- now the vast majority of our flaky tests are gone, we don't need the
bigger machines
- however, browser tests seem to be a little slow, so we can try giving
them more power than the defaults
- this should also help with resource contention when we have multiple
jobs running at the same time, as we have double the numbers of
runners on the free machines
refs https://ghost.slack.com/archives/C02G9E68C/p1677753889082979
- Firefox tests have historically been flaky in CI and a real
distraction when developing
- Firefox is, unfortunately :(, not a widely used browser at only 2.9%
of global market share
- we've not had any Firefox-specific bugs that were detected in CI for
a very long time, so it doesn't add anything anyway
- given SQLite3 is only supported for development, we don't really care about
running tests on Node versions which aren't the recommended version
- this saves 2 jobs per CI run, which helps improve the health of CI in
general
- turns out our concurrency on these 8 core machines is only 10 jobs, so
everything is running really slowly
- by opening up to `linux`, we allow executions on 4, 8 and 16 core
machines with a total concurrency of 30
- we're seeing low availability for the 16 core machines and they might
be constrained as everyone jumps to the highest spec
- in theory, we don't need super fast multi-core machines to run tests,
so I'll try with 8 core ones
refs https://github.com/TryGhost/Toolbox/issues/488
- Node 18 is now LTS so we're adding support for it
- this adds Node 18.12.1 (the latest security release) to our supported
ranges and CI
fixes https://github.com/TryGhost/Toolbox/issues/395
- we seem to be encountering issues with codecov where it randomly
reports huge jumps in test coverage %
- I'm still not sure of the reason why, but the general trend in CI with
codecov is to store test coverage files as artifacts and then submit
them to codecov in one go
- this commit implements that in our CI
- let's see if this improves anything
refs https://github.com/TryGhost/Toolbox/issues/395
- I think we regressed somewhere along the line here, but we seem to be
submitting two test coverage reports for the Admin tests and the unit
tests
- I'm not sure if this fixes the random issues we've been seeing with
-5%/+14% coverage bumps but it should clean this issue up first
- this was originally added because we had an issue with a production
build of Admin during a release
- since then, we've got a canary build that runs for every commit on
`main`
- I think this test is superfluous now, so this commit removes it and
saves ~6 mins of CI time per commit
refs https://github.com/actions/runner-images/issues/6002
- Ubuntu 18.04 is now deprecated in GHA
- this commit moves all tests onto 22.04 except the Admin integration
tests because they're failing to pick Firefox up
- I can look into this after we've unblocked CI
- this required some reworking for the new monorepo setup, but this
commit should now enable the Ghost-CLI tests in CI again
- also switched upgrade test to v4 to support Node 16
- because of how the npm scripts were set up, we were running the full
Admin integration tests during the unit tests phase of CI
- this commit renames the majority of `test` to `test:unit` in the
package.json files, and aliases `test` to `test:unit`
- special packages like Admin have no-op'd `test:unit` scripts so we
don't end up running its tests
refs https://github.com/TryGhost/Toolbox/issues/365
- the .github/workflows/test.yml in Admin needed to be merged with the
top-level GHA folder so this test needed porting
- there is still the browser matrix testing to port once things are
stable
refs https://github.com/TryGhost/Toolbox/issues/354
- this commit turns the Ghost repo into a monorepo so we can bring our
internal packages back in, which makes life easier when working on
Ghost
refs https://github.com/TryGhost/Toolbox/issues/244
- due to the way we currently set the env vars, they get applied across
all matrix variations
- this means we're leaking the variables for SQLite to the MySQL test
runs and this shows a warning because of how strict `mysql2` is
- this commit switches to optionally setting the env variables
- this is a partial workaround for fbcdacbd83/core/shared/config/utils.js (L55-L76) not seeming to work
- we needed to bump the major version so 5.0 migrations will run in CI
- I'm also switching the Migrations test to delete all 5.x migrations
and re-run them to test idempotency
refs https://github.com/TryGhost/Toolbox/issues/267
- Node 12 becomes EOL on April 30th so we're going to be dropping
support for it in Ghost
- this commit updates the Node engine ranges so CLI can pick this up,
and drops 12.22.1 from the CI matrix
- we seem to be getting some odd numbers when running c8 on Node 12, and
it's causing our CI checks to fail
- even when we're adding tests, the coverage value goes down
- this is disrupting the team from shipping, so we need to change that
- this commit alters the setup to run unit tests w/ c8 for Node 16 unit
tests, and without c8 for other versions
- `yarn test:unit` is kept the same for everyday use
refs https://github.com/TryGhost/Toolbox/issues/174
- this commit switches Ghost from using the `mysql` library to the
`mysql2` one
- we've done this for several reasons:
- `mysql2` is more actively maintained
- `mysql2` natively supports the default auth plugin on MySQL 8
- `mysql2` is fasterrrr
- there have been various other commits refactoring the groundwork for
this commit but this commit should be short and sweet:
- alias `mysql` to `mysql2` client so we maintain backwards
compatibility with all configs who use `"client": "mysql"`
- enabled `decimalNumbers` so we maintain the same functionality as
`mysql`
- replaced the dependencies and updated `knex-migrator`
- hardcoded the newer authentication plugin in MySQL 8 CI. Before
switching to `mysql2`, this would break because it didn't support
this
- Updating our config to have `--check-coverage` enforces that the coverage meets a certain level.
- The default is 95 I believe, but our coverage is lower.
- I've set the levels to our current levels, so any drop below these numbers will cause the build to fail.
- I've also set the reporters to be text, html and cobertura so we always have a mini report, the full HTML files to navigate and cobertura for CI
- Cleaned up CI so we don't use the cov:unit command as we're now using codecov
- This also means we can remove the cov:unit command which was weird to use because it uses the last test run, which can be confusing
refs https://github.com/TryGhost/Toolbox/issues/208
- we currently run all test commands separately in CI and this can make
it harder to replicate any issues we see in CI because we have to
manually go and copy the order of the tests
- it's also nicer if there's just one command that runs so we can make
changes globally to CI
- this commit adds a test:ci yarn command which will run the tests in
order of speed/importance, with the -b AKA fail-fast flag so we
don't have to wait for all tests to run before finding out about
issues
- this also cleans up running unit tests in the DB matrix
refs https://github.com/TryGhost/Toolbox/issues/208
- unit tests (shouldn't) be database dependent so they don't need to be
included in the DB matrix
- this prevents an extra 6 executions of unit tests, which isn't a big
deal given they only take 10s to run, but semantically they belong
outside so this commit does that
- also updates the canary step to rely on this extra test run
no issue
- we might fail the tests but not produce any logs in Ghost-CLI
- concatenating the logs would then fail because of the lack of existence
of a directory
- this commit adds a check that files exist before trying to get the
contents of the files
refs https://github.com/TryGhost/Toolbox/issues/181
- we officially support MySQL 8 but we don't run tests for it -- oops
- this commit adds MySQL 8 to the DB matrix in CI tests
- I had to switch to a fork of the `mysql-action` repo so I could get
the native authentication plugin working, but I expect to revert this
once we've merged support for mysql2
refs
https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions#example-using-a-fallback-value
- CI sometimes becomes clogged up with concurrent runs for a PR if the
PR is updated
- this sometimes happens when we merge a bunch of Renovate PRs and other
PRs rebase on top of `main` multiple times
- we shouldn't continue to run tests for PRs if the branches have been pushed
to since they started
- from the referenced link above, this is the built-in solution that
cancels previous in-progress runs if a new job starts
