fixes https://github.com/TryGhost/Toolbox/issues/493
- `nodemon` doesn't properly wait for graceful shutdown if there's a job
running in Ghost when it tries to reload, so it ends up producing
errors because two versions of Ghost are started
- Node 18 added a `--watch` flag, which reloads if any required file is
changed
- switching from nodemon to this native feature should keep the
core functionality the same but fix this edge case that disrupts the
development flow
- by using the same group (the workflow name), we should be able to
limit the concurrency to 1 execution, which might help with some
flaky tests we keep seeing
- now the vast majority of our flaky tests are gone, we don't need the
bigger machines
- however, browser tests seem to be a little slow, so we can try giving
them more power than the defaults
- this should also help with resource contention when we have multiple
jobs running at the same time, as we have double the numbers of
runners on the free machines
refs https://ghost.slack.com/archives/C02G9E68C/p1677753889082979
- Firefox tests have historically been flaky in CI and a real
distraction when developing
- Firefox is, unfortunately :(, not a widely used browser at only 2.9%
of global market share
- we've not had any Firefox-specific bugs that were detected in CI for
a very long time, so it doesn't add anything anyway
- given SQLite3 is only supported for development, we don't really care about
running tests on Node versions which aren't the recommended version
- this saves 2 jobs per CI run, which helps improve the health of CI in
general
- turns out our concurrency on these 8 core machines is only 10 jobs, so
everything is running really slowly
- by opening up to `linux`, we allow executions on 4, 8 and 16 core
machines with a total concurrency of 30
- we're seeing low availability for the 16 core machines and they might
be constrained as everyone jumps to the highest spec
- in theory, we don't need super fast multi-core machines to run tests,
so I'll try with 8 core ones
refs https://github.com/TryGhost/Toolbox/issues/488
- Node 18 is now LTS so we're adding support for it
- this adds Node 18.12.1 (the latest security release) to our supported
ranges and CI
refs: https://github.com/TryGhost/Toolbox/issues/481
* Correctly setup environment variable to run both local & staging browser-based tests
* Use non-production Ghost Admin build, since production builds require HTTPS to use Stripe Connect
- the create-release-branch workflow works by getting the most recent
tag, bumping it and using that to create the new tag
- now we've moved Portal into the monorepo, we've got two different
types of flags, but the Portal ones aren't valid semver so the
workflow fails if the most recent tag is one for Portal
- this fixes that by ensuring we only fetch tags matching the pattern we
use for tagging Ghost
- our use-case for this is to ensure that people don't push to
`main` without running linting, as this can block CI from passing
until the linting issue is resolved
- however, it can become annoying to run linting on non-main branches,
especially when you just want to WIP some changes without caring for
linting
- generally speaking, anyone who creates commits on a non-main branch is
going to open them as a PR, so linting is run anyway
- this commit get the branch name and only runs linting if we're on
`main`
fixes https://github.com/TryGhost/Toolbox/issues/459
- in the situation where the `yarn dev` command fails, it pumps out a
massive error message, which is incredibly distracting to the dev
experience
- this changes the output to just be a log line with some suggestions on
how to fix it
- `refs/closes` can be confusing to those who are not familiar with GitHub when we actually mean "use refs or fixes (or closes)"
- we generally don't want to use emojis for alpha features because adding it to the release notes is unnecessary until the feature is GA
refs https://ghost.slack.com/archives/C02G9E68C/p1665497363885949
- we've seen an issue with `lint-staged` in the Admin package because it
doesn't pick up the lint-todo file, so it incorrectly flags linting
issues that we're ignoring
- this is happening because it runs the command from cwd, where the lint
exclusion file does not exist
- thankfully, `lint-staged` has `--relative` which will run the command
from the directory where the command is defined in config, so `ghost/admin`
in our case, and that means the lint file is present and picked up
- up until this commit, git hooks were only used by a handful of people
because they were a pain:
- they'd only be set up when you did `yarn setup`
- the existing hooks ran `yarn lint` on all projects, which was
incredibly slow
- as a result, not many of us actually had them enabled, but this would
cause issues in CI because people were pushing un-linted commits
- other JS projects tend to use husky to automate the git hook setup and
lint-staged to speed up linting on changed files
- this commit switches to using them both
- `lint-staged` only runs `eslint` on staged JS files that are about to
be committed - if there's a linting error, it will stop the commit
- I've configured the pre-commit hook to successfully exit in CI because we
don't want to run pre-commit hooks right now
- this means we can remove Grunt - yay!
- I lowered the code coverage on the repo to the point where
it started failing because I added a new export to the config library
- this wasn't easy to add tests for because the existing config tests
use the loader directly and not the library export
- instead, I'm just going to make the dev script access the loader, and
make a note to clean this up in the future when we pull out the config
module
- because the cwd of `.github/dev.js` is not `ghost/core`, it doesn't
pick up config.local.json files, so any configuration you set in there
isn't applied
- this meant that developers with HTTPS configured locally couldn't use
`--stripe` because it wouldn't configure the Stripe listening URL
correctly
- this adds an exports to the config lib to allow passing options in,
which I then utilize to pass the directory that config resides in
- this should fix the aforementioned problem with HTTPS
refs https://github.com/TryGhost/Toolbox/issues/426
- this adds support for `--portal` in `yarn dev` to support the upcoming
migration of Portal into the monorepo
- also adds the Portal folder to the ignorelist for nodemon so Ghost
doesn't bootloop
- daily is a little to infrequent for us as it means new comments don't
cause the issue/PR to be unlabeled as `stale` for up to 24hrs
- this commit increases the frequency to hourly
refs https://github.com/TryGhost/Toolbox/issues/381
- when using `pull_request`, the workflow doesn't have permissions to
write to the PR in question
- there is another trigger - `pull_request_target` - but this comes with
a heap of security warnings
- our use of it should be OK because we're only checking out our own
Action and not code from the PR
- this commit also adds permissions to specify we only want write access
to issues and PRs
refs https://github.com/TryGhost/Toolbox/issues/426
- we're going to need to support more complex combinations of dev
commands soon, with other packages optionally running and env
variables being altered
- this command pulls out a lot of the dev env scripting into a single
scripts
- also cleans up the use of grunt-shell so we can remove the dependency
fixes https://github.com/TryGhost/Toolbox/issues/395
- we seem to be encountering issues with codecov where it randomly
reports huge jumps in test coverage %
- I'm still not sure of the reason why, but the general trend in CI with
codecov is to store test coverage files as artifacts and then submit
them to codecov in one go
- this commit implements that in our CI
- let's see if this improves anything
- we didn't clone the repo at the point when we were running a git
command, so that failed
- switched to a different method of checking out `main` and then
checking out the most recent tag
refs https://github.com/TryGhost/Toolbox/issues/381
- we want to start collecting events from pull requests so we can label
and reply upon certain events
- this adds the `closed` and `labeled` events to the workflow triggers
refs https://github.com/TryGhost/Toolbox/issues/395
- I think we regressed somewhere along the line here, but we seem to be
submitting two test coverage reports for the Admin tests and the unit
tests
- I'm not sure if this fixes the random issues we've been seeing with
-5%/+14% coverage bumps but it should clean this issue up first
- right now, we use an internal CI solution to create branches for patch
releases, but it's difficult to use
- this workflow should allow the team to create release branches from
the GitHub UI, without delving into our internal tooling
- this was originally added because we had an issue with a production
build of Admin during a release
- since then, we've got a canary build that runs for every commit on
`main`
- I think this test is superfluous now, so this commit removes it and
saves ~6 mins of CI time per commit
refs https://github.com/actions/runner-images/issues/6002
- Ubuntu 18.04 is now deprecated in GHA
- this commit moves all tests onto 22.04 except the Admin integration
tests because they're failing to pick Firefox up
- I can look into this after we've unblocked CI
- this required some reworking for the new monorepo setup, but this
commit should now enable the Ghost-CLI tests in CI again
- also switched upgrade test to v4 to support Node 16
refs 7913283c96
- you can supply a branch and a version to call it, and it'll produce a
zip of the tarball you can use
- this should avoid the need for local builds, which are error prone
- because of how the npm scripts were set up, we were running the full
Admin integration tests during the unit tests phase of CI
- this commit renames the majority of `test` to `test:unit` in the
package.json files, and aliases `test` to `test:unit`
- special packages like Admin have no-op'd `test:unit` scripts so we
don't end up running its tests
refs https://github.com/TryGhost/Toolbox/issues/365
- the .github/workflows/test.yml in Admin needed to be merged with the
top-level GHA folder so this test needed porting
- there is still the browser matrix testing to port once things are
stable
refs https://github.com/TryGhost/Toolbox/issues/354
- this commit turns the Ghost repo into a monorepo so we can bring our
internal packages back in, which makes life easier when working on
Ghost
- up until now, we've been running `grunt release` before publishing to
NPM or pushing the canary zip
- this command runs the production asset build and generates a zip
- this zip isn't used by the NPM publishing task because that does an
`npm pack`
- we only use it for the canary build, but this should be brought more
inline with the NPM process to make the gaps smaller
- this commit refactors the `grunt release` task to become a lot smaller
by removing the generated zip steps
- the expected workflow is now to just to an `npm pack`, which will run
the `prepack` task to generate a `.tgz` archive
- this should still respect `.npmignore`, so it'll just include the
files we expect
- the test of the canary workflow is being updated to handle this
- also cleans up a dev dependency that is no longer used, along with 2
imports
refs https://github.com/TryGhost/Toolbox/issues/244
- due to the way we currently set the env vars, they get applied across
all matrix variations
- this means we're leaking the variables for SQLite to the MySQL test
runs and this shows a warning because of how strict `mysql2` is
- this commit switches to optionally setting the env variables
- this is a partial workaround for fbcdacbd83/core/shared/config/utils.js (L55-L76) not seeming to work
fixes https://github.com/TryGhost/Toolbox/issues/285
- without this, some of the workflows that are only useful for the core
team will run on forks
- this commit adds checks to ensure we're running on a repo maintained
by the TryGhost organization for a handful of workflows
- we needed to bump the major version so 5.0 migrations will run in CI
- I'm also switching the Migrations test to delete all 5.x migrations
and re-run them to test idempotency
refs 002cf5b0eb
- The hook file has to be executable to be triggered in the pre-push stage:
hint: The '.git/hooks/pre-push' hook was ignored because it's not set as executable.
refs 81cd5fac7e
- While developing locally it's common to commit small WIP changes which might contain linting errors. Having the check done once on a pre-push phase gives enoght protection from pushing out broken code and reduces frustration when developing locally
refs 648530009d
- Naz has broken the main too many times - it's time to stop the atrocities.
- Having a lint check as a pre-commit hook will make it really hard commiting code with linting errors
refs https://github.com/TryGhost/Toolbox/issues/267
- Node 12 becomes EOL on April 30th so we're going to be dropping
support for it in Ghost
- this commit updates the Node engine ranges so CLI can pick this up,
and drops 12.22.1 from the CI matrix
refs 4aee97472e
- the referenced commit copied the workflow file to our shared templates
repository
- this commit switches to using that template
- this should help reduce duplication when we're doing v5
- we seem to be getting some odd numbers when running c8 on Node 12, and
it's causing our CI checks to fail
- even when we're adding tests, the coverage value goes down
- this is disrupting the team from shipping, so we need to change that
- this commit alters the setup to run unit tests w/ c8 for Node 16 unit
tests, and without c8 for other versions
- `yarn test:unit` is kept the same for everyday use
refs https://github.com/TryGhost/Toolbox/issues/174
- this commit switches Ghost from using the `mysql` library to the
`mysql2` one
- we've done this for several reasons:
- `mysql2` is more actively maintained
- `mysql2` natively supports the default auth plugin on MySQL 8
- `mysql2` is fasterrrr
- there have been various other commits refactoring the groundwork for
this commit but this commit should be short and sweet:
- alias `mysql` to `mysql2` client so we maintain backwards
compatibility with all configs who use `"client": "mysql"`
- enabled `decimalNumbers` so we maintain the same functionality as
`mysql`
- replaced the dependencies and updated `knex-migrator`
- hardcoded the newer authentication plugin in MySQL 8 CI. Before
switching to `mysql2`, this would break because it didn't support
this
- Updating our config to have `--check-coverage` enforces that the coverage meets a certain level.
- The default is 95 I believe, but our coverage is lower.
- I've set the levels to our current levels, so any drop below these numbers will cause the build to fail.
- I've also set the reporters to be text, html and cobertura so we always have a mini report, the full HTML files to navigate and cobertura for CI
- Cleaned up CI so we don't use the cov:unit command as we're now using codecov
- This also means we can remove the cov:unit command which was weird to use because it uses the last test run, which can be confusing
refs https://github.com/TryGhost/Toolbox/issues/207
- we want to start writing some E2E tests that involve automated
pointing and clicking around the frontend of Ghost to test that
members of Ghost sites can still do what we expect
- we've decided to look in to Playwright for this - it looks __really__
nice
- this is a VERY basic first test - it'll check for a 200 on the
homepage of whatever we provide as the TEST_URL env variable, or
default to a (manually-run) Ghost instance on port 2368
- also adds a `yarn test:browser` command to run the tests using the
Playwright CLI, and a sample GitHub Action workflow which we can
manually run with a site URL
- there's a lot more to add here in terms of test framework but this
gets us started
refs https://github.com/TryGhost/Toolbox/issues/208
- we currently run all test commands separately in CI and this can make
it harder to replicate any issues we see in CI because we have to
manually go and copy the order of the tests
- it's also nicer if there's just one command that runs so we can make
changes globally to CI
- this commit adds a test:ci yarn command which will run the tests in
order of speed/importance, with the -b AKA fail-fast flag so we
don't have to wait for all tests to run before finding out about
issues
- this also cleans up running unit tests in the DB matrix
refs https://github.com/TryGhost/Toolbox/issues/208
- unit tests (shouldn't) be database dependent so they don't need to be
included in the DB matrix
- this prevents an extra 6 executions of unit tests, which isn't a big
deal given they only take 10s to run, but semantically they belong
outside so this commit does that
- also updates the canary step to rely on this extra test run
no issue
- we might fail the tests but not produce any logs in Ghost-CLI
- concatenating the logs would then fail because of the lack of existence
of a directory
- this commit adds a check that files exist before trying to get the
contents of the files
refs https://github.com/TryGhost/Toolbox/issues/181
- we officially support MySQL 8 but we don't run tests for it -- oops
- this commit adds MySQL 8 to the DB matrix in CI tests
- I had to switch to a fork of the `mysql-action` repo so I could get
the native authentication plugin working, but I expect to revert this
once we've merged support for mysql2
refs
https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions#example-using-a-fallback-value
- CI sometimes becomes clogged up with concurrent runs for a PR if the
PR is updated
- this sometimes happens when we merge a bunch of Renovate PRs and other
PRs rebase on top of `main` multiple times
- we shouldn't continue to run tests for PRs if the branches have been pushed
to since they started
- from the referenced link above, this is the built-in solution that
cancels previous in-progress runs if a new job starts
no issue
Includes a timeout if TailScale is completely inaccessible, and a
continue-on-error statement which should handle all forms of
failure. The following step also includes a continue-on-error step, as
reporting metrics depends on TailScale succeeding, and should also
never fail the build.
refs https://github.com/TryGhost/Ghost/pull/13716
refs https://github.com/actions/setup-node/issues/317#issuecomment-929694556
- the `setup-node` GitHub Action seems to use a shell command to get the
cache path, but these are colorised when `FORCE_COLOR` is enabled
- this causes the Action to fail to read the path correctly
- the comment referenced above suggests to remove `FORCE_COLOR` but it's
nice to have colored output for our tests
- instead, I'm disabling the environment variable on the `setup-node`
action so it still works
- I've tested with the referenced PR and this unblocks dependency caching 🎉
refs 2a19e6151c
- we want to ensure that the `needs triage` label is removed when an
issue is closed
- the event was handled in the labelling Action but it needs the event
adding here to trigger execution
no-issue
- The workflow runs in the pull_request_target context which has access to repo secrets even when triggered from a fork
- Pinned the GH Action to a specific version to guard against upstream changes to the Action which may abuse access to secrets
refs https://github.com/TryGhost/Toolbox/issues/81
- the existing `label-actions` tooling was deprecated and shut down but
after reviewing, it wasn't expressive enough for our workflow use cases
- we wanted a tool we could drop into our repos and it works without
extra configuration
- I've developed the `tryghost/label-actions` GitHub Action which will
supports all our labeling flows for triaging
- this commit switches the repo over to using that
- configured the scheduled tasks to run at midnight daily
Refs CORE-120
When test runs execute, use the new @tryghost/metrics package to send metrics to configurable backends.
At the moment, we're just sending the test run duration to our metric store in preparation of changes to the test suite.
refs https://github.com/TryGhost/Toolbox/issues/71
- Node 16 is now LTS so we're adding support in Ghost
- we're also bumping the minimum Node 14 version to 14.17.0 so we can merge
dependencies who have higher Node 14 requirements than current
- this commit adds Node 16 to the `package.json` engines and to CI
- also bumps all dependencies that needed new versions to add Node 16
support
refs linear.app/tryghost/issue/CORE-74/improve-the-test-situation
- this commit adds the codecov GitHub Action into CI so we can upload
coverage reports
- the coverage files need to be in XML for them to work with
codecov, so this commit also adds cobertura (XML) as a reporter
no issue
- Renovate will open PRs to bump dependencies but these are unassigned
- Renovate does have the ability to set assignees, but these are blanket
assigns not based upon the package that is being upgraded
- we want to assign them to the owners of the package that is being
upgraded
- I've created the `gh-auto-assign` GitHub CLI exntension which reads a
`.github/AUTO_ASSIGN` file and assign PRs based upon the entries listed
- this commit adds the initial `AUTO_ASSIGN` file and GitHub Actions
workflow to trigger when a PR is opened
no issue
- coverage allows us to see how much of our code we're hitting in tests
and it's time we started to get visibility on this in Ghost
- we can then make informed decisions on which well-tested internal libraries
can be pulled out into their own packages
- this commit:
- adds the c8 dev dependency to Ghost
- prepends the `test:unit` command with c8 with the `text-summary` reporter
- adds a `coverage:unit` command to get the `text` report
- you can do `yarn coverage:unit --reporter html` to get a HTML report etc
- uses this new test coverage reporter command in CI tests
no issue
- as we're making more and more use of branches for releases, we want
tests to run on them
- this commit adds the v4 wildcard to the GitHub workflow so it'll
trigger when we push to them
no issue
- `RELEASE_TOKEN` is currently a GitHub personal token, but this
has some downsides:
- if the token ever expires and I'm unaware, it'll break the release
process
- GitHub Releases say the creator was `daniellockyer` even if someone
else actually did the release
- this commit switches over to using the built-in `GITHUB_TOKEN`, which
is owned by the `github-actions` app and should never expire
- aside from that, Ghost releases will be created by the neutral
`github-actions` account
refs 7e6800b2b8
- referenced commit deprecated `grunt main` in favor of `yarn main` but
this wasn't updated so we never installed dependencies for Admin,
which caused the build to break
- this commit switches to the new command
This commit achieves a few things:
- ☑️ No longer having to remember whether a command is yarn something or grunt something
- ☑️ Simplification of tools hopefully making them easier to remember and use
- ☑️ Complete removal of the need for grunt from our test tooling
Several of the tools still use grunt under the hood, but the **entrypoint** should aways be `yarn xxx`.
- `grunt main` -> `yarn main`
- `grunt dev` -> `yarn dev`
- `grunt build` -> `yarn build`
- `grunt test:file-or-folder` -> `yarn test file-or-folder`
- `grunt test-unit` -> `yarn test:unit`
- `grunt test-acceptance` -> `yarn test:acceptance`
- `grunt test-regression` -> `yarn test:regression`
- `grunt validate` -> removed due to lack of use
There is now also `yarn test:all` to run all 3 classes of tests
This PR also reorders & restructures the Gruntfile extensively so that:
- The remaining useful commands are all at the top of the file
- Config and other blah happens after all the useful commands
- All release-only config happens in the release task at the very end of the file
---
DONE:
* Removed all references to npm/bower
* Removed all references to lint / deprecated command
* Moved debug to yarn dev:debug
* Removed all references to travis
* Removed broken help task + useless comment
* Removed unused knex-migrator and clean:test setup tasks
* Added new test commands, removed grunt validate
* Moved stubClientFiles to test utility and use in the few tests that need it
* Used mocha in yarn directly except grunt test:x
* Swapped grunt test for yarn test
* extensive cleanup and reshuffling
no issue
- if Ghost-CLI tests fail, the CLI will put some extra debug logs in
`~/.ghost/logs`
- these are useful to help see what the issue is, so we should print
them out if we're experiencing a failure
- this commit cats the contents of the folder so we can see it in CI
no issue
- by extracting it to a different workflow that uses the
`workflow_dispatch` trigger, we can manually run a build, but also
trigger it from Admin so we can always stay up-to-date
no issue
- Node 14 is now our recommended version and it's easier if we build
Ghost with this version
- this also works around the fact that 12.18.0 would no longer build
Ghost because we have bumped our minimum 12 version
refs 57ff38da8a
- we're dropping support for Node 10 which involved bumping the Node
version we run our Ghost-CLI tests on but they fail because Ghost v1
and v2 don't support Node 12
- this commit disables these tests until we come up with a good
workaround
refs https://github.com/TryGhost/Team/issues/658
- Node 10 become EOL as of April 30th so it's time to drop support
- this commit:
- removes the Node 10 range from the `node` `engines` block
- removes Node 10 from CI tests
- switches Node 10 in the CLI test to Node 12 so we can ensure it
installs on our oldest supported Node version
no issue
- now we've officially released v4, we need to check updates from v3
- this commit adds a test copied from the other tests above
- also renames the steps from Upgrade to Update to match internal terminology
no issue
- when building the canary zip, we currently use `main` within Casper,
but the 4.0 changes are on the 4.0 branch
- this commit updates the workflow to checkout that branch as part of
the pipeline
no issue
- some new GScan changes mean that CI now throws a warning when using v3
Casper, which causes tests to fail
- we need to switch to v4 to get the tests to pass
- we can't put v4 Casper in `main` because it'll autodeploy and be
unusable for anyone downloading until Ghost 4.0 is released
no issue
- Ghost-CLI uses gscan under the hood to check the theme, but GScan has
not yet been updated to support v4
- disabling these tests for now until it does
no issue
- in 4.0, we don't expect to rollback to 3.0 because there are
irreversible migrations
- the existing test attempts to do this because it was written for v3
- this commit updates the test to apply to v4, although this will need
to change soon when we switch branches
no issue
- we want tests to run on the 4.0 branch, along with Slack notifications
- we also want a canary build of the 4.0 branch, but this requires some
alterations to bump to a premajor + use a different artifact name
- because the Admin submodule has been updated, there aren't any
differences that can be committed so we shouldn't do it
- this should check if we have uncommitted changes and only commit if so
