refs https://github.com/TryGhost/DevOps/issues/39
- up until now, we've had a CI job which does a really basic test for
migrations, but it barely functions and misses bugs all the time
- this commit removes that and switches to an actual test suite for our
migrations, so we can ensure they function as expected
- also removes the env var hack I came up with for those migrations
tests
- this should lead to safer migrations and faster tests
refs https://github.com/TryGhost/Toolbox/issues/609
- this rewrites the CI workflow to include a pre-test step which will
download and cache dependencies, and will only run tests when the
associated code changes
- this provides a huge improvement over the existing setup, and will
save us a lot of time in CI
refs https://github.com/TryGhost/Team/issues/3504
This migrates comments-ui to TypeScript. Only `App.js` is left to
migrate, but since this isn't using hooks yet, it will need a bigger
rewrite so this will need to happen in a separate PR.
refs https://github.com/TryGhost/Toolbox/issues/602
- the new regex allows for the filename to contain seconds, which would have
helped prevent a bug we had with migrations being in the wrong order
- mentioned filename ordering in the migration review comment to bring
that to the forefront of the mind
no issue
Safari doesn't allow to load non-https scripts in a https environment, so we need to load Lexical from https using a Caddy proxy. This adds the `--https` option to `yarn dev --lexical`.
Might consider to somehow include a caddyfile somehow in the repo to make this work everywhere without changes.
- in the migration tests we need to boot Ghost and then kill it
afterwards
- because there was no easy way to do this, the workflow waits for 20s
and then kills the last process ID
- aside from being a terrible idea, it means we're also just arbitrarily
waiting for 20s, which burns time when it takes shorter to boot Ghost
- this commit implements an environment variable that will kill the
server once it has run the whole boot process, and then fixes the
workflow to use that
refs https://github.com/TryGhost/Toolbox/issues/592
- this commit extracts the regression tests into a separate workflow
- this means they run in parallel and reduce the time we have to wait
for DB tests in general
- also fixes a test that was reliant on being run after the E2E
tests (!)
refs https://github.com/TryGhost/Toolbox/issues/528
- we're moving towards making Node 18 our recommended version, so that
involves ensuring all of CI is running Node 18
- we still have some Node 16 matrix runs to ensure compatibility
no issue
Added support for `--signup` to `yarn dev`. This will make sure that the signup form preview server is served (with rebuilding) and can be used in admin (instead of using the published version). This is also automatically started when using `yarn dev --all`, but with a twist:
If you only run `yarn dev --all`, it will only build and watch the production build of the signup form. If you use `yarn dev --signup` or `yarn dev --all --signup` it will also serve the demo app on `http://localhost:6173`.
refs https://ghost.slack.com/archives/C02G9E68C/p1685446707169999
- `actions/setup-node` has a bug where it doesn't parse paths where the path is pretty-printed with colors, so we have to explicitly disable colors
- Added whitespace change inside signup-form to trigger new CI run
refs https://github.com/TryGhost/Toolbox/issues/583
- removed default config so we just have separate numbers for the flags
- added threshold to flags config so we can protect against small
changes in variance
refs https://github.com/TryGhost/Toolbox/issues/583
- this commit adds config for codecov because this repo needs some
changes that aren't suitable for the global config
- this probably needs some adjusting over time but should get us started
for now
- this will allow us to see which set of tests are consuming the most
amount of time in CI
- in order to split apart the commands, I've had to override the
coverage thresholds for integration+regression tests in order to keep
c8 happy
- also sprinkled some more labels into the workflows to make things
clearer to read
fixes https://github.com/TryGhost/Toolbox/issues/583
- we currently upload the codecov for unit tests and e2e/admin tests
separately
- unfortunately this means that when one submits before the other, the
codecov % will be incorrect
- this leaves a big red cross on the commit until the other coverage is
uploaded
- this should fix that by upload the coverage in one step once CI is
happy
refs https://github.com/TryGhost/Toolbox/issues/584
- we should strive to provide a good development experience by keeping
the number of "other" setup steps to a minimum
- this adds support for the Announcement Bar app to the dev script so a
simple flag spins up the dev server too
refs https://github.com/TryGhost/Team/issues/3240
- this adds support for `--lexical` to `yarn dev`, which sets the
env variable so Ghost uses a local Lexical instance and not the CDN version
This is the first pass at an abstract class for our in-memory repository
implementations. It's based off of the existing implementations and
should speed up local development and deduplicate code
- I switched over `build` to run in production mode because we now do
`lerna run build` during the release process, but this was bundling
development assets
- this adds `build:dev` and switches the browser test to use that, so
they should use the development assets moving forwards
This is an initial start to using TypeScript in our non-core Ghost packages.
- Adds a prepare script to build the project after installing deps
- Adds an initial tsconfig.json which is compatible with our node env
- Migrates all of the code to TypeScript, including tests
- Updates tests to use ts-node so that we don't need to compile the tests
- ts-node is installed at the top level because the env is weird with lerna and
doesn't work otherwise
- Updates the yarn dev script to build the project with the --all and --revisions flag
- Updates the prepare script in the top level to run prepare on packages, so
that packages can be built when running `yarn`
- Updates the build script in ghost/core to run build on packages, so that
packages are built before being monobundled
- Updates monobundle to be a dependency and use the new TryGhost repo, which
includes some minor fixes and improvements, such as supporting devDeps
- Updates the GitHub workflows to run the build command in the top level
directory rather than ghost/core so that other packages are built, too.
- by default, yarn will try and fetch dependencies from the network, and
fallback to local cache if the network is down
- `--prefer-offline` switches that around so we can take advantage of
local caches before defaulting to the network
- hopefully this speeds up dependency install steps in CI
- because we now ignore git submodule changes, they didn't show up in
`git diff --cached...`, so it was possible to get submodules to be
committed
- you can re-enable submodules to be shown with `--ignore-submodules=none`
- this implements that
- we never want to allow submodules to be committed, so I've removed the
prompt for faster feedback
fixes https://github.com/TryGhost/Toolbox/issues/532
- we should protect against failures entering `main` which could be
avoided by running a quick unit test beforehand
- this reintroduces Lerna as it supports parallelisation and `--since`,
to run linting and unit tests on packages that have changed since
upstream
fixes https://github.com/TryGhost/Toolbox/issues/493
- `nodemon` doesn't properly wait for graceful shutdown if there's a job
running in Ghost when it tries to reload, so it ends up producing
errors because two versions of Ghost are started
- Node 18 added a `--watch` flag, which reloads if any required file is
changed
- switching from nodemon to this native feature should keep the
core functionality the same but fix this edge case that disrupts the
development flow
- by using the same group (the workflow name), we should be able to
limit the concurrency to 1 execution, which might help with some
flaky tests we keep seeing
- now the vast majority of our flaky tests are gone, we don't need the
bigger machines
- however, browser tests seem to be a little slow, so we can try giving
them more power than the defaults
- this should also help with resource contention when we have multiple
jobs running at the same time, as we have double the numbers of
runners on the free machines
refs https://ghost.slack.com/archives/C02G9E68C/p1677753889082979
- Firefox tests have historically been flaky in CI and a real
distraction when developing
- Firefox is, unfortunately :(, not a widely used browser at only 2.9%
of global market share
- we've not had any Firefox-specific bugs that were detected in CI for
a very long time, so it doesn't add anything anyway
- given SQLite3 is only supported for development, we don't really care about
running tests on Node versions which aren't the recommended version
- this saves 2 jobs per CI run, which helps improve the health of CI in
general
- turns out our concurrency on these 8 core machines is only 10 jobs, so
everything is running really slowly
- by opening up to `linux`, we allow executions on 4, 8 and 16 core
machines with a total concurrency of 30
- we're seeing low availability for the 16 core machines and they might
be constrained as everyone jumps to the highest spec
- in theory, we don't need super fast multi-core machines to run tests,
so I'll try with 8 core ones
refs https://github.com/TryGhost/Toolbox/issues/488
- Node 18 is now LTS so we're adding support for it
- this adds Node 18.12.1 (the latest security release) to our supported
ranges and CI
refs: https://github.com/TryGhost/Toolbox/issues/481
* Correctly setup environment variable to run both local & staging browser-based tests
* Use non-production Ghost Admin build, since production builds require HTTPS to use Stripe Connect
- the create-release-branch workflow works by getting the most recent
tag, bumping it and using that to create the new tag
- now we've moved Portal into the monorepo, we've got two different
types of flags, but the Portal ones aren't valid semver so the
workflow fails if the most recent tag is one for Portal
- this fixes that by ensuring we only fetch tags matching the pattern we
use for tagging Ghost
- our use-case for this is to ensure that people don't push to
`main` without running linting, as this can block CI from passing
until the linting issue is resolved
- however, it can become annoying to run linting on non-main branches,
especially when you just want to WIP some changes without caring for
linting
- generally speaking, anyone who creates commits on a non-main branch is
going to open them as a PR, so linting is run anyway
- this commit get the branch name and only runs linting if we're on
`main`
fixes https://github.com/TryGhost/Toolbox/issues/459
- in the situation where the `yarn dev` command fails, it pumps out a
massive error message, which is incredibly distracting to the dev
experience
- this changes the output to just be a log line with some suggestions on
how to fix it
- `refs/closes` can be confusing to those who are not familiar with GitHub when we actually mean "use refs or fixes (or closes)"
- we generally don't want to use emojis for alpha features because adding it to the release notes is unnecessary until the feature is GA
refs https://ghost.slack.com/archives/C02G9E68C/p1665497363885949
- we've seen an issue with `lint-staged` in the Admin package because it
doesn't pick up the lint-todo file, so it incorrectly flags linting
issues that we're ignoring
- this is happening because it runs the command from cwd, where the lint
exclusion file does not exist
- thankfully, `lint-staged` has `--relative` which will run the command
from the directory where the command is defined in config, so `ghost/admin`
in our case, and that means the lint file is present and picked up
- up until this commit, git hooks were only used by a handful of people
because they were a pain:
- they'd only be set up when you did `yarn setup`
- the existing hooks ran `yarn lint` on all projects, which was
incredibly slow
- as a result, not many of us actually had them enabled, but this would
cause issues in CI because people were pushing un-linted commits
- other JS projects tend to use husky to automate the git hook setup and
lint-staged to speed up linting on changed files
- this commit switches to using them both
- `lint-staged` only runs `eslint` on staged JS files that are about to
be committed - if there's a linting error, it will stop the commit
- I've configured the pre-commit hook to successfully exit in CI because we
don't want to run pre-commit hooks right now
- this means we can remove Grunt - yay!
- I lowered the code coverage on the repo to the point where
it started failing because I added a new export to the config library
- this wasn't easy to add tests for because the existing config tests
use the loader directly and not the library export
- instead, I'm just going to make the dev script access the loader, and
make a note to clean this up in the future when we pull out the config
module
- because the cwd of `.github/dev.js` is not `ghost/core`, it doesn't
pick up config.local.json files, so any configuration you set in there
isn't applied
- this meant that developers with HTTPS configured locally couldn't use
`--stripe` because it wouldn't configure the Stripe listening URL
correctly
- this adds an exports to the config lib to allow passing options in,
which I then utilize to pass the directory that config resides in
- this should fix the aforementioned problem with HTTPS
refs https://github.com/TryGhost/Toolbox/issues/426
- this adds support for `--portal` in `yarn dev` to support the upcoming
migration of Portal into the monorepo
- also adds the Portal folder to the ignorelist for nodemon so Ghost
doesn't bootloop
- daily is a little to infrequent for us as it means new comments don't
cause the issue/PR to be unlabeled as `stale` for up to 24hrs
- this commit increases the frequency to hourly
refs https://github.com/TryGhost/Toolbox/issues/381
- when using `pull_request`, the workflow doesn't have permissions to
write to the PR in question
- there is another trigger - `pull_request_target` - but this comes with
a heap of security warnings
- our use of it should be OK because we're only checking out our own
Action and not code from the PR
- this commit also adds permissions to specify we only want write access
to issues and PRs
refs https://github.com/TryGhost/Toolbox/issues/426
- we're going to need to support more complex combinations of dev
commands soon, with other packages optionally running and env
variables being altered
- this command pulls out a lot of the dev env scripting into a single
scripts
- also cleans up the use of grunt-shell so we can remove the dependency
fixes https://github.com/TryGhost/Toolbox/issues/395
- we seem to be encountering issues with codecov where it randomly
reports huge jumps in test coverage %
- I'm still not sure of the reason why, but the general trend in CI with
codecov is to store test coverage files as artifacts and then submit
them to codecov in one go
- this commit implements that in our CI
- let's see if this improves anything
- we didn't clone the repo at the point when we were running a git
command, so that failed
- switched to a different method of checking out `main` and then
checking out the most recent tag
refs https://github.com/TryGhost/Toolbox/issues/381
- we want to start collecting events from pull requests so we can label
and reply upon certain events
- this adds the `closed` and `labeled` events to the workflow triggers
refs https://github.com/TryGhost/Toolbox/issues/395
- I think we regressed somewhere along the line here, but we seem to be
submitting two test coverage reports for the Admin tests and the unit
tests
- I'm not sure if this fixes the random issues we've been seeing with
-5%/+14% coverage bumps but it should clean this issue up first
- right now, we use an internal CI solution to create branches for patch
releases, but it's difficult to use
- this workflow should allow the team to create release branches from
the GitHub UI, without delving into our internal tooling
- this was originally added because we had an issue with a production
build of Admin during a release
- since then, we've got a canary build that runs for every commit on
`main`
- I think this test is superfluous now, so this commit removes it and
saves ~6 mins of CI time per commit
refs https://github.com/actions/runner-images/issues/6002
- Ubuntu 18.04 is now deprecated in GHA
- this commit moves all tests onto 22.04 except the Admin integration
tests because they're failing to pick Firefox up
- I can look into this after we've unblocked CI
- this required some reworking for the new monorepo setup, but this
commit should now enable the Ghost-CLI tests in CI again
- also switched upgrade test to v4 to support Node 16
refs 7913283c96
- you can supply a branch and a version to call it, and it'll produce a
zip of the tarball you can use
- this should avoid the need for local builds, which are error prone
- because of how the npm scripts were set up, we were running the full
Admin integration tests during the unit tests phase of CI
- this commit renames the majority of `test` to `test:unit` in the
package.json files, and aliases `test` to `test:unit`
- special packages like Admin have no-op'd `test:unit` scripts so we
don't end up running its tests
refs https://github.com/TryGhost/Toolbox/issues/365
- the .github/workflows/test.yml in Admin needed to be merged with the
top-level GHA folder so this test needed porting
- there is still the browser matrix testing to port once things are
stable
refs https://github.com/TryGhost/Toolbox/issues/354
- this commit turns the Ghost repo into a monorepo so we can bring our
internal packages back in, which makes life easier when working on
Ghost
- up until now, we've been running `grunt release` before publishing to
NPM or pushing the canary zip
- this command runs the production asset build and generates a zip
- this zip isn't used by the NPM publishing task because that does an
`npm pack`
- we only use it for the canary build, but this should be brought more
inline with the NPM process to make the gaps smaller
- this commit refactors the `grunt release` task to become a lot smaller
by removing the generated zip steps
- the expected workflow is now to just to an `npm pack`, which will run
the `prepack` task to generate a `.tgz` archive
- this should still respect `.npmignore`, so it'll just include the
files we expect
- the test of the canary workflow is being updated to handle this
- also cleans up a dev dependency that is no longer used, along with 2
imports
refs https://github.com/TryGhost/Toolbox/issues/244
- due to the way we currently set the env vars, they get applied across
all matrix variations
- this means we're leaking the variables for SQLite to the MySQL test
runs and this shows a warning because of how strict `mysql2` is
- this commit switches to optionally setting the env variables
- this is a partial workaround for fbcdacbd83/core/shared/config/utils.js (L55-L76) not seeming to work
fixes https://github.com/TryGhost/Toolbox/issues/285
- without this, some of the workflows that are only useful for the core
team will run on forks
- this commit adds checks to ensure we're running on a repo maintained
by the TryGhost organization for a handful of workflows
- we needed to bump the major version so 5.0 migrations will run in CI
- I'm also switching the Migrations test to delete all 5.x migrations
and re-run them to test idempotency
refs 002cf5b0eb
- The hook file has to be executable to be triggered in the pre-push stage:
hint: The '.git/hooks/pre-push' hook was ignored because it's not set as executable.
refs 81cd5fac7e
- While developing locally it's common to commit small WIP changes which might contain linting errors. Having the check done once on a pre-push phase gives enoght protection from pushing out broken code and reduces frustration when developing locally
refs 648530009d
- Naz has broken the main too many times - it's time to stop the atrocities.
- Having a lint check as a pre-commit hook will make it really hard commiting code with linting errors
refs https://github.com/TryGhost/Toolbox/issues/267
- Node 12 becomes EOL on April 30th so we're going to be dropping
support for it in Ghost
- this commit updates the Node engine ranges so CLI can pick this up,
and drops 12.22.1 from the CI matrix
refs 4aee97472e
- the referenced commit copied the workflow file to our shared templates
repository
- this commit switches to using that template
- this should help reduce duplication when we're doing v5
