no issue
- The response from the server might not always contain an array with errors (for example when there's a generic 503 error). This changes prevents tests from hanging and adds at least some relevant output
refs https://github.com/TryGhost/Toolbox/issues/129
- Adds content folder creation based on fixtures for e2e tests using the e2e framework
- Mostly a copy-paste for now from existing e2e utils with some simplifications, so we can "complicate things" later only if needed
refs 24505db918
- The code is heavily borrowed from the referenced commit by @ErisDS. It starts a fresh abstraction layer for all e2e tests to use, provides simle interface for request agent initialization and state management (read the e2e-framework top file description for more)
- Main methods the framework exposes are:
- getAgent - to initialize request agent with a fresh Ghost instance
- initFixtures - to initializer a fresh db state
- resetDb - cleans up the db state if there were state manipulations during the test suite run (POST, PUT requests or non-default fixtures)
refs https://github.com/TryGhost/Toolbox/issues/129
- Having a fresh module should allow building new testing utility concepts from the ground up without being tied by the mystery baggage of the legacy localUtils/testUtils
- The outline of the concepts the framework will be handling is in the commont on the top of the e2e-framework file
refs https://github.com/TryGhost/Toolbox/issues/152
- Being able to set up test suites without blocking a port opens a door to new ways to run tests - for example this has been one of the blockers for running mocha tests in parallel
- Additional benefit is lighter statrup, which reducec the test execution time slightly. Doesn't seem like much but these things stack up!
no issue
- we've seen some instances of SQLite saying "database disk image is malformed"
- I think this happens because we copy the file whilst we are still connected to
the old DB
- this commit destroys the connection before copying the clean file to the live
DB file
refs https://github.com/TryGhost/Toolbox/issues/136
- we nuke and reinitialize the DB many times between tests
- this forms a good portion of the time taken and we shouldn't be spending
so much time on just resetting the DB back to a known state
- this commit switches out the knex-migrator reset + init calls for SQLite to
a function which keeps a clean copy of the DB and copies the file back
when we "reset"
- for MySQL, existing functionality is kept
- this massively speeds up tests because it saves ~700ms+ for every reset
- whilst this change seems to work, it's just the start so there's a
lot more refactoring needed. this change is currently gated to CI until
it's deemed safe/sane enough to run on local machines without blowing
up
refs: TryGhost/Toolbox#147
* Replaces all references to isIgnitionError with isGhostError
* Switches use of GhostError to InternalServerError - as GhostError is no longer public
There are places where InternalServerError is not the valid error, and new errors should be added to the @tryghost/errors package to ensure that we can use semantically correct errors in those cases.
https://github.com/TryGhost/Toolbox/issues/140
- Allows to fully start an instance with a custom routes.yaml file without a need to do workarounds - e.g. we used to call an internal Admin API from a regression test suite, which is not a good practice
- Providing "routesFilePath" to startGhost method copies that file to the default settings location and start the instance with that configuration. No need to do API calls or check if the routing service "isFinished"
no issue
- this commit adds a counter for the number of boots we do in tests
- which therefore allows us to calculate the average boot time we
experience
- only useful for debugging test performance
refs https://github.com/TryGhost/Toolbox/issues/135
- This optimization is expected to play a role in more consistent "backend-only" boot where the previous test state might have left over a different theme version which might cause in unwanted URL Services reainitializations.
- What has been happening here is the themes.test.js suite was uploading a theme with a v4 api and when the users api test suite loaded up it switched back to a default v2 theme, which caused routing reinitialization
- The root problem here is the themese suite is leaving a mess behind so a "restartModeGhostStart" is not really possible anymore - this should be cleaned up separately
refs https://github.com/TryGhost/Toolbox/issues/135
- The global default should stay the same as it used to be and we can introduce an override for "withFrontend:false" on casa-by-case or area-by-area bases
refs https://github.com/TryGhost/Toolbox/issues/135
- When running e2e-API test in most cases there's no need to boot Ghost instance with full frontend. This should improve the boot time which should reflect on the speed of running test suites
- The tests where the "forceStart" and "withFrontend" are used together indicate that there's still some work to do to fully separate frontend/backend boot line. The force start is also unnecessary, but was needed to reinitialize all services properly - should be investigated!
refs https://github.com/TryGhost/Toolbox/issues/135
- These fixture had to be updated due to hardcoded ids introduced in the fixtures.json for test
- This allows to boot up a Ghost instance without recalculating urls for most common database states
refs https://github.com/TryGhost/Toolbox/issues/135
- These fixtures needs id's to be able to form a stable urls and resources cache that would be used across all e2e tests when booting without a frontend
- These two things are meant to improve performance at the cost of reliability.
- Perfect for testing, however I think they make a minimal impact on modern SSDs :(
- Still worth a shot to see if it helps with CI
refs https://github.com/TryGhost/Toolbox/issues/116
- Allows to have deterministic routing on each start of the instance - can use cache when starting tests
- To make use of the new fixtures need to place them in the content/data
folder when the test suite is initialized, then the cache detection will
kick in and boot up the UrlService in a constant time intead of having
to check "urlServiceUtils.isFinished"
refs: https://github.com/TryGhost/Toolbox/issues/133
- instead of just a collection of utils, we now have a class that manages fixtures
- this should allow us to change the path to fixtures, e.g. between prod/dev and test, so that different fixtures can be loaded by default
- also makes it easier to test the fixture manager code itself
refs https://github.com/TryGhost/Members/commit/9e59f5a9
Since we have a DynamicRedirectManager for handling adding/removing
redirects at runtime, we no longer need the custom-redirects middleware.
The redirects service does however need an init method now to add the
custom redirects at Ghost boot, so it's been refactored into our Class &
DI pattern.
refs https://github.com/TryGhost/Team/issues/1175
We found the ETag header sent when serving the Admin template for /ghost/ was not changing between versions which after an upgrade could result in out of date cached content being served containing links to JS/CSS files that no longer existed.
The culprit is weak etags served by Node's `send` package, coupled with Admin template filesize not changing between versions and `npm pack` setting a fixed modification date for every file. See https://github.com/pillarjs/send/issues/176 for more details.
- updated the Admin app's controller to read the template and generate an md5 hash of the contents so we can serve a strong ETag header value when serving the `/ghost/` html
- done a fastest-possible overhaul on this style of tests to try to get them to work independently again
This is a pattern that was introduced a while ago to try to speed up our e2e tests and I'm not sure if it's staying or going
It uses a minimal frontend-only version of the boot process and a custom-built express testing tool
However it's really old and out of date because of the boot refactor and several changes since
This highlights the key problem with it - it doesn't rely on any of our "core" boot process, it makes it up, and therefore how reliable are these tests?
Ideally we need to get these tests working with the real boot process in some capacity
We would then need to make sure we have all the tests in e2e-frontend written in this style
refs https://github.com/TryGhost/Team/issues/1104
- bumped `@tryghost/custom-theme-settings-service` so it throws a more appropriate `ValidationError` when setting keys don't exist or a select value is not known
- changed the custom theme settings service to have a `.init()` method which creates an instance of the service under `.api` so that we're able to create the instance at a particular point in the boot process when we know the models have been initialised
- there were problems in tests because the service was being initialised through the require chain before models were initialised through the boot process
- fixed incorrect `camelCase` of resource name in API responses
refs https://github.com/TryGhost/Team/issues/1070
- stores values of custom theme settings
- will be merged with full settings data parsed from themes for API output
- will be cached and made available for lookup in themes to avoid db roundtrips
- stores type of custom theme settings so we can coerce values and know if the type has changed when syncing
- records will be synced with themes upon activation
refs https://github.com/TryGhost/Team/issues/946
This adds the initial bulk actions endpoint used for the members
filtering feature. The idea is to eventually move bulk destroy into this
endpoint to and provide a consistent interface for applying bulk actions
to members.
The @tryghost/members-api package has been bumped to include the new
bulkEdit method.
The sinon.restore in tests was moved to an afterEach so that stubs did
not effect other tests.
refs https://github.com/TryGhost/Team/issues/714
In order to order products by their monthly price we need to apply a
join with the stripe_prices table when querying so we have access to the
amount column of stripe_prices.
As this ordering is core to how the tiers feature is intended to work,
we have added it as the default order. But this can be overriden by
manually passing the order option.
Also ensured that we do not create duplicate products in test fixtures
- This isn't really a "service" - it's a set of utilities for working with labs flags
- It's also required all over the place, and doesn't require anything that isn't shared
- Therefore, it should live in shared
- This isn't really a "service" - it's a set of utilities for working with labs flags
- It's also required all over the place, and doesn't require anything that isn't shared
- Therefore, it should live in shared
- In an ideal world, our acceptance tests would be much, much faster
- ATM we output how much time is spent on reloading Ghost for each suite, but this is output with console
- Changed this to use DEBUG, so we don't clutter the UI normally
- Added further debug statements, and a cumulative time, so we can see where time is spent/wasted
- Added a DEBUG command for running acceptance tests with this output
- This shows us that reloading Ghost accounts for 50% of the test time
- this was skipped on boot, but then called in the test utilities, but only on restart 🙈
- this means that yarn test:acceptance (i.e. running all tests) works, but if you try to run just test/api-acceptance/themes_spec.js it would fail because that uses a fresh boot not a restart/reload
- I've changed this as keeping the test using the real boot, rather than the made-up acceptance-utils tasks as much as possible is way better
This commit achieves a few things:
- ☑️ No longer having to remember whether a command is yarn something or grunt something
- ☑️ Simplification of tools hopefully making them easier to remember and use
- ☑️ Complete removal of the need for grunt from our test tooling
Several of the tools still use grunt under the hood, but the **entrypoint** should aways be `yarn xxx`.
- `grunt main` -> `yarn main`
- `grunt dev` -> `yarn dev`
- `grunt build` -> `yarn build`
- `grunt test:file-or-folder` -> `yarn test file-or-folder`
- `grunt test-unit` -> `yarn test:unit`
- `grunt test-acceptance` -> `yarn test:acceptance`
- `grunt test-regression` -> `yarn test:regression`
- `grunt validate` -> removed due to lack of use
There is now also `yarn test:all` to run all 3 classes of tests
This PR also reorders & restructures the Gruntfile extensively so that:
- The remaining useful commands are all at the top of the file
- Config and other blah happens after all the useful commands
- All release-only config happens in the release task at the very end of the file
---
DONE:
* Removed all references to npm/bower
* Removed all references to lint / deprecated command
* Moved debug to yarn dev:debug
* Removed all references to travis
* Removed broken help task + useless comment
* Removed unused knex-migrator and clean:test setup tasks
* Added new test commands, removed grunt validate
* Moved stubClientFiles to test utility and use in the few tests that need it
* Used mocha in yarn directly except grunt test:x
* Swapped grunt test for yarn test
* extensive cleanup and reshuffling
- This is part of the quest to separate the frontend and server & get rid of all the places where there are cross-requires
- At the moment the settings cache is one big shared cache used by the frontend and server liberally
- This change doesn't really solve the fundamental problems, as we still depend on events, and requires from inside frontend
- However it allows us to control the misuse slightly better by getting rid of restricted requires and turning on that eslint ruleset
