closes: https://github.com/TryGhost/Toolbox/issues/193
- Our Casper fixture was several years out of date.
- I'd already updated the ghost-api to point to the latest API version, which was the main difference
- This makes sure the full fixture is up to date and correct, and therefore that we're truly testing if Ghost right now works
- It also adds instructions for how to update it in future
GOTCHA: the mock-express-style tests are failing if the API difference between test-theme and casper are different
- I've tried to look into why this is - it's something to do with the overridden route settings not working properly if the API version changes
- Given that we may not keep this style of testing AND we are definitely not keeping API versions, I'm ignoreing this for now
- To get around it, I'm setting both themes to be v4 API, so that Casper is exactly as in main
refs: https://github.com/TryGhost/Toolbox/issues/168
- Upgraded the remaining themes to be pinned to the canary API
- This required one minor fix because the edit URL has changed for authors in v4/canary
- I also ripped out everywhere that the theme ghost-api version was being pinned to canary, as this was unreliable and only happening in a few places
- It's also going to be unnecessary code as soon as we finish changing to only having one API version
- This test failed for me intermittently because the posts would be out of order
- I assume this is due to my super-powered M1 mac 😂
- This rewrite only aims to remove the dependency between the insertion order and the output order
- Everything else should be the same, and it still tests that the posts that are meant to be members only are exactly that
refs: https://github.com/TryGhost/Toolbox/issues/168
- The fixture represented Casper at Ghost version 1.0 which pre-dated the introduction of the posts_per_page config in package.json
- When Casper was upgraded to 2.0 in the fixtures, the lack of pagination broke the e2e tests for pagination
- This change introduces proper code to stub and override posts_per_page rather than keeping and using the old casper-1.4 fixture
- It also required me to remove a handful of CSS-based checks which are no longer true in the new theme version, but also didn't really add anything to the tests
no-issue
This test should have always been a unit test, but it now no longer
serves a purpose, as we do not rely on the event being emitted - it
would also not break Ghost if the event _was_ emitted, so we should not
be testing for that.
refs: https://github.com/TryGhost/Toolbox/issues/168
- All of our unversioned tests should be running against canary already
- These tests are erroneously running on the wrong version
We're going to be dropping the idea of having multiple versions of the API in each Ghost version.
Because this has not achieved the goal of making it easier to make breaking changes, but it has
created an ordinate amount of technical debt and maintenance overhead.
As we know this is going away in the next major, there is no benefit to us constantly running tests
that check if those versions still work, especially given how long they take.
Instead we're starting work to ensure that all of our test work on canary, and that canary has
excellent test coverage so that we can be sure that our one API version works really well and that
any changes, no matter how subtle are deliberate, tracked and understood.
refs: https://github.com/TryGhost/Toolbox/issues/168
- These are all places where we reference an API version like v2 or v3 but it's not actually
used or relevant.
- The aim is to get rid of all mentions of these old versions to make it clearer that we're only running tests on canary
refs: https://github.com/TryGhost/Toolbox/issues/168
We're going to be dropping the idea of having multiple versions of the API in each Ghost version.
Because this has not achieved the goal of making it easier to make breaking changes, but it has
created an ordinate amount of technical debt and maintenance overhead.
As we know this is going away in the next major, there is no benefit to us constantly running tests
that check if those versions still work, especially given how long they take.
Instead we're starting work to ensure that all of our test work on canary, and that canary has
excellent test coverage so that we can be sure that our one API version works really well and that
any changes, no matter how subtle are deliberate, tracked and understood.
refs https://github.com/TryGhost/Team/issues/1257
Offer Redemptions were being overcounted due to the way we were updating
Stripe configuration for the Members service. We would create a new
instance of the members-api, which would have event handlers for
creating Offer Redemptions - by creating a new instance each time Stripe
config changed, we would overcount them.
Here we've pulled out Stripe related logic into the Stripe service, and
updated it internally - rather than creating a new instance. This means
that we've been able to remove all of the logic for re-instantiating the
members-api.
- Bumped members-api & stripe-service
- Removed reinstantiation of members-api
- Used stripe service to execute migrations
- Updated Stripe Service to handle webhooks & migrations
- Used webhook controller from stripe service
- Used disconnect method from stripe service
- Removed unused stripe dependency
- Removed Stripe webhook config from members-api
Since we now have 2 products by default for all ghost sites, free and default paid, the usage of default product which so far was using first product needs to be updated to use the first paid product.
- updates default product usage to use first paid tier
- updates tests
refs https://github.com/TryGhost/Team/issues/1189
Support for AMP is slowly in decline, and makes developing new cards trickier,
since AMP no longer has an effect of SEO we're going to disable it by default
as a first step toward moving away from it.
Co-authored-by: Thibaut Patel <thibaut@ghost.org>
closes https://github.com/TryGhost/Zapier/issues/56
- fixes tag creation when creating posts with `tags: [{slug: 'new'}]` which should be supported
- assigning tags with only `{slug: 'new'}` was triggering our validation for the required `name` property then bubbling up to the `bookshelf-relations` library resulting in a 500 error
- the fix applied here is to set the `name` field to the same as the `slug` field if a name is not provided
refs https://github.com/TryGhost/Toolbox/issues/129
- The "Origin" header is require in Admin API requests so it makes sense to bake it into a default request made by most tests. Reduces unnecesary fluf around test request setup and removes a "config" dependency in each tests suite using the "e2e framework"
refs https://github.com/TryGhost/Toolbox/issues/158
- Allows for much smaller amount of code to configure a test to work with chai-jest-snapshots. They now work automatically for all regression tests and could be enabled for other suites by adding the "--require=./test/utils/snapshots.js" parameter in respective test:* package script
- Regenerated snapshot for authentication test as the naming structure
changed a little with the snapshot metadata being taken on a higher
level in the test (uses the suite name instead of a specific describe it
used to be called from)
refs https://github.com/TryGhost/Toolbox/issues/158
- Moving common mocking/stubbing/spying logic into an outside utils module allows test suites to keep agnostic towards which framework powers mocking etc. Should also substitute email service stubbing used in multiple places
- Bonus, got rid of should dependency, which is deprecated
refs https://github.com/TryGhost/Toolbox/issues/129
- This is an example of how the converted test suite syntax could look like when using jest snapshots.
- The tradeoff is not that visible just yet as these tests were mostly checking few fields, but when the whole range of admin API tests is convered we'll be able to get rid of the "checkResponse" utiliti methods along with all the supporting luggage!
refs https://github.com/TryGhost/Toolbox/issues/129
- Having a fresh module should allow building new testing utility concepts from the ground up without being tied by the mystery baggage of the legacy localUtils/testUtils
- The outline of the concepts the framework will be handling is in the commont on the top of the e2e-framework file
refs https://github.com/TryGhost/Toolbox/issues/152
- This refactor fixes the rest of authentication test suite and outlines the state setup steps that are absolutely needed for tests that rely on any db state. It's still counterintuitive why the fixture initialization has to be called and in that specific order, so next will be refactoring in this area to simplify the initi code
refs https://github.com/TryGhost/Toolbox/issues/152
- Passing around plain options object tends to become quite unreadable long term. While these new utils are being shaped up it's still easy to change interface and introduce new parameters with time as needed.
refs https://github.com/TryGhost/Toolbox/issues/152
refs 5bea089dfe
refs 16ad5f73c4
- The refactor is heavily inspired by the referenced commit from @ErisDS
- This refactor is meant to serve as a template for further refactors and unification of the abstraction over "request". Should allow us to be more agnostic towards the library that's powering the request thing. For example, mock-express style of request handling could substitute or get substututed easily if all tests are behind similar "get(Authenticated)Agent" interface
refs https://github.com/TryGhost/Toolbox/issues/152
- This is a continuation of an experiment to switch over to serverless boot in regression tests. Just a proofe of concept that authentication scenarios would also work and the expectations don't collapse unexpectedly.
- Nothing too crazy so far, easy and straight forward substitution of the config "url" with an express app passed to the supertest agent
refs https://github.com/TryGhost/Toolbox/issues/152
- Being able to set up test suites without blocking a port opens a door to new ways to run tests - for example this has been one of the blockers for running mocha tests in parallel
- Additional benefit is lighter statrup, which reducec the test execution time slightly. Doesn't seem like much but these things stack up!
refs https://github.com/TryGhost/Toolbox/issues/152
- Have skimmed through the test suites in hopes to find some quick performance wins to bring the runtime speed closer to the one in "main". Haven't been successful to identify major wins, cleaned up a couple of small bits.
- We'll have to live with a tradeoff between maintainability/unified boot VS cost of mainitaining a fake boot process. Imo extra couple seconds of runtime is worth it.
refs 3c7a8dead4
- The tests needed adjustments with the native boot mechanism.
- The number of returned posts changed in the test resutls because duing native boot we also insert fixtures which add to the number of initial posts
- The vhost regression suite is still failing and I had no strength to figure out why. The redirect it fails with makes no sense, the clue here is that the test doesn't fail when running in isolation, so probably has to do with some leftover overrides from the previous test cases.
refs 3c7a8dead4
- The boot process has been using an asyc method to load the routes file, which is the case now for these tests since the switch to raw boot method instead of mimicking it manually
refs 3c7a8dead4
- Simplifies the state initialization code significantly and reuses native boot mechanism instead of mimicking it (it was a headache to maintain with all the internal services moving around)
refs: TryGhost/Toolbox#147
* Replaces all references to isIgnitionError with isGhostError
* Switches use of GhostError to InternalServerError - as GhostError is no longer public
There are places where InternalServerError is not the valid error, and new errors should be added to the @tryghost/errors package to ensure that we can use semantically correct errors in those cases.
closes https://github.com/TryGhost/Toolbox/issues/148
- These regression tests introduce very little additional value in exchange for an expensive time to run them. Because we mostly care about stable support for the latest stable API version, the older API version test for "internal" API can go away
- In case there are bugs found we do care about in the v2/v3 APIs we can always revert some of these tests.
https://github.com/TryGhost/Toolbox/issues/140
- This test was bloating the regression/site suite and was using hacks (calling the Admin API) to create a custom redirects state
- It suits way better in e2e frontend test suite with less hacky approach to start the Ghsot instance - using custom routes file path to initialize the instance
refs https://github.com/TryGhost/Toolbox/issues/140
- Same/similar RSS tests were present in regression and e2e tests suites. It made sense to move missing cases from regression to e2e. This saves us time bootstraping db state in multiple places and keeps all test cases regarding single feature in same place
refs https://github.com/TryGhost/Toolbox/issues/139
- With ec2aed5ce8 the DynamiRedirectsManager has reached 100% test coverage and most of the tests present in the removed regression suite have been ported to unit tests
- No need to keep slow tests around! :)
refs https://github.com/TryGhost/Toolbox/issues/139
- The regression test suite for redirects functionality for way too big. And each restart was causing massive overhead. It's enough to have a single exhaustive test using multiple input files
- The tests testing API endpoints should've been e2e tests to start with
- The rest is covered in the unit tests for redirects api service
refs 91efa4605c
- Referenced commit introduced a double json-stringification to uploaded redirects.json files.
- The endpoint has no stability index of any sort and is meant to be dropped in Ghost v5. It's best to rework the redirects to the yaml format as descirbe here - https://ghost.org/docs/tutorials/implementing-redirects/#file-structure
refs https://github.com/TryGhost/Toolbox/issues/139
- The v2 and v3 redirects APIs are unofficial and should not be used by anyone in production. There's no good reason to maintain expensive to run test suites for old unofficial APIs.
- The test cases in canary suite covers the functionality of redirects enough to be sure they work as expeted
refs https://github.com/TryGhost/Toolbox/issues/138
- These tests still rely on the frontend to be present. Needs further investigation to remove "frontend: true" flag - it slows down test runs!
refs https://github.com/TryGhost/Toolbox/issues/138
- Final batch of the refactor to async/await syntax. Doing these refactors before modifying "testUtils.startGhost" everywhere to boot only with the backend
refs https://github.com/TryGhost/Toolbox/issues/138
- This is a continuation of a bigger refactor to use async/await syntax before migrating "startGhost" methods to only use backend boot
- Removed a little bit of dead code (like admin user creation) which should speed up test execution too!
- Refactored user variables to be declared closer to their usecases instead of being high up in a global scope - variables shoul not live that far apart from the code that uses them
refs https://github.com/TryGhost/Toolbox/issues/138
- First batch of the refactor to async/await syntax. Next one will cover the rest. Doing these refactors before modifying "testUtils.startGhost" everywhere to boot only with the backend
refs https://github.com/TryGhost/Toolbox/issues/138
- There is no good reason to keep this extra variable around just call "stop" in couple very specific cases. Even for those cases, there's `testUtils.stopGhost` method which achieves the same without additional variable to track.
refs https://github.com/TryGhost/Toolbox/issues/138
- Having the "ghost" alias only added cognitive load when reading through the test code and didn't provide any additional value. Removed the pattern to keep things simpler and more explicit
refs https://github.com/TryGhost/Toolbox/issues/138
- Using asycn/await syntax is way more readable and allows to identify further reusable patterns in test initialization. This refactor also served as an exploreation around how the code looks like at this point
