- This part of the framework can be used in isolation
- Using mockManager everywhere makes it more visible how to use it
- Aside: fixed .getAdminAPIAgent not needing a URL in site tests whilst cleaning up
refs daeb06e835
- This is an additive change, and the parameter is optional so it's better to ship it right away (testing was hard as it's a chicken and an egg problem to have a feature flag in the setup route).
refs: https://github.com/TryGhost/Toolbox/issues/158
- We only use chai in a handful of places now, and it seems totally unnecessary
- Use assert instead
- Made other minor changes with a view to this being a reference aka "perfect" test suite
refs: https://github.com/TryGhost/Toolbox/issues/158
- if we had to combine a large list of params then this would make sense
- we could/should also add a .query() method to our agent
- however, I don't think this is worthwhile/necessary for just 2 params right now :)
refs: https://github.com/TryGhost/Toolbox/issues/158
- continuing to turn this into a reference test suite for how to do things with the new framework
- use fixtureManager and mockManager properly
- testUtils should _not_ be required anymore!
- TODO: cleanup labs mocking and side effect assertion using models
refs: https://github.com/TryGhost/Toolbox/issues/158
- rather than just exposing any, anything and string matching, expose more specific matchers.
- this was triggered by `any(Date)` not working for dates in our API
- it seems poor to match `any(String)` for something we want to be a well formatted date
- establishes the pattern of using our defined matchers instead of requiring any/anything from jest
refs: https://github.com/TryGhost/Toolbox/issues/158
- Working up to having these be the defacto "reference" tests
- Changing this allows for the fact that there will be a getContentAPIAgent, and probably more in future
- Abstracting away the path to a single location will make updating API paths easier later too
refs TryGhost/Toolbox#158
refs TryGhost/Toolbox#129 (comment)
We now have @tryghost/jest-snapshot - a suite of utilities for using snapshots with mocha, should, etc
The @tryghost/express-test lib also uses it to provide chained snapshot assertions
This library was created because all the existing implementations I could find, including the in use @ethanresnick/chai-jest-snapshot
didn't properly support property matching. @ethanresnick/chai-jest-snapshot supposedly supported it, but the implementation was incorrect
and frequently lead to false test passes.
This library also has (after some back and forth) path resolution so that snapshots can live in a local __snapshots__ folder
The idea here is to keep the concept of _what_ we're asserting (that an email was sent)
away from the implementation of how we check that assertion
This means that tests can have a consistent api like .sentEmail(), and if we change how that function works,
we only have to update the assertion function in the MockManager
Much more cleanup to come behind the scenes, but the aim is to make the tests as clean as possible
- encapsulated concerns within individual objects
- this will allow us to refactor these into classes or move them around later
- also makes it clearer how methods like restore relate to other methods
- e.g mocks.restore() restores mocks, whilst fixtureManager.restore() restores the database
- Ensure that the DB is always reset between tests
- We assume that the DB will be torn down and rebuilt between tests
- Without this, previous tests clobber the current tests, and it's not possible to tell when running files in isolation
which makes developing really tricky
refs https://github.com/TryGhost/Team/issues/1296
- The `theme` must be a github `org/repo` string
- This uses the internal API instead of the services because the API has extra implementation details not present in the services.
- This is so that we can start refactoring more tests to use our new e2d framework
- That framework doesn't support file uploads yet, so tests with uploads are split out
refs https://github.com/TryGhost/Team/issues/1071
Default content visibility for a post can be one of `public|members|paid|tiers`, where `tiers` denotes visibility restricted to specific tiers. This change adds a new setting to store the tier ids when default content visibility is set to `tiers`. This closely matches how the visibility is stored on `posts` table as well, with `visibility` stored as `tiers` and tiers data is stored on tiers pivot table.
refs TryGhost/Team#1071
- new `tiers` key is now attached to posts/pages API response to include tiers visibility
- updates expected response for post/page in tests to include `tiers`
- This test failed for me intermittently because the posts would be out of order
- I assume this is due to my super-powered M1 mac 😂
- This rewrite only aims to remove the dependency between the insertion order and the output order
- Everything else should be the same, and it still tests that the posts that are meant to be members only are exactly that
refs: https://github.com/TryGhost/Toolbox/issues/168
- These are all places where we reference an API version like v2 or v3 but it's not actually
used or relevant.
- The aim is to get rid of all mentions of these old versions to make it clearer that we're only running tests on canary
refs: https://github.com/TryGhost/Toolbox/issues/168
We're going to be dropping the idea of having multiple versions of the API in each Ghost version.
Because this has not achieved the goal of making it easier to make breaking changes, but it has
created an ordinate amount of technical debt and maintenance overhead.
As we know this is going away in the next major, there is no benefit to us constantly running tests
that check if those versions still work, especially given how long they take.
Instead we're starting work to ensure that all of our test work on canary, and that canary has
excellent test coverage so that we can be sure that our one API version works really well and that
any changes, no matter how subtle are deliberate, tracked and understood.
closes https://github.com/TryGhost/Zapier/issues/56
- fixes tag creation when creating posts with `tags: [{slug: 'new'}]` which should be supported
- assigning tags with only `{slug: 'new'}` was triggering our validation for the required `name` property then bubbling up to the `bookshelf-relations` library resulting in a 500 error
- the fix applied here is to set the `name` field to the same as the `slug` field if a name is not provided
refs https://github.com/TryGhost/Toolbox/issues/129
- The "Origin" header is require in Admin API requests so it makes sense to bake it into a default request made by most tests. Reduces unnecesary fluf around test request setup and removes a "config" dependency in each tests suite using the "e2e framework"
refs https://github.com/TryGhost/Toolbox/issues/158
- Allows for much smaller amount of code to configure a test to work with chai-jest-snapshots. They now work automatically for all regression tests and could be enabled for other suites by adding the "--require=./test/utils/snapshots.js" parameter in respective test:* package script
- Regenerated snapshot for authentication test as the naming structure
changed a little with the snapshot metadata being taken on a higher
level in the test (uses the suite name instead of a specific describe it
used to be called from)
refs https://github.com/TryGhost/Toolbox/issues/158
- Moving common mocking/stubbing/spying logic into an outside utils module allows test suites to keep agnostic towards which framework powers mocking etc. Should also substitute email service stubbing used in multiple places
- Bonus, got rid of should dependency, which is deprecated
refs https://github.com/TryGhost/Toolbox/issues/129
- This is an example of how the converted test suite syntax could look like when using jest snapshots.
- The tradeoff is not that visible just yet as these tests were mostly checking few fields, but when the whole range of admin API tests is convered we'll be able to get rid of the "checkResponse" utiliti methods along with all the supporting luggage!
refs https://github.com/TryGhost/Toolbox/issues/129
- Having a fresh module should allow building new testing utility concepts from the ground up without being tied by the mystery baggage of the legacy localUtils/testUtils
- The outline of the concepts the framework will be handling is in the commont on the top of the e2e-framework file
refs https://github.com/TryGhost/Toolbox/issues/152
- This refactor fixes the rest of authentication test suite and outlines the state setup steps that are absolutely needed for tests that rely on any db state. It's still counterintuitive why the fixture initialization has to be called and in that specific order, so next will be refactoring in this area to simplify the initi code
refs https://github.com/TryGhost/Toolbox/issues/152
- Passing around plain options object tends to become quite unreadable long term. While these new utils are being shaped up it's still easy to change interface and introduce new parameters with time as needed.
refs https://github.com/TryGhost/Toolbox/issues/152
refs 5bea089dfe
refs 16ad5f73c4
- The refactor is heavily inspired by the referenced commit from @ErisDS
- This refactor is meant to serve as a template for further refactors and unification of the abstraction over "request". Should allow us to be more agnostic towards the library that's powering the request thing. For example, mock-express style of request handling could substitute or get substututed easily if all tests are behind similar "get(Authenticated)Agent" interface
refs https://github.com/TryGhost/Toolbox/issues/152
- This is a continuation of an experiment to switch over to serverless boot in regression tests. Just a proofe of concept that authentication scenarios would also work and the expectations don't collapse unexpectedly.
- Nothing too crazy so far, easy and straight forward substitution of the config "url" with an express app passed to the supertest agent
refs https://github.com/TryGhost/Toolbox/issues/152
- Being able to set up test suites without blocking a port opens a door to new ways to run tests - for example this has been one of the blockers for running mocha tests in parallel
- Additional benefit is lighter statrup, which reducec the test execution time slightly. Doesn't seem like much but these things stack up!
closes https://github.com/TryGhost/Toolbox/issues/148
- These regression tests introduce very little additional value in exchange for an expensive time to run them. Because we mostly care about stable support for the latest stable API version, the older API version test for "internal" API can go away
- In case there are bugs found we do care about in the v2/v3 APIs we can always revert some of these tests.
refs https://github.com/TryGhost/Toolbox/issues/139
- The regression test suite for redirects functionality for way too big. And each restart was causing massive overhead. It's enough to have a single exhaustive test using multiple input files
- The tests testing API endpoints should've been e2e tests to start with
- The rest is covered in the unit tests for redirects api service
refs 91efa4605c
- Referenced commit introduced a double json-stringification to uploaded redirects.json files.
- The endpoint has no stability index of any sort and is meant to be dropped in Ghost v5. It's best to rework the redirects to the yaml format as descirbe here - https://ghost.org/docs/tutorials/implementing-redirects/#file-structure
refs https://github.com/TryGhost/Toolbox/issues/139
- The v2 and v3 redirects APIs are unofficial and should not be used by anyone in production. There's no good reason to maintain expensive to run test suites for old unofficial APIs.
- The test cases in canary suite covers the functionality of redirects enough to be sure they work as expeted
refs https://github.com/TryGhost/Toolbox/issues/138
- These tests still rely on the frontend to be present. Needs further investigation to remove "frontend: true" flag - it slows down test runs!
refs https://github.com/TryGhost/Toolbox/issues/138
- Final batch of the refactor to async/await syntax. Doing these refactors before modifying "testUtils.startGhost" everywhere to boot only with the backend
refs https://github.com/TryGhost/Toolbox/issues/138
- This is a continuation of a bigger refactor to use async/await syntax before migrating "startGhost" methods to only use backend boot
- Removed a little bit of dead code (like admin user creation) which should speed up test execution too!
- Refactored user variables to be declared closer to their usecases instead of being high up in a global scope - variables shoul not live that far apart from the code that uses them
refs https://github.com/TryGhost/Toolbox/issues/138
- First batch of the refactor to async/await syntax. Next one will cover the rest. Doing these refactors before modifying "testUtils.startGhost" everywhere to boot only with the backend
refs https://github.com/TryGhost/Toolbox/issues/138
- There is no good reason to keep this extra variable around just call "stop" in couple very specific cases. Even for those cases, there's `testUtils.stopGhost` method which achieves the same without additional variable to track.
refs https://github.com/TryGhost/Toolbox/issues/138
- Having the "ghost" alias only added cognitive load when reading through the test code and didn't provide any additional value. Removed the pattern to keep things simpler and more explicit
