refs https://github.com/TryGhost/Team/issues/1469
Previously, members were subscribed to all available newsletters by default when added. This change updates the default newsletters subscription for member to take into account newsletter preferences for auto opt-in(`subscribe_on_signup`) as well as `visibility`.
closes https://github.com/TryGhost/Team/issues/1491
With multiple newsletters feature, a site should always have at-least one newsletter by default. Also, as with the default product, the default newsletter also needs to be renamed to the site title during the setup flow.
- adds default newsletter to main and test fixtures
- updates setup flow to rename newsletter name and sender name to site title
- updates model to extend default value for fields
- updates test
refs https://github.com/TryGhost/Team/issues/1457
We want to save the MRR with a subscription to simplify the calculation of the total MRR once, in 5.0, we also take 'forever' offers into account into the MRR (so we can just SUM the MRR of all subscriptions).
- Sets the MRR to 0 for now.
- Separate commit will fill in all the values in a data migration, but this needs to get merged first because we need this new column in order to update the members-api package (so we already save the MRR before doing the data migration).
- Updated `test/e2e-api/admin/legacy-members.test.js` with improved body assertions.
refs https://github.com/TryGhost/Team/issues/1500
The newsletter table schema has bunch of changes to go through for new and existing columns, this consolidates the schema changes into a single re-create table migration that drops and adds the newsletter table with correct schema. The table re-create migration needs to run before any of the tables using newsletter as foreign key. The changes include -
- new columns for design related fields
- new slug column for filtering
- unique constraint to `name` column
- remove `default` column (noops the existing default column migration)
- `sender_reply_to` has a default of newsletter and a validation of ['newsletter', 'support']
- updated default values for `subscribe_on_signup` and `recipient_filter`
refs https://github.com/TryGhost/Team/issues/1469
Currently, all new members get auto subscribed to the default newsletter. This change adds same behavior with multiple newsletters by auto subscribing all available newsletters on site for new members(If flag is enabled).
Note: In future, this will also take into consideration the `subscribe_on_signup` flag for a newsletter to filter which newsletters should a member be auto-subscribed.
- adds newsletters service for working with newsletter data
- bumps `@tryghost/members-api` package which handles default subscription
- adds new test fixture/data for newsletters
refs https://github.com/TryGhost/Team/issues/1463
- Allow admins to perform all newsletter operations
- We can adjust and be more permissive in the future if needed
- Added the tests back as permissions are configured correctly now
- Allows mocking lib/common/events - stubs event.emit
- Means we can check that events fired, but their sideeffects won't happen, useful for things like bulk email
- I also reorganised the mock manager file to group related functions together
refs: https://github.com/TryGhost/Team/issues/1446
- changed mail API tests to use the new e2e test framework
- added the missing retry test - which has the wrong response format :(
mail
refs https://github.com/TryGhost/Toolbox/issues/241
- The `engines.ghost-api` property has been deprecated and the support for it will be dropped in Ghost v5 due to versionless nature of the Content API.
- When uploading a new theme or activating existing one that uses ghost-api in it's config a warning will be shown to the user
refs https://github.com/TryGhost/Toolbox/issues/241
- The `engines.ghost-api` property has been deprecated and the support for it will be dropped in Ghost v5 due to versionless nature of the Content API.
- When uploading a new theme or activating existing one that uses ghost-api in it's config a warning will be shown to the user
- Attempted to make it clearer what the db utils do and what each one is for
- Clearly marked the old clearData & initData as deprecated, we don't want to mix db and url service resets
- The fast truncateAll method can fail
- If it does, use knex-migrate to do a safe reset of the DB instead
- This is being done to try to make test runs more reliable in the face of database errors
- if the database does not exist, ignore the error
- tables that don't exist don't need truncating
- instead wait until the next thing calls init, that will cause a fresh db to be created in the correct state
- inside of reset, if the truncate fails then do a full reset instead
- some code in tests were still using various unreliable methods to determine the database in use
- DatabaseInfo is the correct method to check if the db is mysql or sqlite
- Exposed DatabaseInfo via our test db-utils to make this easier
- We've been seeing weird errors with tables not existing when running tests locally
- This appears to happen after an error causes the tests to abort
- This change includes two fixes:
1. we triggers a full DB reset just before the entire test suite runs
- this is done by wrapping the override file for tests that use a db, and supplying a mochaGlobalSetup hook
2. catch errors when attempting to do a fast truncation-based reset & init, and do a full reset & init instead
- These two changes should ensure the DB is always in the state we expect when running a new test suite
closes https://github.com/TryGhost/Team/issues/1426
When fetching tiers using the content API, we incorrectly returned all tiers including archived ones unless the active:true filter is passed. Correct behaviour is to always hide archived tiers, so this filter should not be required.
- forces `active:true` filter for tiers content api browse
- updates test to check for archived test removal in tiers content api
- the setup() function returns a function that's expected to be called as a mocha hook
- wrapping this in a function means it doesn't get called properly
- therefore the db setup was never being called for this test
refs https://github.com/TryGhost/Toolbox/issues/169
- Before releasing Ghost v5 we would like to move all canary-related URLs to a non-versioned format, which will become a default in v5.
- 'canary' is by definition unstable, so breaking any unprepared client explicitly using the canary is expected
- Removed the aliased /content/ and /admin/ apps from app.js because with updated configuration they become duplicates of 'canary' endpoints
refs https://github.com/TryGhost/Team/issues/1363
- When uploading a zip of images in Settings > Labs > [Import], it will skip images that have an uppercase extension, citing an 'unsupported file type' error.
- Cause: Glob ignored those files when matching extensions in ImportManager
- Fix: Added nocase option where needed
- Extended tests to also test the processZip method of ImportManager with getFilesFromZip
- Added isValidZip for zip with uppercase image
- Cleaned up JSDoc in ImportManager, and replaced some older JS syntax
Fixed zipContainsMultipleDataFormats error never thrown:
When a zip combines two data formats, no error was thrown.
- The promise error was only returned in an _.each loop, but never thrown
- Previously when combining multiple data types in a zip file, no error got thrown
- Added a test for this error
- Also added a test for noContentToImport error
Other errors and fixes:
- Added missing length in getBaseDirectory check
- getContentTypes fixed (returned duplicate values). Type error came up after adding all JSDocs
- updated tests to match real types from JSDoc and pass type validations
- Rewrote some methods in the async await syntax
- Added tests for ImportManager clean up
Some tests need to run without Stripe connected/enabled, which is the
default at the moment. This method will disconnect stripe for the
lifecycle of the API agent.
no issue
- The support for the misformated roon imports was temporary anyway, based on the comments in code. It's also unecessary to keep around any code related to Roon as it's been _ages_ since anybody needed this kind of migration
refs f48892028d
refs 33da584161
- There was another mather added recently: anyDateWithTimezoneOffset. The naming of date-related matchers was getting long and confusing. Renamed existing date matcher to better reflect what they do and they types of matches they are responsible for, following renames have been done:
- anyShortDate -> anyISODate
- anyDate -> anyISODateTime
- anyDateWithTimezoneOffset -> anyISODateTimeWithTZ
refs https://github.com/TryGhost/Toolbox/issues/215
- The ContentAPI needs it's own test agent, so we can write e2e tests.
- The main method mostly to be used by the test suites is "authenticate" - it add necessary authentication keys to the request. The agent is not authenticated by default because there are suites that need to test the "non authenticated" requests. Also, there's a need to have the default API key inserted from fixtures level before authenticating (it's not strictly necessary because the key is not dynamic, but I think coupling this point would be a bad move)
refs https://github.com/TryGhost/Toolbox/issues/215
- Ghost tests had difficulty running sometimes when the versions for jest-snapshot package did not match in Ghost and @tryghost/express-test
- This is the error that was showing up: `IncorrectUsageError: Unable to run snapshot tests, current test was not configured`
- The reason why snapshot tests were misconfigured was multiple instances of SnapshotManager, which broke the singleton pattern
- Having jest-snapshot embeded within express-test makes sure the versions stay the same across the clients
- The version bump also introduces passing "queryParams" parameter into the Agent constructor - enables configuring query parameters that would appear in each agent's request. Example usecase - Content API authentication parameter "key" would be nice to "remember" and add to every request URL
refs https://github.com/TryGhost/Toolbox/issues/214
- These variables need to be present in the configuration during the Ghost's boot time initialization, which caused a need to remember to mock settings cache before agent initialization.
- By moving the values into default settings config it removes the need to do any work during test environment setup. Yey!
- We should put default values in to test-specific settings-defaults.json In similar situations. Specifically, when we find a need to mock settings cache to be able to start Ghost instance in a certain state
refs https://github.com/TryGhost/Toolbox/issues/214
- Having this config for test environment allows to pre-populate default settings values in the settings table
- Right now the default-settings.json is an exact copy of the original "/data/schema/default-settings/default-settings.json". Having a starter file as an exact copy, allows to track the differences between environments as they are introduced easier
refs https://github.com/TryGhost/Toolbox/issues/214
- TestAgent was used to initialize both Admin & Member API agents, which is somewhat confusing because Member API does not have the same "loginAs" functionality like Admin API does
- Having distinct agents for each API makes the class API cleaner with possibility to extract common functionality even further
- Fixed test fixtures so that members with subscriptions also have products/tiers
- Fixed test fixtures so that default&free tiers can be updated for tests
- Added tests for the signin functionality and welcome page redirects
- Extended `setupStripe` to setup other Members settings - this needs some more
thought around how we proceed
- the previous logic only allowed one flag to be mocked at a time because it kept recalling sinon.stub
- now it's possible to mock multiple flags with different settings as we always just add to the same stub
no issue
- The test was failing because the match for a the data has changed, which was returned in a short format YYYY-MM-DD.
- Added a new matcher for short date format -anyShortDate. Can be used for match dynamic dates in short format
refs https://github.com/TryGhost/Toolbox/issues/214
- Having to fill out `process.env` variables in tests is a frustrating developer experience. Test environment should be configured with smart defaults, so the developer writing test cases would modify variables like this only is special test cases.
refs https://github.com/TryGhost/Toolbox/issues/214
- Calling `getMembersAPIAgent` and `getAdminAPIAgent` separately was booting Ghost twice, which caused a significant performance degradation.
- Additionally, having two calls was slightly ugly and having once utility function that delivers multiple agents at once feels like more readable syntax
- Often in our API we want to check that the location string looks roughly right for a resource
- At the moment we're matching any String, this upgrades the check to look for resource URLs
- I noticed our test fixture theme was still throwing errors during tests because it had partials that are no longer in use
- Updated the docs on how to update Casper to include ensuring Casper was on a valid version AND removing Casper before
copying it across
- Performed the exact steps in the README
- This ensures that the Casper version we use in our tests is not throwing tonnes of errors and is up to date
refs https://github.com/TryGhost/Team/issues/1338
This adds some initial scaffolding to make it easier to test Members with Stripe
- `mockStripe` method to disable the network, so we can use nock to intercept
Stripe API calls
- `setupStripe` method to be called _before_ getting an agent so that Stripe is
configured
- `getMembersAPIAgent` to get an agent for the Members frontend API
refs: https://github.com/Ghost/Ghost/commit/b5ee17b25
- When moving the site tests into e2e-api they broke because they were getting the changed settings value from the settings test
- The solution is to ensure the settingsCache is reset between tests, which is what this commit does
- This commit also renames the shutdown method to reset, because this is not a permanent operation
- It also renames the resetDb method to resetData, because the concept is we want the internal data to be reset, not just the DB
no-issue
Because we were returning the call to `boot`, rather than awaiting it,
it meant that once a test was provided with an API Agent the Ghost
application hadn't necessarily started. This was noticed whilst working
on the Members API which requires the frontend to be loaded.
When the frontend is loaded we must also wait for the url service to
have finished initialising, so that we do not run into 503 maintenance
errors when hitting the frontend.
no-issue
The Members API is served on a different endpoint to the Admin API, and
also requires that the frontend is booted. This agent is to be used when
testing the Members API, e.g. Stripe webhooks or Members config.
- The admin API test agent doesn't boot the Ghost frontend, which is where members routes are currently located
- This means we can't test members properly as we don't have access to webhooks etc
- This change adds a members option to getAdminAPIAgent(), which in turn enables the frontend
- We do it this way so that we can easily change the implementation later, e.g. if we have the option to boot members directly
- When starting ghost for e2e tests we create a content folder in the os tmp dir
- This means that the folder can change between suites as ghost is started and restarted
- For the most part this is fine, but theme storage caches the path to config (which makes sense, it's not meant to change whilst Ghost is in-memory)
- This is a quick-n-dirty fix that just makes it possible to update that path in the tests, so we know it's in sync
- Ideally we'd not cache the path, use a function to fetch it etc, or fully reset the theme storage layer, but this is the fix I have working today
and so it's going in to unblock things for now
- Can now easily mock labs flags as enabled or disabled using mockManager
- Updated some bits of code that directly mocked labs
- Aside: improved error thrown when things go wrong booting Ghost
refs: https://github.com/TryGhost/Toolbox/issues/158
- continuing to turn this into a reference test suite for how to do things with the new framework
- use fixtureManager and mockManager properly
- testUtils should _not_ be required anymore!
- TODO: cleanup labs mocking and side effect assertion using models
refs: https://github.com/TryGhost/Toolbox/issues/158
- rather than just exposing any, anything and string matching, expose more specific matchers.
- this was triggered by `any(Date)` not working for dates in our API
- it seems poor to match `any(String)` for something we want to be a well formatted date
- establishes the pattern of using our defined matchers instead of requiring any/anything from jest
refs: https://github.com/TryGhost/Toolbox/issues/158
- Working up to having these be the defacto "reference" tests
- Changing this allows for the fact that there will be a getContentAPIAgent, and probably more in future
- Abstracting away the path to a single location will make updating API paths easier later too
refs TryGhost/Toolbox#158
refs TryGhost/Toolbox#129 (comment)
We now have @tryghost/jest-snapshot - a suite of utilities for using snapshots with mocha, should, etc
The @tryghost/express-test lib also uses it to provide chained snapshot assertions
This library was created because all the existing implementations I could find, including the in use @ethanresnick/chai-jest-snapshot
didn't properly support property matching. @ethanresnick/chai-jest-snapshot supposedly supported it, but the implementation was incorrect
and frequently lead to false test passes.
This library also has (after some back and forth) path resolution so that snapshots can live in a local __snapshots__ folder
refs a3cc66be50
- in the referenced commit, I made a util to speed up resetting the DB
for SQLite by copying the database file
- I inadvertently removed an optimization we had before - where we
truncate the tables and insert the fixtures instead of dropping the
entire database
- this would be missing on MySQL tests
- this seems to have a big difference so this commit re-adds the
optimization in
The idea here is to keep the concept of _what_ we're asserting (that an email was sent)
away from the implementation of how we check that assertion
This means that tests can have a consistent api like .sentEmail(), and if we change how that function works,
we only have to update the assertion function in the MockManager
Much more cleanup to come behind the scenes, but the aim is to make the tests as clean as possible
- encapsulated concerns within individual objects
- this will allow us to refactor these into classes or move them around later
- also makes it clearer how methods like restore relate to other methods
- e.g mocks.restore() restores mocks, whilst fixtureManager.restore() restores the database
- Ensure that the DB is always reset between tests
- We assume that the DB will be torn down and rebuilt between tests
- Without this, previous tests clobber the current tests, and it's not possible to tell when running files in isolation
which makes developing really tricky
closes: https://github.com/TryGhost/Toolbox/issues/193
- Our Casper fixture was several years out of date.
- I'd already updated the ghost-api to point to the latest API version, which was the main difference
- This makes sure the full fixture is up to date and correct, and therefore that we're truly testing if Ghost right now works
- It also adds instructions for how to update it in future
GOTCHA: the mock-express-style tests are failing if the API difference between test-theme and casper are different
- I've tried to look into why this is - it's something to do with the overridden route settings not working properly if the API version changes
- Given that we may not keep this style of testing AND we are definitely not keeping API versions, I'm ignoreing this for now
- To get around it, I'm setting both themes to be v4 API, so that Casper is exactly as in main
refs: https://github.com/TryGhost/Toolbox/issues/193
- The Casper fixture in the codebase is copied to a tmp folder structure every single time we start Ghost in an e2e test
- We shouldn't keep files here that don't actually get used!
- The files I'm removing are src files, used for building, but we already have the built files
refs: https://github.com/TryGhost/Toolbox/issues/168
- Upgraded the remaining themes to be pinned to the canary API
- This required one minor fix because the edit URL has changed for authors in v4/canary
- I also ripped out everywhere that the theme ghost-api version was being pinned to canary, as this was unreliable and only happening in a few places
- It's also going to be unnecessary code as soon as we finish changing to only having one API version
refs: https://github.com/TryGhost/Toolbox/issues/168
- The fixture represented Casper at Ghost version 1.0 which pre-dated the introduction of the posts_per_page config in package.json
- When Casper was upgraded to 2.0 in the fixtures, the lack of pagination broke the e2e tests for pagination
- This change introduces proper code to stub and override posts_per_page rather than keeping and using the old casper-1.4 fixture
- It also required me to remove a handful of CSS-based checks which are no longer true in the new theme version, but also didn't really add anything to the tests
refs: https://github.com/TryGhost/Toolbox/issues/168
- Having large theme fixtures makes tests run slower, so we're working to reduce them
- This fixture was a full copy of a very old version of Casper, but all we needed was a handful of files so that the tests can run
- This theme is also used for testing members, not just price data so I've renamed it for clarity
- The remaining files is the bare minimum we need to test these features
refs: https://github.com/TryGhost/Toolbox/issues/168
- This file was pinning various tests to v3
We're going to be dropping the idea of having multiple versions of the API in each Ghost version.
Because this has not achieved the goal of making it easier to make breaking changes, but it has
created an ordinate amount of technical debt and maintenance overhead.
As we know this is going away in the next major, there is no benefit to us constantly running tests
that check if those versions still work, especially given how long they take.
Instead we're starting work to ensure that all of our test work on canary, and that canary has
excellent test coverage so that we can be sure that our one API version works really well and that
any changes, no matter how subtle are deliberate, tracked and understood.
refs https://github.com/TryGhost/Team/issues/1252
We need a way to signal whether or not a Tier is active or archived, and
we'll be using the active flag in the same way we do for Offers.
Since we now have 2 products by default for all ghost sites, free and default paid, the usage of default product which so far was using first product needs to be updated to use the first paid product.
- updates default product usage to use first paid tier
- updates tests
refs: https://github.com/TryGhost/Toolbox/issues/150
Instead of loading all themes for each set of tests in the e2e suite, only load the frontend at all for frontend tests, and only load themes for the theme tests.
refs https://github.com/TryGhost/Toolbox/issues/158
- To make the development quicker the copy-paste code was left in the codebase but gave immediate smell. Removed it to stop bothering me every time I look at the related code :)
refs https://github.com/TryGhost/Toolbox/issues/129
- The "Origin" header is require in Admin API requests so it makes sense to bake it into a default request made by most tests. Reduces unnecesary fluf around test request setup and removes a "config" dependency in each tests suite using the "e2e framework"
refs https://github.com/TryGhost/Toolbox/issues/158
- Allows for much smaller amount of code to configure a test to work with chai-jest-snapshots. They now work automatically for all regression tests and could be enabled for other suites by adding the "--require=./test/utils/snapshots.js" parameter in respective test:* package script
- Regenerated snapshot for authentication test as the naming structure
changed a little with the snapshot metadata being taken on a higher
level in the test (uses the suite name instead of a specific describe it
used to be called from)
refs https://github.com/TryGhost/Toolbox/issues/158
- Moving common mocking/stubbing/spying logic into an outside utils module allows test suites to keep agnostic towards which framework powers mocking etc. Should also substitute email service stubbing used in multiple places
- Bonus, got rid of should dependency, which is deprecated
no issue
- The response from the server might not always contain an array with errors (for example when there's a generic 503 error). This changes prevents tests from hanging and adds at least some relevant output
refs https://github.com/TryGhost/Toolbox/issues/129
- Adds content folder creation based on fixtures for e2e tests using the e2e framework
- Mostly a copy-paste for now from existing e2e utils with some simplifications, so we can "complicate things" later only if needed
refs 24505db918
- The code is heavily borrowed from the referenced commit by @ErisDS. It starts a fresh abstraction layer for all e2e tests to use, provides simle interface for request agent initialization and state management (read the e2e-framework top file description for more)
- Main methods the framework exposes are:
- getAgent - to initialize request agent with a fresh Ghost instance
- initFixtures - to initializer a fresh db state
- resetDb - cleans up the db state if there were state manipulations during the test suite run (POST, PUT requests or non-default fixtures)
refs https://github.com/TryGhost/Toolbox/issues/129
- Having a fresh module should allow building new testing utility concepts from the ground up without being tied by the mystery baggage of the legacy localUtils/testUtils
- The outline of the concepts the framework will be handling is in the commont on the top of the e2e-framework file
refs https://github.com/TryGhost/Toolbox/issues/152
- Being able to set up test suites without blocking a port opens a door to new ways to run tests - for example this has been one of the blockers for running mocha tests in parallel
- Additional benefit is lighter statrup, which reducec the test execution time slightly. Doesn't seem like much but these things stack up!
no issue
- we've seen some instances of SQLite saying "database disk image is malformed"
- I think this happens because we copy the file whilst we are still connected to
the old DB
- this commit destroys the connection before copying the clean file to the live
DB file
refs https://github.com/TryGhost/Toolbox/issues/136
- we nuke and reinitialize the DB many times between tests
- this forms a good portion of the time taken and we shouldn't be spending
so much time on just resetting the DB back to a known state
- this commit switches out the knex-migrator reset + init calls for SQLite to
a function which keeps a clean copy of the DB and copies the file back
when we "reset"
- for MySQL, existing functionality is kept
- this massively speeds up tests because it saves ~700ms+ for every reset
- whilst this change seems to work, it's just the start so there's a
lot more refactoring needed. this change is currently gated to CI until
it's deemed safe/sane enough to run on local machines without blowing
up
refs: TryGhost/Toolbox#147
* Replaces all references to isIgnitionError with isGhostError
* Switches use of GhostError to InternalServerError - as GhostError is no longer public
There are places where InternalServerError is not the valid error, and new errors should be added to the @tryghost/errors package to ensure that we can use semantically correct errors in those cases.
https://github.com/TryGhost/Toolbox/issues/140
- Allows to fully start an instance with a custom routes.yaml file without a need to do workarounds - e.g. we used to call an internal Admin API from a regression test suite, which is not a good practice
- Providing "routesFilePath" to startGhost method copies that file to the default settings location and start the instance with that configuration. No need to do API calls or check if the routing service "isFinished"
no issue
- this commit adds a counter for the number of boots we do in tests
- which therefore allows us to calculate the average boot time we
experience
- only useful for debugging test performance
