Commit Graph

296 Commits

Author SHA1 Message Date
Naz
0f4aeaaa80 Added basic framework for webhook e2e tests
refs https://github.com/TryGhost/Toolbox/issues/320

- This is an **MVP** to be able to intercept and match webhook request
snapshots. The concept is similar to the one used in API E2E tests using
same "matchBodySnapshot" and other "match*" methods to test the webhook
**request** data
- Next up here would be:
1. Header matcher
2. Mocking more than one webhook (and doing something nicer with the way
the fixture data is inserted, does this logic belong to the mock-receiver?
2022-06-03 11:57:39 +08:00
Naz
29e5d08210 Refactored Admin API test agent to use async/await
no issue

- Improves readability of what's going on in the code.
2022-06-01 17:48:08 +08:00
Naz
939496487d Removed subscriber-related webhook fixtures
refs https://github.com/TryGhost/Toolbox/issues/308
refs 0f5c22a38d

- There is no concept of "subscriber" in the codebase any longer. Keeping removed webhooks around does not make sense
2022-05-26 14:45:36 +08:00
Matt Hanley
eae0a6a3b9
Added products and Stripe data to exports and imports (#14873)
- The migration path from 4.x on SQLite to 5.0 on MySQL requires an export/import
- Exports don't include the Stripe info required to map members to tiers correctly on import. This change fixes that.

Co-authored-by: Simon Backx <simon@ghost.org>
Co-authored-by: Hannah Wolfe <github.erisds@gmail.com>
2022-05-20 21:13:58 +01:00
Hannah Wolfe
d6d6841186
Added benefits, offers and snippets to exporter
- These are more things that we should write an importer for
2022-05-20 20:30:55 +01:00
Matt Hanley
3471f6cca0
Updated tests 2022-05-20 20:30:55 +01:00
Naz
a18469a3be Added root Ghost server/API test agent
refs https://github.com/TryGhost/Team/issues/1640

- Some tests require making request to the root of the mounted server path like `GET /ghost/.well-known/jwks.json`. These are not stricly APIs like Admin, Content, and Members. They do need a separate agent to distinguish the configuration - for this agetn we can stop loading some parts like "server" in the future to speed things up (didn't work straight out of the box)
2022-05-20 13:17:00 +08:00
Naz
f97b141a3a Added "anyContentVersion" and anyContentVersion matchers to e2e framework
refs https://github.com/TryGhost/Toolbox/issues/335

- Similarly to `anyEtag` pattern for `etag` headers there's a need to match dynamic content-length and content-version headers.
- The `anyContentLength` is the same as `anyStringNumber` matcher, created a separate one to show the intention over a function when reading the code (maybe we could introduce something like floating number range matcher in the future to be more precise in situations where we expect the content-length to float in +-2 length range)
2022-05-19 13:19:05 +08:00
Naz
9e7bea5bb3 Added "anyStringNumber" matcher to e2e framework
refs https://github.com/TryGhost/Toolbox/issues/335

- Existing `anyNumber` matcher does not do a job when the number is stringified, so introduced a new string matcher to match stringified numbers
2022-05-19 13:06:50 +08:00
Naz
1cc3f66d5f Reworked preview controller output
refs https://github.com/TryGhost/Toolbox/issues/332

- With 'page' property being gone from page/post resources we need consistency across internal "preview API" too
- the handling is still hacky as we are exposing the "type" property to the preview pages, which is not exposed through Content API. this a long outstanding issue will have to be handled separately
2022-05-17 17:21:38 +08:00
Hannah Wolfe
8ec8a21b71
Renamed "client" references to "admin"
refs: https://github.com/TryGhost/Toolbox/issues/299

- renamed lots of things that reference Ghost admin as "client"
- these things make even less sense in a post core/client world
2022-05-17 09:05:44 +01:00
Matt Hanley
09804244f5 Added newsletters table to default export
refs https://github.com/TryGhost/Team/issues/1629

- We want to export the default newsletter so that we don't lose the post<>newsletter relationship when exporting
- The newsletter_id is currently ignored in the post importer (https://github.com/TryGhost/Ghost/pull/14720) so imports won't work right now
- This change means that exports created in 5.0+ will work later on with no data loss once the importer supports it
- The counts displayed for the newsletters are a count of email records, not post records - and we don't export/import emails so the count will always be 0 after importing a newsletter
2022-05-16 17:49:25 +01:00
Hannah Wolfe
12aff14dff Removed oauth experiment
refs: https://github.com/TryGhost/Team/issues/1625

- this didn't work the way we wanted to
- removing this will free up the namespaces to start over
2022-05-16 17:37:13 +01:00
Simon Backx
6b3a657f88
Renamed newsletter_id and email_recipient_filter options (#14798)
refs https://github.com/TryGhost/Team/issues/1596

- Renamed `newsletter_id` to `newsletter` option, the `newsletter` option expects a slug instead of an id
- Renamed `email_recipient_filter` to `email_segment` option
- Default `email_segment` to `all`. Ignored if no newsletter is set
- `email_segment` is ignored if no newsletter is set
- When reverting a post to a draft, both `newsletter` and `email_segment` are reset to their default values (null, all)
- Removed legacy mapping from old email_recipient_filter values 'paid' and 'free' (already a migration in place)
- Dropped legacy throwing errors when email_recipient_filter is paid or free in transformEmailRecipientFilter
- Reorganized transformEmailRecipientFilter parameters for the now required newsletter parameter
- Fixed an issue where the newsletter filter wasn't working because it wasn't in permittedoptions
- Fixed an issue where you could send to an archived newsletter
- Added an extra protection when scheduling to an active, and later archiving the newsletter
- Dropped support for `send_email_when_published` in API
- When importing posts we currently don't have a system in place to set the newsletter_id to map the `send_email_when_published` behaviour. Since this was already the case, I won't include a fix in this PR.
- Stripped `email_recipient_filter`/`email_segment` from Content API (https://ghost.slack.com/archives/C02G9E68C/p1652363211841359?thread_ts=1650623650.233229&cid=C02G9E68C)
- Updated `admin-api-schema` to 3.2.0, which includes the new email_segment property
- Contains a temporary fix for https://github.com/TryGhost/Team/issues/1626, where the `.related('newsletter').fetch` call fails when the newsletter relation is already loaded, because of the overridden `formatOnWrite` method.

Since the `email_recipient_filter` is no longer used without a newsletter, the `none` value is no longer used. A migration transforms all those values to `all`. This should be safe, because we only send an email now when newsletter_id is not null (scheduled posts should already have a newsletter_id, even if at the time of scheduling they didn't add the newsletter_id option, because at that time, we defaulted to the default newsletter).

Admin changes to make this work: https://github.com/TryGhost/Admin/pull/2380
2022-05-16 10:18:04 +02:00
Hannah Wolfe
37818a1b71
Removed settings that are no longer in use (#14802)
refs: TryGhost/Team#1625
refs: TryGhost/Team#1558

- none of these settings are actively used anymore

Co-authored-by: Daniel Lockyer <hi@daniellockyer.com>
2022-05-13 21:19:21 +01:00
Hannah Wolfe
b6d9389124
🔥 Removed legacy product + price helpers from themes
refs: https://github.com/TryGhost/Team/issues/1145
refs: https://github.com/TryGhost/Ghost/issues/14446

- remove legacy members theme helpers @products @product @price and @members.products
- all of these have been replaced with new concepts with the introduction of tiers
2022-05-12 18:58:01 +01:00
Matt Hanley
0d379a9264 Added permissions for Authors to read labels
refs https://github.com/TryGhost/Team/issues/1618

- Authors require access to labels as part of the publish + send workflow
2022-05-12 16:07:05 +01:00
Hannah Wolfe
de118b0b04
Renamed lang and session_secret default settings (#14791)
refs: https://github.com/TryGhost/Toolbox/issues/327

- lang / locale has had a lot of churn, but we decided this setting should always be locale
- session_secret is too generic as we have multiples of these
2022-05-12 15:07:05 +01:00
Matt Hanley
1fabd76391 Added newsletter permissions for Editors and Authors
refs https://github.com/TryGhost/Team/issues/1618

- Editors and Authors require read access to newsletters as part of the publish + send workflow
2022-05-12 14:46:55 +01:00
Rishabh Garg
725c2673ea
Added offer read endpoint to content API (#14794)
refs TryGhost/Team#1599

- allows offer details to be fetched via content API directly
2022-05-12 18:16:10 +05:30
Rishabh Garg
3836030950
Allowed tiers include and data for member endpoints (#14790)
refs https://github.com/TryGhost/Team/issues/1145

- allows members endpoint to accept `?include=tiers`
- allows members endpoint to return `tiers` data
2022-05-11 22:26:03 +05:30
Hannah Wolfe
409dc3b534
Added frontend key to ghost_head for portal (#14782)
refs: https://github.com/TryGhost/Team/issues/1599
refs: f3d5d9cf6b

- this commit adds the concept of a frontend data service, intended for passing data to the frontend from the server in a clean way. This is the start of a new & improved pattern, to hopefully reduce coupling
- the newly added internal frontend key is then exposed through this pattern so that the frontend can make use of it
- the first use case is so that portal can use it to talk to the content API instead of having weird endpoints for portal
- this key will also be used by other internal scripts in future, it's public and therefore safe to expose, but it's meant for internal use only and therefore is not exposed in a generic way e.g. as a helper
2022-05-11 17:34:31 +01:00
Hannah Wolfe
196b8ad0da
Removed active_timezone, default_locale & locale compat (#14788)
- These settings no longer exist, having been renamed to timezone and lang
- As of 5.0 we no longer need any kind of backwards compatibility outside of the importer
- We making breaking changes and cleaning up as many old code paths as possible 
- We have not really exposed the admin Settings API, meaning backwards compatibility was more for internal use
- We will be changing lang back to locale, but that's a separate issue and won't need backwards compatibility
2022-05-11 16:22:25 +01:00
Hannah Wolfe
54b4a3c351
Renamed settings index.js to settings-service.js
- in line with our policy of not keeping logic in index.js files
- Note: callign all services service.js is no more helpful :D
2022-05-10 20:55:31 +01:00
Rishabh Garg
4d6b3568c5
Enabled Admin integration for newsletters API (#14714)
refs https://github.com/TryGhost/Team/issues/1546

- allows newsletters API to work with Admin API keys
- updates fixtures to add permissions to admin integration role for new sites
- adds migration to update existing sites to have correct permissions for role
- whitelists add/edit/read/browse on newsletters API for integrations
2022-05-09 20:46:34 +05:30
Naz
7fbf2a62ac Removed support for single "author" shorthand in fixtures
refs https://github.com/TryGhost/Toolbox/issues/230

 - The test fixtures should be using `authors: [{id:...}]` syntax instead of relyin on `author_id` or `author` - these are deprecated concepts that should go away from the codebase
2022-05-09 20:43:19 +08:00
Naz
b71c1895fb Removed use of author_id syntax in fixtures
refs https://github.com/TryGhost/Toolbox/issues/230

 - The `author_id` would be ignored as a parameter in the API or Post model, so would produce a falsy results if specific fixtures were used to compare test results
2022-05-09 20:43:19 +08:00
Naz
e38c8a4662 Fixed model posts test suite
refs https://github.com/TryGhost/Toolbox/issues/230

- Code tweaks to make test suite for post modela and utils play nicely without single post author concept
2022-05-09 20:34:06 +08:00
Naz
a372a84e82 Fixed test query using outdated author_id filter
refs https://github.com/TryGhost/Toolbox/issues/230

- The `author_id` column is gone, so is the support for filtering posts by single author's id. Using author's slug(s) is the closest alternative to achieve the same result
2022-05-09 20:34:06 +08:00
Naz
a3005dc328 Fixed authors initializaion on SQLite
refs https://github.com/TryGhost/Toolbox/issues/230

 - The plain ["id"] syntax doesn't work for SQLite - has to be an explicit property to catch up.
2022-05-09 20:34:06 +08:00
Naz
9821c3785e Fixed multiple author fixture initializaion
refs https://github.com/TryGhost/Toolbox/issues/230

 - The fixture manager has to initialize User/Roles fixtures first to be able to insert multiple authors as a relation in post fixtures. Otherwise the posts could not find correct authors and were failing trying to assign default "owner user" to each post
- The order of running fixtures matters, and till now the order wasn't taken into account at all when populating the db
2022-05-09 20:34:06 +08:00
Naz
e88b4b5c1b Removed single author concept form data generator
refs https://github.com/TryGhost/Toolbox/issues/230

- Data generator for test's state should not be inserting author_id columns now that it's not present in schema
2022-05-09 20:34:06 +08:00
Naz
2f98ad68ba Removed author_id useage form fixtures
refs https://github.com/TryGhost/Toolbox/issues/230

 - The column has been dropped form shema, so should not be used in fixtures either
2022-05-09 20:34:06 +08:00
Hannah Wolfe
7618f925e1
Cleaned newsletter e2e tests, improved framework
- attempting to make our framework a one-stop-shop for all requires (except assert)
- using utils for tests instead of lots of requires makes it easier to reason about how our tests interface with our code
- helps with refactoring later, and making sure that tests really do what we expect
2022-05-06 12:15:12 +01:00
renovate[bot]
9aa8e3e7ff
Update dependency @tryghost/url-utils to v3 (#14705)
- updated usage of url-utils.urlFor to work with v3

Co-authored-by: Renovate Bot <bot@renovateapp.com>
Co-authored-by: Hannah Wolfe <github.erisds@gmail.com>
2022-05-05 16:10:18 +01:00
Naz
734ef66e6c Fixed deadlock when adding multiple authors in tests
refs 275107d423

- Because there might be multiple authors being added at the same time with different values to the posts_authors table these operations should not be done in parallel! Making post insertion sequential fixed the deadlock
2022-05-03 16:28:07 +08:00
Naz
6fa77cc0e6 Fixed fatal errors in theme tests
refs 03c8e7f010
refs https://github.com/TryGhost/Toolbox/issues/226

- With a bump of gscan the default "Casper" theme in test fixtures should have been updated, otherwise it's leaving "fatal error" logs all over the place
2022-05-03 12:24:39 +08:00
Naz
03c8e7f010 Switched gscan to validate against v5 set of rules
refs https://github.com/TryGhost/Toolbox/issues/226

- We are on the finish line to release 5.0 - the instance should be validating it's themes agains v5 set of rules.
- The prominent deprecations are `{{author}}` and `@blog` helpers removals - will throw 'fatal' errors when encountered.
2022-05-02 20:56:25 +08:00
Daniel Lockyer
0f5c22a38d
Cleaned up mention of subscribers in test folder
refs https://github.com/TryGhost/Toolbox/issues/308

- we had a few mentions of `subscribers` in the test suite data generator
  but this shouldn't be used any more because it's an ancienttttt concept
- removing this for v5 as it helps to clean the codebase
2022-04-28 15:37:12 +01:00
Hannah Wolfe
4332546a56
Removed theme engines API versioning concept
refs: https://github.com/TryGhost/Toolbox/issues/228

- we are getting rid of the concept of api versions from Ghost
- this means getting rid of them from the frontend as well, and from themes
2022-04-28 15:35:27 +01:00
Hannah Wolfe
a703185497
Fixed mockLabs disabling all other flags (#14621)
refs TryGhost/Team#1566

- Mocking a labs flag (regardless of enabled/disabled) currently has a side effect of setting any other flag to undefined.
- This meant in a test where we set a flag e.g. members-importer where we set multipleProducts, multipleNewsletters is always disabled
- This fix preserves the default state of all labs flags that are not mocked so that labs behaves how we expect
- Removed usage of GA flags in tests
- Removed tests that had GA flags disabled

Co-authored-by: Simon Backx <simon@ghost.org>
2022-04-28 10:55:20 +02:00
Fabien 'egg' O'Carroll
d94859f2e5
Added /stats/subscriptions API (#14547)
refs https://github.com/TryGhost/Team/issues/1505
refs https://github.com/TryGhost/Team/issues/1466

Exposes an API for historical counts broken down by tier and cadence.

Counts backwards from the current stats like MRR to minimize inaccruate
data due to missing/superfluous events.
2022-04-27 14:53:32 +01:00
Rishabh Garg
756f86dbdc
Added uuid to newsletter schema (#14600)
refs https://github.com/TryGhost/Team/issues/1561

With multiple newsletters, unsubscribe links will also need to have a unique reference to the newsletter that the email is for, so that we can unsubscribe members from the particular newsletter automatically when they click on the link.
As our existing pattern for members is to use UUID as the external unique reference, this change adds UUID to newsletter schema and populates the existing newsletters with a UUID value.

- adds new `uuid` column to newsletter schema
- updates newsletter model to add default uuid
- updates default newsletter migration to add `uuid`
- drops nullable on `uuid` column later in migrations once we have populated existing newsletters
2022-04-27 19:20:25 +05:30
Hannah Wolfe
37a684c2c5 Added explicit enum to newsletters.status
closes: https://github.com/TryGhost/Team/issues/1553

- we want to be explicit in what values are supported
- we want the values that are supported to be supported to also be explicit and clear
- without this API users can set the value to anything and the active/not active logic will work, until such time as we introduce further statuses
- this means introducing a new status could be a potential breaking change and lands us in horrible hot water
2022-04-26 15:51:02 +01:00
Hannah Wolfe
6e92229a07 Remapped member.subscribed value using newsletter subscriptions (#14563)
refs https://github.com/TryGhost/Team/issues/1545

- Remapped `member.subscribed` value based on newsletter subscriptions in API output
- Enabled filtering by subscribed status for multiple newsletters

Co-authored-by: Matt Hanley <git@matthanley.co.uk>
2022-04-26 12:31:34 +01:00
Hannah Wolfe
2d36e5881b Simplified newsletter e2e tests
- Many of these tests were using API calls to get IDs or check side effects
- This makes snap files much harder to read, where keeping tests more minimal gives us the same test coverage
- Also updated the tests to include members, so we have some real live counts
2022-04-26 12:31:34 +01:00
Hannah Wolfe
9da57fa6bb Updated test fixtures for multiple newsletters
- Updated the member fixtures to have 6 subscribed and 2 unsubscribed members
- Added an inactive newsletter
- Updated newsletter-member relations to reflect the 6 subscribed and 2 unsubscribed, but with different cases:
   - 3 subscribed to default
   - 1 subscribed to secondary only
   - 1 subscribed to default + secondary
   - 1 subscribed to secondary + inactive
   - 1 subscribed to inactive only (i.e. not subscribed)
   - 1 not subscribed at all

- With these changes, I needed to update the members snap as 2 members appear as subscribed:false in many tests
- I also needed to update some posts regression tests, as 2 less members get emailed in 2 tests
2022-04-26 12:31:34 +01:00
Naz
1ecb837981 Added version_notifications key to settings table
refs https://github.com/TryGhost/Toolbox/issues/292

- When version missmatch handling is done in Ghost we need to store the 'Accept-Version' header values that have been already processed in the past (to avoid sending notifications about the same mismatch multiple times)
- The `version_notifications` will be storing an array with handled versions like so: `['v3.44', 'v4.23', 'v4.39']`.
- The emailing logic and processing is slightly similar to how "notification" key is handled, that's why I've placed the definition of this new key close by.
2022-04-21 20:34:19 +08:00
Thibaut Patel
ae664e9cad
Added newsletter read permission (#14519)
refs https://github.com/TryGhost/Team/issues/1533

- Needed to create the read newsletter endpoint to make the newsletter resource more consistent with the other resources
- Read is available to admins like other newsletter actions
2022-04-20 15:25:41 +02:00
Simon Backx
a3215a6edf
Updated calculation of MRR history to use mrr column (#14481)
refs https://github.com/TryGhost/Team/issues/1518

- Uses mrr column now
- Doesn't account for `cancel_at_period_end` until a migration fixes the MRR values: https://github.com/TryGhost/Ghost/pull/14480
2022-04-19 17:07:20 +02:00