Commit Graph

1261 Commits

Author SHA1 Message Date
Simon Backx
efdc42c257
Fixed bulk unsubscribe and updated member import tests (#14610)
refs https://github.com/TryGhost/Team/issues/1567

**Changes in members-api**
- Compare changes: https://github.com/TryGhost/Members/compare/%40tryghost/members-api%406.1.0...%40tryghost/members-api%406.2.2
- Fixed bulk unsubscribe
- Deletes the newsletter relations instead of setting subscribed to false

**Test fail fix**
refs https://github.com/TryGhost/Ghost/pull/14621
refs https://ghost.slack.com/archives/C02G9E68C/p1651126990299689?thread_ts=1651072733.859939&cid=C02G9E68C

- Events didn't always have the same created_at as created members
- This caused a test to fail randomly in the main repo

**Changes**
- Added required helpers for members-api package
- Version bumps of other packages are only tooling related

**Tests**
- Tests if member import still works with the legacy `subscribed` flag
- Updated member importer to use multipleNewsletters flag
- Dropped legacy members tests
2022-04-28 09:50:05 +02:00
Daniel Lockyer
7583d379be
Revert "Dropped nullable from subscribe events' newsletter id (#14598)"
This reverts commit dc12e6490a.
2022-04-27 20:01:32 +01:00
Matt Hanley
b2e36b2f6f
Promoted multiple newsletters flag (backend) to GA (#14609)
refs https://github.com/TryGhost/Team/issues/1566
2022-04-27 19:02:56 +01:00
Thibaut Patel
f5bd647100
Updated emails to use newsletter settings (#14588)
refs https://github.com/TryGhost/Team/issues/1550

- Updated email template and seder options to use the settings specified for the related newsletter
- Falls back to the default newsletter, and uses the default newsletter settings for the publishing preview because we only assign a newsletter at the point a post is published

Co-authored-by: Thibaut Patel <thibaut.patel@gmail.com>
Co-authored-by: Matt Hanley <git@matthanley.co.uk>
2022-04-27 18:48:36 +01:00
Fabien 'egg' O'Carroll
810c3077e8
Wired up LimitService to NewsletterService (#14602)
refs https://github.com/TryGhost/Team/issues/1549

This allows us to restrict certain sites to a single newsletter
2022-04-27 17:44:16 +01:00
Simon Backx
dc12e6490a
Dropped nullable from subscribe events' newsletter id (#14598)
refs https://github.com/TryGhost/Team/issues/1478

- Depends on https://github.com/TryGhost/Ghost/pull/14597
- Column shouldn't be nullable now it is filled correctly
2022-04-27 17:52:31 +02:00
Simon Backx
21af34a0d4
Added mapping from member subscribed to newsletters on edit/create (#14596)
refs https://github.com/TryGhost/Team/issues/1545

**Changes (`members-api`)**
- Compare via https://github.com/TryGhost/Members/compare/%40tryghost/members-api%406.0.0...%40tryghost/members-api%406.1.0
- Added mapping from member subscribed to newsletters on edit/create
- When editing or creating a member with the subscribed property, it is mapped to the corresponding newletters value
- Defaults to all active newsletters with visibility = members and subscribe_on_signup = true

**Tests**
- Adds test that adds a member with subscribed = true
- Adds test that adds a member with subscribed = false
- Adds test that edits a member with subscribed = true
- Adds test that edits a member with subscribed = false
2022-04-27 17:04:55 +02:00
Simon Backx
2cf76cb031
Added newsletter relation to subscribe events (#14585)
refs https://github.com/TryGhost/Team/issues/1478

**Changes**
- Added the newsletter relation to subscribe events

**Changes in `members-api`**
- Compare: https://github.com/TryGhost/Members/compare/%40tryghost/members-api%406.0.0-alpha.0...%40tryghost/members-api%406.0.0
- Makes sure the newsletter relation is returned in the activity feed for susbcribe events (aka newsletter events).

**Tests**
- Added first test for activity feed to check if the newsletter relation is correctly fetched
2022-04-27 16:44:27 +02:00
Fabien 'egg' O'Carroll
d94859f2e5
Added /stats/subscriptions API (#14547)
refs https://github.com/TryGhost/Team/issues/1505
refs https://github.com/TryGhost/Team/issues/1466

Exposes an API for historical counts broken down by tier and cadence.

Counts backwards from the current stats like MRR to minimize inaccruate
data due to missing/superfluous events.
2022-04-27 14:53:32 +01:00
Rishabh Garg
756f86dbdc
Added uuid to newsletter schema (#14600)
refs https://github.com/TryGhost/Team/issues/1561

With multiple newsletters, unsubscribe links will also need to have a unique reference to the newsletter that the email is for, so that we can unsubscribe members from the particular newsletter automatically when they click on the link.
As our existing pattern for members is to use UUID as the external unique reference, this change adds UUID to newsletter schema and populates the existing newsletters with a UUID value.

- adds new `uuid` column to newsletter schema
- updates newsletter model to add default uuid
- updates default newsletter migration to add `uuid`
- drops nullable on `uuid` column later in migrations once we have populated existing newsletters
2022-04-27 19:20:25 +05:30
Hannah Wolfe
37a684c2c5 Added explicit enum to newsletters.status
closes: https://github.com/TryGhost/Team/issues/1553

- we want to be explicit in what values are supported
- we want the values that are supported to be supported to also be explicit and clear
- without this API users can set the value to anything and the active/not active logic will work, until such time as we introduce further statuses
- this means introducing a new status could be a potential breaking change and lands us in horrible hot water
2022-04-26 15:51:02 +01:00
Rishabh Garg
4f3743bb60
Added cache invalidation on updating newsletters (#14587)
- allows portal/apis to not show stale newsletter information
2022-04-26 18:06:41 +05:30
Hannah Wolfe
327c209a74 🐛 Added default sort order for members.browse
refs: https://github.com/TryGhost/Team/issues/1557

- There is currently no default order for the members API
- This is done at the API level purely for the endpoint, not in the model using orderDefaultOptions
- This is because orderDefaultOptions affects findPage and findPage is wrapped by membersAPI.members.list
- That in turn is used in multiple places, e.g. getting member counts for emails and getting members for exports
- There is currently no created_at DESC index on the table, so we don't to impact performance too much
- This ensures it's only affected when paginating
2022-04-26 13:02:27 +01:00
Simon Backx
32b0f69e55 Added newsletters to subscribe events (#14579)
refs https://github.com/TryGhost/Team/issues/1478

- Moved all admin API members tests to enable the multiple newsletters flag
- Checks if the susbcribe events are added correctly when adding or removing newsletters
- Checks if susbcribe events are added for default newsletters
2022-04-26 12:31:34 +01:00
Thibaut Patel
7422f3e64d Moved to using newsletter design settings in email serializer (#14562)
refs https://github.com/TryGhost/Team/issues/1550

- Switched to using the newsletter design settings over the global settings
- Made the `newsletter_id` property available in the Admin API Post resource
- Added the `showHeaderName` variable that can be used in the post html template
2022-04-26 12:31:34 +01:00
Matt Hanley
5f31f52139 Allow opt-in for existing members when creating a newsletter (#14567)
- Added changes to opt-in existing members for new newsletters

Co-authored-by: Hannah Wolfe <github.erisds@gmail.com>
2022-04-26 12:31:34 +01:00
Hannah Wolfe
6e92229a07 Remapped member.subscribed value using newsletter subscriptions (#14563)
refs https://github.com/TryGhost/Team/issues/1545

- Remapped `member.subscribed` value based on newsletter subscriptions in API output
- Enabled filtering by subscribed status for multiple newsletters

Co-authored-by: Matt Hanley <git@matthanley.co.uk>
2022-04-26 12:31:34 +01:00
Matt Hanley
8662252ae1 Updated default newsletter model sort order (#14571)
refs https://github.com/TryGhost/Team/issues/1552

- The API doesn't enforce a unique sort order but we infer the "default newsletter" based on ordering so we need to ensure a consistent and deterministic ordering behaviour
2022-04-26 12:31:34 +01:00
Rishabh Garg
05b5059918 Updated portal to manage single newsletter with existing UI (#14573)
refs https://github.com/TryGhost/Team/issues/1554

- bumps portal to manage single site newsletter with old subscribe UI
- adds unit test for portal site endpoint
2022-04-26 12:31:34 +01:00
Hannah Wolfe
2d36e5881b Simplified newsletter e2e tests
- Many of these tests were using API calls to get IDs or check side effects
- This makes snap files much harder to read, where keeping tests more minimal gives us the same test coverage
- Also updated the tests to include members, so we have some real live counts
2022-04-26 12:31:34 +01:00
Thibaut Patel
ed29c7addf Filtered member email recipients based on the newsletter subscriptions (#14489)
refs https://github.com/TryGhost/Team/issues/1524

- We need to fetch the post newsletter to grab the slug as it's needed for the member NQL filter.
- We can then use the newsletter slug and append it in the existing member NQL filter.
- Removed `subscribed:true` when an email is sent to a newsletter and replaced it with the newsletter id
- Added `status:-free` when an email is sent to a newsletter with `visibility` set to `paid`
- Added tests what happens when you publish without newsletter_id
- Added tests what happens when you publish with newsletter_id

Co-authored-by: Simon Backx <simon@ghost.org>
2022-04-26 12:31:34 +01:00
Hannah Wolfe
9da57fa6bb Updated test fixtures for multiple newsletters
- Updated the member fixtures to have 6 subscribed and 2 unsubscribed members
- Added an inactive newsletter
- Updated newsletter-member relations to reflect the 6 subscribed and 2 unsubscribed, but with different cases:
   - 3 subscribed to default
   - 1 subscribed to secondary only
   - 1 subscribed to default + secondary
   - 1 subscribed to secondary + inactive
   - 1 subscribed to inactive only (i.e. not subscribed)
   - 1 not subscribed at all

- With these changes, I needed to update the members snap as 2 members appear as subscribed:false in many tests
- I also needed to update some posts regression tests, as 2 less members get emailed in 2 tests
2022-04-26 12:31:34 +01:00
Thibaut Patel
48799e1c68 Added the newsletter API ?include=count.posts and ?include=count.members query options (#14525)
refs https://github.com/TryGhost/Team/issues/1524

- This enables admins in the ghost admin to have an overview of the total posts/members associated with a newsletter.
- Follows the `?include=count.x` convention used by other resources
2022-04-26 12:31:34 +01:00
Rishabh Garg
a330165e4f Handled email unsubscribe with multiple newsletters (#14560)
refs https://github.com/TryGhost/Team/issues/1495

- removes subscription from all newsletters for a member on click of unsubscribe link in email
- allows the new multiple newsletter system to work with existing unsubscribe flow
2022-04-26 12:31:34 +01:00
Simon Backx
50ab1e0d73 Added created_at/updated_at to newsletters (#14559)
no issue

Added the missing timestamps to the newsletters table
2022-04-26 12:31:34 +01:00
Thibaut Patel
c49d2d0b7e Fixed the newsletter_id assignement in the post API (#14485)
refs https://github.com/TryGhost/Team/issues/1502

- Assigned posts to the default newsletter when sent as email
2022-04-26 12:31:34 +01:00
Simon Backx
c15cb1b5be Added newsletter_id column to subscribe events (#14553)
refs https://github.com/TryGhost/Team/issues/1478

The `newsletter_id` is nullable for now to remain compatible until we have a proper data migration + updated code to set it on inserts
2022-04-26 12:31:34 +01:00
Kevin Ansfield
a8687b35b9 Added newsletter from address verification (#14491)
refs https://github.com/TryGhost/Team/issues/1498
refs https://github.com/TryGhost/Team/issues/584

- Added newsletter `from` address verification

Co-authored-by: Hannah Wolfe <github.erisds@gmail.com>
2022-04-26 12:31:34 +01:00
Matt Hanley
298599ce91 Added show_header_name column to newsletters table
refs https://github.com/TryGhost/Team/issues/1532

- Added before the migration in https://github.com/TryGhost/Ghost/pull/14468 to populate the default newsletter
- The fixture for the default newsletter has a different value than the model and schema default
- This is because by default the newsletter name is the same as the site title, and the site title is already shown
2022-04-26 12:31:34 +01:00
Naz
2a592a9bf3 Added test coverage to settings bread service
no issue

- This serivce was missing any unit test coverage. Covering few cases gives a good groundwork to expand on
2022-04-22 16:31:56 +08:00
Naz
cc71bbfd61 Hooked up api version compatibility middleware
refs https://github.com/TryGhost/Toolbox/issues/280

- ctd of putting pieces together to allow Ghost notifying owner and admin users about version mismatch errors
-  The `@tryghost/mw-api-version-mismatch` in a combination with api version compatibility service make the whole notification process play nicely :)
- The flow of the logic from the request to a sent notification email is following:
1. Request comes is with an Accept-Version header that's behind current Ghost version and is not supported
2. mw-error-handler middleware's 'resourceNotFound' detects such request and returns a 406 with a special 'code' identifying if the version of the client is ahead or behind
3. mw-api-version-mismatch intercepts the 406 request with "code === 'UPDATE_CLIENT'` and calls up APIVersionCompatibilityService
4. emails are sent out to active owner and admin users

- The above flow is also illustratd in the e2e tests that come with the changeset
2022-04-22 16:31:56 +08:00
Rishabh
2ab78bcdc9 Updated settings test snapshot for new flag 2022-04-22 12:50:33 +05:30
Fabien 'egg' O'Carroll
19a08cd11d
Replaced stats service with @tryghost/stats-service (#14527)
- Pulls logic out of Ghost core and into the Analytics repository.
- Test coverage has increased for the stats service
- Interface simplified
2022-04-21 14:57:07 +01:00
Naz
1ecb837981 Added version_notifications key to settings table
refs https://github.com/TryGhost/Toolbox/issues/292

- When version missmatch handling is done in Ghost we need to store the 'Accept-Version' header values that have been already processed in the past (to avoid sending notifications about the same mismatch multiple times)
- The `version_notifications` will be storing an array with handled versions like so: `['v3.44', 'v4.23', 'v4.39']`.
- The emailing logic and processing is slightly similar to how "notification" key is handled, that's why I've placed the definition of this new key close by.
2022-04-21 20:34:19 +08:00
Simon Backx
b9646cceb1
Updated members API to return subscriptions' offers from the offer_id (#14515)
refs https://github.com/TryGhost/Team/issues/1520

**Changes in members repo**

Bumped to `5.12.0`, with the following changes:

- Compare differences via https://github.com/TryGhost/Members/compare/%40tryghost/members-api%405.11.1...%40tryghost/members-api%405.12.0
- Instead of doing the matching of the offers and subscriptions by looking at the offer redemptions, we can now look at the offer_id from subscriptions.
- This also fixes an issue where we don't attach the offer object to subscriptions in the members' browse method
- Updated browse behaviour to match the read behaviour of members (product relation needs to get loaded because it is missing in member.products if the subscription is expired).

**Tests**
- Includes test to see if the API correctly returns the offer object when fetching one or multiple members
- Check if the return format is the same for the read, edit and browse members admin API endpoints (offer was missing in subscriptions)
- Snapshot files have been updated because now the offer is returned in subscriptions (content length increased)
2022-04-21 12:06:06 +02:00
Thibaut Patel
52a855f314
Added the newsletter read endpoint (#14520)
refs https://github.com/TryGhost/Team/issues/1533

- Retrieves one newsletter
- Makes the newsletter resource consistent with the other resources
- Solves an issue with the admin expecting the route to exist
2022-04-21 09:59:44 +02:00
Naz
f28e65b4e7 Added staus code assertions to version test suite
refs https://github.com/TryGhost/Toolbox/issues/280

- All e2e tests should be checking returned status code! This was missed when writing tests
2022-04-21 12:45:21 +08:00
Kevin Ansfield
fbaf49b882 Updated settings API snapshot for new labs flag
refs 05da593516

- test was failing because the `labs` property returned by the settings endpoint changed after the new labs flag was added
2022-04-20 16:29:51 +01:00
Thibaut Patel
ae664e9cad
Added newsletter read permission (#14519)
refs https://github.com/TryGhost/Team/issues/1533

- Needed to create the read newsletter endpoint to make the newsletter resource more consistent with the other resources
- Read is available to admins like other newsletter actions
2022-04-20 15:25:41 +02:00
Simon Backx
a3215a6edf
Updated calculation of MRR history to use mrr column (#14481)
refs https://github.com/TryGhost/Team/issues/1518

- Uses mrr column now
- Doesn't account for `cancel_at_period_end` until a migration fixes the MRR values: https://github.com/TryGhost/Ghost/pull/14480
2022-04-19 17:07:20 +02:00
Simon Backx
0c72e78e6a
Stored offer_id in subscriptions (#14488)
refs https://github.com/TryGhost/Team/issues/1519

**Tests:**
- Tests whether the metadata from an offer is read correctly and stored in the database
- Test that invalid offer ids are ignored

**Changes in members repository:** 
- Compare changes here: https://github.com/TryGhost/Members/compare/%40tryghost/members-api%405.9.2...%40tryghost/members-api%405.11.1
- The `offer_id` column of subscriptions is set based on the coupon id from Stripe
- `getByStripeCouponId` method added in the offers repository (required to look up an offer from a stripe_coupon_id)
- the `members-payments` package was bumped twice (once for changes, once for undoing those changes, my bad). Nothing else has changed in that package.
2022-04-19 17:06:53 +02:00
Hannah Wolfe
1d121c52f4
Replaced white/black list terminology
refs 92986b77e3

- I thought we did this a while ago, but uses in comments and elsewhere in the codebase were missed
2022-04-19 11:19:59 +01:00
Rishabh Garg
88ed60d079
Updated default newsletter subscription logic for members (#14482)
refs https://github.com/TryGhost/Team/issues/1469

Previously, members were subscribed to all available newsletters by default when added. This change updates the default newsletters subscription for member to take into account newsletter preferences for auto opt-in(`subscribe_on_signup`) as well as `visibility`.
2022-04-18 13:48:44 +05:30
Naz
d9ac05f97a Fixed snapshot tests using version headers
refs 3381dae1e6
refs 4a4b0cc8a6
refs f432ee9aa6

- The initial tests didn't take into account the veersion of the Ghost instance would be changing contantly. The "context" part of the returned version missmatch errors should be a regex with only partially hardcoded message.
2022-04-18 10:35:29 +08:00
Daniel Lockyer
f432ee9aa6
Fixed linting re: skipped tests
refs 4a4b0cc8a6

- we're temporarily skipping the tests until they're fixed but then
  linting started failing........
2022-04-15 16:24:07 +01:00
Daniel Lockyer
4a4b0cc8a6
Skipped tests containing faulty snapshot
refs 3381dae1e6

- these tests contain snapshots that hardcode the Ghost version into them
- this is incorrect but we need to unblock `main` so I'm commenting them
  out until someone can look at them
2022-04-15 16:18:26 +01:00
Simon Backx
9b23424e48
Added offer_id column to members_stripe_customers_subscriptions (#14487)
refs https://github.com/TryGhost/Team/issues/1519

Migration that adds the (nullable) offer_id column to members_stripe_customers_subscriptions.
- New reliable way to know which offer is active for a given subscription (currently we compare the tier and cadence in offer redemptions)
- We'll create a separate migration to backfill all the offer_ids, but only after we updated the code to also store them correctly for new or updated subscriptions (https://github.com/TryGhost/Team/issues/1520)
- Allows us to backfill the MRR of all subscriptions to account for forever offers
2022-04-15 13:47:15 +02:00
Simon Backx
a696d99f20
Added tests to check MRR and MRR_delta for subscriptions with offers (#14470)
refs https://github.com/TryGhost/Team/issues/1451
refs https://github.com/TryGhost/Team/issues/1456

Tests for updating `members-api` package to `5.9.0` (happened in earlier commit), which includes the following changes since `5.8.0`:
* Simplifies the calculation of MRR deltas, which will make it easier to update MRR to include offers and cancellations in the future.
* Adjusted MRR and MRR delta calculation to consider "forever" duration offers (only if dashboardv5 flag is enabled)
* Uses the discount information from Stripe to calculate the MRR (this was the easiest way to include it + also supports manually created discounts from users)
* Full difference in https://github.com/TryGhost/Members/pull/387

New tests:
- Checks calculation of MRR when using forever vs repeating discounts
- Checks calculation of MRR with dashboard v5 flag enabled/disabled
- Checks calculation of MRR for yearly and monthly subscriptions with forever offers
- Checks updates of MRR and MRR_delta when adding a forever discount to an existing subscription
- Checks updates of MRR and MRR_delta when canceling a subscription with a discount
2022-04-15 11:16:50 +02:00
Rishabh Garg
1362750640
Updated newsletter fixtures to remove nullable sender name (#14479)
refs https://github.com/TryGhost/Team/issues/1513

Since `sender_name` for newsletter is now nullable field, this change removes hardcoded `sender_name` from fixtures.
2022-04-14 23:45:59 +05:30
Rishabh Garg
5218cf194d
Updated newsletter sender name column as nullable (#14476)
refs TryGhost/Team#1513

- nullable `sender_name` allows us to use auto fallback of site title for sender name without setting any explicit value for it.
2022-04-14 17:42:20 +05:30