Commit Graph

16177 Commits

Author SHA1 Message Date
Hannah Wolfe
2d36e5881b Simplified newsletter e2e tests
- Many of these tests were using API calls to get IDs or check side effects
- This makes snap files much harder to read, where keeping tests more minimal gives us the same test coverage
- Also updated the tests to include members, so we have some real live counts
2022-04-26 12:31:34 +01:00
Thibaut Patel
ed29c7addf Filtered member email recipients based on the newsletter subscriptions (#14489)
refs https://github.com/TryGhost/Team/issues/1524

- We need to fetch the post newsletter to grab the slug as it's needed for the member NQL filter.
- We can then use the newsletter slug and append it in the existing member NQL filter.
- Removed `subscribed:true` when an email is sent to a newsletter and replaced it with the newsletter id
- Added `status:-free` when an email is sent to a newsletter with `visibility` set to `paid`
- Added tests what happens when you publish without newsletter_id
- Added tests what happens when you publish with newsletter_id

Co-authored-by: Simon Backx <simon@ghost.org>
2022-04-26 12:31:34 +01:00
Hannah Wolfe
9da57fa6bb Updated test fixtures for multiple newsletters
- Updated the member fixtures to have 6 subscribed and 2 unsubscribed members
- Added an inactive newsletter
- Updated newsletter-member relations to reflect the 6 subscribed and 2 unsubscribed, but with different cases:
   - 3 subscribed to default
   - 1 subscribed to secondary only
   - 1 subscribed to default + secondary
   - 1 subscribed to secondary + inactive
   - 1 subscribed to inactive only (i.e. not subscribed)
   - 1 not subscribed at all

- With these changes, I needed to update the members snap as 2 members appear as subscribed:false in many tests
- I also needed to update some posts regression tests, as 2 less members get emailed in 2 tests
2022-04-26 12:31:34 +01:00
Thibaut Patel
48799e1c68 Added the newsletter API ?include=count.posts and ?include=count.members query options (#14525)
refs https://github.com/TryGhost/Team/issues/1524

- This enables admins in the ghost admin to have an overview of the total posts/members associated with a newsletter.
- Follows the `?include=count.x` convention used by other resources
2022-04-26 12:31:34 +01:00
Matt Hanley
d4700e048a Mapped subscribers to newsletter (#14518)
Closes https://github.com/TryGhost/Team/issues/1508
Refs https://github.com/TryGhost/Ghost/pull/14468

- Maps existing subscribers to the default newsletter

A note on performance:
We loop over the rows in a potentially large table (members) but I've minimised the impact by limiting the columns we fetch. The alternative is a raw SQL query like the one below: the SQL version takes ~0.9s vs ~1.1s for the migration (my laptop, ~30k members). The disadvantage of the raw SQL implementation is the approximation of the ObjectID (instead of a legit bson ID) which isn't sequential and may impact index size/performance.

```sql
insert into members_newsletters (id, member_id, newsletter_id)
    select
        substr(replace(uuid(),'-',''),1,24),
        id,
        '62595dcbfad4e031d85d166f'
    from members
    where members.subscribed=true;
```
2022-04-26 12:31:34 +01:00
Rishabh Garg
a330165e4f Handled email unsubscribe with multiple newsletters (#14560)
refs https://github.com/TryGhost/Team/issues/1495

- removes subscription from all newsletters for a member on click of unsubscribe link in email
- allows the new multiple newsletter system to work with existing unsubscribe flow
2022-04-26 12:31:34 +01:00
Matt Hanley
2d27c9d42a Added a down() operation to newsletters
- Without the down operation, the previous migration fails to re-add the
created_at column after a rollback
2022-04-26 12:31:34 +01:00
Simon Backx
69f31074e0 Updated default newsletter migration for created_at
refs https://ghost.slack.com/archives/C02G9E68C/p1650632604790259?thread_ts=1650619639.619819&cid=C02G9E68C

The new column needs to be included in the migration explicitly, since there is no default in the database (and we don't use the model).
2022-04-26 12:31:34 +01:00
Simon Backx
50ab1e0d73 Added created_at/updated_at to newsletters (#14559)
no issue

Added the missing timestamps to the newsletters table
2022-04-26 12:31:34 +01:00
Thibaut Patel
c49d2d0b7e Fixed the newsletter_id assignement in the post API (#14485)
refs https://github.com/TryGhost/Team/issues/1502

- Assigned posts to the default newsletter when sent as email
2022-04-26 12:31:34 +01:00
Simon Backx
c15cb1b5be Added newsletter_id column to subscribe events (#14553)
refs https://github.com/TryGhost/Team/issues/1478

The `newsletter_id` is nullable for now to remain compatible until we have a proper data migration + updated code to set it on inserts
2022-04-26 12:31:34 +01:00
Thibaut Patel
c8230060a9 Added the default newsletter (#14468)
refs https://github.com/TryGhost/Team/issues/1473

- Added the default newsletter
- We use the title to populate the newsletter same, slug and sender name
- We use the description to populate the newsletter description
- We use the global design settings to populate the corresponding newsletter design settings

Co-authored-by: Matt Hanley <git@matthanley.co.uk>
2022-04-26 12:31:34 +01:00
Kevin Ansfield
a8687b35b9 Added newsletter from address verification (#14491)
refs https://github.com/TryGhost/Team/issues/1498
refs https://github.com/TryGhost/Team/issues/584

- Added newsletter `from` address verification

Co-authored-by: Hannah Wolfe <github.erisds@gmail.com>
2022-04-26 12:31:34 +01:00
Matt Hanley
298599ce91 Added show_header_name column to newsletters table
refs https://github.com/TryGhost/Team/issues/1532

- Added before the migration in https://github.com/TryGhost/Ghost/pull/14468 to populate the default newsletter
- The fixture for the default newsletter has a different value than the model and schema default
- This is because by default the newsletter name is the same as the site title, and the site title is already shown
2022-04-26 12:31:34 +01:00
Renovate Bot
65c2958f2a
Update dependency express to v4.18.0 2022-04-25 21:12:45 +00:00
Naz
f5937c6f65 Refactored destroyUser method to async/await
refs https://github.com/TryGhost/Toolbox/issues/268

- Before doing a functional change to this method wanted to make sure it's more readable to keep better track of changes
2022-04-25 10:50:04 +08:00
Daniel Lockyer
39f4abbbcd v4.45.0 2022-04-22 16:00:32 +01:00
Daniel Lockyer
11799b048b Updated Admin to v4.45.0 2022-04-22 16:00:32 +01:00
Renovate Bot
5bd44a05da Update dependency glob to v8 2022-04-22 13:53:15 +01:00
Renovate Bot
98c4956ff4 Update dependency c8 to v7.11.2 2022-04-22 13:51:14 +01:00
renovate[bot]
24d3b7cfd3
Pinned dependency @tryghost/stats-service to v0.1.0 (#14528)
Co-authored-by: Renovate Bot <bot@renovateapp.com>
2022-04-22 13:47:19 +01:00
Daniel Lockyer
d0553331db
Switched to commit hash for TryGhost/mock-knex
- I had to fork this dependency to add support for Knex 2.0 but yarn is
  weird and won't bump it if we use `master` because "nothing changed"
- Using a hash should force yarn to pull the changes if the hash gets
  updated
2022-04-22 12:24:02 +01:00
Daniel Lockyer
a394a00618
Fixed date-fns missing as a dependency declaration
refs fccee0614f

- `@tryghost/nql-lang` had `date-fns` declared as a devDependency but it
  was used within the library itself
- the referenced commit moved it to `dependencies` and this commit bumps
  the packages in Ghost
2022-04-22 11:41:48 +01:00
Daniel Lockyer
bead1bbe14 ℹ️ Fixed common build issues with SQLite
- we had to switch to `@vscode/sqlite3` a while back because `sqlite3`
  was unmaintained
- this fork didn't come with prebuilt binaries, so everyone had to
  compile them on their machine
- this brought a lot of issues with installing Ghost
- since then, the Ghost team have picked up maintenance of `sqlite3` and
  Knex has switched back, so we can switch back here too
2022-04-22 10:24:42 +01:00
Naz
2a592a9bf3 Added test coverage to settings bread service
no issue

- This serivce was missing any unit test coverage. Covering few cases gives a good groundwork to expand on
2022-04-22 16:31:56 +08:00
Naz
2d92e98392 Added a note about unusual implemenation
refs https://github.com/TryGhost/Toolbox/issues/280

- When passing the sendEmail parameter with a boutnd 'send' method the e2e tests were failing becuase mock mailer did not pick up the send method. Using this slightly more verbose implementation as it "just works". Could be rewritten into something nicer if there's such a need in the future
2022-04-22 16:31:56 +08:00
Naz
7b9a80fbf4 Extracted data access code to external module
refs https://github.com/TryGhost/Toolbox/issues/280

- Keeps the code in the core to the minimum. Ideally we should not be adding any logic apart form "hooking things together"
2022-04-22 16:31:56 +08:00
Naz
cc71bbfd61 Hooked up api version compatibility middleware
refs https://github.com/TryGhost/Toolbox/issues/280

- ctd of putting pieces together to allow Ghost notifying owner and admin users about version mismatch errors
-  The `@tryghost/mw-api-version-mismatch` in a combination with api version compatibility service make the whole notification process play nicely :)
- The flow of the logic from the request to a sent notification email is following:
1. Request comes is with an Accept-Version header that's behind current Ghost version and is not supported
2. mw-error-handler middleware's 'resourceNotFound' detects such request and returns a 406 with a special 'code' identifying if the version of the client is ahead or behind
3. mw-api-version-mismatch intercepts the 406 request with "code === 'UPDATE_CLIENT'` and calls up APIVersionCompatibilityService
4. emails are sent out to active owner and admin users

- The above flow is also illustratd in the e2e tests that come with the changeset
2022-04-22 16:31:56 +08:00
Naz
b56557ea04 Hooked up api version compatibility service
refs https://github.com/TryGhost/Toolbox/issues/280

- This is continuation of putting pieces together to allow Ghost notifying owner and admin users about version mismatch errors
-  This changeset adds the api-version-compatibility intialization during the boot (needed to be able to pospone the send email initialization so that it's testeable from the e2e tests)
- There's also a data service which handles fetching/saving notifications and fetching emails of users that should be notified
2022-04-22 16:31:56 +08:00
Naz
b7189a99e4 Added support for version missmatch handling
refs https://github.com/TryGhost/Toolbox/issues/280

- Email notification handling logic needs to be added to Ghost. Ideally there should be as little code landing in the core as possible - mostly data fetching and hooking modules together.
- The primary email handling logic for the `Accept-Version`/`Conent-Version` header missmatch is done in `api-version-compatibility-service` module, and the `mw-api-version-mismatch` allows to intercept request with the missmatch and call the api version compatibility service to do it's job. The mw-error-handler gives each case of client BEHIND or AHEAD of the Ghost version a unique error code, so that the versioning compatibility service has data to distinguish different cases
2022-04-22 16:31:56 +08:00
Rishabh
2ab78bcdc9 Updated settings test snapshot for new flag 2022-04-22 12:50:33 +05:30
Rishabh
05583d254f Added multiple newsletters UI feature flag 2022-04-22 12:40:51 +05:30
Renovate Bot
2bde20b7de Update dependency sinon to v13.0.2 2022-04-21 16:32:08 +00:00
Fabien 'egg' O'Carroll
19a08cd11d
Replaced stats service with @tryghost/stats-service (#14527)
- Pulls logic out of Ghost core and into the Analytics repository.
- Test coverage has increased for the stats service
- Interface simplified
2022-04-21 14:57:07 +01:00
Naz
1ecb837981 Added version_notifications key to settings table
refs https://github.com/TryGhost/Toolbox/issues/292

- When version missmatch handling is done in Ghost we need to store the 'Accept-Version' header values that have been already processed in the past (to avoid sending notifications about the same mismatch multiple times)
- The `version_notifications` will be storing an array with handled versions like so: `['v3.44', 'v4.23', 'v4.39']`.
- The emailing logic and processing is slightly similar to how "notification" key is handled, that's why I've placed the definition of this new key close by.
2022-04-21 20:34:19 +08:00
Simon Backx
b9646cceb1
Updated members API to return subscriptions' offers from the offer_id (#14515)
refs https://github.com/TryGhost/Team/issues/1520

**Changes in members repo**

Bumped to `5.12.0`, with the following changes:

- Compare differences via https://github.com/TryGhost/Members/compare/%40tryghost/members-api%405.11.1...%40tryghost/members-api%405.12.0
- Instead of doing the matching of the offers and subscriptions by looking at the offer redemptions, we can now look at the offer_id from subscriptions.
- This also fixes an issue where we don't attach the offer object to subscriptions in the members' browse method
- Updated browse behaviour to match the read behaviour of members (product relation needs to get loaded because it is missing in member.products if the subscription is expired).

**Tests**
- Includes test to see if the API correctly returns the offer object when fetching one or multiple members
- Check if the return format is the same for the read, edit and browse members admin API endpoints (offer was missing in subscriptions)
- Snapshot files have been updated because now the offer is returned in subscriptions (content length increased)
2022-04-21 12:06:06 +02:00
Simon Backx
9def4e6edc
Added migration to backfill offer ids in subscriptions (#14511)
refs https://github.com/TryGhost/Team/issues/1520

- Sets the `offer_id` in the `members_stripe_customers_subscriptions` table based on the `offer_redemptions` that have the same tier and cadence
- We currently use the same subscription <-> offer linking when viewing a member
- The MySQL query is quite optimized in a single UPDATE query, but in SQLite we'll need to run (maximum) one UPDATE query for every offer (not per subscription).
- Best to merge this migration in 4.x (not in 5.0) because it is better (less error prone) to run this migration before starting to fill the offer_id field for updated migrations instead of after (https://github.com/TryGhost/Ghost/pull/14488)
- We need the SQLite migration for sites that will only migrate to MySQL at 5.0
2022-04-21 11:45:24 +02:00
Thibaut Patel
52a855f314
Added the newsletter read endpoint (#14520)
refs https://github.com/TryGhost/Team/issues/1533

- Retrieves one newsletter
- Makes the newsletter resource consistent with the other resources
- Solves an issue with the admin expecting the route to exist
2022-04-21 09:59:44 +02:00
Naz
f28e65b4e7 Added staus code assertions to version test suite
refs https://github.com/TryGhost/Toolbox/issues/280

- All e2e tests should be checking returned status code! This was missed when writing tests
2022-04-21 12:45:21 +08:00
Renovate Bot
065078c397
Update dependency express-jwt to v6.1.2 2022-04-20 18:02:43 +00:00
Kevin Ansfield
fbaf49b882 Updated settings API snapshot for new labs flag
refs 05da593516

- test was failing because the `labs` property returned by the settings endpoint changed after the new labs flag was added
2022-04-20 16:29:51 +01:00
Kevin Ansfield
05da593516 Added publishingFlow labs flag
no issue

- alpha flag for testing a rework of the publishing flow in Admin
2022-04-20 16:20:01 +01:00
Thibaut Patel
ae664e9cad
Added newsletter read permission (#14519)
refs https://github.com/TryGhost/Team/issues/1533

- Needed to create the read newsletter endpoint to make the newsletter resource more consistent with the other resources
- Read is available to admins like other newsletter actions
2022-04-20 15:25:41 +02:00
Rishabh
5a2219bec3 Removed archived newsletters from Portal settings
Archived newsletters won't be shown in Portal for selection, this change filters to only include active newsletters in Portal settings.
2022-04-20 14:18:42 +05:30
Rishabh
6092b31c40 Refined newsletter selection in Portal UI 2022-04-20 14:18:27 +05:30
Simon Backx
a3215a6edf
Updated calculation of MRR history to use mrr column (#14481)
refs https://github.com/TryGhost/Team/issues/1518

- Uses mrr column now
- Doesn't account for `cancel_at_period_end` until a migration fixes the MRR values: https://github.com/TryGhost/Ghost/pull/14480
2022-04-19 17:07:20 +02:00
Simon Backx
0c72e78e6a
Stored offer_id in subscriptions (#14488)
refs https://github.com/TryGhost/Team/issues/1519

**Tests:**
- Tests whether the metadata from an offer is read correctly and stored in the database
- Test that invalid offer ids are ignored

**Changes in members repository:** 
- Compare changes here: https://github.com/TryGhost/Members/compare/%40tryghost/members-api%405.9.2...%40tryghost/members-api%405.11.1
- The `offer_id` column of subscriptions is set based on the coupon id from Stripe
- `getByStripeCouponId` method added in the offers repository (required to look up an offer from a stripe_coupon_id)
- the `members-payments` package was bumped twice (once for changes, once for undoing those changes, my bad). Nothing else has changed in that package.
2022-04-19 17:06:53 +02:00
Daniel Lockyer
3171df4102
🔥 Dropped support for Node 12
refs https://github.com/TryGhost/Toolbox/issues/267

- Node 12 becomes EOL on April 30th so we're going to be dropping
  support for it in Ghost
- this commit updates the Node engine ranges so CLI can pick this up,
  and drops 12.22.1 from the CI matrix
2022-04-19 15:53:44 +01:00
Daniel Lockyer
6d5a8c6b31 Bumped Ghost-CLI test Node version to v14
refs https://github.com/TryGhost/Toolbox/issues/267

- we're about to remove support for Node 12 so this needs bumping to
  keep things chugging along
2022-04-19 15:44:43 +01:00
Hannah Wolfe
9e8de9ab01
Moved cors middleware to own file
- This cors middleware function has been stuck at the top of the site file for a while
- Move it to it's own file in keeping with our usual patterns for middleware
2022-04-19 11:19:59 +01:00