Commit Graph

10376 Commits

Author SHA1 Message Date
Rishabh
05583d254f Added multiple newsletters UI feature flag 2022-04-22 12:40:51 +05:30
Fabien 'egg' O'Carroll
19a08cd11d
Replaced stats service with @tryghost/stats-service (#14527)
- Pulls logic out of Ghost core and into the Analytics repository.
- Test coverage has increased for the stats service
- Interface simplified
2022-04-21 14:57:07 +01:00
Naz
1ecb837981 Added version_notifications key to settings table
refs https://github.com/TryGhost/Toolbox/issues/292

- When version missmatch handling is done in Ghost we need to store the 'Accept-Version' header values that have been already processed in the past (to avoid sending notifications about the same mismatch multiple times)
- The `version_notifications` will be storing an array with handled versions like so: `['v3.44', 'v4.23', 'v4.39']`.
- The emailing logic and processing is slightly similar to how "notification" key is handled, that's why I've placed the definition of this new key close by.
2022-04-21 20:34:19 +08:00
Simon Backx
9def4e6edc
Added migration to backfill offer ids in subscriptions (#14511)
refs https://github.com/TryGhost/Team/issues/1520

- Sets the `offer_id` in the `members_stripe_customers_subscriptions` table based on the `offer_redemptions` that have the same tier and cadence
- We currently use the same subscription <-> offer linking when viewing a member
- The MySQL query is quite optimized in a single UPDATE query, but in SQLite we'll need to run (maximum) one UPDATE query for every offer (not per subscription).
- Best to merge this migration in 4.x (not in 5.0) because it is better (less error prone) to run this migration before starting to fill the offer_id field for updated migrations instead of after (https://github.com/TryGhost/Ghost/pull/14488)
- We need the SQLite migration for sites that will only migrate to MySQL at 5.0
2022-04-21 11:45:24 +02:00
Thibaut Patel
52a855f314
Added the newsletter read endpoint (#14520)
refs https://github.com/TryGhost/Team/issues/1533

- Retrieves one newsletter
- Makes the newsletter resource consistent with the other resources
- Solves an issue with the admin expecting the route to exist
2022-04-21 09:59:44 +02:00
Kevin Ansfield
05da593516 Added publishingFlow labs flag
no issue

- alpha flag for testing a rework of the publishing flow in Admin
2022-04-20 16:20:01 +01:00
Thibaut Patel
ae664e9cad
Added newsletter read permission (#14519)
refs https://github.com/TryGhost/Team/issues/1533

- Needed to create the read newsletter endpoint to make the newsletter resource more consistent with the other resources
- Read is available to admins like other newsletter actions
2022-04-20 15:25:41 +02:00
Rishabh
5a2219bec3 Removed archived newsletters from Portal settings
Archived newsletters won't be shown in Portal for selection, this change filters to only include active newsletters in Portal settings.
2022-04-20 14:18:42 +05:30
Rishabh
6092b31c40 Refined newsletter selection in Portal UI 2022-04-20 14:18:27 +05:30
Simon Backx
a3215a6edf
Updated calculation of MRR history to use mrr column (#14481)
refs https://github.com/TryGhost/Team/issues/1518

- Uses mrr column now
- Doesn't account for `cancel_at_period_end` until a migration fixes the MRR values: https://github.com/TryGhost/Ghost/pull/14480
2022-04-19 17:07:20 +02:00
Hannah Wolfe
9e8de9ab01
Moved cors middleware to own file
- This cors middleware function has been stuck at the top of the site file for a while
- Move it to it's own file in keeping with our usual patterns for middleware
2022-04-19 11:19:59 +01:00
Hannah Wolfe
1d121c52f4
Replaced white/black list terminology
refs 92986b77e3

- I thought we did this a while ago, but uses in comments and elsewhere in the codebase were missed
2022-04-19 11:19:59 +01:00
Daniel Lockyer
7ac7652dd9 Updated Admin to v4.44.0 2022-04-15 16:00:39 +01:00
Simon Backx
9b23424e48
Added offer_id column to members_stripe_customers_subscriptions (#14487)
refs https://github.com/TryGhost/Team/issues/1519

Migration that adds the (nullable) offer_id column to members_stripe_customers_subscriptions.
- New reliable way to know which offer is active for a given subscription (currently we compare the tier and cadence in offer redemptions)
- We'll create a separate migration to backfill all the offer_ids, but only after we updated the code to also store them correctly for new or updated subscriptions (https://github.com/TryGhost/Team/issues/1520)
- Allows us to backfill the MRR of all subscriptions to account for forever offers
2022-04-15 13:47:15 +02:00
Rishabh Garg
1362750640
Updated newsletter fixtures to remove nullable sender name (#14479)
refs https://github.com/TryGhost/Team/issues/1513

Since `sender_name` for newsletter is now nullable field, this change removes hardcoded `sender_name` from fixtures.
2022-04-14 23:45:59 +05:30
Sam Lord
5d62758e64 Fixed verification trigger for imports
refs: https://github.com/TryGhost/Toolbox/issues/293

This import verification check now works by testing all imports in the last 30 days.
2022-04-14 16:04:26 +01:00
Rishabh Garg
5218cf194d
Updated newsletter sender name column as nullable (#14476)
refs TryGhost/Team#1513

- nullable `sender_name` allows us to use auto fallback of site title for sender name without setting any explicit value for it.
2022-04-14 17:42:20 +05:30
Naz
fa82722b03 Uppercased Content-Version header use
refs https://github.com/TryGhost/Toolbox/issues/227

- The convention across the codebase is to define headers with capitalized first letters. This change does not affect the output though as all headers are served lowercased anyway.
- Might be a good idea to make all headers lowercased one day to match the casing with the outputs
2022-04-14 15:21:35 +08:00
Thibaut Patel
bfbece948e Fixed the previous commit issue
refs 7a55e1a60a
2022-04-13 16:51:10 +02:00
Rishabh
00e5e87d4b Added newsletter preference selection to Portal signup flows
refs https://github.com/TryGhost/Team/issues/1490

- adds intermediary newsletter pref selection page to signup flows in case a site has multiple newsletters
  - for members signing up as free via magic link
  - for members signing up as paid via stripe checkout
  - for members signing up as paid via offer url
2022-04-13 20:06:27 +05:30
Thibaut Patel
7a55e1a60a
Added the newsletter_id query string to the post edit API route (#14449)
refs https://github.com/TryGhost/Team/issues/1502

- Support the `newsletter_id` only when sending a newsletter
- Default to the default newsletter when `newsletter_id` isn't specified
- Ignore the `newsletter_id` parameter when passed in the post body
2022-04-13 15:27:43 +02:00
Simon Backx
cb1f0802d4
Added migration that fills the mrr column in members_stripe_customers_subscriptions (#14465)
refs https://github.com/TryGhost/Team/issues/1456

- Sets the MRR of active subscriptions
- Mirrors the behaviour of the getMRR method from MembersRepository in `members-api`
2022-04-13 14:07:34 +02:00
Thibaut Patel
ffc57301f6
Fixed the newsletters api permissions (#14458)
refs https://github.com/TryGhost/Team/issues/1504

- The permissions were missing in the fixture file
- This caused some Ghost installs to not have the right permissions
- This is fixed by adding the missing permissions to the fixture file and creating a migration to resolve the missing permissions
2022-04-13 12:02:06 +02:00
Rishabh Garg
e6f96d4711
Added filtering for members api on newsletter subscriptions (#14460)
refs https://github.com/TryGhost/Team/issues/1484

While sending a post to a specific newsletter, we'll need to get list/count of members eligible to receive the post. This change enables members admin API to filter list of members on specific newsletter by their slug.
2022-04-12 21:01:49 +05:30
Rishabh Garg
95670eed52
Handled default newsletter fixtures setup (#14459)
closes https://github.com/TryGhost/Team/issues/1491

With multiple newsletters feature, a site should always have at-least one newsletter by default. Also, as with the default product, the default newsletter also needs to be renamed to the site title during the setup flow.

- adds default newsletter to main and test fixtures
- updates setup flow to rename newsletter name and sender name to site title
- updates model to extend default value for fields
- updates test
2022-04-12 19:44:21 +05:30
Simon Backx
2db746e559
Added mrr column to subscriptions (#14448)
refs https://github.com/TryGhost/Team/issues/1457

We want to save the MRR with a subscription to simplify the calculation of the total MRR once, in 5.0, we also take 'forever' offers into account into the MRR (so we can just SUM the MRR of all subscriptions).

- Sets the MRR to 0 for now.
- Separate commit will fill in all the values in a data migration, but this needs to get merged first because we need this new column in order to update the members-api package (so we already save the MRR before doing the data migration).
- Updated `test/e2e-api/admin/legacy-members.test.js` with improved body assertions.
2022-04-12 11:02:18 +02:00
Fabien 'egg' O'Carroll
cf6eef60e3
Added migration to populate cancellation events (#14438)
refs https://github.com/TryGhost/Team/issues/1302

We do not want to update MRR calculations until 5.0 - so will be adding
a separate migration to populate the mrr_delta column for 5.0

We've only added events for non-expired subscriptions as this is simpler
and won't impact the mrr events when mrr_delta is updated
2022-04-11 15:18:12 +01:00
Daniel Lockyer
184dc861cb
Merged v4.43.1 into main
v4.43.1
2022-04-11 12:21:16 +01:00
Daniel Lockyer
10248f71e6 Updated Admin to v4.43.1 2022-04-11 12:16:38 +01:00
Fabien 'egg' O'Carroll
ee9190b912
Populated type column of existing events (#14437)
refs https://github.com/TryGhost/Team/issues/1302

Currently we only have three event types: created, updated & expired.

A created event always has a `from_plan` of null

An updated event will always have a different `to_plan` and `from_plan`
as the subscription has changed tier/cadence.

An expired event _should_ have a `to_plan` of null, but due to a bug we
have events with the same `from_plan` and `to_plan`.
2022-04-11 11:41:44 +01:00
Rishabh Garg
cb76b738ce
Fixed newsletter schema column validation (#14456)
refs https://github.com/TryGhost/Team/issues/1500

- newsletter schema columns were using incorrect syntax for `isIn` validations
- adds schema test to make sure schema `isIn` validations are in correct format
2022-04-11 15:05:02 +05:30
Daniel Lockyer
3df2158d23 Updated Admin to v4.43.0 2022-04-08 16:00:32 +01:00
Thibaut Patel
1ba72edce6
Added the emails-newsletters relation (#14434)
refs https://github.com/TryGhost/Team/issues/1497

- This enables us to keep track of which newsletter an email was sent to even if the related post is deleted.
- Adds the `newsletter_id` property to the email API
2022-04-08 16:16:23 +02:00
Rishabh Garg
b0df387deb
Cleaned up newsletter schema migrations (#14441)
refs https://github.com/TryGhost/Team/issues/1500

The newsletter table schema has bunch of changes to go through for new and existing columns, this consolidates the schema changes into a single re-create table migration that drops and adds the newsletter table with correct schema. The table re-create migration needs to run before any of the tables using newsletter as foreign key. The changes include -

- new columns for design related fields
- new slug column for filtering
- unique constraint to `name` column
- remove `default` column (noops the existing default column migration)
- `sender_reply_to` has a default of newsletter and a validation of ['newsletter', 'support']
- updated default values for `subscribe_on_signup` and  `recipient_filter`
2022-04-08 19:06:30 +05:30
Fabien 'egg' O'Carroll
7ff0247e8d
Added subscription_id to MRR events (#14445)
* Added subscription_id to MRR events

refs https://github.com/TryGhost/Team/issues/1453

As part of 5.0 we want to have cancelled evetns affect MRR. We are going
to backfill and begin populating cancelled events _before_ 5.0 however,
adding a migration to set the MRR deltas as part of 5.0

This migration will need to associate the cancelled events with a
subscription, so we are adding the subscription_id now. This will allow
us to guarantee that all cancelled events will have a subscription_id.

The column is missing a NOT NULL and REFERENCES constraint because
we have not populated the missing values. These will be added in the future
once we have populated the column for all rows
2022-04-08 14:04:40 +01:00
Fabien 'egg' O'Carroll
022c8c8e69
Added support for 'FOR UPDATE' lock (#14433)
refs https://github.com/TryGhost/Team/issues/1248

This is the underlying cause of the problems we've seen whilst handling
Stripe webhooks. A transaction ensures that the operations are atomic,
but not that they can run concurrently.

If you have some code which does this, concurrently:
1. Starts a transaction
2. Reads a value
3. Changes the values
4. Ends the transaction

Without applying the `FOR UPDATE` lock - you will have both sequenes
read the same value at step 2.

With the `FOR UPDATE` lock - one of the sequences will hang at step 2,
waiting for the other transaction to end, at which point it will resume
and read the _changed_ value.

Because the `edit` method explicitly does a read followed by a write, we
have also add the `FOR UPDATE` lock to this by default, to avoid any race
conditions. This does however require that `edit` is called within a
transaction. An issue here https://github.com/TryGhost/Team/issues/1503
considers running in a transaction by default.
2022-04-08 12:52:33 +01:00
Naz
76aa2479f8 Added 'content-version' header response
refs https://github.com/TryGhost/Toolbox/issues/280

- In response to 'Accept-Version' header in the request headers, Ghost will always respond with a content-version header indicating the version of the Ghost install that is responding. This should signal to the client the content version that is bein g served
- This is a bare bones implementation and more logic with edge cases where `content-version` is served with a  version value of "best format API could respond with" will be added later.
2022-04-08 20:41:54 +12:00
Simon Backx
132726fe20
Added MRR stats service and endpoint (#14427)
refs https://github.com/TryGhost/Team/issues/1470

Instead of counting the MRR by resolving all the deltas from the past until now, we should start with the current calculated MRR and resolve it until the first event. That would give a more accurate recent MRR (in exchange for a less accurate MRR for older data) and allows us to limit the amount of returned days in the future.

- Includes MRR stats service that can fetch the current MRR per currency
- The service can return a history of the MRR for every day and currency
- New admin API endpoint /stats/mrr that returns the MRR history
- Includes tests for these new service and endpoint
2022-04-08 09:18:04 +02:00
Fabien 'egg' O'Carroll
6507a0b88e
Added type column to members_paid_subscription_events (#14432)
refs https://github.com/TryGhost/Team/issues/1302

This column allows us to store multiple events types in the table, which
can be differentiated by their `type`
2022-04-07 11:29:14 +01:00
Thibaut Patel
88877312b5
Removed the default column from the newsletters table (#14428)
refs https://github.com/TryGhost/Team/issues/1474

- The `default` concept will be replaced by the first newsletter based on the `sort_order`
- This removes the `default` value from the newsletter API
- This simplifies the design to make the api and datastructure more maintainable
2022-04-07 11:50:20 +02:00
Thibaut Patel
2bfd8f8b7e
Added the post-newsletter relation (#14411)
refs https://github.com/TryGhost/Team/issues/1471

- This is a many-to-one relation so that many posts can be linked to a specific newsletter
- The `newsletters` table had to come first in the schema file so that it's initialized before the `posts` table (because of the foreign key)
- Updated the model to make sure the new field doesn't leak in the API for now
- This migration isn't using the `createAddColumnMigration` util because of a performance issue. In MySQL, adding/dropping a column without `algorithm=copy` uses the INPLACE algorithm which was too slow on big posts tables (~3 minutes for 10k posts). Switching to the COPY algorithm fixed the issue (~3 seconds for 10k posts).
- SQLite isn't using the codepath where we run a raw SQL query because `knex` is doing multiple queries to add/remove a column
2022-04-07 10:26:37 +02:00
Rishabh Garg
90e7887007
Added default newsletter subscription for new members (#14431)
refs https://github.com/TryGhost/Team/issues/1469

Currently, all new members get auto subscribed to the default newsletter. This change adds same behavior with multiple newsletters by auto subscribing all available newsletters on site for new members(If flag is enabled). 
Note: In future, this will also take into consideration the `subscribe_on_signup` flag for a newsletter to filter which newsletters should a member be auto-subscribed.

- adds newsletters service for working with newsletter data
- bumps `@tryghost/members-api` package which handles default subscription
- adds new test fixture/data for newsletters
2022-04-07 08:30:00 +05:30
Simon Backx
1957b5b789
Removed pagination from members stats endpoint and added extra day to output (#14429) 2022-04-06 17:10:47 +02:00
Naz
cf8be34c43 Changed frames http module to use async/await
no issue

- It was hard to plug in with additional code into current `.then` based chain. Refactoring to use a more modern syntax helps with readability and allows for easier edits
2022-04-06 14:48:30 +08:00
Hannah Wolfe
08479f3816
Moved routing helpers to rendering service
- The helpers folder was full of things used for rendering pages
- It belongs as its own service so that we can see what it really does
2022-04-05 20:12:20 +01:00
Hannah Wolfe
c902d91c81
Renamed rendering service to handlebars
- This fits more closely, as this service is to so with rendering helpers and small parts
- Whereas we want to use "rendering" for things concerned with rendering pages
2022-04-05 20:10:33 +01:00
Rishabh
5a3d9ddf2e Wired newsletter preference page in Portal to API
refs https://github.com/TryGhost/Team/issues/1469

- wires newsletter preferences for a member in Portal to view/edit real data
- allows members to control their subscription to multiple newsletters
2022-04-05 22:49:09 +05:30
Rishabh
473775788d Disabled after hook for members newsletters pivot table
refs https://github.com/TryGhost/Team/issues/1469

We have an after hook to update `sort_order` for pivot tables after update as base bookshelf plugin. Since new `members_newsletters` table doesn't has a `sort_order` column on it, this change disables the after hook for it to avoid errors.
2022-04-05 22:23:39 +05:30
Rishabh
16f35df448 Added newsletters to Portal site data
refs https://github.com/TryGhost/Team/issues/1469

- allows Portal access to multiple newsletters on a site
- allows Portal to update member's newsletter subscription preference
2022-04-05 22:23:39 +05:30
Rishabh
a1417e86b7 Added relation between newsletters and members
refs https://github.com/TryGhost/Team/issues/1469

- updates member model to add relation to newsletter via pivot table
- updates member api serializer to include newsletter data
- updates tests
2022-04-05 22:23:39 +05:30