refs https://ghost.slack.com/archives/C02G9E68C/p1651484563907609
- When the site has an empty name, it is set to `NULL` in settings.
- The name column is not nullable in newsletters, breaking the migration in that case.
- Fixed by excluding all non-nullable columns
- Replaced JS Date object with raw SQL `CURRENT_TIMESTAMP`
refs https://github.com/TryGhost/Toolbox/issues/300
- the previous commit deleted all v1/v2/v3 migrations
- Ghost-CLI forces you to update to the latest minor in your major
before you upgrade but people who aren't on the latest minor, who don't
use Ghost-CLI and try to update to v5 might have missed migrations that
I've just deleted
- the way to protect against this is to add some migrations for the last
minor in each major, that will throw an error if they get run
- this uses a feature of knex-migrator where it will always try to
backfill missing migrations when you run Ghost, so these new migrations
should _only_ be run if the Ghost DB hasn't already run the same number
of migrations in that minor
- by throwing an error, it'll cause knex-migrator to fail and the user
shouldn't be able to update, which is good
- v2 and v3 only have 1 migration so I can just replace that, but v1 has
2 migrations. I think it makes more sense that the first one errors
and the second one is a no-op otherwise it'll run the first migration,
succeed, run the second, error, and then rollback the second and first
one
- the new migration names are different from the original ones but that
shouldn't matter because we're not comparing nor storing them
refs https://github.com/TryGhost/Toolbox/issues/300
- due to Node compatibility, it only makes sense that users on the latest
v3 (and v4) can update to v5
- therefore, we don't need v1/2/3 migrations as it's more maintenance in
the long run
- this deletes over 5000 lines of code (!!)
refs https://github.com/TryGhost/Team/issues/1561
With multiple newsletters, unsubscribe links will also need to have a unique reference to the newsletter that the email is for, so that we can unsubscribe members from the particular newsletter automatically when they click on the link.
As our existing pattern for members is to use UUID as the external unique reference, this change adds UUID to newsletter schema and populates the existing newsletters with a UUID value.
- adds new `uuid` column to newsletter schema
- updates newsletter model to add default uuid
- updates default newsletter migration to add `uuid`
- drops nullable on `uuid` column later in migrations once we have populated existing newsletters
closes: https://github.com/TryGhost/Team/issues/1553
- we want to be explicit in what values are supported
- we want the values that are supported to be supported to also be explicit and clear
- without this API users can set the value to anything and the active/not active logic will work, until such time as we introduce further statuses
- this means introducing a new status could be a potential breaking change and lands us in horrible hot water
refs https://github.com/TryGhost/Team/issues/1551
- Updated existing migration to insert a nullable created_at column
- Added a migration to update all the created_at values to now
- Added a migration to drop nullable
- Also includes new helper methods to set and drop nullable
Closes https://github.com/TryGhost/Team/issues/1508
Refs https://github.com/TryGhost/Ghost/pull/14468
- Maps existing subscribers to the default newsletter
A note on performance:
We loop over the rows in a potentially large table (members) but I've minimised the impact by limiting the columns we fetch. The alternative is a raw SQL query like the one below: the SQL version takes ~0.9s vs ~1.1s for the migration (my laptop, ~30k members). The disadvantage of the raw SQL implementation is the approximation of the ObjectID (instead of a legit bson ID) which isn't sequential and may impact index size/performance.
```sql
insert into members_newsletters (id, member_id, newsletter_id)
select
substr(replace(uuid(),'-',''),1,24),
id,
'62595dcbfad4e031d85d166f'
from members
where members.subscribed=true;
```
refs https://github.com/TryGhost/Team/issues/1478
The `newsletter_id` is nullable for now to remain compatible until we have a proper data migration + updated code to set it on inserts
refs https://github.com/TryGhost/Team/issues/1473
- Added the default newsletter
- We use the title to populate the newsletter same, slug and sender name
- We use the description to populate the newsletter description
- We use the global design settings to populate the corresponding newsletter design settings
Co-authored-by: Matt Hanley <git@matthanley.co.uk>
refs https://github.com/TryGhost/Team/issues/1532
- Added before the migration in https://github.com/TryGhost/Ghost/pull/14468 to populate the default newsletter
- The fixture for the default newsletter has a different value than the model and schema default
- This is because by default the newsletter name is the same as the site title, and the site title is already shown
refs https://github.com/TryGhost/Toolbox/issues/292
- When version missmatch handling is done in Ghost we need to store the 'Accept-Version' header values that have been already processed in the past (to avoid sending notifications about the same mismatch multiple times)
- The `version_notifications` will be storing an array with handled versions like so: `['v3.44', 'v4.23', 'v4.39']`.
- The emailing logic and processing is slightly similar to how "notification" key is handled, that's why I've placed the definition of this new key close by.
refs https://github.com/TryGhost/Team/issues/1520
- Sets the `offer_id` in the `members_stripe_customers_subscriptions` table based on the `offer_redemptions` that have the same tier and cadence
- We currently use the same subscription <-> offer linking when viewing a member
- The MySQL query is quite optimized in a single UPDATE query, but in SQLite we'll need to run (maximum) one UPDATE query for every offer (not per subscription).
- Best to merge this migration in 4.x (not in 5.0) because it is better (less error prone) to run this migration before starting to fill the offer_id field for updated migrations instead of after (https://github.com/TryGhost/Ghost/pull/14488)
- We need the SQLite migration for sites that will only migrate to MySQL at 5.0
refs https://github.com/TryGhost/Team/issues/1533
- Needed to create the read newsletter endpoint to make the newsletter resource more consistent with the other resources
- Read is available to admins like other newsletter actions
refs https://github.com/TryGhost/Team/issues/1519
Migration that adds the (nullable) offer_id column to members_stripe_customers_subscriptions.
- New reliable way to know which offer is active for a given subscription (currently we compare the tier and cadence in offer redemptions)
- We'll create a separate migration to backfill all the offer_ids, but only after we updated the code to also store them correctly for new or updated subscriptions (https://github.com/TryGhost/Team/issues/1520)
- Allows us to backfill the MRR of all subscriptions to account for forever offers
refs TryGhost/Team#1513
- nullable `sender_name` allows us to use auto fallback of site title for sender name without setting any explicit value for it.
refs https://github.com/TryGhost/Team/issues/1504
- The permissions were missing in the fixture file
- This caused some Ghost installs to not have the right permissions
- This is fixed by adding the missing permissions to the fixture file and creating a migration to resolve the missing permissions
closes https://github.com/TryGhost/Team/issues/1491
With multiple newsletters feature, a site should always have at-least one newsletter by default. Also, as with the default product, the default newsletter also needs to be renamed to the site title during the setup flow.
- adds default newsletter to main and test fixtures
- updates setup flow to rename newsletter name and sender name to site title
- updates model to extend default value for fields
- updates test
refs https://github.com/TryGhost/Team/issues/1457
We want to save the MRR with a subscription to simplify the calculation of the total MRR once, in 5.0, we also take 'forever' offers into account into the MRR (so we can just SUM the MRR of all subscriptions).
- Sets the MRR to 0 for now.
- Separate commit will fill in all the values in a data migration, but this needs to get merged first because we need this new column in order to update the members-api package (so we already save the MRR before doing the data migration).
- Updated `test/e2e-api/admin/legacy-members.test.js` with improved body assertions.
refs https://github.com/TryGhost/Team/issues/1302
We do not want to update MRR calculations until 5.0 - so will be adding
a separate migration to populate the mrr_delta column for 5.0
We've only added events for non-expired subscriptions as this is simpler
and won't impact the mrr events when mrr_delta is updated
refs https://github.com/TryGhost/Team/issues/1302
Currently we only have three event types: created, updated & expired.
A created event always has a `from_plan` of null
An updated event will always have a different `to_plan` and `from_plan`
as the subscription has changed tier/cadence.
An expired event _should_ have a `to_plan` of null, but due to a bug we
have events with the same `from_plan` and `to_plan`.
refs https://github.com/TryGhost/Team/issues/1500
- newsletter schema columns were using incorrect syntax for `isIn` validations
- adds schema test to make sure schema `isIn` validations are in correct format
refs https://github.com/TryGhost/Team/issues/1497
- This enables us to keep track of which newsletter an email was sent to even if the related post is deleted.
- Adds the `newsletter_id` property to the email API
refs https://github.com/TryGhost/Team/issues/1500
The newsletter table schema has bunch of changes to go through for new and existing columns, this consolidates the schema changes into a single re-create table migration that drops and adds the newsletter table with correct schema. The table re-create migration needs to run before any of the tables using newsletter as foreign key. The changes include -
- new columns for design related fields
- new slug column for filtering
- unique constraint to `name` column
- remove `default` column (noops the existing default column migration)
- `sender_reply_to` has a default of newsletter and a validation of ['newsletter', 'support']
- updated default values for `subscribe_on_signup` and `recipient_filter`
* Added subscription_id to MRR events
refs https://github.com/TryGhost/Team/issues/1453
As part of 5.0 we want to have cancelled evetns affect MRR. We are going
to backfill and begin populating cancelled events _before_ 5.0 however,
adding a migration to set the MRR deltas as part of 5.0
This migration will need to associate the cancelled events with a
subscription, so we are adding the subscription_id now. This will allow
us to guarantee that all cancelled events will have a subscription_id.
The column is missing a NOT NULL and REFERENCES constraint because
we have not populated the missing values. These will be added in the future
once we have populated the column for all rows
refs https://github.com/TryGhost/Team/issues/1474
- The `default` concept will be replaced by the first newsletter based on the `sort_order`
- This removes the `default` value from the newsletter API
- This simplifies the design to make the api and datastructure more maintainable
refs https://github.com/TryGhost/Team/issues/1471
- This is a many-to-one relation so that many posts can be linked to a specific newsletter
- The `newsletters` table had to come first in the schema file so that it's initialized before the `posts` table (because of the foreign key)
- Updated the model to make sure the new field doesn't leak in the API for now
- This migration isn't using the `createAddColumnMigration` util because of a performance issue. In MySQL, adding/dropping a column without `algorithm=copy` uses the INPLACE algorithm which was too slow on big posts tables (~3 minutes for 10k posts). Switching to the COPY algorithm fixed the issue (~3 seconds for 10k posts).
- SQLite isn't using the codepath where we run a raw SQL query because `knex` is doing multiple queries to add/remove a column
refs: https://github.com/TryGhost/Ghost/commit/11867ab43
- These checks live in the wrong place. They are mostly a frontend thing
- The only server place they were used was slack and that was fixed in 11867ab43
- Moving these to the frontend they fit neatly into the frontend data service
- We had an incident where a migration file was misnamed.
- This is caused by using slimer with spaces rather than hyphens
- We didn't imagine this case when writing the regex for matching filenames
- However, now we know it, it's easy to tweak the regex to match this case
- This requires adding an override to the one badly named file
refs https://github.com/TryGhost/Team/issues/1469
With multiple newsletters, members will now be able to subscribe to one or more newsletters on the site. Previously, the subscription to default newsletter for a member was controlled via a single boolean `subscribed` column on the member table.
This change allows mapping multiple newsletters to a member via new pivot table that stores relation between a member and newsletter.
- adds new `members_newsletters` pivot table
- update tests
- if we add a column with a foreign key reference, the `down` migration
will try to remove that column
- you can't remove a column without deleting the foreign key reference
first
- our migration utils didn't take that into account and there's nothing
in Knex to do this for us
- this commit deletes the foreign key before removing the column if we
have one referenced in the column spec
- also updates the code to pass the column spec into the util
- this commit removes the colons from the log messages to make each
line flow easier when you're reading it
- also removes a couple of superfluous words that aren't needed
refs https://github.com/TryGhost/Team/issues/1463
- Allow admins to perform all newsletter operations
- We can adjust and be more permissive in the future if needed
- Added the tests back as permissions are configured correctly now
refs https://github.com/TryGhost/Team/issues/1433
- The `default` property stores whether a newsletter is set as default by the admin
- The `status` property stores whether a newsletter is archived or not
- The `recipient_filter` property is only storing whether a newsletter is "paid-only" or not for now, although it can be expanded to more specific filters in the future
- The `subscribe_on_signup` property stores whether a new member should be automatically signed up to the newsletter
- The `sort_order` property enables displaying the newsletter list in an order chosen by the admins
As multiple tiers is now GA, we want to allow devs to be able to work with Tiers and offers via content/Admin API. This change -
- updates fixtures to add permissions to admin integration role for new sites
- adds migration to update existing sites to have correct permissions for role
- whitelists add/edit/read/browse on tiers and offers API for integrations
