Commit Graph

43 Commits

Author SHA1 Message Date
Thibaut Patel
e52dc87b7c Added a foreign key from webhooks to integrations
issue https://github.com/TryGhost/Team/issues/477
2021-03-04 10:56:58 +01:00
Thibaut Patel
126f9004b4 Updated a duplicate migration id
no issue
2021-03-03 10:45:21 +01:00
Thibaut Patel
8b7d7ba1f1 Fixed the add/drop foreign key on sqlite
issue https://github.com/TryGhost/Ghost/pull/12713
ref https://github.com/knex/knex/issues/4155
2021-03-03 10:35:10 +01:00
Thibaut Patel
0843ab6a37 Added 'on delete cascade' to several foreign keys in sqlite
issue https://github.com/TryGhost/Team/issues/476
blocked by https://github.com/TryGhost/Ghost/pull/12702
2021-03-03 10:35:10 +01:00
Fabien O'Carroll
d8fa54024b Fixed migrations to 4.x from 1.x
no-issue

The slack setting in 1.x did not have a `username` property. When
updating from 1.x, the migration to move the slack setting from a JSON
blob to two individual settings, would assume that the setting in the
database would have a value for the `username` property. This resulted
in errors on SQLite "sqlite does not support inserting default values."

The fix here is to add defaults when reading from the database, meaning
that we will _never_ attempt to insert `undefined`
2021-03-01 15:41:02 +00:00
Fabien 'egg' O'Carroll
3b6c55ec53
Remove orphaned Stripe data from SQLite (#12704)
refs https://github.com/TryGhost/Team/issues/476

* Moved paid subscription events population migration

This migration relies on the members_stripe_customers and
members_stripe_customer_subscriptions tables having no orphaned records
in order for it to correcly generate its data.

The migration to clean up orphaned records in those tables has not been
implemented yet, moving this migration free's up the "14" slot

* Removed orphaned stripe data from SQLite3

SQLite databases do not handle removing orphaned stripe records after a
member has been deleted. Our migration to populate the paid subscription
events relies on each customer and subscription being associated with a
member.
2021-03-01 14:32:49 +00:00
Fabien O'Carroll
68008baf0e 🐛 Fixed email permissions for all roles
refs https://github.com/TryGhost/Team/issues/494

The migrations in 3.1.0 which added email permissions did not add those
permissions to the roles. This means that whilst we have the permissions
in the database, only the Owner role could use any of them.

This migration ensures that the email related permissions are added to
the correct roles.
2021-03-01 13:17:43 +00:00
Thibaut Patel
4a939054ba Added a primary key to the brute table
issue https://github.com/TryGhost/Team/issues/267
2021-03-01 12:17:26 +01:00
Kevin Ansfield
722825055f
Added posts html regeneration migration (#12660)
refs https://github.com/TryGhost/Team/issues/467
refs https://github.com/TryGhost/Team/issues/221

- we've introduced backwards-compatible changes to rendering in 3.0 such as srcset and sizes which will only have taken effect on posts created or edited since the changes were made
- 4.0 brings additional changes such as image card width/height
- re-generating the `html` field of all posts from the `mobiledoc` brings all content up to latest rendering output
2021-02-23 17:32:07 +00:00
Thibaut Patel
5b72b20d1e Update the migration number for orphaned webhooks
no issue
2021-02-23 10:46:33 +01:00
Thibaut Patel
2250cd79e1 💡 Resolve orphaned webhooks
issue https://github.com/TryGhost/Ghost/issues/12567
2021-02-23 10:14:34 +01:00
Kevin Ansfield
08e1268aed
Added migration to remove surrounding <> in email_batches.provider_id (#12673)
refs https://github.com/TryGhost/Team/issues/221#issuecomment-759105424

- Mailgun responds to an email send with a provider id in the format `<x@y.com>` but everywhere else it's used in their API it uses the format `x@y.com`
- updates email batch save to strip the brackets, and migration removes brackets from existing records so we no longer have to add special handling for the stored id any time we use it
2021-02-23 08:48:21 +00:00
Daniel Lockyer
a2dd7c7f7d
Unified migration logging styles
no issue

- all migrations should contain a logging message
- info for successful things, or warn if we deviated from what was
  expected for some reason
- also added some spacing to make them easier to read
2021-02-18 17:35:32 +00:00
Daniel Lockyer
bf960bcecb
Fixed typo in migration message
no issue

- simple rewording to clean up the message
2021-02-18 17:05:00 +00:00
Naz
abb8c1df74 Updated unsplash setting migration to be idempotent
refs https://github.com/TryGhost/Ghost/issues/10318

- Previous version of migration was not following the idempotence rule of migrations
2021-02-18 10:55:59 +13:00
Naz
e0cc314f91 Removed duplicate migration
no issue

- The migration didn't want to go away after rebases combined with renames
2021-02-18 10:55:59 +13:00
Fabien O'Carroll
7785fb6dc5 Refactored usage of pre v4 "unsplash" setting format
refs https://github.com/TryGhost/Ghost/issues/10318

- API changes introduced:

canary/v4 Admin API
GET /settings/ (browse)

+ "unsplash" present in response as boolean value

GET /settings/:settingName (read)

+ "unsplash" present in response as boolean value

PUT /settings/ (edit)

+ "unsplash" updates setting, accepts ONLY  boolean format

v3  Admin API
GET /settings/ (browse)

+ "unsplash" present in response with object value

GET /settings/:settingName (read)

+ "unsplash" present in response with object value

PUT /settings/ (edit)

+ "unsplash" updates setting, accepts either boolean or object formats

v2 Admin API
GET /settings/ (browse)

+ "unsplash" present in response with object value

GET /settings/:settingName (read)

+ "unsplash" present in response with object value

PUT /settings/ (edit)

+ "unsplash" updates setting, accepts object format
2021-02-18 10:55:59 +13:00
Fabien O'Carroll
9cb069eb8e 🏗 Updated "unsplash" setting handling to match updated format
refs https://github.com/TryGhost/Ghost/issues/10318

- JSON object format used in previous "unsplash" setting was considered an
anti-pattern. Flat structure was extracted out of the "unsplash.isActive" JSON.
- The naming convention uses `amp` as  a precedent (https://github.com/TryGhost/Team/issues/331#issuecomment-658815017)
2021-02-18 10:55:59 +13:00
Naz
0097f2037c Fixed slack setting migration to follow idempotance rule
refs https://github.com/TryGhost/Ghost/pull/12631

- Initialy writen migration would break if executed twice
- Also accounted for situation if only part of the migration executed leaving only one of the values migrated
2021-02-18 10:55:32 +13:00
Naz
ea6d656457 🏗 Removed labs from setting
refs https://github.com/TryGhost/Team/issues/332

- The last value that has been used in the code was "members"
- By default members will be always "on" starting Ghost 4.0, so there's no need for this flag anymore
- Therefore there's no real need to keep "labs" around
2021-02-17 19:55:16 +13:00
Kevin Ansfield
cabf718ae1
Added migration to transform all post content urls from absolute to relative (#12658)
refs https://github.com/TryGhost/Team/issues/467
refs https://github.com/TryGhost/Team/issues/221

- we added robust url transformation in Ghost 3.0 which ensures newly created or edited content is stored with relative URLs in the database
- this migration loops over all posts (and posts_meta) in the database performing the absolute->relative transformation to normalise data for old posts
2021-02-16 19:00:04 +00:00
Fabien O'Carroll
037b2ce761 Renamed migration to fill 01
no-issue
2021-02-16 18:19:09 +00:00
Fabien O'Carroll
87471fd5c6 Revert "Updated plan_nickname column to be nullable"
This reverts commit 5d122c3c62.
2021-02-16 18:19:09 +00:00
Kevin Ansfield
3dfbb6296b Fixed duplicate migration number 17 2021-02-16 12:33:24 +00:00
Kevin Ansfield
66fe678cb7
Added 4.0 mobiledoc update migration (#12648)
refs https://github.com/TryGhost/Ghost/issues/12646

- adds `ghostVersion` property to mobiledoc of all posts so that we can pin the rendering output of existing content to protect against future breaking changes
- cleans any deprecated markdown card names from old Ghost 1.0 content
- cleans up HR cards that had an [accidental payload added](610d801bcf)
2021-02-16 12:13:30 +00:00
Fabien 'egg' O'Carroll
f4cb5c57c6
Updated members_status_events table (#12647)
refs https://github.com/TryGhost/Ghost/issues/12602

* Updated members_status_events table

By replacing the `status` column with a `from_status` and `to_status`
column, we are able to track the changes between multiple statuses
easier, and accumulate the data. e.g. the delta of paid members in a
given time range is the sum of the `to_status` columns set to 'paid'
minus the sum of the `from_status` columns set to 'paid' within that
time range

* Updated MEGA to handle addition of 'comped' status

With the addition of the 'comped' status, we need to ensure that MEGA
will still send emails to the correct recipients. I've opted to use an
"inverse" filter, as that is the intention of the free/paid split in
MEGA - as far as MEGA is concerned, "free" is the opposite of "paid"

* Updated customQuery for MemberStatusEvent

With the `status` column replaced with `from_status` and `to_status`
this allows us to fix and update the customQuery to correctly accumulate
the data into deltas over time, broken down by day.

* Populated members_status_events table

As the table will be used to generate deltas, we need to backfill the
data so that existing sites will be able to sum up the deltas and
calculate correct data.

The assumptions used in backfilling is that a Member's current status,
is their only status.
2021-02-16 10:38:36 +00:00
Fabien 'egg' O'Carroll
da9cd3b9d6
Updated status population to handle comped status (#12651)
refs https://github.com/TryGhost/Ghost/issues/12602

As part of collecting Member event data, we have added a third status
for members "comped" - this fixes the population of the column to handle
this
2021-02-16 10:37:06 +00:00
Fabien O'Carroll
8880cc82c7 Refactored usage of the pre v4 "slack" setting
refs https://github.com/TryGhost/Ghost/issues/10318

- Change/compatibility table for this changeset:

Importer

+ "slack_username"  import
- "slack_url" ignore
+ "slack.username" import
- "slack.url" ignore

v3 & canary/v4 Admin APIs
GET /settings/ (browse)

+ "slack_username" present in response
+ "slack_url" present in response
+ "slack" present in response

GET /settings/:settingName (read)

+ "slack_username" present in response
+ "slack_url" present in response
+ "slack" present in response

PUT /settings/ (edit)

+ "slack_username" updates setting
+ "slack_url" updates setting
+ "slack" !NOTE! updates setting unless "slack_username" or "slack_ur"l are set (updated keys take priority)

v2 Admin API
GET /settings/ (browse)

- "slack_username" NOT present in response
- "slack_url" NOT present in response
+ "slack" present in response

GET /settings/:settingName (read)

- "slack_username" ignored (404)
- "slack_url" ignored (404)
- "slack" present in response

PUT /settings/ (edit)

- "slack_username" ignored (no error)
- "slack_url" ignored (no error)
+ "slack" updates setting
2021-02-15 15:25:54 +13:00
Fabien O'Carroll
12c28fe66e 🏗 Updated "slack" setting into "slack_{url,username}"
refs https://github.com/TryGhost/Ghost/issues/10318

- Object format used in previous "slack" setting was considered an
anti-pattern. Flag structure of separate slack_url and slack_username
values was extracted out of the "slack" JSON.
2021-02-15 15:25:54 +13:00
Rishabh Garg
1d281581eb
Updated members_email_change_events to store previous email (#12641)
refs https://github.com/TryGhost/Ghost/issues/12602

As we only want to add events for when an email is changed, we need to store the previous email if we want a complete log of all emails for a member
2021-02-12 16:15:31 +00:00
Thibaut Patel
2f547b2974 Added the frontmatter column to meta
issue https://github.com/TryGhost/Team/issues/452
2021-02-12 15:31:04 +01:00
Fabien O'Carroll
905421b774 Populated members_paid_subscription_events table
refs https://github.com/TryGhost/Ghost/issues/12602

By backfilling the data, we allow exisiting sites to view their MRR over
time
2021-02-12 13:55:14 +00:00
Fabien 'egg' O'Carroll
f745fa7621
Updated members_paid_subscription_events table (#12636)
refs https://github.com/TryGhost/Ghost/issues/12602

This allows us to store the stripe_id of the subscription events
2021-02-11 12:45:18 +00:00
Fabien O'Carroll
5624e14b1b Renamed duplicate migration
no-issue
2021-02-11 11:02:18 +00:00
Fabien 'egg' O'Carroll
601d135b43
Added members event tables (#12618)
refs https://github.com/TryGhost/Ghost/issues/12602

* Added members_payment_events table

This table will store successful and unsuccessful payment attempts, and
can be used to calculate gross volume over time.

* Added members_login_events table

This table can be used to audit member logins

* Added members_email_change_events table

This table will allow us to store a history of email addresses associated with a member

* Added members_status_events table

This table will allow us to track the change in status over time for members, as well
as calculate aggregates over time, e.g. paid members over time

* Added members_paid_subscription_events

This table will allow us to track subscriptions changes for members, as well as 
calculating MRR over time
2021-02-11 10:54:30 +00:00
Naz
e0178099a2 🏗 Altered slug unique constraint in posts table
refs https://github.com/TryGhost/Ghost/issues/12567

- Changing unique constraint from slug to slug+type should allow for posts and pages to be created with the same slug
- The constraint will be present on application layer for API v4 while we figure out how to deal with it in API v5
2021-02-09 13:31:12 +13:00
Naz
47843bbfe9 Refactored migration to use transactional utility 2021-02-08 12:25:52 +13:00
Naz
d254a697b9 🏗 Cleanedup app-related settings
refs https://github.com/TryGhost/Ghost/issues/12565
refs f74d459dad

- These fields should have been cleaned up as a result of changed done in referenced commit
2021-02-08 12:25:52 +13:00
Fabien O'Carroll
065bbaeafa Populated members_subscribe_events table
refs https://github.com/TryGhost/Ghost/issues/12602

This allows us to backfill the data for testing purposes as well as for
use by existing sites. The mechanism for determining the date of an
unsubscribe event is crude, we use the date of the last update, which we
might expect to be _sometimes_ correct, but not always. We store the
`source` as NULL for unsubscribes as we cannot determine that with any
certainty whatsoever.
2021-02-05 16:02:08 +00:00
Fabien O'Carroll
e39644bd2d Added members_subscribe_events table
refs https://github.com/TryGhost/Ghost/issues/12602
2021-02-05 16:02:08 +00:00
Naz
606212d187 🏗 Dropped apps related tables
refs https://github.com/TryGhost/Ghost/issues/12565

- Dropped apps_* tables which were not used since the concept of "apps" has been abandoned
- Uses newly introduced "dropTables" migration utility
- Review for these changes has been completed in
(https://github.com/TryGhost/Ghost/pull/12592)
2021-02-03 12:20:08 +13:00
Fabien O'Carroll
5d122c3c62
Updated plan_nickname column to be nullable
no-issue

This value is nullable from the Stripe API so we're making sure that we
can store exact values locally.

SQLite3 does not supports altering columns so instead we have to:
1. Create a temp table and copy the data to it
2. Delete the original table
3. Recreate the original table with the necessary modifications
4. Copy the data from the temp table
5. Drop the temp table
2021-02-02 16:12:10 +00:00
Fabien 'egg' O'Carroll
a79ed1170d
Added status property to members (#12570)
refs #12160

This flag will allow us easier filtering of members via the API

* Added status column to members table

This flag will be used to determine if a member is free or paid, rather
than relying on joins with the customers and subscriptions tables.

* Added migration to populate members.status

As we add the column with a default value of "free" we only need to care
about the paid members here. We also preemptively handle migrations for
SQLite where there are > 998 paid members.
2021-02-02 16:11:34 +00:00