Commit Graph

8451 Commits

Author SHA1 Message Date
Fabien 'egg' O'Carroll
601d135b43
Added members event tables (#12618)
refs https://github.com/TryGhost/Ghost/issues/12602

* Added members_payment_events table

This table will store successful and unsuccessful payment attempts, and
can be used to calculate gross volume over time.

* Added members_login_events table

This table can be used to audit member logins

* Added members_email_change_events table

This table will allow us to store a history of email addresses associated with a member

* Added members_status_events table

This table will allow us to track the change in status over time for members, as well
as calculate aggregates over time, e.g. paid members over time

* Added members_paid_subscription_events

This table will allow us to track subscriptions changes for members, as well as 
calculating MRR over time
2021-02-11 10:54:30 +00:00
Daniel Lockyer
8e05a375ad Updated code to reflect js-yaml dependency changes
no issue

- as per
  https://github.com/nodeca/js-yaml/blob/master/migrate_v3_to_v4.md,
  `safeLoad` is now `load` and safe by default, so we can just switch to
  that
2021-02-10 11:39:30 +00:00
Fabien O'Carroll
4dd1423026 Added missing semicolon
no-issue
2021-02-10 10:14:29 +00:00
Fabien O'Carroll
9a142cd906 Added aggregateSubscriptionDeltas to permitted options
no-issue

This was missing from https://github.com/TryGhost/Ghost/commit/1dc6fdc.
Without it the option will never be passed into the customQuery method.
2021-02-10 10:06:06 +00:00
Naz
864fcd8a0c Removed settings populateDefaults pre-migration
refs https://github.com/TryGhost/Ghost/issues/12568
refs 5fbc40430b

- Having populateDefaults run before migrations creates a chicken and egg problem where populate defaults can create records that are "non-migratable" as happened in https://github.com/TryGhost/Ghost/issues/12026
2021-02-10 10:34:08 +13:00
Daniel Lockyer
94e9f60193
Updated Ghost-Admin to v4.0.0-alpha.1 2021-02-09 17:34:48 +00:00
Daniel Lockyer
1a370a3708
Fixed path to analytics-events in boot file
refs 0b79abf5b2

- in the referenced commit, we refactored the boot code to simplify it
- one of the required files was not updated with the change, so the path
  was wrong
- this wouldn't have been easily visible from local testing because it's
  based upon a config flag that isn't on by default
- this commit updates the path to the file
2021-02-09 12:51:06 +00:00
Sanne de Vries
67b1745bcb Fixed bug for newsletter bookmarks without author displaying redundant separator 2021-02-09 09:35:45 +01:00
Rish
1272968848 Updated {{content}} helper partial template name
no issue
refs e3a0bb535f

Previously, {{content}} helper was updated use default or custom template to show a CTA when trying to use helper without access to content.

While the change was expected to not affect any existing themes as `{{content}}` helper is not supposed to be used on member sites without `access` check, we found quite a few themes use a pattern of using `content.hbs` with `{{content}}` helper inside them as default, which causes infinite loop in how content helper works in certain cases.

Whilst this followed the pattern of the other helpers using a partial with the same name, there are 2 key differences:

- this partial template pattern is being introduced way after the inception of themes with a fairly generic name “content”
- this partial template isn’t used to render the helper all the time - just the CTA part under certain circumstances.

This change updates the template name to `content-cta.hbs` to which makes it less likely to clash, and makes more sense as to what it is.
2021-02-09 13:59:52 +05:30
Naz
e0178099a2 🏗 Altered slug unique constraint in posts table
refs https://github.com/TryGhost/Ghost/issues/12567

- Changing unique constraint from slug to slug+type should allow for posts and pages to be created with the same slug
- The constraint will be present on application layer for API v4 while we figure out how to deal with it in API v5
2021-02-09 13:31:12 +13:00
Sanne de Vries
118ceae29c Fixed gmail bug for newsletter bookmark cards not displaying correctly 2021-02-08 16:49:13 +01:00
Hannah Wolfe
0b79abf5b2 Added new, simpler, linear boot process
Background:
- Ghosts existing boot process is split across multiple files, has affordances for outdated ways of running Ghost and is generally non-linear making it nigh-impossible to follow
- The web of dependencies that are loaded on boot are also impossible to unpick, which makes it really hard to decouple Ghost
- With 4.0 we want to introduce a new, linear, simpler, clearer way to boot up Ghost to unlock decoupling Ghost into much smaller pieces

This commit:
- adds a new ghost.js file which switches between boot mode with `node index` or `node index old` so that if we find bugs we can work around them this week
   - Note: the old boot process will go away very soon, but ghost.js will remain as the interface between the command to start Ghost and the application code
- reworks the database migration process into a standalone utility, so that the DB is handled as one simple step of the boot process, decoupled from everything else
- is missing tests for this new db utility
- leaves a lot of work to do around loading core code, services, express apps in a sensible order, as work to fix this would start to break the old boot process
- doesn't use the new maintenance app because we aren't restarting the server here, instead we have the concept of a "core app" that starts in maintenance mode - need to think about how apps will be decoupled in the near future
2021-02-08 11:56:44 +00:00
Fabien O'Carroll
93b1035df5 🐛 Fixed Member model removing labels when unset
closes https://github.com/TryGhost/Ghost/issues/12600

The bookshelf-relations plugin which we use will **remove** all
relations when they are set to an empty array, but will leave them alone
if it's set to undefined.

Our logic to deduplicate uppercase & lowercase version of the same label
was in advertently always setting the labels to an array, but when the
model was saved without passing the labels, this array would be empty.

Here we've added a check which will skip all label handling, if there
are no labels set.
2021-02-08 09:39:46 +00:00
Naz
47843bbfe9 Refactored migration to use transactional utility 2021-02-08 12:25:52 +13:00
Naz
d254a697b9 🏗 Cleanedup app-related settings
refs https://github.com/TryGhost/Ghost/issues/12565
refs f74d459dad

- These fields should have been cleaned up as a result of changed done in referenced commit
2021-02-08 12:25:52 +13:00
Naz
7e7a94def3 Added a note to locale columns
refs https://github.com/TryGhost/Ghost/pull/12599

- These columns are reserved for future features and should not be dropped.
2021-02-08 12:09:23 +13:00
Hannah Wolfe
2b3c00ac32 Changed exit codes to be > 0
ref: https://tldp.org/LDP/abs/html/exitcodes.html

- exit codes being -1 is a mistake that got perpetuated, should be 1 or greater
- for now, just use 1
2021-02-05 20:13:50 +00:00
Fabien O'Carroll
1dc6fdcd66 Added customQuery to handle subscribed aggregates
refs https://github.com/TryGhost/Ghost/issues/12602

In order to build up a list of un/subscribes over time we have to use
the customQuery functionality to run SQL aggregates - this is "hidden"
behind an option, so that we can find{All,Page} as usual.
2021-02-05 16:02:08 +00:00
Fabien O'Carroll
3dbc7ef5b4 Passed MemberSubscribeEvent to members-api
refs https://github.com/TryGhost/Ghost/issues/12602

This allows the Members module to create MemberSubscribeEvents when
appropriate
2021-02-05 16:02:08 +00:00
Fabien O'Carroll
5aab41931f Added skeleton MemberSubscribeEvent model
refs https://github.com/TryGhost/Ghost/issues/12602

Member Events in general are read-only after they've been created, so
we've explicitly disallowed destroying and editing via the model
2021-02-05 16:02:08 +00:00
Fabien O'Carroll
065bbaeafa Populated members_subscribe_events table
refs https://github.com/TryGhost/Ghost/issues/12602

This allows us to backfill the data for testing purposes as well as for
use by existing sites. The mechanism for determining the date of an
unsubscribe event is crude, we use the date of the last update, which we
might expect to be _sometimes_ correct, but not always. We store the
`source` as NULL for unsubscribes as we cannot determine that with any
certainty whatsoever.
2021-02-05 16:02:08 +00:00
Fabien O'Carroll
e39644bd2d Added members_subscribe_events table
refs https://github.com/TryGhost/Ghost/issues/12602
2021-02-05 16:02:08 +00:00
Fabien O'Carroll
873d21d975 Updated addTable util to take optional tableSpec
no-issue

Migrations should contain all the necessary information rather than
relying on external data
2021-02-05 16:02:08 +00:00
Hannah Wolfe
256b77b0e4 Moved sqlite3 override rules to db connection lib
- We have an already established location for special rules based on db client
- Removes the need to require config inside of overrides
2021-02-05 11:51:56 +00:00
Hannah Wolfe
f6a7abdf5a Removed oembed-parser from overrides
- oembed-parser no longer depends on promise-wtf (it's now an archived project)
- therefore the reason for having this here is goooooone
2021-02-05 11:51:51 +00:00
Naz
f3e0949c73 Updated twitter/og structured data rules
refs https://github.com/TryGhost/Ghost/issues/10921
refs https://github.com/TryGhost/Ghost/pull/11068

- When there is no twitter or og image specified for a post or a page the new default falls back to appropriate site-wide twitter/og image or site's cover image.
- New rules of follback follow this logic:

post/page/amp = post.twitter_image || post.feature_image || settings.twitter_image || settings.cover_image;

post/page/amp = post.og_image || post.feature_image  || settings.og_image || settings.cover_image;
2021-02-05 18:50:11 +13:00
Naz
4b74230624 Refactored createTable migration utility
refs https://github.com/TryGhost/Ghost/issues/12567

- The method was using unneeded lodash dependency and was too complex for what it was doing
- Reshuffled internal code to use native JS filter/forEach iterators
2021-02-04 13:37:20 +13:00
Naz
6b61bcf123 Added unique constraint notation to schema definition
refs https://github.com/TryGhost/Ghost/issues/12567

- Introduced here @@UNIQUE_CONSTRAINTS@@ notation allows to create unique contraints over multiple database fields. This will be needed to change posts' table unique constraint from `slug` to `slug+type`.
- The notation is equivalent to SQL's: UNIQUE(column_name1,column_name2)
- Example use in schema:
posts: {
    slug: {type: 'string', maxlength: 191, nullable: false},
    type: {type: 'string', maxlength: 50, nullable: false, defaultTo: 'post', validations: {isIn: [['post', 'page']]}},
    '@@UNIQUES@@': [
        ['slug', 'type']
    ]
}
2021-02-04 13:37:20 +13:00
Kevin Ansfield
7195a904ba Fixed settings validation error not distinguishing between publishable/secret keys
no issue

- when saving Stripe keys with `stripeDirect: true` config, if either key didn't match the key format the returned validation error always contained `stripe_secret_key`
- updated to output `stripe_publishable_key` if it was the publishable key that was invalid
2021-02-03 16:42:51 +00:00
Thibaut Patel
7d4596adf6 Migrate the default api version from v2 to v3
issue https://github.com/TryGhost/Team/issues/221
2021-02-03 15:02:51 +01:00
Rish
b0ea919e42 Bumped portal to 1.0.0-rc
no refs
2021-02-03 08:43:11 +05:30
Naz
606212d187 🏗 Dropped apps related tables
refs https://github.com/TryGhost/Ghost/issues/12565

- Dropped apps_* tables which were not used since the concept of "apps" has been abandoned
- Uses newly introduced "dropTables" migration utility
- Review for these changes has been completed in
(https://github.com/TryGhost/Ghost/pull/12592)
2021-02-03 12:20:08 +13:00
Daniel Lockyer
fc84bb8d8d
Updated Ghost-Admin to v4.0.0-alpha.0 2021-02-02 16:19:39 +00:00
Daniel Lockyer
0958654457
Updated migration warn message
no issue

- fixed typo and improved message formatting
2021-02-02 16:12:53 +00:00
Naz
96f6ca6218
Fixed drop tables migration utility
refs b5b65b3a60

- The utility was meant to be "irreversible" instead of "transactional"
2021-02-02 16:12:53 +00:00
Naz
23c5ca0795
Added documentation and JSDoc to migration commands
refs https://github.com/TryGhost/Ghost/issues/12567

- Functions were missing documentation and were able to perform operations on multiple columns instead of single one after checking the knex docs (http://knexjs.org/#Schema-unique)
2021-02-02 16:12:53 +00:00
Naz
bf1fcd25ce
Added migration utils for dropping tables
refs https://github.com/TryGhost/Ghost/issues/12565

- Dropping tables happens sporadicaly, usually during major version releases. It made sense to create a utility based on previous migrations of this type (e.g.:  3.0/03-drop-client-auth-tables.js migration) to avoid code duplication in the future
2021-02-02 16:12:53 +00:00
Fabien 'egg' O'Carroll
73f6fd8c51
Enabled Members for all sites (#12582)
no-issue

This removes all references to the members labs setting, any code that was run conditionally behind this flag now runs unconditionally. 

* Removed usage of Members labs flag
* Removed tests for Members disabled
* Added dynamic keypair generation for when setting is missing
2021-02-02 16:12:53 +00:00
Rishabh Garg
26ee648397
🏗 Flattened members subscriptions data in API (#12581)
no refs

- Updates member model serializer to directly set subscriptions on member object instead of `stripe.subscriptions`
- Updates all references to members subscriptions from nested `stripe.subscriptions` to `subscriptions`
- Updates v3 API serializer to still use `stripe.subscriptions`
- Updates tests
2021-02-02 16:12:53 +00:00
Fabien 'egg' O'Carroll
6af2706f10
Updated Admin API and Mega to use status flag (#12579)
no-issue

* Removed support for paid param from v3 & canary API
* Updated active subscription checks to use status flag
* Updated MEGA to use status filter over paid flag
* Removed support for paid option at model level
* Installed @tryghost/members-api@1.0.0-rc.0
* Updated members fixtures
2021-02-02 16:12:51 +00:00
Fabien 'egg' O'Carroll
229295d671
Exited process when members-api fatally errors (#12489)
closes https://github.com/TryGhost/Ghost/issues/12448

This adds handling to force the process to exit when we receive an error
from the members-api on initialisation.
2021-02-02 16:12:10 +00:00
Thibaut Patel
8ecaeb4af9
💡 Renamed the cookie used for the private mode
issue https://github.com/TryGhost/Ghost/issues/12564

- This change means users who have already entered the site password will need to do it again
2021-02-02 16:12:10 +00:00
Fabien O'Carroll
5d122c3c62
Updated plan_nickname column to be nullable
no-issue

This value is nullable from the Stripe API so we're making sure that we
can store exact values locally.

SQLite3 does not supports altering columns so instead we have to:
1. Create a temp table and copy the data to it
2. Delete the original table
3. Recreate the original table with the necessary modifications
4. Copy the data from the temp table
5. Drop the temp table
2021-02-02 16:12:10 +00:00
Fabien O'Carroll
cc17de16ba
Added ability to specific table definition to createTable
no-issue
2021-02-02 16:12:10 +00:00
Naz
e33599432d
Fixed maintenance app initialization 2021-02-02 16:12:10 +00:00
Naz
9bff415572
Moved maintenance app to separate package
refs #12568

- Made a package out of maintenance app to keep as little code in the repository
2021-02-02 16:12:08 +00:00
Naz
dcb89f2be0
Changed maintenance app debug code 2021-02-02 16:11:35 +00:00
Naz
244e2374d3
Refactored maintenance app to be a class with DI
refs #12568

- DI pattern through constructor allows for very clear separation of concerns and makes it easy to extract the class into a separate module
2021-02-02 16:11:35 +00:00
Naz
7d9f056e9b
Added error handling and response formatting in maintenance 2021-02-02 16:11:35 +00:00
Naz
d60e36ab57
Added "maintenance" app serving 503 page
refs #12568

- When the server is in maintenance mode (e.g. during migration) it needs to serve a 503 page. Previously this role was delegated to the "frontend" which tightly coupled server bootup and frontend.
- With a dedicated HTTP application serving up 503 server is no longer coupled to the frontend during maintenance
2021-02-02 16:11:35 +00:00