Commit Graph

1133 Commits

Author SHA1 Message Date
Naz
77a5ea5659 Added JSDoc to addTable method
no issue

- This mehod has an important `tableSpec` parameter which MUST be present when creating a new table migration. Having a description in form of the JSDoc somewhat helps this cause
- Next best improvement would be throwing an error if the parameter wasn't present, but that would require a bigger refactor backporting all usages of `addTable` method
2021-07-14 11:39:23 +04:00
Hannah Wolfe
c29c118fcf
Moved labs utlity to shared
- This isn't really a "service" - it's a set of utilities for working with labs flags
- It's also required all over the place, and doesn't require anything that isn't shared
- Therefore, it should live in shared
2021-07-08 09:05:41 +01:00
Hannah Wolfe
8d38957bd7
Revert "Moved labs utlity to shared"
This reverts commit 782de52678.
2021-07-08 07:09:13 +01:00
Hannah Wolfe
782de52678
Moved labs utlity to shared
- This isn't really a "service" - it's a set of utilities for working with labs flags
- It's also required all over the place, and doesn't require anything that isn't shared
- Therefore, it should live in shared
2021-07-07 21:41:34 +01:00
Fabien O'Carroll
fa137ed16e Added support for the 'comped' status for members
refs https://github.com/TryGhost/Team/issues/790

The schema validations are used at the model layer to validate inputs
and need to be updated in order for us to reintroduce the 'comped'
status.
2021-07-06 12:00:25 +01:00
Fabien O'Carroll
8a87eb9e36 Migrated members_status_events for comped members
refs https://github.com/TryGhost/Team/issues/790

Since version 4.6 the 'comped' status has not been used. Any members
which were given complimentary plans since then will have had a `status`
of 'paid', and therefore the corresponding members_status_events row
would have a `to_status` of 'paid'.

This migration is designed to fix these members_status_events rows by
ensuring that the last (chronologically) members_status_event row for a
comped member has a to status of 'comped'.

Unfortuantely this migration loses information which makes writing a
perfect inverse migraion impossible. Alternative down migrations were
considered, but these would lose further information.
2021-07-06 12:00:03 +01:00
Fabien O'Carroll
47cf21514e Migrated comped members to 'comped' status
refs https://github.com/TryGhost/Team/issues/790

In order to track when a member was comped, as well as to differentiate
paid members from comped, we are reintroducing the 'comped' status. This
migration will updated members with a Complimentary Stripe Subscription
to a status of 'comped'. It is essentially a reversal of the 4.6
migration.
2021-07-06 12:00:03 +01:00
Kevin Ansfield
541fb4d2db
🐛 Fixed potential for partial content re-generation in 4.0 migrations (#13120)
no issue

- incorrect syntax was used in the error handlers inside of the `for` loop, by using `return` when logging the whole for-loop was aborted whereas we want to log and continue processing the rest of the items
2021-07-06 10:15:32 +01:00
Kevin Ansfield
05b317af9d
🐛 Fixed incorrectly stored URLs after migration from 3.x to 4.6.1-4.8.4 (#13109)
refs https://github.com/TryGhost/Team/issues/853

A refactor of `urlUtils` usage in 4.6.1 left a buggy 4.0 migration that did not transform URLs inside of mobiledoc cards. Anyone upgrading from 3.x to 4.6.1-4.8.4 would end up with inconsistent URL formats and potentially broken images.

- fixed 4.0 migration by passing our mobiledoc cards list in when transforming mobiledoc urls
- added a new migration that re-applies the missed URL transforms and content re-generation for any site that did a 3.x upgrade to a buggy 4.x version
2021-07-06 10:15:00 +01:00
Hannah Wolfe
bd597db829
Moved settings/cache to shared/settings-cache
- This is part of the quest to separate the frontend and server & get rid of all the places where there are cross-requires
- At the moment the settings cache is one big shared cache used by the frontend and server liberally
- This change doesn't really solve the fundamental problems, as we still depend on events, and requires from inside frontend
- However it allows us to control the misuse slightly better by getting rid of restricted requires and turning on that eslint ruleset
2021-06-30 15:49:10 +01:00
Thibaut Patel
a792b92e94 Added member segment to email batches
issue https://github.com/TryGhost/Team/issues/826
2021-06-30 14:56:20 +02:00
Fabien O'Carroll
b894b2a913 Added products_benefits table
refs https://github.com/TryGhost/Team/issues/804

The associative table is used to implement the many-to-many relationship
between Products and Benefits. The `sort_order` column is needed because
a product's benefits should be orderable by an admin.
2021-06-24 17:21:01 +01:00
Fabien O'Carroll
65b2e8a0a5 Added benefits table
refs https://github.com/TryGhost/Team/issues/804

Benefits are tag-like resources which will be associated with Products.
The first iteration just requires a name for the benefit, which will be
stored as plaintext.
2021-06-24 17:21:01 +01:00
Thibaut Patel
dc91eabe3c Moved the "Reset all passwords" permission migration to Ghost 4.9
commit f12f64e87b
2021-06-23 17:08:44 +02:00
Thibaut Patel
f12f64e87b
🔒 Added a "reset all passwords" feature (#13005)
issue https://github.com/TryGhost/Team/issues/750

- Only accessible by admins
- Resets all staff users' passwords and prevents them to log-in
- Sends them a reset email password to give them back access to their account
- Closes all existing staff user sessions
2021-06-23 14:54:28 +02:00
Kevin Ansfield
446993a905
Added additional newsletter customisation settings (#13074)
refs https://github.com/TryGhost/Team/issues/793

New settings added for newsletter customisation options:

- `newsletter_header_image` - `null/"$url"`
- `newsletter_show_header_icon` - `"true/false"`
- `newsletter_show_header_title` - `"true/false"`
- `newsletter_title_alignment` - `"center/left"`
- `newsletter_title_font_category` - `"serif/sans_serif"`
- `newsletter_show_feature_image` - `"true/false"`

`newsletter_show_header` has been dropped because the same functionality can be achieved by setting both `newsletter_show_header_icon` and `newsletter_show_header_title` to `false`

---

- migration to convert and delete `newsletter_show_header` setting
- removed `newsletter_show_header` from default settings to ensure it doesn't get re-created
- replaced main labs template and template settings generation with the labs template
- deleted labs template
2021-06-21 13:40:40 +01:00
Rishabh Garg
8f104f67b1
Added new portal_products setting (#13055)
refs https://github.com/TryGhost/Team/issues/768

- `portal_products` stores list of products available in Portal
- adds new `portal_products` setting to default settings
- adds migration to populate `portal_products` with current product so its available by default
- update tests
2021-06-21 14:01:50 +05:30
Sam Lord
24332c3d24 Replaced ghost-version.js with @tryghost/version
no issue
Part of the effort to break up the Ghost codebase into smaller, decoupled modules.
2021-06-16 13:16:15 +01:00
Hannah Wolfe
526993965a
Switch to @trghost/validator, remove validator
- Part of the effort to split Ghost down into smaller, decoupled pieces
- Moved out our internal validator tooling to a separate library
- Replaced all usage of our own tooling and validatorjs directly with @tryghost/validator
- Removed the validatorjs dependency and removed the renovate pin
- This gives us a consistant, smaller, clearer public API for validations
- It will eventually be used on Ghost Admin too
- This way we can start getting up to date with validator whilst not increasing build size
2021-06-16 08:11:22 +01:00
Sam Lord
35e51e364b Switch to @tryghost/debug, remove ghost-ignition
no issue
The only pieces of Ghost-Ignition used in Ghost were debug and
logging. Both of these modules have been superceded by the Framework
monorepo, and all usages of Ignition have now been removed, replaced
with @tryghost/debug and @tryghost/logging.
2021-06-15 17:24:22 +01:00
Sam Lord
caea330647 Change to use @tryghost/logging
no issue

Logging is now controlled by a logginrc.js file in the root of the project - and now we can just import @tryghost/logging everywhere
2021-06-15 15:59:11 +01:00
Hannah Wolfe
0fe8426f97
Renamed validation to validator + better public API
- renamed our internal validation library to "validator" - which is the same as the tool it wraps
- updated the public api so that validator methods are directly exposed
- this will make it a drop-in replacement for validator-js
- in turn, this allows us to pull this out into @tryghost/validator, and use our own wrapper instead of the 3rd party library
2021-06-15 15:32:36 +01:00
Hannah Wolfe
1688b17c49
Refactored + cleaned up validation tools
- General code cleanup
- Removed unused notContains rule
- Swapped custom empty rule for builtin isEmpty rule
- Dropped usage of .extend on validator, as this was removed 2 years ago!
   - This will allow us to upgrade the validator dependency to a much newer version
- Changed our internal validator module to only expose the functions we use.
   - This gives us a clearer Public API
   - It makes it easier to see if we are affected by changes in validator
   - It's still easy to add another validator, we just have to update what we require
   - We can potentially use this to make smaller builds esp for client-side usage
   - Once ripped out into a module we can use ES imports :D
- Rejigged and _slightly_ improved the tests
2021-06-15 15:17:20 +01:00
Hannah Wolfe
4e49aeb9a0
Moved password validation into a library
- This is super specific code relating only to validating passwords.
- It's needed as a shared validator as we use other funnels to help people setup Ghost on Pro, but currently it's hard-baked into Ghost
- It's also not the greatest code. It'd be nice to be able to rework it and know that would automatically update everywhere passwords are set
2021-06-15 12:33:14 +01:00
Hannah Wolfe
d3cc85c920
Moved schema validator into the schema module
- This is a really specific piece of code related to validating models against our internal schema.js format
- This doesn't make sense without a schema.js file
- It does depend on the internal validator and validate tools - but those are used elsewhere too, and can reasonably be moved out of the codebase
- I don't see schema.js moving out of the codebase any time soon. We can move the validator but it would be a class that requires schema via DI
- For now my focus is on getting the data/validation tooling separated and making clear sense
- Improving data/schema can come later :)
2021-06-15 11:54:34 +01:00
Hannah Wolfe
68d60a1834
Removed i18n from validation modules 2021-06-15 11:30:37 +01:00
Hannah Wolfe
c37de311ea
Split validation index into separate files
- The data/validation module is made up of several loosely related things with lots of dependencies
- Separating out the various components makes it possible to see what's what, and importantly what has complex dependencies
- validator + validate probably go togetheri in an external module, the other two files should probably have their own homes in related areas of ghost e.g. schema -> data/schema/validate.js
2021-06-14 18:18:28 +01:00
Daniel Lockyer
e3ea40e268
Fixed JSDoc for DB helper functions
no issue

- the docs were referring to different parameter names so this commit
  bring them in line to make the warnings go away
2021-06-11 16:03:42 +01:00
Kevin Ansfield
1bc57b584a
Added posts_meta.feature_image_{alt,caption} columns (#13030)
refs https://github.com/TryGhost/Team/issues/770

We want post feature image functionality to better match what's available inside the editor, to do that we'll need somewhere to store alt and caption meta data. `posts_meta` chosen because even though we want to make this generic for other tables in the future those tables also have a `feature_image` (or closely related) field.

- updated schema with new columns
- added migration to create columns
- cleaned new columns from API output
  - not output on v2/v3
  - conditionally output on v4/canary output based on labs flag
- bumped `@tryghost/admin-api-schema` to allow new columns through in canary API requests
  - silently clean properties from input when labs flag is disabled
  - updated acceptance tests so they fail if `admin-api-schema` is not letting the new fields through
2021-06-10 20:35:56 +01:00
Kevin Ansfield
f49f7699aa
Enabled all labs flags when testing (#13036)
no issue

Shows impact of new code behind labs flags through the existing acceptance/regression tests. Allows for existing tests to be updated to match new behaviour rather than requiring separate tests where individual flags are enabled. Should result in minimal test updating once code reaches GA.

- adds a forced `'labs:enabled'` fixture op that edits the `labs` setting to enable all flags then restarts the settings service to pick up the new setting
- modifies labs service to not remove ALPHA_FEATURE labs settings when running in a testing environment
2021-06-10 12:54:34 +01:00
Kevin Ansfield
485a50794e Added extra newsletter settings
refs https://github.com/TryGhost/Team/issues/755

Additional customisation settings for newsletter emails are being introduced. Defaults match the existing email design.

- adds new settings
  - `newsletter_header_image` - stores URL from image upload
  - `newsletter_show_header_icon` - boolean toggle, default `true`
  - `newsletter_show_header_title` - boolean toggle, default `true`
  - `newsletter_show_feature_image` - boolean toggle, default `true`
  - `newsletter_title_alignment` - "left" or "center" selection, default "center"
  - `newsletter_title_font_category` - "serif" or "sans_serif" selection, default "sans_serif" (matches body font category selection)
2021-06-07 16:31:50 +01:00
Naz
d8230f3343 Added labs setting import validation
refs https://github.com/TryGhost/Team/issues/757

- Similarly to the API input serrialization importer should not pass through unknown labs keys to avoid soiling data
2021-06-08 02:18:17 +12:00
Naz
49ba26373d 🏗 Added "labs" setting enabling feature flags
refs https://github.com/TryGhost/Team/issues/757
refs https://github.com/TryGhost/Team/issues/332
refs ea6d656457

- We have a need a quick way to add features behind flags. The old way of "labs" is the quickest way to achieve this. It has ready tooling around it and well understood pitfalls. This change reintroduces "labs" group & key in settings table in the same shape it used to be (see reffed commit)
- Next step will be introducing very basic guard rails to protect from pitfalls previous implementation of "labs" had. This will include an allowlist based input validation for lab's object's data
-  The labs being an "object" type is an EXCEPTION. Even though it's an antipattern we aim to move away from, for now it's the lowest impact solution that will unblock the use of flags in the system. A proper solution will come at some point.
2021-06-08 02:18:17 +12:00
Fabien 'egg' O'Carroll
bdd7c0f821
Added {monthly,yearly}_price columns to products
refs https://github.com/TryGhost/Team/issues/710
refs https://github.com/TryGhost/Team/issues/725

Products will now have a single monthly and yearly price which will be
used throughout Themes, Portal & Admin. These columns will be used to
track the current prices for each of them, and will update anytime we
change the pricing of a product.

Due to a circular table dependency we have not added a foreign key
constraint to the new columns, this will be handled at a later date. It
is tracked in issue 725 references above
2021-06-01 09:38:15 +01:00
Kevin Ansfield
c2793eedd3 Fixed no-restricted-require warning for core/shared/url-utils.js
no issue

Our server-defined `mobiledoc` object was required by `UrlUtils.cardTransformers` property to help set up a shortcut for url transform functions but that was breaking the independence of `UrlUtils` by crossing the shared/server boundary.

- `cardTransformers` is only needed for the `mobiledocToTransformReady` utility function that will only be used by the server
- removed `UrlUtils.cardTransformers` (and associated require) from our `UrlUtils` instance and updated the few areas the server uses `mobiledocToTransformReady()` to pass in the mobiledoc card objects directly as an option
2021-05-25 21:13:57 +01:00
Fabien O'Carroll
730811fb6e Handled week & day interval in population of mrr events
refs https://github.com/TryGhost/Team/issues/635

This is to ensure we don't break migrations for any sites which have
imported external subscriptions which have an interval of 'week' or
'day'

The bump to members-api includes the handling of these intervals for
ongoing population of mrr events
2021-05-21 17:57:17 +01:00
Naz
459e8215b9 Split data exporter's index file into separate module
refs https://github.com/TryGhost/Team/issues/610

- This is a next step removing bloat from export module's index.js file, which get's rid of eslisn error completely
2021-05-21 19:13:55 +04:00
Naz
79439bdac5 Extracted table allowlists into a separate module
refs https://github.com/TryGhost/Team/issues/610

- Before introducing a new test for the refed issue doing a linting cleanup. The result will be removing one of `File has too many lines ` lint warnings
2021-05-21 19:13:55 +04:00
Fabien O'Carroll
5880edd722 Replaced members 'comped' status with 'paid'
refs https://github.com/TryGhost/Team/issues/693

Since we've got rid of the concept of Complimentary with the Custom
Prices work, we're removing the 'comped' status from members. This
involves a migration for existing members, a schema update for the
validation, and a bump to members-api to no longer use the 'comped'
status for new members.

We also update the aggregation of the MemberStatusEvent to consider the
'comped' status as 'paid', and that there are 0 'comped' status events
in the database.

We can consider a migration for this data in the future, either adding
new status events moving from 'comped' to 'paid', or by modifying
existing status events. However both of these are very difficulty to
write a down migration for, and might be best saved for a major version.

- @tryghost/members-api@1.7.0 is the version that includes the required
  changes, however we have already bumped to 1.8.0 in Ghost
2021-05-19 18:49:18 +01:00
Rishabh
4627d1c26a Added settings for monthly/yearly price ids
no refs

Since backend now allows multiple prices but we want the prices to be currently limited to monthly/yearly on UI, we need new settings to store the current monthly/yearly price by the site owner. These settings determine the active prices shown in Admin / Portal for the site till we allow all custom products/prices again.
2021-05-18 13:34:31 +05:30
Kevin Ansfield
c36e749820
Added support for gating content by member labels and products (#12946)
refs https://github.com/TryGhost/Team/issues/581
closes https://github.com/TryGhost/Team/issues/582

Emails can now be sent to members with specific associated labels or products by specifying an NQL string. We want to bring the same members segment feature to content by allowing `visibility` to be an NQL filter string on top of the `public/members/paid` special-case strings.

As an example it's possible to set `posts.visibility` to `label:vip` to make a post available only to those members with the `vip` label.

- removed enum validations for `visibility` so it now accepts any string or `null`
    - bumped `@tryghost/admin-api-schema` for API-level validation changes
- added nql validation to API input validators by running the visibility query against the members model
- added transform of NQL to special-case visibility values when saving post model
    - ensures there's a single way of representing "members" and "paid" where NQL gives multiple ways of representing the same segment
    - useful for keeping theme-level checks such as `{{#has visibility="paid"}}` working as expected
- updated content-gating to parse nql from post's visibility and use it to query the currently logged in member to see if there's a match
    - bumped @tryghost/members-api to include label and product data when loading member
2021-05-10 19:32:11 +01:00
Rishabh
cfaddf82e8 Added migration to rename product as site title
refs https://github.com/TryGhost/Team/issues/667

On clean and existing installs, the default product created should be named the same as the site title in the first setup so the UX on Portal and everywhere is consistent. This change adds a migration to update existing sites which already have a default product created via fixture, and rename them to their current site title. The rename is only done if the Product name is still the same as in fixture - `Default Product`.
2021-05-11 00:00:12 +05:30
Rishabh
c199918465 Updated free price default description
refs https://github.com/TryGhost/Team/issues/648

We use sentence case everywhere for labels, the default description for free price should be the same
2021-05-10 18:24:36 +05:30
Rishabh
2497af0072 Updated default description for free price
refs https://github.com/TryGhost/Team/issues/671

When turning on custom products, existing sites should have default price descriptions that match existing values for prices. This change sets the default description for Free price to match existing hardcoded value.
2021-05-10 14:19:02 +05:30
Fabien O'Carroll
6106d1fdc4 Updated active column type from boolean -> bool
no-issue

Our base model will only automatically convert numbers to booleans if
the type is 'bool' - however this column was incorrectly added with a
type of 'boolean'. Lucklily - knex with both MySQL & SQLite3 will add
a column with the same type for both of these, so no migration is needed
to fix it.
2021-05-07 16:37:38 +01:00
Fabien O'Carroll
b35f060bef Added settings for members 'free' price
refs https://github.com/TryGhost/Team/issues/637

The "free" price - when Members signup without using Stripe, should have
a name and description, so that it can be displayed in Portal in a
similar way to paid price's. As there is only ever one, and it is not a
fully fledged price, a setting makes more sense than a dedicated db
table.
2021-05-07 16:32:12 +01:00
Fabien O'Carroll
a138586c83 Updated stripe_plans & portal_plans settings
refs https://github.com/TryGhost/Team/issues/586

We are no longer using the `stripe_plans` setting, instead we are using
the `stripe_prices` database table. However, we must keep the setting as
the migration from the setting to the database is not done as a standard
migration, but in code. This means our code has to still read and pass
the setting because we will never know if the migration in code has run
yet.

The `portal_plans` setting has been updated to only include 'free' by
default, because the setting must include id's now rather than names.
2021-05-07 16:31:08 +01:00
Fabien 'egg' O'Carroll
e9687d359d
Removed unique constraint from Product names (#12928)
refs https://github.com/TryGhost/Team/issues/586

A discussion in the Members team resulted in us determining that we do
not need to enforce unique names for Products. Stripe does not enforce
uniqueness for their Products, and we feel it's not necessary for us to.
2021-05-07 13:34:23 +01:00
Kevin Ansfield
322664a145
Added ability to send a newsletter to members with a certain label or product (#12932)
refs https://github.com/TryGhost/Team/issues/581
refs https://github.com/TryGhost/Team/issues/582

When publishing a post via the API it was possible to send it using `?email_recipient_filter=all/free/paid` which allowed you to send to members only based on their payment status which is quite limiting for some sites.

This PR updates the `?email_recipient_filter` query param to support Ghost's `?filter` param syntax which enables more specific recipient lists, eg:

`?email_recipient_filter=status:free` = free members only
`?email_recipient_filter=status:paid` = paid members only
`?email_recipient_filter=label:vip` = members that have the `vip` label attached
`?email_recipient_filter=status:paid,label:vip` = paid members and members that have the `vip` label attached

The older `free/paid` values are still supported by the API for backwards compatibility.

- updates `Post` and `Email` models to transform legacy `free` and `paid` values to their NQL equivalents on read/write
  - lets us not worry about supporting legacy values elsewhere in the code
  - cleanup migration to transform all rows slated for 5.0
- removes schema and API `isIn` validations for recipient filters so allow free-form filters
- updates posts API input serializers to transform `free` and `paid` values in the `?email_recipient_filter` param to their NQL equivalents for backwards compatibility
- updates Post API controllers `edit` methods to run a query using the supplied filter to verify that it's valid
- updates `mega` service to use the filter directly when selecting recipients
2021-05-07 11:56:41 +01:00
Kevin Ansfield
1ee97ccfbc
Fixed editors not being able to read labels when assigning member segments (#12934)
refs https://github.com/TryGhost/Team/issues/581

Editors are allowed to restrict post visibility and send emails to particular member segments, they need to be able to read labels so that they can select them in a member segment.
2021-05-07 10:51:44 +01:00
Kevin Ansfield
b08b95085e
Added default email recipients settings (#12925)
refs https://github.com/TryGhost/Team/issues/496

We want to give more control over the default selection of email recipients when publishing a post, to do that we need somewhere to store those settings. These settings are site-wide and intended for use by admins to control the default editor behaviour for all staff users. They _do not_ control API behaviour, if you want to send email when publishing via the API it's still necessary to explicitly opt in to that using the `?email_recipients_filter=` query param.

- new `editor` settings group to indicate that these settings only affect the UI rather than the API
- `editor_default_email_recipients` controls overall behaviour, string/enum with these allowed values:
  - `'disabled'`: no option to send email is shown in the editor's publishing dropdown
  - `'visibility'`: (default) selected member segment is dynamic and matches the post visibility filter
  - `'filter'`: specific member filter defined in `editor_default_email_recipients_filter` setting
- `editor_default_email_recipients_filter` is an NQL string for selecting members, used when `editor_default_email_recipients` is set to `'filter'`
  - default value is `'all'`
  - the segment string can be any valid NQL filter with the additional special-case values of `'all'` and `'none'`
2021-05-07 09:30:57 +01:00
Hannah Wolfe
814a0054e7 Improved single author deprecated notices
- single authors were deprecated in v1.22 when we added multiple authors
- we always thought we'd clean this up a lot sooner, but it's stuck because it's an annoying thing to break people's shit over
- still saying "remove in vX" isn't useful, we need to know how long a feature has been deprecated so we can judge whether it's safe to remove
2021-05-04 17:11:12 +01:00
Fabien 'egg' O'Carroll
b4d9ee0b4e
Added description to Products & Prices (#12916)
refs https://github.com/TryGhost/Team/issues/586

The `products` and `stripe_prices` tables are missing a description
column which will be used by Portal to display information about the
products and prices
2021-05-04 13:41:48 +01:00
Hannah Wolfe
273e220327 Moved i18n to shared
refs 829e8ed010

- i18n is used everywhere but only requires shared or external packages, therefore it's a good candidate for living in shared
- this reduces invalid requires across frontend and server, and lets us use it everywhere until we come up with a better option
2021-05-04 13:03:38 +01:00
Kevin Ansfield
65862b437e
Migrated 'invite' to 'all' for sites that were not truly invite-only (#12907)
refs https://github.com/TryGhost/Team/issues/579

The previous `members_allow_free_signup` -> `members_signup_access` migration made a direct correlation between the toggle `true/false` to `all/invite` under the assumption that behaviour between the two settings would be identical. The assumption was incorrect and the behaviour is changing so `invite` forces invite-only mode, stopping all front-end signup to free or paid plans with the free plan now being disabled via the portal plans setting.

- check existing `members_signup_access` setting and if it's `'invite'` migrate it to `'all'` where signup should still be possible. The "invite-only" mode should only be active if certain conditions are met:
  - Stripe is not configured ("allow free member signup" off and no Stripe showed "invite-only" in portal)
  - Stripe is configured but no plans are selected in portal (no plans showed "invite-only" in portal)
- when migrating `'invite'` to `'all'`, also remove `'free'` plan from the `portal_plans` setting to avoid previously paid-only sites unexpectedly showing a free plan on signup
2021-05-04 09:50:29 +01:00
Kevin Ansfield
0f41d1aa49
🐛 Restored correct "allow free signup" setting from backup after buggy 4.3 upgrade (#12905)
refs https://github.com/TryGhost/Team/issues/634

- find earliest backup file created when a 4.3 migration was run, if found use the `members_allow_free_signup` value from there to change `members_signup_access` from `'all'` to `'invite'` if necessary
2021-05-04 09:49:33 +01:00
Hannah Wolfe
829e8ed010 Expanded requires of lib/common i18n and events
- Having these as destructured from the same package is hindering refactoring now
- Events should really only ever be used server-side
- i18n should be a shared module for now so it can be used everywhere until we figure out something better
- Having them seperate also allows us to lint them properly
2021-05-03 17:14:52 +01:00
Kevin Ansfield
10b6fbfc82
🐛 Fixed Ghost 4.3.0 migration that put all sites into "allow free members signup" (#12904)
refs https://github.com/TryGhost/Team/issues/634

- the migration moving `members_allow_free_signup` to `members_signup_access` was expecting a raw boolean setting value but the actual value is a string so always evaluated as truthy making all sites look like they had "allow free members signup" toggled on when generating the new setting's value
- updated to check for an explicit string value in `up` and set an explicit string value in `down`
2021-04-27 15:37:41 +01:00
Daniel Lockyer
e9b21fdbd1 Updated bson-objectid calls to match API change
refs c873899e49

- as of `bson-objectid` v2.0.0, this library exports the function
  to generate an ObjectID directly, and then you need to use `.toHexString()`
  to get the 24 character hex string - 6696f27d82
- this commit removes all uses of `.generate()` and replaces with this
  change
2021-04-21 16:23:52 +01:00
Fabien 'egg' O'Carroll
5b936c052f
Added Default Product fixture (#12892)
refs https://github.com/TryGhost/Team/issues/619

As part of the Custom Products work, we are linking members to products
when updating their subscriptions. This requires that we have at least
one product in the database. For existing sites that are using Members
this is handled by the v4.3 03 migration. But for new sites we must
include a fixture.

Also fixes the tests to not reply on the order of the fixtures
2021-04-20 16:39:27 +01:00
Rishabh Garg
48a2d24497
Added stripe_price_id column to subscriptions table (#12881)
refs https://github.com/TryGhost/Team/issues/586

- Adds new `stripe_price_id` column to subscriptions table to store stripe price ids with `index`
- Populates `stripe_price_id` column value to current `plan_id` making the `plan_*` values redundant
- Updates tests
2021-04-20 16:37:59 +05:30
Rishabh Garg
5da4ae90b2
Added cascade delete for stripe products and prices (#12891)
refs https://github.com/TryGhost/Team/issues/586

If a product inside Ghost is deleted, we want to cascade delete all associated Stripe products and prices as they always need to refer back to a ghost product and will hang without any reason otherwise. This change adds cascade delete for products -> stripe_products -> stripe_prices to avoid broken states
2021-04-20 15:24:52 +05:30
Kevin Ansfield
fff6a04c54
Migrated members_allow_free_signup setting to members_signup_access (#12886)
refs https://github.com/TryGhost/Team/issues/579

Currently the members signup setting is explicitly yes/no to allowing free members signup, with the implication that when set to "no" members is still active but members have to be created via Stripe or the admin API.

This change renames the setting and changes its type to allow more than a binary option.

- migration to create/update the new setting based on the old value
  - free signup = "all", no free signup = "invite"; matches the current UI for this setting
- rename setting everywhere it's used/tested against
- modify `getAllowSelfSignup()` used to configure members packages to only return `true` when the new setting is set to `'all'` to match behaviour to the older setting
- update importer to rename the setting when importing from an older Ghost version
2021-04-19 16:36:30 +01:00
naz
d964224312
Added settings keys to store OAuth related data (#12879)
refs https://github.com/TryGhost/Team/issues/618

- The `oauth_client_id` and `oauth_client_secret` are placeholders to store OAuths related data.
- The flag for `oauth_enabled` or anything along those lines was not added intentionally in favour of checking if the `oauth_client_id` & `oauth_client_secret` are null.
2021-04-16 10:02:21 +02:00
Fabien 'egg' O'Carroll
22924c62d4
Added permissions for the Products resource (#12875)
refs https://github.com/TryGhost/Team/issues/616

All roles which can publish posts should be able to read/browse products, as content gating
will be based on products going forward.

Creating, updating & destroying products will often make modifications to Stripe which requires
Administrator or Owner roles.

We also improve the permissions tests so that we no longer rely on things being in a particular
order.
2021-04-15 14:34:45 +01:00
Rishabh Garg
eb66c715fd
Updated stripe prices table schema (#12863)
refs https://github.com/TryGhost/Team/issues/586

- Change the interval column to be `nullable` as one time payments won't have any interval
- Remove the `livemode` column as we store the connected account's livemode status at top level
2021-04-12 19:18:19 +05:30
Naz
e5b13ee5d2 Added todo to users' schema to audit status field
no issue

- There is a valid subset of statuses that can be set for the users but there's no "isIn" validation for possible values
- Additionally some of the statuses like warn-1, warn-2, etc. don't have a clear usecase (or at least nothing was found in the codebase for them to be used). They might be up for removal if this assumption is correct
2021-04-09 18:59:38 +12:00
Rishabh Garg
94766c05bf
Added stripe_products and stripe_prices tables (#12858)
refs https://github.com/TryGhost/Team/issues/586

- Add the `stripe_products` table, so that we can map Stripe Products to Products in Ghost
- Add the `stripe_prices` table, so that we can associate Stripe Prices to Products table
2021-04-08 20:41:00 +05:30
Fabien 'egg' O'Carroll
25182b7b82
Added products and members_products tables (#12844)
refs https://github.com/TryGhost/Team/issues/586

- Add the products table, so that we can store Products in Ghost 
- Add the members_products table, so that we can associate Members w/ Products
- Use sort_order on the members_products table to follow the same convention in members_labels
- Populate the products table with a single product, using the name from the stripe_product_name setting
- Populate the members_products table with relations based on the status column of the members table

Populating the tables allows us to transition from the current system, which does not care about products, into the
new system, where Products are used to group members. The intention is that all existing paid members have the
same product
2021-04-08 14:15:30 +01:00
Rishabh Garg
dba2a2970b
Added migration to fix incorrect mrr events (#12837)
closes https://github.com/TryGhost/Team/issues/595

Due to a bug in `mrr_delta` calculation, we ended up reducing the MRR delta by twice the original amount when a subscription goes from active to canceled and storing it in `members_paid_subscription_events` table, which is used to show the MRR chart on Dashboard. The way we identify the incorrect events in the table which got the double negative value is by checking if they match certain criteria - Both `from_plan` and `to_plan` have same value as a subscription changes status while being on same plan.

This migration halves the `mrr_delta` for incorrect events to restore the correct MRR change for the site.
2021-04-06 21:14:54 +05:30
naz
ffd866cedc
💥 Reduced default exports content for DB APIs (#12818)
refs https://github.com/TryGhost/Team/issues/555

- Export files included a lot of data which was not used in the importer, for example: members,  labels, migrations and many more. This lead to a lot of clutter in the import files and made it hard to reason about their purpose.
- The main purpose of exports - is to export importable resources. These are posts, tags, and users. The rest of data like members or migrations either have their own importer (like CSV importer for members) or does not and should not have any ways to be imported.
- These changes are in now way complete. It's a first step towards resource-based exports which could be properly versioned in the future on API level and not be a mirror of the DB structure.
- This is sort of a breaking change. But we are doing it because: (1) its an internal API that should not be used by external clients, (2) there was no public contract to have this API stable at any point, (3) we really need to get back the control over export files structure and size
- In case an external client was dependent on some structure of the exported json file they can still pass in ALL of previously exported data by passing table names in `include` query parameter.
2021-03-25 19:27:49 +13:00
Naz
aaa54c603c Refactored exporter to use "allowlist" table filtering
refs https://github.com/TryGhost/Team/issues/555

- Previous blocklist approach was resulting in adding every single new table into an export automatically. Which creates possibility to leak sensitive data if not used porperly. Allowlist approach gives better control over what is exported, makes this information explicit, and version-control friendlier
2021-03-25 16:46:56 +13:00
Naz
eccb15a988 Removed unecessaryly complex logic from exporter
refs https://github.com/TryGhost/Team/issues/555

- The getVersionAndTables was doing too much and was only used once creating clutter in doExport method. Refactored code doing 2 direct calls instead of destructuring "dances".
2021-03-25 15:27:39 +13:00
Naz
2e9ec65b63 Refactored exporter to use async/await
refs https://github.com/TryGhost/Team/issues/555

- async/await is much more readable and is a preferred style within the codebase
2021-03-25 15:27:39 +13:00
Rishabh Garg
1a84b8d81b
🐛 Fixed unable to delete member (#12784)
closes https://github.com/TryGhost/Team/issues/557

After updating to v4 on SQLite from v3, we were unable to delete members due to a foreign key mismatch error. This is because the migrations which recreates the tables for `members_stripe_customers_subscriptions` and `members_stripe_customers` doesn't add back the unique constraint which is needed by FKs for reference.

The migration creates the missing UNIQUE constraints on the tables.
2021-03-18 22:20:58 +05:30
Fabien 'egg' O'Carroll
8318391d4c
Fixed typo in migration to add backupContent permission (#12777)
refs https://github.com/TryGhost/Team/issues/553

This was introduced in https://github.com/TryGhost/Ghost/commit/79c3709f

This migration has a `noop` for the down, as we never want to revert
Ghost to a broken state
2021-03-18 14:52:12 +00:00
Hannah Wolfe
67adfe6054 Updated fixture copy
closes: https://github.com/TryGhost/Team/issues/528

- This only changes one sentence slightly to a minor grammar issue 💪
2021-03-11 19:20:30 +00:00
Hannah Wolfe
fdc043227a Updated fixtures for Ghost v4.0
- With 4.0 we have a brand new version of Casper, new fixtures and new default settings
- Fixture posts cover the key features and give users an introduction to how to use their site
- This all comes from the marketing and design teams to refresh the look and feel of Ghost and give users the best possible onboarding experience
Note: this fixture overhaul includes
- new content for new 4.0 features
- regenerated post content using our updated mobiledoc structure
- a switch from British to US English
2021-03-10 17:37:16 +00:00
Hannah Wolfe
44060d3515 Refined navigation default settings for 4.0
refs 2bba9989db

- Note: this will require new fixtures so that the navigation links actually work
- These updates are all in aid of getting the best possible default setup and onboarding experinence for new Ghost users
2021-03-10 10:42:21 +00:00
Hannah Wolfe
2bba9989db Updated default settings for 4.0
- With 4.0 we have a brand new version of Casper, new fixtures and new default settings
- This all comes from the marketing and design teams to refresh the look and feel of Ghost

Note on accent color:

This commit changes the default accent colour again.

The intention is that new sites should get #FF1A75 (pink) as their default.
Any existing sites that do not have an accent colour set yet, should get #15171A (black) on upgrading to 4.0.

These are different as they are different experinces. Fresh sites will be guided to pick a color, so
a bright color is more visible and helps to see what can be done, whilst existing sites get a muted
black, that should be a sensible fall back color.
2021-03-09 20:19:59 +00:00
Fabien 'egg' O'Carroll
15978689c0
Added default value to accent_color setting (#12743)
refs TryGhost/Team#535

We want to ensure that a site will always have a default value of
`'#15171A'` for the accent_color setting.

Since the boot process changed we have three cases to account for:

1. Setting does not exist
2. Setting exists with no value
3. Setting exists with a value

It is only in the case of 2. that we want the migration to update the
database with a default value.

In the case of 3. the site owner has already set a value, which we do
not want to override.

In the case of 1. the setting will be created (and populated with
default value) from the default-settings.json file, by the
populateDefaults method called from the settings service

We also update the accent_color setting to include a non-empty
validation, to ensure that the setting will always have a value, as
sites before 4.x may have an empty accent_color, we must update the
importer to set the default value if one is not present. Otherwise we
would run into validation errors and even if we didn't would have an
invalid database state.
2021-03-09 16:15:10 +00:00
Fabien 'egg' O'Carroll
52d1940460
Removed queries against information_schema (#12739)
no-issue

We were originally checking the state of the database, e.g. if a foreign
key constraint existed, so that we could conditionally act upon it. This
was to ensure that our migrations are idempotent.

Some database configurations, for example if you have many databases on
a single MySQL instance, would cause these information_schema queries to
take an exceptionally long time.

In order to speed up migrations, we instead attempt the action we want
to apply to the database, and then catch relevant errors to ensure the
migration is idempotent.

SQLite does not error when adding duplicate foreign or primary key
constraints, meaning that we must keep in pre-checks for these
operations, when running on SQLite

Co-authored-by: Daniel Lockyer <hi@daniellockyer.com>
2021-03-09 16:12:10 +00:00
Thibaut Patel
2eb5f19090 🐛 Fixed a foreign key to point to the right column
pr https://github.com/TryGhost/Ghost/pull/12713
2021-03-09 15:19:06 +01:00
Kevin Ansfield
cbbf5b4ad4
Fixed mobiledoc errors when upgrading from v1/v2 to v4 (#12741)
no issue

Upgrading from v1 or v2 can result in successful upgrades but with mobiledoc errors showing in the logs:

```
NAME: InternalServerError
MESSAGE: Mobiledoc card 'card-markdown' not found.
```

The errors do not signify a problem as long as the 4.0 migrations run because those rename the deprecated card before re-rendering.

- `@tryghost/kg-default-cards` dropped support for `card-markdown` cards. 4.0 migrations handled this by renaming all `card-markdown` cards to `markdown` before re-generating any content
- 2.0 and 3.0 also had migrations that re-generated content but they are run before the 4.0 card rename migration meaning that the mobiledoc renderer sees cards that it doesn't know about. The behaviour for unknown cards is to log an error and skip rendering of that card
- by NOOPing the 2.0 and 3.0 migrations we eliminate the incompatibility errors and reduce the amount of processing the upgrade needs to perform
2021-03-09 10:51:44 +00:00
Kevin Ansfield
a6f5eb71be
🏗 Changed internal URL storage format to use __GHOST_URL__ (#12731)
closes https://github.com/TryGhost/Team/issues/467

- switches to storing "transform-ready" URLs in the database
- transform-ready URLs contain a `__GHOST_URL__` placeholder that corresponds to the configured url that gives a few benefits
  - much faster and less memory intensive output transformations through not needing to parse html or markdown - the transform can be achieved using a straightforward regex find+replace
  - ability to change to/from or rename subdirectory without any manual updates to the database
- modified existing 4.0 url-transformation migration rather than adding another one and repeating the transformation on posts rows
2021-03-05 13:54:01 +00:00
Fabien O'Carroll
f03e1dd863 Excluded member events tables from site exports
refs https://github.com/TryGhost/Team/issues/525

We expect the member event tables to be large, and they contain what is
considered metadata. For this reason we do not want to include them in
the export of sites.
2021-03-05 11:41:00 +00:00
Hannah Wolfe
e30b9735fa Added staff user limit
refs: https://github.com/TryGhost/Team/issues/510

- In the case that host config is provided, keep staff users within the limiti
- The definition of a staff user is a user with a role other than Contributor, and whose status is not inactive
   - Contributors don't count
   - Suspended (status inactive) users don't count
   - Locked users DO count
   - Invited users DO count
- You can't invite more staff users whilst there are pending invites
- You can't unsuspend a user, or change the role on a user in such a way as will take you over your limit
- You can't import staff users - all imported users are automatically set to Contributors
- As part of this work, we are changing the default Ghost user to a Contributor otherwise it uses up a staff user

Note: there is one known active bug with this commit.
- Assume you have one remaining user within your limit. You send an invite, this works.
- You cannot "resend" that invite, it will think you're sending a new invite and hit the limit
- You must "revoke" that invite first, and create a new one
- This bug exists because the resend function uses the add endpoint & does a delete+add, but this hits the permission check before the delete
2021-03-04 16:13:04 +00:00
Daniel Lockyer
4e18606942
Updated logging messages in migration commands
no issue

- by switching around the the columns and tables, it becomes a
  lot easier to read the log line in a natural order
2021-03-04 13:53:06 +00:00
Naz
38fbfe0bde Added note about missing migration for itegration_id 2021-03-04 10:56:58 +01:00
Thibaut Patel
e52dc87b7c Added a foreign key from webhooks to integrations
issue https://github.com/TryGhost/Team/issues/477
2021-03-04 10:56:58 +01:00
Naz
2e2d185087 Added note to webhooks.api_version column
refs https://github.com/TryGhost/Team/issues/513
refs https://github.com/TryGhost/Team/issues/477

- We have skipped work to improve the 'defaultTo' value when working on Ghost 4.0 release, so adding this comment while context loaded
-  defaultTo should not be set to anything as it leads to more maintenance work during major version bump
- having validation might make sense but could lead to similar maintenance work unless it's linked to some global nosion of "supported API versions" used everywhere
2021-03-04 13:09:31 +13:00
Naz
55e9ef957f Fixed typo 2021-03-04 11:27:31 +13:00
Thibaut Patel
126f9004b4 Updated a duplicate migration id
no issue
2021-03-03 10:45:21 +01:00
Thibaut Patel
8b7d7ba1f1 Fixed the add/drop foreign key on sqlite
issue https://github.com/TryGhost/Ghost/pull/12713
ref https://github.com/knex/knex/issues/4155
2021-03-03 10:35:10 +01:00
Thibaut Patel
0843ab6a37 Added 'on delete cascade' to several foreign keys in sqlite
issue https://github.com/TryGhost/Team/issues/476
blocked by https://github.com/TryGhost/Ghost/pull/12702
2021-03-03 10:35:10 +01:00
Thibaut Patel
874ccaef53 💡 Enabled foreign key checks on sqlite3
issue https://github.com/TryGhost/Team/issues/476
2021-03-02 11:13:19 +01:00
Thibaut Patel
c8af2d4a04 Added a foreign key in migration utils
no issue

- This makes it easy to add/remove foreign key in both mysql and sqlite
2021-03-02 11:13:19 +01:00
Fabien O'Carroll
d8fa54024b Fixed migrations to 4.x from 1.x
no-issue

The slack setting in 1.x did not have a `username` property. When
updating from 1.x, the migration to move the slack setting from a JSON
blob to two individual settings, would assume that the setting in the
database would have a value for the `username` property. This resulted
in errors on SQLite "sqlite does not support inserting default values."

The fix here is to add defaults when reading from the database, meaning
that we will _never_ attempt to insert `undefined`
2021-03-01 15:41:02 +00:00
Fabien 'egg' O'Carroll
3b6c55ec53
Remove orphaned Stripe data from SQLite (#12704)
refs https://github.com/TryGhost/Team/issues/476

* Moved paid subscription events population migration

This migration relies on the members_stripe_customers and
members_stripe_customer_subscriptions tables having no orphaned records
in order for it to correcly generate its data.

The migration to clean up orphaned records in those tables has not been
implemented yet, moving this migration free's up the "14" slot

* Removed orphaned stripe data from SQLite3

SQLite databases do not handle removing orphaned stripe records after a
member has been deleted. Our migration to populate the paid subscription
events relies on each customer and subscription being associated with a
member.
2021-03-01 14:32:49 +00:00
Fabien O'Carroll
68008baf0e 🐛 Fixed email permissions for all roles
refs https://github.com/TryGhost/Team/issues/494

The migrations in 3.1.0 which added email permissions did not add those
permissions to the roles. This means that whilst we have the permissions
in the database, only the Owner role could use any of them.

This migration ensures that the email related permissions are added to
the correct roles.
2021-03-01 13:17:43 +00:00