refs https://github.com/TryGhost/Team/issues/757
- To safeguard from mise of a very permissing "object" value of the "labs" setting this change introduces an "allowlist" approach to filtering unrecognized labs flags
- Should allow maintainers to have a clear view of which labs flags are currently in use and manage them accordingly
refs https://github.com/TryGhost/Team/issues/710
This allows us to fetch the default monthly and yearly price models for
a product model, which is important since we no longer want to expose
the entire list of prices, but just the designated monthly & yearly prices.
refs https://github.com/TryGhost/Team/issues/660
In case stripe price for a subscription is missing in `stripe_prices` table, it will cause the API to load members list to fail with 500 as we try to serialize the stripe price on member subscription using empty object. This fixes the guard against populating price object for missing data in DB.
Note: This is only a short-term fix till we add a proper fix to cleanup the DB in the subsequent release.
no issue
Our server-defined `mobiledoc` object was required by `UrlUtils.cardTransformers` property to help set up a shortcut for url transform functions but that was breaking the independence of `UrlUtils` by crossing the shared/server boundary.
- `cardTransformers` is only needed for the `mobiledocToTransformReady` utility function that will only be used by the server
- removed `UrlUtils.cardTransformers` (and associated require) from our `UrlUtils` instance and updated the few areas the server uses `mobiledocToTransformReady()` to pass in the mobiledoc card objects directly as an option
refs https://github.com/TryGhost/Team/issues/693
Since we've got rid of the concept of Complimentary with the Custom
Prices work, we're removing the 'comped' status from members. This
involves a migration for existing members, a schema update for the
validation, and a bump to members-api to no longer use the 'comped'
status for new members.
We also update the aggregation of the MemberStatusEvent to consider the
'comped' status as 'paid', and that there are 0 'comped' status events
in the database.
We can consider a migration for this data in the future, either adding
new status events moving from 'comped' to 'paid', or by modifying
existing status events. However both of these are very difficulty to
write a down migration for, and might be best saved for a major version.
- @tryghost/members-api@1.7.0 is the version that includes the required
changes, however we have already bumped to 1.8.0 in Ghost
refs https://github.com/TryGhost/Team/issues/581
closes https://github.com/TryGhost/Team/issues/582
Emails can now be sent to members with specific associated labels or products by specifying an NQL string. We want to bring the same members segment feature to content by allowing `visibility` to be an NQL filter string on top of the `public/members/paid` special-case strings.
As an example it's possible to set `posts.visibility` to `label:vip` to make a post available only to those members with the `vip` label.
- removed enum validations for `visibility` so it now accepts any string or `null`
- bumped `@tryghost/admin-api-schema` for API-level validation changes
- added nql validation to API input validators by running the visibility query against the members model
- added transform of NQL to special-case visibility values when saving post model
- ensures there's a single way of representing "members" and "paid" where NQL gives multiple ways of representing the same segment
- useful for keeping theme-level checks such as `{{#has visibility="paid"}}` working as expected
- updated content-gating to parse nql from post's visibility and use it to query the currently logged in member to see if there's a match
- bumped @tryghost/members-api to include label and product data when loading member
refs https://github.com/TryGhost/Team/issues/581
refs https://github.com/TryGhost/Team/issues/582
When publishing a post via the API it was possible to send it using `?email_recipient_filter=all/free/paid` which allowed you to send to members only based on their payment status which is quite limiting for some sites.
This PR updates the `?email_recipient_filter` query param to support Ghost's `?filter` param syntax which enables more specific recipient lists, eg:
`?email_recipient_filter=status:free` = free members only
`?email_recipient_filter=status:paid` = paid members only
`?email_recipient_filter=label:vip` = members that have the `vip` label attached
`?email_recipient_filter=status:paid,label:vip` = paid members and members that have the `vip` label attached
The older `free/paid` values are still supported by the API for backwards compatibility.
- updates `Post` and `Email` models to transform legacy `free` and `paid` values to their NQL equivalents on read/write
- lets us not worry about supporting legacy values elsewhere in the code
- cleanup migration to transform all rows slated for 5.0
- removes schema and API `isIn` validations for recipient filters so allow free-form filters
- updates posts API input serializers to transform `free` and `paid` values in the `?email_recipient_filter` param to their NQL equivalents for backwards compatibility
- updates Post API controllers `edit` methods to run a query using the supplied filter to verify that it's valid
- updates `mega` service to use the filter directly when selecting recipients
refs https://github.com/TryGhost/Team/issues/586
refs aa12770329
Using `id` as ghost id for subscription prices can be confusing as everything in the method refers ids to be stripe ids. This change updates the ghost id value to use `price_id` key in the serialization
refs https://github.com/TryGhost/Team/issues/586
refs 33f26fbf32
As part of serializing subscriptions with prices, we previously attached only the stripe price id to the price object for subscription. This change updates the price object to include both Ghost id and stripe price id for the object, as Portal needs to check the Ghost price id for logged in members to verify their current plan.
- single authors were deprecated in v1.22 when we added multiple authors
- we always thought we'd clean this up a lot sooner, but it's stuck because it's an annoying thing to break people's shit over
- still saying "remove in vX" isn't useful, we need to know how long a feature has been deprecated so we can judge whether it's safe to remove
refs https://github.com/TryGhost/Team/issues/637
With custom prices, Portal now needs to show all available custom prices in the UI instead of just `monthly` and `yearly` prices. This change adds a list of all custom prices to Portal site settings for the default product which Portal will use to show the available prices in UI.
Note: As part of cleanup, the stripe price ids will be removed from the prices list.
Also:
- Fixes product name in serialised subscriptions
- Adds `type` value in serialised price object
refs 829e8ed010
- i18n is used everywhere but only requires shared or external packages, therefore it's a good candidate for living in shared
- this reduces invalid requires across frontend and server, and lets us use it everywhere until we come up with a better option
- Having these as destructured from the same package is hindering refactoring now
- Events should really only ever be used server-side
- i18n should be a shared module for now so it can be used everywhere until we figure out something better
- Having them seperate also allows us to lint them properly
refs https://github.com/TryGhost/Team/issues/616
We need a way to assign Products to Members via a Subscription, and we've
followed the same pattern as the editSubscription method for the Members API
controller, which acts upon Subscriptions as a nested resource.
Subscriptions now are linked to products, and we've included those links by
default in the Member Admin API as we already include subscriptions by
default, and Products are now a core part of the Members feature-set.
refs c873899e49
- as of `bson-objectid` v2.0.0, this library exports the function
to generate an ObjectID directly, and then you need to use `.toHexString()`
to get the 24 character hex string - 6696f27d82
- this commit removes all uses of `.generate()` and replaces with this
change
refs https://github.com/TryGhost/Team/issues/586
We have to use `belongsToMany` because of the way bookshelf relations
work. In reality the relationship is 'hasMany', e.g. a Product has many
Stripe Prices.
These relations are the minimal needed to satisfy the following
relationships without transforming the results. (e.g. flattening the
StripePrices from a list of StripeProducts for a Product)
Product -> StripeProduct: product.related('stripeProducts')
StripeProduct -> StripePrice: stripeProduct.related('stripePrices');
Product -> StripePrice: product.related('stripePrices');
StripePrice -> Product: stripePrice.related('stripeProduct.product');
refs https://github.com/TryGhost/Team/issues/586
- Member model now has `products` relation, sorted using `sort_order`, following convention from `labels`
- Product model has handling to set `slug` from name, following convention of Label model
- Updated filter plugin to handle filtering Member models by their `product` relations e.g. `product:[slug, slug]`
closes https://github.com/TryGhost/Team/issues/571
- `post.plaintext` values were being transformed with the markdown URL transformer but that wasn't picking up the link format used in our plaintext fields resulting in absolute URLs being stored in the database rather than `__GHOST_URL__` URLs
- meant that if the `url` config is changed then plaintext and other calculated fields that used it would have URLs that referred to the old domain rather than the new one
- re-saving the posts would have updated it but that's not feasible to do manually for large sites
- bumped `@tryghost/url-utils` to a version that has plaintext transform utils and updated the post model's transform map
refs https://github.com/TryGhost/Team/issues/510
- contributors don't count towards the staff limit, therefore they should be allowed to be unsuspended
- currently, we don't check the role when unsuspending, which is incorrect
- this bug is pure oversight!
closes https://github.com/TryGhost/Ghost/issues/12791
closes https://github.com/TryGhost/Team/issues/566https://github.com/TryGhost/Ghost/pull/12787 introduced a significant performance regression due to a misunderstanding of when Bookshelf calls `.format()` ([related upstream issue](https://github.com/bookshelf/bookshelf/issues/668)). We expected `.format()` to only be called on save but it's also called when Bookshelf performs fetching and eager loading which happens frequently. `.format()` can be a heavy method as it needs to parse and serialize html and markdown so it should be performed as infrequently as possible.
- override `sync()` in the base model so we can call our own `.formatOnWrite()` method to transform attributes on `update` and `insert` operations
- this was the only feasible location in Bookshelf I could find that is low enough level to not require modifying model instance attributes
- gives models the option to perform heavy transform operations only when writing to the database compared to the usual `.format()` method that is also called on fetch in many situations
closes https://github.com/TryGhost/Team/issues/552
Refactors URL transforms so they take place at the model layer rather than the API serializer layer. Continuation of the pattern created for the settings model in https://github.com/TryGhost/Ghost/pull/12738
- Added checks to all front-end tests to ensure output does not contain the magic replacement string
- includes failing acceptance test for `__GHOST_URL__` appearing in sitemaps
- Removed all transform-ready URL transforms from API serializers
- input serializers transform image urls relative->absolute to keep absolute-urls as the consistent "outside of the database" format
- output serializers should not need to perform any URL transforms as that will be done at the model layer
- Added url transforms to models layer
- removes knowledge from the API serializers which shouldn't need to know how data is stored internally in the database
- makes absolute urls the consistent "outside of the database" URL format
- adds transform step to the sitemap generator because the data used for that is fetched directly via knex which will not run through the bookshelf `parse()` methods
refs: https://github.com/TryGhost/Team/issues/510
refs: e30b9735fa
- In the case that there is a staff user limit, and the limit is maxed out, it is no longer possible to invite new staff users
- However, Contributors are not considered staff users and therefore it should always be possible to invite new Contributors
refs https://github.com/TryGhost/Ghost/pull/12736
refs https://github.com/TryGhost/Team/issues/467
knex's `parse()` method is only called on data when directly fetched from the db. This was causing problems when model instances are passed around via events for example because `.get('key')` will return data that was directly set on the model without having gone through the `parse()` transformations. The result of this inconsistency was settings appearing correct when Ghost started up but then being broken as soon as a setting was changed.
- moved absolute/relative->transform-ready URL transformations from the API input serializers to the model's `format()` method and replaced with a relative->absolute transform in API input serializers
- results in consistency because `.get()` on a settings model will always return an URL
- removed transform-ready->absolute transforms from the API output serializers as that is now handled at the model-layer
refs https://github.com/TryGhost/Team/issues/467
refs https://github.com/TryGhost/Ghost/pull/12731
- settings are mostly fetched directly from the settings cache rather than via the API so they aren't subject to the API-level output serializers that transform URLs meaning that URLs in the front-end ended up with raw `__GHOST_URL__` replacement strings
- added images to the Settings model's `parse()` method so they are transformed immediately when fetching from the database
closes https://github.com/TryGhost/Team/issues/467
- switches to storing "transform-ready" URLs in the database
- transform-ready URLs contain a `__GHOST_URL__` placeholder that corresponds to the configured url that gives a few benefits
- much faster and less memory intensive output transformations through not needing to parse html or markdown - the transform can be achieved using a straightforward regex find+replace
- ability to change to/from or rename subdirectory without any manual updates to the database
- modified existing 4.0 url-transformation migration rather than adding another one and repeating the transformation on posts rows
refs: https://github.com/TryGhost/Team/issues/510
- The current member limit was implemented as a member-specific concept
- The new limit service is much more generic, here we are swapping old for new
- The updated concept here is blocking all publishing, not just email sending, when a site is over its member limit
- To determine that we are publishing a post, we must be in the model layer. The code has been moved to the permissible function which makes sense as this is a permissions error that we are throwing
- I've left the extra check for email retries in, in case there is some loophole here (but we may wish to change it)
refs: https://github.com/TryGhost/Team/issues/510
- In the case that host config is provided, keep staff users within the limiti
- The definition of a staff user is a user with a role other than Contributor, and whose status is not inactive
- Contributors don't count
- Suspended (status inactive) users don't count
- Locked users DO count
- Invited users DO count
- You can't invite more staff users whilst there are pending invites
- You can't unsuspend a user, or change the role on a user in such a way as will take you over your limit
- You can't import staff users - all imported users are automatically set to Contributors
- As part of this work, we are changing the default Ghost user to a Contributor otherwise it uses up a staff user
Note: there is one known active bug with this commit.
- Assume you have one remaining user within your limit. You send an invite, this works.
- You cannot "resend" that invite, it will think you're sending a new invite and hit the limit
- You must "revoke" that invite first, and create a new one
- This bug exists because the resend function uses the add endpoint & does a delete+add, but this hits the permission check before the delete
refs https://github.com/TryGhost/Team/issues/510
- When the host config was introduced it was incorrectly introduced as host_settings instead of hostSettings
- All other Ghost config uses camelCase, so changing this now before it becomes a problem
- Note: Also removed some rogue return awaits that don't make sense. It's not possible to hit that case, but cleaning up anyway
refs https://github.com/TryGhost/Ghost/issues/12716
- The default API in Ghost v4 will be `'v4'`. When a new webhook is created or any data passes through serialization it should assume `v4` as a fallback if not specified.
refs 17feb14e4a
- The original commit adding this intended to add transactions, following the pattern of always forcing a transaction when we use bookshelf-relations
- (We use bookshelf-relations here because integrations have api-keys and webhooks associated wtih them and we upsert as one)
- These add and edit methods were inadvertently added to the wrong argument object/section of bookshelf (really fucking easily done, one day we will fix bookshelf so its easier to work with)
- Bottom line: these methods have never been called
- I tried moving them to the right section, but this created test failures throughout our acceptance tests:
- Error: Transaction query already complete, run with DEBUG=knex:tx for more info
- This is likely because we need to account for integrations being used as part of the auth step in the before part of tests
- In terms of yak-shaving, fixing these tests is one step too far right now. I think not having this code here at all is a better state than having it look like it works when it doesn't
closes https://github.com/TryGhost/Team/issues/466
- upgraded kg-default-cards to include paywall card
- extracted `htmlToPlaintext` from post model to shared util for re-use
- updated post-gating to set html+plaintext to the free preview if a paywall card has been used
- re-generates plaintext from the truncated html using `htmlToPlaintext` util
- display free content in the `{{content}}` helper via the default CTA template
refs https://github.com/TryGhost/Ghost/issues/12602
* Updated members_status_events table
By replacing the `status` column with a `from_status` and `to_status`
column, we are able to track the changes between multiple statuses
easier, and accumulate the data. e.g. the delta of paid members in a
given time range is the sum of the `to_status` columns set to 'paid'
minus the sum of the `from_status` columns set to 'paid' within that
time range
* Updated MEGA to handle addition of 'comped' status
With the addition of the 'comped' status, we need to ensure that MEGA
will still send emails to the correct recipients. I've opted to use an
"inverse" filter, as that is the intention of the free/paid split in
MEGA - as far as MEGA is concerned, "free" is the opposite of "paid"
* Updated customQuery for MemberStatusEvent
With the `status` column replaced with `from_status` and `to_status`
this allows us to fix and update the customQuery to correctly accumulate
the data into deltas over time, broken down by day.
* Populated members_status_events table
As the table will be used to generate deltas, we need to backfill the
data so that existing sites will be able to sum up the deltas and
calculate correct data.
The assumptions used in backfilling is that a Member's current status,
is their only status.
refs https://github.com/TryGhost/Ghost/issues/12602
The gross volume aggregate query was incorrectly naming the selected `SUM(amount)` value as `gross_volume` instead of expected `volume_delta`
closes https://github.com/TryGhost/Ghost/issues/12600
The bookshelf-relations plugin which we use will **remove** all
relations when they are set to an empty array, but will leave them alone
if it's set to undefined.
Our logic to deduplicate uppercase & lowercase version of the same label
was in advertently always setting the labels to an array, but when the
model was saved without passing the labels, this array would be empty.
Here we've added a check which will skip all label handling, if there
are no labels set.
refs https://github.com/TryGhost/Ghost/issues/12602
In order to build up a list of un/subscribes over time we have to use
the customQuery functionality to run SQL aggregates - this is "hidden"
behind an option, so that we can find{All,Page} as usual.
refs https://github.com/TryGhost/Ghost/issues/12602
Member Events in general are read-only after they've been created, so
we've explicitly disallowed destroying and editing via the model
