refs https://github.com/TryGhost/Team/issues/586
We have to use `belongsToMany` because of the way bookshelf relations
work. In reality the relationship is 'hasMany', e.g. a Product has many
Stripe Prices.
These relations are the minimal needed to satisfy the following
relationships without transforming the results. (e.g. flattening the
StripePrices from a list of StripeProducts for a Product)
Product -> StripeProduct: product.related('stripeProducts')
StripeProduct -> StripePrice: stripeProduct.related('stripePrices');
Product -> StripePrice: product.related('stripePrices');
StripePrice -> Product: stripePrice.related('stripeProduct.product');
refs https://github.com/TryGhost/Team/issues/586
- Member model now has `products` relation, sorted using `sort_order`, following convention from `labels`
- Product model has handling to set `slug` from name, following convention of Label model
- Updated filter plugin to handle filtering Member models by their `product` relations e.g. `product:[slug, slug]`
closes https://github.com/TryGhost/Team/issues/571
- `post.plaintext` values were being transformed with the markdown URL transformer but that wasn't picking up the link format used in our plaintext fields resulting in absolute URLs being stored in the database rather than `__GHOST_URL__` URLs
- meant that if the `url` config is changed then plaintext and other calculated fields that used it would have URLs that referred to the old domain rather than the new one
- re-saving the posts would have updated it but that's not feasible to do manually for large sites
- bumped `@tryghost/url-utils` to a version that has plaintext transform utils and updated the post model's transform map
refs https://github.com/TryGhost/Team/issues/510
- contributors don't count towards the staff limit, therefore they should be allowed to be unsuspended
- currently, we don't check the role when unsuspending, which is incorrect
- this bug is pure oversight!
closes https://github.com/TryGhost/Ghost/issues/12791
closes https://github.com/TryGhost/Team/issues/566https://github.com/TryGhost/Ghost/pull/12787 introduced a significant performance regression due to a misunderstanding of when Bookshelf calls `.format()` ([related upstream issue](https://github.com/bookshelf/bookshelf/issues/668)). We expected `.format()` to only be called on save but it's also called when Bookshelf performs fetching and eager loading which happens frequently. `.format()` can be a heavy method as it needs to parse and serialize html and markdown so it should be performed as infrequently as possible.
- override `sync()` in the base model so we can call our own `.formatOnWrite()` method to transform attributes on `update` and `insert` operations
- this was the only feasible location in Bookshelf I could find that is low enough level to not require modifying model instance attributes
- gives models the option to perform heavy transform operations only when writing to the database compared to the usual `.format()` method that is also called on fetch in many situations
closes https://github.com/TryGhost/Team/issues/552
Refactors URL transforms so they take place at the model layer rather than the API serializer layer. Continuation of the pattern created for the settings model in https://github.com/TryGhost/Ghost/pull/12738
- Added checks to all front-end tests to ensure output does not contain the magic replacement string
- includes failing acceptance test for `__GHOST_URL__` appearing in sitemaps
- Removed all transform-ready URL transforms from API serializers
- input serializers transform image urls relative->absolute to keep absolute-urls as the consistent "outside of the database" format
- output serializers should not need to perform any URL transforms as that will be done at the model layer
- Added url transforms to models layer
- removes knowledge from the API serializers which shouldn't need to know how data is stored internally in the database
- makes absolute urls the consistent "outside of the database" URL format
- adds transform step to the sitemap generator because the data used for that is fetched directly via knex which will not run through the bookshelf `parse()` methods
refs: https://github.com/TryGhost/Team/issues/510
refs: e30b9735fa
- In the case that there is a staff user limit, and the limit is maxed out, it is no longer possible to invite new staff users
- However, Contributors are not considered staff users and therefore it should always be possible to invite new Contributors
refs https://github.com/TryGhost/Ghost/pull/12736
refs https://github.com/TryGhost/Team/issues/467
knex's `parse()` method is only called on data when directly fetched from the db. This was causing problems when model instances are passed around via events for example because `.get('key')` will return data that was directly set on the model without having gone through the `parse()` transformations. The result of this inconsistency was settings appearing correct when Ghost started up but then being broken as soon as a setting was changed.
- moved absolute/relative->transform-ready URL transformations from the API input serializers to the model's `format()` method and replaced with a relative->absolute transform in API input serializers
- results in consistency because `.get()` on a settings model will always return an URL
- removed transform-ready->absolute transforms from the API output serializers as that is now handled at the model-layer
refs https://github.com/TryGhost/Team/issues/467
refs https://github.com/TryGhost/Ghost/pull/12731
- settings are mostly fetched directly from the settings cache rather than via the API so they aren't subject to the API-level output serializers that transform URLs meaning that URLs in the front-end ended up with raw `__GHOST_URL__` replacement strings
- added images to the Settings model's `parse()` method so they are transformed immediately when fetching from the database
closes https://github.com/TryGhost/Team/issues/467
- switches to storing "transform-ready" URLs in the database
- transform-ready URLs contain a `__GHOST_URL__` placeholder that corresponds to the configured url that gives a few benefits
- much faster and less memory intensive output transformations through not needing to parse html or markdown - the transform can be achieved using a straightforward regex find+replace
- ability to change to/from or rename subdirectory without any manual updates to the database
- modified existing 4.0 url-transformation migration rather than adding another one and repeating the transformation on posts rows
refs: https://github.com/TryGhost/Team/issues/510
- The current member limit was implemented as a member-specific concept
- The new limit service is much more generic, here we are swapping old for new
- The updated concept here is blocking all publishing, not just email sending, when a site is over its member limit
- To determine that we are publishing a post, we must be in the model layer. The code has been moved to the permissible function which makes sense as this is a permissions error that we are throwing
- I've left the extra check for email retries in, in case there is some loophole here (but we may wish to change it)
refs: https://github.com/TryGhost/Team/issues/510
- In the case that host config is provided, keep staff users within the limiti
- The definition of a staff user is a user with a role other than Contributor, and whose status is not inactive
- Contributors don't count
- Suspended (status inactive) users don't count
- Locked users DO count
- Invited users DO count
- You can't invite more staff users whilst there are pending invites
- You can't unsuspend a user, or change the role on a user in such a way as will take you over your limit
- You can't import staff users - all imported users are automatically set to Contributors
- As part of this work, we are changing the default Ghost user to a Contributor otherwise it uses up a staff user
Note: there is one known active bug with this commit.
- Assume you have one remaining user within your limit. You send an invite, this works.
- You cannot "resend" that invite, it will think you're sending a new invite and hit the limit
- You must "revoke" that invite first, and create a new one
- This bug exists because the resend function uses the add endpoint & does a delete+add, but this hits the permission check before the delete
refs https://github.com/TryGhost/Team/issues/510
- When the host config was introduced it was incorrectly introduced as host_settings instead of hostSettings
- All other Ghost config uses camelCase, so changing this now before it becomes a problem
- Note: Also removed some rogue return awaits that don't make sense. It's not possible to hit that case, but cleaning up anyway
refs https://github.com/TryGhost/Ghost/issues/12716
- The default API in Ghost v4 will be `'v4'`. When a new webhook is created or any data passes through serialization it should assume `v4` as a fallback if not specified.
refs 17feb14e4a
- The original commit adding this intended to add transactions, following the pattern of always forcing a transaction when we use bookshelf-relations
- (We use bookshelf-relations here because integrations have api-keys and webhooks associated wtih them and we upsert as one)
- These add and edit methods were inadvertently added to the wrong argument object/section of bookshelf (really fucking easily done, one day we will fix bookshelf so its easier to work with)
- Bottom line: these methods have never been called
- I tried moving them to the right section, but this created test failures throughout our acceptance tests:
- Error: Transaction query already complete, run with DEBUG=knex:tx for more info
- This is likely because we need to account for integrations being used as part of the auth step in the before part of tests
- In terms of yak-shaving, fixing these tests is one step too far right now. I think not having this code here at all is a better state than having it look like it works when it doesn't
closes https://github.com/TryGhost/Team/issues/466
- upgraded kg-default-cards to include paywall card
- extracted `htmlToPlaintext` from post model to shared util for re-use
- updated post-gating to set html+plaintext to the free preview if a paywall card has been used
- re-generates plaintext from the truncated html using `htmlToPlaintext` util
- display free content in the `{{content}}` helper via the default CTA template
refs https://github.com/TryGhost/Ghost/issues/12602
* Updated members_status_events table
By replacing the `status` column with a `from_status` and `to_status`
column, we are able to track the changes between multiple statuses
easier, and accumulate the data. e.g. the delta of paid members in a
given time range is the sum of the `to_status` columns set to 'paid'
minus the sum of the `from_status` columns set to 'paid' within that
time range
* Updated MEGA to handle addition of 'comped' status
With the addition of the 'comped' status, we need to ensure that MEGA
will still send emails to the correct recipients. I've opted to use an
"inverse" filter, as that is the intention of the free/paid split in
MEGA - as far as MEGA is concerned, "free" is the opposite of "paid"
* Updated customQuery for MemberStatusEvent
With the `status` column replaced with `from_status` and `to_status`
this allows us to fix and update the customQuery to correctly accumulate
the data into deltas over time, broken down by day.
* Populated members_status_events table
As the table will be used to generate deltas, we need to backfill the
data so that existing sites will be able to sum up the deltas and
calculate correct data.
The assumptions used in backfilling is that a Member's current status,
is their only status.
refs https://github.com/TryGhost/Ghost/issues/12602
The gross volume aggregate query was incorrectly naming the selected `SUM(amount)` value as `gross_volume` instead of expected `volume_delta`
closes https://github.com/TryGhost/Ghost/issues/12600
The bookshelf-relations plugin which we use will **remove** all
relations when they are set to an empty array, but will leave them alone
if it's set to undefined.
Our logic to deduplicate uppercase & lowercase version of the same label
was in advertently always setting the labels to an array, but when the
model was saved without passing the labels, this array would be empty.
Here we've added a check which will skip all label handling, if there
are no labels set.
refs https://github.com/TryGhost/Ghost/issues/12602
In order to build up a list of un/subscribes over time we have to use
the customQuery functionality to run SQL aggregates - this is "hidden"
behind an option, so that we can find{All,Page} as usual.
refs https://github.com/TryGhost/Ghost/issues/12602
Member Events in general are read-only after they've been created, so
we've explicitly disallowed destroying and editing via the model
refs https://github.com/TryGhost/Ghost/issues/12567
- Introduced here @@UNIQUE_CONSTRAINTS@@ notation allows to create unique contraints over multiple database fields. This will be needed to change posts' table unique constraint from `slug` to `slug+type`.
- The notation is equivalent to SQL's: UNIQUE(column_name1,column_name2)
- Example use in schema:
posts: {
slug: {type: 'string', maxlength: 191, nullable: false},
type: {type: 'string', maxlength: 50, nullable: false, defaultTo: 'post', validations: {isIn: [['post', 'page']]}},
'@@UNIQUES@@': [
['slug', 'type']
]
}
no issue
- when saving Stripe keys with `stripeDirect: true` config, if either key didn't match the key format the returned validation error always contained `stripe_secret_key`
- updated to output `stripe_publishable_key` if it was the publishable key that was invalid
no-issue
This removes all references to the members labs setting, any code that was run conditionally behind this flag now runs unconditionally.
* Removed usage of Members labs flag
* Removed tests for Members disabled
* Added dynamic keypair generation for when setting is missing
no refs
- Updates member model serializer to directly set subscriptions on member object instead of `stripe.subscriptions`
- Updates all references to members subscriptions from nested `stripe.subscriptions` to `subscriptions`
- Updates v3 API serializer to still use `stripe.subscriptions`
- Updates tests
no-issue
* Removed support for paid param from v3 & canary API
* Updated active subscription checks to use status flag
* Updated MEGA to use status filter over paid flag
* Removed support for paid option at model level
* Installed @tryghost/members-api@1.0.0-rc.0
* Updated members fixtures
refs #12160
This flag will allow us easier filtering of members via the API
* Added status column to members table
This flag will be used to determine if a member is free or paid, rather
than relying on joins with the customers and subscriptions tables.
* Added migration to populate members.status
As we add the column with a default value of "free" we only need to care
about the paid members here. We also preemptively handle migrations for
SQLite where there are > 998 paid members.
closes https://github.com/TryGhost/Ghost/issues/12347
This change allows a token to be used multiple times for the first 10
seconds after its initial use, this will stop dynamic link checking
software from invaliding magic links.
closes https://github.com/TryGhost/Ghost/issues/12493
- bumps `bookshelf-relations` to new version that allows for `hasMany` children to be kept when a parent record is destroyed
- adds necessary config to the `Member.email_recipients` relationship to keep related records when a member is deleted
closes#12181
refs https://github.com/TryGhost/Ghost/pull/12265
- The 'core' value is invalid for settings 'type' column
- The 'core' default value for 'type' column in model is also invalid
- Both need to be removed as they are never used and only introduce confusion into the codebase
refs c1d66f0b01
- fixed base model allowing '@@INDEXES@@' as a permitted attribute/order
- fixed base model automatically setting `@@INDEXES@@` to null on the model when creating
- added `doAuth('members:emails')`
- creates an `email_batch` record attached to the first email in the fixtures
- creates an `email_recipients` record for each member
- runs analytics aggregation so the email and member counts are as expected
- added acceptance test for `/member/:id/?include=email_recipients`
refs https://github.com/TryGhost/Ghost-Admin/pull/1796
We want to be able to display an email activity timeline in Ghost-Admin for each member. The quickest way to achieve that right now is to provide access to the `email_recipient` data for the member when fetching, this will allow clients to build up a timeline based on the event timestamps included with each email_recipient/email pair.
- sets up `email_recipients` relationship in `Member` model
- updates members API read endpoint to accept an `email_recipients` include parameter
- appends `email_recipients.email` to the `withRelated` array when `email_recipients` is included so that we have data available for email subject and html/plaintext for previews
- updates members API output serializer to include the email_recipients object in the output
refs https://github.com/TryGhost/Ghost/issues/12461
- adds `members.email_count` and `members.email_opened_count` columns to contain cached counts for faster queries when outputting member data via API
- adds migration to populate cached counts with existing data
- tested locally on ~50k members which took ~4sec on mysql
- updates members output serializer to include the new fields in API output
refs https://github.com/TryGhost/Ghost/issues/12421
- add `orderRawQuery` function to members model so that we can ensure members with an open rate are ordered before members without an open rate no matter the order direction chosen
- added `email_open_rate` to members in the test fixtures to allow testing of order
