no issue
- The new Portal config flag allows switching on Portal conditionally with config
- The dev experiment flag still works for enabling Portal
- The flag currently defaults to `false` as Portal is still a beta feature and switched off by default
- We expose it on the admin api config endpoint so that the Ghost-Admin client can use it to conditionally render Portal settings
refs #2635
- Adds 'Location' header to endpoints which create new resources and have corresponding `GET` endpoint as speced in JSON API - https://jsonapi.org/format/#crud-creating-responses-201. Specifically:
/posts/
/pages/
/integrations/
/tags/
/members/
/labels/
/notifications/
/invites/
- Adding the header should allow for better resource discoverability and improved logging readability
- Added `url` property to the frame constructor. Data in `url` should give enough information to later build up the `Location` header URL for created resource.
- Added Location header to headers handler. The Location value is built up from a combination of request URL and the id that is present in the response for the resource. The header is automatically added to requests coming to `add` controller methods which return `id` property in the frame result
- Excluded Webhooks API as there is no "GET" endpoint available to fetch the resource
closes#12045
- When member's email is updated to an already existing email of different member it caused table's unique constraint error, which was not handled properly.
- Added handling for this error similar to one in members `add` method.
closes#11999
- When the routes.yaml file changes (manually or through API) we need
to store a checksum to be able to optimize routes reloads in the future
- Added mechanism to detect differences between stored and current routes.yaml hash value
- Added routes.yaml sync on server boot
- Added routes.yaml handling in controllers
- Added routes hash synchronization method in core settings. It lives in core settings
as it needs access to model layer. To avoid coupling with the frontend settings it accepts
a function which has to resolve to a routes hash
- Added note about settings validation side-effect. It mutates input!
- Added async check for currently loaded routes hash
- Extended frontend settings loader with async loader. The default behavior of the loader is
to load settings syncronously for reasons spelled in 0ac19dcf84
To avoid blocking the eventloop added async loading method
- Refactored frontend setting loader for reusability of settings file path
- Added integrity check test for routes.yaml file
no issue
- The async/await syntax makes it easier to reason about the code. Because adding 'Location' header is in the works it's a prep-work in a sense
no issue
- Updated magic link generation and validation methods for email update API to handle new support address
- Updated importer to ignore the new support address as it can only be updated via verification
- Updated members service to listen on settings edit for new support/reply address fields as well
- Updated tests to include the new settings
closes#12167
- Tags API v2 was ignoring `count.posts` include parameter.
- Regression was introduced with a3f693b472
- Introduced regression tests across all Content API versions to avoid similar bug in the future
no issue
- When an import was done and there were no "global labels" present Ghost created generic `import-[data]` label which later helped to find a specific batch of imported data
- It did not make sense to create such generic label when user provided their own unique label
- The rules that work now are:
1. When there is no global provided Ghost generates on and removes it in case there are no imported records
2. When there is a unique new global label provided no new label is generated, but the label stays even if there are no imported records
no issue
- When no members are succesfully imported through CSV import process the import label should not be created. Otherwise after multiple failed attempts to import there are orphaned labels in the system
* 3.30.2:
Bumped @tryghost/members-api to 0.28.1 in lockfile
Bumbed @tryghost/members-api to 0.28.1
🐛 Fixed unable to delete member when stripe is connected
refs #12127
- Adds new `editSubscription` endpoint for members admin API which allows updating individual subscription for a member - `PUT /members/:id/subscriptions/:subscription_id/`
- `editSubscription` has same permissions as member's `edit` endpoint
- Currently allows toggling of cancellation at period end for an active subscription
no issue
- When batch insert fails handling should be more granular and aim to retry and insert as many records from the batch as possible.
- Added retry logic for failed member's batch inserts. It's a sequential insert for each record in the batch. This implementation was chosen to keep it as simple as possible
- Added filtering of "toCreate" records when member fails to insert. We should not try inserting related members_labels/members_stripe_customers/members_stripe_customer_subscriptions records because they would definitely fail insertion without associated member record
no issue
- When stripe is disconnected and there are Stripe-connected records present in imported set they should not be processed and proper error should be thrown
no-issue
* Added bulkAdd method to Member,Customer&Subscription model
This allows us to keep the db access in the model layer
* Updated @tryghost/members-api to 0.27.2
This includes fixes for rate-limiting of requests, and exposes necessary
Stripe methods for creating customers and complimentary subscriptions,
without affecting the database.
* Refactored importer to parallelise tasks where possible
By parallelising our tasks we are able to improve the speed at which the
entire import completes.
no issue
- Additional validation is needed for imported data because in case of bulk insertions (through knex) we bypass model layer validation - this could lead to invalid data in the database, which would be hard to fix.
- Chose validation method we use for other endpoints - through JSON Schema. It proved to be very performant (200ms overhead for 50k records). When comparing it with iterative method (validating each record separately) this was adding about 17s of overhead.
- Refactored returned values from "sanitizeInput" method to encapsulate more logic so that the caller doesn't have to calculate amount of invalid records and deal with error types
- Whole sanitizeInput method could now be easily extracted into separate module (somewhere close to members importer)
- Bumped members-csv package. It is meant to handle empty string values - '' and null, which should allow validating member records more consistently!
no issue
- Updated sanitization logic to be self contained and return sanitieze input along with error stats
- This should give a nice place for validations to fit in
no-issue
* Added stripeSubscriptions relation to member model
This allows us to fetch the subscriptions for a member via standard
model usage, e.g. `withRelated: ['stripeSubscriptions']` rather than
offloading to loops and `decorateWithSubscriptions` functions, this is
more performant and less non-standard than the existing method.
* Updated serialize methods to match existing format
The current usage of `decorateWithSubscriptions` and the usage of
members throughout the codebase has a subscriptions array on a stripe
object on the member, this ensures that when we serialize members to
JSON that we are using the same format.
There is definitely room to change this in future, but this is an
attempt to create as few breaking changes as possible.
* Installed @tryghost/members-api@0.26.0
This includes the required API changes so that everywhere can use
members-api directly rather than models and/or helper methods
no issue
- The code in controller was becoming hard to reason about.
- Having a single module shows exactly how many dependencies are there to do an import for single batch.
- Having a separate module would make it easier to extract into it's own package in Members monorepo
no issue
- Member's labels have to have sort_order assigned when added/edited. This was lacking from batched importer.
- Implementation is based on logic used in model's base - e484709e73/core/server/models/base/index.js (L81-L86)
- deleted files under `core/server/lib/promise` and related test files
- added `@tryghost/promise` as a dependency
- fixed all local requires to point to the new package
no issue
- New Member API batched import is meant to be a substitution to current import
with improved performance while keeping same behaviore. Current
import processes 1 record at a time using internal API calls and times
out consistently when large number of members has to be imported (~10k
records without Stripe).
- New import's aim is to improve performance and process >50K
records without timing out both with and without Stripe connected
members
- Batched import can be conceptually devided into 3 stages which have
their own ways to improve performance:
1. labels - can be at current performance as number of
labels is usually small, but could also be improved through batching
2. member records + member<->labels relations - these could
be performed as batched inserts into the database
3. Stripe connections - most challanging bottleneck to solve because
API request are slow by it's nature and have to deal with rate limits of
Stripe's API itself
- It's a heavy WIP, with lots of known pitfalls which are marked with
TODOs. Will be solved iteratively through time untill the method can be
declared stable
- The new batched import method will be hidden behind 'enableDeveloperExperiments' flag to
allow early testing
refs 173e3292fa
- The bug was initially introduced in referenced commit. When request is done with `api_key` context, there should always be an `integration` object associated with it - 71c17539d8/core/server/services/permissions/parse-context.js (L36) . An `id` from `context.integration` not `context.api_key` has to be assigned to newly created webhook!
- The webhooks API is about to be declared stable in upcoming release, so no migration will be done
no issue
- When processing entries with new labels in parallel Bookshelf relations is trying to create them which caused unique key constraints to fail. To avoid the failure, all labels should be pre-created before proceeding with creating members
- Member limit code was duplicated in 2 places unnecessarily
- Also used member api code that fetched members and subscriptions fully hyrated when we only need a count
- Using a raw query significantly improves performance here
no-issue
This updates the Admin API Member resource to *not* cancel subscriptions
by default, and adds a `cancel` option. This can be used over HTTP by
including a `cancel=true` query parameter.
no issue
Having all members created during an import labelled with a specific "import label" is useful for later operations such as bulk delete/edit or simply recording how and when a member was created.
- automatically create a label with the date/time the members CSV import occurred and assign it to all imported members
- return the import label data in the API response so that clients can react accordingly such as automatically filtering the members list by the label once an import finishes
refs https://github.com/TryGhost/Ghost/issues/11971
- Added statusCode from bulk email provider to API response
- Updated error messages for different bulk email(mailgun) failure states
- Added `context` to preview mail API error message with mail provider's error message
closes#12033
- Added webhooks schemas and definitions.
- Added validation checking if integration_id is present when using session auth. This is needed to prevent orphan webhooks.
- Integrated webhook schemas into frame's validation layer.
- Added isLowerCase ajv keyword support. This is needed to be able to do isLowerCase validation using JSON Schema for webhooks.
closes#11994
- Adds support for ordering based on slug filter that contains a slug-is-in filter. It is applied only to Content API's resources - post, page, tag, author. The order is applied in the same order in which slugs appear in the filter.
- For, example providing following query parameter filter for any of the above resources: `?filter=slug:[kitchen-sink,bacon,chorizo]`, would filter them by these slugs and order in the same way defined in the filter
- Can be used in handlebars templates in following way: `{{#get "tags" filter="slug:[slugs,of,the,tags,in,order]"}}`
- The property conteining this new order is assigned to `autoOrder` instead of `rawOrder` intentionally. This explicit asstignment would allow distinguishing where the 'orderRaw' comes from the model or the API layer. Apart from adding necessary context this separation makes it easier to refactor separately model layer and API specific ordering in the future
- This commit also fixes default filtering for `author` resource in Content API. The serializer was never used before as it was missing from `serializers/index.js` module.
no issue
- Changes introduced to both API v3 and v2
- Makes sure to use the same integration_id as authenticated integration for the webhook's data.
- Makde it is impossible to create orphaned webhooks using token authentication
- Allowed only parent integration to edit it's children webhooks. Throwing permission error otherwise
no issue
- Adds new portal button settings to members site data for portal script
- Updates settings input/output serializers to handle portal icon image url
* tag '3.22.2':
v3.22.2
Updated Ghost-Admin to v3.22.2
Emitted all settings events on reinit of cache (#12012)
🐛 Updated access to be true by default in v3 API
Hardened members subscription migration against missing data (#12009)
closes#11990
- access should be a members feature, but it was already accidentally exposed to the theme layer
- it has now been added to the API even if members is disabled
- access defaults to true, unless members is enabled
- when members is enabled, access is set to the currently logged in members' access
