refs https://github.com/TryGhost/Team/issues/982
These calls to the edit method were missing the transaction option from
the parent which meant that they ran outside of the transaction and
would cause the method to timeout.
no-issue
As subscriptions are a default relation, and we now require products to
populate subscriptions for comped members, we need to include products
by default when reading members.
refs https://github.com/TryGhost/Team/issues/986
The getMemberIdentityData is a relic of time past. Originally it was
used before we had anything like the member repository or bread
controller as a way for things inside of the Members ecosystem to get
access to member data.
This updates it to use the same interface as everything else for
fetching members so that we can rely on the shape of the data that we
consider a member.
This update will ensure that themes have access to the dummy
subscriptions created by the `read` method of the MemberBREADService.
refs https://github.com/TryGhost/Team/issues/979
This correctly handles updates to subscriptions so that if the product a
subscription is for has changed, we will remove the previous product, if
and only if there is not another subscription which gives access to it.
refs https://github.com/TryGhost/Team/issues/873
The `get` method of the member repository will return null when no
member is found - we must ensure that we don't attempt to call toJSON!
It is also possible for a member to not have any products, in which case
we should not attempt to iterate over them, and we can return early.
refs https://github.com/TryGhost/Team/issues/873
This is an unexpected state, but possible if the alpha version of tier
has been enabled previous to the events being added.
refs https://github.com/TryGhost/Team/issues/873
This adds a dummy subscription for each product that a member has
without an associated stripe subscription. It allows clients to deal
with things like a created date for comped members.
no-issue
The idea of this service is to sit infront of the repository and handle
application logic which does not belong at the data layer. The exact
naming and structure is TBC but this gives us a place to start pulling
logic out of the controllers, without having to mash it all into the
repository.
Also important to note is that is does not return instances of bookshelf
models, but a JSON representation of the model, this allows us to not
leak internal implementation to consumers.
refs https://github.com/TryGhost/Team/issues/873
This handles the creation of product events when a members access to
products is changed. This can happen on creation, update, and any
changes to stripe subscriptions.
We manually workout the difference between the current products and the
new products, and add the events accordingly.
no-issue
Calling ObjectId doesn't return a string a but an ObjectId object.
Whilst this object is cast to a string via the toJSON and toString
methods, this is not enough for MySQL. Instead we should explicitly cast
this to a string ourselves and the application level.
refs https://github.com/TryGhost/Team/issues/870
- using `c8` allows us to see test coverage for all packages in the repo
- this commit adds `c8` as a dev dependency and prepends the `mocha`
command with `c8` so it runs on all tests
refs https://github.com/TryGhost/Team/issues/958
- The import threshold has to become a dynamic number based on external parameters. Because of this it makes most sense to have it as an async function parameter
- There's a package API change for importThreshold constructor paramter becoming a fetchThreshold async funciton parameter
refs https://github.com/TryGhost/Team/issues/946
This adds the bulk edit method which handles bulk edit operations to members
to be used by the filtering feature. They have been combined into a single method
as that is how they are exposed to the API. This is definitely a candidate for a
refactor in the form of a service in front of the repository.
refs https://github.com/TryGhost/Ghost/commit/1dd52075
- Fixes bulkDestroy being passed the context
- Fixes passing options.search to the model layer
- Updates return value since the changes in referenced commit
no-issue
The logic for bulk destroy is currently incorrectly inside of the
members api controller in Ghost core. Moving it out to here allows us to
simplify the controller to rely on the service, rather than implement
the logic.
refs https://github.com/TryGhost/Team/issues/959
Because we were using the pre-existing products to determine a members
status, instead of the products _after_ we have handled the updates to
subscriptions, members with a paid subscription which was later canceled
were changed to 'comped' rather than 'free'. This adds a final check to
set a member to 'free' if their new set of products is empty.
refs https://github.com/TryGhost/Team/issues/912
- We need a way to check if the import threshold has been reached within the members importer. See more deets in refed issue
refs https://github.com/TryGhost/Team/issues/909
The member identity data currently attaches several extra data points to member information which is not used/needed, and causes multiple DB queries on each page load when Portal requests for member via `/members/api/member` endpoint. This change removes all the unused data points on member - `labels`, `stripe_customer`, products`, `stripe_product` cutting DB queries in half.
no issue
- After each test run there were csv files lefover which polluted the workspace.
- Ideally the write file would be some kind of a smart mock, so we'd won't have to do synchronous file remoal after each test but that's for later improvement if it becomes problematic
refs https://github.com/TryGhost/Team/issues/916
- These tests are nowhere close to perfect of even good. They barely pass "good enough" watermark... it's a start for new tests to be added easier in the future once the features are developed around CSV improrter
refs https://github.com/TryGhost/Team/issues/916
- The constructor API should have as small of a surface as possible, there's no need to pass around whole ghostMailer instance
refs https://github.com/TryGhost/Team/issues/916
- The constructor API should have as small of a surface as possible, there's no need to pass around whole settingsCache instance
refs https://github.com/TryGhost/Team/issues/916
- The refactor was done follow the DI Constructor pattern with single options Object parameter
- It didn't make sense to have a "config" object inside of options object containing just one property
refs https://github.com/TryGhost/Team/issues/916
- While investigating members importer related codebase this legacy module was spotted. It's not used anywhere and doesn't serve any particular purpose.
refs 2f1123d6ca
Usage of the raw Error class has been deprecated in favour of our own
errors, which are more descriptive and have built in HTTP status codes.
This also updates the same errors to use @tryghost/tpl for the error
messages, which is the new pattern we are following in order for us to
deprecate the i18n module.
- This isn't really a "service" - it's a set of utilities for working with labs flags
- It's also required all over the place, and doesn't require anything that isn't shared
- Therefore, it should live in shared
- This isn't really a "service" - it's a set of utilities for working with labs flags
- It's also required all over the place, and doesn't require anything that isn't shared
- Therefore, it should live in shared
refs https://github.com/TryGhost/Team/issues/765
This supercedes the `complimentary_plan` flag, as it is more precise
because it determines _which_ product(s) a member has access to. Because
of this, if the `products` column is present the `complimentary_plan`
column is not used.
refs https://github.com/TryGhost/Team/issues/765
As part of the multiple products feature, we're not longer using Stripe
subscriptions to denote Complimentary access, instead we're linking
members directly to products. Here we update the importer to follow
suit, so long as the flag is enabled.
no issue
The only pieces of Ghost-Ignition used in Ghost were debug and
logging. Both of these modules have been superceded by the Framework
monorepo, and all usages of Ignition have now been removed, replaced
with @tryghost/debug and @tryghost/logging.
no-issue
When importing Members it is possible to have both the
complimentary_plan and the stripe_customer_id columns set, this can
result in unusual outcomes, for example when importing a customer with a
zero-amount subscription, they would end up with two "comped"
subscriptions, and there would be two "comped" prices in the database.
As we are deprecating the use of "comped" in favour of creating a
subscription with a specific price, we're updating the import to prefer
`stripe_customer_id` column, only using the `complimentary_plan` column
when it is the only of the two columns passed.
refs b51bba7b0e
- i18n is used everywhere but only requires shared or external packages, therefore it's a good candidate for living in shared
- this reduces invalid requires across frontend and server, and lets us use it everywhere until we come up with a better option
- Having these as destructured from the same package is hindering refactoring now
- Events should really only ever be used server-side
- i18n should be a shared module for now so it can be used everywhere until we figure out something better
- Having them seperate also allows us to lint them properly
refs c873899e49
- as of `bson-objectid` v2.0.0, this library exports the function
to generate an ObjectID directly, and then you need to use `.toHexString()`
to get the 24 character hex string - 6696f27d82
- this commit removes all uses of `.generate()` and replaces with this
change
no-issue
This module encapsulates the work around performing imports, it
currently uses the concept of a "Job" which at the moment is not
persisted to the database, however when we want to look at resuming
imports after a server restart, this should give us the flexibility to
do it.
- users imported from CSV with no created_at date where having their created_at date being stored as an int rather than a datetime.
- this was causing parsing issues with the graph so this commit fixes the formatting
closes#12045
- When member's email is updated to an already existing email of different member it caused table's unique constraint error, which was not handled properly.
- Added handling for this error similar to one in members `add` method.
no issue
- When an import was done and there were no "global labels" present Ghost created generic `import-[data]` label which later helped to find a specific batch of imported data
- It did not make sense to create such generic label when user provided their own unique label
- The rules that work now are:
1. When there is no global provided Ghost generates on and removes it in case there are no imported records
2. When there is a unique new global label provided no new label is generated, but the label stays even if there are no imported records
no issue
- This is handled on input sanitization layer with date
format check in JSON schema validation, so there's no need to do this
check again in the importer.
no issue
- When bulk insert fails there is no transactional logic to revert
related records form being inserted. Also, previously there were no
attempts to "retry" the insert.
- To avoid complex retry logic, an iterative one-by-one insert retry
approach was taken. If this becomes a bottleneck in the future, the
retry algorithm could be improved.
- To avoid a lot of code duplication refactored model's `bulkAdd` & `bulkDestroy`
methods to use 'bulk-operations' module.
- Updated error handling and logging for bulk delete operations. It's very
unlikely for error to happen here, but still need to make sure there is
a proper logging in place to trace back the failure.
- Added debug logs. This should improve debugging experience and
performance measurements.
- Added handling for unrecognized errors. Handling inspired by current unrecognized
error handling by ghost importer -10e5d5f3d4/core/server/data/importer/importers/data/base.js (L148-L154)
no issue
- Similar handling to one introduced in 31db3c86800b3268da5485417b16e0fcd8e6579a
- Having granular tracking for failed to remove id's would make it possible to return more specific errors to the client
no issue
- When batch insert fails handling should be more granular and aim to retry and insert as many records from the batch as possible.
- Added retry logic for failed member's batch inserts. It's a sequential insert for each record in the batch. This implementation was chosen to keep it as simple as possible
- Added filtering of "toCreate" records when member fails to insert. We should not try inserting related members_labels/members_stripe_customers/members_stripe_customer_subscriptions records because they would definitely fail insertion without associated member record
no issue
- When stripe is disconnected and there are Stripe-connected records present in imported set they should not be processed and proper error should be thrown
no-issue
* Added bulkAdd method to Member,Customer&Subscription model
This allows us to keep the db access in the model layer
* Updated @tryghost/members-api to 0.27.2
This includes fixes for rate-limiting of requests, and exposes necessary
Stripe methods for creating customers and complimentary subscriptions,
without affecting the database.
* Refactored importer to parallelise tasks where possible
By parallelising our tasks we are able to improve the speed at which the
entire import completes.
no issue
- There were many failed import records due to rate-limit errors. With concurrency of 9 imports go through with 100% success
- Would need to verify these limits with live API to make the most of it
no-issue
* Added stripeSubscriptions relation to member model
This allows us to fetch the subscriptions for a member via standard
model usage, e.g. `withRelated: ['stripeSubscriptions']` rather than
offloading to loops and `decorateWithSubscriptions` functions, this is
more performant and less non-standard than the existing method.
* Updated serialize methods to match existing format
The current usage of `decorateWithSubscriptions` and the usage of
members throughout the codebase has a subscriptions array on a stripe
object on the member, this ensures that when we serialize members to
JSON that we are using the same format.
There is definitely room to change this in future, but this is an
attempt to create as few breaking changes as possible.
* Installed @tryghost/members-api@0.26.0
This includes the required API changes so that everywhere can use
members-api directly rather than models and/or helper methods
no issue
- The code in controller was becoming hard to reason about.
- Having a single module shows exactly how many dependencies are there to do an import for single batch.
- Having a separate module would make it easier to extract into it's own package in Members monorepo
refs https://github.com/TryGhost/Team/issues/919
As we pass the `benefits` to the Product model on creation, we do not
need to manually fetch them again. In fact doing so causes a strange SQL
error, where we attempt to run `SELECT undefined.*`.
refs https://github.com/TryGhost/Team/issues/908
The `cookies` module will unset a cookie if `null` or `undefined` is
passed as the value, or if the value is not passed. The previous call
was passing the options, which were being read as the value, and
resulting in `'[Object object]'` being stored as a cookie.
Explicitly passing `null` as the value makes this code correct and
easier to maintain.
refs https://github.com/TryGhost/Team/issues/664
The well known controller is designed to handle any requests to the
/.well-known endpoint where the members app is mounted. The first and
only requirement so far is that we expose a JSON Web Key Set so that
external services are able to validate Members JWT's
closes https://github.com/TryGhost/Team/issues/778
- cleans up the stripe migration to add default monthly/yearly prices for sites, which had a possibility of using complimentary (0 amount prices) in edge cases
- adds missing return in the same migration for an unlikely failure to parse stripe plans
refs https://github.com/TryGhost/Team/issues/542
Importing members with a created_at date will incorrectly create events
for the member for the date of the import. This updates our event
handling to use either the passed created_at date, or in the case of
subscriptions the start_date of the subscription. We're using start_date
for subscriptions rather than created, as this is more accurate because
start_date works correctly for backdated subscriptions in Stripe.
closes https://github.com/TryGhost/Team/issues/660
All subscriptions in Ghost are expected to have a corresponding price details in `stripe_price` table, which is used to determine the Stripe price a subscription is on. In some edge cases, specially before we started deleted old Stripe data during Stripe disconnect, it's possible that a subscription exists in DB without having a corresponding Stripe price in the DB. These subscriptions are not active for the connected Stripe account, and are save to remove. Going forward, all existing subscriptions with connected account will be removed when disconnecting stripe so we shouldn't have invalid subscriptions in DB in future.
The goal of this migration is to clean all such subscriptions from the DB to avoid any issues around missing price with invalid subscriptions.
refs https://github.com/TryGhost/Team/issues/858
Replacing the check for subscriptions with products ensures that Stripe
Checkout is not able to be opened by comped members.
refs https://github.com/TryGhost/Team/issues/790
We were missing a check for the existence of memberData.products before
attempting to read the length property from it, which can result in an
Uncaught TypeError
refs https://github.com/TryGhost/Team/issues/790
When linking a subscription to a member we must also update their
status. We could be in one of many states, so we start with the initial
values of either 'free' or 'comped' based on whether the member has any
products. We then make sure to updated the status to 'paid' if we find
any active subscriptions associated with the member, otherwise we leave
it as the initial value.
refs https://github.com/TryGhost/Team/issues/790
Both creating and updating members only ever need to explicitly set
either the 'comped' or 'free' status as these methods do not deal with
Stripe. When updating a member if the products are not changing, we do
not attempt to change the status either.
no issue
- we're killing off `ghost-ignition` in favor of explicit packages
containing its individual components
- this commit switches the Members repo to using the
`@tryghost/ignition-errors` and `@tryghost/debug` dependencies,
updates the code with relevant changes and removes the `ghost-ignition`
dependency
no issue
- these dependencies are used within the `members-csv` repo but were
never included in the `package.json`
- this commit adds in the missing dependencies
refs https://github.com/TryGhost/Team/issues/765
Support for multiple products means we can no longer map a members state
to a csv row using just the `complimentary_plan` option. Instead we must
include the product(s) that a member has. This ensures that we can read
and write this data from/to csv files.
refs https://github.com/TryGhost/Team/issues/748
This ensures that you cannot add more than one product to a Member.
However it does allow a Member which already exists with more than one
Product to continue using the API. This is to account for edgecases such
as a Member going through the Stripe flow twice and ending up with
multiple subscriptions for multiple products
refs https://github.com/TryGhost/Team/issues/748
Previously when a change happened to a subscription the member's
products would be reset purely based on the subscriptions. Since we now
support setting products outside of subscriptions, we must make sure
than a change to a subscription will only affect the product for which
the subscription was for
refs https://github.com/TryGhost/Team/issues/775
Exposing this to the Admin API means that we want to be able to cancel
by id, not just email. The same pattern from editSubscription has been
used.
refs 7d8e18c802
Since we have now gone back to using `monthly` / `yearly` named values in portal plan settings instead of price ids, the old portal plan migration which was converting named values to ids is now redundant. Moreoever, the old migration is incorrectly resetting the `portal_plans` setting to `[free]` by removing the paid plans as its not found in `stripePlans` setting anymore.
refs fd40e04105
The current implementation of setting legacy prices as inactive has a bug where it also sets the active price to inactive if only one of monthly/yearly price is edited while the other is not changed. This is a temporary reversal till we can narrow down the issue and fix to correctly set legacy plans to inactive.
closes https://github.com/TryGhost/Team/issues/753
Currently, the portal_plans setting is storing price ids for active monthly/yearly prices for the default product, which was done to allow multiple prices in Portal. Since we only want to limit the prices for a Product to monthly/yearly, we are reverting the earlier migration and only store the available prices as monthly / yearly in portal setting instead of ids. Its also in sync with the approach in theme/API where we use named prices for monthly/yearly instead of price id list.
refs https://github.com/TryGhost/Team/issues/711
As this migration relies on the settings being populated, it cannot be a
standard migration in Ghost core, as the settings are populated by these
code migrations.
refs https://github.com/TryGhost/Team/issues/704
Currently when attempting to create stripe_prices without a Stripe
connection, it will fail silently. This is an issue when initially
configuring Members as the Stripe connection can take some time to be
established. By erroring we allow the client to be notifed that the
connection does not yet exist, so that it can be retried later.
refs https://github.com/TryGhost/Team/issues/635
It's possible that we have subscriptions in the system which have been
created externally, and so using an interval of week or day. This change
ensures that we handle the mrr_delta for these subscriptions correctly.
refs https://github.com/TryGhost/Team/issues/657
Since we have removed the getPlan method we need to update the migration
which is the last piece of code which used it. Prices are backwards
compatible with Plans - so this will continue to work as expected.
refs https://github.com/TryGhost/Team/issues/657
- Removes the use of "plans" from token service - method wasn't used
- Updates to use async/await so the code is clearer
- Install types for the node-jose module
- Install types for the jsonwebtoken module
refs https://github.com/TryGhost/Team/issues/657
We must still use the "plan" approach of creating Checkout Sessions so
that we are able to maintain our current functionality of setting the
free trial from the one associated with the Plan/Price used for the
Checkout Session.
In future we would handle free trials internally in Ghost and apply them
on a subscription-by-subscription basis.
refs https://github.com/TryGhost/Team/issues/698
As this migration relies on the `stripe_prices` table being populated,
it can not be in a standard versioned migration in Ghost core.
refs https://github.com/TryGhost/Team/issues/693
With the new system of Custom Products, the concept of Complimentary is
not longer a thing, and will instead be handled by explicitly creating
prices with no amount. This means that the 'comped' status for members
will be replaced with 'paid'.
closes https://github.com/TryGhost/Team/issues/682
This ensures that the Stripe Product name is updated during the
migrations of an existing site and any future updates to the Product
name.
no-issue
Since we do not necessarily have a single stripe product anymore, we
should be checking if an invoice webhook is for a stripe product which
we know about. We use the Products repository to search our database for
one.
closes https://github.com/TryGhost/Team/issues/650
Despite the fact we're getting rid of the concept of Complimentary, we
must maintain backwards compatibility for the `comped` flag in the Admin
API & the importer. This flag is handled via the `setComplimentary`
method, which has been updated to work with the new system.
