refs: https://github.com/TryGhost/Toolbox/issues/166
New package handles the email verification workflow to prevent spammers. It currently handles MembersSubscribeEvent to detect potential abuse of the API to add members, and exposes methods for checking the threshold / starting the verification process for use by other areas of the code (at the moment - just member imports).
The import package no longer needs to handle anything related to verification since it can be handled in the wrapper function in Ghost, and the API package doesn't need to do anything other than dispatch the new event.
refs https://github.com/TryGhost/Team/issues/1277
- When a user signs-up, two events are created, the code was only keeping one of these events.
- This was introduce in commit 58c9c1c649 when the only usage of the function was to extract the 5 most recent events. Any duplication was creating too much noise.
- This was creating issues now that we introduce event filtering. Some `newsletter_event` events would appear from nowhere we we were filtering-out `signup_event` events.
- We removed the deduplication when the `membersActivityFeed` flag is enabled.
refs https://github.com/TryGhost/Team/issues/1277
- In `getEventTimeline` we filter to only perform the relevant queries, passing to each query function the filters (subset of NQL)
- In each query function, we rewrite the filters to adapt them to the internal data shape.
- We need to do this rewrite to allow API consumers to create filters based on the output on the API instead of the internal data structure.
- Added partial unit tests as there is a lot of repetition between the query functions.
refs https://github.com/TryGhost/Team/issues/1277
- This will allow to filter events within `getEventTimeline`
- The subset of NQL has the following rules:
- Only one level of filters, now parenthesis allowed
- Only three filter keys allowed
- No `or` allowed outside of the bracket notation (this is allowed: `type:-[email_opened_event,email_failed_event]` but this isn't: `type:1,data.created_at:1`)
- The return is an object with a NQL filter by allowed filter key
refs https://github.com/TryGhost/Team/issues/1252
Although we filter out archived tiers from being shown in Portal - we
must also persist this information, so that when they are unarchived,
they continue to not be shown in Portal.
Unfortunately this information is stored in a setting, rather than on
the Tier object itself, two things to consider for the future are:
1. Representing the display value on the Tier object in the API
2. Updating the DB tables to allow storing the display value on the Tier
refs https://github.com/TryGhost/Team/issues/1277
- Doing a filtered DB query wasn't a good solution because we query several tables with different shapes.
- Filtering in memory is done with the NQL library
- Removed the side-effect of changing options.filter for each query, now that options.filter will already contain NQL
- The filtering is done in the reduce function to avoid looping one more time over the in-memory events
refs https://github.com/TryGhost/Team/issues/1277
- Added the pre-existing framework filters to each event query.
- Made sure we aren't modifying the original `options.filter` to avoid side-effects between event queries.
refs https://github.com/TryGhost/Team/issues/1252
This will be used to archive and unarchive Tiers. There is a restriction
on archiving "free" Tiers because our current system expects only one,
and it should always be active.
refs https://github.com/TryGhost/Team/issues/1277
- The members weren't included in the serialized version of the new events
- Some properties weren't using the bookshelf `get` method as they should have
refs https://github.com/TryGhost/Team/issues/1277
- Adds 3 new requests to the `email_recipients` table in the `getEventTimeline` method
- This allows to extract new member events from the table: `email_delivered_event`, `email_opened_event`, `email_failed_event`
refs https://github.com/TryGhost/Team/issues/1037
Tiers have a new `type` column to differentiate between `free` and `paid` tiers. This change -
- sets type as paid for all new tiers created, as `free` tier is created by default
- excludes any price/stripe data change for free tier
- updates all usages of default product to fetch the first paid product from the products list in DB instead of just the first product it finds.
refs https://github.com/TryGhost/Team/issues/1275
We want to be able to track where member subscriptions came from, so
that we can use the information to reduce spam imports of members.
We were missing information when members were uploaded via the Admin
API, and setting the source to 'member' be default - this fixes that
both when creating members and when updating their subscription status.
refs https://github.com/TryGhost/Team/issues/1092
refs https://github.com/TryGhost/Team/issues/1135
This was missed in the initial due to the issue tracking the task being
superceded, and the task not being copied across to the superceding
issue.
A new method to remove coupons has been added, as opposed to updating
the existing change subscription price method, because the removal of a
coupon is not the concern of an auxillary stripe service, but a busines
concern that should be explicit in the members-api codebase.
no-issue
Without forcing linkSubscription to run inside a transaction - it's
possible to have race conditions where it is called twice, and attempt
to insert duplicate rows into the database.
refs https://github.com/TryGhost/Team/issues/1132
We have to include the Offer on the metadata for the Stripe Checkout -
as Offers with a duration of 'once' will not always be present on the
Subscription after fetching it.
Once we receive the Stripe Checkout webhook we emit an event for
subscription created - the reason we use an event is because this logic
should eventually live in a Payments/Stripe module - and we'd want to
decouple it from the Members module.
The Members module is in charge of writing Offer Redemptions - rather
than the Offers module - because Offer Redemptions are "owned" by a
Member - and merely reference and Offer. Eventually Offer Redemptions
could be replaced by Subscriptions.
refs https://github.com/TryGhost/Team/issues/1156
Because we were only attempting to add the product to the members if the
subscription was new AND active - we would not add it for incomplete
subscriptions transitioning to active.
Instead we always attempt to add the product to a member for an active
subscription - it doesn't matter if it's a new one. We later have logic
to filter out duplicate products if the member already has access to the
product.
refs https://github.com/TryGhost/Team/issues/1057
This method will validate a token, and then return the member associated
with it. Rather than exposing token validation and coupling consumers to
the structure of the token response data.
refs https://github.com/TryGhost/Team/issues/995
Since we reintroduced the comped status, we did not update the
subscription handling to correctly set members to a status of comped
when they were on a 'Complimentary' plan.
no-issue
Since updating the product repository to force transactions, the options
parameter was used in every call, meaning it wasn't optional any more,
which broke usage. This updates the parameter to have a default so that
existing usage still works.
no-issue
Since we run our product repository methods in transactions now we must
ensure that all database interations in the method use the transaction.
This adds the missing options to the reading of existing prices so that
they happen inside of the transaction.
refs https://github.com/TryGhost/Team/issues/982
These calls to the edit method were missing the transaction option from
the parent which meant that they ran outside of the transaction and
would cause the method to timeout.
refs https://github.com/TryGhost/Team/issues/979
This correctly handles updates to subscriptions so that if the product a
subscription is for has changed, we will remove the previous product, if
and only if there is not another subscription which gives access to it.
refs https://github.com/TryGhost/Team/issues/873
This handles the creation of product events when a members access to
products is changed. This can happen on creation, update, and any
changes to stripe subscriptions.
We manually workout the difference between the current products and the
new products, and add the events accordingly.
no-issue
Calling ObjectId doesn't return a string a but an ObjectId object.
Whilst this object is cast to a string via the toJSON and toString
methods, this is not enough for MySQL. Instead we should explicitly cast
this to a string ourselves and the application level.
refs https://github.com/TryGhost/Team/issues/946
This adds the bulk edit method which handles bulk edit operations to members
to be used by the filtering feature. They have been combined into a single method
as that is how they are exposed to the API. This is definitely a candidate for a
refactor in the form of a service in front of the repository.
refs https://github.com/TryGhost/Ghost/commit/1dd52075
- Fixes bulkDestroy being passed the context
- Fixes passing options.search to the model layer
- Updates return value since the changes in referenced commit
no-issue
The logic for bulk destroy is currently incorrectly inside of the
members api controller in Ghost core. Moving it out to here allows us to
simplify the controller to rely on the service, rather than implement
the logic.
refs https://github.com/TryGhost/Team/issues/959
Because we were using the pre-existing products to determine a members
status, instead of the products _after_ we have handled the updates to
subscriptions, members with a paid subscription which was later canceled
were changed to 'comped' rather than 'free'. This adds a final check to
set a member to 'free' if their new set of products is empty.
refs https://github.com/TryGhost/Team/issues/919
As we pass the `benefits` to the Product model on creation, we do not
need to manually fetch them again. In fact doing so causes a strange SQL
error, where we attempt to run `SELECT undefined.*`.
refs https://github.com/TryGhost/Team/issues/542
Importing members with a created_at date will incorrectly create events
for the member for the date of the import. This updates our event
handling to use either the passed created_at date, or in the case of
subscriptions the start_date of the subscription. We're using start_date
for subscriptions rather than created, as this is more accurate because
start_date works correctly for backdated subscriptions in Stripe.
refs https://github.com/TryGhost/Team/issues/790
We were missing a check for the existence of memberData.products before
attempting to read the length property from it, which can result in an
Uncaught TypeError
refs https://github.com/TryGhost/Team/issues/790
When linking a subscription to a member we must also update their
status. We could be in one of many states, so we start with the initial
values of either 'free' or 'comped' based on whether the member has any
products. We then make sure to updated the status to 'paid' if we find
any active subscriptions associated with the member, otherwise we leave
it as the initial value.
refs https://github.com/TryGhost/Team/issues/790
Both creating and updating members only ever need to explicitly set
either the 'comped' or 'free' status as these methods do not deal with
Stripe. When updating a member if the products are not changing, we do
not attempt to change the status either.
refs https://github.com/TryGhost/Team/issues/748
This ensures that you cannot add more than one product to a Member.
However it does allow a Member which already exists with more than one
Product to continue using the API. This is to account for edgecases such
as a Member going through the Stripe flow twice and ending up with
multiple subscriptions for multiple products
refs https://github.com/TryGhost/Team/issues/748
Previously when a change happened to a subscription the member's
products would be reset purely based on the subscriptions. Since we now
support setting products outside of subscriptions, we must make sure
than a change to a subscription will only affect the product for which
the subscription was for
refs https://github.com/TryGhost/Team/issues/775
Exposing this to the Admin API means that we want to be able to cancel
by id, not just email. The same pattern from editSubscription has been
used.
refs fd40e04105
The current implementation of setting legacy prices as inactive has a bug where it also sets the active price to inactive if only one of monthly/yearly price is edited while the other is not changed. This is a temporary reversal till we can narrow down the issue and fix to correctly set legacy plans to inactive.
refs https://github.com/TryGhost/Team/issues/704
Currently when attempting to create stripe_prices without a Stripe
connection, it will fail silently. This is an issue when initially
configuring Members as the Stripe connection can take some time to be
established. By erroring we allow the client to be notifed that the
connection does not yet exist, so that it can be retried later.
refs https://github.com/TryGhost/Team/issues/635
It's possible that we have subscriptions in the system which have been
created externally, and so using an interval of week or day. This change
ensures that we handle the mrr_delta for these subscriptions correctly.
refs https://github.com/TryGhost/Team/issues/693
With the new system of Custom Products, the concept of Complimentary is
not longer a thing, and will instead be handled by explicitly creating
prices with no amount. This means that the 'comped' status for members
will be replaced with 'paid'.
closes https://github.com/TryGhost/Team/issues/682
This ensures that the Stripe Product name is updated during the
migrations of an existing site and any future updates to the Product
name.
closes https://github.com/TryGhost/Team/issues/650
Despite the fact we're getting rid of the concept of Complimentary, we
must maintain backwards compatibility for the `comped` flag in the Admin
API & the importer. This flag is handled via the `setComplimentary`
method, which has been updated to work with the new system.
refs https://github.com/TryGhost/Team/issues/637
All the APIs that currently work with price names needs to be updated to work with price ids instead to work with custom prices/products. This change updates APIs to work with Price IDs in `checkout` , `updateSubscription` and other APIs/methods.
no-issue
When updating a Product we can pass existing Stripe Prices, these will
either be adding to the database, or updated if they already exist. When
updating them we were attemping to use the `id` passed in the update,
which is not necessarily included. Instead we should use the `id` of the
StripePrice which we have already retrieved from the database.
refs https://github.com/TryGhost/Team/issues/616
This is a generic method for adding a subscription to a member for a
particular price/product pair. This will be used in the Admin for e.g.
giving complimentary subscriptions.
refs https://github.com/TryGhost/Team/issues/616
For existing prices linked to a Product, we only allow site owners/admins to edit their nickname and nothing else. This change handles the update in Ghost, but needs extension to update the name in Stripe as well.
closes https://github.com/TryGhost/Team/issues/628
refs 9010a62d54
Following up on last commit, this moves up the expansion of Stripe customer fetch to always include `subscriptions` by default in api service so we don't accidentally miss it.
no refs
The latest version of Stripe doesn't return the `subscriptions` object on `Customer` resource by default and needs an extra param to do so. As we recently updated Members to use the latest Stripe version, the importer tries to fetch all subscriptions of a customer to map in Ghost but was failing due to missing `subscriptions` data. Fix updates the Stripe API to include `subscriptions` by default in response.
refs https://github.com/TryGhost/Team/issues/619
On linking a stripe subscription to a member, this change -
- Adds missing stripe price or stripe product from subscription to DB
- Missing Stripe price is attached to the first Ghost Product if no matching Product exists
- Updates usage from plan to price in the `linkSubscription` method
- Updates products associated with a member based on active subscriptions
refs https://github.com/TryGhost/Team/issues/616
Working with ProductRepository as a separate package was more trouble
than it was worth, so it's been moved into members-api. We expose the
product repository so that Ghost Admin API can access it.
refs https://github.com/TryGhost/Team/issues/595
For a canceled subscription, the desired MRR delta is to reduce by negative of original amount, but our logic was incorrectly reducing it by double which led to big gap between real MRR and one shown on Dashboard.
- Fixes calculation for MRR change for canceled subscriptions
no-issue
The Admin API uses a Member id rather than email to update
subscriptions, this ensures that we provide an interface that will
continue to work with the Admin API
refs https://github.com/TryGhost/Team/issues/530
The RouterController was a grab bag of all controller methods, making it
difficult to mock & test. This adds a MemberController with a smaller
API - making it easier to test.
no-issue
When seeding the database with fake members & stripe data, it's possible
to create stripe plans without a nickname. Similarly some other services
do not have a nickname on their plans. This ensures that we do not error
when working with these plans.
no-issue
If we are to perform the `linkSubscription` method inside of a
transaction, the addition of the paid subscription events would happen
outside of the transaction, and cause errors. This ensures that we pass
the options object (containing the transaction) to the models calls to
add paid subscription events
refs https://github.com/TryGhost/Team/issues/469
In order to reduce noise, we want to only display newsletter
subscription events which are not likely to be the result of a member
signup. The approach we've taken is to remove any newsletter
subscription (not unsubscription) event, if when sorted in chronological
order, it is to reside next to a signup event for the same member.
An improvement to this approach might be to add some kind of transaction
id to events which would allow us to group together events which should
be considered to have happened simultaneously.
refs https://github.com/TryGhost/Team/issues/469
Signup events are captured by status changes with no `from_status`, this
means that the member did not have a status (did not exist) before this
change.
refs https://github.com/TryGhost/Team/issues/469
We order the set of all events by created_at, but were not fetching the
individual events with the same order applies, this resulted in
incorrect results.
refs https://github.com/TryGhost/Ghost/issues/12711
We must wait for the stripeSubscriptions relation to be loaded before
attempting to loop through them. As well as this we should use `upsert`
so that we can edit a subscription record by `subscription_id`, rather
than the (internal) `id`
no refs
The member id assigned when creating a new status event on member creation was incorrectly using `data.id` instead of `member.id`, which was undefined causing a validation error.
refs https://github.com/TryGhost/Ghost/issues/12602
* Added Event Repository
** Added method for MRR over time
** Added method for newsletter subscriptions over time
** Added method for gross volume over time
** Added method for status segment size over time
* Captured login events
* Captured newsletter subscription/unsubscription
* Captured email address change events
* Captured paid subscription events
* Captured payment events
* Captured status events
refs https://github.com/TryGhost/Team/issues/475
The subscription object here is a database model rather than an
ISubscription from the stripe library, and we need to 1) use the `get`
method to read attributes and 2) read the `subscription_id` attribute,
as the `id` is our internal one.
no-issue
If we receive webhooks out of order, e.g. a
`customer.subscription.updated` with a status of 'active', followed by a
`customer.subscription.created` with a status of 'incomplete'. We would
overwrite the correct value with data from the "older" webhook. This
ensures that we always fetch the latest data from the Stripe API before
storing in the database.
no-issue
This refactors the members-api module so that it is easier to test going forward,
as well as easier to understand & navigate. The Stripe API no longer contains
storage code, this is all handled via the member repository. And we have dedicated
services for webhooks, and stripe plans initialisation.
