refs https://github.com/TryGhost/Team/issues/1674
- While preparing the changes had a look around and made small refactors to understand the codebase a little better. In general it's best to keep the method parameters as small and precise as possible instead of passing around a "bag-of-all-the-things" like "data" around
refs https://github.com/TryGhost/Team/issues/1526
We weren't using the `req.body.customerEmail` to load a member and
check their existing tiers, this meant that existing members which
were signed out and attempted to create a stripe checkout session were
able to.
refs https://ghost.slack.com/archives/C02G9E68C/p1652980792270029
When bulk unsubscribing members, the number of deleted newsletter relations are returned instead of the number of members with newsletters that were cleared. This update deletes newsletter relations on member_id, so we can return the count of members instead of newsletter relations that were deleted.
Tests in https://github.com/TryGhost/Ghost/pull/14871
- In case of an offer id present in stripe checkout session, the cadence and tier id values can be null
- this fixes the invalid 400 thrown on missing cadence value while creating stripe checkout session for offer
no issue
- Setting a dedicated source for imported members was already done with 455778662c
- This unifies the same source but keeps the usage of context setting on init alive
refs https://github.com/TryGhost/Team/issues/1577
The call to `edit` was not loading the newsletter relations which is needed
by the serializer used by the webhooks service.
Co-authored-by: Fabien "egg" O'Carroll <fabien@allou.is>
no issue
Using a designated source for members being added within the `importer` context will better describe the source in our members event table. It's more consistent with the usage of the other possible sources.
no issue
When importing members, the members-importer isn't aware of the context and therefore falls back to use `member` as a source in members event table. This makes it impossible to determine imported members from others.
- Added an `context` property to the options in the members-importer constructor
- Checked for `importer` context when creating member events and assigned the source `admin` to it
refs https://github.com/TryGhost/Team/issues/1478
- When creating or editing a member, we'll add the newsletter ids to the subscribe events
- When susbcribing to multiple newsletters, we'll create multiple subscribe events
- When removing newsletters from a member, we'll create one or more subscribe events
- Tests in https://github.com/TryGhost/Ghost/pull/14579
refs https://github.com/TryGhost/Team/issues/1520
- Instead of doing the matching of the offers and subscriptions by looking at the offer redemptions, we can now look at the offer_id from subscriptions.
- This also fixes an issue where we don't attach the offer object to subscriptions in the members' browse method
- Updated browse behaviour to match the read behaviour of members (product relation needs to get loaded because it is missing in member.products if the subscription is expired).
Tests in https://github.com/TryGhost/Ghost/pull/14515
refs https://github.com/TryGhost/Team/issues/1519
- Removed offer metadata again from offer (added in one of the previous commits)
- Added `getByStripeCouponId` method in offer repository (required to find an offer based on the stripe_coupon_id)
- Match discounts from Stripe based on the stripe_coupon_id instead of metadata
refs https://github.com/TryGhost/Team/issues/1519
- Added offer repository dependency to member repository (offerAPI didn't work because it creates a new transaction that resulted in a deadlock during tests)
- Store the offer id from the Stripe subscription metadata in the subscription (only if the discount is still active)
- Also added the offer id to the metadata for a Stripe coupon, this will make adding and removing coupons a bit more foolproof
- Prefer the usage of the offer metadata from a coupon if it is present
- When no discount is applied to a subscription, it always sets the offer id to null, even when the metadata still contains the offer
- The offer_id remains stored when a subscription is canceled/expired
refs https://github.com/TryGhost/Team/issues/1469
Previously, members were subscribed to all available newsletters by default when added. This change updates the default newsletters subscription for member to take into account newsletter preferences for auto opt-in as well as visibility.
refs: https://github.com/TryGhost/Toolbox/issues/293
Things needed to create this:
* MemberSubscriptionEvent now has an import source
* Importer now creates events with this type
* Verification trigger logic changed to use 30 day window of imports
* Updated member mrr_delta calculation to use stored mrr
refs https://github.com/TryGhost/Team/issues/1456
Simplifies the calculation of MRR deltas, which will make it easier to update MRR to include offers and cancellations in the future.
* Adjusted MRR and MRR delta calculation to consider "forever" duration offers
refs https://github.com/TryGhost/Team/issues/1451
Uses the discount information from Stripe to calculate the MRR (this was the easiest way to include it + also supports manually created discounts from users)
refs https://github.com/TryGhost/Team/issues/1490
With multiple newsletters, free members can choose their newsletter subscription preference while signing up.
This change -
- includes newsletters data in magic link token creation
- attaches newsletter data to new members created via magic link
refs https://github.com/TryGhost/Team/issues/1456
- Sets the `mrr` column correctly when updating a subscription
- We can use this value in the future to have an improved calculation of mrr_delta, but we first need to have the migration in place
- Updated missing cancelled -> canceled
refs https://github.com/TryGhost/Team/issues/1469
When a new member signs up on the site, by default they should get subscribed to all newsletters. A new member can get added via different ways (stripe webhook, admin add, free signup), so this change updates the base member repository which gets used irrespective of how member is added to ensure default newsletters are always included for new member.
If a member already has a custom newsletters list attached, we don't change anything.
refs https://github.com/TryGhost/Team/issues/1469
With multiple newsletters, members can now have one or more newsletter subscriptions that is attached to them. This change updates the member BREAD service to handle attaching newsletter data to member, based on the newsletter flag.
- if newsletter flag is disabled, add/edit methods delete any newsletter data attached to member
- sets `newsletters` as a default relation for read/browse methods so a member always has newsletter data attached to them
no issue
- When a new member is added via the API, with a stripe_customer_id, the member bread service is passed also with a default option: `withRelated: ['labels']`.
- This option is passed along to the member repository, and further and is also passed when loading relations that don't have a relationship called 'labels'.
- This results in a `labels is not defined on the model` error when you try to create a new member with a stripe customer id.
- Test that found this issue will be added to the Ghost repo.
refs https://github.com/TryGhost/Team/issues/1141
- when a member had canceled subscriptions the check we have to match products to subscriptions to determine whether to insert the hardcoded complimentary subscription was incorrectly matching against the canceled subscriptions
- updated to match only active subscriptions
refs https://github.com/TryGhost/Team/issues/1141
- The goal is to retrieve the product name for canceled subscriptions
- The property is located at `member.subscriptions.price.product.name`
- We can't easily get the full product as products are retrieved using a join through `members_products`, and there is only a `members_products` row when the subscription is active
refs https://github.com/TryGhost/Team/issues/1387
First, this default value didn't make sense as it was written when we
had a boolean visible column, second, we don't need to add defaults here
- this implementation is super tied to the models anyway, so we may as
well rely on the model defaults - if we do pull it out similar to
Offers, then it may make sense to add a default there.
We also update the JSDoc to match the updated data structure.
refs: 23b383bedf
- @tryghost/error constructors take an object, not a string - the expectation is that message, context & help should all be set
- This does the bare minimum and just ensures message is set correctly
refs https://github.com/TryGhost/Team/issues/1322
We no longer restart the Members service based on the Stripe service
being updated, which meant that if it was initially configured with
missing URL's and later Stripe connected, it would not get the new
config until a server restart. This moves the last of Stripe config into
the Stripe service, so that all things concerning Stripe can be handled
in one place and updated together.
refs https://github.com/TryGhost/Team/issues/1319
Rather than allowing our code to attempt to speak with Stripe and error
there, we opt to fail fast and throw validation errors so that our API
will respond with a 422, the existing pattern for validation errors.
no-issue
We were incorrectly mixing transactional and non-transactional
operations. An e2e test in Ghost will be merged shortly which caught
this problem.
refs https://github.com/TryGhost/Team/issues/1299
- Convert `created_at` to the right column in each function query
- Renamed the misspelled `getEmailDeliveredEvents` function
- Updated existing unit tests to cover the order
refs: https://github.com/TryGhost/Toolbox/issues/166
New package handles the email verification workflow to prevent spammers. It currently handles MembersSubscribeEvent to detect potential abuse of the API to add members, and exposes methods for checking the threshold / starting the verification process for use by other areas of the code (at the moment - just member imports).
The import package no longer needs to handle anything related to verification since it can be handled in the wrapper function in Ghost, and the API package doesn't need to do anything other than dispatch the new event.
refs https://github.com/TryGhost/Team/issues/1277
- When a user signs-up, two events are created, the code was only keeping one of these events.
- This was introduce in commit 58c9c1c649 when the only usage of the function was to extract the 5 most recent events. Any duplication was creating too much noise.
- This was creating issues now that we introduce event filtering. Some `newsletter_event` events would appear from nowhere we we were filtering-out `signup_event` events.
- We removed the deduplication when the `membersActivityFeed` flag is enabled.
refs https://github.com/TryGhost/Team/issues/1277
- In `getEventTimeline` we filter to only perform the relevant queries, passing to each query function the filters (subset of NQL)
- In each query function, we rewrite the filters to adapt them to the internal data shape.
- We need to do this rewrite to allow API consumers to create filters based on the output on the API instead of the internal data structure.
- Added partial unit tests as there is a lot of repetition between the query functions.
refs https://github.com/TryGhost/Team/issues/1277
- This will allow to filter events within `getEventTimeline`
- The subset of NQL has the following rules:
- Only one level of filters, now parenthesis allowed
- Only three filter keys allowed
- No `or` allowed outside of the bracket notation (this is allowed: `type:-[email_opened_event,email_failed_event]` but this isn't: `type:1,data.created_at:1`)
- The return is an object with a NQL filter by allowed filter key
refs https://github.com/TryGhost/Team/issues/1252
Although we filter out archived tiers from being shown in Portal - we
must also persist this information, so that when they are unarchived,
they continue to not be shown in Portal.
Unfortunately this information is stored in a setting, rather than on
the Tier object itself, two things to consider for the future are:
1. Representing the display value on the Tier object in the API
2. Updating the DB tables to allow storing the display value on the Tier
refs https://github.com/TryGhost/Team/issues/1277
- Doing a filtered DB query wasn't a good solution because we query several tables with different shapes.
- Filtering in memory is done with the NQL library
- Removed the side-effect of changing options.filter for each query, now that options.filter will already contain NQL
- The filtering is done in the reduce function to avoid looping one more time over the in-memory events
refs https://github.com/TryGhost/Team/issues/1277
- Added the pre-existing framework filters to each event query.
- Made sure we aren't modifying the original `options.filter` to avoid side-effects between event queries.
refs https://github.com/TryGhost/Team/issues/1252
This will be used to archive and unarchive Tiers. There is a restriction
on archiving "free" Tiers because our current system expects only one,
and it should always be active.
refs https://github.com/TryGhost/Team/issues/1277
- The members weren't included in the serialized version of the new events
- Some properties weren't using the bookshelf `get` method as they should have
refs https://github.com/TryGhost/Team/issues/1277
- Adds 3 new requests to the `email_recipients` table in the `getEventTimeline` method
- This allows to extract new member events from the table: `email_delivered_event`, `email_opened_event`, `email_failed_event`
refs https://github.com/TryGhost/Team/issues/1037
Tiers have a new `type` column to differentiate between `free` and `paid` tiers. This change -
- sets type as paid for all new tiers created, as `free` tier is created by default
- excludes any price/stripe data change for free tier
- updates all usages of default product to fetch the first paid product from the products list in DB instead of just the first product it finds.
refs https://github.com/TryGhost/Team/issues/1275
We want to be able to track where member subscriptions came from, so
that we can use the information to reduce spam imports of members.
We were missing information when members were uploaded via the Admin
API, and setting the source to 'member' be default - this fixes that
both when creating members and when updating their subscription status.
no-issue
Running these in a transaction ensures that they do not partially execute or run
into race conditions with simultaneous operations via the API.
refs https://github.com/TryGhost/Team/issues/1257
This gets us closer to not having to reload the MembersAPI when config
is changed which will help stop bugs arising from multiple instances of
the MembersAPI being created.
refs https://github.com/TryGhost/Team/issues/1259
These errors are thrown by nodemailer and can occur when an invalid
email address is used. Without special handling these cause a 500 error.
refs https://github.com/TryGhost/Team/issues/789
We are still having issues with duplicate subscriptions being inserted,
despite running our code in transactions. For now we will catch these
errors and response ot Stripe with a 409 so that it'll retry later - and
it stops us from throwing 500's
closes https://github.com/TryGhost/Team/issues/1238
- previously returned 500 errors when a subscription had multiple prices due to external tampering on Stripe directly
- instead now returns 400 Bad Request error when subscriptions don't have right number of prices
refs https://github.com/TryGhost/Team/issues/1067
This decouples the contents/type of email from the webhooks service,
allowing us to easily make changes to the type of email sent, without
having to make changes to the webhooks service.
refs https://github.com/TryGhost/Team/issues/1243
It's possible to get into strange states where a subscription in Ghost
doesn't have an associated Price. This then has knock on effects because
we're dealing with data in an undefined state. Rather than add guards
against this throughout the entire stack, we stop returning it from the
BREAD API. It might be worth considering removing these subscriptions
from the response of the repository, but for now this is the most
minimal change that fixes the problem.
refs https://github.com/TryGhost/Team/issues/1092
refs https://github.com/TryGhost/Team/issues/1135
This was missed in the initial due to the issue tracking the task being
superceded, and the task not being copied across to the superceding
issue.
A new method to remove coupons has been added, as opposed to updating
the existing change subscription price method, because the removal of a
coupon is not the concern of an auxillary stripe service, but a busines
concern that should be explicit in the members-api codebase.
refs https://github.com/TryGhost/Team/issues/1067
The auto-login behaviour obliterates the concept of a success URL,
because the Member is redirected in a logged in state, to the welcome
URL - rather than a logged out state to the success URL.
In order to not disrupt existing flows, we disable auto login if a
success URL is provided.
refs https://github.com/TryGhost/Team/issues/1067
This is the MVP for auto-login of Members, it does not support custom
redirects, and will always just redirect to the same place that the
signin & signup links do. Behind a feature flag whilst we iron out the
functionality.
refs https://github.com/TryGhost/Team/issues/887
Our invoice webhook handling code assumed that every invoice would be
for a subscription, but that is not the case. There are valid use-cases
of using the same Stripe account in order to sell items with a one-off
purchase. Here we update the handling to ignore all invoices which are
not for subscriptions.
no-issue
When adding a new member, we allow an email to be sent, and the type of
email to be chosen. This choice was being overriden by our signup email
logic - here we allow the BREAD API to have full control over which
email is sent.
refs https://github.com/TryGhost/Team/issues/885
This webhook isn't used and can cause issues when Checkout Sessions are
completed but with a failed payment. Removing it will remove those
errors.
refs https://github.com/TryGhost/Team/issues/1135
We use the OffersAPI to fetch Offers, so that we can be using the same
format for Offers in all of our APIs.
We will not attach the Offer to the Subscription if either the Tier or
the Cadence do not match. This is because the Offer would no longer
apply to this Subscription.
We do however retain the data, so that a Member can still be filtered on
the Offers which they've redeemed.
refs https://github.com/TryGhost/Team/issues/1166
By using the PaymentsService to fetch coupon information - we ensure
that the coupons are created if they're missing. Like in the case of a
Stripe disconnect/connect cycle.
no-issue
Without forcing linkSubscription to run inside a transaction - it's
possible to have race conditions where it is called twice, and attempt
to insert duplicate rows into the database.
refs https://github.com/TryGhost/Team/issues/1132
We have to include the Offer on the metadata for the Stripe Checkout -
as Offers with a duration of 'once' will not always be present on the
Subscription after fetching it.
Once we receive the Stripe Checkout webhook we emit an event for
subscription created - the reason we use an event is because this logic
should eventually live in a Payments/Stripe module - and we'd want to
decouple it from the Members module.
The Members module is in charge of writing Offer Redemptions - rather
than the Offers module - because Offer Redemptions are "owned" by a
Member - and merely reference and Offer. Eventually Offer Redemptions
could be replaced by Subscriptions.
refs https://github.com/TryGhost/Team/issues/1156
Because we were only attempting to add the product to the members if the
subscription was new AND active - we would not add it for incomplete
subscriptions transitioning to active.
Instead we always attempt to add the product to a member for an active
subscription - it doesn't matter if it's a new one. We later have logic
to filter out duplicate products if the member already has access to the
product.
refs https://github.com/TryGhost/Members/commit/5172e40646
When we updated to use the OffersAPI instead of OfferRepository this was
missed, and we were passing blank coupon to Stripe Checkout. This should
eventually be replaced with a call like `getCoupon(offerId)` from a
payments service.
refs https://github.com/TryGhost/Team/issues/1133
An archived Offer is intended to be disabled from a redemption point of
view. This ensures that we do not allow Stripe Checkout Sessions to be
created for them.
no-issue
The OfferRepository deals with domain objects in the Offers module, and
as such is not suitable for use with "external" services. This update
means that MembersAPI can deal with POJO DTOs so that there is not a
dependency on the internals of the Offers module. Just on the contract
it holds with the outside world.
refs https://github.com/TryGhost/Team/issues/1090
Instead of the hardcoded 1-day version for Offers, we can now talk
directly to the Offers repository and use the real values for Stripe
Checkout.
refs https://github.com/TryGhost/Team/issues/1083
The Offers service is going to need access to the StripeAPIService too,
so we must pull its initialisation out of this module up to the Ghost
application layer, which will allow us to pass a reference of the
StripeAPIService to wherever needs it.
refs https://github.com/TryGhost/Team/issues/1090
This 1-day version of Offers allows us to test the full flow of the
Offers feature without having to implement all of it. The focus here is
that we can pass an Offer ID when creating a Stripe Checkout session and
have it apply. Here we use hardcoded Stripe Coupons as we haven't yet
got persistence implemented for Offers & their related Stripe Coupons
no-issue
Without a return after ending the response, the code will continue to
attempt to send emails and then send another response which results in
an uncaught error.
refs https://github.com/TryGhost/Ghost/security/advisories/GHSA-65p7-pjj8-ggmr
The email address change flow was built on top of the unauthenticated
signin/signup flow. This meant that ownership of the email being changed
wasn't verified and allowed a malicious actore to change the email
address of arbitrary accounts to an email address which they controlled.
We remove the ability to change email addresses from the signin/signup
flow and instead create a dedicated, authenticated flow for changing
email address.
refs https://github.com/TryGhost/Team/issues/1054
We need to instantiate the MemberAnalyticsService so that we can start
listening to events and storing them, this is the minium glue code
required to get us going.
refs https://github.com/TryGhost/Team/issues/1057
This method will validate a token, and then return the member associated
with it. Rather than exposing token validation and coupling consumers to
the structure of the token response data.
refs https://github.com/TryGhost/Team/issues/873
This ensures that all requests to the API will include the mock
subscriptions for comped members. Allowing the Admin to correctly show
the subscription information after adding and editing members. As well
as having the correct information when navigating from the list of
members to an individual member.
no-issue
Previously we would not create an instance of the StripeAPIService if
Stripe was not configured, but that is not the case any more, instead we
have a configured flag on the service. The webhook route handler was not
updated to use this flag and so would attempt to handle webhooks without
having any of the required data. This would result in an uncaught error.
refs https://github.com/TryGhost/Team/issues/1006
When disconnecting from Stripe, we currently do not remove the webhooks,
this will result in the webhooks from Stripe failing, and tending toward
a 100% error rate, which will ultimately result in emails from Stripe
about the failing webhook.
In order to stop all of that from happening, we should make sure that we
actively remove the webhook from Stripe when disconnecting.
refs https://github.com/TryGhost/Team/issues/1006
As part of the work to handle cleaning up webhooks when we disconnect
from Stripe, I'm moving the logic to clear out the Stripe related data
from the database into a disconnectStripe method. This then allows us to
start handling the cleanup of webhooks via the Stripe API.
refs https://github.com/TryGhost/Team/issues/995
Since we reintroduced the comped status, we did not update the
subscription handling to correctly set members to a status of comped
when they were on a 'Complimentary' plan.
no-issue
Since updating the product repository to force transactions, the options
parameter was used in every call, meaning it wasn't optional any more,
which broke usage. This updates the parameter to have a default so that
existing usage still works.
no-issue
Since we run our product repository methods in transactions now we must
ensure that all database interations in the method use the transaction.
This adds the missing options to the reading of existing prices so that
they happen inside of the transaction.
refs https://github.com/TryGhost/Team/issues/982
These calls to the edit method were missing the transaction option from
the parent which meant that they ran outside of the transaction and
would cause the method to timeout.
no-issue
As subscriptions are a default relation, and we now require products to
populate subscriptions for comped members, we need to include products
by default when reading members.
refs https://github.com/TryGhost/Team/issues/986
The getMemberIdentityData is a relic of time past. Originally it was
used before we had anything like the member repository or bread
controller as a way for things inside of the Members ecosystem to get
access to member data.
This updates it to use the same interface as everything else for
fetching members so that we can rely on the shape of the data that we
consider a member.
This update will ensure that themes have access to the dummy
subscriptions created by the `read` method of the MemberBREADService.
refs https://github.com/TryGhost/Team/issues/979
This correctly handles updates to subscriptions so that if the product a
subscription is for has changed, we will remove the previous product, if
and only if there is not another subscription which gives access to it.
refs https://github.com/TryGhost/Team/issues/873
The `get` method of the member repository will return null when no
member is found - we must ensure that we don't attempt to call toJSON!
It is also possible for a member to not have any products, in which case
we should not attempt to iterate over them, and we can return early.
refs https://github.com/TryGhost/Team/issues/873
This is an unexpected state, but possible if the alpha version of tier
has been enabled previous to the events being added.
refs https://github.com/TryGhost/Team/issues/873
This adds a dummy subscription for each product that a member has
without an associated stripe subscription. It allows clients to deal
with things like a created date for comped members.
no-issue
The idea of this service is to sit infront of the repository and handle
application logic which does not belong at the data layer. The exact
naming and structure is TBC but this gives us a place to start pulling
logic out of the controllers, without having to mash it all into the
repository.
Also important to note is that is does not return instances of bookshelf
models, but a JSON representation of the model, this allows us to not
leak internal implementation to consumers.
refs https://github.com/TryGhost/Team/issues/873
This handles the creation of product events when a members access to
products is changed. This can happen on creation, update, and any
changes to stripe subscriptions.
We manually workout the difference between the current products and the
new products, and add the events accordingly.
no-issue
Calling ObjectId doesn't return a string a but an ObjectId object.
Whilst this object is cast to a string via the toJSON and toString
methods, this is not enough for MySQL. Instead we should explicitly cast
this to a string ourselves and the application level.
refs https://github.com/TryGhost/Team/issues/946
This adds the bulk edit method which handles bulk edit operations to members
to be used by the filtering feature. They have been combined into a single method
as that is how they are exposed to the API. This is definitely a candidate for a
refactor in the form of a service in front of the repository.
refs https://github.com/TryGhost/Ghost/commit/1dd52075
- Fixes bulkDestroy being passed the context
- Fixes passing options.search to the model layer
- Updates return value since the changes in referenced commit
