fix https://linear.app/tryghost/issue/SLO-82/query-error-unexpected-character-in-filter-at-char-1
- previously, we weren't handling a parsing error, and just bubbling it
back up the chain
- this would result in an InternalServerError somewhere, which caused
500s
- we can handle this, because it's just a bad filter
- this adds handling so we return a 422 upon receiving an invalid filter
ref https://linear.app/tryghost/issue/KTLO-1/members-spam-signups
- Some customers are seeing many spammy signups ("hundreds a day") — our
hypothesis is that bots and/or email link checkers are able to signup by
simply following the link in the email without even loading the page in
a browser.
- Currently new members signup by clicking a magic link in an email,
which is a simple GET request. When the user (or a bot) clicks that link, Ghost
creates the member and signs them in for the first time.
- This change, behind an alpha flag, requires a new member to click the
link in the email, which takes them to a new frontend route `/confirm_signup/`, then submit a form on the page which sends a POST request to the
server. If JavaScript is enabled, the form will be submitted
automatically so the only change to the user is an extra flash/redirect
before being signed in and redirected to the homepage.
- This change is behind the alpha flag `membersSpamPrevention` so we can
test it out on a few customer's sites and see if it helps reduce the
spam signups. With the flag off, the signup flow remains the same as
before.
refs KTLO-19
When we need to migrate subscriptions from a platform with platform
fees, we need to recreate the subscriptions. That can cause the same
subscription to be attached multiple times to the same member in Ghost.
This is a problem because all MRR, subscriptions and cancellations stats
are no longer correct. Ghost will add a MRR event for the duplicated
subscription from the start time, so there is a sudden peak in MRR and a
dip after the migration because all those duplicate subscriptions are
suddenly cancelled 'today'.
The migrator tool adds a ghost_migrated_to metadata field to the old
subscription. Ghost can use this to detect the old subscription and
delete the subscription and corresponding events.
fixes PROD-102
When a newsletter has a sender_email stored in the database that Ghost
is not allowed to send from, we no longer return it as sender_email in
the API. Instead we return it as the sender_reply_to. That way the
expected behaviour is shown correctly in the frontend and the API result
also makes more sense.
In addition to that, when a change is made to a newsletters reply_to
address we'll clear any invalid sender_email values in that newsletter.
That makes sure we can clear the sender_reply_to value instead of
keeping the current fallback to sender_email if that one is stored.
On top of that, this change correclty updates the browse endpoint to use
the newsletter service instead of directly using the model.
fixes https://github.com/TryGhost/Product/issues/3738https://www.notion.so/ghost/Member-Session-Invalidation-13254316f2244c34bcbc65c101eb5cc4
- Adds the transient_id column to the members table. This defaults to
email, to keep it backwards compatible (not logging out all existing
sessions)
- Instead of using the email in the cookies, we now use the transient_id
- Updating the transient_id means invalidating all sessions of a member
- Adds an endpoint to the admin api to log out a member from all devices
- Added the `all` body property to the DELETE session endpoint in the
members API. Setting it to true will sign a member out from all devices.
- Adds a UI button in Admin to sign a member out from all devices
- Portal 'sign out of all devices' will not be added for now
Related changes (added because these areas were affected by the code
changes):
- Adds a serializer to member events / activity feed endpoints - all
member fields were returned here, so the transient_id would also be
returned - which is not needed and bloats the API response size
(`transient_id` is not a secret because the cookies are signed)
- Removed `loadMemberSession` from public settings browse (not used
anymore + bad pattern)
Performance tests on site with 50.000 members (on Macbook M1 Pro):
- Migrate: 6s (adding column 4s, setting to email is 1s, dropping
nullable: 1s)
- Rollback: 2s
fixes https://github.com/TryGhost/Product/issues/4108
- Updates filters behind a new alpha feature flag so you can also filter
on members who have email disabled (because the email had a permanent
bounce, they reported spam or the email address is invalid)
- When returning members, we now also use the email_disabled flag to set
email_suppression.suppressed correctly (in case they are out of sync,
which should normally never happen).
closes https://github.com/TryGhost/Product/issues/4046
- when editing the member's email in Admin, the email_disabled field was
not recalculated, making it inconsistent with the suppression list
- now, if the new email is part of the suppression list, we set
email_disabled to true. Otherwise set it to false
refs https://github.com/TryGhost/Product/issues/3875
When a member had a comped subscription, the portal was showing an
incorrect expiry date. This was because the `expiry_date` was being set
to the `created_at` date of the subscription, rather than the
`expiry_date` of the comped subscription
fixes https://github.com/TryGhost/Product/issues/3752
- Added some extra tests for edge cases
- Updated handling of multiple subscriptions so they are handled better
- Canceling a subscription when the member still has other subscriptions will now get handled correctly where the status and products of the member stay intact
refs https://github.com/TryGhost/Product/issues/3648
- Refactored Members API RouterController.createCheckoutSession: Split the method into smaller parts so we can reuse individual parts for the upcoming donation checkout session.
- Wired up donation checkout creation
- Added donation events
closes https://github.com/TryGhost/Product/issues/3595
- when importing paid members with a coupon in Stripe, we currently
search for the corresponding offer in our database and attach it to the
subscription if found. However, if an offer doesn't exist in the
database, we do not create one and don't attach any offer to the
subscription
- with this change, we now support the creation of a new offer, based on
a Stripe coupon, if it didn't exist already
closes https://github.com/TryGhost/Product/issues/3593
- when importing members via CSV, it's now possible to add the value "auto" for the "stripe_customer_id" field. When this option is passed, the importer will search for a Stripe customer based on the email address provided
- if there are multiple Stripe customers with the same email address, the customer with the most recent subscription is returned
This prepares us to return a DTO rather than BookshelfModel to the serialiser
layer. When passing a BookshelfModel, the serialisation layer uses the model to
read from when building computed properties. By stripping values out in the
toJSON method it means that the DTO will be missing them and the computed
properties won't be able to be calculated. Instead we return ALL values to the
serialisation layer, and then strip out the ones that weren't requested in the
"clean" step.
This also inadvertently fixes the issue with `reading_time` requiring the
`html` field to be requested, we can now request just `reading_time`, as well
as have it included by default.
As discussed with the product team we want to enforce kebab-case file names for
all files, with the exception of files which export a single class, in which
case they should be PascalCase and reflect the class which they export.
This will help find classes faster, and should push better naming for them too.
Some files and packages have been excluded from this linting, specifically when
a library or framework depends on the naming of a file for the functionality
e.g. Ember, knex-migrator, adapter-manager
no issue
This commit removes the `memberAttribution` feature flag from the
codebase. Some CSS classes are not removed as removing them and updating
the associated CSS files have side effects sadly.
no issue
This pull request removes the `suppressionList` feature flag and all its
dependencies from the codebase. It makes the suppression list feature
the default and consistent behavior for all email events and
newsletters. It simplifies the UI, logic, and data related to email
events and newsletters. It affects several files in the
`ghost/admin/app`, `ghost/core/core`, and `ghost/members-api`
directories.
refs https://github.com/TryGhost/Team/issues/2674
When going to /#/portal/account when not signed in, you are redirected
to the login page. But once signed in, you aren't redirected back to the
account page. This fixes this issue by adding an extra and optional
redirect parameter when requesting a magic token via email.
This new parameter allows to override the default behaviour of using the
Referer HTTP header, which doesn't include the hash/fragment part of the
URL.
The referrer is already restricted to only allow redirects to the site,
not external URLs.
fixes https://github.com/TryGhost/Team/issues/2783
refs cb05fae5a3
The root cause of the issue was the fact we no longer checked for lack of `newsletters` property on member data before checking its `subscribed` property which is now deprecated. This caused a cascading effect where `subscribed:false` property on a member overrides the value for `newsletters` data. The check was accidentally removed in a previous bug fix.
So for members that were not subscribed to any newsletters, saving a newsletter subscription failed as they had their `subscribed` set to `false`, and it was resetting the newsletter subscription to empty always.
- by default, got retries failed requests, which is causing issues in
tests because we've disabled the network with `nock`
- this is causing huge idle time because got pauses before retrying
- this change disables the retries if we're running tests, so things are
more stable
no issue
The Stripe Mocker mocks the Stripe API in memory, to make it much easier
to test subscription flows. Currently it is more a POC to see if it
works well. It probably needs a bit more work to support more scenarios.
- Added new tests for the subscription stats endpoint for 3D secure +
free trial flows using the new Stripe Mocker
- Updated members admin api tests to use Stripe Mocker (+ added new test
for deleting members with Stripe cancellation)
- Some tests called mockStripe at the beginning, but that method did
nothing apart from disabling network (which is the default now), then
they mocked Stripe inside the tests file... so I've removed those
because those conflict with the new mocker that is enabled when calling
mockStripe. We'll need to port those over later.
Refs TryGhost/Team#2459
-upgraded got from v9.6.0 to v11.8.6 to support following redirects (and
other fixes)
-got v12+ requires ESM, so we do not want to upgrade further at this
time
-required changes to a few libraries that use externalRequests
-mention discovery service tests updated to test for follow redirects
fixes https://github.com/TryGhost/Team/issues/2542
fixes https://github.com/TryGhost/Team/issues/2543
fixes https://github.com/TryGhost/Team/issues/2544
- Hides incomplete subscriptions
- Shows Past Due subscriptions
- Fixed UI issues with 3+ subscriptions
- Fixed missing complimentary subscription when one subscription was
incomplete/inactive
- Fixed sending a paid subscription started email for incomplete
subscriptions. This change also required us to actually send the email
when the incomplete subscription eventually becomes active. So the
introduction of a new `SubscriptionActivatedEvent` made sense/was
required (because sending a SubscriptionCreatedEvent again would cause
other issues).
refs https://github.com/TryGhost/Team/issues/2235
We found some cases which can cause a site to have member emails that have invalid characters like `member@example.com�`. This happened due to the `validator` version used by Ghost not able to catch some specific cases as invalid email, allowing members to be created with them either via Admin or Importer or direct signup. Portal UI already blocked these email as invalid. This change:
- updates `@tryghost/validator` to include a latest version of email validator that catches these invalid cases
- doesn't allow member creation with invalid email like above
- doesn't allow existing member emails to be edited to invalid
fixes https://github.com/TryGhost/Team/issues/2386
**Issue:**
- When trying to import a member that already exists, and has
'subscribed' set to 'true' in the CSV, the newsletters the member is
subscribed to are reset to the default newsletters.
- When ediging a member with the API and setting `subscribed` to true,
the same happens.
**Cause:**
A faulty check for the `status` property of a newsletter.
Fixed and added a new E2E test.
refs https://github.com/TryGhost/Team/issues/2367
We're not releasing this immediately so need to put it behind the flag
so that we continue to send emails to the correct members.
refs https://github.com/TryGhost/Team/issues/2367
This ensures that a Member is not considered subscribed to any emails, so that
counts for newsletter recipients are correct. Eventually we will filter members
on their email suppression status but this is not implemented yet.
refs https://ghost.slack.com/archives/C02G9E68C/p1670215917451249
When a member is deleted, and we receive an opened event for an email to
that member. We threw an uncaught Bookshelf EmptyResponse error.
- This change makes fetching the member not a requirement when handling
that event in the last seen at updater.
- It also adds try catches for all event listeners in the last seen at
updater
closesTryGhost/Team#2313
- Added Sent event to Post analytics and Members feed. Now post can be
Sent or Received or Bounced.
- Excluded Delivered event from Sent filter on backend.
refs https://github.com/TryGhost/Ghost/security/advisories/GHSA-9gh8-wp53-ccc6
refs https://github.com/TryGhost/Toolbox/issues/465
- Bookshelf relations allows us to edit relational records by default, which was used liberally in the codebase.
- Not having a clear track record of editable relations left the model layer prone to triggering unwanted nested saves and created a vulnerability where members were able to edit newsletter settings.
- With explicit editable relations it's easier to keep track of relations having editable access to related records. Makes the relational data modification pattern safer to use too.
- Anyone running 5.x should update to 5.24.1
Credits: Dave McDaniel and other members of [Cisco Talos](https://talosintelligence.com/vulnerability_reports)
