no-issue
* Added SingleUseTokenProvider to members service
This implements the TokenProvider interface required by members-api to
generate magic links. It handles checking if the token is expired and
pulls out any associated data.
Future improvments may include the email in the error for expired
tokens, which would make resending a token simpler.
* Passed SingleUseTokenProvider to members-api
This sets up the members-api module to use the new single use tokens
* Installed @tryghost/members-api@0.30.0
This includes the change to allow us to pass a token provider to the members-api
no-issue
This is a model for the tokens table, which handles the single use
aspect by customising the `findOne` method to automatically destroy the
model after reading from it
no-issue
refs: https://github.com/TryGhost/Members/commit/63942f03
The above commit updated all magic-link methods to be async, this change
ensures that we will handle a Promise return when it's updated in Ghost.
`await` has no effect on non Promise return values so it's safe to add
this now.
no-issue
After discussion with Matt, we decided that 192 bits for the token is a
good number, as it has no padding when base64 encoded and is more secure
than 128 bits, whilst still a managable size.
no-issue
This is a table to store single use tokens for use in magic links, the
columns are as simple as possible at the moment and are designed as:
id - standard ObjectID like all of our tables
token - 128bit base64 encoded string
data - arbitrary data to store against the token
created_at - timestamp to allow for expiry to be implemented for tokens
no issue
- In a recent change to ownership verification email flow, we changed the FROM address of ownership verification mails to use the same email as the one we are verifying, aka TO address.
- Email clients like Gmail flags off such emails as possible spam
- Fix updates the `FROM` address to `noreply@domain.com` where domain.com is domain for TO address
- In case the TO is already noreply@domain.com, we use no-reply@domain.com to bypass the same address restriction.
no issue
- The new Portal config flag allows switching on Portal conditionally with config
- The dev experiment flag still works for enabling Portal
- The flag currently defaults to `false` as Portal is still a beta feature and switched off by default
- We expose it on the admin api config endpoint so that the Ghost-Admin client can use it to conditionally render Portal settings
no issue
- Members site data was not appending blog domain for default support address which is `noreply`
- The change allows Portal to use default support address correctly
requires https://github.com/TryGhost/Ghost/pull/12192
- added initial `EmailBatch` and `EmailRecipient` model definitions with defaults and relationships
- added missing `post` relationship function to email model
- fetch member list without bookshelf
- bookshelf can add around 3x overhead when fetching the members list for an email
- we don't need full members at this point, only having the data is fine
- if we need full models later on we can push the model hydration into background jobs where recipient batches are fetched ready for an email to be sent
- bookshelf model instantiation of many models blocks the event loop, using knex directly keeps concurrent requests fast
- store recipient list before sending email
- chunk already-fetched members list into batches and insert records into the `email_recipients` table via knex
- chunked into batches of 1000 to match the number of emails that Mailgun accepts in a single API request but this may not be the absolute fastest batch size for recipient insertion:
| Batch size | Batch time | Total time |
| ---------- | ---------- | ---------- |
| 500 | 20ms | 4142ms |
| 1000 | 50ms | 4651ms |
| 5000 | 170ms | 3540ms |
| 10000 | 370ms | 3684ms |
- create an email_batch record before inserting recipient rows so we can effeciently fetch recipients by batch and store the overall batch status
no issue
We want to store a list of recipients for each bulk email so that we have a consistent set of data that background processing/sending jobs can work from without worrying about moving large data sets around or member data changing mid-send.
- `email_batches` table acts as a join table with status for email<->email_recipient
- stores a provider-specific ID that we get back when submitting a batch for sending to the bulk email provider
- `status` allows for batch-specific status updates and picking up where we left off when submitting batches if needed
- explicitly tying a list of email recipients to a batch allows for partial retries
- `email_recipients` table acts as a join table for email<->member
- `member_id` does not have a foreign key constraint because members can be deleted but does have an index so that we can efficiently query which emails a member has received
- stores static copies of the member info present at the time of sending an email for consistency in background jobs and auditing/historical data
refs #2635
- Adds 'Location' header to endpoints which create new resources and have corresponding `GET` endpoint as speced in JSON API - https://jsonapi.org/format/#crud-creating-responses-201. Specifically:
/posts/
/pages/
/integrations/
/tags/
/members/
/labels/
/notifications/
/invites/
- Adding the header should allow for better resource discoverability and improved logging readability
- Added `url` property to the frame constructor. Data in `url` should give enough information to later build up the `Location` header URL for created resource.
- Added Location header to headers handler. The Location value is built up from a combination of request URL and the id that is present in the response for the resource. The header is automatically added to requests coming to `add` controller methods which return `id` property in the frame result
- Excluded Webhooks API as there is no "GET" endpoint available to fetch the resource
closes#12045
- When member's email is updated to an already existing email of different member it caused table's unique constraint error, which was not handled properly.
- Added handling for this error similar to one in members `add` method.
closes#11999
- When the routes.yaml file changes (manually or through API) we need
to store a checksum to be able to optimize routes reloads in the future
- Added mechanism to detect differences between stored and current routes.yaml hash value
- Added routes.yaml sync on server boot
- Added routes.yaml handling in controllers
- Added routes hash synchronization method in core settings. It lives in core settings
as it needs access to model layer. To avoid coupling with the frontend settings it accepts
a function which has to resolve to a routes hash
- Added note about settings validation side-effect. It mutates input!
- Added async check for currently loaded routes hash
- Extended frontend settings loader with async loader. The default behavior of the loader is
to load settings syncronously for reasons spelled in 0ac19dcf84
To avoid blocking the eventloop added async loading method
- Refactored frontend setting loader for reusability of settings file path
- Added integrity check test for routes.yaml file
closes#12060
- A 500 error what happening when invited user provided an email that is associated with an existing user
- Additional validation for existing email address was added to prevent invalid data hitting db constraint error
refs #11999
- The `routes_hash` setting will be used during the boot process to update the hash
of currently loaded routes.yaml file in case it's different from last restart
no issue
- The async/await syntax makes it easier to reason about the code. Because adding 'Location' header is in the works it's a prep-work in a sense
closes https://github.com/TryGhost/members.js/issues/94
- The member-api package was recently updated to work directly with models and needs explicit `withRelated` options to attack relations
- Without options, the endpoint was returning the default member data without subscriptions attached, which in Portal showed paid member as free
- Fix updates the middleware for updating member data to correctly pass the relations needed to populate the member
no issue
- By default for new sites, support address is set same as from address to `noreply` , with full email address using the domain for `@`
- For newsletter emails, the support address was missing the default site domain to be added to address if its `noreply`
- Fix updates the support address to use the same format as from address and add relevant domain for default case
no refs
- The `update` method in members-api package was edited to return Model object instead of JSON directly - TryGhost/Members@a28bcc5
- This unsubscribe handler was returning the raw member object returned from `update` method, which is now a model object and not able to access `member.email`
- Fix updates the unsubscribe request handler to return the member JSON again
no issue
- members who have trial subscriptions added directly via Stripe will have a status of `"trialed"` in their Ghost subscription
- the `paid: true` filter was not taking that into account meaning trial users were not receiving newsletters sent to paid members even though they have a "paid" subscription
no issue
- We used existing "from" address as sender for mails sent to new email address for verification, but that breaks the flow to update if the current "from" address has DMARC policy set.
- This updates the flow to always send the ownership verification email TO the new address and FROM the new address which both verifies the email deliverability for new address and ownership
no issue
- The newsletter emails are sent out with `from` address as sender
- The new `members_reply_address` setting is now used to set reply-to address for emails, which can be either newsletter or support address
no issue
- Member auth emails were previously using the `from` address as sender
- New `members_support_address` was introduced with default as original "from" address
- Auth emails use the new support address as sender
no issue
- Updated magic link generation and validation methods for email update API to handle new support address
- Updated importer to ignore the new support address as it can only be updated via verification
- Updated members service to listen on settings edit for new support/reply address fields as well
- Updated tests to include the new settings
no issue
- Added default settings for the two new setting fields - `members_support_address` and `members_reply_address`
- Added migrations for setting group for new email settings
- Migration sets current from address as new support address default
- Added migration to set new support address same as from address
- Updated tests for new settings
- `members_support_address` - How members can reach for help with their account, public setting
- `members_reply_address` - Where you receive responses to newsletters
closes#12167
- Tags API v2 was ignoring `count.posts` include parameter.
- Regression was introduced with a3f693b472
- Introduced regression tests across all Content API versions to avoid similar bug in the future
no issue
- Added default settings for the two new setting fields - `members_support_address` and `members_reply_address`
- Added migrations for setting group for new email settings
- Migration sets current from address as new support address default
- Added migration to set new support address same as from address
- Updated tests for new settings
- `members_support_address` - How members can reach for help with their account
- `members_reply_address` - Where you receive responses to newsletters
no issue
- When an import was done and there were no "global labels" present Ghost created generic `import-[data]` label which later helped to find a specific batch of imported data
- It did not make sense to create such generic label when user provided their own unique label
- The rules that work now are:
1. When there is no global provided Ghost generates on and removes it in case there are no imported records
2. When there is a unique new global label provided no new label is generated, but the label stays even if there are no imported records
no issue
- tested performance between knex raw, knex `count()` and bookshelf `count()` and found no difference over 1000 iterations of each (each ~19,500ms +- 500ms for 104k members locally)
- switched to using bookshelf as the code is the simplest
no issue
- extract filtering of an collection into a separate function
- use extracted function in `findAll()` so that it's query behaviour matches `findPage()`
no issue
- for large result sets or complex queries the count query itself can be quite time consuming
- when `limit: 'all'` is passed as an option there's no need to perform a separate count query because we can determine the pagination data from the final result set
- skipped count query when `limit: 'all'` option is present
- re-ordered comments to be closer to the code they reference (ie, why we have our own count query instead of Bookshelf's `.count()`
no issue
- This is handled on input sanitization layer with date
format check in JSON schema validation, so there's no need to do this
check again in the importer.
no issue
- When bulk insert fails there is no transactional logic to revert
related records form being inserted. Also, previously there were no
attempts to "retry" the insert.
- To avoid complex retry logic, an iterative one-by-one insert retry
approach was taken. If this becomes a bottleneck in the future, the
retry algorithm could be improved.
- To avoid a lot of code duplication refactored model's `bulkAdd` & `bulkDestroy`
methods to use 'bulk-operations' module.
- Updated error handling and logging for bulk delete operations. It's very
unlikely for error to happen here, but still need to make sure there is
a proper logging in place to trace back the failure.
- Added debug logs. This should improve debugging experience and
performance measurements.
- Added handling for unrecognized errors. Handling inspired by current unrecognized
error handling by ghost importer -10e5d5f3d4/core/server/data/importer/importers/data/base.js (L148-L154)
no issue
- When no members are succesfully imported through CSV import process the import label should not be created. Otherwise after multiple failed attempts to import there are orphaned labels in the system
no issue
- updates `@tryghost/kg-default-cards` which contains two fixes
- removes email-specific output being added to post html (had no visual impact due to use of conditional comments but keeps rendered html smaller+cleaner)
- adds a background-url style to the thumbnail container to give two options for styling
- updates member email template styling to hide the `<img>` element in bookmark cards and use a background image instead to get consistent rendering across email clients
no issue
- adds a `members:emailTemplate` config object
- `showSiteHeader` - defaults to `true`, shows the site title and icon in member emails
- `showPoweredBy` - defaults to `false`, adds a "Publish with Ghost" button to member email footer
- updates member newsletter email template with hideable site header and "powered by" badge
closes https://github.com/TryGhost/members.js/issues/87
- The `update` method in members-api package was edited to return Model object instead of JSON directly [here](a28bcc5b2a)
- This caused the update member API on member endpoint to return partial response only as most properties couldn't be fetched
- Fix updates the middleware to correctly call `toJSON` before formatting response
* 3.30.2:
Bumped @tryghost/members-api to 0.28.1 in lockfile
Bumbed @tryghost/members-api to 0.28.1
🐛 Fixed unable to delete member when stripe is connected
refs #12127
- Adds new `editSubscription` endpoint for members admin API which allows updating individual subscription for a member - `PUT /members/:id/subscriptions/:subscription_id/`
- `editSubscription` has same permissions as member's `edit` endpoint
- Currently allows toggling of cancellation at period end for an active subscription
no issue
- Similar handling to one introduced in 8418c829de
- Having granular tracking for failed to remove id's would make it possible to return more specific errors to the client
