refs https://github.com/TryGhost/Team/issues/1621
We want to allow previewing emails based on the selected newsletter. The post model doesn't get a newsletter attached until a publish occurs so we can't use `post.newsletter` and need to give the option of specifying which newsletter to preview via query params.
- added support for `newsletter` query param on the `GET /email_previews/posts/:id/` endpoint where the value is a newsletter slug
- updated `generateEmailContent()` signature to use an options object because the order of memberSegment/newsletter arguments doesn't matter and is difficult to reason about if not named
- adjusted `generateEmailContent()` to fetch the newsletter matching the provided slug, falling back to the default newsletter if no slug is provided
- `mysql2` doesn't support this and we haven't had it enabled for a long time
- if we need this feature, we can re-add it with a working implementation
refs: https://github.com/TryGhost/Ghost/issues/11407
- there's no need for us to process the whole document as it's only used for excerpts
- emails need the whole document, but they're managed seprately now
- this is more efficient and means we could potentially do it on the fly 🤔
refs: https://github.com/TryGhost/Team/issues/1609
- Split html-to-plaintext into a function for excerpts and emails
- Updated all usages so they use the correct function
- There's currently no difference between the two
refs: https://github.com/TryGhost/Team/issues/1609
- remove the 80 char wordwrap
- this makes the function the same as used in the post serializer
- updated references everywhere to use the same function
refs https://github.com/TryGhost/Team/issues/1596
- Renamed `newsletter_id` to `newsletter` option, the `newsletter` option expects a slug instead of an id
- Renamed `email_recipient_filter` to `email_segment` option
- Default `email_segment` to `all`. Ignored if no newsletter is set
- `email_segment` is ignored if no newsletter is set
- When reverting a post to a draft, both `newsletter` and `email_segment` are reset to their default values (null, all)
- Removed legacy mapping from old email_recipient_filter values 'paid' and 'free' (already a migration in place)
- Dropped legacy throwing errors when email_recipient_filter is paid or free in transformEmailRecipientFilter
- Reorganized transformEmailRecipientFilter parameters for the now required newsletter parameter
- Fixed an issue where the newsletter filter wasn't working because it wasn't in permittedoptions
- Fixed an issue where you could send to an archived newsletter
- Added an extra protection when scheduling to an active, and later archiving the newsletter
- Dropped support for `send_email_when_published` in API
- When importing posts we currently don't have a system in place to set the newsletter_id to map the `send_email_when_published` behaviour. Since this was already the case, I won't include a fix in this PR.
- Stripped `email_recipient_filter`/`email_segment` from Content API (https://ghost.slack.com/archives/C02G9E68C/p1652363211841359?thread_ts=1650623650.233229&cid=C02G9E68C)
- Updated `admin-api-schema` to 3.2.0, which includes the new email_segment property
- Contains a temporary fix for https://github.com/TryGhost/Team/issues/1626, where the `.related('newsletter').fetch` call fails when the newsletter relation is already loaded, because of the overridden `formatOnWrite` method.
Since the `email_recipient_filter` is no longer used without a newsletter, the `none` value is no longer used. A migration transforms all those values to `all`. This should be safe, because we only send an email now when newsletter_id is not null (scheduled posts should already have a newsletter_id, even if at the time of scheduling they didn't add the newsletter_id option, because at that time, we defaulted to the default newsletter).
Admin changes to make this work: https://github.com/TryGhost/Admin/pull/2380
refs: TryGhost/Team#1625
refs: TryGhost/Team#1558
- none of these settings are actively used anymore
Co-authored-by: Daniel Lockyer <hi@daniellockyer.com>
ref: https://github.com/TryGhost/Team/issues/1145
ref: 8f8b7e7364
- The /products/ endpoint was replaced with /tiers/ some time ago but we didn't finish the switch
- The work is complete now, so can remove the endpoint entirely and cleanup remaining usages
- portal now uses content API to load site data
- handles auto unsubscribe for multiple newsletters
- design refinements
- more test coverage
- better fixtures
refs https://github.com/TryGhost/Team/issues/1599
- adds `portal_*` settings to public settings endpoint
- adds calculated `firstpromoter_account` setting for public settings endpoint
- also adds Ghost `version` information
refs: https://github.com/TryGhost/Toolbox/issues/327
- lang / locale has had a lot of churn, but we decided this setting should always be locale
- session_secret is too generic as we have multiples of these
refs https://github.com/TryGhost/Team/issues/1583
- Check limits when unarchiving newsletters
- Added tests for more scenarios
- When editing/adding newsletters, the limit check happens in the same transaction.
- `limit-service` was bumped to add transactions support
- Added transaction support for edit in newsletter service
refs https://github.com/TryGhost/Toolbox/issues/283
- The header is needed to signal to the webhook subscribers the content version they are being served. This should imrove API version compatibility and allow for the client to handle incoming data better
refs https://github.com/TryGhost/Toolbox/issues/283
- In tests we need assurance that the triggering of webhooks has been finished before making assertions. Doing this was impossible with a previous fire-and-forget style of the request call.
- The change also adds an optional "request" parameter to be able to override the request library used internally - this is purely for testing purposes.
refs https://github.com/TryGhost/Toolbox/issues/283
- Current trigger module handling webhook paypload delivery isn't testable! It sucks to add features to it without assurance things still work
- Apart from expanding the test suite this changeset also needs live testing - setting up webhooks etc.
refs: https://github.com/TryGhost/Team/issues/1599
refs: f3d5d9cf6b
- this commit adds the concept of a frontend data service, intended for passing data to the frontend from the server in a clean way. This is the start of a new & improved pattern, to hopefully reduce coupling
- the newly added internal frontend key is then exposed through this pattern so that the frontend can make use of it
- the first use case is so that portal can use it to talk to the content API instead of having weird endpoints for portal
- this key will also be used by other internal scripts in future, it's public and therefore safe to expose, but it's meant for internal use only and therefore is not exposed in a generic way e.g. as a helper
refs https://github.com/TryGhost/Team/issues/1603
When previewing a scheduled/published post via Post editor menu > E-mail newsletter > Preview in browser. The e-mail template from the default newsletter was used instead of the newsletter that was selected when scheduling the post.
no issue
The message was removed somewhere, but the git history looks to be modified so I couldn't find the reference to when and why it was removed (the message seems to be added and removed in the same commit).
Added in: e849167472
No reference to where it was removed. But after the same commit above, it wasn't present any longer.
- These settings no longer exist, having been renamed to timezone and lang
- As of 5.0 we no longer need any kind of backwards compatibility outside of the importer
- We making breaking changes and cleaning up as many old code paths as possible
- We have not really exposed the admin Settings API, meaning backwards compatibility was more for internal use
- We will be changing lang back to locale, but that's a separate issue and won't need backwards compatibility
closes: https://github.com/TryGhost/Toolbox/issues/324
refs: https://github.com/TryGhost/Ghost/issues/14446
- Currently, if url is configured to http but a request is marked secure, Ghost will handle upgrading all internal URLs to https so that there are no mixed content warnings
- From 5.0 that feature is going away, in favour of strictly honouring the configured URL
- Ghost will serve URLs exactly as configured and won't upgrade http to https anymore
- This use case was common when Ghost was first built, but in 2022 the web is mostly https.
- The code needed to support the feature creates a lot of additional complexity & maintenance overhead, so removing this gives us space to do more cool and useful stuff in 2022
refs https://github.com/TryGhost/Toolbox/issues/309
- this column is not used and I was going to add `validation` to it but
it's better to clean it up and re-add the column if we need it again
- async/await makes the code easier to read
- also means we can get rid of usages of Promise, and some unnecessary usages of Bluebird
- this refactor is done for the 4 async helpers and the code that wraps async helpers so the whole pipeline is async/await and more understandable
refs: https://github.com/TryGhost/Team/issues/626
- calculated settings are simplified settings (booleans) that are based on other settings or data
- they make it easier for us to determine what state features are in elsewhere in ghost e.g. admin and themes
- this duplicates some of the members config concepts in the settings service
no issue
- When applying an incorrect limits config, or missing expected values, Ghost would not boot as the errors would interrupt this process, which should not happen
- This commit catches the error thrown by the limit-service on boot sequence and transforms it into a warning if it's an `IncorectUsageError`. Other errors are handled as before
- Added a test for the limit-service service
refs: e68cb8b314
- a couple of months ago when improving the test coverage here I found some weird behaviour with falsey values
- turned out it didn't matter at the time because we didn't have any settings that are false
- with the introduction of calculated settings we will have: https://github.com/TryGhost/Ghost/pull/14766
- whilst building that, I found settings that should be returned as false were being returned as null
- fixing it in a separate commit to keep the work clean
- by default, the model will auto-refresh when you save
- in MySQL, which doesn't support RETURNING, this is implemented by
doing a SELECT after your actual query
- `email_recipients` is a huge table and `processed_at` is not an
indexed column, so the query times can become realllyyyyyyy big
- given we don't even need the result of the model save, we can just
disable fetching all the affected records again
- Bookshelf gives us this ability in the form of `autoRefresh: false`
refs https://github.com/TryGhost/Toolbox/issues/309
- these stopped being added to the DB in v3 but there was never a
migration to clean up existing permissions and the role link in the DB
- we now have the utils to do this cleanly, so we can drop all the
permissions in this migration
- The recently refactored path matching code forgot to take into account that originalUrl can include the subdir
- Added more permutations to tests and ensured that all tests pass
- This means we don't have to worry about what sort of path we pass to the function, it'll figure out the version and api info
- this will drop the link between a role and a permission, and then
delete the permission
- required minor refactoring to aid in creating migrations for dropping
the links and permission
- I'm refactoring the utils to make way for dropping permissions but the
utils are in a real mess
- to aid my refactoring, I want to make the utils a folder, so the
migrations that require utils as a file need updating
- this commit does that
refs: https://github.com/TryGhost/Team/issues/1599
- add an internal integration for Ghost's frontend to talk to the content API
- this is so that portal and future features can access our APIs through the correct mechanism of an API key
refs https://github.com/TryGhost/Toolbox/issues/325
- this was used for an alpha proof-of-concept for member activity data
collection but we're rethinking the strategy so this is the easiest
way to ensure it can't be enabled when the database table has been deleted
refs https://github.com/TryGhost/Toolbox/issues/292
- Providing user-defined Integration name instead of API client's UserAgent gives a lot more control to instance administrators identifying which integration is being used incorrectly.
- It's best practice to create an Integration with a set of API keys per API client - which should be enough to identify an outdated one.
refs https://github.com/TryGhost/Toolbox/issues/309
- this table was used an an experiment for member analytics
- as we rethink the strategy, we can take the opportunity to clean the table up
- if the table doesn't exist, it's not necessarily a failure that we
should be informing the user about
- this brings the message wording in line with other log lines of
similar operations
refs https://github.com/TryGhost/Toolbox/issues/309
- this column is now a calculated value based upon the relation of a
member to a newsletter
- we should no longer need `subscribed`, so this migrations cleans up
the column in the DB
refs https://ghost.slack.com/archives/C02G9E68C/p1651939076681719
Cause:
- When a scheduled post was published via the post scheduler, no `newsletter_id` option is passed when editing the post.
- When editing a post via the posts service, without the `newsletter_id` option, the `newsletter_id` option is automatically set to the default newsletter's id.
- Inside the post model, this new `newsletter_id` was not saved, because it was already set, and changing it is prevented.
- The `mega` service wasn't using the (unchanged) post's newsletter_id, but used the option instead, which contained the default newsletter's id.
Fix:
- Always using the newsletter_id from the post and requiring the newsletter associated with a post to exist.
- This behaviour can be/is tested by publishing a scheduled post without any option.
Also cleaned up some `Object.assign` usages.
refs https://github.com/TryGhost/Team/issues/1546
- allows newsletters API to work with Admin API keys
- updates fixtures to add permissions to admin integration role for new sites
- adds migration to update existing sites to have correct permissions for role
- whitelists add/edit/read/browse on newsletters API for integrations
refs https://github.com/TryGhost/Toolbox/issues/309
- this commit adds a validation array of valid user `status` values to
the schema
- this also includes a migration to update users with invalid statuses
to `inactive`, which I've seen with `invited` and `invited-pending`
statuses that pre-dated proper invitations
- this also deletes tests that were wrong and written 7 years ago before
invites was added
- `subscribed` will soon cease to be a column on a member and I get errors
in tests that stem from this block of code because it no longer exists
- AFAICT this block of code is only used to verify the filter is valid
to filter on, which we should be able to do even if we remove
`subscribed`
- this commit removes `subscribed` from the filter
refs https://github.com/TryGhost/Toolbox/issues/309
- I originally started looking at this because I wanted to change the default of
`emails.recipient_filter` for old DBs to `status:-free`
- we changed these columns to a `text` type, which doesn't support
defaults
- the tables already have defaults set in the model, so the only change
needed here is to delete the `defaultTo` in the schema to avoid
confusion
- on the way, I ended up fixing 51498abb5c too
refs https://github.com/TryGhost/Toolbox/issues/309
- we're removing the OAuth prototype and the table was never used, so we
should be good to drop it
- this commit adds a migration to drop the table
refs https://github.com/TryGhost/Toolbox/issues/309
- we started off with "free" and "paid" in these columns but since moved
to NQL strings
- there is code in Ghost to rewrite these values to NQL strings, but we
still have posts/emails in the DB with these old values so they need
to be updated
- once this migration is merged, we can probably clean a lot of that
aliasing up
- this commit migrates values in `posts.email_recipient_filter` and
`emails.recipient_filter` from `free` to `status:free` and `paid` to
`status:-free`
- I've left the down as a no-op because we don't want to translate
values back, and we don't know which ones we originally migrated in
the first place
refs https://github.com/TryGhost/Toolbox/issues/230
- The fixture manager has to initialize User/Roles fixtures first to be able to insert multiple authors as a relation in post fixtures. Otherwise the posts could not find correct authors and were failing trying to assign default "owner user" to each post
- The order of running fixtures matters, and till now the order wasn't taken into account at all when populating the db
refs https://github.com/TryGhost/Toolbox/issues/230
- This is a first step to removal of `author_id` concept from the codebase. The aim is to see what parts break down from this changes an work backwards from there
refs https://github.com/TryGhost/Toolbox/issues/292
- The version mismatch notification emails were missing a URL of the endpoint that was being accessed by an outdated integraton
- Also squeezed in a refactor simplifying APIVersionCompatibilityService initialization
refs https://github.com/TryGhost/Team/issues/1571
- With the addition of multiple newsletters, all emails sent previously should be assigned to the default newsletter
- This will make sure that the sent count for the default newsletter displays correctly
refs https://github.com/TryGhost/Toolbox/issues/309
refs 2bfd8f8b7e
- we've previously had issues when dropping a column on `posts`
because it's a large table and it can take a veeeeeeery long time with
the default SQL produced by Knex
- we found a magic incantation that makes it super speedy (context in
commit above) => `, algorithm=copy`
- for that migration, we did it all manually but we can now change the
utility to always append this to the generated SQL so we don't have to
think about the specific table size when adding or dropping columns
- this changes the `addColumn` and `dropColumn` utilities to append the
string to SQL in MySQL, or just executes the SQL for SQLite
refs https://github.com/TryGhost/Team/issues/1595
- Since adding multiple newsletters, posts may be linked to a related newsletter
- We don't export newsletters, so the related newsletter_id doesn't exist and fails the FK check on import
refs: https://github.com/TryGhost/Toolbox/issues/229
- We're getting rid of the concept of api versions inside of Ghost
- Instead of storing the supposed api version a webhook was created with, store the current ghost version
- This way we can determine if anything signicant changes in future and we need to update webhooks or something
refs: https://github.com/TryGhost/Toolbox/issues/229
- Removed the final usage (logging only) of config.get('api:versions') and removed the config
- This is one of the last references to API versioning in the codebase & we don't need it anymore
- Cleaned up a random comment whilst searching for refs
refs https://github.com/TryGhost/Team/issues/1529
- the sender email addresses for newsletters require verification to set.
- this ensures there isn't a way around that by modifying an export file then importing it by setting it to null on import.
This pattern is similar to the current `members_from_address` setting which is excluded when importing.
refs: https://github.com/TryGhost/Toolbox/issues/229
- our api-key audience handling code is still relying on internal api version config
- the regex used is also buggy (it expects 3 parts, which isn't true without versions) and doesn't always match, in which case it can cause the tests to hang
- we already had some very similar code in the version-rewrite middleware which is also validates exact values for version and api type
- moved this code into a util inside api-version-compatibility-service
- using this code, all the tests still pass as is, but when I start to adjust them to cover more cases, none hang (test changes coming in a separate commit)
- updated usage of url-utils.urlFor to work with v3
Co-authored-by: Renovate Bot <bot@renovateapp.com>
Co-authored-by: Hannah Wolfe <github.erisds@gmail.com>
- When both parameters passed to `update` resolve to `undefined` we throw an `Empty .update() call detected` error
- This change always updates the `sender_email` to null rather than skipping the field
- `sender_reply_to` should still be `undefined` so we don't override an existing non-null value
refs https://github.com/TryGhost/Toolbox/issues/309
- I've just ran into a problem when deleting the `defaultTo` field on a
non-nullable `text` column in our schema because this validation
thinks there should be a value set
- `text` fields cannot have defaults so the schema is incorrect, and the
validation triggering is a bug that's preventing it from being
cleaned up
- the default is defined on the model so I don't think we're losing
anything here
refs https://github.com/TryGhost/Toolbox/issues/292
- This change allows to reuse existing pattern present in Ghost on the version mismatch service layer, where we define the contents of the sent email through email templates instead of plain text.
- Apart form templates, there's now failed request URL present in the data passed to the email template along with site title and site url
refs https://github.com/TryGhost/Team/issues/1579
- When writing to the database, the header_image is tranformed to the transformReady path
- When reading from the database, the transformReady path is transformed to an absolute path
- Includes a test when adding a newsletter
Migration:
- Updates all newsletter who have a header_image to make sure it is saved in transform ready format
- Down operation is required to work with the old model logic and transforms it back to an absolute format
refs https://github.com/TryGhost/Team/issues/1581
- The original migration to create the default newsletter omitted the from address and reply-to settings
- `sender_reply_to` and `members_reply_address` are both enums with the same values and copy straight across
- `members_from_address` had a default value of 'noreply' as the fallback, which is remapped to NULL in the newsletters table
- We apply the change to all newsletters (there should only be one outside of alpha) which haven't already been reconfigured
refs https://github.com/TryGhost/Team/issues/1558
- `members_from_address` is no longer configurable in Admin but was still used as a fallback
- This change removes the fallback so we default straight to noreply@<domain> when a from-address isn't set for a newsletter
closes: https://github.com/TryGhost/Toolbox/issues/315
- For all the current versioned URLs, rewrite the URL as unversioned
- Add the accept-version header
- Add the deprecation header
- Add the link header
- This then does the content-version middleware afterwards, ensuring that rewritten requests get this in the response
refs https://github.com/TryGhost/Team/issues/1495
For single newsletters, the unsubscribe link on emails auto unsubscribed member from the newsletter. In case of multiple newsletters, we were missing the newsletter information on unsubscribe URL that will allow us to auto unsubscribe member from that specific newsletter as they intended, while allowing them option to manage other newsletter preferences via Portal UI. This change -
- adds relevant newsletter UUID on the unsubscribe url in emails
- allows portal to auto unsubscribe members from desired newsletter
refs https://github.com/TryGhost/Team/issues/1577
The last seen at was being updated via the model rather than the
respository, which mean that the default relations were not being
loaded.
This fix updates the repository to load the newseletters relation,
updates the last seen at updated to use the repository and updates the
output serializer to handle missing newsletter relation.
We also update all packages relying on the domain-events package to
ensure that they're all using the same version.
Co-authored-by: Fabien 'egg' O'Carroll <fabien@allou.is>
refs https://github.com/TryGhost/Toolbox/issues/292
- Following the concept of having as little code in Ghost core as possible :) The email content generation is also needed to be reused in the version mismatch handling package.
refs https://github.com/TryGhost/Toolbox/issues/292
- There's a need to reuse these utils in the version mismatch notification service. Having loads of tightly coupled dependencies makes it super hard to rip out this module for reuse
- It's a groundwork for extraction of the email-utils package
- Rewrote the unit tests that were written for these utils previously - they weren't testing anything useful. The goal of this util is to generate specific content based on provided data and available templates - now the tests do test those specific things, not the mailer itself!
- This is preparation work for getting rid of API versions
- The existing code used api versions for members, but the members API is not versioned
- This caused a bug as issuer was begin set to {{admin_url}}/ghost/api/undefined
- The updated code returns the correct value and is unit tested
- Whilst cleaning up I also swapped the usage of urlUtils to consistently use urlFor, as that is our main helper
closes https://github.com/TryGhost/Team/issues/1570
- Includes utils to disable foreign key checks when dropping nullable from columns
- Migration to drop nullable from members_subscribe_events.newsletter_id
refs https://github.com/TryGhost/Team/issues/1564
- While creating a newsletter
- While editing a newsletter
- Includes tests and updated snapshots
- Igored sort_order to snapshot test because sort_order is different in CI than locally so had to ignore it in the snapshot.
This reverts commit 9a0d143fb1
- main is now a precursor of 5.0 which should have email notification turned on
- had to add missing `err` in the errorHandler middleware as it was not triggering the versionMissmatchHandler otherwise
closes: https://github.com/TryGhost/Toolbox/issues/319
- at the moment, content-version is only set if one of our endpoints touches the request
- this was demonstrated in the e2e tests, where many of the tests that set accept-version did not receive accept-version
- by moving the middleware out of the http module and onto the api app we ensure it's always done
- I put the code in the api-version-compatibility service to keep it all co-located
- ideally we will refactor that service slightly so it only exposes middleware
closes: https://github.com/TryGhost/Toolbox/issues/317
- Added two tests for unknown versions with accept-versions set ahead and behind
- Ahead passes, but we get an error for behind
- Refactored the api-version-compatibility-service to expose its own middleware so the init sequence is correct
refs https://github.com/TryGhost/Toolbox/issues/226
- We are on the finish line to release 5.0 - the instance should be validating it's themes agains v5 set of rules.
- The prominent deprecations are `{{author}}` and `@blog` helpers removals - will throw 'fatal' errors when encountered.
- we have two JSON error response formats one old, one new (v2)
- we couldn't use the new one everywhere before without changing the response from older versions
- that is totally irrelevant in Ghost 5.0 as there is only one API version
- therefore we can and should use the new response format everywhere
- eventually we should rename it so it doesn't have v2 in it
refs https://ghost.slack.com/archives/C02G9E68C/p1651484563907609
- When the site has an empty name, it is set to `NULL` in settings.
- The name column is not nullable in newsletters, breaking the migration in that case.
- Fixed by excluding all non-nullable columns
- Replaced JS Date object with raw SQL `CURRENT_TIMESTAMP`
refs https://github.com/TryGhost/Toolbox/issues/308
- we have a pattern of using plurals around Ghost but this was singular
- this shouldn't change any API functionality, it's just code
refactoring
closes: https://github.com/TryGhost/Toolbox/issues/296
- This is a small change to permit any known API version to redirect to an unversioned URL
- We include v2 because although it should have been deleted in 5.0 anyway, in the spirit of the change away from versioned URLs there's
absolutely no sense in forcing people to update clients that still work for no reason.
- We use a 307, because this preserves the original HTTP method, allowing POSTS, PUTs and DELETEs through as well as GETs
- We set the accept-version header on the redirect, meaning that for example with a request to the old /v4/ api, Ghost will respond as though
the client sent `accept-version: v4.0` and if there are known breaking changes, it may choose to inform the admin and owner users of these
refs https://github.com/TryGhost/Toolbox/issues/308
- this endpoint is currently used to send a test email with the post
- it currently returns a 200 with whatever the response of the mail
service is
- this body isn't used in Admin nor is useful generally because it just
contains the ID of the mailgun response
- it's better than we change it to 204 and no response
- this commit does that and updates the tests
refs https://github.com/TryGhost/Toolbox/issues/268
- When the user is removed our current pattern was deleting their posts. This didn't work well and created all sorts of problems
- As a solution we now reassign any posts that are only authored by the deleted user to the owner user
- This change also reduced the dependency on "author" field
refs https://github.com/TryGhost/Toolbox/issues/308
- I recently pluralised the API endpoint but never made the changes to
the controller file and everywhere else it's needed
- this commit cleans up that inconsistency so it should be clearer
refs https://github.com/TryGhost/Toolbox/issues/308
- these endpoints return no body but they were implemented to return a
200 error code
- 204 would be more suitable so this commit changes that for v5
- also removes the passthrough serializer and updates the snapshots to
reflect the changes
