refs https://github.com/TryGhost/Product/issues/4153
- We need use the `created_at` timestamp in the new AdminX offers. The
API doesn't return that value.
- With this change the API returns the created_at property so that we
can consume it.
---
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖[[deprecated]](https://githubnext.com/copilot-for-prs-sunset)
Generated by Copilot at dc282af</samp>
This pull request adds a `createdAt` property to the offer domain model,
data transfer object, and repository. This allows tracking and auditing
the creation and modification of offers and offer codes in
`ghost/offers`.
fixes https://github.com/TryGhost/Product/issues/3738https://www.notion.so/ghost/Member-Session-Invalidation-13254316f2244c34bcbc65c101eb5cc4
- Adds the transient_id column to the members table. This defaults to
email, to keep it backwards compatible (not logging out all existing
sessions)
- Instead of using the email in the cookies, we now use the transient_id
- Updating the transient_id means invalidating all sessions of a member
- Adds an endpoint to the admin api to log out a member from all devices
- Added the `all` body property to the DELETE session endpoint in the
members API. Setting it to true will sign a member out from all devices.
- Adds a UI button in Admin to sign a member out from all devices
- Portal 'sign out of all devices' will not be added for now
Related changes (added because these areas were affected by the code
changes):
- Adds a serializer to member events / activity feed endpoints - all
member fields were returned here, so the transient_id would also be
returned - which is not needed and bloats the API response size
(`transient_id` is not a secret because the cookies are signed)
- Removed `loadMemberSession` from public settings browse (not used
anymore + bad pattern)
Performance tests on site with 50.000 members (on Macbook M1 Pro):
- Migrate: 6s (adding column 4s, setting to email is 1s, dropping
nullable: 1s)
- Rollback: 2s
fixes https://github.com/TryGhost/Product/issues/4118
The newsletter uuids were not passed when fetching all the members current newsletters. Therefore, Portals logic broke to remove all newsletters that matched the uuid that was passed to the unsubscribe link. No newsletters were removed, still the notification toast said that the member was unsubscribed from the newsletter.
ref https://github.com/TryGhost/Product/issues/4110
Made this change to increase clarity in data export
---
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 6c0508d</samp>
Renamed a column in posts export data and updated the corresponding test
case. This change makes the export data more consistent and clear for
users who have different member features enabled.
no issue
When a custom welcome page is set for a tier, the recommendations modal
didn't show. If recommendations were disabled, there was also no toast
to confirm the sign up.
To fix this, we'll need to set the success and action query parameters
on the welcome page, but only if it is not an external site.
fixes https://github.com/TryGhost/Product/issues/4102
E.g. you recommend myghostsite.com, while that site redirects all
traffic to [www.myghostsite.com](#):
The redirect causes CORS issues, which means we cannot detect
one-click-subscribe support.
- This is fixed by moving the whole detection to the backend, which has
the additional benefit that we can update it in the background without
the frontend, and update it on every recommendation change.
- This change also fixes existing recommendations by doing a check on
boot (we can move this to a background job in the future).
closes https://github.com/TryGhost/Product/issues/4046
- when editing the member's email in Admin, the email_disabled field was
not recalculated, making it inconsistent with the suppression list
- now, if the new email is part of the suppression list, we set
email_disabled to true. Otherwise set it to false
refs https://github.com/TryGhost/Product/issues/4088
The Content API should not expose the lexical/mobiledoc source content because it's not membership-gated and although not used at the present time may in future contain additional internal metadata. We were handling this for the more-typical `?formats` param but it was still possible to access this data using the `?fields` param.
- updated post mapper used in our API output serializers to strip the `mobiledoc` and `lexical` fields ready for API output
- credits to Prathap Puthran for reporting
no issue
- flag is no longer used in Admin so we can clean it up in Core too
- updated Post model to set blank document to `lexical` field rather than `mobiledoc` as a default value
- switched over to returning `mobiledoc,lexical` as default formats in Admin API
no issue
- Casper and Source theme files were out of date — this commit updates the theme fixtures, and fixes up a few tests to pass with the updated themes
closesTryGhost/Product#4032
- the api flag ?convert_to_lexical converts a mobiledoc string to
lexical
- if run on a post/page with lexical content, would null it out
closes https://github.com/TryGhost/Product/issues/4007
- bumps Koenig packages containing extended TextNode and HeadingNode with extra DOM parsing support for the non-semantic HTML generated by Word when copy/pasting
fixes https://github.com/TryGhost/Product/issues/4005
We no longer use the 'reason' of a recommendation, but allow a flexible
description instead. Because this is a breaking change in the API, we do
this before making this feature GA.
- Added new database utils for renaming a column
- Added new migration to rename the column
- Updated all references in code
fixesTryGhost/Product#3970
- When saving a tier, it is impossible to change the capitalization of a
benefit
- The logic when saving a tier matches benefits by `name.toLowercase()`
and then overwrites the incoming change with the name from the previous
version of the benefit
- This changes the logic to match benefits the same way using
`name.toLowercase()`, but use the _incoming_ benefit's capitalization
rather than the old benefit's, to allow users to e.g. change 'TEst
benefit' to 'Test benefit'
Promoted our beta editor to the default editor. Keep an eye on (or subscribe to) https://ghost.org/changelog/ for release announcements with full details.
- moved the beta editor (Lexical-based editor) to the default editor; all pages and posts will now use it
- all mobiledoc (previous editor) posts will remain mobiledoc until opened in the editor at which point will be converted to Lexical on the fly and open in the new editor
refs https://github.com/TryGhost/DevOps/issues/83
- this will now continue use the dev server assets if we tell it to,
or copy the dependency package files to the built folder otherwise
- removes `editor` from config API because it's no longer needed
- removes dependency on `editor.url` in tests, as this no longer exists
- edits dev script to pass dev server URL as env var
- adds `@tryghost/koenig-lexical` dependency to Admin
refs TryGhost/Product#3510
- Added `TryGhost/Source` as a submodule in `ghost/core/content/themes` so `Source` will ship with Ghost (along with Casper)
- With this change, new installs will use `Source` as the default theme. Existing sites will have `Source` installed, but not activated, as this is a large change and we don't want to drastically change existing sites without warning. Users can upgrade to use `Source` simply by clicking 'Activate' in design settings.
- Updated protections to prevent users from uploading their own conflicting version of `Source`
refs https://github.com/TryGhost/Product/issues/3957
This changes how we fetch recommendations:
- Relations can be included in one query instead of extra queries
- Sorting is now possible by click or subscriber counts
no issue
- bulk edit actions bypass the Bookshelf model hooks which meant our page reset behaviour in `onSaving` and `onDestroyed` was not being hit
- added overrides to `bulkEdit` and `bulkDestroy` to add the same page-reset behaviour any time we have a bulk edit or destroy
no issue
- including a body snapshot for the pages API collection card tests causes issues because the generated HTML is dynamic and contains post creation times meaning the snapshot was unstable
- removed the body snapshot for the tests concerned for now as they are mostly there to catch saving issues rather than rendering issues
no issue
- act as regression tests for internal collection code changes
- useful to test as we've hit missing transaction passthrough for sqlite a couple of times that wasn't caught
refs https://github.com/TryGhost/Arch/issues/95
- We are releasing with a slight risk of failing over, so need a switch to allow disabling collections on hosted environment.
refs https://github.com/TryGhost/Arch/issues/95
Rather than storing all of the relations between the latest collection and
posts, we know that it contains all posts. This means we don't have to keep the
collections posts in sync. Instead we can fetch them from the posts table. This
saves a lot of work during recalculation.
refs https://github.com/TryGhost/Arch/issues/95
Rather than a big nested loop to reconcile the in-memory vs. persisted
PostCollections we can instead use the events to know which rows we have to
delete and which we have to insert. This removes a tonne of work.
This implementation isn't perfect, and misses cases where the same post is
added and removed, our use-cases don't currently support that however.
no issue
Collection cards contain dynamic data that can change when there's any change to a published post but in Ghost all post/page content is rendered once on save and stored as a static string meaning we need a new approach for triggering a re-render of pages that plays well with caching.
- fixed typo in the relations/authors code that meant we weren't correctly calling the prototype method on the Post model inside the `onFetchedCollection` event handler
- updated Post model to clear the `html` field of all pages when saving or deleting a published post
- updated Post model to re-render `html` fields when fetching individual posts or a collection of posts
- modified `insertExtraPostsTags` fixture util to wrap it's concurrent post edits in a transaction otherwise MySQL errors because it hits a deadlock
closes https://github.com/TryGhost/Product/issues/3818
- instead of fetching all recommendations and matching URLs on the frontend, we now query the database directly to find an existing Recommendation by URL. When comparing URLs, we don't take into account the protocol, www, query parameters nor hash fragments
fixes https://github.com/TryGhost/Product/issues/3911
For now we decided that we don't want to enable one-click-subscribe in
case a site has a required checkbox (which isn't shown during the
one-click-subscribe flow)
This reverts commit 3e9da6df0c.
- changes introduced an error fetching `/admin/pages/` when using MySQL
- "The values in where clause must not be object or array"
no issue
Collection cards contain dynamic data that can change when there's any change to a published post but in Ghost all post/page content is rendered once on save and stored as a static string meaning we need a new approach for triggering a re-render of pages that plays well with caching.
- fixed typo in the relations/authors code that meant we weren't correctly calling the prototype method on the Post model inside the `onFetchedCollection` event handler
- updated Post model to clear the `html` field of all pages when saving or deleting a published post
- updated Post model to re-render `html` fields when fetching individual posts or a collection of posts
- modified `insertExtraPostsTags` fixture util to wrap it's concurrent post edits in a transaction otherwise MySQL errors because it hits a deadlock
fixes https://github.com/TryGhost/Product/issues/3830
This endpoint is required for recommendations to work: admin-x loads the incoming recommendations by querying the mentions endpoint. If the mentions flag was not enabled, this endpoint wasn't available.
refs https://github.com/TryGhost/DevOps/issues/80
- as part of moving Admin-X-Setting towards GA, we want to change it from
loading the settings externally via a CDN, to bundling it in with
Admin
- the bulk of the changes here are removing the config in Ghost, setting
up the copy to the Admin assets dir, and loading the new path in Admin
- several other changes have come along the way as I've cleaned up
unneeded code
refs https://github.com/TryGhost/Arch/issues/86
- Creating bookshelf models for each collection_post relation created a
massive overhead. On a dataset with 500k collections_posts records the
timing was roughly 7s comparing to 810ms after the optimization.
- Optimized memory and performance of collections fetching by querying post
ids only by default
fixes https://github.com/TryGhost/Product/issues/3900
1. The service never returns a Recommendation Entity, but always plain
objects (which for now is the same as Recommendation without the
methods).
2. Updated the controller to be more readable and minimal (we keep this
controller, in addition to the existing endpoints and serializers)
- The controller does minimal validation and allows for type checking
(so we get compile time errors in case the service expects new fields)
- The controller uses the `UnsafeData` class to easily validate the
input from requests, and throws appropriate errors (with correct field
descriptions — "Expected a string at recommendations.0.title") without
too much boilerplate code. In addition the interface is typed, so we get
compile errors if there are breaking changes in the service.
- Removed `EntityWithIncludes`, since we now use plain objects, we
inject the relations directly into those plain objects (with some new
types that add type support)
- Added new tests to make sure that edits only affect the given fields,
and never undefined fields
closes https://github.com/TryGhost/Product/issues/3818
- in Admin, when adding a recommendation, the URL is compared against all existing ones. If the URL is already recommended, the publisher is shown an error: "A recommendation with this URL already exists.". Protocol, www, query parameters and hash fragments are ignored during the URL comparison.
- on the backend, there is another uniqueness validation for the recommendation URL. This check is redundant when adding a recommendation from Admin, but helps to keep data integrity when recommendations are added through other paths (e.g. via the API)
refs https://github.com/TryGhost/Product/issues/3875
When a member had a comped subscription, the portal was showing an
incorrect expiry date. This was because the `expiry_date` was being set
to the `created_at` date of the subscription, rather than the
`expiry_date` of the comped subscription
https://github.com/TryGhost/Arch/issues/90
- When a post.deleted event is emitted the original 'data' object does not contain an 'id' property. The logic in collections service assumes the id would be present to update the collections efficiently.
refs https://github.com/TryGhost/Arch/issues/86
bookshelf-relations was generating tonnes of select queries from the
posts table in order to update the relations. We've instead implemented
this ourselves, so as to avoid the superfluous fetches. Working closer to
the db like this is nice, and makes you think more about performance.
This logic could be pulled out into a util (not bookshelf plugin) where
it could be used explicitly, but with the complexity hidden, we'll see ig.
refs https://github.com/TryGhost/Arch/issues/87
- The newsletters in members payload have leaked internal properties from Public Members API. The code skipped the output serialization step, which is now in place.
- The newsletter resource returned from the API consistently returns these properties:
id,
name,
description,
sort_order
refs https://github.com/TryGhost/Arch/issues/87
- The Members Admin API and members.* webhooks were returning too many fields in the nested `newsletters` objects. There was no "allowlist" serializer for the newsletter object, which meant every time we add a new field to the database we would unintentionally return extra fields without a second thought.
- With this change only following fields will be returned with `members[x].newsletters[x]`:
'id',
'name',
'description',
'status'
refs https://github.com/TryGhost/Arch/issues/87
- Round 2 for the previous commit. Removes use of `anyArray` for all
- Using `anyArray` in snapshot test is an anti-pattern which leads to leaking output fields unintentionally when the API changes.
- Adding these fixes is fundamental work before changing the output of 'member.newsletters' property
refs https://github.com/TryGhost/Arch/issues/87
- Using `anyArray` in snapshot test is an anti-pattern which leads to leaking output fields unintentionally when the API changes.
- Adding these fixes is fundamental work before changing the output of 'member.newsletters' property
no issue
- Do not set ?ref in recommendations if analytics is disabled
- Do not send url_history if analytics is disabled
- Expose outboundLinkTagging as a public setting
fixes https://github.com/TryGhost/Product/issues/3851
- Order was not applied via the CRUD plugin
- Removed usage of CRUD findAll, and swapped to Bookshelf fetchAll
instead, to decrease dependencies of invisible Bookshelf plugins logic
- Reverted page and limit options possibility via findAll method
fixes https://github.com/TryGhost/Product/issues/3822
fixes https://github.com/TryGhost/Product/issues/3838
This PR became a bit big because it affected multiple parts of Ghost
that needed to be updated to prevent breaking anything.
### Backend
- Added pagination to the recommendations API's
- Updated BookshelfRepository template implementation to handle
pagination
- Allow to pass `page` and `limit` options to Models `findAll`, to allow
fetching a page without also fetching the count/metadata (=> in the
repository pattern we prefer to fetch the count explicitly if we need
pagination metadata)
- Added E2E tests for public recommendations API (content API)
- Extended E2E tests of admin recommendations API
### Portal
- Corrected recommendations always loaded in Portal. Instead they are
now only fetched when the recommendations page is opened.
### Admin-X
- Added `usePagination` hook: internally used in the new
`usePaginatedQuery` hook. This automatically adds working pagination to
a query that can be used to display in a table by passing the
`pagination` and `isLoading` results to the `<Table>`
- Added placeholder `<LoadingIndicator>` component
- Added a loading indicator to `<Table>`. This remembers the previous
height of the table, to avoid layout jumps when going to the next page.
closes https://github.com/TryGhost/Product/issues/3803
Previously when the beta editor was enabled, using `?source=html` to create posts via the API would create posts in the old editor rather than the beta. This change switches conversion over to the new editor format when the beta is enabled so the full flow can be tested.
- added `htmlToLexicalConverter` method to our lexical library
- updated post and page input serializers to add html-to-lexical conversion when the beta editor is enabled
- updated post model to handle the mobiledoc+lexical co-existing state
- this is a special case that is only valid for `?source=html` because providing both directly via the API is prohibited
- we need the extra check here because at the input serializer layer we don't have access to the model to check if we're updating a mobiledoc post or a lexical post so the serializer sets both formats on a `?source=html` request when the beta is enabled and lets the model handle choosing the correct one
fixes https://github.com/TryGhost/Product/issues/3820
- This adds a new public site endpoint in the members API to check if a
site can offer the one-click-subscribe feature
- This is implemented on the members API as a copy of the `site`
endpoint because the admin API site endpoint is protected by CORS and
mainly because it can be served on a different domain than the
recommended site and this is hard to detect reliably from the frontend
- Added a new calculated setting `allow_self_signup`, which can replace
the setting that is currently used in Portal (best to do this after a
release otherwise we risk creating issues if a patch release happens)
refs https://github.com/TryGhost/Arch/issues/80
refs 3960bfac1d
- The killswitch (a setting in host settings) is needed to control the feature on a hosted environment, so we can safely turn it off if it causes any major issues.
refs https://github.com/TryGhost/Arch/issues/73
This is just an initial stab at making sure we don't introduce extra DB
queries related to collections without being aware of it.
refs https://ghost.slack.com/archives/C02G9E68C/p1692816097875899
- With introduction of extra e2e test coverage for Collections some tests started to fail at random. The root issue here was the transaction processing collections was started before the original bulk action (bulk edit, bulk publish/unpublish, etc.) was fully committed. The full transaction commit happens with the bulkAction method return inside of `if (!options.transacting) {` block.
refs https://github.com/TryGhost/Arch/issues/77
- We were missing e2e test coverage for when the tag used in collection filters was removed. This changeset improves the situation.
refs https://github.com/TryGhost/Arch/issues/77
- During initial development we have missed to support collections update when tags are added to posts in bulk. It's especially valid usecase since we can define automatic collection with a filter containing not yet existing tags.
refs https://github.com/TryGhost/Arch/issues/47
This ensures that we only have collections which have a valid filter in terms of
- Valid NQL string
- Uses only properties which are valid to filter on
- Only has an empty filter in the case of the "latest" collection
fixes https://github.com/TryGhost/Product/issues/3752
- Added some extra tests for edge cases
- Updated handling of multiple subscriptions so they are handled better
- Canceling a subscription when the member still has other subscriptions will now get handled correctly where the status and products of the member stay intact
refs https://github.com/TryGhost/Arch/issues/71
- With describe/it block levels mixed on the top level of the test suite the order fo test execution was scattered around. Having "describe" groups for each of the BREAD methods makes things more organized and readable.
fixes https://github.com/TryGhost/Product/issues/3728
- When importing members from Stripe with an existing offer, that didn't
exist in Ghost, the offer never got linked with the imported
subscription because of a missing return statement.
- Fixes importing offers with duplicate names
- Added E2E tests for creating members from a Stripe Customer ID
closes https://github.com/TryGhost/Arch/issues/76
- The posts test suite was failing when run in isolation. This was due to "collections" labs flag not being turned on, the events were not going through to collections service correctly
refs TryGhost/Product#3638
- Added `convert_to_lexical` flag to the posts/pages edit endpoint
- Added 'convertToLexical' feature flag so we can enable/disable this
feature independently from the main lexical beta flag
- Modified admin posts/pages list to point to the lexical editor for
_all_ posts, regardless of mobiledoc vs lexical (if the flag is on)
- Added call to edit endpoint with `convert_to_lexical` in the lexical
editor admin route if the page/post is currently in mobiledoc and the
flag is enabled
fixes https://github.com/TryGhost/Product/issues/3687
After this change, relative URLs in emails will be replaced with
absolute URLs using the post URL. Making relative Portal URLs possible
etc.
Updates the test data generator to fix invalid URL encoding (somehow a
backslash + escaped double quote was added when it wasn't required).
no issue
- Snapshot tests were incorrect but we weren't catching it because CI
was retrying them and yielding false passes
- This fix just fixes the broken tests, which will allow us to fix the
issue with CI yielding false passing results
closes https://github.com/TryGhost/Product/issues/3666
- added computed setting "donations_enabled"
- added logic to persist "donations_suggested_amount" and "donations_currency"
- used "donations_suggested_amount" and "donations_currency" when initiating a new Stripe Checkout for donations
- added copy functionality to "your link" in Tips & Donations settings
refs https://github.com/TryGhost/Arch/issues/16
- When posts produce PostsBulkFeaturedEvent/PostsBulkUnfeaturedEvent the collections having a featured filter should update the posts belonging to them.
refs https://github.com/TryGhost/Product/issues/3651
- This is a security fix that addresses an issue causing malicious users
to abuse the test / preview email API endpoint.
- We have multiple procedures in place now to limit such users.
- First, we now only allow one email address to be passed into the
`sendTestEmail` method. This method only have one purpose, which is to
compliment the test email functionality within the Editor in Admin and
therefore have no reason to send to more than one email address at a
time.
- We then add an additional rate limiter to prevent a user from making
multiple requests, eg via a script.
- The new imposed limit is 10 test emails per hour.
closes https://github.com/TryGhost/Arch/issues/58
- Following assumptions were broken:
- Posts Admin API should include posts of all statuses when filtering by collection
- Posts Content API should not include any unpublished posts
- Updated the "status" filter which fixes the problem. We still disallow any custom filters to be applied on top of collections filter.
Unfortuantely our framework is bookshelf centric so we have to refer to the
`withRelated` property rather than a more generic `include` property.
The collection entity already contains the list of post ids, so we can just
return the length of that array.
The test was addign an extra collection, but not cleaning it up - which makes it
hard to reason about other tests, especially when running them in isolation and
the state is different. This just cleans up the test and updates the browse test
to match the right snapshot.
The only usecases we need to support at the moment are reading individual
collections by ID and by Slug. We can extend this API as we get more usescases
in future.
The correct mechanism for fetching posts from a collection is via the Posts API.
This removes all functionality of getting posts from the Collections API.
Co-authored-by: Naz <hi@nazavo.com>
refs https://github.com/TryGhost/Arch/issues/46
- Similarly to post filters, collection filters now support both 'tag' and 'tags' nql filter keys when defining a filter for related tag slugs. For example, both `tag:avocado` and `tags:avocado` would both be valid collection filters that would filter by the same 'slug' property of the tags assigned to a post.
- Along with these changes had to rework the tags property of the collection posts to match the shape used in post resources. Moved from:
`tags: ['bacon', 'broc']`
to
`tags:[{slug: 'bacon'}, {slug: 'broc'}]`
