refs https://github.com/TryGhost/Product/issues/3874
- the new collections card needs to access the Content API rather than the Admin API in order to show the card as it will appear on the front-end but we don't have a default integration that can be fetched via the Admin API for Admin to use when fetching from the Content API
- adds a new "Ghost Core Content API" integration with the `core` type so that it can be read via the `/admin/integrations/` endpoint and used in Admin to make Content API requests
refs https://github.com/TryGhost/Arch/issues/86
bookshelf-relations was generating tonnes of select queries from the
posts table in order to update the relations. We've instead implemented
this ourselves, so as to avoid the superfluous fetches. Working closer to
the db like this is nice, and makes you think more about performance.
This logic could be pulled out into a util (not bookshelf plugin) where
it could be used explicitly, but with the complexity hidden, we'll see ig.
refs https://github.com/TryGhost/Ghost/pull/18028
- The previous conditional meant that if the "host settings" collections enabled flag was set, we couldn't disable collections. The referenced pull request would also disable the collections across all of the hosted environment instances. The updated logic optionally takes into account the "labs" flag, as it should have from the very beginning.
refs https://github.com/TryGhost/Arch/issues/87
- The newsletters in members payload have leaked internal properties from Public Members API. The code skipped the output serialization step, which is now in place.
- The newsletter resource returned from the API consistently returns these properties:
id,
name,
description,
sort_order
refs https://github.com/TryGhost/Arch/issues/87
- There was no test whatsoever! Adding a super basic test to have some certainty the output doesn't change after a refactor
refs https://github.com/TryGhost/Arch/issues/87
- The Members Admin API and members.* webhooks were returning too many fields in the nested `newsletters` objects. There was no "allowlist" serializer for the newsletter object, which meant every time we add a new field to the database we would unintentionally return extra fields without a second thought.
- With this change only following fields will be returned with `members[x].newsletters[x]`:
'id',
'name',
'description',
'status'
refs https://github.com/TryGhost/Arch/issues/87
- Round 2 for the previous commit. Removes use of `anyArray` for all
- Using `anyArray` in snapshot test is an anti-pattern which leads to leaking output fields unintentionally when the API changes.
- Adding these fixes is fundamental work before changing the output of 'member.newsletters' property
refs https://github.com/TryGhost/Arch/issues/87
- Using `anyArray` in snapshot test is an anti-pattern which leads to leaking output fields unintentionally when the API changes.
- Adding these fixes is fundamental work before changing the output of 'member.newsletters' property
no issue
- Do not set ?ref in recommendations if analytics is disabled
- Do not send url_history if analytics is disabled
- Expose outboundLinkTagging as a public setting
fixes https://github.com/TryGhost/Product/issues/3851
- Order was not applied via the CRUD plugin
- Removed usage of CRUD findAll, and swapped to Bookshelf fetchAll
instead, to decrease dependencies of invisible Bookshelf plugins logic
- Reverted page and limit options possibility via findAll method
fixes https://github.com/TryGhost/Product/issues/3822
fixes https://github.com/TryGhost/Product/issues/3838
This PR became a bit big because it affected multiple parts of Ghost
that needed to be updated to prevent breaking anything.
### Backend
- Added pagination to the recommendations API's
- Updated BookshelfRepository template implementation to handle
pagination
- Allow to pass `page` and `limit` options to Models `findAll`, to allow
fetching a page without also fetching the count/metadata (=> in the
repository pattern we prefer to fetch the count explicitly if we need
pagination metadata)
- Added E2E tests for public recommendations API (content API)
- Extended E2E tests of admin recommendations API
### Portal
- Corrected recommendations always loaded in Portal. Instead they are
now only fetched when the recommendations page is opened.
### Admin-X
- Added `usePagination` hook: internally used in the new
`usePaginatedQuery` hook. This automatically adds working pagination to
a query that can be used to display in a table by passing the
`pagination` and `isLoading` results to the `<Table>`
- Added placeholder `<LoadingIndicator>` component
- Added a loading indicator to `<Table>`. This remembers the previous
height of the table, to avoid layout jumps when going to the next page.
closes https://github.com/TryGhost/Product/issues/3803
Previously when the beta editor was enabled, using `?source=html` to create posts via the API would create posts in the old editor rather than the beta. This change switches conversion over to the new editor format when the beta is enabled so the full flow can be tested.
- added `htmlToLexicalConverter` method to our lexical library
- updated post and page input serializers to add html-to-lexical conversion when the beta editor is enabled
- updated post model to handle the mobiledoc+lexical co-existing state
- this is a special case that is only valid for `?source=html` because providing both directly via the API is prohibited
- we need the extra check here because at the input serializer layer we don't have access to the model to check if we're updating a mobiledoc post or a lexical post so the serializer sets both formats on a `?source=html` request when the beta is enabled and lets the model handle choosing the correct one
fixes https://github.com/TryGhost/Product/issues/3820
- This adds a new public site endpoint in the members API to check if a
site can offer the one-click-subscribe feature
- This is implemented on the members API as a copy of the `site`
endpoint because the admin API site endpoint is protected by CORS and
mainly because it can be served on a different domain than the
recommended site and this is hard to detect reliably from the frontend
- Added a new calculated setting `allow_self_signup`, which can replace
the setting that is currently used in Portal (best to do this after a
release otherwise we risk creating issues if a patch release happens)
closes https://github.com/TryGhost/Product/issues/3799
- the recommendations_enabled setting is updated when a recommendation
is created or deleted. It's enabled as soon as there is at least one
recommendation in the database
- the recommendations_enabled setting exists to avoid fetching the
recommendation count from the database directly in themes. The setting
is cached and doesn't need a read every time from the database
closes https://github.com/TryGhost/Product/issues/3827
- links following a line break in the editor were being rendered before the line break when previewing/publishing
- bumps Koenig packages which includes relevant fix in `@tryghost/kg-lexical-html-renderer`
refs https://github.com/TryGhost/Ghost/issues/17932
- we were missing `chunk.208.dbf172ad32f72f21a5dc.js` from our published tarball
- turns out this is due to the lines in the `.npmignore` file to remove
.db files, which also matched this file
- we can make the regex more specific to avoid these cases
closes https://github.com/TryGhost/Product/issues/3675
refs c98bf80248
As part of our architecture guidelines Repository implementations should protect
against invalid or malformed data in persistence. We do not want read operations
of Entities to throw because of such data. For some fields that bad data can be
fixed or handled in the constructor or static create factory method and replaced
with valid data, others will cause the factory to throw.
This means that Repositories should catch these errors and exclude those
entities from their results. We log the errors in Sentry so that we have
visibility on the state of bad data in DBs
JSDoc has a problem with using values as types across repositories, rather than
getting `Offer` as the type we end up with `typeof Offer` as the type - which is
incorrect. Instead we use `import` syntax inside of JSDoc which resolves correctly
As per our architecture guidelines we want to keep bookshelf implementations of
Repositories in Ghost core, so that all the bookshelf code is kept together, and
the packages implementing business logic with entities and services require less
dependencies to test. This separation should also help us inadvertently add
business logic to repository implementations by having a more "physical"
boundary between them.
refs https://forum.ghost.org/t/anyone-else-seeing-page-too-wide-errors/40695
- google search console snapshots websites with their full height instead of scrolling
- that makes the value of vmax too much, especially when a page has lots of content
- this fixes the issue by adding max limit to the padding values with vmax
no issue
Previously the beta editor only worked for newly created posts/pages, any older content would open with the original editor. This change enables automatic conversion of old content to the new content format when a post/page is opened in the admin interface allowing new features like signup and advanced header cards to be used on existing content.
- removed `convertToLexical` feature flag
- where necessary switched to using just the `lexicalEditor` feature flag in its place
- moved the "L"/"M" indicators on the posts list to a new `lexicalIndicators` feature flag to make debugging/development easier
- added a redirect to the original editor route so that any route to opening the editor (such as the `/edit` front-end shortcut, or other areas of Admin) will open in the beta editor
- avoids confusing/inconsistent behavior
refs https://github.com/TryGhost/Arch/issues/80
refs 3960bfac1d
- The killswitch (a setting in host settings) is needed to control the feature on a hosted environment, so we can safely turn it off if it causes any major issues.
refs https://github.com/TryGhost/DevOps/issues/68
- without a name, tools such as New Relic report the function as
`<anonymous>`, which makes it incredible hard to follow the code flow
- this commit adds a function name to all middleware I can find that
doesn't already have one, which should fill in a lot of those gaps
refs https://github.com/TryGhost/DevOps/issues/68
- we want New Relic to be one of the first modules to load so it can
instrument the rest of our code
- previously this would not have been the case, and it would have missed
out on instrumenting the config and logging code
- this moves the require to be the first step in the process if the
PRO_ENV env var is set
closes https://github.com/TryGhost/Arch/issues/13
- Model to Domain event interceptor is a class that does not strictly belong to Collections. It's supposed to be used in any new code that depends on legacy bookshelf model events. Extracted it's initialization to it's own service for clarity and visibility.
The future work around collections requires the read only portions
of the Collections Admin API to be enabled without a flag.
The write endpoints are only for custom collections which aren't required.
refs https://github.com/TryGhost/Arch/issues/73
Since we've fixed the issues which were executing performance affecting DB
queries, we can enable the background syncing of collections again. This couple
with a new migration to populate the default collections will allow us to start
work on the collections card and will make rolling that card out to GA smoother
and easier as all the required data will be in place.
refs https://github.com/TryGhost/Arch/issues/73
This is just an initial stab at making sure we don't introduce extra DB
queries related to collections without being aware of it.
refs https://ghost.slack.com/archives/C02G9E68C/p1692816097875899
- With introduction of extra e2e test coverage for Collections some tests started to fail at random. The root issue here was the transaction processing collections was started before the original bulk action (bulk edit, bulk publish/unpublish, etc.) was fully committed. The full transaction commit happens with the bulkAction method return inside of `if (!options.transacting) {` block.
refs https://github.com/TryGhost/Product/issues/3504
- This adds support for translations, but doesn't yet translate every
possible string in the app.
- Only active if beta translations is enabled
refs https://github.com/TryGhost/Arch/issues/77
- We were missing e2e test coverage for when the tag used in collection filters was removed. This changeset improves the situation.
refs https://github.com/TryGhost/Arch/issues/77
- During initial development we have missed to support collections update when tags are added to posts in bulk. It's especially valid usecase since we can define automatic collection with a filter containing not yet existing tags.
refs https://github.com/TryGhost/Arch/issues/47
This ensures that we only have collections which have a valid filter in terms of
- Valid NQL string
- Uses only properties which are valid to filter on
- Only has an empty filter in the case of the "latest" collection
refs https://github.com/TryGhost/Arch/issues/47
This change allows us to throw errors when instantiating invalid entities,
whilst not breaking things when we have bad data in the database. What we can
do is act as if the bad rows are not present, whilst surfacing an error in
sentry to alert us to such cases.
refs https://github.com/TryGhost/Arch/issues/47
We've configured bookshelf to force empty strings to null, but this is
undesired behaviour here, so unfortunately we have to leak some business
logic into the repository.
This needs to be done to correctly support our filter validation logic.
fixes https://github.com/TryGhost/Product/issues/3752
- Added some extra tests for edge cases
- Updated handling of multiple subscriptions so they are handled better
- Canceling a subscription when the member still has other subscriptions will now get handled correctly where the status and products of the member stay intact
refs https://github.com/TryGhost/Arch/issues/71
- With describe/it block levels mixed on the top level of the test suite the order fo test execution was scattered around. Having "describe" groups for each of the BREAD methods makes things more organized and readable.
refs https://github.com/TryGhost/Arch/issues/73
With the latest version of bookshelf-relations we're able to define a model
specific hook which allows us to ignore sort_order updates on automatic
collections, which don't require their order to be persisted.
fixes https://github.com/TryGhost/Product/issues/3728
- When importing members from Stripe with an existing offer, that didn't
exist in Ghost, the offer never got linked with the imported
subscription because of a missing return statement.
- Fixes importing offers with duplicate names
- Added E2E tests for creating members from a Stripe Customer ID
closes https://github.com/TryGhost/Arch/issues/76
- The posts test suite was failing when run in isolation. This was due to "collections" labs flag not being turned on, the events were not going through to collections service correctly
fixes https://github.com/TryGhost/Product/issues/3723
This also fixes usage of localhost instead of 127.0.0.1 as a test URL
for playwright. This caused issues for cookies because the member
impersonation navigated to 127.0.0.1 instead of localhost, meaning that
the next page.goto call would go to localhost and lose the cookies.
closes https://github.com/TryGhost/Ghost/issues/17681
- updated `prepareContextResource()` to make sure `show_title_and_feature_image` is always removed from pages
- updated `formatResponse.entries()` to apply the same `@page` local behaviour when it's passed a `data.page` object to account for custom routed pages
no issue
- We were loading the Explore iframe together with all assets in Admin
- This change will avoid that and only start loading and rendering Explore, once it's clicked
refs https://github.com/TryGhost/Ghost/pull/17609
- The tests for content gating started to fail with introduction of the index on `published_at` data in for `posts` table. The reason for the failure was identical `published_at` date set during the fixture insertion, making the returned results change order non-deterministically. The problem is mostly in how the test is set up as it's quite unrealistic to have multiple posts in the system inserted at the same time down to millisecond. Maybe... by some coincidence, but thats not a problem we should care too much about imo.
refs https://github.com/TryGhost/Arch/issues/18
- The prev/next helpers are slow and are causing major performance issues. The helpers are using `posts.published_at` for comparisons extensively, which causes a full table scan - bad for query performance.
- We use published_at in other queries too (like default order for queries fetching all posts), so there might be a slight performance boost across the system with this new index.
closes https://github.com/TryGhost/Product/issues/3661
- until now, Portal was not loaded if members were disabled. With the
introduction of Tips & Donations, signed-off readers can also make
payments, using the Portal link /#/portal/support.
- now, Portal is loaded when Tips & Donations are enabled, even if
Memberships are disabled
- depending on the member signup access, the top bar / trigger button
Portal buttons are hidden as before (signup/subscribe hidden if access is set to none, subscribe hidden if
access is set to invite-only)
- for any other signup / signin Portal links (e.g., added by the theme,
or added via a Post/Page), a new popup informs the reader when
Memberships are disabled: "Memberships unavailable, contact the site
owner for access".
no issue
- Keep an eye on Ghost's changelog (https://ghost.org/changelog/) for
the full feature announcement. 🎉
---
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 8367091</samp>
This pull request removes the `headerUpgrade` feature flag and enables
the new header card by default in the editor. It also hides the old
header card from the editor menu unless the `labs.headerV1` setting is
enabled.
closes https://github.com/TryGhost/Product/issues/3661
- until now, Portal was not loaded if members were disabled. With the
introduction of Tips & Donations, signed-off readers can also make
payments, using the Portal link /#/portal/support.
- now, Portal is loaded when Tips & Donations are enabled, even if
Memberships are disabled
- depending on the member signup access, the "sign in" / "subscribe"
Portal buttons are hidden (both hidden if none, signup hidden if
invite-only)
- for any other signup / signin Portal links (e.g., added by the theme,
or added via a Post/Page), a new popup informs the reader as such when
Memberships are disabled: "Memberships unavailable, contact the site owner for access".
refs: https://github.com/TryGhost/DevOps/issues/56
The new monobundle package now ignores any package directory if it doesn't contain a package.json. These non-packages are occasionally restored from cache when pulling dependencies.
refs TryGhost/Product#3638
- Added `convert_to_lexical` flag to the posts/pages edit endpoint
- Added 'convertToLexical' feature flag so we can enable/disable this
feature independently from the main lexical beta flag
- Modified admin posts/pages list to point to the lexical editor for
_all_ posts, regardless of mobiledoc vs lexical (if the flag is on)
- Added call to edit endpoint with `convert_to_lexical` in the lexical
editor admin route if the page/post is currently in mobiledoc and the
flag is enabled
fixes https://github.com/TryGhost/Product/issues/3687
After this change, relative URLs in emails will be replaced with
absolute URLs using the post URL. Making relative Portal URLs possible
etc.
Updates the test data generator to fix invalid URL encoding (somehow a
backslash + escaped double quote was added when it wasn't required).
no issue
---
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 2f86a35</samp>
Updated packages for rendering post and page content. This fixes some
bugs and improves the handling of HTML elements.
no issue
- some tweaks inside `kg-default-nodes`, requires an update in Ghost.
---
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
copilot:summary
no issue
- made adjustment to `kg-default-nodes` that needs bumping on Ghost to
have those changes reflected.
---
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 77ea2d6</samp>
Updated Koenig Editor dependencies to fix bugs and enhance performance.
This affects the `@tryghost/kg-default-nodes` and
`@tryghost/kg-lexical-html-renderer` packages in
`ghost/core/package.json`.
By using the `collections_posts` table as a pivot table, I couldn't see
a way to get bookshelf to *not* load the Post models for the relation.
We don't actually need those Post models for our usescases, the the
queries were causing issues with our database servers! Here we've added
a new CollectionPost model which allows us to treat the
collections_posts table as a resource ratherthan pivot, and means we can
use the hasMany relation rather than belongsToMany relation. This
removes all queries to the posts table when fetching collections with
relations
no issue
---
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at ee5f8bf</samp>
Updated Koenig Editor dependencies in `package.json` to improve editor
performance and stability.
no issue
- Snapshot tests were incorrect but we weren't catching it because CI
was retrying them and yielding false passes
- This fix just fixes the broken tests, which will allow us to fix the
issue with CI yielding false passing results
closes https://github.com/TryGhost/Product/issues/3666
- added computed setting "donations_enabled"
- added logic to persist "donations_suggested_amount" and "donations_currency"
- used "donations_suggested_amount" and "donations_currency" when initiating a new Stripe Checkout for donations
- added copy functionality to "your link" in Tips & Donations settings
refs: https://github.com/TryGhost/DevOps/issues/11
This is a pretty huge commit, but the relevant points are:
* Each importer no longer needs to be passed a set of data, it just gets the data it needs
* Each importer specifies its dependencies, so that the order of import can be determined at runtime using a topological sort
* The main data generator function can just tell each importer to import the data it has
This makes working on the data generator much easier.
Some other benefits are:
* Batched importing, massively speeding up the whole process
* `--tables` to set the exact tables you want to import, and specify the quantity of each
