closes https://github.com/TryGhost/Product/issues/3818
- instead of fetching all recommendations and matching URLs on the frontend, we now query the database directly to find an existing Recommendation by URL. When comparing URLs, we don't take into account the protocol, www, query parameters nor hash fragments
fixes https://github.com/TryGhost/Product/issues/3911
For now we decided that we don't want to enable one-click-subscribe in
case a site has a required checkbox (which isn't shown during the
one-click-subscribe flow)
refs https://github.com/TryGhost/Product/issues/3850
- Added a recheck for recommendation related webmentions after boot (to
check missed webmentions during down time)
- Increased general timeouts to 15s for all webmention related HTTP
requests. Instead, increased retries to 3.
- Increased timeout for fetching webmention metadata from 2s to 15s
- Added more logging about verification and deletion status of
webmentions
refs https://ghost.slack.com/archives/C0568LN2CGJ/p1695149803260239
refs 290bc71d6a
- previous versions of `@tryghost/limit-service` did a full import of
lodash, which would bloat the bundle size, especially when we only use
a few of its functions
- I've since fixed that and this commit bumps Ghost to the smaller
version
This reverts commit 3e9da6df0c.
- changes introduced an error fetching `/admin/pages/` when using MySQL
- "The values in where clause must not be object or array"
no issue
Collection cards contain dynamic data that can change when there's any change to a published post but in Ghost all post/page content is rendered once on save and stored as a static string meaning we need a new approach for triggering a re-render of pages that plays well with caching.
- fixed typo in the relations/authors code that meant we weren't correctly calling the prototype method on the Post model inside the `onFetchedCollection` event handler
- updated Post model to clear the `html` field of all pages when saving or deleting a published post
- updated Post model to re-render `html` fields when fetching individual posts or a collection of posts
- modified `insertExtraPostsTags` fixture util to wrap it's concurrent post edits in a transaction otherwise MySQL errors because it hits a deadlock
fixes https://github.com/TryGhost/Product/issues/3830
This endpoint is required for recommendations to work: admin-x loads the incoming recommendations by querying the mentions endpoint. If the mentions flag was not enabled, this endpoint wasn't available.
closes https://github.com/TryGhost/Arch/issues/74
refs b5d1245be1
- We have turned off the collections feature flag after a unsuccessful attempt to make collections GA. With the flag turned off, collections_posts data has gone stale and needs repopulation to function properly again.
- This migration is meant to clear the data on collections_posts table and repopulated it again the same way initial migration did in 5.5/2023-07-10-05-16-55-add-built-in-collection-posts.js script.
refs https://github.com/TryGhost/Arch/issues/86
- Creating bookshelf models for each collection_post relation created a
massive overhead. On a dataset with 500k collections_posts records the
timing was roughly 7s comparing to 810ms after the optimization.
- Optimized memory and performance of collections fetching by querying post
ids only by default
refs https://github.com/TryGhost/DevOps/issues/80
- as part of moving Admin-X-Setting towards GA, we want to change it from
loading the settings externally via a CDN, to bundling it in with
Admin
- the bulk of the changes here are removing the config in Ghost, setting
up the copy to the Admin assets dir, and loading the new path in Admin
- several other changes have come along the way as I've cleaned up
unneeded code
refs https://github.com/TryGhost/Arch/issues/86
- Creating bookshelf models for each collection_post relation created a
massive overhead. On a dataset with 500k collections_posts records the
timing was roughly 7s comparing to 810ms after the optimization.
- Optimized memory and performance of collections fetching by querying post
ids only by default
refs https://github.com/TryGhost/Arch/issues/83
As we've only used the status and tiers in the cache key generation (as those
are currently the only ones that are present in DB's) we want to make sure that
content gating doesn't use any other properties, which means the cache behaviour
will match the content gating behaviour
refs https://github.com/TryGhost/Arch/issues/83
The options included are the only ones which have an effect on the response
data, as well as that we are using the properties of members which are used by
content-gating module. For the read operation we need to include the ID too.
refs https://github.com/TryGhost/Arch/issues/83
This allows endpoints to implement their own key generation, with access to the
frame object they can be smart about key generation and use only options and
context values that are appropriate.
closes https://github.com/TryGhost/Product/issues/3881
We observe the height of the recommendation table, and keep it fixed between pages — so that the UX is smoother when navigating between pages.
fixes https://github.com/TryGhost/Product/issues/3900
1. The service never returns a Recommendation Entity, but always plain
objects (which for now is the same as Recommendation without the
methods).
2. Updated the controller to be more readable and minimal (we keep this
controller, in addition to the existing endpoints and serializers)
- The controller does minimal validation and allows for type checking
(so we get compile time errors in case the service expects new fields)
- The controller uses the `UnsafeData` class to easily validate the
input from requests, and throws appropriate errors (with correct field
descriptions — "Expected a string at recommendations.0.title") without
too much boilerplate code. In addition the interface is typed, so we get
compile errors if there are breaking changes in the service.
- Removed `EntityWithIncludes`, since we now use plain objects, we
inject the relations directly into those plain objects (with some new
types that add type support)
- Added new tests to make sure that edits only affect the given fields,
and never undefined fields
no issue
- the Content API is served from the Admin URL not the Frontend URL but we were fetching from the Frontend URL. That resulted in a 302 response with no CORS headers so the request was blocked by the browser
- the schema.org data fo an author should primarily use the authors image as the image item instead of the cover image.
- otherwise the schema.org metadata will be invalid, since the image item is missing when no cover image has been uploaded.
closes https://github.com/TryGhost/Product/issues/3818
- in Admin, when adding a recommendation, the URL is compared against all existing ones. If the URL is already recommended, the publisher is shown an error: "A recommendation with this URL already exists.". Protocol, www, query parameters and hash fragments are ignored during the URL comparison.
- on the backend, there is another uniqueness validation for the recommendation URL. This check is redundant when adding a recommendation from Admin, but helps to keep data integrity when recommendations are added through other paths (e.g. via the API)
no issue
- added missing `context.public = true` option that tells our data fetching layer that this is a "Frontend/Content API" request and relevant filters should be applied
- adjusted require of posts service so it's only grabbed on the first render rather than every render
refs
https://www.notion.so/ghost/df5bdea8f7ea4aca9d25eceb6a1bf34c?v=be2f15b6b58b4c27a0e11374282bead0&p=163762d9513a4e6dbd60c28e19228fdc&pm=s
- Added a modal to confirm that the new support email has been verified.
- to achieve that a couple of adjustments had to be made
- Updated the RoutingProvider to handle routes with query params.
- Added a new useQueryParams hook to grab query params where needed.
- wired up the email verification api.
- added feature flags / labs logic to the core package with the new URL and updated test.
---
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 3ff8add</samp>
This pull request adds email verification functionality for the support
email address in the portal settings. It fixes a bug in the routing
provider, adds a new API function, a new custom hook, and a new modal
component to handle the verification process. It also updates the
settings query with the verified email address.
refs 488af56ef9
- The referenced commit introduced a postsService initializaiton at the top level of the module - causing cascading failure all the way down in the URL service, whe the ENV variables are not set.
- This fix is just a quick fix to unblock the main branch. A proper initialization of the service should be done ensuring we don't have to re-create a posts-service instance on each render method call.
refs TryGhost/Product#3883
- passes endpoint through to the lexical renderer for collections
rendering
- ghost still needs a `kg-default-nodes` and `kg-lexical-html-renderer`
update to support this completely
no issue
`PostsService` and `CollectionsService` were missing some passthroughs and had differing naming for a transaction instance on the `options` object which meant SQLite would hang if the Lexical renderer called out to `PostsService.browsePosts`
- added passthrough of `transacting` to the Lexical renderer ready for implementation of the collection-fetching function
- added rename of `options.transacting` to `options.transaction` and passthrough from `PostsService` to `CollectionsService` (passthrough from collections repository to bookshelf and required `transaction->transacting` was already in place)
no issue
When filtering and using the BookshelfRepository, you had to use the column names instead of the entitiy field names. This is fixed by adding a transformer. Long term we want to move the NQL parsing away from the repository and into the API layer, and pass along the Mongo filter probably.
refs. https://github.com/TryGhost/Product/issues/3349
- fix tiers save & close bug
- show trial days on tier card
- removed 0's from monthly and yearly tier prices
- added color to learn more link in tier modal
- set background to white in newsletter preview
- fixed newsletter default sender data
- removed underline in "View in browser" link in newsletter preview
- updated copy in newsletters
- added Integrations' active indicator
- scrolling menu under searchbar to give search more prominance
- updated Portal modal buttons to be consistent with design settings
- fixed bug in AdminX loading Orb so that it actually starts auto-playing
refs https://github.com/TryGhost/Product/issues/3832
---
This pull request introduces a new `MainContent` component that handles
the role-based access and rendering of the settings page and the
sidebar. It also refactors and improves the UI and logic of the
`UserDetailModal` and the `Users` components, and updates the footer
component to use the new settings page and profile modal for editors.
Additionally, it removes unused code and adds new helper functions for
checking the user's roles and permissions.
refs https://github.com/TryGhost/Product/issues/3875
When a member had a comped subscription, the portal was showing an
incorrect expiry date. This was because the `expiry_date` was being set
to the `created_at` date of the subscription, rather than the
`expiry_date` of the comped subscription
https://github.com/TryGhost/Arch/issues/90
- When a tag.deleted event is emitted the original 'data' object does not contain an 'id' property. The logic in collections service assumes the id would be present to update the collections efficiently.
https://github.com/TryGhost/Arch/issues/90
- When a post.deleted event is emitted the original 'data' object does not contain an 'id' property. The logic in collections service assumes the id would be present to update the collections efficiently.
no issues
- removed (default) label from the custom theme settings
- the label was redundant especially when multiple select settings are next to each other
- instead, the default value should always be the first option in the select fields
- more consistent spacing between settings with/without description
- less spacing between boolean settings
closes https://github.com/TryGhost/Product/issues/3874
- fetch Content API key if we don't already have it
- use the `frontend` service to fetch posts via the Content API
- uses same ordering and published-only filtering as default front-end requests
refs https://github.com/TryGhost/Product/issues/3874
- the new collections card needs to access the Content API rather than the Admin API in order to show the card as it will appear on the front-end but we don't have a default integration that can be fetched via the Admin API for Admin to use when fetching from the Content API
- adds a new "Ghost Core Content API" integration with the `core` type so that it can be read via the `/admin/integrations/` endpoint and used in Admin to make Content API requests
refs https://github.com/TryGhost/Arch/issues/86
bookshelf-relations was generating tonnes of select queries from the
posts table in order to update the relations. We've instead implemented
this ourselves, so as to avoid the superfluous fetches. Working closer to
the db like this is nice, and makes you think more about performance.
This logic could be pulled out into a util (not bookshelf plugin) where
it could be used explicitly, but with the complexity hidden, we'll see ig.
refs https://github.com/TryGhost/Ghost/pull/18028
- The previous conditional meant that if the "host settings" collections enabled flag was set, we couldn't disable collections. The referenced pull request would also disable the collections across all of the hosted environment instances. The updated logic optionally takes into account the "labs" flag, as it should have from the very beginning.
refs https://github.com/TryGhost/Arch/issues/87
- The newsletters in members payload have leaked internal properties from Public Members API. The code skipped the output serialization step, which is now in place.
- The newsletter resource returned from the API consistently returns these properties:
id,
name,
description,
sort_order
refs https://github.com/TryGhost/Arch/issues/87
- There was no test whatsoever! Adding a super basic test to have some certainty the output doesn't change after a refactor
refs https://github.com/TryGhost/Arch/issues/87
- The Members Admin API and members.* webhooks were returning too many fields in the nested `newsletters` objects. There was no "allowlist" serializer for the newsletter object, which meant every time we add a new field to the database we would unintentionally return extra fields without a second thought.
- With this change only following fields will be returned with `members[x].newsletters[x]`:
'id',
'name',
'description',
'status'
refs https://github.com/TryGhost/Arch/issues/87
- Round 2 for the previous commit. Removes use of `anyArray` for all
- Using `anyArray` in snapshot test is an anti-pattern which leads to leaking output fields unintentionally when the API changes.
- Adding these fixes is fundamental work before changing the output of 'member.newsletters' property
refs https://github.com/TryGhost/Arch/issues/87
- Using `anyArray` in snapshot test is an anti-pattern which leads to leaking output fields unintentionally when the API changes.
- Adding these fixes is fundamental work before changing the output of 'member.newsletters' property
- we have this set as a higher resolution anyway, but I think having
some hardcoded lower versions causes some weird yarn issues
- at the very least, this removes any mention of 1.2.21 from our
yarn.lock
no issue
- Do not set ?ref in recommendations if analytics is disabled
- Do not send url_history if analytics is disabled
- Expose outboundLinkTagging as a public setting
fixes https://github.com/TryGhost/Product/issues/3846
- when the lexical editor is turned off, some action bar icons appear as black in dark mode
- this fixes the icon and divider color for the mobiledoc editor
fixes https://github.com/TryGhost/Product/issues/3851
- Order was not applied via the CRUD plugin
- Removed usage of CRUD findAll, and swapped to Bookshelf fetchAll
instead, to decrease dependencies of invisible Bookshelf plugins logic
- Reverted page and limit options possibility via findAll method
refs https://github.com/TryGhost/Product/issues/3843
If you used relative URLs in the beta editor, when it came to leaving the editor you would get stuck due to an infinite save loop occurring in the background requiring a refresh to get back to a working state.
- when saving relative URLs the server will convert them to absolute for consistency and to ensure URLs work in other situations such as emails, RSS, 3rd party editors, etc
- although we get different data back from the server we don't overwrite the content in the editor with it as that would cause loss of changes since the save as well as loss of the cursor position
- when leaving the editor we compare content from the last save revision with the current editor content to see if we need to save a new revision but if the server data has been modified with relative->absolute URLs then we'd enter an infinite loop because the content would never match
- relative->absolute URLs should be the only thing to ever be modified in the underlying data when saving so we can work around the issue by replacing each instance of the site's URL before comparing revision data to current data
refs https://github.com/TryGhost/DevOps/issues/50
- when creating a TS config in our `eslint-plugin-ghost` dependency, I
only extended the recommended config, which left out a lot of
stylistic things we used to enforce in JS
- this fixes that by bumping the dependency to a version which extends
those shared configs, and fixes all the code that currently goes
against those rules
fixes https://github.com/TryGhost/Product/issues/3822
fixes https://github.com/TryGhost/Product/issues/3838
This PR became a bit big because it affected multiple parts of Ghost
that needed to be updated to prevent breaking anything.
### Backend
- Added pagination to the recommendations API's
- Updated BookshelfRepository template implementation to handle
pagination
- Allow to pass `page` and `limit` options to Models `findAll`, to allow
fetching a page without also fetching the count/metadata (=> in the
repository pattern we prefer to fetch the count explicitly if we need
pagination metadata)
- Added E2E tests for public recommendations API (content API)
- Extended E2E tests of admin recommendations API
### Portal
- Corrected recommendations always loaded in Portal. Instead they are
now only fetched when the recommendations page is opened.
### Admin-X
- Added `usePagination` hook: internally used in the new
`usePaginatedQuery` hook. This automatically adds working pagination to
a query that can be used to display in a table by passing the
`pagination` and `isLoading` results to the `<Table>`
- Added placeholder `<LoadingIndicator>` component
- Added a loading indicator to `<Table>`. This remembers the previous
height of the table, to avoid layout jumps when going to the next page.
refs https://github.com/TryGhost/Arch/issues/86
- When PostEditedEvent data contains no visible changes we can skip the matching collections update process alltogether. Each call to `updatePostInMatchingCollections` creates a transaction in addition to fetching all collections. There's no need to process anything when there are no relevant changes in the post edit!
closes https://github.com/TryGhost/Arch/issues/85
- Added a cache configuration option to signal "reuse of redis connection" for Redis cache adapter. The connection reuse it turned on by default to be shared between caches. They rely on unique "keyPrefix" structure, so there is no collision side-effects when reusing same Redis Store.
- The Redis connection options like "ttl" are shared with the first connection that's crated. So if there's a need to have unique configuration, a separate connection has to be created by passing `"reuseConnection": false` parameter
closes https://github.com/TryGhost/Product/issues/3803
Previously when the beta editor was enabled, using `?source=html` to create posts via the API would create posts in the old editor rather than the beta. This change switches conversion over to the new editor format when the beta is enabled so the full flow can be tested.
- added `htmlToLexicalConverter` method to our lexical library
- updated post and page input serializers to add html-to-lexical conversion when the beta editor is enabled
- updated post model to handle the mobiledoc+lexical co-existing state
- this is a special case that is only valid for `?source=html` because providing both directly via the API is prohibited
- we need the extra check here because at the input serializer layer we don't have access to the model to check if we're updating a mobiledoc post or a lexical post so the serializer sets both formats on a `?source=html` request when the beta is enabled and lets the model handle choosing the correct one
fixes https://github.com/TryGhost/Product/issues/3820
- This adds a new public site endpoint in the members API to check if a
site can offer the one-click-subscribe feature
- This is implemented on the members API as a copy of the `site`
endpoint because the admin API site endpoint is protected by CORS and
mainly because it can be served on a different domain than the
recommended site and this is hard to detect reliably from the frontend
- Added a new calculated setting `allow_self_signup`, which can replace
the setting that is currently used in Portal (best to do this after a
release otherwise we risk creating issues if a patch release happens)
no issue
- recently added code to grab apple touch icons or SVGs before falling back to the default metascraper behaviour wrongly assumed that every size would have a `rel` and `href` attribute which is not the case
no issue
- recently added code to grab apple touch icons or SVGs before falling back to the default metascraper behaviour wrongly assumed that every size would have a `rel` and `href` attribute which is not the case
closes https://github.com/TryGhost/Product/issues/3799
- the recommendations_enabled setting is updated when a recommendation
is created or deleted. It's enabled as soon as there is at least one
recommendation in the database
- the recommendations_enabled setting exists to avoid fetching the
recommendation count from the database directly in themes. The setting
is cached and doesn't need a read every time from the database
closes https://github.com/TryGhost/Product/issues/3827
- links following a line break in the editor were being rendered before the line break when previewing/publishing
- bumps Koenig packages which includes relevant fix in `@tryghost/kg-lexical-html-renderer`
