refs: https://github.com/TryGhost/DevOps/issues/78
Re-introduce parallel browser tests
These were adding in a previous PR, but the difference between local
running tests and using CI introduced failures.
Added additional fixes to ensure the Stripe API key is used in the CLI when running in CI.
fixesTryGhost/Product#3792
- Previously, if you had a filter set in the members list (e.g. `status=paid`), then you searched for a member by email address, the original filter would be ignored, and any members matching the search would be returned, regardless of whether they matched the filters. Effectively, the logic was `member matches filters OR member matches search`. To make this worse, the UI still showed both the filters and the search query, leading to confusing results.
- This small change to the backend logic changes the behavior to only return members that match the filter AND the search query, so if you search for a member that does not meet the current filters, they will not be returned.
refs https://github.com/TryGhost/Product/issues/3831
---
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 5af23a9</samp>
This pull request updates and adds some test cases for the editor and
the publishing flow in Ghost. It uses a helper function to test the date
picker component in the editor, and removes a redundant test case from
the `publish-flow-test.js` file. It also adds two test cases to the
`publishing.spec.js` file, using the Playwright framework, to check the
publish time and the newsletter settings of a post.
no issue
Whilst debugging I discovered that the webhooks weren't being received by some tests, causing them to fail. I added log lines for the output from the webhook server to see if I could fix it, and the bug vanished. I narrowed it down to event listeners on the webhook server fixing it.
I'm not sure exactly how this fixes it, I'm guessing the extra events in the event queue have something to do with it.
refs: https://github.com/TryGhost/DevOps/issues/78
Some tests needed to disconnect Stripe + reconnect, so the code had to be changed to match the new Stripe Connect method.
Some tests in publishing.spec have been modified to work out-of-order, which will enable us to use fullyParallel in future (a change which brings test runs down from 85~ seconds to 50~ seconds).
refs: https://github.com/TryGhost/DevOps/issues/78
Instead of running a single instance of Ghost, we now run an instance of Ghost for each test worker.
This has the unfortunate effect that a test failing will close and restart a new instance of Ghost, but in general will be multiple times faster than sequential execution of tests.
closes https://github.com/TryGhost/Product/issues/4008
- bumps all Koenig packages
- `kg-default-nodes` contains a rendering fix that avoids creating invalid class attributes from bad `backgroundColor` values
- `kg-converters` contains an update to avoid copying over bad `backgroundColor` values when converting from mobiledoc to lexical
refs https://github.com/TryGhost/Product/issues/3832
---
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 8a40cbc</samp>
This pull request adds a feature to verify the sender email address for
a newsletter in the admin settings app. It implements a UI component, a
custom hook, and a server-side service to handle the verification
process. It also fixes a minor typo and a navigation issue in the
newsletters module.
closes https://github.com/TryGhost/Product/issues/4007
- bumps Koenig packages containing extended TextNode and HeadingNode with extra DOM parsing support for the non-semantic HTML generated by Word when copy/pasting
fixes https://github.com/TryGhost/Product/issues/4005
We no longer use the 'reason' of a recommendation, but allow a flexible
description instead. Because this is a breaking change in the API, we do
this before making this feature GA.
- Added new database utils for renaming a column
- Added new migration to rename the column
- Updated all references in code
no issue
- Removed redundant Admin X flags from the email service since we are preparing for release.
- updated snapshots
---
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at fb0df07</samp>
Simplified staff URL generation for emails by removing unused feature
flag and code. This affects the `StaffServiceEmails.js` file and the
`adminXSettings` flag.
fixesTryGhost/Product#3970
- When saving a tier, it is impossible to change the capitalization of a
benefit
- The logic when saving a tier matches benefits by `name.toLowercase()`
and then overwrites the incoming change with the name from the previous
version of the benefit
- This changes the logic to match benefits the same way using
`name.toLowercase()`, but use the _incoming_ benefit's capitalization
rather than the old benefit's, to allow users to e.g. change 'TEst
benefit' to 'Test benefit'
refs https://github.com/TryGhost/Product/issues/3978
- added "GET /incoming_recommendations/" browse endpoint to the Admin
API - we store incoming recommendations as mentions in the database. The
new endpoint reuses the Mentions API underneath to fetch verified
mentions of type recommendation - recommendation-specific attributes are
returned by the new endpoint, including calculated fields such as the
"RecommendingBack" boolean
- show "Recommend back" option for sites recommending me, only if I
haven't recommended the site already
`generateOptionsData` was not taking into account the query params
`include`, `fields` and `format` could also be an array if included in
the request query like: `?fields[]=title&fields[]=slug` or
`?fields=title&fields=slug`
refs https://github.com/TryGhost/Ghost/pull/18422
- now that we're using the externally-loaded editor we need to wait for it to load and be ready before moving focus to it and starting to type
Promoted our beta editor to the default editor. Keep an eye on (or subscribe to) https://ghost.org/changelog/ for release announcements with full details.
- moved the beta editor (Lexical-based editor) to the default editor; all pages and posts will now use it
- all mobiledoc (previous editor) posts will remain mobiledoc until opened in the editor at which point will be converted to Lexical on the fly and open in the new editor
refs https://github.com/TryGhost/DevOps/issues/83
- this will now continue use the dev server assets if we tell it to,
or copy the dependency package files to the built folder otherwise
- removes `editor` from config API because it's no longer needed
- removes dependency on `editor.url` in tests, as this no longer exists
- edits dev script to pass dev server URL as env var
- adds `@tryghost/koenig-lexical` dependency to Admin
no issue
- After updating the default theme to be Source instead of Casper, the
browser tests broke because they depended on the class names in Casper,
which have changed
- This fixes that by updating the classes to be compatible with the
Source theme
refs TryGhost/Product#3510
- Added `TryGhost/Source` as a submodule in `ghost/core/content/themes` so `Source` will ship with Ghost (along with Casper)
- With this change, new installs will use `Source` as the default theme. Existing sites will have `Source` installed, but not activated, as this is a large change and we don't want to drastically change existing sites without warning. Users can upgrade to use `Source` simply by clicking 'Activate' in design settings.
- Updated protections to prevent users from uploading their own conflicting version of `Source`
refs https://github.com/TryGhost/Product/issues/3969
- this now allows themes to have up to 20 custom theme settings
- also bumps `@tryghost/zip` to try and ensure it uses the version
without fs-extra
refs https://github.com/TryGhost/Product/issues/3957
This changes how we fetch recommendations:
- Relations can be included in one query instead of extra queries
- Sorting is now possible by click or subscriber counts
refs https://github.com/TryGhost/Product/issues/3818
- added a check to compare hostname and pathnames of URLs. Different
subdomain or different pathname = different URLs, but protocol, www,
query parameters and hash fragments are ignored.
refs https://github.com/TryGhost/Product/issues/3832
---
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 1347a85</samp>
Added search functionality to the settings page using a custom hook and
a service. The `useSearch` hook uses the `useSearchService` function to
create a search service object that provides the filter and highlight
logic. The `highlightKeywords` function from the search service is
passed to the `SettingsGroupHeader` component to render the settings
with the matching keywords.
no issue
- bulk edit actions bypass the Bookshelf model hooks which meant our page reset behaviour in `onSaving` and `onDestroyed` was not being hit
- added overrides to `bulkEdit` and `bulkDestroy` to add the same page-reset behaviour any time we have a bulk edit or destroy
refs https://github.com/TryGhost/Product/issues/3938
- Updated "Recommendation received" email design
- Slightly improved how dark favicons show in dark mode
- Added TabView with counter variant to Storybook
refs https://github.com/TryGhost/Product/issues/3958
- Disabled automatic network retries for external site lookups (=> timed
out to 5s in every situation because it returned 404 when a site doesn't
implement the Ghost api)
- Disabled representing a modal when it is already present on hash
changes
- Added support for search params in modals
- Handle `?url` search param in the addRecommendationModal
refs https://github.com/TryGhost/Product/issues/3940
- the {{recommendation}} helper fetches recommendations from the Content
API and renders a HTML template with pre-defined CSS classes
- the HTML template can be overridden in themes, by uploading a file
under partials/recommendations.hbs
- the CSS classes are not pre-defined, they need to be defined in
individual themes
- if there are no recommendations, nothing is rendered
- the {{recommendations}} helper currently accepts "page", "limit",
"filter", and "order" as options
fixes https://github.com/TryGhost/Ghost/issues/17937
- We used a global Handlebars instance, which means it was reused across
Ghost
- Partials are different between parts of Ghost, that means the partials
were overwritten every time a normal Mailgun email was send
- All staff emails send after a normal newsletter would have invalid
styles because the partials for styles were overwritten
refs: https://github.com/TryGhost/Product/issues/3782
Latest version of request avoids using the OS-level name resolution system. This prevents long shutdowns from occuring where the OS is blocking the process from exiting cleanly.
The new version uses `got` 13, which has many breaking changes. Some of these are resolved within @tryghost/request (like response errors), but input parameters need to be changed.
no issue
- including a body snapshot for the pages API collection card tests causes issues because the generated HTML is dynamic and contains post creation times meaning the snapshot was unstable
- removed the body snapshot for the tests concerned for now as they are mostly there to catch saving issues rather than rendering issues
no issue
- added passthrough of `transaction` property when fetching post IDs otherwise SQLite will error with ` "Knex: Timeout acquiring a connection. The pool is probably full. Are you missing a .transacting(trx) call?"`
no issue
- act as regression tests for internal collection code changes
- useful to test as we've hit missing transaction passthrough for sqlite a couple of times that wasn't caught
fixes https://github.com/TryGhost/DevOps/issues/72
- in order to have greater control of labs flags outside of Ghost, this
commit allows Ghost to respect the value of `labs: { flagName: boolean }`
- this means we can hardcode a value to true or false, irrespective of
the value in the DB or GA flags array
- also adds tests to check functionality
refs https://github.com/TryGhost/Arch/issues/95
- We are releasing with a slight risk of failing over, so need a switch to allow disabling collections on hosted environment.
refs https://github.com/TryGhost/Arch/issues/95
refs 00e5f84d88
- Maintaining "latest" collection's static content is causing performance issues we are not able to overcome in a short period of time. Instead we leave it as a "dynamic" collection, which equals the contents of the Posts API. The "featured" collection will be saved in a "static" form as previously.
- This is roughly the same migration as in 5.64 version (see refed commit). It wipes out all of the stored collections data and only populates the featured collection.
refs https://github.com/TryGhost/Arch/issues/95
Rather than storing all of the relations between the latest collection and
posts, we know that it contains all posts. This means we don't have to keep the
collections posts in sync. Instead we can fetch them from the posts table. This
saves a lot of work during recalculation.
refs https://github.com/TryGhost/Arch/issues/95
Rather than a big nested loop to reconcile the in-memory vs. persisted
PostCollections we can instead use the events to know which rows we have to
delete and which we have to insert. This removes a tonne of work.
This implementation isn't perfect, and misses cases where the same post is
added and removed, our use-cases don't currently support that however.
no issue
Collection cards contain dynamic data that can change when there's any change to a published post but in Ghost all post/page content is rendered once on save and stored as a static string meaning we need a new approach for triggering a re-render of pages that plays well with caching.
- fixed typo in the relations/authors code that meant we weren't correctly calling the prototype method on the Post model inside the `onFetchedCollection` event handler
- updated Post model to clear the `html` field of all pages when saving or deleting a published post
- updated Post model to re-render `html` fields when fetching individual posts or a collection of posts
- modified `insertExtraPostsTags` fixture util to wrap it's concurrent post edits in a transaction otherwise MySQL errors because it hits a deadlock
refs https://ghost.slack.com/archives/C02G9E68C/p1695296293667689
- We block all outgoing networking by default in tests. When editing a
post/page, Ghost tries to send a webmention. Because of my earlier
changes
(1e3232cf82)
it tries 3 times - all network requests fail instantly when networking
is disabled - but it adds a delay in between those retries. So my change
disables retries during tests.
- Adds a warning when afterEach hook tooks longer than 2s due to
awaiting jobs/events
- We reduce the amount of webmentions that are send, by only sending
webmentions for added or removed urls, no longer when a post html is
changed (unless post is published/removed).
closes https://github.com/TryGhost/Product/issues/3818
- instead of fetching all recommendations and matching URLs on the frontend, we now query the database directly to find an existing Recommendation by URL. When comparing URLs, we don't take into account the protocol, www, query parameters nor hash fragments
fixes https://github.com/TryGhost/Product/issues/3911
For now we decided that we don't want to enable one-click-subscribe in
case a site has a required checkbox (which isn't shown during the
one-click-subscribe flow)
refs https://github.com/TryGhost/Product/issues/3850
- Added a recheck for recommendation related webmentions after boot (to
check missed webmentions during down time)
- Increased general timeouts to 15s for all webmention related HTTP
requests. Instead, increased retries to 3.
- Increased timeout for fetching webmention metadata from 2s to 15s
- Added more logging about verification and deletion status of
webmentions
refs https://ghost.slack.com/archives/C0568LN2CGJ/p1695149803260239
refs 290bc71d6a
- previous versions of `@tryghost/limit-service` did a full import of
lodash, which would bloat the bundle size, especially when we only use
a few of its functions
- I've since fixed that and this commit bumps Ghost to the smaller
version
This reverts commit 3e9da6df0c.
- changes introduced an error fetching `/admin/pages/` when using MySQL
- "The values in where clause must not be object or array"
no issue
Collection cards contain dynamic data that can change when there's any change to a published post but in Ghost all post/page content is rendered once on save and stored as a static string meaning we need a new approach for triggering a re-render of pages that plays well with caching.
- fixed typo in the relations/authors code that meant we weren't correctly calling the prototype method on the Post model inside the `onFetchedCollection` event handler
- updated Post model to clear the `html` field of all pages when saving or deleting a published post
- updated Post model to re-render `html` fields when fetching individual posts or a collection of posts
- modified `insertExtraPostsTags` fixture util to wrap it's concurrent post edits in a transaction otherwise MySQL errors because it hits a deadlock
fixes https://github.com/TryGhost/Product/issues/3830
This endpoint is required for recommendations to work: admin-x loads the incoming recommendations by querying the mentions endpoint. If the mentions flag was not enabled, this endpoint wasn't available.
closes https://github.com/TryGhost/Arch/issues/74
refs b5d1245be1
- We have turned off the collections feature flag after a unsuccessful attempt to make collections GA. With the flag turned off, collections_posts data has gone stale and needs repopulation to function properly again.
- This migration is meant to clear the data on collections_posts table and repopulated it again the same way initial migration did in 5.5/2023-07-10-05-16-55-add-built-in-collection-posts.js script.
refs https://github.com/TryGhost/Arch/issues/86
- Creating bookshelf models for each collection_post relation created a
massive overhead. On a dataset with 500k collections_posts records the
timing was roughly 7s comparing to 810ms after the optimization.
- Optimized memory and performance of collections fetching by querying post
ids only by default
refs https://github.com/TryGhost/DevOps/issues/80
- as part of moving Admin-X-Setting towards GA, we want to change it from
loading the settings externally via a CDN, to bundling it in with
Admin
- the bulk of the changes here are removing the config in Ghost, setting
up the copy to the Admin assets dir, and loading the new path in Admin
- several other changes have come along the way as I've cleaned up
unneeded code
refs https://github.com/TryGhost/Arch/issues/86
- Creating bookshelf models for each collection_post relation created a
massive overhead. On a dataset with 500k collections_posts records the
timing was roughly 7s comparing to 810ms after the optimization.
- Optimized memory and performance of collections fetching by querying post
ids only by default
refs https://github.com/TryGhost/Arch/issues/83
As we've only used the status and tiers in the cache key generation (as those
are currently the only ones that are present in DB's) we want to make sure that
content gating doesn't use any other properties, which means the cache behaviour
will match the content gating behaviour
refs https://github.com/TryGhost/Arch/issues/83
The options included are the only ones which have an effect on the response
data, as well as that we are using the properties of members which are used by
content-gating module. For the read operation we need to include the ID too.
refs https://github.com/TryGhost/Arch/issues/83
This allows endpoints to implement their own key generation, with access to the
frame object they can be smart about key generation and use only options and
context values that are appropriate.
closes https://github.com/TryGhost/Product/issues/3881
We observe the height of the recommendation table, and keep it fixed between pages — so that the UX is smoother when navigating between pages.
fixes https://github.com/TryGhost/Product/issues/3900
1. The service never returns a Recommendation Entity, but always plain
objects (which for now is the same as Recommendation without the
methods).
2. Updated the controller to be more readable and minimal (we keep this
controller, in addition to the existing endpoints and serializers)
- The controller does minimal validation and allows for type checking
(so we get compile time errors in case the service expects new fields)
- The controller uses the `UnsafeData` class to easily validate the
input from requests, and throws appropriate errors (with correct field
descriptions — "Expected a string at recommendations.0.title") without
too much boilerplate code. In addition the interface is typed, so we get
compile errors if there are breaking changes in the service.
- Removed `EntityWithIncludes`, since we now use plain objects, we
inject the relations directly into those plain objects (with some new
types that add type support)
- Added new tests to make sure that edits only affect the given fields,
and never undefined fields
- the schema.org data fo an author should primarily use the authors image as the image item instead of the cover image.
- otherwise the schema.org metadata will be invalid, since the image item is missing when no cover image has been uploaded.
closes https://github.com/TryGhost/Product/issues/3818
- in Admin, when adding a recommendation, the URL is compared against all existing ones. If the URL is already recommended, the publisher is shown an error: "A recommendation with this URL already exists.". Protocol, www, query parameters and hash fragments are ignored during the URL comparison.
- on the backend, there is another uniqueness validation for the recommendation URL. This check is redundant when adding a recommendation from Admin, but helps to keep data integrity when recommendations are added through other paths (e.g. via the API)
no issue
- added missing `context.public = true` option that tells our data fetching layer that this is a "Frontend/Content API" request and relevant filters should be applied
- adjusted require of posts service so it's only grabbed on the first render rather than every render
refs
https://www.notion.so/ghost/df5bdea8f7ea4aca9d25eceb6a1bf34c?v=be2f15b6b58b4c27a0e11374282bead0&p=163762d9513a4e6dbd60c28e19228fdc&pm=s
- Added a modal to confirm that the new support email has been verified.
- to achieve that a couple of adjustments had to be made
- Updated the RoutingProvider to handle routes with query params.
- Added a new useQueryParams hook to grab query params where needed.
- wired up the email verification api.
- added feature flags / labs logic to the core package with the new URL and updated test.
---
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 3ff8add</samp>
This pull request adds email verification functionality for the support
email address in the portal settings. It fixes a bug in the routing
provider, adds a new API function, a new custom hook, and a new modal
component to handle the verification process. It also updates the
settings query with the verified email address.
refs 488af56ef9
- The referenced commit introduced a postsService initializaiton at the top level of the module - causing cascading failure all the way down in the URL service, whe the ENV variables are not set.
- This fix is just a quick fix to unblock the main branch. A proper initialization of the service should be done ensuring we don't have to re-create a posts-service instance on each render method call.
