The future work around collections requires the read only portions
of the Collections Admin API to be enabled without a flag.
The write endpoints are only for custom collections which aren't required.
refs https://github.com/TryGhost/Arch/issues/73
Since we've fixed the issues which were executing performance affecting DB
queries, we can enable the background syncing of collections again. This couple
with a new migration to populate the default collections will allow us to start
work on the collections card and will make rolling that card out to GA smoother
and easier as all the required data will be in place.
refs https://github.com/TryGhost/Arch/issues/73
This is just an initial stab at making sure we don't introduce extra DB
queries related to collections without being aware of it.
refs https://ghost.slack.com/archives/C02G9E68C/p1692816097875899
- With introduction of extra e2e test coverage for Collections some tests started to fail at random. The root issue here was the transaction processing collections was started before the original bulk action (bulk edit, bulk publish/unpublish, etc.) was fully committed. The full transaction commit happens with the bulkAction method return inside of `if (!options.transacting) {` block.
refs https://github.com/TryGhost/Product/issues/3504
- This adds support for translations, but doesn't yet translate every
possible string in the app.
- Only active if beta translations is enabled
refs https://github.com/TryGhost/Arch/issues/77
- We were missing e2e test coverage for when the tag used in collection filters was removed. This changeset improves the situation.
refs https://github.com/TryGhost/Arch/issues/77
- During initial development we have missed to support collections update when tags are added to posts in bulk. It's especially valid usecase since we can define automatic collection with a filter containing not yet existing tags.
refs https://github.com/TryGhost/Arch/issues/47
This ensures that we only have collections which have a valid filter in terms of
- Valid NQL string
- Uses only properties which are valid to filter on
- Only has an empty filter in the case of the "latest" collection
refs https://github.com/TryGhost/Arch/issues/47
This change allows us to throw errors when instantiating invalid entities,
whilst not breaking things when we have bad data in the database. What we can
do is act as if the bad rows are not present, whilst surfacing an error in
sentry to alert us to such cases.
refs https://github.com/TryGhost/Arch/issues/47
We've configured bookshelf to force empty strings to null, but this is
undesired behaviour here, so unfortunately we have to leak some business
logic into the repository.
This needs to be done to correctly support our filter validation logic.
fixes https://github.com/TryGhost/Product/issues/3752
- Added some extra tests for edge cases
- Updated handling of multiple subscriptions so they are handled better
- Canceling a subscription when the member still has other subscriptions will now get handled correctly where the status and products of the member stay intact
refs https://github.com/TryGhost/Arch/issues/71
- With describe/it block levels mixed on the top level of the test suite the order fo test execution was scattered around. Having "describe" groups for each of the BREAD methods makes things more organized and readable.
refs https://github.com/TryGhost/Arch/issues/73
With the latest version of bookshelf-relations we're able to define a model
specific hook which allows us to ignore sort_order updates on automatic
collections, which don't require their order to be persisted.
fixes https://github.com/TryGhost/Product/issues/3728
- When importing members from Stripe with an existing offer, that didn't
exist in Ghost, the offer never got linked with the imported
subscription because of a missing return statement.
- Fixes importing offers with duplicate names
- Added E2E tests for creating members from a Stripe Customer ID
closes https://github.com/TryGhost/Arch/issues/76
- The posts test suite was failing when run in isolation. This was due to "collections" labs flag not being turned on, the events were not going through to collections service correctly
fixes https://github.com/TryGhost/Product/issues/3723
This also fixes usage of localhost instead of 127.0.0.1 as a test URL
for playwright. This caused issues for cookies because the member
impersonation navigated to 127.0.0.1 instead of localhost, meaning that
the next page.goto call would go to localhost and lose the cookies.
closes https://github.com/TryGhost/Ghost/issues/17681
- updated `prepareContextResource()` to make sure `show_title_and_feature_image` is always removed from pages
- updated `formatResponse.entries()` to apply the same `@page` local behaviour when it's passed a `data.page` object to account for custom routed pages
no issue
- We were loading the Explore iframe together with all assets in Admin
- This change will avoid that and only start loading and rendering Explore, once it's clicked
refs https://github.com/TryGhost/Ghost/pull/17609
- The tests for content gating started to fail with introduction of the index on `published_at` data in for `posts` table. The reason for the failure was identical `published_at` date set during the fixture insertion, making the returned results change order non-deterministically. The problem is mostly in how the test is set up as it's quite unrealistic to have multiple posts in the system inserted at the same time down to millisecond. Maybe... by some coincidence, but thats not a problem we should care too much about imo.
refs https://github.com/TryGhost/Arch/issues/18
- The prev/next helpers are slow and are causing major performance issues. The helpers are using `posts.published_at` for comparisons extensively, which causes a full table scan - bad for query performance.
- We use published_at in other queries too (like default order for queries fetching all posts), so there might be a slight performance boost across the system with this new index.
closes https://github.com/TryGhost/Product/issues/3661
- until now, Portal was not loaded if members were disabled. With the
introduction of Tips & Donations, signed-off readers can also make
payments, using the Portal link /#/portal/support.
- now, Portal is loaded when Tips & Donations are enabled, even if
Memberships are disabled
- depending on the member signup access, the top bar / trigger button
Portal buttons are hidden as before (signup/subscribe hidden if access is set to none, subscribe hidden if
access is set to invite-only)
- for any other signup / signin Portal links (e.g., added by the theme,
or added via a Post/Page), a new popup informs the reader when
Memberships are disabled: "Memberships unavailable, contact the site
owner for access".
no issue
- Keep an eye on Ghost's changelog (https://ghost.org/changelog/) for
the full feature announcement. 🎉
---
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 8367091</samp>
This pull request removes the `headerUpgrade` feature flag and enables
the new header card by default in the editor. It also hides the old
header card from the editor menu unless the `labs.headerV1` setting is
enabled.
closes https://github.com/TryGhost/Product/issues/3661
- until now, Portal was not loaded if members were disabled. With the
introduction of Tips & Donations, signed-off readers can also make
payments, using the Portal link /#/portal/support.
- now, Portal is loaded when Tips & Donations are enabled, even if
Memberships are disabled
- depending on the member signup access, the "sign in" / "subscribe"
Portal buttons are hidden (both hidden if none, signup hidden if
invite-only)
- for any other signup / signin Portal links (e.g., added by the theme,
or added via a Post/Page), a new popup informs the reader as such when
Memberships are disabled: "Memberships unavailable, contact the site owner for access".
refs: https://github.com/TryGhost/DevOps/issues/56
The new monobundle package now ignores any package directory if it doesn't contain a package.json. These non-packages are occasionally restored from cache when pulling dependencies.
refs TryGhost/Product#3638
- Added `convert_to_lexical` flag to the posts/pages edit endpoint
- Added 'convertToLexical' feature flag so we can enable/disable this
feature independently from the main lexical beta flag
- Modified admin posts/pages list to point to the lexical editor for
_all_ posts, regardless of mobiledoc vs lexical (if the flag is on)
- Added call to edit endpoint with `convert_to_lexical` in the lexical
editor admin route if the page/post is currently in mobiledoc and the
flag is enabled
fixes https://github.com/TryGhost/Product/issues/3687
After this change, relative URLs in emails will be replaced with
absolute URLs using the post URL. Making relative Portal URLs possible
etc.
Updates the test data generator to fix invalid URL encoding (somehow a
backslash + escaped double quote was added when it wasn't required).
no issue
---
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 2f86a35</samp>
Updated packages for rendering post and page content. This fixes some
bugs and improves the handling of HTML elements.
no issue
- some tweaks inside `kg-default-nodes`, requires an update in Ghost.
---
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
copilot:summary
no issue
- made adjustment to `kg-default-nodes` that needs bumping on Ghost to
have those changes reflected.
---
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 77ea2d6</samp>
Updated Koenig Editor dependencies to fix bugs and enhance performance.
This affects the `@tryghost/kg-default-nodes` and
`@tryghost/kg-lexical-html-renderer` packages in
`ghost/core/package.json`.
By using the `collections_posts` table as a pivot table, I couldn't see
a way to get bookshelf to *not* load the Post models for the relation.
We don't actually need those Post models for our usescases, the the
queries were causing issues with our database servers! Here we've added
a new CollectionPost model which allows us to treat the
collections_posts table as a resource ratherthan pivot, and means we can
use the hasMany relation rather than belongsToMany relation. This
removes all queries to the posts table when fetching collections with
relations
no issue
---
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at ee5f8bf</samp>
Updated Koenig Editor dependencies in `package.json` to improve editor
performance and stability.
no issue
- Snapshot tests were incorrect but we weren't catching it because CI
was retrying them and yielding false passes
- This fix just fixes the broken tests, which will allow us to fix the
issue with CI yielding false passing results
closes https://github.com/TryGhost/Product/issues/3666
- added computed setting "donations_enabled"
- added logic to persist "donations_suggested_amount" and "donations_currency"
- used "donations_suggested_amount" and "donations_currency" when initiating a new Stripe Checkout for donations
- added copy functionality to "your link" in Tips & Donations settings
refs: https://github.com/TryGhost/DevOps/issues/11
This is a pretty huge commit, but the relevant points are:
* Each importer no longer needs to be passed a set of data, it just gets the data it needs
* Each importer specifies its dependencies, so that the order of import can be determined at runtime using a topological sort
* The main data generator function can just tell each importer to import the data it has
This makes working on the data generator much easier.
Some other benefits are:
* Batched importing, massively speeding up the whole process
* `--tables` to set the exact tables you want to import, and specify the quantity of each
We've been seeing an increased number of database queries, which are potentially
caused by the collections syncing logic. Whilst we haven't narrowed the cause
down precisely we want to disable this logic unless the flag is enabled.
refs TryGhost/Product#3609
- author and publisher styles were mixed up in mobiledoc
- made consistent for author coming after publisher for md and lexical
- updated renderer packages
refs https://github.com/TryGhost/Product/issues/3676
- add filter for sidebar display of theme errors (angry red box)
- filter specific to each page feature, will need to add each one by this approach
We have a global hook for the bookshelf-relations plugin which updates the
sort_order for pivot tables when saving the parent model. This hook requires
that we fetch each row in the pivot table related to the model and then run an
update on each one. Since we have a "latest" Collection this means at least N
update queries where N is the number of Posts for a site. For large sites this
was crippling the database. We only need the sort_order to be updated for
Collections with a type of "manual". We currently don't have a way to disable
the update based on model attributes, so instead we have disabled the update for
all Collections - this is okay because 1. Collections is not released and 2. we
don't have full support for manual Collections yet anyway.
refs https://github.com/TryGhost/Arch/issues/16
- When posts produce PostsBulkFeaturedEvent/PostsBulkUnfeaturedEvent the collections having a featured filter should update the posts belonging to them.
refs https://github.com/TryGhost/Product/issues/3648
- Refactored Members API RouterController.createCheckoutSession: Split the method into smaller parts so we can reuse individual parts for the upcoming donation checkout session.
- Wired up donation checkout creation
- Added donation events
no issue
- We made some styling changes in the rendering nodes for the new header
card.
- This bumps the affected packages.
---
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 655e595</samp>
Updated content rendering packages to fix bugs and improve output. This
affects how Ghost converts markdown to `HTML`, `AMP`, or `email`
formats.
refs https://github.com/TryGhost/Product/issues/3651
- This is a security fix that addresses an issue causing malicious users
to abuse the test / preview email API endpoint.
- We have multiple procedures in place now to limit such users.
- First, we now only allow one email address to be passed into the
`sendTestEmail` method. This method only have one purpose, which is to
compliment the test email functionality within the Editor in Admin and
therefore have no reason to send to more than one email address at a
time.
- We then add an additional rate limiter to prevent a user from making
multiple requests, eg via a script.
- The new imposed limit is 10 test emails per hour.
refs https://github.com/TryGhost/Product/issues/3651
- This is a security fix that addresses an issue causing malicious users
to abuse the test / preview email API endpoint.
- We have multiple procedures in place now to limit such users.
- First, we now only allow one email address to be passed into the
`sendTestEmail` method. This method only have one purpose, which is to
compliment the test email functionality within the Editor in Admin and
therefore have no reason to send to more than one email address at a
time.
- We then add an additional rate limiter to prevent a user from making
multiple requests, eg via a script.
- The new imposed limit is 10 test emails per hour.
closes https://github.com/TryGhost/Arch/issues/58
- Following assumptions were broken:
- Posts Admin API should include posts of all statuses when filtering by collection
- Posts Content API should not include any unpublished posts
- Updated the "status" filter which fixes the problem. We still disallow any custom filters to be applied on top of collections filter.
no issues
- using .kg-v2 as an indicator to add the v2-specific styles
- .kg-header-card and .kg-header-card-button are the class names that confict with the existing Header card, so only using .kg-v2 to style those elements
refs https://github.com/TryGhost/Product/issues/3561
- Added a static section to Membership settings with a button to copy
the Tips & donations Stripe link
- Added a `tipsAndDonations` feature flag
This ensures that collections are updated in the background regardless of
whether or not the labs flag is enabled, which is important for the stability
of the collections database tables. In order to make sure we don't add a tonne
of event listeners during the tests we have to add a flag to make sure that the
service is treated as a singleton and only instanciated once. This should only
affect code running in tests, as we don't initialise services multiple times
Unfortuantely our framework is bookshelf centric so we have to refer to the
`withRelated` property rather than a more generic `include` property.
The collection entity already contains the list of post ids, so we can just
return the length of that array.
The test was addign an extra collection, but not cleaning it up - which makes it
hard to reason about other tests, especially when running them in isolation and
the state is different. This just cleans up the test and updates the browse test
to match the right snapshot.
The only usecases we need to support at the moment are reading individual
collections by ID and by Slug. We can extend this API as we get more usescases
in future.
The correct mechanism for fetching posts from a collection is via the Posts API.
This removes all functionality of getting posts from the Collections API.
Co-authored-by: Naz <hi@nazavo.com>
refs https://github.com/TryGhost/Arch/issues/46
- Similarly to post filters, collection filters now support both 'tag' and 'tags' nql filter keys when defining a filter for related tag slugs. For example, both `tag:avocado` and `tags:avocado` would both be valid collection filters that would filter by the same 'slug' property of the tags assigned to a post.
- Along with these changes had to rework the tags property of the collection posts to match the shape used in post resources. Moved from:
`tags: ['bacon', 'broc']`
to
`tags:[{slug: 'bacon'}, {slug: 'broc'}]`
no issues
- some themes have global styles applied to inputs and it breaks the card input layout because of the style conflict
- this adds some default margin values to fix the issue
closes https://github.com/TryGhost/Arch/issues/45
- This endpoint is here to keep the convention of being able to fetch the resource by it's slug through a `GET /{resource_name}/slug/:slug`. It has identical output as the `GET /collections/:id` endpoint
- The alternative would be having an alias and try fetching by :id and then by slug if the result for id was null, but that would be a completely new pattern we have not used anywhere else yet.
We were not passing the `slug` to the `apiOptions` so the posts were not
correctly filtered, and on top of that the `collection` option had not been
added to the allow list of the Posts Content API. With these two fixes the
collection helper works as expected.
refs https://github.com/TryGhost/Arch/issues/16
- We made a decision to limit the API for fetching posts belonging to a certain collection to only the Posts API. The endpoint on the collections was an experiment that only brings unnecessary maintenance at this point of time.
refs https://github.com/TryGhost/Arch/issues/16
- Using the API directly on the repository level prevented us from ensuring collection consistency through transactions.
- This change migrates the PostsRepository to use Bookshelf model layer directly, which also allows to put queries into transactions.
- Additional optimization here was removing the `getAllPosts` method from CollectionService. This is an attempt to reduce the API surface of the of the service before calling it a GA.
refs https://github.com/TryGhost/Arch/issues/16
- Having transactional collection post updates makes sure there are no race conditions when updating collection_posts relations. Without the transactions collection was prone to update relations based on a stale state causing problems like described in the linked issue
refs https://github.com/TryGhost/DevOps/issues/48
- due to changes in the build workflow, we no longer depended on
`ghost`'s own `build` before producing the tarball
- as a result, the minified css was not produced and private pages were
unstyled
- this adds our own `build` step to the list of dependencies for
`archive`
no issue
---
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at bfc259e</samp>
Updated content rendering packages to fix bugs and improve performance.
This affects how Ghost outputs HTML and AMP pages from the
`@tryghost/kg-default-nodes` and `@tryghost/kg-lexical-html-renderer`
modules.
refs https://github.com/TryGhost/Arch/issues/41
- When an new collection is created the relational "tags" filter is now picked up properly and appropriate posts matching the tag filter are assigned and stored in the collection. Example collection filter that is now supported: `tags:['bacon']`
- Additionally cleaned up returned collection post DTOs, so we return as little data as possible and add only the fields that are needed
no issue
- bumps `@tryghost/kg-lexical-html-renderer` which includes a change to drastically reduce the number of `jsdom` instantiations made when rendering text content
no issue
- bumps `@tryghost/kg-lexical-html-renderer` which includes a change to drastically reduce the number of `jsdom` instantiations made when rendering text content
refs https://github.com/TryGhost/Arch/issues/16
- There's a race condition happening when processing multiple collection updates at the same time. It causes the state to be inconsistent between the runs.
- Once the event handling is improved these tests should be put back into action
refs https://github.com/TryGhost/Product/issues/3508
- copied over styles from the signup card, which is what the new header
card is loosely based on.
- this is meant to be hacked till we have the new header card look
pretty.
---
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 150aa92</samp>
This change introduces a new header card v2 component that allows users
to create custom headers for their Ghost site. It adds a new CSS file
`header_v2.css` that defines the styles and responsive behavior for the
component. The file uses modern CSS features to achieve a flexible and
consistent design. The file is part of the frontend module of the Ghost
core repository.
refs https://github.com/TryGhost/Arch/issues/25
- When run against different DB Engines the returned order of collections belonging to a post is not consistent (SQLite vs MySQL). Having a primitive ordering by slug allows to keep the order compatible
refs https://github.com/TryGhost/Ghost/pull/17247
- When the instance is started the posts_collections records should be pre-filled for built-in collections. This is to avoid expensive processing during every instance startup.
https://github.com/TryGhost/Arch/issues/25
- This naming matches how the collection will be visible in the cards and how it's marketed. Naming it "index" even for internal purposes will get confusing.
refs https://github.com/TryGhost/Arch/issues/25
- The instance should have two built-in collections "latest" (prviously known as "index") and "featured". These have been filled through in-memory tricks before, now they should come pre-populated through fixtures mechanism.
This is an initial implementation which uses the Posts Content API rather than
the Collections Content API, this is because we haven't added the Collections
Content API yet, but we can added it later when necessary.
We want to keep the differences between the Content & Admin API's in the API
layer, rather than at the service layer. The caching functionality hasn't been
moved yet though, and can be handled separately.
no issue
- Tests were failing for me locally because the snapshot for testing
page event payloads didn't include the new
`show_title_and_feature_image` property
- Updated snapshot to include this property, which also changed a few content lengths and `x-cache-invalidate` headers as a side effect
- this switches us to using Nx for `ghost:dev`, which means we can
configure its dependencies and ensure that the TS projects are built
beforehand
- also switches to Nx for `ghost:archive` so we don't need to run
`build:ts` for all `yarn` commands
no issue
- These repositories were leftover from first phases of collections development. Not needed any longer as we have more specialized bookshelf repositories in the core code
no issue
- updated frontend's `formatResponse` method to add `@page` to the local template data
- added here because it's the first place we have both context and page data available
- makes the property available outside of the page context so it can be used to modify layouts
- updated `prepareContextResource()` to delete `show_title_and_feature_image`
- ensures `@page.show_title_and_feature_image` is the only way of accessing that property
- method is used when rendering multiple entries so it also prevents differences in context data between looped pages and single pages
refs https://github.com/TryGhost/DevOps/issues/45
- this switches the monorepo over to using Nx instead of Lerna, because
we don't currently need the versioning+publishing capabilities
- this also adds an `nx.json`, which allows us to enable task caching
- also adds `build:ts` to the TS projects, which is cached for fast execution
- how these interact with the dev.js script will hopefully soon be
reworked to be a better experience
closes https://github.com/TryGhost/Arch/issues/27
- We need a more convenient method of fetching posts belonging to a collection than by collection's "id". This change adds an alias to the existing endpoint `GET /collections/:id/posts/`. A non-valid ObjectID in the parameter is treated as a slug.
refs https://github.com/TryGhost/Product/issues/3508
- added a feature flag for Header Cards that's being upgraded.
---
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 10ac1e4</samp>
This pull request adds a new feature flag `headerUpgrade` to the server
and the admin settings, which allows the user to opt-in to a new header
design. The feature flag is only available if the labs toggle is
enabled.
- due to schema changes between versions, we need to completely nuke the
DB between these tests
- this is definitely not the best way to do it but I'll fix properly next week
no issue
`show_title_and_feature_image` leads to more intuitive logic in themes and we can use `posts` rather than `posts_meta` as there are no longer row-length issues with MySQL 8.
- removed original add-column migration that was never in a release
- added new add-column migration that puts
`show_title_and_feature_image` column with a default of `true` on the `posts` table
- renamed property and default value everywhere
- bumped `@tryghost/admin-api-schema` to allow the new property through at the API level
refs @TryGhost/Product#3551
- PostsImporter would convert the HTML from the import file into
Mobiledoc, even if the post was written in Lexical
- As a result, the imported posts would have both mobiledoc & lexical
fields populated, which prevents the post from being updated in the
Lexical editor
- Added a check to see if the post was written in Lexical, and if so,
skip the HTML > Mobiledoc conversion
