refs https://github.com/TryGhost/Arch/issues/86
- Creating bookshelf models for each collection_post relation created a
massive overhead. On a dataset with 500k collections_posts records the
timing was roughly 7s comparing to 810ms after the optimization.
- Optimized memory and performance of collections fetching by querying post
ids only by default
fixes https://github.com/TryGhost/Product/issues/3900
1. The service never returns a Recommendation Entity, but always plain
objects (which for now is the same as Recommendation without the
methods).
2. Updated the controller to be more readable and minimal (we keep this
controller, in addition to the existing endpoints and serializers)
- The controller does minimal validation and allows for type checking
(so we get compile time errors in case the service expects new fields)
- The controller uses the `UnsafeData` class to easily validate the
input from requests, and throws appropriate errors (with correct field
descriptions — "Expected a string at recommendations.0.title") without
too much boilerplate code. In addition the interface is typed, so we get
compile errors if there are breaking changes in the service.
- Removed `EntityWithIncludes`, since we now use plain objects, we
inject the relations directly into those plain objects (with some new
types that add type support)
- Added new tests to make sure that edits only affect the given fields,
and never undefined fields
- the schema.org data fo an author should primarily use the authors image as the image item instead of the cover image.
- otherwise the schema.org metadata will be invalid, since the image item is missing when no cover image has been uploaded.
closes https://github.com/TryGhost/Product/issues/3818
- in Admin, when adding a recommendation, the URL is compared against all existing ones. If the URL is already recommended, the publisher is shown an error: "A recommendation with this URL already exists.". Protocol, www, query parameters and hash fragments are ignored during the URL comparison.
- on the backend, there is another uniqueness validation for the recommendation URL. This check is redundant when adding a recommendation from Admin, but helps to keep data integrity when recommendations are added through other paths (e.g. via the API)
refs
https://www.notion.so/ghost/df5bdea8f7ea4aca9d25eceb6a1bf34c?v=be2f15b6b58b4c27a0e11374282bead0&p=163762d9513a4e6dbd60c28e19228fdc&pm=s
- Added a modal to confirm that the new support email has been verified.
- to achieve that a couple of adjustments had to be made
- Updated the RoutingProvider to handle routes with query params.
- Added a new useQueryParams hook to grab query params where needed.
- wired up the email verification api.
- added feature flags / labs logic to the core package with the new URL and updated test.
---
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 3ff8add</samp>
This pull request adds email verification functionality for the support
email address in the portal settings. It fixes a bug in the routing
provider, adds a new API function, a new custom hook, and a new modal
component to handle the verification process. It also updates the
settings query with the verified email address.
refs https://github.com/TryGhost/Product/issues/3875
When a member had a comped subscription, the portal was showing an
incorrect expiry date. This was because the `expiry_date` was being set
to the `created_at` date of the subscription, rather than the
`expiry_date` of the comped subscription
https://github.com/TryGhost/Arch/issues/90
- When a post.deleted event is emitted the original 'data' object does not contain an 'id' property. The logic in collections service assumes the id would be present to update the collections efficiently.
refs https://github.com/TryGhost/Product/issues/3874
- the new collections card needs to access the Content API rather than the Admin API in order to show the card as it will appear on the front-end but we don't have a default integration that can be fetched via the Admin API for Admin to use when fetching from the Content API
- adds a new "Ghost Core Content API" integration with the `core` type so that it can be read via the `/admin/integrations/` endpoint and used in Admin to make Content API requests
refs https://github.com/TryGhost/Arch/issues/86
bookshelf-relations was generating tonnes of select queries from the
posts table in order to update the relations. We've instead implemented
this ourselves, so as to avoid the superfluous fetches. Working closer to
the db like this is nice, and makes you think more about performance.
This logic could be pulled out into a util (not bookshelf plugin) where
it could be used explicitly, but with the complexity hidden, we'll see ig.
refs https://github.com/TryGhost/Arch/issues/87
- The newsletters in members payload have leaked internal properties from Public Members API. The code skipped the output serialization step, which is now in place.
- The newsletter resource returned from the API consistently returns these properties:
id,
name,
description,
sort_order
refs https://github.com/TryGhost/Arch/issues/87
- There was no test whatsoever! Adding a super basic test to have some certainty the output doesn't change after a refactor
refs https://github.com/TryGhost/Arch/issues/87
- The Members Admin API and members.* webhooks were returning too many fields in the nested `newsletters` objects. There was no "allowlist" serializer for the newsletter object, which meant every time we add a new field to the database we would unintentionally return extra fields without a second thought.
- With this change only following fields will be returned with `members[x].newsletters[x]`:
'id',
'name',
'description',
'status'
refs https://github.com/TryGhost/Arch/issues/87
- Round 2 for the previous commit. Removes use of `anyArray` for all
- Using `anyArray` in snapshot test is an anti-pattern which leads to leaking output fields unintentionally when the API changes.
- Adding these fixes is fundamental work before changing the output of 'member.newsletters' property
refs https://github.com/TryGhost/Arch/issues/87
- Using `anyArray` in snapshot test is an anti-pattern which leads to leaking output fields unintentionally when the API changes.
- Adding these fixes is fundamental work before changing the output of 'member.newsletters' property
no issue
- Do not set ?ref in recommendations if analytics is disabled
- Do not send url_history if analytics is disabled
- Expose outboundLinkTagging as a public setting
fixes https://github.com/TryGhost/Product/issues/3851
- Order was not applied via the CRUD plugin
- Removed usage of CRUD findAll, and swapped to Bookshelf fetchAll
instead, to decrease dependencies of invisible Bookshelf plugins logic
- Reverted page and limit options possibility via findAll method
fixes https://github.com/TryGhost/Product/issues/3822
fixes https://github.com/TryGhost/Product/issues/3838
This PR became a bit big because it affected multiple parts of Ghost
that needed to be updated to prevent breaking anything.
### Backend
- Added pagination to the recommendations API's
- Updated BookshelfRepository template implementation to handle
pagination
- Allow to pass `page` and `limit` options to Models `findAll`, to allow
fetching a page without also fetching the count/metadata (=> in the
repository pattern we prefer to fetch the count explicitly if we need
pagination metadata)
- Added E2E tests for public recommendations API (content API)
- Extended E2E tests of admin recommendations API
### Portal
- Corrected recommendations always loaded in Portal. Instead they are
now only fetched when the recommendations page is opened.
### Admin-X
- Added `usePagination` hook: internally used in the new
`usePaginatedQuery` hook. This automatically adds working pagination to
a query that can be used to display in a table by passing the
`pagination` and `isLoading` results to the `<Table>`
- Added placeholder `<LoadingIndicator>` component
- Added a loading indicator to `<Table>`. This remembers the previous
height of the table, to avoid layout jumps when going to the next page.
closes https://github.com/TryGhost/Product/issues/3803
Previously when the beta editor was enabled, using `?source=html` to create posts via the API would create posts in the old editor rather than the beta. This change switches conversion over to the new editor format when the beta is enabled so the full flow can be tested.
- added `htmlToLexicalConverter` method to our lexical library
- updated post and page input serializers to add html-to-lexical conversion when the beta editor is enabled
- updated post model to handle the mobiledoc+lexical co-existing state
- this is a special case that is only valid for `?source=html` because providing both directly via the API is prohibited
- we need the extra check here because at the input serializer layer we don't have access to the model to check if we're updating a mobiledoc post or a lexical post so the serializer sets both formats on a `?source=html` request when the beta is enabled and lets the model handle choosing the correct one
fixes https://github.com/TryGhost/Product/issues/3820
- This adds a new public site endpoint in the members API to check if a
site can offer the one-click-subscribe feature
- This is implemented on the members API as a copy of the `site`
endpoint because the admin API site endpoint is protected by CORS and
mainly because it can be served on a different domain than the
recommended site and this is hard to detect reliably from the frontend
- Added a new calculated setting `allow_self_signup`, which can replace
the setting that is currently used in Portal (best to do this after a
release otherwise we risk creating issues if a patch release happens)
refs https://github.com/TryGhost/Arch/issues/80
refs 3960bfac1d
- The killswitch (a setting in host settings) is needed to control the feature on a hosted environment, so we can safely turn it off if it causes any major issues.
refs https://github.com/TryGhost/Arch/issues/73
This is just an initial stab at making sure we don't introduce extra DB
queries related to collections without being aware of it.
refs https://ghost.slack.com/archives/C02G9E68C/p1692816097875899
- With introduction of extra e2e test coverage for Collections some tests started to fail at random. The root issue here was the transaction processing collections was started before the original bulk action (bulk edit, bulk publish/unpublish, etc.) was fully committed. The full transaction commit happens with the bulkAction method return inside of `if (!options.transacting) {` block.
refs https://github.com/TryGhost/Arch/issues/77
- We were missing e2e test coverage for when the tag used in collection filters was removed. This changeset improves the situation.
refs https://github.com/TryGhost/Arch/issues/77
- During initial development we have missed to support collections update when tags are added to posts in bulk. It's especially valid usecase since we can define automatic collection with a filter containing not yet existing tags.
refs https://github.com/TryGhost/Arch/issues/47
This ensures that we only have collections which have a valid filter in terms of
- Valid NQL string
- Uses only properties which are valid to filter on
- Only has an empty filter in the case of the "latest" collection
fixes https://github.com/TryGhost/Product/issues/3752
- Added some extra tests for edge cases
- Updated handling of multiple subscriptions so they are handled better
- Canceling a subscription when the member still has other subscriptions will now get handled correctly where the status and products of the member stay intact
refs https://github.com/TryGhost/Arch/issues/71
- With describe/it block levels mixed on the top level of the test suite the order fo test execution was scattered around. Having "describe" groups for each of the BREAD methods makes things more organized and readable.
refs https://github.com/TryGhost/Arch/issues/73
With the latest version of bookshelf-relations we're able to define a model
specific hook which allows us to ignore sort_order updates on automatic
collections, which don't require their order to be persisted.
fixes https://github.com/TryGhost/Product/issues/3728
- When importing members from Stripe with an existing offer, that didn't
exist in Ghost, the offer never got linked with the imported
subscription because of a missing return statement.
- Fixes importing offers with duplicate names
- Added E2E tests for creating members from a Stripe Customer ID
closes https://github.com/TryGhost/Arch/issues/76
- The posts test suite was failing when run in isolation. This was due to "collections" labs flag not being turned on, the events were not going through to collections service correctly
fixes https://github.com/TryGhost/Product/issues/3723
This also fixes usage of localhost instead of 127.0.0.1 as a test URL
for playwright. This caused issues for cookies because the member
impersonation navigated to 127.0.0.1 instead of localhost, meaning that
the next page.goto call would go to localhost and lose the cookies.
closes https://github.com/TryGhost/Ghost/issues/17681
- updated `prepareContextResource()` to make sure `show_title_and_feature_image` is always removed from pages
- updated `formatResponse.entries()` to apply the same `@page` local behaviour when it's passed a `data.page` object to account for custom routed pages
refs https://github.com/TryGhost/Ghost/pull/17609
- The tests for content gating started to fail with introduction of the index on `published_at` data in for `posts` table. The reason for the failure was identical `published_at` date set during the fixture insertion, making the returned results change order non-deterministically. The problem is mostly in how the test is set up as it's quite unrealistic to have multiple posts in the system inserted at the same time down to millisecond. Maybe... by some coincidence, but thats not a problem we should care too much about imo.
refs https://github.com/TryGhost/Arch/issues/18
- The prev/next helpers are slow and are causing major performance issues. The helpers are using `posts.published_at` for comparisons extensively, which causes a full table scan - bad for query performance.
- We use published_at in other queries too (like default order for queries fetching all posts), so there might be a slight performance boost across the system with this new index.
closes https://github.com/TryGhost/Product/issues/3661
- until now, Portal was not loaded if members were disabled. With the
introduction of Tips & Donations, signed-off readers can also make
payments, using the Portal link /#/portal/support.
- now, Portal is loaded when Tips & Donations are enabled, even if
Memberships are disabled
- depending on the member signup access, the top bar / trigger button
Portal buttons are hidden as before (signup/subscribe hidden if access is set to none, subscribe hidden if
access is set to invite-only)
- for any other signup / signin Portal links (e.g., added by the theme,
or added via a Post/Page), a new popup informs the reader when
Memberships are disabled: "Memberships unavailable, contact the site
owner for access".
refs TryGhost/Product#3638
- Added `convert_to_lexical` flag to the posts/pages edit endpoint
- Added 'convertToLexical' feature flag so we can enable/disable this
feature independently from the main lexical beta flag
- Modified admin posts/pages list to point to the lexical editor for
_all_ posts, regardless of mobiledoc vs lexical (if the flag is on)
- Added call to edit endpoint with `convert_to_lexical` in the lexical
editor admin route if the page/post is currently in mobiledoc and the
flag is enabled
fixes https://github.com/TryGhost/Product/issues/3687
After this change, relative URLs in emails will be replaced with
absolute URLs using the post URL. Making relative Portal URLs possible
etc.
Updates the test data generator to fix invalid URL encoding (somehow a
backslash + escaped double quote was added when it wasn't required).
no issue
- Snapshot tests were incorrect but we weren't catching it because CI
was retrying them and yielding false passes
- This fix just fixes the broken tests, which will allow us to fix the
issue with CI yielding false passing results
closes https://github.com/TryGhost/Product/issues/3666
- added computed setting "donations_enabled"
- added logic to persist "donations_suggested_amount" and "donations_currency"
- used "donations_suggested_amount" and "donations_currency" when initiating a new Stripe Checkout for donations
- added copy functionality to "your link" in Tips & Donations settings
refs https://github.com/TryGhost/Product/issues/3676
- add filter for sidebar display of theme errors (angry red box)
- filter specific to each page feature, will need to add each one by this approach
We have a global hook for the bookshelf-relations plugin which updates the
sort_order for pivot tables when saving the parent model. This hook requires
that we fetch each row in the pivot table related to the model and then run an
update on each one. Since we have a "latest" Collection this means at least N
update queries where N is the number of Posts for a site. For large sites this
was crippling the database. We only need the sort_order to be updated for
Collections with a type of "manual". We currently don't have a way to disable
the update based on model attributes, so instead we have disabled the update for
all Collections - this is okay because 1. Collections is not released and 2. we
don't have full support for manual Collections yet anyway.
refs https://github.com/TryGhost/Arch/issues/16
- When posts produce PostsBulkFeaturedEvent/PostsBulkUnfeaturedEvent the collections having a featured filter should update the posts belonging to them.
refs https://github.com/TryGhost/Product/issues/3648
- Refactored Members API RouterController.createCheckoutSession: Split the method into smaller parts so we can reuse individual parts for the upcoming donation checkout session.
- Wired up donation checkout creation
- Added donation events
refs https://github.com/TryGhost/Product/issues/3651
- This is a security fix that addresses an issue causing malicious users
to abuse the test / preview email API endpoint.
- We have multiple procedures in place now to limit such users.
- First, we now only allow one email address to be passed into the
`sendTestEmail` method. This method only have one purpose, which is to
compliment the test email functionality within the Editor in Admin and
therefore have no reason to send to more than one email address at a
time.
- We then add an additional rate limiter to prevent a user from making
multiple requests, eg via a script.
- The new imposed limit is 10 test emails per hour.
closes https://github.com/TryGhost/Arch/issues/58
- Following assumptions were broken:
- Posts Admin API should include posts of all statuses when filtering by collection
- Posts Content API should not include any unpublished posts
- Updated the "status" filter which fixes the problem. We still disallow any custom filters to be applied on top of collections filter.
Unfortuantely our framework is bookshelf centric so we have to refer to the
`withRelated` property rather than a more generic `include` property.
The collection entity already contains the list of post ids, so we can just
return the length of that array.
The test was addign an extra collection, but not cleaning it up - which makes it
hard to reason about other tests, especially when running them in isolation and
the state is different. This just cleans up the test and updates the browse test
to match the right snapshot.
The only usecases we need to support at the moment are reading individual
collections by ID and by Slug. We can extend this API as we get more usescases
in future.
The correct mechanism for fetching posts from a collection is via the Posts API.
This removes all functionality of getting posts from the Collections API.
Co-authored-by: Naz <hi@nazavo.com>
refs https://github.com/TryGhost/Arch/issues/46
- Similarly to post filters, collection filters now support both 'tag' and 'tags' nql filter keys when defining a filter for related tag slugs. For example, both `tag:avocado` and `tags:avocado` would both be valid collection filters that would filter by the same 'slug' property of the tags assigned to a post.
- Along with these changes had to rework the tags property of the collection posts to match the shape used in post resources. Moved from:
`tags: ['bacon', 'broc']`
to
`tags:[{slug: 'bacon'}, {slug: 'broc'}]`
closes https://github.com/TryGhost/Arch/issues/45
- This endpoint is here to keep the convention of being able to fetch the resource by it's slug through a `GET /{resource_name}/slug/:slug`. It has identical output as the `GET /collections/:id` endpoint
- The alternative would be having an alias and try fetching by :id and then by slug if the result for id was null, but that would be a completely new pattern we have not used anywhere else yet.
refs https://github.com/TryGhost/Arch/issues/16
- Using the API directly on the repository level prevented us from ensuring collection consistency through transactions.
- This change migrates the PostsRepository to use Bookshelf model layer directly, which also allows to put queries into transactions.
- Additional optimization here was removing the `getAllPosts` method from CollectionService. This is an attempt to reduce the API surface of the of the service before calling it a GA.
refs https://github.com/TryGhost/Arch/issues/41
- When an new collection is created the relational "tags" filter is now picked up properly and appropriate posts matching the tag filter are assigned and stored in the collection. Example collection filter that is now supported: `tags:['bacon']`
- Additionally cleaned up returned collection post DTOs, so we return as little data as possible and add only the fields that are needed
refs https://github.com/TryGhost/Arch/issues/16
- There's a race condition happening when processing multiple collection updates at the same time. It causes the state to be inconsistent between the runs.
- Once the event handling is improved these tests should be put back into action
refs https://github.com/TryGhost/Arch/issues/25
- When run against different DB Engines the returned order of collections belonging to a post is not consistent (SQLite vs MySQL). Having a primitive ordering by slug allows to keep the order compatible
refs https://github.com/TryGhost/Arch/issues/25
- The instance should have two built-in collections "latest" (prviously known as "index") and "featured". These have been filled through in-memory tricks before, now they should come pre-populated through fixtures mechanism.
This is an initial implementation which uses the Posts Content API rather than
the Collections Content API, this is because we haven't added the Collections
Content API yet, but we can added it later when necessary.
no issue
- Tests were failing for me locally because the snapshot for testing
page event payloads didn't include the new
`show_title_and_feature_image` property
- Updated snapshot to include this property, which also changed a few content lengths and `x-cache-invalidate` headers as a side effect
no issue
- These repositories were leftover from first phases of collections development. Not needed any longer as we have more specialized bookshelf repositories in the core code
no issue
- updated frontend's `formatResponse` method to add `@page` to the local template data
- added here because it's the first place we have both context and page data available
- makes the property available outside of the page context so it can be used to modify layouts
- updated `prepareContextResource()` to delete `show_title_and_feature_image`
- ensures `@page.show_title_and_feature_image` is the only way of accessing that property
- method is used when rendering multiple entries so it also prevents differences in context data between looped pages and single pages
closes https://github.com/TryGhost/Arch/issues/27
- We need a more convenient method of fetching posts belonging to a collection than by collection's "id". This change adds an alias to the existing endpoint `GET /collections/:id/posts/`. A non-valid ObjectID in the parameter is treated as a slug.
- due to schema changes between versions, we need to completely nuke the
DB between these tests
- this is definitely not the best way to do it but I'll fix properly next week
no issue
`show_title_and_feature_image` leads to more intuitive logic in themes and we can use `posts` rather than `posts_meta` as there are no longer row-length issues with MySQL 8.
- removed original add-column migration that was never in a release
- added new add-column migration that puts
`show_title_and_feature_image` column with a default of `true` on the `posts` table
- renamed property and default value everywhere
- bumped `@tryghost/admin-api-schema` to allow the new property through at the API level
refs @TryGhost/Product#3551
- PostsImporter would convert the HTML from the import file into
Mobiledoc, even if the post was written in Lexical
- As a result, the imported posts would have both mobiledoc & lexical
fields populated, which prevents the post from being updated in the
Lexical editor
- Added a check to see if the post was written in Lexical, and if so,
skip the HTML > Mobiledoc conversion
closes https://github.com/TryGhost/Product/issues/3557
- bumped `@tryghost/admin-api-schema` to allow passthrough of the new property in API requests
- updated output mapper to ensure property always returns a boolean rather than `null` in the case where `posts_meta` doesn't exist for a page
- updated `PostsService.copyPost()` to include the new property when copying
- updated `checkResponse` test util and snapshots to expect `hide_title_and_feature_image` property in page API responses
- fixed pages e2e test so it doesn't inadvertently modify the match object breaking later tests
closes https://github.com/TryGhost/Team/issues/3550
We want to allow an option to hide the title and feature image on a
per-page basis, to do that we need somewhere to store the setting value.
The existing `posts_meta` table is the simplest candidate, especially as
this is a single setting and we don't have a desire to introduce many
such settings.
- added migration that adds the `hide_title_and_feature_image` column to
the `posts_meta` table with a `boolean` data type and a default value of
`false` (matches behaviour of all existing pages)
- updated schema file for initial database creation
- removed property from API output via serializers to keep migration PR
minimal
refs https://github.com/TryGhost/DevOps/issues/39
- up until now, we've had a CI job which does a really basic test for
migrations, but it barely functions and misses bugs all the time
- this commit removes that and switches to an actual test suite for our
migrations, so we can ensure they function as expected
- also removes the env var hack I came up with for those migrations
tests
- this should lead to safer migrations and faster tests
fixes https://github.com/TryGhost/Team/issues/2937
Bumps juice to 9.1.0:
- Support for 'auto' width and height attributes
- Fixed a bug with counter-reset styles
- Dependencies updates
fixes https://github.com/TryGhost/Team/issues/3541
The email preheader, which is only present in the html version of an
email, is also included in the plaintext version of all emails. This
results in all text being duplicated twice in plaintext emails.
When we end up wiring this to the database, this generator will also ensure
uniqueness by appending/incrementing a number on the end of the slug. Long term
it would be good to offload this to a shared slug service, this could also
ensure that slugs are unique globally or between multiple tables, if desired
refs https://github.com/TryGhost/Team/issues/3169
- To be able to apply NQL filtering on Collection Posts the dates should be serialized to be ISO Date Strings instead of raw Dates. Otherwise, NQL filtering fails to compare Date with a Date String.
refs https://github.com/TryGhost/Ghost/issues/15725
This pull request adds a new configuration option for the Mailgun email
provider that allows the user to set the maximum number of recipients
per email batch via a new config option `bulkEmail.batchSize`
refs: https://github.com/TryGhost/Toolbox/issues/595
We're rolling out new rules around the node assert library, the first of which is enforcing the use of assert/strict. This means we don't need to use the strict version of methods, as the standard version will work that way by default.
This caught some gotchas in our existing usage of assert where the lack of strict mode had unexpected results:
- Url matching needs to be done on `url.href` see aa58b354a4
- Null and undefined are not the same thing, there were a few cases of this being confused
- Particularly questionable changes in [PostExporter tests](c1a468744b) tracked [here](https://github.com/TryGhost/Team/issues/3505).
- A typo see eaac9c293a
Moving forward, using assert strict should help us to catch unexpected behaviour, particularly around nulls and undefineds during implementation.
refs https://github.com/TryGhost/Toolbox/issues/592
- it turns out that `TRUNCATE` in CI takes ~300ms for all tables, but
`DELETE FROM` takes ~30ms
- whilst truncating is generally known to be faster, I believe it's only
faster on large tables
- this saves 90% of the time it takes to reset the DB in MySQL
closes https://github.com/TryGhost/Team/issues/3423
- For convenience we need a way to fetch posts that belong to a certain collection. This change adds support for `collection` query parameter: `/?collection=` which can be either an id or slug of the collections we are trying to fetch.
- When posts are fetched by collection we ignore any filters passed along in query parameters as collection is a "filter" by it's very nature.
no issue
This was a bit of an oversight from our feature built at the retreat. We
didn't take revisions into account for pages at all, but luckily it made
revisions without issues regardless.
It just wasn't accessible and users weren't able to restore via ADMIN
because the API didn't serve them at all.
This wires up the revisions relation to be served by the API so we can
retrieve it in Admin.
We've got some fairly simple diffing logic here to update the collections which
a post is in, the bulk of the changes here are to support the return of a DTO
rather than Bookshelf Model. This also helps improve the architecture because
we are step closer to removing infrastructure concerns (HTTP Response Headers)
from the business logic layer.
For now there is a crappy EventString which can be passed back to the
controller which can then handle any HTTP related concerns, although long term
these should be actual events like PostPublished or PostUpdated.
This prepares us to return a DTO rather than BookshelfModel to the serialiser
layer. When passing a BookshelfModel, the serialisation layer uses the model to
read from when building computed properties. By stripping values out in the
toJSON method it means that the DTO will be missing them and the computed
properties won't be able to be calculated. Instead we return ALL values to the
serialisation layer, and then strip out the ones that weren't requested in the
"clean" step.
This also inadvertently fixes the issue with `reading_time` requiring the
`html` field to be requested, we can now request just `reading_time`, as well
as have it included by default.
refs https://github.com/TryGhost/Team/issues/3423
- When querying for posts that belong to a collection we should be returning full post information just like we do for Posts API.
no issue
- We need to send information about Stripe being enabled or disabled in live mode to analytics
- This hooks up the Domain events listeners in the analytics service and processes this information accordingly
refs https://github.com/TryGhost/Toolbox/issues/592
- this commit extracts the regression tests into a separate workflow
- this means they run in parallel and reduce the time we have to wait
for DB tests in general
- also fixes a test that was reliant on being run after the E2E
tests (!)
refs https://github.com/TryGhost/Toolbox/issues/592
- async-await makes the code easier to read
- also performs a small optimization to only load the foreign_keys
pragma once for SQLite
refs https://github.com/TryGhost/Toolbox/issues/592
- we should reset the URL service to avoid event listeners piling up and
slowing down CI due to the number of events it has to process
refs https://github.com/TryGhost/Toolbox/issues/592
- heads up, I'm not really sure about this fix
- when we're wrapping `setTimeout`, time stops and mysql2 starts doing
weird things because we then shift time and it hits timeouts
- apparently `shouldAdvanceTime` should fix this by automatically
incrementing time along with the system clock
- given the problem is quite difficult to hit, I could just be seeing a
lack of this due to some other factor
- also removed unnecessary sinon sandbox creation as this is superfluous
closes https://github.com/TryGhost/Team/issues/3425
- Index collection is needed to support one of the usecases we have in the near future where we'd hold all posts that would be displayed on the "index" page.
closes https://github.com/TryGhost/Team/issues/3431
- We don't currently have a clear usecase to use the new pattern of updating posts as nested resource (of a collection). To simplify the API we are sticking with the approach of controlling where the post belongs to only through the Posts Admin API.
no issue
- The class should not rely on being passed a specific dependency, but rather needs to communicate with types what structure and method it needs to function correctly.
- Replaced the specific dependency to `sentry` with a generic definition of what is expected.
no issue
- In order to listen to `DomainEvents` for `MilestoneCreatedEvents` we need to add a `DomainEvents` listener and handler to the Segment analytics service.
- For better readability and to be more consistent with how code is currently written in Ghost, I refactored the service index file and split the two types of event listener into separate classes which is much cleaner and easier to test.
closes https://github.com/TryGhost/Team/issues/3325
Awaited `DomainEvents.allSettled()` to ensure domain event is fully
processed before asserting member was successfully updated
refs https://github.com/TryGhost/Team/issues/3145
Updates pintura integration to be switched on by default for all sites by adding a migration to update the default value for the setting.
refs https://github.com/TryGhost/Team/issues/3376
fixes b4a97d084f
- The in-memory stores are not cleaned up when the Ghost instance is "shallow restarted" between test suite runs, causing the initialization of built-in collections to run multiple times. The initialization should ever add the collections once.
refs https://github.com/TryGhost/Team/issues/3376
- When the Ghost instance is initialized it has to have a set of built-in collections. With these changes Ghost starts with a "featured posts" collection - available to be used right away.
refs https://github.com/TryGhost/Team/issues/3170
- This implementation allows to create an automatic collection with a filter defining automatically populated posts that belong to a collection
- To populate collection using a filter the API client can send a `filter` property along with a collection request
- Filter values are compatible with the filters used in Content API (https://ghost.org/docs/content-api/#filter)
fixes https://github.com/TryGhost/Team/issues/3331
This adds attribution tracking to the signup form. It sends a newly
created url history when sending the signup API call, this url history
will get translated to a proper attribution and saved on the backend. We
send a history with only a single item that contains the referrer
source, medium and path of the Embed form.
This also makes some changes to the E2E tests so that the tests run
in an https environment instead of about:blank.
refs https://github.com/TryGhost/Team/issues/3260
- We need a way to remove posts form collections without fetching the whole collection's content. This API method allows to remove posts from manual collections by collection id and post id.
- As a response it returns up to date collection state without the removed post.
refs https://github.com/TryGhost/Team/issues/3260
- We need a way to append posts to collections without sending over all of the posts that are already in the collection
- The API would receive post_id and collection_id as required fields and will optionally take in sort_order to control the ordering in the manual collection
refs https://github.com/TryGhost/Team/issues/2808
Updated the test to ensure that the date assertions do not unexpectedly
fail if the dates used are computed precisely at the start of a second
(no milliseconds `.000Z`)
refs https://github.com/TryGhost/Team/issues/3234
Added an e2e for the update check script to detect potential breakages
in the script due to uninitialised dependencies in the isolated
execution environment
fixes https://github.com/TryGhost/Team/issues/3296
Adds a new `signupForm` feature flag, that will enable/disable the new embeddable signup form code generation.
Since the new flag shares its name with a new config value (that contains the script location), this also fixes the feature helper to only use a config with the same name if it is a boolean.
closes https://github.com/TryGhost/Team/issues/3259
- API output mappers (soon to be serializers) are meant to work based on allowlist set of output properties. Having the allowlist early on will allow to track the API evolution consistently.
refs https://github.com/TryGhost/Team/issues/3248
The current test fixtures didn't include any hidden Tiers, so I've added
a new fixture to test the filtering of hidden Tiers. It's not enabled by
default to avoid breaking the existing tests.
refs https://github.com/TryGhost/Team/issues/3167
- This is scaffolding for collections API. Contains wiring for service wrapper, e2e test, and a browse endpoint
- Adds basic implementation of the GET /collections endpoint to build up upon
- Note, there are no permissions in this version as they will be added in later stages of development with migrations etc
refs https://github.com/TryGhost/Team/issues/3224
When a product has a slug that is a single letter, checking if a user
had access to view a post associated with that product would cause a 500
error. The underlying cause of this issue is
https://github.com/TryGhost/NQL/issues/20 This fix circumvents this
issue by providing a value that the nql lexer will not error out on
refs: https://github.com/TryGhost/Team/issues/3139https://github.com/TryGhost/Team/issues/3140
- Added duplicate post functionality to post list context menu
- Currently only a single post can be duplicated at a time
- Currently only enabled via the `Making it rain` flag
- Added admin API endpoint to copy a post - `POST ghost/api/admin/posts/<post_id>/copy/`
- Added admin API endpoint to copy a page - `POST ghost/api/admin/pages/<page_id>/copy/`
refs https://github.com/TryGhost/Toolbox/issues/515
Tiers are very frequently queried and we want to reduce the number of DB calls
we're making. We can store the Tiers in-memory, using the existing in-memory
repository patterns, but still persisting writes the the database.
We also have to update our test helpers, because they were bypassing the
repository for writes, but using it for reads resulting in an invalid cache
We want to cache access to Tiers, and it's easier to do that in the
TierRepository. So we update a heavy user of Tiers to use the Tier
service so it can take adv of caching. The serializers are a big
offender for making calls to fetch Tiers.
closes https://github.com/TryGhost/Team/issues/3176
We were waiting for the welcome email to send before responding to the
client that setup is complete, this was causing the client to hang when
running `ghost install local` as mail isn't configured by default.
We were incorrectly specifying the path of the file rather than relying on the
package exports, and this broke in the previous commit to a file rename. It
wasn't caught by CI because the browser tests are not run on pull requests
As discussed with the product team we want to enforce kebab-case file names for
all files, with the exception of files which export a single class, in which
case they should be PascalCase and reflect the class which they export.
This will help find classes faster, and should push better naming for them too.
Some files and packages have been excluded from this linting, specifically when
a library or framework depends on the naming of a file for the functionality
e.g. Ember, knex-migrator, adapter-manager
Whilst Admin API Integrations had the permissions to create invites they were
blocked from doing so at the HTTP level. We've removed this restriction for
creating Invites as well as browsing Roles, because a Role ID is necessary to
create an invite. The code was also not setup to support Admin API Integrations
as it made assumptions about the existence of a User. That has been updated in
the permissions layer - so that the Invites are limited to Contributors,
Authors and Editors as well as at the email layer, which has has the copy and
from address updated to reflect the lack of a User creating the Invite.
no issue
This commit removes the `memberAttribution` feature flag from the
codebase. Some CSS classes are not removed as removing them and updating
the associated CSS files have side effects sadly.
refs TryGhost/Team#3122
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 87727d9</samp>
Added `data-testid` attributes to various elements in the announcement
bar settings feature to enable Playwright testing. Fixed a potential bug
with the `visibilitySettings` getter in the `visibility.js` component.
Added Playwright tests for the announcement bar settings feature in
`announcement-bar-settings.spec.js`.
refs TryGhost/Ghost#16048
- When attempting to embed a Youtube video that has had embedding
disabled by its owner/author, Ghost displayed a generic error message
that didn't indicate the reason for the failed emebed.
- This change updated the error message when Youtube (or any provider)
returns 401: Unauthorized to indicate that the owner of the resource has
explicitly disabled embedding.
no issue
This pull request removes the `suppressionList` feature flag and all its
dependencies from the codebase. It makes the suppression list feature
the default and consistent behavior for all email events and
newsletters. It simplifies the UI, logic, and data related to email
events and newsletters. It affects several files in the
`ghost/admin/app`, `ghost/core/core`, and `ghost/members-api`
directories.
refs https://github.com/TryGhost/Ghost/security/advisories/GHSA-r97q-ghch-82j9
Because our filtering layer is so coupled to the DB and we don't generally
apply restrictions, it was possible to fetch authors and filter by their
password or email field. Coupled with the "starts with" operator this can be
used to brute force the first character of these fields by trying random
combinations until an author is included in the filter. After which the next
character can be brute forced, and so on until the data has been leaked
completely.
no issue
There was an error when generating the snapshot for this test. It never ran, so the snapshot was never committed. On top of that, the generated snapshot would change every time because the email verification token was not replaced with a static value.
https://github.com/TryGhost/Team/issues/3121
- Because the announcement data has to be available with member's context, it's only possible to have it in cross-origin requests in the Members API.
- Exposed the announcement bar data through `GET /members/api/announcement` endpoint
no issue
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at ebd1973</samp>
This pull request adds support for storing and tracking the status of
posts in revisions. It introduces a new `post_status` column and
property in the `post` and `PostRevision` models, and updates the
`PostRevisions.formatInput` method to handle it.
refs https://github.com/TryGhost/Team/issues/3099
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 83140b5</samp>
This pull request adds a new column `post_status` to the
`post_revisions` table to store the status of the post revision. It also
updates the schema definition, the migration file, and the integrity
test to reflect the new column.
refs @TryGhost/Team#3076
- added `save_revision` option to edit post endpoint
- this change covers the following cases:
1. we will not save a `post_revision` on every background autosave that
occurs after 3 seconds of inactivity in the editor
2. we will save a `post_revision` when the user hits `cmd+s` in the
editor to explicitly save
3. we will save a `post_revision` when the user navigates away from the
editor (e.g. by clicking the 'Posts' breadcrumb in the editor)
4. we will save a `post_revision` when the user publishes a post
5. we will save a `post_revision` when a user updates an already
published post
refs https://github.com/TryGhost/Ghost/pull/14264
- With a requirement change we need to transform `announcement_visibility` setting to be an "array" instead of a "string". Array structure will allow us to hold multiple filters at once giving more coverage to the audience targetting usecases.
- Example filter variations we'll support are:
[ ] Logged out visitors
[ ] Members
[ ] Free members
[ ] Paid members
refs https://github.com/TryGhost/Team/issues/3034
- adds new integration page for Pintura in Admin
- allows site owners to enable/disable the image editor integration
- allows self-hosters to upload the files for enabling Pintura image
editor
---------
Co-authored-by: Sodbileg Gansukh <sodbileg.gansukh@gmail.com>
refs https://github.com/TryGhost/Team/issues/3034
- adds 3 new settings for managing pintura image editor integration in
admin
- `pintura` setting controls the enabling of the feature
- `pintura_css_url` is used to store path to pintura css file uploaded
on integration page
- `pintura_js_url` is used to store path to pintura js file uploaded on
integration page
refs https://github.com/TryGhost/Team/issues/3011
- This is a data structure needed to support Announcement Bar feature -
allows to create custom site-wide announcements tailored to the
audience.
- The `announcement_content` is meant to hold displayed HTML content of
the announcement and will be exposed through unauthenticated Content
Site API
- The `announcement_visibility` sets the target audience to display the
Announcement Bart to:
- `public` - Everyone
- `visitors` - Logged out visitors only
- `members` - Members only
- `paid` - Paid members only
- The `announcement_background` sets the CSS class that should be
applied to the Announcement Bar. and will be exposed through
unauthenticated Content Site API. Three styles are available:
- `accent` - matches the color of the site accent
- `dark` - dark style
- `light` - light style
no issue
- added full author object to the post_revisions array on the /posts endpoint
- to be used in the post history modal to display who authored each revision
- the autogenerated constraint name was too long for MySQL, yielding the
error below (although it seems to be fine for sqlite)
- this change adds an explicit, shorthand name for the constraint to
stay under the limit on MySQL8
Error: alter table `post_revisions` add constraint
`post_revisions_author_id_foreign` foreign key (`author_id`) references
`users` (`id`), algorithm=copy - Specified key was too long; max key
length is 3072 bytes
no issue
- post_revisions will now be included in any request to the /posts
endpoint
- updated admin models to include post_revisions
- post revisions can now be accessed in the modal-portal-history via
this.post.post_revisions
no issue
- with this change, you can access a post's lexical revisions via the
api (e.g. `/posts/:id/?include=post_revisions`)
> _We are the post revisions, we won't be erased_
> _We rise from the ashes of the mobiledoc waste_
> _We join the API response, we claim our rightful place_
> _We are the post revisions, we show the truth of your face_
refs TryGhost/Team#2904
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at b3f5423</samp>
This pull request adds support for multiple formats of snippet content,
especially the `lexical` format, to the Ghost CMS. It modifies the
snippets API, model, and test files to handle the format conversion,
filtering, and serialization of snippets.
refs TryGhost/Team#2904
<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 04a6b06</samp>
This pull request adds a new `lexical` column to the `snippets` table to
store lexical information for snippets. It includes a migration script,
a schema update, and a schema hash update.
closesTryghost/Team#2975
- The test was failing because the order of the posts was not guaranteed
- The posts receive a published_at timestamp = new Date() when they are
created, unless a published_at date is passed in
- ~1/3 times the tests would run, the ordering would change and this
test would fail
- This commit fixes the test by passing in a published_at date to ensure
the order is always the same
refs TryGhost/Team#2833
- for mocha tests, we can add `this.retries(1)` to any flaky tests
- for playwright tests, we can add `test.describe.configure({ retries:
1})` to any `describe` block
- not a long-term solution, but it should help mitigate issues with flaky
tests in short term
refs https://github.com/TryGhost/Team/issues/2691
refs
939f25a987
- Resurrected refed commit that was adding tests for versioning API,
this time it's using dynamic replacements to match dynamic content of
the email using matchHTMLSnapshot / matchPlaintextSnapshot with dynamic
content replacements.
refs https://github.com/TryGhost/Team/issues/2691
- This bump changes the "sentEmailCount" method to a more descriptive "assertSentEmailCount" and adds chaining to this method.
refs TryGhost/Team#2691
- The bump adds possibility to make email's html/text snapshots with dynamic content. The breaking change here is with separate "matchPlaintextSnapshot" method extracted out of "matchMetadataSnapshot" to handle dynamic content in "text" part of the sent email.
