Commit Graph

472 Commits

Author SHA1 Message Date
Simon Backx
b545dfa0cc
Improved recommendation controller structure (#18208)
fixes https://github.com/TryGhost/Product/issues/3900

1. The service never returns a Recommendation Entity, but always plain
objects (which for now is the same as Recommendation without the
methods).
2. Updated the controller to be more readable and minimal (we keep this
controller, in addition to the existing endpoints and serializers)
- The controller does minimal validation and allows for type checking
(so we get compile time errors in case the service expects new fields)
- The controller uses the `UnsafeData` class to easily validate the
input from requests, and throws appropriate errors (with correct field
descriptions — "Expected a string at recommendations.0.title") without
too much boilerplate code. In addition the interface is typed, so we get
compile errors if there are breaking changes in the service.
- Removed `EntityWithIncludes`, since we now use plain objects, we
inject the relations directly into those plain objects (with some new
types that add type support)
- Added new tests to make sure that edits only affect the given fields,
and never undefined fields
2023-09-18 14:36:49 +00:00
Simon Backx
4e2710ada2 Added stats to recommendations endpoints
fixes https://github.com/TryGhost/Product/issues/3854
fixes https://github.com/TryGhost/Product/issues/3864
2023-09-15 15:57:27 +02:00
Sag
6e68c43f78
Added uniqueness validation for the recommendation URL (#18163)
closes https://github.com/TryGhost/Product/issues/3818

- in Admin, when adding a recommendation, the URL is compared against all existing ones. If the URL is already recommended, the publisher is shown an error: "A recommendation with this URL already exists.". Protocol, www, query parameters and hash fragments are ignored during the URL comparison.
- on the backend, there is another uniqueness validation for the recommendation URL. This check is redundant when adding a recommendation from Admin, but helps to keep data integrity when recommendations are added through other paths (e.g. via the API)
2023-09-15 13:14:47 +00:00
Naz
99f29a169c Removed post delete related event handling
closes https://github.com/TryGhost/Arch/issues/91

- We have on cascade delete (a9f9f6121a/ghost/core/core/server/data/schema/schema.js (L1068)) on `post_id` column which handles post deletion logic automatically on DB level.
- The commented out handlers in the long term should be hooked up with public CollectionService methods on the client side.
2023-09-15 13:50:25 +08:00
Simon Backx
82079c1dc5 Added click tracking endpoints for recommendations
fixes https://github.com/TryGhost/Product/issues/3853
2023-09-14 14:47:02 +02:00
Michael Barrett
a1f056ee86
🐛 Fixed portal showing incorrect expiry date for comped subscription (#18120)
refs https://github.com/TryGhost/Product/issues/3875

When a member had a comped subscription, the portal was showing an
incorrect expiry date. This was because the `expiry_date` was being set
to the `created_at` date of the subscription, rather than the
`expiry_date` of the comped subscription
2023-09-14 08:46:23 +01:00
Naz
1d7bb44a94 Fixed post.deleted event mapping
https://github.com/TryGhost/Arch/issues/90

- When a post.deleted event is emitted the original 'data' object does not contain an 'id' property. The logic in collections service assumes the id would be present to update the collections efficiently.
2023-09-14 14:31:23 +08:00
Fabien 'egg' O'Carroll
9dde39b2a4
Handled CollectionPost relations manually (#18081)
refs https://github.com/TryGhost/Arch/issues/86

bookshelf-relations was generating tonnes of select queries from the
posts table in order to update the relations. We've instead implemented
this ourselves, so as to avoid the superfluous fetches. Working closer to
the db like this is nice, and makes you think more about performance.

This logic could be pulled out into a util (not bookshelf plugin) where
it could be used explicitly, but with the complexity hidden, we'll see ig.
2023-09-13 14:16:22 +07:00
Naz
bbb5b64ec5 Cleaned up member's neswletter test suite
refs https://github.com/TryGhost/Arch/issues/87

- We just introduce member's newsletters serialization, so removed the comments reminding to do so.
2023-09-13 13:18:43 +08:00
Naz
282de7f0b4 Added newsletter payload serializer
refs https://github.com/TryGhost/Arch/issues/87

- The newsletters in members payload have leaked internal properties from Public Members API. The code skipped the output serialization step, which is now in place.
- The newsletter resource returned from the API consistently returns these properties:
id,
name,
description,
sort_order
2023-09-13 13:18:43 +08:00
Naz
e73b16979f Trimmed down Members Admin API response data
refs https://github.com/TryGhost/Arch/issues/87

- The Members Admin API and members.* webhooks were returning too many fields in the nested `newsletters` objects. There was no "allowlist" serializer for the newsletter object, which meant every time we add a new field to the database we would unintentionally return extra fields without a second thought.
- With this change only following fields will be returned with `members[x].newsletters[x]`:
'id',
'name',
'description',
'status'
2023-09-13 13:18:43 +08:00
Naz
2e84cdca36 Added specific newsletters and attribution tests
refs https://github.com/TryGhost/Arch/issues/87

- Round 2 for the previous commit. Removes use of `anyArray` for all
- Using `anyArray` in snapshot test is an anti-pattern which leads to leaking output fields unintentionally when the API changes.
- Adding these fixes is fundamental work before changing the output of 'member.newsletters' property
2023-09-13 13:18:43 +08:00
Naz
4999523289 Added specific newsletters and attribution tests
refs https://github.com/TryGhost/Arch/issues/87

- Using `anyArray` in snapshot test is an anti-pattern which leads to leaking output fields unintentionally when the API changes.
- Adding these fixes is fundamental work before changing the output of 'member.newsletters' property
2023-09-13 13:18:43 +08:00
Simon Backx
d166977ae4
Updated Portal to always honor outboundLinkTagging setting (#18079)
no issue

- Do not set ?ref in recommendations if analytics is disabled
- Do not send url_history if analytics is disabled
- Expose outboundLinkTagging as a public setting
2023-09-12 10:27:04 +00:00
Simon Backx
f566729ed6
Fixed recommendation order (#18060)
fixes https://github.com/TryGhost/Product/issues/3851

- Order was not applied via the CRUD plugin
- Removed usage of CRUD findAll, and swapped to Bookshelf fetchAll
instead, to decrease dependencies of invisible Bookshelf plugins logic
- Reverted page and limit options possibility via findAll method
2023-09-11 11:04:34 +00:00
Simon Backx
669be72673
Wired up pagination to recommendations (#18018)
fixes https://github.com/TryGhost/Product/issues/3822
fixes https://github.com/TryGhost/Product/issues/3838

This PR became a bit big because it affected multiple parts of Ghost
that needed to be updated to prevent breaking anything.

### Backend
- Added pagination to the recommendations API's
- Updated BookshelfRepository template implementation to handle
pagination
- Allow to pass `page` and `limit` options to Models `findAll`, to allow
fetching a page without also fetching the count/metadata (=> in the
repository pattern we prefer to fetch the count explicitly if we need
pagination metadata)
- Added E2E tests for public recommendations API (content API)
- Extended E2E tests of admin recommendations API

### Portal
- Corrected recommendations always loaded in Portal. Instead they are
now only fetched when the recommendations page is opened.

### Admin-X
- Added `usePagination` hook: internally used in the new
`usePaginatedQuery` hook. This automatically adds working pagination to
a query that can be used to display in a table by passing the
`pagination` and `isLoading` results to the `<Table>`
- Added placeholder `<LoadingIndicator>` component
- Added a loading indicator to `<Table>`. This remembers the previous
height of the table, to avoid layout jumps when going to the next page.
2023-09-08 12:32:06 +02:00
Kevin Ansfield
8c91662a47
Added conversion to beta editor format when creating content via ?source=html (#18000)
closes https://github.com/TryGhost/Product/issues/3803

Previously when the beta editor was enabled, using `?source=html` to create posts via the API would create posts in the old editor rather than the beta. This change switches conversion over to the new editor format when the beta is enabled so the full flow can be tested.

- added `htmlToLexicalConverter` method to our lexical library
- updated post and page input serializers to add html-to-lexical conversion when the beta editor is enabled
- updated post model to handle the mobiledoc+lexical co-existing state
  - this is a special case that is only valid for `?source=html` because providing both directly via the API is prohibited
  - we need the extra check here because at the input serializer layer we don't have access to the model to check if we're updating a mobiledoc post or a lexical post so the serializer sets both formats on a `?source=html` request when the beta is enabled and lets the model handle choosing the correct one
2023-09-06 21:16:40 +00:00
Simon Backx
f71c074d31
Added one-click-subscribe detection (#17995)
fixes https://github.com/TryGhost/Product/issues/3820

- This adds a new public site endpoint in the members API to check if a
site can offer the one-click-subscribe feature
- This is implemented on the members API as a copy of the `site`
endpoint because the admin API site endpoint is protected by CORS and
mainly because it can be served on a different domain than the
recommended site and this is hard to detect reliably from the frontend
- Added a new calculated setting `allow_self_signup`, which can replace
the setting that is currently used in Portal (best to do this after a
release otherwise we risk creating issues if a patch release happens)
2023-09-06 15:11:14 +00:00
Simon Backx
8600ccf387 Added BookshelfRepository and BookshelfRecommendationRepository
refs https://github.com/TryGhost/Product/issues/3800
2023-09-01 15:10:34 +02:00
Simon Backx
d5c8804e23 Updated InMemoryRecommendationRepository to follow newer pattern
refs https://github.com/TryGhost/Product/issues/3800
2023-09-01 15:10:34 +02:00
Sag
96d9099195
Revert "Added support for importing Stripe Coupons as Offers (#17415)" (#17915)
refs https://github.com/TryGhost/Product/issues/3674
refs https://github.com/TryGhost/Product/issues/3675

- this reverts commits 8a32941ae8 and
b587429008
- the reverted commits added some logic to create offers based on a
Stripe coupon. However, the logic bypassed the Offer entity, and
therefore skipped any validations/constraints — causing invalid data in
the database and some sites to crash.
2023-09-01 09:49:29 +02:00
Simon Backx
96fefaea69
Added well known recommendations service (#17895)
fixes https://github.com/TryGhost/Product/issues/3797 
fixes https://github.com/TryGhost/Product/issues/3776 
fixes https://github.com/TryGhost/Product/issues/3798

- Added support for storing json webmentions
- Improved handling deleted webmentions (set deleted to true instead of verified to false)
2023-08-31 14:57:18 +00:00
Simon Backx
78ae776c5e
Added Recommendations browse API to admin-x-settings (#17870)
refs https://github.com/TryGhost/Product/issues/3786
2023-08-30 10:25:31 +00:00
Naz
b54191dbe4 Added killswitch for collection sync functionality
refs https://github.com/TryGhost/Arch/issues/80
refs 3960bfac1d

- The killswitch (a setting in host settings) is needed to control the feature on a hosted environment, so we can safely turn it off if it causes any major issues.
2023-08-30 18:22:22 +08:00
Sag
f1cd6432a8
Added new setting to enable Recommendations (#17856)
closes https://github.com/TryGhost/Product/issues/3772

- Recommendations are disabled by default and will get enabled when a
first recommendation is added. The "recommendations_enabled" setting
will be not shown explicitly in Admin, it will rather be set implicitly
based on whether there is at least one recommendation
- The "recommendations_enabled" setting belongs to a new setting group
called "recommendations"

Tech Spec: https://www.notion.so/ghost/Recommentions-Tech-Spec-598f09e4109f4eef8d9e32db7245a90c?pvs=4
2023-08-30 08:56:21 +00:00
Simon Backx
935ac43584
Added recommendations CRUD api (#17845)
fixes https://github.com/TryGhost/Product/issues/3784

- Includes migrations for new permissions needed for the new endpoints
2023-08-29 15:06:57 +00:00
Fabien 'egg' O'Carroll
44ae0dcbe1
Added rudimentary tests for Collections DB queries (#17820)
refs https://github.com/TryGhost/Arch/issues/73

This is just an initial stab at making sure we don't introduce extra DB
queries related to collections without being aware of it.
2023-08-24 13:08:36 +00:00
Naz
46f8c15448 Fixed flaky test behavior for bulk actions
refs https://ghost.slack.com/archives/C02G9E68C/p1692816097875899

- With introduction of extra e2e test coverage for Collections some tests started to fail at random. The root issue here was the transaction processing collections was started before the original bulk action (bulk edit, bulk publish/unpublish, etc.) was fully committed. The full transaction commit happens with the bulkAction method return inside of `if (!options.transacting) {` block.
2023-08-24 16:06:41 +08:00
Naz
437a080580 Added test coverage for collection update on tag removal
refs https://github.com/TryGhost/Arch/issues/77

- We were missing e2e test coverage for when the tag used in collection filters was removed. This changeset improves the situation.
2023-08-23 20:44:08 +08:00
Naz
acccc16614 Added collections update after bulk adding tags
refs https://github.com/TryGhost/Arch/issues/77

- During initial development we have missed to support collections update when tags are added to posts in bulk. It's especially valid usecase since we can define automatic collection with a filter containing not yet existing tags.
2023-08-23 17:12:08 +08:00
Fabien "egg" O'Carroll
62d5ca558d Validated filters for collections
refs https://github.com/TryGhost/Arch/issues/47

This ensures that we only have collections which have a valid filter in terms of
  - Valid NQL string
  - Uses only properties which are valid to filter on
  - Only has an empty filter in the case of the "latest" collection
2023-08-23 15:49:06 +07:00
Ghost CI
cebe8d7db6 Merged v5.59.4 into main 2023-08-22 11:46:13 +00:00
Simon Backx
a011151e24
🐛 Fixed handling multiple Stripe subscriptions for same member (#17773)
fixes https://github.com/TryGhost/Product/issues/3752

- Added some extra tests for edge cases
- Updated handling of multiple subscriptions so they are handled better
- Canceling a subscription when the member still has other subscriptions will now get handled correctly where the status and products of the member stay intact
2023-08-22 13:27:21 +02:00
Naz
df3aaed3c8 Added test coverage for collection updates
refs https://github.com/TryGhost/Arch/issues/77

- The test covers collection updates on post resource add/edit/delete operations
2023-08-22 17:16:35 +08:00
Naz
e638d72e70 Refactored collections e2e Admin API test suite
refs https://github.com/TryGhost/Arch/issues/71

- With describe/it block levels mixed on the top level of the test suite the order fo test execution was scattered around. Having "describe" groups for each of the BREAD methods makes things more organized and readable.
2023-08-22 17:16:35 +08:00
Simon Backx
b587429008
🐛 Fixed importing offers when importing members from Stripe (#17739)
fixes https://github.com/TryGhost/Product/issues/3728

- When importing members from Stripe with an existing offer, that didn't
exist in Ghost, the offer never got linked with the imported
subscription because of a missing return statement.
- Fixes importing offers with duplicate names
- Added E2E tests for creating members from a Stripe Customer ID
2023-08-16 13:25:57 +00:00
Naz
49a4a60d82 Fixed posts Content API test run in isolation
closes https://github.com/TryGhost/Arch/issues/76

- The posts test suite was failing when run in isolation. This was due to "collections" labs flag not being turned on, the events were not going through to collections service correctly
2023-08-16 16:07:01 +08:00
Simon Backx
e14df6479b
Added E2E tests for donations API (#17694)
fixes https://github.com/TryGhost/Product/issues/3722
2023-08-11 14:25:53 +02:00
Steve Larson
8273671425
Updated bookmark card renderer (#17662)
refs https://github.com/TryGhost/Product/issues/3609
-reverted css changes
-reverted mobiledoc renderer
-made lexical renderer consistent with mobiledoc
-css classes were inverted for theme backwards compatibility
2023-08-10 13:41:59 +00:00
Chris Raible
9ea4fbd7a7
Added feature to convert and open mobiledoc posts in the lexical editor (#17453)
refs TryGhost/Product#3638

- Added `convert_to_lexical` flag to the posts/pages edit endpoint
- Added 'convertToLexical' feature flag so we can enable/disable this
feature independently from the main lexical beta flag
- Modified admin posts/pages list to point to the lexical editor for
_all_ posts, regardless of mobiledoc vs lexical (if the flag is on)
- Added call to edit endpoint with `convert_to_lexical` in the lexical
editor admin route if the page/post is currently in mobiledoc and the
flag is enabled
2023-08-08 15:44:54 -07:00
Simon Backx
4c8179312d
🎨 Added support for relative links in emails (#17630)
fixes https://github.com/TryGhost/Product/issues/3687

After this change, relative URLs in emails will be replaced with
absolute URLs using the post URL. Making relative Portal URLs possible
etc.

Updates the test data generator to fix invalid URL encoding (somehow a
backslash + escaped double quote was added when it wasn't required).
2023-08-08 13:22:56 +02:00
Simon Backx
66675a1383
Added donation notifications column (#17607)
fixes https://github.com/TryGhost/Product/issues/3691

When a site receives a donation, we'll send an email to all
administrator. They can disable this in their profile settings. This
migration adds the required column to make this possible. It defaults to
true.

Link to tech spec:
https://www.notion.so/ghost/Tech-Spec-5cd6929f7960462ebcbf198176e0d899?pvs=4#156834b0231d4968a203d797c6817d21
2023-08-07 15:31:36 +02:00
Chris Raible
a4c04da443
Updated Posts Content API snapshots (#17600)
no issue

- Snapshot tests were incorrect but we weren't catching it because CI
was retrying them and yielding false passes
- This fix just fixes the broken tests, which will allow us to fix the
issue with CI yielding false passing results
2023-08-04 22:31:54 +00:00
Sag
81c3555106
Added logic for currency and suggested amount for Tips & Donations (#17599)
closes https://github.com/TryGhost/Product/issues/3666

- added computed setting "donations_enabled"
- added logic to persist "donations_suggested_amount" and "donations_currency"
- used  "donations_suggested_amount" and "donations_currency" when initiating a new Stripe Checkout for donations
- added copy functionality to "your link" in Tips & Donations settings
2023-08-04 18:17:35 +00:00
Sag
19bdb0efef
Added migrations for Tips & Donations' settings (#17576)
closes https://github.com/TryGhost/Product/issues/3668

- Tips and Donations feature offers two settings: "donations_currency", and "donations_suggested_amount"
    - "donation_currency": the currency to be used for the donation. Defaults to "USD", not nullable.
    - "donation_suggested_amount": an anchor price for the donation. Defaults to 0, not nullable.
- Both settings belong to a new group "donations"

Tech Spec: https://www.notion.so/ghost/Tech-Spec-5cd6929f7960462ebcbf198176e0d899?pvs=4#6e8b34c45f0c4c78b48c9e7725a307c8
2023-08-03 22:13:47 +02:00
Naz
7464dbc1af Added e2e test coverage for bulk collection edits
refs https://github.com/TryGhost/Arch/issues/16

- Makes sure post bulk actions also update collection posts
2023-08-01 15:11:00 +08:00
Naz
8635f4efeb Added collections handling for PostsBulkFeaturedEvent/PostsBulkUnfeaturedEvent
refs https://github.com/TryGhost/Arch/issues/16

- When posts produce PostsBulkFeaturedEvent/PostsBulkUnfeaturedEvent the collections having a featured filter should update the posts belonging to them.
2023-08-01 15:11:00 +08:00
Ronald Langeveld
0029c444ad
Added test email rate limiting (#17505)
refs https://github.com/TryGhost/Product/issues/3651

- This is a security fix that addresses an issue causing malicious users
to abuse the test / preview email API endpoint.
- We have multiple procedures in place now to limit such users.
- First, we now only allow one email address to be passed into the
`sendTestEmail` method. This method only have one purpose, which is to
compliment the test email functionality within the Editor in Admin and
therefore have no reason to send to more than one email address at a
time.
- We then add an additional rate limiter to prevent a user from making
multiple requests, eg via a script.
- The new imposed limit is 10 test emails per hour.
2023-07-27 08:46:50 +02:00
Naz
48ccea818a Fixed unpublished collection posts filtering
closes https://github.com/TryGhost/Arch/issues/58

- Following assumptions were broken:
- Posts Admin API should include posts of all statuses when filtering by collection
-  Posts Content API should not include any unpublished posts
- Updated the "status" filter which fixes the problem. We still disallow any custom filters to be applied on top of collections filter.
2023-07-26 16:45:57 +08:00
Fabien "egg" O'Carroll
3d0825ea2f Supported ?include=count.posts for Collections API
Unfortuantely our framework is bookshelf centric so we have to refer to the
`withRelated` property rather than a more generic `include` property.

The collection entity already contains the list of post ids, so we can just
return the length of that array.
2023-07-25 16:56:22 +02:00