Commit Graph

38358 Commits

Author SHA1 Message Date
Ghost CI
cf9ba60524 v5.82.4 2024-04-26 16:04:30 +00:00
Chris Raible
dcd65bfa4f
Added caching to the LinkRedirectRepository (#20036)
ref
https://linear.app/tryghost/issue/ENG-851/implement-a-minimal-but-complete-version-of-redirect-caching-to
ref https://app.incident.io/ghost/incidents/55

Often immediately after sending an email, sites receive a large volume
of requests to LinkRedirect endpoints from members clicking on the links in
the email.

We currently don't cache any of these requests in our CDN, because we
also record click events, update the member's `last_seen_at` timestamp,
and send webhooks in response to these clicks, so Ghost needs to handle
each of these requests itself. This means that each of these LinkRedirect requests
hits Ghost, and currently all these requests hit the database to lookup
where to redirect the member to.

Each one of these requests can make up to 11 database queries, which can
quickly exhaust Ghost's database connection pool. Even though the
LinkRedirect lookup query is fairly cheap and quick, these queries aren't
prioritized over the "record" queries Ghost needs to handle, so they can
get stuck behind other queries in the queue and eventually timeout.

The result is that members are unable to actually reach the destination
of the link they clicked on, instead receiving a 500 error in Ghost, or
it can take a long time (60s+) for the redirect to happen.

This PR uses our existing `adapterManager` to cache the redirect lookups
either in-memory or in Redis (if configured — by default there is no caching). This only removes 1 out of
11 queries per redirect request, so it won't reduce the load on the DB
drastically, but it at least decouples the serving of the LinkRedirect from
the DB so the member can be redirected even if the DB is under heavy
load.

Local load testing results have shown a decrease in response times from
60 seconds to ~50ms for the redirect requests when handling 500 requests
per second, and reduced the 500 error rate to 0.
2024-04-25 19:17:25 -07:00
Daniel Lockyer
892b9ab397 Moved internalLinking and stripeAutomaticTax flags to private beta
refs https://ghost.slack.com/archives/C02G9E68C/p1714051665654659?thread_ts=1713970812.191919&cid=C02G9E68C

- this enables us to enable the flags on sites without the need to
  enable developer experiments
- added `(private beta)` to the end of the relevant UI flags, because
  they're still in alpha in the UI despite being in beta in backend
2024-04-25 16:37:55 +02:00
Ronald Langeveld
601d05de0d
Removed duplicated ActivityPub labs key (#20089)
no issue

- rebased from main without seeing there's an existing flag.
- this removes the duplicated flag
2024-04-25 14:15:47 +00:00
Steve Larson
a0b7476794
Updated staff deletion logic (#20069)
ref https://linear.app/tryghost/issue/ENG-826

- Changed staff deletion logic to do a bulk insert when adding a tag to
the users' associated posts

Staff deletion logic has really poor performance at scale because we do
individual updates for every post. If a user has dozens+ posts
(especially in a large db with thousands of posts), this can take >60s
and look like a timeout. Ultimately this should probably be a jobbed off
process, but for the time being we can improve this by doing a bulk
insert.

Note that this update uses the pattern for the bulk tagging of posts
from the right click (bulk) actions in the posts lists in Admin. With
bulk actions, **we do not trigger web hooks or the post.edited events**.
We will document this and follow up on this separately.
2024-04-25 08:19:11 -05:00
Djordje Vlaisavljevic
7a3bbfde10
Added ActivityPub playground (#20081)
ref MOM61

- Adds admin-x react app we’ll use as ActivityPub playground to the
sidebar nav behind the feature flag.
- Wired up routing to Ember
- Setup the project as `admin-x-activitypub`

---------

Co-authored-by: Ronald Langeveld <hi@ronaldlangeveld.com>
2024-04-25 16:44:29 +08:00
Fabien O'Carroll
af02ca7044 Initial wire up of Posts -> Outbox flow
ref https://linear.app/tryghost/issue/MOM-29

This is very rough, and all still behind a flag. The idea is that any public
post which is published gets added to the Outbox of the site Actor. We also
dispatch an event, which will be used to deliver the Activity to any relevant
inboxes, but that is outside the scope of this commit.
2024-04-25 11:10:17 +07:00
Fabien O'Carroll
e01c9cb546 Moved frontend controllers into dedicated directory
no-issue

These should never have been in the admin directory!
2024-04-25 11:10:17 +07:00
Fabien O'Carroll
299f7c408e Added very basic Outbox for Actors
ref https://linear.app/tryghost/issue/MOM-28
ref https://linear.app/tryghost/issue/MOM-29
ref https://linear.app/tryghost/issue/MOM-30

Basic wire up of Create Activities, Articles for Posts & Actor's Outbox!
I'd definitely like to rethink the whole storage layer and how we split things
out - I think separating the Outbox from the Actor would make sense, otherwise
the size of thsi is gonna grow, or we're gonna have to deal with sub-pagination.
2024-04-25 11:10:17 +07:00
Fabien O'Carroll
d592b1e9c9 Expanded ActivityPub type definitions
ref https://linear.app/tryghost/issue/MOM-25

All these intersection types are getting a bit out of hand - but we can clean
up all of this once we're past prototyping phase.
2024-04-25 11:10:17 +07:00
Djordje Vlaisavljevic
7f915215f3 Made subscription link in newsletter preview use brand color
closes https://linear.app/tryghost/issue/DES-220/email-preview-manage-subscription-link-in-sub-details-is-hardcoded-to
2024-04-24 16:42:32 +01:00
Djordje Vlaisavljevic
4b4af65fce Fixed spacing bug
closes https://linear.app/tryghost/issue/DES-258/not-receiving-emails-link-is-offset-on-account-page
2024-04-24 13:13:21 +01:00
Peter Zimon
9f19e334c1
Improve technical errors (#20046)
ref DES-229

Some of the error messages in Ghost and more specifically in Settings
were very technical, e.g.
`ValidationError: Validation (isEmpty) failed for locale`

This PR deals with some of the occurances for a more human error
communication.
2024-04-24 08:42:22 +02:00
renovate[bot]
571171bad5 Update storybook monorepo to v7.6.18 2024-04-23 17:10:07 +02:00
renovate[bot]
25657b6a99 Update sentry-javascript monorepo to v7.112.1 2024-04-23 16:39:50 +02:00
renovate[bot]
627e8b66bb
Fixed BookmarkNode not auto selecting in Editor
refs 8166363359

- updated dependency @tryghost/koenig-lexical to v1.1.6
2024-04-23 14:36:00 +00:00
renovate[bot]
3960e8caa1 Update sentry-javascript monorepo to v7.112.0 2024-04-23 14:11:30 +02:00
Daniel Lockyer
fda8aa5bfe Removed Sentry logging from XMLRPC service
refs https://ghost-foundation.sentry.io/issues/5135326925/

- the service tends to 503 all the time, and we don't really care enough
  for it to ping us in Sentry, as it's not something we control
- we can still keep logging the errors in case we need to go and look at
  what went wrong
2024-04-23 12:48:33 +02:00
renovate[bot]
197d510c9c Update dependency tailwindcss to v3.4.3 2024-04-23 11:27:43 +02:00
renovate[bot]
3fc9262204 Update dependency clsx to v2.1.1 2024-04-23 07:09:19 +00:00
Fabien O'Carroll
2deee35673 Used more specific Content-Type for ActivityPub
no-issue
2024-04-23 11:53:27 +07:00
Fabien O'Carroll
da2ba37f63 Made the default ID a little more fun
no-issue

A bunch of zeroes is boring :(
2024-04-23 11:53:27 +07:00
Fabien O'Carroll
dfeb965878 Added extra properties to Actors JSON-LD
ref https://linear.app/tryghost/issue/MOM-25
ref https://docs.joinmastodon.org/spec/activitypub/#properties-used-1

This adds a bunch of extra properties, most of which are placeholders. They're
all taken from the mastodon docs for properties used on ActivityPub Actors
2024-04-23 11:53:27 +07:00
Chris Raible
d8672cb1ff
Fixed length of from field in the RedirectsImporter (#20034)
no issue

- The `RedirectsImporter` used by the data generator was creating
redirects with the wrong length for the `from` field, which didn't match
the actual behavior of Ghost.
- This commit corrects the length from 32 to 8, which is the actual
length of the `from` field in production.
- This change has no impact on Ghost's behavior, but makes the data
generator more representative of real world data for more accurate
testing.
2024-04-22 18:11:14 -07:00
Sanne de Vries
a2bbe6b3c3
Updated 'Users' to 'Staff' in search dropdown component (#20064)
No ref

- This copy change allows us to use consistent copy in both the general
search as well as the internal linking search component in the editor.
2024-04-22 14:49:43 +00:00
Daniël van der Winden
3771b2fca4
Fixed email footer text styling inconsistencies (#20063)
fixes
https://linear.app/tryghost/issue/DES-260/footer-link-text-smaller-than-regular-text

There was a bit of CSS in a media query aimed at other parts of the
newsletter template that was causing the footer styling to break. I
added some more specific styling for the footer as well, to make sure
span's within the `<p>` element are covered as well.
2024-04-22 15:43:19 +02:00
renovate[bot]
33c5ce057c Update dependency terser to v5.30.4 2024-04-22 12:42:44 +00:00
renovate[bot]
d69a0aa1e6 Update dependency knex-migrator to v5.2.1 2024-04-22 12:29:48 +02:00
Kevin Ansfield
8dac340492
Updated editor link search to group results (#20058)
closes https://linear.app/tryghost/issue/MOM-49

- bumped koenig-lexical so the bookmark card has group support for testing
- updated `searchLinks` function passed to Koenig to match expected grouped results shape
2024-04-22 05:35:18 +00:00
Fabien O'Carroll
99bfa30f81 Added Content-Type Header to actor & outbox endpoints
ref https://linear.app/tryghost/issue/MOM-25

This is the correct Content-Type and whilst not required, better in than out
2024-04-22 11:18:03 +07:00
Fabien O'Carroll
8542766094 Updated Actor key id to use Actor URL with fragment
ref https://linear.app/tryghost/issue/MOM-25

This matches the way that mastodon handles the key url and may be the reason
these documents are incompatible. This also removes the `username` key as that
isn't used anywhere, instead we have a username property which is rendered as
the ActivityPub compat preferredUsername key.
2024-04-22 11:18:03 +07:00
renovate[bot]
e4ffc7b8c0 Update dependency mysql2 to v3.9.7 2024-04-22 01:37:24 +00:00
renovate[bot]
69372d9018 Update dependency @sentry/profiling-node to v7.111.0 2024-04-19 16:56:32 +00:00
renovate[bot]
7fea4151d8 Update dependency @sentry/profiling-node to v7 2024-04-19 18:43:59 +02:00
Ghost CI
cb838c4bb6 v5.82.3 2024-04-19 16:04:16 +00:00
renovate[bot]
b24ad1ccc6 Update nest monorepo to v10.3.8 2024-04-19 15:05:38 +02:00
renovate[bot]
80fe672525 Update sentry-javascript monorepo to v7.111.0 2024-04-19 08:54:26 +02:00
Daniel Lockyer
f31814c804 Updated mysql2 dependency
refs #20050

- Renovate seems to be unable to bump the package past the security
  release, but unfortunately this release contains a breaking bug
- this commit manually bumps the package so we can get things flowing
  again
- the security release doesn't really affect us, but we should still try
  and keep on the latest
2024-04-19 08:48:24 +02:00
Kevin Ansfield
7132619115
Fixed internal linking not correctly filtering to published-only (#20054)
no issue

- updated search to add `status` to the search results
- added filtering to the editor's `searchLinks()` method
- prevented TaskCancellation errors being thrown from the search task being cast to a Promise
2024-04-18 17:18:37 +00:00
Chris Raible
a10b13916a
🐛 Fixed admin error when deleting an unsaved or imported post (#20053)
ref
https://linear.app/tryghost/issue/ENG-845/error-attempted-to-set-lexical-on-the-deleted-record
ref
[https://linear.app/tryghost/issue/ENG-854/🐛-deleting-imported-posts-makes-ghost-unresponsive](https://linear.app/tryghost/issue/ENG-854/%F0%9F%90%9B-deleting-imported-posts-makes-ghost-unresponsive)

- When deleting a post in the editor's Post Settings Menu, if the post
has unsaved changes (indicated by the hasDirtyAttributes property in the
editor), Admin will crash because it tries to save a post revision
before leaving the editor, but the post has already been deleted so
saving fails.
- This can occur when editing a post and quickly deleting it from the
Post Settings Menu before saving is completed.
- It can also occur when attempting to delete an imported post, as the
editor will parse the lexical from the server and may make some minor,
invisible-to-the-user changes to the lexical string locally (e.g. JSON
formatting, or updating the JSON to use extended version of base lexical
nodes), which triggers the same error.
- This fix bypasses the attempt to save a post revision when leaving the
editor if the post is already deleted, which allows the transition back
to the Posts route to succeed.
2024-04-18 10:02:02 -07:00
Daniel Lockyer
7ab6a219b0 Updated @doist/react-interpolate to v1.0.0
refs Doist/react-interpolate@ff466d4
refs https://ghost.slack.com/archives/CFH10N79S/p1713450302767339?thread_ts=1713439122.675249&cid=CFH10N79S
refs https://linear.app/tryghost/issue/ENG-765/add-support-for-node-20

- this version has support for Node 20, which we need in order to add
  support for it
2024-04-18 17:00:54 +02:00
Kevin Ansfield
182fa62759
Bumped Koenig packages (#20047)
closes https://linear.app/tryghost/issue/MOM-39/

- fixes clicking on search results in internal linking PoC
- adds support for Node 20
2024-04-18 14:58:01 +00:00
Daniel Lockyer
10e81aeed8
ℹ️ Added support for Node 20
ref https://linear.app/tryghost/issue/ENG-765/add-support-for-node-20

- this adds support for Node 20 to Ghost and CI, as Node 20 is an LTS
  version and we should pick it up
2024-04-18 13:17:21 +02:00
Daniel Lockyer
8e0ad1a6fb
Fixed test on Node 20
refs f39d1d3aa3

- similar to the commit above, the JSON parser changed between Node 18
  and Node 20, so the error message changed too
- we actually just want to check the error is forwarded to the user, so
  we can do that by getting the error message from JSON.parse and check
  against that
2024-04-18 13:17:16 +02:00
Kevin Ansfield
96adb0a5d8
Wired up bookmark card internal linking proof of concept (#20044)
closes https://linear.app/tryghost/issue/MOM-1/

- added `feature.internalLinking` and `searchLinks` properties to the `cardConfig` object passed to the editor
- `searchLinks()` uses Admin's internal search to fetch and filter results
  - called with no search term to obtain default links to show as soon as the bookmark card is inserted, in our case we show the last 5 published posts. Result is cached for the duration of the editing session to avoid API queries/loading state after the first fetch
  - flattens search results for now because Koenig doesn't yet support grouped results
- bumps version of `@tryghost/koenig-lexical` to support the feature flag
2024-04-18 11:16:19 +00:00
renovate[bot]
ef6f79c272 Update dependency gscan to v4.43.0 2024-04-18 12:48:42 +02:00
Fabien O'Carroll
9e98be9268 Disabled caching for ActivityPub & WebFinger
ref https://linear.app/tryghost/issue/MOM-25

Whilst we're experimenting it's gonna be easier to not have to think about
caching affecting things. We'll disable it completely for now, and then decide
on a caching strategy that suits us down the line.
2024-04-18 17:40:44 +07:00
renovate[bot]
e8ea2e4db0 Update GitHub Artifact Actions to v4 2024-04-18 12:36:39 +02:00
renovate[bot]
4ab31122a4 Update actions/cache action to v4 2024-04-18 12:36:07 +02:00
Fabien O'Carroll
e346423db2 Added missing JSON-LD context for publicKey definition
ref https://linear.app/tryghost/issue/MOM-25

This _might_ be the reason that Mastodon doesn't recognise our Actor, but
either way it's the correct thing to do so that JSON-LD parsers correctly
understand that publicKey field
2024-04-18 16:48:09 +07:00