Commit Graph

2980 Commits

Author SHA1 Message Date
Daniel Lockyer
7950122ffe Protected against deleting non-existent image during upload
fix https://linear.app/tryghost/issue/SLO-93/undefined-path-error-with-bad-image-upload

- in the event we receive a request to upload an image, that doesn't
  contain an image, we still try and unlink the files
- this is a dangling promise, so it doesn't cause an explicit HTTP
  error, but it does show up as a console error
- fixed it by checking for the path, and early returning if it doesn't
  exist
- also added a test that would fail without this
2024-05-02 17:29:03 +02:00
Sag
e996213122
Moved POST /members/api/member behind alpha flag (#20124)
ref https://linear.app/tryghost/issue/SLO-78

- the `POST /members/api/member` endpoint is solely used by the alpha
feature `membersSpamPrevention` and should not be available otherwise
2024-05-02 16:34:32 +02:00
Daniel Lockyer
cc76fda3e8 Enabled includeLocalVariables option in Sentry
ref https://linear.app/tryghost/issue/SLO-92/enable-extra-sentry-integrations
refs https://docs.sentry.io/platforms/javascript/guides/node/configuration/options/#include-local-variables

- this will capture the local stack variables when producing a stack
  trace, which would be super useful when debugging
2024-05-02 13:52:40 +02:00
Daniel Lockyer
5b28dc9246 Enabled Sentry's ExtraErrorData integration
refs https://docs.sentry.io/platforms/javascript/guides/node/configuration/integrations/extraerrordata/
ref https://linear.app/tryghost/issue/SLO-92/enable-extra-sentry-integrations

- this enables the ExtraErrorData integration, which should help us
  capture more of the properties of the errors we're producing, which
  _may_ help with debugging
2024-05-02 13:52:40 +02:00
renovate[bot]
a33dccf8cd Update TryGhost packages 2024-05-01 17:01:41 +02:00
Daniel Lockyer
3f7a7fff44 Fixed HTTP 500 when adding unknown products to member
fix https://linear.app/tryghost/issue/SLO-89/cannot-read-properties-of-null-reading-get-an-unexpected-error

- if we pass an invalid ID when updating the products on a member, we
  throw a HTTP 500 error because `product` is `null`
- we can check for this and return a BadRequestError, because the user
  supplied an incorrect ID
2024-05-01 16:54:35 +02:00
Ghost CI
6b32548e5c Merged v5.82.5 into main 2024-05-01 13:22:28 +00:00
Ghost CI
a4c478958b v5.82.5 2024-05-01 13:22:26 +00:00
renovate[bot]
746df63307
Update dependency gscan to v4.43.1 (#20109)
ref https://linear.app/tryghost/issue/ENG-842/gluster-file-name-length-limit

- gscan v4.43.1 contains a bug fix to throw an large filenames
2024-05-01 11:26:21 +00:00
Daniel Lockyer
ddac3a9e8b Removed Sentry error capturing for failed URL decoding
fix https://linear.app/tryghost/issue/SLO-79/incorrectusageerror-the-url-httpsblogkongregatecompercentc0-couldnt-be

- we added this Sentry captureException whilst fixing a bug where
  decodeUrl could fail, and throw a 500 exception
- we added handling for that case and returned an empty string, but we
  also added Sentry error capturing
- at this point, I don't think we need to be capturing errors in Sentry,
  because the issue is already handled, and it only usually happens with
  malicious/incorrect URLs
- this is our #2 cause of Sentry alerts, so it's good to clean it up
2024-05-01 10:15:13 +02:00
renovate[bot]
60a3b5a913 Update TryGhost packages 2024-05-01 08:49:04 +02:00
Daniel Lockyer
7109743282 Fixed overly verbose DatabaseInfo require
- we don't need to deep require into the library as it exports what we
  need on the surface
- this should unblock https://github.com/TryGhost/Ghost/pull/19002, as
  it's randomly failing with this require
2024-04-30 21:39:56 +02:00
Michael Barrett
4cd85ab8b7
Added timeout when resizing an image (#20087)
refs
[ENG-827](https://linear.app/tryghost/issue/ENG-827/🐛-crash-on-resizing-animated-gif)

Added a timeout to the image resizing middleware to prevent crashes when
an image is taking too long to resize. When the timeout is reached and
the image has not been resized, the middleware will return the original
image
2024-04-30 08:39:30 +01:00
Ronald Langeveld
b2970cb4e0
Added integrity test for flags (#20094)
ref
https://ghost.slack.com/archives/C02G9E68C/p1714047709694639?thread_ts=1713956576.497899&cid=C02G9E68C
    
    - Ensures unique feature flags, avoiding configuration conflicts.
    - Enhances code reliability and simplifies feature tracking.
    - Prevents bad rebases was the reason for the initial duplication.
2024-04-29 02:39:15 +00:00
renovate[bot]
305029bc38 Update dependency ws to v8.17.0 2024-04-29 01:23:51 +00:00
renovate[bot]
f18f08928f Update dependency yjs to v13.6.15 2024-04-29 01:22:59 +00:00
Ghost CI
cf9ba60524 v5.82.4 2024-04-26 16:04:30 +00:00
Chris Raible
dcd65bfa4f
Added caching to the LinkRedirectRepository (#20036)
ref
https://linear.app/tryghost/issue/ENG-851/implement-a-minimal-but-complete-version-of-redirect-caching-to
ref https://app.incident.io/ghost/incidents/55

Often immediately after sending an email, sites receive a large volume
of requests to LinkRedirect endpoints from members clicking on the links in
the email.

We currently don't cache any of these requests in our CDN, because we
also record click events, update the member's `last_seen_at` timestamp,
and send webhooks in response to these clicks, so Ghost needs to handle
each of these requests itself. This means that each of these LinkRedirect requests
hits Ghost, and currently all these requests hit the database to lookup
where to redirect the member to.

Each one of these requests can make up to 11 database queries, which can
quickly exhaust Ghost's database connection pool. Even though the
LinkRedirect lookup query is fairly cheap and quick, these queries aren't
prioritized over the "record" queries Ghost needs to handle, so they can
get stuck behind other queries in the queue and eventually timeout.

The result is that members are unable to actually reach the destination
of the link they clicked on, instead receiving a 500 error in Ghost, or
it can take a long time (60s+) for the redirect to happen.

This PR uses our existing `adapterManager` to cache the redirect lookups
either in-memory or in Redis (if configured — by default there is no caching). This only removes 1 out of
11 queries per redirect request, so it won't reduce the load on the DB
drastically, but it at least decouples the serving of the LinkRedirect from
the DB so the member can be redirected even if the DB is under heavy
load.

Local load testing results have shown a decrease in response times from
60 seconds to ~50ms for the redirect requests when handling 500 requests
per second, and reduced the 500 error rate to 0.
2024-04-25 19:17:25 -07:00
Daniel Lockyer
892b9ab397 Moved internalLinking and stripeAutomaticTax flags to private beta
refs https://ghost.slack.com/archives/C02G9E68C/p1714051665654659?thread_ts=1713970812.191919&cid=C02G9E68C

- this enables us to enable the flags on sites without the need to
  enable developer experiments
- added `(private beta)` to the end of the relevant UI flags, because
  they're still in alpha in the UI despite being in beta in backend
2024-04-25 16:37:55 +02:00
Ronald Langeveld
601d05de0d
Removed duplicated ActivityPub labs key (#20089)
no issue

- rebased from main without seeing there's an existing flag.
- this removes the duplicated flag
2024-04-25 14:15:47 +00:00
Steve Larson
a0b7476794
Updated staff deletion logic (#20069)
ref https://linear.app/tryghost/issue/ENG-826

- Changed staff deletion logic to do a bulk insert when adding a tag to
the users' associated posts

Staff deletion logic has really poor performance at scale because we do
individual updates for every post. If a user has dozens+ posts
(especially in a large db with thousands of posts), this can take >60s
and look like a timeout. Ultimately this should probably be a jobbed off
process, but for the time being we can improve this by doing a bulk
insert.

Note that this update uses the pattern for the bulk tagging of posts
from the right click (bulk) actions in the posts lists in Admin. With
bulk actions, **we do not trigger web hooks or the post.edited events**.
We will document this and follow up on this separately.
2024-04-25 08:19:11 -05:00
Djordje Vlaisavljevic
7a3bbfde10
Added ActivityPub playground (#20081)
ref MOM61

- Adds admin-x react app we’ll use as ActivityPub playground to the
sidebar nav behind the feature flag.
- Wired up routing to Ember
- Setup the project as `admin-x-activitypub`

---------

Co-authored-by: Ronald Langeveld <hi@ronaldlangeveld.com>
2024-04-25 16:44:29 +08:00
Fabien O'Carroll
af02ca7044 Initial wire up of Posts -> Outbox flow
ref https://linear.app/tryghost/issue/MOM-29

This is very rough, and all still behind a flag. The idea is that any public
post which is published gets added to the Outbox of the site Actor. We also
dispatch an event, which will be used to deliver the Activity to any relevant
inboxes, but that is outside the scope of this commit.
2024-04-25 11:10:17 +07:00
renovate[bot]
25657b6a99 Update sentry-javascript monorepo to v7.112.1 2024-04-23 16:39:50 +02:00
renovate[bot]
3960e8caa1 Update sentry-javascript monorepo to v7.112.0 2024-04-23 14:11:30 +02:00
Daniel Lockyer
fda8aa5bfe Removed Sentry logging from XMLRPC service
refs https://ghost-foundation.sentry.io/issues/5135326925/

- the service tends to 503 all the time, and we don't really care enough
  for it to ping us in Sentry, as it's not something we control
- we can still keep logging the errors in case we need to go and look at
  what went wrong
2024-04-23 12:48:33 +02:00
renovate[bot]
d69a0aa1e6 Update dependency knex-migrator to v5.2.1 2024-04-22 12:29:48 +02:00
renovate[bot]
e4ffc7b8c0 Update dependency mysql2 to v3.9.7 2024-04-22 01:37:24 +00:00
renovate[bot]
69372d9018 Update dependency @sentry/profiling-node to v7.111.0 2024-04-19 16:56:32 +00:00
renovate[bot]
7fea4151d8 Update dependency @sentry/profiling-node to v7 2024-04-19 18:43:59 +02:00
Ghost CI
cb838c4bb6 v5.82.3 2024-04-19 16:04:16 +00:00
renovate[bot]
80fe672525 Update sentry-javascript monorepo to v7.111.0 2024-04-19 08:54:26 +02:00
Daniel Lockyer
f31814c804 Updated mysql2 dependency
refs #20050

- Renovate seems to be unable to bump the package past the security
  release, but unfortunately this release contains a breaking bug
- this commit manually bumps the package so we can get things flowing
  again
- the security release doesn't really affect us, but we should still try
  and keep on the latest
2024-04-19 08:48:24 +02:00
Chris Raible
a10b13916a
🐛 Fixed admin error when deleting an unsaved or imported post (#20053)
ref
https://linear.app/tryghost/issue/ENG-845/error-attempted-to-set-lexical-on-the-deleted-record
ref
[https://linear.app/tryghost/issue/ENG-854/🐛-deleting-imported-posts-makes-ghost-unresponsive](https://linear.app/tryghost/issue/ENG-854/%F0%9F%90%9B-deleting-imported-posts-makes-ghost-unresponsive)

- When deleting a post in the editor's Post Settings Menu, if the post
has unsaved changes (indicated by the hasDirtyAttributes property in the
editor), Admin will crash because it tries to save a post revision
before leaving the editor, but the post has already been deleted so
saving fails.
- This can occur when editing a post and quickly deleting it from the
Post Settings Menu before saving is completed.
- It can also occur when attempting to delete an imported post, as the
editor will parse the lexical from the server and may make some minor,
invisible-to-the-user changes to the lexical string locally (e.g. JSON
formatting, or updating the JSON to use extended version of base lexical
nodes), which triggers the same error.
- This fix bypasses the attempt to save a post revision when leaving the
editor if the post is already deleted, which allows the transition back
to the Posts route to succeed.
2024-04-18 10:02:02 -07:00
Kevin Ansfield
182fa62759
Bumped Koenig packages (#20047)
closes https://linear.app/tryghost/issue/MOM-39/

- fixes clicking on search results in internal linking PoC
- adds support for Node 20
2024-04-18 14:58:01 +00:00
Daniel Lockyer
10e81aeed8
ℹ️ Added support for Node 20
ref https://linear.app/tryghost/issue/ENG-765/add-support-for-node-20

- this adds support for Node 20 to Ghost and CI, as Node 20 is an LTS
  version and we should pick it up
2024-04-18 13:17:21 +02:00
Daniel Lockyer
8e0ad1a6fb
Fixed test on Node 20
refs f39d1d3aa3

- similar to the commit above, the JSON parser changed between Node 18
  and Node 20, so the error message changed too
- we actually just want to check the error is forwarded to the user, so
  we can do that by getting the error message from JSON.parse and check
  against that
2024-04-18 13:17:16 +02:00
renovate[bot]
ef6f79c272 Update dependency gscan to v4.43.0 2024-04-18 12:48:42 +02:00
Fabien O'Carroll
d34884fc6d Moved ActivityPub API to frontend URL
ref https://linear.app/tryghost/issue/MOM-48

This required some structural changes to our NestJS setup so that we can mount
it on multiple parts of the Ghost express app.

We've used the RouterModule to allow adding submodules that are mounted on
different paths, and we've had to be explicit about the base path for each
module. We've also had to switch back to using the Module decorator, because
RouterModule doesn't work with DynamicModule definitions.

Now that the NestJS app has knowledge of the full path, we need to "reset" the
url & baseUrl when passing the request into NestJS so that it can correctly
match the path. This is probably needed for the frontend too, for subdirs, but
that causes further issues - as this in prototype stage, we'll look later

Another issue is that NestJS replaces the express app instance with its own,
which isn't an issue for the Admin API (though we've fixed it anyway for
consistency), but did cause problems for the frontend, because the express app
is where view engine and directory information is stored.

The fix for this is to save a reference to the original ghost express
application, and reattach it to the request if it is not handled by Nest

Now that we have the Nest app mounted on the frontend, we're able to have it
handle the /.well-known/webfinger route with a proper controller, which is nice!
2024-04-18 13:26:24 +07:00
Fabien O'Carroll
885dc537d5 Added initial support for /.well-known/webfinger
ref https://linear.app/tryghost/issue/MOM-26
ref https://linear.app/tryghost/issue/MOM-27
ref https://webfinger.net/spec/

WebFinger is the protocol which allows different ActivityPub implementers to
find information about Actors, it's kinda like the entrypoint.

Given a username like @user@site.com, we can look up the URL for the Actor at

   https://website.com/.well-known/webfinger?resource=acct:user@site.com

This would then give us the info needed to discover the Actor's Inbox & Outbox
2024-04-17 16:12:57 +07:00
Fabien O'Carroll
ffbf2680ef Enabled Nest when ActivityPub flag is set
ref MOM-31
ref https://linear.app/tryghost/issue/MOM-31

We'll be building a lot of the new code for ActivityPub in Nest, so we'll need
to have it enabled in Ghost to work.
2024-04-17 14:00:07 +07:00
Fabien O'Carroll
0fb290a841 Added Feature Flag for ActivityPub
ref MOM-31
ref https://linear.app/tryghost/issue/MOM-31

Ronseal - let's goooooooo
2024-04-17 14:00:07 +07:00
renovate[bot]
96d0883928
🐛 Fixed file card button not being linked in emails (#20023)
ref https://linear.app/tryghost/issue/DES-202/

- bumped Koenig packages to include fix for incorrectly wrapped download image link in email rendering of file card
2024-04-16 10:37:28 +00:00
renovate[bot]
2af9c04477 Update CSS preprocessors 2024-04-16 09:44:36 +02:00
renovate[bot]
0802b9533c Update dependency semver to v7.6.0 2024-04-16 09:41:01 +02:00
Nicholas Mizoguchi
d6b7ebb517
Enforced more Mocha lint rules (#19720)
ref https://github.com/TryGhost/Ghost/issues/11038

1. Enforced lint rule
**[ghost/mocha/no-identical-title](https://github.com/lo1tuma/eslint-plugin-mocha/blob/main/docs/rules/no-identical-title.md)**
- Fixed relevant tests
2. Enforced lint rule
**[ghost/mocha/max-top-level-suites](https://github.com/lo1tuma/eslint-plugin-mocha/blob/main/docs/rules/max-top-level-suites.md)**
- No required fixes, as tests are compliant already

#### Additional details
Specifically for `ghost/mocha/no-identical-title` most fixes were simple
test description updates. Added comments to aid the PR review for the
ones that had relevant changes, and might require more attention. They
are as follows:
*
[e2e-api/admin/invites.test.js](https://github.com/TryGhost/Ghost/pull/19720#discussion_r1496397548):
Removed duplicated test (exact same code on both);
*
[e2e-api/admin/members.test.js](https://github.com/TryGhost/Ghost/pull/19720#discussion_r1496399107):
From the[ PR this was
introduced](73466c1c40 (diff-4dbc7e96e356428561085147e00e9acb5c71b58d4c1bd3d9fc9ac30e77c45be0L236-L237))
seems like author based his test on an existing one but possibly forgot
to rename it;
*
[unit/api/canary/utils/serializers/input/pages.test.js](https://github.com/TryGhost/Ghost/pull/19720#discussion_r1496400143):
The [page filter](https://github.com/TryGhost/Ghost/pull/14829/files)
was removed, so changed the description accordingly;
*
[unit/api/canary/utils/serializers/input/posts.test.js](https://github.com/TryGhost/Ghost/pull/19720#discussion_r1496400329):
The [page filter](https://github.com/TryGhost/Ghost/pull/14829/files)
was removed, so changed the description accordingly;
*
[unit/frontend/services/rendering/templates.test.js](https://github.com/TryGhost/Ghost/pull/19720#discussion_r1496402430):
Removed duplicated test
*
[unit/server/models/post.test.js](https://github.com/TryGhost/Ghost/pull/19720#discussion_r1496403529):
the change in [this
PR](https://github.com/TryGhost/Ghost/pull/14586/files#diff-c351cb589adefbb886570cfadb33b33eb8fdc12bde1024d1188cd18c165fc5e8L1010)
made three tests here mostly the same. Deduplicated them and kept only
one.
2024-04-16 09:37:06 +02:00
renovate[bot]
39a9a9bfd8 Update sentry-javascript monorepo to v7.110.1 2024-04-16 09:09:22 +02:00
renovate[bot]
b008c00feb Update sentry-javascript monorepo to v7.110.0 2024-04-15 09:47:44 +02:00
Ghost CI
ff03b2eedb v5.82.2 2024-04-15 01:06:07 +00:00
Ghost CI
bc0496e5d3 🎨 Updated Source to v1.2.2 2024-04-15 01:06:06 +00:00