Commit Graph

3194 Commits

Author SHA1 Message Date
Ronald Langeveld
4f3bfebfea
Revert "Added latest_event_timestamp to email table (#20118)" (#20149)
This reverts commit 3246a8d2c9.
2024-05-07 08:58:42 +00:00
Daniel Lockyer
c298db912c Added JSDoc types for API controllers
- this adds a simple set of types to the @tryghost/api-framework
  package that should describe all of the keys available on a
  controller, and then rolls it out to all API controllers
- unfortunately, due to https://github.com/microsoft/TypeScript/issues/47107, we have
  to split apart `module.exports` into a variable assignment in order for type-checking
  to be done
- the main benefit of this is that `frame` is now typed, and editors understand what keys
  are available, so intellisense works properly
2024-05-07 10:49:44 +02:00
Ronald Langeveld
3246a8d2c9
Added latest_event_timestamp to email table (#20118)
ref ENG-832

- Added migrations for `latest_event_timestamp` column in emails table.
- updated schema
- updated emails model
2024-05-07 16:12:51 +08:00
Daniel Lockyer
de435238f4 Fixed wrong status code type
- `statusCode` should be a number, but we were passing a string
- this doesn't really affect anything, but tsserver was flagging it up
  as the wrong type
2024-05-07 08:29:48 +02:00
Daniel Lockyer
b29c897da3 Fixed passing error to IncorrectUsageError
- we should pass it as `err` and not `error`
- this probably slipped in because the catch parameter is called
  `error`, so I've updated that and fixed the references
2024-05-07 08:29:48 +02:00
Daniel Lockyer
efc59dd315 Fixed extra parameter to function
- `serializeTier` only takes 2 parameters
- flagged by tsserver
2024-05-07 08:29:48 +02:00
Daniel Lockyer
f72d7b77ac Fixed miscellaneous jsdoc comments
- this helps tsserver figure out what the type of things is around our
  codebase
- nothing crazy, mostly Express types for the middleware, application and router levels
2024-05-07 08:29:48 +02:00
Daniel Lockyer
265a8dd16f Added function names to more middleware
refs 319f251ad2

- this helps debugging because all middleware in the stack will have a
  function name, so it'll show up instead of `<anonymous>`
2024-05-06 17:51:39 +02:00
Daniel Lockyer
a50bb8d79e Added missing error messages for members uploader
fix https://linear.app/tryghost/issue/SLO-97/missing-messages-for-members-file-upload

- these were missing, so if the members importer wasn't given a file, it
  would crash with an HTTP 500 error
- also added a test to ensure we get a 422 back
2024-05-06 15:17:25 +02:00
Daniel Lockyer
5a8145139a Fixed handling cutoff boundary data in image + media upload
fix https://linear.app/tryghost/issue/SLO-95/unexpected-end-of-multipart-data-for-broken-image-upload-request

- in the event the client sends an invalid body to the image or media
  upload endpoints, Dicer will throw an error if the boundary data is
  malformed
- previously, we've just been bubbling that up as an InternalServerError
  and that results in an HTTP 500
- we can capture errors produced by dicer and return a handled
  BadRequestError, as it's the client's fault
- also includes breaking tests
2024-05-06 13:41:25 +02:00
Daniel Lockyer
3e79712466 Fixed handling malformed image + media upload requests
fix https://linear.app/tryghost/issue/SLO-94/unexpected-field-when-given-broken-image-upload-request

- in the event the body of an image or media upload request is malformed
  (broken metadata / blob or something), we get a MulterError and this
  bubbles up as an InternalServerError and spits out a HTTP 500
- we can capture this and return a BadRequestError, as it's the client's
  fault for not providing the correct body
- this implements that and adds breaking tests
2024-05-06 13:24:26 +02:00
Ghost CI
f34c33f330 v5.82.6 2024-05-03 16:04:14 +00:00
Chris Raible
d8b629c713
Added an optional timeout parameter to AdapterCacheRedis (#20131)
ref
https://linear.app/tryghost/issue/ENG-902/add-an-optional-timeout-in-the-redis-cache-adapter-in-case-redis

- Added an optional timeout parameter to AdapterCacheRedis, so that the
`get(key)` method will return `null` after the timeout if it hasn't
received a response from Redis
- When load testing the `LinkRedirectRepository` with the Redis cache
enabled on staging, we noticed that for some reason Redis stopped
responding to commands for ~30 seconds.
- The `LinkRedirectRepository` was waiting for the Redis cache to
respond and resulted in a drastic increase in response times for link
redirects
- This change will allow us to set a timeout on the `get(key)` method,
so that if Redis doesn't respond within the timeout, the method will
return `null` as if it were a cache miss.
- Then the `LinkRedirectRepository` will fall back to the database and
return the link redirect from the database instead of waiting
indefinitely for Redis to respond
2024-05-02 20:39:23 -07:00
Chris Raible
b9f7ea65e9
Revert "Added new member signup flow behind labs flag (#19986)" (#20130)
ref https://linear.app/tryghost/issue/KTLO-1/members-spam-signups

This reverts commit 01d0b2b304.

- Removed the new member signup flow because it didn't solve the
problems with spam signups
2024-05-02 13:02:32 -07:00
Daniel Lockyer
7950122ffe Protected against deleting non-existent image during upload
fix https://linear.app/tryghost/issue/SLO-93/undefined-path-error-with-bad-image-upload

- in the event we receive a request to upload an image, that doesn't
  contain an image, we still try and unlink the files
- this is a dangling promise, so it doesn't cause an explicit HTTP
  error, but it does show up as a console error
- fixed it by checking for the path, and early returning if it doesn't
  exist
- also added a test that would fail without this
2024-05-02 17:29:03 +02:00
Sag
e996213122
Moved POST /members/api/member behind alpha flag (#20124)
ref https://linear.app/tryghost/issue/SLO-78

- the `POST /members/api/member` endpoint is solely used by the alpha
feature `membersSpamPrevention` and should not be available otherwise
2024-05-02 16:34:32 +02:00
Daniel Lockyer
cc76fda3e8 Enabled includeLocalVariables option in Sentry
ref https://linear.app/tryghost/issue/SLO-92/enable-extra-sentry-integrations
refs https://docs.sentry.io/platforms/javascript/guides/node/configuration/options/#include-local-variables

- this will capture the local stack variables when producing a stack
  trace, which would be super useful when debugging
2024-05-02 13:52:40 +02:00
Daniel Lockyer
5b28dc9246 Enabled Sentry's ExtraErrorData integration
refs https://docs.sentry.io/platforms/javascript/guides/node/configuration/integrations/extraerrordata/
ref https://linear.app/tryghost/issue/SLO-92/enable-extra-sentry-integrations

- this enables the ExtraErrorData integration, which should help us
  capture more of the properties of the errors we're producing, which
  _may_ help with debugging
2024-05-02 13:52:40 +02:00
renovate[bot]
a33dccf8cd Update TryGhost packages 2024-05-01 17:01:41 +02:00
Daniel Lockyer
3f7a7fff44 Fixed HTTP 500 when adding unknown products to member
fix https://linear.app/tryghost/issue/SLO-89/cannot-read-properties-of-null-reading-get-an-unexpected-error

- if we pass an invalid ID when updating the products on a member, we
  throw a HTTP 500 error because `product` is `null`
- we can check for this and return a BadRequestError, because the user
  supplied an incorrect ID
2024-05-01 16:54:35 +02:00
Ghost CI
6b32548e5c Merged v5.82.5 into main 2024-05-01 13:22:28 +00:00
Ghost CI
a4c478958b v5.82.5 2024-05-01 13:22:26 +00:00
renovate[bot]
746df63307
Update dependency gscan to v4.43.1 (#20109)
ref https://linear.app/tryghost/issue/ENG-842/gluster-file-name-length-limit

- gscan v4.43.1 contains a bug fix to throw an large filenames
2024-05-01 11:26:21 +00:00
Daniel Lockyer
ddac3a9e8b Removed Sentry error capturing for failed URL decoding
fix https://linear.app/tryghost/issue/SLO-79/incorrectusageerror-the-url-httpsblogkongregatecompercentc0-couldnt-be

- we added this Sentry captureException whilst fixing a bug where
  decodeUrl could fail, and throw a 500 exception
- we added handling for that case and returned an empty string, but we
  also added Sentry error capturing
- at this point, I don't think we need to be capturing errors in Sentry,
  because the issue is already handled, and it only usually happens with
  malicious/incorrect URLs
- this is our #2 cause of Sentry alerts, so it's good to clean it up
2024-05-01 10:15:13 +02:00
renovate[bot]
60a3b5a913 Update TryGhost packages 2024-05-01 08:49:04 +02:00
Daniel Lockyer
7109743282 Fixed overly verbose DatabaseInfo require
- we don't need to deep require into the library as it exports what we
  need on the surface
- this should unblock https://github.com/TryGhost/Ghost/pull/19002, as
  it's randomly failing with this require
2024-04-30 21:39:56 +02:00
Michael Barrett
4cd85ab8b7
Added timeout when resizing an image (#20087)
refs
[ENG-827](https://linear.app/tryghost/issue/ENG-827/🐛-crash-on-resizing-animated-gif)

Added a timeout to the image resizing middleware to prevent crashes when
an image is taking too long to resize. When the timeout is reached and
the image has not been resized, the middleware will return the original
image
2024-04-30 08:39:30 +01:00
Ronald Langeveld
b2970cb4e0
Added integrity test for flags (#20094)
ref
https://ghost.slack.com/archives/C02G9E68C/p1714047709694639?thread_ts=1713956576.497899&cid=C02G9E68C
    
    - Ensures unique feature flags, avoiding configuration conflicts.
    - Enhances code reliability and simplifies feature tracking.
    - Prevents bad rebases was the reason for the initial duplication.
2024-04-29 02:39:15 +00:00
renovate[bot]
305029bc38 Update dependency ws to v8.17.0 2024-04-29 01:23:51 +00:00
renovate[bot]
f18f08928f Update dependency yjs to v13.6.15 2024-04-29 01:22:59 +00:00
Ghost CI
cf9ba60524 v5.82.4 2024-04-26 16:04:30 +00:00
Chris Raible
dcd65bfa4f
Added caching to the LinkRedirectRepository (#20036)
ref
https://linear.app/tryghost/issue/ENG-851/implement-a-minimal-but-complete-version-of-redirect-caching-to
ref https://app.incident.io/ghost/incidents/55

Often immediately after sending an email, sites receive a large volume
of requests to LinkRedirect endpoints from members clicking on the links in
the email.

We currently don't cache any of these requests in our CDN, because we
also record click events, update the member's `last_seen_at` timestamp,
and send webhooks in response to these clicks, so Ghost needs to handle
each of these requests itself. This means that each of these LinkRedirect requests
hits Ghost, and currently all these requests hit the database to lookup
where to redirect the member to.

Each one of these requests can make up to 11 database queries, which can
quickly exhaust Ghost's database connection pool. Even though the
LinkRedirect lookup query is fairly cheap and quick, these queries aren't
prioritized over the "record" queries Ghost needs to handle, so they can
get stuck behind other queries in the queue and eventually timeout.

The result is that members are unable to actually reach the destination
of the link they clicked on, instead receiving a 500 error in Ghost, or
it can take a long time (60s+) for the redirect to happen.

This PR uses our existing `adapterManager` to cache the redirect lookups
either in-memory or in Redis (if configured — by default there is no caching). This only removes 1 out of
11 queries per redirect request, so it won't reduce the load on the DB
drastically, but it at least decouples the serving of the LinkRedirect from
the DB so the member can be redirected even if the DB is under heavy
load.

Local load testing results have shown a decrease in response times from
60 seconds to ~50ms for the redirect requests when handling 500 requests
per second, and reduced the 500 error rate to 0.
2024-04-25 19:17:25 -07:00
Daniel Lockyer
892b9ab397 Moved internalLinking and stripeAutomaticTax flags to private beta
refs https://ghost.slack.com/archives/C02G9E68C/p1714051665654659?thread_ts=1713970812.191919&cid=C02G9E68C

- this enables us to enable the flags on sites without the need to
  enable developer experiments
- added `(private beta)` to the end of the relevant UI flags, because
  they're still in alpha in the UI despite being in beta in backend
2024-04-25 16:37:55 +02:00
Ronald Langeveld
601d05de0d
Removed duplicated ActivityPub labs key (#20089)
no issue

- rebased from main without seeing there's an existing flag.
- this removes the duplicated flag
2024-04-25 14:15:47 +00:00
Steve Larson
a0b7476794
Updated staff deletion logic (#20069)
ref https://linear.app/tryghost/issue/ENG-826

- Changed staff deletion logic to do a bulk insert when adding a tag to
the users' associated posts

Staff deletion logic has really poor performance at scale because we do
individual updates for every post. If a user has dozens+ posts
(especially in a large db with thousands of posts), this can take >60s
and look like a timeout. Ultimately this should probably be a jobbed off
process, but for the time being we can improve this by doing a bulk
insert.

Note that this update uses the pattern for the bulk tagging of posts
from the right click (bulk) actions in the posts lists in Admin. With
bulk actions, **we do not trigger web hooks or the post.edited events**.
We will document this and follow up on this separately.
2024-04-25 08:19:11 -05:00
Djordje Vlaisavljevic
7a3bbfde10
Added ActivityPub playground (#20081)
ref MOM61

- Adds admin-x react app we’ll use as ActivityPub playground to the
sidebar nav behind the feature flag.
- Wired up routing to Ember
- Setup the project as `admin-x-activitypub`

---------

Co-authored-by: Ronald Langeveld <hi@ronaldlangeveld.com>
2024-04-25 16:44:29 +08:00
Fabien O'Carroll
af02ca7044 Initial wire up of Posts -> Outbox flow
ref https://linear.app/tryghost/issue/MOM-29

This is very rough, and all still behind a flag. The idea is that any public
post which is published gets added to the Outbox of the site Actor. We also
dispatch an event, which will be used to deliver the Activity to any relevant
inboxes, but that is outside the scope of this commit.
2024-04-25 11:10:17 +07:00
renovate[bot]
25657b6a99 Update sentry-javascript monorepo to v7.112.1 2024-04-23 16:39:50 +02:00
renovate[bot]
3960e8caa1 Update sentry-javascript monorepo to v7.112.0 2024-04-23 14:11:30 +02:00
Daniel Lockyer
fda8aa5bfe Removed Sentry logging from XMLRPC service
refs https://ghost-foundation.sentry.io/issues/5135326925/

- the service tends to 503 all the time, and we don't really care enough
  for it to ping us in Sentry, as it's not something we control
- we can still keep logging the errors in case we need to go and look at
  what went wrong
2024-04-23 12:48:33 +02:00
renovate[bot]
d69a0aa1e6 Update dependency knex-migrator to v5.2.1 2024-04-22 12:29:48 +02:00
renovate[bot]
e4ffc7b8c0 Update dependency mysql2 to v3.9.7 2024-04-22 01:37:24 +00:00
renovate[bot]
69372d9018 Update dependency @sentry/profiling-node to v7.111.0 2024-04-19 16:56:32 +00:00
renovate[bot]
7fea4151d8 Update dependency @sentry/profiling-node to v7 2024-04-19 18:43:59 +02:00
Ghost CI
cb838c4bb6 v5.82.3 2024-04-19 16:04:16 +00:00
renovate[bot]
80fe672525 Update sentry-javascript monorepo to v7.111.0 2024-04-19 08:54:26 +02:00
Daniel Lockyer
f31814c804 Updated mysql2 dependency
refs #20050

- Renovate seems to be unable to bump the package past the security
  release, but unfortunately this release contains a breaking bug
- this commit manually bumps the package so we can get things flowing
  again
- the security release doesn't really affect us, but we should still try
  and keep on the latest
2024-04-19 08:48:24 +02:00
Chris Raible
a10b13916a
🐛 Fixed admin error when deleting an unsaved or imported post (#20053)
ref
https://linear.app/tryghost/issue/ENG-845/error-attempted-to-set-lexical-on-the-deleted-record
ref
[https://linear.app/tryghost/issue/ENG-854/🐛-deleting-imported-posts-makes-ghost-unresponsive](https://linear.app/tryghost/issue/ENG-854/%F0%9F%90%9B-deleting-imported-posts-makes-ghost-unresponsive)

- When deleting a post in the editor's Post Settings Menu, if the post
has unsaved changes (indicated by the hasDirtyAttributes property in the
editor), Admin will crash because it tries to save a post revision
before leaving the editor, but the post has already been deleted so
saving fails.
- This can occur when editing a post and quickly deleting it from the
Post Settings Menu before saving is completed.
- It can also occur when attempting to delete an imported post, as the
editor will parse the lexical from the server and may make some minor,
invisible-to-the-user changes to the lexical string locally (e.g. JSON
formatting, or updating the JSON to use extended version of base lexical
nodes), which triggers the same error.
- This fix bypasses the attempt to save a post revision when leaving the
editor if the post is already deleted, which allows the transition back
to the Posts route to succeed.
2024-04-18 10:02:02 -07:00
Kevin Ansfield
182fa62759
Bumped Koenig packages (#20047)
closes https://linear.app/tryghost/issue/MOM-39/

- fixes clicking on search results in internal linking PoC
- adds support for Node 20
2024-04-18 14:58:01 +00:00
Daniel Lockyer
10e81aeed8
ℹ️ Added support for Node 20
ref https://linear.app/tryghost/issue/ENG-765/add-support-for-node-20

- this adds support for Node 20 to Ghost and CI, as Node 20 is an LTS
  version and we should pick it up
2024-04-18 13:17:21 +02:00
Daniel Lockyer
8e0ad1a6fb
Fixed test on Node 20
refs f39d1d3aa3

- similar to the commit above, the JSON parser changed between Node 18
  and Node 20, so the error message changed too
- we actually just want to check the error is forwarded to the user, so
  we can do that by getting the error message from JSON.parse and check
  against that
2024-04-18 13:17:16 +02:00
renovate[bot]
ef6f79c272 Update dependency gscan to v4.43.0 2024-04-18 12:48:42 +02:00
Fabien O'Carroll
d34884fc6d Moved ActivityPub API to frontend URL
ref https://linear.app/tryghost/issue/MOM-48

This required some structural changes to our NestJS setup so that we can mount
it on multiple parts of the Ghost express app.

We've used the RouterModule to allow adding submodules that are mounted on
different paths, and we've had to be explicit about the base path for each
module. We've also had to switch back to using the Module decorator, because
RouterModule doesn't work with DynamicModule definitions.

Now that the NestJS app has knowledge of the full path, we need to "reset" the
url & baseUrl when passing the request into NestJS so that it can correctly
match the path. This is probably needed for the frontend too, for subdirs, but
that causes further issues - as this in prototype stage, we'll look later

Another issue is that NestJS replaces the express app instance with its own,
which isn't an issue for the Admin API (though we've fixed it anyway for
consistency), but did cause problems for the frontend, because the express app
is where view engine and directory information is stored.

The fix for this is to save a reference to the original ghost express
application, and reattach it to the request if it is not handled by Nest

Now that we have the Nest app mounted on the frontend, we're able to have it
handle the /.well-known/webfinger route with a proper controller, which is nice!
2024-04-18 13:26:24 +07:00
Fabien O'Carroll
885dc537d5 Added initial support for /.well-known/webfinger
ref https://linear.app/tryghost/issue/MOM-26
ref https://linear.app/tryghost/issue/MOM-27
ref https://webfinger.net/spec/

WebFinger is the protocol which allows different ActivityPub implementers to
find information about Actors, it's kinda like the entrypoint.

Given a username like @user@site.com, we can look up the URL for the Actor at

   https://website.com/.well-known/webfinger?resource=acct:user@site.com

This would then give us the info needed to discover the Actor's Inbox & Outbox
2024-04-17 16:12:57 +07:00
Fabien O'Carroll
ffbf2680ef Enabled Nest when ActivityPub flag is set
ref MOM-31
ref https://linear.app/tryghost/issue/MOM-31

We'll be building a lot of the new code for ActivityPub in Nest, so we'll need
to have it enabled in Ghost to work.
2024-04-17 14:00:07 +07:00
Fabien O'Carroll
0fb290a841 Added Feature Flag for ActivityPub
ref MOM-31
ref https://linear.app/tryghost/issue/MOM-31

Ronseal - let's goooooooo
2024-04-17 14:00:07 +07:00
renovate[bot]
96d0883928
🐛 Fixed file card button not being linked in emails (#20023)
ref https://linear.app/tryghost/issue/DES-202/

- bumped Koenig packages to include fix for incorrectly wrapped download image link in email rendering of file card
2024-04-16 10:37:28 +00:00
renovate[bot]
2af9c04477 Update CSS preprocessors 2024-04-16 09:44:36 +02:00
renovate[bot]
0802b9533c Update dependency semver to v7.6.0 2024-04-16 09:41:01 +02:00
Nicholas Mizoguchi
d6b7ebb517
Enforced more Mocha lint rules (#19720)
ref https://github.com/TryGhost/Ghost/issues/11038

1. Enforced lint rule
**[ghost/mocha/no-identical-title](https://github.com/lo1tuma/eslint-plugin-mocha/blob/main/docs/rules/no-identical-title.md)**
- Fixed relevant tests
2. Enforced lint rule
**[ghost/mocha/max-top-level-suites](https://github.com/lo1tuma/eslint-plugin-mocha/blob/main/docs/rules/max-top-level-suites.md)**
- No required fixes, as tests are compliant already

#### Additional details
Specifically for `ghost/mocha/no-identical-title` most fixes were simple
test description updates. Added comments to aid the PR review for the
ones that had relevant changes, and might require more attention. They
are as follows:
*
[e2e-api/admin/invites.test.js](https://github.com/TryGhost/Ghost/pull/19720#discussion_r1496397548):
Removed duplicated test (exact same code on both);
*
[e2e-api/admin/members.test.js](https://github.com/TryGhost/Ghost/pull/19720#discussion_r1496399107):
From the[ PR this was
introduced](73466c1c40 (diff-4dbc7e96e356428561085147e00e9acb5c71b58d4c1bd3d9fc9ac30e77c45be0L236-L237))
seems like author based his test on an existing one but possibly forgot
to rename it;
*
[unit/api/canary/utils/serializers/input/pages.test.js](https://github.com/TryGhost/Ghost/pull/19720#discussion_r1496400143):
The [page filter](https://github.com/TryGhost/Ghost/pull/14829/files)
was removed, so changed the description accordingly;
*
[unit/api/canary/utils/serializers/input/posts.test.js](https://github.com/TryGhost/Ghost/pull/19720#discussion_r1496400329):
The [page filter](https://github.com/TryGhost/Ghost/pull/14829/files)
was removed, so changed the description accordingly;
*
[unit/frontend/services/rendering/templates.test.js](https://github.com/TryGhost/Ghost/pull/19720#discussion_r1496402430):
Removed duplicated test
*
[unit/server/models/post.test.js](https://github.com/TryGhost/Ghost/pull/19720#discussion_r1496403529):
the change in [this
PR](https://github.com/TryGhost/Ghost/pull/14586/files#diff-c351cb589adefbb886570cfadb33b33eb8fdc12bde1024d1188cd18c165fc5e8L1010)
made three tests here mostly the same. Deduplicated them and kept only
one.
2024-04-16 09:37:06 +02:00
renovate[bot]
39a9a9bfd8 Update sentry-javascript monorepo to v7.110.1 2024-04-16 09:09:22 +02:00
renovate[bot]
b008c00feb Update sentry-javascript monorepo to v7.110.0 2024-04-15 09:47:44 +02:00
Ghost CI
ff03b2eedb v5.82.2 2024-04-15 01:06:07 +00:00
Ghost CI
bc0496e5d3 🎨 Updated Source to v1.2.2 2024-04-15 01:06:06 +00:00
Ghost CI
5e6cc0d746 🎨 Updated Casper to v5.7.2 2024-04-15 01:06:06 +00:00
Hannah Wolfe
145a184967 Added logging to RSS cache
- The RSS cache has lived for a really long time, but I'm not sure it's useful
- Want to be able to determine if it gets used much, and if not, then we can remove it
2024-04-11 10:33:53 +01:00
Michael Barrett
9e78412268
Added queue depth to requests (#19987)
refs
[CFR-14](https://linear.app/tryghost/issue/CFR-14/ensure-queue-depth-is-always-set-on-req)

Added queue depth to any request that passes through the request queue
middleware instead of only adding it to the request if it is queued.
This makes it easier to report on the queue depth within Elastic.
2024-04-11 09:24:04 +01:00
Kevin Ansfield
ebd36f2503
Added internalLinking labs flag (#20006)
ref https://linear.app/tryghost/issue/MOM-1
2024-04-10 12:13:37 +01:00
Ghost CI
f046442bd8 v5.82.1 2024-04-08 14:42:52 +00:00
Kevin Ansfield
5c05ebe6cb
Fixed browser tests broken by onboarding changes (#19998)
ref 78311591d0

- updated tests to not click a button on the setup/done screen that is no longer shown
- fixed setup flow showing an alert bar due to not handling the `TransitionAborted` error that is thrown by the setup/done->dashboard redirect
2024-04-08 15:15:04 +01:00
Kevin Ansfield
78311591d0
🎨 Improved post-setup onboarding flow (#19996)
ref https://linear.app/tryghost/issue/IPC-66/onboarding-checklist-v1

- replaced the setup/done screen with a new onboarding checklist shown on the dashboard
2024-04-08 13:03:41 +01:00
Ghost CI
b6195d204c v5.82.0 2024-04-05 16:05:12 +00:00
Chris Raible
01d0b2b304
Added new member signup flow behind labs flag (#19986)
ref https://linear.app/tryghost/issue/KTLO-1/members-spam-signups

- Some customers are seeing many spammy signups ("hundreds a day") — our
hypothesis is that bots and/or email link checkers are able to signup by
simply following the link in the email without even loading the page in
a browser.
- Currently new members signup by clicking a magic link in an email,
which is a simple GET request. When the user (or a bot) clicks that link, Ghost
creates the member and signs them in for the first time.
- This change, behind an alpha flag, requires a new member to click the
link in the email, which takes them to a new frontend route `/confirm_signup/`, then submit a form on the page which sends a POST request to the
server. If JavaScript is enabled, the form will be submitted
automatically so the only change to the user is an extra flash/redirect
before being signed in and redirected to the homepage.
- This change is behind the alpha flag `membersSpamPrevention` so we can
test it out on a few customer's sites and see if it helps reduce the
spam signups. With the flag off, the signup flow remains the same as
before.
2024-04-04 15:25:41 -07:00
Fabien O'Carroll
a262a64eea Moved additional payment methods into beta
ref ENG-812
ref https://linear.app/tryghost/issue/ENG-812

We want to roll this out as a beta feature
2024-04-04 23:54:01 +07:00
Fabien O'Carroll
c1b72b3997 Wired up additional payment types with labs flag
ref ENG-812
ref https://linear.app/tryghost/issue/ENG-812
2024-04-04 23:13:10 +07:00
Fabien O'Carroll
781599de62 Added a feature flag for additional payment methods
ref ENG-812
ref https://linear.app/tryghost/issue/ENG-812
2024-04-04 23:13:10 +07:00
Fabien O'Carroll
0bd4a800a5 Fixed error message for custom theme settings
ref ENG-809
ref https://linear.app/tryghost/issue/ENG-809

This includes the fix to the error message from gscan
2024-04-04 01:39:30 +07:00
Kevin Ansfield
d5a9731845
Fixed email_recipients indexes to match query usage (#19918)
closes https://linear.app/tryghost/issue/ENG-791/migration-to-fix-email-recipients-indexes

Our indexes over single columns (`delivered_at`, `opened_at`, `failed_at`) were ineffective because the only time we query those is alongside `email_id` meaning we were frequently performing full table scans on very large tables during our email analytics jobs.

- added migration to add new indexes covering `email_id` and the respective columns
- added migration to drop the old indexes that weren't being used in any query plans

Local runtime with ~2M email_recipient rows:
- before: 1.7s
- after: 99ms

Explain output...

before:
```
+----+-------------+------------------+------------+-------+----------------------------------------------------------------------------------+----------------------------------------------+---------+-------+--------+----------+------------------------------------+
| id | select_type | table            | partitions | type  | possible_keys                                                                    | key                                          | key_len | ref   | rows   | filtered | Extra                              |
+----+-------------+------------------+------------+-------+----------------------------------------------------------------------------------+----------------------------------------------+---------+-------+--------+----------+------------------------------------+
|  1 | UPDATE      | emails           | NULL       | index | NULL                                                                             | PRIMARY                                      | 98      | NULL  |      1 |   100.00 | Using where                        |
|  4 | SUBQUERY    | email_recipients | NULL       | range | email_recipients_email_id_member_email_index,email_recipients_failed_at_index    | email_recipients_failed_at_index             | 6       | NULL  |   2343 |     7.76 | Using index condition; Using where |
|  3 | SUBQUERY    | email_recipients | NULL       | ref   | email_recipients_email_id_member_email_index,email_recipients_opened_at_index    | email_recipients_email_id_member_email_index | 98      | const | 159126 |    50.00 | Using where                        |
|  2 | SUBQUERY    | email_recipients | NULL       | ref   | email_recipients_email_id_member_email_index,email_recipients_delivered_at_index | email_recipients_email_id_member_email_index | 98      | const | 159126 |    50.00 | Using where                        |
+----+-------------+------------------+------------+-------+----------------------------------------------------------------------------------+----------------------------------------------+---------+-------+--------+----------+------------------------------------+
```

after:
```
+----+-------------+------------------+------------+-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------+---------+------+--------+----------+--------------------------+
| id | select_type | table            | partitions | type  | possible_keys                                                                                                                                                                 | key                                          | key_len | ref  | rows   | filtered | Extra                    |
+----+-------------+------------------+------------+-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------+---------+------+--------+----------+--------------------------+
|  1 | UPDATE      | emails           | NULL       | index | NULL                                                                                                                                                                          | PRIMARY                                      | 98      | NULL |      1 |   100.00 | Using where;             |
|  4 | SUBQUERY    | email_recipients | NULL       | range | email_recipients_email_id_member_email_index,email_recipients_email_id_delivered_at_index,email_recipients_email_id_opened_at_index,email_recipients_email_id_failed_at_index | email_recipients_email_id_failed_at_index    | 104     | NULL |     60 |   100.00 | Using where; Using index |
|  3 | SUBQUERY    | email_recipients | NULL       | range | email_recipients_email_id_member_email_index,email_recipients_email_id_delivered_at_index,email_recipients_email_id_opened_at_index,email_recipients_email_id_failed_at_index | email_recipients_email_id_opened_at_index    | 104     | NULL | 119496 |   100.00 | Using where; Using index |
|  2 | SUBQUERY    | email_recipients | NULL       | range | email_recipients_email_id_member_email_index,email_recipients_email_id_delivered_at_index,email_recipients_email_id_opened_at_index,email_recipients_email_id_failed_at_index | email_recipients_email_id_delivered_at_index | 104     | NULL | 146030 |   100.00 | Using where; Using index |
+----+-------------+------------------+------------+-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------+---------+------+--------+----------+--------------------------+
```
2024-04-03 17:52:52 +01:00
Kevin Ansfield
bd93bf0dea Optimised email stats aggregation query for typical column usage
ref https://linear.app/tryghost/issue/ENG-790/remove-use-of-sub-queries-in-email-analytics

- the `delivered_at` column is typically entirely/nearly entirely filled with values meaning the `IS NOT NULL` query matches a huge number of rows that MySQL has to fetch from the index to count
- using `IS NULL` switches that behaviour around as it will now match very few rows which has been shown in testing to be considerably quicker
- after switching to `IS NULL` the query returns an "undelivered" count rather than a "delivered" count, in order to keep the rest of the system behaviour the same we can calculate the delivered count by subtracting the query result from the total number of emails sent which we can fetch using a very fast primary key lookup query on the `emails` table
2024-04-03 16:27:23 +01:00
Kevin Ansfield
4f5a7b420e Removed use of subqueries in email analytics queries
closes https://linear.app/tryghost/issue/ENG-790/remove-use-of-sub-queries-in-email-analytics

Avoiding sub queries means we don't have a process tied up for longer than necessary and we can more easily see if one of the queries is non-performant.

- extracted the count queries into separate queries and used the retrieved values in the final update query
- removed a query by moving the email open rate calculation into JS as we've already fetched the necessary data before that point
2024-04-03 16:27:23 +01:00
renovate[bot]
541395434d Update sentry-javascript monorepo to v7.109.0 2024-04-01 14:10:07 +02:00
Ghost CI
d4d69e2ac5 v5.81.1 2024-04-01 06:16:31 +00:00
Steve Larson
78d2a5e3c0
🐛 Fixed flaky browser tests (#19929)
ref https://linear.app/tryghost/issue/CFR-13
- enabled saving traces on browser test failure; this makes troubleshooting a lot easier
- updated handling in offers tests to ensure the tier has fully loaded in the UI (not just `networkidle`)
- updated publishing test to examine the publish button reaction to the save action response instead of a 300ms pause

In general, our tests use a lot of watching for 'networkidle' - and sometimes just raw timeouts - which do not scale well into running tests on CI. In particular, 'networkidle' does not work if we're expecting to see React components' state updates propagate and re-render. We should always instead look to the content which encapsulates the response and the UI updates. This is something we should tackle on a larger scale.
2024-03-27 13:57:53 -05:00
renovate[bot]
dcbbfbba70 Update dependency express to v4.19.2 [SECURITY] 2024-03-27 11:18:44 +01:00
Sag
5c4a4e812c
Removed Powered by Ghost clicks in publisher analytics (#19926)
fixes https://linear.app/tryghost/issue/TRI-65/add-powered-by-ghost-badge-tracking

- clicks on the "Powered by Ghost" badge were unintentionally surfaced
in publisher analytics, under Newsletter Clicks
2024-03-26 17:51:23 +01:00
renovate[bot]
ec25aed06f fix(deps): update dependency mysql2 to v3.9.3 2024-03-26 07:46:09 +00:00
Fabien 'egg' O'Carroll
3d3b3ff701
Fixed Editors being able to invite Editors (#19904)
ref ENG-774
ref https://linear.app/tryghost/issue/ENG-774

Staff Tokens will have both a `user` and an `apiKey` present on the
`loadedPermissions`.

The check here for `apiKey` was written when we could assume that an
`apiKey` was an Admin Integration - so it completely overwrote the
previous `allowed` list. When we added the concept of Staff Tokens -
this resulted in a privilege escalation.

This is a good lesson in not using proxies or indicators for data, as
changes elsewhere can invalidate them - if we had been specific and
checked the role of the current actor we wouldn't've had this bug!
2024-03-26 00:45:08 +07:00
Daniël van der Winden
3fa363f944
Fixed design issue DES-4 (#19662)
Fixed inconsistencies in typography for footer and featured images, on desktop and mobile.
2024-03-25 12:08:34 +01:00
renovate[bot]
97c63e1735 chore(deps): update sentry-javascript monorepo to v7.108.0 2024-03-25 08:34:53 +00:00
Ghost CI
5d714f8a05 v5.81.0 2024-03-22 16:05:44 +00:00
Steve Larson
a1c4e64994
Added queueing middleware to handle high request volume (#19887)
ref https://linear.app/tryghost/issue/CFR-4/
- added request queueing middleware (express-queue) to handle high
request volume
- added new config option `optimization.requestQueue`
- added new config option `optimization.requestConcurrency`
- added logging of request queue depth - `req.queueDepth`

We've done a fair amount of investigation around improving Ghost's
resiliency to high request volume. While we believe this to be partly
due to database connection contention, it also seems Ghost gets
overwhelmed by the requests themselves. Implementing a simple queueing
system allows us a simple lever to change the volume of requests Ghost
is actually ingesting at any given time and gives us options besides
simply increasing database connection pool size.

---------

Co-authored-by: Michael Barrett <mike@ghost.org>
2024-03-21 09:25:07 -05:00
renovate[bot]
dfdd4e5cfa Update dependency express to v4.19.1 2024-03-21 11:50:48 +01:00
Ghost CI
b88ef5f816 Merged v5.80.5 into main 2024-03-21 09:51:52 +00:00
Ghost CI
40277465ba v5.80.5 2024-03-21 09:51:50 +00:00
Sag
c2320cd2ea Revert "Added referral tracking to the powered-by-ghost newsletter badge" (#19899)
refs https://ghost.slack.com/archives/CTH5NDJMS/p1710976281912809

- this reverts commit 9869d9adb6
- the referral query parameter is unintentionally surfacing in publisher
analytics
2024-03-21 10:37:42 +01:00
Sag
5477d70a0c
Revert "Added referral tracking to the powered-by-ghost newsletter badge" (#19899)
refs https://ghost.slack.com/archives/CTH5NDJMS/p1710976281912809

- this reverts commit 9869d9adb6
- the referral query parameter is unintentionally surfacing in publisher
analytics
2024-03-21 10:02:17 +01:00
renovate[bot]
28c851be6a Update dependency sanitize-html to v2.13.0 2024-03-20 19:10:32 +00:00
Fabien O'Carroll
cb72835af1 Removed support for id specific permissions
ref ENG-728
ref https://linear.app/tryghost/issue/ENG-728

This is not used anywhere, and makes the code more complicated, it's a good
step toward simplifying permissions and pulling them out of the database.
2024-03-21 00:21:40 +07:00
Fabien O'Carroll
e29c653ef6 Removed permissible method from setting model
ref ENG-728
ref https://linear.app/tryghost/issue/ENG-728

This implementation is essentially a no-op so we can remove it completely
2024-03-21 00:21:24 +07:00
Daniel Lockyer
27cc32ec25 Added comments count endpoint to robots.txt disallow list
fix https://linear.app/tryghost/issue/ENG-771/add-comments-count-endpoint-to-robotstxt-ignorelist

- we've seen web scrapers hitting this endpoint a lot, but the value to
  be taken from it is minimal for SEO purposes
- adding it to robots.txt should encourage web scrapers to ignore it,
  and we should see less traffic as a result
2024-03-20 14:48:54 +01:00
Fabien 'egg' O'Carroll
7cc65c18cc
Added missing permissions to Contributor & Editor (#19881)
ref ENG-728
ref https://linear.app/tryghost/issue/ENG-728

This is NOT a functionality change. The Post#permissible method unit
tests have been updated to pass `true` as `hasUserPermission` and we can
see that the permission functionality remains the same.

The permissible method of the post model is responsible for removing
permission based on the data that is being modified, but the permissions
module is setup to allow the permissible method to grant permission -
this means that we call permissible, even if the current actor doesn't
have permission, this results in code that is hard to understand and
manage.

We are going to be instead returning early if an actor does not have
permission, this will allow permissible method signatures to be greatly
simplified (removing the need for hasUserPermission, hasApiKeyPermission
& hasMemberPermission arguments).
2024-03-20 20:36:07 +07:00
renovate[bot]
38f8e05a3e Update dependency knex-migrator to v5.2.0 2024-03-20 10:54:21 +01:00
renovate[bot]
3ff2eecb76 Update dependency bookshelf-relations to v2.7.0 2024-03-19 11:20:22 +00:00
Daniel Lockyer
134c33cef5
🐛 Fixed missing source + resized images producing rendered 404 (#19869)
fixes https://linear.app/tryghost/issue/ENG-746/http-500-responses-when-handle-image-sizes-middleware-hits-missing

- in the event a request comes in for a resized image, but the source
image does not exist, we return a rendered 404 page
- we do this because we pass the NotFoundError to `next`, which skips
over the static asset code where we return a plaintext 404
- also included a breaking test that ensure we go to the next middleware
without an error
2024-03-18 18:32:10 +01:00
Fabien 'egg' O'Carroll
3f27ca5c00
Cached api controller pipelines (#19880)
ref ENG-761
ref https://linear.app/tryghost/issue/ENG-761

Creating these pipelines is expensive, and we don't want to do it
repeatedly for the same controller. Adding caching should reduce the
amount of time spent setting up pipelines for each usage of the `get`
helper.
2024-03-19 00:29:41 +07:00
Djordje Vlaisavljevic
a67342b06a
Add WIP onboarding checklist behind the flag (#19801)
ref https://linear.app/tryghost/issue/IPC-66/onboarding-checklist-v1

- Adds a basic version of a new onboarding checklist behind the feature
flag, without incomplete/complete state logic
- Links to Design settings, Members screen and new post
- Opens amodal that we’ll use as Share modal

---------

Co-authored-by: Daniël van der Winden <danielvanderwinden@ghost.org>
2024-03-18 14:53:01 +00:00
Daniel Lockyer
48ae822b9f Removed NewRelic from Ghost
fix https://linear.app/tryghost/issue/SLO-47/remove-newrelic-from-ghost

- we no longer need this in Ghost because we've doubled down on Sentry
  Performance
- we can remove this because we've seen it costs 200-300ms to include
  the dependency
2024-03-18 15:22:33 +01:00
Michael Barrett
949b157775
Fixed naming of connection pool metrics (#19874)
no refs
2024-03-18 09:59:16 +00:00
Ghost CI
715ff56032 v5.80.4 2024-03-18 09:42:10 +00:00
Ghost CI
6849aa9891 v5.80.3 2024-03-15 16:04:57 +00:00
Ghost CI
a52c645c44 🎨 Updated Source to v1.2.1 2024-03-15 16:04:57 +00:00
Ghost CI
8b438fa015 🎨 Updated Casper to v5.7.1 2024-03-15 16:04:57 +00:00
renovate[bot]
566a065f36 Update dependency socket.io to v4.7.5 2024-03-14 17:30:47 +00:00
Fabien 'egg' O'Carroll
6a35f6e4cc
Fixed get helper cache optimizations (#19865)
ref [ENG-747](https://linear.app/tryghost/issue/ENG-747/)
ref https://linear.app/tryghost/issue/ENG-747

H'okay - so what we're trying to do here is make get helper queries more
cacheable. The way we're doing that is by modifying the filter used when
we're trying to remove a single post from the query.

The idea is that we can remove that restriction on the filter, increase
the number of posts fetched by 1 and then filter the fetched posts back
down, this means that the same query, but filtering different posts,
will be updated to make _exactly_ the same query, and so share a cache!

We've been purposefully restrictive in the types of filters we
manipulate, so that we only deal with the simplest cases and the code is
easier to understand.
2024-03-15 00:18:15 +07:00
Fabien 'egg' O'Carroll
e607556913
Stopped us from rendering no html when error occurs
no-issue

We need to make sure that we return, otherwise we'll end up skipping the
error handler middleware and trying to render.
2024-03-14 16:34:13 +00:00
Fabien 'egg' O'Carroll
aaa19a535a
🐛 Fixed 500 errors for invalid theme layouts (#19848)
ref ENG-742
ref https://linear.app/tryghost/issue/ENG-742

We don't do any parsing of layouts in gscan, which means themes can be
uploaded which use non-existent files for their layout.

We can catch the error in the res.render call, and wrap it, just like we
do for missing templates (e.g. the StaticRoutesRouter)
2024-03-14 23:12:26 +07:00
Aileen Booker
f16d9802d0 Added ability to pass minThreshold for Milestone Slack notifications
closes ENG-632

- This listens to a new property in the `milestones` config to set a minimum value of Milestones we wanna use the Slack notification service for
2024-03-14 12:06:43 -04:00
Michael Barrett
60d81b2003
🐛 Fixed /p/ redirects not being indexed by search engines (#19864)
ref
[ENG-741](https://linear.app/tryghost/issue/ENG-741/🐛-our-robotstxt-config-causes-indexing-issues-for-customers-who-have)

`/p/` has been dropped from the `robots.txt` file so that search engines
can index the pages at these locations. In the event that the page at
the location is a preview page, the existing robots meta tag on the page
will prevent indexing.
2024-03-14 14:44:54 +00:00
Fabien O'Carroll
39da5a1f88 Revert "Optimised queries made by get helper for posts"
no-issue

This was incorrectly merged - reverting until the work is complete
2024-03-14 20:26:01 +07:00
Sag
7a40ab52fb
🐛 Fixed adding recommendation when oembed fails (#19861)
refs https://linear.app/tryghost/issue/ENG-750

- when adding a recommendation, we fetch the recommended site's metadata
- before this change, if the metadata fetch failed for some reason, we'd show an error and block the recommendation from being added
- after this change, we use fallback values if the metadata fails to fetch, instead of blocking the recommendation from being added. We use the site domain as the title and leave the rest empty (no favicon, no description)
- this change also means we are not checking whether a site exists or not for the publisher anymore. It’s then up to the publisher to make sure they don’t enter broken URLs
2024-03-14 11:36:28 +01:00
Fabien 'egg' O'Carroll
52a28c0059
Optimised queries made by get helper for posts (#19859)
ref ENG-747
ref https://linear.app/tryghost/issue/ENG-747

H'okay - so what we're trying to do here is make get helper queries more
cacheable. The way we're doing that is by modifying the filter used when
we're trying to remove a single post from the query.

The idea is that we can remove that restriction on the filter, increase
the number of posts fetched by 1 and then filter the fetched posts back
down, this means that the same query, but filtering different posts,
will be updated to make _exactly_ the same query, and so share a cache!

We've been purposefully restrictive in the types of filters we
manipulate, so that we only deal with the simplest cases and the code is
easier to understand.
2024-03-13 19:27:27 +00:00
renovate[bot]
48782df301 Update dependency newrelic to v11.13.0 2024-03-13 17:14:53 +00:00
Kevin Ansfield
47e6911ca0
Added cache-control header back to /auth-frame/ response (#19858)
ref https://linear.app/tryghost/issue/ENG-721

- when changing the response to a `204` for requests with no cookie we'd lost the `Cache-Control: public, max-age: 0` header which meant some cache systems weren't caching as efficiently as possible
2024-03-13 16:00:46 +00:00
Sag
9869d9adb6
Added referral tracking to the powered-by-ghost newsletter badge (#19850)
ref https://linear.app/tryghost/issue/TRI-65

In the context of referrals, we want to understand how useful our
“Powered by Ghost” badges are.

There are currently a few places where the “Powered by Ghost” badge can
be rendered:
- in newsletters (enabled/disabled by publisher, on a newsletter basis)
- in Portal popups, e.g. member signup/signin/account settings
- in the footer of some themes, including Source & Casper

We're adding the query param ?via to evaluate the usage of the badge in
newsletters.
2024-03-13 16:03:13 +01:00
Fabien "egg" O'Carroll
0fb0c6c2b5 Added NestJS Playground
no-issue

This adds the barebones of a NestJS application wired up to the Admin API
behind a feature flag, so that we can experiement with how to use Nest in the
context of Ghost
2024-03-13 19:44:06 +07:00
Fabien "egg" O'Carroll
d2620171ea Refactored auth services so they can be used in Nest
no-issue

This decouples the business logic from the express middleware so that it can be
used inside of a NestJS application.
2024-03-13 19:44:06 +07:00
Sag
59bbade630
Fixed browser tests (#19852)
no issue

- browser tests were failing due to the renaming of a button
2024-03-13 12:54:19 +01:00
Fabien 'egg' O'Carroll
5a5ddcb609
🐛 Fixed Tiers API erroring when invalid filter passed (#19845)
closes ENG-730
closes https://linear.app/tryghost/issue/ENG-730/

We've updated the input serializer to parse the filter, and responded
with an error if it cannot be parsed correctly.

Now that it's parsed, we can pass a mongo query object through the
stack, which will lend itself to better typing for this code, which is a
direction we want to go in anyway. We've had to update all the internal
usages of the `browse` method to use mongo query objects.
2024-03-13 00:25:42 +07:00
Daniel Lockyer
55791a8c64 Switched to throwing error upon failed image processing
ref https://linear.app/tryghost/issue/ENG-740/http-500-error-when-image-processing-fails
refs 4aad551c72

- upon further discussion, we've decided it's better to throw an error
  in this case because the uploaded image is deemed invalid and storing
  it on the filesystem might cause more issues with resizing/further
  processing in the future
- this commit implements that and alters the tests
2024-03-12 16:24:29 +01:00
Daniel Lockyer
4aad551c72 🐛 Fixed HTTP 500 error when image processing fails during upload
fixes ENG-740
fixes https://linear.app/tryghost/issue/ENG-740/http-500-error-when-image-processing-fails

- in the event the image transform library throws (which can happen for
  many reasons; sharp/libvips can come across a number of errors), we
  currently return this as a HTTP 500 error to the user
- in this case, we should just try-catch the call and jump to the
  non-processing flow where it just saves the original image
- also added breaking test
2024-03-12 15:33:17 +01:00
Kevin Ansfield
ef143978e7
🎨 Reduced requests and 403 responses for comments auth check (#19840)
closes https://linear.app/tryghost/issue/ENG-721
ref https://linear.app/tryghost/issue/ENG-708

Comments-UI loads `/ghost/admin-frame/` in an iframe to check if a Staff User is authenticated in order to  show moderation options. That iframe request loads a HTML page which in turn contains a script that fires off an API request that attempts to fetch the logged-in user details, resulting in a 403 "error" showing up when not authenticated. In the vast majority of cases there will be no staff user authenticated so lots of extra requests and "errors" are seen unnecessarily.

- adjusted the `/ghost/auth-frame/` endpoint to check if the request contains an Admin session cookie
  - if it does, continue as before with rendering the HTML page so the script is loaded
  - if it doesn't, return an empty 204 response avoiding the script request and subsequent 403-generating API request
- eliminates the 403 error being generated for all typical visitor traffic, the error should only be seen when an Admin was previously logged in but their cookie is no longer valid (either from logging out, or going past the 6month validity period)
2024-03-12 12:27:18 +00:00
Daniel Lockyer
5fa4496d52 🐛 Fixed HTTP 500 responses when oembed endpoint receives error
fixes https://github.com/TryGhost/Product/issues/4237

- this fixes the fact that we return a HTTP 500 response when the oembed
  library receives an error, such as a 401 or 403
- includes special handling for cases where we want to return a slightly
  different error message
- also adds unit tests for @tryghost/oembed-service package
2024-03-12 12:31:44 +01:00
Daniel Lockyer
6842d599e9 🐛 Fixed handling of image uploads with overly long filenames
fixes ENG-733
ref https://linear.app/tryghost/issue/ENG-733/handle-image-uploads-where-name-is-too-long

- filesystems usually have a filename length limit; ie. on macOS it is
  255 characters
- if a file is uploaded with a longer filename, we'll return a HTTP 500
- we shouldn't do this as it is user error, so we can just catch the
  error code and return BadRequest
- this implements that, and adds a breaking test
2024-03-12 12:31:44 +01:00
Daniel Lockyer
6db20fc14b Fixed minor code nits
- made fixes for the following:
  - jsdoc definitions
  - typos
  - extra parameter to function
  - missing `utf-8` to fs file read
2024-03-12 12:31:44 +01:00
Daniel Lockyer
360ecf15ae 🐛 Fixed HTTP 500 error when given incorrect Range header
ref ENG-729
ref https://linear.app/tryghost/issue/ENG-729/incorrect-range-header-leads-to-http-500-errors

- we didn't have handling here for the `RangeNotSatisfiableError` that
  can come from express/serve-static/send
- as a result, passing an invalid range would cause a 500 error
- this prevents that and adds a breaking test
2024-03-11 19:14:30 +01:00
Daniel Lockyer
162f438c63 Updated @tryghost/errors dependency
- this version is written in TS, but was published a few months ago and
  needs to be bumped here
- also updates a previous deep include into the library, which was
  unnecessary anyway
2024-03-11 17:33:51 +01:00
Ghost CI
f83d51c1e3 v5.80.2 2024-03-08 16:04:08 +00:00
Ghost CI
76383b4295 🎨 Updated Source to v1.2.0 2024-03-08 16:04:08 +00:00
renovate[bot]
3301332253 Update dependency express to v4.18.3 2024-03-07 13:42:27 +01:00
Ghost CI
624168ead5 Merged v5.80.1 into main 2024-03-07 09:04:51 +00:00
Ghost CI
0a8716b0ae v5.80.1 2024-03-07 09:04:50 +00:00
Sag
ae95e8de8c Fixed tiers paywall selecting all paid tiers (#19817)
refs INC-36

- oversight in parent commit 00cff0a
2024-03-06 22:35:43 +01:00
Sag
69466ecab9 🐛 Fixed free tier showing in the tiers-only paywall in posts (#19807)
refs INC-36
fixes https://github.com/TryGhost/Ghost/issues/19796

- The tiers-only paywall was incorrectly rendering "Free". Example:
"This post is for subscribers of the Free, Silver and Gold tiers only"
- Steps to reproduce the issue:
    1. Create a post with public visibility, publish it
    2. Then swap the visibility to specific tiers. The default selects all
paid tiers. Leave it like that
    3. Update the post. The paywall show Free, even though it should be
showing only the paid tiers
- This fix filters out the "free" tier when visibility is set to tiers,
before updating a Post or a Page. The fix includes bulk updates from the
list of Posts and Pages (right-click on a Post/Page > Change Access).
2024-03-06 22:35:33 +01:00
Sag
656846018a
Fixed tiers paywall selecting all paid tiers (#19817)
refs INC-36

- oversight in parent commit 00cff0a
2024-03-06 22:14:17 +01:00
Sag
00cff0aece
🐛 Fixed free tier showing in the tiers-only paywall in posts (#19807)
refs INC-36
fixes https://github.com/TryGhost/Ghost/issues/19796

- The tiers-only paywall was incorrectly rendering "Free". Example:
"This post is for subscribers of the Free, Silver and Gold tiers only"
- Steps to reproduce the issue:
    1. Create a post with public visibility, publish it
    2. Then swap the visibility to specific tiers. The default selects all
paid tiers. Leave it like that
    3. Update the post. The paywall show Free, even though it should be
showing only the paid tiers
- This fix filters out the "free" tier when visibility is set to tiers,
before updating a Post or a Page. The fix includes bulk updates from the
list of Posts and Pages (right-click on a Post/Page > Change Access).
2024-03-06 21:30:00 +01:00
Kevin Ansfield
3090f8ec95
🎨 Improved lazy-loading of comments data (#19809)
no issue

Bumps `Comments-UI` app version that contains an improvement to data loading:

- within the comments block we only use Admin auth to show moderation options on each displayed comment but we were always pre-emptively loading the `admin-auth` frame and making the associated Admin API user request. That loading has now been deferred until at least one comment has been displayed cutting down unnecessary requests on each post view
2024-03-06 10:29:55 +00:00
Kevin Ansfield
78aba5b22a
🎨 Improved lazy-loading of comments data (#19809)
no issue

Bumps `Comments-UI` app version that contains an improvement to data loading:

- within the comments block we only use Admin auth to show moderation options on each displayed comment but we were always pre-emptively loading the `admin-auth` frame and making the associated Admin API user request. That loading has now been deferred until at least one comment has been displayed cutting down unnecessary requests on each post view
2024-03-06 10:17:32 +00:00
Kevin Ansfield
b704530d74
🐛 Fixed unexpected conversion of single-quoted attributes in HTML cards (#19727)
closes ENG-627

We were using `cheerio` to parse+modify+serialize our rendered HTML to modify links for member attribution. Cheerio's serializer has a [long-standing issue](https://github.com/cheeriojs/cheerio/issues/720) (that we've [had to deal with before](https://github.com/TryGhost/SDK/issues/124)) where it replaces single-quote attributes with double-quote attributes. That was resulting in broken rendering when content used single-quotes such as in HTML cards that have JSON data inside a `data-` attribute or otherwise used single-quotes to avoid escaping double-quotes in an attribute value.

- swapped the implementation that uses `cheerio` for one that uses `html5parser` to tokenize the html string, from there we can loop over the tokens and replace the href attribute values in the original string without touching any other part of the content. Avoids a full parse+serialize process which is both more costly and can result unexpected content changes due to serializer opinions.
  - fixes the quote change bug
  - uses tokenization directly to avoid cost of building a full AST
- updated Content API Posts snapshot
  - one of our fixtures has a missing closing tag which we're no longer "fixing" with a full parse+serialize step in the link replacer (keeps modified src closer to original and better matches behaviour elsewhere in the app / without member-attribution applied)
  - the link replacer no longer converts `attr=""` to `attr` (these are equivalent in the HTML spec so no change in behaviour other than preserving the original source html)
- added a benchmark test file comparing the two implementations because the link replacer runs on render so it's used in a hot path
  - new implementation has a 3x performance improvement
  - the separate files with the old/new implementations have been cleaned up but I've left the benchmark test file in place for future reference

Benchmark results comparing implementations:

```
❯ node test/benchmark.js

LinkReplacer
├─ cheerio: 5.03K /s ±2.20%
├─ html5parser: 16.5K /s ±0.43%

Completed benchmark in 0.9976526670455933s
┌─────────────┬─────────┬────────────┬─────────┬───────┐
│   (index)   │ percent │ iterations │ current │  max  │
├─────────────┼─────────┼────────────┼─────────┼───────┤
│   cheerio   │   ''    │ '5.03K/s'  │  5037   │ 5037  │
│ html5parser │   ''    │ '16.5K/s'  │  16534  │ 16534 │
└─────────────┴─────────┴────────────┴─────────┴───────┘
```
2024-03-06 09:11:49 +00:00
Fabien O'Carroll
d9fb4787ec Removed whitelist of JWT errors
refs https://linear.app/tryghost/issue/ENG-712/

I don't think we ever need to respond with a 500 here, if the verify call
fails, we know that the token is unauthorized for use.
2024-03-05 03:04:34 +07:00
Fabien O'Carroll
dcbd168585 🐛 Fixed 500 error for premature api token use
refs https://linear.app/tryghost/issue/ENG-712

We weren't handling the NotBeforeError and instead responing with a 500 which
is not correct.
2024-03-05 03:04:34 +07:00
renovate[bot]
b6b2e2ea31 Update dependency newrelic to v11.12.0 2024-03-04 18:35:26 +00:00
Ghost CI
9df5148427 v5.80.0 2024-03-01 16:04:20 +00:00
renovate[bot]
69459e9b42 Update dependency yjs to v13.6.14 2024-03-01 11:32:05 +00:00
renovate[bot]
81f1b63cca Update dependency yjs to v13.6.13 2024-02-29 19:32:37 +00:00
Peter Zimon
8f3617aaa8
Content card design improvements (#19737)
refs. https://linear.app/tryghost/issue/DES-122/bookmark-card-issues

This PR addresses the following content card related problems:

1. The design of the following cards are more self-contained so it makes
more sense to use `px` for their font-sizes and spacings so it looks the
same regardless of the theme. Of course themes still can override these
values.

Updated cards to use `px` for font sizing:
- audio
- bookmark
- file
- product

2. So far header and signup cards had been using `rem` for font-sizes
and some sizing. This commit updates these to use `em` instead so that
it's consistent with all other cards.

3. The favicon sometimes is not available for bookmark cards. This PR also
fixes that by providing a default favicon for these cases.
2024-02-29 17:09:34 +01:00
Sanne de Vries
c7e475feb0
Remove comment icon at the top of email template (#19771)
Refs
https://linear.app/tryghost/issue/DES-80/newsletter-view-in-browser-breaking-to-next-line-with-incorrect
2024-02-29 14:45:38 +01:00
Kevin Ansfield
e0d8e18785
Added lazy-loading of comments data (#19778)
no issue

Bumps `Comments-UI` app version that contains a few changes:

- comments data is now lazy-loaded with API requests being deferred until the comments block is scrolled into view, saving up-front visitor data usage as well as reducing server-load for page views where the comments are never seen
- comments data is now fetched from `/members/api/comments/{post_id}/` rather than using the post_id in the `filter` param to enable cache bucketing and cache invalidation
- `created_at` timestamp has been dropped from the initial comments data request so the results can be cached, on pagination requests the timestamp has been improved to use the created_at data from the response so it remains consistent and can also be cached
- `order` param has been dropped from API requests as the API has been updated to include our default ordering
2024-02-29 10:21:05 +00:00
Fabien 'egg' O'Carroll
a489d5a3d8
Added /comments/:id/replies/ to X-Cache-Invalidate
refs https://linear.app/tryghost/issue/ENG-682/

This should allow us to bust both endpoints cache when write operations
are made to comments.
2024-02-28 23:15:02 +00:00
Kevin Ansfield
44e602b447
Switched to default ordering for comments API requests (#19774)
closes ENG-681

There's no need to provide an `order` param with every request in Comments-UI if the API has default ordering that matches our requirements. The order param makes logs more noisy/harder to read than they need to be so we want to get rid of it.

- modified comments API input serializer to add a default order param to the browse and replies endpoints when none is provided
- removed order param from the requests that Comments-UI makes
2024-02-28 18:42:02 +00:00
Fabien O'Carroll
4c6f7715ef Cleaned up comments controller
no-issue

This removes some redundant calls to `get` and makes refactoring
easier in future.
2024-02-29 00:24:34 +07:00
Fabien O'Carroll
001f2b0b91 Invalidated post comments cache on like&unlike
refs https://linear.app/tryghost/issue/ENG-676/

We want to make sure that we're not serving stale liked counts for
comments, which means we need to cache bust when they're liked/unliked

Unfortuantely this means we need to fetch the comment from the db so
that we have access to the post id.
2024-02-29 00:24:34 +07:00
Fabien O'Carroll
58dd79ccb4 Invalidated the new comments endpoint cache on add & edit
refs https://linear.app/tryghost/issue/ENG-676/

This is the meat of the change and actually causes the cache to be
invalidated on adds and edits to the comments endpoints.

It doesn't currently include the liked/unliked actions at the moment
as we don't have easy access to the post id from those endpoints.
2024-02-28 22:40:56 +07:00
Fabien O'Carroll
2c6321472c Added endpoint for comments/post/:post_id
refs https://linear.app/tryghost/issue/ENG-676/

This is pretty simple as we can reuse the existing browse method
on the CommentsController, but we need to add support for the post_id
option to the endpoint, for it to be added to the frame.

We also need to update the browse method to enforce the post_id on the
NQL filter. I initially tried this with string concatenation, but ran
into way too many bugs, so we're using a mongo transformer instead.
2024-02-28 22:40:56 +07:00
Djordje Vlaisavljevic
f032f11d8a Added hrefs to paywall links for improved SEO
refs DES-150
2024-02-28 15:28:41 +00:00
Fabien O'Carroll
ec697051dc 🐛 Fixed cache invalidation header race conditions
refs https://linear.app/tryghost/issue/ENG-674/

This ensures that all of our dynamic cache invalidation header logic
is applied on a per-request basis!
2024-02-28 21:31:04 +07:00
Fabien 'egg' O'Carroll
5fae212416
Ensured comment counts route doesn't load member (#19762)
refs https://linear.app/tryghost/issue/ENG-672/

The comment counts endpoint does not need member authentication.
This saves us a bunch of db queries for each request
2024-02-28 07:36:35 +07:00
Fabien 'egg' O'Carroll
7f392c305b
Added output to the get helper when the timeout is exceeded (#19761)
refs https://linear.app/tryghost/issue/ENG-670

We keep running into issues with a sites content not being correct,
and slow get helpers being the suspect - but it's difficult to prove.
The idea behind this it to give us concrete evidence, which will allow
us to diagnose the problem faster.
2024-02-28 07:35:16 +07:00
renovate[bot]
0c5f1daf96 Update dependency newrelic to v11.11.0 2024-02-27 22:55:16 +00:00
Kevin Ansfield
f3e36bcd4e
🐛 Fixed extra whitespace in plaintext transactional member emails (#19736)
closes ENG-660

- added tagged template function to strip leading whitespace from our plaintext email strings without making the source file harder to read
2024-02-27 16:24:38 +00:00
Ghost CI
536f67398c Merged v5.79.6 into main 2024-02-26 17:18:43 +00:00
Ghost CI
3b0d0934eb v5.79.6 2024-02-26 17:18:41 +00:00
Kevin Ansfield
ab7c1cfa92
Fixed incorrect cache invalidation headers for slugs Admin API endpoint (#19753)
closes ENG-666

- the Admin API `GET /slugs/{type}/{slug}/` endpoint is used by Admin to check when a potential slug needs de-duping by adding a `-{x}` suffix. Most often this occurs when setting a draft post title
- the endpoint was returning a full-site cache invalidation header meaning hosting services could be blowing away their site caches and needlessly hurting performance because this endpoint is purely a read operation and makes no changes to the site
- updated the endpoint to return no cache invalidation header
2024-02-26 17:02:09 +00:00
Kevin Ansfield
ca9c0a4055
Fixed incorrect cache invalidation headers for slugs Admin API endpoint (#19753)
closes ENG-666

- the Admin API `GET /slugs/{type}/{slug}/` endpoint is used by Admin to check when a potential slug needs de-duping by adding a `-{x}` suffix. Most often this occurs when setting a draft post title
- the endpoint was returning a full-site cache invalidation header meaning hosting services could be blowing away their site caches and needlessly hurting performance because this endpoint is purely a read operation and makes no changes to the site
- updated the endpoint to return no cache invalidation header
2024-02-26 16:59:29 +00:00
renovate[bot]
65a9a04959 Update dependency mysql2 to v3.9.2 2024-02-26 16:25:50 +00:00
Ghost CI
3a0fd45958 v5.79.5 2024-02-26 06:26:13 +00:00
renovate[bot]
49fc876fe1 Update dependency sanitize-html to v2.12.1 2024-02-22 19:17:35 +00:00
Steve Larson
02edc5ad4f
Updated browser tests to be less flaky (#19701)
no refs

- Offers browser tests were subject to a race condition. I'm guessing
this dates back to when we moved to Settings X (and React), as it seems
the url for the offer is not present on the first render of the page -
despite being returned in the `POST` request of the offer creation, the
component does a `GET` on render to get the link. This is now awaited.
- The Publishing timezone test also seemed to suffer from a race
condition. This is less sure of a fix as it's a much less frequent
failure. The date time picker input is now validated in the test before
continuing.
- Offers browser tests often timed out so the timeout has been moved to
90s for these tests.
- All tests were bumped to 75s timeout as we generally would
occasionally hit the timeout.
2024-02-21 17:47:44 +00:00
renovate[bot]
bcbd3fbcc8 Update dependency sanitize-html to v2.12.0 2024-02-21 16:47:38 +00:00
Kevin Ansfield
a3f8aa110d
🐛 Fixed explicit HTML entities being decoded when rendering HTML cards (#19728)
closes ENG-608

- bumps Koenig rendering packages to include fix for HTML entities in HTML card content being decoded during rendering which could result in unexpected/broken output
2024-02-21 16:36:04 +00:00
Sag
034df3714e
Removed leftover alpha flag (#19717)
no issue
2024-02-20 14:07:03 +01:00
Sag
ba3ea9b36c
Fixed browser tests (#19719)
no issue
2024-02-20 14:03:36 +01:00
Sag
e9f5af110c
🎨 Added option to change the name of the free tier (#19715)
ref ENG-607

- also added the option to show the monthly pricing by default during
signup

Co-authored-by: Simon Backx <simon@ghost.org>
Co-authored-by: Djordje Vlaisavljevic <dzvlais@gmail.com>
2024-02-20 12:54:41 +01:00
Ghost CI
98b5992f63 v5.79.4 2024-02-19 11:26:01 +00:00
Ghost CI
0c4d62880e v5.79.3 2024-02-16 18:20:06 +00:00
Ghost CI
49bf6d2754 v5.79.2 2024-02-16 16:06:40 +00:00
renovate[bot]
c0325da0e7 Update dependency newrelic to v11.10.4 2024-02-13 21:27:42 +00:00
renovate[bot]
cd8b5f3ab3 Update dependency yjs to v13.6.12 2024-02-12 02:16:00 +00:00
Ghost CI
300563eb95 v5.79.1 2024-02-09 16:04:10 +00:00
renovate[bot]
46866788dd Update dependency newrelic to v11.10.3 2024-02-07 20:03:13 +00:00
Ronald Langeveld
b460dabf68
Revert "Added headers if making a preview site request (#19668)" (#19669) 2024-02-07 16:54:27 +02:00
Ronald Langeveld
d3e16bb885
Added headers if making a preview site request (#19668)
no issue

- to test if we can access Private Sites in Admin when set as a private
site.
- the issue is, we have CORS issues that doesn't allow a cookie to be
passed via Admin when the site uses a custom domain.
- generally does not affect self hosters.
2024-02-07 11:37:59 +00:00
Steve Larson
5f371027a3
Updated editor lexical packages (#19664)
no refs
- updated koenig packages with the newest lexical version to bring in bugfixes and improvements
2024-02-06 22:13:54 +00:00
Ghost CI
c6c66d2a20 v5.79.0 2024-02-02 16:04:15 +00:00
Steve Larson
2c166582fd
Added config option to disable db backups (#19614)
refs https://linear.app/tryghost/issue/ENG-600
- users need an option so they can perform actions like delete users
without blowing up Ghost as large dbs can OOM node
2024-02-01 12:09:41 -06:00
Sag
c12e279e0c
Added migration to fix data discrepancy in free tier visibility (#19624)
refs INC-18

- release v5.69.0 introduced a data discrepancy in the free tier
visibility: the "free" tier visibility got out of sync with the
"portal_plans" setting due to a bug in the new Admin settings. The bug
was corrected in a patch release rolled out a few days later, v5.69.4
- however, the data discrepancy has not been corrected for all
customers; this data migration fixes the data discrepancy
2024-02-01 16:53:40 +01:00
renovate[bot]
1bd7dea79d Update dependency newrelic to v11.10.2 2024-01-31 19:53:07 +00:00
Steve Larson
1ed34aebac
Added check for Stripe Pass in Stripe Connect test utils (#19633)
refs https://linear.app/tryghost/issue/ENG-599
- Portal tests occasionally failed without clear cause on CI, possibly
due to GH runner region
- Portal tests never successfully ran locally for US-based IPs because
of a required prompt for Stripe Pass
2024-01-31 11:52:09 -06:00
Fabien 'egg' O'Carroll
38b29d0566
Fixed boot crashing when segment config is present (#19625)
refs https://github.com/TryGhost/Ghost/commit/c4912665e5d5af2c25e

We removed the segment service but continued to attempt to load it when
the segment config was present.
2024-01-31 13:01:34 +07:00
Steve Larson
d5077ac1bf
Cleared member count cache on manual member add/delete (#19623)
refs https://linear.app/tryghost/issue/ENG-599
- member count is based on the cache which only updates ~every minute
- forced cache clear on manual member add/delete (not import)
- tests were failing based on the assumption that a new site that adds a
member has a nonzero member count, although the cache did not reflect
this quickly enough for the test to pass

Previously on a new site if you tried to publish a newsletter, it would
require at least one member. If you quickly added a member and tried to
send a newsletter, it would stop you saying you need at least one
member, requiring a browser refresh. This was a bug that is resolved
with this changes, as well as odd behaviour to try to write tests
around.
2024-01-30 15:08:27 -06:00
Daniel Lockyer
f76bb91122 Updated @elastic/elasticsearch to latest version
refs https://github.com/TryGhost/Toolbox/issues/501

- at this point, we have no real reason to keep this behind as it wasn't
  proven what the cause of the high CPU was, and it's just causing more
  lockfile issues with the resolution
2024-01-30 21:41:33 +01:00
Aileen Booker
c4912665e5 Removed DomainEventsAnalytics
closes CA-11

- Segment events in Ghost core are not used currently
2024-01-30 13:58:22 -04:00
Kevin Ansfield
3c56005d44
🐛 Fixed error when converting or pasting HTML content with headings+text nested inside lists (#19618)
closes https://github.com/TryGhost/Product/issues/4234

- bumps Koenig packages to version containing a fix to our denest transform so it properly handles denesting element nodes inside list item nodes
2024-01-30 13:45:58 +00:00
Ghost CI
5d19c75482 v5.78.0 2024-01-30 11:17:31 +00:00
Simon Backx
7d0be3f1a9
Improved sending email addresses for self-hosters (#19617)
ref https://github.com/TryGhost/Ghost/issues/12802
fixes DMA-27

- You can choose any support and newsletter email address in the UI
without verification (as long as your SMTP-server / Mailgun can send
from it)
- All emails will use the mail.from config as the from address as a
default:
- Staff notification emails no longer use the made up ghost@domain email
address
    - Newsletters no longer default to 'noreply@domain' 
- Member related emails (signin/signup/comment notifications...) will
continue to be send from the chosen support address (Portal settings →
Account page), but will now default to the mail.from config instead of
noreply@domain if no support address is set.
2024-01-30 11:21:08 +01:00
Steve Larson
68dda65a12
Added refresh to publishing test (#19612)
refs ENG-599
- added refresh to publishing workflow test
- member count is cached and not updated immediately upon adding a
member, but a count >0 is required in order to send a newsletter (what
this test tests)
- we are looking at updating the cached count; until then, a refresh
will be a performance hit but allow this test to pass
2024-01-29 14:15:56 -06:00
Chris Raible
975bb6849f
Renamed performanceMonitoring config to telemetry (#19613)
no issue

- Renaming the configuration parameter created in this commit:
e0dae46dfc
- No functional difference, this change just makes the configuration a
bit more succinct
2024-01-29 11:56:17 -08:00
Chris Raible
e0dae46dfc
Added basic instrumentation to the database connection pool (#19589)
no issue

- To help debug potential causes of slow/aborted get helpers, it would
be cool to get more visibility into how Ghost handles database
connections, particularly if it has to spend a long time waiting to
acquire a new connection from the pool.
- Under the hood, knex uses a package called tarn
(https://github.com/Vincit/tarn.js/tree/3.0.2) to manage the connection
pool. Tarn provides some hooks for instrumentation, so we can use those
to get some basic visibility into the connection pool.
- This PR adds handling for creating, acquiring and releasing
connections from Tarn's connection pool which logs some basic metrics,
particularly the queue length and time it takes to acquire a connection.
2024-01-29 10:25:35 -08:00
Ghost CI
1d4b076670 v5.77.0 2024-01-29 15:36:59 +00:00
renovate[bot]
e45eb3e222 Update dependency mysql2 to v3.9.1 2024-01-29 13:06:38 +00:00
renovate[bot]
b2712065df Update dependency express-session to v1.18.0 2024-01-29 01:10:26 +00:00
Ronald Langeveld
b490534983
Fixed flaky portal tests (#19596)
no issue
2024-01-28 23:04:39 +04:00
Sag
1e988cccff
Revert Portal changes (#19594)
no issue

- This revert recent changes made to Portal, that may be causing the
Free tier to not render properly on Ghost sites
2024-01-27 01:46:59 +01:00
Sag
24952ab3df
🎨 Improved Portal and Portal settings (#19584)
no issue

---------

Co-authored-by: Simon Backx <simon@ghost.org>
Co-authored-by: Djordje Vlaisavljevic <dzvlais@gmail.com>
2024-01-26 10:38:00 +01:00
renovate[bot]
07dbcb0715 Update dependency mysql2 to v3.9.0 2024-01-26 08:38:20 +00:00
renovate[bot]
8b36aa03d1 Update dependency newrelic to v11.10.1 2024-01-25 22:04:39 +00:00
Ronald Langeveld
0c95111f8e
🎨 Admin X - Offers (#19520)
no issue

- Offers rebuilt in React and now located in Settings.
2024-01-25 12:41:54 +00:00
Chris Raible
794ef85968
Added Sentry instrumentation for get helpers (#19576)
no issue

- To help debug ABORTED_GET_HELPER errors, this PR adds Sentry
instrumentation to the get helpers
- It also adds the homepage, any pages/posts, the tag page, and the
author page to the list of transactions that will send to Sentry
2024-01-24 18:50:48 -08:00
Ghost CI
5e0bcc5a38 Merged v5.76.2 into main 2024-01-24 17:37:40 +00:00
Ghost CI
922af6defe v5.76.2 2024-01-24 17:37:39 +00:00
Steve Larson
f9adc59774 🐛 Fixed custom excerpts sometimes being cut off (#19560)
refs TryGhost/Ghost#19559
- custom excerpts are truncated based on character length
- escaped characters added extra length but we didn't account for this,
resulting in poor truncation of excerpts
2024-01-24 10:55:49 -06:00
renovate[bot]
f22e0eb2dd Update dependency cookie-session to v2.1.0 2024-01-24 05:11:56 +00:00
renovate[bot]
b0a9d3541e Update dependency mysql2 to v3.8.0 2024-01-23 20:56:42 +00:00
Steve Larson
40891272dc
🐛 Fixed custom excerpts sometimes being cut off (#19560)
refs TryGhost/Ghost#19559
- custom excerpts are truncated based on character length
- escaped characters added extra length but we didn't account for this,
resulting in poor truncation of excerpts
2024-01-23 14:45:27 -06:00
Sag
5469e76852
Fixed reply-to address to stay the same after dmarc changes (#19542)
fixes PROD-102
- after dmarc changes, replies from members should keep going to any previously set
reply-to email address by the publisher
2024-01-23 16:22:40 +01:00
Simon Backx
eb063f7a40
Fixed clearing invalid sender_email when changing newsletter sender_reply_to (#19555)
fixes PROD-102

When a newsletter has a sender_email stored in the database that Ghost
is not allowed to send from, we no longer return it as sender_email in
the API. Instead we return it as the sender_reply_to. That way the
expected behaviour is shown correctly in the frontend and the API result
also makes more sense.

In addition to that, when a change is made to a newsletters reply_to
address we'll clear any invalid sender_email values in that newsletter.
That makes sure we can clear the sender_reply_to value instead of
keeping the current fallback to sender_email if that one is stored.

On top of that, this change correclty updates the browse endpoint to use
the newsletter service instead of directly using the model.
2024-01-23 16:10:11 +01:00
Michael Barrett
57810cd34e
Added allowlist for Sentry transactions (#19538)
refs
[ARCH-41](https://linear.app/tryghost/issue/ARCH-41/add-allowlist-for-sentry-transactions)

Added allowlist for Sentry transactions so that we can better control
the data we are putting into Sentry
2024-01-23 08:22:57 +00:00
renovate[bot]
aa5cd13aec Update dependency newrelic to v11.10.0 2024-01-22 22:21:29 +00:00
Kevin Ansfield
15897096b0
🐛 Fixed broken access to preview of scheduled email-only posts (#19539)
no issue

- we recently added a redirect to disable access to the preview endpoint for sent email-only posts but the condition was too broad and also disabled access to scheduled email-only posts
- adjusted so we only apply the /p/ -> /email/ redirect for sent posts
2024-01-22 14:20:50 +00:00
Ghost CI
f4e20ad247 Merged v5.76.1 into main 2024-01-22 09:00:44 +00:00
Ghost CI
5a630b6aa4 v5.76.1 2024-01-22 09:00:42 +00:00
Chris Raible
d1f9dab1d5
🐛 Fixed externally hosted images overflowing in Outlook (#19527)
refs TryGhost/Product#4243

- Externally hosted images added in the editor were not populating the
`width` and `height` attributes, which could result in overflowing
images in certain email clients, particularly Outlook.
- This fix populates the `width` and `height` attributes in the editor
when adding an external image by URL or copy/pasting, which in turn
corrects the rendering in Outlook.
- Various other fixes and improvements to editor related packages, see
https://github.com/tryghost/koenig repo for more info
2024-01-22 09:44:05 +01:00
Djordje Vlaisavljevic
8511fbbdae
Updated design for sender and reply-to email address flow PROD-215 PROD-216
refs PROD-215 PROD-216

- Added toast notifications for successful sender and reply-to email
address change behind the flag, instead of the modal
- Updated email template for verifying new sender or reply-to email
2024-01-22 09:43:48 +01:00
renovate[bot]
c9d571354f
🐛 Fixed rare rendering issue of lists appearing as headings (#19511)
closes https://github.com/TryGhost/Product/issues/4247

- bumps `@tryghost/kg-default-transforms` with a fix to our de-nesting transform so ListNode is no longer ignored as a badly nested child node which can occur through copy/paste from other editors

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-01-22 09:43:05 +01:00
renovate[bot]
8168fdb9be Update dependency yjs to v13.6.11 2024-01-22 00:22:10 +00:00
Chris Raible
21fc34d088
🐛 Fixed externally hosted images overflowing in Outlook (#19527)
refs TryGhost/Product#4243

- Externally hosted images added in the editor were not populating the
`width` and `height` attributes, which could result in overflowing
images in certain email clients, particularly Outlook.
- This fix populates the `width` and `height` attributes in the editor
when adding an external image by URL or copy/pasting, which in turn
corrects the rendering in Outlook.
- Various other fixes and improvements to editor related packages, see
https://github.com/tryghost/koenig repo for more info
2024-01-18 12:38:09 -08:00
Djordje Vlaisavljevic
501b1a2640
Updated design for sender and reply-to email address flow PROD-215 PROD-216
refs PROD-215 PROD-216

- Added toast notifications for successful sender and reply-to email
address change behind the flag, instead of the modal
- Updated email template for verifying new sender or reply-to email
2024-01-18 15:21:48 +00:00
Kevin Ansfield
0c5cdbf4d2
🐛 Fixed embed service trying http before https for oembed providers (#19521)
no issue

- issue reported via the forum https://forum.ghost.org/t/video-embed-break-page-on-mobile/44172
- due to historical issues we check against http/https and non-www/www URLs to match an oembed provider in case our library's provider list is out of date. However we checked http first which could match and then update the original URL to be `http` in place of `https` leading to potentially broken oembed fetch requests as was the case with http://odysee.com URLs
2024-01-18 14:42:28 +00:00
Aileen Booker
75874151fd Removed ModelEventsAnalytics
refs https://linear.app/tryghost/issue/BIZ-6/[wip]-update-segment-events

- With the removal of the `integration.added` event, we have no more model events remaining to listen to for our analytics
- Removal of the function entirely seems the easier and more straightforward way
2024-01-18 10:29:56 -04:00
Aileen Booker
e4b9305e2a Removed unneeded analytics for model events
refs https://linear.app/tryghost/issue/BIZ-6/[wip]-update-segment-events

- Removed model events to listen to: `post.published`, `page.published`, and `theme.uploaded` in segment service,  as we're not actively using those.
- Updated tests to reflect the changes (from 4 events to 1 model event)
2024-01-18 10:29:56 -04:00
Simon Backx
b30558c77c
Added cache config to stats endpoints (#19481)
no issue

Allows to enable cache via hostSettings.statsCache.enabled. This will
need proper cache timeouts in order to function correctly.

Usage in config:
```
"hostSettings": {
        "statsCache": {
            "enabled": true
        }
    },
    "adapters": {
        "cache": {
            "Redis": {
                "host": "127.0.0.1",
                "port": 6379,
                "username": "",
                "password": "",
                "ttl": 60,
                "storeConfig": {
                    "maxRetriesPerRequest": 1,
                    "enableOfflineQueue": false,
                    "retryConnectSeconds": 60
                }
            },
            "stats": {
                "adapter": "Redis",
                "ttl": 3600,
                "refreshAheadFactor": 1,
                "keyPrefix": "site:123456:stats"
            }
        }
    },
    ```
2024-01-18 15:26:49 +01:00
Daniel Lockyer
57c5f92770 Reverted "🎨 Added Offers to the new Settings (#19493)"
This reverts commit c7d7b883cc.
2024-01-18 15:04:59 +01:00
Fabien "egg" O'Carroll
c60dd779c9 Removed usage of EventAwareCacheAdapter
This logic is so simple it isn't worth having the indirection of another class.

This also removes the indirection of wrapped getters/setters, which is useful
because otherwise we need to update the wrapper with new methods each time
theunderlying implementation is changed. There was a note about losing the
context of this, but I haven't found anywhere that the context is lost.
2024-01-18 20:16:36 +07:00
Ronald Langeveld
c7d7b883cc
🎨 Added Offers to the new Settings (#19493)
no issue

- Removes flags for the new Offers in Admin X (Settings)
- Removes old Offers from the sidebar.
- See a new version of Offers in Settings. 🎨
2024-01-18 12:56:08 +00:00
renovate[bot]
89a24c3e8b
🐛 Fixed rare rendering issue of lists appearing as headings (#19511)
closes https://github.com/TryGhost/Product/issues/4247

- bumps `@tryghost/kg-default-transforms` with a fix to our de-nesting transform so ListNode is no longer ignored as a badly nested child node which can occur through copy/paste from other editors

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-01-18 07:19:38 +00:00
renovate[bot]
ecd54b1a63 Update dependency mysql2 to v3.7.1 2024-01-18 00:28:28 +00:00
Simon Backx
a60704c588
Revert "Added support for "Refresh Ahead" caching strategy" (#19502)
Reverts TryGhost/Ghost#19499
2024-01-17 13:12:58 +00:00
Fabien 'egg' O'Carroll
aaaa3ba797
Added support for "Refresh Ahead" caching strategy (#19499)
The main changes are:
- Updating the pipeline to allow for doing a background refresh of the
cache
- Remove the use of the EventAwareCacheWrapper for the posts public
cache

### Background refresh

This is just an initial implementation, and tbh it doesn't sit right
with me that the logic for this is in the pipeline - I think this should
sit in the cache implementation itself, and then we call out to it with
something like: `cache.get(key, fetchData)` and then the updates can
happen internally.

The `cache-manager` project actually has a method like this called
`wrap` - but every time I've used it it hangs, and debugging was a pain,
so I don't really trust it.

### EventAwareCacheWrapper

This is such a small amount of logic, I don't think it's worth creating
an entire wrapper for it, at least not a class based one. I would be
happy to refactor this to use a `Proxy` too, so that we don't have to
add methods to it each time we wanna change the underlying cache
implementation.
2024-01-17 14:00:24 +01:00
Ghost CI
3b0f99d455 v5.76.0 2024-01-17 09:16:09 +00:00
Kevin Ansfield
100e7b70c6
Added TK Reminders feature (#19491)
no issue

- keep an eye on on https://ghost.org/changelog/ for full details
2024-01-17 08:57:35 +00:00
renovate[bot]
c37642a67b Update dependency json-stable-stringify to v1.1.1 2024-01-16 23:05:24 +00:00