Commit Graph

2863 Commits

Author SHA1 Message Date
Djordje Vlaisavljevic
8511fbbdae
Updated design for sender and reply-to email address flow PROD-215 PROD-216
refs PROD-215 PROD-216

- Added toast notifications for successful sender and reply-to email
address change behind the flag, instead of the modal
- Updated email template for verifying new sender or reply-to email
2024-01-22 09:43:48 +01:00
renovate[bot]
c9d571354f
🐛 Fixed rare rendering issue of lists appearing as headings (#19511)
closes https://github.com/TryGhost/Product/issues/4247

- bumps `@tryghost/kg-default-transforms` with a fix to our de-nesting transform so ListNode is no longer ignored as a badly nested child node which can occur through copy/paste from other editors

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-01-22 09:43:05 +01:00
renovate[bot]
8168fdb9be Update dependency yjs to v13.6.11 2024-01-22 00:22:10 +00:00
Chris Raible
21fc34d088
🐛 Fixed externally hosted images overflowing in Outlook (#19527)
refs TryGhost/Product#4243

- Externally hosted images added in the editor were not populating the
`width` and `height` attributes, which could result in overflowing
images in certain email clients, particularly Outlook.
- This fix populates the `width` and `height` attributes in the editor
when adding an external image by URL or copy/pasting, which in turn
corrects the rendering in Outlook.
- Various other fixes and improvements to editor related packages, see
https://github.com/tryghost/koenig repo for more info
2024-01-18 12:38:09 -08:00
Djordje Vlaisavljevic
501b1a2640
Updated design for sender and reply-to email address flow PROD-215 PROD-216
refs PROD-215 PROD-216

- Added toast notifications for successful sender and reply-to email
address change behind the flag, instead of the modal
- Updated email template for verifying new sender or reply-to email
2024-01-18 15:21:48 +00:00
Kevin Ansfield
0c5cdbf4d2
🐛 Fixed embed service trying http before https for oembed providers (#19521)
no issue

- issue reported via the forum https://forum.ghost.org/t/video-embed-break-page-on-mobile/44172
- due to historical issues we check against http/https and non-www/www URLs to match an oembed provider in case our library's provider list is out of date. However we checked http first which could match and then update the original URL to be `http` in place of `https` leading to potentially broken oembed fetch requests as was the case with http://odysee.com URLs
2024-01-18 14:42:28 +00:00
Aileen Booker
75874151fd Removed ModelEventsAnalytics
refs https://linear.app/tryghost/issue/BIZ-6/[wip]-update-segment-events

- With the removal of the `integration.added` event, we have no more model events remaining to listen to for our analytics
- Removal of the function entirely seems the easier and more straightforward way
2024-01-18 10:29:56 -04:00
Aileen Booker
e4b9305e2a Removed unneeded analytics for model events
refs https://linear.app/tryghost/issue/BIZ-6/[wip]-update-segment-events

- Removed model events to listen to: `post.published`, `page.published`, and `theme.uploaded` in segment service,  as we're not actively using those.
- Updated tests to reflect the changes (from 4 events to 1 model event)
2024-01-18 10:29:56 -04:00
Simon Backx
b30558c77c
Added cache config to stats endpoints (#19481)
no issue

Allows to enable cache via hostSettings.statsCache.enabled. This will
need proper cache timeouts in order to function correctly.

Usage in config:
```
"hostSettings": {
        "statsCache": {
            "enabled": true
        }
    },
    "adapters": {
        "cache": {
            "Redis": {
                "host": "127.0.0.1",
                "port": 6379,
                "username": "",
                "password": "",
                "ttl": 60,
                "storeConfig": {
                    "maxRetriesPerRequest": 1,
                    "enableOfflineQueue": false,
                    "retryConnectSeconds": 60
                }
            },
            "stats": {
                "adapter": "Redis",
                "ttl": 3600,
                "refreshAheadFactor": 1,
                "keyPrefix": "site:123456:stats"
            }
        }
    },
    ```
2024-01-18 15:26:49 +01:00
Daniel Lockyer
57c5f92770 Reverted "🎨 Added Offers to the new Settings (#19493)"
This reverts commit c7d7b883cc.
2024-01-18 15:04:59 +01:00
Fabien "egg" O'Carroll
c60dd779c9 Removed usage of EventAwareCacheAdapter
This logic is so simple it isn't worth having the indirection of another class.

This also removes the indirection of wrapped getters/setters, which is useful
because otherwise we need to update the wrapper with new methods each time
theunderlying implementation is changed. There was a note about losing the
context of this, but I haven't found anywhere that the context is lost.
2024-01-18 20:16:36 +07:00
Ronald Langeveld
c7d7b883cc
🎨 Added Offers to the new Settings (#19493)
no issue

- Removes flags for the new Offers in Admin X (Settings)
- Removes old Offers from the sidebar.
- See a new version of Offers in Settings. 🎨
2024-01-18 12:56:08 +00:00
renovate[bot]
89a24c3e8b
🐛 Fixed rare rendering issue of lists appearing as headings (#19511)
closes https://github.com/TryGhost/Product/issues/4247

- bumps `@tryghost/kg-default-transforms` with a fix to our de-nesting transform so ListNode is no longer ignored as a badly nested child node which can occur through copy/paste from other editors

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2024-01-18 07:19:38 +00:00
renovate[bot]
ecd54b1a63 Update dependency mysql2 to v3.7.1 2024-01-18 00:28:28 +00:00
Simon Backx
a60704c588
Revert "Added support for "Refresh Ahead" caching strategy" (#19502)
Reverts TryGhost/Ghost#19499
2024-01-17 13:12:58 +00:00
Fabien 'egg' O'Carroll
aaaa3ba797
Added support for "Refresh Ahead" caching strategy (#19499)
The main changes are:
- Updating the pipeline to allow for doing a background refresh of the
cache
- Remove the use of the EventAwareCacheWrapper for the posts public
cache

### Background refresh

This is just an initial implementation, and tbh it doesn't sit right
with me that the logic for this is in the pipeline - I think this should
sit in the cache implementation itself, and then we call out to it with
something like: `cache.get(key, fetchData)` and then the updates can
happen internally.

The `cache-manager` project actually has a method like this called
`wrap` - but every time I've used it it hangs, and debugging was a pain,
so I don't really trust it.

### EventAwareCacheWrapper

This is such a small amount of logic, I don't think it's worth creating
an entire wrapper for it, at least not a class based one. I would be
happy to refactor this to use a `Proxy` too, so that we don't have to
add methods to it each time we wanna change the underlying cache
implementation.
2024-01-17 14:00:24 +01:00
Ghost CI
3b0f99d455 v5.76.0 2024-01-17 09:16:09 +00:00
Kevin Ansfield
100e7b70c6
Added TK Reminders feature (#19491)
no issue

- keep an eye on on https://ghost.org/changelog/ for full details
2024-01-17 08:57:35 +00:00
renovate[bot]
c37642a67b Update dependency json-stable-stringify to v1.1.1 2024-01-16 23:05:24 +00:00
Kevin Ansfield
f88fdfe363
Increased test timeout for HTML transform unit tests (#19490)
closes https://github.com/TryGhost/Product/issues/4086

- JSDOM require on CI has been found to occasionally be very slow causing random timeouts
- doubled test time to eliminate the noise
2024-01-16 11:16:46 +00:00
Simon Backx
709a0cf3c4
🐛 Fixed error logging crash when email recipients count if off by 1% (#19485)
no issue

When creating the batches when sending an email, we log a message to
Sentry when there is an unexpected offset of 1% between creating the
email and actually creating the batch recipients. We used a method that
was not mapped in our Sentry proxy.

Location of error: ghost/email-service/lib/BatchSendingService.js:286
2024-01-15 16:21:11 +01:00
Michael Barrett
ed0762fb51
Removed usage of yg when using NQL (#19287)
refs https://github.com/TryGhost/NQL/pull/73

The referenced PR removes `yg` from the parsed NQL output, so we also
need to remove any usage of it in Ghost
2024-01-15 14:40:01 +00:00
renovate[bot]
524f73c545 Update dependency socket.io to v4.7.4 2024-01-14 20:54:29 +00:00
Daniel Lockyer
be6b9e437f Refactored fetching schema tables in data generator
- we want to pass in the schema tables instead of cross requiring them
  from a different package because it means the package isn't standalone
  and moving the code structure around breaks the data generator
2024-01-13 18:28:14 +01:00
renovate[bot]
09921fd2b4 Update dependency @sentry/profiling-node to v1.3.5 2024-01-12 18:05:56 +00:00
renovate[bot]
84c2fe9051 Update dependency @sentry/profiling-node to v1.3.4 2024-01-11 17:05:58 +00:00
renovate[bot]
78e2c10d3c Update dependency newrelic to v11.9.0 2024-01-10 22:17:49 +00:00
renovate[bot]
3f9598d14c Update dependency @sentry/profiling-node to v1.3.3 2024-01-08 21:57:00 +00:00
renovate[bot]
0b0c177952 Update dependency mysql2 to v3.7.0 2024-01-08 01:45:46 +00:00
Daniel Lockyer
18599fb9ce
Merged v5.75.3 into main
v5.75.3
2024-01-05 15:24:59 +01:00
Ghost CI
f21f025659 v5.75.3 2024-01-05 13:58:49 +00:00
Simon Backx
d2cb23c3fa
Wired up Docker setup script and increased data generation performance (#19420)
ref PROD-233

- Stored whether Docker is used in the config files
- When running `yarn setup`, any existing Docker container will be
reset. Run with `-y` to skip the confirmation step.
- `yarn setup` will always init the database and generate fake data
- Increased amount of default generated data to 100,000 members + 500
posts.
- Made lots of performance improvements in the data generator so we can
generate the default data in ±170s
2024-01-05 13:42:30 +00:00
Daniel Lockyer
d2ab091599
Updated snapshots for the new year
- these shouldn't be hardcoded but it changes so infrequently that we
  can just update the snapshots for now and then fix it at a later point
2024-01-05 13:31:07 +00:00
Daniel Lockyer
a667b160c1
Lazyloaded @sentry/profiling-node dependency
- we don't need to load this if we haven't configured Node profiling to occur
- this might help fix random segfaults we've been seeing in CI, which
  only started occurring after this dependency was added
2024-01-05 13:30:21 +00:00
Michael Barrett
b639993a1b
Removed Sentry Express integration (#19443)
no refs

Removed Sentry Express integration as it is not compatible with Ghost's
use of Express
2024-01-05 13:02:29 +00:00
Michael Barrett
06a413c807
Updated Sentry env to use PRO_ENV when available (#19441)
refs
[ARCH-33](https://linear.app/tryghost/issue/ARCH-33/fix-sentry-environment)

To ensure that we are correctly identifying the environment that data is
being sent to Sentry from, we can use the `PRO_ENV` environment variable
if it is available. This will be set to `production` in production and
`staging` in staging. If `PRO_ENV` is not available, we will fall back
to retrieving the environment from config (`env`)
2024-01-05 13:02:29 +00:00
Chris Raible
0feebfcf63
Added Sentry Profiling to Ghost server (#19319)
refs ARCH-29

- Added Sentry Profiling to collect more detailed performance data on
the backend.
- This feature is opt-in behind a config. To enable profiling, first
enable tracing with `sentry.tracing.enabled: true`, then set
`sentry.profiling.enabled: true` and `sentry.profiling.sampleRate` to a
decimal number between 0 and 1.
2024-01-05 12:57:18 +00:00
Michael Barrett
bd6bfe13c0
Added custom Sentry integration for Knex.js (#19315)
no refs

Added custom Sentry integration for Knex.js to trace database queries in
Sentry
2024-01-05 12:54:26 +00:00
Chris Raible
f91d046f5e
Added Sentry performance monitoring to Ghost Server (#19243)
refs ARCH-21

- We currently have NewRelic setup for a few of our largest customers
for monitoring performance, but it is too expensive to enable across all
sites
- Sentry has similar (but simpler) performance monitoring tools to keep
track of response times that are available to us for free, but we just
haven't configured them
- This PR sets up Sentry Performance monitoring for API requests so we
can have one place for monitoring errors + performance so we can stay on
top of response times more easily.
- Tracing is disabled by default, so there is no additional overhead
unless `sentry.tracing.enabled` is set to `true` in the site's config.
Additionally, `sentry.tracing.sampleRate` should be set to a decimal
value between 0 and 1. This value defaults to 0 to avoid accidentally
blowing through quota, and requires a value to explicitly be set in
order to send the traces to Sentry.
2024-01-05 12:53:36 +00:00
Michael Barrett
1263cf148e
Updated Sentry env to use PRO_ENV when available (#19441)
refs
[ARCH-33](https://linear.app/tryghost/issue/ARCH-33/fix-sentry-environment)

To ensure that we are correctly identifying the environment that data is
being sent to Sentry from, we can use the `PRO_ENV` environment variable
if it is available. This will be set to `production` in production and
`staging` in staging. If `PRO_ENV` is not available, we will fall back
to retrieving the environment from config (`env`)
2024-01-05 12:10:39 +00:00
renovate[bot]
1fa2a11cbc Update dependency knex-migrator to v5.1.7 2024-01-05 12:32:45 +01:00
renovate[bot]
85f3ef3d14 Update dependency postcss to v8.4.33 2024-01-05 11:05:49 +01:00
renovate[bot]
ce5466d017 Update dependency sqlite3 to v5.1.7 2024-01-05 11:05:15 +01:00
Michael Barrett
2d28dbe2fd
Removed Sentry Express integration (#19443)
no refs

Removed Sentry Express integration as it is not compatible with Ghost's
use of Express
2024-01-04 14:31:57 +00:00
renovate[bot]
dc45d5285a Update dependency cssnano to v6.0.3 2024-01-04 10:44:39 +01:00
Sag
1f5a42d34c
Added webmentions endpoint to robots.txt disallow (#19433)
fixes PROD-290

- in order to receive webmentions (e.g. recommendations), Ghost sites
expose a /webmentions/receive endpoint. This endpoint is wrongly being
indexed by Google as a regular page, and causes indexing errors in
Google Search Console
2024-01-03 17:30:37 +00:00
renovate[bot]
668e51e631 Update dependency newrelic to v11.8.0 2024-01-03 16:20:33 +00:00
renovate[bot]
56fd992a1f Update dependency image-size to v1.1.1 2024-01-02 17:41:16 +00:00
renovate[bot]
9e2558931f
🐛 Fixed signup card background color and editor crash when typing :, or :| (#19421)
refs https://github.com/TryGhost/Ghost/issues/19282
refs https://github.com/TryGhost/Koenig/pull/1136

- fixes signup card background color
- fixes crash when typing `:,` or `:|` or similar
2024-01-02 17:26:55 +00:00
renovate[bot]
43dbc4ca89 Update dependency cssnano to v6.0.2 2024-01-02 09:46:50 +01:00
renovate[bot]
e90e403aca Update dependency ws to v8.16.0 2024-01-02 08:39:49 +00:00
Daniel Lockyer
a86bf46347 Updated snapshots for the new year
- these shouldn't be hardcoded but it changes so infrequently that we
  can just update the snapshots for now and then fix it at a later point
2024-01-02 09:07:55 +01:00
Daniel Lockyer
d21ab1aa4e Lazyloaded @sentry/profiling-node dependency
- we don't need to load this if we haven't configured Node profiling to occur
- this might help fix random segfaults we've been seeing in CI, which
  only started occurring after this dependency was added
2024-01-02 09:07:55 +01:00
renovate[bot]
9f2365209d Update dependency image-size to v1.1.0 2023-12-28 15:32:30 +00:00
renovate[bot]
bce90d5337 Update dependency newrelic to v11.7.0 2023-12-14 22:01:01 +00:00
Daniel Lockyer
47f50e2d35
Merged v5.75.2 into main
v5.75.2
2023-12-14 13:51:23 +01:00
Ghost CI
c2ad349b78 v5.75.2 2023-12-14 12:34:56 +00:00
Sanne de Vries
255d1b1740
Added site url link to newsletter header image (#19380)
No ref
2023-12-14 12:08:10 +00:00
Chris Raible
a33ce7c20c
Added Sentry Profiling to Ghost server (#19319)
refs ARCH-29

- Added Sentry Profiling to collect more detailed performance data on
the backend.
- This feature is opt-in behind a config. To enable profiling, first
enable tracing with `sentry.tracing.enabled: true`, then set
`sentry.profiling.enabled: true` and `sentry.profiling.sampleRate` to a
decimal number between 0 and 1.
2023-12-13 21:53:19 -08:00
Joel DeSante
dc7e2b9261
🐛Fixed XSS vulnerability involving post excerpts (#17190)
closes https://github.com/TryGhost/Ghost/issues/17058

- Uses the lodash `escape` function.
- Avoids XSS vulnerabilities in post excerpts.
2023-12-13 15:23:48 -06:00
Daniel Lockyer
20b0890a02 Cleaned up duplicate await
- noticed whilst bouncing around the codebase
- shouldn't change anything but it gets rid of some tsserver warnings
2023-12-13 11:54:31 +01:00
Chris Raible
c90e033fcf
Added an email rendering test for all Koenig cards (#19059)
refs TryGhost/Product#4125

This PR adds two new integration tests to ensure all our Koenig cards
are rendered properly after going through the EmailRenderer. Although we
have thorough tests for the cards themselves in the Koenig repo, the
EmailRenderer does post-processing on the rendered HTML, such as
inlining CSS, which can adversely impact the rendered output of our
cards in email clients (usually Outlook).

Since email newsletters are a core feature of Ghost, these bugs are
typically fairly urgent, and since it is email, they are also quite
difficult to troubleshoot and fix. These two tests are intended to
prevent bugs of this sort, which in the past have been created by
seemingly harmless changes like bumping dependencies that are used in
the EmailRenderer.

The idea is to create a 'Golden Post' which has at least 1 of every card
from Koenig, run that post through the EmailRenderer, and take a
snapshot of the rendered HTML. In the future, if we make any changes to
the EmailRenderer or the Koenig cards themselves, this will trigger us
to carefully consider the changes, and it provides an 'expected' output
to compare our changes against.

Additionally, the second test simply checks that all cards from
`kg-default-nodes` are included in the 'Golden Post'. This protects
against any new cards that we will add in the future — as soon as we add
them to Koenig and bump `kg-default-nodes` in Ghost, this test will
fail, prompting us to add the new card to the Golden Post and update the
snapshots.

We should also run the 'Golden Post' through a test in Litmus, which
allows us to visually inspect the rendered email across many different
email clients. Ideally we would create a process to review the output of
the 'Golden Post' in Litmus whenever we update the snapshot as well.
2023-12-12 16:05:04 -08:00
renovate[bot]
45891f83b1 Update dependency ws to v8.15.1 2023-12-12 21:10:38 +00:00
Kevin Ansfield
565b9b245e
🐛 Fixed callout card not rendering all inline formats (#19343)
refs https://github.com/TryGhost/Ghost/issues/19129

- bumps Koenig packages containing fix for callout card rendering
2023-12-12 19:00:32 +00:00
Kevin Ansfield
9706754d6a
🐛 Fixed quote and aside formatting being lost in single-block snippets (#19341)
refs https://github.com/TryGhost/Product/issues/4197

- bumped Koenig packages containing fix for snippets capturing plain text when only the text of an aside or quote block is selected
2023-12-12 17:56:21 +00:00
Simon Backx
60fb2e3139
Added quantities and seed option to the data generator (#19330)
ref PROD-243
2023-12-12 12:50:55 +01:00
Michael Barrett
95eaaad459
Added custom Sentry integration for Knex.js (#19315)
no refs

Added custom Sentry integration for Knex.js to trace database queries in
Sentry
2023-12-12 11:09:49 +00:00
Sanne de Vries
98ff45647c
🎨 Updated editor layout to be more mobile friendly (#19327)
Refs https://github.com/TryGhost/Ghost/issues/18690
2023-12-12 11:51:59 +01:00
Steve Larson
ccc9c9bdd8
Removed emoji picker feature flag (#19314)
closes TryGhost/Product#4109
- requires new editor packages which are bumped in this commit
2023-12-11 09:51:37 -06:00
Kevin Ansfield
1479c55068
🐛 Fixed video uploads hanging in editor when using iOS (#19302)
refs https://github.com/TryGhost/Koenig/issues/1121

- bumps `@tryghost/koenig-lexical` to version including fix (https://github.com/TryGhost/Koenig/pull/1122)
2023-12-11 11:27:59 +00:00
Ronald Langeveld
c969dd18a8
Revert "Fixed private mode cookie for local development (#17938)" (#19298)
This reverts commit f303eee8a4.

refs https://ghost.slack.com/archives/C0568LN2CGJ/p1702277420152709
https://linear.app/tryghost/issue/PROD-46/rss-url-for-private-mode-site-is-hardcoded
2023-12-11 09:18:03 +00:00
renovate[bot]
9fae565673 Update sentry-javascript monorepo to v7.86.0 2023-12-11 09:48:49 +01:00
renovate[bot]
2caf1ec93a Update dependency ws to v8.15.0 2023-12-11 03:41:38 +00:00
renovate[bot]
8cbf133614 Update dependency newrelic to v11.6.1 2023-12-07 19:41:46 +00:00
Daniel Lockyer
db16e565bc Added --print-dependencies to data generator
refs https://github.com/TryGhost/DevOps/issues/119

- this allows you to debug the dependency chain to understand why a
  particular table is being generated
2023-12-07 14:44:00 +01:00
Chris Raible
1b43b5c60a
Added Sentry performance monitoring to Ghost Server (#19243)
refs ARCH-21

- We currently have NewRelic setup for a few of our largest customers
for monitoring performance, but it is too expensive to enable across all
sites
- Sentry has similar (but simpler) performance monitoring tools to keep
track of response times that are available to us for free, but we just
haven't configured them
- This PR sets up Sentry Performance monitoring for API requests so we
can have one place for monitoring errors + performance so we can stay on
top of response times more easily.
- Tracing is disabled by default, so there is no additional overhead
unless `sentry.tracing.enabled` is set to `true` in the site's config.
Additionally, `sentry.tracing.sampleRate` should be set to a decimal
value between 0 and 1. This value defaults to 0 to avoid accidentally
blowing through quota, and requires a value to explicitly be set in
order to send the traces to Sentry.
2023-12-06 15:04:35 -08:00
Steve Larson
d696e8b2e2
Added support for TK tracking inside cards (#19247)
refs https://github.com/TryGhost/Product/issues/4209

- bumped Koenig packages
  - `koenig-lexical` added nested editor TK support
  - all packages dropped Node 16 support
- switched to using `isTKEnabled` prop and `<TKCountPlugin>`

Co-authored-by: Kevin Ansfield <kevin@lookingsideways.co.uk>
2023-12-06 11:32:36 +00:00
Simon Backx
3f6ea04c43
Added portal default plan setting (#19238)
fixes PROD-61

This adds a new default plan setting. It defaults to yearly, which is
the current default selected interval in Portal.

Behind the new portal improvements feature flag, the default plan can be
changed. It will also change automatically if the available intervals
are changed.

This PR also wires up passing the new setting to the Portal preview.
2023-12-06 11:39:58 +01:00
Simon Backx
7c8a141264 Bumped Portal to 2.37.x
no issue
2023-12-06 11:30:47 +01:00
renovate[bot]
351e93ebca Update dependency lib0 to v0.2.88 2023-12-04 22:09:17 +01:00
renovate[bot]
e2a6a83fb6 Update sentry-javascript monorepo to v7.85.0 2023-12-04 22:05:42 +01:00
Djordje Vlaisavljevic
36294c6482 Added feature flag for portal improvements
refs GRO-154
2023-12-04 18:16:23 +00:00
Ghost CI
feb15d2273 Merged v5.75.1 into main 2023-12-04 14:56:08 +00:00
Ghost CI
9ac050dfe9 v5.75.1 2023-12-04 14:56:05 +00:00
renovate[bot]
50d40f298f Update dependency knex-migrator to v5.1.6 2023-12-04 15:21:00 +01:00
renovate[bot]
6f3d16f75b Update dependency postcss to v8.4.32 2023-12-04 08:37:46 +01:00
Ghost CI
90656aa047 v5.75.0 2023-12-01 16:04:16 +00:00
Ghost CI
643ec589a9 🎨 Updated Source to v1.1.2 2023-12-01 16:04:16 +00:00
renovate[bot]
de2482736e Update sentry-javascript monorepo to v7.84.0 2023-12-01 08:58:32 +01:00
Jono M
642c7f39cd
Fixed bugs with newsletter creation in admin (#19201)
refs ADM-4
2023-11-30 15:07:02 +00:00
Daniel Lockyer
4d029c4812 Fixed generating fallback context unnecessarily
fixes https://github.com/TryGhost/Product/issues/4216

When generating page/post metadata, we generate a "context object" from
several meta helpers. In the event there is no context from the resource
type, we generate a fallback object.

However... we generate this fallback object no matter what.

Now, the fallback object is just 3x settingsCache.get, which should be
fast, but it's not. I've opened a separate issue for that: https://github.com/TryGhost/Product/issues/4217

In the mean time, we can switch this logic around to only do these calls
when we have no other context.

From testing, this allows for 10% more throughput on a post 🤯
2023-11-30 10:39:35 +01:00
Simon Backx
ab21b8ae1d
Allowed custom/empty replyTo for newsletters with managed sending domain (#19183)
fixes GRO-75
fixes GRO-100

And allow them to be empty
2023-11-30 09:16:03 +00:00
renovate[bot]
8fc6fef168 Update dependency newrelic to v11.6.0 2023-11-29 15:59:56 +00:00
Sag
f8ad5fb2ea
Updated Portal to use calculated support and default email addresses (#19163)
fixes GRO-72

- added "default_email_address" and "support_email_address" to the
public settings
- when available, use these addresses in Portal. Otherwise, fallback to
current logic
2023-11-29 09:48:05 -03:00
renovate[bot]
d5f3b26e87 Update sentry-javascript monorepo to v7.83.0 2023-11-29 12:29:34 +01:00
Jono M
3f70cc08b7
Fixed create offer API not returning created_at (#19143)
refs https://github.com/TryGhost/Product/issues/4196

The offers API basically returns the data you pass to it, rather than
the created database record. It looks like this is how it was intended
to work in the first place; the `setMilliseconds` is because the test
helper expects `.000Z`, which I assume is because MySQL will strip off
the milliseconds when it's saved.
2023-11-28 15:11:35 +00:00
Simon Backx
3687feca07
Updated support email address verification for DMARC changes (#19147)
fixes GRO-71

- Current flow: unchanged
- New managed flow: verification required
- New managed flow with custom sending domain: only verification
required for different domains
- Self hosters (feature flag): no verification required
2023-11-28 15:06:58 +01:00
Ghost CI
be6916f066 Merged v5.74.5 into main 2023-11-28 12:58:26 +00:00
Ghost CI
df3eea908a v5.74.5 2023-11-28 12:58:24 +00:00
Fabien "egg" O'Carroll
07f9564eea 🔐 Restricted reading files from outside the theme directory
closes https://github.com/TryGhost/Product/issues/4191

Without this patch, themes can read arbitrary files from your system and
expose them to the internet via the layout feature of express-hbs.

For example `{{!< ../../../../config.production.json}}` would spit out config,
which can contain secrets.

As theme upload is restricted to users with the Admin role, this mostly effects
hosting providers which use their own secret keys for e.g. mail or database config
2023-11-28 12:46:06 +00:00
Fabien 'egg' O'Carroll
616247b6d5
Indexed over members_newsletters(newsletter_id, member_id) (#19156)
refs https://github.com/TryGhost/Product/issues/4181

We were seeing slow queries when joining on this table, and the index
speeds them up. The down migration is tricky because when we add the
index MySQL can optimise away some `KEY` indexes on the `newsletter_id`
column. When we then go to remove the newly created index, there is no
index for the FK! 

We also remove the use of `force index` as 1. the index we're forcing is
optimised away and 2. we don't need it anymore!


Co-authored-by: Daniel Lockyer <hi@daniellockyer.com>
2023-11-28 12:20:14 +00:00
renovate[bot]
7d564d4173 Update dependency fs-extra to v11.2.0 2023-11-28 11:54:44 +01:00
Steve Larson
e2807475f8
Updated editor packages (#19151)
closes TryGhost/Product#4204
- updated editor packages to get improvements to TK Plugin
2023-11-27 18:02:26 -06:00
Sag
54ee8d3389
Updated newsletter email verification to render the verified address (#19148)
refs GRO-80
- added a new meta field "email_verified" to the /verification endpoint
for newsletters. This meta field contains which email has been verified,
"sender_email" or "sender_reply_to"
- updated copy in newsletter settings, based on which email has been
verified
2023-11-27 18:28:57 +00:00
Simon Backx
6c37382bc3
Fixed requiring verification email for default address (#19140)
fixes GRO-79

When changing the Reply-To to the default address, the verification
email is now skipped.
2023-11-27 10:45:34 +00:00
Ghost CI
d30a6633ab v5.74.4 2023-11-27 07:56:41 +00:00
Ghost CI
bae7ca957d 🎨 Updated Source to v1.1.1 2023-11-27 07:56:41 +00:00
Daniel Lockyer
28e59b5444
Merged tag 'v5.74.3' into main
v5.74.3
2023-11-27 08:48:04 +01:00
Ghost CI
c6b86abb12 v5.74.3 2023-11-24 14:55:14 +00:00
Simon Backx
a037ce3dde
Added calculated email address settings (#19115)
fixes GRO-73

We need to avoid duplicating the complex logic for determining the
default email address and the support email address. So these are now
exposed as calculated settings.
2023-11-23 13:07:15 +00:00
Simon Backx
17ec1e8937
Added email address alignment protections (#19094)
ref GRO-54
fixes GRO-63
fixes GRO-62
fixes GRO-69

When the config `hostSettings:managedEmail:enabled` is enabled, or the
new flag (`newEmailAddresses`) is enabled for self-hosters, we'll start
to check the from addresses of all outgoing emails more strictly.

- Current flow: nothing changes if the managedEmail config is not set or
the `newEmailAddresses` feature flag is not set
- When managedEmail is enabled: never allow to send an email from any
chosen email. We always use `mail.from` for all outgoing emails. Custom
addresses should be set as replyTo instead. Changing the newsletter
sender_email is not allowed anymore (and ignored if it is set).
- When managedEmail is enabled with a custom sending domain: if a from
address doesn't match the sending domain, we'll default to mail.from and
use the original as a replyTo if appropriate and only when no other
replyTo was set. A newsletter sender email addresss can only be set to
an email address on this domain.
- When `newEmailAddresses` is enabled: self hosters are free to set all
email addresses to whatever they want, without verification. In addition
to that, we stop making up our own email addresses and send from
`mail.from` by default instead of generating a `noreply`+ `@` +
`sitedomain.com` address

A more in depth example of all cases can be seen in
`ghost/core/test/integration/services/email-addresses.test.js`

Includes lots of new E2E tests for most new situations. Apart from that,
all email snapshots are changed because the from and replyTo addresses
are now included in snapshots (so we can see unexpected changes in the
future).

Dropped test coverage requirement, because tests were failing coverage
locally, but not in CI

Fixed settings test that set the site title to an array - bug tracked in
GRO-68
2023-11-23 10:25:30 +01:00
Chris Raible
a97e2b823e
Added error handling to Sentry's beforeSend (#19109)
refs TryGhost/Product#4175

- Added error handling to Sentry's beforeSend function in both Admin and
Core, so if there is any error in beforeSend, we will still send the
unmodified event to Sentry
- This is in response to an incident yesterday wherein the beforeSend
function threw an error due to an unexpected missing value in the
exception. The event sent to Sentry was the error in the beforeSend
function, and the original error never reached Sentry.
- If the original event had reached Sentry, even if unmodified by the
logic in beforeSend, we could have been alerted to the issue sooner and
more easily identified all affected sites.
- Also added defensive logic to protect for certain values in the
exception passed to beforeSend not existing and added unit tests for the
beforeSend function in admin and core
2023-11-23 02:46:50 +00:00
Jono Mingard
efc0f68b96 Merge tag 'v5.74.2'
v5.74.2
2023-11-22 14:53:42 +00:00
Ghost CI
5299d1176e v5.74.2 2023-11-22 14:37:51 +00:00
Steve Larson
0954e08cf9
🐛 Fixed creating posts with an empty root (#19098) (#19099)
refs TryGhost/Product#4156
- bumped renderer package to handle empty root node
2023-11-22 08:26:09 -06:00
Steve Larson
03fec65fd5
🐛 Fixed creating posts with an empty root (#19098)
refs TryGhost/Product#4156
- bumped renderer package to handle empty root node
2023-11-22 14:18:26 +00:00
Daniel Lockyer
8debc4eb02 Forced use of specific index on members_newsletters
refs https://github.com/TryGhost/Product/issues/4181

- we've seen MySQL change its query planner to use a different index
  than the ideal one, resulting in drastically slower query performance
  when fetching newsletters with the member count
- this forces the use of the ideal index on MySQL
- this kept many of the Ghost team up overnight, so I hope MySQL is
  happy
2023-11-22 15:12:42 +01:00
renovate[bot]
e01c87800e Update dependency mysql2 to v3.6.5 2023-11-22 00:59:57 +00:00
Ghost CI
6941ef0148 Merged v5.74.1 into main 2023-11-21 16:49:06 +00:00
Ghost CI
34dc2f8c1e v5.74.1 2023-11-21 16:49:03 +00:00
renovate[bot]
56d18edf0e Update sentry-javascript monorepo to v7.81.1 2023-11-21 15:05:41 +01:00
renovate[bot]
98941cef3a Update dependency yjs to v13.6.10 2023-11-21 11:53:03 +00:00
Simon Backx
b6519e0f1f
Removed usage of unquoted ids in filter strings (#19070)
fixes GRO-34
fixes GRO-33

This is a revision of a previous commit, that broke the browser tests
because changes in the data generator (requiring bookshelf had side
effects).

This adds a new way to run all tests with enforced numeric ObjectIDs.
These numeric ids cause issues if they are used withing NQL filters. So
they surface tiny bugs in our codebase.

You can run tests using this option via:
NUMERIC_IDS=1 yarn test:e2e

Removed some defensive logic that could be explained by this discovered
issue.
2023-11-21 09:45:36 +01:00
renovate[bot]
fedda8b898 Update dependency mysql2 to v3.6.4 2023-11-21 08:14:56 +00:00
Ronald Langeveld
83a1060983
Added last redeemed property to Offers (#19066)
refs https://github.com/TryGhost/Product/issues/4153

- wired up a new last_redeemed prop to the Offers API endpoint.
2023-11-21 08:02:15 +00:00
Steve Larson
a398067159
Added TK reminders labs feature (#19057)
refs TryGhost/Product#4156
2023-11-20 23:30:16 +00:00
renovate[bot]
7a3ae4ab75 Update sentry-javascript monorepo to v7.81.0 2023-11-20 16:57:32 +01:00
Michael Barrett
55392646e1
Refined cache invalidation when updating a user (#19028)
refs https://github.com/TryGhost/Arch/issues/101

Refined the cache invalidation logic so that when updating a user, we
only invalidate the cache when an attribute of the user that is used on
the frontend changes.
2023-11-20 14:14:30 +00:00
Simon Backx
d5492bd63c
Revert "Removed usage of unquoted ids in filter strings" (#19052)
Reverts TryGhost/Ghost#19031

Browser tests are failing with an unknown cause
2023-11-20 13:50:07 +00:00
Jono M
a93c665d20
Created a skeleton AdminX demo app (#19005)
refs https://github.com/TryGhost/Product/issues/4152

---

<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖[[deprecated]](https://githubnext.com/copilot-for-prs-sunset)
Generated by Copilot at a28462f</samp>

This pull request adds a new admin-x app called `admin-x-demo`, which
demonstrates how to use the shared packages `admin-x-framework` and
`admin-x-design-system` to create a simple app that renders a button and
a modal. It also improves the development workflow, the vite
integration, the dependency management, and the type checking for the
admin-x apps and packages. It modifies some files in the
`admin-x-framework` and `admin-x-design-system` packages to make the
modals prop optional, to introduce a new type for the props from the
Ember app, to fix the z-index of the modal backdrop, and to use
consistent file extensions and module syntax.
2023-11-20 13:30:15 +00:00
Simon Backx
320eaac4c4
Removed usage of unquoted ids in filter strings (#19031)
fixes GRO-34
fixes GRO-33

This also adds a new way to run all tests with enforced numeric ObjectIDs.
These numeric ids cause issues if they are used withing NQL filters. So they
surface tiny bugs in our codebase.

You can run tests using this option via:
NUMERIC_IDS=1 yarn test:e2e

Also removed some defensive logic that could be explained by unquoted ids.
2023-11-20 14:00:31 +01:00
Kevin Ansfield
1be490ae9e
Reduced size of social metadata images (#19048)
refs https://github.com/TryGhost/Product/issues/4140

- added `social-image` image size to our `internalImagesSizes` list with a max-width of 1200
- extracted image utils from `{{img_url}}` helper to a utils file for re-use
- updated `getImageDimensions` method that reads image dimensions and modifies the finalised `metaData` object before use to adjust dimensions and associated URLs to match max width of 1200px
2023-11-20 12:39:51 +00:00
renovate[bot]
0cf25d0afe Update dependency yjs to v13.6.9 2023-11-20 12:25:48 +00:00
Jono M
5e057dee11
Added tests to AdminX framework package (#19022)
refs https://github.com/TryGhost/Product/issues/4159

---

<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖[[deprecated]](https://githubnext.com/copilot-for-prs-sunset)
Generated by Copilot at 9e68f4d</samp>

This pull request refactors several components in the `admin-x-settings`
app to use common hooks from the `@tryghost/admin-x-framework` package,
which reduces code duplication and improves consistency. It also updates
the `package.json` file and adds unit tests for the `admin-x-framework`
package, which improves the formatting, testing, and dependency
management. Additionally, it makes some minor changes to the `hooks.ts`,
`FrameworkProvider.tsx`, and `.eslintrc.cjs` files in the
`admin-x-framework` package, which enhance the public API and the
linting configuration.
2023-11-20 11:00:51 +00:00
Ghost CI
a1d7aa6dba v5.74.0 2023-11-17 16:05:58 +00:00
Ghost CI
d77ef5de7c 🎨 Updated Source to v1.1.1 2023-11-17 16:05:58 +00:00
Simon Backx
32d0d2b293 Added filter to filter members by email disabled
fixes GRO-23

Co-authored-by: Djordje Vlaisavljevic <dzvlais@gmail.com>
Co-authored-by: Sag <guptazy@gmail.com>
2023-11-16 19:16:20 +01:00
Sag
69ee4a5ff5
Added option to unsubscribe in one-click from emails (#19032)
refs https://github.com/TryGhost/Product/issues/4051
- added a "List-Unsubscribe" header to emails, in compliance with the
RFC 8058 requirements
- Gmail, Apple Mail, Yahoo Mail, and other popular email clients offer
an option to unsubscribe in one-click, based on the "List-Unsubscribe"
header. Some require an HTTPS endpoint, some a mailto address; both
options are provided in the "List-Unsubscribe" header

Co-authored-by: Simon Backx <simon@ghost.org>
Co-authored-by: Djordje Vlaisavljevic <dzvlais@gmail.com>
2023-11-16 14:50:24 -03:00
Daniel Lockyer
5acecf2cae ️ Improved GScan performance for themes with many partials
refs https://github.com/TryGhost/DevOps/issues/3
refs b6d8e0192a

- see referenced commit for full context but this should improve the
  theme check time for themes with a large number of files and partials
- locally, checking a particularly heavy theme goes from 5s to 1.7s with this
  commit, and the improvement is larger on slower machines
2023-11-16 13:55:57 +01:00
Simon Backx
e3cdc24d6b
Added test to test default theme against linked gscan version (#19020)
fixes GRO-32

If we ever introduce errors in the linked source theme and linked gscan
version combination, this test will catch it.
2023-11-16 12:03:00 +00:00
Simon Backx
3a78cf48c9
Fixed deleting session when requesting identity for invalid session (#19017)
ref https://ghost.slack.com/archives/C02G9E68C/p1700129928489809

- When the GET /api/session endpoint is called, the session is deleted
if it is invalid
- We don't have a body parser for this GET endoint, and the request
object was passed to the deleteSession handler. This caused a type error
(cannot read properties of undefined)
- We had dangling promise because deleteSession is async and wasn't
awaited, causing random errors in tests
- Added a test that would have caught this earlier
2023-11-16 11:01:50 +00:00
Ronald Langeveld
2bd597fe0d
Added createdAt prop to Offers API request. (#19012)
refs https://github.com/TryGhost/Product/issues/4153

- We need use the `created_at` timestamp in the new AdminX offers. The
API doesn't return that value.
- With this change the API returns the created_at property so that we
can consume it.
---

<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖[[deprecated]](https://githubnext.com/copilot-for-prs-sunset)
Generated by Copilot at dc282af</samp>

This pull request adds a `createdAt` property to the offer domain model,
data transfer object, and repository. This allows tracking and auditing
the creation and modification of offers and offer codes in
`ghost/offers`.
2023-11-16 17:33:01 +07:00
Simon Backx
e5f644c27f
🐛 Fixed contain/starts/endsWith filters with /, _ or % in them (#19015)
fixes GRO-25

Updated @tryghost/nql to 0.12.0 and other packages that depend on it

1. SQLite: when a filter string contains /.

When we use a NQL contain/starts/endsWith filter that contains a slash,
underlyingly the whole filter will get converted to a MongoDB query, in
which we just use a regexp to represent the filter. In here we will
escape the slash: \/ as expected in a regexp. Later when we convert this
MongoDB query back to knex/SQL, we use a SQL LIKE query. Currently we
don't remove the escaping here for a normal slash. MySQL seems to ignore
this (kinda incorrect). SQLite doesn't like it, and this breaks queries
on SQLite that use slashes. The solution here is simple: remove the
backslash escaping when converting the regexp to LIKE, just like we do
with other special regexp characters.

2. We don't escape % and _, which have a special meaning in LIKE queries

Usage of % and _ is now as expected and doesn't have the special SQL
meaning anymore.
2023-11-16 09:35:20 +00:00
Steve Larson
d0fa385848
Added jpeg compression (#19006)
refs TryGhost/Product#4140
- bumped image-transform to add jpeg compression
2023-11-15 23:30:48 +00:00
Steve Larson
40a8e969b8
Revert "Bumped image transform package (#19003)" (#19004)
This reverts commit 557c01f48c.
2023-11-15 20:25:29 +00:00
Steve Larson
557c01f48c
Bumped image transform package (#19003)
refs TryGhost/Product#4140
- adds compression for jpeg > jpeg or any > jpeg asset generation
2023-11-15 19:14:23 +00:00
Simon Backx
75bb53f065
🔒 Added support for logging out members on all devices (#18935)
fixes https://github.com/TryGhost/Product/issues/3738
https://www.notion.so/ghost/Member-Session-Invalidation-13254316f2244c34bcbc65c101eb5cc4

- Adds the transient_id column to the members table. This defaults to
email, to keep it backwards compatible (not logging out all existing
sessions)
- Instead of using the email in the cookies, we now use the transient_id
- Updating the transient_id means invalidating all sessions of a member
- Adds an endpoint to the admin api to log out a member from all devices
- Added the `all` body property to the DELETE session endpoint in the
members API. Setting it to true will sign a member out from all devices.
- Adds a UI button in Admin to sign a member out from all devices
- Portal 'sign out of all devices' will not be added for now

Related changes (added because these areas were affected by the code
changes):
- Adds a serializer to member events / activity feed endpoints - all
member fields were returned here, so the transient_id would also be
returned - which is not needed and bloats the API response size
(`transient_id` is not a secret because the cookies are signed)
- Removed `loadMemberSession` from public settings browse (not used
anymore + bad pattern)

Performance tests on site with 50.000 members (on Macbook M1 Pro):
- Migrate: 6s (adding column 4s, setting to email is 1s, dropping
nullable: 1s)
- Rollback: 2s
2023-11-15 17:10:28 +01:00
Kevin Ansfield
3358ba305b
🐛 Fixed highlight formatting not showing in rendered posts (#18997)
closes https://github.com/TryGhost/Product/issues/4144

- bumped `@tryghost/kg-lexical-html-renderer` which adds highlight (`<mark>`) support
2023-11-15 14:08:14 +00:00
Kevin Ansfield
26554bc206
Improved lexical render performance (#18967)
closes https://github.com/TryGhost/Product/issues/4133

- we were creating a new JSDOM instance every time we rendered a card which lowered performance because JSDOM instantiation is heavy
- updated Koenig packages to remove the need for passing in an external `createDocument` option method as they now re-use the renderer's internal single instance of JSDOM
2023-11-15 12:33:22 +00:00
Daniel Lockyer
05977d5c26 Added boot time performance test to CI
refs https://github.com/TryGhost/DevOps/issues/105

- this will run a small benchmark on the boot time for Ghost, and push
  the stats to another repository which has GitHub Pages enabled, so we
  can visualise the change over time
- will only run on pushes to `main` so we don't pollute the stats with
  commits from PRs
- just a test for now to see if we get stable numbers
2023-11-15 13:24:56 +01:00