Commit Graph

32708 Commits

Author SHA1 Message Date
Fabien "egg" O'Carroll
78bebc6279 Refactored the assignment of Mention metadata
refs https://github.com/TryGhost/Team/issues/2535

Moving this into a separate method allows us to set the metadata externally from
the Mention entity and keep all of the validation, without having duplicate code
2023-02-10 11:43:16 +07:00
renovate[bot]
b271d5a53a Update metascraper to v5.33.8 2023-02-10 02:02:53 +00:00
renovate[bot]
3a11525ad7
Update dependency mysql2 to v3.1.2 2023-02-09 20:10:27 +00:00
Daniel Lockyer
49e9be4368 Reconfigured clean-expired-comped job to execute during morning UTC
refs https://ghost.slack.com/archives/C04MSE4MKJT/p1675948815531779

- running at a fixed hh:mm every day means a platform with a large number
  of Ghost sites will get hammered with DB requests when they all start
  up
- this reconfigures the cron to run at a random minute and second
  between 0am and 5am, which gives a 6 hour window
2023-02-09 15:58:43 +01:00
John O'Nolan
5b4abea56d
Fixed broken link 2023-02-09 10:53:53 -04:00
Ronald Langeveld
4096571b8d Commented out flaky webmention test
refs https://ghost.slack.com/archives/C02G9E68C/p1675934676146349

- Commented out the rate limiter test as it proves to be flaky on CI,
  till we find a better way of implementing it.
2023-02-09 21:35:15 +08:00
Fabien 'egg' O'Carroll
4e8af72305
Added support for deleting mentions (#16238)
refs https://github.com/TryGhost/Team/issues/2534

As we're using soft deletes for mentions we need to store the `deleted` column
as well as enforce a `'deleted:false'` filter on the bookshelf model. 

We've also implemented the handling for deleting mentions. Where we remove a
mention anytime we receive and update from or to a page which no longer exists.

Co-authored-by: Steve Larson <9larsons@gmail.com>
2023-02-09 17:29:13 +07:00
Steve Larson
d20696805f
Added mentions deleted column (#16251)
refs https://github.com/TryGhost/Team/issues/2534

This is so that we can support soft deletes for Mentions.
We need to add the defaults to the model so that write continue to work.

Co-authored-by: Fabien "egg" O'Carroll <fabien@allou.is>
2023-02-09 16:19:07 +07:00
Daniel Lockyer
d750612508
Merged v5.33.8 into main
v5.33.8
2023-02-09 10:08:43 +01:00
Ghost CI
83bf4ccb53 v5.33.8 2023-02-09 09:06:34 +00:00
Ronald Langeveld
ce567b9816
Added Mentions email notification settings (#16245)
closes https://github.com/TryGhost/Team/issues/2526

- Mention emails can now be toggled inside staff user' profiles, if they
have the webmention flag enabled on their Ghost site.
- Removed the flag dedicated to webmention email notifications and is
now handled by the `webmention` flag.
- Does not send email notification if `webmention` flag is not enabled.
- Updated tests.

---------

Co-authored-by: Fabien "egg" O'Carroll <fabien@allou.is>
2023-02-09 17:03:03 +08:00
Simon Backx
da97519082 Moved email event fetching to main thread (#16231)
refs https://github.com/TryGhost/Team/issues/2482

This moves the processing of Mailgun events to the main thread. By using a simple approach where we emit a start event on the worker thread (via the job manager) and listen for it on the main thread. This is needed because for now the job manager doesn't support scheduling periodic jobs on the main thread (not offloaded).

Apart from that, the email processor now uses the email event storage directy instead of emitting events (it is still emitting event for now). This makes sure we await for the event to be processed before continuing with the next event.
2023-02-09 09:44:44 +01:00
Simon Backx
ea2c69565f
Moved email event fetching to main thread (#16231) 2023-02-09 09:36:39 +01:00
renovate[bot]
fd79ca3f5a Update dependency html-validate to v7.13.2 2023-02-09 09:30:14 +01:00
Simon Backx
b7be02f057
🐛 Fixed throwing errors during link redirects
no issue

Errors weren't correctly passed on to Express in the middleware.
2023-02-09 09:26:45 +01:00
Kevin Ansfield
f7aefa2feb
🐛 Fixed default email recipients always being "everyone" rather than matching post visibility (#16247)
refs https://github.com/TryGhost/Ghost/issues/16125

The previous fix for incorrect recipient details being shown when
re-sending a failed email introduced another bug that prevented the
"Match post visibility" default recipients setting from working.

- the server always sets `post.emailSegment` to `'all'` for new posts so
the publish flow recipient filter logic that checked for
`post.emailSegment` being present always defaulted to `'all'` rather
than falling back to the selected default recipients setting
- when a post has been published but the email failed it will have its
`newsletter` value set so we can use that as a check for using the
`post.emailSegment` value in place of the default recipients setting
2023-02-09 09:24:50 +01:00
Ronald Langeveld
30dc2a8228
Added mention_notifications column (#16242)
refs https://github.com/TryGhost/Team/issues/2526

- created a migration for a new boolean column in users that would
determine if the staff user gets an email when the publication receive a
new mention.
2023-02-09 16:15:54 +08:00
Ronald Langeveld
9fc13cfe65
Implemented mentions rate limiter (#16223)
closes https://github.com/TryGhost/Team/issues/2419

- adds a rate limiter implementation to the mentions receiving
endpoint.
- Current configuration is `{"minWait": 10,
             "maxWait": 100,
             "lifetime": 1000,
             "freeRetries": 100}` which is still very open and almost unrestricted. 
- currently makes use of database storage to track the limits, but can be relatively easily swapped out to something eg Redis should we find this endpoint getting hit too often and maliciously.
2023-02-09 14:57:48 +08:00
Naz
93a10d8f4f Optimized queries for tiers (aka products)
refs https://github.com/TryGhost/Toolbox/issues/515
refs dd4d6aeae5

- The `productRepository.list` call produced 5 db queries and a transaction wrapping this call.
- Transaction is not needed in this situation as there are no possible writes in the meantime (transaction wrapping code was put in there through refed commit to guard against failing Stripe API calls, which are no longer involved when calling the list method)
-  The `limit: 'all'` makes sure all product entries are fetched AND removes an extra aggregation query called over stripe_prices join
- The 'monthlyPrice' and 'yearlyPrice' relations are not needed because this data is not used in downstream code - only slug and type are used for visiblity/content gating  (ref. 1 3b6759ca6d/ghost/core/core/server/services/members/content-gating.js (L44-L55), ref. 2 3b6759ca6d/ghost/core/core/server/api/endpoints/utils/serializers/output/mappers/posts.js (L39-L54))
2023-02-09 13:07:57 +08:00
Steve Larson
6c97edec25
moved sending mentions to job (#16234)
refs TryGhost/Team#2523
-implemented jobsService within mention sending service
-updated tests so we don't need to sleep
2023-02-08 16:29:12 -06:00
Sam Lord
0d7944861c Data generator: Swap order of free and premium newsletters
no issue
Free and premium newsletters were the other way around in the demo-data. This was a good opportunity to stop the email table importer from relying on the newsletter name, and use the order alone.
2023-02-08 22:12:19 +00:00
Kevin Ansfield
923a9a1cd6
🐛 Fixed default email recipients always being "everyone" rather than matching post visibility (#16247)
refs https://github.com/TryGhost/Ghost/issues/16125

The previous fix for incorrect recipient details being shown when
re-sending a failed email introduced another bug that prevented the
"Match post visibility" default recipients setting from working.

- the server always sets `post.emailSegment` to `'all'` for new posts so
the publish flow recipient filter logic that checked for
`post.emailSegment` being present always defaulted to `'all'` rather
than falling back to the selected default recipients setting
- when a post has been published but the email failed it will have its
`newsletter` value set so we can use that as a check for using the
`post.emailSegment` value in place of the default recipients setting
2023-02-08 11:41:48 +00:00
Simon Backx
4436a1cd18 🐛 Fixed throwing errors during link redirects
no issue

Errors weren't correctly passed on to Express in the middleware.
2023-02-08 11:50:22 +01:00
Aileen Booker
3b6759ca6d
Added initial basic milestone emails in-memory repository (#16216)
refs
https://www.notion.so/ghost/Marketing-Milestone-email-campaigns-1d2c9dee3cfa4029863edb16092ad5c4

This adds a milestone entity and in-memory repository in a new
`milestone-emails` package. This also adds a first initial definition of
milestones and their types which is held in the default config to avoid
DB changes when, e. g. values change.

This should get everything in place to begin with the service
implementation.
2023-02-07 12:47:35 +02:00
Ghost CI
c518446c17 Merged v5.33.7 into main 2023-02-07 10:32:47 +00:00
Ghost CI
ca568817dd v5.33.7 2023-02-07 10:32:43 +00:00
Simon Backx
ac2daccf95 Added short lived caching to email batch body (#16233)
fixes https://github.com/TryGhost/Team/issues/2522

When sending an email for multiple batches at the same time, we now
reuse the same email body for each batch in the same segment. This
reduces the amount of database queries and makes the sending more
reliable in case of database failures.

The cache is short lived. After sending the email it is automatically
garbage collected.
2023-02-07 11:30:37 +01:00
Simon Backx
fb3cbe5fc8
Added short lived caching to email batch body (#16233)
fixes https://github.com/TryGhost/Team/issues/2522

When sending an email for multiple batches at the same time, we now
reuse the same email body for each batch in the same segment. This
reduces the amount of database queries and makes the sending more
reliable in case of database failures.

The cache is short lived. After sending the email it is automatically
garbage collected.
2023-02-07 11:01:49 +01:00
renovate[bot]
aa6242eb6d Update peter-evans/create-or-update-comment digest to 521ce1b 2023-02-07 09:45:48 +01:00
renovate[bot]
654a991c98 Update dependency mysql2 to v3.1.1 2023-02-07 09:20:11 +01:00
renovate[bot]
244fd05035 Update dependency lint-staged to v13.1.1 2023-02-07 09:09:59 +01:00
Sam Lord
e0bcb26600 Improved data-generator tooling
no issue

This makes the data-generator more customisable.
2023-02-06 10:42:56 +00:00
renovate[bot]
c01875b85c Update sentry-javascript monorepo to v7.36.0 2023-02-06 10:26:32 +01:00
renovate[bot]
b76466946e Update dependency @types/express to v4.17.17 2023-02-06 10:26:06 +01:00
renovate[bot]
d60fea59e2 Update dependency terser to v5.16.3 2023-02-04 04:47:16 +00:00
Steve Larson
efd01f649b removed suppressionList from labs settings 2023-02-03 14:57:30 -06:00
Steve Larson
5ccb778ac8
enabled always sending mentions (#16227)
refs TryGhost/Team#2507
-removed labs check from sending service
-labs flag controls seeing mentions in admin and receiving mentions
2023-02-03 14:52:20 -06:00
Steve Larson
02953490a5
added mention sending service e2e tests (#16226)
refs TryGhost/Team#2508
-added sending service e2e tests
-should job off this sending service for better tests
-and for ghost to finish processing the job before shutdown
2023-02-03 10:45:50 -06:00
Ghost CI
d37ed200ab Merged v5.33.6 into main 2023-02-03 13:41:48 +00:00
Ghost CI
8e72332c77 v5.33.6 2023-02-03 13:41:45 +00:00
Simon Backx
30b9d02f70 Increased wait time between Mailgun events to 200ms
no issue

Reduce concurrency if the database is going a bit slower, until we have more permanent fix for this.
2023-02-03 14:38:44 +01:00
Djordje Vlaisavljevic
3f7b717c0f Added logic to display target post titles
refs https://github.com/TryGhost/Team/issues/2481
2023-02-03 11:18:03 +00:00
Djordje Vlaisavljevic
5f1bdd6d87 Fixed scroll state issue
refs https://github.com/TryGhost/Team/issues/2434
2023-02-03 11:18:03 +00:00
Djordje Vlaisavljevic
2e379809d0 Updated timestamp design
refs https://github.com/TryGhost/Team/issues/2434
2023-02-03 11:18:03 +00:00
Simon Backx
29e1e93ca1 Increased wait time between Mailgun events to 150ms
no issue

Reduce concurrency if the database is going a bit slower, until we have more permanent fix for this.
2023-02-03 10:39:55 +01:00
Simon Backx
4fad1986a0 🐛 Fixed link to posts on audio cards in emails
refs https://ghost.slack.com/archives/CTH5NDJMS/p1675356984043299

The url property doesn't exists on post, this was copied from the old MEGA flow that serialized the model first.
2023-02-03 10:37:02 +01:00
Naz
b35b1ae98b
Added cleanup note
refs https://github.com/TryGhost/Toolbox/issues/516
refs 4c60cdce9c

- The syntax used to skip non-`/r/` routes is unreadable. Needs a little love during cleanup
2023-02-03 13:11:59 +08:00
Simon Backx
0f08f39a1b 🐛 Fixed link to posts on audio cards in emails
refs https://ghost.slack.com/archives/CTH5NDJMS/p1675356984043299

The url property doesn't exists on post, this was copied from the old MEGA flow that serialized the model first.
2023-02-02 18:19:31 +01:00
Ghost CI
509affb1f7 Merged v5.33.5 into main 2023-02-02 16:33:50 +00:00
Ghost CI
771a6182c7 v5.33.5 2023-02-02 16:33:46 +00:00