Commit Graph

46 Commits

Author SHA1 Message Date
Simon Backx
6631071dd3
Improved handling large amounts of email events (#16189)
refs https://github.com/TryGhost/Team/issues/2486

Stop the event fetching loop as soon as we receive events that were
created later then when we started the loop. This ensures that we don't
miss events if we receive a giant batch of events that take a long time
to process.
2023-01-26 16:06:15 +01:00
Daniel Lockyer
34fe49b110
Added version information to log lines
refs https://github.com/TryGhost/Toolbox/issues/501

- this reverts commit 48dda23554
- also includes a resolution for `@elastic/elasticsearch` so we don't
  run a version that is potentially problematic - see referenced issue
  for context
2023-01-20 13:18:44 +01:00
Daniel Lockyer
48dda23554
Reverted "Added version information to log lines"
refs https://github.com/TryGhost/Toolbox/issues/501

- this reverts commit f2116357b7
- something with Elasticsearch is causing high CPU usage, so this commit
  reverts that for now
2023-01-17 13:20:47 +01:00
Daniel Lockyer
c4391567f3
Updated jsdoc for mailgun-client
- in the event the Mailgun config doesn't exist, we return `null` from
  this function
- this updates the jsdoc to correct the return type of `getInstance`
2023-01-13 16:10:32 +01:00
Sam Lord
f2116357b7 Added version information to log lines
refs: https://github.com/TryGhost/Toolbox/issues/502

Updated to @tryghost/logging@2.4.0 to allow metadata to be logged
2023-01-12 10:34:23 +00:00
renovate[bot]
b86eaf8e71
Update dependency nock to v13.3.0 2023-01-10 21:36:44 +00:00
Daniel Lockyer
08b786af3b Bumped TryGhost-owned dependencies and lockfile
- this was all getting terribly behind so I've done several things:
  - majority of `@tryghost/*` except Lexical packages
  - gscan + knex-migrator to remove old `@tryghost/errors` usage
  - bumped lockfile
2023-01-02 20:55:22 +01:00
Hannah Wolfe
abc7e64941 Added new unique tag to all mailgun requests
refs: https://github.com/TryGhost/Ghost/issues/15725

- our users are having difficulties getting onboarded with mailgun
- we're adding an explicit and unique tag to all requests, to help mailgun detect when mail is being sent from Ghost
2022-12-14 12:20:25 +00:00
Simon Backx
47cd7a7095
🐛 Handled unknown Mailgun events (#15995)
refs https://ghost.slack.com/archives/C02G9E68C/p1670916538764019

- We receive events that don't have an emailId or providerId.
- We filter those events now and log them as an error
2022-12-14 11:17:45 +01:00
Daniel Lockyer
6f4e663d74
Updated @tryghost dependencies (#16005)
- also includes `knex-migrator` with a simple `sqlite3` bump
2022-12-14 11:18:55 +07:00
renovate[bot]
13abcf6c9d
Update dependency mocha to v10.2.0 2022-12-12 13:20:22 +00:00
Fabien "egg" O'Carroll
c6c962d5d6 Added methods for removing suppressions from Mailgun
refs https://github.com/TryGhost/Team/issues/2255

These methods will be used by the Mailgun implementation of EmailSuppressionList
so that emails are removed from both our internal list and Mailguns.
2022-12-01 20:14:30 +07:00
Simon Backx
d8187123af
Added storage for email failures (#15901)
fixes https://github.com/TryGhost/Team/issues/2332

Saves events in the database and collects error information.

Do note that we can emit the same events multiple times, and as a result
out of order. That means we should correctly handle that a delivered
event might be fired after a permanent failure. So a delivered event is
ignored if the email is already marked as failed. Also delivered_at is
reset to null when we receive a permanent failure.
2022-12-01 10:00:53 +01:00
renovate[bot]
313c1788d9
Update dependency mailgun.js to v8.0.6 2022-11-17 22:31:28 +00:00
renovate[bot]
174525bc52
Update dependency mailgun.js to v8.0.4 2022-11-16 17:46:29 +00:00
renovate[bot]
8fa9f1e7e6
Update Test & linting packages 2022-11-07 20:39:48 +00:00
renovate[bot]
f8cf220762
Update dependency mailgun.js to v8.0.2 2022-10-25 20:20:41 +00:00
renovate[bot]
603c78755d
Updated @tryghost dependencies (#15631)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2022-10-24 10:30:40 +07:00
Daniel Lockyer
2dcc4139b1
Merged v5.19.2 into main
v5.19.2
2022-10-18 17:04:23 +07:00
Simon Backx
d1e6870740
🐛 Fixed large mailgun recipient data (#15638)
fixes https://github.com/TryGhost/Team/issues/2096

When generating the recipient data for emails, the email clicks
implementation is resulting in a recipient variable being added called
replacement_xxx once for each link containing the same UUID.

This generates a lot of unnecessary data overhead for emails, and it
turns out that mailgun has a 25MB message limit. We wouldn't have come
close if we only included the uuid once.
2022-10-18 10:32:50 +02:00
renovate[bot]
e9587e02d0
Update dependency mocha to v10.1.0 2022-10-17 08:41:28 +00:00
Sam Lord
6ff34fb49f Added logging & metrics to mailgun API calls
refs: https://github.com/TryGhost/Toolbox/issues/439

Adds timed metrics to ensure we know how long Mailgun takes to respond to queries - as well as the status codes we hit.
2022-10-11 15:11:46 +01:00
renovate[bot]
2c2ee81adb
Update Test & linting packages 2022-10-05 00:36:08 +00:00
renovate[bot]
9eb3c84a23
Updated @tryghost dependencies (#15434)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2022-09-27 08:31:35 +07:00
renovate[bot]
225765241c
Updated @tryghost dependencies (#15404)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2022-09-19 12:09:33 +01:00
Daniel Lockyer
7a2f766668
Added logging configuration option for timestamps to use the local timezone
fixes https://github.com/TryGhost/Ghost/issues/15190
refs https://github.com/TryGhost/framework/pull/76

- log output always uses UTC timestamps, but it may be desirable to
  configure logs to use the local machine timezone
- a new config option has been added to `@tryghost/logging` so you can
  switch the logs to the local timezone
- this commit bumps the package and sets the default config option to
  `false`, so it doesn't suddenly change the timezone of the logs
- docs will be updated soon but if you'd like to use the
  timezone-altered timestamps, you can set `logging.useLocalTime` to
  `true`
- credits to https://github.com/levee223 for the implementation and PR
2022-08-31 10:29:55 +01:00
renovate[bot]
9355c6d8fa
Update dependency mailgun.js to v8.0.1 2022-08-30 08:10:52 +00:00
Daniel Lockyer
36c0b7d2bd
Added 60s timeout to Mailgun API calls
refs https://github.com/TryGhost/Toolbox/issues/164

- see referenced issue for more context but Ghost sometimes has issues
  with the email analytics job getting stuck
- we don't provide a timeout to the Mailgun library, so we just
  sit there idling for eternity if something between us and Mailgun is
  causing issues
- this commit adds a 60s timeout so we can at least error out and try
  again next time
2022-08-24 09:13:13 +02:00
Daniel Lockyer
da7b7650c0 Fixed use of static class field
refs 3d80fd982c

- referenced commit moved it from an exported field to a static class
  variable, but I forgot to update it here
2022-08-18 22:34:34 +02:00
Daniel Lockyer
3d80fd982c
Changed BATCH_SIZE to static class field
refs fd6f48d6b2

- this is cleaner and would have avoided the issue fixed in the
  referenced commit
2022-08-18 22:14:54 +02:00
Daniel Lockyer
fd6f48d6b2
Fixed batch size when sending bulk emails
- during a refactor, I moved the `BATCH_SIZE` variable around
- putting the variable export above a more general export means it get
  overwritten and the value is `undefined` outside of the module
- when we chunk the emails, we were chunking in sized of `undefined`,
  so I'm guessing it just defaulted to 1
- this means the email batches were of size 1 instead of 1000 - oops
2022-08-18 22:04:00 +02:00
Daniel Lockyer
f51226e5fb Organized package dependencies
- cleaned up unused dependencies
- adds missing dependencies that are used in the code
- this should help us be more explicit about the dependencies a package
  uses
2022-08-18 11:55:49 +02:00
Daniel Lockyer
3cf64a1e78 Switched Mailgun SDK from mailgun-js to mailgun.js
refs https://github.com/TryGhost/Toolbox/issues/363

- this commit switches us to using the official and maintained
  `mailgun.js` SDK, and updates the `mailgun-client` code to reflect the
  changes between the two
2022-08-17 08:45:21 +02:00
Daniel Lockyer
f48a48914a Updated mailgun-client test fixtures
- some of the existing tests return `items` as an empty array
- the upcoming change to switch `mailgun-js` to `mailgun.js` means it's
  more strict about requiring `paging` too
- this commit adds a new empty-response fixture so we can standardize
  using that across tests
2022-08-17 08:45:21 +02:00
Daniel Lockyer
54aa9f016b Fixed full Admin test suite running during unit tests
- because of how the npm scripts were set up, we were running the full
  Admin integration tests during the unit tests phase of CI
- this commit renames the majority of `test` to `test:unit` in the
  package.json files, and aliases `test` to `test:unit`
- special packages like Admin have no-op'd `test:unit` scripts so we
  don't end up running its tests
2022-08-15 15:34:52 +02:00
Daniel Lockyer
2a07b94bd4
Fixed small boot time regression due to lack of dependency lazyloading
- when I extracted `mailgun-client` to a separate package, I
  accidentally removed the lazyloading for the `mailgun-js` library,
  which takes a non-negligible amount of time to require on boot
- this fixes that by moving the require into the function where it's
  used
2022-08-15 10:10:26 +02:00
renovate[bot]
ea39ee3269 Update dependency @tryghost/logging to v2.2.4 2022-08-11 20:13:21 +02:00
Daniel Lockyer
b226065c04
Added mailgun-client test to ensure config values are prioritised over settings
- this is the current functionality of the code, as it has always been,
  but this test ensures we prioritise the values in the config over
  those in settings
2022-08-11 09:10:49 +02:00
Daniel Lockyer
f9b90a3cfe
Added test to check sending emails when Mailgun is not configured
- this tests the early-return path linked to Mailgun not being
  configured
2022-08-11 09:10:49 +02:00
Daniel Lockyer
ddbee90c1c
Fixed bug when checking email batch size
- even though we don't do anything yet, the code was calling `.length`
  on an object, which is `undefined`
- this fixes that by checking the length of the keys
- also moves the code block down so we can early-return if mailgun is
  not configured
2022-08-11 09:10:49 +02:00
Daniel Lockyer
5d177c2a11
Added optional chaining to mailgun-client package
- this helps keep the code clean
2022-08-11 09:10:49 +02:00
Daniel Lockyer
3e6416431e
Added test to ensure events are only fetched if configured
refs https://github.com/TryGhost/Toolbox/issues/363

- this adds coverage for a branch in the code, and ensures we don't
  fetch any events if mailgun is not configured
2022-08-11 08:19:03 +02:00
Daniel Lockyer
8a32efac1f
Added further tests to mailgun-client package
refs https://github.com/TryGhost/Toolbox/issues/363

- this copies over tests from `email-analytics-provider-mailgun` that
  are more relevant here
- there is now duplication in tests across the two packages but this
  will be resolved soon
2022-08-10 19:02:53 +02:00
Daniel Lockyer
9401d835ce
Moved Mailgun settings test to mailgun-client
- this test checks that the mailgun client respects the changes in
  settings, which is something that we used to ask
  `email-analytics-provider-mailgun` to do when the mailgun client was
  made in that package
- since then, we've pulled it out, so we should move the test to the
  `mailgun-client` library
2022-08-10 18:24:35 +02:00
Daniel Lockyer
98bd5423fa
Added private getConfig function to Mailgun client lib
- the code to get the mailgun info from the config/settings will be used in
  multiple places so this pulls it out into a function
2022-08-10 17:43:19 +02:00
Daniel Lockyer
bf254b9c6a Extracted Mailgun client to separate package
refs https://github.com/TryGhost/Toolbox/issues/363

- this commit pulls all code involving the Mailgun client SDK into one
  new package called `mailgun-client`
- this means we should be able to replace `mailgun-js` (deprecated) with
  `mailgun.js` (the new, official one) without editing code all over the
  place
- this also lays some groundwork for better testing of smaller
  components
2022-08-10 17:12:37 +02:00