Commit Graph

20 Commits

Author SHA1 Message Date
Simon Backx
c47891c3f6
🐛 Fixed setting delivered_at to null after hard bounce (#15942)
refs https://ghost.slack.com/archives/C02G9E68C/p1670075366333929?thread_ts=1669963540.980309&cid=C02G9E68C

When we receive a permanent bounce/failure, we set delivered_at to null.
But we don't want to lose this information.

Instead we should be able to handle recipients that both have failed_at
and delivered_at set.
2022-12-06 10:26:54 +05:30
Simon Backx
bededf4520
🐛 Fixed errors of old events from deleted members (#15944)
refs https://ghost.slack.com/archives/C02G9E68C/p1670215917451249

When a member is deleted, and we receive an opened event for an email to
that member. We threw an uncaught Bookshelf EmptyResponse error.

- This change makes fetching the member not a requirement when handling
that event in the last seen at updater.
- It also adds try catches for all event listeners in the last seen at
updater
2022-12-05 14:36:23 +01:00
Simon Backx
2519f4b20d Disabled autoRefresh for batch sending service to increase performance
fixes https://github.com/TryGhost/Team/issues/2363
2022-12-05 12:09:30 +01:00
Simon Backx
ea72934c41
Improved email concurrency sending (#15923)
no issue

Send 10 batches at the same time
2022-12-02 15:30:02 +01:00
Simon Backx
1958d0db63
Removed spaces and quotes from replacement ids (#15921)
fixes https://github.com/TryGhost/Team/issues/2350
2022-12-02 10:49:01 +01:00
Simon Backx
6fbda0730d Added unit test for paywall and segments 2022-12-02 10:48:29 +01:00
Simon Backx
53b5c152d9 Added email event storage unit tests
refs https://github.com/TryGhost/Team/issues/2339
2022-12-02 10:21:39 +01:00
Rishabh Garg
f9161876b3
Updated unit test coverage for email service (#15913)
refs https://github.com/TryGhost/Team/issues/2339

- adds unit test for email renderer, segmenter and sending service
2022-12-02 00:33:28 +05:30
Simon Backx
0182965aa3 Updated batch sending to log errors separately
fixes https://github.com/TryGhost/Team/issues/2347

Makes sure we can use the error code
2022-12-01 16:35:54 +01:00
Simon Backx
4c166e11df
Added E2E tests for batch sending (#15910)
refs https://github.com/TryGhost/Team/issues/2339

- Includes a new pattern in the job manager that allows us to properly
await jobs.
- Added new convenience mocking methods to stub settings
- Tests the main flows for bulk sending:
    - Sending in multiple batches
    - Sending to multiple segments
    - Handling a failed batch and retrying that batch
- Fixes bug in batch generation (ordering not working)

In a different PR I'll add more detailed tests.
2022-12-01 13:43:49 +01:00
Fabien "egg" O'Carroll
12219fed65 Handled receiving duplicate SpamComplaintEvents
We need to check for both ER_DUP_ENTRY and SQLITE_CONSTRAINT because
different DB engines give us back different error codes.
2022-12-01 17:27:06 +07:00
Fabien "egg" O'Carroll
f9fdee1b28 Wired up storage of EmailSpamComplaintEvents
refs https://github.com/TryGhost/Team/issues/2337

This also removes the code which unsubscribes members from newsletters because
that is not the spec of the feature.
2022-12-01 17:26:59 +07:00
Simon Backx
d8187123af
Added storage for email failures (#15901)
fixes https://github.com/TryGhost/Team/issues/2332

Saves events in the database and collects error information.

Do note that we can emit the same events multiple times, and as a result
out of order. That means we should correctly handle that a delivered
event might be fired after a permanent failure. So a delivered event is
ignored if the email is already marked as failed. Also delivered_at is
reset to null when we receive a permanent failure.
2022-12-01 10:00:53 +01:00
Simon Backx
26d51687b1
Implemented email previews and tests using new email flow (#15899)
fixes https://github.com/TryGhost/Team/issues/2330

Uses new flow for previewing and testing emails (only if email stability
flag is enabled)
2022-11-30 13:56:28 +01:00
Rishabh Garg
42f9d392a3
Added mailgun provider for sending emails (#15896)
closes https://github.com/TryGhost/Team/issues/2309

- adds new mailgun provider to send out batch emails
- updates sending service to send email id for mailgun provider, allows tagging mail with email id
2022-11-30 16:21:58 +05:30
Simon Backx
f5045b9bf7
Added email renderer implementation draft (#15877)
fixes https://github.com/TryGhost/Team/issues/2308

- Still has some missing pieces, but mostly works.
- Uses new handlebars template for emails
- When sending emails with the new email stability flag enabled, one
test email is now sent via the default smtp ghost mailer.
2022-11-29 11:27:17 +01:00
Simon Backx
f4fdb4fa6c
Added new email event processor (#15879)
fixes https://github.com/TryGhost/Team/issues/2310

This moves the processing of the events from the event-processor to a
new email-event-processor in the email-service package.

- The `EmailEventProcessor` only translates events from
providerId/emailId to their known emailId, memberId and recipientId, and
dispatches the corresponding events.
- Since `EmailEventProcessor` runs in a separate worker thread, we can't
listen for the dispatched events on the main thread. To accomplish this
communication, the events dispatched from the `EmailEventProcessor`
class are 'posted' via the postMessage method and redispatched on the
main thread.
- A new `EmailEventStorage` class reacts to the email events and stores
it in the database. This code mostly corresponds to the (now deleted)
subclass of the old `EmailEventProcessor`
- Updating a members last_seen_at timestamp has moved to the
lastSeenAtUpdater.
- Email events no longer store `ObjectID` because these are not
encodable across threads via postMessage
- Includes new E2E tests that test the storage of all supported Mailgun
events. Note that in these tests we run the processing on the main
thread instead of on a separate thread (couldn't do this because
stubbing is not possible across threads)

There are some missing pieces that will get added in later PRs (this PR
focuses on porting the existing functionality):
- Handling temporary failures/bounces
- Capturing the error messages of bounce events
2022-11-29 11:15:19 +01:00
renovate[bot]
e48072e471 Update dependency bson-objectid to v2.0.4 2022-11-24 11:24:07 +07:00
Simon Backx
4b4592630f
Added new email batch sending service (#15865)
fixes https://github.com/TryGhost/Team/issues/2284

New batch sending flow (still WIP). Logs the sent emails instead of actually sending them. Unit tests are coming in later commits.
2022-11-23 11:33:44 +01:00
Simon Backx
44f189b56a
Added email service package (#15849)
fixes https://github.com/TryGhost/Team/issues/2282

Added a new email service package that is used when the email stability
flag is enabled. Currently not yet implemented so will throw an error
for all entry points (if flag enabled).

Removed usage of `labs.isSet.bind` across the code, because that breaks
the stubbing of labs by `mockManager.mockLabsEnabled` and
`mockManager.mockLabsDisabled`. `flag => labs.isSet(flag)` should be
used instead.

All email depending tests now disable the `emailStability` feature flag
to keep the tests passing + make sure we still run all the tests for the
old flow while the email stability package is being built.
2022-11-21 10:29:53 +01:00