fixes https://github.com/TryGhost/Team/issues/2310
This moves the processing of the events from the event-processor to a
new email-event-processor in the email-service package.
- The `EmailEventProcessor` only translates events from
providerId/emailId to their known emailId, memberId and recipientId, and
dispatches the corresponding events.
- Since `EmailEventProcessor` runs in a separate worker thread, we can't
listen for the dispatched events on the main thread. To accomplish this
communication, the events dispatched from the `EmailEventProcessor`
class are 'posted' via the postMessage method and redispatched on the
main thread.
- A new `EmailEventStorage` class reacts to the email events and stores
it in the database. This code mostly corresponds to the (now deleted)
subclass of the old `EmailEventProcessor`
- Updating a members last_seen_at timestamp has moved to the
lastSeenAtUpdater.
- Email events no longer store `ObjectID` because these are not
encodable across threads via postMessage
- Includes new E2E tests that test the storage of all supported Mailgun
events. Note that in these tests we run the processing on the main
thread instead of on a separate thread (couldn't do this because
stubbing is not possible across threads)
There are some missing pieces that will get added in later PRs (this PR
focuses on porting the existing functionality):
- Handling temporary failures/bounces
- Capturing the error messages of bounce events
refs https://github.com/TryGhost/Team/issues/2291
When sending out mails to individual recipients, its possible that recipient gets a temporary or permanent failure for receiving the mail. Temporary failures can generally get resolved after a bit when the recipient’s mail server accepts the email, unlike permanent failures. For both customer visibility and easier debugging on what went wrong while delivering to a particular recipient, we’ll store the permanent/temporary failure for a recipient.
- migration adds a new table that stores the failure information for the recipients
refs https://github.com/TryGhost/Toolbox/issues/475
refs https://github.com/TryGhost/Toolbox/issues/117
- The frontend/src folded would never get picked up by a code coverage tooling as scripts there are dynamically minimized and served from the server
- There's nothing to cover under frontend/public
- Logic behind these changes is the same as in second referenced issue
fixes https://github.com/TryGhost/Team/issues/2326
When importing more than 500 members, we didn't testImportThreshold at
the right time. It was called too early because the importing job was
not awaited. This also adds an E2E test for this case.
closesTryGhost/Team#2313
- Added Sent event to Post analytics and Members feed. Now post can be
Sent or Received or Bounced.
- Excluded Delivered event from Sent filter on backend.
refs https://github.com/TryGhost/Ghost/security/advisories/GHSA-9gh8-wp53-ccc6
refs https://github.com/TryGhost/Toolbox/issues/465
- Bookshelf relations allows us to edit relational records by default, which was used liberally in the codebase.
- Not having a clear track record of editable relations left the model layer prone to triggering unwanted nested saves and created a vulnerability where members were able to edit newsletter settings.
- With explicit editable relations it's easier to keep track of relations having editable access to related records. Makes the relational data modification pattern safer to use too.
- Anyone running 5.x should update to 5.24.1
Credits: Dave McDaniel and other members of [Cisco Talos](https://talosintelligence.com/vulnerability_reports)
refs https://github.com/TryGhost/Ghost/security/advisories/GHSA-9gh8-wp53-ccc6
refs https://github.com/TryGhost/Toolbox/issues/465
- Bookshelf relations allows us to edit relational records by default, which was used liberally in the codebase.
- Not having a clear track record of editable relations left the model layer prone to triggering unwanted nested saves and created a vulnerability where members were able to edit newsletter settings.
- With explicit editable relations it's easier to keep track of relations having editable access to related records. Makes the relational data modification pattern safer to use too.
- Anyone running 5.x should update to 5.24.1
Credits: Dave McDaniel and other members of [Cisco Talos](https://talosintelligence.com/vulnerability_reports)
closes Sentry ADMIN-CC8
closes Sentry ADMIN-DDM
closes Sentry ADMIN-C9F
- there are situations where the editor range when key commands are triggered does not have a head section which then throws errors due to the commands assuming there will always be a section present
- added a guard to key commands that use the head section to abort early and fall back to default Mobiledoc behaviour when the section is missing
closes sentry ADMIN-C7S
- we can't guarantee that the iframe being swapped to has rendered yet at the time we swap iframes so we need a guard around the `contentDocument.body` property existing before setting it's `scrollTop` value
refs https://github.com/TryGhost/Team/issues/2280
refs 9b0c21e0a2
As part of the email stability work, we added new `source` and `source_type` columns to `email` table, which allows us to store the email source information. The source for all existing emails before the stability work was always `html`, while newer emails will store `mobiledoc` or `lexical` directly.
While the `source` for all existing emails was populated with the `html` as part of above migration, we also need to store the right `source` for all new emails created till the feature is under a flag.
This change updates the current email flow to also store `source` with html data, so it can be used in future with new email service and allows removing old `html` column.
closesTryGhost/Team#2264
- Instead of relative paths, we can use absolute. It helps to keep code cleaner and don't worry about import when doing refactoring.
Relative paths require rewriting them in case a file is moved to another directory.
refs https://github.com/TryGhost/Team/issues/2253
refs https://github.com/TryGhost/Team/issues/2254
This package is analogous to the @tryghost/member-events package. The
events here will be consumed by the EmailSuppressionList
implementation and used to add emails to said list. They'll be
dispatched by the code which handles events received from Mailgun.
refs: https://github.com/TryGhost/Team/issues/1121
- We've decided on one preferred message for unexpected errors
- We want to use this everywhere where we don't know what to display
- We now have a GENERIC_ERROR_MESSAGE constant that we should use
refs: https://github.com/TryGhost/Team/issues/1121
- showAPIError is a method intended for formatting errors from the Ghost API
- Ghost API Errors do not have a detail field, therefore this code was redundant
- there are also no related tests
- removing now because I'm trying to cleanup and streamline all our error handling code
refs: https://github.com/TryGhost/Team/issues/1121
- In certain cases our API sends the same data for message and context.
- We will also fix this server-side, but we should also be smart in the UI and not show duplicate info
no issue
- "unhandled" `TransitionAborted` errors almost always occur as part of expected application behaviour and were causing a lot of noise in Sentry making it harder to track down real errors
- when a `TransitionAborted` error occurs outside of expected behaviour it will usually be accompanied by other errors that do get logged
- there's a long-standing Ember issue about how aborted transition errors should be handled at https://github.com/emberjs/ember.js/issues/12505
refs: https://github.com/TryGhost/Team/issues/1121
refs: 54574025e0
- The previous change to fall back to a generic error on the server side is resulting in lots of much less useful Sentry reports
- For unexpected errors, change what's sent to Sentry back to context
- This is done by adding a specific code, so we don't have to match on a string that might change
- Also add the error type, id, code & statusCode as tags to the events - these are searchable structured data
- Adding code as a tag also makes it possible to find all errors that showed the generic message
- As demonstrated by my comments in the boot file, I thought sentry was already depending on the version package
- IMO it's undesirable to require package.json directly esp when we have a tool setup and ready for tis
- Added a bunch of tests to show that Sentry does roughly what we think
