Commit Graph

13 Commits

Author SHA1 Message Date
Hannah Wolfe
6161f94910
Updated to use assert/strict everywhere (#17047)
refs: https://github.com/TryGhost/Toolbox/issues/595

We're rolling out new rules around the node assert library, the first of which is enforcing the use of assert/strict. This means we don't need to use the strict version of methods, as the standard version will work that way by default.

This caught some gotchas in our existing usage of assert where the lack of strict mode had unexpected results:
- Url matching needs to be done on `url.href` see aa58b354a4
- Null and undefined are not the same thing,  there were a few cases of this being confused
- Particularly questionable changes in [PostExporter tests](c1a468744b) tracked [here](https://github.com/TryGhost/Team/issues/3505).
- A typo see eaac9c293a

Moving forward, using assert strict should help us to catch unexpected behaviour, particularly around nulls and undefineds during implementation.
2023-06-21 09:56:59 +01:00
Simon Backx
913ad18b71
Added DomainEvents.allSettled utility method (#16075)
no issue

With the increased usage of DomainEvents, it gets harder to build
reliable tests without having to resort to timeouts. This utility method
allows us to wait for all events to be processed before continuing with
the test.

This change should speed up tests and make them more reliable.

It only adds extra code when running tests and shouldn't impact
production.
2023-01-04 14:30:35 +01:00
Simon Backx
94e85dc09e
Reduced webhook calls when updating last_seen_at for email opens (#16008)
refs https://ghost.slack.com/archives/C02G9E68C/p1670960248186789

This reverts a change that was made here:

f4fdb4fa6c (r93071549),
but it still moved the original code to a new location in the
LastSeenAtUpdater

It includes a new E2E test to make sure timezones are supported
correctly.

- By not using Bookshelf, we no longer fire webhook calls
- By not using the member repository, we don't fetch and update the
member model and the labels relation in a forUpdate transaction, which
caused deadlock issues on the labels/members_labels tables which were
hard to resolve. Until now I was unable to find the other conflicting
transaction that caused this deadlock. Moving to raw knex (instead of
Bookshelf) and only updating the last_updated_at column should remove
the deadlock issue.

This removed the test for the email service wrapper, since it started
failing for an unknown reason and the test didn't make much sense (was
added earlier only to bump test threshold).
2022-12-14 17:50:42 +01:00
Simon Backx
f4fdb4fa6c
Added new email event processor (#15879)
fixes https://github.com/TryGhost/Team/issues/2310

This moves the processing of the events from the event-processor to a
new email-event-processor in the email-service package.

- The `EmailEventProcessor` only translates events from
providerId/emailId to their known emailId, memberId and recipientId, and
dispatches the corresponding events.
- Since `EmailEventProcessor` runs in a separate worker thread, we can't
listen for the dispatched events on the main thread. To accomplish this
communication, the events dispatched from the `EmailEventProcessor`
class are 'posted' via the postMessage method and redispatched on the
main thread.
- A new `EmailEventStorage` class reacts to the email events and stores
it in the database. This code mostly corresponds to the (now deleted)
subclass of the old `EmailEventProcessor`
- Updating a members last_seen_at timestamp has moved to the
lastSeenAtUpdater.
- Email events no longer store `ObjectID` because these are not
encodable across threads via postMessage
- Includes new E2E tests that test the storage of all supported Mailgun
events. Note that in these tests we run the processing on the main
thread instead of on a separate thread (couldn't do this because
stubbing is not possible across threads)

There are some missing pieces that will get added in later PRs (this PR
focuses on porting the existing functionality):
- Handling temporary failures/bounces
- Capturing the error messages of bounce events
2022-11-29 11:15:19 +01:00
Simon Backx
74ecde73db
Moved attribution event handler to events service (#15379)
fixes https://github.com/TryGhost/Team/issues/1821

This change moves all the event storage logic to one new place: the event storage class in the MembersEventsService, which is initialised in a new members events service wrapper.

Apart from this, this includes some improvements:
- Removed DomainEvents from the constructor arguments to the subscribe method (to make it more clear where to subscribe to and decrease dependencies)
- LastSeenAtUpdater doesn't subscribe in the constructor any longer (removes unclear side effect)
- Moved LastSeenAtUpdater initialisation to new members events service wrapper
- Added missing tests to LastSeenAtUpdater to assure that the MembersEventsService package has 100% coverage.
2022-09-07 16:41:59 +02:00
Simon Backx
31a4135fec
Added members.last_commented_at and last_seen_at update when commenting (#15088)
refs https://github.com/TryGhost/Team/issues/1717

- Updates last_commented_at and last_seen_at (only once a day)
- Used the LastSeenAtUpdater, so we can combine updating last_commented_at and last_seen_at in one query + used same pattern
- Updated comments service to await emails in order to make E2E tests more stable (as we don't have any method to await emails and test emails otherwise). This removed the email sending logic from the `onCreated` hook of the model.
2022-07-25 17:35:46 +02:00
Simon Backx
8a40d8e76b Fixed member webhooks (#394)
refs https://github.com/TryGhost/Team/issues/1577

The call to `edit` was not loading the newsletter relations which is needed
by the serializer used by the webhooks service.

Co-authored-by: Fabien "egg" O'Carroll <fabien@allou.is>
2022-05-02 19:07:30 +01:00
Thibaut Patel
30681319b5 Fixed the model update method
refs https://github.com/TryGhost/Ghost/pull/14197

- Uses the right method to update a model (`edit`)
- Also fixes the `updateLastSeenAt` comment that wasn't reflecting the code
2022-03-01 17:35:02 +01:00
Thibaut Patel
c12c638974 Fixed typo in members-events-service dependency
refs https://github.com/TryGhost/Ghost/pull/14197

- `domainEvents` should be plural
2022-03-01 17:19:34 +01:00
Thibaut Patel
88f8b622f1 Moved the DomainEvents service as a dependency of members-events-service
refs https://github.com/TryGhost/Ghost/pull/14197

- Using the package directly was creating a second instance and was never triggering the subscriber
- Passing DomainEvents as a dependency solves this issue
2022-03-01 17:12:59 +01:00
Thibaut Patel
bc5b8109e6 Moved the last-seen-at-updater to use the publication timezone
refs https://github.com/TryGhost/Team/issues/1306

- This removes the limitation described in commit ff46449ad6
- The only edge case is that when a publication changes their timezone, it will have maximum 24 hours where the member last_seen_at could be incorrect
2022-03-01 10:28:45 +01:00
Thibaut Patel
d2d7fb3fe7 Moved from a floating 24h window to a UTC-aligned window
refs https://github.com/TryGhost/Ghost/pull/14197

- Moved from updating the last_seen_at value "at most every 24h" to "at most every UTC day".
- It will simplify explaining the following behavior: a publication is set in UTC-10, a user visits at 2pm on Monday and at 1pm on Tuesday, the last_seen_at value is still Monday.
- There is no way to go around the above issue due to the technical constraint of updating the `last_seen_at` value at most once a day.
- This might create database write spikes at midnight UTC
2022-02-28 14:42:17 +01:00
Thibaut Patel
161c0d7330 Added the members-events-service package
refs https://github.com/TryGhost/Team/issues/1306

- Contains all services that listen on member events
- Only contains the last-seen-at-updater service for now
- Listens for `MemberViewEvent` events to update the `member.last_seen_at` timestamp
- Updates after 24hours of the last timestamp to avoid too many writes
- Also updates when the value is NULL
- This is using the existing `last_seen_at` value to avoid an SQL query when no writes are required
2022-02-28 14:42:17 +01:00