Commit Graph

18 Commits

Author SHA1 Message Date
Simon Backx
94e85dc09e
Reduced webhook calls when updating last_seen_at for email opens ()
refs https://ghost.slack.com/archives/C02G9E68C/p1670960248186789

This reverts a change that was made here:

f4fdb4fa6c (r93071549),
but it still moved the original code to a new location in the
LastSeenAtUpdater

It includes a new E2E test to make sure timezones are supported
correctly.

- By not using Bookshelf, we no longer fire webhook calls
- By not using the member repository, we don't fetch and update the
member model and the labels relation in a forUpdate transaction, which
caused deadlock issues on the labels/members_labels tables which were
hard to resolve. Until now I was unable to find the other conflicting
transaction that caused this deadlock. Moving to raw knex (instead of
Bookshelf) and only updating the last_updated_at column should remove
the deadlock issue.

This removed the test for the email service wrapper, since it started
failing for an unknown reason and the test didn't make much sense (was
added earlier only to bump test threshold).
2022-12-14 17:50:42 +01:00
Simon Backx
bededf4520
🐛 Fixed errors of old events from deleted members ()
refs https://ghost.slack.com/archives/C02G9E68C/p1670215917451249

When a member is deleted, and we receive an opened event for an email to
that member. We threw an uncaught Bookshelf EmptyResponse error.

- This change makes fetching the member not a requirement when handling
that event in the last seen at updater.
- It also adds try catches for all event listeners in the last seen at
updater
2022-12-05 14:36:23 +01:00
Simon Backx
f4fdb4fa6c
Added new email event processor ()
fixes https://github.com/TryGhost/Team/issues/2310

This moves the processing of the events from the event-processor to a
new email-event-processor in the email-service package.

- The `EmailEventProcessor` only translates events from
providerId/emailId to their known emailId, memberId and recipientId, and
dispatches the corresponding events.
- Since `EmailEventProcessor` runs in a separate worker thread, we can't
listen for the dispatched events on the main thread. To accomplish this
communication, the events dispatched from the `EmailEventProcessor`
class are 'posted' via the postMessage method and redispatched on the
main thread.
- A new `EmailEventStorage` class reacts to the email events and stores
it in the database. This code mostly corresponds to the (now deleted)
subclass of the old `EmailEventProcessor`
- Updating a members last_seen_at timestamp has moved to the
lastSeenAtUpdater.
- Email events no longer store `ObjectID` because these are not
encodable across threads via postMessage
- Includes new E2E tests that test the storage of all supported Mailgun
events. Note that in these tests we run the processing on the main
thread instead of on a separate thread (couldn't do this because
stubbing is not possible across threads)

There are some missing pieces that will get added in later PRs (this PR
focuses on porting the existing functionality):
- Handling temporary failures/bounces
- Capturing the error messages of bounce events
2022-11-29 11:15:19 +01:00
Simon Backx
076e3c02b2
Added linking between member and subscription created events ()
fixes https://github.com/TryGhost/Team/issues/2160

- Adds a `batch_id` to both events that contain the same ID if they were created at the same time.
- Removes duplicate signup/conversion events using the batch_id
- Requires an update in mongo-knex to work (refs https://ghost.slack.com/archives/C02G9E68C/p1666773313272409?thread_ts=1666767872.375009&cid=C02G9E68C)
- Some dependencies needed an update to load the latest mongo-knex
- Added tiers to membersUtils, loaded on startup (we can start to use this instead of fetching it every time)
2022-10-27 11:44:19 +02:00
Rishabh Garg
b46258bbc6
Allowed storing attribution data on member events ()
refs 90034577b8

- this change was missed in above commit and would have blocked capturing attribution data for sites without the flag
2022-09-28 15:42:04 +05:30
Rishabh Garg
31733657a6
Updated naming for referrer attribution ()
- renames `refSource`, `refMedium` and `refUrl` to `referrerSource`, `referrerMedium` and `referrerUrl` respectively for consistent naming across files and usages
2022-09-28 00:58:06 +05:30
Simon Backx
1290477d71
Added member last seen update on link click ()
fixes https://github.com/TryGhost/Team/issues/1952

Adds a new MemberLinkClickEvent event that is fired when a member clicks a link. This code has been added to the `linkClickRepository` because that is the only place that has access to the member model (and the event requires the id and current last seen at value). The LastSeenAtUpdater listens for this event and updates the timestamp if required.
2022-09-23 10:34:33 +02:00
Rishabh
bb0d900937 Handled storing referrer information in DB
refs https://github.com/TryGhost/Team/issues/1931

- stores `referrer_source`, `referrer_medium` and `referrer_url` in event tables for new members and paid subscriptions
2022-09-21 19:32:18 +05:30
Simon Backx
74ecde73db
Moved attribution event handler to events service ()
fixes https://github.com/TryGhost/Team/issues/1821

This change moves all the event storage logic to one new place: the event storage class in the MembersEventsService, which is initialised in a new members events service wrapper.

Apart from this, this includes some improvements:
- Removed DomainEvents from the constructor arguments to the subscribe method (to make it more clear where to subscribe to and decrease dependencies)
- LastSeenAtUpdater doesn't subscribe in the constructor any longer (removes unclear side effect)
- Moved LastSeenAtUpdater initialisation to new members events service wrapper
- Added missing tests to LastSeenAtUpdater to assure that the MembersEventsService package has 100% coverage.
2022-09-07 16:41:59 +02:00
Simon Backx
31a4135fec
Added members.last_commented_at and last_seen_at update when commenting ()
refs https://github.com/TryGhost/Team/issues/1717

- Updates last_commented_at and last_seen_at (only once a day)
- Used the LastSeenAtUpdater, so we can combine updating last_commented_at and last_seen_at in one query + used same pattern
- Updated comments service to await emails in order to make E2E tests more stable (as we don't have any method to await emails and test emails otherwise). This removed the email sending logic from the `onCreated` hook of the model.
2022-07-25 17:35:46 +02:00
Simon Backx
8a40d8e76b Fixed member webhooks ()
refs https://github.com/TryGhost/Team/issues/1577

The call to `edit` was not loading the newsletter relations which is needed
by the serializer used by the webhooks service.

Co-authored-by: Fabien "egg" O'Carroll <fabien@allou.is>
2022-05-02 19:07:30 +01:00
Thibaut Patel
30681319b5 Fixed the model update method
refs https://github.com/TryGhost/Ghost/pull/14197

- Uses the right method to update a model (`edit`)
- Also fixes the `updateLastSeenAt` comment that wasn't reflecting the code
2022-03-01 17:35:02 +01:00
Thibaut Patel
c12c638974 Fixed typo in members-events-service dependency
refs https://github.com/TryGhost/Ghost/pull/14197

- `domainEvents` should be plural
2022-03-01 17:19:34 +01:00
Thibaut Patel
88f8b622f1 Moved the DomainEvents service as a dependency of members-events-service
refs https://github.com/TryGhost/Ghost/pull/14197

- Using the package directly was creating a second instance and was never triggering the subscriber
- Passing DomainEvents as a dependency solves this issue
2022-03-01 17:12:59 +01:00
Thibaut Patel
36bb014fc0 Simplified time comparison in last-seen-at-updater
no issue

- The code had un-needed complexity
- Removing the extra complexity led to the same behavior, as validated by the unit tests
2022-03-01 13:07:37 +01:00
Thibaut Patel
bc5b8109e6 Moved the last-seen-at-updater to use the publication timezone
refs https://github.com/TryGhost/Team/issues/1306

- This removes the limitation described in commit ff46449ad6
- The only edge case is that when a publication changes their timezone, it will have maximum 24 hours where the member last_seen_at could be incorrect
2022-03-01 10:28:45 +01:00
Thibaut Patel
d2d7fb3fe7 Moved from a floating 24h window to a UTC-aligned window
refs https://github.com/TryGhost/Ghost/pull/14197

- Moved from updating the last_seen_at value "at most every 24h" to "at most every UTC day".
- It will simplify explaining the following behavior: a publication is set in UTC-10, a user visits at 2pm on Monday and at 1pm on Tuesday, the last_seen_at value is still Monday.
- There is no way to go around the above issue due to the technical constraint of updating the `last_seen_at` value at most once a day.
- This might create database write spikes at midnight UTC
2022-02-28 14:42:17 +01:00
Thibaut Patel
161c0d7330 Added the members-events-service package
refs https://github.com/TryGhost/Team/issues/1306

- Contains all services that listen on member events
- Only contains the last-seen-at-updater service for now
- Listens for `MemberViewEvent` events to update the `member.last_seen_at` timestamp
- Updates after 24hours of the last timestamp to avoid too many writes
- Also updates when the value is NULL
- This is using the existing `last_seen_at` value to avoid an SQL query when no writes are required
2022-02-28 14:42:17 +01:00