Commit Graph

60 Commits

Author SHA1 Message Date
Simon Backx
94e85dc09e
Reduced webhook calls when updating last_seen_at for email opens (#16008)
refs https://ghost.slack.com/archives/C02G9E68C/p1670960248186789

This reverts a change that was made here:

f4fdb4fa6c (r93071549),
but it still moved the original code to a new location in the
LastSeenAtUpdater

It includes a new E2E test to make sure timezones are supported
correctly.

- By not using Bookshelf, we no longer fire webhook calls
- By not using the member repository, we don't fetch and update the
member model and the labels relation in a forUpdate transaction, which
caused deadlock issues on the labels/members_labels tables which were
hard to resolve. Until now I was unable to find the other conflicting
transaction that caused this deadlock. Moving to raw knex (instead of
Bookshelf) and only updating the last_updated_at column should remove
the deadlock issue.

This removed the test for the email service wrapper, since it started
failing for an unknown reason and the test didn't make much sense (was
added earlier only to bump test threshold).
2022-12-14 17:50:42 +01:00
Daniel Lockyer
6f4e663d74
Updated @tryghost dependencies (#16005)
- also includes `knex-migrator` with a simple `sqlite3` bump
2022-12-14 11:18:55 +07:00
renovate[bot]
13abcf6c9d
Update dependency mocha to v10.2.0 2022-12-12 13:20:22 +00:00
Simon Backx
bededf4520
🐛 Fixed errors of old events from deleted members (#15944)
refs https://ghost.slack.com/archives/C02G9E68C/p1670215917451249

When a member is deleted, and we receive an opened event for an email to
that member. We threw an uncaught Bookshelf EmptyResponse error.

- This change makes fetching the member not a requirement when handling
that event in the last seen at updater.
- It also adds try catches for all event listeners in the last seen at
updater
2022-12-05 14:36:23 +01:00
Simon Backx
f4fdb4fa6c
Added new email event processor (#15879)
fixes https://github.com/TryGhost/Team/issues/2310

This moves the processing of the events from the event-processor to a
new email-event-processor in the email-service package.

- The `EmailEventProcessor` only translates events from
providerId/emailId to their known emailId, memberId and recipientId, and
dispatches the corresponding events.
- Since `EmailEventProcessor` runs in a separate worker thread, we can't
listen for the dispatched events on the main thread. To accomplish this
communication, the events dispatched from the `EmailEventProcessor`
class are 'posted' via the postMessage method and redispatched on the
main thread.
- A new `EmailEventStorage` class reacts to the email events and stores
it in the database. This code mostly corresponds to the (now deleted)
subclass of the old `EmailEventProcessor`
- Updating a members last_seen_at timestamp has moved to the
lastSeenAtUpdater.
- Email events no longer store `ObjectID` because these are not
encodable across threads via postMessage
- Includes new E2E tests that test the storage of all supported Mailgun
events. Note that in these tests we run the processing on the main
thread instead of on a separate thread (couldn't do this because
stubbing is not possible across threads)

There are some missing pieces that will get added in later PRs (this PR
focuses on porting the existing functionality):
- Handling temporary failures/bounces
- Capturing the error messages of bounce events
2022-11-29 11:15:19 +01:00
renovate[bot]
8fa9f1e7e6
Update Test & linting packages 2022-11-07 20:39:48 +00:00
Simon Backx
076e3c02b2
Added linking between member and subscription created events (#15693)
fixes https://github.com/TryGhost/Team/issues/2160

- Adds a `batch_id` to both events that contain the same ID if they were created at the same time.
- Removes duplicate signup/conversion events using the batch_id
- Requires an update in mongo-knex to work (refs https://ghost.slack.com/archives/C02G9E68C/p1666773313272409?thread_ts=1666767872.375009&cid=C02G9E68C)
- Some dependencies needed an update to load the latest mongo-knex
- Added tiers to membersUtils, loaded on startup (we can start to use this instead of fetching it every time)
2022-10-27 11:44:19 +02:00
renovate[bot]
e9587e02d0
Update dependency mocha to v10.1.0 2022-10-17 08:41:28 +00:00
renovate[bot]
6fc497743d
Updated @tryghost dependencies (#15479)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2022-10-12 10:16:05 +07:00
renovate[bot]
2c2ee81adb
Update Test & linting packages 2022-10-05 00:36:08 +00:00
Rishabh Garg
b46258bbc6
Allowed storing attribution data on member events (#15487)
refs 90034577b8

- this change was missed in above commit and would have blocked capturing attribution data for sites without the flag
2022-09-28 15:42:04 +05:30
Rishabh Garg
31733657a6
Updated naming for referrer attribution (#15486)
- renames `refSource`, `refMedium` and `refUrl` to `referrerSource`, `referrerMedium` and `referrerUrl` respectively for consistent naming across files and usages
2022-09-28 00:58:06 +05:30
renovate[bot]
9eb3c84a23
Updated @tryghost dependencies (#15434)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2022-09-27 08:31:35 +07:00
Simon Backx
1290477d71
Added member last seen update on link click (#15459)
fixes https://github.com/TryGhost/Team/issues/1952

Adds a new MemberLinkClickEvent event that is fired when a member clicks a link. This code has been added to the `linkClickRepository` because that is the only place that has access to the member model (and the event requires the id and current last seen at value). The LastSeenAtUpdater listens for this event and updates the timestamp if required.
2022-09-23 10:34:33 +02:00
Rishabh
bb0d900937 Handled storing referrer information in DB
refs https://github.com/TryGhost/Team/issues/1931

- stores `referrer_source`, `referrer_medium` and `referrer_url` in event tables for new members and paid subscriptions
2022-09-21 19:32:18 +05:30
renovate[bot]
225765241c
Updated @tryghost dependencies (#15404)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2022-09-19 12:09:33 +01:00
Simon Backx
74ecde73db
Moved attribution event handler to events service (#15379)
fixes https://github.com/TryGhost/Team/issues/1821

This change moves all the event storage logic to one new place: the event storage class in the MembersEventsService, which is initialised in a new members events service wrapper.

Apart from this, this includes some improvements:
- Removed DomainEvents from the constructor arguments to the subscribe method (to make it more clear where to subscribe to and decrease dependencies)
- LastSeenAtUpdater doesn't subscribe in the constructor any longer (removes unclear side effect)
- Moved LastSeenAtUpdater initialisation to new members events service wrapper
- Added missing tests to LastSeenAtUpdater to assure that the MembersEventsService package has 100% coverage.
2022-09-07 16:41:59 +02:00
renovate[bot]
5f4c170d05 Update dependency @tryghost/errors to v1.2.15 2022-08-18 12:02:38 +02:00
Daniel Lockyer
f51226e5fb Organized package dependencies
- cleaned up unused dependencies
- adds missing dependencies that are used in the code
- this should help us be more explicit about the dependencies a package
  uses
2022-08-18 11:55:49 +02:00
Daniel Lockyer
54aa9f016b Fixed full Admin test suite running during unit tests
- because of how the npm scripts were set up, we were running the full
  Admin integration tests during the unit tests phase of CI
- this commit renames the majority of `test` to `test:unit` in the
  package.json files, and aliases `test` to `test:unit`
- special packages like Admin have no-op'd `test:unit` scripts so we
  don't end up running its tests
2022-08-15 15:34:52 +02:00
Daniel Lockyer
a9cffd9550
Pinned all dependencies
- we should pin all dependencies so we know what is getting shipped in
  the application
2022-07-27 08:44:50 +02:00
Daniel Lockyer
a12047b224
Added cobertura reporter to members-events-service
- without this, we don't get coverage reports for the package
2022-07-26 15:37:33 +02:00
Simon Backx
31a4135fec
Added members.last_commented_at and last_seen_at update when commenting (#15088)
refs https://github.com/TryGhost/Team/issues/1717

- Updates last_commented_at and last_seen_at (only once a day)
- Used the LastSeenAtUpdater, so we can combine updating last_commented_at and last_seen_at in one query + used same pattern
- Updated comments service to await emails in order to make E2E tests more stable (as we don't have any method to await emails and test emails otherwise). This removed the email sending logic from the `onCreated` hook of the model.
2022-07-25 17:35:46 +02:00
Daniel Lockyer
308a28d31a
Tidied up package READMEs
refs https://github.com/TryGhost/Toolbox/issues/354

- these READMEs were migrated over from when each package was in a
  different repo
- they also assume you're going to be publishing the packages because it
  mentions install instructions
- only a few of them contain custom content
- this commit deletes the majority of these files because they're now
  not useful
- any that contained other instructions have been cut down
2022-07-25 15:17:12 +02:00
Daniel Lockyer
61125d7605
Removed repository from component package.json files
refs https://github.com/TryGhost/Toolbox/issues/354

- these repository links made sense when they were in different repos
  and published to NPM but we don't publish these packages any more
- this commit deletes those keys from the files
2022-07-25 11:15:16 +02:00
Daniel Lockyer
d73d7da7ef
Deleted non-root LICENSE files
- these were copied over during the monorepo conversion but we're not
  going to be publishing these packages so the top-level LICENSE file
  covers all packages here
2022-07-25 08:35:59 +02:00
Daniel Lockyer
17bf358b47
Pinned internal packages
- we're going to be pinning all dependencies within the monorepo
- this shouldn't change anything anyway because we're using the same
  version across all packages
2022-07-21 16:35:19 +02:00
Daniel Lockyer
48fd5ca8cb
Removed posttest linting step
- linting is done as a separate step in CI and it's a git hook upon
  pushing locally, so we don't need to run it after tests
2022-07-21 10:09:36 +02:00
Daniel Lockyer
9ec83a6e21
Disabled publishing for Members packages
- we don't want to publish these anymore so this commit disables the
  ability to
- also fixes up a missing version that wasn't reset
2022-07-21 09:26:04 +02:00
Daniel Lockyer
376ee24600
Switched to unversioned Members packages
- these packages are split apart for local development, but will be
  bundled into Ghost when publishing
- therefore, these packages won't be published so we are resetting the
  versions to make them cleaner
2022-07-21 09:15:29 +02:00
renovate[bot]
bc4aebf163 Update Test & linting packages 2022-07-19 18:55:22 +00:00
Fabien "egg" O'Carroll
c41f067ea8 Published new versions
- @tryghost/domain-events@0.1.14
 - @tryghost/express-dynamic-redirects@0.2.13
 - @tryghost/magic-link@1.0.26
 - @tryghost/member-analytics-service@0.1.16
 - @tryghost/member-events@0.4.6
 - @tryghost/members-analytics-ingress@0.1.17
 - @tryghost/members-api@8.0.0
 - @tryghost/members-csv@1.2.15
 - @tryghost/members-events-service@0.4.3
 - @tryghost/members-importer@0.5.14
 - @tryghost/members-ssr@1.0.28
 - @tryghost/members-offers@0.11.6
 - @tryghost/members-payments@0.3.6
 - @tryghost/members-stripe-service@0.10.5
 - @tryghost/verification-trigger@0.2.5
2022-05-16 19:29:05 +01:00
Renovate Bot
a599830920 Update dependency c8 to v7.11.3 2022-05-16 05:20:36 +00:00
Aileen Nowak
152a2ea41f Published new versions
- @tryghost/domain-events@0.1.13
 - @tryghost/express-dynamic-redirects@0.2.12
 - @tryghost/magic-link@1.0.25
 - @tryghost/member-analytics-service@0.1.15
 - @tryghost/member-events@0.4.5
 - @tryghost/members-analytics-ingress@0.1.16
 - @tryghost/members-api@7.0.1
 - @tryghost/members-csv@1.2.14
 - @tryghost/members-events-service@0.4.2
 - @tryghost/members-importer@0.5.13
 - @tryghost/members-ssr@1.0.27
 - @tryghost/members-offers@0.11.5
 - @tryghost/members-payments@0.3.5
 - @tryghost/members-stripe-service@0.10.4
 - @tryghost/verification-trigger@0.2.4
2022-05-10 11:40:41 -04:00
Renovate Bot
7af0eb6fdf Update dependency sinon to v14 2022-05-09 04:41:01 +00:00
Simon Backx
0111e23fb7 Published new versions
- @tryghost/domain-events@0.1.12
 - @tryghost/express-dynamic-redirects@0.2.11
 - @tryghost/magic-link@1.0.24
 - @tryghost/member-analytics-service@0.1.14
 - @tryghost/member-events@0.4.4
 - @tryghost/members-analytics-ingress@0.1.15
 - @tryghost/members-api@6.3.1
 - @tryghost/members-csv@1.2.13
 - @tryghost/members-events-service@0.4.1
 - @tryghost/members-importer@0.5.12
 - @tryghost/members-ssr@1.0.26
 - @tryghost/members-offers@0.11.4
 - @tryghost/members-payments@0.3.4
 - @tryghost/members-stripe-service@0.10.3
 - @tryghost/verification-trigger@0.2.3
2022-05-03 16:40:39 +02:00
Renovate Bot
bddc9a5159 Update dependency c8 to v7.11.2 2022-05-02 21:39:21 +00:00
Fabien "egg" O'Carroll
b8545d2912 Published new versions
- @tryghost/domain-events@0.1.11
 - @tryghost/express-dynamic-redirects@0.2.10
 - @tryghost/magic-link@1.0.23
 - @tryghost/member-analytics-service@0.1.13
 - @tryghost/member-events@0.4.3
 - @tryghost/members-analytics-ingress@0.1.14
 - @tryghost/members-api@6.3.0
 - @tryghost/members-csv@1.2.12
 - @tryghost/members-events-service@0.4.0
 - @tryghost/members-importer@0.5.11
 - @tryghost/members-ssr@1.0.25
 - @tryghost/members-offers@0.11.3
 - @tryghost/members-payments@0.3.3
 - @tryghost/members-stripe-service@0.10.2
 - @tryghost/verification-trigger@0.2.2
2022-05-02 19:11:55 +01:00
Simon Backx
8a40d8e76b Fixed member webhooks (#394)
refs https://github.com/TryGhost/Team/issues/1577

The call to `edit` was not loading the newsletter relations which is needed
by the serializer used by the webhooks service.

Co-authored-by: Fabien "egg" O'Carroll <fabien@allou.is>
2022-05-02 19:07:30 +01:00
Renovate Bot
cddf87863e Update dependency mocha to v10 2022-05-02 02:29:13 +00:00
Simon Backx
a0a50f7acc Published new versions
- @tryghost/domain-events@0.1.10
 - @tryghost/express-dynamic-redirects@0.2.9
 - @tryghost/magic-link@1.0.22
 - @tryghost/member-analytics-service@0.1.12
 - @tryghost/member-events@0.4.2
 - @tryghost/members-analytics-ingress@0.1.13
 - @tryghost/members-api@6.2.0
 - @tryghost/members-csv@1.2.11
 - @tryghost/members-events-service@0.3.4
 - @tryghost/members-importer@0.5.9
 - @tryghost/members-ssr@1.0.24
 - @tryghost/members-offers@0.11.2
 - @tryghost/members-payments@0.3.2
 - @tryghost/members-stripe-service@0.10.1
 - @tryghost/verification-trigger@0.2.1
2022-04-27 19:08:40 +02:00
Renovate Bot
a49bb037c4 Update Test & linting packages 2022-04-27 12:16:56 +00:00
Rishabh
0e09a57971 Published new versions
- @tryghost/express-dynamic-redirects@0.2.8
 - @tryghost/members-api@5.5.0
 - @tryghost/members-csv@1.2.10
 - @tryghost/members-events-service@0.3.3
 - @tryghost/members-importer@0.5.7
 - @tryghost/members-stripe-service@0.9.3
2022-04-04 21:08:04 +05:30
Daniel Lockyer
b84e54b0cf Unpinned dependencies
- libraries should have dependencies unpinned so we can rely on
  lockfiles and reduce duplicates
- this commit unpins a handful of dependencies
2022-03-24 11:06:57 +00:00
Thibaut Patel
c37021c575 Published new versions
- @tryghost/domain-events@0.1.9
 - @tryghost/express-dynamic-redirects@0.2.7
 - @tryghost/magic-link@1.0.21
 - @tryghost/member-analytics-service@0.1.11
 - @tryghost/member-events@0.4.1
 - @tryghost/members-analytics-ingress@0.1.12
 - @tryghost/members-api@5.3.0
 - @tryghost/members-csv@1.2.7
 - @tryghost/members-events-service@0.3.2
 - @tryghost/members-importer@0.5.4
 - @tryghost/members-ssr@1.0.23
 - @tryghost/members-offers@0.10.9
 - @tryghost/members-payments@0.1.11
 - @tryghost/members-stripe-service@0.9.1
 - @tryghost/verification-trigger@0.1.6
2022-03-11 22:45:26 +01:00
Renovate Bot
0ad973c8b5 Update dependency mocha to v9.2.2 2022-03-11 18:04:58 +00:00
Thibaut Patel
64e2f49bab Published new versions
- @tryghost/members-events-service@0.3.1
2022-03-01 17:35:27 +01:00
Thibaut Patel
30681319b5 Fixed the model update method
refs https://github.com/TryGhost/Ghost/pull/14197

- Uses the right method to update a model (`edit`)
- Also fixes the `updateLastSeenAt` comment that wasn't reflecting the code
2022-03-01 17:35:02 +01:00
Thibaut Patel
0ad1e65ff6 Published new versions
- @tryghost/members-events-service@0.3.0
2022-03-01 17:20:14 +01:00
Thibaut Patel
c12c638974 Fixed typo in members-events-service dependency
refs https://github.com/TryGhost/Ghost/pull/14197

- `domainEvents` should be plural
2022-03-01 17:19:34 +01:00