Commit Graph

32434 Commits

Author SHA1 Message Date
Kevin Ansfield
a855973ac6 Wired up top-of-document handling in Lexical editor
closes https://github.com/TryGhost/Team/issues/2285

- added passthrough of `cursorDidExitAtTop` action so the editor can trigger title input focus on key commands that trigger the cursor to leave the top of document
2022-12-01 16:46:06 +00:00
Peter Zimon
a51f3a124d Added static batch errors to error debug
refs. https://github.com/TryGhost/Team/issues/2327

- the batch errors tab content was just a placeholder, this adds static version to be able to test and wire it in
2022-12-01 17:30:13 +01:00
Simon Backx
0182965aa3 Updated batch sending to log errors separately
fixes https://github.com/TryGhost/Team/issues/2347

Makes sure we can use the error code
2022-12-01 16:35:54 +01:00
Peter Zimon
dee00e36a0 Added static UI to email debug screen
refs. https://github.com/TryGhost/Team/issues/2327

- static UI was needed on the email debug screen for testing and backend wiring
2022-12-01 15:13:48 +01:00
Fabien "egg" O'Carroll
6849a4ef66 Removed mock data from event fetcher
closes https://github.com/TryGhost/Team/issues/2277

This data is no longer needed since we wired up to the mailgun implementation
2022-12-01 20:19:59 +07:00
Fabien "egg" O'Carroll
c5b233ac65 Moved the filtering of suppressed emails behind a flag check
We want to be able to release this code without restricting the
sending of emails on our end, so this logic needs to be behind a
feature flag
2022-12-01 20:14:30 +07:00
Fabien "egg" O'Carroll
2350cacf3a Wired up MailgunEmailSuppressionList
This switches out the in-memory implementation for the mailgun one and
is the final piece in wirign up everything to real data.
2022-12-01 20:14:30 +07:00
Fabien "egg" O'Carroll
a25fdf4052 Added MailgunEmailSuppressionList
refs https://github.com/TryGhost/Team/issues/2253
refs https://github.com/TryGhost/Team/issues/2254
refs https://github.com/TryGhost/Team/issues/2255

This is the implementation of the EmailSuppressionList specific to
Mailgun, it listens for the new events and stores suppressions based
on those.
2022-12-01 20:14:30 +07:00
Fabien "egg" O'Carroll
5e72e42d32 Moved InMemoryEmailSuppressionList to own file
This cleans up the service file so we can make the switch to a Mailgun specific
implementation later.
2022-12-01 20:14:30 +07:00
Fabien "egg" O'Carroll
c6c962d5d6 Added methods for removing suppressions from Mailgun
refs https://github.com/TryGhost/Team/issues/2255

These methods will be used by the Mailgun implementation of EmailSuppressionList
so that emails are removed from both our internal list and Mailguns.
2022-12-01 20:14:30 +07:00
Simon Backx
4c166e11df
Added E2E tests for batch sending (#15910)
refs https://github.com/TryGhost/Team/issues/2339

- Includes a new pattern in the job manager that allows us to properly
await jobs.
- Added new convenience mocking methods to stub settings
- Tests the main flows for bulk sending:
    - Sending in multiple batches
    - Sending to multiple segments
    - Handling a failed batch and retrying that batch
- Fixes bug in batch generation (ordering not working)

In a different PR I'll add more detailed tests.
2022-12-01 13:43:49 +01:00
Rishabh Garg
a5ca64f693
Added basic email debug design (#15914)
refs https://github.com/TryGhost/Team/issues/2327

As part of improving visibility for email failures, this change adds a new debug screen that allows visualising the email failures for a post. The screen is hidden on the UI for now and only accessible via URL directly.

Co-authored-by: Djordje Vlaisavljevic <dzvlais@gmail.com>
2022-12-01 17:58:37 +05:30
Fabien "egg" O'Carroll
12219fed65 Handled receiving duplicate SpamComplaintEvents
We need to check for both ER_DUP_ENTRY and SQLITE_CONSTRAINT because
different DB engines give us back different error codes.
2022-12-01 17:27:06 +07:00
Fabien "egg" O'Carroll
c0e91c73d3 Added unique constraint to email_spam_complaint_events table
We can fetch the same event multiple times from Mailgun so we need to
be able to protect against inserting duplicate events in the
database. This will allow us to catch duplicate errors on insert when
handling complaint events.
2022-12-01 17:27:02 +07:00
Fabien "egg" O'Carroll
f9fdee1b28 Wired up storage of EmailSpamComplaintEvents
refs https://github.com/TryGhost/Team/issues/2337

This also removes the code which unsubscribes members from newsletters because
that is not the spec of the feature.
2022-12-01 17:26:59 +07:00
Fabien "egg" O'Carroll
9c691f3ea9 Added support for EmailSpamComplaintEvent to activity feed
We need to add relations and filterRelations to the model so that we can
correctly filter events by post_id and member_id
2022-12-01 17:26:24 +07:00
Naz
04d9cbb1eb Removed unused members settings module
no issue

- This module is marked for deletion with a todo comment.  It also popped up as least covered place in the codebase, because it was never used.
2022-12-01 16:12:09 +07:00
Simon Backx
d8187123af
Added storage for email failures (#15901)
fixes https://github.com/TryGhost/Team/issues/2332

Saves events in the database and collects error information.

Do note that we can emit the same events multiple times, and as a result
out of order. That means we should correctly handle that a delivered
event might be fired after a permanent failure. So a delivered event is
ignored if the email is already marked as failed. Also delivered_at is
reset to null when we receive a permanent failure.
2022-12-01 10:00:53 +01:00
Kevin Ansfield
a1ee04b08f Wired up title->editor keyboard handling in Lexical editor
closes https://github.com/TryGhost/Team/issues/2286

- use the `registerAPI` prop to get access to a basic API for focusing and inserting paragraphs
- replaced commented mobiledoc based title key handling with lexical handling
2022-11-30 22:04:02 +00:00
Elena Baidakova
95c3a68c34
Updated the filter naming in Post Analytics (#15898)
closes TryGhost/Team#2329
- Replace 'Received' on 'Sent' in member's filter
- Moved links for feedback analytics from chart to table
2022-11-30 17:39:37 +04:00
Simon Backx
26d51687b1
Implemented email previews and tests using new email flow (#15899)
fixes https://github.com/TryGhost/Team/issues/2330

Uses new flow for previewing and testing emails (only if email stability
flag is enabled)
2022-11-30 13:56:28 +01:00
Rishabh Garg
42f9d392a3
Added mailgun provider for sending emails (#15896)
closes https://github.com/TryGhost/Team/issues/2309

- adds new mailgun provider to send out batch emails
- updates sending service to send email id for mailgun provider, allows tagging mail with email id
2022-11-30 16:21:58 +05:30
Naz
0cfef77a01
Removed db schema config from e2e test coverage
refs a2d487e074

- Same reasoning as in referenced commit: "Database schema definition file is a special type of "configuration" file containing mostly static declarations. This sort of code should not be tested by unit tests, rather by e2e tests"
2022-11-30 17:08:47 +07:00
Naz
130f379176
Added "frontend/helpers" to coverage ignore list
refs https://github.com/TryGhost/Toolbox/issues/486

- Frontend helpers are extremely hard to cover fully in e2e tests and are better suited to be covered by unit tests (which they have very hight coverage with)
2022-11-30 17:06:27 +07:00
Naz
aa8fe65665
Added "testmode" app to coverage ignore list
refs https://github.com/TryGhost/Toolbox/issues/486

- Testmode is a special app used for manual testing and has no value in code coverage reports
2022-11-30 16:45:22 +07:00
Naz
80decf8303
Fixed invalid query strings in tests
closes https://github.com/TryGhost/Team/issues/2324

- It seemed like the "limit" query parameter did not work properly returning multiple entries from the endpoint. In reality the whole query string was ignored because of an error in the "filter" part of the query ^_^
2022-11-30 13:31:06 +07:00
Naz
65d910f43b
Increased timeout for the import job test
refs 9fe1274fbe

- Temporary fix to let the CI pass the test. Needs investigation into why MySQL is so much slower comparing to SQLite
2022-11-30 13:25:18 +07:00
Daniel Lockyer
52fc66762a
Updated .npmignore with new files
- these files should not be included in the published package as they're
  for development purposes
2022-11-30 12:40:47 +07:00
renovate[bot]
40c4ae4e00 Update dependency @babel/plugin-proposal-decorators to v7.20.5 2022-11-30 12:28:57 +07:00
renovate[bot]
491cf44f52 Update dependency @sinonjs/fake-timers to v10 2022-11-30 12:26:53 +07:00
renovate[bot]
d6a9b464b3 Update dependency jwks-rsa to v3 2022-11-30 12:24:08 +07:00
renovate[bot]
8682100e58 Update dependency @types/mocha to v10.0.1 2022-11-30 12:23:06 +07:00
Naz
9fe1274fbe
Fixed member import job test
refs https://github.com/TryGhost/Team/issues/2326

-  The job takes considerably longer to run with MySQL, so needed a longer sleep time. It's a temporary fix to unblock a broken build. We should investigate why the job takes so long to run on MySQL
2022-11-30 11:15:12 +07:00
Naz
ff1c17d661
Bumped e2e test coverage thresholds
refs fc9f8aebc1

- With integration & regression test suites included in the e2e coverage reports the coverage has jumped up nicely. We should keep the threshold as hight as possible from accidentally reducing the coverage quality
2022-11-30 10:41:32 +07:00
Naz
fc9f8aebc1
Fixed ci test runs
refs 3e0ec26408

- The refed commit excluded integration/regression test suites from CI test runs by accident
2022-11-30 10:39:32 +07:00
renovate[bot]
fc16bf97da Update dependency postcss-cli to v10.1.0 2022-11-30 10:09:41 +07:00
renovate[bot]
b05434ca91
Update metascraper to v5.32.2 2022-11-29 17:02:07 +00:00
Kevin Ansfield
457c672c6a
Added URL transform for image cards in Lexical documents (#15890)
refs https://github.com/TryGhost/Team/issues/2225

- updated the `formatOnWrite` transform map for posts to include the new `nodes` and `transformMap` options used by `urlUtils` for transforming node payload data
- added `nodes` to the `lexicalLib` module that pulls in our default nodes to be passed in to the URL transform utilities
- added `urlTransformMap` to the `lexicalLib` module that maps transform type and data type to URL transform utility functions that accept a single URL argument
2022-11-29 16:57:01 +00:00
Elena Baidakova
86abab4f1d
Remove member's activity double scrolling (#15897)
closes TryGhost/Team#2325
2022-11-29 16:08:11 +04:00
Fabien "egg" O'Carroll
ba5b8ea33d Added email_spam_complaint_events table and model
refs https://github.com/TryGhost/Team/issues/2318

As with our other events, we've disabled destroy and edit static methods
on the bookshelf model.
2022-11-29 18:13:12 +07:00
Fabien "egg" O'Carroll
83be54af42 Added suppressions table and model
refs https://github.com/TryGhost/Team/issues/2317

This table is used for persisting the email suppression list.
We don't have a member_id column because emails, not members are suppressed.
2022-11-29 18:12:24 +07:00
Simon Backx
f5045b9bf7
Added email renderer implementation draft (#15877)
fixes https://github.com/TryGhost/Team/issues/2308

- Still has some missing pieces, but mostly works.
- Uses new handlebars template for emails
- When sending emails with the new email stability flag enabled, one
test email is now sent via the default smtp ghost mailer.
2022-11-29 11:27:17 +01:00
Simon Backx
f4fdb4fa6c
Added new email event processor (#15879)
fixes https://github.com/TryGhost/Team/issues/2310

This moves the processing of the events from the event-processor to a
new email-event-processor in the email-service package.

- The `EmailEventProcessor` only translates events from
providerId/emailId to their known emailId, memberId and recipientId, and
dispatches the corresponding events.
- Since `EmailEventProcessor` runs in a separate worker thread, we can't
listen for the dispatched events on the main thread. To accomplish this
communication, the events dispatched from the `EmailEventProcessor`
class are 'posted' via the postMessage method and redispatched on the
main thread.
- A new `EmailEventStorage` class reacts to the email events and stores
it in the database. This code mostly corresponds to the (now deleted)
subclass of the old `EmailEventProcessor`
- Updating a members last_seen_at timestamp has moved to the
lastSeenAtUpdater.
- Email events no longer store `ObjectID` because these are not
encodable across threads via postMessage
- Includes new E2E tests that test the storage of all supported Mailgun
events. Note that in these tests we run the processing on the main
thread instead of on a separate thread (couldn't do this because
stubbing is not possible across threads)

There are some missing pieces that will get added in later PRs (this PR
focuses on porting the existing functionality):
- Handling temporary failures/bounces
- Capturing the error messages of bounce events
2022-11-29 11:15:19 +01:00
Fabien 'egg' O'Carroll
a95cc4e7ac
Lowered minimum function coverage tests (#15894)
refs https://github.com/TryGhost/Ghost/pull/15877
refs https://github.com/TryGhost/Ghost/pull/15892

We've been running into issues with unit test coverage in the references
PR's and have tried adding unit tests, but the problem persists. We're
lowering this to unblock development.
2022-11-29 16:55:53 +07:00
Rishabh
64ac47f4ef Added table to store email recipient failures
refs https://github.com/TryGhost/Team/issues/2291

When sending out mails to individual recipients, its possible that recipient gets a temporary or permanent failure for receiving the mail. Temporary failures can generally get resolved after a bit when the recipient’s mail server accepts the email, unlike permanent failures. For both customer visibility and easier debugging on what went wrong while delivering to a particular recipient, we’ll store the permanent/temporary failure for a recipient.

- migration adds a new table that stores the failure information for the recipients
2022-11-29 15:19:36 +05:30
Naz
a2d487e074
Removed db schema config from test coverage
refs https://ghost.slack.com/archives/C02G9E68C/p1669707419081529?thread_ts=1669702577.532999&cid=C02G9E68C
refs https://github.com/TryGhost/Toolbox/issues/117

- Database schema definition file is a special type of "configuration" file containing mostly static declarations. This sort of code should not be tested by unit tests, rather by e2e tests
2022-11-29 14:50:28 +07:00
Naz
5dd6cb909f
Tweaked code coverage config for e2e tests
refs https://github.com/TryGhost/Toolbox/issues/475
refs https://github.com/TryGhost/Toolbox/issues/117

- The frontend/src folded would never get picked up by a code coverage tooling as scripts there are dynamically minimized and served from the server
- There's nothing to cover under frontend/public
- Logic behind these changes is the same as in second referenced issue
2022-11-29 14:22:27 +07:00
Simon Backx
709b40144d Added JSDoc for GhostMailer replyTo message option
no issue
2022-11-28 18:37:51 +01:00
Ghost CI
4a3ac50a82 Merged v5.24.2 into main 2022-11-28 17:36:37 +00:00
Ghost CI
a176d1d36f v5.24.2 2022-11-28 17:36:34 +00:00