Commit Graph

32020 Commits

Author SHA1 Message Date
Fabien "egg" O'Carroll
f9fdee1b28 Wired up storage of EmailSpamComplaintEvents
refs https://github.com/TryGhost/Team/issues/2337

This also removes the code which unsubscribes members from newsletters because
that is not the spec of the feature.
2022-12-01 17:26:59 +07:00
Fabien "egg" O'Carroll
9c691f3ea9 Added support for EmailSpamComplaintEvent to activity feed
We need to add relations and filterRelations to the model so that we can
correctly filter events by post_id and member_id
2022-12-01 17:26:24 +07:00
Naz
04d9cbb1eb Removed unused members settings module
no issue

- This module is marked for deletion with a todo comment.  It also popped up as least covered place in the codebase, because it was never used.
2022-12-01 16:12:09 +07:00
Simon Backx
d8187123af
Added storage for email failures (#15901)
fixes https://github.com/TryGhost/Team/issues/2332

Saves events in the database and collects error information.

Do note that we can emit the same events multiple times, and as a result
out of order. That means we should correctly handle that a delivered
event might be fired after a permanent failure. So a delivered event is
ignored if the email is already marked as failed. Also delivered_at is
reset to null when we receive a permanent failure.
2022-12-01 10:00:53 +01:00
Kevin Ansfield
a1ee04b08f Wired up title->editor keyboard handling in Lexical editor
closes https://github.com/TryGhost/Team/issues/2286

- use the `registerAPI` prop to get access to a basic API for focusing and inserting paragraphs
- replaced commented mobiledoc based title key handling with lexical handling
2022-11-30 22:04:02 +00:00
Elena Baidakova
95c3a68c34
Updated the filter naming in Post Analytics (#15898)
closes TryGhost/Team#2329
- Replace 'Received' on 'Sent' in member's filter
- Moved links for feedback analytics from chart to table
2022-11-30 17:39:37 +04:00
Simon Backx
26d51687b1
Implemented email previews and tests using new email flow (#15899)
fixes https://github.com/TryGhost/Team/issues/2330

Uses new flow for previewing and testing emails (only if email stability
flag is enabled)
2022-11-30 13:56:28 +01:00
Rishabh Garg
42f9d392a3
Added mailgun provider for sending emails (#15896)
closes https://github.com/TryGhost/Team/issues/2309

- adds new mailgun provider to send out batch emails
- updates sending service to send email id for mailgun provider, allows tagging mail with email id
2022-11-30 16:21:58 +05:30
Naz
0cfef77a01
Removed db schema config from e2e test coverage
refs a2d487e074

- Same reasoning as in referenced commit: "Database schema definition file is a special type of "configuration" file containing mostly static declarations. This sort of code should not be tested by unit tests, rather by e2e tests"
2022-11-30 17:08:47 +07:00
Naz
130f379176
Added "frontend/helpers" to coverage ignore list
refs https://github.com/TryGhost/Toolbox/issues/486

- Frontend helpers are extremely hard to cover fully in e2e tests and are better suited to be covered by unit tests (which they have very hight coverage with)
2022-11-30 17:06:27 +07:00
Naz
aa8fe65665
Added "testmode" app to coverage ignore list
refs https://github.com/TryGhost/Toolbox/issues/486

- Testmode is a special app used for manual testing and has no value in code coverage reports
2022-11-30 16:45:22 +07:00
Naz
80decf8303
Fixed invalid query strings in tests
closes https://github.com/TryGhost/Team/issues/2324

- It seemed like the "limit" query parameter did not work properly returning multiple entries from the endpoint. In reality the whole query string was ignored because of an error in the "filter" part of the query ^_^
2022-11-30 13:31:06 +07:00
Naz
65d910f43b
Increased timeout for the import job test
refs 9fe1274fbe

- Temporary fix to let the CI pass the test. Needs investigation into why MySQL is so much slower comparing to SQLite
2022-11-30 13:25:18 +07:00
Daniel Lockyer
52fc66762a
Updated .npmignore with new files
- these files should not be included in the published package as they're
  for development purposes
2022-11-30 12:40:47 +07:00
renovate[bot]
40c4ae4e00 Update dependency @babel/plugin-proposal-decorators to v7.20.5 2022-11-30 12:28:57 +07:00
renovate[bot]
491cf44f52 Update dependency @sinonjs/fake-timers to v10 2022-11-30 12:26:53 +07:00
renovate[bot]
d6a9b464b3 Update dependency jwks-rsa to v3 2022-11-30 12:24:08 +07:00
renovate[bot]
8682100e58 Update dependency @types/mocha to v10.0.1 2022-11-30 12:23:06 +07:00
Naz
9fe1274fbe
Fixed member import job test
refs https://github.com/TryGhost/Team/issues/2326

-  The job takes considerably longer to run with MySQL, so needed a longer sleep time. It's a temporary fix to unblock a broken build. We should investigate why the job takes so long to run on MySQL
2022-11-30 11:15:12 +07:00
Naz
ff1c17d661
Bumped e2e test coverage thresholds
refs fc9f8aebc1

- With integration & regression test suites included in the e2e coverage reports the coverage has jumped up nicely. We should keep the threshold as hight as possible from accidentally reducing the coverage quality
2022-11-30 10:41:32 +07:00
Naz
fc9f8aebc1
Fixed ci test runs
refs 3e0ec26408

- The refed commit excluded integration/regression test suites from CI test runs by accident
2022-11-30 10:39:32 +07:00
renovate[bot]
fc16bf97da Update dependency postcss-cli to v10.1.0 2022-11-30 10:09:41 +07:00
renovate[bot]
b05434ca91
Update metascraper to v5.32.2 2022-11-29 17:02:07 +00:00
Kevin Ansfield
457c672c6a
Added URL transform for image cards in Lexical documents (#15890)
refs https://github.com/TryGhost/Team/issues/2225

- updated the `formatOnWrite` transform map for posts to include the new `nodes` and `transformMap` options used by `urlUtils` for transforming node payload data
- added `nodes` to the `lexicalLib` module that pulls in our default nodes to be passed in to the URL transform utilities
- added `urlTransformMap` to the `lexicalLib` module that maps transform type and data type to URL transform utility functions that accept a single URL argument
2022-11-29 16:57:01 +00:00
Elena Baidakova
86abab4f1d
Remove member's activity double scrolling (#15897)
closes TryGhost/Team#2325
2022-11-29 16:08:11 +04:00
Fabien "egg" O'Carroll
ba5b8ea33d Added email_spam_complaint_events table and model
refs https://github.com/TryGhost/Team/issues/2318

As with our other events, we've disabled destroy and edit static methods
on the bookshelf model.
2022-11-29 18:13:12 +07:00
Fabien "egg" O'Carroll
83be54af42 Added suppressions table and model
refs https://github.com/TryGhost/Team/issues/2317

This table is used for persisting the email suppression list.
We don't have a member_id column because emails, not members are suppressed.
2022-11-29 18:12:24 +07:00
Simon Backx
f5045b9bf7
Added email renderer implementation draft (#15877)
fixes https://github.com/TryGhost/Team/issues/2308

- Still has some missing pieces, but mostly works.
- Uses new handlebars template for emails
- When sending emails with the new email stability flag enabled, one
test email is now sent via the default smtp ghost mailer.
2022-11-29 11:27:17 +01:00
Simon Backx
f4fdb4fa6c
Added new email event processor (#15879)
fixes https://github.com/TryGhost/Team/issues/2310

This moves the processing of the events from the event-processor to a
new email-event-processor in the email-service package.

- The `EmailEventProcessor` only translates events from
providerId/emailId to their known emailId, memberId and recipientId, and
dispatches the corresponding events.
- Since `EmailEventProcessor` runs in a separate worker thread, we can't
listen for the dispatched events on the main thread. To accomplish this
communication, the events dispatched from the `EmailEventProcessor`
class are 'posted' via the postMessage method and redispatched on the
main thread.
- A new `EmailEventStorage` class reacts to the email events and stores
it in the database. This code mostly corresponds to the (now deleted)
subclass of the old `EmailEventProcessor`
- Updating a members last_seen_at timestamp has moved to the
lastSeenAtUpdater.
- Email events no longer store `ObjectID` because these are not
encodable across threads via postMessage
- Includes new E2E tests that test the storage of all supported Mailgun
events. Note that in these tests we run the processing on the main
thread instead of on a separate thread (couldn't do this because
stubbing is not possible across threads)

There are some missing pieces that will get added in later PRs (this PR
focuses on porting the existing functionality):
- Handling temporary failures/bounces
- Capturing the error messages of bounce events
2022-11-29 11:15:19 +01:00
Fabien 'egg' O'Carroll
a95cc4e7ac
Lowered minimum function coverage tests (#15894)
refs https://github.com/TryGhost/Ghost/pull/15877
refs https://github.com/TryGhost/Ghost/pull/15892

We've been running into issues with unit test coverage in the references
PR's and have tried adding unit tests, but the problem persists. We're
lowering this to unblock development.
2022-11-29 16:55:53 +07:00
Rishabh
64ac47f4ef Added table to store email recipient failures
refs https://github.com/TryGhost/Team/issues/2291

When sending out mails to individual recipients, its possible that recipient gets a temporary or permanent failure for receiving the mail. Temporary failures can generally get resolved after a bit when the recipient’s mail server accepts the email, unlike permanent failures. For both customer visibility and easier debugging on what went wrong while delivering to a particular recipient, we’ll store the permanent/temporary failure for a recipient.

- migration adds a new table that stores the failure information for the recipients
2022-11-29 15:19:36 +05:30
Naz
a2d487e074
Removed db schema config from test coverage
refs https://ghost.slack.com/archives/C02G9E68C/p1669707419081529?thread_ts=1669702577.532999&cid=C02G9E68C
refs https://github.com/TryGhost/Toolbox/issues/117

- Database schema definition file is a special type of "configuration" file containing mostly static declarations. This sort of code should not be tested by unit tests, rather by e2e tests
2022-11-29 14:50:28 +07:00
Naz
5dd6cb909f
Tweaked code coverage config for e2e tests
refs https://github.com/TryGhost/Toolbox/issues/475
refs https://github.com/TryGhost/Toolbox/issues/117

- The frontend/src folded would never get picked up by a code coverage tooling as scripts there are dynamically minimized and served from the server
- There's nothing to cover under frontend/public
- Logic behind these changes is the same as in second referenced issue
2022-11-29 14:22:27 +07:00
Simon Backx
709b40144d Added JSDoc for GhostMailer replyTo message option
no issue
2022-11-28 18:37:51 +01:00
Ghost CI
4a3ac50a82 Merged v5.24.2 into main 2022-11-28 17:36:37 +00:00
Ghost CI
a176d1d36f v5.24.2 2022-11-28 17:36:34 +00:00
Simon Backx
2a832645dd Added getNoReplyAddress settings helper
no issue

New settings method used for https://github.com/TryGhost/Ghost/pull/15877, which shouldn't belong in the member service like it currently is.
2022-11-28 18:28:15 +01:00
Simon Backx
6e72767a50
Fixed verification trigger not working for large imports (#15887)
fixes https://github.com/TryGhost/Team/issues/2326

When importing more than 500 members, we didn't testImportThreshold at
the right time. It was called too early because the importing job was
not awaited. This also adds an E2E test for this case.
2022-11-28 18:22:10 +01:00
Elena Baidakova
1b784b5ec5
Added events to Post Analytics page (#15886)
closes TryGhost/Team#2313
- Added Sent event to Post analytics and Members feed. Now post can be
Sent or Received or Bounced.
- Excluded Delivered event from Sent filter on backend.
2022-11-28 17:43:35 +04:00
Elena Baidakova
fbf761b0ac
Fixed post title in the filter dropdown on the Members page (#15884)
closes TryGhost/Team#2265
2022-11-28 16:47:50 +04:00
Ghost CI
d9b4057fe1 Merged v5.24.1 into main 2022-11-28 11:51:06 +00:00
Ghost CI
2a28fa411b v5.24.1 2022-11-28 11:51:02 +00:00
Naz
60d066b243
🔒 Disabled editable relations by default
refs https://github.com/TryGhost/Ghost/security/advisories/GHSA-9gh8-wp53-ccc6
refs https://github.com/TryGhost/Toolbox/issues/465

- Bookshelf relations allows us to edit relational records by default, which was used liberally in the codebase.
- Not having a clear track record of editable relations left the model layer prone to triggering unwanted nested saves and created a vulnerability where members were able to edit newsletter settings.
- With explicit editable relations it's easier to keep track of relations having editable access to related records. Makes the relational data modification pattern safer to use too.
- Anyone running 5.x should update to 5.24.1

Credits: Dave McDaniel and other members of [Cisco Talos](https://talosintelligence.com/vulnerability_reports)
2022-11-28 18:40:26 +07:00
Naz
22738b1b50 🔒 Disabled editable relations by default
refs https://github.com/TryGhost/Ghost/security/advisories/GHSA-9gh8-wp53-ccc6
refs https://github.com/TryGhost/Toolbox/issues/465

- Bookshelf relations allows us to edit relational records by default, which was used liberally in the codebase.
- Not having a clear track record of editable relations left the model layer prone to triggering unwanted nested saves and created a vulnerability where members were able to edit newsletter settings.
- With explicit editable relations it's easier to keep track of relations having editable access to related records. Makes the relational data modification pattern safer to use too.
- Anyone running 5.x should update to 5.24.1

Credits: Dave McDaniel and other members of [Cisco Talos](https://talosintelligence.com/vulnerability_reports)
2022-11-28 18:39:39 +07:00
Sam Lord
60b924f173 Use headless mode by default for Playwright tests
no issue

The setting `headless: false` is superceded by using `PWDEBUG=1` as an
environment variable during debug sessions / while working on tests.
2022-11-28 10:52:38 +00:00
Elena Baidakova
1161ab98f9
Rename column to Sent on Dashboard (#15883)
closes TryGhost/Team#2321
2022-11-28 11:22:35 +04:00
renovate[bot]
79ac85e63f Update dependency lint-staged to v13.0.4 2022-11-28 08:43:37 +07:00
Ghost CI
e1279c74b4 v5.24.0 2022-11-25 15:00:47 +00:00
Kevin Ansfield
d4ea9c877d Fixed editor key commands throwing errors when range has no section
closes Sentry ADMIN-CC8
closes Sentry ADMIN-DDM
closes Sentry ADMIN-C9F

- there are situations where the editor range when key commands are triggered does not have a head section which then throws errors due to the commands assuming there will always be a section present
- added a guard to key commands that use the head section to abort early and fall back to default Mobiledoc behaviour when the section is missing
2022-11-25 08:56:38 +00:00
Gabriel Ribeiro
64329d716e
Fixed missing amp-youtube properties (#15880)
fixes: https://github.com/TryGhost/Ghost/issues/15878

- Fix `amp-youtube` component, add mandatory `width` and `height`
properties. Also change the `layout` property to `responsive`, so that
the width adapts to mobile devices.

---

Error log:
`The mandatory attribute 'height' is missing in tag 'amp-youtube'.`

Documentation `amp-youtube`:
https://amp.dev/documentation/components/amp-youtube
2022-11-25 07:26:33 +00:00