Commit Graph

130 Commits

Author SHA1 Message Date
Simon Backx
17ec1e8937
Added email address alignment protections (#19094)
ref GRO-54
fixes GRO-63
fixes GRO-62
fixes GRO-69

When the config `hostSettings:managedEmail:enabled` is enabled, or the
new flag (`newEmailAddresses`) is enabled for self-hosters, we'll start
to check the from addresses of all outgoing emails more strictly.

- Current flow: nothing changes if the managedEmail config is not set or
the `newEmailAddresses` feature flag is not set
- When managedEmail is enabled: never allow to send an email from any
chosen email. We always use `mail.from` for all outgoing emails. Custom
addresses should be set as replyTo instead. Changing the newsletter
sender_email is not allowed anymore (and ignored if it is set).
- When managedEmail is enabled with a custom sending domain: if a from
address doesn't match the sending domain, we'll default to mail.from and
use the original as a replyTo if appropriate and only when no other
replyTo was set. A newsletter sender email addresss can only be set to
an email address on this domain.
- When `newEmailAddresses` is enabled: self hosters are free to set all
email addresses to whatever they want, without verification. In addition
to that, we stop making up our own email addresses and send from
`mail.from` by default instead of generating a `noreply`+ `@` +
`sitedomain.com` address

A more in depth example of all cases can be seen in
`ghost/core/test/integration/services/email-addresses.test.js`

Includes lots of new E2E tests for most new situations. Apart from that,
all email snapshots are changed because the from and replyTo addresses
are now included in snapshots (so we can see unexpected changes in the
future).

Dropped test coverage requirement, because tests were failing coverage
locally, but not in CI

Fixed settings test that set the site title to an array - bug tracked in
GRO-68
2023-11-23 10:25:30 +01:00
Simon Backx
b6519e0f1f
Removed usage of unquoted ids in filter strings (#19070)
fixes GRO-34
fixes GRO-33

This is a revision of a previous commit, that broke the browser tests
because changes in the data generator (requiring bookshelf had side
effects).

This adds a new way to run all tests with enforced numeric ObjectIDs.
These numeric ids cause issues if they are used withing NQL filters. So
they surface tiny bugs in our codebase.

You can run tests using this option via:
NUMERIC_IDS=1 yarn test:e2e

Removed some defensive logic that could be explained by this discovered
issue.
2023-11-21 09:45:36 +01:00
Simon Backx
d5492bd63c
Revert "Removed usage of unquoted ids in filter strings" (#19052)
Reverts TryGhost/Ghost#19031

Browser tests are failing with an unknown cause
2023-11-20 13:50:07 +00:00
Simon Backx
320eaac4c4
Removed usage of unquoted ids in filter strings (#19031)
fixes GRO-34
fixes GRO-33

This also adds a new way to run all tests with enforced numeric ObjectIDs.
These numeric ids cause issues if they are used withing NQL filters. So they
surface tiny bugs in our codebase.

You can run tests using this option via:
NUMERIC_IDS=1 yarn test:e2e

Also removed some defensive logic that could be explained by unquoted ids.
2023-11-20 14:00:31 +01:00
Sag
6db7cc8156
Removed email from Mailgun's suppression list on unsubscribe (#18922)
closes https://github.com/TryGhost/Product/issues/4075

- when a member clicks on "Unsubscribe from that list" from Apple Mail,
the member's email is put into Mailgun's Unsubscribe suppression list.
Ghost listens for "Unsubscribe" events from Mailgun, and unsubscribes
the member from all the newsletters
- now, the member is only unsubscribed from the newsletter they
unsubscribe to (not all of them)
- now, the email is also deleted from Mailgun's suppression list, so
that it doesn't affect any other membership
2023-11-13 16:56:37 -03:00
Simon Backx
14927ee24b
Added quotes to NQL filters with ids (#18958)
refs https://github.com/TryGhost/Product/issues/4120

Updated some places where we don't add quotes around ids in NQL filters,
which can be an issue when the id is a number
2023-11-13 12:00:20 +01:00
Sanne de Vries
dbe00929d3
Changed "latest posts" layout in email template on mobile (#18894)
Refs https://ghost.slack.com/archives/C019B1K4FAM/p1699127038805739

- Removed mobile styles so that latest posts are always in the same
layout, instead of in a single column on mobile
- Fixed img caption font size rendering too large on mobile
2023-11-07 11:23:03 +01:00
Sanne de Vries
a38461a00d
Fixed email paywall styling bug on mobile (#18818)
Closes https://github.com/TryGhost/Product/issues/4089
2023-11-01 14:13:51 +00:00
Kevin Ansfield
9b1f79f601
Cleaned up lexicalEditor labs flag, switched Post model to lexical-by-default (#18607)
no issue

- flag is no longer used in Admin so we can clean it up in Core too
- updated Post model to set blank document to `lexical` field rather than `mobiledoc` as a default value
- switched over to returning `mobiledoc,lexical` as default formats in Admin API
2023-10-23 17:51:34 +01:00
Chris Raible
80a6fe17d0 Added Source as the new default theme
refs TryGhost/Product#3510

- Added `TryGhost/Source` as a submodule in `ghost/core/content/themes` so `Source` will ship with Ghost (along with Casper)
- With this change, new installs will use `Source` as the default theme. Existing sites will have `Source` installed, but not activated, as this is a large change and we don't want to drastically change existing sites without warning. Users can upgrade to use `Source` simply by clicking 'Activate' in design settings.
- Updated protections to prevent users from uploading their own conflicting version of `Source`
2023-10-03 14:02:08 +02:00
Steve Larson
c0f70144fc
🐛 Fixed collection card post excerpts (#18232)
refs TryGhost/Product#3913
- posts starting with a lexical card resulted in blank space in excerpts
2023-09-19 23:12:57 +00:00
Simon Backx
e888a4e820
Added recommendation events tables (#18084)
fixes https://github.com/TryGhost/Product/issues/3852

Tech spec:
https://www.notion.so/ghost/Recommentions-Tech-Spec-598f09e4109f4eef8d9e32db7245a90c?pvs=4#d30e0081613a4f51860acd40169a4d68
2023-09-13 17:18:40 +02:00
Sag
89e9e4a2b8
Added Recommendations table (#17862)
closes https://github.com/TryGhost/Product/issues/3775

- Recommendations tech spec:
https://www.notion.so/ghost/Recommentions-Tech-Spec-598f09e4109f4eef8d9e32db7245a90c?pvs=4#d59dd64cf40144bc944319b65436a730
2023-08-30 10:33:49 +02:00
Simon Backx
935ac43584
Added recommendations CRUD api (#17845)
fixes https://github.com/TryGhost/Product/issues/3784

- Includes migrations for new permissions needed for the new endpoints
2023-08-29 15:06:57 +00:00
Simon Backx
18cf5dd582
Added migrations for donations API (#17536)
fixes https://github.com/TryGhost/Product/issues/3655

- This adds a new table for storing donation events in the database.
- Makes it possible to store stripe_products without associating it with
a tier/Ghost product (required for one time purchases).
- Updates the schema for stripe_prices to allow enum value of
`donation`, which is required to query specifically on the prices used
for donations (existing one_time is not enough).

Database changes are discussed in the Tech Spec:
https://www.notion.so/ghost/Tech-Spec-5cd6929f7960462ebcbf198176e0d899?pvs=4#1f18cc5a38294f61a091e5be63fe1059
2023-07-31 17:29:36 +02:00
Michael Barrett
184c6ae951
Retain newsletter subscriptions on suppression (#17373)
refs https://github.com/TryGhost/Product/issues/2610
2023-07-24 10:47:57 +01:00
Daniel Lockyer
6f5f608bcd Fixed resetting database in migrations tests
- due to schema changes between versions, we need to completely nuke the
  DB between these tests
- this is definitely not the best way to do it but I'll fix properly next week
2023-07-07 18:42:18 +02:00
Daniel Lockyer
289e459283 Added tests for migrations
refs https://github.com/TryGhost/DevOps/issues/39

- up until now, we've had a CI job which does a really basic test for
  migrations, but it barely functions and misses bugs all the time
- this commit removes that and switches to an actual test suite for our
  migrations, so we can ensure they function as expected
- also removes the env var hack I came up with for those migrations
  tests
- this should lead to safer migrations and faster tests
2023-07-05 11:00:06 +02:00
Simon Backx
1b33634495
🐛 Fixed sending newsletters with counter-reset CSS styles in custom HTML (#17194)
fixes https://github.com/TryGhost/Team/issues/2937

Bumps juice to 9.1.0:
- Support for 'auto' width and height attributes
- Fixed a bug with counter-reset styles
- Dependencies updates
2023-07-04 10:18:16 +02:00
Simon Backx
8b610d46b1
Cleaned up batch email utils (#17165)
no issue
2023-06-29 15:29:16 +02:00
Simon Backx
c90424542e
🐛 Fixed signup card in post plaintext and email preheader (#17163)
fixes https://github.com/TryGhost/Team/issues/3542

The signup card text was included in the post plaintext/excerpt and
email preheader
2023-06-29 14:45:30 +02:00
Simon Backx
e6dbc0bc4c
🐛 Fixed repeating text in plaintext version of emails (#17162)
fixes https://github.com/TryGhost/Team/issues/3541

The email preheader, which is only present in the html version of an
email, is also included in the plaintext version of all emails. This
results in all text being duplicated twice in plaintext emails.
2023-06-29 12:47:17 +02:00
Simon Backx
c41694f5e6
Added E2E Koenig Lexical card tests for paragraphs and signup card (#17159)
refs https://ghost.slack.com/archives/C04TMVA1D7A/p1687987311455289
2023-06-29 11:10:56 +02:00
Kuba
9601285c3d
Added bulkEmail.batchSize option to configure batch size
refs https://github.com/TryGhost/Ghost/issues/15725

This pull request adds a new configuration option for the Mailgun email
provider that allows the user to set the maximum number of recipients
per email batch via a new config option `bulkEmail.batchSize`
2023-06-26 12:57:44 +02:00
Fabien "egg" O'Carroll
bd6e476c57 Added tables to persist Collections
refs https://github.com/TryGhost/Team/issues/3427
2023-06-23 16:11:55 +02:00
Michael Barrett
6f5baca849
Add endpoint to record mail events (#16990)
refs https://github.com/TryGhost/Team/issues/3319
2023-06-23 12:22:01 +01:00
Daniel Lockyer
08e2fbeacb Removed Bluebird from tests
refs https://github.com/TryGhost/Ghost/issues/14882

- we're moving away from using Bluebird in favor of native Promises, so
  this commit removes nearly all instances from tests
2023-06-23 10:03:03 +02:00
Hannah Wolfe
6161f94910
Updated to use assert/strict everywhere (#17047)
refs: https://github.com/TryGhost/Toolbox/issues/595

We're rolling out new rules around the node assert library, the first of which is enforcing the use of assert/strict. This means we don't need to use the strict version of methods, as the standard version will work that way by default.

This caught some gotchas in our existing usage of assert where the lack of strict mode had unexpected results:
- Url matching needs to be done on `url.href` see aa58b354a4
- Null and undefined are not the same thing,  there were a few cases of this being confused
- Particularly questionable changes in [PostExporter tests](c1a468744b) tracked [here](https://github.com/TryGhost/Team/issues/3505).
- A typo see eaac9c293a

Moving forward, using assert strict should help us to catch unexpected behaviour, particularly around nulls and undefineds during implementation.
2023-06-21 09:56:59 +01:00
Naz
05bb82cf1a Added collections CRUD permissions
refs https://github.com/TryGhost/Team/issues/3220

- Added permissions for collection resources
2023-06-09 21:13:06 +07:00
Chris Raible
04e6b9763c
Added retries to a flaky batch sending test (#16591)
refs TryGhost/Team#2949
2023-04-07 17:44:11 -07:00
Chris Raible
563793c7ed
Added retries for all known flaky tests (#16582)
refs TryGhost/Team#2833

- for mocha tests, we can add `this.retries(1)` to any flaky tests
- for playwright tests, we can add `test.describe.configure({ retries:
1})` to any `describe` block
- not a long-term solution, but it should help mitigate issues with flaky
tests in short term
2023-04-07 00:37:01 -07:00
Chris Raible
16c625a630
Added retries to flaky email test (#16556)
refs TryGhost/Team#2891

- test was flaking frequently enough that we had to remove it — not a perfect fix but figure it's better to enable retries than to completely remove the test
- ran CI 5 times (x 4 environments) and it passed 5 times in a row
2023-04-05 22:01:59 -07:00
Sanne de Vries
92663ea731
Fixed email width and dark mode images in email template (#16566)
Refs https://github.com/TryGhost/Team/issues/2845

---------

Co-authored-by: Fabien "egg" O'Carroll <fabien@allou.is>
Co-authored-by: Simon Backx <simon@ghost.org>
2023-04-05 15:53:51 +02:00
Fabien "egg" O'Carroll
41e8c20d88 Deleted flakey test
refs https://github.com/TryGhost/Team/issues/2891

This test is failing more and more frequently and is being removed with the
intention of reinstating it once it has been fixed
2023-04-04 20:16:44 +07:00
Sanne de Vries
57fcfe76d8
Updated images in email template to be work with light/dark mode
refs https://github.com/TryGhost/Team/issues/2845

We needed to update the html out of the cards to include images for light
and dark mode, and then we've used CSS to show/hide them 

Co-authored-by: Fabien "egg" O'Carroll <fabien@allou.is>
2023-04-04 00:32:08 +07:00
Fabien 'egg' O'Carroll
0f72816951
Newsletter email template customisation (#16523)
Refs https://github.com/TryGhost/Team/issues/2845

Co-authored-by: Sanne de Vries <sannedv@protonmail.com>
2023-03-30 15:33:46 +02:00
Simon Backx
868adf30fd Fixed batch sending snapshot date matching
no issue

The replacement didn't match dates with only one number.
2023-03-27 10:24:34 +02:00
Sanne de Vries
387dfa59c9
Fixed issue with email template in Outlook (#16486)
Refs https://github.com/TryGhost/Team/issues/2801

- It was not possible to click latest post links in Outlook due to <a>
tag wrapping around a table
- The post meta data wouldn't display properly when centered in Outlook

---------

Co-authored-by: Simon Backx <simon@ghost.org>
2023-03-24 14:54:16 +01:00
Simon Backx
d9c92816e0 Added dynamic text truncation in email latest posts
refs https://github.com/TryGhost/Team/issues/2675

Truncate text depending on mobile/desktop and feature image.
2023-03-24 12:15:16 +01:00
Naz
fe4e9897fc
Added migration for Self-Serve Integration permissions
refs https://github.com/TryGhost/Team/issues/2790

- This migration adds permissions for Self-Serve Migration Integration to have access to Admin APIs:

POST /ghost/api/admin/db
POST /ghost/api/admin/db/media/inline
POST /ghost/api/admin/members/upload
GET  /ghost/api/admin/tags/:id
GET  /ghost/api/admin/tags/slug/:slug
2023-03-24 11:31:25 +01:00
Naz
0b107f5af5
Added migrations for Self-Serve Migration Integration and API key
refs https://github.com/TryGhost/Team/issues/2790
refs 3747df1bc8 (diff-396038cecd7a381616d00954ae18a655ae2a8af71ea65866bf09d2c7cc1b5235)

- This integration will be used to perform self-serve migrations.
- The integration will be limited to these Admin API endpoints:
POST /ghost/api/admin/db
POST /ghost/api/admin/db/media/inline
POST /ghost/api/admin/members/upload
GET   /ghost/api/admin/tags/:id
GET   /ghost/api/admin/tags/slug/:slug
2023-03-24 11:28:29 +01:00
Simon Backx
21ded0b4b8 Corrected email link in subscription box
refs https://github.com/TryGhost/Team/issues/2674

The replacement didn't work inside an attribute, so I removed it to a link that doesn't do anything.
2023-03-24 10:23:51 +01:00
Simon Backx
a9791e95cc Fixed including current email in last 3 posts in newsletters
refs https://github.com/TryGhost/Team/issues/2675

The current post could be inclnuded in the last 3 posts if it was also published. This is now fixed and covered by a test.
2023-03-24 09:20:32 +01:00
Simon Backx
0107d2bb77 Fixed subscription status not showing correctly in emails
refs https://github.com/TryGhost/Team/issues/2674

- The segment detection doesn't work outside the main post content. So the data-gh-segment attribute didn't work. It is now replaced with just a simple email replacement that is empty for a free member.
- Fixed that a trialing member was shown as 'paid'. This is now replaced with 'trialing'.

This commit also includes E2E tests for a couple of member statusses.
2023-03-24 08:55:36 +01:00
Sanne de Vries
3f0ec1af3d Updated subscription box in email preview and email template
Refs https://github.com/TryGhost/Team/issues/2738

- With all footer additions and changes, the subscription box did not match the rest of the design and is therefore updated.
2023-03-23 19:39:58 +01:00
Sanne de Vries
0455672832 Updatet latest post section in email template
Refs https://github.com/TryGhost/Team/issues/2675
2023-03-23 13:16:48 +01:00
Daniel Lockyer
2941154353
Added dynamic date matching to email sending snapshot
- we don't want to include the actual date here because it'll change
- this adds a regex to match the date and replace it with a standard
  "date" string, which won't change
2023-03-23 09:20:23 +01:00
Simon Backx
860048a89d Fixed snapshots 2023-03-22 17:03:11 +01:00
Simon Backx
55c281debf Reverted removeStyleTags from email-renderer
no issue

Some things break in some email clients with this new setting. Disabled it for now and moved the required css style to hide the member name row to @media all.
2023-03-22 16:40:35 +01:00
Simon Backx
4eebf6612a Added image dimensions to the latest posts mobile version in email
fixes https://github.com/TryGhost/Team/issues/2799

Separate image dimensions for mobile and normal images.
2023-03-22 15:54:27 +01:00