Commit Graph

1133 Commits

Author SHA1 Message Date
Simon Backx
17ec1e8937
Added email address alignment protections (#19094)
ref GRO-54
fixes GRO-63
fixes GRO-62
fixes GRO-69

When the config `hostSettings:managedEmail:enabled` is enabled, or the
new flag (`newEmailAddresses`) is enabled for self-hosters, we'll start
to check the from addresses of all outgoing emails more strictly.

- Current flow: nothing changes if the managedEmail config is not set or
the `newEmailAddresses` feature flag is not set
- When managedEmail is enabled: never allow to send an email from any
chosen email. We always use `mail.from` for all outgoing emails. Custom
addresses should be set as replyTo instead. Changing the newsletter
sender_email is not allowed anymore (and ignored if it is set).
- When managedEmail is enabled with a custom sending domain: if a from
address doesn't match the sending domain, we'll default to mail.from and
use the original as a replyTo if appropriate and only when no other
replyTo was set. A newsletter sender email addresss can only be set to
an email address on this domain.
- When `newEmailAddresses` is enabled: self hosters are free to set all
email addresses to whatever they want, without verification. In addition
to that, we stop making up our own email addresses and send from
`mail.from` by default instead of generating a `noreply`+ `@` +
`sitedomain.com` address

A more in depth example of all cases can be seen in
`ghost/core/test/integration/services/email-addresses.test.js`

Includes lots of new E2E tests for most new situations. Apart from that,
all email snapshots are changed because the from and replyTo addresses
are now included in snapshots (so we can see unexpected changes in the
future).

Dropped test coverage requirement, because tests were failing coverage
locally, but not in CI

Fixed settings test that set the site title to an array - bug tracked in
GRO-68
2023-11-23 10:25:30 +01:00
Chris Raible
a97e2b823e
Added error handling to Sentry's beforeSend (#19109)
refs TryGhost/Product#4175

- Added error handling to Sentry's beforeSend function in both Admin and
Core, so if there is any error in beforeSend, we will still send the
unmodified event to Sentry
- This is in response to an incident yesterday wherein the beforeSend
function threw an error due to an unexpected missing value in the
exception. The event sent to Sentry was the error in the beforeSend
function, and the original error never reached Sentry.
- If the original event had reached Sentry, even if unmodified by the
logic in beforeSend, we could have been alerted to the issue sooner and
more easily identified all affected sites.
- Also added defensive logic to protect for certain values in the
exception passed to beforeSend not existing and added unit tests for the
beforeSend function in admin and core
2023-11-23 02:46:50 +00:00
Simon Backx
b6519e0f1f
Removed usage of unquoted ids in filter strings (#19070)
fixes GRO-34
fixes GRO-33

This is a revision of a previous commit, that broke the browser tests
because changes in the data generator (requiring bookshelf had side
effects).

This adds a new way to run all tests with enforced numeric ObjectIDs.
These numeric ids cause issues if they are used withing NQL filters. So
they surface tiny bugs in our codebase.

You can run tests using this option via:
NUMERIC_IDS=1 yarn test:e2e

Removed some defensive logic that could be explained by this discovered
issue.
2023-11-21 09:45:36 +01:00
Ronald Langeveld
83a1060983
Added last redeemed property to Offers (#19066)
refs https://github.com/TryGhost/Product/issues/4153

- wired up a new last_redeemed prop to the Offers API endpoint.
2023-11-21 08:02:15 +00:00
Michael Barrett
55392646e1
Refined cache invalidation when updating a user (#19028)
refs https://github.com/TryGhost/Arch/issues/101

Refined the cache invalidation logic so that when updating a user, we
only invalidate the cache when an attribute of the user that is used on
the frontend changes.
2023-11-20 14:14:30 +00:00
Simon Backx
d5492bd63c
Revert "Removed usage of unquoted ids in filter strings" (#19052)
Reverts TryGhost/Ghost#19031

Browser tests are failing with an unknown cause
2023-11-20 13:50:07 +00:00
Simon Backx
320eaac4c4
Removed usage of unquoted ids in filter strings (#19031)
fixes GRO-34
fixes GRO-33

This also adds a new way to run all tests with enforced numeric ObjectIDs.
These numeric ids cause issues if they are used withing NQL filters. So they
surface tiny bugs in our codebase.

You can run tests using this option via:
NUMERIC_IDS=1 yarn test:e2e

Also removed some defensive logic that could be explained by unquoted ids.
2023-11-20 14:00:31 +01:00
Kevin Ansfield
1be490ae9e
Reduced size of social metadata images (#19048)
refs https://github.com/TryGhost/Product/issues/4140

- added `social-image` image size to our `internalImagesSizes` list with a max-width of 1200
- extracted image utils from `{{img_url}}` helper to a utils file for re-use
- updated `getImageDimensions` method that reads image dimensions and modifies the finalised `metaData` object before use to adjust dimensions and associated URLs to match max width of 1200px
2023-11-20 12:39:51 +00:00
Simon Backx
32d0d2b293 Added filter to filter members by email disabled
fixes GRO-23

Co-authored-by: Djordje Vlaisavljevic <dzvlais@gmail.com>
Co-authored-by: Sag <guptazy@gmail.com>
2023-11-16 19:16:20 +01:00
Sag
69ee4a5ff5
Added option to unsubscribe in one-click from emails (#19032)
refs https://github.com/TryGhost/Product/issues/4051
- added a "List-Unsubscribe" header to emails, in compliance with the
RFC 8058 requirements
- Gmail, Apple Mail, Yahoo Mail, and other popular email clients offer
an option to unsubscribe in one-click, based on the "List-Unsubscribe"
header. Some require an HTTPS endpoint, some a mailto address; both
options are provided in the "List-Unsubscribe" header

Co-authored-by: Simon Backx <simon@ghost.org>
Co-authored-by: Djordje Vlaisavljevic <dzvlais@gmail.com>
2023-11-16 14:50:24 -03:00
Simon Backx
e3cdc24d6b
Added test to test default theme against linked gscan version (#19020)
fixes GRO-32

If we ever introduce errors in the linked source theme and linked gscan
version combination, this test will catch it.
2023-11-16 12:03:00 +00:00
Simon Backx
3a78cf48c9
Fixed deleting session when requesting identity for invalid session (#19017)
ref https://ghost.slack.com/archives/C02G9E68C/p1700129928489809

- When the GET /api/session endpoint is called, the session is deleted
if it is invalid
- We don't have a body parser for this GET endoint, and the request
object was passed to the deleteSession handler. This caused a type error
(cannot read properties of undefined)
- We had dangling promise because deleteSession is async and wasn't
awaited, causing random errors in tests
- Added a test that would have caught this earlier
2023-11-16 11:01:50 +00:00
Ronald Langeveld
2bd597fe0d
Added createdAt prop to Offers API request. (#19012)
refs https://github.com/TryGhost/Product/issues/4153

- We need use the `created_at` timestamp in the new AdminX offers. The
API doesn't return that value.
- With this change the API returns the created_at property so that we
can consume it.
---

<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖[[deprecated]](https://githubnext.com/copilot-for-prs-sunset)
Generated by Copilot at dc282af</samp>

This pull request adds a `createdAt` property to the offer domain model,
data transfer object, and repository. This allows tracking and auditing
the creation and modification of offers and offer codes in
`ghost/offers`.
2023-11-16 17:33:01 +07:00
Simon Backx
75bb53f065
🔒 Added support for logging out members on all devices (#18935)
fixes https://github.com/TryGhost/Product/issues/3738
https://www.notion.so/ghost/Member-Session-Invalidation-13254316f2244c34bcbc65c101eb5cc4

- Adds the transient_id column to the members table. This defaults to
email, to keep it backwards compatible (not logging out all existing
sessions)
- Instead of using the email in the cookies, we now use the transient_id
- Updating the transient_id means invalidating all sessions of a member
- Adds an endpoint to the admin api to log out a member from all devices
- Added the `all` body property to the DELETE session endpoint in the
members API. Setting it to true will sign a member out from all devices.
- Adds a UI button in Admin to sign a member out from all devices
- Portal 'sign out of all devices' will not be added for now

Related changes (added because these areas were affected by the code
changes):
- Adds a serializer to member events / activity feed endpoints - all
member fields were returned here, so the transient_id would also be
returned - which is not needed and bloats the API response size
(`transient_id` is not a secret because the cookies are signed)
- Removed `loadMemberSession` from public settings browse (not used
anymore + bad pattern)

Performance tests on site with 50.000 members (on Macbook M1 Pro):
- Migrate: 6s (adding column 4s, setting to email is 1s, dropping
nullable: 1s)
- Rollback: 2s
2023-11-15 17:10:28 +01:00
Steve Larson
0fe573b1d0
🐛 Redirected email previews to /email/ route (#18976)
closes TryGhost/Product#4136
- the `/p/` route is only intended for drafts, not published content
(e.g. sent newsletters)
- email-only posts (newsletters) do not get assigned a slug, and could
still be viewed at `/p/:uuid`, which didn't hide paid/member content
2023-11-14 13:31:41 -06:00
Sag
6db7cc8156
Removed email from Mailgun's suppression list on unsubscribe (#18922)
closes https://github.com/TryGhost/Product/issues/4075

- when a member clicks on "Unsubscribe from that list" from Apple Mail,
the member's email is put into Mailgun's Unsubscribe suppression list.
Ghost listens for "Unsubscribe" events from Mailgun, and unsubscribes
the member from all the newsletters
- now, the member is only unsubscribed from the newsletter they
unsubscribe to (not all of them)
- now, the email is also deleted from Mailgun's suppression list, so
that it doesn't affect any other membership
2023-11-13 16:56:37 -03:00
Simon Backx
14927ee24b
Added quotes to NQL filters with ids (#18958)
refs https://github.com/TryGhost/Product/issues/4120

Updated some places where we don't add quotes around ids in NQL filters,
which can be an issue when the id is a number
2023-11-13 12:00:20 +01:00
Djordje Vlaisavljevic
057d952e27
Made new recommendation email more flexible (#18915)
refs https://github.com/TryGhost/Product/issues/4095

- Removes `min-height` of the card in order to make the template more
flexible half-empty states look better (missing description, short
description, featured image...)
- Fixes wrong variables used in Outlook-specific template
2023-11-08 16:44:48 +00:00
Simon Backx
986296cac9 🐛 Fixed unsubscribe button requiring manual action + wrong confirmation
fixes https://github.com/TryGhost/Product/issues/4118

The newsletter uuids were not passed when fetching all the members current newsletters. Therefore, Portals logic broke to remove all newsletters that matched the uuid that was passed to the unsubscribe link. No newsletters were removed, still the notification toast said that the member was unsubscribed from the newsletter.
2023-11-08 15:05:47 +01:00
Sanne de Vries
dbe00929d3
Changed "latest posts" layout in email template on mobile (#18894)
Refs https://ghost.slack.com/archives/C019B1K4FAM/p1699127038805739

- Removed mobile styles so that latest posts are always in the same
layout, instead of in a single column on mobile
- Fixed img caption font size rendering too large on mobile
2023-11-07 11:23:03 +01:00
Nick Moreton
b9b1502772
🐛Changed "free_signups" to "signups" in Post Exporter (#18883)
ref https://github.com/TryGhost/Product/issues/4110

Made this change to increase clarity in data export

---

<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 6c0508d</samp>

Renamed a column in posts export data and updated the corresponding test
case. This change makes the export data more consistent and clear for
users who have different member features enabled.
2023-11-07 09:52:28 +00:00
Daniel Lockyer
9c6dc044b3 Skipped random flaky HTML -> Lexical test
refs https://github.com/TryGhost/Product/issues/4086

- this test is randomly timing out so we're disabling it until we figure out
  the cause
2023-11-07 09:50:53 +01:00
Steve Larson
ae53464cac
Enabled emoji picker in editor (#18880)
closes TryGhost/Product#4107
- emoji picker is now available for inline emoji insertion
- open the menu with typing a : (colon), e.g. 
2023-11-06 16:07:55 +00:00
Jono M
fa3898113b
Updated publishing spec to use a more consistent timezone (#18873)
no issue

---

<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 474a274</samp>

This pull request fixes a timezone display bug in the admin and post
settings. It also updates the `publishing.spec.js` file to test the
timezone functionality with a consistent option.
2023-11-06 07:49:43 +00:00
Jono M
e8e0d84d50
Deleted old Ember settings pages (#18740)
refs https://github.com/TryGhost/Product/issues/4055

---

<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 3b24693</samp>

Removed unused components related to announcement settings, custom theme
settings, and file upload. These components were part of a deprecated
feature or a legacy UI.
2023-11-06 06:46:31 +00:00
Simon Backx
bf859c38ee
🐛 Fixed showing recommendations on custom welcome pages (#18864)
no issue

When a custom welcome page is set for a tier, the recommendations modal
didn't show. If recommendations were disabled, there was also no toast
to confirm the sign up.

To fix this, we'll need to set the success and action query parameters
on the welcome page, but only if it is not an external site.
2023-11-03 15:02:46 +00:00
Simon Backx
fee402a340
🐛 Fixed adding recommendation with URL redirect breaking one-click-subscribe (#18863)
fixes https://github.com/TryGhost/Product/issues/4102

E.g. you recommend myghostsite.com, while that site redirects all
traffic to [www.myghostsite.com](#):

The redirect causes CORS issues, which means we cannot detect
one-click-subscribe support.
- This is fixed by moving the whole detection to the backend, which has
the additional benefit that we can update it in the background without
the frontend, and update it on every recommendation change.
- This change also fixes existing recommendations by doing a check on
boot (we can move this to a background job in the future).
2023-11-03 15:02:45 +01:00
Sag
b3c8055efe
Fixed email_disabled field after member update in Admin (#18827)
closes https://github.com/TryGhost/Product/issues/4046
- when editing the member's email in Admin, the email_disabled field was
not recalculated, making it inconsistent with the suppression list
- now, if the new email is part of the suppression list, we set
email_disabled to true. Otherwise set it to false
2023-11-02 17:15:03 +00:00
Michael Barrett
213e54aa71
Fixed error caused by uploading empty redirects YAML file (#18820)
no refs

Fixed error caused by uploading empty redirects YAML file:

```
Cannot read properties of undefined (reading '302')
```

This error was occurring due to `yaml.load` returning `undefined` when
the provided yaml file was empty. I've made the check on the return
value of `yaml.load` stricter (i.e we only want an `object`) to prevent
this error from occurring.
2023-11-02 08:17:52 +00:00
Ghost CI
2c9d42f599 Merged v5.71.1 into main 2023-11-01 16:53:42 +00:00
Kevin Ansfield
281fbc973e
🐛 Fixed mobiledoc and lexical content being available via Content API
refs https://github.com/TryGhost/Product/issues/4088

The Content API should not expose the lexical/mobiledoc source content because it's not membership-gated and although not used at the present time may in future contain additional internal metadata. We were handling this for the more-typical `?formats` param but it was still possible to access this data using the `?fields` param.

- updated post mapper used in our API output serializers to strip the `mobiledoc` and `lexical` fields ready for API output
- credits to Prathap Puthran for reporting
2023-11-01 16:10:41 +01:00
Sanne de Vries
a38461a00d
Fixed email paywall styling bug on mobile (#18818)
Closes https://github.com/TryGhost/Product/issues/4089
2023-11-01 14:13:51 +00:00
Simon Backx
97d0cddb50
Added indexes to members_created_events and members_subscription_created_events (#18805)
fixes https://github.com/TryGhost/Product/issues/4085

Increases the performance for the post analytics export by adding new
indexes. These indexes are used when counting the amount of (paid)
subscribers that were attributed to a given post. With the indexes, the
time required to export 700 posts with 300k members decreases from 40s
to 0.6s.

Tests show that adding these indexes should be very fast (< 1 s) if the
tables contain up to 300k rows.
2023-10-31 15:11:24 +00:00
Hannah Wolfe
0049b74a2d Added well-known assetlinks.json to allowed files
refs: https://github.com/google/digitalassetlinks/blob/master/well-known/specification.md
refs: https://github.com/google/digitalassetlinks/blob/master/well-known/details.md

- allow themes to include assetlinks.json files to comply with the Google digital asset links spec
2023-10-31 15:10:58 +00:00
Sag
6db0deb58e
Fixed missing favicon layout in the Recommendations template (#18766)
refs https://github.com/TryGhost/Product/issues/3940
- when a favicon url is null, don't render the img element at all
- when a favicon fails to load, hide the img element from the DOM
2023-10-25 19:13:37 +00:00
Daniel Lockyer
96b7ba4755 Fixed minor Node deprecation warning
- resolves `DeprecationWarning: In future versions of Node.js, fs.rmdir(path, { recursive: true }) will be removed. Use fs.rm(path, { recursive: true }) instead` in tests
2023-10-25 18:06:44 +02:00
Simon Backx
6cc19e1851
Added List-Unsubscribe https endpoint (#18758)
refs TryGhost/Product#4052
2023-10-25 16:16:31 +02:00
Sag
1b82efe5d2
Added Recommendations feature (#18743)
refs https://github.com/TryGhost/Product/issues/3744


Co-authored-by: Simon Backx <simon@ghost.org>
Co-authored-by: Djordje Vlaisavljevic <dzvlais@gmail.com>
2023-10-24 11:00:11 -03:00
Kevin Ansfield
9b1f79f601
Cleaned up lexicalEditor labs flag, switched Post model to lexical-by-default (#18607)
no issue

- flag is no longer used in Admin so we can clean it up in Core too
- updated Post model to set blank document to `lexical` field rather than `mobiledoc` as a default value
- switched over to returning `mobiledoc,lexical` as default formats in Admin API
2023-10-23 17:51:34 +01:00
Princi Vershwal
3324009381
Added fix for browser test (#18710) 2023-10-20 16:06:50 +05:30
Jono M
caae0edf19
Fixed and restored new AdminX E2E tests (#18606)
refs https://github.com/TryGhost/Product/issues/3831

---

### <samp>🤖 Generated by Copilot at 539c2d3</samp>

This pull request updates and adds some test cases for the date picker
and newsletter features in the Ghost admin panel. It introduces a test
helper function for the date picker in `editor-test.js`, and removes a
redundant test case from `publish-flow-test.js`. It also adds two new
test cases in `publishing.spec.js` using the Playwright framework to
verify the timezone and recipient settings for publishing posts.
2023-10-20 10:00:35 +01:00
Chris Raible
5a5a756415
Updated theme fixtures for casper and source (#18677)
no issue

- Casper and Source theme files were out of date — this commit updates the theme fixtures, and fixes up a few tests to pass with the updated themes
2023-10-18 14:51:35 -07:00
Sag
5c43945b94
Revert "Added Recommendations feature (#18664)" (#18694)
refs https://github.com/TryGhost/Product/issues/3744

- this reverts GA commit 4e338114de, as
launch has been postponed by a week
2023-10-18 17:03:35 +00:00
Steve Larson
a521c23cd3
🐛 Prevented subsequent lexical conversion wiping out content (#18672)
closes TryGhost/Product#4032
- the api flag ?convert_to_lexical converts a mobiledoc string to
lexical
- if run on a post/page with lexical content, would null it out
2023-10-17 20:38:51 +00:00
Sag
9ba1d47b04
Updated copy in the Recommendation email notification (#18671)
closes https://github.com/TryGhost/Product/issues/4031
2023-10-17 17:03:24 +00:00
Sag
22618aad65
Fixed broken link in Recommendations email notification (#18666)
no issue

- paths in the new settings are not using /settings-x/... anymore
2023-10-17 10:44:28 -03:00
Sag
4e338114de
Added Recommendations feature (#18664)
refs https://github.com/TryGhost/Product/issues/3744

Co-authored-by: Simon Backx <simon@ghost.org>
Co-authored-by: Djordje Vlaisavljevic <dzvlais@gmail.com>
2023-10-17 13:04:35 +00:00
Sam Lord
7f67e98e28 Memoized the getStripeAccountId function
no issue

Some flaky tests found, and it seems as though they're being caused by an invalid Stripe account id.

It's possible that by re-using the worker after a test which calls `setupStripe` could cause some Stripe functionality to not work.
2023-10-16 22:14:10 +01:00
Djordje Vlaisavljevic
08f81dbf90
Updated recommendations design (#18614)
refs https://github.com/TryGhost/Product/issues/4024, https://github.com/TryGhost/Product/issues/3938

- Improved email notification template
- Added `for you` next to the click/subscribe stats

---------

Co-authored-by: Sag <guptazy@gmail.com>
2023-10-16 18:04:15 +01:00
Daniel Lockyer
3a564237a5 Temporarily increased timeout for saving Offer in tests
- if Stripe is slow, this might take longer than we expect to finish
- this helps prevent flaky tests in CI
2023-10-16 17:37:42 +02:00