Commit Graph

274 Commits

Author SHA1 Message Date
Aileen Booker
75874151fd Removed ModelEventsAnalytics
refs https://linear.app/tryghost/issue/BIZ-6/[wip]-update-segment-events

- With the removal of the `integration.added` event, we have no more model events remaining to listen to for our analytics
- Removal of the function entirely seems the easier and more straightforward way
2024-01-18 10:29:56 -04:00
Aileen Booker
e4b9305e2a Removed unneeded analytics for model events
refs https://linear.app/tryghost/issue/BIZ-6/[wip]-update-segment-events

- Removed model events to listen to: `post.published`, `page.published`, and `theme.uploaded` in segment service,  as we're not actively using those.
- Updated tests to reflect the changes (from 4 events to 1 model event)
2024-01-18 10:29:56 -04:00
Michael Barrett
1263cf148e
Updated Sentry env to use PRO_ENV when available (#19441)
refs
[ARCH-33](https://linear.app/tryghost/issue/ARCH-33/fix-sentry-environment)

To ensure that we are correctly identifying the environment that data is
being sent to Sentry from, we can use the `PRO_ENV` environment variable
if it is available. This will be set to `production` in production and
`staging` in staging. If `PRO_ENV` is not available, we will fall back
to retrieving the environment from config (`env`)
2024-01-05 12:10:39 +00:00
Simon Backx
3f6ea04c43
Added portal default plan setting (#19238)
fixes PROD-61

This adds a new default plan setting. It defaults to yearly, which is
the current default selected interval in Portal.

Behind the new portal improvements feature flag, the default plan can be
changed. It will also change automatically if the available intervals
are changed.

This PR also wires up passing the new setting to the Portal preview.
2023-12-06 11:39:58 +01:00
Simon Backx
3687feca07
Updated support email address verification for DMARC changes (#19147)
fixes GRO-71

- Current flow: unchanged
- New managed flow: verification required
- New managed flow with custom sending domain: only verification
required for different domains
- Self hosters (feature flag): no verification required
2023-11-28 15:06:58 +01:00
Fabien 'egg' O'Carroll
616247b6d5
Indexed over members_newsletters(newsletter_id, member_id) (#19156)
refs https://github.com/TryGhost/Product/issues/4181

We were seeing slow queries when joining on this table, and the index
speeds them up. The down migration is tricky because when we add the
index MySQL can optimise away some `KEY` indexes on the `newsletter_id`
column. When we then go to remove the newly created index, there is no
index for the FK! 

We also remove the use of `force index` as 1. the index we're forcing is
optimised away and 2. we don't need it anymore!


Co-authored-by: Daniel Lockyer <hi@daniellockyer.com>
2023-11-28 12:20:14 +00:00
Simon Backx
17ec1e8937
Added email address alignment protections (#19094)
ref GRO-54
fixes GRO-63
fixes GRO-62
fixes GRO-69

When the config `hostSettings:managedEmail:enabled` is enabled, or the
new flag (`newEmailAddresses`) is enabled for self-hosters, we'll start
to check the from addresses of all outgoing emails more strictly.

- Current flow: nothing changes if the managedEmail config is not set or
the `newEmailAddresses` feature flag is not set
- When managedEmail is enabled: never allow to send an email from any
chosen email. We always use `mail.from` for all outgoing emails. Custom
addresses should be set as replyTo instead. Changing the newsletter
sender_email is not allowed anymore (and ignored if it is set).
- When managedEmail is enabled with a custom sending domain: if a from
address doesn't match the sending domain, we'll default to mail.from and
use the original as a replyTo if appropriate and only when no other
replyTo was set. A newsletter sender email addresss can only be set to
an email address on this domain.
- When `newEmailAddresses` is enabled: self hosters are free to set all
email addresses to whatever they want, without verification. In addition
to that, we stop making up our own email addresses and send from
`mail.from` by default instead of generating a `noreply`+ `@` +
`sitedomain.com` address

A more in depth example of all cases can be seen in
`ghost/core/test/integration/services/email-addresses.test.js`

Includes lots of new E2E tests for most new situations. Apart from that,
all email snapshots are changed because the from and replyTo addresses
are now included in snapshots (so we can see unexpected changes in the
future).

Dropped test coverage requirement, because tests were failing coverage
locally, but not in CI

Fixed settings test that set the site title to an array - bug tracked in
GRO-68
2023-11-23 10:25:30 +01:00
Simon Backx
75bb53f065
🔒 Added support for logging out members on all devices (#18935)
fixes https://github.com/TryGhost/Product/issues/3738
https://www.notion.so/ghost/Member-Session-Invalidation-13254316f2244c34bcbc65c101eb5cc4

- Adds the transient_id column to the members table. This defaults to
email, to keep it backwards compatible (not logging out all existing
sessions)
- Instead of using the email in the cookies, we now use the transient_id
- Updating the transient_id means invalidating all sessions of a member
- Adds an endpoint to the admin api to log out a member from all devices
- Added the `all` body property to the DELETE session endpoint in the
members API. Setting it to true will sign a member out from all devices.
- Adds a UI button in Admin to sign a member out from all devices
- Portal 'sign out of all devices' will not be added for now

Related changes (added because these areas were affected by the code
changes):
- Adds a serializer to member events / activity feed endpoints - all
member fields were returned here, so the transient_id would also be
returned - which is not needed and bloats the API response size
(`transient_id` is not a secret because the cookies are signed)
- Removed `loadMemberSession` from public settings browse (not used
anymore + bad pattern)

Performance tests on site with 50.000 members (on Macbook M1 Pro):
- Migrate: 6s (adding column 4s, setting to email is 1s, dropping
nullable: 1s)
- Rollback: 2s
2023-11-15 17:10:28 +01:00
Simon Backx
986296cac9 🐛 Fixed unsubscribe button requiring manual action + wrong confirmation
fixes https://github.com/TryGhost/Product/issues/4118

The newsletter uuids were not passed when fetching all the members current newsletters. Therefore, Portals logic broke to remove all newsletters that matched the uuid that was passed to the unsubscribe link. No newsletters were removed, still the notification toast said that the member was unsubscribed from the newsletter.
2023-11-08 15:05:47 +01:00
Michael Barrett
213e54aa71
Fixed error caused by uploading empty redirects YAML file (#18820)
no refs

Fixed error caused by uploading empty redirects YAML file:

```
Cannot read properties of undefined (reading '302')
```

This error was occurring due to `yaml.load` returning `undefined` when
the provided yaml file was empty. I've made the check on the return
value of `yaml.load` stricter (i.e we only want an `object`) to prevent
this error from occurring.
2023-11-02 08:17:52 +00:00
Simon Backx
97d0cddb50
Added indexes to members_created_events and members_subscription_created_events (#18805)
fixes https://github.com/TryGhost/Product/issues/4085

Increases the performance for the post analytics export by adding new
indexes. These indexes are used when counting the amount of (paid)
subscribers that were attributed to a given post. With the indexes, the
time required to export 700 posts with 300k members decreases from 40s
to 0.6s.

Tests show that adding these indexes should be very fast (< 1 s) if the
tables contain up to 300k rows.
2023-10-31 15:11:24 +00:00
Jono M
81d2fa09da
Redesigned Admin settings (#18545)
refs https://github.com/TryGhost/Product/issues/3832

Last piece of cleanup, removing the old feature flag. See
https://github.com/TryGhost/Ghost/pull/18360 for the commit that
actually switched over the routes.
2023-10-10 07:37:01 +01:00
Steve Larson
dcb9bfa7fc
Support twitter embeds using x as host (#18550)
closes TryGhost/Product#4017
- fixed users reporting x.com not creating twitter embed
2023-10-09 15:55:26 -05:00
Simon Backx
9abd466397
Renamed reason to description in recommendations table (#18527)
fixes https://github.com/TryGhost/Product/issues/4005

We no longer use the 'reason' of a recommendation, but allow a flexible
description instead. Because this is a breaking change in the API, we do
this before making this feature GA.
- Added new database utils for renaming a column
- Added new migration to rename the column
- Updated all references in code
2023-10-09 11:19:44 -03:00
Chris Raible
80a6fe17d0 Added Source as the new default theme
refs TryGhost/Product#3510

- Added `TryGhost/Source` as a submodule in `ghost/core/content/themes` so `Source` will ship with Ghost (along with Casper)
- With this change, new installs will use `Source` as the default theme. Existing sites will have `Source` installed, but not activated, as this is a large change and we don't want to drastically change existing sites without warning. Users can upgrade to use `Source` simply by clicking 'Activate' in design settings.
- Updated protections to prevent users from uploading their own conflicting version of `Source`
2023-10-03 14:02:08 +02:00
Sam Lord
fd7ead6ce6 Upgraded to latest version of @tryghost/request
refs: https://github.com/TryGhost/Product/issues/3782

Latest version of request avoids using the OS-level name resolution system. This prevents long shutdowns from occuring where the OS is blocking the process from exiting cleanly.

The new version uses `got` 13, which has many breaking changes. Some of these are resolved within @tryghost/request (like response errors), but input parameters need to be changed.
2023-09-26 12:45:14 +01:00
Daniel Lockyer
500507ff21 Moved labs unit test to correct folder
refs https://github.com/TryGhost/DevOps/issues/72

- labs.js is in the shared folder, so I've moved it to the corresponding
  folder to make understanding the tests easier
2023-09-25 11:24:32 +02:00
Simon Backx
b82a93beac
Added recommendation_notifications column to users table (#18304)
fixes https://github.com/TryGhost/Product/issues/3939
2023-09-22 15:35:55 +00:00
Naz
1fd4861e97 Cleaned up fixtures from latest collections entry
refs https://github.com/TryGhost/Arch/issues/95
refs 45c1a82909

- Latest collection has become "dynamic" in the referenced commit, this means we don't need stored entries in the database.
2023-09-22 22:23:30 +08:00
Ronald Langeveld
851504030c
Added Support Email verification modals to Admin X (#18152)
refs
https://www.notion.so/ghost/df5bdea8f7ea4aca9d25eceb6a1bf34c?v=be2f15b6b58b4c27a0e11374282bead0&p=163762d9513a4e6dbd60c28e19228fdc&pm=s

- Added a modal to confirm that the new support email has been verified.
- to achieve that a couple of adjustments had to be made
- Updated the RoutingProvider to handle routes with query params.
- Added a new useQueryParams hook to grab query params where needed.
- wired up the email verification api.
- added feature flags / labs logic to the core package with the new URL and updated test.

---

<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 3ff8add</samp>

This pull request adds email verification functionality for the support
email address in the portal settings. It fixes a bug in the routing
provider, adds a new API function, a new custom hook, and a new modal
component to handle the verification process. It also updates the
settings query with the verified email address.
2023-09-15 16:00:24 +07:00
Kevin Ansfield
12621d8157
Added core content API integration fixture (#18114)
refs https://github.com/TryGhost/Product/issues/3874

- the new collections card needs to access the Content API rather than the Admin API in order to show the card as it will appear on the front-end but we don't have a default integration that can be fetched via the Admin API for Admin to use when fetching from the Content API
- adds a new "Ghost Core Content API" integration with the `core` type so that it can be read via the `/admin/integrations/` endpoint and used in Admin to make Content API requests
2023-09-13 17:04:49 +01:00
Simon Backx
e888a4e820
Added recommendation events tables (#18084)
fixes https://github.com/TryGhost/Product/issues/3852

Tech spec:
https://www.notion.so/ghost/Recommentions-Tech-Spec-598f09e4109f4eef8d9e32db7245a90c?pvs=4#d30e0081613a4f51860acd40169a4d68
2023-09-13 17:18:40 +02:00
Sag
f1cd6432a8
Added new setting to enable Recommendations (#17856)
closes https://github.com/TryGhost/Product/issues/3772

- Recommendations are disabled by default and will get enabled when a
first recommendation is added. The "recommendations_enabled" setting
will be not shown explicitly in Admin, it will rather be set implicitly
based on whether there is at least one recommendation
- The "recommendations_enabled" setting belongs to a new setting group
called "recommendations"

Tech Spec: https://www.notion.so/ghost/Recommentions-Tech-Spec-598f09e4109f4eef8d9e32db7245a90c?pvs=4
2023-08-30 08:56:21 +00:00
Sag
89e9e4a2b8
Added Recommendations table (#17862)
closes https://github.com/TryGhost/Product/issues/3775

- Recommendations tech spec:
https://www.notion.so/ghost/Recommentions-Tech-Spec-598f09e4109f4eef8d9e32db7245a90c?pvs=4#d59dd64cf40144bc944319b65436a730
2023-08-30 10:33:49 +02:00
Simon Backx
935ac43584
Added recommendations CRUD api (#17845)
fixes https://github.com/TryGhost/Product/issues/3784

- Includes migrations for new permissions needed for the new endpoints
2023-08-29 15:06:57 +00:00
Naz
d068409fac Added index on posts table for published_at column
refs https://github.com/TryGhost/Arch/issues/18

- The prev/next helpers are slow and are causing major performance issues. The helpers are using `posts.published_at` for comparisons extensively, which causes a full table scan - bad for query performance.
- We use published_at in other queries too (like default order for queries fetching all posts), so there  might be a slight performance boost across the system with this new index.
2023-08-11 17:31:13 +08:00
Simon Backx
66675a1383
Added donation notifications column (#17607)
fixes https://github.com/TryGhost/Product/issues/3691

When a site receives a donation, we'll send an email to all
administrator. They can disable this in their profile settings. This
migration adds the required column to make this possible. It defaults to
true.

Link to tech spec:
https://www.notion.so/ghost/Tech-Spec-5cd6929f7960462ebcbf198176e0d899?pvs=4#156834b0231d4968a203d797c6817d21
2023-08-07 15:31:36 +02:00
Sag
19bdb0efef
Added migrations for Tips & Donations' settings (#17576)
closes https://github.com/TryGhost/Product/issues/3668

- Tips and Donations feature offers two settings: "donations_currency", and "donations_suggested_amount"
    - "donation_currency": the currency to be used for the donation. Defaults to "USD", not nullable.
    - "donation_suggested_amount": an anchor price for the donation. Defaults to 0, not nullable.
- Both settings belong to a new group "donations"

Tech Spec: https://www.notion.so/ghost/Tech-Spec-5cd6929f7960462ebcbf198176e0d899?pvs=4#6e8b34c45f0c4c78b48c9e7725a307c8
2023-08-03 22:13:47 +02:00
Simon Backx
841e52ccfe
Added donations API (#17495)
refs https://github.com/TryGhost/Product/issues/3648

- Refactored Members API RouterController.createCheckoutSession: Split the method into smaller parts so we can reuse individual parts for the upcoming donation checkout session.
- Wired up donation checkout creation
- Added donation events
2023-07-31 16:00:52 +00:00
Simon Backx
18cf5dd582
Added migrations for donations API (#17536)
fixes https://github.com/TryGhost/Product/issues/3655

- This adds a new table for storing donation events in the database.
- Makes it possible to store stripe_products without associating it with
a tier/Ghost product (required for one time purchases).
- Updates the schema for stripe_prices to allow enum value of
`donation`, which is required to query specifically on the prices used
for donations (existing one_time is not enough).

Database changes are discussed in the Tech Spec:
https://www.notion.so/ghost/Tech-Spec-5cd6929f7960462ebcbf198176e0d899?pvs=4#1f18cc5a38294f61a091e5be63fe1059
2023-07-31 17:29:36 +02:00
Aileen Booker
eb303ec221 Fixed Segment event named wrong 2023-07-24 15:49:33 +01:00
Michael Barrett
184c6ae951
Retain newsletter subscriptions on suppression (#17373)
refs https://github.com/TryGhost/Product/issues/2610
2023-07-24 10:47:57 +01:00
Steve Larson
22441fe730
Made lexical rendering async (#17438)
-moved lexical rendering to async
-includes rendering for front end and email
-necessary to pull dynamic data into render method, e.g. collections
2023-07-20 17:48:48 -05:00
Naz
ea2defb76c Updated collection descriptions
refs https://github.com/TryGhost/Arch/issues/25

- Updated collection descriptions to be less wordy and avoid a redundant "Collection" in them.
2023-07-17 18:43:21 +08:00
Naz
acbe3a250e Added built in collection fixtures
refs https://github.com/TryGhost/Arch/issues/25

- The instance should have two built-in collections "latest" (prviously known as "index") and "featured". These have been filled through in-memory tricks before, now they should come pre-populated through fixtures mechanism.
2023-07-17 18:43:21 +08:00
Naz
382d7fa25f Removed unused collections repositories
no issue

- These repositories were leftover from first phases of collections development. Not needed any longer as we have more specialized bookshelf repositories in the core code
2023-07-14 14:20:37 +08:00
Steve Larson
42d87d1437
Updated koenig lexical packages (#17325)
no refs
-updated to get collections card changes
2023-07-12 10:59:20 -05:00
Kevin Ansfield
d8d0bc8bd9
Moved posts_meta.hide_title_and_feature_image to posts.show_title_and_feature_image (#17239)
no issue

`show_title_and_feature_image` leads to more intuitive logic in themes and we can use `posts` rather than `posts_meta` as there are no longer row-length issues with MySQL 8.

- removed original add-column migration that was never in a release
- added new add-column migration that puts
`show_title_and_feature_image` column with a default of `true` on the `posts` table
- renamed property and default value everywhere
- bumped `@tryghost/admin-api-schema` to allow the new property through at the API level
2023-07-07 16:40:22 +01:00
Chris Raible
fc7e150cc2
Fixed importer breaking any posts written in the new editor (#17230)
refs @TryGhost/Product#3551

- PostsImporter would convert the HTML from the import file into
Mobiledoc, even if the post was written in Lexical
- As a result, the imported posts would have both mobiledoc & lexical
fields populated, which prevents the post from being updated in the
Lexical editor
- Added a check to see if the post was written in Lexical, and if so,
skip the HTML > Mobiledoc conversion
2023-07-06 18:37:19 -07:00
Kevin Ansfield
2a340bcab9
Added migration and schema change for posts_meta.hide_title_and_feature_image (#17187)
closes https://github.com/TryGhost/Team/issues/3550

We want to allow an option to hide the title and feature image on a
per-page basis, to do that we need somewhere to store the setting value.
The existing `posts_meta` table is the simplest candidate, especially as
this is a single setting and we don't have a desire to introduce many
such settings.

- added migration that adds the `hide_title_and_feature_image` column to
the `posts_meta` table with a `boolean` data type and a default value of
`false` (matches behaviour of all existing pages)
- updated schema file for initial database creation
- removed property from API output via serializers to keep migration PR
minimal
2023-07-05 14:48:25 +01:00
Fabien "egg" O'Carroll
bd6e476c57 Added tables to persist Collections
refs https://github.com/TryGhost/Team/issues/3427
2023-06-23 16:11:55 +02:00
Michael Barrett
6f5baca849
Add endpoint to record mail events (#16990)
refs https://github.com/TryGhost/Team/issues/3319
2023-06-23 12:22:01 +01:00
Daniel Lockyer
08e2fbeacb Removed Bluebird from tests
refs https://github.com/TryGhost/Ghost/issues/14882

- we're moving away from using Bluebird in favor of native Promises, so
  this commit removes nearly all instances from tests
2023-06-23 10:03:03 +02:00
Hannah Wolfe
6161f94910
Updated to use assert/strict everywhere (#17047)
refs: https://github.com/TryGhost/Toolbox/issues/595

We're rolling out new rules around the node assert library, the first of which is enforcing the use of assert/strict. This means we don't need to use the strict version of methods, as the standard version will work that way by default.

This caught some gotchas in our existing usage of assert where the lack of strict mode had unexpected results:
- Url matching needs to be done on `url.href` see aa58b354a4
- Null and undefined are not the same thing,  there were a few cases of this being confused
- Particularly questionable changes in [PostExporter tests](c1a468744b) tracked [here](https://github.com/TryGhost/Team/issues/3505).
- A typo see eaac9c293a

Moving forward, using assert strict should help us to catch unexpected behaviour, particularly around nulls and undefineds during implementation.
2023-06-21 09:56:59 +01:00
Aileen Booker
d27bb8f075 Added new Stripe events to DomainEventsAnalytics
no issue

- We need to send information about Stripe being enabled or disabled in live mode to analytics
- This hooks up the Domain events listeners in the analytics service and processes this information accordingly
2023-06-14 08:18:19 -04:00
Daniel Lockyer
af8c0dc7a5 Prevented registering multiple Slack event listeners
refs https://github.com/TryGhost/Toolbox/issues/592

- in tests, we boot Ghost over and over
- this inits all the services each time
- it turns out that the Slack event listener is registered 80+ times
- to prevent this, we can check if it has already been registered, like
  we do with webhooks: 4639396c3a/ghost/core/core/server/services/webhooks/listen.js (L67-L69)
- not sure about this pattern in general, but it's something we can
  review in coming weeks
2023-06-12 15:04:11 +02:00
Naz
05bb82cf1a Added collections CRUD permissions
refs https://github.com/TryGhost/Team/issues/3220

- Added permissions for collection resources
2023-06-09 21:13:06 +07:00
Fabien 'egg' O'Carroll
92172aca8e
Wired up collections to posts endpoint (#16945)
- Added support for `include=collections` to the Posts Admin API behind a flag
- Refactored some of the collections work to support it
2023-06-07 15:06:15 +02:00
Aileen Booker
9372124d61 Decouple sentry from analytics events handlers by expecting general exception handler
no issue

- The class should not rely on being passed a specific dependency, but rather needs to communicate with types what structure and method it needs to function correctly.
- Replaced the specific dependency to `sentry` with a generic definition of what is expected.
2023-06-07 09:00:49 -04:00
Aileen Booker
5db1950090 Added tests ModelEventsAnalytics
no issue

- Added full unit tests for `ModelEventsAnalytics` class
2023-06-07 09:00:49 -04:00