Commit Graph

130 Commits

Author SHA1 Message Date
Simon Backx
076e3c02b2
Added linking between member and subscription created events (#15693)
fixes https://github.com/TryGhost/Team/issues/2160

- Adds a `batch_id` to both events that contain the same ID if they were created at the same time.
- Removes duplicate signup/conversion events using the batch_id
- Requires an update in mongo-knex to work (refs https://ghost.slack.com/archives/C02G9E68C/p1666773313272409?thread_ts=1666767872.375009&cid=C02G9E68C)
- Some dependencies needed an update to load the latest mongo-knex
- Added tiers to membersUtils, loaded on startup (we can start to use this instead of fetching it every time)
2022-10-27 11:44:19 +02:00
Elena Baidakova
e3ab868b83
Added email feedback column (#15698)
closes TryGhost/Team#2159
- Added column to email table
- Hide the feedback tab on frontend depending on the column value

Co-authored-by: Daniel Lockyer <daniellockyer@fastmail.com>
2022-10-27 11:22:50 +04:00
Simon Backx
1f138893ff Fixed snapshot tests
refs 30ecaef329
2022-10-24 14:27:52 +02:00
Hannah Wolfe
3ab1c418bc
Updated webhook test snapshots
refs: 717a27c85c
refs: 6380b82793

- The snapshots just needed updating
2022-10-21 14:36:20 +01:00
Rishabh
d8bacf12d1 Added endpoint for fixing newsletter links
refs https://github.com/TryGhost/Team/issues/2104

- adds new bulk edit endpoint for links, updates all matching link with the current redirect url and update to new url
2022-10-20 17:50:02 +05:30
Daniel Lockyer
02c8690e87 Added ghost_subscription_id column to members_stripe_customers_subscriptions
refs https://github.com/TryGhost/Team/issues/2034

- this table will be used to link Stripe subscriptions to Ghost
  subscriptions via a foreign key that we add at a later point
- this also includes `constraintName` as the auto-generated one would be
  too long for MySQL 8
2022-10-20 10:59:36 +07:00
Rishabh Garg
318a5a809c
Added permissions for link edit endpoints (#15664)
refs https://github.com/TryGhost/Team/issues/2104

- adds edit permissions for links endpoints to fixtures
- new `bulkEdit` endpoint will use the permissions and allow fixing newsletter links via Admin
2022-10-20 09:11:26 +05:30
Rishabh Garg
60b10ad69a
Fixed permissions for links endpoint (#15656)
refs 5fcf5098a8

- links browse endpoint had permissions switched off unintentionally and was also missing the necessary permissions in fixtures.
- enables permissions for browse endpoint and adds migration insert permissions in DB
2022-10-20 08:18:29 +05:30
Daniel Lockyer
9b8c33484d
Merged v5.19.3 into main
v5.19.3
2022-10-19 06:22:38 +07:00
Simon Backx
a822c5a8c5 Added test to check if feedback buttons are hidden if alpha flag is disabled 2022-10-18 16:47:06 +02:00
Simon Backx
a01fb5f1aa
Added post_id filter and total to activity feed API (#15650)
fixes https://github.com/TryGhost/Team/issues/2091
fixes https://github.com/TryGhost/Team/issues/2089

- Added new fixtures to make testing easier for the activity feed
- Improved E2E test coverage of activity feed with separate test file
- Added data.post_id filter to enable filtering by events related to a
given post
- Fixed return types in JSDoc of test agents (TypeScript interprets
these as `typeof Agent` if we don't add `InstanceType<Agent>`)
- Added total pagination metadata to activity feed API (to allow a basic
type of pagination using filters)
2022-10-18 15:52:04 +02:00
Daniel Lockyer
2dcc4139b1
Merged v5.19.2 into main
v5.19.2
2022-10-18 17:04:23 +07:00
Simon Backx
d1e6870740
🐛 Fixed large mailgun recipient data (#15638)
fixes https://github.com/TryGhost/Team/issues/2096

When generating the recipient data for emails, the email clicks
implementation is resulting in a recipient variable being added called
replacement_xxx once for each link containing the same UUID.

This generates a lot of unnecessary data overhead for emails, and it
turns out that mailgun has a 25MB message limit. We wouldn't have come
close if we only included the uuid once.
2022-10-18 10:32:50 +02:00
Daniel Lockyer
3858f255b9 Dropped nullable status on subscriptions.tier_id
fixes https://github.com/TryGhost/Team/issues/2102

- this column was added with `nullable: true` but it should never be
  nullable, so we should drop the nullable status whilst it's easy to
2022-10-18 14:16:30 +07:00
Daniel Lockyer
c9d43b8fe1
Allowed constraintName in schema column spec
refs 0ba3d6df49

- this is used to indicate the name of the foreign key constraint and so
  we should let it through the schema checks
2022-10-18 10:29:55 +07:00
Elena Baidakova
46141efe05
Updated test snapshot after bumping Portal (#15623) 2022-10-14 20:29:46 +04:00
Elena Baidakova
e831be6bc2
Added the feedback buttons in the emails (#15619)
closes TryGhost/Team#2046
closes TryGhost/Team#2045
- Added feedback buttons markup.
- Added feedback links generation.
2022-10-14 18:12:17 +04:00
Fabien 'egg' O'Carroll
bd0f4b4b8c
Added Tier price and currency data to products table (#15366)
refs https://github.com/TryGhost/Team/issues/1765

In order to better handle deleted objects in Stripe we want to decouple
Members from Stripe.

These changes allow us to have the Tier concept completely independent
of the Stripe tables, such that the Stripe data can be generated as/when
it's needed - which will help to protect against missing data.
2022-10-14 06:40:17 +01:00
Rishabh Garg
e05889cd53
🐛 Fixed missing accent color for default content cta (#15611)
refs https://github.com/TryGhost/Casper/issues/901

- the site accent color property on default cta for upgrade link had a typo and was using wrong property
2022-10-13 13:42:21 +05:30
Naz
4db0be603f Added subscriptions table
refs https://github.com/TryGhost/Team/issues/2030

- adds `subscriptions` table to the DB schema
- this new table is aimed to support a native "subscription" primitive in Ghost
  that most resembles previously used `members_stripe_customers_subscriptions` table
2022-10-13 11:19:13 +07:00
Daniel Lockyer
74e1d52d6a Fixed incorrect maximum length definition in schema
refs https://github.com/TryGhost/Toolbox/issues/441

- whilst reviewing another PR, I noticed we were incorrectly using
  `maxLength` instead of `maxlength` in the schema column definition
- it turns out we've already been doing this wrong for a while with
  other columns
- this key is not acted upon, so the maximum column length was not applied
- fixing up the DB to the correct maximum length is something to fix in the
  future but right now, the schema does not reflect the size of the
  column that actually got created
- the fallback when `maxlength` is not provided is currently 191 [0], so
  this commit switches the schema and migrations to using the correct
  key name and column length that they are using when applied

[0]: 24670aa555/ghost/core/core/server/data/schema/commands.js (L27)
2022-10-13 09:58:19 +07:00
Daniel Lockyer
143ae857c9 Removed bool type from schema
refs https://github.com/TryGhost/Toolbox/issues/441

- we tend to have a mix of `bool` and `boolean` in the schema and
  migrations, which has become a real nit for me at this point
- we don't do any special handling between `bool` and `boolean`, it's
  just something we pass to Knex
- `bool` is an alias for `boolean` but `boolean` is actually documented - https://knexjs.org/guide/schema-builder.html#boolean
- this commit switches Ghost to only using `boolean` in the schema and
  migrations, and removes `bool` from the allowlist in tests to prevent
  us from adding it again in the future
- this should make absolutely no difference to the DB because both
  resulted in the same column
2022-10-13 09:37:38 +07:00
Daniel Lockyer
24670aa555
Fixed validating numbers as booleans in schema validator
refs https://github.com/TryGhost/Toolbox/issues/441

- I'm currently working on cleaning up our uses of `bool` and `boolean`
  in favor of `boolean`, and I've noticed we only handle converting
  numbers into booleans when the type is `bool`, so validation would
  otherwise fail
- given these can be used interchangeably, we should also support
  converting the numbers into booleans when the type is `boolean`
- this is going to get cleaned up again when I remove `bool` but this
  fixes the validation bug for now
2022-10-13 08:20:31 +07:00
jbenezech
3d44e37cbd
🐛 Fixed sitemaps with no content (#15571)
closes: https://github.com/TryGhost/Ghost/issues/14981

- Taxonomy-specific sitemaps were invalid xml when there was no data
- These invalid empty sitemaps were referenced in the index sitemap causing SEO tools to report errors
2022-10-12 14:11:19 +01:00
Daniel Lockyer
845f8d965e
Added test to validate schema structure
refs https://github.com/TryGhost/Toolbox/issues/441

- this is only v1 of the test I would like but it validates the keys on
  a column definition are part of an allowlist
- this has already uncovered a bug with `maxLength` (vs `maxlength`)
2022-10-12 19:19:33 +07:00
Daniel Lockyer
e2ba19b0df
Fixed import of bson-objectid in accordance to the typings
- there's a few different ways we can import it but I've chosen to
  append `.default` as we have done in several other places in the code
2022-10-12 14:54:35 +07:00
Simon Backx
68bdc1afea
Added post sentiment (#15592)
fixes https://github.com/TryGhost/Team/issues/2054

This change adds the sentiment and positive_feedback counts to the posts models. This change isn't really ideal because there are some problems here:
- sentiment isn't really a count
- we don't need to include the sentiment and positive_feedback as a default for posts (but the same is true for attribution)

It would make sense to move this to separate endpoints that only fetch the analytics for a given post when the analytics page is opened. But for our initial skateboard version of audience feedback this should be a good start to already see the data.
2022-10-11 17:52:14 +02:00
Elena Baidakova
1221ba5d1d
Added feedback_enabled to newsletters table (#15589)
closes TryGhost/Team#2042
- Added ability to enable audience feedback per newsletter (just on BE side).
2022-10-11 16:06:26 +04:00
Simon Backx
74d749fa63
Added members_feedback table (#15581)
fixes https://github.com/TryGhost/Team/issues/2041
2022-10-11 13:21:31 +02:00
Simon Backx
b96ff6ae4a Fixed snapshots for Portal update
refs e86e78fb6b
2022-10-05 12:52:50 +02:00
Fabien 'egg' O'Carroll
45d65663f4
Simplified link tracking related tables naming (#15480)
- Removes superfluous "link" from table names
- Fixes type definititon of dropTables util
- Updates & renames models
- Noop existing migrations to avoid unnecessary work
2022-09-29 22:08:45 +01:00
Simon Backx
0cd0fc838d
Added email track clicks column and cleaned up frontend checks (#15501)
fixes https://github.com/TryGhost/Team/issues/2008

- New column that stores email click tracking at the time it was created
- Improved frontend side checks for when to show analytics
2022-09-29 16:42:45 +02:00
Naz
8cbf913582 Increased Vary granularity for versioned requests
refs https://github.com/TryGhost/Toolbox/issues/425
refs https://github.com/TryGhost/Toolbox/issues/280

- The versioned API responses vary based on requested version (passed in request's 'accept-version' header). shared caches that sit between Ghost's origin server and the browser would be putting responses with same Vary into the same caching bucket, which is incorrect.
- This change makes response's Vary more granular and tells caching mechanisms to take 'Accept-Version' request header into account when caching.
- Informative read on the topic - https://www.fastly.com/blog/getting-most-out-vary-fastly
2022-09-28 14:48:43 +08:00
Rishabh
d886bc4b0d Fixed tests for portal bump
refs 8848fd0f59

- last commit didn't update test snapshots for new portal version bump
2022-09-28 00:04:32 +05:30
Rishabh Garg
90034577b8
Added member attribution history frontend script (#15482)
- bumps member attribution script from alpha feature to now load for all sites. The script captures recent url history in localstorage to capture correct attribution for members.
- script is only loaded on the site if members is enabled
2022-09-27 23:45:41 +05:30
Rishabh
0349acb7e3 Updated content cta to use global accent color property
refs https://github.com/TryGhost/Ghost/pull/15471#discussion_r979902374

- the accent color value used by default content cta was copying the global site property which is redundant, and can be directly used
- originally, the accentColor property was extended to allow a fallback value for content ctas, but was later removed as we added default value to global site property directly
- the accentColor property is now deprecated and will be removed in next version, as existing themes might be relying on it for custom cta helpers
2022-09-27 09:41:54 +05:30
Rishabh
1410a4237e 🐛 Fixed default content CTA message to reflect page vs post
closes https://github.com/TryGhost/Team/issues/1898

- the default content cta always used the terminology as `post` when showing message that users don't have access to some content
- this caused confusion when users were looking at a page and message showed "This post is for subscribers only"
- updates the message to correctly reflect `page` vs `post` on the default cta
2022-09-27 09:41:54 +05:30
Rishabh Garg
b99c5428d0
Added referrer attribution columns to events table (#15436)
refs TryGhost/Team#1931

- referrer source, medium and url will be stored in the events table along with rest of attribution data
- stores referrer information on two tables
  - `members_created_events` for signups
  - `members_subscription_created_events` for paid conversions
2022-09-21 19:01:36 +05:30
Hannah Wolfe
f31a50270d Removed search labs flag
- the search helper was promoted to GA and we no longer need the feature flag
2022-09-20 15:22:06 +01:00
Daniel Lockyer
74f1f1bba6
Fixed CDN URLs in tests
refs de416629e6

- I forgot to update them when making the referenced commit
2022-09-19 14:51:31 +01:00
Daniel Lockyer
1ba6d2e5a7
Updated test snapshot
refs de416629e6

- this was forgotten after I'd changed the endpoint in the referenced
  commit
2022-09-19 14:43:07 +01:00
Simon Backx
901a28f3c6
Added link_redirects and members_link_click_events tables (#15421)
closes https://github.com/TryGhost/Team/issues/1912
closes https://github.com/TryGhost/Team/issues/1913
2022-09-19 11:20:36 +02:00
Aileen Nowak
be45d4ebcf Added post stats service to return total posts in Explore endpoint
no issue

- The explore endpoint needs to expose the total amount of published posts
- To be more consistent, this PR creates a PostStats class which is exposed as `stats` method within the PostService; just like it's done with the MemberService
- Moved existing method to return the date of the most recently published post into the stats service
- Updated the explore service test to reflect the new return property
2022-09-16 13:56:14 +01:00
Kevin Ansfield
3b21d26be7
Wired up creation of post_revisions entries when saving posts with lexical (#15422)
no issue

- added `PostRevsion` model
- duplicated `mobiledoc_revision` creation routine in Post model's onSaving hook to create `post_revision` when model's `lexical` field has changed
- updated `mobiledoc_revision` creation to skip when `lexical` field is populated
2022-09-16 11:59:35 +01:00
Kevin Ansfield
1581f439e9
Added post_revisions table (#15420)
no issue

- initially this will perform the same function as `mobiledoc_revisions` but storing `lexical` instead of `mobiledoc`
- naming is intentionally generic ready for later expansions
2022-09-16 10:19:05 +01:00
Simon Backx
a7b583050c
Added link tracking to paywall (#15414)
closes https://github.com/TryGhost/Team/issues/1908

### Problem:
- We need tracking on the paywall links in each email. (we cannot ignore them because those buttons are probably gonna have a higher paid conversion attribution than others).
- Currently we only add the paywall HTML to an email when processing each batch. So if we batch an email to 1.000 recipients per 100, we'll generate the paywall HTML 10 times. 
- We cannot replace links in `renderEmailForSegment` because that methods will get called multiple times. We don't want to have multiple redirect instances created for the same link in the same email.

###  Solution:
- Move the generation of the paywall to the `serialize` method of the post email serializer.
- Surround the generated paywall with HTML-comments so we can remove it if required in `renderEmailForSegment` depending on the member segment we are sending the email to.

---

### Before:

**Serialize output:**
```html
<html>
    <body>
        <h1>Generated email header</h1>
        <p>Generated text</p>

        <div>
            <!-- POST CONTENT START -->
                <h1>Post title</h1>
                <p>Content visible for all members</p>
            <!--members-only-->
                <p>Content visible for paid members only</p>
            <!-- POST CONTENT END -->
        </div>
    </body>
</html>
```

To be modified later by  `renderEmailForSegment`:

**Paid members (nothing changed):**
```html
<html>
    <body>
        <h1>Generated email header</h1>
        <p>Generated text</p>

        <div>
            <!-- POST CONTENT START -->
                <h1>Post title</h1>
                <p>Content visible for all members</p>
            <!--members-only-->
                <p>Content visible for paid members only</p>
            <!-- POST CONTENT END -->
        </div>
    </body>
</html>
```

**Free members (paywall _added_):**
```html
<html>
    <body>
        <h1>Generated email header</h1>
        <p>Generated text</p>

        <div>
            <!-- POST CONTENT START -->
                <h1>Post title</h1>
                <p>Content visible for all members</p>
                <h2>Generated paywall here</h2>
                <a href="https://subscribe.com">Subscribe to read the full post</a>
            <!-- POST CONTENT END -->
        </div>
    </body>
</html>
```

### After this change:

**Serialize output:**
```html
<html>
    <body>
        <h1>Generated email header</h1>
        <p>Generated text</p>

        <div>
            <!-- POST CONTENT START -->
                <h1>Post title</h1>
                <p>Content visible for all members</p>
            <!--members-only-->
                <p>Content visible for paid members only</p>
             <!-- PAYWALL -->
                <h2>Generated paywall here</h2>
                <a href="https://subscribe.com/?tracked">Subscribe to read the full post</a>
            <!-- POST CONTENT END -->
        </div>
    </body>
</html>
```

To be modified later by  `renderEmailForSegment`:

**Paid members (paywall removed):**
```html
<html>
    <body>
        <h1>Generated email header</h1>
        <p>Generated text</p>

        <div>
            <!-- POST CONTENT START -->
                <h1>Post title</h1>
                <p>Content visible for all members</p>
            <!--members-only-->
                <p>Content visible for paid members only</p>
            <!-- POST CONTENT END -->
        </div>
    </body>
</html>
```

**Free members (members-only content removed):**
```html
<html>
    <body>
        <h1>Generated email header</h1>
        <p>Generated text</p>

        <div>
            <!-- POST CONTENT START -->
                <h1>Post title</h1>
                <p>Content visible for all members</p>
            <!-- PAYWALL -->
                <h2>Generated paywall here</h2>
                <a href="https://subscribe.com/?tracked">Subscribe to read the full post</a>
            <!-- POST CONTENT END -->
        </div>
    </body>
</html>
```
2022-09-16 10:08:12 +02:00
Kevin Ansfield
c240f7afa4
Added rendering of posts.lexical to posts.html when saving (#15416)
no issue

- added `@tryghost/kg-lexical-html-renderer` dependency
- added `lexical` lib following the same pattern as our `mobiledoc` lib
- updated the Post model's `onSaving` hook to generate the `html` value from `lexical` when present
2022-09-15 16:49:14 +01:00
Simon Backx
e9974d8cc0
🐛 Fixed feature image caption escaped twice in newsletters (#15417)
fixes https://github.com/TryGhost/Team/issues/1909

- The feature image caption is already escaped on the frontend
- Doing it again in the backend breaks the possibility to add links to the caption
- I checked and the `feature_image_alt` is not escaped in the frontend.
2022-09-15 17:07:10 +02:00
Simon Backx
699e67f4e4
Added email_track_clicks setting (#15409)
fixes https://github.com/TryGhost/Team/issues/1900
refs https://github.com/TryGhost/Team/issues/1901

- Defaults to the same value as the current email_track_opens setting for existing installations, otherwise defaults to true
- Had to use a custom migration because the `addSetting` helper doesn't support using an existing setting as current value
- Added a minimal UI to change the setting, but this still needs some design magic 🪄
- Link replacement is disabled if `email_track_clicks` is disabled. In the future we might consider to still do parial additions, such as source attribution and maybe redirects (to discuss).
2022-09-15 15:48:22 +02:00
Fabien "egg" O'Carroll
4726742673 Renamed redirects to custom-redirects
We're going to be adding more redirection logic into Ghost and it's
going to get confusing if we have names this generic. This makes it
clear which feature this service is related to.

Ideally in the future we can combine all of these into one redirects
service, but for now we will be running a specific service per feature
2022-09-14 13:24:17 -04:00