Commit Graph

209 Commits

Author SHA1 Message Date
Simon Backx
f33e7a22fa
Added E2E test for whole link tracking flow (#15481)
refs https://github.com/TryGhost/Team/issues/1967

- Test is good to test if the whole flow works as expected, and works together
- We can test independent parts in separate tests that have better coverage of more edge cases
- Adds a basic helper to get an agent for the frontend (spent too much time on a better solution so I decided to keep the existing supertest agent)
2022-09-27 18:44:20 +02:00
Kevin Ansfield
89d4e3daf9
Updated error messages for invalid mobiledoc+lexical post/page API requests (#15477)
closes https://github.com/TryGhost/Team/issues/1896

- updated message to be clearer, added context and help
2022-09-27 10:30:28 +01:00
Daniel Lockyer
9818634b63
Merged v5.16.1 into main
v5.16.1
2022-09-27 11:21:48 +07:00
Rishabh
0349acb7e3 Updated content cta to use global accent color property
refs https://github.com/TryGhost/Ghost/pull/15471#discussion_r979902374

- the accent color value used by default content cta was copying the global site property which is redundant, and can be directly used
- originally, the accentColor property was extended to allow a fallback value for content ctas, but was later removed as we added default value to global site property directly
- the accentColor property is now deprecated and will be removed in next version, as existing themes might be relying on it for custom cta helpers
2022-09-27 09:41:54 +05:30
Rishabh
1410a4237e 🐛 Fixed default content CTA message to reflect page vs post
closes https://github.com/TryGhost/Team/issues/1898

- the default content cta always used the terminology as `post` when showing message that users don't have access to some content
- this caused confusion when users were looking at a page and message showed "This post is for subscribers only"
- updates the message to correctly reflect `page` vs `post` on the default cta
2022-09-27 09:41:54 +05:30
Simon Backx
281cd2e7a3
Renamed count.conversions to count.paid_conversions in posts (#15460)
fixes https://github.com/TryGhost/Team/issues/1943
2022-09-26 14:25:27 +02:00
Naz
4528cba1b9 Changed Content API caching to public
refs https://github.com/TryGhost/Toolbox/issues/410

- Private cache control was preventing browser or shared caches from storing Content APIs response. The type of data served through the Content API is very much of a "public" nature, so should be cacheable.
- Right now the 'max-age' value of 'cache-control' header is hardcoded to '0', without 'must-revalidate' value, to allow browsers to cache content slightly more aggressively. In the future the 'max-age' value will most-likely become configurable to allow even more aggressive HTTP caching.
2022-09-26 14:54:50 +08:00
Naz
7b009bf1fe Enabled shared caching of 404 error responses
refs https://github.com/TryGhost/Toolbox/issues/410

- The 'private' value in 'Cache-Control' response header for all errors made it impossible for shared caches (e.g.: Fastly, Cloudflare) to cache 404 responses efficiently.
- The change substitutes 'max-age=0' which should not effect the browser cache behavior but would allow shared caches to process such requests efficiently.
- A more loose caching logic only applies to 404 responses from GET requests that are not user-specific (non-authenticated, non-cookie containing requests)
2022-09-26 14:54:50 +08:00
Rishabh
6c85c75b86 Added referrer attribution data to member api
refs https://github.com/TryGhost/Team/issues/1961

- includes referrer source and medium information in member api
2022-09-24 17:46:57 +05:30
Naz
225a046bb8
Made Admin assets aggressively cacheable
closes https://github.com/TryGhost/Toolbox/issues/372

- The admin assets are served with a unique hash depending on the build with a year-long "max-age" value in the response cache-control header. The client browsers still do send 'If-None-Match' requests when there is a hard-refresh on the client side. There's no need for 'If-None-Match' requests though!
- With 'immutable' value in the cache-control header, the browser caches are treating responses as "hard-fresh" without sending redundant requests.
- For more about 'immutable' value read https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#immutable
2022-09-23 13:46:49 +08:00
Rishabh Garg
b048b02f67
Added new referrer source stats API (#15449)
closes https://github.com/TryGhost/Team/issues/1939

- adds new endpoint that returns count of referrer sources by date for admin dashboard
2022-09-22 16:34:26 +05:30
Simon Backx
7437d92d50
Added post referrers stats API (#15448)
closes https://github.com/TryGhost/Team/issues/1942

- Added data fixtures for referrers
- Added new endpoint to fetch referrer stats for a given post: `/stats/referrers/posts/:id`
- Added new ReferrersStatsService, responsible for calculating referrer stats
2022-09-21 18:16:56 +02:00
Rishabh Garg
b99c5428d0
Added referrer attribution columns to events table (#15436)
refs TryGhost/Team#1931

- referrer source, medium and url will be stored in the events table along with rest of attribution data
- stores referrer information on two tables
  - `members_created_events` for signups
  - `members_subscription_created_events` for paid conversions
2022-09-21 19:01:36 +05:30
Simon Backx
b8041f0a60
Added clicks to activity feed (#15439)
closes https://github.com/TryGhost/Team/issues/1933

- Added click_events to activity feed
- Added support for parsing click_events in the frontend
- Moved url parsing (transform ready) to model layer of LinkRedirect
- Moved `getEventTimeline` method to the top of the event repository
- Added description field to parsed events in the frontend (because we need a second line)
- Fixed: member email not returned in comment_event
2022-09-21 10:25:51 +02:00
Hannah Wolfe
f31a50270d Removed search labs flag
- the search helper was promoted to GA and we no longer need the feature flag
2022-09-20 15:22:06 +01:00
Simon Backx
63103c2251
Added click counts to posts admin API (#15435)
closes https://github.com/TryGhost/Team/issues/1928
2022-09-20 10:05:41 +02:00
Kevin Ansfield
b2b6be9cb5 Fixed content-length matcher in posts API test snapshot
refs d5f03ec0b1

- underlying error message varies across node versions so the content-length can't be fixed
- applied any-content-length matcher to the right test this time
2022-09-19 16:37:54 +01:00
Kevin Ansfield
1cc8176d4f Allowed any error message length in posts API error test snapshot
refs d5f03ec0b1

- underlying error message varies across node versions so the content-length can't be fixed
2022-09-19 16:22:22 +01:00
Kevin Ansfield
201d4ef228 Loosened error message snapshot matching for posts API test error output
refs d5f03ec0b1

- underlying error message varies across node versions
- adjusted to match only the part we explicitly set
2022-09-19 16:06:34 +01:00
Kevin Ansfield
744534fde6 Updated snapshot for posts API test error output
refs d5f03ec0b1
2022-09-19 15:56:30 +01:00
Daniel Lockyer
74f1f1bba6
Fixed CDN URLs in tests
refs de416629e6

- I forgot to update them when making the referenced commit
2022-09-19 14:51:31 +01:00
Daniel Lockyer
1ba6d2e5a7
Updated test snapshot
refs de416629e6

- this was forgotten after I'd changed the endpoint in the referenced
  commit
2022-09-19 14:43:07 +01:00
Kevin Ansfield
3bdfb68ff6
Added __GHOST_URL__ transform support to posts.lexical field (#15433)
no issue

- bumped `@tryghost/url-utils` to get access to the new lexical transform utilities
- updated the Post model's `parse()` and `formatOnWrite()` methods to transform the `lexical` field contents when reading/writing to ensure any links in content point at the correct place with `site.url` config changes
2022-09-19 11:30:38 +01:00
Simon Backx
901a28f3c6
Added link_redirects and members_link_click_events tables (#15421)
closes https://github.com/TryGhost/Team/issues/1912
closes https://github.com/TryGhost/Team/issues/1913
2022-09-19 11:20:36 +02:00
Rishabh
c765c3230e Updated attribution service to handle referrer information
refs TryGhost/Team#1907

- calculates final attribution source and medium using captured referrer information in history
- adds new referrer-translator that goes through available history and based to determine most valid referrer info
- includes referrer url, source and medium in the attribution data for storage
2022-09-19 12:29:22 +05:30
Aileen Nowak
be45d4ebcf Added post stats service to return total posts in Explore endpoint
no issue

- The explore endpoint needs to expose the total amount of published posts
- To be more consistent, this PR creates a PostStats class which is exposed as `stats` method within the PostService; just like it's done with the MemberService
- Moved existing method to return the date of the most recently published post into the stats service
- Updated the explore service test to reflect the new return property
2022-09-16 13:56:14 +01:00
Kevin Ansfield
3b21d26be7
Wired up creation of post_revisions entries when saving posts with lexical (#15422)
no issue

- added `PostRevsion` model
- duplicated `mobiledoc_revision` creation routine in Post model's onSaving hook to create `post_revision` when model's `lexical` field has changed
- updated `mobiledoc_revision` creation to skip when `lexical` field is populated
2022-09-16 11:59:35 +01:00
Kevin Ansfield
1581f439e9
Added post_revisions table (#15420)
no issue

- initially this will perform the same function as `mobiledoc_revisions` but storing `lexical` instead of `mobiledoc`
- naming is intentionally generic ready for later expansions
2022-09-16 10:19:05 +01:00
Simon Backx
a7b583050c
Added link tracking to paywall (#15414)
closes https://github.com/TryGhost/Team/issues/1908

### Problem:
- We need tracking on the paywall links in each email. (we cannot ignore them because those buttons are probably gonna have a higher paid conversion attribution than others).
- Currently we only add the paywall HTML to an email when processing each batch. So if we batch an email to 1.000 recipients per 100, we'll generate the paywall HTML 10 times. 
- We cannot replace links in `renderEmailForSegment` because that methods will get called multiple times. We don't want to have multiple redirect instances created for the same link in the same email.

###  Solution:
- Move the generation of the paywall to the `serialize` method of the post email serializer.
- Surround the generated paywall with HTML-comments so we can remove it if required in `renderEmailForSegment` depending on the member segment we are sending the email to.

---

### Before:

**Serialize output:**
```html
<html>
    <body>
        <h1>Generated email header</h1>
        <p>Generated text</p>

        <div>
            <!-- POST CONTENT START -->
                <h1>Post title</h1>
                <p>Content visible for all members</p>
            <!--members-only-->
                <p>Content visible for paid members only</p>
            <!-- POST CONTENT END -->
        </div>
    </body>
</html>
```

To be modified later by  `renderEmailForSegment`:

**Paid members (nothing changed):**
```html
<html>
    <body>
        <h1>Generated email header</h1>
        <p>Generated text</p>

        <div>
            <!-- POST CONTENT START -->
                <h1>Post title</h1>
                <p>Content visible for all members</p>
            <!--members-only-->
                <p>Content visible for paid members only</p>
            <!-- POST CONTENT END -->
        </div>
    </body>
</html>
```

**Free members (paywall _added_):**
```html
<html>
    <body>
        <h1>Generated email header</h1>
        <p>Generated text</p>

        <div>
            <!-- POST CONTENT START -->
                <h1>Post title</h1>
                <p>Content visible for all members</p>
                <h2>Generated paywall here</h2>
                <a href="https://subscribe.com">Subscribe to read the full post</a>
            <!-- POST CONTENT END -->
        </div>
    </body>
</html>
```

### After this change:

**Serialize output:**
```html
<html>
    <body>
        <h1>Generated email header</h1>
        <p>Generated text</p>

        <div>
            <!-- POST CONTENT START -->
                <h1>Post title</h1>
                <p>Content visible for all members</p>
            <!--members-only-->
                <p>Content visible for paid members only</p>
             <!-- PAYWALL -->
                <h2>Generated paywall here</h2>
                <a href="https://subscribe.com/?tracked">Subscribe to read the full post</a>
            <!-- POST CONTENT END -->
        </div>
    </body>
</html>
```

To be modified later by  `renderEmailForSegment`:

**Paid members (paywall removed):**
```html
<html>
    <body>
        <h1>Generated email header</h1>
        <p>Generated text</p>

        <div>
            <!-- POST CONTENT START -->
                <h1>Post title</h1>
                <p>Content visible for all members</p>
            <!--members-only-->
                <p>Content visible for paid members only</p>
            <!-- POST CONTENT END -->
        </div>
    </body>
</html>
```

**Free members (members-only content removed):**
```html
<html>
    <body>
        <h1>Generated email header</h1>
        <p>Generated text</p>

        <div>
            <!-- POST CONTENT START -->
                <h1>Post title</h1>
                <p>Content visible for all members</p>
            <!-- PAYWALL -->
                <h2>Generated paywall here</h2>
                <a href="https://subscribe.com/?tracked">Subscribe to read the full post</a>
            <!-- POST CONTENT END -->
        </div>
    </body>
</html>
```
2022-09-16 10:08:12 +02:00
Kevin Ansfield
c240f7afa4
Added rendering of posts.lexical to posts.html when saving (#15416)
no issue

- added `@tryghost/kg-lexical-html-renderer` dependency
- added `lexical` lib following the same pattern as our `mobiledoc` lib
- updated the Post model's `onSaving` hook to generate the `html` value from `lexical` when present
2022-09-15 16:49:14 +01:00
Simon Backx
e9974d8cc0
🐛 Fixed feature image caption escaped twice in newsletters (#15417)
fixes https://github.com/TryGhost/Team/issues/1909

- The feature image caption is already escaped on the frontend
- Doing it again in the backend breaks the possibility to add links to the caption
- I checked and the `feature_image_alt` is not escaped in the frontend.
2022-09-15 17:07:10 +02:00
Simon Backx
699e67f4e4
Added email_track_clicks setting (#15409)
fixes https://github.com/TryGhost/Team/issues/1900
refs https://github.com/TryGhost/Team/issues/1901

- Defaults to the same value as the current email_track_opens setting for existing installations, otherwise defaults to true
- Had to use a custom migration because the `addSetting` helper doesn't support using an existing setting as current value
- Added a minimal UI to change the setting, but this still needs some design magic 🪄
- Link replacement is disabled if `email_track_clicks` is disabled. In the future we might consider to still do parial additions, such as source attribution and maybe redirects (to discuss).
2022-09-15 15:48:22 +02:00
Simon Backx
972c25edc7
Wired up member attribution from email clicks (#15407)
refs https://github.com/TryGhost/Team/issues/1899

- Added `addEmailAttributionToUrl` method to MemberAttributionService. This adds both the source attribution (`rel=newsletter`) and member attribution (`?attribution_id=123&attribution_type=post`) to a URL.
- The URLHistory can now contain a new sort of items: `{type: 'post', id: 'post-id', time: 123}`.
- Updated frontend script to read `?attribution_id=123&attribution_type=post` from the URL and add it to the URLHistory + clear it from the URL.
- Wired up some external dependencies to LinkReplacementService and added some dummy code.
- Increased test coverage of attribution service
- Moved all logic that removes the subdirectory from a URL to the UrlTranslator instead of the AttributionBuilder
- The UrlTranslator now parses a URLHistoryItem to an object that can be used to build an Attribution instance
- Excluded sites with different domain from member id and attribution tracking
2022-09-14 15:50:54 -04:00
Fabien "egg" O'Carroll
4726742673 Renamed redirects to custom-redirects
We're going to be adding more redirection logic into Ghost and it's
going to get confusing if we have names this generic. This makes it
clear which feature this service is related to.

Ideally in the future we can combine all of these into one redirects
service, but for now we will be running a specific service per feature
2022-09-14 13:24:17 -04:00
Simon Backx
d5b332ab02 Added temporary fix for random test failures in comments
refs https://ghost.slack.com/archives/C02G9E68C/p1663162175224299

This requires a better fix in the future that properly awaits the emails (not really possible at the moment) or disables sending new member emails when using loginAs
2022-09-14 17:02:13 +02:00
Kevin Ansfield
a7c4991af5 Wired up lexical editor saving
no issue

- fixed API returning "Invalid mobiledoc structure" errors when `mobiledoc:null` is sent in the payload alongside `lexical: '{...}'`
- updated Admin's `posts` and `pages` adapters to always add `?formats=mobiledoc,lexical` because the API doesn't return `lexical` by default
- added `lexical` attribute to Admin's Post model
- updated `lexical-editor` controller and related components to work with `lexical` always being a JSON string rather than a parsed object
- updated `<KoenigLexicalEditor>` to pass through the lexical state string as initial state and wired up the `onChange` prop
2022-09-13 21:01:53 +01:00
Kevin Ansfield
6fc9cd5f80
Added passthrough + saving of lexical property on posts/pages (#15403)
no issue

- bumped `@tryghost/admin-api-schema` to allow passthrough of the `lexical` property on post and page API endpoints
- prevented saving of blank document in the `mobiledoc` field if `lexical` is provided
- prevented API input containing both `mobiledoc` and `lexical` fields to avoid issues when both are present:
  - not possible to know which content is latest/has precedence
  - not possible to know which editor should be displayed in Admin
2022-09-13 17:29:37 +01:00
Kevin Ansfield
c8dc23cbb5 Fixed Content API posts/pages e2e tests
refs 7ad1be2555

- snapshot comparisons were missing matchers for dynamic fields in the body response
2022-09-13 15:05:53 +01:00
Kevin Ansfield
30611cf2c4 Really fixed e2e Admin API posts test
refs 9471384020

- previously added tests (any subsequent matcher updates) for browse endpoint were not using matchers that sufficiently covered the dynamic portions of the body
2022-09-13 14:09:45 +01:00
Kevin Ansfield
eebdb1d5df Fixed e2e Admin API posts test
refs 9471384020

- previously added tests for browse endpoint were not using matchers that sufficiently covered the dynamic portions of the body
2022-09-13 14:05:03 +01:00
Kevin Ansfield
9471384020 Added tests for Admin API not returning lexical by default but including when requested
no issue

- left `mobiledoc` as the only default format added in the post/page input serializers for now to minimize API/test churn during these early stages of lexical development
- tested that the `lexical` field is not returned by default but can be requested via `?formats=lexical`
2022-09-13 13:30:29 +01:00
Kevin Ansfield
7ad1be2555 Fixed Content API returning lexical format when requested
no issue

- similar to the `mobiledoc` field, the Content API should not return the source `lexical` field if requested via `?formats=`
  - renamed `removeMobiledocFormat()` to `removeSourceFormats()` to better match it's behaviour
2022-09-13 13:30:29 +01:00
Daniel Lockyer
f8679f22d7
Updated settings snapshot
refs 067bfe92a4

- this was missed in the previous commit
2022-09-13 12:37:33 +01:00
Kevin Ansfield
8cdd2e10b7
Added posts.lexical database field (#15397)
closes https://github.com/TryGhost/Team/issues/1884

- adds `post.lexical` ready for use by the lexical-powered editor re-write
- fulfils the same purpose as `posts.mobiledoc` so uses the same field properties
- added `lexical` to allowed formats in Post model so it won't be included by default in API responses meaning tests/snapshots don't need updating at present
2022-09-13 11:21:47 +01:00
Rishabh
054833992e Wired events for triggering email alerts for subscription creation/cancellation
refs https://github.com/TryGhost/Team/issues/1865

- refactors subscription creation/cancellation to dispatch proper events which are used for email alerts
- cleanup
2022-09-10 11:06:34 +05:30
Rishabh
2fbaa7b9bc Moved member email alert trigger to member creation
closes https://github.com/TryGhost/Team/issues/1864
refs https://github.com/TryGhost/Team/issues/1881

- triggers free member email alert via event dispatch from member create method
- passes subscription/stripe data to member creation for paid members so free member alert can be ignored for them
- moves subscription created event being called from webhook controller to `linkSubscription`, allows creating subscription events for all new subscriptions instead of ones just via webhooks
2022-09-10 11:06:34 +05:30
Rishabh
4187f0da54 Updated staff service to trigger alerts via events
refs https://github.com/TryGhost/Team/issues/1865

- refactors staff service to listen to member and subscription events
- triggers email alerts based on events instead of directly calling the service
- removes staff service dependency for members api
2022-09-10 11:06:34 +05:30
Hannah Wolfe
04f3ac37d3 Removed defunct applyPublicRules code
closes: https://github.com/TryGhost/Toolbox/issues/369
refs: https://github.com/TryGhost/Toolbox/issues/229

- this code is defunct as we no longer accept a status parameter via the API for posts, pages, users or authors
2022-09-09 17:14:15 +01:00
Hannah Wolfe
b0234dd58e
Removed apiVersions from test urlUtils
refs: https://github.com/TryGhost/Toolbox/issues/229

- These properties were removed from urlUtils in 5.0 and no longer do anything
2022-09-09 13:32:30 +01:00
Ronald Langeveld
c9e6f42ca8 Bumped Portal to 2.12.0
ref https://github.com/TryGhost/Team/issues/1800

- Adds v 2.12.0 of Portal
- Updates snapshots for tests
2022-09-09 13:54:43 +02:00