Commit Graph

920 Commits

Author SHA1 Message Date
Sam Lord
08427809ec Allowed data-generation script to run in all environments
refs: https://github.com/TryGhost/Toolbox/issues/453
2022-10-27 16:10:02 +01:00
Sam Lord
1b7df74703 Added option to use existing data when generating demo data
refs: https://github.com/TryGhost/Toolbox/issues/440
2022-10-27 16:10:02 +01:00
Rishabh Garg
6a619310f6
Removed attribution script if tracking sources is disabled (#15707)
refs https://github.com/TryGhost/Team/issues/2168

- site owners can now disable tracking sources from analytics settings.
- this change removes the loading of attribution script if tracking is
turned off so we don't capture any post/page or external source
attributions
2022-10-27 18:52:11 +05:30
Rishabh Garg
cca0f7d7dc
Added new setting to toggle tracking of member sources (#15705)
refs https://github.com/TryGhost/Team/issues/2168

- the new setting allows site owners to control if they want to track
the sources for new member signups and subscriptions
- its switched on by default, but can be toggled off from new analytics
settings page
2022-10-27 17:24:46 +05:30
Sam Lord
da559e5a7f Added @tryghost/data-generator dependency to Ghost core
refs: https://github.com/TryGhost/Toolbox/issues/440

This was working locally where the dependency is resolved implicitly, but when deployed there is no @tryghost/data-generator in the node_modules folder.
2022-10-27 11:51:09 +01:00
Simon Backx
b911208b41
Improved filter support in activity API to allow pagination (#15684)
fixes https://github.com/TryGhost/Team/issues/2129

- This changes how the activity feed API parses the filter.
- We now parse the filter early to a MongoDB filter, and split it in two. One of the filters is applied to the pageActions, and the other one is used individually for every event type. We now allow to use grouping and OR's inside the filters because of this change. As long as we don't combine filters on 'type' with other filters inside grouped filters or OR, then it is allowed.
- We make use of mongoTransformer to manually inject a mongo filter without needing to parse it from a string value again (that would make it a lot harder because we would have to convert the splitted filter back to a string and we currently don't have methods for that).
- Added sorting by id for events with the same timestamp (required for reliable pagination)
- Added id to each event (required for pagination)
- Added more tests for filters
- Added test for pagination
- Removed unsued getSubscriptions and getVolume methods

Used new mongo utility methods introduced here: https://github.com/TryGhost/NQL/pull/49
2022-10-27 12:13:24 +02:00
Simon Backx
076e3c02b2
Added linking between member and subscription created events (#15693)
fixes https://github.com/TryGhost/Team/issues/2160

- Adds a `batch_id` to both events that contain the same ID if they were created at the same time.
- Removes duplicate signup/conversion events using the batch_id
- Requires an update in mongo-knex to work (refs https://ghost.slack.com/archives/C02G9E68C/p1666773313272409?thread_ts=1666767872.375009&cid=C02G9E68C)
- Some dependencies needed an update to load the latest mongo-knex
- Added tiers to membersUtils, loaded on startup (we can start to use this instead of fetching it every time)
2022-10-27 11:44:19 +02:00
Daniel Lockyer
30327d62cd
Refactored DB backup lib to async-await
- also improves jsdoc comments to make it easier to see the types of
  variables around the code
2022-10-27 15:52:12 +07:00
Naz
6bd2769146
Hardened tiers-related snapshot tests
refs https://github.com/TryGhost/Team/issues/2077

- Members and Posts test suites were using a broad tiers property matcher, which is an anti-pattern for snapshot tests. Without more specific snapshots it would be very hard to track down tier-related breaking changes!
- This change is groundwork for a refactor coming in tier usage at API's output serializers
2022-10-27 15:54:28 +08:00
Elena Baidakova
e3ab868b83
Added email feedback column (#15698)
closes TryGhost/Team#2159
- Added column to email table
- Hide the feedback tab on frontend depending on the column value

Co-authored-by: Daniel Lockyer <daniellockyer@fastmail.com>
2022-10-27 11:22:50 +04:00
Sam Lord
28b11e6fed
Added command to generate demo data (#15691)
refs: https://github.com/TryGhost/Toolbox/issues/440

New command to generate demo data, creates data for over 20 tables in
Ghost, suitable for testing most features of the dashboard, as well as
making guided product tours using newsletters, tiers, many posts and
tags.

Usage: `yarn start generate-data`

Optionally, keep your existing posts / tags with: `yarn start generate-data --use-existing-tags --use-existing-posts`
2022-10-26 17:55:08 +01:00
Elena Baidakova
57817eefc8
Added feedback buttons to emails (#15695)
closes TryGhost/Team#2075
2022-10-26 19:19:05 +04:00
James Morris
f1565fe92a Updated the copy for the audience feedback in preview and email based on feedback
refs https://github.com/TryGhost/Team/issues/2171
2022-10-26 12:17:07 +01:00
Aileen Nowak
b79006d7f3 Added Ghost Explore app
no issue

- bumps Ghost Explore integrated app  to GA from alpha
2022-10-26 10:05:50 +01:00
Naz
cdd65f25ac
Migrated members importer to use tiers
refs https://github.com/TryGhost/Team/issues/2077

- The "productRepository" methods have been deprecated in favor of "tiers" and "Tiers API".
- The changes migrated usages of  "productRepository.getDefaultProduct" to Tiers API's "readDefaultTier"
2022-10-26 14:26:21 +08:00
renovate[bot]
d034526fe6
Update dependency supertest to v6.3.1 2022-10-26 02:54:43 +00:00
Daniel Lockyer
857dacbf16 Fixed missing column values for default paid tiers
fixes https://github.com/TryGhost/Toolbox/issues/455
refs https://github.com/TryGhost/Ghost/blob/main/ghost/core/core/server/data/migrations/versions/5.19/2022-09-02-20-52-backfill-new-product-columns.js

- the referenced migration does not handle backfilling the
  currency/monthly_price/yearly_price for the default paid tiers where
  they do not originate from Stripe
- this is causing issues in Ghost because of the missing data
- this migration backfills the columns for products where they are paid
  but do not currently contain values due to the bug above with the
  values for the default tier we usually use
2022-10-26 08:49:10 +07:00
Naz
a7f5ee0ad5
Simplified members CSV importer constructor
refs https://github.com/TryGhost/Team/issues/2077

- Passing in the whole "getMembersApi" is just too much state to know about for the importer - it only uses a concept of default tier and members repository, the rest is distracting fluff making it hard to reason about what the importer **has to** know to function
- Passing in two functions breaking up the above state simplifies the constructor API.
- This is also a groundwork before substituting productsRepository for tiersRepository (refed issue objective)
2022-10-25 16:40:28 +08:00
renovate[bot]
0d3d85df64 Update dependency html-validate to v7.7.1 2022-10-25 10:40:33 +07:00
Fabien "egg" O'Carroll
25d8d694a0 Wired up the payment service to create stripe checkout sessions
refs https://github.com/TryGhost/Team/issues/2078
2022-10-25 09:03:04 +07:00
Fabien "egg" O'Carroll
31610f9b94 Wired up Tiers service to Tiers Content & Admin API
refs https://github.com/TryGhost/Team/issues/2078
2022-10-25 09:02:59 +07:00
Fabien "egg" O'Carroll
cc14ce2b20 Added initial tiers service to Ghost
refs https://github.com/TryGhost/Team/issues/2078

This allows us to start wiring up the new package to the Admin & Content API's
2022-10-25 09:02:56 +07:00
renovate[bot]
68f1df545c
Update dependency express-jwt to v7.7.7 2022-10-24 21:21:25 +00:00
Hannah Wolfe
d260a7c78c
Updated webhook test snapshots
refs: 26d049911c
refs: 8c2f832573

- snapshots fell behind between the two referenced commits, and needed updating
2022-10-24 19:44:46 +01:00
Kritika Sharma
26d049911c
Added e2e tests for post.published.edited webhook (#15642)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.

Co-authored-by: Kritika Sharma <kritikasharma@Kritikas-MacBook-Pro-2.local>
2022-10-24 14:55:25 +01:00
Dominik Picheta
8c2f832573
Added e2e tests for post.unscheduled webhook (#15675)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-10-24 14:12:49 +01:00
Simon Backx
1f138893ff Fixed snapshot tests
refs 30ecaef329
2022-10-24 14:27:52 +02:00
James Morris
30ecaef329 Update the portal version number
no issue
2022-10-24 12:47:00 +01:00
Simon Backx
fd91f7eebb
Added email sent events (#15682)
fixes https://github.com/TryGhost/Team/issues/2137

For the analytics page, we need the sent events to show up immediately
after sending an email. Otherwise we need to wait for emails to be
marked as received (which takes too long) before being able to show them
on the analytics page.

This adds the email_sent_event, which is hidden by default everywhere
and used on the analytics page.
2022-10-24 11:11:44 +02:00
Simon Backx
a650ae2138
Fixed default feedback enabled when flag is disabled (#15660)
fixes https://github.com/TryGhost/Team/issues/2114
fixes https://github.com/TryGhost/Team/issues/2115

When a new newsletter is created, the frontend will send feedback_enabled to true. We'll catch this in the backend and don't allow setting feedback_enabled to true when audience_feedback flag is disabled. This is also handled for editing newsletters.

To fix this in existing sites, I added a migration that disables feedback for all sites (since this is an alpha feature). Once we'll release the feature later, it will be disabled for existing newsletters, just like expected.
2022-10-24 10:14:58 +02:00
renovate[bot]
bbd6c47d01 Update dependency html-validate to v7.7.0 2022-10-24 11:36:30 +07:00
renovate[bot]
603c78755d
Updated @tryghost dependencies (#15631)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2022-10-24 10:30:40 +07:00
renovate[bot]
44f54fa4cc Update dependency inquirer to v8.2.5 2022-10-24 10:22:16 +07:00
renovate[bot]
3021a7dc50 Update dependency eslint to v8.26.0 2022-10-24 10:19:56 +07:00
Halldor Thorhallsson
dbad621b91
Removed bluebird from fixture-utils.js (#15626)
refs: https://github.com/TryGhost/Ghost/issues/14882

- Opted to use the in-house `sequence` function when refactoring Bluebird's `Promise.each` to avoid deadlock issues (see 734ef66e6c). 
-It's hard to know without tonnes of context if any `Promise.each` are safe to refactor to `Promise.all`.
2022-10-21 20:56:21 +01:00
Elena Baidakova
9720459f55
Added pie chart for activity feed (#15673)
closes TryGhost/Team#2088
- Added pie chart to feedback event
- Added `negative_feedback` field to response from BE
2022-10-21 21:34:05 +04:00
Halldor Thorhallsson
39e246aaf8
Removed bluebird from fixture-manager.js (#15629)
refs: https://github.com/TryGhost/Ghost/issues/14882

- Removing bluebird specific methods in favour of the Ghost sequence method so we can remove the bluebird dependency

Co-authored-by: Hannah Wolfe <github.erisds@gmail.com>
2022-10-21 18:08:44 +01:00
Simon Backx
7c824556c2
Added new members filters and refactored filters (#15667)
fixes https://github.com/TryGhost/Team/issues/2112

- Removed a bit of duplicate code across templates and components that was used to handle filters
- Updated filter objects to contain information about the filter
- Added resource filters that are able to select a single resource, which can be used in columns
- Filters can now define columns by themselves. Not all columns already make use of this functionality, but we can move those over later (cleanup: https://github.com/TryGhost/Team/issues/2133)
- The filter definitions became quite long. We should move them to separate files in the future: https://github.com/TryGhost/Team/issues/2134
- Filters can now have custom NQL parsing
- Improved support for parsing recursive or grouped NQL queries
- Added support for filtering members by feedback
2022-10-21 17:18:00 +02:00
Ghost CI
67c742606d v5.20.0 2022-10-21 16:00:38 +01:00
Rishabh Garg
05330316e3
Allowed fixing newsletter links (#15672)
refs https://github.com/TryGhost/Team/issues/2116

- allows site owners to edit a link in a post that has already been sent out, fixing any typos or other mistakes
- resets click counter for the edited link back to 0 so site owners can see the clicks on new link, doesn't change the overall click count
2022-10-21 19:38:57 +05:30
illiteratewriter
a701f2114d
Added importer for custom theme settings (#15596)
closes: https://github.com/TryGhost/Ghost/issues/15542

- custom theme settings were not reinstated on import
- importing custom theme settings for the current active theme requires the theme be re-activated
2022-10-21 15:02:32 +01:00
Hannah Wolfe
3ab1c418bc
Updated webhook test snapshots
refs: 717a27c85c
refs: 6380b82793

- The snapshots just needed updating
2022-10-21 14:36:20 +01:00
Halldor Thorhallsson
717a27c85c
Added e2e tests for page.unpublished webhook (#15613)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-10-21 14:16:13 +01:00
Rishabh
3d355ea41d Fixed test snapshots
refs 5b283930f0
2022-10-21 18:14:57 +05:30
Samprit JC
f7738dc1a2
Added e2e test for post.unpublished webhook (#15628)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-10-21 13:08:54 +01:00
Shashank Gupta
092fb7813d
Added e2e tests for page.edited webhook (#15627)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-10-21 12:45:38 +01:00
illiteratewriter
4ba254b339
Added e2e tests for member.edited webhook (#15620)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-10-21 12:43:30 +01:00
Shubhadeep Das
5b283930f0
Added e2e tests for post.edited webhook (#15625)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-10-21 12:39:24 +01:00
Naz
b552727b69
Changed members export to contain tiers
refs https://github.com/TryGhost/Team/issues/1076

- The "products" is a legacy term for what is now "tiers" since multiple tiers feature introduction in https://github.com/TryGhost/Ghost/releases/tag/v4.39.0
- Note, the "tiers" is a field meant for informational purposes and cannot be imported back into Ghost site.
2022-10-21 18:58:59 +08:00
Rishabh
ba6fe63250 Fixed cached redirects for edited newsletter links
refs https://github.com/TryGhost/Team/issues/2135

The email link redirects on Pro are cached as 302 redirects in Varnish, so we're missing further clicks after the first one for each member, until the cache is invalidated. This change invalidates cache on link edits to ensure that we correctly redirect members to updated link everytime
2022-10-21 16:07:34 +05:30
Rishabh
d7b3721036 Added edited property to links api
refs https://github.com/TryGhost/Team/issues/2104

- adds a boolean `edited` property to links api that denotes if the link has been edited
2022-10-20 18:17:44 +05:30
Rishabh
6312b064f5 Added reset for link click count on edit
refs https://github.com/TryGhost/Team/issues/2104

When a newsletter link is edited, we reset its click count to 0 to show only the clicks on newly edited links. This is done by only counting the member click events for a link which are greater than its last updated at, so that all previous click events are not counted for the link, but are included in the total count of all links on the page.
2022-10-20 17:50:02 +05:30
Rishabh
d8bacf12d1 Added endpoint for fixing newsletter links
refs https://github.com/TryGhost/Team/issues/2104

- adds new bulk edit endpoint for links, updates all matching link with the current redirect url and update to new url
2022-10-20 17:50:02 +05:30
Simon Backx
adf10f0e76
Added support for filtering email events by post_id (#15666)
refs https://github.com/TryGhost/Team/issues/2093
2022-10-20 13:29:00 +02:00
Daniel Lockyer
02c8690e87 Added ghost_subscription_id column to members_stripe_customers_subscriptions
refs https://github.com/TryGhost/Team/issues/2034

- this table will be used to link Stripe subscriptions to Ghost
  subscriptions via a foreign key that we add at a later point
- this also includes `constraintName` as the auto-generated one would be
  too long for MySQL 8
2022-10-20 10:59:36 +07:00
Rishabh Garg
318a5a809c
Added permissions for link edit endpoints (#15664)
refs https://github.com/TryGhost/Team/issues/2104

- adds edit permissions for links endpoints to fixtures
- new `bulkEdit` endpoint will use the permissions and allow fixing newsletter links via Admin
2022-10-20 09:11:26 +05:30
Rishabh Garg
60b10ad69a
Fixed permissions for links endpoint (#15656)
refs 5fcf5098a8

- links browse endpoint had permissions switched off unintentionally and was also missing the necessary permissions in fixtures.
- enables permissions for browse endpoint and adds migration insert permissions in DB
2022-10-20 08:18:29 +05:30
Simon Backx
6380b82793
Added sentiment ordering and include for posts (#15657)
fixes https://github.com/TryGhost/Team/issues/2090

- This changes how sentiment is exposed in the API. Now it is exposed as a `sentiment` relation, directly on the model (no longer in counts). Internally we still use `count.sentiment`.
- Content API users (and themes) can include the 'sentiment' relation and order by sentiment.
- Updated Admin to use sentiment instead of count.sentiment
2022-10-19 16:50:58 +02:00
Sam Lord
68955aa704 Remove Grunt from yarn setup in Ghost core
no issue
2022-10-19 12:58:10 +01:00
Elena Baidakova
17cfdcd3a9
Updated feedback buttons url (#15655)
closes TryGhost/Team#2080
- If the post was published and emailed the link leads the user to the
post.
- If the post was just emailed the link leads the user to the home page.
2022-10-19 15:21:43 +04:00
Naz
b589a66cd4
Fixed broken CSV importer tests
refs 90768e9985

- With introduction of strict field mapping the regression test testing for "imports of not mapped fields" failed.
2022-10-19 18:33:47 +08:00
Naz
6c9cfe8f24
Fixed typo 2022-10-19 18:10:29 +08:00
Naz
7a1389954b
Updated JSDoc and fixed typos 2022-10-19 18:10:29 +08:00
Daniel Lockyer
9b8c33484d
Merged v5.19.3 into main
v5.19.3
2022-10-19 06:22:38 +07:00
Ghost CI
e1e5a7b35e v5.19.3 2022-10-18 16:18:29 +01:00
Simon Backx
a822c5a8c5 Added test to check if feedback buttons are hidden if alpha flag is disabled 2022-10-18 16:47:06 +02:00
Simon Backx
8ecf0a5858 🐛 Fixed alpha feature visible in new newsletters
no issue
2022-10-18 16:38:59 +02:00
Simon Backx
a01fb5f1aa
Added post_id filter and total to activity feed API (#15650)
fixes https://github.com/TryGhost/Team/issues/2091
fixes https://github.com/TryGhost/Team/issues/2089

- Added new fixtures to make testing easier for the activity feed
- Improved E2E test coverage of activity feed with separate test file
- Added data.post_id filter to enable filtering by events related to a
given post
- Fixed return types in JSDoc of test agents (TypeScript interprets
these as `typeof Agent` if we don't add `InstanceType<Agent>`)
- Added total pagination metadata to activity feed API (to allow a basic
type of pagination using filters)
2022-10-18 15:52:04 +02:00
Daniel Lockyer
2dcc4139b1
Merged v5.19.2 into main
v5.19.2
2022-10-18 17:04:23 +07:00
Ghost CI
14a259b828 v5.19.2 2022-10-18 09:51:41 +01:00
Simon Backx
d1e6870740
🐛 Fixed large mailgun recipient data (#15638)
fixes https://github.com/TryGhost/Team/issues/2096

When generating the recipient data for emails, the email clicks
implementation is resulting in a recipient variable being added called
replacement_xxx once for each link containing the same UUID.

This generates a lot of unnecessary data overhead for emails, and it
turns out that mailgun has a 25MB message limit. We wouldn't have come
close if we only included the uuid once.
2022-10-18 10:32:50 +02:00
Daniel Lockyer
18c52f2a2a
Improved jsdoc for permission migration utils
- this helps with readability and editor autocomplete
2022-10-18 14:38:58 +07:00
Daniel Lockyer
3858f255b9 Dropped nullable status on subscriptions.tier_id
fixes https://github.com/TryGhost/Team/issues/2102

- this column was added with `nullable: true` but it should never be
  nullable, so we should drop the nullable status whilst it's easy to
2022-10-18 14:16:30 +07:00
Daniel Lockyer
2d324ea315
Fixed various code nits with schema command utils
- de-duped the exports at the bottom if they export the same name as the
  function
- added types to all functions, or fixed existing ones
- renamed `table` to `tableBuilder` to represent it better
- these should help with code readability and autocomplete in editors
2022-10-18 11:07:12 +07:00
Daniel Lockyer
c9d43b8fe1
Allowed constraintName in schema column spec
refs 0ba3d6df49

- this is used to indicate the name of the foreign key constraint and so
  we should let it through the schema checks
2022-10-18 10:29:55 +07:00
Daniel Lockyer
0ba3d6df49 Added support for supplying the foreign key constraint name
- this allows us to choose the foreign key constraint name when the
  auto-generated one would be too long
2022-10-18 10:25:14 +07:00
Simon Backx
0bb7538cd1
Added feedback events to activity feed (#15639)
fixes https://github.com/TryGhost/Team/issues/2051
fixes https://github.com/TryGhost/Team/issues/2052
2022-10-17 15:44:18 +02:00
Simon Backx
22fe1c01de
Added conversions count and separate analytics page (#15637)
fixes https://github.com/TryGhost/Team/issues/2084

- When audience feedback is enabled, we use a single 'conversions' count instead of having separate ones for signups and paid conversions.
- The analytics component is separated so we can change it without breaking the existing page.
2022-10-17 13:02:39 +02:00
Daniel Lockyer
b6d2d97af2
Merged v5.19.1 into main
v5.19.1
2022-10-17 17:48:02 +07:00
Ghost CI
9a2fcba68a v5.19.1 2022-10-17 11:45:59 +01:00
Naz
66c2d3748a
🐛 Fixed 404 collection links for new tags
closes https://github.com/TryGhost/Ghost/issues/15608
closes https://github.com/TryGhost/Toolbox/issues/437
refs https://github.com/bookshelf/bookshelf/issues/2111
refs https://github.com/knex/knex/issues/1641

- When new tag was attached to the post the tag collection link returned 404 - instead of a collection with one post
- The root cause of the issue and it's flaky behavior (sometimes the collection link was returning correctly) was a race condition between event propagation in routing for "tag.attached" event and the post+tag+relations transaction completion
- The race condition was happening as the bookshelf-transaction-events plugin was emitting the 'committed' event BEFORE the transaction was committed!
2022-10-17 18:21:10 +08:00
renovate[bot]
e9587e02d0
Update dependency mocha to v10.1.0 2022-10-17 08:41:28 +00:00
Daniel Lockyer
eb30e464d3
Added handling for unsuccessful comments API requests
refs https://github.com/TryGhost/Team/issues/2082

- in the event the API doesn't return a 200 OK, we shouldn't be
  processing the response from it, as we can end up doing weird things
  if, for example, an error object is returned
2022-10-17 12:07:18 +07:00
Daniel Lockyer
1dab16c9c1 Added handling for unsuccessful comments API requests
refs https://github.com/TryGhost/Team/issues/2082

- in the event the API doesn't return a 200 OK, we shouldn't be
  processing the response from it, as we can end up doing weird things
  if, for example, an error object is returned
2022-10-17 11:59:32 +07:00
Daniel Lockyer
7ac0fdb23d
Fixed unnecessary requests with loading comment counts
refs https://github.com/TryGhost/Team/issues/2082

- if a site has comments enabled but doesn't use the `comments_count`
  helper, the comments-count.min.js will still be loaded and it'll send
  a POST request to Ghost with an empty array of post IDs to fetch
- this is unnecessary and we should avoid this extra request for pages
  that don't need to show comment counts
- this commit prevents the comment-counts JS from sending the request if
  there are no post IDs to fetch
2022-10-17 11:51:14 +07:00
Daniel Lockyer
fd9401503b Fixed unnecessary requests with loading comment counts
refs https://github.com/TryGhost/Team/issues/2082

- if a site has comments enabled but doesn't use the `comments_count`
  helper, the comments-count.min.js will still be loaded and it'll send
  a POST request to Ghost with an empty array of post IDs to fetch
- this is unnecessary and we should avoid this extra request for pages
  that don't need to show comment counts
- this commit prevents the comment-counts JS from sending the request if
  there are no post IDs to fetch
2022-10-17 11:07:16 +07:00
Ronald Langeveld
6b7088ba20 Bumped kg-lexical-html-renderer version
no issue

- Bumped from 0.0.8 to 0.0.9
2022-10-17 08:44:30 +07:00
Daniel Lockyer
54c143a1b4
Fixed optional syntax style for jsdoc
refs https://jsdoc.app/tags-param.html#optional-parameters-and-default-values

- using an equals sign in the type definition is part of the Google
  Closure syntax but we use the JSDoc syntax in all other places, and
  tsc detects the different syntax
- this commit standardizes the syntax ahead of enforcing a certain style
  down the line
2022-10-16 14:48:05 +07:00
Elena Baidakova
46141efe05
Updated test snapshot after bumping Portal (#15623) 2022-10-14 20:29:46 +04:00
Elena Baidakova
d381ff87b8
Added ability to handle feedback links from emails (#15622) 2022-10-14 20:11:44 +04:00
Ghost CI
4e3afadfef v5.19.0 2022-10-14 16:00:45 +01:00
Elena Baidakova
e831be6bc2
Added the feedback buttons in the emails (#15619)
closes TryGhost/Team#2046
closes TryGhost/Team#2045
- Added feedback buttons markup.
- Added feedback links generation.
2022-10-14 18:12:17 +04:00
Fabien 'egg' O'Carroll
8afc6777c0
🐛 Removed redirects from search engine indexing (#15617)
refs https://github.com/TryGhost/Team/issues/2072

Google is indexing our redirects and storign the redirected content
against the redirect URL in search results. This seems to be caused by
us using a 302 redirect rather than 301. We don't want to switch to a
301 however, so that we can support the ability to update redirects in
the future.
2022-10-14 15:51:43 +07:00
Fabien 'egg' O'Carroll
bd0f4b4b8c
Added Tier price and currency data to products table (#15366)
refs https://github.com/TryGhost/Team/issues/1765

In order to better handle deleted objects in Stripe we want to decouple
Members from Stripe.

These changes allow us to have the Tier concept completely independent
of the Stripe tables, such that the Stripe data can be generated as/when
it's needed - which will help to protect against missing data.
2022-10-14 06:40:17 +01:00
Barno
5c361670ce
Replaced Promise.each() with .all() in models/base/bookshelf (#15509)
refs: https://github.com/TryGhost/Ghost/issues/14882

- Removing bluebird specific methods in favour of native promises so we can remove the bluebird dependency.

Co-authored-by: Carol-Barno <cbarno@innovexsolutions.co.ke>
Co-authored-by: Hannah Wolfe <github.erisds@gmail.com>
2022-10-13 15:24:05 +01:00
jbenezech
75811f35bc
🐛 Fixed embedded cards for non-utf8 content (#15578)
closes: https://github.com/TryGhost/Ghost/issues/14973

- When fetching content using a non-standard charset, characters were notproperly decoded to utf-8 resulting in mangled text in the editor -> Detect charset and use iconv to decode the page text

- When requesting a non bookmark card, if no oembed data could be foundand we fallback to bookmark, a second network request to fetch the content was issued. This seemed unnecessary -> refactored to avoid that
2022-10-13 12:19:47 +01:00
Kritika Sharma
76e906d498
Added e2e tests for post.tag.detached webhoo (#15610)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.

Co-authored-by: Kritika Sharma <kritikasharma@Kritikas-MacBook-Pro-2.local>
2022-10-13 11:53:18 +01:00
Shubhadeep Das
e2124314ed
Added e2e tests for page.scheduled webhook (#15609)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-10-13 11:20:42 +01:00
Halldor Thorhallsson
74f7b7b3ee
Added e2e test for site.changed webhook event (#15595)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-10-13 11:05:31 +01:00
Rishabh Garg
e05889cd53
🐛 Fixed missing accent color for default content cta (#15611)
refs https://github.com/TryGhost/Casper/issues/901

- the site accent color property on default cta for upgrade link had a typo and was using wrong property
2022-10-13 13:42:21 +05:30
Daniel Lockyer
bd3add9747
Switched to US spelling of cancelled in schema validation
refs https://github.com/TryGhost/Team/issues/2030

- we tend to use US spellings in the code and this was merged with the
  British spelling
- nothing has been added to this table yet so it's safe to switch
2022-10-13 11:53:07 +07:00
Naz
4db0be603f Added subscriptions table
refs https://github.com/TryGhost/Team/issues/2030

- adds `subscriptions` table to the DB schema
- this new table is aimed to support a native "subscription" primitive in Ghost
  that most resembles previously used `members_stripe_customers_subscriptions` table
2022-10-13 11:19:13 +07:00
Daniel Lockyer
74e1d52d6a Fixed incorrect maximum length definition in schema
refs https://github.com/TryGhost/Toolbox/issues/441

- whilst reviewing another PR, I noticed we were incorrectly using
  `maxLength` instead of `maxlength` in the schema column definition
- it turns out we've already been doing this wrong for a while with
  other columns
- this key is not acted upon, so the maximum column length was not applied
- fixing up the DB to the correct maximum length is something to fix in the
  future but right now, the schema does not reflect the size of the
  column that actually got created
- the fallback when `maxlength` is not provided is currently 191 [0], so
  this commit switches the schema and migrations to using the correct
  key name and column length that they are using when applied

[0]: 24670aa555/ghost/core/core/server/data/schema/commands.js (L27)
2022-10-13 09:58:19 +07:00
Daniel Lockyer
143ae857c9 Removed bool type from schema
refs https://github.com/TryGhost/Toolbox/issues/441

- we tend to have a mix of `bool` and `boolean` in the schema and
  migrations, which has become a real nit for me at this point
- we don't do any special handling between `bool` and `boolean`, it's
  just something we pass to Knex
- `bool` is an alias for `boolean` but `boolean` is actually documented - https://knexjs.org/guide/schema-builder.html#boolean
- this commit switches Ghost to only using `boolean` in the schema and
  migrations, and removes `bool` from the allowlist in tests to prevent
  us from adding it again in the future
- this should make absolutely no difference to the DB because both
  resulted in the same column
2022-10-13 09:37:38 +07:00
Daniel Lockyer
24670aa555
Fixed validating numbers as booleans in schema validator
refs https://github.com/TryGhost/Toolbox/issues/441

- I'm currently working on cleaning up our uses of `bool` and `boolean`
  in favor of `boolean`, and I've noticed we only handle converting
  numbers into booleans when the type is `bool`, so validation would
  otherwise fail
- given these can be used interchangeably, we should also support
  converting the numbers into booleans when the type is `boolean`
- this is going to get cleaned up again when I remove `bool` but this
  fixes the validation bug for now
2022-10-13 08:20:31 +07:00
renovate[bot]
f5774fad0c Update dependency postcss to v8.4.18 2022-10-13 08:08:05 +07:00
Kritika Sharma
2cb4282fa8
Added e2e tests for post.tag.attached webhook (#15576)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.

Co-authored-by: Kritika Sharma <kritikasharma@Kritikas-MacBook-Pro-2.local>
2022-10-12 21:53:02 +01:00
jennyxchang
0bc31ded1b
Added e2e tests for post.scheduled event (#15561)
refs https://github.com/TryGhost/Ghost/issues/15537

- Added e2e test for post.scheduled event to increase webhook test coverage
2022-10-12 21:18:08 +01:00
Christa
87d21662bc
🐛 fixed error message code for HB translate helper (#15529)
closes: https://github.com/TryGhost/Ghost/issues/15500

- Per the issue, Ghost has a policy to never throw 500 Internal Server errors for theme issues. This change adds a check inside of `ghost\core\core\frontend\helpers\t.js` if `text` or `options` is undefined, to throw an `IncorrectUsageError` error within the function.
- Messaging was borrowed from `ghost\core\core\frontend\web\middleware\error-handler.js`.
2022-10-12 21:14:53 +01:00
Shubhadeep Das
8374c73e52
Added e2e tests for tag.edited webhook (#15555)
refs: https://github.com/TryGhost/Ghost/issues/15537

- this adds an e2e test and test snapshot for the `tag.edited` webhook so we can prevent regressions and bugs in the future

Co-authored-by: Hannah Wolfe <github.erisds@gmail.com>
2022-10-12 20:00:04 +01:00
Sam Lord
0c18060f61 Added metric for sending transactional email with mailgun
refs: https://github.com/TryGhost/Toolbox/issues/439

This case completes the monitoring of mailgun.js usage within Ghost.
2022-10-12 16:52:51 +01:00
Hannah Wolfe
168fa64395
Updated webhook snapshot for member.deleted
refs: 3ae1e48917
refs: 1221ba5d1d

- We added a new field between the PR being raised and merged, so the snapshot went out-of-date
2022-10-12 16:04:57 +01:00
illiteratewriter
3ae1e48917
Added e2e tests for member.deleted webhook (#15570)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-10-12 14:35:24 +01:00
Kritika Sharma
2ce0727794
Added e2e tests for post.deleted webhook (#15572)
refs: https://github.com/TryGhost/Ghost/issues/15537

- Added  missing e2e tests for post.deleted webhook to increase coverage 

Co-authored-by: Kritika Sharma <kritikasharma@Kritikas-MacBook-Pro-2.local>
2022-10-12 14:32:01 +01:00
jbenezech
3d44e37cbd
🐛 Fixed sitemaps with no content (#15571)
closes: https://github.com/TryGhost/Ghost/issues/14981

- Taxonomy-specific sitemaps were invalid xml when there was no data
- These invalid empty sitemaps were referenced in the index sitemap causing SEO tools to report errors
2022-10-12 14:11:19 +01:00
jbenezech
b9dd9f066d
🐛 Fixed settings overriden when updated from multiple tabs (#15536)
closes: https://github.com/TryGhost/Ghost/issues/15470

- When multiple browser tabs are open, each manipulate a different copy of ember data model, changes to the model in one tab are not reflected in the model of the other tab.
- When updating some settings, all current settings were sent to the API.
- As a result, when updating two different categories of settings (navigation/code inspection) in different tabs, the second update was overriding the first one.
- From a user perspective, this is not a natural behaviour. Only settings visible on-screen when clicking save should be modified.

Co-authored-by: Kevin Ansfield <kevin@lookingsideways.co.uk>
2022-10-12 14:03:54 +01:00
Daniel Lockyer
845f8d965e
Added test to validate schema structure
refs https://github.com/TryGhost/Toolbox/issues/441

- this is only v1 of the test I would like but it validates the keys on
  a column definition are part of an allowlist
- this has already uncovered a bug with `maxLength` (vs `maxlength`)
2022-10-12 19:19:33 +07:00
renovate[bot]
178df69ae2 Update dependency @playwright/test to v1.27.1 2022-10-12 17:55:29 +07:00
Daniel Lockyer
e2ba19b0df
Fixed import of bson-objectid in accordance to the typings
- there's a few different ways we can import it but I've chosen to
  append `.default` as we have done in several other places in the code
2022-10-12 14:54:35 +07:00
Daniel Lockyer
2422f4a95c
Fixed minor jsdoc typing issue regarding arrays
- `[string]` indicates an array with exactly one item but we actually
  want the type to be `string[]` to indicate an array of strings
2022-10-12 14:52:59 +07:00
Daniel Lockyer
b3ee6a4c9e
Promoted sourceAttribution flag to private beta
refs https://www.notion.so/ghost/Source-attribution-7ad2f4f75b4a458f8d8090ad75d9359e

- this allows us to enable the feature for private testing before we
  continue with GA'ing it
2022-10-12 13:46:35 +07:00
renovate[bot]
6fc497743d
Updated @tryghost dependencies (#15479)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2022-10-12 10:16:05 +07:00
Simon Backx
68bdc1afea
Added post sentiment (#15592)
fixes https://github.com/TryGhost/Team/issues/2054

This change adds the sentiment and positive_feedback counts to the posts models. This change isn't really ideal because there are some problems here:
- sentiment isn't really a count
- we don't need to include the sentiment and positive_feedback as a default for posts (but the same is true for attribution)

It would make sense to move this to separate endpoints that only fetch the analytics for a given post when the analytics page is opened. But for our initial skateboard version of audience feedback this should be a good start to already see the data.
2022-10-11 17:52:14 +02:00
Djordje Vlaisavljevic
88d4872346 Added fix links in newsletter feature flag
refs https://github.com/TryGhost/Team/issues/2063
2022-10-11 17:08:41 +02:00
Simon Backx
08309f8d88 Fixed audience feedback package dependencies
refs e540344ef2
2022-10-11 16:53:54 +02:00
Simon Backx
e540344ef2
Added audience feedback service and storage (#15584)
fixes https://github.com/TryGhost/Team/issues/2049
fixes https://github.com/TryGhost/Team/issues/2053

- This adds a new audience feedback package to Ghost. 
- A new members API to give feedback on posts using the `/api/feedback` endpoint.
- Added a new authentication middleware that supports both uuid-based and session based authentication.
2022-10-11 16:32:28 +02:00
Elena Baidakova
1221ba5d1d
Added feedback_enabled to newsletters table (#15589)
closes TryGhost/Team#2042
- Added ability to enable audience feedback per newsletter (just on BE side).
2022-10-11 16:06:26 +04:00
Simon Backx
74d749fa63
Added members_feedback table (#15581)
fixes https://github.com/TryGhost/Team/issues/2041
2022-10-11 13:21:31 +02:00
Naz
714e108d40
Fixed typo 2022-10-11 17:24:11 +08:00
Naz
9b34bd70a2
Added test coverage for Subscription edit API
refs https://github.com/TryGhost/Team/issues/2047

- We anticipate upcoming changes in the PUT /members/:id/subscriptions/:subscription_id endpoint , so covered it with a snapshot test to track the differences more precisely.
- Note, the test case contains a more explicit outgoing HTTP request mocking.
2022-10-11 17:24:00 +08:00
Daniel Lockyer
9f04475918
Refactored JSON content importer handler to async-await
- this makes the code easier to read and removes an import of Bluebird
2022-10-11 10:22:13 +07:00
renovate[bot]
3b62e8c52f Update sentry-javascript monorepo to v7.15.0 2022-10-11 09:24:03 +07:00
renovate[bot]
aa29478057 Update dependency @playwright/test to v1.27.0 2022-10-11 09:23:33 +07:00
renovate[bot]
9f5ca0ede0 Update dependency jwks-rsa to v2.1.5 2022-10-11 09:23:12 +07:00
John Grisham
c41f431fc7
Added e2e tests for page.added webhook (#15548)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-10-10 19:56:53 +01:00
Fabien 'egg' O'Carroll
dc8617a1e6
Added full flow test for click tracking (#15546)
refs https://github.com/TryGhost/Team/issues/1967

This tests the full flow of publishing a newsletter, and then checking
that clicked links will increase the click count, generate events for
the member which clicked the link as well as the redirects contain the
correct query params.
2022-10-10 10:15:31 -05:00
Daniel Lockyer
8ec071a47c
Update dependency eslint-plugin-ghost to v2.16.0 (#15551)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2022-10-10 10:23:24 +01:00
Daniel Lockyer
c80d6d5b1c
Updated .eslintignore list for core
- this file path no longer exists
2022-10-10 15:12:52 +07:00
Daniel Lockyer
a37b2cd24e Switched to Husky + lint-staged for git hooks
- up until this commit, git hooks were only used by a handful of people
  because they were a pain:
  - they'd only be set up when you did `yarn setup`
  - the existing hooks ran `yarn lint` on all projects, which was
    incredibly slow
- as a result, not many of us actually had them enabled, but this would
  cause issues in CI because people were pushing un-linted commits
- other JS projects tend to use husky to automate the git hook setup and
  lint-staged to speed up linting on changed files
- this commit switches to using them both
  - `lint-staged` only runs `eslint` on staged JS files that are about to
    be committed - if there's a linting error, it will stop the commit
  - I've configured the pre-commit hook to successfully exit in CI because we
    don't want to run pre-commit hooks right now
- this means we can remove Grunt - yay!
2022-10-10 15:10:48 +07:00
Simon Backx
73f5fd92eb Added audience feedback feature flag
closes https://github.com/TryGhost/Team/issues/2039
2022-10-10 09:27:45 +02:00
Naz
0125f51780
Fixed typo/reference in db schema
refs https://github.com/TryGhost/Team/issues/2030

- There was a typo and a reference to a wrong field in members_stripe_customers_subscriptions schema definition
2022-10-10 14:03:42 +08:00
Daniel Lockyer
75f83718f5
Fixed comment typos in schema definition 2022-10-10 10:58:51 +07:00
renovate[bot]
34439f2e28 Update dependency html-validate to v7.6.0 2022-10-10 10:33:10 +07:00
renovate[bot]
82441e943d Update dependency express to v4.18.2 2022-10-10 10:29:34 +07:00
Ghost CI
b5fd02c9e8 v5.18.0 2022-10-07 16:00:53 +01:00
Ghost CI
a6177b46b5 🎨 Updated Casper to v5.3.2 2022-10-07 16:00:52 +01:00
Aileen Booker
c4188c1a9e
Added Ghost Explore in Ghost as iframe app behind feature flag (#15495)
no issue

- Added Ghost Explore screen behind alpha flag
- Moved existing /explore route to /explore/connect which we'll redirect to for outside requests
- Added iframe communication with Ghost Explore App
2022-10-07 14:32:54 +01:00
Simon Backx
7e3b41f643
Removed emailClicks feature flag (#15556)
fixes https://github.com/TryGhost/Team/issues/2028

Since link clicks became GA, some older components and templates are no longer used.
2022-10-07 14:27:57 +02:00
Daniel Lockyer
9fc830e662
Removed patch from comments-ui version
- we only need to provide the patch if we want to force Ghost to use a
  specific version
- otherwise, we can just use major.minor because we use the tilde
  versioning method
- having the patch version here just encourages you to bump it
  unnecessarily, so removing it cleans up the usage for now
2022-10-07 16:22:16 +07:00
illiteratewriter
a0ec94fbfe
Added e2e test for member.added webhook (#15554)
refs https://github.com/TryGhost/Ghost/issues/15537

- this adds an e2e test and test snapshot for the `member.added` webhook so we can prevent regressions and bugs in the future
2022-10-07 15:54:24 +07:00
Shubhadeep Das
426168f73d
Added e2e test for tag.deleted webhook (#15553)
refs https://github.com/TryGhost/Ghost/issues/15537

- this adds an e2e test and test snapshot for the `tag.deleted` webhook so we can prevent regressions and bugs in the future
2022-10-07 15:50:16 +07:00
Naz
06f6fc11a7
Cleaned up tags e2e webhook test
refs https://github.com/TryGhost/Ghost/issues/15537
refs 4110ffaa2c

- The test had minor formatting issues not worth an extra back-forth during the PR review
2022-10-07 10:20:41 +08:00
Shubhadeep Das
4110ffaa2c Added e2e tests for tag.added webhook (#15537) 2022-10-07 09:46:21 +08:00
renovate[bot]
84549838a4 Update sentry-javascript monorepo to v7.14.2 2022-10-07 08:41:20 +07:00
renovate[bot]
0370dd258d
Update dependency body-parser to v1.20.1 2022-10-06 21:20:56 +00:00
Simon Backx
9d27014aff Reverted change in post email serializer
refs d4540012dc

This was committed by accident
2022-10-06 20:16:02 +02:00
James Morris
e871aabb70 Updated the comments to 0.10.2 2022-10-06 16:39:42 +01:00
Naz
1880c7c1ec
Updated webhook post.published test
refs https://github.com/TryGhost/Toolbox/issues/320

- Added more complex mobiledoc structure in the post.published test to check for correct transformation of special purpose `__GHOST_URL__`. The snapshot has a correct URL transformation, which gives confidence it works properly
2022-10-06 17:51:14 +08:00
Simon Backx
d4540012dc Added tests for click events in the activity feed
fixes https://github.com/TryGhost/Team/issues/2018

- Includes new test fixtures for redirects and click events
- Tests if post, and links are returned in the click events
2022-10-06 11:43:39 +02:00
Daniel Lockyer
7308bb9122
Switched to accessing config loader directly
- I lowered the code coverage on the repo to the point where
  it started failing because I added a new export to the config library
- this wasn't easy to add tests for because the existing config tests
  use the loader directly and not the library export
- instead, I'm just going to make the dev script access the loader, and
  make a note to clean this up in the future when we pull out the config
  module
2022-10-06 16:25:29 +07:00
Daniel Lockyer
0bfbee5523
Fixed yarn dev --stripe ignoring HTTPS configured sites
- because the cwd of `.github/dev.js` is not `ghost/core`, it doesn't
  pick up config.local.json files, so any configuration you set in there
  isn't applied
- this meant that developers with HTTPS configured locally couldn't use
  `--stripe` because it wouldn't configure the Stripe listening URL
  correctly
- this adds an exports to the config lib to allow passing options in,
  which I then utilize to pass the directory that config resides in
- this should fix the aforementioned problem with HTTPS
2022-10-06 15:58:51 +07:00
Naz
78c97d10a6
Improved post's webhook test annotations
refs https://github.com/TryGhost/Toolbox/issues/320

- There noe "roles" attached to the post's author when the 'post.added' event is fired. Webhooks function based of the model events and differ slightly with it's output comparing to the API response. For example, in case of Posts API, there'a an additional 'findOne' call (ref.: https://github.com/TryGhost/Ghost/blob/main/ghost/core/core/server/models/post.js#L1224-L1227) before returning the post to the endpoint handler and then passing that to the output serializer.
- If we want to have 1:1 copy of webhooks outputs and API outputs, we should rethink how we rely on model event data which is never the same as API controller level data.
2022-10-06 10:50:02 +08:00
Naz
4315b21d25
Fixed note copy 2022-10-06 10:26:23 +08:00
Naz
fe1d0e44b4
Moved Ghost agent matcher to common framework
refs a499f866f3
refs d817e5830d

- The user-agent used in outgoing Ghost requests (webhooks mostly) is dependent on the Ghost version - snapshots break if the matcher is not dynamic.
- There will be a few more webhooks tests coming soon, so makes sense to have this matcher moved to a common "framework matchers"
2022-10-06 08:56:10 +08:00
renovate[bot]
57f09fc8b7 Update dependency semver to v7.3.8 2022-10-05 23:34:22 +00:00
renovate[bot]
fa7a582c78
Update dependency knex-migrator to v5.0.7 2022-10-05 15:06:09 +00:00
Simon Backx
a499f866f3 Prevented posts webhook tests from breaking on every release 2022-10-05 14:25:00 +02:00
Simon Backx
f17934a5d2 Updated snapshots for latest release 2022-10-05 14:18:29 +02:00
Daniel Lockyer
c4981a71a2
Merged v5.17.2 into main
v5.17.2
2022-10-05 18:33:12 +07:00
Ghost CI
267f1530f0 v5.17.2 2022-10-05 12:32:04 +01:00
Simon Backx
8900db8614
Fixed snapshots for Portal update
refs e86e78fb6b
2022-10-05 18:11:06 +07:00
Simon Backx
811f37e18a
Bumped used Portal version to v2.14.x
refs eac8fbfdfd
refs e7378520a0
refs https://github.com/TryGhost/Ghost/issues/14508
2022-10-05 18:11:06 +07:00
Simon Backx
41a0945592
🐛 Prevented member creation when logging in (#15526)
fixes https://github.com/TryGhost/Ghost/issues/14508

This change requires the frontend to send an explicit `emailType` when sending a magic link. We default to `subscribe` (`signin` for invite only sites) for now to remain compatible with the existing behaviour.

**Problem:**
When a member tries to login and that member doesn't exist, we created a new member in the past.

- This caused the creation of duplicate accounts when members were guessing the email address they used.
- This caused the creation of new accounts when using an old impersonation token, login link or email change link that was sent before member deletion.

**Fixed:**
- Trying to login with an email address that doesn't exist will throw an error now.
- Added new and separate rate limiting to login (to prevent user enumeration). This rate limiting has a higher default limit of 8. I think it needs a higher default limit (because it is rate limited on every call instead of per email address. And it should be configurable independent from administrator rate limiting. It also needs a lower lifetime value because it is never reset.
- Updated error responses in the `sendMagicLink` endpoint to use the default error encoding middleware.
- The type (`signin`, `signup`, `updateEmail` or `subscribe`) is now stored in the magic link. This is used to prevent signups with a sign in token.

**Notes:**
- Between tests, we truncate the database, but this is not enough for the rate limits to be truly reset. I had to add a method to the spam prevention service to reset all the instances between tests. Not resetting them caused random failures because every login in every test was hitting those spam prevention middlewares and somehow left a trace of that in those instances (even when the brute table is reset). Maybe those instances were doing some in memory caching.
2022-10-05 18:11:06 +07:00
Simon Backx
b96ff6ae4a Fixed snapshots for Portal update
refs e86e78fb6b
2022-10-05 12:52:50 +02:00
Simon Backx
e86e78fb6b Bumped used Portal version to v2.14.x
refs eac8fbfdfd
refs e7378520a0
refs https://github.com/TryGhost/Ghost/issues/14508
2022-10-05 12:47:03 +02:00
Simon Backx
e7378520a0
🔒 Prevented member creation when logging in (#15526)
fixes https://github.com/TryGhost/Ghost/issues/14508

This change requires the frontend to send an explicit `emailType` when sending a magic link. We default to `subscribe` (`signin` for invite only sites) for now to remain compatible with the existing behaviour.

**Problem:**
When a member tries to login and that member doesn't exist, we created a new member in the past.

- This caused the creation of duplicate accounts when members were guessing the email address they used.
- This caused the creation of new accounts when using an old impersonation token, login link or email change link that was sent before member deletion.

**Fixed:**
- Trying to login with an email address that doesn't exist will throw an error now.
- Added new and separate rate limiting to login (to prevent user enumeration). This rate limiting has a higher default limit of 8. I think it needs a higher default limit (because it is rate limited on every call instead of per email address. And it should be configurable independent from administrator rate limiting. It also needs a lower lifetime value because it is never reset.
- Updated error responses in the `sendMagicLink` endpoint to use the default error encoding middleware.
- The type (`signin`, `signup`, `updateEmail` or `subscribe`) is now stored in the magic link. This is used to prevent signups with a sign in token.

**Notes:**
- Between tests, we truncate the database, but this is not enough for the rate limits to be truly reset. I had to add a method to the spam prevention service to reset all the instances between tests. Not resetting them caused random failures because every login in every test was hitting those spam prevention middlewares and somehow left a trace of that in those instances (even when the brute table is reset). Maybe those instances were doing some in memory caching.
2022-10-05 12:42:42 +02:00
Naz
0bf6268091
Updated content-length header matchers
no issue

- All content-length snapshots should be using the same matcher for consistency - anyContentLength. It's more explicit about what the matcher is all about and might be useful to have content-length matchers in one place if it ever changes (the header value should be a damn digit after all, not a string!) (ref. https://www.rfc-editor.org/rfc/rfc7230#section-3.3.2)
2022-10-05 17:34:17 +08:00
Fabien "egg" O'Carroll
28de1720c1 🔒 Fixed magic link endpoint sending multiple emails
refs https://github.com/TryGhost/Team/issues/2024

Without validation it was possible to send a string of comma separated
email addresses to the endpoint, and an email would be sent to each
address, bypassing any rate limiting.

This bug does not allow for an authentication bypass exploit. It is purely a
spam email concern.

Credit: Sandip Maity <maitysandip925@gmail.com>
2022-10-05 10:28:13 +01:00
Naz
2288289ae9
Added notes for maxAge config using express.static
no issue

- The milliseconds configuration here is different to "seconds" used in the max-age header value itself and other middlewares (like CORS). It's not going to be fixed upstream, so whenever this piece of code is touched again would be smart to get our own converter from seconds to milliseconds going, or some other mechanism making max-age configuration uniform across codebase
2022-10-05 17:26:21 +08:00
Naz
320c6e0dd3
Abstracted a hacky local URL matcher
refs https://github.com/TryGhost/Toolbox/issues/320

- The URL matcher is very likely to be reused in the future, so having it abstracted away gives two benefits:
1. Central place to document hacky behavior and easier future cleanup
2. The implementer of the e2e test does not have to see the "hacky note" and just concentrate on the implementation of the test
2022-10-05 17:23:02 +08:00
Naz
d817e5830d
Added header snapshots to webhook e2e tests
refs https://github.com/TryGhost/Toolbox/issues/320

- Header snapshot matching was missing from webhook e2e tests. With a bumped version of webhook-mock-receiver it's now possible to record and match webhook request headers.
2022-10-05 17:23:02 +08:00
renovate[bot]
2c2ee81adb
Update Test & linting packages 2022-10-05 00:36:08 +00:00
renovate[bot]
8751245fa5 Update sentry-javascript monorepo to v7.14.1 2022-10-04 08:07:37 +07:00
Daniel Lockyer
44f6840a26
Merged v5.17.1 into main
v5.17.1
2022-10-03 15:42:30 +07:00
Ghost CI
f1dad0519c v5.17.1 2022-10-03 09:41:39 +01:00
Fabien 'egg' O'Carroll
1613470a8c
Used the higher of click count and open count for email open count (#15508)
fixes https://github.com/TryGhost/Team/issues/2017

We process clicks much faster than we process Mailgun events which can result in a higher click rater than open rate shown on the dashboard. This ensures that the open rate will never be lower than the click rate. This is a stopgap solution until we can get click events updating the opened_at time for email_recipients
2022-10-03 10:29:58 +02:00
Simon Backx
b80c2cd81b 🐛 Fixed broken activity feed and click filter
fixes https://github.com/TryGhost/Ghost/issues/15515

- The link relation of a member-click-event was still using the link_id as foreign key instead of redirect_id.
- The members_link_click_events table was renamed to members_click_events, but this change was not reflected in a recent change in the member model (which has the custom filters).
2022-10-03 09:52:24 +02:00
renovate[bot]
5140f8a887 Update dependency sqlite3 to v5.1.2 2022-10-03 14:21:24 +07:00
renovate[bot]
828ad5915d Update dependency postcss to v8.4.17 2022-10-03 07:38:42 +07:00
Ghost CI
a7556ad9ea v5.17.0 2022-09-30 16:00:36 +01:00
Fabien 'egg' O'Carroll
45d65663f4
Simplified link tracking related tables naming (#15480)
- Removes superfluous "link" from table names
- Fixes type definititon of dropTables util
- Updates & renames models
- Noop existing migrations to avoid unnecessary work
2022-09-29 22:08:45 +01:00
Rishabh Garg
e3600d70ef
Added referrer attribution from request context (#15499)
closes TryGhost/Team#2007

- uses request context to add referrer source and medium for a new member
- uses integration name as referrer medium if exists
2022-09-29 22:31:48 +05:30
Simon Backx
648811690a Added email click tracking
no issue

Bumped flag to GA.
2022-09-29 18:14:15 +02:00
Simon Backx
0cd0fc838d
Added email track clicks column and cleaned up frontend checks (#15501)
fixes https://github.com/TryGhost/Team/issues/2008

- New column that stores email click tracking at the time it was created
- Improved frontend side checks for when to show analytics
2022-09-29 16:42:45 +02:00
renovate[bot]
68ad829e1f Update sentry-javascript monorepo to v7.14.0 2022-09-29 07:44:21 +07:00
Simon Backx
b905085d6f
Added opened, clicked and received email filtering to members (#15492)
fixes https://github.com/TryGhost/Team/issues/1993

- Allows filtering members by opened, clicked and received email
- Adds clicked_links filter relation to Member model.
- Adds emails filter relation to Member model.
- Adds opened_emails filter expansion to Member model.
- Updated GhResourceSelect to be able to only show list posts by setting the `type` attribute to `email`.
- Improved code reuse in `filter-value` component.
2022-09-28 17:14:32 +02:00
Simon Backx
8af422c601
Updated email_track_clicks to current email_track_opens value (#15484)
fixes https://github.com/TryGhost/Team/issues/1990

We need to set the current track clicks setting to the current track opens setting, just before release.
2022-09-28 17:12:29 +02:00
Naz
8cbf913582 Increased Vary granularity for versioned requests
refs https://github.com/TryGhost/Toolbox/issues/425
refs https://github.com/TryGhost/Toolbox/issues/280

- The versioned API responses vary based on requested version (passed in request's 'accept-version' header). shared caches that sit between Ghost's origin server and the browser would be putting responses with same Vary into the same caching bucket, which is incorrect.
- This change makes response's Vary more granular and tells caching mechanisms to take 'Accept-Version' request header into account when caching.
- Informative read on the topic - https://www.fastly.com/blog/getting-most-out-vary-fastly
2022-09-28 14:48:43 +08:00
renovate[bot]
b61c13e228 Update dependency @playwright/test to v1.26.1 2022-09-28 12:38:09 +07:00
Naz
874d0bf81b Made Content API caching configurable
refs https://github.com/TryGhost/Toolbox/issues/411

- Having hardcoded cache control values in the codebase makes it impossible to experiment with new values without a version release.
- Having all values configurable by default will allow for easier caching experiments and customizations on self-hosting instances.
2022-09-28 07:32:27 +08:00
Naz
e45eb4d5dd Made robots.txt caching configurable
refs https://github.com/TryGhost/Toolbox/issues/411

- Having hardcoded cache control values in the codebase makes it impossible to experiment with new values without a version release.
- Having all values configurable by default will allow for easier caching experiments and customizations on self-hosting instances.
- Brings caching across both private and public robots file caching to same consistent and configurable value.
2022-09-28 07:32:27 +08:00
Naz
f6c7df4018 Fixed overly long robots.txt caching for private sites
refs https://github.com/TryGhost/Toolbox/issues/411
refs 78ac63d8ad (diff-fb6792aa9cc3b71f5b146129fb1b58765206ecf4c5f367b72e0826b4b2934508R74)

- The value should have been in SECONDS not MILLISECONDS from the very beginning (see referenced commit)
2022-09-28 07:32:27 +08:00
Naz
a26689c264 Added backend JWKS endpoint caching
refs https://github.com/TryGhost/Toolbox/issues/411
refs f58b5984cb

- Backend JWKS endpoint did not have any HTTP caching
- It is recommended to cache JWKS endpoints (ref.: https://docs.apigee.com/api-platform/reference/policies/jwt-policies-overview)
- This change also brings the endpoint up to speed with what members JWKS
endpoint does
- Above point gives another reminder to extract JWKS functionality into a
common module that should be reused
2022-09-28 07:32:27 +08:00
Naz
8c47819194 Made members JWKS endpoint caching configurable
refs https://github.com/TryGhost/Toolbox/issues/411
refs f58b5984cb

- Having hardcoded cache control values in the codebase makes it impossible to experiment with new values without a version release.
- Having all values configurable by default will allow for easier caching experiments and customizations on self-hosting instances.
- This change only changes the members endpoint caching configurability. The other JWKS endpoint will be modified separately (following commit), to keep changes concise
2022-09-28 07:32:27 +08:00
Naz
5331ba3999 Made public built assets caching configurable
refs https://github.com/TryGhost/Toolbox/issues/411

- Having hardcoded cache control values in the codebase makes it impossible to experiment with new values without a version release.
- Having all values configurable by default will allow for easier caching experiments and customizations on self-hosting instances.
-NOTE:  caching of `public/ghost.css` increases here from one HOUR to one YEAR (did not find any good reason to keep caching to short window for a built asset that has cache-busting mechanism)
2022-09-28 07:32:27 +08:00
Naz
381e419471 Made sitemap.xls caching configurable
refs https://github.com/TryGhost/Toolbox/issues/411

- Having hardcoded cache control values in the codebase makes it impossible to experiment with new values without a version release.
- Having all values configurable by default will allow for easier caching experiments and customizations on self-hosting instances.
2022-09-28 07:32:27 +08:00
Rishabh Garg
31733657a6
Updated naming for referrer attribution (#15486)
- renames `refSource`, `refMedium` and `refUrl` to `referrerSource`, `referrerMedium` and `referrerUrl` respectively for consistent naming across files and usages
2022-09-28 00:58:06 +05:30
Rishabh
d886bc4b0d Fixed tests for portal bump
refs 8848fd0f59

- last commit didn't update test snapshots for new portal version bump
2022-09-28 00:04:32 +05:30
Rishabh
8848fd0f59 Handled attribution history capture via portal data attributes
- updates portal to capture member attribution history when signing up via theme using data-attributes
2022-09-27 23:59:06 +05:30
Rishabh Garg
90034577b8
Added member attribution history frontend script (#15482)
- bumps member attribution script from alpha feature to now load for all sites. The script captures recent url history in localstorage to capture correct attribution for members.
- script is only loaded on the site if members is enabled
2022-09-27 23:45:41 +05:30
Simon Backx
f33e7a22fa
Added E2E test for whole link tracking flow (#15481)
refs https://github.com/TryGhost/Team/issues/1967

- Test is good to test if the whole flow works as expected, and works together
- We can test independent parts in separate tests that have better coverage of more edge cases
- Adds a basic helper to get an agent for the frontend (spent too much time on a better solution so I decided to keep the existing supertest agent)
2022-09-27 18:44:20 +02:00
Simon Backx
6489cb01b5
Disabled link replacement when link click tracking is disabled (#15483)
fixes https://github.com/TryGhost/Team/issues/1988

- We don't want to replace links when link click tracking is disabled (also not add ref)
- Cleaned up some comments and methods
2022-09-27 18:20:34 +02:00
Simon Backx
8c87bb8158 Fixed missing times in activity feed click events
fixes https://github.com/TryGhost/Team/issues/1983
2022-09-27 18:10:05 +02:00
Daniel Lockyer
18e3d38bb0
Merged v5.16.2 into main
v5.16.2
2022-09-27 21:19:32 +07:00
Ghost CI
effe66e3a5 v5.16.2 2022-09-27 13:45:58 +01:00
Kevin Ansfield
89d4e3daf9
Updated error messages for invalid mobiledoc+lexical post/page API requests (#15477)
closes https://github.com/TryGhost/Team/issues/1896

- updated message to be clearer, added context and help
2022-09-27 10:30:28 +01:00
Daniel Lockyer
9818634b63
Merged v5.16.1 into main
v5.16.1
2022-09-27 11:21:48 +07:00
Ghost CI
ddb9c1b30c v5.16.1 2022-09-27 05:21:10 +01:00
Rishabh
0349acb7e3 Updated content cta to use global accent color property
refs https://github.com/TryGhost/Ghost/pull/15471#discussion_r979902374

- the accent color value used by default content cta was copying the global site property which is redundant, and can be directly used
- originally, the accentColor property was extended to allow a fallback value for content ctas, but was later removed as we added default value to global site property directly
- the accentColor property is now deprecated and will be removed in next version, as existing themes might be relying on it for custom cta helpers
2022-09-27 09:41:54 +05:30
Rishabh
1410a4237e 🐛 Fixed default content CTA message to reflect page vs post
closes https://github.com/TryGhost/Team/issues/1898

- the default content cta always used the terminology as `post` when showing message that users don't have access to some content
- this caused confusion when users were looking at a page and message showed "This post is for subscribers only"
- updates the message to correctly reflect `page` vs `post` on the default cta
2022-09-27 09:41:54 +05:30
renovate[bot]
9eb3c84a23
Updated @tryghost dependencies (#15434)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2022-09-27 08:31:35 +07:00
Simon Backx
86022b136b
Allowed free signups regardless of Portal settings (#15472)
fixes https://github.com/TryGhost/Team/issues/1966

- Currently you can only do a free self signup when 'free' is enabled in Portal or when Stripe is disabled
- Some themes, such as the Edition theme add a free signup form to the theme. That theme stops working if we don't allow self signup.
- The portal settings shouldn't be used to determine if free signup is allowed or not.
2022-09-26 18:25:06 +02:00
Simon Backx
281cd2e7a3
Renamed count.conversions to count.paid_conversions in posts (#15460)
fixes https://github.com/TryGhost/Team/issues/1943
2022-09-26 14:25:27 +02:00
Daniel Lockyer
a94c93e1c0
Added yarn build command
refs https://github.com/TryGhost/Toolbox/issues/390

- this allows us to run `yarn build` and make it output a .tgz file
  which can be installed with `ghost install --archive ...`
2022-09-26 18:28:30 +07:00
Kevin Ansfield
2de0c1358f 🐛 Fixed product card not displaying with just an image+button
closes https://github.com/TryGhost/Team/issues/1877

- bumped `@tryghost/kg-default-cards` which includes updated "should render" dependencies that adds a fully enabled button to the list of possible requirements for the product card to render. Now any one of the following will render the product card:
  - title is present
  - description is present
  - button url is enabled and button text+url are present
2022-09-26 09:38:49 +01:00
Simon Backx
bf008bee2d 🐛 Fixed deleting users with draft posts
fixes https://github.com/TryGhost/Team/issues/1945

- When deleting a user, a private tag is assigned to their existing posts.
- In that loop, it tries to find the post, but the post model had a default filter to only return published posts.
- An error was thrown because the post model was not fetched.
2022-09-26 10:19:33 +02:00
renovate[bot]
d02401c1f0
Update dependency eslint to v8.24.0 2022-09-26 06:58:26 +00:00
Naz
4528cba1b9 Changed Content API caching to public
refs https://github.com/TryGhost/Toolbox/issues/410

- Private cache control was preventing browser or shared caches from storing Content APIs response. The type of data served through the Content API is very much of a "public" nature, so should be cacheable.
- Right now the 'max-age' value of 'cache-control' header is hardcoded to '0', without 'must-revalidate' value, to allow browsers to cache content slightly more aggressively. In the future the 'max-age' value will most-likely become configurable to allow even more aggressive HTTP caching.
2022-09-26 14:54:50 +08:00
Naz
7b009bf1fe Enabled shared caching of 404 error responses
refs https://github.com/TryGhost/Toolbox/issues/410

- The 'private' value in 'Cache-Control' response header for all errors made it impossible for shared caches (e.g.: Fastly, Cloudflare) to cache 404 responses efficiently.
- The change substitutes 'max-age=0' which should not effect the browser cache behavior but would allow shared caches to process such requests efficiently.
- A more loose caching logic only applies to 404 responses from GET requests that are not user-specific (non-authenticated, non-cookie containing requests)
2022-09-26 14:54:50 +08:00
Naz
2acb0fca74 Refactored error cache control logic to middleware
refs https://github.com/TryGhost/Toolbox/issues/410

- This is groundwork for split cache-control rules for Admin app endpoints and the rest of Ghost apps.
2022-09-26 14:54:50 +08:00
Rishabh
6c85c75b86 Added referrer attribution data to member api
refs https://github.com/TryGhost/Team/issues/1961

- includes referrer source and medium information in member api
2022-09-24 17:46:57 +05:30
Ghost CI
c6865ffe6c v5.16.0 2022-09-23 16:00:34 +01:00
Simon Backx
e658f7622a
Added LinkClickTrackingService unit tests and renamed wrapper (#15462)
refs https://github.com/TryGhost/Team/issues/1958

- Renamed wrapper service link-click-tracking to link-tracking to be consistent with the package name
- Added unit tests for LinkClickTrackingService
- Added DomainEvents dependency to LinkClickTrackingService
- Fixes dependencies in link-tracking package
2022-09-23 16:19:16 +02:00
Daniel Lockyer
2bff2a22e0
Extracted dev tooling to separate script
refs https://github.com/TryGhost/Toolbox/issues/426

- we're going to need to support more complex combinations of dev
  commands soon, with other packages optionally running and env
  variables being altered
- this command pulls out a lot of the dev env scripting into a single
  scripts
- also cleans up the use of grunt-shell so we can remove the dependency
2022-09-23 15:02:35 +02:00
Simon Backx
3056e3cb51 Fixed subdirectories saved in the link_redirects table
refs https://github.com/TryGhost/Team/issues/1954

- Subdir should be removed before storing it as the 'from' URL
- Should be removed before checking a URL
2022-09-23 14:41:26 +02:00
Simon Backx
1290477d71
Added member last seen update on link click (#15459)
fixes https://github.com/TryGhost/Team/issues/1952

Adds a new MemberLinkClickEvent event that is fired when a member clicks a link. This code has been added to the `linkClickRepository` because that is the only place that has access to the member model (and the event requires the id and current last seen at value). The LastSeenAtUpdater listens for this event and updates the timestamp if required.
2022-09-23 10:34:33 +02:00
Naz
225a046bb8
Made Admin assets aggressively cacheable
closes https://github.com/TryGhost/Toolbox/issues/372

- The admin assets are served with a unique hash depending on the build with a year-long "max-age" value in the response cache-control header. The client browsers still do send 'If-None-Match' requests when there is a hard-refresh on the client side. There's no need for 'If-None-Match' requests though!
- With 'immutable' value in the cache-control header, the browser caches are treating responses as "hard-fresh" without sending redundant requests.
- For more about 'immutable' value read https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#immutable
2022-09-23 13:46:49 +08:00
Kevin Ansfield
c59ea73a80 🐛 Fixed distorted images in newsletters for product cards
refs https://github.com/TryGhost/Team/issues/1949

- bumps `@tryghost/kg-default-cards` which updates the rendered output for emails
- added `height: auto` style to the img element so clients don't render the image at the fixed image height retrieved from the `height="x"` attribute
2022-09-22 16:02:04 +02:00
Fabien 'egg' O'Carroll
5fcf5098a8
Added links API (#15446)
closes https://github.com/TryGhost/Team/issues/1927

This expose the /links endpoint on the Admin API, which is filterable by Post ID.

Co-authored-by: Simon Backx <simon@ghost.org>
2022-09-22 13:39:52 +02:00
Rishabh Garg
b048b02f67
Added new referrer source stats API (#15449)
closes https://github.com/TryGhost/Team/issues/1939

- adds new endpoint that returns count of referrer sources by date for admin dashboard
2022-09-22 16:34:26 +05:30
Simon Backx
7437d92d50
Added post referrers stats API (#15448)
closes https://github.com/TryGhost/Team/issues/1942

- Added data fixtures for referrers
- Added new endpoint to fetch referrer stats for a given post: `/stats/referrers/posts/:id`
- Added new ReferrersStatsService, responsible for calculating referrer stats
2022-09-21 18:16:56 +02:00
Rishabh Garg
b99c5428d0
Added referrer attribution columns to events table (#15436)
refs TryGhost/Team#1931

- referrer source, medium and url will be stored in the events table along with rest of attribution data
- stores referrer information on two tables
  - `members_created_events` for signups
  - `members_subscription_created_events` for paid conversions
2022-09-21 19:01:36 +05:30
Simon Backx
14f91093eb Fixed post counts cleared when member attribution flag was disabled 2022-09-21 11:32:00 +02:00
Simon Backx
719e5321d6 Fixed wrong usage of frontend labs in server
no issue
2022-09-21 11:25:29 +02:00
Simon Backx
b8041f0a60
Added clicks to activity feed (#15439)
closes https://github.com/TryGhost/Team/issues/1933

- Added click_events to activity feed
- Added support for parsing click_events in the frontend
- Moved url parsing (transform ready) to model layer of LinkRedirect
- Moved `getEventTimeline` method to the top of the event repository
- Added description field to parsed events in the frontend (because we need a second line)
- Fixed: member email not returned in comment_event
2022-09-21 10:25:51 +02:00
Kevin Ansfield
9f0bf7e40c Added aside and blockquote lexical rendering support
no issue

- bumped `@tryghost/kg-lexical-html-renderer` with support for `quote` and `aside` node types
2022-09-21 08:30:30 +01:00
renovate[bot]
5573e111cd Update dependency @playwright/test to v1.26.0 2022-09-21 07:31:56 +01:00
Ludovic Toinel
79ffdd9b7e
Improved handling of mailgun errors on bulk send (#15445)
- show both the status code and original error from mailgun
- clarify that the error is from mailgun
- swap from error to err as we're rolling out that pattern everywhere

Co-authored-by: Hannah Wolfe <github.erisds@gmail.com>
2022-09-20 22:46:55 +01:00
Sam Lord
364f92f021 Added mailgun as transaction email transport
refs: https://github.com/TryGhost/Toolbox/issues/281

As well as SMTP, we can now use Mailgun's API to send transactional email.
2022-09-20 15:44:26 +01:00
Hannah Wolfe
f31a50270d Removed search labs flag
- the search helper was promoted to GA and we no longer need the feature flag
2022-09-20 15:22:06 +01:00