Commit Graph

989 Commits

Author SHA1 Message Date
Simon Backx
e879406659
Added outbound link tagging setting (#16146)
fixes https://github.com/TryGhost/Team/issues/2432
    
Adds outbound_link_tagging setting (enabled by default and behind
feature flag). If the feature flag is enabled, and the setting is
disabled, we won't add ?ref to links in emails.
    
This includes new E2E tests for email click tracking, which were also
extended to check outbound link tagging (for both MEGA and the new email
stability flow).

Also fixes a test fixture for the comments_enabled setting.
2023-01-20 13:41:36 +01:00
Simon Backx
8a95c62ff1
🐛 Fixed post links being marked as edited when they were not (#16153)
fixes https://github.com/TryGhost/Team/issues/2461

- Ignores 'edited' links when there is only one second differences.
- Make sure we don't set updatedAt when linking a post to a redirect
2023-01-20 13:31:54 +01:00
Ronald Langeveld
3061fb2b3b
Added mentions database table (#16150)
closes https://github.com/TryGhost/Team/issues/2417

- added new `mentions` database table to be able to store incoming webmentions.
- updated schema and tests to match.
2023-01-20 19:08:07 +08:00
Fabien "egg" O'Carroll
d0d45d45bc Refactored OEmbed service into a proper service
This allows us to share the implementation with other parts of the codebase, the
specific usecase here being fetching the metadata from webmention sources, for
display in the mentions UI, which will be borrowing a lot of stuff from the
bookmark card.
2023-01-19 18:41:49 +07:00
Daniel Lockyer
5b2fc983c6
Added email snapshots to settings BREAD service test
refs https://github.com/TryGhost/Toolbox/issues/499

- switching to email snapshots allows us to track more of our output to check for regresssions
2023-01-18 14:19:27 +01:00
Simon Backx
6c2af0793c Fixed failing Stripe webhook test
refs acf0baa8c7

Due to the bump in express-test, we now handle string bodies 'properly'. So they now pass all the Express middlewares. In the past this failing test did not really pass by the bodyParser.raw middleware,
so the content-type check on the `bodyParser.raw({type: 'application/json'})` middleware was not executed. Now it is, and the test fails because the content-type header was not set to application/json.
2023-01-18 12:05:55 +01:00
Daniel Lockyer
9ba251238a Added Content-Version header to all API requests
refs https://github.com/TryGhost/Team/issues/2400

- we've deemed it useful to start to return `Content-Version` for all
  API requests, because it becomes useful to know which version of Ghost
  a response has come from in logs
- this should also help us detect Admin<->Ghost API mismatches, which
  was the cause of a bug recently (ref'd issue)
2023-01-18 08:38:07 +01:00
Daniel Lockyer
85051199e3
Added email snapshots for API versioning tests (#16139) 2023-01-18 08:09:28 +01:00
Naz
967dd7d244
Added deprecation notice to mockManager's sentEmailCount
refs https://github.com/TryGhost/Toolbox/issues/499

- The mockManager's sentEmailCount is left here to avoid breaking many tests that already depend on this method. With future improvements to email snapshot tests this method should not be used. Instead, emailMockReceiver's own sentEmailCount method should be used directly.
2023-01-18 12:55:45 +08:00
Simon Backx
34d4e5c8d0
Added logging to webmention endpoint (#16137)
closes https://github.com/TryGhost/Team/issues/2424

Adds the new webmention endpoint, responds with the correct statuscode
and empty body at `https://site.ghost/webmentions/receive/`.

Also updates the head link tag to the new endpoint location. We don't
use /webmention because that could conflict with post slugs.
2023-01-17 14:26:43 +01:00
Naz
112ab23968
Removed direct use of @tryghost/jest-snapshot
refs https://github.com/TryGhost/Toolbox/issues/499
refs 6bcc47a0ad

- Using module directly caused issues with snapshots manager instance initialization (mocha hooks did not apply to a correct instance)
- See refed commit for more
2023-01-17 12:50:49 +08:00
Naz
1f6b2b4d5e
Added email content testing capabilities
refs https://github.com/TryGhost/Toolbox/issues/499

- Outgoing emails have been a weak point of Ghost's stability recently. The concept of "emailMockReceiver" similarly to "webhookMockReceiver", allows to test side-effects like outgoing emails.
- This is a first iteration which should lay groundwork for testing all outgoing emails in the future
- The change adds a new concept of "email mock receiver" which is very similar to how the "webhook mock receiver" works. The email mock receiver exposes two methods to record and verify snapshots:
- matchHTMLSnapshot - records and verifies only the HTML content of the outgoint email
- matchMetadataSnapshot - records and verifies all the non-HTML properties sent along an email content, e.g.: to address, plaintext, subject, etc.
- What's missing is matching content based on dynamic content like dates, links with JWT tokens, etc.
2023-01-17 12:50:49 +08:00
Fabien 'egg' O'Carroll
a30bbd5c69
Added webmention endpoint discovery link to ghost_head (#16126)
We've wrapped both changes in a try/catch to make sure this has no
adverse affects. The endpoint currently doesn't exist - we're only
adding this to get an idea of how much traffic we'll expect to see.

Long term we'll want to read the endpoint from the webmention service.
2023-01-16 12:32:04 +07:00
Fabien 'egg' O'Carroll
14f282d640
Improved newsletter delivery rates and email suppression handling
This introduces the new suppressions feature which will automatically
unsubscribe members from newsletters when their email is added to the
suppression list in Mailgun, this is usually due to emails either
permanently bouncing to the address, or the member making a spam
complaint.

Both Members and Admins are able to see that the email has been added to
the list, and Members are be able to request their email be removed from
the list via Portal.

Overall this feature should improve delivery rates of newsletters and
improve the rating of the domain you're sending from.
2023-01-13 15:46:57 +07:00
Naz
c2b18f0c87
🔥 Removed support for {{lang}} helper
refs https://github.com/TryGhost/Toolbox/issues/497
refs https://github.com/TryGhost/Toolbox/issues/406

- Removes {{lang}} helper which has been completely deprecated since Ghost 5.0. When using the helper in themes with 5.x backend the output will be empty
2023-01-13 13:12:53 +07:00
Rishabh Garg
755e031017
Fixed hidden free tier in Portal settings (#16072)
closes https://github.com/TryGhost/Team/issues/2338

If a site has the Free tier hidden from the Portal, and subsequently the Stripe connection is disconnected, this produces a dead-end state where no new members can sign up and the Free tier cannot be reactivated again in Portal settings as its hidden. This change -

- enables free tier toggle to be always shown on site irrespective of Stripe connection
2023-01-12 19:29:17 +05:30
Rishabh
28bea791dd Added playwright tests for portal links
refs https://github.com/TryGhost/Team/issues/2371
2023-01-12 13:02:41 +05:30
Simon Backx
8a2303413d Restricted email-service package to 100% test coverage
fixes https://github.com/TryGhost/Team/issues/2339

The email service is now fully covered by tests, and this commit also forces the test coverage to remain 100% after future changes.
2023-01-11 13:54:26 +01:00
Ronald Langeveld
0e2d1b3afd
Added redeemed offers filtering for members (#16071)
closes https://github.com/TryGhost/Team/issues/2011

- Gives publishers the ability to filter members based on which offer they used (redeemed) when they subscribed for a paid membership.
- On the offers page, the redemption count number links to a the members page with the filter already applied making it easy to have insight on which members used the offer / coupon.
2023-01-11 20:13:09 +08:00
Elena Baidakova
1aae5ba2eb
Added e2e tests for default tier (#16103)
refs TryGhost/Team#2371
2023-01-11 15:44:34 +04:00
Elena Baidakova
a3f7188369
Added e2e tests for archive/unarchive tiers (#16101)
refs TryGhost/Team#2371
2023-01-11 13:53:07 +04:00
Fabien "egg" O'Carroll
49af26279d Updated MailgunEmailSuppressionList to only handle previous bounces
We're seeing behaviour from Mailgun where permanent failures with a
5xx error code are not being added to their internal suppression list,
which is resulting in the Ghost list becoming out of sync with
Mailgun.

Rather than adding emails to the suppression list when Mailgun does,
we're instead going to add emails _after_ Mailgun does, by waiting for
an error code which tells us the email is already on the suppression
list.

Those codes are 605 for previous bounces and 607 for previous spam complaints.
2023-01-11 15:10:39 +07:00
Elena Baidakova
9c664c8d02
Added data attributes to admin/tiers (#16095)
refs TryGhost/Team#2371
2023-01-11 11:35:55 +04:00
Simon Backx
117afb232c Improved stability of batch sending test
refs https://github.com/TryGhost/Ghost/actions/runs/3884901582/jobs/6628075997

The 'Doesn't include members created after the email in the batches' test sometimes failed randomly because of timing. This change ensures a strict timing.
2023-01-10 17:12:56 +01:00
Simon Backx
9ca2e3f183 🐛 Fixed storing email recipient failures
fixes https://github.com/TryGhost/Team/issues/2398

There was an error when fetching the existing email recipient failure. It ended up matching all recipient failures. The result was that only one failure was stored in the database.
2023-01-10 15:41:42 +01:00
Rishabh
7a74ec8daa Updated playwright tests to use updated data ids
refs https://github.com/TryGhost/Team/issues/2371

- cleans up and adds comments for portal playwright tests
- updates data test attributes for portal trigger and popup selectors for consistency
- updates data attribute usage for offers
2023-01-10 13:33:12 +05:30
Rishabh
628b80778f Updated create tier playwright util for all archived tiers
refs https://github.com/TryGhost/Team/issues/2371

- in case all tiers are archived before new tier is created, the add tier section can be collapsed and will need to be opened first before going through add tier flow
2023-01-10 09:34:21 +05:30
Simon Backx
26635f192f Added visible theme errors in admin
fixes https://github.com/TryGhost/Team/issues/2393
2023-01-06 15:51:34 +01:00
Simon Backx
f6e463ea43 Fixed missing awaits in publishing Playwright test
no issue
2023-01-06 14:18:54 +01:00
Rishabh
54b27f7d55 Fixed offers playwright test for redemption count
- updates check of redemption count using data test attributes to avoid unreliable member count check
2023-01-06 18:39:38 +05:30
Simon Backx
7b3712a15b
Added visible theme errors in admin (#16081)
refs https://github.com/TryGhost/Team/issues/2393

- During boot and loading the active theme, we now cache the result of
the gscan validation. Cache configuration can happen in
`adapters.cache.gscan`
- We now also return non-fatal errors when activating or adding a theme.
- When the `themeErrorsNotification` feature flag is on, we fetch the
active theme (which includes the validation information) when loading
admin
- If the currently active theme has errors, we show an error
notification that can open the error modal
- Added a new endpoint: `/ghost/api/admin/themes/active/` that returns
the result of the last gscan validation of the active theme. If no cache
is available, it will run a new gscan validation.
- Added new permissions for the active action/endpoint (author, editor,
administrator)
2023-01-06 13:44:27 +01:00
Naz
3e209218d7
Made resource mismatch error more specific
refs https://github.com/TryGhost/Toolbox/issues/497

- During gscan fatal error downgrade to non-fatal some of the deprecated helpers were a bit vague to debug with no information on which exact "resource" was invalid
- Added resource name to the log for clarity. Should make life easier when debugging potential get helper misuses
2023-01-06 18:05:07 +07:00
Sam Lord
36e7e79eec Improved Playwright test, selector for redemption count
no issue

Selector for redemption count was not ideal - this makes it more specific & clear.
2023-01-06 10:56:19 +00:00
Rishabh
30c9dfd68d Fixed failing playwright test to remove member labels
- the test was using incorrect test state that was copied over from adding label test
- also adds guard for empty newsletters in member filters as in some cases it might not exist as found by test
2023-01-06 16:17:34 +05:30
Ronald Langeveld
e52f29231f Added Playwright test - remove labels from members
ref https://github.com/TryGhost/Team/issues/2371

- check that a fitered list of members can have a label removed from
  them at once.
2023-01-06 14:36:13 +08:00
Fabien "egg" O'Carroll
953f3856a8 Handled EmailBounceEvent with 605 error code
When Mailgun fails to deliver an email to an address because the
address has already bounced before, it gives us a permanent fail event
with a 605 error code rather than a 5xx one. Because we want to
"backfill" our suppressions data with previously bounced email
addresses, we want to handle this specific error code.

We may update this logic in the future based on new information from
Mailgun with respect to their 6xx error codes and the
meanings/underlying cause of theme.

This also moves the tests which check for whether or not emails are
suppressed into their own fail so that we do not pollute the event
storage tests, and adds more tests cases.

We also fix a leaky sinon stub which we were not resetting in the email
event storage tests
2023-01-05 17:11:37 +07:00
Fabien "egg" O'Carroll
2d9114450c Fixed email_recipients fixtures for tests
The email_recipient fixtures were using duplicate and mismatched email addresses
rather than having them correctly map to the Members, which is required for testing
email suppressions.
2023-01-05 17:11:06 +07:00
Simon Backx
913ad18b71
Added DomainEvents.allSettled utility method (#16075)
no issue

With the increased usage of DomainEvents, it gets harder to build
reliable tests without having to resort to timeouts. This utility method
allows us to wait for all events to be processed before continuing with
the test.

This change should speed up tests and make them more reliable.

It only adds extra code when running tests and shouldn't impact
production.
2023-01-04 14:30:35 +01:00
Rishabh
f5aa07a095 🐛 Removed free trial message shown on portal for invite only sites
closes https://github.com/TryGhost/Team/issues/2361

If a free trial tier existed on site and its set to 'Invite only' in membership settings, the free trial copy still showed on portal.

- removes free trial copy from portal if site is invite only
- adds playwright test to make sure free trial copy is not shown for invite only sites
2023-01-04 17:11:55 +05:30
Fabien 'egg' O'Carroll
50e99e013c
Added migrations to drop and recreate the suppressions table (#16070)
There are currently two issues with the suppressions table:
  - We have some incorrect rows
  - We have missing UNIQUE constraints

We want to completely wipe the tables and start fresh, as well as make
sure that the UNIQUE constraints are added, so we drop the table
completely, and then re-add it, which should result in an empty
suppressions table with all expected constraints.

We've also renamed the `email_address` column to `email` to match our
`users` & `members` tables
2023-01-04 17:26:57 +07:00
Simon Backx
819d0d884c
Improved email verification required checks (#16060)
fixes https://github.com/TryGhost/Team/issues/2366
refs https://ghost.slack.com/archives/C02G9E68C/p1670232405014209

Probem described in issue.

In the old MEGA flow:
- The `email_verification_required` check is now repeated inside the job

In the new email service flow:
- The `email_verification_required` is now checked (didn't happen
before)
- When generating the email batch recipients, we only include members
that were created before the email was created. That way it is
impossible to avoid limit checks by inserting new members between
creating an email and sending an email.
- We don't need to repeat the check inside the job because of the above
changes

Improved handling of large imports:
- When checking `email_verification_required`, we now also check if the
import threshold is reached (a new method is introduced in
vertificationTrigger specifically for this usage). If it is, we start
the verification progress. This is required for long running imports
that only check the verification threshold at the very end.
- This change increases the concurrency of fastq to 3 (refs
https://ghost.slack.com/archives/C02G9E68C/p1670232405014209). So when
running a long import, it is now possible to send emails without having
to wait for the import. Above change makes sure it is not possible to
get around the verification limits.

Refactoring:
- Removed the need to use `updateVerificationTrigger` by making
thresholds getters instead of fixed variables.
- Improved awaiting of members import job in regression test
2023-01-04 11:22:12 +01:00
Fabien 'egg' O'Carroll
e78612bb66
Fixed MailgunEmailSuppressionList adding non-5xx failures to the list
The MailgunEmailSuppression list was incorrectly adding emails
to the suppression list for permanent failure events which have
an error code outside of the 5xx range.
2023-01-04 17:03:52 +07:00
Simon Backx
789e2c96c0
🐛 Fixed SingleUseTokens being cleared on boot (#15999)
fixes https://github.com/TryGhost/Team/issues/1996

**Issue**
Our Magic links are valid for 24 hours. After first usage, the token
lives for a further 10 minutes, so that in the case of email servers or
clients that "visit" links, the token can still be used.

The implementation of the 10 minute window uses setTimeout, meaning if
the process is interrupted, the 10 minute window is ignored completely,
and the token will continue to live for the remainder of it's 24 hour
validity period. To prevent that, the tokens are cleared on boot at the
moment.

**Solution**

To remove the boot clearing logic, we need to make sure the tokens are
only valid for 10 minutes after first use even during restarts.

This commit adds 3 new fields to the SingleUseToken model:
- updated_at: for storing the last time the token was changed/used). Not
really used atm.
- first_used_at: for storing the first time the token was used
- used_count: for storing the number of times the token has been used

Using these fields:
- A token can only be used 3 times
- A token is only valid for 10 minutes after first use, even if the
server restarts in between
- A token is only valid for 24 hours after creation (not changed)

We now also delete expired tokens in a separate job instead of on boot /
in a timeout.
2023-01-04 09:49:39 +01:00
Kevin Ansfield
0b80a7f9eb
Fixed new tier creation browser tests (#16066)
refs https://github.com/TryGhost/Ghost/pull/16013

- ordering of tiers changed so newly added tiers are no longer always at the end of the list in Admin
2023-01-03 17:59:33 +00:00
Simon Backx
803bb18b8d Improved unit test coverage for the output mappers
no issue

Increases the unit test coverage to a bit above 60% to increase margin
2023-01-03 15:58:31 +01:00
Elena Baidakova
235446b034
🐛 Fixed tiers order by monthly_price (#16013)
refs TryGhost/Team#2273
- According to documentation (https://ghost.org/docs/content-api/#order)
tiers should be ordered from lowest to highest monthly price.
2023-01-03 12:59:18 +04:00
Simon Backx
f568b35f26 Fixed 2023 snapshot tests
no issue

Updated the snapshot files for the email preview test snapshots that contain 2023 in the email footer.
2023-01-02 11:18:07 +01:00
Rishabh Garg
2eac41b1f4
🐛 Fixed invalid email getting saved for members (#16021)
refs https://github.com/TryGhost/Team/issues/2235

We found some cases which can cause a site to have member emails that have invalid characters like `member@example.com�`. This happened due to the `validator` version used by Ghost not able to catch some specific cases as invalid email, allowing members to be created with them either via Admin or Importer or direct signup. Portal UI already blocked these email as invalid. This change:

- updates `@tryghost/validator` to include a latest version of email validator that catches these invalid cases
- doesn't allow member creation with invalid email like above
- doesn't allow existing member emails to be edited to invalid
2022-12-16 16:47:52 +05:30
Hannah Wolfe
5f90baf6fe
Added Revue Importer (#16012)
refs: https://www.getrevue.co/app/offboard

- Revue is stopping all paid subscriptions on 20th Dec, and shutting down on Jan 18th.
- This update allows Ghost to accept and handle the zip file Revue are providing as an export in Labs > Importer
- It will import posts (as best as we can with the data provided) and subscribers as free members
- At present it doesn't import paid subscribers, as we don't have that info, but you can disconnect Revue from your Stripe account to prevent all your subscriptions being cancelled & there's the option this can be fixed later
- There will be further updates to polish up this tooling - this is just a first pass to try to get something in people's hands

Co-authored-by: Paul Davis <PaulAdamDavis@users.noreply.github.com>
2022-12-15 17:22:54 +00:00
Simon Backx
0825a2d7f4
🐛 Fixed importing existing member resetting newsletters (#16017)
fixes https://github.com/TryGhost/Team/issues/2386

**Issue:**
- When trying to import a member that already exists, and has
'subscribed' set to 'true' in the CSV, the newsletters the member is
subscribed to are reset to the default newsletters.
- When ediging a member with the API and setting `subscribed` to true,
the same happens.

**Cause:**
A faulty check for the `status` property of a newsletter.

Fixed and added a new E2E test.
2022-12-15 17:45:11 +01:00
Hannah Wolfe
5fad27a3c0
Added html -> mobiledoc conversion to the importer (#16016)
- Now that the importer runs in a job, it seems sensble that we should
do this
- If posts are imported with HTML set, but not mobiledoc, we now convert html -> mobiledoc
- Note: This also converts the mobiledoc -> html so _may_ be lossy
- Without this, imports that only have HTML, not mobiledoc, would have
resulted in empty posts, so lossy > empty
2022-12-15 16:10:36 +00:00
Simon Backx
94e85dc09e
Reduced webhook calls when updating last_seen_at for email opens (#16008)
refs https://ghost.slack.com/archives/C02G9E68C/p1670960248186789

This reverts a change that was made here:

f4fdb4fa6c (r93071549),
but it still moved the original code to a new location in the
LastSeenAtUpdater

It includes a new E2E test to make sure timezones are supported
correctly.

- By not using Bookshelf, we no longer fire webhook calls
- By not using the member repository, we don't fetch and update the
member model and the labels relation in a forUpdate transaction, which
caused deadlock issues on the labels/members_labels tables which were
hard to resolve. Until now I was unable to find the other conflicting
transaction that caused this deadlock. Moving to raw knex (instead of
Bookshelf) and only updating the last_updated_at column should remove
the deadlock issue.

This removed the test for the email service wrapper, since it started
failing for an unknown reason and the test didn't make much sense (was
added earlier only to bump test threshold).
2022-12-14 17:50:42 +01:00
Hannah Wolfe
270f288c48 Added a timeout to the get helper
- The get helper can sometimes take a long time, and in themes that have many get helpers, the request can take far too long to respond
- This adds a timeout to the get helper, so that the page render doesn't block forever
- This won't abort the request to the DB, but instead just means the page will render sooner, and without the get block
2022-12-14 15:35:07 +00:00
Kevin Ansfield
a69ffb94c1 Added playwright test for publishing posts for specific tiers
refs https://github.com/TryGhost/Team/issues/2371

- test publishes a post with access for a single tier then checks the front-end with no member, member on wrong tier, and member on right tier
2022-12-14 12:20:55 +00:00
Rishabh
25c530293f Added playwright tests for forever and multiple month offers
- adds test that cover creating and signing up to multiple-month/forever offers
- checks that the offer information is shown to members during signup and in account detail
2022-12-14 17:13:20 +05:30
Rishabh
199d8644a5 Fixed offers playwright test for one-time discount
refs aa693039a3

- one-time offer discounts are no longer shown to member in Portal settings, as portal shows only the next payment info.
2022-12-14 16:27:01 +05:30
Simon Backx
47cd7a7095
🐛 Handled unknown Mailgun events (#15995)
refs https://ghost.slack.com/archives/C02G9E68C/p1670916538764019

- We receive events that don't have an emailId or providerId.
- We filter those events now and log them as an error
2022-12-14 11:17:45 +01:00
Rishabh
aa693039a3 🐛 Removed expired offers shown in portal account detail
closes https://github.com/TryGhost/Team/issues/2376#event-8026429598

- if an offer is expired/in past, we no longer show it in member account info against the price
- one-time offers are never showed in portal in member account detail, as the payment information shown to member in Portal points to charge at next payment
- if trial days are over for a subscription, portal doesn't show any offer data on member account detail
2022-12-14 15:27:49 +05:30
Elena Baidakova
9579791185
Resubscribe member to default newsletters after removing from suppression list (#15974)
refs TryGhost/Team#2372
2022-12-13 13:31:50 +04:00
Elena Baidakova
7ef157b17f
Improved count of items in e2e tests (#15986)
refs TryGhost/Team#2371
- We need to wait selectors before count them
https://github.com/microsoft/playwright/issues/14278

These changes needed to make the results of tests more stable on the CI
2022-12-12 16:51:31 +04:00
Rishabh
ce45571dc0 Updated email faqs in portal to show sender email address
refs https://github.com/TryGhost/Team/issues/2348

- updates email faqs to show the sender email from newsletter data
2022-12-12 15:09:32 +05:30
Rishabh
4e766e2584 Fixed broken playwright tests
refs https://github.com/TryGhost/Team/issues/2371

- playwright tests were broken due to state changes based on prev tests that were not accounted for
- in case of multiple newsletters, portal tests expected another step between stripe checkout for newsletter selection
- site settings test was disabling members, but not re-enabling it back
2022-12-12 14:50:34 +05:30
Elena Baidakova
88b056dc86
Added Playwright member's actions tests (#15968)
refs TryGhost/Team#2371
- check that members can unsubscribe from newsletters by toggling
preferences in their account settings
- check that member can log out
2022-12-12 10:13:08 +04:00
Rishabh Garg
109bce8096
Fixed portal comp upgrade playwright tests (#15977)
refs
f5aae1e2c5
refs
0f9ed54a6f

- changing playwright portal tests to work for single tier setup caused failure for comped upgrade tests as they were relying on button text that changed
2022-12-09 19:19:13 +05:30
Peter Zimon
527d718b76
Added extra assertions to Offers tests (#15956)
refs. https://github.com/TryGhost/Team/issues/2371

- two extra assertion was needed for discount and free-trials to check
if the offers are listed in ‘Active’ offers and the URLs load portal
2022-12-09 08:56:41 +01:00
Naz
ffd07ea744
Added Playwright "Private site" test
refs https://github.com/TryGhost/Team/issues/2371

- Test enabling private site and checking access with a password
- The test flow is lacking a check for site access through password
due to a Playwirght bug. This should be cleaned up in the future
2022-12-09 14:01:19 +07:00
Peter Zimon
be84593684 Fixed offers archive test
refs. https://github.com/TryGhost/Team/issues/2371

- deleteAllMembers was an unnecessary step
- since there's a generated code appended to the name of the archived offer, it had to be shorter to avoid potential naming conflicts
2022-12-09 07:48:24 +01:00
Ronald Langeveld
d62f09c588 Added filtered list labelling playwright test
ref https://github.com/TryGhost/Team/issues/2371

- Filters and then adds label to filtered members
2022-12-09 12:56:34 +07:00
Simon Backx
e7572fc111 Added Playwright test for disabled subscription access
refs https://github.com/TryGhost/Team/issues/2371

Portal should be disabled on the frontend, sending as a newsletter should no longer be an option when publishing
2022-12-08 17:58:03 +01:00
Djordje Vlaisavljevic
82aa9522dc Added playwright test for updating atier
refs TryGhost/Team#2371

- Tests that tier details can be updated
- Tests that the updates are reflected in Portal
2022-12-08 17:40:42 +01:00
Simon Backx
6af149120a Added Playwright test for invite only sites
refs https://github.com/TryGhost/Team/issues/2371

Tests if an invite only site disables sign up.
2022-12-08 17:27:30 +01:00
Rishabh
f5aae1e2c5 Updated playwright tests for portal member upgrade
refs https://github.com/TryGhost/Team/issues/2371

- updated setup to ensure single tier for testing single tier flows
- updated portal test to use utils and new data test ids
2022-12-08 21:09:19 +05:30
Simon Backx
08206f367f Removed publishing spec only test 2022-12-08 15:30:27 +01:00
Simon Backx
7c6e9c6ecf Added Playwright tests for scheduled post emails
refs https://github.com/TryGhost/Team/issues/2371

- Added some more utility methods to the Publishing spec
- Added tests for scheduled pusblish+send and send flows
2022-12-08 15:24:37 +01:00
Kevin Ansfield
f2e1d77a0f Fixed linter error 2022-12-08 14:02:27 +00:00
Simon Backx
80258cd396 Added Playwright tests for publishing post with email
refs https://github.com/TryGhost/Team/issues/2371

- Adds a test for publishing and sending
- Adds a test for email only sending
- Updated some util methods in the publishing spec to remove the dependency on the post bookmark (which is not present for email only posts)
2022-12-08 14:38:12 +01:00
Sam Lord
3449c49c10 Tidy up the mocking utility for Mailgun
no issue
Moved the mocking into the Playwright specific codebase, since this will only be used in browser based testing
2022-12-08 13:01:35 +00:00
Sam Lord
249802d62b Added mailgun mock to allow publishing workflow to run
no issue
2022-12-08 13:01:35 +00:00
Naz
4c6a86eca4 Added mocked Mailgun client to browser test env
refs https://github.com/TryGhost/Ghost/pull/15959

- To be able to test and intercept emails we need a mock in test environment - avoids making calls to the Mailgun API
2022-12-08 13:01:35 +00:00
Sodbileg Gansukh
3d6753a54a Added create comped member test
refs https://github.com/TryGhost/Team/issues/2371

A member can be granted a comp in admin, that account should be able to access paid content.
2022-12-08 20:59:12 +08:00
Fabien "egg" O'Carroll
0f9ed54a6f Tested upgrading to a paid subscription from comped
refs https://github.com/TryGhost/Team/issues/2371

Note that the "Choose" button is "Continue" when running this test
standalone so currently it needs to run with the full suite.
2022-12-08 19:54:56 +07:00
Kevin Ansfield
88c8bf7dbc Added compedPlan option to createMember test util
refs https://github.com/TryGhost/Team/issues/2371

Usage:

```
await createTier(page, {name: 'Silver', monthlyPrice: 5, yearlyPrice: 50});
await createMember(page, {email: 'silver@example.com', compedPlan: 'Silver'});
```
2022-12-08 12:17:31 +00:00
Kevin Ansfield
1f22edb47c Removed errant page.pause() 2022-12-08 12:09:54 +00:00
Kevin Ansfield
4eeaca7ab8 Extracted openPostSettingsMenu and setPostVisibility test utils
refs https://github.com/TryGhost/Team/issues/2371

- extracting the re-used actions to utils allows tests to be self-descriptive rather than relying on comments and keeps the selectors and related actions in one place to help refactoring if/when they change
2022-12-08 12:02:22 +00:00
James Morris
e44f10bc5d Updated Playwright test: Sign up for paid plan via portal - single tier
refs https://github.com/TryGhost/Team/issues/2371

- Includes new data-test-* attributes
- Includes better checking of paid tier from member
2022-12-08 11:54:32 +00:00
Kevin Ansfield
9d434b8105 Removed errant page.pause() 2022-12-08 11:27:47 +00:00
Peter Zimon
8e9245bb91 Added archive offers test
refs. https://github.com/TryGhost/Team/issues/2371

- Test for archived offers should be moved to ‘Archived’ view of the offer list in Admin, and the offer URL should redirect to the site's homepage for logged out visitors
2022-12-08 12:21:52 +01:00
Kevin Ansfield
df72182e2d
Further attempt to fix update-published-post test in CI (#15964)
refs https://github.com/TryGhost/Team/issues/2371

- bumped timeout between saving and refreshing to account for slower
speeds in CI
- increased specificity for the frontend text comparisons so the output
when failing is smaller and easier to parse
2022-12-08 10:36:33 +00:00
James Morris
991b60a342 Added the Playwright test: Sign up for paid plan via portal - single tier 2022-12-08 10:23:35 +00:00
Simon Backx
641364876c Added Playwright test to unschedule a post
refs https://github.com/TryGhost/Team/issues/2371

- Schedule and unschedule a post
- Also includes some extra status checks
2022-12-08 11:13:54 +01:00
Rishabh
ae3d85d2c4 Added playwright test for paid member switching plans
refs https://github.com/TryGhost/Team/issues/2371

- tests that a paid member can switch between monthly and yearly plans after logging in
2022-12-08 15:35:41 +05:30
Kevin Ansfield
7371addbc5 Another attempt to fix update-published-post test in CI
refs https://github.com/TryGhost/Team/issues/2371

- the 100ms timeout was enough for local tests to pass but was still failing on CI
- bumped to 200ms and skipped the creation of a new paragraph to reduce what the editor is doing
2022-12-08 09:59:20 +00:00
Kevin Ansfield
8ddf1f0215 Fixed update-published-post Playwright test
refs https://github.com/TryGhost/Team/issues/2371

- added timeout between clicking the editor and starting to type otherwise some of the typing events could be missed causing a mismatch in actual vs expected output
2022-12-08 09:39:05 +00:00
Ronald Langeveld
3e62764a64 Added csv validation, members export Playwright test
ref https://github.com/TryGhost/Team/issues/2371

- updated Member exports with csv validation
- added member fixtures to be loaded into Ghost to ensure filtering
  works correctly when downloading / exporting members csv.
2022-12-08 16:33:27 +07:00
Simon Backx
c1fc0d8888 Added Playwright test to schedule a Page
refs https://github.com/TryGhost/Team/issues/2371

Schedule a page and check if it is only published at the scheduled time.
2022-12-08 10:25:30 +01:00
Simon Backx
0dbc10a41a Added Playwright test for publishing a page
refs https://github.com/TryGhost/Team/issues/2371

Publish a page and verify that the page is published.
2022-12-08 10:24:35 +01:00
Naz
739bda9bb8
Fixed publishing type selectors in playwright
refs https://github.com/TryGhost/Team/issues/2371

- The "data-test-*" selectors in playwright did not work with publishing channel selectors. This is a quick hack to enable working around it
2022-12-08 14:38:59 +07:00
Ronald Langeveld
9a99b81ea4 Added Playwright test for filtered members export
ref https://github.com/TryGhost/Team/issues/2371

- Added test for exporting a set of filtered members
2022-12-08 13:20:29 +07:00
Fabien "egg" O'Carroll
939e3ce96f Fixed suppression list handling of spam complaint events
refs https://github.com/TryGhost/Team/issues/2351

We were storing all suppressions with a reason of bounced, rather than
spam for spam complaint events.
2022-12-08 13:02:36 +07:00
Fabien "egg" O'Carroll
5749a17910 Cleaned up EmailEventStorage tests
This ensures that services aren't required before boot, so that they are
initialised in the correct order.
2022-12-08 13:02:36 +07:00
Fabien "egg" O'Carroll
9736d942e1 Unsubscribed Members from newsletters when their email is suppressed
refs https://github.com/TryGhost/Team/issues/2367

This ensures that a Member is not considered subscribed to any emails, so that
counts for newsletter recipients are correct. Eventually we will filter members
on their email suppression status but this is not implemented yet.
2022-12-08 13:02:36 +07:00
Ronald Langeveld
7567997dbf Added Playwright test for member exports
ref https://github.com/TryGhost/Team/issues/2371

- Tests whether a CSV can be exported with all members
2022-12-08 12:27:41 +07:00
Sanne de Vries
0ebb3cbefe Added Playwright "Post visibility" tests
Refs https://github.com/TryGhost/Team/issues/2371

- Tests whether the post access selection of public, members, or paid-members matches the expected post visibility on the frontend.
2022-12-08 11:26:44 +07:00
Kevin Ansfield
460b031969 Added additional post setting changes to e2e published post update test
refs https://github.com/TryGhost/Team/issues/2371

- tests both the post body and settings in the PSM take effect when saving
2022-12-07 18:18:48 +00:00
Rishabh
e24a67b882 Fixed member upgrade portal playwright test
refs b6db85a5d6

- portal upgrade test used duplicate email which caused the member creation to fail
- also updates the tier selection for checkout
2022-12-07 22:45:12 +05:30
Simon Backx
b615c9f7d2
Added post scheduling Playwright test and loosened time restrictions (#15960)
refs https://github.com/TryGhost/Team/issues/2371

- Adds a test that schedules a post 5 seconds in the future and waits
for it to be published
- Reduced the time restrictions for scheduling: 
    - The minimum time in the frontend is now 5 seconds in the future (came
from 5 minutes in the future)
    - The time picker now suggests 10 minutes in the future instead of the
minimum scheduling time (came from 5 minutes)
    - In the backend, a post will be allowed to be scheduled if it is at
least 2 minutes in the past (came from 2 minutes in the future)
    - The scheduler will publish a post if it is at least 5 minutes in the
past, and maximum 5 minutes in the future (came from 2 minutes)
2022-12-07 17:29:36 +01:00
Djordje Vlaisavljevic
2464d45b4a Added playwright test for creating additional tier
refs TryGhost/Team#2371

- Tests that an additional tier can be created and that it'll appear in portal settings unselected by default
2022-12-07 17:26:11 +01:00
Rishabh Garg
b6db85a5d6
Added playwright test for free member upgrade via portal (#15961)
refs https://github.com/TryGhost/Team/issues/2371

- tests that a free member can upgrade to a paid tier via stripe checkout and the payment details are reflected in portal and member detail page on admin
2022-12-07 21:41:55 +05:30
Hannah Wolfe
205664f75f
Updated Playwright createTier util to be repeatable
- we have to close the premium tier list after creating a tier, else we can't create another one
2022-12-07 14:32:26 +00:00
Kevin Ansfield
3bd9e2ecfe
Added Playwright "Update published post" test (#15958)
refs https://github.com/TryGhost/Team/issues/2371

- tests modifying the content of a published post
- extracted publish flow into a `publishPost` function that returns a new browser page object with the newly created post loaded
2022-12-07 14:13:23 +00:00
Ronald Langeveld
5bd82bc6d7 Added Playwright test for Member Impersonation
ref https://github.com/TryGhost/Team/issues/2371

- tests whether an impersonation link gets generated as well as whether
  the link authenticates the user on a new page.
2022-12-07 19:26:23 +07:00
Simon Backx
31855ba75a Added Playwright publish only test
refs https://github.com/TryGhost/Team/issues/2371

Test publishing flow for publish only posts.
2022-12-07 12:03:56 +01:00
Sam Lord
d6072ea0e8 Record tests using same setup as the test runner
no issue
2022-12-07 10:59:45 +00:00
Ronald Langeveld
8c41807978 Added Playwright test for deleting members from Admin.
ref https://github.com/TryGhost/Team/issues/2371

- This test will delete the previously added and editor member, returning the initial member state with no members.
2022-12-07 17:00:36 +07:00
Daniel Lockyer
1e27dd638f
Added Playwright check that Offer redemption count was incremented
refs https://github.com/TryGhost/Team/issues/2369

- this checks whether the Offer redemption count is set to 1, which
  would be indicative that the Offer was successfully counted as
  redeemed
2022-12-07 16:52:58 +07:00
Ronald Langeveld
1a879f9550 Added label and unsubscribe toggle on members test.
ref https://github.com/TryGhost/Team/issues/2371

- Editing members requires Label prefilled before the test runs so we can test making changes to that data.
2022-12-07 15:45:00 +07:00
Ronald Langeveld
f5bc7353b0 Added Playwright test for editing a member in admin
ref https://github.com/TryGhost/Team/issues/2371

- tests for editing a member inside admin
2022-12-07 13:02:48 +07:00
Ronald Langeveld
d60c50751b Added playwright test for creating a new member in admin
ref https://www.notion.so/ghost/Critical-Paths-980dd089c3e74a6fbc619271f5a9ce42

- this test will add a new member via the admin.
- it also tests to ensure the added member is displayed after heading back to the members list page.
2022-12-07 12:35:16 +07:00
Daniel Lockyer
d64d8c0ddd
Added Playwright test for creating and using a free-trial Offer
refs https://www.notion.so/ghost/Critical-Paths-980dd089c3e74a6fbc619271f5a9ce42

- this test will create a free-trial Offer and go through Stripe
  checkout, ensuring the member is created as paid
- also changes some utils to support creating the free-trial offer vs
  just a discounted Offer
2022-12-07 11:29:05 +07:00
Daniel Lockyer
41e1f0b568
Removed workaround for Members modal bug
refs 3817f583fa

- this should no longer be needed because the underlying bug was fixed
  and the modal should no longer show
2022-12-07 10:51:44 +07:00
Daniel Lockyer
16f3ba573b
Updated ghost_head snapshots
refs de97d90cf9

- this should have been updated when I deleted some extraneous config
  that was lurking in the testing config files
2022-12-07 10:24:44 +07:00
Daniel Lockyer
570b697b4a
Fixed double slashes in Stripe webhook URL
- `getSiteUrl` already returns a trailing slash so we shouldn't add
  another one in the path
2022-12-07 10:18:21 +07:00
Naz
6ef5e6a7e0 Fixed local environment for playwright tests
refs https://github.com/TryGhost/Toolbox/issues/479

- this includes a handful of improvements to get Playwright working on a
  local environment including:
  - adding `testing-browser` environment so we don't nuke `development`
    environments, and makes all the necessary changes to get Ghost to
    behave when this is running
  - stopped running one global instance of Ghost as this doesn't provide
    a clean environment
  - copies a few default fixtures that are needed for the new
    environment
2022-12-07 09:44:05 +07:00
Daniel Lockyer
d04abc90b4
Added test group names
- this helps differentiate between the different tests we have available
2022-12-06 19:18:16 +07:00
Daniel Lockyer
97e5ca1de6
Extracted Playwright tests into separate files
- we should start to keep tests grouped by their area, so first we split
  by Admin tests and then Portal tests, and within that we split into
  setup/Tiers/Offers etc
2022-12-06 19:08:09 +07:00
Elena Baidakova
62681037f0
Added email_suppression property to response for PUT api/member method (#15946)
closes TryGhost/Team#2359
- Keep the same response structure for `api/member` GET and PUT methods
2022-12-06 13:51:26 +04:00
Ghost CI
54b693a359 Merged v5.25.1 into main 2022-12-06 05:14:41 +00:00
Simon Backx
c47891c3f6
🐛 Fixed setting delivered_at to null after hard bounce (#15942)
refs https://ghost.slack.com/archives/C02G9E68C/p1670075366333929?thread_ts=1669963540.980309&cid=C02G9E68C

When we receive a permanent bounce/failure, we set delivered_at to null.
But we don't want to lose this information.

Instead we should be able to handle recipients that both have failed_at
and delivered_at set.
2022-12-06 10:26:54 +05:30
Naz
879aad263d
Added email verification trigger test
refs https://github.com/TryGhost/Toolbox/issues/476

- The email verification trigger and host settings related bugs have been a cause of bugs in past releases.  The admin client verification source did not have any test coverage in the past.
- The members test suite size is getting out of hand. This test is quite verbose, because of the state it's trying to check.
- In the future we should consider splitting up Member API (and probably other) test suites into smaller pieces.
2022-12-05 18:24:28 +07:00
Naz
e170f293e3
Extracted sleep method to e2e framework module
no issue

- The sleep method has been used in 8 modules reimplementing the same thing over and over again. It's usually a sign of async event processing outside of the request/response loop. It's good to have a single point of implementation for a "hack" like this, so we could track it easier and address the even processing delay in a more optimal way centrally if it ever becomes a bottleneck
2022-12-05 17:26:29 +07:00
Sam Lord
00d223991d Improved browser test CI integration
refs: https://github.com/TryGhost/Toolbox/issues/481
2022-12-02 17:04:04 +00:00
Sam Lord
7bfe5db716 Ensure tests pass when pointing at a remote server
no issue

This will need some work, since we are introducing a 500ms delay to wait for a network request to return. Ideally the tier expander should eventually populate itself.
2022-12-02 17:04:04 +00:00
Sam Lord
47526bc175 Fixed Portal test in browser suite
no issue
2022-12-02 17:04:04 +00:00
Sam Lord
40903c3136 Fixed browser-based tests in local context
no issue

Local tests can now setup Stripe during the global setup process, and the webhook server is run out-of-process.
Running tests in CI against localhost will use environment variables to setup Stripe.
Providing a test URL will avoid setting up Stripe and will assume that it is already done.
2022-12-02 17:04:04 +00:00
Daniel Lockyer
bc0b1f6adc Added CI for running Playwright tests
🚧
2022-12-02 17:04:04 +00:00
Sam Lord
668e523ab4 Allow playwright tests to run locally and remotely
no issue

This commit allows tests to run remotely by replacing selectors with production-suitable ones (no [data-test...]).

It also allows running locally with Stripe webhooks by adding a new global setup function.
2022-12-02 17:04:04 +00:00
Simon Backx
ea72934c41
Improved email concurrency sending (#15923)
no issue

Send 10 batches at the same time
2022-12-02 15:30:02 +01:00
Simon Backx
2a3a72e4d5
Added email debug API (#15915)
fixes https://github.com/TryGhost/Team/issues/2346

- Adds email batch browse endpoint
- Adds email recipient failures browse endpoint
- Adds new fixtures and E2E tests for the new API
- Added support for snapshot tests to have 'nullable' types.
2022-12-02 09:44:32 +01:00
Simon Backx
4c166e11df
Added E2E tests for batch sending (#15910)
refs https://github.com/TryGhost/Team/issues/2339

- Includes a new pattern in the job manager that allows us to properly
await jobs.
- Added new convenience mocking methods to stub settings
- Tests the main flows for bulk sending:
    - Sending in multiple batches
    - Sending to multiple segments
    - Handling a failed batch and retrying that batch
- Fixes bug in batch generation (ordering not working)

In a different PR I'll add more detailed tests.
2022-12-01 13:43:49 +01:00
Fabien "egg" O'Carroll
c0e91c73d3 Added unique constraint to email_spam_complaint_events table
We can fetch the same event multiple times from Mailgun so we need to
be able to protect against inserting duplicate events in the
database. This will allow us to catch duplicate errors on insert when
handling complaint events.
2022-12-01 17:27:02 +07:00
Fabien "egg" O'Carroll
f9fdee1b28 Wired up storage of EmailSpamComplaintEvents
refs https://github.com/TryGhost/Team/issues/2337

This also removes the code which unsubscribes members from newsletters because
that is not the spec of the feature.
2022-12-01 17:26:59 +07:00
Simon Backx
d8187123af
Added storage for email failures (#15901)
fixes https://github.com/TryGhost/Team/issues/2332

Saves events in the database and collects error information.

Do note that we can emit the same events multiple times, and as a result
out of order. That means we should correctly handle that a delivered
event might be fired after a permanent failure. So a delivered event is
ignored if the email is already marked as failed. Also delivered_at is
reset to null when we receive a permanent failure.
2022-12-01 10:00:53 +01:00
Naz
80decf8303
Fixed invalid query strings in tests
closes https://github.com/TryGhost/Team/issues/2324

- It seemed like the "limit" query parameter did not work properly returning multiple entries from the endpoint. In reality the whole query string was ignored because of an error in the "filter" part of the query ^_^
2022-11-30 13:31:06 +07:00
Naz
65d910f43b
Increased timeout for the import job test
refs 9fe1274fbe

- Temporary fix to let the CI pass the test. Needs investigation into why MySQL is so much slower comparing to SQLite
2022-11-30 13:25:18 +07:00
Naz
9fe1274fbe
Fixed member import job test
refs https://github.com/TryGhost/Team/issues/2326

-  The job takes considerably longer to run with MySQL, so needed a longer sleep time. It's a temporary fix to unblock a broken build. We should investigate why the job takes so long to run on MySQL
2022-11-30 11:15:12 +07:00
Naz
fc9f8aebc1
Fixed ci test runs
refs 3e0ec26408

- The refed commit excluded integration/regression test suites from CI test runs by accident
2022-11-30 10:39:32 +07:00
Kevin Ansfield
457c672c6a
Added URL transform for image cards in Lexical documents (#15890)
refs https://github.com/TryGhost/Team/issues/2225

- updated the `formatOnWrite` transform map for posts to include the new `nodes` and `transformMap` options used by `urlUtils` for transforming node payload data
- added `nodes` to the `lexicalLib` module that pulls in our default nodes to be passed in to the URL transform utilities
- added `urlTransformMap` to the `lexicalLib` module that maps transform type and data type to URL transform utility functions that accept a single URL argument
2022-11-29 16:57:01 +00:00
Fabien "egg" O'Carroll
ba5b8ea33d Added email_spam_complaint_events table and model
refs https://github.com/TryGhost/Team/issues/2318

As with our other events, we've disabled destroy and edit static methods
on the bookshelf model.
2022-11-29 18:13:12 +07:00
Fabien "egg" O'Carroll
83be54af42 Added suppressions table and model
refs https://github.com/TryGhost/Team/issues/2317

This table is used for persisting the email suppression list.
We don't have a member_id column because emails, not members are suppressed.
2022-11-29 18:12:24 +07:00
Simon Backx
f5045b9bf7
Added email renderer implementation draft (#15877)
fixes https://github.com/TryGhost/Team/issues/2308

- Still has some missing pieces, but mostly works.
- Uses new handlebars template for emails
- When sending emails with the new email stability flag enabled, one
test email is now sent via the default smtp ghost mailer.
2022-11-29 11:27:17 +01:00
Simon Backx
f4fdb4fa6c
Added new email event processor (#15879)
fixes https://github.com/TryGhost/Team/issues/2310

This moves the processing of the events from the event-processor to a
new email-event-processor in the email-service package.

- The `EmailEventProcessor` only translates events from
providerId/emailId to their known emailId, memberId and recipientId, and
dispatches the corresponding events.
- Since `EmailEventProcessor` runs in a separate worker thread, we can't
listen for the dispatched events on the main thread. To accomplish this
communication, the events dispatched from the `EmailEventProcessor`
class are 'posted' via the postMessage method and redispatched on the
main thread.
- A new `EmailEventStorage` class reacts to the email events and stores
it in the database. This code mostly corresponds to the (now deleted)
subclass of the old `EmailEventProcessor`
- Updating a members last_seen_at timestamp has moved to the
lastSeenAtUpdater.
- Email events no longer store `ObjectID` because these are not
encodable across threads via postMessage
- Includes new E2E tests that test the storage of all supported Mailgun
events. Note that in these tests we run the processing on the main
thread instead of on a separate thread (couldn't do this because
stubbing is not possible across threads)

There are some missing pieces that will get added in later PRs (this PR
focuses on porting the existing functionality):
- Handling temporary failures/bounces
- Capturing the error messages of bounce events
2022-11-29 11:15:19 +01:00
Rishabh
64ac47f4ef Added table to store email recipient failures
refs https://github.com/TryGhost/Team/issues/2291

When sending out mails to individual recipients, its possible that recipient gets a temporary or permanent failure for receiving the mail. Temporary failures can generally get resolved after a bit when the recipient’s mail server accepts the email, unlike permanent failures. For both customer visibility and easier debugging on what went wrong while delivering to a particular recipient, we’ll store the permanent/temporary failure for a recipient.

- migration adds a new table that stores the failure information for the recipients
2022-11-29 15:19:36 +05:30
Ghost CI
4a3ac50a82 Merged v5.24.2 into main 2022-11-28 17:36:37 +00:00
Simon Backx
6e72767a50
Fixed verification trigger not working for large imports (#15887)
fixes https://github.com/TryGhost/Team/issues/2326

When importing more than 500 members, we didn't testImportThreshold at
the right time. It was called too early because the importing job was
not awaited. This also adds an E2E test for this case.
2022-11-28 18:22:10 +01:00
Elena Baidakova
1b784b5ec5
Added events to Post Analytics page (#15886)
closes TryGhost/Team#2313
- Added Sent event to Post analytics and Members feed. Now post can be
Sent or Received or Bounced.
- Excluded Delivered event from Sent filter on backend.
2022-11-28 17:43:35 +04:00
Naz
60d066b243
🔒 Disabled editable relations by default
refs https://github.com/TryGhost/Ghost/security/advisories/GHSA-9gh8-wp53-ccc6
refs https://github.com/TryGhost/Toolbox/issues/465

- Bookshelf relations allows us to edit relational records by default, which was used liberally in the codebase.
- Not having a clear track record of editable relations left the model layer prone to triggering unwanted nested saves and created a vulnerability where members were able to edit newsletter settings.
- With explicit editable relations it's easier to keep track of relations having editable access to related records. Makes the relational data modification pattern safer to use too.
- Anyone running 5.x should update to 5.24.1

Credits: Dave McDaniel and other members of [Cisco Talos](https://talosintelligence.com/vulnerability_reports)
2022-11-28 18:40:26 +07:00
Naz
22738b1b50 🔒 Disabled editable relations by default
refs https://github.com/TryGhost/Ghost/security/advisories/GHSA-9gh8-wp53-ccc6
refs https://github.com/TryGhost/Toolbox/issues/465

- Bookshelf relations allows us to edit relational records by default, which was used liberally in the codebase.
- Not having a clear track record of editable relations left the model layer prone to triggering unwanted nested saves and created a vulnerability where members were able to edit newsletter settings.
- With explicit editable relations it's easier to keep track of relations having editable access to related records. Makes the relational data modification pattern safer to use too.
- Anyone running 5.x should update to 5.24.1

Credits: Dave McDaniel and other members of [Cisco Talos](https://talosintelligence.com/vulnerability_reports)
2022-11-28 18:39:39 +07:00
Sam Lord
8b80233ae6 Added sample Portal test to PlayWright suite
refs: https://github.com/TryGhost/Toolbox/issues/479
2022-11-24 17:47:04 +00:00
Sam Lord
6fe5023e91 Added working Playwright tests
refs: https://github.com/TryGhost/Toolbox/issues/479
2022-11-24 15:11:40 +00:00
Hannah Wolfe
62cd52ff98 Improved Sentry server side error reporting
refs: https://github.com/TryGhost/Team/issues/1121
refs: 54574025e0

- The previous change to fall back to a generic error on the server side is resulting in lots of much less useful Sentry reports
- For unexpected errors, change what's sent to Sentry back to context
- This is done by adding a specific code, so we don't have to match on a string that might change
- Also add the error type, id, code & statusCode as tags to the events - these are searchable structured data
- Adding code as a tag also makes it possible to find all errors that showed the generic message
2022-11-23 12:37:24 +00:00
Hannah Wolfe
36b80f2dda Updated sentry to use @tryghost/version
- As demonstrated by my comments in the boot file, I thought sentry was already depending on the version package
- IMO it's undesirable to require package.json directly esp when we have a tool setup and ready for tis
- Added a bunch of tests to show that Sentry does roughly what we think
2022-11-23 12:37:24 +00:00
Elena Baidakova
8d9d22e5a7
Added member API for removing email from suppression list (#15867)
closes TryGhost/Team#2306
2022-11-23 14:41:00 +04:00
Simon Backx
4b4592630f
Added new email batch sending service (#15865)
fixes https://github.com/TryGhost/Team/issues/2284

New batch sending flow (still WIP). Logs the sent emails instead of actually sending them. Unit tests are coming in later commits.
2022-11-23 11:33:44 +01:00
Rishabh
5780fc2a93 Added new source and source type columns to emails table
refs https://github.com/TryGhost/Team/issues/2280

We are moving away from storing html and plaintext on email and instead will store the email data in source and source_type columns which allows us to store the email in other formats like mobiledoc and lexical. Storing in those formats allows greater flexibility for later html generation

- adds new `source` column that stores `mobiledoc`/`lexical`/`html` data for a newsletter
- adds new `source_type` column that stores one of `mobiledoc`/`lexical`/`html` to identify type of source
2022-11-23 15:04:11 +05:30
Rishabh Garg
d3267dd5b0
Added columns to store error information for email batches (#15859)
closes https://github.com/TryGhost/Team/issues/2290

Currently, if the whole batch of email fails to send we don’t capture
any errors directly tied to the batch. This makes it hard to debug which
and why a batch failed when debugging email errors. Going forward we'll
store the error information for a failing email batch directly that
allows easier debugging for batch.

- `error_status_code` : Captures statusCode returned by Mailgun,
available in error.status from the example batch error
- `error_message` : Captures short error message from Mailgun and
status, available in context object of batch error
- `error_data` : Captures while whole error json for a batch. As
mentioned in pitch, this will be huge data and we’ll figure out long
term how to best use this.
2022-11-23 13:13:49 +05:30
Sam Lord
f6870fa846 Added browser-based testing framework
refs: https://github.com/TryGhost/Toolbox/issues/479

Framework includes:
* command to run tests
* command to record tests
* mechanism for starting and stopping Ghost before and after each suite of tests
* mechanism for loading fixtures into Ghost before starting tests
* sample test for controlling Ghost Admin
2022-11-22 14:12:34 +00:00
Simon Backx
44f189b56a
Added email service package (#15849)
fixes https://github.com/TryGhost/Team/issues/2282

Added a new email service package that is used when the email stability
flag is enabled. Currently not yet implemented so will throw an error
for all entry points (if flag enabled).

Removed usage of `labs.isSet.bind` across the code, because that breaks
the stubbing of labs by `mockManager.mockLabsEnabled` and
`mockManager.mockLabsDisabled`. `flag => labs.isSet(flag)` should be
used instead.

All email depending tests now disable the `emailStability` feature flag
to keep the tests passing + make sure we still run all the tests for the
old flow while the email stability package is being built.
2022-11-21 10:29:53 +01:00
Simon Backx
5c2f0b9a4b
Removed getCreatedEvents and added verification trigger test (#15832)
refs https://github.com/TryGhost/Team/issues/2266

This removes the deprecated `getCreatedEvents` method in the event
repository and adds tests to the verification trigger to see if we don't
break anything.

Changes extracted from https://github.com/TryGhost/Ghost/pull/15831
2022-11-18 15:05:15 +01:00
Simon Backx
2220686113 Sped up comments and feedback tests by reusing Ghost app instance
refs https://ghost.slack.com/archives/C02G9E68C/p1668777066462859

Added a duplicate method to the mmebers agent, so we can reuse the same app instance and options, whenever we need multiple agents (each with their own cookies).
2022-11-18 14:58:35 +01:00
Fabien 'egg' O'Carroll
8283de99c8
Wired up EmailSuppressionList to Members Admin API (#15848)
refs https://github.com/TryGhost/Team/issues/2268

The approach of using the service to lead email suppression data as
opposed to bookshelf relations allows us to wire things up without
having implemented the database. The getBulkSuppressionData allows us to
do this without much of a DB performance hit.
2022-11-18 16:28:13 +07:00
Ronald Langeveld
08232bd538
Cleaned up GA Flags (#15762)
closes https://github.com/TryGhost/Team/issues/2126

- Cleaned up the following GA flags: `newsletterPaywall`, `freeTrial`, `compExpiring`, `searchHelper`, `emailAlerts`, `fixNewsletterLinks`.
2022-11-18 16:09:25 +07:00
Sam Lord
49605a9559 Enabled browser-based tests to start Ghost with default test fixtures
refs: https://github.com/TryGhost/Toolbox/issues/479
2022-11-17 17:00:54 +00:00
Sam Lord
9e54128a39 Reverted unrelated changes in test file
no issue
2022-11-17 14:40:24 +00:00
Sam Lord
5619348c26 Replaced "Open Importer" button with one that works
no issue
2022-11-17 14:40:24 +00:00
Sam Lord
fc291240d5 Updated importer test to use new object return format
no issue
2022-11-17 14:40:24 +00:00
Simon Backx
69228b2947
🐛 Fixed race condition when sending email (#15829)
refs https://github.com/TryGhost/Team/issues/2246

- This change helps avoid race conditions due to a lack of a transaction
in the email job. It also moves the status check before creating the
email batches (can take a while) to prevent other timing issues in case
the job got scheduled multiple times.
- Sets the patch option to true when changing the status of an email
batch. If we don't do this, the bookshelf-relations plugin might try to
save relations too. This could have caused a 'no rows updated' error.
- Added a test that tests if the email job can only run once
- Added logging to batching logic
2022-11-17 13:36:52 +01:00
Ronald Langeveld
e0787b4e83
Added specific newsletter support for bulk unsubscribes (#15742)
closes https://github.com/TryGhost/Team/issues/2013

Added support to bulk unsubscribe a selected (filtered) list on members from specific, selected newsletters.
2022-11-16 14:29:00 +07:00
Sodbileg Gansukh
78bff39c23
Update the cover image in default fixtures (#15817)
- updated the cover image to be simpler
- made the change in text fixtures as well, just to keep the fixtures in sync

Co-authored-by: Hannah Wolfe <github.erisds@gmail.com>
2022-11-15 21:19:50 +00:00
Simon Backx
cdccf5c46a
Reduced default post relations (#15798)
fixes https://github.com/TryGhost/Team/issues/2085

Don't load relations we don't need anymore for the posts table. And
reload the individual post when we open the analytics page with more
relations that we actually need.
2022-11-15 10:17:26 +01:00
Rishabh
d4c3f86ce0 Cleaned up old member activity code usage
refs https://github.com/TryGhost/Team/issues/2216

The `membersActivity` flag was an alpha feature to test the first versions of member analytics, and is no longer active or in use. This change removes the remaining pieces of code that are setup behind that flag and are no longer in use or accessible.
2022-11-14 10:44:10 +05:30
Sam Lord
ce05d84940 Removed Bluebird from importer
refs: https://github.com/TryGhost/Ghost/issues/14882

This commit totally removes Bluebird from the importer. Updated `@tryghost/promise` to use native async/await and refactored importer logic to avoid the need of `reflect()`.
2022-11-09 16:14:37 +00:00
Simon Backx
f4660b440b
Fixed sending emails when using MembersAgent.loginAs (#15791)
fixes https://github.com/TryGhost/Team/issues/1903

MembersAgent.loginAs sends email, asynchronously via events. Which
conflicts with tests that also test emails. We cannot properly await
these events, so this is currently fixed with a timeout of 200ms. But
this was too random and unreliable.
2022-11-09 16:25:42 +01:00
Ronald Langeveld
6214812ac0
Added e2e test for bulk member deletion (#15730)
ref https://github.com/TryGhost/Team/issues/1446

- tests the `delete` api endpoint method for deleting a bulk set of members.
2022-11-08 17:02:55 +07:00
Ghost CI
4b7dd71851 Merged heads/v5.22.9 into main 2022-11-08 09:04:04 +00:00
Fabien "egg" O'Carroll
69aa52bd8e 🐛 Handled deleted Stripe objects in the Stripe Checkout flow
closes https://github.com/TryGhost/Team/issues/2222

Whilst we were checking for Stripe objects being active, we were not
checking for them existing in Stripe. This adds handling to all read
request to Stripe in the payment link flow, so that we can gracefully
handle deleted objects.

We've also included an automated test which fails without this fix.

We've also improved the query to find Stripe Prices which will result
in less request to the Stripe API to check if it is valid.
2022-11-08 16:03:07 +07:00
Simon Backx
7cefd4f70b
Updated ghost head test to ignore Portal/Search version (#15777)
fixes https://github.com/TryGhost/Team/issues/2152

When updating the Portal of Search version, the snapshots will now
remain the same.
2022-11-07 15:09:26 +01:00
Simon Backx
231ceedc72
Readded activity feed pagination tests (#15776)
fixes https://github.com/TryGhost/Team/issues/2218

The test should run a lot faster now.
2022-11-07 15:08:56 +01:00
Ghost CI
1575304627 Merged heads/v5.22.7 into main 2022-11-07 10:46:01 +00:00
Naz
6989d08b19
Fixed validation for Members API newsletters endpoint
refs https://github.com/TryGhost/Toolbox/issues/465
refs 31546a6fd3/packages/admin-api-schema/lib/schemas/members.json (L93-L103)

- Updated Members API newsletter validation to match the subset of Admin API's members validation schema.
2022-11-07 17:18:36 +07:00
leonyangela
8c0b5520ce
Added e2e tests for post.rescheduled webhook (#15735)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-11-05 13:52:03 +00:00
leonyangela
e8d8e07dad
Added e2e tests for page.rescheduled webhook (#15736)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-11-05 13:51:03 +00:00
Kevin Ansfield
6a573d4511
Added lexical post support to email renderer (#15767)
closes https://github.com/TryGhost/Team/issues/2207

- adds conditional to the post email serializer to switch between
`mobiledocLib` and `lexicalLib` depending on which format the post
contains
2022-11-04 11:19:40 +00:00
Daniel Lockyer
c32a013087
Merged v5.22.5 into main
v5.22.5
2022-11-04 13:05:33 +07:00
Fabien "egg" O'Carroll
5a1364e46d 🐛 Fixed importer importing invalid Tier pricing data
closes https://github.com/TryGhost/Team/issues/2211

We were allowing paid Tiers to be imported with non-integer prices which was
causing the Admin to be bricked when attempting to load them. This adds some
validation to the price data of Tiers.
2022-11-04 11:27:15 +07:00
Naz
ac46c2f2e9
Fixed CORS vary header modification
refs https://github.com/TryGhost/Toolbox/issues/461

- The 'vary' header with 'Origin' value should only be set when an OPTIONS header is processed. Otherwise we are prone to leaking the vary header modification to further down in the request pipeline
2022-11-03 11:16:13 +08:00
Naz
9b2e36e4fb
Fixed CORS middleware unit test
refs https://github.com/TryGhost/Toolbox/issues/461

- The unit test was never using the "OPTIONS" request method, which did not actually trigger the full logic of the "cors" module used under the hood.
- Using the correct request method triggers all the right pathways and tests the state that's closer to the real world - for example the response does get "ended" instead of calling the "next" middleware.
2022-11-03 11:16:13 +08:00
Robin Cussol
0bccbcb083
Added e2e tests for page.deleted webhook (#15723)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-11-02 17:25:07 +00:00
Naz
a8ba8cc444
Added Vary value for CORS in Frontend
refs https://github.com/TryGhost/Toolbox/issues/461

- Having a 'Origin' in vary header value present on each `OPTIONS` allows to correctly bucket "allowed CORS" and "disallowed CORS" responses in shared caches
2022-11-02 17:23:47 +08:00
Naz
f581e33400
Added Vary value for CORS in Admin API
refs https://github.com/TryGhost/Toolbox/issues/461

- Having a 'Origin' in vary header value present on each `OPTIONS` allows to correctly bucket "allowed CORS" and "disallowed CORS" responses in shared caches
2022-11-02 17:23:47 +08:00
Naz
2f2aa36c75
Renamed options test suite cases
refs https://github.com/TryGhost/Toolbox/issues/461

- The previous names were poorly representing the state that was produced and checked by the test cases
2022-11-02 17:23:22 +08:00
e.baidakova
81e6a7c5bd 🐛 Add ability to cache comments count endpoint
closes TryGhost/Team#2094
- Comment counts request was changed from `post` to `get` to enable request caching.
2022-11-02 10:54:12 +04:00
Naz
26b0bbc623 Added a test suite for OPTIONS requests
refs https://github.com/TryGhost/Toolbox/issues/461

- The codebase has ambiguous behavior with OPTIONS request. Adding tests covering edge cases for all possible variations of OPTIONS responses is the first step to solving cahceability of these requests.
- The obvious question if you look into the changeset itself would also be: "WTF did you do with test suite naming? What are these changes in admin and click tracking suites? You having a bad day Naz?". The answer is "yes"  (╯°□°)╯︵ ┻━┻
- On a serious note. I've introduced multiple hacks here that should be fixed:
1. Forced test suite execution order for options request - extreme blasphemy. This was last resort decision. I went deep into trying to fixup the server shutdown in the "admin" test suite, which cascaded into failing "click tracking" suite, which has shortcomings on it's own (see notes left in that suite)
2. Exposed "ghostServer" from the e2e-framework's "getAgentsWithFrontend" method. Exposing ghostServer to be able to shut it down (or do other manipulations) was one of the pitfalls we had in the previous test utils, which ended up plaguing the test codebase. Ideally the framework should only be exposing the agents and the rest would happen behind the scenes.
- To fix the hacks above I've raised a cleanup issue (https://github.com/TryGhost/Toolbox/issues/471). I'm very sorry for this mess. The issue at hand has very little to do with fixing the e2e framework, so leaving things "as is".
2022-11-02 13:43:30 +08:00
Naz
5fe80c82c5 Exposed Content API agent in e2e test utils
refs https://github.com/TryGhost/Toolbox/issues/461

- When testing OPTIONS requests there is a need to get all possible agents available in the system. The "getAgentsWithFrontend" serves exactly this purpose - create all possible agents while starting Ghost instance only once
- This is groundwork for OPTIONS request caching tests and improvements
2022-11-02 13:43:30 +08:00
Daniel Lockyer
ab678c3a90
Merged v5.22.4 into main
v5.22.4
2022-11-02 07:09:23 +07:00
Fabien 'egg' O'Carroll
1ff1b75a69
🐛 Fixed errors with Stripe Checkout (#15749)
refs https://github.com/TryGhost/Ghost/commit/1f300fb781f0

The full customer object was not being passed to the StripeAPI service
when it already exists, this was resulting in inconsistent behaviour when
sending the customerEmail param to the API, causing `invalid_email`
errors to be thrown from Stripe and breaking the checkout.
2022-11-02 02:18:30 +07:00
Hannah Wolfe
85c5a19f33
Merged v5.22.3 into main
v5.22.3
2022-11-01 17:29:16 +00:00
Fabien "egg" O'Carroll
ba41f308c7 🐛 Fixed upgrading to a paid plan
closes https://github.com/TryGhost/Team/issues/2196

We were incorrectly assuming that all requests would have the
`customerEmail` passed in the body. Instead we were incorrectly
passing `undefined` or `''` as the `customerEmail` property to stripe,
which resulted in a validation error.

We've updated the code to pass `null` in the case of a falsy value,
which the Stripe API handles without error.
2022-11-02 00:11:33 +07:00
Fabien "egg" O'Carroll
1f300fb781 🐛 Fixed checkout sessions when using Offers
closes https://github.com/TryGhost/Team/issues/2195

The issue here is two-fold, and specific to using Offers so was not
caught by any automated tests. First, we were incorrectly comparing
the tier.id to the offer.tier.id - this is because the Tier objects id
property is an instance of ObjectID rather than a string.

Secondly we were passing through the cadence parameter from the
request body, but when using Offers this is not including in the
request, so we must pull the data off of the Offer object instead and
pass that to the payments service.
2022-11-02 00:11:33 +07:00
Daniel Lockyer
498bec08cf
Merged v5.22.2 into main
v5.22.2
2022-11-01 18:24:58 +07:00
Fabien 'egg' O'Carroll
a0ebb9a6f3
🐛 Fixed Tier description not being set (#15741)
closes https://github.com/TryGhost/Ghost/issues/15740

The validation function for a Tier description was not returning the
validated value, which meant we were unable to set the Tier
description.
2022-11-01 16:25:55 +07:00
Robin Cussol
e7e12aba09
Added e2e tests for page.published.edited webhook (#15724)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-10-31 19:51:04 +00:00
Daniel Lockyer
92740e8967
Merged v5.22.0 into main
v5.22.0
2022-10-31 17:49:27 +07:00
Fabien "egg" O'Carroll
f878e84707 Fixed Tiers importer not correctly mapping price data
refs https://github.com/TryGhost/Toolbox/issues/464

Bceause the import does not use the API, any backwards compat code we put in the
API does not get run for imports, this means we need to update the importer to
map the stripe_prices data onto the products table so that we have valid data in
the database.
2022-10-31 17:30:16 +07:00
Hannah Wolfe
25d5839e96
Added shouldCompileToError test util
refs: a8b1676734

- Extended the newly created handlebars test utils with a shouldCompileToError method
- Updated the price helper tests tp use shouldCompileToExpected and shouldCompileToError
- This allows us to test our handlebars helpers in a much more conisstent way
2022-10-30 16:55:53 +00:00
leonyangela
7df2b7625f
Added e2e tests for page.tag.attached webhook (#15648)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-10-30 16:45:51 +00:00
Hannah Wolfe
a8b1676734
Refactored shouldCompileToExpected into shared util
no issue

- There are currently two patterns in our handlebars helper unit tests:
  1. Treating the helper as a function, and doing a function call
   - This is the original way the tests were done, and they're not great as they're approximating how the helpers are really used
  2. Using a template string, and rendering the string using a method called shouldCompileToExpected
   - These tests are more realistic and powerful and also easier to read

- The new method is only being used in a few places so far, and each place had re-created the `shouldCompileToExpected` method
- Therefore I've moved this method into a util that should make it easier to write unit tests for handlebars helpers
- I also renamed the method in the excerpt tests, because it doesn't do the same thing, it's just a wrapper around a function call rather than compiling a string

The aim is to refactor all of our handlebars helper tests to use `shouldCompileToExpected`
2022-10-30 15:26:29 +00:00
Vedant Bhamare
3f289edc26
Added e2e tests for page.tag.detached webhook (#15651)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-10-30 14:59:12 +00:00
Hannah Wolfe
5fe6687c2a
Skipped slow activity feed tests temporarily
- These tests are very slow, and make the build fail about 2/3 times
- Temporarily skipping until we can fix, as I want to get all our outstanding hacktoberfest PRs merged
2022-10-30 14:52:30 +00:00
Hannah Wolfe
3ec0057567
Removed accidentally committed file
- This was accidentally included in a fixup commit
2022-10-30 14:38:59 +00:00
e.baidakova
d8fa21e43c Revert "Update portal version"
This reverts commit b5b7ce5f7a.
2022-10-27 23:36:58 +04:00
e.baidakova
b5b7ce5f7a Update portal version 2022-10-27 23:31:48 +04:00
Simon Backx
66e282f7e3 Fixed default setting count regression test 2022-10-27 18:37:45 +02:00
Elena Baidakova
df51a852f5
Added new feedback flow to portal (#15716)
- Updated the way we're handling feedback from user, due to email
restrictions (email client opened all links in email).
2022-10-27 20:34:45 +04:00
Simon Backx
13fd64ebf7 Fixed tests for flag bumps and new setting
no issue
2022-10-27 18:26:46 +02:00
Rishabh
6795e95483 Fixed settings test
refs f1aff45dc7

- adds missing default setting for member source tracking
2022-10-27 21:30:31 +05:30
Rishabh Garg
f1aff45dc7
Disabled attribution calculation when tracking is disabled (#15710)
refs https://github.com/TryGhost/Team/issues/2168

- forces attribution service to use empty history or context if attribution tracking is disabled
2022-10-27 21:10:03 +05:30
Simon Backx
b916300ceb
Added aggregated click events (#15713)
fixes https://github.com/TryGhost/Team/issues/2175

- New event type `aggregated_click_event` that is disabled by default in all the existing activity feeds
- This returns click events, but only the first click events for each member/post combination.
- It includes the total count of unique link clicks for that member on that post combination
- Had to resort to some custom knex queries to make this work easily
- Requires `@tryghost/bookshelf-pagination@0.1.31`, included in `@tryghost/bookshelf-plugins@0.6.1` (this fixes an issue with custom selects breaking the total count query of pages)
- Went a bit overboard with the pagination tests to cover as much unknown edge cases as possible
2022-10-27 17:23:45 +02:00
Rishabh Garg
6a619310f6
Removed attribution script if tracking sources is disabled (#15707)
refs https://github.com/TryGhost/Team/issues/2168

- site owners can now disable tracking sources from analytics settings.
- this change removes the loading of attribution script if tracking is
turned off so we don't capture any post/page or external source
attributions
2022-10-27 18:52:11 +05:30
Rishabh Garg
cca0f7d7dc
Added new setting to toggle tracking of member sources (#15705)
refs https://github.com/TryGhost/Team/issues/2168

- the new setting allows site owners to control if they want to track
the sources for new member signups and subscriptions
- its switched on by default, but can be toggled off from new analytics
settings page
2022-10-27 17:24:46 +05:30
Simon Backx
b911208b41
Improved filter support in activity API to allow pagination (#15684)
fixes https://github.com/TryGhost/Team/issues/2129

- This changes how the activity feed API parses the filter.
- We now parse the filter early to a MongoDB filter, and split it in two. One of the filters is applied to the pageActions, and the other one is used individually for every event type. We now allow to use grouping and OR's inside the filters because of this change. As long as we don't combine filters on 'type' with other filters inside grouped filters or OR, then it is allowed.
- We make use of mongoTransformer to manually inject a mongo filter without needing to parse it from a string value again (that would make it a lot harder because we would have to convert the splitted filter back to a string and we currently don't have methods for that).
- Added sorting by id for events with the same timestamp (required for reliable pagination)
- Added id to each event (required for pagination)
- Added more tests for filters
- Added test for pagination
- Removed unsued getSubscriptions and getVolume methods

Used new mongo utility methods introduced here: https://github.com/TryGhost/NQL/pull/49
2022-10-27 12:13:24 +02:00
Simon Backx
076e3c02b2
Added linking between member and subscription created events (#15693)
fixes https://github.com/TryGhost/Team/issues/2160

- Adds a `batch_id` to both events that contain the same ID if they were created at the same time.
- Removes duplicate signup/conversion events using the batch_id
- Requires an update in mongo-knex to work (refs https://ghost.slack.com/archives/C02G9E68C/p1666773313272409?thread_ts=1666767872.375009&cid=C02G9E68C)
- Some dependencies needed an update to load the latest mongo-knex
- Added tiers to membersUtils, loaded on startup (we can start to use this instead of fetching it every time)
2022-10-27 11:44:19 +02:00
Naz
6bd2769146
Hardened tiers-related snapshot tests
refs https://github.com/TryGhost/Team/issues/2077

- Members and Posts test suites were using a broad tiers property matcher, which is an anti-pattern for snapshot tests. Without more specific snapshots it would be very hard to track down tier-related breaking changes!
- This change is groundwork for a refactor coming in tier usage at API's output serializers
2022-10-27 15:54:28 +08:00
Elena Baidakova
e3ab868b83
Added email feedback column (#15698)
closes TryGhost/Team#2159
- Added column to email table
- Hide the feedback tab on frontend depending on the column value

Co-authored-by: Daniel Lockyer <daniellockyer@fastmail.com>
2022-10-27 11:22:50 +04:00
Elena Baidakova
57817eefc8
Added feedback buttons to emails (#15695)
closes TryGhost/Team#2075
2022-10-26 19:19:05 +04:00
Fabien "egg" O'Carroll
25d8d694a0 Wired up the payment service to create stripe checkout sessions
refs https://github.com/TryGhost/Team/issues/2078
2022-10-25 09:03:04 +07:00
Fabien "egg" O'Carroll
31610f9b94 Wired up Tiers service to Tiers Content & Admin API
refs https://github.com/TryGhost/Team/issues/2078
2022-10-25 09:02:59 +07:00
Hannah Wolfe
d260a7c78c
Updated webhook test snapshots
refs: 26d049911c
refs: 8c2f832573

- snapshots fell behind between the two referenced commits, and needed updating
2022-10-24 19:44:46 +01:00
Kritika Sharma
26d049911c
Added e2e tests for post.published.edited webhook (#15642)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.

Co-authored-by: Kritika Sharma <kritikasharma@Kritikas-MacBook-Pro-2.local>
2022-10-24 14:55:25 +01:00
Dominik Picheta
8c2f832573
Added e2e tests for post.unscheduled webhook (#15675)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-10-24 14:12:49 +01:00
Simon Backx
1f138893ff Fixed snapshot tests
refs 30ecaef329
2022-10-24 14:27:52 +02:00
Simon Backx
fd91f7eebb
Added email sent events (#15682)
fixes https://github.com/TryGhost/Team/issues/2137

For the analytics page, we need the sent events to show up immediately
after sending an email. Otherwise we need to wait for emails to be
marked as received (which takes too long) before being able to show them
on the analytics page.

This adds the email_sent_event, which is hidden by default everywhere
and used on the analytics page.
2022-10-24 11:11:44 +02:00
Halldor Thorhallsson
dbad621b91
Removed bluebird from fixture-utils.js (#15626)
refs: https://github.com/TryGhost/Ghost/issues/14882

- Opted to use the in-house `sequence` function when refactoring Bluebird's `Promise.each` to avoid deadlock issues (see 734ef66e6c). 
-It's hard to know without tonnes of context if any `Promise.each` are safe to refactor to `Promise.all`.
2022-10-21 20:56:21 +01:00
Elena Baidakova
9720459f55
Added pie chart for activity feed (#15673)
closes TryGhost/Team#2088
- Added pie chart to feedback event
- Added `negative_feedback` field to response from BE
2022-10-21 21:34:05 +04:00
Rishabh Garg
05330316e3
Allowed fixing newsletter links (#15672)
refs https://github.com/TryGhost/Team/issues/2116

- allows site owners to edit a link in a post that has already been sent out, fixing any typos or other mistakes
- resets click counter for the edited link back to 0 so site owners can see the clicks on new link, doesn't change the overall click count
2022-10-21 19:38:57 +05:30
Hannah Wolfe
3ab1c418bc
Updated webhook test snapshots
refs: 717a27c85c
refs: 6380b82793

- The snapshots just needed updating
2022-10-21 14:36:20 +01:00
Halldor Thorhallsson
717a27c85c
Added e2e tests for page.unpublished webhook (#15613)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-10-21 14:16:13 +01:00
Rishabh
3d355ea41d Fixed test snapshots
refs 5b283930f0
2022-10-21 18:14:57 +05:30
Samprit JC
f7738dc1a2
Added e2e test for post.unpublished webhook (#15628)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-10-21 13:08:54 +01:00
Shashank Gupta
092fb7813d
Added e2e tests for page.edited webhook (#15627)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-10-21 12:45:38 +01:00
illiteratewriter
4ba254b339
Added e2e tests for member.edited webhook (#15620)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-10-21 12:43:30 +01:00
Shubhadeep Das
5b283930f0
Added e2e tests for post.edited webhook (#15625)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-10-21 12:39:24 +01:00
Naz
b552727b69
Changed members export to contain tiers
refs https://github.com/TryGhost/Team/issues/1076

- The "products" is a legacy term for what is now "tiers" since multiple tiers feature introduction in https://github.com/TryGhost/Ghost/releases/tag/v4.39.0
- Note, the "tiers" is a field meant for informational purposes and cannot be imported back into Ghost site.
2022-10-21 18:58:59 +08:00
Rishabh
ba6fe63250 Fixed cached redirects for edited newsletter links
refs https://github.com/TryGhost/Team/issues/2135

The email link redirects on Pro are cached as 302 redirects in Varnish, so we're missing further clicks after the first one for each member, until the cache is invalidated. This change invalidates cache on link edits to ensure that we correctly redirect members to updated link everytime
2022-10-21 16:07:34 +05:30
Rishabh
d7b3721036 Added edited property to links api
refs https://github.com/TryGhost/Team/issues/2104

- adds a boolean `edited` property to links api that denotes if the link has been edited
2022-10-20 18:17:44 +05:30
Rishabh
d8bacf12d1 Added endpoint for fixing newsletter links
refs https://github.com/TryGhost/Team/issues/2104

- adds new bulk edit endpoint for links, updates all matching link with the current redirect url and update to new url
2022-10-20 17:50:02 +05:30
Simon Backx
adf10f0e76
Added support for filtering email events by post_id (#15666)
refs https://github.com/TryGhost/Team/issues/2093
2022-10-20 13:29:00 +02:00
Daniel Lockyer
02c8690e87 Added ghost_subscription_id column to members_stripe_customers_subscriptions
refs https://github.com/TryGhost/Team/issues/2034

- this table will be used to link Stripe subscriptions to Ghost
  subscriptions via a foreign key that we add at a later point
- this also includes `constraintName` as the auto-generated one would be
  too long for MySQL 8
2022-10-20 10:59:36 +07:00
Rishabh Garg
318a5a809c
Added permissions for link edit endpoints (#15664)
refs https://github.com/TryGhost/Team/issues/2104

- adds edit permissions for links endpoints to fixtures
- new `bulkEdit` endpoint will use the permissions and allow fixing newsletter links via Admin
2022-10-20 09:11:26 +05:30
Rishabh Garg
60b10ad69a
Fixed permissions for links endpoint (#15656)
refs 5fcf5098a8

- links browse endpoint had permissions switched off unintentionally and was also missing the necessary permissions in fixtures.
- enables permissions for browse endpoint and adds migration insert permissions in DB
2022-10-20 08:18:29 +05:30
Simon Backx
6380b82793
Added sentiment ordering and include for posts (#15657)
fixes https://github.com/TryGhost/Team/issues/2090

- This changes how sentiment is exposed in the API. Now it is exposed as a `sentiment` relation, directly on the model (no longer in counts). Internally we still use `count.sentiment`.
- Content API users (and themes) can include the 'sentiment' relation and order by sentiment.
- Updated Admin to use sentiment instead of count.sentiment
2022-10-19 16:50:58 +02:00
Naz
b589a66cd4
Fixed broken CSV importer tests
refs 90768e9985

- With introduction of strict field mapping the regression test testing for "imports of not mapped fields" failed.
2022-10-19 18:33:47 +08:00
Daniel Lockyer
9b8c33484d
Merged v5.19.3 into main
v5.19.3
2022-10-19 06:22:38 +07:00
Simon Backx
a822c5a8c5 Added test to check if feedback buttons are hidden if alpha flag is disabled 2022-10-18 16:47:06 +02:00
Simon Backx
a01fb5f1aa
Added post_id filter and total to activity feed API (#15650)
fixes https://github.com/TryGhost/Team/issues/2091
fixes https://github.com/TryGhost/Team/issues/2089

- Added new fixtures to make testing easier for the activity feed
- Improved E2E test coverage of activity feed with separate test file
- Added data.post_id filter to enable filtering by events related to a
given post
- Fixed return types in JSDoc of test agents (TypeScript interprets
these as `typeof Agent` if we don't add `InstanceType<Agent>`)
- Added total pagination metadata to activity feed API (to allow a basic
type of pagination using filters)
2022-10-18 15:52:04 +02:00
Daniel Lockyer
2dcc4139b1
Merged v5.19.2 into main
v5.19.2
2022-10-18 17:04:23 +07:00
Simon Backx
d1e6870740
🐛 Fixed large mailgun recipient data (#15638)
fixes https://github.com/TryGhost/Team/issues/2096

When generating the recipient data for emails, the email clicks
implementation is resulting in a recipient variable being added called
replacement_xxx once for each link containing the same UUID.

This generates a lot of unnecessary data overhead for emails, and it
turns out that mailgun has a 25MB message limit. We wouldn't have come
close if we only included the uuid once.
2022-10-18 10:32:50 +02:00
Daniel Lockyer
3858f255b9 Dropped nullable status on subscriptions.tier_id
fixes https://github.com/TryGhost/Team/issues/2102

- this column was added with `nullable: true` but it should never be
  nullable, so we should drop the nullable status whilst it's easy to
2022-10-18 14:16:30 +07:00
Daniel Lockyer
c9d43b8fe1
Allowed constraintName in schema column spec
refs 0ba3d6df49

- this is used to indicate the name of the foreign key constraint and so
  we should let it through the schema checks
2022-10-18 10:29:55 +07:00
Simon Backx
22fe1c01de
Added conversions count and separate analytics page (#15637)
fixes https://github.com/TryGhost/Team/issues/2084

- When audience feedback is enabled, we use a single 'conversions' count instead of having separate ones for signups and paid conversions.
- The analytics component is separated so we can change it without breaking the existing page.
2022-10-17 13:02:39 +02:00
Elena Baidakova
46141efe05
Updated test snapshot after bumping Portal (#15623) 2022-10-14 20:29:46 +04:00
Elena Baidakova
e831be6bc2
Added the feedback buttons in the emails (#15619)
closes TryGhost/Team#2046
closes TryGhost/Team#2045
- Added feedback buttons markup.
- Added feedback links generation.
2022-10-14 18:12:17 +04:00
Fabien 'egg' O'Carroll
8afc6777c0
🐛 Removed redirects from search engine indexing (#15617)
refs https://github.com/TryGhost/Team/issues/2072

Google is indexing our redirects and storign the redirected content
against the redirect URL in search results. This seems to be caused by
us using a 302 redirect rather than 301. We don't want to switch to a
301 however, so that we can support the ability to update redirects in
the future.
2022-10-14 15:51:43 +07:00
Fabien 'egg' O'Carroll
bd0f4b4b8c
Added Tier price and currency data to products table (#15366)
refs https://github.com/TryGhost/Team/issues/1765

In order to better handle deleted objects in Stripe we want to decouple
Members from Stripe.

These changes allow us to have the Tier concept completely independent
of the Stripe tables, such that the Stripe data can be generated as/when
it's needed - which will help to protect against missing data.
2022-10-14 06:40:17 +01:00
jbenezech
75811f35bc
🐛 Fixed embedded cards for non-utf8 content (#15578)
closes: https://github.com/TryGhost/Ghost/issues/14973

- When fetching content using a non-standard charset, characters were notproperly decoded to utf-8 resulting in mangled text in the editor -> Detect charset and use iconv to decode the page text

- When requesting a non bookmark card, if no oembed data could be foundand we fallback to bookmark, a second network request to fetch the content was issued. This seemed unnecessary -> refactored to avoid that
2022-10-13 12:19:47 +01:00
Kritika Sharma
76e906d498
Added e2e tests for post.tag.detached webhoo (#15610)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.

Co-authored-by: Kritika Sharma <kritikasharma@Kritikas-MacBook-Pro-2.local>
2022-10-13 11:53:18 +01:00
Shubhadeep Das
e2124314ed
Added e2e tests for page.scheduled webhook (#15609)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-10-13 11:20:42 +01:00
Halldor Thorhallsson
74f7b7b3ee
Added e2e test for site.changed webhook event (#15595)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-10-13 11:05:31 +01:00
Rishabh Garg
e05889cd53
🐛 Fixed missing accent color for default content cta (#15611)
refs https://github.com/TryGhost/Casper/issues/901

- the site accent color property on default cta for upgrade link had a typo and was using wrong property
2022-10-13 13:42:21 +05:30
Naz
4db0be603f Added subscriptions table
refs https://github.com/TryGhost/Team/issues/2030

- adds `subscriptions` table to the DB schema
- this new table is aimed to support a native "subscription" primitive in Ghost
  that most resembles previously used `members_stripe_customers_subscriptions` table
2022-10-13 11:19:13 +07:00
Daniel Lockyer
74e1d52d6a Fixed incorrect maximum length definition in schema
refs https://github.com/TryGhost/Toolbox/issues/441

- whilst reviewing another PR, I noticed we were incorrectly using
  `maxLength` instead of `maxlength` in the schema column definition
- it turns out we've already been doing this wrong for a while with
  other columns
- this key is not acted upon, so the maximum column length was not applied
- fixing up the DB to the correct maximum length is something to fix in the
  future but right now, the schema does not reflect the size of the
  column that actually got created
- the fallback when `maxlength` is not provided is currently 191 [0], so
  this commit switches the schema and migrations to using the correct
  key name and column length that they are using when applied

[0]: 24670aa555/ghost/core/core/server/data/schema/commands.js (L27)
2022-10-13 09:58:19 +07:00
Daniel Lockyer
143ae857c9 Removed bool type from schema
refs https://github.com/TryGhost/Toolbox/issues/441

- we tend to have a mix of `bool` and `boolean` in the schema and
  migrations, which has become a real nit for me at this point
- we don't do any special handling between `bool` and `boolean`, it's
  just something we pass to Knex
- `bool` is an alias for `boolean` but `boolean` is actually documented - https://knexjs.org/guide/schema-builder.html#boolean
- this commit switches Ghost to only using `boolean` in the schema and
  migrations, and removes `bool` from the allowlist in tests to prevent
  us from adding it again in the future
- this should make absolutely no difference to the DB because both
  resulted in the same column
2022-10-13 09:37:38 +07:00
Daniel Lockyer
24670aa555
Fixed validating numbers as booleans in schema validator
refs https://github.com/TryGhost/Toolbox/issues/441

- I'm currently working on cleaning up our uses of `bool` and `boolean`
  in favor of `boolean`, and I've noticed we only handle converting
  numbers into booleans when the type is `bool`, so validation would
  otherwise fail
- given these can be used interchangeably, we should also support
  converting the numbers into booleans when the type is `boolean`
- this is going to get cleaned up again when I remove `bool` but this
  fixes the validation bug for now
2022-10-13 08:20:31 +07:00
Kritika Sharma
2cb4282fa8
Added e2e tests for post.tag.attached webhook (#15576)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.

Co-authored-by: Kritika Sharma <kritikasharma@Kritikas-MacBook-Pro-2.local>
2022-10-12 21:53:02 +01:00
jennyxchang
0bc31ded1b
Added e2e tests for post.scheduled event (#15561)
refs https://github.com/TryGhost/Ghost/issues/15537

- Added e2e test for post.scheduled event to increase webhook test coverage
2022-10-12 21:18:08 +01:00
Shubhadeep Das
8374c73e52
Added e2e tests for tag.edited webhook (#15555)
refs: https://github.com/TryGhost/Ghost/issues/15537

- this adds an e2e test and test snapshot for the `tag.edited` webhook so we can prevent regressions and bugs in the future

Co-authored-by: Hannah Wolfe <github.erisds@gmail.com>
2022-10-12 20:00:04 +01:00
Hannah Wolfe
168fa64395
Updated webhook snapshot for member.deleted
refs: 3ae1e48917
refs: 1221ba5d1d

- We added a new field between the PR being raised and merged, so the snapshot went out-of-date
2022-10-12 16:04:57 +01:00
illiteratewriter
3ae1e48917
Added e2e tests for member.deleted webhook (#15570)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-10-12 14:35:24 +01:00
Kritika Sharma
2ce0727794
Added e2e tests for post.deleted webhook (#15572)
refs: https://github.com/TryGhost/Ghost/issues/15537

- Added  missing e2e tests for post.deleted webhook to increase coverage 

Co-authored-by: Kritika Sharma <kritikasharma@Kritikas-MacBook-Pro-2.local>
2022-10-12 14:32:01 +01:00
jbenezech
3d44e37cbd
🐛 Fixed sitemaps with no content (#15571)
closes: https://github.com/TryGhost/Ghost/issues/14981

- Taxonomy-specific sitemaps were invalid xml when there was no data
- These invalid empty sitemaps were referenced in the index sitemap causing SEO tools to report errors
2022-10-12 14:11:19 +01:00
jbenezech
b9dd9f066d
🐛 Fixed settings overriden when updated from multiple tabs (#15536)
closes: https://github.com/TryGhost/Ghost/issues/15470

- When multiple browser tabs are open, each manipulate a different copy of ember data model, changes to the model in one tab are not reflected in the model of the other tab.
- When updating some settings, all current settings were sent to the API.
- As a result, when updating two different categories of settings (navigation/code inspection) in different tabs, the second update was overriding the first one.
- From a user perspective, this is not a natural behaviour. Only settings visible on-screen when clicking save should be modified.

Co-authored-by: Kevin Ansfield <kevin@lookingsideways.co.uk>
2022-10-12 14:03:54 +01:00
Daniel Lockyer
845f8d965e
Added test to validate schema structure
refs https://github.com/TryGhost/Toolbox/issues/441

- this is only v1 of the test I would like but it validates the keys on
  a column definition are part of an allowlist
- this has already uncovered a bug with `maxLength` (vs `maxlength`)
2022-10-12 19:19:33 +07:00
Daniel Lockyer
e2ba19b0df
Fixed import of bson-objectid in accordance to the typings
- there's a few different ways we can import it but I've chosen to
  append `.default` as we have done in several other places in the code
2022-10-12 14:54:35 +07:00
Simon Backx
68bdc1afea
Added post sentiment (#15592)
fixes https://github.com/TryGhost/Team/issues/2054

This change adds the sentiment and positive_feedback counts to the posts models. This change isn't really ideal because there are some problems here:
- sentiment isn't really a count
- we don't need to include the sentiment and positive_feedback as a default for posts (but the same is true for attribution)

It would make sense to move this to separate endpoints that only fetch the analytics for a given post when the analytics page is opened. But for our initial skateboard version of audience feedback this should be a good start to already see the data.
2022-10-11 17:52:14 +02:00
Simon Backx
e540344ef2
Added audience feedback service and storage (#15584)
fixes https://github.com/TryGhost/Team/issues/2049
fixes https://github.com/TryGhost/Team/issues/2053

- This adds a new audience feedback package to Ghost. 
- A new members API to give feedback on posts using the `/api/feedback` endpoint.
- Added a new authentication middleware that supports both uuid-based and session based authentication.
2022-10-11 16:32:28 +02:00
Elena Baidakova
1221ba5d1d
Added feedback_enabled to newsletters table (#15589)
closes TryGhost/Team#2042
- Added ability to enable audience feedback per newsletter (just on BE side).
2022-10-11 16:06:26 +04:00
Simon Backx
74d749fa63
Added members_feedback table (#15581)
fixes https://github.com/TryGhost/Team/issues/2041
2022-10-11 13:21:31 +02:00
Naz
714e108d40
Fixed typo 2022-10-11 17:24:11 +08:00
Naz
9b34bd70a2
Added test coverage for Subscription edit API
refs https://github.com/TryGhost/Team/issues/2047

- We anticipate upcoming changes in the PUT /members/:id/subscriptions/:subscription_id endpoint , so covered it with a snapshot test to track the differences more precisely.
- Note, the test case contains a more explicit outgoing HTTP request mocking.
2022-10-11 17:24:00 +08:00
John Grisham
c41f431fc7
Added e2e tests for page.added webhook (#15548)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2022-10-10 19:56:53 +01:00
Fabien 'egg' O'Carroll
dc8617a1e6
Added full flow test for click tracking (#15546)
refs https://github.com/TryGhost/Team/issues/1967

This tests the full flow of publishing a newsletter, and then checking
that clicked links will increase the click count, generate events for
the member which clicked the link as well as the redirects contain the
correct query params.
2022-10-10 10:15:31 -05:00
Simon Backx
7e3b41f643
Removed emailClicks feature flag (#15556)
fixes https://github.com/TryGhost/Team/issues/2028

Since link clicks became GA, some older components and templates are no longer used.
2022-10-07 14:27:57 +02:00
illiteratewriter
a0ec94fbfe
Added e2e test for member.added webhook (#15554)
refs https://github.com/TryGhost/Ghost/issues/15537

- this adds an e2e test and test snapshot for the `member.added` webhook so we can prevent regressions and bugs in the future
2022-10-07 15:54:24 +07:00
Shubhadeep Das
426168f73d
Added e2e test for tag.deleted webhook (#15553)
refs https://github.com/TryGhost/Ghost/issues/15537

- this adds an e2e test and test snapshot for the `tag.deleted` webhook so we can prevent regressions and bugs in the future
2022-10-07 15:50:16 +07:00
Naz
06f6fc11a7
Cleaned up tags e2e webhook test
refs https://github.com/TryGhost/Ghost/issues/15537
refs 4110ffaa2c

- The test had minor formatting issues not worth an extra back-forth during the PR review
2022-10-07 10:20:41 +08:00
Shubhadeep Das
4110ffaa2c Added e2e tests for tag.added webhook (#15537) 2022-10-07 09:46:21 +08:00
Naz
1880c7c1ec
Updated webhook post.published test
refs https://github.com/TryGhost/Toolbox/issues/320

- Added more complex mobiledoc structure in the post.published test to check for correct transformation of special purpose `__GHOST_URL__`. The snapshot has a correct URL transformation, which gives confidence it works properly
2022-10-06 17:51:14 +08:00
Simon Backx
d4540012dc Added tests for click events in the activity feed
fixes https://github.com/TryGhost/Team/issues/2018

- Includes new test fixtures for redirects and click events
- Tests if post, and links are returned in the click events
2022-10-06 11:43:39 +02:00
Naz
78c97d10a6
Improved post's webhook test annotations
refs https://github.com/TryGhost/Toolbox/issues/320

- There noe "roles" attached to the post's author when the 'post.added' event is fired. Webhooks function based of the model events and differ slightly with it's output comparing to the API response. For example, in case of Posts API, there'a an additional 'findOne' call (ref.: https://github.com/TryGhost/Ghost/blob/main/ghost/core/core/server/models/post.js#L1224-L1227) before returning the post to the endpoint handler and then passing that to the output serializer.
- If we want to have 1:1 copy of webhooks outputs and API outputs, we should rethink how we rely on model event data which is never the same as API controller level data.
2022-10-06 10:50:02 +08:00
Naz
fe1d0e44b4
Moved Ghost agent matcher to common framework
refs a499f866f3
refs d817e5830d

- The user-agent used in outgoing Ghost requests (webhooks mostly) is dependent on the Ghost version - snapshots break if the matcher is not dynamic.
- There will be a few more webhooks tests coming soon, so makes sense to have this matcher moved to a common "framework matchers"
2022-10-06 08:56:10 +08:00
Simon Backx
a499f866f3 Prevented posts webhook tests from breaking on every release 2022-10-05 14:25:00 +02:00
Simon Backx
f17934a5d2 Updated snapshots for latest release 2022-10-05 14:18:29 +02:00
Daniel Lockyer
c4981a71a2
Merged v5.17.2 into main
v5.17.2
2022-10-05 18:33:12 +07:00
Simon Backx
8900db8614
Fixed snapshots for Portal update
refs e86e78fb6b
2022-10-05 18:11:06 +07:00
Simon Backx
41a0945592
🐛 Prevented member creation when logging in (#15526)
fixes https://github.com/TryGhost/Ghost/issues/14508

This change requires the frontend to send an explicit `emailType` when sending a magic link. We default to `subscribe` (`signin` for invite only sites) for now to remain compatible with the existing behaviour.

**Problem:**
When a member tries to login and that member doesn't exist, we created a new member in the past.

- This caused the creation of duplicate accounts when members were guessing the email address they used.
- This caused the creation of new accounts when using an old impersonation token, login link or email change link that was sent before member deletion.

**Fixed:**
- Trying to login with an email address that doesn't exist will throw an error now.
- Added new and separate rate limiting to login (to prevent user enumeration). This rate limiting has a higher default limit of 8. I think it needs a higher default limit (because it is rate limited on every call instead of per email address. And it should be configurable independent from administrator rate limiting. It also needs a lower lifetime value because it is never reset.
- Updated error responses in the `sendMagicLink` endpoint to use the default error encoding middleware.
- The type (`signin`, `signup`, `updateEmail` or `subscribe`) is now stored in the magic link. This is used to prevent signups with a sign in token.

**Notes:**
- Between tests, we truncate the database, but this is not enough for the rate limits to be truly reset. I had to add a method to the spam prevention service to reset all the instances between tests. Not resetting them caused random failures because every login in every test was hitting those spam prevention middlewares and somehow left a trace of that in those instances (even when the brute table is reset). Maybe those instances were doing some in memory caching.
2022-10-05 18:11:06 +07:00
Simon Backx
b96ff6ae4a Fixed snapshots for Portal update
refs e86e78fb6b
2022-10-05 12:52:50 +02:00
Simon Backx
e7378520a0
🔒 Prevented member creation when logging in (#15526)
fixes https://github.com/TryGhost/Ghost/issues/14508

This change requires the frontend to send an explicit `emailType` when sending a magic link. We default to `subscribe` (`signin` for invite only sites) for now to remain compatible with the existing behaviour.

**Problem:**
When a member tries to login and that member doesn't exist, we created a new member in the past.

- This caused the creation of duplicate accounts when members were guessing the email address they used.
- This caused the creation of new accounts when using an old impersonation token, login link or email change link that was sent before member deletion.

**Fixed:**
- Trying to login with an email address that doesn't exist will throw an error now.
- Added new and separate rate limiting to login (to prevent user enumeration). This rate limiting has a higher default limit of 8. I think it needs a higher default limit (because it is rate limited on every call instead of per email address. And it should be configurable independent from administrator rate limiting. It also needs a lower lifetime value because it is never reset.
- Updated error responses in the `sendMagicLink` endpoint to use the default error encoding middleware.
- The type (`signin`, `signup`, `updateEmail` or `subscribe`) is now stored in the magic link. This is used to prevent signups with a sign in token.

**Notes:**
- Between tests, we truncate the database, but this is not enough for the rate limits to be truly reset. I had to add a method to the spam prevention service to reset all the instances between tests. Not resetting them caused random failures because every login in every test was hitting those spam prevention middlewares and somehow left a trace of that in those instances (even when the brute table is reset). Maybe those instances were doing some in memory caching.
2022-10-05 12:42:42 +02:00
Naz
0bf6268091
Updated content-length header matchers
no issue

- All content-length snapshots should be using the same matcher for consistency - anyContentLength. It's more explicit about what the matcher is all about and might be useful to have content-length matchers in one place if it ever changes (the header value should be a damn digit after all, not a string!) (ref. https://www.rfc-editor.org/rfc/rfc7230#section-3.3.2)
2022-10-05 17:34:17 +08:00
Fabien "egg" O'Carroll
28de1720c1 🔒 Fixed magic link endpoint sending multiple emails
refs https://github.com/TryGhost/Team/issues/2024

Without validation it was possible to send a string of comma separated
email addresses to the endpoint, and an email would be sent to each
address, bypassing any rate limiting.

This bug does not allow for an authentication bypass exploit. It is purely a
spam email concern.

Credit: Sandip Maity <maitysandip925@gmail.com>
2022-10-05 10:28:13 +01:00
Naz
320c6e0dd3
Abstracted a hacky local URL matcher
refs https://github.com/TryGhost/Toolbox/issues/320

- The URL matcher is very likely to be reused in the future, so having it abstracted away gives two benefits:
1. Central place to document hacky behavior and easier future cleanup
2. The implementer of the e2e test does not have to see the "hacky note" and just concentrate on the implementation of the test
2022-10-05 17:23:02 +08:00
Naz
d817e5830d
Added header snapshots to webhook e2e tests
refs https://github.com/TryGhost/Toolbox/issues/320

- Header snapshot matching was missing from webhook e2e tests. With a bumped version of webhook-mock-receiver it's now possible to record and match webhook request headers.
2022-10-05 17:23:02 +08:00
Fabien 'egg' O'Carroll
45d65663f4
Simplified link tracking related tables naming (#15480)
- Removes superfluous "link" from table names
- Fixes type definititon of dropTables util
- Updates & renames models
- Noop existing migrations to avoid unnecessary work
2022-09-29 22:08:45 +01:00
Rishabh Garg
e3600d70ef
Added referrer attribution from request context (#15499)
closes TryGhost/Team#2007

- uses request context to add referrer source and medium for a new member
- uses integration name as referrer medium if exists
2022-09-29 22:31:48 +05:30
Simon Backx
648811690a Added email click tracking
no issue

Bumped flag to GA.
2022-09-29 18:14:15 +02:00
Simon Backx
0cd0fc838d
Added email track clicks column and cleaned up frontend checks (#15501)
fixes https://github.com/TryGhost/Team/issues/2008

- New column that stores email click tracking at the time it was created
- Improved frontend side checks for when to show analytics
2022-09-29 16:42:45 +02:00
Naz
8cbf913582 Increased Vary granularity for versioned requests
refs https://github.com/TryGhost/Toolbox/issues/425
refs https://github.com/TryGhost/Toolbox/issues/280

- The versioned API responses vary based on requested version (passed in request's 'accept-version' header). shared caches that sit between Ghost's origin server and the browser would be putting responses with same Vary into the same caching bucket, which is incorrect.
- This change makes response's Vary more granular and tells caching mechanisms to take 'Accept-Version' request header into account when caching.
- Informative read on the topic - https://www.fastly.com/blog/getting-most-out-vary-fastly
2022-09-28 14:48:43 +08:00
Naz
f6c7df4018 Fixed overly long robots.txt caching for private sites
refs https://github.com/TryGhost/Toolbox/issues/411
refs 78ac63d8ad (diff-fb6792aa9cc3b71f5b146129fb1b58765206ecf4c5f367b72e0826b4b2934508R74)

- The value should have been in SECONDS not MILLISECONDS from the very beginning (see referenced commit)
2022-09-28 07:32:27 +08:00
Naz
a26689c264 Added backend JWKS endpoint caching
refs https://github.com/TryGhost/Toolbox/issues/411
refs f58b5984cb

- Backend JWKS endpoint did not have any HTTP caching
- It is recommended to cache JWKS endpoints (ref.: https://docs.apigee.com/api-platform/reference/policies/jwt-policies-overview)
- This change also brings the endpoint up to speed with what members JWKS
endpoint does
- Above point gives another reminder to extract JWKS functionality into a
common module that should be reused
2022-09-28 07:32:27 +08:00
Rishabh Garg
31733657a6
Updated naming for referrer attribution (#15486)
- renames `refSource`, `refMedium` and `refUrl` to `referrerSource`, `referrerMedium` and `referrerUrl` respectively for consistent naming across files and usages
2022-09-28 00:58:06 +05:30
Rishabh
d886bc4b0d Fixed tests for portal bump
refs 8848fd0f59

- last commit didn't update test snapshots for new portal version bump
2022-09-28 00:04:32 +05:30
Rishabh Garg
90034577b8
Added member attribution history frontend script (#15482)
- bumps member attribution script from alpha feature to now load for all sites. The script captures recent url history in localstorage to capture correct attribution for members.
- script is only loaded on the site if members is enabled
2022-09-27 23:45:41 +05:30
Simon Backx
f33e7a22fa
Added E2E test for whole link tracking flow (#15481)
refs https://github.com/TryGhost/Team/issues/1967

- Test is good to test if the whole flow works as expected, and works together
- We can test independent parts in separate tests that have better coverage of more edge cases
- Adds a basic helper to get an agent for the frontend (spent too much time on a better solution so I decided to keep the existing supertest agent)
2022-09-27 18:44:20 +02:00
Kevin Ansfield
89d4e3daf9
Updated error messages for invalid mobiledoc+lexical post/page API requests (#15477)
closes https://github.com/TryGhost/Team/issues/1896

- updated message to be clearer, added context and help
2022-09-27 10:30:28 +01:00
Daniel Lockyer
9818634b63
Merged v5.16.1 into main
v5.16.1
2022-09-27 11:21:48 +07:00
Rishabh
0349acb7e3 Updated content cta to use global accent color property
refs https://github.com/TryGhost/Ghost/pull/15471#discussion_r979902374

- the accent color value used by default content cta was copying the global site property which is redundant, and can be directly used
- originally, the accentColor property was extended to allow a fallback value for content ctas, but was later removed as we added default value to global site property directly
- the accentColor property is now deprecated and will be removed in next version, as existing themes might be relying on it for custom cta helpers
2022-09-27 09:41:54 +05:30
Rishabh
1410a4237e 🐛 Fixed default content CTA message to reflect page vs post
closes https://github.com/TryGhost/Team/issues/1898

- the default content cta always used the terminology as `post` when showing message that users don't have access to some content
- this caused confusion when users were looking at a page and message showed "This post is for subscribers only"
- updates the message to correctly reflect `page` vs `post` on the default cta
2022-09-27 09:41:54 +05:30
Simon Backx
281cd2e7a3
Renamed count.conversions to count.paid_conversions in posts (#15460)
fixes https://github.com/TryGhost/Team/issues/1943
2022-09-26 14:25:27 +02:00
Naz
4528cba1b9 Changed Content API caching to public
refs https://github.com/TryGhost/Toolbox/issues/410

- Private cache control was preventing browser or shared caches from storing Content APIs response. The type of data served through the Content API is very much of a "public" nature, so should be cacheable.
- Right now the 'max-age' value of 'cache-control' header is hardcoded to '0', without 'must-revalidate' value, to allow browsers to cache content slightly more aggressively. In the future the 'max-age' value will most-likely become configurable to allow even more aggressive HTTP caching.
2022-09-26 14:54:50 +08:00
Naz
7b009bf1fe Enabled shared caching of 404 error responses
refs https://github.com/TryGhost/Toolbox/issues/410

- The 'private' value in 'Cache-Control' response header for all errors made it impossible for shared caches (e.g.: Fastly, Cloudflare) to cache 404 responses efficiently.
- The change substitutes 'max-age=0' which should not effect the browser cache behavior but would allow shared caches to process such requests efficiently.
- A more loose caching logic only applies to 404 responses from GET requests that are not user-specific (non-authenticated, non-cookie containing requests)
2022-09-26 14:54:50 +08:00
Rishabh
6c85c75b86 Added referrer attribution data to member api
refs https://github.com/TryGhost/Team/issues/1961

- includes referrer source and medium information in member api
2022-09-24 17:46:57 +05:30
Naz
225a046bb8
Made Admin assets aggressively cacheable
closes https://github.com/TryGhost/Toolbox/issues/372

- The admin assets are served with a unique hash depending on the build with a year-long "max-age" value in the response cache-control header. The client browsers still do send 'If-None-Match' requests when there is a hard-refresh on the client side. There's no need for 'If-None-Match' requests though!
- With 'immutable' value in the cache-control header, the browser caches are treating responses as "hard-fresh" without sending redundant requests.
- For more about 'immutable' value read https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#immutable
2022-09-23 13:46:49 +08:00
Rishabh Garg
b048b02f67
Added new referrer source stats API (#15449)
closes https://github.com/TryGhost/Team/issues/1939

- adds new endpoint that returns count of referrer sources by date for admin dashboard
2022-09-22 16:34:26 +05:30
Simon Backx
7437d92d50
Added post referrers stats API (#15448)
closes https://github.com/TryGhost/Team/issues/1942

- Added data fixtures for referrers
- Added new endpoint to fetch referrer stats for a given post: `/stats/referrers/posts/:id`
- Added new ReferrersStatsService, responsible for calculating referrer stats
2022-09-21 18:16:56 +02:00
Rishabh Garg
b99c5428d0
Added referrer attribution columns to events table (#15436)
refs TryGhost/Team#1931

- referrer source, medium and url will be stored in the events table along with rest of attribution data
- stores referrer information on two tables
  - `members_created_events` for signups
  - `members_subscription_created_events` for paid conversions
2022-09-21 19:01:36 +05:30
Simon Backx
b8041f0a60
Added clicks to activity feed (#15439)
closes https://github.com/TryGhost/Team/issues/1933

- Added click_events to activity feed
- Added support for parsing click_events in the frontend
- Moved url parsing (transform ready) to model layer of LinkRedirect
- Moved `getEventTimeline` method to the top of the event repository
- Added description field to parsed events in the frontend (because we need a second line)
- Fixed: member email not returned in comment_event
2022-09-21 10:25:51 +02:00
Hannah Wolfe
f31a50270d Removed search labs flag
- the search helper was promoted to GA and we no longer need the feature flag
2022-09-20 15:22:06 +01:00
Simon Backx
63103c2251
Added click counts to posts admin API (#15435)
closes https://github.com/TryGhost/Team/issues/1928
2022-09-20 10:05:41 +02:00
Kevin Ansfield
b2b6be9cb5 Fixed content-length matcher in posts API test snapshot
refs d5f03ec0b1

- underlying error message varies across node versions so the content-length can't be fixed
- applied any-content-length matcher to the right test this time
2022-09-19 16:37:54 +01:00
Kevin Ansfield
1cc8176d4f Allowed any error message length in posts API error test snapshot
refs d5f03ec0b1

- underlying error message varies across node versions so the content-length can't be fixed
2022-09-19 16:22:22 +01:00
Kevin Ansfield
201d4ef228 Loosened error message snapshot matching for posts API test error output
refs d5f03ec0b1

- underlying error message varies across node versions
- adjusted to match only the part we explicitly set
2022-09-19 16:06:34 +01:00
Kevin Ansfield
744534fde6 Updated snapshot for posts API test error output
refs d5f03ec0b1
2022-09-19 15:56:30 +01:00
Daniel Lockyer
74f1f1bba6
Fixed CDN URLs in tests
refs de416629e6

- I forgot to update them when making the referenced commit
2022-09-19 14:51:31 +01:00
Daniel Lockyer
1ba6d2e5a7
Updated test snapshot
refs de416629e6

- this was forgotten after I'd changed the endpoint in the referenced
  commit
2022-09-19 14:43:07 +01:00
Kevin Ansfield
3bdfb68ff6
Added __GHOST_URL__ transform support to posts.lexical field (#15433)
no issue

- bumped `@tryghost/url-utils` to get access to the new lexical transform utilities
- updated the Post model's `parse()` and `formatOnWrite()` methods to transform the `lexical` field contents when reading/writing to ensure any links in content point at the correct place with `site.url` config changes
2022-09-19 11:30:38 +01:00
Simon Backx
901a28f3c6
Added link_redirects and members_link_click_events tables (#15421)
closes https://github.com/TryGhost/Team/issues/1912
closes https://github.com/TryGhost/Team/issues/1913
2022-09-19 11:20:36 +02:00
Rishabh
c765c3230e Updated attribution service to handle referrer information
refs TryGhost/Team#1907

- calculates final attribution source and medium using captured referrer information in history
- adds new referrer-translator that goes through available history and based to determine most valid referrer info
- includes referrer url, source and medium in the attribution data for storage
2022-09-19 12:29:22 +05:30
Aileen Nowak
be45d4ebcf Added post stats service to return total posts in Explore endpoint
no issue

- The explore endpoint needs to expose the total amount of published posts
- To be more consistent, this PR creates a PostStats class which is exposed as `stats` method within the PostService; just like it's done with the MemberService
- Moved existing method to return the date of the most recently published post into the stats service
- Updated the explore service test to reflect the new return property
2022-09-16 13:56:14 +01:00
Kevin Ansfield
3b21d26be7
Wired up creation of post_revisions entries when saving posts with lexical (#15422)
no issue

- added `PostRevsion` model
- duplicated `mobiledoc_revision` creation routine in Post model's onSaving hook to create `post_revision` when model's `lexical` field has changed
- updated `mobiledoc_revision` creation to skip when `lexical` field is populated
2022-09-16 11:59:35 +01:00
Kevin Ansfield
1581f439e9
Added post_revisions table (#15420)
no issue

- initially this will perform the same function as `mobiledoc_revisions` but storing `lexical` instead of `mobiledoc`
- naming is intentionally generic ready for later expansions
2022-09-16 10:19:05 +01:00
Simon Backx
a7b583050c
Added link tracking to paywall (#15414)
closes https://github.com/TryGhost/Team/issues/1908

### Problem:
- We need tracking on the paywall links in each email. (we cannot ignore them because those buttons are probably gonna have a higher paid conversion attribution than others).
- Currently we only add the paywall HTML to an email when processing each batch. So if we batch an email to 1.000 recipients per 100, we'll generate the paywall HTML 10 times. 
- We cannot replace links in `renderEmailForSegment` because that methods will get called multiple times. We don't want to have multiple redirect instances created for the same link in the same email.

###  Solution:
- Move the generation of the paywall to the `serialize` method of the post email serializer.
- Surround the generated paywall with HTML-comments so we can remove it if required in `renderEmailForSegment` depending on the member segment we are sending the email to.

---

### Before:

**Serialize output:**
```html
<html>
    <body>
        <h1>Generated email header</h1>
        <p>Generated text</p>

        <div>
            <!-- POST CONTENT START -->
                <h1>Post title</h1>
                <p>Content visible for all members</p>
            <!--members-only-->
                <p>Content visible for paid members only</p>
            <!-- POST CONTENT END -->
        </div>
    </body>
</html>
```

To be modified later by  `renderEmailForSegment`:

**Paid members (nothing changed):**
```html
<html>
    <body>
        <h1>Generated email header</h1>
        <p>Generated text</p>

        <div>
            <!-- POST CONTENT START -->
                <h1>Post title</h1>
                <p>Content visible for all members</p>
            <!--members-only-->
                <p>Content visible for paid members only</p>
            <!-- POST CONTENT END -->
        </div>
    </body>
</html>
```

**Free members (paywall _added_):**
```html
<html>
    <body>
        <h1>Generated email header</h1>
        <p>Generated text</p>

        <div>
            <!-- POST CONTENT START -->
                <h1>Post title</h1>
                <p>Content visible for all members</p>
                <h2>Generated paywall here</h2>
                <a href="https://subscribe.com">Subscribe to read the full post</a>
            <!-- POST CONTENT END -->
        </div>
    </body>
</html>
```

### After this change:

**Serialize output:**
```html
<html>
    <body>
        <h1>Generated email header</h1>
        <p>Generated text</p>

        <div>
            <!-- POST CONTENT START -->
                <h1>Post title</h1>
                <p>Content visible for all members</p>
            <!--members-only-->
                <p>Content visible for paid members only</p>
             <!-- PAYWALL -->
                <h2>Generated paywall here</h2>
                <a href="https://subscribe.com/?tracked">Subscribe to read the full post</a>
            <!-- POST CONTENT END -->
        </div>
    </body>
</html>
```

To be modified later by  `renderEmailForSegment`:

**Paid members (paywall removed):**
```html
<html>
    <body>
        <h1>Generated email header</h1>
        <p>Generated text</p>

        <div>
            <!-- POST CONTENT START -->
                <h1>Post title</h1>
                <p>Content visible for all members</p>
            <!--members-only-->
                <p>Content visible for paid members only</p>
            <!-- POST CONTENT END -->
        </div>
    </body>
</html>
```

**Free members (members-only content removed):**
```html
<html>
    <body>
        <h1>Generated email header</h1>
        <p>Generated text</p>

        <div>
            <!-- POST CONTENT START -->
                <h1>Post title</h1>
                <p>Content visible for all members</p>
            <!-- PAYWALL -->
                <h2>Generated paywall here</h2>
                <a href="https://subscribe.com/?tracked">Subscribe to read the full post</a>
            <!-- POST CONTENT END -->
        </div>
    </body>
</html>
```
2022-09-16 10:08:12 +02:00
Kevin Ansfield
c240f7afa4
Added rendering of posts.lexical to posts.html when saving (#15416)
no issue

- added `@tryghost/kg-lexical-html-renderer` dependency
- added `lexical` lib following the same pattern as our `mobiledoc` lib
- updated the Post model's `onSaving` hook to generate the `html` value from `lexical` when present
2022-09-15 16:49:14 +01:00
Simon Backx
e9974d8cc0
🐛 Fixed feature image caption escaped twice in newsletters (#15417)
fixes https://github.com/TryGhost/Team/issues/1909

- The feature image caption is already escaped on the frontend
- Doing it again in the backend breaks the possibility to add links to the caption
- I checked and the `feature_image_alt` is not escaped in the frontend.
2022-09-15 17:07:10 +02:00
Simon Backx
699e67f4e4
Added email_track_clicks setting (#15409)
fixes https://github.com/TryGhost/Team/issues/1900
refs https://github.com/TryGhost/Team/issues/1901

- Defaults to the same value as the current email_track_opens setting for existing installations, otherwise defaults to true
- Had to use a custom migration because the `addSetting` helper doesn't support using an existing setting as current value
- Added a minimal UI to change the setting, but this still needs some design magic 🪄
- Link replacement is disabled if `email_track_clicks` is disabled. In the future we might consider to still do parial additions, such as source attribution and maybe redirects (to discuss).
2022-09-15 15:48:22 +02:00
Simon Backx
972c25edc7
Wired up member attribution from email clicks (#15407)
refs https://github.com/TryGhost/Team/issues/1899

- Added `addEmailAttributionToUrl` method to MemberAttributionService. This adds both the source attribution (`rel=newsletter`) and member attribution (`?attribution_id=123&attribution_type=post`) to a URL.
- The URLHistory can now contain a new sort of items: `{type: 'post', id: 'post-id', time: 123}`.
- Updated frontend script to read `?attribution_id=123&attribution_type=post` from the URL and add it to the URLHistory + clear it from the URL.
- Wired up some external dependencies to LinkReplacementService and added some dummy code.
- Increased test coverage of attribution service
- Moved all logic that removes the subdirectory from a URL to the UrlTranslator instead of the AttributionBuilder
- The UrlTranslator now parses a URLHistoryItem to an object that can be used to build an Attribution instance
- Excluded sites with different domain from member id and attribution tracking
2022-09-14 15:50:54 -04:00
Fabien "egg" O'Carroll
4726742673 Renamed redirects to custom-redirects
We're going to be adding more redirection logic into Ghost and it's
going to get confusing if we have names this generic. This makes it
clear which feature this service is related to.

Ideally in the future we can combine all of these into one redirects
service, but for now we will be running a specific service per feature
2022-09-14 13:24:17 -04:00
Simon Backx
d5b332ab02 Added temporary fix for random test failures in comments
refs https://ghost.slack.com/archives/C02G9E68C/p1663162175224299

This requires a better fix in the future that properly awaits the emails (not really possible at the moment) or disables sending new member emails when using loginAs
2022-09-14 17:02:13 +02:00
Kevin Ansfield
a7c4991af5 Wired up lexical editor saving
no issue

- fixed API returning "Invalid mobiledoc structure" errors when `mobiledoc:null` is sent in the payload alongside `lexical: '{...}'`
- updated Admin's `posts` and `pages` adapters to always add `?formats=mobiledoc,lexical` because the API doesn't return `lexical` by default
- added `lexical` attribute to Admin's Post model
- updated `lexical-editor` controller and related components to work with `lexical` always being a JSON string rather than a parsed object
- updated `<KoenigLexicalEditor>` to pass through the lexical state string as initial state and wired up the `onChange` prop
2022-09-13 21:01:53 +01:00
Kevin Ansfield
6fc9cd5f80
Added passthrough + saving of lexical property on posts/pages (#15403)
no issue

- bumped `@tryghost/admin-api-schema` to allow passthrough of the `lexical` property on post and page API endpoints
- prevented saving of blank document in the `mobiledoc` field if `lexical` is provided
- prevented API input containing both `mobiledoc` and `lexical` fields to avoid issues when both are present:
  - not possible to know which content is latest/has precedence
  - not possible to know which editor should be displayed in Admin
2022-09-13 17:29:37 +01:00
Kevin Ansfield
c8dc23cbb5 Fixed Content API posts/pages e2e tests
refs 7ad1be2555

- snapshot comparisons were missing matchers for dynamic fields in the body response
2022-09-13 15:05:53 +01:00
Kevin Ansfield
30611cf2c4 Really fixed e2e Admin API posts test
refs 9471384020

- previously added tests (any subsequent matcher updates) for browse endpoint were not using matchers that sufficiently covered the dynamic portions of the body
2022-09-13 14:09:45 +01:00
Kevin Ansfield
eebdb1d5df Fixed e2e Admin API posts test
refs 9471384020

- previously added tests for browse endpoint were not using matchers that sufficiently covered the dynamic portions of the body
2022-09-13 14:05:03 +01:00
Kevin Ansfield
9471384020 Added tests for Admin API not returning lexical by default but including when requested
no issue

- left `mobiledoc` as the only default format added in the post/page input serializers for now to minimize API/test churn during these early stages of lexical development
- tested that the `lexical` field is not returned by default but can be requested via `?formats=lexical`
2022-09-13 13:30:29 +01:00
Kevin Ansfield
7ad1be2555 Fixed Content API returning lexical format when requested
no issue

- similar to the `mobiledoc` field, the Content API should not return the source `lexical` field if requested via `?formats=`
  - renamed `removeMobiledocFormat()` to `removeSourceFormats()` to better match it's behaviour
2022-09-13 13:30:29 +01:00
Daniel Lockyer
f8679f22d7
Updated settings snapshot
refs 067bfe92a4

- this was missed in the previous commit
2022-09-13 12:37:33 +01:00
Kevin Ansfield
8cdd2e10b7
Added posts.lexical database field (#15397)
closes https://github.com/TryGhost/Team/issues/1884

- adds `post.lexical` ready for use by the lexical-powered editor re-write
- fulfils the same purpose as `posts.mobiledoc` so uses the same field properties
- added `lexical` to allowed formats in Post model so it won't be included by default in API responses meaning tests/snapshots don't need updating at present
2022-09-13 11:21:47 +01:00
Rishabh
054833992e Wired events for triggering email alerts for subscription creation/cancellation
refs https://github.com/TryGhost/Team/issues/1865

- refactors subscription creation/cancellation to dispatch proper events which are used for email alerts
- cleanup
2022-09-10 11:06:34 +05:30
Rishabh
2fbaa7b9bc Moved member email alert trigger to member creation
closes https://github.com/TryGhost/Team/issues/1864
refs https://github.com/TryGhost/Team/issues/1881

- triggers free member email alert via event dispatch from member create method
- passes subscription/stripe data to member creation for paid members so free member alert can be ignored for them
- moves subscription created event being called from webhook controller to `linkSubscription`, allows creating subscription events for all new subscriptions instead of ones just via webhooks
2022-09-10 11:06:34 +05:30
Rishabh
4187f0da54 Updated staff service to trigger alerts via events
refs https://github.com/TryGhost/Team/issues/1865

- refactors staff service to listen to member and subscription events
- triggers email alerts based on events instead of directly calling the service
- removes staff service dependency for members api
2022-09-10 11:06:34 +05:30
Hannah Wolfe
04f3ac37d3 Removed defunct applyPublicRules code
closes: https://github.com/TryGhost/Toolbox/issues/369
refs: https://github.com/TryGhost/Toolbox/issues/229

- this code is defunct as we no longer accept a status parameter via the API for posts, pages, users or authors
2022-09-09 17:14:15 +01:00
Hannah Wolfe
b0234dd58e
Removed apiVersions from test urlUtils
refs: https://github.com/TryGhost/Toolbox/issues/229

- These properties were removed from urlUtils in 5.0 and no longer do anything
2022-09-09 13:32:30 +01:00
Ronald Langeveld
c9e6f42ca8 Bumped Portal to 2.12.0
ref https://github.com/TryGhost/Team/issues/1800

- Adds v 2.12.0 of Portal
- Updates snapshots for tests
2022-09-09 13:54:43 +02:00
Naz
235d716048
Refactored notifications e2e tests to use test framework
no issue

- Bumped into these tests when doing cleanup in the notifications service. Having full snapshot of requests is useful to have as a sanity check, so migrated this test suite quickly.
2022-09-09 19:51:50 +08:00
Ronald Langeveld
eb6534bd7f
Replaced all 'bio' references with 'expertise' for member comments. (#15359)
closes https://github.com/TryGhost/Team/issues/1772

- The user facing side of comments recently replaced `bio` with `expertise`.
- To remain consistent we replaced all the references of `bio` with `expertise` throughout the codebase.
- This includes a database column name changing migration, within the `members` table.
- Bumped up the comments-ui version to a new minor (0.10.x) as its a breaking change.
2022-09-09 10:14:49 +02:00
Daniel Lockyer
790e4c5598
Added history log for staff actions
fixes https://github.com/TryGhost/Toolbox/issues/356

- this feature allows site Administrators to view a history log of staff
  actions on their site so they can audit when and by whom that something happened
- this commit promotes the History log to GA
2022-09-08 18:23:39 +01:00
Hannah Wolfe
7084217d3d
Added same-origin referrer rule to post previews
- this prevents the referrer/referer header being sent for requests that go to external domains
- this in turn prevents preview URLs from appearing in the analytics of sites that are linked to and clicked on from previews
- otherwise, preview URLs can be leaked to the owners of the linked and clicked sites
2022-09-08 12:39:13 +01:00
rw4nn
dc84983550
🐛 Fixed square brackets being % encoded in URLs (#14977)
fixes: #14863
refs: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/encodeURI#encoding_for_ipv6

- added a simple Regex replace for the percent-encoded square brackets to get them back to non-encoded
- a preferred solution might be using new URL(), but that causes other issues. The regex solves the immediate need.
2022-09-08 12:09:40 +01:00
Simon Backx
6bffa893b1
Added snapshot tests to ghost_head helper (#15327)
refs https://github.com/TryGhost/Team/issues/1795

- Snapshots help us detect unexpected changes in the `<head>` of all sites (e.g., newly introduced script tags)
- Added ghost_head tests for comment count helper
2022-09-08 13:04:34 +02:00
Simon Backx
4534b693e4
Added test that validates output HTML of email template (#15365)
refs https://github.com/TryGhost/Team/issues/1871

This commit adds a test to the serialize method of `post-emaiserializer`. It checks whether the generated email HTML is valid and standard HTML5 and that all properties are escaped.

To do this validation, I depend on the new `html-validate` dev dependency. Just parsing the HTML with a HTML parser is not enough to guarantee that the HTML is okay.

Apart from that this fixes:
- Removed the sanitizeHTML method and replaced it with normal HTML escaping. We don't want to allow any HTML in the escaped fields. Whereas `sanitizeHTML` still allows valid HTML, but we don't want that and want the same behaviour as on the site. E.g., a post with a title `All your need to know about the <br /> tag` should actually render the same title and non-html content, being `All your need to know about the &lt;br /&gt; tag`
- The file, nft and audio card didn't (always) escape the injected HTML fields (new version @tryghost/kg-default-cards) 
- `@tryghost/string` is bumped because it contains the new escapeHtml method
2022-09-08 10:11:01 +02:00
Daniel Lockyer
04e3ee9f10 Added property cleaning to tag relations on pages + posts
refs https://github.com/TryGhost/Ghost/pull/15375

- we currently pass all properties for the `tags` property of a
  `page`/`post` body down further into Ghost, which is causing issues
  because it's handling properties it doesn't expect
- this is showing up because it's triggering save history events for
  tags when a post is edited
- this commit introduces a clean util which has an allowlist of
  properties allows on tag relations
- this list was taken from the schema: 128f8fb006/packages/admin-api-schema/lib/schemas/posts.json (L214-L227)
2022-09-07 22:28:56 +01:00
Daniel Lockyer
4a6f57b105
Merged v5.13.2 into main
v5.13.2
2022-09-06 16:45:52 +01:00
Fabien 'egg' O'Carroll
f7a58ecafc
🐛 Fixed OpenSea NFT OEmbeds (#15372)
refs https://github.com/TryGhost/Team/issues/1879

OpenSea updated their URL format for NFTs after adding support for Solana
which broke our regex, this updates to support the new format.
2022-09-06 11:29:35 -04:00
Simon Backx
8b4d5504e8
Moved (un)like endpoint code to comments service (#15371)
fixes https://github.com/TryGhost/Team/issues/1861

- Moved like and unlike endpoint handling to comments service and controller
- Moved small part of report logic to comments controller
- Added proper 401 authentication error when not authenticated as member
2022-09-06 17:20:55 +02:00
Ronald Langeveld
1f177e1c17
Added optional data-attribute to enable and disable auto redirection. (#15335)
closes https://github.com/TryGhost/Ghost/issues/15104 https://github.com/TryGhost/Team/issues/1800

- On custom sign up and login forms, creators often wouldn't want their members to be redirected to that page after signing in.
- This takes a new data-attribute value (eg `data-members-autoredirect="false"`) that can be set on [custom sign up / login forms](https://ghost.org/docs/themes/members/#signup-forms) into account before parsing the referrer on the magic link URL that gets sent to the member for login.
2022-09-06 14:36:06 +02:00
Naz
920a3aeb4c
Fixed adapter-related unit test
refs 37dd187fe6
refs c36575627d/ghost/core/core/server/data/importer/handlers/image.js (L16)

- The tests were failing because they were stubbing a "generic" adapter, instead of the one which the module under test was using (see referenced code to see what I mean)
2022-09-06 18:46:32 +08:00
Naz
c36575627d
Fixed unit test
refs 37dd187fe6

- The referenced commit lacked cleanup after module removal
2022-09-06 18:11:22 +08:00
Naz
a96a7340c0
Added JSDoc to adapter options resolver
refs https://github.com/TryGhost/Toolbox/issues/384

- Added jsdoc for intellisence/typechecking
- Cleaned up the naming of returned values to resemble the usecases a bit better
2022-09-06 17:51:57 +08:00
Naz
67df9a6105
Removed unused adapterType variable
refs https://github.com/TryGhost/Toolbox/issues/384

- The adapter manager can parse the adapter type internally from the "type:feature" syntax, so there's no need to pass it around.
2022-09-06 17:51:57 +08:00
Naz
1fc8c8d671
Added more explicit adapter config syntax
refs https://github.com/TryGhost/Toolbox/issues/384

- Existing adapter config was based on the notion there can only be one configuration per one adapter class. With adapter cache now allowing instantiating multiple adapter instances with the same base class it opened up a possibility to have shared configuration for a base class and then extend/override it in "feature" configurations (see tests in this commit for specific examples)
2022-09-06 17:51:57 +08:00
Naz
a0d0c38aaf
Fixed typo complementary -> complimentary 2022-09-06 17:51:56 +08:00
Hannah Wolfe
db6fb2d6d0
Removed unused fixture tasks
- these old concepts aren't used anymore
2022-09-06 10:18:55 +01:00
Daniel Lockyer
79368f565f
Fixed Tier events being created when Posts are edited
refs https://github.com/TryGhost/Team/issues/1875

- due to an misbehavior in our model layer, when `tiers` is set on a Post, it'll
  trigger a save of the Tier, and this produces an extra event in the
  `actions` table
- mapping the Tier(s) to just the ID prevents bookshelf-relations from
  editing the Tier and thus prevents the extra event
- also fixed tests which were implicitly assuming supplying a slug to a
  post would create the product
2022-09-05 17:19:27 +01:00
Hannah Wolfe
f1bc8026b7
Removed unused resetRoles method
- working on cleaning up our fixture mechanism and making it easier to understand
2022-09-05 09:32:54 +01:00
Simon Backx
2e85ae98be
🐛 Fixed sending emails from email domain that includes www subdomain (#15348)
fixes https://github.com/TryGhost/Team/issues/1855
fixes https://github.com/TryGhost/Team/issues/1866

This commit moves all duplicate methods to get the support email address to a single location. Also methods to get the default email domain are moved.

For the location, I initially wanted to put it at the settings service. But that service doesn't feel like the right place. Instead I created a new settings helpers service. This service takes the settingsCache, urlUtils and config and calculates some special 'calculated' settings based on those:

- Support email methods
- Stripe (active) keys / stripe connected (also removed some duplicate code that calculated the keys in a couple of places)
- All the calculated settings are moved to the settings helpers

I'm not 100% confident in whether this is the right place to put the helpers. Suggestions are welcome.
2022-09-02 16:57:59 +02:00
Simon Backx
51ddc39fa7 Updated snapshots of email preview tests
refs dd2bfb8c0e
2022-09-02 16:19:28 +02:00
Daniel Lockyer
dd2bfb8c0e
Merged v5.12.4 into main
v5.12.4
2022-09-02 15:13:37 +01:00
Simon Backx
999b111fce
🐛 Fixed paid email preview stopped working in emails (#15356)
fixes https://github.com/TryGhost/Team/issues/1870

Disables email sanitization that was enabled earlier because this bug is more important and urgent.

The recently introduced email sanitzation removes HTML comments from the post html.
- This breaks the email paid preview, because it depends on the `<!--members-only-->` comment.
- Breaks the Outlook comments `<!--[if !mso !vml]-->`

This commit reverts this change.
2022-09-02 15:49:39 +02:00
Hannah Wolfe
409a4783a3
Renamed content api agent auth method
- the query param is called key, so key is easier to remember
2022-09-02 10:48:03 +01:00
Hannah Wolfe
642b6ff8ae
Added loginAs[Role] to e2e framework with example
closes: https://github.com/TryGhost/Toolbox/issues/342
refs: 032a26f9f3
refs: 588c9d04e8

- Now that the old `users:no-owner` (now named 'users') is working correctly :)
- Was able to add loginAs[Role] methods for each staff role, so that it's possible to execute tests as that user and check permissions
- Refactored the email preview tests to use the new e2e framework and these methods, as an example
2022-09-02 10:38:22 +01:00
Hannah Wolfe
588c9d04e8
Renamed users:no-owner to users as main user fixture
- This fixture is the main user fixture you'd want to use when testing staff roles
- At the moment it has a weird name that makes it less likely people will use it
- A tiny step in trying to make our fixture system make a tiny bit more sense
2022-09-02 10:08:37 +01:00
Hannah Wolfe
032a26f9f3
Fixed users:no-owner fixture to add roles correctly
- This fixture would only work if the roles were inserted by the fixture system
- In most cases, this fixture was adding users without their associated roles
- Now we assume the roles exist already, and that we need to map users to each role
- This will allow us to more easily test user roles in e2e tests
2022-09-02 08:26:30 +01:00
Fabien 'egg' O'Carroll
e4cbb3d24d
Reset magic link rate limiting upon successful login (#15345)
refs https://github.com/TryGhost/Team/issues/1771

We don't have access to `req.brute.reset` due to the way the flow
works, we have one endpoint which sends an email with a magic link,
and another route which handles the login. We don't want to apply
brute force protection to both because our rate limiting is designed
for API requests not web page visits (which is how login is handled).

Because of this we require access to the underlying ExpressBrute
instance exposed by the spam-protection module, so that we can
perform the reset.
2022-09-01 08:54:14 -04:00
Fabien 'egg' O'Carroll
c9f782a3fc
🔒 Fixed rate limiting for user login (#15336)
refs https://github.com/TryGhost/Team/issues/1074

Rather than relying on the global block to stop malicious actors from
enumerating email addresses to determine who is and isn't a user, we
want our user login brute force protection to be on an IP basis,
rather than tied to the username.
2022-09-01 13:29:59 +01:00
Naz
945ebd4806
Fixed ERR_NOCK_NO_MATCH warning during test runs
refs https://github.com/TryGhost/Toolbox/issues/389

- The e2e test suite log was full of ERR_NOCK_NO_MATCH warnings when the logging level was set to "warn". The cause of this warning was legit duplicated webhook trigger processing on test environment. Gah!
- The source of duplicate webhook processing was duplication of event handlers. Event handlers were registered multiple times for same event because of the singleton nature of the "common/events" module - it remains the same instance and is not cleaned up between reboots. The deeper issue of events module initialization should be solved separately, this slightly hacky approach fixes the problem now and highlights it to be tackled in the future.
2022-09-01 12:25:47 +08:00
Naz
88e0ae892c
Fixed typo 2022-09-01 11:01:15 +08:00
Fabien 'egg' O'Carroll
2ff81cc5d3
🔒 Fixed rate limiting for user login (#15336)
refs https://github.com/TryGhost/Team/issues/1074

Rather than relying on the global block to stop malicious actors from
enumerating email addresses to determine who is and isn't a user, we
want our user login brute force protection to be on an IP basis,
rather than tied to the username.
2022-08-31 10:33:42 -04:00
Daniel Lockyer
c2b399fc2c Fixed warning about aborted connection in tests
refs https://github.com/TryGhost/Toolbox/issues/389

- if we enable warning logs in E2E tests, we get a bunch of error
  messages saying `ERROR Unhandled rejection: aborted` coming from the
  SQLite DB reset code
- specifically, it's coming from the line that resets the DB by copying
  the file
- this line was initially added because we would see random SQLite
  "malformed database" errors
- I have a feeling that was due to something else, but I can't be sure
- I'm also not sure how else we should shut the DB connection, as this
  is the recommended way but it throws an unhandled rejection
- this commit is a bit of a gamble because I'm not actually sure what
  was causing the problem, but it gets rid of the errors locally and
  doesn't regress on the random failures
2022-08-31 12:15:30 +01:00
Simon Backx
8cd2b3182a
🐛 Fixed commenting on tier-only posts (#15333)
fixes https://github.com/TryGhost/Team/issues/1860

**Problem:**
Members were not able to comment on a post that was only visible for members with a specific tier.

**Causes:**
Content gating was done on models with missing relations.
- The products relation was not loaded on the member when doing content gating
- The tiers relation was not loaded on the post when doing content gating

**Tests:**
- Added for tier-only posts
- Added for paid-only commenting
2022-08-30 16:48:47 +01:00
Simon Backx
aec2badc6c
🐛 Fixed removing comped subscriptions for members with active subs (#15332)
fixes https://github.com/TryGhost/Team/issues/1859

**Problem:**
When for some reason a member has an active subscription (or legacy comped subscription) for product A, and a comped subscription for product B. You cannot remove comped subscription B.

**Fixed by:**
Updating the API to allow more flexible product changes on members.
- Allow the removal of (comped) products on a member, as long as that product doesn't have a related subscription
- (still) allow the addition of comped products to a member, as long as that member doesn't have other active subscriptions. This matches the existing behaviour, but now this is only checked for added products.
- Includes tests for these edge cases
2022-08-30 16:48:44 +01:00
Simon Backx
4282ead3a7
🐛 Fixed commenting on tier-only posts (#15333)
fixes https://github.com/TryGhost/Team/issues/1860

**Problem:**
Members were not able to comment on a post that was only visible for members with a specific tier.

**Causes:**
Content gating was done on models with missing relations.
- The products relation was not loaded on the member when doing content gating
- The tiers relation was not loaded on the post when doing content gating

**Tests:**
- Added for tier-only posts
- Added for paid-only commenting
2022-08-30 17:38:58 +02:00
Simon Backx
e7786ca482
🐛 Fixed removing comped subscriptions for members with active subs (#15332)
fixes https://github.com/TryGhost/Team/issues/1859

**Problem:**
When for some reason a member has an active subscription (or legacy comped subscription) for product A, and a comped subscription for product B. You cannot remove comped subscription B.

**Fixed by:**
Updating the API to allow more flexible product changes on members.
- Allow the removal of (comped) products on a member, as long as that product doesn't have a related subscription
- (still) allow the addition of comped products to a member, as long as that member doesn't have other active subscriptions. This matches the existing behaviour, but now this is only checked for added products.
- Includes tests for these edge cases
2022-08-30 17:36:52 +02:00
Fabien 'egg' O'Carroll
21e473ff78
🐛 Fixed newsletters not rendering with non-HTML safe chars (#15331)
Co-authored-by: Kevin Ansfield <kevin@lookingsideways.co.uk>
2022-08-30 11:26:01 +01:00
Simon Backx
f2da1229d8
Removed unused support email verificaton endpoints (#15328)
fixes https://github.com/TryGhost/Team/issues/1679

These endpoints are safe to be removed, as they are only used by the admin app and usage has been removed over there. It is very unlikely that this endpoint has been used in a third party integration (in which case they will get a notification email).
2022-08-29 15:16:13 +02:00
Elijah
3c94812ee5
Added missing return in create-stripe-update-session
no issue

- Return was missing for `res.end` if an invalid subscription_id was passed
- Added explicit `text/plain` `Content-Type` headers to error messages to avoid MIME sniffing

Signed-off-by: Elijah Conners <business@elijahpepe.com>
Co-authored-by: Simon Backx <simon@ghost.org>
2022-08-29 14:02:58 +02:00
Rishabh Garg
594ef34871
Enabled member email alerts (#15321)
closes https://github.com/TryGhost/Team/issues/1825
closes https://github.com/TryGhost/Team/issues/1826

- allows site owners/admins to receive email notifications when somebody signs up, becomes paid, or cancels subscription
- owners/admins can set their email preference from staff settings
2022-08-26 09:38:12 +05:30
Fabien "egg" O'Carroll
4718171b1d Removed out of date history items from UrlHistory
In case there is an issue with the filtering of items in our client
side attribution script, we also check for and remove out of date
items here. This ensures that we do not erroneously attribute signups
or conversions to webpages from more than 24h ago.
2022-08-25 16:09:34 -04:00
Simon Backx
67163209e1
Enabled member attribution flag in all tests (#15317)
closes https://github.com/TryGhost/Team/issues/1852

Updates all tests to run with memberAttribution flag enabled
2022-08-25 15:25:01 -04:00
Rishabh
216eeb9d71 Added paid subscription start email alert
refs TryGhost/Team#1826

- triggers paid subscription start email via staff service
2022-08-25 19:53:02 +05:30
Rishabh
aeadf8a5e1 Added email alert trigger for free member signup
refs TryGhost/Team#1826

- fires email alert on free member creation after they finish signing up via checkout link
2022-08-25 19:53:02 +05:30
Simon Backx
a8239bfa97
Added ENUM validation for member/subscription created events (#15312)
closes https://github.com/TryGhost/Team/issues/1842

- members_created_events: source + attribution_type
- members_subscription_created_events: attribution_type
- members_subscribe_events: source
2022-08-25 15:39:37 +02:00
Simon Backx
232882daa2 Mapped '/' attribution url to homepage
fixes https://github.com/TryGhost/Team/issues/1846

- Shows homepage instead of / for attribution values
2022-08-25 14:51:38 +02:00
Rishabh
effd5af615 Handled fetching staff users for email alerts
refs TryGhost/Team#1826

- adds a method on user model which fetches all eligible users for a type of email alert
- restricts users to active `Owner` and `Administrators` with setting turned on
2022-08-25 18:01:52 +05:30
Hannah Wolfe
c9864ee63f Added {{search}} theme helper
closes: https://github.com/TryGhost/Team/issues/1732

- adds a theme helper which outputs a working search button with a standard icon
- the icon adopts whatever the current color is from css, and has a set of default styles
- styles can be overridden with !important or the data attribute
- alternatively, any element in a theme may be turned into a search button by adding data-ghost-search
- this is meant to be a simple tool for non-theme-developers to easily add a search icon to their themes in a way that doesn't require css or html knowledge
2022-08-24 21:34:20 +01:00
Hannah Wolfe
96f7b8fdc8 Fixed content-length, again
refs: 203c8036fa
refs: 1fadbacdec
refs: 22fd7f289c

- There is something seriously weird about how content-length changes...
- It's different on CI to local sometimes...
- This particular test should not change IMO
2022-08-24 20:38:35 +01:00
Hannah Wolfe
203c8036fa
Fixed more content-length labs issues
refs: 1fadbacdec
refs: https://github.com/TryGhost/Ghost/commit/x5447985ee2e8a4b497e9c1afbad07

- I'm trying to make it so that changing labs flags doesn't require changes to these snapshot files!
2022-08-24 19:45:20 +01:00
Simon Backx
f124d142c9 Added member attributions to activity feed (#15283)
refs https://github.com/TryGhost/Team/issues/1833
refs https://github.com/TryGhost/Team/issues/1834

We've added the attribution property to subscription and signup events when the
flag is enabled. The attributions resource is fetched by creating multiple relations
on the model, rather than polymorphic as we ran into issues with that as they can't
be nullable/optional.

The parse-member-event structure has been updated to make it easier to work with,
specifically `getObject` is only used when the event is clickable, and there is now a 
join property which makes it easier to join the action and the object.
2022-08-24 11:17:28 -04:00
Fabien "egg" O'Carroll
3c431bd8da Revert "Added member attributions to activity feed (#15283)"
This reverts commit e986b78458.

The tests were not passing for the PR and it was erroneously
merged into main
2022-08-24 11:01:47 -04:00
Emmanuel Gatwech
d9f0db6a22
Replaced Promise.join() with .all() in user model (#14972)
refs: https://github.com/TryGhost/Ghost/issues/14882

- Usage of bluebird is deprecated in favour of using native promises
2022-08-24 15:32:44 +01:00
Simon Backx
e986b78458
Added member attributions to activity feed (#15283)
refs https://github.com/TryGhost/Team/issues/1833
refs https://github.com/TryGhost/Team/issues/1834

We've added the attribution property to subscription and signup events when the
flag is enabled. The attributions resource is fetched by creating multiple relations
on the model, rather than polymorphic as we ran into issues with that as they can't
be nullable/optional.

The parse-member-event structure has been updated to make it easier to work with,
specifically `getObject` is only used when the event is clickable, and there is now a 
join property which makes it easier to join the action and the object.
2022-08-24 10:11:25 -04:00
Rishabh
326bb97d2c Enabled free trials via tiers and offers
refs https://github.com/TryGhost/Team/issues/1724
refs https://github.com/TryGhost/Team/issues/1726

- allows site owners to add a default free trial period to their tiers
- allows site owners to create custom offers that allow free trials on tiers
2022-08-24 19:24:31 +05:30
Rishabh Garg
9abfae2ddb
Switched off email alerts for subscription cancellation by default (#15304)
refs TryGhost/Team#1825

- all sites are expected to have cancellation alerts off by default
2022-08-24 18:54:00 +05:30
Hannah Wolfe
af94855349 Removed bluebird catch predicates from API endpoints
refs: https://github.com/TryGhost/Ghost/issues/14882

- I found a common pattern where catch predicates were being used to catch non-existent models in destroy methods, and sometimes elsewhere in the API endpoints
- The use of predicates is deprecated, and we're working to remove them from everywhere, so that we can remove bluebird
- In order to still handle these errors correctly, we needed a small change to mw-error-handler so that it can detect EmptyResponse errors from bookshelf, as well as 404s
Note: there is a small change as a result of this - the context on these errors now says "Resource not found" instead of "{ModelName} not found".
- I think this is acceptable for now, as we will be reviewing these errors in more depth later. It's quite easy to make changes, we just have to decide what with proper design input
2022-08-24 11:27:09 +01:00
Rishabh Garg
c48c65cc88
Added member email alert notification columns for staff (#15276)
refs https://github.com/TryGhost/Team/issues/1825

- adds 3 new columns to users table for storing email alert preferences for member signups/cancellation
- adds column for new member signup alert
- adds column for paid subscription started alert
- adds column for paid subscription canceled alert
- Updated default fixtures and tests for new columns
2022-08-23 22:11:38 +05:30
Daniel Lockyer
7f0996d986
Implemented resource linking in Audit Log
refs https://github.com/TryGhost/Toolbox/issues/356

- we have a very crude version of this before but it just wasn't
  maintainable
- one of the first things I did here was to add `include=resource` on
  the API call, so it returns the fields we need without extra API
  requests
- after we have the id/slug, I could build a route and model array
  dynamically, or return null if we can't redirect to the object (it
  doesn't exist)
2022-08-23 17:48:11 +02:00
Fabien "egg" O'Carroll
73466c1c40 Added ability to filter members on conversion attribution
refs https://github.com/TryGhost/Team/issues/1830
2022-08-23 11:36:56 -04:00
Fabien "egg" O'Carroll
83f2bf4757 Added ability to filter members on signup attribution
refs https://github.com/TryGhost/Team/issues/1831
2022-08-23 11:36:56 -04:00
Simon Backx
d91d6c1311
Added attribution counts to Pages API (#15289)
fixes https://github.com/TryGhost/Team/issues/1835

This was missed in the initial pass when adding to the Posts API
2022-08-23 10:26:34 -04:00
Hannah Wolfe
4cd210c29c Added post deletion tests using new e2e framework
- copied over and rewrote the deletion test from the legacy file
- added a new test that checks that we get a 404 when attempting to delete an unknown post
- this is a guard to protect and futureproof the API whilst we do refactoring to improve 404 handling from bookshelf
- in turn this is aimed at helping to get rid of a bunch of catch predicates from the API
2022-08-23 14:49:29 +01:00
Hannah Wolfe
a2a71c8e67 Renamed e2e admin post tests to legacy
- I want to start rewriting the post tests using the new e2e framework, but it's quite a big task
- For now I have renamed the existing file, and will use the correct file name for writing modern versions of tests
- Note: I have a specific test that I'd like to add which is far easier to write in the new framework
- This change should facilitate moving forward more with the new framework
2022-08-23 14:49:29 +01:00
Daniel Lockyer
9effa119c6 Implemented context on Actions events
refs https://github.com/TryGhost/Toolbox/issues/356

- in order to show data that we might not necessarily still have around
  (ie. when you delete a post, you might want the title), we're going to
  start utilizing the `context` column
- right now, we store the `primary_name` for deleted events, and we also
  store the `setting` `key` and `group` so we can reference it in the
  audit log
2022-08-23 14:58:41 +02:00
Aileen Nowak
e5b604cca0 Updated camelCase properties in Explore endpoint to be snake_case 2022-08-23 12:04:07 +01:00
Aileen Nowak
4892f1d0d5 Added publication language to admin site endpoint
no issue

- The site locale should be exposed within the public site config in order to handle i18n in third party apps
- Added the locale to Explore service to simplify fetching it when submitting a new site
2022-08-23 12:04:07 +01:00
Simon Backx
fe3430202a
Fixed member attribution for subdirectories (#15277)
fixes https://github.com/TryGhost/Team/issues/1829

- Remove the subdirectories when creating the Attribution instances
- URLs are now always stored relative to the subdirectory instead of the root directory (makes changing the subdirectory easier)
- Fixed returning absolute urls
- Added tests
2022-08-22 17:16:18 +02:00
Simon Backx
0943daad72
Added member attribution to member details page (#15266)
refs https://github.com/TryGhost/Team/issues/1817

Co-authored-by: James Morris <moreofmorris@users.noreply.github.com>
2022-08-19 16:39:18 -04:00
Daniel Lockyer
b023f716ab
Fixed content-length header in Explore snapshot
- this endpoint returns the Ghost version, of which the minor just hit
  double digits
- because of this, the content-length size changed, and the snapshot was
  incorrect
- we've previously allowed overrides for the content-length to be any number (see
  1fadbacdec)
- this commit allows the header to be any number so it doesn't fail when
  the Ghost version is incremented
2022-08-19 17:14:04 +02:00
Hannah Wolfe
809c1a6e08 🐛 Fixed error deleting post with comment replies
closes: https://github.com/TryGhost/Ghost/issues/15252

- comments are deleted when posts are deleted. Without cascade delete on parent_id, replies cannot be deleted
- this change means that deleting a post will delete all comments and replies without error
2022-08-19 15:49:58 +01:00
Hannah Wolfe
d2acf3aada Fixed attribution table missing on cascade delete
refs: https://github.com/TryGhost/Ghost/issues/15252

- all columns with a foreign key (references prop) must have a deletion strategy
- we just found a bug with this in the comments table - see referenced issue
- this fix adjusts the schema and migration for this change before its released so we don't have to write a horrible migration later
2022-08-19 15:28:45 +01:00
Rishabh
1258156c38 Handled storing complimentary subscription expiry
refs https://github.com/TryGhost/Team/issues/1727

- if feature flag is enabled, handles storing expiry date on complimentary subscriptions in `expiry_at` column of `members_products`
- updates the expiry value on both member edit or add with tiers
- expiry is passed as `expiry_at` in `tiers` list of a member
- includes `expiry_at` on tiers data of a member when flag is enabled
2022-08-19 18:20:52 +05:30
Hannah Wolfe
704f17ff96
Added beta search helper implementation (#15236)
refs: TryGhost/Team#1732

- We're testing out the feasibility of having a {{search}} helper that outputs an pre-styled icon to trigger search.
2022-08-19 12:27:38 +01:00
Hannah Wolfe
1fadbacdec
Fixed content-length in labs snapshot
refs: 22fd7f289c

- in the mentioned commit I changed the tests so that we don't need to update snapshots for every labs flag change
- this commit does the same for content-length which didn't get picked up locally, but does on CI for some reason
- the goal is to allow the team to add and remove flags without needing to update a random snapshot
2022-08-19 10:51:43 +01:00
Rishabh Garg
43b8ad5069
Added expiry column for complimentary subscriptions (#15241)
refs https://github.com/TryGhost/Team/issues/1727

- allows adding expiry date to complimentary subscriptions, effectively allowing cardless trials.
- expiry is stored in members<>products mapping table
2022-08-19 08:41:40 +05:30
Fabien "egg" O'Carroll
20726c8fc2 Added signup & conversion counts to Posts API
refs https://github.com/TryGhost/Team/issues/1822

Exposing the values through the API is restricted behind the alpha flag.
We're exposing the values by default when the flag is enabled for now,
but can reconsider that later.
2022-08-18 19:24:54 -04:00
Simon Backx
da24d13601
Added member attribution events and storage (#15243)
refs https://github.com/TryGhost/Team/issues/1808
refs https://github.com/TryGhost/Team/issues/1809
refs https://github.com/TryGhost/Team/issues/1820
refs https://github.com/TryGhost/Team/issues/1814

### Changes in `member-events` package

- Added MemberCreatedEvent (event, not model)
- Added SubscriptionCreatedEvent (event, not model) 

### Added `member-attribution` package (new)

- Added the AttributionBuilder class which is able to convert a url history to an attribution object (exposed as getAttribution on the service itself, which handles the dependencies)
```
[{
    "path": "/",
    "time": 123
}]
```
to
```
{
    "url": "/",
    "id": null,
    "type": "url"
}
```

- event handler listens for MemberCreatedEvent and SubscriptionCreatedEvent and creates the corresponding models in the database.

### Changes in `members-api` package

- Added urlHistory to `sendMagicLink` endpoint body + convert the urlHistory to an attribution object that is stored in the tokenData of the magic link (sent by Portal in this PR: https://github.com/TryGhost/Portal/pull/256).
- Added urlHistory to `createCheckoutSession` endpoint + convert the urlHistory to attribution keys that are saved in the Stripe Session metadata (sent by Portal in this PR: https://github.com/TryGhost/Portal/pull/256).

- Added attribution data property to member repository's create method (when a member is created)
- Dispatch MemberCreatedEvent with attribution

###  Changes in `members-stripe-service` package (`ghost/stripe`)

- Dispatch SubscriptionCreatedEvent in WebhookController on subscription checkout (with attribution from session metadata)
2022-08-18 17:38:42 +02:00
Aileen Nowak
e0602b8159 Fixed explore snapshot 2022-08-18 15:50:28 +01:00
Fabien "egg" O'Carroll
37ef0582e6 Added members_subscription_created_events table & model
refs https://github.com/TryGhost/Team/issues/1803
2022-08-18 10:45:53 -04:00
Fabien "egg" O'Carroll
03155a61ff Added members_created_events table & model
refs https://github.com/TryGhost/Team/issues/1802
2022-08-18 10:45:53 -04:00
Naz
6cf49d8f89 Fixed inconsistent format of image sizes cache
refs https://github.com/TryGhost/Toolbox/issues/364
refs 147ec91162

- This looks like a subtle bug that has gone unnoticed for years. Have checked if we rely on the logic anywhere (mostly used in image-dimensions frontend helper) - we don't access the "url" directly.
- There is no reasoning attached behind why the cached size was stored as a url (see refed commit)

- WHY is this even being fixed? Caches can store anything... does not mean we should! Inconsistent data becomes a real PITA  if the cache is persisted and is hard to repopulate (e.g. to migrate the cached data format).
2022-08-16 15:20:49 +02:00
Naz
e4d02657ee Swapped image size "Map" cache for adapter cache
refs https://github.com/TryGhost/Toolbox/issues/364

- The "new Map()" cache was a "hidden cache" that did not follow any specific pattern. Following the cache adapter pattern here makes it possible swapping out the cache for alternative implementations - e.g. Redis storage
2022-08-16 12:35:35 +02:00
Naz
05eb719d0a Swapped image size cache with InMemoryCache
refs https://github.com/TryGhost/Toolbox/issues/364

- The InMemoryCache is an implementation of the cache adapter interface and allows to test cache in the works which is "close to the real world". Being able to do so in tests for image sizes cache manager proves we can use other cache adapters such as Redis based ones.
2022-08-16 12:35:35 +02:00
Naz
e549528985 Simplified image size cache module constructor
refs https://github.com/TryGhost/Toolbox/issues/364

- As little as possible should be passed in in the parameters for any method/constructor/whatever. Specific Function > vague Object
2022-08-16 12:35:35 +02:00
Naz
92d6c998b3 Fixed error handling bug in image size cache
refs https://github.com/TryGhost/Toolbox/issues/364

- Doing the `.catch(errors.NotFoundError...` was throwing another error as this syntax did not work with native promises. Checking `instanceof` works 100% and is way more explicit/readable way to handle this type of error differently
2022-08-16 12:35:35 +02:00
Naz
38c76847e9 Refactored cache image size test suite
refs https://github.com/TryGhost/Toolbox/issues/364

- It was using an outdated syntax and relied on Bluebird depencency. Updated the syntax to async/await and dropped the Bluebird dependency.
2022-08-16 12:35:35 +02:00
Naz
7d3b678d4e Refactored cached image size to have cache as DI
refs https://github.com/TryGhost/Toolbox/issues/364

- Groundwork before swapping out existing cache for in-memory cache from the adapter
2022-08-16 12:35:35 +02:00
Fabien "egg" O'Carroll
c37670ea40 Fixed Settings API test snapshot
refs https://github.com/TryGhost/Team/issues/1801

Adding a new labs flag increased the content length of the settings snapshot
2022-08-15 17:20:43 -04:00
Fabien 'egg' O'Carroll
96cd6df6fa
🐛 Fixed Comments administration for self hosters (#15239)
refs https://github.com/TryGhost/Team/issues/1799

Rather than using the `adminAuthAssets` config which is not updated to
be aware of running in a different directory to the cwd, we use the
getContentPath method which handles all of the directory checking.

Without this, we were unable to serve the admin-auth iframe, as the
directory was incorrect for self hosters.
2022-08-15 15:38:19 -04:00
Daniel Lockyer
b27ea8f504
Optimized database resets for local testing
- we've had an optimization in CI that copies a fresh SQLite DB to a
  file, and copies it back when we need to do a DB reset
- I originally only let this run in CI but we've had it around for a
  while so we should GA it to run on local machines
- there may be edge cases, but we should fix them instead of letting
  tests run slower for development
- this also makes sure we clean up any existing files before
  initializing the DB
2022-08-15 16:58:56 +02:00
Hannah Wolfe
22fd7f289c
Removed need for snapshot updates for feature flags
- prior to this commit, if you add or remove a faeture flag, you also have to update the snapshots for the settings tests
- feature flags are intended to be very easy to add and remove, and so this extra step doesn't fit with our needs
- it's also unnecessary, we don't need to verify the exact contents of the labs setting
2022-08-15 15:36:14 +01:00
Daniel Lockyer
70aa1dd3c8
Added auditLog labs flag
refs https://github.com/TryGhost/Toolbox/issues/356

- allows work on the Audit Log project whilst in alpha
2022-08-15 14:59:32 +02:00
Simon Backx
a666b846e6
🐛 Fixed unsubscribe flow for comment reply emails (#15232)
refs https://github.com/TryGhost/Team/issues/1790

- Currently we go to the account settings when you click 'Manage your email preferences' in the footer of an email that informs a comment author that they received a reply.
- Related Portal changes are here: https://github.com/TryGhost/Portal/pull/255
2022-08-15 11:36:08 +02:00
Simon Backx
27df4e536f
Added native comments (#15223)
no issue

- Removes the comments lab flag (bumping it to GA).
- Bumps comments-ui to v0.8
2022-08-12 16:06:06 +02:00
Aileen Nowak
77e4be6b81 Updated tests to incl. host limit cases for core integrations
- added core and builtin integrations to test fixtures
- allowed passing a custom api key id to generate JWT
- updated admin key auth test to make successful request with a `core` integration, which doesn't work atm because relations are not returned
2022-08-12 14:18:44 +01:00
Aileen Nowak
5ac779f86b Added core type integrations to API serializer 2022-08-12 14:18:44 +01:00
Aileen Nowak
0e284edbad Allowed core integrations to work with Admin API 2022-08-12 14:18:44 +01:00
Daniel Lockyer
687e68d5de Extracted shared API framework to separate package
refs https://github.com/TryGhost/Toolbox/issues/363

- this API framework is standalone and should be pulled out into a
  separate package so we can define its boundaries more clearly, and
  promote better testing of smaller parts
2022-08-11 17:44:59 +02:00
Daniel Lockyer
dc0cec56ea Moved settings mappers from shared frame to endpoints
- I'm still not sure where these are meant to be but they don't really
  belong in shared and I'm about to pull that folder out into a separate
  package
2022-08-11 17:44:59 +02:00
Daniel Lockyer
7d52ae52cf Deleted concept of external user
refs https://github.com/TryGhost/Toolbox/issues/363

- AFAICT, this is an outdated and unused concept from when subscribers
  were a thing, but members are now how we do things, and this is causing code
  to linger around that we don't need
2022-08-11 16:05:00 +02:00
Daniel Lockyer
42e722d627
Moved Cache-Control middleware to separate package
refs https://github.com/TryGhost/Toolbox/issues/363

- this middleware is standalone and I suspect we're going to be touching
  it further when we work on Ghost's caching in the near future
2022-08-11 14:14:20 +02:00
Rishabh
843bbfa55d Handled stripe setup for free trial offers
refs https://github.com/TryGhost/Team/issues/1726

- free trial offers don't need a stripe coupon created for them
- checkout sessions for free trial offers ignore stripe coupon and directly pass the trial days value
- trial days of an offer take precedence over trial days added as default to a tier
2022-08-11 11:04:39 +05:30
Daniel Lockyer
bf254b9c6a Extracted Mailgun client to separate package
refs https://github.com/TryGhost/Toolbox/issues/363

- this commit pulls all code involving the Mailgun client SDK into one
  new package called `mailgun-client`
- this means we should be able to replace `mailgun-js` (deprecated) with
  `mailgun.js` (the new, official one) without editing code all over the
  place
- this also lays some groundwork for better testing of smaller
  components
2022-08-10 17:12:37 +02:00
Simon Backx
82a3133ace
Added replies/likes count and limited replies to comments (#15206)
refs https://github.com/TryGhost/Team/issues/1723

- Added count.replies to comments
- Added replies endpoint
- Limited returned replies to 3.
- Replaced likes_count with count.likes in comments
- Instead of fetching all the likes of a comment to determine the total count, we'll now use count.likes
- Instead of fetching all the likes of a comment to determine whether a member liked a comment, we'll now use count.liked (which returns the amount of likes of the current member, being 0 or 1). This is mapped to `liked` to make it more natural to work with.

The `members.test.snap` file changed because we no longer include `liked: false` if we didn't fetch the liked relation. And in the comments events of the activity feed the liked property is therefore removed.

These changes requires an update to the `bookshelf-include-count` plugin:
- Updated to also work for nested relations
- This moves the count queries from the `bookshelf-include-count` plugin to the `countRelations` method of each model.
- Updated to keep the counts after saving a model (crud.edit didn't return the counts before)
2022-08-10 16:12:35 +02:00
Fabien 'egg' O'Carroll
f34740d6d0
Added support for autowrap and class to the comment_count helper (#15203)
refs https://github.com/TryGhost/Team/issues/1760

This allows theme developers to wrap the output of the comment_count
helper in an element, which will only be shown when there is content
to output.

This makes styling a lot easier, as the default output for no comments
is nothing, meaning that separators defined with CSS will not be rendered.
2022-08-09 13:08:36 +01:00
Rishabh Garg
098f40bbe3
Added trial info to member subscription detail (#15193)
refs https://github.com/TryGhost/Team/issues/1757

- exposes trial start and end dates in member's subscription object
- allows portal and admin to show member's trial information in UI
2022-08-09 17:28:00 +05:30
Rishabh
ce80d250bf Handled null trial days on tiers
refs e26c977c66

- handles null trial days in admin and API, sets trial days as 0 for null values
2022-08-09 14:14:22 +05:30
Fabien 'egg' O'Carroll
5165a0b298
Redacted comment html for deleted and hidden comments (#15173)
refs https://github.com/TryGhost/Team/issues/1745

This is done at the mapper layer, so that the model can be used for the
Admin API - where will probably want to expose the underlying content.

We've also disabled editing of deleted/hidden comments, to avoid
accidentally overriding the comments html when sending up deleted
comments.
2022-08-08 13:29:27 +01:00
Simon Backx
eb4d882bb2 Fixed whitespace in bio not removed
fixes https://github.com/TryGhost/Team/issues/1755
2022-08-05 17:10:56 +02:00
Rishabh Garg
5704ac061e
Handled storing of trial start/end info for subscription (#15161)
refs https://github.com/TryGhost/Team/issues/1724

With free trials, members can start subscriptions with a trial period. This change stores the information about trial start and end date for every subscription so it can be shown on Admin/Portal for member.

- adds new `trial_start_at` column for storing trial start date on Stripe subscription. Will in most cases match the start of subscription date.
- adds new `trial_end_at` column for storing trial end date on Stripe subscription.
- wires storing trial start and end values on stripe subscription
2022-08-05 17:50:40 +05:30
Rishabh
2d12d9aa89 Wired trial days to tiers API
refs https://github.com/TryGhost/Team/issues/1724

- Added `trial_days` to api serializers
2022-08-05 17:23:40 +05:30
arsereg
182a7ea07c
💡 Upgraded Tenor API to v2 (#15087)
closes: https://github.com/TryGhost/Ghost/issues/14980
refs: cc276486f0

- Tenor is now operated by Google, and the old v1 Tenor API has been decommissioned
- At present anyone with a pre-configured tenor integration will get intermittent errors, whilst it is impossible to setup a new tenor API integration
- Sadly old keys do not work with the new API, and new keys do not work with the old API, so there is no happy path forward.
- After this lands, everyone will need to go and get a new Google API Key for Tenor, update their config, and then the integration will work properly again.
- This particular change renames the API key from `publicReadOnlyApiKey` to `googleApiKey` to reflect that the key itself changes in type and behaviour

Co-authored-by: Hannah Wolfe <github.erisds@gmail.com>
2022-08-05 12:13:27 +01:00
Aileen Nowak
085958d53d Added Ghost Explore integration
no issue

- bumps Ghost Explore to GA from alpha
2022-08-05 10:06:10 +01:00
Naz
fa8d94fce2 Fixed the typo
refs e9bfc4ef01

- Did a typo in the find and replace... and now correcting a typo of a typo  -_-
2022-08-04 15:38:32 +01:00
Naz
2de7688eaf Used in memory cache adapter in settigs cache manager
refs https://github.com/TryGhost/Toolbox/issues/364

- Settings Manager used to store all of it's settings values in a hash - an in memory cache in disguise. Having a hidden cache made it hard to reason about it's impact of memory usage and did not allow to swap it out for an alternative storage metchanism easily. Having a cache storage abstraction in Settings Manager allows to get rid of long lasting memory problems + decouples storage mechanism from the logic around transforming stored values.
2022-08-05 02:28:33 +12:00
Naz
ed79d3e9b3 Added basic memory cache storage implementation
refs https://github.com/TryGhost/Toolbox/issues/364

- This is groundwork to substitute in memory caches we use across the codebase. The first candidate would be settings cache. The interface of the memory cache was kept to the minimum. The "keys" method is a somewhat acceptable alternative to the "getAll" method used in the codebase right now.
- The next iteration over this would be adding async methods are alternative key/value storage methanisms like Redis.
2022-08-05 02:28:33 +12:00