Commit Graph

485 Commits

Author SHA1 Message Date
Simon Backx
480c1a7004 Removed name from subscription details if missing
refs https://github.com/TryGhost/Team/issues/2736

If the name is not known for a member, we'll hide the name row in the subscription details in an email. This method is supported in most email clients, and requires the support of `<style>` in `<head>`.
2023-03-22 15:32:07 +01:00
Sanne de Vries
47e343ec18 Updated latest posts UI in email template
Refs https://github.com/TryGhost/Team/issues/2801
2023-03-22 15:16:55 +01:00
Sanne de Vries
792bfdb498 Added UI for subscription box in newsletters
Refs https://github.com/TryGhost/Team/issues/2738

- Added preview to email newsletter settings
- Added subscription box UI to email template
2023-03-21 11:40:04 +01:00
Rishabh
0b7e09c986 Updated fallback for storing source site title
refs https://github.com/TryGhost/Team/issues/2754

Previously, we didn't have any backup for storing source site title and it could have stored as empty if missing. This change ensures the source site title is stored as site host instead as fallback if not present.
2023-03-21 16:09:21 +05:30
Simon Backx
07ec33fb3a
Added latest posts to email template (#16448)
refs https://github.com/TryGhost/Team/issues/2769

Needs some extra styling and design, this is only a minimal version
behind the feature flag.
2023-03-20 14:30:42 +01:00
Simon Backx
a513542b56
Added show_latest_posts column to newsletter table (#16402) 2023-03-20 11:10:49 +01:00
Steve Larson
0a4d568f19 updated snapshot
no refs
-updated snapshot to pass tests
2023-03-17 10:57:50 -05:00
Steve Larson
067b35de6b fixed member newsletter test
no refs
-fixed flaky test that seemed to have sync issues with fixture data and db
2023-03-17 10:27:09 -05:00
Simon Backx
b3c410a923 Reverted stats snapshot change 2023-03-17 16:06:00 +01:00
Sanne de Vries
48a3159d3d Added mobile feedback buttons to email template
Refs https://github.com/TryGhost/Team/issues/2740
2023-03-17 15:58:14 +01:00
Simon Backx
298e3da745 Fixed comment CTA button link
refs https://github.com/TryGhost/Team/issues/2672

- Use #ghost-comments instead of #ghost-comments-root
- Fixed snapshots
2023-03-17 10:27:23 +01:00
Simon Backx
423337a048 Fixed snapshots 2023-03-17 10:24:21 +01:00
Sanne de Vries
717ed114f4 Fixed center aligned post title in emails
Refs https://github.com/TryGhost/Team/issues/2740
2023-03-17 10:07:17 +01:00
Chris Raible
0640b9c4aa
Modified outbound link tagger to use ?ref=domain instead of ?ref=siteTitle (#16431)
refs TryGhost/Team#2755

- Outbound link tagger will now append ?ref=example.com or ?ref=example.ghost.io instead of ?ref=siteTitle
2023-03-17 14:18:26 +05:30
Steve Larson
7f223b56dd
fixed member newsletter test (#16425)
no ref
-fixed flaky test that seemed to have sync issues with fixture data and
db
2023-03-16 10:43:17 -05:00
Sanne de Vries
cdcb3dcd6f
Updated email template to include comment link in header and footer (#16423)
Refs https://github.com/TryGhost/Team/issues/2740

---------

Co-authored-by: Steve Larson <9larsons@gmail.com>
2023-03-16 16:10:53 +01:00
Simon Backx
450e01d1c0 Added placeholder basic subscription details to email template
refs https://github.com/TryGhost/Team/issues/2736

The renewal date is still missing, and style and desin is only a placeholder.
2023-03-15 17:12:56 +01:00
Simon Backx
6e5ddc6a9a Fixed email preview snapshots 2023-03-15 14:03:24 +01:00
Sanne de Vries
e0c3573079 Updated email template snapshot resulting in test failure
No ref
2023-03-15 12:45:54 +01:00
Simon Backx
da2e56916e
Added show_subscription_details column to newsletter table (#16403)
fixes https://github.com/TryGhost/Team/issues/2708

Adds `show_subscription_details` column to newsletter table. Disabled by
default.
2023-03-15 11:53:45 +01:00
Simon Backx
bb019a0ab5 Added E2E test for sending email without post title
fixes https://github.com/TryGhost/Team/issues/2724

This change also includes new snapshots for email sending (similar for email previews, but this time for the real emails to make sure we catch changes).
2023-03-14 16:05:18 +01:00
Simon Backx
350b291c34
Added show_comment_cta column to newsletter table (#16401)
fixes https://github.com/TryGhost/Team/issues/2707

Adds the new column for storing whether we need to show a comment CTA in
emails. Enabled by default.
2023-03-14 15:03:02 +01:00
Simon Backx
1e435fb328
Implemented showPostTitleSection (#16404)
fixes https://github.com/TryGhost/Team/issues/2705

- Added showPostTitleSection to newsletter model in admin
- Wired up UI to admin model so it saves to the database
- Implemented showPostTitleSection in newsletter preview and added some
minor temporary css styling
- Implemented showPostTitleSection in newsletter template in backend,
and added some extra CSS styling to fix spacing
2023-03-14 11:29:43 +01:00
Simon Backx
69d07c892c
Fixed getLazyRelation returning new model for optional relations (#16411)
no issue

When using `getLazyRelation` on an optional relation that is not set, it
will return a newly created model instead of a model from the database.
- Adds a new require option to `getLazyRelation`, that throws an error
if the relation is not set (off by default to match existing use cases)
- This caused a bug (not visible because we always pass a newsletter id)
in email previews, where when the newsletter id was not explicitly set,
it would use `newsletter = (await post.getLazyRelation('newsletter')) ??
(await this.models.Newsletter.getDefaultNewsletter());`, which always
returned the first one, and could return a newly initiated newsletter
with all properties set to undefined.
- Some page snapshots are altered by this, because the usage of
`getLazyRelation` on a post no longer sets the email relation to some
new model.
2023-03-14 10:58:58 +01:00
Simon Backx
ff5e7202c6
Added show_post_title_section column to newsletters table (#16397)
fixes https://github.com/TryGhost/Team/issues/2706

Only includes the migration. The setting is not used at the moment, but
will be enabled by default.
2023-03-14 09:48:13 +01:00
Fabien 'egg' O'Carroll
4525c7293c
Removed email notifications for new mentions
These are no longer desired and will be replaced with a digest style
email.
2023-03-13 18:33:57 +07:00
Simon Backx
80cec29144
Added Stripe Mocker to make testing easier (#16396)
no issue

The Stripe Mocker mocks the Stripe API in memory, to make it much easier
to test subscription flows. Currently it is more a POC to see if it
works well. It probably needs a bit more work to support more scenarios.
- Added new tests for the subscription stats endpoint for 3D secure +
free trial flows using the new Stripe Mocker
- Updated members admin api tests to use Stripe Mocker (+ added new test
for deleting members with Stripe cancellation)
- Some tests called mockStripe at the beginning, but that method did
nothing apart from disabling network (which is the default now), then
they mocked Stripe inside the tests file... so I've removed those
because those conflict with the new mocker that is enabled when calling
mockStripe. We'll need to port those over later.
2023-03-10 18:43:35 +01:00
Daniel Lockyer
89493893d1 Removed all unused variables from test files
- this cleans up all imports or variables that aren't currently being used
- this really helps keep the tests clean by only allowing what is needed
- I've left `should` as an exemption for now because we need to clean up
  how it is used
2023-03-10 14:29:55 +01:00
Daniel Lockyer
08f84475e7 Removed sleep lib from test utils
- this is generally an anti-pattern in tests and leads to flaky
  behaviour when tests are ran on different machines/loads
- this is currently unused so it is an easy removal
2023-03-10 14:29:55 +01:00
Ronald Langeveld
ab01e4fcbf Added new Tier e2e test
refs https://github.com/TryGhost/Team/issues/1446

- Added a test for successfully creating a new tier.
2023-03-10 13:29:25 +08:00
Simon Backx
832610fd2a
🐛 Fixed retrying failed emails when rescheduling them (#16383)
fixes https://github.com/TryGhost/Team/issues/2560

When an email fails, and you reschedule the post, the error dialog was
shown (from the previous try). The retry button on that page allowed you
to retry sending the email immediately, which could be very confusing.

- The email error dialog is no longer shown for scheduled emails
- The email status is no longer polled for scheduled emails
- Retrying an email is not possible via the API if the post status is
not published or sent
- Added some extra snapshot tests
- When retrying an email, we immediately update the email status to
'pending' to have a better API response (instead of still returning
failed).
- Disabled email sending retrying in development (otherwise very hard to
test failed emails if it takes 10 mins before it gives up automatic
retrying)
2023-03-09 12:32:22 +01:00
Steve Larson
cb05fae5a3
🐛 Fixed maintaining archived newsletter subscriptions for members (#16375)
refs #16355
-archived newsletters will no longer be unsubscribed when saving member
-fixed bug with initialMember load
-fixed errors in unit tests
2023-03-08 16:39:28 -06:00
Simon Backx
4184b279d2
🐛 Fixed HTML escaping when using outbound link tagging (#16380)
fixes https://github.com/TryGhost/Team/issues/2666

- Somehow occurrences of `&map_` got replaced with `&#x21A6;`
- Disables escaping &, ', " and other HTML characters when not needed
(escaping is already handled by mobiledoc/lexical)
- Bumps unit test coverage of link replacer to 100%
2023-03-08 16:30:54 +01:00
Simon Backx
3d89dbb775 Fixed whistelisted domains in Playwright tests and network connectivity
no issue

- Nock doesn't support multiple calls to enableNetConnect -> only the last one counts. This fixes that issue.
- Some tests interacted directly with nock instead of using the mockManager to restore everything.
2023-03-08 12:40:21 +01:00
Simon Backx
38de815d98
Removed old email flow (#16349)
fixes https://github.com/TryGhost/Team/issues/2611

The old email flow is no longer used since we introduced the email stability flow. This commit removes the related code and tests. The general test coverage decreased a bit as a result, because the old email flow probably had a high test coverage. The new flow is in separate packages, so it couldn't contribute to a higher test coverage (but it does have 100% unit test coverage).
2023-03-07 16:08:40 +01:00
Simon Backx
3db434736b
🐛 Fixed replacements with fallback in plaintext newsletters (#16372)
fixes https://github.com/TryGhost/Team/issues/2683

When sending a newsletter with a replacement that has a fallback, the
replacement only happens in the HTML version of the newsletter. The
plaintext version isn't replaced.

This commit fixes the issue and adds some tests to make sure it doesn't
happen again.

The cause of the issue was that we used the original matched Regex text
to replace. But that was calculated on the HTML version, so double
quotes were encoded. This change updates the generated 'token' regex to
also match on both a double quote as the escaped double quote.
2023-03-07 15:34:43 +01:00
Simon Backx
f50224f445
Improved network access and mocking in tests (#16371)
refs https://github.com/TryGhost/Team/issues/2667

Some tests still accessed the internet. Now network access is disabled
by default. This change also introduces two helper methods related to
networking (mocking Slack and Mailgun).

This fixes two unreliable tests:
- Staff service was accessing a Slack test API -> timeout possible
- MentionSendingService was trying to send webmentions for every post
publish/change -> possible timeouts and job issues
2023-03-07 13:20:28 +01:00
Tim Gates
5394fa81fc
Fixed typos (#16359) 2023-03-06 13:11:45 +01:00
Naz
33316a6440
Added e2e test covering media inliner job start
refs https://github.com/TryGhost/Toolbox/issues/523

- The test is useful for future iterations of the response format and as a quick reference on which parameters the media inlining endpoint accepts.
2023-03-06 17:43:33 +08:00
Sam Lord
13b0f51b13 Prevent simple error logs by stubbing log library
refs: https://github.com/TryGhost/Toolbox/issues/389

This removes many error logs when the end-to-end test suite is run with the log-level set to error. Many errors are intentional, so the resolution is typically to stub the error log function and assert that it would have been called.
2023-03-03 18:25:53 +00:00
Sam Lord
cd5aca7227 Prevented .well-known file tests from attempting to send mail
refs: https://github.com/TryGhost/Toolbox/issues/389

These tests would log errors while attempting to send email.
2023-03-03 18:25:53 +00:00
Sam Lord
5b2adbe4c5 Prevents errors from being logged during the offers tests
refs: https://github.com/TryGhost/Toolbox/issues/389

Intentional errors now call a logging library stub
2023-03-03 18:25:53 +00:00
Sam Lord
64d94c8c8c Prevent posts-legacy tests from logging errors
refs: https://github.com/TryGhost/Toolbox/issues/389

The newsletter fixtures no longer errors when accessing the test image, and tests which intentionally error now stub the logging call
2023-03-03 18:25:53 +00:00
Ronald Langeveld
7c805b7176 Revert "Added new created tier e2e test"
This reverts commit dc98868961.
2023-03-02 16:39:17 +08:00
Ronald Langeveld
dc98868961 Added new created tier e2e test
ref https://github.com/TryGhost/Team/issues/1446

- Added e2e test to check if a new tier can be added successfully.
2023-03-02 16:11:12 +08:00
Reupen Shah
c5a4ee89c9 🐛 Fixed uploads of m4a files with audio/mp4 content type
fixes https://github.com/TryGhost/Ghost/issues/16301

Previously, audio/x-m4a was allowed but not audio/mp4. This meant
uploads of m4a files failed in some cases e.g. Firefox on Windows.
2023-03-01 08:32:21 +01:00
Simon Backx
3ded0bbee8
Added outbound link tagging setting (#16324)
no issue

Enable or disable outbound link tagging in both web posts and
newsletters.
2023-02-23 14:37:23 +01:00
Simon Backx
f6bc3479f9
Added webmentions emails flag (#16321)
fixes https://github.com/TryGhost/Team/issues/2599

Disables sending webmentions notification emails, and hides it behind a new flag instead.
2023-02-23 11:42:14 +01:00
Steve Larson
18eeb9e523 moved spam prevention test
no refs
-spam prevention test was causing subsequent tests to fail randomly
-moving to the end ensures (for now) we don't interrupt other tests
-seems to be an issue with awaiting the jobservice which do concurrent
2023-02-22 09:52:26 -06:00
Simon Backx
a7090dddcd Email snapshot updates 2023-02-22 16:49:28 +01:00
Simon Backx
f45d1810a6 Improved webmention receiving E2E test reliability
refs https://github.com/TryGhost/Team/issues/2596
2023-02-22 16:19:56 +01:00
Ronald Langeveld
76640d9b7e
Added page resources to emails (#16315)
refs https://github.com/TryGhost/Team/issues/2572

- Passes post resource data to emails.
- Updated email templates accordingly.
- Added snapshot e2e tests.
2023-02-22 21:02:15 +08:00
Aileen Booker
4556e9be88
Added milestone_notifications column (#16306)
refs
https://www.notion.so/ghost/Marketing-Milestone-email-campaigns-1d2c9dee3cfa4029863edb16092ad5c4?pvs=4

- created a migration for a new boolean column in users that would
determine if the staff user gets an email when a new milestone is
achieved.
2023-02-22 14:13:50 +02:00
Ghost CI
d3563d9c06 Merged v5.35.1 into main 2023-02-21 15:36:02 +00:00
Simon Backx
461634c57c Fixed snapshot tests 2023-02-21 16:15:00 +01:00
Steve Larson
2d84b7d990
Upgraded got package from v9.6.0 to v11.8.6 (#16261)
Refs TryGhost/Team#2459
-upgraded got from v9.6.0 to v11.8.6 to support following redirects (and
other fixes)
-got v12+ requires ESM, so we do not want to upgrade further at this
time
-required changes to a few libraries that use externalRequests
-mention discovery service tests updated to test for follow redirects
2023-02-20 09:33:11 -06:00
Simon Backx
52a26a7f80 Removed unreliable upload test
no issue

This test failed randomly. It is hard to improve the test without defeating the purpose of the test.
2023-02-20 13:58:01 +01:00
Fabien "egg" O'Carroll
8908a51547 Improved verification logic for Mentions
refs https://github.com/TryGhost/Team/issues/2550

By using cheerio to parse the HTML we can correctly look for elements
which use the target URL as the href attribute, rather than doing a
plaintext search. This closer to what the spec says.
2023-02-17 19:07:11 +07:00
Fabien 'egg' O'Carroll
101cabb188
Added separate job service for webmentions (#16285)
[Added initial mentions-jobs
service](3656190114)

This is the result of running `cp -r jobs mentions-jobs` in the services
directory.

[Waited for mentions-jobs queue before
shutdown](2bb1a12a89)

This matches the functionality of the existing jobs service where we
will wait
for jobs to complete before closing the process.

[Used mentions-jobs service in the mentions
service](4e4f9fdd00)

This ensures that any delays in the mentions jobs queue does not effect
other
parts of the application.
2023-02-17 13:05:36 +01:00
Naz
f27c7ba044
Fixed variable name typo 2023-02-17 17:27:51 +08:00
Simon Backx
8dfa490638
Added outbound link tagging for web posts (#16201)
fixes https://github.com/TryGhost/Team/issues/2433

- Moved all outbound link tagging code to separate OutboundLinkTagger
- Because a site can easily enable/disable this feature, we don't store
the ?refs in the HTML but add them on the fly for now in the Content
API.
2023-02-16 11:26:35 +01:00
Fabien "egg" O'Carroll
c6b90b6ac7 Added verified property to Mentions Admin API
closes https://github.com/TryGhost/Team/issues/2551

Rather than blindly passing all data through the API we explicitly include each
new property. This allows us to make changes to the core entities without
affecting the API. The verified property is being added now to give design the
ability to display these mentions differently.

We also needed to include the verified property in the return value of toJSON,
this was missed as part of the original entity changes
2023-02-16 14:31:05 +07:00
Ronald Langeveld
f9eeec10db Updated email-preview test snapshot
no issue

- fixed up tests failing from the outdated email snapshot.
2023-02-15 11:21:37 +08:00
Simon Backx
4e361a65fa Added webmention delete test and cleaned up webmention tests
refs https://github.com/TryGhost/Team/issues/2553

- Added webmention delete test
- Fixed dangling promise in test
- Fixed missing .persist in tests
2023-02-13 17:15:34 +01:00
Fabien "egg" O'Carroll
bf6bcdbff3 Added e2e tests for updating Mentions
closes https://github.com/TryGhost/Team/issues/2552

We send a Webmention for the same URL twice, but change the contents
of the source document, and we check that the source metadata is
updated appropriately.

We should consider extending all of these tests to include featured
images and logos etc...
2023-02-13 19:09:31 +07:00
Fabien 'egg' O'Carroll
57695d03df
Fixed rate limit test (#16258)
closes https://github.com/TryGhost/Team/issues/2547

Changed the configuration for testing to be a bit more strict, by slowing down the amount of requests it can handle to give CI enough time to kick in the rate limiter. Before this, CI simply wasn't hitting the API fast enough to trigger the rate limiter.

Co-authored-by: Ronald Langeveld <hi@ronaldlangeveld.com>
2023-02-13 15:16:56 +08:00
Ronald Langeveld
4096571b8d Commented out flaky webmention test
refs https://ghost.slack.com/archives/C02G9E68C/p1675934676146349

- Commented out the rate limiter test as it proves to be flaky on CI,
  till we find a better way of implementing it.
2023-02-09 21:35:15 +08:00
Ronald Langeveld
ce567b9816
Added Mentions email notification settings (#16245)
closes https://github.com/TryGhost/Team/issues/2526

- Mention emails can now be toggled inside staff user' profiles, if they
have the webmention flag enabled on their Ghost site.
- Removed the flag dedicated to webmention email notifications and is
now handled by the `webmention` flag.
- Does not send email notification if `webmention` flag is not enabled.
- Updated tests.

---------

Co-authored-by: Fabien "egg" O'Carroll <fabien@allou.is>
2023-02-09 17:03:03 +08:00
Ronald Langeveld
30dc2a8228
Added mention_notifications column (#16242)
refs https://github.com/TryGhost/Team/issues/2526

- created a migration for a new boolean column in users that would
determine if the staff user gets an email when the publication receive a
new mention.
2023-02-09 16:15:54 +08:00
Ronald Langeveld
9fc13cfe65
Implemented mentions rate limiter (#16223)
closes https://github.com/TryGhost/Team/issues/2419

- adds a rate limiter implementation to the mentions receiving
endpoint.
- Current configuration is `{"minWait": 10,
             "maxWait": 100,
             "lifetime": 1000,
             "freeRetries": 100}` which is still very open and almost unrestricted. 
- currently makes use of database storage to track the limits, but can be relatively easily swapped out to something eg Redis should we find this endpoint getting hit too often and maliciously.
2023-02-09 14:57:48 +08:00
Fabien "egg" O'Carroll
e5d31e2900 Added resource data to Mentions Admin API
refs https://github.com/TryGhost/Team/issues/2503

This is in the MentionController atm as it's considered a presentation
concern. We might want to consider moving this into the MentionsAPI in
future so that we can simplify the controller and even remove it
completely in favour of putting the data-mapping in the endpoint definition.
2023-02-02 16:13:49 +07:00
Ronald Langeveld
bef3516f72 Added e2e test for filtering members by tier id
no issue

- Added a test for filtering members by tier id since we now have a new
  way of filtering members in the admin.
2023-02-02 15:10:19 +08:00
Simon Backx
8e66edee2b
🐛 Fixed storing original files for images (#16117)
fixes https://github.com/TryGhost/Team/issues/481

This change fixes an issue when multiple images with the same name are
uploaded in parallel. The current system does not guarantee that the
original filename is stored under NAME+`_o`, because the upload for the
original file and the resized file are happening in parallel.

Solution:
- Wait for the storage of the resized image (= the image without the _o
suffix) before storing the original file.
- When that is stored, use the generated file name of the stored image
to generate the filename with the _o suffix. This way, it will always
match and we don't risk both files to have a different number suffix.
We'll also set the `targetDir` argument when saving the file, to avoid
storing the original file in a different directory (when uploading a
file around midnight both files could be stored in 2023/01 and 2023/02).

Some extra optimisations needed with this fix:
- Previously when uploading image.jpg, while it already exists, it would
store two filenames on e.g., `image-3.jpg` and `image_o-3.jpg`. Note the
weird positioning of `_o`. This probably caused bugs when uploading
files named `image-3.jpg`, which would store the original in
`image-3_o.jpg`, but this original would never be used by the
handle-image-sizes middleware (it would look for `image_o-3.jpg`). This
fix would solve this weird naming issue, and make it more consistent.
But we need to make sure our middlewares (including handle-image-sizes)
will be able to handle both file locations to remain compatible with the
old format. This isn't additional work, because it would fix the old bug
too.
- Prevent uploading files that end with `_o`, e.g. by automatically
stripping that suffix from uploaded files. To prevent collisions.

Advantage(s):
- We keep the original file name, which is better for SEO.
- No changes required to the storage adapters.

Downside(s):
- The storage of both files will nog happen parallel any longer. But I
expect the performance implications to be minimal.
- Changes to the routing: normalize middleware is removed
2023-01-30 16:40:50 +01:00
Simon Backx
762238c7e7 Cleaned DomainEvents import 2023-01-30 15:46:31 +01:00
Simon Backx
60670c21ce Fixed webmentions test awaiting events
no issue

The test was not waiting for dispatched events.
2023-01-30 15:45:16 +01:00
Steve Larson
477295a262
added mentions admin browse api e2e tests (#16177)
refs TryGhost/Team#2468
-added simple browse api tests for webmentions
-need to add user access tests and possibly site access (private sites)
2023-01-30 07:59:00 -06:00
Simon Backx
8f8ca481a6
Fixed configUtils and adapter cache issues in E2E tests (#16167)
no issue

There are a couple of issues with resetting the Ghost instance between
E2E test files:

These issues came to the surface because of new tests written in
https://github.com/TryGhost/Ghost/pull/16117

**1. configUtils.restore does not work correctly**
`config.reset()` is a callback based method. On top of that, it doesn't
really work reliably (https://github.com/indexzero/nconf/issues/93)

What kinda happens, is that you first call `config.reset` but
immediately after you correcty reset the config using the `config.set`
calls afterwards. But since `config.reset` is async, that reset will
happen after all those sets, and the end result is that it isn't reset
correctly.

This mainly caused issues in the new updated images tests, which were
updating the config `imageOptimization.contentImageSizes`, which is a
deeply nested config value. Maybe some references to objects are reused
in nconf that cause this issue?

Wrapping `config.reset()` in a promise does fix the issue.

**2. Adapters cache not reset between tests**
At the start of each test, we set `paths:contentPath` to a nice new
temporary directory. But if a previous test already requests a
localStorage adapter, that adapter would have been created and in the
constructor `paths:contentPath` would have been passed. That same
instance will be reused in the next test run. So it won't read the new
config again. To fix this, we need to reset the adapter instances
between E2E tests.

How was this visible? Test uploads were stored in the actual git
repository, and not in a temporary directory. When writing the new image
upload tests, this also resulted in unreliable test runs because some
image names were already taken (from previous test runs).

**3. Old 2E2 test Ghost server not stopped**
Sometimes we still need access to the frontend test server using
`getAgentsWithFrontend`. But that does start a new Ghost server which is
actually listening for HTTP traffic. This could result in a fatal error
in tests because the port is already in use. The issue is that old E2E
tests also start a HTTP server, but they don't stop the server. When you
used the old `startGhost` util, it would check if a server was already
running and stop it first. The new `getAgentsWithFrontend` now also has
the same functionality to fix that issue.
2023-01-30 14:06:20 +01:00
Fabien 'egg' O'Carroll
73bddef7c5
Used job queue for processing incoming Webmentions
refs https://github.com/TryGhost/Team/issues/2419

We use a job queue to ensure that webmentions can be processed outside of
the request/response cycle, but still finish executing if the processed is closed.

With this we're able to update the e2e tests to await the processing of the mention
rather than sleepign for arbitrary lengths of time, and we've reintroduced the tests
removed previously

- aa14207b69
- 48e9393159
2023-01-30 18:11:30 +07:00
Naz
d3ff4348c8
Fixed test name as content version header is always present
refs 9ba251238a

- The test name should have been updated along with referenced change.
2023-01-26 19:20:15 +08:00
Ronald Langeveld
dd74f42376
Added mentions email notifications (#16170)
closes https://github.com/TryGhost/Team/issues/2429

- sends email notifications to staff users when their site receives a Webmention.
- currently behind a flag, that can be toggled in the labs settings.
2023-01-25 21:10:29 +08:00
Daniel Lockyer
977c95157a
Removed flaky test
refs https://ghost.slack.com/archives/C02G9E68C/p1674558376712339

- this test is also flaky and causing random failures
2023-01-25 11:41:59 +01:00
Fabien "egg" O'Carroll
48e9393159 Removed flaky test
This test is failing because the `sleep` isn't long enough. Removing this test
until we've refactored to use the jobs service, at which point we can remove the
sleep and wait for the job to be complete.
2023-01-24 18:50:04 +07:00
Daniel Lockyer
56b0e65583
Revert "Added email snapshots for API versioning tests (#16139)"
- this reverts commit 85051199e3
- the tests here rely on dynamic content (the Ghost version number)
2023-01-20 18:05:36 +01:00
Fabien 'egg' O'Carroll
33ebe971f8
Added BookshelfMentionRepository implementation (#16156)
fixes https://github.com/TryGhost/Team/issues/2418

This stores the received webmentions in the database.

Co-authored-by: Simon Backx <simon@ghost.org>
2023-01-20 14:32:50 +01:00
Rishabh Garg
c7230f1858
Improved email stability (#16159)
- bumps new email flows to GA improving overall email stability
2023-01-20 18:55:20 +05:30
Simon Backx
e879406659
Added outbound link tagging setting (#16146)
fixes https://github.com/TryGhost/Team/issues/2432
    
Adds outbound_link_tagging setting (enabled by default and behind
feature flag). If the feature flag is enabled, and the setting is
disabled, we won't add ?ref to links in emails.
    
This includes new E2E tests for email click tracking, which were also
extended to check outbound link tagging (for both MEGA and the new email
stability flow).

Also fixes a test fixture for the comments_enabled setting.
2023-01-20 13:41:36 +01:00
Simon Backx
8a95c62ff1
🐛 Fixed post links being marked as edited when they were not (#16153)
fixes https://github.com/TryGhost/Team/issues/2461

- Ignores 'edited' links when there is only one second differences.
- Make sure we don't set updatedAt when linking a post to a redirect
2023-01-20 13:31:54 +01:00
Simon Backx
6c2af0793c Fixed failing Stripe webhook test
refs acf0baa8c7

Due to the bump in express-test, we now handle string bodies 'properly'. So they now pass all the Express middlewares. In the past this failing test did not really pass by the bodyParser.raw middleware,
so the content-type check on the `bodyParser.raw({type: 'application/json'})` middleware was not executed. Now it is, and the test fails because the content-type header was not set to application/json.
2023-01-18 12:05:55 +01:00
Daniel Lockyer
9ba251238a Added Content-Version header to all API requests
refs https://github.com/TryGhost/Team/issues/2400

- we've deemed it useful to start to return `Content-Version` for all
  API requests, because it becomes useful to know which version of Ghost
  a response has come from in logs
- this should also help us detect Admin<->Ghost API mismatches, which
  was the cause of a bug recently (ref'd issue)
2023-01-18 08:38:07 +01:00
Daniel Lockyer
85051199e3
Added email snapshots for API versioning tests (#16139) 2023-01-18 08:09:28 +01:00
Naz
1f6b2b4d5e
Added email content testing capabilities
refs https://github.com/TryGhost/Toolbox/issues/499

- Outgoing emails have been a weak point of Ghost's stability recently. The concept of "emailMockReceiver" similarly to "webhookMockReceiver", allows to test side-effects like outgoing emails.
- This is a first iteration which should lay groundwork for testing all outgoing emails in the future
- The change adds a new concept of "email mock receiver" which is very similar to how the "webhook mock receiver" works. The email mock receiver exposes two methods to record and verify snapshots:
- matchHTMLSnapshot - records and verifies only the HTML content of the outgoint email
- matchMetadataSnapshot - records and verifies all the non-HTML properties sent along an email content, e.g.: to address, plaintext, subject, etc.
- What's missing is matching content based on dynamic content like dates, links with JWT tokens, etc.
2023-01-17 12:50:49 +08:00
Fabien 'egg' O'Carroll
14f282d640
Improved newsletter delivery rates and email suppression handling
This introduces the new suppressions feature which will automatically
unsubscribe members from newsletters when their email is added to the
suppression list in Mailgun, this is usually due to emails either
permanently bouncing to the address, or the member making a spam
complaint.

Both Members and Admins are able to see that the email has been added to
the list, and Members are be able to request their email be removed from
the list via Portal.

Overall this feature should improve delivery rates of newsletters and
improve the rating of the domain you're sending from.
2023-01-13 15:46:57 +07:00
Ronald Langeveld
0e2d1b3afd
Added redeemed offers filtering for members (#16071)
closes https://github.com/TryGhost/Team/issues/2011

- Gives publishers the ability to filter members based on which offer they used (redeemed) when they subscribed for a paid membership.
- On the offers page, the redemption count number links to a the members page with the filter already applied making it easy to have insight on which members used the offer / coupon.
2023-01-11 20:13:09 +08:00
Simon Backx
26635f192f Added visible theme errors in admin
fixes https://github.com/TryGhost/Team/issues/2393
2023-01-06 15:51:34 +01:00
Simon Backx
7b3712a15b
Added visible theme errors in admin (#16081)
refs https://github.com/TryGhost/Team/issues/2393

- During boot and loading the active theme, we now cache the result of
the gscan validation. Cache configuration can happen in
`adapters.cache.gscan`
- We now also return non-fatal errors when activating or adding a theme.
- When the `themeErrorsNotification` feature flag is on, we fetch the
active theme (which includes the validation information) when loading
admin
- If the currently active theme has errors, we show an error
notification that can open the error modal
- Added a new endpoint: `/ghost/api/admin/themes/active/` that returns
the result of the last gscan validation of the active theme. If no cache
is available, it will run a new gscan validation.
- Added new permissions for the active action/endpoint (author, editor,
administrator)
2023-01-06 13:44:27 +01:00
Fabien "egg" O'Carroll
2d9114450c Fixed email_recipients fixtures for tests
The email_recipient fixtures were using duplicate and mismatched email addresses
rather than having them correctly map to the Members, which is required for testing
email suppressions.
2023-01-05 17:11:06 +07:00
Simon Backx
913ad18b71
Added DomainEvents.allSettled utility method (#16075)
no issue

With the increased usage of DomainEvents, it gets harder to build
reliable tests without having to resort to timeouts. This utility method
allows us to wait for all events to be processed before continuing with
the test.

This change should speed up tests and make them more reliable.

It only adds extra code when running tests and shouldn't impact
production.
2023-01-04 14:30:35 +01:00
Simon Backx
819d0d884c
Improved email verification required checks (#16060)
fixes https://github.com/TryGhost/Team/issues/2366
refs https://ghost.slack.com/archives/C02G9E68C/p1670232405014209

Probem described in issue.

In the old MEGA flow:
- The `email_verification_required` check is now repeated inside the job

In the new email service flow:
- The `email_verification_required` is now checked (didn't happen
before)
- When generating the email batch recipients, we only include members
that were created before the email was created. That way it is
impossible to avoid limit checks by inserting new members between
creating an email and sending an email.
- We don't need to repeat the check inside the job because of the above
changes

Improved handling of large imports:
- When checking `email_verification_required`, we now also check if the
import threshold is reached (a new method is introduced in
vertificationTrigger specifically for this usage). If it is, we start
the verification progress. This is required for long running imports
that only check the verification threshold at the very end.
- This change increases the concurrency of fastq to 3 (refs
https://ghost.slack.com/archives/C02G9E68C/p1670232405014209). So when
running a long import, it is now possible to send emails without having
to wait for the import. Above change makes sure it is not possible to
get around the verification limits.

Refactoring:
- Removed the need to use `updateVerificationTrigger` by making
thresholds getters instead of fixed variables.
- Improved awaiting of members import job in regression test
2023-01-04 11:22:12 +01:00
Simon Backx
789e2c96c0
🐛 Fixed SingleUseTokens being cleared on boot (#15999)
fixes https://github.com/TryGhost/Team/issues/1996

**Issue**
Our Magic links are valid for 24 hours. After first usage, the token
lives for a further 10 minutes, so that in the case of email servers or
clients that "visit" links, the token can still be used.

The implementation of the 10 minute window uses setTimeout, meaning if
the process is interrupted, the 10 minute window is ignored completely,
and the token will continue to live for the remainder of it's 24 hour
validity period. To prevent that, the tokens are cleared on boot at the
moment.

**Solution**

To remove the boot clearing logic, we need to make sure the tokens are
only valid for 10 minutes after first use even during restarts.

This commit adds 3 new fields to the SingleUseToken model:
- updated_at: for storing the last time the token was changed/used). Not
really used atm.
- first_used_at: for storing the first time the token was used
- used_count: for storing the number of times the token has been used

Using these fields:
- A token can only be used 3 times
- A token is only valid for 10 minutes after first use, even if the
server restarts in between
- A token is only valid for 24 hours after creation (not changed)

We now also delete expired tokens in a separate job instead of on boot /
in a timeout.
2023-01-04 09:49:39 +01:00
Elena Baidakova
235446b034
🐛 Fixed tiers order by monthly_price (#16013)
refs TryGhost/Team#2273
- According to documentation (https://ghost.org/docs/content-api/#order)
tiers should be ordered from lowest to highest monthly price.
2023-01-03 12:59:18 +04:00
Simon Backx
f568b35f26 Fixed 2023 snapshot tests
no issue

Updated the snapshot files for the email preview test snapshots that contain 2023 in the email footer.
2023-01-02 11:18:07 +01:00
Simon Backx
0825a2d7f4
🐛 Fixed importing existing member resetting newsletters (#16017)
fixes https://github.com/TryGhost/Team/issues/2386

**Issue:**
- When trying to import a member that already exists, and has
'subscribed' set to 'true' in the CSV, the newsletters the member is
subscribed to are reset to the default newsletters.
- When ediging a member with the API and setting `subscribed` to true,
the same happens.

**Cause:**
A faulty check for the `status` property of a newsletter.

Fixed and added a new E2E test.
2022-12-15 17:45:11 +01:00
Elena Baidakova
9579791185
Resubscribe member to default newsletters after removing from suppression list (#15974)
refs TryGhost/Team#2372
2022-12-13 13:31:50 +04:00
Rishabh
ce45571dc0 Updated email faqs in portal to show sender email address
refs https://github.com/TryGhost/Team/issues/2348

- updates email faqs to show the sender email from newsletter data
2022-12-12 15:09:32 +05:30
Elena Baidakova
62681037f0
Added email_suppression property to response for PUT api/member method (#15946)
closes TryGhost/Team#2359
- Keep the same response structure for `api/member` GET and PUT methods
2022-12-06 13:51:26 +04:00
Naz
879aad263d
Added email verification trigger test
refs https://github.com/TryGhost/Toolbox/issues/476

- The email verification trigger and host settings related bugs have been a cause of bugs in past releases.  The admin client verification source did not have any test coverage in the past.
- The members test suite size is getting out of hand. This test is quite verbose, because of the state it's trying to check.
- In the future we should consider splitting up Member API (and probably other) test suites into smaller pieces.
2022-12-05 18:24:28 +07:00
Naz
e170f293e3
Extracted sleep method to e2e framework module
no issue

- The sleep method has been used in 8 modules reimplementing the same thing over and over again. It's usually a sign of async event processing outside of the request/response loop. It's good to have a single point of implementation for a "hack" like this, so we could track it easier and address the even processing delay in a more optimal way centrally if it ever becomes a bottleneck
2022-12-05 17:26:29 +07:00
Simon Backx
2a3a72e4d5
Added email debug API (#15915)
fixes https://github.com/TryGhost/Team/issues/2346

- Adds email batch browse endpoint
- Adds email recipient failures browse endpoint
- Adds new fixtures and E2E tests for the new API
- Added support for snapshot tests to have 'nullable' types.
2022-12-02 09:44:32 +01:00
Simon Backx
4c166e11df
Added E2E tests for batch sending (#15910)
refs https://github.com/TryGhost/Team/issues/2339

- Includes a new pattern in the job manager that allows us to properly
await jobs.
- Added new convenience mocking methods to stub settings
- Tests the main flows for bulk sending:
    - Sending in multiple batches
    - Sending to multiple segments
    - Handling a failed batch and retrying that batch
- Fixes bug in batch generation (ordering not working)

In a different PR I'll add more detailed tests.
2022-12-01 13:43:49 +01:00
Naz
80decf8303
Fixed invalid query strings in tests
closes https://github.com/TryGhost/Team/issues/2324

- It seemed like the "limit" query parameter did not work properly returning multiple entries from the endpoint. In reality the whole query string was ignored because of an error in the "filter" part of the query ^_^
2022-11-30 13:31:06 +07:00
Elena Baidakova
1b784b5ec5
Added events to Post Analytics page (#15886)
closes TryGhost/Team#2313
- Added Sent event to Post analytics and Members feed. Now post can be
Sent or Received or Bounced.
- Excluded Delivered event from Sent filter on backend.
2022-11-28 17:43:35 +04:00
Naz
22738b1b50 🔒 Disabled editable relations by default
refs https://github.com/TryGhost/Ghost/security/advisories/GHSA-9gh8-wp53-ccc6
refs https://github.com/TryGhost/Toolbox/issues/465

- Bookshelf relations allows us to edit relational records by default, which was used liberally in the codebase.
- Not having a clear track record of editable relations left the model layer prone to triggering unwanted nested saves and created a vulnerability where members were able to edit newsletter settings.
- With explicit editable relations it's easier to keep track of relations having editable access to related records. Makes the relational data modification pattern safer to use too.
- Anyone running 5.x should update to 5.24.1

Credits: Dave McDaniel and other members of [Cisco Talos](https://talosintelligence.com/vulnerability_reports)
2022-11-28 18:39:39 +07:00
Elena Baidakova
8d9d22e5a7
Added member API for removing email from suppression list (#15867)
closes TryGhost/Team#2306
2022-11-23 14:41:00 +04:00
Simon Backx
4b4592630f
Added new email batch sending service (#15865)
fixes https://github.com/TryGhost/Team/issues/2284

New batch sending flow (still WIP). Logs the sent emails instead of actually sending them. Unit tests are coming in later commits.
2022-11-23 11:33:44 +01:00
Rishabh
5780fc2a93 Added new source and source type columns to emails table
refs https://github.com/TryGhost/Team/issues/2280

We are moving away from storing html and plaintext on email and instead will store the email data in source and source_type columns which allows us to store the email in other formats like mobiledoc and lexical. Storing in those formats allows greater flexibility for later html generation

- adds new `source` column that stores `mobiledoc`/`lexical`/`html` data for a newsletter
- adds new `source_type` column that stores one of `mobiledoc`/`lexical`/`html` to identify type of source
2022-11-23 15:04:11 +05:30
Simon Backx
44f189b56a
Added email service package (#15849)
fixes https://github.com/TryGhost/Team/issues/2282

Added a new email service package that is used when the email stability
flag is enabled. Currently not yet implemented so will throw an error
for all entry points (if flag enabled).

Removed usage of `labs.isSet.bind` across the code, because that breaks
the stubbing of labs by `mockManager.mockLabsEnabled` and
`mockManager.mockLabsDisabled`. `flag => labs.isSet(flag)` should be
used instead.

All email depending tests now disable the `emailStability` feature flag
to keep the tests passing + make sure we still run all the tests for the
old flow while the email stability package is being built.
2022-11-21 10:29:53 +01:00
Simon Backx
2220686113 Sped up comments and feedback tests by reusing Ghost app instance
refs https://ghost.slack.com/archives/C02G9E68C/p1668777066462859

Added a duplicate method to the mmebers agent, so we can reuse the same app instance and options, whenever we need multiple agents (each with their own cookies).
2022-11-18 14:58:35 +01:00
Fabien 'egg' O'Carroll
8283de99c8
Wired up EmailSuppressionList to Members Admin API (#15848)
refs https://github.com/TryGhost/Team/issues/2268

The approach of using the service to lead email suppression data as
opposed to bookshelf relations allows us to wire things up without
having implemented the database. The getBulkSuppressionData allows us to
do this without much of a DB performance hit.
2022-11-18 16:28:13 +07:00
Ronald Langeveld
08232bd538
Cleaned up GA Flags (#15762)
closes https://github.com/TryGhost/Team/issues/2126

- Cleaned up the following GA flags: `newsletterPaywall`, `freeTrial`, `compExpiring`, `searchHelper`, `emailAlerts`, `fixNewsletterLinks`.
2022-11-18 16:09:25 +07:00
Ronald Langeveld
e0787b4e83
Added specific newsletter support for bulk unsubscribes (#15742)
closes https://github.com/TryGhost/Team/issues/2013

Added support to bulk unsubscribe a selected (filtered) list on members from specific, selected newsletters.
2022-11-16 14:29:00 +07:00
Sodbileg Gansukh
78bff39c23
Update the cover image in default fixtures (#15817)
- updated the cover image to be simpler
- made the change in text fixtures as well, just to keep the fixtures in sync

Co-authored-by: Hannah Wolfe <github.erisds@gmail.com>
2022-11-15 21:19:50 +00:00
Simon Backx
cdccf5c46a
Reduced default post relations (#15798)
fixes https://github.com/TryGhost/Team/issues/2085

Don't load relations we don't need anymore for the posts table. And
reload the individual post when we open the analytics page with more
relations that we actually need.
2022-11-15 10:17:26 +01:00
Simon Backx
f4660b440b
Fixed sending emails when using MembersAgent.loginAs (#15791)
fixes https://github.com/TryGhost/Team/issues/1903

MembersAgent.loginAs sends email, asynchronously via events. Which
conflicts with tests that also test emails. We cannot properly await
these events, so this is currently fixed with a timeout of 200ms. But
this was too random and unreliable.
2022-11-09 16:25:42 +01:00
Ronald Langeveld
6214812ac0
Added e2e test for bulk member deletion (#15730)
ref https://github.com/TryGhost/Team/issues/1446

- tests the `delete` api endpoint method for deleting a bulk set of members.
2022-11-08 17:02:55 +07:00
Ghost CI
4b7dd71851 Merged heads/v5.22.9 into main 2022-11-08 09:04:04 +00:00
Fabien "egg" O'Carroll
69aa52bd8e 🐛 Handled deleted Stripe objects in the Stripe Checkout flow
closes https://github.com/TryGhost/Team/issues/2222

Whilst we were checking for Stripe objects being active, we were not
checking for them existing in Stripe. This adds handling to all read
request to Stripe in the payment link flow, so that we can gracefully
handle deleted objects.

We've also included an automated test which fails without this fix.

We've also improved the query to find Stripe Prices which will result
in less request to the Stripe API to check if it is valid.
2022-11-08 16:03:07 +07:00
Simon Backx
231ceedc72
Readded activity feed pagination tests (#15776)
fixes https://github.com/TryGhost/Team/issues/2218

The test should run a lot faster now.
2022-11-07 15:08:56 +01:00
e.baidakova
81e6a7c5bd 🐛 Add ability to cache comments count endpoint
closes TryGhost/Team#2094
- Comment counts request was changed from `post` to `get` to enable request caching.
2022-11-02 10:54:12 +04:00
Fabien 'egg' O'Carroll
1ff1b75a69
🐛 Fixed errors with Stripe Checkout (#15749)
refs https://github.com/TryGhost/Ghost/commit/1f300fb781f0

The full customer object was not being passed to the StripeAPI service
when it already exists, this was resulting in inconsistent behaviour when
sending the customerEmail param to the API, causing `invalid_email`
errors to be thrown from Stripe and breaking the checkout.
2022-11-02 02:18:30 +07:00
Fabien "egg" O'Carroll
ba41f308c7 🐛 Fixed upgrading to a paid plan
closes https://github.com/TryGhost/Team/issues/2196

We were incorrectly assuming that all requests would have the
`customerEmail` passed in the body. Instead we were incorrectly
passing `undefined` or `''` as the `customerEmail` property to stripe,
which resulted in a validation error.

We've updated the code to pass `null` in the case of a falsy value,
which the Stripe API handles without error.
2022-11-02 00:11:33 +07:00
Fabien "egg" O'Carroll
1f300fb781 🐛 Fixed checkout sessions when using Offers
closes https://github.com/TryGhost/Team/issues/2195

The issue here is two-fold, and specific to using Offers so was not
caught by any automated tests. First, we were incorrectly comparing
the tier.id to the offer.tier.id - this is because the Tier objects id
property is an instance of ObjectID rather than a string.

Secondly we were passing through the cadence parameter from the
request body, but when using Offers this is not including in the
request, so we must pull the data off of the Offer object instead and
pass that to the payments service.
2022-11-02 00:11:33 +07:00
Fabien 'egg' O'Carroll
a0ebb9a6f3
🐛 Fixed Tier description not being set (#15741)
closes https://github.com/TryGhost/Ghost/issues/15740

The validation function for a Tier description was not returning the
validated value, which meant we were unable to set the Tier
description.
2022-11-01 16:25:55 +07:00
Hannah Wolfe
5fe6687c2a
Skipped slow activity feed tests temporarily
- These tests are very slow, and make the build fail about 2/3 times
- Temporarily skipping until we can fix, as I want to get all our outstanding hacktoberfest PRs merged
2022-10-30 14:52:30 +00:00
Simon Backx
13fd64ebf7 Fixed tests for flag bumps and new setting
no issue
2022-10-27 18:26:46 +02:00
Simon Backx
b916300ceb
Added aggregated click events (#15713)
fixes https://github.com/TryGhost/Team/issues/2175

- New event type `aggregated_click_event` that is disabled by default in all the existing activity feeds
- This returns click events, but only the first click events for each member/post combination.
- It includes the total count of unique link clicks for that member on that post combination
- Had to resort to some custom knex queries to make this work easily
- Requires `@tryghost/bookshelf-pagination@0.1.31`, included in `@tryghost/bookshelf-plugins@0.6.1` (this fixes an issue with custom selects breaking the total count query of pages)
- Went a bit overboard with the pagination tests to cover as much unknown edge cases as possible
2022-10-27 17:23:45 +02:00
Simon Backx
b911208b41
Improved filter support in activity API to allow pagination (#15684)
fixes https://github.com/TryGhost/Team/issues/2129

- This changes how the activity feed API parses the filter.
- We now parse the filter early to a MongoDB filter, and split it in two. One of the filters is applied to the pageActions, and the other one is used individually for every event type. We now allow to use grouping and OR's inside the filters because of this change. As long as we don't combine filters on 'type' with other filters inside grouped filters or OR, then it is allowed.
- We make use of mongoTransformer to manually inject a mongo filter without needing to parse it from a string value again (that would make it a lot harder because we would have to convert the splitted filter back to a string and we currently don't have methods for that).
- Added sorting by id for events with the same timestamp (required for reliable pagination)
- Added id to each event (required for pagination)
- Added more tests for filters
- Added test for pagination
- Removed unsued getSubscriptions and getVolume methods

Used new mongo utility methods introduced here: https://github.com/TryGhost/NQL/pull/49
2022-10-27 12:13:24 +02:00
Simon Backx
076e3c02b2
Added linking between member and subscription created events (#15693)
fixes https://github.com/TryGhost/Team/issues/2160

- Adds a `batch_id` to both events that contain the same ID if they were created at the same time.
- Removes duplicate signup/conversion events using the batch_id
- Requires an update in mongo-knex to work (refs https://ghost.slack.com/archives/C02G9E68C/p1666773313272409?thread_ts=1666767872.375009&cid=C02G9E68C)
- Some dependencies needed an update to load the latest mongo-knex
- Added tiers to membersUtils, loaded on startup (we can start to use this instead of fetching it every time)
2022-10-27 11:44:19 +02:00
Naz
6bd2769146
Hardened tiers-related snapshot tests
refs https://github.com/TryGhost/Team/issues/2077

- Members and Posts test suites were using a broad tiers property matcher, which is an anti-pattern for snapshot tests. Without more specific snapshots it would be very hard to track down tier-related breaking changes!
- This change is groundwork for a refactor coming in tier usage at API's output serializers
2022-10-27 15:54:28 +08:00
Elena Baidakova
e3ab868b83
Added email feedback column (#15698)
closes TryGhost/Team#2159
- Added column to email table
- Hide the feedback tab on frontend depending on the column value

Co-authored-by: Daniel Lockyer <daniellockyer@fastmail.com>
2022-10-27 11:22:50 +04:00
Elena Baidakova
57817eefc8
Added feedback buttons to emails (#15695)
closes TryGhost/Team#2075
2022-10-26 19:19:05 +04:00
Fabien "egg" O'Carroll
25d8d694a0 Wired up the payment service to create stripe checkout sessions
refs https://github.com/TryGhost/Team/issues/2078
2022-10-25 09:03:04 +07:00
Fabien "egg" O'Carroll
31610f9b94 Wired up Tiers service to Tiers Content & Admin API
refs https://github.com/TryGhost/Team/issues/2078
2022-10-25 09:02:59 +07:00
Simon Backx
fd91f7eebb
Added email sent events (#15682)
fixes https://github.com/TryGhost/Team/issues/2137

For the analytics page, we need the sent events to show up immediately
after sending an email. Otherwise we need to wait for emails to be
marked as received (which takes too long) before being able to show them
on the analytics page.

This adds the email_sent_event, which is hidden by default everywhere
and used on the analytics page.
2022-10-24 11:11:44 +02:00
Elena Baidakova
9720459f55
Added pie chart for activity feed (#15673)
closes TryGhost/Team#2088
- Added pie chart to feedback event
- Added `negative_feedback` field to response from BE
2022-10-21 21:34:05 +04:00
Rishabh Garg
05330316e3
Allowed fixing newsletter links (#15672)
refs https://github.com/TryGhost/Team/issues/2116

- allows site owners to edit a link in a post that has already been sent out, fixing any typos or other mistakes
- resets click counter for the edited link back to 0 so site owners can see the clicks on new link, doesn't change the overall click count
2022-10-21 19:38:57 +05:30
Naz
b552727b69
Changed members export to contain tiers
refs https://github.com/TryGhost/Team/issues/1076

- The "products" is a legacy term for what is now "tiers" since multiple tiers feature introduction in https://github.com/TryGhost/Ghost/releases/tag/v4.39.0
- Note, the "tiers" is a field meant for informational purposes and cannot be imported back into Ghost site.
2022-10-21 18:58:59 +08:00
Rishabh
ba6fe63250 Fixed cached redirects for edited newsletter links
refs https://github.com/TryGhost/Team/issues/2135

The email link redirects on Pro are cached as 302 redirects in Varnish, so we're missing further clicks after the first one for each member, until the cache is invalidated. This change invalidates cache on link edits to ensure that we correctly redirect members to updated link everytime
2022-10-21 16:07:34 +05:30
Rishabh
d7b3721036 Added edited property to links api
refs https://github.com/TryGhost/Team/issues/2104

- adds a boolean `edited` property to links api that denotes if the link has been edited
2022-10-20 18:17:44 +05:30
Rishabh
d8bacf12d1 Added endpoint for fixing newsletter links
refs https://github.com/TryGhost/Team/issues/2104

- adds new bulk edit endpoint for links, updates all matching link with the current redirect url and update to new url
2022-10-20 17:50:02 +05:30
Simon Backx
adf10f0e76
Added support for filtering email events by post_id (#15666)
refs https://github.com/TryGhost/Team/issues/2093
2022-10-20 13:29:00 +02:00
Simon Backx
6380b82793
Added sentiment ordering and include for posts (#15657)
fixes https://github.com/TryGhost/Team/issues/2090

- This changes how sentiment is exposed in the API. Now it is exposed as a `sentiment` relation, directly on the model (no longer in counts). Internally we still use `count.sentiment`.
- Content API users (and themes) can include the 'sentiment' relation and order by sentiment.
- Updated Admin to use sentiment instead of count.sentiment
2022-10-19 16:50:58 +02:00
Simon Backx
a01fb5f1aa
Added post_id filter and total to activity feed API (#15650)
fixes https://github.com/TryGhost/Team/issues/2091
fixes https://github.com/TryGhost/Team/issues/2089

- Added new fixtures to make testing easier for the activity feed
- Improved E2E test coverage of activity feed with separate test file
- Added data.post_id filter to enable filtering by events related to a
given post
- Fixed return types in JSDoc of test agents (TypeScript interprets
these as `typeof Agent` if we don't add `InstanceType<Agent>`)
- Added total pagination metadata to activity feed API (to allow a basic
type of pagination using filters)
2022-10-18 15:52:04 +02:00
Simon Backx
22fe1c01de
Added conversions count and separate analytics page (#15637)
fixes https://github.com/TryGhost/Team/issues/2084

- When audience feedback is enabled, we use a single 'conversions' count instead of having separate ones for signups and paid conversions.
- The analytics component is separated so we can change it without breaking the existing page.
2022-10-17 13:02:39 +02:00
Elena Baidakova
e831be6bc2
Added the feedback buttons in the emails (#15619)
closes TryGhost/Team#2046
closes TryGhost/Team#2045
- Added feedback buttons markup.
- Added feedback links generation.
2022-10-14 18:12:17 +04:00
Fabien 'egg' O'Carroll
bd0f4b4b8c
Added Tier price and currency data to products table (#15366)
refs https://github.com/TryGhost/Team/issues/1765

In order to better handle deleted objects in Stripe we want to decouple
Members from Stripe.

These changes allow us to have the Tier concept completely independent
of the Stripe tables, such that the Stripe data can be generated as/when
it's needed - which will help to protect against missing data.
2022-10-14 06:40:17 +01:00
jbenezech
75811f35bc
🐛 Fixed embedded cards for non-utf8 content (#15578)
closes: https://github.com/TryGhost/Ghost/issues/14973

- When fetching content using a non-standard charset, characters were notproperly decoded to utf-8 resulting in mangled text in the editor -> Detect charset and use iconv to decode the page text

- When requesting a non bookmark card, if no oembed data could be foundand we fallback to bookmark, a second network request to fetch the content was issued. This seemed unnecessary -> refactored to avoid that
2022-10-13 12:19:47 +01:00
Daniel Lockyer
e2ba19b0df
Fixed import of bson-objectid in accordance to the typings
- there's a few different ways we can import it but I've chosen to
  append `.default` as we have done in several other places in the code
2022-10-12 14:54:35 +07:00
Simon Backx
68bdc1afea
Added post sentiment (#15592)
fixes https://github.com/TryGhost/Team/issues/2054

This change adds the sentiment and positive_feedback counts to the posts models. This change isn't really ideal because there are some problems here:
- sentiment isn't really a count
- we don't need to include the sentiment and positive_feedback as a default for posts (but the same is true for attribution)

It would make sense to move this to separate endpoints that only fetch the analytics for a given post when the analytics page is opened. But for our initial skateboard version of audience feedback this should be a good start to already see the data.
2022-10-11 17:52:14 +02:00
Simon Backx
e540344ef2
Added audience feedback service and storage (#15584)
fixes https://github.com/TryGhost/Team/issues/2049
fixes https://github.com/TryGhost/Team/issues/2053

- This adds a new audience feedback package to Ghost. 
- A new members API to give feedback on posts using the `/api/feedback` endpoint.
- Added a new authentication middleware that supports both uuid-based and session based authentication.
2022-10-11 16:32:28 +02:00
Elena Baidakova
1221ba5d1d
Added feedback_enabled to newsletters table (#15589)
closes TryGhost/Team#2042
- Added ability to enable audience feedback per newsletter (just on BE side).
2022-10-11 16:06:26 +04:00
Naz
714e108d40
Fixed typo 2022-10-11 17:24:11 +08:00
Naz
9b34bd70a2
Added test coverage for Subscription edit API
refs https://github.com/TryGhost/Team/issues/2047

- We anticipate upcoming changes in the PUT /members/:id/subscriptions/:subscription_id endpoint , so covered it with a snapshot test to track the differences more precisely.
- Note, the test case contains a more explicit outgoing HTTP request mocking.
2022-10-11 17:24:00 +08:00
Simon Backx
7e3b41f643
Removed emailClicks feature flag (#15556)
fixes https://github.com/TryGhost/Team/issues/2028

Since link clicks became GA, some older components and templates are no longer used.
2022-10-07 14:27:57 +02:00
Simon Backx
d4540012dc Added tests for click events in the activity feed
fixes https://github.com/TryGhost/Team/issues/2018

- Includes new test fixtures for redirects and click events
- Tests if post, and links are returned in the click events
2022-10-06 11:43:39 +02:00
Daniel Lockyer
c4981a71a2
Merged v5.17.2 into main
v5.17.2
2022-10-05 18:33:12 +07:00
Simon Backx
41a0945592
🐛 Prevented member creation when logging in (#15526)
fixes https://github.com/TryGhost/Ghost/issues/14508

This change requires the frontend to send an explicit `emailType` when sending a magic link. We default to `subscribe` (`signin` for invite only sites) for now to remain compatible with the existing behaviour.

**Problem:**
When a member tries to login and that member doesn't exist, we created a new member in the past.

- This caused the creation of duplicate accounts when members were guessing the email address they used.
- This caused the creation of new accounts when using an old impersonation token, login link or email change link that was sent before member deletion.

**Fixed:**
- Trying to login with an email address that doesn't exist will throw an error now.
- Added new and separate rate limiting to login (to prevent user enumeration). This rate limiting has a higher default limit of 8. I think it needs a higher default limit (because it is rate limited on every call instead of per email address. And it should be configurable independent from administrator rate limiting. It also needs a lower lifetime value because it is never reset.
- Updated error responses in the `sendMagicLink` endpoint to use the default error encoding middleware.
- The type (`signin`, `signup`, `updateEmail` or `subscribe`) is now stored in the magic link. This is used to prevent signups with a sign in token.

**Notes:**
- Between tests, we truncate the database, but this is not enough for the rate limits to be truly reset. I had to add a method to the spam prevention service to reset all the instances between tests. Not resetting them caused random failures because every login in every test was hitting those spam prevention middlewares and somehow left a trace of that in those instances (even when the brute table is reset). Maybe those instances were doing some in memory caching.
2022-10-05 18:11:06 +07:00
Simon Backx
e7378520a0
🔒 Prevented member creation when logging in (#15526)
fixes https://github.com/TryGhost/Ghost/issues/14508

This change requires the frontend to send an explicit `emailType` when sending a magic link. We default to `subscribe` (`signin` for invite only sites) for now to remain compatible with the existing behaviour.

**Problem:**
When a member tries to login and that member doesn't exist, we created a new member in the past.

- This caused the creation of duplicate accounts when members were guessing the email address they used.
- This caused the creation of new accounts when using an old impersonation token, login link or email change link that was sent before member deletion.

**Fixed:**
- Trying to login with an email address that doesn't exist will throw an error now.
- Added new and separate rate limiting to login (to prevent user enumeration). This rate limiting has a higher default limit of 8. I think it needs a higher default limit (because it is rate limited on every call instead of per email address. And it should be configurable independent from administrator rate limiting. It also needs a lower lifetime value because it is never reset.
- Updated error responses in the `sendMagicLink` endpoint to use the default error encoding middleware.
- The type (`signin`, `signup`, `updateEmail` or `subscribe`) is now stored in the magic link. This is used to prevent signups with a sign in token.

**Notes:**
- Between tests, we truncate the database, but this is not enough for the rate limits to be truly reset. I had to add a method to the spam prevention service to reset all the instances between tests. Not resetting them caused random failures because every login in every test was hitting those spam prevention middlewares and somehow left a trace of that in those instances (even when the brute table is reset). Maybe those instances were doing some in memory caching.
2022-10-05 12:42:42 +02:00
Naz
0bf6268091
Updated content-length header matchers
no issue

- All content-length snapshots should be using the same matcher for consistency - anyContentLength. It's more explicit about what the matcher is all about and might be useful to have content-length matchers in one place if it ever changes (the header value should be a damn digit after all, not a string!) (ref. https://www.rfc-editor.org/rfc/rfc7230#section-3.3.2)
2022-10-05 17:34:17 +08:00
Fabien "egg" O'Carroll
28de1720c1 🔒 Fixed magic link endpoint sending multiple emails
refs https://github.com/TryGhost/Team/issues/2024

Without validation it was possible to send a string of comma separated
email addresses to the endpoint, and an email would be sent to each
address, bypassing any rate limiting.

This bug does not allow for an authentication bypass exploit. It is purely a
spam email concern.

Credit: Sandip Maity <maitysandip925@gmail.com>
2022-10-05 10:28:13 +01:00
Rishabh Garg
e3600d70ef
Added referrer attribution from request context (#15499)
closes TryGhost/Team#2007

- uses request context to add referrer source and medium for a new member
- uses integration name as referrer medium if exists
2022-09-29 22:31:48 +05:30
Simon Backx
648811690a Added email click tracking
no issue

Bumped flag to GA.
2022-09-29 18:14:15 +02:00
Simon Backx
0cd0fc838d
Added email track clicks column and cleaned up frontend checks (#15501)
fixes https://github.com/TryGhost/Team/issues/2008

- New column that stores email click tracking at the time it was created
- Improved frontend side checks for when to show analytics
2022-09-29 16:42:45 +02:00
Naz
8cbf913582 Increased Vary granularity for versioned requests
refs https://github.com/TryGhost/Toolbox/issues/425
refs https://github.com/TryGhost/Toolbox/issues/280

- The versioned API responses vary based on requested version (passed in request's 'accept-version' header). shared caches that sit between Ghost's origin server and the browser would be putting responses with same Vary into the same caching bucket, which is incorrect.
- This change makes response's Vary more granular and tells caching mechanisms to take 'Accept-Version' request header into account when caching.
- Informative read on the topic - https://www.fastly.com/blog/getting-most-out-vary-fastly
2022-09-28 14:48:43 +08:00
Rishabh Garg
31733657a6
Updated naming for referrer attribution (#15486)
- renames `refSource`, `refMedium` and `refUrl` to `referrerSource`, `referrerMedium` and `referrerUrl` respectively for consistent naming across files and usages
2022-09-28 00:58:06 +05:30
Kevin Ansfield
89d4e3daf9
Updated error messages for invalid mobiledoc+lexical post/page API requests (#15477)
closes https://github.com/TryGhost/Team/issues/1896

- updated message to be clearer, added context and help
2022-09-27 10:30:28 +01:00
Simon Backx
281cd2e7a3
Renamed count.conversions to count.paid_conversions in posts (#15460)
fixes https://github.com/TryGhost/Team/issues/1943
2022-09-26 14:25:27 +02:00
Naz
4528cba1b9 Changed Content API caching to public
refs https://github.com/TryGhost/Toolbox/issues/410

- Private cache control was preventing browser or shared caches from storing Content APIs response. The type of data served through the Content API is very much of a "public" nature, so should be cacheable.
- Right now the 'max-age' value of 'cache-control' header is hardcoded to '0', without 'must-revalidate' value, to allow browsers to cache content slightly more aggressively. In the future the 'max-age' value will most-likely become configurable to allow even more aggressive HTTP caching.
2022-09-26 14:54:50 +08:00
Naz
7b009bf1fe Enabled shared caching of 404 error responses
refs https://github.com/TryGhost/Toolbox/issues/410

- The 'private' value in 'Cache-Control' response header for all errors made it impossible for shared caches (e.g.: Fastly, Cloudflare) to cache 404 responses efficiently.
- The change substitutes 'max-age=0' which should not effect the browser cache behavior but would allow shared caches to process such requests efficiently.
- A more loose caching logic only applies to 404 responses from GET requests that are not user-specific (non-authenticated, non-cookie containing requests)
2022-09-26 14:54:50 +08:00
Rishabh
6c85c75b86 Added referrer attribution data to member api
refs https://github.com/TryGhost/Team/issues/1961

- includes referrer source and medium information in member api
2022-09-24 17:46:57 +05:30
Rishabh Garg
b048b02f67
Added new referrer source stats API (#15449)
closes https://github.com/TryGhost/Team/issues/1939

- adds new endpoint that returns count of referrer sources by date for admin dashboard
2022-09-22 16:34:26 +05:30
Simon Backx
7437d92d50
Added post referrers stats API (#15448)
closes https://github.com/TryGhost/Team/issues/1942

- Added data fixtures for referrers
- Added new endpoint to fetch referrer stats for a given post: `/stats/referrers/posts/:id`
- Added new ReferrersStatsService, responsible for calculating referrer stats
2022-09-21 18:16:56 +02:00
Rishabh Garg
b99c5428d0
Added referrer attribution columns to events table (#15436)
refs TryGhost/Team#1931

- referrer source, medium and url will be stored in the events table along with rest of attribution data
- stores referrer information on two tables
  - `members_created_events` for signups
  - `members_subscription_created_events` for paid conversions
2022-09-21 19:01:36 +05:30
Simon Backx
b8041f0a60
Added clicks to activity feed (#15439)
closes https://github.com/TryGhost/Team/issues/1933

- Added click_events to activity feed
- Added support for parsing click_events in the frontend
- Moved url parsing (transform ready) to model layer of LinkRedirect
- Moved `getEventTimeline` method to the top of the event repository
- Added description field to parsed events in the frontend (because we need a second line)
- Fixed: member email not returned in comment_event
2022-09-21 10:25:51 +02:00
Simon Backx
63103c2251
Added click counts to posts admin API (#15435)
closes https://github.com/TryGhost/Team/issues/1928
2022-09-20 10:05:41 +02:00
Kevin Ansfield
b2b6be9cb5 Fixed content-length matcher in posts API test snapshot
refs d5f03ec0b1

- underlying error message varies across node versions so the content-length can't be fixed
- applied any-content-length matcher to the right test this time
2022-09-19 16:37:54 +01:00
Kevin Ansfield
1cc8176d4f Allowed any error message length in posts API error test snapshot
refs d5f03ec0b1

- underlying error message varies across node versions so the content-length can't be fixed
2022-09-19 16:22:22 +01:00
Kevin Ansfield
201d4ef228 Loosened error message snapshot matching for posts API test error output
refs d5f03ec0b1

- underlying error message varies across node versions
- adjusted to match only the part we explicitly set
2022-09-19 16:06:34 +01:00
Kevin Ansfield
744534fde6 Updated snapshot for posts API test error output
refs d5f03ec0b1
2022-09-19 15:56:30 +01:00
Aileen Nowak
be45d4ebcf Added post stats service to return total posts in Explore endpoint
no issue

- The explore endpoint needs to expose the total amount of published posts
- To be more consistent, this PR creates a PostStats class which is exposed as `stats` method within the PostService; just like it's done with the MemberService
- Moved existing method to return the date of the most recently published post into the stats service
- Updated the explore service test to reflect the new return property
2022-09-16 13:56:14 +01:00
Kevin Ansfield
3b21d26be7
Wired up creation of post_revisions entries when saving posts with lexical (#15422)
no issue

- added `PostRevsion` model
- duplicated `mobiledoc_revision` creation routine in Post model's onSaving hook to create `post_revision` when model's `lexical` field has changed
- updated `mobiledoc_revision` creation to skip when `lexical` field is populated
2022-09-16 11:59:35 +01:00
Kevin Ansfield
c240f7afa4
Added rendering of posts.lexical to posts.html when saving (#15416)
no issue

- added `@tryghost/kg-lexical-html-renderer` dependency
- added `lexical` lib following the same pattern as our `mobiledoc` lib
- updated the Post model's `onSaving` hook to generate the `html` value from `lexical` when present
2022-09-15 16:49:14 +01:00
Simon Backx
699e67f4e4
Added email_track_clicks setting (#15409)
fixes https://github.com/TryGhost/Team/issues/1900
refs https://github.com/TryGhost/Team/issues/1901

- Defaults to the same value as the current email_track_opens setting for existing installations, otherwise defaults to true
- Had to use a custom migration because the `addSetting` helper doesn't support using an existing setting as current value
- Added a minimal UI to change the setting, but this still needs some design magic 🪄
- Link replacement is disabled if `email_track_clicks` is disabled. In the future we might consider to still do parial additions, such as source attribution and maybe redirects (to discuss).
2022-09-15 15:48:22 +02:00
Simon Backx
d5b332ab02 Added temporary fix for random test failures in comments
refs https://ghost.slack.com/archives/C02G9E68C/p1663162175224299

This requires a better fix in the future that properly awaits the emails (not really possible at the moment) or disables sending new member emails when using loginAs
2022-09-14 17:02:13 +02:00
Kevin Ansfield
a7c4991af5 Wired up lexical editor saving
no issue

- fixed API returning "Invalid mobiledoc structure" errors when `mobiledoc:null` is sent in the payload alongside `lexical: '{...}'`
- updated Admin's `posts` and `pages` adapters to always add `?formats=mobiledoc,lexical` because the API doesn't return `lexical` by default
- added `lexical` attribute to Admin's Post model
- updated `lexical-editor` controller and related components to work with `lexical` always being a JSON string rather than a parsed object
- updated `<KoenigLexicalEditor>` to pass through the lexical state string as initial state and wired up the `onChange` prop
2022-09-13 21:01:53 +01:00
Kevin Ansfield
6fc9cd5f80
Added passthrough + saving of lexical property on posts/pages (#15403)
no issue

- bumped `@tryghost/admin-api-schema` to allow passthrough of the `lexical` property on post and page API endpoints
- prevented saving of blank document in the `mobiledoc` field if `lexical` is provided
- prevented API input containing both `mobiledoc` and `lexical` fields to avoid issues when both are present:
  - not possible to know which content is latest/has precedence
  - not possible to know which editor should be displayed in Admin
2022-09-13 17:29:37 +01:00
Kevin Ansfield
c8dc23cbb5 Fixed Content API posts/pages e2e tests
refs 7ad1be2555

- snapshot comparisons were missing matchers for dynamic fields in the body response
2022-09-13 15:05:53 +01:00
Kevin Ansfield
30611cf2c4 Really fixed e2e Admin API posts test
refs 9471384020

- previously added tests (any subsequent matcher updates) for browse endpoint were not using matchers that sufficiently covered the dynamic portions of the body
2022-09-13 14:09:45 +01:00
Kevin Ansfield
eebdb1d5df Fixed e2e Admin API posts test
refs 9471384020

- previously added tests for browse endpoint were not using matchers that sufficiently covered the dynamic portions of the body
2022-09-13 14:05:03 +01:00
Kevin Ansfield
9471384020 Added tests for Admin API not returning lexical by default but including when requested
no issue

- left `mobiledoc` as the only default format added in the post/page input serializers for now to minimize API/test churn during these early stages of lexical development
- tested that the `lexical` field is not returned by default but can be requested via `?formats=lexical`
2022-09-13 13:30:29 +01:00
Kevin Ansfield
7ad1be2555 Fixed Content API returning lexical format when requested
no issue

- similar to the `mobiledoc` field, the Content API should not return the source `lexical` field if requested via `?formats=`
  - renamed `removeMobiledocFormat()` to `removeSourceFormats()` to better match it's behaviour
2022-09-13 13:30:29 +01:00
Daniel Lockyer
f8679f22d7
Updated settings snapshot
refs 067bfe92a4

- this was missed in the previous commit
2022-09-13 12:37:33 +01:00
Rishabh
054833992e Wired events for triggering email alerts for subscription creation/cancellation
refs https://github.com/TryGhost/Team/issues/1865

- refactors subscription creation/cancellation to dispatch proper events which are used for email alerts
- cleanup
2022-09-10 11:06:34 +05:30
Rishabh
2fbaa7b9bc Moved member email alert trigger to member creation
closes https://github.com/TryGhost/Team/issues/1864
refs https://github.com/TryGhost/Team/issues/1881

- triggers free member email alert via event dispatch from member create method
- passes subscription/stripe data to member creation for paid members so free member alert can be ignored for them
- moves subscription created event being called from webhook controller to `linkSubscription`, allows creating subscription events for all new subscriptions instead of ones just via webhooks
2022-09-10 11:06:34 +05:30
Naz
235d716048
Refactored notifications e2e tests to use test framework
no issue

- Bumped into these tests when doing cleanup in the notifications service. Having full snapshot of requests is useful to have as a sanity check, so migrated this test suite quickly.
2022-09-09 19:51:50 +08:00
Ronald Langeveld
eb6534bd7f
Replaced all 'bio' references with 'expertise' for member comments. (#15359)
closes https://github.com/TryGhost/Team/issues/1772

- The user facing side of comments recently replaced `bio` with `expertise`.
- To remain consistent we replaced all the references of `bio` with `expertise` throughout the codebase.
- This includes a database column name changing migration, within the `members` table.
- Bumped up the comments-ui version to a new minor (0.10.x) as its a breaking change.
2022-09-09 10:14:49 +02:00
Daniel Lockyer
790e4c5598
Added history log for staff actions
fixes https://github.com/TryGhost/Toolbox/issues/356

- this feature allows site Administrators to view a history log of staff
  actions on their site so they can audit when and by whom that something happened
- this commit promotes the History log to GA
2022-09-08 18:23:39 +01:00
Simon Backx
4534b693e4
Added test that validates output HTML of email template (#15365)
refs https://github.com/TryGhost/Team/issues/1871

This commit adds a test to the serialize method of `post-emaiserializer`. It checks whether the generated email HTML is valid and standard HTML5 and that all properties are escaped.

To do this validation, I depend on the new `html-validate` dev dependency. Just parsing the HTML with a HTML parser is not enough to guarantee that the HTML is okay.

Apart from that this fixes:
- Removed the sanitizeHTML method and replaced it with normal HTML escaping. We don't want to allow any HTML in the escaped fields. Whereas `sanitizeHTML` still allows valid HTML, but we don't want that and want the same behaviour as on the site. E.g., a post with a title `All your need to know about the <br /> tag` should actually render the same title and non-html content, being `All your need to know about the &lt;br /&gt; tag`
- The file, nft and audio card didn't (always) escape the injected HTML fields (new version @tryghost/kg-default-cards) 
- `@tryghost/string` is bumped because it contains the new escapeHtml method
2022-09-08 10:11:01 +02:00
Simon Backx
8b4d5504e8
Moved (un)like endpoint code to comments service (#15371)
fixes https://github.com/TryGhost/Team/issues/1861

- Moved like and unlike endpoint handling to comments service and controller
- Moved small part of report logic to comments controller
- Added proper 401 authentication error when not authenticated as member
2022-09-06 17:20:55 +02:00
Ronald Langeveld
1f177e1c17
Added optional data-attribute to enable and disable auto redirection. (#15335)
closes https://github.com/TryGhost/Ghost/issues/15104 https://github.com/TryGhost/Team/issues/1800

- On custom sign up and login forms, creators often wouldn't want their members to be redirected to that page after signing in.
- This takes a new data-attribute value (eg `data-members-autoredirect="false"`) that can be set on [custom sign up / login forms](https://ghost.org/docs/themes/members/#signup-forms) into account before parsing the referrer on the magic link URL that gets sent to the member for login.
2022-09-06 14:36:06 +02:00
Naz
a0d0c38aaf
Fixed typo complementary -> complimentary 2022-09-06 17:51:56 +08:00
Simon Backx
2e85ae98be
🐛 Fixed sending emails from email domain that includes www subdomain (#15348)
fixes https://github.com/TryGhost/Team/issues/1855
fixes https://github.com/TryGhost/Team/issues/1866

This commit moves all duplicate methods to get the support email address to a single location. Also methods to get the default email domain are moved.

For the location, I initially wanted to put it at the settings service. But that service doesn't feel like the right place. Instead I created a new settings helpers service. This service takes the settingsCache, urlUtils and config and calculates some special 'calculated' settings based on those:

- Support email methods
- Stripe (active) keys / stripe connected (also removed some duplicate code that calculated the keys in a couple of places)
- All the calculated settings are moved to the settings helpers

I'm not 100% confident in whether this is the right place to put the helpers. Suggestions are welcome.
2022-09-02 16:57:59 +02:00
Simon Backx
51ddc39fa7 Updated snapshots of email preview tests
refs dd2bfb8c0e
2022-09-02 16:19:28 +02:00
Hannah Wolfe
642b6ff8ae
Added loginAs[Role] to e2e framework with example
closes: https://github.com/TryGhost/Toolbox/issues/342
refs: 032a26f9f3
refs: 588c9d04e8

- Now that the old `users:no-owner` (now named 'users') is working correctly :)
- Was able to add loginAs[Role] methods for each staff role, so that it's possible to execute tests as that user and check permissions
- Refactored the email preview tests to use the new e2e framework and these methods, as an example
2022-09-02 10:38:22 +01:00
Hannah Wolfe
588c9d04e8
Renamed users:no-owner to users as main user fixture
- This fixture is the main user fixture you'd want to use when testing staff roles
- At the moment it has a weird name that makes it less likely people will use it
- A tiny step in trying to make our fixture system make a tiny bit more sense
2022-09-02 10:08:37 +01:00
Hannah Wolfe
032a26f9f3
Fixed users:no-owner fixture to add roles correctly
- This fixture would only work if the roles were inserted by the fixture system
- In most cases, this fixture was adding users without their associated roles
- Now we assume the roles exist already, and that we need to map users to each role
- This will allow us to more easily test user roles in e2e tests
2022-09-02 08:26:30 +01:00
Fabien 'egg' O'Carroll
e4cbb3d24d
Reset magic link rate limiting upon successful login (#15345)
refs https://github.com/TryGhost/Team/issues/1771

We don't have access to `req.brute.reset` due to the way the flow
works, we have one endpoint which sends an email with a magic link,
and another route which handles the login. We don't want to apply
brute force protection to both because our rate limiting is designed
for API requests not web page visits (which is how login is handled).

Because of this we require access to the underlying ExpressBrute
instance exposed by the spam-protection module, so that we can
perform the reset.
2022-09-01 08:54:14 -04:00
Fabien 'egg' O'Carroll
2ff81cc5d3
🔒 Fixed rate limiting for user login (#15336)
refs https://github.com/TryGhost/Team/issues/1074

Rather than relying on the global block to stop malicious actors from
enumerating email addresses to determine who is and isn't a user, we
want our user login brute force protection to be on an IP basis,
rather than tied to the username.
2022-08-31 10:33:42 -04:00
Simon Backx
4282ead3a7
🐛 Fixed commenting on tier-only posts (#15333)
fixes https://github.com/TryGhost/Team/issues/1860

**Problem:**
Members were not able to comment on a post that was only visible for members with a specific tier.

**Causes:**
Content gating was done on models with missing relations.
- The products relation was not loaded on the member when doing content gating
- The tiers relation was not loaded on the post when doing content gating

**Tests:**
- Added for tier-only posts
- Added for paid-only commenting
2022-08-30 17:38:58 +02:00
Simon Backx
e7786ca482
🐛 Fixed removing comped subscriptions for members with active subs (#15332)
fixes https://github.com/TryGhost/Team/issues/1859

**Problem:**
When for some reason a member has an active subscription (or legacy comped subscription) for product A, and a comped subscription for product B. You cannot remove comped subscription B.

**Fixed by:**
Updating the API to allow more flexible product changes on members.
- Allow the removal of (comped) products on a member, as long as that product doesn't have a related subscription
- (still) allow the addition of comped products to a member, as long as that member doesn't have other active subscriptions. This matches the existing behaviour, but now this is only checked for added products.
- Includes tests for these edge cases
2022-08-30 17:36:52 +02:00
Simon Backx
f2da1229d8
Removed unused support email verificaton endpoints (#15328)
fixes https://github.com/TryGhost/Team/issues/1679

These endpoints are safe to be removed, as they are only used by the admin app and usage has been removed over there. It is very unlikely that this endpoint has been used in a third party integration (in which case they will get a notification email).
2022-08-29 15:16:13 +02:00
Rishabh Garg
594ef34871
Enabled member email alerts (#15321)
closes https://github.com/TryGhost/Team/issues/1825
closes https://github.com/TryGhost/Team/issues/1826

- allows site owners/admins to receive email notifications when somebody signs up, becomes paid, or cancels subscription
- owners/admins can set their email preference from staff settings
2022-08-26 09:38:12 +05:30
Fabien "egg" O'Carroll
4718171b1d Removed out of date history items from UrlHistory
In case there is an issue with the filtering of items in our client
side attribution script, we also check for and remove out of date
items here. This ensures that we do not erroneously attribute signups
or conversions to webpages from more than 24h ago.
2022-08-25 16:09:34 -04:00
Simon Backx
67163209e1
Enabled member attribution flag in all tests (#15317)
closes https://github.com/TryGhost/Team/issues/1852

Updates all tests to run with memberAttribution flag enabled
2022-08-25 15:25:01 -04:00
Rishabh
216eeb9d71 Added paid subscription start email alert
refs TryGhost/Team#1826

- triggers paid subscription start email via staff service
2022-08-25 19:53:02 +05:30
Rishabh
aeadf8a5e1 Added email alert trigger for free member signup
refs TryGhost/Team#1826

- fires email alert on free member creation after they finish signing up via checkout link
2022-08-25 19:53:02 +05:30
Simon Backx
232882daa2 Mapped '/' attribution url to homepage
fixes https://github.com/TryGhost/Team/issues/1846

- Shows homepage instead of / for attribution values
2022-08-25 14:51:38 +02:00
Hannah Wolfe
c9864ee63f Added {{search}} theme helper
closes: https://github.com/TryGhost/Team/issues/1732

- adds a theme helper which outputs a working search button with a standard icon
- the icon adopts whatever the current color is from css, and has a set of default styles
- styles can be overridden with !important or the data attribute
- alternatively, any element in a theme may be turned into a search button by adding data-ghost-search
- this is meant to be a simple tool for non-theme-developers to easily add a search icon to their themes in a way that doesn't require css or html knowledge
2022-08-24 21:34:20 +01:00
Hannah Wolfe
96f7b8fdc8 Fixed content-length, again
refs: 203c8036fa
refs: 1fadbacdec
refs: 22fd7f289c

- There is something seriously weird about how content-length changes...
- It's different on CI to local sometimes...
- This particular test should not change IMO
2022-08-24 20:38:35 +01:00
Hannah Wolfe
203c8036fa
Fixed more content-length labs issues
refs: 1fadbacdec
refs: https://github.com/TryGhost/Ghost/commit/x5447985ee2e8a4b497e9c1afbad07

- I'm trying to make it so that changing labs flags doesn't require changes to these snapshot files!
2022-08-24 19:45:20 +01:00
Simon Backx
f124d142c9 Added member attributions to activity feed (#15283)
refs https://github.com/TryGhost/Team/issues/1833
refs https://github.com/TryGhost/Team/issues/1834

We've added the attribution property to subscription and signup events when the
flag is enabled. The attributions resource is fetched by creating multiple relations
on the model, rather than polymorphic as we ran into issues with that as they can't
be nullable/optional.

The parse-member-event structure has been updated to make it easier to work with,
specifically `getObject` is only used when the event is clickable, and there is now a 
join property which makes it easier to join the action and the object.
2022-08-24 11:17:28 -04:00
Fabien "egg" O'Carroll
3c431bd8da Revert "Added member attributions to activity feed (#15283)"
This reverts commit e986b78458.

The tests were not passing for the PR and it was erroneously
merged into main
2022-08-24 11:01:47 -04:00
Simon Backx
e986b78458
Added member attributions to activity feed (#15283)
refs https://github.com/TryGhost/Team/issues/1833
refs https://github.com/TryGhost/Team/issues/1834

We've added the attribution property to subscription and signup events when the
flag is enabled. The attributions resource is fetched by creating multiple relations
on the model, rather than polymorphic as we ran into issues with that as they can't
be nullable/optional.

The parse-member-event structure has been updated to make it easier to work with,
specifically `getObject` is only used when the event is clickable, and there is now a 
join property which makes it easier to join the action and the object.
2022-08-24 10:11:25 -04:00
Rishabh
326bb97d2c Enabled free trials via tiers and offers
refs https://github.com/TryGhost/Team/issues/1724
refs https://github.com/TryGhost/Team/issues/1726

- allows site owners to add a default free trial period to their tiers
- allows site owners to create custom offers that allow free trials on tiers
2022-08-24 19:24:31 +05:30
Rishabh Garg
9abfae2ddb
Switched off email alerts for subscription cancellation by default (#15304)
refs TryGhost/Team#1825

- all sites are expected to have cancellation alerts off by default
2022-08-24 18:54:00 +05:30
Hannah Wolfe
af94855349 Removed bluebird catch predicates from API endpoints
refs: https://github.com/TryGhost/Ghost/issues/14882

- I found a common pattern where catch predicates were being used to catch non-existent models in destroy methods, and sometimes elsewhere in the API endpoints
- The use of predicates is deprecated, and we're working to remove them from everywhere, so that we can remove bluebird
- In order to still handle these errors correctly, we needed a small change to mw-error-handler so that it can detect EmptyResponse errors from bookshelf, as well as 404s
Note: there is a small change as a result of this - the context on these errors now says "Resource not found" instead of "{ModelName} not found".
- I think this is acceptable for now, as we will be reviewing these errors in more depth later. It's quite easy to make changes, we just have to decide what with proper design input
2022-08-24 11:27:09 +01:00
Rishabh Garg
c48c65cc88
Added member email alert notification columns for staff (#15276)
refs https://github.com/TryGhost/Team/issues/1825

- adds 3 new columns to users table for storing email alert preferences for member signups/cancellation
- adds column for new member signup alert
- adds column for paid subscription started alert
- adds column for paid subscription canceled alert
- Updated default fixtures and tests for new columns
2022-08-23 22:11:38 +05:30
Daniel Lockyer
7f0996d986
Implemented resource linking in Audit Log
refs https://github.com/TryGhost/Toolbox/issues/356

- we have a very crude version of this before but it just wasn't
  maintainable
- one of the first things I did here was to add `include=resource` on
  the API call, so it returns the fields we need without extra API
  requests
- after we have the id/slug, I could build a route and model array
  dynamically, or return null if we can't redirect to the object (it
  doesn't exist)
2022-08-23 17:48:11 +02:00
Fabien "egg" O'Carroll
73466c1c40 Added ability to filter members on conversion attribution
refs https://github.com/TryGhost/Team/issues/1830
2022-08-23 11:36:56 -04:00
Fabien "egg" O'Carroll
83f2bf4757 Added ability to filter members on signup attribution
refs https://github.com/TryGhost/Team/issues/1831
2022-08-23 11:36:56 -04:00
Simon Backx
d91d6c1311
Added attribution counts to Pages API (#15289)
fixes https://github.com/TryGhost/Team/issues/1835

This was missed in the initial pass when adding to the Posts API
2022-08-23 10:26:34 -04:00
Hannah Wolfe
4cd210c29c Added post deletion tests using new e2e framework
- copied over and rewrote the deletion test from the legacy file
- added a new test that checks that we get a 404 when attempting to delete an unknown post
- this is a guard to protect and futureproof the API whilst we do refactoring to improve 404 handling from bookshelf
- in turn this is aimed at helping to get rid of a bunch of catch predicates from the API
2022-08-23 14:49:29 +01:00
Hannah Wolfe
a2a71c8e67 Renamed e2e admin post tests to legacy
- I want to start rewriting the post tests using the new e2e framework, but it's quite a big task
- For now I have renamed the existing file, and will use the correct file name for writing modern versions of tests
- Note: I have a specific test that I'd like to add which is far easier to write in the new framework
- This change should facilitate moving forward more with the new framework
2022-08-23 14:49:29 +01:00
Daniel Lockyer
9effa119c6 Implemented context on Actions events
refs https://github.com/TryGhost/Toolbox/issues/356

- in order to show data that we might not necessarily still have around
  (ie. when you delete a post, you might want the title), we're going to
  start utilizing the `context` column
- right now, we store the `primary_name` for deleted events, and we also
  store the `setting` `key` and `group` so we can reference it in the
  audit log
2022-08-23 14:58:41 +02:00
Aileen Nowak
e5b604cca0 Updated camelCase properties in Explore endpoint to be snake_case 2022-08-23 12:04:07 +01:00
Aileen Nowak
4892f1d0d5 Added publication language to admin site endpoint
no issue

- The site locale should be exposed within the public site config in order to handle i18n in third party apps
- Added the locale to Explore service to simplify fetching it when submitting a new site
2022-08-23 12:04:07 +01:00
Simon Backx
0943daad72
Added member attribution to member details page (#15266)
refs https://github.com/TryGhost/Team/issues/1817

Co-authored-by: James Morris <moreofmorris@users.noreply.github.com>
2022-08-19 16:39:18 -04:00
Daniel Lockyer
b023f716ab
Fixed content-length header in Explore snapshot
- this endpoint returns the Ghost version, of which the minor just hit
  double digits
- because of this, the content-length size changed, and the snapshot was
  incorrect
- we've previously allowed overrides for the content-length to be any number (see
  1fadbacdec)
- this commit allows the header to be any number so it doesn't fail when
  the Ghost version is incremented
2022-08-19 17:14:04 +02:00
Rishabh
1258156c38 Handled storing complimentary subscription expiry
refs https://github.com/TryGhost/Team/issues/1727

- if feature flag is enabled, handles storing expiry date on complimentary subscriptions in `expiry_at` column of `members_products`
- updates the expiry value on both member edit or add with tiers
- expiry is passed as `expiry_at` in `tiers` list of a member
- includes `expiry_at` on tiers data of a member when flag is enabled
2022-08-19 18:20:52 +05:30
Hannah Wolfe
1fadbacdec
Fixed content-length in labs snapshot
refs: 22fd7f289c

- in the mentioned commit I changed the tests so that we don't need to update snapshots for every labs flag change
- this commit does the same for content-length which didn't get picked up locally, but does on CI for some reason
- the goal is to allow the team to add and remove flags without needing to update a random snapshot
2022-08-19 10:51:43 +01:00
Fabien "egg" O'Carroll
20726c8fc2 Added signup & conversion counts to Posts API
refs https://github.com/TryGhost/Team/issues/1822

Exposing the values through the API is restricted behind the alpha flag.
We're exposing the values by default when the flag is enabled for now,
but can reconsider that later.
2022-08-18 19:24:54 -04:00