Commit Graph

138 Commits

Author SHA1 Message Date
Fabien 'egg' O'Carroll
57695d03df
Fixed rate limit test (#16258)
closes https://github.com/TryGhost/Team/issues/2547

Changed the configuration for testing to be a bit more strict, by slowing down the amount of requests it can handle to give CI enough time to kick in the rate limiter. Before this, CI simply wasn't hitting the API fast enough to trigger the rate limiter.

Co-authored-by: Ronald Langeveld <hi@ronaldlangeveld.com>
2023-02-13 15:16:56 +08:00
Simon Backx
ea2c69565f
Moved email event fetching to main thread (#16231) 2023-02-09 09:36:39 +01:00
Ronald Langeveld
30dc2a8228
Added mention_notifications column (#16242)
refs https://github.com/TryGhost/Team/issues/2526

- created a migration for a new boolean column in users that would
determine if the staff user gets an email when the publication receive a
new mention.
2023-02-09 16:15:54 +08:00
Ronald Langeveld
c77984e6ab
Added mentions permissions (#16200)
closes https://github.com/TryGhost/Team/issues/2420

- Added user roles and permissions for the mentions admin API.
- We only have a `browse` function for our current use case, accessible
by `administrator` and `admin integration`.
2023-01-31 16:40:44 +08:00
Simon Backx
8e66edee2b
🐛 Fixed storing original files for images (#16117)
fixes https://github.com/TryGhost/Team/issues/481

This change fixes an issue when multiple images with the same name are
uploaded in parallel. The current system does not guarantee that the
original filename is stored under NAME+`_o`, because the upload for the
original file and the resized file are happening in parallel.

Solution:
- Wait for the storage of the resized image (= the image without the _o
suffix) before storing the original file.
- When that is stored, use the generated file name of the stored image
to generate the filename with the _o suffix. This way, it will always
match and we don't risk both files to have a different number suffix.
We'll also set the `targetDir` argument when saving the file, to avoid
storing the original file in a different directory (when uploading a
file around midnight both files could be stored in 2023/01 and 2023/02).

Some extra optimisations needed with this fix:
- Previously when uploading image.jpg, while it already exists, it would
store two filenames on e.g., `image-3.jpg` and `image_o-3.jpg`. Note the
weird positioning of `_o`. This probably caused bugs when uploading
files named `image-3.jpg`, which would store the original in
`image-3_o.jpg`, but this original would never be used by the
handle-image-sizes middleware (it would look for `image_o-3.jpg`). This
fix would solve this weird naming issue, and make it more consistent.
But we need to make sure our middlewares (including handle-image-sizes)
will be able to handle both file locations to remain compatible with the
old format. This isn't additional work, because it would fix the old bug
too.
- Prevent uploading files that end with `_o`, e.g. by automatically
stripping that suffix from uploaded files. To prevent collisions.

Advantage(s):
- We keep the original file name, which is better for SEO.
- No changes required to the storage adapters.

Downside(s):
- The storage of both files will nog happen parallel any longer. But I
expect the performance implications to be minimal.
- Changes to the routing: normalize middleware is removed
2023-01-30 16:40:50 +01:00
Steve Larson
477295a262
added mentions admin browse api e2e tests (#16177)
refs TryGhost/Team#2468
-added simple browse api tests for webmentions
-need to add user access tests and possibly site access (private sites)
2023-01-30 07:59:00 -06:00
Simon Backx
8f8ca481a6
Fixed configUtils and adapter cache issues in E2E tests (#16167)
no issue

There are a couple of issues with resetting the Ghost instance between
E2E test files:

These issues came to the surface because of new tests written in
https://github.com/TryGhost/Ghost/pull/16117

**1. configUtils.restore does not work correctly**
`config.reset()` is a callback based method. On top of that, it doesn't
really work reliably (https://github.com/indexzero/nconf/issues/93)

What kinda happens, is that you first call `config.reset` but
immediately after you correcty reset the config using the `config.set`
calls afterwards. But since `config.reset` is async, that reset will
happen after all those sets, and the end result is that it isn't reset
correctly.

This mainly caused issues in the new updated images tests, which were
updating the config `imageOptimization.contentImageSizes`, which is a
deeply nested config value. Maybe some references to objects are reused
in nconf that cause this issue?

Wrapping `config.reset()` in a promise does fix the issue.

**2. Adapters cache not reset between tests**
At the start of each test, we set `paths:contentPath` to a nice new
temporary directory. But if a previous test already requests a
localStorage adapter, that adapter would have been created and in the
constructor `paths:contentPath` would have been passed. That same
instance will be reused in the next test run. So it won't read the new
config again. To fix this, we need to reset the adapter instances
between E2E tests.

How was this visible? Test uploads were stored in the actual git
repository, and not in a temporary directory. When writing the new image
upload tests, this also resulted in unreliable test runs because some
image names were already taken (from previous test runs).

**3. Old 2E2 test Ghost server not stopped**
Sometimes we still need access to the frontend test server using
`getAgentsWithFrontend`. But that does start a new Ghost server which is
actually listening for HTTP traffic. This could result in a fatal error
in tests because the port is already in use. The issue is that old E2E
tests also start a HTTP server, but they don't stop the server. When you
used the old `startGhost` util, it would check if a server was already
running and stop it first. The new `getAgentsWithFrontend` now also has
the same functionality to fix that issue.
2023-01-30 14:06:20 +01:00
Simon Backx
3ac282598c Fixed Playwright tests for new email flow
no issue

Tests stopped working because the Mailgun mocker stopped working since we moved to the new email flow.

This also fixes a unit test that needed to get updated.
2023-01-20 19:00:13 +01:00
Fabien 'egg' O'Carroll
33ebe971f8
Added BookshelfMentionRepository implementation (#16156)
fixes https://github.com/TryGhost/Team/issues/2418

This stores the received webmentions in the database.

Co-authored-by: Simon Backx <simon@ghost.org>
2023-01-20 14:32:50 +01:00
Simon Backx
e879406659
Added outbound link tagging setting (#16146)
fixes https://github.com/TryGhost/Team/issues/2432
    
Adds outbound_link_tagging setting (enabled by default and behind
feature flag). If the feature flag is enabled, and the setting is
disabled, we won't add ?ref to links in emails.
    
This includes new E2E tests for email click tracking, which were also
extended to check outbound link tagging (for both MEGA and the new email
stability flow).

Also fixes a test fixture for the comments_enabled setting.
2023-01-20 13:41:36 +01:00
Naz
967dd7d244
Added deprecation notice to mockManager's sentEmailCount
refs https://github.com/TryGhost/Toolbox/issues/499

- The mockManager's sentEmailCount is left here to avoid breaking many tests that already depend on this method. With future improvements to email snapshot tests this method should not be used. Instead, emailMockReceiver's own sentEmailCount method should be used directly.
2023-01-18 12:55:45 +08:00
Naz
112ab23968
Removed direct use of @tryghost/jest-snapshot
refs https://github.com/TryGhost/Toolbox/issues/499
refs 6bcc47a0ad

- Using module directly caused issues with snapshots manager instance initialization (mocha hooks did not apply to a correct instance)
- See refed commit for more
2023-01-17 12:50:49 +08:00
Naz
1f6b2b4d5e
Added email content testing capabilities
refs https://github.com/TryGhost/Toolbox/issues/499

- Outgoing emails have been a weak point of Ghost's stability recently. The concept of "emailMockReceiver" similarly to "webhookMockReceiver", allows to test side-effects like outgoing emails.
- This is a first iteration which should lay groundwork for testing all outgoing emails in the future
- The change adds a new concept of "email mock receiver" which is very similar to how the "webhook mock receiver" works. The email mock receiver exposes two methods to record and verify snapshots:
- matchHTMLSnapshot - records and verifies only the HTML content of the outgoint email
- matchMetadataSnapshot - records and verifies all the non-HTML properties sent along an email content, e.g.: to address, plaintext, subject, etc.
- What's missing is matching content based on dynamic content like dates, links with JWT tokens, etc.
2023-01-17 12:50:49 +08:00
Simon Backx
7b3712a15b
Added visible theme errors in admin (#16081)
refs https://github.com/TryGhost/Team/issues/2393

- During boot and loading the active theme, we now cache the result of
the gscan validation. Cache configuration can happen in
`adapters.cache.gscan`
- We now also return non-fatal errors when activating or adding a theme.
- When the `themeErrorsNotification` feature flag is on, we fetch the
active theme (which includes the validation information) when loading
admin
- If the currently active theme has errors, we show an error
notification that can open the error modal
- Added a new endpoint: `/ghost/api/admin/themes/active/` that returns
the result of the last gscan validation of the active theme. If no cache
is available, it will run a new gscan validation.
- Added new permissions for the active action/endpoint (author, editor,
administrator)
2023-01-06 13:44:27 +01:00
Fabien "egg" O'Carroll
2d9114450c Fixed email_recipients fixtures for tests
The email_recipient fixtures were using duplicate and mismatched email addresses
rather than having them correctly map to the Members, which is required for testing
email suppressions.
2023-01-05 17:11:06 +07:00
Rishabh Garg
2eac41b1f4
🐛 Fixed invalid email getting saved for members (#16021)
refs https://github.com/TryGhost/Team/issues/2235

We found some cases which can cause a site to have member emails that have invalid characters like `member@example.com�`. This happened due to the `validator` version used by Ghost not able to catch some specific cases as invalid email, allowing members to be created with them either via Admin or Importer or direct signup. Portal UI already blocked these email as invalid. This change:

- updates `@tryghost/validator` to include a latest version of email validator that catches these invalid cases
- doesn't allow member creation with invalid email like above
- doesn't allow existing member emails to be edited to invalid
2022-12-16 16:47:52 +05:30
Sam Lord
3449c49c10 Tidy up the mocking utility for Mailgun
no issue
Moved the mocking into the Playwright specific codebase, since this will only be used in browser based testing
2022-12-08 13:01:35 +00:00
Naz
4c6a86eca4 Added mocked Mailgun client to browser test env
refs https://github.com/TryGhost/Ghost/pull/15959

- To be able to test and intercept emails we need a mock in test environment - avoids making calls to the Mailgun API
2022-12-08 13:01:35 +00:00
Naz
6ef5e6a7e0 Fixed local environment for playwright tests
refs https://github.com/TryGhost/Toolbox/issues/479

- this includes a handful of improvements to get Playwright working on a
  local environment including:
  - adding `testing-browser` environment so we don't nuke `development`
    environments, and makes all the necessary changes to get Ghost to
    behave when this is running
  - stopped running one global instance of Ghost as this doesn't provide
    a clean environment
  - copies a few default fixtures that are needed for the new
    environment
2022-12-07 09:44:05 +07:00
Naz
e170f293e3
Extracted sleep method to e2e framework module
no issue

- The sleep method has been used in 8 modules reimplementing the same thing over and over again. It's usually a sign of async event processing outside of the request/response loop. It's good to have a single point of implementation for a "hack" like this, so we could track it easier and address the even processing delay in a more optimal way centrally if it ever becomes a bottleneck
2022-12-05 17:26:29 +07:00
Simon Backx
2a3a72e4d5
Added email debug API (#15915)
fixes https://github.com/TryGhost/Team/issues/2346

- Adds email batch browse endpoint
- Adds email recipient failures browse endpoint
- Adds new fixtures and E2E tests for the new API
- Added support for snapshot tests to have 'nullable' types.
2022-12-02 09:44:32 +01:00
Simon Backx
4c166e11df
Added E2E tests for batch sending (#15910)
refs https://github.com/TryGhost/Team/issues/2339

- Includes a new pattern in the job manager that allows us to properly
await jobs.
- Added new convenience mocking methods to stub settings
- Tests the main flows for bulk sending:
    - Sending in multiple batches
    - Sending to multiple segments
    - Handling a failed batch and retrying that batch
- Fixes bug in batch generation (ordering not working)

In a different PR I'll add more detailed tests.
2022-12-01 13:43:49 +01:00
Simon Backx
6e72767a50
Fixed verification trigger not working for large imports (#15887)
fixes https://github.com/TryGhost/Team/issues/2326

When importing more than 500 members, we didn't testImportThreshold at
the right time. It was called too early because the importing job was
not awaited. This also adds an E2E test for this case.
2022-11-28 18:22:10 +01:00
Naz
60d066b243
🔒 Disabled editable relations by default
refs https://github.com/TryGhost/Ghost/security/advisories/GHSA-9gh8-wp53-ccc6
refs https://github.com/TryGhost/Toolbox/issues/465

- Bookshelf relations allows us to edit relational records by default, which was used liberally in the codebase.
- Not having a clear track record of editable relations left the model layer prone to triggering unwanted nested saves and created a vulnerability where members were able to edit newsletter settings.
- With explicit editable relations it's easier to keep track of relations having editable access to related records. Makes the relational data modification pattern safer to use too.
- Anyone running 5.x should update to 5.24.1

Credits: Dave McDaniel and other members of [Cisco Talos](https://talosintelligence.com/vulnerability_reports)
2022-11-28 18:40:26 +07:00
Simon Backx
5c2f0b9a4b
Removed getCreatedEvents and added verification trigger test (#15832)
refs https://github.com/TryGhost/Team/issues/2266

This removes the deprecated `getCreatedEvents` method in the event
repository and adds tests to the verification trigger to see if we don't
break anything.

Changes extracted from https://github.com/TryGhost/Ghost/pull/15831
2022-11-18 15:05:15 +01:00
Simon Backx
2220686113 Sped up comments and feedback tests by reusing Ghost app instance
refs https://ghost.slack.com/archives/C02G9E68C/p1668777066462859

Added a duplicate method to the mmebers agent, so we can reuse the same app instance and options, whenever we need multiple agents (each with their own cookies).
2022-11-18 14:58:35 +01:00
Sodbileg Gansukh
78bff39c23
Update the cover image in default fixtures (#15817)
- updated the cover image to be simpler
- made the change in text fixtures as well, just to keep the fixtures in sync

Co-authored-by: Hannah Wolfe <github.erisds@gmail.com>
2022-11-15 21:19:50 +00:00
Simon Backx
f4660b440b
Fixed sending emails when using MembersAgent.loginAs (#15791)
fixes https://github.com/TryGhost/Team/issues/1903

MembersAgent.loginAs sends email, asynchronously via events. Which
conflicts with tests that also test emails. We cannot properly await
these events, so this is currently fixed with a timeout of 200ms. But
this was too random and unreliable.
2022-11-09 16:25:42 +01:00
Naz
26b0bbc623 Added a test suite for OPTIONS requests
refs https://github.com/TryGhost/Toolbox/issues/461

- The codebase has ambiguous behavior with OPTIONS request. Adding tests covering edge cases for all possible variations of OPTIONS responses is the first step to solving cahceability of these requests.
- The obvious question if you look into the changeset itself would also be: "WTF did you do with test suite naming? What are these changes in admin and click tracking suites? You having a bad day Naz?". The answer is "yes"  (╯°□°)╯︵ ┻━┻
- On a serious note. I've introduced multiple hacks here that should be fixed:
1. Forced test suite execution order for options request - extreme blasphemy. This was last resort decision. I went deep into trying to fixup the server shutdown in the "admin" test suite, which cascaded into failing "click tracking" suite, which has shortcomings on it's own (see notes left in that suite)
2. Exposed "ghostServer" from the e2e-framework's "getAgentsWithFrontend" method. Exposing ghostServer to be able to shut it down (or do other manipulations) was one of the pitfalls we had in the previous test utils, which ended up plaguing the test codebase. Ideally the framework should only be exposing the agents and the rest would happen behind the scenes.
- To fix the hacks above I've raised a cleanup issue (https://github.com/TryGhost/Toolbox/issues/471). I'm very sorry for this mess. The issue at hand has very little to do with fixing the e2e framework, so leaving things "as is".
2022-11-02 13:43:30 +08:00
Naz
5fe80c82c5 Exposed Content API agent in e2e test utils
refs https://github.com/TryGhost/Toolbox/issues/461

- When testing OPTIONS requests there is a need to get all possible agents available in the system. The "getAgentsWithFrontend" serves exactly this purpose - create all possible agents while starting Ghost instance only once
- This is groundwork for OPTIONS request caching tests and improvements
2022-11-02 13:43:30 +08:00
Rishabh
6795e95483 Fixed settings test
refs f1aff45dc7

- adds missing default setting for member source tracking
2022-10-27 21:30:31 +05:30
Rishabh Garg
f1aff45dc7
Disabled attribution calculation when tracking is disabled (#15710)
refs https://github.com/TryGhost/Team/issues/2168

- forces attribution service to use empty history or context if attribution tracking is disabled
2022-10-27 21:10:03 +05:30
Fabien "egg" O'Carroll
25d8d694a0 Wired up the payment service to create stripe checkout sessions
refs https://github.com/TryGhost/Team/issues/2078
2022-10-25 09:03:04 +07:00
Kritika Sharma
26d049911c
Added e2e tests for post.published.edited webhook (#15642)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.

Co-authored-by: Kritika Sharma <kritikasharma@Kritikas-MacBook-Pro-2.local>
2022-10-24 14:55:25 +01:00
Halldor Thorhallsson
dbad621b91
Removed bluebird from fixture-utils.js (#15626)
refs: https://github.com/TryGhost/Ghost/issues/14882

- Opted to use the in-house `sequence` function when refactoring Bluebird's `Promise.each` to avoid deadlock issues (see 734ef66e6c). 
-It's hard to know without tonnes of context if any `Promise.each` are safe to refactor to `Promise.all`.
2022-10-21 20:56:21 +01:00
Rishabh
d8bacf12d1 Added endpoint for fixing newsletter links
refs https://github.com/TryGhost/Team/issues/2104

- adds new bulk edit endpoint for links, updates all matching link with the current redirect url and update to new url
2022-10-20 17:50:02 +05:30
Simon Backx
adf10f0e76
Added support for filtering email events by post_id (#15666)
refs https://github.com/TryGhost/Team/issues/2093
2022-10-20 13:29:00 +02:00
Rishabh Garg
318a5a809c
Added permissions for link edit endpoints (#15664)
refs https://github.com/TryGhost/Team/issues/2104

- adds edit permissions for links endpoints to fixtures
- new `bulkEdit` endpoint will use the permissions and allow fixing newsletter links via Admin
2022-10-20 09:11:26 +05:30
Rishabh Garg
60b10ad69a
Fixed permissions for links endpoint (#15656)
refs 5fcf5098a8

- links browse endpoint had permissions switched off unintentionally and was also missing the necessary permissions in fixtures.
- enables permissions for browse endpoint and adds migration insert permissions in DB
2022-10-20 08:18:29 +05:30
Naz
b589a66cd4
Fixed broken CSV importer tests
refs 90768e9985

- With introduction of strict field mapping the regression test testing for "imports of not mapped fields" failed.
2022-10-19 18:33:47 +08:00
Simon Backx
a01fb5f1aa
Added post_id filter and total to activity feed API (#15650)
fixes https://github.com/TryGhost/Team/issues/2091
fixes https://github.com/TryGhost/Team/issues/2089

- Added new fixtures to make testing easier for the activity feed
- Improved E2E test coverage of activity feed with separate test file
- Added data.post_id filter to enable filtering by events related to a
given post
- Fixed return types in JSDoc of test agents (TypeScript interprets
these as `typeof Agent` if we don't add `InstanceType<Agent>`)
- Added total pagination metadata to activity feed API (to allow a basic
type of pagination using filters)
2022-10-18 15:52:04 +02:00
Fabien 'egg' O'Carroll
bd0f4b4b8c
Added Tier price and currency data to products table (#15366)
refs https://github.com/TryGhost/Team/issues/1765

In order to better handle deleted objects in Stripe we want to decouple
Members from Stripe.

These changes allow us to have the Tier concept completely independent
of the Stripe tables, such that the Stripe data can be generated as/when
it's needed - which will help to protect against missing data.
2022-10-14 06:40:17 +01:00
jbenezech
3d44e37cbd
🐛 Fixed sitemaps with no content (#15571)
closes: https://github.com/TryGhost/Ghost/issues/14981

- Taxonomy-specific sitemaps were invalid xml when there was no data
- These invalid empty sitemaps were referenced in the index sitemap causing SEO tools to report errors
2022-10-12 14:11:19 +01:00
Daniel Lockyer
e2ba19b0df
Fixed import of bson-objectid in accordance to the typings
- there's a few different ways we can import it but I've chosen to
  append `.default` as we have done in several other places in the code
2022-10-12 14:54:35 +07:00
Simon Backx
d4540012dc Added tests for click events in the activity feed
fixes https://github.com/TryGhost/Team/issues/2018

- Includes new test fixtures for redirects and click events
- Tests if post, and links are returned in the click events
2022-10-06 11:43:39 +02:00
Naz
fe1d0e44b4
Moved Ghost agent matcher to common framework
refs a499f866f3
refs d817e5830d

- The user-agent used in outgoing Ghost requests (webhooks mostly) is dependent on the Ghost version - snapshots break if the matcher is not dynamic.
- There will be a few more webhooks tests coming soon, so makes sense to have this matcher moved to a common "framework matchers"
2022-10-06 08:56:10 +08:00
Simon Backx
e7378520a0
🔒 Prevented member creation when logging in (#15526)
fixes https://github.com/TryGhost/Ghost/issues/14508

This change requires the frontend to send an explicit `emailType` when sending a magic link. We default to `subscribe` (`signin` for invite only sites) for now to remain compatible with the existing behaviour.

**Problem:**
When a member tries to login and that member doesn't exist, we created a new member in the past.

- This caused the creation of duplicate accounts when members were guessing the email address they used.
- This caused the creation of new accounts when using an old impersonation token, login link or email change link that was sent before member deletion.

**Fixed:**
- Trying to login with an email address that doesn't exist will throw an error now.
- Added new and separate rate limiting to login (to prevent user enumeration). This rate limiting has a higher default limit of 8. I think it needs a higher default limit (because it is rate limited on every call instead of per email address. And it should be configurable independent from administrator rate limiting. It also needs a lower lifetime value because it is never reset.
- Updated error responses in the `sendMagicLink` endpoint to use the default error encoding middleware.
- The type (`signin`, `signup`, `updateEmail` or `subscribe`) is now stored in the magic link. This is used to prevent signups with a sign in token.

**Notes:**
- Between tests, we truncate the database, but this is not enough for the rate limits to be truly reset. I had to add a method to the spam prevention service to reset all the instances between tests. Not resetting them caused random failures because every login in every test was hitting those spam prevention middlewares and somehow left a trace of that in those instances (even when the brute table is reset). Maybe those instances were doing some in memory caching.
2022-10-05 12:42:42 +02:00
Naz
320c6e0dd3
Abstracted a hacky local URL matcher
refs https://github.com/TryGhost/Toolbox/issues/320

- The URL matcher is very likely to be reused in the future, so having it abstracted away gives two benefits:
1. Central place to document hacky behavior and easier future cleanup
2. The implementer of the e2e test does not have to see the "hacky note" and just concentrate on the implementation of the test
2022-10-05 17:23:02 +08:00
Simon Backx
f33e7a22fa
Added E2E test for whole link tracking flow (#15481)
refs https://github.com/TryGhost/Team/issues/1967

- Test is good to test if the whole flow works as expected, and works together
- We can test independent parts in separate tests that have better coverage of more edge cases
- Adds a basic helper to get an agent for the frontend (spent too much time on a better solution so I decided to keep the existing supertest agent)
2022-09-27 18:44:20 +02:00
Naz
7b009bf1fe Enabled shared caching of 404 error responses
refs https://github.com/TryGhost/Toolbox/issues/410

- The 'private' value in 'Cache-Control' response header for all errors made it impossible for shared caches (e.g.: Fastly, Cloudflare) to cache 404 responses efficiently.
- The change substitutes 'max-age=0' which should not effect the browser cache behavior but would allow shared caches to process such requests efficiently.
- A more loose caching logic only applies to 404 responses from GET requests that are not user-specific (non-authenticated, non-cookie containing requests)
2022-09-26 14:54:50 +08:00
Naz
225a046bb8
Made Admin assets aggressively cacheable
closes https://github.com/TryGhost/Toolbox/issues/372

- The admin assets are served with a unique hash depending on the build with a year-long "max-age" value in the response cache-control header. The client browsers still do send 'If-None-Match' requests when there is a hard-refresh on the client side. There's no need for 'If-None-Match' requests though!
- With 'immutable' value in the cache-control header, the browser caches are treating responses as "hard-fresh" without sending redundant requests.
- For more about 'immutable' value read https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#immutable
2022-09-23 13:46:49 +08:00
Simon Backx
7437d92d50
Added post referrers stats API (#15448)
closes https://github.com/TryGhost/Team/issues/1942

- Added data fixtures for referrers
- Added new endpoint to fetch referrer stats for a given post: `/stats/referrers/posts/:id`
- Added new ReferrersStatsService, responsible for calculating referrer stats
2022-09-21 18:16:56 +02:00
Simon Backx
699e67f4e4
Added email_track_clicks setting (#15409)
fixes https://github.com/TryGhost/Team/issues/1900
refs https://github.com/TryGhost/Team/issues/1901

- Defaults to the same value as the current email_track_opens setting for existing installations, otherwise defaults to true
- Had to use a custom migration because the `addSetting` helper doesn't support using an existing setting as current value
- Added a minimal UI to change the setting, but this still needs some design magic 🪄
- Link replacement is disabled if `email_track_clicks` is disabled. In the future we might consider to still do parial additions, such as source attribution and maybe redirects (to discuss).
2022-09-15 15:48:22 +02:00
Fabien "egg" O'Carroll
4726742673 Renamed redirects to custom-redirects
We're going to be adding more redirection logic into Ghost and it's
going to get confusing if we have names this generic. This makes it
clear which feature this service is related to.

Ideally in the future we can combine all of these into one redirects
service, but for now we will be running a specific service per feature
2022-09-14 13:24:17 -04:00
Hannah Wolfe
b0234dd58e
Removed apiVersions from test urlUtils
refs: https://github.com/TryGhost/Toolbox/issues/229

- These properties were removed from urlUtils in 5.0 and no longer do anything
2022-09-09 13:32:30 +01:00
Simon Backx
6bffa893b1
Added snapshot tests to ghost_head helper (#15327)
refs https://github.com/TryGhost/Team/issues/1795

- Snapshots help us detect unexpected changes in the `<head>` of all sites (e.g., newly introduced script tags)
- Added ghost_head tests for comment count helper
2022-09-08 13:04:34 +02:00
Hannah Wolfe
db6fb2d6d0
Removed unused fixture tasks
- these old concepts aren't used anymore
2022-09-06 10:18:55 +01:00
Hannah Wolfe
f1bc8026b7
Removed unused resetRoles method
- working on cleaning up our fixture mechanism and making it easier to understand
2022-09-05 09:32:54 +01:00
Simon Backx
2e85ae98be
🐛 Fixed sending emails from email domain that includes www subdomain (#15348)
fixes https://github.com/TryGhost/Team/issues/1855
fixes https://github.com/TryGhost/Team/issues/1866

This commit moves all duplicate methods to get the support email address to a single location. Also methods to get the default email domain are moved.

For the location, I initially wanted to put it at the settings service. But that service doesn't feel like the right place. Instead I created a new settings helpers service. This service takes the settingsCache, urlUtils and config and calculates some special 'calculated' settings based on those:

- Support email methods
- Stripe (active) keys / stripe connected (also removed some duplicate code that calculated the keys in a couple of places)
- All the calculated settings are moved to the settings helpers

I'm not 100% confident in whether this is the right place to put the helpers. Suggestions are welcome.
2022-09-02 16:57:59 +02:00
Hannah Wolfe
409a4783a3
Renamed content api agent auth method
- the query param is called key, so key is easier to remember
2022-09-02 10:48:03 +01:00
Hannah Wolfe
642b6ff8ae
Added loginAs[Role] to e2e framework with example
closes: https://github.com/TryGhost/Toolbox/issues/342
refs: 032a26f9f3
refs: 588c9d04e8

- Now that the old `users:no-owner` (now named 'users') is working correctly :)
- Was able to add loginAs[Role] methods for each staff role, so that it's possible to execute tests as that user and check permissions
- Refactored the email preview tests to use the new e2e framework and these methods, as an example
2022-09-02 10:38:22 +01:00
Hannah Wolfe
588c9d04e8
Renamed users:no-owner to users as main user fixture
- This fixture is the main user fixture you'd want to use when testing staff roles
- At the moment it has a weird name that makes it less likely people will use it
- A tiny step in trying to make our fixture system make a tiny bit more sense
2022-09-02 10:08:37 +01:00
Hannah Wolfe
032a26f9f3
Fixed users:no-owner fixture to add roles correctly
- This fixture would only work if the roles were inserted by the fixture system
- In most cases, this fixture was adding users without their associated roles
- Now we assume the roles exist already, and that we need to map users to each role
- This will allow us to more easily test user roles in e2e tests
2022-09-02 08:26:30 +01:00
Naz
88e0ae892c
Fixed typo 2022-09-01 11:01:15 +08:00
Daniel Lockyer
c2b399fc2c Fixed warning about aborted connection in tests
refs https://github.com/TryGhost/Toolbox/issues/389

- if we enable warning logs in E2E tests, we get a bunch of error
  messages saying `ERROR Unhandled rejection: aborted` coming from the
  SQLite DB reset code
- specifically, it's coming from the line that resets the DB by copying
  the file
- this line was initially added because we would see random SQLite
  "malformed database" errors
- I have a feeling that was due to something else, but I can't be sure
- I'm also not sure how else we should shut the DB connection, as this
  is the recommended way but it throws an unhandled rejection
- this commit is a bit of a gamble because I'm not actually sure what
  was causing the problem, but it gets rid of the errors locally and
  doesn't regress on the random failures
2022-08-31 12:15:30 +01:00
Simon Backx
f124d142c9 Added member attributions to activity feed (#15283)
refs https://github.com/TryGhost/Team/issues/1833
refs https://github.com/TryGhost/Team/issues/1834

We've added the attribution property to subscription and signup events when the
flag is enabled. The attributions resource is fetched by creating multiple relations
on the model, rather than polymorphic as we ran into issues with that as they can't
be nullable/optional.

The parse-member-event structure has been updated to make it easier to work with,
specifically `getObject` is only used when the event is clickable, and there is now a 
join property which makes it easier to join the action and the object.
2022-08-24 11:17:28 -04:00
Fabien "egg" O'Carroll
3c431bd8da Revert "Added member attributions to activity feed (#15283)"
This reverts commit e986b78458.

The tests were not passing for the PR and it was erroneously
merged into main
2022-08-24 11:01:47 -04:00
Simon Backx
e986b78458
Added member attributions to activity feed (#15283)
refs https://github.com/TryGhost/Team/issues/1833
refs https://github.com/TryGhost/Team/issues/1834

We've added the attribution property to subscription and signup events when the
flag is enabled. The attributions resource is fetched by creating multiple relations
on the model, rather than polymorphic as we ran into issues with that as they can't
be nullable/optional.

The parse-member-event structure has been updated to make it easier to work with,
specifically `getObject` is only used when the event is clickable, and there is now a 
join property which makes it easier to join the action and the object.
2022-08-24 10:11:25 -04:00
Rishabh Garg
9abfae2ddb
Switched off email alerts for subscription cancellation by default (#15304)
refs TryGhost/Team#1825

- all sites are expected to have cancellation alerts off by default
2022-08-24 18:54:00 +05:30
Rishabh Garg
c48c65cc88
Added member email alert notification columns for staff (#15276)
refs https://github.com/TryGhost/Team/issues/1825

- adds 3 new columns to users table for storing email alert preferences for member signups/cancellation
- adds column for new member signup alert
- adds column for paid subscription started alert
- adds column for paid subscription canceled alert
- Updated default fixtures and tests for new columns
2022-08-23 22:11:38 +05:30
Fabien "egg" O'Carroll
73466c1c40 Added ability to filter members on conversion attribution
refs https://github.com/TryGhost/Team/issues/1830
2022-08-23 11:36:56 -04:00
Fabien "egg" O'Carroll
83f2bf4757 Added ability to filter members on signup attribution
refs https://github.com/TryGhost/Team/issues/1831
2022-08-23 11:36:56 -04:00
Simon Backx
da24d13601
Added member attribution events and storage (#15243)
refs https://github.com/TryGhost/Team/issues/1808
refs https://github.com/TryGhost/Team/issues/1809
refs https://github.com/TryGhost/Team/issues/1820
refs https://github.com/TryGhost/Team/issues/1814

### Changes in `member-events` package

- Added MemberCreatedEvent (event, not model)
- Added SubscriptionCreatedEvent (event, not model) 

### Added `member-attribution` package (new)

- Added the AttributionBuilder class which is able to convert a url history to an attribution object (exposed as getAttribution on the service itself, which handles the dependencies)
```
[{
    "path": "/",
    "time": 123
}]
```
to
```
{
    "url": "/",
    "id": null,
    "type": "url"
}
```

- event handler listens for MemberCreatedEvent and SubscriptionCreatedEvent and creates the corresponding models in the database.

### Changes in `members-api` package

- Added urlHistory to `sendMagicLink` endpoint body + convert the urlHistory to an attribution object that is stored in the tokenData of the magic link (sent by Portal in this PR: https://github.com/TryGhost/Portal/pull/256).
- Added urlHistory to `createCheckoutSession` endpoint + convert the urlHistory to attribution keys that are saved in the Stripe Session metadata (sent by Portal in this PR: https://github.com/TryGhost/Portal/pull/256).

- Added attribution data property to member repository's create method (when a member is created)
- Dispatch MemberCreatedEvent with attribution

###  Changes in `members-stripe-service` package (`ghost/stripe`)

- Dispatch SubscriptionCreatedEvent in WebhookController on subscription checkout (with attribution from session metadata)
2022-08-18 17:38:42 +02:00
Daniel Lockyer
b27ea8f504
Optimized database resets for local testing
- we've had an optimization in CI that copies a fresh SQLite DB to a
  file, and copies it back when we need to do a DB reset
- I originally only let this run in CI but we've had it around for a
  while so we should GA it to run on local machines
- there may be edge cases, but we should fix them instead of letting
  tests run slower for development
- this also makes sure we clean up any existing files before
  initializing the DB
2022-08-15 16:58:56 +02:00
Aileen Nowak
77e4be6b81 Updated tests to incl. host limit cases for core integrations
- added core and builtin integrations to test fixtures
- allowed passing a custom api key id to generate JWT
- updated admin key auth test to make successful request with a `core` integration, which doesn't work atm because relations are not returned
2022-08-12 14:18:44 +01:00
Aileen Nowak
0e284edbad Allowed core integrations to work with Admin API 2022-08-12 14:18:44 +01:00
Naz
fa8d94fce2 Fixed the typo
refs e9bfc4ef01

- Did a typo in the find and replace... and now correcting a typo of a typo  -_-
2022-08-04 15:38:32 +01:00
Naz
e9bfc4ef01 Changed the lingo to US of A variation
refs 16728a3ef1

- It's 'merica time!
2022-08-05 02:28:33 +12:00
Daniel Lockyer
7cf4f595f7 🐛 Fixed missing published Admin assets when running in development
refs https://forum.ghost.org/t/admin-template-issues-default-install/31750

- we recently switched to using different folders within `core/built`, to
  indicate the assets that are applicable for development/production
  environments
- unfortunately, this came with the side effect of the "development" assets
  missing in the published tarball, which meant Admin wouldn't load when
  running in development mode
- this was a regression from how it previously worked because we used to
  just copy the production HTML file to the development HTML name, and
  use the same assets
- after thinking about it, I think we can get rid of the split folders
  for assets, because I don't think the use-case is there for having
  them:
  - if you run Ghost from source, you're 99% only using the
    development-built assets
  - if you want production ones, you can run with a flag, but the
    development ones get wiped anyway
  - those running Ghost from a published package are using the same
    assets and HTML file
- therefore, I think we can make our lives simpler by removing the env
  folders and using a folder under `core/built/admin/...`
- this commit implements that across Ghost and Admin
2022-08-04 10:55:35 +02:00
Kevin Ansfield
0a34be4012
Updated admin asset serving for ember-auto-import@2 compatibility (#15128)
refs https://github.com/TryGhost/Admin/pull/2252
closes https://github.com/TryGhost/Team/issues/1182

- Admin now copies it's build output to a single env-specific directory rather than splitting html and assets
  - `core/built/admin/{development|production}/*`
- updated the admin app's `serveStatic` definition for assets and controller's html serving to reflect the new asset paths
2022-08-02 13:43:45 +01:00
Aileen Booker
304b660c9d
Added migrations to change Ghost Explore integration type 2022-08-01 09:08:06 +02:00
Sam Lord
48550c81aa Improved performance of prepareContentFolder function
fixes https://github.com/TryGhost/Toolbox/issues/150

- up until now, the test framework has copied all theme fixtures to the
  test directory when it boots Ghost
- the vast majority of tests don't need all the themes, so this is quite
  a wasteful operation
- this commit disables copying all themes by default, and provides the
  `copyThemes` boot option to enable this
- also adds a `copySettings` option, and defaults `redirectsFile` to
  false to further reduce the number of file copies
2022-08-01 08:58:13 +02:00
Daniel Lockyer
4b5b3de11c
Moved test agents into subfolder
- these agent files are all very similar and should be grouped together
- moving them into a subfolder makes the `test/utils` folder a little
  bit easier to browse
2022-07-26 18:00:37 +02:00
Daniel Lockyer
044b342de3
Fixed random test failure due to asynchronous directory creation
- we keep seeing random failures that complain about a directory already
  existing when trying to create it
- the error arises from the `fs.copySync` in this `prepareContentFolder`
  function, because it tries to create the folder if it doesn't exist
- it turns out we're using the asynchronous `fs.ensureDir` without an
  await just before, so it doesn't block on creating the folder
- there's a veeeery small window where the code within `copySync` thinks
  the folder doesn't exist, `ensureDir` creates the folder, and
  then `copySync` tries to create the folder => 💥
- it looks like we're already `await`-ing `prepareContentFolder`, so we
  can just switch all the calls to the Promise-based ones and await them
- the other `prepareContentFolder` uses the sync versions of the
  functions, but we can fix that in the future: 7b7767d483/ghost/core/test/utils/e2e-utils.js (L73-L79)
2022-07-26 17:28:16 +02:00
Simon Backx
5235d67fed
Added comment events to activity feed (#15064)
refs https://github.com/TryGhost/Team/issues/1709

- New event type `comment_event` (comments and replies of a member in the activity feed)
- Includes member, post and parent relation by default
- Added new output mapper for ActivityFeed events

**Changes to `Comment` model:**
* **Only limit comment fetched to root comments when not authenticated as a user:** 
`enforcedFilters` is applied to all queries, which is a problem because for the activity feed we also need to fetch comments which have a parent_id that is not null (`Member x replied to a comment`). The current filter in the model is specifically for the members API, not the admin API (so checking the user should fix that, not sure if that is a good pattern but couldn’t find a better alternative).
* **Only set default relations for comments when withRelated is empty or not set:**
`defaultRelations`: Right now, for every fetch it would force all these relations. But we don’t need all those relations for the activity feed; So I updated the pattern to only set the default relations when it is empty (which we also do on a couple of other places and seems like a good pattern). I also updated the comments-ui frontend to not send ?include
2022-07-25 17:48:23 +02:00
Simon Backx
57a743e3aa
Fixed TXT content of report emails (#15090)
fixes https://github.com/TryGhost/Team/issues/1718

- Text content of report emails still had some placeholder text
- Converts HTML comments to TXT to include in the TXT version of the email
- Added support for Regexp matchers in the email mocker
- Added tests to check if the email content is in the new comment/report emails
2022-07-25 15:27:38 +01:00
Simon Backx
30c4f11e27
Added report API for comments (#15043)
closes https://github.com/TryGhost/Team/issues/1684

**Migrations:**
- Added report permissions (fixtures + migrations)
- Dropped reason field in reports (no textarea in reports in V1)
- Dropped nullable from comment_likes.member_id (can't be null)
- Added SET NULL/CASCADE foreign keys for comments related tables(*)

(*):
fixes https://github.com/TryGhost/Team/issues/1687
refs https://ghost.slack.com/archives/C02G9E68C/p1658217288591369

This commit adds support for `SET NULL` foreign keys in schema and migration helpers + also fixes the foreign keys for the comment_reports, comment_likes and comments tables.

- When a member is deleted, we **do** want to keep their reports (SET NULL)
- When a member is deleted, we **do not** want to keep their likes (CASCADE)
- When a member is deleted, we **do** want to keep the comments (SET NULL)

**Changes:**
- Added report API: `POST /members/api/comments/{id}}/report/`
- Sends an email to the owner when a comment is reported
- Saves a report to the database (not used for now, but might be useful later)
2022-07-22 12:03:05 +02:00
Daniel Lockyer
3d989eba23 Converted Ghost repo into a monorepo
refs https://github.com/TryGhost/Toolbox/issues/354

- this commit turns the Ghost repo into a monorepo so we can bring our
  internal packages back in, which makes life easier when working on
  Ghost
2022-07-20 16:41:05 +02:00