Commit Graph

1228 Commits

Author SHA1 Message Date
Fabien "egg" O'Carroll
605aa18d6a Added collections helper
This is an initial implementation which uses the Posts Content API rather than
the Collections Content API, this is because we haven't added the Collections
Content API yet, but we can added it later when necessary.
2023-07-17 11:35:30 +01:00
Chris Raible
b625fe76b1
Updated snapshots to include show_title_and_feature_image in event payload (#17378)
no issue

- Tests were failing for me locally because the snapshot for testing
page event payloads didn't include the new
`show_title_and_feature_image` property
- Updated snapshot to include this property, which also changed a few content lengths and `x-cache-invalidate` headers as a side effect
2023-07-14 16:03:40 -07:00
Naz
382d7fa25f Removed unused collections repositories
no issue

- These repositories were leftover from first phases of collections development. Not needed any longer as we have more specialized bookshelf repositories in the core code
2023-07-14 14:20:37 +08:00
Kevin Ansfield
3fdf5adbfd
Moved show_title_and_feature_image to @page.show_title_and_feature_image (#17353)
no issue

- updated frontend's `formatResponse` method to add `@page` to the local template data
  - added here because it's the first place we have both context and page data available
  - makes the property available outside of the page context so it can be used to modify layouts
- updated `prepareContextResource()` to delete `show_title_and_feature_image`
  - ensures `@page.show_title_and_feature_image` is the only way of accessing that property
  - method is used when rendering multiple entries so it also prevents differences in context data between looped pages and single pages
2023-07-13 14:26:33 +00:00
Steve Larson
42d87d1437
Updated koenig lexical packages (#17325)
no refs
-updated to get collections card changes
2023-07-12 10:59:20 -05:00
Naz
cdae2a978d Added ability to fetch collection posts by slug
closes https://github.com/TryGhost/Arch/issues/27

- We need a more convenient method of fetching posts belonging to a collection than by collection's "id". This change adds an alias to the existing endpoint `GET /collections/:id/posts/`. A non-valid ObjectID in the parameter is treated as a slug.
2023-07-11 09:30:01 +08:00
Daniel Lockyer
6f5f608bcd Fixed resetting database in migrations tests
- due to schema changes between versions, we need to completely nuke the
  DB between these tests
- this is definitely not the best way to do it but I'll fix properly next week
2023-07-07 18:42:18 +02:00
Kevin Ansfield
d8d0bc8bd9
Moved posts_meta.hide_title_and_feature_image to posts.show_title_and_feature_image (#17239)
no issue

`show_title_and_feature_image` leads to more intuitive logic in themes and we can use `posts` rather than `posts_meta` as there are no longer row-length issues with MySQL 8.

- removed original add-column migration that was never in a release
- added new add-column migration that puts
`show_title_and_feature_image` column with a default of `true` on the `posts` table
- renamed property and default value everywhere
- bumped `@tryghost/admin-api-schema` to allow the new property through at the API level
2023-07-07 16:40:22 +01:00
Fabien "egg" O'Carroll
2808df7f87 Switched to bookshelf repository 2023-07-07 14:24:42 +08:00
Chris Raible
fc7e150cc2
Fixed importer breaking any posts written in the new editor (#17230)
refs @TryGhost/Product#3551

- PostsImporter would convert the HTML from the import file into
Mobiledoc, even if the post was written in Lexical
- As a result, the imported posts would have both mobiledoc & lexical
fields populated, which prevents the post from being updated in the
Lexical editor
- Added a check to see if the post was written in Lexical, and if so,
skip the HTML > Mobiledoc conversion
2023-07-06 18:37:19 -07:00
Kevin Ansfield
5d8ea79788
Implemented pages.hide_title_and_feature_image property in API (#17211)
closes https://github.com/TryGhost/Product/issues/3557

- bumped `@tryghost/admin-api-schema` to allow passthrough of the new property in API requests
- updated output mapper to ensure property always returns a boolean rather than `null` in the case where `posts_meta` doesn't exist for a page
- updated `PostsService.copyPost()` to include the new property when copying
- updated `checkResponse` test util and snapshots to expect `hide_title_and_feature_image` property in page API responses
- fixed pages e2e test so it doesn't inadvertently modify the match object breaking later tests
2023-07-05 14:01:24 +00:00
Kevin Ansfield
2a340bcab9
Added migration and schema change for posts_meta.hide_title_and_feature_image (#17187)
closes https://github.com/TryGhost/Team/issues/3550

We want to allow an option to hide the title and feature image on a
per-page basis, to do that we need somewhere to store the setting value.
The existing `posts_meta` table is the simplest candidate, especially as
this is a single setting and we don't have a desire to introduce many
such settings.

- added migration that adds the `hide_title_and_feature_image` column to
the `posts_meta` table with a `boolean` data type and a default value of
`false` (matches behaviour of all existing pages)
- updated schema file for initial database creation
- removed property from API output via serializers to keep migration PR
minimal
2023-07-05 14:48:25 +01:00
Daniel Lockyer
289e459283 Added tests for migrations
refs https://github.com/TryGhost/DevOps/issues/39

- up until now, we've had a CI job which does a really basic test for
  migrations, but it barely functions and misses bugs all the time
- this commit removes that and switches to an actual test suite for our
  migrations, so we can ensure they function as expected
- also removes the env var hack I came up with for those migrations
  tests
- this should lead to safer migrations and faster tests
2023-07-05 11:00:06 +02:00
Simon Backx
1b33634495
🐛 Fixed sending newsletters with counter-reset CSS styles in custom HTML (#17194)
fixes https://github.com/TryGhost/Team/issues/2937

Bumps juice to 9.1.0:
- Support for 'auto' width and height attributes
- Fixed a bug with counter-reset styles
- Dependencies updates
2023-07-04 10:18:16 +02:00
Simon Backx
8b610d46b1
Cleaned up batch email utils (#17165)
no issue
2023-06-29 15:29:16 +02:00
Simon Backx
c90424542e
🐛 Fixed signup card in post plaintext and email preheader (#17163)
fixes https://github.com/TryGhost/Team/issues/3542

The signup card text was included in the post plaintext/excerpt and
email preheader
2023-06-29 14:45:30 +02:00
Simon Backx
e6dbc0bc4c
🐛 Fixed repeating text in plaintext version of emails (#17162)
fixes https://github.com/TryGhost/Team/issues/3541

The email preheader, which is only present in the html version of an
email, is also included in the plaintext version of all emails. This
results in all text being duplicated twice in plaintext emails.
2023-06-29 12:47:17 +02:00
Simon Backx
c41694f5e6
Added E2E Koenig Lexical card tests for paragraphs and signup card (#17159)
refs https://ghost.slack.com/archives/C04TMVA1D7A/p1687987311455289
2023-06-29 11:10:56 +02:00
Fabien "egg" O'Carroll
cf83d169db Added use of a slug generator to offload calculation of slugs
When we end up wiring this to the database, this generator will also ensure
uniqueness by appending/incrementing a number on the end of the slug. Long term
it would be good to offload this to a shared slug service, this could also
ensure that slugs are unique globally or between multiple tables, if desired
2023-06-29 01:40:05 +01:00
Michael Barrett
636c916715
Fixed leaking pivot fields (#17142)
fixes https://github.com/TryGhost/Team/issues/2657

The `omitPivot` option does not have an affect on a models
`_previousAttributes`. When we serialise a model and want to retrieve
the previous attributes we need to ensure we manually remove the pivot
fields

See
7704fbc5e8/lib/base/model.js (L512)
2023-06-28 13:16:50 +01:00
Naz
f74588f9c2 Fixed date filtering in collections
refs https://github.com/TryGhost/Team/issues/3169

- To be able to apply NQL filtering on Collection Posts the dates should be serialized to be ISO Date Strings instead of raw Dates. Otherwise, NQL filtering fails to compare Date with a Date String.
2023-06-27 12:52:10 +07:00
Kuba
9601285c3d
Added bulkEmail.batchSize option to configure batch size
refs https://github.com/TryGhost/Ghost/issues/15725

This pull request adds a new configuration option for the Mailgun email
provider that allows the user to set the maximum number of recipients
per email batch via a new config option `bulkEmail.batchSize`
2023-06-26 12:57:44 +02:00
Fabien "egg" O'Carroll
bd6e476c57 Added tables to persist Collections
refs https://github.com/TryGhost/Team/issues/3427
2023-06-23 16:11:55 +02:00
Simon Backx
78405a9cb8
Corrected image CORS (#17119)
refs
018f97e679
2023-06-23 14:33:00 +02:00
Michael Barrett
6f5baca849
Add endpoint to record mail events (#16990)
refs https://github.com/TryGhost/Team/issues/3319
2023-06-23 12:22:01 +01:00
Daniel Lockyer
08e2fbeacb Removed Bluebird from tests
refs https://github.com/TryGhost/Ghost/issues/14882

- we're moving away from using Bluebird in favor of native Promises, so
  this commit removes nearly all instances from tests
2023-06-23 10:03:03 +02:00
Princi Vershwal
2798d43663
Removed Bluebird Promise.each from db.utils (#17032)
refs https://github.com/TryGhost/Ghost/issues/14882

Co-authored-by: Princi Vershwal <princi.vershwal@Princis-MacBook-Pro.local>
Co-authored-by: Daniel Lockyer <hi@daniellockyer.com>
2023-06-23 09:46:41 +02:00
Princi Vershwal
308a3b286a
Replaced Bluebird Promise.mapSeries with sequence util (#17008)
refs https://github.com/TryGhost/Ghost/issues/14882

Co-authored-by: Princi Vershwal <princi.vershwal@Princis-MacBook-Pro.local>
2023-06-23 09:04:37 +02:00
Simon Backx
8d6fb51908 Added Playwright tests to comments-ui
refs https://github.com/TryGhost/Team/issues/3504

Not complete yet, but contains the basic structure and a few tests that work and should run in CI.
2023-06-22 15:06:13 +02:00
Hannah Wolfe
6161f94910
Updated to use assert/strict everywhere (#17047)
refs: https://github.com/TryGhost/Toolbox/issues/595

We're rolling out new rules around the node assert library, the first of which is enforcing the use of assert/strict. This means we don't need to use the strict version of methods, as the standard version will work that way by default.

This caught some gotchas in our existing usage of assert where the lack of strict mode had unexpected results:
- Url matching needs to be done on `url.href` see aa58b354a4
- Null and undefined are not the same thing,  there were a few cases of this being confused
- Particularly questionable changes in [PostExporter tests](c1a468744b) tracked [here](https://github.com/TryGhost/Team/issues/3505).
- A typo see eaac9c293a

Moving forward, using assert strict should help us to catch unexpected behaviour, particularly around nulls and undefineds during implementation.
2023-06-21 09:56:59 +01:00
Simon Backx
0c28cebfd4 Updated rate limiting snapshot for updated copy
refs https://ghost.slack.com/archives/C02G9E68C/p1687303342271029
2023-06-21 10:36:53 +02:00
Fabien 'egg' O'Carroll
41cbc40353
Moved collections initialisation behind labs flag (#17057)
We ran into an issue where the large `published:true` query was
affecting the boot time of large sites which had knock-on effects with
availability.
2023-06-19 16:40:15 +02:00
Daniel Lockyer
99aeb73ecc Optimized DB reset during tests
refs https://github.com/TryGhost/Toolbox/issues/592

- it turns out that `TRUNCATE` in CI takes ~300ms for all tables, but
  `DELETE FROM` takes ~30ms
- whilst truncating is generally known to be faster, I believe it's only
  faster on large tables
- this saves 90% of the time it takes to reset the DB in MySQL
2023-06-16 13:15:17 +02:00
Naz
09264261b7 Added ability to fetch posts by collection
closes https://github.com/TryGhost/Team/issues/3423

- For convenience we need a way to fetch posts that belong to a certain collection. This change adds support for `collection` query parameter: `/?collection=` which can be either an id or slug of the collections we are trying to fetch.
- When posts are fetched by collection we ignore any filters passed along in query parameters as collection is a "filter" by it's very nature.
2023-06-16 17:28:12 +07:00
Ronald Langeveld
cfbc97b033
🐛 Fixed revisions relation not linked to Posts api (#17037)
no issue

This was a bit of an oversight from our feature built at the retreat. We
didn't take revisions into account for pages at all, but luckily it made
revisions without issues regardless.
It just wasn't accessible and users weren't able to restore via ADMIN
because the API didn't serve them at all.

This wires up the revisions relation to be served by the API so we can
retrieve it in Admin.
2023-06-16 09:49:12 +02:00
Fabien "egg" O'Carroll
0d7f98f4d1 Supported adding/removing post to collection via Posts API
We've got some fairly simple diffing logic here to update the collections which
a post is in, the bulk of the changes here are to support the return of a DTO
rather than Bookshelf Model. This also helps improve the architecture because
we are step closer to removing infrastructure concerns (HTTP Response Headers)
from the business logic layer.

For now there is a crappy EventString which can be passed back to the
controller which can then handle any HTTP related concerns, although long term
these should be actual events like PostPublished or PostUpdated.
2023-06-16 09:28:19 +02:00
Fabien "egg" O'Carroll
f3f9e5a2f3 Moved serialisation of formats into the serialiser-layer
This prepares us to return a DTO rather than BookshelfModel to the serialiser
layer. When passing a BookshelfModel, the serialisation layer uses the model to
read from when building computed properties. By stripping values out in the
toJSON method it means that the DTO will be missing them and the computed
properties won't be able to be calculated. Instead we return ALL values to the
serialisation layer, and then strip out the ones that weren't requested in the
"clean" step.

This also inadvertently fixes the issue with `reading_time` requiring the
`html` field to be requested, we can now request just `reading_time`, as well
as have it included by default.
2023-06-16 09:17:47 +02:00
Fabien "egg" O'Carroll
41716a06ae Fixed mutation of shared state for matcher
This can cause bugs due to the matcher constraints changing in each test
2023-06-16 09:17:47 +02:00
Naz
6335033466
Added full post data when returning collection's posts
refs https://github.com/TryGhost/Team/issues/3423

- When querying for posts that belong to a collection we should be returning full post information just like we do for Posts API.
2023-06-15 18:35:58 +07:00
Aileen Booker
d27bb8f075 Added new Stripe events to DomainEventsAnalytics
no issue

- We need to send information about Stripe being enabled or disabled in live mode to analytics
- This hooks up the Domain events listeners in the analytics service and processes this information accordingly
2023-06-14 08:18:19 -04:00
Naz
34f7b77190
Fixed collections snapshot
no issue

- The snapshot has save a wrong amount of "pages" for collections and was falsely passing after a retry. The correct amount should be 6.
2023-06-14 14:17:23 +07:00
Daniel Lockyer
01e90dfb65 Extracted regression tests to a separate workflow
refs https://github.com/TryGhost/Toolbox/issues/592

- this commit extracts the regression tests into a separate workflow
- this means they run in parallel and reduce the time we have to wait
  for DB tests in general
- also fixes a test that was reliant on being run after the E2E
  tests (!)
2023-06-13 16:47:19 +02:00
Daniel Lockyer
082ab6dc3e Refactored truncateAll util to async-await
refs https://github.com/TryGhost/Toolbox/issues/592

- async-await makes the code easier to read
- also performs a small optimization to only load the foreign_keys
  pragma once for SQLite
2023-06-13 14:46:57 +02:00
Daniel Lockyer
264773ccd1 Reset URL service between test boots
refs https://github.com/TryGhost/Toolbox/issues/592

- we should reset the URL service to avoid event listeners piling up and
  slowing down CI due to the number of events it has to process
2023-06-13 12:52:03 +02:00
Daniel Lockyer
cedfb14924 Fixed random timeouts with mysql2 library during tests
refs https://github.com/TryGhost/Toolbox/issues/592

- heads up, I'm not really sure about this fix
- when we're wrapping `setTimeout`, time stops and mysql2 starts doing
  weird things because we then shift time and it hits timeouts
- apparently `shouldAdvanceTime` should fix this by automatically
  incrementing time along with the system clock
- given the problem is quite difficult to hit, I could just be seeing a
  lack of this due to some other factor
- also removed unnecessary sinon sandbox creation as this is superfluous
2023-06-13 12:52:03 +02:00
Naz
7717021331 Added GET /collections/:id/posts endpoint
refs https://github.com/TryGhost/Team/issues/3423

- The Admin API endpoint allows to browse collection posts and paginate over them
2023-06-13 17:23:09 +07:00
Daniel Lockyer
af8c0dc7a5 Prevented registering multiple Slack event listeners
refs https://github.com/TryGhost/Toolbox/issues/592

- in tests, we boot Ghost over and over
- this inits all the services each time
- it turns out that the Slack event listener is registered 80+ times
- to prevent this, we can check if it has already been registered, like
  we do with webhooks: 4639396c3a/ghost/core/core/server/services/webhooks/listen.js (L67-L69)
- not sure about this pattern in general, but it's something we can
  review in coming weeks
2023-06-12 15:04:11 +02:00
Daniel Lockyer
4639396c3a
Collected Ghost boot timing in test framework
refs https://github.com/TryGhost/Toolbox/issues/592

- this should help us track and calculate how much time is being spent
  in the boot/DB reset process during tests
2023-06-12 12:07:04 +02:00
Daniel Lockyer
3ac28c7a95
Fixed snapshot for settings
- this deviated somewhere, which I am investigating
2023-06-12 10:04:50 +02:00
Naz
bb4e2fcadb
Added "index" built in collection
closes https://github.com/TryGhost/Team/issues/3425

- Index collection is needed to support one of the usecases we have in the near future where we'd hold all posts that would be displayed on the "index" page.
2023-06-12 12:57:52 +07:00
Naz
6421e202e5
Removed API endpoints adding posts to collections
closes https://github.com/TryGhost/Team/issues/3431

- We don't currently have a clear usecase to use the new pattern of updating posts as nested resource (of a collection). To simplify the API we are sticking with the approach of controlling where the post belongs to only through the Posts Admin API.
2023-06-12 12:08:37 +07:00
Naz
05bb82cf1a Added collections CRUD permissions
refs https://github.com/TryGhost/Team/issues/3220

- Added permissions for collection resources
2023-06-09 21:13:06 +07:00
Michael Barrett
9da246ac84
Enforce explicit cacheInvalidate header (#16826)
refs https://github.com/TryGhost/Team/issues/3005
2023-06-07 15:13:45 +01:00
Fabien 'egg' O'Carroll
92172aca8e
Wired up collections to posts endpoint (#16945)
- Added support for `include=collections` to the Posts Admin API behind a flag
- Refactored some of the collections work to support it
2023-06-07 15:06:15 +02:00
Aileen Booker
9372124d61 Decouple sentry from analytics events handlers by expecting general exception handler
no issue

- The class should not rely on being passed a specific dependency, but rather needs to communicate with types what structure and method it needs to function correctly.
- Replaced the specific dependency to `sentry` with a generic definition of what is expected.
2023-06-07 09:00:49 -04:00
Aileen Booker
5db1950090 Added tests ModelEventsAnalytics
no issue

- Added full unit tests for `ModelEventsAnalytics` class
2023-06-07 09:00:49 -04:00
Aileen Booker
6efc1cce82 Added JSDoc types and inject more deps for DomainEventsAnalytics
no issue
2023-06-07 09:00:49 -04:00
Aileen Booker
591ca2c9bf Added first test structure for ModelEventsAnalytics
no issue

- The added `ModelEventsAnalytics` class didn't have tests before the refactor. Added unit tests to verify functionality.
2023-06-07 09:00:49 -04:00
Aileen Booker
9b27804276 Added tests for DomainEventsAnalytics
no issue

- Added unit tests for newly added `DomainEventsAnalytics` class to verify intended functionality
2023-06-07 09:00:49 -04:00
Aileen Booker
2c4d9e2776 First pass on adding domainevents to segment service
no issue

- In order to listen to `DomainEvents` for `MilestoneCreatedEvents` we need to add a `DomainEvents` listener and handler to the Segment analytics service.
- For better readability and to be more consistent with how code is currently written in Ghost, I refactored the service index file and split the two types of event listener into separate classes which is much cleaner and easier to test.
2023-06-07 09:00:49 -04:00
Naz
fab5b1845c
Made builtin collections un-deletable
closes https://github.com/TryGhost/Team/issues/3376

- It should not be possible to delete a built-in collection.
2023-06-06 22:04:16 +07:00
Michael Barrett
a1b08f15b2
Fixed flaky frontend members test (#16947)
closes https://github.com/TryGhost/Team/issues/3325

Awaited `DomainEvents.allSettled()` to ensure domain event is fully
processed before asserting member was successfully updated
2023-06-06 14:57:46 +01:00
Rishabh Garg
5c209abdc0
Updated default value for image editor integration setting (#16897)
refs https://github.com/TryGhost/Team/issues/3145

Updates pintura integration to be switched on by default for all sites by adding a migration to update the default value for the setting.
2023-06-06 18:15:01 +05:30
Naz
5a8eec402f
Restricted built-in collection init to run once
refs https://github.com/TryGhost/Team/issues/3376
fixes b4a97d084f

- The in-memory stores are not cleaned up when the Ghost instance is "shallow restarted" between test suite runs, causing the initialization of built-in collections to run multiple times. The initialization should ever add the collections once.
2023-06-06 13:35:06 +07:00
Naz
b4a97d084f
Fixed snapshot for failing collections test
refs bdbd9327d9

- The "build" command was not run for the @tryghost/collecitons package and caused outdated response
2023-06-06 12:50:55 +07:00
Naz
bdbd9327d9
Added built-in automatic featured collection
refs https://github.com/TryGhost/Team/issues/3376

- When the Ghost instance is initialized it has to have a set of built-in collections. With these changes Ghost starts with a "featured posts" collection - available to be used right away.
2023-06-06 12:41:37 +07:00
Naz
c86859d0a7
Added test coverage to automatic collections
refs https://github.com/TryGhost/Team/issues/3170

- The test confirms relational filters like `tag:kitchen-sink` filtering works for automatic collections
2023-06-05 16:37:17 +07:00
Naz
260ca0548a
Added test coverage to automatic collections
refs https://github.com/TryGhost/Team/issues/3170

- The test confirms `published_at` filtering works for automatic collections
2023-06-05 14:59:49 +07:00
Naz
bfefcfd4df
Added automatic collection creation based on filter
refs https://github.com/TryGhost/Team/issues/3170

- This implementation allows to create an automatic collection with a filter defining automatically populated posts that belong to a collection
- To populate collection using a filter the API client can send a `filter` property along with a collection request
- Filter values are compatible with the filters used in Content API (https://ghost.org/docs/content-api/#filter)
2023-06-05 13:39:52 +07:00
Naz
f64276b26e
Added basic test an automatic collection
refs https://github.com/TryGhost/Team/issues/3170

- This test confirms we can create a collection with "automatic" type and assign a "filter" value to it
2023-06-05 13:39:49 +07:00
Daniel Lockyer
f908a03fff
Enabled and tracked retries in CI tests (#16909) 2023-06-02 09:36:28 +02:00
Michael Barrett
09ded37d91
Removed emailErrors flag (#16900)
refs https://github.com/TryGhost/Team/issues/3337

Removed emailErrors flag as it is no longer needed
2023-06-01 12:11:01 +01:00
Simon Backx
7e27d3f3e8
Added signup form attribution (#16899)
fixes https://github.com/TryGhost/Team/issues/3331

This adds attribution tracking to the signup form. It sends a newly
created url history when sending the signup API call, this url history
will get translated to a proper attribution and saved on the backend. We
send a history with only a single item that contains the referrer
source, medium and path of the Embed form.

This also makes some changes to the E2E tests so that the tests run
in an https environment instead of about:blank.
2023-06-01 10:18:11 +02:00
Naz
e8220b1387
Added DEL /collections/:id/posts/:post_id to Admin API
refs https://github.com/TryGhost/Team/issues/3260

- We need a way to remove posts form collections without fetching the whole collection's content. This API method allows to remove posts from manual collections by collection id and post id.
- As a response it returns up to date collection state without the removed post.
2023-06-01 14:59:05 +07:00
Fabien "egg" O'Carroll
93bad82a24 Wired up adding post to collections to API 2023-06-01 14:25:49 +07:00
Naz
bf5becfb22 Added POST /collections/:id/posts to Admin API
refs https://github.com/TryGhost/Team/issues/3260

- We need a way to append posts to collections without sending over all of the posts that are already in the collection
- The API would receive post_id and collection_id as required fields and will optionally take in sort_order to control the ordering in the manual collection
2023-06-01 14:25:49 +07:00
Naz
87df8754ee Moved bookshelf posts repository to core
refs https://github.com/TryGhost/Team/issues/3260

- Moved the posts bookshelf repository into core codebase where it should belong.
2023-05-31 22:55:35 +07:00
Naz
4fe9e5fac0 Added posts editing to collections in Admin API
refs https://github.com/TryGhost/Team/issues/3260

- Allows to manually manage posts assigned to collections through Collections Admin API
2023-05-31 22:55:35 +07:00
Naz
c7954fa695 Refactored basic edit test case
refs https://github.com/TryGhost/Team/issues/3260

- We need to have a reusable collection to edit throughout the describe, so we don't do repeatable work over and over
2023-05-31 22:55:35 +07:00
Naz
988f7e69c2 Grouped edit collection test cases
refs https://github.com/TryGhost/Team/issues/3260

- Cleanup before adding more test cases to the suite
2023-05-31 22:55:35 +07:00
Michael Barrett
88f3161903
Fixed flaky legacy posts api test (#16871)
refs https://github.com/TryGhost/Team/issues/2808

Updated the test to ensure that the date assertions do not unexpectedly
fail if the dates used are computed precisely at the start of a second
(no milliseconds `.000Z`)
2023-05-25 15:40:17 +01:00
Michael Barrett
8fc4e0fdcc
Add e2e test for update check script (#16840)
refs https://github.com/TryGhost/Team/issues/3234

Added an e2e for the update check script to detect potential breakages
in the script due to uninitialised dependencies in the isolated
execution environment
2023-05-25 14:36:36 +01:00
Simon Backx
488aa983d0 Added signup form code generation to admin
fixes https://github.com/TryGhost/Team/issues/3296

Adds a new `signupForm` feature flag, that will enable/disable the new embeddable signup form code generation.

Since the new flag shares its name with a new config value (that contains the script location), this also fixes the feature helper to only use a config with the same name if it is a boolean.
2023-05-25 15:18:40 +02:00
Naz
4cb5cc9087
Added default collection values
refs https://github.com/TryGhost/Team/issues/3259

- For collection entity creation consistency have set defaults
2023-05-24 17:01:28 +07:00
Naz
b52ec948b0
Added property mapper to collections responses
closes https://github.com/TryGhost/Team/issues/3259

- API output mappers (soon to be serializers) are meant to work based on allowlist set of output properties. Having the allowlist early on will allow to track the API evolution consistently.
2023-05-24 17:01:28 +07:00
Fabien "egg" O'Carroll
82acf85b29 Tested filtering visibility in Tiers Content API
refs https://github.com/TryGhost/Team/issues/3248

The current test fixtures didn't include any hidden Tiers, so I've added
a new fixture to test the filtering of hidden Tiers. It's not enabled by
default to avoid breaking the existing tests.
2023-05-19 13:12:33 -04:00
Naz
e302f8cc1d Added DELETE /collections/id to Admin API
refs https://github.com/TryGhost/Team/issues/3167

- This is part of scaffolding for collections API. Allows to delete collection resource
2023-05-19 20:42:46 +07:00
Naz
e82fcbfc5e Added GET /collections/:id to Admin API
refs https://github.com/TryGhost/Team/issues/3167

- This is part of scaffolding for collections API. Allows to read a collection resource by id
2023-05-19 20:42:46 +07:00
Naz
735edf5f87 Fixed editing unexistent collection behavior
refs https://github.com/TryGhost/Team/issues/3167

- When editing collection that does not exist the API should be returning a 404 instead of creating a new collection
2023-05-19 20:42:46 +07:00
Naz
f3f3d58acf Added PUT /collections/id to Admin API
refs https://github.com/TryGhost/Team/issues/3167

- This is part of scaffolding for collections API. Allows to edit collection resource
2023-05-19 20:42:46 +07:00
Naz
d3a8aad319 Added POST /collections to Admin API
refs https://github.com/TryGhost/Team/issues/3167

- This is part of scaffolding for collections API. Allows to add new collection records
2023-05-19 20:42:46 +07:00
Naz
36eff3a481 Added GET /collections to Admin API
refs https://github.com/TryGhost/Team/issues/3167

- This is scaffolding for collections API. Contains wiring for service wrapper, e2e test, and a browse endpoint
- Adds basic implementation of the GET /collections endpoint to build up upon
- Note, there are no permissions in this version as they will be added in later stages of development with migrations etc
2023-05-19 20:42:46 +07:00
Michael Barrett
cde30eb469
🐛 Fixed issue where single letter product slugs cause 500 error (#16821)
refs https://github.com/TryGhost/Team/issues/3224

When a product has a slug that is a single letter, checking if a user
had access to view a post associated with that product would cause a 500
error. The underlying cause of this issue is
https://github.com/TryGhost/NQL/issues/20 This fix circumvents this
issue by providing a value that the nql lexer will not error out on
2023-05-18 09:38:30 +01:00
Benjamin Rancourt
5841f30d30
🐛 Fixed trailing slash and space in HTML metadata elements (#16778) 2023-05-17 08:51:32 +02:00
Rishabh
f9866f97ae Updated config test to include adminX property
refs https://github.com/TryGhost/Team/issues/3151
2023-05-16 13:10:00 +05:30
Michael Barrett
59fe794b0c
Implemented duplicate post functionality (#16767)
refs: https://github.com/TryGhost/Team/issues/3139 https://github.com/TryGhost/Team/issues/3140

- Added duplicate post functionality to post list context menu
  - Currently only a single post can be duplicated at a time
  - Currently only enabled via the `Making it rain` flag
- Added admin API endpoint to copy a post - `POST ghost/api/admin/posts/<post_id>/copy/`
- Added admin API endpoint to copy a page - `POST ghost/api/admin/pages/<page_id>/copy/`
2023-05-15 09:30:32 +01:00
Fabien "egg" O'Carroll
08597b47ba Added unit test for TiersRepository
We were completely missing tests for this, and the new logic pushed us under
the coverage threshold.
2023-05-12 16:38:32 -04:00
Fabien "egg" O'Carroll
c0ca7b16f6 Added caching to TierRepository
refs https://github.com/TryGhost/Toolbox/issues/515

Tiers are very frequently queried and we want to reduce the number of DB calls
we're making. We can store the Tiers in-memory, using the existing in-memory
repository patterns, but still persisting writes the the database.

We also have to update our test helpers, because they were bypassing the
repository for writes, but using it for reads resulting in an invalid cache
2023-05-12 16:38:32 -04:00
Fabien "egg" O'Carroll
6ab862568c Used TierRepository in serializers
We want to cache access to Tiers, and it's easier to do that in the
TierRepository. So we update a heavy user of Tiers to use the Tier
service so it can take adv of caching. The serializers are a big
offender for making calls to fetch Tiers.
2023-05-12 16:38:32 -04:00
Fabien 'egg' O'Carroll
13a18711d0
🐛 Fixed site setup hanging when mail isn't configured
closes https://github.com/TryGhost/Team/issues/3176

We were waiting for the welcome email to send before responding to the
client that setup is complete, this was causing the client to hang when
running `ghost install local` as mail isn't configured by default.
2023-05-11 11:58:27 -04:00
Deepam Kapur
3a39aa361b
🐛 Fixed generating card assets with include allowlist (#16766)
fixes https://github.com/TryGhost/Ghost/issues/16652

- we need `@` keyword before '(' according to the globrex package used in tiny-glob
2023-05-11 14:20:35 +02:00
Sodbileg Gansukh
4898fd401b Updated signup confiramtion email snapshot
no issues

- updated the snapshot based on the changes added to the email template
2023-05-10 18:18:10 +08:00
Fabien "egg" O'Carroll
c5dc5a2675 Fixed @tryghost/mailgun-client import
We were incorrectly specifying the path of the file rather than relying on the
package exports, and this broke in the previous commit to a file rename. It
wasn't caught by CI because the browser tests are not run on pull requests
2023-05-09 13:00:18 -04:00
Fabien "egg" O'Carroll
104f84f252 Added eslint rule for file naming convention
As discussed with the product team we want to enforce kebab-case file names for
all files, with the exception of files which export a single class, in which
case they should be PascalCase and reflect the class which they export.

This will help find classes faster, and should push better naming for them too.

Some files and packages have been excluded from this linting, specifically when
a library or framework depends on the naming of a file for the functionality
e.g. Ember, knex-migrator, adapter-manager
2023-05-09 12:34:34 -04:00
Fabien 'egg' O'Carroll
0b8c3747c5
Supported inviting users using an Admin API Integration
Whilst Admin API Integrations had the permissions to create invites they were
blocked from doing so at the HTTP level. We've removed this restriction for
creating Invites as well as browsing Roles, because a Role ID is necessary to
create an invite. The code was also not setup to support Admin API Integrations
as it made assumptions about the existence of a User. That has been updated in
the permissions layer - so that the Invites are limited to Contributors,
Authors and Editors as well as at the email layer, which has has the copy and
from address updated to reflect the lack of a User creating the Invite.
2023-05-08 15:27:15 -04:00
Simon Backx
6566903df5
Cleaned up member attribution flag (#16745)
no issue

This commit removes the `memberAttribution` feature flag from the
codebase. Some CSS classes are not removed as removing them and updating
the associated CSS files have side effects sadly.
2023-05-05 15:04:14 +02:00
Elena Baidakova
4207c9d0d1
Added browser tests for announcement bar (#16742)
refs TryGhost/Team#3122

<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 87727d9</samp>

Added `data-testid` attributes to various elements in the announcement
bar settings feature to enable Playwright testing. Fixed a potential bug
with the `visibilitySettings` getter in the `visibility.js` component.
Added Playwright tests for the announcement bar settings feature in
`announcement-bar-settings.spec.js`.
2023-05-05 14:11:26 +04:00
Simon Backx
05bba5135d
Cleaned up sourceAttribution flag (#16740)
no issue

This commit removes the `sourceAttribution` feature flag from the
codebase.
2023-05-05 10:57:26 +02:00
Chris Raible
27e4523aec
🐛 Improved error message for unauthorized YouTube embeds (#16374)
refs TryGhost/Ghost#16048

- When attempting to embed a Youtube video that has had embedding
disabled by its owner/author, Ghost displayed a generic error message
that didn't indicate the reason for the failed emebed.
- This change updated the error message when Youtube (or any provider)
returns 401: Unauthorized to indicate that the owner of the resource has
explicitly disabled embedding.
2023-05-04 16:04:58 -07:00
Simon Backx
848b2d82a1
Cleaned up suppressionList feature flag (#16736)
no issue

This pull request removes the `suppressionList` feature flag and all its
dependencies from the codebase. It makes the suppression list feature
the default and consistent behavior for all email events and
newsletters. It simplifies the UI, logic, and data related to email
events and newsletters. It affects several files in the
`ghost/admin/app`, `ghost/core/core`, and `ghost/members-api`
directories.
2023-05-04 14:47:04 +02:00
Fabien "egg" O'Carroll
b3caf16005 🔒 Fixed filtering on private Author fields in Content API
refs https://github.com/TryGhost/Ghost/security/advisories/GHSA-r97q-ghch-82j9

Because our filtering layer is so coupled to the DB and we don't generally
apply restrictions, it was possible to fetch authors and filter by their
password or email field. Coupled with the "starts with" operator this can be
used to brute force the first character of these fields by trying random
combinations until an author is included in the filter. After which the next
character can be brute forced, and so on until the data has been leaked
completely.
2023-05-03 08:43:20 -04:00
Simon Backx
514c8917c0 Readded matchMetadataSnapshot
refs bac2f4d4d4 (diff-473dc0c49e80df6c07569ed5730445ebdaaa8138fc819182548727db50eb55aa)
2023-05-03 14:14:10 +02:00
Simon Backx
fd69ab181d Removed duplicate matchHTMLSnapshot 2023-05-03 14:13:04 +02:00
Simon Backx
bac2f4d4d4 Fixed snapshot tests for MySQL only newsletter test
no issue

There was an error when generating the snapshot for this test. It never ran, so the snapshot was never committed. On top of that, the generated snapshot would change every time because the email verification token was not replaced with a static value.
2023-05-03 14:05:53 +02:00
Elena Baidakova
ef25e8dda7
Fixed tests (#16718)
no issue
2023-04-27 21:05:01 +04:00
Simon Backx
51473b3f7d Added tests for pages bulk API
fixes https://github.com/TryGhost/Team/issues/2925
2023-04-27 15:06:19 +02:00
Simon Backx
9fc98417b5 Added bulk post unpublishing test
refs https://github.com/TryGhost/Team/issues/2925
2023-04-27 14:58:27 +02:00
Naz
06c0a19718 Moved announcement bar settings to Frontend Members API
https://github.com/TryGhost/Team/issues/3121

- Because the announcement data has to be available with member's context, it's only possible to have it in cross-origin requests in the Members API.
- Exposed the announcement bar data through `GET /members/api/announcement` endpoint
2023-04-27 14:01:36 +02:00
Simon Backx
57557cb2f7 Added E2E tests for bulk post API
refs https://github.com/TryGhost/Team/issues/2925
2023-04-27 12:45:24 +02:00
Chris Raible
58efca6c04
Added background saves every 10 mins for post-revisions (#16703)
no issue
2023-04-21 16:04:54 +01:00
Fabien "egg" O'Carroll
bbdbcd02ef Added columns to store feature image metadata for revision
We need this to correctly display the difference with feature images and to
restore them.
2023-04-21 15:25:11 +01:00
Ronald Langeveld
f68936900c
Added post status saving (#16702)
no issue 

<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at ebd1973</samp>

This pull request adds support for storing and tracking the status of
posts in revisions. It introduces a new `post_status` column and
property in the `post` and `PostRevision` models, and updates the
`PostRevisions.formatInput` method to handle it.
2023-04-21 15:17:25 +01:00
Chris Raible
3aec11328f
Populated reason field in post-revisions when revision is created (#16700)
no issue
2023-04-21 14:36:35 +01:00
Naz
cddf786424
Added filtering of announcement bar content
refs https://github.com/TryGhost/Team/issues/3051

-  We need to show the announcement_content to specific audiences based on the announcement_visibility filter
2023-04-21 13:46:37 +02:00
Ronald Langeveld
3cf6800e3e
Added post-status and reason col to post_revisions (#16693)
refs https://github.com/TryGhost/Team/issues/3099

<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 83140b5</samp>

This pull request adds a new column `post_status` to the
`post_revisions` table to store the status of the post revision. It also
updates the schema definition, the migration file, and the integrity
test to reflect the new column.
2023-04-21 12:32:35 +01:00
Fabien 'egg' O'Carroll
169a56d1bb
Added feature_image to post_revisions (#16695)
This will allow us to store historical data for feature images so we can
diff and restore them
2023-04-21 12:02:04 +01:00
Chris Raible
06262ecf33
Added logic for saving revisions on explicit saves (#16688)
refs @TryGhost/Team#3076

- added `save_revision` option to edit post endpoint
- this change covers the following cases:
1. we will not save a `post_revision` on every background autosave that
occurs after 3 seconds of inactivity in the editor
2. we will save a `post_revision` when the user hits `cmd+s` in the
editor to explicitly save
3. we will save a `post_revision` when the user navigates away from the
editor (e.g. by clicking the 'Posts' breadcrumb in the editor)
4. we will save a `post_revision` when the user publishes a post
5. we will save a `post_revision` when a user updates an already
published post
2023-04-21 10:04:05 +01:00
Naz
c21f136834 Corrected announcement_visibility setting type
refs https://github.com/TryGhost/Ghost/pull/14264

- With a requirement change we need to transform `announcement_visibility` setting to be an "array" instead of a "string". Array structure will allow us to hold multiple filters at once giving more coverage to the audience targetting usecases.
- Example filter variations we'll support are:
[ ] Logged out visitors
[ ] Members
[ ] Free members
[ ] Paid members
2023-04-21 10:18:50 +02:00
Rishabh Garg
d3c6d8ad13
Added Pintura integration page in Admin (#16686)
refs https://github.com/TryGhost/Team/issues/3034

- adds new integration page for Pintura in Admin
- allows site owners to enable/disable the image editor integration
- allows self-hosters to upload the files for enabling Pintura image
editor

---------

Co-authored-by: Sodbileg Gansukh <sodbileg.gansukh@gmail.com>
2023-04-20 21:20:07 +05:30
Rishabh Garg
bd127eae6d
Added new settings for pintura editor integration (#16682)
refs https://github.com/TryGhost/Team/issues/3034

- adds 3 new settings for managing pintura image editor integration in
admin
- `pintura` setting controls the enabling of the feature
- `pintura_css_url` is used to store path to pintura css file uploaded
on integration page
- `pintura_js_url` is used to store path to pintura js file uploaded on
integration page
2023-04-20 19:19:56 +05:30
Elena Baidakova
4c4e5a9c95 Hide announcement bar script if announcement_content is empty
refs TryGhost/Team#3009
2023-04-20 14:05:37 +04:00
Naz
de41b07228
Exposed announcement settings in Content API
refs https://github.com/TryGhost/Team/issues/3011

- Allows to read following settings fields:
  - announcement - alias to announcement_content
  - announcement_background
2023-04-19 16:01:50 +02:00
Naz
388a625f37
Added announcement_* fields to Settings Admin API
refs https://github.com/TryGhost/Team/issues/3011

- Allows to edit and read following settings fields:
  - announcement_content
  - announcement_background
  - announcement_visibility
2023-04-19 16:01:50 +02:00
naz
865df1e143
Added announcement fields to settings table (#16654)
refs https://github.com/TryGhost/Team/issues/3011

- This is a data structure needed to support Announcement Bar feature -
allows to create custom site-wide announcements tailored to the
audience.
- The `announcement_content` is meant to hold displayed HTML content of
the announcement and will be exposed through unauthenticated Content
Site API

- The `announcement_visibility` sets the target audience to display the
Announcement Bart to:
  - `public` - Everyone
  - `visitors` - Logged out visitors only
  - `members` - Members only
  - `paid` - Paid members only

- The `announcement_background` sets the CSS class that should be
applied to the Announcement Bar. and will be exposed through
unauthenticated Content Site API. Three styles are available:
  - `accent` - matches the color of the site accent
  - `dark` - dark style
  - `light` - light style
2023-04-19 14:25:25 +02:00
Chris Raible
f95012066a
Added author details to post_revisions in posts endpoint (#16674)
no issue

- added full author object to the post_revisions array on the /posts endpoint
- to be used in the post history modal to display who authored each revision
2023-04-19 12:05:55 +01:00
Chris Raible
0130c9f553
Renamed foreign key constraint in migration v5.45 (#16664)
- the autogenerated constraint name was too long for MySQL, yielding the
error below (although it seems to be fine for sqlite)
- this change adds an explicit, shorthand name for the constraint to
stay under the limit on MySQL8

Error: alter table `post_revisions` add constraint
`post_revisions_author_id_foreign` foreign key (`author_id`) references
`users` (`id`), algorithm=copy - Specified key was too long; max key
length is 3072 bytes
2023-04-18 15:02:13 +01:00
Michael Barrett
9911e6be78
Persisted post revision author and title (#16653)
no issue

Persisted post revision author and title
2023-04-18 14:15:26 +01:00
Chris Raible
e7f9f7c101 Fixed post revisions not loading in modal-post-history 2023-04-18 12:19:31 +01:00
Michael Barrett
75e6ce261c
Added author & title to post revisions (#16649)
no issue

Added `created_by` and `title` columns to `post_revisions` table
2023-04-18 11:47:57 +01:00
Chris Raible
af367a2a18
Added post_revisions as a default include for posts (#16660)
no issue

- post_revisions will now be included in any request to the /posts
endpoint
- updated admin models to include post_revisions
- post revisions can now be accessed in the modal-portal-history via
this.post.post_revisions
2023-04-18 11:12:35 +01:00
Chris Raible
4c893e5a63
Added post_revisions to posts endpoint as an include option (#16648)
no issue

- with this change, you can access a post's lexical revisions via the
api (e.g. `/posts/:id/?include=post_revisions`)

> _We are the post revisions, we won't be erased_
> _We rise from the ashes of the mobiledoc waste_
> _We join the API response, we claim our rightful place_
> _We are the post revisions, we show the truth of your face_
2023-04-17 13:46:27 +01:00
Elena Baidakova
7f184d2451
Added support for filtering snippets to mobiledoc/lexical (#16636)
refs TryGhost/Team#2904


<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at b3f5423</samp>

This pull request adds support for multiple formats of snippet content,
especially the `lexical` format, to the Ghost CMS. It modifies the
snippets API, model, and test files to handle the format conversion,
filtering, and serialization of snippets.
2023-04-17 10:54:08 +04:00
Elena Baidakova
eaf6e3c7e5
Added lexical column to snippets table (#16632)
refs TryGhost/Team#2904

<!-- Leave the line below if you'd like GitHub Copilot to generate a
summary from your commit -->
<!--
copilot:summary
-->
### <samp>🤖 Generated by Copilot at 04a6b06</samp>

This pull request adds a new `lexical` column to the `snippets` table to
store lexical information for snippets. It includes a migration script,
a schema update, and a schema hash update.
2023-04-14 13:18:41 +04:00
Chris Raible
7a800ec06e
Fixed flaky test: posts API export can export with order (#16605)
closes Tryghost/Team#2975

- The test was failing because the order of the posts was not guaranteed
- The posts receive a published_at timestamp = new Date() when they are
created, unless a published_at date is passed in
- ~1/3 times the tests would run, the ordering would change and this
test would fail
- This commit fixes the test by passing in a published_at date to ensure
the order is always the same
2023-04-10 18:40:03 -07:00
Chris Raible
04e6b9763c
Added retries to a flaky batch sending test (#16591)
refs TryGhost/Team#2949
2023-04-07 17:44:11 -07:00
Naz
cdc8cb2d16 Added email snapshot test to member suite
refs https://github.com/TryGhost/Team/issues/2691
refs 53fd78cc86

- One more example using emailMockReceiver instead of outdated mockManager's assertions.
2023-04-07 16:12:51 +02:00
Naz
f77aa03e91
Added missing snapshots for version tests
refs 7552873072

- These snapshot were missing with original refed commit.
2023-04-07 15:31:18 +02:00
Naz
53fd78cc86
Added email snapshot tests to newsletter suites
https://github.com/TryGhost/Team/issues/2691

- This change is meant to serve as an example for testing outbound emails. It uses emailMockReceiver and it's html/test/metadata snapshotting features.
- The wider goal is to move away from using "mockManager.assert.sentEmail" that only tested some of the outgoing email contents to more wide range testing through "emailMockReceiver"
- For more on best practices using emailMockReceiver check out codex section on "email testing" - https://ghost.notion.site/End-to-end-Testing-6a2ef073b1754b18aff42e24a632a007#cf33c7f0-fba6-4951-bf50-09080374719c
2023-04-07 14:34:57 +02:00
Naz
f264c1d1f8
Removed unnecessary mockMail call
no issue

- This test has nothing to do with outgoing emails, so no need to mock mail
2023-04-07 14:34:57 +02:00
Chris Raible
d49241dd29
Fixing error in browser test (#16583) 2023-04-07 01:11:37 -07:00
Ghost CI
7b6805580c Merged v5.42.1 into main 2023-04-07 08:51:06 +01:00
Daniel Lockyer
378dd913aa
🔒 Fixed path traversal issue in theme files
refs https://github.com/TryGhost/Team/issues/2843

- Using encoded path traversal characters in URL's path allowed to fetch
  any file within active theme's folder, which is disallowed
- credits to: fuomag9 https://kiwi.fuo.fi/@fuomag9
2023-04-07 09:45:59 +02:00
Chris Raible
563793c7ed
Added retries for all known flaky tests (#16582)
refs TryGhost/Team#2833

- for mocha tests, we can add `this.retries(1)` to any flaky tests
- for playwright tests, we can add `test.describe.configure({ retries:
1})` to any `describe` block
- not a long-term solution, but it should help mitigate issues with flaky
tests in short term
2023-04-07 00:37:01 -07:00
naz
7552873072
Added email content snapshots to API versioning tests (#16577)
refs https://github.com/TryGhost/Team/issues/2691
refs
939f25a987

- Resurrected refed commit that was adding tests for versioning API,
this time it's using dynamic replacements to match dynamic content of
the email using matchHTMLSnapshot / matchPlaintextSnapshot with dynamic
content replacements.
2023-04-06 22:42:11 +02:00
Naz
3f78e959b3
Bumped email-mock-receiver
refs https://github.com/TryGhost/Team/issues/2691

- This bump changes the "sentEmailCount" method to a more descriptive "assertSentEmailCount" and adds chaining to this method.
2023-04-06 22:13:17 +02:00
naz
70ae3efd5c
Update email mock receive (#16578)
refs TryGhost/Team#2691

- The bump adds possibility to make email's html/text snapshots with dynamic content. The breaking change here is with separate "matchPlaintextSnapshot" method extracted out of "matchMetadataSnapshot" to handle dynamic content in "text" part of the sent email.
2023-04-06 17:24:23 +02:00
Simon Backx
ba8f082d41
Added awaiting jobs and events by default to all tests (#16505)
no issue

This change waits for domain events and jobs before continuing with the
next test. This prevents issues where background tasks in tests are
executed when the next test is running and the configurations have
changed, causing random error logs and test failures.

It also includes a change in Stripe mocking in one E2E test to make use
of the new StripeMocker instead of custom mocking in each test (also to
reduce error logs).
2023-04-06 09:05:16 +02:00
Chris Raible
16c625a630
Added retries to flaky email test (#16556)
refs TryGhost/Team#2891

- test was flaking frequently enough that we had to remove it — not a perfect fix but figure it's better to enable retries than to completely remove the test
- ran CI 5 times (x 4 environments) and it passed 5 times in a row
2023-04-05 22:01:59 -07:00
Sanne de Vries
92663ea731
Fixed email width and dark mode images in email template (#16566)
Refs https://github.com/TryGhost/Team/issues/2845

---------

Co-authored-by: Fabien "egg" O'Carroll <fabien@allou.is>
Co-authored-by: Simon Backx <simon@ghost.org>
2023-04-05 15:53:51 +02:00
Rishabh Garg
bb21dc47f0
Updated member snapshot for comments tests
refs TryGhost/Team#2888
2023-04-05 00:25:31 +05:30
Rishabh
76ed31464b Fixed snapshots for frontend frontend member data APIs
refs https://github.com/TryGhost/Team/issues/2888
2023-04-05 00:01:55 +05:30
Rishabh
ace4d17b3e Updated unit tests for frontend member data 2023-04-04 23:51:45 +05:30
Chris Raible
e95c531e8b
🐛 Fixed member signup emails being sent with escaped subject line (#16544)
closes TryGhost/Team#2895

- this was caused by the subject line being passed through the i18n
translator, which was escaping the content
- passing in `interpolation: {escapeValue: false}` when retrieving the
value prevents the content from being escaped
- modified a test to ensure the subject line is not escaped
2023-04-04 10:12:28 -07:00
Simon Backx
d0042b550a
Fixed redirecting back to account home after sign in in Portal (#16487)
refs https://github.com/TryGhost/Team/issues/2674

When going to /#/portal/account when not signed in, you are redirected
to the login page. But once signed in, you aren't redirected back to the
account page. This fixes this issue by adding an extra and optional
redirect parameter when requesting a magic token via email.

This new parameter allows to override the default behaviour of using the
Referer HTTP header, which doesn't include the hash/fragment part of the
URL.

The referrer is already restricted to only allow redirects to the site,
not external URLs.
2023-04-04 18:07:37 +02:00
Fabien "egg" O'Carroll
41e8c20d88 Deleted flakey test
refs https://github.com/TryGhost/Team/issues/2891

This test is failing more and more frequently and is being removed with the
intention of reinstating it once it has been fixed
2023-04-04 20:16:44 +07:00
Simon Backx
e13f052af8
Added Portal signup terms settings and migration (#16545)
fixes https://github.com/TryGhost/Team/issues/2885 
fixes https://github.com/TryGhost/Team/issues/2896 
fixes https://github.com/TryGhost/Team/issues/2877

This change adds the new portal_signup_terms_html setting and
portal_signup_checkbox_required setting and the corresponding migration.
2023-04-04 10:04:12 +02:00
Sanne de Vries
57fcfe76d8
Updated images in email template to be work with light/dark mode
refs https://github.com/TryGhost/Team/issues/2845

We needed to update the html out of the cards to include images for light
and dark mode, and then we've used CSS to show/hide them 

Co-authored-by: Fabien "egg" O'Carroll <fabien@allou.is>
2023-04-04 00:32:08 +07:00
Simon Backx
d06e3dca60 Added id column to post analytics export
refs https://github.com/TryGhost/Team/issues/2678

Also fixes snapshots
2023-04-03 14:50:43 +02:00
Simon Backx
bef310cb2a Readded failing test
no issue

Test was failing and removed temporarily.
2023-04-03 11:35:40 +02:00
Simon Backx
33237c4df7 Added newsletter auto border color and dynamic color picker
fixes https://github.com/TryGhost/Team/issues/2879
fixes https://github.com/TryGhost/Team/issues/2880

- Replaced black border color with 'auto' based on background color.
- When a color is 'auto', the color that are visible in the UI (color pciker) will be dynamic based on the background color.
2023-04-03 11:27:57 +02:00
Daniel Lockyer
5cd67f83ea Added i18n labs flag
- this adds the labs flag and scaffolding to wire it into Ghost + Portal
2023-03-30 18:15:57 +02:00
Fabien 'egg' O'Carroll
0f72816951
Newsletter email template customisation (#16523)
Refs https://github.com/TryGhost/Team/issues/2845

Co-authored-by: Sanne de Vries <sannedv@protonmail.com>
2023-03-30 15:33:46 +02:00
Rishabh Garg
dc811af070
Merged mentions UI and email lab flags (#16518)
closes https://github.com/TryGhost/Team/issues/2851

We had two separate flags to manage the Mentions beta, one for showing
UI and other for sending emails. This change combines them both under
the single `webmentions` flag that was previously only used to show the
UI.
2023-03-29 01:12:56 +05:30
Simon Backx
109cdeb492 Fixed exporting post metrics without email
no issue

Bookshelf by default returns an empty model when requesting .related('email') for a post without an email. So we need to be a bit smarter to know if a post has an email or not. This fixed an issue where we always showed 'published and emailed' instead of 'published only'.

Since this change also included some changes to test helpers, it also made some changes to the email service because coverage dropped below 100% as a result of fixing the .related method mocking. Ideally we want to move test test helpers to a seperate package in the future.
2023-03-28 12:41:32 +02:00
Simon Backx
7366f726e5 Removed post access copy in post exporter
no issue
2023-03-28 12:08:00 +02:00
Naz
2dd94d4300
Fixed self-serve api key type
refs https://github.com/TryGhost/Ghost/pull/16461

- The referenced migration had an incorrect 'type' assigned to the self-serve integration's api key. Should have been "admin" instead of "core"
2023-03-28 11:59:17 +02:00
Simon Backx
37397e88b6 Renamed paid_signups to paid_conversions in post metrics export
no issue
2023-03-28 11:43:37 +02:00
Simon Backx
59f94e552a
Added newsletter color columns (#16506)
fixes https://github.com/TryGhost/Team/issues/2825

Adds 3 new columns to the newsletters table to allow more customisation:
- background_color, default to 'light'
- border_color, nullable, null = no border
- title_color, nullable, null = auto
2023-03-28 11:38:43 +02:00
Fabien "egg" O'Carroll
349f3b01eb Fixed snapshots for post stats export e2e-api test
The Content-Disposition header was not matching because we use the date in the
filename for the exported CSV file. I haven't created a new matcher here because
it's quite unique, and we don't want an `anyContentDisposition` matcher because
the filename is a part of the API.
2023-03-28 09:48:17 +07:00
Simon Backx
465c2ee8d5 Updated post export E2E test ordering
no issue

Ordering on title differs between MySQL and SQLite, so replaced it with published_at.
2023-03-27 11:22:33 +02:00
Simon Backx
2fedce8a37 Added E2E tests for post export endpoint
fixes https://github.com/TryGhost/Team/issues/2788
2023-03-27 11:11:49 +02:00
Simon Backx
868adf30fd Fixed batch sending snapshot date matching
no issue

The replacement didn't match dates with only one number.
2023-03-27 10:24:34 +02:00
Rishabh Garg
0c743d67af
🐛 Fixed member newsletter subscription not saving in Admin (#16490)
fixes https://github.com/TryGhost/Team/issues/2783 
refs cb05fae5a3

The root cause of the issue was the fact we no longer checked for lack of `newsletters` property on member data before checking its `subscribed` property which is now deprecated. This caused a cascading effect where `subscribed:false` property on a member overrides the value for `newsletters` data. The check was accidentally removed in a previous bug fix.

So for members that were not subscribed to any newsletters, saving a newsletter subscription failed as they had their `subscribed` set to `false`, and it was resetting the newsletter subscription to empty always.
2023-03-24 23:29:49 +05:30
Sanne de Vries
387dfa59c9
Fixed issue with email template in Outlook (#16486)
Refs https://github.com/TryGhost/Team/issues/2801

- It was not possible to click latest post links in Outlook due to <a>
tag wrapping around a table
- The post meta data wouldn't display properly when centered in Outlook

---------

Co-authored-by: Simon Backx <simon@ghost.org>
2023-03-24 14:54:16 +01:00
Simon Backx
d9c92816e0 Added dynamic text truncation in email latest posts
refs https://github.com/TryGhost/Team/issues/2675

Truncate text depending on mobile/desktop and feature image.
2023-03-24 12:15:16 +01:00
Naz
2231981880
Added e2e test checking integration access
refs https://github.com/TryGhost/Team/issues/2790

- The Self-Serve Integration should only be accessible to the Owner and Admin user roles otherwise we risk accidental indirect increase in role permissions - Self-Serve Integration has permissions which editors/contributors don't have.
2023-03-24 11:31:25 +01:00
Naz
fe4e9897fc
Added migration for Self-Serve Integration permissions
refs https://github.com/TryGhost/Team/issues/2790

- This migration adds permissions for Self-Serve Migration Integration to have access to Admin APIs:

POST /ghost/api/admin/db
POST /ghost/api/admin/db/media/inline
POST /ghost/api/admin/members/upload
GET  /ghost/api/admin/tags/:id
GET  /ghost/api/admin/tags/slug/:slug
2023-03-24 11:31:25 +01:00
Naz
0b107f5af5
Added migrations for Self-Serve Migration Integration and API key
refs https://github.com/TryGhost/Team/issues/2790
refs 3747df1bc8 (diff-396038cecd7a381616d00954ae18a655ae2a8af71ea65866bf09d2c7cc1b5235)

- This integration will be used to perform self-serve migrations.
- The integration will be limited to these Admin API endpoints:
POST /ghost/api/admin/db
POST /ghost/api/admin/db/media/inline
POST /ghost/api/admin/members/upload
GET   /ghost/api/admin/tags/:id
GET   /ghost/api/admin/tags/slug/:slug
2023-03-24 11:28:29 +01:00
Simon Backx
21ded0b4b8 Corrected email link in subscription box
refs https://github.com/TryGhost/Team/issues/2674

The replacement didn't work inside an attribute, so I removed it to a link that doesn't do anything.
2023-03-24 10:23:51 +01:00
Simon Backx
a9791e95cc Fixed including current email in last 3 posts in newsletters
refs https://github.com/TryGhost/Team/issues/2675

The current post could be inclnuded in the last 3 posts if it was also published. This is now fixed and covered by a test.
2023-03-24 09:20:32 +01:00
Simon Backx
0107d2bb77 Fixed subscription status not showing correctly in emails
refs https://github.com/TryGhost/Team/issues/2674

- The segment detection doesn't work outside the main post content. So the data-gh-segment attribute didn't work. It is now replaced with just a simple email replacement that is empty for a free member.
- Fixed that a trialing member was shown as 'paid'. This is now replaced with 'trialing'.

This commit also includes E2E tests for a couple of member statusses.
2023-03-24 08:55:36 +01:00
Sanne de Vries
3f0ec1af3d Updated subscription box in email preview and email template
Refs https://github.com/TryGhost/Team/issues/2738

- With all footer additions and changes, the subscription box did not match the rest of the design and is therefore updated.
2023-03-23 19:39:58 +01:00
Aileen Nowak
37383fde90 Moved Milestone emails to GA
- Moves Milestone emails from public beta to GA Moved Milestone emails to GA

- Moves Milestone emails from public beta to GA Moved Milestone emails to GA

- Moves Milestone emails from public beta to GA Moved Milestone emails to GA

- Moves Milestone emails from public beta to GA Moved Milestone emails to GA

- Moves Milestone emails from public beta to GA Moved Milestone emails to GA

- Moves Milestone emails from public beta to GA Moved Milestone emails to GA

- Moves Milestone emails from public beta to GA Moved Milestone emails to GA

- Moves Milestone emails from public beta to GA Moved Milestone emails to GA

- Moves Milestone emails from public beta to GA
2023-03-23 17:35:23 +00:00
Sanne de Vries
0455672832 Updatet latest post section in email template
Refs https://github.com/TryGhost/Team/issues/2675
2023-03-23 13:16:48 +01:00
Daniel Lockyer
2941154353
Added dynamic date matching to email sending snapshot
- we don't want to include the actual date here because it'll change
- this adds a regex to match the date and replace it with a standard
  "date" string, which won't change
2023-03-23 09:20:23 +01:00
Sam Lord
5065950401 Mocked mail to prevent errors in e2e test suite
refs: https://github.com/TryGhost/Toolbox/issues/389
2023-03-22 16:25:23 +00:00
Sam Lord
47d9b4a51d Fixed errors being logged by unit tests
refs: https://github.com/TryGhost/Toolbox/issues/389
2023-03-22 16:25:23 +00:00
Simon Backx
860048a89d Fixed snapshots 2023-03-22 17:03:11 +01:00
Simon Backx
46f305efaf Fixed hiding name if empty in email subscription box
no issue

Moved some CSS around because I moved the wrong code in the previous commit.
2023-03-22 16:51:56 +01:00
Simon Backx
55c281debf Reverted removeStyleTags from email-renderer
no issue

Some things break in some email clients with this new setting. Disabled it for now and moved the required css style to hide the member name row to @media all.
2023-03-22 16:40:35 +01:00
Simon Backx
4eebf6612a Added image dimensions to the latest posts mobile version in email
fixes https://github.com/TryGhost/Team/issues/2799

Separate image dimensions for mobile and normal images.
2023-03-22 15:54:27 +01:00
Simon Backx
480c1a7004 Removed name from subscription details if missing
refs https://github.com/TryGhost/Team/issues/2736

If the name is not known for a member, we'll hide the name row in the subscription details in an email. This method is supported in most email clients, and requires the support of `<style>` in `<head>`.
2023-03-22 15:32:07 +01:00
Sanne de Vries
47e343ec18 Updated latest posts UI in email template
Refs https://github.com/TryGhost/Team/issues/2801
2023-03-22 15:16:55 +01:00
Sam Lord
3718d4acda Enable labs in playwright tests
no issue

Allows us to write tests for features which are still in development.
2023-03-22 11:16:49 +00:00
Simon Backx
bc0126c54e
Added subscription status text in newsletters (#16442)
fixes https://github.com/TryGhost/Team/issues/2736

Shows the actual subscription status (expires on DD MMM YYYY) in every
email when show subscription details is enabled.
2023-03-22 11:52:41 +01:00
Daniel Lockyer
247163c710
Fixed tests for verification during batch sending
refs 59df99388c

- it seems the `getSignupEvents` method is now called twice during these
  tests
2023-03-22 09:44:01 +01:00
Aileen Booker
8d290c4560
Milestone emails templates and sending implementation (#16318)
refs
https://www.notion.so/ghost/Marketing-Milestone-email-campaigns-1d2c9dee3cfa4029863edb16092ad5c4?pvs=4

- Added email template for milestones with using a configuration file
for different member milestone values, as we're sending different
content for each one
- Implement sending the email to users who have
`milestone-notifications` enabled, currently still behind a flag

Co-authored-by: Peter Zimon <peter.zimon@gmail.com>
2023-03-21 13:39:40 +00:00
Sanne de Vries
792bfdb498 Added UI for subscription box in newsletters
Refs https://github.com/TryGhost/Team/issues/2738

- Added preview to email newsletter settings
- Added subscription box UI to email template
2023-03-21 11:40:04 +01:00
Rishabh
0b7e09c986 Updated fallback for storing source site title
refs https://github.com/TryGhost/Team/issues/2754

Previously, we didn't have any backup for storing source site title and it could have stored as empty if missing. This change ensures the source site title is stored as site host instead as fallback if not present.
2023-03-21 16:09:21 +05:30
Simon Backx
a016f53b7e Added resolveMx stub to all E2E tests
refs https://ghost.slack.com/archives/C04TMVA1D7A/p1679391135060629
refs 1b0adcf4ab

The same issue when sending emails from E2E tests, where the tests timeout when resolving mx records for invalid domains.
2023-03-21 11:25:06 +01:00
Simon Backx
a5dff4207e Moved posts service to its own package
fixes https://github.com/TryGhost/Team/issues/2778

It is easier to add extra classes using the latest patterns if it has its own package.
2023-03-20 16:06:42 +01:00
Elena Baidakova
aca8c5805c Improved members-actions e2e browser test
refs TryGhost/Team#2667
- Added notification check before checking that user successfully unsubscribed from all newsletters. It helps to make the test more stable
2023-03-20 18:48:35 +04:00
Simon Backx
07ec33fb3a
Added latest posts to email template (#16448)
refs https://github.com/TryGhost/Team/issues/2769

Needs some extra styling and design, this is only a minimal version
behind the feature flag.
2023-03-20 14:30:42 +01:00
Simon Backx
a513542b56
Added show_latest_posts column to newsletter table (#16402) 2023-03-20 11:10:49 +01:00
Sanne de Vries
1b0adcf4ab Fixed missing stub for DNS in GhostMailer Direct tests
- without this, Node will try and resolve the domain name but local DNS
  resolvers can take a while to timeout, which causes the tests to timeout
- `nodemailer-direct-transport` calls `dns.resolveMx`, so if we stub that
  function and return an empty array, we can avoid any real DNS lookups
2023-03-17 18:16:30 +01:00
Simon Backx
a6654b5e6c Updated GhostMailer Direct unit tests to async await style
no issue

Tests were timing out for some users, probably because not all errors are caught.
2023-03-17 17:34:01 +01:00
Steve Larson
0a4d568f19 updated snapshot
no refs
-updated snapshot to pass tests
2023-03-17 10:57:50 -05:00
Peter Zimon
9d25c2a058 Fixed mobile size footer buttons in newsletters
refs. https://github.com/TryGhost/Team/issues/2740

- alignment of buttons in the footer of newsletter email template was off
2023-03-17 16:34:23 +01:00
Steve Larson
067b35de6b fixed member newsletter test
no refs
-fixed flaky test that seemed to have sync issues with fixture data and db
2023-03-17 10:27:09 -05:00
Simon Backx
b3c410a923 Reverted stats snapshot change 2023-03-17 16:06:00 +01:00
Simon Backx
c591f1f86a Updated email batch sending snapshots 2023-03-17 16:00:14 +01:00
Sanne de Vries
48a3159d3d Added mobile feedback buttons to email template
Refs https://github.com/TryGhost/Team/issues/2740
2023-03-17 15:58:14 +01:00
Daniel Lockyer
b7a03ca765 🐛 Fixed mainEntityOfPage property within page metadata
fixes https://github.com/TryGhost/Ghost/issues/16367
refs https://github.com/schemaorg/schemaorg/discussions/3274

- after a discussion with the schemaorg maintainers, it looks like
  `mainEntityOfPage` should refer to the entity itself vs the parent
  entity
- this commit updates the code to use the entity URL and fixes the
  associated tests
2023-03-17 15:13:28 +01:00
Simon Backx
005b5f20fb Added E2E tests for comment CTA
fixes https://github.com/TryGhost/Team/issues/2727

Adds some E2E tests and snapshots for comment CTA in all possible situations.
2023-03-17 11:12:47 +01:00
Simon Backx
298e3da745 Fixed comment CTA button link
refs https://github.com/TryGhost/Team/issues/2672

- Use #ghost-comments instead of #ghost-comments-root
- Fixed snapshots
2023-03-17 10:27:23 +01:00
Simon Backx
423337a048 Fixed snapshots 2023-03-17 10:24:21 +01:00
Sanne de Vries
717ed114f4 Fixed center aligned post title in emails
Refs https://github.com/TryGhost/Team/issues/2740
2023-03-17 10:07:17 +01:00
Chris Raible
0640b9c4aa
Modified outbound link tagger to use ?ref=domain instead of ?ref=siteTitle (#16431)
refs TryGhost/Team#2755

- Outbound link tagger will now append ?ref=example.com or ?ref=example.ghost.io instead of ?ref=siteTitle
2023-03-17 14:18:26 +05:30
Naz
c487b12518 Reverted zip's root file copying fix
refs https://github.com/TryGhost/Toolbox/issues/523

- The reverted fix did not take into account the "original path" of the
files would be truncated. This path has to be full relative to the root
of the zip to later be used during importer url substitution logic.
- This reverts commit 831a76505c.
2023-03-16 21:52:24 +01:00
Steve Larson
7f223b56dd
fixed member newsletter test (#16425)
no ref
-fixed flaky test that seemed to have sync issues with fixture data and
db
2023-03-16 10:43:17 -05:00
Sanne de Vries
cdcb3dcd6f
Updated email template to include comment link in header and footer (#16423)
Refs https://github.com/TryGhost/Team/issues/2740

---------

Co-authored-by: Steve Larson <9larsons@gmail.com>
2023-03-16 16:10:53 +01:00
Elena Baidakova
60b3731199 Separated admin and portal tests to different projects
refs TryGhost/Team#2667
- Portal tests are much more stable when running in the separate DB from admin tests.
2023-03-16 18:35:09 +04:00
Simon Backx
450e01d1c0 Added placeholder basic subscription details to email template
refs https://github.com/TryGhost/Team/issues/2736

The renewal date is still missing, and style and desin is only a placeholder.
2023-03-15 17:12:56 +01:00
Simon Backx
80043f2699 Fixed snapshot tests in batch sending 2023-03-15 14:11:05 +01:00
Simon Backx
6e5ddc6a9a Fixed email preview snapshots 2023-03-15 14:03:24 +01:00
Sanne de Vries
e0c3573079 Updated email template snapshot resulting in test failure
No ref
2023-03-15 12:45:54 +01:00
Simon Backx
da2e56916e
Added show_subscription_details column to newsletter table (#16403)
fixes https://github.com/TryGhost/Team/issues/2708

Adds `show_subscription_details` column to newsletter table. Disabled by
default.
2023-03-15 11:53:45 +01:00
Fabien 'egg' O'Carroll
bdaddd1a64
Added setting to store last mention report email date (#16399)
Because there is no guarantee about a daily job running exactly once a
day, we need to store the last time that the email was sent, so that we
can refrain from sending one if it's been less than a day since the
last.

A setting has been used for this as we don't currently have a pattern
for it, we might want to consider moving this to some kind of cache
based solution in future. This has been added as a core setting so that
we don't expose it via the API.

The setting is stored as a number to allow us to store value as unix timestamp.

---------

Co-authored-by: Rishabh <zrishabhgarg@gmail.com>
2023-03-15 15:34:41 +05:30
Simon Backx
b27c7bc707 Fixed batch sending snapshots dates
no issue

The snapshots contained the current date, so they broke every day. This commit fixes the issue by setting a fixed date.
2023-03-15 09:45:36 +01:00
Simon Backx
bb019a0ab5 Added E2E test for sending email without post title
fixes https://github.com/TryGhost/Team/issues/2724

This change also includes new snapshots for email sending (similar for email previews, but this time for the real emails to make sure we catch changes).
2023-03-14 16:05:18 +01:00
Simon Backx
350b291c34
Added show_comment_cta column to newsletter table (#16401)
fixes https://github.com/TryGhost/Team/issues/2707

Adds the new column for storing whether we need to show a comment CTA in
emails. Enabled by default.
2023-03-14 15:03:02 +01:00
Simon Backx
1e435fb328
Implemented showPostTitleSection (#16404)
fixes https://github.com/TryGhost/Team/issues/2705

- Added showPostTitleSection to newsletter model in admin
- Wired up UI to admin model so it saves to the database
- Implemented showPostTitleSection in newsletter preview and added some
minor temporary css styling
- Implemented showPostTitleSection in newsletter template in backend,
and added some extra CSS styling to fix spacing
2023-03-14 11:29:43 +01:00
Simon Backx
69d07c892c
Fixed getLazyRelation returning new model for optional relations (#16411)
no issue

When using `getLazyRelation` on an optional relation that is not set, it
will return a newly created model instead of a model from the database.
- Adds a new require option to `getLazyRelation`, that throws an error
if the relation is not set (off by default to match existing use cases)
- This caused a bug (not visible because we always pass a newsletter id)
in email previews, where when the newsletter id was not explicitly set,
it would use `newsletter = (await post.getLazyRelation('newsletter')) ??
(await this.models.Newsletter.getDefaultNewsletter());`, which always
returned the first one, and could return a newly initiated newsletter
with all properties set to undefined.
- Some page snapshots are altered by this, because the usage of
`getLazyRelation` on a post no longer sets the email relation to some
new model.
2023-03-14 10:58:58 +01:00
Simon Backx
ff5e7202c6
Added show_post_title_section column to newsletters table (#16397)
fixes https://github.com/TryGhost/Team/issues/2706

Only includes the migration. The setting is not used at the moment, but
will be enabled by default.
2023-03-14 09:48:13 +01:00
Aileen Booker
eeb7546abb
Added handling for initial and skipped Milestones (#16405)
refs
https://www.notion.so/ghost/Marketing-Milestone-email-campaigns-1d2c9dee3cfa4029863edb16092ad5c4?pvs=4

- When milestones will be activated we would send out emails to users
that are way above the achieved milestone, as we didn't record
milestones before
- The plan is to implement a 0 milestone and don't send an email for
achieving those and also add all achieved milestones in the first run
until a first milestone is stored in the DB, then increment from there.
- This change takes care of two cases:
1. Milestones gets enabled and runs initially. We don't want to send
emails unless there's already at least one milestone achieved. For that
we add a 0 milestone helper and add a `initial` reason to the meta
object for the milestone event, so we can choose not to ping Slack and
also disable email sending for all milestones achieved in this initial
run.
2. All achieved milestones will be stored in the DB, even when that
means we skip some. This introduces the `skipped` reason which also
doesn't send emails for the skipped milestones, but will do for
correctly achieved milestones (always the highest one).
- Added handling for slack notifications to not attempt sending when
reason is `skipped` or `initial`
2023-03-13 19:01:11 +02:00
Fabien 'egg' O'Carroll
4525c7293c
Removed email notifications for new mentions
These are no longer desired and will be replaced with a digest style
email.
2023-03-13 18:33:57 +07:00
Steve Larson
8d76832b07 added e2e tests to cover page events
no refs
-added e2e tests for sending mentions on various page events
2023-03-10 15:25:10 -06:00
Simon Backx
80cec29144
Added Stripe Mocker to make testing easier (#16396)
no issue

The Stripe Mocker mocks the Stripe API in memory, to make it much easier
to test subscription flows. Currently it is more a POC to see if it
works well. It probably needs a bit more work to support more scenarios.
- Added new tests for the subscription stats endpoint for 3D secure +
free trial flows using the new Stripe Mocker
- Updated members admin api tests to use Stripe Mocker (+ added new test
for deleting members with Stripe cancellation)
- Some tests called mockStripe at the beginning, but that method did
nothing apart from disabling network (which is the default now), then
they mocked Stripe inside the tests file... so I've removed those
because those conflict with the new mocker that is enabled when calling
mockStripe. We'll need to port those over later.
2023-03-10 18:43:35 +01:00
Daniel Lockyer
89493893d1 Removed all unused variables from test files
- this cleans up all imports or variables that aren't currently being used
- this really helps keep the tests clean by only allowing what is needed
- I've left `should` as an exemption for now because we need to clean up
  how it is used
2023-03-10 14:29:55 +01:00
Daniel Lockyer
42333a3349 Removed custom assertions
- these are no longer used in favor of snapshots in tests
2023-03-10 14:29:55 +01:00
Daniel Lockyer
08f84475e7 Removed sleep lib from test utils
- this is generally an anti-pattern in tests and leads to flaky
  behaviour when tests are ran on different machines/loads
- this is currently unused so it is an easy removal
2023-03-10 14:29:55 +01:00
Elena Baidakova
6f9f83bee8 Revert "Removed explicit timeout for offer creation helper"
This reverts commit ac427e4253.
2023-03-10 13:44:45 +04:00
Elena Baidakova
ac427e4253 Removed explicit timeout for offer creation helper
refs TryGhost/Team#2667
- Removed timeout to make it the same as a global one
2023-03-10 13:24:38 +04:00
Ronald Langeveld
ab01e4fcbf Added new Tier e2e test
refs https://github.com/TryGhost/Team/issues/1446

- Added a test for successfully creating a new tier.
2023-03-10 13:29:25 +08:00
Simon Backx
832610fd2a
🐛 Fixed retrying failed emails when rescheduling them (#16383)
fixes https://github.com/TryGhost/Team/issues/2560

When an email fails, and you reschedule the post, the error dialog was
shown (from the previous try). The retry button on that page allowed you
to retry sending the email immediately, which could be very confusing.

- The email error dialog is no longer shown for scheduled emails
- The email status is no longer polled for scheduled emails
- Retrying an email is not possible via the API if the post status is
not published or sent
- Added some extra snapshot tests
- When retrying an email, we immediately update the email status to
'pending' to have a better API response (instead of still returning
failed).
- Disabled email sending retrying in development (otherwise very hard to
test failed emails if it takes 10 mins before it gives up automatic
retrying)
2023-03-09 12:32:22 +01:00
Elena Baidakova
8b191a8d01 Removed all tiers archive from portal tests to reduce relation between tests
refs TryGhost/Team#2667
2023-03-09 12:55:27 +04:00
Ronald Langeveld
7539a681fe
Excludes sites with canonical_url meta from sitemap. (#16376)
closes https://github.com/TryGhost/Team/issues/2531

This commit fixes the issue where non-canonical URLs are included in the
XML sitemap, leading to poor SEO for our user's sites. The solution
implemented is to exclude any page or post that specifies a canonical
URL in its metadata from the sitemap.

To achieve this, a condition has been added to the 'addUrl' method,
which checks for the existence of a canonical URL in the metadata of the
resource being added to the sitemap. If a canonical URL is present, the
resource is excluded from the sitemap.

With this fix, our user's sites will have better SEO and improved search
engine visibility.
2023-03-09 16:38:43 +08:00
Steve Larson
cb05fae5a3
🐛 Fixed maintaining archived newsletter subscriptions for members (#16375)
refs #16355
-archived newsletters will no longer be unsubscribed when saving member
-fixed bug with initialMember load
-fixed errors in unit tests
2023-03-08 16:39:28 -06:00
Simon Backx
4184b279d2
🐛 Fixed HTML escaping when using outbound link tagging (#16380)
fixes https://github.com/TryGhost/Team/issues/2666

- Somehow occurrences of `&map_` got replaced with `&#x21A6;`
- Disables escaping &, ', " and other HTML characters when not needed
(escaping is already handled by mobiledoc/lexical)
- Bumps unit test coverage of link replacer to 100%
2023-03-08 16:30:54 +01:00
Elena Baidakova
00cda68125
Disabled fraud detection for e2e browser tests (#16379)
refs TryGhost/Team#2667
2023-03-08 16:01:08 +04:00
Simon Backx
3d89dbb775 Fixed whistelisted domains in Playwright tests and network connectivity
no issue

- Nock doesn't support multiple calls to enableNetConnect -> only the last one counts. This fixes that issue.
- Some tests interacted directly with nock instead of using the mockManager to restore everything.
2023-03-08 12:40:21 +01:00
Simon Backx
df9df4be80 Allowed Stripe network access in Playwright tests
no issue

Previous test change that blocks network access broke the Playwright tests, that need to access the Stripe checkout page.
2023-03-08 11:49:36 +01:00
Naz
831a76505c
Fixed root zip image/media/files copying during import
https://github.com/TryGhost/Toolbox/issues/523

- During import process of content files the files from the root directory were also copied over. This is causing chaos in the root of content folder with files that only needed for data import. For example, the csv files needed for Revue import were also copied over by "file importer" even though those do not belong to any content.
- Any content import files - images, media, files, should be in according folders in the imported zip file. The root files in the base zip directory are for data-related imports
2023-03-08 11:03:47 +08:00
Simon Backx
38de815d98
Removed old email flow (#16349)
fixes https://github.com/TryGhost/Team/issues/2611

The old email flow is no longer used since we introduced the email stability flow. This commit removes the related code and tests. The general test coverage decreased a bit as a result, because the old email flow probably had a high test coverage. The new flow is in separate packages, so it couldn't contribute to a higher test coverage (but it does have 100% unit test coverage).
2023-03-07 16:08:40 +01:00
Simon Backx
3db434736b
🐛 Fixed replacements with fallback in plaintext newsletters (#16372)
fixes https://github.com/TryGhost/Team/issues/2683

When sending a newsletter with a replacement that has a fallback, the
replacement only happens in the HTML version of the newsletter. The
plaintext version isn't replaced.

This commit fixes the issue and adds some tests to make sure it doesn't
happen again.

The cause of the issue was that we used the original matched Regex text
to replace. But that was calculated on the HTML version, so double
quotes were encoded. This change updates the generated 'token' regex to
also match on both a double quote as the escaped double quote.
2023-03-07 15:34:43 +01:00
Simon Backx
f50224f445
Improved network access and mocking in tests (#16371)
refs https://github.com/TryGhost/Team/issues/2667

Some tests still accessed the internet. Now network access is disabled
by default. This change also introduces two helper methods related to
networking (mocking Slack and Mailgun).

This fixes two unreliable tests:
- Staff service was accessing a Slack test API -> timeout possible
- MentionSendingService was trying to send webmentions for every post
publish/change -> possible timeouts and job issues
2023-03-07 13:20:28 +01:00
Tim Gates
5394fa81fc
Fixed typos (#16359) 2023-03-06 13:11:45 +01:00
Naz
33316a6440
Added e2e test covering media inliner job start
refs https://github.com/TryGhost/Toolbox/issues/523

- The test is useful for future iterations of the response format and as a quick reference on which parameters the media inlining endpoint accepts.
2023-03-06 17:43:33 +08:00
Sam Lord
472e52a536 Made URL class warn on duplicate URLs in the test suite
refs: https://github.com/TryGhost/Toolbox/issues/389

Instead of logging errors, this will warn when adding a duplicate URL in the test environment.

At the moment, this is happening a lot in the test suite. While we also need to fix the root cause of this so we're not erroring in the product, it's a massive amount of spam in the logs when running the test suite which could prevent us from finding other errors which are causing issues.
2023-03-03 18:25:53 +00:00
Sam Lord
13b0f51b13 Prevent simple error logs by stubbing log library
refs: https://github.com/TryGhost/Toolbox/issues/389

This removes many error logs when the end-to-end test suite is run with the log-level set to error. Many errors are intentional, so the resolution is typically to stub the error log function and assert that it would have been called.
2023-03-03 18:25:53 +00:00
Sam Lord
cd5aca7227 Prevented .well-known file tests from attempting to send mail
refs: https://github.com/TryGhost/Toolbox/issues/389

These tests would log errors while attempting to send email.
2023-03-03 18:25:53 +00:00
Sam Lord
5b2adbe4c5 Prevents errors from being logged during the offers tests
refs: https://github.com/TryGhost/Toolbox/issues/389

Intentional errors now call a logging library stub
2023-03-03 18:25:53 +00:00
Sam Lord
64d94c8c8c Prevent posts-legacy tests from logging errors
refs: https://github.com/TryGhost/Toolbox/issues/389

The newsletter fixtures no longer errors when accessing the test image, and tests which intentionally error now stub the logging call
2023-03-03 18:25:53 +00:00
Sam Lord
a0acccc088 Prevent errors in E2E tests where front-end is disabled and Ghost Admin accesses themes
refs: https://github.com/TryGhost/Toolbox/issues/389
2023-03-03 18:25:53 +00:00
Steve Larson
3e18715caa added twitter embed tests
refs TryGhost/Team#2663
-added basic coverage for twitter embeds
2023-03-03 10:54:03 -06:00
Naz
a701724903
Renamed content file importer test
refs https://github.com/TryGhost/Toolbox/issues/523

- Test was not renamed after the "image" module changed it's name to "ContentFileImporter"
2023-03-02 17:59:36 +08:00
Naz
ab3959f3f2
Added document file support to the zip importer
refs https://github.com/TryGhost/Toolbox/issues/523

- When migrating or importing ZIP files into Ghost there's often a need to include document files.
- When document files are present in the imported zip file they are now copied across and processed along with the rest of import files: json, images, csvs, etc.
- The importer also searches for use of the document files in the imported "posts" substituting the links with local ones
- The document files importer recognizes media files inside of "files" or "content/files" folders present in the zip.
- The supported media file extensions are same as for file upload widget:
".pdf",".json",".jsonld",".odp",".ods",".odt",".ppt",".pptx",".rtf",".txt",".xls",".xlsx",".xml"

with following content-types:
"application/pdf", "application/json", "application/ld+json", "application/vnd.oasis.opendocument.presentation", "application/vnd.oasis.opendocument.spreadsheet", "application/vnd.oasis.opendocument.text", "application/vnd.ms-powerpoint", "application/vnd.openxmlformats-officedocument.presentationml.presentation", "application/rtf", "text/plain", "application/vnd.ms-excel", "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", "application/xml", "application/atom+xml"
2023-03-02 17:54:19 +08:00
Naz
cd49992cd9
Added generic file import handler
refs https://github.com/TryGhost/Toolbox/issues/523
refs c2534e3c86/packages/mg-assetscraper/lib/AssetScraper.js (L14-L16)
refs https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types

- Importer needs to process and recognize document files like pdfs, presentations etc to be able to import them into sites file storage.
- The handler allows a new root directory "files" to place imported documents
- The handler adds validation and processing for following file extensions:
".pdf",
".json",
".jsonld",
".odp",
".ods",
".odt",
".ppt",
".pptx",
".rtf",
".txt",
".xls",
".xlsx",
".xml"
- With following content types:
"application/pdf",
"application/json",
"application/ld+json",
"application/vnd.oasis.opendocument.presentation",
"application/vnd.oasis.opendocument.spreadsheet",
"application/vnd.oasis.opendocument.text",
"application/vnd.ms-powerpoint",
"application/vnd.openxmlformats-officedocument.presentationml.presentation",
"application/rtf",
"text/plain",
"application/vnd.ms-excel",
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
"application/xml",
"application/atom+xml"
2023-03-02 17:54:18 +08:00
Ronald Langeveld
7c805b7176 Revert "Added new created tier e2e test"
This reverts commit dc98868961.
2023-03-02 16:39:17 +08:00
Ronald Langeveld
dc98868961 Added new created tier e2e test
ref https://github.com/TryGhost/Team/issues/1446

- Added e2e test to check if a new tier can be added successfully.
2023-03-02 16:11:12 +08:00
Naz
2361669bb6
Added media file support to the zip importer
refs https://github.com/TryGhost/Toolbox/issues/523

- When migrating or importing ZIP files into Ghost there's often a need to include more than just post content and images.
- When media files are present in the imported zip file they are now copied across and processed along with the rest of import files: json, images, csvs, etc.
- The importer also searches for use of the media files in the imported "posts" substituting the links with local ones
- The media importer recognizes media files inside of "media" or "content/media" folders present in the zip. The supported media file extensions are same as for media upload widget:
".mp4", ".webm", ".ogv", ".mp3", ".wav", ".ogg", ".m4a"

with following content-types:
 "video/mp4", "video/webm", "video/ogg", "audio/mpeg", "audio/vnd.wav", "audio/wave", "audio/wav", "audio/x-wav", "audio/ogg", "audio/mp4", "audio/x-m4a"
2023-03-02 14:35:37 +08:00
Naz
49553e5942
Renamed "image" importer to generic ContentFileImporter
refs https://github.com/TryGhost/Toolbox/issues/523

- This is ground work before introducing a "media" content type importer
- Previous "image" file name was not describing well what the importer was capable of doing
2023-03-02 14:35:37 +08:00
Sam Lord
90ef8f008e Prevent error logs from occuring in Ghost unit test suite
refs: https://github.com/TryGhost/Toolbox/issues/389
2023-03-01 16:37:46 +00:00
Naz
eb8d9df923
Made image importer type dynamic
refs https://github.com/TryGhost/Toolbox/issues/523

- Having a dynamic importer type allows to reuse the image importer implementation for other file types e.g.: 'media'
2023-03-01 18:28:17 +08:00
Naz
edebe1571c
Extracted image store to constructor parameter
refs https://github.com/TryGhost/Toolbox/issues/523

- We need to be able to use different storage mechanisms when importing different types of content
- Having the storage passed in using constructor DI allows to have more flexible storage mechanism in the Images importer (soon to become a generic file importer)
2023-03-01 18:21:46 +08:00
Naz
4573d8b4b8
Refactored image importer to use class
refs https://github.com/TryGhost/Toolbox/issues/523

- The class syntax would allow swapping out the storage mechanism in the importer making it universal to use with other file types like media or generic files.
2023-03-01 18:10:39 +08:00
Naz
2bf98e288f
Removed unused module from test suite
refs https://github.com/TryGhost/Toolbox/issues/523
2023-03-01 18:10:39 +08:00
Naz
e14520115a
Extracted Image Importer test suite
refs https://github.com/TryGhost/Toolbox/issues/523
refs 3119a5cc4c

- The "index.test.js" is doing too much, has to be broken down to allow things to move into new packages
2023-03-01 16:48:38 +08:00
Daniel Lockyer
4c2ff0539d
Fixed tests for ImportManager audio types
refs c5a4ee89c9

- as of the referenced commit, the supported list of mime types has been
  updated
2023-03-01 09:14:09 +01:00
Naz
3119a5cc4c
Extracted ImageHandler test suite
refs https://github.com/TryGhost/Toolbox/issues/523

- The "importer/index.test.js" test suite is testing more than it should. ImageHandler test suite section is one of the examples of test cases that should live in a separate file.
- Having these tests in different files makes it easier to reason about coverage and extract to it's own packages.
2023-03-01 15:55:10 +08:00
Naz
ce5eea5707
Added media file handling to Ghost importer
refs https://github.com/TryGhost/Toolbox/issues/523

- When a zip file is imported into Ghost we need to recognize and process media files with following extensions:
".mp4",".webm", ".ogv", ".mp3", ".wav", ".ogg", ".m4a"
- The media files can come from a "media" or "content/media" folder inside of zip file
2023-03-01 15:55:10 +08:00
Naz
23e2fdde95
Added image importer test coverage
refs https://github.com/TryGhost/Toolbox/issues/523

- The test cases for "content" and "content/images" zip import file structure were missing.
2023-03-01 15:55:10 +08:00
monkey sees
838516efb0 🐛 Fixed Handlebars’ asset helper for SafeString input
closes https://github.com/TryGhost/Ghost/issues/16332

Passing `SafeString` input to `asset` helper was resulting in the
exception being thrown. This meant that we couldn’t combine `asset`
helper with other helpers which produce `SafeString` e.g. `concat`
helper for string concatenation.
2023-03-01 08:39:14 +01:00
Reupen Shah
c5a4ee89c9 🐛 Fixed uploads of m4a files with audio/mp4 content type
fixes https://github.com/TryGhost/Ghost/issues/16301

Previously, audio/x-m4a was allowed but not audio/mp4. This meant
uploads of m4a files failed in some cases e.g. Firefox on Windows.
2023-03-01 08:32:21 +01:00
Sam Lord
579ecd5b0b Prevent error logging in Urls tests from being output
refs: https://github.com/TryGhost/Toolbox/issues/389
2023-02-28 11:14:11 +00:00
Sam Lord
6512fc770d Prevent xmlprc tests from outputting errors for failure cases
refs: https://github.com/TryGhost/Toolbox/issues/389
2023-02-28 11:14:11 +00:00
Sam Lord
007adb6a33 Prevent tests from erroring when calling validate.init()
refs: https://github.com/TryGhost/Toolbox/issues/389

Calling validate always uses the cache system, so this commit makes sure that the cache system is always initialised correctly by the tests.
2023-02-28 11:14:11 +00:00
Daniel Lockyer
93e0898c12
Extracted version-match middleware to separate package
- this middlware is standalone and has 100% test coverage, therefore we
  can move it out to a separate package and remap the requires
  accordingly
2023-02-27 18:34:03 +01:00
Aileen Nowak
d441fe2e2a Fixed tests 2023-02-23 17:21:20 +02:00
Aileen Nowak
086da740a2 Changed maxPercentageFromMilestone to 10% 2023-02-23 17:21:20 +02:00
Aileen Nowak
74a65f3b1c Simplyfied milestone event meta data
no issue

- Reduced the amount of diffeerent properties by not populating a `currentARR` and `currentMembers` fields, but use a `currentValue` instead.
- The type of milestone can still be determined by its `type` property, so we actually don't need two different props here
2023-02-23 17:21:20 +02:00
Aileen Nowak
d820b961b0 Avoid sending milestone email when actual value is too far from milestone
no issue

- Switches to used newly added config values throughout the services
- Updated the `shouldSendEmail` fn to check if actual value is too far from achieved milestone as determined by the percentage setting (e. g. 998 members should not accidentally receive an email for achieving 100 members)
2023-02-23 17:21:20 +02:00
Simon Backx
89ababb71f
Updated email event fetching to stop when begin and end are the same (#16326)
no issue

Optimization that makes sure we stop fetching when it is no longer
needed.
2023-02-23 15:44:01 +01:00
Simon Backx
3ded0bbee8
Added outbound link tagging setting (#16324)
no issue

Enable or disable outbound link tagging in both web posts and
newsletters.
2023-02-23 14:37:23 +01:00
Aileen Nowak
ff3568293c Updated milestones e2e test to better check for last email sent safeguard
refs https://www.notion.so/ghost/Marketing-Milestone-email-campaigns-1d2c9dee3cfa4029863edb16092ad5c4?pvs=4

- Didn't check for the specific case before and fixed a bug that was allowiing the test to pass even though it should fail
2023-02-23 14:10:12 +02:00
Simon Backx
f6bc3479f9
Added webmentions emails flag (#16321)
fixes https://github.com/TryGhost/Team/issues/2599

Disables sending webmentions notification emails, and hides it behind a new flag instead.
2023-02-23 11:42:14 +01:00
Aileen Booker
e0331bbfcf
Added MilestoneCreatedEvent to staff service notifications (#16307)
refs
https://www.notion.so/ghost/Marketing-Milestone-email-campaigns-1d2c9dee3cfa4029863edb16092ad5c4?pvs=4

- Added MilestoneCreatedEvent subscription to staff-service incl. first
handling
- Logs information for now instead of actually sending an email
2023-02-23 11:20:13 +02:00
Aileen Nowak
80ebd1c5be Fixed random timeouts in milestone service e2e test
no issue

- Passed specific config to `useFakeTimers` so the test only fakes the `setTimeout` call
2023-02-22 20:26:53 +02:00
Simon Backx
7220be92e2 Webmentions sending tests stability improvements
no issue
2023-02-22 18:25:06 +01:00
Steve Larson
18eeb9e523 moved spam prevention test
no refs
-spam prevention test was causing subsequent tests to fail randomly
-moving to the end ensures (for now) we don't interrupt other tests
-seems to be an issue with awaiting the jobservice which do concurrent
2023-02-22 09:52:26 -06:00
Simon Backx
a7090dddcd Email snapshot updates 2023-02-22 16:49:28 +01:00
Simon Backx
f45d1810a6 Improved webmention receiving E2E test reliability
refs https://github.com/TryGhost/Team/issues/2596
2023-02-22 16:19:56 +01:00
Aileen Booker
3aa75ee5dc
Improved milestone e2e test (#16314)
refs
https://www.notion.so/ghost/Marketing-Milestone-email-campaigns-1d2c9dee3cfa4029863edb16092ad5c4?pvs=4

- Check with more realistic dates and validate changes on milestone
model
2023-02-22 16:15:21 +02:00
Aileen Booker
cf7d34d862
Added BookshelfMilestoneRepository implementation (#16305)
refs
https://www.notion.so/ghost/Marketing-Milestone-email-campaigns-1d2c9dee3cfa4029863edb16092ad5c4?pvs=4

This stores the received milestones in the database.
2023-02-22 15:53:29 +02:00
Ronald Langeveld
76640d9b7e
Added page resources to emails (#16315)
refs https://github.com/TryGhost/Team/issues/2572

- Passes post resource data to emails.
- Updated email templates accordingly.
- Added snapshot e2e tests.
2023-02-22 21:02:15 +08:00
Aileen Booker
4556e9be88
Added milestone_notifications column (#16306)
refs
https://www.notion.so/ghost/Marketing-Milestone-email-campaigns-1d2c9dee3cfa4029863edb16092ad5c4?pvs=4

- created a migration for a new boolean column in users that would
determine if the staff user gets an email when a new milestone is
achieved.
2023-02-22 14:13:50 +02:00
Aileen Booker
32630c27cc
Added milestones database table (#16300)
refs
https://www.notion.so/ghost/Marketing-Milestone-email-campaigns-1d2c9dee3cfa4029863edb16092ad5c4?pvs=4

- added new `milestones` database table to be able to store created
milestones.
- updated schema and tests to match.
2023-02-22 10:46:43 +02:00
Naz
ad6097b695
Refactored post's extra-attributes to use options
refs https://github.com/TryGhost/Toolbox/issues/522

- Having simpler method signature makes it easier to use it in different context - needed for changes in public resource repository
- TLDR of the changes - reduced parameter 'frame.options' -> 'options'
2023-02-22 15:46:56 +08:00
Ghost CI
d3563d9c06 Merged v5.35.1 into main 2023-02-21 15:36:02 +00:00
Simon Backx
461634c57c Fixed snapshot tests 2023-02-21 16:15:00 +01:00
Simon Backx
b665b1a3cc Implemented email analytics retrying (#16273)
fixes https://github.com/TryGhost/Team/issues/2562

New event fetching loops:
- Reworked the analytics fetching algorithm. Instead of starting again
where we stopped during the last fetching minus 30 minutes, we now just
continue where we stopped. But with ms precision (because no longer
database dependent after first fetch), and we stop at NOW - 1 minute to
reduce chance of missing events.
- Apart from that, a missing fetching loop is introduced. This fetches
events that are older than 30 minutes, and just processes all events a
second time to make sure we didn't skip any because of storage delays in
the Mailgun API.
- A new scheduled fetching loop, that allows us to schedule between a
given start/end date (currently only persisted in memory, so stops after
a reboot)

UI and endpoint changes:
- New UI to show the state of the analytics 'loops'
- New endpoint to request the analytics loop status
- New endpoint to schedule analytics
- New endpoint to cancel scheduled analytics
- Some number formatting improvements, and introduction of 'opened'
count in debug screen
- Live reload of data in the debug screen

Other changes:
- This also improves the support for maxEvents. We can now stop a
fetching loop after x events without worrying about lost events. This is
used to reduce the fetched events in the missing and scheduled event
loop (e.g. when the main one is fetching lots of events, we skip the
other loops).
- Prevents fetching the same events over and over again if no new events
come in (because we always started at the same begin timestamp). The
code increases the begin timestamp with 1 second if it is safe to do so,
to prevent the API from returning the same events over and over again.
- Some optimisations in handing the processing results (less merges to
reduce CPU usage in cases we have lots of events).

Testing:
- You can test with lots of events using the new mailgun mocking server
(Toolbox repo `scripts/mailgun-mock-server`). This can also simulate
events that are only returned after x minutes because of storage delays.
2023-02-21 16:00:32 +01:00
Simon Backx
923c522778
Implemented email analytics retrying (#16273)
fixes https://github.com/TryGhost/Team/issues/2562

New event fetching loops:
- Reworked the analytics fetching algorithm. Instead of starting again
where we stopped during the last fetching minus 30 minutes, we now just
continue where we stopped. But with ms precision (because no longer
database dependent after first fetch), and we stop at NOW - 1 minute to
reduce chance of missing events.
- Apart from that, a missing fetching loop is introduced. This fetches
events that are older than 30 minutes, and just processes all events a
second time to make sure we didn't skip any because of storage delays in
the Mailgun API.
- A new scheduled fetching loop, that allows us to schedule between a
given start/end date (currently only persisted in memory, so stops after
a reboot)

UI and endpoint changes:
- New UI to show the state of the analytics 'loops'
- New endpoint to request the analytics loop status
- New endpoint to schedule analytics
- New endpoint to cancel scheduled analytics
- Some number formatting improvements, and introduction of 'opened'
count in debug screen
- Live reload of data in the debug screen

Other changes:
- This also improves the support for maxEvents. We can now stop a
fetching loop after x events without worrying about lost events. This is
used to reduce the fetched events in the missing and scheduled event
loop (e.g. when the main one is fetching lots of events, we skip the
other loops).
- Prevents fetching the same events over and over again if no new events
come in (because we always started at the same begin timestamp). The
code increases the begin timestamp with 1 second if it is safe to do so,
to prevent the API from returning the same events over and over again.
- Some optimisations in handing the processing results (less merges to
reduce CPU usage in cases we have lots of events).

Testing:
- You can test with lots of events using the new mailgun mocking server
(Toolbox repo `scripts/mailgun-mock-server`). This can also simulate
events that are only returned after x minutes because of storage delays.
2023-02-20 16:44:13 +01:00
Steve Larson
2d84b7d990
Upgraded got package from v9.6.0 to v11.8.6 (#16261)
Refs TryGhost/Team#2459
-upgraded got from v9.6.0 to v11.8.6 to support following redirects (and
other fixes)
-got v12+ requires ESM, so we do not want to upgrade further at this
time
-required changes to a few libraries that use externalRequests
-mention discovery service tests updated to test for follow redirects
2023-02-20 09:33:11 -06:00
Simon Backx
52a26a7f80 Removed unreliable upload test
no issue

This test failed randomly. It is hard to improve the test without defeating the purpose of the test.
2023-02-20 13:58:01 +01:00
Aileen Booker
386298310b
Added timeout for running milestone service (#16290)
no issue

- Instead of running milestone service directly on boot, set a random
timeout of 0-4 days to run after boot
- Updated tests
- Service is still behind a beta flag
2023-02-20 14:14:06 +02:00
Simon Backx
ec3f262ea4 Increased scheduling default test timeout 2023-02-17 13:29:07 +01:00
Fabien "egg" O'Carroll
8908a51547 Improved verification logic for Mentions
refs https://github.com/TryGhost/Team/issues/2550

By using cheerio to parse the HTML we can correctly look for elements
which use the target URL as the href attribute, rather than doing a
plaintext search. This closer to what the spec says.
2023-02-17 19:07:11 +07:00
Fabien 'egg' O'Carroll
101cabb188
Added separate job service for webmentions (#16285)
[Added initial mentions-jobs
service](3656190114)

This is the result of running `cp -r jobs mentions-jobs` in the services
directory.

[Waited for mentions-jobs queue before
shutdown](2bb1a12a89)

This matches the functionality of the existing jobs service where we
will wait
for jobs to complete before closing the process.

[Used mentions-jobs service in the mentions
service](4e4f9fdd00)

This ensures that any delays in the mentions jobs queue does not effect
other
parts of the application.
2023-02-17 13:05:36 +01:00
Daniel Lockyer
2edfd8b8e0
Increased timeout to allow for delays in browser e2e test
- this test was randomly timing out, and the tests above allow for a
  greater timeout before failing
2023-02-17 12:13:09 +01:00
Aileen Booker
2f57e95a5d
Slack notifications service for Milestones behind flag (#16281)
refs
https://www.notion.so/ghost/Marketing-Milestone-email-campaigns-1d2c9dee3cfa4029863edb16092ad5c4?pvs=4

- Added a `slack-notifications` repository which handles sending Slack
messages to a URL as defined in our Ghost(Pro) config (also includes a
global switch to disable the feature if needed) and listens to
`MilestoneCreatedEvents`.
- Added a `slack-notification` service which listens to the events on
boot.
- In order to have access to further information such as the reason why
a Milestone email hasn't been sent, or the current ARR or Member value
as comparison to the achieved milestone, I added a `meta` object to the
`MilestoneCreatedEvent` which then gets accessible by the event
subscriber. This avoid doing further requests to the DB as we need to
have this information in relation to the event occurred.

---------

Co-authored-by: Fabien "egg" O'Carroll <fabien@allou.is>
2023-02-17 12:59:18 +02:00
Steve Larson
1e00bd2af9 Handled empty content when sending webmentions
refs https://ghost.slack.com/archives/C02G9E68C/p1676564978732119

- `cheerio` errors when trying to parse `null`
2023-02-17 10:46:48 +01:00
Naz
f27c7ba044
Fixed variable name typo 2023-02-17 17:27:51 +08:00
Simon Backx
8dfa490638
Added outbound link tagging for web posts (#16201)
fixes https://github.com/TryGhost/Team/issues/2433

- Moved all outbound link tagging code to separate OutboundLinkTagger
- Because a site can easily enable/disable this feature, we don't store
the ?refs in the HTML but add them on the fly for now in the Content
API.
2023-02-16 11:26:35 +01:00
Fabien "egg" O'Carroll
c6b90b6ac7 Added verified property to Mentions Admin API
closes https://github.com/TryGhost/Team/issues/2551

Rather than blindly passing all data through the API we explicitly include each
new property. This allows us to make changes to the core entities without
affecting the API. The verified property is being added now to give design the
ability to display these mentions differently.

We also needed to include the verified property in the return value of toJSON,
this was missed as part of the original entity changes
2023-02-16 14:31:05 +07:00
Aileen Nowak
a400353881 Renamed milestone-emails to milestones
no issue

- The way we're going to implement milestones diverged from the original idea of handling email sending within the milestone-emails package, as we'll be sending events instead and will utilise the StaffService to listen to them and send the emails
- This renames the package as well as the service in core itself and all relevant tests
2023-02-15 10:43:05 +02:00
Aileen Booker
41e4132a17
Refactored milestones service to preserve API instance (#16272)
no issue

- For better testability with in-memory repository, refactor the
milestones service to preserve the API instance
- Fetching the information about Stripe live mode from Stripe service
was causing difficulties when testing. As a workaround we switched to
reading the live mode keys and determine it that way.

---------

Co-authored-by: Fabien "egg" O'Carroll <fabien@allou.is>
2023-02-15 08:43:40 +02:00
Ronald Langeveld
f9eeec10db Updated email-preview test snapshot
no issue

- fixed up tests failing from the outdated email snapshot.
2023-02-15 11:21:37 +08:00
Ronald Langeveld
c651f5b10f
Added verified mention column (#16264)
refs https://github.com/TryGhost/Team/issues/2549

- Added `verified` column to mentions table
- Added a new migration
- Updated schema
- Updated integrity test
2023-02-14 15:46:07 +08:00
Kevin Ansfield
c489343831
Fixed saving and rendering of Lexical posts containing new cards
closes https://github.com/TryGhost/Team/issues/2558

- bumped `kg-lexical` packages so we're working with latest suite of default nodes and renderer
- added a `render()` method directly to our `lexicalLib` object
  - allows us to pass through all of Ghost's config for image transforms etc in one place rather than every time we want to render something
2023-02-13 16:35:34 +00:00
Simon Backx
4e361a65fa Added webmention delete test and cleaned up webmention tests
refs https://github.com/TryGhost/Team/issues/2553

- Added webmention delete test
- Fixed dangling promise in test
- Fixed missing .persist in tests
2023-02-13 17:15:34 +01:00
Daniel Lockyer
4bca725215
Added basic unit test for cache adapter module
- this helps improve our testing capabilities of the new cache adapters
2023-02-13 17:10:42 +01:00
Aileen Booker
6f0d1b0ff9
Added milestone email service behind alpha flag (#16241)
refs
https://www.notion.so/ghost/Marketing-Milestone-email-campaigns-1d2c9dee3cfa4029863edb16092ad5c4

Added milestone email service behind a flag. The service will currently
run on boot and is meant to be scheduled soon, which should happen in
the next step. For now it's protected behind the alpha flag.
2023-02-13 16:29:01 +02:00
Simon Backx
48f9485f46
🐛 Fixed storing email failures with an empty message (#16260)
no issue

- When we receive an email failure with an empty message, the saving of
the model would fail because of schema validation that requires strings
to be non-empty.
- This adds more logging to the email analytics service to help debug
future issues
- Performance improvement to storing delivered, opened and failed emails
by replacing COALESCE with WHERE X IS NULL (tested and should give a
decent performance boost locally).
2023-02-13 15:25:36 +01:00
Fabien "egg" O'Carroll
bf6bcdbff3 Added e2e tests for updating Mentions
closes https://github.com/TryGhost/Team/issues/2552

We send a Webmention for the same URL twice, but change the contents
of the source document, and we check that the source metadata is
updated appropriately.

We should consider extending all of these tests to include featured
images and logos etc...
2023-02-13 19:09:31 +07:00
Simon Backx
77032262c4
🐛 Fixed subscriptions visible as "Active" within Ghost Admin (#16255)
fixes https://github.com/TryGhost/Team/issues/2542 
fixes https://github.com/TryGhost/Team/issues/2543 
fixes https://github.com/TryGhost/Team/issues/2544

- Hides incomplete subscriptions
- Shows Past Due subscriptions
- Fixed UI issues with 3+ subscriptions
- Fixed missing complimentary subscription when one subscription was
incomplete/inactive
- Fixed sending a paid subscription started email for incomplete
subscriptions. This change also required us to actually send the email
when the incomplete subscription eventually becomes active. So the
introduction of a new `SubscriptionActivatedEvent` made sense/was
required (because sending a SubscriptionCreatedEvent again would cause
other issues).
2023-02-13 13:07:53 +01:00
Fabien 'egg' O'Carroll
57695d03df
Fixed rate limit test (#16258)
closes https://github.com/TryGhost/Team/issues/2547

Changed the configuration for testing to be a bit more strict, by slowing down the amount of requests it can handle to give CI enough time to kick in the rate limiter. Before this, CI simply wasn't hitting the API fast enough to trigger the rate limiter.

Co-authored-by: Ronald Langeveld <hi@ronaldlangeveld.com>
2023-02-13 15:16:56 +08:00
Ronald Langeveld
4096571b8d Commented out flaky webmention test
refs https://ghost.slack.com/archives/C02G9E68C/p1675934676146349

- Commented out the rate limiter test as it proves to be flaky on CI,
  till we find a better way of implementing it.
2023-02-09 21:35:15 +08:00
Steve Larson
d20696805f
Added mentions deleted column (#16251)
refs https://github.com/TryGhost/Team/issues/2534

This is so that we can support soft deletes for Mentions.
We need to add the defaults to the model so that write continue to work.

Co-authored-by: Fabien "egg" O'Carroll <fabien@allou.is>
2023-02-09 16:19:07 +07:00
Ronald Langeveld
ce567b9816
Added Mentions email notification settings (#16245)
closes https://github.com/TryGhost/Team/issues/2526

- Mention emails can now be toggled inside staff user' profiles, if they
have the webmention flag enabled on their Ghost site.
- Removed the flag dedicated to webmention email notifications and is
now handled by the `webmention` flag.
- Does not send email notification if `webmention` flag is not enabled.
- Updated tests.

---------

Co-authored-by: Fabien "egg" O'Carroll <fabien@allou.is>
2023-02-09 17:03:03 +08:00
Simon Backx
ea2c69565f
Moved email event fetching to main thread (#16231) 2023-02-09 09:36:39 +01:00