Commit Graph

16065 Commits

Author SHA1 Message Date
Daniel Lockyer
2edfd8b8e0
Increased timeout to allow for delays in browser e2e test
- this test was randomly timing out, and the tests above allow for a
  greater timeout before failing
2023-02-17 12:13:09 +01:00
Aileen Booker
2f57e95a5d
Slack notifications service for Milestones behind flag (#16281)
refs
https://www.notion.so/ghost/Marketing-Milestone-email-campaigns-1d2c9dee3cfa4029863edb16092ad5c4?pvs=4

- Added a `slack-notifications` repository which handles sending Slack
messages to a URL as defined in our Ghost(Pro) config (also includes a
global switch to disable the feature if needed) and listens to
`MilestoneCreatedEvents`.
- Added a `slack-notification` service which listens to the events on
boot.
- In order to have access to further information such as the reason why
a Milestone email hasn't been sent, or the current ARR or Member value
as comparison to the achieved milestone, I added a `meta` object to the
`MilestoneCreatedEvent` which then gets accessible by the event
subscriber. This avoid doing further requests to the DB as we need to
have this information in relation to the event occurred.

---------

Co-authored-by: Fabien "egg" O'Carroll <fabien@allou.is>
2023-02-17 12:59:18 +02:00
Steve Larson
034a230365 Added alpha feature to demonstrate websockets
refs https://github.com/TryGhost/Team/issues/2561

- added simple socket-io implementation to Ghost server
- added alpha flag for websockets
- added route in admin to test websockets using a simple counter stored in server local memory (refreshes on reboot)
2023-02-17 11:55:15 +01:00
Daniel Lockyer
e52efb31b8
Updated coverage exemption list
- like the members jobs, we don't need to run unit tests on these
  because they're not library code
2023-02-17 11:43:13 +01:00
Steve Larson
1e00bd2af9 Handled empty content when sending webmentions
refs https://ghost.slack.com/archives/C02G9E68C/p1676564978732119

- `cheerio` errors when trying to parse `null`
2023-02-17 10:46:48 +01:00
Naz
f27c7ba044
Fixed variable name typo 2023-02-17 17:27:51 +08:00
Naz
3ddfe59e15
Added resource caching to Posts/Pages Content API
refs https://github.com/TryGhost/Toolbox/issues/522

- Browse endpoints for Posts and Pages are creating the most database traffic in the system. These are read-only endpoints that don't have to be fresh 100% of the time. Having optional cache allows to offload some of the database querying to more efficient storage.
- To enable cache for Posts/Pages browse endpoints there are two prerequisites:
  - set 'hostSettings:postsPublicCache:enabled' to 'true' in the configuration file
  - add 'postsPublic' cache adapter in cache configuration

- Example config for adapters with 60s TTL for a cached resource:
```
    "adapters": {
        "cache": {
            "postsPublic": {
                "adapter": "Redis",
                "ttl": 60,
                "keyPrefix": "site_id_here:posts-content-api:"
            }
        }
    },
```
2023-02-17 17:27:51 +08:00
Naz
c3d7104367
Refactored tags repository to be model-agnostic
refs https://github.com/TryGhost/Toolbox/issues/522

- The caching rules for Content API resources like Tags / Posts / Pages / Authors are the same, so it makes sense to make a common repository which would take in a model as a parameter and perform the queries on that level instead of implementing repository-per-model.
- With this refactor getting Posts Content API caching would be much simpler change.
2023-02-17 17:27:51 +08:00
Steve Larson
5d00b64e27
Added timeout to external requests
no refs

- got doesn't have a timeout by default
- we think we might have hanging jobs because of this
2023-02-17 09:16:58 +01:00
Daniel Lockyer
9a2d765c73
Added one-off job name and current queue length into log line
- this makes it easier to know what jobs are being scheduled and what
  the current queue length is
2023-02-17 09:16:40 +01:00
Aileen Nowak
cd3930d08e Promote milestoneEmails to beta feature
no issue

- Promotes `milestoneEmails` to a beta feature so it can be a private beta to be tested
2023-02-17 07:45:51 +02:00
Aileen Nowak
1c0ee675fe Updated milestone values in default config 2023-02-17 07:45:51 +02:00
renovate[bot]
7de35f1ca3 Update dependency node-jose to v2.2.0 [SECURITY] 2023-02-17 05:29:46 +00:00
renovate[bot]
cf7ecb492c
Update dependency c8 to v7.13.0 2023-02-16 22:15:50 +00:00
Sam Lord
b4c90bf07d Reduced number of login events in large-scale data generation
no issue

This should massively increase the speed of importing for the large dataset, which is important as the time to import it on Pro is >10 minutes at the moment
2023-02-16 20:54:14 +00:00
renovate[bot]
38642af435 Update dependency cssnano to v5.1.15 2023-02-16 14:59:59 +01:00
Sam Lord
b8f59f07f2 Added --clear-database option to make it easier to use the data generator
no issue
2023-02-16 12:11:08 +00:00
Simon Backx
f8e77ad012 🐛 Fixed uploading images with custom storage adapters
fixes https://github.com/TryGhost/Ghost/issues/16278

`urlToPath` is not a method that is defined on the StorageBase, and thus is not implemented on any custom storage adapter.

We need this method to prevent uploading a file in two different directories. Currently this is an edge case:
- If you upload a file at midnight between a month shift, it is possible that we upload one file at `/2023/02/file.png` and the original at `/2023/03/file_o.png`
- To make sure we upload the second at the same directory, we need the `urlToPath` implementation. The save method sadly only returns the URL, which could be different than the stored file path, so we cannot use that return value.

This fix only uses `urlToPath` if available, to prevent above issue.
2023-02-16 11:49:37 +01:00
Simon Backx
8dfa490638
Added outbound link tagging for web posts (#16201)
fixes https://github.com/TryGhost/Team/issues/2433

- Moved all outbound link tagging code to separate OutboundLinkTagger
- Because a site can easily enable/disable this feature, we don't store
the ?refs in the HTML but add them on the fly for now in the Content
API.
2023-02-16 11:26:35 +01:00
Fabien "egg" O'Carroll
a310ec2e3e Fixed Mention unit test
refs https://github.com/TryGhost/Ghost/commit/c6b90b6ac
2023-02-16 14:37:22 +07:00
Fabien "egg" O'Carroll
c6b90b6ac7 Added verified property to Mentions Admin API
closes https://github.com/TryGhost/Team/issues/2551

Rather than blindly passing all data through the API we explicitly include each
new property. This allows us to make changes to the core entities without
affecting the API. The verified property is being added now to give design the
ability to display these mentions differently.

We also needed to include the verified property in the return value of toJSON,
this was missed as part of the original entity changes
2023-02-16 14:31:05 +07:00
Ghost CI
dd48f65712 Merged v5.34.1 into main 2023-02-16 07:23:52 +00:00
Ghost CI
443085ab87 v5.34.1 2023-02-16 07:23:49 +00:00
Naz
8a0c20b54c
Fixed Redis cluster ttl configuration
refs https://github.com/TryGhost/Toolbox/issues/520

- The cluster config is taking over local adapter ttl configuration - the priority should be reverse adapter config first followed by cluster config
- In addition if we add nested config merging to adapter manager we could achieve the same effect by having per-adapter configuraiton with "clusterConfig.options.ttl" value
2023-02-16 14:13:44 +08:00
John O'Nolan
14c5e48190
Improved micro copy on Portal email tips 2023-02-15 12:30:03 -04:00
Aileen Nowak
5f12c7ad40 Updated Milstone test to expect no event for existing milestone 2023-02-15 14:28:57 +02:00
Aileen Nowak
dc8dc90212 Exposed MilestoneCreatedEvent as module 2023-02-15 14:28:57 +02:00
Aileen Nowak
d482046a25 Moved event creation to all save events and not only new ones 2023-02-15 14:28:57 +02:00
Aileen Nowak
b55a95b944 Removed usage of GhostMailer in MilestonesService
no issue

- With the switch of using a `MilestoneCreatedEvent` we'll be decoupling the mailing functionality and not need `GhostMailer` as dependency in the package anymore
2023-02-15 14:28:57 +02:00
Aileen Nowak
513b7d1df4 Added MilestoneCreatedEvent using DomainEvents
no issue

- In preparation of using event emitting for Milestone achievements, we needed to add a dedicated `MilestoneCreatedEvent` to the `Milestone` entity.
- The event will be emitted using `DomainEvents` when a new milesteone is saved, which will allow us to listen to these events.
2023-02-15 14:28:57 +02:00
renovate[bot]
636ff6d4ef Update sentry-javascript monorepo to v7.37.2 2023-02-15 11:25:44 +01:00
Aileen Nowak
a400353881 Renamed milestone-emails to milestones
no issue

- The way we're going to implement milestones diverged from the original idea of handling email sending within the milestone-emails package, as we'll be sending events instead and will utilise the StaffService to listen to them and send the emails
- This renames the package as well as the service in core itself and all relevant tests
2023-02-15 10:43:05 +02:00
Aileen Booker
41e4132a17
Refactored milestones service to preserve API instance (#16272)
no issue

- For better testability with in-memory repository, refactor the
milestones service to preserve the API instance
- Fetching the information about Stripe live mode from Stripe service
was causing difficulties when testing. As a workaround we switched to
reading the live mode keys and determine it that way.

---------

Co-authored-by: Fabien "egg" O'Carroll <fabien@allou.is>
2023-02-15 08:43:40 +02:00
Naz
d62eba6c7f
Fixed Redis cluster ttl configuration
refs https://github.com/TryGhost/Toolbox/issues/520

- The cluster config is taking over local adapter ttl configuration - the priority should be reverse adapter config first followed by cluster config
- In addition if we add nested config merging to adapter manager we could achieve the same effect by having per-adapter configuraiton with "clusterConfig.options.ttl" value
2023-02-15 11:39:04 +08:00
Ronald Langeveld
f9eeec10db Updated email-preview test snapshot
no issue

- fixed up tests failing from the outdated email snapshot.
2023-02-15 11:21:37 +08:00
Peter Zimon
5bb5977a47 Fixed issue with bookmark cards width on Android
no refs.

- On Android Gmail client the bookmark cards were not taking the full width of the screen
2023-02-14 14:31:16 +01:00
Ronald Langeveld
c651f5b10f
Added verified mention column (#16264)
refs https://github.com/TryGhost/Team/issues/2549

- Added `verified` column to mentions table
- Added a new migration
- Updated schema
- Updated integrity test
2023-02-14 15:46:07 +08:00
renovate[bot]
7f278bd762 Update Test & linting packages 2023-02-14 04:25:29 +00:00
John O'Nolan
0334595a0e Updated post preview meta description prompt 2023-02-13 16:58:28 -04:00
John O'Nolan
4f3733a7cf 🎨 Improved meta description preview 2023-02-13 16:30:02 -04:00
Ghost CI
e8d3add2bd v5.34.0 2023-02-13 17:04:36 +00:00
Kevin Ansfield
6fe8154329
Fixed email rendering of lexical posts
refs c489343831

- `lexicalLib.lexicalHtmlRenderer` can't be used directly because it's missing required config for nodes to render properly
- updated email serializers to use `lexicalLib.render()` instead
2023-02-13 16:48:30 +00:00
Kevin Ansfield
c489343831
Fixed saving and rendering of Lexical posts containing new cards
closes https://github.com/TryGhost/Team/issues/2558

- bumped `kg-lexical` packages so we're working with latest suite of default nodes and renderer
- added a `render()` method directly to our `lexicalLib` object
  - allows us to pass through all of Ghost's config for image transforms etc in one place rather than every time we want to render something
2023-02-13 16:35:34 +00:00
Simon Backx
4e361a65fa Added webmention delete test and cleaned up webmention tests
refs https://github.com/TryGhost/Team/issues/2553

- Added webmention delete test
- Fixed dangling promise in test
- Fixed missing .persist in tests
2023-02-13 17:15:34 +01:00
Daniel Lockyer
4bca725215
Added basic unit test for cache adapter module
- this helps improve our testing capabilities of the new cache adapters
2023-02-13 17:10:42 +01:00
Kevin Ansfield
247d0f1f92
Removed editor.url from config.development.json
no issue

- local development Ghost installs should not default to a locally served `koenig-lexical.umd.js` file but should use the default CDN version
- a locally served file should be set in `config.local.json` when you want to develop against an unreleased version
2023-02-13 15:10:49 +00:00
Kevin Ansfield
440d4c41f9
Fixed media thumbnail uploads in koenig-lexical editor
no issue

- added `requestMethod` to `fileTypes` map
- added pass-through of `formData` options to `upload(file, options)` so `url` property can be passed to map uploaded image file to media file
- fixed `resourceName` for media thumbnail uploads
2023-02-13 15:00:29 +00:00
Simon Backx
0e8d13bdde 🐛 Fixed email replacements without fallback
fixes https://github.com/TryGhost/Team/issues/2557

When a member doen't have a name, and the first_name replacement doesn't have a fallback, we did show %recipient.first_name% instead of an empty string.
2023-02-13 15:58:40 +01:00
Aileen Booker
6f0d1b0ff9
Added milestone email service behind alpha flag (#16241)
refs
https://www.notion.so/ghost/Marketing-Milestone-email-campaigns-1d2c9dee3cfa4029863edb16092ad5c4

Added milestone email service behind a flag. The service will currently
run on boot and is meant to be scheduled soon, which should happen in
the next step. For now it's protected behind the alpha flag.
2023-02-13 16:29:01 +02:00
Simon Backx
48f9485f46
🐛 Fixed storing email failures with an empty message (#16260)
no issue

- When we receive an email failure with an empty message, the saving of
the model would fail because of schema validation that requires strings
to be non-empty.
- This adds more logging to the email analytics service to help debug
future issues
- Performance improvement to storing delivered, opened and failed emails
by replacing COALESCE with WHERE X IS NULL (tested and should give a
decent performance boost locally).
2023-02-13 15:25:36 +01:00
Kevin Ansfield
fb882e2e2a
Updated koenig-lexical upload function to handle multiple file types
refs https://github.com/TryGhost/Koenig/pull/491

- `<KoenigComposer>` now takes a `fileUpoader` object in place of `imageUploadFunction`
- updated the upload functions in `koenig-lexical-editor.js` to match expected patterns, handle multiple files and file types, and return expected upload progress, result, and error details
2023-02-13 13:38:35 +00:00
Sam Lord
9e42d1356a Updated avatar styles for webmentions
no issue

This makes the style for generated webmentions much better.
2023-02-13 13:16:24 +00:00
Fabien "egg" O'Carroll
90b7a3f4d0 Added verified member and verify method to Mention entity
closes https://github.com/TryGhost/Team/issues/2548

Rather than use a setter here we've used a verify method which takes the HTML
string and naively validates that the target URL is present. This is so that the
logic of verification is encapsulated in the Mention, and should mean that
erroneous verification doesn't happen.

We could consider down the line that the verify method fetches the content
itself, but if we're to do that we should pass in `got` as a param, so that it's
possible to stub in tests.

One thing to think about when it comes time to making this as performant as
possible is doing a single fetch of the source document and using that for
verification and metadata extraction. At that point we should probably
consolidate both of those operations, either moving the metadata extraction into
the Mention entity (passing in any necessary deps) OR we move the verification
out to the same layer as metadata extraction.
2023-02-13 19:23:34 +07:00
Fabien "egg" O'Carroll
bf6bcdbff3 Added e2e tests for updating Mentions
closes https://github.com/TryGhost/Team/issues/2552

We send a Webmention for the same URL twice, but change the contents
of the source document, and we check that the source metadata is
updated appropriately.

We should consider extending all of these tests to include featured
images and logos etc...
2023-02-13 19:09:31 +07:00
Simon Backx
77032262c4
🐛 Fixed subscriptions visible as "Active" within Ghost Admin (#16255)
fixes https://github.com/TryGhost/Team/issues/2542 
fixes https://github.com/TryGhost/Team/issues/2543 
fixes https://github.com/TryGhost/Team/issues/2544

- Hides incomplete subscriptions
- Shows Past Due subscriptions
- Fixed UI issues with 3+ subscriptions
- Fixed missing complimentary subscription when one subscription was
incomplete/inactive
- Fixed sending a paid subscription started email for incomplete
subscriptions. This change also required us to actually send the email
when the incomplete subscription eventually becomes active. So the
introduction of a new `SubscriptionActivatedEvent` made sense/was
required (because sending a SubscriptionCreatedEvent again would cause
other issues).
2023-02-13 13:07:53 +01:00
Naz
7c18263227 Extracted adapter-cache-redis from core codebase
refs https://github.com/TryGhost/Toolbox/issues/515

- This implementation allows to use Redis cluster as a caching adapter. The cache adapter can be configured through same adapter configuration interface as others. It accepts following config values:
  - "ttl" - time in SECONDS for stored entity to live in Redis cache
  - "keyPrefix" - special cache key prefix to use with stored entities
  - "host" / "port" / "password" / "clusterConfig" - Redis instance specific configs
- Set test coverage to non-standard 75% because the code mostly consists of the glue code that would require unnecessary Redis server mocking and would be a bad ROI. This module has been used in production for a long time already, so can be considered pretty stable.
2023-02-13 19:01:43 +08:00
Naz
b3252f956f Added adapter-cache-redis package
refs https://github.com/TryGhost/Toolbox/issues/515

- Redis-based caches can be used on hosted-environments to store information with high memory impact - when in-memory caches would be too impractical to use
- This is a placeholder package for a cluster-aware Redis cache implementation compatible with Ghost's cache adapter interface (a41d351f16/packages/adapter-base-cache)
2023-02-13 19:01:43 +08:00
Naz
1c12d789f9 Added async cache support to Tags repository
refs https://github.com/TryGhost/Toolbox/issues/515

- The caches relying on external storage like e.g.: Redis, the get/set operations are usually async. The tags repository should be working with these as caching is expected to be non-in-memory for these data.
2023-02-13 19:01:43 +08:00
Naz
9d4ea7b8c9 Moved cache initialization behind the hostSetting
refs https://github.com/TryGhost/Toolbox/issues/515

- By moving the cache initialization behind the hostSettings configuration we can limit the experimental feature to only hosted environments with special capacities
- An example configuration to enable Tags caching looks like this:
```
    "hostSettings": {
        "tagsPublicCache": {
            "enabled": true
        },
```
In addition to have the caching backed by a Redis backend or even InMemoryTTL cache the site configuration should include a cache adapter configuration like this:
```
    "adapters": {
        "cache": {
            "active": "Memory",
            "tagsPublic": {
                "adapter": "TTL",
                "ttl": 60000 // 60 * 1000 minute
            },
```
2023-02-13 19:01:43 +08:00
Naz
5216220541 Public tags repository caching
refs https://github.com/TryGhost/Toolbox/issues/515

- There are a lot of repeated cacheable tag-related queries coming from
{{get}} helpers in themes that can be cached.
- Having a repository layer deal with very specific type of query allows
to add extra functionality, like caching, on top of the database query
- This commit is wiring code that addds a default in-memory cache to
all db queries. Note, it lasts forever and has no "reset" listeners. The
production cache is mean to have a short time-to-live (TTL) - removes a need
to keep the cache always fresh.
- Kept the cache key shortened. Without a "context" and any other non-model options the cache-key can store more variations of queries. For example, there is no member-specific or integration-specific query results, so having those in the cache key would only partition the cache and use up more memory.
2023-02-13 19:01:43 +08:00
renovate[bot]
b48262b5d8 Update sentry-javascript monorepo to v7.37.1 2023-02-13 09:22:57 +01:00
renovate[bot]
ccfe6dd638 Update dependency minimist to v1.2.8 2023-02-13 09:21:59 +01:00
Fabien 'egg' O'Carroll
57695d03df
Fixed rate limit test (#16258)
closes https://github.com/TryGhost/Team/issues/2547

Changed the configuration for testing to be a bit more strict, by slowing down the amount of requests it can handle to give CI enough time to kick in the rate limiter. Before this, CI simply wasn't hitting the API fast enough to trigger the rate limiter.

Co-authored-by: Ronald Langeveld <hi@ronaldlangeveld.com>
2023-02-13 15:16:56 +08:00
renovate[bot]
2c9fb2ad5e
Update dependency ember-template-lint to v5.5.0 2023-02-10 11:43:30 +00:00
Kevin Ansfield
d6b8ff124b
Updated <KoenigComposer> props to use generic cardConfig object
refs aed55d415d

- `unsplashConfig` has changed to a generic `cardConfig` object
2023-02-10 11:13:57 +00:00
Ronald Langeveld
b37aff52ca Mention alert toggle only visible to admins
refs https://ghost.slack.com/archives/C02G9E68C/p1675916876727609

- Added the mentions alert toggle to only be visible to owners and
  admins, since it will not have any effect for non-admins staff users.
2023-02-10 17:24:02 +08:00
Fabien "egg" O'Carroll
3df19235b0 Added support for updating Mentions when Webmentions are resent
refs https://github.com/TryGhost/Team/issues/2535

At the moment we only update the metadata of the Webmention source, so
that we can capture update titles, excerpts, etc... when a post is updated.
2023-02-10 11:43:16 +07:00
Fabien "egg" O'Carroll
78bebc6279 Refactored the assignment of Mention metadata
refs https://github.com/TryGhost/Team/issues/2535

Moving this into a separate method allows us to set the metadata externally from
the Mention entity and keep all of the validation, without having duplicate code
2023-02-10 11:43:16 +07:00
renovate[bot]
b271d5a53a Update metascraper to v5.33.8 2023-02-10 02:02:53 +00:00
renovate[bot]
3a11525ad7
Update dependency mysql2 to v3.1.2 2023-02-09 20:10:27 +00:00
Daniel Lockyer
49e9be4368 Reconfigured clean-expired-comped job to execute during morning UTC
refs https://ghost.slack.com/archives/C04MSE4MKJT/p1675948815531779

- running at a fixed hh:mm every day means a platform with a large number
  of Ghost sites will get hammered with DB requests when they all start
  up
- this reconfigures the cron to run at a random minute and second
  between 0am and 5am, which gives a 6 hour window
2023-02-09 15:58:43 +01:00
John O'Nolan
5b4abea56d
Fixed broken link 2023-02-09 10:53:53 -04:00
Ronald Langeveld
4096571b8d Commented out flaky webmention test
refs https://ghost.slack.com/archives/C02G9E68C/p1675934676146349

- Commented out the rate limiter test as it proves to be flaky on CI,
  till we find a better way of implementing it.
2023-02-09 21:35:15 +08:00
Fabien 'egg' O'Carroll
4e8af72305
Added support for deleting mentions (#16238)
refs https://github.com/TryGhost/Team/issues/2534

As we're using soft deletes for mentions we need to store the `deleted` column
as well as enforce a `'deleted:false'` filter on the bookshelf model. 

We've also implemented the handling for deleting mentions. Where we remove a
mention anytime we receive and update from or to a page which no longer exists.

Co-authored-by: Steve Larson <9larsons@gmail.com>
2023-02-09 17:29:13 +07:00
Steve Larson
d20696805f
Added mentions deleted column (#16251)
refs https://github.com/TryGhost/Team/issues/2534

This is so that we can support soft deletes for Mentions.
We need to add the defaults to the model so that write continue to work.

Co-authored-by: Fabien "egg" O'Carroll <fabien@allou.is>
2023-02-09 16:19:07 +07:00
Daniel Lockyer
d750612508
Merged v5.33.8 into main
v5.33.8
2023-02-09 10:08:43 +01:00
Ghost CI
83bf4ccb53 v5.33.8 2023-02-09 09:06:34 +00:00
Ronald Langeveld
ce567b9816
Added Mentions email notification settings (#16245)
closes https://github.com/TryGhost/Team/issues/2526

- Mention emails can now be toggled inside staff user' profiles, if they
have the webmention flag enabled on their Ghost site.
- Removed the flag dedicated to webmention email notifications and is
now handled by the `webmention` flag.
- Does not send email notification if `webmention` flag is not enabled.
- Updated tests.

---------

Co-authored-by: Fabien "egg" O'Carroll <fabien@allou.is>
2023-02-09 17:03:03 +08:00
Simon Backx
da97519082 Moved email event fetching to main thread (#16231)
refs https://github.com/TryGhost/Team/issues/2482

This moves the processing of Mailgun events to the main thread. By using a simple approach where we emit a start event on the worker thread (via the job manager) and listen for it on the main thread. This is needed because for now the job manager doesn't support scheduling periodic jobs on the main thread (not offloaded).

Apart from that, the email processor now uses the email event storage directy instead of emitting events (it is still emitting event for now). This makes sure we await for the event to be processed before continuing with the next event.
2023-02-09 09:44:44 +01:00
Simon Backx
ea2c69565f
Moved email event fetching to main thread (#16231) 2023-02-09 09:36:39 +01:00
renovate[bot]
fd79ca3f5a Update dependency html-validate to v7.13.2 2023-02-09 09:30:14 +01:00
Simon Backx
b7be02f057
🐛 Fixed throwing errors during link redirects
no issue

Errors weren't correctly passed on to Express in the middleware.
2023-02-09 09:26:45 +01:00
Kevin Ansfield
f7aefa2feb
🐛 Fixed default email recipients always being "everyone" rather than matching post visibility (#16247)
refs https://github.com/TryGhost/Ghost/issues/16125

The previous fix for incorrect recipient details being shown when
re-sending a failed email introduced another bug that prevented the
"Match post visibility" default recipients setting from working.

- the server always sets `post.emailSegment` to `'all'` for new posts so
the publish flow recipient filter logic that checked for
`post.emailSegment` being present always defaulted to `'all'` rather
than falling back to the selected default recipients setting
- when a post has been published but the email failed it will have its
`newsletter` value set so we can use that as a check for using the
`post.emailSegment` value in place of the default recipients setting
2023-02-09 09:24:50 +01:00
Ronald Langeveld
30dc2a8228
Added mention_notifications column (#16242)
refs https://github.com/TryGhost/Team/issues/2526

- created a migration for a new boolean column in users that would
determine if the staff user gets an email when the publication receive a
new mention.
2023-02-09 16:15:54 +08:00
Ronald Langeveld
9fc13cfe65
Implemented mentions rate limiter (#16223)
closes https://github.com/TryGhost/Team/issues/2419

- adds a rate limiter implementation to the mentions receiving
endpoint.
- Current configuration is `{"minWait": 10,
             "maxWait": 100,
             "lifetime": 1000,
             "freeRetries": 100}` which is still very open and almost unrestricted. 
- currently makes use of database storage to track the limits, but can be relatively easily swapped out to something eg Redis should we find this endpoint getting hit too often and maliciously.
2023-02-09 14:57:48 +08:00
Naz
93a10d8f4f Optimized queries for tiers (aka products)
refs https://github.com/TryGhost/Toolbox/issues/515
refs dd4d6aeae5

- The `productRepository.list` call produced 5 db queries and a transaction wrapping this call.
- Transaction is not needed in this situation as there are no possible writes in the meantime (transaction wrapping code was put in there through refed commit to guard against failing Stripe API calls, which are no longer involved when calling the list method)
-  The `limit: 'all'` makes sure all product entries are fetched AND removes an extra aggregation query called over stripe_prices join
- The 'monthlyPrice' and 'yearlyPrice' relations are not needed because this data is not used in downstream code - only slug and type are used for visiblity/content gating  (ref. 1 3b6759ca6d/ghost/core/core/server/services/members/content-gating.js (L44-L55), ref. 2 3b6759ca6d/ghost/core/core/server/api/endpoints/utils/serializers/output/mappers/posts.js (L39-L54))
2023-02-09 13:07:57 +08:00
Steve Larson
6c97edec25
moved sending mentions to job (#16234)
refs TryGhost/Team#2523
-implemented jobsService within mention sending service
-updated tests so we don't need to sleep
2023-02-08 16:29:12 -06:00
Sam Lord
0d7944861c Data generator: Swap order of free and premium newsletters
no issue
Free and premium newsletters were the other way around in the demo-data. This was a good opportunity to stop the email table importer from relying on the newsletter name, and use the order alone.
2023-02-08 22:12:19 +00:00
Kevin Ansfield
923a9a1cd6
🐛 Fixed default email recipients always being "everyone" rather than matching post visibility (#16247)
refs https://github.com/TryGhost/Ghost/issues/16125

The previous fix for incorrect recipient details being shown when
re-sending a failed email introduced another bug that prevented the
"Match post visibility" default recipients setting from working.

- the server always sets `post.emailSegment` to `'all'` for new posts so
the publish flow recipient filter logic that checked for
`post.emailSegment` being present always defaulted to `'all'` rather
than falling back to the selected default recipients setting
- when a post has been published but the email failed it will have its
`newsletter` value set so we can use that as a check for using the
`post.emailSegment` value in place of the default recipients setting
2023-02-08 11:41:48 +00:00
Simon Backx
4436a1cd18 🐛 Fixed throwing errors during link redirects
no issue

Errors weren't correctly passed on to Express in the middleware.
2023-02-08 11:50:22 +01:00
Aileen Booker
3b6759ca6d
Added initial basic milestone emails in-memory repository (#16216)
refs
https://www.notion.so/ghost/Marketing-Milestone-email-campaigns-1d2c9dee3cfa4029863edb16092ad5c4

This adds a milestone entity and in-memory repository in a new
`milestone-emails` package. This also adds a first initial definition of
milestones and their types which is held in the default config to avoid
DB changes when, e. g. values change.

This should get everything in place to begin with the service
implementation.
2023-02-07 12:47:35 +02:00
Ghost CI
c518446c17 Merged v5.33.7 into main 2023-02-07 10:32:47 +00:00
Ghost CI
ca568817dd v5.33.7 2023-02-07 10:32:43 +00:00
Simon Backx
ac2daccf95 Added short lived caching to email batch body (#16233)
fixes https://github.com/TryGhost/Team/issues/2522

When sending an email for multiple batches at the same time, we now
reuse the same email body for each batch in the same segment. This
reduces the amount of database queries and makes the sending more
reliable in case of database failures.

The cache is short lived. After sending the email it is automatically
garbage collected.
2023-02-07 11:30:37 +01:00
Simon Backx
fb3cbe5fc8
Added short lived caching to email batch body (#16233)
fixes https://github.com/TryGhost/Team/issues/2522

When sending an email for multiple batches at the same time, we now
reuse the same email body for each batch in the same segment. This
reduces the amount of database queries and makes the sending more
reliable in case of database failures.

The cache is short lived. After sending the email it is automatically
garbage collected.
2023-02-07 11:01:49 +01:00
renovate[bot]
654a991c98 Update dependency mysql2 to v3.1.1 2023-02-07 09:20:11 +01:00
Sam Lord
e0bcb26600 Improved data-generator tooling
no issue

This makes the data-generator more customisable.
2023-02-06 10:42:56 +00:00
renovate[bot]
c01875b85c Update sentry-javascript monorepo to v7.36.0 2023-02-06 10:26:32 +01:00
renovate[bot]
b76466946e Update dependency @types/express to v4.17.17 2023-02-06 10:26:06 +01:00
renovate[bot]
d60fea59e2 Update dependency terser to v5.16.3 2023-02-04 04:47:16 +00:00
Steve Larson
efd01f649b removed suppressionList from labs settings 2023-02-03 14:57:30 -06:00
Steve Larson
5ccb778ac8
enabled always sending mentions (#16227)
refs TryGhost/Team#2507
-removed labs check from sending service
-labs flag controls seeing mentions in admin and receiving mentions
2023-02-03 14:52:20 -06:00
Steve Larson
02953490a5
added mention sending service e2e tests (#16226)
refs TryGhost/Team#2508
-added sending service e2e tests
-should job off this sending service for better tests
-and for ghost to finish processing the job before shutdown
2023-02-03 10:45:50 -06:00
Ghost CI
d37ed200ab Merged v5.33.6 into main 2023-02-03 13:41:48 +00:00
Ghost CI
8e72332c77 v5.33.6 2023-02-03 13:41:45 +00:00
Simon Backx
30b9d02f70 Increased wait time between Mailgun events to 200ms
no issue

Reduce concurrency if the database is going a bit slower, until we have more permanent fix for this.
2023-02-03 14:38:44 +01:00
Djordje Vlaisavljevic
3f7b717c0f Added logic to display target post titles
refs https://github.com/TryGhost/Team/issues/2481
2023-02-03 11:18:03 +00:00
Djordje Vlaisavljevic
5f1bdd6d87 Fixed scroll state issue
refs https://github.com/TryGhost/Team/issues/2434
2023-02-03 11:18:03 +00:00
Djordje Vlaisavljevic
2e379809d0 Updated timestamp design
refs https://github.com/TryGhost/Team/issues/2434
2023-02-03 11:18:03 +00:00
Simon Backx
29e1e93ca1 Increased wait time between Mailgun events to 150ms
no issue

Reduce concurrency if the database is going a bit slower, until we have more permanent fix for this.
2023-02-03 10:39:55 +01:00
Simon Backx
4fad1986a0 🐛 Fixed link to posts on audio cards in emails
refs https://ghost.slack.com/archives/CTH5NDJMS/p1675356984043299

The url property doesn't exists on post, this was copied from the old MEGA flow that serialized the model first.
2023-02-03 10:37:02 +01:00
Naz
b35b1ae98b
Added cleanup note
refs https://github.com/TryGhost/Toolbox/issues/516
refs 4c60cdce9c

- The syntax used to skip non-`/r/` routes is unreadable. Needs a little love during cleanup
2023-02-03 13:11:59 +08:00
Simon Backx
0f08f39a1b 🐛 Fixed link to posts on audio cards in emails
refs https://ghost.slack.com/archives/CTH5NDJMS/p1675356984043299

The url property doesn't exists on post, this was copied from the old MEGA flow that serialized the model first.
2023-02-02 18:19:31 +01:00
Ghost CI
509affb1f7 Merged v5.33.5 into main 2023-02-02 16:33:50 +00:00
Ghost CI
771a6182c7 v5.33.5 2023-02-02 16:33:46 +00:00
Naz
32cd3b9433 Optimized link redirects handling
refs https://github.com/TryGhost/Toolbox/issues/515

- The link redirect handled was querying database on every single frontend request causing a significant amount of unnecessary traffic
- The optimization is returning early if the incoming URL does not start with a common "r/" prefix
2023-02-02 17:27:30 +01:00
Ghost CI
f90d6ca66b Merged v5.33.4 into main 2023-02-02 13:33:14 +00:00
Ghost CI
3f5e34f840 v5.33.4 2023-02-02 13:33:10 +00:00
Kevin Ansfield
1ffb92dd27
Re-build .lint-todo file
no issue

- deleted the file and ran `yarn lint:hbs --update-todo` to clean up fixed errors in the todo list and reset the hide->warn dates
2023-02-02 13:13:59 +00:00
Simon Backx
06de10786f
Added automatic database query retrying to the email service (#16218)
fixes https://github.com/TryGhost/Team/issues/2512

The email sending is a crucial flow that should not be interrupted. If
there are database connection issues, we should try to recover from them
automatically by retrying individual database queries and transactions
for a limited amount of time.

This adds a helper method to retry database operations. It limits all
database queries before sending the actual email to maximum 10 minutes.
Database operations that happen after sending have a higher retry time
because they are more crucial to prevent data loss (e.g. saving that an
email was sent).
2023-02-02 14:12:54 +01:00
Kevin Ansfield
5f5ae79897
Fixed template lint error
refs 7f1e970a0b

- `koenig-card-callout.hbs` was touched without fixing the lint errors or updating to todo list causing the lint todos to become out of sync
- fixed the lint error and updated the todo list
2023-02-02 13:12:03 +00:00
Robin Cussol
1c019a3f35
Added e2e tests for page.unscheduled webhook (#15722)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2023-02-02 13:03:23 +00:00
Tihomir Valkanov
7f1e970a0b
🐛 Fixed emoji causing page jump in safari (#16026)
fixes: https://github.com/TryGhost/Ghost/issues/15295

- By design Safari [does not support focus on a button
element](https://bugs.webkit.org/show_bug.cgi?id=22261). We have to keep
the focus somehow otherwise we cause page jump in Safari when changing
the emoji. Tabindex -1 is a good choice because it does not mess with a
tab sequence if we have one.
2023-02-02 12:59:24 +00:00
Fabien "egg" O'Carroll
669b1927aa Promoted webmentions to private beta
refs https://github.com/TryGhost/Team/issues/2509
2023-02-02 16:42:13 +07:00
Fabien "egg" O'Carroll
e5d31e2900 Added resource data to Mentions Admin API
refs https://github.com/TryGhost/Team/issues/2503

This is in the MentionController atm as it's considered a presentation
concern. We might want to consider moving this into the MentionsAPI in
future so that we can simplify the controller and even remove it
completely in favour of putting the data-mapping in the endpoint definition.
2023-02-02 16:13:49 +07:00
Ronald Langeveld
bef3516f72 Added e2e test for filtering members by tier id
no issue

- Added a test for filtering members by tier id since we now have a new
  way of filtering members in the admin.
2023-02-02 15:10:19 +08:00
Naz
4c60cdce9c
Optimized link redirects handling
refs https://github.com/TryGhost/Toolbox/issues/515

- The link redirect handled was querying database on every single frontend request causing a significant amount of unnecessary traffic
- The optimization is returning early if the incoming URL does not start with a common "r/" prefix
2023-02-02 15:04:52 +08:00
Ronald Langeveld
e02c67dd3a
Mentions ordering (#16215)
closes https://github.com/TryGhost/Team/issues/2498

- can now order mentions via the API via params eg `api/admin/mentions/?order=created_at%20desc`.
2023-02-02 13:25:09 +08:00
Steve Larson
855bcb8613 added tests to cover internal data checks
no ref
-tests didn't cover imported data escape
-tests didn't cover internal data escape
2023-02-01 17:40:29 -06:00
Steve Larson
3186323663 updated listeners in test to be fully inclusive
no ref
-test wasn't previously updated to include all event listeners
2023-02-01 17:20:24 -06:00
Steve Larson
ae92d1425d
updated events that send mentions (#16219)
refs TryGhost/Team#2477
-removed post.edited as it was too inclusive
-changed to post.published, post.published.edited, post.unpublished
-blocked import and internal data from triggering mentions
2023-02-01 16:49:58 -06:00
renovate[bot]
5367fa94cc
Update dependency sanitize-html to v2.9.0 2023-02-01 15:39:45 +00:00
Ronald Langeveld
24681398af
🐛 Fixed members tier filtering (#16212)
refs https://ghost.slack.com/archives/CTH5NDJMS/p1675194453354469 https://secure.helpscout.net/conversation/2127479135/94711

- replaced tiers filtering by `slug` with `id` as we hit an edge case where the NQL filtering is struggling to handle slugs with a single character, eg `a`, `b`. Now filters by the tier's `id` instead.
- added new member, `tier_id` relation to make it possible to filter by id, eg `/?filter=tier_id:<id>`.
2023-02-01 20:33:57 +08:00
Sam Lord
1c0a0e0549 Added email table to data generator
refs: https://github.com/TryGhost/Toolbox/issues/454
2023-02-01 11:58:20 +00:00
Ghost CI
a444d2e703 Merged v5.33.3 into main 2023-02-01 11:07:22 +00:00
Ghost CI
e82ed951f4 v5.33.3 2023-02-01 11:07:19 +00:00
Kevin Ansfield
3c695064bd
🐛 Fixed blank email previews when opening from member activity feeds (#16207)
refs https://github.com/TryGhost/Team/issues/2506

`email` objects no longer contain a `html` field and the fallback logic in the email preview modal was failing resulting in a 404 from trying to fetch an email preview using the email id rather than a post id.

- added quick-fix to the preview modal logic to use `data.post_id || data.id` for generating the preview URL (previous logic never expected to reach the fallback when working with an email record)
2023-02-01 09:52:45 +00:00
Fabien "egg" O'Carroll
72d7580461 Added source_is_ghost flag to outgoing Webmentions
This is a pretty simple way for us to track which webmentions are sent
by Ghost. Although it's easily spoofed, so are other approaches like
using a header (e.g. User-Agent). If we find that this data is being
spoofed we can look at different approach.

Becuase our receiving implementation stores the payload of the
Webmention, we'll be able to know inside Ghost which Mentions
originated from another Ghost installation, which is useful for stats
and gives us the possibility to display that information in the feed.

Longer term we might want to consider storing this data in a separate
column for Mentions, rather than the `payload` column - but that is
outside the scope of this change.
2023-02-01 13:44:55 +07:00
renovate[bot]
c34735f8a2 Update dependency terser to v5.16.2 2023-01-31 19:05:50 +00:00
Simon Backx
4f889586ad
🐛 Fixed email header image width not set (#16210)
refs
https://ghost.slack.com/archives/CTH5NDJMS/p1675182725388589?thread_ts=1675181248.903819&cid=CTH5NDJMS

Missing dependency was causing image widths not set
2023-01-31 18:51:36 +01:00
Ghost CI
356c370905 Merged v5.33.2 into main 2023-01-31 14:37:47 +00:00
Ghost CI
444ca6f5e5 v5.33.2 2023-01-31 14:37:44 +00:00
Naz
25d81c99ca
Fixed {{price}} helper to render empty instead of throwing
refs https://github.com/TryGhost/Toolbox/issues/497
refs fb7532bf5d

- We downgraded the 'GS090-NO-PRICE-DATA-CURRENCY-CONTEXT' rule in gscan to non-fatal, meaning Ghost should not be throwing an error but instead render an empty value for {{price}} helper when price data is empty.
- For example, a legacy syntax like this: '{{price currency=@price.currency}}' should not cause a page render error but return an empty price string.
- The pattern of returning an empty string instead of crashing is used in other helpers like {{img_url}} and and {{url}}
2023-01-31 22:14:43 +08:00
Naz
9390f0953f
Fixed {{price}} helper to render empty instead of throwing
refs https://github.com/TryGhost/Toolbox/issues/497
refs fb7532bf5d

- We downgraded the 'GS090-NO-PRICE-DATA-CURRENCY-CONTEXT' rule in gscan to non-fatal, meaning Ghost should not be throwing an error but instead render an empty value for {{price}} helper when price data is empty.
- For example, a legacy syntax like this: '{{price currency=@price.currency}}' should not cause a page render error but return an empty price string.
- The pattern of returning an empty string instead of crashing is used in other helpers like {{img_url}} and and {{url}}
2023-01-31 21:01:19 +08:00
Simon Backx
62ab5073ac Fixed Playwright test not awaiting Stripe setup
no issue

The test closed the setup modal to fast, which caused the @task to be cancelled and the save would not complete (portalPlans not saved).
2023-01-31 11:26:42 +01:00
renovate[bot]
a85c2ad3d6 Update metascraper to v5.33.7 2023-01-31 10:16:20 +01:00
renovate[bot]
02ca1940f8 Update dependency mysql2 to v3.1.0 2023-01-31 10:16:05 +01:00
Hannah Wolfe
9475c9404c
Fixed minor comment typo in boot
- nothign to see here!
2023-01-31 09:07:08 +00:00
Ronald Langeveld
c77984e6ab
Added mentions permissions (#16200)
closes https://github.com/TryGhost/Team/issues/2420

- Added user roles and permissions for the mentions admin API.
- We only have a `browse` function for our current use case, accessible
by `administrator` and `admin integration`.
2023-01-31 16:40:44 +08:00
renovate[bot]
5ab7654695 Update Test & linting packages 2023-01-31 05:44:54 +00:00
Djordje Vlaisavljevic
f24a3d649f Fixed link to mentions feed
refs https://github.com/TryGhost/Team/issues/2481
2023-01-30 21:53:05 +00:00
Djordje Vlaisavljevic
368d110b1b Fixed feature images not showing in mentions feed
refs https://github.com/TryGhost/Team/issues/2502
2023-01-30 21:49:40 +00:00
Simon Backx
8a153a83ad Improved job manager unit test stability 2023-01-30 17:26:38 +01:00
Ghost CI
b9b51ee309 Merged v5.33.1 into main 2023-01-30 16:18:18 +00:00
Ghost CI
249942892a v5.33.1 2023-01-30 16:18:14 +00:00
Sanne de Vries
365fda2a12 Fixed padding bug on dashboard on mobile 2023-01-30 17:11:10 +01:00
Simon Backx
aea127b17f Fixed Playwright tests for new Stripe connect flow
no issue
2023-01-30 17:09:09 +01:00
Simon Backx
b50e3915ea 🐛 Fixed HTML escaping of feature_image_caption in newsletters
no issue

feature_image_caption was escaped in the new email stability flow, while that should not happen (bold/underline/...).
2023-01-30 16:58:59 +01:00
Fabien 'egg' O'Carroll
8a8bd60d1c 🐛 Fixed newsletter resending showing incorrect recipient data
closes https://github.com/TryGhost/Ghost/issues/16125

We weren't taking into account any existing email segment set on the
post. This is usually not an issue because during the publishing flow
the post.emailSegment and the selectedRecipientFilter are kept in sync,
but it becomes and issue when the email fails to send and is later
retried - we now have an inconsistency between the two values.
2023-01-30 16:58:47 +01:00
Simon Backx
8e66edee2b
🐛 Fixed storing original files for images (#16117)
fixes https://github.com/TryGhost/Team/issues/481

This change fixes an issue when multiple images with the same name are
uploaded in parallel. The current system does not guarantee that the
original filename is stored under NAME+`_o`, because the upload for the
original file and the resized file are happening in parallel.

Solution:
- Wait for the storage of the resized image (= the image without the _o
suffix) before storing the original file.
- When that is stored, use the generated file name of the stored image
to generate the filename with the _o suffix. This way, it will always
match and we don't risk both files to have a different number suffix.
We'll also set the `targetDir` argument when saving the file, to avoid
storing the original file in a different directory (when uploading a
file around midnight both files could be stored in 2023/01 and 2023/02).

Some extra optimisations needed with this fix:
- Previously when uploading image.jpg, while it already exists, it would
store two filenames on e.g., `image-3.jpg` and `image_o-3.jpg`. Note the
weird positioning of `_o`. This probably caused bugs when uploading
files named `image-3.jpg`, which would store the original in
`image-3_o.jpg`, but this original would never be used by the
handle-image-sizes middleware (it would look for `image_o-3.jpg`). This
fix would solve this weird naming issue, and make it more consistent.
But we need to make sure our middlewares (including handle-image-sizes)
will be able to handle both file locations to remain compatible with the
old format. This isn't additional work, because it would fix the old bug
too.
- Prevent uploading files that end with `_o`, e.g. by automatically
stripping that suffix from uploaded files. To prevent collisions.

Advantage(s):
- We keep the original file name, which is better for SEO.
- No changes required to the storage adapters.

Downside(s):
- The storage of both files will nog happen parallel any longer. But I
expect the performance implications to be minimal.
- Changes to the routing: normalize middleware is removed
2023-01-30 16:40:50 +01:00
Djordje Vlaisavljevic
4cdd12214b Updated names
refs https://github.com/TryGhost/Team/issues/2481
2023-01-30 15:16:01 +00:00
Simon Backx
f6149813a9 Renamed try again to view details in newsletter error message
no issue
2023-01-30 15:57:50 +01:00
Simon Backx
762238c7e7 Cleaned DomainEvents import 2023-01-30 15:46:31 +01:00
Simon Backx
60670c21ce Fixed webmentions test awaiting events
no issue

The test was not waiting for dispatched events.
2023-01-30 15:45:16 +01:00
Steve Larson
68ef33800c
stopped private sites from sending mentions (#16160)
refs [TryGhost/Team#2467](https://github.com/TryGhost/Team/issues/2467)
- sites marked private should not send mentions to external sites
2023-01-30 08:13:22 -06:00
Steve Larson
477295a262
added mentions admin browse api e2e tests (#16177)
refs TryGhost/Team#2468
-added simple browse api tests for webmentions
-need to add user access tests and possibly site access (private sites)
2023-01-30 07:59:00 -06:00
Simon Backx
19b9696fe2 🐛 Fixed HTML escaping of feature_image_caption in newsletters
no issue

feature_image_caption was escaped in the new email stability flow, while that should not happen (bold/underline/...).
2023-01-30 14:39:08 +01:00
Simon Backx
af31e5d910 Improved job manager test stability
no issue

Delays can cause unpredictable test failures. This commit removes a couple delays.
2023-01-30 14:19:16 +01:00
Simon Backx
8f8ca481a6
Fixed configUtils and adapter cache issues in E2E tests (#16167)
no issue

There are a couple of issues with resetting the Ghost instance between
E2E test files:

These issues came to the surface because of new tests written in
https://github.com/TryGhost/Ghost/pull/16117

**1. configUtils.restore does not work correctly**
`config.reset()` is a callback based method. On top of that, it doesn't
really work reliably (https://github.com/indexzero/nconf/issues/93)

What kinda happens, is that you first call `config.reset` but
immediately after you correcty reset the config using the `config.set`
calls afterwards. But since `config.reset` is async, that reset will
happen after all those sets, and the end result is that it isn't reset
correctly.

This mainly caused issues in the new updated images tests, which were
updating the config `imageOptimization.contentImageSizes`, which is a
deeply nested config value. Maybe some references to objects are reused
in nconf that cause this issue?

Wrapping `config.reset()` in a promise does fix the issue.

**2. Adapters cache not reset between tests**
At the start of each test, we set `paths:contentPath` to a nice new
temporary directory. But if a previous test already requests a
localStorage adapter, that adapter would have been created and in the
constructor `paths:contentPath` would have been passed. That same
instance will be reused in the next test run. So it won't read the new
config again. To fix this, we need to reset the adapter instances
between E2E tests.

How was this visible? Test uploads were stored in the actual git
repository, and not in a temporary directory. When writing the new image
upload tests, this also resulted in unreliable test runs because some
image names were already taken (from previous test runs).

**3. Old 2E2 test Ghost server not stopped**
Sometimes we still need access to the frontend test server using
`getAgentsWithFrontend`. But that does start a new Ghost server which is
actually listening for HTTP traffic. This could result in a fatal error
in tests because the port is already in use. The issue is that old E2E
tests also start a HTTP server, but they don't stop the server. When you
used the old `startGhost` util, it would check if a server was already
running and stop it first. The new `getAgentsWithFrontend` now also has
the same functionality to fix that issue.
2023-01-30 14:06:20 +01:00
Aileen Nowak
b8fe582378 Added milestone emails alpha flag
refs https://www.notion.so/ghost/Marketing-Milestone-email-campaigns-1d2c9dee3cfa4029863edb16092ad5c4

- Added an alpha feature flag toggle for milestone emails
2023-01-30 13:02:31 +00:00
Djordje Vlaisavljevic
5b3a16213b Fixed the test
refs https://github.com/TryGhost/Team/issues/2491
2023-01-30 12:45:28 +00:00
Djordje Vlaisavljevic
050e739c59 Updated new mention notification email design
refs https://github.com/TryGhost/Team/issues/2491
2023-01-30 12:24:01 +00:00
Fabien 'egg' O'Carroll
73bddef7c5
Used job queue for processing incoming Webmentions
refs https://github.com/TryGhost/Team/issues/2419

We use a job queue to ensure that webmentions can be processed outside of
the request/response cycle, but still finish executing if the processed is closed.

With this we're able to update the e2e tests to await the processing of the mention
rather than sleepign for arbitrary lengths of time, and we've reintroduced the tests
removed previously

- aa14207b69
- 48e9393159
2023-01-30 18:11:30 +07:00
Naz
478eb6ead6
Limited integrations triggering version mismatch emails
refs https://github.com/TryGhost/Toolbox/issues/500
refs https://ghost.notion.site/Data-Types-e5dc54dd0078443f9afd6b2abda443c4

- There current notification logic for incompatible integrations did not take into account the source of the trigger, which might have been causing emails to instance owners that did not ever set up custom integration - so they had nothing to fix.
- The "internal" and "core" integrations are maintained/controlled by the Ghost team, so there should never be a notification going out to the instance owner about possible incompatibility in the code they do not control.
- Along with changed updated the unit test threshold in the packages that were touched to 100%. As that's the standard for all new packages.
2023-01-30 17:57:14 +08:00
Fabien 'egg' O'Carroll
c11e79765e
🐛 Fixed newsletter resending showing incorrect recipient data
closes https://github.com/TryGhost/Ghost/issues/16125

We weren't taking into account any existing email segment set on the
post. This is usually not an issue because during the publishing flow
the post.emailSegment and the selectedRecipientFilter are kept in sync,
but it becomes and issue when the email fails to send and is later
retried - we now have an inconsistency between the two values.
2023-01-30 16:44:57 +07:00
Ghost CI
829c325e51 v5.33.0 2023-01-27 16:00:43 +00:00
John O'Nolan
d4725bfe87
Updated email tips copy 2023-01-26 13:12:03 -04:00
Simon Backx
c798f383f9 🐛 Fixed not reactivating email analytics jobs
no issue

When a site doesn't have any emails on boot, it doesn't schedule the email analytics job. With this change, the new email flow will also restart that job after an email has been created.
2023-01-26 17:35:45 +01:00
Simon Backx
6631071dd3
Improved handling large amounts of email events (#16189)
refs https://github.com/TryGhost/Team/issues/2486

Stop the event fetching loop as soon as we receive events that were
created later then when we started the loop. This ensures that we don't
miss events if we receive a giant batch of events that take a long time
to process.
2023-01-26 16:06:15 +01:00
Aileen Booker
488d9bb135
Added referrals invite notification (#16187)
no issue

- Ghost users that make >= $100 MRR will see a dismissible notification that invites them to the Ghost Referral program
- Only applies to Admin and Owner users and when Stripe is setup and connected in live mode
- By saving a `referralInviteDismissed` property to the users' `accessibility` JSON object we can determine if the notification has been dismissed and won't show it again
- Added new `gh-referral-invite` component
2023-01-26 14:42:11 +00:00
Peter Zimon
63d216c321
Email debug updates (#16185)
no refs.

This commit fixes a couple of UX issues on the email debug screen:

- shows [...] button only for errors actually longer than the available
space to avoid confusion about where there's more error text
- use actual avatars instead of fake red/blue dots to make it consistent
with the rest of the app
- adds click through to member details screen to easily access member
data if needed
- updates text select for provider ID for easier copying
- removes unused "Download full error message" icon
2023-01-26 15:12:26 +01:00
Naz
d3ff4348c8
Fixed test name as content version header is always present
refs 9ba251238a

- The test name should have been updated along with referenced change.
2023-01-26 19:20:15 +08:00
Steve Larson
a3064d95cb
completely blocked external requests in Webmentions tests (#16186)
no issue
-had timeout issues in test suite
-tests were still doing dns lookup via external request
2023-01-25 10:43:57 -06:00
Daniel Lockyer
693db59f73
Merged v5.32.0 into main
v5.32.0
2023-01-25 15:34:26 +01:00
Ghost CI
b1b24a6884 v5.32.0 2023-01-25 14:32:26 +00:00
Simon Backx
2d11c29695
🐛 Fixed email segment generation (#16182)
fixes https://github.com/TryGhost/Team/issues/2484

The flow only send the email to segments that were targeted in the email
content. But if a part of the email is only visible for `status:free`,
that doesn't mean we don't want to send the email to `status:-free`.
This has been corrected in the new email flow.
2023-01-25 14:59:56 +01:00
Simon Backx
846d033e20
Improved email error logging (#16184)
no issue

Logs errors to Sentry and adds error codes.
2023-01-25 14:59:49 +01:00
Simon Backx
5e1e6c9863
Improved email error logging (#16184)
no issue

Logs errors to Sentry and adds error codes.
2023-01-25 14:57:10 +01:00
Simon Backx
f4c55d123c
🐛 Fixed email segment generation (#16182)
fixes https://github.com/TryGhost/Team/issues/2484

The flow only send the email to segments that were targeted in the email
content. But if a part of the email is only visible for `status:free`,
that doesn't mean we don't want to send the email to `status:-free`.
This has been corrected in the new email flow.
2023-01-25 14:56:37 +01:00
Simon Backx
fc990e856a
Reduced email error messages (#16183)
fixes https://github.com/TryGhost/Team/issues/2487
2023-01-25 14:27:35 +01:00
Simon Backx
27619dd8a7
🐛 Reduced concurrency when fetching Mailgun events (#16176)
refs https://github.com/TryGhost/Team/issues/2482

This change adds a small sleep in between dispatching events in the
worker thread that reads the events from Mailgun. That should reduce the
amount of queries we fire parallel to each other and could cause the
connection pool to run out of connections.

It also reduces the amount of concurrent sending to 2 from 10. Also to
make sure the connection pool doesn't run out of connections while
sending emails, and to reduce the chance of new connections falling back
on a (delayed) replicated database.
2023-01-25 14:26:41 +01:00
Ronald Langeveld
dd74f42376
Added mentions email notifications (#16170)
closes https://github.com/TryGhost/Team/issues/2429

- sends email notifications to staff users when their site receives a Webmention.
- currently behind a flag, that can be toggled in the labs settings.
2023-01-25 21:10:29 +08:00
Djordje Vlaisavljevic
c630fae60e Fixed dark mode colors
refs https://github.com/TryGhost/Team/issues/2434
2023-01-25 13:01:39 +00:00
Djordje Vlaisavljevic
03593805d5 Hooked up recent mentions widget
refs https://github.com/TryGhost/Team/issues/2481
2023-01-25 12:47:48 +00:00
Djordje Vlaisavljevic
f0cf740df5 Changed ordering of mentions
refs https://github.com/TryGhost/Team/issues/2434
2023-01-25 12:47:48 +00:00
Simon Backx
17e0c0b856
Reduced email error messages (#16183)
fixes https://github.com/TryGhost/Team/issues/2487
2023-01-25 13:20:50 +01:00
Simon Backx
4b0ca9399d 🐛 Reduced concurrency when fetching Mailgun events (#16176)
refs https://github.com/TryGhost/Team/issues/2482

This change adds a small sleep in between dispatching events in the
worker thread that reads the events from Mailgun. That should reduce the
amount of queries we fire parallel to each other and could cause the
connection pool to run out of connections.

It also reduces the amount of concurrent sending to 2 from 10. Also to
make sure the connection pool doesn't run out of connections while
sending emails, and to reduce the chance of new connections falling back
on a (delayed) replicated database.
2023-01-25 13:07:01 +01:00
Rishabh Garg
503a9ebe51
🐛 Fixed invalid expiry for member tier subscriptions (#16174)
refs https://github.com/TryGhost/Team/issues/2476

When upgrading from a Complimentary subscription with an expiry, to a paid Subscription of the same Tier, the Member was eventually losing access to the Tier when the complimentary subscription expires as the `expiry_at` on the mapping was not removed. This change fixes the code by setting expiry as null when a member upgrades their subscription to paid. This also adds 2 migrations to fix any side-effects on existing sites -

- Removed invalid expiry tier expiry date for paid members
- Restored missing tier mapping for paid members
2023-01-25 12:29:27 +01:00
Daniel Lockyer
1d32931d0a
Revert "Added email snapshots for API versioning tests (#16139)"
- this reverts commit 85051199e3
- the tests here rely on dynamic content (the Ghost version number)
2023-01-25 12:29:27 +01:00
Daniel Lockyer
977c95157a
Removed flaky test
refs https://ghost.slack.com/archives/C02G9E68C/p1674558376712339

- this test is also flaky and causing random failures
2023-01-25 11:41:59 +01:00