Commit Graph

16021 Commits

Author SHA1 Message Date
Simon Backx
fb3cbe5fc8
Added short lived caching to email batch body (#16233)
fixes https://github.com/TryGhost/Team/issues/2522

When sending an email for multiple batches at the same time, we now
reuse the same email body for each batch in the same segment. This
reduces the amount of database queries and makes the sending more
reliable in case of database failures.

The cache is short lived. After sending the email it is automatically
garbage collected.
2023-02-07 11:01:49 +01:00
renovate[bot]
654a991c98 Update dependency mysql2 to v3.1.1 2023-02-07 09:20:11 +01:00
Sam Lord
e0bcb26600 Improved data-generator tooling
no issue

This makes the data-generator more customisable.
2023-02-06 10:42:56 +00:00
renovate[bot]
c01875b85c Update sentry-javascript monorepo to v7.36.0 2023-02-06 10:26:32 +01:00
renovate[bot]
b76466946e Update dependency @types/express to v4.17.17 2023-02-06 10:26:06 +01:00
renovate[bot]
d60fea59e2 Update dependency terser to v5.16.3 2023-02-04 04:47:16 +00:00
Steve Larson
efd01f649b removed suppressionList from labs settings 2023-02-03 14:57:30 -06:00
Steve Larson
5ccb778ac8
enabled always sending mentions (#16227)
refs TryGhost/Team#2507
-removed labs check from sending service
-labs flag controls seeing mentions in admin and receiving mentions
2023-02-03 14:52:20 -06:00
Steve Larson
02953490a5
added mention sending service e2e tests (#16226)
refs TryGhost/Team#2508
-added sending service e2e tests
-should job off this sending service for better tests
-and for ghost to finish processing the job before shutdown
2023-02-03 10:45:50 -06:00
Ghost CI
d37ed200ab Merged v5.33.6 into main 2023-02-03 13:41:48 +00:00
Ghost CI
8e72332c77 v5.33.6 2023-02-03 13:41:45 +00:00
Simon Backx
30b9d02f70 Increased wait time between Mailgun events to 200ms
no issue

Reduce concurrency if the database is going a bit slower, until we have more permanent fix for this.
2023-02-03 14:38:44 +01:00
Djordje Vlaisavljevic
3f7b717c0f Added logic to display target post titles
refs https://github.com/TryGhost/Team/issues/2481
2023-02-03 11:18:03 +00:00
Djordje Vlaisavljevic
5f1bdd6d87 Fixed scroll state issue
refs https://github.com/TryGhost/Team/issues/2434
2023-02-03 11:18:03 +00:00
Djordje Vlaisavljevic
2e379809d0 Updated timestamp design
refs https://github.com/TryGhost/Team/issues/2434
2023-02-03 11:18:03 +00:00
Simon Backx
29e1e93ca1 Increased wait time between Mailgun events to 150ms
no issue

Reduce concurrency if the database is going a bit slower, until we have more permanent fix for this.
2023-02-03 10:39:55 +01:00
Simon Backx
4fad1986a0 🐛 Fixed link to posts on audio cards in emails
refs https://ghost.slack.com/archives/CTH5NDJMS/p1675356984043299

The url property doesn't exists on post, this was copied from the old MEGA flow that serialized the model first.
2023-02-03 10:37:02 +01:00
Naz
b35b1ae98b
Added cleanup note
refs https://github.com/TryGhost/Toolbox/issues/516
refs 4c60cdce9c

- The syntax used to skip non-`/r/` routes is unreadable. Needs a little love during cleanup
2023-02-03 13:11:59 +08:00
Simon Backx
0f08f39a1b 🐛 Fixed link to posts on audio cards in emails
refs https://ghost.slack.com/archives/CTH5NDJMS/p1675356984043299

The url property doesn't exists on post, this was copied from the old MEGA flow that serialized the model first.
2023-02-02 18:19:31 +01:00
Ghost CI
509affb1f7 Merged v5.33.5 into main 2023-02-02 16:33:50 +00:00
Ghost CI
771a6182c7 v5.33.5 2023-02-02 16:33:46 +00:00
Naz
32cd3b9433 Optimized link redirects handling
refs https://github.com/TryGhost/Toolbox/issues/515

- The link redirect handled was querying database on every single frontend request causing a significant amount of unnecessary traffic
- The optimization is returning early if the incoming URL does not start with a common "r/" prefix
2023-02-02 17:27:30 +01:00
Ghost CI
f90d6ca66b Merged v5.33.4 into main 2023-02-02 13:33:14 +00:00
Ghost CI
3f5e34f840 v5.33.4 2023-02-02 13:33:10 +00:00
Kevin Ansfield
1ffb92dd27
Re-build .lint-todo file
no issue

- deleted the file and ran `yarn lint:hbs --update-todo` to clean up fixed errors in the todo list and reset the hide->warn dates
2023-02-02 13:13:59 +00:00
Simon Backx
06de10786f
Added automatic database query retrying to the email service (#16218)
fixes https://github.com/TryGhost/Team/issues/2512

The email sending is a crucial flow that should not be interrupted. If
there are database connection issues, we should try to recover from them
automatically by retrying individual database queries and transactions
for a limited amount of time.

This adds a helper method to retry database operations. It limits all
database queries before sending the actual email to maximum 10 minutes.
Database operations that happen after sending have a higher retry time
because they are more crucial to prevent data loss (e.g. saving that an
email was sent).
2023-02-02 14:12:54 +01:00
Kevin Ansfield
5f5ae79897
Fixed template lint error
refs 7f1e970a0b

- `koenig-card-callout.hbs` was touched without fixing the lint errors or updating to todo list causing the lint todos to become out of sync
- fixed the lint error and updated the todo list
2023-02-02 13:12:03 +00:00
Robin Cussol
1c019a3f35
Added e2e tests for page.unscheduled webhook (#15722)
refs: https://github.com/TryGhost/Ghost/issues/15537

- snapshot test created to add confidence to webhook stability and increase overall test coverage.
2023-02-02 13:03:23 +00:00
Tihomir Valkanov
7f1e970a0b
🐛 Fixed emoji causing page jump in safari (#16026)
fixes: https://github.com/TryGhost/Ghost/issues/15295

- By design Safari [does not support focus on a button
element](https://bugs.webkit.org/show_bug.cgi?id=22261). We have to keep
the focus somehow otherwise we cause page jump in Safari when changing
the emoji. Tabindex -1 is a good choice because it does not mess with a
tab sequence if we have one.
2023-02-02 12:59:24 +00:00
Fabien "egg" O'Carroll
669b1927aa Promoted webmentions to private beta
refs https://github.com/TryGhost/Team/issues/2509
2023-02-02 16:42:13 +07:00
Fabien "egg" O'Carroll
e5d31e2900 Added resource data to Mentions Admin API
refs https://github.com/TryGhost/Team/issues/2503

This is in the MentionController atm as it's considered a presentation
concern. We might want to consider moving this into the MentionsAPI in
future so that we can simplify the controller and even remove it
completely in favour of putting the data-mapping in the endpoint definition.
2023-02-02 16:13:49 +07:00
Ronald Langeveld
bef3516f72 Added e2e test for filtering members by tier id
no issue

- Added a test for filtering members by tier id since we now have a new
  way of filtering members in the admin.
2023-02-02 15:10:19 +08:00
Naz
4c60cdce9c
Optimized link redirects handling
refs https://github.com/TryGhost/Toolbox/issues/515

- The link redirect handled was querying database on every single frontend request causing a significant amount of unnecessary traffic
- The optimization is returning early if the incoming URL does not start with a common "r/" prefix
2023-02-02 15:04:52 +08:00
Ronald Langeveld
e02c67dd3a
Mentions ordering (#16215)
closes https://github.com/TryGhost/Team/issues/2498

- can now order mentions via the API via params eg `api/admin/mentions/?order=created_at%20desc`.
2023-02-02 13:25:09 +08:00
Steve Larson
855bcb8613 added tests to cover internal data checks
no ref
-tests didn't cover imported data escape
-tests didn't cover internal data escape
2023-02-01 17:40:29 -06:00
Steve Larson
3186323663 updated listeners in test to be fully inclusive
no ref
-test wasn't previously updated to include all event listeners
2023-02-01 17:20:24 -06:00
Steve Larson
ae92d1425d
updated events that send mentions (#16219)
refs TryGhost/Team#2477
-removed post.edited as it was too inclusive
-changed to post.published, post.published.edited, post.unpublished
-blocked import and internal data from triggering mentions
2023-02-01 16:49:58 -06:00
renovate[bot]
5367fa94cc
Update dependency sanitize-html to v2.9.0 2023-02-01 15:39:45 +00:00
Ronald Langeveld
24681398af
🐛 Fixed members tier filtering (#16212)
refs https://ghost.slack.com/archives/CTH5NDJMS/p1675194453354469 https://secure.helpscout.net/conversation/2127479135/94711

- replaced tiers filtering by `slug` with `id` as we hit an edge case where the NQL filtering is struggling to handle slugs with a single character, eg `a`, `b`. Now filters by the tier's `id` instead.
- added new member, `tier_id` relation to make it possible to filter by id, eg `/?filter=tier_id:<id>`.
2023-02-01 20:33:57 +08:00
Sam Lord
1c0a0e0549 Added email table to data generator
refs: https://github.com/TryGhost/Toolbox/issues/454
2023-02-01 11:58:20 +00:00
Ghost CI
a444d2e703 Merged v5.33.3 into main 2023-02-01 11:07:22 +00:00
Ghost CI
e82ed951f4 v5.33.3 2023-02-01 11:07:19 +00:00
Kevin Ansfield
3c695064bd
🐛 Fixed blank email previews when opening from member activity feeds (#16207)
refs https://github.com/TryGhost/Team/issues/2506

`email` objects no longer contain a `html` field and the fallback logic in the email preview modal was failing resulting in a 404 from trying to fetch an email preview using the email id rather than a post id.

- added quick-fix to the preview modal logic to use `data.post_id || data.id` for generating the preview URL (previous logic never expected to reach the fallback when working with an email record)
2023-02-01 09:52:45 +00:00
Fabien "egg" O'Carroll
72d7580461 Added source_is_ghost flag to outgoing Webmentions
This is a pretty simple way for us to track which webmentions are sent
by Ghost. Although it's easily spoofed, so are other approaches like
using a header (e.g. User-Agent). If we find that this data is being
spoofed we can look at different approach.

Becuase our receiving implementation stores the payload of the
Webmention, we'll be able to know inside Ghost which Mentions
originated from another Ghost installation, which is useful for stats
and gives us the possibility to display that information in the feed.

Longer term we might want to consider storing this data in a separate
column for Mentions, rather than the `payload` column - but that is
outside the scope of this change.
2023-02-01 13:44:55 +07:00
renovate[bot]
c34735f8a2 Update dependency terser to v5.16.2 2023-01-31 19:05:50 +00:00
Simon Backx
4f889586ad
🐛 Fixed email header image width not set (#16210)
refs
https://ghost.slack.com/archives/CTH5NDJMS/p1675182725388589?thread_ts=1675181248.903819&cid=CTH5NDJMS

Missing dependency was causing image widths not set
2023-01-31 18:51:36 +01:00
Ghost CI
356c370905 Merged v5.33.2 into main 2023-01-31 14:37:47 +00:00
Ghost CI
444ca6f5e5 v5.33.2 2023-01-31 14:37:44 +00:00
Naz
25d81c99ca
Fixed {{price}} helper to render empty instead of throwing
refs https://github.com/TryGhost/Toolbox/issues/497
refs fb7532bf5d

- We downgraded the 'GS090-NO-PRICE-DATA-CURRENCY-CONTEXT' rule in gscan to non-fatal, meaning Ghost should not be throwing an error but instead render an empty value for {{price}} helper when price data is empty.
- For example, a legacy syntax like this: '{{price currency=@price.currency}}' should not cause a page render error but return an empty price string.
- The pattern of returning an empty string instead of crashing is used in other helpers like {{img_url}} and and {{url}}
2023-01-31 22:14:43 +08:00
Naz
9390f0953f
Fixed {{price}} helper to render empty instead of throwing
refs https://github.com/TryGhost/Toolbox/issues/497
refs fb7532bf5d

- We downgraded the 'GS090-NO-PRICE-DATA-CURRENCY-CONTEXT' rule in gscan to non-fatal, meaning Ghost should not be throwing an error but instead render an empty value for {{price}} helper when price data is empty.
- For example, a legacy syntax like this: '{{price currency=@price.currency}}' should not cause a page render error but return an empty price string.
- The pattern of returning an empty string instead of crashing is used in other helpers like {{img_url}} and and {{url}}
2023-01-31 21:01:19 +08:00
Simon Backx
62ab5073ac Fixed Playwright test not awaiting Stripe setup
no issue

The test closed the setup modal to fast, which caused the @task to be cancelled and the save would not complete (portalPlans not saved).
2023-01-31 11:26:42 +01:00
renovate[bot]
a85c2ad3d6 Update metascraper to v5.33.7 2023-01-31 10:16:20 +01:00
renovate[bot]
02ca1940f8 Update dependency mysql2 to v3.1.0 2023-01-31 10:16:05 +01:00
Hannah Wolfe
9475c9404c
Fixed minor comment typo in boot
- nothign to see here!
2023-01-31 09:07:08 +00:00
Ronald Langeveld
c77984e6ab
Added mentions permissions (#16200)
closes https://github.com/TryGhost/Team/issues/2420

- Added user roles and permissions for the mentions admin API.
- We only have a `browse` function for our current use case, accessible
by `administrator` and `admin integration`.
2023-01-31 16:40:44 +08:00
renovate[bot]
5ab7654695 Update Test & linting packages 2023-01-31 05:44:54 +00:00
Djordje Vlaisavljevic
f24a3d649f Fixed link to mentions feed
refs https://github.com/TryGhost/Team/issues/2481
2023-01-30 21:53:05 +00:00
Djordje Vlaisavljevic
368d110b1b Fixed feature images not showing in mentions feed
refs https://github.com/TryGhost/Team/issues/2502
2023-01-30 21:49:40 +00:00
Simon Backx
8a153a83ad Improved job manager unit test stability 2023-01-30 17:26:38 +01:00
Ghost CI
b9b51ee309 Merged v5.33.1 into main 2023-01-30 16:18:18 +00:00
Ghost CI
249942892a v5.33.1 2023-01-30 16:18:14 +00:00
Sanne de Vries
365fda2a12 Fixed padding bug on dashboard on mobile 2023-01-30 17:11:10 +01:00
Simon Backx
aea127b17f Fixed Playwright tests for new Stripe connect flow
no issue
2023-01-30 17:09:09 +01:00
Simon Backx
b50e3915ea 🐛 Fixed HTML escaping of feature_image_caption in newsletters
no issue

feature_image_caption was escaped in the new email stability flow, while that should not happen (bold/underline/...).
2023-01-30 16:58:59 +01:00
Fabien 'egg' O'Carroll
8a8bd60d1c 🐛 Fixed newsletter resending showing incorrect recipient data
closes https://github.com/TryGhost/Ghost/issues/16125

We weren't taking into account any existing email segment set on the
post. This is usually not an issue because during the publishing flow
the post.emailSegment and the selectedRecipientFilter are kept in sync,
but it becomes and issue when the email fails to send and is later
retried - we now have an inconsistency between the two values.
2023-01-30 16:58:47 +01:00
Simon Backx
8e66edee2b
🐛 Fixed storing original files for images (#16117)
fixes https://github.com/TryGhost/Team/issues/481

This change fixes an issue when multiple images with the same name are
uploaded in parallel. The current system does not guarantee that the
original filename is stored under NAME+`_o`, because the upload for the
original file and the resized file are happening in parallel.

Solution:
- Wait for the storage of the resized image (= the image without the _o
suffix) before storing the original file.
- When that is stored, use the generated file name of the stored image
to generate the filename with the _o suffix. This way, it will always
match and we don't risk both files to have a different number suffix.
We'll also set the `targetDir` argument when saving the file, to avoid
storing the original file in a different directory (when uploading a
file around midnight both files could be stored in 2023/01 and 2023/02).

Some extra optimisations needed with this fix:
- Previously when uploading image.jpg, while it already exists, it would
store two filenames on e.g., `image-3.jpg` and `image_o-3.jpg`. Note the
weird positioning of `_o`. This probably caused bugs when uploading
files named `image-3.jpg`, which would store the original in
`image-3_o.jpg`, but this original would never be used by the
handle-image-sizes middleware (it would look for `image_o-3.jpg`). This
fix would solve this weird naming issue, and make it more consistent.
But we need to make sure our middlewares (including handle-image-sizes)
will be able to handle both file locations to remain compatible with the
old format. This isn't additional work, because it would fix the old bug
too.
- Prevent uploading files that end with `_o`, e.g. by automatically
stripping that suffix from uploaded files. To prevent collisions.

Advantage(s):
- We keep the original file name, which is better for SEO.
- No changes required to the storage adapters.

Downside(s):
- The storage of both files will nog happen parallel any longer. But I
expect the performance implications to be minimal.
- Changes to the routing: normalize middleware is removed
2023-01-30 16:40:50 +01:00
Djordje Vlaisavljevic
4cdd12214b Updated names
refs https://github.com/TryGhost/Team/issues/2481
2023-01-30 15:16:01 +00:00
Simon Backx
f6149813a9 Renamed try again to view details in newsletter error message
no issue
2023-01-30 15:57:50 +01:00
Simon Backx
762238c7e7 Cleaned DomainEvents import 2023-01-30 15:46:31 +01:00
Simon Backx
60670c21ce Fixed webmentions test awaiting events
no issue

The test was not waiting for dispatched events.
2023-01-30 15:45:16 +01:00
Steve Larson
68ef33800c
stopped private sites from sending mentions (#16160)
refs [TryGhost/Team#2467](https://github.com/TryGhost/Team/issues/2467)
- sites marked private should not send mentions to external sites
2023-01-30 08:13:22 -06:00
Steve Larson
477295a262
added mentions admin browse api e2e tests (#16177)
refs TryGhost/Team#2468
-added simple browse api tests for webmentions
-need to add user access tests and possibly site access (private sites)
2023-01-30 07:59:00 -06:00
Simon Backx
19b9696fe2 🐛 Fixed HTML escaping of feature_image_caption in newsletters
no issue

feature_image_caption was escaped in the new email stability flow, while that should not happen (bold/underline/...).
2023-01-30 14:39:08 +01:00
Simon Backx
af31e5d910 Improved job manager test stability
no issue

Delays can cause unpredictable test failures. This commit removes a couple delays.
2023-01-30 14:19:16 +01:00
Simon Backx
8f8ca481a6
Fixed configUtils and adapter cache issues in E2E tests (#16167)
no issue

There are a couple of issues with resetting the Ghost instance between
E2E test files:

These issues came to the surface because of new tests written in
https://github.com/TryGhost/Ghost/pull/16117

**1. configUtils.restore does not work correctly**
`config.reset()` is a callback based method. On top of that, it doesn't
really work reliably (https://github.com/indexzero/nconf/issues/93)

What kinda happens, is that you first call `config.reset` but
immediately after you correcty reset the config using the `config.set`
calls afterwards. But since `config.reset` is async, that reset will
happen after all those sets, and the end result is that it isn't reset
correctly.

This mainly caused issues in the new updated images tests, which were
updating the config `imageOptimization.contentImageSizes`, which is a
deeply nested config value. Maybe some references to objects are reused
in nconf that cause this issue?

Wrapping `config.reset()` in a promise does fix the issue.

**2. Adapters cache not reset between tests**
At the start of each test, we set `paths:contentPath` to a nice new
temporary directory. But if a previous test already requests a
localStorage adapter, that adapter would have been created and in the
constructor `paths:contentPath` would have been passed. That same
instance will be reused in the next test run. So it won't read the new
config again. To fix this, we need to reset the adapter instances
between E2E tests.

How was this visible? Test uploads were stored in the actual git
repository, and not in a temporary directory. When writing the new image
upload tests, this also resulted in unreliable test runs because some
image names were already taken (from previous test runs).

**3. Old 2E2 test Ghost server not stopped**
Sometimes we still need access to the frontend test server using
`getAgentsWithFrontend`. But that does start a new Ghost server which is
actually listening for HTTP traffic. This could result in a fatal error
in tests because the port is already in use. The issue is that old E2E
tests also start a HTTP server, but they don't stop the server. When you
used the old `startGhost` util, it would check if a server was already
running and stop it first. The new `getAgentsWithFrontend` now also has
the same functionality to fix that issue.
2023-01-30 14:06:20 +01:00
Aileen Nowak
b8fe582378 Added milestone emails alpha flag
refs https://www.notion.so/ghost/Marketing-Milestone-email-campaigns-1d2c9dee3cfa4029863edb16092ad5c4

- Added an alpha feature flag toggle for milestone emails
2023-01-30 13:02:31 +00:00
Djordje Vlaisavljevic
5b3a16213b Fixed the test
refs https://github.com/TryGhost/Team/issues/2491
2023-01-30 12:45:28 +00:00
Djordje Vlaisavljevic
050e739c59 Updated new mention notification email design
refs https://github.com/TryGhost/Team/issues/2491
2023-01-30 12:24:01 +00:00
Fabien 'egg' O'Carroll
73bddef7c5
Used job queue for processing incoming Webmentions
refs https://github.com/TryGhost/Team/issues/2419

We use a job queue to ensure that webmentions can be processed outside of
the request/response cycle, but still finish executing if the processed is closed.

With this we're able to update the e2e tests to await the processing of the mention
rather than sleepign for arbitrary lengths of time, and we've reintroduced the tests
removed previously

- aa14207b69
- 48e9393159
2023-01-30 18:11:30 +07:00
Naz
478eb6ead6
Limited integrations triggering version mismatch emails
refs https://github.com/TryGhost/Toolbox/issues/500
refs https://ghost.notion.site/Data-Types-e5dc54dd0078443f9afd6b2abda443c4

- There current notification logic for incompatible integrations did not take into account the source of the trigger, which might have been causing emails to instance owners that did not ever set up custom integration - so they had nothing to fix.
- The "internal" and "core" integrations are maintained/controlled by the Ghost team, so there should never be a notification going out to the instance owner about possible incompatibility in the code they do not control.
- Along with changed updated the unit test threshold in the packages that were touched to 100%. As that's the standard for all new packages.
2023-01-30 17:57:14 +08:00
Fabien 'egg' O'Carroll
c11e79765e
🐛 Fixed newsletter resending showing incorrect recipient data
closes https://github.com/TryGhost/Ghost/issues/16125

We weren't taking into account any existing email segment set on the
post. This is usually not an issue because during the publishing flow
the post.emailSegment and the selectedRecipientFilter are kept in sync,
but it becomes and issue when the email fails to send and is later
retried - we now have an inconsistency between the two values.
2023-01-30 16:44:57 +07:00
Ghost CI
829c325e51 v5.33.0 2023-01-27 16:00:43 +00:00
John O'Nolan
d4725bfe87
Updated email tips copy 2023-01-26 13:12:03 -04:00
Simon Backx
c798f383f9 🐛 Fixed not reactivating email analytics jobs
no issue

When a site doesn't have any emails on boot, it doesn't schedule the email analytics job. With this change, the new email flow will also restart that job after an email has been created.
2023-01-26 17:35:45 +01:00
Simon Backx
6631071dd3
Improved handling large amounts of email events (#16189)
refs https://github.com/TryGhost/Team/issues/2486

Stop the event fetching loop as soon as we receive events that were
created later then when we started the loop. This ensures that we don't
miss events if we receive a giant batch of events that take a long time
to process.
2023-01-26 16:06:15 +01:00
Aileen Booker
488d9bb135
Added referrals invite notification (#16187)
no issue

- Ghost users that make >= $100 MRR will see a dismissible notification that invites them to the Ghost Referral program
- Only applies to Admin and Owner users and when Stripe is setup and connected in live mode
- By saving a `referralInviteDismissed` property to the users' `accessibility` JSON object we can determine if the notification has been dismissed and won't show it again
- Added new `gh-referral-invite` component
2023-01-26 14:42:11 +00:00
Peter Zimon
63d216c321
Email debug updates (#16185)
no refs.

This commit fixes a couple of UX issues on the email debug screen:

- shows [...] button only for errors actually longer than the available
space to avoid confusion about where there's more error text
- use actual avatars instead of fake red/blue dots to make it consistent
with the rest of the app
- adds click through to member details screen to easily access member
data if needed
- updates text select for provider ID for easier copying
- removes unused "Download full error message" icon
2023-01-26 15:12:26 +01:00
Naz
d3ff4348c8
Fixed test name as content version header is always present
refs 9ba251238a

- The test name should have been updated along with referenced change.
2023-01-26 19:20:15 +08:00
Steve Larson
a3064d95cb
completely blocked external requests in Webmentions tests (#16186)
no issue
-had timeout issues in test suite
-tests were still doing dns lookup via external request
2023-01-25 10:43:57 -06:00
Daniel Lockyer
693db59f73
Merged v5.32.0 into main
v5.32.0
2023-01-25 15:34:26 +01:00
Ghost CI
b1b24a6884 v5.32.0 2023-01-25 14:32:26 +00:00
Simon Backx
2d11c29695
🐛 Fixed email segment generation (#16182)
fixes https://github.com/TryGhost/Team/issues/2484

The flow only send the email to segments that were targeted in the email
content. But if a part of the email is only visible for `status:free`,
that doesn't mean we don't want to send the email to `status:-free`.
This has been corrected in the new email flow.
2023-01-25 14:59:56 +01:00
Simon Backx
846d033e20
Improved email error logging (#16184)
no issue

Logs errors to Sentry and adds error codes.
2023-01-25 14:59:49 +01:00
Simon Backx
5e1e6c9863
Improved email error logging (#16184)
no issue

Logs errors to Sentry and adds error codes.
2023-01-25 14:57:10 +01:00
Simon Backx
f4c55d123c
🐛 Fixed email segment generation (#16182)
fixes https://github.com/TryGhost/Team/issues/2484

The flow only send the email to segments that were targeted in the email
content. But if a part of the email is only visible for `status:free`,
that doesn't mean we don't want to send the email to `status:-free`.
This has been corrected in the new email flow.
2023-01-25 14:56:37 +01:00
Simon Backx
fc990e856a
Reduced email error messages (#16183)
fixes https://github.com/TryGhost/Team/issues/2487
2023-01-25 14:27:35 +01:00
Simon Backx
27619dd8a7
🐛 Reduced concurrency when fetching Mailgun events (#16176)
refs https://github.com/TryGhost/Team/issues/2482

This change adds a small sleep in between dispatching events in the
worker thread that reads the events from Mailgun. That should reduce the
amount of queries we fire parallel to each other and could cause the
connection pool to run out of connections.

It also reduces the amount of concurrent sending to 2 from 10. Also to
make sure the connection pool doesn't run out of connections while
sending emails, and to reduce the chance of new connections falling back
on a (delayed) replicated database.
2023-01-25 14:26:41 +01:00
Ronald Langeveld
dd74f42376
Added mentions email notifications (#16170)
closes https://github.com/TryGhost/Team/issues/2429

- sends email notifications to staff users when their site receives a Webmention.
- currently behind a flag, that can be toggled in the labs settings.
2023-01-25 21:10:29 +08:00
Djordje Vlaisavljevic
c630fae60e Fixed dark mode colors
refs https://github.com/TryGhost/Team/issues/2434
2023-01-25 13:01:39 +00:00
Djordje Vlaisavljevic
03593805d5 Hooked up recent mentions widget
refs https://github.com/TryGhost/Team/issues/2481
2023-01-25 12:47:48 +00:00
Djordje Vlaisavljevic
f0cf740df5 Changed ordering of mentions
refs https://github.com/TryGhost/Team/issues/2434
2023-01-25 12:47:48 +00:00
Simon Backx
17e0c0b856
Reduced email error messages (#16183)
fixes https://github.com/TryGhost/Team/issues/2487
2023-01-25 13:20:50 +01:00
Simon Backx
4b0ca9399d 🐛 Reduced concurrency when fetching Mailgun events (#16176)
refs https://github.com/TryGhost/Team/issues/2482

This change adds a small sleep in between dispatching events in the
worker thread that reads the events from Mailgun. That should reduce the
amount of queries we fire parallel to each other and could cause the
connection pool to run out of connections.

It also reduces the amount of concurrent sending to 2 from 10. Also to
make sure the connection pool doesn't run out of connections while
sending emails, and to reduce the chance of new connections falling back
on a (delayed) replicated database.
2023-01-25 13:07:01 +01:00
Rishabh Garg
503a9ebe51
🐛 Fixed invalid expiry for member tier subscriptions (#16174)
refs https://github.com/TryGhost/Team/issues/2476

When upgrading from a Complimentary subscription with an expiry, to a paid Subscription of the same Tier, the Member was eventually losing access to the Tier when the complimentary subscription expires as the `expiry_at` on the mapping was not removed. This change fixes the code by setting expiry as null when a member upgrades their subscription to paid. This also adds 2 migrations to fix any side-effects on existing sites -

- Removed invalid expiry tier expiry date for paid members
- Restored missing tier mapping for paid members
2023-01-25 12:29:27 +01:00
Daniel Lockyer
1d32931d0a
Revert "Added email snapshots for API versioning tests (#16139)"
- this reverts commit 85051199e3
- the tests here rely on dynamic content (the Ghost version number)
2023-01-25 12:29:27 +01:00
Daniel Lockyer
977c95157a
Removed flaky test
refs https://ghost.slack.com/archives/C02G9E68C/p1674558376712339

- this test is also flaky and causing random failures
2023-01-25 11:41:59 +01:00
renovate[bot]
8123f883bd Update dependency @types/express to v4.17.16 2023-01-25 11:28:02 +01:00
Rishabh Garg
a81620e37d
🐛 Fixed invalid expiry for member tier subscriptions (#16174)
refs https://github.com/TryGhost/Team/issues/2476

When upgrading from a Complimentary subscription with an expiry, to a paid Subscription of the same Tier, the Member was eventually losing access to the Tier when the complimentary subscription expires as the `expiry_at` on the mapping was not removed. This change fixes the code by setting expiry as null when a member upgrades their subscription to paid. This also adds 2 migrations to fix any side-effects on existing sites -

- Removed invalid expiry tier expiry date for paid members
- Restored missing tier mapping for paid members
2023-01-25 13:59:43 +05:30
renovate[bot]
270d2b26e5 Update sentry-javascript monorepo to v7.33.0 2023-01-25 08:40:51 +01:00
renovate[bot]
cbdd866dc7 Update dependency @playwright/test to v1.30.0 2023-01-25 08:40:36 +01:00
Aileen Booker
e243b583f6
Added Stripe guide to Stripe connect modal (#16178)
closes https://www.notion.so/ghost/Promote-Stripe-Guide-in-admin-a5740ce2afd7413bae7e1245cfc4d9ce

- Added a guide to Ghost Resources after successful connection with Stripe
- Updated modal layout
- Fixed bug where template condition was listening to wrong task descriptor and rendering a flash in between running and finished state
2023-01-24 18:55:16 +00:00
Djordje Vlaisavljevic
3dc85b0397 Added static Mentions widget behind the flag
refs https://github.com/TryGhost/Team/issues/2481
2023-01-24 15:55:37 +00:00
Cathy Sarisky
9db27cfd50
remove hardcoded values from Portal (#15661)
Portal currently has a Globals.js file that spells out all the colors in use in Portal, which should make it easy to customize the portal colors to match the chosen theme. There are a bunch of hardcoded values and this PR deals with those. The final outcome of these changes is absolutely invisible.

Co-authored-by: Peter Zimon <zimo@ghost.org>
2023-01-24 13:23:11 +01:00
Fabien "egg" O'Carroll
48e9393159 Removed flaky test
This test is failing because the `sleep` isn't long enough. Removing this test
until we've refactored to use the jobs service, at which point we can remove the
sleep and wait for the job to be complete.
2023-01-24 18:50:04 +07:00
Fabien "egg" O'Carroll
169eb6046e Fixed RoutingService checks for resource existence
We were incorrectly handling a "no resource found" return value from the
ResourceService, instead of an object with `null` values, we were expecting a
`null` value - so we were considering all URL's to be pointing toward a
resource.
2023-01-24 18:00:52 +07:00
Rishabh Garg
88979c852b
Updated email sending to remove invalid recipient emails (#16171)
closes https://github.com/TryGhost/Team/issues/2388

We have seen examples of sites with member emails that have invalid characters that can cause an entire email send to fail, or just cause a failure to those addresses. The issue that allowed members with invalid email address to be saved was patched earlier, but its possible there are still sites that contain some of those invalid email addresses.

This change updates new sending service to filter out the recipients with invalid email address before passing them to mail provider, so these rogue addresses don't affect the whole batch in anyway. We also trim the recipient emails to clear out any spaces first, which is the most likely culprit.

- uses new email validator that detects invalid email addresses with special chars
2023-01-24 16:13:10 +05:30
Fabien "egg" O'Carroll
9df131ee5a Checked for existence of page via a network request
refs https://github.com/TryGhost/Team/issues/2466

Now that we're checking for resources at the URL and rejecting if
there isn't one found, we want to make sure that we can handle pages
which are not a resource.

The idea here is to make a HEAD request to determine whether or not
the page exists. We don't need the full response so HEAD saves us some
bandwidth and we allow both 2xx and 3xx status codes because Ghost has
redirects to add missing trailing slashes, which may not be present in
the URL we're passed.
2023-01-24 16:17:40 +07:00
Fabien "egg" O'Carroll
919d0a80c0 Checked for existence of a resource to determine page existence
refs https://github.com/TryGhost/Team/issues/2466

The existing implementation was a very basic check to get us to the
first milestone. By checking if the page points to a resource we can
know for sure the URL exists on the site.
2023-01-24 16:17:40 +07:00
Naz
a22799e133
Added last_seen to authors routing config exclude
refs https://github.com/TryGhost/Toolbox/issues/503

- The "last_seen" property is not used in routing calculations. Without it the routing service was triggering an expensive process on each user login.
2023-01-24 12:29:09 +08:00
Naz
62e3caba2c
Fixed route update error on attach/detach events
refs https://github.com/TryGhost/Toolbox/issues/503

- There was an error thrown due to empty "model._changed" field
- When attached or detached events (e.g. tag.attached) are sent through, their models do not contain any _changed properties. This was taken into account when checking for route related resource changes
2023-01-24 12:29:09 +08:00
Djordje Vlaisavljevic
d394ce6638 Added Mentions page empty state
refs https://github.com/TryGhost/Team/issues/2479
2023-01-23 21:42:55 +00:00
Djordje Vlaisavljevic
bbbdd26f3f Added conditionals for displaying mention info
refs https://github.com/TryGhost/Team/issues/2434
2023-01-23 21:42:55 +00:00
Sam Lord
24bdcf5e9d Added WebMentions to data generator
closes: https://github.com/TryGhost/Toolbox/issues/509
closes: https://github.com/TryGhost/Toolbox/issues/507
2023-01-23 17:39:00 +00:00
renovate[bot]
90ad60dd85 Update sentry-javascript monorepo to v7.32.0 2023-01-23 16:46:58 +01:00
renovate[bot]
d4cd6be9f7
Update metascraper to v5.33.5 2023-01-23 12:04:33 +00:00
renovate[bot]
929d32dddf Update dependency knex to v2.4.2 2023-01-23 13:00:53 +01:00
Fabien "egg" O'Carroll
182e0b831d Implemented and tested ResourceService as separate class
refs https://github.com/TryGhost/Team/issues/2465

This code is still in the Ghost package for now as it's essentially glue code.
2023-01-23 18:34:52 +07:00
Fabien "egg" O'Carroll
f746f223cd Implemented and tested RoutingService as separate class
refs https://github.com/TryGhost/Team/issues/2466

This code is still in the Ghost package for now as it's essentially glue code.
2023-01-23 18:34:52 +07:00
Sagar Gupta
bba4743739
Removed bluebird dependency from unit tests in core (#16096)
refs https://github.com/TryGhost/Ghost/issues/14882

- Replaced `new Promise.reject()` / `new Promise.resolve()` with the
static methods `Promise.reject` / `Promise/resolve` from native promises
- Replaced `Promise.delay()` with a promisified `setTimeout()`
2023-01-23 11:23:48 +00:00
Kevin Ansfield
42b30ac613 Fixed email preview input dropdown not closing when clicking on iframe
no issue

- clicks on the iframe never bubble out of the iframe so weren't captured by the dropdown-closing event listener
- added an event listener directly on the iframe's body element when we render the iframe's content that manually calls out to our generic dropdown closing method
2023-01-23 09:47:46 +00:00
Sanne de Vries
4b61f23bd0 Fixed minor email preview visual bugs
No ref
2023-01-23 10:46:30 +01:00
Naz
2a01dd0481
Added test coverage for sitemap base generator
refs https://github.com/TryGhost/Toolbox/issues/503

- The "updateURL" method was not covered during implementation. Covering the gap with basic tests for the "updateURL" method
2023-01-23 16:33:41 +08:00
Naz
4aacd50fee
Added coverage for URLResourceUpdatedEvent
refs https://github.com/TryGhost/Toolbox/issues/503

- The listener was not covered during quick and dirty implementation. While in the area did some cleanup to the sitemap manager test
- One of the problems I've stumbled upon when adding a test is having multiple instances of SiteManager in the test, which in turn created multiple "subscribe" events and repeat handle executions. Fixed it by having just one site manager instance (a singleton) as that's the pattern that used in main codebase
2023-01-23 16:33:41 +08:00
Naz
714a6f6900
Updated sitemaps to react to a partial resource update
refs https://github.com/TryGhost/Toolbox/issues/503

- The Dynamic URL service no longer generates "url.added" event when only a partial resource update happened - only non-url forming properties were modified. The sitemaps service still needs to know when to update the lastmod ("Last Modified") field associated with specific URL.
2023-01-23 16:33:41 +08:00
Naz
39ef1d20db
Excluded 'tiers' from fields that affect url generation for posts
refs https://github.com/TryGhost/Toolbox/issues/503

- Tier's are sometimes dynamically generated and are present in the "_changed" properties, causing full URL regeneration. They have no effect on post's URL, so should not trigger URL regeneration.
2023-01-23 16:33:41 +08:00
Naz
fbf9442e39
Added extra JSDocs
no issue
2023-01-23 16:33:41 +08:00
Naz
ccb485110b
Short-circuited resource URL regeneration when it's not necessary
refs https://github.com/TryGhost/Toolbox/issues/503

- Full URL regeneration process was happening even when only unrelated to URL generation fields were updated (e.g. 'plaintext' change in post does not affect the URL of the post). Stopping the  "resource updated" event processing early circumvents full url regeneration inside of DynamicRouting, which can be quite heavy depending on routing configuration
- The URLResourceUpdatedEvent is supposed to be emmited whenever there's an update to the resource already associated with the URL and no url-affecting fields were touched.
2023-01-23 16:33:41 +08:00
Naz
ec4045cb57
Added dynamic-routing-events package
refs https://github.com/TryGhost/Toolbox/issues/503

- Reusing existing events inside of dynamic routing would only contribute to general confusion that is already there. Having separate "DomainEvents" is the best practice used throughout the code which is substituting generic events.
- The URLResourceUpdatedEvent is supposed to be emmited whenever there's an updated to the resource already associated with the URL circumventing full url regeneration process inside of DynamicRouting
2023-01-23 16:33:41 +08:00
Ronald Langeveld
95b6a9d569
Added mentions email notification alpha flag (#16164)
refs https://github.com/TryGhost/Team/issues/2429

- Added an alpha feature flag toggle for webmentions' email notifications
2023-01-23 15:42:48 +08:00
renovate[bot]
e3b0152ae0
Update dependency ember-template-lint to v5.3.2 2023-01-23 02:01:35 +00:00
Simon Backx
3ac282598c Fixed Playwright tests for new email flow
no issue

Tests stopped working because the Mailgun mocker stopped working since we moved to the new email flow.

This also fixes a unit test that needed to get updated.
2023-01-20 19:00:13 +01:00
Simon Backx
693216c29e Improved failed email error messages
refs https://ghost.slack.com/archives/C02G9E68C/p1673570945616369?thread_ts=1672700158.659209&cid=C02G9E68C

Fixed a typo in the error messages and made the errors more clear. Also hide the email configuration part when there is no email configuration.
2023-01-20 18:37:43 +01:00
Daniel Lockyer
56b0e65583
Revert "Added email snapshots for API versioning tests (#16139)"
- this reverts commit 85051199e3
- the tests here rely on dynamic content (the Ghost version number)
2023-01-20 18:05:36 +01:00
Ghost CI
ff1039bcbe v5.31.0 2023-01-20 16:01:01 +00:00
Ghost CI
d3557c2879 🎨 Updated Casper to v5.4.5 2023-01-20 16:01:00 +00:00
Simon Backx
a269fbe15a Fixed mention card target link 2023-01-20 15:54:22 +01:00
Simon Backx
5287aeb320 Added noopener security to mention cards 2023-01-20 15:21:23 +01:00
Fabien 'egg' O'Carroll
33ebe971f8
Added BookshelfMentionRepository implementation (#16156)
fixes https://github.com/TryGhost/Team/issues/2418

This stores the received webmentions in the database.

Co-authored-by: Simon Backx <simon@ghost.org>
2023-01-20 14:32:50 +01:00
Rishabh Garg
c7230f1858
Improved email stability (#16159)
- bumps new email flows to GA improving overall email stability
2023-01-20 18:55:20 +05:30
Rishabh
0a65f3405e Added new urls to known referrers list for source attribution
- adds new urls for Google and LinkedIn as known referrers so they are grouped together on Admin
2023-01-20 18:13:05 +05:30
Simon Backx
e879406659
Added outbound link tagging setting (#16146)
fixes https://github.com/TryGhost/Team/issues/2432
    
Adds outbound_link_tagging setting (enabled by default and behind
feature flag). If the feature flag is enabled, and the setting is
disabled, we won't add ?ref to links in emails.
    
This includes new E2E tests for email click tracking, which were also
extended to check outbound link tagging (for both MEGA and the new email
stability flow).

Also fixes a test fixture for the comments_enabled setting.
2023-01-20 13:41:36 +01:00
Simon Backx
8a95c62ff1
🐛 Fixed post links being marked as edited when they were not (#16153)
fixes https://github.com/TryGhost/Team/issues/2461

- Ignores 'edited' links when there is only one second differences.
- Make sure we don't set updatedAt when linking a post to a redirect
2023-01-20 13:31:54 +01:00
Daniel Lockyer
34fe49b110
Added version information to log lines
refs https://github.com/TryGhost/Toolbox/issues/501

- this reverts commit 48dda23554
- also includes a resolution for `@elastic/elasticsearch` so we don't
  run a version that is potentially problematic - see referenced issue
  for context
2023-01-20 13:18:44 +01:00
Fabien "egg" O'Carroll
40c280f6ac Improved naming of output serializer url util 2023-01-20 18:49:26 +07:00
Fabien "egg" O'Carroll
68f469c78b Fixed mock instantiation
This mock will fail by default now, which stops us from adding bad data in prod.
2023-01-20 18:49:26 +07:00
Fabien "egg" O'Carroll
833182bc7f Implemented a basic RoutingService for Mentions
refs https://github.com/TryGhost/Team/issues/2466

This initial implementation just checks that we're on the right origin and
subdomain, but should be extended to check if the URL actually resolves to a
page hosted on the site!
2023-01-20 18:49:26 +07:00
Fabien "egg" O'Carroll
5b4bc01504 Added initial inline ResourceService implementation
refs https://github.com/TryGhost/Team/issues/2465

We've restricted this to Post resources for now until we update the Mention
entity to be able to handle multiple resource types.
2023-01-20 18:49:26 +07:00
Naz
d44386dae3
Refactored resource config as a DI in Resources
refs https://github.com/TryGhost/Toolbox/issues/503
refs https://github.com/TryGhost/Toolbox/issues/406

- In Ghost 5.x we dropped multi-versioned API, which means there's no need to track resource configs dynamically as there can only be one version
- Along with removed "initResourceConfig" refactored the "config" file itself to be injected into Resource's constructor - allows for easier testing.
2023-01-20 19:47:26 +08:00
Ronald Langeveld
3061fb2b3b
Added mentions database table (#16150)
closes https://github.com/TryGhost/Team/issues/2417

- added new `mentions` database table to be able to store incoming webmentions.
- updated schema and tests to match.
2023-01-20 19:08:07 +08:00
Simon Backx
8c9893e568 Improved MentionSendingService timeout
no issue

The current timeouts were unreliable.
2023-01-20 11:46:18 +01:00
Steve Larson
8895d22602
Added mention discovery service (#16154)
fixes https://github.com/TryGhost/Team/issues/2407

The MentionDiscoveryService fetches mentioned sites to return the webmention endpoint.
2023-01-20 11:45:48 +01:00
Fabien 'egg' O'Carroll
b1e6eb0b5e
Unsubscribed existing Members with suppressed emails (#15952)
refs https://github.com/TryGhost/Team/issues/2367

We already have existing Members which have their emails suppressed that
need to have their newsletter subscriptions removed.
2023-01-20 16:14:50 +07:00
renovate[bot]
81fe4840dd Update dependency postcss to v8.4.21 2023-01-20 09:07:51 +01:00
Djordje Vlaisavljevic
f7f2224fc5 Added logic for displaying author and publisher
refs https://github.com/TryGhost/Team/issues/2434
2023-01-19 22:38:30 +00:00
renovate[bot]
3737788d75 Update dependency i18next to v22.4.9 2023-01-19 17:58:16 +00:00
Simon Backx
a596acf7d2
Added MentionSendingService (#16151)
fixes https://github.com/TryGhost/Team/issues/2409

The MentionSendingService listens for post changes and sends webmentions
for outbound links in the post.
2023-01-19 17:35:10 +01:00
Daniel Lockyer
1f32a7be8a
Added CI test to ensure i18n strings are extracted out
refs https://github.com/TryGhost/Ghost/issues/15502

- this will test if we have strings that aren't present in the JSON
  files in CI, so we don't miss out on translations
2023-01-19 16:20:55 +01:00
Djordje Vlaisavljevic
437e1cc936 Updated Mention card design
refs https://github.com/TryGhost/Team/issues/2434
2023-01-19 12:52:28 +00:00
Djordje Vlaisavljevic
62d4746469 Added missing properties
refs https://github.com/TryGhost/Team/issues/2434
2023-01-19 12:50:57 +00:00
Kevin Ansfield
7e98f1b9f4 Fixed editor cards being accessible when their availability checks failed
no issue

Using the slash menu it was possible to insert cards that shouldn't have been accessible based on their availability checks. This was happening because we were only hiding the visibility of the cards in the template rather than completely removing them from the slash command matching logic.

- added `{{card-menu-items}}` helper that combines the availability matching and snippet section addition to return a complete array of sections+items that match the current system state and post type
- added `@menuItems` argument set to the output of `{{card-menu-items}}` to the two card menu components so they are working against a pre-filtered list of menu items
  - lets us remove duplication of code that handled pushing snippets section into the menus
- removed availability check conditionals from `<KoenigMenuContent>` as the menu items passed in are now pre-filtered
2023-01-19 12:38:49 +00:00
Fabien "egg" O'Carroll
111c5742c9 Implemented WebmentionMetadata using the oembed service
https://github.com/TryGhost/Team/issues/2458

This is an initial pass at pulling metadata from webmention sources, we've also
updated the fake data to pull from some real-world sites which implement
webmentions. We've reused the oembed service here, long term it would be nice to
pull the metadata parsing/pulling part out, so that we can have more generic
error messages.
2023-01-19 19:25:52 +07:00
Fabien "egg" O'Carroll
4215786bf1 Improved internal types for the Mention entity
This makes development a little more streamlined, and helped catch a wee bug!
2023-01-19 19:25:19 +07:00
Fabien "egg" O'Carroll
3a0e0831ec Updated Mention entity validation of properties
Based on a discussion in slack we want to make all metadata properties optional,
with the exception of the title, which will default to the host of the source
URL if it's missing.

This is so that we can accept as many webmentions as possible and convert them
into Mentions. If we were to have strictly validation, we'd end up having to
drop webmentions that didn't match our criteria, and lose important data.

Giving the title a default allows us to provide a consistent UI experience too.
2023-01-19 19:24:07 +07:00
Fabien "egg" O'Carroll
743da542d0 Wired up the Webmention receive endpoint to the Mentions service
refs https://github.com/TryGhost/Team/issues/2419

This is the initial stab at having everything wired up, we're not
using a queue but we are handling the processing of the Webmention
asyncrounsly so that the HTTP response can be end immediately.

We've also laid the groundwork for extending and implementing the
correct processing of Webmentions, for example checking if the target
URL exists in the system, pulling out the metadata from the Webmention
source and fetching any internal resources.
2023-01-19 18:41:49 +07:00
Fabien "egg" O'Carroll
d0d45d45bc Refactored OEmbed service into a proper service
This allows us to share the implementation with other parts of the codebase, the
specific usecase here being fetching the metadata from webmention sources, for
display in the mentions UI, which will be borrowing a lot of stuff from the
bookmark card.
2023-01-19 18:41:49 +07:00
Fabien "egg" O'Carroll
a7eeb8f628 Fixed Mention Admin API mapper
There was a typo when reading the featured image property causing it not to show
up in the API responses.
2023-01-19 18:41:49 +07:00
Fabien "egg" O'Carroll
52892e0695 Added source_site_title and source_author to Mentions Admin API
refs https://github.com/TryGhost/Team/issues/2435

We need to update the API to explicitly include the new data from our Mention
entity
2023-01-19 18:41:49 +07:00
Fabien "egg" O'Carroll
560d168169 Added site title and author to Mention entity
refs https://github.com/TryGhost/Team/issues/2435

We've made these fields optional, and we may need to extend this to other fields
too as we discover more about the data we're able to get access to.
2023-01-19 18:41:49 +07:00
Daniel Lockyer
f45bba21f6
Disabled autoRefresh for inserted Actions events
- we don't end up using the inserted model from Bookshelf, so we
  shouldn't be performing a SELECT on the entry
- this disables refreshing the model using Bookshelf's `autoRefresh:
  false` and allows the key through the sanitization for `add
2023-01-19 09:28:28 +01:00
renovate[bot]
150f2f8a5f Update dependency eslint-plugin-react to v7.32.1 2023-01-19 01:55:30 +00:00
Daniel Lockyer
fbbea1c8a1
Configured i18next-parser settings
refs https://github.com/TryGhost/Ghost/issues/15502

- prevent it from creating `_old` files
- set JSON indentation to 4 spaces
2023-01-18 18:29:54 +01:00
Daniel Lockyer
2cf161168b
Added yarn translate:portal command
refs https://github.com/TryGhost/Ghost/issues/15502

- the amazing `i18next-parser` dependency will extract our translated
  strings from Portal and dump them into locale files, so we never have
  to add them manually
2023-01-18 18:23:53 +01:00
Djordje Vlaisavljevic
3634815a51 Updated HTML tags
refs https://github.com/TryGhost/Team/issues/2434
2023-01-18 16:49:04 +00:00
Djordje Vlaisavljevic
a3fcd5be6a Added static mention card design
refs https://github.com/TryGhost/Team/issues/2434
2023-01-18 16:42:44 +00:00
Daniel Lockyer
fb030353fe
Moved i18n translations into JSON files
refs https://github.com/TryGhost/Ghost/issues/15502

- plain JSON files are cleaner and less overwhelming than boilerplate JS
  files, and given they're going to be automatically generated, we
  probably won't be able to support comments anyway
2023-01-18 17:03:22 +01:00
Elena Baidakova
7ba40a92a1
Added @tryghost/kg-simplemde package (#16147)
refs TryGhost/Team#2427
2023-01-18 19:54:59 +04:00
Sanne de Vries
f5d03409c9 Fixed video card thumbnail error message incorrectly shown
No ref
2023-01-18 15:17:16 +01:00
Daniel Lockyer
5b2fc983c6
Added email snapshots to settings BREAD service test
refs https://github.com/TryGhost/Toolbox/issues/499

- switching to email snapshots allows us to track more of our output to check for regresssions
2023-01-18 14:19:27 +01:00
Daniel Lockyer
0d9268ad86 Added i18n package
refs https://github.com/TryGhost/Ghost/issues/15502

- this is an early implementation of an i18n provider by
  exporting an instance of `i18next`
- there's a lot more to be done here but baby steps :)
2023-01-18 13:54:14 +01:00
Simon Backx
4d54880113
Added error handling for ENAMETOOLONG import error (#16054)
fixes https://github.com/TryGhost/Team/issues/2200

When zipping a folder that contains files with UTF-8 characters in the filename, using the MacOS Archive Utility, the resulting zip will be missing some UTF-8 configuration bit. This breaks the unzipper, causing it to decode the filenames using the wrong encodign.

When the file names are long, and become longer than the length allowed by the OS, an ENAMETOOLONG error is thrown. This error is not handled by the importer, and causes the import to fail.

This adds a specific check for this error so we can show a clear error message to the user, that helps them to resolve the issue. We are currently unable to fix the issue on our side, because of a lack of well supported zip libraries for node.
2023-01-18 13:28:36 +01:00
Simon Backx
6c2af0793c Fixed failing Stripe webhook test
refs acf0baa8c7

Due to the bump in express-test, we now handle string bodies 'properly'. So they now pass all the Express middlewares. In the past this failing test did not really pass by the bodyParser.raw middleware,
so the content-type check on the `bodyParser.raw({type: 'application/json'})` middleware was not executed. Now it is, and the test fails because the content-type header was not set to application/json.
2023-01-18 12:05:55 +01:00
Simon Backx
acf0baa8c7 Updated express-test to 0.13.0
no issue

This includes support for uploading files in tests.
2023-01-18 11:42:42 +01:00
Simon Backx
2dc9c7dbba
🐛 Fixed suppression list data for emails with plus sign (#16140)
no issue

When fetching the suppression list data for emails with a plus sign, the
parsing of the NQL filter fails:

```at Child.applyDefaultAndCustomFilters (/Ghost/node_modules/@tryghost/bookshelf-filter/lib/bookshelf-filter.js:66:23)
[ghost] email:[simon+test@ghos
[ghost] ------------^
[ghost] Expecting 'OR', 'RBRACKET', got 'AND'
```
2023-01-18 10:17:57 +01:00
Daniel Lockyer
9ba251238a Added Content-Version header to all API requests
refs https://github.com/TryGhost/Team/issues/2400

- we've deemed it useful to start to return `Content-Version` for all
  API requests, because it becomes useful to know which version of Ghost
  a response has come from in logs
- this should also help us detect Admin<->Ghost API mismatches, which
  was the cause of a bug recently (ref'd issue)
2023-01-18 08:38:07 +01:00
Daniel Lockyer
85051199e3
Added email snapshots for API versioning tests (#16139) 2023-01-18 08:09:28 +01:00
renovate[bot]
28ed23c13e Update dependency knex to v2.4.1 2023-01-18 07:43:07 +01:00
Naz
967dd7d244
Added deprecation notice to mockManager's sentEmailCount
refs https://github.com/TryGhost/Toolbox/issues/499

- The mockManager's sentEmailCount is left here to avoid breaking many tests that already depend on this method. With future improvements to email snapshot tests this method should not be used. Instead, emailMockReceiver's own sentEmailCount method should be used directly.
2023-01-18 12:55:45 +08:00
Simon Backx
87be76ebb3 Added externalAttribution feature flag
fixes https://github.com/TryGhost/Team/issues/2430
2023-01-17 15:15:32 +01:00
Djordje Vlaisavljevic
5a90975288 Fixed staff page bug when logged in as editor
no refs
2023-01-17 14:02:01 +00:00
Simon Backx
34d4e5c8d0
Added logging to webmention endpoint (#16137)
closes https://github.com/TryGhost/Team/issues/2424

Adds the new webmention endpoint, responds with the correct statuscode
and empty body at `https://site.ghost/webmentions/receive/`.

Also updates the head link tag to the new endpoint location. We don't
use /webmention because that could conflict with post slugs.
2023-01-17 14:26:43 +01:00
Daniel Lockyer
48dda23554
Reverted "Added version information to log lines"
refs https://github.com/TryGhost/Toolbox/issues/501

- this reverts commit f2116357b7
- something with Elasticsearch is causing high CPU usage, so this commit
  reverts that for now
2023-01-17 13:20:47 +01:00
Ronald Langeveld
57bc14e00f
Added Mentions endpoint to Admin (#16136)
ref https://github.com/TryGhost/Team/issues/2421

- added the Mentions API endpoint to Admin
- setup initial mention model in the Ember Store to be able to dev with the endpoint
- added basic routing to access the `/mentions` page that is currently behind feature flags
- Setup basic testing with a mirage mock endpoint.
2023-01-17 20:16:18 +08:00
Fabien "egg" O'Carroll
008d83ad08 Fixed the Mention mapper
This had some leftover code in that shouldn't have been merged.
2023-01-17 17:06:22 +07:00
Fabien "egg" O'Carroll
1babf6126a Added initial basic Mentions implementation
refs https://github.com/TryGhost/Team/issues/2416

This extends the mock API to use a more formal pattern of moving our
entity code into a separate package, and use the service/repository
patterns we've been work toward.

The repository is currently in memory, this allows us to start using
the API without having to make commitments to the database structure.

We've also injected a single fake webmention for testing. I'd expect
the Mention object to change a lot from this initial definition as we
gain more information about the type of data we expect to see.
2023-01-17 17:01:20 +07:00
renovate[bot]
3d79d10ddf Update dependency mysql2 to v3.0.1 2023-01-17 08:22:53 +01:00
Fabien 'egg' O'Carroll
a8a279d667
Wired up mock Mentions Admin API (#16134)
refs https://github.com/TryGhost/Team/issues/2416

This doesn't even return a Mention in the correct format at the moment,
but it's just to get an endpoint there, behind a flag and returning data
so that we can start playing with the API and having it hooked up the
the Admin.

The next step will be fleshing this out further and defining the
services and repository to back it, as well as updating the Admin so that
we can fetch mentions to display in the UI
2023-01-17 12:39:32 +07:00
Ronald Langeveld
e55405691a
Added webmentions package (#16133)
refs https://github.com/TryGhost/Team/issues/2408

- Added webmentions package 
- Setup basic folder structure, controllers, repositories & services to
get started.
- note package haven't been published yet
2023-01-17 13:18:16 +08:00
Naz
112ab23968
Removed direct use of @tryghost/jest-snapshot
refs https://github.com/TryGhost/Toolbox/issues/499
refs 6bcc47a0ad

- Using module directly caused issues with snapshots manager instance initialization (mocha hooks did not apply to a correct instance)
- See refed commit for more
2023-01-17 12:50:49 +08:00
Naz
1f6b2b4d5e
Added email content testing capabilities
refs https://github.com/TryGhost/Toolbox/issues/499

- Outgoing emails have been a weak point of Ghost's stability recently. The concept of "emailMockReceiver" similarly to "webhookMockReceiver", allows to test side-effects like outgoing emails.
- This is a first iteration which should lay groundwork for testing all outgoing emails in the future
- The change adds a new concept of "email mock receiver" which is very similar to how the "webhook mock receiver" works. The email mock receiver exposes two methods to record and verify snapshots:
- matchHTMLSnapshot - records and verifies only the HTML content of the outgoint email
- matchMetadataSnapshot - records and verifies all the non-HTML properties sent along an email content, e.g.: to address, plaintext, subject, etc.
- What's missing is matching content based on dynamic content like dates, links with JWT tokens, etc.
2023-01-17 12:50:49 +08:00
Steve Larson
9199547bfe add webmentions feature flag
refs #2406
Adding the alpha feature flag so we can begin testing.
2023-01-16 15:03:36 -06:00
renovate[bot]
e95cff2ef4 Update dependency glob to v8.1.0 2023-01-16 09:59:45 +01:00
renovate[bot]
15c6fbb12a Update dependency html-validate to v7.13.1 2023-01-16 09:57:58 +01:00
Ghost CI
460d2dc339 v5.30.1 2023-01-16 05:35:14 +00:00
Fabien 'egg' O'Carroll
a30bbd5c69
Added webmention endpoint discovery link to ghost_head (#16126)
We've wrapped both changes in a try/catch to make sure this has no
adverse affects. The endpoint currently doesn't exist - we're only
adding this to get an idea of how much traffic we'll expect to see.

Long term we'll want to read the endpoint from the webmention service.
2023-01-16 12:32:04 +07:00
renovate[bot]
9dda708305
Update dependency eslint to v8.32.0 2023-01-16 00:19:30 +00:00
Ghost CI
2884521195 v5.30.0 2023-01-13 16:00:44 +00:00
Rishabh
fea75365c7 Updated guard for site data in Portal
closes https://github.com/TryGhost/Team/issues/2276

Portal had died with an unintelligible error about portal plans/includes being undefined when there was missing site data in some extreme edge cases. This change catches any errors in site transformation and logs it to console instead of crashing portal unexpectedly
2023-01-13 21:01:58 +05:30
Daniel Lockyer
c4391567f3
Updated jsdoc for mailgun-client
- in the event the Mailgun config doesn't exist, we return `null` from
  this function
- this updates the jsdoc to correct the return type of `getInstance`
2023-01-13 16:10:32 +01:00
Rishabh Garg
7d060e5edb
Bumped email stability flag to beta (#16116)
- allows email stability to be turned on without developer experiments enabled
2023-01-13 20:30:15 +05:30
Rishabh
8d4bc97dbe Fixed admin utils test
refs 9fc9e4311d
2023-01-13 19:49:01 +05:30
Peter Zimon
1ccb2f08e7 Improved mobile popup for feedbacks
no refs.

Post feedback popups were not optimised for mobile usage at all. All the content was on top of the window which is really hard to reach using a single hand/thumb and it just looked like a scaled down version of a desktop modal.
2023-01-13 15:14:49 +01:00
renovate[bot]
a990769e31 Update dependency jwks-rsa to v3.0.1 2023-01-13 14:07:10 +01:00
Rishabh
9fc9e4311d Removed www prefix from newsletter link table
closes https://github.com/TryGhost/Team/issues/2206

- removes `www.` from the url shown on links table in post analytics
- we had previously removed http(s) protocol from it as well, and they are only shown while editing the url
2023-01-13 15:38:54 +05:30
Sanne de Vries
1645f551bc Fixed error message display bug in video card
No ref
2023-01-13 09:56:36 +01:00
Fabien 'egg' O'Carroll
14f282d640
Improved newsletter delivery rates and email suppression handling
This introduces the new suppressions feature which will automatically
unsubscribe members from newsletters when their email is added to the
suppression list in Mailgun, this is usually due to emails either
permanently bouncing to the address, or the member making a spam
complaint.

Both Members and Admins are able to see that the email has been added to
the list, and Members are be able to request their email be removed from
the list via Portal.

Overall this feature should improve delivery rates of newsletters and
improve the rating of the domain you're sending from.
2023-01-13 15:46:57 +07:00
Naz
c2b18f0c87
🔥 Removed support for {{lang}} helper
refs https://github.com/TryGhost/Toolbox/issues/497
refs https://github.com/TryGhost/Toolbox/issues/406

- Removes {{lang}} helper which has been completely deprecated since Ghost 5.0. When using the helper in themes with 5.x backend the output will be empty
2023-01-13 13:12:53 +07:00
Rishabh Garg
755e031017
Fixed hidden free tier in Portal settings (#16072)
closes https://github.com/TryGhost/Team/issues/2338

If a site has the Free tier hidden from the Portal, and subsequently the Stripe connection is disconnected, this produces a dead-end state where no new members can sign up and the Free tier cannot be reactivated again in Portal settings as its hidden. This change -

- enables free tier toggle to be always shown on site irrespective of Stripe connection
2023-01-12 19:29:17 +05:30
renovate[bot]
bb2ffee69d Update dependency @playwright/test to v1.29.2 2023-01-12 11:35:22 +01:00
Sam Lord
f2116357b7 Added version information to log lines
refs: https://github.com/TryGhost/Toolbox/issues/502

Updated to @tryghost/logging@2.4.0 to allow metadata to be logged
2023-01-12 10:34:23 +00:00
renovate[bot]
02e84bf62a Update sentry-javascript monorepo to v7.30.0 2023-01-12 11:32:44 +01:00
Daniel Lockyer
9b38bd9509
🐛 Fixed ECONNRESET error when connecting to Azure MySQL DB
fixes https://github.com/TryGhost/Ghost/issues/14990

- there was a bug in `mysql2` [1] when connecting to Azure DBs, but this was
  subsequently fixed, so this commit bumps the package in Ghost and
  `knex-migrator`, where this was also bumped
- of note, this release includes https://github.com/sidorares/node-mysql2/pull/1666 and
  https://github.com/sidorares/node-mysql2/pull/1751, which are very interesting

[1]: https://github.com/sidorares/node-mysql2/pull/1438
2023-01-12 10:46:09 +01:00
Rishabh
ee7474db45 Fixed portal failure on opening email prefs for logged out member
- opening email preferences page via portal link doesn't work if member is not logged in.
- opens the signin page now instead of silently crashing
2023-01-12 13:02:41 +05:30
Rishabh
28bea791dd Added playwright tests for portal links
refs https://github.com/TryGhost/Team/issues/2371
2023-01-12 13:02:41 +05:30
Ronald Langeveld
f0e98c8bcb Fixed TypeError in filter-value
no issue

- Fixes a `TypeError` that sometimes crops up when setting filters from
  the URL parameters.
2023-01-12 13:18:55 +08:00
renovate[bot]
0fc72f6bd7
Update dependency eslint-plugin-react to v7.32.0 2023-01-11 16:27:56 +00:00
Elena Baidakova
a7366249bc
Added data-test attr for e2e tests to Portal (#16104)
refs TryGhost/Team#2371
2023-01-11 19:39:29 +04:00
Simon Backx
d5a359f2db Removed possibility to use {uuid} replacements in email
fixes https://github.com/TryGhost/Team/issues/2383

A user could use `{uuid}` inside an email only content and it would work. This currently isn't supposed to be used outside internal features (link click tracking, feedback buttons). For now this is only fixed in the new email flow under the email stability flag.
2023-01-11 15:56:15 +01:00
Sanne de Vries
c616f56a2a Replaced copy that uses "See" to "View" across Admin
Refs https://www.notion.so/ghost/Member-activity-improvements-247080c54dac4ebf86d10b38508d2a19
2023-01-11 15:44:23 +01:00
Simon Backx
8a2303413d Restricted email-service package to 100% test coverage
fixes https://github.com/TryGhost/Team/issues/2339

The email service is now fully covered by tests, and this commit also forces the test coverage to remain 100% after future changes.
2023-01-11 13:54:26 +01:00
Simon Backx
ee36a0fce9 Added 100% test coverage to SendingService
refs https://github.com/TryGhost/Team/issues/2339
2023-01-11 13:39:25 +01:00
Ronald Langeveld
0e2d1b3afd
Added redeemed offers filtering for members (#16071)
closes https://github.com/TryGhost/Team/issues/2011

- Gives publishers the ability to filter members based on which offer they used (redeemed) when they subscribed for a paid membership.
- On the offers page, the redemption count number links to a the members page with the filter already applied making it easy to have insight on which members used the offer / coupon.
2023-01-11 20:13:09 +08:00
Elena Baidakova
1aae5ba2eb
Added e2e tests for default tier (#16103)
refs TryGhost/Team#2371
2023-01-11 15:44:34 +04:00
Sanne de Vries
48ea2a871b
Updated editor re-authentication modal (#16102) 2023-01-11 12:39:04 +01:00
Simon Backx
9650521376 Fixed variable date formatting in new email flow
refs https://github.com/TryGhost/Ghost/actions/runs/3892250013/jobs/6643448457

Depending on the environment the date formatting in a newsletter could change. Now it is always set to the DD MMM YYYY format (like the old flow).
2023-01-11 12:26:09 +01:00
Simon Backx
6a664d11b9 Added 100% test coverage to email renderer and fixed authors bug
refs https://github.com/TryGhost/Team/issues/2339

This fixed a bug in the new email flow that more than 2 authors were displayed as 'undefined & 2 others'.
2023-01-11 12:14:02 +01:00
Elena Baidakova
a3f7188369
Added e2e tests for archive/unarchive tiers (#16101)
refs TryGhost/Team#2371
2023-01-11 13:53:07 +04:00
Daniel Lockyer
f8fe009f9f
Merged v5.29.0 into main 2023-01-11 09:30:13 +01:00
Ghost CI
d8e3f29a76 v5.29.0 2023-01-11 08:23:41 +00:00
Fabien "egg" O'Carroll
1e393472dd Truncated suppressions table
We have been adding emails to the suppressions table which are not on
the suppression list in Mailgun due to a misunderstanding of how
Mailgun handles 5xx error codes.
2023-01-11 15:14:08 +07:00
Fabien "egg" O'Carroll
9fe3555897 Updated MailgunEmailSuppressionList to only handle previous bounces
We're seeing behaviour from Mailgun where permanent failures with a
5xx error code are not being added to their internal suppression list,
which is resulting in the Ghost list becoming out of sync with
Mailgun.

Rather than adding emails to the suppression list when Mailgun does,
we're instead going to add emails _after_ Mailgun does, by waiting for
an error code which tells us the email is already on the suppression
list.

Those codes are 605 for previous bounces and 607 for previous spam complaints.
2023-01-11 15:13:57 +07:00