Commit Graph

32999 Commits

Author SHA1 Message Date
Naz
46508b7299
Fixed external media inliner logs
refs https://github.com/TryGhost/Toolbox/issues/524

- The "tableName" property is not present on the bookshelf model instance - it was a mistake assumption using it for logs. In case the logs are too ambiguous we could figure pass around model names separately into the "inlineSimpleFields" method
2023-03-08 15:24:21 +08:00
Naz
6fa00faaa2
Added naive domain matching when inlining media
refs https://github.com/TryGhost/Toolbox/issues/524

- Mobiledoc from a migrated content could contain html/markdown cards that could have a variety of different resource <> url pairs. To avoid complex logic and parsing html/md going with a simplest approach - matching external content URLs purely based on provided domains. This gives useful enough of a tool to migrate external content for a specific service (e.g Revue)
- In cases where the content is not supported the fetching will fail with a message and move on to the next match - which is a reasonable behavior for a migration tool
2023-03-08 15:22:51 +08:00
Naz
93ea9a2976
Added a not on media inlining perf improvement
refs https://github.com/TryGhost/Toolbox/issues/524

- Fetching media from a remote server is an expensive network operation. Given there's probability for the content to reuse the same image in different posts or in multiple places, we could save on extra fetches by adding caching to the remote media fetch method
2023-03-08 14:16:28 +08:00
Naz
831a76505c
Fixed root zip image/media/files copying during import
https://github.com/TryGhost/Toolbox/issues/523

- During import process of content files the files from the root directory were also copied over. This is causing chaos in the root of content folder with files that only needed for data import. For example, the csv files needed for Revue import were also copied over by "file importer" even though those do not belong to any content.
- Any content import files - images, media, files, should be in according folders in the imported zip file. The root files in the base zip directory are for data-related imports
2023-03-08 11:03:47 +08:00
renovate[bot]
ec869cfa0d
Update dependency mysql2 to v3.2.0 2023-03-07 23:33:30 +00:00
renovate[bot]
a613e5f9b0
Update dependency mailgun.js to v8.2.1 2023-03-07 21:13:12 +00:00
Steve Larson
c111f8bef4 🐛 Fixed referrals to web archive
no refs
-removed ref param for archive.org domain
-ref links to archive.org were broken
2023-03-07 13:35:34 -06:00
renovate[bot]
f6280d835d
Update dependency i18next to v22.4.11 2023-03-07 15:12:44 +00:00
Simon Backx
38de815d98
Removed old email flow (#16349)
fixes https://github.com/TryGhost/Team/issues/2611

The old email flow is no longer used since we introduced the email stability flow. This commit removes the related code and tests. The general test coverage decreased a bit as a result, because the old email flow probably had a high test coverage. The new flow is in separate packages, so it couldn't contribute to a higher test coverage (but it does have 100% unit test coverage).
2023-03-07 16:08:40 +01:00
Simon Backx
3db434736b
🐛 Fixed replacements with fallback in plaintext newsletters (#16372)
fixes https://github.com/TryGhost/Team/issues/2683

When sending a newsletter with a replacement that has a fallback, the
replacement only happens in the HTML version of the newsletter. The
plaintext version isn't replaced.

This commit fixes the issue and adds some tests to make sure it doesn't
happen again.

The cause of the issue was that we used the original matched Regex text
to replace. But that was calculated on the HTML version, so double
quotes were encoded. This change updates the generated 'token' regex to
also match on both a double quote as the escaped double quote.
2023-03-07 15:34:43 +01:00
Peter Zimon
d58f01bac7 Refined new signup email templates
refs. https://github.com/TryGhost/Team/issues/2489

- Alignment of avatar in free and paid signup email template was off
- Paragraphs was mixed in free signup email template which resulted in an unwanted space
2023-03-07 14:19:30 +01:00
Simon Backx
f50224f445
Improved network access and mocking in tests (#16371)
refs https://github.com/TryGhost/Team/issues/2667

Some tests still accessed the internet. Now network access is disabled
by default. This change also introduces two helper methods related to
networking (mocking Slack and Mailgun).

This fixes two unreliable tests:
- Staff service was accessing a Slack test API -> timeout possible
- MentionSendingService was trying to send webmentions for every post
publish/change -> possible timeouts and job issues
2023-03-07 13:20:28 +01:00
Daniel Lockyer
9fde77a95c
Temporarily removed Slack notification from failed browser tests
- this is becoming far too noisy and there are some dependency issues we
  need to fix first
2023-03-07 12:18:28 +01:00
Daniel Lockyer
b3d36709ab
Downgraded expect and @playwright/test packages
refs 6460522352

- these were both bumped around the time that Playwright browser tests
  started going awry, so they may be the cause of some random failures
2023-03-07 12:11:05 +01:00
Simon Backx
a58396b2de Removed email sending retrying in test mode
no issue

Disables email sending retrying in test mode by default. This is to prevent test timeouts and to make testing more reliable in case where we manually let a batch fail.
2023-03-07 11:52:10 +01:00
Naz
bd9b76c5ec
Fixed inlined media storage path
refs https://github.com/TryGhost/Toolbox/issues/523

- When storing a Buffer through saveRaw the target path *has to* be relative path instead of a full path.
2023-03-07 18:42:58 +08:00
Simon Backx
b2e00221e1 Fixed MentionSendingService listening to events multiple times in tests
no issue
2023-03-07 11:24:02 +01:00
Simon Backx
4147663cde 🐛 Fixed ability to cancel Stripe subscriptio when deleting member
fixes https://github.com/TryGhost/Team/issues/2670

"Also cancel subscription in Stripe" checkbox was not showing up when deleting a member with active subscriptions.
2023-03-07 10:38:04 +01:00
Naz
6d4dcdf45d
Extended external media inlining
refs https://github.com/TryGhost/Toolbox/issues/524

- We need to be able to inline external media in all internal resources: tags, users, post's meta fields.
- This change adds media inlining logic to all these resources
2023-03-07 15:22:56 +08:00
renovate[bot]
47b6c0b61d Update sentry-javascript monorepo to v7.41.0 2023-03-07 06:51:35 +01:00
renovate[bot]
d140e656c5 Update dependency expect to v29.5.0 2023-03-07 06:49:58 +01:00
Daniel Lockyer
5a6b9e6d84
Added debug logging to measure model require time
- these models should be as lightweight as possible to require, so we
  need to measure how long they take
- this has already found a couple of models which need optimizing
2023-03-06 13:11:59 +01:00
Tim Gates
5394fa81fc
Fixed typos (#16359) 2023-03-06 13:11:45 +01:00
Naz
33316a6440
Added e2e test covering media inliner job start
refs https://github.com/TryGhost/Toolbox/issues/523

- The test is useful for future iterations of the response format and as a quick reference on which parameters the media inlining endpoint accepts.
2023-03-06 17:43:33 +08:00
Rishabh
38b356a748 🎨 Added newsletter preference page link in portal settings
- adds data attribute and link to member's newsletter preference page in portal to portal links table in settings
2023-03-06 15:09:24 +05:30
Rishabh Garg
d6a1d98aca
🎨 Added source attribution info to email alerts (#16360)
refs https://github.com/TryGhost/Team/issues/2489

- adds attribution info for new free and paid members in email alerts
2023-03-06 15:06:47 +05:30
Naz
74067f23bd
Moved the media inliner into a job
refs https://github.com/TryGhost/Toolbox/issues/523

- Media inlining is a long running piece of logic that suits perfectly for a job. It is not as critical to be offloaded into a separate thread at this point, so leaving it as "inline" while the feature is in experimental stage.
2023-03-06 17:10:02 +08:00
Naz
2ce992ed00
Added media inliner for mobiledoc content
refs https://github.com/TryGhost/Toolbox/issues/523

- This is a first pass media inliner going through all posts and checking to inline media from specified domains
- As a working copy the inliner looks for image content from  Revue and Substack
2023-03-06 15:44:22 +08:00
renovate[bot]
0b2f88c100
Update dependency nodemon to v2.0.21 2023-03-06 03:45:22 +00:00
Sam Lord
472e52a536 Made URL class warn on duplicate URLs in the test suite
refs: https://github.com/TryGhost/Toolbox/issues/389

Instead of logging errors, this will warn when adding a duplicate URL in the test environment.

At the moment, this is happening a lot in the test suite. While we also need to fix the root cause of this so we're not erroring in the product, it's a massive amount of spam in the logs when running the test suite which could prevent us from finding other errors which are causing issues.
2023-03-03 18:25:53 +00:00
Sam Lord
13b0f51b13 Prevent simple error logs by stubbing log library
refs: https://github.com/TryGhost/Toolbox/issues/389

This removes many error logs when the end-to-end test suite is run with the log-level set to error. Many errors are intentional, so the resolution is typically to stub the error log function and assert that it would have been called.
2023-03-03 18:25:53 +00:00
Sam Lord
cd5aca7227 Prevented .well-known file tests from attempting to send mail
refs: https://github.com/TryGhost/Toolbox/issues/389

These tests would log errors while attempting to send email.
2023-03-03 18:25:53 +00:00
Sam Lord
5b2adbe4c5 Prevents errors from being logged during the offers tests
refs: https://github.com/TryGhost/Toolbox/issues/389

Intentional errors now call a logging library stub
2023-03-03 18:25:53 +00:00
Sam Lord
8a11ac936d Prevent app services from starting when Ghost is booted without frontend
refs: https://github.com/TryGhost/Toolbox/issues/389

Prevents errors caused when attempting to setup the appService, which relies on the frontend.
2023-03-03 18:25:53 +00:00
Sam Lord
64d94c8c8c Prevent posts-legacy tests from logging errors
refs: https://github.com/TryGhost/Toolbox/issues/389

The newsletter fixtures no longer errors when accessing the test image, and tests which intentionally error now stub the logging call
2023-03-03 18:25:53 +00:00
Sam Lord
a0acccc088 Prevent errors in E2E tests where front-end is disabled and Ghost Admin accesses themes
refs: https://github.com/TryGhost/Toolbox/issues/389
2023-03-03 18:25:53 +00:00
Steve Larson
3e18715caa added twitter embed tests
refs TryGhost/Team#2663
-added basic coverage for twitter embeds
2023-03-03 10:54:03 -06:00
Ghost CI
349e46c454 v5.37.0 2023-03-03 16:00:38 +00:00
Ghost CI
dfde0acf12 🎨 Updated Casper to v5.4.7 2023-03-03 16:00:37 +00:00
Naz
1f95fd50df
Fixed Revue import
refs https://github.com/TryGhost/Toolbox/issues/523

- When importing a zip with Revue data and no "files" or "media" the importer blew up. This is due to a bug where the files from the root directory are still being copied over (the should not be!)
- Added protective extra ".data" check to make sure code doest not blow up when there's no data property to be processed
2023-03-03 17:46:02 +08:00
Naz
29b3da26c3
Added ExternalMediaInliner package scaffolding
refs https://github.com/TryGhost/Toolbox/issues/524

- This is groundwork to contain logic related to external media scraping/inlining for posts.
2023-03-03 17:07:50 +08:00
renovate[bot]
ddd43b7daf Update dependency @playwright/test to v1.31.2 2023-03-03 09:18:21 +01:00
Naz
901485c47b
Added media inlining button in settings>labs
refs https://github.com/TryGhost/Toolbox/issues/524

- Having a button in Admin UI allows to manually trigger media inlining job once import has been processed.
- Added a "start" button hidden behind labs calling Admin API endpoint that starts external media inlining job.
2023-03-03 14:41:24 +08:00
Naz
de87479fb9
Bootstrapped media inlining trigger endpoint
refs https://github.com/TryGhost/Toolbox/issues/524

- Added an endpoint to Admin API under `POST /db/media/inline` that accepts `domains` as JSON payload triggering media inlining job
2023-03-03 14:26:52 +08:00
Naz
da78e235dd
Added media inliner service scaffolding
refs https://github.com/TryGhost/Toolbox/issues/524

- This is groundwork to build up on when adding media inlining functionality. This service will be used by the API to trigger post's content processing and media inlining.
2023-03-03 14:10:30 +08:00
Naz
a479b719ea
Added media inliner feature alpha flag
refs https://github.com/TryGhost/Toolbox/issues/524

- This flag will be used to test the trigger to external media inliner job
- The Admin UI is still unclear so sticking it behind the flag for some experimentation before shipping the feature
2023-03-03 10:54:01 +08:00
Djordje Vlaisavljevic
7bc7881088 Added a temporary fix for wrapping multiple mentions dropdown
refs https://github.com/TryGhost/Team/issues/2606
2023-03-02 20:06:53 +00:00
Djordje Vlaisavljevic
c6185017be Added mentions widget empty state
refs https://github.com/TryGhost/Team/issues/2608
2023-03-02 20:06:53 +00:00
Djordje Vlaisavljevic
683a0726df Added dark mode hover state
refs https://github.com/TryGhost/Team/issues/2608
2023-03-02 20:06:53 +00:00
Kevin Ansfield
7c84ef8c2d
Wired up click-to-refresh upgrade banner
closes https://github.com/TryGhost/Team/issues/2400

- used semver comparison to detect when the app version is less than the content-version header in any API response to toggle `upgradeStatus.requiresRefresh` that is used to conditionally show the upgrade banner
- only works on minors as we don't store the full Ghost patch version in `config.APP.version`
2023-03-02 18:39:38 +00:00