Commit Graph

15 Commits

Author SHA1 Message Date
Paul Davis
489fae98eb
Improve external media inliner URL handling (#18428) 2023-10-19 11:58:41 +01:00
Daniel Lockyer
85098e07d4 Configured all unit tests to use dot reporter
refs https://ghost.slack.com/archives/C02G9E68C/p1696490748701419

- this configures mocha to use the dot reporter because the default is
  way too verbose in CI
2023-10-05 12:24:24 +02:00
Sam Lord
fd7ead6ce6 Upgraded to latest version of @tryghost/request
refs: https://github.com/TryGhost/Product/issues/3782

Latest version of request avoids using the OS-level name resolution system. This prevents long shutdowns from occuring where the OS is blocking the process from exiting cleanly.

The new version uses `got` 13, which has many breaking changes. Some of these are resolved within @tryghost/request (like response errors), but input parameters need to be changed.
2023-09-26 12:45:14 +01:00
Daniel Lockyer
c6cb35074a Updated linting and testing packages 2023-09-01 15:51:17 +02:00
renovate[bot]
7dce046786 Update Test & linting packages 2023-07-11 15:26:07 +02:00
Hannah Wolfe
6161f94910
Updated to use assert/strict everywhere (#17047)
refs: https://github.com/TryGhost/Toolbox/issues/595

We're rolling out new rules around the node assert library, the first of which is enforcing the use of assert/strict. This means we don't need to use the strict version of methods, as the standard version will work that way by default.

This caught some gotchas in our existing usage of assert where the lack of strict mode had unexpected results:
- Url matching needs to be done on `url.href` see aa58b354a4
- Null and undefined are not the same thing,  there were a few cases of this being confused
- Particularly questionable changes in [PostExporter tests](c1a468744b) tracked [here](https://github.com/TryGhost/Team/issues/3505).
- A typo see eaac9c293a

Moving forward, using assert strict should help us to catch unexpected behaviour, particularly around nulls and undefineds during implementation.
2023-06-21 09:56:59 +01:00
renovate[bot]
83373e1751 Update Test & linting packages 2023-04-05 15:16:08 +02:00
renovate[bot]
2223db5379
Update Test & linting packages 2023-03-13 02:36:20 +00:00
Naz
46508b7299
Fixed external media inliner logs
refs https://github.com/TryGhost/Toolbox/issues/524

- The "tableName" property is not present on the bookshelf model instance - it was a mistake assumption using it for logs. In case the logs are too ambiguous we could figure pass around model names separately into the "inlineSimpleFields" method
2023-03-08 15:24:21 +08:00
Naz
6fa00faaa2
Added naive domain matching when inlining media
refs https://github.com/TryGhost/Toolbox/issues/524

- Mobiledoc from a migrated content could contain html/markdown cards that could have a variety of different resource <> url pairs. To avoid complex logic and parsing html/md going with a simplest approach - matching external content URLs purely based on provided domains. This gives useful enough of a tool to migrate external content for a specific service (e.g Revue)
- In cases where the content is not supported the fetching will fail with a message and move on to the next match - which is a reasonable behavior for a migration tool
2023-03-08 15:22:51 +08:00
Naz
93ea9a2976
Added a not on media inlining perf improvement
refs https://github.com/TryGhost/Toolbox/issues/524

- Fetching media from a remote server is an expensive network operation. Given there's probability for the content to reuse the same image in different posts or in multiple places, we could save on extra fetches by adding caching to the remote media fetch method
2023-03-08 14:16:28 +08:00
Naz
bd9b76c5ec
Fixed inlined media storage path
refs https://github.com/TryGhost/Toolbox/issues/523

- When storing a Buffer through saveRaw the target path *has to* be relative path instead of a full path.
2023-03-07 18:42:58 +08:00
Naz
6d4dcdf45d
Extended external media inlining
refs https://github.com/TryGhost/Toolbox/issues/524

- We need to be able to inline external media in all internal resources: tags, users, post's meta fields.
- This change adds media inlining logic to all these resources
2023-03-07 15:22:56 +08:00
Naz
2ce992ed00
Added media inliner for mobiledoc content
refs https://github.com/TryGhost/Toolbox/issues/523

- This is a first pass media inliner going through all posts and checking to inline media from specified domains
- As a working copy the inliner looks for image content from  Revue and Substack
2023-03-06 15:44:22 +08:00
Naz
29b3da26c3
Added ExternalMediaInliner package scaffolding
refs https://github.com/TryGhost/Toolbox/issues/524

- This is groundwork to contain logic related to external media scraping/inlining for posts.
2023-03-03 17:07:50 +08:00