Commit Graph

9 Commits

Author SHA1 Message Date
Paul Davis
489fae98eb
Improve external media inliner URL handling (#18428) 2023-10-19 11:58:41 +01:00
Sam Lord
fd7ead6ce6 Upgraded to latest version of @tryghost/request
refs: https://github.com/TryGhost/Product/issues/3782

Latest version of request avoids using the OS-level name resolution system. This prevents long shutdowns from occuring where the OS is blocking the process from exiting cleanly.

The new version uses `got` 13, which has many breaking changes. Some of these are resolved within @tryghost/request (like response errors), but input parameters need to be changed.
2023-09-26 12:45:14 +01:00
Naz
46508b7299
Fixed external media inliner logs
refs https://github.com/TryGhost/Toolbox/issues/524

- The "tableName" property is not present on the bookshelf model instance - it was a mistake assumption using it for logs. In case the logs are too ambiguous we could figure pass around model names separately into the "inlineSimpleFields" method
2023-03-08 15:24:21 +08:00
Naz
6fa00faaa2
Added naive domain matching when inlining media
refs https://github.com/TryGhost/Toolbox/issues/524

- Mobiledoc from a migrated content could contain html/markdown cards that could have a variety of different resource <> url pairs. To avoid complex logic and parsing html/md going with a simplest approach - matching external content URLs purely based on provided domains. This gives useful enough of a tool to migrate external content for a specific service (e.g Revue)
- In cases where the content is not supported the fetching will fail with a message and move on to the next match - which is a reasonable behavior for a migration tool
2023-03-08 15:22:51 +08:00
Naz
93ea9a2976
Added a not on media inlining perf improvement
refs https://github.com/TryGhost/Toolbox/issues/524

- Fetching media from a remote server is an expensive network operation. Given there's probability for the content to reuse the same image in different posts or in multiple places, we could save on extra fetches by adding caching to the remote media fetch method
2023-03-08 14:16:28 +08:00
Naz
bd9b76c5ec
Fixed inlined media storage path
refs https://github.com/TryGhost/Toolbox/issues/523

- When storing a Buffer through saveRaw the target path *has to* be relative path instead of a full path.
2023-03-07 18:42:58 +08:00
Naz
6d4dcdf45d
Extended external media inlining
refs https://github.com/TryGhost/Toolbox/issues/524

- We need to be able to inline external media in all internal resources: tags, users, post's meta fields.
- This change adds media inlining logic to all these resources
2023-03-07 15:22:56 +08:00
Naz
2ce992ed00
Added media inliner for mobiledoc content
refs https://github.com/TryGhost/Toolbox/issues/523

- This is a first pass media inliner going through all posts and checking to inline media from specified domains
- As a working copy the inliner looks for image content from  Revue and Substack
2023-03-06 15:44:22 +08:00
Naz
29b3da26c3
Added ExternalMediaInliner package scaffolding
refs https://github.com/TryGhost/Toolbox/issues/524

- This is groundwork to contain logic related to external media scraping/inlining for posts.
2023-03-03 17:07:50 +08:00