ref https://linear.app/tryghost/issue/ONC-197
- YouTube has started responding to video page requests with localised content when requested from certain IPs, with that localised content not containing the required `<link rel="alternate" ...>` tag pointing to the oembed endpoint
- we were fetching video pages rather than the oembed endpoint for YouTube Live URLs because they are not recognised by the oembed extraction library we use
- by modifying the URL from a live URL to a watch URL before we perform oembed lookup/extraction we are able to bypass the (localised) page fetch and instead grab the oembed content directly
closes https://linear.app/tryghost/issue/ENG-762
- nytimes.com and other sites return 403 responses when requests do not match typical browser user-agents
- our bookmark fetching requests were using `Ghost(https://github.com/TryGhost/Ghost)` meaning bookmark creation failed for these user-agent-blocking sites
- switched to using a standard browser user-agent string to avoid such blocks
closes https://github.com/TryGhost/Ghost/issues/20028
It's fairly common practice for oembed providers to skip some of the "required" fields from the oembed spec such as `height` when it doesn't make sense for the embeddable content, this was the case with Bluesky embeds which return `height: null`
- removed validation for `height` being present in the response for it to be recognised as an embed because we don't use it anywhere and the validation is blocking otherwise valid embeds
fixes https://github.com/TryGhost/Product/issues/4237
- this fixes the fact that we return a HTTP 500 response when the oembed
library receives an error, such as a 401 or 403
- includes special handling for cases where we want to return a slightly
different error message
- also adds unit tests for @tryghost/oembed-service package
refs: https://github.com/TryGhost/Toolbox/issues/595
We're rolling out new rules around the node assert library, the first of which is enforcing the use of assert/strict. This means we don't need to use the strict version of methods, as the standard version will work that way by default.
This caught some gotchas in our existing usage of assert where the lack of strict mode had unexpected results:
- Url matching needs to be done on `url.href` see aa58b354a4
- Null and undefined are not the same thing, there were a few cases of this being confused
- Particularly questionable changes in [PostExporter tests](c1a468744b) tracked [here](https://github.com/TryGhost/Team/issues/3505).
- A typo see eaac9c293a
Moving forward, using assert strict should help us to catch unexpected behaviour, particularly around nulls and undefineds during implementation.
refs: https://github.com/TryGhost/Toolbox/issues/188
- some of our older packages used a pattern for linting which missed using test config for linting tests
- we need this to be consistent so that we can add more eslint rules for testing
- two packages also didn't use the lib pattern, which made the lint pattern error - so this was fixed as well
refs https://github.com/TryGhost/Toolbox/issues/363
- the oembed service is completely standalone and could do with some
individual unit tests
- moving it out to a package allows us to draw the boundaries better and
allows us to remove some dependencies from the core package.json
