Commit Graph

5645 Commits

Author SHA1 Message Date
Naz
3b390639c4 Introduced booting from fixture url cache in test environment
refs https://github.com/TryGhost/Toolbox/issues/135

- To be able to reliably start ghost instance without a frontend the process needs access to urls/resources caches
- Storing the configuration in "paths" for now as there's no better place for it untill we are able to mock the content folder in pre-boot
2021-11-22 21:56:32 +13:00
Naz
c12ae81ece Introduced "withFrontend" and "withBackend" flags in boot
refs https://github.com/TryGhost/Toolbox/issues/135

- These flags are meant to control initialization of sections of the boot sequence depending on the needs - with or without bakend (API)/frontend (public handlebars site)
- Ideally these flags should not be passed deep into the components, and if the are (like in the web/parent/app case) it's a smell that we need to move things up into the boot process!
2021-11-22 21:56:32 +13:00
Hannah Wolfe
80358266b2 Added perf-improving PRAGMAs in testing mode
- These two things are meant to improve performance at the cost of reliability.
- Perfect for testing, however I think they make a minimal impact on modern SSDs :(
- Still worth a shot to see if it helps with CI
2021-11-19 10:47:21 +00:00
Daniel Lockyer
80d5f89382 Lazy loaded cheerio dependency
refs https://github.com/TryGhost/Toolbox/issues/136

- `cheerio` isn't needed during the boot but it takes time and memory to
  load the library
- this commit moves `cheerio` requires later into the code to when they
  are needed
2021-11-18 17:31:04 +01:00
Kevin Ansfield
ca26c3ab0a Added logging of unexpected errors in oembed service
no issue

- if we encounter an unexpected error whilst fetching embed details we return a generic validation error so we're not leaking any details about the URL that is being hit, however that meant the error logs were only showing validation errors making debugging difficult
- added explicit logging of the unexpected error before throwing the generic validation error
2021-11-17 13:31:20 +00:00
Peter Zimon
bbceace31c Implemented table based NFT email styles 2021-11-17 14:05:39 +01:00
Sam Lord
0692ddf57d Revert "Replace update user last seen middleware with package version"
This reverts commit 07f8c6efbb.

Revert "Remove unused unit test file"

This reverts commit f1798119d3.
2021-11-17 11:00:55 +00:00
Hannah Wolfe
332beaaf90
Moved "vhost-utils" to config helpers
- These are simple functions that get data from config in a specific format
- They are also used by the topmost part of the application
- Config helpers seems like a reasonable fit to get them out of the web folder
- Functions have also been renamed to try to get them to make more sense
2021-11-17 08:37:08 +00:00
Fabien O'Carroll
e6773d00e6 Updated NFT Embed custom provider
refs https://github.com/TryGhost/Team/issues/1211

Added ?format=json to the URL in an attempt to mitigate any issues with
weird caching and receiving HTML rather than JSON.

Used `type` in place of `card_type` to closer follow how the bookmark
card/embed works.

Removed the html & width/heihgt properties which are not needed at all.
2021-11-16 21:28:36 +02:00
Sanne de Vries
b6b024b132 Fixed missing bottom spacing on email callout cards 2021-11-16 19:19:49 +01:00
Kevin Ansfield
63b7ef7dc6 Added Tenor contentFilter setting to config
refs https://github.com/TryGhost/Team/issues/1217

- moved top-level `tenorApiKey` to `tenor:apiKey` and added `tenor:contentFilter`
- added base config to `defaults.json`
- updated `public-config.js` and API output serializer to use the new top-level `tenor` key
2021-11-16 17:42:26 +00:00
Sam Lord
a5484a0fe1 Fixed transaction logic in data importer
https://github.com/TryGhost/Toolbox/issues/130

The transaction no longer commits in the promise chain, which wasn't
valid logic for a transaction, since it is commited automatically when
the promise chain resolves, and rollsback automatically when the
promise chain rejects.

This makes code which fails during the transaction error in the right
place, instead of getting stuck here. (Especially good for writing
tests).

The tests for this code can now live in the integration folder.
2021-11-16 17:09:55 +00:00
Sam Lord
07f8c6efbb Replace update user last seen middleware with package version
no issue

Part of the goal to move these middleware packages to remove untested code from the codebase
2021-11-16 15:58:45 +00:00
Hannah Wolfe
4f9b72ff43
Renamed middlewares to middleware consistently
- This is a minor bugbare, but it will affect some configuration I'm about to do for c8
- I've been wanting to do it for ages, middleware is plural all on it's own so it's an odd affectation in our codebase
- This also only exists in 2 places, everywhere else we use "middleware"
- Sadly it did result in a lot of churn as I did a full find and replace, but consistency is king!
2021-11-16 15:51:47 +00:00
Hannah Wolfe
b781e47cb9
Changed to use different fixtures during testing
- this keeps production and test fixtures separate, so that changing the prod fixtures doesn't change the shape of our tests.
- we may still want to test that the production fixtures do what we expect, but that can be handled in a separate integration test, by specifically setting the fixture path
2021-11-16 14:03:42 +00:00
Hannah Wolfe
426c8bf918
Refactored fixtures to be loaded by fixture manager
refs: https://github.com/TryGhost/Toolbox/issues/133

- instead of just a collection of utils, we now have a class that manages fixtures
- this should allow us to change the path to fixtures, e.g. between prod/dev and test, so that different fixtures can be loaded by default
- also makes it easier to test the fixture manager code itself
2021-11-16 11:17:03 +00:00
Naz
458f5b894b Moved Urls and Rexources reinitialization
refs https://github.com/TryGhost/Toolbox/issues/127

- New instances or Urls and Resources during init were messing up test suites without an obvious solution. Moved them to only be triggered when the cache is present.
- Note the e2e-frontend tests are failing if thie initialization happens every time when init is called. There seems to be some reliance on Resources being the same intsance between "softReset" which seems wrong, but wasn't able to track it down definitevely
2021-11-17 00:00:23 +13:00
Naz
1a3a80cacc Added resource cache handling to UrlService
refs https://github.com/TryGhost/Toolbox/issues/127

- The resource cache is needed to have quick and reproducible state of the resouces tied to the urls instead of waiting for the db queries to finish.
- Allows to use UrlService without any database connection at all - useful for unit testing
2021-11-17 00:00:23 +13:00
Naz
9b309cf4e7 Refactored URL cache persistance
refs https://github.com/TryGhost/Toolbox/issues/127

- This refactor is needed to accomodate incoming "resource cache" which shares a lot of common logic with how url cache is operated
2021-11-17 00:00:23 +13:00
Hannah Wolfe
88311381ef Added a URL Service boot log
- At the moment there's no way to see in the logs when the URL Service finally finishes
- This is the moment when Ghost stops serving 503s
- Adding this log line so it's clear to see, which migh be useful whilst were refactorising
2021-11-16 19:13:36 +13:00
Sanne de Vries
632e15affc Added callout card email styles
Refs https://github.com/TryGhost/Team/issues/1206
2021-11-15 19:45:33 +01:00
Naz
83a54eeb99 Fixed UrlService's getPermalinkByUrl method
refs https://github.com/TryGhost/Toolbox/issues/127

- The "withUrlOptions" is used internally in the constructor by the frontend's routers. To avoid passing around a whole function from the frontend routers duplicated the functionality from there.
2021-11-15 19:40:44 +04:00
Naz
042618fe93 Fixed failing tests
refs 5a62253466

- The UrlService was relying on a "hidden" identifier field that was passed along with a router object. Now it's passed as an explicit parameter from the "frontend" to the backend's UrlService
2021-11-15 19:37:49 +04:00
Naz
5a62253466 Removed "router" dependency from UrlGenerator
refs https://github.com/TryGhost/Toolbox/issues/127

- Passing around whole instance of a frontend router was an overkill when there are only 3 static pieces of information that needed to be loaded. Extracting the router out makes the UrlGenerator way more readable, tests slimer, and the memory footpring of the process should be slightly lighter
- The toString overloading didn't make sense at the time of this refactor, maybe if there's a concrete usecase we could resurect it in a form of passing in a router's name or something.
2021-11-15 18:32:34 +04:00
Naz
6e205a3f05 Extracted an explicit "permalink" parameter in UrlGenerator constructor
refs https://github.com/TryGhost/Toolbox/issues/127

- This is an effor t to define a precise set of data needed for the UrlGenerator to function, which should help with decoupling it from the frontend routes
- This is almost the last piece to free us up from the massive "router" object that has been passed around
2021-11-15 18:32:34 +04:00
Naz
677ea1073d Extracted an explicit "resourceType" parameter in UrlGenerator constructor
refs https://github.com/TryGhost/Toolbox/issues/127

- This is an effor t to define a precise set of data needed for the UrlGenerator to function, which should help with decoupling it from the frontend routes
2021-11-15 18:32:34 +04:00
Naz
ca2809d432 Extracted an explicit "filter" parameter in UrlGenerator constructor
refs https://github.com/TryGhost/Toolbox/issues/127

- This is an effor t to define a precise set of data needed for the UrlGenerator to function, which should help with decoupling it from the frontend routes
2021-11-15 18:32:34 +04:00
Naz
176c369620 Refactored UrlGenerator to use DI constructor pattern
refs https://github.com/TryGhost/Toolbox/issues/127

- This is a standard across the codebase which is good to follow for better testability
2021-11-15 18:32:34 +04:00
Naz
33d02cddfe Added JSDocs to the UrlGenerator class
refs https://github.com/TryGhost/Toolbox/issues/127

- Before refactoring the URL Generator constructor to have a smaller construcotr, documented current state for future reference
2021-11-15 12:41:52 +04:00
Fabien O'Carroll
31e103be9d Updated NFT OEmbed provider to return metadata
refs https://github.com/TryGhost/Team/issues/1211

Instead of rendering the HTML as an embed - we will send back the
necessary data. This will allow us to keep all the knowledge of HTML
structure in the Koenig repository.
2021-11-15 10:31:35 +02:00
Peter Zimon
a68854c3bd Added NFT card email styles
- added email styles for NFT cards
- fixed buttons having capitalized text
2021-11-12 16:39:35 +01:00
Peter Zimon
661bde3892 NFT card frontend style refinements 2021-11-12 15:55:19 +01:00
Naz
04c6716171 Moved url cache loading logic behind the flag
refs https://github.com/TryGhost/Toolbox/issues/125

- The url loader is not fully working and is in very experimental mode. The aim is to ship this version to allow other team members to play with the feature in limited use-cases like testing evnironment
2021-11-13 01:01:28 +13:00
Naz
4582fd48b5 Parameterized url cache storage path
refs https://github.com/TryGhost/Toolbox/issues/116

- This should allow testing in a bit easier manner and would place the file into a more suitable directory
- ideally we'd put an alfa flag bahind this new "cached routes" feature to have less consequences to deal with if we have to back out
2021-11-13 01:01:28 +13:00
Naz
36bc05447c Booting instance with persisted urls
refs https://github.com/TryGhost/Toolbox/issues/116

- This is a PoC to check out how viable this approach is and if it's worth merging into main as a very quick win
- The `urls.json` is in a bad place right now should probably live in a data folder
2021-11-13 01:01:28 +13:00
Fabien O'Carroll
39f01f4925 Removed SVG from NFT OEmbed
refs https://github.com/TryGhost/Team/issues/1211

The SVG is causing issues with copy/paste in Firefox, we're going to
remove it and later replace it with a PNG.
2021-11-12 12:43:25 +02:00
Naz
b707f325c8 Exposed config initialization from Resources
refs https://github.com/TryGhost/Toolbox/issues/125

- Config initialization for the URL Resouces is a separate stage which doesn't have to be bundled along with resouce fetching method
- It gives even more flexibility when composing different ways to get the "resources" loaded into memory: right now it's from the DB but could come from a cache
2021-11-12 12:18:17 +04:00
Naz
963df37aaa Reworded UrlService's init documentation
refs https://github.com/TryGhost/Toolbox/issues/125

- The "init" method pattern is not used across the codebase and it's pretty standard to start the service in this way. Previous description was outdated and misguiding
2021-11-12 12:16:08 +04:00
Naz
39b480682f Fixed UrlService initialization
refs https://github.com/TryGhost/Toolbox/issues/125

- The await was missing from the "fetchResources" call which made the queue pick up less information than it should've
2021-11-12 12:15:01 +04:00
Naz
376e2f4cf3 Separated event listeners from resource fetching
refs https://github.com/TryGhost/Toolbox/issues/125

- The "fetchResources" method did way to many things extracted event listener setup logic to a separate method
- This allows to call out these stages as needed if we retreive resources separately from a cache of some sort and don't need to wait for the database response
2021-11-12 11:56:16 +04:00
Naz
46c3b53979 Moved queue start into UrlServices init block
refs https://github.com/TryGhost/Toolbox/issues/125

- The "fetchResources" is only used from one place and does way to many things at once:
- Fetches resources from the DB-
- Hooks up event listeners
- Starts the queue

- These are three different things, and should be three different methods ideally
2021-11-12 11:56:16 +04:00
Kevin Ansfield
6431e1a01c
Exposed tenorApiKey via config endpoint if set (#13734)
refs https://github.com/TryGhost/Team/issues/1217

- add `tenorApiKey` to `publicConfig.config()
- update canary config endpoint output serializer to include `tenorApiKey` when the `gifsCard` labs flag is enabled
2021-11-11 18:07:19 +00:00
Fabien O'Carroll
dda8e0249b Wired up NFT custom provider to canary API
refs https://github.com/TryGhost/Team/issues/1211

This registers the NFT custom OEmbed provider to the OEmbed service for
the canary API. This should probably be done in a centralised place -
but we do not have a single instance of the OEmbed service.

When we have more information about why the OEmbed service is
instantiated like this, we can think about moving it into a singleton
service with an `init` method - which is where we can register custom
providers.
2021-11-11 17:16:03 +02:00
Fabien O'Carroll
3b7284b62f Initial OpenSea NFT OEmbed customer provider
refs https://github.com/TryGhost/Team/issues/1211

This adds the initial custom OEmbed provider for OpenSea NFT's - we're
using the new custom provider functionality so that we can easily change
the logic here without affecting the core functionality.
2021-11-11 17:16:03 +02:00
Fabien O'Carroll
e69d112c77 Added support for custom OEmbed providers
refs https://github.com/TryGhost/Team/issues/1211

In order to override the default OEmbeds for OpenSea NFT's we need a way
to provide out own OEmbed data. We will want this in future too for
custom Twitter embeds, so this has been built in a way which allows
extension.
2021-11-11 17:16:03 +02:00
Naz
a64c186179 Fixed typo
no issue
2021-11-11 16:52:11 +04:00
Fabien 'egg' O'Carroll
7fae5b8341
🐛 Fixed setting Tier prices after changing Stripe accounts
refs https://github.com/TryGhost/Team/issues/1212

This now emits the event when the service is reconfigured, rather than
when we issue the reconfigure command, which causes the event and the
action to be run in the wrong order. This would then cause knock on effects
of having the database in an undefined state - with stripe data in not linked
to the current Stripe account.
2021-11-10 14:03:03 +02:00
Naz
4721b75463 Moved `/media/thumbnail/ Admin API endpoint to /media/thumbnail/upload
refs  https://github.com/TryGhost/Toolbox/issues/120

- Changed endpoint name as it suits way better to the `/upload` postfix convention we use along with all other endpoints that support file uploads
2021-11-09 16:39:33 +04:00
Naz
61b82e3ae2 Added thumbnail upload endpoint to Media API
closes https://github.com/TryGhost/Toolbox/issues/120

- Allows to update and upload brand new thumbnail images for previusly uploaded media resources
- The endpoint is available udner alpa flag as part of Admin API at `PUT /media/thumbnail/`
- As an input accepts following parameters:
- *required* `file` field containing an image file
- *required* `url` field containing parent media file URL
- *optional* `ref` as a field to put in an ID to reference the resource on the client side

- The response has following format:
```
{
  media: [{
    url: 'http://127.0.0.1:2369/content/images/1991/11/nicevideo_thumb.png'
    ref: 'unique-id-420'
  }]
}
```
2021-11-09 16:07:23 +04:00
Naz
2a7ef77a7b Added ability to delete existing files through storage adapters
refs https://github.com/TryGhost/Toolbox/issues/120

- When editing an uploaded media thumbnail  file there'a need to remove existing thumbnail to keep media files:thumbnails 1:1. - Because the API client only has a public URL under which the resource is served it can only provide that as an API input, the `urlToPath` was also added to the base class of LocalStorageAdapter (it might be moved up to the BaseAdapter in the future if we see a need)
2021-11-09 16:07:23 +04:00