Commit Graph

5681 Commits

Author SHA1 Message Date
Fabien egg O'Carroll
c4021b6eb4 Added OpenSea API key to config
refs https://github.com/TryGhost/Team/issues/1237

We want to use an API key for OpenSea where possible so that we do not
run into rate limit issues.
2021-11-30 12:58:25 +02:00
Hannah Wolfe
ad9eb35746
Moved theme-specific error handling to frontend
- our themeErrorRenderer is only used in the frontend.. move it there
- this required exposing prepareError as shared middleware
- TODO: move these shared compontents to @tryghost/error
2021-11-29 17:04:01 +00:00
Hannah Wolfe
d7c4168452
Removed HTMLErrorRenderer completely
refs: 0799f02e80
refs: 5e931e2e37

- with the referenced two commits I replaced our old HTML renderer with some code borrowed heavily from finalHandler
- I had intended to modify this further to out put our message, context and help error messages
- However, I ended up doing this in prepareError so it's done for all error renderers
- There's now very little point keeping duplicated code from finalHandler just to output the status code
- If we remove this code, express will fall back to finalHandler anyway, so the output is near identical
2021-11-29 17:00:47 +00:00
Hannah Wolfe
3b069b544f
Cleaned code patterns in error handler
- got rid of old _private & variable pattern in favour of const and module.exports
- changed weird capitalisation naming conventions to be camelCase
- removed some very old TODOs that we're never gonna get TODONE
   - these are mostly old ideas that never made it, and it's been so long they're clearly not important
2021-11-29 16:53:58 +00:00
Sam Lord
a21b91cc71 Added lint rules for migrations
refs: https://github.com/TryGhost/Toolbox/issues/105

Lint rules prevent:

* Invalid naming conventions for new migrations
* Loop constructs in migrations - these should be used with caution
and are therefore a warning rule, use `// eslint-disable-next-line
no-restricted-syntax` to prevent this rule from firing where a loop is
required
* Returing within a loop - this is usually meant to be a
continue/break
* Multiple joins - these can be badly performing migrations, so should
be treated with caution, disable the rule for the line if the risk is
understood / the migration cannot be written without it
2021-11-29 16:21:43 +00:00
Hannah Wolfe
8c93bdff41
Removed unnecessary BasicErrorRenderer
refs: 4474ca1a1d
refs: 0799f02e80

The BasicErrorRenderer was created as a fallback for when we needed to not render templates, which is
chiefly when we're trying to render a 404 for an image. Using a template puts us at risk of an infinite 404 loop
if the missing image is referenced in the 404 template.

As of 0799f02e, the HTMLErrorRenderer no longer uses templates - instead we serve a very simple HTML page.
This can be used instead of the BasicErrorRenderer, as it results in a properly formatted error.
Even when sending responses in plain text, the content type is returned as HTML and therefore having an
unformatted error makes no sense - if we really need a non-html format I guess there should be no body at all.
2021-11-29 15:40:55 +00:00
Naz
fa9d2a4105 Reworked media API thumbnail endpoint
refs https://github.com/TryGhost/Toolbox/issues/120

- Having an "options" parameter in the controller definition was missleading as if the `url` or `ref` parameters were expected as a part of the qurey parameter. These variables should be provided as a part of the request body, thus having them in "data" attribute is more accurate
2021-11-29 13:12:47 +04:00
Naz
b807be9699 Added JSDocs to redirects validator
refs https://github.com/TryGhost/Toolbox/issues/139

- Makes the expected file formatting easier to reason about
2021-11-26 06:18:36 +13:00
Naz
6ed5f64f4b Moved backup path calculation outside redirects module
refs https://github.com/TryGhost/Toolbox/issues/139

- Having tight coupling with backup file path calculation for redirects makes it extremely hard to test. In addition, having it injected will make it easier to swap this dependency to the mechanism similar to one used for routes files
2021-11-26 06:18:36 +13:00
Naz
edfe81e11c Moved validation dependency to constructor param
refs https://github.com/TryGhost/Toolbox/issues/139

- Having no external dependencies makes the class way easier to test
2021-11-26 06:18:36 +13:00
Naz
e6c9bcdf22 Moved custom redirects initialization to frontend init secton
refs https://github.com/TryGhost/Toolbox/issues/139

- The custom redirects services belong in the  initServicesForFrontend because frontend depends on these to function properly. When placed in general init section the middleware would not get initialized properly before it's used by the "frontend express app"
2021-11-26 06:18:36 +13:00
Naz
fd0a0ddf47 Refactored CustomRedirectsAPI constructor
refs https://github.com/TryGhost/Toolbox/issues/139

- The pattern we use accross the codebase is a single "options" object passed into a constructor instead of passing multiple parametes. Fixed the broken pattern in CustomRedirectsAPI constructor
2021-11-26 06:18:36 +13:00
Hannah Wolfe
303ea87897
Removed overly-specific handling of handlebars errors
refs: https://github.com/TryGhost/Ghost/issues/10496
refs: https://github.com/handlebars-lang/handlebars.js/issues/1548

- For a while, handlebars would and could generate quite a lot of obscure sytnax error messages for malformed helpers that we couldn't catch in gscan
- This was fixed in https://github.com/handlebars-lang/handlebars.js/issues/1548
- We have also since added AST parsing to gscan, so if there are more obscure syntax errors, we should be able to add rules to catch them
- Therefore, this additional, overly specific code block isn't needed anymore, so let's clean it up!
2021-11-25 14:53:36 +00:00
Naz
3315ed34ae 🐛 Fixed redirects.json file corruption on upload
refs 91efa4605c

- Referenced commit introduced a double json-stringification to uploaded redirects.json files.
- The endpoint has no stability index of any sort and is meant to be dropped in Ghost v5. It's best to rework the redirects to the yaml format as descirbe here - https://ghost.org/docs/tutorials/implementing-redirects/#file-structure
2021-11-25 18:43:45 +04:00
Hannah Wolfe
621cfd9866
Moved admin redirect middleware to named file
- moving this middleware because we're about to add a second piece of middleware
- it's easier to see what we have when each middleware is in its own file rather than in one big middleware.js file
2021-11-25 14:29:17 +00:00
Fabien 'egg' O'Carroll
eace98d787
Truncated offers.name to 40 characters (#13781)
refs https://github.com/TryGhost/Team/issues/1236

We use Offer names for the Stripe Coupon name - which has a limit of 40
characters. We are now introducing a limit of 40 characters to Offer
names too. This migration ensures that all our data in the DB is valid.
2021-11-25 12:33:22 +02:00
Hannah Wolfe
5e931e2e37
Added err.context & err.help to stack traces
- When we handle errors in Ghost, we are supposed to use a pattern of supplying 3 messages:
   - message: what went wrong
   - context: details about why how or where the error happened
   - help: where the user can go to get help with this error
- We do this in many places and our JSON error handler and CLI error logging tools are designed to output this extra information
- However, stack traces, which start with message as the first line and then output the stack are totally missing this
- By injecting the additional messages into the stack once an error has been "ghostified" we should get clearer messages everywhere

Notes:
- I've additionally injected a "Stack Trace:" line that makes it easier to read the error vs the stack
- This code looks a little weird because the lines are inserted backwards, but that allows us to always to the insert at position 1 as per the comment,
   so we don't have to keep track of whether we already injected something or not
2021-11-24 13:01:06 +00:00
Hannah Wolfe
0799f02e80
Rewritten HTMLErrorRenderer w/o asset helper & template
refs: 2af9e2e12

- This new HTMLErrorRenderer is borrowed heavily from finalHandler
   - This is the module that express uses to render errors if there is no custom errorhandler
- It just renders a really simple html page wrapping err.stack in a <pre>
   - This results in a nicely formatted, but unstyled error page
- I also updated BasicErrorRenderer to use the same res.statusCode + err.stack pattern rather than err.message

Note: This error renderer is _only_ used for renderering errors on the `/ghost/` route
 - In almost all cases, errors here are rendered by Ember
 - The only error that can be rendered here is a missing template error see: 2af9e2e12
2021-11-24 13:01:06 +00:00
Hannah Wolfe
2af9e2e125
Fixed error handling for missing admin templates
- If the admin templates default.html or default-prod.html are missing, don't throw a 500
- Instead throw a well considered 400 error with extra help for what to do to fix it
2021-11-24 12:30:41 +00:00
Hannah Wolfe
0c4c9ce553
Moved JSON error handlers next to each other
- helps for comparing and contrasting
- context is largely lost on why we have these two, need to figure it out and if possible, simplify
2021-11-24 11:42:53 +00:00
Hannah Wolfe
f417c4c732
Merged our two maintenance middleware into one
- Reduced our maintenance middleware code down to the bare minimum!
  - We have an old maintenance middleware in place to handle when a site is forcibly put into maintenance mode, or the urlService hasn't finished booting
    - This maintenance middleware was mounted on every sub app, instead of globally for reasons I no longer remember
  - Recently, we introduced a new, static version of maintenence middleware to show during the boot process so we can get the server started earlier & not drop requests
    - This version has its own HTML template and doesn't depend on any of Ghost's error rendering code
  - To simplify and help with decoupling, this commit merges the two middleware, so that the new independent & static middleware renders its template for any one of the 3 possible maintenance modes
    - It only needs to exist in the top level app 🙌

TODO: move the maintenance middleware to its own file/package so it's not part of the app.js as that is weird
2021-11-24 11:27:18 +00:00
Hannah Wolfe
6247aa4d8b
Added missing async/await calls
- throughout the theme activation flow there are several missing awaits and necessary async keywords
- we should be waiting on these processes, not letting them complete indeterministically
2021-11-23 17:24:50 +00:00
Naz
6ee94f66b4 Fixed invalid settings file path configuration
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
refs e457fd5fe0 (diff-b292e8480eee007786cc602f55ed05006a06b8da9fe6934d51fbef8328013278R36)

- There were two separate instances of the SettingsPathManager in route-settings and settings-loader causing the configured paths missmatching on test environment. Because of this missmatch, uploading and resetting the routes.yaml file didn't work!
2021-11-24 05:32:20 +13:00
Naz
2fed8dbbd5 Moved all settings module bootstrap logic into init method
refs refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- The initialization outside the init method is causing problems when the configs are mocked during the test setup
2021-11-24 05:32:20 +13:00
Naz
cbec6aa49e Refactored route settings module into class
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- It's a step to making the module follow class+DI pattern before fully extracting it into an external libarary
- Reminder, doing in Ghost repo instead of substituting big chunks all at once to have clear history of how the service evolved prior to the extraction into external lib!
2021-11-24 05:32:20 +13:00
Naz
a9d742c2d3 Extracted getBackupFilePath out of route settings module
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
refs e457fd5fe0 (diff-b292e8480eee007786cc602f55ed05006a06b8da9fe6934d51fbef8328013278R36)

- The method is used in a similar fassion in redirects modules, so should be a gret target for extraction and reuse.
- Bonuses: we drop 'path' and 'moment-timezone' dependencies from route settings module!
2021-11-24 05:32:20 +13:00
Naz
6d6aa12afe Extracted routes.yaml path calculation into external package
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
refs e457fd5fe0 (diff-b292e8480eee007786cc602f55ed05006a06b8da9fe6934d51fbef8328013278R36)

- The full logic for  file path calculation for routes.yaml has been duplicated in couple modules. It is aslo following similar pattern used in redirects services and should be reused there once those modules are touched
2021-11-24 05:32:20 +13:00
Kevin Ansfield
1190ff1df0 Fixed errors being swallowed in oembed service
no issue

When switching the oembed service to async/await the error handling was not correctly refactored. `this.errorHandler(url)` was returning a curried function so it could be used as `.catch(this.errorHandler(url))` but that's not how it's being used after the async/await change meaning we were returning a function rather than the result of that function.

- `this.errorHandler(url)` is now only used in one place where `url` is available so removed the method and moved the body of the curried function inline into the `catch` handler
- added a message to the logged error so it's more clear what the log refers to
2021-11-23 10:58:52 +00:00
Sanne de Vries
969921f444 Fixed accent color background in email callout cards
Refs https://github.com/TryGhost/Team/issues/1206
2021-11-22 10:45:03 +01:00
Naz
bbc93ff996 Renamed withBackend/withFrotned to backend/frontend
refs https://github.com/TryGhost/Toolbox/issues/135

- Shorter name still makes sense and the "with" might be causing confusion understanding what's gonig on during the boot time
2021-11-22 21:56:32 +13:00
Naz
149c100b4c Fixed frontend initialization in "mock-express" tests
refs https://github.com/TryGhost/Toolbox/issues/135

- Without sensible defaults the web app was not initializing either the backend nor the frontned parts of the application. Fixed the defaults so the problem doesn't happen again and optimized mock-express-style initialization to only initialize the frontend routing
2021-11-22 21:56:32 +13:00
Naz
d0dfac438d Changed /^testing/ regexp use to startsWith method
no issue

- `startsWith` method is way easier to read and understand. also, **probably** has better performance comparing to building up a regexp and then matching
2021-11-22 21:56:32 +13:00
Naz
ae397eecdf Fixed UrlService initialization
refs https://github.com/TryGhost/Toolbox/issues/135

- When there's no instance of cache available the code to retreive it should not execut. Duh!
2021-11-22 21:56:32 +13:00
Naz
ae5dd8ad30 Fixed event listener pileup
refs https://github.com/TryGhost/Toolbox/issues/135

- When instantiating new Urls/Resources object in the UrlService's init on every test suite start it was loosing track of past events (softReset wasn't doing the cleanup properly). Not initializing new classes "fixes" the problem partially, by not loosing track of those event listeners. The real fix should be proper even listener cleanup on every soft reset!
2021-11-22 21:56:32 +13:00
Naz
34b0196c0f Added writeDisabled flag to URL file cache
refs https://github.com/TryGhost/Toolbox/issues/135

- Allows to turn off overwriting urls/resources JSON file caches on testing environment. This is needed to have predictable state when running multiple test suites that stop the Ghost process and try to persiste URL cache.
2021-11-22 21:56:32 +13:00
Naz
ee4d2dd1a8 Extracted a local file cache class for URLs
refs https://github.com/TryGhost/Toolbox/issues/135

- This extracts the file storage knowledge out of the URL Service an allows to have optional features based on the environment - for example turning off writing cache for when running tests
2021-11-22 21:56:32 +13:00
Naz
3b390639c4 Introduced booting from fixture url cache in test environment
refs https://github.com/TryGhost/Toolbox/issues/135

- To be able to reliably start ghost instance without a frontend the process needs access to urls/resources caches
- Storing the configuration in "paths" for now as there's no better place for it untill we are able to mock the content folder in pre-boot
2021-11-22 21:56:32 +13:00
Naz
c12ae81ece Introduced "withFrontend" and "withBackend" flags in boot
refs https://github.com/TryGhost/Toolbox/issues/135

- These flags are meant to control initialization of sections of the boot sequence depending on the needs - with or without bakend (API)/frontend (public handlebars site)
- Ideally these flags should not be passed deep into the components, and if the are (like in the web/parent/app case) it's a smell that we need to move things up into the boot process!
2021-11-22 21:56:32 +13:00
Hannah Wolfe
80358266b2 Added perf-improving PRAGMAs in testing mode
- These two things are meant to improve performance at the cost of reliability.
- Perfect for testing, however I think they make a minimal impact on modern SSDs :(
- Still worth a shot to see if it helps with CI
2021-11-19 10:47:21 +00:00
Daniel Lockyer
80d5f89382 Lazy loaded cheerio dependency
refs https://github.com/TryGhost/Toolbox/issues/136

- `cheerio` isn't needed during the boot but it takes time and memory to
  load the library
- this commit moves `cheerio` requires later into the code to when they
  are needed
2021-11-18 17:31:04 +01:00
Kevin Ansfield
ca26c3ab0a Added logging of unexpected errors in oembed service
no issue

- if we encounter an unexpected error whilst fetching embed details we return a generic validation error so we're not leaking any details about the URL that is being hit, however that meant the error logs were only showing validation errors making debugging difficult
- added explicit logging of the unexpected error before throwing the generic validation error
2021-11-17 13:31:20 +00:00
Peter Zimon
bbceace31c Implemented table based NFT email styles 2021-11-17 14:05:39 +01:00
Sam Lord
0692ddf57d Revert "Replace update user last seen middleware with package version"
This reverts commit 07f8c6efbb.

Revert "Remove unused unit test file"

This reverts commit f1798119d3.
2021-11-17 11:00:55 +00:00
Hannah Wolfe
332beaaf90
Moved "vhost-utils" to config helpers
- These are simple functions that get data from config in a specific format
- They are also used by the topmost part of the application
- Config helpers seems like a reasonable fit to get them out of the web folder
- Functions have also been renamed to try to get them to make more sense
2021-11-17 08:37:08 +00:00
Fabien O'Carroll
e6773d00e6 Updated NFT Embed custom provider
refs https://github.com/TryGhost/Team/issues/1211

Added ?format=json to the URL in an attempt to mitigate any issues with
weird caching and receiving HTML rather than JSON.

Used `type` in place of `card_type` to closer follow how the bookmark
card/embed works.

Removed the html & width/heihgt properties which are not needed at all.
2021-11-16 21:28:36 +02:00
Sanne de Vries
b6b024b132 Fixed missing bottom spacing on email callout cards 2021-11-16 19:19:49 +01:00
Kevin Ansfield
63b7ef7dc6 Added Tenor contentFilter setting to config
refs https://github.com/TryGhost/Team/issues/1217

- moved top-level `tenorApiKey` to `tenor:apiKey` and added `tenor:contentFilter`
- added base config to `defaults.json`
- updated `public-config.js` and API output serializer to use the new top-level `tenor` key
2021-11-16 17:42:26 +00:00
Sam Lord
a5484a0fe1 Fixed transaction logic in data importer
https://github.com/TryGhost/Toolbox/issues/130

The transaction no longer commits in the promise chain, which wasn't
valid logic for a transaction, since it is commited automatically when
the promise chain resolves, and rollsback automatically when the
promise chain rejects.

This makes code which fails during the transaction error in the right
place, instead of getting stuck here. (Especially good for writing
tests).

The tests for this code can now live in the integration folder.
2021-11-16 17:09:55 +00:00
Sam Lord
07f8c6efbb Replace update user last seen middleware with package version
no issue

Part of the goal to move these middleware packages to remove untested code from the codebase
2021-11-16 15:58:45 +00:00
Hannah Wolfe
4f9b72ff43
Renamed middlewares to middleware consistently
- This is a minor bugbare, but it will affect some configuration I'm about to do for c8
- I've been wanting to do it for ages, middleware is plural all on it's own so it's an odd affectation in our codebase
- This also only exists in 2 places, everywhere else we use "middleware"
- Sadly it did result in a lot of churn as I did a full find and replace, but consistency is king!
2021-11-16 15:51:47 +00:00