Commit Graph

742 Commits

Author SHA1 Message Date
Hannah Wolfe
f5f24d9cbc
Removed 2 test files that weren't deleted when moved
refs: 95d27e7f5

- My IDE is messing me about today, I moved these two files into test/unit/frontend/services in 95d27e7f5
2021-10-06 12:17:26 +01:00
Hannah Wolfe
9e96b04542
Moved server unit tests into the server folder
- this is a small part of a bit of cleanup of our test files
- the goal is to make the existing tests clearer with a view to making it easier to write more tests
- this makes the test structure follow the codebase structure more closely
- eventually we will colocate the tests as we break the codebase down further
2021-10-06 12:01:09 +01:00
Hannah Wolfe
95d27e7f58
Moved frontend unit tests into their own folder
- this is a small part of a bit of cleanup of our test files
- the goal is to make the existing tests clearer with a view to making it easier to write more tests
- this makes the test structure follow the codebase structure more closely
- eventually we will colocate the frontend tests with the frontend code
2021-10-06 11:58:29 +01:00
Naz
e3b56dd99f 🐛 Fixed immediately sent email when scheduling email-only post
closes https://linear.app/tryghost/issue/CORE-78/email-only-scheduling-should-work-the-same-way-as-regular-posts

- The email was going out at the moment of scheduling an email-only post instead of respecting the scheduled time.
2021-10-05 16:42:29 +02:00
Hannah Wolfe
d724c8f95d
Fixed amp not rendering content
refs: 9d7049cd3

- I missed that the amp_content helper was meant to be async when refactoring
- I have updated our proper amp acceptance tests to catch the content not rendering as this is a regression, this should definitely have been caught
- Added missing async property to amp_content helper to fix the issue
2021-10-05 12:27:31 +01:00
Hannah Wolfe
9d7049cd3f
Refactored helper registration code into a service
- The helper registration code is "framework" code and very specific
- At the moment the "theme engine" is full of lots of disparate theme related stuff
- I'm trying to make the frontend framework code clearer and also expand it to  make it more useful
- The helper system now also exposes 3 methods allowing you to register a directory, a helper or an alias
- I've updated the codebase to use these both for our core helpers and for "apps"
2021-10-05 10:04:02 +01:00
Daniel Lockyer
0ec6b425ee
Merged v4.17.1 into main
v4.17.1
2021-10-05 08:33:42 +01:00
Fabien O'Carroll
cda041d424 Moved StripeAPIService to its own service
refs https://github.com/TryGhost/Team/issues/1083

The Offers service is going to need access to the StripeAPIService too,
so we need to move it out of the @tryghost/members-api module and make
it accessible to both.
2021-10-04 19:28:19 +02:00
Naz
5066e65e03 Added validation to redirects config
refs https://linear.app/tryghost/issue/CORE-86/fix-failing-site-instance-when-redirects-file-is-invalid
refs 260a47da83

- Added validation logic to catch redirects files having invalid RegEx expressions when they are introduced into the system (on upload)
- This way the error happening in the refed commit would have not happened as the validator would not have passed it through
- Moved up the "Router" declaration in custom-redirects as it needs to happen before any other bit of logic has a chance to throw
2021-10-04 18:32:40 +02:00
Hannah Wolfe
1bbaf65a22
Removed need for index.js in frontend/helpers
- The index.js file was actually loader code
- It was mainly used by the unit tests, which needed to be rewritten to get each helper individually
2021-10-04 16:46:01 +01:00
Naz
8f5186995d Added unit test coverage for invalid redirects config
refs https://linear.app/tryghost/issue/CORE-86/fix-failing-site-instance-when-redirects-file-is-invalid
refs 260a47da83

- Refed commit was missing a unit test coverage.
- The approach here introduces a new pattern - using `supertest` in unit tests. I've found this to be the most expressive way to test an express app which receives certain middleware dynamically. Because there are very few moving parts the test is still extremely quick to run
2021-10-04 16:44:23 +02:00
Naz
260a47da83 🐛 Fixed 500 when instance has an invalid redirects config
refs https://linear.app/tryghost/issue/CORE-86/fix-failing-site-instance-when-redirects-file-is-invalid

- The site was returning a 500 for all routes whenever an invalid redirects file was uploaded.
- The cause of this issue surfacing was this change - 0962b3ed45 (diff-10784969d51d9abb9d620625a14b31661ff3f98dbfb0ea17186186a8151bacfaL35) . Previously an instance always had some sort of a working router and when the configuration failed it still functioned as an empty redirects router. After the change no router was mounted resulting in a 500
- Test cases fixing the problem will folllow
2021-10-04 15:25:37 +02:00
Hannah Wolfe
b10802f4c6
Updated fixture hash for integrity check
refs: fe2958eb73

Commit fe2958eb73 updated fixtures without updating the integrity hash - not sure how I missed this!
2021-10-04 13:47:12 +01:00
Paul Cushing
fe2958eb73
Fixed typo "in" default About page. (#13271)
- added a missing word
2021-10-04 12:18:15 +01:00
Fabien O'Carroll
58653690bd Fixed member bulk actions acceptance tests
refs https://github.com/TryGhost/Team/issues/1077
refs https://github.com/allouis/Ghost/commit/cdbccdeba

These tests were forgotten in the update to the API.
2021-10-01 14:21:14 +02:00
Rishabh Garg
c4cfd1839a
Added offers table (#13381)
refs https://github.com/TryGhost/Team/issues/1088

- adds schema for new offers table
- adds permission fixtures for new offers table
- adds migrations for new table and permissions

Co-authored-by: Fabien O'Carroll <fabien@allou.is>
2021-10-01 12:18:33 +02:00
Naz
99a2f12cb7 Simplified yaml parser to have fewer dependencies
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
refs 5715aa2155 (diff-48644be82a9b957e5e627bf7b0f2f73cdb1d63851ffad68c7c178c5886495bb8R52-R57)

- Simplified the yaml parser implementation to take in a single parameter, this move will allove to simplify the logic in the route settings + opens a door to unify handling with redirects yaml parsing!
- We loose the "filename" from the error information but that was a generic "routes.yaml" anyway and would be thrown only when somebody uploaded a routes.yaml file (no real added value).
- The debug statement should be moved to contain related filepath+other info to the calling module instead
- An additional error handler was borrowed from the redirects yaml parsing logic that was introduced in a referenced commit - it still makes sense to keep it for routes.yaml configuration
2021-09-30 17:29:43 +02:00
Naz
1ac0ba07de Extracted yarml persed dep out of settings loader
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- Moving internal dependencies to be injected through constructor DI for better testability. This is first step of few more to follow. Not doing it all at once as there's too many thing failing when doing a bulk refactor
2021-09-30 17:29:42 +02:00
Naz
96d075c47d Refactored settings loader to class
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- It's a step to making the module follow class+DI pattern before fully extracting it into an external libarary
- Reminder, doing in Ghost repo instead of substituting big chunks all at once to have clear history of how the service evolved prior to the extraction into external lib!
2021-09-30 17:29:42 +02:00
Naz
a00b994e9e Renamed route loader module to settings-loader
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- The rename was needed to make it something more generic to later fit both routes and redirects setting purposes
2021-09-30 17:29:42 +02:00
Kevin Ansfield
b8e2bb7b6d Allowed custom theme settings to be passed through preview data
refs https://github.com/TryGhost/Team/issues/1097

- added `customThemeSettingKeys` as an argument to `preview.handle()` because we can't know which keys should be allowed through up-front
- added `custom` as a supported setting in the preview header data
  - `custom` should be a JSON object containing any custom theme settings
  - we parse the object but only set properties on `@custom` that are known custom theme setting keys
  - if parsing fails or it's not an object then no custom data is set
- updated `updateLocalTemplateOptions()` to pull `.custom` off of the preview data and pass it through so it's accessible on `@custom` as an override to the saved custom data
2021-09-30 13:23:39 +01:00
Kevin Ansfield
8a17e723a1 Moved usage of preview options from globalTemplateOptions to localTemplateOptions
refs https://github.com/TryGhost/Team/issues/1097

globalTemplateOptions are supposed to be static with localTemplateOptions being merged in per-request, however the per-request preview data was being extracted and set in the global options. Comments suggest that the global data should be static and eventually updated via other means, the usage of the request object to get per-request preview data is working against that.

- adjusted the preview handler to return an object rather than changing properties by reference on a passed in object
- moved preview data fetching out of `getSiteData()` used in `updateGlobalTemplateOptions()` and into `updateLocalTemplateOptions()` so that we're not relying on the request object in `updateGlobalTemplateOptions()`
2021-09-30 12:12:55 +01:00
Kevin Ansfield
4a6bedce7b Fixed theme-engine middleware tests timing out rather than showing failed assertions
no issue

- if any of the assertions in a test failed there was no handling, instead the test just timed out with a timeout error
- wrapping the assertions in a try/catch and calling `done()` with the error object aborts the test immediately and shows a useful assertion failure message
2021-09-30 12:12:55 +01:00
Naz
d4cd1bb865 Refactored ensure settings module into a class with DI
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- Ensure settings had only one method but would benefit from class+DI pattern before extracting it into an outside module.
- The logic is now also less coupled with "routes" and single source/destination paths. It's all configureable instead and might be reused if similar pattern is needed for example with redirect settings defaults.
2021-09-29 20:56:55 +02:00
Hannah Wolfe
fd20f90cca
Divided f/e proxy into true proxy + rendering service
- The original intention of the proxy was to collect up all the requires in our helpers into one place
- This has since been expanded and used in more places, in more ways
- In hindsight there are now multiple different types of requires in the proxy:
   - One: true frontend rendering framework requires (stuff from deep inside theme-engine)
   - Two: data manipulation/sdk stuff, belongs to the frontend, ways to process API data
   - Three: actual core stuff from Ghost, that we wish wasn't here / needs to be passed in a controlled way
- This commit pulls out One into a new rendering service, so at least that stuff is managed independently
- This draws the lines clearly between what's internal to the frontend and what isn't
- It also highlights that the theme-engine needs to be divided up / refactored so that we don't have these deep requires
2021-09-29 13:10:14 +01:00
Naz
0962b3ed45 Made custom-redirects middleware testable
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
refs 7528ec8c3b

- The way the custom redirects middleware was organized made it extremely hard to unit test it (had to stub the redirects service methods etc). With a new organization it's possible to provide needed redirects configs to the method which makes the actual redirects Router logic testable and the code less coupled with redirects services
- This was meant to be an attempt to extract more of the slow redirects regression tests, which failed. Instead found this weak spot that could be improved and gained:
- shaved 4s of time as two slow regression test cases are now gone
- there's now a base to build upon when getting more coverage for the custom redirects middleware
2021-09-28 22:00:32 +02:00
Naz
dd042d69c9 Moved routing setting hash calculation to backend
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- These were the last bits that had to be moved out of the frontend doing route.yaml related operations
- Next steps will be refactoring a "bag of everything" into smaller modules and reducing dependencies where possible
2021-09-28 04:59:41 +13:00
Naz
95706aa37e Moved validate&yaml-parser modules to backend
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- These modules should be colocated along with the rest of routes.yaml related modules
- They will later be extracted into external modules
2021-09-28 04:59:41 +13:00
Naz
7a91917424 Removed use of 'routes' parameter in route settings loader
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- The only allowed route settings name is 'routes.yaml', which removes a need to parameterize the function as the location is permanent anyway
- Simplifying the function in any possible way before extracting the common bits into an external lib
2021-09-28 04:59:41 +13:00
Naz
ba964c549f Moved route settings "getter" to the backend
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- Frontend is not meant to know about the underlying source of the "routes" configuration, so any reads/edits/validations are being moved into a backend service. This should also simplify the coupling of the backend with the frontend where the latter will get a JSON blob with all needed configuration during the boot
- Nother problem the "get" method had was hiding an underlying function it was doing - reading the file from the filesystem SYNCRONOUSLY. It might be a thing we need to do during the "web" app initialization, but there's no clear need to do this in a sync fassion during the bootup for example. Also having a more explicit name should help :)
2021-09-28 04:59:41 +13:00
Naz
484bb2eea2 Moved default-routes.yaml file to backend
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
refs c1c9bf0866

- Actions logic related to file system operations (like ensuring files exist) should be done on the backend. Now the route settings initialization logic lives on the backend it makes sense to keep the file closer to the source.
- The move is the opposite to the one refed in the commit with a
difference that the file now lives in "route-settings"
2021-09-28 04:59:41 +13:00
Naz
31e9434466 Moved route settings initialization to backend
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- Actions logic related to file system operations (like ensuring files exist) should be done on the backend. The frontend will be receiving a unified JSON blob config without needing to know about filesystem
2021-09-28 04:59:41 +13:00
Naz
4a47e8d0a8 Changed settings loader module API signature
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- It was not clear from the module signature/usages that the default method is executing synchronously. The change makes it explicit. Knowing if the method is synchronous is helpful to stop possible pefr bottlenecks!
2021-09-28 04:59:41 +13:00
Naz
748ea9540c Refactored routes hash to be a "single" value
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- It's a continuation of simplifying the concept of multiple frontend settings. If there's just one file we support there should be just one hash
2021-09-28 04:59:41 +13:00
Naz
93af11bdec Simplified knowSettings usage
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- 'knowSettings' was based on a "configurable" array of settings that might be configured in Ghost. The multitude never happened! The only setting the frontend takes care of is routes.yaml file (redirects is also kind of a setting but is a separate concept for now).
- Having just one type of file to deal with allows to simplify implementation significantly, which helps before a big refactor
2021-09-28 04:59:41 +13:00
Naz
010db90a51 Removed unused getAll settings method
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- The getAll function was never used. No need to keep it around!
2021-09-28 04:59:41 +13:00
Naz
71e2a06b25 Reworked ensure-settings module to take in singular file path
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- This is a micro-step towards getting rid of multiple "knownSettings" concept. Since the introduction of an array of knowSettings there was never-ever a need to handle anything but a single `routes.yaml` file. Getting rid of this concept first to have a simpler module. Next step would be getting rid of filesystem reads/writes in the "frontend"
2021-09-28 04:59:41 +13:00
Kevin Ansfield
b6d9bad6dc
Added custom theme settings browse/edit permissions for Administrators (#13361)
refs https://github.com/TryGhost/Team/issues/1070TryGhost/Team#1070

- initial implementation will only allow browse+edit via the API
2021-09-27 09:59:09 +01:00
Kevin Ansfield
57effd9585
Added custom_theme_settings table/model (#13327)
refs https://github.com/TryGhost/Team/issues/1070

- stores values of custom theme settings
  - will be merged with full settings data parsed from themes for API output
  - will be cached and made available for lookup in themes to avoid db roundtrips
- stores type of custom theme settings so we can coerce values and know if the type has changed when syncing
- records will be synced with themes upon activation
2021-09-23 11:51:18 +01:00
Daniel Lockyer
93e4b2eafd 🔒 Fixed remote command injection when using sendmail email transport
refs https://github.com/TryGhost/Ghost/security/advisories/GHSA-wfrj-qqc2-83cm
refs https://github.com/advisories/GHSA-48ww-j4fc-435p

- a vulnerability in `nodemailer` means that the `sendmail` transport is
  vulnerable to command injection for flags passed to the `sendmail`
  binary
- updating to the latest version of Nodemailer required creating
  `@tryghost/nodemailer`, which is a wrapper around Nodemailer and
  several plugins that used to be in the core
- this commit switches to using that package, and fixes up some small
  code + test changes
2021-09-17 16:46:51 +01:00
Hannah Wolfe
7d1d6ec6eb
🐛 Fixed error in sitemap with >50k posts (#13317)
closes: CORE-34
refs: https://github.com/TryGhost/Team/issues/1044

- this is a super basic fix, it adds a max nodes concept and limits the node in each sub-sitemap to 50k by default
- this will prevent the error in google console
- a better fix is in progress, but we want to at least solve the errors ASAP
2021-09-17 11:13:42 +01:00
Fabien 'egg' O'Carroll
2dca63eae2
Added temporary database table for analytic events (#13312)
refs https://github.com/TryGhost/Team/issues/1053

This table is going to be completely deleted at some point in the
future. It serves as a persistent datastore for a spike into collection
analytic events for members. We've opted for a generic table, rather
than a table for each event, so that we can push the DB to the limit in
terms of the length of the table, and find out performance issues A$AP
2021-09-17 11:15:21 +02:00
Hannah Wolfe
ba587ba882
Fixed logic bugs in match helper (#13315)
closes: CORE-33

Two bugs:
- lodash isEmpty and handlebars util isEmpty are not the same
- I literally had the truthy and falsy cases the wrong way around 🙈

Notes:
- I have, for now, copied the isEmpty util from handlebars. It's so small it doesn't seem worth trying to require the util right now, although in future it'd be nice if that was easier to do
- Adding the management for the conditional being a SafeString allows the match helper to be a subexpression of itself, I can see this pattern being useful later in combo with the any and all helpers
2021-09-17 09:47:10 +01:00
Hannah Wolfe
38c7f42baf
Improved match helper tests (#13314)
closes: CORE-32

- The match helper is a very logical helper, designed to support many many type of comparison
- The existing test pattern had gotten long and unweildy already and I'd barely scratched the surface
- Introduce a new pattern, that allows for very quickly and easily describing template strings and what their evaluation should be
- This quickly highlights how many broken cases there are right now!

Notes:
- I could have condensed the pattern further with test groups and iterating over to create describe blocks
- I feel the current implementation is the right balance between condensed and legible to make this maintainable
- I could also have used the same pattern for the block helper, but at that point I felt it more important that the logic I was testing was clear, rather than condensed
2021-09-16 17:11:10 +01:00
Naz
4744349381 Removed method complexity in integrations API controller
refs https://github.com/TryGhost/Team/issues/694
refs https://linear.app/tryghost/issue/CORE-10/tackle-integrationsjs

- The controller code is not meant to contain complex business logic.
- Added a test case checking 'PUT' endpoint for integrations to ensure
proper 'NotFound' handling. Found that previous implemenation was
buggy - threw a 500 as 'models.Integration.NotFoundError' that was removed
in previous commit didn't catch a needed error.
2021-09-16 14:23:48 +03:00
Daniel Lockyer
d4adae775e v4.14.0
-----BEGIN PGP SIGNATURE-----
 
 iHUEABYIAB0WIQTqYa7kNs8D7Oo9dgLSEYbwtHKVrQUCYUB7mgAKCRDSEYbwtHKV
 rYTGAP9dggMBUTq6+2yLyYHChVMqLez2WS/XmgTdC4mc2tsZzgD+J2/zhRObGYX0
 d54Y39pAw7rPV8Z8md9nCm9olPpE4AM=
 =w206
 -----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
 
 iHUEABYIAB0WIQTqYa7kNs8D7Oo9dgLSEYbwtHKVrQUCYUB8kwAKCRDSEYbwtHKV
 rTGVAP4wqFwWwQUFUXX4tLbvcLKQalvHQI3soLFneAzZT1M3DQEAtWO+crkH2auN
 Agt8ND2ndlIzsyGxYywliajBfbQVZwM=
 =nFhH
 -----END PGP SIGNATURE-----

Merged v4.14.0 into main

v4.14.0
2021-09-14 11:42:21 +01:00
Kevin Ansfield
6875796417 Blocked 0.* IP addresses when making oembed requests
no issue

It was possible for authenticated/trusted admin users to make GET requests to localhost via the oembed service by crafting a redirect that used 0.0.0.0.

- added the 0.* default route/routing block to the private IP regex used to block requests when we're contacting external sites
- added an additional IP or localhost check in the oembed service when fetching bookmark card data
2021-09-14 11:35:14 +01:00
Kevin Ansfield
0294918eb9 Fixed labs service unit tests
refs d735e3a4f3
refs c7972c20a5

- the unit tests are currently tied to the labs service data rather than testing independent behaviour meaning that the test needs to be updated when the particular alpha flag they are tied to gets removed
2021-09-14 10:19:26 +01:00
Daniel Lockyer
2d639ad4a1 Replaced removed Bookshelf findWhere function
- as per https://github.com/bookshelf/bookshelf/wiki/Migrating-from-0.15.1-to-1.0.0#collectionfindwhere, the `findWhere` function was removed
- `find` can be used in combination with `matchFunc` and then checking
  the values against each other to keep the same functionality
- also updates the tests to reflect the change in number of function calls
2021-09-10 16:59:11 +01:00
Daniel Lockyer
7a3725fdbf Fixed settings tests due to Bookshelf update
- Bookshelf now seems to error out when the response is undefined and
  expects something
- this commit updates the test to the same empty response we use
  elsewhere
2021-09-10 16:59:11 +01:00