Commit Graph

726 Commits

Author SHA1 Message Date
Naz
99a2f12cb7 Simplified yaml parser to have fewer dependencies
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
refs 5715aa2155 (diff-48644be82a9b957e5e627bf7b0f2f73cdb1d63851ffad68c7c178c5886495bb8R52-R57)

- Simplified the yaml parser implementation to take in a single parameter, this move will allove to simplify the logic in the route settings + opens a door to unify handling with redirects yaml parsing!
- We loose the "filename" from the error information but that was a generic "routes.yaml" anyway and would be thrown only when somebody uploaded a routes.yaml file (no real added value).
- The debug statement should be moved to contain related filepath+other info to the calling module instead
- An additional error handler was borrowed from the redirects yaml parsing logic that was introduced in a referenced commit - it still makes sense to keep it for routes.yaml configuration
2021-09-30 17:29:43 +02:00
Naz
1ac0ba07de Extracted yarml persed dep out of settings loader
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- Moving internal dependencies to be injected through constructor DI for better testability. This is first step of few more to follow. Not doing it all at once as there's too many thing failing when doing a bulk refactor
2021-09-30 17:29:42 +02:00
Naz
96d075c47d Refactored settings loader to class
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- It's a step to making the module follow class+DI pattern before fully extracting it into an external libarary
- Reminder, doing in Ghost repo instead of substituting big chunks all at once to have clear history of how the service evolved prior to the extraction into external lib!
2021-09-30 17:29:42 +02:00
Naz
a00b994e9e Renamed route loader module to settings-loader
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- The rename was needed to make it something more generic to later fit both routes and redirects setting purposes
2021-09-30 17:29:42 +02:00
Kevin Ansfield
b8e2bb7b6d Allowed custom theme settings to be passed through preview data
refs https://github.com/TryGhost/Team/issues/1097

- added `customThemeSettingKeys` as an argument to `preview.handle()` because we can't know which keys should be allowed through up-front
- added `custom` as a supported setting in the preview header data
  - `custom` should be a JSON object containing any custom theme settings
  - we parse the object but only set properties on `@custom` that are known custom theme setting keys
  - if parsing fails or it's not an object then no custom data is set
- updated `updateLocalTemplateOptions()` to pull `.custom` off of the preview data and pass it through so it's accessible on `@custom` as an override to the saved custom data
2021-09-30 13:23:39 +01:00
Kevin Ansfield
8a17e723a1 Moved usage of preview options from globalTemplateOptions to localTemplateOptions
refs https://github.com/TryGhost/Team/issues/1097

globalTemplateOptions are supposed to be static with localTemplateOptions being merged in per-request, however the per-request preview data was being extracted and set in the global options. Comments suggest that the global data should be static and eventually updated via other means, the usage of the request object to get per-request preview data is working against that.

- adjusted the preview handler to return an object rather than changing properties by reference on a passed in object
- moved preview data fetching out of `getSiteData()` used in `updateGlobalTemplateOptions()` and into `updateLocalTemplateOptions()` so that we're not relying on the request object in `updateGlobalTemplateOptions()`
2021-09-30 12:12:55 +01:00
Kevin Ansfield
4a6bedce7b Fixed theme-engine middleware tests timing out rather than showing failed assertions
no issue

- if any of the assertions in a test failed there was no handling, instead the test just timed out with a timeout error
- wrapping the assertions in a try/catch and calling `done()` with the error object aborts the test immediately and shows a useful assertion failure message
2021-09-30 12:12:55 +01:00
Naz
d4cd1bb865 Refactored ensure settings module into a class with DI
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- Ensure settings had only one method but would benefit from class+DI pattern before extracting it into an outside module.
- The logic is now also less coupled with "routes" and single source/destination paths. It's all configureable instead and might be reused if similar pattern is needed for example with redirect settings defaults.
2021-09-29 20:56:55 +02:00
Hannah Wolfe
fd20f90cca
Divided f/e proxy into true proxy + rendering service
- The original intention of the proxy was to collect up all the requires in our helpers into one place
- This has since been expanded and used in more places, in more ways
- In hindsight there are now multiple different types of requires in the proxy:
   - One: true frontend rendering framework requires (stuff from deep inside theme-engine)
   - Two: data manipulation/sdk stuff, belongs to the frontend, ways to process API data
   - Three: actual core stuff from Ghost, that we wish wasn't here / needs to be passed in a controlled way
- This commit pulls out One into a new rendering service, so at least that stuff is managed independently
- This draws the lines clearly between what's internal to the frontend and what isn't
- It also highlights that the theme-engine needs to be divided up / refactored so that we don't have these deep requires
2021-09-29 13:10:14 +01:00
Naz
0962b3ed45 Made custom-redirects middleware testable
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
refs 7528ec8c3b

- The way the custom redirects middleware was organized made it extremely hard to unit test it (had to stub the redirects service methods etc). With a new organization it's possible to provide needed redirects configs to the method which makes the actual redirects Router logic testable and the code less coupled with redirects services
- This was meant to be an attempt to extract more of the slow redirects regression tests, which failed. Instead found this weak spot that could be improved and gained:
- shaved 4s of time as two slow regression test cases are now gone
- there's now a base to build upon when getting more coverage for the custom redirects middleware
2021-09-28 22:00:32 +02:00
Naz
dd042d69c9 Moved routing setting hash calculation to backend
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- These were the last bits that had to be moved out of the frontend doing route.yaml related operations
- Next steps will be refactoring a "bag of everything" into smaller modules and reducing dependencies where possible
2021-09-28 04:59:41 +13:00
Naz
95706aa37e Moved validate&yaml-parser modules to backend
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- These modules should be colocated along with the rest of routes.yaml related modules
- They will later be extracted into external modules
2021-09-28 04:59:41 +13:00
Naz
7a91917424 Removed use of 'routes' parameter in route settings loader
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- The only allowed route settings name is 'routes.yaml', which removes a need to parameterize the function as the location is permanent anyway
- Simplifying the function in any possible way before extracting the common bits into an external lib
2021-09-28 04:59:41 +13:00
Naz
ba964c549f Moved route settings "getter" to the backend
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- Frontend is not meant to know about the underlying source of the "routes" configuration, so any reads/edits/validations are being moved into a backend service. This should also simplify the coupling of the backend with the frontend where the latter will get a JSON blob with all needed configuration during the boot
- Nother problem the "get" method had was hiding an underlying function it was doing - reading the file from the filesystem SYNCRONOUSLY. It might be a thing we need to do during the "web" app initialization, but there's no clear need to do this in a sync fassion during the bootup for example. Also having a more explicit name should help :)
2021-09-28 04:59:41 +13:00
Naz
484bb2eea2 Moved default-routes.yaml file to backend
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
refs c1c9bf0866

- Actions logic related to file system operations (like ensuring files exist) should be done on the backend. Now the route settings initialization logic lives on the backend it makes sense to keep the file closer to the source.
- The move is the opposite to the one refed in the commit with a
difference that the file now lives in "route-settings"
2021-09-28 04:59:41 +13:00
Naz
31e9434466 Moved route settings initialization to backend
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- Actions logic related to file system operations (like ensuring files exist) should be done on the backend. The frontend will be receiving a unified JSON blob config without needing to know about filesystem
2021-09-28 04:59:41 +13:00
Naz
4a47e8d0a8 Changed settings loader module API signature
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- It was not clear from the module signature/usages that the default method is executing synchronously. The change makes it explicit. Knowing if the method is synchronous is helpful to stop possible pefr bottlenecks!
2021-09-28 04:59:41 +13:00
Naz
748ea9540c Refactored routes hash to be a "single" value
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- It's a continuation of simplifying the concept of multiple frontend settings. If there's just one file we support there should be just one hash
2021-09-28 04:59:41 +13:00
Naz
93af11bdec Simplified knowSettings usage
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- 'knowSettings' was based on a "configurable" array of settings that might be configured in Ghost. The multitude never happened! The only setting the frontend takes care of is routes.yaml file (redirects is also kind of a setting but is a separate concept for now).
- Having just one type of file to deal with allows to simplify implementation significantly, which helps before a big refactor
2021-09-28 04:59:41 +13:00
Naz
010db90a51 Removed unused getAll settings method
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- The getAll function was never used. No need to keep it around!
2021-09-28 04:59:41 +13:00
Naz
71e2a06b25 Reworked ensure-settings module to take in singular file path
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings

- This is a micro-step towards getting rid of multiple "knownSettings" concept. Since the introduction of an array of knowSettings there was never-ever a need to handle anything but a single `routes.yaml` file. Getting rid of this concept first to have a simpler module. Next step would be getting rid of filesystem reads/writes in the "frontend"
2021-09-28 04:59:41 +13:00
Kevin Ansfield
b6d9bad6dc
Added custom theme settings browse/edit permissions for Administrators (#13361)
refs https://github.com/TryGhost/Team/issues/1070TryGhost/Team#1070

- initial implementation will only allow browse+edit via the API
2021-09-27 09:59:09 +01:00
Kevin Ansfield
57effd9585
Added custom_theme_settings table/model (#13327)
refs https://github.com/TryGhost/Team/issues/1070

- stores values of custom theme settings
  - will be merged with full settings data parsed from themes for API output
  - will be cached and made available for lookup in themes to avoid db roundtrips
- stores type of custom theme settings so we can coerce values and know if the type has changed when syncing
- records will be synced with themes upon activation
2021-09-23 11:51:18 +01:00
Daniel Lockyer
93e4b2eafd 🔒 Fixed remote command injection when using sendmail email transport
refs https://github.com/TryGhost/Ghost/security/advisories/GHSA-wfrj-qqc2-83cm
refs https://github.com/advisories/GHSA-48ww-j4fc-435p

- a vulnerability in `nodemailer` means that the `sendmail` transport is
  vulnerable to command injection for flags passed to the `sendmail`
  binary
- updating to the latest version of Nodemailer required creating
  `@tryghost/nodemailer`, which is a wrapper around Nodemailer and
  several plugins that used to be in the core
- this commit switches to using that package, and fixes up some small
  code + test changes
2021-09-17 16:46:51 +01:00
Hannah Wolfe
7d1d6ec6eb
🐛 Fixed error in sitemap with >50k posts (#13317)
closes: CORE-34
refs: https://github.com/TryGhost/Team/issues/1044

- this is a super basic fix, it adds a max nodes concept and limits the node in each sub-sitemap to 50k by default
- this will prevent the error in google console
- a better fix is in progress, but we want to at least solve the errors ASAP
2021-09-17 11:13:42 +01:00
Fabien 'egg' O'Carroll
2dca63eae2
Added temporary database table for analytic events (#13312)
refs https://github.com/TryGhost/Team/issues/1053

This table is going to be completely deleted at some point in the
future. It serves as a persistent datastore for a spike into collection
analytic events for members. We've opted for a generic table, rather
than a table for each event, so that we can push the DB to the limit in
terms of the length of the table, and find out performance issues A$AP
2021-09-17 11:15:21 +02:00
Hannah Wolfe
ba587ba882
Fixed logic bugs in match helper (#13315)
closes: CORE-33

Two bugs:
- lodash isEmpty and handlebars util isEmpty are not the same
- I literally had the truthy and falsy cases the wrong way around 🙈

Notes:
- I have, for now, copied the isEmpty util from handlebars. It's so small it doesn't seem worth trying to require the util right now, although in future it'd be nice if that was easier to do
- Adding the management for the conditional being a SafeString allows the match helper to be a subexpression of itself, I can see this pattern being useful later in combo with the any and all helpers
2021-09-17 09:47:10 +01:00
Hannah Wolfe
38c7f42baf
Improved match helper tests (#13314)
closes: CORE-32

- The match helper is a very logical helper, designed to support many many type of comparison
- The existing test pattern had gotten long and unweildy already and I'd barely scratched the surface
- Introduce a new pattern, that allows for very quickly and easily describing template strings and what their evaluation should be
- This quickly highlights how many broken cases there are right now!

Notes:
- I could have condensed the pattern further with test groups and iterating over to create describe blocks
- I feel the current implementation is the right balance between condensed and legible to make this maintainable
- I could also have used the same pattern for the block helper, but at that point I felt it more important that the logic I was testing was clear, rather than condensed
2021-09-16 17:11:10 +01:00
Naz
4744349381 Removed method complexity in integrations API controller
refs https://github.com/TryGhost/Team/issues/694
refs https://linear.app/tryghost/issue/CORE-10/tackle-integrationsjs

- The controller code is not meant to contain complex business logic.
- Added a test case checking 'PUT' endpoint for integrations to ensure
proper 'NotFound' handling. Found that previous implemenation was
buggy - threw a 500 as 'models.Integration.NotFoundError' that was removed
in previous commit didn't catch a needed error.
2021-09-16 14:23:48 +03:00
Daniel Lockyer
d4adae775e v4.14.0
-----BEGIN PGP SIGNATURE-----
 
 iHUEABYIAB0WIQTqYa7kNs8D7Oo9dgLSEYbwtHKVrQUCYUB7mgAKCRDSEYbwtHKV
 rYTGAP9dggMBUTq6+2yLyYHChVMqLez2WS/XmgTdC4mc2tsZzgD+J2/zhRObGYX0
 d54Y39pAw7rPV8Z8md9nCm9olPpE4AM=
 =w206
 -----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
 
 iHUEABYIAB0WIQTqYa7kNs8D7Oo9dgLSEYbwtHKVrQUCYUB8kwAKCRDSEYbwtHKV
 rTGVAP4wqFwWwQUFUXX4tLbvcLKQalvHQI3soLFneAzZT1M3DQEAtWO+crkH2auN
 Agt8ND2ndlIzsyGxYywliajBfbQVZwM=
 =nFhH
 -----END PGP SIGNATURE-----

Merged v4.14.0 into main

v4.14.0
2021-09-14 11:42:21 +01:00
Kevin Ansfield
6875796417 Blocked 0.* IP addresses when making oembed requests
no issue

It was possible for authenticated/trusted admin users to make GET requests to localhost via the oembed service by crafting a redirect that used 0.0.0.0.

- added the 0.* default route/routing block to the private IP regex used to block requests when we're contacting external sites
- added an additional IP or localhost check in the oembed service when fetching bookmark card data
2021-09-14 11:35:14 +01:00
Kevin Ansfield
0294918eb9 Fixed labs service unit tests
refs d735e3a4f3
refs c7972c20a5

- the unit tests are currently tied to the labs service data rather than testing independent behaviour meaning that the test needs to be updated when the particular alpha flag they are tied to gets removed
2021-09-14 10:19:26 +01:00
Daniel Lockyer
2d639ad4a1 Replaced removed Bookshelf findWhere function
- as per https://github.com/bookshelf/bookshelf/wiki/Migrating-from-0.15.1-to-1.0.0#collectionfindwhere, the `findWhere` function was removed
- `find` can be used in combination with `matchFunc` and then checking
  the values against each other to keep the same functionality
- also updates the tests to reflect the change in number of function calls
2021-09-10 16:59:11 +01:00
Daniel Lockyer
7a3725fdbf Fixed settings tests due to Bookshelf update
- Bookshelf now seems to error out when the response is undefined and
  expects something
- this commit updates the test to the same empty response we use
  elsewhere
2021-09-10 16:59:11 +01:00
Rishabh Garg
2272f84cc9
Added beta {{products}} helper for tiers list on post (#13267)
refs https://github.com/TryGhost/Team/issues/1004

- adds new `{{products}}` helper behind `multipleProducts` flag
- `{{products}}` outputs a string with list of products that have access to specific post when used in a post context in theme
- outputs empty string when used out of a post context and without access to `visibility` property
- uses all available posts for a site via the global products data
- updates {{content}} helper cta to use this new helper to show list of tiers with access to post
2021-09-10 09:30:41 +05:30
Rishabh
c7972c20a5 Fixed alpha flag tests
refs 025eb8bd79

`mutipleProducts` has now been moved out of alpha to beta, though the alpha feature tests were still using it to verify alpha behavior. The tests are now updated to use `dashboardTwo` feature as base alpha feature instead.
2021-09-09 17:05:07 +05:30
Naz
4f78bf01a2 Moved default settings population test to regression suite
closes https://github.com/TryGhost/Team/issues/860
refs 5405b6ca7c

- The slow test was running slow because it's not a "unit test" it is testing much more. Moved it to a correct suite - regression which simplified the logic a lot (no need to mock db calls).
- Brought back the 2000ms limit as the bottleneck has been solved
2021-09-01 15:23:57 +04:00
Naz
35e23636ae Fixed 'sent' status setting when publishing a post
refs https://github.com/TryGhost/Team/issues/947

- During the work of the UI and moving `email_only` flag to publish menu it created the situation where the publishing of the post was at the same time as adding `email_only` flag, resulted in not picking up teh `sent` status as the `posts_meta` model and record were's available during save.
- Adding the incoming attribute check for email_only flag covers this situation
2021-08-26 22:25:45 +04:00
Fabien 'egg' O'Carroll
9a1417c8b9
Exposed dummy subscriptions to theme layer (#13257)
refs https://github.com/TryGhost/Team/issues/986

This updates the @tryghost/members-api module to return the full member
object from getMemberIdentityData, which is used to populate req.member
used by themes to construct the `@member` template data.

The full object is read from the service which handles all additional
properties and logic for retrieving members, including the dummy
subscriptions for comped members.
2021-08-26 16:03:32 +01:00
Fabien 'egg' O'Carroll
c7a7828b57
Gave Administrators permission to connect to Stripe (#13228)
refs https://github.com/TryGhost/Team/issues/994

This adds the permission required to connect to Stripe to the
Administrator role, as required by the linked issue.
2021-08-26 11:00:40 +01:00
Fabien O'Carroll
e71114bb8f Added Members bulk actions endpoint
refs https://github.com/TryGhost/Team/issues/946

This adds the initial bulk actions endpoint used for the members
filtering feature. The idea is to eventually move bulk destroy into this
endpoint to and provide a consistent interface for applying bulk actions
to members.

The @tryghost/members-api package has been bumped to include the new
bulkEdit method.

The sinon.restore in tests was moved to an afterEach so that stubs did
not effect other tests.
2021-08-23 16:38:21 +02:00
Fabien 'egg' O'Carroll
2f33292600
Added members_product_events table (#13236)
refs https://github.com/TryGhost/Team/issues/873

This table is to track events related to members be given or having
removed access to products. It will allow us to provide start dates for
access for complimentary members, as well as being able to track access
to products over time, either for individual members or for aggregates.
2021-08-23 16:29:15 +02:00
Fabien 'egg' O'Carroll
26c3e77640
Ordered Products by their monthly price by default (#13234)
refs https://github.com/TryGhost/Team/issues/714

In order to order products by their monthly price we need to apply a
join with the stripe_prices table when querying so we have access to the
amount column of stripe_prices.

As this ordering is core to how the tiers feature is intended to work,
we have added it as the default order. But this can be overriden by
manually passing the order option.

Also ensured that we do not create duplicate products in test fixtures
2021-08-20 14:46:06 +02:00
Naz
937d9e58d1 Switched post routing to uuid instead of slug
refs https://github.com/TryGhost/Team/issues/990

- Relying on uuid instead of slug makes the posts less discoverable and partially soves discoverability through overriden robots.txt files
2021-08-19 12:27:45 +04:00
Naz
74280cfbea Added "email post" frontend routing
refs https://github.com/TryGhost/Team/issues/953

- Emails posts should be not explorable by the rest of the frontend similarly to the draft or scheduled posts. Email posts should also keep the content gating, so that specific parts of content can still be gated based on the post's visibility setup
- A separate frontend router was chosen to implement this part of the system instead of a moutable express app due to increased complexity to introduce the latter approach.
- All "sent" email-only posts will be accessible through the `/email/:slug/` route
2021-08-13 12:09:11 +04:00
Naz
70b359923f Fixed failing unit test
refs 9bc5a279ca

- Adds a new post status introduced in the referenced commit.
2021-08-13 10:22:11 +04:00
Naz
9bc5a279ca Added a 'sent' status to post model
refs https://github.com/TryGhost/Team/issues/953

- We need to track email-only posts that have been sent out. New status was chosen as a way to differenciate such posts.
- Introducing a new "email post" type, conceptually like "page", was considered. Because there is no clear roadmap for "email post" becoming a bigger part of the product yet and a lot of uncertainty around this concept, overhead needed to introduce a new type was just too much to do at this moment. It's still a possibility in the future
2021-08-13 10:06:15 +04:00
Fabien 'egg' O'Carroll
6dba643ef9
Added ability to filter members on subscription data (#13214)
refs https://github.com/TryGhost/Team/issues/944

This will allow us to filter for members which have a canceled
subscription or for members which are currently on trial.
2021-08-12 10:10:51 +01:00
Naz
01f8737d39 Added /email/ route to robots.txt
closes https://github.com/TryGhost/Team/issues/952

- The `/email/` route will be a home for email only posts. We are adding the route preemptively to have the crowlers update their caches before the feature sees the light of The Internet
2021-08-10 13:45:53 +04:00
Naz
5967c080dc Covered current format of robots.txt with a test
closes https://github.com/TryGhost/Team/issues/952

- The format and routes used in default robots.txt is publicly tocumented and changing it can possibly cause breaking the users' usecases.
- When it's changed one should remember to update https://ghost.org/help/modifying-robots-txt/ and have a warning in the release notes describing the change
2021-08-10 13:27:08 +04:00