Commit Graph

397 Commits

Author SHA1 Message Date
Daniel Lockyer
93e4b2eafd 🔒 Fixed remote command injection when using sendmail email transport
refs https://github.com/TryGhost/Ghost/security/advisories/GHSA-wfrj-qqc2-83cm
refs https://github.com/advisories/GHSA-48ww-j4fc-435p

- a vulnerability in `nodemailer` means that the `sendmail` transport is
  vulnerable to command injection for flags passed to the `sendmail`
  binary
- updating to the latest version of Nodemailer required creating
  `@tryghost/nodemailer`, which is a wrapper around Nodemailer and
  several plugins that used to be in the core
- this commit switches to using that package, and fixes up some small
  code + test changes
2021-09-17 16:46:51 +01:00
Hannah Wolfe
7d1d6ec6eb
🐛 Fixed error in sitemap with >50k posts (#13317)
closes: CORE-34
refs: https://github.com/TryGhost/Team/issues/1044

- this is a super basic fix, it adds a max nodes concept and limits the node in each sub-sitemap to 50k by default
- this will prevent the error in google console
- a better fix is in progress, but we want to at least solve the errors ASAP
2021-09-17 11:13:42 +01:00
Fabien 'egg' O'Carroll
2dca63eae2
Added temporary database table for analytic events (#13312)
refs https://github.com/TryGhost/Team/issues/1053

This table is going to be completely deleted at some point in the
future. It serves as a persistent datastore for a spike into collection
analytic events for members. We've opted for a generic table, rather
than a table for each event, so that we can push the DB to the limit in
terms of the length of the table, and find out performance issues A$AP
2021-09-17 11:15:21 +02:00
Hannah Wolfe
ba587ba882
Fixed logic bugs in match helper (#13315)
closes: CORE-33

Two bugs:
- lodash isEmpty and handlebars util isEmpty are not the same
- I literally had the truthy and falsy cases the wrong way around 🙈

Notes:
- I have, for now, copied the isEmpty util from handlebars. It's so small it doesn't seem worth trying to require the util right now, although in future it'd be nice if that was easier to do
- Adding the management for the conditional being a SafeString allows the match helper to be a subexpression of itself, I can see this pattern being useful later in combo with the any and all helpers
2021-09-17 09:47:10 +01:00
Hannah Wolfe
38c7f42baf
Improved match helper tests (#13314)
closes: CORE-32

- The match helper is a very logical helper, designed to support many many type of comparison
- The existing test pattern had gotten long and unweildy already and I'd barely scratched the surface
- Introduce a new pattern, that allows for very quickly and easily describing template strings and what their evaluation should be
- This quickly highlights how many broken cases there are right now!

Notes:
- I could have condensed the pattern further with test groups and iterating over to create describe blocks
- I feel the current implementation is the right balance between condensed and legible to make this maintainable
- I could also have used the same pattern for the block helper, but at that point I felt it more important that the logic I was testing was clear, rather than condensed
2021-09-16 17:11:10 +01:00
Daniel Lockyer
d4adae775e v4.14.0
-----BEGIN PGP SIGNATURE-----
 
 iHUEABYIAB0WIQTqYa7kNs8D7Oo9dgLSEYbwtHKVrQUCYUB7mgAKCRDSEYbwtHKV
 rYTGAP9dggMBUTq6+2yLyYHChVMqLez2WS/XmgTdC4mc2tsZzgD+J2/zhRObGYX0
 d54Y39pAw7rPV8Z8md9nCm9olPpE4AM=
 =w206
 -----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
 
 iHUEABYIAB0WIQTqYa7kNs8D7Oo9dgLSEYbwtHKVrQUCYUB8kwAKCRDSEYbwtHKV
 rTGVAP4wqFwWwQUFUXX4tLbvcLKQalvHQI3soLFneAzZT1M3DQEAtWO+crkH2auN
 Agt8ND2ndlIzsyGxYywliajBfbQVZwM=
 =nFhH
 -----END PGP SIGNATURE-----

Merged v4.14.0 into main

v4.14.0
2021-09-14 11:42:21 +01:00
Kevin Ansfield
6875796417 Blocked 0.* IP addresses when making oembed requests
no issue

It was possible for authenticated/trusted admin users to make GET requests to localhost via the oembed service by crafting a redirect that used 0.0.0.0.

- added the 0.* default route/routing block to the private IP regex used to block requests when we're contacting external sites
- added an additional IP or localhost check in the oembed service when fetching bookmark card data
2021-09-14 11:35:14 +01:00
Kevin Ansfield
0294918eb9 Fixed labs service unit tests
refs d735e3a4f3
refs c7972c20a5

- the unit tests are currently tied to the labs service data rather than testing independent behaviour meaning that the test needs to be updated when the particular alpha flag they are tied to gets removed
2021-09-14 10:19:26 +01:00
Daniel Lockyer
2d639ad4a1 Replaced removed Bookshelf findWhere function
- as per https://github.com/bookshelf/bookshelf/wiki/Migrating-from-0.15.1-to-1.0.0#collectionfindwhere, the `findWhere` function was removed
- `find` can be used in combination with `matchFunc` and then checking
  the values against each other to keep the same functionality
- also updates the tests to reflect the change in number of function calls
2021-09-10 16:59:11 +01:00
Daniel Lockyer
7a3725fdbf Fixed settings tests due to Bookshelf update
- Bookshelf now seems to error out when the response is undefined and
  expects something
- this commit updates the test to the same empty response we use
  elsewhere
2021-09-10 16:59:11 +01:00
Rishabh Garg
2272f84cc9
Added beta {{products}} helper for tiers list on post (#13267)
refs https://github.com/TryGhost/Team/issues/1004

- adds new `{{products}}` helper behind `multipleProducts` flag
- `{{products}}` outputs a string with list of products that have access to specific post when used in a post context in theme
- outputs empty string when used out of a post context and without access to `visibility` property
- uses all available posts for a site via the global products data
- updates {{content}} helper cta to use this new helper to show list of tiers with access to post
2021-09-10 09:30:41 +05:30
Rishabh
c7972c20a5 Fixed alpha flag tests
refs 025eb8bd79

`mutipleProducts` has now been moved out of alpha to beta, though the alpha feature tests were still using it to verify alpha behavior. The tests are now updated to use `dashboardTwo` feature as base alpha feature instead.
2021-09-09 17:05:07 +05:30
Naz
4f78bf01a2 Moved default settings population test to regression suite
closes https://github.com/TryGhost/Team/issues/860
refs 5405b6ca7c

- The slow test was running slow because it's not a "unit test" it is testing much more. Moved it to a correct suite - regression which simplified the logic a lot (no need to mock db calls).
- Brought back the 2000ms limit as the bottleneck has been solved
2021-09-01 15:23:57 +04:00
Fabien 'egg' O'Carroll
c7a7828b57
Gave Administrators permission to connect to Stripe (#13228)
refs https://github.com/TryGhost/Team/issues/994

This adds the permission required to connect to Stripe to the
Administrator role, as required by the linked issue.
2021-08-26 11:00:40 +01:00
Fabien 'egg' O'Carroll
2f33292600
Added members_product_events table (#13236)
refs https://github.com/TryGhost/Team/issues/873

This table is to track events related to members be given or having
removed access to products. It will allow us to provide start dates for
access for complimentary members, as well as being able to track access
to products over time, either for individual members or for aggregates.
2021-08-23 16:29:15 +02:00
Naz
70b359923f Fixed failing unit test
refs 9bc5a279ca

- Adds a new post status introduced in the referenced commit.
2021-08-13 10:22:11 +04:00
Naz
755a3a320e Added email_only column to posts_meta table
closes https://github.com/TryGhost/Team/issues/893

- We need a place to store email-only flag and posts_meta is the best place for it
2021-08-05 19:44:35 +12:00
Naz
2fbc1af165 Added email disabling flag to settings records
refs https://github.com/TryGhost/Team/issues/912

- We need a place to persist the email freeze state between instance restarts - settings table record is the best place for it
2021-07-29 20:30:30 +12:00
Thibaut Patel
ba4f8cc857 Added OAuth user data
issue https://github.com/TryGhost/Team/issues/614

- This allows keeping each user oauth data, to compare on future logins
2021-07-27 17:56:07 +02:00
Kevin Ansfield
8d5e7ed695
🐛 Fixed unsubscribed members receiving email when a post is sent to all members (#13181)
refs https://github.com/TryGhost/Team/issues/935

The problem was incorrect operator precedence when multiple statements existed in the filter original filter when we transform it to enforce `subscribed:true` before sending.

- free only - subscribed:true+status:free - no issue
- paid only - subscribed:true+status:-free - no issue
- all - subscribed:true+status:-free,status:free - the ,status:free part is treated as a separate OR statement meaning the subscribed:true is not applied to it and free members that are unsubscribed will receive the email

- extracted the filter transform into a separate function so it can be unit tested
- updated the transform to use `()` for operator precedence, eg: `subscribed:true+(status:-free,status:free)`
- used transform function in `addEmail()` and `getEmailMemberRows()`
- fixed `sent/send` typo in error message
2021-07-26 12:47:03 +01:00
Naz
810b052e01 Removed use of deprecated new Error() syntax
refs 2f1123d6ca
refs 6f1a3e1774

- As per refed commits, we are removing deprecated use of `new Error()` in the codebase
- Exposed few internal from commands module methods for easier testing, otherwise it was turning into neverending mocking show
2021-07-19 21:31:31 +12:00
Naz
a266c64130 Removed use of deprecated new Error() syntax
refs 2f1123d6ca
refs 6f1a3e1774

- As per refed commits, we are removing deprecated use of `new Error()` in the codebase
2021-07-19 21:31:31 +12:00
Naz
648662af93 Reverted "Moved disguised unit tests into regression test suite"
refs bf587d4055

- The change broke regression tests and due to this change being out of
scope of the cycle dicided to rever it and increase the time limit on
unit tests instead
- We should focus on refactoring moving slow unit tests in the next
cleanup period
2021-07-19 13:22:31 +04:00
Naz
bf587d4055 Moved disguised unit tests into regression test suite
no issue

- Two affected unit tests were triggering the "slow unit test" error on CI which gave a clue they were either:
- written badly
OR
- were not unit tests at all

- The latter proved to be true and the tests were moved where they belong - into the regression test suite with much more forgiving running itme constraints.
- Unit tests should run quick ALWAYS as that's the first thing that gets triggered before pushing improvements to main. Faster - better.
2021-07-19 12:04:42 +04:00
Naz
8a219f4f72 Fixed slow-running external request test
refs https://github.com/TryGhost/Team/issues/860

- Slow unit tests cause longer waiting time to deliver code to main. Before this fix the test was taking a whooping 6s on average
- The main cause of the delay was a downstream's package (got) default retry logic that was taking up a lot of time. The test is meant to check a "500" response handling not a retry logic, so disabling it for this specific test makes sense
2021-07-15 15:21:33 +04:00
Naz
84d0815e47 Refactored scheduling default test to use nock
refs https://github.com/TryGhost/Team/issues/860

- `nock` has been used in other similar unit tests. It was proven to have easy to understand and use API. By not making up custom code through express/httpServer stuff we reduce the amount of boilerplate code significantly - makes tests a lot more readable
- Bonus, removes couplint dependency to "shared" express module
2021-07-15 14:07:11 +04:00
Naz
5ea8e9b926 Removed use of deprecated new Error() syntax
refs 2f1123d6ca
refs 6f1a3e1774

- As per refed commits, we are removing deprecated use of `new Error()` in the codebase
- This bit cleans up the rest of `new Error()` usage in MEGA service
2021-07-14 19:17:35 +04:00
Naz
b045112950 Renamed exposed _getEmailMemberRows in mega
no issue

- Exposing internal methods out of the module is a non-standard practice. Adding `_` prefix allows to signal that this method is not for general use.
- When mega is refactored into a proper class this method will become exposed anyways
2021-07-14 18:56:57 +04:00
Naz
f343e73c92 Removed use of deprecated new Error() syntax
refs 2f1123d6ca
refs 6f1a3e1774

- As per refed commits, we are removing deprecated use of `new Error()` in the codebase
- This bit cleans up `new Error()` usage in MEGA service
2021-07-14 18:44:25 +04:00
Naz
3b7042545a Fixed typo 2021-07-14 13:23:15 +04:00
Hannah Wolfe
c29c118fcf
Moved labs utlity to shared
- This isn't really a "service" - it's a set of utilities for working with labs flags
- It's also required all over the place, and doesn't require anything that isn't shared
- Therefore, it should live in shared
2021-07-08 09:05:41 +01:00
Hannah Wolfe
8d38957bd7
Revert "Moved labs utlity to shared"
This reverts commit 782de52678.
2021-07-08 07:09:13 +01:00
Hannah Wolfe
782de52678
Moved labs utlity to shared
- This isn't really a "service" - it's a set of utilities for working with labs flags
- It's also required all over the place, and doesn't require anything that isn't shared
- Therefore, it should live in shared
2021-07-07 21:41:34 +01:00
Kevin Ansfield
69bc5a9dfd Fixed error when requesting resize of a blank image
closes https://github.com/TryGhost/Team/issues/819

- adds guard for an empty buffer when reading file from storage for resizing, if a blank image is loaded then redirect to the original file
2021-07-07 19:11:24 +01:00
Naz
dfe1089ee2 Bumped default API version export to be canary/v4
refs https://github.com/TryGhost/Team/issues/856

- The default internal version of the API is expected to be the latest one available which is v4/canary at the moment.
- There will be more information posted in the referenced issue later around how to approach the "default version", for now it's just a change to make a small step into a right direction.
2021-07-07 18:44:21 +04:00
Hannah Wolfe
f08a55c21f
Renamed tests to .test.js & updated commands
refs: https://github.com/TryGhost/Team/issues/856
refs: https://github.com/TryGhost/Team/issues/756

- The .test.js extension is better than _spec.js as it's more obvious that it's an extension
- It also meaans we can use the --extension parameter in mocha, which should result in a better default behaviour for `yarn test`
- It also highlights that some of our tests were named incorrectly and were not (and still will not be) run (see https://github.com/TryGhost/Team/issues/856)
- Note: even with this change, `yarn test` is throwing errors, I believe because of this issue https://github.com/TryGhost/Team/issues/756
2021-07-06 20:45:01 +01:00
Naz
b98854150c Added test illustrating filtering of null segments
refs https://github.com/TryGhost/Team/issues/861

- The test case illustrates that it's already possible to achieve what's described in the referenced issue (bug where the content is not filtered correctly when a single email segment card is present)
- The bug fix for a general issue should be using this parser's ability to properly filter html/plaintext content
2021-07-06 17:03:46 +04:00
Naz
2c1ae2e9af 🐛 Fixed a 500 error for incorrect fields parameter in API
refs 8a1fd1f57f
refs 5584430ddc

- The change to async/await in the original commit 558443 was causing problems in downstream dependencies (create-error package) where it was loosing a context of "this". It's not a direct dependency so I didn't go yak shaving into where exacly the context is lost.
- The fix to keep a correct context of "this" was sticking to an existing pattern using regular function returning promises. Once we need to redo them into async/await we can investigate if there's a way around create-error's context prolbem
2021-07-06 11:58:37 +04:00
Rishabh
3e9a23355f Handled visibility filter in post/page API
refs https://github.com/TryGhost/Team/issues/849

As part of work for segmented post access with multiple products, the custom filter for post access is stored in `visibility` field on posts but passed with `visibility_filter` property on API. This change -

- updates input serializer of posts to transform `visibility` and `visibility_filter` properties correctly
- updates output serializer for canary to transform and send `visibility_filter` attribute with filter value
- updates output serializer for v3 to ignore any custom filter on visibility and return `paid` instead as v3 didn't have a concept of custom filter
2021-07-05 17:10:45 +05:30
Hannah Wolfe
84c2154cbe
Fixed missing renames getExcerpt -> generateExcerpt
refs: fbf0636936

- I renamed this pattern in a bunch of places, but missed a few, leaving the code messy and confusing
- This makes the naming consistent
2021-07-02 09:44:18 +01:00
Matt Hanley
4ac89c0176
Added webp image support (#13105)
refs https://github.com/TryGhost/Ghost/issues/9448
refs https://github.com/TryGhost/Ghost/issues/2591

Added API support for webp image files.
2021-07-02 09:37:54 +01:00
Naz
df5290089b Fixed lint error 2021-07-01 21:02:06 +04:00
Naz
986a7526f5 Added member partitioner based on segment
refs https://github.com/TryGhost/Team/issues/828

- Before sending out batches with members we need to partition all members based on the segment they belong to. Special segment "unsegmented" is used in case none of the segments used in the emal cards cover part of the members set (for example only free members card used when emailing all members)
2021-07-01 20:58:24 +04:00
Kevin Ansfield
1b0aa0abd8
Fixed feature_image_caption needing triple-curlies in themes (#13107)
refs https://github.com/TryGhost/Team/issues/845
refs 517d2abc5c

- updated router response formatting functions and `{{#get}}` helper response handling to make any `feature_image_caption` properties in the response a `SafeString` instance so triple-curlies are not needed when using the property in themes
2021-07-01 17:55:44 +01:00
Thibaut Patel
b94c8bcfd4 Render an email correctly according to the associated member segment
issue https://github.com/TryGhost/Team/issues/829
2021-07-01 13:36:42 +02:00
Hannah Wolfe
af4bfb8862
Changed meta filenames to correct casing
- purely for cleanliness!
- we use kebabcase by default
- tests should be kebab-case-file_spec.js (one day we want this to be .test.js)
2021-07-01 12:30:25 +01:00
Hannah Wolfe
fbf0636936
Renamed meta/excerpt meta/generate-excerpt
- This function is quite different to the others, as it generates an excerpt from HTML (truncating)
- Most functions in the meta data folder just contain content negotiation logic, like if post then feature_image else cover_image type things
- This function is more like a library and shouldn't live in Ghost, it should probably be in @tryghost/helpers
- It's definitely something we'd love to rewrite to work better tooooo
2021-07-01 11:47:29 +01:00
Hannah Wolfe
884f39d045
Moved meta + sitemap tests out of data folder
- we moved the meta folder out of data a really long time ago
- we also moved the sitemap out of data/xml into a service
- this moves the tests to roughly match
2021-06-30 17:08:28 +01:00
Hannah Wolfe
bd597db829
Moved settings/cache to shared/settings-cache
- This is part of the quest to separate the frontend and server & get rid of all the places where there are cross-requires
- At the moment the settings cache is one big shared cache used by the frontend and server liberally
- This change doesn't really solve the fundamental problems, as we still depend on events, and requires from inside frontend
- However it allows us to control the misuse slightly better by getting rid of restricted requires and turning on that eslint ruleset
2021-06-30 15:49:10 +01:00
Hannah Wolfe
3ea6df819c
Refactored SettingsCache to get events through DI
- requiring lib/common/events makes the settings cache tightly coupled to the server
- moving this up to settings index means the cache itself can be moved to a shared component/moved out of Ghost
- the index then becomes the settings manager
- questionable whether the event listeners & updater part of this shouldn't be part of a manager, independent of the actual cache 🤔
2021-06-30 15:40:41 +01:00