refs https://github.com/TryGhost/Team/issues/1097
globalTemplateOptions are supposed to be static with localTemplateOptions being merged in per-request, however the per-request preview data was being extracted and set in the global options. Comments suggest that the global data should be static and eventually updated via other means, the usage of the request object to get per-request preview data is working against that.
- adjusted the preview handler to return an object rather than changing properties by reference on a passed in object
- moved preview data fetching out of `getSiteData()` used in `updateGlobalTemplateOptions()` and into `updateLocalTemplateOptions()` so that we're not relying on the request object in `updateGlobalTemplateOptions()`
no issue
- if any of the assertions in a test failed there was no handling, instead the test just timed out with a timeout error
- wrapping the assertions in a try/catch and calling `done()` with the error object aborts the test immediately and shows a useful assertion failure message
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
- Ensure settings had only one method but would benefit from class+DI pattern before extracting it into an outside module.
- The logic is now also less coupled with "routes" and single source/destination paths. It's all configureable instead and might be reused if similar pattern is needed for example with redirect settings defaults.
- The original intention of the proxy was to collect up all the requires in our helpers into one place
- This has since been expanded and used in more places, in more ways
- In hindsight there are now multiple different types of requires in the proxy:
- One: true frontend rendering framework requires (stuff from deep inside theme-engine)
- Two: data manipulation/sdk stuff, belongs to the frontend, ways to process API data
- Three: actual core stuff from Ghost, that we wish wasn't here / needs to be passed in a controlled way
- This commit pulls out One into a new rendering service, so at least that stuff is managed independently
- This draws the lines clearly between what's internal to the frontend and what isn't
- It also highlights that the theme-engine needs to be divided up / refactored so that we don't have these deep requires
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
refs 7528ec8c3b
- The way the custom redirects middleware was organized made it extremely hard to unit test it (had to stub the redirects service methods etc). With a new organization it's possible to provide needed redirects configs to the method which makes the actual redirects Router logic testable and the code less coupled with redirects services
- This was meant to be an attempt to extract more of the slow redirects regression tests, which failed. Instead found this weak spot that could be improved and gained:
- shaved 4s of time as two slow regression test cases are now gone
- there's now a base to build upon when getting more coverage for the custom redirects middleware
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
- The only allowed route settings name is 'routes.yaml', which removes a need to parameterize the function as the location is permanent anyway
- Simplifying the function in any possible way before extracting the common bits into an external lib
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
- Frontend is not meant to know about the underlying source of the "routes" configuration, so any reads/edits/validations are being moved into a backend service. This should also simplify the coupling of the backend with the frontend where the latter will get a JSON blob with all needed configuration during the boot
- Nother problem the "get" method had was hiding an underlying function it was doing - reading the file from the filesystem SYNCRONOUSLY. It might be a thing we need to do during the "web" app initialization, but there's no clear need to do this in a sync fassion during the bootup for example. Also having a more explicit name should help :)
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
refs c1c9bf0866
- Actions logic related to file system operations (like ensuring files exist) should be done on the backend. Now the route settings initialization logic lives on the backend it makes sense to keep the file closer to the source.
- The move is the opposite to the one refed in the commit with a
difference that the file now lives in "route-settings"
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
- 'knowSettings' was based on a "configurable" array of settings that might be configured in Ghost. The multitude never happened! The only setting the frontend takes care of is routes.yaml file (redirects is also kind of a setting but is a separate concept for now).
- Having just one type of file to deal with allows to simplify implementation significantly, which helps before a big refactor
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
- This is a micro-step towards getting rid of multiple "knownSettings" concept. Since the introduction of an array of knowSettings there was never-ever a need to handle anything but a single `routes.yaml` file. Getting rid of this concept first to have a simpler module. Next step would be getting rid of filesystem reads/writes in the "frontend"
refs https://github.com/TryGhost/Team/issues/1070
- stores values of custom theme settings
- will be merged with full settings data parsed from themes for API output
- will be cached and made available for lookup in themes to avoid db roundtrips
- stores type of custom theme settings so we can coerce values and know if the type has changed when syncing
- records will be synced with themes upon activation
refs https://github.com/TryGhost/Ghost/security/advisories/GHSA-wfrj-qqc2-83cm
refs https://github.com/advisories/GHSA-48ww-j4fc-435p
- a vulnerability in `nodemailer` means that the `sendmail` transport is
vulnerable to command injection for flags passed to the `sendmail`
binary
- updating to the latest version of Nodemailer required creating
`@tryghost/nodemailer`, which is a wrapper around Nodemailer and
several plugins that used to be in the core
- this commit switches to using that package, and fixes up some small
code + test changes
closes: CORE-34
refs: https://github.com/TryGhost/Team/issues/1044
- this is a super basic fix, it adds a max nodes concept and limits the node in each sub-sitemap to 50k by default
- this will prevent the error in google console
- a better fix is in progress, but we want to at least solve the errors ASAP
refs https://github.com/TryGhost/Team/issues/1053
This table is going to be completely deleted at some point in the
future. It serves as a persistent datastore for a spike into collection
analytic events for members. We've opted for a generic table, rather
than a table for each event, so that we can push the DB to the limit in
terms of the length of the table, and find out performance issues A$AP
closes: CORE-33
Two bugs:
- lodash isEmpty and handlebars util isEmpty are not the same
- I literally had the truthy and falsy cases the wrong way around 🙈
Notes:
- I have, for now, copied the isEmpty util from handlebars. It's so small it doesn't seem worth trying to require the util right now, although in future it'd be nice if that was easier to do
- Adding the management for the conditional being a SafeString allows the match helper to be a subexpression of itself, I can see this pattern being useful later in combo with the any and all helpers
closes: CORE-32
- The match helper is a very logical helper, designed to support many many type of comparison
- The existing test pattern had gotten long and unweildy already and I'd barely scratched the surface
- Introduce a new pattern, that allows for very quickly and easily describing template strings and what their evaluation should be
- This quickly highlights how many broken cases there are right now!
Notes:
- I could have condensed the pattern further with test groups and iterating over to create describe blocks
- I feel the current implementation is the right balance between condensed and legible to make this maintainable
- I could also have used the same pattern for the block helper, but at that point I felt it more important that the logic I was testing was clear, rather than condensed
refs https://github.com/TryGhost/Team/issues/694
refs https://linear.app/tryghost/issue/CORE-10/tackle-integrationsjs
- The controller code is not meant to contain complex business logic.
- Added a test case checking 'PUT' endpoint for integrations to ensure
proper 'NotFound' handling. Found that previous implemenation was
buggy - threw a 500 as 'models.Integration.NotFoundError' that was removed
in previous commit didn't catch a needed error.
no issue
It was possible for authenticated/trusted admin users to make GET requests to localhost via the oembed service by crafting a redirect that used 0.0.0.0.
- added the 0.* default route/routing block to the private IP regex used to block requests when we're contacting external sites
- added an additional IP or localhost check in the oembed service when fetching bookmark card data
refs d735e3a4f3
refs c7972c20a5
- the unit tests are currently tied to the labs service data rather than testing independent behaviour meaning that the test needs to be updated when the particular alpha flag they are tied to gets removed
- Bookshelf now seems to error out when the response is undefined and
expects something
- this commit updates the test to the same empty response we use
elsewhere
refs https://github.com/TryGhost/Team/issues/1004
- adds new `{{products}}` helper behind `multipleProducts` flag
- `{{products}}` outputs a string with list of products that have access to specific post when used in a post context in theme
- outputs empty string when used out of a post context and without access to `visibility` property
- uses all available posts for a site via the global products data
- updates {{content}} helper cta to use this new helper to show list of tiers with access to post
refs 025eb8bd79
`mutipleProducts` has now been moved out of alpha to beta, though the alpha feature tests were still using it to verify alpha behavior. The tests are now updated to use `dashboardTwo` feature as base alpha feature instead.
closes https://github.com/TryGhost/Team/issues/860
refs 5405b6ca7c
- The slow test was running slow because it's not a "unit test" it is testing much more. Moved it to a correct suite - regression which simplified the logic a lot (no need to mock db calls).
- Brought back the 2000ms limit as the bottleneck has been solved
refs https://github.com/TryGhost/Team/issues/947
- During the work of the UI and moving `email_only` flag to publish menu it created the situation where the publishing of the post was at the same time as adding `email_only` flag, resulted in not picking up teh `sent` status as the `posts_meta` model and record were's available during save.
- Adding the incoming attribute check for email_only flag covers this situation
refs https://github.com/TryGhost/Team/issues/986
This updates the @tryghost/members-api module to return the full member
object from getMemberIdentityData, which is used to populate req.member
used by themes to construct the `@member` template data.
The full object is read from the service which handles all additional
properties and logic for retrieving members, including the dummy
subscriptions for comped members.
refs https://github.com/TryGhost/Team/issues/946
This adds the initial bulk actions endpoint used for the members
filtering feature. The idea is to eventually move bulk destroy into this
endpoint to and provide a consistent interface for applying bulk actions
to members.
The @tryghost/members-api package has been bumped to include the new
bulkEdit method.
The sinon.restore in tests was moved to an afterEach so that stubs did
not effect other tests.
refs https://github.com/TryGhost/Team/issues/873
This table is to track events related to members be given or having
removed access to products. It will allow us to provide start dates for
access for complimentary members, as well as being able to track access
to products over time, either for individual members or for aggregates.
refs https://github.com/TryGhost/Team/issues/714
In order to order products by their monthly price we need to apply a
join with the stripe_prices table when querying so we have access to the
amount column of stripe_prices.
As this ordering is core to how the tiers feature is intended to work,
we have added it as the default order. But this can be overriden by
manually passing the order option.
Also ensured that we do not create duplicate products in test fixtures
refs https://github.com/TryGhost/Team/issues/990
- Relying on uuid instead of slug makes the posts less discoverable and partially soves discoverability through overriden robots.txt files
refs https://github.com/TryGhost/Team/issues/953
- Emails posts should be not explorable by the rest of the frontend similarly to the draft or scheduled posts. Email posts should also keep the content gating, so that specific parts of content can still be gated based on the post's visibility setup
- A separate frontend router was chosen to implement this part of the system instead of a moutable express app due to increased complexity to introduce the latter approach.
- All "sent" email-only posts will be accessible through the `/email/:slug/` route
refs https://github.com/TryGhost/Team/issues/953
- We need to track email-only posts that have been sent out. New status was chosen as a way to differenciate such posts.
- Introducing a new "email post" type, conceptually like "page", was considered. Because there is no clear roadmap for "email post" becoming a bigger part of the product yet and a lot of uncertainty around this concept, overhead needed to introduce a new type was just too much to do at this moment. It's still a possibility in the future
closes https://github.com/TryGhost/Team/issues/952
- The `/email/` route will be a home for email only posts. We are adding the route preemptively to have the crowlers update their caches before the feature sees the light of The Internet
refs https://github.com/TryGhost/Team/issues/948
- The frontend route `/email/:uuid` is aliased to the preview as a temporary solution. It fulfills the premise of the email-only post anyway - not being accessible publicly and only shared through email.
- The tests for the new route are missing as adding them was way more problematic than I envisoned. They are in the works and will be added as a follow up commit next.
refs https://github.com/TryGhost/Team/issues/948
- When working on related problem noticed these declarations were scattered all over the place in the test without special purpose. Moved them all up into a central "once per suite" stub. Not sure if this subing is working at all, but at least it will be easier to change it once per suite in the future once someone spots a problem with this approach
refs https://github.com/TryGhost/Team/issues/948
- While working on referenced issue came around adding tests to new top level route - '/email/' and it was pretty hard to navigate through single blob of tests. Broke them down by the area/route into separate suites for better mainteinability
refs https://github.com/TryGhost/Team/issues/949
- When post is marked as "email-only" we can send it out to the selected audience when publishing without making the post publicly available
- The feature is available for experimentation behind "email only" alpha flag available in labs
refs https://github.com/TryGhost/Team/issues/912
- We need a place to persist the email freeze state between instance restarts - settings table record is the best place for it
refs https://github.com/TryGhost/Team/issues/935
The problem was incorrect operator precedence when multiple statements existed in the filter original filter when we transform it to enforce `subscribed:true` before sending.
- free only - subscribed:true+status:free - no issue
- paid only - subscribed:true+status:-free - no issue
- all - subscribed:true+status:-free,status:free - the ,status:free part is treated as a separate OR statement meaning the subscribed:true is not applied to it and free members that are unsubscribed will receive the email
- extracted the filter transform into a separate function so it can be unit tested
- updated the transform to use `()` for operator precedence, eg: `subscribed:true+(status:-free,status:free)`
- used transform function in `addEmail()` and `getEmailMemberRows()`
- fixed `sent/send` typo in error message
refs d60d348c88
- When the import triggers a background job the meta response should contain no data otherwise the client can mistake it for completed import
refs 2f1123d6ca
refs 6f1a3e1774
- As per refed commits, we are removing deprecated use of `new Error()` in the codebase
- Exposed few internal from commands module methods for easier testing, otherwise it was turning into neverending mocking show
refs bf587d4055
- The change broke regression tests and due to this change being out of
scope of the cycle dicided to rever it and increase the time limit on
unit tests instead
- We should focus on refactoring moving slow unit tests in the next
cleanup period
no issue
- Two affected unit tests were triggering the "slow unit test" error on CI which gave a clue they were either:
- written badly
OR
- were not unit tests at all
- The latter proved to be true and the tests were moved where they belong - into the regression test suite with much more forgiving running itme constraints.
- Unit tests should run quick ALWAYS as that's the first thing that gets triggered before pushing improvements to main. Faster - better.
refs https://github.com/TryGhost/Team/issues/542
The members-api has been updated to use the passed `created_at`
property when creating members for the status change events, as well as
using the subscription start_date for members paid subscription events
This is a duplicate of 7eb821b3f5 but
with updated dependencies.
refs https://github.com/TryGhost/Team/issues/860
- Slow unit tests cause longer waiting time to deliver code to main. Before this fix the test was taking a whooping 6s on average
- The main cause of the delay was a downstream's package (got) default retry logic that was taking up a lot of time. The test is meant to check a "500" response handling not a retry logic, so disabling it for this specific test makes sense
refs https://github.com/TryGhost/Team/issues/860
- `nock` has been used in other similar unit tests. It was proven to have easy to understand and use API. By not making up custom code through express/httpServer stuff we reduce the amount of boilerplate code significantly - makes tests a lot more readable
- Bonus, removes couplint dependency to "shared" express module
refs 2f1123d6ca
refs 6f1a3e1774
- As per refed commits, we are removing deprecated use of `new Error()` in the codebase
- This bit cleans up the rest of `new Error()` usage in MEGA service
no issue
- Exposing internal methods out of the module is a non-standard practice. Adding `_` prefix allows to signal that this method is not for general use.
- When mega is refactored into a proper class this method will become exposed anyways
refs 2f1123d6ca
refs 6f1a3e1774
- As per refed commits, we are removing deprecated use of `new Error()` in the codebase
- This bit cleans up `new Error()` usage in MEGA service
refs: f9a3f7d955
- The test for overriding a theme (uploading a theme with the same name as the currently active theme) doesn't test the right codepath
- It incorrectly assumes uploading the same theme twice results in an override, but this is only true for the active theme
- This change splits the override test out into it's own test, and only tests overriding by changing the active theme first
- Also fixed a minor comment type whilst here
closes https://github.com/TryGhost/Team/issues/607
- patch was added in `bookshelf-include-count@0.1.3` to fix member counts for labels
- bumps `bookshelf-plugins` to include the patch
- This isn't really a "service" - it's a set of utilities for working with labs flags
- It's also required all over the place, and doesn't require anything that isn't shared
- Therefore, it should live in shared
- This isn't really a "service" - it's a set of utilities for working with labs flags
- It's also required all over the place, and doesn't require anything that isn't shared
- Therefore, it should live in shared
closes https://github.com/TryGhost/Team/issues/819
- adds guard for an empty buffer when reading file from storage for resizing, if a blank image is loaded then redirect to the original file
issue https://github.com/TryGhost/Team/issues/859
- Added invalidation to PUT /authentication/setup
- Added invalidation to POST /db
- Added invalidation to DELETE /db
- Added invalidation to GET /slugs/:type/:name
- Removed invalidation from PUT /users/:id/token
refs https://github.com/TryGhost/Team/issues/856
- The default internal version of the API is expected to be the latest one available which is v4/canary at the moment.
- There will be more information posted in the referenced issue later around how to approach the "default version", for now it's just a change to make a small step into a right direction.
refs https://github.com/TryGhost/Team/issues/856
- These tests were created during work on v4 without a particular goal behind them. Because they don't do anything extra comparing to the existing tests there is no sense to keep them around
- In an ideal world, our acceptance tests would be much, much faster
- ATM we output how much time is spent on reloading Ghost for each suite, but this is output with console
- Changed this to use DEBUG, so we don't clutter the UI normally
- Added further debug statements, and a cumulative time, so we can see where time is spent/wasted
- Added a DEBUG command for running acceptance tests with this output
- This shows us that reloading Ghost accounts for 50% of the test time
- this was skipped on boot, but then called in the test utilities, but only on restart 🙈
- this means that yarn test:acceptance (i.e. running all tests) works, but if you try to run just test/api-acceptance/themes_spec.js it would fail because that uses a fresh boot not a restart/reload
- I've changed this as keeping the test using the real boot, rather than the made-up acceptance-utils tasks as much as possible is way better
refs https://github.com/TryGhost/Team/issues/861
- The test case illustrates that it's already possible to achieve what's described in the referenced issue (bug where the content is not filtered correctly when a single email segment card is present)
- The bug fix for a general issue should be using this parser's ability to properly filter html/plaintext content
refs 8a1fd1f57f
refs 5584430ddc
- The change to async/await in the original commit 558443 was causing problems in downstream dependencies (create-error package) where it was loosing a context of "this". It's not a direct dependency so I didn't go yak shaving into where exacly the context is lost.
- The fix to keep a correct context of "this" was sticking to an existing pattern using regular function returning promises. Once we need to redo them into async/await we can investigate if there's a way around create-error's context prolbem
This commit achieves a few things:
- ☑️ No longer having to remember whether a command is yarn something or grunt something
- ☑️ Simplification of tools hopefully making them easier to remember and use
- ☑️ Complete removal of the need for grunt from our test tooling
Several of the tools still use grunt under the hood, but the **entrypoint** should aways be `yarn xxx`.
- `grunt main` -> `yarn main`
- `grunt dev` -> `yarn dev`
- `grunt build` -> `yarn build`
- `grunt test:file-or-folder` -> `yarn test file-or-folder`
- `grunt test-unit` -> `yarn test:unit`
- `grunt test-acceptance` -> `yarn test:acceptance`
- `grunt test-regression` -> `yarn test:regression`
- `grunt validate` -> removed due to lack of use
There is now also `yarn test:all` to run all 3 classes of tests
This PR also reorders & restructures the Gruntfile extensively so that:
- The remaining useful commands are all at the top of the file
- Config and other blah happens after all the useful commands
- All release-only config happens in the release task at the very end of the file
---
DONE:
* Removed all references to npm/bower
* Removed all references to lint / deprecated command
* Moved debug to yarn dev:debug
* Removed all references to travis
* Removed broken help task + useless comment
* Removed unused knex-migrator and clean:test setup tasks
* Added new test commands, removed grunt validate
* Moved stubClientFiles to test utility and use in the few tests that need it
* Used mocha in yarn directly except grunt test:x
* Swapped grunt test for yarn test
* extensive cleanup and reshuffling
closes https://github.com/TryGhost/Team/issues/817
refs 6d083ee00e/packages/bookshelf-pagination/lib/bookshelf-pagination.js (L256)
- The 500 error is not the best we can do in this situation and throwing a 400 just like we doo in a referenced commit would keep the convention
- The underlying problem of the bug is bigger - we allow the fields named the same way as relations to leak into the db query and that causes an incorrect SQL syntax. It's a bigger problem which would need a separate, holistic approach
refs https://github.com/TryGhost/Team/issues/849
As part of work for segmented post access with multiple products, the custom filter for post access is stored in `visibility` field on posts but passed with `visibility_filter` property on API. This change -
- updates input serializer of posts to transform `visibility` and `visibility_filter` properties correctly
- updates output serializer for canary to transform and send `visibility_filter` attribute with filter value
- updates output serializer for v3 to ignore any custom filter on visibility and return `paid` instead as v3 didn't have a concept of custom filter
refs: fbf0636936
- I renamed this pattern in a bunch of places, but missed a few, leaving the code messy and confusing
- This makes the naming consistent
refs https://github.com/TryGhost/Team/issues/828
- Before sending out batches with members we need to partition all members based on the segment they belong to. Special segment "unsegmented" is used in case none of the segments used in the emal cards cover part of the members set (for example only free members card used when emailing all members)
refs https://github.com/TryGhost/Team/issues/845
refs 517d2abc5c
- updated router response formatting functions and `{{#get}}` helper response handling to make any `feature_image_caption` properties in the response a `SafeString` instance so triple-curlies are not needed when using the property in themes
- This function is quite different to the others, as it generates an excerpt from HTML (truncating)
- Most functions in the meta data folder just contain content negotiation logic, like if post then feature_image else cover_image type things
- This function is more like a library and shouldn't live in Ghost, it should probably be in @tryghost/helpers
- It's definitely something we'd love to rewrite to work better tooooo
- we moved the meta folder out of data a really long time ago
- we also moved the sitemap out of data/xml into a service
- this moves the tests to roughly match
- This is part of the quest to separate the frontend and server & get rid of all the places where there are cross-requires
- At the moment the settings cache is one big shared cache used by the frontend and server liberally
- This change doesn't really solve the fundamental problems, as we still depend on events, and requires from inside frontend
- However it allows us to control the misuse slightly better by getting rid of restricted requires and turning on that eslint ruleset
