refs https://linear.app/tryghost/issue/CORE-84/have-a-look-at-the-eggs-redirects-refactor-branch
- The problem this change is addressing is inability to override config values once the code is extracted into a class+DI pattern
- The work around is restarting the instance with the configuration testing expected behavior - in this case missing or existing types of redirects files
no issue
The way GA flags were introduced means that they stop existing in the `'labs'` setting in the db and are instead forced to always return `true` when checking the flag in the labs service. However, Admin which uses the flags fetches them via the `/settings/` API endpoint which was only returning the raw labs setting db value meaning GA flags appeared to be disabled unless the flag had previously been enabled and no settings save had occured.
- updated the settings bread service to replace the labs setting value with the JSON stringified output of `labs.getAll()` which is the ultimate source-of-truth for a feature being enabled/disabled
- extracted `browse()` behaviour to an internal `_formatBrowse()` method so we can apply the same filtering/modification for output of `browse()` and `edit()`
Co-authored-by: Fabien O'Carroll <fabien@allou.is>
closes https://github.com/TryGhost/Team/issues/1150
Our override of the base Bookshelf `insert` operation so that our own `formatOnWrite()` method is called on attributes was working on a false assumption that an `attrs` attribute is passed in as it is for the `update` operation. Instead Bookshelf's base update uses the `model.attributes` values to create an `attrs` object that is then passed through the usual `.format()` method meaning that our `insert` override was not actually doing anything.
- added a failing regression test for the `formatOnWrite()` override behaviour
- adjusted our insert/update overrides to set an internal `_isWriting` property on the model, then if that property is true our `.format()` override (which is called by Bookshelf on a generated `attrs` object during inserts) we manually call our `.formatOnWrite()` method
- updated both overrides even though `update` was working for consistency and less cognitive overhead for reasoning between two different approaches
- These don't make sense and we're working on improving testing across the board
- We'll make sure our testing best practices are documented when they've settled
- the integrationTesting utils are specific to the express mock style of testing
- all other tests can use the url-service-utils to check the url service is finished
- done a fastest-possible overhaul on this style of tests to try to get them to work independently again
This is a pattern that was introduced a while ago to try to speed up our e2e tests and I'm not sure if it's staying or going
It uses a minimal frontend-only version of the boot process and a custom-built express testing tool
However it's really old and out of date because of the boot refactor and several changes since
This highlights the key problem with it - it doesn't rely on any of our "core" boot process, it makes it up, and therefore how reliable are these tests?
Ideally we need to get these tests working with the real boot process in some capacity
We would then need to make sure we have all the tests in e2e-frontend written in this style
- this test file uses a different pattern to the other test files
- not yet sure if the pattern is terrible or genius, need to assess before moving it into a folder full of what are meant to be exemplary tests
closes https://github.com/TryGhost/Team/issues/1125
refs 3c822e0457
- Email-only is not considered a general availability feature and can be used without special flags.
- It allows to publish a new post type "email only" that only goes out as an email newletter and is available through an undescoverable URL (does not appear anywhere publicly similarly to preview posts) on the site.
- e2e tests are tests that cover critical functionality by booting ghost
- integration tests are more like unit tests, but need to initialise and use a db
- so settings shouldn't start Ghost, url service is critical and should be in integration, and preview is critical and should be in e2e
- some tests are necessarily driven from the db
- these are like unit tests, except they only make sense if using the db - else you have to stub too much to make them worthwhile
- for these rare but important cases, we have the clear concept of integration tests
- We have a bunch of important server-related e2e tests
- Make these clear in their own folder
- "server" is everything that isn't the api or the frontend - kind of a catch-all concept
refs https://github.com/TryGhost/Team/issues/1088
- adds schema for new offers table
- adds permission fixtures for new offers table
- adds migrations for new table and permissions
Co-authored-by: Fabien O'Carroll <fabien@allou.is>
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
refs 7528ec8c3b
- The way the custom redirects middleware was organized made it extremely hard to unit test it (had to stub the redirects service methods etc). With a new organization it's possible to provide needed redirects configs to the method which makes the actual redirects Router logic testable and the code less coupled with redirects services
- This was meant to be an attempt to extract more of the slow redirects regression tests, which failed. Instead found this weak spot that could be improved and gained:
- shaved 4s of time as two slow regression test cases are now gone
- there's now a base to build upon when getting more coverage for the custom redirects middleware
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
- Frontend is not meant to know about the underlying source of the "routes" configuration, so any reads/edits/validations are being moved into a backend service. This should also simplify the coupling of the backend with the frontend where the latter will get a JSON blob with all needed configuration during the boot
- Nother problem the "get" method had was hiding an underlying function it was doing - reading the file from the filesystem SYNCRONOUSLY. It might be a thing we need to do during the "web" app initialization, but there's no clear need to do this in a sync fassion during the bootup for example. Also having a more explicit name should help :)
refs https://github.com/TryGhost/Team/issues/1070
- stores values of custom theme settings
- will be merged with full settings data parsed from themes for API output
- will be cached and made available for lookup in themes to avoid db roundtrips
- stores type of custom theme settings so we can coerce values and know if the type has changed when syncing
- records will be synced with themes upon activation
refs https://github.com/TryGhost/Team/issues/1053
This table is going to be completely deleted at some point in the
future. It serves as a persistent datastore for a spike into collection
analytic events for members. We've opted for a generic table, rather
than a table for each event, so that we can push the DB to the limit in
terms of the length of the table, and find out performance issues A$AP
closes https://github.com/TryGhost/Team/issues/860
refs 5405b6ca7c
- The slow test was running slow because it's not a "unit test" it is testing much more. Moved it to a correct suite - regression which simplified the logic a lot (no need to mock db calls).
- Brought back the 2000ms limit as the bottleneck has been solved
refs https://github.com/TryGhost/Team/issues/947
- During the work of the UI and moving `email_only` flag to publish menu it created the situation where the publishing of the post was at the same time as adding `email_only` flag, resulted in not picking up teh `sent` status as the `posts_meta` model and record were's available during save.
- Adding the incoming attribute check for email_only flag covers this situation
refs https://github.com/TryGhost/Team/issues/873
This table is to track events related to members be given or having
removed access to products. It will allow us to provide start dates for
access for complimentary members, as well as being able to track access
to products over time, either for individual members or for aggregates.
refs https://github.com/TryGhost/Team/issues/990
- Relying on uuid instead of slug makes the posts less discoverable and partially soves discoverability through overriden robots.txt files
refs https://github.com/TryGhost/Team/issues/953
- Emails posts should be not explorable by the rest of the frontend similarly to the draft or scheduled posts. Email posts should also keep the content gating, so that specific parts of content can still be gated based on the post's visibility setup
- A separate frontend router was chosen to implement this part of the system instead of a moutable express app due to increased complexity to introduce the latter approach.
- All "sent" email-only posts will be accessible through the `/email/:slug/` route
refs https://github.com/TryGhost/Team/issues/953
- We need to track email-only posts that have been sent out. New status was chosen as a way to differenciate such posts.
- Introducing a new "email post" type, conceptually like "page", was considered. Because there is no clear roadmap for "email post" becoming a bigger part of the product yet and a lot of uncertainty around this concept, overhead needed to introduce a new type was just too much to do at this moment. It's still a possibility in the future
refs https://github.com/TryGhost/Team/issues/948
- The frontend route `/email/:uuid` is aliased to the preview as a temporary solution. It fulfills the premise of the email-only post anyway - not being accessible publicly and only shared through email.
- The tests for the new route are missing as adding them was way more problematic than I envisoned. They are in the works and will be added as a follow up commit next.
refs https://github.com/TryGhost/Team/issues/948
- When working on related problem noticed these declarations were scattered all over the place in the test without special purpose. Moved them all up into a central "once per suite" stub. Not sure if this subing is working at all, but at least it will be easier to change it once per suite in the future once someone spots a problem with this approach
refs https://github.com/TryGhost/Team/issues/948
- While working on referenced issue came around adding tests to new top level route - '/email/' and it was pretty hard to navigate through single blob of tests. Broke them down by the area/route into separate suites for better mainteinability
refs https://github.com/TryGhost/Team/issues/949
- When post is marked as "email-only" we can send it out to the selected audience when publishing without making the post publicly available
- The feature is available for experimentation behind "email only" alpha flag available in labs
refs https://github.com/TryGhost/Team/issues/912
- We need a place to persist the email freeze state between instance restarts - settings table record is the best place for it
refs d60d348c88
- When the import triggers a background job the meta response should contain no data otherwise the client can mistake it for completed import
refs bf587d4055
- The change broke regression tests and due to this change being out of
scope of the cycle dicided to rever it and increase the time limit on
unit tests instead
- We should focus on refactoring moving slow unit tests in the next
cleanup period
no issue
- Two affected unit tests were triggering the "slow unit test" error on CI which gave a clue they were either:
- written badly
OR
- were not unit tests at all
- The latter proved to be true and the tests were moved where they belong - into the regression test suite with much more forgiving running itme constraints.
- Unit tests should run quick ALWAYS as that's the first thing that gets triggered before pushing improvements to main. Faster - better.
- This isn't really a "service" - it's a set of utilities for working with labs flags
- It's also required all over the place, and doesn't require anything that isn't shared
- Therefore, it should live in shared
- This isn't really a "service" - it's a set of utilities for working with labs flags
- It's also required all over the place, and doesn't require anything that isn't shared
- Therefore, it should live in shared
issue https://github.com/TryGhost/Team/issues/859
- Added invalidation to PUT /authentication/setup
- Added invalidation to POST /db
- Added invalidation to DELETE /db
- Added invalidation to GET /slugs/:type/:name
- Removed invalidation from PUT /users/:id/token
refs 8a1fd1f57f
refs 5584430ddc
- The change to async/await in the original commit 558443 was causing problems in downstream dependencies (create-error package) where it was loosing a context of "this". It's not a direct dependency so I didn't go yak shaving into where exacly the context is lost.
- The fix to keep a correct context of "this" was sticking to an existing pattern using regular function returning promises. Once we need to redo them into async/await we can investigate if there's a way around create-error's context prolbem
This commit achieves a few things:
- ☑️ No longer having to remember whether a command is yarn something or grunt something
- ☑️ Simplification of tools hopefully making them easier to remember and use
- ☑️ Complete removal of the need for grunt from our test tooling
Several of the tools still use grunt under the hood, but the **entrypoint** should aways be `yarn xxx`.
- `grunt main` -> `yarn main`
- `grunt dev` -> `yarn dev`
- `grunt build` -> `yarn build`
- `grunt test:file-or-folder` -> `yarn test file-or-folder`
- `grunt test-unit` -> `yarn test:unit`
- `grunt test-acceptance` -> `yarn test:acceptance`
- `grunt test-regression` -> `yarn test:regression`
- `grunt validate` -> removed due to lack of use
There is now also `yarn test:all` to run all 3 classes of tests
This PR also reorders & restructures the Gruntfile extensively so that:
- The remaining useful commands are all at the top of the file
- Config and other blah happens after all the useful commands
- All release-only config happens in the release task at the very end of the file
---
DONE:
* Removed all references to npm/bower
* Removed all references to lint / deprecated command
* Moved debug to yarn dev:debug
* Removed all references to travis
* Removed broken help task + useless comment
* Removed unused knex-migrator and clean:test setup tasks
* Added new test commands, removed grunt validate
* Moved stubClientFiles to test utility and use in the few tests that need it
* Used mocha in yarn directly except grunt test:x
* Swapped grunt test for yarn test
* extensive cleanup and reshuffling
closes https://github.com/TryGhost/Team/issues/817
refs 6d083ee00e/packages/bookshelf-pagination/lib/bookshelf-pagination.js (L256)
- The 500 error is not the best we can do in this situation and throwing a 400 just like we doo in a referenced commit would keep the convention
- The underlying problem of the bug is bigger - we allow the fields named the same way as relations to leak into the db query and that causes an incorrect SQL syntax. It's a bigger problem which would need a separate, holistic approach
- This is part of the quest to separate the frontend and server & get rid of all the places where there are cross-requires
- At the moment the settings cache is one big shared cache used by the frontend and server liberally
- This change doesn't really solve the fundamental problems, as we still depend on events, and requires from inside frontend
- However it allows us to control the misuse slightly better by getting rid of restricted requires and turning on that eslint ruleset
refs https://github.com/TryGhost/Team/issues/754
refs a7dec233ba
- Additional validation protects from problems like the ones in refed commit from even getting through to the database.
- At the moment only used notificatons and couple more settings to ensure they are arrays when passed into the API. This is to avoid making big change in settings straight away - this is a problematic area which needs cautious approach.
- Ideally in the future the list of settings to check the "array" type (and other types) should be automatically generated based on the default-settings.json (or whatever way we define settings in the db a that moment)
- There's an ugly code-tripplication going on in this change. This is a separate topic that will be addressed once we work on API cleanup.
refs https://github.com/TryGhost/Team/issues/804
The associative table is used to implement the many-to-many relationship
between Products and Benefits. The `sort_order` column is needed because
a product's benefits should be orderable by an admin.
refs https://github.com/TryGhost/Team/issues/804
Benefits are tag-like resources which will be associated with Products.
The first iteration just requires a name for the benefit, which will be
stored as plaintext.
issue https://github.com/TryGhost/Team/issues/750
- Only accessible by admins
- Resets all staff users' passwords and prevents them to log-in
- Sends them a reset email password to give them back access to their account
- Closes all existing staff user sessions
refs https://github.com/TryGhost/Team/issues/793
New settings added for newsletter customisation options:
- `newsletter_header_image` - `null/"$url"`
- `newsletter_show_header_icon` - `"true/false"`
- `newsletter_show_header_title` - `"true/false"`
- `newsletter_title_alignment` - `"center/left"`
- `newsletter_title_font_category` - `"serif/sans_serif"`
- `newsletter_show_feature_image` - `"true/false"`
`newsletter_show_header` has been dropped because the same functionality can be achieved by setting both `newsletter_show_header_icon` and `newsletter_show_header_title` to `false`
---
- migration to convert and delete `newsletter_show_header` setting
- removed `newsletter_show_header` from default settings to ensure it doesn't get re-created
- replaced main labs template and template settings generation with the labs template
- deleted labs template
refs https://github.com/TryGhost/Team/issues/768
- `portal_products` stores list of products available in Portal
- adds new `portal_products` setting to default settings
- adds migration to populate `portal_products` with current product so its available by default
- update tests
- Part of the effort to split Ghost down into smaller, decoupled pieces
- Moved out our internal validator tooling to a separate library
- Replaced all usage of our own tooling and validatorjs directly with @tryghost/validator
- Removed the validatorjs dependency and removed the renovate pin
- This gives us a consistant, smaller, clearer public API for validations
- It will eventually be used on Ghost Admin too
- This way we can start getting up to date with validator whilst not increasing build size
refs https://github.com/TryGhost/Team/issues/770
We want post feature image functionality to better match what's available inside the editor, to do that we'll need somewhere to store alt and caption meta data. `posts_meta` chosen because even though we want to make this generic for other tables in the future those tables also have a `feature_image` (or closely related) field.
- updated schema with new columns
- added migration to create columns
- cleaned new columns from API output
- not output on v2/v3
- conditionally output on v4/canary output based on labs flag
- bumped `@tryghost/admin-api-schema` to allow new columns through in canary API requests
- silently clean properties from input when labs flag is disabled
- updated acceptance tests so they fail if `admin-api-schema` is not letting the new fields through
no issue
Shows impact of new code behind labs flags through the existing acceptance/regression tests. Allows for existing tests to be updated to match new behaviour rather than requiring separate tests where individual flags are enabled. Should result in minimal test updating once code reaches GA.
- adds a forced `'labs:enabled'` fixture op that edits the `labs` setting to enable all flags then restarts the settings service to pick up the new setting
- modifies labs service to not remove ALPHA_FEATURE labs settings when running in a testing environment
refs https://github.com/TryGhost/Team/issues/757
- The "type" value in settings is meant to be representing the data type stored in the "value" field. It was an overlooked bug in v4 API adding a mapper to group->type
refs https://github.com/TryGhost/Team/issues/757
- Groups are meant to represent the group to which the settings key belongs and have the same value as stored in the DB. This was an evolution of v2/v3 API's "type"
- Having a fixed value for groups in a test will make it easier to track the "type" change that is about to land
refs https://github.com/TryGhost/Team/issues/757
- There is no usecase for editing "labs" settings outside of canary/v4 API versions. Removing support for older versions makes the supported API surface smaller (easy maintenance).
refs https://github.com/TryGhost/Team/issues/757
- To safeguard from mise of a very permissing "object" value of the "labs" setting this change introduces an "allowlist" approach to filtering unrecognized labs flags
- Should allow maintainers to have a clear view of which labs flags are currently in use and manage them accordingly
* 🐛 Fixed saving Members with Complimentary plans
refs https://github.com/TryGhost/Team/issues/758
Since 4.6 The Admin is using the comped flag again, rather than creating
subscriptions for zero-amount prices directly. With the `comped` flag
removed, the default state was for it to be falsy in the Admin, and when
saved would trigger the legacy comped flow, cancelling the subscription.
This reverts commit 57a176ff3d.
refs https://github.com/TryGhost/Team/issues/728
- This is the end of the refactor for update check tests. There are no dependencies for Ghost server process to run this test suite!
- Next steps will be extracting the service to an outside module
refs https://github.com/TryGhost/Team/issues/728
- Rewire is usually a sign of massive module that is impossible to unit test without mocking big parts of that module. With a refactored UpdateCheckService and a brand new test coverage there is no need to use rewire and leave the "smell"
refs https://github.com/TryGhost/Team/issues/728
- This is a continuation of the test coverage for the UpdateCheckService.
- Covers scpecial cases of notification processing within Update Check
- The refactor inside the update check service was a convenience to get rid or the Bluebird dependency completely. Also, some minor preventative code added to avoid errors from referencing undefined objects
refs https://github.com/TryGhost/Team/issues/728
- This is a continuation of the test coverage for the UpdateCheckService.
- Covers a basic check for what kind of data is sent from the service to external analytics
refs https://github.com/TryGhost/Team/issues/728
- This approach is moving away from stubbing and "rewiring" massive internal modules and instead only stubs needed dependencies for the UpdateCheckService class
- Also eliminates the need to depend on the database fixtures which should speed up test execution and move it to proper "unit test" class when the test rewrite is complete
- starting and stopping Ghost is part of our rather unloved acceptance test framework
- moving them into their own file to make the different pieces clearer and also to start to make improvements
- first improvement had to happen as an aside - exposing the existingData property via a function and making the API clearer
- this was a weird thing set on module.exports, very hidden and hard to follow
- Note: stopGhost is only used once in the regression/modles/model_posts_spec.js file to make the test run fast enough...
- These tests are effectively "DB integration" tests e.g. non-unit tests because they do use the DB, they need their own framework
- These tests protect some advanced, yet important behaviour around URLs
- Moved them out of regression tests into acceptance, as we're slowly trying to clean up regression tests :)
- Checking for differences in behaviour when urls are configured with or without slashes should fall to unit testing
- Generally testing that a correctly configured subdirectory site does what's expected is an acceptance test
- Ghost's 301 to trailing slash behaviour is also well tested, and doesn't need extensive acceptance tests when using a subdir
- Instead, test that the non-subdirectory equivalent route is 404ing as expected, to check that routes are ONLY mounted on the subdir, as well as mounted at all
refs https://github.com/TryGhost/Team/issues/726
- When UpdateCheck service sends a notification with "type: 'alert'" an email goes out to admin users with the "message" content of the notification.
- This functionality is aimed to handling critical messages like urgent instance updates
- Next step will be getting as much of the update check code extracted into a "service" and then moved out of Ghost's codebase
refs: f1be3418d9
- Since I refactored the boot process & subsequently the test tools, I have been hitting an issue where subdirectory tests don't work if the app has already been started without a subdirectory
- Turns out it's due to the rootApp getting cached, and not replaced no matter what you do, even though if you debug it _looks_ like it's been replaced
- This makes sense as the rootApp is in a separate file and therefore subject to the node module cache and we're using const everywhere
- Therefore, I have added a single line to the test utils to destroy this cache for this file, and the tests now work perfectly
- Next: refactor the test utilities some more to make all this much, much more straightforward and easy to understand
no-issue
These tests were skipped out due to adding with the subscriptions
relation not working. However this functionality is redundant, as it is
not used anywhere. The tests have been updated to test that we can read
the subscriptions relation correctly.
no-issue
We no longer want to include skipped tests in the codebase. These tests
were added as placeholders until a point at which we were able to mock
Stripe.
We haven't got time during cleanup to mock Stripe so removing these
tests for now.
refs https://github.com/TryGhost/Team/issues/694
- Unskipping the test does not seem to be causing a failure any longer. Removed the skip as it would be causing linting errors as we tighten up the `.skip` rules
refs https://github.com/TryGhost/Team/issues/694
- The previous `.then` chaining was outdated, while in this part of code did a tiny cleanup which should improve future maintenance slightly
refs https://github.com/TryGhost/Team/issues/694
- The previous `.then` chaining was outdated and was causing 2x ghost instance initialization per test suite
- With a refactor there's only one intance initialization per suite (saves running time!) and we use more readable async/await syntax, which should make things more maintainable
refs https://github.com/TryGhost/Team/issues/694
- The tests were skipped and were throwing an eslint warning, which soon will become an error.
- They were skipped initially as they used an outdated authorization mechanism. With migration to token based auth the tests are green again
no issue
- The test was skipped, which should not have been commited into the trunk in the first place. The checks were reworked to reflect the reality - the endpoint ignores the "group" qurey parameter
no-issue
This protects our tests against changes to the database schema, which
helps us decouple the API from the database, and make tests less
brittle. It also forces us to manually update the tests if we do make a
change to the API!
closes https://github.com/TryGhost/Team/issues/699
With custom products, saving a member with subscriptions on member detail page in Admin throws errors on console, though the save is successful. This breaks the Admin as user needs to refresh the screen again to get rid of error. This change -
- updates the response on member save to return `price` object in subscription
- updates tests
no refs
Since backend now allows multiple prices but we want the prices to be currently limited to monthly/yearly on UI, we need new settings to store the current monthly/yearly price by the site owner. These settings determine the active prices shown in Admin / Portal for the site till we allow all custom products/prices again.
refs https://github.com/TryGhost/Team/issues/687
- The frontmatter field has leaked into the API layer unintentionally when it was introduced into the DB schema during 4.0 release.
- The fix add the field to "trim" list in all API. A proper validation and handling will be add per API as usecase for the field becomes clear
refs https://github.com/TryGhost/Team/issues/687
- The approach of generating validation properties using `/server/data/schema` package's tables object is prone to leaking unwanted database fields into API responses
- This refactor takes a tiny step into direction of relying on "allowlist" approach for properties in the API response resources.
- Apart from solving the described property leak problem it also moves toward decoupling tests from `/core/server` dependencies!
refs https://github.com/TryGhost/Team/issues/581
closes https://github.com/TryGhost/Team/issues/582
Emails can now be sent to members with specific associated labels or products by specifying an NQL string. We want to bring the same members segment feature to content by allowing `visibility` to be an NQL filter string on top of the `public/members/paid` special-case strings.
As an example it's possible to set `posts.visibility` to `label:vip` to make a post available only to those members with the `vip` label.
- removed enum validations for `visibility` so it now accepts any string or `null`
- bumped `@tryghost/admin-api-schema` for API-level validation changes
- added nql validation to API input validators by running the visibility query against the members model
- added transform of NQL to special-case visibility values when saving post model
- ensures there's a single way of representing "members" and "paid" where NQL gives multiple ways of representing the same segment
- useful for keeping theme-level checks such as `{{#has visibility="paid"}}` working as expected
- updated content-gating to parse nql from post's visibility and use it to query the currently logged in member to see if there's a match
- bumped @tryghost/members-api to include label and product data when loading member
