refs https://github.com/TryGhost/Toolbox/issues/139
- The regression test suite for redirects functionality for way too big. And each restart was causing massive overhead. It's enough to have a single exhaustive test using multiple input files
- The tests testing API endpoints should've been e2e tests to start with
- The rest is covered in the unit tests for redirects api service
refs https://github.com/TryGhost/Toolbox/issues/139
- Having tight coupling with backup file path calculation for redirects makes it extremely hard to test. In addition, having it injected will make it easier to swap this dependency to the mechanism similar to one used for routes files
refs 91efa4605c
- Referenced commit introduced a double json-stringification to uploaded redirects.json files.
- The endpoint has no stability index of any sort and is meant to be dropped in Ghost v5. It's best to rework the redirects to the yaml format as descirbe here - https://ghost.org/docs/tutorials/implementing-redirects/#file-structure
- moving this middleware because we're about to add a second piece of middleware
- it's easier to see what we have when each middleware is in its own file rather than in one big middleware.js file
no issue
- this commit adds a counter for the number of boots we do in tests
- which therefore allows us to calculate the average boot time we
experience
- only useful for debugging test performance
refs https://github.com/TryGhost/Toolbox/issues/139
- The v2 and v3 redirects APIs are unofficial and should not be used by anyone in production. There's no good reason to maintain expensive to run test suites for old unofficial APIs.
- The test cases in canary suite covers the functionality of redirects enough to be sure they work as expeted
refs https://github.com/TryGhost/Toolbox/issues/138
- These tests still rely on the frontend to be present. Needs further investigation to remove "frontend: true" flag - it slows down test runs!
refs https://github.com/TryGhost/Toolbox/issues/138
- Final batch of the refactor to async/await syntax. Doing these refactors before modifying "testUtils.startGhost" everywhere to boot only with the backend
refs https://github.com/TryGhost/Toolbox/issues/138
- This is a continuation of a bigger refactor to use async/await syntax before migrating "startGhost" methods to only use backend boot
- Removed a little bit of dead code (like admin user creation) which should speed up test execution too!
- Refactored user variables to be declared closer to their usecases instead of being high up in a global scope - variables shoul not live that far apart from the code that uses them
refs https://github.com/TryGhost/Toolbox/issues/138
- First batch of the refactor to async/await syntax. Next one will cover the rest. Doing these refactors before modifying "testUtils.startGhost" everywhere to boot only with the backend
refs https://github.com/TryGhost/Toolbox/issues/138
- There is no good reason to keep this extra variable around just call "stop" in couple very specific cases. Even for those cases, there's `testUtils.stopGhost` method which achieves the same without additional variable to track.
refs https://github.com/TryGhost/Toolbox/issues/138
- Having the "ghost" alias only added cognitive load when reading through the test code and didn't provide any additional value. Removed the pattern to keep things simpler and more explicit
refs https://github.com/TryGhost/Toolbox/issues/138
- Using asycn/await syntax is way more readable and allows to identify further reusable patterns in test initialization. This refactor also served as an exploreation around how the code looks like at this point
refs: 2af9e2e12
- This new HTMLErrorRenderer is borrowed heavily from finalHandler
- This is the module that express uses to render errors if there is no custom errorhandler
- It just renders a really simple html page wrapping err.stack in a <pre>
- This results in a nicely formatted, but unstyled error page
- I also updated BasicErrorRenderer to use the same res.statusCode + err.stack pattern rather than err.message
Note: This error renderer is _only_ used for renderering errors on the `/ghost/` route
- In almost all cases, errors here are rendered by Ember
- The only error that can be rendered here is a missing template error see: 2af9e2e12
- Card asset reloading was incorrectly only happening if the API version changed 🙈
- In addition, having an init function was redundant, as theme activation happens on boot
- This meant that the card assets were being generated twice on boot
- Instead, we now only generate them on theme activation, which covers the boot case and simplifies all the logic
- Currently it's assumed that public files are 100% static
- With card assets, we're using it for files that are partially static, but can change between reboots and theme changes
- We already have a system for managing cache busting across theme changes and restarts - the ?v= key that is added via the asset helper
- This was already in place and used, but servePublicFile's internal cache didn't honor this key, and cached for the lifetime of boot
- This small change means that if a ?v= query param is present on a request for a public file, that we pay attention to it. Else we cache as before
refs https://github.com/TryGhost/Toolbox/issues/135
- Looking closer into the reason why the test was failing without "forceStart" revealed that the server only start was overoptimized - "initServicesForFrontend" should be a part of a backend as those are generic theme services.
no issue
- Having rewire here doens't do any difference and should not be used if absolutely needed. Usually using rewire gives a code "smell" so there's some sort of coupling that's going on and probably has to be addressed first
refs https://github.com/TryGhost/Toolbox/issues/135
- The reason the test **settings** test was failing when the force start flag was removed in the **custom themes** was the bridge! The bridge was trying to execute function on the frontend when the boot was done without initializing the frontend. The setting test was changing locale and the timezone which triggered events calling up on frontend components - we clearly don't want to do this when the instance is booted without the frontend
- To make event initialization conditional moved it to the "init". This way the event listeners are only set up when we boot with the "frontend" flag set to true
refs https://github.com/TryGhost/Toolbox/issues/135
- Without sensible defaults the web app was not initializing either the backend nor the frontned parts of the application. Fixed the defaults so the problem doesn't happen again and optimized mock-express-style initialization to only initialize the frontend routing
refs https://github.com/TryGhost/Toolbox/issues/135
- This optimization is expected to play a role in more consistent "backend-only" boot where the previous test state might have left over a different theme version which might cause in unwanted URL Services reainitializations.
- What has been happening here is the themes.test.js suite was uploading a theme with a v4 api and when the users api test suite loaded up it switched back to a default v2 theme, which caused routing reinitialization
- The root problem here is the themese suite is leaving a mess behind so a "restartModeGhostStart" is not really possible anymore - this should be cleaned up separately
refs https://github.com/TryGhost/Toolbox/issues/135
- Allows to turn off overwriting urls/resources JSON file caches on testing environment. This is needed to have predictable state when running multiple test suites that stop the Ghost process and try to persiste URL cache.
refs https://github.com/TryGhost/Toolbox/issues/135
- This extracts the file storage knowledge out of the URL Service an allows to have optional features based on the environment - for example turning off writing cache for when running tests
refs https://github.com/TryGhost/Toolbox/issues/135
- The global default should stay the same as it used to be and we can introduce an override for "withFrontend:false" on casa-by-case or area-by-area bases
refs https://github.com/TryGhost/Toolbox/issues/135
- Going though local utils allows to have the "withFrontend: false" flag applied only to the e2e-api test suites. This way we can gradually introduce the no-frontend change across all test suites
refs https://github.com/TryGhost/Toolbox/issues/135
- Having the proxy in local utils allows to disable frontend gradually in test suites instead of all at once (the latter approach didn't work out as too many other suites still depend on full frontend initialization)
refs https://github.com/TryGhost/Toolbox/issues/135
- When running e2e-API test in most cases there's no need to boot Ghost instance with full frontend. This should improve the boot time which should reflect on the speed of running test suites
- The tests where the "forceStart" and "withFrontend" are used together indicate that there's still some work to do to fully separate frontend/backend boot line. The force start is also unnecessary, but was needed to reinitialize all services properly - should be investigated!
refs https://github.com/TryGhost/Toolbox/issues/135
- To be able to reliably start ghost instance without a frontend the process needs access to urls/resources caches
- Storing the configuration in "paths" for now as there's no better place for it untill we are able to mock the content folder in pre-boot
refs https://github.com/TryGhost/Toolbox/issues/135
- These fixture had to be updated due to hardcoded ids introduced in the fixtures.json for test
- This allows to boot up a Ghost instance without recalculating urls for most common database states
refs https://github.com/TryGhost/Toolbox/issues/135
- These fixtures needs id's to be able to form a stable urls and resources cache that would be used across all e2e tests when booting without a frontend
refs https://github.com/TryGhost/Toolbox/issues/135
- This way the test is performed on a "full" set of posts that will become unified within all e2e tests for performance optimization
closes: https://github.com/TryGhost/Ghost/issues/13739
- Ghost cannot write to the core folder in correctly configured production installations
- Built assets therefore need to be written to the content directory
- Ghost does not overwrite anything in the content folder as part of an upgrade, therefore static files that are provided by Ghost
must still live inside /core
- So as a result, we now have core/frontend/public and content/public
- These two things are meant to improve performance at the cost of reliability.
- Perfect for testing, however I think they make a minimal impact on modern SSDs :(
- Still worth a shot to see if it helps with CI
refs: https://github.com/TryGhost/Toolbox/issues/130
The API version stays at v2 unless we stub the getFrontendApiVersion method. But stubbing the method doesn't get picked up unless we actually restart Ghost.
TODO: Maybe change the default here so we don't need to restart Ghost just to test the current version's API
- These are simple functions that get data from config in a specific format
- They are also used by the topmost part of the application
- Config helpers seems like a reasonable fit to get them out of the web folder
- Functions have also been renamed to try to get them to make more sense
refs https://github.com/TryGhost/Team/issues/1217
- moved top-level `tenorApiKey` to `tenor:apiKey` and added `tenor:contentFilter`
- added base config to `defaults.json`
- updated `public-config.js` and API output serializer to use the new top-level `tenor` key
https://github.com/TryGhost/Toolbox/issues/130
The transaction no longer commits in the promise chain, which wasn't
valid logic for a transaction, since it is commited automatically when
the promise chain resolves, and rollsback automatically when the
promise chain rejects.
This makes code which fails during the transaction error in the right
place, instead of getting stuck here. (Especially good for writing
tests).
The tests for this code can now live in the integration folder.
- This is a minor bugbare, but it will affect some configuration I'm about to do for c8
- I've been wanting to do it for ages, middleware is plural all on it's own so it's an odd affectation in our codebase
- This also only exists in 2 places, everywhere else we use "middleware"
- Sadly it did result in a lot of churn as I did a full find and replace, but consistency is king!
refs https://github.com/TryGhost/Toolbox/issues/116
- Allows to have deterministic routing on each start of the instance - can use cache when starting tests
- To make use of the new fixtures need to place them in the content/data
folder when the test suite is initialized, then the cache detection will
kick in and boot up the UrlService in a constant time intead of having
to check "urlServiceUtils.isFinished"
- this keeps production and test fixtures separate, so that changing the prod fixtures doesn't change the shape of our tests.
- we may still want to test that the production fixtures do what we expect, but that can be handled in a separate integration test, by specifically setting the fixture path
refs: https://github.com/TryGhost/Toolbox/issues/133
- instead of just a collection of utils, we now have a class that manages fixtures
- this should allow us to change the path to fixtures, e.g. between prod/dev and test, so that different fixtures can be loaded by default
- also makes it easier to test the fixture manager code itself
refs 042618fe93
- There's no longer "getValue" method passed around in UrlService, so the were unnecessary and did a wrong thing here :) Yey, less code!
refs https://github.com/TryGhost/Toolbox/issues/127
- Passing around whole instance of a frontend router was an overkill when there are only 3 static pieces of information that needed to be loaded. Extracting the router out makes the UrlGenerator way more readable, tests slimer, and the memory footpring of the process should be slightly lighter
- The toString overloading didn't make sense at the time of this refactor, maybe if there's a concrete usecase we could resurect it in a form of passing in a router's name or something.
refs https://github.com/TryGhost/Toolbox/issues/127
- This is an effor t to define a precise set of data needed for the UrlGenerator to function, which should help with decoupling it from the frontend routes
- This is almost the last piece to free us up from the massive "router" object that has been passed around
refs https://github.com/TryGhost/Toolbox/issues/127
- This is an effor t to define a precise set of data needed for the UrlGenerator to function, which should help with decoupling it from the frontend routes
refs https://github.com/TryGhost/Toolbox/issues/127
- This is an effor t to define a precise set of data needed for the UrlGenerator to function, which should help with decoupling it from the frontend routes
refs https://github.com/TryGhost/Team/issues/1217
- add `tenorApiKey` to `publicConfig.config()
- update canary config endpoint output serializer to include `tenorApiKey` when the `gifsCard` labs flag is enabled
refs https://github.com/TryGhost/Team/issues/1211
In order to override the default OEmbeds for OpenSea NFT's we need a way
to provide out own OEmbed data. We will want this in future too for
custom Twitter embeds, so this has been built in a way which allows
extension.
refs https://github.com/TryGhost/Team/issues/1212
This now emits the event when the service is reconfigured, rather than
when we issue the reconfigure command, which causes the event and the
action to be run in the wrong order. This would then cause knock on effects
of having the database in an undefined state - with stripe data in not linked
to the current Stripe account.
refs https://github.com/TryGhost/Toolbox/issues/120
- Changed endpoint name as it suits way better to the `/upload` postfix convention we use along with all other endpoints that support file uploads
closes https://github.com/TryGhost/Toolbox/issues/120
- Allows to update and upload brand new thumbnail images for previusly uploaded media resources
- The endpoint is available udner alpa flag as part of Admin API at `PUT /media/thumbnail/`
- As an input accepts following parameters:
- *required* `file` field containing an image file
- *required* `url` field containing parent media file URL
- *optional* `ref` as a field to put in an ID to reference the resource on the client side
- The response has following format:
```
{
media: [{
url: 'http://127.0.0.1:2369/content/images/1991/11/nicevideo_thumb.png'
ref: 'unique-id-420'
}]
}
```
refs https://github.com/TryGhost/Toolbox/issues/120
- When editing an uploaded media thumbnail file there'a need to remove existing thumbnail to keep media files:thumbnails 1:1. - Because the API client only has a public URL under which the resource is served it can only provide that as an API input, the `urlToPath` was also added to the base class of LocalStorageAdapter (it might be moved up to the BaseAdapter in the future if we see a need)
refs https://github.com/TryGhost/Toolbox/issues/114
- Files API is meant to be used for non-executable file uploads of all sorts
- The files are stored and retrieved for download as-is
no issue
- Not sure what the purpose of this test is. Seems like it's kind of like an db integrity check where it makes sure the change is double checked before commiting
- This comment removes the block on themes controlling card assets via config
- It also changes the default behaviour from "false" config (doing nothing) to excluding bookmark and gallery card assets
- This is essentially the same thing, as only bookmark and gallery card assets exist at the moment, but it's being done because it makes this feature future-proof for all theme developers.
- As we add new cards, all themes will automatically get the assets to make them work
- As theme developers want to, they can create their own custom assets and disble assets for any cards they support by adding them to the exclude list
- They can also remove any custom code they currently have to support bookmark and gallery cards, and set card_assets: true in package.json to use the defaults instead
- If we register the serve public file middleware for a file that doesn't exist, this will currently throw an ENOENT error
- Instead, we want to fall back to a standard 404 so that this behaves normally
- This will be useful for the card asset service, where the cards.min.css and cards.min.js files may or may not exist
refs https://github.com/TryGhost/Toolbox/issues/95
- The uploaded media thumbnail name should have a "_thumb" postfix to be able to distinguish thumbnails from other files. This can be handy if we decide to store them in a different location in the future.
refs https://github.com/TryGhost/Team/issues/807
The launch wizard completed flag was previously stored at per user level in accessibility column of user table, so an administrator still got the option to complete the launch wizard even if the owner had completed it previously, which is not expected pattern. This change moves the launch complete flag for Admin to common settings from per user level so a site only needs to complete the launch wizard once irrespective of which user completes it
- adds new `editor_is_launch_complete` setting to track if a site launch steps are completed in Admin
- adds new migration util to easily allow adding new setting
- adds migration to introduce new `editor_is_launch_complete` setting
- adds migration to update launch complete flag for a site if any of the users have already completed the launch steps
- Requires the new @tryghost/minifier package
- Adds a new service that will handle taking config from the theme and optionally including assets for Koenig editor cards
- It supports both css and js as cards may need one or both
- For any given config, the tool can find the matching files to include and concat and minify them into one file per type
- Currently has an override in place so that this is not yet customisable in the theme - will remove this override when we're ready for the feature
closes https://github.com/TryGhost/Toolbox/issues/111
- Just like with invalid JSON redirects files we should return a BadRequestError instead of throwing a generic 500 when the redirects.yaml file fails parsing
refs https://github.com/TryGhost/Toolbox/issues/95
- Each media file quires a thumbnail and these changes provide a capability to upload them along with media files.
- The thumbnail file is always required and has to be the format of already supported image formats
- The thumbnail should be uploaded as a part of "thumbnail" attachment in the request
- The regression tests added with this changeset will be claened up and moved to unit-tests (this is a dirty-but-working version!)
- The thumbnail always gets a name of the uploaded media file and keeps it's own extension.
- The thumbnails is accessible under the url present in the "thumbnail_url" reponse field
refs https://github.com/TryGhost/Team/issues/1190
- The assets were broken in Admin when the frontend and admin urls were different
- Fixed the issue by changing the `asset` helper to output absolute URLs when the frontend/admin urls are differents
refs 91efa4605c
- When the instance is booted without any redirects files configured it's not supposed to error but rather default to an "empty" [] redirects configuration.
- Ideally the logic shoudl not contain try/catch block at all and fail as soon as there's any error during the initialization. This wasn't changed at this time due to possible break of existing Ghost instances
refs refs https://linear.app/tryghost/issue/CORE-84/have-a-look-at-the-eggs-redirects-refactor-branch
- The tests needed to have a clean state with empty redirects file, which was previously ensured through "configUtils". Because configUtils don't play ball with the class initialization pattern this approach was chosen
- It's an end-to-end test with lots of logic and pobably would be enough to run against single API endpoint. Leaving it as is and to be improved in the future
refs https://linear.app/tryghost/issue/CORE-84/have-a-look-at-the-eggs-redirects-refactor-branch
- The problem this change is addressing is inability to override config values once the code is extracted into a class+DI pattern
- The work around is restarting the instance with the configuration testing expected behavior - in this case missing or existing types of redirects files
refs https://github.com/TryGhost/Members/commit/9e59f5a9
Since we have a DynamicRedirectManager for handling adding/removing
redirects at runtime, we no longer need the custom-redirects middleware.
The redirects service does however need an init method now to add the
custom redirects at Ghost boot, so it's been refactored into our Class &
DI pattern.
refs https://linear.app/tryghost/issue/CORE-121/create-a-video-storage-adapter
- This is an experimental implementation of video file upload support (audio is yet to follow)
- The storage adapter still needs more thinking as it's almost the same as the "LocalStorgeAdapter" that stores images.
- Also the output serializer skipped use of url utils in favor of inline implementatoin - this should almost certainly be it's own package
refs https://github.com/TryGhost/Team/issues/1178
The "up" migration that this util generates correctly throws if the
pre-requisite data cannot be found in the database. The "down" migration
however was incorrectly mirroring this behaviour of throwing - which
meant that it wasn't idempotent, as it does not require a permission or
role to existing if it wants to move relations between them.
no issue
The way GA flags were introduced means that they stop existing in the `'labs'` setting in the db and are instead forced to always return `true` when checking the flag in the labs service. However, Admin which uses the flags fetches them via the `/settings/` API endpoint which was only returning the raw labs setting db value meaning GA flags appeared to be disabled unless the flag had previously been enabled and no settings save had occured.
- updated the settings bread service to replace the labs setting value with the JSON stringified output of `labs.getAll()` which is the ultimate source-of-truth for a feature being enabled/disabled
- extracted `browse()` behaviour to an internal `_formatBrowse()` method so we can apply the same filtering/modification for output of `browse()` and `edit()`
Co-authored-by: Fabien O'Carroll <fabien@allou.is>
refs https://github.com/TryGhost/Team/issues/1115
This allows users to create Offers for Tier/Cadence pairs in order to
provide discounted subscriptions to Members! We have support for
percentage based discounts & fixed price discounts, either for the first
payment, all payments, or a number of monthly payments.
Offers also have a code, which can be used as an easy way to share them,
as visiting https://site.com/offer-code will automatically open Portal
with the Offer prepopulated.
closes https://github.com/TryGhost/Team/issues/1164
Themes can now define custom settings via their `package.json` file, and use them in templates via `@custom.{setting}`. Values for custom settings can be changed by site owners through a redesigned "Design settings" area in the admin interface.
Full announcement, documentation, and examples will be made available soon.
Co-authored-by:
- Sanne de Vries (@sanne-san)
- Thibaut Patel (@tpatel)
refs https://github.com/TryGhost/Team/issues/1175
We found the ETag header sent when serving the Admin template for /ghost/ was not changing between versions which after an upgrade could result in out of date cached content being served containing links to JS/CSS files that no longer existed.
The culprit is weak etags served by Node's `send` package, coupled with Admin template filesize not changing between versions and `npm pack` setting a fixed modification date for every file. See https://github.com/pillarjs/send/issues/176 for more details.
- updated the Admin app's controller to read the template and generate an md5 hash of the contents so we can serve a strong ETag header value when serving the `/ghost/` html
refs https://github.com/TryGhost/Team/issues/1163
We want to make the title for Offers optional, our nullable validation
means that we cannot store an empty string, so we must remove the NOT
NULL constraint from the column if we want to store either an empty
value or null.
There is a bug with editing columns in SQLite with `knex` which strips all
the indexes, so we have to manually add them afterwards.
- we're slowly trying to draw the lines between the backend and the frontend correctly
- these files deal only with serving the frontend so they should live there
- there are lots of mixed requires in these files, so having them in the right place makes that clear
refs https://linear.app/tryghost/issue/CORE-1/multiple-adapters-per-type
- There's a need to support multiple adapter variations per given adapter type (storage, sso, etc.)
- With the introduced changes we can specify a version of an adapter that should be fetched based on `:feature` postfix. For example:
`adapterManager.getAdapter('storage')` - would return the default adapter listed under "active" configuration
`adapterManager.getAdapter('storage:videos') - would return an adapter configured for videos *feature*
- Here's an example configuration for a custom video storage:
```
"storage": {
"active": "LocalFileStorage",
"videos": "ghost-storage-custom-video",
"ghost-storage-custom-video": {
"custom": "configHere"
}
}
```
- We were not requiring the full path to our fixture file, and therefore we were testing the fixtures + utility functions
- This updates the test to clearly require the exact files we are checking, except the routes.yaml where we can't
- Also updates the hash to be the correct hash for just the fixture content
closes https://github.com/TryGhost/Team/issues/1150
Our override of the base Bookshelf `insert` operation so that our own `formatOnWrite()` method is called on attributes was working on a false assumption that an `attrs` attribute is passed in as it is for the `update` operation. Instead Bookshelf's base update uses the `model.attributes` values to create an `attrs` object that is then passed through the usual `.format()` method meaning that our `insert` override was not actually doing anything.
- added a failing regression test for the `formatOnWrite()` override behaviour
- adjusted our insert/update overrides to set an internal `_isWriting` property on the model, then if that property is true our `.format()` override (which is called by Bookshelf on a generated `attrs` object during inserts) we manually call our `.formatOnWrite()` method
- updated both overrides even though `update` was working for consistency and less cognitive overhead for reasoning between two different approaches
no-issue
* Removed NOT_NULL constraint from stripe_coupon_id
When handling disconnecting from Stripe - we remove all Stripe data from
our database to ensure we do not have bad/invalid data stored. Removing
this constraint will allow us to set the value to NULL.
* Added created_at column to offer_redemptions
Offer Redemptions are not just a joining table, but an event. A created_at
date allows them to be ordered
Because this is in alpha it is simpler to just drop the tables and
re-add them, due to offer_redemptions depending on offers, we also drop
this table and re-add it.
- These don't make sense and we're working on improving testing across the board
- We'll make sure our testing best practices are documented when they've settled
- the integrationTesting utils are specific to the express mock style of testing
- all other tests can use the url-service-utils to check the url service is finished
- done a fastest-possible overhaul on this style of tests to try to get them to work independently again
This is a pattern that was introduced a while ago to try to speed up our e2e tests and I'm not sure if it's staying or going
It uses a minimal frontend-only version of the boot process and a custom-built express testing tool
However it's really old and out of date because of the boot refactor and several changes since
This highlights the key problem with it - it doesn't rely on any of our "core" boot process, it makes it up, and therefore how reliable are these tests?
Ideally we need to get these tests working with the real boot process in some capacity
We would then need to make sure we have all the tests in e2e-frontend written in this style
refs https://linear.app/tryghost/issue/CORE-1/multiple-adapters-per-type
- While digging around this area refactored the code to use more readable async/await syntax instead of chaining then's
- Dropped unneeded "catch" block in save method as all it was doing was rethrowing an error with out additional handling
refs https://linear.app/tryghost/issue/CORE-104/decouple-frontend-routing-events-from-urlserver-events
- The "bootstrap" didn't give enough credit to everything this module was doing - it's responsible for managing correct initialization and reinitialization of the frontend Routes as well as passing router creation information back to the frontend's URL service
- The refactor is done in two steps - the "bootstrap.js" will be renamed in the follow-up commit to have a clean history of how the file evolved
refs https://linear.app/tryghost/issue/CORE-104/decouple-frontend-routing-events-from-urlserver-events
- "routerCreated" call was causing a need to create a dependency on the frontend Router level which didn't fit nicely with the refactor of the bootstrap into a class, it's also makes way more sense having it as an independent parameter instead of a call on a module (makes testing way more readable too!)
refs https://linear.app/tryghost/issue/CORE-104/decouple-frontend-routing-events-from-urlserver-events
- The 'settings.timezone.edited' event triggers a roundtrip chain of calls in the frontend routing to the url services. It was all handled by event listeners and handler that clearly don't belong there.
- Extracted event realted listeners/handlers into methods and moved most of the logic to the "bootstrap" module, which soon is going to become a "RoutesManger"
- The result of this refactor - no more events going back and forth between frontend routing and the backend!
- core/shared/i18n is no longer used. Remove it to prevent temptation!
- this class needs merging with the one in themes, but for now just co-locate them as that's quicker and easier
refs https://github.com/TryGhost/Ghost/issues/13380
- Now that i18n.t has been removed everywhere, we can cleanup the final usages
- Still TODO: merge the i18n logic into themeI18n, and get rid of shared/i18n entirely
refs: https://github.com/TryGhost/Team/issues/759
- The match helper allows for basic equals and not equals comparisons,
Example:
{{match title "=" "Getting Started"}}
{{match slug "!=" "welcome"}}
- There's a lot more functionality we want to add here, so that it ends up being a replacement for {{#has}}
- However, this first iteration is already useful, especially in the context of custom theme settings
- Therefore we are adding it early, and will document it along with custom theme settings when that goes GA very soon
refs https://github.com/TryGhost/Team/issues/1107
- updated schema validation to allow `'image'` through as a known setting type now that Admin has support
- added transformation of setting values for `'image'` types because they will be URLs and should be stored with `__GHOST_URL__`
refs: https://github.com/TryGhost/Team/issues/759
- No matter what, a handlebars helper outputs a string. So if you return true, you'll always get 'true'.
- SafeStrings are handlebars's way of passing around a string whilst also maintaining a record of the original value e.g. new SafeString(true) results in {string: true}
- We need this for the match helper, so that we know when doing a comparison that we're meant to be comparing against a boolean true, not a string true
- Therefore, we need to putput SafeStrings, but also process them when passed in
The logic
- Figuring out the correct logic here has been a little tricky but essentially:
- {{match safestring}} with a single arg, will return true for any truthy value
- {{match safestring "=" true}} does a direct comparison with the original value of the safe string, so if it was a boolean true, the match will be true else false
- {{match (match something) "=" true}} will therefore work for any level of nesting
- this can result in slightly inconsistent results, but feels correct and documentable
This is documented extensively through the test cases
refs https://github.com/TryGhost/Ghost/commit/b93e7d7f7c
Our CI wasn't running integration tests so this slipped through. When
adding a new table we must update the exporter to ensure it's exported,
and that means the tests need to be updated to check for it!
refs https://linear.app/tryghost/issue/CORE-103/decouple-internal-frontend-code-from-url-module
- We need to decouple all frontend services from URL service as much as possible. "bootstrap" module is now a central point to substitute (proxy really) function previously done by the URL service and this move changes direct usage of URL service to "bootstraps" internal proxy function
refs https://linear.app/tryghost/issue/CORE-103/decouple-internal-frontend-code-from-url-module
- By becoming a parameter in the routing bootstrap process URL is Service no longer a "require" inside the frontend controllers but rather becomes a part of the "internal API" of the bootstrapper. This is not the end form of it, rather a step closer to decouplint routing from the URL serivce.
- The bootstrap module needs a facelift to have cleaner distinction between init/start methods. This is left for another time
refs https://github.com/TryGhost/Team/issues/1106
- updated schema validation to add `'boolean'` as an allowed `type` value
- added `format()` and `parse()` methods to `CustomThemeSetting` model to match `Settings` model behaviour for boolean-type settings
refs https://github.com/TryGhost/Team/issues/1132
This will be used to store Offer Redemptions, which will be used to list
the Offers which a Member has redeemed, as well as the number of times
an Offer has been redeemed.
refs https://github.com/TryGhost/Team/issues/1104
- bumped `@tryghost/custom-theme-settings-service` so it throws a more appropriate `ValidationError` when setting keys don't exist or a select value is not known
- changed the custom theme settings service to have a `.init()` method which creates an instance of the service under `.api` so that we're able to create the instance at a particular point in the boot process when we know the models have been initialised
- there were problems in tests because the service was being initialised through the require chain before models were initialised through the boot process
- fixed incorrect `camelCase` of resource name in API responses
refs: #13380
- The i18n package is deprecated. It is being replaced with the tpl package.
Co-authored-by: Aleksander Chromik <aleksander.chromik@footballco.com>
- this test file uses a different pattern to the other test files
- not yet sure if the pattern is terrible or genius, need to assess before moving it into a folder full of what are meant to be exemplary tests
closes https://github.com/TryGhost/Team/issues/1125
refs 3c822e0457
- Email-only is not considered a general availability feature and can be used without special flags.
- It allows to publish a new post type "email only" that only goes out as an email newletter and is available through an undescoverable URL (does not appear anywhere publicly similarly to preview posts) on the site.
- e2e tests are tests that cover critical functionality by booting ghost
- integration tests are more like unit tests, but need to initialise and use a db
- so settings shouldn't start Ghost, url service is critical and should be in integration, and preview is critical and should be in e2e
- some tests are necessarily driven from the db
- these are like unit tests, except they only make sense if using the db - else you have to stub too much to make them worthwhile
- for these rare but important cases, we have the clear concept of integration tests
- We have a bunch of important server-related e2e tests
- Make these clear in their own folder
- "server" is everything that isn't the api or the frontend - kind of a catch-all concept
- this is a small part of a bit of cleanup of our test files
- the goal is to make the existing tests clearer with a view to making it easier to write more tests
- this makes the test structure follow the codebase structure more closely
- eventually we will colocate the tests as we break the codebase down further
- this is a small part of a bit of cleanup of our test files
- the goal is to make the existing tests clearer with a view to making it easier to write more tests
- this makes the test structure follow the codebase structure more closely
- eventually we will colocate the frontend tests with the frontend code
refs: 9d7049cd3
- I missed that the amp_content helper was meant to be async when refactoring
- I have updated our proper amp acceptance tests to catch the content not rendering as this is a regression, this should definitely have been caught
- Added missing async property to amp_content helper to fix the issue
- The helper registration code is "framework" code and very specific
- At the moment the "theme engine" is full of lots of disparate theme related stuff
- I'm trying to make the frontend framework code clearer and also expand it to make it more useful
- The helper system now also exposes 3 methods allowing you to register a directory, a helper or an alias
- I've updated the codebase to use these both for our core helpers and for "apps"
refs https://github.com/TryGhost/Team/issues/1083
The Offers service is going to need access to the StripeAPIService too,
so we need to move it out of the @tryghost/members-api module and make
it accessible to both.
refs https://linear.app/tryghost/issue/CORE-86/fix-failing-site-instance-when-redirects-file-is-invalid
refs 260a47da83
- Added validation logic to catch redirects files having invalid RegEx expressions when they are introduced into the system (on upload)
- This way the error happening in the refed commit would have not happened as the validator would not have passed it through
- Moved up the "Router" declaration in custom-redirects as it needs to happen before any other bit of logic has a chance to throw
refs https://linear.app/tryghost/issue/CORE-86/fix-failing-site-instance-when-redirects-file-is-invalid
refs 260a47da83
- Refed commit was missing a unit test coverage.
- The approach here introduces a new pattern - using `supertest` in unit tests. I've found this to be the most expressive way to test an express app which receives certain middleware dynamically. Because there are very few moving parts the test is still extremely quick to run
refs https://github.com/TryGhost/Team/issues/1088
- adds schema for new offers table
- adds permission fixtures for new offers table
- adds migrations for new table and permissions
Co-authored-by: Fabien O'Carroll <fabien@allou.is>
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
refs 5715aa2155 (diff-48644be82a9b957e5e627bf7b0f2f73cdb1d63851ffad68c7c178c5886495bb8R52-R57)
- Simplified the yaml parser implementation to take in a single parameter, this move will allove to simplify the logic in the route settings + opens a door to unify handling with redirects yaml parsing!
- We loose the "filename" from the error information but that was a generic "routes.yaml" anyway and would be thrown only when somebody uploaded a routes.yaml file (no real added value).
- The debug statement should be moved to contain related filepath+other info to the calling module instead
- An additional error handler was borrowed from the redirects yaml parsing logic that was introduced in a referenced commit - it still makes sense to keep it for routes.yaml configuration
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
- It's a step to making the module follow class+DI pattern before fully extracting it into an external libarary
- Reminder, doing in Ghost repo instead of substituting big chunks all at once to have clear history of how the service evolved prior to the extraction into external lib!
refs https://github.com/TryGhost/Team/issues/1097
- added `customThemeSettingKeys` as an argument to `preview.handle()` because we can't know which keys should be allowed through up-front
- added `custom` as a supported setting in the preview header data
- `custom` should be a JSON object containing any custom theme settings
- we parse the object but only set properties on `@custom` that are known custom theme setting keys
- if parsing fails or it's not an object then no custom data is set
- updated `updateLocalTemplateOptions()` to pull `.custom` off of the preview data and pass it through so it's accessible on `@custom` as an override to the saved custom data
refs https://github.com/TryGhost/Team/issues/1097
globalTemplateOptions are supposed to be static with localTemplateOptions being merged in per-request, however the per-request preview data was being extracted and set in the global options. Comments suggest that the global data should be static and eventually updated via other means, the usage of the request object to get per-request preview data is working against that.
- adjusted the preview handler to return an object rather than changing properties by reference on a passed in object
- moved preview data fetching out of `getSiteData()` used in `updateGlobalTemplateOptions()` and into `updateLocalTemplateOptions()` so that we're not relying on the request object in `updateGlobalTemplateOptions()`
no issue
- if any of the assertions in a test failed there was no handling, instead the test just timed out with a timeout error
- wrapping the assertions in a try/catch and calling `done()` with the error object aborts the test immediately and shows a useful assertion failure message
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
- Ensure settings had only one method but would benefit from class+DI pattern before extracting it into an outside module.
- The logic is now also less coupled with "routes" and single source/destination paths. It's all configureable instead and might be reused if similar pattern is needed for example with redirect settings defaults.
- The original intention of the proxy was to collect up all the requires in our helpers into one place
- This has since been expanded and used in more places, in more ways
- In hindsight there are now multiple different types of requires in the proxy:
- One: true frontend rendering framework requires (stuff from deep inside theme-engine)
- Two: data manipulation/sdk stuff, belongs to the frontend, ways to process API data
- Three: actual core stuff from Ghost, that we wish wasn't here / needs to be passed in a controlled way
- This commit pulls out One into a new rendering service, so at least that stuff is managed independently
- This draws the lines clearly between what's internal to the frontend and what isn't
- It also highlights that the theme-engine needs to be divided up / refactored so that we don't have these deep requires
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
refs 7528ec8c3b
- The way the custom redirects middleware was organized made it extremely hard to unit test it (had to stub the redirects service methods etc). With a new organization it's possible to provide needed redirects configs to the method which makes the actual redirects Router logic testable and the code less coupled with redirects services
- This was meant to be an attempt to extract more of the slow redirects regression tests, which failed. Instead found this weak spot that could be improved and gained:
- shaved 4s of time as two slow regression test cases are now gone
- there's now a base to build upon when getting more coverage for the custom redirects middleware
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
- The only allowed route settings name is 'routes.yaml', which removes a need to parameterize the function as the location is permanent anyway
- Simplifying the function in any possible way before extracting the common bits into an external lib
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
- Frontend is not meant to know about the underlying source of the "routes" configuration, so any reads/edits/validations are being moved into a backend service. This should also simplify the coupling of the backend with the frontend where the latter will get a JSON blob with all needed configuration during the boot
- Nother problem the "get" method had was hiding an underlying function it was doing - reading the file from the filesystem SYNCRONOUSLY. It might be a thing we need to do during the "web" app initialization, but there's no clear need to do this in a sync fassion during the bootup for example. Also having a more explicit name should help :)
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
refs c1c9bf0866
- Actions logic related to file system operations (like ensuring files exist) should be done on the backend. Now the route settings initialization logic lives on the backend it makes sense to keep the file closer to the source.
- The move is the opposite to the one refed in the commit with a
difference that the file now lives in "route-settings"
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
- 'knowSettings' was based on a "configurable" array of settings that might be configured in Ghost. The multitude never happened! The only setting the frontend takes care of is routes.yaml file (redirects is also kind of a setting but is a separate concept for now).
- Having just one type of file to deal with allows to simplify implementation significantly, which helps before a big refactor
refs https://linear.app/tryghost/issue/CORE-35/refactor-route-and-redirect-settings
- This is a micro-step towards getting rid of multiple "knownSettings" concept. Since the introduction of an array of knowSettings there was never-ever a need to handle anything but a single `routes.yaml` file. Getting rid of this concept first to have a simpler module. Next step would be getting rid of filesystem reads/writes in the "frontend"
refs https://github.com/TryGhost/Team/issues/1070
- stores values of custom theme settings
- will be merged with full settings data parsed from themes for API output
- will be cached and made available for lookup in themes to avoid db roundtrips
- stores type of custom theme settings so we can coerce values and know if the type has changed when syncing
- records will be synced with themes upon activation
refs https://github.com/TryGhost/Ghost/security/advisories/GHSA-wfrj-qqc2-83cm
refs https://github.com/advisories/GHSA-48ww-j4fc-435p
- a vulnerability in `nodemailer` means that the `sendmail` transport is
vulnerable to command injection for flags passed to the `sendmail`
binary
- updating to the latest version of Nodemailer required creating
`@tryghost/nodemailer`, which is a wrapper around Nodemailer and
several plugins that used to be in the core
- this commit switches to using that package, and fixes up some small
code + test changes
closes: CORE-34
refs: https://github.com/TryGhost/Team/issues/1044
- this is a super basic fix, it adds a max nodes concept and limits the node in each sub-sitemap to 50k by default
- this will prevent the error in google console
- a better fix is in progress, but we want to at least solve the errors ASAP
refs https://github.com/TryGhost/Team/issues/1053
This table is going to be completely deleted at some point in the
future. It serves as a persistent datastore for a spike into collection
analytic events for members. We've opted for a generic table, rather
than a table for each event, so that we can push the DB to the limit in
terms of the length of the table, and find out performance issues A$AP
closes: CORE-33
Two bugs:
- lodash isEmpty and handlebars util isEmpty are not the same
- I literally had the truthy and falsy cases the wrong way around 🙈
Notes:
- I have, for now, copied the isEmpty util from handlebars. It's so small it doesn't seem worth trying to require the util right now, although in future it'd be nice if that was easier to do
- Adding the management for the conditional being a SafeString allows the match helper to be a subexpression of itself, I can see this pattern being useful later in combo with the any and all helpers
