no issue
- we increase the client in-memory expiry for production built assets
- as soon as there will be another release, a new asset hash is generated and the client cache is invalidated automatically (doesn't matter how long we store the file in the client)
- the next step is to get rid of having asset hashs part as query params
- ghost-sdk.min.js?v=1234 is becoming e.g. ghost-sdk-1234.min.js
- reasons:
- A: performance tools complain about it
- B: we no longer invalidate the asset hashs for built assets if the theme changes
no issue
- discovered while testing
- activate theme
- download theme
- modify theme
- upload theme
- override? yes
- translation files are not reloaded, because the database is up-to-date
- remove un-used events in theme api layer
- trigger event from theme service
closes#5071
- Remove hardcoded notification in admin controller
- NOTE: update check notifications are no longer blocking the admin rendering
- this is one of the most import changes
- we remove the hardcoded release message
- we also remove adding a notification manually in here, because this will work differently from now on
-> you receive a notification (release or custom) in the update check module and this module adds the notification as is to our database
- Change default core settings keys
- remove displayUpdateNotification
-> this was used to store the release version number send from the UCS
-> based on this value, Ghost creates a notification container with self defined values
-> not needed anymore
- rename seenNotifications to notifications
-> the new notifications key will hold both
1. the notification from the USC
2. the information about if a notification was seen or not
- this key hold only one release notification
- and n custom notifications
- Update Check Module: Request to the USC depends on the privacy configuration
- useUpdateCheck: true -> does a checkin in the USC (exposes data)
- useUpdateCheck: false -> does only a GET query to the USC (does not expose any data)
- make the request handling dynamic, so it depends on the flag
- add an extra logic to be able to define a custom USC endpoint (helpful for testing)
- add an extra logic to be able to force the request to the service (helpful for testing)
- Update check module: re-work condition when a check should happen
- only if the env is not correct
- remove deprecated config.updateCheck
- remove isPrivacyDisabled check (handled differently now, explained in last commit)
- Update check module: remove `showUpdateNotification` and readability
- showUpdateNotification was used in the admin controller to fetch the latest release version number from the db
- no need to check against semver in general, the USC takes care of that (no need to double check)
- improve readability of `nextUpdateCheck` condition
- Update check module: refactor `updateCheckResponse`
- remove db call to displayUpdateNotification, not used anymore
- support receiving multiple custom notifications
- support custom notification groups
- the default group is `all` - this will always be consumed
- groups can be extended via config e.g. `notificationGroups: ['migration']`
- Update check module: refactor createCustomNotification helper
- get rid of taking over notification duplication handling (this is not the task of the update check module)
- ensure we have good fallback values for non present attributes in a notification
- get rid of semver check (happens in the USC) - could be reconsidered later if LTS is gone
- Refactor notification API
- reason: get rid of in process notification store
-> this was an object hold in process
-> everything get's lost after restart
-> not helpful anymore, because imagine the following case
-> you get a notification
-> you store it in process
-> you mark this notification as seen
-> you restart Ghost, you will receive the same notification on the next check again
-> because we are no longer have a separate seen notifications object
- use database settings key `notification` instead
- refactor all api endpoints to support reading and storing into the `notifications` object
- most important: notification deletion happens via a `seen` property (the notification get's physically deleted 3 month automatically)
-> we have to remember a seen property, because otherwise you don't know which notification was already received/seen
- Add listener to remove seen notifications automatically after 3 month
- i just decided for 3 month (we can decrease?)
- at the end it doesn't really matter, as long as the windows is not tooooo short
- listen on updates for the notifications settings
- check if notification was seen and is older than 3 month
- ignore release notification
- Updated our privacy document
- Updated docs.ghost.org for privacy config behaviour
- contains a migration script to remove old settings keys
refs #5345, refs #3801
- Blog localisation
- default is `en` (English)
- you can change the language code in the admin panel, see https://github.com/TryGhost/Ghost-Admin/pull/703
- blog behaviour changes depending on the language e.g. date helper format
- theme translation get's loaded if available depending on the language setting
- falls back to english if not available
- Theme translation
- complete automatic translation of Ghost's frontend for site visitors (themes, etc.), to quickly deploy a site in a non-English language
- added {{t}} and {{lang}} helper
- no backend or admin panel translations (!)
- easily readable translation keys - very simple translation
- server restart required when adding new language files or changing existing files in the theme
- no language code validation for now (will be added soon)
- a full theme translation requires to translate Ghost core templates (e.g. subscriber form)
- when activating a different theme, theme translations are auto re-loaded
- when switching language of blog, theme translations are auto re-loaded
- Bump gscan to version 1.3.0 to support more known helpers
**Documentation can be found at https://themes.ghost.org/v1.20.0/docs/i18n.**
no issue
- our API layer uses a unit to combine incoming data and options
- e.g. `options.data` is the end result
- we have to take care that we don't pass data into the model layer
Credits: Olivier Arteau
closes#9381
Fixes a bug where the date helper would ignore any timezone settings, when called with a specific date option, e. g. `published_at`, as `timezone` was only ever assigned when called without options.
no issue
- decreases chance of not-loaded modules or circular dependencies
- e.g. the i18n implementation will use the settings-cache and the settings-cache uses lib/common/events
closes#9022
Images without extensions don't need to be manipulated, as we're now reading the bytes and pass those to the `image-size` lib.
This PR adds another `user-agent` to emulate multiple browser requests, as I stumbled over an example where the image without extension is protected otherwise.
Added a test, that works with above mentioned image, but is currently mocked. Nevertheless, the image worked as a PoC, that we're able to read the bytes of an image without its extension and still return the dimensions of the image.
refs https://github.com/TryGhost/Ghost-Release/issues/24
- differentiate between
1. original package.json version (can contain pre and build suffix)
2. full package.json version X.X.X-{pre} (optional)
3. safe package.json version X.X (major+minor)
no issue
- with 29e143fa9a import queries no longer run in parallel
- this commit simply adds a small code snippet to reflect the importer behaviour
1) duplicate slugs *within* a file are getting ignored
2) existing posts in the database and posts to import with the same slug, result in duplicates
Further improvements regarding duplication detection will happen via #8717.
closes#8717
- this is now required, because we run import queries sequentiell
- this code protects two cases:
- you have duplicate slugs in the JSON file (the first get's inserted, the second get's ignored)
- you have an existing slug in the database and you try to import the same slug, get's ignored
closes#9348
- do not run import with `Promise.all`
- with a large import file, we run an enormous amount of queries in parallel, which does not allow Node to cleanup memory
- tested with an 13mb import file
- requires bookshelf-relations 0.1.4
refs #9178, refs #8988
With 7353c87d7f we use Bluebird globally for Promises. Therefore, the request lib doesn't need to be wrapped in a bluebird Promise anymore.
This was originally done, so we can work with catch predicated in our image-size lib.
Updated the tests to proof, that the catch predicates work.
The tests fail, as soon as the Promise overwrite is commented out.
refs #8868
- Loading the admin prior to a build results in: Failed to lookup view "error-404" in views directory
- This fixes that error, by splitting the HTMLErrorRenderer and the ThemeErrorRenderer into two separate things
no issue
- required for #8437
- one instance of hyphenated key changed; the rest of keys in file
_core/server/translations/en.json_ are already camelCase
- also converted `common.i18n.t()` calls to this key in file
_core/server/update-check.js_
- this allows to simplify i18n to an unified use of `jsonpath`
refs #9178
- this util uses the url services (!)
- moving this file into lib would not make sense right now
- that would mean a module requires first ../lib/url, which then requires ../services/url
- the url service definitely need a clean up 😃
refs #9178
- not 100% sure about this, but i think it makes right now the most sense
- we have already a url service and creating another lib/url is confusing at the moment
- i'll copy the last utility `makeAbsoluteUrls` to the url service for now
- see next commit for explanation (!)
refs #9178
- `checkFileExists` and `checkFileIsValid` where dirty required from web/middleware
- these two functions are only used in the target middleware
- let's move them
refs #9178
- i am not super happy about `const imageLib = require('../lib/image')`
- i don't really like the name `imageLib`
- but i had no better idea 😃
- if we use the same name in the whole project, it's very easy to rename the folder or the variable
no issue
> Deprecation warning: value provided is not in a recognized ISO format. moment construction falls back to js Date(), which is not reliable across all browsers and versions.
no issue
> (node:63849) Warning: Possible EventEmitter memory leak detected. 101 settings.edited listeners added. Use emitter.setMaxListeners() to increase limit
- the settings cache was initialised per test
- it registered the model events over and over again
- add a simple shutdown function, which can be called from the test env
refs #9178
- they definitely don't belong to server/utils
- i think the best place is putting them into the card apps
- the the post model needs to ask the app for it's converters
- move tests as well
refs #9178
- first iteration of tidying up the unit tests
- this is useful in the current stage, because if i move files in the server folder, i need a clean folder/file structure to detect which tests needs to move
- this is a simple cleanup to reflect the current server folder structure
no issue
- this test fails sometimes, i recently increased the socket delay (maybe it was not enough)
- because these are milliseconds
- let's try 100ms
refs #9178
- Ghost uses the Node crypto lib always direct (require('crypto'))
- it doesn't make sense to outsource a single crypto statement (for the asset hash)
- we either have to write a crypto wrapper to avoid writing long crypto statements or we keep the direct usages for every case
- for now, wrapping the crypto calls into a lib/crypto has no priority
refs #9178
- continue with killing our global utils folder
- i haven't found any better naming for lib/promise
- so, require single files for now
- instead of doing `promiseLib = require('../lib/promise')`
- we can optimise the requires later
refs #9178
- each package/module has a local utility (e.g. api, helpers, adapters)
- these are very small utility functions which are only used from this package
- they don't belong into the global lib/utils
refs #9178
- this logic belongs to a static model helper
- the visibility property is a model property, the knowledge about the visibility values belongs to the model
- rename the functions, so they make more sense
refs #9178
- we have to take care that we don't end up in circular dependencies
- e.g. API requires UrlService and UrlService needs to require the API (for requesting data)
- update the references
- we would like to get rid of the utils folder, this is/was the most complicated change
no issue
- rename the config option to disable preloading the urls
- always expose the urlservice as singleton
- do the initialisation of the service inside the constructor
refs #9178
- move express apps to one place (called `web`)
- requires https://github.com/TryGhost/Ghost-Admin/pull/923
- any further improvements are not part of this PR
- this PR just moves the files and ensures the paths are up-to-date
refs #9178
- the first test will start the ghost server
- all following test will reuse the server and only truncate the database tables
- you can use `forceStart` if you need
- we have to ensure that reusing the server requires us to
- e.g. reload the settings
- e.g. reload active theme
- remove fork functionality (you can use `forceStart: true`)
- no test has changed, just the way we start the server
no issue
- adapt major changes of knex-migrator v3
- adapt migration scripts, simplify and add `down` (rollback) hook if possible
- clear Ghost cache after init hook (because of `knex-migrator migrate --init`)
- ensure db migrations work with the CLI
- updated troubleshooting guide (https://docs.ghost.org/v1/docs/troubleshooting#section-task-execute-is-not-a-function)
**For development only: Please ensure you run `npm i -g knex-migrator@latest` to update your global installation to v3. We always prefer the local installation, but v3 has modified and added binaries.**
no issue
- i saw that the channels routing tests were sometimes not running in Travis
e.g. https://travis-ci.org/TryGhost/Ghost/jobs/311303939
> Channel Routes
> Unhandled rejection Error: select "webhooks".* from "webhooks" where "event" = 'subscriber.added' - SQLITE_ERROR: no such table: webhooks
no issue
- https://github.com/mochajs/mocha/blob/master/CHANGELOG.md#400--2017-10-02
- the new `--exit` flag might be interesting at some point
> In Mocha v3.0.0 and newer, returning a Promise and calling done() will result in an exception.
- adapt teardown/setup test utility
- adapt other mixed usages of callback && Promise usage
refs https://github.com/TryGhost/Team/issues/41
- differentiate error codes
- return 404 if image was not found
- else return a 500
- use i18n keys
- use errors.utils.isIgnitionError (!)
refs https://github.com/TryGhost/Team/issues/41, refs https://github.com/TryGhost/gscan/issues/85
- if you are using the pagination helper not inside a resource context, you will receive an error
- improve error message, because it was not clear what happened
- downgrade error level to normal, because it's not a critical error from Ghost's perspective, from user perspective it is
- added help docs link and added a callout to our docs
refs https://github.com/TryGhost/Team/issues/41
- if you add invalid handlebars logic e.g. {{if condition condition}}, handlebars throws an error
- in case of having invalid hbs in an amp page, the amp component throwed another syntax error (which is fixed in this PR)
- furthermore the `setTemplate` helper function had a logic bug, which did not handle errors correctly
- if there is an error and a template is set (e.g. amp), we have to still render the error page and not the amp page
- this fix only ensures that the error handling is correct, we still see the error of the "ugly" handlebars message
- e.g. [amp.hbs] Cannot read property 'includeZero' of undefined
- but no longer -> Cannot read property 'html' of undefined (which was a syntax error in Ghost)
refs https://github.com/TryGhost/Team/issues/41
- if you send invalid encoded url components in the path, the server tried to decode the url
- if it contains invalid characters like /AF%, it throwed a 500
- we return a page not found error instead
no issue
Support for http://resthooks.org style webhooks that can be used with Zapier triggers. This can currently be used in two ways:
a) adding a webhook record to the DB manually
b) using the API with password auth and POSTing to /webhooks/ (this is private API so not documented)
⚠️ only _https_ URLs are supported in the webhook `target_url` field 🚨
- add `webhooks` table to store event names and target urls
- add `POST` and `DELETE` endpoints for `/webhooks/`
- configure `subscribers.added` and `subscribers.deleted` events to trigger registered webhooks
no issue
- added https://github.com/TryGhost/bookshelf-relations as dependency
- remove existing tag handling
---
* Important: Ensure we trigger parent initialize function
- otherwise the plugin is unable to listen on model events
- important: event order for listeners is Ghost -> Plugin
- Ghost should be able to listen on the events as first instance
- e.g. be able to modify/validate relationships
* Fix tag validation
- we detect lower/update case slugs for tags manually
- this can't be taken over from the plugin obviously
- ensure we update the target model e.g. this.set('tags', ...)
* override base fn: `permittedAttributes`
- ensure we call the base
- put relations on top
- each relation is allowed to be passed
- the plugin will auto-unset any relations to it does not reach the database
* Ensure we run add/edit/delete within a transaction
- updating nested relationships requires sql queries
- all sql statements have to run in a single transaction to ensure we rollback everything if an error occurs
- use es6
closes#8143
Fixed a potential issue (edge-case), where our generated and validated (in terms of check for existance and add a counter) would return a slug, that will exceed the maximum length of the slug fields (191 chars).
This is mostly possible for the post title, which can be 255 chars long and would generate a slug with the same length. This would prevent the user from actually saving a post.
I tried first to determine the expected length for a slug that already exists, but decided that the **easier** and simplyfied implementation is to always cut a slug to **185 chars** (+ counter). This makes it easier to find duplicates and includes a possible high number of counts (edge-edge-case).
The slug will not be cut down to 185 chars if it's an import.
refs #9192
- Introduces a url service that can be initialised
- Added a concept of Resources and resource config.json that contains details about the resources in the system that we may want to make customisable
- Note that individual resources know how to create their own Urls... this is important for later
- Url Service loads all of the resources, and stores their URLs
- The UrlService binds to all events, so that when a resource changes its url and related data can be updated if needed
- There is a temporary config guard so that this can be turned off easily
refs #8143
Add max length validations to settings:
- `blog.title`: 150 chars
- `blog.description`: 200 chars
The `validateSettings` fn in our validations checks for existing `validations` properties in our `default-settings.json` file, similar to other tables in our `schema.js`.
no issue
- when calling `doesTranslationKeyExist`, we want to know if a key exists, we don't want to log if the key was not found
- this can mess up the server log
no issue
- useful for managing subscribers via external systems/API calls where it's likely only the e-mail address will be known
- adds `GET /subscribers/email/:email/`
- adds `DELETE /subscribers/email/:email/`
no issue
Had a couple of people ask about how to delete welcome posts easily, so adding a bio to the default user to draw a little more attention to it
no issue
- it can happen that concurrent requests try to renew access tokens with the same refresh token
- in this case it could happen that you received a token deletion error
- add propert locking
- ensure we don't run into deadlocks
- manual testing with async.times for parallel requests (was able to reproduce the error)
refs #5091, #9192
- Renderer figures out templates, contexts, and does a render call
- Templating is now handled with a single function
- Context call is made in the renderer
Note: to make this work, all controllers now define a little bit of config, currently stored in res._route. (That's a totally temporary location, as is res._template... when a sensible naming convention reveals itself I'll get rid of the weird _). This exposes a type and for custom routes a template name & default.
refs #9192, refs #5091
- Moved all url generation into generate-feed.js, so we can see as much data processing as possible in a single place.
- Refactored the way res.locals were used, to be more like how express uses them prior to rendering
- Removed a bunch of code & tests todo with context for RSS - I can't see any way that'd be used, unless we switched the rendering to use a template.
- moved the RSS rendering to be part of the service, not controller
- updated the tests significantly
Note: RSS generate-feed has a complete duplication of the code used in the excerpt helper in order to create an item description
refs #8613, refs #9228
- if you send a request to /authentication/token with `grant_type:password` and a Bearer token, Ghost was not able to handle this combination
- because it skipped the client authentication, see https://github.com/TryGhost/Ghost/blob/1.17.0/core/server/auth/authenticate.js#L13
- and OAuth detects the `grant_type: password` and jumps in the target implementation
- the target implementation for password authentication **again** tried to fetch the client and failed, because it relied on the previous client authentication
- see https://github.com/TryGhost/Ghost/blob/1.17.0/core/server/auth/oauth.js#L40 (client.slug is undefined if client authentication is skipped)
- ^ so this is the bug
- we **can** skip client authentication for requests to the API to fetch data for example e.g. GET /posts (including Bearer)
- so when is a client authentication required?
- RFC (https://tools.ietf.org/html/rfc6749#page-38) differentiates between confidential and public clients, Ghost has no implementation for this at the moment
- so in theory, public clients don't have to be authenticated, only if the credentials are included
- to not invent a breaking change, i decided to only make the client authentication required for password authentication
- we could change this in Ghost 2.0
I have removed the extra client request to the database for the password authentication, this is not needed. We already do client password authentication [here](https://github.com/TryGhost/Ghost/blob/1.17.0/core/server/auth/auth-strategies.js#L19);
If a Bearer token is present and you have not send a `grant_type` (which signalises OAuth to do authentication), you can skip the client authentication.
refs #9192, refs #9178
After trying to progress with current implementation, it became clear that the route service can't control the boot sequence, because then we end up with circular dependencies between the route service and the channel service.
The route service now exposes:
- a siteRouter
- a way for apps to register routes.
- ParentRouter base class for other modules to use
- the registry
...
- moved the default route setup back to site/routes.js 🙈
- moved the parent channel router back to the channel service (this makes way more sense imo)
- this structure prevents circular dependencies
- split the registry out into it's own thing
- fixed-up various bits of tests and comments
- DEBUG will print a list of routes 🎉
refs #9192, #5091
- changed channels to use our new base class
- keep the flexible structure, so that channels can be reloaded
- I had to move the router into the route service otherwise we get circular dependencies
- Don't _really_ want to keep it like this - need a way to define base classes as shared
no issue
- moved isLocalImage fn to storage utils used the RegExp of getLocalFileStoragePath to detect also relative image paths and added tests.
- Added test for independent protocol request (skip, because not supported/implemented)
refs #9192
- Moving towards a centralised concept of routing / routes
- The base router now wraps express router, and offers us the features we need
- Site Router is the parent router, it gets initialised with all of our default routing
- App Router is a sub router for apps - apps register their routes/routers onto it.
- TODO: refactor channels subrouter to work this same way
- MAYBE: move the app router to the apps service
refs #9192, refs #5091, refs #9178
- moved channels from controllers to a service
- split out the parent router from the remaining individual router logic
- moved the tests to match
refs #9192
- Admin redirects should really happen first, up with custom redirects
- Later we can package this up, maybe
- For now, let's focus the site router on site-related things
refs #5091, refs #9192
- There are several theme template "renderers" all over the codebase
- Some are in apps, and were called "controllers"
- One is in error handling
- All of them now have comments marking out how they share logic/steps
- Other comments describe routes & controllers where they live
refs #9192
- The AMP app is nothing more than a custom controller - this will come clear soon
- Moved enabled/disabled logic into router
- Removed error-related code, as this wasn't used
- Changed logic for static pages to be based on req.body, not context
- Improved the tests to match
refs #5091, #9192, #9178
- Get the RSS module into a much better shape
- Controller -> /controllers/rss
- Remainder -> /services/rss
- Moved tests to match & updated requires
refs #5091, refs #9192
- This is similar to #9218, in that I'm revealing bits of code that are "controllers" in our codebase. As opposed to routes, services, renderers etc.
- This also reveals some code which is identical to the channels controller
- There is more to do here, but for now I've got the module split up, and the tests split and improved.
- Next I'll split RSS into controller + service, DRY up the controller code, etc
closes#9200
- Registered new server helper `{{reading_time}}`.
- Added new global util `word-count` based on the util in Ghost admin, which returns the number of words in an HTML string.
- Based on the word count of the post html, the helper calculated the estimated reading time:
- 275 words per minute
- additional 12 seconds when post has feature image
- Renders a string like 'x min red', unless reading time is less than a minute. In this case, the rendered string is '< 1 min read'.
refs #5091, refs #9192
- render channel was always a weird file
- now it's clearly 2 things
- we're slowly getting towards closing #5091... 🎉
- added some extra tests
refs #9192, refs #5091
- Using a class allows for easy shared logic
- Loading is designed to work from config right now, but could be DB driven, etc
- Provided configuration can be simplified and extended in the constructor / class methods
- Update tests, move custom assertions to utils
refs #9192
To anyone seeing this go by - I'm about to start some fairly major refactoring work on the url utility. Before I do that, I wanted to make sure I had 100% coverage, and understanding of some of the weird cases.
The majority of the changes I've made are adding tests, but I was also able to clean up a little bit, remove a few lines or change them to make use of other tools.
refs #9192
- Instead of `setupRoutes` function in apps that gets passed a router, there is now a registerRouter function as part of the proxy
- Moved towards a route service, which will know about all routes
- Using classes to abstract away shared behaviour
Notes:
- changing the app proxy didn't result in a test failure!
- structure of route service is totally new and may change a lot yet
refs #9178
* Add eslint deps, remove old lint deps
* Add eslint config, remove old lint configs
* Config for server and tests are different
* Tweaked rules to suit us
* Fix linting in codebase - lots of indent changes.
* Fix a real broken test
refs #9192
- Each setting is saved individually
- Update this to only happen on import, or when a value changes
- Reduces the amount of work Ghost does on every setting change
