closes#9520
- it contains a dependency bump of the latest Bookshelf release
- Bookshelf introduced a bug in the last release
- see https://github.com/bookshelf/bookshelf/pull/1583
- see https://github.com/bookshelf/bookshelf/pull/1798
- this has caused trouble in Ghost
- the `updated_at` attribute was not automatically set anymore
---
The bookshelf added one breaking change: it's allow to pass custom `updated_at` and `created_at`.
We already have a protection for not being able to override the `created_at` date on update.
We had to add another protection to now allow to only change the `updated_at` property.
You can only change `updated_at` if you actually change something else e.g. the title of a post.
To be able to implement this check i discovered that Bookshelfs `model.changed` object has a tricky behaviour.
It remembers **all** attributes, which where changed, doesn't matter if they are valid or invalid model properties.
We had to add a line of code to avoid remembering none valid model attributes in this object.
e.g. you change `tag.parent` (no valid model attribute). The valid property is `tag.parent_id`.
If you pass `tag.parent` but the value has **not** changed (`tag.parent` === `tag.parent_id`), it will output you `tag.changed.parent`. But this is wrong.
Bookshelf detects `changed` attributes too early. Or if you think the other way around, Ghost detects valid attributes too late.
But the current earliest possible stage is the `onSaving` event, there is no earlier way to pick valid attributes (except of `.forge`, but we don't use this fn ATM).
Later: the API should transform `tag.parent` into `tag.parent_id`, but we are not using it ATM, so no need to pre-optimise.
The API already transforms `post.author` into `post.author_id`.
closes#9507
- Changed the utils.wordCount implementation to the one used by simpleMDE
- Added extra À-ÿ to the regex to support diacritics characters
- Added corresponding text with Chinese text mentioned in the issue
closes#9495
- Added a clause for amp being disabled
- In this clause, we strip the final 'amp/' part of the url, and redirect
- Changed corresponding test in frontend_spec.js
- Used `urlService.utils.redirect301()` instead of `res.redirect()`
refs https://github.com/TryGhost/Ghost/issues/9311
- very basic implementation, still needs proper classes and default stylesheet implementation
- change image card output to a `<figure>` with optional `<figcaption>`
- add optional `<p>` caption output to the html card
refs #9200
- We have not yet counted the images within your html, this commit counts images based on the this algorithm: https://blog.medium.com/read-time-and-you-bc2048ab620c
- Added imageCount utility, which counts images using an img-tag regex, amended from the general tag-regex found in wordCount
- Added this imageCount to the {{reading_time}} helper, adding 12 seconds to the reading time for every image
- The feature image is still counted as before
- The first image adds 12 seconds, the second 11, the third 10, and so on
- Images from the tenth onwards add 3 seconds to the reading time
closes#9085
Fixes an issue, where the client sets image properties to `""` after deleting the image. This causes problems with the query filter (see https://github.com/TryGhost/GQL/issues/24), as they have to be `null`.
Added a check in the model layer saving method to set value to `null`, when the property is empty.
Affected models and properties:
- `posts`:
- `feature_image`
- `og_image`
- `twitter_image`
- `users`:
- `profile_image`
- `cover_image`
- `tags`:
- `feature_image`
no issue
- the tests were failing since beginning of March
- this was caused by a wrong assertion in one of our authentication tests
- we work with a static 6 month ms number for token expiry
- this static ms number is based on 30 days per month
no issue
- extended functionality
- the knex mock simply parses the sql statements and serves data from memory
- i've tested the memory mode of sqlite, but could not get it working
- but maybe for the future to test again
no issue
- the handling here was not correct
- if you've passed no mobiledoc, it wasn't adding mobiledoc and an undefined html value
- we need a default mobiledoc+html value in case you don't pass the values within the test cases
no issue
- `post.author_id` has no reference to any table currently, see https://github.com/TryGhost/Ghost/blob/1.21.3/core/server/data/schema/schema.js#L19
- that's why it is right now possible to insert none existent author id's
- with multiple authors, this get's protected (see https://github.com/TryGhost/Ghost/pull/9426)
- you would get a proper error message
- it is not allowed to insert invalid author id's
- as soon as you do `include=author` you would receive an error
- fixed one test case where we inserted an invalid author id via the API
no issue
- just discovered that we had confusing function names in our test utility
- e.g. `posts` -> default posts from the data generator
- e.g. `users` -> extra users not from our data generator
- now:
- e.g. `posts` -> default posts from the data generator
- e.g. `users` -> default users from the data generator
- e.g. `users:extra` -> extra users not from our data generator
no issue
- currently if you would like to edit a resource (e.g. post) and you pass an invalid model id, the following happens
- permission check calls `Post.permissible`
- the Post could not find the post, but ignored it and returned `userPermissions:true`
- then the model layer is queried again and figured out that the post does not exist
- A: there is no need to query the model twice
- B: we needed proper error handling for post and role model
no issue
- replace logic for preparing nested tags
- if you have nested tags in your file, we won't update or update the target tag
- we simply would like to add the relationship to the database
- use same approach as base class
- add `posts_tags` to target post model
- update identifiers
- insert relation by foreign key `tag_id`
- bump bookshelf-relations to 0.1.10
no issue
- change behaviour from updating user references after the actual import to update the user reference before the actual import
- updating user references after the import is way less case intense
- that was the initial decision for updating the references afterwards
- but that does not play well with adding nested relations by identifier
- the refactoring is required for multiple authors
- if we e.g. store invalid author id's, we won't be able to add a belongs-to-many relation for multiple authors
- bookshelf-relations is generic and always tries to find a matching target before attching a model
- invalid user references won't work anymore
- this change has a very good side affect
- 17mb takes on master ~1,5seconds
- on this branch it takes ~45seconds
- also the memory usage is way lower and stabler
- 40mb takes 1,6s (times out on master)
no issue
- Ghost does not support adding an author by relation (`post.author = {id: '..'}`)
- Ghost does not support editing an author by relation (`post.author = {id: '..'}`)
- only `author_id` is allowed
refs https://github.com/TryGhost/Ghost/issues/3658
- the `validateSchema` helper was a bit broken
- if you add a user without email, you will receive a database error
- but the validation error should catch that email is passed with null
- it was broken, because:
- A: it called `toJSON` -> this can remove properties from the output (e.g. password)
- B: we only validated fields, which were part of the JSON data (model.hasOwnProperty)
- we now differentiate between schema validation for update and insert
- fixed one broken import test
- if you import a post without a status, it should not error
- it falls back to the default value
- removed user model `onValidate`
- the user model added a custom implementation of `onValidate`, because of a bug which we experienced (see https://github.com/TryGhost/Ghost/issues/3638)
- with the refactoring this is no longer required - we only validate fields which have changed when updating resources
- also, removed extra safe catch when logging in (no longer needed - unit tested)
- add lot's of unit tests to proof the code change
- always call the base class, except you have a good reason
no issue
- `isNew` does not work in Ghost, because Ghost does not use auto increment id's
- see https://github.com/bookshelf/bookshelf/issues/1265
- see https://github.com/bookshelf/bookshelf/blob/0.10.3/src/base/model.js#L211
- we only had one occurance, which was anyway redundant
- if you add a user, `hasChanged('password') is true
- if you edit a user and the password has changed, `hasChanged('password')` is true as well
NOTE #1:
1. We can't override `isNew` and throw an error, because bookshelf makes use of `isNew` as well, but it's a fallback if `options.method` is not set.
2. It's hard to re-implement `isNew` based on `options.method`, because then we need to ensure that this value is always set (requires a couple of changes)
NOTE #2:
If we need to differentiate if a model is new or edited, we should manually check for `options.method === insert`.
NOTE #3:
The unit tests are much faster compared to the model integration tests.
I did a comparision with the same test assertion:
- unit test takes 70ms
- integration test takes 190ms
no issue
- added https://github.com/colonyamerican/mock-knex as dev dependency
- the mock serves our data generator test data by default
- but you can define your own if you want
- we need a proper mock for unit testing
- we should not mock bookshelf if possible, otherwise we can't test event flows
no issue
- move password hashing and password comparison to lib/security/password
- added two unit test
- FYI: password hashing takes ~100ms
- we could probably mock password hashing in certain cases when unit testing
no issue
- this commit cleans up the usages of `include` and `withRelated`.
### API layer (`include`)
- as request parameter e.g. `?include=roles,tags`
- as theme API parameter e.g. `{{get .... include="author"}}`
- as internal API access e.g. `api.posts.browse({include: 'author,tags'})`
- the `include` notation is more readable than `withRelated`
- and it allows us to use a different easier format (comma separated list)
- the API utility transforms these more readable properties into model style (or into Ghost style)
### Model access (`withRelated`)
- e.g. `models.Post.findPage({withRelated: ['tags']})`
- driven by bookshelf
---
Commits explained.
* Reorder the usage of `convertOptions`
- 1. validation
- 2. options convertion
- 3. permissions
- the reason is simple, the permission layer access the model layer
- we have to prepare the options before talking to the model layer
- added `convertOptions` where it was missed (not required, but for consistency reasons)
* Use `withRelated` when accessing the model layer and use `include` when accessing the API layer
* Change `convertOptions` API utiliy
- API Usage
- ghost.api(..., {include: 'tags,authors'})
- `include` should only be used when calling the API (either via request or via manual usage)
- `include` is only for readability and easier format
- Ghost (Model Layer Usage)
- models.Post.findOne(..., {withRelated: ['tags', 'authors']})
- should only use `withRelated`
- model layer cannot read 'tags,authors`
- model layer has no idea what `include` means, speaks a different language
- `withRelated` is bookshelf
- internal usage
* include-count plugin: use `withRelated` instead of `include`
- imagine you outsource this plugin to git and publish it to npm
- `include` is an unknown option in bookshelf
* Updated `permittedOptions` in base model
- `include` is no longer a known option
* Remove all occurances of `include` in the model layer
* Extend `filterOptions` base function
- this function should be called as first action
- we clone the unfiltered options
- check if you are using `include` (this is a protection which could help us in the beginning)
- check for permitted and (later on default `withRelated`) options
- the usage is coming in next commit
* Ensure we call `filterOptions` as first action
- use `ghostBookshelf.Model.filterOptions` as first action
- consistent naming pattern for incoming options: `unfilteredOptions`
- re-added allowed options for `toJSON`
- one unsolved architecture problem:
- if you override a function e.g. `edit`
- then you should call `filterOptions` as first action
- the base implementation of e.g. `edit` will call it again
- future improvement
* Removed `findOne` from Invite model
- no longer needed, the base implementation is the same
no issue
- Date comparisons are possible via API, but there's no way to inject a valid date into the get helper
- JavaScript's Date.toString() function outputs dates in a useless format
- Swap to using Date.toISOString() and now the format can be understood anywhere!
- {{#get "posts" filter="published_at:<='{{published_at}}'"}}{{/get}} works now as expected
closes#9445
- redirects all asset requests if https is configured (theme, core, images)
- re-use and extend our url-redirect middleware
- add proper integration tests for our express site app (no db interaction, component testing required for such important use cases)
- i added some more general tests
- should avoid mixed content warnings in the browser
closes#9314
* added fixtures for contributor role
* update post api tests to prevent contributor publishing post
* update permissible function in role/user model
* fix additional author code in invites
* update contributor role migration for knex-migrator v3
* fix paths in contrib migration
* ensure contributors can't edit or delete published posts, fix routing tests [ci skip]
* update db fixtures hash
* strip tags from post if contributor
* cleanup post permissible function
* excludedAttrs to ignore tag updates for now (might be removed later)
* ensure contributors can't edit another's post
* migration script for 1.21
no issue
- returning and remembering the data, which was imported, is...
- not required when using the API
- not required when importing via script
- required for tests
- added an option to have control over it
- make more usage of local variables
- the GC cannot tidy up variables, which are defined outside of a loop, but used in the loop
- try to keep less memory in process
- reduce the number of properties we have to remember
no issue
- if you import a JSON file with a post, which has an unknown author,
the target user was removed from the blog
- Ghost can handle this case and still succeeds with import
- but we have stored an `author_id` in the database, which does not map to any user and won't map in the future
- this can trouble if we add support for multiple authors
- currently, we only return the `author_id` to the client and the client can map with `author_id` with users fetched by the API
- if it does not find a user, it just falls back to a different user
- but multiple authors have to be included explicit (`include=authors`) and we will return a mapped (author_id => user) result
- it won't be able to find the user, because we lookup the database
- this would result in an error
- there is in general no reason to import (or store) an unknown/invalid `author_id` into the database
- on import, we show you a warning and you can choose a different author if you want
- solution: fallback to owner user and extend warning
- it's not a behaviour change, you still can import unknown author id's and the import won't fail
- but we ensure valid author id's
- updated test
- further more: returning `author={}` when requesting `include=author` could trouble with ember currently
- it expects the author to be returned
refs #9127
- permission checks can happen everywhere in the code base
- we would like to create a context class
- global access to `options.context.is(...)`
- please read more about the access plugin in #9127 section "Model layer and the access plugin".
- removed the plugin and use direct context checks
no issue
- the affected test had a wrong payload, but passed anyway because the previous test edited the same title ;)
- this routing test does currently truncate the tables after each test case
- the test can run faster
- if we achieve reducing the routing tests, we can reconsider truncating the db
no issue
- reported in slack (https://ghost.slack.com/files/U8QV8DXQB/F8TSBQ532/image.png)
- do not expose old release notification
- e.g. you are on 1.20.0
- you receive a notification for 1.20.1 to update
- you update to 1.20.1
- ensure we protect exposing the release notification (compare against blog version)
- protect against wrong formats
- @TODO: the notifications could store a `version` property
- by that we could use `notification.version` and don't have to match the version in the message
closes#5071
- Remove hardcoded notification in admin controller
- NOTE: update check notifications are no longer blocking the admin rendering
- this is one of the most import changes
- we remove the hardcoded release message
- we also remove adding a notification manually in here, because this will work differently from now on
-> you receive a notification (release or custom) in the update check module and this module adds the notification as is to our database
- Change default core settings keys
- remove displayUpdateNotification
-> this was used to store the release version number send from the UCS
-> based on this value, Ghost creates a notification container with self defined values
-> not needed anymore
- rename seenNotifications to notifications
-> the new notifications key will hold both
1. the notification from the USC
2. the information about if a notification was seen or not
- this key hold only one release notification
- and n custom notifications
- Update Check Module: Request to the USC depends on the privacy configuration
- useUpdateCheck: true -> does a checkin in the USC (exposes data)
- useUpdateCheck: false -> does only a GET query to the USC (does not expose any data)
- make the request handling dynamic, so it depends on the flag
- add an extra logic to be able to define a custom USC endpoint (helpful for testing)
- add an extra logic to be able to force the request to the service (helpful for testing)
- Update check module: re-work condition when a check should happen
- only if the env is not correct
- remove deprecated config.updateCheck
- remove isPrivacyDisabled check (handled differently now, explained in last commit)
- Update check module: remove `showUpdateNotification` and readability
- showUpdateNotification was used in the admin controller to fetch the latest release version number from the db
- no need to check against semver in general, the USC takes care of that (no need to double check)
- improve readability of `nextUpdateCheck` condition
- Update check module: refactor `updateCheckResponse`
- remove db call to displayUpdateNotification, not used anymore
- support receiving multiple custom notifications
- support custom notification groups
- the default group is `all` - this will always be consumed
- groups can be extended via config e.g. `notificationGroups: ['migration']`
- Update check module: refactor createCustomNotification helper
- get rid of taking over notification duplication handling (this is not the task of the update check module)
- ensure we have good fallback values for non present attributes in a notification
- get rid of semver check (happens in the USC) - could be reconsidered later if LTS is gone
- Refactor notification API
- reason: get rid of in process notification store
-> this was an object hold in process
-> everything get's lost after restart
-> not helpful anymore, because imagine the following case
-> you get a notification
-> you store it in process
-> you mark this notification as seen
-> you restart Ghost, you will receive the same notification on the next check again
-> because we are no longer have a separate seen notifications object
- use database settings key `notification` instead
- refactor all api endpoints to support reading and storing into the `notifications` object
- most important: notification deletion happens via a `seen` property (the notification get's physically deleted 3 month automatically)
-> we have to remember a seen property, because otherwise you don't know which notification was already received/seen
- Add listener to remove seen notifications automatically after 3 month
- i just decided for 3 month (we can decrease?)
- at the end it doesn't really matter, as long as the windows is not tooooo short
- listen on updates for the notifications settings
- check if notification was seen and is older than 3 month
- ignore release notification
- Updated our privacy document
- Updated docs.ghost.org for privacy config behaviour
- contains a migration script to remove old settings keys
refs #5345, refs #3801
- Blog localisation
- default is `en` (English)
- you can change the language code in the admin panel, see https://github.com/TryGhost/Ghost-Admin/pull/703
- blog behaviour changes depending on the language e.g. date helper format
- theme translation get's loaded if available depending on the language setting
- falls back to english if not available
- Theme translation
- complete automatic translation of Ghost's frontend for site visitors (themes, etc.), to quickly deploy a site in a non-English language
- added {{t}} and {{lang}} helper
- no backend or admin panel translations (!)
- easily readable translation keys - very simple translation
- server restart required when adding new language files or changing existing files in the theme
- no language code validation for now (will be added soon)
- a full theme translation requires to translate Ghost core templates (e.g. subscriber form)
- when activating a different theme, theme translations are auto re-loaded
- when switching language of blog, theme translations are auto re-loaded
- Bump gscan to version 1.3.0 to support more known helpers
**Documentation can be found at https://themes.ghost.org/v1.20.0/docs/i18n.**
closes#9022
Images without extensions don't need to be manipulated, as we're now reading the bytes and pass those to the `image-size` lib.
This PR adds another `user-agent` to emulate multiple browser requests, as I stumbled over an example where the image without extension is protected otherwise.
Added a test, that works with above mentioned image, but is currently mocked. Nevertheless, the image worked as a PoC, that we're able to read the bytes of an image without its extension and still return the dimensions of the image.
refs https://github.com/TryGhost/Ghost-Release/issues/24
- differentiate between
1. original package.json version (can contain pre and build suffix)
2. full package.json version X.X.X-{pre} (optional)
3. safe package.json version X.X (major+minor)
no issue
- with 29e143fa9a import queries no longer run in parallel
- this commit simply adds a small code snippet to reflect the importer behaviour
1) duplicate slugs *within* a file are getting ignored
2) existing posts in the database and posts to import with the same slug, result in duplicates
Further improvements regarding duplication detection will happen via #8717.
closes#8717
- this is now required, because we run import queries sequentiell
- this code protects two cases:
- you have duplicate slugs in the JSON file (the first get's inserted, the second get's ignored)
- you have an existing slug in the database and you try to import the same slug, get's ignored
refs #9178, refs #8988
With 7353c87d7f we use Bluebird globally for Promises. Therefore, the request lib doesn't need to be wrapped in a bluebird Promise anymore.
This was originally done, so we can work with catch predicated in our image-size lib.
Updated the tests to proof, that the catch predicates work.
The tests fail, as soon as the Promise overwrite is commented out.
refs #9178
- this util uses the url services (!)
- moving this file into lib would not make sense right now
- that would mean a module requires first ../lib/url, which then requires ../services/url
- the url service definitely need a clean up 😃
refs #9178
- not 100% sure about this, but i think it makes right now the most sense
- we have already a url service and creating another lib/url is confusing at the moment
- i'll copy the last utility `makeAbsoluteUrls` to the url service for now
- see next commit for explanation (!)
