closes#9972
* Added breaking test for node v6 session auth
* Updated session middleware to support node v6
This uses the legacy url to obtain the origin rather than the WHATWG
URL class in order to support node <6.14.4
closes#9927
- Added post model implementation to be able to store up to 10 versions of mobiledoc
- Bumped GQL to support filtering on the mobiledoc revision table
- Added tests ensuring new functionality works
* Added api_key_id to frame.context
refs #9865
This is to allow controllers to check permissions using api_key_id data.
* Removed client and client_id from frame.context
refs #9865
This is unused as we only support oauth on v0.1 API.
* Fixed indentation for login function
no-issue
* Updated login to work with session auth
refs #9866
This allows the 201 response from session auth and will resolve with
the cookie, or the access_token if the cookie does not exist.
* Added basic doAuth to v2 admin test utils
refs #9866
This will allow functional tests to use session based authentication.
closes#9959
This issue existed because the logic assumed that if there were no
query parameters then there would be no `query` object. However this is
not the case. What we really wanted to check was for the existence of an
"r" query param - the code has been refactor to explicitly do this now.
refs #9866
- just some 🤪
- some were really slow in general, because they had to start/stop Ghost within the test file twice or so
- removed some obvious test cases
- if we copy over the controllers and add functional tests for v2, we can maybe remove some more test cases for v2
refs #9866
- test/functional/
- test/functional/api
- test/functional/api/v0.1
- test/functional/api/v0.1/utils
- test/functional/api/v2
- test/functional/api/v2/admin
- test/functional/api/v2/admin/utils
- test/functional/api/v2/content
- test/functional/api/v2/content/utils
- updated grunt file
- instead of `grunt test-routes`, you now need to use `grunt test-functional` (docs are updated)
You can use `localUtils.API.getApiQuery('posts/')` and it will generate the correct API url.
refs #9866
- moved the tests either to unit tests or routing tests
- or removed test case (a lot)
- this commit is very big 🤪, it was not rly possible to create clean commits for this
- it only changes the test env, no real code is touched
Next steps:
- optimise folder structure + make v2 testing possible
- reduce some more tests from routing and model integeration tests
refs #9866
- prep for v2
- you can better unit test the permissible function
- this avoids copying over the permission handling to v2 controller
- it was possible to move this logic into the model layer, because we now support `unsafeAttrs`
no issue
- last commit failed on mysql
- reverted added a new user to knex test utility
- added the user in the target test
@TODO: we should in the future add resources per test, without using a "global" set,
otherwise you always run into trouble that you have to update a lot of tests when you add a new resource to the test set
refs #9866
- try to keep API interactions for routing tests in a single place, because it gives a better overview
- tiny improvement
- there are lot's of other things we could do, but we want to limit the changes for now
Goal:
e2e/api/v0.1
utils (local)
e2e/api/v2
utils (local)
utils
api (shared)
refs #9866
- the global utility should for now not generate API urls
- we only have one test which generates the API url
- for now it will live in this test file
- if we have multiple cases, we can reconsider this
refs #9866
- the sdk does not use the url utility
- it just respects what you pass in
- there is no need to test this behaviour for multiple API versions
refs #9866
- we want to keep shared API helpers
- the local utility just wraps this call and forwards the correct url
- ability to override the behaviour completely
refs #9866
- we would like to avoid doing for each api version e.g.
testUtils.API.getQueryUrl('webhooks/', v2)
testUtils.API.getQueryUrl('webhooks/', v3, admin)
- the URL should be easily accessible per api version routing folder
- so i thought it makes the most sense to add a local utils file
e.g. localUtils.API.getQueryUrl('webhooks/') => returns target api url for target api folder by default
e.g. localUtils.API.content.getQueryUrl('webhooks/') => returns content api url for target api folder
- it also makes it easy for us to copy the routing tests over without changing anything
- we just need to provide the local utility
no issue
- why? this is a unit test (!)
- we start+stop Ghost in the routing tests a lot, this implicit tested
- it has no priority for now to move this test to a unit test
- this was the only module test, removed related grunt tasks
refs #9326, refs #9866
**ATTENTION: This is the first iteration. Bugs are expected.**
Main Goals:
- add support for multiple API versions.
- do not touch v0.1 implementation
- do not break v0.1
## Problems with the existing v0.1 implementation
1. It tried to be generic and helpful, but it was a mixture of generic and explicit logic living in basically two files: utils.js and index.js.
2. Supporting multiple api versions means, you want to have as less as possible code per API version. With v0.1 it is impossible to reduce the API controller implementation.
----
This commit adds three things:
1. The tiny framework with well-defined API stages.
2. An example implementation of serving static pages via /pages for the content v2 API.
3. Unit tests to prove that the API framework works in general.
## API Stages
- validation
- input serialization
- permissions
- query
- output serialization
Each request should go through these stages. It is possible to disable stages, but it's not recommended.
The code for each stage will either live in a shared folder or in the API version itself. It depends how API specific the validation or serialization is. Depends on the use case.
We should add a specific API validator or serializer if the use case is API format specific.
We should put everything else to shared.
The goal is to add as much as possible into the shared API layer to reduce the logic per API version.
---
Serializers and validators can be added:
- for each request
- for specific controllers
- for specific actions
---
There is room for improvements/extensions:
1. Remove http header configuration from the API controller, because the API controller should not know about http - decouple.
2. Put permissions helpers into shared. I've just extracted and capsulated the permissions helpers into a single file for now. It had no priority. The focus was on the framework itself.
etc.
---
You can find more information about it in the API README.md (api/README.md)
- e.g. find more information about the structure
- e.g. example controllers
The docs are not perfect. We will improve the docs in the next two weeks.
---
Upcoming tasks:
- prepare test env to test multiple API versions
- copy over the controllers from v0.1 to v2
- adapt the v2 express app to use the v2 controllers
refs #9866
- Added test util method for api path based on version
- Updated all hardcoded v0.1 tests strings to use dynamic string from util method
- Updated hardcoded v0.1 tests using regex match to use string equal with new util method
refs #9865
Note that this controller is the singular, that's because we plan to
make a session resource controller to be used with /sessions, wheras
this is on /session
Moved URL attributes logic from the model into API layer
refs #9866
- Moved URL related attribute calculation for posts, users, and tags into API layer
- Added test coverage for url attributes in tags/authors/primary_tags/primary_authors
no issue
- support promise and none promise tasks
- helpful if you create an array of operations and not all of the operations/tasks are async
- `response instanceof Promise` does not work for all cases e.g. some usages return a transaction/bookshelf chain
refs https://github.com/TryGhost/Ghost/issues/9865
- schema migrations
- adds `integrations` and `api_keys` tables
- inserts `integration` and `api_key` permissions and Administrator role relationships
- inserts `Admin Integration` role and permissions
- adds `Integration` model
- adds `ApiKey` model
- creates default secret if not given
- hardcodes associated role based on key type
- `admin` = `Admin API Client`
- `content` = no role
- updates `Role` model to use `bookshelf-relations` for auto cleanup of permission relationships on destroy
refs #9865
* This service handles the session store and exporting middleware to be
used for creating and managing sessions
* Updates the auth service index.js file in line with how we do things elsewhere
* After wrapping the exports in a getter, the usage of rewire had broken
the authenticate tests, this commit _removes_ rewire from the tests, calls `init` on
the models before the tests (needed because rewire isn't there) and also
cleans up the use of var.
refs #9866
- preparation for v2
- moved api/ to api/v0.1
- do export v0.1 straight from the api folder, we don't want to touch this right now
- that means currently if you require the api folder, we return v0.1 by default
- there were some direct requires of api files in the test env
- some of them use rewire
- for now, we just correct the require path to require api/v0.1/
- we touch the test env next week
**Docs about V2 design are coming soon!**
refs #9866
- Removed `toJSON` call in `findPage`
- Added JSON serialization on API layer
- Reason: model and api layer were coupled - all other model actions just returned the raw data and no specific format
- Corrected test suites to serialize fetched models to JSON
- Removed `absolute_urls` attribute from validOptions findPage methods as it's no longer needed in the data layer
- Changed 'include' test as this option is now tolerated and returns data
refs #9866
- added api version config to overrides, which makes it possible to have a centralized api versioning configuration
- the next PR will use this config in the web folder
- make api url generation in url service flexible and dynamic
- remove hardcoded API_PATH
- updated all places which used `urlFor('api'..)` -> we now ask for explicit api version
refs #8576
- adds new API endpoint `/uploads/profile-image` for uploading profile images
- new validation which fails with error message if uploaded image is not square
- Renamed getImageSizeFromFilePath to getImageSizeFromStoragePath, because it's more explicit
- Add new getImageSizeFromPath method, which is used in the new dimensions middleware
- Ensure we use the sharp middleware to auto-resize the uploaded profile pictures
- Ensure the new route get's added to v2
While this makes sure all future profile images uploaded are square, this doesn’t affect any existing non-square profile image. Needs more thought on how to handle existing non-square profile images for the purpose of making theming easier in future.
Extended uncapitalise unit tests
refs #9866
- Proved that the middleware works for many API versions
- Added test case to prove that the version identifier gets lowercased too
refs #9866
- Removed `res.isAdmin` flag in v2 express app
- Did not touch v0.1 express app
- Separated url redirect middleware for admin and content API
refs #9866
- Registered Content API under /ghost/api/v2/content/
- Registered Admin API under /ghost/api/v2/admin/
- Moved API v0.1 implementation to web/api/v0.1
- Created web/api/v2 for the new api endpoints
- Started with reducing the implementation for the new Content API (the Content api does not serve admin api endpoints, that's why it was reducible)
- Covered parent-app module with basic test checking correct applications/routes are being mounted
- Added a readme file, which contains a warning using v2, because it's under active development!
- This PR does only make the new endpoints available, we have not:
- optimised the web folder (e.g. res.isAdmin)
- started with different API controllers
- reason: we want to do more preparation tasks before we copy the api controllers
closes#9749
- disallow indexing of /p/ in robots.txt
- add meta robots tag to preview pages
- robots.txt wont' be applied for blogs not mounted to the root (i.e. https://example.com/blog/)
no issue
- private blogging was once part of a single big express app, see: c1a2601514
- therefor we needed to differentiate if the request is for /api/v0.1 or for the actual site
- nowadays, we register apps only for the site express app
refs #9848
- Disabled image optimization for .gif files as a temporary solution until sharp library starts supporting latest version of libvips
- Disabled image optimization for .svg/.svgz files as permanent solution as these file types are being converted to .png
closes#9832
The API _should_ be returning absolute URLs for everything, 3rd party applications require absolute urls to read and display ghost data correctly. Currently they have to concat the blog url and the resource url, which is very uncomfortable.
Changing the public api like this would be considered a breaking change however so we've opted to put it behind a query parameter named `absolute_urls`.
refs #4453
* On by default
* Added config to disable resizing
* Added basic image optimization processing
* Added dep: sharp (optional dep)
* Added resize middleware
* Take care of rotation based on EXIF information
* Removed all meta data from optimised image
* Added handling if sharp could not get installed
* Do not read ext twice - optimisation
* Do not call sharp if config is disabled
* Do not remove the original image which was uploaded (store 2 images)
* Support of `req.files` for internal logic
* Disabled cache to enable file removal on Windows
closes#9134
- Added form_id, input_id, and button_id parameters to subscribe_form helper
- Added id parameter to input_email helper
- Added test coverage to input_email helper
- Added quotes to id attributes for consistency
- Added subscribe_form helper tests
- Updated express to v4 in helper tests
closes#9822
- Fixed the post count issue for co authors
- Corrected and refactored tests related to users post count
- Consistency fix, because we return all posts where the author is primary or co author for the author page already
closes#9786
- Make GET request when url has no provider match
- The HEAD request was made in order to send less data over the wire when
checking for redirects for urls that do not have an oembed provider
match. We are now going to look for provider metatags withing the
response of the request - rather than making a HEAD followed by a GET if
no redirect is found, this condenses that to a single request.
- Try to get OEmbed data from tag if no provider
- Here we parse the HTML response of the resource and look for a link tag
that will give us the oembed resource url which we can use to fetch the
embed html
closes#9644
- Removed google blogsearch from pingList, because it's dead (https://plus.google.com/+GoogleWebmasters/posts/46W7ZwVrwqg)
- Updated pingomatic method weblogUpdate -> weblogUpdates
- Updated RPC response handler to check for errors, log improvement
- the pingomatic service sends back a 200 even when it errors, so we
check the xml response to see if it's good, it throws and passes of to
the catch handler already in place
- Avoid multiline XML error message strings, but
includes a catch in case our regex stops working with a fallback to
multiline XML error message
- we check for arbitrary whitespace between different XML tags,
which I haven't seen in any of the responses - but it could change I
guess. I haven't added support of whitespace between the tags, as I
believe that would be a different value with XML spec. And we wanna
match on exact values here.
no issue
- we have to explicitly reset the previous `updated_at` field, because Bookshelf auto-updates this field on each update
- we have to extend the condition to avoid updating the `updated_by` field
- detect and respect `options.migrating`
* 🐛 Dynamic Routing Beta: Shortform `author.foo` is not allowed
refs #9601
- otherwise you get access to {{author}} in the theme, which is deprecated & causes errors
- recommend not using {{author}} as data longform name in the yaml validator
- (internal usage is still allowed)
- also warn against using reserved data key names like filter, resource, limit etc - maybe remove this restriction later but seems like a sensible validation right now.
no issue
- Organising your content
From: Here, the theme would assign the post publicly displayed tags of Blog - but it would also keep a private record of the post being tagged with #video.
To: Here, the theme would assign the post publicly displayed tags of News - but it would also keep a private record of the post being tagged with #video.
refs #9601
- when using the short form `data: tag.welcome` the redirect is enabled by default
- /tag/welcome/ will redirect to the channel/collection which makes use of the data key
- you can disable the redirect by using the long form
e.g. data:
tag:
resource: tags
type: read
slug: welcome
redirect: false
closes#9774, refs #9742
- added new fixture posts for Ghost 2.0
- added migration file to remove old fixture posts
- only remove them if they are owned by the Ghost author and if they are tagged with getting-started
- added new fixture posts if you had all (!) old fixture posts
- ensure on rollback we remove the new fixture posts again
- updated default settings
refs #9742
- removed usage of single permalink setting
- with dynamic routing this configuration does no longer makes sense
- because you can configure your permalinks in the routes.yaml
- furthermore you can have multiple collections with multiple permalinks
- removed @blog.permalinks
- do not export permalink setting
- do not import permalink setting
- permalink setting UI will be removed soon
- get rid of {globals.permalink} completely
- remove yaml in-built migration
- do not expose settings.permalinks via the private API
- do not allow to edit this setting
- keep phyiscal value in case a blog needs to rollback from v2 to v1
- sorted out when the routers should be created
- ensure routes.yaml file doesn't get validated before Ghost is fully ready to start
refs #9742, refs #9724
- handle König Editor format for 2.0
- adapted importer to be able to import 1.0 and 2.0 exports
- added migration scripts
- remove labs flag for Koenig
- migrate all old editor posts to new editor format
- ensure we protect the code against mobiledoc or html field being null
- ensure we create a blank mobiledoc structure if mobiledoc field is null (model layer)
- ensure you can fully rollback 2.0 to 1.0
- keep mobiledoc/markdown version 1 logic to be able to rollback (deprecated code)
refs #9744
- added two new endpoints to upload/download routes.yaml
- reload site express app on successful/valid upload
- reload url service on sucessfuly upload
- force clear cache of pages
- ensure we keep a backup of the routes.yaml file
- this feature was mostly tested manually
- @TODO: i have to write unit tests - will do later
- @TODO: do a memory test to ensure we haven't introduced any memory leaks with this feature
refs #9742
- rebase against master updated some docs links again
- go over code base again and double check that all docs links are correct
- 2.0 will become the latest version on our readme pages
no issue
- these tests were a mess
- we had many duplicated tests
- it was very hard to work with the exporter files
- i have tidied up the whole file
- first section: all general importer tests based on the current Ghost version
- second section: 1.0 tests
- everything is now JSON based (much easier to control)
refs 9742
- when we've introduced Ghost 1.0, we have noticed that we broke Disqus comments
- Disqus comments use a unique identifier, which is the post id
- that means if you have exported your LTS content and imported it into 1.0, all resource identifiers are regenerated
- but the Disqus must use the original post resource id
- that's why we have imported the old post id and remembered it in the `amp` field 🤠
- that was the only field which was available and un-used
- now in Ghost 2.0, we would like to rename the `amp` field to `comment_id`
- string length: 50 (i thought it might be safer to not use 24 characters, because who knows if support other comment id's in the future)
refs https://github.com/TryGhost/Ghost/issues/9724
- perform a HEAD request on a url if we don't find a matching provider, following any redirects until we hit success response before looking up providers for the resulting url
refs #9601
- on resource update/add we have cached mobiledoc, html etc
- we have to ensure we exclude the fields (same procdure happens on bootstrap)
- these excluded fields don't have to be cached
- otherwise memory usage is higher in general
- ensure we cache relations with a minimal field set on resource update/add
refs #9742, refs #8719
- make it possible to import more tables (optional)
- available tables: clients, trusted domains
- by default we won't import these tables, you have to tell Ghost using `include` (same syntax on export)
- we won't announce this ability for now (stays hidden)
refs #9742, refs #8719
- you can now use `include` to export extra tables e.g. `include=clients`
- admin client won't make use of this option yet, maybe later and optional
- we won't announce this new ability for now (stays hidden)
no issue
- add a new migration for 1.25 to insert the draft demo post for existing blogs
- ensure new blogs get the draft demo post as well
- tested on sqlite3 + mysql
- added handling if Ghost Author user doesn't exist anymore (fallback to owner user)
refs https://github.com/TryGhost/Ghost/issues/9742
We've identified some changes we need to make to the HTML output of the [new Koenig editor](
https://forum.ghost.org/t/koenig-editor-beta-release/1284/102) for future proofing and consistency across cards.
- the `<div class="kg-post">` wrapper around post content has been removed
- for image cards the `.kg-image-wide` and `.kg-image-full` classes have been changed to `.kg-width-wide` and `.kg-width-full` and applied to the `<figure>` element rather than the `<img>` element
Before:
```html
<div class="kg-post">
<figure class="kg-image-card">
<img class="kg-image kg-image-wide" src="...">
<figcaption>example wide image</figcaption>
</figure>
</div>
```
After:
```html
<figure class="kg-image-card kg-width-wide">
<img class="kg-image" src="...">
<figcaption>example wide image</figcaption>
</figure>
```
refs #9742
- Ghost 2.0 is coming
- all doc links in 1.0 must use concrete links e.g. docs.ghost.org/v1 or themes.ghost.org/v1.23.0/
- if we release Ghost 2.0, docs.ghost.org will show 2.0 docs
closes#9727
- this is a short term fix to proof that Disqus comments on preview pages no longer appear on other threads
- this is not a full solution to the problem
- the private API still returns /404/, which is right now inconsistent, but not critical in any way
- the url helper will now output the post preview url if you serve a draft/scheduled post
- this should register unique page urls at Disqus and ensure uniquness for threads
- i still don't understand why the cross posting happens at all, because we also pass an unique identifier to Disqus (the post ID)
- it could be that comments, which are added on the preview page, won't appear on the published urls, because
the published url !== preview url. I wasn't able to figure this out via testing or reading their docs
refs https://github.com/TryGhost/Ghost/issues/9623
- added `DomModifier` class to walk a SimpleDom document and modify as needed
- adds `id` attributes to `h1`, `h2`, etc heading tags
- converts H* tag content to a dasherized string for the id attribute (dasherized id's are different to the smushed ids that are generated by our markdown converted but there are no backwards-compatibility concerns here)
- if a duplicate id is detected then add a `-1`, `-2`, etc suffix to the id
- use `DomModifier` after converting mobiledoc to SimpleDom but before serialising to html
- switched top-level var declarations to es6
refs #9601
- this was already working for collections or channels
- but the `routeName` was not parsed for static routes
- ensure we push the route name into the context object
e.g. /about/: about
-> name of the route is "about"
no issue
- discovered while testing (issue doesn't exist)
- with dynamic routing we have introduced a bug
- we pre-generate urls and if your permalink contains a date (dated permalink),
we have to regenerate the urls, otherwise the dated urls do not respect your blog timezone
- collection router has to subscribe to the timezone event
- collection router must trigger an update on it's url generator if the timezone changes and the it's permalink is dated
- ensure we also update the urls on import
no issue
- reported in the forum: https://forum.ghost.org/t/publishing-with-a-single-post-request-to-posts/1648
- the defaults are defined in two places
1. on the schema level (defaults for the database)
2. on the ORM (model layer)
- the defaults on the db layer are set correctly when inserting a new resource
- but if we don't apply all defaults on the model layer, it will happen that model events are emitted without the correct defaults
- see comment in code base
- it's caused by the fact that knex only returns the inserted resource id (probably caused by the fact knex has to support x databases)
- components/modules are listening on model events and expect:
1. a complete set of attributes
2. a complete set of defaults
3. sanitized values e.g. bool, date
- this commit fixes:
1. added missing defaults for user & post model
2. sanitize booleans (0|1 => false|true)
3. added tests to ensure this works as expected
4. clarfies the usage of `defaults`
Regarding https://forum.ghost.org/t/publishing-with-a-single-post-request-to-posts/1648:
- the post event was emitted with the following values {page: undefined, featured: undefined}
- the urlservice receives this event and won't match the resource against collection filters correctly
- NOTE: the post data in the db were correct
refs #9601
- replace jsonpath with [NQL](https://github.com/NexesJS/NQL)
- jsonpath was just a temporary solution (a short-term fix)
- with NQL we are able to filter collections more powerful in the near future
- NQL is not feature complete
- we still support `featured:true` for collections
refs #9601
- you can now use `rss:false`
- ability to define a custom rss url with a target template (+ content_type)
- ability to disable rss for channel or collection
refs #9601
- refactor architecture of routing so you can define a channel
- a channel is a different way of looking at your posts (a view)
- a channel does not change the url of a resource
Example channel
```
routes:
/worldcup-2018-russia/:
controller: channel
filter: tag:football18
data: tag.football18
```
- added ability to redirect resources to a channel/static route
- support templates for channels
- ensure we still support static routes (e.g. /about/: home)
- ensure pagination + rss works out of the box
refs #9601
- support data, limit and order for collections
- limit definition in routes.yaml is stronger than theme package.json limit configuration
- ensure we update hbs template options
refs #9601
- the home.hbs behaviour for the index collection (`/`) is hardcoded in Ghost
- we would like to migrate all existing routes.yaml files
- we only replace the file if the contents of the routes.yaml file equals the old routes.yaml format (with home.hbs as template)
- updated README of settings folder
- if we don't remove the home.hbs template from the default routes.yaml file, home.hbs will be rendered for any page of the index collection
- the backwards compatible behaviour was different
- only render home.hbs for page 1
- remember: the default routes.yaml file reflects how Ghost was working without dynamic routing
refs #9601
Example:
```
collections:
/podcast/:
permalink: /{slug}/
```
- the name of the collection is remembered as `routerName` (in the case above: "podcast")
- the name of the collection is important for two things
1. context value
2. template name
- the context value is available for specific theme helpers e.g. is helper, body_class helper
- we auto-lookup the collection name in your theme e.g. podcast.hbs
- this logic does not apply to static routes
- if you define templates on your collection, they are stronger than the collection name
refs #9601
- implementation of resource listener updated
- if you define two collections: `featured:true` (1) and `page:false` (2) you can run into the following bug:
- you create a published post (owned by (2))
- you change the status to featured
- still owned by (2), because the filter still matches (it's still not a static page)
- this adaption fixes the behaviour
- less smart logic, but less error prone
closes https://github.com/TryGhost/Ghost/issues/9674
- with dynamic routing the first collection get's the "index" context attached
- the index context signalises the main post listening route (first collection)
- this behaviour was present < 1.24 - we have to keep this behaviour
no issue
- if you define no collections, but a static route, it can happen that the target template to render
makes use of the {{ghost_head}} helper
- the {{ghost_head}} helper tries to create the primary rss feed url
- at the moment: no collections, no primary rss feed url
- if we offer the option to define custom rss rules, this function might need an extension
no issue
- this mock eat already too much of my/our time
- the idea of adding a knex mock was definitely a failed approach/try
- it's too much to maintaince and have not found a module which does this already
- we have to support any query format
- this is too crazy
- the idea was to use the knex mock for model unit tests, because if we want to unit test models we have to
run through bookshelf, because the whole model layer depends on bookshelf e.g. events
- for now we simply use the real database
- we could use the sqlite3 memory mode, but that would mean every unit test runs on sqlite3
- something to consider for later e.g. run unit tests on one matrix
- run the rest on another matrix for sqlite + mysql
closes#9675
- with dynamic routing we have introduced a breaking change, which we have overseen
- Ghost does not return absolute urls, that's why the clients need to concat the blog url and the resource url
- with 1.24.0 Ghost returned resource urls including the subdirectory
- this caused trouble for e.g. zapier or the preview feature in the admin client
- revert breaking change and ensure we only expose resource urls without subdirectory
refs https://github.com/TryGhost/Ghost/issues/9623
- add `oembed-parser` module for checking provider availability for a url and fetching data from the provider
- require it in the `overrides.js` file before the general Promise override so that the `promise-wrt` sub-dependency doesn't attempt to extend the Bluebird promise implementation
- add `/oembed` authenticated endpoint
- takes `?url=` query parameter to match against known providers
- adds safeguard against oembed-parser's providers list not recognising http+https and www+non-www
- responds with `ValidationError` if no provider is found
- responds with oembed response from matched provider's oembed endpoint if match is found
no issue
- was introduced with dynamic routing beta: https://github.com/TryGhost/Ghost/releases/tag/1.24.0
- the slug param wasn't forwarded correctly
- you were not able to render a custom tag or author template e.g. `tag-news.hbs`
closes#9674
- the collection router had a hardcoded default context "home"
- this is wrong
- the context array get's automatically filled for the collection
- if you are serving a page e.g. /page/2/ -> it's "paged"
- if you are serving / -> it's "home"
- same for {{body_class}}, it outputs "home-template" on "/"
- this is the same behaviour as in 1.23.x
no issue
- reverse must happen once in the constructor
- otherwise we reverse the array on each request
- Ghost would randomly pick the first and then the second template
no issue
- from now on: you have to manually reconfigure your slack hook after importing your data
- we were running into trouble that Ghost had import slack hooks, because it can happen very fast
that you are importing someone's slack hook
no issue
- there was a timing bug in Ghost
- we do operations in parallel on bootstrap
- 1) we fetch the resources as early as possible
- 2) we do all the rest (express bootstrapping, theme loading, router registration) etc.
- it can happen that (2) happens too slow and ends in the situation that the queue, which is responsible
to handle both parallel actions, does not wait for the routers and closes the event
- this is a short term fix
- i need to reconsider if there is a better long term fix
refs #9601
### Dynamic Routing
This is the beta version of dynamic routing.
- we had a initial implementation of "channels" available in the codebase
- we have removed and moved this implementation
- there is now a centralised place for dynamic routing - server/services/routing
- each routing component is represented by a router type e.g. collections, routes, static pages, taxonomies, rss, preview of posts
- keep as much as possible logic of routing helpers, middlewares and controllers
- ensure test coverage
- connect all the things together
- yaml file + validation
- routing + routers
- url service
- sitemaps
- url access
- deeper implementation of yaml validations
- e.g. hard require slashes
- ensure routing hierarchy/order
- e.g. you enable the subscriber app
- you have a custom static page, which lives under the same slug /subscribe
- static pages are stronger than apps
- e.g. the first collection owns the post it has filtered
- a post cannot live in two collections
- ensure apps are still working and hook into the routers layer (or better said: and register in the routing service)
- put as much as possible comments to the code base for better understanding
- ensure a clean debug log
- ensure we can unmount routes
- e.g. you have a collection permalink of /:slug/ represented by {globals.permalink}
- and you change the permalink in the admin to dated permalink
- the express route get's refreshed from /:slug/ to /:year/:month/:day/:slug/
- unmount without server restart, yey
- ensure we are backwards compatible
- e.g. render home.hbs for collection index if collection route is /
- ensure you can access your configured permalink from the settings table with {globals.permalink}
### Render 503 if url service did not finish
- return 503 if the url service has not finished generating the resource urls
### Rewrite sitemaps
- we have rewritten the sitemaps "service", because the url generator does no longer happen on runtime
- we generate all urls on bootstrap
- the sitemaps service will consume created resource and router urls
- these urls will be shown on the xml pages
- we listen on url events
- we listen on router events
- we no longer have to fetch the resources, which is nice
- the urlservice pre-fetches resources and emits their urls
- the urlservice is the only component who knows which urls are valid
- i made some ES6 adaptions
- we keep the caching logic -> only regenerate xml if there is a change
- updated tests
- checked test coverage (100%)
### Re-work usage of Url utility
- replace all usages of `urlService.utils.urlFor` by `urlService.getByResourceId`
- only for resources e.g. post, author, tag
- this is important, because with dynamic routing we no longer create static urls based on the settings permalink on runtime
- adapt url utility
- adapt tests
no issue
- discovered while coding
- the value was always false, because we've tried to read the value from the config object
- the value lives in the database and is accessible via the labs service
no issue
- we sanitise any incoming slug on the model layer e.g uppercase -> lowercase
- and when importing e.g. an uppercase slug, the importer was trying to compare the uppercase slug with the sanitised slug
closes https://github.com/TryGhost/Ghost/issues/9620
- adjust the `deduplicateSubDir` function's regex to only match duplicate subdirectories when the `url` is only a path rather than full url or the duplicate match starts with a `/`
closesTryGhost/Support#426
refs TryGhost/gscan#106
needs TryGhost/gscan#107
GScan can return errors, which was not handled in our theme validator and caused Ghost to crash completely. GScan will now return an Ignition error when its not able to read the `.zip` file.
e. g.: `{"errors":[{"message":"Failed to read zip file","context":"tife.zip","errorType":"ValidationError","errorDetails":"invalid relative path: ../tife/"}]}`
refs https://github.com/TryGhost/Ghost/issues/9505
- remove requirement for the `enableDeveloperExperiments` flag to be able to use Koenig
- it's now possible to enable as a standard Labs beta feature
refs https://github.com/TryGhost/Ghost/issues/9505
- updates mobiledoc converter's `render` method to accept a `version` argument
- `1` === Ghost 1.0's markdown-only renderer output
- `2` === Koenig's full mobiledoc renderer output
- switch between mobiledoc renderer versions in Post model's `onSaving` hook
- version 1 by default
- version 2 if Koenig is enabled (currently behind dev experiments config + labs flag)
- version 2 if the post's mobiledoc is not compatible with the markdown-only renderer
- "version 2" full-Koenig mobiledoc renderer output
- wraps content in a `.kg-post` div
- removes wrapper around markdown and html card output
- adds classes to image card output including selected image size/style
- standardises es6 usage across mobiledoc related files
refs #9601
- replace raw knex queries by Bookshelf queries
- optimise lot's of test setups, so we don't experience a massive slow down in the test run
- this has troubled in the past e.g. with normalisation, any custom model logic - the test env always had to simulate things
- there are for sure thousands things which can be optimised now, but because of time, we do them step by step
- this is especially important for the url service (https://github.com/TryGhost/Ghost/issues/9601), because we have to ensure that inserting/updating/removing resources will trigger model events
`grunt test-all` with SQLite finishes in 2,5-3min. (on master: 1-2min)
`grunt test-all` with MySQL finishes in 4min. (on master: 3min)
**NOTE: We want to move as much as possible routing and integration tests to unit tests. This will speed up the test run again.** See #9342. But we need to find time for that. Any help is welcome!
refs https://github.com/TryGhost/Ghost/pull/9592
- we add bookshelf-relations step by step if we need it
- with https://github.com/TryGhost/Ghost/pull/9592 we have rewritten the test env to use Bookshelf
- this is important for our new url service
- because the service is listening on model updates and updates the urls based on the model events
- so with moving to Bookshelf, we need any easy way to add relations
- the test env inserts test fixtures
- it adds permissions and each permission get's roles attached
- `models.Permission.add({roles: [...]})
no issue
- replaced token creation by `lib.common.security`
- added unit tests for adding invites
- allow a different invite status for internal access
refs https://github.com/TryGhost/Team/issues/65
- it's easier for the architecture if we read the setting files synchronously,
because the dynamic routing component is part of the express bootstrap and
the whole routing bootstrap is synchronously
- for now: we only read one file anyway
- it's for now easier to read the file synchronously, then i don't have to change
any existing express bootstrap architecture
refs https://github.com/TryGhost/Team/issues/65
- this is just the first optimisation regarding relative/absolute urls
- the full strike will happen when i start with the url utility re-write
- for now: there will be only one subscriber of url events -> the sitemaps service
- the sitemaps service outputs absolute urls
- we don't want to receive an url event and ask the url service again to get an absolute version of the url
closes#9569
- Removed the `<1 min read` time clause, effectively making `1 min read` the minimum reading time
- Removed the `seconds` option for i18n strings, which contained the less than one minute display string
- Kept the other i18n string options the same
- Amended and improved tests for new functionality
refs https://github.com/TryGhost/Team/issues/65
- currently we generate a relative resource url
- if you configure a subdirectory, the urls have to respect that
- e.g. you configure `localhost:2368/blog`, your url results in e.g. `/blog/my-post/`
- this is not yet a critical bug, because the url service is not connected yet
- @TODO: consider absolute vs. relative urls in the url service
no issue
- removed the `routeKeywords` property from the config and used hard coded keywords.
- removed `routeKeywords` from public configuration API endpoint, as it's no longer used in the Admin.
refs https://github.com/TryGhost/Team/issues/65
We are currently work on dynamic routing (aka channels).
An important piece of this feature is the url service, which always knows the url of a resource at any time.
Resources can belong to collections or taxonomies, which can be defined in a [routing yaml file](https://github.com/TryGhost/Ghost/issues/9528). We are currently shipping portions, which will at end form the full dynamic routing feature.
### Key Notes
- each routing type (collections, taxonomies, static pages) is registered in order - depending on the yaml routes file configuration
- static pages are an internal concept - they sit at the end of the subscriber queue
- we make use of a temporary [`Channels2`](https://github.com/TryGhost/Ghost/pull/9550/files#diff-9e7251409844521470c9829013cd1563) file, which simulates the current static routing in Ghost (this file will be modified, removed or whatever - this is one of the next steps)
- two way binding: you can ask for a resource url based on the resource id, you can ask for the resource based on the url
- in theory it's possible that multiple resources generate the same url: we don't handle this with collision (because this is error prone), we handle this with the order of serving content. if you ask the service for a resource, which lives behind e.g. /test/, you will get the resource which is served
- loose error handling -> log errors and handle instead of throw error and do nothing (we log the errors with a specific code, so we can react in case there is a bug)
- the url services fetches all resources on bootstrap. we only fetch and keep a reduced set of attributes (basically the main body of a resource)
- the bootstrap time will decrease a very little (depending on the amount of resources you have in your database)
- we still offer the option to disable url preloading (in your config `disableUrlPreload: true`) - this option will be removed as soon as the url service is connected. You can disable the service in case you encounter a problem
- **the url service is not yet connected, we will connect the service step by step. The first version should be released to pre-catch bugs. The next version will add 503 handling if the url service is not ready and it will consume urls for resources.**
----
- the url service generates urls based on resources (posts, pages, users, tags)
- the url service keeps track of resource changes
- the url service keeps track of resource removal/insert
- the architecture:
- each routing type is represented by a url generator
- a routing type is a collection, a taxonomiy or static pages
- a queue which ensures that urls are unique and can be owned by one url generator
- the hierarchy of registration defines that
- we query knex, because bookshelf is too slow
- removed old url service files + logic
- added temp channels alternative (Channels2) -> this file will look different soon, it's for now the temporary connector to the url service. Also the name of the file is not optimal, but that is not really important right now.
no issue
- support more cases
- e.g. multiple where matches
- @TODO
- take time to look for NPM module, which does this already
- test sqlite3 :memory: mode again
closes#9528
These code changes introduce a YAML parser which will load and parse YAML files from the `/content/settings` directory. There are three major parts involved:
1. `ensure-settings.js`: this fn takes care that on bootstrap, the supported files are present in the `/content/settings` directory. If the files are not present, they get copied back from our default files. The default files to copy from are located in `core/server/services/settings`.
2. `loader.js`: the settings loader reads the requested `yaml` file from the disk and passes it to the yaml parser, which returns a `json` object of the file. The settings loader throws an error, if the file is not accessible, e. g. because of permission errors.
3. `yaml-parser`: gets passed a `yaml` file and returns a `json` object. If the file is not parseable, it returns a clear error that contains the information, what and where the parsing error occurred (e. g. line number and reason).
- added a `get()` fn to settings services, that returns the settings object that's asked for. e. g. `settings.get('routes').then(()...` will return the `routes` settings.
- added a `getAll()` fn to settings services, that returns all available settings in an object. The object looks like: `{routes: {routes: {}, collections: {}, resources: {}}, globals: {value: {}}`, assuming that we have to supported settings `routes` and `globals`.
Further additions:
- config `contentPath` for `settings`
- config overrides for default `yaml` files location in `/core/server/services/settings`
**Important**: These code changes are in preparation for Dynamic Routing and not yet used. The process of copying the supported `yaml` files (in this first step, the `routes.yaml` file) is not yet activated.
no issue
- required for model events
- otherwise you won't receive a full data set
- in worst case you have to re-fetch the post
- required for the url service
- the url service always needs relations (authors,tags) to be able to generate the url properly
@IMPORTANT
- no API change, we still return what you are asking for
- we first edit/add the resource
- then we fetch the data with the API options
- @TODO: this can be optimised and will improve performance
picking/selecting it from the insert/update response
- this is an internal change
closes#9547
- you setup a blog with the following owner:
- email: test@ghost.org
- name: test
- slug: test
- now you import a JSON db file, which holds the exact same owner
- this owner won't be imported, because it's a duplicate
- but the slug is different (!)
- the importer tries to find a matching existing user, but won't find anything
- the importer then send an empty authors array `post.authors=[]` into the model layer
- this is not allowed -> this would mean, you are actively trying to unset all authors
no issue
- if multiple queries run in a transaction, the model events are triggered before the txn finished
- if the txn rolls back, the events are anyway emitted
- the events are triggered too early
- solution:
- `emitChange` needs to detect that a transaction is happening
- it listens on a txn event to determine if events should be triggered
no issue
- if you delete all content, we expect two events
- `post.deleted` and `post.unpublished`
- `post.unpublished` was never triggered, because the api implementation made use of `collection.invoke(`destroy`)`
- what happened?
- you fetch all posts (columns:id)
- you destroy the post (only id column is available)
- the model events are triggered
- but you have no access to a default set of data
- the result is that the event handler can't even tell if this is a post or a page
- added a proper test to ensure which events are triggered
refs #9548
- we always receive date strings from the client in ISO format
- we ensure that we transform these strings into JS dates for comparison
- when the client sends relations, we need to ensure that relations are checked as well
- will only work for the post model for now, because this is the only model which uses `bookshelf-relations`
- added unit tests
- removed some model tests, which do the same
refs #9548
- do not forward `tag.parent` to the model layer
- the model layer should only know `tag.parent_id`
- and the API should only expose `tag.parent` (this is an API feature)
- currently Ghost has a mixture of using `toJSON` and the API validation layer for this
- we just continue with this for now (no time to fix this)
- disallow sending nested-nested relations
- unsupported
- see comment for more information
- this can cause problems with calling `hasChanged` on relations
- add unit tests
no issue
This PR adds the server side logic for multiple authors. This adds the ability to add multiple authors per post. We keep and support single authors (maybe till the next major - this is still in discussion)
### key notes
- `authors` are not fetched by default, only if we need them
- the migration script iterates over all posts and figures out if an author_id is valid and exists (in master we can add invalid author_id's) and then adds the relation (falls back to owner if invalid)
- ~~i had to push a fork of bookshelf to npm because we currently can't bump bookshelf + the two bugs i discovered are anyway not yet merged (https://github.com/kirrg001/bookshelf/commits/master)~~ replaced by new bookshelf release
- the implementation of single & multiple authors lives in a single place (introduction of a new concept: model relation)
- if you destroy an author, we keep the behaviour for now -> remove all posts where the primary author id matches. furthermore, remove all relations in posts_authors (e.g. secondary author)
- we make re-use of the `excludeAttrs` concept which was invented in the contributors PR (to protect editing authors as author/contributor role) -> i've added a clear todo that we need a logic to make a diff of the target relation -> both for tags and authors
- `authors` helper available (same as `tags` helper)
- `primary_author` computed field available
- `primary_author` functionality available (same as `primary_tag` e.g. permalinks, prev/next helper etc)
closes#9520
- it contains a dependency bump of the latest Bookshelf release
- Bookshelf introduced a bug in the last release
- see https://github.com/bookshelf/bookshelf/pull/1583
- see https://github.com/bookshelf/bookshelf/pull/1798
- this has caused trouble in Ghost
- the `updated_at` attribute was not automatically set anymore
---
The bookshelf added one breaking change: it's allow to pass custom `updated_at` and `created_at`.
We already have a protection for not being able to override the `created_at` date on update.
We had to add another protection to now allow to only change the `updated_at` property.
You can only change `updated_at` if you actually change something else e.g. the title of a post.
To be able to implement this check i discovered that Bookshelfs `model.changed` object has a tricky behaviour.
It remembers **all** attributes, which where changed, doesn't matter if they are valid or invalid model properties.
We had to add a line of code to avoid remembering none valid model attributes in this object.
e.g. you change `tag.parent` (no valid model attribute). The valid property is `tag.parent_id`.
If you pass `tag.parent` but the value has **not** changed (`tag.parent` === `tag.parent_id`), it will output you `tag.changed.parent`. But this is wrong.
Bookshelf detects `changed` attributes too early. Or if you think the other way around, Ghost detects valid attributes too late.
But the current earliest possible stage is the `onSaving` event, there is no earlier way to pick valid attributes (except of `.forge`, but we don't use this fn ATM).
Later: the API should transform `tag.parent` into `tag.parent_id`, but we are not using it ATM, so no need to pre-optimise.
The API already transforms `post.author` into `post.author_id`.
closes#9507
- Changed the utils.wordCount implementation to the one used by simpleMDE
- Added extra À-ÿ to the regex to support diacritics characters
- Added corresponding text with Chinese text mentioned in the issue
closes#9495
- Added a clause for amp being disabled
- In this clause, we strip the final 'amp/' part of the url, and redirect
- Changed corresponding test in frontend_spec.js
- Used `urlService.utils.redirect301()` instead of `res.redirect()`
refs https://github.com/TryGhost/Ghost/issues/9311
- very basic implementation, still needs proper classes and default stylesheet implementation
- change image card output to a `<figure>` with optional `<figcaption>`
- add optional `<p>` caption output to the html card
refs #9200
- We have not yet counted the images within your html, this commit counts images based on the this algorithm: https://blog.medium.com/read-time-and-you-bc2048ab620c
- Added imageCount utility, which counts images using an img-tag regex, amended from the general tag-regex found in wordCount
- Added this imageCount to the {{reading_time}} helper, adding 12 seconds to the reading time for every image
- The feature image is still counted as before
- The first image adds 12 seconds, the second 11, the third 10, and so on
- Images from the tenth onwards add 3 seconds to the reading time
closes#9085
Fixes an issue, where the client sets image properties to `""` after deleting the image. This causes problems with the query filter (see https://github.com/TryGhost/GQL/issues/24), as they have to be `null`.
Added a check in the model layer saving method to set value to `null`, when the property is empty.
Affected models and properties:
- `posts`:
- `feature_image`
- `og_image`
- `twitter_image`
- `users`:
- `profile_image`
- `cover_image`
- `tags`:
- `feature_image`
no issue
- the tests were failing since beginning of March
- this was caused by a wrong assertion in one of our authentication tests
- we work with a static 6 month ms number for token expiry
- this static ms number is based on 30 days per month
no issue
- extended functionality
- the knex mock simply parses the sql statements and serves data from memory
- i've tested the memory mode of sqlite, but could not get it working
- but maybe for the future to test again
no issue
- the handling here was not correct
- if you've passed no mobiledoc, it wasn't adding mobiledoc and an undefined html value
- we need a default mobiledoc+html value in case you don't pass the values within the test cases
no issue
- `post.author_id` has no reference to any table currently, see https://github.com/TryGhost/Ghost/blob/1.21.3/core/server/data/schema/schema.js#L19
- that's why it is right now possible to insert none existent author id's
- with multiple authors, this get's protected (see https://github.com/TryGhost/Ghost/pull/9426)
- you would get a proper error message
- it is not allowed to insert invalid author id's
- as soon as you do `include=author` you would receive an error
- fixed one test case where we inserted an invalid author id via the API
no issue
- just discovered that we had confusing function names in our test utility
- e.g. `posts` -> default posts from the data generator
- e.g. `users` -> extra users not from our data generator
- now:
- e.g. `posts` -> default posts from the data generator
- e.g. `users` -> default users from the data generator
- e.g. `users:extra` -> extra users not from our data generator
no issue
- currently if you would like to edit a resource (e.g. post) and you pass an invalid model id, the following happens
- permission check calls `Post.permissible`
- the Post could not find the post, but ignored it and returned `userPermissions:true`
- then the model layer is queried again and figured out that the post does not exist
- A: there is no need to query the model twice
- B: we needed proper error handling for post and role model
no issue
- replace logic for preparing nested tags
- if you have nested tags in your file, we won't update or update the target tag
- we simply would like to add the relationship to the database
- use same approach as base class
- add `posts_tags` to target post model
- update identifiers
- insert relation by foreign key `tag_id`
- bump bookshelf-relations to 0.1.10
no issue
- change behaviour from updating user references after the actual import to update the user reference before the actual import
- updating user references after the import is way less case intense
- that was the initial decision for updating the references afterwards
- but that does not play well with adding nested relations by identifier
- the refactoring is required for multiple authors
- if we e.g. store invalid author id's, we won't be able to add a belongs-to-many relation for multiple authors
- bookshelf-relations is generic and always tries to find a matching target before attching a model
- invalid user references won't work anymore
- this change has a very good side affect
- 17mb takes on master ~1,5seconds
- on this branch it takes ~45seconds
- also the memory usage is way lower and stabler
- 40mb takes 1,6s (times out on master)
no issue
- Ghost does not support adding an author by relation (`post.author = {id: '..'}`)
- Ghost does not support editing an author by relation (`post.author = {id: '..'}`)
- only `author_id` is allowed
refs https://github.com/TryGhost/Ghost/issues/3658
- the `validateSchema` helper was a bit broken
- if you add a user without email, you will receive a database error
- but the validation error should catch that email is passed with null
- it was broken, because:
- A: it called `toJSON` -> this can remove properties from the output (e.g. password)
- B: we only validated fields, which were part of the JSON data (model.hasOwnProperty)
- we now differentiate between schema validation for update and insert
- fixed one broken import test
- if you import a post without a status, it should not error
- it falls back to the default value
- removed user model `onValidate`
- the user model added a custom implementation of `onValidate`, because of a bug which we experienced (see https://github.com/TryGhost/Ghost/issues/3638)
- with the refactoring this is no longer required - we only validate fields which have changed when updating resources
- also, removed extra safe catch when logging in (no longer needed - unit tested)
- add lot's of unit tests to proof the code change
- always call the base class, except you have a good reason
no issue
- `isNew` does not work in Ghost, because Ghost does not use auto increment id's
- see https://github.com/bookshelf/bookshelf/issues/1265
- see https://github.com/bookshelf/bookshelf/blob/0.10.3/src/base/model.js#L211
- we only had one occurance, which was anyway redundant
- if you add a user, `hasChanged('password') is true
- if you edit a user and the password has changed, `hasChanged('password')` is true as well
NOTE #1:
1. We can't override `isNew` and throw an error, because bookshelf makes use of `isNew` as well, but it's a fallback if `options.method` is not set.
2. It's hard to re-implement `isNew` based on `options.method`, because then we need to ensure that this value is always set (requires a couple of changes)
NOTE #2:
If we need to differentiate if a model is new or edited, we should manually check for `options.method === insert`.
NOTE #3:
The unit tests are much faster compared to the model integration tests.
I did a comparision with the same test assertion:
- unit test takes 70ms
- integration test takes 190ms
no issue
- added https://github.com/colonyamerican/mock-knex as dev dependency
- the mock serves our data generator test data by default
- but you can define your own if you want
- we need a proper mock for unit testing
- we should not mock bookshelf if possible, otherwise we can't test event flows
no issue
- move password hashing and password comparison to lib/security/password
- added two unit test
- FYI: password hashing takes ~100ms
- we could probably mock password hashing in certain cases when unit testing
no issue
- this commit cleans up the usages of `include` and `withRelated`.
### API layer (`include`)
- as request parameter e.g. `?include=roles,tags`
- as theme API parameter e.g. `{{get .... include="author"}}`
- as internal API access e.g. `api.posts.browse({include: 'author,tags'})`
- the `include` notation is more readable than `withRelated`
- and it allows us to use a different easier format (comma separated list)
- the API utility transforms these more readable properties into model style (or into Ghost style)
### Model access (`withRelated`)
- e.g. `models.Post.findPage({withRelated: ['tags']})`
- driven by bookshelf
---
Commits explained.
* Reorder the usage of `convertOptions`
- 1. validation
- 2. options convertion
- 3. permissions
- the reason is simple, the permission layer access the model layer
- we have to prepare the options before talking to the model layer
- added `convertOptions` where it was missed (not required, but for consistency reasons)
* Use `withRelated` when accessing the model layer and use `include` when accessing the API layer
* Change `convertOptions` API utiliy
- API Usage
- ghost.api(..., {include: 'tags,authors'})
- `include` should only be used when calling the API (either via request or via manual usage)
- `include` is only for readability and easier format
- Ghost (Model Layer Usage)
- models.Post.findOne(..., {withRelated: ['tags', 'authors']})
- should only use `withRelated`
- model layer cannot read 'tags,authors`
- model layer has no idea what `include` means, speaks a different language
- `withRelated` is bookshelf
- internal usage
* include-count plugin: use `withRelated` instead of `include`
- imagine you outsource this plugin to git and publish it to npm
- `include` is an unknown option in bookshelf
* Updated `permittedOptions` in base model
- `include` is no longer a known option
* Remove all occurances of `include` in the model layer
* Extend `filterOptions` base function
- this function should be called as first action
- we clone the unfiltered options
- check if you are using `include` (this is a protection which could help us in the beginning)
- check for permitted and (later on default `withRelated`) options
- the usage is coming in next commit
* Ensure we call `filterOptions` as first action
- use `ghostBookshelf.Model.filterOptions` as first action
- consistent naming pattern for incoming options: `unfilteredOptions`
- re-added allowed options for `toJSON`
- one unsolved architecture problem:
- if you override a function e.g. `edit`
- then you should call `filterOptions` as first action
- the base implementation of e.g. `edit` will call it again
- future improvement
* Removed `findOne` from Invite model
- no longer needed, the base implementation is the same
no issue
- Date comparisons are possible via API, but there's no way to inject a valid date into the get helper
- JavaScript's Date.toString() function outputs dates in a useless format
- Swap to using Date.toISOString() and now the format can be understood anywhere!
- {{#get "posts" filter="published_at:<='{{published_at}}'"}}{{/get}} works now as expected
closes#9445
- redirects all asset requests if https is configured (theme, core, images)
- re-use and extend our url-redirect middleware
- add proper integration tests for our express site app (no db interaction, component testing required for such important use cases)
- i added some more general tests
- should avoid mixed content warnings in the browser
closes#9314
* added fixtures for contributor role
* update post api tests to prevent contributor publishing post
* update permissible function in role/user model
* fix additional author code in invites
* update contributor role migration for knex-migrator v3
* fix paths in contrib migration
* ensure contributors can't edit or delete published posts, fix routing tests [ci skip]
* update db fixtures hash
* strip tags from post if contributor
* cleanup post permissible function
* excludedAttrs to ignore tag updates for now (might be removed later)
* ensure contributors can't edit another's post
* migration script for 1.21
no issue
- returning and remembering the data, which was imported, is...
- not required when using the API
- not required when importing via script
- required for tests
- added an option to have control over it
- make more usage of local variables
- the GC cannot tidy up variables, which are defined outside of a loop, but used in the loop
- try to keep less memory in process
- reduce the number of properties we have to remember
no issue
- if you import a JSON file with a post, which has an unknown author,
the target user was removed from the blog
- Ghost can handle this case and still succeeds with import
- but we have stored an `author_id` in the database, which does not map to any user and won't map in the future
- this can trouble if we add support for multiple authors
- currently, we only return the `author_id` to the client and the client can map with `author_id` with users fetched by the API
- if it does not find a user, it just falls back to a different user
- but multiple authors have to be included explicit (`include=authors`) and we will return a mapped (author_id => user) result
- it won't be able to find the user, because we lookup the database
- this would result in an error
- there is in general no reason to import (or store) an unknown/invalid `author_id` into the database
- on import, we show you a warning and you can choose a different author if you want
- solution: fallback to owner user and extend warning
- it's not a behaviour change, you still can import unknown author id's and the import won't fail
- but we ensure valid author id's
- updated test
- further more: returning `author={}` when requesting `include=author` could trouble with ember currently
- it expects the author to be returned
refs #9127
- permission checks can happen everywhere in the code base
- we would like to create a context class
- global access to `options.context.is(...)`
- please read more about the access plugin in #9127 section "Model layer and the access plugin".
- removed the plugin and use direct context checks
no issue
- the affected test had a wrong payload, but passed anyway because the previous test edited the same title ;)
- this routing test does currently truncate the tables after each test case
- the test can run faster
- if we achieve reducing the routing tests, we can reconsider truncating the db
no issue
- reported in slack (https://ghost.slack.com/files/U8QV8DXQB/F8TSBQ532/image.png)
- do not expose old release notification
- e.g. you are on 1.20.0
- you receive a notification for 1.20.1 to update
- you update to 1.20.1
- ensure we protect exposing the release notification (compare against blog version)
- protect against wrong formats
- @TODO: the notifications could store a `version` property
- by that we could use `notification.version` and don't have to match the version in the message
closes#5071
- Remove hardcoded notification in admin controller
- NOTE: update check notifications are no longer blocking the admin rendering
- this is one of the most import changes
- we remove the hardcoded release message
- we also remove adding a notification manually in here, because this will work differently from now on
-> you receive a notification (release or custom) in the update check module and this module adds the notification as is to our database
- Change default core settings keys
- remove displayUpdateNotification
-> this was used to store the release version number send from the UCS
-> based on this value, Ghost creates a notification container with self defined values
-> not needed anymore
- rename seenNotifications to notifications
-> the new notifications key will hold both
1. the notification from the USC
2. the information about if a notification was seen or not
- this key hold only one release notification
- and n custom notifications
- Update Check Module: Request to the USC depends on the privacy configuration
- useUpdateCheck: true -> does a checkin in the USC (exposes data)
- useUpdateCheck: false -> does only a GET query to the USC (does not expose any data)
- make the request handling dynamic, so it depends on the flag
- add an extra logic to be able to define a custom USC endpoint (helpful for testing)
- add an extra logic to be able to force the request to the service (helpful for testing)
- Update check module: re-work condition when a check should happen
- only if the env is not correct
- remove deprecated config.updateCheck
- remove isPrivacyDisabled check (handled differently now, explained in last commit)
- Update check module: remove `showUpdateNotification` and readability
- showUpdateNotification was used in the admin controller to fetch the latest release version number from the db
- no need to check against semver in general, the USC takes care of that (no need to double check)
- improve readability of `nextUpdateCheck` condition
- Update check module: refactor `updateCheckResponse`
- remove db call to displayUpdateNotification, not used anymore
- support receiving multiple custom notifications
- support custom notification groups
- the default group is `all` - this will always be consumed
- groups can be extended via config e.g. `notificationGroups: ['migration']`
- Update check module: refactor createCustomNotification helper
- get rid of taking over notification duplication handling (this is not the task of the update check module)
- ensure we have good fallback values for non present attributes in a notification
- get rid of semver check (happens in the USC) - could be reconsidered later if LTS is gone
- Refactor notification API
- reason: get rid of in process notification store
-> this was an object hold in process
-> everything get's lost after restart
-> not helpful anymore, because imagine the following case
-> you get a notification
-> you store it in process
-> you mark this notification as seen
-> you restart Ghost, you will receive the same notification on the next check again
-> because we are no longer have a separate seen notifications object
- use database settings key `notification` instead
- refactor all api endpoints to support reading and storing into the `notifications` object
- most important: notification deletion happens via a `seen` property (the notification get's physically deleted 3 month automatically)
-> we have to remember a seen property, because otherwise you don't know which notification was already received/seen
- Add listener to remove seen notifications automatically after 3 month
- i just decided for 3 month (we can decrease?)
- at the end it doesn't really matter, as long as the windows is not tooooo short
- listen on updates for the notifications settings
- check if notification was seen and is older than 3 month
- ignore release notification
- Updated our privacy document
- Updated docs.ghost.org for privacy config behaviour
- contains a migration script to remove old settings keys
refs #5345, refs #3801
- Blog localisation
- default is `en` (English)
- you can change the language code in the admin panel, see https://github.com/TryGhost/Ghost-Admin/pull/703
- blog behaviour changes depending on the language e.g. date helper format
- theme translation get's loaded if available depending on the language setting
- falls back to english if not available
- Theme translation
- complete automatic translation of Ghost's frontend for site visitors (themes, etc.), to quickly deploy a site in a non-English language
- added {{t}} and {{lang}} helper
- no backend or admin panel translations (!)
- easily readable translation keys - very simple translation
- server restart required when adding new language files or changing existing files in the theme
- no language code validation for now (will be added soon)
- a full theme translation requires to translate Ghost core templates (e.g. subscriber form)
- when activating a different theme, theme translations are auto re-loaded
- when switching language of blog, theme translations are auto re-loaded
- Bump gscan to version 1.3.0 to support more known helpers
**Documentation can be found at https://themes.ghost.org/v1.20.0/docs/i18n.**
closes#9022
Images without extensions don't need to be manipulated, as we're now reading the bytes and pass those to the `image-size` lib.
This PR adds another `user-agent` to emulate multiple browser requests, as I stumbled over an example where the image without extension is protected otherwise.
Added a test, that works with above mentioned image, but is currently mocked. Nevertheless, the image worked as a PoC, that we're able to read the bytes of an image without its extension and still return the dimensions of the image.
refs https://github.com/TryGhost/Ghost-Release/issues/24
- differentiate between
1. original package.json version (can contain pre and build suffix)
2. full package.json version X.X.X-{pre} (optional)
3. safe package.json version X.X (major+minor)
no issue
- with 29e143fa9a import queries no longer run in parallel
- this commit simply adds a small code snippet to reflect the importer behaviour
1) duplicate slugs *within* a file are getting ignored
2) existing posts in the database and posts to import with the same slug, result in duplicates
Further improvements regarding duplication detection will happen via #8717.
closes#8717
- this is now required, because we run import queries sequentiell
- this code protects two cases:
- you have duplicate slugs in the JSON file (the first get's inserted, the second get's ignored)
- you have an existing slug in the database and you try to import the same slug, get's ignored
refs #9178, refs #8988
With 7353c87d7f we use Bluebird globally for Promises. Therefore, the request lib doesn't need to be wrapped in a bluebird Promise anymore.
This was originally done, so we can work with catch predicated in our image-size lib.
Updated the tests to proof, that the catch predicates work.
The tests fail, as soon as the Promise overwrite is commented out.
refs #9178
- this util uses the url services (!)
- moving this file into lib would not make sense right now
- that would mean a module requires first ../lib/url, which then requires ../services/url
- the url service definitely need a clean up 😃
refs #9178
- not 100% sure about this, but i think it makes right now the most sense
- we have already a url service and creating another lib/url is confusing at the moment
- i'll copy the last utility `makeAbsoluteUrls` to the url service for now
- see next commit for explanation (!)
refs #9178
- `checkFileExists` and `checkFileIsValid` where dirty required from web/middleware
- these two functions are only used in the target middleware
- let's move them
refs #9178
- i am not super happy about `const imageLib = require('../lib/image')`
- i don't really like the name `imageLib`
- but i had no better idea 😃
- if we use the same name in the whole project, it's very easy to rename the folder or the variable
no issue
> Deprecation warning: value provided is not in a recognized ISO format. moment construction falls back to js Date(), which is not reliable across all browsers and versions.
no issue
> (node:63849) Warning: Possible EventEmitter memory leak detected. 101 settings.edited listeners added. Use emitter.setMaxListeners() to increase limit
- the settings cache was initialised per test
- it registered the model events over and over again
- add a simple shutdown function, which can be called from the test env
refs #9178
- they definitely don't belong to server/utils
- i think the best place is putting them into the card apps
- the the post model needs to ask the app for it's converters
- move tests as well
refs #9178
- first iteration of tidying up the unit tests
- this is useful in the current stage, because if i move files in the server folder, i need a clean folder/file structure to detect which tests needs to move
- this is a simple cleanup to reflect the current server folder structure
no issue
- this test fails sometimes, i recently increased the socket delay (maybe it was not enough)
- because these are milliseconds
- let's try 100ms
refs #9178
- continue with killing our global utils folder
- i haven't found any better naming for lib/promise
- so, require single files for now
- instead of doing `promiseLib = require('../lib/promise')`
- we can optimise the requires later
refs #9178
- each package/module has a local utility (e.g. api, helpers, adapters)
- these are very small utility functions which are only used from this package
- they don't belong into the global lib/utils
refs #9178
- this logic belongs to a static model helper
- the visibility property is a model property, the knowledge about the visibility values belongs to the model
- rename the functions, so they make more sense
refs #9178
- we have to take care that we don't end up in circular dependencies
- e.g. API requires UrlService and UrlService needs to require the API (for requesting data)
- update the references
- we would like to get rid of the utils folder, this is/was the most complicated change
refs #9178
- move express apps to one place (called `web`)
- requires https://github.com/TryGhost/Ghost-Admin/pull/923
- any further improvements are not part of this PR
- this PR just moves the files and ensures the paths are up-to-date
refs #9178
- the first test will start the ghost server
- all following test will reuse the server and only truncate the database tables
- you can use `forceStart` if you need
- we have to ensure that reusing the server requires us to
- e.g. reload the settings
- e.g. reload active theme
- remove fork functionality (you can use `forceStart: true`)
- no test has changed, just the way we start the server
no issue
- adapt major changes of knex-migrator v3
- adapt migration scripts, simplify and add `down` (rollback) hook if possible
- clear Ghost cache after init hook (because of `knex-migrator migrate --init`)
- ensure db migrations work with the CLI
- updated troubleshooting guide (https://docs.ghost.org/v1/docs/troubleshooting#section-task-execute-is-not-a-function)
**For development only: Please ensure you run `npm i -g knex-migrator@latest` to update your global installation to v3. We always prefer the local installation, but v3 has modified and added binaries.**
no issue
- i saw that the channels routing tests were sometimes not running in Travis
e.g. https://travis-ci.org/TryGhost/Ghost/jobs/311303939
> Channel Routes
> Unhandled rejection Error: select "webhooks".* from "webhooks" where "event" = 'subscriber.added' - SQLITE_ERROR: no such table: webhooks
no issue
- https://github.com/mochajs/mocha/blob/master/CHANGELOG.md#400--2017-10-02
- the new `--exit` flag might be interesting at some point
> In Mocha v3.0.0 and newer, returning a Promise and calling done() will result in an exception.
- adapt teardown/setup test utility
- adapt other mixed usages of callback && Promise usage
refs https://github.com/TryGhost/Team/issues/41
- differentiate error codes
- return 404 if image was not found
- else return a 500
- use i18n keys
- use errors.utils.isIgnitionError (!)
refs https://github.com/TryGhost/Team/issues/41, refs https://github.com/TryGhost/gscan/issues/85
- if you are using the pagination helper not inside a resource context, you will receive an error
- improve error message, because it was not clear what happened
- downgrade error level to normal, because it's not a critical error from Ghost's perspective, from user perspective it is
- added help docs link and added a callout to our docs
no issue
Support for http://resthooks.org style webhooks that can be used with Zapier triggers. This can currently be used in two ways:
a) adding a webhook record to the DB manually
b) using the API with password auth and POSTing to /webhooks/ (this is private API so not documented)
⚠️ only _https_ URLs are supported in the webhook `target_url` field 🚨
- add `webhooks` table to store event names and target urls
- add `POST` and `DELETE` endpoints for `/webhooks/`
- configure `subscribers.added` and `subscribers.deleted` events to trigger registered webhooks
no issue
- added https://github.com/TryGhost/bookshelf-relations as dependency
- remove existing tag handling
---
* Important: Ensure we trigger parent initialize function
- otherwise the plugin is unable to listen on model events
- important: event order for listeners is Ghost -> Plugin
- Ghost should be able to listen on the events as first instance
- e.g. be able to modify/validate relationships
* Fix tag validation
- we detect lower/update case slugs for tags manually
- this can't be taken over from the plugin obviously
- ensure we update the target model e.g. this.set('tags', ...)
* override base fn: `permittedAttributes`
- ensure we call the base
- put relations on top
- each relation is allowed to be passed
- the plugin will auto-unset any relations to it does not reach the database
* Ensure we run add/edit/delete within a transaction
- updating nested relationships requires sql queries
- all sql statements have to run in a single transaction to ensure we rollback everything if an error occurs
- use es6
no issue
- useful for managing subscribers via external systems/API calls where it's likely only the e-mail address will be known
- adds `GET /subscribers/email/:email/`
- adds `DELETE /subscribers/email/:email/`
no issue
Had a couple of people ask about how to delete welcome posts easily, so adding a bio to the default user to draw a little more attention to it
no issue
- it can happen that concurrent requests try to renew access tokens with the same refresh token
- in this case it could happen that you received a token deletion error
- add propert locking
- ensure we don't run into deadlocks
- manual testing with async.times for parallel requests (was able to reproduce the error)
refs #5091, #9192
- Renderer figures out templates, contexts, and does a render call
- Templating is now handled with a single function
- Context call is made in the renderer
Note: to make this work, all controllers now define a little bit of config, currently stored in res._route. (That's a totally temporary location, as is res._template... when a sensible naming convention reveals itself I'll get rid of the weird _). This exposes a type and for custom routes a template name & default.
refs #9192, refs #5091
- Moved all url generation into generate-feed.js, so we can see as much data processing as possible in a single place.
- Refactored the way res.locals were used, to be more like how express uses them prior to rendering
- Removed a bunch of code & tests todo with context for RSS - I can't see any way that'd be used, unless we switched the rendering to use a template.
- moved the RSS rendering to be part of the service, not controller
- updated the tests significantly
Note: RSS generate-feed has a complete duplication of the code used in the excerpt helper in order to create an item description
refs #8613, refs #9228
- if you send a request to /authentication/token with `grant_type:password` and a Bearer token, Ghost was not able to handle this combination
- because it skipped the client authentication, see https://github.com/TryGhost/Ghost/blob/1.17.0/core/server/auth/authenticate.js#L13
- and OAuth detects the `grant_type: password` and jumps in the target implementation
- the target implementation for password authentication **again** tried to fetch the client and failed, because it relied on the previous client authentication
- see https://github.com/TryGhost/Ghost/blob/1.17.0/core/server/auth/oauth.js#L40 (client.slug is undefined if client authentication is skipped)
- ^ so this is the bug
- we **can** skip client authentication for requests to the API to fetch data for example e.g. GET /posts (including Bearer)
- so when is a client authentication required?
- RFC (https://tools.ietf.org/html/rfc6749#page-38) differentiates between confidential and public clients, Ghost has no implementation for this at the moment
- so in theory, public clients don't have to be authenticated, only if the credentials are included
- to not invent a breaking change, i decided to only make the client authentication required for password authentication
- we could change this in Ghost 2.0
I have removed the extra client request to the database for the password authentication, this is not needed. We already do client password authentication [here](https://github.com/TryGhost/Ghost/blob/1.17.0/core/server/auth/auth-strategies.js#L19);
If a Bearer token is present and you have not send a `grant_type` (which signalises OAuth to do authentication), you can skip the client authentication.
refs #9192, refs #9178
After trying to progress with current implementation, it became clear that the route service can't control the boot sequence, because then we end up with circular dependencies between the route service and the channel service.
The route service now exposes:
- a siteRouter
- a way for apps to register routes.
- ParentRouter base class for other modules to use
- the registry
...
- moved the default route setup back to site/routes.js 🙈
- moved the parent channel router back to the channel service (this makes way more sense imo)
- this structure prevents circular dependencies
- split the registry out into it's own thing
- fixed-up various bits of tests and comments
- DEBUG will print a list of routes 🎉
refs #9192, #5091
- changed channels to use our new base class
- keep the flexible structure, so that channels can be reloaded
- I had to move the router into the route service otherwise we get circular dependencies
- Don't _really_ want to keep it like this - need a way to define base classes as shared
no issue
- moved isLocalImage fn to storage utils used the RegExp of getLocalFileStoragePath to detect also relative image paths and added tests.
- Added test for independent protocol request (skip, because not supported/implemented)
refs #9192, refs #5091, refs #9178
- moved channels from controllers to a service
- split out the parent router from the remaining individual router logic
- moved the tests to match
refs #5091, refs #9192
- There are several theme template "renderers" all over the codebase
- Some are in apps, and were called "controllers"
- One is in error handling
- All of them now have comments marking out how they share logic/steps
- Other comments describe routes & controllers where they live
refs #9192
- The AMP app is nothing more than a custom controller - this will come clear soon
- Moved enabled/disabled logic into router
- Removed error-related code, as this wasn't used
- Changed logic for static pages to be based on req.body, not context
- Improved the tests to match
refs #5091, #9192, #9178
- Get the RSS module into a much better shape
- Controller -> /controllers/rss
- Remainder -> /services/rss
- Moved tests to match & updated requires
refs #5091, refs #9192
- This is similar to #9218, in that I'm revealing bits of code that are "controllers" in our codebase. As opposed to routes, services, renderers etc.
- This also reveals some code which is identical to the channels controller
- There is more to do here, but for now I've got the module split up, and the tests split and improved.
- Next I'll split RSS into controller + service, DRY up the controller code, etc
closes#9200
- Registered new server helper `{{reading_time}}`.
- Added new global util `word-count` based on the util in Ghost admin, which returns the number of words in an HTML string.
- Based on the word count of the post html, the helper calculated the estimated reading time:
- 275 words per minute
- additional 12 seconds when post has feature image
- Renders a string like 'x min red', unless reading time is less than a minute. In this case, the rendered string is '< 1 min read'.
refs #5091, refs #9192
- render channel was always a weird file
- now it's clearly 2 things
- we're slowly getting towards closing #5091... 🎉
- added some extra tests
refs #9192, refs #5091
- Using a class allows for easy shared logic
- Loading is designed to work from config right now, but could be DB driven, etc
- Provided configuration can be simplified and extended in the constructor / class methods
- Update tests, move custom assertions to utils
refs #9192
To anyone seeing this go by - I'm about to start some fairly major refactoring work on the url utility. Before I do that, I wanted to make sure I had 100% coverage, and understanding of some of the weird cases.
The majority of the changes I've made are adding tests, but I was also able to clean up a little bit, remove a few lines or change them to make use of other tools.
refs #9178
* Add eslint deps, remove old lint deps
* Add eslint config, remove old lint configs
* Config for server and tests are different
* Tweaked rules to suit us
* Fix linting in codebase - lots of indent changes.
* Fix a real broken test
no issue
- we had to fork the original repository at one point, because of slow maintenance
- maintenance is back now
- https://github.com/maxogden/extract-zip/pull/52 was merged and released
refs #9178
* Moved app handling code into services/apps
- Apps is a service, that allows for the App lifecycle
- /server/apps = contains internal apps
- /server/services/apps = contains code for managing/handling app life cycle, providing the proxy, etc
* Split apps service tests into separate files
* Moved internal app tests into test folders
- Problem: Not all the tests in apps were unit tests, yet they were treated like they were in Gruntfile.js
- Unit tests now live in /test/unit/apps
- Route tests now live in /test/functional/routes/apps
- Gruntfile.js has been updated to match
* Switch api.read usage for settingsCache
* Add tests to cover the basic App lifecycle
* Simplify some of the init logic
- This happened by accident when the model filters were introduced
- filters_spec.js was supposed to be renamed to apps_filters_spec.js
- Removing the duplicate file now
refs #8995
- move the getClient lookup from ghost_head into middleware
- use res.locals to keep track of the information (res.locals.client)
- make the middleware global to all frontend routes
- ghost_head: get locals from options.data not this (!)
- adapt lot's of tests
refs #9150
- Moves the password length fn from `models/user` to `data/validation` where the other validator functions live.
- Added password validation rules. Password rules added:
- Disallow obviously bad passwords: '1234567890', 'qwertyuiop', 'asdfghjkl;' and 'asdfghjklm' for example
- Disallow passwords that contain the words 'password' or 'ghost'
- Disallow passwords that match the user's email address
- Disallow passwords that match the blog domain or blog title
- Disallow passwords that include 50% or more of the same characters: 'aaaaaaaaaa', '1111111111' and 'ababababab' for example.
- Password validation returns an `Object` now, that includes an `isValid` and `message` property to differentiate between the two error messages (password too short or password insecure).
- Use a catch predicate in `api/authentication` on `passwordReset`, so the correct `ValidationError` will be thrown during the password reset flow rather then an `UnauthorizedError`.
- When in setup flow, the blog title is not available yet from `settingsCache`. We therefore supply it from the received form data in the user model `setup` method to have it accessible for the validation.
refs #5091
- Move all of the code to do with handling channels into one folder
- Still keeping all the shared/simlar code for rendering etc inside weird
frontend folder until I am sure what this will look like
refs #9178
- Introduce the /services/ folder
- Move xmlrpc there
- Move slack there
- In slack: remove a usage of the settings API that should use settingsCache
- In slack: Simplify the tests
- Various tiny changes to move towards code consistency
refs #5091
- remove the use of functions
- remove unnecessary quotes from tag filter
- move channel config to be a JSOn file called config.channels.json
- accept external config
- new channelUtils for tests
- remove channelConfig.get
- refactor so tests work as expected
- refactor away duplicate 'name' value
closes#9164
- check options.importing on xmlrpc
- also don't ping if private
- cleanup slack to work the same way
- update tests
- TODO: we need to prevent this event happening altogether
refs #5091
- There is very little that changes here, just code readability
- However I've expanded out the tests getting ready to be able to test more deeply as I refactor the routing
refs #9141
- adds support for `{{#prev_post in="author"}}{{/prev_post}}` & `{{#next_post in="author"}}{{/next_post}}`
- "author.slug" is the author equivalent of "primary_tag.slug" - there is only one
- added tests to cover both cases in the prev/next helpers
closes#9136
Changed the functionality in `make-absolute-urls.js` util to not convert the URL when starting which an `#`, as it indicates and internal link.
The util is used inside of the `{{amp_content}}` helper and to render the RSS feed. I tested the changes with the most popular RSS reader 'Feedly' and it seems like these internal links get converted to absolute URL inside of Feedly automatically.
no issue
- This started as an attempt to simplify the admin redirect code
- I realised we were sometimes using utils.redirect301 and sometimes not
- Decided to move this into utils.url as it's more relevant to URL generation
- Unified usage of redirects in the codebase
- Updated tests & ensured we have basic coverage
- rename adminRedirect -> redirectToAdmin
- Tweak method signature, fix channel edit redirects
- Tests: Optimised test descriptions for url-redirects_spec.js
- ensure caching works as expected
closes#9060
- Update `gscan` - it now extracts custom templates and exposes them to Ghost
- Add `custom_template` field to post schema w/ 1.13 migration
- Return `templates` array for the active theme in `/themes/` requests
- Users with Author/Editor roles can now request `/themes/`
- Front-end will render `custom_template` for posts if it exists, template priority is now:
1. `post/page-{{slug}}.hbs`
2. `{{custom_template}}.hbs`
3. `post/page.hbs`
closes#8668, refs #8920
- Updated tests to include internal tags
- Tests had no example of an internal tag
- Need this to show that the new filtering works as expected
- primary_tag is a calculated field
- This ensures that we can alias the field to equivalent logic in API filters
- By replacing primary_tag by a lookup based on a tag which has order 0
- bump ghost-gql to 0.0.8
**NOTE:**
Until GQL is refactored, there are limitations on what else can be filtered when using primary_tag in a filter e.g. it wont be possible to do a filter based on primary_tag AND/OR other tag filters.
refs #5091
- This removes hardcoded config to generate feed urls
- This means that RSS feeds work properly for custom channels
- Remaining assumption is that paginated feeds will end /pageNum/
- Added extra tests
- Don't pass through query params
refs #5091
- This simple change allows custom contexts to use existing channel logic
- E.g. if we want to create a custom tag-based channel, it can pass "tag" as the context, and get all the same metadata logic
refs #9043
- Cleanups / refactors to make the code more manageable
- Move remaining code out of index.js
- Only "init" function is left. Actions map cache and init function is based heavily on the settings cache module
- refactor the odd way of exporting
- This was cleaned up naturally by moving the actionsMap object out
- rename "effective" -> "providers"
- "Providers" provide permissions for different things that can have permissions (users, apps, in future clients).
closes#9089
- use the current date any time a post is fetched if the database contains an invalid date
- raise an error any time an attempt is made to save an invalidate date via the API
closes https://github.com/TryGhost/Ghost/issues/8943
- if you send a tag name with a hash, it's an internal tag
- ensure that the visibility property is forced to `internal`
- add a proper test
no issue
* Comment current state of toJSON for user model
- currently the user model does not return the email if the context is app/external/public OR if there is no context object at all
- i am not 100% sure why if there is no context we should not return the email address
- i think no context means internal access
- maybe change this condition cc @ErisDS
* Extend our access rules plugin
- we already have a instance method to determine which context is used
- this relies on passing options into `.forge` - but we almost never pass the context into the forge call
- added @TODO
- provide another static method to determine the context based on the options object passed from outside
* Use the new static function for existing code
* Add comment where the external context is used
* Remove certain fields from a public request (User model only)
* Tests: support `checkResponse` for a public request
- start with an optional option pattern
- i would love to get rid of checkResponse('user', null, null, null)
- still support old style for now
- a resoure can define the default response fields and public response fields
* Tests: adapt public api test
* Tests: adapt api user test
- use new option pattern for `checkResponse`
- eww null, null, null, null....
* Revert the usage of the access rules plugin
no issue
- it's not allowed to change/add these attributes via the API
- created_at = is only once set on adding the resource
- created_by = is only once set on adding the resource
- updated_by = is set on the server side when updating the model (based on who is logged in)
- updated_at = is set on the server side when updating the model
* Revert the usage of the access rules plugin
closes#9077
- because of our API layer refactoring, see https://github.com/TryGhost/Ghost/pull/9068
- we can now see that code was written wrong because of this horrible API bug
- this fixes the formats parameter for querying a single post
no issue
- the logic here bypasses filtering options!
- that is wrong, because if we filter out certain options e.g. include
- the tests from the previous commit fail because of this
- if we don't fix this logic, the tests won't pass, because as said, you can bypass certain logic e.g. remove roles from include
- this has worked before, because we passed the wrong options via the API layer
- was introduced here 014e2c88dd, because of https://github.com/TryGhost/Ghost/pull/6122
- add proper tests to proof that these queries work!!
🐛 Fixed author role permission to change author
no issue
- To be able to fix this bug, we had to solve tasks from #9043
- This bug affects the private / undocumented API only
- Author role users should not be allowed to change the author of a post
refs #8602
- Add the wiring to pass attributes around the permission system
- Allows us to get access to the important "unsafe" attributes that are changing
- E.g. status for posts
- This can then be used to determine whether a user has permission to perform an attribute-based action
- E.g. publish a post (change status)
no issue
- our public API is still a beta/labs feature
- from api.ghost.org
> The API is still under very (very) heavy development and subject to regular breaking changes.
- users should expect breaking changes in any release (independent from semver versions)
- the public user API never returns any email addresses to decrease the information we expose
- there is no need to keep the support fetching a user by email address
refs #9043
- this is preparation for adding Author-specific tests later
- the changes the posts_spec.js, so that all the tests are inside an "As Owner" describe block, similar to the users_spec.js
- Added new util for creating a specific post
- This will make it easier to do routing tests on the post model in future
- Our `index.js` file in test/utils really needs a bit of love 🙈
- Also added all the framework for author role tests in post_spec.js
- Added a single test, showing we can edit posts, including author_id
no issue
- this bug fix affects all endpoints for the public user access
- we allowed fetching `roles` via the public api by accident
- see our docs: https://api.ghost.org/docs/users)
- we only allow `count.posts`
- returning roles via the public api exposes too many details
- this was never intentional
refs #9043
- Move api util tests into api section
- Adding export test to utils to see the amount of functions which are exported
- Adding basic handlePermissions tests
refs #9028
- if you upload a redirects file and a redirects file exists already, we backup this file to `data/redirects-YYYY-MM-DD-HH-mm-ss.json`
- decrease chance of random test failures by not comparing date format with seconds
no issue
- this bug fix affects all endpoints for the public user access
- we allowed fetching `roles` via the public api by accident
- see our docs: https://api.ghost.org/docs/users)
- we only allow `count.posts`
- returning roles via the public api exposes too many details
- this was never attentional
refs #9043
- Split public-related and context code into logical components
- Split tests up to match
- Ensure we have 100% unit test coverage
- General cleanup
refs #9028
- add two new endpoints for uploading/downloading the redirects (file based)
- reload/re-register redirects on runtime
- migration for 1.9 to add permissions for redirects download/upload
no issue
- use latest casper in test fixtures
- never ever use the root content folder for tests
- if we start/fork Ghost for the tests, we use a tmp folder
- this change is required to for an upcoming PR (#9029)
- i've added a TODO to create a helper fn for stopping the ghost server, so we can cleanup the tmp folder
* Care about TODO's in our channels spec
- add the 1.4 compatible casper theme to fixtures
- so as soon as you start Ghost, the test env will provide the content folder in /tmp something with the activated latest default casper and the 1.4 compatible old casper
- there are tests which tests different logici e.g. pagination
- therefor we need a different theme, we are simply using our 1.4 casper
refs https://github.com/TryGhost/Ghost/issues/8859
There are four cases:
- unsplash setting is empty (default), admin can enable the app by default (hardcoded isActive:true)
- unsplash settings are set, unsplash is disabled, admin detects that app was disabled on purpose
- unsplash setting is set, unsplash is enabled and has a key, app is enabled, old key get's ignored and overridden on the next save
- unsplash setting is set, unsplash is enabled and has no key, app is enabled
refs https://github.com/TryGhost/Ghost/issues/8859
- We don't need the config option for Unsplash anymore
- The private endpoint (/configuration/private) was introduced for Unsplash
* Improved log output for welcome email error
no issue
- if Ghost is unable to send a welcome email, the server log printe a huge error log
- the reason was that each component wrapped the original error into a new error instance
- so the stack grows and grows
- the golden rule should always be: the smallest/lowest component should instanitate a specifc error
- the caller can expect to receive a custom Ghost error
* Tidy up error messages for mail failures and fix tests
- We never use "Error:" notation in our translations
- Make the error messages consistent and show a reason if possible
no issue
- mirror LTS behaviour to master
- if your blog or admin url is configured to http, it's still possible that e.g. nginx allows both https/http
- that's why we should generate the api url without protocol in this case
- so it depends how you serve your blog, example:
- blog url is http://example.com
- generated api url for the sdk is //example.com (dynamic protocol allowed)
- you serve your blog via https://example.com, protocol is https
- you serve your blog via http://example.com, protocol is http
closes#8342
- no need to add a migration, because when we'released 1.0, OAuth was never an option
- it was disabled in April, 1.0-beta was released in June
- remove all remote authentication code
no issue
- if you blog runs on a custom domain, but your admin panel is configured using a different domain
-> Ghost losts the origin header
- we had this situation once with pretty urls (your request get's redirected from /posts to /posts/, see https://github.com/TryGhost/Ghost/pull/8094)
- we've moved all our redirect logic to Ghost and ran into the same situation
- i've added proper test to ensure it won't happen again
refs #7470
- the importer test causes problems with the order of posts
- the importer is greedy and tries to add data in parallel, but the tests simply fetch the raw data from knex without any order
- while i was improving the order problem, i found this amp/disqus edge case
Order Random Failure Example:
1) Import (new test structure) 1.0: basic import test keeps the value of the amp field:
AssertionError: expected '59a952be7d79ed06b0d21128' to equal '1'
+ expected - actual
-59a952be7d79ed06b0d21128
+1
refs #8868
* 📐 Use request util in image-size
- swapped the usage of `got` for requests with the request util
* 💄 Use catch predicates
- Uses catch predicates instead of conditionals in `getImageSizeFromUrl`
- Return `NotFoundError` if applicable in `getImageSizeFromFilePath` as the caller function `cachedImageSizeFromUrl` is differentiating those between this error and others.
* 🐛 Fixed ImageObject URL & simplify no protocol URL logic
- Using `ImageObject` as a global var resulted in having the `url` property being the same for all requests coming in.
- The logic that checked for an existing protocol (e. g. gravatar URLs) was overly complicated. Refactored it to be more simple.
- Passing the correct value to `fetchDimensionsFromBuffer` as the population of `imageObject.url` happens there. These are used in our structured data and need to be full URLs (in case of locally stored files) or the original URL (in case of URLs missing the protocol)
- Added two more debug logs in `getCachedImageSizeFromUrl` so it's logged when an image is added to the cache even tho it was returned as error.
* 👀 Differentiate error codes between request and storage
* 🔥 Remove not needed `Promise.resolve()`
We're always resolving the result in `getCachedImageSizeFromUrl`, so there's no need to return the values with a `Promise.resolve()`. The caller fn uses waits for the Promises to be fulfilled.
* ☂️ Wrap already rejected predicate errors in catch all
* Use errorDetails instead of context
* ☂️ Support /assets/ image paths
- adds a guard that checks the image URL for `/assets/` in the beginning and passes a completed URL to the request util to try and fetch the image size
- adds tests
no issue
This PR includes a new util which wraps the `got` library. It is not used in the codebase yet, but tested with `image-size` util:
- wraps `got` request library in its own `request.js` util that returns bluebird promises and validates URL before starting a request
- adds tests
no issue
- test cases were trying to fetch image sizes for `localhost:port/favicon.ico` but no server is running so they time out
- stub the `getImageSizeFromUrl` method so it resolves instantly
refs #8868
- Removed image-size in blog logo fn for meta data and made it synchronous
- Renamed `image-size-from-url.js` to `image-size.js` (incl. the test)
- Added second fn `getImageSizeFromFilePath` that reads from local file storage
- Added guard in `getImageSizeFromUrl` that checks if the image should be on local file storage and uses the new fn then instead
- Added a fn `fetchDimensionsFromBuffer` that takes the file buffer and returns an `imageObject` with dimensions.
- Added a new utils.js in `adapters/storage` for getting the file storage path
closes#8963
- if an LTS export is imported into a 1.0 blog, then the 1.0 blog is
exported and re-imported into another 1.0 blog, any post ids from the
lts import were getting clobbered. This only saves the post id if the
amp field does not already exist
- add failing test that passes w/change
no issue
- reduce the number of redirects
- before: you are redirected from example.com/ghost to admin.example.com/ghost and Ghost would detect a missing slash and redirect you to /ghost/
- now: you are redirected from example.com/ghost to admin.example.com/ghost/
no issue
- this bug was invented with this commit 25c4e5025a
- the updated logic ensures that
- only if you have configured a custom admin url and your requested host does not match, we redirect you
- we still keep the wish of no force redirect if you have only configured a custom blog url and you navigate to /ghost
refs #8703
- Instead of throwing errors, throw warnings for incorrect usage of the img_url helper
- Differentiate between no attribute passed, and attribute evaluating to undefined
no issue
- adds a ghost-backup client
- adds a client authenticated endpoint to export blog for ghost-backup client only
- allows some additional overrides during import
- allows for an import by file to override locking a user and double hashing the password
closes#8821
- Use semver to do constraint matching
- Use client to generate a caret constraint
- E.g. if the client is 1.1, then the constraint ^1.1.0 will match >=1.1.0 <2.0.0
- Updated tests
refs #8868
The `image-size` library supports now `.ico` files, which means there is no longer need to use the `icojs` library.
- removes unnecessary `icojs` dependency
- refactors `getIconDimensions` fn in blog icon util to fetch image sizes synchronus
- removes unnecessary `getIconDimensions` fn in blog icon validation, as there is no longer need to use different image size fn for different file extensions, and uses `getIconDimensions` from blog util fn instead.
- updates and adds more tests
refs #8901
- Adds support for:
```
{{#has number="3"}} // A single number
{{#has number="3, 6, 9"}} // list the numbers you want to match against
{{#has number="nth:3"}} // special syntax for nth item
```
And
```
{{#has index="3"}} // A single number
{{#has index="3, 6, 9"}} // list the numbers you want to match against
{{#has index="nth:3"}} // special syntax for nth item
```
refs #8859
- adds new `configuration/private` endpoint for exposing config that should not be accessible without authentication
- adds `unsplashAPI` to private config
- adds empty `unsplash` config to default settings
no issue
- Upgraded ghost-ignition
- Use debug from ghost-ignition everywhere in the code base
- Remove debug dependency
- Fixed random typo in Gruntfile.js
refs #5091
- This tiny refactor opens the door for using channel config inside of helpers
- This means that ghost_head, and the next_post/prev_post helpers can be context aware
refs #5091
- occurred to me whilst documenting the custom homepage config, that RSS and pagination
need to be optional
- added a very quick if statement & tests
- needs further refactoring & test improvements
- this will not disable the RSS url output in meta data yet 😔
closes#8334
- adds title, image and description to structured data to be rendered as open graph and twitter data.
- if meta title and description for a post exists already, the custom structured data will overwrite those for `og:` and `twitter:` data. `JSON-LD` (Schema.org`) is not affected and will stay the same.
- adds tests
- adds new og and twitter fields to schema incl. migration
no issue
- add 1.4 database migration to add two new fields to the database (use type text, because of max row size)
- handle global code injection vs. post code injection
- add tests
refs #8700
- if you used a url e.g. /page/2abc/ ghost would interpret the 2 as /page/2/
- these urls should have returned 404, but instead were responding correctly
- this effectively creates duplicate pages
- added a test, but needed a dirty hack to get it to work 😞
- TODO: update casper fixture and use it in channel tests!
closes#8793
- 1.3 post excerpt migration
- add 1.3 migration to add `excerpt` to post schema
NOTE:
- knex-migrator relies on the package.json safe version
- so right now Ghost is on 1.2
- the migration script is for 1.3
- if you pull down the PR (or if we merge this PR into master), you have to run `knex-migrator migrate --v 1.3 --force`
- knex-migrator will tell you what you have todo
- Bump dependencies
- knex-migrator@2.1.3
- Soft limit for custom_excerpt
- Extended {{excerpt}} to use custom excerpt
- when a `custom_excerpt` field exists, the `{{excerpt}}` helper will output this and fall back to autogenerated excerpt if not.
- Refactored behaviour of (meta) description
- html tag `<meta name="description" />` for posts, tags and author doesn't get rendered if not provided.
- fallback for `author.bio` removed
- fallback for `tag.description` removed
- structured data and schema.org for `post` context takes the following order to render description fields:
1. custom excerpt
2. meta description
3. automated excerpt (50 words)
- updated and added tests to reflect the changes
closes#8781
- when the ownership get's transferred, the id of the new owner is not '1' anymore
- we previously added a database rule, which signalises if the blog is setup or not, see 827aa15757 (diff-7a2fe80302d7d6bf67f97cdccef1f71fR542)
- this database rule is based on the owner id being '1', which is wrong when you transfer ownership
- we should keep in mind, that the owner id being '1' is only the default Ghost setup, but it can change
- blog is setup if the owner is locked
closes#8605
- This file has already been moved, might as well get the rename out of the way
- Especially as we don't migrate clients - everyone will now need to make just one change
refs #8756
- there was a bug in one of the last LTS releases, which produced duplicated attached roles to users
- we want to prevent that on import and take the latest created based on the autoincrement id
closes#8760
- we have to remember the old post id's when migrating a blog from LTS to 1.0
- otherwise we would break disqus comments, because they rely on the post id
- this should fix the discovered situation
closes#8645, closes#8710
- locked users were once part of the category "active users", but were moved to the inactive category
-> we have added a protection of not being able to edit yourself when you are either suspended or locked
- but they are not really active users, they are restricted, because they have no access to the admin panel
- support three categories: active, inactive, restricted
* - revert restricted states
- instead, update permission layer: fallback to `all` by default, because you are able to serve any user status
- add more tests
- ATTENTION: there is a behaviour change, that a blog owner's author page can be served before setting up the blog, see conversation on slack
-> LTS serves 404
-> 1.0 would serve 200
closes#8691
There was a condition added when i've refactored the importer.
> if (models.User.isOwnerUser(obj[key])) {
This condition is absolutely wrong! If you import an owner user, this owner user get's imported as administrator. But the original owner user id reference must be updated as well, so that the reference points to the new administrator id ✌🏻
no issue
- if you delete an active user, Ghost logs an error message (Ghost does not crash!)
- but the event logic is not triggered, that means we don't delete the users tokens
- token deletion happens on: suspend a user and delete a user
closes#8601
- This makes sure that when you do an import, you still get the LATEST
default settings for labs. Even if you had a different value before.
- LTS -> 1.0 is an upgrade, and Public API should be on by default, even if you
had deliberately turned it off before.
- Cheeky test added
refs #8620
Adds a new Ghost Author user, which is the author of the new welcome blog posts. The user is set to active, so the author slug works (otherwise it would render a 404, when user is suspended). Furthermore, there's one little fix in the user model, which was checking only for `active` user to decide the signup or setup process for the UI. Adding one more conditional to check if the found active user is also the owner, prevents to get redirected to sign in.
closes#8601
- this doesn't take the feature out of beta, but does enable it by default
- no need to enable the public api in the test anymore
- because public api is enabled by default
closes#8565
- isPasswordCorrect fn returns a specific error, which we simply forward
- no need to wrap a custom error into a new custom error
- the rule is always: if you are using a Ghost unit/function, you can expect that this unit returns a custom error
closes#8568
- use our `urlJoin` util to concatenate the URL (not the query part of it, as this is not supported in `urlJoin`) and to prevent possible missing or double slashes, as `config.apiUrl` could be with or without trailing slash
closes#8542
- updates default post fixtures
- adds default logo and cover images to settings fixtures
- update tests due to coupling to dev/prod fixtures
refs #8222
- differentiate between errors and fatal errors
- use gscan errors in theme middleware
- Adds a new `error()` method to `currentActiveTheme` constructor which will return the errors we receive from gscan
- In middleware, if a theme couldn't be activated because it's invalid, we'll fetch the erros and send them to our error handler. We also use a new property `hideStack` to control, if the stack (in dev mode and if available) should be shown or the gscan errors (in prod mode, or in dev if no stack error)
- In our error handler we use this conditional to send a new property `gscan` to our error theme
- In `error.hbs` we'll iterate through possible `gscan` error objects and render them.
- remove stack printing
- stack for theme developers in development mode doesn't make sense
- stack in production doesn't make sense
- the stack is usually hard to read
- if you are developer you can read the error stack on the server log
- utils.packages: transform native error into Ghost error
- use `onlyFatalErrors` for gscan format and differeniate fatal errors vo.2
- optimise bootstrap error handling
- transform theme is missing into an error
- add new translation key
- show html tags for error.hbs template: rule
refs #8141
- update importer for LTS fields
- optimise for LTS export fixtures
- add image/language test for LTS import
- ensure post image is mapped to feature_image
- create mobiledoc values from markdown and html
- if mobiledoc is null, use markdown or html to create a mobiledoc markdown card
- update import mapping to use locale
- defaultLang in settings now maps to default_locale
- language for post and user models now maps to locale
- posts are not always loaded in correct same order so we select the posts we want to validate
- ensure if mobiledoc field is not in export we can still import from markdown
- map last_login to last_seen
- for users the importer maps last_login to last_seen
- add warning for legacyActiveTheme
- for export with old activeTheme key provide a warning that theme is not installed
- add importer test for LTS user long email
- add a test for LTS export where email address could be longer than alpha
- fix for importer date tests on mysql
- use valueOf in moment to compare times stored in different formats
- ignore warnings for not found settings in import
- use a flag to ignore NotFound Entries for settings during import
no issue
- this PR references indirecty to https://github.com/TryGhost/Ghost/pull/8437
- i would like to have the settings change already in place before we release the beta
- the i18n feature is able to change the locale of Ghost
- most i18n libraries use locale
- adding/changing settings doesn't require a migration file, but it can make the database a bit messy (because you can end up with default_locale and lang)
- furthermore we agreed that the default locale for Ghost should be simply `en`, not `en_US` or `en_GB`
closes#8479
- removes `markdown` field from schema
- removes `legacyMarkdown` converter
- updates tests to work with `mobiledoc` field instead of `markdown` and adapt for mobiledoc HTML output where necessary
refs #8152
- as long as OAuth is disabled, we can revert the url redirection (see comment)
- the redirect only happens if you configure a specific `admin.url`
- add another test case, which was missing
refs #5422
- we can support null titles after this PR if we want
- user model: fix getAuthorRole
- user model: support adding roles by name
- we support this for roles as well, this makes it easier when importing related user roles (because usually roles already exists in the database and the related id's are wrong e.g. roles_users)
- base model: support for null created_at or updated_at values
- post or tag slugs are always safe strings
- enable an import of a null slug, no need to crash or to cover this on import layer
- add new DataImporter logic
- uses a class inheritance mechanism to achieve an easier readability and maintenance
- schema validation (happens on model layer) was ignored
- allow to import unknown user id's (see https://github.com/TryGhost/Ghost/issues/8365)
- most of the duplication handling happens on model layer (we can use the power of unique fields and errors from the database)
- the import is splitted into three steps:
- beforeImport
--> prepares the data to import, sorts out relations (roles, tags), detects fields (for LTS)
- doImport
--> does the actual import
- afterImport
--> updates the data after successful import e.g. update all user reference fields e.g. published_by (compares the imported data with the current state of the database)
- import images: markdown can be null
- show error message when json handler can't parse file
- do not request gravatar if email is null
- return problems/warnings after successful import
- optimise warnings in importer
- do not return warnings for role duplications, no helpful information
- error handler: return context information of error
- we show the affected json entries as one line in the UI
- show warning for: detected duplicated tag
- schema validation: fix valueMustBeBoolean translation
- remove context property from json parse error
refs https://github.com/TryGhost/Ghost-Admin/pull/690, closes#1501, closes#2093, closes#4592, closes#4627, closes#4659, closes#5039, closes#5237, closes#5587, closes#5625, closes#5632, closes#5822, closes#5939, closes#6840, closes#7183, closes#7536
- replace custom showdown fork with markdown-it
- swaps showdown for markdown-it when rendering markdown
- match existing header ID behaviour
- allow headers without a space after the #s
- add duplicate header ID handling
- remove legacy markdown spec
- move markdown-it setup into markdown-converter util
- update mobiledoc specs to match markdown-it newline behaviour
- update data-generator HTML to match markdown-it newline behaviour
- fix Post "converts html to plaintext" test
- update rss spec to match markdown-it newline behaviour
- close almost all related showdown bugs
closes#8354
- i thought about transforming scheduled posts into drafts on export, but this has two disadvantages:
1. existing exports with scheduled posts won't import
2. if you schedule a post for next week and you export/import earlier, the post is back to draft
- by this we ensure that we can simply import the post back to a scheduled post
- if the published_at is already in the past, the scheduler will care and instantly publish the post
closes#8436
- this is how the from field looks like "blog title <owner@blog.com>"
- so if you set your blog title with double quotes, it throws a syntax error from the smtp library
no issue
- now that we've switched to using a SimpleMDE based editor in Ghost-Admin the default post needs to match the expected single-markdown-card format
* 🙀 change database schema for images
- rename user/post/tag images
- contains all the required changes from the schema change
* Refactor helper/meta data
- rename cover to cover_image
- also rename default settings to match the pattern
- rename image to profile_image for user
- rename image to feature_image for tags/posts
* {{image}} >>> {{img_url}}
- rename
- change the functionality
- attr is required
- e.g. {{img_url feature_image}}
* gscan 1.0.0
- update yarn.lock
* Update casper reference: 1.0-changes
- see 5487b4da8d
closes#5599
If two users edit the same post, it can happen that they override each others content or post settings. With this change this won't happen anymore.
✨ Update collision for posts
- add a new bookshelf plugin to detect these changes
- use the `changed` object of bookshelf -> we don't have to create our own diff
- compare client and server updated_at field
- run editing posts in a transaction (see comments in code base)
🙀 update collision for tags
- `updateTags` for adding posts on `onCreated` - happens after the post was inserted
--> it's "okay" to attach the tags afterwards on insert
--> there is no need to add collision for inserting data
--> it's very hard to move the updateTags call to `onCreating`, because the `updateTags` function queries the database to look up the affected post
- `updateTags` while editing posts on `onSaving` - all operations run in a transactions and are rolled back if something get's rejected
- Post model edit: if we push a transaction from outside, take this one
✨ introduce options.forUpdate
- if two queries happening in a transaction we have to signalise knex/mysql that we select for an update
- otherwise the following case happens:
>> you fetch posts for an update
>> a user requests comes in and updates the post (e.g. sets title to "X")
>> you update the fetched posts, title would get overriden to the old one
use options.forUpdate and protect internal post updates: model listeners
- use a transaction for listener updates
- signalise forUpdate
- write a complex test
use options.forUpdate and protect internal post updates: scheduling
- publish endpoint runs in a transaction
- add complex test
- @TODO: right now scheduling api uses posts api, therefor we had to extend the options for api's
>> allowed to pass transactions through it
>> but these are only allowed if defined from outside {opts: [...]}
>> so i think this is fine and not dirty
>> will wait for opinions
>> alternatively we have to re-write the scheduling endpoint to use the models directly
no issue
- the UTC offset diff of the current and previous timezone must switch
- i have added more tests and more example case descriptions to understand why
refs #8221, closes#7688, refs #7558🙇 Improve meta data publisher logo behaviour
This is a follow-up PR for #8285.
Reasons: The code changes of #8285 caused error messages when falling back to the default `favicon.ico`, as the `image-size` tool doesn't support `ico` files.
This PR takes the logic to decide which logo needs to be listed in our schema into a new fn `blog_logo.js`. There we have now three decisions:
1. If we have a publication **logo**, we'll take that one
2. If we have no publication logo, but an **icon** we'll use this one.
3. If we have none of the above things, we fall back to our default `favicon.ico`
Additional, we're hard coding image dimensions for whenever the logo is an `.ico` file and built and extra decision to not call `image-size` when the dimension are already given.
I will create another follow-up PR, which checks the extension type for the file and offers it as a util.
🛠 Blog icon util
refs #7688
Serve functionality around the blog icon in its own util:
- getIconDimensions -> async function that takes the filepath of on ico file and returns its dimensions
- isIcoImageType -> returns true if file has `.ico` extension
- getIconType -> returns icon-type (`x-icon` or `png`)
- getIconUrl -> returns the absolut or relativ URL for the favicon: `[subdirectory or not]favicon.[ico or png]`
📖 Get .ico sizes for meta data & logo improvement
refs #7558
refs #8221
Use the new `blogIconUtil` in meta data to fetch the dimensions of `.ico` files.
Improvements for `publisher.logo`: We're now returning a hard-coded 'faked' image dimensions value to render an `imageObject` and prevent error our schema (Google structured data). As soon as an image (`.ico` or non-`.ico`) is too large, but - in case of non-`.ico` - a square format, be set the image-dimensions to 60px width and height. This reduces the chances of getting constantly error messages from Googles' webmaster tools.
- add getIconPath util
refs #8275
- If the HTML field has changed, update the plaintext field
- Use html-to-text to generate a plaintext version of the HTML which retains some structure
- Add a couple of tests - although there's much to do here!
