closes#9822
- Fixed the post count issue for co authors
- Corrected and refactored tests related to users post count
- Consistency fix, because we return all posts where the author is primary or co author for the author page already
closes#9786
- Make GET request when url has no provider match
- The HEAD request was made in order to send less data over the wire when
checking for redirects for urls that do not have an oembed provider
match. We are now going to look for provider metatags withing the
response of the request - rather than making a HEAD followed by a GET if
no redirect is found, this condenses that to a single request.
- Try to get OEmbed data from tag if no provider
- Here we parse the HTML response of the resource and look for a link tag
that will give us the oembed resource url which we can use to fetch the
embed html
closes#9644
- Removed google blogsearch from pingList, because it's dead (https://plus.google.com/+GoogleWebmasters/posts/46W7ZwVrwqg)
- Updated pingomatic method weblogUpdate -> weblogUpdates
- Updated RPC response handler to check for errors, log improvement
- the pingomatic service sends back a 200 even when it errors, so we
check the xml response to see if it's good, it throws and passes of to
the catch handler already in place
- Avoid multiline XML error message strings, but
includes a catch in case our regex stops working with a fallback to
multiline XML error message
- we check for arbitrary whitespace between different XML tags,
which I haven't seen in any of the responses - but it could change I
guess. I haven't added support of whitespace between the tags, as I
believe that would be a different value with XML spec. And we wanna
match on exact values here.
no issue
- we have to explicitly reset the previous `updated_at` field, because Bookshelf auto-updates this field on each update
- we have to extend the condition to avoid updating the `updated_by` field
- detect and respect `options.migrating`
* 🐛 Dynamic Routing Beta: Shortform `author.foo` is not allowed
refs #9601
- otherwise you get access to {{author}} in the theme, which is deprecated & causes errors
- recommend not using {{author}} as data longform name in the yaml validator
- (internal usage is still allowed)
- also warn against using reserved data key names like filter, resource, limit etc - maybe remove this restriction later but seems like a sensible validation right now.
no issue
- Organising your content
From: Here, the theme would assign the post publicly displayed tags of Blog - but it would also keep a private record of the post being tagged with #video.
To: Here, the theme would assign the post publicly displayed tags of News - but it would also keep a private record of the post being tagged with #video.
refs #9601
- when using the short form `data: tag.welcome` the redirect is enabled by default
- /tag/welcome/ will redirect to the channel/collection which makes use of the data key
- you can disable the redirect by using the long form
e.g. data:
tag:
resource: tags
type: read
slug: welcome
redirect: false
closes#9774, refs #9742
- added new fixture posts for Ghost 2.0
- added migration file to remove old fixture posts
- only remove them if they are owned by the Ghost author and if they are tagged with getting-started
- added new fixture posts if you had all (!) old fixture posts
- ensure on rollback we remove the new fixture posts again
- updated default settings
refs #9742
- removed usage of single permalink setting
- with dynamic routing this configuration does no longer makes sense
- because you can configure your permalinks in the routes.yaml
- furthermore you can have multiple collections with multiple permalinks
- removed @blog.permalinks
- do not export permalink setting
- do not import permalink setting
- permalink setting UI will be removed soon
- get rid of {globals.permalink} completely
- remove yaml in-built migration
- do not expose settings.permalinks via the private API
- do not allow to edit this setting
- keep phyiscal value in case a blog needs to rollback from v2 to v1
- sorted out when the routers should be created
- ensure routes.yaml file doesn't get validated before Ghost is fully ready to start
refs #9742, refs #9724
- handle König Editor format for 2.0
- adapted importer to be able to import 1.0 and 2.0 exports
- added migration scripts
- remove labs flag for Koenig
- migrate all old editor posts to new editor format
- ensure we protect the code against mobiledoc or html field being null
- ensure we create a blank mobiledoc structure if mobiledoc field is null (model layer)
- ensure you can fully rollback 2.0 to 1.0
- keep mobiledoc/markdown version 1 logic to be able to rollback (deprecated code)
refs #9744
- added two new endpoints to upload/download routes.yaml
- reload site express app on successful/valid upload
- reload url service on sucessfuly upload
- force clear cache of pages
- ensure we keep a backup of the routes.yaml file
- this feature was mostly tested manually
- @TODO: i have to write unit tests - will do later
- @TODO: do a memory test to ensure we haven't introduced any memory leaks with this feature
refs #9742
- rebase against master updated some docs links again
- go over code base again and double check that all docs links are correct
- 2.0 will become the latest version on our readme pages
no issue
- these tests were a mess
- we had many duplicated tests
- it was very hard to work with the exporter files
- i have tidied up the whole file
- first section: all general importer tests based on the current Ghost version
- second section: 1.0 tests
- everything is now JSON based (much easier to control)
refs 9742
- when we've introduced Ghost 1.0, we have noticed that we broke Disqus comments
- Disqus comments use a unique identifier, which is the post id
- that means if you have exported your LTS content and imported it into 1.0, all resource identifiers are regenerated
- but the Disqus must use the original post resource id
- that's why we have imported the old post id and remembered it in the `amp` field 🤠
- that was the only field which was available and un-used
- now in Ghost 2.0, we would like to rename the `amp` field to `comment_id`
- string length: 50 (i thought it might be safer to not use 24 characters, because who knows if support other comment id's in the future)
refs https://github.com/TryGhost/Ghost/issues/9724
- perform a HEAD request on a url if we don't find a matching provider, following any redirects until we hit success response before looking up providers for the resulting url
refs #9601
- on resource update/add we have cached mobiledoc, html etc
- we have to ensure we exclude the fields (same procdure happens on bootstrap)
- these excluded fields don't have to be cached
- otherwise memory usage is higher in general
- ensure we cache relations with a minimal field set on resource update/add
refs #9742, refs #8719
- make it possible to import more tables (optional)
- available tables: clients, trusted domains
- by default we won't import these tables, you have to tell Ghost using `include` (same syntax on export)
- we won't announce this ability for now (stays hidden)
refs #9742, refs #8719
- you can now use `include` to export extra tables e.g. `include=clients`
- admin client won't make use of this option yet, maybe later and optional
- we won't announce this new ability for now (stays hidden)
no issue
- add a new migration for 1.25 to insert the draft demo post for existing blogs
- ensure new blogs get the draft demo post as well
- tested on sqlite3 + mysql
- added handling if Ghost Author user doesn't exist anymore (fallback to owner user)
refs https://github.com/TryGhost/Ghost/issues/9742
We've identified some changes we need to make to the HTML output of the [new Koenig editor](
https://forum.ghost.org/t/koenig-editor-beta-release/1284/102) for future proofing and consistency across cards.
- the `<div class="kg-post">` wrapper around post content has been removed
- for image cards the `.kg-image-wide` and `.kg-image-full` classes have been changed to `.kg-width-wide` and `.kg-width-full` and applied to the `<figure>` element rather than the `<img>` element
Before:
```html
<div class="kg-post">
<figure class="kg-image-card">
<img class="kg-image kg-image-wide" src="...">
<figcaption>example wide image</figcaption>
</figure>
</div>
```
After:
```html
<figure class="kg-image-card kg-width-wide">
<img class="kg-image" src="...">
<figcaption>example wide image</figcaption>
</figure>
```
refs #9742
- Ghost 2.0 is coming
- all doc links in 1.0 must use concrete links e.g. docs.ghost.org/v1 or themes.ghost.org/v1.23.0/
- if we release Ghost 2.0, docs.ghost.org will show 2.0 docs
closes#9727
- this is a short term fix to proof that Disqus comments on preview pages no longer appear on other threads
- this is not a full solution to the problem
- the private API still returns /404/, which is right now inconsistent, but not critical in any way
- the url helper will now output the post preview url if you serve a draft/scheduled post
- this should register unique page urls at Disqus and ensure uniquness for threads
- i still don't understand why the cross posting happens at all, because we also pass an unique identifier to Disqus (the post ID)
- it could be that comments, which are added on the preview page, won't appear on the published urls, because
the published url !== preview url. I wasn't able to figure this out via testing or reading their docs
refs https://github.com/TryGhost/Ghost/issues/9623
- added `DomModifier` class to walk a SimpleDom document and modify as needed
- adds `id` attributes to `h1`, `h2`, etc heading tags
- converts H* tag content to a dasherized string for the id attribute (dasherized id's are different to the smushed ids that are generated by our markdown converted but there are no backwards-compatibility concerns here)
- if a duplicate id is detected then add a `-1`, `-2`, etc suffix to the id
- use `DomModifier` after converting mobiledoc to SimpleDom but before serialising to html
- switched top-level var declarations to es6
refs #9601
- this was already working for collections or channels
- but the `routeName` was not parsed for static routes
- ensure we push the route name into the context object
e.g. /about/: about
-> name of the route is "about"
no issue
- discovered while testing (issue doesn't exist)
- with dynamic routing we have introduced a bug
- we pre-generate urls and if your permalink contains a date (dated permalink),
we have to regenerate the urls, otherwise the dated urls do not respect your blog timezone
- collection router has to subscribe to the timezone event
- collection router must trigger an update on it's url generator if the timezone changes and the it's permalink is dated
- ensure we also update the urls on import
no issue
- reported in the forum: https://forum.ghost.org/t/publishing-with-a-single-post-request-to-posts/1648
- the defaults are defined in two places
1. on the schema level (defaults for the database)
2. on the ORM (model layer)
- the defaults on the db layer are set correctly when inserting a new resource
- but if we don't apply all defaults on the model layer, it will happen that model events are emitted without the correct defaults
- see comment in code base
- it's caused by the fact that knex only returns the inserted resource id (probably caused by the fact knex has to support x databases)
- components/modules are listening on model events and expect:
1. a complete set of attributes
2. a complete set of defaults
3. sanitized values e.g. bool, date
- this commit fixes:
1. added missing defaults for user & post model
2. sanitize booleans (0|1 => false|true)
3. added tests to ensure this works as expected
4. clarfies the usage of `defaults`
Regarding https://forum.ghost.org/t/publishing-with-a-single-post-request-to-posts/1648:
- the post event was emitted with the following values {page: undefined, featured: undefined}
- the urlservice receives this event and won't match the resource against collection filters correctly
- NOTE: the post data in the db were correct
refs #9601
- replace jsonpath with [NQL](https://github.com/NexesJS/NQL)
- jsonpath was just a temporary solution (a short-term fix)
- with NQL we are able to filter collections more powerful in the near future
- NQL is not feature complete
- we still support `featured:true` for collections
refs #9601
- you can now use `rss:false`
- ability to define a custom rss url with a target template (+ content_type)
- ability to disable rss for channel or collection
refs #9601
- refactor architecture of routing so you can define a channel
- a channel is a different way of looking at your posts (a view)
- a channel does not change the url of a resource
Example channel
```
routes:
/worldcup-2018-russia/:
controller: channel
filter: tag:football18
data: tag.football18
```
- added ability to redirect resources to a channel/static route
- support templates for channels
- ensure we still support static routes (e.g. /about/: home)
- ensure pagination + rss works out of the box
refs #9601
- support data, limit and order for collections
- limit definition in routes.yaml is stronger than theme package.json limit configuration
- ensure we update hbs template options
refs #9601
- the home.hbs behaviour for the index collection (`/`) is hardcoded in Ghost
- we would like to migrate all existing routes.yaml files
- we only replace the file if the contents of the routes.yaml file equals the old routes.yaml format (with home.hbs as template)
- updated README of settings folder
- if we don't remove the home.hbs template from the default routes.yaml file, home.hbs will be rendered for any page of the index collection
- the backwards compatible behaviour was different
- only render home.hbs for page 1
- remember: the default routes.yaml file reflects how Ghost was working without dynamic routing
refs #9601
Example:
```
collections:
/podcast/:
permalink: /{slug}/
```
- the name of the collection is remembered as `routerName` (in the case above: "podcast")
- the name of the collection is important for two things
1. context value
2. template name
- the context value is available for specific theme helpers e.g. is helper, body_class helper
- we auto-lookup the collection name in your theme e.g. podcast.hbs
- this logic does not apply to static routes
- if you define templates on your collection, they are stronger than the collection name
refs #9601
- implementation of resource listener updated
- if you define two collections: `featured:true` (1) and `page:false` (2) you can run into the following bug:
- you create a published post (owned by (2))
- you change the status to featured
- still owned by (2), because the filter still matches (it's still not a static page)
- this adaption fixes the behaviour
- less smart logic, but less error prone
closes https://github.com/TryGhost/Ghost/issues/9674
- with dynamic routing the first collection get's the "index" context attached
- the index context signalises the main post listening route (first collection)
- this behaviour was present < 1.24 - we have to keep this behaviour
no issue
- if you define no collections, but a static route, it can happen that the target template to render
makes use of the {{ghost_head}} helper
- the {{ghost_head}} helper tries to create the primary rss feed url
- at the moment: no collections, no primary rss feed url
- if we offer the option to define custom rss rules, this function might need an extension
no issue
- this mock eat already too much of my/our time
- the idea of adding a knex mock was definitely a failed approach/try
- it's too much to maintaince and have not found a module which does this already
- we have to support any query format
- this is too crazy
- the idea was to use the knex mock for model unit tests, because if we want to unit test models we have to
run through bookshelf, because the whole model layer depends on bookshelf e.g. events
- for now we simply use the real database
- we could use the sqlite3 memory mode, but that would mean every unit test runs on sqlite3
- something to consider for later e.g. run unit tests on one matrix
- run the rest on another matrix for sqlite + mysql
closes#9675
- with dynamic routing we have introduced a breaking change, which we have overseen
- Ghost does not return absolute urls, that's why the clients need to concat the blog url and the resource url
- with 1.24.0 Ghost returned resource urls including the subdirectory
- this caused trouble for e.g. zapier or the preview feature in the admin client
- revert breaking change and ensure we only expose resource urls without subdirectory
refs https://github.com/TryGhost/Ghost/issues/9623
- add `oembed-parser` module for checking provider availability for a url and fetching data from the provider
- require it in the `overrides.js` file before the general Promise override so that the `promise-wrt` sub-dependency doesn't attempt to extend the Bluebird promise implementation
- add `/oembed` authenticated endpoint
- takes `?url=` query parameter to match against known providers
- adds safeguard against oembed-parser's providers list not recognising http+https and www+non-www
- responds with `ValidationError` if no provider is found
- responds with oembed response from matched provider's oembed endpoint if match is found
no issue
- was introduced with dynamic routing beta: https://github.com/TryGhost/Ghost/releases/tag/1.24.0
- the slug param wasn't forwarded correctly
- you were not able to render a custom tag or author template e.g. `tag-news.hbs`
closes#9674
- the collection router had a hardcoded default context "home"
- this is wrong
- the context array get's automatically filled for the collection
- if you are serving a page e.g. /page/2/ -> it's "paged"
- if you are serving / -> it's "home"
- same for {{body_class}}, it outputs "home-template" on "/"
- this is the same behaviour as in 1.23.x
no issue
- reverse must happen once in the constructor
- otherwise we reverse the array on each request
- Ghost would randomly pick the first and then the second template
no issue
- from now on: you have to manually reconfigure your slack hook after importing your data
- we were running into trouble that Ghost had import slack hooks, because it can happen very fast
that you are importing someone's slack hook
no issue
- there was a timing bug in Ghost
- we do operations in parallel on bootstrap
- 1) we fetch the resources as early as possible
- 2) we do all the rest (express bootstrapping, theme loading, router registration) etc.
- it can happen that (2) happens too slow and ends in the situation that the queue, which is responsible
to handle both parallel actions, does not wait for the routers and closes the event
- this is a short term fix
- i need to reconsider if there is a better long term fix
refs #9601
### Dynamic Routing
This is the beta version of dynamic routing.
- we had a initial implementation of "channels" available in the codebase
- we have removed and moved this implementation
- there is now a centralised place for dynamic routing - server/services/routing
- each routing component is represented by a router type e.g. collections, routes, static pages, taxonomies, rss, preview of posts
- keep as much as possible logic of routing helpers, middlewares and controllers
- ensure test coverage
- connect all the things together
- yaml file + validation
- routing + routers
- url service
- sitemaps
- url access
- deeper implementation of yaml validations
- e.g. hard require slashes
- ensure routing hierarchy/order
- e.g. you enable the subscriber app
- you have a custom static page, which lives under the same slug /subscribe
- static pages are stronger than apps
- e.g. the first collection owns the post it has filtered
- a post cannot live in two collections
- ensure apps are still working and hook into the routers layer (or better said: and register in the routing service)
- put as much as possible comments to the code base for better understanding
- ensure a clean debug log
- ensure we can unmount routes
- e.g. you have a collection permalink of /:slug/ represented by {globals.permalink}
- and you change the permalink in the admin to dated permalink
- the express route get's refreshed from /:slug/ to /:year/:month/:day/:slug/
- unmount without server restart, yey
- ensure we are backwards compatible
- e.g. render home.hbs for collection index if collection route is /
- ensure you can access your configured permalink from the settings table with {globals.permalink}
### Render 503 if url service did not finish
- return 503 if the url service has not finished generating the resource urls
### Rewrite sitemaps
- we have rewritten the sitemaps "service", because the url generator does no longer happen on runtime
- we generate all urls on bootstrap
- the sitemaps service will consume created resource and router urls
- these urls will be shown on the xml pages
- we listen on url events
- we listen on router events
- we no longer have to fetch the resources, which is nice
- the urlservice pre-fetches resources and emits their urls
- the urlservice is the only component who knows which urls are valid
- i made some ES6 adaptions
- we keep the caching logic -> only regenerate xml if there is a change
- updated tests
- checked test coverage (100%)
### Re-work usage of Url utility
- replace all usages of `urlService.utils.urlFor` by `urlService.getByResourceId`
- only for resources e.g. post, author, tag
- this is important, because with dynamic routing we no longer create static urls based on the settings permalink on runtime
- adapt url utility
- adapt tests
no issue
- discovered while coding
- the value was always false, because we've tried to read the value from the config object
- the value lives in the database and is accessible via the labs service
no issue
- we sanitise any incoming slug on the model layer e.g uppercase -> lowercase
- and when importing e.g. an uppercase slug, the importer was trying to compare the uppercase slug with the sanitised slug
closes https://github.com/TryGhost/Ghost/issues/9620
- adjust the `deduplicateSubDir` function's regex to only match duplicate subdirectories when the `url` is only a path rather than full url or the duplicate match starts with a `/`
closesTryGhost/Support#426
refs TryGhost/gscan#106
needs TryGhost/gscan#107
GScan can return errors, which was not handled in our theme validator and caused Ghost to crash completely. GScan will now return an Ignition error when its not able to read the `.zip` file.
e. g.: `{"errors":[{"message":"Failed to read zip file","context":"tife.zip","errorType":"ValidationError","errorDetails":"invalid relative path: ../tife/"}]}`
refs https://github.com/TryGhost/Ghost/issues/9505
- remove requirement for the `enableDeveloperExperiments` flag to be able to use Koenig
- it's now possible to enable as a standard Labs beta feature
refs https://github.com/TryGhost/Ghost/issues/9505
- updates mobiledoc converter's `render` method to accept a `version` argument
- `1` === Ghost 1.0's markdown-only renderer output
- `2` === Koenig's full mobiledoc renderer output
- switch between mobiledoc renderer versions in Post model's `onSaving` hook
- version 1 by default
- version 2 if Koenig is enabled (currently behind dev experiments config + labs flag)
- version 2 if the post's mobiledoc is not compatible with the markdown-only renderer
- "version 2" full-Koenig mobiledoc renderer output
- wraps content in a `.kg-post` div
- removes wrapper around markdown and html card output
- adds classes to image card output including selected image size/style
- standardises es6 usage across mobiledoc related files
refs #9601
- replace raw knex queries by Bookshelf queries
- optimise lot's of test setups, so we don't experience a massive slow down in the test run
- this has troubled in the past e.g. with normalisation, any custom model logic - the test env always had to simulate things
- there are for sure thousands things which can be optimised now, but because of time, we do them step by step
- this is especially important for the url service (https://github.com/TryGhost/Ghost/issues/9601), because we have to ensure that inserting/updating/removing resources will trigger model events
`grunt test-all` with SQLite finishes in 2,5-3min. (on master: 1-2min)
`grunt test-all` with MySQL finishes in 4min. (on master: 3min)
**NOTE: We want to move as much as possible routing and integration tests to unit tests. This will speed up the test run again.** See #9342. But we need to find time for that. Any help is welcome!
refs https://github.com/TryGhost/Ghost/pull/9592
- we add bookshelf-relations step by step if we need it
- with https://github.com/TryGhost/Ghost/pull/9592 we have rewritten the test env to use Bookshelf
- this is important for our new url service
- because the service is listening on model updates and updates the urls based on the model events
- so with moving to Bookshelf, we need any easy way to add relations
- the test env inserts test fixtures
- it adds permissions and each permission get's roles attached
- `models.Permission.add({roles: [...]})
no issue
- replaced token creation by `lib.common.security`
- added unit tests for adding invites
- allow a different invite status for internal access
refs https://github.com/TryGhost/Team/issues/65
- it's easier for the architecture if we read the setting files synchronously,
because the dynamic routing component is part of the express bootstrap and
the whole routing bootstrap is synchronously
- for now: we only read one file anyway
- it's for now easier to read the file synchronously, then i don't have to change
any existing express bootstrap architecture
refs https://github.com/TryGhost/Team/issues/65
- this is just the first optimisation regarding relative/absolute urls
- the full strike will happen when i start with the url utility re-write
- for now: there will be only one subscriber of url events -> the sitemaps service
- the sitemaps service outputs absolute urls
- we don't want to receive an url event and ask the url service again to get an absolute version of the url
closes#9569
- Removed the `<1 min read` time clause, effectively making `1 min read` the minimum reading time
- Removed the `seconds` option for i18n strings, which contained the less than one minute display string
- Kept the other i18n string options the same
- Amended and improved tests for new functionality
refs https://github.com/TryGhost/Team/issues/65
- currently we generate a relative resource url
- if you configure a subdirectory, the urls have to respect that
- e.g. you configure `localhost:2368/blog`, your url results in e.g. `/blog/my-post/`
- this is not yet a critical bug, because the url service is not connected yet
- @TODO: consider absolute vs. relative urls in the url service
no issue
- removed the `routeKeywords` property from the config and used hard coded keywords.
- removed `routeKeywords` from public configuration API endpoint, as it's no longer used in the Admin.
refs https://github.com/TryGhost/Team/issues/65
We are currently work on dynamic routing (aka channels).
An important piece of this feature is the url service, which always knows the url of a resource at any time.
Resources can belong to collections or taxonomies, which can be defined in a [routing yaml file](https://github.com/TryGhost/Ghost/issues/9528). We are currently shipping portions, which will at end form the full dynamic routing feature.
### Key Notes
- each routing type (collections, taxonomies, static pages) is registered in order - depending on the yaml routes file configuration
- static pages are an internal concept - they sit at the end of the subscriber queue
- we make use of a temporary [`Channels2`](https://github.com/TryGhost/Ghost/pull/9550/files#diff-9e7251409844521470c9829013cd1563) file, which simulates the current static routing in Ghost (this file will be modified, removed or whatever - this is one of the next steps)
- two way binding: you can ask for a resource url based on the resource id, you can ask for the resource based on the url
- in theory it's possible that multiple resources generate the same url: we don't handle this with collision (because this is error prone), we handle this with the order of serving content. if you ask the service for a resource, which lives behind e.g. /test/, you will get the resource which is served
- loose error handling -> log errors and handle instead of throw error and do nothing (we log the errors with a specific code, so we can react in case there is a bug)
- the url services fetches all resources on bootstrap. we only fetch and keep a reduced set of attributes (basically the main body of a resource)
- the bootstrap time will decrease a very little (depending on the amount of resources you have in your database)
- we still offer the option to disable url preloading (in your config `disableUrlPreload: true`) - this option will be removed as soon as the url service is connected. You can disable the service in case you encounter a problem
- **the url service is not yet connected, we will connect the service step by step. The first version should be released to pre-catch bugs. The next version will add 503 handling if the url service is not ready and it will consume urls for resources.**
----
- the url service generates urls based on resources (posts, pages, users, tags)
- the url service keeps track of resource changes
- the url service keeps track of resource removal/insert
- the architecture:
- each routing type is represented by a url generator
- a routing type is a collection, a taxonomiy or static pages
- a queue which ensures that urls are unique and can be owned by one url generator
- the hierarchy of registration defines that
- we query knex, because bookshelf is too slow
- removed old url service files + logic
- added temp channels alternative (Channels2) -> this file will look different soon, it's for now the temporary connector to the url service. Also the name of the file is not optimal, but that is not really important right now.
no issue
- support more cases
- e.g. multiple where matches
- @TODO
- take time to look for NPM module, which does this already
- test sqlite3 :memory: mode again
closes#9528
These code changes introduce a YAML parser which will load and parse YAML files from the `/content/settings` directory. There are three major parts involved:
1. `ensure-settings.js`: this fn takes care that on bootstrap, the supported files are present in the `/content/settings` directory. If the files are not present, they get copied back from our default files. The default files to copy from are located in `core/server/services/settings`.
2. `loader.js`: the settings loader reads the requested `yaml` file from the disk and passes it to the yaml parser, which returns a `json` object of the file. The settings loader throws an error, if the file is not accessible, e. g. because of permission errors.
3. `yaml-parser`: gets passed a `yaml` file and returns a `json` object. If the file is not parseable, it returns a clear error that contains the information, what and where the parsing error occurred (e. g. line number and reason).
- added a `get()` fn to settings services, that returns the settings object that's asked for. e. g. `settings.get('routes').then(()...` will return the `routes` settings.
- added a `getAll()` fn to settings services, that returns all available settings in an object. The object looks like: `{routes: {routes: {}, collections: {}, resources: {}}, globals: {value: {}}`, assuming that we have to supported settings `routes` and `globals`.
Further additions:
- config `contentPath` for `settings`
- config overrides for default `yaml` files location in `/core/server/services/settings`
**Important**: These code changes are in preparation for Dynamic Routing and not yet used. The process of copying the supported `yaml` files (in this first step, the `routes.yaml` file) is not yet activated.
no issue
- required for model events
- otherwise you won't receive a full data set
- in worst case you have to re-fetch the post
- required for the url service
- the url service always needs relations (authors,tags) to be able to generate the url properly
@IMPORTANT
- no API change, we still return what you are asking for
- we first edit/add the resource
- then we fetch the data with the API options
- @TODO: this can be optimised and will improve performance
picking/selecting it from the insert/update response
- this is an internal change
closes#9547
- you setup a blog with the following owner:
- email: test@ghost.org
- name: test
- slug: test
- now you import a JSON db file, which holds the exact same owner
- this owner won't be imported, because it's a duplicate
- but the slug is different (!)
- the importer tries to find a matching existing user, but won't find anything
- the importer then send an empty authors array `post.authors=[]` into the model layer
- this is not allowed -> this would mean, you are actively trying to unset all authors
no issue
- if multiple queries run in a transaction, the model events are triggered before the txn finished
- if the txn rolls back, the events are anyway emitted
- the events are triggered too early
- solution:
- `emitChange` needs to detect that a transaction is happening
- it listens on a txn event to determine if events should be triggered
no issue
- if you delete all content, we expect two events
- `post.deleted` and `post.unpublished`
- `post.unpublished` was never triggered, because the api implementation made use of `collection.invoke(`destroy`)`
- what happened?
- you fetch all posts (columns:id)
- you destroy the post (only id column is available)
- the model events are triggered
- but you have no access to a default set of data
- the result is that the event handler can't even tell if this is a post or a page
- added a proper test to ensure which events are triggered
refs #9548
- we always receive date strings from the client in ISO format
- we ensure that we transform these strings into JS dates for comparison
- when the client sends relations, we need to ensure that relations are checked as well
- will only work for the post model for now, because this is the only model which uses `bookshelf-relations`
- added unit tests
- removed some model tests, which do the same
refs #9548
- do not forward `tag.parent` to the model layer
- the model layer should only know `tag.parent_id`
- and the API should only expose `tag.parent` (this is an API feature)
- currently Ghost has a mixture of using `toJSON` and the API validation layer for this
- we just continue with this for now (no time to fix this)
- disallow sending nested-nested relations
- unsupported
- see comment for more information
- this can cause problems with calling `hasChanged` on relations
- add unit tests
no issue
This PR adds the server side logic for multiple authors. This adds the ability to add multiple authors per post. We keep and support single authors (maybe till the next major - this is still in discussion)
### key notes
- `authors` are not fetched by default, only if we need them
- the migration script iterates over all posts and figures out if an author_id is valid and exists (in master we can add invalid author_id's) and then adds the relation (falls back to owner if invalid)
- ~~i had to push a fork of bookshelf to npm because we currently can't bump bookshelf + the two bugs i discovered are anyway not yet merged (https://github.com/kirrg001/bookshelf/commits/master)~~ replaced by new bookshelf release
- the implementation of single & multiple authors lives in a single place (introduction of a new concept: model relation)
- if you destroy an author, we keep the behaviour for now -> remove all posts where the primary author id matches. furthermore, remove all relations in posts_authors (e.g. secondary author)
- we make re-use of the `excludeAttrs` concept which was invented in the contributors PR (to protect editing authors as author/contributor role) -> i've added a clear todo that we need a logic to make a diff of the target relation -> both for tags and authors
- `authors` helper available (same as `tags` helper)
- `primary_author` computed field available
- `primary_author` functionality available (same as `primary_tag` e.g. permalinks, prev/next helper etc)
closes#9520
- it contains a dependency bump of the latest Bookshelf release
- Bookshelf introduced a bug in the last release
- see https://github.com/bookshelf/bookshelf/pull/1583
- see https://github.com/bookshelf/bookshelf/pull/1798
- this has caused trouble in Ghost
- the `updated_at` attribute was not automatically set anymore
---
The bookshelf added one breaking change: it's allow to pass custom `updated_at` and `created_at`.
We already have a protection for not being able to override the `created_at` date on update.
We had to add another protection to now allow to only change the `updated_at` property.
You can only change `updated_at` if you actually change something else e.g. the title of a post.
To be able to implement this check i discovered that Bookshelfs `model.changed` object has a tricky behaviour.
It remembers **all** attributes, which where changed, doesn't matter if they are valid or invalid model properties.
We had to add a line of code to avoid remembering none valid model attributes in this object.
e.g. you change `tag.parent` (no valid model attribute). The valid property is `tag.parent_id`.
If you pass `tag.parent` but the value has **not** changed (`tag.parent` === `tag.parent_id`), it will output you `tag.changed.parent`. But this is wrong.
Bookshelf detects `changed` attributes too early. Or if you think the other way around, Ghost detects valid attributes too late.
But the current earliest possible stage is the `onSaving` event, there is no earlier way to pick valid attributes (except of `.forge`, but we don't use this fn ATM).
Later: the API should transform `tag.parent` into `tag.parent_id`, but we are not using it ATM, so no need to pre-optimise.
The API already transforms `post.author` into `post.author_id`.
closes#9507
- Changed the utils.wordCount implementation to the one used by simpleMDE
- Added extra À-ÿ to the regex to support diacritics characters
- Added corresponding text with Chinese text mentioned in the issue
closes#9495
- Added a clause for amp being disabled
- In this clause, we strip the final 'amp/' part of the url, and redirect
- Changed corresponding test in frontend_spec.js
- Used `urlService.utils.redirect301()` instead of `res.redirect()`
refs https://github.com/TryGhost/Ghost/issues/9311
- very basic implementation, still needs proper classes and default stylesheet implementation
- change image card output to a `<figure>` with optional `<figcaption>`
- add optional `<p>` caption output to the html card
refs #9200
- We have not yet counted the images within your html, this commit counts images based on the this algorithm: https://blog.medium.com/read-time-and-you-bc2048ab620c
- Added imageCount utility, which counts images using an img-tag regex, amended from the general tag-regex found in wordCount
- Added this imageCount to the {{reading_time}} helper, adding 12 seconds to the reading time for every image
- The feature image is still counted as before
- The first image adds 12 seconds, the second 11, the third 10, and so on
- Images from the tenth onwards add 3 seconds to the reading time
closes#9085
Fixes an issue, where the client sets image properties to `""` after deleting the image. This causes problems with the query filter (see https://github.com/TryGhost/GQL/issues/24), as they have to be `null`.
Added a check in the model layer saving method to set value to `null`, when the property is empty.
Affected models and properties:
- `posts`:
- `feature_image`
- `og_image`
- `twitter_image`
- `users`:
- `profile_image`
- `cover_image`
- `tags`:
- `feature_image`
no issue
- the tests were failing since beginning of March
- this was caused by a wrong assertion in one of our authentication tests
- we work with a static 6 month ms number for token expiry
- this static ms number is based on 30 days per month
no issue
- extended functionality
- the knex mock simply parses the sql statements and serves data from memory
- i've tested the memory mode of sqlite, but could not get it working
- but maybe for the future to test again
no issue
- the handling here was not correct
- if you've passed no mobiledoc, it wasn't adding mobiledoc and an undefined html value
- we need a default mobiledoc+html value in case you don't pass the values within the test cases
no issue
- `post.author_id` has no reference to any table currently, see https://github.com/TryGhost/Ghost/blob/1.21.3/core/server/data/schema/schema.js#L19
- that's why it is right now possible to insert none existent author id's
- with multiple authors, this get's protected (see https://github.com/TryGhost/Ghost/pull/9426)
- you would get a proper error message
- it is not allowed to insert invalid author id's
- as soon as you do `include=author` you would receive an error
- fixed one test case where we inserted an invalid author id via the API
no issue
- just discovered that we had confusing function names in our test utility
- e.g. `posts` -> default posts from the data generator
- e.g. `users` -> extra users not from our data generator
- now:
- e.g. `posts` -> default posts from the data generator
- e.g. `users` -> default users from the data generator
- e.g. `users:extra` -> extra users not from our data generator
no issue
- currently if you would like to edit a resource (e.g. post) and you pass an invalid model id, the following happens
- permission check calls `Post.permissible`
- the Post could not find the post, but ignored it and returned `userPermissions:true`
- then the model layer is queried again and figured out that the post does not exist
- A: there is no need to query the model twice
- B: we needed proper error handling for post and role model
no issue
- replace logic for preparing nested tags
- if you have nested tags in your file, we won't update or update the target tag
- we simply would like to add the relationship to the database
- use same approach as base class
- add `posts_tags` to target post model
- update identifiers
- insert relation by foreign key `tag_id`
- bump bookshelf-relations to 0.1.10
no issue
- change behaviour from updating user references after the actual import to update the user reference before the actual import
- updating user references after the import is way less case intense
- that was the initial decision for updating the references afterwards
- but that does not play well with adding nested relations by identifier
- the refactoring is required for multiple authors
- if we e.g. store invalid author id's, we won't be able to add a belongs-to-many relation for multiple authors
- bookshelf-relations is generic and always tries to find a matching target before attching a model
- invalid user references won't work anymore
- this change has a very good side affect
- 17mb takes on master ~1,5seconds
- on this branch it takes ~45seconds
- also the memory usage is way lower and stabler
- 40mb takes 1,6s (times out on master)
no issue
- Ghost does not support adding an author by relation (`post.author = {id: '..'}`)
- Ghost does not support editing an author by relation (`post.author = {id: '..'}`)
- only `author_id` is allowed
refs https://github.com/TryGhost/Ghost/issues/3658
- the `validateSchema` helper was a bit broken
- if you add a user without email, you will receive a database error
- but the validation error should catch that email is passed with null
- it was broken, because:
- A: it called `toJSON` -> this can remove properties from the output (e.g. password)
- B: we only validated fields, which were part of the JSON data (model.hasOwnProperty)
- we now differentiate between schema validation for update and insert
- fixed one broken import test
- if you import a post without a status, it should not error
- it falls back to the default value
- removed user model `onValidate`
- the user model added a custom implementation of `onValidate`, because of a bug which we experienced (see https://github.com/TryGhost/Ghost/issues/3638)
- with the refactoring this is no longer required - we only validate fields which have changed when updating resources
- also, removed extra safe catch when logging in (no longer needed - unit tested)
- add lot's of unit tests to proof the code change
- always call the base class, except you have a good reason
no issue
- `isNew` does not work in Ghost, because Ghost does not use auto increment id's
- see https://github.com/bookshelf/bookshelf/issues/1265
- see https://github.com/bookshelf/bookshelf/blob/0.10.3/src/base/model.js#L211
- we only had one occurance, which was anyway redundant
- if you add a user, `hasChanged('password') is true
- if you edit a user and the password has changed, `hasChanged('password')` is true as well
NOTE #1:
1. We can't override `isNew` and throw an error, because bookshelf makes use of `isNew` as well, but it's a fallback if `options.method` is not set.
2. It's hard to re-implement `isNew` based on `options.method`, because then we need to ensure that this value is always set (requires a couple of changes)
NOTE #2:
If we need to differentiate if a model is new or edited, we should manually check for `options.method === insert`.
NOTE #3:
The unit tests are much faster compared to the model integration tests.
I did a comparision with the same test assertion:
- unit test takes 70ms
- integration test takes 190ms
no issue
- added https://github.com/colonyamerican/mock-knex as dev dependency
- the mock serves our data generator test data by default
- but you can define your own if you want
- we need a proper mock for unit testing
- we should not mock bookshelf if possible, otherwise we can't test event flows
no issue
- move password hashing and password comparison to lib/security/password
- added two unit test
- FYI: password hashing takes ~100ms
- we could probably mock password hashing in certain cases when unit testing
no issue
- this commit cleans up the usages of `include` and `withRelated`.
### API layer (`include`)
- as request parameter e.g. `?include=roles,tags`
- as theme API parameter e.g. `{{get .... include="author"}}`
- as internal API access e.g. `api.posts.browse({include: 'author,tags'})`
- the `include` notation is more readable than `withRelated`
- and it allows us to use a different easier format (comma separated list)
- the API utility transforms these more readable properties into model style (or into Ghost style)
### Model access (`withRelated`)
- e.g. `models.Post.findPage({withRelated: ['tags']})`
- driven by bookshelf
---
Commits explained.
* Reorder the usage of `convertOptions`
- 1. validation
- 2. options convertion
- 3. permissions
- the reason is simple, the permission layer access the model layer
- we have to prepare the options before talking to the model layer
- added `convertOptions` where it was missed (not required, but for consistency reasons)
* Use `withRelated` when accessing the model layer and use `include` when accessing the API layer
* Change `convertOptions` API utiliy
- API Usage
- ghost.api(..., {include: 'tags,authors'})
- `include` should only be used when calling the API (either via request or via manual usage)
- `include` is only for readability and easier format
- Ghost (Model Layer Usage)
- models.Post.findOne(..., {withRelated: ['tags', 'authors']})
- should only use `withRelated`
- model layer cannot read 'tags,authors`
- model layer has no idea what `include` means, speaks a different language
- `withRelated` is bookshelf
- internal usage
* include-count plugin: use `withRelated` instead of `include`
- imagine you outsource this plugin to git and publish it to npm
- `include` is an unknown option in bookshelf
* Updated `permittedOptions` in base model
- `include` is no longer a known option
* Remove all occurances of `include` in the model layer
* Extend `filterOptions` base function
- this function should be called as first action
- we clone the unfiltered options
- check if you are using `include` (this is a protection which could help us in the beginning)
- check for permitted and (later on default `withRelated`) options
- the usage is coming in next commit
* Ensure we call `filterOptions` as first action
- use `ghostBookshelf.Model.filterOptions` as first action
- consistent naming pattern for incoming options: `unfilteredOptions`
- re-added allowed options for `toJSON`
- one unsolved architecture problem:
- if you override a function e.g. `edit`
- then you should call `filterOptions` as first action
- the base implementation of e.g. `edit` will call it again
- future improvement
* Removed `findOne` from Invite model
- no longer needed, the base implementation is the same
no issue
- Date comparisons are possible via API, but there's no way to inject a valid date into the get helper
- JavaScript's Date.toString() function outputs dates in a useless format
- Swap to using Date.toISOString() and now the format can be understood anywhere!
- {{#get "posts" filter="published_at:<='{{published_at}}'"}}{{/get}} works now as expected
closes#9445
- redirects all asset requests if https is configured (theme, core, images)
- re-use and extend our url-redirect middleware
- add proper integration tests for our express site app (no db interaction, component testing required for such important use cases)
- i added some more general tests
- should avoid mixed content warnings in the browser
closes#9314
* added fixtures for contributor role
* update post api tests to prevent contributor publishing post
* update permissible function in role/user model
* fix additional author code in invites
* update contributor role migration for knex-migrator v3
* fix paths in contrib migration
* ensure contributors can't edit or delete published posts, fix routing tests [ci skip]
* update db fixtures hash
* strip tags from post if contributor
* cleanup post permissible function
* excludedAttrs to ignore tag updates for now (might be removed later)
* ensure contributors can't edit another's post
* migration script for 1.21
no issue
- returning and remembering the data, which was imported, is...
- not required when using the API
- not required when importing via script
- required for tests
- added an option to have control over it
- make more usage of local variables
- the GC cannot tidy up variables, which are defined outside of a loop, but used in the loop
- try to keep less memory in process
- reduce the number of properties we have to remember
no issue
- if you import a JSON file with a post, which has an unknown author,
the target user was removed from the blog
- Ghost can handle this case and still succeeds with import
- but we have stored an `author_id` in the database, which does not map to any user and won't map in the future
- this can trouble if we add support for multiple authors
- currently, we only return the `author_id` to the client and the client can map with `author_id` with users fetched by the API
- if it does not find a user, it just falls back to a different user
- but multiple authors have to be included explicit (`include=authors`) and we will return a mapped (author_id => user) result
- it won't be able to find the user, because we lookup the database
- this would result in an error
- there is in general no reason to import (or store) an unknown/invalid `author_id` into the database
- on import, we show you a warning and you can choose a different author if you want
- solution: fallback to owner user and extend warning
- it's not a behaviour change, you still can import unknown author id's and the import won't fail
- but we ensure valid author id's
- updated test
- further more: returning `author={}` when requesting `include=author` could trouble with ember currently
- it expects the author to be returned
refs #9127
- permission checks can happen everywhere in the code base
- we would like to create a context class
- global access to `options.context.is(...)`
- please read more about the access plugin in #9127 section "Model layer and the access plugin".
- removed the plugin and use direct context checks
no issue
- the affected test had a wrong payload, but passed anyway because the previous test edited the same title ;)
- this routing test does currently truncate the tables after each test case
- the test can run faster
- if we achieve reducing the routing tests, we can reconsider truncating the db
no issue
- reported in slack (https://ghost.slack.com/files/U8QV8DXQB/F8TSBQ532/image.png)
- do not expose old release notification
- e.g. you are on 1.20.0
- you receive a notification for 1.20.1 to update
- you update to 1.20.1
- ensure we protect exposing the release notification (compare against blog version)
- protect against wrong formats
- @TODO: the notifications could store a `version` property
- by that we could use `notification.version` and don't have to match the version in the message
closes#5071
- Remove hardcoded notification in admin controller
- NOTE: update check notifications are no longer blocking the admin rendering
- this is one of the most import changes
- we remove the hardcoded release message
- we also remove adding a notification manually in here, because this will work differently from now on
-> you receive a notification (release or custom) in the update check module and this module adds the notification as is to our database
- Change default core settings keys
- remove displayUpdateNotification
-> this was used to store the release version number send from the UCS
-> based on this value, Ghost creates a notification container with self defined values
-> not needed anymore
- rename seenNotifications to notifications
-> the new notifications key will hold both
1. the notification from the USC
2. the information about if a notification was seen or not
- this key hold only one release notification
- and n custom notifications
- Update Check Module: Request to the USC depends on the privacy configuration
- useUpdateCheck: true -> does a checkin in the USC (exposes data)
- useUpdateCheck: false -> does only a GET query to the USC (does not expose any data)
- make the request handling dynamic, so it depends on the flag
- add an extra logic to be able to define a custom USC endpoint (helpful for testing)
- add an extra logic to be able to force the request to the service (helpful for testing)
- Update check module: re-work condition when a check should happen
- only if the env is not correct
- remove deprecated config.updateCheck
- remove isPrivacyDisabled check (handled differently now, explained in last commit)
- Update check module: remove `showUpdateNotification` and readability
- showUpdateNotification was used in the admin controller to fetch the latest release version number from the db
- no need to check against semver in general, the USC takes care of that (no need to double check)
- improve readability of `nextUpdateCheck` condition
- Update check module: refactor `updateCheckResponse`
- remove db call to displayUpdateNotification, not used anymore
- support receiving multiple custom notifications
- support custom notification groups
- the default group is `all` - this will always be consumed
- groups can be extended via config e.g. `notificationGroups: ['migration']`
- Update check module: refactor createCustomNotification helper
- get rid of taking over notification duplication handling (this is not the task of the update check module)
- ensure we have good fallback values for non present attributes in a notification
- get rid of semver check (happens in the USC) - could be reconsidered later if LTS is gone
- Refactor notification API
- reason: get rid of in process notification store
-> this was an object hold in process
-> everything get's lost after restart
-> not helpful anymore, because imagine the following case
-> you get a notification
-> you store it in process
-> you mark this notification as seen
-> you restart Ghost, you will receive the same notification on the next check again
-> because we are no longer have a separate seen notifications object
- use database settings key `notification` instead
- refactor all api endpoints to support reading and storing into the `notifications` object
- most important: notification deletion happens via a `seen` property (the notification get's physically deleted 3 month automatically)
-> we have to remember a seen property, because otherwise you don't know which notification was already received/seen
- Add listener to remove seen notifications automatically after 3 month
- i just decided for 3 month (we can decrease?)
- at the end it doesn't really matter, as long as the windows is not tooooo short
- listen on updates for the notifications settings
- check if notification was seen and is older than 3 month
- ignore release notification
- Updated our privacy document
- Updated docs.ghost.org for privacy config behaviour
- contains a migration script to remove old settings keys
refs #5345, refs #3801
- Blog localisation
- default is `en` (English)
- you can change the language code in the admin panel, see https://github.com/TryGhost/Ghost-Admin/pull/703
- blog behaviour changes depending on the language e.g. date helper format
- theme translation get's loaded if available depending on the language setting
- falls back to english if not available
- Theme translation
- complete automatic translation of Ghost's frontend for site visitors (themes, etc.), to quickly deploy a site in a non-English language
- added {{t}} and {{lang}} helper
- no backend or admin panel translations (!)
- easily readable translation keys - very simple translation
- server restart required when adding new language files or changing existing files in the theme
- no language code validation for now (will be added soon)
- a full theme translation requires to translate Ghost core templates (e.g. subscriber form)
- when activating a different theme, theme translations are auto re-loaded
- when switching language of blog, theme translations are auto re-loaded
- Bump gscan to version 1.3.0 to support more known helpers
**Documentation can be found at https://themes.ghost.org/v1.20.0/docs/i18n.**
closes#9022
Images without extensions don't need to be manipulated, as we're now reading the bytes and pass those to the `image-size` lib.
This PR adds another `user-agent` to emulate multiple browser requests, as I stumbled over an example where the image without extension is protected otherwise.
Added a test, that works with above mentioned image, but is currently mocked. Nevertheless, the image worked as a PoC, that we're able to read the bytes of an image without its extension and still return the dimensions of the image.
refs https://github.com/TryGhost/Ghost-Release/issues/24
- differentiate between
1. original package.json version (can contain pre and build suffix)
2. full package.json version X.X.X-{pre} (optional)
3. safe package.json version X.X (major+minor)
no issue
- with 29e143fa9a import queries no longer run in parallel
- this commit simply adds a small code snippet to reflect the importer behaviour
1) duplicate slugs *within* a file are getting ignored
2) existing posts in the database and posts to import with the same slug, result in duplicates
Further improvements regarding duplication detection will happen via #8717.
closes#8717
- this is now required, because we run import queries sequentiell
- this code protects two cases:
- you have duplicate slugs in the JSON file (the first get's inserted, the second get's ignored)
- you have an existing slug in the database and you try to import the same slug, get's ignored
refs #9178, refs #8988
With 7353c87d7f we use Bluebird globally for Promises. Therefore, the request lib doesn't need to be wrapped in a bluebird Promise anymore.
This was originally done, so we can work with catch predicated in our image-size lib.
Updated the tests to proof, that the catch predicates work.
The tests fail, as soon as the Promise overwrite is commented out.
refs #9178
- this util uses the url services (!)
- moving this file into lib would not make sense right now
- that would mean a module requires first ../lib/url, which then requires ../services/url
- the url service definitely need a clean up 😃
refs #9178
- not 100% sure about this, but i think it makes right now the most sense
- we have already a url service and creating another lib/url is confusing at the moment
- i'll copy the last utility `makeAbsoluteUrls` to the url service for now
- see next commit for explanation (!)
refs #9178
- `checkFileExists` and `checkFileIsValid` where dirty required from web/middleware
- these two functions are only used in the target middleware
- let's move them
refs #9178
- i am not super happy about `const imageLib = require('../lib/image')`
- i don't really like the name `imageLib`
- but i had no better idea 😃
- if we use the same name in the whole project, it's very easy to rename the folder or the variable
no issue
> Deprecation warning: value provided is not in a recognized ISO format. moment construction falls back to js Date(), which is not reliable across all browsers and versions.
no issue
> (node:63849) Warning: Possible EventEmitter memory leak detected. 101 settings.edited listeners added. Use emitter.setMaxListeners() to increase limit
- the settings cache was initialised per test
- it registered the model events over and over again
- add a simple shutdown function, which can be called from the test env
refs #9178
- they definitely don't belong to server/utils
- i think the best place is putting them into the card apps
- the the post model needs to ask the app for it's converters
- move tests as well
refs #9178
- first iteration of tidying up the unit tests
- this is useful in the current stage, because if i move files in the server folder, i need a clean folder/file structure to detect which tests needs to move
- this is a simple cleanup to reflect the current server folder structure
no issue
- this test fails sometimes, i recently increased the socket delay (maybe it was not enough)
- because these are milliseconds
- let's try 100ms
refs #9178
- continue with killing our global utils folder
- i haven't found any better naming for lib/promise
- so, require single files for now
- instead of doing `promiseLib = require('../lib/promise')`
- we can optimise the requires later
refs #9178
- each package/module has a local utility (e.g. api, helpers, adapters)
- these are very small utility functions which are only used from this package
- they don't belong into the global lib/utils
refs #9178
- this logic belongs to a static model helper
- the visibility property is a model property, the knowledge about the visibility values belongs to the model
- rename the functions, so they make more sense
refs #9178
- we have to take care that we don't end up in circular dependencies
- e.g. API requires UrlService and UrlService needs to require the API (for requesting data)
- update the references
- we would like to get rid of the utils folder, this is/was the most complicated change
refs #9178
- move express apps to one place (called `web`)
- requires https://github.com/TryGhost/Ghost-Admin/pull/923
- any further improvements are not part of this PR
- this PR just moves the files and ensures the paths are up-to-date
refs #9178
- the first test will start the ghost server
- all following test will reuse the server and only truncate the database tables
- you can use `forceStart` if you need
- we have to ensure that reusing the server requires us to
- e.g. reload the settings
- e.g. reload active theme
- remove fork functionality (you can use `forceStart: true`)
- no test has changed, just the way we start the server
no issue
- adapt major changes of knex-migrator v3
- adapt migration scripts, simplify and add `down` (rollback) hook if possible
- clear Ghost cache after init hook (because of `knex-migrator migrate --init`)
- ensure db migrations work with the CLI
- updated troubleshooting guide (https://docs.ghost.org/v1/docs/troubleshooting#section-task-execute-is-not-a-function)
**For development only: Please ensure you run `npm i -g knex-migrator@latest` to update your global installation to v3. We always prefer the local installation, but v3 has modified and added binaries.**
no issue
- i saw that the channels routing tests were sometimes not running in Travis
e.g. https://travis-ci.org/TryGhost/Ghost/jobs/311303939
> Channel Routes
> Unhandled rejection Error: select "webhooks".* from "webhooks" where "event" = 'subscriber.added' - SQLITE_ERROR: no such table: webhooks
no issue
- https://github.com/mochajs/mocha/blob/master/CHANGELOG.md#400--2017-10-02
- the new `--exit` flag might be interesting at some point
> In Mocha v3.0.0 and newer, returning a Promise and calling done() will result in an exception.
- adapt teardown/setup test utility
- adapt other mixed usages of callback && Promise usage
refs https://github.com/TryGhost/Team/issues/41
- differentiate error codes
- return 404 if image was not found
- else return a 500
- use i18n keys
- use errors.utils.isIgnitionError (!)
refs https://github.com/TryGhost/Team/issues/41, refs https://github.com/TryGhost/gscan/issues/85
- if you are using the pagination helper not inside a resource context, you will receive an error
- improve error message, because it was not clear what happened
- downgrade error level to normal, because it's not a critical error from Ghost's perspective, from user perspective it is
- added help docs link and added a callout to our docs
no issue
Support for http://resthooks.org style webhooks that can be used with Zapier triggers. This can currently be used in two ways:
a) adding a webhook record to the DB manually
b) using the API with password auth and POSTing to /webhooks/ (this is private API so not documented)
⚠️ only _https_ URLs are supported in the webhook `target_url` field 🚨
- add `webhooks` table to store event names and target urls
- add `POST` and `DELETE` endpoints for `/webhooks/`
- configure `subscribers.added` and `subscribers.deleted` events to trigger registered webhooks
no issue
- added https://github.com/TryGhost/bookshelf-relations as dependency
- remove existing tag handling
---
* Important: Ensure we trigger parent initialize function
- otherwise the plugin is unable to listen on model events
- important: event order for listeners is Ghost -> Plugin
- Ghost should be able to listen on the events as first instance
- e.g. be able to modify/validate relationships
* Fix tag validation
- we detect lower/update case slugs for tags manually
- this can't be taken over from the plugin obviously
- ensure we update the target model e.g. this.set('tags', ...)
* override base fn: `permittedAttributes`
- ensure we call the base
- put relations on top
- each relation is allowed to be passed
- the plugin will auto-unset any relations to it does not reach the database
* Ensure we run add/edit/delete within a transaction
- updating nested relationships requires sql queries
- all sql statements have to run in a single transaction to ensure we rollback everything if an error occurs
- use es6
no issue
- useful for managing subscribers via external systems/API calls where it's likely only the e-mail address will be known
- adds `GET /subscribers/email/:email/`
- adds `DELETE /subscribers/email/:email/`
no issue
Had a couple of people ask about how to delete welcome posts easily, so adding a bio to the default user to draw a little more attention to it
no issue
- it can happen that concurrent requests try to renew access tokens with the same refresh token
- in this case it could happen that you received a token deletion error
- add propert locking
- ensure we don't run into deadlocks
- manual testing with async.times for parallel requests (was able to reproduce the error)
refs #5091, #9192
- Renderer figures out templates, contexts, and does a render call
- Templating is now handled with a single function
- Context call is made in the renderer
Note: to make this work, all controllers now define a little bit of config, currently stored in res._route. (That's a totally temporary location, as is res._template... when a sensible naming convention reveals itself I'll get rid of the weird _). This exposes a type and for custom routes a template name & default.
refs #9192, refs #5091
- Moved all url generation into generate-feed.js, so we can see as much data processing as possible in a single place.
- Refactored the way res.locals were used, to be more like how express uses them prior to rendering
- Removed a bunch of code & tests todo with context for RSS - I can't see any way that'd be used, unless we switched the rendering to use a template.
- moved the RSS rendering to be part of the service, not controller
- updated the tests significantly
Note: RSS generate-feed has a complete duplication of the code used in the excerpt helper in order to create an item description
refs #8613, refs #9228
- if you send a request to /authentication/token with `grant_type:password` and a Bearer token, Ghost was not able to handle this combination
- because it skipped the client authentication, see https://github.com/TryGhost/Ghost/blob/1.17.0/core/server/auth/authenticate.js#L13
- and OAuth detects the `grant_type: password` and jumps in the target implementation
- the target implementation for password authentication **again** tried to fetch the client and failed, because it relied on the previous client authentication
- see https://github.com/TryGhost/Ghost/blob/1.17.0/core/server/auth/oauth.js#L40 (client.slug is undefined if client authentication is skipped)
- ^ so this is the bug
- we **can** skip client authentication for requests to the API to fetch data for example e.g. GET /posts (including Bearer)
- so when is a client authentication required?
- RFC (https://tools.ietf.org/html/rfc6749#page-38) differentiates between confidential and public clients, Ghost has no implementation for this at the moment
- so in theory, public clients don't have to be authenticated, only if the credentials are included
- to not invent a breaking change, i decided to only make the client authentication required for password authentication
- we could change this in Ghost 2.0
I have removed the extra client request to the database for the password authentication, this is not needed. We already do client password authentication [here](https://github.com/TryGhost/Ghost/blob/1.17.0/core/server/auth/auth-strategies.js#L19);
If a Bearer token is present and you have not send a `grant_type` (which signalises OAuth to do authentication), you can skip the client authentication.
refs #9192, refs #9178
After trying to progress with current implementation, it became clear that the route service can't control the boot sequence, because then we end up with circular dependencies between the route service and the channel service.
The route service now exposes:
- a siteRouter
- a way for apps to register routes.
- ParentRouter base class for other modules to use
- the registry
...
- moved the default route setup back to site/routes.js 🙈
- moved the parent channel router back to the channel service (this makes way more sense imo)
- this structure prevents circular dependencies
- split the registry out into it's own thing
- fixed-up various bits of tests and comments
- DEBUG will print a list of routes 🎉
refs #9192, #5091
- changed channels to use our new base class
- keep the flexible structure, so that channels can be reloaded
- I had to move the router into the route service otherwise we get circular dependencies
- Don't _really_ want to keep it like this - need a way to define base classes as shared
no issue
- moved isLocalImage fn to storage utils used the RegExp of getLocalFileStoragePath to detect also relative image paths and added tests.
- Added test for independent protocol request (skip, because not supported/implemented)
refs #9192, refs #5091, refs #9178
- moved channels from controllers to a service
- split out the parent router from the remaining individual router logic
- moved the tests to match
refs #5091, refs #9192
- There are several theme template "renderers" all over the codebase
- Some are in apps, and were called "controllers"
- One is in error handling
- All of them now have comments marking out how they share logic/steps
- Other comments describe routes & controllers where they live
refs #9192
- The AMP app is nothing more than a custom controller - this will come clear soon
- Moved enabled/disabled logic into router
- Removed error-related code, as this wasn't used
- Changed logic for static pages to be based on req.body, not context
- Improved the tests to match
refs #5091, #9192, #9178
- Get the RSS module into a much better shape
- Controller -> /controllers/rss
- Remainder -> /services/rss
- Moved tests to match & updated requires
refs #5091, refs #9192
- This is similar to #9218, in that I'm revealing bits of code that are "controllers" in our codebase. As opposed to routes, services, renderers etc.
- This also reveals some code which is identical to the channels controller
- There is more to do here, but for now I've got the module split up, and the tests split and improved.
- Next I'll split RSS into controller + service, DRY up the controller code, etc
closes#9200
- Registered new server helper `{{reading_time}}`.
- Added new global util `word-count` based on the util in Ghost admin, which returns the number of words in an HTML string.
- Based on the word count of the post html, the helper calculated the estimated reading time:
- 275 words per minute
- additional 12 seconds when post has feature image
- Renders a string like 'x min red', unless reading time is less than a minute. In this case, the rendered string is '< 1 min read'.
refs #5091, refs #9192
- render channel was always a weird file
- now it's clearly 2 things
- we're slowly getting towards closing #5091... 🎉
- added some extra tests
refs #9192, refs #5091
- Using a class allows for easy shared logic
- Loading is designed to work from config right now, but could be DB driven, etc
- Provided configuration can be simplified and extended in the constructor / class methods
- Update tests, move custom assertions to utils
refs #9192
To anyone seeing this go by - I'm about to start some fairly major refactoring work on the url utility. Before I do that, I wanted to make sure I had 100% coverage, and understanding of some of the weird cases.
The majority of the changes I've made are adding tests, but I was also able to clean up a little bit, remove a few lines or change them to make use of other tools.
refs #9178
* Add eslint deps, remove old lint deps
* Add eslint config, remove old lint configs
* Config for server and tests are different
* Tweaked rules to suit us
* Fix linting in codebase - lots of indent changes.
* Fix a real broken test
no issue
- we had to fork the original repository at one point, because of slow maintenance
- maintenance is back now
- https://github.com/maxogden/extract-zip/pull/52 was merged and released
refs #9178
* Moved app handling code into services/apps
- Apps is a service, that allows for the App lifecycle
- /server/apps = contains internal apps
- /server/services/apps = contains code for managing/handling app life cycle, providing the proxy, etc
* Split apps service tests into separate files
* Moved internal app tests into test folders
- Problem: Not all the tests in apps were unit tests, yet they were treated like they were in Gruntfile.js
- Unit tests now live in /test/unit/apps
- Route tests now live in /test/functional/routes/apps
- Gruntfile.js has been updated to match
* Switch api.read usage for settingsCache
* Add tests to cover the basic App lifecycle
* Simplify some of the init logic
- This happened by accident when the model filters were introduced
- filters_spec.js was supposed to be renamed to apps_filters_spec.js
- Removing the duplicate file now
refs #8995
- move the getClient lookup from ghost_head into middleware
- use res.locals to keep track of the information (res.locals.client)
- make the middleware global to all frontend routes
- ghost_head: get locals from options.data not this (!)
- adapt lot's of tests
refs #9150
- Moves the password length fn from `models/user` to `data/validation` where the other validator functions live.
- Added password validation rules. Password rules added:
- Disallow obviously bad passwords: '1234567890', 'qwertyuiop', 'asdfghjkl;' and 'asdfghjklm' for example
- Disallow passwords that contain the words 'password' or 'ghost'
- Disallow passwords that match the user's email address
- Disallow passwords that match the blog domain or blog title
- Disallow passwords that include 50% or more of the same characters: 'aaaaaaaaaa', '1111111111' and 'ababababab' for example.
- Password validation returns an `Object` now, that includes an `isValid` and `message` property to differentiate between the two error messages (password too short or password insecure).
- Use a catch predicate in `api/authentication` on `passwordReset`, so the correct `ValidationError` will be thrown during the password reset flow rather then an `UnauthorizedError`.
- When in setup flow, the blog title is not available yet from `settingsCache`. We therefore supply it from the received form data in the user model `setup` method to have it accessible for the validation.
refs #5091
- Move all of the code to do with handling channels into one folder
- Still keeping all the shared/simlar code for rendering etc inside weird
frontend folder until I am sure what this will look like
refs #9178
- Introduce the /services/ folder
- Move xmlrpc there
- Move slack there
- In slack: remove a usage of the settings API that should use settingsCache
- In slack: Simplify the tests
- Various tiny changes to move towards code consistency
refs #5091
- remove the use of functions
- remove unnecessary quotes from tag filter
- move channel config to be a JSOn file called config.channels.json
- accept external config
- new channelUtils for tests
- remove channelConfig.get
- refactor so tests work as expected
- refactor away duplicate 'name' value
closes#9164
- check options.importing on xmlrpc
- also don't ping if private
- cleanup slack to work the same way
- update tests
- TODO: we need to prevent this event happening altogether
refs #5091
- There is very little that changes here, just code readability
- However I've expanded out the tests getting ready to be able to test more deeply as I refactor the routing
