refs https://github.com/TryGhost/Ghost/issues/10124
- This PR introduced additional db calls in URL service due to the need for a model recalculation (we can't rely on the objects that come with events)
refs #10124
- Author model returns only users that have published non-page posts
- Added a public controller for tags (should be extracted to separate Content API controller https://github.com/TryGhost/Ghost/issues/10106)
- Made resource configuration dynamic based on current theme engine
- This needs a follow-up PR with fixes to the problems described in the PR
* 🐛 Protected Ghost blog against invalid uploaded routes.yaml
no issue
- e.g. you upload `filter:tag=this is a wrong filter value`
- ask the url service if it has finished it's work to ensure the upload was successful
- wait 5 seconds till Ghost will bring back the last uploaded valid version
* fixed test
refs #9866
- Refactored overrides config to include direct version configs(v0.1, v2), supported versions map to direct version
- Refactored `getApiPath` to handle direct versions as well as mappings of supported version
refs #9866
- the global utility should for now not generate API urls
- we only have one test which generates the API url
- for now it will live in this test file
- if we have multiple cases, we can reconsider this
refs #9866
- Added test util method for api path based on version
- Updated all hardcoded v0.1 tests strings to use dynamic string from util method
- Updated hardcoded v0.1 tests using regex match to use string equal with new util method
Moved URL attributes logic from the model into API layer
refs #9866
- Moved URL related attribute calculation for posts, users, and tags into API layer
- Added test coverage for url attributes in tags/authors/primary_tags/primary_authors
refs #9866
- added api version config to overrides, which makes it possible to have a centralized api versioning configuration
- the next PR will use this config in the web folder
- make api url generation in url service flexible and dynamic
- remove hardcoded API_PATH
- updated all places which used `urlFor('api'..)` -> we now ask for explicit api version
closes#9832
The API _should_ be returning absolute URLs for everything, 3rd party applications require absolute urls to read and display ghost data correctly. Currently they have to concat the blog url and the resource url, which is very uncomfortable.
Changing the public api like this would be considered a breaking change however so we've opted to put it behind a query parameter named `absolute_urls`.
refs #9601
- on resource update/add we have cached mobiledoc, html etc
- we have to ensure we exclude the fields (same procdure happens on bootstrap)
- these excluded fields don't have to be cached
- otherwise memory usage is higher in general
- ensure we cache relations with a minimal field set on resource update/add
closes#9727
- this is a short term fix to proof that Disqus comments on preview pages no longer appear on other threads
- this is not a full solution to the problem
- the private API still returns /404/, which is right now inconsistent, but not critical in any way
- the url helper will now output the post preview url if you serve a draft/scheduled post
- this should register unique page urls at Disqus and ensure uniquness for threads
- i still don't understand why the cross posting happens at all, because we also pass an unique identifier to Disqus (the post ID)
- it could be that comments, which are added on the preview page, won't appear on the published urls, because
the published url !== preview url. I wasn't able to figure this out via testing or reading their docs
refs #9601
- replace jsonpath with [NQL](https://github.com/NexesJS/NQL)
- jsonpath was just a temporary solution (a short-term fix)
- with NQL we are able to filter collections more powerful in the near future
- NQL is not feature complete
- we still support `featured:true` for collections
refs #9601
- implementation of resource listener updated
- if you define two collections: `featured:true` (1) and `page:false` (2) you can run into the following bug:
- you create a published post (owned by (2))
- you change the status to featured
- still owned by (2), because the filter still matches (it's still not a static page)
- this adaption fixes the behaviour
- less smart logic, but less error prone
no issue
- this mock eat already too much of my/our time
- the idea of adding a knex mock was definitely a failed approach/try
- it's too much to maintaince and have not found a module which does this already
- we have to support any query format
- this is too crazy
- the idea was to use the knex mock for model unit tests, because if we want to unit test models we have to
run through bookshelf, because the whole model layer depends on bookshelf e.g. events
- for now we simply use the real database
- we could use the sqlite3 memory mode, but that would mean every unit test runs on sqlite3
- something to consider for later e.g. run unit tests on one matrix
- run the rest on another matrix for sqlite + mysql
closes#9675
- with dynamic routing we have introduced a breaking change, which we have overseen
- Ghost does not return absolute urls, that's why the clients need to concat the blog url and the resource url
- with 1.24.0 Ghost returned resource urls including the subdirectory
- this caused trouble for e.g. zapier or the preview feature in the admin client
- revert breaking change and ensure we only expose resource urls without subdirectory
no issue
- there was a timing bug in Ghost
- we do operations in parallel on bootstrap
- 1) we fetch the resources as early as possible
- 2) we do all the rest (express bootstrapping, theme loading, router registration) etc.
- it can happen that (2) happens too slow and ends in the situation that the queue, which is responsible
to handle both parallel actions, does not wait for the routers and closes the event
- this is a short term fix
- i need to reconsider if there is a better long term fix
refs #9601
### Dynamic Routing
This is the beta version of dynamic routing.
- we had a initial implementation of "channels" available in the codebase
- we have removed and moved this implementation
- there is now a centralised place for dynamic routing - server/services/routing
- each routing component is represented by a router type e.g. collections, routes, static pages, taxonomies, rss, preview of posts
- keep as much as possible logic of routing helpers, middlewares and controllers
- ensure test coverage
- connect all the things together
- yaml file + validation
- routing + routers
- url service
- sitemaps
- url access
- deeper implementation of yaml validations
- e.g. hard require slashes
- ensure routing hierarchy/order
- e.g. you enable the subscriber app
- you have a custom static page, which lives under the same slug /subscribe
- static pages are stronger than apps
- e.g. the first collection owns the post it has filtered
- a post cannot live in two collections
- ensure apps are still working and hook into the routers layer (or better said: and register in the routing service)
- put as much as possible comments to the code base for better understanding
- ensure a clean debug log
- ensure we can unmount routes
- e.g. you have a collection permalink of /:slug/ represented by {globals.permalink}
- and you change the permalink in the admin to dated permalink
- the express route get's refreshed from /:slug/ to /:year/:month/:day/:slug/
- unmount without server restart, yey
- ensure we are backwards compatible
- e.g. render home.hbs for collection index if collection route is /
- ensure you can access your configured permalink from the settings table with {globals.permalink}
### Render 503 if url service did not finish
- return 503 if the url service has not finished generating the resource urls
### Rewrite sitemaps
- we have rewritten the sitemaps "service", because the url generator does no longer happen on runtime
- we generate all urls on bootstrap
- the sitemaps service will consume created resource and router urls
- these urls will be shown on the xml pages
- we listen on url events
- we listen on router events
- we no longer have to fetch the resources, which is nice
- the urlservice pre-fetches resources and emits their urls
- the urlservice is the only component who knows which urls are valid
- i made some ES6 adaptions
- we keep the caching logic -> only regenerate xml if there is a change
- updated tests
- checked test coverage (100%)
### Re-work usage of Url utility
- replace all usages of `urlService.utils.urlFor` by `urlService.getByResourceId`
- only for resources e.g. post, author, tag
- this is important, because with dynamic routing we no longer create static urls based on the settings permalink on runtime
- adapt url utility
- adapt tests
closes https://github.com/TryGhost/Ghost/issues/9620
- adjust the `deduplicateSubDir` function's regex to only match duplicate subdirectories when the `url` is only a path rather than full url or the duplicate match starts with a `/`
refs https://github.com/TryGhost/Team/issues/65
- this is just the first optimisation regarding relative/absolute urls
- the full strike will happen when i start with the url utility re-write
- for now: there will be only one subscriber of url events -> the sitemaps service
- the sitemaps service outputs absolute urls
- we don't want to receive an url event and ask the url service again to get an absolute version of the url
refs https://github.com/TryGhost/Team/issues/65
- currently we generate a relative resource url
- if you configure a subdirectory, the urls have to respect that
- e.g. you configure `localhost:2368/blog`, your url results in e.g. `/blog/my-post/`
- this is not yet a critical bug, because the url service is not connected yet
- @TODO: consider absolute vs. relative urls in the url service
refs https://github.com/TryGhost/Team/issues/65
We are currently work on dynamic routing (aka channels).
An important piece of this feature is the url service, which always knows the url of a resource at any time.
Resources can belong to collections or taxonomies, which can be defined in a [routing yaml file](https://github.com/TryGhost/Ghost/issues/9528). We are currently shipping portions, which will at end form the full dynamic routing feature.
### Key Notes
- each routing type (collections, taxonomies, static pages) is registered in order - depending on the yaml routes file configuration
- static pages are an internal concept - they sit at the end of the subscriber queue
- we make use of a temporary [`Channels2`](https://github.com/TryGhost/Ghost/pull/9550/files#diff-9e7251409844521470c9829013cd1563) file, which simulates the current static routing in Ghost (this file will be modified, removed or whatever - this is one of the next steps)
- two way binding: you can ask for a resource url based on the resource id, you can ask for the resource based on the url
- in theory it's possible that multiple resources generate the same url: we don't handle this with collision (because this is error prone), we handle this with the order of serving content. if you ask the service for a resource, which lives behind e.g. /test/, you will get the resource which is served
- loose error handling -> log errors and handle instead of throw error and do nothing (we log the errors with a specific code, so we can react in case there is a bug)
- the url services fetches all resources on bootstrap. we only fetch and keep a reduced set of attributes (basically the main body of a resource)
- the bootstrap time will decrease a very little (depending on the amount of resources you have in your database)
- we still offer the option to disable url preloading (in your config `disableUrlPreload: true`) - this option will be removed as soon as the url service is connected. You can disable the service in case you encounter a problem
- **the url service is not yet connected, we will connect the service step by step. The first version should be released to pre-catch bugs. The next version will add 503 handling if the url service is not ready and it will consume urls for resources.**
----
- the url service generates urls based on resources (posts, pages, users, tags)
- the url service keeps track of resource changes
- the url service keeps track of resource removal/insert
- the architecture:
- each routing type is represented by a url generator
- a routing type is a collection, a taxonomiy or static pages
- a queue which ensures that urls are unique and can be owned by one url generator
- the hierarchy of registration defines that
- we query knex, because bookshelf is too slow
- removed old url service files + logic
- added temp channels alternative (Channels2) -> this file will look different soon, it's for now the temporary connector to the url service. Also the name of the file is not optimal, but that is not really important right now.
refs #9178
- this util uses the url services (!)
- moving this file into lib would not make sense right now
- that would mean a module requires first ../lib/url, which then requires ../services/url
- the url service definitely need a clean up 😃
