refs #10438
- "null" means the resource does not exist (it was sett to "null"), which is not true
- we won't serve primary_tag and primary_author by default
- TODO: add the same change to the Content API v2 (raise issue)
no issue
- Content API v2 served primary_tag by default if members flag is enabled
- reference: b2201d4179
- it's safe to remove, because members is behind the dev flag
refs #10438
- To make response structure future proof and conform to the rest of API responses /images* now returns an object with url property instead of plain url string
no issue
- make use of filter instead of status=all or data.page
- nql was designed to filter data on database layer
- do not break v0.1
- we just got rid of the "status" query param, you should use the filter instead
- get rid of the ugly condition to remove page field if "fields" param was used
- allow filtering on model layer for "findOne"
- do not allow filtering for "findOne" on API layer for now
- the API controller defines what is allowed
- the model layer can allow more by default
- we can re-use the powerful filter logic without adding hacks
refs #10438, refs #10106
* Renamed existing pages ctrl
* Splitted posts & pages for Admin API v2
* Added pages JSON input schema for Admin API v2
* Removed single author for Content & Admin API v2
- single author is not documented
- single author usage is deprecated in v0.1
- single author usage is removed in API v2
* Splitted posts & postsPublic controller for v2
* Removed requirement to send `status=all` from Admin API v2
* Removed `status` option from pages Content API v2
* Removed `status` options from Users Admin API v2
no issue
- this is either documented, not does it work
- the Content API returns authors independing on the status
- filtering by status should not work, because otherwise you could guess the status of a user
- we do not expose the status (!)
refs #10438
- the `updated_at` functions as version control value
- it is required for collision detection
- we might redesign this feature at some point
no-issue
* Refactored hideMembersOnlyContent to 3 "stages"
* Exported paymentConfigured flag from members service
* Updated Content-API to check members service for paymentConfigured
* Updated members content output serializer to remove content if plan required and no plan
* Updated isContentAPI method
* Moved api util test
refs #10438
- we now try to match by slug or id or email
- fallback to owner
- you cannot create a user via post endpoint
- Ghost uses the invite flow to add users
- get rid of `id` restriction on API level
refs #10438
refs #9100
- Added 'strip' attributes to properties that need to be ignored
- Relaxed 'uri' format to 'uri-reference'
- Made input array for posts more restrictive
Added JSON Schema validations for /tags endpoints
refs #10438
refs #9100
- Added JSON Schemas for POST/PUT /tags endpoints
- Added 'strip' keyword definition schema allowing to strip data and not throw errors on further validation stages
refs #10438
- these fields are not used
- no need to expose them in v2
- we will either remove them in the next major or use them for new features (will see)
refs #10472
- Moved config related variable into function scope, so it can be reset by unit tests
- e47d1e275f broke the build and is being fixed by this commit
refs #10438
- Added validation helper based on JSON schema
- Added schema validation for POST/PUT in /posts endpoints
- Refactored existing authors validation test suite
- Extended test coverage with a minimally required structure of post.add validator
refs #10461
- do not break the existing webhooks by keeping both payload formats for subscribers events
- refactored webhooks service to run models through target API version
- added new events described in the target issue reference
- this refactoring & enhancement is undocumented, further breaking changes will happen because we are actively working on: https://github.com/TryGhost/Ghost/issues/10438
no issue
- With the changes in 79ca6c575c we removed old unused events
- The theme upload event is still used and needed to be put back
- Added the event emit right after the successful upload of the theme
- Renamed analytics events for more consistency
- We need to add the same event emitter to the v0.1 API as it's not deprecated
- emits a `theme.uploaded` event after the theme was successfully uploaded and saved
refs #10438
- integrations != users
- Ghost's assumption is: if you create a post, the primary author becomes the logged in user
- we have to require authors for integrations
- short fix and needs some more thoughts later
closes#10174
- Introduced upload middleware that cleans up temporary files stored by mutler after the request is finished
- Removed redundant fs.remove calls as this work is now handled in newly introduced middleware
refs #9865
- Changed id passed for api_key to an object to be able to differenciate between admin and content api requests
- Added integration id to frame context
- Small refactoring of frame context initialization
refs #9865
- Changed key format to {id}:{secret} so API consumer only has to worry about copying a single value during setup
- Updated key expiration time in getValidAdminToken test helper to match server side expiration check
* Updated docs links to best equivalents
- Our documentation has been overhauled, this updates the all the old links sprinkled through Ghost
* Update integrity hash
closes#10373
- ghost_head & ghost_foot are deprecated from now on
- we want to remove them in v3
- this short fix is dirty (!)
- we return codeinjection_* for admin & content api
- this is a consistentency change e.g. posts return `post.codeinjection_*`
- need to raise a decoupling refactoring issue for the code comments
* Added clause in validation for include to not error
refs #10337
Here we forgo erroring when an invalid property for include is sent, and
instead remove the invalid properties.
* Fixed authors test
* Fixed validators tests
closes#10062
- return `post.excerpt` for Content API v2
- do not use `downsize`, because we might want to get rid of it if we drop v0.1 (downsize does not create good excerpts)
- simple substring of the plaintext field
refs #10318
- This settings endpoint returns the commonly used, public information from our settings.
- The values are whitelisted each with a custom name for returning from the endpoint
refs #10318
- cheap and dirty way of removing settings we don't use
- rewritten the settingsFilter function as that was unnecessarily complex
- aslo fixed the require of default-settings in the importer
* Removed unused fields from v2 Content API
- We want to ship the v2 Content API as clean and lean as we can
- Many fields in the DB aren't actually used, we shouldn't return these values
- Other values aren't useful outside of Admin clients, and shouldn't be returned either
Fields removed:
- tags: created_at, updated_at, parent
- authors: locale, accessibility, tour
- posts: locale, author status, page
refs #10124
- Author model returns only users that have published non-page posts
- Added a public controller for tags (should be extracted to separate Content API controller https://github.com/TryGhost/Ghost/issues/10106)
- Made resource configuration dynamic based on current theme engine
- This needs a follow-up PR with fixes to the problems described in the PR
refs #10286
- v2 no longer exposes x_by fields (published_by, updated_by, created_by)
- we will add a brand new concept called activity stream/actions soon
no issue
- This change is a follow up to this bugfix https://github.com/TryGhost/Ghost/pull/10299
- Added default export JSON to keep the state of db test suite intact
- Small typo fixe that noticed while debugging
no issue
- We need to be able to not send the welcome email if needed
- Intruduces a new possible config setting `sendWelcomeEmail` which is set to `true` by default
refs #10286
- we want to deprecate all `x_by` fields
- we would like to get rid of all usages to be able to easily remove the fields in the future
- `invitedBy` is not used in the admin client
* 🐛 Protected Ghost blog against invalid uploaded routes.yaml
no issue
- e.g. you upload `filter:tag=this is a wrong filter value`
- ask the url service if it has finished it's work to ensure the upload was successful
- wait 5 seconds till Ghost will bring back the last uploaded valid version
* fixed test
closes#10118
All behind a members labs switch for now
* Added filter for member only content
* Updated frame context
* Cleaned up members content check
* Cleanup
* Cleanup
* Ensured members filtering works without include=tags
* Protected against missing query
* Fixed usage of include vs withRelated
* Moved includeTags logic for members behind members flag to use tags
* Cleanup
* Update input serializer dependency
Co-Authored-By: rishabhgrg <zrishabhgarg@gmail.com>
* Added some explanations
closes#10065
- Added UTC offset to dates returned by Content API
- Added test checking new format is compatible with Admin API
- Refactored output serializer mapping logic
Closes#10041
1. Why is this change neccesary?
String.prototype.length returns the number of code units in the string (number
of characters) while Buffer.byteLength returns the actual byte length of a
string.
2. How does it address the issue?
Places that use String.prototype.length to calculate Content-Length
were switched to Buffer.byteLength instead.
no issue
- webhooks UI requires the ability to edit webhooks
- added `edit` permission for `webhook`
- added `edit` method to v2 webhook controller
- added `PUT /webhooks/:id` route to v2 Admin API routes
refs #9942
* Added new middleware to trigger events
* Refactored webhooks service
- added new trigger service, moved listen service to its own file
- started listening to new site.changed event
- cleaned up trigger service to work with new webhook fields
- cleaned up tests
- removed redundant trigger method in v0.1 controller
refs #9866
- invent preview api, but only used internally
- the idea of a preview api is definitiely reaslistic and came up in the past a couple of times
- by that we don't have to differentiate between pages or posts controller
- still support v0.1
- preview controller is not registered for http, only internal handling
refs #9865
* Added generic messaging for resource not found
* Ensured integration model uses transaction for writes
* Created POST /integrations endpoint
* Created GET /integrations/:id endpoint
* Created GET /integrations endpoint
* Created PUT /integrations/:id endpoint
* Created DELETE /integrations/:id endpoint
closes#10024
- Updated input serializers for posts/tags/users to handle absolute urls conversion
-------
1. Ghost stores relative images urls
2. API V2 returns images with absolute urls
3. Ghost-Admin sends absolute urls back on any save e.g. update user
**Current behavior**: This will override the relative image path in db to absolute, which in turn won't get updated in future if domain or protocol changes for e.g.
**Fix**: On save/update, input serializers converts any absolute image url paths back to relative if the base URL from image fields matches the configured URL
closes#10029
- allowed page option for users, posts, & tags browse
- The page query param was not forwarding to the query, meaning that when the admin client requested the next page of users or posts, it would receive the first page again.
no-issue
- Added spam prevention to POST /session
- This blocks repeated requests the the /session endpoint preventing brute
force password attacks
- Updated session controller to reset brute middleware
- This updates the session controller to reset the brute force protection
on a successful login. This is required so that a user is not locked out
forever :o!!
refs #9866
- Switched update checker to api v2
- Updated and cleaned up the corresponding test suite
- Updated the frame pipeline to respect context passed in with Frame instance
- Exposed 'active' verison from api index module
no-issue
There are a few libraries, including node core that when given an array
for a query parameter will encode it as repeated query params. e.g.
```
{someParam: ['a', 'b']}
// becomes
'?someParam=a&someParam=b'
```
This adds a check for the value to stop us 500ing on repeated keys and
to add easier interop with http clients
refs #9866
- Extracted url decoration logic to utility in output serializers in posts, pages, users, and tags
- Added test cases for url usage by child object (tags of posts)
refs #9866
- Refactored overrides config to include direct version configs(v0.1, v2), supported versions map to direct version
- Refactored `getApiPath` to handle direct versions as well as mappings of supported version
closes#9962
- Fixed the bug with url being set to /404 when id was not present on the model
- Added a functional test to cover this bug
- Refactored url decorating methods to be more clear about the nature of passed parameters
* Added API Key auth middleware to v2 content API
refs #9865
- add `auth.authenticate.authenticateContentApiKey` middleware
- accepts `?key=` query param, sets `req.api_key` if it's a known Content API key
- add `requiresAuthorizedUserOrApiKey` authorization middleware
- passes if either `req.user` or `req.api_key` exists
- update `authenticatePublic` middleware stack for v2 content routes
* Fixed functional content api tests
no-issue
This fixes the functional content api tests so they use the content api
auth.
* Fixed context check and removed skip
* Updated cors middleware for content api
* Removed client_id from frame.context
no-issue
The v2 api doesn't have a notion of clients as we do not use oauth for it
* Fixed tests for posts input serializer
refs #9866
- Added logic ensuring page filter is always set to false in posts endpoint for Content API
- Added functional tests to pages and posts
- Added absolute_url logic in pages controller
refs #9866
- by default it used `options.id`, which tells the permission layer the target id
- but some controllers want to use a different identifier
- e.g. settings -> settings.key
- e.g. password changes -> password[0].user_id
- Added slugs controller to v2 API
- Added slugs tests to v2 API
- Updated generic validation error message in shared validator to return validation error with sub-message
refs #9866
- there was a missing step in the shared validator
- we have to differentiate between data validation for browse/read and data validation for add/edit
- furthermore, the data validation for add/edit was missing and was not copied over from v0.1 (check structure of incoming body)
- adds the ability to require properties from req.body.docName[0]
* Added api_key_id to frame.context
refs #9865
This is to allow controllers to check permissions using api_key_id data.
* Removed client and client_id from frame.context
refs #9865
This is unused as we only support oauth on v0.1 API.
refs #9866
- prep for v2
- you can better unit test the permissible function
- this avoids copying over the permission handling to v2 controller
- it was possible to move this logic into the model layer, because we now support `unsafeAttrs`
no issue
- now that we have a concept of `unsafeAttrs`, we can move the custom permissions to the invite model
Why doing now?
A) We won't copy this controller code to v2.
B) Makes it easier to unit test this behaviour
refs #9326, refs #9866
**ATTENTION: This is the first iteration. Bugs are expected.**
Main Goals:
- add support for multiple API versions.
- do not touch v0.1 implementation
- do not break v0.1
## Problems with the existing v0.1 implementation
1. It tried to be generic and helpful, but it was a mixture of generic and explicit logic living in basically two files: utils.js and index.js.
2. Supporting multiple api versions means, you want to have as less as possible code per API version. With v0.1 it is impossible to reduce the API controller implementation.
----
This commit adds three things:
1. The tiny framework with well-defined API stages.
2. An example implementation of serving static pages via /pages for the content v2 API.
3. Unit tests to prove that the API framework works in general.
## API Stages
- validation
- input serialization
- permissions
- query
- output serialization
Each request should go through these stages. It is possible to disable stages, but it's not recommended.
The code for each stage will either live in a shared folder or in the API version itself. It depends how API specific the validation or serialization is. Depends on the use case.
We should add a specific API validator or serializer if the use case is API format specific.
We should put everything else to shared.
The goal is to add as much as possible into the shared API layer to reduce the logic per API version.
---
Serializers and validators can be added:
- for each request
- for specific controllers
- for specific actions
---
There is room for improvements/extensions:
1. Remove http header configuration from the API controller, because the API controller should not know about http - decouple.
2. Put permissions helpers into shared. I've just extracted and capsulated the permissions helpers into a single file for now. It had no priority. The focus was on the framework itself.
etc.
---
You can find more information about it in the API README.md (api/README.md)
- e.g. find more information about the structure
- e.g. example controllers
The docs are not perfect. We will improve the docs in the next two weeks.
---
Upcoming tasks:
- prepare test env to test multiple API versions
- copy over the controllers from v0.1 to v2
- adapt the v2 express app to use the v2 controllers
refs #9865
Note that this controller is the singular, that's because we plan to
make a session resource controller to be used with /sessions, wheras
this is on /session
Moved URL attributes logic from the model into API layer
refs #9866
- Moved URL related attribute calculation for posts, users, and tags into API layer
- Added test coverage for url attributes in tags/authors/primary_tags/primary_authors
refs #9866
- preparation for v2
- moved api/ to api/v0.1
- do export v0.1 straight from the api folder, we don't want to touch this right now
- that means currently if you require the api folder, we return v0.1 by default
- there were some direct requires of api files in the test env
- some of them use rewire
- for now, we just correct the require path to require api/v0.1/
- we touch the test env next week
**Docs about V2 design are coming soon!**
refs #9866
- Removed `toJSON` call in `findPage`
- Added JSON serialization on API layer
- Reason: model and api layer were coupled - all other model actions just returned the raw data and no specific format
- Corrected test suites to serialize fetched models to JSON
- Removed `absolute_urls` attribute from validOptions findPage methods as it's no longer needed in the data layer
- Changed 'include' test as this option is now tolerated and returns data
refs #9865
This is to ensure that if a controller returns a function, it will
always get called regardless of method.
Also cleaned up top level const usage
refs #9866
- added api version config to overrides, which makes it possible to have a centralized api versioning configuration
- the next PR will use this config in the web folder
- make api url generation in url service flexible and dynamic
- remove hardcoded API_PATH
- updated all places which used `urlFor('api'..)` -> we now ask for explicit api version
refs #9866
- if we start with v2 controllers, the code base should not require specific api controllers
- because e.g. `require('../api/posts')` will no longer exist
- if you require the api folder, you will get the latest available version by default e.g. `require('../api').posts`
- this branch does not touch the test env (!)
refs #9866
- if I want to do a project search and looks for model usages e.g. `models.`, then I won't find these usages
- normalise how we require models -> consistency
closes#9832
The API _should_ be returning absolute URLs for everything, 3rd party applications require absolute urls to read and display ghost data correctly. Currently they have to concat the blog url and the resource url, which is very uncomfortable.
Changing the public api like this would be considered a breaking change however so we've opted to put it behind a query parameter named `absolute_urls`.
refs #4453
* On by default
* Added config to disable resizing
* Added basic image optimization processing
* Added dep: sharp (optional dep)
* Added resize middleware
* Take care of rotation based on EXIF information
* Removed all meta data from optimised image
* Added handling if sharp could not get installed
* Do not read ext twice - optimisation
* Do not call sharp if config is disabled
* Do not remove the original image which was uploaded (store 2 images)
* Support of `req.files` for internal logic
* Disabled cache to enable file removal on Windows
closes#9786
- Make GET request when url has no provider match
- The HEAD request was made in order to send less data over the wire when
checking for redirects for urls that do not have an oembed provider
match. We are now going to look for provider metatags withing the
response of the request - rather than making a HEAD followed by a GET if
no redirect is found, this condenses that to a single request.
- Try to get OEmbed data from tag if no provider
- Here we parse the HTML response of the resource and look for a link tag
that will give us the oembed resource url which we can use to fetch the
embed html
refs #9742
- removed usage of single permalink setting
- with dynamic routing this configuration does no longer makes sense
- because you can configure your permalinks in the routes.yaml
- furthermore you can have multiple collections with multiple permalinks
- removed @blog.permalinks
- do not export permalink setting
- do not import permalink setting
- permalink setting UI will be removed soon
- get rid of {globals.permalink} completely
- remove yaml in-built migration
- do not expose settings.permalinks via the private API
- do not allow to edit this setting
- keep phyiscal value in case a blog needs to rollback from v2 to v1
- sorted out when the routers should be created
- ensure routes.yaml file doesn't get validated before Ghost is fully ready to start
refs #9744
- added two new endpoints to upload/download routes.yaml
- reload site express app on successful/valid upload
- reload url service on sucessfuly upload
- force clear cache of pages
- ensure we keep a backup of the routes.yaml file
- this feature was mostly tested manually
- @TODO: i have to write unit tests - will do later
- @TODO: do a memory test to ensure we haven't introduced any memory leaks with this feature
refs https://github.com/TryGhost/Ghost/issues/9724
- perform a HEAD request on a url if we don't find a matching provider, following any redirects until we hit success response before looking up providers for the resulting url
refs #8719
- initial commit: 40c8eacd44
- we have forgotten that there is another endpoint which triggers an export (the backup endpoint)
- this endpoint needs to accept the new `include` query param as well (was missing)
refs #9742, refs #8719
- make it possible to import more tables (optional)
- available tables: clients, trusted domains
- by default we won't import these tables, you have to tell Ghost using `include` (same syntax on export)
- we won't announce this ability for now (stays hidden)
refs #9742, refs #8719
- you can now use `include` to export extra tables e.g. `include=clients`
- admin client won't make use of this option yet, maybe later and optional
- we won't announce this new ability for now (stays hidden)
refs https://github.com/TryGhost/Ghost/issues/9623
- add `oembed-parser` module for checking provider availability for a url and fetching data from the provider
- require it in the `overrides.js` file before the general Promise override so that the `promise-wrt` sub-dependency doesn't attempt to extend the Bluebird promise implementation
- add `/oembed` authenticated endpoint
- takes `?url=` query parameter to match against known providers
- adds safeguard against oembed-parser's providers list not recognising http+https and www+non-www
- responds with `ValidationError` if no provider is found
- responds with oembed response from matched provider's oembed endpoint if match is found
refs #9601
### Dynamic Routing
This is the beta version of dynamic routing.
- we had a initial implementation of "channels" available in the codebase
- we have removed and moved this implementation
- there is now a centralised place for dynamic routing - server/services/routing
- each routing component is represented by a router type e.g. collections, routes, static pages, taxonomies, rss, preview of posts
- keep as much as possible logic of routing helpers, middlewares and controllers
- ensure test coverage
- connect all the things together
- yaml file + validation
- routing + routers
- url service
- sitemaps
- url access
- deeper implementation of yaml validations
- e.g. hard require slashes
- ensure routing hierarchy/order
- e.g. you enable the subscriber app
- you have a custom static page, which lives under the same slug /subscribe
- static pages are stronger than apps
- e.g. the first collection owns the post it has filtered
- a post cannot live in two collections
- ensure apps are still working and hook into the routers layer (or better said: and register in the routing service)
- put as much as possible comments to the code base for better understanding
- ensure a clean debug log
- ensure we can unmount routes
- e.g. you have a collection permalink of /:slug/ represented by {globals.permalink}
- and you change the permalink in the admin to dated permalink
- the express route get's refreshed from /:slug/ to /:year/:month/:day/:slug/
- unmount without server restart, yey
- ensure we are backwards compatible
- e.g. render home.hbs for collection index if collection route is /
- ensure you can access your configured permalink from the settings table with {globals.permalink}
### Render 503 if url service did not finish
- return 503 if the url service has not finished generating the resource urls
### Rewrite sitemaps
- we have rewritten the sitemaps "service", because the url generator does no longer happen on runtime
- we generate all urls on bootstrap
- the sitemaps service will consume created resource and router urls
- these urls will be shown on the xml pages
- we listen on url events
- we listen on router events
- we no longer have to fetch the resources, which is nice
- the urlservice pre-fetches resources and emits their urls
- the urlservice is the only component who knows which urls are valid
- i made some ES6 adaptions
- we keep the caching logic -> only regenerate xml if there is a change
- updated tests
- checked test coverage (100%)
### Re-work usage of Url utility
- replace all usages of `urlService.utils.urlFor` by `urlService.getByResourceId`
- only for resources e.g. post, author, tag
- this is important, because with dynamic routing we no longer create static urls based on the settings permalink on runtime
- adapt url utility
- adapt tests
no issue
- discovered while coding
- the value was always false, because we've tried to read the value from the config object
- the value lives in the database and is accessible via the labs service
no issue
- removed the `routeKeywords` property from the config and used hard coded keywords.
- removed `routeKeywords` from public configuration API endpoint, as it's no longer used in the Admin.
no issue
- if you delete all content, we expect two events
- `post.deleted` and `post.unpublished`
- `post.unpublished` was never triggered, because the api implementation made use of `collection.invoke(`destroy`)`
- what happened?
- you fetch all posts (columns:id)
- you destroy the post (only id column is available)
- the model events are triggered
- but you have no access to a default set of data
- the result is that the event handler can't even tell if this is a post or a page
- added a proper test to ensure which events are triggered
