no issue
- With df1ba8aee1 landing in master, there was an error introduced in integration tests, which was caused by URL service being reset at the same time as it was populating its cache.
* Revert "Removed brute force middleware form content api (#10353)"
This reverts commit 63c8c310fb.
* Updated content api spam prevention to use memory store
* Used TooManyRequestsError instead of InternalServer
* Added clause in validation for include to not error
refs #10337
Here we forgo erroring when an invalid property for include is sent, and
instead remove the invalid properties.
* Fixed authors test
* Fixed validators tests
refs https://github.com/TryGhost/Ghost/issues/10124
- This PR introduced additional db calls in URL service due to the need for a model recalculation (we can't rely on the objects that come with events)
refs #10124
- one clean v0.1 and v2 config file for routing!
- solves one underlying bug reported in #10124
- the alias handling was just a hotfix to support v2 for the site
- but it was hard to read, ugly
- now we have two clean configs
- we'll see how useful it is
- need to do proper manual testing on Monday
closes#10062
- return `post.excerpt` for Content API v2
- do not use `downsize`, because we might want to get rid of it if we drop v0.1 (downsize does not create good excerpts)
- simple substring of the plaintext field
no issue
- See explanation: ef98c65040 (r31840536)
- that should not break anything, because resource consumption is based on resource type
- the alias pattern was only invented to make v2 work, it was a little dirty. i wanted to refactor that out anyway
- Use the new getPublic function which does the same thing as this code
- This removes the inclusion of the amp property, but this is undocumented and should not be there anyway!
- This also adds the ghost_head and ghost_foot property, which are public settings
refs #10318
- This settings endpoint returns the commonly used, public information from our settings.
- The values are whitelisted each with a custom name for returning from the endpoint
* Removed unused fields from v2 Content API
- We want to ship the v2 Content API as clean and lean as we can
- Many fields in the DB aren't actually used, we shouldn't return these values
- Other values aren't useful outside of Admin clients, and shouldn't be returned either
Fields removed:
- tags: created_at, updated_at, parent
- authors: locale, accessibility, tour
- posts: locale, author status, page
refs #10124
- Author model returns only users that have published non-page posts
- Added a public controller for tags (should be extracted to separate Content API controller https://github.com/TryGhost/Ghost/issues/10106)
- Made resource configuration dynamic based on current theme engine
- This needs a follow-up PR with fixes to the problems described in the PR
refs #10286
- v2 no longer exposes x_by fields (published_by, updated_by, created_by)
- we will add a brand new concept called activity stream/actions soon
no issue
- we need to maintain the expected properties per API version
- each API version can return a different result
- move the expected properties to the local utility and call global test helper
closes#10301
* Redirected to original image for gifs & svgs
* Created canTransformFileExtension method
* Updated image middlewares to use canTransformFileExtension
no issue
- This change is a follow up to this bugfix https://github.com/TryGhost/Ghost/pull/10299
- Added default export JSON to keep the state of db test suite intact
- Small typo fixe that noticed while debugging
- Removed dependency on the database
- We should not have integration tests for this kind of code
- Note: had to fix rss url tests as sandbox wasn't restored afterwards
no-issue
* Added InternalServerError to resizeImage
* Added a redirect to original image if sharp is missing
* Improved naming - safeMethod -> method
* Updated process method to follow same sharp check pattern
* Refactor safety wrapper into makeSafe function
* Moved generic manipulation error to makeSafe function
* Refactored unsafeProcess to use unsafeResizeImage
* Removed CRAZY catch
refs #10181
* Added initial handleImageSizes middleware
* Implemented saveRaw method on local file storage
* Wired up handleImageSizes middleware
* Implemented delete for LocalFileStorage
* Removed delete method from theme Storage class
* Deleted sizes directory when theme is activated
* Ensured that smaller images are not enlarged
* Renamed sizes -> size
* Exited middleware as early as possible
* Called getStorage as late as possible
* Updated image sizes middleware to handle dimension paths
* Revert "Deleted sizes directory when theme is activated"
This reverts commit 9204dfcc73a6a79d597dbf23651817bcbfc59991.
* Revert "Removed delete method from theme Storage class"
This reverts commit b45fdb405a05faeaf4bd87e977c4ac64ff96b057.
* Revert "Implemented delete for LocalFileStorage"
This reverts commit a587cd6bae45b68a293b2d5cfd9b7705a29e7bfa.
* Fixed typo
Co-Authored-By: allouis <fabien@allou.is>
* Redirected to original image if no image_sizes config
* Refactored redirection because rule of three
* Updated comments
* Added rubbish tests
* Added @TODO comment for handleImageSizes tests
* Added safeResizeImage method to image manipulator
* Used image manipulator lib in image_size middleware
refs #10181
Adds support to request a size in the img_url helper using syntax like:
<img src="{{img_url profile_image size="small"}}"/>
Requires the image_sizes config to be defined in the themes package.json
closes#10266
- the Public API labs flag refers to the v0.1 API only
- if it is disabled, the v0.1 API should be disabled
- if the theme is using v2 API, then the get helper should be available regardless
* Updated auth service members middleware
refs #10213
* Wired up members api router to the ghost api endpoints
refs #10213
* Created members app for the static pages
refs #10213
* Wired up the members app
refs #10213
closes#9791
- we only made use of the redirect middleware, who detects if a redirect should happen, for taxonomies (tags, authors)
- `data: page.team` will now redirect too
- `data: post.team` will now redirect too
- you can disable the redirect using the long form
* 🐛 Protected Ghost blog against invalid uploaded routes.yaml
no issue
- e.g. you upload `filter:tag=this is a wrong filter value`
- ask the url service if it has finished it's work to ensure the upload was successful
- wait 5 seconds till Ghost will bring back the last uploaded valid version
* fixed test
refs #10105
- `options.where` is an older deprecated logic
- before the filter language was invented, Ghost generates statements for knex
- if we want to replace GQL with NQL, we can't generate these statements
- they are not understood from NQL, because NQL uses mongo JSON
- go through usages and rewrite the statements
- invent `extraFilters` for now
- we need to keep the support for `status` or `staticPages` for now (API requirement)
- IMO both shortcuts in the extra filters should be removed in the future
This commit is required for https://github.com/TryGhost/Ghost/pull/10159!
closes#10144
- When the input image is well optimized and has smaller byte size than the processed one it's still being used
- Bumped sharp version to have access to `size` property
* Added updateLastSeen method to user model
refs #10138
* Refactor codebase to use user.updateLastSeen
refs #10138
This is to ensure all updates go via the same method, meaning any
specific logic can be handled in one place, it also helps with grepping
the codebase to find where this occurs
* Created updateUserLastSeen middleware for v2 admin
refs #10138
This is intended to be used with the v2 admin api and _possibly_ the
content api, to give us an accruate report on thelast time a user access
a ghost instance.
* Wired updateUserLastSeen up to v2 Admin API
closes#10138
* Fixed broken test for v2 admin api
no-issue
This test was broken because it was incorrectly testing for a method to
be called exactly once - this was irrelevant to the functionality being
tested for.
* Updated user check method to set status to active
no-issue
* Debounced the updateUserLastSeen middlware an hour
no-issue
* Resolved some PR comments
no issue
Assets moved from gh-pages to https://github.com/tryghost/static and hostname changed, redirects already in place. Can be tested on https://demo.ghost.io (image should all work fine, try visiting one directly to verify redirect works)
closes#10065
- Added UTC offset to dates returned by Content API
- Added test checking new format is compatible with Admin API
- Refactored output serializer mapping logic
no issue
- When importing Ghost 2.0 blogs into 2.0 blogs...
- The Koenig image card would lose it's cardWidth setting,
- because it'd be overridden by the imageStyle setting, which was null
- The importer previous _only_ kept the width if importing 1.0 blogs
refs #7470
These tests had a random failure on travis for Node v6 w/ sqlite. I
figured I would update them to return promises, and remove state, to
make debugging easier, and hopefully fix any issues.
closes#7470
This refactors the spam prevention tests to all return promises,
hopefully making it easier to see where a test timeout might occur, it
also refactors the tests to remove the count state, which made it hard
to follow exactly what was happening or being tested.
no issue
- webhooks UI requires the ability to edit webhooks
- added `edit` permission for `webhook`
- added `edit` method to v2 webhook controller
- added `PUT /webhooks/:id` route to v2 Admin API routes
refs #9942
* Added new middleware to trigger events
* Refactored webhooks service
- added new trigger service, moved listen service to its own file
- started listening to new site.changed event
- cleaned up trigger service to work with new webhook fields
- cleaned up tests
- removed redundant trigger method in v0.1 controller
refs #9866
- invent preview api, but only used internally
- the idea of a preview api is definitiely reaslistic and came up in the past a couple of times
- by that we don't have to differentiate between pages or posts controller
- still support v0.1
- preview controller is not registered for http, only internal handling
refs #9866
- the entry helper is used for static pages and post lookups
- now that we support changing the api version, we have to respect the resource type
- for v2: we ask the pages controller for static pages
- in v0.1: pages and posts lived on the same route
- we are talking about the content API (!) - not admin api
refs #9866
- the static pages router uses the entry controller
- and the entry controller uses the lookup helper
- the lookup helper needs to either fetch static pages or posts
- v2 uses pages and posts controller
refs #9866
- we fallback to v0.1 by default
- we support different formats
- this opens the box to switch the ghost api version for the whole blog site
- i had to add a different notation for overrides.json, because the structure is not optimal (i only want the versions, not the shortcuts)
refs #9865
* Added generic messaging for resource not found
* Ensured integration model uses transaction for writes
* Created POST /integrations endpoint
* Created GET /integrations/:id endpoint
* Created GET /integrations endpoint
* Created PUT /integrations/:id endpoint
* Created DELETE /integrations/:id endpoint
closes#10024
- Updated input serializers for posts/tags/users to handle absolute urls conversion
-------
1. Ghost stores relative images urls
2. API V2 returns images with absolute urls
3. Ghost-Admin sends absolute urls back on any save e.g. update user
**Current behavior**: This will override the relative image path in db to absolute, which in turn won't get updated in future if domain or protocol changes for e.g.
**Fix**: On save/update, input serializers converts any absolute image url paths back to relative if the base URL from image fields matches the configured URL
closes#10029
- allowed page option for users, posts, & tags browse
- The page query param was not forwarding to the query, meaning that when the admin client requested the next page of users or posts, it would receive the first page again.
no-issue
- Added spam prevention to POST /session
- This blocks repeated requests the the /session endpoint preventing brute
force password attacks
- Updated session controller to reset brute middleware
- This updates the session controller to reset the brute force protection
on a successful login. This is required so that a user is not locked out
forever :o!!
refs #9866
- Switched update checker to api v2
- Updated and cleaned up the corresponding test suite
- Updated the frame pipeline to respect context passed in with Frame instance
- Exposed 'active' verison from api index module
* Extended webhooks schema/model and connected with integrations
refs #9942
- Updated webhooks schema with new columns - name, integration_id, secret, last_triggered_at, api_version
- Updated webhooks and integration model to map relationships
- Updated schema hash
- Updated test utils to exclude new webhooks columns for response comparison
* Added migration script for new webhooks columns
refs #9942
- Added migration script in 2.3 to add new columns to webhooks
* Updated schema hash
* Updated maxLength for api_version in schema
* Removed concurrency value from migration script
* Added defaults for webhooks model
* Added status field to webhooks for last trigger status
* Updated schema hash
* Fixed tests with status field
* Removed concurrency value in migration script
* Cleanup
* Updated schema with new fields
- last_triggered_status, last_triggered_error
no-issue
There are a few libraries, including node core that when given an array
for a query parameter will encode it as repeated query params. e.g.
```
{someParam: ['a', 'b']}
// becomes
'?someParam=a&someParam=b'
```
This adds a check for the value to stop us 500ing on repeated keys and
to add easier interop with http clients
refs #9866
- Extracted url decoration logic to utility in output serializers in posts, pages, users, and tags
- Added test cases for url usage by child object (tags of posts)
refs #9866
- Refactored overrides config to include direct version configs(v0.1, v2), supported versions map to direct version
- Refactored `getApiPath` to handle direct versions as well as mappings of supported version
closes#9962
- Fixed the bug with url being set to /404 when id was not present on the model
- Added a functional test to cover this bug
- Refactored url decorating methods to be more clear about the nature of passed parameters
* Added API Key auth middleware to v2 content API
refs #9865
- add `auth.authenticate.authenticateContentApiKey` middleware
- accepts `?key=` query param, sets `req.api_key` if it's a known Content API key
- add `requiresAuthorizedUserOrApiKey` authorization middleware
- passes if either `req.user` or `req.api_key` exists
- update `authenticatePublic` middleware stack for v2 content routes
* Fixed functional content api tests
no-issue
This fixes the functional content api tests so they use the content api
auth.
* Fixed context check and removed skip
* Updated cors middleware for content api
* Removed client_id from frame.context
no-issue
The v2 api doesn't have a notion of clients as we do not use oauth for it
* Fixed tests for posts input serializer
refs #9866
- Added logic ensuring page filter is always set to false in posts endpoint for Content API
- Added functional tests to pages and posts
- Added absolute_url logic in pages controller
- Added slugs controller to v2 API
- Added slugs tests to v2 API
- Updated generic validation error message in shared validator to return validation error with sub-message
* Stopped api key from assigning the 'Owner' role
refs #9865
We do not want api keys to be able to assign the Owner role to any other
key or user.
* Cleaned up Role model permissible method
no-issue
refs #9866
- there was a missing step in the shared validator
- we have to differentiate between data validation for browse/read and data validation for add/edit
- furthermore, the data validation for add/edit was missing and was not copied over from v0.1 (check structure of incoming body)
- adds the ability to require properties from req.body.docName[0]
closes#9972
* Added breaking test for node v6 session auth
* Updated session middleware to support node v6
This uses the legacy url to obtain the origin rather than the WHATWG
URL class in order to support node <6.14.4
closes#9927
- Added post model implementation to be able to store up to 10 versions of mobiledoc
- Bumped GQL to support filtering on the mobiledoc revision table
- Added tests ensuring new functionality works
* Added api_key_id to frame.context
refs #9865
This is to allow controllers to check permissions using api_key_id data.
* Removed client and client_id from frame.context
refs #9865
This is unused as we only support oauth on v0.1 API.
* Fixed indentation for login function
no-issue
* Updated login to work with session auth
refs #9866
This allows the 201 response from session auth and will resolve with
the cookie, or the access_token if the cookie does not exist.
* Added basic doAuth to v2 admin test utils
refs #9866
This will allow functional tests to use session based authentication.
closes#9959
This issue existed because the logic assumed that if there were no
query parameters then there would be no `query` object. However this is
not the case. What we really wanted to check was for the existence of an
"r" query param - the code has been refactor to explicitly do this now.
refs #9866
- just some 🤪
- some were really slow in general, because they had to start/stop Ghost within the test file twice or so
- removed some obvious test cases
- if we copy over the controllers and add functional tests for v2, we can maybe remove some more test cases for v2
refs #9866
- test/functional/
- test/functional/api
- test/functional/api/v0.1
- test/functional/api/v0.1/utils
- test/functional/api/v2
- test/functional/api/v2/admin
- test/functional/api/v2/admin/utils
- test/functional/api/v2/content
- test/functional/api/v2/content/utils
- updated grunt file
- instead of `grunt test-routes`, you now need to use `grunt test-functional` (docs are updated)
You can use `localUtils.API.getApiQuery('posts/')` and it will generate the correct API url.
refs #9866
- moved the tests either to unit tests or routing tests
- or removed test case (a lot)
- this commit is very big 🤪, it was not rly possible to create clean commits for this
- it only changes the test env, no real code is touched
Next steps:
- optimise folder structure + make v2 testing possible
- reduce some more tests from routing and model integeration tests
refs #9866
- prep for v2
- you can better unit test the permissible function
- this avoids copying over the permission handling to v2 controller
- it was possible to move this logic into the model layer, because we now support `unsafeAttrs`
no issue
- last commit failed on mysql
- reverted added a new user to knex test utility
- added the user in the target test
@TODO: we should in the future add resources per test, without using a "global" set,
otherwise you always run into trouble that you have to update a lot of tests when you add a new resource to the test set
refs #9866
- try to keep API interactions for routing tests in a single place, because it gives a better overview
- tiny improvement
- there are lot's of other things we could do, but we want to limit the changes for now
Goal:
e2e/api/v0.1
utils (local)
e2e/api/v2
utils (local)
utils
api (shared)
refs #9866
- the global utility should for now not generate API urls
- we only have one test which generates the API url
- for now it will live in this test file
- if we have multiple cases, we can reconsider this
refs #9866
- the sdk does not use the url utility
- it just respects what you pass in
- there is no need to test this behaviour for multiple API versions
refs #9866
- we want to keep shared API helpers
- the local utility just wraps this call and forwards the correct url
- ability to override the behaviour completely
refs #9866
- we would like to avoid doing for each api version e.g.
testUtils.API.getQueryUrl('webhooks/', v2)
testUtils.API.getQueryUrl('webhooks/', v3, admin)
- the URL should be easily accessible per api version routing folder
- so i thought it makes the most sense to add a local utils file
e.g. localUtils.API.getQueryUrl('webhooks/') => returns target api url for target api folder by default
e.g. localUtils.API.content.getQueryUrl('webhooks/') => returns content api url for target api folder
- it also makes it easy for us to copy the routing tests over without changing anything
- we just need to provide the local utility
no issue
- why? this is a unit test (!)
- we start+stop Ghost in the routing tests a lot, this implicit tested
- it has no priority for now to move this test to a unit test
- this was the only module test, removed related grunt tasks
refs #9326, refs #9866
**ATTENTION: This is the first iteration. Bugs are expected.**
Main Goals:
- add support for multiple API versions.
- do not touch v0.1 implementation
- do not break v0.1
## Problems with the existing v0.1 implementation
1. It tried to be generic and helpful, but it was a mixture of generic and explicit logic living in basically two files: utils.js and index.js.
2. Supporting multiple api versions means, you want to have as less as possible code per API version. With v0.1 it is impossible to reduce the API controller implementation.
----
This commit adds three things:
1. The tiny framework with well-defined API stages.
2. An example implementation of serving static pages via /pages for the content v2 API.
3. Unit tests to prove that the API framework works in general.
## API Stages
- validation
- input serialization
- permissions
- query
- output serialization
Each request should go through these stages. It is possible to disable stages, but it's not recommended.
The code for each stage will either live in a shared folder or in the API version itself. It depends how API specific the validation or serialization is. Depends on the use case.
We should add a specific API validator or serializer if the use case is API format specific.
We should put everything else to shared.
The goal is to add as much as possible into the shared API layer to reduce the logic per API version.
---
Serializers and validators can be added:
- for each request
- for specific controllers
- for specific actions
---
There is room for improvements/extensions:
1. Remove http header configuration from the API controller, because the API controller should not know about http - decouple.
2. Put permissions helpers into shared. I've just extracted and capsulated the permissions helpers into a single file for now. It had no priority. The focus was on the framework itself.
etc.
---
You can find more information about it in the API README.md (api/README.md)
- e.g. find more information about the structure
- e.g. example controllers
The docs are not perfect. We will improve the docs in the next two weeks.
---
Upcoming tasks:
- prepare test env to test multiple API versions
- copy over the controllers from v0.1 to v2
- adapt the v2 express app to use the v2 controllers
refs #9866
- Added test util method for api path based on version
- Updated all hardcoded v0.1 tests strings to use dynamic string from util method
- Updated hardcoded v0.1 tests using regex match to use string equal with new util method
