no issue
- With the changes in 79ca6c575c we removed old unused events
- The theme upload event is still used and needed to be put back
- Added the event emit right after the successful upload of the theme
- Renamed analytics events for more consistency
- We need to add the same event emitter to the v0.1 API as it's not deprecated
- emits a `theme.uploaded` event after the theme was successfully uploaded and saved
closes#10447
- Get helper message talks about the old API, but upgrading is the best way to solve the problem
- Had to create a way to add a custom message to a labs enabled helper to achieve this
closes#10448
- using @site.lang to read posts is a valid use case for the get helper filters
- get helper filters have special treatment of anything wrapped in {{}}, in the form of resolvePaths
- resolvePaths uses some custom logic + jsonpath to find the right bit of data to inject
- this function had no handling for globals starting with `@`, and also didn't have access to them
refs #9248
- Bookshelf gives access to ".changed" before the update
- Discussion: https://github.com/bookshelf/bookshelf/issues/1943
- We also need to know what has changed after the update to be able to decide if we should trigger events
- Furthermore: Bookshelf cannot handle relation updates, it always marks relations as changed even though they did not change
- Bumped bookshelf-relations to be able to
- know if relations were updated
- ensure we unset relations on bookshelf's ".changed"
refs #10438
- integrations != users
- Ghost's assumption is: if you create a post, the primary author becomes the logged in user
- we have to require authors for integrations
- short fix and needs some more thoughts later
refs #10438
- make /images/ available
- we want to document this endpoint notiation, because it is more specific and fits better, because you can only upload images
- either we drop /uploads/ by the end of the project cycle or we keep both for now
- the Admin API v2 is currently undocumented and allows breaking changes in theory
refs #10174
- Improved importer cleanUp method usage, so the cleanup is called in cases when there is an error during an import stage
- Simplified files to clean up tracking as removal of files is now partially handled in uploader middleware
closes#10174
- Introduced upload middleware that cleans up temporary files stored by mutler after the request is finished
- Removed redundant fs.remove calls as this work is now handled in newly introduced middleware
no-issue
- revert the migration because migrations that (potentially) touch every row should be kept to major releases where possible
- the migration was safe to run and won't cause any problems for anyone who has already upgraded
- reversion keeps the migration file but changes the contents to a no-op so that `migrations` table state is the same for all users whether they migrated with the full migration or the no-op version
closes#10388
This migration finds all tables with nullable columns, it then loops through the tables and their nullable columns, updating each column to a null when its current value is an empty string.
no-issue
Currently the `user-agent` header is the for outgoing webhook calls is the `got` default: `User-Agent: got/8.3.2 (https://github.com/sindresorhus/got)`.
This is pretty unfriendly to the receiver of the webhook who may wish to perform analytics on calling systems, implement security features based on calling system or take action based on different versions of a client.
This PR sets the header to: `User-Agent: Ghost/2.12.0 (https://github.com/TryGhost/Ghost)` which is much more descriptive.
refs #10388
This updates the base model to retrieve column information, and explicitly set every property whose column is `nullable` and content is the empty string (`""`) to `null`
refs #9865
- Changed id passed for api_key to an object to be able to differenciate between admin and content api requests
- Added integration id to frame context
- Small refactoring of frame context initialization
refs #9865
- Changed key format to {id}:{secret} so API consumer only has to worry about copying a single value during setup
- Updated key expiration time in getValidAdminToken test helper to match server side expiration check
refs #9389
- eslint@5.12.1, eslint-plugin-ghost@0.1.0, grunt-contrib-clean@2.0.0, grunt-contrib-uglify@4.0.0, grunt-eslint@21.0.0, grunt-mocha-cli@4.0.0, grunt-shell@3.0.1, mocha@5.2
.0, nock@10.0.6, rewire@4.0.1
All of them dropped Node v4. I was not able to find any other big breaking changes, which affect us right now.
refs #10286
- this is just a hotfix for v0.1
- we keep the x_by fields for now and deprecate them
- as soon as an integration updates/inserts a resource, we just store the owner ID
- we currently work on a new concept for v2
- v2 no longer exposes or uses x_by, see https://github.com/TryGhost/Ghost/pull/10294
- we need to iterate on this change, because we currently use the naming `context.api_key_id` in the auth/API layer
closes#10391
- We use "relative protocol" urls for gravatar images, which were
incorrectly getting treated as relative path urls.
- Refactored getBlogUrl calls into const
refs #9865
- small refactoring to make both session and admin api key handling similar
- admin api key authentication is still disabled, but easy to enable
- added proof test how to authenticate using admin api keys
refs #9865
- the outer authentication layer wants a consistent interface of each authentication package
- admin.authenticate
- session.authenticate
- furthermore, there is no need to put the full feature into the exposed function name
refs #9865
- Added `auth.authenticate.authenticateAdminApiKey` middleware
- accepts signed JWT in an `Authorization: Ghost [token]` header
- sets `req.api_key` if the token is valid
- Updated `authenticatePrivate` middleware stack for v2 admin routes
refs #9865
This updates all current permissible methods to use the new function
signature which includes the hasApiKeyPermissions parameter. It also
makes sure that the hasApiKeyPermissions argument is taken into account
whenever checking before returning a resolved promise.
To be continued. This is just a tiny part of the big picture. None of these changes are fully committed to stay as they are.
refs #9865
- Enabled the permissions module to lookup permissions based on an api_key id.
- Updated the "can this" part of the permissions service to load permissions for any api key in the context, and correctly use that to determine whether an action is permissible. It also updates the permissible interface that models can implement to pass in the hasApiKeyPermissions param.
* Updated docs links to best equivalents
- Our documentation has been overhauled, this updates the all the old links sprinkled through Ghost
* Update integrity hash
no issue
- Fixes a case where a post that doesn't belong to a collection throws a 500 Resource Not Found Error
- This should be a 404 Resource Not Found error
- 500 suggests something went very wrong and is our fault,
- but this is a user error where the collections or posts are misconfigured, and some content doesn't have a home
closes#10383
- Upgrades got to 8.3.2, which contains better error handling and resolves the issue with uncaught exceptions
- Note: Got 9.x stream doesn't support Node v6
- Requires us to hardcode http:// for xmlrpc because there is a breaking change where got now defaults to https instead of http
closes#10373
- ghost_head & ghost_foot are deprecated from now on
- we want to remove them in v3
- this short fix is dirty (!)
- we return codeinjection_* for admin & content api
- this is a consistentency change e.g. posts return `post.codeinjection_*`
- need to raise a decoupling refactoring issue for the code comments
* Revert "Removed brute force middleware form content api (#10353)"
This reverts commit 63c8c310fb.
* Updated content api spam prevention to use memory store
* Used TooManyRequestsError instead of InternalServer
* Added clause in validation for include to not error
refs #10337
Here we forgo erroring when an invalid property for include is sent, and
instead remove the invalid properties.
* Fixed authors test
* Fixed validators tests
closes#10283
Updated middleware for dynamic image sizes to attempt to read the unoptimized image first, taking into account the `-n` suffix for duplicate image names, by using a regex.
refs https://github.com/TryGhost/Ghost/issues/10124
- This PR introduced additional db calls in URL service due to the need for a model recalculation (we can't rely on the objects that come with events)
* Added spam config for content api key
no-issue
* Created contentApiKey spam prevention method
* Added contentApiKey brute middleware
no-issue
This middleware attaches a listener for when the request has completed,
if the request ends with a successful response code, we reset any spam
prevention data for that ip.
* Added contentApiKey brute middleware to the content api
* Multipled maxWait by 24, to 24 hours
refs #10124
- one clean v0.1 and v2 config file for routing!
- solves one underlying bug reported in #10124
- the alias handling was just a hotfix to support v2 for the site
- but it was hard to read, ugly
- now we have two clean configs
- we'll see how useful it is
- need to do proper manual testing on Monday
closes#10062
- return `post.excerpt` for Content API v2
- do not use `downsize`, because we might want to get rid of it if we drop v0.1 (downsize does not create good excerpts)
- simple substring of the plaintext field
no issue
- See explanation: ef98c65040 (r31840536)
- that should not break anything, because resource consumption is based on resource type
- the alias pattern was only invented to make v2 work, it was a little dirty. i wanted to refactor that out anyway
- Use the new getPublic function which does the same thing as this code
- This removes the inclusion of the amp property, but this is undocumented and should not be there anyway!
- This also adds the ghost_head and ghost_foot property, which are public settings
refs #10318
- This settings endpoint returns the commonly used, public information from our settings.
- The values are whitelisted each with a custom name for returning from the endpoint
refs #10318
- cheap and dirty way of removing settings we don't use
- rewritten the settingsFilter function as that was unnecessarily complex
- aslo fixed the require of default-settings in the importer
* Removed unused fields from v2 Content API
- We want to ship the v2 Content API as clean and lean as we can
- Many fields in the DB aren't actually used, we shouldn't return these values
- Other values aren't useful outside of Admin clients, and shouldn't be returned either
Fields removed:
- tags: created_at, updated_at, parent
- authors: locale, accessibility, tour
- posts: locale, author status, page
refs #10124
- Author model returns only users that have published non-page posts
- Added a public controller for tags (should be extracted to separate Content API controller https://github.com/TryGhost/Ghost/issues/10106)
- Made resource configuration dynamic based on current theme engine
- This needs a follow-up PR with fixes to the problems described in the PR
refs #10286
- v2 no longer exposes x_by fields (published_by, updated_by, created_by)
- we will add a brand new concept called activity stream/actions soon
closes#10301
* Redirected to original image for gifs & svgs
* Created canTransformFileExtension method
* Updated image middlewares to use canTransformFileExtension
no issue
- This change is a follow up to this bugfix https://github.com/TryGhost/Ghost/pull/10299
- Added default export JSON to keep the state of db test suite intact
- Small typo fixe that noticed while debugging
no issue
- We need to be able to not send the welcome email if needed
- Intruduces a new possible config setting `sendWelcomeEmail` which is set to `true` by default
refs #10286
- we want to deprecate all `x_by` fields
- we would like to get rid of all usages to be able to easily remove the fields in the future
- `invitedBy` is not used in the admin client
* Update mobile modal animations
* Member popup input error and placeholder refinements
* Adding close animation to members auth popups
* Improve members auth dialog
* Refine members reset password design
no-issue
the ssoOriginCheck exists to ensure that we only allow signin/signup to
be called from the specified auth page, this is a very minor security
feature in that it forces signins to go via the page you've designated.
signout however does not need this protection as the call to signout
completely bypasses any UI (this is the same for the call to /token)
no-issue
* Added InternalServerError to resizeImage
* Added a redirect to original image if sharp is missing
* Improved naming - safeMethod -> method
* Updated process method to follow same sharp check pattern
* Refactor safety wrapper into makeSafe function
* Moved generic manipulation error to makeSafe function
* Refactored unsafeProcess to use unsafeResizeImage
* Removed CRAZY catch
refs #10181
* Added initial handleImageSizes middleware
* Implemented saveRaw method on local file storage
* Wired up handleImageSizes middleware
* Implemented delete for LocalFileStorage
* Removed delete method from theme Storage class
* Deleted sizes directory when theme is activated
* Ensured that smaller images are not enlarged
* Renamed sizes -> size
* Exited middleware as early as possible
* Called getStorage as late as possible
* Updated image sizes middleware to handle dimension paths
* Revert "Deleted sizes directory when theme is activated"
This reverts commit 9204dfcc73a6a79d597dbf23651817bcbfc59991.
* Revert "Removed delete method from theme Storage class"
This reverts commit b45fdb405a05faeaf4bd87e977c4ac64ff96b057.
* Revert "Implemented delete for LocalFileStorage"
This reverts commit a587cd6bae45b68a293b2d5cfd9b7705a29e7bfa.
* Fixed typo
Co-Authored-By: allouis <fabien@allou.is>
* Redirected to original image if no image_sizes config
* Refactored redirection because rule of three
* Updated comments
* Added rubbish tests
* Added @TODO comment for handleImageSizes tests
* Added safeResizeImage method to image manipulator
* Used image manipulator lib in image_size middleware
refs #10181
Adds support to request a size in the img_url helper using syntax like:
<img src="{{img_url profile_image size="small"}}"/>
Requires the image_sizes config to be defined in the themes package.json
closes#10266
- the Public API labs flag refers to the v0.1 API only
- if it is disabled, the v0.1 API should be disabled
- if the theme is using v2 API, then the get helper should be available regardless
* Updated auth service members middleware
refs #10213
* Wired up members api router to the ghost api endpoints
refs #10213
* Created members app for the static pages
refs #10213
* Wired up the members app
refs #10213
* Added members library inc. gateway
refs #10213
* Added the auth pages and build steps for them
refs #10213
* Cleaned up logs
* Updated gruntfile to run yarn for member auth
* Design refinements on members popups
* UI refinements
* Updated backend call to trigger only if frontend validation passes
* Design refinements for error messages
* Added error message for email failure
* Updated request-password-reset to not attempt to send headers twice
* Updated preact publicPath to relative path
* Build auth pages on init
no-issue
When trying to use /api/v2/content from a different domain, the requests
were failing with CORS errors. This doesn't use the shared cors middleware,
because it should be open to all hosts, and not locked down via our
whitelist or trusted domains.
closes#10226
- Middleware emits site-changed event used to trigger webhook, was configured to v2 admin api only.
- Change allows all versions of api to emit event in case of cache invalidation
closes#9791
- we only made use of the redirect middleware, who detects if a redirect should happen, for taxonomies (tags, authors)
- `data: page.team` will now redirect too
- `data: post.team` will now redirect too
- you can disable the redirect using the long form
refs #10082
- this is a requirement if a static route represents a single resource
e.g. `data: page.team`
- the page resource will no longer live on it's original static url
- instead, it now lives somewhere else
- that means the whole site needs to act the same than the original static url
- the resource does not contain any relations
- we don't forward the correct context (page, post, user?)
- we override the `include` property for now
- need to wait for more use cases or bug reports for this controller
- more changes will follow asap
refs #9584
- The Importer checks if a user reference is null. But if the post is a draft and published_by is null, we should ignore the user reference detection.
- This change will avoid showing an incorrect user reference warning in the importer report for draft posts.
* 🐛 Protected Ghost blog against invalid uploaded routes.yaml
no issue
- e.g. you upload `filter:tag=this is a wrong filter value`
- ask the url service if it has finished it's work to ensure the upload was successful
- wait 5 seconds till Ghost will bring back the last uploaded valid version
* fixed test
refs #10105
- `options.where` is an older deprecated logic
- before the filter language was invented, Ghost generates statements for knex
- if we want to replace GQL with NQL, we can't generate these statements
- they are not understood from NQL, because NQL uses mongo JSON
- go through usages and rewrite the statements
- invent `extraFilters` for now
- we need to keep the support for `status` or `staticPages` for now (API requirement)
- IMO both shortcuts in the extra filters should be removed in the future
This commit is required for https://github.com/TryGhost/Ghost/pull/10159!
closes#10118
All behind a members labs switch for now
* Added filter for member only content
* Updated frame context
* Cleaned up members content check
* Cleanup
* Cleanup
* Ensured members filtering works without include=tags
* Protected against missing query
* Fixed usage of include vs withRelated
* Moved includeTags logic for members behind members flag to use tags
* Cleanup
* Update input serializer dependency
Co-Authored-By: rishabhgrg <zrishabhgarg@gmail.com>
* Added some explanations
closes#10144
- When the input image is well optimized and has smaller byte size than the processed one it's still being used
- Bumped sharp version to have access to `size` property
* Added updateLastSeen method to user model
refs #10138
* Refactor codebase to use user.updateLastSeen
refs #10138
This is to ensure all updates go via the same method, meaning any
specific logic can be handled in one place, it also helps with grepping
the codebase to find where this occurs
* Created updateUserLastSeen middleware for v2 admin
refs #10138
This is intended to be used with the v2 admin api and _possibly_ the
content api, to give us an accruate report on thelast time a user access
a ghost instance.
* Wired updateUserLastSeen up to v2 Admin API
closes#10138
* Fixed broken test for v2 admin api
no-issue
This test was broken because it was incorrectly testing for a method to
be called exactly once - this was irrelevant to the functionality being
tested for.
* Updated user check method to set status to active
no-issue
* Debounced the updateUserLastSeen middlware an hour
no-issue
* Resolved some PR comments
closes#10114
* Members lab enabled to be always true behind developer experiments flag
* Members lab set to true for themes behind developer experiments flag
Note: This change uses hard-coded labs value for members based on enableDeveloperExperiments flag, ideal implementation for later is to pick those value from settings.
no-issue
This is because the Content API will eventually be accessed not just
from Content API keys. The addition of a Content API specific
authorization middleware is because:
1. content api should not authorize based on req.user
2. content api will need separate authorization than admin api
no issue
Assets moved from gh-pages to https://github.com/tryghost/static and hostname changed, redirects already in place. Can be tested on https://demo.ghost.io (image should all work fine, try visiting one directly to verify redirect works)
closes#10065
- Added UTC offset to dates returned by Content API
- Added test checking new format is compatible with Admin API
- Refactored output serializer mapping logic
* Added Node v10 Support
no issue
Signed-off-by: kirrg001 <katharina.irrgang@googlemail.com>
* Bump amperize to version 0.3.8
no issue
* Bump mysql to version 2.16.0
no issue
- mysql 2.15.0 uses a deprecated notation for timers
- e.g. timers.unenroll()
* Bump sub dependencies
no issue
- e.g. knex-migrator used mysql 2.15.0
* Bump dependencies
no issue
* Replaced `new Buffer` with `Buffer.from`
no issue
- Buffer() is deprecated due to security and usability issues.
- https://nodejs.org/en/docs/guides/buffer-constructor-deprecation/
no issue
- When importing Ghost 2.0 blogs into 2.0 blogs...
- The Koenig image card would lose it's cardWidth setting,
- because it'd be overridden by the imageStyle setting, which was null
- The importer previous _only_ kept the width if importing 1.0 blogs
Closes#10041
1. Why is this change neccesary?
String.prototype.length returns the number of code units in the string (number
of characters) while Buffer.byteLength returns the actual byte length of a
string.
2. How does it address the issue?
Places that use String.prototype.length to calculate Content-Length
were switched to Buffer.byteLength instead.
no issue
- webhooks UI requires the ability to edit webhooks
- added `edit` permission for `webhook`
- added `edit` method to v2 webhook controller
- added `PUT /webhooks/:id` route to v2 Admin API routes
refs #9942
* Added new middleware to trigger events
* Refactored webhooks service
- added new trigger service, moved listen service to its own file
- started listening to new site.changed event
- cleaned up trigger service to work with new webhook fields
- cleaned up tests
- removed redundant trigger method in v0.1 controller
refs #9866
- invent preview api, but only used internally
- the idea of a preview api is definitiely reaslistic and came up in the past a couple of times
- by that we don't have to differentiate between pages or posts controller
- still support v0.1
- preview controller is not registered for http, only internal handling
refs #9866
- the entry helper is used for static pages and post lookups
- now that we support changing the api version, we have to respect the resource type
- for v2: we ask the pages controller for static pages
- in v0.1: pages and posts lived on the same route
- we are talking about the content API (!) - not admin api
refs #9866
- the static pages router uses the entry controller
- and the entry controller uses the lookup helper
- the lookup helper needs to either fetch static pages or posts
- v2 uses pages and posts controller
refs #9866
- we fallback to v0.1 by default
- we support different formats
- this opens the box to switch the ghost api version for the whole blog site
- i had to add a different notation for overrides.json, because the structure is not optimal (i only want the versions, not the shortcuts)
refs #9865
* Added generic messaging for resource not found
* Ensured integration model uses transaction for writes
* Created POST /integrations endpoint
* Created GET /integrations/:id endpoint
* Created GET /integrations endpoint
* Created PUT /integrations/:id endpoint
* Created DELETE /integrations/:id endpoint
closes#10024
- Updated input serializers for posts/tags/users to handle absolute urls conversion
-------
1. Ghost stores relative images urls
2. API V2 returns images with absolute urls
3. Ghost-Admin sends absolute urls back on any save e.g. update user
**Current behavior**: This will override the relative image path in db to absolute, which in turn won't get updated in future if domain or protocol changes for e.g.
**Fix**: On save/update, input serializers converts any absolute image url paths back to relative if the base URL from image fields matches the configured URL
closes#10029
- allowed page option for users, posts, & tags browse
- The page query param was not forwarding to the query, meaning that when the admin client requested the next page of users or posts, it would receive the first page again.
no-issue
- Added spam prevention to POST /session
- This blocks repeated requests the the /session endpoint preventing brute
force password attacks
- Updated session controller to reset brute middleware
- This updates the session controller to reset the brute force protection
on a successful login. This is required so that a user is not locked out
forever :o!!
refs #9866
- Switched update checker to api v2
- Updated and cleaned up the corresponding test suite
- Updated the frame pipeline to respect context passed in with Frame instance
- Exposed 'active' verison from api index module
* Extended webhooks schema/model and connected with integrations
refs #9942
- Updated webhooks schema with new columns - name, integration_id, secret, last_triggered_at, api_version
- Updated webhooks and integration model to map relationships
- Updated schema hash
- Updated test utils to exclude new webhooks columns for response comparison
* Added migration script for new webhooks columns
refs #9942
- Added migration script in 2.3 to add new columns to webhooks
* Updated schema hash
* Updated maxLength for api_version in schema
* Removed concurrency value from migration script
* Added defaults for webhooks model
* Added status field to webhooks for last trigger status
* Updated schema hash
* Fixed tests with status field
* Removed concurrency value in migration script
* Cleanup
* Updated schema with new fields
- last_triggered_status, last_triggered_error
no-issue
There are a few libraries, including node core that when given an array
for a query parameter will encode it as repeated query params. e.g.
```
{someParam: ['a', 'b']}
// becomes
'?someParam=a&someParam=b'
```
This adds a check for the value to stop us 500ing on repeated keys and
to add easier interop with http clients
refs #9866
- Extracted url decoration logic to utility in output serializers in posts, pages, users, and tags
- Added test cases for url usage by child object (tags of posts)
refs #9866
- Refactored overrides config to include direct version configs(v0.1, v2), supported versions map to direct version
- Refactored `getApiPath` to handle direct versions as well as mappings of supported version
closes#9962
- Fixed the bug with url being set to /404 when id was not present on the model
- Added a functional test to cover this bug
- Refactored url decorating methods to be more clear about the nature of passed parameters
no-issue
The content API only supports GET requests so has no need for cors
middleware on OPTIONS. This also removes the router.del helper as it's
not used
* Added API Key auth middleware to v2 content API
refs #9865
- add `auth.authenticate.authenticateContentApiKey` middleware
- accepts `?key=` query param, sets `req.api_key` if it's a known Content API key
- add `requiresAuthorizedUserOrApiKey` authorization middleware
- passes if either `req.user` or `req.api_key` exists
- update `authenticatePublic` middleware stack for v2 content routes
* Fixed functional content api tests
no-issue
This fixes the functional content api tests so they use the content api
auth.
* Fixed context check and removed skip
* Updated cors middleware for content api
* Removed client_id from frame.context
no-issue
The v2 api doesn't have a notion of clients as we do not use oauth for it
* Fixed tests for posts input serializer
refs #9866
- Added logic ensuring page filter is always set to false in posts endpoint for Content API
- Added functional tests to pages and posts
- Added absolute_url logic in pages controller
refs #9866
- by default it used `options.id`, which tells the permission layer the target id
- but some controllers want to use a different identifier
- e.g. settings -> settings.key
- e.g. password changes -> password[0].user_id
- Added slugs controller to v2 API
- Added slugs tests to v2 API
- Updated generic validation error message in shared validator to return validation error with sub-message
no-issue
With the new framework it is hard to handle 404 errors outside of the
serialization layer, this is because we cannot force destroy, edit or
findOne to error if the model is missing. This lets us do that.
* Stopped api key from assigning the 'Owner' role
refs #9865
We do not want api keys to be able to assign the Owner role to any other
key or user.
* Cleaned up Role model permissible method
no-issue
refs #9866
- there was a missing step in the shared validator
- we have to differentiate between data validation for browse/read and data validation for add/edit
- furthermore, the data validation for add/edit was missing and was not copied over from v0.1 (check structure of incoming body)
- adds the ability to require properties from req.body.docName[0]
closes#9982
This adds the subdirectory to the path for the session cookie, enabling
cookies to be sent/set/parsed for the session authentication to work.
closes#9983
- everything is described in the target issue
- this PR fixes both problems described in the issue
- TryGhost/Ghost-CLI#839 was raised to avoid this problem in the future
closes#9972
* Added breaking test for node v6 session auth
* Updated session middleware to support node v6
This uses the legacy url to obtain the origin rather than the WHATWG
URL class in order to support node <6.14.4
closes#9927
- Added post model implementation to be able to store up to 10 versions of mobiledoc
- Bumped GQL to support filtering on the mobiledoc revision table
- Added tests ensuring new functionality works
* Added api_key_id to frame.context
refs #9865
This is to allow controllers to check permissions using api_key_id data.
* Removed client and client_id from frame.context
refs #9865
This is unused as we only support oauth on v0.1 API.
refs #9865
Both the Post and the Author model implement the permissible method,
however the Post model does not abide by the signature of the
permissible method and add their own parameter "result" at the end.
This makes changes to the permissible method difficult as we have to
take into account multiple signatures.
This changes the Post model permissible method to the correct signature,
but still retains the current functionality. This will make it easier to
break up future permission related PR's so they can be reviwed easier
and faster!
closes#9959
This issue existed because the logic assumed that if there were no
query parameters then there would be no `query` object. However this is
not the case. What we really wanted to check was for the existence of an
"r" query param - the code has been refactor to explicitly do this now.
refs #9866
- moved the tests either to unit tests or routing tests
- or removed test case (a lot)
- this commit is very big 🤪, it was not rly possible to create clean commits for this
- it only changes the test env, no real code is touched
Next steps:
- optimise folder structure + make v2 testing possible
- reduce some more tests from routing and model integeration tests
refs #9866
- prep for v2
- you can better unit test the permissible function
- this avoids copying over the permission handling to v2 controller
- it was possible to move this logic into the model layer, because we now support `unsafeAttrs`
no issue
- now that we have a concept of `unsafeAttrs`, we can move the custom permissions to the invite model
Why doing now?
A) We won't copy this controller code to v2.
B) Makes it easier to unit test this behaviour
* Added admin specific auth{enticate,orize} middleware
refs #9865
This middleware will be used by the admin api to authenticate and
authorize requests
* Update v2/admin to use authAdminApi middleware
refs #9865
This changes thh auth middleware to use the adminApi authenticate and
authorize middlewares underneath, it also renames the middleware to be
consistent with the naming of the api.
* Removed oauth specific endpoints from /v2/admin
refs #9865
These are not to be used in v2/admin
* Wired up the session controller to the admin api
refs #9865
These endpoints will be used by ghost admin to login, confirm logged in status and logout
refs #9326, refs #9866
**ATTENTION: This is the first iteration. Bugs are expected.**
Main Goals:
- add support for multiple API versions.
- do not touch v0.1 implementation
- do not break v0.1
## Problems with the existing v0.1 implementation
1. It tried to be generic and helpful, but it was a mixture of generic and explicit logic living in basically two files: utils.js and index.js.
2. Supporting multiple api versions means, you want to have as less as possible code per API version. With v0.1 it is impossible to reduce the API controller implementation.
----
This commit adds three things:
1. The tiny framework with well-defined API stages.
2. An example implementation of serving static pages via /pages for the content v2 API.
3. Unit tests to prove that the API framework works in general.
## API Stages
- validation
- input serialization
- permissions
- query
- output serialization
Each request should go through these stages. It is possible to disable stages, but it's not recommended.
The code for each stage will either live in a shared folder or in the API version itself. It depends how API specific the validation or serialization is. Depends on the use case.
We should add a specific API validator or serializer if the use case is API format specific.
We should put everything else to shared.
The goal is to add as much as possible into the shared API layer to reduce the logic per API version.
---
Serializers and validators can be added:
- for each request
- for specific controllers
- for specific actions
---
There is room for improvements/extensions:
1. Remove http header configuration from the API controller, because the API controller should not know about http - decouple.
2. Put permissions helpers into shared. I've just extracted and capsulated the permissions helpers into a single file for now. It had no priority. The focus was on the framework itself.
etc.
---
You can find more information about it in the API README.md (api/README.md)
- e.g. find more information about the structure
- e.g. example controllers
The docs are not perfect. We will improve the docs in the next two weeks.
---
Upcoming tasks:
- prepare test env to test multiple API versions
- copy over the controllers from v0.1 to v2
- adapt the v2 express app to use the v2 controllers
no issue
- optimised only for web/ folder, because it has used very general namespaces
- the debug namespace must be specific, otherwise i run `DEBUG=ghost:api:*` and i get web debug logs and api folder debug logs
- we can come up with a new namespace system, but for now it must be explicit enough
Refs #9936
- Updated method to take single options param with version and admin instead of separate values
- Updated urlFor method to use the updated syntax
- Updated parent-app to use updated syntax
refs #9865
Note that this controller is the singular, that's because we plan to
make a session resource controller to be used with /sessions, wheras
this is on /session
Moved URL attributes logic from the model into API layer
refs #9866
- Moved URL related attribute calculation for posts, users, and tags into API layer
- Added test coverage for url attributes in tags/authors/primary_tags/primary_authors
refs #9866
- req.body is undefined if we don't use the body parser
- the content API only offers "fetch" endpoints, but if a component/module in Ghost relies on req.body being present, it can crash
- e.g. the authentication service checks for the existence of client_id + client_secret in req.query or req.body
- we could theoretically change it from `if (!req.body.client_id` to `if (req.body && !req.body.client_id)`, but that makes the code very hard to read + maintain
- we will use the body parser for the content API now
- req.body will be {}
no issue
- support promise and none promise tasks
- helpful if you create an array of operations and not all of the operations/tasks are async
- `response instanceof Promise` does not work for all cases e.g. some usages return a transaction/bookshelf chain
refs https://github.com/TryGhost/Ghost/issues/9865
- schema migrations
- adds `integrations` and `api_keys` tables
- inserts `integration` and `api_key` permissions and Administrator role relationships
- inserts `Admin Integration` role and permissions
- adds `Integration` model
- adds `ApiKey` model
- creates default secret if not given
- hardcodes associated role based on key type
- `admin` = `Admin API Client`
- `content` = no role
- updates `Role` model to use `bookshelf-relations` for auto cleanup of permission relationships on destroy
refs #9865
* This service handles the session store and exporting middleware to be
used for creating and managing sessions
* Updates the auth service index.js file in line with how we do things elsewhere
* After wrapping the exports in a getter, the usage of rewire had broken
the authenticate tests, this commit _removes_ rewire from the tests, calls `init` on
the models before the tests (needed because rewire isn't there) and also
cleans up the use of var.
refs #9866
- preparation for v2
- moved api/ to api/v0.1
- do export v0.1 straight from the api folder, we don't want to touch this right now
- that means currently if you require the api folder, we return v0.1 by default
- there were some direct requires of api files in the test env
- some of them use rewire
- for now, we just correct the require path to require api/v0.1/
- we touch the test env next week
**Docs about V2 design are coming soon!**
refs #9866
- Removed `toJSON` call in `findPage`
- Added JSON serialization on API layer
- Reason: model and api layer were coupled - all other model actions just returned the raw data and no specific format
- Corrected test suites to serialize fetched models to JSON
- Removed `absolute_urls` attribute from validOptions findPage methods as it's no longer needed in the data layer
- Changed 'include' test as this option is now tolerated and returns data
refs #9865
This is to ensure that if a controller returns a function, it will
always get called regardless of method.
Also cleaned up top level const usage
refs #9866
- added api version config to overrides, which makes it possible to have a centralized api versioning configuration
- the next PR will use this config in the web folder
- make api url generation in url service flexible and dynamic
- remove hardcoded API_PATH
- updated all places which used `urlFor('api'..)` -> we now ask for explicit api version
refs #8576
- adds new API endpoint `/uploads/profile-image` for uploading profile images
- new validation which fails with error message if uploaded image is not square
- Renamed getImageSizeFromFilePath to getImageSizeFromStoragePath, because it's more explicit
- Add new getImageSizeFromPath method, which is used in the new dimensions middleware
- Ensure we use the sharp middleware to auto-resize the uploaded profile pictures
- Ensure the new route get's added to v2
While this makes sure all future profile images uploaded are square, this doesn’t affect any existing non-square profile image. Needs more thought on how to handle existing non-square profile images for the purpose of making theming easier in future.
refs #9866
- if we start with v2 controllers, the code base should not require specific api controllers
- because e.g. `require('../api/posts')` will no longer exist
- if you require the api folder, you will get the latest available version by default e.g. `require('../api').posts`
- this branch does not touch the test env (!)
refs #9866
- if I want to do a project search and looks for model usages e.g. `models.`, then I won't find these usages
- normalise how we require models -> consistency
refs #9881
This is because when extending these methods, you need to know the
contents of the extraAllowedProperties to replicate it in the subclass,
breaking the principle of open/closed.
Extended uncapitalise unit tests
refs #9866
- Proved that the middleware works for many API versions
- Added test case to prove that the version identifier gets lowercased too
refs #9866
- Removed `res.isAdmin` flag in v2 express app
- Did not touch v0.1 express app
- Separated url redirect middleware for admin and content API
refs #9866
- Registered Content API under /ghost/api/v2/content/
- Registered Admin API under /ghost/api/v2/admin/
- Moved API v0.1 implementation to web/api/v0.1
- Created web/api/v2 for the new api endpoints
- Started with reducing the implementation for the new Content API (the Content api does not serve admin api endpoints, that's why it was reducible)
- Covered parent-app module with basic test checking correct applications/routes are being mounted
- Added a readme file, which contains a warning using v2, because it's under active development!
- This PR does only make the new endpoints available, we have not:
- optimised the web folder (e.g. res.isAdmin)
- started with different API controllers
- reason: we want to do more preparation tasks before we copy the api controllers
closes#9749
- disallow indexing of /p/ in robots.txt
- add meta robots tag to preview pages
- robots.txt wont' be applied for blogs not mounted to the root (i.e. https://example.com/blog/)
no issue
- private blogging was once part of a single big express app, see: c1a2601514
- therefor we needed to differentiate if the request is for /api/v0.1 or for the actual site
- nowadays, we register apps only for the site express app
no issue
- the 'rss.feed' filter used to get an empty feed as a parameter, because item generation is done via promises
- now it waits until all items have been added
refs #9848
- Disabled image optimization for .gif files as a temporary solution until sharp library starts supporting latest version of libvips
- Disabled image optimization for .svg/.svgz files as permanent solution as these file types are being converted to .png
closes#9832
The API _should_ be returning absolute URLs for everything, 3rd party applications require absolute urls to read and display ghost data correctly. Currently they have to concat the blog url and the resource url, which is very uncomfortable.
Changing the public api like this would be considered a breaking change however so we've opted to put it behind a query parameter named `absolute_urls`.
refs #4453
* On by default
* Added config to disable resizing
* Added basic image optimization processing
* Added dep: sharp (optional dep)
* Added resize middleware
* Take care of rotation based on EXIF information
* Removed all meta data from optimised image
* Added handling if sharp could not get installed
* Do not read ext twice - optimisation
* Do not call sharp if config is disabled
* Do not remove the original image which was uploaded (store 2 images)
* Support of `req.files` for internal logic
* Disabled cache to enable file removal on Windows
closes#9134
- Added form_id, input_id, and button_id parameters to subscribe_form helper
- Added id parameter to input_email helper
- Added test coverage to input_email helper
- Added quotes to id attributes for consistency
- Added subscribe_form helper tests
- Updated express to v4 in helper tests
closes#9822
- Fixed the post count issue for co authors
- Corrected and refactored tests related to users post count
- Consistency fix, because we return all posts where the author is primary or co author for the author page already
closes#9786
- Make GET request when url has no provider match
- The HEAD request was made in order to send less data over the wire when
checking for redirects for urls that do not have an oembed provider
match. We are now going to look for provider metatags withing the
response of the request - rather than making a HEAD followed by a GET if
no redirect is found, this condenses that to a single request.
- Try to get OEmbed data from tag if no provider
- Here we parse the HTML response of the resource and look for a link tag
that will give us the oembed resource url which we can use to fetch the
embed html
closes#9644
- Removed google blogsearch from pingList, because it's dead (https://plus.google.com/+GoogleWebmasters/posts/46W7ZwVrwqg)
- Updated pingomatic method weblogUpdate -> weblogUpdates
- Updated RPC response handler to check for errors, log improvement
- the pingomatic service sends back a 200 even when it errors, so we
check the xml response to see if it's good, it throws and passes of to
the catch handler already in place
- Avoid multiline XML error message strings, but
includes a catch in case our regex stops working with a fallback to
multiline XML error message
- we check for arbitrary whitespace between different XML tags,
which I haven't seen in any of the responses - but it could change I
guess. I haven't added support of whitespace between the tags, as I
believe that would be a different value with XML spec. And we wanna
match on exact values here.
no issue
- we have to explicitly reset the previous `updated_at` field, because Bookshelf auto-updates this field on each update
- we have to extend the condition to avoid updating the `updated_by` field
- detect and respect `options.migrating`
no issue
- do only look for published old fixture posts
- otherwise we detect draft old fixture post
- and then we would replace them with published new fixture posts, which is not a very nice experience for the user
- ensure, if we have found all old fixture posts, replace all of them with the correct date
- otherwise they are getting replaced and the date is "now"
- in general, this migration script is tricky and it tries to be smart, but there are so many cases we can run into
- to remember: the goal was to replace all old with new fixture posts (e.g. you just installed 1.25 and straight migrate to 2.0 - the old fixture posts should get replaced)
- added more protections to ensure we never delete custom posts using the same fixture post slugs
closes#9802
- we have to trigger both functions within Ghost core, otherwise people who are using Ghost as NPM module have to call these functions
- this is internal logic
- plus: this logic is conditional, because of our internal maintenance flag
- make it backwards compatible in case you call announceServerStart or announceServerStopped twice
- tested with "Ghost as NPM module" and with the CLI on production
* 🐛 Dynamic Routing Beta: Shortform `author.foo` is not allowed
refs #9601
- otherwise you get access to {{author}} in the theme, which is deprecated & causes errors
- recommend not using {{author}} as data longform name in the yaml validator
- (internal usage is still allowed)
- also warn against using reserved data key names like filter, resource, limit etc - maybe remove this restriction later but seems like a sensible validation right now.
no issue
- Organising your content
From: Here, the theme would assign the post publicly displayed tags of Blog - but it would also keep a private record of the post being tagged with #video.
To: Here, the theme would assign the post publicly displayed tags of News - but it would also keep a private record of the post being tagged with #video.
no issue
- if you have a bootstrap socket configured, but the socket does not appear, Ghost would crash
- cases:
- your server bootstraps (Ghost-CLI automatically starts Ghost via systemd)
- you add a bootstrap config in your Ghost config manually, but you don't start a net server
- added handling to catch errors and retry 3 times
no issue
- IPC communication happens before we trigger process.exit
- this is a timing issue
- we should give IPC a little time to finish sending the message over to the CLI
- wait 100ms before exiting the process
refs #9601, refs #9744
- the express router reference wasn't updated fully
- the stack was the old router stack when you have uploaded a new routes.yaml file
- this has caused e.g. that new redirects for channels/collections didn't work after the upload
refs #9601
- when using the short form `data: tag.welcome` the redirect is enabled by default
- /tag/welcome/ will redirect to the channel/collection which makes use of the data key
- you can disable the redirect by using the long form
e.g. data:
tag:
resource: tags
type: read
slug: welcome
redirect: false
closes#9774, refs #9742
- added new fixture posts for Ghost 2.0
- added migration file to remove old fixture posts
- only remove them if they are owned by the Ghost author and if they are tagged with getting-started
- added new fixture posts if you had all (!) old fixture posts
- ensure on rollback we remove the new fixture posts again
- updated default settings
refs #9742
- removed usage of single permalink setting
- with dynamic routing this configuration does no longer makes sense
- because you can configure your permalinks in the routes.yaml
- furthermore you can have multiple collections with multiple permalinks
- removed @blog.permalinks
- do not export permalink setting
- do not import permalink setting
- permalink setting UI will be removed soon
- get rid of {globals.permalink} completely
- remove yaml in-built migration
- do not expose settings.permalinks via the private API
- do not allow to edit this setting
- keep phyiscal value in case a blog needs to rollback from v2 to v1
- sorted out when the routers should be created
- ensure routes.yaml file doesn't get validated before Ghost is fully ready to start
refs #9601, refs #9742
- Upgraded NQL to 0.1.0
- The new version of NQL supports aliases e.g. `tag: tags.slug`, which makes it possible to define `filter=tag:support`
- Furthermore, this allows us to support advanced filtering like tag:[a,b]
- In dynamic routing, we use mingo via NQL which has a slightly different feature set to GQL in the API:
- AND NOT, OR and other advanced logic combos DO work on joined tables
- Counts are not yet supported
- The Dynamic Routing beta docs should describe that API filtering and Dynamic Routing filtering is different
refs #9742, refs #9724
- handle König Editor format for 2.0
- adapted importer to be able to import 1.0 and 2.0 exports
- added migration scripts
- remove labs flag for Koenig
- migrate all old editor posts to new editor format
- ensure we protect the code against mobiledoc or html field being null
- ensure we create a blank mobiledoc structure if mobiledoc field is null (model layer)
- ensure you can fully rollback 2.0 to 1.0
- keep mobiledoc/markdown version 1 logic to be able to rollback (deprecated code)
refs #9744
- added two new endpoints to upload/download routes.yaml
- reload site express app on successful/valid upload
- reload url service on sucessfuly upload
- force clear cache of pages
- ensure we keep a backup of the routes.yaml file
- this feature was mostly tested manually
- @TODO: i have to write unit tests - will do later
- @TODO: do a memory test to ensure we haven't introduced any memory leaks with this feature
refs #9742
- rebase against master updated some docs links again
- go over code base again and double check that all docs links are correct
- 2.0 will become the latest version on our readme pages
refs #9742, refs https://github.com/TryGhost/Ghost-CLI/issues/759
- required a reordering of Ghost's bootstrap file, because:
- we have to ensure that no database queries are executed within Ghost during the migrations
- make 3 sections: check if db needs initialisation, bootstrap Ghost with minimal components (db/models, express apps, load settings+theme)
- create a new `migrator` utility, which tells you which state your db is in and offers an API to execute knex-migrator based on this state
- ensure we still detect an incompatible db: you connect your 2.0 blog with a 0.11 database
- enable maintenance mode if migrations are missing
- if the migration have failed, knex-migrator roll auto rollback
- you can automatically switch to 1.0 again
- added socket communication for the CLI
refs #9742
- rename column from amp -> comment_id
- iterate over all posts and ensure we use the resource id or the original amp value
- provide down hook to undo this change
no issue
- these tests were a mess
- we had many duplicated tests
- it was very hard to work with the exporter files
- i have tidied up the whole file
- first section: all general importer tests based on the current Ghost version
- second section: 1.0 tests
- everything is now JSON based (much easier to control)
refs 9742
- when we've introduced Ghost 1.0, we have noticed that we broke Disqus comments
- Disqus comments use a unique identifier, which is the post id
- that means if you have exported your LTS content and imported it into 1.0, all resource identifiers are regenerated
- but the Disqus must use the original post resource id
- that's why we have imported the old post id and remembered it in the `amp` field 🤠
- that was the only field which was available and un-used
- now in Ghost 2.0, we would like to rename the `amp` field to `comment_id`
- string length: 50 (i thought it might be safer to not use 24 characters, because who knows if support other comment id's in the future)
refs https://github.com/TryGhost/Ghost/issues/9724
- perform a HEAD request on a url if we don't find a matching provider, following any redirects until we hit success response before looking up providers for the resulting url
refs #9601
- on resource update/add we have cached mobiledoc, html etc
- we have to ensure we exclude the fields (same procdure happens on bootstrap)
- these excluded fields don't have to be cached
- otherwise memory usage is higher in general
- ensure we cache relations with a minimal field set on resource update/add
closes#9587
- when `useNullAsDefault` was invented in knex 0.10, it was a breaking change, that update/insert etc no longer set's null as default if a field is missing
- at this time we thought it only affects our test env, because the test generator doesn't generate all fields
- but turned out the importer is affected as well e.g. you import a post with missing fields
- the importer doesn't iterate over all fields and checks if the field is present or not
- as this only happens with SQlite3, we should enable `useNullAsDefault` by default
- you can still disable this option if you want, but not recommended
- the reason why knex added this breaking change was that some applications want "undefined" as value
- this is not the case in Ghost, so it's fine to make use of the default null behaviour
refs #8719
- initial commit: 40c8eacd44
- we have forgotten that there is another endpoint which triggers an export (the backup endpoint)
- this endpoint needs to accept the new `include` query param as well (was missing)
no issue
- discovered while testing
- SQLite3 has a restriction that you cannot query more than 999 SQL variables (e.g. fetch all tag relations for all posts)
- if you have more than 999 posts in your database, Ghost wasn't able to finish the bootstrap
- the url service tried to fetch all posts & relations, but could not complete, because SQlite3 has thrown
-> "too many SQL variables"
- i have added a recursive query strategy only for SQLite3
- use offset/limit
refs #9742, refs #8719
- make it possible to import more tables (optional)
- available tables: clients, trusted domains
- by default we won't import these tables, you have to tell Ghost using `include` (same syntax on export)
- we won't announce this ability for now (stays hidden)
refs #9742, refs #8719
- you can now use `include` to export extra tables e.g. `include=clients`
- admin client won't make use of this option yet, maybe later and optional
- we won't announce this new ability for now (stays hidden)
refs #9751
- the mobiledoc field can be null
- e.g. if you import a JSON with no markdown/mobiledoc or html field
The migration script for 1.25 had only the purpose to migrate existing Koenig Beta posts.
no issue
- add a new migration for 1.25 to insert the draft demo post for existing blogs
- ensure new blogs get the draft demo post as well
- tested on sqlite3 + mysql
- added handling if Ghost Author user doesn't exist anymore (fallback to owner user)
refs https://github.com/TryGhost/Ghost/issues/9742
We've identified some changes we need to make to the HTML output of the [new Koenig editor](
https://forum.ghost.org/t/koenig-editor-beta-release/1284/102) for future proofing and consistency across cards.
- the `<div class="kg-post">` wrapper around post content has been removed
- for image cards the `.kg-image-wide` and `.kg-image-full` classes have been changed to `.kg-width-wide` and `.kg-width-full` and applied to the `<figure>` element rather than the `<img>` element
Before:
```html
<div class="kg-post">
<figure class="kg-image-card">
<img class="kg-image kg-image-wide" src="...">
<figcaption>example wide image</figcaption>
</figure>
</div>
```
After:
```html
<figure class="kg-image-card kg-width-wide">
<img class="kg-image" src="...">
<figcaption>example wide image</figcaption>
</figure>
```
refs #9742
- Ghost 2.0 is coming
- all doc links in 1.0 must use concrete links e.g. docs.ghost.org/v1 or themes.ghost.org/v1.23.0/
- if we release Ghost 2.0, docs.ghost.org will show 2.0 docs
closes#9727
- this is a short term fix to proof that Disqus comments on preview pages no longer appear on other threads
- this is not a full solution to the problem
- the private API still returns /404/, which is right now inconsistent, but not critical in any way
- the url helper will now output the post preview url if you serve a draft/scheduled post
- this should register unique page urls at Disqus and ensure uniquness for threads
- i still don't understand why the cross posting happens at all, because we also pass an unique identifier to Disqus (the post ID)
- it could be that comments, which are added on the preview page, won't appear on the published urls, because
the published url !== preview url. I wasn't able to figure this out via testing or reading their docs
refs https://github.com/TryGhost/Ghost/issues/9623
- added `DomModifier` class to walk a SimpleDom document and modify as needed
- adds `id` attributes to `h1`, `h2`, etc heading tags
- converts H* tag content to a dasherized string for the id attribute (dasherized id's are different to the smushed ids that are generated by our markdown converted but there are no backwards-compatibility concerns here)
- if a duplicate id is detected then add a `-1`, `-2`, etc suffix to the id
- use `DomModifier` after converting mobiledoc to SimpleDom but before serialising to html
- switched top-level var declarations to es6
closes#9715
- changed the `urlWithoutSubdirectoryWithoutAmp` variable in the amp
router to only match /amp or /amp/ at the end of the url string, instead
of just matching any occurrence of /amp in the url string
no issue
- replaced `querySelector` with `querySelectorAll` and a loop so that all subscribe form inputs have their values updated rather than only the first form on the page
- made the selector more specific so that it only updates `<input>` elements
- switched to a template string so it's easier to read/write
refs #9601
- this was already working for collections or channels
- but the `routeName` was not parsed for static routes
- ensure we push the route name into the context object
e.g. /about/: about
-> name of the route is "about"
no issue
- discovered while testing (issue doesn't exist)
- with dynamic routing we have introduced a bug
- we pre-generate urls and if your permalink contains a date (dated permalink),
we have to regenerate the urls, otherwise the dated urls do not respect your blog timezone
- collection router has to subscribe to the timezone event
- collection router must trigger an update on it's url generator if the timezone changes and the it's permalink is dated
- ensure we also update the urls on import
no issue
- reported in the forum: https://forum.ghost.org/t/publishing-with-a-single-post-request-to-posts/1648
- the defaults are defined in two places
1. on the schema level (defaults for the database)
2. on the ORM (model layer)
- the defaults on the db layer are set correctly when inserting a new resource
- but if we don't apply all defaults on the model layer, it will happen that model events are emitted without the correct defaults
- see comment in code base
- it's caused by the fact that knex only returns the inserted resource id (probably caused by the fact knex has to support x databases)
- components/modules are listening on model events and expect:
1. a complete set of attributes
2. a complete set of defaults
3. sanitized values e.g. bool, date
- this commit fixes:
1. added missing defaults for user & post model
2. sanitize booleans (0|1 => false|true)
3. added tests to ensure this works as expected
4. clarfies the usage of `defaults`
Regarding https://forum.ghost.org/t/publishing-with-a-single-post-request-to-posts/1648:
- the post event was emitted with the following values {page: undefined, featured: undefined}
- the urlservice receives this event and won't match the resource against collection filters correctly
- NOTE: the post data in the db were correct
- it’s good practice with password reset emails to offer peace of mind
to the user if they didn’t initiate the request and to let them know
that if they didn’t, it’s safe to ignore
refs #9601
- replace jsonpath with [NQL](https://github.com/NexesJS/NQL)
- jsonpath was just a temporary solution (a short-term fix)
- with NQL we are able to filter collections more powerful in the near future
- NQL is not feature complete
- we still support `featured:true` for collections
refs #9601
- you can now use `rss:false`
- ability to define a custom rss url with a target template (+ content_type)
- ability to disable rss for channel or collection
refs #9601
- refactor architecture of routing so you can define a channel
- a channel is a different way of looking at your posts (a view)
- a channel does not change the url of a resource
Example channel
```
routes:
/worldcup-2018-russia/:
controller: channel
filter: tag:football18
data: tag.football18
```
- added ability to redirect resources to a channel/static route
- support templates for channels
- ensure we still support static routes (e.g. /about/: home)
- ensure pagination + rss works out of the box
refs #9601
- you can define a redirect in your routes.yaml
e.g. from `page.home` to /about/
- we have to check for a possible redirect before rendering the target static page
refs #9601
- support data, limit and order for collections
- limit definition in routes.yaml is stronger than theme package.json limit configuration
- ensure we update hbs template options
refs #9601
- the home.hbs behaviour for the index collection (`/`) is hardcoded in Ghost
- we would like to migrate all existing routes.yaml files
- we only replace the file if the contents of the routes.yaml file equals the old routes.yaml format (with home.hbs as template)
- updated README of settings folder
- if we don't remove the home.hbs template from the default routes.yaml file, home.hbs will be rendered for any page of the index collection
- the backwards compatible behaviour was different
- only render home.hbs for page 1
- remember: the default routes.yaml file reflects how Ghost was working without dynamic routing
refs #9601
Example:
```
collections:
/podcast/:
permalink: /{slug}/
```
- the name of the collection is remembered as `routerName` (in the case above: "podcast")
- the name of the collection is important for two things
1. context value
2. template name
- the context value is available for specific theme helpers e.g. is helper, body_class helper
- we auto-lookup the collection name in your theme e.g. podcast.hbs
- this logic does not apply to static routes
- if you define templates on your collection, they are stronger than the collection name
refs #9601
- implementation of resource listener updated
- if you define two collections: `featured:true` (1) and `page:false` (2) you can run into the following bug:
- you create a published post (owned by (2))
- you change the status to featured
- still owned by (2), because the filter still matches (it's still not a static page)
- this adaption fixes the behaviour
- less smart logic, but less error prone
closes https://github.com/TryGhost/Ghost/issues/9674
- with dynamic routing the first collection get's the "index" context attached
- the index context signalises the main post listening route (first collection)
- this behaviour was present < 1.24 - we have to keep this behaviour
refs #9601
- if you call `express.Router()`, the router's name is always "router"
- that is caused by the closure behaviour in express:
- https://github.com/expressjs/express/blob/4.16.3/lib/router/index.js#L46
- Ghost creates a couple of express routers for dynamic routing
- it depends how much you configure in your routes.yaml file
- but every router is called "router"
- this is hard to work with
- with this router wrapping logic, we are able to give each router an exact name
If you enable `DEBUG=ghost:services:routing:*`, you have seen this before
> ghost:services:routing:ParentRouter site: mountRouter: router +0ms
With the wrapper logic, you will see:
> ghost:services:routing:ParentRouter site: mountRouter: StaticPagesRouter +0ms
- furthermore, if you have to access the router stack (`app.router.stack`), you can easily identify and find router instances by name
no issue
- if you define no collections, but a static route, it can happen that the target template to render
makes use of the {{ghost_head}} helper
- the {{ghost_head}} helper tries to create the primary rss feed url
- at the moment: no collections, no primary rss feed url
- if we offer the option to define custom rss rules, this function might need an extension
closes#9675
- with dynamic routing we have introduced a breaking change, which we have overseen
- Ghost does not return absolute urls, that's why the clients need to concat the blog url and the resource url
- with 1.24.0 Ghost returned resource urls including the subdirectory
- this caused trouble for e.g. zapier or the preview feature in the admin client
- revert breaking change and ensure we only expose resource urls without subdirectory
refs https://github.com/TryGhost/Ghost/issues/9623
- add `oembed-parser` module for checking provider availability for a url and fetching data from the provider
- require it in the `overrides.js` file before the general Promise override so that the `promise-wrt` sub-dependency doesn't attempt to extend the Bluebird promise implementation
- add `/oembed` authenticated endpoint
- takes `?url=` query parameter to match against known providers
- adds safeguard against oembed-parser's providers list not recognising http+https and www+non-www
- responds with `ValidationError` if no provider is found
- responds with oembed response from matched provider's oembed endpoint if match is found
no issue
- was introduced with dynamic routing beta: https://github.com/TryGhost/Ghost/releases/tag/1.24.0
- the slug param wasn't forwarded correctly
- you were not able to render a custom tag or author template e.g. `tag-news.hbs`
refs #9681
- we already had a protection against these situations when serving the site (theme)
- it can happen that we have to initialise the express engine in the error handler in case the first request to /ghost produces an error (e.g. 503)
- otherwise the underlying error message is hidden and Ghost doesn't render the error html template correctly
closes#9674
- the collection router had a hardcoded default context "home"
- this is wrong
- the context array get's automatically filled for the collection
- if you are serving a page e.g. /page/2/ -> it's "paged"
- if you are serving / -> it's "home"
- same for {{body_class}}, it outputs "home-template" on "/"
- this is the same behaviour as in 1.23.x
no issue
- reverse must happen once in the constructor
- otherwise we reverse the array on each request
- Ghost would randomly pick the first and then the second template
no issue
- from now on: you have to manually reconfigure your slack hook after importing your data
- we were running into trouble that Ghost had import slack hooks, because it can happen very fast
that you are importing someone's slack hook
no issue
- there was a timing bug in Ghost
- we do operations in parallel on bootstrap
- 1) we fetch the resources as early as possible
- 2) we do all the rest (express bootstrapping, theme loading, router registration) etc.
- it can happen that (2) happens too slow and ends in the situation that the queue, which is responsible
to handle both parallel actions, does not wait for the routers and closes the event
- this is a short term fix
- i need to reconsider if there is a better long term fix
refs #9601
### Dynamic Routing
This is the beta version of dynamic routing.
- we had a initial implementation of "channels" available in the codebase
- we have removed and moved this implementation
- there is now a centralised place for dynamic routing - server/services/routing
- each routing component is represented by a router type e.g. collections, routes, static pages, taxonomies, rss, preview of posts
- keep as much as possible logic of routing helpers, middlewares and controllers
- ensure test coverage
- connect all the things together
- yaml file + validation
- routing + routers
- url service
- sitemaps
- url access
- deeper implementation of yaml validations
- e.g. hard require slashes
- ensure routing hierarchy/order
- e.g. you enable the subscriber app
- you have a custom static page, which lives under the same slug /subscribe
- static pages are stronger than apps
- e.g. the first collection owns the post it has filtered
- a post cannot live in two collections
- ensure apps are still working and hook into the routers layer (or better said: and register in the routing service)
- put as much as possible comments to the code base for better understanding
- ensure a clean debug log
- ensure we can unmount routes
- e.g. you have a collection permalink of /:slug/ represented by {globals.permalink}
- and you change the permalink in the admin to dated permalink
- the express route get's refreshed from /:slug/ to /:year/:month/:day/:slug/
- unmount without server restart, yey
- ensure we are backwards compatible
- e.g. render home.hbs for collection index if collection route is /
- ensure you can access your configured permalink from the settings table with {globals.permalink}
### Render 503 if url service did not finish
- return 503 if the url service has not finished generating the resource urls
### Rewrite sitemaps
- we have rewritten the sitemaps "service", because the url generator does no longer happen on runtime
- we generate all urls on bootstrap
- the sitemaps service will consume created resource and router urls
- these urls will be shown on the xml pages
- we listen on url events
- we listen on router events
- we no longer have to fetch the resources, which is nice
- the urlservice pre-fetches resources and emits their urls
- the urlservice is the only component who knows which urls are valid
- i made some ES6 adaptions
- we keep the caching logic -> only regenerate xml if there is a change
- updated tests
- checked test coverage (100%)
### Re-work usage of Url utility
- replace all usages of `urlService.utils.urlFor` by `urlService.getByResourceId`
- only for resources e.g. post, author, tag
- this is important, because with dynamic routing we no longer create static urls based on the settings permalink on runtime
- adapt url utility
- adapt tests
no issue
- the dot notation only works if you install a single lodash dependency e.g. `yarn install lodash.get`
- otherwise we have to use `lodash/get`
no issue
- discovered while coding
- the value was always false, because we've tried to read the value from the config object
- the value lives in the database and is accessible via the labs service
no issue
- we sanitise any incoming slug on the model layer e.g uppercase -> lowercase
- and when importing e.g. an uppercase slug, the importer was trying to compare the uppercase slug with the sanitised slug
closes https://github.com/TryGhost/Ghost/issues/9620
- adjust the `deduplicateSubDir` function's regex to only match duplicate subdirectories when the `url` is only a path rather than full url or the duplicate match starts with a `/`
closesTryGhost/Support#426
refs TryGhost/gscan#106
needs TryGhost/gscan#107
GScan can return errors, which was not handled in our theme validator and caused Ghost to crash completely. GScan will now return an Ignition error when its not able to read the `.zip` file.
e. g.: `{"errors":[{"message":"Failed to read zip file","context":"tife.zip","errorType":"ValidationError","errorDetails":"invalid relative path: ../tife/"}]}`
refs https://github.com/TryGhost/Ghost/issues/9505
- remove requirement for the `enableDeveloperExperiments` flag to be able to use Koenig
- it's now possible to enable as a standard Labs beta feature
refs https://github.com/TryGhost/Ghost/issues/9505
- updates mobiledoc converter's `render` method to accept a `version` argument
- `1` === Ghost 1.0's markdown-only renderer output
- `2` === Koenig's full mobiledoc renderer output
- switch between mobiledoc renderer versions in Post model's `onSaving` hook
- version 1 by default
- version 2 if Koenig is enabled (currently behind dev experiments config + labs flag)
- version 2 if the post's mobiledoc is not compatible with the markdown-only renderer
- "version 2" full-Koenig mobiledoc renderer output
- wraps content in a `.kg-post` div
- removes wrapper around markdown and html card output
- adds classes to image card output including selected image size/style
- standardises es6 usage across mobiledoc related files
refs #9601
- while i was testing different collections and different filters, i somehow thought that the default
collection does not contain featured posts 😀🙊
- this is wrong (!!!!)
- the url service is not yet connected
- so: this is not a bug
refs https://github.com/TryGhost/Ghost/pull/9592
- we add bookshelf-relations step by step if we need it
- with https://github.com/TryGhost/Ghost/pull/9592 we have rewritten the test env to use Bookshelf
- this is important for our new url service
- because the service is listening on model updates and updates the urls based on the model events
- so with moving to Bookshelf, we need any easy way to add relations
- the test env inserts test fixtures
- it adds permissions and each permission get's roles attached
- `models.Permission.add({roles: [...]})
no issue
- replaced token creation by `lib.common.security`
- added unit tests for adding invites
- allow a different invite status for internal access
refs https://github.com/TryGhost/Team/issues/65
- it's easier for the architecture if we read the setting files synchronously,
because the dynamic routing component is part of the express bootstrap and
the whole routing bootstrap is synchronously
- for now: we only read one file anyway
- it's for now easier to read the file synchronously, then i don't have to change
any existing express bootstrap architecture
refs https://github.com/TryGhost/Team/issues/65
- this is just the first optimisation regarding relative/absolute urls
- the full strike will happen when i start with the url utility re-write
- for now: there will be only one subscriber of url events -> the sitemaps service
- the sitemaps service outputs absolute urls
- we don't want to receive an url event and ask the url service again to get an absolute version of the url
closes#9569
- Removed the `<1 min read` time clause, effectively making `1 min read` the minimum reading time
- Removed the `seconds` option for i18n strings, which contained the less than one minute display string
- Kept the other i18n string options the same
- Amended and improved tests for new functionality
refs #9584
- object by reference 🎡
- we modify the object later on
- this has resulted in a wrong context output in the admin client
- e.g. we've output an updated user reference
refs https://github.com/TryGhost/Team/issues/65
- currently we generate a relative resource url
- if you configure a subdirectory, the urls have to respect that
- e.g. you configure `localhost:2368/blog`, your url results in e.g. `/blog/my-post/`
- this is not yet a critical bug, because the url service is not connected yet
- @TODO: consider absolute vs. relative urls in the url service
no issue
- removed the `routeKeywords` property from the config and used hard coded keywords.
- removed `routeKeywords` from public configuration API endpoint, as it's no longer used in the Admin.
refs https://github.com/TryGhost/Team/issues/65
We are currently work on dynamic routing (aka channels).
An important piece of this feature is the url service, which always knows the url of a resource at any time.
Resources can belong to collections or taxonomies, which can be defined in a [routing yaml file](https://github.com/TryGhost/Ghost/issues/9528). We are currently shipping portions, which will at end form the full dynamic routing feature.
### Key Notes
- each routing type (collections, taxonomies, static pages) is registered in order - depending on the yaml routes file configuration
- static pages are an internal concept - they sit at the end of the subscriber queue
- we make use of a temporary [`Channels2`](https://github.com/TryGhost/Ghost/pull/9550/files#diff-9e7251409844521470c9829013cd1563) file, which simulates the current static routing in Ghost (this file will be modified, removed or whatever - this is one of the next steps)
- two way binding: you can ask for a resource url based on the resource id, you can ask for the resource based on the url
- in theory it's possible that multiple resources generate the same url: we don't handle this with collision (because this is error prone), we handle this with the order of serving content. if you ask the service for a resource, which lives behind e.g. /test/, you will get the resource which is served
- loose error handling -> log errors and handle instead of throw error and do nothing (we log the errors with a specific code, so we can react in case there is a bug)
- the url services fetches all resources on bootstrap. we only fetch and keep a reduced set of attributes (basically the main body of a resource)
- the bootstrap time will decrease a very little (depending on the amount of resources you have in your database)
- we still offer the option to disable url preloading (in your config `disableUrlPreload: true`) - this option will be removed as soon as the url service is connected. You can disable the service in case you encounter a problem
- **the url service is not yet connected, we will connect the service step by step. The first version should be released to pre-catch bugs. The next version will add 503 handling if the url service is not ready and it will consume urls for resources.**
----
- the url service generates urls based on resources (posts, pages, users, tags)
- the url service keeps track of resource changes
- the url service keeps track of resource removal/insert
- the architecture:
- each routing type is represented by a url generator
- a routing type is a collection, a taxonomiy or static pages
- a queue which ensures that urls are unique and can be owned by one url generator
- the hierarchy of registration defines that
- we query knex, because bookshelf is too slow
- removed old url service files + logic
- added temp channels alternative (Channels2) -> this file will look different soon, it's for now the temporary connector to the url service. Also the name of the file is not optimal, but that is not really important right now.
