refs #9248
- Bookshelf gives access to ".changed" before the update
- Discussion: https://github.com/bookshelf/bookshelf/issues/1943
- We also need to know what has changed after the update to be able to decide if we should trigger events
- Furthermore: Bookshelf cannot handle relation updates, it always marks relations as changed even though they did not change
- Bumped bookshelf-relations to be able to
- know if relations were updated
- ensure we unset relations on bookshelf's ".changed"
refs #10438
- integrations != users
- Ghost's assumption is: if you create a post, the primary author becomes the logged in user
- we have to require authors for integrations
- short fix and needs some more thoughts later
refs #10388
This updates the base model to retrieve column information, and explicitly set every property whose column is `nullable` and content is the empty string (`""`) to `null`
refs #9865
- Changed id passed for api_key to an object to be able to differenciate between admin and content api requests
- Added integration id to frame context
- Small refactoring of frame context initialization
refs #10286
- this is just a hotfix for v0.1
- we keep the x_by fields for now and deprecate them
- as soon as an integration updates/inserts a resource, we just store the owner ID
- we currently work on a new concept for v2
- v2 no longer exposes or uses x_by, see https://github.com/TryGhost/Ghost/pull/10294
- we need to iterate on this change, because we currently use the naming `context.api_key_id` in the auth/API layer
refs #9865
This updates all current permissible methods to use the new function
signature which includes the hasApiKeyPermissions parameter. It also
makes sure that the hasApiKeyPermissions argument is taken into account
whenever checking before returning a resolved promise.
To be continued. This is just a tiny part of the big picture. None of these changes are fully committed to stay as they are.
refs https://github.com/TryGhost/Ghost/issues/10124
- This PR introduced additional db calls in URL service due to the need for a model recalculation (we can't rely on the objects that come with events)
* Removed unused fields from v2 Content API
- We want to ship the v2 Content API as clean and lean as we can
- Many fields in the DB aren't actually used, we shouldn't return these values
- Other values aren't useful outside of Admin clients, and shouldn't be returned either
Fields removed:
- tags: created_at, updated_at, parent
- authors: locale, accessibility, tour
- posts: locale, author status, page
refs #10124
- Author model returns only users that have published non-page posts
- Added a public controller for tags (should be extracted to separate Content API controller https://github.com/TryGhost/Ghost/issues/10106)
- Made resource configuration dynamic based on current theme engine
- This needs a follow-up PR with fixes to the problems described in the PR
refs #10105
- `options.where` is an older deprecated logic
- before the filter language was invented, Ghost generates statements for knex
- if we want to replace GQL with NQL, we can't generate these statements
- they are not understood from NQL, because NQL uses mongo JSON
- go through usages and rewrite the statements
- invent `extraFilters` for now
- we need to keep the support for `status` or `staticPages` for now (API requirement)
- IMO both shortcuts in the extra filters should be removed in the future
This commit is required for https://github.com/TryGhost/Ghost/pull/10159!
* Added updateLastSeen method to user model
refs #10138
* Refactor codebase to use user.updateLastSeen
refs #10138
This is to ensure all updates go via the same method, meaning any
specific logic can be handled in one place, it also helps with grepping
the codebase to find where this occurs
* Created updateUserLastSeen middleware for v2 admin
refs #10138
This is intended to be used with the v2 admin api and _possibly_ the
content api, to give us an accruate report on thelast time a user access
a ghost instance.
* Wired updateUserLastSeen up to v2 Admin API
closes#10138
* Fixed broken test for v2 admin api
no-issue
This test was broken because it was incorrectly testing for a method to
be called exactly once - this was irrelevant to the functionality being
tested for.
* Updated user check method to set status to active
no-issue
* Debounced the updateUserLastSeen middlware an hour
no-issue
* Resolved some PR comments
refs #9865
* Added generic messaging for resource not found
* Ensured integration model uses transaction for writes
* Created POST /integrations endpoint
* Created GET /integrations/:id endpoint
* Created GET /integrations endpoint
* Created PUT /integrations/:id endpoint
* Created DELETE /integrations/:id endpoint
* Extended webhooks schema/model and connected with integrations
refs #9942
- Updated webhooks schema with new columns - name, integration_id, secret, last_triggered_at, api_version
- Updated webhooks and integration model to map relationships
- Updated schema hash
- Updated test utils to exclude new webhooks columns for response comparison
* Added migration script for new webhooks columns
refs #9942
- Added migration script in 2.3 to add new columns to webhooks
* Updated schema hash
* Updated maxLength for api_version in schema
* Removed concurrency value from migration script
* Added defaults for webhooks model
* Added status field to webhooks for last trigger status
* Updated schema hash
* Fixed tests with status field
* Removed concurrency value in migration script
* Cleanup
* Updated schema with new fields
- last_triggered_status, last_triggered_error
no-issue
With the new framework it is hard to handle 404 errors outside of the
serialization layer, this is because we cannot force destroy, edit or
findOne to error if the model is missing. This lets us do that.
* Stopped api key from assigning the 'Owner' role
refs #9865
We do not want api keys to be able to assign the Owner role to any other
key or user.
* Cleaned up Role model permissible method
no-issue
closes#9983
- everything is described in the target issue
- this PR fixes both problems described in the issue
- TryGhost/Ghost-CLI#839 was raised to avoid this problem in the future
closes#9927
- Added post model implementation to be able to store up to 10 versions of mobiledoc
- Bumped GQL to support filtering on the mobiledoc revision table
- Added tests ensuring new functionality works
refs #9865
Both the Post and the Author model implement the permissible method,
however the Post model does not abide by the signature of the
permissible method and add their own parameter "result" at the end.
This makes changes to the permissible method difficult as we have to
take into account multiple signatures.
This changes the Post model permissible method to the correct signature,
but still retains the current functionality. This will make it easier to
break up future permission related PR's so they can be reviwed easier
and faster!
refs #9866
- prep for v2
- you can better unit test the permissible function
- this avoids copying over the permission handling to v2 controller
- it was possible to move this logic into the model layer, because we now support `unsafeAttrs`
no issue
- now that we have a concept of `unsafeAttrs`, we can move the custom permissions to the invite model
Why doing now?
A) We won't copy this controller code to v2.
B) Makes it easier to unit test this behaviour
Moved URL attributes logic from the model into API layer
refs #9866
- Moved URL related attribute calculation for posts, users, and tags into API layer
- Added test coverage for url attributes in tags/authors/primary_tags/primary_authors
refs https://github.com/TryGhost/Ghost/issues/9865
- schema migrations
- adds `integrations` and `api_keys` tables
- inserts `integration` and `api_key` permissions and Administrator role relationships
- inserts `Admin Integration` role and permissions
- adds `Integration` model
- adds `ApiKey` model
- creates default secret if not given
- hardcodes associated role based on key type
- `admin` = `Admin API Client`
- `content` = no role
- updates `Role` model to use `bookshelf-relations` for auto cleanup of permission relationships on destroy
refs #9866
- Removed `toJSON` call in `findPage`
- Added JSON serialization on API layer
- Reason: model and api layer were coupled - all other model actions just returned the raw data and no specific format
- Corrected test suites to serialize fetched models to JSON
- Removed `absolute_urls` attribute from validOptions findPage methods as it's no longer needed in the data layer
- Changed 'include' test as this option is now tolerated and returns data
refs #9881
This is because when extending these methods, you need to know the
contents of the extraAllowedProperties to replicate it in the subclass,
breaking the principle of open/closed.
closes#9832
The API _should_ be returning absolute URLs for everything, 3rd party applications require absolute urls to read and display ghost data correctly. Currently they have to concat the blog url and the resource url, which is very uncomfortable.
Changing the public api like this would be considered a breaking change however so we've opted to put it behind a query parameter named `absolute_urls`.
closes#9822
- Fixed the post count issue for co authors
- Corrected and refactored tests related to users post count
- Consistency fix, because we return all posts where the author is primary or co author for the author page already
no issue
- we have to explicitly reset the previous `updated_at` field, because Bookshelf auto-updates this field on each update
- we have to extend the condition to avoid updating the `updated_by` field
- detect and respect `options.migrating`
refs #9742, refs #9724
- handle König Editor format for 2.0
- adapted importer to be able to import 1.0 and 2.0 exports
- added migration scripts
- remove labs flag for Koenig
- migrate all old editor posts to new editor format
- ensure we protect the code against mobiledoc or html field being null
- ensure we create a blank mobiledoc structure if mobiledoc field is null (model layer)
- ensure you can fully rollback 2.0 to 1.0
- keep mobiledoc/markdown version 1 logic to be able to rollback (deprecated code)
refs #9742
- rebase against master updated some docs links again
- go over code base again and double check that all docs links are correct
- 2.0 will become the latest version on our readme pages
refs 9742
- when we've introduced Ghost 1.0, we have noticed that we broke Disqus comments
- Disqus comments use a unique identifier, which is the post id
- that means if you have exported your LTS content and imported it into 1.0, all resource identifiers are regenerated
- but the Disqus must use the original post resource id
- that's why we have imported the old post id and remembered it in the `amp` field 🤠
- that was the only field which was available and un-used
- now in Ghost 2.0, we would like to rename the `amp` field to `comment_id`
- string length: 50 (i thought it might be safer to not use 24 characters, because who knows if support other comment id's in the future)
no issue
- discovered while testing
- SQLite3 has a restriction that you cannot query more than 999 SQL variables (e.g. fetch all tag relations for all posts)
- if you have more than 999 posts in your database, Ghost wasn't able to finish the bootstrap
- the url service tried to fetch all posts & relations, but could not complete, because SQlite3 has thrown
-> "too many SQL variables"
- i have added a recursive query strategy only for SQLite3
- use offset/limit
no issue
- add a new migration for 1.25 to insert the draft demo post for existing blogs
- ensure new blogs get the draft demo post as well
- tested on sqlite3 + mysql
- added handling if Ghost Author user doesn't exist anymore (fallback to owner user)
refs #9742
- Ghost 2.0 is coming
- all doc links in 1.0 must use concrete links e.g. docs.ghost.org/v1 or themes.ghost.org/v1.23.0/
- if we release Ghost 2.0, docs.ghost.org will show 2.0 docs
no issue
- reported in the forum: https://forum.ghost.org/t/publishing-with-a-single-post-request-to-posts/1648
- the defaults are defined in two places
1. on the schema level (defaults for the database)
2. on the ORM (model layer)
- the defaults on the db layer are set correctly when inserting a new resource
- but if we don't apply all defaults on the model layer, it will happen that model events are emitted without the correct defaults
- see comment in code base
- it's caused by the fact that knex only returns the inserted resource id (probably caused by the fact knex has to support x databases)
- components/modules are listening on model events and expect:
1. a complete set of attributes
2. a complete set of defaults
3. sanitized values e.g. bool, date
- this commit fixes:
1. added missing defaults for user & post model
2. sanitize booleans (0|1 => false|true)
3. added tests to ensure this works as expected
4. clarfies the usage of `defaults`
Regarding https://forum.ghost.org/t/publishing-with-a-single-post-request-to-posts/1648:
- the post event was emitted with the following values {page: undefined, featured: undefined}
- the urlservice receives this event and won't match the resource against collection filters correctly
- NOTE: the post data in the db were correct
refs #9601
- the home.hbs behaviour for the index collection (`/`) is hardcoded in Ghost
- we would like to migrate all existing routes.yaml files
- we only replace the file if the contents of the routes.yaml file equals the old routes.yaml format (with home.hbs as template)
- updated README of settings folder
- if we don't remove the home.hbs template from the default routes.yaml file, home.hbs will be rendered for any page of the index collection
- the backwards compatible behaviour was different
- only render home.hbs for page 1
- remember: the default routes.yaml file reflects how Ghost was working without dynamic routing
refs #9601
### Dynamic Routing
This is the beta version of dynamic routing.
- we had a initial implementation of "channels" available in the codebase
- we have removed and moved this implementation
- there is now a centralised place for dynamic routing - server/services/routing
- each routing component is represented by a router type e.g. collections, routes, static pages, taxonomies, rss, preview of posts
- keep as much as possible logic of routing helpers, middlewares and controllers
- ensure test coverage
- connect all the things together
- yaml file + validation
- routing + routers
- url service
- sitemaps
- url access
- deeper implementation of yaml validations
- e.g. hard require slashes
- ensure routing hierarchy/order
- e.g. you enable the subscriber app
- you have a custom static page, which lives under the same slug /subscribe
- static pages are stronger than apps
- e.g. the first collection owns the post it has filtered
- a post cannot live in two collections
- ensure apps are still working and hook into the routers layer (or better said: and register in the routing service)
- put as much as possible comments to the code base for better understanding
- ensure a clean debug log
- ensure we can unmount routes
- e.g. you have a collection permalink of /:slug/ represented by {globals.permalink}
- and you change the permalink in the admin to dated permalink
- the express route get's refreshed from /:slug/ to /:year/:month/:day/:slug/
- unmount without server restart, yey
- ensure we are backwards compatible
- e.g. render home.hbs for collection index if collection route is /
- ensure you can access your configured permalink from the settings table with {globals.permalink}
### Render 503 if url service did not finish
- return 503 if the url service has not finished generating the resource urls
### Rewrite sitemaps
- we have rewritten the sitemaps "service", because the url generator does no longer happen on runtime
- we generate all urls on bootstrap
- the sitemaps service will consume created resource and router urls
- these urls will be shown on the xml pages
- we listen on url events
- we listen on router events
- we no longer have to fetch the resources, which is nice
- the urlservice pre-fetches resources and emits their urls
- the urlservice is the only component who knows which urls are valid
- i made some ES6 adaptions
- we keep the caching logic -> only regenerate xml if there is a change
- updated tests
- checked test coverage (100%)
### Re-work usage of Url utility
- replace all usages of `urlService.utils.urlFor` by `urlService.getByResourceId`
- only for resources e.g. post, author, tag
- this is important, because with dynamic routing we no longer create static urls based on the settings permalink on runtime
- adapt url utility
- adapt tests
refs https://github.com/TryGhost/Ghost/issues/9505
- remove requirement for the `enableDeveloperExperiments` flag to be able to use Koenig
- it's now possible to enable as a standard Labs beta feature
refs https://github.com/TryGhost/Ghost/issues/9505
- updates mobiledoc converter's `render` method to accept a `version` argument
- `1` === Ghost 1.0's markdown-only renderer output
- `2` === Koenig's full mobiledoc renderer output
- switch between mobiledoc renderer versions in Post model's `onSaving` hook
- version 1 by default
- version 2 if Koenig is enabled (currently behind dev experiments config + labs flag)
- version 2 if the post's mobiledoc is not compatible with the markdown-only renderer
- "version 2" full-Koenig mobiledoc renderer output
- wraps content in a `.kg-post` div
- removes wrapper around markdown and html card output
- adds classes to image card output including selected image size/style
- standardises es6 usage across mobiledoc related files
refs https://github.com/TryGhost/Ghost/pull/9592
- we add bookshelf-relations step by step if we need it
- with https://github.com/TryGhost/Ghost/pull/9592 we have rewritten the test env to use Bookshelf
- this is important for our new url service
- because the service is listening on model updates and updates the urls based on the model events
- so with moving to Bookshelf, we need any easy way to add relations
- the test env inserts test fixtures
- it adds permissions and each permission get's roles attached
- `models.Permission.add({roles: [...]})
no issue
- replaced token creation by `lib.common.security`
- added unit tests for adding invites
- allow a different invite status for internal access
refs https://github.com/TryGhost/Team/issues/65
We are currently work on dynamic routing (aka channels).
An important piece of this feature is the url service, which always knows the url of a resource at any time.
Resources can belong to collections or taxonomies, which can be defined in a [routing yaml file](https://github.com/TryGhost/Ghost/issues/9528). We are currently shipping portions, which will at end form the full dynamic routing feature.
### Key Notes
- each routing type (collections, taxonomies, static pages) is registered in order - depending on the yaml routes file configuration
- static pages are an internal concept - they sit at the end of the subscriber queue
- we make use of a temporary [`Channels2`](https://github.com/TryGhost/Ghost/pull/9550/files#diff-9e7251409844521470c9829013cd1563) file, which simulates the current static routing in Ghost (this file will be modified, removed or whatever - this is one of the next steps)
- two way binding: you can ask for a resource url based on the resource id, you can ask for the resource based on the url
- in theory it's possible that multiple resources generate the same url: we don't handle this with collision (because this is error prone), we handle this with the order of serving content. if you ask the service for a resource, which lives behind e.g. /test/, you will get the resource which is served
- loose error handling -> log errors and handle instead of throw error and do nothing (we log the errors with a specific code, so we can react in case there is a bug)
- the url services fetches all resources on bootstrap. we only fetch and keep a reduced set of attributes (basically the main body of a resource)
- the bootstrap time will decrease a very little (depending on the amount of resources you have in your database)
- we still offer the option to disable url preloading (in your config `disableUrlPreload: true`) - this option will be removed as soon as the url service is connected. You can disable the service in case you encounter a problem
- **the url service is not yet connected, we will connect the service step by step. The first version should be released to pre-catch bugs. The next version will add 503 handling if the url service is not ready and it will consume urls for resources.**
----
- the url service generates urls based on resources (posts, pages, users, tags)
- the url service keeps track of resource changes
- the url service keeps track of resource removal/insert
- the architecture:
- each routing type is represented by a url generator
- a routing type is a collection, a taxonomiy or static pages
- a queue which ensures that urls are unique and can be owned by one url generator
- the hierarchy of registration defines that
- we query knex, because bookshelf is too slow
- removed old url service files + logic
- added temp channels alternative (Channels2) -> this file will look different soon, it's for now the temporary connector to the url service. Also the name of the file is not optimal, but that is not really important right now.
no issue
- required for model events
- otherwise you won't receive a full data set
- in worst case you have to re-fetch the post
- required for the url service
- the url service always needs relations (authors,tags) to be able to generate the url properly
@IMPORTANT
- no API change, we still return what you are asking for
- we first edit/add the resource
- then we fetch the data with the API options
- @TODO: this can be optimised and will improve performance
picking/selecting it from the insert/update response
- this is an internal change
closes#9547
- you setup a blog with the following owner:
- email: test@ghost.org
- name: test
- slug: test
- now you import a JSON db file, which holds the exact same owner
- this owner won't be imported, because it's a duplicate
- but the slug is different (!)
- the importer tries to find a matching existing user, but won't find anything
- the importer then send an empty authors array `post.authors=[]` into the model layer
- this is not allowed -> this would mean, you are actively trying to unset all authors
no issue
- if multiple queries run in a transaction, the model events are triggered before the txn finished
- if the txn rolls back, the events are anyway emitted
- the events are triggered too early
- solution:
- `emitChange` needs to detect that a transaction is happening
- it listens on a txn event to determine if events should be triggered
no issue
- add a big comment
- describe:
- how this works
- why this is in place
- what does currently not work
- and why it will work with channels
- @TODO:
- figure out how to disallow:
- `models.Post.findAll({columns: id})`
- `post.save(data)`
- this will trigger bookshelf events and model events
- url generation currently needs a set of attributes (e.g. slug, published_at)
- will be auto-fixed with channels, because you can call `urlService.getUrl(post.id)`
- but what doesn't get solved is our model events
- e.g. `emitChange` needs `post.get('page')` to determine if it's a page
no issue
- this is no longer needed for now
- it was anyway a little bit ugly to modify bookshelf's `changed` object
- if we want to change something about figuring out if a model has changed (including relations)
-> we probably need to override bookshelf
refs #9548
- we always receive date strings from the client in ISO format
- we ensure that we transform these strings into JS dates for comparison
- when the client sends relations, we need to ensure that relations are checked as well
- will only work for the post model for now, because this is the only model which uses `bookshelf-relations`
- added unit tests
- removed some model tests, which do the same
no issue
This PR adds the server side logic for multiple authors. This adds the ability to add multiple authors per post. We keep and support single authors (maybe till the next major - this is still in discussion)
### key notes
- `authors` are not fetched by default, only if we need them
- the migration script iterates over all posts and figures out if an author_id is valid and exists (in master we can add invalid author_id's) and then adds the relation (falls back to owner if invalid)
- ~~i had to push a fork of bookshelf to npm because we currently can't bump bookshelf + the two bugs i discovered are anyway not yet merged (https://github.com/kirrg001/bookshelf/commits/master)~~ replaced by new bookshelf release
- the implementation of single & multiple authors lives in a single place (introduction of a new concept: model relation)
- if you destroy an author, we keep the behaviour for now -> remove all posts where the primary author id matches. furthermore, remove all relations in posts_authors (e.g. secondary author)
- we make re-use of the `excludeAttrs` concept which was invented in the contributors PR (to protect editing authors as author/contributor role) -> i've added a clear todo that we need a logic to make a diff of the target relation -> both for tags and authors
- `authors` helper available (same as `tags` helper)
- `primary_author` computed field available
- `primary_author` functionality available (same as `primary_tag` e.g. permalinks, prev/next helper etc)
closes#9520
- it contains a dependency bump of the latest Bookshelf release
- Bookshelf introduced a bug in the last release
- see https://github.com/bookshelf/bookshelf/pull/1583
- see https://github.com/bookshelf/bookshelf/pull/1798
- this has caused trouble in Ghost
- the `updated_at` attribute was not automatically set anymore
---
The bookshelf added one breaking change: it's allow to pass custom `updated_at` and `created_at`.
We already have a protection for not being able to override the `created_at` date on update.
We had to add another protection to now allow to only change the `updated_at` property.
You can only change `updated_at` if you actually change something else e.g. the title of a post.
To be able to implement this check i discovered that Bookshelfs `model.changed` object has a tricky behaviour.
It remembers **all** attributes, which where changed, doesn't matter if they are valid or invalid model properties.
We had to add a line of code to avoid remembering none valid model attributes in this object.
e.g. you change `tag.parent` (no valid model attribute). The valid property is `tag.parent_id`.
If you pass `tag.parent` but the value has **not** changed (`tag.parent` === `tag.parent_id`), it will output you `tag.changed.parent`. But this is wrong.
Bookshelf detects `changed` attributes too early. Or if you think the other way around, Ghost detects valid attributes too late.
But the current earliest possible stage is the `onSaving` event, there is no earlier way to pick valid attributes (except of `.forge`, but we don't use this fn ATM).
Later: the API should transform `tag.parent` into `tag.parent_id`, but we are not using it ATM, so no need to pre-optimise.
The API already transforms `post.author` into `post.author_id`.
refs #9519
- `errors.models.posts.postNotFound` -> wrong
- `errors.models.post.postNotFound` -> correct
- the i18n lib just logs the error and falls back to a valid error key
- wrong i18n keys will never break Ghost
closes#9085
Fixes an issue, where the client sets image properties to `""` after deleting the image. This causes problems with the query filter (see https://github.com/TryGhost/GQL/issues/24), as they have to be `null`.
Added a check in the model layer saving method to set value to `null`, when the property is empty.
Affected models and properties:
- `posts`:
- `feature_image`
- `og_image`
- `twitter_image`
- `users`:
- `profile_image`
- `cover_image`
- `tags`:
- `feature_image`
no issue
- currently if you would like to edit a resource (e.g. post) and you pass an invalid model id, the following happens
- permission check calls `Post.permissible`
- the Post could not find the post, but ignored it and returned `userPermissions:true`
- then the model layer is queried again and figured out that the post does not exist
- A: there is no need to query the model twice
- B: we needed proper error handling for post and role model
no issue
- replace logic for preparing nested tags
- if you have nested tags in your file, we won't update or update the target tag
- we simply would like to add the relationship to the database
- use same approach as base class
- add `posts_tags` to target post model
- update identifiers
- insert relation by foreign key `tag_id`
- bump bookshelf-relations to 0.1.10
no issue
- change behaviour from updating user references after the actual import to update the user reference before the actual import
- updating user references after the import is way less case intense
- that was the initial decision for updating the references afterwards
- but that does not play well with adding nested relations by identifier
- the refactoring is required for multiple authors
- if we e.g. store invalid author id's, we won't be able to add a belongs-to-many relation for multiple authors
- bookshelf-relations is generic and always tries to find a matching target before attching a model
- invalid user references won't work anymore
- this change has a very good side affect
- 17mb takes on master ~1,5seconds
- on this branch it takes ~45seconds
- also the memory usage is way lower and stabler
- 40mb takes 1,6s (times out on master)
no issue
- otherwise we will have trouble in the future fetching relations by foreign key
- e.g. `tag_id: {id}`
- this won't work if we don't explicitly define the name of the keys
- bookshelf can't fulfil the request
- this does not change any behaviour, it just makes use of the ability to define the names of your foreign keys
refs https://github.com/TryGhost/Ghost/issues/3658
- the `validateSchema` helper was a bit broken
- if you add a user without email, you will receive a database error
- but the validation error should catch that email is passed with null
- it was broken, because:
- A: it called `toJSON` -> this can remove properties from the output (e.g. password)
- B: we only validated fields, which were part of the JSON data (model.hasOwnProperty)
- we now differentiate between schema validation for update and insert
- fixed one broken import test
- if you import a post without a status, it should not error
- it falls back to the default value
- removed user model `onValidate`
- the user model added a custom implementation of `onValidate`, because of a bug which we experienced (see https://github.com/TryGhost/Ghost/issues/3638)
- with the refactoring this is no longer required - we only validate fields which have changed when updating resources
- also, removed extra safe catch when logging in (no longer needed - unit tested)
- add lot's of unit tests to proof the code change
- always call the base class, except you have a good reason
no issue
- `isNew` does not work in Ghost, because Ghost does not use auto increment id's
- see https://github.com/bookshelf/bookshelf/issues/1265
- see https://github.com/bookshelf/bookshelf/blob/0.10.3/src/base/model.js#L211
- we only had one occurance, which was anyway redundant
- if you add a user, `hasChanged('password') is true
- if you edit a user and the password has changed, `hasChanged('password')` is true as well
NOTE #1:
1. We can't override `isNew` and throw an error, because bookshelf makes use of `isNew` as well, but it's a fallback if `options.method` is not set.
2. It's hard to re-implement `isNew` based on `options.method`, because then we need to ensure that this value is always set (requires a couple of changes)
NOTE #2:
If we need to differentiate if a model is new or edited, we should manually check for `options.method === insert`.
NOTE #3:
The unit tests are much faster compared to the model integration tests.
I did a comparision with the same test assertion:
- unit test takes 70ms
- integration test takes 190ms
no issue
- move password hashing and password comparison to lib/security/password
- added two unit test
- FYI: password hashing takes ~100ms
- we could probably mock password hashing in certain cases when unit testing
no issue
- this commit cleans up the usages of `include` and `withRelated`.
### API layer (`include`)
- as request parameter e.g. `?include=roles,tags`
- as theme API parameter e.g. `{{get .... include="author"}}`
- as internal API access e.g. `api.posts.browse({include: 'author,tags'})`
- the `include` notation is more readable than `withRelated`
- and it allows us to use a different easier format (comma separated list)
- the API utility transforms these more readable properties into model style (or into Ghost style)
### Model access (`withRelated`)
- e.g. `models.Post.findPage({withRelated: ['tags']})`
- driven by bookshelf
---
Commits explained.
* Reorder the usage of `convertOptions`
- 1. validation
- 2. options convertion
- 3. permissions
- the reason is simple, the permission layer access the model layer
- we have to prepare the options before talking to the model layer
- added `convertOptions` where it was missed (not required, but for consistency reasons)
* Use `withRelated` when accessing the model layer and use `include` when accessing the API layer
* Change `convertOptions` API utiliy
- API Usage
- ghost.api(..., {include: 'tags,authors'})
- `include` should only be used when calling the API (either via request or via manual usage)
- `include` is only for readability and easier format
- Ghost (Model Layer Usage)
- models.Post.findOne(..., {withRelated: ['tags', 'authors']})
- should only use `withRelated`
- model layer cannot read 'tags,authors`
- model layer has no idea what `include` means, speaks a different language
- `withRelated` is bookshelf
- internal usage
* include-count plugin: use `withRelated` instead of `include`
- imagine you outsource this plugin to git and publish it to npm
- `include` is an unknown option in bookshelf
* Updated `permittedOptions` in base model
- `include` is no longer a known option
* Remove all occurances of `include` in the model layer
* Extend `filterOptions` base function
- this function should be called as first action
- we clone the unfiltered options
- check if you are using `include` (this is a protection which could help us in the beginning)
- check for permitted and (later on default `withRelated`) options
- the usage is coming in next commit
* Ensure we call `filterOptions` as first action
- use `ghostBookshelf.Model.filterOptions` as first action
- consistent naming pattern for incoming options: `unfilteredOptions`
- re-added allowed options for `toJSON`
- one unsolved architecture problem:
- if you override a function e.g. `edit`
- then you should call `filterOptions` as first action
- the base implementation of e.g. `edit` will call it again
- future improvement
* Removed `findOne` from Invite model
- no longer needed, the base implementation is the same
refs #6103
- simplify `toJSON`
- `baseKey` was not used - have not find a single use case
- all the functionality of our `toJSON` is offered in bookshelf
- `omitPivot` does remove pivot elements from the JSON obj (bookshelf feature)
- `shallow` allows you to not return relations
- make use of `serialize`, see http://bookshelfjs.org/docs/src_base_model.js.html#line260
- fetching nested relations e.g. `users.roles` still works (unrelated to this refactoring)
> pick('shallow', 'baseKey', 'include', 'context')
We will re-add options validation in https://github.com/TryGhost/Ghost/pull/9427, but then with the official way: use `filterOptions`.
---
We return all fetched relations (pre-defined with `withRelated`) by default.
You can disable it with `shallow:true`.
closes#9314
* added fixtures for contributor role
* update post api tests to prevent contributor publishing post
* update permissible function in role/user model
* fix additional author code in invites
* update contributor role migration for knex-migrator v3
* fix paths in contrib migration
* ensure contributors can't edit or delete published posts, fix routing tests [ci skip]
* update db fixtures hash
* strip tags from post if contributor
* cleanup post permissible function
* excludedAttrs to ignore tag updates for now (might be removed later)
* ensure contributors can't edit another's post
* migration script for 1.21
no issue
- the warning is "Transaction was already complete"
- destroying a user happens in a transaction, but the event is not asynchronous
- so we have to ensure that we don't operate on a finished transaction
refs #9127
- permission checks can happen everywhere in the code base
- we would like to create a context class
- global access to `options.context.is(...)`
- please read more about the access plugin in #9127 section "Model layer and the access plugin".
- removed the plugin and use direct context checks
closes#5071
- Remove hardcoded notification in admin controller
- NOTE: update check notifications are no longer blocking the admin rendering
- this is one of the most import changes
- we remove the hardcoded release message
- we also remove adding a notification manually in here, because this will work differently from now on
-> you receive a notification (release or custom) in the update check module and this module adds the notification as is to our database
- Change default core settings keys
- remove displayUpdateNotification
-> this was used to store the release version number send from the UCS
-> based on this value, Ghost creates a notification container with self defined values
-> not needed anymore
- rename seenNotifications to notifications
-> the new notifications key will hold both
1. the notification from the USC
2. the information about if a notification was seen or not
- this key hold only one release notification
- and n custom notifications
- Update Check Module: Request to the USC depends on the privacy configuration
- useUpdateCheck: true -> does a checkin in the USC (exposes data)
- useUpdateCheck: false -> does only a GET query to the USC (does not expose any data)
- make the request handling dynamic, so it depends on the flag
- add an extra logic to be able to define a custom USC endpoint (helpful for testing)
- add an extra logic to be able to force the request to the service (helpful for testing)
- Update check module: re-work condition when a check should happen
- only if the env is not correct
- remove deprecated config.updateCheck
- remove isPrivacyDisabled check (handled differently now, explained in last commit)
- Update check module: remove `showUpdateNotification` and readability
- showUpdateNotification was used in the admin controller to fetch the latest release version number from the db
- no need to check against semver in general, the USC takes care of that (no need to double check)
- improve readability of `nextUpdateCheck` condition
- Update check module: refactor `updateCheckResponse`
- remove db call to displayUpdateNotification, not used anymore
- support receiving multiple custom notifications
- support custom notification groups
- the default group is `all` - this will always be consumed
- groups can be extended via config e.g. `notificationGroups: ['migration']`
- Update check module: refactor createCustomNotification helper
- get rid of taking over notification duplication handling (this is not the task of the update check module)
- ensure we have good fallback values for non present attributes in a notification
- get rid of semver check (happens in the USC) - could be reconsidered later if LTS is gone
- Refactor notification API
- reason: get rid of in process notification store
-> this was an object hold in process
-> everything get's lost after restart
-> not helpful anymore, because imagine the following case
-> you get a notification
-> you store it in process
-> you mark this notification as seen
-> you restart Ghost, you will receive the same notification on the next check again
-> because we are no longer have a separate seen notifications object
- use database settings key `notification` instead
- refactor all api endpoints to support reading and storing into the `notifications` object
- most important: notification deletion happens via a `seen` property (the notification get's physically deleted 3 month automatically)
-> we have to remember a seen property, because otherwise you don't know which notification was already received/seen
- Add listener to remove seen notifications automatically after 3 month
- i just decided for 3 month (we can decrease?)
- at the end it doesn't really matter, as long as the windows is not tooooo short
- listen on updates for the notifications settings
- check if notification was seen and is older than 3 month
- ignore release notification
- Updated our privacy document
- Updated docs.ghost.org for privacy config behaviour
- contains a migration script to remove old settings keys
closes#8717
- this is now required, because we run import queries sequentiell
- this code protects two cases:
- you have duplicate slugs in the JSON file (the first get's inserted, the second get's ignored)
- you have an existing slug in the database and you try to import the same slug, get's ignored
refs #9178
- i am not super happy about `const imageLib = require('../lib/image')`
- i don't really like the name `imageLib`
- but i had no better idea 😃
- if we use the same name in the whole project, it's very easy to rename the folder or the variable
no issue
> Deprecation warning: value provided is not in a recognized ISO format. moment construction falls back to js Date(), which is not reliable across all browsers and versions.
refs #9178
- they definitely don't belong to server/utils
- i think the best place is putting them into the card apps
- the the post model needs to ask the app for it's converters
- move tests as well
refs #9178
- continue with killing our global utils folder
- i haven't found any better naming for lib/promise
- so, require single files for now
- instead of doing `promiseLib = require('../lib/promise')`
- we can optimise the requires later
refs #9178
- this logic belongs to a static model helper
- the visibility property is a model property, the knowledge about the visibility values belongs to the model
- rename the functions, so they make more sense
refs #9178
- we have to take care that we don't end up in circular dependencies
- e.g. API requires UrlService and UrlService needs to require the API (for requesting data)
- update the references
- we would like to get rid of the utils folder, this is/was the most complicated change
no issue
- adapt major changes of knex-migrator v3
- adapt migration scripts, simplify and add `down` (rollback) hook if possible
- clear Ghost cache after init hook (because of `knex-migrator migrate --init`)
- ensure db migrations work with the CLI
- updated troubleshooting guide (https://docs.ghost.org/v1/docs/troubleshooting#section-task-execute-is-not-a-function)
**For development only: Please ensure you run `npm i -g knex-migrator@latest` to update your global installation to v3. We always prefer the local installation, but v3 has modified and added binaries.**
no issue
Support for http://resthooks.org style webhooks that can be used with Zapier triggers. This can currently be used in two ways:
a) adding a webhook record to the DB manually
b) using the API with password auth and POSTing to /webhooks/ (this is private API so not documented)
⚠️ only _https_ URLs are supported in the webhook `target_url` field 🚨
- add `webhooks` table to store event names and target urls
- add `POST` and `DELETE` endpoints for `/webhooks/`
- configure `subscribers.added` and `subscribers.deleted` events to trigger registered webhooks
no issue
- added https://github.com/TryGhost/bookshelf-relations as dependency
- remove existing tag handling
---
* Important: Ensure we trigger parent initialize function
- otherwise the plugin is unable to listen on model events
- important: event order for listeners is Ghost -> Plugin
- Ghost should be able to listen on the events as first instance
- e.g. be able to modify/validate relationships
* Fix tag validation
- we detect lower/update case slugs for tags manually
- this can't be taken over from the plugin obviously
- ensure we update the target model e.g. this.set('tags', ...)
* override base fn: `permittedAttributes`
- ensure we call the base
- put relations on top
- each relation is allowed to be passed
- the plugin will auto-unset any relations to it does not reach the database
* Ensure we run add/edit/delete within a transaction
- updating nested relationships requires sql queries
- all sql statements have to run in a single transaction to ensure we rollback everything if an error occurs
- use es6
closes#8143
Fixed a potential issue (edge-case), where our generated and validated (in terms of check for existance and add a counter) would return a slug, that will exceed the maximum length of the slug fields (191 chars).
This is mostly possible for the post title, which can be 255 chars long and would generate a slug with the same length. This would prevent the user from actually saving a post.
I tried first to determine the expected length for a slug that already exists, but decided that the **easier** and simplyfied implementation is to always cut a slug to **185 chars** (+ counter). This makes it easier to find duplicates and includes a possible high number of counts (edge-edge-case).
The slug will not be cut down to 185 chars if it's an import.
refs #9178
* Add eslint deps, remove old lint deps
* Add eslint config, remove old lint configs
* Config for server and tests are different
* Tweaked rules to suit us
* Fix linting in codebase - lots of indent changes.
* Fix a real broken test
refs #9192
- Each setting is saved individually
- Update this to only happen on import, or when a value changes
- Reduces the amount of work Ghost does on every setting change
refs #9150
- Moves the password length fn from `models/user` to `data/validation` where the other validator functions live.
- Added password validation rules. Password rules added:
- Disallow obviously bad passwords: '1234567890', 'qwertyuiop', 'asdfghjkl;' and 'asdfghjklm' for example
- Disallow passwords that contain the words 'password' or 'ghost'
- Disallow passwords that match the user's email address
- Disallow passwords that match the blog domain or blog title
- Disallow passwords that include 50% or more of the same characters: 'aaaaaaaaaa', '1111111111' and 'ababababab' for example.
- Password validation returns an `Object` now, that includes an `isValid` and `message` property to differentiate between the two error messages (password too short or password insecure).
- Use a catch predicate in `api/authentication` on `passwordReset`, so the correct `ValidationError` will be thrown during the password reset flow rather then an `UnauthorizedError`.
- When in setup flow, the blog title is not available yet from `settingsCache`. We therefore supply it from the received form data in the user model `setup` method to have it accessible for the validation.
refs #9150
- move data manipulation for importing users from `importers/data/users` to `model/user` for more consistency (see behaviour of post imports)
- changed importing logic in `onSaving` fn for user model:
- when importing, we set the password to a random uid and don't validate, just hash it and lock the user
- when importing with `importPersistUser` we check if the password is a bcrypt hash already and fall back to normal behaviour if not (set random password, lock user, and hash password)
- don't run validations when importing
closes#8668, refs #8920
- Updated tests to include internal tags
- Tests had no example of an internal tag
- Need this to show that the new filtering works as expected
- primary_tag is a calculated field
- This ensures that we can alias the field to equivalent logic in API filters
- By replacing primary_tag by a lookup based on a tag which has order 0
- bump ghost-gql to 0.0.8
**NOTE:**
Until GQL is refactored, there are limitations on what else can be filtered when using primary_tag in a filter e.g. it wont be possible to do a filter based on primary_tag AND/OR other tag filters.
refs #9001
When a blog is in private mode there is now an unguessable URL that allows access to the RSS feed for internal use, commenting systems, etc.
- add public hash for private blogging
- auto generate on bootstrap if missing
- global hash, we can re-use in the future
- update private blogging middleware to detect the private RSS URL and rewrite it so that the normal rss route/code is used for display
- if a normal `/rss/` route is accessed with a private session return a 404
closes#9089
- use the current date any time a post is fetched if the database contains an invalid date
- raise an error any time an attempt is made to save an invalidate date via the API
closes https://github.com/TryGhost/Ghost/issues/8943
- if you send a tag name with a hash, it's an internal tag
- ensure that the visibility property is forced to `internal`
- add a proper test
no issue
* Comment current state of toJSON for user model
- currently the user model does not return the email if the context is app/external/public OR if there is no context object at all
- i am not 100% sure why if there is no context we should not return the email address
- i think no context means internal access
- maybe change this condition cc @ErisDS
* Extend our access rules plugin
- we already have a instance method to determine which context is used
- this relies on passing options into `.forge` - but we almost never pass the context into the forge call
- added @TODO
- provide another static method to determine the context based on the options object passed from outside
* Use the new static function for existing code
* Add comment where the external context is used
* Remove certain fields from a public request (User model only)
* Tests: support `checkResponse` for a public request
- start with an optional option pattern
- i would love to get rid of checkResponse('user', null, null, null)
- still support old style for now
- a resoure can define the default response fields and public response fields
* Tests: adapt public api test
* Tests: adapt api user test
- use new option pattern for `checkResponse`
- eww null, null, null, null....
* Revert the usage of the access rules plugin
no issue
- it's not allowed to change/add these attributes via the API
- created_at = is only once set on adding the resource
- created_by = is only once set on adding the resource
- updated_by = is set on the server side when updating the model (based on who is logged in)
- updated_at = is set on the server side when updating the model
* Revert the usage of the access rules plugin
no issue
- the logic here bypasses filtering options!
- that is wrong, because if we filter out certain options e.g. include
- the tests from the previous commit fail because of this
- if we don't fix this logic, the tests won't pass, because as said, you can bypass certain logic e.g. remove roles from include
- this has worked before, because we passed the wrong options via the API layer
- was introduced here 014e2c88dd, because of https://github.com/TryGhost/Ghost/pull/6122
- add proper tests to proof that these queries work!!
🐛 Fixed author role permission to change author
no issue
- To be able to fix this bug, we had to solve tasks from #9043
- This bug affects the private / undocumented API only
- Author role users should not be allowed to change the author of a post
no issue
- we store dates without milliseconds in the database
- our test environment does not use our model layer to insert data, this is related to https://github.com/TryGhost/Ghost/issues/7196
- so it can happen that the test env inserts unix timestamps instead of a formatted string
- e.g. adding data via the model layer (e.g. via the API) the format is always normalised to `YYYY-MM-DD HH:mm:ss`
- if we fetch the date from the database, we have a hook which sorts out knex returning different formats for dates
- this hook wraps the returned date into a UTC moment date, but adds the current milliseconds on top
- which can collide in tests when you have specific assertions
- use `startOf` to ignore milliseconds
- furthermore: remove the mentionings of `pg` (postgres)
refs #8602
- Add the wiring to pass attributes around the permission system
- Allows us to get access to the important "unsafe" attributes that are changing
- E.g. status for posts
- This can then be used to determine whether a user has permission to perform an attribute-based action
- E.g. publish a post (change status)
no issue
- this bug fix affects all endpoints for the public user access
- we allowed fetching `roles` via the public api by accident
- see our docs: https://api.ghost.org/docs/users)
- we only allow `count.posts`
- returning roles via the public api exposes too many details
- this was never intentional
no issue
- this bug fix affects all endpoints for the public user access
- we allowed fetching `roles` via the public api by accident
- see our docs: https://api.ghost.org/docs/users)
- we only allow `count.posts`
- returning roles via the public api exposes too many details
- this was never attentional
no issue
- while i was testing random failures, i discovered an edge case for disqus
- you start a new 1.0 blog, you add disqus, the unique identifer is the post id (object id)
- now you export your content and import it on a new instance
- the importer detects that the amp field is null and imports the old object id as comment id
- but the post model is not prepared for this case
- see next commit for tests
**NOTE**: The comment id had two different data types (Number or String). Disqus expects a string. So this should not change any behaviour, now that the comment_id is always a string.
closes#8757
- update the markdown card render method to use SimpleDOM's `createRawHtmlSection`. This avoids SimpleDOM parsing and tokenization of broken or unsupported free-form HTML that markdown allows
- replace markdown extraction/render with mobiledoc's renderer in the `Post` model
- removes `jsdom` as it's no longer necessary
fixes#8920
- Implements logic such that internal tags cannot be primary tags
- If the first tag on a post is an internal tag, that post will not have a primary tag
no issue
- adds a ghost-backup client
- adds a client authenticated endpoint to export blog for ghost-backup client only
- allows some additional overrides during import
- allows for an import by file to override locking a user and double hashing the password
fixes#8845
- We had a report of weird URLS being output in admin stories view
- This is due to plaintext being incorrectly generated
- In order for a URL to be correct, it would need to already contain the subdirectory
- This line in the post model adds it as well, causing a duplicate
- Hence removing this line is the fix
fixes#8898
- This is a user error, not a system error
- Downgrading to a 4xx status code means it doesn't appear in logs where it shouldn't
- We didn't have a suitable error available so I added UpdateCollisionError with 409 status
closes#8781
- when the ownership get's transferred, the id of the new owner is not '1' anymore
- we previously added a database rule, which signalises if the blog is setup or not, see 827aa15757 (diff-7a2fe80302d7d6bf67f97cdccef1f71fR542)
- this database rule is based on the owner id being '1', which is wrong when you transfer ownership
- we should keep in mind, that the owner id being '1' is only the default Ghost setup, but it can change
- blog is setup if the owner is locked
refs https://github.com/TryGhost/Ghost/issues/8757
- remove mobiledoc parsing, it's reliance on SimpleDom makes it too
fragile when dealing with the unconstrained user-entered HTML that is
allowed in markdown
closes#8760
- we have to remember the old post id's when migrating a blog from LTS to 1.0
- otherwise we would break disqus comments, because they rely on the post id
- this should fix the discovered situation
no issue
- if you upload a huge import file, parallel operations can throw errors e.g. lock wait exceeds
- this can happen if multiple transactions run in parallel
- there is no need to run:
1. the removal of active tokens on import, because imported users have no active session
2. rescheduling logic on timezone, because importing scheduled posts works out of the box via the model layer (if a published date is detected and it's in the future, the post get's scheduled)
closes#8645, closes#8710
- locked users were once part of the category "active users", but were moved to the inactive category
-> we have added a protection of not being able to edit yourself when you are either suspended or locked
- but they are not really active users, they are restricted, because they have no access to the admin panel
- support three categories: active, inactive, restricted
* - revert restricted states
- instead, update permission layer: fallback to `all` by default, because you are able to serve any user status
- add more tests
- ATTENTION: there is a behaviour change, that a blog owner's author page can be served before setting up the blog, see conversation on slack
-> LTS serves 404
-> 1.0 would serve 200
no issue
- if you delete an active user, Ghost logs an error message (Ghost does not crash!)
- but the event logic is not triggered, that means we don't delete the users tokens
- token deletion happens on: suspend a user and delete a user
no issue
- if you destroy a user with an unknown user id, Ghost would crash
- because `userModel.hasRole` is undefined
- there is actually a bigger underlying architectual problem:
- the permission check should rely on an existing user
- so there should be a first api layer, which 1. validates (this code exists) and 2. ensures that requested database id's exist
- but this requires a bigger refactoring
refs #8620
Adds a new Ghost Author user, which is the author of the new welcome blog posts. The user is set to active, so the author slug works (otherwise it would render a 404, when user is suspended). Furthermore, there's one little fix in the user model, which was checking only for `active` user to decide the signup or setup process for the UI. Adding one more conditional to check if the found active user is also the owner, prevents to get redirected to sign in.
closes#8565
- isPasswordCorrect fn returns a specific error, which we simply forward
- no need to wrap a custom error into a new custom error
- the rule is always: if you are using a Ghost unit/function, you can expect that this unit returns a custom error
closes#8479
- removes `markdown` field from schema
- removes `legacyMarkdown` converter
- updates tests to work with `mobiledoc` field instead of `markdown` and adapt for mobiledoc HTML output where necessary
refs #5422
- we can support null titles after this PR if we want
- user model: fix getAuthorRole
- user model: support adding roles by name
- we support this for roles as well, this makes it easier when importing related user roles (because usually roles already exists in the database and the related id's are wrong e.g. roles_users)
- base model: support for null created_at or updated_at values
- post or tag slugs are always safe strings
- enable an import of a null slug, no need to crash or to cover this on import layer
- add new DataImporter logic
- uses a class inheritance mechanism to achieve an easier readability and maintenance
- schema validation (happens on model layer) was ignored
- allow to import unknown user id's (see https://github.com/TryGhost/Ghost/issues/8365)
- most of the duplication handling happens on model layer (we can use the power of unique fields and errors from the database)
- the import is splitted into three steps:
- beforeImport
--> prepares the data to import, sorts out relations (roles, tags), detects fields (for LTS)
- doImport
--> does the actual import
- afterImport
--> updates the data after successful import e.g. update all user reference fields e.g. published_by (compares the imported data with the current state of the database)
- import images: markdown can be null
- show error message when json handler can't parse file
- do not request gravatar if email is null
- return problems/warnings after successful import
- optimise warnings in importer
- do not return warnings for role duplications, no helpful information
- error handler: return context information of error
- we show the affected json entries as one line in the UI
- show warning for: detected duplicated tag
- schema validation: fix valueMustBeBoolean translation
- remove context property from json parse error
refs https://github.com/TryGhost/Ghost-Admin/pull/690, closes#1501, closes#2093, closes#4592, closes#4627, closes#4659, closes#5039, closes#5237, closes#5587, closes#5625, closes#5632, closes#5822, closes#5939, closes#6840, closes#7183, closes#7536
- replace custom showdown fork with markdown-it
- swaps showdown for markdown-it when rendering markdown
- match existing header ID behaviour
- allow headers without a space after the #s
- add duplicate header ID handling
- remove legacy markdown spec
- move markdown-it setup into markdown-converter util
- update mobiledoc specs to match markdown-it newline behaviour
- update data-generator HTML to match markdown-it newline behaviour
- fix Post "converts html to plaintext" test
- update rss spec to match markdown-it newline behaviour
- close almost all related showdown bugs
closes#8354
- i thought about transforming scheduled posts into drafts on export, but this has two disadvantages:
1. existing exports with scheduled posts won't import
2. if you schedule a post for next week and you export/import earlier, the post is back to draft
- by this we ensure that we can simply import the post back to a scheduled post
- if the published_at is already in the past, the scheduler will care and instantly publish the post
closes#8426
- if you import posts with updated_at=null, you are not able to save this post anymore
- i am not sure how this is even possible, but maybe there is a case where updated_at can be null
* 🙀 change database schema for images
- rename user/post/tag images
- contains all the required changes from the schema change
* Refactor helper/meta data
- rename cover to cover_image
- also rename default settings to match the pattern
- rename image to profile_image for user
- rename image to feature_image for tags/posts
* {{image}} >>> {{img_url}}
- rename
- change the functionality
- attr is required
- e.g. {{img_url feature_image}}
* gscan 1.0.0
- update yarn.lock
* Update casper reference: 1.0-changes
- see 5487b4da8d
closes#5599
If two users edit the same post, it can happen that they override each others content or post settings. With this change this won't happen anymore.
✨ Update collision for posts
- add a new bookshelf plugin to detect these changes
- use the `changed` object of bookshelf -> we don't have to create our own diff
- compare client and server updated_at field
- run editing posts in a transaction (see comments in code base)
🙀 update collision for tags
- `updateTags` for adding posts on `onCreated` - happens after the post was inserted
--> it's "okay" to attach the tags afterwards on insert
--> there is no need to add collision for inserting data
--> it's very hard to move the updateTags call to `onCreating`, because the `updateTags` function queries the database to look up the affected post
- `updateTags` while editing posts on `onSaving` - all operations run in a transactions and are rolled back if something get's rejected
- Post model edit: if we push a transaction from outside, take this one
✨ introduce options.forUpdate
- if two queries happening in a transaction we have to signalise knex/mysql that we select for an update
- otherwise the following case happens:
>> you fetch posts for an update
>> a user requests comes in and updates the post (e.g. sets title to "X")
>> you update the fetched posts, title would get overriden to the old one
use options.forUpdate and protect internal post updates: model listeners
- use a transaction for listener updates
- signalise forUpdate
- write a complex test
use options.forUpdate and protect internal post updates: scheduling
- publish endpoint runs in a transaction
- add complex test
- @TODO: right now scheduling api uses posts api, therefor we had to extend the options for api's
>> allowed to pass transactions through it
>> but these are only allowed if defined from outside {opts: [...]}
>> so i think this is fine and not dirty
>> will wait for opinions
>> alternatively we have to re-write the scheduling endpoint to use the models directly
no issue
- client dates are sent as ISO format (moment(..).format())
- server dates are in JS Date format
>> when bookshelf fetches data from the database, all dates are transformed into JS dates
>> see `parse` helper function
- Bookshelf updates the model with the client data via Bookshelf's `set` function
- therefor Bookshelf uses a simple `isEqual` function from lodash to detect changes
- .previous(attr) and .get(attr) return false
- that has the concequence that dates are always marked as "changed"
- internally we use our `hasDateChanged` if we have to compare previous/updated dates
- but Bookshelf is not in our control for this case
no issue
- the UTC offset diff of the current and previous timezone must switch
- i have added more tests and more example case descriptions to understand why
refs #8275
- If the HTML field has changed, update the plaintext field
- Use html-to-text to generate a plaintext version of the HTML which retains some structure
- Add a couple of tests - although there's much to do here!
no issue
- i don't know if this never worked or has worked and something changed in bookshelf
- but this fixes: saving the content (no change for published_at) of a scheduled post within the 2minutes window
- add `beforeWrite` option to hasDateChanged helper, see comment
- use previous for `beforeWrite` operations
- add a test and fix some other small issues in the scheduler tests
refs #8258
* 🎨 change last_login to last_seen
- rename the column
- a change in Ghost-Admin is required as well
* test utils: revert export examples
* revert line breaks
refs #7429
- ☢️👷🏻♀️ This PR removes the dependency on Ghost-Editor and replaces it with the Mobiledoc DOM renderer. It includes new DOM based default cards and atoms.
refs #8111
- Ghost returns now all (active+none active) users by default
- protect login with suspended status
- test permissions and add extra protection for suspending myself
- if a user is suspended and tries to activate himself, he won't be able to proceed the login to get a new token
refs #8093✨ Add activate theme permission
- add permission to activate themes
- update tests
- also: update tests for invites
TODO: change how the active theme setting is updated to reduce extra permissions
✨ Move theme validation to gscan
- add a new gscan validation method and use it for upload
- update activate endpoint to do validation also using gscan
- change to using SettingsModel instead of API so that we don't call validation or permissions on the settings API
- remove validation from the settings model
- remove the old validation function
- add new invalid theme message to translations & remove a bunch of theme validation related unused keys
📖 Planned changes
🚨 Tests for theme activation API endpoint
🐛 Don't allow deleting the active theme
🚫 Prevent activeTheme being set via settings API
- We want to control how this happens in future.
- We still want to store the information in settings, via the model.
- We just don't want to be able to change this info via the settings edit endpoint
🐛✨ Fix warnings for uploads & add for activations
- warnings for uploads were broken in f8b498d
- fix the response + adds tests to cover that warnings are correctly returned
- add the same response to activations + more tests
- activations now return a single theme object - the theme that was activated + any warnings
🎨 Improve how we generate theme API responses
- remove the requirement to pass in the active theme!
- move this to a specialist function, away from the list
🎨 Do not load gscan on boot
no issue
🔥 Remove unnecessary cache update
🎨 simplify updateSettingsCache()
🎨 Simplify readSettingsResult
- although this is more code, it's now much clearer what happens in the two cases
🎨 Don't use readSettingResult for edit
🎨 Simplify updateSettingsCache further
🔥 Remove now unused readSettingsResult
🎨 Change populateDefault to return all
🎨 Move the findAll call out of updateSettingsCache
🔥 Remove updateSettingsCache!!
🎨 Restructure init & finish up settingsCache
- move initialisation into settingsCache.init AT LAST
- change settingCache to use cloneDeep, so that the object can't be modified outside of the functions
- add lots of docs to settings cache
🎨 Cleanup db api endpoints
🔥 Don't populate settings in migrations
* 🎨 deny auto switch
no issue
- deny auth switch after the blog was setup
- setup completed depends on the status of the user right now, see comments
* Updates from comments
- re-use statuses in user model
- update error message
no issue
🔥 remove unused loadThemes API method
🚨 Add tests for themes.readOne
🔥 Don't update settings cache for imports
- this isn't needed as of #8057
- settings.edit fires an event, that will result in the update happening automatically
🎨 Move validation to themes
- slowly collecting all theme-related code together
🔥 Reduce DEBUG output
- all this info is a bit tooooo much!
closes#8037🔥 Remove API-level default settings population
- This is a relic!
- We ALWAYS populate defaults on server start therefore this code could never run.
- This was a lot of complicated code that wasn't even needed!!
🎨 Move settings cache
- Move settings cache to be its own thing
- Update all references
- Adds TODOs for further cleanup
🎨 Create settings initialisation step
- Create new settings library, which will eventually house more code
- Unify the interface for initialising settings (will be more useful later)
- Reduce number of calls to updateSettingsCache
* ✨ ghost auth: sync email
refs #7452
- sync email changes in background (every hour right now)
- sync logged in user only!
- no sync if auth strategy password is used
- GET /users/me is triggered on every page refresh
- added TODO to support or add long polling for syncing data later
- no tests yet on purpose, as i would like to get a basic review first
* 🐩 use events
- remember sync per user
refs #2182
* 🔥 Remove unused options from server init
- this is left over from old code and is now unused
* 🎨 Move knex-migrator check to db health
- Move complex check function into own module
- Call module from server/index.js
- This just improves the readability of server/index.js
* 🔥 Remove old comments
- These comments all make no sense now!
* 🎨 ⏱ Move model init out of promise chain
- Model.init() does not return a promise
- Therefore, we can move it to the top of the init function, outside of the promise change
- This should be a minor optimisation, and again improves readability /clarity of what's happening
* ✨⁉️ Move DBHash init / first run to Settings model
- this structure is left over from when we had code we executed on the first run of Ghost
- the implementation used the API to initialise one setting before populateDefaults is called
- this had lots of dependencies - the whole model, API, and permissions structure had to be initialised for it to work
- the new implementation is simpler, it captures the dbHash getting initialised during populateDefaults()
- it also adds an event, so we can do first-run code later if we really want to (or maybe apps can?!)
- perhaps this is hiding behaviour, and there's a nicer way to do it, but populateDefaults seems like a sane place to populate a default setting 😁
* ⏱ Optimise require order so config is first
- the first require to config will cause the files to be read etc
- this ensures that it happens early, and isn't confusingly timed as part of loading a different module
* 🎨 Simplify settings model changes
closes#7256
- original code changes made by @golya in https://github.com/TryGhost/Ghost/pull/7304
- refactored edit method in user model to validate an existing email address
- added test coverage for existing email update in user model spec
refs #7489
- as we are now using a different migration approach (knex-migrator), we don't need to remember the database version anymore
- it was once used to check the state of a database and based on it we decided to migrate or not
- with knex-migrator everything depends on the migration table entries and the current ghost version you are on
- on current master the leftover usage is to add the db version when exporting the database, which can be replaced by reading the ghost version
- removing this solves also an interesting migration case with knex-migrator:
- you are on 1.0
- you update to 1.1, but 1.1 has no migrations
- the db version would remain in 1.0
- because the db version was only updated when knex migrator executed a migration
* 🛠 bookshelf tarball, bson-objectid
* 🎨 schema changes
- change increment type to string
- add a default fallback for string length 191 (to avoid adding this logic to every single column which uses an ID)
- remove uuid, because ID now represents a global resource identifier
- keep uuid for post, because we are using this as preview id
- keep uuid for clients for now - we are using this param for Ghost-Auth
* ✨ base model: generate ObjectId on creating event
- each new resource get's a auto generate ObjectId
- this logic won't work for attached models, this commit comes later
* 🎨 centralised attach method
When attaching models there are two things important two know
1. To be able to attach an ObjectId, we need to register the `onCreating` event the fetched model!This is caused by the Bookshelf design in general. On this target model we are attaching the new model.
2. We need to manually fetch the target model, because Bookshelf has a weird behaviour (which is known as a bug, see see https://github.com/tgriesser/bookshelf/issues/629). The most important property when attaching a model is `parentFk`, which is the foreign key. This can be null when fetching the model with the option `withRelated`. To ensure quality and consistency, the custom attach wrapper always fetches the target model manual. By fetching the target model (again) is a little performance decrease, but it also has advantages: we can register the event, and directly unregister the event again. So very clean code.
Important: please only use the custom attach wrapper in the future.
* 🎨 token model had overriden the onCreating function because of the created_at field
- we need to ensure that the base onCreating hook get's triggered for ALL models
- if not, they don't get an ObjectId assigned
- in this case: be smart and check if the target model has a created_at field
* 🎨 we don't have a uuid field anymore, remove the usages
- no default uuid creation in models
- i am pretty sure we have some more definitions in our tests (for example in the export json files), but that is too much work to delete them all
* 🎨 do not parse ID to Number
- we had various occurances of parsing all ID's to numbers
- we don't need this behaviour anymore
- ID is string
- i will adapt the ID validation in the next commit
* 🎨 change ID regex for validation
- we only allow: ID as ObjectId, ID as 1 and ID as me
- we need to keep ID 1, because our whole software relies on ID 1 (permissions etc)
* 🎨 owner fixture
- roles: [4] does not work anymore
- 4 means -> static id 4
- this worked in an auto increment system (not even in a system with distributed writes)
- with ObjectId we generate each ID automatically (for static and dynamic resources)
- it is possible to define all id's for static resources still, but that means we need to know which ID is already used and for consistency we have to define ObjectId's for these static resources
- so no static id's anymore, except of: id 1 for owner and id 0 for external usage (because this is required from our permission system)
- NOTE: please read through the comment in the user model
* 🎨 tests: DataGenerator and test utils
First of all: we need to ensure using ObjectId's in the tests. When don't, we can't ensure that ObjectId's work properly.
This commit brings lot's of dynamic into all the static defined id's.
In one of the next commits, i will adapt all the tests.
* 🚨 remove counter in Notification API
- no need to add a counter
- we simply generate ObjectId's (they are auto incremental as well)
- our id validator does only allow ObjectId as id,1 and me
* 🎨 extend contextUser in Base Model
- remove isNumber check, because id's are no longer numbers, except of id 0/1
- use existing isExternalUser
- support id 0/1 as string or number
* ✨ Ghost Owner has id 1
- ensure we define this id in the fixtures.json
- doesn't matter if number or string
* 🎨 functional tests adaptions
- use dynamic id's
* 🎨 fix unit tests
* 🎨 integration tests adaptions
* 🎨 change importer utils
- all our export examples (test/fixtures/exports) contain id's as numbers
- fact: but we ignore them anyway when inserting into the database, see https://github.com/TryGhost/Ghost/blob/master/core/server/data/import/utils.js#L249
- in 0e6ed957cd (diff-70f514a06347c048648be464819503c4L67) i removed parsing id's to integers
- i realised that this ^ check just existed, because the userIdToMap was an object key and object keys are always strings!
- i think this logic is a little bit complicated, but i don't want to refactor this now
- this commit ensures when trying to find the user, the id comparison works again
- i've added more documentation to understand this logic ;)
- plus i renamed an attribute to improve readability
* 🎨 Data-Generator: add more defaults to createUser
- if i use the function DataGenerator.forKnex.createUser i would like to get a full set of defaults
* 🎨 test utils: change/extend function set for functional tests
- functional tests work a bit different
- they boot Ghost and seed the database
- some functional tests have mis-used the test setup
- the test setup needs two sections: integration/unit and functional tests
- any functional test is allowed to either add more data or change data in the existing Ghost db
- but what it should not do is: add test fixtures like roles or users from our DataGenerator and cross fingers it will work
- this commit adds a clean method for functional tests to add extra users
* 🎨 functional tests adaptions
- use last commit to insert users for functional tests clean
- tidy up usage of testUtils.setup or testUtils.doAuth
* 🐛 test utils: reset database before init
- ensure we don't have any left data from other tests in the database when starting ghost
* 🐛 fix test (unrelated to this PR)
- fixes a random failure
- return statement was missing
* 🎨 make changes for invites
* 🎨 schema change
- simply role_id attribute
* 🎨 update invite model
- remove all methods we don't need
- ensure we remove the relation from the model
- ensure we do not allow to call withRelated
* 🎨 adapt api changes
* 🎨 adapt auth module
* 🎨 adapt tests
* 🎨 better error handling
* schema update
refs #7494, refs #7495
This PR is an extracted clean up feature of #7495.
We are using everywhere static id checks (userId === 0 or userId === 1).
This PR moves the static values into the Base model.
This makes it 1. way more readable and 2. we can change the id's in a central place.
I changed the most important occurrences - no tests are touched (yet!).
The background is: when changing from auto increment id (number) to ObjectId's (string) we still need to support id 1 and 0, because Ghost relies on these two static id's.
I would like to support using both: 0/1 as string and 0/1 as number.
1 === owner/internal
0 === external
Another important change:
User Model does not longer define the contextUser method, because i couldn't find a reason?
I looked in Git history, see 6e48275160
no issue
- removes count from user checks model
- uses brute express brute with brute-knex adaptor to store persisted data on spam prevention
- implement brute force protection for password/token exchange, password resets and private blogging
* 🔥 remove User model functions
- validateToken
- generateToken
- resetPassword
- all this logic will re-appear in a different way
Token logic:
- was already extracted as separate PR, see https://github.com/TryGhost/Ghost/pull/7554
- we will use this logic in the controller, you will see in the next commits
Reset Password:
Was just a wrapper for calling the token logic and change the password.
We can reconsider keeping the function to call: changePassword and activate the status of the user - but i think it's fine to trigger these two actions from the controlling unit.
* 🔥 remove password reset tests from User model
- we already have unit tests for change password and the token logic
- i will re-check at the end if any test case is missing - but for now i will just burn the tests
* ✨ add token logic to controlling unit
generateResetToken endpoint
- the only change here is instead of calling the User model to generate a token, we generate the token via utils
- we fetch the user by email, and generate a hash and return
resetPassword endpoint
- here we have changed a little bit more
- first of all: we have added the validation check if the new passwords match
- a new helper method to extract the token informations
- the brute force security check, which can be handled later from the new bruteforce middleware (see TODO)
- the actual reset function is doing the steps: load me the user, compare the token, change the password and activate the user
- we can think of wrapping these steps into a User model function
- i was not sure about it, because it is actually part of the controlling unit
[ci skip]
* 🎨 tidy up
- jscs
- jshint
- naming functions
- fixes
* ✨ add a test for resetting the password
- there was none
- added a test to reset the password
* 🎨 add more token tests
- ensure quality
- ensure logic we had
* 🔥 remove compare new password check from User Model
- this part of controlling unit
* ✨ compare new passwords for user endpoint
- we deleted the logic in User Model
- we are adding the logic to controlling unit
* 🐛 spam prevention forgotten can crash
- no validation happend before this middleware
- it just assumes that the root key is present
- when we work on our API, we need to ensure that
1. pre validation happens
2. we call middlewares
3. ...
* 🎨 token translation key
* 🎨 do not call generateSlug twice for User.setup
* 🎨 call generatePasswordHash onSaving only
- now we can add defaults to User Model
- it was not possible before because add User model did the following:
1. validate password length
2. hash password manually
3. call ghostBookshelf.Model.add and THEN bookshelf defaults fn gets triggered
- call generatePasswordHash in onSaving hook for all use case
- add more tests to user model, juhu
refs #7432
- all models implemented it's own initialize fn to register events
- we can register all events in the base model
- important: we only listen on the event, if the model has defined a hook for it
- this is just a small clean up PR
- register more bookshelf events
no issue
- preperation for User model refactoring
- the rule is:
--> when calling a unit, this unit should return something new
--> and NOT modifying an existing object and return it (this is an unexpected behaviour, especially for utils and libs)
* 🎨 move heart of fixtures to schema folder and change user model
- add fixtures.json to schema folder
- add fixture utils to schema folder
- keep all the logic!
--> FIXTURE.JSON
- add owner user with roles
--> USER MODEL
- add password as default
- findAll: allow querying inactive users when internal context (defaultFilters)
- findOne: do not remove values from original object!
- add: do not remove values from original object!
* 🔥 remove migrations key from default_settings.json
- this was a temporary invention for an older migration script
- sephiroth keep alls needed information in a migration collection
* 🔥 add code property to errors
- add code property to errors
- IMPORTANT: please share your opinion about that
- this is a copy paste behaviour of how node is doing that (errno, code etc.)
- so code specifies a GhostError
* 🎨 change error handling in versioning
- no need to throw specific database errors anymore (this was just a temporary solution)
- now: we are throwing real DatabaseVersionErrors
- specified by a code
- background: the versioning unit has not idea about seeding and population of the database
- it just throws what it knows --> database version does not exist or settings table does not exist
* 🎨 sephiroth optimisations
- added getPath function to get the path to init scripts and migration scripts
- migrationPath is still hardcoded (see TODO)
- tidy up database naming to transacting
* ✨ migration init scripts are now complete
- 1. add tables
- 2. add fixtures
- 3. add default settings
* 🎨 important: make bootup script smaller!
- remove all TODO'S except of one
- no seeding logic in bootup script anymore 🕵🏻
* ✨ sephiroth: allow params for init command
- param: skip (do not run this script)
- param: only (only run this script)
- very simple way
* 🎨 adapt tests and test env
- do not use migrate.populate anymore
- use sephiroth instead
- jscs/jshint
* 🎨 fix User model status checks
closes#6629
- i had the case that in gravatar process.env.NODE_ENV was undefined and indexOf of undefined crashe my application
- so always use config to read current env
closes#6165
- internal tags has been in labs for a couple of months, we've fixed some bugs & are ready to ship
- removes all code that tests for the labs flag
- also refactors the various usage of the visibility filter into a single util
- all the tests still pass!!!
- this marks #6165 as closed because I think the remaining UI tasks will be handled as part of a larger piece of work
refs #7429
Finally it's starting to feel like a real editor, although there will be another version bump over the weekend which improves the toolbar behaviour and usability, and enables image uploading.
- Added the start of a new toolbar, what we're (well I am) calling the Owesome bar, not to be confused with the Firefox Awesome bar. It's a cultural thing. (google "O for awesome").
- The idea of dragging and dropping cards has been removed for now, although the code will still be in there as we will support dragging cards around fairly shortly. When apps are included a better card interface will be required for a larger amount of app created content cards (Oh yeah!)
- Ghost Server now pulls in it's configuration from Ghost-Editor, this allows Ghost-Editor to a) keep cards up to date, and b) define what happens if a card is missing.
- The whole cards in admin written in ember and cards in server written in javascript thing is still very much a work in progress, it's kind of messy as we find the optimum solution (which isn't the current sollution).
So yeah, this is a WIP not the final styling, not the final interactions, not the final anything... :)
Adds a new mobile doc editor which has:
- A new toolbar
- Basic image uploading capability
refs #7116, refs #2001
- Changes the way Ghost errors are implemented to benefit from proper inheritance
- Moves all error definitions into a single file
- Changes the error constructor to take an options object, rather than needing the arguments to be passed in the correct order.
- Provides a wrapper so that any errors that haven't already been converted to GhostErrors get converted before they are displayed.
Summary of changes:
* 🐛 set NODE_ENV in config handler
* ✨ add GhostError implementation (core/server/errors.js)
- register all errors in one file
- inheritance from GhostError
- option pattern
* 🔥 remove all error files
* ✨ wrap all errors into GhostError in case of HTTP
* 🎨 adaptions
- option pattern for errors
- use GhostError when needed
* 🎨 revert debug deletion and add TODO for error id's
- 🛠 add bunyan and prettyjson, remove morgan
- ✨ add logging module
- GhostLogger class that handles setup of bunyan
- PrettyStream for stdout
- ✨ config for logging
- @TODO: testing level fatal?
- ✨ log each request via GhostLogger (express middleware)
- @TODO: add errors to output
- 🔥 remove errors.updateActiveTheme
- we can read the value from config
- 🔥 remove 15 helper functions in core/server/errors/index.js
- all these functions get replaced by modules:
1. logging
2. error middleware handling for html/json
3. error creation (which will be part of PR #7477)
- ✨ add express error handler for html/json
- one true error handler for express responses
- contains still some TODO's, but they are not high priority for first implementation/integration
- this middleware only takes responsibility of either rendering html responses or return json error responses
- 🎨 use new express error handler in middleware/index
- 404 and 500 handling
- 🎨 return error instead of error message in permissions/index.js
- the rule for error handling should be: if you call a unit, this unit should return a custom Ghost error
- 🎨 wrap serve static module
- rule: if you call a module/unit, you should always wrap this error
- it's always the same rule
- so the caller never has to worry about what comes back
- it's always a clear error instance
- in this case: we return our notfounderror if serve static does not find the resource
- this avoid having checks everywhere
- 🎨 replace usages of errors/index.js functions and adapt tests
- use logging.error, logging.warn
- make tests green
- remove some usages of logging and throwing api errors -> because when a request is involved, logging happens automatically
- 🐛 return errorDetails to Ghost-Admin
- errorDetails is used for Theme error handling
- 🎨 use 500er error for theme is missing error in theme-handler
- 🎨 extend file rotation to 1w
This release includes:
Ghost Editor, this is required to get access to the built in cards
Ghost Editor includes:
Responsive toolbars 🔨🔧
Both Ember and Plain javascript cards 🎴
An embeded HTML card ✍️
This is still an early release, but things are moving in the right direction. :)
We're still defining the spec for the UI, so expect drastic changes over the next couple of weeks.
This is going to be a great writing experience and we can't wait to show what we have planned.
refs #7452
- remove references to 'patronus' in favour of GhostAuth, Note: this will require databases to be deleted ;)
- remove email addresses from test data
issue #7452
Remote oauth2 authentication with Ghost.org.
This PR supports:
- oauth2 login or local login
- authentication on blog setup
- authentication on invite
- normal authentication
- does not contain many, many tests, but we'll improve in the next alpha weeks
closes#6625
- "url" and "author" fields depend on {id, published_at, slug, author_id} to construct post url.
- implemented a generic solution by defining defaultColumnsToFetch() in
base class for models.
- findPage() calls defaultColumnsToFetch() before loading models
- results are transformed by filtering out additional properties to return just the requested fields
- Added a test case to check for url and author fields
- Renamed allColumns as requestedColumns and used _.map instead of Promise.map
closes#6932
- new default order of posts: scheduled, draft, published
- invent orderDefaultRaw fn for each model
- each model is able to create a default raw order query
- separate count and fetch query for fetchPage, because the count query where group/order statements attached
refs #6413
- PUT endpoint to publish a post/page for the scheduler
- fn endpoint to get all scheduled posts (with from/to query params) for the scheduler
- hardcoded permission handling for scheduler client
- fix event bug: unscheduled
- basic structure for scheduling
- post scheduling basics
- offer easy option to change adapter
- integrate the default scheduler adapter
- update scheduled posts when blog TZ changes
- safety check before scheduler can publish a post (not allowed to publish in the future or past)
- add force flag to allow publishing in the past
- invalidate cache header for /schedules/posts/:id
closes#6406
- created listeners.js connector
- merged listeners.js with events.js (in models/base)
- set a post to draft when published_at would be in the past
- reschedule a post when published_at would be in the future
fixes#6919, refs #6917
- resolves the problem by guarding for `tags` being undefined or null
- If it is undefined, we don't do any tag processing
- If it is null, we don't do any tag processing
- To delete all tags, you would provide `tags: []`
- adds tests at both the model (using withRelated) and API (using includes) layers
- moves the tests for the post updateTags functionality from the tag model spec to the post model spec & cleans up a bit
issues #6406#6399
- all dates are stored as UTC with this commit
- use moment.tz.setDefault('UTC')
- add migration file to recalculate local datetimes to UTC
- store all dates in same format into our three supported databases
- add option to remeber migrations inside settings (core)
- support DST offset for migration
- ensure we force UTC in test env
- run whole migration as transaction
- extend: Settings.findOne function
no issue
- add some more tests, optimise tests and finish tests
- subscriber model checks external context permissions in permissible fn
- add missing permissions for subscriber csv
closes#6462
- monkey-patch validator.extends() since it was dropped by validator @5.0.0
- coerce input to string prior to validation (custom toString func)
- need to handle boolean validation based on column type not isIn()
- use `lodash.tostring` to convert input values to strings
Closes#6625
- Adds a failing test for not returning computed columns as well
as for the bookshelf bug where extra columns passed into a fetch
will result in the model having an extra "quoted" column.
- Filter model attributes for passing into "fetch" but used the
entire list of columns for `toJSON`.
refs #6301, #4176
- always check existence of items before attempting to create them, in order to prevent duplicates
- provide stats on how many object creations are expected vs done
- split out and improve fixture utils tests (100% covers utils)
Closes#6620
* Changed it from always returning true, to evaluate if it is the
current logged in user, and if so, check the old password. If not,
ignore
- Simplify the `init` method in `models/index.js` so that it no longer
returns a promise. Easier to use.
- Eliminates the `deleteAllContent` method from `models/index.js` as it
can all be handled at the API layer in a single spot.
- Optimize `destroyAllContent` in `api/db.js`. Eliminates
double-fetching every post from the database and converting it to
JSON. Also only fetches ids from the database instead of the entire
model.
- Eliminates the custom static method `destroy` in the Post model in
favor of handling detaching tag relations in a single place (the
`destroying` event). This also eliminates a big source of unneeded
database round trips--needing to get post ids to feed into
`Post.destroy()` which then re-fetches the post again.
refs #6301
- Move secret generation logic to the model defaults, so there's no need to handle this in fixtures
- Tested upgrades from 003 & fresh installs -> all is well
no issue
- request is quite a heavy dependency
- we were only using request in 3 places: a test, storing contrib images in the gruntfile & the gravatar lookup
- all 3 are relatively simple to do with the http/https module
- refactored all 3, removed request
refs #6301
- In the migration folder, commands.js changed to builder.js to resolve conflict with the 'commands' inside data/utils/clients/.
- a new data/schema/ folder has been added to hold all the code related to the database schema
- data/utils/clients have been moved to data/schema/clients
- data/utils/index.js has become data/schema/commands.js
- data/schema.js has been split, the definition of the DB schema stays put, the additional checks have moved to data/schema/checks.js
- data/validation/index.js has become data/schema/versioning.js
- data/fixtures has moved to data/migration/fixtures
- data/default-settings.json has moved to data/schema/default-settings.json
no issue
- Cache the permalinks & postsPerPage settings on the config.theme object
- Use the config.theme cache to reference these items throughout the frontend of a blog
- Removes the need for workarounds and extra code to handle async fetches
- Makes these values accessible to all themes, which is very useful now we have the API stuff
refs #6122
- restore original options after delete
- this is a fix for one use case, long term we should aim to leave
options untouched and execute special queries with temporary data
refs #6009
- This is a straight rename, no functionality is added
- The dot syntax requires pre/post processing to convert the name
- This PR also includes several updates to the tests, as they weren't being run as part of Travis!
- pass debug: true to the API to get some useful debug output
- does not work in production mode
Note: I have added these lines back in so many times in the past month or so so that I could
figure out what was happening, I figured everyone else might find them useful.
TODO: use a proper logging method dependent on env
refs #5614, #5943
- adds a new 'filter' bookshelf plugin which extends the model
- the filter plugin provides handling for merging/combining various filters (enforced, defaults and custom/user-provided)
- the filter plugin also handles the calls to gql
- post processing is also moved to the plugin, to be further refactored/removed in future
- adds tests showing how filter could be abused prior to this commit
refs #5614
- change isPublicContext to detectPublicContext
- behaviour now expands the context object out
- this is a bit of a sideeffect, but this is the simplest change
that makes it possible to use the context in the model layer without
significant wider changes
- add new access rules plugin
- takes a context object as part of `forge()` & caches it on the model instance
- provides helper functions for testing access rules later on
refs #5943
- no longer assume the options in processOptions are set
- set where to a new GQL JSON-like statement object
- rather than setting options, add statements which can be understood by knexify
- pass the statements through knexify to build the query
refs #6009, #5615
- minimal refactor to remove the addition of count from pagination and other various points
- create a include count plugin that overrides fetch and fetchAll
- this ensures that counts get added at the right points
no issue
- switching from using fetch to fetch all means some code can be removed from the fetchPage method
- updating tests to reflect cleaner code
- ensure coverage is at 100%
refs #5602
- add "order" to default browse options
- parse order parameter in Base model
- accept "order" option in Post, User and Tag models
- add tests for posts order
- add tests for tags order
- add tests for users order
refs #5943
- removed featured, tag and author parameters from posts API
- featured was only used in tests
- removed role filter from users API
- role was only used in tests
- fixed up the tests, skipping those that don't quite work yet
refs #5604, refs #5463
- deps: ghost-gql@0.0.2
- adds code to wire up the filtering to a paginated query
- updated pagination plugin count query to use 'distinct' so it's more robust
- rename paginationUtils.query to addLimitAndOffset to be more explicit and make the code clearer
- add a new 'advanced browsing spec' set of tests for tracking these features as they are built out
closes#5492
- remove core/server/require-tree.js and split it into modules
- add read-directory module to recursively read directories
- add validate-themes module to scan themes and return errors/warnings
- add parse-package-json module to parse json and validate requirements
- rewrite core/server/models/index.js to manually require models
closes#5913
Sitemap deletion is based on the page.unpublished event. The previous
logic was always sending post.unpublished instead. If page or post
event is triggered is based on the ‘page’ attribute of the model. When
the destroyed handler all attributes are already cleared from the model
which makes this logic always fall back to post.
The fix is to move to the destroying event which still has all the
model values in place.
refs #5909, #4577
- removes accidental '.only' which was hiding issues with the findAll changes
- deleteAllContent and importer still need to use a hard 'findAll' as findPage({limit: 'all'}) doesn't have the same behaviour
- published_by should be set by business logic, rather than by users
Credits: An anonymous researcher working with Beyond Security's SecuriTeam Secure Disclosure program
refs #5727, #5602
- Add new 'order' column to posts_tags table
- Migrate all existing posts_tags to have a correct value for 'order'
- Rewrite updateTags to not remove all tags, and to correctly maintain order
- Add transaction support for tag operations
- Many tests
closes#5692, refs felixrieseberg/Ghost-Azure#1
- fix broken promise code
- fix incorrect handling of hash in user.add which causes 'Error: Invalid salt version 2' if owner user fixture is not present
refs #5614 and #5503
- update private blog type, including update to settings.edit
- switch order of populate settings & update fixtures + populate all settings
Private blog settings should not be returned by public endpoints
therefore they need a type which is not `blog` or `theme`.
`core` doesn't suit either, as those settings don't usually have UI
To resolve this, I created a new type `private` which can be used
for any setting which has a UI but should not be public data
no issue
- extract handlePermissions to utils
- added NoPermissionError when canThis() rejects
- omitted users.js because it uses special permission handling
closes#5150
- Post API understands next.author, next.tags, previous.author and previous.tags
- Post Read request filters out those properties and does the right thing with them
- Prev/Next post helpers send extra include properties
- Tests updated
closes#5551
- adds new test fixture generator and tests for tag pagination
- changes how post_count is added to use a select subquery rather than a join
issue #5500
- make `changePassword` and `resetPassword` methods on `user` model
consistent: use `object` and `options` arguments instead of multiple
different arguments
- change User API `changePassword` method to use these new arguments
closes#5490
- use same event handling pattern as fetchAll
- add support for `fetching:collection` to post model
- add tests to check that url is fetched via findAll and findPage
- extends clobbers the first argument you pass to it, so that should not be a variable that is used elsewhere, if you're also assigning the value, as it will have unintended side effects.
closes#2896
- move default options / custom code into model functions
- move most of the filtering logic into base/utils.filtering (to be relocated)
- move the remainder of findPage back into base/index.js and remove from posts/users&tags
- move pagination-specific logic to a separate 'plugin' file
- pagination provides new fetchPage function, similar to fetchAll but handling pagination
- findPage model method uses fetchPage
- plugin is fully unit-tested and documented
refs #2896
- remove duplicate query-building code
- use the same approach for creating the count query from the main query
- restructure the code to match more closely across the 3 findPage functions (prep for further refactoring)
refs #2896
- moves repeated code out of models
- creates a new file for unit-testable code (this should be moved in future)
- adds a default for `page` as that seems sensible
- adds 100% test coverage for the new file
closes#4778
- If import contains an owner that does not match original owner they are downgraded to admin
- Change error message for locked users to more generic message to account for imported users
- Adds duplicated user test
- Process Roles and maps import roles to db roles
- reduces export files in test fixtures
temp commit
closes#4782
- Still achieves the same goal of stripping out reserved characters
- Changes from removal to replacement
- This helps word separators from being removed
- Apostrophes (') are unaffected
fixes#5177
- we now pass API/model options directly to toJSON, which is unsafe as these options haven't always been filtered before they are passed.
- this fix adds a filter so that toJSON only uses the options it needs
- additionally, rename the 'name' option to something more specific to prevent clashes
refs #2330
- Pass through `options` to all toJSON calls on posts, tags, and users
- Use options.context.user to determine whether it's OK to return user.email
- Remove author.email handling code from frontend.js
fixes#5104, refs #4348, #2263
- Create a centralised event module
- Hook it up for posts, pages, tags and users
- Use it in sitemaps instead of direct method calls
- Use it for xmlrpc calls
- Check events are fired in model tests
- Update sitemap tests to work with new code
- Fix a bug where invited users were appearing in sitemaps
- Move sitemaps and xmlrpc into a directory together
closes#5066
- fix a problem that user model will fail to save if mysql has option of
STRICT_TRANS_TABLES
- add a parent call in user.format override function, which should fix
dateTime format before user saved.
no issue
- We already maintain our own fork of showdown, this moves our custom extensions to our fork
- Code duplication is removed
- Tests are also moved to the other repo
No Issue
- Set 'status: all` when calling generateSlug from the user model
so that all user slugs are checked for duplicates instead of
only active users.
closes#4262
- implementation based on #1545
- added integration test. Modified mocked posts because code requires published_at timestamps to be different.
- fixed 2 broken tests that depended on mocked posts to have "new Date()" as their timestamps
- added checks to only query db if next/previous post requested
closes#4770
- Check for Gravatar now allows the highest rating for images
- Defaults to silhouette in the event a user deletes their Gravatar
- Allows highest rating for Gravatars (x), since Ghost should have no opinion on image ratings.
Closes#4697
- Run tag add operations in sequence instead of in parallel
since generateSlug will hand out duplicate slugs until one of
them is committed to the database.
- Add test.
- Misc cleanup of method parameter names and jshint hints.
Refs #4521
- Handle 'include' query param in tags API.
- Add post_count support when fetching a tag with findOne.
- Remove post_count from options.include after processing.
- Extra database query no longer used to fetch post_count.
No Issue
- Reorder promise chains to defer database queries until they
are needed.
- Execute database queries that are not dependent on each other in
parallel instead of sequentially.
- Reduce the number of variables used to hold state across multiple
promise blocks.
- Do not go async unless necessary.
closes#4521
- if '?include=post_count' present, query db for tag/post_count information
and append it to returned JSON
- fixed linter errors
- added integration test
- parsing to int as knex (or PostGres driver) is returning bigint as strings
- iterating through a smaller collection to avoid exception
closes#4624
- added user_id to password reset request
- hide old password field
- updated changePassword method to check permissions
- updated changePassword method to work without oldPassword
- fixed bug for errors shown as [Object object]
closes#4445
- post model gets permalink format
- post model queries urlPathForPost to return computed url
- url helper modified to use post url
- urlForPost method abolished and replaced where necessary
- updated tests
closes#4498
- remove toJSON code which returns only IDs from objects
- don't auto-include tags & fields in post responses
- don't auto-include roles in user responses
- fix #allthethings that made assumptions about the auto-includes, or otherwise were only working because of the auto-include
Closes#623
- Add basic init and eventing scaffold
- Add sitemap-index.xml generation
- Broke out generators to individual files, added request handler
- Add page, author and tag xml files; add index mapping
- Add SiteMapManager unit tests
- Add Generators tests
- Cache invalidation headers for sitemap-*.xml
- Redirect sitemap.xml to index and rename to sitemap-index
- Handle page convert and publish/draft changes
- Add very basic functional test for route existence
- Add cache headers to sitemap routes
Base64 encoding causes some issues when the token is URL encoded
as the = symbol is not a valid URL character. We replace any =
symbols with a - as this is valid in a URL, but is guaranteed
not to appear in a base64 string. This fix ensures that Ghost
password resets work with mail providers such as Mailgun that
add their own tracking redirects
closes#3872 (for real this time)
