Commit Graph

833 Commits

Author SHA1 Message Date
Nazar Gargol
d299e8676f Fixed JSON Schema validation error message format
refs #10438
2019-02-08 12:17:39 +00:00
Nazar Gargol
28986b3b77 Fixed post.tags validation schema
refs #10438

- Change oneOf to anyOf to allow any of the required fields
2019-02-08 11:29:42 +00:00
Nazar Gargol
96cdde80b9 Fixed validation error when creating new tag from PSM
refs #10438

- Expanded required fields for related tags to have either one of 'id', 'slug' or a 'name' field
2019-02-08 11:02:18 +00:00
Naz Gargol
19643c75dd
Added JSON Schema validation for /posts endpoint (#10468)
refs #10438

- Added  validation helper based on JSON schema
- Added schema validation for POST/PUT in /posts endpoints
- Refactored existing authors validation test suite
- Extended test coverage with a minimally required structure of post.add validator
2019-02-08 10:40:58 +00:00
kirrg001
e1436e2985 Required "event" & "target_url" for webhook creation
refs #10461
2019-02-07 23:21:59 +01:00
kirrg001
b83232bf0c Added more webhooks & changed payload
refs #10461

- do not break the existing webhooks by keeping both payload formats for subscribers events
- refactored webhooks service to run models through target API version
- added new events described in the target issue reference
- this refactoring & enhancement is undocumented, further breaking changes will happen because we are actively working on: https://github.com/TryGhost/Ghost/issues/10438
2019-02-07 23:14:27 +01:00
kirrg001
7810140f15 Avoided site.changed event if nothing has changed
refs #10461
2019-02-07 22:08:36 +01:00
kirrg001
a8e0a173c2 Redefined context.integration
no issue

- we only have access to the ID at the moment
- we really don't want to end up in the same situation as with "context.user"
2019-02-06 21:36:09 +01:00
kirrg001
2fd4cbb93b Added v2 actions endpoint
refs #10431

- added v2 endpoint with browse permissions
- context.integration was never accessible in the model layer
  - why? https://github.com/TryGhost/Ghost/issues/10099
2019-02-06 21:36:09 +01:00
Aileen Nowak
93b936d2fb Added back "theme.uploaded" analytics event (#10450)
no issue

- With the changes in 79ca6c575c we removed old unused events
- The theme upload event is still used and needed to be put back
- Added the event emit right after the successful upload of the theme
- Renamed analytics events for more consistency
- We need to add the same event emitter to the v0.1 API as it's not deprecated
- emits a `theme.uploaded` event after the theme was successfully uploaded and saved
2019-02-05 17:38:40 +01:00
kirrg001
93092c8f20 Removed author from Admin API v2 output
refs #10438

- single author is deprecated and get's completely dropped in v3
2019-01-31 16:27:56 +01:00
kirrg001
7f7b477ce9 Fixed error message being unclear when admin api keys create posts without authors
refs #10438

- integrations != users
- Ghost's assumption is: if you create a post, the primary author becomes the logged in user
- we have to require authors for integrations
- short fix and needs some more thoughts later
2019-01-31 16:27:56 +01:00
Nazar Gargol
a463a56971 Changed path returned for uploads to absolute
refs #10438

- This change affects all /uploads/* and /images/* endpoints in Admin API
2019-01-31 15:02:33 +00:00
kirrg001
eafbaaeba5 Added v2 theme controller
refs #10060
2019-01-30 19:45:02 +01:00
Rishabh Garg
8fd4b3f09f
Added new admin API for members (#10435)
no issue

- Added read and browse admin API for members
2019-01-30 17:06:09 +05:30
notanengineercom
631716053a 🐛 Fixed files staying in temp directory after upload is done
closes #10174

- Introduced upload middleware that cleans up temporary files stored by mutler after the request is finished
- Removed redundant fs.remove calls as this work is now handled in newly introduced middleware
2019-01-30 09:24:29 +00:00
Nazar Gargol
bc90b8ec27 🐛 Fixed plaintext attribute error when it is empty
refs #10388

- Issue was caused by migration of empty strings to null 0edacf3fc1
2019-01-29 12:12:10 +00:00
Nazar Gargol
6318b65cab Changed context.api_key_id to an object containing key type information
refs #9865

- Changed id passed for api_key to an object to be able to differenciate between admin and content api requests
- Added integration id to frame context
- Small refactoring of frame context initialization
2019-01-24 17:22:58 +00:00
Nazar Gargol
5fbad09a56 Modified Admin API key output format
refs #9865

- Changed key format to {id}:{secret} so API consumer only has to worry about copying a single value during setup
- Updated key expiration time in getValidAdminToken test helper to match server side expiration check
2019-01-23 17:18:50 +00:00
Katharina Irrgang
4acc375fb6
Bumped knex & bookshelf dependencies (#10404)
refs #9389, refs #9248

- https://github.com/bookshelf/bookshelf/releases/tag/0.14.0
- Bookshelf has fixed it's previous attr handling, see https://github.com/bookshelf/bookshelf/pull/1848
- SQlite3 double slashes was merged into knex and released 👻tgriesser/knex@c746dea
2019-01-21 21:53:11 +01:00
Nazar Gargol
c3e22b6ab9 🐛 Fixed forced page filter not being respected in Content API
closes https://github.com/TryGhost/Ghost/issues/10395, https://github.com/TryGhost/Ghost/issues/10390

- The grouping was introduced due to a flasy conjunction of last filter in 'or' group with enforced page filter, due to this the rest of filters in the 'or' group were bypassed
- Additional grouping added here increases the possiblity of https://github.com/NexesJS/NQL-Lang/issues/7 bug happening
2019-01-21 13:26:45 +00:00
Hannah Wolfe
909f9012ed
Updated docs links to best equivalents (#10386)
* Updated docs links to best equivalents
   - Our documentation has been overhauled, this updates the all the old links sprinkled through Ghost
* Update integrity hash
2019-01-17 06:57:37 +00:00
Rishabh Garg
9d9690987a
🐛 Updated v2 content API output to return null instead of “” for empty fields (#10378)
refs #10345

- We are standardising on returning null from the Content API for any empty values
- Updated `post`, `tag` and `author` APIs
2019-01-15 17:58:51 +05:30
Naz Gargol
1a4497fc9a
🐛 Fixed default resource ordering in Content API (#10371)
refs https://github.com/TryGhost/Ghost/issues/10354

- Added ordering on input serialization layer for posts, pages, tags, and authors
- At the moment ordering is dependent on DB engine which will be resolved with https://github.com/TryGhost/Ghost/issues/6104
2019-01-15 12:21:04 +00:00
Katharina Irrgang
3924acd152
🗑Deprecated ghost_head & ghost_foot in favour of codeinjection_* for Settings API v2 (#10380)
closes #10373

- ghost_head & ghost_foot are deprecated from now on
- we want to remove them in v3
- this short fix is dirty (!)
- we return codeinjection_* for admin & content api
- this is a consistentency change e.g. posts return `post.codeinjection_*`
- need to raise a decoupling refactoring issue for the code comments
2019-01-15 13:03:17 +01:00
kirrg001
95b5fb1c81 Fixed relative image paths for settings v2 admin api
refs #10363
2019-01-14 21:01:12 +01:00
Katharina Irrgang
56db4ddd75
🐛 Fixed API v2 Settings browse returns relative image path (#10370)
closes #10363
2019-01-14 20:36:41 +01:00
Katharina Irrgang
732f97a074
De-aliased api versions in codebase (#10375)
closes #10357
2019-01-14 19:49:55 +01:00
Naz Gargol
48d6e7298a
Refactored content-disposition header handling in API v2 (#10374)
closes https://github.com/TryGhost/Ghost/issues/10331

- Left only a filename part to be handled by controller configuration, the rest was extracted to more generic headers layer
2019-01-14 18:05:16 +00:00
Rishabh Garg
a7ba90dfa1
🐛 Fixed filtering of page attribute when fields query param is used (#10352)
closes #10351

- Updated filtering logic for `page` in output serialisers of post/page in v2
2019-01-11 22:26:34 +05:30
Fabien O'Carroll
76bb40b7c0
Added clause in validation for include to not error (#10350)
* Added clause in validation for include to not error

refs #10337

Here we forgo erroring when an invalid property for include is sent, and
instead remove the invalid properties.

* Fixed authors test

* Fixed validators tests
2019-01-08 12:29:15 +01:00
Hannah Wolfe
954ca8c48f Removed unused visiiblity fields from Content API
refs #10337

- Trying to make the API output as clean as possible,
- We can add these back later if we start using them
2019-01-08 10:00:27 +00:00
Naz Gargol
4177548a84
Migrated redirects controller to API v2 (#10053)
refs #9866

- Migrated redirect routes to use new redirect controller
2019-01-07 11:32:53 +00:00
kirrg001
5548bbb2c1 Removed post.status from Content API v2 response
refs #10335

- only used for the preview feature, which does not remove the status field
2019-01-05 12:11:27 +01:00
kirrg001
a8dc29e794 Reverted page and status not being returned from Content API v2
refs https://github.com/TryGhost/Ghost/issues/10335

- temporary till Monday!!
2019-01-04 20:18:11 +01:00
Katharina Irrgang
30a0b1794a
Added calculated excerpt field to Content API v2 (#10326)
closes #10062

- return `post.excerpt` for Content API v2
- do not use `downsize`, because we might want to get rid of it if we drop v0.1 (downsize does not create good excerpts)
- simple substring of the plaintext field
2019-01-04 19:00:45 +01:00
Hannah Wolfe
5d977f23d4 Added Settings endpoint to V2 Content API
refs #10318

- This settings endpoint returns the commonly used, public information from our settings.
- The values are whitelisted each with a custom name for returning from the endpoint
2019-01-04 12:17:32 +00:00
Hannah Wolfe
80f9765a35 Removed deprecated settings
refs #10318

- cheap and dirty way of removing settings we don't use
- rewritten the settingsFilter function as that was unnecessarily complex
- aslo fixed the require of default-settings in the importer
2019-01-04 12:17:32 +00:00
Hannah Wolfe
ef98c65040
Clean v2 Content API (#10329)
* Removed unused fields from v2 Content API

- We want to ship the v2 Content API as clean and lean as we can
- Many fields in the DB aren't actually used, we shouldn't return these values
- Other values aren't useful outside of Admin clients, and shouldn't be returned either

Fields removed:
- tags: created_at, updated_at, parent
- authors: locale, accessibility, tour
- posts: locale, author status, page
2019-01-04 11:21:21 +00:00
Naz Gargol
d3f3b3dc20
Added plugin based author and public tag models in API v2 (#10284)
refs #10124

- Author model returns only users that have published non-page posts
- Added a public controller for tags (should be extracted to separate Content API controller https://github.com/TryGhost/Ghost/issues/10106)
- Made resource configuration dynamic based on current theme engine
- This needs a follow-up PR with fixes to the problems described in the PR
2019-01-03 20:30:35 +01:00
kirrg001
75037bcb28 Added some deprecation notes for x_by fields
closes #10286

- we will come up with a new activity stream/actions concept soon
2019-01-03 17:14:52 +01:00
kirrg001
789a3c0715 Removed x_by fields from API v2 response
refs #10286

- v2 no longer exposes x_by fields (published_by, updated_by, created_by)
- we will add a brand new concept called activity stream/actions soon
2019-01-03 16:38:52 +01:00
Nazar Gargol
6797da599b Added functional test for import endpoint in API v2
no issue

- This change is a follow up to this bugfix https://github.com/TryGhost/Ghost/pull/10299
- Added default export JSON to keep the state of db test suite intact
- Small typo fixe that noticed while debugging
2019-01-02 14:10:25 +01:00
Kevin Kuang
4a5a5acba7 🐛 Fixed broken content import in Admin API v2 (#10299)
no issue
2018-12-20 11:01:48 +01:00
Aileen Nowak
506d013f25
Ability to not send welcome email (#10293)
no issue

- We need to be able to not send the welcome email if needed
- Intruduces a new possible config setting `sendWelcomeEmail` which is set to `true` by default
2018-12-17 15:02:47 +01:00
Naz Gargol
aa8e75914d
Migrated db controller to API v2 (#10051)
refs #9866

- Migrated db import/export routes to use new db controller
2018-12-17 14:45:09 +01:00
Katharina Irrgang
5d2ab19881
Removed invitedBy from admin API: invitation endpoint (#10291)
refs #10286

- we want to deprecate all `x_by` fields
- we would like to get rid of all usages to be able to easily remove the fields in the future
- `invitedBy` is not used in the admin client
2018-12-17 14:16:39 +01:00
Naz Gargol
aca887a35d
Added async file header handling for API v2 (#10292)
refs #9866
2018-12-17 12:47:19 +01:00
Katharina Irrgang
9d7c3bd726
🐛 Fixed all known filter limitations (#10159)
refs #10105, closes #10108, closes https://github.com/TryGhost/Ghost/issues/9950, refs https://github.com/TryGhost/Ghost/issues/9923, refs https://github.com/TryGhost/Ghost/issues/9916, refs https://github.com/TryGhost/Ghost/issues/9574, refs https://github.com/TryGhost/Ghost/issues/6345, refs https://github.com/TryGhost/Ghost/issues/6309, refs https://github.com/TryGhost/Ghost/issues/6158, refs https://github.com/TryGhost/GQL/issues/16

- removed GQL dependency
- replaced GQL with our brand new NQL implementation
- fixed all known filter limitations
- GQL suffered from some underlying filter bugs, which NQL tried to fix
- the bugs were mostly in how we query the database for relation filtering
- the underlying problem was caused by a too simple implementation of querying the relations
- mongo-knex has implemented a more robust and complex filtering mechanism for relations
- replaced logic in our bookshelf filter plugin
- we pass the custom, default and override filters from Ghost to NQL, which then are getting parsed and merged into a mongo JSON object. The mongo JSON is getting attached by mongo-knex.

NQL: https://github.com/NexesJS/NQL
mongo-knex: https://github.com/NexesJS/mongo-knex
2018-12-11 11:53:40 +01:00
kirrg001
7af2802e14 Added validation for null|undefined values for required keys
closes #10071
2018-12-10 16:33:39 +01:00
kirrg001
ccd9541f75 Replaced hardcoded translation in shared input validator
no issue

- re-use existing keys
2018-12-10 16:33:39 +01:00
kirrg001
812e533554 Corrected incorrect syntax for required propierties
no issue
2018-12-10 16:33:39 +01:00
kirrg001
11f0f1c314 Fixed incorrect validation message for required values
no issue

- the message always showed: `Validation (FieldIsRequired) failed for key`
2018-12-10 16:33:39 +01:00
Nazar Gargol
da7fdfeae6 Extracted trimAndLowerCase utility to shared utility module
refs #9866
2018-12-10 13:56:42 +01:00
Rishabh Garg
bf295a96a8
Added configuration controller to v2 API (#10056)
refs #10060
2018-12-04 09:27:36 +05:30
Katharina Irrgang
e48c28b98f 🐛 Protected Ghost blog against invalid uploaded routes.yaml (#10100)
* 🐛  Protected Ghost blog against invalid uploaded routes.yaml

no issue

- e.g. you upload `filter:tag=this is a wrong filter value`
- ask the url service if it has finished it's work to ensure the upload was successful
- wait 5 seconds till Ghost will bring back the last uploaded valid version

* fixed test
2018-11-26 16:28:22 +07:00
Fabien O'Carroll
95ba6a55ec
Checked options.include when filtering tags (#10163)
no-issue
2018-11-14 17:54:15 +07:00
Rishabh Garg
b2201d4179 Removed formats from private posts in content api (#10154)
closes #10118

All behind a members labs switch for now

* Added filter for member only content

* Updated frame context

* Cleaned up members content check

* Cleanup

* Cleanup

* Ensured members filtering works without include=tags

* Protected against missing query

* Fixed usage of include vs withRelated

* Moved includeTags logic for members behind members flag to use tags

* Cleanup

* Update input serializer dependency

Co-Authored-By: rishabhgrg <zrishabhgarg@gmail.com>

* Added some explanations
2018-11-14 14:32:14 +07:00
Fabien O'Carroll
7323258415 🚧 Added req.member to the v2 api context object
closes #10112
2018-11-08 13:32:41 +07:00
Katharina Irrgang
ff6bf5f318 Renamed /users to /authors for Content API V2 (#10096)
refs #10061

- Made /authors endpoint available in Content API V2
2018-11-07 15:29:37 +01:00
Rishabh Garg
3b8621e19c Added slack controller to v2 API (#10086)
refs #10060

- Added slack controller to v2 admin API
- Added new API test for slack API controller
2018-11-06 23:20:43 +01:00
Naz Gargol
ec03b3cfc5
Content API v2 date formatting (#10095)
closes #10065 

- Added UTC offset to dates returned by Content API
- Added test checking new format is compatible with Admin API
- Refactored output serializer mapping logic
2018-11-06 16:12:12 +01:00
Rishabh Garg
7b38986316 Removed mobiledoc format from Content API V2 response (#10098)
closes #10097

- removed formats `mobiledoc` option directly in post input serializer for v2 Content API
2018-11-06 14:39:16 +01:00
Rishabh Garg
3345618731
Added helper to check for content API (#10104)
no issue

* Added helper to check if API called is content API
2018-11-06 17:36:22 +05:30
kirrg001
22911b5812 Fixed frame context being empty
no issue
2018-11-06 14:44:06 +05:30
Katharina Irrgang
1b9c61eed1
Returned relative paths in html for Content API V2 by default (#10091)
refs #10083

- you can send `?absolute_urls=true` and Ghost will also transform the paths in the content (this is optional/conditional)
2018-11-05 18:07:45 +01:00
Rishabh Garg
94b3735c6e Added oembed controller to v2 api (#10057)
refs #9866

- Added oembed controller to v2 API
2018-11-02 09:32:26 +01:00
kirrg001
8d12c8908f 🐛 Fixed missing filename when exporting subscribers csv
closes #10075

- the filename was missing
2018-10-27 18:39:39 +02:00
kirrg001
5f7401051c 🐛 Fixed pagination for subscribers
closes #10074

- the `page` option was missing
2018-10-27 18:27:35 +02:00
root@andrea:~#
3f91a9e8a2 Corrected 'Content-Length' header by using Buffer.byteLength (#10055)
Closes #10041
1. Why is this change neccesary?
String.prototype.length returns the number of code units in the string (number
of characters) while Buffer.byteLength returns the actual byte length of a
string.

2. How does it address the issue?
Places that use String.prototype.length to calculate Content-Length
were switched to Buffer.byteLength instead.
2018-10-25 09:18:36 +07:00
Fabien O'Carroll
7fafa1e152 Fixed mail api usage of the notifcations api
closes #10047

Requiring in the api module correctly wraps the notification api and
exposes it as a function.
2018-10-24 13:18:51 +05:30
Kevin Ansfield
71f0c08a34
Added edit webhook route to v2 Admin API (#10033)
no issue
- webhooks UI requires the ability to edit webhooks
- added `edit` permission for `webhook`
- added `edit` method to v2 webhook controller
- added `PUT /webhooks/:id` route to v2 Admin API routes
2018-10-19 18:35:55 +01:00
kirrg001
99a2f3ba3a Fixed read pages with user access
refs #9866

- we still serve pages via /posts with user access
- we only differentiate this behaviour for the content api
2018-10-19 10:54:05 +02:00
kirrg001
759c25d03e Fixed read posts/pages for v2
refs #9866

- read does not support `filter`
2018-10-19 10:40:47 +02:00
Rishabh Garg
8ad951d7f3
Added new site.changed event and webhook trigger service (#10019)
refs #9942

* Added new middleware to trigger events

* Refactored webhooks service
- added new trigger service, moved listen service to its own file
- started listening to new site.changed event
- cleaned up trigger service to work with new webhook fields
- cleaned up tests
- removed redundant trigger method in v0.1 controller
2018-10-19 00:01:30 +05:30
kirrg001
e302be2749 Changed preview controller to support v0.1 and v2
refs #9866

- invent preview api, but only used internally
  - the idea of a preview api is definitiely reaslistic and came up in the past a couple of times
- by that we don't have to differentiate between pages or posts controller
- still support v0.1
- preview controller is not registered for http, only internal handling
2018-10-18 19:41:07 +02:00
kirrg001
cbf2817e39 Added missing read pages endpoint
refs #9866

- the endpoints were missing
- the site app needs pages.read for v2
2018-10-18 19:41:07 +02:00
Fabien O'Carroll
17feb14e4a Added HTTP BREAD for integrations resource (#9985)
refs #9865

* Added generic messaging for resource not found
* Ensured integration model uses transaction for writes
* Created POST /integrations endpoint
* Created GET /integrations/:id endpoint
* Created GET /integrations endpoint
* Created PUT /integrations/:id endpoint
* Created DELETE /integrations/:id endpoint
2018-10-18 14:03:56 +01:00
Rishabh Garg
915d5612a1 🐛 Fixed relative image URLs becoming absolute URLs on save (#10025)
closes #10024

- Updated input serializers for posts/tags/users to handle absolute urls conversion

-------
1. Ghost stores relative images urls
2. API V2 returns images with absolute urls
3. Ghost-Admin sends absolute urls back on any save e.g. update user

**Current behavior**: This will override the relative image path in db to absolute, which in turn won't get updated in future if domain or protocol changes for e.g.
**Fix**: On save/update, input serializers converts any absolute image url paths back to relative if the base URL from image fields matches the configured URL
2018-10-18 12:18:47 +01:00
Fabien O'Carroll
e865d2218c 🐛 Fixed pagination/duplicate posts and users appearing in admin area (#10031)
closes #10029

- allowed page option for users, posts, & tags browse 
  - The page query param was not forwarding to the query, meaning that when the admin client requested the next page of users or posts, it would receive the first page again.
2018-10-18 10:05:51 +01:00
Fabien O'Carroll
ae71f2deca Added spam prevention for v2 sessions (#10030)
no-issue

- Added spam prevention to POST /session
  - This blocks repeated requests the the /session endpoint preventing brute
force password attacks
- Updated session controller to reset brute middleware
  - This updates the session controller to reset the brute force protection
on a successful login. This is required so that a user is not locked out
forever :o!!
2018-10-18 09:58:29 +01:00
Nazar Gargol
fd958addb6 Migrated update check to use api v2
refs #9866

- Switched update checker to api v2
- Updated and cleaned up the corresponding test suite
- Updated the frame pipeline to respect context passed in with Frame instance
- Exposed 'active' verison from api index module
2018-10-18 00:13:31 +02:00
Fabien O'Carroll
4f1866a263
Allowed for repeated query parameters for arrays (#10021)
no-issue

There are a few libraries, including node core that when given an array
for a query parameter will encode it as repeated query params. e.g.

```
{someParam: ['a', 'b']}
// becomes
'?someParam=a&someParam=b'
```

This adds a check for the value to stop us 500ing on repeated keys and
to add easier interop with http clients
2018-10-17 13:43:32 +07:00
kirrg001
14a1bdbcf6 Removed invite before adding in v2
refs #9866

- the logic in v2 was missing
- it exists in v0.1
2018-10-16 17:26:24 +02:00
kirrg001
a65702df1d Extended api/index.js to export all available api versions
refs #9866
2018-10-16 16:03:32 +02:00
Nazar Gargol
d582c06eee Optimized usage of urls in API v2
refs #9866

- Extracted url decoration logic to utility in output serializers in posts, pages, users, and tags
- Added test cases for url usage by child object (tags of posts)
2018-10-16 13:02:04 +02:00
Fabien O'Carroll
2fbc5aa257
Added apiImpl.data to apiOptions for serialisation (#10016)
no-issue

This is to give serializers access to the expected data properties so
that can be used for filtering.
2018-10-16 16:51:50 +07:00
Rishabh Garg
51dde1e38c
Refactored config to handle direct calls for specific version (#10012)
refs #9866

- Refactored overrides config to include direct version configs(v0.1, v2), supported versions map to direct version
- Refactored `getApiPath` to handle direct versions as well as mappings of supported version
2018-10-16 15:20:51 +05:30
Nazar Gargol
9fd9186557
🐛 Fixed 'url' attribute miscalculation when when requested as the only part of fields filter (#9969)
closes #9962

- Fixed the bug with url being set to /404 when id was not present on the model
- Added a functional test to cover this bug
- Refactored url decorating methods to be more clear about the nature of passed parameters
2018-10-15 14:47:56 +02:00
Fabien O'Carroll
3db102a776
Added API Key auth middleware to v2 content API (#10005)
* Added API Key auth middleware to v2 content API

refs #9865

- add `auth.authenticate.authenticateContentApiKey` middleware
  - accepts `?key=` query param, sets `req.api_key` if it's a known Content API key
- add `requiresAuthorizedUserOrApiKey` authorization middleware
  - passes if either `req.user` or `req.api_key` exists
- update `authenticatePublic` middleware stack for v2 content routes

* Fixed functional content api tests

no-issue

This fixes the functional content api tests so they use the content api
auth.

* Fixed context check and removed skip

* Updated cors middleware for content api

* Removed client_id from frame.context

no-issue

The v2 api doesn't have a notion of clients as we do not use oauth for it

* Fixed tests for posts input serializer
2018-10-15 16:23:34 +07:00
Nazar Gargol
76f4a4bb03
Enforced non-page posts only to be returned by /posts endpoint from Content API (#10002)
refs #9866

- Added logic ensuring page filter is always set to false in posts endpoint for Content API
- Added functional tests to pages and posts
- Added absolute_url logic in pages controller
2018-10-13 00:48:49 +02:00
Katharina Irrgang
4aaff31890
Added users ctrl to v2 (#10001)
refs #9866
2018-10-12 23:27:30 +02:00
Katharina Irrgang
1ee4d53bfe
Added tags ctrl to v2 (#10000)
refs #9866
2018-10-12 23:10:43 +02:00
Nazar Gargol
310526b6c5 Added upload controller to v1 (#9997)
refs #9866
2018-10-12 22:41:39 +02:00
Nazar Gargol
cfea6375ab Added subscribers controller to v2
refs #9866
2018-10-12 22:27:37 +02:00
kirrg001
0338ba56c0 Added removal of null values in v2
refs #9866

- also moved id mismatch to global validator
- this is not specific to posts
2018-10-12 21:46:16 +02:00
kirrg001
adc5b18fb7 Added invites ctrl to v2
refs #9866
2018-10-12 21:13:20 +02:00
kirrg001
1472035137 Added mail ctrl to v2
refs #9866
2018-10-12 21:13:20 +02:00
kirrg001
9f2d68a027 Added notifications ctrl to v2
refs #9866
2018-10-12 21:13:20 +02:00
kirrg001
b899a6fec8 Added settings ctrl to v2
refs #9866
2018-10-12 21:13:20 +02:00
kirrg001
8b54cfea81 Supported status code as function
refs #9866
2018-10-12 21:13:20 +02:00
kirrg001
850e3139ee Added api permissions before hook support
refs #9866
2018-10-12 21:13:20 +02:00
kirrg001
4dcf256371 Added ability to define permission identifier
refs #9866

- by default it used `options.id`, which tells the permission layer the target id
- but some controllers want to use a different identifier
- e.g. settings -> settings.key
- e.g. password changes -> password[0].user_id
2018-10-12 20:02:08 +02:00
kirrg001
a153400164 Added posts controller to v2
refs #9866
2018-10-12 19:21:43 +02:00
kirrg001
27714075b5 Added handling for empty query options
refs #9866

- it's fine if you pass e.g. `?formats=`
- same behaviour as v0.1
2018-10-12 19:21:43 +02:00
kirrg001
5d3b026cd9 Fixed missing return statement in shared validators
refs #9866

- if the fn returns a Promise, it won't be returned
2018-10-12 18:34:57 +02:00
kirrg001
551f14d9ec Moved toJSON call to api v0.1 controller for ownership transfer
refs #9866
2018-10-12 18:12:16 +02:00
Rish
0a834d4223 Added webhooks controller to API v2
refs #9866

- Added new webhooks controller to v2 API
- Added webhooks tests to v2 API
2018-10-12 18:42:23 +05:30
Rishabh Garg
5683204371
Added v2 controller for slugs (#9978)
- Added slugs controller to v2 API
- Added slugs tests to v2 API
- Updated generic validation error message in shared validator to return validation error with sub-message
2018-10-12 17:55:20 +05:30
Fabien O'Carroll
caccda1aab
Implemented global validation on defined fields (#9992)
no-issue

This is to allow global validation to run on fields that have some user
validation defined.
2018-10-12 15:16:12 +07:00
kirrg001
7a73dfd9bc Extended all shared validator
refs #9866

- there was a missing step in the shared validator
- we have to differentiate between data validation for browse/read and data validation for add/edit
- furthermore, the data validation for add/edit was missing and was not copied over from v0.1 (check structure of incoming body)
- adds the ability to require properties from req.body.docName[0]
2018-10-12 09:40:34 +02:00
Nazar Gargol
86e9c35c3c Allowed passing an array directly instead of requiring object with values key for validation options
noissue
2018-10-10 16:48:22 +02:00
Nazar Gargol
eb0bc3068c Migrated roles controller to API v2
refs #9866

- Added new controller to v2 API
- Added roles tests to v2 API
2018-10-10 16:48:22 +02:00
Fabien O'Carroll
1b9aa2546f
Updated frame.context to use req.api_key_id (#9965)
* Added api_key_id to frame.context

refs #9865

This is to allow controllers to check permissions using api_key_id data.

* Removed client and client_id from frame.context

refs #9865

This is unused as we only support oauth on v0.1 API.
2018-10-09 18:28:38 +07:00
kirrg001
3e397275d1 Added protection against an empty notification response
no issue

- discovered in https://github.com/TryGhost/Ghost/pull/9940
- pulled the change out and push straight to master
2018-10-06 21:01:42 +02:00
kirrg001
2e3876b477 Moved user controller permission handling to user permissible fn
refs #9866

- prep for v2
- you can better unit test the permissible function
- this avoids copying over the permission handling to v2 controller
- it was possible to move this logic into the model layer, because we now support `unsafeAttrs`
2018-10-06 02:25:46 +02:00
kirrg001
301b18b0ed Moved custom invite permission to permissible fn
no issue

- now that we have a concept of `unsafeAttrs`, we can move the custom permissions to the invite model

Why doing now?

A) We won't copy this controller code to v2.
B) Makes it easier to unit test this behaviour
2018-10-05 15:38:14 +02:00
Katharina Irrgang
959912eca3
Added tiny framework to support multiple API versions (#9933)
refs #9326, refs #9866

**ATTENTION: This is the first iteration. Bugs are expected.**

Main Goals: 

- add support for multiple API versions.
- do not touch v0.1 implementation
- do not break v0.1

## Problems with the existing v0.1 implementation

1. It tried to be generic and helpful, but it was a mixture of generic and explicit logic living in basically two files: utils.js and index.js.

2. Supporting multiple api versions means, you want to have as less as possible code per API version. With v0.1 it is impossible to reduce the API controller implementation. 

----

This commit adds three things:

1. The tiny framework with well-defined API stages.
2. An example implementation of serving static pages via /pages for the content v2 API.
3. Unit tests to prove that the API framework works in general.

## API Stages

- validation
- input serialization
- permissions
- query
- output serialization

Each request should go through these stages. It is possible to disable stages, but it's not recommended.

The code for each stage will either live in a shared folder or in the API version itself. It depends how API specific the validation or serialization is. Depends on the use case.

We should add a specific API validator or serializer if the use case is API format specific.
We should put everything else to shared.

The goal is to add as much as possible into the shared API layer to reduce the logic per API version.

---

Serializers and validators can be added:

- for each request
- for specific controllers
- for specific actions

---

There is room for improvements/extensions:

1. Remove http header configuration from the API controller, because the API controller should not know about http - decouple.

2. Put permissions helpers into shared. I've just extracted and capsulated the permissions helpers into a single file for now. It had no priority. The focus was on the framework itself.

etc.

---

You can find more information about it in the API README.md (api/README.md)

- e.g. find more information about the structure
- e.g. example controllers

The docs are not perfect. We will improve the docs in the next two weeks.

---

Upcoming tasks:

- prepare test env to test multiple API versions
- copy over the controllers from v0.1 to v2
- adapt the v2 express app to use the v2 controllers
2018-10-05 00:50:45 +02:00
Fabien O'Carroll
b185892b7b
Created session controller (#9911)
refs #9865

Note that this controller is the singular, that's because we plan to
make a session resource controller to be used with /sessions, wheras
this is on /session
2018-10-03 20:45:42 +07:00
Nazar Gargol
82b7aea641
Refactor URL generation from models (#9917)
Moved URL attributes logic from the model into API layer 

refs #9866

- Moved URL related attribute calculation for posts, users, and tags into API layer
- Added test coverage for url attributes in tags/authors/primary_tags/primary_authors
2018-10-03 15:44:30 +02:00
Katharina Irrgang
efc5219afa Added empty api v2 + shared folder and README.md (#9920)
refs #9866
2018-09-28 00:03:21 +05:30
Rishabh Garg
6163d1f128 Updated overrides config for api versions with v0.1 to deprecated (#9921)
refs #9866

- v0.1 == deprecated
- v2 == active
2018-09-27 19:34:12 +02:00
Katharina Irrgang
b43ab65d8a
Moved api controllers into api/v0.1 (#9918)
refs #9866

- preparation for v2
- moved api/ to api/v0.1
- do export v0.1 straight from the api folder, we don't want to touch this right now
- that means currently if you require the api folder, we return v0.1 by default
- there were some direct requires of api files in the test env
  - some of them use rewire
  - for now, we just correct the require path to require api/v0.1/
  - we touch the test env next week

**Docs about V2 design are coming soon!**
2018-09-27 16:06:57 +02:00
Nazar Gargol
4c5bff0f49 Removed toJSON serialization in findPage method (#9899)
refs #9866

- Removed `toJSON` call in `findPage`
- Added JSON serialization on API layer
- Reason: model and api layer were coupled - all other model actions just returned the raw data and no specific format
- Corrected test suites to serialize fetched models to JSON
- Removed `absolute_urls` attribute from validOptions findPage methods as it's no longer needed in the data layer
- Changed 'include' test as this option is now tolerated and returns data
2018-09-26 14:11:22 +02:00
Fabien O'Carroll
6c35de7d95 Updated api.http handling of controller returned fns (#9907)
refs #9865

This is to ensure that if a controller returns a function, it will
always get called regardless of method.

Also cleaned up top level const usage
2018-09-26 08:13:41 +02:00
Rishabh Garg
9624c88f3e Refactored url utility to generate multiple API version URLs (#9897)
refs #9866

- added api version config to overrides, which makes it possible to have a centralized api versioning configuration
- the next PR will use this config in the web folder
- make api url generation in url service flexible and dynamic
- remove hardcoded API_PATH
- updated all places which used `urlFor('api'..)` -> we now ask for explicit api version
2018-09-24 21:49:20 +02:00
kirrg001
bc4b637e30 Refactored direct usages of api controllers
refs #9866

- if we start with v2 controllers, the code base should not require specific api controllers
- because e.g. `require('../api/posts')` will no longer exist
- if you require the api folder, you will get the latest available version by default e.g. `require('../api').posts`
- this branch does not touch the test env (!)
2018-09-21 15:18:22 +02:00
Katharina Irrgang
727ff183f4 Normalised how we require models
refs #9866

- if I want to do a project search and looks for model usages e.g. `models.`, then I won't find these usages
- normalise how we require models -> consistency
2018-09-21 15:16:19 +02:00
Katharina Irrgang
213474835b Refactored how we require shared middlewares from web/ (#9893)
refs #9866

- use package notation
- get rid of x requires for middlewares
- improved readability
- do not refactor web/api/v0.1
2018-09-21 16:17:11 +05:30
Rishabh Garg
fcd275f6c0 Refactored web/middleware and web/utils to web/shared (#9892)
refs #9866

- Moved web/middleware to web/shared/middlewares
- Moved util file to web/shared/utils
2018-09-20 20:04:34 +02:00
Rishabh Garg
0549c9f7d4 ES6 migration: server/api/ (#9876)
refs #9589
2018-09-18 15:59:56 +02:00
Sumedh Nimkarde
efd1587ee9 Switched to eslint-plugin-ghost (#9835)
refs #9834

- @TODO: the test env eslint needs to use the plugin, not part of this PR
2018-09-17 20:49:30 +02:00
Mandeep Singh Gulati
03af23cddf ES6 migration: server/api (#9761)
refs #9589

- not all of the api files were updated
2018-09-10 14:30:48 +02:00
Fabien O'Carroll
c9b8ddde4b 🎨Added absolute_url flag to public api (#9833)
closes #9832

The API _should_ be returning absolute URLs for everything, 3rd party applications require absolute urls to read and display ghost data correctly. Currently they have to concat the blog url and the resource url, which is very uncomfortable.

Changing the public api like this would be considered a breaking change however so we've opted to put it behind a query parameter named `absolute_urls`.
2018-08-31 11:02:39 +01:00
Nazar Gargol
0faf89b5ab Added ability to resize and compress images on upload (#9837)
refs #4453

* On by default

* Added config to disable resizing

* Added basic image optimization processing

* Added dep: sharp (optional dep)

* Added resize middleware

* Take care of rotation based on EXIF information

* Removed all meta data from optimised image

* Added handling if sharp could not get installed

* Do not read ext twice - optimisation

* Do not call sharp if config is disabled

* Do not remove the original image which was uploaded (store 2 images)

* Support of `req.files` for internal logic

* Disabled cache to enable file removal on Windows
2018-08-30 17:30:36 +01:00
Fabien O'Carroll
8ccf27340b Oembed meta tag fallback for unknown providers (#9827)
closes #9786

- Make GET request when url has no provider match
  - The HEAD request was made in order to send less data over the wire when
checking for redirects for urls that do not have an oembed provider
match. We are now going to look for provider metatags withing the
response of the request - rather than making a HEAD followed by a GET if
no redirect is found, this condenses that to a single request.

- Try to get OEmbed data from tag if no provider
  - Here we parse the HTML response of the resource and look for a link tag
that will give us the oembed resource url which we can use to fetch the
embed html
2018-08-27 15:02:03 +01:00
kirrg001
bbde22a687 Fixed tests
no issue

- replace test themes
  - otherwise they are all invalid with the new GScan version
- fix general tests because of Ghost 2.0 logic
2018-08-16 12:13:24 +02:00
kirrg001
8bb7088ba0 🔥 Removed permalink setting
refs #9742

- removed usage of single permalink setting
  - with dynamic routing this configuration does no longer makes sense
  - because you can configure your permalinks in the routes.yaml
  - furthermore you can have multiple collections with multiple permalinks
- removed @blog.permalinks
- do not export permalink setting
- do not import permalink setting
- permalink setting UI will be removed soon
- get rid of {globals.permalink} completely
- remove yaml in-built migration
- do not expose settings.permalinks via the private API
- do not allow to edit this setting
- keep phyiscal value in case a blog needs to rollback from v2 to v1
- sorted out when the routers should be created
  - ensure routes.yaml file doesn't get validated before Ghost is fully ready to start
2018-08-16 12:13:24 +02:00
kirrg001
52a9eebabf Changed comments and deprecation notes
refs #9742
2018-08-16 12:13:24 +02:00
kirrg001
c40454f23c Added ability to upload/reload routes.yaml
refs #9744

- added two new endpoints to upload/download routes.yaml
- reload site express app on successful/valid upload
- reload url service on sucessfuly upload
- force clear cache of pages
- ensure we keep a backup of the routes.yaml file
- this feature was mostly tested manually
- @TODO: i have to write unit tests - will do later
- @TODO: do a memory test to ensure we haven't introduced any memory leaks with this feature
2018-08-16 12:13:24 +02:00
kirrg001
7b9c6af9d7 Updated docs links
refs #9742

- 2.0 will become the latest version on our readme pages
2018-08-16 12:13:24 +02:00
kirrg001
529bec8089 Renamed core/server/data/export to core/server/data/exporter
no issue

- i'vejust renamed the folder
- it makes more sense
  - data/importer
  - data/exporter
2018-08-16 12:13:24 +02:00
Kevin Ansfield
cfd9ff3993 🎨 Koenig - Added support for shortened URLs in embed card (#9781)
refs https://github.com/TryGhost/Ghost/issues/9724

- perform a HEAD request on a url if we don't find a matching provider, following any redirects until we hit success response before looking up providers for the resulting url
2018-08-12 15:57:19 +02:00
kirrg001
1ce504bb2d Fixed exporting extra tables for endpoint db/backup
refs #8719

- initial commit: 40c8eacd44
- we have forgotten that there is another endpoint which triggers an export (the backup endpoint)
- this endpoint needs to accept the new `include` query param as well (was missing)
2018-08-10 15:31:54 +02:00
kirrg001
75cc60c20a Added option to import clients and trusted domains
refs #9742, refs #8719

- make it possible to import more tables (optional)
  - available tables: clients, trusted domains
- by default we won't import these tables, you have to tell Ghost using `include` (same syntax on export)
- we won't announce this ability for now (stays hidden)
2018-08-03 14:12:06 +02:00
kirrg001
40c8eacd44 Added option to export extra tables
refs #9742, refs #8719

- you can now use `include` to export extra tables e.g. `include=clients`
- admin client won't make use of this option yet, maybe later and optional
- we won't announce this new ability for now (stays hidden)
2018-08-03 14:11:45 +02:00
Mandeep Singh Gulati
104a4a5c92 ES6 migration: server/api (#9756)
refs #9589
2018-07-30 12:28:05 +02:00
Mandeep Singh Gulati
75bcfba71b ES6 migration: server/api (#9733)
refs #9589
2018-07-23 14:38:40 +02:00
Antony Garand
742eed99bd Fixed typo in server/api/utils (#9714)
no issue

- removed the extra `*` from the jsdoc of the handlePermissions function
2018-07-10 12:20:43 +02:00
Kevin Ansfield
865acecaef
🐛 Koenig - Support schema-less URLs in embed card (#9725)
refs https://github.com/TryGhost/Ghost/issues/9724
- adjust the "base url" regex in the oembed endpoint to strip schemaless scheme `//` as well as `https?://`
2018-07-09 10:32:39 +01:00
Kevin Ansfield
ca20f3a6b0 Added /oembed API endpoint
refs https://github.com/TryGhost/Ghost/issues/9623
- add `oembed-parser` module for checking provider availability for a url and fetching data from the provider
  - require it in the `overrides.js` file before the general Promise override so that the `promise-wrt` sub-dependency doesn't attempt to extend the Bluebird promise implementation
- add `/oembed` authenticated endpoint
  - takes `?url=` query parameter to match against known providers
  - adds safeguard against oembed-parser's providers list not recognising http+https and www+non-www
  - responds with `ValidationError` if no provider is found
  - responds with oembed response from matched provider's oembed endpoint if match is found
2018-06-12 16:18:01 +01:00
Katharina Irrgang
b392d1925a
Dynamic Routing Beta (#9596)
refs #9601

### Dynamic Routing

This is the beta version of dynamic routing. 

- we had a initial implementation of "channels" available in the codebase
- we have removed and moved this implementation 
- there is now a centralised place for dynamic routing - server/services/routing
- each routing component is represented by a router type e.g. collections, routes, static pages, taxonomies, rss, preview of posts
- keep as much as possible logic of routing helpers, middlewares and controllers
- ensure test coverage
- connect all the things together
  - yaml file + validation
  - routing + routers
  - url service
  - sitemaps
  - url access
- deeper implementation of yaml validations
  - e.g. hard require slashes
- ensure routing hierarchy/order
  - e.g. you enable the subscriber app
  - you have a custom static page, which lives under the same slug /subscribe
  - static pages are stronger than apps
  - e.g. the first collection owns the post it has filtered
  - a post cannot live in two collections
- ensure apps are still working and hook into the routers layer (or better said: and register in the routing service)
- put as much as possible comments to the code base for better understanding
- ensure a clean debug log
- ensure we can unmount routes
  - e.g. you have a collection permalink of /:slug/ represented by {globals.permalink}
  - and you change the permalink in the admin to dated permalink
  - the express route get's refreshed from /:slug/ to /:year/:month/:day/:slug/
  - unmount without server restart, yey
- ensure we are backwards compatible
  - e.g. render home.hbs for collection index if collection route is /
  - ensure you can access your configured permalink from the settings table with {globals.permalink}

### Render 503 if url service did not finish

- return 503 if the url service has not finished generating the resource urls

### Rewrite sitemaps

- we have rewritten the sitemaps "service", because the url generator does no longer happen on runtime
- we generate all urls on bootstrap
- the sitemaps service will consume created resource and router urls
- these urls will be shown on the xml pages
- we listen on url events
- we listen on router events
- we no longer have to fetch the resources, which is nice
  - the urlservice pre-fetches resources and emits their urls
- the urlservice is the only component who knows which urls are valid
- i made some ES6 adaptions
- we keep the caching logic -> only regenerate xml if there is a change
- updated tests
- checked test coverage (100%)

### Re-work usage of Url utility

- replace all usages of `urlService.utils.urlFor` by `urlService.getByResourceId`
  - only for resources e.g. post, author, tag
- this is important, because with dynamic routing we no longer create static urls based on the settings permalink on runtime
- adapt url utility
- adapt tests
2018-06-05 19:02:20 +02:00
kirrg001
af5717762d Fixed incorrect return value for publicAPI from the configuration endpoint
no issue

- discovered while coding
- the value was always false, because we've tried to read the value from the config object
- the value lives in the database and is accessible via the labs service
2018-05-31 15:14:59 +02:00
kirrg001
5f5f0021db 🔥 Drop Node v4 Support
no issue

- support ends today
- see https://github.com/nodejs/Release
- removed `use strict`
2018-05-01 14:06:18 +02:00
kirrg001
e43bdad818 Fixed broken i18n keys
refs #9519

- discovered by https://github.com/TryGhost/Ghost/issues/9519#issuecomment-374891164
2018-04-18 15:05:20 +02:00
Aileen Nowak
23f59c341c Replaced routeKeywords in config with hard coded keywords (#9561)
no issue
- removed the `routeKeywords` property from the config and used hard coded keywords.
- removed `routeKeywords` from public configuration API endpoint, as it's no longer used in the Admin.
2018-04-17 10:36:05 +01:00
kirrg001
25cd7c7756 Simplify destroy post API endpoint implementation
no issue

- no need to fetch the post before
- the model implementation does that already
2018-04-06 15:49:25 +02:00
kirrg001
c8f2dd11ba Fixed post.unpublished when deleting all content
no issue

- if you delete all content, we expect two events
  - `post.deleted` and `post.unpublished`
- `post.unpublished` was never triggered, because the api implementation made use of `collection.invoke(`destroy`)`
- what happened?
  - you fetch all posts (columns:id)
  - you destroy the post (only id column is available)
  - the model events are triggered
  - but you have no access to a default set of data
  - the result is that the event handler can't even tell if this is a post or a page
- added a proper test to ensure which events are triggered
2018-04-06 15:49:25 +02:00
kirrg001
bda76acba6 Extended the API object validation
refs #9548

- do not forward `tag.parent` to the model layer
  - the model layer should only know `tag.parent_id`
  - and the API should only expose `tag.parent` (this is an API feature)
  - currently Ghost has a mixture of using `toJSON` and the API validation layer for this
  - we just continue with this for now (no time to fix this)
- disallow sending nested-nested relations
  - unsupported
  - see comment for more information
  - this can cause problems with calling `hasChanged` on relations
- add unit tests
2018-04-05 18:51:58 +02:00
Katharina Irrgang
40d0a745df Multiple authors (#9426)
no issue

This PR adds the server side logic for multiple authors. This adds the ability to add multiple authors per post. We keep and support single authors (maybe till the next major - this is still in discussion)

### key notes

- `authors` are not fetched by default, only if we need them
- the migration script iterates over all posts and figures out if an author_id is valid and exists (in master we can add invalid author_id's) and then adds the relation (falls back to owner if invalid)
- ~~i had to push a fork of bookshelf to npm because we currently can't bump bookshelf + the two bugs i discovered are anyway not yet merged (https://github.com/kirrg001/bookshelf/commits/master)~~ replaced by new bookshelf release
- the implementation of single & multiple authors lives in a single place (introduction of a new concept: model relation)
- if you destroy an author, we keep the behaviour for now -> remove all posts where the primary author id matches. furthermore, remove all relations in posts_authors (e.g. secondary author)
- we make re-use of the `excludeAttrs` concept which was invented in the contributors PR (to protect editing authors as author/contributor role) -> i've added a clear todo that we need a logic to make a diff of the target relation -> both for tags and authors
- `authors` helper available (same as `tags` helper)
- `primary_author` computed field available
- `primary_author` functionality available (same as `primary_tag` e.g. permalinks, prev/next helper etc)
2018-03-27 15:16:15 +01:00
kirrg001
e01b61dcf4 Proper error handling for permissible implementations
no issue

- currently if you would like to edit a resource (e.g. post) and you pass an invalid model id, the following happens
  - permission check calls `Post.permissible`
  - the Post could not find the post, but ignored it and returned `userPermissions:true`
  - then the model layer is queried again and figured out that the post does not exist
- A: there is no need to query the model twice
- B: we needed proper error handling for post and role model
2018-02-21 16:59:48 +01:00
Katharina Irrgang
c6a95c6478
Sorted out the mixed usages of include and withRelated (#9425)
no issue

- this commit cleans up the usages of `include` and `withRelated`.

### API layer (`include`)
- as request parameter e.g. `?include=roles,tags`
- as theme API parameter e.g. `{{get .... include="author"}}`
- as internal API access e.g. `api.posts.browse({include: 'author,tags'})`
- the `include` notation is more readable than `withRelated`
- and it allows us to use a different easier format (comma separated list)
- the API utility transforms these more readable properties into model style (or into Ghost style)

### Model access (`withRelated`)
- e.g. `models.Post.findPage({withRelated: ['tags']})`
- driven by bookshelf

---

Commits explained.

* Reorder the usage of `convertOptions`

- 1. validation
- 2. options convertion
- 3. permissions
- the reason is simple, the permission layer access the model layer
  - we have to prepare the options before talking to the model layer
- added `convertOptions` where it was missed (not required, but for consistency reasons)

* Use `withRelated` when accessing the model layer and use `include` when accessing the API layer

* Change `convertOptions` API utiliy

- API Usage
  - ghost.api(..., {include: 'tags,authors'})
  - `include` should only be used when calling the API (either via request or via manual usage)
  - `include` is only for readability and easier format
- Ghost (Model Layer Usage)
  - models.Post.findOne(..., {withRelated: ['tags', 'authors']})
  - should only use `withRelated`
  - model layer cannot read 'tags,authors`
  - model layer has no idea what `include` means, speaks a different language
  - `withRelated` is bookshelf
  - internal usage

* include-count plugin: use `withRelated` instead of `include`

- imagine you outsource this plugin to git and publish it to npm
- `include` is an unknown option in bookshelf

* Updated `permittedOptions` in base model

- `include` is no longer a known option

* Remove all occurances of `include` in the model layer

* Extend `filterOptions` base function

- this function should be called as first action
- we clone the unfiltered options
- check if you are using `include` (this is a protection which could help us in the beginning)
- check for permitted and (later on default `withRelated`) options
- the usage is coming in next commit

* Ensure we call `filterOptions` as first action

- use `ghostBookshelf.Model.filterOptions` as first action
- consistent naming pattern for incoming options: `unfilteredOptions`
- re-added allowed options for `toJSON`
- one unsolved architecture problem:
  - if you override a function e.g. `edit`
  - then you should call `filterOptions` as first action
  - the base implementation of e.g. `edit` will call it again
  - future improvement

* Removed `findOne` from Invite model

- no longer needed, the base implementation is the same
2018-02-15 10:53:53 +01:00
Austin Burdine
777247cbc7 Contributor Role (#9315)
closes #9314 

* added fixtures for contributor role
* update post api tests to prevent contributor publishing post
* update permissible function in role/user model
* fix additional author code in invites
* update contributor role migration for knex-migrator v3
* fix paths in contrib migration
* ensure contributors can't edit or delete published posts, fix routing tests [ci skip]
* update db fixtures hash
* strip tags from post if contributor
* cleanup post permissible function
* excludedAttrs to ignore tag updates for now (might be removed later)
* ensure contributors can't edit another's post
* migration script for 1.21
2018-02-07 10:46:22 +01:00
Katharina Irrgang
a274d61a8c Removed html usage in error messages (#9444)
no issue

- all of the error message keys were unused
- the only html anchor i found was for mail, but this doesn't change anything, because the admin does only show the message and not the context at the moment
2018-02-07 09:35:48 +01:00
Kevin Ansfield
69d5fac61e
Resurrect the old alpha Koenig editor (#9277)
requires https://github.com/TryGhost/Ghost-Admin/pull/916
- add "enableDeveloperExperiments" config flag
- allow any HTML payload through in the HTML mobiledoc card
  - same approach as taken in the markdown card, running the markup through SimpleDOM isn't necessary and is prone to breaking because of it's limited parsing and error handling abilities

To use Koenig modify your `config.development.json` file and add the following flag to the top-level object:

```
"enableDeveloperExperiments": true
```

If you restart the dev server you will then see a new section on the Labs screen with a Koenig Editor checkbox to enable/disable the editor.

⚠️ The editor is in a _very_ broken state, it's there for developer testing and on-going development. _Do not_ try to use this on any production data!
2018-01-18 15:43:22 +00:00
kirrg001
357ea3dffd 🐛 Fixed showing old release notifications in the about page
no issue

- reported in slack (https://ghost.slack.com/files/U8QV8DXQB/F8TSBQ532/image.png)
- do not expose old release notification
  - e.g. you are on 1.20.0
  - you receive a notification for 1.20.1 to update
  - you update to 1.20.1
- ensure we protect exposing the release notification (compare against blog version)
- protect against wrong formats
- @TODO: the notifications could store a `version` property
  - by that we could use `notification.version` and don't have to match the version in the message
2018-01-18 12:19:55 +01:00
Katharina Irrgang
e480c7c50d 🐛 Reload translations if active theme get's overridden (#9392)
no issue

- discovered while testing
  - activate theme
  - download theme
  - modify theme
  - upload theme
  - override? yes
  - translation files are not reloaded, because the database is up-to-date
- remove un-used events in theme api layer
- trigger event from theme service
2018-01-12 09:19:28 +00:00
David Wolfe
b69b9780a9 Backup endpoint improvements (#9365)
closes #9297

- backup endpoint returns JSON
- allows setting of export filename
- DRY up code
- the endpoint is not documented, no breaking change
2018-01-11 16:03:21 +01:00
Katharina Irrgang
5b77f052d9
Update Notification improvements (#9123)
closes #5071

- Remove hardcoded notification in admin controller
  - NOTE: update check notifications are no longer blocking the admin rendering
  - this is one of the most import changes
  - we remove the hardcoded release message
  - we also remove adding a notification manually in here, because this will work differently from now on
    -> you receive a notification (release or custom) in the update check module and this module adds the notification as is to our database

- Change default core settings keys
  - remove displayUpdateNotification
    -> this was used to store the release version number send from the UCS
    -> based on this value, Ghost creates a notification container with self defined values
    -> not needed anymore

- rename seenNotifications to notifications
  -> the new notifications key will hold both
     1. the notification from the USC
     2. the information about if a notification was seen or not
  - this key hold only one release notification
  - and n custom notifications

- Update Check Module: Request to the USC depends on the privacy configuration
  - useUpdateCheck: true -> does a checkin in the USC (exposes data)
  - useUpdateCheck: false -> does only a GET query to the USC (does not expose any data)
  - make the request handling dynamic, so it depends on the flag
  - add an extra logic to be able to define a custom USC endpoint (helpful for testing)
  - add an extra logic to be able to force the request to the service (helpful for testing)

- Update check module: re-work condition when a check should happen
  - only if the env is not correct
  - remove deprecated config.updateCheck
  - remove isPrivacyDisabled check (handled differently now, explained in last commit)

- Update check module: remove `showUpdateNotification` and readability
  - showUpdateNotification was used in the admin controller to fetch the latest release version number from the db
  - no need to check against semver in general, the USC takes care of that (no need to double check)
  - improve readability of `nextUpdateCheck` condition

- Update check module: refactor `updateCheckResponse`
  - remove db call to displayUpdateNotification, not used anymore
  - support receiving multiple custom notifications
  - support custom notification groups
  - the default group is `all` - this will always be consumed
  - groups can be extended via config e.g. `notificationGroups: ['migration']`

- Update check module: refactor createCustomNotification helper
  - get rid of taking over notification duplication handling (this is not the task of the update check module)
  - ensure we have good fallback values for non present attributes in a notification
  - get rid of semver check (happens in the USC) - could be reconsidered later if LTS is gone

- Refactor notification API
  - reason: get rid of in process notification store
    -> this was an object hold in process
    -> everything get's lost after restart
    -> not helpful anymore, because imagine the following case
      -> you get a notification
      -> you store it in process
      -> you mark this notification as seen
      -> you restart Ghost, you will receive the same notification on the next check again
      -> because we are no longer have a separate seen notifications object
  - use database settings key `notification` instead
  - refactor all api endpoints to support reading and storing into the `notifications` object
  - most important: notification deletion happens via a `seen` property (the notification get's physically deleted 3 month automatically)
    -> we have to remember a seen property, because otherwise you don't know which notification was already received/seen

- Add listener to remove seen notifications automatically after 3 month
  - i just decided for 3 month (we can decrease?)
  - at the end it doesn't really matter, as long as the windows is not tooooo short
  - listen on updates for the notifications settings
  - check if notification was seen and is older than 3 month
  - ignore release notification

- Updated our privacy document
- Updated docs.ghost.org for privacy config behaviour
- contains a migration script to remove old settings keys
2018-01-09 15:20:00 +01:00
Katharina Irrgang
dcb2aa9ad4
Improved forwarding model options in the API layer (#9380)
no issue

- our API layer uses a unit to combine incoming data and options
- e.g. `options.data` is the end result
- we have to take care that we don't pass data into the model layer

Credits: Olivier Arteau
2018-01-09 13:26:09 +01:00
kirrg001
b474fb0d16 Moved ghost-version to lib
refs #9178
2017-12-14 22:14:55 +01:00
kirrg001
1a9a10c82b Moved zip folder, read csv and package-json to lib/fs
refs #9178, refs 849e97640f

- i've reconsidered, these modules belong to lib
- prettify package-json module
2017-12-14 22:07:53 +01:00
kirrg001
82597080be Burn dirty require of api utils
refs #9178

- `checkFileExists` and `checkFileIsValid` where dirty required from web/middleware
- these two functions are only used in the target middleware
- let's move them
2017-12-14 21:25:51 +01:00
kirrg001
a3091a3012 Moved utils constants to lib/constants
refs #9178
2017-12-14 14:13:40 +01:00
kirrg001
bb06a8426d Moved tokens, url safe and safe string utility to lib/security
refs #9178

- we could now also move any crypto usages to lib/security, but no priority
- the main goal is to tidy up our utils folder
2017-12-14 13:38:00 +01:00
kirrg001
4fe7c9d7fc Moved validateRedirects helper to data/validation
refs #9178
2017-12-14 11:55:10 +01:00
Katharina Irrgang
2956c2c247 Use request lib in webhooks api (#9336)
refs #9178

- add retries and timeout
2017-12-14 10:00:34 +00:00
Katharina Irrgang
192ebb1739
Moved labs, auth, permissions, settings, mail, themes to services (#9339)
refs #9178

- move tests as well
2017-12-14 03:01:23 +01:00
kirrg001
f83cbf6117 Moved pipeline/sequence to lib/promise
refs #9178

- continue with killing our global utils folder
- i haven't found any better naming for lib/promise
- so, require single files for now
- instead of doing `promiseLib = require('../lib/promise')`
- we can optimise the requires later
2017-12-13 22:20:02 +01:00
kirrg001
0d65e7f24f Renamed apiUtils to localUtils - consistency change
refs #9178

- we should always use the same naming patterns
2017-12-13 22:14:19 +01:00
kirrg001
485c264c69 Use Promise support which was added in fs-extra 3.x
no issue

- now that we use bluebird globally, we can use the promise support from fs-extra
2017-12-13 20:57:11 +01:00
kirrg001
d83f474ff8 Always require fs-extra - consistency change
no issue
2017-12-13 20:57:11 +01:00
kirrg001
6f6c8f4521 Import lib/common only
refs #9178

- avoid importing 4 modules (logging, errors, events and i18n)
- simply require common in each file
2017-12-12 10:28:13 +01:00
kirrg001
ac2578b419 Moved errors,logging,i18n and events to lib/common
refs #9178
2017-12-12 10:28:13 +01:00
kirrg001
4265afe580 Moved utils/url.js to UrlService
refs #9178

- we have to take care that we don't end up in circular dependencies
  - e.g. API requires UrlService and UrlService needs to require the API (for requesting data)
- update the references
- we would like to get rid of the utils folder, this is/was the most complicated change
2017-12-11 20:05:33 +01:00
Katharina Irrgang
7bcccc71dc
Moved apps into web folder (#9308)
refs #9178

- move express apps to one place (called `web`)
- requires https://github.com/TryGhost/Ghost-Admin/pull/923
- any further improvements are not part of this PR
- this PR just moves the files and ensures the paths are up-to-date
2017-12-06 17:37:54 +01:00
Kevin Ansfield
bffb3dbd90
Webhooks support for subscriber events (#9230)
no issue

Support for http://resthooks.org style webhooks that can be used with Zapier triggers. This can currently be used in two ways:

a) adding a webhook record to the DB manually
b) using the API with password auth and POSTing to /webhooks/ (this is private API so not documented)

⚠️ only _https_ URLs are supported in the webhook `target_url` field 🚨

- add `webhooks` table to store event names and target urls
- add `POST` and `DELETE` endpoints for `/webhooks/`
- configure `subscribers.added` and `subscribers.deleted` events to trigger registered webhooks
2017-11-21 15:43:14 +00:00
Kevin Ansfield
bcc98e5536 Added GET/DELETE /subscribers/email/:email/ endpoints (#9238)
no issue

- useful for managing subscribers via external systems/API calls where it's likely only the e-mail address will be known
- adds `GET /subscribers/email/:email/`
- adds `DELETE /subscribers/email/:email/`
2017-11-14 12:09:41 +01:00
Hannah Wolfe
bcf5a1bc34
Switch to Eslint (#9197)
refs #9178

* Add eslint deps, remove old lint deps
* Add eslint config, remove old lint configs
* Config for server and tests are different
* Tweaked rules to suit us
* Fix linting in codebase - lots of indent changes.
* Fix a real broken test
2017-11-01 13:44:54 +00:00
Aileen Nowak
c8cbbc4eb6 Improved password validation rules (#9171)
refs #9150 

- Moves the password length fn from `models/user` to `data/validation` where the other validator functions live.
- Added password validation rules. Password rules added:
   - Disallow obviously bad passwords: '1234567890', 'qwertyuiop', 'asdfghjkl;' and 'asdfghjklm' for example
   - Disallow passwords that contain the words 'password' or 'ghost'
   - Disallow passwords that match the user's email address
   - Disallow passwords that match the blog domain or blog title
   - Disallow passwords that include 50% or more of the same characters: 'aaaaaaaaaa', '1111111111' and 'ababababab' for example.
- Password validation returns an `Object` now, that includes an `isValid` and `message` property to differentiate between the two error messages (password too short or password insecure).
- Use a catch predicate in `api/authentication` on `passwordReset`, so the correct `ValidationError` will be thrown during the password reset flow rather then an `UnauthorizedError`.
- When in setup flow, the blog title is not available yet from `settingsCache`. We therefore supply it from the received form data in the user model `setup` method to have it accessible for the validation.
2017-10-26 11:01:24 +01:00
Hannah Wolfe
1c382792ef Prev & next post filtering, with primary tag support (#9141)
closes #9140
* Rip out existing prev/next implementation
* New implementation using filter
* Support next/prev in primary_tag
2017-10-13 15:44:39 +01:00
Katharina Irrgang
594b0c2d14 Custom post templates (#9073)
closes #9060

- Update `gscan` - it now extracts custom templates and exposes them to Ghost
- Add `custom_template` field to post schema w/ 1.13 migration
- Return `templates` array for the active theme in `/themes/` requests
- Users with Author/Editor roles can now request `/themes/`
- Front-end will render `custom_template` for posts if it exists, template priority is now:
  1. `post/page-{{slug}}.hbs`
  2. `{{custom_template}}.hbs`
  3. `post/page.hbs`
2017-10-10 13:36:35 +01:00
Katharina Irrgang
d3d04a8e72 Fixed wrong handling of formats param (#9078)
closes #9077

- because of our API layer refactoring, see https://github.com/TryGhost/Ghost/pull/9068
- we can now see that code was written wrong because of this horrible API bug
- this fixes the formats parameter for querying a single post
2017-09-28 13:38:32 +01:00
kirrg001
1e2beface1 Refactored the API layer: do not handle API response after pipelining
no issue

- this has a big underlying problem
- each task in the pipeline can modify the options
- e.g. add a proper permission context
- if we chain after the pipeline, we don't have access to the modified options object
- and then we pass the wrong options into the `toJSON` function of a model
- the toJSON function decides what to return based on options
- this is the easiest solution for now, but i am going to write a spec if we can solve this problem differently
2017-09-28 10:18:18 +01:00
Katharina Irrgang
a6d57d6324 🐛 Fixed missing cache invalidation header when uploading redirects (#9071)
closes #9065
2017-09-27 18:58:33 +01:00
Hannah Wolfe
fcd3c6847b 🐛 Fixed author role permission to change author (#9067)
🐛  Fixed author role permission to change author

no issue

- To be able to fix this bug, we had to solve tasks from #9043
- This bug affects the private / undocumented API only
- Author role users should not be allowed to change the author of a post
2017-09-27 13:12:53 +02:00
Katharina Irrgang
baf8116d6b Removed OAuth leftover: emit event when calling /users/me (#9061)
refs #8342

- was added in this commit 319a388277
- we can remove this event
2017-09-27 04:07:39 +01:00
Hannah Wolfe
b468d6dbe2 Support for attribute-based permissions (#9025)
refs #8602

- Add the wiring to pass attributes around the permission system
- Allows us to get access to the important "unsafe" attributes that are changing
- E.g. status for posts
- This can then be used to determine whether a user has permission to perform an attribute-based action
- E.g. publish a post (change status)
2017-09-26 18:06:14 +02:00
Katharina Irrgang
a80a09e483 🔥 Removed public API endpoint to fetch users by email address (#9059)
no issue

- our public API is still a beta/labs feature
- from api.ghost.org
  > The API is still under very (very) heavy development and subject to regular breaking changes.
- users should expect breaking changes in any release (independent from semver versions)
- the public user API never returns any email addresses to decrease the information we expose
- there is no need to keep the support fetching a user by email address
2017-09-26 16:42:58 +01:00
Katharina Irrgang
22017b8ede 🎨 Backup redirects.json file before overriding (#9051)
refs #9028

- if you upload a redirects file and a redirects file exists already, we backup this file to `data/redirects-YYYY-MM-DD-HH-mm-ss.json`
- decrease chance of random test failures by not comparing date format with seconds
2017-09-25 18:35:57 +01:00
Katharina Irrgang
472858f262 Removed unused API endpoint: POST /users (#9052)
no issue

- this endpoint does not exist anymore
- if you want to add a new user, you have to invite him via the invites API
- on invite accept, the user is inserted
2017-09-25 16:58:14 +01:00
Katharina Irrgang
d943fc7cc9 Allow Upload/Download of redirects.json (#9029)
refs #9028

- add two new endpoints for uploading/downloading the redirects (file based)
- reload/re-register redirects on runtime
- migration for 1.9 to add permissions for redirects download/upload
2017-09-21 16:01:03 +01:00
kirrg001
79fead5516 Removed private configuration endpoint
refs https://github.com/TryGhost/Ghost/issues/8859

- We don't need the config option for Unsplash anymore
- The private endpoint (/configuration/private) was introduced for Unsplash
2017-09-20 11:44:47 +01:00
Katharina Irrgang
edf2348394 Improved log output for welcome email error (#9016)
* Improved log output for welcome email error

no issue

- if Ghost is unable to send a welcome email, the server log printe a huge error log
- the reason was that each component wrapped the original error into a new error instance
  - so the stack grows and grows
- the golden rule should always be: the smallest/lowest component should instanitate a specifc error
  - the caller can expect to receive a custom Ghost error

* Tidy up error messages for mail failures and fix tests

- We never use "Error:" notation in our translations
- Make the error messages consistent and show a reason if possible
2017-09-19 14:24:20 +01:00
Katharina Irrgang
7b4c3fc085 Removed defunct Ghost OAuth code (#9014)
closes #8342

- no need to add a migration, because when we'released 1.0, OAuth was never an option
- it was disabled in April, 1.0-beta was released in June
- remove all remote authentication code
2017-09-18 13:01:58 +01:00
kirrg001
79959d9581 🐛 Fixed public api access on custom domain
no issue

- if you blog runs on a custom domain, but your admin panel is configured using a different domain
  -> Ghost losts the origin header
- we had this situation once with pretty urls (your request get's redirected from /posts to /posts/, see https://github.com/TryGhost/Ghost/pull/8094)
- we've moved all our redirect logic to Ghost and ran into the same situation
- i've added proper test to ensure it won't happen again
2017-09-14 07:55:14 +07:00
Hannah Wolfe
4237446277 Misc cleanup & consistency amends (#9002)
no issue

- Consistent naming for postLookup
   - makes it easier to search and inspect the various usages
- Cleanup unneeded code
- Make res.render calls more consistent
- add some consistency to the calls to res.render
- Remove ancient reference to dataProvider
- Let's call it models everywhere now...
- Use consistent formatting across the API
- we're no longer using alignment in vars
- Misc other consistency changes in API
- always refer to local utils as apiUtils
- logical grouping of requires - dependencies, utils, "lib common" etc
- use xAPI to refer to API endpoints, e.g. mailAPI, settingsAPI for clarity
2017-09-12 17:31:14 +02:00
Vikas
8606ea22a2 Normalize docs URLs (#8949)
no issue

v1.0.0 is no longer the standard in the docs, so I updated all of the URLs containing it with v1
Note: I tried squashing commits, but failed. I'll try again in the future with throwaway changes
Secondary Note: I tested most of the URLs listed and got no 404s!
2017-08-31 11:45:59 +02:00
David Wolfe
c3fcb3105f Add ghost-backup client to trigger export (#8911)
no issue
- adds a ghost-backup client
- adds a client authenticated endpoint to export blog for ghost-backup client only
- allows some additional overrides during import
- allows for an import by file to override locking a user and double hashing the password
2017-08-22 11:15:40 +01:00
Hannah Wolfe
4f3a8f6b38 API express app routing & middleware improvements (#8883)
no issue

- Split routes out from the API app 🎨
    - Use the same pattern as the blog app
    - General cleanup/unification across all of the `app.js` files

- Split middleware config out from API routes
    - Logical groupings make it easier to see WTF is going on 😬
2017-08-17 17:52:58 +07:00
Kevin Ansfield
d064eda229 Add configuration/private endpoint and settings for Unsplash (#8895)
refs #8859

- adds new `configuration/private` endpoint for exposing config that should not be accessible without authentication
- adds `unsplashAPI` to private config
- adds empty `unsplash` config to default settings
2017-08-15 15:59:16 +01:00
Hannah Wolfe
b6b299a8f7 Used ghost-ignition.debug, removed debug dep (#8881)
no issue

- Upgraded ghost-ignition
- Use debug from ghost-ignition everywhere in the code base
- Remove debug dependency
- Fixed random typo in Gruntfile.js
2017-08-15 18:29:27 +07:00
kirrg001
91f36fc241 🐛 invite existing users
refs #8692

- protect invite endpoint
2017-07-18 18:24:36 +01:00
Hannah Wolfe
117aeaa61d Pass clientExtensions to Ghost Admin (#8617)
no issue

- pass through this additional config if it exists
2017-06-22 20:45:35 +02:00
Kevin Ansfield
88ba07f657 📖 shorten mail config docs links (#8592)
no issue
- URL on docs.ghost.org has been shortened for more readable display in
errors
2017-06-15 18:04:48 +01:00
Katharina Irrgang
f5ef8dbb52 📖 replace any support.ghost.org link (#8555)
refs #7421
2017-06-08 20:34:20 +01:00
Katharina Irrgang
9023ff0b68 😝 replace removeSync by sync (#8513)
refs #8510

- nothing to see here!
2017-06-01 19:11:50 +07:00
Hannah Wolfe
3e60941054 Add ?formats param to Posts API (#8305)
refs #8275
- Adds support for `formats` param
- Returns `html` by default
- Can optionally return other formats by providing a comma-separated list
2017-05-30 11:40:39 +01:00
Katharina Irrgang
1f37ff6053 🎨 refactor the importer (#8473)
refs #5422

- we can support null titles after this PR if we want
- user model: fix getAuthorRole
- user model: support adding roles by name
- we support this for roles as well, this makes it easier when importing related user roles (because usually roles already exists in the database and the related id's are wrong e.g. roles_users)
- base model: support for null created_at or updated_at values
- post or tag slugs are always safe strings
- enable an import of a null slug, no need to crash or to cover this on import layer
- add new DataImporter logic
    - uses a class inheritance mechanism to achieve an easier readability and maintenance
    - schema validation (happens on model layer) was ignored
    - allow to import unknown user id's (see https://github.com/TryGhost/Ghost/issues/8365)
    - most of the duplication handling happens on model layer (we can use the power of unique fields and errors from the database)
- the import is splitted into three steps:
  - beforeImport
    --> prepares the data to import, sorts out relations (roles, tags), detects fields (for LTS)
  - doImport
    --> does the actual import
  - afterImport
    --> updates the data after successful import e.g. update all user reference fields e.g. published_by (compares the imported data with the current state of the database)
- import images: markdown can be null
- show error message when json handler can't parse file
- do not request gravatar if email is null
- return problems/warnings after successful import
- optimise warnings in importer
- do not return warnings for role duplications, no helpful information
- error handler: return context information of error
- we show the affected json entries as one line in the UI
- show warning for: detected duplicated tag
- schema validation: fix valueMustBeBoolean translation
- remove context property from json parse error
2017-05-23 17:18:13 +01:00
Aileen Nowak
1b965fab95 🎨 Move scheduling and storage in adapters/ (#8435)
no issue

Move `core/server/scheduling` to `core/server/adapters/scheduling` and
`core/server/storage` to `core/server/adapters/storage`
2017-05-15 12:52:01 +02:00
Katharina Irrgang
4e2474a018 🎨 settings inconsistency (#8381)
no issue
- replace camelCase settings keys with underscore_case for consistency
- discussed here https://github.com/TryGhost/Ghost-Admin/pull/661#discussion_r112939982
2017-04-24 18:41:00 +01:00
Katharina Irrgang
c93f03b87e post update collision detection (#8328) (#8362)
closes #5599

If two users edit the same post, it can happen that they override each others content or post settings. With this change this won't happen anymore.

 Update collision for posts
- add a new bookshelf plugin to detect these changes
- use the `changed` object of bookshelf -> we don't have to create our own diff
- compare client and server updated_at field
- run editing posts in a transaction (see comments in code base)

🙀  update collision for tags
- `updateTags` for adding posts on `onCreated` - happens after the post was inserted
   --> it's "okay" to attach the tags afterwards on insert
   --> there is no need to add collision for inserting data
   --> it's very hard to move the updateTags call to `onCreating`, because the `updateTags` function queries the database to look up the affected post
- `updateTags` while editing posts on `onSaving` - all operations run in a transactions and are rolled back if something get's rejected

- Post model edit: if we push a transaction from outside, take this one

  introduce options.forUpdate
- if two queries happening in a transaction we have to signalise knex/mysql that we select for an update
- otherwise the following case happens:
  >> you fetch posts for an update
  >> a user requests comes in and updates the post (e.g. sets title to "X")
  >> you update the fetched posts, title would get overriden to the old one

use options.forUpdate and protect internal post updates: model listeners
- use a transaction for listener updates
- signalise forUpdate
- write a complex test

use options.forUpdate and protect internal post updates: scheduling
- publish endpoint runs in a transaction
- add complex test
- @TODO: right now scheduling api uses posts api, therefor we had to extend the options for api's
  >> allowed to pass transactions through it
  >> but these are only allowed if defined from outside {opts: [...]}
  >> so i think this is fine and not dirty
  >> will wait for opinions
  >> alternatively we have to re-write the scheduling endpoint to use the models directly
2017-04-19 14:53:23 +01:00
Katharina Irrgang
1bcd25fdbf 🎨 delete all content: do not delete subscribers (#8282)
closes #7875
2017-04-05 21:57:41 +01:00
Katharina Irrgang
817b8d09ca 😱 🎨 Refactor storage adapter (#8229)
refs #7687

There are four main changes in this PR:

we have outsourced the base storage adapter to npm, because for storage developers it's annoying to inherit from a script within Ghost
we hacked theme storage handling into the default local storage adapter - this was reverted, instead we have added a static theme storage here
use classes instead of prototyping
optimise the storage adapter in general - everything is explained in each commit

----

* rename local-file-store to LocalFileStorage

I would like to keep the name pattern i have used for scheduling.
If a file is a class, the file name reflects the class name.
We can discuss this, if concerns are raised.

* Transform LocalFileStorage to class and inherit from new base

- inherit from npm ghost-storage-base
- rewrite to class
- no further refactoring, happens later

* Rename core/test/unit/storage/local-file-store_spec.js -> core/test/unit/storage/LocalFileStorage_spec.js

* Fix wrong require in core/test/unit/storage/LocalFileStorage_spec.js

* remove base storage and test

- see https://github.com/kirrg001/Ghost-Storage-Base
- the test has moved to this repo as well

* Use npm ghost-storage-base in storage/index.js

* remove the concept of getStorage('themes')

This concept was added when we added themes as a feature.
Back then, we have changed the local storage adapter to support images and themes.
This has added some hacks into the local storage adapters.
We want to revert this change and add a simple static theme storage.

Will adapt the api/themes layer in the next commits.

* Revert LocalFileStorage

- revert serve
- revert delete

* add storagePath as property to LocalFileStorage

- define one property which holds the storage path
- could be considered to pass from outside, but found that not helpful, as other storage adapters do not need this property
- IMPORTANT: save has no longer a targetDir option, because this was used to pass the alternative theme storage path
- IMPORTANT: exists has now an alternative targetDir, this makes sense, because
  - you can either ask the storage exists('my-file') and it will look in the base storage path
  - or you pass a specific path where to look exists('my-file', /path/to/dir)

* LocalFileStorage: get rid of store pattern

- getUniqueFileName(THIS)
- this doesn't make sense, instances always have access to this by default

* Add static theme storage

- inherits from the local file storage, because they both operate on the file system
- IMPORTANT: added a TODO to consider a merge of themes/loader and themes/storage
- but will be definitely not part of this PR

* Use new static theme storage in api/themes

- storage functions are simplified!

* Add https://github.com/kirrg001/Ghost-Storage-Base as dependency

- tarball for now, as i am still testing
- will release if PR review get's accepted

* Adapt tests and jscs/jshint

* 🐛  fix storage.read in favicon utility

- wrong implementation of error handling

* 🎨  optimise error messages for custom storage adapter errors

* little renaming in the storage utlity

- purpose is to have access to the custom storage instance and to the custom storage class
- see next commit why

* optimise instanceof base storage

- instanceof is always tricky in javascript
- if multiple modules exist, it can happen that instanceof is false

* fix getTargetDir

- the importer uses the `targetDir` option to ensure that images land in the correct folder

* ghost-storage-base@0.0.1 package.json dependency
2017-04-05 15:10:34 +01:00
Hannah Wolfe
495eee7747 Use "mounting" concept for active theme (#8193)
no issue

🔥 Remove DIRTY HACK for API
- this is no longer needed, because themes get mounted in every case

 Switch to concept of 'mounted' theme
- check if active theme is mounted
- if not, mount it
- mounting is a function OF the active theme

🎨 Move theme middleware to theme module

🎨 Update theme middleware function names
- update the function names and comments to be more representative of their current functions
- this was pretty old and out of date!

🚨 Fixup tests for middleware
- ensure the objects match what we expect
- based partially on theme docs

Update TODO
2017-03-21 10:03:09 +01:00
Hannah Wolfe
fad0ac7213 🐛🎨 Theme API minor improvements (#8197)
no issue

🎨 🐛 Ensure cache is updated correctly for themes
- Insure the cache invalidation headers are always set correctly for the themes API

📖  Theme API comments / function naming
- this is an update for clarity, so we can see what further improvements can be made

🐛 🎨 Add permissions to themes.browse
2017-03-20 19:02:44 +01:00
Katharina Irrgang
974adee932 🔥 remove fileStorage option (#8144)
refs #8032

- this was used to disable the upload image functionality in Ghost-Admin
- we no longer need this boolean, because people can add their own storage adapter
2017-03-14 10:31:33 +00:00
Hannah Wolfe
b06f03b370 New fully-loaded & validated activeTheme concept (#8146)
📡 Add debug for the 3 theme activation methods
There are 3 different ways that a theme can be activated in Ghost:

A. On boot: we load the active theme from the file system, according to the `activeTheme` setting
B. On API "activate": when an /activate/ request is triggered for a theme, we validate & change the `activeTheme` setting
C. On API "override": if uploading a theme with the same name, we override. Using a dirty hack to make this work.

A: setting is done, should load & validate + next request does mounting
B: load is done, should validate & change setting + next request does mounting
C: load, validate & setting are all done + a hack is needed to ensure the next request does mounting

 Validate w/ gscan when theme activating on boot
- use the new gscan validation validate.check() method when activating on boot

 New concept of active theme
- add ActiveTheme class
- make it possible to set a theme to be active, and to get the active theme
- call the new themes.activate() method in all 3 cases where we activate a theme

🎨 Use new activeTheme to simplify theme code
- make use of the new concept where we can, to reduce & simplify code
- use new hasPartials() method so we don't have to do file lookups
- use path & name getters to reduce use of getContentPath etc
- remove requirement on req.app.get('activeTheme') from static-theme middleware (more on this soon)

🚨 Improve theme unit tests (TODO: fix inter-dep)
- The theme unit tests are borked! They all pass because they don't test the right things.
- This improves them, but they are still dependent on each-other
- configHbsForContext tests don't pass if the activateTheme tests aren't run first
- I will fix this in a later PR
2017-03-13 21:13:17 +01:00
Katharina Irrgang
7556e68c48 🎨 Ghost bootstrap: optimise requires (#8121)
* 🎨  Ghost bootstrap: optimise requires

no issue

- require as less as possible on bootstrap

* do not load icojs on bootstrap
2017-03-13 20:07:12 +00:00
Katharina Irrgang
c9f551eb96 suspend user feature (#8114)
refs #8111 
- Ghost returns now all (active+none active) users by default
- protect login with suspended status
- test permissions and add extra protection for suspending myself
- if a user is suspended and tries to activate himself, he won't be able to proceed the login to get a new token
2017-03-13 12:03:26 +00:00
Hannah Wolfe
b2f1d0559b Themes API activation permissions & validation (#8104)
refs #8093

 Add activate theme permission
- add permission to activate themes
- update tests
- also: update tests for invites
TODO: change how the active theme setting is updated to reduce extra permissions

 Move theme validation to gscan
- add a new gscan validation method and use it for upload
- update activate endpoint to do validation also using gscan
- change to using SettingsModel instead of API so that we don't call validation or permissions on the settings API
- remove validation from the settings model
- remove the old validation function
- add new invalid theme message to translations & remove a bunch of theme validation related unused keys

📖  Planned changes

🚨 Tests for theme activation API endpoint
🐛 Don't allow deleting the active theme

🚫 Prevent activeTheme being set via settings API
- We want to control how this happens in future.
- We still want to store the information in settings, via the model.
- We just don't want to be able to change this info via the settings edit endpoint

🐛  Fix warnings for uploads & add for activations
- warnings for uploads were broken in f8b498d
- fix the response + adds tests to cover that warnings are correctly returned
- add the same response to activations + more tests
- activations now return a single theme object - the theme that was activated + any warnings

🎨 Improve how we generate theme API responses
- remove the requirement to pass in the active theme!
- move this to a specialist function, away from the list

🎨 Do not load gscan on boot
2017-03-13 12:44:44 +01:00
Hannah Wolfe
94d53cf5fb Move activation to themes endpoint (#8093)
no issue
- browse will now include the correct activated theme again
- PUT /theme/:name/activate will activate a theme
- tests now read from a temp directory not content/themes
- all tests check errors and responses
2017-03-08 10:46:03 +00:00
Katharina Irrgang
773eb92960 🐛 fix cors middleware (#8094)
no issue

- same fix as https://github.com/TryGhost/Ghost/pull/8066
- just for master
2017-03-03 16:41:20 +00:00
Hannah Wolfe
a5ab2ffc13 🔥 🎨 No more updateSettingsCache (#8090)
no issue

🔥 Remove unnecessary cache update
🎨 simplify updateSettingsCache()
🎨 Simplify readSettingsResult
- although this is more code, it's now much clearer what happens in the two cases
🎨 Don't use readSettingResult for edit
🎨 Simplify updateSettingsCache further
🔥 Remove now unused readSettingsResult
🎨 Change populateDefault to return all
🎨 Move the findAll call out of updateSettingsCache
🔥 Remove updateSettingsCache!!
🎨 Restructure init & finish up settingsCache
- move initialisation into settingsCache.init AT LAST
- change settingCache to use cloneDeep, so that the object can't be modified outside of the functions
- add lots of docs to settings cache
🎨 Cleanup db api endpoints
🔥 Don't populate settings in migrations
2017-03-02 23:00:01 +01:00
Katharina Irrgang
9fafc38b79 🎨 deny auto switch (#8086)
* 🎨  deny auto switch

no issue

- deny auth switch after the blog was setup
- setup completed depends on the status of the user right now, see comments

* Updates from comments

- re-use statuses in user model
- update error message
2017-03-02 19:50:58 +00:00
Hannah Wolfe
f8b498d6e7 🔥 No more availableThemes (#8085)
no issue

🎨 Switch themes API to use config.availableThemes
- this gets rid of the only places where settings.availableThemes are used

🔥 Get rid of settings.availableThemes
- this is no longer used anywhere
- also get rid of every related call to updateSettingsCache

🔥 Replace config.availableThemes with theme cache
- Creates a tailor-made in-memory cache for themes inside the theme module
- Add methods for getting & setting items on the cache
- Move all references to config.availableThemes to use the new cache
- This can be abstracted later to support other kinds of caches?

🎨 Start improving theme lib's API
Still TODO: simplifying/clarifying:
- what is the structure of the internal list
- what is the difference between a package list, and a theme list?
- what is the difference between reading a theme and loading it?
- how do we update the theme list (add/remove)
- how do we refresh the theme list? (hot reload?!)
- how do we get from an internal list, to one that is sent as part of the API?
- how are we going to handle theme storage: read/write, such that the path is configurable

🎨 Use themeList consistently
🎨 Update list after storage
2017-03-02 17:53:48 +01:00
Hannah Wolfe
c70fbc2c7e 🎨 Collect & simplify package utils (#8080)
closes #8056

🎨 Collect together the package-related utils
- read directory actually reads a directory of packages
- parse package json is very tighly related to this

🎨 Move filterPaths -> packages.filterPackages
- this function is related to packages, not settings
- move the function to the new utils/packages
- add 100% test coverage

🎨 Simplify filterPackages code
🎨 Simplify reading of packages & themes
- This massively reduces all the complex code in the read packages & themes utils
- Added full test coverage

🎨 Improve & clarify active prop in filterPackages
- active is returned from API endpoints to combine data from multiple sources
- see https://github.com/TryGhost/Ghost/pull/8064#discussion_r103514810

🎨 Better error handling
🔥 Temporarily remove custom error templates
- we will reimplement this later when we have got a better concept of loading the active theme in place
- refs #8079
2017-03-01 14:09:31 +01:00
Katharina Irrgang
fa38257170 🐛 🎨 old accesstokens are not cleaned up (#8065)
closes #8035
- create auth/utils
- use authUtils.createTokens for all cases
- decrease the expiry of the old access token before creating a new one
2017-03-01 10:12:03 +00:00
Hannah Wolfe
690ff05588 🔥 🎨 Themes & settings misc cleanup (#8061)
no issue

🔥 remove unused loadThemes API method
🚨 Add tests for themes.readOne
🔥 Don't update settings cache for imports
- this isn't needed as of #8057
- settings.edit fires an event, that will result in the update happening automatically
🎨 Move validation to themes
- slowly collecting all theme-related code together
🔥 Reduce DEBUG output
- all this info is a bit tooooo much!
2017-02-27 23:30:49 +01:00
Hannah Wolfe
63723aa36a 🎨 Move settings cache & cleanup settings API (#8057)
closes #8037

🔥 Remove API-level default settings population
- This is a relic!
- We ALWAYS populate defaults on server start therefore this code could never run.
- This was a lot of complicated code that wasn't even needed!!

🎨 Move settings cache
- Move settings cache to be its own thing
- Update all references
- Adds TODOs for further cleanup

🎨 Create settings initialisation step
- Create new settings library, which will eventually house more code
- Unify the interface for initialising settings (will be more useful later)
- Reduce number of calls to updateSettingsCache
2017-02-27 16:53:04 +01:00
Katharina Irrgang
319a388277 ghost auth: sync email (#8027)
*   ghost auth: sync email

refs #7452

- sync email changes in background (every hour right now)
- sync logged in user only!
- no sync if auth strategy password is used
- GET /users/me is triggered on every page refresh
- added TODO to support or add long polling for syncing data later
- no tests yet on purpose, as i would like to get a basic review first

* 🐩  use events

- remember sync per user
2017-02-23 18:04:24 +00:00
Hannah Wolfe
fe90cf2be2 Theme loading part 1 (#7989)
no issue

*  Add new server start & stop events
* 🔥 Get rid of unused availableApps concept
- when we need an API endpoint for a list of apps, we'll build one 😝
*  Move theme loading into a module
- move loading from API method to a module method and use as needed
- wire up read one vs read all as per LTS
- read one (the active theme) on boot, and read the rest after
- fudge validation - this isn't all that helpful
* Settings API tests need to preload themes
- this used to automatically happen as part of loading settings
- now we need to trigger this to happen specifically for this test
2017-02-22 00:26:19 +01:00
Hannah Wolfe
294561cac7 Theme Browse API endpoint (#8022)
no issue
- adds `GET /themes/` endpoint for requesting all themes
2017-02-21 14:59:03 +00:00
Katharina Irrgang
29fb68137f 🎨 increase token expiry (#7971)
refs #5202
- please read https://github.com/TryGhost/Ghost/issues/5202#issuecomment-278934768
2017-02-10 13:35:58 +00:00
Katharina Irrgang
72e9f0a95e 🐛 delete all content if subscriber is associated with a post (#7962)
closes #7875
- we need to delete the subscribers before deleting the posts
2017-02-08 15:40:47 +00:00
Vivek Kannan
4718d55630 Fixed issue where all only a few subscribers where exported via CSV. (#7925)
closes #7850

- export subscribers to CSV now uses .findAll() instead of .findPage()
2017-02-08 11:37:09 +01:00
Katharina Irrgang
f3d16352b1 🎨 😎 config env usages (#7929)
refs #7488

- remove all ugly env checks
- rather use config properties
- replace process.env.NODE_ENV by config.get('env')
2017-02-03 18:25:39 +00:00
Katharina Irrgang
a68592a6b9 🔥 remove forceAdminSSL and urlSSL, add admin url (#7937)
* 🔥  kill apiUrl helper, use urlFor helper instead

More consistency of creating urls.
Creates an easier ability to add config changes.

Attention: urlFor function is getting a little nesty, BUT that is for now wanted to make easier and centralised changes to the configs.
The url util need's refactoring anyway.

* 🔥  urlSSL

Remove all urlSSL usages.
Add TODO's for the next commit to re-add logic for deleted logic.

e.g.

- cors helper generated an array of url's to allow requests from the defined config url's -> will be replaced by the admin url if available
- theme handler prefered the urlSSL in case it was defined -> will be replaced by using the urlFor helper to get the blog url (based on the request secure flag)

The changes in this commit doesn't have to be right, but it helped going step by step.
The next commit is the more interesting one.

* 🔥    remove forceAdminSSL, add new admin url and adapt logic

I wanted to remove the forceAdminSSL as separate commit, but was hard to realise.
That's why both changes are in one commit:

1. remove forceAdminSSL
2. add admin.url option

- fix TODO's from last commits
- rewrite the ssl middleware!
- create some private helper functions in the url helper to realise the changes
- rename some wordings and functions e.g. base === blog (we have so much different wordings)
- i would like to do more, but this would end in a non readable PR
- this commit contains the most important changes to offer admin.url option

* 🤖  adapt tests

IMPORTANT
- all changes in the routing tests were needed, because each routing test did not start the ghost server
- they just required the ghost application, which resulted in a random server port
- having a random server port results in a redirect, caused by the ssl/redirect middleware

* 😎  rename check-ssl middleware

* 🎨  fix theme-handler because of master rebase
2017-02-03 18:13:22 +00:00
Katharina Irrgang
0201c431d7 🔥 do not store settings in config (#7924)
* 🎨  🔥  do not store settings in config and make settings cache easier available

- remove remembering settings value in theme config
- if we need a cache value, we are asking the settings cache directly
- instead of settings.getSettingSync we use settings.cache.get

- added TODO:
  - think about moving the settings cache out of api/settings
  - we could create a folder named cache cache/settings
  - this settings cache listens on model changes for settings
  - decoupling

* 🔥  remove timezone from config

- no need to store in overrides config and in defaults settings

* 🎨  context object helper

- replace config.get('theme') by settings cache

* 🎨  replace config.get('theme') by settings.cache.get

* 🎨  adapt tests

* fixes from comments
2017-02-03 13:15:11 +00:00
Katharina Irrgang
16f5d1fdaf 🎨 add urlFor('admin') and increase usage of urlFor helper (#7935)
refs #7488

- to be able to refactor the url configuration in ghost, we need to go step by step making this possible
- reduce the usage of forceAdminSSL
- add a urlFor('admin') helper, which returns the admin url + path e.g. http://my-blog.com/blog/ghost
- increase usage of urlFor helper
- do not expose getBaseUrl, use urlFor('home') (home === blog)
2017-02-02 18:51:35 +00:00