Commit Graph

1481 Commits

Author SHA1 Message Date
Naz Gargol
a0712d23e8
Shortened admin key length (#10418)
refs #10156

- Updated ApiKey model to use shorter secrets for admin keys
2019-01-24 13:46:33 +00:00
Nazar Gargol
5fbad09a56 Modified Admin API key output format
refs #9865

- Changed key format to {id}:{secret} so API consumer only has to worry about copying a single value during setup
- Updated key expiration time in getValidAdminToken test helper to match server side expiration check
2019-01-23 17:18:50 +00:00
Katharina Irrgang
5921d9ce4a
Removed unit tests who use the database (#10407)
refs #9178

## Rule

- do not use the database in unit tests
- re-add correct unit tests if you work on something which is not tested properly (we have to bring them back at some point, but without using the database)
2019-01-22 13:36:30 +01:00
Katharina Irrgang
4acc375fb6
Bumped knex & bookshelf dependencies (#10404)
refs #9389, refs #9248

- https://github.com/bookshelf/bookshelf/releases/tag/0.14.0
- Bookshelf has fixed it's previous attr handling, see https://github.com/bookshelf/bookshelf/pull/1848
- SQlite3 double slashes was merged into knex and released 👻tgriesser/knex@c746dea
2019-01-21 21:53:11 +01:00
Katharina Irrgang
fb044e6d88
Bumped sinon from 4.4.6 to 7.3.2 (#10400)
refs #9389

- https://github.com/sinonjs/sinon/blob/master/CHANGELOG.md

Breaking changes for Ghost:

- no need to create a sandbox anymore, each file get's it's own sandbox
- just require sinon and use this sandbox
- you can still create separate sandboxes with .createSandbox
- reset single stubs: use .resetHistory instead of .reset

This is a global replace for any sandbox creation.

---

From https://sinonjs.org/releases/v7.2.3/sandbox/

> Default sandbox
> Since sinon@5.0.0, the sinon object is a default sandbox. Unless you have a very advanced setup or need a special configuration, you probably want to just use that one.
2019-01-21 17:53:44 +01:00
Nazar Gargol
c3e22b6ab9 🐛 Fixed forced page filter not being respected in Content API
closes https://github.com/TryGhost/Ghost/issues/10395, https://github.com/TryGhost/Ghost/issues/10390

- The grouping was introduced due to a flasy conjunction of last filter in 'or' group with enforced page filter, due to this the rest of filters in the 'or' group were bypassed
- Additional grouping added here increases the possiblity of https://github.com/NexesJS/NQL-Lang/issues/7 bug happening
2019-01-21 13:26:45 +00:00
Katharina Irrgang
a927aecc3f
🐛 Fixed prev_next helper when using Content API v2 (#10397)
closes #10389
2019-01-21 11:48:18 +01:00
Fabien O'Carroll
6f87f2af93 Fixed img_url to work with protocol relative urls
closes #10391

- We use "relative protocol" urls for gravatar images, which were
  incorrectly getting treated as relative path urls.

- Refactored getBlogUrl calls into const
2019-01-21 11:41:23 +01:00
kirrg001
1126997244 Added easy way to enable admin api key authentication
refs #9865

- small refactoring to make both session and admin api key handling similar
- admin api key authentication is still disabled, but easy to enable
- added proof test how to authenticate using admin api keys
2019-01-18 18:57:58 +01:00
kirrg001
e90148e7c3 Switched to use new implementation of authorizeAdminApi
refs #9865

- see code comments
2019-01-18 17:54:08 +01:00
kirrg001
3f758c6a0a Protected against empty admin api key
refs #9865
2019-01-18 17:54:04 +01:00
kirrg001
1b5b95e198 Renamed authenticateAdminApiKey to authenticate for admin api key auth
refs #9865

- the outer authentication layer wants a consistent interface of each authentication package
  - admin.authenticate
  - session.authenticate

- furthermore, there is no need to put the full feature into the exposed function name
2019-01-18 17:30:07 +01:00
kirrg001
462865981e Updated admin api key authentication to expect api key id in payload
refs #9865

- see https://github.com/TryGhost/Ghost/blob/2.11.1/core/server/lib/members/index.js#L52
- consistency
2019-01-18 17:22:19 +01:00
Fabien O'Carroll
1c56221d80 Added API Key auth middleware to v2 Admin API (#10006)
refs #9865

- Added `auth.authenticate.authenticateAdminApiKey` middleware
  - accepts signed JWT in an `Authorization: Ghost [token]` header
  - sets `req.api_key` if the token is valid
- Updated `authenticatePrivate` middleware stack for v2 admin routes
2019-01-18 12:45:06 +00:00
Fabien O'Carroll
809a167a55 Updated permissible methods for hasApiKeyPermissions (#9970)
refs #9865

This updates all current permissible methods to use the new function
signature which includes the hasApiKeyPermissions parameter. It also
makes sure that the hasApiKeyPermissions argument is taken into account
whenever checking before returning a resolved promise.

To be continued. This is just a tiny part of the big picture. None of these changes are fully committed to stay as they are.
2019-01-18 13:39:53 +01:00
kirrg001
873d63dc5d Fixed unit test: services/permissions/parse-context_spec.js
no issue
2019-01-18 13:16:28 +01:00
Fabien O'Carroll
fc73cd71bb Updated permissions service to handle api keys (#9967)
refs #9865

- Enabled the permissions module to lookup permissions based on an api_key id.
- Updated the "can this" part of the permissions service to load permissions for any api key in the context, and correctly use that to determine whether an action is permissible. It also updates the permissible interface that models can implement to pass in the hasApiKeyPermissions param.
2019-01-18 11:17:11 +00:00
Fabien O'Carroll
42a1313bff
Removed members internal app (#10379)
no-issue

* Mounted members static router directly to parentApp

* Removed members internal app
2019-01-17 12:03:40 +01:00
Hannah Wolfe
909f9012ed
Updated docs links to best equivalents (#10386)
* Updated docs links to best equivalents
   - Our documentation has been overhauled, this updates the all the old links sprinkled through Ghost
* Update integrity hash
2019-01-17 06:57:37 +00:00
Hannah Wolfe
d58ecda9a9 🐛 Fixed responsive images for .icos
closes #10301

- we allow .ico files for icons, but we cannot resize these specialist files at present
2019-01-15 17:41:25 +00:00
Katharina Irrgang
732f97a074
De-aliased api versions in codebase (#10375)
closes #10357
2019-01-14 19:49:55 +01:00
Naz Gargol
48d6e7298a
Refactored content-disposition header handling in API v2 (#10374)
closes https://github.com/TryGhost/Ghost/issues/10331

- Left only a filename part to be handled by controller configuration, the rest was extracted to more generic headers layer
2019-01-14 18:05:16 +00:00
Fabien O'Carroll
9837df0b6f
Added brute protection to content api (#10362)
* Revert "Removed brute force middleware form content api (#10353)"

This reverts commit 63c8c310fb.

* Updated content api spam prevention to use memory store

* Used TooManyRequestsError instead of InternalServer
2019-01-14 14:11:48 +01:00
Naz Gargol
aa49f77504
🐛 Fixed url not being immediately available after post publish (#10361)
🐛  Fixed URL not being immediately available after post publish

closes https://github.com/TryGhost/Ghost/issues/10360

- This change makes sure posts and pages URLs are populated in
synchronous manner
- Further cleanup and restructuring of this hacky solution is planned to
be done in https://github.com/TryGhost/Ghost/issues/10360
2019-01-11 10:30:36 +00:00
Fabien O'Carroll
63c8c310fb
Removed brute force middleware form content api (#10353)
* Removed brute force middleware form content api

no-issue

This is just temporary

* Skipped brute test for content api
2019-01-09 11:38:03 +01:00
Fabien O'Carroll
76bb40b7c0
Added clause in validation for include to not error (#10350)
* Added clause in validation for include to not error

refs #10337

Here we forgo erroring when an invalid property for include is sent, and
instead remove the invalid properties.

* Fixed authors test

* Fixed validators tests
2019-01-08 12:29:15 +01:00
Naz Gargol
df1ba8aee1
Added ability to notify and update url service about changes in related resources (#10336)
refs https://github.com/TryGhost/Ghost/issues/10124

- This PR introduced additional db calls in URL service due to the need for a model recalculation (we can't rely on the objects that come with events)
2019-01-08 09:48:53 +00:00
Hannah Wolfe
da17b2c82b Settings API should return null instead of ""
refs #10345

- We are standardising on returning null from the Content API for any empty values
2019-01-07 15:02:24 +00:00
Fabien O'Carroll
9ce160df78
Added tests for content api brute force protection (#10344)
no-issue
2019-01-07 21:25:19 +07:00
Katharina Irrgang
6b758bda79
Refactored routing config for multiple api versions (#10333)
refs #10124

- one clean v0.1 and v2 config file for routing!
- solves one underlying bug reported in #10124
- the alias handling was just a hotfix to support v2 for the site
- but it was hard to read, ugly
- now we have two clean configs
- we'll see how useful it is
- need to do proper manual testing on Monday
2019-01-04 21:59:39 +01:00
Katharina Irrgang
30a0b1794a
Added calculated excerpt field to Content API v2 (#10326)
closes #10062

- return `post.excerpt` for Content API v2
- do not use `downsize`, because we might want to get rid of it if we drop v0.1 (downsize does not create good excerpts)
- simple substring of the plaintext field
2019-01-04 19:00:45 +01:00
Katharina Irrgang
6448c7bdc8
Fixed site using API v2 (#10332)
no issue

- See explanation: ef98c65040 (r31840536)
- that should not break anything, because resource consumption is based on resource type
- the alias pattern was only invented to make v2 work, it was a little dirty. i wanted to refactor that out anyway
2019-01-04 18:24:00 +01:00
Fabien O'Carroll
b6152684a8
Shortened content api key lengths (#10322)
* Lowered the min length for api keys to 26

* Updated ApiKey model to use shorter secrets for content keys
2019-01-04 19:39:54 +07:00
Hannah Wolfe
652e818cae Used settingsCache.getPublic() in theme middleware
- Use the new getPublic function which does the same thing as this code
- This removes the inclusion of the amp property, but this is undocumented and should not be there anyway!
- This also adds the ghost_head and ghost_foot property, which are public settings
2019-01-04 12:17:32 +00:00
Hannah Wolfe
5d977f23d4 Added Settings endpoint to V2 Content API
refs #10318

- This settings endpoint returns the commonly used, public information from our settings.
- The values are whitelisted each with a custom name for returning from the endpoint
2019-01-04 12:17:32 +00:00
Hannah Wolfe
ef98c65040
Clean v2 Content API (#10329)
* Removed unused fields from v2 Content API

- We want to ship the v2 Content API as clean and lean as we can
- Many fields in the DB aren't actually used, we shouldn't return these values
- Other values aren't useful outside of Admin clients, and shouldn't be returned either

Fields removed:
- tags: created_at, updated_at, parent
- authors: locale, accessibility, tour
- posts: locale, author status, page
2019-01-04 11:21:21 +00:00
Naz Gargol
d3f3b3dc20
Added plugin based author and public tag models in API v2 (#10284)
refs #10124

- Author model returns only users that have published non-page posts
- Added a public controller for tags (should be extracted to separate Content API controller https://github.com/TryGhost/Ghost/issues/10106)
- Made resource configuration dynamic based on current theme engine
- This needs a follow-up PR with fixes to the problems described in the PR
2019-01-03 20:30:35 +01:00
kirrg001
789a3c0715 Removed x_by fields from API v2 response
refs #10286

- v2 no longer exposes x_by fields (published_by, updated_by, created_by)
- we will add a brand new concept called activity stream/actions soon
2019-01-03 16:38:52 +01:00
Fabien O'Carroll
0f32209e65
🐛 Fixed responsive images for gifs & svgs (#10315)
closes #10301

* Redirected to original image for gifs & svgs

* Created canTransformFileExtension method

* Updated image middlewares to use canTransformFileExtension
2019-01-03 16:28:37 +07:00
Hannah Wolfe
8d6def159a Reworked ghost_head tests as unit tests
- Removed dependency on the database
- We should not have integration tests for this kind of code

- Note: had to fix rss url tests as sandbox wasn't restored afterwards
2019-01-02 09:51:38 +00:00
Naz Gargol
aca887a35d
Added async file header handling for API v2 (#10292)
refs #9866
2018-12-17 12:47:19 +01:00
Fabien O'Carroll
2d92793b3f
Cleaned up image manipulator (#10282)
no-issue

* Added InternalServerError to resizeImage

* Added a redirect to original image if sharp is missing

* Improved naming - safeMethod -> method

* Updated process method to follow same sharp check pattern

* Refactor safety wrapper into makeSafe function

* Moved generic manipulation error to makeSafe function

* Refactored unsafeProcess to use unsafeResizeImage

* Removed CRAZY catch
2018-12-14 11:54:52 +07:00
Fabien O'Carroll
7099dd45a5
Supported dynamic image resizing for LocalFileStorage(#10184)
refs #10181 

* Added initial handleImageSizes middleware

* Implemented saveRaw method on local file storage

* Wired up handleImageSizes middleware

* Implemented delete for LocalFileStorage

* Removed delete method from theme Storage class

* Deleted sizes directory when theme is activated

* Ensured that smaller images are not enlarged

* Renamed sizes -> size

* Exited middleware as early as possible

* Called getStorage as late as possible

* Updated image sizes middleware to handle dimension paths

* Revert "Deleted sizes directory when theme is activated"

This reverts commit 9204dfcc73a6a79d597dbf23651817bcbfc59991.

* Revert "Removed delete method from theme Storage class"

This reverts commit b45fdb405a05faeaf4bd87e977c4ac64ff96b057.

* Revert "Implemented delete for LocalFileStorage"

This reverts commit a587cd6bae45b68a293b2d5cfd9b7705a29e7bfa.

* Fixed typo

Co-Authored-By: allouis <fabien@allou.is>

* Redirected to original image if no image_sizes config

* Refactored redirection because rule of three

* Updated comments

* Added rubbish tests

* Added @TODO comment for handleImageSizes tests

* Added safeResizeImage method to image manipulator

* Used image manipulator lib in image_size middleware
2018-12-13 20:25:24 +07:00
Fabien O'Carroll
c2275ed131
Added size attribute support to img_url helper (#10182)
refs #10181 

Adds support to request a size in the img_url helper using syntax like:
    <img src="{{img_url profile_image size="small"}}"/>

Requires the image_sizes config to be defined in the themes package.json
2018-12-13 19:14:08 +07:00
Dexter Leng
8c3d29edb2 🐛 Striped invisible unicode characters from slugs (#10252)
refs #9826

- Striped invisible characters from slug based on regexp pulled directly from XRegExp library, similarly to the approach taken in 9eef2616e4 (diff-50bf43eef863952813617ae9997538d0R4)
2018-12-13 13:06:59 +01:00
Hannah Wolfe
7cce71d997
🐛 Fixed get helper when API v0.1 is disabled (#10270)
closes #10266

- the Public API labs flag refers to the v0.1 API only
- if it is disabled, the v0.1 API should be disabled
- if the theme is using v2 API, then the get helper should be available regardless
2018-12-12 15:38:35 +00:00
Katharina Irrgang
9d7c3bd726
🐛 Fixed all known filter limitations (#10159)
refs #10105, closes #10108, closes https://github.com/TryGhost/Ghost/issues/9950, refs https://github.com/TryGhost/Ghost/issues/9923, refs https://github.com/TryGhost/Ghost/issues/9916, refs https://github.com/TryGhost/Ghost/issues/9574, refs https://github.com/TryGhost/Ghost/issues/6345, refs https://github.com/TryGhost/Ghost/issues/6309, refs https://github.com/TryGhost/Ghost/issues/6158, refs https://github.com/TryGhost/GQL/issues/16

- removed GQL dependency
- replaced GQL with our brand new NQL implementation
- fixed all known filter limitations
- GQL suffered from some underlying filter bugs, which NQL tried to fix
- the bugs were mostly in how we query the database for relation filtering
- the underlying problem was caused by a too simple implementation of querying the relations
- mongo-knex has implemented a more robust and complex filtering mechanism for relations
- replaced logic in our bookshelf filter plugin
- we pass the custom, default and override filters from Ghost to NQL, which then are getting parsed and merged into a mongo JSON object. The mongo JSON is getting attached by mongo-knex.

NQL: https://github.com/NexesJS/NQL
mongo-knex: https://github.com/NexesJS/mongo-knex
2018-12-11 11:53:40 +01:00
Fabien O'Carroll
48923ac327
Wired members service up to api and app (#10262)
* Updated auth service members middleware

refs #10213

* Wired up members api router to the ghost api endpoints

refs #10213

* Created members app for the static pages

refs #10213

* Wired up the members app

refs #10213
2018-12-11 15:18:07 +07:00
Fabien O'Carroll
0bf5dd212a Added members related db changes and models
refs #10213
2018-12-11 11:53:55 +07:00
kirrg001
7af2802e14 Added validation for null|undefined values for required keys
closes #10071
2018-12-10 16:33:39 +01:00
kirrg001
11f0f1c314 Fixed incorrect validation message for required values
no issue

- the message always showed: `Validation (FieldIsRequired) failed for key`
2018-12-10 16:33:39 +01:00
Nazar Gargol
da7fdfeae6 Extracted trimAndLowerCase utility to shared utility module
refs #9866
2018-12-10 13:56:42 +01:00
Sumedh Nimkarde
8b816af1c2 🐛 Fixed invalid imported subscribers count (#10229)
closes #9987

- Fixed the count of invalid records when importing single column csv of subscribers
2018-12-03 20:45:55 +01:00
Katharina Irrgang
fc21b25895
🐛Fixed auto redirect for extra data queries on post and page resources (#9828)
closes #9791

- we only made use of the redirect middleware, who detects if a redirect should happen, for taxonomies (tags, authors)
- `data: page.team` will now redirect too
- `data: post.team` will now redirect too
- you can disable the redirect using the long form
2018-12-03 20:31:48 +01:00
anaplian
c539d39875 🐛 Fixed missing 404 for unknown API routes (#10070)
closes #10020

* Append trailing slash to version path
2018-11-26 16:35:38 +07:00
Katharina Irrgang
e48c28b98f 🐛 Protected Ghost blog against invalid uploaded routes.yaml (#10100)
* 🐛  Protected Ghost blog against invalid uploaded routes.yaml

no issue

- e.g. you upload `filter:tag=this is a wrong filter value`
- ask the url service if it has finished it's work to ensure the upload was successful
- wait 5 seconds till Ghost will bring back the last uploaded valid version

* fixed test
2018-11-26 16:28:22 +07:00
Katharina Irrgang
090a936e16
Moved raw order for author filtering to correct place (#10166) (#10171)
refs #10105

- ordering !== filtering
2018-11-15 16:17:51 +01:00
Katharina Irrgang
e89a27f3ab
Replaced options.where GQL statements with filter notation (#10160)
refs #10105

- `options.where` is an older deprecated logic
- before the filter language was invented, Ghost generates statements for knex
- if we want to replace GQL with NQL, we can't generate these statements
- they are not understood from NQL, because NQL uses mongo JSON
- go through usages and rewrite the statements
- invent `extraFilters` for now
- we need to keep the support for `status` or `staticPages` for now (API requirement)
- IMO both shortcuts in the extra filters should be removed in the future

This commit is required for https://github.com/TryGhost/Ghost/pull/10159!
2018-11-15 15:53:24 +01:00
Katharina Irrgang
2e81852b22
Moved raw order for tag filtering to correct place (#10166)
refs #10105

- ordering !== filtering
2018-11-15 15:27:31 +01:00
Nazar Gargol
28a686a936 🐛 Fixed image optimisation for input image being smaller than optimized one
closes #10144

- When the input image is well optimized and has smaller byte size than the processed one it's still being used
- Bumped sharp version to have access to `size` property
2018-11-13 13:19:31 +01:00
Fabien O'Carroll
8046f4d437
🐛 Fix last seen for users (#10141)
* Added updateLastSeen method to user model

refs #10138

* Refactor codebase to use user.updateLastSeen

refs #10138

This is to ensure all updates go via the same method, meaning any
specific logic can be handled in one place, it also helps with grepping
the codebase to find where this occurs

* Created updateUserLastSeen middleware for v2 admin

refs #10138

This is intended to be used with the v2 admin api and _possibly_ the
content api, to give us an accruate report on thelast time a user access
a ghost instance.

* Wired updateUserLastSeen up to v2 Admin API

closes #10138

* Fixed broken test for v2 admin api

no-issue

This test was broken because it was incorrectly testing for a method to
be called exactly once - this was irrelevant to the functionality being
tested for.

* Updated user check method to set status to active

no-issue

* Debounced the updateUserLastSeen middlware an hour

no-issue

* Resolved some PR comments
2018-11-13 18:27:10 +07:00
Sumedh Nimkarde
f3e9218ee3 🎨 Modified slack webhook payload, added info about published post (#9850)
closes #8936

- Modified the slack webhook payload to provide more info about a published post
2018-11-12 13:04:50 +01:00
Fabien O'Carroll
7323258415 🚧 Added req.member to the v2 api context object
closes #10112
2018-11-08 13:32:41 +07:00
Fabien O'Carroll
2a6e9aac55 🚧 Created members auth middleware
closes #10110
2018-11-08 13:32:09 +07:00
John O'Nolan
9f7295853f Changed casper.ghost.org to static.ghost.org references (#10109)
no issue

Assets moved from gh-pages to https://github.com/tryghost/static and hostname changed, redirects already in place. Can be tested on https://demo.ghost.io (image should all work fine, try visiting one directly to verify redirect works)
2018-11-07 15:50:38 +01:00
Katharina Irrgang
ff6bf5f318 Renamed /users to /authors for Content API V2 (#10096)
refs #10061

- Made /authors endpoint available in Content API V2
2018-11-07 15:29:37 +01:00
Naz Gargol
ec03b3cfc5
Content API v2 date formatting (#10095)
closes #10065 

- Added UTC offset to dates returned by Content API
- Added test checking new format is compatible with Admin API
- Refactored output serializer mapping logic
2018-11-06 16:12:12 +01:00
Rishabh Garg
7b38986316 Removed mobiledoc format from Content API V2 response (#10098)
closes #10097

- removed formats `mobiledoc` option directly in post input serializer for v2 Content API
2018-11-06 14:39:16 +01:00
kirrg001
46c806358b Extended resource lookup in {{get}} helper
refs #10061

- prep task to add authors for content API
2018-11-06 14:08:13 +01:00
Rishabh Garg
3345618731
Added helper to check for content API (#10104)
no issue

* Added helper to check if API called is content API
2018-11-06 17:36:22 +05:30
kirrg001
22911b5812 Fixed frame context being empty
no issue
2018-11-06 14:44:06 +05:30
Katharina Irrgang
1b9c61eed1
Returned relative paths in html for Content API V2 by default (#10091)
refs #10083

- you can send `?absolute_urls=true` and Ghost will also transform the paths in the content (this is optional/conditional)
2018-11-05 18:07:45 +01:00
Fabien O'Carroll
923969b38c
Removed user_id constraint when upserting session (#10085)
no-issue

This is to stop an issue when creating a session, if you already have an existing session
2018-10-30 16:15:48 +07:00
Fabien O'Carroll
1c21153b85
Refactored request unit tests to return promises (#10045)
refs #7470

These tests had a random failure on travis for Node v6 w/ sqlite. I
figured I would update them to return promises, and remove state, to
make debugging easier, and hopefully fix any issues.
2018-10-24 12:14:44 +07:00
Kevin Ansfield
71f0c08a34
Added edit webhook route to v2 Admin API (#10033)
no issue
- webhooks UI requires the ability to edit webhooks
- added `edit` permission for `webhook`
- added `edit` method to v2 webhook controller
- added `PUT /webhooks/:id` route to v2 Admin API routes
2018-10-19 18:35:55 +01:00
Rishabh Garg
e2d0b2e543 Updated webhook request header and last triggered error mssg (#10035)
refs #9942

- Added error message if unknown status code on request failure
- Added event to payload
2018-10-19 14:11:03 +01:00
kirrg001
99a2f3ba3a Fixed read pages with user access
refs #9866

- we still serve pages via /posts with user access
- we only differentiate this behaviour for the content api
2018-10-19 10:54:05 +02:00
kirrg001
759c25d03e Fixed read posts/pages for v2
refs #9866

- read does not support `filter`
2018-10-19 10:40:47 +02:00
Rishabh Garg
8ad951d7f3
Added new site.changed event and webhook trigger service (#10019)
refs #9942

* Added new middleware to trigger events

* Refactored webhooks service
- added new trigger service, moved listen service to its own file
- started listening to new site.changed event
- cleaned up trigger service to work with new webhook fields
- cleaned up tests
- removed redundant trigger method in v0.1 controller
2018-10-19 00:01:30 +05:30
kirrg001
e302be2749 Changed preview controller to support v0.1 and v2
refs #9866

- invent preview api, but only used internally
  - the idea of a preview api is definitiely reaslistic and came up in the past a couple of times
- by that we don't have to differentiate between pages or posts controller
- still support v0.1
- preview controller is not registered for http, only internal handling
2018-10-18 19:41:07 +02:00
kirrg001
12ff70497f Changed entry lookup helper to respect the resource type
refs #9866

- the entry helper is used for static pages and post lookups
- now that we support changing the api version, we have to respect the resource type
- for v2: we ask the pages controller for static pages
- in v0.1: pages and posts lived on the same route
- we are talking about the content API (!) - not admin api
2018-10-18 19:41:07 +02:00
kirrg001
803a325ade Renamed post-lookup to entry-lookup
refs #9866

- the static pages router uses the entry controller
- and the entry controller uses the lookup helper
- the lookup helper needs to either fetch static pages or posts
- v2 uses pages and posts controller
2018-10-18 19:41:07 +02:00
kirrg001
640290c31b Updated blog site components to respect the api version
refs #9866

- the api call must be dynamic based on the api version information
2018-10-18 19:41:07 +02:00
kirrg001
3f9d0f51f5 Changed how we determine if a resource is owned by an url generator
refs #9601

- api v2 returns absolute urls
- api v0.1 returns relative urls
- it's easier if we compare by id
2018-10-18 19:41:07 +02:00
kirrg001
987e41e8d6 Attached api version to res.locals context
refs #9866

- each request get's the ghost api version attached
- this makes it possible to access the version in all steps (routing, theme helpers)
2018-10-18 19:41:07 +02:00
kirrg001
1f55c90037 Supported reading ghost api engine
refs #9866

- we fallback to v0.1 by default
- we support different formats
- this opens the box to switch the ghost api version for the whole blog site
- i had to add a different notation for overrides.json, because the structure is not optimal (i only want the versions, not the shortcuts)
2018-10-18 19:41:07 +02:00
Rishabh Garg
915d5612a1 🐛 Fixed relative image URLs becoming absolute URLs on save (#10025)
closes #10024

- Updated input serializers for posts/tags/users to handle absolute urls conversion

-------
1. Ghost stores relative images urls
2. API V2 returns images with absolute urls
3. Ghost-Admin sends absolute urls back on any save e.g. update user

**Current behavior**: This will override the relative image path in db to absolute, which in turn won't get updated in future if domain or protocol changes for e.g.
**Fix**: On save/update, input serializers converts any absolute image url paths back to relative if the base URL from image fields matches the configured URL
2018-10-18 12:18:47 +01:00
Fabien O'Carroll
ae71f2deca Added spam prevention for v2 sessions (#10030)
no-issue

- Added spam prevention to POST /session
  - This blocks repeated requests the the /session endpoint preventing brute
force password attacks
- Updated session controller to reset brute middleware
  - This updates the session controller to reset the brute force protection
on a successful login. This is required so that a user is not locked out
forever :o!!
2018-10-18 09:58:29 +01:00
Nazar Gargol
49a729f77f Excluded sessions table from exporter (#9961)
refs #9865

- Made sessions table to be skipped during the export process
2018-10-17 18:29:24 +07:00
Rishabh Garg
c638863cb9 Updated webhooks schema and add relation (#10018)
* Extended webhooks schema/model and connected with integrations

refs #9942

- Updated webhooks schema with new columns - name, integration_id, secret, last_triggered_at, api_version
- Updated webhooks and integration model to map relationships
- Updated schema hash
- Updated test utils to exclude new webhooks columns for response comparison

* Added migration script for new webhooks columns

refs #9942

- Added migration script in 2.3 to add new columns to webhooks

* Updated schema hash

* Updated maxLength for api_version in schema

* Removed concurrency value from migration script

* Added defaults for webhooks model

* Added status field to webhooks for last trigger status

* Updated schema hash

* Fixed tests with status field

* Removed concurrency value in migration script

* Cleanup

* Updated schema with new fields

 - last_triggered_status, last_triggered_error
2018-10-17 18:17:13 +07:00
Fabien O'Carroll
4f1866a263
Allowed for repeated query parameters for arrays (#10021)
no-issue

There are a few libraries, including node core that when given an array
for a query parameter will encode it as repeated query params. e.g.

```
{someParam: ['a', 'b']}
// becomes
'?someParam=a&someParam=b'
```

This adds a check for the value to stop us 500ing on repeated keys and
to add easier interop with http clients
2018-10-17 13:43:32 +07:00
kirrg001
15a8951bc7 Respected hasUserPermissions & hasAppPermissions in invite model
no issue

- the permission service expects from the permissible function to respect the input values
  - hasUserPermissions
  - hasAppPermissions
2018-10-16 18:00:05 +02:00
Nazar Gargol
d582c06eee Optimized usage of urls in API v2
refs #9866

- Extracted url decoration logic to utility in output serializers in posts, pages, users, and tags
- Added test cases for url usage by child object (tags of posts)
2018-10-16 13:02:04 +02:00
Fabien O'Carroll
2fbc5aa257
Added apiImpl.data to apiOptions for serialisation (#10016)
no-issue

This is to give serializers access to the expected data properties so
that can be used for filtering.
2018-10-16 16:51:50 +07:00
Rishabh Garg
51dde1e38c
Refactored config to handle direct calls for specific version (#10012)
refs #9866

- Refactored overrides config to include direct version configs(v0.1, v2), supported versions map to direct version
- Refactored `getApiPath` to handle direct versions as well as mappings of supported version
2018-10-16 15:20:51 +05:30
Nazar Gargol
9fd9186557
🐛 Fixed 'url' attribute miscalculation when when requested as the only part of fields filter (#9969)
closes #9962

- Fixed the bug with url being set to /404 when id was not present on the model
- Added a functional test to cover this bug
- Refactored url decorating methods to be more clear about the nature of passed parameters
2018-10-15 14:47:56 +02:00
Fabien O'Carroll
3db102a776
Added API Key auth middleware to v2 content API (#10005)
* Added API Key auth middleware to v2 content API

refs #9865

- add `auth.authenticate.authenticateContentApiKey` middleware
  - accepts `?key=` query param, sets `req.api_key` if it's a known Content API key
- add `requiresAuthorizedUserOrApiKey` authorization middleware
  - passes if either `req.user` or `req.api_key` exists
- update `authenticatePublic` middleware stack for v2 content routes

* Fixed functional content api tests

no-issue

This fixes the functional content api tests so they use the content api
auth.

* Fixed context check and removed skip

* Updated cors middleware for content api

* Removed client_id from frame.context

no-issue

The v2 api doesn't have a notion of clients as we do not use oauth for it

* Fixed tests for posts input serializer
2018-10-15 16:23:34 +07:00
Nazar Gargol
76f4a4bb03
Enforced non-page posts only to be returned by /posts endpoint from Content API (#10002)
refs #9866

- Added logic ensuring page filter is always set to false in posts endpoint for Content API
- Added functional tests to pages and posts
- Added absolute_url logic in pages controller
2018-10-13 00:48:49 +02:00
kirrg001
0338ba56c0 Added removal of null values in v2
refs #9866

- also moved id mismatch to global validator
- this is not specific to posts
2018-10-12 21:46:16 +02:00
kirrg001
a153400164 Added posts controller to v2
refs #9866
2018-10-12 19:21:43 +02:00
kirrg001
27714075b5 Added handling for empty query options
refs #9866

- it's fine if you pass e.g. `?formats=`
- same behaviour as v0.1
2018-10-12 19:21:43 +02:00
Fabien O'Carroll
05330482e6
Stopped api key from assigning the 'Owner' role (#9971)
* Stopped api key from assigning the 'Owner' role

refs #9865

We do not want api keys to be able to assign the Owner role to any other
key or user.

* Cleaned up Role model permissible method

no-issue
2018-10-12 15:38:57 +07:00
Fabien O'Carroll
caccda1aab
Implemented global validation on defined fields (#9992)
no-issue

This is to allow global validation to run on fields that have some user
validation defined.
2018-10-12 15:16:12 +07:00
kirrg001
7a73dfd9bc Extended all shared validator
refs #9866

- there was a missing step in the shared validator
- we have to differentiate between data validation for browse/read and data validation for add/edit
- furthermore, the data validation for add/edit was missing and was not copied over from v0.1 (check structure of incoming body)
- adds the ability to require properties from req.body.docName[0]
2018-10-12 09:40:34 +02:00
Nazar Gargol
86e9c35c3c Allowed passing an array directly instead of requiring object with values key for validation options
noissue
2018-10-10 16:48:22 +02:00
Fabien O'Carroll
3516b15f9e
Updated session middleware to support node v6 (#9980)
closes #9972

* Added breaking test for node v6 session auth
* Updated session middleware to support node v6

This uses the legacy url to obtain the origin rather than the WHATWG
URL class in order to support node <6.14.4
2018-10-10 20:07:31 +07:00
Katharina Irrgang
a7b0029471 Added mobiledoc revisions functionality
closes #9927

- Added post model implementation to be able to store up to 10 versions of mobiledoc
- Bumped GQL to support filtering on the mobiledoc revision table
- Added tests ensuring new functionality works
2018-10-09 15:31:09 +02:00
Fabien O'Carroll
1b9aa2546f
Updated frame.context to use req.api_key_id (#9965)
* Added api_key_id to frame.context

refs #9865

This is to allow controllers to check permissions using api_key_id data.

* Removed client and client_id from frame.context

refs #9865

This is unused as we only support oauth on v0.1 API.
2018-10-09 18:28:38 +07:00
Fabien O'Carroll
e392fe55ae
🐛 Fixed redirect issue with private sites (#9960)
closes #9959

This issue existed because the logic assumed that if there were no
query parameters then there would be no `query` object. However this is
not the case. What we really wanted to check was for the existence of an
"r" query param - the code has been refactor to explicitly do this now.
2018-10-08 16:31:08 +07:00
Katharina Irrgang
d699daeb35 🐛Fixed sitemap duplicates after routes.yaml upload (#9957)
closes #9956

- sitemap reset was missing
2018-10-08 10:29:21 +07:00
Katharina Irrgang
d2baf80d58
Optimised test folder structure (#9958)
refs #9866

- test/functional/
- test/functional/api
- test/functional/api/v0.1
- test/functional/api/v0.1/utils
- test/functional/api/v2
- test/functional/api/v2/admin
- test/functional/api/v2/admin/utils
- test/functional/api/v2/content
- test/functional/api/v2/content/utils

- updated grunt file
- instead of `grunt test-routes`, you now need to use `grunt test-functional` (docs are updated)

You can use `localUtils.API.getApiQuery('posts/')` and it will generate the correct API url.
2018-10-07 16:36:02 +02:00
Katharina Irrgang
db1d2f62dd
Removed api integration tests (#9940)
refs #9866 

- moved the tests either to unit tests or routing tests
- or removed test case (a lot)
- this commit is very big 🤪, it was not rly possible to create clean commits for this
- it only changes the test env, no real code is touched

Next steps:
- optimise folder structure + make v2 testing possible
- reduce some more tests from routing and model integeration tests
2018-10-06 22:13:52 +02:00
kirrg001
b855f2d8f0 Removed testUtils.API.getApiPath
refs #9866

- the global utility should for now not generate API urls
- we only have one test which generates the API url
- for now it will live in this test file
- if we have multiple cases, we can reconsider this
2018-10-05 16:49:46 +02:00
kirrg001
b74879c64e Refactored Ghost SDK unit test
refs #9866

- the sdk does not use the url utility
- it just respects what you pass in
- there is no need to test this behaviour for multiple API versions
2018-10-05 16:49:46 +02:00
kirrg001
1e9d87624a Moved test/unit/api files to test/unit/api/v0.1
refs #9866

- these test files belong to v0.1
2018-10-05 15:11:35 +02:00
Fabien O'Carroll
b6e80ef557
Added refreshSecret method to ApiKey model (#9947)
refs #9865

This is to allow the secret of an api_key to be refreshed, in the event of a secret being compromised.
2018-10-05 15:51:13 +07:00
Katharina Irrgang
959912eca3
Added tiny framework to support multiple API versions (#9933)
refs #9326, refs #9866

**ATTENTION: This is the first iteration. Bugs are expected.**

Main Goals: 

- add support for multiple API versions.
- do not touch v0.1 implementation
- do not break v0.1

## Problems with the existing v0.1 implementation

1. It tried to be generic and helpful, but it was a mixture of generic and explicit logic living in basically two files: utils.js and index.js.

2. Supporting multiple api versions means, you want to have as less as possible code per API version. With v0.1 it is impossible to reduce the API controller implementation. 

----

This commit adds three things:

1. The tiny framework with well-defined API stages.
2. An example implementation of serving static pages via /pages for the content v2 API.
3. Unit tests to prove that the API framework works in general.

## API Stages

- validation
- input serialization
- permissions
- query
- output serialization

Each request should go through these stages. It is possible to disable stages, but it's not recommended.

The code for each stage will either live in a shared folder or in the API version itself. It depends how API specific the validation or serialization is. Depends on the use case.

We should add a specific API validator or serializer if the use case is API format specific.
We should put everything else to shared.

The goal is to add as much as possible into the shared API layer to reduce the logic per API version.

---

Serializers and validators can be added:

- for each request
- for specific controllers
- for specific actions

---

There is room for improvements/extensions:

1. Remove http header configuration from the API controller, because the API controller should not know about http - decouple.

2. Put permissions helpers into shared. I've just extracted and capsulated the permissions helpers into a single file for now. It had no priority. The focus was on the framework itself.

etc.

---

You can find more information about it in the API README.md (api/README.md)

- e.g. find more information about the structure
- e.g. example controllers

The docs are not perfect. We will improve the docs in the next two weeks.

---

Upcoming tasks:

- prepare test env to test multiple API versions
- copy over the controllers from v0.1 to v2
- adapt the v2 express app to use the v2 controllers
2018-10-05 00:50:45 +02:00
kirrg001
80b963fb5a Moved test/unit/api/decorators
no issue

- to test/unit/api/v0.1/decorators/urls_spec
2018-10-04 19:09:52 +02:00
Rishabh Garg
f78d9d3914
Refactored hardcoded v0.1 url unit tests to support multiple versions (#9945)
refs #9866

- Added test util method for api path based on version
- Updated all hardcoded v0.1 tests strings to use dynamic string from util method
- Updated hardcoded v0.1 tests using regex match to use string equal with new util method
2018-10-04 21:43:12 +05:30
Rish
dd151f3713 Refactored method urlFor to use versionType instead of admin: true
no issue

- Updated urlFor to work with versionType instead of admin:true
- Updated tests to use the correct method call
2018-10-04 16:49:55 +05:30
Fabien O'Carroll
b185892b7b
Created session controller (#9911)
refs #9865

Note that this controller is the singular, that's because we plan to
make a session resource controller to be used with /sessions, wheras
this is on /session
2018-10-03 20:45:42 +07:00
Nazar Gargol
82b7aea641
Refactor URL generation from models (#9917)
Moved URL attributes logic from the model into API layer 

refs #9866

- Moved URL related attribute calculation for posts, users, and tags into API layer
- Added test coverage for url attributes in tags/authors/primary_tags/primary_authors
2018-10-03 15:44:30 +02:00
kirrg001
27996db5e9 Extended sequence utility
no issue

- support promise and none promise tasks
- helpful if you create an array of operations and not all of the operations/tasks are async
- `response instanceof Promise` does not work for all cases e.g. some usages return a transaction/bookshelf chain
2018-10-03 00:17:45 +02:00
Rish
91f8e03244 Added mobiledoc_revisions schema, migration script and updated tests 2018-10-02 19:15:23 +02:00
Kevin Ansfield
1db3aefb9b
Set up schema and models for API Key authentication (#9904)
refs https://github.com/TryGhost/Ghost/issues/9865
- schema migrations
  - adds `integrations` and `api_keys` tables
  - inserts `integration` and `api_key` permissions and Administrator role relationships
  - inserts `Admin Integration` role and permissions
- adds `Integration` model
- adds `ApiKey` model
  - creates default secret if not given
  - hardcodes associated role based on key type
    - `admin` = `Admin API Client`
    - `content` = no role
- updates `Role` model to use `bookshelf-relations` for auto cleanup of permission relationships on destroy
2018-10-02 17:46:38 +01:00
Fabien O'Carroll
ecf47f3b7b Added some missing tests to base model (#9931)
no-issue
2018-10-02 16:43:19 +02:00
Fabien O'Carroll
cb0c5dc582
Session auth service (#9910)
refs #9865

* This service handles the session store and exporting middleware to be
used for creating and managing sessions

* Updates the auth service index.js file in line with how we do things elsewhere

* After wrapping the exports in a getter, the usage of rewire had broken
the authenticate tests, this commit _removes_ rewire from the tests, calls `init` on
the models before the tests (needed because rewire isn't there) and also
cleans up the use of var.
2018-10-02 15:35:23 +07:00
Fabien O'Carroll
6cd1dc8005 🐛Fixed blog not responding when uploading routes.yaml with unknown taxonomy (#9926)
closes #9870

Instead of 500'ing when uploading a routes.yaml with an invalid taxonomy
we will now send back an error saying it's invalid.
2018-10-01 20:40:29 +02:00
Fabien O'Carroll
af12f21db7 Updated base model destroy fn to be more generic (#9928)
refs #9914

This allows us to destroy models on properties other than the id.
2018-10-01 14:44:52 +02:00
Rishabh Garg
6163d1f128 Updated overrides config for api versions with v0.1 to deprecated (#9921)
refs #9866

- v0.1 == deprecated
- v2 == active
2018-09-27 19:34:12 +02:00
Katharina Irrgang
b43ab65d8a
Moved api controllers into api/v0.1 (#9918)
refs #9866

- preparation for v2
- moved api/ to api/v0.1
- do export v0.1 straight from the api folder, we don't want to touch this right now
- that means currently if you require the api folder, we return v0.1 by default
- there were some direct requires of api files in the test env
  - some of them use rewire
  - for now, we just correct the require path to require api/v0.1/
  - we touch the test env next week

**Docs about V2 design are coming soon!**
2018-09-27 16:06:57 +02:00
Fabien O'Carroll
4d9414b5d2 Created Session Model (#9909)
refs #9865

- Created session model for session table
- Added model relations
- Added unit test coverage
2018-09-27 14:31:39 +02:00
Fabien O'Carroll
1d17f2aa91 Created session table migration (#9908)
refs #9865

- This table will be used for storing user sessions in
2018-09-27 09:51:46 +02:00
Fabien O'Carroll
045ce0a800 Updated private-sites to not redirect to full urls
no-issue

- Parse redirects as URL with blog as base
- Redirect to the pathname property of parsed URL

Credits: @j3ssie
2018-09-25 17:32:17 +07:00
Rishabh Garg
9624c88f3e Refactored url utility to generate multiple API version URLs (#9897)
refs #9866

- added api version config to overrides, which makes it possible to have a centralized api versioning configuration
- the next PR will use this config in the web folder
- make api url generation in url service flexible and dynamic
- remove hardcoded API_PATH
- updated all places which used `urlFor('api'..)` -> we now ask for explicit api version
2018-09-24 21:49:20 +02:00
Rishabh Garg
39485d17c0 Added new endpoint to upload square profile images with dimension validation (#9862)
refs #8576

- adds new API endpoint `/uploads/profile-image` for uploading profile images
- new validation which fails with error message if uploaded image is not square
- Renamed getImageSizeFromFilePath to getImageSizeFromStoragePath, because it's more explicit
- Add new getImageSizeFromPath method, which is used in the new dimensions middleware
- Ensure we use the sharp middleware to auto-resize the uploaded profile pictures
- Ensure the new route get's added to v2

While this makes sure all future profile images uploaded are square, this doesn’t affect any existing non-square profile image. Needs more thought on how to handle existing non-square profile images for the purpose of making theming easier in future.
2018-09-24 21:42:58 +02:00
Daman Mulye
214d682ea3 Removed change frequency and priority fields from sitemap generator (#9771)
closes #9765

- Google says that change frequency and priority are not important when search engines crawl through pages. Therefore, these two properties are unimportant and removed from Ghost. Reference: https://www.seroundtable.com/google-priority-change-frequency-xml-sitemap-20273.html
- Credits to @damanm24
2018-09-24 19:16:31 +02:00
Katharina Irrgang
b913618c03 Extended uncapitalise unit tests (#9891)
Extended uncapitalise unit tests

refs #9866

- Proved that the middleware works for many API versions
- Added test case to prove that the version identifier gets lowercased too
2018-09-20 22:27:38 +02:00
Rishabh Garg
fcd275f6c0 Refactored web/middleware and web/utils to web/shared (#9892)
refs #9866

- Moved web/middleware to web/shared/middlewares
- Moved util file to web/shared/utils
2018-09-20 20:04:34 +02:00
Nazar Gargol
7e17c56feb Removed res.isAdmin usages from v2 express app (#9884)
refs #9866

- Removed `res.isAdmin` flag in v2 express app
- Did not touch v0.1 express app
- Separated url redirect middleware for admin and content API
2018-09-20 13:58:45 +02:00
Nazar Gargol
57271127f4 Added v2 api endpoints (#9874)
refs #9866

- Registered Content API under /ghost/api/v2/content/
- Registered Admin API under /ghost/api/v2/admin/
- Moved API v0.1 implementation to web/api/v0.1
- Created web/api/v2 for the new api endpoints
- Started with reducing the implementation for the new Content API (the Content api does not serve admin api endpoints, that's why it was reducible)
- Covered parent-app module with basic test checking correct applications/routes are being mounted
- Added a readme file, which contains a warning using v2, because it's under active development!
- This PR does only make the new endpoints available, we have not:
  - optimised the web folder (e.g. res.isAdmin)
  - started with different API controllers
  - reason: we want to do more preparation tasks before we copy the api controllers
2018-09-18 15:59:06 +02:00
kirrg001
242f7ef4c8 Removed usage of res.isAdmin from private blogging app
no issue

- private blogging was once part of a single big express app, see: c1a2601514
- therefor we needed to differentiate if the request is for /api/v0.1 or for the actual site
- nowadays, we register apps only for the site express app
2018-09-15 08:58:53 +02:00
Nazar Gargol
9613de7cc1 🐛 Fixed gif images are not converted to png (#9853)
refs #9848

- Disabled image optimization for .gif files as a temporary solution until sharp library starts supporting latest version of libvips
- Disabled image optimization for .svg/.svgz files as permanent solution as these file types are being converted to .png
2018-09-06 11:42:42 +02:00
Kevin Ansfield
8e4c5d0f34 Fixed mobiledoc-converter test
no issue
- 👻
2018-08-31 12:03:08 +01:00
Kevin Ansfield
bba3049106 Updated gallery renderer to cope with invalid images
no issue
- skip rendering images that do not have all required fields
- do not render anything if there are no valid images
2018-08-31 11:13:24 +01:00
Rish
d29e376367 Added kg-card class to mobiledoc cards
No issue
2018-08-31 11:05:47 +01:00
Fabien O'Carroll
c9b8ddde4b 🎨Added absolute_url flag to public api (#9833)
closes #9832

The API _should_ be returning absolute URLs for everything, 3rd party applications require absolute urls to read and display ghost data correctly. Currently they have to concat the blog url and the resource url, which is very uncomfortable.

Changing the public api like this would be considered a breaking change however so we've opted to put it behind a query parameter named `absolute_urls`.
2018-08-31 11:02:39 +01:00
Kevin Ansfield
402d26a23c Added mobiledoc renderer for gallery card
no issue
- basic renderer for working with Koenig's gallery card
2018-08-30 17:57:17 +01:00
Nazar Gargol
0faf89b5ab Added ability to resize and compress images on upload (#9837)
refs #4453

* On by default

* Added config to disable resizing

* Added basic image optimization processing

* Added dep: sharp (optional dep)

* Added resize middleware

* Take care of rotation based on EXIF information

* Removed all meta data from optimised image

* Added handling if sharp could not get installed

* Do not read ext twice - optimisation

* Do not call sharp if config is disabled

* Do not remove the original image which was uploaded (store 2 images)

* Support of `req.files` for internal logic

* Disabled cache to enable file removal on Windows
2018-08-30 17:30:36 +01:00
Nazar Gargol
154366f448 🎨 Added additional parameters to subscribe_form and input_email helpers (#9820)
closes #9134

- Added form_id, input_id, and button_id parameters to subscribe_form helper
- Added id parameter to input_email helper
- Added test coverage to input_email helper
- Added quotes to id attributes for consistency
- Added subscribe_form helper tests
- Updated express to v4 in helper tests
2018-08-30 17:17:27 +01:00
Fabien O'Carroll
8ccf27340b Oembed meta tag fallback for unknown providers (#9827)
closes #9786

- Make GET request when url has no provider match
  - The HEAD request was made in order to send less data over the wire when
checking for redirects for urls that do not have an oembed provider
match. We are now going to look for provider metatags withing the
response of the request - rather than making a HEAD followed by a GET if
no redirect is found, this condenses that to a single request.

- Try to get OEmbed data from tag if no provider
  - Here we parse the HTML response of the resource and look for a link tag
that will give us the oembed resource url which we can use to fetch the
embed html
2018-08-27 15:02:03 +01:00
Fabien O'Carroll
2376c614b3 Fixed RPC pings (#9816)
closes #9644

- Removed google blogsearch from pingList, because it's dead (https://plus.google.com/+GoogleWebmasters/posts/46W7ZwVrwqg)
- Updated pingomatic method weblogUpdate -> weblogUpdates
- Updated RPC response handler to check for errors, log improvement
- the pingomatic service sends back a 200 even when it errors, so we
check the xml response to see if it's good, it throws and passes of to
the catch handler already in place
- Avoid multiline XML error message strings, but
includes a catch in case our regex stops working with a fallback to
multiline XML error message
- we check for arbitrary whitespace between different XML tags,
which I haven't seen in any of the responses - but it could change I
guess. I haven't added support of whitespace between the tags, as I
believe that would be a different value with XML spec. And we wanna
match on exact values here.
2018-08-23 15:20:29 +02:00
Katharina Irrgang
4b2ebee67b
🐛 Added logic to avoid updating the updated_at and updated_by field when migrating (#9814)
no issue

- we have to explicitly reset the previous `updated_at` field, because Bookshelf auto-updates this field on each update
- we have to extend the condition to avoid updating the `updated_by` field
- detect and respect `options.migrating`
2018-08-22 13:57:12 +02:00
Katharina Irrgang
cd690efad1 🐛 Dynamic Routing Beta: Disallow using author resource and author data key name (#9790)
* 🐛 Dynamic Routing Beta: Shortform `author.foo` is not allowed

refs #9601

- otherwise you get access to {{author}} in the theme, which is deprecated & causes errors
- recommend not using {{author}} as data longform name in the yaml validator
- (internal usage is still allowed)
- also warn against using reserved data key names like filter, resource, limit etc - maybe remove this restriction later but seems like a sensible validation right now.
2018-08-16 12:13:24 +02:00
kirrg001
fd8502448a Updated fixture post content
no issue

- Organising your content

From: Here, the theme would assign the post publicly displayed tags of Blog - but it would also keep a private record of the post being tagged with #video.
To: Here, the theme would assign the post publicly displayed tags of News - but it would also keep a private record of the post being tagged with #video.
2018-08-16 12:13:24 +02:00
kirrg001
bbde22a687 Fixed tests
no issue

- replace test themes
  - otherwise they are all invalid with the new GScan version
- fix general tests because of Ghost 2.0 logic
2018-08-16 12:13:24 +02:00
kirrg001
17833a669b 🎨 Dynamic Routing Beta: Enabled redirect by default when using data key
refs #9601

- when using the short form `data: tag.welcome` the redirect is enabled by default
  - /tag/welcome/ will redirect to the channel/collection which makes use of the data key
- you can disable the redirect by using the long form
  e.g. data:
	tag:
	  resource: tags
          type: read
          slug: welcome
          redirect: false
2018-08-16 12:13:24 +02:00
Katharina Irrgang
220d0456b4 🎨 Changed fixture data and default settings (#9778)
closes #9774, refs #9742

- added new fixture posts for Ghost 2.0
- added migration file to remove old fixture posts
  - only remove them if they are owned by the Ghost author and if they are tagged with getting-started
  - added new fixture posts if you had all (!) old fixture posts
  - ensure on rollback we remove the new fixture posts again
- updated default settings
2018-08-16 12:13:24 +02:00
Katharina Irrgang
1caa941d20 🔥 Removed demo post (#9769)
refs #9742

- removed demo post fixture for fresh blogs
- added migration to remove the demo post from the database
- on rollback: bring back demo post
2018-08-16 12:13:24 +02:00
kirrg001
8bb7088ba0 🔥 Removed permalink setting
refs #9742

- removed usage of single permalink setting
  - with dynamic routing this configuration does no longer makes sense
  - because you can configure your permalinks in the routes.yaml
  - furthermore you can have multiple collections with multiple permalinks
- removed @blog.permalinks
- do not export permalink setting
- do not import permalink setting
- permalink setting UI will be removed soon
- get rid of {globals.permalink} completely
- remove yaml in-built migration
- do not expose settings.permalinks via the private API
- do not allow to edit this setting
- keep phyiscal value in case a blog needs to rollback from v2 to v1
- sorted out when the routers should be created
  - ensure routes.yaml file doesn't get validated before Ghost is fully ready to start
2018-08-16 12:13:24 +02:00
kirrg001
52a9eebabf Changed comments and deprecation notes
refs #9742
2018-08-16 12:13:24 +02:00
Katharina Irrgang
c39df004dc Changes for Koenig and Ghost 2.0 (#9750)
refs #9742, refs #9724

- handle König Editor format for 2.0
- adapted importer to be able to import 1.0 and 2.0 exports
- added migration scripts
  - remove labs flag for Koenig
  - migrate all old editor posts to new editor format
- ensure we protect the code against mobiledoc or html field being null
- ensure we create a blank mobiledoc structure if mobiledoc field is null (model layer)
- ensure you can fully rollback 2.0 to 1.0
- keep mobiledoc/markdown version 1 logic to be able to rollback (deprecated code)
2018-08-16 12:13:24 +02:00
kirrg001
d518f23b48 Updated docs links
refs #9742

- rebase against master updated some docs links again
- go over code base again and double check that all docs links are correct
- 2.0 will become the latest version on our readme pages
2018-08-16 12:13:24 +02:00
kirrg001
91152efdc1 Changed http to https links
no issue

- use https
- replace broken links e.g. docs.ghost.org/themes
2018-08-16 12:13:24 +02:00
kirrg001
529bec8089 Renamed core/server/data/export to core/server/data/exporter
no issue

- i'vejust renamed the folder
- it makes more sense
  - data/importer
  - data/exporter
2018-08-16 12:13:24 +02:00
kirrg001
e99015e700 Refactored importer tests
no issue

- these tests were a mess
- we had many duplicated tests
- it was very hard to work with the exporter files
- i have tidied up the whole file
  - first section: all general importer tests based on the current Ghost version
  - second section: 1.0 tests
- everything is now JSON based (much easier to control)
2018-08-16 12:13:24 +02:00
kirrg001
7db0739296 Renamed amp post column to column_id
refs 9742

- when we've introduced Ghost 1.0, we have noticed that we broke Disqus comments
- Disqus comments use a unique identifier, which is the post id
- that means if you have exported your LTS content and imported it into 1.0, all resource identifiers are regenerated
- but the Disqus must use the original post resource id
- that's why we have imported the old post id and remembered it in the `amp` field 🤠
- that was the only field which was available and un-used
- now in Ghost 2.0, we would like to rename the `amp` field to `comment_id`
- string length: 50 (i thought it might be safer to not use 24 characters, because who knows if support other comment id's in the future)
2018-08-16 12:13:24 +02:00
Kevin Ansfield
66fb0955a4 Koenig - Render image title attribute if present
refs https://github.com/TryGhost/Ghost/issues/9724
- `<img>` elements can have both `alt ` and `title` attributes, ensure we render both of them
2018-08-13 09:48:04 +01:00
Kevin Ansfield
cfd9ff3993 🎨 Koenig - Added support for shortened URLs in embed card (#9781)
refs https://github.com/TryGhost/Ghost/issues/9724

- perform a HEAD request on a url if we don't find a matching provider, following any redirects until we hit success response before looking up providers for the resulting url
2018-08-12 15:57:19 +02:00
kirrg001
860718f584 Minimized cached data on resource add/update
refs #9601

- on resource update/add we have cached mobiledoc, html etc
- we have to ensure we exclude the fields (same procdure happens on bootstrap)
- these excluded fields don't have to be cached
- otherwise memory usage is higher in general
- ensure we cache relations with a minimal field set on resource update/add
2018-08-12 14:23:32 +02:00
Kevin Ansfield
0c06a47b9b Koenig - Added rich-text caption support
refs https://github.com/TryGhost/Ghost/issues/9724
- captions can have HTML so we need to render as HTML rather than as a text node so special chars don't get escaped
2018-08-08 14:29:20 +01:00
kirrg001
40c8eacd44 Added option to export extra tables
refs #9742, refs #8719

- you can now use `include` to export extra tables e.g. `include=clients`
- admin client won't make use of this option yet, maybe later and optional
- we won't announce this new ability for now (stays hidden)
2018-08-03 14:11:45 +02:00
Kevin Ansfield
5b80ec44ab 🐛 Koenig - Fixed missing alt text on images
refs https://github.com/TryGhost/Ghost/issues/9724
- render `alt` attribute if the image card payload has an `alt` property
2018-07-30 10:10:11 +01:00
Katharina Irrgang
76b9a49eb8
🎨 Added Koenig Demo Post (#9747)
no issue

- add a new migration for 1.25 to insert the draft demo post for existing blogs
- ensure new blogs get the draft demo post as well
- tested on sqlite3 + mysql
- added handling if Ghost Author user doesn't exist anymore (fallback to owner user)
2018-07-24 14:37:17 +02:00
Kevin Ansfield
f57268daae
Koenig - Finalise Koenig HTML output and migrate existing content (#9741)
refs https://github.com/TryGhost/Ghost/issues/9742

We've identified some changes we need to make to the HTML output of the [new Koenig editor](
https://forum.ghost.org/t/koenig-editor-beta-release/1284/102) for future proofing and consistency across cards.

- the `<div class="kg-post">` wrapper around post content has been removed
- for image cards the `.kg-image-wide` and `.kg-image-full` classes have been changed to `.kg-width-wide` and `.kg-width-full` and applied to the `<figure>` element rather than the `<img>` element

Before:
```html
<div class="kg-post">
    <figure class="kg-image-card">
        <img class="kg-image kg-image-wide" src="...">
        <figcaption>example wide image</figcaption>
    </figure>
</div>
```

After:
```html
<figure class="kg-image-card kg-width-wide">
    <img class="kg-image" src="...">
    <figcaption>example wide image</figcaption>
</figure>
```
2018-07-23 12:23:02 +01:00
kirrg001
132df78940 Updated docs links
refs #9742

- Ghost 2.0 is coming
- all doc links in 1.0 must use concrete links e.g. docs.ghost.org/v1 or themes.ghost.org/v1.23.0/
- if we release Ghost 2.0, docs.ghost.org will show 2.0 docs
2018-07-20 23:49:16 +02:00
kirrg001
b335002269 🐛 Fixed Disqus comments on post preview pages
closes #9727

- this is a short term fix to proof that Disqus comments on preview pages no longer appear on other threads
- this is not a full solution to the problem
- the private API still returns /404/, which is right now inconsistent, but not critical in any way
- the url helper will now output the post preview url if you serve a draft/scheduled post
- this should register unique page urls at Disqus and ensure uniquness for threads
  - i still don't understand why the cross posting happens at all, because we also pass an unique identifier to Disqus (the post ID)
- it could be that comments, which are added on the preview page, won't appear on the published urls, because
  the published url !== preview url. I wasn't able to figure this out via testing or reading their docs
2018-07-11 00:20:55 +02:00
Kevin Ansfield
a4aab19403 🎨 Koenig - Added ID attributes to heading tags when rendering (#9720)
refs https://github.com/TryGhost/Ghost/issues/9623

- added `DomModifier` class to walk a SimpleDom document and modify as needed
  - adds `id` attributes to `h1`, `h2`, etc heading tags
    - converts H* tag content to a dasherized string for the id attribute (dasherized id's are different to the smushed ids that are generated by our markdown converted but there are no backwards-compatibility concerns here)
    - if a duplicate id is detected then add a `-1`, `-2`, etc suffix to the id
- use `DomModifier` after converting mobiledoc to SimpleDom but before serialising to html
- switched top-level var declarations to es6
2018-07-11 00:03:25 +02:00
Kevin Ansfield
865acecaef
🐛 Koenig - Support schema-less URLs in embed card (#9725)
refs https://github.com/TryGhost/Ghost/issues/9724
- adjust the "base url" regex in the oembed endpoint to strip schemaless scheme `//` as well as `https?://`
2018-07-09 10:32:39 +01:00
kirrg001
c5bad55d6f Dynamic Routing Beta: StaticRoutesRouter has to respect the route name
refs #9601

- this was already working for collections or channels
- but the `routeName` was not parsed for static routes
- ensure we push the route name into the context object

e.g. /about/: about

-> name of the route is "about"
2018-06-26 19:06:42 +02:00
kirrg001
af58de0490 🐛 Fixed dated urls when timezone changes
no issue

- discovered while testing (issue doesn't exist)
- with dynamic routing we have introduced a bug
- we pre-generate urls and if your permalink contains a date (dated permalink),
  we have to regenerate the urls, otherwise the dated urls do not respect your blog timezone
- collection router has to subscribe to the timezone event
- collection router must trigger an update on it's url generator if the timezone changes and the it's permalink is dated
- ensure we also update the urls on import
2018-06-26 18:18:11 +02:00
kirrg001
00cf043e15 Fixed missing defaults in model layer
no issue

- reported in the forum: https://forum.ghost.org/t/publishing-with-a-single-post-request-to-posts/1648
- the defaults are defined in two places
  1. on the schema level (defaults for the database)
  2. on the ORM (model layer)
- the defaults on the db layer are set correctly when inserting a new resource
- but if we don't apply all defaults on the model layer, it will happen that model events are emitted without the correct defaults
  - see comment in code base
  - it's caused by the fact that knex only returns the inserted resource id (probably caused by the fact knex has to support x databases)
- components/modules are listening on model events and expect:
  1. a complete set of attributes
  2. a complete set of defaults
  3. sanitized values e.g. bool, date
- this commit fixes:
  1. added missing defaults for user & post model
  2. sanitize booleans (0|1 => false|true)
  3. added tests to ensure this works as expected
  4. clarfies the usage of `defaults`

Regarding https://forum.ghost.org/t/publishing-with-a-single-post-request-to-posts/1648:
  - the post event was emitted with the following values {page: undefined, featured: undefined}
  - the urlservice receives this event and won't match the resource against collection filters correctly
  - NOTE: the post data in the db were correct
2018-06-26 16:35:23 +02:00
Katharina Irrgang
7027980ad2
Dynamic Routing Beta: Filter collections with NQL (#9704)
refs #9601

- replace jsonpath with [NQL](https://github.com/NexesJS/NQL)
- jsonpath was just a temporary solution (a short-term fix)
- with NQL we are able to filter collections more powerful in the near future
- NQL is not feature complete
- we still support `featured:true` for collections
2018-06-26 01:54:51 +02:00
Katharina Irrgang
fc9da07025
Dynamic Routing Beta: Added ability to disable+override rss (#9693)
refs #9601

- you can now use `rss:false`
- ability to define a custom rss url with a target template (+ content_type)
- ability to disable rss for channel or collection
2018-06-26 01:33:29 +02:00
Katharina Irrgang
13cccfa9ee
Dynamic Routing Beta: Refactor res.routerOptions (#9705)
refs #9601

- sort out `res._route` vs. `res.locals.routerOptions`
- it was super hard to maintain two different objects
2018-06-26 01:12:50 +02:00
David Balderston
74a6a413ed Changed theme fixture post: point to forum instead of slack (#9703)
no issue

 - changed the default themes post to direct users to the Ghost forum
rather than slack
- updated the tests to reflect this change
2018-06-25 22:06:21 +02:00
kirrg001
07c72d735e Dynamic Routing Beta: Render default.hbs as fallback for static routes
refs refs #9601

- instead of index.hbs (that doesn't make sense)
2018-06-24 02:06:58 +02:00
kirrg001
c2fa469c4d Dynamic Routing Beta: Channels
refs #9601

- refactor architecture of routing so you can define a channel
- a channel is a different way of looking at your posts (a view)
- a channel does not change the url of a resource

Example channel

```
routes:
  /worldcup-2018-russia/:
    controller: channel
    filter: tag:football18
    data: tag.football18
```

- added ability to redirect resources to a channel/static route
- support templates for channels
- ensure we still support static routes (e.g. /about/: home)
- ensure pagination + rss works out of the box
2018-06-24 02:06:58 +02:00
kirrg001
4ed10aa76a 🎨 Dynamic Routing Beta: Extended collection feature (limit, order, data)
refs #9601

- support data, limit and order for collections
- limit definition in routes.yaml is stronger than theme package.json limit configuration
- ensure we update hbs template options
2018-06-24 02:06:58 +02:00
kirrg001
51e6286924 Dynamic Routing Beta: Added redirect middleware to ParentRouter
refs #9601

- middleware to control dominant router redirects
2018-06-24 02:06:58 +02:00
kirrg001
1952c55d33 Dynamic Routing Beta: Added redirect helper to ParentRouter
refs #9601

- this function will help us to determine if a redirect should happen or not
2018-06-24 02:06:58 +02:00
kirrg001
935f064357 Dynamic Routing Beta: Added yaml validation for data key
refs #9601

- support short and long form of data key
- always return the expanded version (long) to the routers
2018-06-24 02:06:58 +02:00
Katharina Irrgang
5a61f99467
Dynamic Routing: Added migration for routes.yaml file (#9692)
refs #9601

- the home.hbs behaviour for the index collection (`/`) is hardcoded in Ghost
- we would like to migrate all existing routes.yaml files
- we only replace the file if the contents of the routes.yaml file equals the old routes.yaml format (with home.hbs as template)
- updated README of settings folder
- if we don't remove the home.hbs template from the default routes.yaml file, home.hbs will be rendered for any page of the index collection
  - the backwards compatible behaviour was different
  - only render home.hbs for page 1
- remember: the default routes.yaml file reflects how Ghost was working without dynamic routing
2018-06-22 20:28:01 +02:00
kirrg001
a1b55509df Dynamic Routing Beta: collection name behaviour
refs #9601

Example:

```
collections:
  /podcast/:
    permalink: /{slug}/
```

- the name of the collection is remembered as `routerName` (in the case above: "podcast")
- the name of the collection is important for two things
  1. context value
  2. template name
- the context value is available for specific theme helpers e.g. is helper, body_class helper
- we auto-lookup the collection name in your theme e.g. podcast.hbs
- this logic does not apply to static routes
- if you define templates on your collection, they are stronger than the collection name
2018-06-21 20:59:43 +02:00
kirrg001
0046dce39f Dynamic Routing Beta: Better template support
refs #9601

- single or multiple template definition
- possible formats:

```
routes:
  /about/: about
```

```
routes:
  /about/:
    template: about
```

```
routes:
  /about/:
    template:
      - about
      - me
```

```
collections
  /posts/:
    template:
      - posts
      - general
```

```
collections
  /posts/:
    template: posts
```
2018-06-21 16:22:45 +02:00
kirrg001
1a79aac673 Fixed collection ownership if a published post status changes to featured
refs #9601

- implementation of resource listener updated
- if you define two collections: `featured:true` (1) and `page:false` (2) you can run into the following bug:
  - you create a published post (owned by (2))
  - you change the status to featured
  - still owned by (2), because the filter still matches (it's still not a static page)
- this adaption fixes the behaviour
- less smart logic, but less error prone
2018-06-17 10:41:05 +02:00
Kevin Ansfield
a16077a8e3 Koenig - Do not render image cards with no src on the front-end
refs https://github.com/TryGhost/Ghost/issues/9623
- blank images may be used in the editor as placeholders, don't render them on the front-end
2018-06-14 14:57:09 +01:00
kirrg001
87c01c131b 🐛 Fixed {{#is "index"}}
closes https://github.com/TryGhost/Ghost/issues/9674

- with dynamic routing the first collection get's the "index" context attached
- the index context signalises the main post listening route (first collection)
- this behaviour was present < 1.24 - we have to keep this behaviour
2018-06-14 11:53:13 +02:00
Katharina Irrgang
835fd6c45b
Removed knex mock (#9685)
no issue

- this mock eat already too much of my/our time
- the idea of adding a knex mock was definitely a failed approach/try
- it's too much to maintaince and have not found a module which does this already
  - we have to support any query format
  - this is too crazy
- the idea was to use the knex mock for model unit tests, because if we want to unit test models we have to
  run through bookshelf, because the whole model layer depends on bookshelf e.g. events
- for now we simply use the real database
  - we could use the sqlite3 memory mode, but that would mean every unit test runs on sqlite3
  - something to consider for later e.g. run unit tests on one matrix
  - run the rest on another matrix for sqlite + mysql
2018-06-12 20:26:16 +02:00
Katharina Irrgang
7b0d5d465b 🐛 Fixed preview url and Zapier on subdirectory (#9683)
closes #9675

- with dynamic routing we have introduced a breaking change, which we have overseen
- Ghost does not return absolute urls, that's why the clients need to concat the blog url and the resource url
- with 1.24.0 Ghost returned resource urls including the subdirectory
- this caused trouble for e.g. zapier or the preview feature in the admin client
- revert breaking change and ensure we only expose resource urls without subdirectory
2018-06-12 16:36:58 +01:00