Commit Graph

2669 Commits

Author SHA1 Message Date
Katharina Irrgang
5b77f052d9
Update Notification improvements (#9123)
closes #5071

- Remove hardcoded notification in admin controller
  - NOTE: update check notifications are no longer blocking the admin rendering
  - this is one of the most import changes
  - we remove the hardcoded release message
  - we also remove adding a notification manually in here, because this will work differently from now on
    -> you receive a notification (release or custom) in the update check module and this module adds the notification as is to our database

- Change default core settings keys
  - remove displayUpdateNotification
    -> this was used to store the release version number send from the UCS
    -> based on this value, Ghost creates a notification container with self defined values
    -> not needed anymore

- rename seenNotifications to notifications
  -> the new notifications key will hold both
     1. the notification from the USC
     2. the information about if a notification was seen or not
  - this key hold only one release notification
  - and n custom notifications

- Update Check Module: Request to the USC depends on the privacy configuration
  - useUpdateCheck: true -> does a checkin in the USC (exposes data)
  - useUpdateCheck: false -> does only a GET query to the USC (does not expose any data)
  - make the request handling dynamic, so it depends on the flag
  - add an extra logic to be able to define a custom USC endpoint (helpful for testing)
  - add an extra logic to be able to force the request to the service (helpful for testing)

- Update check module: re-work condition when a check should happen
  - only if the env is not correct
  - remove deprecated config.updateCheck
  - remove isPrivacyDisabled check (handled differently now, explained in last commit)

- Update check module: remove `showUpdateNotification` and readability
  - showUpdateNotification was used in the admin controller to fetch the latest release version number from the db
  - no need to check against semver in general, the USC takes care of that (no need to double check)
  - improve readability of `nextUpdateCheck` condition

- Update check module: refactor `updateCheckResponse`
  - remove db call to displayUpdateNotification, not used anymore
  - support receiving multiple custom notifications
  - support custom notification groups
  - the default group is `all` - this will always be consumed
  - groups can be extended via config e.g. `notificationGroups: ['migration']`

- Update check module: refactor createCustomNotification helper
  - get rid of taking over notification duplication handling (this is not the task of the update check module)
  - ensure we have good fallback values for non present attributes in a notification
  - get rid of semver check (happens in the USC) - could be reconsidered later if LTS is gone

- Refactor notification API
  - reason: get rid of in process notification store
    -> this was an object hold in process
    -> everything get's lost after restart
    -> not helpful anymore, because imagine the following case
      -> you get a notification
      -> you store it in process
      -> you mark this notification as seen
      -> you restart Ghost, you will receive the same notification on the next check again
      -> because we are no longer have a separate seen notifications object
  - use database settings key `notification` instead
  - refactor all api endpoints to support reading and storing into the `notifications` object
  - most important: notification deletion happens via a `seen` property (the notification get's physically deleted 3 month automatically)
    -> we have to remember a seen property, because otherwise you don't know which notification was already received/seen

- Add listener to remove seen notifications automatically after 3 month
  - i just decided for 3 month (we can decrease?)
  - at the end it doesn't really matter, as long as the windows is not tooooo short
  - listen on updates for the notifications settings
  - check if notification was seen and is older than 3 month
  - ignore release notification

- Updated our privacy document
- Updated docs.ghost.org for privacy config behaviour
- contains a migration script to remove old settings keys
2018-01-09 15:20:00 +01:00
juan-g
f671f9d2c9 Theme translations and blog localisation (#8437)
refs #5345, refs #3801

- Blog localisation
  - default is `en` (English)
  - you can change the language code in the admin panel, see https://github.com/TryGhost/Ghost-Admin/pull/703
  - blog behaviour changes depending on the language e.g. date helper format
  - theme translation get's loaded if available depending on the language setting
  - falls back to english if not available

- Theme translation
  - complete automatic translation of Ghost's frontend for site visitors (themes, etc.), to quickly deploy a site in a non-English language
  - added {{t}} and {{lang}} helper
  - no backend or admin panel translations (!)
  - easily readable translation keys - very simple translation
  - server restart required when adding new language files or changing existing files in the theme
  - no language code validation for now (will be added soon)
  - a full theme translation requires to translate Ghost core templates (e.g. subscriber form)
  - when activating a different theme, theme translations are auto re-loaded
  - when switching language of blog, theme translations are auto re-loaded

- Bump gscan to version 1.3.0 to support more known helpers

**Documentation can be found at https://themes.ghost.org/v1.20.0/docs/i18n.**
2018-01-09 14:50:57 +01:00
Katharina Irrgang
dcb2aa9ad4
Improved forwarding model options in the API layer (#9380)
no issue

- our API layer uses a unit to combine incoming data and options
- e.g. `options.data` is the end result
- we have to take care that we don't pass data into the model layer

Credits: Olivier Arteau
2018-01-09 13:26:09 +01:00
Aileen Nowak
430973b85b 🐛 Fixed date helper timezone bug
closes #9381

Fixes a bug where the date helper would ignore any timezone settings, when called with a specific date option, e. g. `published_at`, as `timezone` was only ever assigned when called without options.
2018-01-09 13:05:54 +01:00
Aileen Nowak
b99d3bc70a Revert "🐛 Fixed date helper timezone bug (#9382)"
This reverts commit c3b0219d70.
2018-01-09 13:05:54 +01:00
汪磊
c3b0219d70 🐛 Fixed date helper timezone bug (#9382)
closes #9381

Fixes a bug where the date helper would ignore any timezone settings, when called with a specific date option, e. g. `published_at`.
2018-01-09 17:44:01 +07:00
kirrg001
a1393762f6 Lazily require modules in lib/common/index
no issue

- decreases chance of not-loaded modules or circular dependencies
- e.g. the i18n implementation will use the settings-cache and the settings-cache uses lib/common/events
2018-01-08 21:12:16 +01:00
Aileen Nowak
19a6c8a426 Added test for images w/o extensions for image size util (#9367)
closes #9022

Images without extensions don't need to be manipulated, as we're now reading the bytes and pass those to the `image-size` lib.

This PR adds another `user-agent` to emulate multiple browser requests, as I stumbled over an example where the image without extension is protected otherwise.

Added a test, that works with above mentioned image, but is currently mocked. Nevertheless, the image worked as a PoC, that we're able to read the bytes of an image without its extension and still return the dimensions of the image.
2018-01-04 12:03:41 +01:00
Katharina Irrgang
341f719d92
Extended ghost version utility (#9278)
refs https://github.com/TryGhost/Ghost-Release/issues/24

- differentiate between
  1. original package.json version (can contain pre and build suffix)
  2. full package.json version X.X.X-{pre} (optional)
  3. safe package.json version X.X (major+minor)
2018-01-03 20:20:18 +01:00
kirrg001
eb0a11d53a Fixed import test: post duplication detection within a file to import
no issue

- with 29e143fa9a import queries no longer run in parallel
- this commit simply adds a small code snippet to reflect the importer behaviour

1) duplicate slugs *within* a file are getting ignored
2) existing posts in the database and posts to import with the same slug, result in duplicates

Further improvements regarding duplication detection will happen via #8717.
2018-01-03 13:34:15 +00:00
kirrg001
428008e63d Revert "🐛 Fixed importer duplicate detection for posts"
refs #8717

- we decided to not changing the current importer behaviour
- no slug duplication detection means, importing posts can result in duplicates
2018-01-03 13:34:15 +00:00
kirrg001
02bd71d0f5 🐛 Fixed importer duplicate detection for posts
closes #8717

- this is now required, because we run import queries sequentiell
- this code protects two cases:
  - you have duplicate slugs in the JSON file (the first get's inserted, the second get's ignored)
  - you have an existing slug in the database and you try to import the same slug, get's ignored
2018-01-03 00:07:41 +01:00
kirrg001
29e143fa9a 🐛 Fixed large imports
closes #9348

- do not run import with `Promise.all`
- with a large import file, we run an enormous amount of queries in parallel, which does not allow Node to cleanup memory
- tested with an 13mb import file
- requires bookshelf-relations 0.1.4
2018-01-02 23:21:45 +01:00
Aileen Nowak
ae741b1a18 Removed bluebird promise wrap in request lib (#9343)
refs #9178, refs #8988

With 7353c87d7f we use Bluebird globally for Promises. Therefore, the request lib doesn't need to be wrapped in a bluebird Promise anymore.

This was originally done, so we can work with catch predicated in our image-size lib.

Updated the tests to proof, that the catch predicates work.

The tests fail, as soon as the Promise overwrite is commented out.
2018-01-02 12:18:56 +01:00
汪磊
7a98c99cba Renamed BasicErorRenderer to BasicErrorRenderer (#9358)
no issue

- typo 🙊
2018-01-02 10:33:43 +01:00
Hannah Wolfe
bcf6e9f517 Fixed error trying to render 404 for missing asset
refs #8868

- Loading the admin prior to a build results in: Failed to lookup view "error-404" in views directory
- This fixes that error, by splitting the HTMLErrorRenderer and the ThemeErrorRenderer into two separate things
2017-12-22 20:08:59 +00:00
juan-g
4cca2353e0 consistency change for english translation keys: use camel case (#9347)
no issue

- required for #8437 
- one instance of hyphenated key changed; the rest of keys in file
_core/server/translations/en.json_ are already camelCase
- also converted `common.i18n.t()` calls to this key in file
_core/server/update-check.js_
- this allows to simplify i18n to an unified use of `jsonpath`
2017-12-18 15:47:55 +01:00
Katharina Irrgang
a5af7d6384 Used request lib in xmlrpc (#9333)
refs #9178, refs https://github.com/TryGhost/Ghost/pull/8980
2017-12-15 16:50:18 +07:00
Katharina Irrgang
6e94cedfa2 Ensure cors check happens for /authentication/token route (#9317)
no issue

- otherwise external browser clients run into cors problems
2017-12-15 09:35:48 +00:00
Abijeet Patro
73c0bf554a 🐛 Fixed emoji in Slack test notification not rendered by Discord (#9313)
closes #9263

- changed emoji for Slack test notification to 😄 which will be correctly rendered as emoji by both, Slack and Discord.
2017-12-15 11:19:37 +07:00
kirrg001
991ccb1d35 Moved make-absolute-urls to url service
refs #9178

- this util uses the url services (!)
- moving this file into lib would not make sense right now
  - that would mean a module requires first ../lib/url, which then requires ../services/url
- the url service definitely need a clean up 😃
2017-12-14 22:34:37 +01:00
kirrg001
64626dedd1 Moved social utility to lib/social
refs #9178

- not 100% sure about this, but i think it makes right now the most sense
- we have already a url service and creating another lib/url is confusing at the moment
- i'll copy the last utility `makeAbsoluteUrls` to the url service for now
- see next commit for explanation (!)
2017-12-14 22:34:05 +01:00
kirrg001
b474fb0d16 Moved ghost-version to lib
refs #9178
2017-12-14 22:14:55 +01:00
kirrg001
1a9a10c82b Moved zip folder, read csv and package-json to lib/fs
refs #9178, refs 849e97640f

- i've reconsidered, these modules belong to lib
- prettify package-json module
2017-12-14 22:07:53 +01:00
kirrg001
82597080be Burn dirty require of api utils
refs #9178

- `checkFileExists` and `checkFileIsValid` where dirty required from web/middleware
- these two functions are only used in the target middleware
- let's move them
2017-12-14 21:25:51 +01:00
kirrg001
849e97640f Moved zip folder utility to services/themes
refs #9178

- it's only used for themes currently
- this is definitely a util which could be useful lib
- if we have a second use case, we can move it
2017-12-14 21:02:47 +01:00
kirrg001
fc5b4dd934 Moved image utils to lib/image
refs #9178

- i am not super happy about `const imageLib = require('../lib/image')`
- i don't really like the name `imageLib`
- but i had no better idea 😃
- if we use the same name in the whole project, it's very easy to rename the folder or the variable
2017-12-14 20:46:53 +01:00
kirrg001
740b247a80 Avoid moment deprecation warning when validating incoming dates
no issue

> Deprecation warning: value provided is not in a recognized ISO format. moment construction falls back to js Date(), which is not reliable across all browsers and versions.
2017-12-14 17:04:06 +01:00
kirrg001
634fdbfa96 Tests: Do not re-register model events in settings cache
no issue

> (node:63849) Warning: Possible EventEmitter memory leak detected. 101 settings.edited listeners added. Use emitter.setMaxListeners() to increase limit

- the settings cache was initialised per test
- it registered the model events over and over again
- add a simple shutdown function, which can be called from the test env
2017-12-14 16:41:30 +01:00
Katharina Irrgang
18e15934fd Used request lib in slack service (#9335)
refs #9178

- tested with slack
2017-12-14 22:08:48 +07:00
Katharina Irrgang
50b65bca0c Moved default-cards app to lib/mobiledoc (#9341)
refs #9178, refs https://github.com/TryGhost/Ghost/pull/9338
2017-12-14 14:44:01 +00:00
kirrg001
a3091a3012 Moved utils constants to lib/constants
refs #9178
2017-12-14 14:13:40 +01:00
kirrg001
c5169e23c4 Moved unique identifier generation to lib/security
refs #9178
2017-12-14 13:52:20 +01:00
kirrg001
bb06a8426d Moved tokens, url safe and safe string utility to lib/security
refs #9178

- we could now also move any crypto usages to lib/security, but no priority
- the main goal is to tidy up our utils folder
2017-12-14 13:38:00 +01:00
Katharina Irrgang
9de13ae3ad Moved mobiledoc/markdown converters to apps/default-cards (#9338)
refs #9178

- they definitely don't belong to server/utils
- i think the best place is putting them into the card apps
- the the post model needs to ask the app for it's converters
- move tests as well
2017-12-14 11:09:54 +00:00
kirrg001
4fe7c9d7fc Moved validateRedirects helper to data/validation
refs #9178
2017-12-14 11:55:10 +01:00
Katharina Irrgang
2956c2c247 Use request lib in webhooks api (#9336)
refs #9178

- add retries and timeout
2017-12-14 10:00:34 +00:00
Katharina Irrgang
941379ddba Use request lib in gravatar utility (#9337)
refs #9178
2017-12-14 09:51:05 +00:00
Katharina Irrgang
192ebb1739
Moved labs, auth, permissions, settings, mail, themes to services (#9339)
refs #9178

- move tests as well
2017-12-14 03:01:23 +01:00
kirrg001
dac9ffc473 Removed leftover usages for globalUtils.sequence
refs #9178
2017-12-14 01:40:15 +01:00
kirrg001
660a407c16 Burn asset-hash utility
refs #9178

- Ghost uses the Node crypto lib always direct (require('crypto'))
- it doesn't make sense to outsource a single crypto statement (for the asset hash)
- we either have to write a crypto wrapper to avoid writing long crypto statements or we keep the direct usages for every case
- for now, wrapping the crypto calls into a lib/crypto has no priority
2017-12-14 01:33:01 +01:00
kirrg001
11a8245ca6 Added TODO to core/server/data/validation/index.js
no issue

- we are modifying the global validator dependency, which is bad
- see https://github.com/chriso/validator.js/issues/525#issuecomment-213149570
- we should create our own class and expose a singleton for the whole Ghost project
2017-12-14 01:15:12 +01:00
Katharina Irrgang
11765351cc
Use request lib in update-check module (#9334)
refs #9178

- works with local update check service
- tested success/error cases
2017-12-14 00:26:11 +01:00
kirrg001
7460076206 Moved request to lib/request
refs #9178

- next steps are to
  A: test if global.Promise works with `got` (see https://github.com/TryGhost/Ghost/pull/8988#issuecomment-351532881)
  B: re-use request utility everywhere
  C: request lib requires data/validator, which is dirty
2017-12-13 22:37:42 +01:00
kirrg001
f83cbf6117 Moved pipeline/sequence to lib/promise
refs #9178

- continue with killing our global utils folder
- i haven't found any better naming for lib/promise
- so, require single files for now
- instead of doing `promiseLib = require('../lib/promise')`
- we can optimise the requires later
2017-12-13 22:20:02 +01:00
kirrg001
0d65e7f24f Renamed apiUtils to localUtils - consistency change
refs #9178

- we should always use the same naming patterns
2017-12-13 22:14:19 +01:00
kirrg001
6e915e8e89 Moved removeOpenRedirectFromUrl to local web utils
refs #9178

- see https://github.com/TryGhost/Ghost/issues/9178#issuecomment-351521897
2017-12-13 22:06:31 +01:00
kirrg001
485c264c69 Use Promise support which was added in fs-extra 3.x
no issue

- now that we use bluebird globally, we can use the promise support from fs-extra
2017-12-13 20:57:11 +01:00
kirrg001
d83f474ff8 Always require fs-extra - consistency change
no issue
2017-12-13 20:57:11 +01:00
kirrg001
7353c87d7f Set Promise to be Bluebird globally
closes #9064
2017-12-13 20:57:11 +01:00
Katharina Irrgang
4f35f86713
Moved word-count utility to helpers/utils.js (#9328)
refs #9178

- each package/module has a local utility (e.g. api, helpers, adapters)
- these are very small utility functions which are only used from this package
- they don't belong into the global lib/utils
2017-12-13 14:05:53 +01:00
Katharina Irrgang
397400b4f8
Moved visibility utility to static model fn (#9327)
refs #9178

- this logic belongs to a static model helper
- the visibility property is a model property, the knowledge about the visibility values belongs to the model
- rename the functions, so they make more sense
2017-12-13 13:19:51 +01:00
Hannah Wolfe
79ca6c575c Cleared out old unused events
no issue

- Because of 9248 we send too many events and this is not reliable
- Remove everything except the well understood published events
2017-12-13 09:08:25 +00:00
kirrg001
2c2d1e93cc Fixed wrong require path in webhook-permissions
no issue

- the require path to the logging module was wrong
- because of ac2578b419
2017-12-12 15:23:07 +01:00
Katharina Irrgang
a98346e2ce Migration for webhook permissions (#9320)
refs https://github.com/TryGhost/Ghost/pull/9230
2017-12-12 14:12:34 +00:00
kirrg001
6f6c8f4521 Import lib/common only
refs #9178

- avoid importing 4 modules (logging, errors, events and i18n)
- simply require common in each file
2017-12-12 10:28:13 +01:00
kirrg001
ac2578b419 Moved errors,logging,i18n and events to lib/common
refs #9178
2017-12-12 10:28:13 +01:00
kirrg001
4265afe580 Moved utils/url.js to UrlService
refs #9178

- we have to take care that we don't end up in circular dependencies
  - e.g. API requires UrlService and UrlService needs to require the API (for requesting data)
- update the references
- we would like to get rid of the utils folder, this is/was the most complicated change
2017-12-11 20:05:33 +01:00
kirrg001
7d99ee8466 Small UrlService optimisations
no issue

- rename the config option to disable preloading the urls
- always expose the urlservice as singleton
- do the initialisation of the service inside the constructor
2017-12-11 20:05:33 +01:00
Katharina Irrgang
7bcccc71dc
Moved apps into web folder (#9308)
refs #9178

- move express apps to one place (called `web`)
- requires https://github.com/TryGhost/Ghost-Admin/pull/923
- any further improvements are not part of this PR
- this PR just moves the files and ensures the paths are up-to-date
2017-12-06 17:37:54 +01:00
Katharina Irrgang
0bb81bb3c4
Bump knex-migrator to version 3.1.1 (#9199)
no issue

- adapt major changes of knex-migrator v3
- adapt migration scripts, simplify and add `down` (rollback) hook if possible
- clear Ghost cache after init hook (because of `knex-migrator migrate --init`)
- ensure db migrations work with the CLI
- updated troubleshooting guide (https://docs.ghost.org/v1/docs/troubleshooting#section-task-execute-is-not-a-function)

**For development only: Please ensure you run `npm i -g knex-migrator@latest` to update your global installation to v3. We always prefer the local installation, but v3 has modified and added binaries.**
2017-12-05 09:14:55 +01:00
kirrg001
533ae22c2b Added protection for incorrect i18n variable names
closes #9305

- add a try/catch when formatting messages
- fallback to the general error message if error ocurred
- log the error
2017-12-04 17:13:12 +01:00
kirrg001
0be734a05c Fixed incorrect i18n variable names
no issue
2017-12-04 16:58:02 +01:00
Katharina Irrgang
9e388aee4d 🎨 Improved error handling for images on file storage which don't exist (#9282)
refs https://github.com/TryGhost/Team/issues/41

- differentiate error codes
- return 404 if image was not found
- else return a 500
- use i18n keys
- use errors.utils.isIgnitionError (!)
2017-11-28 14:27:18 +00:00
kirrg001
be9ce107bd Re-attach error to request object if error occurs when rendering error template
no issue

- otherwise we won't see the error in the logs
2017-11-28 13:44:14 +00:00
kirrg001
e8075262eb 🎨 Improved pagination misusage error
refs https://github.com/TryGhost/Team/issues/41, refs https://github.com/TryGhost/gscan/issues/85

- if you are using the pagination helper not inside a resource context, you will receive an error
- improve error message, because it was not clear what happened
- downgrade error level to normal, because it's not a critical error from Ghost's perspective, from user perspective it is
- added help docs link and added a callout to our docs
2017-11-28 13:44:14 +00:00
Katharina Irrgang
d87fbeca49 🐛 Fixed error handling for html response (#9280)
refs https://github.com/TryGhost/Team/issues/41

- if you add invalid handlebars logic e.g. {{if condition condition}}, handlebars throws an error
- in case of having invalid hbs in an amp page, the amp component throwed another syntax error (which is fixed in this PR)
- furthermore the `setTemplate` helper function had a logic bug, which did not handle errors correctly
  - if there is an error and a template is set (e.g. amp), we have to still render the error page and not the amp page
- this fix only ensures that the error handling is correct, we still see the error of the "ugly" handlebars message
  - e.g. [amp.hbs] Cannot read property 'includeZero' of undefined
  - but no longer -> Cannot read property 'html' of undefined (which was a syntax error in Ghost)
2017-11-28 12:00:43 +00:00
Katharina Irrgang
3004e03bc6 🐛 Fixed throwing 500 for invalid encoded urls (#9279)
refs https://github.com/TryGhost/Team/issues/41

- if you send invalid encoded url components in the path, the server tried to decode the url
- if it contains invalid characters like /AF%, it throwed a 500
- we return a page not found error instead
2017-11-28 11:39:38 +00:00
Guillem Andreu
860b38a1a7 🐛 Fixed Sitemap when permalink contains Primary Tag (#9273)
closes #9272

- included tags in sitemap post-generator
2017-11-27 12:38:56 +01:00
Kevin Ansfield
94360c8a32
🐛 Fixed import and redirect uploads for Win10/Edge (#9267)
closes https://github.com/TryGhost/Ghost/issues/9236
- added `text/plain` to the allowed mime types for db and redirect uploads
2017-11-22 17:30:53 +00:00
Kevin Ansfield
bffb3dbd90
Webhooks support for subscriber events (#9230)
no issue

Support for http://resthooks.org style webhooks that can be used with Zapier triggers. This can currently be used in two ways:

a) adding a webhook record to the DB manually
b) using the API with password auth and POSTing to /webhooks/ (this is private API so not documented)

⚠️ only _https_ URLs are supported in the webhook `target_url` field 🚨

- add `webhooks` table to store event names and target urls
- add `POST` and `DELETE` endpoints for `/webhooks/`
- configure `subscribers.added` and `subscribers.deleted` events to trigger registered webhooks
2017-11-21 15:43:14 +00:00
Katharina Irrgang
dfd4afea19 Add bookshelf-relations (#9252)
no issue

- added https://github.com/TryGhost/bookshelf-relations as dependency
- remove existing tag handling

--- 

* Important: Ensure we trigger parent initialize function

- otherwise the plugin is unable to listen on model events
- important: event order for listeners is Ghost -> Plugin
- Ghost should be able to listen on the events as first instance
- e.g. be able to modify/validate relationships

* Fix tag validation

- we detect lower/update case slugs for tags manually
- this can't be taken over from the plugin obviously
- ensure we update the target model e.g. this.set('tags', ...)

* override base fn: `permittedAttributes`

- ensure we call the base
- put relations on top
- each relation is allowed to be passed
- the plugin will auto-unset any relations to it does not reach the database

* Ensure we run add/edit/delete within a transaction

- updating nested relationships requires sql queries
- all sql statements have to run in a single transaction to ensure we rollback everything if an error occurs
- use es6
2017-11-21 13:28:05 +00:00
Aileen Nowak
982a75d6be 🐛 Fixed slugs from exceeding db limit (#9251)
closes #8143

Fixed a potential issue (edge-case), where our generated and validated (in terms of check for existance and add a counter) would return a slug, that will exceed the maximum length of the slug fields (191 chars).

This is mostly possible for the post title, which can be 255 chars long and would generate a slug with the same length. This would prevent the user from actually saving a post.

I tried first to determine the expected length for a slug that already exists, but decided that the **easier** and simplyfied implementation is to always cut a slug to **185 chars** (+ counter). This makes it easier to find duplicates and includes a possible high number of counts (edge-edge-case).

The slug will not be cut down to 185 chars if it's an import.
2017-11-21 14:21:22 +01:00
Hannah Wolfe
5e5b90ac29
Added Url Service to track all URLs in the system (#9247)
refs #9192

- Introduces a url service that can be initialised
- Added a concept of Resources and resource config.json that contains details about the resources in the system that we may want to make customisable 
- Note that individual resources know how to create their own Urls... this is important for later
- Url Service loads all of the resources, and stores their URLs
- The UrlService binds to all events, so that when a resource changes its url and related data can be updated if needed
- There is a temporary config guard so that this can be turned off easily
2017-11-16 13:03:24 +00:00
Aileen Nowak
1bb9d4ff00 Set soft limits for blog title and description (#9250)
refs #8143

Add max length validations to settings:
- `blog.title`: 150 chars
- `blog.description`: 200 chars

The `validateSettings` fn in our validations checks for existing `validations` properties in our `default-settings.json` file, similar to other tables in our `schema.js`.
2017-11-16 13:58:22 +01:00
kirrg001
c4044aa498 Do not log error if i18n.doesTranslationKeyExist is called
no issue

- when calling `doesTranslationKeyExist`, we want to know if a key exists, we don't want to log if the key was not found
- this can mess up the server log
2017-11-14 14:03:48 +01:00
Katharina Irrgang
76689ecbee 🐛 Fixed pagination error (#9243)
no issue

- see explanation https://github.com/TryGhost/Ghost/pull/8129#issuecomment-286061529
- we catch the db error if it occurs, we can't simply check the length of the page param, because sqlite/mysql behaves differently
2017-11-14 12:47:58 +00:00
Kevin Ansfield
bcc98e5536 Added GET/DELETE /subscribers/email/:email/ endpoints (#9238)
no issue

- useful for managing subscribers via external systems/API calls where it's likely only the e-mail address will be known
- adds `GET /subscribers/email/:email/`
- adds `DELETE /subscribers/email/:email/`
2017-11-14 12:09:41 +01:00
John O'Nolan
0d54326121 Give default Ghost user fixture a helpful bio (#9241)
no issue

Had a couple of people ask about how to delete welcome posts easily, so adding a bio to the default user to draw a little more attention to it
2017-11-14 10:23:38 +00:00
kirrg001
0eb84d7f8a 🐛 Fixed concurrent renew of access tokens
no issue

- it can happen that concurrent requests try to renew access tokens with the same refresh token
- in this case it could happen that you received a token deletion error
- add propert locking
- ensure we don't run into deadlocks
- manual testing with async.times for parallel requests (was able to reproduce the error)
2017-11-14 10:22:09 +00:00
kirrg001
4b21fc1d59 Allow forUpdate for any model
no issue

- this was only supported for the Post Model until now
- locking should be possible for every resource depending on the use case
2017-11-14 10:22:09 +00:00
Hannah Wolfe
1445678035 Fixed channel path to work with Ghost-CLI
- Just, DERP 🙈
2017-11-10 20:52:50 +00:00
Hannah Wolfe
98f5ae00fc
Introduced renderer to DRY up controllers (#9235)
refs #5091, #9192

- Renderer figures out templates, contexts, and does a render call
- Templating is now handled with a single function
- Context call is made in the renderer

Note:  to make this work, all controllers now define a little bit of config, currently stored in res._route. (That's a totally temporary location, as is res._template... when a sensible naming convention reveals itself I'll get rid of the weird _). This exposes a type and for custom routes a template name & default.
2017-11-10 12:44:29 +00:00
Hannah Wolfe
e41d0c76fb
RSS service + controller improved for consistency (#9233)
refs #9192, refs #5091 

- Moved all url generation into generate-feed.js, so we can see as much data processing as possible in a single place.
- Refactored the way res.locals were used, to be more like how express uses them prior to rendering
- Removed a bunch of code & tests todo with context for RSS - I can't see any way that'd be used, unless we switched the rendering to use a template.
- moved the RSS rendering to be part of the service, not controller
- updated the tests significantly 

Note: RSS generate-feed has a complete duplication of the code used in the excerpt helper in order to create an item description
2017-11-10 07:36:39 +00:00
Aileen Nowak
a35c0c29e0 Added more database soft limits (#9225)
refs #8143

Sets soft limits for certain db fields:

- `posts`:
	- `title`: 255 chars (current hard limit: 2,000 chars)
	- `meta_title`: 300 chars (current hard limit: 2,000 chars)
	- `meta_description`: 500 chars (current hard limit: 2,000 chars)
- `users`:
	- `bio`: 200 chars (current hard limit: 65,535 chars)
	- `location`: 150 chars (current hard limit: 65,535 chars)
	- `meta_description`: 500 chars (current hard limit: 2,000 chars)
	- `meta_title`: 300 chars (current hard limit: 2,000 chars)
- `tags`:
	- `description`: 500 chars (current hard limit: 65,535 chars)
	- `meta_title`: 300 chars (current hard limit: 2,000 chars)
	- `meta_description`: 500 chars (current hard limit: 2,000 chars)

- same error message for isLength validator as for hard limits (more improvements are comming with https://github.com/TryGhost/Ghost/issues/6050)
- added more tests for importer
- added dynamic translation key handling for validators errors (isLength is only supported atm)
2017-11-09 15:22:20 +01:00
Katharina Irrgang
f22a2784f7 🐛 Fixed error for password authentication with Bearer Token (#9227)
refs #8613, refs #9228

- if you send a request to /authentication/token with `grant_type:password` and a Bearer token, Ghost was not able to handle this combination
- because it skipped the client authentication, see https://github.com/TryGhost/Ghost/blob/1.17.0/core/server/auth/authenticate.js#L13
- and OAuth detects the `grant_type: password` and jumps in the target implementation
- the target implementation for password authentication **again** tried to fetch the client and failed, because it relied on the previous client authentication
- see https://github.com/TryGhost/Ghost/blob/1.17.0/core/server/auth/oauth.js#L40 (client.slug is undefined if client authentication is skipped)
- ^ so this is the bug
- we **can** skip client authentication for requests to the API to fetch data for example e.g. GET /posts (including Bearer)
- so when is a client authentication required?
- RFC (https://tools.ietf.org/html/rfc6749#page-38) differentiates between confidential and public clients, Ghost has no implementation for this at the moment
  - so in theory, public clients don't have to be authenticated, only if the credentials are included
- to not invent a breaking change, i decided to only make the client authentication required for password authentication
- we could change this in Ghost 2.0

I have removed the extra client request to the database for the password authentication, this is not needed. We already do client password authentication [here](https://github.com/TryGhost/Ghost/blob/1.17.0/core/server/auth/auth-strategies.js#L19);
If a Bearer token is present and you have not send a `grant_type` (which signalises OAuth to do authentication), you can skip the client authentication.
2017-11-09 14:11:29 +00:00
Hannah Wolfe
016ee17ebb
Rework route service to prevent circular dependencies (#9229)
refs #9192, refs #9178  

After trying to progress with current implementation, it became clear that the route service can't control the boot sequence, because then we end up with circular dependencies between the route service and the channel service.

The route service now exposes:
-  a siteRouter 
- a way for apps to register routes.
- ParentRouter base class for other modules to use
- the registry

...

- moved the default route setup back to site/routes.js 🙈
- moved the parent channel router back to the channel service (this makes way more sense imo)
- this structure prevents circular dependencies
- split the registry out into it's own thing
- fixed-up various bits of tests and comments
- DEBUG will print a list of routes 🎉
2017-11-09 13:58:22 +00:00
Hannah Wolfe
27b4688cea Changed channelsRouter to use new base class
refs #9192, #5091

- changed channels to use our new base class
- keep the flexible structure, so that channels can be reloaded
- I had to move the router into the route service otherwise we get circular dependencies
- Don't _really_ want to keep it like this - need a way to define base classes as shared
2017-11-09 10:47:20 +00:00
Aileen Nowak
6a41104969 Moved isLocalFile fn to storage utils (#8985)
no issue

- moved isLocalImage fn to storage utils used the RegExp of getLocalFileStoragePath to detect also relative image paths and added tests.
- Added test for independent protocol request (skip, because not supported/implemented)
2017-11-09 11:11:54 +01:00
Hannah Wolfe
7656d0bdda
Wrapped express router & expose from route service (#9206)
refs #9192

- Moving towards a centralised concept of routing / routes
- The base router now wraps express router, and offers us the features we need
- Site Router is the parent router, it gets initialised with all of our default routing
- App Router is a sub router for apps - apps register their routes/routers onto it.
- TODO: refactor channels subrouter to work this same way
- MAYBE: move the app router to the apps service
2017-11-09 10:08:11 +00:00
Hannah Wolfe
90cfdbe7a6 Moved Channels module -> services
refs #9192, refs #5091, refs #9178

- moved channels from controllers to a service
- split out the parent router from the remaining individual router logic
- moved the tests to match
2017-11-08 15:45:23 +00:00
Hannah Wolfe
f0f0735437 Renamed single / post -> entry
refs #9192

- an entry is a post or a page, represented by a post model
2017-11-08 10:07:52 +00:00
Hannah Wolfe
187c38991e Move admin redirects out of site router
refs #9192

- Admin redirects should really happen first, up with custom redirects
- Later we can package this up, maybe
- For now, let's focus the site router on site-related things
2017-11-08 09:45:34 +00:00
Hannah Wolfe
abaf0461cf Highlighted routes, controllers & renderers
refs #5091, refs #9192

- There are several theme template "renderers" all over the codebase
- Some are in apps, and were called "controllers"
- One is in error handling
- All of them now have comments marking out how they share logic/steps
- Other comments describe routes & controllers where they live
2017-11-08 09:45:12 +00:00
Hannah Wolfe
474e9234a6 Simplified AMP internal app
refs #9192

- The AMP app is nothing more than a custom controller - this will come clear soon
- Moved enabled/disabled logic into router
- Removed error-related code, as this wasn't used
- Changed logic for static pages to be based on req.body, not context
- Improved the tests to match
2017-11-08 08:25:25 +00:00
Hannah Wolfe
86c6cec433 Moved RSS module -> services & controllers
refs #5091, #9192, #9178

- Get the RSS module into a much better shape
- Controller -> /controllers/rss
- Remainder -> /services/rss
- Moved tests to match & updated requires
2017-11-08 08:09:44 +00:00
David Wolfe
f44a495cfc Added rescheduleOnBoot option for post scheduling (#9196)
no issue

- allows custom scheduling adapters with persistent data to not reschedule posts when Ghost is restarted
2017-11-08 00:24:34 +01:00
Hannah Wolfe
60fd98679f
Split RSS module into controller + other (#9224)
refs #5091, refs #9192

- This is similar to #9218, in that I'm revealing bits of code that are "controllers" in our codebase. As opposed to routes, services, renderers etc.
- This also reveals some code which is identical to the channels controller
- There is more to do here, but for now I've got the module split up, and the tests split and improved.
- Next I'll split RSS into controller + service, DRY up the controller code, etc
2017-11-07 20:00:03 +00:00
rambii
341948966d 🐛 Fixed custom_excerpt not being used in RSS feeds (#9219)
closes #9210

* use `custom_excerpt` before `meta_description` in rss
2017-11-07 10:16:14 +00:00
Aileen Nowak
d507eab3e8 Changed logic for importPersistUser option (#9203)
no issue

- `importing` and `importPersistUser` are two different concepts
2017-11-07 09:09:57 +01:00
Aileen Nowak
dbd22d7447 Feature: {{reading_time}} theme helper (#9217)
closes #9200

- Registered new server helper `{{reading_time}}`.
- Added new global util `word-count` based on the util in Ghost admin, which returns the number of words in an HTML string.
- Based on the word count of the post html, the helper calculated the estimated reading time:
   - 275 words per minute
   - additional 12 seconds when post has feature image
- Renders a string like 'x min red', unless reading time is less than a minute. In this case, the rendered string is '< 1 min read'.
2017-11-06 18:40:07 +00:00
Hannah Wolfe
5dac1c97fc
Split renderChannel into controller + renderer (#9218)
refs #5091, refs #9192

- render channel was always a weird file
- now it's clearly 2 things
- we're slowly getting towards closing #5091... 🎉
- added some extra tests
2017-11-06 12:17:24 +00:00
Hannah Wolfe
4c5ef16bc3
Represent channels as class instances (#9209)
refs #9192, refs #5091

- Using a class allows for easy shared logic
- Loading is designed to work from config right now, but could be DB driven, etc
- Provided configuration can be simplified and extended in the constructor / class methods
- Update tests, move custom assertions to utils
2017-11-05 10:04:59 +00:00
Hannah Wolfe
4ee522069c
Increased URL utility coverage to 100% (#9201)
refs #9192

To anyone seeing this go by - I'm about to start some fairly major refactoring work on the url utility. Before I do that, I wanted to make sure I had 100% coverage, and understanding of some of the weird cases.

The majority of the changes I've made are adding tests, but I was also able to clean up a little bit, remove a few lines or change them to make use of other tools.
2017-11-02 20:35:58 +00:00
Hannah Wolfe
329289a2b4 Classes must use strict mode 2017-11-01 15:55:06 +00:00
Hannah Wolfe
5319fd4e35 Added concept of ghost-query debug mode
- Outputting all queries is too much debug info for normal dev
- Use DEBUG=ghost:*,ghost-query to debug queries
- Or just DEBUG=ghost-query
2017-11-01 15:18:53 +00:00
Hannah Wolfe
9b54ed0689 Refactored apps to have access to a router
refs #9192

- Instead of `setupRoutes` function in apps that gets passed a router, there is now a registerRouter function as part of the proxy
- Moved towards a route service, which will know about all routes
- Using classes to abstract away shared behaviour

Notes:

- changing the app proxy didn't result in a test failure!
- structure of route service is totally new and may change a lot yet
2017-11-01 15:02:25 +00:00
Hannah Wolfe
bcf5a1bc34
Switch to Eslint (#9197)
refs #9178

* Add eslint deps, remove old lint deps
* Add eslint config, remove old lint configs
* Config for server and tests are different
* Tweaked rules to suit us
* Fix linting in codebase - lots of indent changes.
* Fix a real broken test
2017-11-01 13:44:54 +00:00
Hannah Wolfe
bbf59fc6c1
Refactored to save settings only if value changes (#9194)
refs #9192

- Each setting is saved individually
- Update this to only happen on import, or when a value changes
- Reduces the amount of work Ghost does on every setting change
2017-10-31 15:47:30 +00:00
kirrg001
9d4ed8c61c Replaced extract-zip-fork by extract-zip
no issue

- we had to fork the original repository at one point, because of slow maintenance
- maintenance is back now
- https://github.com/maxogden/extract-zip/pull/52 was merged and released
2017-10-31 12:11:59 +01:00
Hannah Wolfe
882a2361ee
Moved apps to /services/ & moved individual tests (#9187)
refs #9178

* Moved app handling code into services/apps
  - Apps is a service, that allows for the App lifecycle 
  - /server/apps = contains internal apps 
   - /server/services/apps = contains code for managing/handling app life cycle, providing the proxy, etc
* Split apps service tests into separate files
* Moved internal app tests into test folders
    - Problem: Not all the tests in apps were unit tests, yet they were treated like they were in Gruntfile.js
    - Unit tests now live in /test/unit/apps
    - Route tests now live in /test/functional/routes/apps
    - Gruntfile.js has been updated to match
* Switch api.read usage for settingsCache
* Add tests to cover the basic App lifecycle
* Simplify some of the init logic
2017-10-30 12:31:04 +00:00
Hannah Wolfe
b5228f5c43 Rename blog -> site
- We're going to be moving towards this naming convention more
- Doing this now makes it easier to name other things
2017-10-26 18:09:56 +01:00
Hannah Wolfe
b8a8360219 Use t_ for translations in channel routes
refs #5091

- rkw is something I made up on the spot
- t_ looks like a translation function, which is what this IS!
2017-10-26 13:04:01 +01:00
Katharina Irrgang
34054a32c0 🐛 Theme name is point number (#9184)
closes #9182

- e.g. "1.4"
- extend settings cache to ensure we return strings for numbers and floating point numbers
2017-10-26 13:54:18 +02:00
Katharina Irrgang
88eab9898c Moved fetching client out of our ghost_head helper (#9180)
refs #8995

- move the getClient lookup from ghost_head into middleware
- use res.locals to keep track of the information (res.locals.client)
- make the middleware global to all frontend routes
- ghost_head: get locals from options.data not this (!)
- adapt lot's of tests
2017-10-26 12:03:53 +02:00
Aileen Nowak
c8cbbc4eb6 Improved password validation rules (#9171)
refs #9150 

- Moves the password length fn from `models/user` to `data/validation` where the other validator functions live.
- Added password validation rules. Password rules added:
   - Disallow obviously bad passwords: '1234567890', 'qwertyuiop', 'asdfghjkl;' and 'asdfghjklm' for example
   - Disallow passwords that contain the words 'password' or 'ghost'
   - Disallow passwords that match the user's email address
   - Disallow passwords that match the blog domain or blog title
   - Disallow passwords that include 50% or more of the same characters: 'aaaaaaaaaa', '1111111111' and 'ababababab' for example.
- Password validation returns an `Object` now, that includes an `isValid` and `message` property to differentiate between the two error messages (password too short or password insecure).
- Use a catch predicate in `api/authentication` on `passwordReset`, so the correct `ValidationError` will be thrown during the password reset flow rather then an `UnauthorizedError`.
- When in setup flow, the blog title is not available yet from `settingsCache`. We therefore supply it from the received form data in the user model `setup` method to have it accessible for the validation.
2017-10-26 11:01:24 +01:00
Hannah Wolfe
05729d2f29 Group channel-handling code together
refs #5091

- Move all of the code to do with handling channels into one folder
- Still keeping all the shared/simlar code for rendering etc inside weird
  frontend folder until I am sure what this will look like
2017-10-25 18:48:47 +01:00
Hannah Wolfe
e659766f55 Move xmlrpc & slack to services (#9179)
refs #9178

- Introduce the /services/ folder
- Move xmlrpc there
- Move slack there
- In slack: remove a usage of the settings API that should use settingsCache
- In slack: Simplify the tests 
- Various tiny changes to move towards code consistency
2017-10-25 15:27:56 +01:00
Hannah Wolfe
050f1751c4 Simplify config for channels (#9158)
refs #5091

- remove the use of functions
- remove unnecessary quotes from tag filter
- move channel config to be a JSOn file called config.channels.json
- accept external config
- new channelUtils for tests
- remove channelConfig.get 
- refactor so tests work as expected
- refactor away duplicate 'name' value
2017-10-24 17:18:35 +01:00
Hannah Wolfe
ac3feb96d6 🐛 Prevents xmlrpc pings happening on import (#9165)
closes #9164

- check options.importing on xmlrpc
- also don't ping if private
- cleanup slack to work the same way
- update tests
- TODO: we need to prevent this event happening altogether
2017-10-23 18:30:33 +01:00
Hannah Wolfe
c20a6aa7f7 Improve channels router code (#9166)
refs #5091

- There is very little that changes here, just code readability
- However I've expanded out the tests getting ready to be able to test more deeply as I refactor the routing
2017-10-23 11:42:48 +01:00
Aileen Nowak
d4b6390fd6 Improved importer logic for password in users (#9161)
refs #9150

- move data manipulation for importing users from `importers/data/users` to `model/user` for more consistency (see behaviour of post imports)
- changed importing logic in `onSaving` fn for user model:
   - when importing, we set the password to a random uid and don't validate, just hash it and lock the user
   - when importing with `importPersistUser` we check if the password is a bcrypt hash already and fall back to normal behaviour if not (set random password, lock user, and hash password)
   - don't run validations when importing
2017-10-19 10:43:52 +01:00
aimingoo
958544ae90 support prev/next filtering by author (#9149)
refs #9141

- adds support for `{{#prev_post in="author"}}{{/prev_post}}` & `{{#next_post in="author"}}{{/next_post}}` 
- "author.slug" is the author equivalent of "primary_tag.slug" - there is only one
- added tests to cover both cases in the prev/next helpers
2017-10-18 18:12:20 +01:00
Aileen Nowak
7d388cb9e1 🐛 Fixed internal links converting to absolute URLs (#9143)
closes #9136

Changed the functionality in `make-absolute-urls.js` util to not convert the URL when starting which an `#`, as it indicates and internal link.
The util is used inside of the `{{amp_content}}` helper and to render the RSS feed. I tested the changes with the most popular RSS reader 'Feedly' and it seems like these internal links get converted to absolute URL inside of Feedly automatically.
2017-10-18 17:54:17 +01:00
Aileen Nowak
0ed92959c8 Increase minimum password length to 10 characters (#9152)
refs #9150

- Sets password min length in validator to 10
- Updates tests
2017-10-18 17:45:41 +01:00
Hannah Wolfe
1c382792ef Prev & next post filtering, with primary tag support (#9141)
closes #9140
* Rip out existing prev/next implementation
* New implementation using filter
* Support next/prev in primary_tag
2017-10-13 15:44:39 +01:00
Katharina Irrgang
8de691d575 🐛 HTML page error: correct templateData (#9144)
no issue

- `this.statusCode` was always undefined
- see HTML error handler
- it's hard to add a test for this case
- manual test only
2017-10-13 14:26:42 +01:00
Hannah Wolfe
9f9d8b2fec Always use our redirects utility (#9132)
no issue

- This started as an attempt to simplify the admin redirect code
- I realised we were sometimes using utils.redirect301 and sometimes not
- Decided to move this into utils.url as it's more relevant to URL generation
- Unified usage of redirects in the codebase
- Updated tests & ensured we have basic coverage
- rename adminRedirect -> redirectToAdmin
- Tweak method signature, fix channel edit redirects
- Tests: Optimised test descriptions for url-redirects_spec.js
- ensure caching works as expected
2017-10-12 14:36:50 +02:00
Kevin Ansfield
9699016cca 🎨 Improved accessibility markup in default navigation.hbs partial (#9137)
refs #9135

- remove `role="presentation"` attribute from list items
- add explicit `menu` and `menuitem` roles to improve screen ready accessibility
2017-10-12 13:57:18 +02:00
Hannah Wolfe
c25c5e2395 🐛 Fixed showing "theme missing" error incorrectly (#9129)
closes #8222

- There are still some cases where Ghost shows "the currently active theme X is missing" when it isn't
- This is due to the error handling masking several cases
- This PR resolves that, ensuring errors from gscan and the underlying environment don't get masked
2017-10-11 15:19:12 +02:00
Katharina Irrgang
594b0c2d14 Custom post templates (#9073)
closes #9060

- Update `gscan` - it now extracts custom templates and exposes them to Ghost
- Add `custom_template` field to post schema w/ 1.13 migration
- Return `templates` array for the active theme in `/themes/` requests
- Users with Author/Editor roles can now request `/themes/`
- Front-end will render `custom_template` for posts if it exists, template priority is now:
  1. `post/page-{{slug}}.hbs`
  2. `{{custom_template}}.hbs`
  3. `post/page.hbs`
2017-10-10 13:36:35 +01:00
Hannah Wolfe
7999c38fa2 Support filtering based on primary_tag (#9124)
closes #8668, refs #8920

- Updated tests to include internal tags
  - Tests had no example of an internal tag
  - Need this to show that the new filtering works as expected
- primary_tag is a calculated field
- This ensures that we can alias the field to equivalent logic in API filters
- By replacing primary_tag by a lookup based on a tag which has order 0
- bump ghost-gql to 0.0.8

**NOTE:**
Until GQL is refactored, there are limitations on what else can be filtered when using primary_tag in a filter e.g. it wont be possible to do a filter based on primary_tag AND/OR other tag filters.
2017-10-10 14:07:44 +02:00
Hannah Wolfe
2941932ebe Removed hardcoded url structures from RSS (#9113)
refs #5091

- This removes hardcoded config to generate feed urls
- This means that RSS feeds work properly for custom channels
- Remaining assumption is that paginated feeds will end /pageNum/
- Added extra tests
- Don't pass through query params
2017-10-09 16:08:53 +02:00
Katharina Irrgang
8c54f3cca9 Added request id header (#9122)
no issue

- be able to track a request through different systems
- case insensitive
- manual test only
2017-10-09 14:48:32 +02:00
Hannah Wolfe
d77e000113 Splitted preview & single controllers from index (#9121)
refs #5091

- one step towards having a less confusing controller/routing system for blogs
2017-10-09 13:56:44 +02:00
Hannah Wolfe
75fdcd168e Added support for custom channel contexts (#9114)
refs #5091

- This simple change allows custom contexts to use existing channel logic
- E.g. if we want to create a custom tag-based channel, it can pass "tag" as the context, and get all the same metadata logic
2017-10-09 11:59:46 +02:00
Hannah Wolfe
6760ccc8ec Permissions: minor refactors (#9104)
refs #9043

- Cleanups / refactors to make the code more manageable
- Move remaining code out of index.js 
   - Only "init" function is left. Actions map cache and init function is based heavily on the settings cache module
- refactor the odd way of exporting
   - This was cleaned up naturally by moving the actionsMap object out
- rename "effective" -> "providers"
  - "Providers" provide permissions for different things that can have permissions (users, apps, in future clients).
2017-10-05 21:01:34 +02:00
Aileen Nowak
19167c1af2 🐛 Fixed author helper not returning the correct url (#9102)
closes #9101

With 506a0c3e9e we don't expose the `status` field for author context anymore, which is used to determine the correct URL for the `{{url}}` helper in https://github.com/TryGhost/Ghost/blob/master/core/server/data/schema/checks.js#L13

This fix uses the field `profile_image` instead and adds a missing test for author context to the `{{url}}` helper test.
2017-10-05 13:50:55 +02:00
Katharina Irrgang
30e790bf12 Debug: Update Collision (#9103)
refs #8969

- we would like to figure out how often people get the error and with which context
2017-10-05 12:24:21 +01:00
Katharina Irrgang
7800ed3d8b Private RSS feed (#9088)
refs #9001

When a blog is in private mode there is now an unguessable URL that allows access to the RSS feed for internal use, commenting systems, etc.

- add public hash for private blogging
  - auto generate on bootstrap if missing
  - global hash, we can re-use in the future
- update private blogging middleware to detect the private RSS URL and rewrite it so that the normal rss route/code is used for display
- if a normal `/rss/` route is accessed with a private session return a 404
2017-10-05 11:07:32 +01:00
Katharina Irrgang
a36ca9c14e Debug: Measure AMP parse (#9097)
no issue
2017-10-04 10:32:00 +01:00
kirrg001
78ac98ee27 Debug: LocalStorage response time
no issue
2017-10-04 10:05:35 +01:00
kirrg001
41c9db30b5 Debug: MySQL logging hook
no issue
2017-10-04 10:05:35 +01:00
Katharina Irrgang
73568ba55a Image Size Utility: optimised unknown request error handling (#9094)
no issue

- we want to know and learn from the full original error
2017-10-04 10:03:26 +01:00
Katharina Irrgang
3d6227e11b Brute store error is an internal server error (#9092)
no issue
- If the brute store throws an error and the `handleStoreError` is called, then the storage is unable to get/set values.
- This is not a PermissionError. The result is that the user has no access, because the brute store has problems reading/writing to the storage.
2017-10-04 10:02:22 +01:00
Katharina Irrgang
c4b2e8035d Debug: Add post.url if amp parse error happens (#9091)
no issue
- if AMP parse failed, we want to figure out which url is/was affected
2017-10-04 10:01:00 +01:00
Katharina Irrgang
5f44972d44 🐛 Fixed being able to store invalid date formats (#9090)
closes #9089
- use the current date any time a post is fetched if the database contains an invalid date
- raise an error any time an attempt is made to save an invalidate date via the API
2017-10-04 09:56:09 +01:00
Katharina Irrgang
15446766bf Protected internal tags visibility (#9076)
closes https://github.com/TryGhost/Ghost/issues/8943

- if you send a tag name with a hash, it's an internal tag
- ensure that the visibility property is forced to `internal`
- add a proper test
2017-10-03 13:00:33 +01:00
Katharina Irrgang
66f78af6a1 Refactored private blogging app: use settings cache (#9086)
no issue

- preparation for #9001
- no need to require the settings API, we can simply fetch the data from the settings cache
- the settings API uses the settings cache anyway
2017-10-03 12:40:53 +01:00
Kevin Ansfield
b82932b492 🎨 Disabled auto-linking of domain names in markdown (#9084)
closes https://github.com/TryGhost/Ghost/issues/8987

- set `linkify-it` `fuzzyLink` option to false so that it only auto-links URLs starting with `http(s)://` or other valid schemes
2017-10-03 12:14:53 +02:00
Vikas
b7e82f3b00 Improved the usage of missing translation keys (#9081)
closes #9079

- log error if missing key was used
- send client generic error message (they can look at logs to get a
better understanding)
2017-10-03 09:47:48 +02:00
Katharina Irrgang
506a0c3e9e 🔥 Removed certain fields from public user response (#9069)
no issue 

* Comment current state of toJSON for user model

- currently the user model does not return the email if the context is app/external/public OR if there is no context object at all
- i am not 100% sure why if there is no context we should not return the email address
- i think no context means internal access
- maybe change this condition cc @ErisDS

* Extend our access rules plugin

- we already have a instance method to determine which context is used
- this relies on passing options into `.forge` - but we almost never pass the context into the forge call
  - added @TODO
- provide another static method to determine the context based on the options object passed from outside

* Use the new static function for existing code

* Add comment where the external context is used

* Remove certain fields from a public request (User model only)

* Tests: support `checkResponse` for a public request

- start with an optional option pattern
- i would love to get rid of checkResponse('user', null, null, null)
- still support old style for now
- a resoure can define the default response fields and public response fields

* Tests: adapt public api test

* Tests: adapt api user test

- use new option pattern for `checkResponse`
- eww null, null, null, null....

* Revert the usage of the access rules plugin
2017-09-28 14:00:52 +01:00
Katharina Irrgang
42af268d1b 🎨 User is not allowed to add/modify certain fields (#9053)
no issue

- it's not allowed to change/add these attributes via the API
  - created_at = is only once set on adding the resource
  - created_by = is only once set on adding the resource
  - updated_by = is set on the server side when updating the model (based on who is logged in)
  - updated_at = is set on the server side when updating the model

* Revert the usage of the access rules plugin
2017-09-28 13:59:42 +01:00
Katharina Irrgang
d3d04a8e72 Fixed wrong handling of formats param (#9078)
closes #9077

- because of our API layer refactoring, see https://github.com/TryGhost/Ghost/pull/9068
- we can now see that code was written wrong because of this horrible API bug
- this fixes the formats parameter for querying a single post
2017-09-28 13:38:32 +01:00
kirrg001
e347163940 Removed bypassing option filtering in User model
no issue

- the logic here bypasses filtering options!
- that is wrong, because if we filter out certain options e.g. include
- the tests from the previous commit fail because of this
- if we don't fix this logic, the tests won't pass, because as said, you can bypass certain logic e.g. remove roles from include
- this has worked before, because we passed the wrong options via the API layer
- was introduced here 014e2c88dd, because of https://github.com/TryGhost/Ghost/pull/6122
- add proper tests to proof that these queries work!!
2017-09-28 10:18:18 +01:00
kirrg001
1e2beface1 Refactored the API layer: do not handle API response after pipelining
no issue

- this has a big underlying problem
- each task in the pipeline can modify the options
- e.g. add a proper permission context
- if we chain after the pipeline, we don't have access to the modified options object
- and then we pass the wrong options into the `toJSON` function of a model
- the toJSON function decides what to return based on options
- this is the easiest solution for now, but i am going to write a spec if we can solve this problem differently
2017-09-28 10:18:18 +01:00
Katharina Irrgang
a6d57d6324 🐛 Fixed missing cache invalidation header when uploading redirects (#9071)
closes #9065
2017-09-27 18:58:33 +01:00
Hannah Wolfe
fcd3c6847b 🐛 Fixed author role permission to change author (#9067)
🐛  Fixed author role permission to change author

no issue

- To be able to fix this bug, we had to solve tasks from #9043
- This bug affects the private / undocumented API only
- Author role users should not be allowed to change the author of a post
2017-09-27 13:12:53 +02:00
Katharina Irrgang
baf8116d6b Removed OAuth leftover: emit event when calling /users/me (#9061)
refs #8342

- was added in this commit 319a388277
- we can remove this event
2017-09-27 04:07:39 +01:00
Katharina Irrgang
3002747b68 Return dates from the database without milliseconds (#9054)
no issue

- we store dates without milliseconds in the database
- our test environment does not use our model layer to insert data, this is related to  https://github.com/TryGhost/Ghost/issues/7196
- so it can happen that the test env inserts unix timestamps instead of a formatted string
- e.g. adding data via the model layer (e.g. via the API) the format is always normalised to `YYYY-MM-DD HH:mm:ss`
- if we fetch the date from the database, we have a hook which sorts out knex returning different formats for dates
- this hook wraps the returned date into a UTC moment date, but adds the current milliseconds on top
- which can collide in tests when you have specific assertions
- use `startOf` to ignore milliseconds
- furthermore: remove the mentionings of `pg` (postgres)
2017-09-26 17:16:46 +01:00
Hannah Wolfe
b468d6dbe2 Support for attribute-based permissions (#9025)
refs #8602

- Add the wiring to pass attributes around the permission system
- Allows us to get access to the important "unsafe" attributes that are changing
- E.g. status for posts
- This can then be used to determine whether a user has permission to perform an attribute-based action
- E.g. publish a post (change status)
2017-09-26 18:06:14 +02:00
Katharina Irrgang
a80a09e483 🔥 Removed public API endpoint to fetch users by email address (#9059)
no issue

- our public API is still a beta/labs feature
- from api.ghost.org
  > The API is still under very (very) heavy development and subject to regular breaking changes.
- users should expect breaking changes in any release (independent from semver versions)
- the public user API never returns any email addresses to decrease the information we expose
- there is no need to keep the support fetching a user by email address
2017-09-26 16:42:58 +01:00
Katharina Irrgang
af01f51204 🐛 Fixed returning roles for the public user resource (#9039)
no issue

- this bug fix affects all endpoints for the public user access
- we allowed fetching `roles` via the public api by accident
- see our docs: https://api.ghost.org/docs/users)
  - we only allow `count.posts`
- returning roles via the public api exposes too many details
- this was never intentional
2017-09-26 15:43:21 +01:00
Katharina Irrgang
e921c7a044 Revert "🐛 Fixed returning roles for the public user resource (#9039)" (#9062)
This reverts commit 217bc6914d.

- NOTE: will be released in the next minor release
2017-09-26 14:28:34 +01:00
Katharina Irrgang
22017b8ede 🎨 Backup redirects.json file before overriding (#9051)
refs #9028

- if you upload a redirects file and a redirects file exists already, we backup this file to `data/redirects-YYYY-MM-DD-HH-mm-ss.json`
- decrease chance of random test failures by not comparing date format with seconds
2017-09-25 18:35:57 +01:00
Katharina Irrgang
472858f262 Removed unused API endpoint: POST /users (#9052)
no issue

- this endpoint does not exist anymore
- if you want to add a new user, you have to invite him via the invites API
- on invite accept, the user is inserted
2017-09-25 16:58:14 +01:00
Kevin Ansfield
ec6e25674c Removed markdown-it-named-headers and unused string deps (#8994)
refs https://github.com/TryGhost/Ghost-Admin/pull/856

- moves `markdown-it-named-headers` functionality into our own app code without requiring the [`string.js`](http://stringjs.com) sub-dependency
- matches Ghost-Admin markdown-it code
2017-09-25 16:36:34 +02:00
Katharina Irrgang
1dd365778f 🐛 Fixed persistent upgrade notifications showing for the currently installed version (#9048)
closes #9040

- introduced by https://github.com/TryGhost/Ghost/pull/9009
- a condition was missing, was removed by mistake
2017-09-25 12:22:56 +01:00
Katharina Irrgang
217bc6914d 🐛 Fixed returning roles for the public user resource (#9039)
no issue

- this bug fix affects all endpoints for the public user access
- we allowed fetching `roles` via the public api by accident
- see our docs: https://api.ghost.org/docs/users)
  - we only allow `count.posts`
- returning roles via the public api exposes too many details
- this was never attentional
2017-09-25 11:18:23 +01:00
Hannah Wolfe
9da7b956d5 Permissions: code cleanup & basic unit tests (#9037)
refs #9043

- Split public-related and context code into logical components
- Split tests up to match
- Ensure we have 100% unit test coverage
- General cleanup
2017-09-25 11:17:06 +02:00
Vikas
e8f85fd3da 🐛 Fixed redirects upload for various browsers/systems (#9042)
closes #9036
2017-09-22 21:57:43 +02:00
Katharina Irrgang
d943fc7cc9 Allow Upload/Download of redirects.json (#9029)
refs #9028

- add two new endpoints for uploading/downloading the redirects (file based)
- reload/re-register redirects on runtime
- migration for 1.9 to add permissions for redirects download/upload
2017-09-21 16:01:03 +01:00
kirrg001
f478e4f9c8 🎨 Enabled Unsplash by default
refs https://github.com/TryGhost/Ghost/issues/8859

There are four cases:

- unsplash setting is empty (default), admin can enable the app by default (hardcoded isActive:true)
- unsplash settings are set, unsplash is disabled, admin detects that app was disabled on purpose
- unsplash setting is set, unsplash is enabled and has a key, app is enabled, old key get's ignored and overridden on the next save
- unsplash setting is set, unsplash is enabled and has no key, app is enabled
2017-09-20 11:44:47 +01:00
kirrg001
79fead5516 Removed private configuration endpoint
refs https://github.com/TryGhost/Ghost/issues/8859

- We don't need the config option for Unsplash anymore
- The private endpoint (/configuration/private) was introduced for Unsplash
2017-09-20 11:44:47 +01:00
Katharina Irrgang
edf2348394 Improved log output for welcome email error (#9016)
* Improved log output for welcome email error

no issue

- if Ghost is unable to send a welcome email, the server log printe a huge error log
- the reason was that each component wrapped the original error into a new error instance
  - so the stack grows and grows
- the golden rule should always be: the smallest/lowest component should instanitate a specifc error
  - the caller can expect to receive a custom Ghost error

* Tidy up error messages for mail failures and fix tests

- We never use "Error:" notation in our translations
- Make the error messages consistent and show a reason if possible
2017-09-19 14:24:20 +01:00
Aileen Nowak
0ce24b48bd Moved published_at creation to fixtures/utils (#8595)
no issue

- follow-up from #8573
- bove the hack that creates published_at values from the migration fn to our fixture util
2017-09-19 12:54:01 +02:00
Katharina Irrgang
4ac34a7f33 🐛 Fixed api url for the ghost sdk (#9013)
no issue

- mirror LTS behaviour to master
- if your blog or admin url is configured to http, it's still possible that e.g. nginx allows both https/http
- that's why we should generate the api url without protocol in this case
- so it depends how you serve your blog, example:
  - blog url is http://example.com
  - generated api url for the sdk is //example.com (dynamic protocol allowed)
  - you serve your blog via https://example.com, protocol is https
  - you serve your blog via http://example.com, protocol is http
2017-09-18 16:28:22 +01:00
Katharina Irrgang
abb84d065e Improved debug logs for auth utils (#9015)
no issue

- reduce the debug logs
- it's okay to log the old token to delete, because this token is getting removed anyway
2017-09-18 16:23:47 +01:00
Katharina Irrgang
7b4c3fc085 Removed defunct Ghost OAuth code (#9014)
closes #8342

- no need to add a migration, because when we'released 1.0, OAuth was never an option
- it was disabled in April, 1.0-beta was released in June
- remove all remote authentication code
2017-09-18 13:01:58 +01:00
kirrg001
18abb425fc 🐛 Fixed doubled query params for url/admin-url redirection
no issue

- express adds the query parameters to the `originalUrl`
- we have to ensure that we don't add the query params twice
2017-09-14 07:55:14 +07:00
kirrg001
79959d9581 🐛 Fixed public api access on custom domain
no issue

- if you blog runs on a custom domain, but your admin panel is configured using a different domain
  -> Ghost losts the origin header
- we had this situation once with pretty urls (your request get's redirected from /posts to /posts/, see https://github.com/TryGhost/Ghost/pull/8094)
- we've moved all our redirect logic to Ghost and ran into the same situation
- i've added proper test to ensure it won't happen again
2017-09-14 07:55:14 +07:00
Katharina Irrgang
85f8498bd6 🎨 Changed showing update notifications for minor/major only (#9009)
closes #9006

- this is a temporary fix to only show update notifications for minor/major releases
- the notification refactoring is in the pipeline, but not yet merged into 1.X/LTS, see https://github.com/TryGhost/Ghost/pull/8871
2017-09-13 18:12:41 +07:00
Hannah Wolfe
4237446277 Misc cleanup & consistency amends (#9002)
no issue

- Consistent naming for postLookup
   - makes it easier to search and inspect the various usages
- Cleanup unneeded code
- Make res.render calls more consistent
- add some consistency to the calls to res.render
- Remove ancient reference to dataProvider
- Let's call it models everywhere now...
- Use consistent formatting across the API
- we're no longer using alignment in vars
- Misc other consistency changes in API
- always refer to local utils as apiUtils
- logical grouping of requires - dependencies, utils, "lib common" etc
- use xAPI to refer to API endpoints, e.g. mailAPI, settingsAPI for clarity
2017-09-12 17:31:14 +02:00
kirrg001
c99557d9a3 🐛 Fixed disqus comment id when exporting/importing 1.x content
no issue

- while i was testing random failures, i discovered an edge case for disqus
- you start a new 1.0 blog, you add disqus, the unique identifer is the post id (object id)
- now you export your content and import it on a new instance
- the importer detects that the amp field is null and imports the old object id as comment id
- but the post model is not prepared for this case
- see next commit for tests

**NOTE**: The comment id had two different data types (Number or String). Disqus expects a string. So this should not change any behaviour, now that the comment_id is always a string.
2017-09-12 16:29:59 +01:00
Aileen Nowak
a45a91c906 🐛 Fix invalid image URLs not being cached and causing timeouts (#8986)
refs #8868

* 📐  Use request util in image-size
- swapped the usage of `got` for requests with the request util

* 💄  Use catch predicates
- Uses catch predicates instead of conditionals in `getImageSizeFromUrl`
- Return `NotFoundError` if applicable in `getImageSizeFromFilePath` as the caller function `cachedImageSizeFromUrl` is differentiating those between this error and others.

* 🐛  Fixed ImageObject URL & simplify no protocol URL logic

- Using `ImageObject` as a global var resulted in having the `url` property being the same for all requests coming in.
- The logic that checked for an existing protocol (e. g. gravatar URLs) was overly complicated. Refactored it to be more simple.
- Passing the correct value to `fetchDimensionsFromBuffer` as the population of `imageObject.url` happens there. These are used in our structured data and need to be full URLs (in case of locally stored files) or the original URL (in case of URLs missing the protocol)
- Added two more debug logs in `getCachedImageSizeFromUrl` so it's logged when an image is added to the cache even tho it was returned as error.

* 👀  Differentiate error codes between request and storage

* 🔥  Remove not needed `Promise.resolve()`

We're always resolving the result in `getCachedImageSizeFromUrl`, so there's no need to return the values with a `Promise.resolve()`. The caller fn uses waits for the Promises to be fulfilled.

* ☂️  Wrap already rejected predicate errors in catch all

* Use errorDetails instead of context

* ☂️  Support /assets/ image paths

- adds a guard that checks the image URL for `/assets/` in the beginning and passes a completed URL to the request util to try and fetch the image size
- adds tests
2017-09-12 12:53:18 +01:00
Katharina Irrgang
add9e541c2 🐛 Fixed private blogging leaking post information (#8999)
* 🐛  Fixed private blogging leaking post information

closes #8990

- a condition in the private blogging app redirected rss && sitemap to 404, which can possibly leak content
- remove this condition and ensure we always redirect to /private

* lint 😋
2017-09-11 14:09:19 +01:00
Katharina Irrgang
7e211a307c 🐛 Fixed custom redirects with query/search params (#8998)
closes #8997

- improved the logic for custom redirects
- added more tests
2017-09-11 13:20:29 +01:00
Hannah Wolfe
45fd2d437f Properly return a bluebird promise (#8988)
refs #8980

- ☹️ apparently this is actually the only way
2017-09-07 20:48:20 +07:00
Aileen Nowak
6c216b81be Request util to wrap got library (#8980)
no issue

This PR includes a new util which wraps the `got` library. It is not used in the codebase yet, but tested with `image-size` util:
- wraps `got` request library in its own `request.js` util that returns bluebird promises and validates URL before starting a request
- adds tests
2017-09-07 12:17:24 +01:00
Hannah Wolfe
cdf6a10490 ghost_head improvements (#8983)
no issue

- Added debug statements to ghost_head
  - useful for determining how much render time is spent in ghost head
- Make promises more readable
- Used join instead of props for less code
2017-09-07 12:59:02 +02:00
Aileen Nowak
56d64e53bf Logging for image size (#8978)
no issue

- added debug logs to image size util and related fn:
    - when fetched via network request
    - when fetched from storage
    - when added to cache
    - when read from cache
2017-09-07 10:34:10 +01:00
Hannah Wolfe
6c47285bba Added error handling for ghost_head (#8982)
refs #8945

- Ensure that errors in ghost_head are logged
- Render some content despite the error!
2017-09-07 09:29:44 +02:00
Katharina Irrgang
d460cf1291 🐛 Fixed post scheduling (#8976)
closes #8975

- recursive logic was broken
- caused via bf47397ac2
2017-09-05 19:23:11 +01:00
Hannah Wolfe
69657a1ac6 Keep the original error in registerAsyncTheme (#8977)
refs #8945

- pass the original error as part of the errorDetails
- improves logging, so we know what really went wrong
- for debug purposes - can be removed at some point
2017-09-05 20:11:59 +02:00
Katharina Irrgang
71cee504c2 Added default logging rotation config (#8974)
no issue

- define `period` and `count` in our defaults.json config
- advantage: easier access and better overview how logging is configured by default
- period is 1d and count is 10
2017-09-05 17:25:20 +02:00
Katharina Irrgang
1fe87a6110 Tests: Stub image request for dimensions (#8973)
no issue
- test cases were trying to fetch image sizes for `localhost:port/favicon.ico` but no server is running so they time out
- stub the `getImageSizeFromUrl` method so it resolves instantly
2017-09-05 16:24:04 +01:00
Aileen Nowak
eef7932e94 Refactor: fetch image dimensions from local file storage (#8900)
refs #8868

- Removed image-size in blog logo fn for meta data and made it synchronous
- Renamed `image-size-from-url.js` to `image-size.js` (incl. the test)
- Added second fn `getImageSizeFromFilePath` that reads from local file storage
- Added guard in `getImageSizeFromUrl` that checks if the image should be on local file storage and uses the new fn then instead
- Added a fn `fetchDimensionsFromBuffer` that takes the file buffer and returns an `imageObject` with dimensions.
- Added a new utils.js in `adapters/storage` for getting the file storage path
2017-09-05 14:13:22 +02:00
Austin Burdine
a9e668a949 🐛 Fixed only save imported post ids if amp field is empty (#8967)
closes #8963

- if an LTS export is imported into a 1.0 blog, then the 1.0 blog is
exported and re-imported into another 1.0 blog, any post ids from the
lts import were getting clobbered. This only saves the post id if the
amp field does not already exist
- add failing test that passes w/change
2017-09-04 09:48:56 +02:00
Kevin Ansfield
47322e4239 Re-instate mobiledoc dom rendering with bypass of SimpleDOM parsing (#8937)
closes #8757

- update the markdown card render method to use SimpleDOM's `createRawHtmlSection`. This avoids SimpleDOM parsing and tokenization of broken or unsupported free-form HTML that markdown allows
- replace markdown extraction/render with mobiledoc's renderer in the `Post` model
- removes `jsdom` as it's no longer necessary
2017-08-31 12:09:02 +02:00
Vikas
8606ea22a2 Normalize docs URLs (#8949)
no issue

v1.0.0 is no longer the standard in the docs, so I updated all of the URLs containing it with v1
Note: I tried squashing commits, but failed. I'll try again in the future with throwaway changes
Secondary Note: I tested most of the URLs listed and got no 404s!
2017-08-31 11:45:59 +02:00