Commit Graph

1074 Commits

Author SHA1 Message Date
Katharina Irrgang
5b77f052d9
Update Notification improvements (#9123)
closes #5071

- Remove hardcoded notification in admin controller
  - NOTE: update check notifications are no longer blocking the admin rendering
  - this is one of the most import changes
  - we remove the hardcoded release message
  - we also remove adding a notification manually in here, because this will work differently from now on
    -> you receive a notification (release or custom) in the update check module and this module adds the notification as is to our database

- Change default core settings keys
  - remove displayUpdateNotification
    -> this was used to store the release version number send from the UCS
    -> based on this value, Ghost creates a notification container with self defined values
    -> not needed anymore

- rename seenNotifications to notifications
  -> the new notifications key will hold both
     1. the notification from the USC
     2. the information about if a notification was seen or not
  - this key hold only one release notification
  - and n custom notifications

- Update Check Module: Request to the USC depends on the privacy configuration
  - useUpdateCheck: true -> does a checkin in the USC (exposes data)
  - useUpdateCheck: false -> does only a GET query to the USC (does not expose any data)
  - make the request handling dynamic, so it depends on the flag
  - add an extra logic to be able to define a custom USC endpoint (helpful for testing)
  - add an extra logic to be able to force the request to the service (helpful for testing)

- Update check module: re-work condition when a check should happen
  - only if the env is not correct
  - remove deprecated config.updateCheck
  - remove isPrivacyDisabled check (handled differently now, explained in last commit)

- Update check module: remove `showUpdateNotification` and readability
  - showUpdateNotification was used in the admin controller to fetch the latest release version number from the db
  - no need to check against semver in general, the USC takes care of that (no need to double check)
  - improve readability of `nextUpdateCheck` condition

- Update check module: refactor `updateCheckResponse`
  - remove db call to displayUpdateNotification, not used anymore
  - support receiving multiple custom notifications
  - support custom notification groups
  - the default group is `all` - this will always be consumed
  - groups can be extended via config e.g. `notificationGroups: ['migration']`

- Update check module: refactor createCustomNotification helper
  - get rid of taking over notification duplication handling (this is not the task of the update check module)
  - ensure we have good fallback values for non present attributes in a notification
  - get rid of semver check (happens in the USC) - could be reconsidered later if LTS is gone

- Refactor notification API
  - reason: get rid of in process notification store
    -> this was an object hold in process
    -> everything get's lost after restart
    -> not helpful anymore, because imagine the following case
      -> you get a notification
      -> you store it in process
      -> you mark this notification as seen
      -> you restart Ghost, you will receive the same notification on the next check again
      -> because we are no longer have a separate seen notifications object
  - use database settings key `notification` instead
  - refactor all api endpoints to support reading and storing into the `notifications` object
  - most important: notification deletion happens via a `seen` property (the notification get's physically deleted 3 month automatically)
    -> we have to remember a seen property, because otherwise you don't know which notification was already received/seen

- Add listener to remove seen notifications automatically after 3 month
  - i just decided for 3 month (we can decrease?)
  - at the end it doesn't really matter, as long as the windows is not tooooo short
  - listen on updates for the notifications settings
  - check if notification was seen and is older than 3 month
  - ignore release notification

- Updated our privacy document
- Updated docs.ghost.org for privacy config behaviour
- contains a migration script to remove old settings keys
2018-01-09 15:20:00 +01:00
juan-g
f671f9d2c9 Theme translations and blog localisation (#8437)
refs #5345, refs #3801

- Blog localisation
  - default is `en` (English)
  - you can change the language code in the admin panel, see https://github.com/TryGhost/Ghost-Admin/pull/703
  - blog behaviour changes depending on the language e.g. date helper format
  - theme translation get's loaded if available depending on the language setting
  - falls back to english if not available

- Theme translation
  - complete automatic translation of Ghost's frontend for site visitors (themes, etc.), to quickly deploy a site in a non-English language
  - added {{t}} and {{lang}} helper
  - no backend or admin panel translations (!)
  - easily readable translation keys - very simple translation
  - server restart required when adding new language files or changing existing files in the theme
  - no language code validation for now (will be added soon)
  - a full theme translation requires to translate Ghost core templates (e.g. subscriber form)
  - when activating a different theme, theme translations are auto re-loaded
  - when switching language of blog, theme translations are auto re-loaded

- Bump gscan to version 1.3.0 to support more known helpers

**Documentation can be found at https://themes.ghost.org/v1.20.0/docs/i18n.**
2018-01-09 14:50:57 +01:00
Aileen Nowak
19a6c8a426 Added test for images w/o extensions for image size util (#9367)
closes #9022

Images without extensions don't need to be manipulated, as we're now reading the bytes and pass those to the `image-size` lib.

This PR adds another `user-agent` to emulate multiple browser requests, as I stumbled over an example where the image without extension is protected otherwise.

Added a test, that works with above mentioned image, but is currently mocked. Nevertheless, the image worked as a PoC, that we're able to read the bytes of an image without its extension and still return the dimensions of the image.
2018-01-04 12:03:41 +01:00
Katharina Irrgang
341f719d92
Extended ghost version utility (#9278)
refs https://github.com/TryGhost/Ghost-Release/issues/24

- differentiate between
  1. original package.json version (can contain pre and build suffix)
  2. full package.json version X.X.X-{pre} (optional)
  3. safe package.json version X.X (major+minor)
2018-01-03 20:20:18 +01:00
Aileen Nowak
ae741b1a18 Removed bluebird promise wrap in request lib (#9343)
refs #9178, refs #8988

With 7353c87d7f we use Bluebird globally for Promises. Therefore, the request lib doesn't need to be wrapped in a bluebird Promise anymore.

This was originally done, so we can work with catch predicated in our image-size lib.

Updated the tests to proof, that the catch predicates work.

The tests fail, as soon as the Promise overwrite is commented out.
2018-01-02 12:18:56 +01:00
Katharina Irrgang
a5af7d6384 Used request lib in xmlrpc (#9333)
refs #9178, refs https://github.com/TryGhost/Ghost/pull/8980
2017-12-15 16:50:18 +07:00
kirrg001
991ccb1d35 Moved make-absolute-urls to url service
refs #9178

- this util uses the url services (!)
- moving this file into lib would not make sense right now
  - that would mean a module requires first ../lib/url, which then requires ../services/url
- the url service definitely need a clean up 😃
2017-12-14 22:34:37 +01:00
kirrg001
64626dedd1 Moved social utility to lib/social
refs #9178

- not 100% sure about this, but i think it makes right now the most sense
- we have already a url service and creating another lib/url is confusing at the moment
- i'll copy the last utility `makeAbsoluteUrls` to the url service for now
- see next commit for explanation (!)
2017-12-14 22:34:05 +01:00
kirrg001
1a9a10c82b Moved zip folder, read csv and package-json to lib/fs
refs #9178, refs 849e97640f

- i've reconsidered, these modules belong to lib
- prettify package-json module
2017-12-14 22:07:53 +01:00
kirrg001
82597080be Burn dirty require of api utils
refs #9178

- `checkFileExists` and `checkFileIsValid` where dirty required from web/middleware
- these two functions are only used in the target middleware
- let's move them
2017-12-14 21:25:51 +01:00
kirrg001
849e97640f Moved zip folder utility to services/themes
refs #9178

- it's only used for themes currently
- this is definitely a util which could be useful lib
- if we have a second use case, we can move it
2017-12-14 21:02:47 +01:00
kirrg001
fc5b4dd934 Moved image utils to lib/image
refs #9178

- i am not super happy about `const imageLib = require('../lib/image')`
- i don't really like the name `imageLib`
- but i had no better idea 😃
- if we use the same name in the whole project, it's very easy to rename the folder or the variable
2017-12-14 20:46:53 +01:00
Katharina Irrgang
18e15934fd Used request lib in slack service (#9335)
refs #9178

- tested with slack
2017-12-14 22:08:48 +07:00
Katharina Irrgang
50b65bca0c Moved default-cards app to lib/mobiledoc (#9341)
refs #9178, refs https://github.com/TryGhost/Ghost/pull/9338
2017-12-14 14:44:01 +00:00
kirrg001
a3091a3012 Moved utils constants to lib/constants
refs #9178
2017-12-14 14:13:40 +01:00
kirrg001
c5169e23c4 Moved unique identifier generation to lib/security
refs #9178
2017-12-14 13:52:20 +01:00
kirrg001
bb06a8426d Moved tokens, url safe and safe string utility to lib/security
refs #9178

- we could now also move any crypto usages to lib/security, but no priority
- the main goal is to tidy up our utils folder
2017-12-14 13:38:00 +01:00
Katharina Irrgang
9de13ae3ad Moved mobiledoc/markdown converters to apps/default-cards (#9338)
refs #9178

- they definitely don't belong to server/utils
- i think the best place is putting them into the card apps
- the the post model needs to ask the app for it's converters
- move tests as well
2017-12-14 11:09:54 +00:00
Katharina Irrgang
228c1c16fd
Tidy up unit test files (#9340)
refs #9178

- first iteration of tidying up the unit tests
- this is useful in the current stage, because if i move files in the server folder, i need a clean folder/file structure to detect which tests needs to move
- this is a simple cleanup to reflect the current server folder structure
2017-12-14 03:36:50 +01:00
Katharina Irrgang
192ebb1739
Moved labs, auth, permissions, settings, mail, themes to services (#9339)
refs #9178

- move tests as well
2017-12-14 03:01:23 +01:00
kirrg001
2bfff4ecc2 Decrease chance for random failure for unit/lib/request_spec.js
no issue

- this test fails sometimes, i recently increased the socket delay (maybe it was not enough)
- because these are milliseconds
- let's try 100ms
2017-12-14 02:24:14 +01:00
kirrg001
7460076206 Moved request to lib/request
refs #9178

- next steps are to
  A: test if global.Promise works with `got` (see https://github.com/TryGhost/Ghost/pull/8988#issuecomment-351532881)
  B: re-use request utility everywhere
  C: request lib requires data/validator, which is dirty
2017-12-13 22:37:42 +01:00
kirrg001
f83cbf6117 Moved pipeline/sequence to lib/promise
refs #9178

- continue with killing our global utils folder
- i haven't found any better naming for lib/promise
- so, require single files for now
- instead of doing `promiseLib = require('../lib/promise')`
- we can optimise the requires later
2017-12-13 22:20:02 +01:00
kirrg001
485c264c69 Use Promise support which was added in fs-extra 3.x
no issue

- now that we use bluebird globally, we can use the promise support from fs-extra
2017-12-13 20:57:11 +01:00
kirrg001
d83f474ff8 Always require fs-extra - consistency change
no issue
2017-12-13 20:57:11 +01:00
Katharina Irrgang
4f35f86713
Moved word-count utility to helpers/utils.js (#9328)
refs #9178

- each package/module has a local utility (e.g. api, helpers, adapters)
- these are very small utility functions which are only used from this package
- they don't belong into the global lib/utils
2017-12-13 14:05:53 +01:00
Katharina Irrgang
397400b4f8
Moved visibility utility to static model fn (#9327)
refs #9178

- this logic belongs to a static model helper
- the visibility property is a model property, the knowledge about the visibility values belongs to the model
- rename the functions, so they make more sense
2017-12-13 13:19:51 +01:00
kirrg001
6f6c8f4521 Import lib/common only
refs #9178

- avoid importing 4 modules (logging, errors, events and i18n)
- simply require common in each file
2017-12-12 10:28:13 +01:00
kirrg001
ac2578b419 Moved errors,logging,i18n and events to lib/common
refs #9178
2017-12-12 10:28:13 +01:00
kirrg001
4265afe580 Moved utils/url.js to UrlService
refs #9178

- we have to take care that we don't end up in circular dependencies
  - e.g. API requires UrlService and UrlService needs to require the API (for requesting data)
- update the references
- we would like to get rid of the utils folder, this is/was the most complicated change
2017-12-11 20:05:33 +01:00
kirrg001
6d87703c2e Decrease chance of random test failure in core/test/unit/utils/request_spec.js
no issue
2017-12-11 12:53:13 +01:00
kirrg001
bac71db996 Updated comment in core/test/unit/migration_spec.js
no issue
2017-12-11 12:35:27 +01:00
Katharina Irrgang
7bcccc71dc
Moved apps into web folder (#9308)
refs #9178

- move express apps to one place (called `web`)
- requires https://github.com/TryGhost/Ghost-Admin/pull/923
- any further improvements are not part of this PR
- this PR just moves the files and ensures the paths are up-to-date
2017-12-06 17:37:54 +01:00
kirrg001
511c26745b Decrease chance of random test failure in core/test/unit/server_utils_spec.js
no issue
2017-12-05 12:32:04 +01:00
kirrg001
ee7710ba68 Bump mocha to version 4.0.1
no issue

- https://github.com/mochajs/mocha/blob/master/CHANGELOG.md#400--2017-10-02
- the new `--exit` flag might be interesting at some point

> In Mocha v3.0.0 and newer, returning a Promise and calling done() will result in an exception.

- adapt teardown/setup test utility
- adapt other mixed usages of callback && Promise usage
2017-11-29 10:34:36 +01:00
kirrg001
c6e64fc7a8 Speed up SchedulingDefault_spec.js unit test
refs #9178

- the test used lots of bigger and unneeded timeouts
- reduce them to ensure unit tests run faster
- they now run with 845ms on my mac
2017-11-28 21:42:14 +01:00
Katharina Irrgang
b9a44bacf6
Bump should to version 13.1.3 (#9284)
refs https://github.com/TryGhost/Ghost/issues/9178

- adapt major changes
- see https://github.com/shouldjs/should.js/blob/master/History.md
- should.have.enumerable was removed
2017-11-28 19:41:16 +01:00
Katharina Irrgang
b4b5da2a75
Bump sinon to version 4.1.2 (#9283)
refs #9178

- adapt major changes
2017-11-28 18:19:23 +01:00
Katharina Irrgang
9e388aee4d 🎨 Improved error handling for images on file storage which don't exist (#9282)
refs https://github.com/TryGhost/Team/issues/41

- differentiate error codes
- return 404 if image was not found
- else return a 500
- use i18n keys
- use errors.utils.isIgnitionError (!)
2017-11-28 14:27:18 +00:00
kirrg001
e8075262eb 🎨 Improved pagination misusage error
refs https://github.com/TryGhost/Team/issues/41, refs https://github.com/TryGhost/gscan/issues/85

- if you are using the pagination helper not inside a resource context, you will receive an error
- improve error message, because it was not clear what happened
- downgrade error level to normal, because it's not a critical error from Ghost's perspective, from user perspective it is
- added help docs link and added a callout to our docs
2017-11-28 13:44:14 +00:00
Kevin Ansfield
bffb3dbd90
Webhooks support for subscriber events (#9230)
no issue

Support for http://resthooks.org style webhooks that can be used with Zapier triggers. This can currently be used in two ways:

a) adding a webhook record to the DB manually
b) using the API with password auth and POSTing to /webhooks/ (this is private API so not documented)

⚠️ only _https_ URLs are supported in the webhook `target_url` field 🚨

- add `webhooks` table to store event names and target urls
- add `POST` and `DELETE` endpoints for `/webhooks/`
- configure `subscribers.added` and `subscribers.deleted` events to trigger registered webhooks
2017-11-21 15:43:14 +00:00
John O'Nolan
0d54326121 Give default Ghost user fixture a helpful bio (#9241)
no issue

Had a couple of people ask about how to delete welcome posts easily, so adding a bio to the default user to draw a little more attention to it
2017-11-14 10:23:38 +00:00
kirrg001
0eb84d7f8a 🐛 Fixed concurrent renew of access tokens
no issue

- it can happen that concurrent requests try to renew access tokens with the same refresh token
- in this case it could happen that you received a token deletion error
- add propert locking
- ensure we don't run into deadlocks
- manual testing with async.times for parallel requests (was able to reproduce the error)
2017-11-14 10:22:09 +00:00
Hannah Wolfe
98f5ae00fc
Introduced renderer to DRY up controllers (#9235)
refs #5091, #9192

- Renderer figures out templates, contexts, and does a render call
- Templating is now handled with a single function
- Context call is made in the renderer

Note:  to make this work, all controllers now define a little bit of config, currently stored in res._route. (That's a totally temporary location, as is res._template... when a sensible naming convention reveals itself I'll get rid of the weird _). This exposes a type and for custom routes a template name & default.
2017-11-10 12:44:29 +00:00
Hannah Wolfe
e41d0c76fb
RSS service + controller improved for consistency (#9233)
refs #9192, refs #5091 

- Moved all url generation into generate-feed.js, so we can see as much data processing as possible in a single place.
- Refactored the way res.locals were used, to be more like how express uses them prior to rendering
- Removed a bunch of code & tests todo with context for RSS - I can't see any way that'd be used, unless we switched the rendering to use a template.
- moved the RSS rendering to be part of the service, not controller
- updated the tests significantly 

Note: RSS generate-feed has a complete duplication of the code used in the excerpt helper in order to create an item description
2017-11-10 07:36:39 +00:00
Katharina Irrgang
f22a2784f7 🐛 Fixed error for password authentication with Bearer Token (#9227)
refs #8613, refs #9228

- if you send a request to /authentication/token with `grant_type:password` and a Bearer token, Ghost was not able to handle this combination
- because it skipped the client authentication, see https://github.com/TryGhost/Ghost/blob/1.17.0/core/server/auth/authenticate.js#L13
- and OAuth detects the `grant_type: password` and jumps in the target implementation
- the target implementation for password authentication **again** tried to fetch the client and failed, because it relied on the previous client authentication
- see https://github.com/TryGhost/Ghost/blob/1.17.0/core/server/auth/oauth.js#L40 (client.slug is undefined if client authentication is skipped)
- ^ so this is the bug
- we **can** skip client authentication for requests to the API to fetch data for example e.g. GET /posts (including Bearer)
- so when is a client authentication required?
- RFC (https://tools.ietf.org/html/rfc6749#page-38) differentiates between confidential and public clients, Ghost has no implementation for this at the moment
  - so in theory, public clients don't have to be authenticated, only if the credentials are included
- to not invent a breaking change, i decided to only make the client authentication required for password authentication
- we could change this in Ghost 2.0

I have removed the extra client request to the database for the password authentication, this is not needed. We already do client password authentication [here](https://github.com/TryGhost/Ghost/blob/1.17.0/core/server/auth/auth-strategies.js#L19);
If a Bearer token is present and you have not send a `grant_type` (which signalises OAuth to do authentication), you can skip the client authentication.
2017-11-09 14:11:29 +00:00
Hannah Wolfe
016ee17ebb
Rework route service to prevent circular dependencies (#9229)
refs #9192, refs #9178  

After trying to progress with current implementation, it became clear that the route service can't control the boot sequence, because then we end up with circular dependencies between the route service and the channel service.

The route service now exposes:
-  a siteRouter 
- a way for apps to register routes.
- ParentRouter base class for other modules to use
- the registry

...

- moved the default route setup back to site/routes.js 🙈
- moved the parent channel router back to the channel service (this makes way more sense imo)
- this structure prevents circular dependencies
- split the registry out into it's own thing
- fixed-up various bits of tests and comments
- DEBUG will print a list of routes 🎉
2017-11-09 13:58:22 +00:00
Hannah Wolfe
27b4688cea Changed channelsRouter to use new base class
refs #9192, #5091

- changed channels to use our new base class
- keep the flexible structure, so that channels can be reloaded
- I had to move the router into the route service otherwise we get circular dependencies
- Don't _really_ want to keep it like this - need a way to define base classes as shared
2017-11-09 10:47:20 +00:00
Aileen Nowak
6a41104969 Moved isLocalFile fn to storage utils (#8985)
no issue

- moved isLocalImage fn to storage utils used the RegExp of getLocalFileStoragePath to detect also relative image paths and added tests.
- Added test for independent protocol request (skip, because not supported/implemented)
2017-11-09 11:11:54 +01:00
Hannah Wolfe
90cfdbe7a6 Moved Channels module -> services
refs #9192, refs #5091, refs #9178

- moved channels from controllers to a service
- split out the parent router from the remaining individual router logic
- moved the tests to match
2017-11-08 15:45:23 +00:00