refs #7687
There are four main changes in this PR:
we have outsourced the base storage adapter to npm, because for storage developers it's annoying to inherit from a script within Ghost
we hacked theme storage handling into the default local storage adapter - this was reverted, instead we have added a static theme storage here
use classes instead of prototyping
optimise the storage adapter in general - everything is explained in each commit
----
* rename local-file-store to LocalFileStorage
I would like to keep the name pattern i have used for scheduling.
If a file is a class, the file name reflects the class name.
We can discuss this, if concerns are raised.
* Transform LocalFileStorage to class and inherit from new base
- inherit from npm ghost-storage-base
- rewrite to class
- no further refactoring, happens later
* Rename core/test/unit/storage/local-file-store_spec.js -> core/test/unit/storage/LocalFileStorage_spec.js
* Fix wrong require in core/test/unit/storage/LocalFileStorage_spec.js
* remove base storage and test
- see https://github.com/kirrg001/Ghost-Storage-Base
- the test has moved to this repo as well
* Use npm ghost-storage-base in storage/index.js
* remove the concept of getStorage('themes')
This concept was added when we added themes as a feature.
Back then, we have changed the local storage adapter to support images and themes.
This has added some hacks into the local storage adapters.
We want to revert this change and add a simple static theme storage.
Will adapt the api/themes layer in the next commits.
* Revert LocalFileStorage
- revert serve
- revert delete
* add storagePath as property to LocalFileStorage
- define one property which holds the storage path
- could be considered to pass from outside, but found that not helpful, as other storage adapters do not need this property
- IMPORTANT: save has no longer a targetDir option, because this was used to pass the alternative theme storage path
- IMPORTANT: exists has now an alternative targetDir, this makes sense, because
- you can either ask the storage exists('my-file') and it will look in the base storage path
- or you pass a specific path where to look exists('my-file', /path/to/dir)
* LocalFileStorage: get rid of store pattern
- getUniqueFileName(THIS)
- this doesn't make sense, instances always have access to this by default
* Add static theme storage
- inherits from the local file storage, because they both operate on the file system
- IMPORTANT: added a TODO to consider a merge of themes/loader and themes/storage
- but will be definitely not part of this PR
* Use new static theme storage in api/themes
- storage functions are simplified!
* Add https://github.com/kirrg001/Ghost-Storage-Base as dependency
- tarball for now, as i am still testing
- will release if PR review get's accepted
* Adapt tests and jscs/jshint
* 🐛 fix storage.read in favicon utility
- wrong implementation of error handling
* 🎨 optimise error messages for custom storage adapter errors
* little renaming in the storage utlity
- purpose is to have access to the custom storage instance and to the custom storage class
- see next commit why
* optimise instanceof base storage
- instanceof is always tricky in javascript
- if multiple modules exist, it can happen that instanceof is false
* fix getTargetDir
- the importer uses the `targetDir` option to ensure that images land in the correct folder
* ghost-storage-base@0.0.1 package.json dependency
closes#8187
- if you start Ghost via the Ghost-CLI, the path to the favicon must be absolute
- because the CLI spawns a Ghost process from the root folder of the CLI folder e.g. node current/index.js
no issue
🔥 Remove DIRTY HACK for API
- this is no longer needed, because themes get mounted in every case
✨ Switch to concept of 'mounted' theme
- check if active theme is mounted
- if not, mount it
- mounting is a function OF the active theme
🎨 Move theme middleware to theme module
🎨 Update theme middleware function names
- update the function names and comments to be more representative of their current functions
- this was pretty old and out of date!
🚨 Fixup tests for middleware
- ensure the objects match what we expect
- based partially on theme docs
Update TODO
no issue
- change out should.equal for // jshint ignore:line
- ensure should is the first require in every test, and ALWAYS require
- make sinon the second require, and sandbox the last thing
- ALWAYS use sandbox, futureproofs tests against contributors who don't know it
- change require formatting
closes#8131
- Remove ppp from default-settings.json
- Remove ppp from meta (unused?\!)
- ✨ Basic concept of theme config
- use theme config ppp setting
- ✨ Make @config.posts_per_page helper available
- rather than @blog.posts_per_page, we now have @config.posts_per_page
- 🚨 Test updates
- Adding TODO note
refs #8140✨ Support new default-prod.hbs template for admin
✨ Redirect ghost admin urls without a #
✨ Update admin urls to include #
🎨 Move the admin templates
🔥 Remove redirect to setup middleware
🚨 Tests for new middleware
📡 Add debug for the 3 theme activation methods
There are 3 different ways that a theme can be activated in Ghost:
A. On boot: we load the active theme from the file system, according to the `activeTheme` setting
B. On API "activate": when an /activate/ request is triggered for a theme, we validate & change the `activeTheme` setting
C. On API "override": if uploading a theme with the same name, we override. Using a dirty hack to make this work.
A: setting is done, should load & validate + next request does mounting
B: load is done, should validate & change setting + next request does mounting
C: load, validate & setting are all done + a hack is needed to ensure the next request does mounting
✨ Validate w/ gscan when theme activating on boot
- use the new gscan validation validate.check() method when activating on boot
✨ New concept of active theme
- add ActiveTheme class
- make it possible to set a theme to be active, and to get the active theme
- call the new themes.activate() method in all 3 cases where we activate a theme
🎨 Use new activeTheme to simplify theme code
- make use of the new concept where we can, to reduce & simplify code
- use new hasPartials() method so we don't have to do file lookups
- use path & name getters to reduce use of getContentPath etc
- remove requirement on req.app.get('activeTheme') from static-theme middleware (more on this soon)
🚨 Improve theme unit tests (TODO: fix inter-dep)
- The theme unit tests are borked! They all pass because they don't test the right things.
- This improves them, but they are still dependent on each-other
- configHbsForContext tests don't pass if the activateTheme tests aren't run first
- I will fix this in a later PR
no issue
🎨 simplify loader - use loadOneTheme for init
- use loadOneTheme for init
- move updateThemeList to the one place that it is used
- this just reduces the surface area of the loader
🎨 Move init up to index temporarily
- need to figure out what stuff goes in here as well as loading themes
- will move it again later once I've got it figured out
🎨 Reorder & cleanup theme middleware
- move the order in blog/app.js so that theme middleware isn't called for shared assets
- add comments & cleanup in the middleware itself, for clarity
🎨 Simplify the logic in themes middleware
- Separate out config dependent on settings changing and config dependent on request
- Move blogApp.set('views') - no reason why this isn't in the theme activation method as
it's actually simpler if it is there, we already know the active theme exists & can remove the if-guard
🎨 Improve error handling for missing theme
- ensure we display a warning
- don't have complex logic for handling errors
- move loading of an empty hbs object into the error-handler as this will support more cases
🐛 Fix assetHash clearing bug on theme switch
- asset hash wasn't correctly being set on theme switch
🎨 Remove themes.read & test loader instead
- Previously, we've simplified loader & improved error handling
- We are now able to completely remove theme.read as it's nothing more than a wrapper for package.read
- This also means we can change our tests from testing the theme reader to loader
no issue
🎨 Switch themes API to use config.availableThemes
- this gets rid of the only places where settings.availableThemes are used
🔥 Get rid of settings.availableThemes
- this is no longer used anywhere
- also get rid of every related call to updateSettingsCache
🔥 Replace config.availableThemes with theme cache
- Creates a tailor-made in-memory cache for themes inside the theme module
- Add methods for getting & setting items on the cache
- Move all references to config.availableThemes to use the new cache
- This can be abstracted later to support other kinds of caches?
🎨 Start improving theme lib's API
Still TODO: simplifying/clarifying:
- what is the structure of the internal list
- what is the difference between a package list, and a theme list?
- what is the difference between reading a theme and loading it?
- how do we update the theme list (add/remove)
- how do we refresh the theme list? (hot reload?!)
- how do we get from an internal list, to one that is sent as part of the API?
- how are we going to handle theme storage: read/write, such that the path is configurable
🎨 Use themeList consistently
🎨 Update list after storage
closes#8037🔥 Remove API-level default settings population
- This is a relic!
- We ALWAYS populate defaults on server start therefore this code could never run.
- This was a lot of complicated code that wasn't even needed!!
🎨 Move settings cache
- Move settings cache to be its own thing
- Update all references
- Adds TODOs for further cleanup
🎨 Create settings initialisation step
- Create new settings library, which will eventually house more code
- Unify the interface for initialising settings (will be more useful later)
- Reduce number of calls to updateSettingsCache
refs #7488
- we have recently changed our url redirects
- see https://github.com/TryGhost/Ghost/pull/7937
- the url has a canonical meaning and that's why Ghost shouldn't force redirect to the blog url
* 🔥 kill apiUrl helper, use urlFor helper instead
More consistency of creating urls.
Creates an easier ability to add config changes.
Attention: urlFor function is getting a little nesty, BUT that is for now wanted to make easier and centralised changes to the configs.
The url util need's refactoring anyway.
* 🔥 urlSSL
Remove all urlSSL usages.
Add TODO's for the next commit to re-add logic for deleted logic.
e.g.
- cors helper generated an array of url's to allow requests from the defined config url's -> will be replaced by the admin url if available
- theme handler prefered the urlSSL in case it was defined -> will be replaced by using the urlFor helper to get the blog url (based on the request secure flag)
The changes in this commit doesn't have to be right, but it helped going step by step.
The next commit is the more interesting one.
* 🔥✨ remove forceAdminSSL, add new admin url and adapt logic
I wanted to remove the forceAdminSSL as separate commit, but was hard to realise.
That's why both changes are in one commit:
1. remove forceAdminSSL
2. add admin.url option
- fix TODO's from last commits
- rewrite the ssl middleware!
- create some private helper functions in the url helper to realise the changes
- rename some wordings and functions e.g. base === blog (we have so much different wordings)
- i would like to do more, but this would end in a non readable PR
- this commit contains the most important changes to offer admin.url option
* 🤖 adapt tests
IMPORTANT
- all changes in the routing tests were needed, because each routing test did not start the ghost server
- they just required the ghost application, which resulted in a random server port
- having a random server port results in a redirect, caused by the ssl/redirect middleware
* 😎 rename check-ssl middleware
* 🎨 fix theme-handler because of master rebase
* 🎨🔥 do not store settings in config and make settings cache easier available
- remove remembering settings value in theme config
- if we need a cache value, we are asking the settings cache directly
- instead of settings.getSettingSync we use settings.cache.get
- added TODO:
- think about moving the settings cache out of api/settings
- we could create a folder named cache cache/settings
- this settings cache listens on model changes for settings
- decoupling
* 🔥 remove timezone from config
- no need to store in overrides config and in defaults settings
* 🎨 context object helper
- replace config.get('theme') by settings cache
* 🎨 replace config.get('theme') by settings.cache.get
* 🎨 adapt tests
* fixes from comments
closes#7688
- Use `/favicon.ico` and `/favicon.png` in blog app. Depending on type of storage (custom upload = local file storage), serves either from storage adapter with `read()` method or reads the bytes via `fs`.
- Redirects requests for `favicon.ico` to `favicon.png` if custom `png` icon is uploaded and vice versa.
- Redirect requests for `favicon.png` to `favicon.ico` if default icon is used (in `core/shared`).
- Changes the `{{asset}}` helper for favicon to not serve from theme assets anymore. It will either be served the custom blog-icon or the default one.
- The `{{@blog.icon}}` helper renders the url of the **uploaded** blog icon. It won't render the default icon.
no issue
- removes count from user checks model
- uses brute express brute with brute-knex adaptor to store persisted data on spam prevention
- implement brute force protection for password/token exchange, password resets and private blogging
closes#4172, closes#6948, refs #7491, refs #7488, refs #7542, refs #7484
* 🎨 Co-locate all admin-related code in /admin
- move all the admin related code from controllers, routes and helpers into a single location
- add error handling middleware explicitly to adminApp
- re-order blogApp middleware to ensure the shared middleware is mounted after the adminApp
- TODO: rethink the structure of /admin, this should probably be an internal app
* 💄 Group global middleware together
- There are only a few pieces of middleware which are "global"
- These are needed for the admin, blog and api
- Everything else is only needed in one or two places
* ✨ Introduce a separate blogApp
- create a brand-new blogApp
- mount all blog/theme only middleware etc onto blogApp
- mount error handling on blogApp only
* 🎨 Separate error handling for HTML & API JSON
- split JSON and HTML error handling into separate functions
- re-introduce a way to not output the stack for certain errors
- add more tests around errors & an assertion framework for checking JSON Errors
- TODO: better 404 handling for static assets
Rationale:
The API is very different to the blog/admin panel:
- It is intended to only ever serve JSON, never HTML responses
- It is intended to always serve JSON
Meanwhile the blog and admin panel have no need for JSON errors,
when an error happens on those pages, we should serve HTML pages
which are nicely formatted with the error & using the correct template
* 🐛 Fix checkSSL to work for subapps
- in order to make this work on a sub app we need to use the pattern `req.originalUrl || req.url`
* 🔥 Get rid of decide-is-admin (part 1/2)
- delete decide-is-admin & tests
- add two small functions to apiApp and adminApp to set res.isAdmin
- mount checkSSL on all the apps
- TODO: deduplicate the calls to checkSSL by making blogApp a subApp :D
- PART 2/2: finish cleaning this up by removing it from where it's not needed and giving it a more specific name
Rationale:
Now that we have both an adminApp and an apiApp,
we can temporarily replace this weird path-matching middleware
with middleware that sets res.isAdmin for api & admin
* 🎨 Wire up prettyURLs on all Apps
- prettyURLs is needed for all requests
- it cannot be global because it has to live after asset middleware, and before routing
- this does not result in duplicate redirects, but does result in duplicate checks
- TODO: resolve extra middleware in stack by making blogApp a sub app
* ⏱ Add debug to API setup
* 🎨 Rename blogApp -> parentApp in middleware
* 🎨 Co-locate all blog-related code in /blog
- Move all of the blogApp code from middleware/index.js to blog/app.js
- Move routes/frontend.js to blog/routes.js
- Remove the routes/index.js and routes folder, this is empty now!
- @TODO is blog the best name for this? 🤔
- @TODO sort out the big hunk of asset-related mess
- @TODO also separate out the concept of theme from blog
* 🎉 Replace middleware index with server/app.js
- The final piece of the puzzle! 🎉🎈🎂
- We no longer have our horrendous middleware/index.js
- Instead, we have a set of app.js files, which all use a familiar pattern
* 💄 Error handling fixups
refs #4172
* 🎨 Use bodyParser only where it is needed
This is a pretty extreme optimisation, however in the interests of killing middleware/index.js it
seemed prudent to move towards not having in there that wasn't strictly necessary 😁
We should reassess how apps do this sort of thing, but it seems pretty sane to declare bodyParsing
if and only if it is necessary.
* 🎨 Move all API code to API router
* 🎨 Refactor API into an App, not just a router
- Apps have their own rendering engines, only the frontend & the admin panel need views
- The API should be JSON only, with minimal middleware
- Individual sections within the API could/should be treated as Routers
* 🎨 Flatten API middleware inclusion
- get rid of the weird middleware object
- move the api-only middleware into the middleware/api folder
* 💄 Combine slashes & uncapitalise middleware
- these bits of middleware belong together
- ideally they should be optimised
* 🎨 Move ghostLocals out of themeHandler
GhostLocals sets several important values which are needed for every part of the application,
admin, api and theme. Therefore, it doesn't make sense for it to be bundled in the themeHandler.
* 🐛 Fix the uncapitalise middleware
- Updated to make correct use of req.baseUrl, req.path, req.url & req.originalUrl
- Updated the tests to actually cover our weird cases
* 🎨 Move ghostVersion logic out of config
* 💄 Group static / asset-related middleware together
* 🔥 Remove /shared/ asset handling
- The 5 files which are located in `/shared/` are all handled by individual calls to `serveSharedFile`
- Therefore this code is redundant
refs #7116, refs #2001
- Changes the way Ghost errors are implemented to benefit from proper inheritance
- Moves all error definitions into a single file
- Changes the error constructor to take an options object, rather than needing the arguments to be passed in the correct order.
- Provides a wrapper so that any errors that haven't already been converted to GhostErrors get converted before they are displayed.
Summary of changes:
* 🐛 set NODE_ENV in config handler
* ✨ add GhostError implementation (core/server/errors.js)
- register all errors in one file
- inheritance from GhostError
- option pattern
* 🔥 remove all error files
* ✨ wrap all errors into GhostError in case of HTTP
* 🎨 adaptions
- option pattern for errors
- use GhostError when needed
* 🎨 revert debug deletion and add TODO for error id's
- 🛠 add bunyan and prettyjson, remove morgan
- ✨ add logging module
- GhostLogger class that handles setup of bunyan
- PrettyStream for stdout
- ✨ config for logging
- @TODO: testing level fatal?
- ✨ log each request via GhostLogger (express middleware)
- @TODO: add errors to output
- 🔥 remove errors.updateActiveTheme
- we can read the value from config
- 🔥 remove 15 helper functions in core/server/errors/index.js
- all these functions get replaced by modules:
1. logging
2. error middleware handling for html/json
3. error creation (which will be part of PR #7477)
- ✨ add express error handler for html/json
- one true error handler for express responses
- contains still some TODO's, but they are not high priority for first implementation/integration
- this middleware only takes responsibility of either rendering html responses or return json error responses
- 🎨 use new express error handler in middleware/index
- 404 and 500 handling
- 🎨 return error instead of error message in permissions/index.js
- the rule for error handling should be: if you call a unit, this unit should return a custom Ghost error
- 🎨 wrap serve static module
- rule: if you call a module/unit, you should always wrap this error
- it's always the same rule
- so the caller never has to worry about what comes back
- it's always a clear error instance
- in this case: we return our notfounderror if serve static does not find the resource
- this avoid having checks everywhere
- 🎨 replace usages of errors/index.js functions and adapt tests
- use logging.error, logging.warn
- make tests green
- remove some usages of logging and throwing api errors -> because when a request is involved, logging happens automatically
- 🐛 return errorDetails to Ghost-Admin
- errorDetails is used for Theme error handling
- 🎨 use 500er error for theme is missing error in theme-handler
- 🎨 extend file rotation to 1w
issue #7452
Remote oauth2 authentication with Ghost.org.
This PR supports:
- oauth2 login or local login
- authentication on blog setup
- authentication on invite
- normal authentication
- does not contain many, many tests, but we'll improve in the next alpha weeks
closes#6948
- the hbs engine was never initialised when server starts
- when you request a page which does not exist, express jumps directly into the error handlers
- delete some dynamic hbs engine setters in theme handler
refs #6949
- Adds a new VersionMismatchError with status 400 (bad request)
- Adds middleware that checks the X-Ghost-Version header if it is provided
- If it is not provided, the middleware does nothing
- If it is provided, and the versions match, the middleware does nothing
- If it is provided, and the versions don't match, the middleware returns a VersionMismatchError
- Includes both unit and a functional test to prove the middleware works alone and as part of the whole system
refs #6644
- urls specified in config.js should be considered whitelisted/trusted
- this is not quite straightforward because config.js is not ready at the point the middleware is required
- tests have been updated to cover these new cases + use rewire to override the internal whitelist cache
closes#5914, #6589
- moves all private-blogging related code & tests into /server/apps/private-blogging/
- rework Grunt to run private-blogging tests
- modify server apps code to have a place for internal apps
Refs #6644
- deps: cors@2.7.1; Add express cors package.
- Adds new middleware for proper CORS support.
- Handles CORS pre-flight checks.
- Separates request authentication/authorization from
CORS.
Refs #6642
- Do not send CORS headers on an invalid "origin"
header, but otherwise allow the response to
proceed normally. This enforces CORS for the browser
but does not blow up non-CORS requests.
- Simplify the `init` method in `models/index.js` so that it no longer
returns a promise. Easier to use.
- Eliminates the `deleteAllContent` method from `models/index.js` as it
can all be handled at the API layer in a single spot.
- Optimize `destroyAllContent` in `api/db.js`. Eliminates
double-fetching every post from the database and converting it to
JSON. Also only fetches ids from the database instead of the entire
model.
- Eliminates the custom static method `destroy` in the Post model in
favor of handling detaching tag relations in a single place (the
`destroying` event). This also eliminates a big source of unneeded
database round trips--needing to get post ids to feed into
`Post.destroy()` which then re-fetches the post again.
no issue
- removes should-sinon dependency from package.json
- rewrites all usages of should-sinon to use normal should assertions
Unfortunately, should-sinon has very minimal documentation and therefore it is hard to discern what is considered a correctly-written assertion:
- in some cases, refactoring to use should-sinon causes false positives
- in other cases, assertions that work written in the normal way fail when written using should-sinon (e.g. getters, combos with rewire)
The additional overhead created by these issues outweigh any benefit from the easier-to-read assertions
closes#6505
-Removed all of the /*jshint expr:true*/ comments from the tests
-Removed all of the should.equal(true, true) statements from the tests
-Removed should from the greenkeeper ignores
closes#6448
-upgraded should.js to the latest version (8.2.1)
-Changed the tests so that they comply with the breaking changes introduced in the new version of should.js
-Installs the package should-http so should.be.json() can be used
-Installs the package should-sinon so that should.be.calledOnce() can be used
refs #5942, #6150
There were a few key problems I was looking to solve with this:
- Introduce a single point of truth for what the URL for accessing the API should be
- Provide a simple way to configure the utility (much like a true SDK)
As of this commit, this utility is still automatically available in a Ghost theme.
To use it on an external site, the code would look like:
```
<script type="text/javascript" src="http://my-ghost-blog.com/shared/ghost-url.min.js"></script>
<script type="text/javascript">
ghost.init({
clientId: "<your-client-id>",
clientSecret: "<your-client-secret>"
});
</script>
```
To achieve this, there have been a number of changes:
- A new `apiUrl` function has been added to config, which calculates the correct URL. This needs to be unified with the other url generation functions as a separate piece of work.
- The serveSharedFile middleware has been updated, so that it can serve files from / or /shared and to substitute `{{api-url}}` as it does `{{blog-url}}`.
- ghost-url.js and ghost-url.min.js have been updated to be served via the serveSharedFile middleware
- ghost-url.js has been changed slightly, to take the url from an inline variable which is substituted the first time it is served
- `{{ghost_head}}` has been updated, removing the api url handling which is now in config/url.js and removing the configuration of the utility in favour of calling `init()` after the script is required
- `{{ghost_head}}` has also had the meta tags for client id and secret removed
- tests have been updated
no issue
- provide a single point for accessing config in unit tests
- create a single way to set and restore config
- ensure that restore deletes top level optional keys that are now undefined
- use this._config in check deprecations, otherwise the config gets cached
- solves issues with interdependent tests
no issue
- If client credentials are missing, or not valid, output a clear message in the server console
- Still defaults to sending the 'access denied to url' error to the frontend
