refs https://github.com/TryGhost/Team/issues/41
- if you add invalid handlebars logic e.g. {{if condition condition}}, handlebars throws an error
- in case of having invalid hbs in an amp page, the amp component throwed another syntax error (which is fixed in this PR)
- furthermore the `setTemplate` helper function had a logic bug, which did not handle errors correctly
- if there is an error and a template is set (e.g. amp), we have to still render the error page and not the amp page
- this fix only ensures that the error handling is correct, we still see the error of the "ugly" handlebars message
- e.g. [amp.hbs] Cannot read property 'includeZero' of undefined
- but no longer -> Cannot read property 'html' of undefined (which was a syntax error in Ghost)
refs https://github.com/TryGhost/Team/issues/41
- if you send invalid encoded url components in the path, the server tried to decode the url
- if it contains invalid characters like /AF%, it throwed a 500
- we return a page not found error instead
no issue
Support for http://resthooks.org style webhooks that can be used with Zapier triggers. This can currently be used in two ways:
a) adding a webhook record to the DB manually
b) using the API with password auth and POSTing to /webhooks/ (this is private API so not documented)
⚠️ only _https_ URLs are supported in the webhook `target_url` field 🚨
- add `webhooks` table to store event names and target urls
- add `POST` and `DELETE` endpoints for `/webhooks/`
- configure `subscribers.added` and `subscribers.deleted` events to trigger registered webhooks
no issue
- added https://github.com/TryGhost/bookshelf-relations as dependency
- remove existing tag handling
---
* Important: Ensure we trigger parent initialize function
- otherwise the plugin is unable to listen on model events
- important: event order for listeners is Ghost -> Plugin
- Ghost should be able to listen on the events as first instance
- e.g. be able to modify/validate relationships
* Fix tag validation
- we detect lower/update case slugs for tags manually
- this can't be taken over from the plugin obviously
- ensure we update the target model e.g. this.set('tags', ...)
* override base fn: `permittedAttributes`
- ensure we call the base
- put relations on top
- each relation is allowed to be passed
- the plugin will auto-unset any relations to it does not reach the database
* Ensure we run add/edit/delete within a transaction
- updating nested relationships requires sql queries
- all sql statements have to run in a single transaction to ensure we rollback everything if an error occurs
- use es6
closes#8143
Fixed a potential issue (edge-case), where our generated and validated (in terms of check for existance and add a counter) would return a slug, that will exceed the maximum length of the slug fields (191 chars).
This is mostly possible for the post title, which can be 255 chars long and would generate a slug with the same length. This would prevent the user from actually saving a post.
I tried first to determine the expected length for a slug that already exists, but decided that the **easier** and simplyfied implementation is to always cut a slug to **185 chars** (+ counter). This makes it easier to find duplicates and includes a possible high number of counts (edge-edge-case).
The slug will not be cut down to 185 chars if it's an import.
refs #9192
- Introduces a url service that can be initialised
- Added a concept of Resources and resource config.json that contains details about the resources in the system that we may want to make customisable
- Note that individual resources know how to create their own Urls... this is important for later
- Url Service loads all of the resources, and stores their URLs
- The UrlService binds to all events, so that when a resource changes its url and related data can be updated if needed
- There is a temporary config guard so that this can be turned off easily
refs #8143
Add max length validations to settings:
- `blog.title`: 150 chars
- `blog.description`: 200 chars
The `validateSettings` fn in our validations checks for existing `validations` properties in our `default-settings.json` file, similar to other tables in our `schema.js`.
no issue
- when calling `doesTranslationKeyExist`, we want to know if a key exists, we don't want to log if the key was not found
- this can mess up the server log
no issue
- useful for managing subscribers via external systems/API calls where it's likely only the e-mail address will be known
- adds `GET /subscribers/email/:email/`
- adds `DELETE /subscribers/email/:email/`
no issue
Had a couple of people ask about how to delete welcome posts easily, so adding a bio to the default user to draw a little more attention to it
no issue
- it can happen that concurrent requests try to renew access tokens with the same refresh token
- in this case it could happen that you received a token deletion error
- add propert locking
- ensure we don't run into deadlocks
- manual testing with async.times for parallel requests (was able to reproduce the error)
refs #5091, #9192
- Renderer figures out templates, contexts, and does a render call
- Templating is now handled with a single function
- Context call is made in the renderer
Note: to make this work, all controllers now define a little bit of config, currently stored in res._route. (That's a totally temporary location, as is res._template... when a sensible naming convention reveals itself I'll get rid of the weird _). This exposes a type and for custom routes a template name & default.
refs #9192, refs #5091
- Moved all url generation into generate-feed.js, so we can see as much data processing as possible in a single place.
- Refactored the way res.locals were used, to be more like how express uses them prior to rendering
- Removed a bunch of code & tests todo with context for RSS - I can't see any way that'd be used, unless we switched the rendering to use a template.
- moved the RSS rendering to be part of the service, not controller
- updated the tests significantly
Note: RSS generate-feed has a complete duplication of the code used in the excerpt helper in order to create an item description
refs #8613, refs #9228
- if you send a request to /authentication/token with `grant_type:password` and a Bearer token, Ghost was not able to handle this combination
- because it skipped the client authentication, see https://github.com/TryGhost/Ghost/blob/1.17.0/core/server/auth/authenticate.js#L13
- and OAuth detects the `grant_type: password` and jumps in the target implementation
- the target implementation for password authentication **again** tried to fetch the client and failed, because it relied on the previous client authentication
- see https://github.com/TryGhost/Ghost/blob/1.17.0/core/server/auth/oauth.js#L40 (client.slug is undefined if client authentication is skipped)
- ^ so this is the bug
- we **can** skip client authentication for requests to the API to fetch data for example e.g. GET /posts (including Bearer)
- so when is a client authentication required?
- RFC (https://tools.ietf.org/html/rfc6749#page-38) differentiates between confidential and public clients, Ghost has no implementation for this at the moment
- so in theory, public clients don't have to be authenticated, only if the credentials are included
- to not invent a breaking change, i decided to only make the client authentication required for password authentication
- we could change this in Ghost 2.0
I have removed the extra client request to the database for the password authentication, this is not needed. We already do client password authentication [here](https://github.com/TryGhost/Ghost/blob/1.17.0/core/server/auth/auth-strategies.js#L19);
If a Bearer token is present and you have not send a `grant_type` (which signalises OAuth to do authentication), you can skip the client authentication.
refs #9192, refs #9178
After trying to progress with current implementation, it became clear that the route service can't control the boot sequence, because then we end up with circular dependencies between the route service and the channel service.
The route service now exposes:
- a siteRouter
- a way for apps to register routes.
- ParentRouter base class for other modules to use
- the registry
...
- moved the default route setup back to site/routes.js 🙈
- moved the parent channel router back to the channel service (this makes way more sense imo)
- this structure prevents circular dependencies
- split the registry out into it's own thing
- fixed-up various bits of tests and comments
- DEBUG will print a list of routes 🎉
refs #9192, #5091
- changed channels to use our new base class
- keep the flexible structure, so that channels can be reloaded
- I had to move the router into the route service otherwise we get circular dependencies
- Don't _really_ want to keep it like this - need a way to define base classes as shared
no issue
- moved isLocalImage fn to storage utils used the RegExp of getLocalFileStoragePath to detect also relative image paths and added tests.
- Added test for independent protocol request (skip, because not supported/implemented)
refs #9192
- Moving towards a centralised concept of routing / routes
- The base router now wraps express router, and offers us the features we need
- Site Router is the parent router, it gets initialised with all of our default routing
- App Router is a sub router for apps - apps register their routes/routers onto it.
- TODO: refactor channels subrouter to work this same way
- MAYBE: move the app router to the apps service
refs #9192, refs #5091, refs #9178
- moved channels from controllers to a service
- split out the parent router from the remaining individual router logic
- moved the tests to match
refs #9192
- Admin redirects should really happen first, up with custom redirects
- Later we can package this up, maybe
- For now, let's focus the site router on site-related things
refs #5091, refs #9192
- There are several theme template "renderers" all over the codebase
- Some are in apps, and were called "controllers"
- One is in error handling
- All of them now have comments marking out how they share logic/steps
- Other comments describe routes & controllers where they live
refs #9192
- The AMP app is nothing more than a custom controller - this will come clear soon
- Moved enabled/disabled logic into router
- Removed error-related code, as this wasn't used
- Changed logic for static pages to be based on req.body, not context
- Improved the tests to match
refs #5091, #9192, #9178
- Get the RSS module into a much better shape
- Controller -> /controllers/rss
- Remainder -> /services/rss
- Moved tests to match & updated requires
refs #5091, refs #9192
- This is similar to #9218, in that I'm revealing bits of code that are "controllers" in our codebase. As opposed to routes, services, renderers etc.
- This also reveals some code which is identical to the channels controller
- There is more to do here, but for now I've got the module split up, and the tests split and improved.
- Next I'll split RSS into controller + service, DRY up the controller code, etc
closes#9200
- Registered new server helper `{{reading_time}}`.
- Added new global util `word-count` based on the util in Ghost admin, which returns the number of words in an HTML string.
- Based on the word count of the post html, the helper calculated the estimated reading time:
- 275 words per minute
- additional 12 seconds when post has feature image
- Renders a string like 'x min red', unless reading time is less than a minute. In this case, the rendered string is '< 1 min read'.
refs #5091, refs #9192
- render channel was always a weird file
- now it's clearly 2 things
- we're slowly getting towards closing #5091... 🎉
- added some extra tests
refs #9192, refs #5091
- Using a class allows for easy shared logic
- Loading is designed to work from config right now, but could be DB driven, etc
- Provided configuration can be simplified and extended in the constructor / class methods
- Update tests, move custom assertions to utils
refs #9192
To anyone seeing this go by - I'm about to start some fairly major refactoring work on the url utility. Before I do that, I wanted to make sure I had 100% coverage, and understanding of some of the weird cases.
The majority of the changes I've made are adding tests, but I was also able to clean up a little bit, remove a few lines or change them to make use of other tools.
refs #9192
- Instead of `setupRoutes` function in apps that gets passed a router, there is now a registerRouter function as part of the proxy
- Moved towards a route service, which will know about all routes
- Using classes to abstract away shared behaviour
Notes:
- changing the app proxy didn't result in a test failure!
- structure of route service is totally new and may change a lot yet
refs #9178
* Add eslint deps, remove old lint deps
* Add eslint config, remove old lint configs
* Config for server and tests are different
* Tweaked rules to suit us
* Fix linting in codebase - lots of indent changes.
* Fix a real broken test
refs #9192
- Each setting is saved individually
- Update this to only happen on import, or when a value changes
- Reduces the amount of work Ghost does on every setting change
no issue
- we had to fork the original repository at one point, because of slow maintenance
- maintenance is back now
- https://github.com/maxogden/extract-zip/pull/52 was merged and released
refs #9178
* Moved app handling code into services/apps
- Apps is a service, that allows for the App lifecycle
- /server/apps = contains internal apps
- /server/services/apps = contains code for managing/handling app life cycle, providing the proxy, etc
* Split apps service tests into separate files
* Moved internal app tests into test folders
- Problem: Not all the tests in apps were unit tests, yet they were treated like they were in Gruntfile.js
- Unit tests now live in /test/unit/apps
- Route tests now live in /test/functional/routes/apps
- Gruntfile.js has been updated to match
* Switch api.read usage for settingsCache
* Add tests to cover the basic App lifecycle
* Simplify some of the init logic
- This happened by accident when the model filters were introduced
- filters_spec.js was supposed to be renamed to apps_filters_spec.js
- Removing the duplicate file now
refs #8995
- move the getClient lookup from ghost_head into middleware
- use res.locals to keep track of the information (res.locals.client)
- make the middleware global to all frontend routes
- ghost_head: get locals from options.data not this (!)
- adapt lot's of tests
refs #9150
- Moves the password length fn from `models/user` to `data/validation` where the other validator functions live.
- Added password validation rules. Password rules added:
- Disallow obviously bad passwords: '1234567890', 'qwertyuiop', 'asdfghjkl;' and 'asdfghjklm' for example
- Disallow passwords that contain the words 'password' or 'ghost'
- Disallow passwords that match the user's email address
- Disallow passwords that match the blog domain or blog title
- Disallow passwords that include 50% or more of the same characters: 'aaaaaaaaaa', '1111111111' and 'ababababab' for example.
- Password validation returns an `Object` now, that includes an `isValid` and `message` property to differentiate between the two error messages (password too short or password insecure).
- Use a catch predicate in `api/authentication` on `passwordReset`, so the correct `ValidationError` will be thrown during the password reset flow rather then an `UnauthorizedError`.
- When in setup flow, the blog title is not available yet from `settingsCache`. We therefore supply it from the received form data in the user model `setup` method to have it accessible for the validation.
refs #5091
- Move all of the code to do with handling channels into one folder
- Still keeping all the shared/simlar code for rendering etc inside weird
frontend folder until I am sure what this will look like
refs #9178
- Introduce the /services/ folder
- Move xmlrpc there
- Move slack there
- In slack: remove a usage of the settings API that should use settingsCache
- In slack: Simplify the tests
- Various tiny changes to move towards code consistency
refs #5091
- remove the use of functions
- remove unnecessary quotes from tag filter
- move channel config to be a JSOn file called config.channels.json
- accept external config
- new channelUtils for tests
- remove channelConfig.get
- refactor so tests work as expected
- refactor away duplicate 'name' value
closes#9164
- check options.importing on xmlrpc
- also don't ping if private
- cleanup slack to work the same way
- update tests
- TODO: we need to prevent this event happening altogether
refs #5091
- There is very little that changes here, just code readability
- However I've expanded out the tests getting ready to be able to test more deeply as I refactor the routing
refs #9150
- move data manipulation for importing users from `importers/data/users` to `model/user` for more consistency (see behaviour of post imports)
- changed importing logic in `onSaving` fn for user model:
- when importing, we set the password to a random uid and don't validate, just hash it and lock the user
- when importing with `importPersistUser` we check if the password is a bcrypt hash already and fall back to normal behaviour if not (set random password, lock user, and hash password)
- don't run validations when importing
refs #9141
- adds support for `{{#prev_post in="author"}}{{/prev_post}}` & `{{#next_post in="author"}}{{/next_post}}`
- "author.slug" is the author equivalent of "primary_tag.slug" - there is only one
- added tests to cover both cases in the prev/next helpers
closes#9136
Changed the functionality in `make-absolute-urls.js` util to not convert the URL when starting which an `#`, as it indicates and internal link.
The util is used inside of the `{{amp_content}}` helper and to render the RSS feed. I tested the changes with the most popular RSS reader 'Feedly' and it seems like these internal links get converted to absolute URL inside of Feedly automatically.
no issue
- This started as an attempt to simplify the admin redirect code
- I realised we were sometimes using utils.redirect301 and sometimes not
- Decided to move this into utils.url as it's more relevant to URL generation
- Unified usage of redirects in the codebase
- Updated tests & ensured we have basic coverage
- rename adminRedirect -> redirectToAdmin
- Tweak method signature, fix channel edit redirects
- Tests: Optimised test descriptions for url-redirects_spec.js
- ensure caching works as expected
closes#8222
- There are still some cases where Ghost shows "the currently active theme X is missing" when it isn't
- This is due to the error handling masking several cases
- This PR resolves that, ensuring errors from gscan and the underlying environment don't get masked
closes#9060
- Update `gscan` - it now extracts custom templates and exposes them to Ghost
- Add `custom_template` field to post schema w/ 1.13 migration
- Return `templates` array for the active theme in `/themes/` requests
- Users with Author/Editor roles can now request `/themes/`
- Front-end will render `custom_template` for posts if it exists, template priority is now:
1. `post/page-{{slug}}.hbs`
2. `{{custom_template}}.hbs`
3. `post/page.hbs`
closes#8668, refs #8920
- Updated tests to include internal tags
- Tests had no example of an internal tag
- Need this to show that the new filtering works as expected
- primary_tag is a calculated field
- This ensures that we can alias the field to equivalent logic in API filters
- By replacing primary_tag by a lookup based on a tag which has order 0
- bump ghost-gql to 0.0.8
**NOTE:**
Until GQL is refactored, there are limitations on what else can be filtered when using primary_tag in a filter e.g. it wont be possible to do a filter based on primary_tag AND/OR other tag filters.
refs #5091
- This removes hardcoded config to generate feed urls
- This means that RSS feeds work properly for custom channels
- Remaining assumption is that paginated feeds will end /pageNum/
- Added extra tests
- Don't pass through query params
refs #5091
- This simple change allows custom contexts to use existing channel logic
- E.g. if we want to create a custom tag-based channel, it can pass "tag" as the context, and get all the same metadata logic
refs #9043
- Cleanups / refactors to make the code more manageable
- Move remaining code out of index.js
- Only "init" function is left. Actions map cache and init function is based heavily on the settings cache module
- refactor the odd way of exporting
- This was cleaned up naturally by moving the actionsMap object out
- rename "effective" -> "providers"
- "Providers" provide permissions for different things that can have permissions (users, apps, in future clients).
refs #9001
When a blog is in private mode there is now an unguessable URL that allows access to the RSS feed for internal use, commenting systems, etc.
- add public hash for private blogging
- auto generate on bootstrap if missing
- global hash, we can re-use in the future
- update private blogging middleware to detect the private RSS URL and rewrite it so that the normal rss route/code is used for display
- if a normal `/rss/` route is accessed with a private session return a 404
no issue
- If the brute store throws an error and the `handleStoreError` is called, then the storage is unable to get/set values.
- This is not a PermissionError. The result is that the user has no access, because the brute store has problems reading/writing to the storage.
closes#9089
- use the current date any time a post is fetched if the database contains an invalid date
- raise an error any time an attempt is made to save an invalidate date via the API
closes https://github.com/TryGhost/Ghost/issues/8943
- if you send a tag name with a hash, it's an internal tag
- ensure that the visibility property is forced to `internal`
- add a proper test
no issue
- preparation for #9001
- no need to require the settings API, we can simply fetch the data from the settings cache
- the settings API uses the settings cache anyway
closes https://github.com/TryGhost/Ghost/issues/8987
- set `linkify-it` `fuzzyLink` option to false so that it only auto-links URLs starting with `http(s)://` or other valid schemes
no issue
* Comment current state of toJSON for user model
- currently the user model does not return the email if the context is app/external/public OR if there is no context object at all
- i am not 100% sure why if there is no context we should not return the email address
- i think no context means internal access
- maybe change this condition cc @ErisDS
* Extend our access rules plugin
- we already have a instance method to determine which context is used
- this relies on passing options into `.forge` - but we almost never pass the context into the forge call
- added @TODO
- provide another static method to determine the context based on the options object passed from outside
* Use the new static function for existing code
* Add comment where the external context is used
* Remove certain fields from a public request (User model only)
* Tests: support `checkResponse` for a public request
- start with an optional option pattern
- i would love to get rid of checkResponse('user', null, null, null)
- still support old style for now
- a resoure can define the default response fields and public response fields
* Tests: adapt public api test
* Tests: adapt api user test
- use new option pattern for `checkResponse`
- eww null, null, null, null....
* Revert the usage of the access rules plugin
no issue
- it's not allowed to change/add these attributes via the API
- created_at = is only once set on adding the resource
- created_by = is only once set on adding the resource
- updated_by = is set on the server side when updating the model (based on who is logged in)
- updated_at = is set on the server side when updating the model
* Revert the usage of the access rules plugin
closes#9077
- because of our API layer refactoring, see https://github.com/TryGhost/Ghost/pull/9068
- we can now see that code was written wrong because of this horrible API bug
- this fixes the formats parameter for querying a single post
no issue
- the logic here bypasses filtering options!
- that is wrong, because if we filter out certain options e.g. include
- the tests from the previous commit fail because of this
- if we don't fix this logic, the tests won't pass, because as said, you can bypass certain logic e.g. remove roles from include
- this has worked before, because we passed the wrong options via the API layer
- was introduced here 014e2c88dd, because of https://github.com/TryGhost/Ghost/pull/6122
- add proper tests to proof that these queries work!!
no issue
- this has a big underlying problem
- each task in the pipeline can modify the options
- e.g. add a proper permission context
- if we chain after the pipeline, we don't have access to the modified options object
- and then we pass the wrong options into the `toJSON` function of a model
- the toJSON function decides what to return based on options
- this is the easiest solution for now, but i am going to write a spec if we can solve this problem differently
🐛 Fixed author role permission to change author
no issue
- To be able to fix this bug, we had to solve tasks from #9043
- This bug affects the private / undocumented API only
- Author role users should not be allowed to change the author of a post
no issue
- we store dates without milliseconds in the database
- our test environment does not use our model layer to insert data, this is related to https://github.com/TryGhost/Ghost/issues/7196
- so it can happen that the test env inserts unix timestamps instead of a formatted string
- e.g. adding data via the model layer (e.g. via the API) the format is always normalised to `YYYY-MM-DD HH:mm:ss`
- if we fetch the date from the database, we have a hook which sorts out knex returning different formats for dates
- this hook wraps the returned date into a UTC moment date, but adds the current milliseconds on top
- which can collide in tests when you have specific assertions
- use `startOf` to ignore milliseconds
- furthermore: remove the mentionings of `pg` (postgres)
refs #8602
- Add the wiring to pass attributes around the permission system
- Allows us to get access to the important "unsafe" attributes that are changing
- E.g. status for posts
- This can then be used to determine whether a user has permission to perform an attribute-based action
- E.g. publish a post (change status)
no issue
- our public API is still a beta/labs feature
- from api.ghost.org
> The API is still under very (very) heavy development and subject to regular breaking changes.
- users should expect breaking changes in any release (independent from semver versions)
- the public user API never returns any email addresses to decrease the information we expose
- there is no need to keep the support fetching a user by email address
refs #9043
- this is preparation for adding Author-specific tests later
- the changes the posts_spec.js, so that all the tests are inside an "As Owner" describe block, similar to the users_spec.js
- Added new util for creating a specific post
- This will make it easier to do routing tests on the post model in future
- Our `index.js` file in test/utils really needs a bit of love 🙈
- Also added all the framework for author role tests in post_spec.js
- Added a single test, showing we can edit posts, including author_id
no issue
- this bug fix affects all endpoints for the public user access
- we allowed fetching `roles` via the public api by accident
- see our docs: https://api.ghost.org/docs/users)
- we only allow `count.posts`
- returning roles via the public api exposes too many details
- this was never intentional
refs #9043
- Move api util tests into api section
- Adding export test to utils to see the amount of functions which are exported
- Adding basic handlePermissions tests
refs #9028
- if you upload a redirects file and a redirects file exists already, we backup this file to `data/redirects-YYYY-MM-DD-HH-mm-ss.json`
- decrease chance of random test failures by not comparing date format with seconds
no issue
- this endpoint does not exist anymore
- if you want to add a new user, you have to invite him via the invites API
- on invite accept, the user is inserted
no issue
- this bug fix affects all endpoints for the public user access
- we allowed fetching `roles` via the public api by accident
- see our docs: https://api.ghost.org/docs/users)
- we only allow `count.posts`
- returning roles via the public api exposes too many details
- this was never attentional
refs #9043
- Split public-related and context code into logical components
- Split tests up to match
- Ensure we have 100% unit test coverage
- General cleanup
refs #9028
- add two new endpoints for uploading/downloading the redirects (file based)
- reload/re-register redirects on runtime
- migration for 1.9 to add permissions for redirects download/upload
no issue
- use latest casper in test fixtures
- never ever use the root content folder for tests
- if we start/fork Ghost for the tests, we use a tmp folder
- this change is required to for an upcoming PR (#9029)
- i've added a TODO to create a helper fn for stopping the ghost server, so we can cleanup the tmp folder
* Care about TODO's in our channels spec
- add the 1.4 compatible casper theme to fixtures
- so as soon as you start Ghost, the test env will provide the content folder in /tmp something with the activated latest default casper and the 1.4 compatible old casper
- there are tests which tests different logici e.g. pagination
- therefor we need a different theme, we are simply using our 1.4 casper
refs https://github.com/TryGhost/Ghost/issues/8859
There are four cases:
- unsplash setting is empty (default), admin can enable the app by default (hardcoded isActive:true)
- unsplash settings are set, unsplash is disabled, admin detects that app was disabled on purpose
- unsplash setting is set, unsplash is enabled and has a key, app is enabled, old key get's ignored and overridden on the next save
- unsplash setting is set, unsplash is enabled and has no key, app is enabled
refs https://github.com/TryGhost/Ghost/issues/8859
- We don't need the config option for Unsplash anymore
- The private endpoint (/configuration/private) was introduced for Unsplash
* Improved log output for welcome email error
no issue
- if Ghost is unable to send a welcome email, the server log printe a huge error log
- the reason was that each component wrapped the original error into a new error instance
- so the stack grows and grows
- the golden rule should always be: the smallest/lowest component should instanitate a specifc error
- the caller can expect to receive a custom Ghost error
* Tidy up error messages for mail failures and fix tests
- We never use "Error:" notation in our translations
- Make the error messages consistent and show a reason if possible
no issue
- mirror LTS behaviour to master
- if your blog or admin url is configured to http, it's still possible that e.g. nginx allows both https/http
- that's why we should generate the api url without protocol in this case
- so it depends how you serve your blog, example:
- blog url is http://example.com
- generated api url for the sdk is //example.com (dynamic protocol allowed)
- you serve your blog via https://example.com, protocol is https
- you serve your blog via http://example.com, protocol is http
closes#8342
- no need to add a migration, because when we'released 1.0, OAuth was never an option
- it was disabled in April, 1.0-beta was released in June
- remove all remote authentication code
no issue
- if you blog runs on a custom domain, but your admin panel is configured using a different domain
-> Ghost losts the origin header
- we had this situation once with pretty urls (your request get's redirected from /posts to /posts/, see https://github.com/TryGhost/Ghost/pull/8094)
- we've moved all our redirect logic to Ghost and ran into the same situation
- i've added proper test to ensure it won't happen again
closes#9006
- this is a temporary fix to only show update notifications for minor/major releases
- the notification refactoring is in the pipeline, but not yet merged into 1.X/LTS, see https://github.com/TryGhost/Ghost/pull/8871
no issue
- Consistent naming for postLookup
- makes it easier to search and inspect the various usages
- Cleanup unneeded code
- Make res.render calls more consistent
- add some consistency to the calls to res.render
- Remove ancient reference to dataProvider
- Let's call it models everywhere now...
- Use consistent formatting across the API
- we're no longer using alignment in vars
- Misc other consistency changes in API
- always refer to local utils as apiUtils
- logical grouping of requires - dependencies, utils, "lib common" etc
- use xAPI to refer to API endpoints, e.g. mailAPI, settingsAPI for clarity
refs #7470
- the importer test causes problems with the order of posts
- the importer is greedy and tries to add data in parallel, but the tests simply fetch the raw data from knex without any order
- while i was improving the order problem, i found this amp/disqus edge case
Order Random Failure Example:
1) Import (new test structure) 1.0: basic import test keeps the value of the amp field:
AssertionError: expected '59a952be7d79ed06b0d21128' to equal '1'
+ expected - actual
-59a952be7d79ed06b0d21128
+1
no issue
- while i was testing random failures, i discovered an edge case for disqus
- you start a new 1.0 blog, you add disqus, the unique identifer is the post id (object id)
- now you export your content and import it on a new instance
- the importer detects that the amp field is null and imports the old object id as comment id
- but the post model is not prepared for this case
- see next commit for tests
**NOTE**: The comment id had two different data types (Number or String). Disqus expects a string. So this should not change any behaviour, now that the comment_id is always a string.
refs #8868
* 📐 Use request util in image-size
- swapped the usage of `got` for requests with the request util
* 💄 Use catch predicates
- Uses catch predicates instead of conditionals in `getImageSizeFromUrl`
- Return `NotFoundError` if applicable in `getImageSizeFromFilePath` as the caller function `cachedImageSizeFromUrl` is differentiating those between this error and others.
* 🐛 Fixed ImageObject URL & simplify no protocol URL logic
- Using `ImageObject` as a global var resulted in having the `url` property being the same for all requests coming in.
- The logic that checked for an existing protocol (e. g. gravatar URLs) was overly complicated. Refactored it to be more simple.
- Passing the correct value to `fetchDimensionsFromBuffer` as the population of `imageObject.url` happens there. These are used in our structured data and need to be full URLs (in case of locally stored files) or the original URL (in case of URLs missing the protocol)
- Added two more debug logs in `getCachedImageSizeFromUrl` so it's logged when an image is added to the cache even tho it was returned as error.
* 👀 Differentiate error codes between request and storage
* 🔥 Remove not needed `Promise.resolve()`
We're always resolving the result in `getCachedImageSizeFromUrl`, so there's no need to return the values with a `Promise.resolve()`. The caller fn uses waits for the Promises to be fulfilled.
* ☂️ Wrap already rejected predicate errors in catch all
* Use errorDetails instead of context
* ☂️ Support /assets/ image paths
- adds a guard that checks the image URL for `/assets/` in the beginning and passes a completed URL to the request util to try and fetch the image size
- adds tests
* 🐛 Fixed private blogging leaking post information
closes#8990
- a condition in the private blogging app redirected rss && sitemap to 404, which can possibly leak content
- remove this condition and ensure we always redirect to /private
* lint 😋
no issue
This PR includes a new util which wraps the `got` library. It is not used in the codebase yet, but tested with `image-size` util:
- wraps `got` request library in its own `request.js` util that returns bluebird promises and validates URL before starting a request
- adds tests
no issue
- Added debug statements to ghost_head
- useful for determining how much render time is spent in ghost head
- Make promises more readable
- Used join instead of props for less code
no issue
- added debug logs to image size util and related fn:
- when fetched via network request
- when fetched from storage
- when added to cache
- when read from cache
refs #8945
- pass the original error as part of the errorDetails
- improves logging, so we know what really went wrong
- for debug purposes - can be removed at some point
no issue
- define `period` and `count` in our defaults.json config
- advantage: easier access and better overview how logging is configured by default
- period is 1d and count is 10
no issue
- test cases were trying to fetch image sizes for `localhost:port/favicon.ico` but no server is running so they time out
- stub the `getImageSizeFromUrl` method so it resolves instantly
refs #8868
- Removed image-size in blog logo fn for meta data and made it synchronous
- Renamed `image-size-from-url.js` to `image-size.js` (incl. the test)
- Added second fn `getImageSizeFromFilePath` that reads from local file storage
- Added guard in `getImageSizeFromUrl` that checks if the image should be on local file storage and uses the new fn then instead
- Added a fn `fetchDimensionsFromBuffer` that takes the file buffer and returns an `imageObject` with dimensions.
- Added a new utils.js in `adapters/storage` for getting the file storage path
closes#8963
- if an LTS export is imported into a 1.0 blog, then the 1.0 blog is
exported and re-imported into another 1.0 blog, any post ids from the
lts import were getting clobbered. This only saves the post id if the
amp field does not already exist
- add failing test that passes w/change
closes#8757
- update the markdown card render method to use SimpleDOM's `createRawHtmlSection`. This avoids SimpleDOM parsing and tokenization of broken or unsupported free-form HTML that markdown allows
- replace markdown extraction/render with mobiledoc's renderer in the `Post` model
- removes `jsdom` as it's no longer necessary
no issue
v1.0.0 is no longer the standard in the docs, so I updated all of the URLs containing it with v1
Note: I tried squashing commits, but failed. I'll try again in the future with throwaway changes
Secondary Note: I tested most of the URLs listed and got no 404s!
no issue
- add caching logic to adapter creation (same as we use for storages)
- add debug logs to the default scheduler
- add `requestTimeout` to the default scheduler to support custom timeouts
- add `isRunning` logic to protect running the scheduler twice
no issue
- reduce the number of redirects
- before: you are redirected from example.com/ghost to admin.example.com/ghost and Ghost would detect a missing slash and redirect you to /ghost/
- now: you are redirected from example.com/ghost to admin.example.com/ghost/
no issue
- this bug was invented with this commit 25c4e5025a
- the updated logic ensures that
- only if you have configured a custom admin url and your requested host does not match, we redirect you
- we still keep the wish of no force redirect if you have only configured a custom blog url and you navigate to /ghost
refs #8703
- Instead of throwing errors, throw warnings for incorrect usage of the img_url helper
- Differentiate between no attribute passed, and attribute evaluating to undefined
refs #8703
- Always always call cb() even if we get an error!
- Ensure the error is handled, and converted to a GhostError if not already
- If we're in development mode, render the error. Else render nothing.
fixes#8920
- Implements logic such that internal tags cannot be primary tags
- If the first tag on a post is an internal tag, that post will not have a primary tag
no issue
- adds a ghost-backup client
- adds a client authenticated endpoint to export blog for ghost-backup client only
- allows some additional overrides during import
- allows for an import by file to override locking a user and double hashing the password
closes#8821
- Use semver to do constraint matching
- Use client to generate a caret constraint
- E.g. if the client is 1.1, then the constraint ^1.1.0 will match >=1.1.0 <2.0.0
- Updated tests
refs #8868
The `image-size` library supports now `.ico` files, which means there is no longer need to use the `icojs` library.
- removes unnecessary `icojs` dependency
- refactors `getIconDimensions` fn in blog icon util to fetch image sizes synchronus
- removes unnecessary `getIconDimensions` fn in blog icon validation, as there is no longer need to use different image size fn for different file extensions, and uses `getIconDimensions` from blog util fn instead.
- updates and adds more tests
no issue
- Split routes out from the API app 🎨
- Use the same pattern as the blog app
- General cleanup/unification across all of the `app.js` files
- Split middleware config out from API routes
- Logical groupings make it easier to see WTF is going on 😬
refs #8901
- Adds support for:
```
{{#has number="3"}} // A single number
{{#has number="3, 6, 9"}} // list the numbers you want to match against
{{#has number="nth:3"}} // special syntax for nth item
```
And
```
{{#has index="3"}} // A single number
{{#has index="3, 6, 9"}} // list the numbers you want to match against
{{#has index="nth:3"}} // special syntax for nth item
```
refs #8859
- adds new `configuration/private` endpoint for exposing config that should not be accessible without authentication
- adds `unsplashAPI` to private config
- adds empty `unsplash` config to default settings
no issue
- Upgraded ghost-ignition
- Use debug from ghost-ignition everywhere in the code base
- Remove debug dependency
- Fixed random typo in Gruntfile.js
fixes#8845
- We had a report of weird URLS being output in admin stories view
- This is due to plaintext being incorrectly generated
- In order for a URL to be correct, it would need to already contain the subdirectory
- This line in the post model adds it as well, causing a duplicate
- Hence removing this line is the fix
fixes#8898
- This is a user error, not a system error
- Downgrading to a 4xx status code means it doesn't appear in logs where it shouldn't
- We didn't have a suitable error available so I added UpdateCollisionError with 409 status
refs #5091
- This tiny refactor opens the door for using channel config inside of helpers
- This means that ghost_head, and the next_post/prev_post helpers can be context aware
refs #5091
- occurred to me whilst documenting the custom homepage config, that RSS and pagination
need to be optional
- added a very quick if statement & tests
- needs further refactoring & test improvements
- this will not disable the RSS url output in meta data yet 😔
no issue
- 404 errors clutter up the log files and stdout when developing
- We don't really need these as more than a single line, like other requests
- This is how it worked in LTS
- This is also more consistent with other software (e.g. nginx)
closes#8334
- adds title, image and description to structured data to be rendered as open graph and twitter data.
- if meta title and description for a post exists already, the custom structured data will overwrite those for `og:` and `twitter:` data. `JSON-LD` (Schema.org`) is not affected and will stay the same.
- adds tests
- adds new og and twitter fields to schema incl. migration
closes#8651
- inactive users are suspended users or the owner user on blog setup
- added a check to see if user is inactive in import
- passes all tests
closes#8808
Problem:
- In certain cases, particularly in production mode, errors would be hidden
- E.g. fatal theme errors could not be seen, users instead saw "Failed to lookup view 'error' in views directory"
- This is extremely unhelpful, particularly for people upgrading from 1.0.0 or 1.0.1 where the disqus rule was added
afterwards and modified casper would error
Solution:
- Ensure that we properly setup handlebars when we throw an error
- If engines is not set, set all the view engine related properties
no issue
- add 1.4 database migration to add two new fields to the database (use type text, because of max row size)
- handle global code injection vs. post code injection
- add tests
refs #8700
- if you used a url e.g. /page/2abc/ ghost would interpret the 2 as /page/2/
- these urls should have returned 404, but instead were responding correctly
- this effectively creates duplicate pages
- added a test, but needed a dirty hack to get it to work 😞
- TODO: update casper fixture and use it in channel tests!
no issue
- if you backup your database and you are in the middle of a transaction, the transaction was not fully forwarded
- we were running into a pool error in knex
closes#8793
- 1.3 post excerpt migration
- add 1.3 migration to add `excerpt` to post schema
NOTE:
- knex-migrator relies on the package.json safe version
- so right now Ghost is on 1.2
- the migration script is for 1.3
- if you pull down the PR (or if we merge this PR into master), you have to run `knex-migrator migrate --v 1.3 --force`
- knex-migrator will tell you what you have todo
- Bump dependencies
- knex-migrator@2.1.3
- Soft limit for custom_excerpt
- Extended {{excerpt}} to use custom excerpt
- when a `custom_excerpt` field exists, the `{{excerpt}}` helper will output this and fall back to autogenerated excerpt if not.
- Refactored behaviour of (meta) description
- html tag `<meta name="description" />` for posts, tags and author doesn't get rendered if not provided.
- fallback for `author.bio` removed
- fallback for `tag.description` removed
- structured data and schema.org for `post` context takes the following order to render description fields:
1. custom excerpt
2. meta description
3. automated excerpt (50 words)
- updated and added tests to reflect the changes
closes#8781
- when the ownership get's transferred, the id of the new owner is not '1' anymore
- we previously added a database rule, which signalises if the blog is setup or not, see 827aa15757 (diff-7a2fe80302d7d6bf67f97cdccef1f71fR542)
- this database rule is based on the owner id being '1', which is wrong when you transfer ownership
- we should keep in mind, that the owner id being '1' is only the default Ghost setup, but it can change
- blog is setup if the owner is locked
closes#8605
- This file has already been moved, might as well get the rename out of the way
- Especially as we don't migrate clients - everyone will now need to make just one change
refs #8756
- there was a bug in one of the last LTS releases, which produced duplicated attached roles to users
- we want to prevent that on import and take the latest created based on the autoincrement id
refs https://github.com/TryGhost/Ghost/issues/8757
- remove mobiledoc parsing, it's reliance on SimpleDom makes it too
fragile when dealing with the unconstrained user-entered HTML that is
allowed in markdown
closes#8760
- we have to remember the old post id's when migrating a blog from LTS to 1.0
- otherwise we would break disqus comments, because they rely on the post id
- this should fix the discovered situation
closes https://github.com/TryGhost/Ghost/issues/8743
- the serializer in our mobiledoc renderer didn't have the list of non-closing HTML tags passed in which meant that tags such as `<br>` in the markdown HTML output were being re-serialized to `<br></br>` which is invalid HTML which (at least Chrome) then attempts to fix by rendering it as `<br><br>` instead
- the elements with incorrect rendering that may result in unwanted "fixing" by browsers are `AREA`, `BASE`, `BR`, `COL`, `COMMAND`, `EMBED`, `HR`, `IMG`, `INPUT`, `KEYGEN`, `LINK`, `META`, `PARAM`, `SOURCE`, `TRACK`, and `WBR`
no issue
- this is usually an edge case, but i investigated because i thought that the importer is broken
- the importer logic is build like this:
- it creates a transaction
- this transactions runs through:
- beforeImport
- doImport
- afterImport
- afterImport corrects user references and if a user could not be imported, we have to protect that
NOTE: we could create two transactions to be more correct, but building this had no priority because of edge cases only
having two transactions would solve: you first add the data (error or success), then you correct the data
- usually a user can be always imported (!), but there are a few edge cases (e.g. multiple roles attached)
no issue
- if you upload a huge import file, parallel operations can throw errors e.g. lock wait exceeds
- this can happen if multiple transactions run in parallel
- there is no need to run:
1. the removal of active tokens on import, because imported users have no active session
2. rescheduling logic on timezone, because importing scheduled posts works out of the box via the model layer (if a published date is detected and it's in the future, the post get's scheduled)
closes#8645, closes#8710
- locked users were once part of the category "active users", but were moved to the inactive category
-> we have added a protection of not being able to edit yourself when you are either suspended or locked
- but they are not really active users, they are restricted, because they have no access to the admin panel
- support three categories: active, inactive, restricted
* - revert restricted states
- instead, update permission layer: fallback to `all` by default, because you are able to serve any user status
- add more tests
- ATTENTION: there is a behaviour change, that a blog owner's author page can be served before setting up the blog, see conversation on slack
-> LTS serves 404
-> 1.0 would serve 200
closes#8691
There was a condition added when i've refactored the importer.
> if (models.User.isOwnerUser(obj[key])) {
This condition is absolutely wrong! If you import an owner user, this owner user get's imported as administrator. But the original owner user id reference must be updated as well, so that the reference points to the new administrator id ✌🏻
no issue
- if you delete an active user, Ghost logs an error message (Ghost does not crash!)
- but the event logic is not triggered, that means we don't delete the users tokens
- token deletion happens on: suspend a user and delete a user
no issue
- if you destroy a user with an unknown user id, Ghost would crash
- because `userModel.hasRole` is undefined
- there is actually a bigger underlying architectual problem:
- the permission check should rely on an existing user
- so there should be a first api layer, which 1. validates (this code exists) and 2. ensures that requested database id's exist
- but this requires a bigger refactoring
closes#8601
- This makes sure that when you do an import, you still get the LATEST
default settings for labs. Even if you had a different value before.
- LTS -> 1.0 is an upgrade, and Public API should be on by default, even if you
had deliberately turned it off before.
- Cheeky test added
no issue
Seems like we forgot to update the AMP template to reflect our image helper changes.
- Replaces `{{image}}` helper with `{{img_url}}` for `feature_image`
- Removes `{{meta_description}}` helper
refs #8620
Adds a new Ghost Author user, which is the author of the new welcome blog posts. The user is set to active, so the author slug works (otherwise it would render a 404, when user is suspended). Furthermore, there's one little fix in the user model, which was checking only for `active` user to decide the signup or setup process for the UI. Adding one more conditional to check if the found active user is also the owner, prevents to get redirected to sign in.
closes#8601
- this doesn't take the feature out of beta, but does enable it by default
- no need to enable the public api in the test anymore
- because public api is enabled by default
closes#8565
- isPasswordCorrect fn returns a specific error, which we simply forward
- no need to wrap a custom error into a new custom error
- the rule is always: if you are using a Ghost unit/function, you can expect that this unit returns a custom error
closes#8568
- use our `urlJoin` util to concatenate the URL (not the query part of it, as this is not supported in `urlJoin`) and to prevent possible missing or double slashes, as `config.apiUrl` could be with or without trailing slash
closes#8562
- before we create our model fixtures, we assign a `published_at` property with a difference of 1 second for each blog post, so the `prev_post` and `next_post` helpers work correctly
closes#8542
- updates default post fixtures
- adds default logo and cover images to settings fixtures
- update tests due to coupling to dev/prod fixtures
refs #8530
- display theme name for non-fatal errors
- minor wording tweak
- remove duplicate lodash require
- use the shiny new calculated property syntax
refs #8222
- differentiate between errors and fatal errors
- use gscan errors in theme middleware
- Adds a new `error()` method to `currentActiveTheme` constructor which will return the errors we receive from gscan
- In middleware, if a theme couldn't be activated because it's invalid, we'll fetch the erros and send them to our error handler. We also use a new property `hideStack` to control, if the stack (in dev mode and if available) should be shown or the gscan errors (in prod mode, or in dev if no stack error)
- In our error handler we use this conditional to send a new property `gscan` to our error theme
- In `error.hbs` we'll iterate through possible `gscan` error objects and render them.
- remove stack printing
- stack for theme developers in development mode doesn't make sense
- stack in production doesn't make sense
- the stack is usually hard to read
- if you are developer you can read the error stack on the server log
- utils.packages: transform native error into Ghost error
- use `onlyFatalErrors` for gscan format and differeniate fatal errors vo.2
- optimise bootstrap error handling
- transform theme is missing into an error
- add new translation key
- show html tags for error.hbs template: rule
refs #8141
- update importer for LTS fields
- optimise for LTS export fixtures
- add image/language test for LTS import
- ensure post image is mapped to feature_image
- create mobiledoc values from markdown and html
- if mobiledoc is null, use markdown or html to create a mobiledoc markdown card
- update import mapping to use locale
- defaultLang in settings now maps to default_locale
- language for post and user models now maps to locale
- posts are not always loaded in correct same order so we select the posts we want to validate
- ensure if mobiledoc field is not in export we can still import from markdown
- map last_login to last_seen
- for users the importer maps last_login to last_seen
- add warning for legacyActiveTheme
- for export with old activeTheme key provide a warning that theme is not installed
- add importer test for LTS user long email
- add a test for LTS export where email address could be longer than alpha
- fix for importer date tests on mysql
- use valueOf in moment to compare times stored in different formats
- ignore warnings for not found settings in import
- use a flag to ignore NotFound Entries for settings during import
