refs #9601
- refactor architecture of routing so you can define a channel
- a channel is a different way of looking at your posts (a view)
- a channel does not change the url of a resource
Example channel
```
routes:
/worldcup-2018-russia/:
controller: channel
filter: tag:football18
data: tag.football18
```
- added ability to redirect resources to a channel/static route
- support templates for channels
- ensure we still support static routes (e.g. /about/: home)
- ensure pagination + rss works out of the box
refs #9601
- the home.hbs behaviour for the index collection (`/`) is hardcoded in Ghost
- we would like to migrate all existing routes.yaml files
- we only replace the file if the contents of the routes.yaml file equals the old routes.yaml format (with home.hbs as template)
- updated README of settings folder
- if we don't remove the home.hbs template from the default routes.yaml file, home.hbs will be rendered for any page of the index collection
- the backwards compatible behaviour was different
- only render home.hbs for page 1
- remember: the default routes.yaml file reflects how Ghost was working without dynamic routing
no issue
- this mock eat already too much of my/our time
- the idea of adding a knex mock was definitely a failed approach/try
- it's too much to maintaince and have not found a module which does this already
- we have to support any query format
- this is too crazy
- the idea was to use the knex mock for model unit tests, because if we want to unit test models we have to
run through bookshelf, because the whole model layer depends on bookshelf e.g. events
- for now we simply use the real database
- we could use the sqlite3 memory mode, but that would mean every unit test runs on sqlite3
- something to consider for later e.g. run unit tests on one matrix
- run the rest on another matrix for sqlite + mysql
no issue
- from now on: you have to manually reconfigure your slack hook after importing your data
- we were running into trouble that Ghost had import slack hooks, because it can happen very fast
that you are importing someone's slack hook
refs #9601
### Dynamic Routing
This is the beta version of dynamic routing.
- we had a initial implementation of "channels" available in the codebase
- we have removed and moved this implementation
- there is now a centralised place for dynamic routing - server/services/routing
- each routing component is represented by a router type e.g. collections, routes, static pages, taxonomies, rss, preview of posts
- keep as much as possible logic of routing helpers, middlewares and controllers
- ensure test coverage
- connect all the things together
- yaml file + validation
- routing + routers
- url service
- sitemaps
- url access
- deeper implementation of yaml validations
- e.g. hard require slashes
- ensure routing hierarchy/order
- e.g. you enable the subscriber app
- you have a custom static page, which lives under the same slug /subscribe
- static pages are stronger than apps
- e.g. the first collection owns the post it has filtered
- a post cannot live in two collections
- ensure apps are still working and hook into the routers layer (or better said: and register in the routing service)
- put as much as possible comments to the code base for better understanding
- ensure a clean debug log
- ensure we can unmount routes
- e.g. you have a collection permalink of /:slug/ represented by {globals.permalink}
- and you change the permalink in the admin to dated permalink
- the express route get's refreshed from /:slug/ to /:year/:month/:day/:slug/
- unmount without server restart, yey
- ensure we are backwards compatible
- e.g. render home.hbs for collection index if collection route is /
- ensure you can access your configured permalink from the settings table with {globals.permalink}
### Render 503 if url service did not finish
- return 503 if the url service has not finished generating the resource urls
### Rewrite sitemaps
- we have rewritten the sitemaps "service", because the url generator does no longer happen on runtime
- we generate all urls on bootstrap
- the sitemaps service will consume created resource and router urls
- these urls will be shown on the xml pages
- we listen on url events
- we listen on router events
- we no longer have to fetch the resources, which is nice
- the urlservice pre-fetches resources and emits their urls
- the urlservice is the only component who knows which urls are valid
- i made some ES6 adaptions
- we keep the caching logic -> only regenerate xml if there is a change
- updated tests
- checked test coverage (100%)
### Re-work usage of Url utility
- replace all usages of `urlService.utils.urlFor` by `urlService.getByResourceId`
- only for resources e.g. post, author, tag
- this is important, because with dynamic routing we no longer create static urls based on the settings permalink on runtime
- adapt url utility
- adapt tests
no issue
- we sanitise any incoming slug on the model layer e.g uppercase -> lowercase
- and when importing e.g. an uppercase slug, the importer was trying to compare the uppercase slug with the sanitised slug
refs #9601
- replace raw knex queries by Bookshelf queries
- optimise lot's of test setups, so we don't experience a massive slow down in the test run
- this has troubled in the past e.g. with normalisation, any custom model logic - the test env always had to simulate things
- there are for sure thousands things which can be optimised now, but because of time, we do them step by step
- this is especially important for the url service (https://github.com/TryGhost/Ghost/issues/9601), because we have to ensure that inserting/updating/removing resources will trigger model events
`grunt test-all` with SQLite finishes in 2,5-3min. (on master: 1-2min)
`grunt test-all` with MySQL finishes in 4min. (on master: 3min)
**NOTE: We want to move as much as possible routing and integration tests to unit tests. This will speed up the test run again.** See #9342. But we need to find time for that. Any help is welcome!
no issue
- replaced token creation by `lib.common.security`
- added unit tests for adding invites
- allow a different invite status for internal access
no issue
- support more cases
- e.g. multiple where matches
- @TODO
- take time to look for NPM module, which does this already
- test sqlite3 :memory: mode again
closes#9528
These code changes introduce a YAML parser which will load and parse YAML files from the `/content/settings` directory. There are three major parts involved:
1. `ensure-settings.js`: this fn takes care that on bootstrap, the supported files are present in the `/content/settings` directory. If the files are not present, they get copied back from our default files. The default files to copy from are located in `core/server/services/settings`.
2. `loader.js`: the settings loader reads the requested `yaml` file from the disk and passes it to the yaml parser, which returns a `json` object of the file. The settings loader throws an error, if the file is not accessible, e. g. because of permission errors.
3. `yaml-parser`: gets passed a `yaml` file and returns a `json` object. If the file is not parseable, it returns a clear error that contains the information, what and where the parsing error occurred (e. g. line number and reason).
- added a `get()` fn to settings services, that returns the settings object that's asked for. e. g. `settings.get('routes').then(()...` will return the `routes` settings.
- added a `getAll()` fn to settings services, that returns all available settings in an object. The object looks like: `{routes: {routes: {}, collections: {}, resources: {}}, globals: {value: {}}`, assuming that we have to supported settings `routes` and `globals`.
Further additions:
- config `contentPath` for `settings`
- config overrides for default `yaml` files location in `/core/server/services/settings`
**Important**: These code changes are in preparation for Dynamic Routing and not yet used. The process of copying the supported `yaml` files (in this first step, the `routes.yaml` file) is not yet activated.
no issue
- required for model events
- otherwise you won't receive a full data set
- in worst case you have to re-fetch the post
- required for the url service
- the url service always needs relations (authors,tags) to be able to generate the url properly
@IMPORTANT
- no API change, we still return what you are asking for
- we first edit/add the resource
- then we fetch the data with the API options
- @TODO: this can be optimised and will improve performance
picking/selecting it from the insert/update response
- this is an internal change
closes#9547
- you setup a blog with the following owner:
- email: test@ghost.org
- name: test
- slug: test
- now you import a JSON db file, which holds the exact same owner
- this owner won't be imported, because it's a duplicate
- but the slug is different (!)
- the importer tries to find a matching existing user, but won't find anything
- the importer then send an empty authors array `post.authors=[]` into the model layer
- this is not allowed -> this would mean, you are actively trying to unset all authors
no issue
This PR adds the server side logic for multiple authors. This adds the ability to add multiple authors per post. We keep and support single authors (maybe till the next major - this is still in discussion)
### key notes
- `authors` are not fetched by default, only if we need them
- the migration script iterates over all posts and figures out if an author_id is valid and exists (in master we can add invalid author_id's) and then adds the relation (falls back to owner if invalid)
- ~~i had to push a fork of bookshelf to npm because we currently can't bump bookshelf + the two bugs i discovered are anyway not yet merged (https://github.com/kirrg001/bookshelf/commits/master)~~ replaced by new bookshelf release
- the implementation of single & multiple authors lives in a single place (introduction of a new concept: model relation)
- if you destroy an author, we keep the behaviour for now -> remove all posts where the primary author id matches. furthermore, remove all relations in posts_authors (e.g. secondary author)
- we make re-use of the `excludeAttrs` concept which was invented in the contributors PR (to protect editing authors as author/contributor role) -> i've added a clear todo that we need a logic to make a diff of the target relation -> both for tags and authors
- `authors` helper available (same as `tags` helper)
- `primary_author` computed field available
- `primary_author` functionality available (same as `primary_tag` e.g. permalinks, prev/next helper etc)
closes#9085
Fixes an issue, where the client sets image properties to `""` after deleting the image. This causes problems with the query filter (see https://github.com/TryGhost/GQL/issues/24), as they have to be `null`.
Added a check in the model layer saving method to set value to `null`, when the property is empty.
Affected models and properties:
- `posts`:
- `feature_image`
- `og_image`
- `twitter_image`
- `users`:
- `profile_image`
- `cover_image`
- `tags`:
- `feature_image`
no issue
- extended functionality
- the knex mock simply parses the sql statements and serves data from memory
- i've tested the memory mode of sqlite, but could not get it working
- but maybe for the future to test again
no issue
- the handling here was not correct
- if you've passed no mobiledoc, it wasn't adding mobiledoc and an undefined html value
- we need a default mobiledoc+html value in case you don't pass the values within the test cases
no issue
- just discovered that we had confusing function names in our test utility
- e.g. `posts` -> default posts from the data generator
- e.g. `users` -> extra users not from our data generator
- now:
- e.g. `posts` -> default posts from the data generator
- e.g. `users` -> default users from the data generator
- e.g. `users:extra` -> extra users not from our data generator
no issue
- replace logic for preparing nested tags
- if you have nested tags in your file, we won't update or update the target tag
- we simply would like to add the relationship to the database
- use same approach as base class
- add `posts_tags` to target post model
- update identifiers
- insert relation by foreign key `tag_id`
- bump bookshelf-relations to 0.1.10
no issue
- change behaviour from updating user references after the actual import to update the user reference before the actual import
- updating user references after the import is way less case intense
- that was the initial decision for updating the references afterwards
- but that does not play well with adding nested relations by identifier
- the refactoring is required for multiple authors
- if we e.g. store invalid author id's, we won't be able to add a belongs-to-many relation for multiple authors
- bookshelf-relations is generic and always tries to find a matching target before attching a model
- invalid user references won't work anymore
- this change has a very good side affect
- 17mb takes on master ~1,5seconds
- on this branch it takes ~45seconds
- also the memory usage is way lower and stabler
- 40mb takes 1,6s (times out on master)
refs https://github.com/TryGhost/Ghost/issues/3658
- the `validateSchema` helper was a bit broken
- if you add a user without email, you will receive a database error
- but the validation error should catch that email is passed with null
- it was broken, because:
- A: it called `toJSON` -> this can remove properties from the output (e.g. password)
- B: we only validated fields, which were part of the JSON data (model.hasOwnProperty)
- we now differentiate between schema validation for update and insert
- fixed one broken import test
- if you import a post without a status, it should not error
- it falls back to the default value
- removed user model `onValidate`
- the user model added a custom implementation of `onValidate`, because of a bug which we experienced (see https://github.com/TryGhost/Ghost/issues/3638)
- with the refactoring this is no longer required - we only validate fields which have changed when updating resources
- also, removed extra safe catch when logging in (no longer needed - unit tested)
- add lot's of unit tests to proof the code change
- always call the base class, except you have a good reason
no issue
- added https://github.com/colonyamerican/mock-knex as dev dependency
- the mock serves our data generator test data by default
- but you can define your own if you want
- we need a proper mock for unit testing
- we should not mock bookshelf if possible, otherwise we can't test event flows
closes#9445
- redirects all asset requests if https is configured (theme, core, images)
- re-use and extend our url-redirect middleware
- add proper integration tests for our express site app (no db interaction, component testing required for such important use cases)
- i added some more general tests
- should avoid mixed content warnings in the browser
closes#9314
* added fixtures for contributor role
* update post api tests to prevent contributor publishing post
* update permissible function in role/user model
* fix additional author code in invites
* update contributor role migration for knex-migrator v3
* fix paths in contrib migration
* ensure contributors can't edit or delete published posts, fix routing tests [ci skip]
* update db fixtures hash
* strip tags from post if contributor
* cleanup post permissible function
* excludedAttrs to ignore tag updates for now (might be removed later)
* ensure contributors can't edit another's post
* migration script for 1.21
no issue
- reported in slack (https://ghost.slack.com/files/U8QV8DXQB/F8TSBQ532/image.png)
- do not expose old release notification
- e.g. you are on 1.20.0
- you receive a notification for 1.20.1 to update
- you update to 1.20.1
- ensure we protect exposing the release notification (compare against blog version)
- protect against wrong formats
- @TODO: the notifications could store a `version` property
- by that we could use `notification.version` and don't have to match the version in the message
refs #5345, refs #3801
- Blog localisation
- default is `en` (English)
- you can change the language code in the admin panel, see https://github.com/TryGhost/Ghost-Admin/pull/703
- blog behaviour changes depending on the language e.g. date helper format
- theme translation get's loaded if available depending on the language setting
- falls back to english if not available
- Theme translation
- complete automatic translation of Ghost's frontend for site visitors (themes, etc.), to quickly deploy a site in a non-English language
- added {{t}} and {{lang}} helper
- no backend or admin panel translations (!)
- easily readable translation keys - very simple translation
- server restart required when adding new language files or changing existing files in the theme
- no language code validation for now (will be added soon)
- a full theme translation requires to translate Ghost core templates (e.g. subscriber form)
- when activating a different theme, theme translations are auto re-loaded
- when switching language of blog, theme translations are auto re-loaded
- Bump gscan to version 1.3.0 to support more known helpers
**Documentation can be found at https://themes.ghost.org/v1.20.0/docs/i18n.**
no issue
> (node:63849) Warning: Possible EventEmitter memory leak detected. 101 settings.edited listeners added. Use emitter.setMaxListeners() to increase limit
- the settings cache was initialised per test
- it registered the model events over and over again
- add a simple shutdown function, which can be called from the test env
refs #9178
- continue with killing our global utils folder
- i haven't found any better naming for lib/promise
- so, require single files for now
- instead of doing `promiseLib = require('../lib/promise')`
- we can optimise the requires later
refs #9178
- we have to take care that we don't end up in circular dependencies
- e.g. API requires UrlService and UrlService needs to require the API (for requesting data)
- update the references
- we would like to get rid of the utils folder, this is/was the most complicated change
refs #9178
- move express apps to one place (called `web`)
- requires https://github.com/TryGhost/Ghost-Admin/pull/923
- any further improvements are not part of this PR
- this PR just moves the files and ensures the paths are up-to-date
refs #9178
- the first test will start the ghost server
- all following test will reuse the server and only truncate the database tables
- you can use `forceStart` if you need
- we have to ensure that reusing the server requires us to
- e.g. reload the settings
- e.g. reload active theme
- remove fork functionality (you can use `forceStart: true`)
- no test has changed, just the way we start the server
no issue
- adapt major changes of knex-migrator v3
- adapt migration scripts, simplify and add `down` (rollback) hook if possible
- clear Ghost cache after init hook (because of `knex-migrator migrate --init`)
- ensure db migrations work with the CLI
- updated troubleshooting guide (https://docs.ghost.org/v1/docs/troubleshooting#section-task-execute-is-not-a-function)
**For development only: Please ensure you run `npm i -g knex-migrator@latest` to update your global installation to v3. We always prefer the local installation, but v3 has modified and added binaries.**
no issue
- https://github.com/mochajs/mocha/blob/master/CHANGELOG.md#400--2017-10-02
- the new `--exit` flag might be interesting at some point
> In Mocha v3.0.0 and newer, returning a Promise and calling done() will result in an exception.
- adapt teardown/setup test utility
- adapt other mixed usages of callback && Promise usage
no issue
Support for http://resthooks.org style webhooks that can be used with Zapier triggers. This can currently be used in two ways:
a) adding a webhook record to the DB manually
b) using the API with password auth and POSTing to /webhooks/ (this is private API so not documented)
⚠️ only _https_ URLs are supported in the webhook `target_url` field 🚨
- add `webhooks` table to store event names and target urls
- add `POST` and `DELETE` endpoints for `/webhooks/`
- configure `subscribers.added` and `subscribers.deleted` events to trigger registered webhooks
refs #9192, #5091
- changed channels to use our new base class
- keep the flexible structure, so that channels can be reloaded
- I had to move the router into the route service otherwise we get circular dependencies
- Don't _really_ want to keep it like this - need a way to define base classes as shared
refs #9192, refs #5091, refs #9178
- moved channels from controllers to a service
- split out the parent router from the remaining individual router logic
- moved the tests to match
refs #9192, refs #5091
- Using a class allows for easy shared logic
- Loading is designed to work from config right now, but could be DB driven, etc
- Provided configuration can be simplified and extended in the constructor / class methods
- Update tests, move custom assertions to utils
refs #9178
* Add eslint deps, remove old lint deps
* Add eslint config, remove old lint configs
* Config for server and tests are different
* Tweaked rules to suit us
* Fix linting in codebase - lots of indent changes.
* Fix a real broken test
refs #8995
- move the getClient lookup from ghost_head into middleware
- use res.locals to keep track of the information (res.locals.client)
- make the middleware global to all frontend routes
- ghost_head: get locals from options.data not this (!)
- adapt lot's of tests
refs #5091
- Move all of the code to do with handling channels into one folder
- Still keeping all the shared/simlar code for rendering etc inside weird
frontend folder until I am sure what this will look like
refs #5091
- remove the use of functions
- remove unnecessary quotes from tag filter
- move channel config to be a JSOn file called config.channels.json
- accept external config
- new channelUtils for tests
- remove channelConfig.get
- refactor so tests work as expected
- refactor away duplicate 'name' value
closes#8668, refs #8920
- Updated tests to include internal tags
- Tests had no example of an internal tag
- Need this to show that the new filtering works as expected
- primary_tag is a calculated field
- This ensures that we can alias the field to equivalent logic in API filters
- By replacing primary_tag by a lookup based on a tag which has order 0
- bump ghost-gql to 0.0.8
**NOTE:**
Until GQL is refactored, there are limitations on what else can be filtered when using primary_tag in a filter e.g. it wont be possible to do a filter based on primary_tag AND/OR other tag filters.
no issue
* Comment current state of toJSON for user model
- currently the user model does not return the email if the context is app/external/public OR if there is no context object at all
- i am not 100% sure why if there is no context we should not return the email address
- i think no context means internal access
- maybe change this condition cc @ErisDS
* Extend our access rules plugin
- we already have a instance method to determine which context is used
- this relies on passing options into `.forge` - but we almost never pass the context into the forge call
- added @TODO
- provide another static method to determine the context based on the options object passed from outside
* Use the new static function for existing code
* Add comment where the external context is used
* Remove certain fields from a public request (User model only)
* Tests: support `checkResponse` for a public request
- start with an optional option pattern
- i would love to get rid of checkResponse('user', null, null, null)
- still support old style for now
- a resoure can define the default response fields and public response fields
* Tests: adapt public api test
* Tests: adapt api user test
- use new option pattern for `checkResponse`
- eww null, null, null, null....
* Revert the usage of the access rules plugin
refs #9043
- this is preparation for adding Author-specific tests later
- the changes the posts_spec.js, so that all the tests are inside an "As Owner" describe block, similar to the users_spec.js
- Added new util for creating a specific post
- This will make it easier to do routing tests on the post model in future
- Our `index.js` file in test/utils really needs a bit of love 🙈
- Also added all the framework for author role tests in post_spec.js
- Added a single test, showing we can edit posts, including author_id
refs #9028
- if you upload a redirects file and a redirects file exists already, we backup this file to `data/redirects-YYYY-MM-DD-HH-mm-ss.json`
- decrease chance of random test failures by not comparing date format with seconds
refs #9028
- add two new endpoints for uploading/downloading the redirects (file based)
- reload/re-register redirects on runtime
- migration for 1.9 to add permissions for redirects download/upload
no issue
- use latest casper in test fixtures
- never ever use the root content folder for tests
- if we start/fork Ghost for the tests, we use a tmp folder
- this change is required to for an upcoming PR (#9029)
- i've added a TODO to create a helper fn for stopping the ghost server, so we can cleanup the tmp folder
* Care about TODO's in our channels spec
- add the 1.4 compatible casper theme to fixtures
- so as soon as you start Ghost, the test env will provide the content folder in /tmp something with the activated latest default casper and the 1.4 compatible old casper
- there are tests which tests different logici e.g. pagination
- therefor we need a different theme, we are simply using our 1.4 casper
closes#8963
- if an LTS export is imported into a 1.0 blog, then the 1.0 blog is
exported and re-imported into another 1.0 blog, any post ids from the
lts import were getting clobbered. This only saves the post id if the
amp field does not already exist
- add failing test that passes w/change
no issue
- adds a ghost-backup client
- adds a client authenticated endpoint to export blog for ghost-backup client only
- allows some additional overrides during import
- allows for an import by file to override locking a user and double hashing the password
no issue
- Upgraded ghost-ignition
- Use debug from ghost-ignition everywhere in the code base
- Remove debug dependency
- Fixed random typo in Gruntfile.js
closes#8793
- 1.3 post excerpt migration
- add 1.3 migration to add `excerpt` to post schema
NOTE:
- knex-migrator relies on the package.json safe version
- so right now Ghost is on 1.2
- the migration script is for 1.3
- if you pull down the PR (or if we merge this PR into master), you have to run `knex-migrator migrate --v 1.3 --force`
- knex-migrator will tell you what you have todo
- Bump dependencies
- knex-migrator@2.1.3
- Soft limit for custom_excerpt
- Extended {{excerpt}} to use custom excerpt
- when a `custom_excerpt` field exists, the `{{excerpt}}` helper will output this and fall back to autogenerated excerpt if not.
- Refactored behaviour of (meta) description
- html tag `<meta name="description" />` for posts, tags and author doesn't get rendered if not provided.
- fallback for `author.bio` removed
- fallback for `tag.description` removed
- structured data and schema.org for `post` context takes the following order to render description fields:
1. custom excerpt
2. meta description
3. automated excerpt (50 words)
- updated and added tests to reflect the changes
closes#8781
- when the ownership get's transferred, the id of the new owner is not '1' anymore
- we previously added a database rule, which signalises if the blog is setup or not, see 827aa15757 (diff-7a2fe80302d7d6bf67f97cdccef1f71fR542)
- this database rule is based on the owner id being '1', which is wrong when you transfer ownership
- we should keep in mind, that the owner id being '1' is only the default Ghost setup, but it can change
- blog is setup if the owner is locked
refs #8756
- there was a bug in one of the last LTS releases, which produced duplicated attached roles to users
- we want to prevent that on import and take the latest created based on the autoincrement id
closes#8760
- we have to remember the old post id's when migrating a blog from LTS to 1.0
- otherwise we would break disqus comments, because they rely on the post id
- this should fix the discovered situation
closes#8645, closes#8710
- locked users were once part of the category "active users", but were moved to the inactive category
-> we have added a protection of not being able to edit yourself when you are either suspended or locked
- but they are not really active users, they are restricted, because they have no access to the admin panel
- support three categories: active, inactive, restricted
* - revert restricted states
- instead, update permission layer: fallback to `all` by default, because you are able to serve any user status
- add more tests
- ATTENTION: there is a behaviour change, that a blog owner's author page can be served before setting up the blog, see conversation on slack
-> LTS serves 404
-> 1.0 would serve 200
closes#8691
There was a condition added when i've refactored the importer.
> if (models.User.isOwnerUser(obj[key])) {
This condition is absolutely wrong! If you import an owner user, this owner user get's imported as administrator. But the original owner user id reference must be updated as well, so that the reference points to the new administrator id ✌🏻
refs #8222
- differentiate between errors and fatal errors
- use gscan errors in theme middleware
- Adds a new `error()` method to `currentActiveTheme` constructor which will return the errors we receive from gscan
- In middleware, if a theme couldn't be activated because it's invalid, we'll fetch the erros and send them to our error handler. We also use a new property `hideStack` to control, if the stack (in dev mode and if available) should be shown or the gscan errors (in prod mode, or in dev if no stack error)
- In our error handler we use this conditional to send a new property `gscan` to our error theme
- In `error.hbs` we'll iterate through possible `gscan` error objects and render them.
- remove stack printing
- stack for theme developers in development mode doesn't make sense
- stack in production doesn't make sense
- the stack is usually hard to read
- if you are developer you can read the error stack on the server log
- utils.packages: transform native error into Ghost error
- use `onlyFatalErrors` for gscan format and differeniate fatal errors vo.2
- optimise bootstrap error handling
- transform theme is missing into an error
- add new translation key
- show html tags for error.hbs template: rule
refs #8141
- update importer for LTS fields
- optimise for LTS export fixtures
- add image/language test for LTS import
- ensure post image is mapped to feature_image
- create mobiledoc values from markdown and html
- if mobiledoc is null, use markdown or html to create a mobiledoc markdown card
- update import mapping to use locale
- defaultLang in settings now maps to default_locale
- language for post and user models now maps to locale
- posts are not always loaded in correct same order so we select the posts we want to validate
- ensure if mobiledoc field is not in export we can still import from markdown
- map last_login to last_seen
- for users the importer maps last_login to last_seen
- add warning for legacyActiveTheme
- for export with old activeTheme key provide a warning that theme is not installed
- add importer test for LTS user long email
- add a test for LTS export where email address could be longer than alpha
- fix for importer date tests on mysql
- use valueOf in moment to compare times stored in different formats
- ignore warnings for not found settings in import
- use a flag to ignore NotFound Entries for settings during import
no issue
- this PR references indirecty to https://github.com/TryGhost/Ghost/pull/8437
- i would like to have the settings change already in place before we release the beta
- the i18n feature is able to change the locale of Ghost
- most i18n libraries use locale
- adding/changing settings doesn't require a migration file, but it can make the database a bit messy (because you can end up with default_locale and lang)
- furthermore we agreed that the default locale for Ghost should be simply `en`, not `en_US` or `en_GB`
closes#8479
- removes `markdown` field from schema
- removes `legacyMarkdown` converter
- updates tests to work with `mobiledoc` field instead of `markdown` and adapt for mobiledoc HTML output where necessary
refs #5422
- we can support null titles after this PR if we want
- user model: fix getAuthorRole
- user model: support adding roles by name
- we support this for roles as well, this makes it easier when importing related user roles (because usually roles already exists in the database and the related id's are wrong e.g. roles_users)
- base model: support for null created_at or updated_at values
- post or tag slugs are always safe strings
- enable an import of a null slug, no need to crash or to cover this on import layer
- add new DataImporter logic
- uses a class inheritance mechanism to achieve an easier readability and maintenance
- schema validation (happens on model layer) was ignored
- allow to import unknown user id's (see https://github.com/TryGhost/Ghost/issues/8365)
- most of the duplication handling happens on model layer (we can use the power of unique fields and errors from the database)
- the import is splitted into three steps:
- beforeImport
--> prepares the data to import, sorts out relations (roles, tags), detects fields (for LTS)
- doImport
--> does the actual import
- afterImport
--> updates the data after successful import e.g. update all user reference fields e.g. published_by (compares the imported data with the current state of the database)
- import images: markdown can be null
- show error message when json handler can't parse file
- do not request gravatar if email is null
- return problems/warnings after successful import
- optimise warnings in importer
- do not return warnings for role duplications, no helpful information
- error handler: return context information of error
- we show the affected json entries as one line in the UI
- show warning for: detected duplicated tag
- schema validation: fix valueMustBeBoolean translation
- remove context property from json parse error
refs https://github.com/TryGhost/Ghost-Admin/pull/690, closes#1501, closes#2093, closes#4592, closes#4627, closes#4659, closes#5039, closes#5237, closes#5587, closes#5625, closes#5632, closes#5822, closes#5939, closes#6840, closes#7183, closes#7536
- replace custom showdown fork with markdown-it
- swaps showdown for markdown-it when rendering markdown
- match existing header ID behaviour
- allow headers without a space after the #s
- add duplicate header ID handling
- remove legacy markdown spec
- move markdown-it setup into markdown-converter util
- update mobiledoc specs to match markdown-it newline behaviour
- update data-generator HTML to match markdown-it newline behaviour
- fix Post "converts html to plaintext" test
- update rss spec to match markdown-it newline behaviour
- close almost all related showdown bugs
closes#8354
- i thought about transforming scheduled posts into drafts on export, but this has two disadvantages:
1. existing exports with scheduled posts won't import
2. if you schedule a post for next week and you export/import earlier, the post is back to draft
- by this we ensure that we can simply import the post back to a scheduled post
- if the published_at is already in the past, the scheduler will care and instantly publish the post
* 🙀 change database schema for images
- rename user/post/tag images
- contains all the required changes from the schema change
* Refactor helper/meta data
- rename cover to cover_image
- also rename default settings to match the pattern
- rename image to profile_image for user
- rename image to feature_image for tags/posts
* {{image}} >>> {{img_url}}
- rename
- change the functionality
- attr is required
- e.g. {{img_url feature_image}}
* gscan 1.0.0
- update yarn.lock
* Update casper reference: 1.0-changes
- see 5487b4da8d
refs #8111
- Ghost returns now all (active+none active) users by default
- protect login with suspended status
- test permissions and add extra protection for suspending myself
- if a user is suspended and tries to activate himself, he won't be able to proceed the login to get a new token
refs #8093✨ Add activate theme permission
- add permission to activate themes
- update tests
- also: update tests for invites
TODO: change how the active theme setting is updated to reduce extra permissions
✨ Move theme validation to gscan
- add a new gscan validation method and use it for upload
- update activate endpoint to do validation also using gscan
- change to using SettingsModel instead of API so that we don't call validation or permissions on the settings API
- remove validation from the settings model
- remove the old validation function
- add new invalid theme message to translations & remove a bunch of theme validation related unused keys
📖 Planned changes
🚨 Tests for theme activation API endpoint
🐛 Don't allow deleting the active theme
🚫 Prevent activeTheme being set via settings API
- We want to control how this happens in future.
- We still want to store the information in settings, via the model.
- We just don't want to be able to change this info via the settings edit endpoint
🐛✨ Fix warnings for uploads & add for activations
- warnings for uploads were broken in f8b498d
- fix the response + adds tests to cover that warnings are correctly returned
- add the same response to activations + more tests
- activations now return a single theme object - the theme that was activated + any warnings
🎨 Improve how we generate theme API responses
- remove the requirement to pass in the active theme!
- move this to a specialist function, away from the list
🎨 Do not load gscan on boot
* 😱🚀🎨 tests: use truncate instead of delete the database
refs #7718, refs #7470
- should bring massive speed improvement
- could also fix random test failures (e.g. sqlite database is busy)
* gruntfile: add knex-migrator command in test-setup
no issue
- browse will now include the correct activated theme again
- PUT /theme/:name/activate will activate a theme
- tests now read from a temp directory not content/themes
- all tests check errors and responses
no issue
🎨 Switch themes API to use config.availableThemes
- this gets rid of the only places where settings.availableThemes are used
🔥 Get rid of settings.availableThemes
- this is no longer used anywhere
- also get rid of every related call to updateSettingsCache
🔥 Replace config.availableThemes with theme cache
- Creates a tailor-made in-memory cache for themes inside the theme module
- Add methods for getting & setting items on the cache
- Move all references to config.availableThemes to use the new cache
- This can be abstracted later to support other kinds of caches?
🎨 Start improving theme lib's API
Still TODO: simplifying/clarifying:
- what is the structure of the internal list
- what is the difference between a package list, and a theme list?
- what is the difference between reading a theme and loading it?
- how do we update the theme list (add/remove)
- how do we refresh the theme list? (hot reload?!)
- how do we get from an internal list, to one that is sent as part of the API?
- how are we going to handle theme storage: read/write, such that the path is configurable
🎨 Use themeList consistently
🎨 Update list after storage
closes#8037🔥 Remove API-level default settings population
- This is a relic!
- We ALWAYS populate defaults on server start therefore this code could never run.
- This was a lot of complicated code that wasn't even needed!!
🎨 Move settings cache
- Move settings cache to be its own thing
- Update all references
- Adds TODOs for further cleanup
🎨 Create settings initialisation step
- Create new settings library, which will eventually house more code
- Unify the interface for initialising settings (will be more useful later)
- Reduce number of calls to updateSettingsCache
no issue
* ✨ Add new server start & stop events
* 🔥 Get rid of unused availableApps concept
- when we need an API endpoint for a list of apps, we'll build one 😝
* ✨ Move theme loading into a module
- move loading from API method to a module method and use as needed
- wire up read one vs read all as per LTS
- read one (the active theme) on boot, and read the rest after
- fudge validation - this isn't all that helpful
* Settings API tests need to preload themes
- this used to automatically happen as part of loading settings
- now we need to trigger this to happen specifically for this test
refs #7432🚨 database: change hard limits and field types
- we went over all schema fields and decided to decrease/increase the hard limits
- the core goal is to have more flexibility in the future
- we reconsidered string vs. text
There are 5 groups:
- small strings: 50 characters
- static strings
- status, visibility, language, role name, permission name, client name etc.
- medium strings: 191 characters
- all unique fields or fields which can be unique in the future
- slug, tokens, user name, password, tag name, email
- large strings: 1000-2000 characters
- these fields need to be very flexible
- these fields get a soft limit attached (in a different PR)
- post title, meta title, meta description, urls
- medium text: 64kb characters
- bio, settings, location, tour
- long text: 1000000000 chars
- html, amp, mobiledoc, markdown
🙄 sort_order for tests
- sort order was not set for the tests, so it was always 0
- mysql could return a different result
in my case:
- field length 156 returned the following related tags ["bacon", "kitchen"]
- field length 157 returned the following related tags ["kitchen", "kitchen"]
Change client.secret to 191
Tweak field lengths
- Add 24 char limit for ids
- Limited fields are the exact length they need
- Unified 1000 and 2000 char string classes to all be 2000
- Changed descriptions to be either 2000, except user & tag which is text 65535 as these may be used to store HTML later?!
- Updated tests
🛠 Update importer tests
- The old 001-003 tests are kind of less relevant now.
- Rather than worrying about past versions of the data structure, we should check that the importer only imports what we consider to be valid data
- I've changed the tests to treat the title-length check as a length-validation check, rather than a test for each of the old versions
🔥 Remove foreign key from subscribers.post_id
- There's no real need to have an index on this column, it just makes deleting posts hard.
- Same as created_by type columns, we can reference ids without needing keys/indexes
