Commit Graph

441 Commits

Author SHA1 Message Date
Katharina Irrgang
7b4c3fc085 Removed defunct Ghost OAuth code (#9014)
closes #8342

- no need to add a migration, because when we'released 1.0, OAuth was never an option
- it was disabled in April, 1.0-beta was released in June
- remove all remote authentication code
2017-09-18 13:01:58 +01:00
Katharina Irrgang
aef3d7f3f3 Tests: Fix one more random failure (#9004)
refs #7470

- this should fix https://github.com/TryGhost/Ghost/issues/7470#issuecomment-321016771
- the importer adds posts in parallel, but the tests read directly from the db without any order
- use findPage (findAll does not support order yet)
2017-09-12 16:51:40 +01:00
kirrg001
2647b754d1 Tests: Improve random failures and optimise comment id tests
refs #7470

- the importer test causes problems with the order of posts
- the importer is greedy and tries to add data in parallel, but the tests simply fetch the raw data from knex without any order
- while i was improving the order problem, i found this amp/disqus edge case

Order Random Failure Example:

1) Import (new test structure) 1.0: basic import test keeps the value of the amp field:
      AssertionError: expected '59a952be7d79ed06b0d21128' to equal '1'
         + expected - actual
           -59a952be7d79ed06b0d21128
           +1
2017-09-12 16:29:59 +01:00
Austin Burdine
a9e668a949 🐛 Fixed only save imported post ids if amp field is empty (#8967)
closes #8963

- if an LTS export is imported into a 1.0 blog, then the 1.0 blog is
exported and re-imported into another 1.0 blog, any post ids from the
lts import were getting clobbered. This only saves the post id if the
amp field does not already exist
- add failing test that passes w/change
2017-09-04 09:48:56 +02:00
David Wolfe
c3fcb3105f Add ghost-backup client to trigger export (#8911)
no issue
- adds a ghost-backup client
- adds a client authenticated endpoint to export blog for ghost-backup client only
- allows some additional overrides during import
- allows for an import by file to override locking a user and double hashing the password
2017-08-22 11:15:40 +01:00
Kevin Ansfield
d064eda229 Add configuration/private endpoint and settings for Unsplash (#8895)
refs #8859

- adds new `configuration/private` endpoint for exposing config that should not be accessible without authentication
- adds `unsplashAPI` to private config
- adds empty `unsplash` config to default settings
2017-08-15 15:59:16 +01:00
Katharina Irrgang
7845617607 Custom Post Excerpt Feature (#8792)
closes #8793

- 1.3 post excerpt migration
  - add 1.3 migration to add `excerpt` to post schema

NOTE:
    - knex-migrator relies on the package.json safe version
    - so right now Ghost is on 1.2
    - the migration script is for 1.3
    - if you pull down the PR (or if we merge this PR into master), you have to run `knex-migrator migrate --v 1.3 --force`
    - knex-migrator will tell you what you have todo

- Bump dependencies
  - knex-migrator@2.1.3
- Soft limit for custom_excerpt
- Extended {{excerpt}} to use custom excerpt
   - when a `custom_excerpt` field exists, the `{{excerpt}}` helper will output this and fall back to autogenerated excerpt if not.
- Refactored behaviour of (meta) description
   - html tag `<meta name="description" />` for posts, tags and author doesn't get rendered if not provided.
        - fallback for `author.bio` removed
        - fallback for `tag.description` removed
   - structured data and schema.org for `post` context takes the following order to render description fields:
        1. custom excerpt
        2. meta description
        3. automated excerpt (50 words)
    - updated and added tests to reflect the changes
2017-08-01 12:39:34 +04:00
Katharina Irrgang
b003a6c173 🐛 fix transfer ownership (#8784)
closes #8781

- when the ownership get's transferred, the id of the new owner is not '1' anymore
- we previously added a database rule, which signalises if the blog is setup or not, see 827aa15757 (diff-7a2fe80302d7d6bf67f97cdccef1f71fR542)
- this database rule is based on the owner id being '1', which is wrong when you transfer ownership
- we should keep in mind, that the owner id being '1' is only the default Ghost setup, but it can change
- blog is setup if the owner is locked
2017-07-31 13:37:37 +04:00
Hannah Wolfe
353e11dafb Primary tag (#8669)
refs #8668

- return primary tag from Post API
- support primary tag in URL
2017-07-31 13:00:03 +04:00
Katharina Irrgang
3462f07f58 🎨 take the latest attached role when importing (#8765)
refs #8756

- there was a bug in one of the last LTS releases, which produced duplicated attached roles to users
- we want to prevent that on import and take the latest created based on the autoincrement id
2017-07-27 13:25:01 +04:00
John O'Nolan
0698be503a Site description fixture update (#8763)
No issue
2017-07-27 12:26:31 +04:00
Katharina Irrgang
ce3830f8a9 🚓 disqus comments (#8762)
closes #8760

- we have to remember the old post id's when migrating a blog from LTS to 1.0
- otherwise we would break disqus comments, because they rely on the post id
- this should fix the discovered situation
2017-07-27 11:55:23 +04:00
Katharina Irrgang
60558a776f 🐛 be able to serve locked users (#8711)
closes #8645, closes #8710

- locked users were once part of the category "active users", but were moved to the inactive category
  -> we have added a protection of not being able to edit yourself when you are either suspended or locked
- but they are not really active users, they are restricted, because they have no access to the admin panel
- support three categories: active, inactive, restricted

* - revert restricted states
- instead, update permission layer: fallback to `all` by default, because you are able to serve any user status
- add more tests

- ATTENTION: there is a behaviour change, that a blog owner's author page can be served before setting up the blog, see conversation on slack
   -> LTS serves 404
   -> 1.0 would serve 200
2017-07-20 12:45:13 +01:00
Katharina Irrgang
59d7302da5 🐛 import invalid dates (#8712)
closes #8703, closes #8015

- add sanitize fn to importer
- check wether an imported date is a valid date
- if not, print a warning
2017-07-20 11:24:23 +01:00
Katharina Irrgang
90fc7a6c06 🐛 respect owner user id references when importing (#8693)
closes #8691 

There was a condition added when i've refactored the importer.

> if (models.User.isOwnerUser(obj[key])) {

This condition is absolutely wrong! If you import an owner user, this owner user get's imported as administrator. But the original owner user id reference must be updated as well, so that the reference points to the new administrator id ✌🏻
2017-07-20 11:11:56 +01:00
kirrg001
91f36fc241 🐛 invite existing users
refs #8692

- protect invite endpoint
2017-07-18 18:24:36 +01:00
Hannah Wolfe
6a45ca2605 🐛 Honour labs defaults on import (#8667)
closes #8601

- This makes sure that when you do an import, you still get the LATEST
  default settings for labs. Even if you had a different value before.
- LTS -> 1.0 is an upgrade, and Public API should be on by default, even if you
  had deliberately turned it off before.
- Cheeky test added
2017-07-09 14:31:53 +02:00
Aileen Nowak
35bd0aeb60 🐛 Fix error message for login when password wrong (#8594)
closes #8565

- isPasswordCorrect fn returns a specific error, which we simply forward
- no need to wrap a custom error into a new custom error
- the rule is always: if you are using a Ghost unit/function, you can expect that this unit returns a custom error
2017-06-19 10:37:58 +02:00
Kevin Ansfield
f675a962cf 📖 new default post content (#8548)
closes #8542
- updates default post fixtures
- adds default logo and cover images to settings fixtures
- update tests due to coupling to dev/prod fixtures
2017-06-08 16:36:14 +01:00
David Wolfe
b081ae34b5 🎨 Support LTS imports (#8498)
refs #8141

- update importer for LTS fields
- optimise for LTS export fixtures
- add image/language test for LTS import
- ensure post image is mapped to feature_image
- create mobiledoc values from markdown and html
- if mobiledoc is null, use markdown or html to create a mobiledoc markdown card
- update import mapping to use locale
- defaultLang in settings now maps to default_locale
- language for post and user models now maps to locale
- posts are not always loaded in correct same order so we select the posts we want to validate
- ensure if mobiledoc field is not in export we can still import from markdown
- map last_login to last_seen
- for users the importer maps last_login to last_seen
- add warning for legacyActiveTheme
- for export with old activeTheme key provide a warning that theme is not installed
- add importer test for LTS user long email
- add a test for LTS export where email address could be longer than alpha
- fix for importer date tests on mysql
- use valueOf in moment to compare times stored in different formats
- ignore warnings for not found settings in import
- use a flag to ignore NotFound Entries for settings during import
2017-06-04 11:53:00 +02:00
Katharina Irrgang
18b71f32b1 🎨 rename language to locale and use en as default locale (#8490)
no issue
- this PR references indirecty to https://github.com/TryGhost/Ghost/pull/8437
- i would like to have the settings change already in place before we release the beta
- the i18n feature is able to change the locale of Ghost
- most i18n libraries use locale
- adding/changing settings doesn't require a migration file, but it can make the database a bit messy (because you can end up with default_locale and lang)
- furthermore we agreed that the default locale for Ghost should be simply `en`, not `en_US` or `en_GB`
2017-05-31 16:05:49 +01:00
Kevin Ansfield
85496f409a 🔥 remove posts.markdown field (#8497)
closes #8479

- removes `markdown` field from schema
- removes `legacyMarkdown` converter
- updates tests to work with `mobiledoc` field instead of `markdown` and adapt for mobiledoc HTML output where necessary
2017-05-31 16:46:29 +02:00
Hannah Wolfe
3e60941054 Add ?formats param to Posts API (#8305)
refs #8275
- Adds support for `formats` param
- Returns `html` by default
- Can optionally return other formats by providing a comma-separated list
2017-05-30 11:40:39 +01:00
John O'Nolan
476caa5c2f 🎨 Update fixtures: Ghost owner (#8460)
no issue

- use a better name for default user pre-setup
- make it clear that this email address is an example, not real
2017-05-29 19:25:19 +02:00
Katharina Irrgang
925d72198c 🐛 Ensure post tags sort order is correct when importing (#8481)
refs #6967, refs #5422

- imports posts tags by post id and sort order
- test: fix order problem (sqlite)
2017-05-23 19:12:08 +01:00
Katharina Irrgang
1f37ff6053 🎨 refactor the importer (#8473)
refs #5422

- we can support null titles after this PR if we want
- user model: fix getAuthorRole
- user model: support adding roles by name
- we support this for roles as well, this makes it easier when importing related user roles (because usually roles already exists in the database and the related id's are wrong e.g. roles_users)
- base model: support for null created_at or updated_at values
- post or tag slugs are always safe strings
- enable an import of a null slug, no need to crash or to cover this on import layer
- add new DataImporter logic
    - uses a class inheritance mechanism to achieve an easier readability and maintenance
    - schema validation (happens on model layer) was ignored
    - allow to import unknown user id's (see https://github.com/TryGhost/Ghost/issues/8365)
    - most of the duplication handling happens on model layer (we can use the power of unique fields and errors from the database)
- the import is splitted into three steps:
  - beforeImport
    --> prepares the data to import, sorts out relations (roles, tags), detects fields (for LTS)
  - doImport
    --> does the actual import
  - afterImport
    --> updates the data after successful import e.g. update all user reference fields e.g. published_by (compares the imported data with the current state of the database)
- import images: markdown can be null
- show error message when json handler can't parse file
- do not request gravatar if email is null
- return problems/warnings after successful import
- optimise warnings in importer
- do not return warnings for role duplications, no helpful information
- error handler: return context information of error
- we show the affected json entries as one line in the UI
- show warning for: detected duplicated tag
- schema validation: fix valueMustBeBoolean translation
- remove context property from json parse error
2017-05-23 17:18:13 +01:00
Katharina Irrgang
9bea2077cf 🐛 ensure import of scheduled posts works (#8454)
closes #8354

- i thought about transforming scheduled posts into drafts on export, but this has two disadvantages:
  1. existing exports with scheduled posts won't import
  2. if you schedule a post for next week and you export/import earlier, the post is back to draft
- by this we ensure that we can simply import the post back to a scheduled post
- if the published_at is already in the past, the scheduler will care and instantly publish the post
2017-05-12 15:11:52 +02:00
Katharina Irrgang
4e2474a018 🎨 settings inconsistency (#8381)
no issue
- replace camelCase settings keys with underscore_case for consistency
- discussed here https://github.com/TryGhost/Ghost-Admin/pull/661#discussion_r112939982
2017-04-24 18:41:00 +01:00
Katharina Irrgang
76bd4fdef6 🙀 Image field naming & new img_url helper (#8364)
* 🙀  change database schema for images
    - rename user/post/tag images
    - contains all the required changes from the schema change

* Refactor helper/meta data
    - rename cover to cover_image
    - also rename default settings to match the pattern
    - rename image to profile_image for user
    - rename image to feature_image for tags/posts

* {{image}} >>> {{img_url}}
    - rename
    - change the functionality
    - attr is required
    - e.g. {{img_url feature_image}}

* gscan 1.0.0
    - update yarn.lock

* Update casper reference: 1.0-changes
    - see 5487b4da8d
2017-04-24 18:21:47 +01:00
Katharina Irrgang
c93f03b87e post update collision detection (#8328) (#8362)
closes #5599

If two users edit the same post, it can happen that they override each others content or post settings. With this change this won't happen anymore.

 Update collision for posts
- add a new bookshelf plugin to detect these changes
- use the `changed` object of bookshelf -> we don't have to create our own diff
- compare client and server updated_at field
- run editing posts in a transaction (see comments in code base)

🙀  update collision for tags
- `updateTags` for adding posts on `onCreated` - happens after the post was inserted
   --> it's "okay" to attach the tags afterwards on insert
   --> there is no need to add collision for inserting data
   --> it's very hard to move the updateTags call to `onCreating`, because the `updateTags` function queries the database to look up the affected post
- `updateTags` while editing posts on `onSaving` - all operations run in a transactions and are rolled back if something get's rejected

- Post model edit: if we push a transaction from outside, take this one

  introduce options.forUpdate
- if two queries happening in a transaction we have to signalise knex/mysql that we select for an update
- otherwise the following case happens:
  >> you fetch posts for an update
  >> a user requests comes in and updates the post (e.g. sets title to "X")
  >> you update the fetched posts, title would get overriden to the old one

use options.forUpdate and protect internal post updates: model listeners
- use a transaction for listener updates
- signalise forUpdate
- write a complex test

use options.forUpdate and protect internal post updates: scheduling
- publish endpoint runs in a transaction
- add complex test
- @TODO: right now scheduling api uses posts api, therefor we had to extend the options for api's
  >> allowed to pass transactions through it
  >> but these are only allowed if defined from outside {opts: [...]}
  >> so i think this is fine and not dirty
  >> will wait for opinions
  >> alternatively we have to re-write the scheduling endpoint to use the models directly
2017-04-19 14:53:23 +01:00
Katharina Irrgang
f7393686f8 🐛 UTC offset for scheduled posts when changing timezone (#8356)
no issue

- the UTC offset diff of the current and previous timezone must switch
- i have added more tests and more example case descriptions to understand why
2017-04-19 10:26:33 +02:00
Hannah Wolfe
06fc5f4508 Autofill plaintext field on save (#8304)
refs #8275

- If the HTML field has changed, update the plaintext field
- Use html-to-text to generate a plaintext version of the HTML which retains some structure
- Add a couple of tests - although there's much to do here!
2017-04-11 10:55:36 +02:00
Katharina Irrgang
59a8911830 🐛 fix hasDateChanged (#8291)
no issue
- i don't know if this never worked or has worked and something changed in bookshelf
- but this fixes: saving the content (no change for published_at) of a scheduled post within the 2minutes window
- add `beforeWrite` option to hasDateChanged helper, see comment
- use previous for `beforeWrite` operations
- add a test and fix some other small issues in the scheduler tests
2017-04-06 17:49:59 +01:00
Katharina Irrgang
1bcd25fdbf 🎨 delete all content: do not delete subscribers (#8282)
closes #7875
2017-04-05 21:57:41 +01:00
Katharina Irrgang
587ff6f026 🎨 change last_login to last_seen (#8259)
refs #8258

* 🎨  change last_login to last_seen

- rename the column
- a change in Ghost-Admin is required as well

* test utils: revert export examples

* revert line breaks
2017-04-05 20:45:55 +01:00
Hannah Wolfe
47e00900cc 💄 🐷 Test consistency (#8199)
no issue

- change out should.equal for // jshint ignore:line
- ensure should is the first require in every test, and ALWAYS require
- make sinon the second require, and sandbox the last thing
- ALWAYS use sandbox, futureproofs tests against contributors who don't know it
- change require formatting
2017-03-21 09:24:11 +01:00
Katharina Irrgang
974adee932 🔥 remove fileStorage option (#8144)
refs #8032

- this was used to disable the upload image functionality in Ghost-Admin
- we no longer need this boolean, because people can add their own storage adapter
2017-03-14 10:31:33 +00:00
Katharina Irrgang
c9f551eb96 suspend user feature (#8114)
refs #8111 
- Ghost returns now all (active+none active) users by default
- protect login with suspended status
- test permissions and add extra protection for suspending myself
- if a user is suspended and tries to activate himself, he won't be able to proceed the login to get a new token
2017-03-13 12:03:26 +00:00
Hannah Wolfe
b2f1d0559b Themes API activation permissions & validation (#8104)
refs #8093

 Add activate theme permission
- add permission to activate themes
- update tests
- also: update tests for invites
TODO: change how the active theme setting is updated to reduce extra permissions

 Move theme validation to gscan
- add a new gscan validation method and use it for upload
- update activate endpoint to do validation also using gscan
- change to using SettingsModel instead of API so that we don't call validation or permissions on the settings API
- remove validation from the settings model
- remove the old validation function
- add new invalid theme message to translations & remove a bunch of theme validation related unused keys

📖  Planned changes

🚨 Tests for theme activation API endpoint
🐛 Don't allow deleting the active theme

🚫 Prevent activeTheme being set via settings API
- We want to control how this happens in future.
- We still want to store the information in settings, via the model.
- We just don't want to be able to change this info via the settings edit endpoint

🐛  Fix warnings for uploads & add for activations
- warnings for uploads were broken in f8b498d
- fix the response + adds tests to cover that warnings are correctly returned
- add the same response to activations + more tests
- activations now return a single theme object - the theme that was activated + any warnings

🎨 Improve how we generate theme API responses
- remove the requirement to pass in the active theme!
- move this to a specialist function, away from the list

🎨 Do not load gscan on boot
2017-03-13 12:44:44 +01:00
Hannah Wolfe
a5ab2ffc13 🔥 🎨 No more updateSettingsCache (#8090)
no issue

🔥 Remove unnecessary cache update
🎨 simplify updateSettingsCache()
🎨 Simplify readSettingsResult
- although this is more code, it's now much clearer what happens in the two cases
🎨 Don't use readSettingResult for edit
🎨 Simplify updateSettingsCache further
🔥 Remove now unused readSettingsResult
🎨 Change populateDefault to return all
🎨 Move the findAll call out of updateSettingsCache
🔥 Remove updateSettingsCache!!
🎨 Restructure init & finish up settingsCache
- move initialisation into settingsCache.init AT LAST
- change settingCache to use cloneDeep, so that the object can't be modified outside of the functions
- add lots of docs to settings cache
🎨 Cleanup db api endpoints
🔥 Don't populate settings in migrations
2017-03-02 23:00:01 +01:00
Hannah Wolfe
63723aa36a 🎨 Move settings cache & cleanup settings API (#8057)
closes #8037

🔥 Remove API-level default settings population
- This is a relic!
- We ALWAYS populate defaults on server start therefore this code could never run.
- This was a lot of complicated code that wasn't even needed!!

🎨 Move settings cache
- Move settings cache to be its own thing
- Update all references
- Adds TODOs for further cleanup

🎨 Create settings initialisation step
- Create new settings library, which will eventually house more code
- Unify the interface for initialising settings (will be more useful later)
- Reduce number of calls to updateSettingsCache
2017-02-27 16:53:04 +01:00
Hannah Wolfe
fe90cf2be2 Theme loading part 1 (#7989)
no issue

*  Add new server start & stop events
* 🔥 Get rid of unused availableApps concept
- when we need an API endpoint for a list of apps, we'll build one 😝
*  Move theme loading into a module
- move loading from API method to a module method and use as needed
- wire up read one vs read all as per LTS
- read one (the active theme) on boot, and read the rest after
- fudge validation - this isn't all that helpful
* Settings API tests need to preload themes
- this used to automatically happen as part of loading settings
- now we need to trigger this to happen specifically for this test
2017-02-22 00:26:19 +01:00
Katharina Irrgang
aa670fa8b4 Updates for alpha.13 (#8026)
* 🐛  delete client if auth url has changed

no issue

* 🎨  update default auth config

refs #7488
2017-02-18 17:23:35 +01:00
Katharina Irrgang
2e1d7fcc42 🚨 database: change hard limits and field types (#7932)
refs #7432

🚨  database: change hard limits and field types

- we went over all schema fields and decided to decrease/increase the hard limits
- the core goal is to have more flexibility in the future
- we reconsidered string vs. text

There are 5 groups:

- small strings: 50 characters
    - static strings
    - status, visibility, language, role name, permission name, client name etc.
- medium strings: 191 characters
    - all unique fields or fields which can be unique in the future
    - slug, tokens, user name, password, tag name, email
- large strings: 1000-2000 characters
    - these fields need to be very flexible
    - these fields get a soft limit attached (in a different PR)
    - post title, meta title, meta description, urls
- medium text: 64kb characters
    - bio, settings, location, tour
- long text: 1000000000 chars
    - html, amp, mobiledoc, markdown

🙄  sort_order for tests

- sort order was not set for the tests, so it was always 0
- mysql could return a different result

in my case:
- field length 156 returned the following related tags ["bacon", "kitchen"]
- field length 157 returned the following related tags ["kitchen", "kitchen"]

Change client.secret to 191

Tweak field lengths

- Add 24 char limit for ids
- Limited fields are the exact length they need
- Unified 1000 and 2000 char string classes to all be 2000
- Changed descriptions to be either 2000, except user & tag which is text 65535 as these may be used to store HTML later?!
- Updated tests

🛠  Update importer tests

- The old 001-003 tests are kind of less relevant now.
- Rather than worrying about past versions of the data structure, we should check that the importer only imports what we consider to be valid data
- I've changed the tests to treat the title-length check as a length-validation check, rather than a test for each of the old versions

🔥 Remove foreign key from subscribers.post_id

- There's no real need to have an index on this column, it just makes deleting posts hard.
- Same as created_by type columns, we can reference ids without needing keys/indexes
2017-02-17 23:20:59 +01:00
Katharina Irrgang
2a5927b4f9 🐛 fix email already in use (#7987)
refs #7981

- usage of data.id was wrong
- usage of id comparison was wrong
2017-02-13 15:36:21 +00:00
Katharina Irrgang
29fb68137f 🎨 increase token expiry (#7971)
refs #5202
- please read https://github.com/TryGhost/Ghost/issues/5202#issuecomment-278934768
2017-02-10 13:35:58 +00:00
Katharina Irrgang
72e9f0a95e 🐛 delete all content if subscriber is associated with a post (#7962)
closes #7875
- we need to delete the subscribers before deleting the posts
2017-02-08 15:40:47 +00:00
janvt
e6662b6929 Add email validation in case of profile update (#7928)
closes #7256

- original code changes made by @golya in https://github.com/TryGhost/Ghost/pull/7304
- refactored edit method in user model to validate an existing email address
- added test coverage for existing email update in user model spec
2017-02-08 10:50:43 +01:00
kirrg001
a1cb8e8c58 🐛 tests: integration test for api notifications failed sometimes
no issue

- sometimes the integration test for api notifications failed
- this was caused because multiple tests are using the notifications API, which has a local notification store
- we need to ensure that tests, who add notifications to this store, reset the store after it's test iteration
2017-02-07 21:04:35 +01:00
Katharina Irrgang
f3d16352b1 🎨 😎 config env usages (#7929)
refs #7488

- remove all ugly env checks
- rather use config properties
- replace process.env.NODE_ENV by config.get('env')
2017-02-03 18:25:39 +00:00
Katharina Irrgang
0201c431d7 🔥 do not store settings in config (#7924)
* 🎨  🔥  do not store settings in config and make settings cache easier available

- remove remembering settings value in theme config
- if we need a cache value, we are asking the settings cache directly
- instead of settings.getSettingSync we use settings.cache.get

- added TODO:
  - think about moving the settings cache out of api/settings
  - we could create a folder named cache cache/settings
  - this settings cache listens on model changes for settings
  - decoupling

* 🔥  remove timezone from config

- no need to store in overrides config and in defaults settings

* 🎨  context object helper

- replace config.get('theme') by settings cache

* 🎨  replace config.get('theme') by settings.cache.get

* 🎨  adapt tests

* fixes from comments
2017-02-03 13:15:11 +00:00
Katharina Irrgang
584bd15b76 🔥 remove database version (#7894)
refs #7489

- as we are now using a different migration approach (knex-migrator), we don't need to remember the database version anymore
- it was once used to check the state of a database and based on it we decided to migrate or not
- with knex-migrator everything depends on the migration table entries and the current ghost version you are on
- on current master the leftover usage is to add the db version when exporting the database, which can be replaced by reading the ghost version
- removing this solves also an interesting migration case with knex-migrator:
  - you are on 1.0
  - you update to 1.1, but 1.1 has no migrations
  - the db version would remain in 1.0
  - because the db version was only updated when knex migrator executed a migration
2017-01-26 12:12:00 +00:00
Katharina Irrgang
2d19ae2c6c 🔥 😎 remove old migrations (#7887)
refs #7489

- remove old migration code
- this logic was sourced out to knex-migrator
2017-01-25 13:47:49 +00:00
Katharina Irrgang
042750f4cf 🐛 fix invite permissions for editor (#7889)
refs #7724

- we already fixed the permissions for the editor
- see 3d3101ad0e
- but as we are inside of a refactoring process, we had two fixtures.json files
- we fixed the fixtures.json in the wrong place
- now that the permissions are used, we can see failing tests
- i have added the correct permissions handling
2017-01-25 12:07:31 +00:00
Katharina Irrgang
a2edc09762 🎨 optimisations for brute (#7867)
closes #7766, refs #7579

- ensure we are using the correct brute keys
- ensure we are using req.ip as Ghost is configured  with trust proxy option
- tidy up a little
2017-01-23 22:44:39 +01:00
Katharina Irrgang
78eacb19e9 🛠 use Ignition for logging/errors (#7869)
no issue

- we started implementing logging and error handling in Ghost
- later we outsourced both into a module
- use the module now in Ghost
- this commit basically just removes the logging and error implementation and uses Ignition
2017-01-23 11:04:01 +00:00
Aileen Nowak
503148058c More consistant usage of urlFor('home') (#7689)
refs #7666 

Using `urlFor('home')` instead `config.get('url')` in Ghost.
When `urlFor('home', true)` returns the absolute adress of the blog as defined in the config.
Will always return a trailing `/`.
2017-01-23 09:22:37 +01:00
Katharina Irrgang
5210271474 🐛 import subscribers (#7834)
closes #7748

- logic for import of subscribers was just missing
2017-01-10 19:38:20 +00:00
Marc Bachmann
353330bb8a chore(package): update uuid to version 3.0.0 (#7742) 2017-01-04 17:10:29 +01:00
Katharina Irrgang
7eb316b786 replace auto increment id's by object id (#7495)
* 🛠  bookshelf tarball, bson-objectid

* 🎨  schema changes

- change increment type to string
- add a default fallback for string length 191 (to avoid adding this logic to every single column which uses an ID)
- remove uuid, because ID now represents a global resource identifier
- keep uuid for post, because we are using this as preview id
- keep uuid for clients for now - we are using this param for Ghost-Auth

*   base model: generate ObjectId on creating event

- each new resource get's a auto generate ObjectId
- this logic won't work for attached models, this commit comes later

* 🎨  centralised attach method

When attaching models there are two things important two know

1. To be able to attach an ObjectId, we need to register the `onCreating` event the fetched model!This is caused by the Bookshelf design in general. On this target model we are attaching the new model.
2. We need to manually fetch the target model, because Bookshelf has a weird behaviour (which is known as a bug, see see https://github.com/tgriesser/bookshelf/issues/629). The most important property when attaching a model is `parentFk`, which is the foreign key. This can be null when fetching the model with the option `withRelated`. To ensure quality and consistency, the custom attach wrapper always fetches the target model manual. By fetching the target model (again) is a little performance decrease, but it also has advantages: we can register the event, and directly unregister the event again. So very clean code.

Important: please only use the custom attach wrapper in the future.

* 🎨  token model had overriden the onCreating function because of the created_at field

- we need to ensure that the base onCreating hook get's triggered for ALL models
- if not, they don't get an ObjectId assigned
- in this case: be smart and check if the target model has a created_at field

* 🎨  we don't have a uuid field anymore, remove the usages

- no default uuid creation in models
- i am pretty sure we have some more definitions in our tests (for example in the export json files), but that is too much work to delete them all

* 🎨  do not parse ID to Number

- we had various occurances of parsing all ID's to numbers
- we don't need this behaviour anymore
- ID is string
- i will adapt the ID validation in the next commit

* 🎨  change ID regex for validation

- we only allow: ID as ObjectId, ID as 1 and ID as me
- we need to keep ID 1, because our whole software relies on ID 1 (permissions etc)

* 🎨  owner fixture

- roles: [4] does not work anymore
- 4 means -> static id 4
- this worked in an auto increment system (not even in a system with distributed writes)
- with ObjectId we generate each ID automatically (for static and dynamic resources)
- it is possible to define all id's for static resources still, but that means we need to know which ID is already used and for consistency we have to define ObjectId's for these static resources
- so no static id's anymore, except of: id 1 for owner and id 0 for external usage (because this is required from our permission system)
- NOTE: please read through the comment in the user model


* 🎨  tests: DataGenerator and test utils

First of all: we need to ensure using ObjectId's in the tests. When don't, we can't ensure that ObjectId's work properly.
This commit brings lot's of dynamic into all the static defined id's.
In one of the next commits, i will adapt all the tests.

* 🚨  remove counter in Notification API

- no need to add a counter
- we simply generate ObjectId's (they are auto incremental as well)
- our id validator does only allow ObjectId as id,1 and me

* 🎨  extend contextUser in Base Model

- remove isNumber check, because id's are no longer numbers, except of id 0/1
- use existing isExternalUser
- support id 0/1 as string or number

*   Ghost Owner has id 1

- ensure we define this id in the fixtures.json
- doesn't matter if number or string

* 🎨  functional tests adaptions

- use dynamic id's

* 🎨  fix unit tests

* 🎨  integration tests adaptions

* 🎨  change importer utils

- all our export examples (test/fixtures/exports) contain id's as numbers
- fact: but we ignore them anyway when inserting into the database, see https://github.com/TryGhost/Ghost/blob/master/core/server/data/import/utils.js#L249
- in 0e6ed957cd (diff-70f514a06347c048648be464819503c4L67) i removed parsing id's to integers
- i realised that this ^ check just existed, because the userIdToMap was an object key and object keys are always strings!
- i think this logic is a little bit complicated, but i don't want to refactor this now
- this commit ensures when trying to find the user, the id comparison works again
- i've added more documentation to understand this logic ;)
- plus i renamed an attribute to improve readability

* 🎨  Data-Generator: add more defaults to createUser

- if i use the function DataGenerator.forKnex.createUser i would like to get a full set of defaults

* 🎨  test utils: change/extend function set for functional tests

- functional tests work a bit different
- they boot Ghost and seed the database
- some functional tests have mis-used the test setup
- the test setup needs two sections: integration/unit and functional tests
- any functional test is allowed to either add more data or change data in the existing Ghost db
- but what it should not do is: add test fixtures like roles or users from our DataGenerator and cross fingers it will work
- this commit adds a clean method for functional tests to add extra users

* 🎨  functional tests adaptions

- use last commit to insert users for functional tests clean
- tidy up usage of testUtils.setup or testUtils.doAuth

* 🐛  test utils: reset database before init

- ensure we don't have any left data from other tests in the database when starting ghost

* 🐛  fix test (unrelated to this PR)

- fixes a random failure
- return statement was missing

* 🎨  make changes for invites
2016-11-17 09:09:11 +00:00
Katharina Irrgang
3d3101ad0e 🐛 invite permissions for Editor (#7724)
closes #7723

- editor role had no permissions assigned for invites
2016-11-16 13:49:55 +00:00
Katharina Irrgang
0f855c538e 🎨 invites roles table into a field on the invites table (#7705)
* 🎨  schema change

- simply role_id attribute

* 🎨  update invite model

- remove all methods we don't need
- ensure we remove the relation from the model
- ensure we do not allow to call withRelated

* 🎨  adapt api changes

* 🎨  adapt auth module

* 🎨  adapt tests

* 🎨  better error handling

* schema update
2016-11-16 09:33:44 +00:00
Katharina Irrgang
b48031fa0e 🎨 unique constraint for permission and role name (#7674)
refs #7494,  refs #7495 

I saw tests adding permissions and roles twice. (see screenshots)
That happened because the setup in the test was mis-used and there is no restriction for static resources to create duplicates.
With this PR i suggest to make name unique.
2016-11-09 15:02:49 +00:00
Katharina Irrgang
3aac3ef6de 🎨 make settings cache available (#7692)
* 🎨  settingsCache is available

- do not destroy the object reference
- added TODO to reconsider the config values for theme
- get one or all cached settings

* 🚨  remove api.init

- this functiion has just wrapped a function to update the settings cache
- if we have multiple tasks todo later, we can re-add
- but for now: this is way easier to read
- adapt test

* 🎨  tests
2016-11-08 13:37:19 +00:00
David Wolfe
68af2145a1 Replace memory spam prevention with brute-express (#7579)
no issue

- removes count from user checks model
- uses brute express brute with brute-knex adaptor to store persisted data on spam prevention
- implement brute force protection for password/token exchange, password resets and private blogging
2016-11-08 12:33:19 +01:00
Katharina Irrgang
bae0de6cd5 knex-migrator v2 (#7605)
* 🎨  knex-migrator reset

[ci skip]

*   add migration example

- hooks
- 1.0

[ci skip]

* 🛠  knex-migrator tarball

- remove when released

[ci skip]

* 🎨  jscs/jshint

* 🕵🏻 do not drop the database connection when running tests

- please read the comments in the commit

* 🔥  remove example migration

* 🛠  knex-migrator 0.1.0

* 🛠  knex-migrator 0.1.1

- fix a single test to ensure we catch the error

* 🛠  knex-migrator 0.1.2

* 🎨  make tests green

- added my keyword: kate-migrations
- i will go over all TODO's when removing the old migrations code

* 🛠  knex-migrator update

* 🛠  knex-migrator 0.2.0
2016-11-07 11:39:49 +00:00
Katharina Irrgang
4e7779b783 🎨 remove token logic from user model (#7622)
* 🔥  remove User model functions

- validateToken
- generateToken
- resetPassword
- all this logic will re-appear in a different way

Token logic:
- was already extracted as separate PR, see https://github.com/TryGhost/Ghost/pull/7554
- we will use this logic in the controller, you will see in the next commits

Reset Password:
Was just a wrapper for calling the token logic and change the password.
We can reconsider keeping the function to call: changePassword and activate the status of the user - but i think it's fine to trigger these two actions from the controlling unit.

* 🔥  remove password reset tests from User model

- we already have unit tests for change password and the token logic
- i will re-check at the end if any test case is missing - but for now i will just burn the tests

*   add token logic to controlling unit

generateResetToken endpoint
- the only change here is instead of calling the User model to generate a token, we generate the token via utils
- we fetch the user by email, and generate a hash and return

resetPassword endpoint
- here we have changed a little bit more
- first of all: we have added the validation check if the new passwords match
- a new helper method to extract the token informations
- the brute force security check, which can be handled later from the new bruteforce middleware (see TODO)
- the actual reset function is doing the steps: load me the user, compare the token, change the password and activate the user
- we can think of wrapping these steps into a User model function
- i was not sure about it, because it is actually part of the controlling unit

[ci skip]

* 🎨  tidy up

- jscs
- jshint
- naming functions
- fixes

*   add a test for resetting the password

- there was none
- added a test to reset the password

* 🎨  add more token tests

- ensure quality
- ensure logic we had

* 🔥  remove compare new password check from User Model

- this part of controlling unit

*   compare new passwords for user endpoint

- we deleted the logic in User Model
- we are adding the logic to controlling unit

* 🐛  spam prevention forgotten can crash

- no validation happend before this middleware
- it just assumes that the root key is present
- when we work on our API, we need to ensure that
  1. pre validation happens
  2. we call middlewares
  3. ...

* 🎨  token translation key
2016-11-07 11:18:50 +00:00
Katharina Irrgang
be183f3441 🐛 fix DST in listeners spec (#7652)
no issue
- we need to calculate the timezone offset dynamically, because of DST
2016-10-31 11:46:29 +00:00
Katharina Irrgang
a55fb0bafe 🎨 public config endpoint (#7631)
closes #7628

With this PR we expose a public configuration endpoint.
When /ghost is requested, we don't load and render the configurations into the template anymore. Instead, Ghost-Admin can request the public configuration endpoint.

* 🎨  make configuration endpoint public
* 🔥  remove loading configurations in admin app
- do not render them into the default html page
*   load client credentials in configuration endpoint
- this is not a security issue, because we have exposed this information anyway before (by rendering them into the requested html page)
* 🎨  extend existing configuration integration test
*   tests: add ghost-auth to data generator
*   add functional test
* 🔥  remove type/value pattern
* 🎨  do not return stringified JSON objects
2016-10-28 14:07:46 +01:00
Katharina Irrgang
02a1f08ba3 🐛 fix changePassword bug (#7590)
no issue
- comparison for isLoggedInUser did not work when userId was a string
- parsing of int was missing
2016-10-21 10:19:09 +01:00
Katharina Irrgang
ca7b5643d5 🎨 more clean code in User Model (#7572)
* 🎨  do not call generateSlug twice for User.setup

* 🎨  call generatePasswordHash onSaving only

- now we can add defaults to User Model
- it was not possible before because add User model did the following:
  1. validate password length
  2. hash password manually
  3. call ghostBookshelf.Model.add and THEN bookshelf defaults fn gets triggered
- call generatePasswordHash in onSaving hook for all use case
- add more tests to user model, juhu
2016-10-14 18:24:38 +01:00
Katharina Irrgang
e2e83a0f7b Migration: New database versioning (#7499)
refs #7489

- new database versioning scheme which is based upon the Ghost version, and so easier to reason about
- massive refactor of all the version related code 

Summary of changes:

*   new error: DatabaseNotSeeded
* 🎨  change versioning module
  - versioning is based on Ghost Version
* 🎨  change bootUp file
  - add big picture description
  - version error get's trigger from versioning module
* 🎨  default setting for database version is null
  - very important change: this is caused by the big picture
  - see bootUp description
  - the database version get's set by the seed script later
  - db version is by default null
  - 1. population happens (we ensure that this has finished, by checking if each table exists)   
  - 2. seeds happening (we ensure that seeds happend if database version is set to X.X)
* 🎨  temporary change for population logic
  - set database version after population happens
  - ensure population of default settings happend before
  - both: get's removed in next iteration
* 🎨  adapt tests && mark TODO's
* 🎨  err instance checking
2016-10-06 14:50:55 +01:00
Katharina Irrgang
32700a0e5a 🔥 remove all migration files (#7498)
refs #7489
- a simple PR to remove all the old migration files
- remove related tests
2016-10-05 19:57:32 +01:00
Katharina Irrgang
1882278b5b 🎨 configurable logging with bunyan (#7431)
- 🛠  add bunyan and prettyjson, remove morgan

-   add logging module
  - GhostLogger class that handles setup of bunyan
  - PrettyStream for stdout

-   config for logging
  - @TODO: testing level fatal?

-   log each request via GhostLogger (express middleware)
  - @TODO: add errors to output

- 🔥  remove errors.updateActiveTheme
  - we can read the value from config

- 🔥  remove 15 helper functions in core/server/errors/index.js
  - all these functions get replaced by modules:
    1. logging
    2. error middleware handling for html/json
    3. error creation (which will be part of PR #7477)

-   add express error handler for html/json
  - one true error handler for express responses
  - contains still some TODO's, but they are not high priority for first implementation/integration
  - this middleware only takes responsibility of either rendering html responses or return json error responses

- 🎨  use new express error handler in middleware/index
  - 404 and 500 handling

- 🎨  return error instead of error message in permissions/index.js
  - the rule for error handling should be: if you call a unit, this unit should return a custom Ghost error

- 🎨  wrap serve static module
  - rule: if you call a module/unit, you should always wrap this error
  - it's always the same rule
  - so the caller never has to worry about what comes back
  - it's always a clear error instance
  - in this case: we return our notfounderror if serve static does not find the resource
  - this avoid having checks everywhere

- 🎨  replace usages of errors/index.js functions and adapt tests
  - use logging.error, logging.warn
  - make tests green
  - remove some usages of logging and throwing api errors -> because when a request is involved, logging happens automatically

- 🐛  return errorDetails to Ghost-Admin
  - errorDetails is used for Theme error handling

- 🎨  use 500er error for theme is missing error in theme-handler

- 🎨  extend file rotation to 1w
2016-10-04 16:33:43 +01:00
Hannah Wolfe
e40290af5d 🎨 Use GhostAuth name and example emails (#7475)
refs #7452

- remove references to 'patronus' in favour of GhostAuth, Note: this will require databases to be deleted ;)
- remove email addresses from test data
2016-10-03 15:11:43 +01:00
Katharina Irrgang
6473c9e858 Ghost OAuth (#7451)
issue #7452

Remote oauth2 authentication with Ghost.org.

This PR supports:

- oauth2 login or local login
- authentication on blog setup
- authentication on invite
- normal authentication
- does not contain many, many tests, but we'll improve in the next alpha weeks
2016-09-30 12:45:59 +01:00
kirrg001
b79a18ca8f 🎨 Separate invites from user
refs #7420
- remove invite logic from user
- add invite model and adapt affected logic for inviting team members
2016-09-26 11:08:43 +02:00
Katharina Irrgang
32a5982430 utf8mb4 support (#7409)
closes #5519, closes #6197

- default encoding is utf8mb4 instead of utf8
- support emojis
- read here why: http://dev.mysql.com/doc/refman/5.7/en/innodb-restrictions.html
- read here why: https://dev.mysql.com/doc/refman/5.5/en/charset-unicode-conversion.html
2016-09-20 15:59:34 +01:00
kirrg001
8115d83782 🔥 remove postgres support
refs #1333, refs #5878, refs #5321
2016-09-20 15:59:34 +01:00
kirrg001
b158a3a944 🚨 change logic for test/utils/configUtils
refs #6982
- adaption because of using nconf
- change tests which changed config directly

[ci skip]
2016-09-20 15:59:34 +01:00
kirrg001
0ae0a0b490 🎨 change how we get and set config
refs #6982
- a replace for all config usages
- always use config.get or config.set
- this a pure replacement, no logic has changed

[ci skip]
2016-09-20 15:59:34 +01:00
Austin Burdine
756d9bcb6e show correct error message the first time account is locked (#7263)
closes #7251

- check if remaining attemps is 0, if so then show account locked error
- adds test
2016-09-19 14:56:55 +01:00
Katharina Irrgang
9349e99e54 🐛 fix delete by author as transaction (#7145)
closes #7137

Deleting the content from the database runs in a transaction. see
https://github.com/TryGhost/Ghost/blob/master/core/server/api/users.js#L390

`destroyByAuthor` is one of the operations we trigger to delete all the conent, see https://github.com/TryGhost/Ghost/blob/master/core/server/models/post.js#L647

The post model has a specific hook for deleting content to delete the relations as well, see https://github.com/TryGhost/Ghost/blob/master/core/server/models/post.js#L122

This hook is part of the transaction. But the `options` are ignored. `(model/*, attr, options*/)` 
We use the `options` to forward the transaction reference, which we need to pass into the bookshelf queries. So `return model.load('tags').call('related', 'tags').call('detach')` does not forward the transaction and that's why it stucks when deleting the content.
2016-09-19 14:45:36 +01:00
Kevin Ansfield
0b6459cb91 Fix upgrade notification type value (#7308)
refs #7305

* 🎨 display upgrade alerts with the correct "info" style
* 💄 update use of notifications status/type/location attrs to reflect current usage
2016-09-01 17:58:46 +02:00
Katharina Irrgang
a91e54cf1a feature: theme upload/download/delete (#7209)
refs #7204

- added 3 new themes permissions
- change core/client
- add theme upload/download logic
- extended local file storage to serve zips
- added gscan dependency
- add ability to handle the express response within the api layer
- restrict theme upload to local file storage
- added 007 migration
2016-08-23 13:07:25 +01:00
Katharina Irrgang
3b8f08e0ec fix: delete unused theme endpoints (#7231)
no issue
2016-08-22 10:54:54 +01:00
Aileen Nowak
ea099af854 💄 Remove duplicate test assertion (#7233)
no issue

One test assertion was a duplicate.
2016-08-19 16:38:56 +02:00
John O'Nolan
61a8845ab5 Switch to new native system font stack (#7219)
no issue

Based on an increasingly popular trend and modern web typography capabilities, switch out Google Fonts for default native system fonts, tailored in a stack to suit every device. Also makes some very minor visual adjustments to suit.

Nixes all references to Google Fonts, and provides a faster rendering experience and fewer http requests. 💃

Reference material:

https://www.smashingmagazine.com/2015/11/using-system-ui-fonts-practical-guide/
https://medium.design/system-shock-6b1dc6d6596f#.rhqx5fmyz
Dependencies:

TryGhost/Ghost-Admin#211
TryGhost/Ghost-Desktop#190
2016-08-18 20:29:46 +01:00
Katharina Irrgang
663b410fd4 feature: upload validation middleware (#7208)
no issue

- Source out validation logic into a upload validation middleware for all upload types (csv, image, subscribers). This unit can be later used for Ghost 1.0 as a pre validation core unit. 
- More usage of route tests than controller tests. These are use case tests, a use case only changes if the product changes
2016-08-18 20:25:51 +01:00
Katharina Irrgang
9cd9e03fdb fix: notification store without duplicates (#7135)
closes #7133
- ensure we don't add duplicate notifications to the in process notification store
2016-08-11 08:58:51 +01:00
Greenkeeper
dd5775c018 Update mocha to version 3.0.2 🚀 (#7165)
* chore(package): update mocha to version 3.0.2

https://greenkeeper.io/

* fix duplicate done/promise handling
2016-08-11 08:51:19 +01:00
Tim Walling
a0288303f6 Check file type and file extension when importing csv (#7185)
issue #7144
- added a check for file type and file extension
- added an error message to the localization file
- added integration test
2016-08-11 08:46:06 +01:00
Katharina Irrgang
c9dc367bf0 improvement: validate timezones (#7143)
no issue
- add isTimezone to schema validation
2016-07-26 10:23:20 +01:00
Vijay Kandy
f2d09df512 Support for custom notifications (#7077)
closes #5071

- Send application/json requests to UpdateCheck service. New UpdateCheck service accepts JSON request
- If UpdateCheck service respponse has messages[] array, iterate over the array and create custom notifications intended for current version
- Save custom notification if its not already in the store AND its uuid is not in seenNotifications array
- When a custom notification is dismissed, store its uuid in seenNotifications array
- setup test fixtures to trigger tests properly
- api_notification_spec test to ensure custom notification can be added to store and added to seenNotifications when dismissed
- update_check_spec test to ensure custom notification can be displayed for a specific Ghost version
- added test to ensure messages meant for other versions don't create notifications
2016-07-22 14:02:10 +01:00
Vijay Kandy
ffd3ec563a fix: "url" field is undefined when restricting returned fields (#7089)
closes #6625

- "url" and "author" fields depend on {id, published_at, slug, author_id} to construct post url.
- implemented a generic solution by defining defaultColumnsToFetch() in
  base class for models.
- findPage() calls defaultColumnsToFetch() before loading models
- results are transformed by filtering out additional properties to return just the requested fields
- Added a test case to check for url and author fields
- Renamed allColumns as requestedColumns and used _.map instead of Promise.map
2016-07-18 22:21:47 +02:00
Katharina Irrgang
8c6ba47b0e fix: offset bug when switching timezones (#7101)
no issue
2016-07-15 17:44:22 +01:00
Katharina Irrgang
748c664b78 feature: maintenance mode (#7019)
closes #6976
- add maintenance mode when running migrations
- refactor update/populate migrations
2016-07-15 17:22:41 +01:00
Katharina Irrgang
416c215018 change default order of posts (#7096)
closes #6932
- new default order of posts: scheduled, draft, published
- invent orderDefaultRaw fn for each model
- each model is able to create a default raw order query
- separate count and fetch query for fetchPage, because the count query where group/order statements attached
2016-07-15 11:04:10 +01:00
Katharina Irrgang
6e1bd2838e improvement: migrations (#7000)
closes #6972, #6574

- run each database version as top level transaction
- run migrations in correct order
2016-07-14 11:59:42 +01:00
kirrg001
f8ba12481d fix: prev next posts
closes #7015
2016-06-20 15:44:02 +02:00