Commit Graph

64 Commits

Author SHA1 Message Date
Katharina Irrgang
1f37ff6053 🎨 refactor the importer (#8473)
refs #5422

- we can support null titles after this PR if we want
- user model: fix getAuthorRole
- user model: support adding roles by name
- we support this for roles as well, this makes it easier when importing related user roles (because usually roles already exists in the database and the related id's are wrong e.g. roles_users)
- base model: support for null created_at or updated_at values
- post or tag slugs are always safe strings
- enable an import of a null slug, no need to crash or to cover this on import layer
- add new DataImporter logic
    - uses a class inheritance mechanism to achieve an easier readability and maintenance
    - schema validation (happens on model layer) was ignored
    - allow to import unknown user id's (see https://github.com/TryGhost/Ghost/issues/8365)
    - most of the duplication handling happens on model layer (we can use the power of unique fields and errors from the database)
- the import is splitted into three steps:
  - beforeImport
    --> prepares the data to import, sorts out relations (roles, tags), detects fields (for LTS)
  - doImport
    --> does the actual import
  - afterImport
    --> updates the data after successful import e.g. update all user reference fields e.g. published_by (compares the imported data with the current state of the database)
- import images: markdown can be null
- show error message when json handler can't parse file
- do not request gravatar if email is null
- return problems/warnings after successful import
- optimise warnings in importer
- do not return warnings for role duplications, no helpful information
- error handler: return context information of error
- we show the affected json entries as one line in the UI
- show warning for: detected duplicated tag
- schema validation: fix valueMustBeBoolean translation
- remove context property from json parse error
2017-05-23 17:18:13 +01:00
Katharina Irrgang
1bcd25fdbf 🎨 delete all content: do not delete subscribers (#8282)
closes #7875
2017-04-05 21:57:41 +01:00
Hannah Wolfe
a5ab2ffc13 🔥 🎨 No more updateSettingsCache (#8090)
no issue

🔥 Remove unnecessary cache update
🎨 simplify updateSettingsCache()
🎨 Simplify readSettingsResult
- although this is more code, it's now much clearer what happens in the two cases
🎨 Don't use readSettingResult for edit
🎨 Simplify updateSettingsCache further
🔥 Remove now unused readSettingsResult
🎨 Change populateDefault to return all
🎨 Move the findAll call out of updateSettingsCache
🔥 Remove updateSettingsCache!!
🎨 Restructure init & finish up settingsCache
- move initialisation into settingsCache.init AT LAST
- change settingCache to use cloneDeep, so that the object can't be modified outside of the functions
- add lots of docs to settings cache
🎨 Cleanup db api endpoints
🔥 Don't populate settings in migrations
2017-03-02 23:00:01 +01:00
Hannah Wolfe
690ff05588 🔥 🎨 Themes & settings misc cleanup (#8061)
no issue

🔥 remove unused loadThemes API method
🚨 Add tests for themes.readOne
🔥 Don't update settings cache for imports
- this isn't needed as of #8057
- settings.edit fires an event, that will result in the update happening automatically
🎨 Move validation to themes
- slowly collecting all theme-related code together
🔥 Reduce DEBUG output
- all this info is a bit tooooo much!
2017-02-27 23:30:49 +01:00
Katharina Irrgang
72e9f0a95e 🐛 delete all content if subscriber is associated with a post (#7962)
closes #7875
- we need to delete the subscribers before deleting the posts
2017-02-08 15:40:47 +00:00
Katharina Irrgang
2d19ae2c6c 🔥 😎 remove old migrations (#7887)
refs #7489

- remove old migration code
- this logic was sourced out to knex-migrator
2017-01-25 13:47:49 +00:00
Katharina Irrgang
d81bc91bd2 Error creation (#7477)
refs #7116, refs #2001

- Changes the way Ghost errors are implemented to benefit from proper inheritance
- Moves all error definitions into a single file
- Changes the error constructor to take an options object, rather than needing the arguments to be passed in the correct order.
- Provides a wrapper so that any errors that haven't already been converted to GhostErrors get converted before they are displayed.

Summary of changes:

* 🐛  set NODE_ENV in config handler
*   add GhostError implementation (core/server/errors.js)
  - register all errors in one file
  - inheritance from GhostError
  - option pattern
* 🔥  remove all error files
*   wrap all errors into GhostError in case of HTTP
* 🎨  adaptions
  - option pattern for errors
  - use GhostError when needed
* 🎨  revert debug deletion and add TODO for error id's
2016-10-06 13:27:35 +01:00
Katharina Irrgang
663b410fd4 feature: upload validation middleware (#7208)
no issue

- Source out validation logic into a upload validation middleware for all upload types (csv, image, subscribers). This unit can be later used for Ghost 1.0 as a pre validation core unit. 
- More usage of route tests than controller tests. These are use case tests, a use case only changes if the product changes
2016-08-18 20:25:51 +01:00
Katharina Irrgang
f644d99460 add small permission improvements
no issue
- do not check client type in auth middleware
- offer filtering for findAll function in base
- add isInternalContext to base model
2016-05-08 14:22:55 +02:00
Jason Williams
c41ee354b1 Replace busboy upload middleware with multer
- deps: multer@1.1.0
2016-04-07 21:30:46 -05:00
Jason Williams
9fe573a0c5 Refactor content deletion
- Simplify the `init` method in `models/index.js` so that it no longer
  returns a promise. Easier to use.
- Eliminates the `deleteAllContent` method from `models/index.js` as it
  can all be handled at the API layer in a single spot.
- Optimize `destroyAllContent` in `api/db.js`. Eliminates
  double-fetching every post from the database and converting it to
  JSON. Also only fetches ids from the database instead of the entire
  model.
- Eliminates the custom static method `destroy` in the Post model in
  favor of handling detaching tag relations in a single place (the
  `destroying` event). This also eliminates a big source of unneeded
  database round trips--needing to get post ids to feed into
  `Post.destroy()` which then re-fetches the post again.
2016-03-22 10:10:09 -05:00
Hannah Wolfe
9309adc511 Exporter cleanup & tests
refs #6301

- change knex getter def to be configurable, else it is not testable
- remove exportPath and lang from config - neither are used
- add client_trusted_domains to tables which shouldn't be exported as there are no clients in the export
- change export signature to be an object with `doExport` function consistent with import & easier to test
- cleanup export code so it is clearer, easier to read & to test:
  - use mapSeries instead of sequence
  - use Promise.props instead of Promise.join
  - split functionality into smaller functions
- add test coverage
2016-03-12 19:01:25 +00:00
cobbspur
8025325b15 Backup database before deletion
No issue

- a json object is now created when a user deletes content in labs
2016-02-14 09:04:53 +00:00
Fabian Becker
3cafc70022 Correctly clear settings cache after import
fixes #6435
2016-02-04 09:23:43 +01:00
rfpe
7abcc43907 Harvest server side strings
closes #5617
- Replace all hard-coded server-side strings with i18n translations
2015-12-19 12:12:16 +01:00
Delgermurun
146bb01657 db api endpoint validation error refactor
No issue
- Raised ValidationError instead of PermissionError on db api validation
- Added & modified integration tests
2015-10-25 09:20:13 +08:00
Delgermurun
b37c0f2e91 Apply pipeline to db api endpoint
refs #5508
- adds pipeline to export, import and delete all methods
2015-10-21 09:33:31 +08:00
Hannah Wolfe
20710b9eba Refactor importer with tests
refs #4605

- Move as much code as possible out of the DB API
- Split the importer into 2 concepts, code which handles different
  filetypes and code which takes the content of files and imports it into
  Ghost in some way
- Split the import process into 4 steps, load the file into a useful
  format, preprocess the data, do the import, generate a report
- Reporting is currently a no-op
- Adds a basic level of testing
2014-12-17 18:22:39 +00:00
Hannah Wolfe
6ff51ad44e Add support for zip import
fixes #4607

- moves file checks from db and upload API endpoints to api utils
- adds code to accept and then extract a zip and pull out a JSON file
- zip handling requires a lot of dependencies - this needs a good refactor
2014-12-10 20:21:06 +00:00
Fabian Becker
72b0a48483 Fix DB API
no ref
- Use correct error type
2014-09-28 19:38:41 +00:00
Harry Wolff
469aaa398c Add jscs task to grunt file and clean up files to adhere to jscs rules.
resolves #1920

- updates all files to conform to style settings.
2014-09-17 21:44:29 -04:00
Jason Williams
07ad400ee0 Replace the when promise library with bluebird.
Closes #968
2014-08-23 17:15:40 +00:00
Hannah Wolfe
0ffc5e6d47 Improve importer error messaging
closes #3274

- Ensure that validation errors are always handled by moving them into the
  importer
- Ensure that db errors are handled consistently across sqlite and mysql
- Change the errors to be output in a table, with a short failure notification
- Add tests for 003 importing bad files
2014-07-29 12:02:18 +01:00
Sebastian Gierlinger
1eff3d9471 Fix importer bug for IE
closes #3339
- removed check for mime type
2014-07-23 16:53:03 +02:00
Sebastian Gierlinger
772d81849e Fix file validation for importer
refs #3339
- wrong call to file validation
- refs the issue because I get application/json on OSX? Is this
something Windows specific?
2014-07-20 23:07:17 +02:00
Sebastian Gierlinger
2957b0175e Move image upload to API
closes #3252
- added `/ghost/api/v0.1/uploads/` endpoint
- removed upload method from `controller/admin.js`
- moved removal of temporary files from storage to endpoint (needed to
account for failed uploads)
- changed and moved tests
- Oversight: I think that we use `.otherwise()` and `.catch()` a bit
too extensive and mask the real error objects. We probably need an
error handling strategy at some point in the future.
2014-07-15 12:40:14 +02:00
Jason Williams
13229fb6a4 Fix server-side validation
Closes #3122
-Fix validation so that all values are validated instead
 of just values that evaluate to true.
-Ensure validation methods consistently return promises
 and switch error handling from try/catch to promise.catch
 to get rid of unhandled rejection warnings.
-Add 0 and 1 to list of acceptable values in boolean validation.
2014-07-03 14:57:57 +00:00
Jason Williams
39c0f219e6 Finish Debug screen for Ember admin
Closes #2847
2014-06-25 04:36:17 +00:00
Jason Williams
9a9593536d Update when.js dependency to version 3.2.3 2014-06-16 04:11:22 +00:00
Hannah Wolfe
57a5b6a188 Full pass at inline API Docs
closes #2622, ref #2125
2014-06-03 14:05:25 +01:00
Sebastian Gierlinger
217e3ed7ad Improve status codes
closes #2187
- added UnsupportedMediaTypeError (code: 415)
- added status code 201 if a new object was created
- Updated tests
2014-05-24 08:05:12 +02:00
Hannah Wolfe
c02ebb0dcf Refactor API arguments
closes #2610, refs #2697

- cleanup API index.js, and add docs
- all API methods take consistent arguments: object & options
- browse, read, destroy take options, edit and add take object and options
- the context is passed as part of options, meaning no more .call
  everywhere
- destroy expects an object, rather than an id all the way down to the model layer
- route params such as :id, :slug, and :key are passed as an option & used
  to perform reads, updates and deletes where possible - settings / themes
  may need work here still
- HTTP posts api can find a post by slug
- Add API utils for checkData
2014-05-15 10:41:05 +01:00
Hannah Wolfe
25ec0f3f75 Merge pull request #2719 from appleYaks/import-backward-compatible
Make DB Import backwards compatible
2014-05-11 16:17:17 +01:00
Sebastian Gierlinger
fd0f5a5028 Add distinct error classes
closes #2690
- added new error classes
- moved errorhandling.js to /errors/index.js
- changed API errors to use new classes
- updated tests
2014-05-09 12:11:29 +02:00
David Arvelo
eb18991ddb Make DB Import backwards compatible
closes #2716
- Continues to parse and allow import with the old format
- Detects the presence of the new JSON-API format and allows import
2014-05-08 20:29:45 -04:00
David Arvelo
2d783ac3d4 DB API returns JSON-API compatible objects. Export triggers 'Save As' dialog.
closes #2647
- GET method returns { db: [exportedData] }
- POST, DELETE methods return { db: [] }
- 'delete all content' test updated
- Attach 'Content-Disposition' header on DB export for 'Save As' browser dialog
- Add DB API functional test for Export
2014-05-07 19:05:09 -04:00
Jacob Gable
298077582b ACL and strict rules for Settings API
Ref #2061

- Add canThis permission checks to settings api calls
- Add strict rules about accessing core settings without internal: true
- Omit core settings in browse() call unless internal: true
- Update unit tests to call api.settings with contexts
- Add a couple unit tests for new scenarios
- Update all api.settings calls in the app to call with internal context
- Re-arrange permissions.init in server startup so config.theme.update
can access settings without permissions error
2014-05-07 10:56:03 -05:00
Sebastian Gierlinger
39e654e9c3 Change error message response
closes #2643
- added error type
- added error property for validations
- wrapped errors in an array
- returns multiple errors for validation
- updated tests and admin
2014-05-05 15:51:21 +02:00
Jacob Gable
7e9880ce8d Settings API Primary Document refactor
Closes #2606

- Refactor settings api responses to { settings: [ ] } format
- Update all code using api.settings to handle new response format
- Update test stubs to return new format
- Update client site settings model to parse new format into one object of key/value pairs
- Refactor to include all setting values
- Remove unused settingsCollection method
- Update settingsCache to store all attributes
- Update settingsResult to send all attributes
- Remove unnecessary when() wraps
- Reject if editing a setting that doesn't exist
- Reject earlier if setting key is empty
- Update tests with new error messages
- Use setting.add instead of edit that was incorrectly adding
- Update importer to properly import activePlugins and installedPlugins
- Update expected setting result fields
- Fix a weird situation where hasOwnProperty didn't exist 🤷
2014-04-30 23:15:27 -05:00
Sebastian Gierlinger
e47e9c62d0 Add permissions to API
closes #2264
- added permissions check to db, users and posts
- added register method to users
- added doesUserExist method to users
- added user from session to internal calls
- changed permissible to overwrite canThis
- removed action map and action type from permissable method
2014-04-16 18:22:22 +02:00
Fabian Becker
f0c6052c73 Unlink import file when finished
fixes #2493
- Update db import version (in case it doesn't exist)
- Properly unlink file after import or error
2014-03-24 10:51:10 +01:00
Fabian Becker
1a9e91f120 Replace JSLint with JSHint.
closes #2277
- Added ES6 linting to core/client/
- Fix typeof array comparison
2014-03-04 15:47:39 +00:00
Hannah Wolfe
708ba6f8b0 Merge pull request #2220 from sebgie/issue#1401
Add validation from schema.js
2014-02-28 23:04:04 +00:00
Sebastian Gierlinger
7155d95f9d Add JSON API tests & cleanup
first 10 % of #2124
- added initial version of JSON API tests
- renamed error.errorCode to error.code
- renamed tags.all to tags.browse for consistency
2014-02-28 22:26:03 +00:00
Sebastian Gierlinger
67611045e7 Remove res.redirect from db.exportContent
closes #1654
- added frontend route /ghost/export/
- removed request handling from API
2014-02-27 16:48:38 +01:00
Sebastian Gierlinger
ac7f4f05c4 Add validation from schema.js
closes #1401
- added data/validation/index.js
- added generic validation for length
- added generic validation for nullable
- added validations object to schema.js for custom validation
- removed pyramid of doom from api/db.js
2014-02-19 18:32:23 +01:00
Sam Saccone
d9e7a05fbc Show parse error message during a failed import
adds support for outputting error message during a failed JSON.parse
This will help people identify the problem with their JSON source
and hopefully be able to aid them in resolving said issue.
2014-02-19 11:52:35 -05:00
Harry Wolff
f16dc290b7 Improve bootstrap flow of a Ghost application
addresses #1789, #1364

- Moves ./core/server/loader -> ./core/bootstrap.
The bootstrap file is only accessed once during startup,
and it’s sole job is to ensure a config.js file exists
(creating one if it doesn’t) and then validates
the contents of the config file.

Since this is directly related to the initializing 
the application is is appropriate to have 
it in the ./core folder, named bootstrap as that
is what it does.

This also improves the dependency graph, as now
the bootstrap file require’s the ./core/server/config
module and is responsible for passing in the validated
config file.

Whereas before we had ./core/server/config
require’ing ./core/server/loader and running its
init code and then passing that value back to itself,
the flow is now more straight forward of
./core/bootstrap handling initialization and then
instatiation of config module

- Merges ./core/server/config/paths into 
./core/server/config
This flow was always confusing me to that some config
options were on the config object, and some were on
the paths object.

This change now incorporates all of the variables
previously defined in config/paths directly
into the config module, and in extension,
the config.js file.

This means that you now have the option of deciding
at startup where the content directory for ghost
should reside.

- broke out loader tests in config_spec to bootstrap_spec

- updated all relevant files to now use config().paths

- moved urlFor and urlForPost function into 
 ./server/config/url.js
2014-02-07 17:34:21 -05:00
John-David Dalton
6eab7b3f92 Replace underscore with lodash. 2014-02-06 14:08:34 +00:00
Sebastian Gierlinger
6220bd19f5 Use ajax for import
closes #1854
- added blueimp file upload to debug.js
- changed POST /ghost/api/v0.1/db to be used with AJAX
- cache invalidation header should now work for import
- moved busboy middleware invocation to routes/api and routes/admin
- moved api.db.import to api.db.importContent (I hated the [] notation)
- moved api.db.export to api.db.exportContent (see above)
2014-01-06 22:55:43 +00:00