closes#7866
- Importer now uses Javascript's Map instead of the normal object to ensure that tags are properly associated with their corresponding posts.
* 🛠 bookshelf tarball, bson-objectid
* 🎨 schema changes
- change increment type to string
- add a default fallback for string length 191 (to avoid adding this logic to every single column which uses an ID)
- remove uuid, because ID now represents a global resource identifier
- keep uuid for post, because we are using this as preview id
- keep uuid for clients for now - we are using this param for Ghost-Auth
* ✨ base model: generate ObjectId on creating event
- each new resource get's a auto generate ObjectId
- this logic won't work for attached models, this commit comes later
* 🎨 centralised attach method
When attaching models there are two things important two know
1. To be able to attach an ObjectId, we need to register the `onCreating` event the fetched model!This is caused by the Bookshelf design in general. On this target model we are attaching the new model.
2. We need to manually fetch the target model, because Bookshelf has a weird behaviour (which is known as a bug, see see https://github.com/tgriesser/bookshelf/issues/629). The most important property when attaching a model is `parentFk`, which is the foreign key. This can be null when fetching the model with the option `withRelated`. To ensure quality and consistency, the custom attach wrapper always fetches the target model manual. By fetching the target model (again) is a little performance decrease, but it also has advantages: we can register the event, and directly unregister the event again. So very clean code.
Important: please only use the custom attach wrapper in the future.
* 🎨 token model had overriden the onCreating function because of the created_at field
- we need to ensure that the base onCreating hook get's triggered for ALL models
- if not, they don't get an ObjectId assigned
- in this case: be smart and check if the target model has a created_at field
* 🎨 we don't have a uuid field anymore, remove the usages
- no default uuid creation in models
- i am pretty sure we have some more definitions in our tests (for example in the export json files), but that is too much work to delete them all
* 🎨 do not parse ID to Number
- we had various occurances of parsing all ID's to numbers
- we don't need this behaviour anymore
- ID is string
- i will adapt the ID validation in the next commit
* 🎨 change ID regex for validation
- we only allow: ID as ObjectId, ID as 1 and ID as me
- we need to keep ID 1, because our whole software relies on ID 1 (permissions etc)
* 🎨 owner fixture
- roles: [4] does not work anymore
- 4 means -> static id 4
- this worked in an auto increment system (not even in a system with distributed writes)
- with ObjectId we generate each ID automatically (for static and dynamic resources)
- it is possible to define all id's for static resources still, but that means we need to know which ID is already used and for consistency we have to define ObjectId's for these static resources
- so no static id's anymore, except of: id 1 for owner and id 0 for external usage (because this is required from our permission system)
- NOTE: please read through the comment in the user model
* 🎨 tests: DataGenerator and test utils
First of all: we need to ensure using ObjectId's in the tests. When don't, we can't ensure that ObjectId's work properly.
This commit brings lot's of dynamic into all the static defined id's.
In one of the next commits, i will adapt all the tests.
* 🚨 remove counter in Notification API
- no need to add a counter
- we simply generate ObjectId's (they are auto incremental as well)
- our id validator does only allow ObjectId as id,1 and me
* 🎨 extend contextUser in Base Model
- remove isNumber check, because id's are no longer numbers, except of id 0/1
- use existing isExternalUser
- support id 0/1 as string or number
* ✨ Ghost Owner has id 1
- ensure we define this id in the fixtures.json
- doesn't matter if number or string
* 🎨 functional tests adaptions
- use dynamic id's
* 🎨 fix unit tests
* 🎨 integration tests adaptions
* 🎨 change importer utils
- all our export examples (test/fixtures/exports) contain id's as numbers
- fact: but we ignore them anyway when inserting into the database, see https://github.com/TryGhost/Ghost/blob/master/core/server/data/import/utils.js#L249
- in 0e6ed957cd (diff-70f514a06347c048648be464819503c4L67) i removed parsing id's to integers
- i realised that this ^ check just existed, because the userIdToMap was an object key and object keys are always strings!
- i think this logic is a little bit complicated, but i don't want to refactor this now
- this commit ensures when trying to find the user, the id comparison works again
- i've added more documentation to understand this logic ;)
- plus i renamed an attribute to improve readability
* 🎨 Data-Generator: add more defaults to createUser
- if i use the function DataGenerator.forKnex.createUser i would like to get a full set of defaults
* 🎨 test utils: change/extend function set for functional tests
- functional tests work a bit different
- they boot Ghost and seed the database
- some functional tests have mis-used the test setup
- the test setup needs two sections: integration/unit and functional tests
- any functional test is allowed to either add more data or change data in the existing Ghost db
- but what it should not do is: add test fixtures like roles or users from our DataGenerator and cross fingers it will work
- this commit adds a clean method for functional tests to add extra users
* 🎨 functional tests adaptions
- use last commit to insert users for functional tests clean
- tidy up usage of testUtils.setup or testUtils.doAuth
* 🐛 test utils: reset database before init
- ensure we don't have any left data from other tests in the database when starting ghost
* 🐛 fix test (unrelated to this PR)
- fixes a random failure
- return statement was missing
* 🎨 make changes for invites
refs #7494, refs #7495
This PR is an extracted clean up feature of #7495.
We are using everywhere static id checks (userId === 0 or userId === 1).
This PR moves the static values into the Base model.
This makes it 1. way more readable and 2. we can change the id's in a central place.
I changed the most important occurrences - no tests are touched (yet!).
The background is: when changing from auto increment id (number) to ObjectId's (string) we still need to support id 1 and 0, because Ghost relies on these two static id's.
I would like to support using both: 0/1 as string and 0/1 as number.
1 === owner/internal
0 === external
Another important change:
User Model does not longer define the contextUser method, because i couldn't find a reason?
I looked in Git history, see 6e48275160
refs #7116, refs #2001
- Changes the way Ghost errors are implemented to benefit from proper inheritance
- Moves all error definitions into a single file
- Changes the error constructor to take an options object, rather than needing the arguments to be passed in the correct order.
- Provides a wrapper so that any errors that haven't already been converted to GhostErrors get converted before they are displayed.
Summary of changes:
* 🐛 set NODE_ENV in config handler
* ✨ add GhostError implementation (core/server/errors.js)
- register all errors in one file
- inheritance from GhostError
- option pattern
* 🔥 remove all error files
* ✨ wrap all errors into GhostError in case of HTTP
* 🎨 adaptions
- option pattern for errors
- use GhostError when needed
* 🎨 revert debug deletion and add TODO for error id's
closes#6361
- Updated bluebird dependency to latest 3.1.2.
- Updated update check to handle promises not resolving to arrays.
- Reviewed all other promise code and it looks good.
- Updated code using settle to use the new reflect function.
refs #5909, #4577
- removes accidental '.only' which was hiding issues with the findAll changes
- deleteAllContent and importer still need to use a hard 'findAll' as findPage({limit: 'all'}) doesn't have the same behaviour
closes#4778
- If import contains an owner that does not match original owner they are downgraded to admin
- Change error message for locked users to more generic message to account for imported users
- Adds duplicated user test
- Process Roles and maps import roles to db roles
- reduces export files in test fixtures
temp commit
refs #2330
- Pass through `options` to all toJSON calls on posts, tags, and users
- Use options.context.user to determine whether it's OK to return user.email
- Remove author.email handling code from frontend.js
ref #4608, #4609, #4690
- fix errors with cleaning up files
- improve handling of base directories, and introduce a simple valid format for zips (must contain importable files or folders, and may contain up to one base directory)
- vastly improve test coverage
refs #4605, #4479
- Removes versioning from the importer
- Fixes an issue with SQLITE errors not being thrown properly for posts
- Ensures that posts have a created_at date
- Makes sure that the API wrapper is properly handled
closes#4498
- remove toJSON code which returns only IDs from objects
- don't auto-include tags & fields in post responses
- don't auto-include roles in user responses
- fix #allthethings that made assumptions about the auto-includes, or otherwise were only working because of the auto-include
Issue #806
- Modify sanitize to check for duplicate posts and tags
- Update posts_tags when referencing removed duplicate tags
- Return both data and problems from importData
- Add tests for duplicate posts and tags with fixtures
Closes#1977, Refs #3473
- Ensure that import operations are run in sequence.
Previously the operations were started in order but subsequent
ops were allowed to begin before the previous finished, which would
result in out-of-order execution.
- Fix bug in attach() where a model property was being passed in
instead of a transaction object. If the call was made when a
transaction was in process, it could cause bookshelf/knex to
hang and never finish the transaction.
Closes#3844
- Before importing, all data is checked for incorrect UUIDs. If check
fails, data is sanitized and outfitted with correct UUIDS.
- Scaffolding allows for easy implementation of additional
sanitization, should that be required in the future.
- Test included, old tests updated.
fixes#3716
- change the importer to not override any user details
- only set published_by if it is not already set
- import users before anything else
- process the import and map user ids to existing users
- test fix - owner should have owner role
- test fix - catch invalid success in importer
closes#3285
- remove apps stuff for now
- if there is a single user, behave the same as before, overriding
non-critical properties of the single owner user
- if there are multiple users, import them like normal resource
Closes#3242
- Add checks for valid fields in tags and posts
- Add unit test with empty tag export data file
- Add unit test with empty post export data file
closes#3274
- Ensure that validation errors are always handled by moving them into the
importer
- Ensure that db errors are handled consistently across sqlite and mysql
- Change the errors to be output in a table, with a short failure notification
- Add tests for 003 importing bad files
fixes#3275, fixes#3290, ref #3086, ref #3084
- Ensure that we use the current logged in user and not just user 1 when
- removing hard coded user: 1 except where absolutely necessary
- passing context, rather than user to models
- base model has a new function to determine what id to use for created_by etc
closes#2610, refs #2697
- cleanup API index.js, and add docs
- all API methods take consistent arguments: object & options
- browse, read, destroy take options, edit and add take object and options
- the context is passed as part of options, meaning no more .call
everywhere
- destroy expects an object, rather than an id all the way down to the model layer
- route params such as :id, :slug, and :key are passed as an option & used
to perform reads, updates and deletes where possible - settings / themes
may need work here still
- HTTP posts api can find a post by slug
- Add API utils for checkData
closes#2643
- added error type
- added error property for validations
- wrapped errors in an array
- returns multiple errors for validation
- updated tests and admin
Closes#2606
- Refactor settings api responses to { settings: [ ] } format
- Update all code using api.settings to handle new response format
- Update test stubs to return new format
- Update client site settings model to parse new format into one object of key/value pairs
- Refactor to include all setting values
- Remove unused settingsCollection method
- Update settingsCache to store all attributes
- Update settingsResult to send all attributes
- Remove unnecessary when() wraps
- Reject if editing a setting that doesn't exist
- Reject earlier if setting key is empty
- Update tests with new error messages
- Use setting.add instead of edit that was incorrectly adding
- Update importer to properly import activePlugins and installedPlugins
- Update expected setting result fields
- Fix a weird situation where hasOwnProperty didn't exist 🤷
closes#2058
- fixed apiContext as suggested in the issue
- added user to options object for models
- added api.users.register() for public registration
- changed models to use options.user for created_by, updated_by,
author_id and published_by
- added override to session model to avoid created_by and updated_by
values
- added user (id: 1) to tests
- added user (id: 1) for registration
- added user (id: 1) for import, fixtures and default settings
- added user (id: 1) for user update
- added user (id: 1) for settings update (dbHash, installedApps, update
check)
- updated bookshelf to version 0.6.8
closes#2138
- Adds new models for AppField and AppSetting
- Removed permitted attributes from App model (handled by base)
- Added reference from Post to AppFields
- Added fixture data to DataGenerator
- Added integration tests for Apps, AppSettings, AppFields
- Added import for Apps
- Added app_fields to default fixtures
closes#1947
- added fieldtype: medium for posts.html
- changed fieldtype: medium for posts.markdown
- added method to fix databases created with wrong field type
- added tests for database version 002
closes#1681
- import doesn't override user credentials
- import doesn't override theme
- import doesn't kill session
- import does refresh the settings cache
- updated tests, they now use a fixture instead of a generated export
- tests check to ensure import is safe
fixes#1696
- this is a temp workaround until full fledged support
is added directly to bookshelfjs
- when importing we use the import json blob’s timestamps
as the value that’s set in the DB
- added tests for this change
- changed cookieSession to session
- added session.regenerate for login and logout
- added bookshelf session store
- added session table to database
- added import for databaseVersion 001
- added grunt task test-api
- cleanup of gruntfile to start express when needed only
- moved api tests to functional tests
