requires https://github.com/TryGhost/Ghost/pull/12192
- added initial `EmailBatch` and `EmailRecipient` model definitions with defaults and relationships
- added missing `post` relationship function to email model
- fetch member list without bookshelf
- bookshelf can add around 3x overhead when fetching the members list for an email
- we don't need full members at this point, only having the data is fine
- if we need full models later on we can push the model hydration into background jobs where recipient batches are fetched ready for an email to be sent
- bookshelf model instantiation of many models blocks the event loop, using knex directly keeps concurrent requests fast
- store recipient list before sending email
- chunk already-fetched members list into batches and insert records into the `email_recipients` table via knex
- chunked into batches of 1000 to match the number of emails that Mailgun accepts in a single API request but this may not be the absolute fastest batch size for recipient insertion:
| Batch size | Batch time | Total time |
| ---------- | ---------- | ---------- |
| 500 | 20ms | 4142ms |
| 1000 | 50ms | 4651ms |
| 5000 | 170ms | 3540ms |
| 10000 | 370ms | 3684ms |
- create an email_batch record before inserting recipient rows so we can effeciently fetch recipients by batch and store the overall batch status
- All var declarations are now const or let as per ES6
- All comma-separated lists / chained declarations are now one declaration per line
- This is for clarity/readability but also made running the var-to-const/let switch smoother
- ESLint rules updated to match
How this was done:
- npm install -g jscodeshift
- git clone https://github.com/cpojer/js-codemod.git
- git clone git@github.com:TryGhost/Ghost.git shallow-ghost
- cd shallow-ghost
- jscodeshift -t ../js-codemod/transforms/unchain-variables.js . -v=2
- jscodeshift -t ../js-codemod/transforms/no-vars.js . -v=2
- yarn
- yarn test
- yarn lint / fix various lint errors (almost all indent) by opening files and saving in vscode
- grunt test-regression
- sorted!
- Apps are marked as removed in 3.0, never officially launched and have been deprecated for at least 2 years.
- We've slowly removed bits that got in our way or were insecure over time meaning they mostly didn't work
- This cleans up the remainder of the logic
- The tables should be cleaned up in a future major
- Apps are marked as removed in 3.0, never officially launched and have been deprecated for at least 2 years.
- We've slowly removed bits that got in our way or were insecure over time meaning they mostly didn't work
- This cleans up the remainder of the logic
- The tables should be cleaned up in a future major
no issue
* Updated sendEmailWithMagicLink syntax
* Updated label name selection from theme
* Updated migration version for labels
* Added labels to export/import of members
* Added member labels sanitization for case-insensitive duplicates
* Fixed tests
* Fixed label serialization bug on import
* Bumped @tryghost/members-api to 0.15.0
* Fixed lint
* Cleanup
refs https://github.com/TryGhost/Ghost/pull/11152
- Added subscribers table drop migration
- Removed subscribers from schema
- Removed subscribers controllers/routes/regression tests
- Removed subscriber related API code
- Removed subscribers from internal apps
- Removed subscriber importer
- Removed subscriber model
- Removed subscriber related permissions
- Removed webhook code related to subscribers
- When upgrading to v3 it is on the site admin to migrate all zapps or any other webhook clients to use members
- Removed subscriber-specific translation
- Removed subscriber lab flag
NOTE: The post metadata table split is purely an internal optimization for v3 and doesn't require or expect any external actions including related API usage in v3
We keep running into issues adding new fields to the post table because there are too many fields making the post table "too wide". We have also hit MySQL limitations in how many bytes can be in a row (64kb) with post table.
In v3, we decided to split the 8 post fields (meta, twitter and og) used for meta data into a posts_meta table as these 8 fields are all "problem" `varchar` fields and make sense logically grouped together. The API layer is unaffected by the split as input/output serializers ensure the data flow works the same way as it was in v2. Only thing to note is json export in v3 will have slightly different structure with posts meta fields as separate.
- Creates new post_meta schema/table with 8 fields (2 meta_* , 3 twitter_* and 3 og_*)
- Update relations between post and post_meta table
- Update input/output serializers to keep existing API behavior
- Avoids new entry in post_meta table for post where all meta fields are null
- Keeps the current fields API param behavior
- Handles migration of existing posts to new table structure
- Updates importer/exporter to work seamlessly with table changes
no issue
- v0.1 is ☠️ so the access/refresh token based auth is no longer used
- removed all code related to the `accesstokens` and `refreshtokens` tables
- removed all `passport` related dependencies as it's no longer used
no issue
- v0.1 is ☠️ so there's no longer any use of client auth
- removes all code related to `clients` and `client_trusted_domains`
- noops the "add backup client" migration in 1.7 because the referenced fixture no longer exists causing migrations and consequently all regression tests to fail
refs #10124
- Author model returns only users that have published non-page posts
- Added a public controller for tags (should be extracted to separate Content API controller https://github.com/TryGhost/Ghost/issues/10106)
- Made resource configuration dynamic based on current theme engine
- This needs a follow-up PR with fixes to the problems described in the PR
closes#9927
- Added post model implementation to be able to store up to 10 versions of mobiledoc
- Bumped GQL to support filtering on the mobiledoc revision table
- Added tests ensuring new functionality works
refs https://github.com/TryGhost/Ghost/issues/9865
- schema migrations
- adds `integrations` and `api_keys` tables
- inserts `integration` and `api_key` permissions and Administrator role relationships
- inserts `Admin Integration` role and permissions
- adds `Integration` model
- adds `ApiKey` model
- creates default secret if not given
- hardcodes associated role based on key type
- `admin` = `Admin API Client`
- `content` = no role
- updates `Role` model to use `bookshelf-relations` for auto cleanup of permission relationships on destroy
no issue
Support for http://resthooks.org style webhooks that can be used with Zapier triggers. This can currently be used in two ways:
a) adding a webhook record to the DB manually
b) using the API with password auth and POSTing to /webhooks/ (this is private API so not documented)
⚠️ only _https_ URLs are supported in the webhook `target_url` field 🚨
- add `webhooks` table to store event names and target urls
- add `POST` and `DELETE` endpoints for `/webhooks/`
- configure `subscribers.added` and `subscribers.deleted` events to trigger registered webhooks
closes#6406
- created listeners.js connector
- merged listeners.js with events.js (in models/base)
- set a post to draft when published_at would be in the past
- reschedule a post when published_at would be in the future
- Simplify the `init` method in `models/index.js` so that it no longer
returns a promise. Easier to use.
- Eliminates the `deleteAllContent` method from `models/index.js` as it
can all be handled at the API layer in a single spot.
- Optimize `destroyAllContent` in `api/db.js`. Eliminates
double-fetching every post from the database and converting it to
JSON. Also only fetches ids from the database instead of the entire
model.
- Eliminates the custom static method `destroy` in the Post model in
favor of handling detaching tag relations in a single place (the
`destroying` event). This also eliminates a big source of unneeded
database round trips--needing to get post ids to feed into
`Post.destroy()` which then re-fetches the post again.
closes#5492
- remove core/server/require-tree.js and split it into modules
- add read-directory module to recursively read directories
- add validate-themes module to scan themes and return errors/warnings
- add parse-package-json module to parse json and validate requirements
- rewrite core/server/models/index.js to manually require models
refs #5909, #4577
- removes accidental '.only' which was hiding issues with the findAll changes
- deleteAllContent and importer still need to use a hard 'findAll' as findPage({limit: 'all'}) doesn't have the same behaviour
Closes#4225
- If a theme is symlinked in the themes directory, follow
the symlink so that the theme object is populated correctly.
- Only do the fallback loading of theme data in the validations
module if it doesn't exist in config.
resolves#2170
- creates a models.init() function that requires all other model files
and caches them. This is opposed to the previous functionality where
when you require('./models') it would immediately require all other models.
Now it's done when you want.
- Updates all tests to reflect the new structure of the model module
Refs #2170
This removes the circular dependency problem from our models thanks to
https://github.com/tgriesser/bookshelf/issues/181
- add the registry plugin
- switch all models and collections to be registered
- switch relationships to be defined using a string, which calls from the registry
This frees us up to enforce one single point of access, thus paving
the way towards allowing us to initialize the models at are request,
and not when it's require().
addresses #2170
closes#2759closes#3027
- added oauth2orize library for server side oAuth handling
- added ember-simple-auth library for admin oAuth handling
- added tables for client, accesstoken and refreshtoken
- implemented RFC6749 4.3 Ressouce Owner Password Credentials Grant
- updated api tests with oAuth
- removed session, authentication is now token based
Known issues:
- Restore spam prevention #3128
- Signin after Signup #3125
- Signin validation #3125
**Attention**
- oldClient doesn't work with this PR anymore, session authentication
was
removed
closes#2610, refs #2697
- cleanup API index.js, and add docs
- all API methods take consistent arguments: object & options
- browse, read, destroy take options, edit and add take object and options
- the context is passed as part of options, meaning no more .call
everywhere
- destroy expects an object, rather than an id all the way down to the model layer
- route params such as :id, :slug, and :key are passed as an option & used
to perform reads, updates and deletes where possible - settings / themes
may need work here still
- HTTP posts api can find a post by slug
- Add API utils for checkData
- The API has the BREAD naming for methods
- The model now has findAll, findOne, findPage (where needed), edit, add and destroy, meaning it is similar but with a bit more flexibility
- browse, read, update, create, and delete, which were effectively just aliases, have all been removed.
- added jsDoc for the model methods
closes#2138
- Adds new models for AppField and AppSetting
- Removed permitted attributes from App model (handled by base)
- Added reference from Post to AppFields
- Added fixture data to DataGenerator
- Added integration tests for Apps, AppSettings, AppFields
- Added import for Apps
- Added app_fields to default fixtures
- Pass permissions loading to buildObjectTypeHandlers to eliminate
shared state
- Load both app and user permissions to check
- Check app permissions if present
- Create apps table and App model
- Move effectiveUserPermissions to permissions/effective
- Change permissable interface to take context; user and app.
- Add unit tests for app canThis checks and effective permissions
hooking into when ghost has finished loading
addresses item 9 in #2078
and makes progress on #2182
- has files that startup ghost return a promise
that is resolved once ghost has finished loading
- moves getSocket into config file
- removes models.reset() as it's not used anywhere
- update functions in server startup
- remove unused version hash variable
fixes#1765fixes#1811
issue #1833
New UrlFor functions
- moved body of url helper to config.path.urlFor, which can generate a URL for various scenarios
- urlFor can take a string (name) or object (relativeUrl: '/') as the first
argument - this is the first step towards issue #1833
- also added config.path.urlForPost which is async and handles getting
permalink setting
- frontend controller, ghost_head helper, cache invalidation all now use
urlFor or urlForPost all urls should be correct and consistent
URL Consistency Improvements
- refactored invalidateCache into cacheInvalidationHeader which returns a
promise so that url can be generated properly by urlForPost
- moved isPost from models to schema, and refactored schema to have a tables object
- deleted posts now return the whole object, not just id and slug,
ensuring cache invalidation header can be set on delete
- frontend controller rss and archive page redirects work properly with subdirectory
- removes {{url}} helper from admin and client, and replaced with adminUrl
helper which also uses urlFor
- in res.locals ghostRoot becomes relativeUrl, and path is removed
- changed cookieSession to session
- added session.regenerate for login and logout
- added bookshelf session store
- added session table to database
- added import for databaseVersion 001
- added grunt task test-api
- cleanup of gruntfile to start express when needed only
- moved api tests to functional tests
