Commit Graph

378 Commits

Author SHA1 Message Date
Katharina Irrgang
8d8d7bdb26 knex migrator (#7565)
refs #7489
- remove sephiroth
- use knex migrator npm
- goodbye bootup script
- 🎨  update README
- 🎨  knex migrator @ 0.0.2
2016-10-17 13:50:29 +01:00
Katharina Irrgang
869a35c97d migrations: seeding is part of init db task (#7545)
* 🎨  move heart of fixtures to schema folder and change user model

- add fixtures.json to schema folder
- add fixture utils to schema folder
- keep all the logic!

--> FIXTURE.JSON
- add owner user with roles

--> USER MODEL
- add password as default
- findAll: allow querying inactive users when internal context (defaultFilters)
- findOne: do not remove values from original object!
- add: do not remove values from original object!

* 🔥  remove migrations key from default_settings.json

- this was a temporary invention for an older migration script
- sephiroth keep alls needed information in a migration collection

* 🔥   add code property to errors

- add code property to errors
- IMPORTANT: please share your opinion about that
- this is a copy paste behaviour of how node is doing that (errno, code etc.)
- so code specifies a GhostError

* 🎨  change error handling in versioning

- no need to throw specific database errors anymore (this was just a temporary solution)
- now: we are throwing real DatabaseVersionErrors
- specified by a code
- background: the versioning unit has not idea about seeding and population of the database
- it just throws what it knows --> database version does not exist or settings table does not exist

* 🎨  sephiroth optimisations

- added getPath function to get the path to init scripts and migration scripts
- migrationPath is still hardcoded (see TODO)
- tidy up database naming to transacting

*   migration init scripts are now complete

- 1. add tables
- 2. add fixtures
- 3. add default settings

* 🎨  important: make bootup script smaller!

- remove all TODO'S except of one
- no seeding logic in bootup script anymore 🕵🏻

*   sephiroth: allow params for init command

- param: skip (do not run this script)
- param: only (only run this script)
- very simple way

* 🎨  adapt tests and test env

- do not use migrate.populate anymore
- use sephiroth instead
- jscs/jshint

* 🎨  fix User model status checks
2016-10-12 16:18:57 +01:00
Katharina Irrgang
677502813e 🎨 replace process.env.NODE_ENV usages by config.get('env') (#7544)
closes #6629

- i had the case that in gravatar process.env.NODE_ENV was undefined and indexOf of undefined crashe my application
- so always use config to read current env
2016-10-11 13:53:52 +01:00
Katharina Irrgang
9fad7f1d69 use migration runner for init db (#7502)
refs #7489

* 🎨  protect error when creating owner
* 🎨  reset migration table
- temporary solution, see TODO's
* 🎨  use sephiroth in bootUp script
- do not populate the database
- ask sephiroth for database state
- do seeding manually (this will be removed in next seeding PR)
* 🎨  rewrite createTableIfNotExists because it causes error when running twice
- see knex issue
- hasTable and createTable
- indexes can cause trouble when calling them twice
* 🎨  tests: populate db in test env
- when forking db
- when starting ghost()
- this basically affects only the functional tests
* 🎨  server spec test adaption
- we now throw an error when database is not populated, instead of populating the database
* 🎨   migration spec adaption
- reset database now deletes migration table
- we will move the reset script into sephiroth and then we make it pretty
* 🎨  error creation adaption in bootUp
* 🎨  fixes
- sephiroth error handling
- fix tests
2016-10-11 13:37:11 +01:00
Hannah Wolfe
59e2694acf Misc Middleware cleanup (#7526)
* 💄 Combine slashes & uncapitalise middleware

- these bits of middleware belong together
- ideally they should be optimised

* 🎨 Move ghostLocals out of themeHandler

GhostLocals sets several important values which are needed for every part of the application,
admin, api and theme. Therefore, it doesn't make sense for it to be bundled in the themeHandler.

* 🐛 Fix the uncapitalise middleware

- Updated to make correct use of req.baseUrl, req.path, req.url & req.originalUrl
- Updated the tests to actually cover our weird cases

* 🎨 Move ghostVersion logic out of config

* 💄 Group static / asset-related middleware together

* 🔥 Remove /shared/ asset handling

- The 5 files which are located in `/shared/` are all handled by individual calls to `serveSharedFile`
- Therefore this code is redundant
2016-10-10 21:14:32 +02:00
Katharina Irrgang
8d9414e8ba 🎨 optimise sephiroth error handling (#7531)
refs #7489
2016-10-10 18:31:32 +01:00
Katharina Irrgang
c4fa34224f Migration runner - first iteration (#7501)
refs #7489
- add independent migratio runner
- add init script
- this is not connected to Ghost yet, but next PR will
2016-10-10 13:27:31 +01:00
Hannah Wolfe
63094d3cc7 Move internal tags out of labs (#7519)
closes #6165

- internal tags has been in labs for a couple of months, we've fixed some bugs & are ready to ship
- removes all code that tests for the labs flag
- also refactors the various usage of the visibility filter into a single util
- all the tests still pass!!!
- this marks #6165 as closed because I think the remaining UI tasks will be handled as part of a larger piece of work
2016-10-10 09:51:03 +01:00
Hannah Wolfe
c74f65683e 🎨 Change asset path to /ghost/assets (#7504)
closes #7503

- Update this server-side to serve assets from ghost/assets
- a Ghost-Admin PR changes the client to always request them from there
2016-10-07 17:05:36 -05:00
Katharina Irrgang
f9561eed29 🎨 use nullable(false) for knex schema builder (#7511)
refs #7470
2016-10-07 10:17:39 +01:00
Katharina Irrgang
e2e83a0f7b Migration: New database versioning (#7499)
refs #7489

- new database versioning scheme which is based upon the Ghost version, and so easier to reason about
- massive refactor of all the version related code 

Summary of changes:

*   new error: DatabaseNotSeeded
* 🎨  change versioning module
  - versioning is based on Ghost Version
* 🎨  change bootUp file
  - add big picture description
  - version error get's trigger from versioning module
* 🎨  default setting for database version is null
  - very important change: this is caused by the big picture
  - see bootUp description
  - the database version get's set by the seed script later
  - db version is by default null
  - 1. population happens (we ensure that this has finished, by checking if each table exists)   
  - 2. seeds happening (we ensure that seeds happend if database version is set to X.X)
* 🎨  temporary change for population logic
  - set database version after population happens
  - ensure population of default settings happend before
  - both: get's removed in next iteration
* 🎨  adapt tests && mark TODO's
* 🎨  err instance checking
2016-10-06 14:50:55 +01:00
Katharina Irrgang
d81bc91bd2 Error creation (#7477)
refs #7116, refs #2001

- Changes the way Ghost errors are implemented to benefit from proper inheritance
- Moves all error definitions into a single file
- Changes the error constructor to take an options object, rather than needing the arguments to be passed in the correct order.
- Provides a wrapper so that any errors that haven't already been converted to GhostErrors get converted before they are displayed.

Summary of changes:

* 🐛  set NODE_ENV in config handler
*   add GhostError implementation (core/server/errors.js)
  - register all errors in one file
  - inheritance from GhostError
  - option pattern
* 🔥  remove all error files
*   wrap all errors into GhostError in case of HTTP
* 🎨  adaptions
  - option pattern for errors
  - use GhostError when needed
* 🎨  revert debug deletion and add TODO for error id's
2016-10-06 13:27:35 +01:00
Katharina Irrgang
32700a0e5a 🔥 remove all migration files (#7498)
refs #7489
- a simple PR to remove all the old migration files
- remove related tests
2016-10-05 19:57:32 +01:00
Katharina Irrgang
1882278b5b 🎨 configurable logging with bunyan (#7431)
- 🛠  add bunyan and prettyjson, remove morgan

-   add logging module
  - GhostLogger class that handles setup of bunyan
  - PrettyStream for stdout

-   config for logging
  - @TODO: testing level fatal?

-   log each request via GhostLogger (express middleware)
  - @TODO: add errors to output

- 🔥  remove errors.updateActiveTheme
  - we can read the value from config

- 🔥  remove 15 helper functions in core/server/errors/index.js
  - all these functions get replaced by modules:
    1. logging
    2. error middleware handling for html/json
    3. error creation (which will be part of PR #7477)

-   add express error handler for html/json
  - one true error handler for express responses
  - contains still some TODO's, but they are not high priority for first implementation/integration
  - this middleware only takes responsibility of either rendering html responses or return json error responses

- 🎨  use new express error handler in middleware/index
  - 404 and 500 handling

- 🎨  return error instead of error message in permissions/index.js
  - the rule for error handling should be: if you call a unit, this unit should return a custom Ghost error

- 🎨  wrap serve static module
  - rule: if you call a module/unit, you should always wrap this error
  - it's always the same rule
  - so the caller never has to worry about what comes back
  - it's always a clear error instance
  - in this case: we return our notfounderror if serve static does not find the resource
  - this avoid having checks everywhere

- 🎨  replace usages of errors/index.js functions and adapt tests
  - use logging.error, logging.warn
  - make tests green
  - remove some usages of logging and throwing api errors -> because when a request is involved, logging happens automatically

- 🐛  return errorDetails to Ghost-Admin
  - errorDetails is used for Theme error handling

- 🎨  use 500er error for theme is missing error in theme-handler

- 🎨  extend file rotation to 1w
2016-10-04 16:33:43 +01:00
Hannah Wolfe
e40290af5d 🎨 Use GhostAuth name and example emails (#7475)
refs #7452

- remove references to 'patronus' in favour of GhostAuth, Note: this will require databases to be deleted ;)
- remove email addresses from test data
2016-10-03 15:11:43 +01:00
Katharina Irrgang
1867e1a75e 🎨 run database population in transaction (#7448)
* 🎨  run database population in transaction

refs #6574, refs #7432

- create transaction for creating tables
- if an error occurs or a container get's destroyed before population finishes, transaction is rolled back

* 🎨  simplify transaction creation and test
2016-09-30 14:05:17 +01:00
Katharina Irrgang
6473c9e858 Ghost OAuth (#7451)
issue #7452

Remote oauth2 authentication with Ghost.org.

This PR supports:

- oauth2 login or local login
- authentication on blog setup
- authentication on invite
- normal authentication
- does not contain many, many tests, but we'll improve in the next alpha weeks
2016-09-30 12:45:59 +01:00
kirrg001
d0356a257e 🐛 fix circular dependency in new bootUp script
closes #7440
2016-09-27 12:18:45 +02:00
kirrg001
b79a18ca8f 🎨 Separate invites from user
refs #7420
- remove invite logic from user
- add invite model and adapt affected logic for inviting team members
2016-09-26 11:08:43 +02:00
Hannah Wolfe
264661ee09 🐛 Refresh assetHash on theme override (#7430)
closes #7423

- Extend our dirty theme override cache clear hack to also reset the asset hash
_ This brings alpha into line with the LTS branch
- This still needs a rewrite for Ghost 1.0.0 🙄
2016-09-23 13:05:44 +02:00
Katharina Irrgang
32a5982430 utf8mb4 support (#7409)
closes #5519, closes #6197

- default encoding is utf8mb4 instead of utf8
- support emojis
- read here why: http://dev.mysql.com/doc/refman/5.7/en/innodb-restrictions.html
- read here why: https://dev.mysql.com/doc/refman/5.5/en/charset-unicode-conversion.html
2016-09-20 15:59:34 +01:00
Hannah Wolfe
9a520f39fb 🎨 Preparation for going alpha (#7404)
- Don't let people start Ghost Alpha with non-alpha databases.
- Provide a new welcome message for development mode (a little bit of positive reinforcment)
- Provide a RED WARNING when in production mode (will still be used for developing, but we can ignore)
- Change package.json to 1.0.0-alpha.0, we won't relelase this, will bump to .1 for release
2016-09-20 15:59:34 +01:00
kirrg001
8115d83782 🔥 remove postgres support
refs #1333, refs #5878, refs #5321
2016-09-20 15:59:34 +01:00
kirrg001
231fea4766 🎨 adaption from master merge
no issue
[ci skip]
2016-09-20 15:59:34 +01:00
kirrg001
6a97873f98 🎨 🔦 refactor content paths (images, apps, themes, storage, scheduling)
refs #6982
- create config util fn: getContentPath
- we can later let the user change the folder names in contentPath
- get rid of custom/default storage paths

[ci skip]
2016-09-20 15:59:34 +01:00
kirrg001
f4bee50e86 🎨 create assetHash if not available in asset_url helper
refs #6982
- was before generated in ConfigManager

[ci skip]
2016-09-20 15:59:34 +01:00
kirrg001
34e48dc49a 🎨 🔦 do not append isPostgreSQL to config
refs #6982
- we cannot add functions to nconf

[ci skip]
2016-09-20 15:59:34 +01:00
kirrg001
0ae0a0b490 🎨 change how we get and set config
refs #6982
- a replace for all config usages
- always use config.get or config.set
- this a pure replacement, no logic has changed

[ci skip]
2016-09-20 15:59:34 +01:00
kirrg001
0b2940d587 🎨 add getSubdir and getProtectedSlugs
refs #6982

- do not add the subdir property as static config value
- create functions to read subdir and protected slugs on runtime
2016-09-20 15:59:34 +01:00
Katharina Irrgang
c8119eee1f 🎨 source out url utils from ConfigManager (#7347)
refs #6982
2016-09-20 15:59:34 +01:00
Hannah Wolfe
a791b43f1e 🐛 😱 Fixing extract-zip derp-moment
There are no words for this!
2016-09-14 22:47:41 +01:00
David Wolfe
1cf0b940ad Do not show internal tags in RSS feed (#7385)
closes #7367

- checks tags are visibile if internal tags are enabled in labs
- dirty quick update to tests
2016-09-14 19:14:08 +01:00
Hannah Wolfe
de66a61f59 hides internal tags from displaying in meta data (#7379)
closes #7348

- only filter keywords if the internal tag lab feature is turned on
2016-09-14 17:33:24 +00:00
Katharina Irrgang
774a662fb2 🐛 re-run migration for sqlite/pg (#7323)
closes #7192

- add 008 migration
- added script to re-run 006/01
- re-run 006/01 migration for postgres in any timezone (transform formats only)
- re-run 006/01 migration for sqlite (transform formats only)
- rely on format checks for sqlite, do not check server TZ
2016-09-14 17:52:45 +01:00
Aileen Nowak
b598656ff1 Use migration to add AMP column in posts table (#7370)
no issue

Uses migration to '008' to add an `amp` column to the `posts` table.
2016-09-14 16:02:35 +01:00
Katharina Irrgang
5f4da361aa 🐛 fix 006 transform dates for sqlite server offset 0 (#7322)
refs #7192, refs #7248, refs #7351
2016-09-14 11:19:39 +01:00
Ryan McCarvill
98a17d5116 fix: Empty Sitemap.xml (#7354)
closes #7341
2016-09-14 10:44:08 +02:00
Hannah Wolfe
761d963893 💄 Output overall from -> to version during migration (#7329)
no issue

- Migrations weren't clear what version the database was
2016-09-06 13:59:54 +02:00
Katharina Irrgang
2142a9c587 🐛 fix direct blog migration and permission fixture options (#7320)
* 🐛 fix direct update

closes #7297
- move sitemap initialisation into sitemap handler
- initialise sitemap on first request to sitemap

* 🐛 fix how we pass options to migration files

refs #7317
- clone options when passing them into the migration/fixture files
- do not use default sequence, because it does not clone the arguments
2016-09-06 13:16:32 +02:00
Aileen Nowak
f7129a0e39 🎨 Util to convert relative urls in absolute (#7264)
no issue

This PR takes the existing function `processUrls` in `data/xml/rss` and refactors it to be a stand-alone util.
The change is needed, as this functionality will be accessed from `apps/amp` to convert relative URLs.
2016-08-25 07:09:40 +01:00
Hannah Wolfe
5739411c51 🐛 Ensure sitemap items are valid (#7261)
closes #7186

- Add a concept of validity to each generator
- Refactor base generator to handle invalid (empty) nodes for both events & the initial generation
- Update the tests a bit, to fix some bugs in the tests
- Ensure the homepage is always present
2016-08-25 07:13:08 +02:00
Katharina Irrgang
a91e54cf1a feature: theme upload/download/delete (#7209)
refs #7204

- added 3 new themes permissions
- change core/client
- add theme upload/download logic
- extended local file storage to serve zips
- added gscan dependency
- add ability to handle the express response within the api layer
- restrict theme upload to local file storage
- added 007 migration
2016-08-23 13:07:25 +01:00
Aileen Nowak
a5c29dfc34 [FEATURE] AMP (#7229)
closes #6588, #7095

* `ImageObject` with image dimensions (#7152, #7151, #7153)
- Returns meta data as promise
    - returns a new Promise from meta data
    - uses `Promise.props()` to resolve `getClient()` and `getMetaData()`

- Adds 'image-size' util
The util returns an object like this
```
{
    height: 50,
    url: 'http://myblog.com/images/cat.jpg',
    width: 50
};
```
if the dimensions can be fetched and rejects with error, if not.
In case we get a locally stored image or a not complete url (like `//www.gravatar.com/andsoon`), we add the protocol to the incomplete one and use `urlFor()` to get the absolute URL. If the request fails or `image-size` is not able to read the file, we reject with error.
- adds 'image-size' module to dependencies
- adds `getImageSizeFromUrl` function that returns image dimensions

- In preparation of AMP support and to improve our schema.org JSON-LD and structured data, I made the following changes:
    - Changes the following properties to be `Objects`, which have a `url` property by default and a `dimensions` property, if `width` and `height` are available:
        - `metaData.coverImage`
        - `metaData.authorImage`
        - `metaData.blog.logo`
    - Checks cache by calling `getCachedImageSizeFromUrl`. If image dimensions were fetched already, returns them from cache instead of fetching them again.
    - If we have image dimensions on hand, the output in our JSON-LD changes from normal urls to be full `ImageObjects`. Applies to all images and logos.
    - Special case for `publisher.logo` as it has size restrictions: if the image doesn't fulfil the restrictions (<=600 width and <=60 height), we simply output the url instead, so like before.
    - Adds new property for schema.org JSON-LD: `mainEntityOfPage` as an Object.
    - Adds additional Open Graph data (if we have the image size): `og:image:width` and `og:image:height`
    - Adds/updates tests

* AMP router and controller (#7171, #7157)
Implements AMP in `/apps/`:
- renders `amp.hbs` if route is `/:slug/amp/`
- updates `setResponseContext` to set context to `['amp', 'post']` for a amp post and `['amp', 'page']` for a page, but will not render amp template for a page
- updates `context_spec`
- registers 'amp' as new internal app
- adds the `amp.hbs` template to `core/server/apps/amp` which will be the default template for AMP posts.
- adds `isAmpURL` to `post-lookup`

* 🎨 Use `context` in meta as array (#7205)
Instead of reading the first value of the context array, we're checking if it includes certain context values.
This is a preparation change for AMP, where the context will be delivered as `['amp', 'post']`.

*  AMP helpers (#7174, #7216, #7215, #7223)
- Adds AMP helpers `{{amp_content}}`, `{{amp_component}}` and  `{{amp_ghost_head}}` to support AMP:
- `{{amp_content}}`:
    - Adds `Amperize` as dependency
    - AMP app uses new helper `{{amp_content}}` to render AMP HTML
    - `Amperize` transforms regular HTML into AMP HTML
    - Adds test for `{{amp_content}}` helper
    - Adds 'Sanitize-HTML` as dependendy
    - After the HTML get 'amperized' we still might have some HTML tags, which are prohibited in AMP HTML, so we use `sanitize-html` to remove those. With every update, `Amperize` gets and it is able to transform more HTML tags, they valid AMP HTML tags (e. g. `video` and `amp-video`) and will therefore not be removed.
- `{{amp_ghost_head}}`:
    - registers `{{amp_ghost_head}}` helper, but uses `{{ghost_head}}` code
    - uses `{{amp_ghost_head}}` in `amp.hbs` instead of `{{ghost_head}}`
- `{{ghost_head}}`:
    - Render `amphtml` link in metadata for post, which links to the amp post (`getAmpUrl`)
    - Updates all test in metadata to support `amp` context
    - Changes context conditionals to work with full array instead of first array value
    - Adds conditionals, so no additional javascript gets rendered in `{{ghost_head}}`
    - Removes trailing `/amp/` in URLs, so only `amphtml` link on regular post renders it
    - Adds a conditional, so no code injection will be included, for an `amp` context.
- `{{amp_components}}`:
    - AMP app uses new helper `{{amp_components}}` to render necessary script tags for AMP extended components as `amp-iframe`, `amp-anime` and `amp-form`
    - Adds test for `{{amp_components}}`
2016-08-22 18:49:27 +02:00
Katharina Irrgang
663b410fd4 feature: upload validation middleware (#7208)
no issue

- Source out validation logic into a upload validation middleware for all upload types (csv, image, subscribers). This unit can be later used for Ghost 1.0 as a pre validation core unit. 
- More usage of route tests than controller tests. These are use case tests, a use case only changes if the product changes
2016-08-18 20:25:51 +01:00
Misha Wakerman
634956e7db Removes internal tags from sitemap (#7199)
closes #7186
2016-08-13 07:00:43 -06:00
Katharina Irrgang
c6f1a8de13 fix: refresh settings cache for migration 006 (#7147)
no issue
- refresh settings cache after 006/001 migration fixture
2016-07-26 13:53:17 +01:00
Katharina Irrgang
e5a0471c9d improvement: set activeTimezone when transforming dates (#7142)
no issue
- sets `settings.activeTimezone` to best-guess based on current server time when performing the timezones migration in order to prevent unexpected changes in timezone when upgrading
2016-07-26 12:05:39 +01:00
Katharina Irrgang
c9dc367bf0 improvement: validate timezones (#7143)
no issue
- add isTimezone to schema validation
2016-07-26 10:23:20 +01:00
Katharina Irrgang
417b9b6b7c fix: enable maintenance mode only if there is an upgrade to perform (#7129)
refs #7125
- create isDatabaseOutOfDate fn for initial migration update checks
- only set maintenance.enabled to true if migration needs to happen
2016-07-26 09:56:07 +01:00
Vijay Kandy
f2d09df512 Support for custom notifications (#7077)
closes #5071

- Send application/json requests to UpdateCheck service. New UpdateCheck service accepts JSON request
- If UpdateCheck service respponse has messages[] array, iterate over the array and create custom notifications intended for current version
- Save custom notification if its not already in the store AND its uuid is not in seenNotifications array
- When a custom notification is dismissed, store its uuid in seenNotifications array
- setup test fixtures to trigger tests properly
- api_notification_spec test to ensure custom notification can be added to store and added to seenNotifications when dismissed
- update_check_spec test to ensure custom notification can be displayed for a specific Ghost version
- added test to ensure messages meant for other versions don't create notifications
2016-07-22 14:02:10 +01:00