Commit Graph

2719 Commits

Author SHA1 Message Date
kirrg001
52a9eebabf Changed comments and deprecation notes
refs #9742
2018-08-16 12:13:24 +02:00
Hannah Wolfe
290f74f815 Added advanced filtering to Dynamic Routing (#9757)
refs #9601, refs #9742

- Upgraded NQL to 0.1.0
- The new version of NQL supports aliases e.g. `tag: tags.slug`, which makes it possible to define `filter=tag:support`
- Furthermore, this allows us to support advanced filtering like tag:[a,b]
- In dynamic routing, we use mingo via NQL which has a slightly different feature set to GQL in the API:
   - AND NOT, OR and other advanced logic combos DO work on joined tables
   - Counts are not yet supported
- The Dynamic Routing beta docs should describe that API filtering and Dynamic Routing filtering is different
2018-08-16 12:13:24 +02:00
Katharina Irrgang
c39df004dc Changes for Koenig and Ghost 2.0 (#9750)
refs #9742, refs #9724

- handle König Editor format for 2.0
- adapted importer to be able to import 1.0 and 2.0 exports
- added migration scripts
  - remove labs flag for Koenig
  - migrate all old editor posts to new editor format
- ensure we protect the code against mobiledoc or html field being null
- ensure we create a blank mobiledoc structure if mobiledoc field is null (model layer)
- ensure you can fully rollback 2.0 to 1.0
- keep mobiledoc/markdown version 1 logic to be able to rollback (deprecated code)
2018-08-16 12:13:24 +02:00
kirrg001
0e521792aa Added bootstrap socket extension
refs https://github.com/TryGhost/Ghost-CLI/issues/759

- Ghost will announce the server start or failure in each case
- therefor you have to configure a bootstrap socket host and port
2018-08-16 12:13:24 +02:00
kirrg001
c40454f23c Added ability to upload/reload routes.yaml
refs #9744

- added two new endpoints to upload/download routes.yaml
- reload site express app on successful/valid upload
- reload url service on sucessfuly upload
- force clear cache of pages
- ensure we keep a backup of the routes.yaml file
- this feature was mostly tested manually
- @TODO: i have to write unit tests - will do later
- @TODO: do a memory test to ensure we haven't introduced any memory leaks with this feature
2018-08-16 12:13:24 +02:00
kirrg001
d518f23b48 Updated docs links
refs #9742

- rebase against master updated some docs links again
- go over code base again and double check that all docs links are correct
- 2.0 will become the latest version on our readme pages
2018-08-16 12:13:24 +02:00
kirrg001
23b4fd26c6 Moved knex-migrator execution into Ghost
refs #9742, refs https://github.com/TryGhost/Ghost-CLI/issues/759

- required a reordering of Ghost's bootstrap file, because:
  - we have to ensure that no database queries are executed within Ghost during the migrations
  - make 3 sections: check if db needs initialisation, bootstrap Ghost with minimal components (db/models, express apps, load settings+theme)
- create a new `migrator` utility, which tells you which state your db is in and offers an API to execute knex-migrator based on this state
- ensure we still detect an incompatible db: you connect your 2.0 blog with a 0.11 database
- enable maintenance mode if migrations are missing
- if the migration have failed, knex-migrator roll auto rollback
  - you can automatically switch to 1.0 again
- added socket communication for the CLI
2018-08-16 12:13:24 +02:00
kirrg001
90b56f925a Changed rollback messages
refs #9742

- use logging.warn
- make sure it's understandable that we are rolling back
2018-08-16 12:13:24 +02:00
kirrg001
ef355e281c ES6 migration: server/data/migrations/versions/2.0
refs #9742

- use ES6
2018-08-16 12:13:24 +02:00
kirrg001
0a556f95c7 Optimised logging for the new migration files
refs #9742
2018-08-16 12:13:24 +02:00
kirrg001
91152efdc1 Changed http to https links
no issue

- use https
- replace broken links e.g. docs.ghost.org/themes
2018-08-16 12:13:24 +02:00
kirrg001
7b9c6af9d7 Updated docs links
refs #9742

- 2.0 will become the latest version on our readme pages
2018-08-16 12:13:24 +02:00
kirrg001
529bec8089 Renamed core/server/data/export to core/server/data/exporter
no issue

- i'vejust renamed the folder
- it makes more sense
  - data/importer
  - data/exporter
2018-08-16 12:13:24 +02:00
kirrg001
ed0dc191b2 Added database migration for amp -> comment_id rename
refs #9742

- rename column from amp -> comment_id
- iterate over all posts and ensure we use the resource id or the original amp value
- provide down hook to undo this change
2018-08-16 12:13:24 +02:00
kirrg001
050e026f8b Improved debug logs in importer
no issue
2018-08-16 12:13:24 +02:00
kirrg001
ad1e9ab038 🎨 Importer disallows LTS imports
refs #9742

- Ghost 2.0 won't support LTS imports
- only Ghost 1.0 imports
2018-08-16 12:13:24 +02:00
kirrg001
e99015e700 Refactored importer tests
no issue

- these tests were a mess
- we had many duplicated tests
- it was very hard to work with the exporter files
- i have tidied up the whole file
  - first section: all general importer tests based on the current Ghost version
  - second section: 1.0 tests
- everything is now JSON based (much easier to control)
2018-08-16 12:13:24 +02:00
kirrg001
7db0739296 Renamed amp post column to column_id
refs 9742

- when we've introduced Ghost 1.0, we have noticed that we broke Disqus comments
- Disqus comments use a unique identifier, which is the post id
- that means if you have exported your LTS content and imported it into 1.0, all resource identifiers are regenerated
- but the Disqus must use the original post resource id
- that's why we have imported the old post id and remembered it in the `amp` field 🤠
- that was the only field which was available and un-used
- now in Ghost 2.0, we would like to rename the `amp` field to `comment_id`
- string length: 50 (i thought it might be safer to not use 24 characters, because who knows if support other comment id's in the future)
2018-08-16 12:13:24 +02:00
kirrg001
8aac74b3e8 Disabled bluebird debug logs
refs https://github.com/TryGhost/Ghost-CLI/issues/778

- enabling the debug logs will eat a huge amount of memory
- ensure we disable the debug logs before we require anything else
2018-08-13 14:01:31 +02:00
Kevin Ansfield
66fb0955a4 Koenig - Render image title attribute if present
refs https://github.com/TryGhost/Ghost/issues/9724
- `<img>` elements can have both `alt ` and `title` attributes, ensure we render both of them
2018-08-13 09:48:04 +01:00
Kevin Ansfield
cfd9ff3993 🎨 Koenig - Added support for shortened URLs in embed card (#9781)
refs https://github.com/TryGhost/Ghost/issues/9724

- perform a HEAD request on a url if we don't find a matching provider, following any redirects until we hit success response before looking up providers for the resulting url
2018-08-12 15:57:19 +02:00
kirrg001
860718f584 Minimized cached data on resource add/update
refs #9601

- on resource update/add we have cached mobiledoc, html etc
- we have to ensure we exclude the fields (same procdure happens on bootstrap)
- these excluded fields don't have to be cached
- otherwise memory usage is higher in general
- ensure we cache relations with a minimal field set on resource update/add
2018-08-12 14:23:32 +02:00
kirrg001
3ed5087deb Released memory in importer as early as possible
no issue

- set bigger objects to null as soon as possible
- this will trigger the GC to free memory
2018-08-12 12:22:23 +02:00
kirrg001
d421b8ccac Fixed offset calculation for SQlite3
no issue
2018-08-11 14:04:25 +02:00
kirrg001
a36191de5c Enabled useNullAsDefault for SQlite3
closes #9587

- when `useNullAsDefault` was invented in knex 0.10, it was a breaking change, that update/insert etc no longer set's null as default if a field is missing
- at this time we thought it only affects our test env, because the test generator doesn't generate all fields
- but turned out the importer is affected as well e.g. you import a post with missing fields
- the importer doesn't iterate over all fields and checks if the field is present or not
- as this only happens with SQlite3, we should enable `useNullAsDefault` by default
- you can still disable this option if you want, but not recommended
- the reason why knex added this breaking change was that some applications want "undefined" as value
- this is not the case in Ghost, so it's fine to make use of the default null behaviour
2018-08-11 13:52:03 +02:00
kirrg001
1ce504bb2d Fixed exporting extra tables for endpoint db/backup
refs #8719

- initial commit: 40c8eacd44
- we have forgotten that there is another endpoint which triggers an export (the backup endpoint)
- this endpoint needs to accept the new `include` query param as well (was missing)
2018-08-10 15:31:54 +02:00
kirrg001
08f54d2f57 Added protection against "too many SQL variables" for SQLite3
no issue

- discovered while testing
- SQLite3 has a restriction that you cannot query more than 999 SQL variables (e.g. fetch all tag relations for all posts)
- if you have more than 999 posts in your database, Ghost wasn't able to finish the bootstrap
  - the url service tried to fetch all posts & relations, but could not complete, because SQlite3 has thrown
    -> "too many SQL variables"
- i have added a recursive query strategy only for SQLite3
  - use offset/limit
2018-08-08 19:17:01 +02:00
kirrg001
6c18ceef25 Importer: Added protection for duplicate trusted domains
refs #8719
2018-08-08 19:00:23 +02:00
Kevin Ansfield
0c06a47b9b Koenig - Added rich-text caption support
refs https://github.com/TryGhost/Ghost/issues/9724
- captions can have HTML so we need to render as HTML rather than as a text node so special chars don't get escaped
2018-08-08 14:29:20 +01:00
kirrg001
75cc60c20a Added option to import clients and trusted domains
refs #9742, refs #8719

- make it possible to import more tables (optional)
  - available tables: clients, trusted domains
- by default we won't import these tables, you have to tell Ghost using `include` (same syntax on export)
- we won't announce this ability for now (stays hidden)
2018-08-03 14:12:06 +02:00
kirrg001
40c8eacd44 Added option to export extra tables
refs #9742, refs #8719

- you can now use `include` to export extra tables e.g. `include=clients`
- admin client won't make use of this option yet, maybe later and optional
- we won't announce this new ability for now (stays hidden)
2018-08-03 14:11:45 +02:00
Mandeep Singh Gulati
104a4a5c92 ES6 migration: server/api (#9756)
refs #9589
2018-07-30 12:28:05 +02:00
Kevin Ansfield
5b80ec44ab 🐛 Koenig - Fixed missing alt text on images
refs https://github.com/TryGhost/Ghost/issues/9724
- render `alt` attribute if the image card payload has an `alt` property
2018-07-30 10:10:11 +01:00
kirrg001
569018c2bc 🐛 Fixed 1.25 migration for html field being null
no issue

- it can happen that the `html` field is null
- the migration script will fail and rollback the changes automatically
2018-07-25 21:54:44 +02:00
Katharina Irrgang
6a9fe1e10c
🐛 Fixed 1.25 migration for mobiledoc field being null (#9752)
refs #9751

- the mobiledoc field can be null
- e.g. if you import a JSON with no markdown/mobiledoc or html field

The migration script for 1.25 had only the purpose to migrate existing Koenig Beta posts.
2018-07-25 11:49:04 +02:00
Katharina Irrgang
76b9a49eb8
🎨 Added Koenig Demo Post (#9747)
no issue

- add a new migration for 1.25 to insert the draft demo post for existing blogs
- ensure new blogs get the draft demo post as well
- tested on sqlite3 + mysql
- added handling if Ghost Author user doesn't exist anymore (fallback to owner user)
2018-07-24 14:37:17 +02:00
Tien Do
14d9a04fb0 ES6 migration: server/apps/amp (#9667)
refs #9589
2018-07-23 14:43:01 +02:00
Mandeep Singh Gulati
75bcfba71b ES6 migration: server/api (#9733)
refs #9589
2018-07-23 14:38:40 +02:00
Kevin Ansfield
f57268daae
Koenig - Finalise Koenig HTML output and migrate existing content (#9741)
refs https://github.com/TryGhost/Ghost/issues/9742

We've identified some changes we need to make to the HTML output of the [new Koenig editor](
https://forum.ghost.org/t/koenig-editor-beta-release/1284/102) for future proofing and consistency across cards.

- the `<div class="kg-post">` wrapper around post content has been removed
- for image cards the `.kg-image-wide` and `.kg-image-full` classes have been changed to `.kg-width-wide` and `.kg-width-full` and applied to the `<figure>` element rather than the `<img>` element

Before:
```html
<div class="kg-post">
    <figure class="kg-image-card">
        <img class="kg-image kg-image-wide" src="...">
        <figcaption>example wide image</figcaption>
    </figure>
</div>
```

After:
```html
<figure class="kg-image-card kg-width-wide">
    <img class="kg-image" src="...">
    <figcaption>example wide image</figcaption>
</figure>
```
2018-07-23 12:23:02 +01:00
kirrg001
132df78940 Updated docs links
refs #9742

- Ghost 2.0 is coming
- all doc links in 1.0 must use concrete links e.g. docs.ghost.org/v1 or themes.ghost.org/v1.23.0/
- if we release Ghost 2.0, docs.ghost.org will show 2.0 docs
2018-07-20 23:49:16 +02:00
Lars Nolden
8295f5f967 ES6 migration: server/web (#9729)
refs #9589
2018-07-12 15:35:35 +02:00
kirrg001
b335002269 🐛 Fixed Disqus comments on post preview pages
closes #9727

- this is a short term fix to proof that Disqus comments on preview pages no longer appear on other threads
- this is not a full solution to the problem
- the private API still returns /404/, which is right now inconsistent, but not critical in any way
- the url helper will now output the post preview url if you serve a draft/scheduled post
- this should register unique page urls at Disqus and ensure uniquness for threads
  - i still don't understand why the cross posting happens at all, because we also pass an unique identifier to Disqus (the post ID)
- it could be that comments, which are added on the preview page, won't appear on the published urls, because
  the published url !== preview url. I wasn't able to figure this out via testing or reading their docs
2018-07-11 00:20:55 +02:00
Kevin Ansfield
a4aab19403 🎨 Koenig - Added ID attributes to heading tags when rendering (#9720)
refs https://github.com/TryGhost/Ghost/issues/9623

- added `DomModifier` class to walk a SimpleDom document and modify as needed
  - adds `id` attributes to `h1`, `h2`, etc heading tags
    - converts H* tag content to a dasherized string for the id attribute (dasherized id's are different to the smushed ids that are generated by our markdown converted but there are no backwards-compatibility concerns here)
    - if a duplicate id is detected then add a `-1`, `-2`, etc suffix to the id
- use `DomModifier` after converting mobiledoc to SimpleDom but before serialising to html
- switched top-level var declarations to es6
2018-07-11 00:03:25 +02:00
Rosco Kalis
541713d73f 🐛Fixed slugs containing the word amp result in 404s when serving the amp version (#9722)
closes #9715

- changed the `urlWithoutSubdirectoryWithoutAmp` variable in the amp
router to only match /amp or /amp/ at the end of the url string, instead
of just matching any occurrence of /amp in the url string
2018-07-10 12:55:08 +02:00
Antony Garand
742eed99bd Fixed typo in server/api/utils (#9714)
no issue

- removed the extra `*` from the jsdoc of the handlePermissions function
2018-07-10 12:20:43 +02:00
Kevin Ansfield
865acecaef
🐛 Koenig - Support schema-less URLs in embed card (#9725)
refs https://github.com/TryGhost/Ghost/issues/9724
- adjust the "base url" regex in the oembed endpoint to strip schemaless scheme `//` as well as `https?://`
2018-07-09 10:32:39 +01:00
Kevin Ansfield
7e105eb5b6
🐛 Fixed missing location/referrer data with multiple {{subscribe_form}} (#9713)
no issue
- replaced `querySelector` with `querySelectorAll` and a loop so that all subscribe form inputs have their values updated rather than only the first form on the page
- made the selector more specific so that it only updates `<input>` elements
- switched to a template string so it's easier to read/write
2018-07-03 10:16:44 +01:00
Tien Do
4b3f3b8d0f ES6 migration: server/apps/private-blogging (#9707)
refs #9589
2018-06-28 10:50:03 +02:00
汪磊
4f22a402a3 Renamed StaticRoutesRouter debug namespace (#9706)
no issue
2018-06-28 10:44:05 +02:00
kirrg001
c5bad55d6f Dynamic Routing Beta: StaticRoutesRouter has to respect the route name
refs #9601

- this was already working for collections or channels
- but the `routeName` was not parsed for static routes
- ensure we push the route name into the context object

e.g. /about/: about

-> name of the route is "about"
2018-06-26 19:06:42 +02:00