Commit Graph

40 Commits

Author SHA1 Message Date
Katharina Irrgang
78eacb19e9 🛠 use Ignition for logging/errors (#7869)
no issue

- we started implementing logging and error handling in Ghost
- later we outsourced both into a module
- use the module now in Ghost
- this commit basically just removes the logging and error implementation and uses Ignition
2017-01-23 11:04:01 +00:00
Katharina Irrgang
7eb316b786 replace auto increment id's by object id (#7495)
* 🛠  bookshelf tarball, bson-objectid

* 🎨  schema changes

- change increment type to string
- add a default fallback for string length 191 (to avoid adding this logic to every single column which uses an ID)
- remove uuid, because ID now represents a global resource identifier
- keep uuid for post, because we are using this as preview id
- keep uuid for clients for now - we are using this param for Ghost-Auth

*   base model: generate ObjectId on creating event

- each new resource get's a auto generate ObjectId
- this logic won't work for attached models, this commit comes later

* 🎨  centralised attach method

When attaching models there are two things important two know

1. To be able to attach an ObjectId, we need to register the `onCreating` event the fetched model!This is caused by the Bookshelf design in general. On this target model we are attaching the new model.
2. We need to manually fetch the target model, because Bookshelf has a weird behaviour (which is known as a bug, see see https://github.com/tgriesser/bookshelf/issues/629). The most important property when attaching a model is `parentFk`, which is the foreign key. This can be null when fetching the model with the option `withRelated`. To ensure quality and consistency, the custom attach wrapper always fetches the target model manual. By fetching the target model (again) is a little performance decrease, but it also has advantages: we can register the event, and directly unregister the event again. So very clean code.

Important: please only use the custom attach wrapper in the future.

* 🎨  token model had overriden the onCreating function because of the created_at field

- we need to ensure that the base onCreating hook get's triggered for ALL models
- if not, they don't get an ObjectId assigned
- in this case: be smart and check if the target model has a created_at field

* 🎨  we don't have a uuid field anymore, remove the usages

- no default uuid creation in models
- i am pretty sure we have some more definitions in our tests (for example in the export json files), but that is too much work to delete them all

* 🎨  do not parse ID to Number

- we had various occurances of parsing all ID's to numbers
- we don't need this behaviour anymore
- ID is string
- i will adapt the ID validation in the next commit

* 🎨  change ID regex for validation

- we only allow: ID as ObjectId, ID as 1 and ID as me
- we need to keep ID 1, because our whole software relies on ID 1 (permissions etc)

* 🎨  owner fixture

- roles: [4] does not work anymore
- 4 means -> static id 4
- this worked in an auto increment system (not even in a system with distributed writes)
- with ObjectId we generate each ID automatically (for static and dynamic resources)
- it is possible to define all id's for static resources still, but that means we need to know which ID is already used and for consistency we have to define ObjectId's for these static resources
- so no static id's anymore, except of: id 1 for owner and id 0 for external usage (because this is required from our permission system)
- NOTE: please read through the comment in the user model


* 🎨  tests: DataGenerator and test utils

First of all: we need to ensure using ObjectId's in the tests. When don't, we can't ensure that ObjectId's work properly.
This commit brings lot's of dynamic into all the static defined id's.
In one of the next commits, i will adapt all the tests.

* 🚨  remove counter in Notification API

- no need to add a counter
- we simply generate ObjectId's (they are auto incremental as well)
- our id validator does only allow ObjectId as id,1 and me

* 🎨  extend contextUser in Base Model

- remove isNumber check, because id's are no longer numbers, except of id 0/1
- use existing isExternalUser
- support id 0/1 as string or number

*   Ghost Owner has id 1

- ensure we define this id in the fixtures.json
- doesn't matter if number or string

* 🎨  functional tests adaptions

- use dynamic id's

* 🎨  fix unit tests

* 🎨  integration tests adaptions

* 🎨  change importer utils

- all our export examples (test/fixtures/exports) contain id's as numbers
- fact: but we ignore them anyway when inserting into the database, see https://github.com/TryGhost/Ghost/blob/master/core/server/data/import/utils.js#L249
- in 0e6ed957cd (diff-70f514a06347c048648be464819503c4L67) i removed parsing id's to integers
- i realised that this ^ check just existed, because the userIdToMap was an object key and object keys are always strings!
- i think this logic is a little bit complicated, but i don't want to refactor this now
- this commit ensures when trying to find the user, the id comparison works again
- i've added more documentation to understand this logic ;)
- plus i renamed an attribute to improve readability

* 🎨  Data-Generator: add more defaults to createUser

- if i use the function DataGenerator.forKnex.createUser i would like to get a full set of defaults

* 🎨  test utils: change/extend function set for functional tests

- functional tests work a bit different
- they boot Ghost and seed the database
- some functional tests have mis-used the test setup
- the test setup needs two sections: integration/unit and functional tests
- any functional test is allowed to either add more data or change data in the existing Ghost db
- but what it should not do is: add test fixtures like roles or users from our DataGenerator and cross fingers it will work
- this commit adds a clean method for functional tests to add extra users

* 🎨  functional tests adaptions

- use last commit to insert users for functional tests clean
- tidy up usage of testUtils.setup or testUtils.doAuth

* 🐛  test utils: reset database before init

- ensure we don't have any left data from other tests in the database when starting ghost

* 🐛  fix test (unrelated to this PR)

- fixes a random failure
- return statement was missing

* 🎨  make changes for invites
2016-11-17 09:09:11 +00:00
Aileen Nowak
06061d5d6c 💄 Improve URL consistency, Part 1: urlJoin (#7668)
refs #7666

Use urlJoin for more consistency instead of concatenating url strings.
2016-11-14 14:38:55 +00:00
Katharina Irrgang
677502813e 🎨 replace process.env.NODE_ENV usages by config.get('env') (#7544)
closes #6629

- i had the case that in gravatar process.env.NODE_ENV was undefined and indexOf of undefined crashe my application
- so always use config to read current env
2016-10-11 13:53:52 +01:00
Hannah Wolfe
63094d3cc7 Move internal tags out of labs (#7519)
closes #6165

- internal tags has been in labs for a couple of months, we've fixed some bugs & are ready to ship
- removes all code that tests for the labs flag
- also refactors the various usage of the visibility filter into a single util
- all the tests still pass!!!
- this marks #6165 as closed because I think the remaining UI tasks will be handled as part of a larger piece of work
2016-10-10 09:51:03 +01:00
Katharina Irrgang
d81bc91bd2 Error creation (#7477)
refs #7116, refs #2001

- Changes the way Ghost errors are implemented to benefit from proper inheritance
- Moves all error definitions into a single file
- Changes the error constructor to take an options object, rather than needing the arguments to be passed in the correct order.
- Provides a wrapper so that any errors that haven't already been converted to GhostErrors get converted before they are displayed.

Summary of changes:

* 🐛  set NODE_ENV in config handler
*   add GhostError implementation (core/server/errors.js)
  - register all errors in one file
  - inheritance from GhostError
  - option pattern
* 🔥  remove all error files
*   wrap all errors into GhostError in case of HTTP
* 🎨  adaptions
  - option pattern for errors
  - use GhostError when needed
* 🎨  revert debug deletion and add TODO for error id's
2016-10-06 13:27:35 +01:00
Katharina Irrgang
1882278b5b 🎨 configurable logging with bunyan (#7431)
- 🛠  add bunyan and prettyjson, remove morgan

-   add logging module
  - GhostLogger class that handles setup of bunyan
  - PrettyStream for stdout

-   config for logging
  - @TODO: testing level fatal?

-   log each request via GhostLogger (express middleware)
  - @TODO: add errors to output

- 🔥  remove errors.updateActiveTheme
  - we can read the value from config

- 🔥  remove 15 helper functions in core/server/errors/index.js
  - all these functions get replaced by modules:
    1. logging
    2. error middleware handling for html/json
    3. error creation (which will be part of PR #7477)

-   add express error handler for html/json
  - one true error handler for express responses
  - contains still some TODO's, but they are not high priority for first implementation/integration
  - this middleware only takes responsibility of either rendering html responses or return json error responses

- 🎨  use new express error handler in middleware/index
  - 404 and 500 handling

- 🎨  return error instead of error message in permissions/index.js
  - the rule for error handling should be: if you call a unit, this unit should return a custom Ghost error

- 🎨  wrap serve static module
  - rule: if you call a module/unit, you should always wrap this error
  - it's always the same rule
  - so the caller never has to worry about what comes back
  - it's always a clear error instance
  - in this case: we return our notfounderror if serve static does not find the resource
  - this avoid having checks everywhere

- 🎨  replace usages of errors/index.js functions and adapt tests
  - use logging.error, logging.warn
  - make tests green
  - remove some usages of logging and throwing api errors -> because when a request is involved, logging happens automatically

- 🐛  return errorDetails to Ghost-Admin
  - errorDetails is used for Theme error handling

- 🎨  use 500er error for theme is missing error in theme-handler

- 🎨  extend file rotation to 1w
2016-10-04 16:33:43 +01:00
kirrg001
0ae0a0b490 🎨 change how we get and set config
refs #6982
- a replace for all config usages
- always use config.get or config.set
- this a pure replacement, no logic has changed

[ci skip]
2016-09-20 15:59:34 +01:00
kirrg001
0b2940d587 🎨 add getSubdir and getProtectedSlugs
refs #6982

- do not add the subdir property as static config value
- create functions to read subdir and protected slugs on runtime
2016-09-20 15:59:34 +01:00
Katharina Irrgang
c8119eee1f 🎨 source out url utils from ConfigManager (#7347)
refs #6982
2016-09-20 15:59:34 +01:00
David Wolfe
1cf0b940ad Do not show internal tags in RSS feed (#7385)
closes #7367

- checks tags are visibile if internal tags are enabled in labs
- dirty quick update to tests
2016-09-14 19:14:08 +01:00
Ryan McCarvill
98a17d5116 fix: Empty Sitemap.xml (#7354)
closes #7341
2016-09-14 10:44:08 +02:00
Katharina Irrgang
2142a9c587 🐛 fix direct blog migration and permission fixture options (#7320)
* 🐛 fix direct update

closes #7297
- move sitemap initialisation into sitemap handler
- initialise sitemap on first request to sitemap

* 🐛 fix how we pass options to migration files

refs #7317
- clone options when passing them into the migration/fixture files
- do not use default sequence, because it does not clone the arguments
2016-09-06 13:16:32 +02:00
Aileen Nowak
f7129a0e39 🎨 Util to convert relative urls in absolute (#7264)
no issue

This PR takes the existing function `processUrls` in `data/xml/rss` and refactors it to be a stand-alone util.
The change is needed, as this functionality will be accessed from `apps/amp` to convert relative URLs.
2016-08-25 07:09:40 +01:00
Hannah Wolfe
5739411c51 🐛 Ensure sitemap items are valid (#7261)
closes #7186

- Add a concept of validity to each generator
- Refactor base generator to handle invalid (empty) nodes for both events & the initial generation
- Update the tests a bit, to fix some bugs in the tests
- Ensure the homepage is always present
2016-08-25 07:13:08 +02:00
Misha Wakerman
634956e7db Removes internal tags from sitemap (#7199)
closes #7186
2016-08-13 07:00:43 -06:00
Austin Burdine
44537bd15f deps: lodash@4.13.1
closes #6911
- update lodash to v4
- remove lodash.tostring override
- remove lodash from greenkeeper ignore
2016-06-11 13:13:55 -06:00
Hannah Wolfe
f489d7df72 xmlrpc + slack init() -> listen() & fixup tests
no issue

- changes xmlrcp & slack `init` function to be `listen`
- update the code to use `listen` instead of `init`
- changes the tests to make sure that event listeners are not wired up
- adds 100% test coverage

Since we added slack event listeners, the xmlrpc event tests have been throwing an error:
 > Unhandled rejection Error
See: http://puu.sh/phvjZ.png

This is because both xmlrpc & slack are listening to `post.published` events.
xmlrpc didn't require any extra stubbing, but the slack listener did
By turning the listeners off after the tests, we reset the environment to not impact the next event test

We probably need to do more work like this to improve the systems around event handling and
make them more robust
2016-06-09 15:38:19 +01:00
Hannah Wolfe
08e5848d4a Fix cache-control header for sitemaps
no issue

- sitemaps were getting max-age=undefined as they were depending on the wrong utils folder
- test + fix included
2016-01-25 17:56:05 +00:00
Hannah Wolfe
4a7a19c86a Merge pull request #6070 from kevinkucharczyk/harvesting-server-side-strings
Harvest server side strings
2016-01-12 20:04:53 +00:00
Jacob Gable
d1348d709f Refactor RSS Image node generation
fixes #6292

- Added createImageNodeFromDatum to BaseSiteMapGenerator
- Refactor some defaults code that was unnecessary
- Add tests for posts, tags, users and posts with images
2016-01-05 21:25:52 -08:00
rfpe
7abcc43907 Harvest server side strings
closes #5617
- Replace all hard-coded server-side strings with i18n translations
2015-12-19 12:12:16 +01:00
Hannah Wolfe
e84b7f3217 Cache permalinks & postsPerPage on config.theme
no issue

- Cache the permalinks & postsPerPage settings on the config.theme object
- Use the config.theme cache to reference these items throughout the frontend of a blog
- Removes the need for workarounds and extra code to handle async fetches
- Makes these values accessible to all themes, which is very useful now we have the API stuff
2015-12-15 08:16:53 +00:00
Hannah Wolfe
8c355349b3 No more soft 404s in pagination
closes #6201

- redirects for page/1/ or rss/1/ are now 301s
- any other invalid page now 404s
2015-12-10 15:00:02 +00:00
Hannah Wolfe
395079cd2f Unify code for picking a template to render with
refs #5091

- 100% coverage for new frontend/templates file
- new module handles the logic for determining which template to render with
2015-12-01 12:05:46 +08:00
Hannah Wolfe
ff7517b801 Switch RSS to use new filter param
refs #5943, #5091

- split out channel config
- use config.theme instead of api calls to grab title & desc
- wrap rss call in a function which sets channel config for RSS feeds
- change rss `getData` function to use the new multiple-query-handling fetchData functionality
- make sure channelConfig is set in all tests
2015-10-27 10:00:51 +00:00
John O'Nolan
1add453733 Remove forum mentions 2015-09-29 13:10:04 +02:00
hwdsl2
14dd95faf2 RSS Fix: Do not convert protocol relative URLs
closes #5567
- Fixed an issue with protocol relative URLs in the RSS feed
- Such URLs should be kept as-is and not prefixed with baseUrl
- Added corresponding test to cover this case
2015-07-21 23:16:04 -05:00
Fabian Becker
4500b14989 Use previousAttributes when model is destroyed
fixes #5589
2015-07-20 14:41:40 +02:00
Fabian Becker
88a7cdbc3d Add rss.item filter 2015-07-14 09:40:31 +02:00
Hannah Wolfe
34b22eefa2 Use urlJoin in RSS + test coverage
refs #1833

- move RSS tests to be in rss_spec.js
- improve RSS test coverage
- fix a bug with RSS title generation for tag RSS feeds
- replace custom code in processUrls with urlJoin
2015-06-10 09:18:52 +01:00
Jason Williams
8196133bc7 Merge pull request #5377 from ErisDS/url-refactor
Refactor URL usage
2015-06-06 09:50:20 -05:00
Jason Williams
60e160d169 Merge pull request #5361 from ErisDS/fix-contexts
Fix context setup (remove dep on req.route.path)
2015-06-03 16:28:16 -05:00
Hannah Wolfe
2700bfa4cc Refactor URL builder
refs #1833

- Refactor url generation to use a base urlJoin method
- urlJoin handles slash de-duping and subdir de-duping
- fixes sitemap xml option
2015-06-02 21:50:17 +01:00
Hannah Wolfe
7eacd2876d Ensure middleware functions are named
refs #5091

- adds names to all middleware functions, for debugging purposes
2015-06-02 14:49:18 +01:00
Hannah Wolfe
cc00c6c64a Fix context setup (remove dep on req.route.path)
refs #5344

- As a result of #5344, context are pretty broken.
- This PR removes all dependence on req.route.path, and uses res.locals.relativeUrl
- res.locals.relativeUrl is used for many things and is dependable
2015-05-29 09:16:04 +01:00
Hannah Wolfe
f6aa46e446 Frontend route refactor / cleanup
refs #5091

- This is step one of several steps towards ending up with dynamic routes for channels
- Refactoring this way makes the similarities between all the routes clearer to see
2015-05-27 11:17:15 +01:00
Wilhansen Li
d02bcbacc2 Provide secure post links if RSS accessed over SSL 2015-04-24 18:20:38 +08:00
Hannah Wolfe
8d1e729f30 RSS Refactor with cache
refs #5091, refs #2263

- Move rss handling out of the frontend controller and into its own module
- Separate the code into logical blocks
- Wrap the generation code in a in-memory cache to prevent it being regenerated on every request
2015-04-10 21:32:14 +01:00
Hannah Wolfe
196bbfce18 Refactor sitemaps to use centralised events
fixes #5104, refs #4348, #2263

- Create a centralised event module
- Hook it up for posts, pages, tags and users
- Use it in sitemaps instead of direct method calls
- Use it for xmlrpc calls
- Check events are fired in model tests
- Update sitemap tests to work with new code
- Fix a bug where invited users were appearing in sitemaps
- Move sitemaps and xmlrpc into a directory together
2015-04-05 19:32:18 +01:00