Commit Graph

12500 Commits

Author SHA1 Message Date
Hannah Wolfe
089655c2e2 Cleaned up boot require order, debugs & inlined mount
- Continuing cleanup and clarification of the boot process
- By locating requires and inits more closely together its easier to see what happens where and to get timings
- Improved consistency of the debug begin/end statements
- Added "Step" comments to the main process
- Inlined the mountGhost function, for readability and consistency with the other steps
2021-02-23 14:28:43 +00:00
Thibaut Patel
05b77397b6 Fixed the tests
commit 26e20d223d
2021-02-23 15:18:32 +01:00
Thibaut Patel
26e20d223d Finished to move the frontend to themes v4
issue https://github.com/TryGhost/Team/issues/221
2021-02-23 15:02:11 +01:00
Fabien O'Carroll
b5e85c55dc Added limit option to /members/events API
refs https://github.com/TryGhost/Team/issues/469

The initial implementation was just to get us going with the dashboard
but the requirements have changed now, rather than updating the code we
allow to pass the `limit` options so the Admin can choose how many
events to display.
2021-02-23 12:38:46 +00:00
Hannah Wolfe
3373b5bbd1 Cleaned up use of express in ghost-server
- There is now one true way to start Ghost - you create a server, and then call start with an express app
- We may well expand this again to improve testing pathways in future, but it will be done with a bit more clarity about expectations
2021-02-23 12:33:00 +00:00
Hannah Wolfe
3c7e2e3ac2 Cleaned up requires in boot error handling
- moved logging variable outside of the try-catch block, so it doesn't need a 2nd require
- I was considering having config,logging etc shared components required in a function together, but it's less readable
- Having a try-catch for loading shared components before main boot is also confusing
- This is simplest and most readable IMO
2021-02-23 12:27:27 +00:00
Thibaut Patel
0f59537b96 Added the v4 version to the themes engine
issue https://github.com/TryGhost/Team/issues/221
2021-02-23 11:38:44 +01:00
Hannah Wolfe
70ed998838 Replaced db.ready event with direct call to urlservice
refs dd715b33dc

- this is the last event that is used to trigger part of the standard boot process
- events make the code harder to read/reason about
- the urlservice is one of the most core and critical components in Ghost, possibly the biggest consumer of time and memory
- we want to have the work it is doing front and center so that we can improve it
2021-02-23 10:05:59 +00:00
Thibaut Patel
5b72b20d1e Update the migration number for orphaned webhooks
no issue
2021-02-23 10:46:33 +01:00
Thibaut Patel
2250cd79e1 💡 Resolve orphaned webhooks
issue https://github.com/TryGhost/Ghost/issues/12567
2021-02-23 10:14:34 +01:00
Kevin Ansfield
519faf8749
Fixed fs error during theme install deleting active theme (#12688)
closes https://github.com/TryGhost/Ghost/issues/12506

- adds an error handler that will rename the backup folder to the original name if the newly uploaded theme wasn't saved successfully
2021-02-23 08:49:48 +00:00
Kevin Ansfield
08e1268aed
Added migration to remove surrounding <> in email_batches.provider_id (#12673)
refs https://github.com/TryGhost/Team/issues/221#issuecomment-759105424

- Mailgun responds to an email send with a provider id in the format `<x@y.com>` but everywhere else it's used in their API it uses the format `x@y.com`
- updates email batch save to strip the brackets, and migration removes brackets from existing records so we no longer have to add special handling for the stored id any time we use it
2021-02-23 08:48:21 +00:00
Renovate Bot
94f5aa45e8
Update dependency lodash to v4.17.21 2021-02-22 21:53:57 +00:00
Hannah Wolfe
6ab1fb4e1d Fixed linting error
refs dd715b33dc

- we no longer use events in the theme service initialisation
2021-02-22 21:16:05 +00:00
Hannah Wolfe
dd715b33dc Replaced themes.ready event with direct call
refs b1a98b0b67

- note: I already replaced server.start with themes.ready in the above commit
- events make the code harder to read/reason about
- long term it would be nice to have a concept of hooks for services, but for now explicit is clearer
2021-02-22 21:05:06 +00:00
Renovate Bot
9c96fbbe41
Update dependency @tryghost/zip to v1.1.10 2021-02-22 17:50:30 +00:00
Thibaut Patel
b77f431f56 Added v4 frontend url config
issue https://github.com/TryGhost/Team/issues/221

We can't alias it so we have to duplicate the configuration already.
2021-02-22 18:19:57 +01:00
Thibaut Patel
a551970315 Updated the configuration to include v4 pointing to canary
issue https://github.com/TryGhost/Team/issues/221
2021-02-22 15:39:48 +01:00
Kevin Ansfield
95105836aa
Removed logging require in db/connection.js (#12690)
refs https://github.com/TryGhost/Ghost/issues/12496

- having the logging require here means that workers wanting to use the db are unable to do so without requiring logging as a side-effect
- `connection.loggingHook` does not appear to be widely used for anything outside of specific debugging scenarios when using MySQL so it should be safe to disable until a proper fix is found for workers+logging leaking file descriptors
2021-02-22 12:58:57 +00:00
Kevin Ansfield
42e452b127
Removed models require from analytics job (#12689)
refs https://github.com/TryGhost/Ghost/issues/12496

By requiring the models layer the shared logging util was being required as a side-effect causing the open file descriptors problem to continue. Removing logging from the models layer isn't feasible due to deep require chains spreading across the codebase, it's much quicker to remove the need for models in the analytics job.

- models layer was only needed because it's used by the session service
- updated analytics job to create it's own instance of `EmailAnalyticsService` rather than the default instance in order to pass in custom dependencies
- pass in custom `logging` object that uses `parentPort.postMessage` as a way of writing log output
- pass in custom `settings` object that returns settings that have been manually fetched and cached during job instantiation
2021-02-22 12:10:19 +00:00
Renovate Bot
f106de99f6
Update dependency @tryghost/vhost-middleware to v1.0.13 2021-02-22 09:30:40 +00:00
Rish
0620ff6ae0 Added new browse email endpoint
refs https://github.com/TryGhost/Ghost/issues/12633

Adds new `browse` endpoint for emails that allows Admin to check performance of newsletters over time and show stats on dashboard as primary usecase
2021-02-22 13:55:20 +05:30
Naz
12a1c60424 Added custom worker message handler
refs https://github.com/TryGhost/Ghost/issues/12496

- Handling logging in the main thread avoids file handle leaks which happen due to leaky implementation of bunyan logger (see referenced issue for more context)
- Bumped job-manager version to allow for `workerMessageHandler` callback funciton
2021-02-22 20:02:00 +13:00
Naz
8a718ca99a Migrated jobs to use parentPort.postMessage
refs https://github.com/TryGhost/Ghost/issues/12496

- Using ghost-ignition logging caused file handle leaks. As there is no straight-forward way to handle write streams with bunyan (ghost-ignition's underlying logging library) this method of logging was chosen as an alternative to keep the amount of open file handles to minimum
- The follow up changes will include custom formatter for jobs service which should make logging match the same format  as has been used inside the jobs
2021-02-22 20:02:00 +13:00
Naz
adebca422f Made addUnique/dropUnique migration utils idempodent
refs https://github.com/TryGhost/Ghost/pull/12598

- This changeset adds idepmotence to situations where unique contraint has to be dropped or added to the table
- Note '4.0/07-alter-unique-constraint-for-posts-slug.js` was migration that was effected by lack of idempotence
2021-02-22 16:13:28 +13:00
Naz
ce50653f89 Removed "object" as a valid value for settings type
refs https://github.com/TryGhost/Ghost/issues/10318

- Storing JSON object in settings has caused multiple bugs in the past and was considered an antipattern
- This is a last bit in the long process of getting rid of "object" settings in the database. At this point there should be no settings with this type. Yey!
2021-02-22 13:23:03 +13:00
Hannah Wolfe
176433e307 Refactored notify to send started + ready
- In the old boot the server wasn't started til we were ready
- In new boot, we start the server immediately and send the old started event
- Then, when we are ready to accept some traffic, we send a ready event
- At the moment, ready isn't quite sent at the right time:
   - It _should_ be when we're ready to serve real traffic, not just send 503s
   - This is after the URL generation has finished
   - But this requires more refactoring work :(
   - So for now we send when everything else is ready
- This really needs some tests
2021-02-19 20:42:10 +00:00
Hannah Wolfe
2527efd6fc Moved notify out of GhostServer
- make this a standalone module
2021-02-19 20:22:56 +00:00
Hannah Wolfe
b65cb7bd7b Renamed announceServerReadiness to notifyServerStarted
- Notify is a more familiar name e.g. systemd has the sd_notify system which this is similar to
- We're actually announcing the server started, it's not actually ready for traffic (will serve 503s)
2021-02-19 20:11:35 +00:00
Renovate Bot
b5f1c3ef6e Update dependency @tryghost/members-csv to v0.4.5 2021-02-19 16:18:18 +00:00
Hannah Wolfe
09eadad667 Removed extra console times for test utils
- Think it's pretty clear these functions take ~30ms consistently
- Not interesting to optimise these when the whole restart takes seconds
2021-02-19 15:30:40 +00:00
Hannah Wolfe
edbf9a7076 💡 Removed main script that no longer exists
refs 481e2425af

- We no longer support using Ghost as an npm module (this was deprecated long ago)
- We've now removed the file that this used to point to (in 481e2425af)
- As we aren't supporting this approach, we are removing the main script rather than updating it
2021-02-19 15:22:50 +00:00
Hannah Wolfe
b1a98b0b67 Cleaned up global server events
- use theme.ready for loading themes instead of server.start and properly clean this up
- remove server.start and server.stop as they are no longer used (only server.start was used, and only for themes)
- we're moving away from the pattern of using global events like this as they are hard to reason about
2021-02-19 15:21:58 +00:00
Hannah Wolfe
d150516ec3 Improved readability of ghost server and boot
- removed some unused code I added preivously
- refactored boot logging into a mini class for readability
2021-02-19 15:04:55 +00:00
Hannah Wolfe
481e2425af Finalised new boot process
- This commit removes the old boot process and any files that are no longer needed as a result
- Remove the duplicate event for triggering inactive themes to load
- Tidied up a few other bits
2021-02-19 13:15:12 +00:00
Thibaut Patel
9e93ed7ef0 Adding sentry logging to all ping errors
issue TryGhost/Team#362
2021-02-19 14:00:50 +01:00
Hannah Wolfe
b28b4e1cb0 Cleaned up, commented + co-located new boot
- moved the DB init back into the boot file so everything is colocated - want it to be EASY to reason about whats happening
- added some more debug, and made the method signatures all the same
- expanded on comments to make it clear what the intentions are here, as there's a lot more work to do
2021-02-19 12:27:49 +00:00
Daniel Lockyer
db1c1ce21d Changed DB path for SQLite tests
no issue

- previously for SQLite tests we were pointing to a DB file within the
  content/data folder, which would be stored on the filesystem
- by pointing this file to be under `/tmp`, the file is stored in memory
  and should be a lot quicker to access
- this works great for me! - 2x faster test suite for SQLite
- however, Windows doesn't have a `/tmp` folder so we need to rewrite that
  part to replace with `os.tmpdir()`
2021-02-19 12:21:00 +00:00
Hannah Wolfe
71521d166e Readded job scheduling to boot process
- This was missing from the new boot process!
- Added it back because obviously it is necessary :D
2021-02-19 10:09:41 +00:00
Hannah Wolfe
4b472615a8 Added Sentry to new boot process
- With the new boot finalised it's clearer to see where this needs to go
- Update comments to add more clarity around what's happening and why
- Cleaned up the version require, as this was prep for maybe using version via config, which won't work in MigratorConfig
2021-02-19 09:20:41 +00:00
Renovate Bot
ab65f70ef1
Update dependency ghost-ignition to v4.4.4 2021-02-19 04:41:24 +00:00
Renovate Bot
cda68d21a3
Update dependency @sentry/node to v6.2.0 2021-02-18 21:59:02 +00:00
Hannah Wolfe
f1be3418d9 Switched tests to use new boot method
- We currently have 2 boot systems in the codebase, until now the tests were still running against the old one
- Regression tests using the forceStart + subdir flag no longer work. These need reworking, but this _should_ be easier later
- Note those tests work fine if they were the first tests run, it is only the case of trying to restart the server with a subdirectory after starting without that doesn't work
2021-02-18 20:24:49 +00:00
Kevin Ansfield
29f34468f5
Updated mobiledoc tests for lazy loaded images (#12672)
refs 074ce4efb0
2021-02-18 19:49:40 +00:00
Kevin Ansfield
074ce4efb0 Added native lazy loading behaviour to content images
refs https://github.com/TryGhost/Ghost/issues/1325

- updated `kg-default-cards` to version that includes lazy loading attributes on image and gallery card output
- updated `kg-markdown-html-renderer` to version that includes lazy loading attributes on markdown images
2021-02-18 18:53:35 +00:00
Thibaut Patel
6e58846c94 Added sentry logging to 429 ping errors
issue https://github.com/TryGhost/Team/issues/362
2021-02-18 19:21:15 +01:00
Daniel Lockyer
a2dd7c7f7d
Unified migration logging styles
no issue

- all migrations should contain a logging message
- info for successful things, or warn if we deviated from what was
  expected for some reason
- also added some spacing to make them easier to read
2021-02-18 17:35:32 +00:00
Daniel Lockyer
bf960bcecb
Fixed typo in migration message
no issue

- simple rewording to clean up the message
2021-02-18 17:05:00 +00:00
Hannah Wolfe
5970f1df9e Split out old boot method from test utils
- clearly mark what counts as "boot" so we can swap it for new boot
2021-02-18 16:11:47 +00:00
Hannah Wolfe
469382eb95 Cleaned test utils start and stop Ghost functions
- create clarity around the two modes of "starting" ghost - one is just a reload of the DiB and some services!
- both of these modes ends the same way - with exposing fixtures and returning the server
2021-02-18 15:38:31 +00:00