Commit Graph

11441 Commits

Author SHA1 Message Date
Renovate Bot
e484709e73
Update dependency nock to v13.0.4 2020-08-11 22:05:36 +00:00
Renovate Bot
307ec64dde
Update dependency @tryghost/zip to v1.1.3 2020-08-11 20:35:08 +00:00
Hannah Wolfe
bbbd011074 Added job-manager & wired up shutdown and testmode
- Bottom line - we need to manage shutting down gracefully when doing long-running tasks
- To achieve that, we're going to use job queues

In this commit:
- added new @tryghost/job-manager dependency
- added a minimal job service, that handles in passing things like logging and (maybe later) config
- job service is wired up to server shutdown, so that the queue finishes before the server exits
- also added a new job endpoint to testmode so that it's easy to test job behaviour without needing to do real work
2020-08-11 21:31:34 +01:00
Renovate Bot
7b04bccd50
Update dependency @tryghost/vhost-middleware to v1.0.8 2020-08-11 19:10:07 +00:00
Hannah Wolfe
624206b6d7 Improved ghost-server with async/await
- Using consistent patterns for shutdown and stop
- Make it clear what each method does
- Use async/await to make the code more readable and simple
- This lays groundwork for having more cleanup tasks in stop than just server.stop()
2020-08-11 19:42:37 +01:00
Renovate Bot
1f69b22b30
Update dependency @tryghost/session-service to v0.1.8 2020-08-11 18:15:53 +00:00
Renovate Bot
bc3a162372
Update dependency @tryghost/mw-session-from-token to v0.1.7 2020-08-11 17:49:08 +00:00
Daniel Lockyer
5b471e1bbe Extracted promise libs and history into @tryghost/promise
- deleted files under `core/server/lib/promise` and related test files
- added `@tryghost/promise` as a dependency
- fixed all local requires to point to the new package
2020-08-11 18:44:21 +01:00
Renovate Bot
6ae884d2e4
Update dependency @tryghost/image-transform to v1.0.3 2020-08-11 16:06:44 +00:00
Renovate Bot
890bb1b90b
Update dependency @tryghost/errors to v0.2.3 2020-08-11 15:03:49 +00:00
Renovate Bot
4c056fdf65
Update dependency @tryghost/bootstrap-socket to v0.2.1 2020-08-11 13:35:53 +00:00
Renovate Bot
1444992bc9 Update dependency @tryghost/members-api to v0.25.2 2020-08-11 14:34:37 +01:00
Daniel Lockyer
b911b2bd5e Removed explicit bcryptjs dependency
- the only bit of code using it was just extracted out to
  @tryghost/security, so we can get rid of it from here
2020-08-11 14:07:41 +01:00
Daniel Lockyer
c9a5b28669 Extracted core/server/lib/security to @tryghost/security package
- code and tests were extracted out to this package
- deletes these files
- replaces all local requires, and adds it as a dependency
2020-08-11 14:06:50 +01:00
Renovate Bot
beabc62c45
Update dependency @tryghost/adapter-manager to v0.1.10 2020-08-11 11:58:38 +00:00
Renovate Bot
b43bf29f43 Update dependency @tryghost/html-to-mobiledoc to v0.7.2 2020-08-11 12:57:34 +01:00
Renovate Bot
4a67ed420a Update dependency @tryghost/magic-link to v0.4.13 2020-08-11 12:57:09 +01:00
Renovate Bot
e227522e50 Update dependency @tryghost/members-ssr to v0.8.5 2020-08-11 12:56:53 +01:00
Renovate Bot
e03ae33cb4 Update dependency @tryghost/social-urls to v0.1.11 2020-08-11 12:56:40 +01:00
Daniel Lockyer
8799feb801 Replaced constants file with @tryghost/constants
- extracted constants file into a new package
- replaced all local requires of the file with new package
2020-08-11 12:51:16 +01:00
Renovate Bot
d8880b8b97
Update dependency @tryghost/helpers to v1.1.29 2020-08-11 09:47:51 +00:00
Daniel Lockyer
2c0fc9b206 v3.29.1 2020-08-11 10:46:43 +01:00
Daniel Lockyer
1950d54c07 Updated Ghost-Admin to v3.29.1 2020-08-11 10:46:42 +01:00
Hannah Wolfe
e84621d6ef 🐛 Readded missing server.start event
closes #12118

- server.start was mistakenly removed in 71f02d25e9
- it is used for loading themes (and other things) and is critical
- added tests to prevent this regressing again in future
2020-08-11 10:09:24 +01:00
Renovate Bot
3b2d3f1d60
Update dependency sinon to v9.0.3 2020-08-11 08:01:30 +00:00
Nazar Gargol
8a7e00c413 Enabled batched members import method through enableDeveloperExperiments flag
no issue

- Allows early testing of batched import method
2020-08-11 18:52:37 +12:00
Nazar Gargol
bbcc0f5178 Added batched members import API method
no issue

- New Member API batched import is meant to be a substitution to current import
with improved performance while keeping same behaviore. Current
import processes 1 record at a time using internal API calls and times
out consistently when large number of members has to be imported (~10k
records without Stripe).
- New import's aim is to improve performance and process >50K
records without timing out both with and without Stripe connected
members
- Batched import can be conceptually devided into 3 stages which have
their own ways to improve performance:
  1. labels - can be at current performance as number of
labels is usually small, but could also be improved through batching
  2. member records + member<->labels relations - these could
be performed as batched inserts into the database
  3. Stripe connections - most challanging bottleneck to solve because
API request are slow by it's nature and have to deal with rate limits of
Stripe's API itself
- It's a heavy WIP, with lots of known pitfalls which are marked with
TODOs. Will be solved iteratively through time untill the method can be
declared stable
- The new batched import method will be hidden behind 'enableDeveloperExperiments' flag to
allow early testing
2020-08-11 18:31:31 +12:00
Renovate Bot
b61ccf0889
Update dependency @tryghost/vhost-middleware to v1.0.7 2020-08-10 21:14:18 +00:00
Renovate Bot
ca4b0cb9a0
Update dependency @tryghost/session-service to v0.1.7 2020-08-10 19:05:39 +00:00
Renovate Bot
7466218f75
Update dependency @tryghost/image-transform to v1.0.2 2020-08-10 18:05:10 +00:00
Renovate Bot
d2fb84d228
Update dependency @tryghost/errors to v0.2.2 2020-08-10 17:17:59 +00:00
Renovate Bot
0e3dd89312
Update dependency @tryghost/adapter-manager to v0.1.9 2020-08-10 16:12:34 +00:00
Daniel Lockyer
30cbfd93f8 v3.29.0 2020-08-10 17:00:10 +01:00
Daniel Lockyer
f83ee621a2 Updated Ghost-Admin to v3.29.0 2020-08-10 17:00:10 +01:00
Hannah Wolfe
b0512f2c25 Added a server testmode to help test behaviour
- A simple way to test behaviours without having to do complex interactions to e.g. generate errors or slow requests
- Makes it easier to test the new shutdown behaviour, among other things
2020-08-10 16:38:49 +01:00
Renovate Bot
a96f434163 Update dependency gscan to v3.5.6 2020-08-10 15:55:34 +01:00
Renovate Bot
edef05c2ae
Update dependency knex to v0.21.4 2020-08-10 14:10:23 +00:00
Hannah Wolfe
a9c6e081cf Added an additional log to notify shutdown start
- there can now be quite a big delay between SIGINT/TERM being received and shutdown finishing
- add an extra log message to acknowledge the SIGINT/TERM to facilitate debugging and just be clear
2020-08-10 14:53:05 +01:00
Renovate Bot
64856dc5a3
Update dependency knex-migrator to v3.4.7 2020-08-10 12:19:33 +00:00
Daniel Lockyer
226dc32ec5 Fixed default shutdown timeout
- should be 60s (60000ms), not 600s (600000ms)
2020-08-10 13:00:02 +01:00
Renovate Bot
a255426c77
Update dependency ghost-ignition to v4.2.2 2020-08-10 10:50:33 +00:00
Hannah Wolfe
19e3b70c7a Added stoppable for graceful shutdown of requests
- stopppable is a dependency that handles closing connections properly, which server.close does not
    - active connections are allowed to complete what they are doing
    - idle connections are closed
    - no new connections are allowed
- we call stoppable in stop() instead of server.close so that idle connections don't hold the server open
- calling await stop() from shutdown then ensures that we have a consistent experience of stop
- all together this allows ghost to shutdown gracefully when there are long-running requests
- @TODO: handle graceful shutdown of long-running processes
- @TODO: consider do we need to send 503s whilst the server is shutting down?
2020-08-10 11:46:36 +01:00
Hannah Wolfe
e72cc193c6 Refactored GhostServer to use a class
- use more up-to-date and less confusing syntax
- makes it easier to use async/await as well
2020-08-10 08:53:09 +01:00
Hannah Wolfe
2289b7c0f7 Moved sig event and stop message handling
- none of this should be inside the start message handling
- move to inside the start function, where it's clearer and makes more sense
2020-08-10 08:53:09 +01:00
Hannah Wolfe
31981d086b Updated stop messages to have a consistent interface
- changed method to logStopMessages, as we use start and stop, not start and shutdown
- changed logStopMesasges to output the "proper" messages and use this method consistently - the closing connections message isn't really useful
- changed uptime message to always be output cos I can't see a case where there isn't interesting/useful
2020-08-10 08:53:09 +01:00
Renovate Bot
e2adc10a52
Update dependency knex to v0.21.3 2020-08-10 06:05:05 +00:00
Renovate Bot
b56c858109
Update dependency @tryghost/zip to v1.1.2 2020-08-10 04:06:07 +00:00
Renovate Bot
45085cdd99
Update dependency @tryghost/vhost-middleware to v1.0.6 2020-08-10 02:06:10 +00:00
Renovate Bot
1f3363e0a9
Update dependency @tryghost/session-service to v0.1.6 2020-08-10 00:08:31 +00:00
Hannah Wolfe
e4ab28b70f Prevent grunt from exiting without letting express stop 2020-08-09 17:40:38 +01:00