Commit Graph

201 Commits

Author SHA1 Message Date
Naz
24782ead02 Cleaned up job manager test suit
no issue

- Small "boyscout" improvements that were noticed while developing a new feature. This boosts the execution time of the test suit a little bit.
2022-08-01 16:47:45 +01:00
Naz
b0a2555465 Updated job manager documentation
no issue

- Minor cleanup of the outdated and ambiguous documentation. It wasn't loved in years, so got it's tiny facelift.
2022-08-01 16:47:45 +01:00
Naz
929946d744 Added documentation for one-off jobs
refs https://github.com/TryGhost/Toolbox/issues/359
refs 1606a10ff8

- One-off jobs have been released and needed a little bit of documentation for engineers to find their feet quick with a new concept.
- One-off jobs have a quality of executing only ever once within the lifetime of Ghost instance. For example this feature enabled moving members-migrations from the main path of boot process  - boosts the boot time significantly (refed commit)
2022-08-01 16:47:45 +01:00
Naz
5adcee1528 Added ability to readd failed one-off job
refs https://github.com/TryGhost/Toolbox/issues/358

- When a one-off job fails it could be restarted during the next call, given it has been cleared from the job queue.
- This readding WILL NOT work for jobs that are restarted within same process (while being kept in the bree's queue). It's specifically targetting one-off jobs like migrations that **might** fail and are only added once per process lifetime.
2022-07-28 19:07:49 +01:00
Naz
195973eae1 Added successful job run check
refs https://github.com/TryGhost/Toolbox/issues/358

- Allows to check for a **successfull** job run and restart/re-add the job in case it was a failed one off job
2022-07-28 19:07:49 +01:00
Naz
3092b62e10 Refactored job manager tests
refs https://github.com/TryGhost/Toolbox/issues/358

- A little cleanup before adding more edge case handling
2022-07-28 19:07:49 +01:00
Naz
ee243f9ea3 Refactored job statuses
refs https://github.com/TryGhost/Toolbox/issues/358

- Without going into the model layer (schema) for a job it's hard to figure out which job statuses are available. Using an object with hard typed properties makes the code less prone to typos.
2022-07-28 19:07:49 +01:00
Naz
c88dcf749b Improved awaitCompletion interface
refs https://github.com/TryGhost/Toolbox/issues/358

- The method is a bit of a dangerous to use in cases when the job takes a long time to execute.
- Returning a boolean value did not make sense and provided no helpful information. Having a job model (or not having one) gives the context in which the "completion" happened.
2022-07-28 19:07:49 +01:00
Naz
8f95759b68 Removed unecessarily long delay in a test
no issue

- The delay is there to allow for the "IPC" (thread <-> process) communication to happen. Does not have to be this long in reality
2022-07-28 19:07:49 +01:00
Naz
6d5a5e90b1 Added job status checks to job manager
refs ttps://github.com/TryGhost/Toolbox/issues/358

- One off jobs need a way to check for prior execution and await for their completion (in cases when it is reasonably short).
- Added `hasExecuted` and `awaitCompletion` methods to the job manager allowing to monitor one off job state
2022-07-27 18:04:19 +01:00
Naz
b7853323ae 🐛 Fixed email sending job execution
refs https://github.com/TryGhost/Toolbox/issues/359

- Sending newsletters got broken because underlying "inline job" execution had a bug.
- The real problem was in the job manager trying to verify inline unnamed job status in the database without having a name.
2022-07-27 15:12:56 +01:00
Naz
7adf3a5410 Added support for one off inline jobs
refs https://github.com/TryGhost/Toolbox/issues/359

- Inline one off jobs are needed in situations when we want to run a certain operation only once in the lifecycle of the Ghost instance. These operations should not be extremely long to execute though (not suited for backups or import types of tasks)
2022-07-26 18:14:25 +01:00
Naz
822549c9ef Added defensive code for non-persistent job manager
refs https://github.com/TryGhost/Toolbox/issues/359

- It's up to a user to decide initializing the job manager without a "jobModel". In these cases the regular recurring job scheduling should work as it did before
2022-07-26 18:14:25 +01:00
Naz
2fe4f66ae3 Cleanedup job manager one off test structure
refs https://github.com/TryGhost/Toolbox/issues/359

- LIttle housekeeping was needed before introducing "inline" one-off jobs
2022-07-26 15:18:48 +01:00
Naz
013051a6c9 Made name parameter required for one off jobs
refs https://github.com/TryGhost/Toolbox/issues/359

- Without a "name" parameter it's impossible to identify a job in the storage. It was missed during the PoC inmplementation
2022-07-26 15:13:12 +01:00
renovate[bot]
679634342a
Pinned dependencies (#15100)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2022-07-26 16:06:00 +02:00
Daniel Lockyer
08abfcafd1
Remove linting as posttest step
- linting is handled separately (in CI and locally, as it's a git
  pre-push hook) so we shouldn't also be running it after tests
2022-07-26 15:26:21 +02:00
Daniel Lockyer
0a5f600dfe
Tidied up package README and LICENSE files
- we shouldn't need individual LICENSE files because these packages
  won't be published, so the top-level one applies
- also cleaned up README files to remove mentions of Lerna monorepos and
  install instructions
2022-07-26 15:22:10 +02:00
Daniel Lockyer
255eb0726b
Cleaned up package metadata
refs https://github.com/TryGhost/Toolbox/issues/354

- set packages to `private: true`
- removed repository link - these packages won't be published so this
  link won't be seen anywhere
- removed `publishConfig`
2022-07-26 15:08:05 +02:00
Daniel Lockyer
5fc7ba59d3
Reset package versions back to 0.0.0
refs https://github.com/TryGhost/Toolbox/issues/354

- these packages won't be published from now on, so setting the versions
  back to 0.0.0 keeps them clean
2022-07-26 14:57:43 +02:00
Naz
d5e38e6fc7 Published new versions
- @tryghost/adapter-manager@0.2.33
 - @tryghost/api-version-compatibility-service@0.4.4
 - @tryghost/bootstrap-socket@0.2.22
 - @tryghost/config-url-helpers@1.0.2
 - @tryghost/constants@1.0.7
 - @tryghost/database-info@0.3.8
 - @tryghost/email-content-generator@0.1.4
 - @tryghost/image-transform@1.2.1
 - @tryghost/job-manager@0.9.0
 - @tryghost/limit-service@1.2.2
 - @tryghost/minifier@0.1.17
 - @tryghost/moleculer-service-from-class@0.2.28
 - @tryghost/mw-api-version-mismatch@0.2.3
 - @tryghost/mw-error-handler@1.0.5
 - @tryghost/mw-session-from-token@0.1.34
 - @tryghost/mw-update-user-last-seen@0.1.8
 - @tryghost/package-json@1.0.23
 - @tryghost/pretty-cli@1.2.29
 - @tryghost/promise@0.1.20
 - @tryghost/release-utils@0.8.1
 - @tryghost/security@0.3.3
 - @tryghost/session-service@0.1.44
 - @tryghost/settings-path-manager@0.1.9
 - @tryghost/version-notifications-data-service@0.2.2
 - @tryghost/vhost-middleware@1.0.28
 - @tryghost/zip@1.1.27
2022-07-22 16:46:56 +01:00
Naz
df8a7f9b6b Fixed formatting 2022-07-22 16:44:28 +01:00
Naz
b0581c778e Added execution progress updates for one off jobs
refs https://github.com/TryGhost/Toolbox/issues/357

- Job persisted in the database need to track job's execution status such as completion, failure, execution start and end times. This changeset allows to hook into job/bree lifecycle to track job's progress.
- NOTE: only supports "offloaded" jobs at the moment. Support for "inline" jobs will be added once there's a clear usecase for it.
- The "started" status and "started_at" timestamp are assigned to a job at the moment when the worker thread is created inside of bree
- The "finished" status and "finished_at" timestamp are assigned to a job when a "done" event is passed from the job script (NOTE: using process.exit(0) will not trigger the "finished" state")
- The "failed" status is assigned when the job execution is interrupted with an error
2022-07-22 16:43:15 +01:00
renovate[bot]
c9ae36ea8c Update dependency date-fns to v2.29.1 2022-07-22 13:04:15 +00:00
Naz
5dae6d6acf Added support for one off jobs
refs https://github.com/TryGhost/Toolbox/issues/357

- This is a scaffolding for what will become a one off job scheduling mechanism. The aim is allowing to run jobs which can be only ever be run once in the lifetime of the instance - persisting through restarts.
2022-07-21 19:39:54 +01:00
Naz
13bfc0746b Revert "Transformed addJob to be an async method"
This reverts commit f8b1877253.
2022-07-21 18:56:35 +01:00
Naz
f8b1877253 Transformed addJob to be an async method
https://github.com/TryGhost/Toolbox/issues/357

- This is a groundwork before adding one-off (solo) jobs with persistance to the job manager
- Making the addJob method async also makes the whole interface consistent - removeJob and shutdown are also async
2022-07-20 20:41:02 +01:00
renovate[bot]
11d0ab9605 Update dependency c8 to v7.12.0 2022-07-19 18:33:27 +00:00
Daniel Lockyer
d8af7189e7 Published new versions
- @tryghost/adapter-manager@0.2.32
 - @tryghost/api-version-compatibility-service@0.4.2
 - @tryghost/bootstrap-socket@0.2.21
 - @tryghost/config-url-helpers@1.0.1
 - @tryghost/constants@1.0.6
 - @tryghost/database-info@0.3.6
 - @tryghost/email-content-generator@0.1.3
 - @tryghost/image-transform@1.0.33
 - @tryghost/job-manager@0.8.25
 - @tryghost/limit-service@1.2.1
 - @tryghost/minifier@0.1.16
 - @tryghost/moleculer-service-from-class@0.2.27
 - @tryghost/mw-api-version-mismatch@0.2.2
 - @tryghost/mw-error-handler@1.0.2
 - @tryghost/mw-session-from-token@0.1.33
 - @tryghost/mw-update-user-last-seen@0.1.7
 - @tryghost/package-json@1.0.22
 - @tryghost/pretty-cli@1.2.28
 - @tryghost/promise@0.1.19
 - @tryghost/release-utils@0.8.0
 - @tryghost/security@0.3.2
 - @tryghost/session-service@0.1.43
 - @tryghost/settings-path-manager@0.1.8
 - @tryghost/version-notifications-data-service@0.2.1
 - @tryghost/vhost-middleware@1.0.26
 - @tryghost/zip@1.1.26
2022-05-24 13:06:28 +02:00
Renovate Bot
74ff5d48de Update dependency c8 to v7.11.3 2022-05-16 04:56:38 +00:00
Naz
c5b8d073d7 Published new versions
- @tryghost/adapter-manager@0.2.31
 - @tryghost/api-version-compatibility-service@0.3.0
 - @tryghost/bootstrap-socket@0.2.20
 - @tryghost/config-url-helpers@0.1.8
 - @tryghost/constants@1.0.5
 - @tryghost/database-info@0.3.5
 - @tryghost/email-content-generator@0.1.2
 - @tryghost/image-transform@1.0.32
 - @tryghost/job-manager@0.8.24
 - @tryghost/limit-service@1.1.3
 - @tryghost/minifier@0.1.15
 - @tryghost/moleculer-service-from-class@0.2.26
 - @tryghost/mw-api-version-mismatch@0.1.4
 - @tryghost/mw-error-handler@1.0.1
 - @tryghost/mw-session-from-token@0.1.32
 - @tryghost/mw-update-user-last-seen@0.1.6
 - @tryghost/package-json@1.0.21
 - @tryghost/pretty-cli@1.2.27
 - @tryghost/promise@0.1.18
 - @tryghost/release-utils@0.7.15
 - @tryghost/security@0.3.1
 - @tryghost/session-service@0.1.42
 - @tryghost/settings-path-manager@0.1.7
 - @tryghost/update-check-service@0.3.4
 - @tryghost/version-notifications-data-service@0.1.2
 - @tryghost/zip@1.1.25
2022-05-09 18:25:48 +08:00
Renovate Bot
82b83743a7 Update dependency sinon to v14 2022-05-09 00:14:50 +00:00
Naz
e5d097fbae Published new versions
- @tryghost/adapter-manager@0.2.30
 - @tryghost/api-version-compatibility-service@0.1.2
 - @tryghost/bootstrap-socket@0.2.19
 - @tryghost/config-url-helpers@0.1.7
 - @tryghost/constants@1.0.4
 - @tryghost/database-info@0.3.4
 - @tryghost/email-content-generator@0.1.0
 - @tryghost/image-transform@1.0.31
 - @tryghost/job-manager@0.8.23
 - @tryghost/limit-service@1.1.1
 - @tryghost/minifier@0.1.14
 - @tryghost/moleculer-service-from-class@0.2.25
 - @tryghost/mw-api-version-mismatch@0.1.2
 - @tryghost/mw-error-handler@0.2.3
 - @tryghost/mw-session-from-token@0.1.31
 - @tryghost/mw-update-user-last-seen@0.1.5
 - @tryghost/package-json@1.0.20
 - @tryghost/pretty-cli@1.2.26
 - @tryghost/promise@0.1.17
 - @tryghost/release-utils@0.7.14
 - @tryghost/security@0.2.17
 - @tryghost/session-service@0.1.41
 - @tryghost/settings-path-manager@0.1.6
 - @tryghost/update-check-service@0.3.3
 - @tryghost/version-notifications-data-service@0.1.1
 - @tryghost/vhost-middleware@1.0.25
 - @tryghost/zip@1.1.24
2022-05-04 17:21:51 +08:00
Renovate Bot
6f93c82c5a Update Test & linting packages 2022-05-02 14:01:35 +00:00
Renovate Bot
9589755925 Update Test & linting packages 2022-05-02 13:58:34 +00:00
Naz
014fafbafb Published new versions
- @tryghost/adapter-manager@0.2.29
 - @tryghost/api-version-compatibility-service@0.1.0
 - @tryghost/bootstrap-socket@0.2.18
 - @tryghost/config-url-helpers@0.1.6
 - @tryghost/constants@1.0.3
 - @tryghost/database-info@0.3.2
 - @tryghost/image-transform@1.0.30
 - @tryghost/job-manager@0.8.22
 - @tryghost/limit-service@1.0.11
 - @tryghost/minifier@0.1.13
 - @tryghost/moleculer-service-from-class@0.2.24
 - @tryghost/mw-api-version-mismatch@0.1.0
 - @tryghost/mw-error-handler@0.2.1
 - @tryghost/mw-session-from-token@0.1.29
 - @tryghost/mw-update-user-last-seen@0.1.4
 - @tryghost/package-json@1.0.19
 - @tryghost/pretty-cli@1.2.25
 - @tryghost/promise@0.1.16
 - @tryghost/release-utils@0.7.13
 - @tryghost/security@0.2.16
 - @tryghost/session-service@0.1.39
 - @tryghost/settings-path-manager@0.1.5
 - @tryghost/vhost-middleware@1.0.23
 - @tryghost/zip@1.1.23
2022-04-21 20:58:17 +08:00
Renovate Bot
430608c9c3 Update dependency c8 to v7.11.2 2022-04-20 18:23:57 +00:00
Renovate Bot
65b91e4895 Update dependency @sinonjs/fake-timers to v9.1.2 2022-04-13 10:09:23 +00:00
Sam Lord
432ebed27c Published new versions
- @tryghost/job-manager@0.8.21
 - @tryghost/mw-error-handler@0.1.4
2022-03-11 10:48:00 +00:00
Renovate Bot
10e29837e5 Update dependency @sinonjs/fake-timers to v9.1.1 2022-03-07 01:24:01 +00:00
Daniel Lockyer
3324ffc788 Published new versions
- @tryghost/adapter-manager@0.2.28
 - @tryghost/bootstrap-socket@0.2.17
 - @tryghost/config-url-helpers@0.1.5
 - @tryghost/constants@1.0.2
 - @tryghost/database-info@0.2.0
 - @tryghost/image-transform@1.0.28
 - @tryghost/job-manager@0.8.20
 - @tryghost/limit-service@1.0.10
 - @tryghost/minifier@0.1.11
 - @tryghost/moleculer-service-from-class@0.2.23
 - @tryghost/mw-error-handler@0.1.3
 - @tryghost/mw-session-from-token@0.1.28
 - @tryghost/mw-update-user-last-seen@0.1.3
 - @tryghost/package-json@1.0.16
 - @tryghost/pretty-cli@1.2.24
 - @tryghost/promise@0.1.15
 - @tryghost/release-utils@0.7.12
 - @tryghost/security@0.2.15
 - @tryghost/session-service@0.1.38
 - @tryghost/settings-path-manager@0.1.4
 - @tryghost/vhost-middleware@1.0.22
 - @tryghost/zip@1.1.20
2022-03-01 08:40:52 +01:00
Daniel Lockyer
3259994e4f Added --all flag to c8 commands
refs https://github.com/TryGhost/Toolbox/issues/203

- without `--all`, c8 will ignore files that aren't covered in tests, so
  they won't pull the test coverage down
- this means we have artificially high coverage scores
- this commit adds `--all` where previously missing
2022-02-21 12:50:26 +01:00
Daniel Lockyer
cfb1ded0d0 Published new versions
- @tryghost/job-manager@0.8.19
 - @tryghost/release-utils@0.7.11
 - @tryghost/update-check-service@0.3.1
 - @tryghost/vhost-middleware@1.0.21
2022-02-04 14:27:23 +01:00
Renovate Bot
30527371fc Update dependency @sinonjs/fake-timers to v9.1.0 2022-02-03 12:13:52 +00:00
Renovate Bot
6885cdbc90 Update dependency @sinonjs/fake-timers to v9 2022-01-28 14:08:17 +00:00
Daniel Lockyer
19d2411f2b Published new versions
- @tryghost/adapter-manager@0.2.27
 - @tryghost/bootstrap-socket@0.2.16
 - @tryghost/config-url-helpers@0.1.4
 - @tryghost/constants@1.0.1
 - @tryghost/database-info@0.1.0
 - @tryghost/errors@1.2.1
 - @tryghost/image-transform@1.0.27
 - @tryghost/job-manager@0.8.18
 - @tryghost/limit-service@1.0.9
 - @tryghost/minifier@0.1.10
 - @tryghost/moleculer-service-from-class@0.2.22
 - @tryghost/mw-error-handler@0.1.2
 - @tryghost/mw-session-from-token@0.1.27
 - @tryghost/mw-update-user-last-seen@0.1.2
 - @tryghost/package-json@1.0.15
 - @tryghost/pretty-cli@1.2.23
 - @tryghost/promise@0.1.14
 - @tryghost/release-utils@0.7.10
 - @tryghost/security@0.2.14
 - @tryghost/session-service@0.1.37
 - @tryghost/settings-path-manager@0.1.3
 - @tryghost/vhost-middleware@1.0.20
 - @tryghost/zip@1.1.19
2022-01-18 09:08:09 +00:00
John O'Nolan
279ce66e71 2022 2022-01-06 09:52:35 +00:00
Renovate Bot
4c288fa50e Update dependency c8 to v7.11.0 2021-12-30 16:28:02 +00:00
Renovate Bot
e0551bd32e Update dependency date-fns to v2.28.0 2021-12-28 16:35:23 +00:00
Sam Lord
90e6e8b37d Published new versions
- @tryghost/bootstrap-socket@0.2.15
 - @tryghost/job-manager@0.8.17
2021-12-06 11:28:22 +00:00