Commit Graph

208 Commits

Author SHA1 Message Date
Daniel Lockyer
7a2f766668
Added logging configuration option for timestamps to use the local timezone
fixes https://github.com/TryGhost/Ghost/issues/15190
refs https://github.com/TryGhost/framework/pull/76

- log output always uses UTC timestamps, but it may be desirable to
  configure logs to use the local machine timezone
- a new config option has been added to `@tryghost/logging` so you can
  switch the logs to the local timezone
- this commit bumps the package and sets the default config option to
  `false`, so it doesn't suddenly change the timezone of the logs
- docs will be updated soon but if you'd like to use the
  timezone-altered timestamps, you can set `logging.useLocalTime` to
  `true`
- credits to https://github.com/levee223 for the implementation and PR
2022-08-31 10:29:55 +01:00
renovate[bot]
02d66ee30a Update dependency date-fns to v2.29.2 2022-08-18 17:15:58 +02:00
renovate[bot]
5f4c170d05 Update dependency @tryghost/errors to v1.2.15 2022-08-18 12:02:38 +02:00
Daniel Lockyer
f51226e5fb Organized package dependencies
- cleaned up unused dependencies
- adds missing dependencies that are used in the code
- this should help us be more explicit about the dependencies a package
  uses
2022-08-18 11:55:49 +02:00
Daniel Lockyer
54aa9f016b Fixed full Admin test suite running during unit tests
- because of how the npm scripts were set up, we were running the full
  Admin integration tests during the unit tests phase of CI
- this commit renames the majority of `test` to `test:unit` in the
  package.json files, and aliases `test` to `test:unit`
- special packages like Admin have no-op'd `test:unit` scripts so we
  don't end up running its tests
2022-08-15 15:34:52 +02:00
Simon Backx
82a3133ace
Added replies/likes count and limited replies to comments (#15206)
refs https://github.com/TryGhost/Team/issues/1723

- Added count.replies to comments
- Added replies endpoint
- Limited returned replies to 3.
- Replaced likes_count with count.likes in comments
- Instead of fetching all the likes of a comment to determine the total count, we'll now use count.likes
- Instead of fetching all the likes of a comment to determine whether a member liked a comment, we'll now use count.liked (which returns the amount of likes of the current member, being 0 or 1). This is mapped to `liked` to make it more natural to work with.

The `members.test.snap` file changed because we no longer include `liked: false` if we didn't fetch the liked relation. And in the comments events of the activity feed the liked property is therefore removed.

These changes requires an update to the `bookshelf-include-count` plugin:
- Updated to also work for nested relations
- This moves the count queries from the `bookshelf-include-count` plugin to the `countRelations` method of each model.
- Updated to keep the counts after saving a model (crud.edit didn't return the counts before)
2022-08-10 16:12:35 +02:00
Daniel Lockyer
c2e45b657f Removed bthreads dependency in favor of native worker_threads
fixes https://github.com/TryGhost/Toolbox/issues/370

- we no longer need `bthreads` because we can use native
  `worker_threads` now we don't have to support Node 10 any longer
- this allows us to clean up a dependency and stick with native
  libraries
- the referenced node-sqlite3 issue should be fixed (or at least, we now
  maintain it so we can fix it if not)
2022-08-09 09:04:59 +02:00
Naz
24782ead02 Cleaned up job manager test suit
no issue

- Small "boyscout" improvements that were noticed while developing a new feature. This boosts the execution time of the test suit a little bit.
2022-08-01 16:47:45 +01:00
Naz
b0a2555465 Updated job manager documentation
no issue

- Minor cleanup of the outdated and ambiguous documentation. It wasn't loved in years, so got it's tiny facelift.
2022-08-01 16:47:45 +01:00
Naz
929946d744 Added documentation for one-off jobs
refs https://github.com/TryGhost/Toolbox/issues/359
refs 1606a10ff8

- One-off jobs have been released and needed a little bit of documentation for engineers to find their feet quick with a new concept.
- One-off jobs have a quality of executing only ever once within the lifetime of Ghost instance. For example this feature enabled moving members-migrations from the main path of boot process  - boosts the boot time significantly (refed commit)
2022-08-01 16:47:45 +01:00
Naz
5adcee1528 Added ability to readd failed one-off job
refs https://github.com/TryGhost/Toolbox/issues/358

- When a one-off job fails it could be restarted during the next call, given it has been cleared from the job queue.
- This readding WILL NOT work for jobs that are restarted within same process (while being kept in the bree's queue). It's specifically targetting one-off jobs like migrations that **might** fail and are only added once per process lifetime.
2022-07-28 19:07:49 +01:00
Naz
195973eae1 Added successful job run check
refs https://github.com/TryGhost/Toolbox/issues/358

- Allows to check for a **successfull** job run and restart/re-add the job in case it was a failed one off job
2022-07-28 19:07:49 +01:00
Naz
3092b62e10 Refactored job manager tests
refs https://github.com/TryGhost/Toolbox/issues/358

- A little cleanup before adding more edge case handling
2022-07-28 19:07:49 +01:00
Naz
ee243f9ea3 Refactored job statuses
refs https://github.com/TryGhost/Toolbox/issues/358

- Without going into the model layer (schema) for a job it's hard to figure out which job statuses are available. Using an object with hard typed properties makes the code less prone to typos.
2022-07-28 19:07:49 +01:00
Naz
c88dcf749b Improved awaitCompletion interface
refs https://github.com/TryGhost/Toolbox/issues/358

- The method is a bit of a dangerous to use in cases when the job takes a long time to execute.
- Returning a boolean value did not make sense and provided no helpful information. Having a job model (or not having one) gives the context in which the "completion" happened.
2022-07-28 19:07:49 +01:00
Naz
8f95759b68 Removed unecessarily long delay in a test
no issue

- The delay is there to allow for the "IPC" (thread <-> process) communication to happen. Does not have to be this long in reality
2022-07-28 19:07:49 +01:00
Naz
6d5a5e90b1 Added job status checks to job manager
refs ttps://github.com/TryGhost/Toolbox/issues/358

- One off jobs need a way to check for prior execution and await for their completion (in cases when it is reasonably short).
- Added `hasExecuted` and `awaitCompletion` methods to the job manager allowing to monitor one off job state
2022-07-27 18:04:19 +01:00
Naz
b7853323ae 🐛 Fixed email sending job execution
refs https://github.com/TryGhost/Toolbox/issues/359

- Sending newsletters got broken because underlying "inline job" execution had a bug.
- The real problem was in the job manager trying to verify inline unnamed job status in the database without having a name.
2022-07-27 15:12:56 +01:00
Naz
7adf3a5410 Added support for one off inline jobs
refs https://github.com/TryGhost/Toolbox/issues/359

- Inline one off jobs are needed in situations when we want to run a certain operation only once in the lifecycle of the Ghost instance. These operations should not be extremely long to execute though (not suited for backups or import types of tasks)
2022-07-26 18:14:25 +01:00
Naz
822549c9ef Added defensive code for non-persistent job manager
refs https://github.com/TryGhost/Toolbox/issues/359

- It's up to a user to decide initializing the job manager without a "jobModel". In these cases the regular recurring job scheduling should work as it did before
2022-07-26 18:14:25 +01:00
Naz
2fe4f66ae3 Cleanedup job manager one off test structure
refs https://github.com/TryGhost/Toolbox/issues/359

- LIttle housekeeping was needed before introducing "inline" one-off jobs
2022-07-26 15:18:48 +01:00
Naz
013051a6c9 Made name parameter required for one off jobs
refs https://github.com/TryGhost/Toolbox/issues/359

- Without a "name" parameter it's impossible to identify a job in the storage. It was missed during the PoC inmplementation
2022-07-26 15:13:12 +01:00
renovate[bot]
679634342a
Pinned dependencies (#15100)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2022-07-26 16:06:00 +02:00
Daniel Lockyer
08abfcafd1
Remove linting as posttest step
- linting is handled separately (in CI and locally, as it's a git
  pre-push hook) so we shouldn't also be running it after tests
2022-07-26 15:26:21 +02:00
Daniel Lockyer
0a5f600dfe
Tidied up package README and LICENSE files
- we shouldn't need individual LICENSE files because these packages
  won't be published, so the top-level one applies
- also cleaned up README files to remove mentions of Lerna monorepos and
  install instructions
2022-07-26 15:22:10 +02:00
Daniel Lockyer
255eb0726b
Cleaned up package metadata
refs https://github.com/TryGhost/Toolbox/issues/354

- set packages to `private: true`
- removed repository link - these packages won't be published so this
  link won't be seen anywhere
- removed `publishConfig`
2022-07-26 15:08:05 +02:00
Daniel Lockyer
5fc7ba59d3
Reset package versions back to 0.0.0
refs https://github.com/TryGhost/Toolbox/issues/354

- these packages won't be published from now on, so setting the versions
  back to 0.0.0 keeps them clean
2022-07-26 14:57:43 +02:00
Naz
d5e38e6fc7 Published new versions
- @tryghost/adapter-manager@0.2.33
 - @tryghost/api-version-compatibility-service@0.4.4
 - @tryghost/bootstrap-socket@0.2.22
 - @tryghost/config-url-helpers@1.0.2
 - @tryghost/constants@1.0.7
 - @tryghost/database-info@0.3.8
 - @tryghost/email-content-generator@0.1.4
 - @tryghost/image-transform@1.2.1
 - @tryghost/job-manager@0.9.0
 - @tryghost/limit-service@1.2.2
 - @tryghost/minifier@0.1.17
 - @tryghost/moleculer-service-from-class@0.2.28
 - @tryghost/mw-api-version-mismatch@0.2.3
 - @tryghost/mw-error-handler@1.0.5
 - @tryghost/mw-session-from-token@0.1.34
 - @tryghost/mw-update-user-last-seen@0.1.8
 - @tryghost/package-json@1.0.23
 - @tryghost/pretty-cli@1.2.29
 - @tryghost/promise@0.1.20
 - @tryghost/release-utils@0.8.1
 - @tryghost/security@0.3.3
 - @tryghost/session-service@0.1.44
 - @tryghost/settings-path-manager@0.1.9
 - @tryghost/version-notifications-data-service@0.2.2
 - @tryghost/vhost-middleware@1.0.28
 - @tryghost/zip@1.1.27
2022-07-22 16:46:56 +01:00
Naz
df8a7f9b6b Fixed formatting 2022-07-22 16:44:28 +01:00
Naz
b0581c778e Added execution progress updates for one off jobs
refs https://github.com/TryGhost/Toolbox/issues/357

- Job persisted in the database need to track job's execution status such as completion, failure, execution start and end times. This changeset allows to hook into job/bree lifecycle to track job's progress.
- NOTE: only supports "offloaded" jobs at the moment. Support for "inline" jobs will be added once there's a clear usecase for it.
- The "started" status and "started_at" timestamp are assigned to a job at the moment when the worker thread is created inside of bree
- The "finished" status and "finished_at" timestamp are assigned to a job when a "done" event is passed from the job script (NOTE: using process.exit(0) will not trigger the "finished" state")
- The "failed" status is assigned when the job execution is interrupted with an error
2022-07-22 16:43:15 +01:00
renovate[bot]
c9ae36ea8c Update dependency date-fns to v2.29.1 2022-07-22 13:04:15 +00:00
Naz
5dae6d6acf Added support for one off jobs
refs https://github.com/TryGhost/Toolbox/issues/357

- This is a scaffolding for what will become a one off job scheduling mechanism. The aim is allowing to run jobs which can be only ever be run once in the lifetime of the instance - persisting through restarts.
2022-07-21 19:39:54 +01:00
Naz
13bfc0746b Revert "Transformed addJob to be an async method"
This reverts commit f8b1877253.
2022-07-21 18:56:35 +01:00
Naz
f8b1877253 Transformed addJob to be an async method
https://github.com/TryGhost/Toolbox/issues/357

- This is a groundwork before adding one-off (solo) jobs with persistance to the job manager
- Making the addJob method async also makes the whole interface consistent - removeJob and shutdown are also async
2022-07-20 20:41:02 +01:00
renovate[bot]
11d0ab9605 Update dependency c8 to v7.12.0 2022-07-19 18:33:27 +00:00
Daniel Lockyer
d8af7189e7 Published new versions
- @tryghost/adapter-manager@0.2.32
 - @tryghost/api-version-compatibility-service@0.4.2
 - @tryghost/bootstrap-socket@0.2.21
 - @tryghost/config-url-helpers@1.0.1
 - @tryghost/constants@1.0.6
 - @tryghost/database-info@0.3.6
 - @tryghost/email-content-generator@0.1.3
 - @tryghost/image-transform@1.0.33
 - @tryghost/job-manager@0.8.25
 - @tryghost/limit-service@1.2.1
 - @tryghost/minifier@0.1.16
 - @tryghost/moleculer-service-from-class@0.2.27
 - @tryghost/mw-api-version-mismatch@0.2.2
 - @tryghost/mw-error-handler@1.0.2
 - @tryghost/mw-session-from-token@0.1.33
 - @tryghost/mw-update-user-last-seen@0.1.7
 - @tryghost/package-json@1.0.22
 - @tryghost/pretty-cli@1.2.28
 - @tryghost/promise@0.1.19
 - @tryghost/release-utils@0.8.0
 - @tryghost/security@0.3.2
 - @tryghost/session-service@0.1.43
 - @tryghost/settings-path-manager@0.1.8
 - @tryghost/version-notifications-data-service@0.2.1
 - @tryghost/vhost-middleware@1.0.26
 - @tryghost/zip@1.1.26
2022-05-24 13:06:28 +02:00
Renovate Bot
74ff5d48de Update dependency c8 to v7.11.3 2022-05-16 04:56:38 +00:00
Naz
c5b8d073d7 Published new versions
- @tryghost/adapter-manager@0.2.31
 - @tryghost/api-version-compatibility-service@0.3.0
 - @tryghost/bootstrap-socket@0.2.20
 - @tryghost/config-url-helpers@0.1.8
 - @tryghost/constants@1.0.5
 - @tryghost/database-info@0.3.5
 - @tryghost/email-content-generator@0.1.2
 - @tryghost/image-transform@1.0.32
 - @tryghost/job-manager@0.8.24
 - @tryghost/limit-service@1.1.3
 - @tryghost/minifier@0.1.15
 - @tryghost/moleculer-service-from-class@0.2.26
 - @tryghost/mw-api-version-mismatch@0.1.4
 - @tryghost/mw-error-handler@1.0.1
 - @tryghost/mw-session-from-token@0.1.32
 - @tryghost/mw-update-user-last-seen@0.1.6
 - @tryghost/package-json@1.0.21
 - @tryghost/pretty-cli@1.2.27
 - @tryghost/promise@0.1.18
 - @tryghost/release-utils@0.7.15
 - @tryghost/security@0.3.1
 - @tryghost/session-service@0.1.42
 - @tryghost/settings-path-manager@0.1.7
 - @tryghost/update-check-service@0.3.4
 - @tryghost/version-notifications-data-service@0.1.2
 - @tryghost/zip@1.1.25
2022-05-09 18:25:48 +08:00
Renovate Bot
82b83743a7 Update dependency sinon to v14 2022-05-09 00:14:50 +00:00
Naz
e5d097fbae Published new versions
- @tryghost/adapter-manager@0.2.30
 - @tryghost/api-version-compatibility-service@0.1.2
 - @tryghost/bootstrap-socket@0.2.19
 - @tryghost/config-url-helpers@0.1.7
 - @tryghost/constants@1.0.4
 - @tryghost/database-info@0.3.4
 - @tryghost/email-content-generator@0.1.0
 - @tryghost/image-transform@1.0.31
 - @tryghost/job-manager@0.8.23
 - @tryghost/limit-service@1.1.1
 - @tryghost/minifier@0.1.14
 - @tryghost/moleculer-service-from-class@0.2.25
 - @tryghost/mw-api-version-mismatch@0.1.2
 - @tryghost/mw-error-handler@0.2.3
 - @tryghost/mw-session-from-token@0.1.31
 - @tryghost/mw-update-user-last-seen@0.1.5
 - @tryghost/package-json@1.0.20
 - @tryghost/pretty-cli@1.2.26
 - @tryghost/promise@0.1.17
 - @tryghost/release-utils@0.7.14
 - @tryghost/security@0.2.17
 - @tryghost/session-service@0.1.41
 - @tryghost/settings-path-manager@0.1.6
 - @tryghost/update-check-service@0.3.3
 - @tryghost/version-notifications-data-service@0.1.1
 - @tryghost/vhost-middleware@1.0.25
 - @tryghost/zip@1.1.24
2022-05-04 17:21:51 +08:00
Renovate Bot
6f93c82c5a Update Test & linting packages 2022-05-02 14:01:35 +00:00
Renovate Bot
9589755925 Update Test & linting packages 2022-05-02 13:58:34 +00:00
Naz
014fafbafb Published new versions
- @tryghost/adapter-manager@0.2.29
 - @tryghost/api-version-compatibility-service@0.1.0
 - @tryghost/bootstrap-socket@0.2.18
 - @tryghost/config-url-helpers@0.1.6
 - @tryghost/constants@1.0.3
 - @tryghost/database-info@0.3.2
 - @tryghost/image-transform@1.0.30
 - @tryghost/job-manager@0.8.22
 - @tryghost/limit-service@1.0.11
 - @tryghost/minifier@0.1.13
 - @tryghost/moleculer-service-from-class@0.2.24
 - @tryghost/mw-api-version-mismatch@0.1.0
 - @tryghost/mw-error-handler@0.2.1
 - @tryghost/mw-session-from-token@0.1.29
 - @tryghost/mw-update-user-last-seen@0.1.4
 - @tryghost/package-json@1.0.19
 - @tryghost/pretty-cli@1.2.25
 - @tryghost/promise@0.1.16
 - @tryghost/release-utils@0.7.13
 - @tryghost/security@0.2.16
 - @tryghost/session-service@0.1.39
 - @tryghost/settings-path-manager@0.1.5
 - @tryghost/vhost-middleware@1.0.23
 - @tryghost/zip@1.1.23
2022-04-21 20:58:17 +08:00
Renovate Bot
430608c9c3 Update dependency c8 to v7.11.2 2022-04-20 18:23:57 +00:00
Renovate Bot
65b91e4895 Update dependency @sinonjs/fake-timers to v9.1.2 2022-04-13 10:09:23 +00:00
Sam Lord
432ebed27c Published new versions
- @tryghost/job-manager@0.8.21
 - @tryghost/mw-error-handler@0.1.4
2022-03-11 10:48:00 +00:00
Renovate Bot
10e29837e5 Update dependency @sinonjs/fake-timers to v9.1.1 2022-03-07 01:24:01 +00:00
Daniel Lockyer
3324ffc788 Published new versions
- @tryghost/adapter-manager@0.2.28
 - @tryghost/bootstrap-socket@0.2.17
 - @tryghost/config-url-helpers@0.1.5
 - @tryghost/constants@1.0.2
 - @tryghost/database-info@0.2.0
 - @tryghost/image-transform@1.0.28
 - @tryghost/job-manager@0.8.20
 - @tryghost/limit-service@1.0.10
 - @tryghost/minifier@0.1.11
 - @tryghost/moleculer-service-from-class@0.2.23
 - @tryghost/mw-error-handler@0.1.3
 - @tryghost/mw-session-from-token@0.1.28
 - @tryghost/mw-update-user-last-seen@0.1.3
 - @tryghost/package-json@1.0.16
 - @tryghost/pretty-cli@1.2.24
 - @tryghost/promise@0.1.15
 - @tryghost/release-utils@0.7.12
 - @tryghost/security@0.2.15
 - @tryghost/session-service@0.1.38
 - @tryghost/settings-path-manager@0.1.4
 - @tryghost/vhost-middleware@1.0.22
 - @tryghost/zip@1.1.20
2022-03-01 08:40:52 +01:00
Daniel Lockyer
3259994e4f Added --all flag to c8 commands
refs https://github.com/TryGhost/Toolbox/issues/203

- without `--all`, c8 will ignore files that aren't covered in tests, so
  they won't pull the test coverage down
- this means we have artificially high coverage scores
- this commit adds `--all` where previously missing
2022-02-21 12:50:26 +01:00
Daniel Lockyer
cfb1ded0d0 Published new versions
- @tryghost/job-manager@0.8.19
 - @tryghost/release-utils@0.7.11
 - @tryghost/update-check-service@0.3.1
 - @tryghost/vhost-middleware@1.0.21
2022-02-04 14:27:23 +01:00