Commit Graph

98 Commits

Author SHA1 Message Date
Steve Larson
a1c4e64994
Added queueing middleware to handle high request volume (#19887)
ref https://linear.app/tryghost/issue/CFR-4/
- added request queueing middleware (express-queue) to handle high
request volume
- added new config option `optimization.requestQueue`
- added new config option `optimization.requestConcurrency`
- added logging of request queue depth - `req.queueDepth`

We've done a fair amount of investigation around improving Ghost's
resiliency to high request volume. While we believe this to be partly
due to database connection contention, it also seems Ghost gets
overwhelmed by the requests themselves. Implementing a simple queueing
system allows us a simple lever to change the volume of requests Ghost
is actually ingesting at any given time and gives us options besides
simply increasing database connection pool size.

---------

Co-authored-by: Michael Barrett <mike@ghost.org>
2024-03-21 09:25:07 -05:00
Daniel Lockyer
85d41d0562 Aligned dependencies with resolution values
- this commit brings all dependencies up-to-date with the version set as
  a resolution
2023-10-13 08:37:36 +02:00
Daniel Lockyer
85098e07d4 Configured all unit tests to use dot reporter
refs https://ghost.slack.com/archives/C02G9E68C/p1696490748701419

- this configures mocha to use the dot reporter because the default is
  way too verbose in CI
2023-10-05 12:24:24 +02:00
Daniel Lockyer
c6cb35074a Updated linting and testing packages 2023-09-01 15:51:17 +02:00
renovate[bot]
7dce046786 Update Test & linting packages 2023-07-11 15:26:07 +02:00
Hannah Wolfe
b80b90229f Added consistent linting pattern to all packages
refs: https://github.com/TryGhost/Toolbox/issues/188

- some of our older packages used a pattern for linting which missed using test config for linting tests
- we need this to be consistent so that we can add more eslint rules for testing
- two packages also didn't use the lib pattern, which made the lint pattern error - so this was fixed as well
2023-06-13 10:43:29 +01:00
renovate[bot]
07545541a8 Update @tryghost 2023-04-07 13:47:12 +02:00
renovate[bot]
83373e1751 Update Test & linting packages 2023-04-05 15:16:08 +02:00
renovate[bot]
2223db5379
Update Test & linting packages 2023-03-13 02:36:20 +00:00
Daniel Lockyer
6b1966ad9b Updated sinon dependency
- this is being done manually instead of merging the Renovate PR because
  the PR bundles another bump which doesn't pass yet
2023-03-02 12:43:42 +01:00
renovate[bot]
9f08732039 Update @tryghost 2023-03-02 09:47:02 +01:00
renovate[bot]
cf7ecb492c
Update dependency c8 to v7.13.0 2023-02-16 22:15:50 +00:00
Daniel Lockyer
34fe49b110
Added version information to log lines
refs https://github.com/TryGhost/Toolbox/issues/501

- this reverts commit 48dda23554
- also includes a resolution for `@elastic/elasticsearch` so we don't
  run a version that is potentially problematic - see referenced issue
  for context
2023-01-20 13:18:44 +01:00
Daniel Lockyer
48dda23554
Reverted "Added version information to log lines"
refs https://github.com/TryGhost/Toolbox/issues/501

- this reverts commit f2116357b7
- something with Elasticsearch is causing high CPU usage, so this commit
  reverts that for now
2023-01-17 13:20:47 +01:00
Sam Lord
f2116357b7 Added version information to log lines
refs: https://github.com/TryGhost/Toolbox/issues/502

Updated to @tryghost/logging@2.4.0 to allow metadata to be logged
2023-01-12 10:34:23 +00:00
Daniel Lockyer
08b786af3b Bumped TryGhost-owned dependencies and lockfile
- this was all getting terribly behind so I've done several things:
  - majority of `@tryghost/*` except Lexical packages
  - gscan + knex-migrator to remove old `@tryghost/errors` usage
  - bumped lockfile
2023-01-02 20:55:22 +01:00
Daniel Lockyer
6f4e663d74
Updated @tryghost dependencies (#16005)
- also includes `knex-migrator` with a simple `sqlite3` bump
2022-12-14 11:18:55 +07:00
renovate[bot]
13abcf6c9d
Update dependency mocha to v10.2.0 2022-12-12 13:20:22 +00:00
renovate[bot]
8fa9f1e7e6
Update Test & linting packages 2022-11-07 20:39:48 +00:00
renovate[bot]
e9587e02d0
Update dependency mocha to v10.1.0 2022-10-17 08:41:28 +00:00
renovate[bot]
2c2ee81adb
Update Test & linting packages 2022-10-05 00:36:08 +00:00
renovate[bot]
9eb3c84a23
Updated @tryghost dependencies (#15434)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2022-09-27 08:31:35 +07:00
renovate[bot]
225765241c
Updated @tryghost dependencies (#15404)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2022-09-19 12:09:33 +01:00
Daniel Lockyer
7a2f766668
Added logging configuration option for timestamps to use the local timezone
fixes https://github.com/TryGhost/Ghost/issues/15190
refs https://github.com/TryGhost/framework/pull/76

- log output always uses UTC timestamps, but it may be desirable to
  configure logs to use the local machine timezone
- a new config option has been added to `@tryghost/logging` so you can
  switch the logs to the local timezone
- this commit bumps the package and sets the default config option to
  `false`, so it doesn't suddenly change the timezone of the logs
- docs will be updated soon but if you'd like to use the
  timezone-altered timestamps, you can set `logging.useLocalTime` to
  `true`
- credits to https://github.com/levee223 for the implementation and PR
2022-08-31 10:29:55 +01:00
Daniel Lockyer
54aa9f016b Fixed full Admin test suite running during unit tests
- because of how the npm scripts were set up, we were running the full
  Admin integration tests during the unit tests phase of CI
- this commit renames the majority of `test` to `test:unit` in the
  package.json files, and aliases `test` to `test:unit`
- special packages like Admin have no-op'd `test:unit` scripts so we
  don't end up running its tests
2022-08-15 15:34:52 +02:00
Simon Backx
82a3133ace
Added replies/likes count and limited replies to comments (#15206)
refs https://github.com/TryGhost/Team/issues/1723

- Added count.replies to comments
- Added replies endpoint
- Limited returned replies to 3.
- Replaced likes_count with count.likes in comments
- Instead of fetching all the likes of a comment to determine the total count, we'll now use count.likes
- Instead of fetching all the likes of a comment to determine whether a member liked a comment, we'll now use count.liked (which returns the amount of likes of the current member, being 0 or 1). This is mapped to `liked` to make it more natural to work with.

The `members.test.snap` file changed because we no longer include `liked: false` if we didn't fetch the liked relation. And in the comments events of the activity feed the liked property is therefore removed.

These changes requires an update to the `bookshelf-include-count` plugin:
- Updated to also work for nested relations
- This moves the count queries from the `bookshelf-include-count` plugin to the `countRelations` method of each model.
- Updated to keep the counts after saving a model (crud.edit didn't return the counts before)
2022-08-10 16:12:35 +02:00
renovate[bot]
679634342a
Pinned dependencies (#15100)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2022-07-26 16:06:00 +02:00
Daniel Lockyer
08abfcafd1
Remove linting as posttest step
- linting is handled separately (in CI and locally, as it's a git
  pre-push hook) so we shouldn't also be running it after tests
2022-07-26 15:26:21 +02:00
Daniel Lockyer
0a5f600dfe
Tidied up package README and LICENSE files
- we shouldn't need individual LICENSE files because these packages
  won't be published, so the top-level one applies
- also cleaned up README files to remove mentions of Lerna monorepos and
  install instructions
2022-07-26 15:22:10 +02:00
Daniel Lockyer
255eb0726b
Cleaned up package metadata
refs https://github.com/TryGhost/Toolbox/issues/354

- set packages to `private: true`
- removed repository link - these packages won't be published so this
  link won't be seen anywhere
- removed `publishConfig`
2022-07-26 15:08:05 +02:00
Daniel Lockyer
5fc7ba59d3
Reset package versions back to 0.0.0
refs https://github.com/TryGhost/Toolbox/issues/354

- these packages won't be published from now on, so setting the versions
  back to 0.0.0 keeps them clean
2022-07-26 14:57:43 +02:00
Naz
d5e38e6fc7 Published new versions
- @tryghost/adapter-manager@0.2.33
 - @tryghost/api-version-compatibility-service@0.4.4
 - @tryghost/bootstrap-socket@0.2.22
 - @tryghost/config-url-helpers@1.0.2
 - @tryghost/constants@1.0.7
 - @tryghost/database-info@0.3.8
 - @tryghost/email-content-generator@0.1.4
 - @tryghost/image-transform@1.2.1
 - @tryghost/job-manager@0.9.0
 - @tryghost/limit-service@1.2.2
 - @tryghost/minifier@0.1.17
 - @tryghost/moleculer-service-from-class@0.2.28
 - @tryghost/mw-api-version-mismatch@0.2.3
 - @tryghost/mw-error-handler@1.0.5
 - @tryghost/mw-session-from-token@0.1.34
 - @tryghost/mw-update-user-last-seen@0.1.8
 - @tryghost/package-json@1.0.23
 - @tryghost/pretty-cli@1.2.29
 - @tryghost/promise@0.1.20
 - @tryghost/release-utils@0.8.1
 - @tryghost/security@0.3.3
 - @tryghost/session-service@0.1.44
 - @tryghost/settings-path-manager@0.1.9
 - @tryghost/version-notifications-data-service@0.2.2
 - @tryghost/vhost-middleware@1.0.28
 - @tryghost/zip@1.1.27
2022-07-22 16:46:56 +01:00
renovate[bot]
11d0ab9605 Update dependency c8 to v7.12.0 2022-07-19 18:33:27 +00:00
Daniel Lockyer
d8af7189e7 Published new versions
- @tryghost/adapter-manager@0.2.32
 - @tryghost/api-version-compatibility-service@0.4.2
 - @tryghost/bootstrap-socket@0.2.21
 - @tryghost/config-url-helpers@1.0.1
 - @tryghost/constants@1.0.6
 - @tryghost/database-info@0.3.6
 - @tryghost/email-content-generator@0.1.3
 - @tryghost/image-transform@1.0.33
 - @tryghost/job-manager@0.8.25
 - @tryghost/limit-service@1.2.1
 - @tryghost/minifier@0.1.16
 - @tryghost/moleculer-service-from-class@0.2.27
 - @tryghost/mw-api-version-mismatch@0.2.2
 - @tryghost/mw-error-handler@1.0.2
 - @tryghost/mw-session-from-token@0.1.33
 - @tryghost/mw-update-user-last-seen@0.1.7
 - @tryghost/package-json@1.0.22
 - @tryghost/pretty-cli@1.2.28
 - @tryghost/promise@0.1.19
 - @tryghost/release-utils@0.8.0
 - @tryghost/security@0.3.2
 - @tryghost/session-service@0.1.43
 - @tryghost/settings-path-manager@0.1.8
 - @tryghost/version-notifications-data-service@0.2.1
 - @tryghost/vhost-middleware@1.0.26
 - @tryghost/zip@1.1.26
2022-05-24 13:06:28 +02:00
Renovate Bot
74ff5d48de Update dependency c8 to v7.11.3 2022-05-16 04:56:38 +00:00
Naz
c5b8d073d7 Published new versions
- @tryghost/adapter-manager@0.2.31
 - @tryghost/api-version-compatibility-service@0.3.0
 - @tryghost/bootstrap-socket@0.2.20
 - @tryghost/config-url-helpers@0.1.8
 - @tryghost/constants@1.0.5
 - @tryghost/database-info@0.3.5
 - @tryghost/email-content-generator@0.1.2
 - @tryghost/image-transform@1.0.32
 - @tryghost/job-manager@0.8.24
 - @tryghost/limit-service@1.1.3
 - @tryghost/minifier@0.1.15
 - @tryghost/moleculer-service-from-class@0.2.26
 - @tryghost/mw-api-version-mismatch@0.1.4
 - @tryghost/mw-error-handler@1.0.1
 - @tryghost/mw-session-from-token@0.1.32
 - @tryghost/mw-update-user-last-seen@0.1.6
 - @tryghost/package-json@1.0.21
 - @tryghost/pretty-cli@1.2.27
 - @tryghost/promise@0.1.18
 - @tryghost/release-utils@0.7.15
 - @tryghost/security@0.3.1
 - @tryghost/session-service@0.1.42
 - @tryghost/settings-path-manager@0.1.7
 - @tryghost/update-check-service@0.3.4
 - @tryghost/version-notifications-data-service@0.1.2
 - @tryghost/zip@1.1.25
2022-05-09 18:25:48 +08:00
Renovate Bot
82b83743a7 Update dependency sinon to v14 2022-05-09 00:14:50 +00:00
Naz
e5d097fbae Published new versions
- @tryghost/adapter-manager@0.2.30
 - @tryghost/api-version-compatibility-service@0.1.2
 - @tryghost/bootstrap-socket@0.2.19
 - @tryghost/config-url-helpers@0.1.7
 - @tryghost/constants@1.0.4
 - @tryghost/database-info@0.3.4
 - @tryghost/email-content-generator@0.1.0
 - @tryghost/image-transform@1.0.31
 - @tryghost/job-manager@0.8.23
 - @tryghost/limit-service@1.1.1
 - @tryghost/minifier@0.1.14
 - @tryghost/moleculer-service-from-class@0.2.25
 - @tryghost/mw-api-version-mismatch@0.1.2
 - @tryghost/mw-error-handler@0.2.3
 - @tryghost/mw-session-from-token@0.1.31
 - @tryghost/mw-update-user-last-seen@0.1.5
 - @tryghost/package-json@1.0.20
 - @tryghost/pretty-cli@1.2.26
 - @tryghost/promise@0.1.17
 - @tryghost/release-utils@0.7.14
 - @tryghost/security@0.2.17
 - @tryghost/session-service@0.1.41
 - @tryghost/settings-path-manager@0.1.6
 - @tryghost/update-check-service@0.3.3
 - @tryghost/version-notifications-data-service@0.1.1
 - @tryghost/vhost-middleware@1.0.25
 - @tryghost/zip@1.1.24
2022-05-04 17:21:51 +08:00
Renovate Bot
6f93c82c5a Update Test & linting packages 2022-05-02 14:01:35 +00:00
Renovate Bot
9589755925 Update Test & linting packages 2022-05-02 13:58:34 +00:00
Naz
014fafbafb Published new versions
- @tryghost/adapter-manager@0.2.29
 - @tryghost/api-version-compatibility-service@0.1.0
 - @tryghost/bootstrap-socket@0.2.18
 - @tryghost/config-url-helpers@0.1.6
 - @tryghost/constants@1.0.3
 - @tryghost/database-info@0.3.2
 - @tryghost/image-transform@1.0.30
 - @tryghost/job-manager@0.8.22
 - @tryghost/limit-service@1.0.11
 - @tryghost/minifier@0.1.13
 - @tryghost/moleculer-service-from-class@0.2.24
 - @tryghost/mw-api-version-mismatch@0.1.0
 - @tryghost/mw-error-handler@0.2.1
 - @tryghost/mw-session-from-token@0.1.29
 - @tryghost/mw-update-user-last-seen@0.1.4
 - @tryghost/package-json@1.0.19
 - @tryghost/pretty-cli@1.2.25
 - @tryghost/promise@0.1.16
 - @tryghost/release-utils@0.7.13
 - @tryghost/security@0.2.16
 - @tryghost/session-service@0.1.39
 - @tryghost/settings-path-manager@0.1.5
 - @tryghost/vhost-middleware@1.0.23
 - @tryghost/zip@1.1.23
2022-04-21 20:58:17 +08:00
Renovate Bot
430608c9c3 Update dependency c8 to v7.11.2 2022-04-20 18:23:57 +00:00
Daniel Lockyer
3324ffc788 Published new versions
- @tryghost/adapter-manager@0.2.28
 - @tryghost/bootstrap-socket@0.2.17
 - @tryghost/config-url-helpers@0.1.5
 - @tryghost/constants@1.0.2
 - @tryghost/database-info@0.2.0
 - @tryghost/image-transform@1.0.28
 - @tryghost/job-manager@0.8.20
 - @tryghost/limit-service@1.0.10
 - @tryghost/minifier@0.1.11
 - @tryghost/moleculer-service-from-class@0.2.23
 - @tryghost/mw-error-handler@0.1.3
 - @tryghost/mw-session-from-token@0.1.28
 - @tryghost/mw-update-user-last-seen@0.1.3
 - @tryghost/package-json@1.0.16
 - @tryghost/pretty-cli@1.2.24
 - @tryghost/promise@0.1.15
 - @tryghost/release-utils@0.7.12
 - @tryghost/security@0.2.15
 - @tryghost/session-service@0.1.38
 - @tryghost/settings-path-manager@0.1.4
 - @tryghost/vhost-middleware@1.0.22
 - @tryghost/zip@1.1.20
2022-03-01 08:40:52 +01:00
Daniel Lockyer
3259994e4f Added --all flag to c8 commands
refs https://github.com/TryGhost/Toolbox/issues/203

- without `--all`, c8 will ignore files that aren't covered in tests, so
  they won't pull the test coverage down
- this means we have artificially high coverage scores
- this commit adds `--all` where previously missing
2022-02-21 12:50:26 +01:00
Daniel Lockyer
19d2411f2b Published new versions
- @tryghost/adapter-manager@0.2.27
 - @tryghost/bootstrap-socket@0.2.16
 - @tryghost/config-url-helpers@0.1.4
 - @tryghost/constants@1.0.1
 - @tryghost/database-info@0.1.0
 - @tryghost/errors@1.2.1
 - @tryghost/image-transform@1.0.27
 - @tryghost/job-manager@0.8.18
 - @tryghost/limit-service@1.0.9
 - @tryghost/minifier@0.1.10
 - @tryghost/moleculer-service-from-class@0.2.22
 - @tryghost/mw-error-handler@0.1.2
 - @tryghost/mw-session-from-token@0.1.27
 - @tryghost/mw-update-user-last-seen@0.1.2
 - @tryghost/package-json@1.0.15
 - @tryghost/pretty-cli@1.2.23
 - @tryghost/promise@0.1.14
 - @tryghost/release-utils@0.7.10
 - @tryghost/security@0.2.14
 - @tryghost/session-service@0.1.37
 - @tryghost/settings-path-manager@0.1.3
 - @tryghost/vhost-middleware@1.0.20
 - @tryghost/zip@1.1.19
2022-01-18 09:08:09 +00:00
John O'Nolan
279ce66e71 2022 2022-01-06 09:52:35 +00:00
Renovate Bot
4c288fa50e Update dependency c8 to v7.11.0 2021-12-30 16:28:02 +00:00
Sam Lord
90e6e8b37d Published new versions
- @tryghost/bootstrap-socket@0.2.15
 - @tryghost/job-manager@0.8.17
2021-12-06 11:28:22 +00:00
Sam Lord
565790083e job-manager should support logging to parent process
refs: https://github.com/TryGhost/Toolbox/issues/146
2021-12-06 11:27:44 +00:00
Sam Lord
c4009fd4cf Published new versions
- @tryghost/bootstrap-socket@0.2.14
 - @tryghost/job-manager@0.8.16
2021-12-02 13:45:25 +00:00