TryGhost/Ghost
1
0
Fork 0
mirror of https://github.com/TryGhost/Ghost.git synced 2024-11-22 19:32:54 +03:00
Code Issues Projects Releases Wiki Activity
2fa-during-setup
Ghost/ghost/prometheus-metrics/package.json

43 lines
1.3 KiB
JSON
Raw Permalink Normal View History

Added a prometheus metrics exporter to Ghost (#21193) ref https://linear.app/tryghost/issue/ENG-1505/add-prometheus-metrics-server-to-allow-monitoring-ghost-metrics # Summary This commit includes two main components: a prometheus client class to collect metrics from Ghost, and a standalone metrics server that exposes a /metrics endpoint at a separate port (9416 by default) from the main Ghost app. The prometheus client is a very thin wrapper around [prom-client](https://github.com/siimon/prom-client). We could use prom-client directly, but this approach should make it easier to switch to a different prometheus client package (or make our own) if we ever need to down the line. The list of default metrics this enables is specified in an e2e test [here](https://github.com/TryGhost/Ghost/pull/21192/files#diff-ebc52236be2cd14b40be89220ae961f48d3f837693f7d1da76db292348915941R66-R92). This also gives us the ability to create and collect custom metrics, although none are included in this commit yet. # Configuration The prometheus client and the metrics server are both disabled by default, but can be enabled by setting the metrics_server:enabled flag to true. You can also define a custom host and port using `metrics_server:host` and `metrics_server:port`. ## Why not expose the /metrics endpoint in one of the existing express apps? The standalone express app exists for two main reasons: 1. We don't want these metrics to be public, and the easiest way to accomplish that is to expose the /metrics endpoint at a different port that won't be exposed to the internet. 2. Creating a standalone express instance decouples the metrics endpoint from the Ghost server, so if Ghost is not responding for whatever reason, we should still be able to scrape metrics to understand what's going on internally. ## Impact on Boot & Shut down time The prometheus client is initialized early in the boot process so we can collect metrics during the boot sequence. Testing locally has shown that this increases boot time by ~20ms. The metrics server which exposes the /metrics endpoint is not initialized until after the background services, and it is not awaited, to avoid impacting boot time. None of this code, including the requires, will run if the metrics_server:enabled flag is set to false (or not set). Shutting down the metrics server is added as a cleanup task for the main Ghost server instance, and is setup to shut down with 0 grace period to avoid impacting shut down time.
2024-10-03 21:34:25 +03:00
{
Added ability to push prometheus metrics to a pushgateway (#21526) ref https://linear.app/ghost/issue/ENG-1746/enable-ghost-to-push-metrics-to-a-pushgateway - We'd like to use prometheus to expose metrics from Ghost, but the "standard" approach of having prometheus scrape the `/metrics` endpoint adds some complexity and additional challenges on Pro. - A suggested simpler alternative is to use a pushgateway, to have Ghost _push_ metrics to prometheus, rather than have prometheus scrape the running instances. - This PR introduces this functionality behind a configuration. - It also includes a refactor to the current metrics-server implementation so all the related code for prometheus is colocated, and the configuration is a bit more organized. `@tryghost/metrics-server` has been renamed to `@tryghost/prometheus-metrics`, and it now includes the metrics server and prometheus-client code itself (including the pushgateway code) - To enable the prometheus client alone, `prometheus:enabled` must be true. This will _not_ enable the metrics server or the pushgateway — it will essentially collect the metrics, but not do anything with them. - To enable the metrics server, set `prometheus:metrics_server:enabled` to true. You can also configure the host and port that the metrics server should export the `/metrics` endpoint on in the `prometheus:metrics_server` block. - To enable the pushgateway, set `prometheus:pushgateway:enabled` to true. You can also configure the pushgateway's `url`, the `interval` it should push metrics in (in milliseconds) and the `jobName` in the `prometheus:pushgateway` block.
2024-11-05 22:50:39 +03:00
"name": "@tryghost/prometheus-metrics",
Added a prometheus metrics exporter to Ghost (#21193) ref https://linear.app/tryghost/issue/ENG-1505/add-prometheus-metrics-server-to-allow-monitoring-ghost-metrics # Summary This commit includes two main components: a prometheus client class to collect metrics from Ghost, and a standalone metrics server that exposes a /metrics endpoint at a separate port (9416 by default) from the main Ghost app. The prometheus client is a very thin wrapper around [prom-client](https://github.com/siimon/prom-client). We could use prom-client directly, but this approach should make it easier to switch to a different prometheus client package (or make our own) if we ever need to down the line. The list of default metrics this enables is specified in an e2e test [here](https://github.com/TryGhost/Ghost/pull/21192/files#diff-ebc52236be2cd14b40be89220ae961f48d3f837693f7d1da76db292348915941R66-R92). This also gives us the ability to create and collect custom metrics, although none are included in this commit yet. # Configuration The prometheus client and the metrics server are both disabled by default, but can be enabled by setting the metrics_server:enabled flag to true. You can also define a custom host and port using `metrics_server:host` and `metrics_server:port`. ## Why not expose the /metrics endpoint in one of the existing express apps? The standalone express app exists for two main reasons: 1. We don't want these metrics to be public, and the easiest way to accomplish that is to expose the /metrics endpoint at a different port that won't be exposed to the internet. 2. Creating a standalone express instance decouples the metrics endpoint from the Ghost server, so if Ghost is not responding for whatever reason, we should still be able to scrape metrics to understand what's going on internally. ## Impact on Boot & Shut down time The prometheus client is initialized early in the boot process so we can collect metrics during the boot sequence. Testing locally has shown that this increases boot time by ~20ms. The metrics server which exposes the /metrics endpoint is not initialized until after the background services, and it is not awaited, to avoid impacting boot time. None of this code, including the requires, will run if the metrics_server:enabled flag is set to false (or not set). Shutting down the metrics server is added as a cleanup task for the main Ghost server instance, and is setup to shut down with 0 grace period to avoid impacting shut down time.
2024-10-03 21:34:25 +03:00
"version": "0.0.0",
Added ability to push prometheus metrics to a pushgateway (#21526) ref https://linear.app/ghost/issue/ENG-1746/enable-ghost-to-push-metrics-to-a-pushgateway - We'd like to use prometheus to expose metrics from Ghost, but the "standard" approach of having prometheus scrape the `/metrics` endpoint adds some complexity and additional challenges on Pro. - A suggested simpler alternative is to use a pushgateway, to have Ghost _push_ metrics to prometheus, rather than have prometheus scrape the running instances. - This PR introduces this functionality behind a configuration. - It also includes a refactor to the current metrics-server implementation so all the related code for prometheus is colocated, and the configuration is a bit more organized. `@tryghost/metrics-server` has been renamed to `@tryghost/prometheus-metrics`, and it now includes the metrics server and prometheus-client code itself (including the pushgateway code) - To enable the prometheus client alone, `prometheus:enabled` must be true. This will _not_ enable the metrics server or the pushgateway — it will essentially collect the metrics, but not do anything with them. - To enable the metrics server, set `prometheus:metrics_server:enabled` to true. You can also configure the host and port that the metrics server should export the `/metrics` endpoint on in the `prometheus:metrics_server` block. - To enable the pushgateway, set `prometheus:pushgateway:enabled` to true. You can also configure the pushgateway's `url`, the `interval` it should push metrics in (in milliseconds) and the `jobName` in the `prometheus:pushgateway` block.
2024-11-05 22:50:39 +03:00
"repository": "https://github.com/TryGhost/Ghost/tree/main/packages/prometheus-metrics",
Added a prometheus metrics exporter to Ghost (#21193) ref https://linear.app/tryghost/issue/ENG-1505/add-prometheus-metrics-server-to-allow-monitoring-ghost-metrics # Summary This commit includes two main components: a prometheus client class to collect metrics from Ghost, and a standalone metrics server that exposes a /metrics endpoint at a separate port (9416 by default) from the main Ghost app. The prometheus client is a very thin wrapper around [prom-client](https://github.com/siimon/prom-client). We could use prom-client directly, but this approach should make it easier to switch to a different prometheus client package (or make our own) if we ever need to down the line. The list of default metrics this enables is specified in an e2e test [here](https://github.com/TryGhost/Ghost/pull/21192/files#diff-ebc52236be2cd14b40be89220ae961f48d3f837693f7d1da76db292348915941R66-R92). This also gives us the ability to create and collect custom metrics, although none are included in this commit yet. # Configuration The prometheus client and the metrics server are both disabled by default, but can be enabled by setting the metrics_server:enabled flag to true. You can also define a custom host and port using `metrics_server:host` and `metrics_server:port`. ## Why not expose the /metrics endpoint in one of the existing express apps? The standalone express app exists for two main reasons: 1. We don't want these metrics to be public, and the easiest way to accomplish that is to expose the /metrics endpoint at a different port that won't be exposed to the internet. 2. Creating a standalone express instance decouples the metrics endpoint from the Ghost server, so if Ghost is not responding for whatever reason, we should still be able to scrape metrics to understand what's going on internally. ## Impact on Boot & Shut down time The prometheus client is initialized early in the boot process so we can collect metrics during the boot sequence. Testing locally has shown that this increases boot time by ~20ms. The metrics server which exposes the /metrics endpoint is not initialized until after the background services, and it is not awaited, to avoid impacting boot time. None of this code, including the requires, will run if the metrics_server:enabled flag is set to false (or not set). Shutting down the metrics server is added as a cleanup task for the main Ghost server instance, and is setup to shut down with 0 grace period to avoid impacting shut down time.
2024-10-03 21:34:25 +03:00
"author": "Ghost Foundation",
"private": true,
"main": "build/index.js",
"types": "build/index.d.ts",
"scripts": {
"dev": "tsc --watch --preserveWatchOutput --sourceMap",
"build": "yarn build:ts",
"build:ts": "tsc",
"prepare": "tsc",
Added ability to push prometheus metrics to a pushgateway (#21526) ref https://linear.app/ghost/issue/ENG-1746/enable-ghost-to-push-metrics-to-a-pushgateway - We'd like to use prometheus to expose metrics from Ghost, but the "standard" approach of having prometheus scrape the `/metrics` endpoint adds some complexity and additional challenges on Pro. - A suggested simpler alternative is to use a pushgateway, to have Ghost _push_ metrics to prometheus, rather than have prometheus scrape the running instances. - This PR introduces this functionality behind a configuration. - It also includes a refactor to the current metrics-server implementation so all the related code for prometheus is colocated, and the configuration is a bit more organized. `@tryghost/metrics-server` has been renamed to `@tryghost/prometheus-metrics`, and it now includes the metrics server and prometheus-client code itself (including the pushgateway code) - To enable the prometheus client alone, `prometheus:enabled` must be true. This will _not_ enable the metrics server or the pushgateway — it will essentially collect the metrics, but not do anything with them. - To enable the metrics server, set `prometheus:metrics_server:enabled` to true. You can also configure the host and port that the metrics server should export the `/metrics` endpoint on in the `prometheus:metrics_server` block. - To enable the pushgateway, set `prometheus:pushgateway:enabled` to true. You can also configure the pushgateway's `url`, the `interval` it should push metrics in (in milliseconds) and the `jobName` in the `prometheus:pushgateway` block.
2024-11-05 22:50:39 +03:00
"test:unit": "NODE_ENV=testing c8 --src src --all --check-coverage --90 --reporter text --reporter cobertura mocha -r ts-node/register './test/**/*.test.ts'",
Added a prometheus metrics exporter to Ghost (#21193) ref https://linear.app/tryghost/issue/ENG-1505/add-prometheus-metrics-server-to-allow-monitoring-ghost-metrics # Summary This commit includes two main components: a prometheus client class to collect metrics from Ghost, and a standalone metrics server that exposes a /metrics endpoint at a separate port (9416 by default) from the main Ghost app. The prometheus client is a very thin wrapper around [prom-client](https://github.com/siimon/prom-client). We could use prom-client directly, but this approach should make it easier to switch to a different prometheus client package (or make our own) if we ever need to down the line. The list of default metrics this enables is specified in an e2e test [here](https://github.com/TryGhost/Ghost/pull/21192/files#diff-ebc52236be2cd14b40be89220ae961f48d3f837693f7d1da76db292348915941R66-R92). This also gives us the ability to create and collect custom metrics, although none are included in this commit yet. # Configuration The prometheus client and the metrics server are both disabled by default, but can be enabled by setting the metrics_server:enabled flag to true. You can also define a custom host and port using `metrics_server:host` and `metrics_server:port`. ## Why not expose the /metrics endpoint in one of the existing express apps? The standalone express app exists for two main reasons: 1. We don't want these metrics to be public, and the easiest way to accomplish that is to expose the /metrics endpoint at a different port that won't be exposed to the internet. 2. Creating a standalone express instance decouples the metrics endpoint from the Ghost server, so if Ghost is not responding for whatever reason, we should still be able to scrape metrics to understand what's going on internally. ## Impact on Boot & Shut down time The prometheus client is initialized early in the boot process so we can collect metrics during the boot sequence. Testing locally has shown that this increases boot time by ~20ms. The metrics server which exposes the /metrics endpoint is not initialized until after the background services, and it is not awaited, to avoid impacting boot time. None of this code, including the requires, will run if the metrics_server:enabled flag is set to false (or not set). Shutting down the metrics server is added as a cleanup task for the main Ghost server instance, and is setup to shut down with 0 grace period to avoid impacting shut down time.
2024-10-03 21:34:25 +03:00
"test": "yarn test:types && yarn test:unit",
"test:types": "tsc --noEmit",
"lint:code": "eslint src/ --ext .ts --cache",
"lint": "yarn lint:code && yarn lint:test",
"lint:test": "eslint -c test/.eslintrc.js test/ --ext .ts --cache"
},
"files": [
"build"
],
"devDependencies": {
"@types/express": "4.17.21",
"@types/stoppable": "1.1.0",
"c8": "10.1.2",
Added connection pool metrics to prometheus client (#21576) ref https://linear.app/ghost/issue/ENG-1592/start-monitoring-connection-pool-utilization-in-ghost - This commit adds prometheus metrics to the connection pool so we can start to track connection pool utilization, number of pending acquires, and also adds some basic SQL query summary metrics like queries per minute and query duration percentiles. - The connection pool has now been theorized to be a main constraint of Ghost for some time, but it's been challenging to get actual visibility into the state of the connection pool. With this change, we should be able to directly observe, monitor and alert on the connection pool. - Updated grafana version to fix a bug in the query editor that was fixed in 8.3, even though this is a couple versions ahead of production
2024-11-08 10:01:34 +03:00
"knex": "2.4.2",
Added a prometheus metrics exporter to Ghost (#21193) ref https://linear.app/tryghost/issue/ENG-1505/add-prometheus-metrics-server-to-allow-monitoring-ghost-metrics # Summary This commit includes two main components: a prometheus client class to collect metrics from Ghost, and a standalone metrics server that exposes a /metrics endpoint at a separate port (9416 by default) from the main Ghost app. The prometheus client is a very thin wrapper around [prom-client](https://github.com/siimon/prom-client). We could use prom-client directly, but this approach should make it easier to switch to a different prometheus client package (or make our own) if we ever need to down the line. The list of default metrics this enables is specified in an e2e test [here](https://github.com/TryGhost/Ghost/pull/21192/files#diff-ebc52236be2cd14b40be89220ae961f48d3f837693f7d1da76db292348915941R66-R92). This also gives us the ability to create and collect custom metrics, although none are included in this commit yet. # Configuration The prometheus client and the metrics server are both disabled by default, but can be enabled by setting the metrics_server:enabled flag to true. You can also define a custom host and port using `metrics_server:host` and `metrics_server:port`. ## Why not expose the /metrics endpoint in one of the existing express apps? The standalone express app exists for two main reasons: 1. We don't want these metrics to be public, and the easiest way to accomplish that is to expose the /metrics endpoint at a different port that won't be exposed to the internet. 2. Creating a standalone express instance decouples the metrics endpoint from the Ghost server, so if Ghost is not responding for whatever reason, we should still be able to scrape metrics to understand what's going on internally. ## Impact on Boot & Shut down time The prometheus client is initialized early in the boot process so we can collect metrics during the boot sequence. Testing locally has shown that this increases boot time by ~20ms. The metrics server which exposes the /metrics endpoint is not initialized until after the background services, and it is not awaited, to avoid impacting boot time. None of this code, including the requires, will run if the metrics_server:enabled flag is set to false (or not set). Shutting down the metrics server is added as a cleanup task for the main Ghost server instance, and is setup to shut down with 0 grace period to avoid impacting shut down time.
2024-10-03 21:34:25 +03:00
"mocha": "10.7.3",
Added connection pool metrics to prometheus client (#21576) ref https://linear.app/ghost/issue/ENG-1592/start-monitoring-connection-pool-utilization-in-ghost - This commit adds prometheus metrics to the connection pool so we can start to track connection pool utilization, number of pending acquires, and also adds some basic SQL query summary metrics like queries per minute and query duration percentiles. - The connection pool has now been theorized to be a main constraint of Ghost for some time, but it's been challenging to get actual visibility into the state of the connection pool. With this change, we should be able to directly observe, monitor and alert on the connection pool. - Updated grafana version to fix a bug in the query editor that was fixed in 8.3, even though this is a couple versions ahead of production
2024-11-08 10:01:34 +03:00
"nock": "13.5.5",
Added a prometheus metrics exporter to Ghost (#21193) ref https://linear.app/tryghost/issue/ENG-1505/add-prometheus-metrics-server-to-allow-monitoring-ghost-metrics # Summary This commit includes two main components: a prometheus client class to collect metrics from Ghost, and a standalone metrics server that exposes a /metrics endpoint at a separate port (9416 by default) from the main Ghost app. The prometheus client is a very thin wrapper around [prom-client](https://github.com/siimon/prom-client). We could use prom-client directly, but this approach should make it easier to switch to a different prometheus client package (or make our own) if we ever need to down the line. The list of default metrics this enables is specified in an e2e test [here](https://github.com/TryGhost/Ghost/pull/21192/files#diff-ebc52236be2cd14b40be89220ae961f48d3f837693f7d1da76db292348915941R66-R92). This also gives us the ability to create and collect custom metrics, although none are included in this commit yet. # Configuration The prometheus client and the metrics server are both disabled by default, but can be enabled by setting the metrics_server:enabled flag to true. You can also define a custom host and port using `metrics_server:host` and `metrics_server:port`. ## Why not expose the /metrics endpoint in one of the existing express apps? The standalone express app exists for two main reasons: 1. We don't want these metrics to be public, and the easiest way to accomplish that is to expose the /metrics endpoint at a different port that won't be exposed to the internet. 2. Creating a standalone express instance decouples the metrics endpoint from the Ghost server, so if Ghost is not responding for whatever reason, we should still be able to scrape metrics to understand what's going on internally. ## Impact on Boot & Shut down time The prometheus client is initialized early in the boot process so we can collect metrics during the boot sequence. Testing locally has shown that this increases boot time by ~20ms. The metrics server which exposes the /metrics endpoint is not initialized until after the background services, and it is not awaited, to avoid impacting boot time. None of this code, including the requires, will run if the metrics_server:enabled flag is set to false (or not set). Shutting down the metrics server is added as a cleanup task for the main Ghost server instance, and is setup to shut down with 0 grace period to avoid impacting shut down time.
2024-10-03 21:34:25 +03:00
"sinon": "19.0.2",
"supertest": "7.0.0",
"ts-node": "10.9.2",
"typescript": "5.6.2"
},
"dependencies": {
Configured local prometheus and pushgateway in docker-compose (#21538) ref https://linear.app/ghost/issue/ENG-1746/enable-ghost-to-push-metrics-to-a-pushgateway - Added prometheus job to scrape the pushgateway - Updated grafana dashboard to use the metrics from the pushgateway - Added some logging to prometheus client to log errors when pushing metrics to pushgateway
2024-11-06 22:36:37 +03:00
"@tryghost/logging": "2.4.19",
Update dependency express to v4.21.1
2024-10-08 22:07:31 +03:00
"express": "4.21.1",
Pin dependency prom-client to 15.1.3
2024-11-06 07:29:04 +03:00
"prom-client": "15.1.3",
Added a prometheus metrics exporter to Ghost (#21193) ref https://linear.app/tryghost/issue/ENG-1505/add-prometheus-metrics-server-to-allow-monitoring-ghost-metrics # Summary This commit includes two main components: a prometheus client class to collect metrics from Ghost, and a standalone metrics server that exposes a /metrics endpoint at a separate port (9416 by default) from the main Ghost app. The prometheus client is a very thin wrapper around [prom-client](https://github.com/siimon/prom-client). We could use prom-client directly, but this approach should make it easier to switch to a different prometheus client package (or make our own) if we ever need to down the line. The list of default metrics this enables is specified in an e2e test [here](https://github.com/TryGhost/Ghost/pull/21192/files#diff-ebc52236be2cd14b40be89220ae961f48d3f837693f7d1da76db292348915941R66-R92). This also gives us the ability to create and collect custom metrics, although none are included in this commit yet. # Configuration The prometheus client and the metrics server are both disabled by default, but can be enabled by setting the metrics_server:enabled flag to true. You can also define a custom host and port using `metrics_server:host` and `metrics_server:port`. ## Why not expose the /metrics endpoint in one of the existing express apps? The standalone express app exists for two main reasons: 1. We don't want these metrics to be public, and the easiest way to accomplish that is to expose the /metrics endpoint at a different port that won't be exposed to the internet. 2. Creating a standalone express instance decouples the metrics endpoint from the Ghost server, so if Ghost is not responding for whatever reason, we should still be able to scrape metrics to understand what's going on internally. ## Impact on Boot & Shut down time The prometheus client is initialized early in the boot process so we can collect metrics during the boot sequence. Testing locally has shown that this increases boot time by ~20ms. The metrics server which exposes the /metrics endpoint is not initialized until after the background services, and it is not awaited, to avoid impacting boot time. None of this code, including the requires, will run if the metrics_server:enabled flag is set to false (or not set). Shutting down the metrics server is added as a cleanup task for the main Ghost server instance, and is setup to shut down with 0 grace period to avoid impacting shut down time.
2024-10-03 21:34:25 +03:00
"stoppable": "1.1.0"
}
}
Reference in New Issue Copy Permalink