INFRA-830: add prometheus metric corresponding to /healthz/sources

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9134
Co-authored-by: awjchen <13142944+awjchen@users.noreply.github.com>
GitOrigin-RevId: 03f2aa72757d405b314c252234e0cd81787c3948

docs/docs/enterprise/metrics.mdx

@@ -284,6 +284,19 @@ Current number of active PostgreSQL connections. Compare this to
 | Type   | Gauge                                                                                                                                                                             |
 | Labels | `source_name`: name of the database<br />`conn_info`: connection url string (password omitted) or name of the connection url environment variable<br />`role`: primary \| replica |
 
+### Hasura source health
+
+Health check status of a particular data source, corresponding to the output of
+`/healthz/sources`, with possible values 0 through 3 indicating, respectively:
+OK, TIMEOUT, FAILED, ERROR. See the [Source Health Check API Reference](/api-reference/source-health.mdx)
+for details.
+
+|        |                                                                                                                                                                                   |
+| ------ | -------------------------------------- |
+| Name   | `hasura_source_health`                 |
+| Type   | Gauge                                  |
+| Labels | `source_name`: name of the database    |
+
 ### Hasura subscription database execution time
 
 The time taken to run the subscription's multiplexed query in the database for a single batch.

server/src-lib/Hasura/RQL/DDL/Schema/Cache.hs

@@ -282,6 +282,10 @@ buildHealthCheckCache sources =
     mkSourceHealthCheck (BackendSourceMetadata sourceMetadata) =
       AB.traverseBackend @Backend sourceMetadata mkSourceHealthCheckBackend
 
+    -- 'Nothing' when no health check is defined. See:
+    -- https://hasura.io/docs/latest/deployment/health-checks/source-health-check/
+    -- We likely choose not to install a default `SELECT 1` health check here,
+    -- since we don't want to spam serverless databases.
     mkSourceHealthCheckBackend :: SourceMetadata b -> Maybe (SourceHealthCheckInfo b)
     mkSourceHealthCheckBackend sourceMetadata =
       let sourceName = _smName sourceMetadata

server/src-lib/Hasura/RQL/Types/Metadata/Common.hs

@@ -142,6 +142,7 @@ data SourceMetadata b = SourceMetadata
     _smConfiguration :: SourceConnConfiguration b,
     _smQueryTags :: Maybe QueryTagsConfig,
     _smCustomization :: SourceCustomization,
+    -- | https://hasura.io/docs/latest/deployment/health-checks/source-health-check/
     _smHealthCheckConfig :: Maybe (HealthCheckConfig b)
   }
   deriving (Generic)