mirror of
https://github.com/hasura/graphql-engine.git
synced 2024-09-20 23:17:30 +03:00
server/pro: enable health check on data sources and report via logging and API
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4868 Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com> Co-authored-by: Sean Park-Ross <94021366+seanparkross@users.noreply.github.com> GitOrigin-RevId: b8d43e3f7d977c4bb37b8506ac87ce7bf289d542
This commit is contained in:
awjchen
hasura-bot
parent
6cf337c0fa
commit
78cf1d544e
102
docs/docs/api-reference/source-health.mdx
Normal file
102
docs/docs/api-reference/source-health.mdx
Normal file
@ -0,0 +1,102 @@
|
||||
---
|
||||
description: Source health check API reference
|
||||
keywords:
|
||||
- hasura
|
||||
- cloud
|
||||
- docs
|
||||
- health API
|
||||
- source health API
|
||||
- API reference
|
||||
sidebar_position: 14
|
||||
sidebar_label: Source Health Check API
|
||||
sidebar_class_name: cloud-icon
|
||||
---
|
||||
|
||||
# Source Health Check API Reference
|
||||
|
||||
<div className='badge badge--primary heading-badge'>Available on: Cloud, Enterprise Edition</div>
|
||||
|
||||
## Introduction
|
||||
|
||||
The Source Health API is an admin-only endpoint which reports the health of sources whose health check is configured.
|
||||
[Documentation here](/deployment/health-checks/source-health-check.mdx).
|
||||
|
||||
## Endpoint
|
||||
|
||||
All requests are `GET` requests to the `/healthz/sources` endpoint.
|
||||
|
||||
## API Spec
|
||||
|
||||
### Request
|
||||
|
||||
```http
|
||||
GET /healthz/sources HTTP/1.1
|
||||
X-Hasura-Role: admin
|
||||
```
|
||||
|
||||
### Response
|
||||
|
||||
The response is an object with the source name as key and health status as value.
|
||||
|
||||
```none
|
||||
{
|
||||
"source_1": HealthStatus,
|
||||
"source_2": HealthStatus,
|
||||
... ...
|
||||
"source_n": HealthStatus
|
||||
|
||||
}
|
||||
```
|
||||
|
||||
The `HealthStatus` is an object with the following members.
|
||||
|
||||
| Name | Type | Description |
|
||||
|-----------|------------|--------------------------------------------------------------------------------------|
|
||||
| status | `string` | The status of the health check |
|
||||
| error | any `json` | An additional field whose value varies based on the `status` |
|
||||
| timestamp | `string` | A [UTC time](https://en.wikipedia.org/wiki/Coordinated_Universal_Time) encoded value |
|
||||
|
||||
Find the possible values of `status` field in the following along with corresponding `error` field value.
|
||||
|
||||
| status | error | Description |
|
||||
|-----------|--------------------|------------------------------------------------------------------------------------|
|
||||
| "OK" | `null` | Health check succeeded with configured test; the source is healthy |
|
||||
| "TIMEOUT" | `null` | Health check timed out |
|
||||
| "ERROR" | `HealthCheckError` | Exceptions occurred after running health check; refer `error` for in-depth details |
|
||||
| "FAILED" | `String` | Health check failed due to bad configuration |
|
||||
|
||||
The `HealthCheckError` is an object with the following members.
|
||||
|
||||
| Name | Type | Description |
|
||||
|---------|------------|--------------------------------------------------------------|
|
||||
| message | `string` | A very short description of the error |
|
||||
| extra | any `json` | An optional value that contains more details about the error |
|
||||
|
||||
### Sample response
|
||||
|
||||
```http
|
||||
HTTP/1.1 200 OK
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"mssql_source_name": {
|
||||
"error": null,
|
||||
"status": "OK",
|
||||
"timestamp": "2022-08-09T09:32:05.235347837Z"
|
||||
},
|
||||
"postgres_source_name": {
|
||||
"error": {
|
||||
"message": "connection error",
|
||||
"extra": "connection to server at \"localhost\" (::1), port 6432 failed: Connection refused\n\tIs the server running on that host and accepting TCP/IP connections?\nconnection to server at \"localhost\" (127.0.0.1), port 6432 failed: Connection refused\n\tIs the server running on that host and accepting TCP/IP connections?\n"
|
||||
},
|
||||
"status": "ERROR",
|
||||
"timestamp": "2022-08-09T09:30:05.235347837Z"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
:::info Note
|
||||
|
||||
The `healthz/sources` API endpoint cannot be disabled.
|
||||
|
||||
:::
|
||||
4
docs/docs/deployment/health-checks/_category_.json
Normal file
4
docs/docs/deployment/health-checks/_category_.json
Normal file
@ -0,0 +1,4 @@
|
||||
{
|
||||
"label": "Health Checks",
|
||||
"position": 3
|
||||
}
|
||||
31
docs/docs/deployment/health-checks/healthz-check.mdx
Normal file
31
docs/docs/deployment/health-checks/healthz-check.mdx
Normal file
@ -0,0 +1,31 @@
|
||||
---
|
||||
description: Hasura installation health check API route
|
||||
title: 'Cloud: Source health check'
|
||||
keywords:
|
||||
- hasura
|
||||
- source health check
|
||||
- source health
|
||||
sidebar_label: Healthz Check
|
||||
sidebar_position: 10
|
||||
---
|
||||
|
||||
# Healthz Check
|
||||
|
||||
Hasura provides a health check endpoint to monitor the status of the GraphQL API. This is available under the
|
||||
`/healthz` endpoint for all Hasura projects (including the OSS GraphQL Engine).
|
||||
|
||||
Make a `GET` request to the `/healthz` endpoint to fetch the status:
|
||||
|
||||
```bash
|
||||
curl -XGET https://advanced-hasura.hasura.app/healthz
|
||||
```
|
||||
|
||||
Replace `advanced-hasura` with your project name.
|
||||
|
||||
The status could be one of the following:
|
||||
|
||||
- `200, OK` - This requires no action. Everything is working as expected.
|
||||
- `200, WARN, inconsistent objects in schema` - This requires a review of metadata since some inconsistent objects have
|
||||
been identified. Usually occurs when there is a metadata apply that had the wrong objects.
|
||||
- `500, ERROR` - This means the API is not working and the [logs](/deployment/logging.mdx#health-check-log-structure)
|
||||
need to be checked.
|
||||
22
docs/docs/deployment/health-checks/index.mdx
Normal file
22
docs/docs/deployment/health-checks/index.mdx
Normal file
@ -0,0 +1,22 @@
|
||||
---
|
||||
description: Check the health of your Hasura installation
|
||||
keywords:
|
||||
- hasura
|
||||
- health
|
||||
- checks
|
||||
- health checks
|
||||
- sources
|
||||
- databases
|
||||
slug: index
|
||||
sidebar_position: 1
|
||||
---
|
||||
|
||||
# Health Checks
|
||||
|
||||
## Introduction
|
||||
|
||||
You can use the `healthz` (Hasura Community Edition) and `healthz/sources` (Hasura Cloud and Enterprise only) API
|
||||
routes to check on the health of your Hasura installation and the database sources connected to it.
|
||||
|
||||
- [Hasura Health Check](/deployment/health-checks/healthz-check.mdx)
|
||||
- [Source Health Check](/deployment/health-checks/source-health-check.mdx)
|
||||
123
docs/docs/deployment/health-checks/source-health-check.mdx
Normal file
123
docs/docs/deployment/health-checks/source-health-check.mdx
Normal file
@ -0,0 +1,123 @@
|
||||
---
|
||||
description: Hasura database source health check
|
||||
title: 'Cloud: Source health check'
|
||||
keywords:
|
||||
- hasura
|
||||
- cloud
|
||||
- enterprise
|
||||
- source health check
|
||||
- source health
|
||||
sidebar_label: Source Health Check
|
||||
sidebar_position: 20
|
||||
sidebar_class_name: cloud-icon
|
||||
---
|
||||
|
||||
import Tabs from '@theme/Tabs';
|
||||
import TabItem from '@theme/TabItem';
|
||||
import HeadingIcon from '@site/src/components/HeadingIcon';
|
||||
|
||||
# Source Health Check
|
||||
|
||||
<div className='badge badge--primary heading-badge'>Available on: Cloud, Enterprise Edition</div>
|
||||
|
||||
## Overview
|
||||
|
||||
Hasura enables users to check the health of connected data sources via the health check API.
|
||||
[API reference here](/api-reference/source-health.mdx).
|
||||
|
||||
## Configuring source health check
|
||||
|
||||
A health check configuration contains information that Hasura uses to determine the health state of a source. You
|
||||
can set the time interval for when Hasura will re-perform the check on the source.
|
||||
|
||||
Currently, Hasura supports enabling health checks on Postgres and MS SQL Server databases. Support for other data
|
||||
sources will be added soon.
|
||||
|
||||
Health check configurations for Postgres and SQL Server sources are identical and are as follows.
|
||||
|
||||
<Tabs className="api-tabs">
|
||||
<TabItem value="console" label="Console">
|
||||
|
||||
Console support will be added soon.
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="cli" label="CLI">
|
||||
|
||||
You can add _health check_ for a source by adding the config to the `/metadata/databases/database.yaml` file:
|
||||
|
||||
```yaml {3-8}
|
||||
- name: <db-name>
|
||||
kind: postgres
|
||||
health_check:
|
||||
test:
|
||||
sql: SELECT 1
|
||||
interval: 10
|
||||
retries: 3
|
||||
retry_interval: 5
|
||||
timeout: 5
|
||||
configuration:
|
||||
connection_info:
|
||||
database_url:
|
||||
from_env: <DATABASE_URL_ENV>
|
||||
pool_settings:
|
||||
idle_timeout: 180
|
||||
max_connections: 50
|
||||
retries: 1
|
||||
```
|
||||
|
||||
Apply the metadata by running:
|
||||
|
||||
```yaml
|
||||
hasura metadata apply
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="api" label="API">
|
||||
|
||||
You can add _health check_ for a database using the
|
||||
[pg_add_source](/api-reference/metadata-api/source.mdx#metadata-pg-add-source) metadata API.
|
||||
|
||||
```http {17-24}
|
||||
POST /v1/metadata HTTP/1.1
|
||||
Content-Type: application/json
|
||||
X-Hasura-Role: admin
|
||||
|
||||
{
|
||||
"type":"pg_add_source",
|
||||
"args":{
|
||||
"name":"<db_name>",
|
||||
"replace_configuration":true,
|
||||
"configuration":{
|
||||
"connection_info":{
|
||||
"database_url":{
|
||||
"from_env":"<DATABASE_URL_ENV>"
|
||||
}
|
||||
}
|
||||
},
|
||||
"health_check": {
|
||||
"test": {
|
||||
"sql": "SELECT 1"
|
||||
},
|
||||
"interval": 300,
|
||||
"timeout": 5,
|
||||
"retries": 3,
|
||||
"retry_interval": 5
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
## Reporting source health check
|
||||
|
||||
### API
|
||||
|
||||
Health check reports of sources can be obtained through a `GET` request from the
|
||||
`/healthz/sources` API, on demand. Learn more about the API [here](/api-reference/source-health.mdx).
|
||||
|
||||
### Logging
|
||||
|
||||
Hasura logs the health check status and other information via `health-check-log` type when enabled.
|
||||
Learn more about the health checks logs [here](/deployment/logging.mdx#health-check-log-structure).
|
||||
@ -59,7 +59,7 @@ Configurable log-types
|
||||
Apart from the above, there are other internal log-types which cannot be configured:
|
||||
|
||||
| Log type | Description | Log Level |
|
||||
| -------------------- | --------------------------------------------------------------------------------------------------- | ------------------ |
|
||||
|----------------------|-----------------------------------------------------------------------------------------------------|--------------------|
|
||||
| `pg-client` | Logs from the postgres client library | `warn` |
|
||||
| `metadata` | Logs inconsistent metadata items | `warn` |
|
||||
| `jwk-refresh-log` | Logs information and errors about periodic refreshing of JWK | `info` and `error` |
|
||||
@ -67,6 +67,7 @@ Apart from the above, there are other internal log-types which cannot be configu
|
||||
| `event-trigger` | Logs HTTP responses from the webhook, HTTP exceptions and internal errors | `info` and `error` |
|
||||
| `ws-server` | Debug logs from the websocket server, mostly used internally for debugging | `debug` |
|
||||
| `schema-sync-thread` | Logs internal events, when it detects schema has changed on Postgres and when it reloads the schema | `info` and `error` |
|
||||
| `health-check-log` | Logs source health check events which includes health status of a data source | `info` and `warn` |
|
||||
|
||||
Internal log-types
|
||||
|
||||
@ -395,6 +396,122 @@ This is how the Websocket logs look like:
|
||||
}
|
||||
```
|
||||
|
||||
### **health-check-log** structure
|
||||
|
||||
The GraphQL Engine does recurring health checks on data sources and logs the status with other
|
||||
details. This is how the health check log looks like:
|
||||
|
||||
- On successful health check
|
||||
|
||||
```json
|
||||
{
|
||||
"level": "info",
|
||||
"timestamp": "2022-07-28T12:23:56.555+0530",
|
||||
"type": "health-check-log",
|
||||
"detail": {
|
||||
"source_name": "mssql",
|
||||
"status": "OK",
|
||||
"timestamp": "2022-07-28T06:53:56.555Z",
|
||||
"error": null,
|
||||
"internal": {
|
||||
"interval": 5,
|
||||
"max_retries": 3,
|
||||
"retry_iteration": 0,
|
||||
"timeout": 3
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- When health check is timed out
|
||||
|
||||
```json
|
||||
{
|
||||
"level": "warn",
|
||||
"timestamp": "2022-07-28T12:28:16.165+0530",
|
||||
"type": "health-check-log",
|
||||
"detail": {
|
||||
"source_name": "mssql",
|
||||
"status": "TIMEOUT",
|
||||
"timestamp": "2022-07-28T06:58:16.165Z",
|
||||
"error": null,
|
||||
"internal": {
|
||||
"interval": 5,
|
||||
"max_retries": 3,
|
||||
"retry_iteration": 3,
|
||||
"timeout": 3
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- When health check results in an error
|
||||
|
||||
```json
|
||||
{
|
||||
"level": "warn",
|
||||
"timestamp": "2022-07-28T12:30:06.643+0530",
|
||||
"type": "health-check-log",
|
||||
"detail": {
|
||||
"source_name": "postgres",
|
||||
"status": "ERROR",
|
||||
"timestamp": "2022-07-28T07:00:06.643Z",
|
||||
"error": {
|
||||
"message": "connection error",
|
||||
"extra": "connection to server at \"localhost\" (::1), port 6432 failed: Connection refused\n\tIs the server running on that host and accepting TCP/IP connections?\nconnection to server at \"localhost\" (127.0.0.1), port 6432 failed: Connection refused\n\tIs the server running on that host and accepting TCP/IP connections?\n"
|
||||
},
|
||||
"internal": {
|
||||
"interval": 10,
|
||||
"max_retries": 3,
|
||||
"retry_iteration": 3,
|
||||
"timeout": 5
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The `type` in the log will be `health-check-log` and details of the health check will be under `detail` key.
|
||||
|
||||
The `detail` field value is an object contains the following members.
|
||||
|
||||
| Name | Type | Description |
|
||||
|---------------|----------------------------|--------------------------------------------------------|
|
||||
| `source_name` | string | The name of the source |
|
||||
| `status` | `HealthCheckStatus` string | The health status of the source |
|
||||
| `timestamp` | string | The timestamp in UTC when the health check is finished |
|
||||
| `error` | `HealthCheckError` value | The details of the error |
|
||||
| `internal` | `Internal` object | Internals of the health check config |
|
||||
|
||||
- **HealthCheckStatus** is a mandatory field whose values are as follows.
|
||||
|
||||
| Health check status | Description | Log level |
|
||||
|---------------------|--------------------------------------------------------------------------------------|-----------|
|
||||
| `OK` | Health check succeeded with no errors. | `info` |
|
||||
| `FAILED` | Health check is failed maybe due to bad connection config. | `warn` |
|
||||
| `TIMEOUT` | Health check is timed out. The timeout value is specified in the healch check config | `warn` |
|
||||
| `ERROR` | Health check results in an exception. | `warn` |
|
||||
|
||||
- **HealthCheckError** contains more information about the health check exception when the status is `ERROR`.
|
||||
For other statuses the value will be `null`. The `error` object contains the following fields
|
||||
- `message`: _string_. A very brief description about the error.
|
||||
- `extra`: _any json_. Contains extra and detailed information about the error.
|
||||
|
||||
- **Internal** is an object contains the following fields.
|
||||
- `interval`: _int_. Health check interval in seconds.
|
||||
- `max_retries`: _int_. Maximum # of retries configured.
|
||||
- `retry_interation`: _int_. The iteration on which the health check is succeeded. In case of unsuccessful health check, the retry iteration is same as `max_retries`.
|
||||
- `retry_interval`: _int_. The retry interval in seconds.
|
||||
- `timeout`: _int_. Health check time out value in seconds.
|
||||
|
||||
:::info Note
|
||||
|
||||
The GraphQL Engine logs the health check status only when
|
||||
|
||||
- the `status` is not `OK`
|
||||
- the previous check `status` was not `OK` and current `status` is `OK`
|
||||
|
||||
:::
|
||||
|
||||
## Monitoring frameworks
|
||||
|
||||
You can integrate the logs emitted by Hasura GraphQL with external monitoring tools for better visibility as per your
|
||||
|
||||
@ -680,6 +680,7 @@ library
|
||||
, Hasura.RQL.Types.EventTrigger
|
||||
, Hasura.RQL.Types.Eventing
|
||||
, Hasura.RQL.Types.Eventing.Backend
|
||||
, Hasura.RQL.Types.HealthCheck
|
||||
, Hasura.RQL.Types.Function
|
||||
, Hasura.RQL.Types.GraphqlSchemaIntrospection
|
||||
, Hasura.RQL.Types.Instances
|
||||
|
||||
@ -47,6 +47,9 @@ instance Backend 'BigQuery where
|
||||
|
||||
type ExtraTableMetadata 'BigQuery = ()
|
||||
|
||||
type HealthCheckTest 'BigQuery = Void
|
||||
defaultHealthCheckTest = error "defaultHealthCheckTest"
|
||||
|
||||
isComparableType :: ScalarType 'BigQuery -> Bool
|
||||
isComparableType = BigQuery.isComparableType
|
||||
|
||||
|
||||
@ -70,6 +70,9 @@ instance Backend 'DataConnector where
|
||||
type XNestedInserts 'DataConnector = XDisable
|
||||
type XStreamingSubscription 'DataConnector = XDisable
|
||||
|
||||
type HealthCheckTest 'DataConnector = Void
|
||||
defaultHealthCheckTest = error "defaultHealthCheckTest: not implemented for Data Connector backend"
|
||||
|
||||
isComparableType :: ScalarType 'DataConnector -> Bool
|
||||
isComparableType = \case
|
||||
IR.S.T.Number -> True
|
||||
|
||||
@ -9,12 +9,15 @@
|
||||
module Hasura.Backends.MSSQL.Connection
|
||||
( MSSQLConnConfiguration (MSSQLConnConfiguration),
|
||||
MSSQLSourceConfig (MSSQLSourceConfig, _mscExecCtx),
|
||||
MSSQLConnectionInfo (..),
|
||||
MSSQLPoolSettings (..),
|
||||
MSSQLExecCtx (..),
|
||||
MonadMSSQLTx (..),
|
||||
createMSSQLPool,
|
||||
getEnv,
|
||||
odbcValueToJValue,
|
||||
mkMSSQLExecCtx,
|
||||
mkMSSQLAnyQueryTx,
|
||||
runMSSQLSourceReadTx,
|
||||
runMSSQLSourceWriteTx,
|
||||
)
|
||||
@ -210,6 +213,12 @@ mkMSSQLExecCtx pool =
|
||||
mssqlDestroyConn = MSPool.drainMSSQLPool pool
|
||||
}
|
||||
|
||||
-- | Run any query discarding its results
|
||||
mkMSSQLAnyQueryTx :: ODBC.Query -> MSTx.TxET QErr IO ()
|
||||
mkMSSQLAnyQueryTx q = do
|
||||
_discard :: [[ODBC.Value]] <- MSTx.multiRowQueryE defaultMSSQLTxErrorHandler q
|
||||
pure ()
|
||||
|
||||
data MSSQLSourceConfig = MSSQLSourceConfig
|
||||
{ _mscConnectionString :: MSPool.ConnectionString,
|
||||
_mscExecCtx :: MSSQLExecCtx
|
||||
|
||||
@ -16,6 +16,7 @@ import Hasura.Backends.MSSQL.Types.Update qualified as MSSQL (BackendUpdate)
|
||||
import Hasura.Base.Error
|
||||
import Hasura.Prelude
|
||||
import Hasura.RQL.Types.Backend
|
||||
import Hasura.RQL.Types.HealthCheck
|
||||
import Hasura.SQL.Backend
|
||||
import Language.GraphQL.Draft.Syntax qualified as G
|
||||
|
||||
@ -57,6 +58,11 @@ instance Backend 'MSSQL where
|
||||
type XNestedInserts 'MSSQL = XDisable
|
||||
type XStreamingSubscription 'MSSQL = XDisable
|
||||
|
||||
type HealthCheckTest 'MSSQL = HealthCheckTestSql
|
||||
|
||||
defaultHealthCheckTest :: HealthCheckTest 'MSSQL
|
||||
defaultHealthCheckTest = defaultHealthCheckTestSql
|
||||
|
||||
isComparableType :: ScalarType 'MSSQL -> Bool
|
||||
isComparableType = MSSQL.isComparableType
|
||||
|
||||
|
||||
@ -42,6 +42,9 @@ instance Backend 'MySQL where
|
||||
type XNestedInserts 'MySQL = XDisable
|
||||
type XStreamingSubscription 'MySQL = XDisable
|
||||
|
||||
type HealthCheckTest 'MySQL = Void
|
||||
defaultHealthCheckTest = error "defaultHealthCheckTest"
|
||||
|
||||
isComparableType :: ScalarType 'MySQL -> Bool
|
||||
isComparableType = isNumType @'MySQL -- TODO: For now we only allow comparisons for numeric types
|
||||
|
||||
|
||||
@ -28,6 +28,7 @@ import Hasura.Base.Error
|
||||
import Hasura.Prelude
|
||||
import Hasura.RQL.IR.BoolExp.AggregationPredicates qualified as Agg
|
||||
import Hasura.RQL.Types.Backend
|
||||
import Hasura.RQL.Types.HealthCheck
|
||||
import Hasura.SQL.Backend
|
||||
import Hasura.SQL.Tag
|
||||
|
||||
@ -107,6 +108,9 @@ instance
|
||||
type XNestedInserts ('Postgres pgKind) = XEnable
|
||||
type XStreamingSubscription ('Postgres pgKind) = XEnable
|
||||
|
||||
type HealthCheckTest ('Postgres pgKind) = HealthCheckTestSql
|
||||
defaultHealthCheckTest = defaultHealthCheckTestSql
|
||||
|
||||
isComparableType = PG.isComparableType
|
||||
isNumType = PG.isNumType
|
||||
textToScalarValue = PG.textToScalarValue
|
||||
|
||||
@ -233,6 +233,8 @@ instance Cacheable Word16 where unchanged _ = (==)
|
||||
|
||||
instance Cacheable Key where unchanged _ = (==)
|
||||
|
||||
instance Cacheable Seconds where unchanged _ = (==)
|
||||
|
||||
-- instances for CronSchedule from package `cron`
|
||||
instance Cacheable StepField
|
||||
|
||||
|
||||
@ -129,6 +129,7 @@ runClearMetadata _ = do
|
||||
(_smConfiguration @b s)
|
||||
Nothing
|
||||
emptySourceCustomization
|
||||
Nothing
|
||||
in emptyMetadata
|
||||
& metaSources %~ OMap.insert defaultSource emptyDefaultSource
|
||||
runReplaceMetadataV1 $ RMWithSources emptyMetadata'
|
||||
|
||||
@ -236,6 +236,22 @@ instance
|
||||
invalidations
|
||||
)
|
||||
|
||||
-- | Generate health checks related cache from sources metadata
|
||||
buildHealthCheckCache :: Sources -> SourceHealthCheckCache
|
||||
buildHealthCheckCache sources =
|
||||
catMaybes $ M.fromList $ map (second mkSourceHealthCheck) (OMap.toList sources)
|
||||
where
|
||||
mkSourceHealthCheck :: BackendSourceMetadata -> Maybe BackendSourceHealthCheckInfo
|
||||
mkSourceHealthCheck (BackendSourceMetadata sourceMetadata) =
|
||||
AB.traverseBackend @Backend sourceMetadata mkSourceHealthCheckBackend
|
||||
|
||||
mkSourceHealthCheckBackend :: SourceMetadata b -> Maybe (SourceHealthCheckInfo b)
|
||||
mkSourceHealthCheckBackend sourceMetadata =
|
||||
let sourceName = _smName sourceMetadata
|
||||
connection = _smConfiguration sourceMetadata
|
||||
healthCheck = _smHealthCheckConfig sourceMetadata
|
||||
in SourceHealthCheckInfo sourceName connection <$> healthCheck
|
||||
|
||||
buildSchemaCacheRule ::
|
||||
-- Note: by supplying BuildReason via MonadReader, it does not participate in caching, which is
|
||||
-- what we want!
|
||||
@ -349,7 +365,8 @@ buildSchemaCacheRule logger env = proc (metadata, invalidationKeys) -> do
|
||||
scSetGraphqlIntrospectionOptions = _metaSetGraphqlIntrospectionOptions metadata,
|
||||
scTlsAllowlist = _boTlsAllowlist resolvedOutputs,
|
||||
scQueryCollections = _boQueryCollections resolvedOutputs,
|
||||
scDataConnectorCapabilities = _boDataConnectorCapabilities resolvedOutputs
|
||||
scDataConnectorCapabilities = _boDataConnectorCapabilities resolvedOutputs,
|
||||
scSourceHealthChecks = buildHealthCheckCache (_metaSources metadata)
|
||||
}
|
||||
where
|
||||
getDataConnectorCapabilitiesIfNeeded ::
|
||||
@ -540,7 +557,7 @@ buildSchemaCacheRule logger env = proc (metadata, invalidationKeys) -> do
|
||||
)
|
||||
`arr` BackendSourceInfo
|
||||
buildSource = proc (allSources, sourceMetadata, sourceConfig, tablesRawInfo, eventTriggerInfoMaps, _dbTables, dbFunctions, remoteSchemaMap, orderedRoles) -> do
|
||||
let SourceMetadata sourceName _backendKind tables functions _ queryTagsConfig sourceCustomization = sourceMetadata
|
||||
let SourceMetadata sourceName _backendKind tables functions _ queryTagsConfig sourceCustomization _healthCheckConfig = sourceMetadata
|
||||
tablesMetadata = OMap.elems tables
|
||||
(_, nonColumnInputs, permissions) = unzip3 $ map mkTableInputs tablesMetadata
|
||||
alignTableMap :: HashMap (TableName b) a -> HashMap (TableName b) c -> HashMap (TableName b) (a, c)
|
||||
@ -658,7 +675,7 @@ buildSchemaCacheRule logger env = proc (metadata, invalidationKeys) -> do
|
||||
HS.fromList $
|
||||
concat $
|
||||
OMap.elems sources >>= \(BackendSourceMetadata e) ->
|
||||
AB.dispatchAnyBackend @Backend e \(SourceMetadata _ _ tables _functions _ _ _) -> do
|
||||
AB.dispatchAnyBackend @Backend e \(SourceMetadata _ _ tables _functions _ _ _ _) -> do
|
||||
table <- OMap.elems tables
|
||||
pure $
|
||||
OMap.keys (_tmInsertPermissions table)
|
||||
|
||||
@ -56,6 +56,7 @@ import Hasura.RQL.Types.Backend
|
||||
import Hasura.RQL.Types.Backend qualified as RQL.Types
|
||||
import Hasura.RQL.Types.Common
|
||||
import Hasura.RQL.Types.Common qualified as Common
|
||||
import Hasura.RQL.Types.HealthCheck (HealthCheckConfig)
|
||||
import Hasura.RQL.Types.Metadata
|
||||
import Hasura.RQL.Types.Metadata qualified as Metadata
|
||||
import Hasura.RQL.Types.Metadata.Backend
|
||||
@ -85,7 +86,8 @@ data AddSource b = AddSource
|
||||
_asBackendKind :: BackendSourceKind b,
|
||||
_asConfiguration :: SourceConnConfiguration b,
|
||||
_asReplaceConfiguration :: Bool,
|
||||
_asCustomization :: SourceCustomization
|
||||
_asCustomization :: SourceCustomization,
|
||||
_asHealthCheckConfig :: Maybe (HealthCheckConfig b)
|
||||
}
|
||||
|
||||
instance (Backend b) => FromJSONWithContext (BackendSourceKind b) (AddSource b) where
|
||||
@ -96,13 +98,14 @@ instance (Backend b) => FromJSONWithContext (BackendSourceKind b) (AddSource b)
|
||||
<*> o .: "configuration"
|
||||
<*> o .:? "replace_configuration" .!= False
|
||||
<*> o .:? "customization" .!= emptySourceCustomization
|
||||
<*> o .:? "health_check"
|
||||
|
||||
runAddSource ::
|
||||
forall m b.
|
||||
(MonadError QErr m, CacheRWM m, MetadataM m, BackendMetadata b) =>
|
||||
AddSource b ->
|
||||
m EncJSON
|
||||
runAddSource (AddSource name backendKind sourceConfig replaceConfiguration sourceCustomization) = do
|
||||
runAddSource (AddSource name backendKind sourceConfig replaceConfiguration sourceCustomization healthCheckConfig) = do
|
||||
sources <- scSources <$> askSchemaCache
|
||||
|
||||
metadataModifier <-
|
||||
@ -118,7 +121,7 @@ runAddSource (AddSource name backendKind sourceConfig replaceConfiguration sourc
|
||||
else throw400 AlreadyExists $ "source with name " <> name <<> " already exists"
|
||||
else do
|
||||
let sourceMetadata =
|
||||
mkSourceMetadata @b name backendKind sourceConfig sourceCustomization
|
||||
mkSourceMetadata @b name backendKind sourceConfig sourceCustomization healthCheckConfig
|
||||
pure $ metaSources %~ OMap.insert name sourceMetadata
|
||||
|
||||
buildSchemaCacheFor (MOSource name) metadataModifier
|
||||
@ -279,7 +282,8 @@ runPostDropSourceHook sourceName sourceInfo = do
|
||||
data UpdateSource b = UpdateSource
|
||||
{ _usName :: SourceName,
|
||||
_usConfiguration :: Maybe (SourceConnConfiguration b),
|
||||
_usCustomization :: Maybe SourceCustomization
|
||||
_usCustomization :: Maybe SourceCustomization,
|
||||
_usHealthCheckConfig :: Maybe (HealthCheckConfig b)
|
||||
}
|
||||
|
||||
instance (Backend b) => FromJSONWithContext (BackendSourceKind b) (UpdateSource b) where
|
||||
@ -288,13 +292,14 @@ instance (Backend b) => FromJSONWithContext (BackendSourceKind b) (UpdateSource
|
||||
<$> o .: "name"
|
||||
<*> o .:? "configuration"
|
||||
<*> o .:? "customization"
|
||||
<*> o .:? "health_check"
|
||||
|
||||
runUpdateSource ::
|
||||
forall m b.
|
||||
(MonadError QErr m, CacheRWM m, MetadataM m, BackendMetadata b) =>
|
||||
UpdateSource b ->
|
||||
m EncJSON
|
||||
runUpdateSource (UpdateSource name sourceConfig sourceCustomization) = do
|
||||
runUpdateSource (UpdateSource name sourceConfig sourceCustomization healthCheckConfig) = do
|
||||
sources <- scSources <$> askSchemaCache
|
||||
|
||||
metadataModifier <-
|
||||
@ -304,7 +309,8 @@ runUpdateSource (UpdateSource name sourceConfig sourceCustomization) = do
|
||||
let sMetadata = metaSources . ix name . toSourceMetadata @b
|
||||
updateConfig = maybe id (\scc -> sMetadata . smConfiguration .~ scc) sourceConfig
|
||||
updateCustomization = maybe id (\scc -> sMetadata . smCustomization .~ scc) sourceCustomization
|
||||
pure $ updateConfig . updateCustomization
|
||||
updateHealthCheckConfig = maybe id (\hcc -> sMetadata . smHealthCheckConfig .~ Just hcc) healthCheckConfig
|
||||
pure $ updateHealthCheckConfig . updateConfig . updateCustomization
|
||||
else do
|
||||
throw400 NotExists $ "source with name " <> name <<> " does not exist"
|
||||
|
||||
|
||||
@ -83,6 +83,7 @@ class
|
||||
Representable (ComputedFieldDefinition b),
|
||||
Representable (ComputedFieldImplicitArguments b),
|
||||
Representable (ComputedFieldReturn b),
|
||||
Representable (HealthCheckTest b),
|
||||
Ord (TableName b),
|
||||
Ord (FunctionName b),
|
||||
Ord (ScalarType b),
|
||||
@ -97,6 +98,7 @@ class
|
||||
FromJSON (ExtraTableMetadata b),
|
||||
FromJSON (ComputedFieldDefinition b),
|
||||
FromJSON (BackendSourceKind b),
|
||||
FromJSON (HealthCheckTest b),
|
||||
FromJSONKey (Column b),
|
||||
HasCodec (BackendSourceKind b),
|
||||
HasCodec (SourceConnConfiguration b),
|
||||
@ -114,6 +116,7 @@ class
|
||||
ToJSON (ComputedFieldDefinition b),
|
||||
ToJSON (ComputedFieldImplicitArguments b),
|
||||
ToJSON (ComputedFieldReturn b),
|
||||
ToJSON (HealthCheckTest b),
|
||||
ToJSONKey (Column b),
|
||||
ToJSONKey (FunctionName b),
|
||||
ToJSONKey (ScalarType b),
|
||||
@ -231,6 +234,12 @@ class
|
||||
-- | Computed field return information
|
||||
type ComputedFieldReturn b :: Type
|
||||
|
||||
-- | A config type for health check tests
|
||||
type HealthCheckTest b :: Type
|
||||
|
||||
-- | Default health check test config
|
||||
defaultHealthCheckTest :: HealthCheckTest b
|
||||
|
||||
-- Backend-specific IR types
|
||||
|
||||
-- | Intermediate Representation of extensions to the shared set of boolean
|
||||
|
||||
106
server/src-lib/Hasura/RQL/Types/HealthCheck.hs
Normal file
106
server/src-lib/Hasura/RQL/Types/HealthCheck.hs
Normal file
@ -0,0 +1,106 @@
|
||||
module Hasura.RQL.Types.HealthCheck
|
||||
( HealthCheckConfig (..),
|
||||
HealthCheckTestSql (..),
|
||||
HealthCheckInterval (..),
|
||||
HealthCheckRetries (..),
|
||||
HealthCheckRetryInterval (..),
|
||||
HealthCheckTimeout (..),
|
||||
defaultHealthCheckTestSql,
|
||||
)
|
||||
where
|
||||
|
||||
import Autodocodec hiding (object, (.=))
|
||||
import Autodocodec qualified as AC
|
||||
import Data.Aeson.Extended
|
||||
import Hasura.Incremental (Cacheable)
|
||||
import Hasura.Metadata.DTO.Placeholder (placeholderCodecViaJSON)
|
||||
import Hasura.Prelude
|
||||
import Hasura.RQL.Types.Backend
|
||||
|
||||
newtype HealthCheckTestSql = HealthCheckTestSql
|
||||
{ _hctSql :: Text
|
||||
}
|
||||
deriving (Eq, Generic, Show, Cacheable, Hashable, NFData)
|
||||
|
||||
instance ToJSON HealthCheckTestSql where
|
||||
toJSON = genericToJSON hasuraJSON {omitNothingFields = True}
|
||||
|
||||
instance FromJSON HealthCheckTestSql where
|
||||
parseJSON = withObject "Object" $ \o ->
|
||||
HealthCheckTestSql <$> o .:? "sql" .!= defaultTestSql
|
||||
|
||||
defaultHealthCheckTestSql :: HealthCheckTestSql
|
||||
defaultHealthCheckTestSql = HealthCheckTestSql defaultTestSql
|
||||
|
||||
defaultTestSql :: Text
|
||||
defaultTestSql = "SELECT 1"
|
||||
|
||||
newtype HealthCheckInterval = HealthCheckInterval {unHealthCheckInterval :: Seconds}
|
||||
deriving (Eq, Generic, Show, Cacheable, ToJSON, FromJSON)
|
||||
|
||||
instance HasCodec HealthCheckInterval where
|
||||
codec = AC.codecViaAeson "HealthCheckInterval"
|
||||
|
||||
newtype HealthCheckRetries = HealthCheckRetries {unHealthCheckRetries :: Int}
|
||||
deriving (Eq, Generic, Show, Cacheable, FromJSON, ToJSON)
|
||||
|
||||
instance HasCodec HealthCheckRetries where
|
||||
codec = AC.codecViaAeson "HealthCheckRetries"
|
||||
|
||||
newtype HealthCheckRetryInterval = HealthCheckRetryInterval {unHealthCheckRetryInterval :: Seconds}
|
||||
deriving (Eq, Generic, Show, Cacheable, ToJSON, FromJSON)
|
||||
|
||||
instance HasCodec HealthCheckRetryInterval where
|
||||
codec = AC.codecViaAeson "HealthCheckRetryInterval"
|
||||
|
||||
newtype HealthCheckTimeout = HealthCheckTimeout {unHealthCheckTimeout :: Seconds}
|
||||
deriving (Eq, Generic, Show, Cacheable, FromJSON, ToJSON)
|
||||
|
||||
instance HasCodec HealthCheckTimeout where
|
||||
codec = AC.codecViaAeson "HealthCheckTimeout"
|
||||
|
||||
data HealthCheckConfig b = HealthCheckConfig
|
||||
{ _hccTest :: HealthCheckTest b,
|
||||
_hccInterval :: HealthCheckInterval,
|
||||
_hccRetries :: HealthCheckRetries,
|
||||
_hccRetryInterval :: HealthCheckRetryInterval,
|
||||
_hccTimeout :: HealthCheckTimeout
|
||||
}
|
||||
deriving (Generic)
|
||||
|
||||
deriving instance (Backend b) => Eq (HealthCheckConfig b)
|
||||
|
||||
deriving instance (Backend b) => Show (HealthCheckConfig b)
|
||||
|
||||
instance (Backend b) => Cacheable (HealthCheckConfig b)
|
||||
|
||||
instance (Backend b) => ToJSON (HealthCheckConfig b) where
|
||||
toJSON = genericToJSON hasuraJSON {omitNothingFields = True}
|
||||
|
||||
instance (Backend b) => FromJSON (HealthCheckConfig b) where
|
||||
parseJSON = withObject "Object" $ \o ->
|
||||
HealthCheckConfig
|
||||
<$> o .:? "test" .!= defaultHealthCheckTest @b
|
||||
<*> o .: "interval"
|
||||
<*> o .:? "retries" .!= defaultRetries
|
||||
<*> o .:? "retry_interval" .!= defaultRetryInterval
|
||||
<*> o .:? "timeout" .!= defaultTimeout
|
||||
|
||||
instance (Backend b) => HasCodec (HealthCheckConfig b) where
|
||||
codec =
|
||||
AC.object "HealthCheckConfig" $
|
||||
HealthCheckConfig
|
||||
<$> optionalFieldWithOmittedDefaultWith' "test" placeholderCodecViaJSON (defaultHealthCheckTest @b) AC..= _hccTest
|
||||
<*> requiredField' "interval" AC..= _hccInterval
|
||||
<*> optionalFieldWithOmittedDefault' "retries" defaultRetries AC..= _hccRetries
|
||||
<*> optionalFieldWithOmittedDefault' "retry_interval" defaultRetryInterval AC..= _hccRetryInterval
|
||||
<*> optionalFieldWithOmittedDefault' "timeout" defaultTimeout AC..= _hccTimeout
|
||||
|
||||
defaultRetries :: HealthCheckRetries
|
||||
defaultRetries = HealthCheckRetries 3
|
||||
|
||||
defaultRetryInterval :: HealthCheckRetryInterval
|
||||
defaultRetryInterval = HealthCheckRetryInterval 10
|
||||
|
||||
defaultTimeout :: HealthCheckTimeout
|
||||
defaultTimeout = HealthCheckTimeout 10
|
||||
@ -59,6 +59,7 @@ module Hasura.RQL.Types.Metadata.Common
|
||||
smQueryTags,
|
||||
smTables,
|
||||
smCustomization,
|
||||
smHealthCheckConfig,
|
||||
sourcesCodec,
|
||||
tmArrayRelationships,
|
||||
tmComputedFields,
|
||||
@ -106,6 +107,7 @@ import Hasura.RQL.Types.Endpoint
|
||||
import Hasura.RQL.Types.EventTrigger
|
||||
import Hasura.RQL.Types.Function
|
||||
import Hasura.RQL.Types.GraphqlSchemaIntrospection
|
||||
import Hasura.RQL.Types.HealthCheck
|
||||
import Hasura.RQL.Types.Permission
|
||||
import Hasura.RQL.Types.QueryCollection
|
||||
import Hasura.RQL.Types.QueryTags
|
||||
@ -408,7 +410,8 @@ data SourceMetadata b = SourceMetadata
|
||||
_smFunctions :: Functions b,
|
||||
_smConfiguration :: SourceConnConfiguration b,
|
||||
_smQueryTags :: Maybe QueryTagsConfig,
|
||||
_smCustomization :: SourceCustomization
|
||||
_smCustomization :: SourceCustomization,
|
||||
_smHealthCheckConfig :: Maybe (HealthCheckConfig b)
|
||||
}
|
||||
deriving (Generic)
|
||||
|
||||
@ -428,6 +431,7 @@ instance (Backend b) => FromJSONWithContext (BackendSourceKind b) (SourceMetadat
|
||||
_smConfiguration <- o .: "configuration"
|
||||
_smQueryTags <- o .:? "query_tags"
|
||||
_smCustomization <- o .:? "customization" .!= emptySourceCustomization
|
||||
_smHealthCheckConfig <- o .:? "health_check"
|
||||
pure SourceMetadata {..}
|
||||
|
||||
backendSourceMetadataCodec :: JSONCodec BackendSourceMetadata
|
||||
@ -481,6 +485,7 @@ instance Backend b => HasCodec (SourceMetadata b) where
|
||||
<*> requiredField' "configuration" .== _smConfiguration
|
||||
<*> optionalFieldOrNullWith' "query_tags" placeholderCodecViaJSON .== _smQueryTags -- TODO: replace placeholder
|
||||
<*> optionalFieldOrNullWithOmittedDefault' "customization" emptySourceCustomization .== _smCustomization
|
||||
<*> optionalFieldOrNull' "health_check" .== _smHealthCheckConfig
|
||||
where
|
||||
(.==) = (AC..=)
|
||||
|
||||
@ -491,8 +496,9 @@ mkSourceMetadata ::
|
||||
BackendSourceKind b ->
|
||||
SourceConnConfiguration b ->
|
||||
SourceCustomization ->
|
||||
Maybe (HealthCheckConfig b) ->
|
||||
BackendSourceMetadata
|
||||
mkSourceMetadata name backendSourceKind config customization =
|
||||
mkSourceMetadata name backendSourceKind config customization healthCheckConfig =
|
||||
BackendSourceMetadata $
|
||||
AB.mkAnyBackend $
|
||||
SourceMetadata
|
||||
@ -504,6 +510,7 @@ mkSourceMetadata name backendSourceKind config customization =
|
||||
config
|
||||
Nothing
|
||||
customization
|
||||
healthCheckConfig
|
||||
|
||||
-- | Source configuration as stored in the Metadata DB for some existentialized backend.
|
||||
newtype BackendSourceMetadata = BackendSourceMetadata {unBackendSourceMetadata :: AB.AnyBackend SourceMetadata}
|
||||
|
||||
@ -115,12 +115,14 @@ sourcesToOrdJSONList sources =
|
||||
customizationPair =
|
||||
guard (_smCustomization /= emptySourceCustomization)
|
||||
*> [("customization", AO.toOrdered _smCustomization)]
|
||||
healthCheckPair = maybe [] (\healthCheckConfig -> [("health_check", AO.toOrdered healthCheckConfig)]) _smHealthCheckConfig
|
||||
in AO.object $
|
||||
[sourceNamePair, sourceKindPair, tablesPair]
|
||||
<> maybeToList functionsPair
|
||||
<> configurationPair
|
||||
<> queryTagsConfigPair
|
||||
<> customizationPair
|
||||
<> healthCheckPair
|
||||
|
||||
tableMetaToOrdJSON :: (Backend b) => TableMetadata b -> AO.Value
|
||||
tableMetaToOrdJSON
|
||||
|
||||
@ -504,7 +504,8 @@ data SchemaCache = SchemaCache
|
||||
scSetGraphqlIntrospectionOptions :: SetGraphqlIntrospectionOptions,
|
||||
scTlsAllowlist :: [TlsAllow],
|
||||
scQueryCollections :: QueryCollections,
|
||||
scDataConnectorCapabilities :: DataConnectorCapabilities
|
||||
scDataConnectorCapabilities :: DataConnectorCapabilities,
|
||||
scSourceHealthChecks :: SourceHealthCheckCache
|
||||
}
|
||||
|
||||
-- WARNING: this can only be used for debug purposes, as it loses all
|
||||
|
||||
@ -26,6 +26,11 @@ module Hasura.RQL.Types.Source
|
||||
SourceResolver,
|
||||
MonadResolveSource (..),
|
||||
MaintenanceModeVersion (..),
|
||||
|
||||
-- * Health check
|
||||
SourceHealthCheckInfo (..),
|
||||
BackendSourceHealthCheckInfo,
|
||||
SourceHealthCheckCache,
|
||||
)
|
||||
where
|
||||
|
||||
@ -38,6 +43,7 @@ import Hasura.Prelude
|
||||
import Hasura.RQL.Types.Backend
|
||||
import Hasura.RQL.Types.Common
|
||||
import Hasura.RQL.Types.Function
|
||||
import Hasura.RQL.Types.HealthCheck
|
||||
import Hasura.RQL.Types.Instances ()
|
||||
import Hasura.RQL.Types.QueryTags
|
||||
import Hasura.RQL.Types.SourceCustomization
|
||||
@ -172,3 +178,16 @@ data MaintenanceModeVersion
|
||||
| -- | should correspond to the latest source catalog version
|
||||
CurrentMMVersion
|
||||
deriving (Show, Eq)
|
||||
|
||||
-------------------------------------------------------------------------------
|
||||
-- Source health check
|
||||
|
||||
data SourceHealthCheckInfo b = SourceHealthCheckInfo
|
||||
{ _shciName :: SourceName,
|
||||
_shciConnection :: SourceConnConfiguration b,
|
||||
_shciHealthCheck :: HealthCheckConfig b
|
||||
}
|
||||
|
||||
type BackendSourceHealthCheckInfo = AB.AnyBackend SourceHealthCheckInfo
|
||||
|
||||
type SourceHealthCheckCache = HashMap SourceName BackendSourceHealthCheckInfo
|
||||
|
||||
@ -159,6 +159,7 @@ migrateCatalog maybeDefaultSourceConfig extensionsSchema maintenanceMode migrati
|
||||
defaultSourceConfig
|
||||
Nothing
|
||||
emptySourceCustomization
|
||||
Nothing
|
||||
sources = OMap.singleton defaultSource $ BackendSourceMetadata defaultSourceMetadata
|
||||
in emptyMetadata {_metaSources = sources}
|
||||
|
||||
@ -328,7 +329,7 @@ migrations maybeDefaultSourceConfig dryRun maintenanceMode =
|
||||
defaultSourceMetadata =
|
||||
BackendSourceMetadata $
|
||||
AB.mkAnyBackend $
|
||||
SourceMetadata defaultSource PostgresVanillaKind _mnsTables _mnsFunctions defaultSourceConfig Nothing emptySourceCustomization
|
||||
SourceMetadata defaultSource PostgresVanillaKind _mnsTables _mnsFunctions defaultSourceConfig Nothing emptySourceCustomization Nothing
|
||||
in Metadata
|
||||
(OMap.singleton defaultSource defaultSourceMetadata)
|
||||
_mnsRemoteSchemas
|
||||
|
||||
Loading…
Reference in New Issue
Block a user