1
0
mirror of https://github.com/hasura/graphql-engine.git synced 2024-12-15 09:22:43 +03:00

Initial advanced config for data connectors

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10091
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Rob Dominguez <24390149+robertjdominguez@users.noreply.github.com>
GitOrigin-RevId: a74bb822338b5f738884958fd4abaebf8d3a1925
This commit is contained in:
Brandon Martin 2023-08-17 08:51:16 -06:00 committed by hasura-bot
parent 3b7a601c6e
commit 0902cddd5f
3 changed files with 229 additions and 0 deletions
docs
docs
api-reference
databases/database-config
static/img/databases

View File

@ -45,6 +45,7 @@ At present, Kriti templating is available for:
- [Event Triggers REST Connectors](/event-triggers/rest-connectors.mdx) - [Event Triggers REST Connectors](/event-triggers/rest-connectors.mdx)
- [Cron Triggers REST Connectors](/scheduled-triggers/create-cron-trigger.mdx#rest-connectors) - [Cron Triggers REST Connectors](/scheduled-triggers/create-cron-trigger.mdx#rest-connectors)
- [Dynamic Database Connection Routing](databases/database-config/dynamic-db-connection.mdx#connection-template) - [Dynamic Database Connection Routing](databases/database-config/dynamic-db-connection.mdx#connection-template)
- [Data Connector Config](databases/database-config/data-connector-config.mdx)
### Example ### Example

View File

@ -0,0 +1,228 @@
---
description: Hasura Cloud and Hasura Enterprise data connector configuration
title: 'Cloud and Enterprise Edition: Data connector configuration'
keywords:
- hasura
- docs
- cloud
- enterprise
- configuration
- data connector
- data connector configuration
- connections
sidebar_label: Data Connector Config
sidebar_position: 4
---
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import Thumbnail from '@site/src/components/Thumbnail';
import HeadingIcon from '@site/src/components/HeadingIcon';
import ProductBadge from '@site/src/components/ProductBadge';
# Data Connector Config
<ProductBadge ce free standard pro ee self />
## Introduction
The configuration of [data connectors](/databases/data-connectors/index.mdx) depends on the specific data connector
you are connecting to. But, two options are available that are common amongst all data
connectors. Those two options are:
1. [Timeout](#timeout)
2. [Template](#template)
:::info This configuration only applies to Hasura Data Connector Agents
Please note that this only applies to [data connectors](/databases/data-connectors/index.mdx). If you are using a
native database connection then this configuration does not apply. You can learn more about connecting to a
native database [here](/databases/overview.mdx#supported-databases).
:::
### Timeout
The timeout setting is related to how long the graphql engine will wait for a response from the
data connector agent before throwing an error. This value defaults to 30 seconds.
As an example, if it takes more than 30 seconds to introspect the schema of a data connector, then it
is likely you may be connecting to a slow or latent data source. In this case, you may want to increase
the timeout value to allow for the introspection to complete.
Timeout has three available setting options:
1. Seconds `seconds` `{ "seconds": 60 }`
2. Milliseconds `milliseconds` `{ "milliseconds": 60000 }`
3. Microseconds `microseconds` `{ "microseconds": 60000000 }`
<Tabs groupId="timeout-setting" className="timeout-settings-tabs">
<TabItem value="console" label="Console">
:::info Console only supports `seconds` timeout
The Console only gives you the option to use `seconds`. If you need to use `milliseconds` or `microseconds` you
will need to use CLI or the API.
:::
In the Console, navigate to the `Data` tab and select your data connector. Then, under the `Advanced` tab you
will can set the timeout value.
<Thumbnail
src="/img/databases/data-connector-config.png"
alt="Data connector advanced config"
/>
</TabItem>
<TabItem value="cli" label="CLI">
You can add a _timeout_ for a data connector database by adding their config to the
`/metadata/databases/database.yaml` file. In the example below, we're using
Snowflake:
```yaml {5}
- name: snowflake
kind: snowflake
configuration:
template: null
timeout: { "seconds": 120 }
value: {
"jdbc_url": "jdbc:snowflake://url.snowflakecomputing.com/?user=user&password=password&warehouse=warehouse&db=db&role=role&schema=schema"
}
```
</TabItem>
<TabItem value="api" label="API">
The _timeout_ via the [{{connector_name}}_add_source](/api-reference/metadata-api/source.mdx) Metadata API. In the example below, we're using
Snowflake:
```http {11}
POST /v1/metadata HTTP/1.1
Content-Type: application/json
X-Hasura-Role: admin
{
"type": "snowflake_add_source",
"args": {
"name": "db",
"configuration": {
"template": null,
"timeout": { "seconds": 120 },
"value": {
"jdbc_url": "jdbc:snowflake://url.snowflakecomputing.com/?user=user&password=password&warehouse=warehouse&db=db&role=role&schema=schema"
}
}
}
}
```
</TabItem>
</Tabs>
### Template
You can use the template field to either set connection strings from `ENV` variables or implement more complex logic
for a connection string based on some other available variables. Currently, the template for data connectors
has access to:
- [Environment variables](#environment-variables)
- [Session variables](#session-variables)
- [Configuration variables](#configuration-variables)
#### Environment variables
Environment variables can be accessed using the variable `$env` or using the function `getEnvironmentVariable`
in your Kriti templating. A simple example would be to use the environment variable `SNOWFLAKE_JDBC_URL` to set
the connection string for a Snowflake data connector:
```
{"jdbc_url": "$env["SNOWFLAKE_JDBC_URL"]"}
{"jdbc_url": "getEnvironmentVariable("SNOWFLAKE_JDBC_URL")"}
```
#### Session variables
You can refer to session variables in the template using the variable `$session`. The session variable keys always contain
`x-hasura-*` as the prefix:
```
{{if (empty($session?['x-hasura-role'])) && (empty($session?['x-hasura-user-id']))}}
{"jdbc_url": "jdbc:snowflake://url.snowflakecomputing.com/?user=getEnvironmentVariable("DEFAULT_USER")&password=getEnvironmentVariable("DEFAULT_PASS")&warehouse=warehouse&db=db&role=role&schema=schema"}
{{else}}
{"jdbc_url": "jdbc:snowflake://url.snowflakecomputing.com/?user={{$session['x-hasura-role']}}&password={{$session['x-hasura-user-id']}}&warehouse=warehouse&db=db&role=role&schema=schema"}
{{end}}
```
In this example, if the session variables `x-hasura-role` and `x-hasura-user-id` are both empty, then the graphql engine will use
the connection string with the `DEFAULT_USER` and the `DEFAULT_PASS` from our environment variables.
Otherwise, the graphql engine will use the values of `x-hasura-role` and `x-hasura-user-id` to generate the
connection string dynamically.
#### Configuration variables
Config variables can be accessed using the variable `$config`. This allows you to access any field from the
data connector's configuration fields. As a simple example you could setup your template to check the value
of the JDBC url configuration field as environment variable or fallback to using the value as the JDBC url.
```
{"jdbc_url":{{$env?[$config.jdbc_url] ?? $config.jdbc_url}}}
```
<Tabs groupId="timeout-setting" className="timeout-settings-tabs">
<TabItem value="console" label="Console">
In the Console, navigate to the `Data` tab and select your data connector. Then, under the `Advanced` tab you
will can set the template value.
<Thumbnail
src="/img/databases/data-connector-config.png"
alt="Data connector advanced config"
/>
</TabItem>
<TabItem value="cli" label="CLI">
You can add a _template_ for a data connector database by adding their config to the
`/metadata/databases/database.yaml` file:
```yaml {5}
- name: snowflake
kind: snowflake
configuration:
template: |
{"jdbc_url": "getEnvironmentVariable("SNOWFLAKE_JDBC_URL")"}
timeout: null
value: {}
```
Apply the Metadata by running:
```yaml
hasura metadata apply
```
</TabItem>
<TabItem value="api" label="API">
The _template_ via the [{{:connector_name}}_add_source](/api-reference/metadata-api/source.mdx) Metadata API.
```http {10}
POST /v1/metadata HTTP/1.1
Content-Type: application/json
X-Hasura-Role: admin
{
"type": "snowflake_add_source",
"args": {
"name": "db",
"configuration": {
"template": "{\"jdbc_url\": \"getEnvironmentVariable(\"SNOWFLAKE_JDBC_URL\")\"}",
"timeout": null,
"value": {}
}
}
}
```
</TabItem>
</Tabs>

Binary file not shown.

After

(image error) Size: 88 KiB