mirror of
https://github.com/hasura/graphql-engine.git
synced 2024-12-15 09:22:43 +03:00
Initial advanced config for data connectors
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10091 Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Rob Dominguez <24390149+robertjdominguez@users.noreply.github.com> GitOrigin-RevId: a74bb822338b5f738884958fd4abaebf8d3a1925
This commit is contained in:
parent
3b7a601c6e
commit
0902cddd5f
docs
docs
static/img/databases
@ -45,6 +45,7 @@ At present, Kriti templating is available for:
|
|||||||
- [Event Triggers REST Connectors](/event-triggers/rest-connectors.mdx)
|
- [Event Triggers REST Connectors](/event-triggers/rest-connectors.mdx)
|
||||||
- [Cron Triggers REST Connectors](/scheduled-triggers/create-cron-trigger.mdx#rest-connectors)
|
- [Cron Triggers REST Connectors](/scheduled-triggers/create-cron-trigger.mdx#rest-connectors)
|
||||||
- [Dynamic Database Connection Routing](databases/database-config/dynamic-db-connection.mdx#connection-template)
|
- [Dynamic Database Connection Routing](databases/database-config/dynamic-db-connection.mdx#connection-template)
|
||||||
|
- [Data Connector Config](databases/database-config/data-connector-config.mdx)
|
||||||
|
|
||||||
### Example
|
### Example
|
||||||
|
|
||||||
|
228
docs/docs/databases/database-config/data-connector-config.mdx
Normal file
228
docs/docs/databases/database-config/data-connector-config.mdx
Normal file
@ -0,0 +1,228 @@
|
|||||||
|
---
|
||||||
|
description: Hasura Cloud and Hasura Enterprise data connector configuration
|
||||||
|
title: 'Cloud and Enterprise Edition: Data connector configuration'
|
||||||
|
keywords:
|
||||||
|
- hasura
|
||||||
|
- docs
|
||||||
|
- cloud
|
||||||
|
- enterprise
|
||||||
|
- configuration
|
||||||
|
- data connector
|
||||||
|
- data connector configuration
|
||||||
|
- connections
|
||||||
|
sidebar_label: Data Connector Config
|
||||||
|
sidebar_position: 4
|
||||||
|
---
|
||||||
|
|
||||||
|
import Tabs from '@theme/Tabs';
|
||||||
|
import TabItem from '@theme/TabItem';
|
||||||
|
import Thumbnail from '@site/src/components/Thumbnail';
|
||||||
|
import HeadingIcon from '@site/src/components/HeadingIcon';
|
||||||
|
import ProductBadge from '@site/src/components/ProductBadge';
|
||||||
|
|
||||||
|
# Data Connector Config
|
||||||
|
|
||||||
|
<ProductBadge ce free standard pro ee self />
|
||||||
|
|
||||||
|
## Introduction
|
||||||
|
|
||||||
|
The configuration of [data connectors](/databases/data-connectors/index.mdx) depends on the specific data connector
|
||||||
|
you are connecting to. But, two options are available that are common amongst all data
|
||||||
|
connectors. Those two options are:
|
||||||
|
|
||||||
|
1. [Timeout](#timeout)
|
||||||
|
2. [Template](#template)
|
||||||
|
|
||||||
|
:::info This configuration only applies to Hasura Data Connector Agents
|
||||||
|
|
||||||
|
Please note that this only applies to [data connectors](/databases/data-connectors/index.mdx). If you are using a
|
||||||
|
native database connection then this configuration does not apply. You can learn more about connecting to a
|
||||||
|
native database [here](/databases/overview.mdx#supported-databases).
|
||||||
|
|
||||||
|
:::
|
||||||
|
|
||||||
|
### Timeout
|
||||||
|
|
||||||
|
The timeout setting is related to how long the graphql engine will wait for a response from the
|
||||||
|
data connector agent before throwing an error. This value defaults to 30 seconds.
|
||||||
|
|
||||||
|
As an example, if it takes more than 30 seconds to introspect the schema of a data connector, then it
|
||||||
|
is likely you may be connecting to a slow or latent data source. In this case, you may want to increase
|
||||||
|
the timeout value to allow for the introspection to complete.
|
||||||
|
|
||||||
|
Timeout has three available setting options:
|
||||||
|
|
||||||
|
1. Seconds `seconds` `{ "seconds": 60 }`
|
||||||
|
2. Milliseconds `milliseconds` `{ "milliseconds": 60000 }`
|
||||||
|
3. Microseconds `microseconds` `{ "microseconds": 60000000 }`
|
||||||
|
|
||||||
|
<Tabs groupId="timeout-setting" className="timeout-settings-tabs">
|
||||||
|
<TabItem value="console" label="Console">
|
||||||
|
|
||||||
|
:::info Console only supports `seconds` timeout
|
||||||
|
The Console only gives you the option to use `seconds`. If you need to use `milliseconds` or `microseconds` you
|
||||||
|
will need to use CLI or the API.
|
||||||
|
:::
|
||||||
|
|
||||||
|
In the Console, navigate to the `Data` tab and select your data connector. Then, under the `Advanced` tab you
|
||||||
|
will can set the timeout value.
|
||||||
|
|
||||||
|
<Thumbnail
|
||||||
|
src="/img/databases/data-connector-config.png"
|
||||||
|
alt="Data connector advanced config"
|
||||||
|
/>
|
||||||
|
|
||||||
|
</TabItem>
|
||||||
|
<TabItem value="cli" label="CLI">
|
||||||
|
|
||||||
|
You can add a _timeout_ for a data connector database by adding their config to the
|
||||||
|
`/metadata/databases/database.yaml` file. In the example below, we're using
|
||||||
|
Snowflake:
|
||||||
|
|
||||||
|
```yaml {5}
|
||||||
|
- name: snowflake
|
||||||
|
kind: snowflake
|
||||||
|
configuration:
|
||||||
|
template: null
|
||||||
|
timeout: { "seconds": 120 }
|
||||||
|
value: {
|
||||||
|
"jdbc_url": "jdbc:snowflake://url.snowflakecomputing.com/?user=user&password=password&warehouse=warehouse&db=db&role=role&schema=schema"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
</TabItem>
|
||||||
|
<TabItem value="api" label="API">
|
||||||
|
|
||||||
|
The _timeout_ via the [{{connector_name}}_add_source](/api-reference/metadata-api/source.mdx) Metadata API. In the example below, we're using
|
||||||
|
Snowflake:
|
||||||
|
|
||||||
|
```http {11}
|
||||||
|
POST /v1/metadata HTTP/1.1
|
||||||
|
Content-Type: application/json
|
||||||
|
X-Hasura-Role: admin
|
||||||
|
|
||||||
|
{
|
||||||
|
"type": "snowflake_add_source",
|
||||||
|
"args": {
|
||||||
|
"name": "db",
|
||||||
|
"configuration": {
|
||||||
|
"template": null,
|
||||||
|
"timeout": { "seconds": 120 },
|
||||||
|
"value": {
|
||||||
|
"jdbc_url": "jdbc:snowflake://url.snowflakecomputing.com/?user=user&password=password&warehouse=warehouse&db=db&role=role&schema=schema"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
</TabItem>
|
||||||
|
</Tabs>
|
||||||
|
|
||||||
|
### Template
|
||||||
|
|
||||||
|
You can use the template field to either set connection strings from `ENV` variables or implement more complex logic
|
||||||
|
for a connection string based on some other available variables. Currently, the template for data connectors
|
||||||
|
has access to:
|
||||||
|
|
||||||
|
- [Environment variables](#environment-variables)
|
||||||
|
- [Session variables](#session-variables)
|
||||||
|
- [Configuration variables](#configuration-variables)
|
||||||
|
|
||||||
|
#### Environment variables
|
||||||
|
|
||||||
|
Environment variables can be accessed using the variable `$env` or using the function `getEnvironmentVariable`
|
||||||
|
in your Kriti templating. A simple example would be to use the environment variable `SNOWFLAKE_JDBC_URL` to set
|
||||||
|
the connection string for a Snowflake data connector:
|
||||||
|
|
||||||
|
```
|
||||||
|
{"jdbc_url": "$env["SNOWFLAKE_JDBC_URL"]"}
|
||||||
|
{"jdbc_url": "getEnvironmentVariable("SNOWFLAKE_JDBC_URL")"}
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Session variables
|
||||||
|
|
||||||
|
You can refer to session variables in the template using the variable `$session`. The session variable keys always contain
|
||||||
|
`x-hasura-*` as the prefix:
|
||||||
|
|
||||||
|
```
|
||||||
|
{{if (empty($session?['x-hasura-role'])) && (empty($session?['x-hasura-user-id']))}}
|
||||||
|
{"jdbc_url": "jdbc:snowflake://url.snowflakecomputing.com/?user=getEnvironmentVariable("DEFAULT_USER")&password=getEnvironmentVariable("DEFAULT_PASS")&warehouse=warehouse&db=db&role=role&schema=schema"}
|
||||||
|
{{else}}
|
||||||
|
{"jdbc_url": "jdbc:snowflake://url.snowflakecomputing.com/?user={{$session['x-hasura-role']}}&password={{$session['x-hasura-user-id']}}&warehouse=warehouse&db=db&role=role&schema=schema"}
|
||||||
|
{{end}}
|
||||||
|
```
|
||||||
|
|
||||||
|
In this example, if the session variables `x-hasura-role` and `x-hasura-user-id` are both empty, then the graphql engine will use
|
||||||
|
the connection string with the `DEFAULT_USER` and the `DEFAULT_PASS` from our environment variables.
|
||||||
|
Otherwise, the graphql engine will use the values of `x-hasura-role` and `x-hasura-user-id` to generate the
|
||||||
|
connection string dynamically.
|
||||||
|
|
||||||
|
#### Configuration variables
|
||||||
|
|
||||||
|
Config variables can be accessed using the variable `$config`. This allows you to access any field from the
|
||||||
|
data connector's configuration fields. As a simple example you could setup your template to check the value
|
||||||
|
of the JDBC url configuration field as environment variable or fallback to using the value as the JDBC url.
|
||||||
|
|
||||||
|
```
|
||||||
|
{"jdbc_url":{{$env?[$config.jdbc_url] ?? $config.jdbc_url}}}
|
||||||
|
```
|
||||||
|
|
||||||
|
<Tabs groupId="timeout-setting" className="timeout-settings-tabs">
|
||||||
|
<TabItem value="console" label="Console">
|
||||||
|
|
||||||
|
In the Console, navigate to the `Data` tab and select your data connector. Then, under the `Advanced` tab you
|
||||||
|
will can set the template value.
|
||||||
|
|
||||||
|
<Thumbnail
|
||||||
|
src="/img/databases/data-connector-config.png"
|
||||||
|
alt="Data connector advanced config"
|
||||||
|
/>
|
||||||
|
|
||||||
|
</TabItem>
|
||||||
|
<TabItem value="cli" label="CLI">
|
||||||
|
|
||||||
|
You can add a _template_ for a data connector database by adding their config to the
|
||||||
|
`/metadata/databases/database.yaml` file:
|
||||||
|
|
||||||
|
```yaml {5}
|
||||||
|
- name: snowflake
|
||||||
|
kind: snowflake
|
||||||
|
configuration:
|
||||||
|
template: |
|
||||||
|
{"jdbc_url": "getEnvironmentVariable("SNOWFLAKE_JDBC_URL")"}
|
||||||
|
timeout: null
|
||||||
|
value: {}
|
||||||
|
```
|
||||||
|
|
||||||
|
Apply the Metadata by running:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
hasura metadata apply
|
||||||
|
```
|
||||||
|
|
||||||
|
</TabItem>
|
||||||
|
<TabItem value="api" label="API">
|
||||||
|
|
||||||
|
The _template_ via the [{{:connector_name}}_add_source](/api-reference/metadata-api/source.mdx) Metadata API.
|
||||||
|
|
||||||
|
```http {10}
|
||||||
|
POST /v1/metadata HTTP/1.1
|
||||||
|
Content-Type: application/json
|
||||||
|
X-Hasura-Role: admin
|
||||||
|
|
||||||
|
{
|
||||||
|
"type": "snowflake_add_source",
|
||||||
|
"args": {
|
||||||
|
"name": "db",
|
||||||
|
"configuration": {
|
||||||
|
"template": "{\"jdbc_url\": \"getEnvironmentVariable(\"SNOWFLAKE_JDBC_URL\")\"}",
|
||||||
|
"timeout": null,
|
||||||
|
"value": {}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
</TabItem>
|
||||||
|
</Tabs>
|
BIN
docs/static/img/databases/data-connector-config.png
vendored
Normal file
BIN
docs/static/img/databases/data-connector-config.png
vendored
Normal file
Binary file not shown.
After (image error) Size: 88 KiB |
Loading…
Reference in New Issue
Block a user