mirror of
https://github.com/hasura/graphql-engine.git
synced 2024-12-14 17:02:49 +03:00
Initial advanced config for data connectors
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10091 Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Rob Dominguez <24390149+robertjdominguez@users.noreply.github.com> GitOrigin-RevId: a74bb822338b5f738884958fd4abaebf8d3a1925
This commit is contained in:
Brandon Martin
hasura-bot
parent
3b7a601c6e
commit
0902cddd5f
@ -45,6 +45,7 @@ At present, Kriti templating is available for:
|
||||
- [Event Triggers REST Connectors](/event-triggers/rest-connectors.mdx)
|
||||
- [Cron Triggers REST Connectors](/scheduled-triggers/create-cron-trigger.mdx#rest-connectors)
|
||||
- [Dynamic Database Connection Routing](databases/database-config/dynamic-db-connection.mdx#connection-template)
|
||||
- [Data Connector Config](databases/database-config/data-connector-config.mdx)
|
||||
|
||||
### Example
|
||||
|
||||
|
||||
228
docs/docs/databases/database-config/data-connector-config.mdx
Normal file
228
docs/docs/databases/database-config/data-connector-config.mdx
Normal file
@ -0,0 +1,228 @@
|
||||
---
|
||||
description: Hasura Cloud and Hasura Enterprise data connector configuration
|
||||
title: 'Cloud and Enterprise Edition: Data connector configuration'
|
||||
keywords:
|
||||
- hasura
|
||||
- docs
|
||||
- cloud
|
||||
- enterprise
|
||||
- configuration
|
||||
- data connector
|
||||
- data connector configuration
|
||||
- connections
|
||||
sidebar_label: Data Connector Config
|
||||
sidebar_position: 4
|
||||
---
|
||||
|
||||
import Tabs from '@theme/Tabs';
|
||||
import TabItem from '@theme/TabItem';
|
||||
import Thumbnail from '@site/src/components/Thumbnail';
|
||||
import HeadingIcon from '@site/src/components/HeadingIcon';
|
||||
import ProductBadge from '@site/src/components/ProductBadge';
|
||||
|
||||
# Data Connector Config
|
||||
|
||||
<ProductBadge ce free standard pro ee self />
|
||||
|
||||
## Introduction
|
||||
|
||||
The configuration of [data connectors](/databases/data-connectors/index.mdx) depends on the specific data connector
|
||||
you are connecting to. But, two options are available that are common amongst all data
|
||||
connectors. Those two options are:
|
||||
|
||||
1. [Timeout](#timeout)
|
||||
2. [Template](#template)
|
||||
|
||||
:::info This configuration only applies to Hasura Data Connector Agents
|
||||
|
||||
Please note that this only applies to [data connectors](/databases/data-connectors/index.mdx). If you are using a
|
||||
native database connection then this configuration does not apply. You can learn more about connecting to a
|
||||
native database [here](/databases/overview.mdx#supported-databases).
|
||||
|
||||
:::
|
||||
|
||||
### Timeout
|
||||
|
||||
The timeout setting is related to how long the graphql engine will wait for a response from the
|
||||
data connector agent before throwing an error. This value defaults to 30 seconds.
|
||||
|
||||
As an example, if it takes more than 30 seconds to introspect the schema of a data connector, then it
|
||||
is likely you may be connecting to a slow or latent data source. In this case, you may want to increase
|
||||
the timeout value to allow for the introspection to complete.
|
||||
|
||||
Timeout has three available setting options:
|
||||
|
||||
1. Seconds `seconds` `{ "seconds": 60 }`
|
||||
2. Milliseconds `milliseconds` `{ "milliseconds": 60000 }`
|
||||
3. Microseconds `microseconds` `{ "microseconds": 60000000 }`
|
||||
|
||||
<Tabs groupId="timeout-setting" className="timeout-settings-tabs">
|
||||
<TabItem value="console" label="Console">
|
||||
|
||||
:::info Console only supports `seconds` timeout
|
||||
The Console only gives you the option to use `seconds`. If you need to use `milliseconds` or `microseconds` you
|
||||
will need to use CLI or the API.
|
||||
:::
|
||||
|
||||
In the Console, navigate to the `Data` tab and select your data connector. Then, under the `Advanced` tab you
|
||||
will can set the timeout value.
|
||||
|
||||
<Thumbnail
|
||||
src="/img/databases/data-connector-config.png"
|
||||
alt="Data connector advanced config"
|
||||
/>
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="cli" label="CLI">
|
||||
|
||||
You can add a _timeout_ for a data connector database by adding their config to the
|
||||
`/metadata/databases/database.yaml` file. In the example below, we're using
|
||||
Snowflake:
|
||||
|
||||
```yaml {5}
|
||||
- name: snowflake
|
||||
kind: snowflake
|
||||
configuration:
|
||||
template: null
|
||||
timeout: { "seconds": 120 }
|
||||
value: {
|
||||
"jdbc_url": "jdbc:snowflake://url.snowflakecomputing.com/?user=user&password=password&warehouse=warehouse&db=db&role=role&schema=schema"
|
||||
}
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="api" label="API">
|
||||
|
||||
The _timeout_ via the [{{connector_name}}_add_source](/api-reference/metadata-api/source.mdx) Metadata API. In the example below, we're using
|
||||
Snowflake:
|
||||
|
||||
```http {11}
|
||||
POST /v1/metadata HTTP/1.1
|
||||
Content-Type: application/json
|
||||
X-Hasura-Role: admin
|
||||
|
||||
{
|
||||
"type": "snowflake_add_source",
|
||||
"args": {
|
||||
"name": "db",
|
||||
"configuration": {
|
||||
"template": null,
|
||||
"timeout": { "seconds": 120 },
|
||||
"value": {
|
||||
"jdbc_url": "jdbc:snowflake://url.snowflakecomputing.com/?user=user&password=password&warehouse=warehouse&db=db&role=role&schema=schema"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
### Template
|
||||
|
||||
You can use the template field to either set connection strings from `ENV` variables or implement more complex logic
|
||||
for a connection string based on some other available variables. Currently, the template for data connectors
|
||||
has access to:
|
||||
|
||||
- [Environment variables](#environment-variables)
|
||||
- [Session variables](#session-variables)
|
||||
- [Configuration variables](#configuration-variables)
|
||||
|
||||
#### Environment variables
|
||||
|
||||
Environment variables can be accessed using the variable `$env` or using the function `getEnvironmentVariable`
|
||||
in your Kriti templating. A simple example would be to use the environment variable `SNOWFLAKE_JDBC_URL` to set
|
||||
the connection string for a Snowflake data connector:
|
||||
|
||||
```
|
||||
{"jdbc_url": "$env["SNOWFLAKE_JDBC_URL"]"}
|
||||
{"jdbc_url": "getEnvironmentVariable("SNOWFLAKE_JDBC_URL")"}
|
||||
```
|
||||
|
||||
#### Session variables
|
||||
|
||||
You can refer to session variables in the template using the variable `$session`. The session variable keys always contain
|
||||
`x-hasura-*` as the prefix:
|
||||
|
||||
```
|
||||
{{if (empty($session?['x-hasura-role'])) && (empty($session?['x-hasura-user-id']))}}
|
||||
{"jdbc_url": "jdbc:snowflake://url.snowflakecomputing.com/?user=getEnvironmentVariable("DEFAULT_USER")&password=getEnvironmentVariable("DEFAULT_PASS")&warehouse=warehouse&db=db&role=role&schema=schema"}
|
||||
{{else}}
|
||||
{"jdbc_url": "jdbc:snowflake://url.snowflakecomputing.com/?user={{$session['x-hasura-role']}}&password={{$session['x-hasura-user-id']}}&warehouse=warehouse&db=db&role=role&schema=schema"}
|
||||
{{end}}
|
||||
```
|
||||
|
||||
In this example, if the session variables `x-hasura-role` and `x-hasura-user-id` are both empty, then the graphql engine will use
|
||||
the connection string with the `DEFAULT_USER` and the `DEFAULT_PASS` from our environment variables.
|
||||
Otherwise, the graphql engine will use the values of `x-hasura-role` and `x-hasura-user-id` to generate the
|
||||
connection string dynamically.
|
||||
|
||||
#### Configuration variables
|
||||
|
||||
Config variables can be accessed using the variable `$config`. This allows you to access any field from the
|
||||
data connector's configuration fields. As a simple example you could setup your template to check the value
|
||||
of the JDBC url configuration field as environment variable or fallback to using the value as the JDBC url.
|
||||
|
||||
```
|
||||
{"jdbc_url":{{$env?[$config.jdbc_url] ?? $config.jdbc_url}}}
|
||||
```
|
||||
|
||||
<Tabs groupId="timeout-setting" className="timeout-settings-tabs">
|
||||
<TabItem value="console" label="Console">
|
||||
|
||||
In the Console, navigate to the `Data` tab and select your data connector. Then, under the `Advanced` tab you
|
||||
will can set the template value.
|
||||
|
||||
<Thumbnail
|
||||
src="/img/databases/data-connector-config.png"
|
||||
alt="Data connector advanced config"
|
||||
/>
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="cli" label="CLI">
|
||||
|
||||
You can add a _template_ for a data connector database by adding their config to the
|
||||
`/metadata/databases/database.yaml` file:
|
||||
|
||||
```yaml {5}
|
||||
- name: snowflake
|
||||
kind: snowflake
|
||||
configuration:
|
||||
template: |
|
||||
{"jdbc_url": "getEnvironmentVariable("SNOWFLAKE_JDBC_URL")"}
|
||||
timeout: null
|
||||
value: {}
|
||||
```
|
||||
|
||||
Apply the Metadata by running:
|
||||
|
||||
```yaml
|
||||
hasura metadata apply
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="api" label="API">
|
||||
|
||||
The _template_ via the [{{:connector_name}}_add_source](/api-reference/metadata-api/source.mdx) Metadata API.
|
||||
|
||||
```http {10}
|
||||
POST /v1/metadata HTTP/1.1
|
||||
Content-Type: application/json
|
||||
X-Hasura-Role: admin
|
||||
|
||||
{
|
||||
"type": "snowflake_add_source",
|
||||
"args": {
|
||||
"name": "db",
|
||||
"configuration": {
|
||||
"template": "{\"jdbc_url\": \"getEnvironmentVariable(\"SNOWFLAKE_JDBC_URL\")\"}",
|
||||
"timeout": null,
|
||||
"value": {}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
BIN
docs/static/img/databases/data-connector-config.png
vendored
Normal file
BIN
docs/static/img/databases/data-connector-config.png
vendored
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 88 KiB |
Loading…
Reference in New Issue
Block a user