diff --git a/docs/docs/api-reference/kriti-templating.mdx b/docs/docs/api-reference/kriti-templating.mdx index b3410a4f7aa..ada7e4308d7 100644 --- a/docs/docs/api-reference/kriti-templating.mdx +++ b/docs/docs/api-reference/kriti-templating.mdx @@ -45,6 +45,7 @@ At present, Kriti templating is available for: - [Event Triggers REST Connectors](/event-triggers/rest-connectors.mdx) - [Cron Triggers REST Connectors](/scheduled-triggers/create-cron-trigger.mdx#rest-connectors) - [Dynamic Database Connection Routing](databases/database-config/dynamic-db-connection.mdx#connection-template) +- [Data Connector Config](databases/database-config/data-connector-config.mdx) ### Example diff --git a/docs/docs/databases/database-config/data-connector-config.mdx b/docs/docs/databases/database-config/data-connector-config.mdx new file mode 100644 index 00000000000..79cdabde4ba --- /dev/null +++ b/docs/docs/databases/database-config/data-connector-config.mdx @@ -0,0 +1,228 @@ +--- +description: Hasura Cloud and Hasura Enterprise data connector configuration +title: 'Cloud and Enterprise Edition: Data connector configuration' +keywords: + - hasura + - docs + - cloud + - enterprise + - configuration + - data connector + - data connector configuration + - connections +sidebar_label: Data Connector Config +sidebar_position: 4 +--- + +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; +import Thumbnail from '@site/src/components/Thumbnail'; +import HeadingIcon from '@site/src/components/HeadingIcon'; +import ProductBadge from '@site/src/components/ProductBadge'; + +# Data Connector Config + + + +## Introduction + +The configuration of [data connectors](/databases/data-connectors/index.mdx) depends on the specific data connector +you are connecting to. But, two options are available that are common amongst all data +connectors. Those two options are: + +1. [Timeout](#timeout) +2. [Template](#template) + +:::info This configuration only applies to Hasura Data Connector Agents + +Please note that this only applies to [data connectors](/databases/data-connectors/index.mdx). If you are using a +native database connection then this configuration does not apply. You can learn more about connecting to a +native database [here](/databases/overview.mdx#supported-databases). + +::: + +### Timeout + +The timeout setting is related to how long the graphql engine will wait for a response from the +data connector agent before throwing an error. This value defaults to 30 seconds. + +As an example, if it takes more than 30 seconds to introspect the schema of a data connector, then it +is likely you may be connecting to a slow or latent data source. In this case, you may want to increase +the timeout value to allow for the introspection to complete. + +Timeout has three available setting options: + +1. Seconds `seconds` `{ "seconds": 60 }` +2. Milliseconds `milliseconds` `{ "milliseconds": 60000 }` +3. Microseconds `microseconds` `{ "microseconds": 60000000 }` + + + + +:::info Console only supports `seconds` timeout +The Console only gives you the option to use `seconds`. If you need to use `milliseconds` or `microseconds` you +will need to use CLI or the API. +::: + +In the Console, navigate to the `Data` tab and select your data connector. Then, under the `Advanced` tab you +will can set the timeout value. + + + + + + +You can add a _timeout_ for a data connector database by adding their config to the +`/metadata/databases/database.yaml` file. In the example below, we're using +Snowflake: + +```yaml {5} +- name: snowflake + kind: snowflake + configuration: + template: null + timeout: { "seconds": 120 } + value: { + "jdbc_url": "jdbc:snowflake://url.snowflakecomputing.com/?user=user&password=password&warehouse=warehouse&db=db&role=role&schema=schema" + } +``` + + + + +The _timeout_ via the [{{connector_name}}_add_source](/api-reference/metadata-api/source.mdx) Metadata API. In the example below, we're using +Snowflake: + +```http {11} +POST /v1/metadata HTTP/1.1 +Content-Type: application/json +X-Hasura-Role: admin + +{ + "type": "snowflake_add_source", + "args": { + "name": "db", + "configuration": { + "template": null, + "timeout": { "seconds": 120 }, + "value": { + "jdbc_url": "jdbc:snowflake://url.snowflakecomputing.com/?user=user&password=password&warehouse=warehouse&db=db&role=role&schema=schema" + } + } + } +} +``` + + + + +### Template + +You can use the template field to either set connection strings from `ENV` variables or implement more complex logic +for a connection string based on some other available variables. Currently, the template for data connectors +has access to: + +- [Environment variables](#environment-variables) +- [Session variables](#session-variables) +- [Configuration variables](#configuration-variables) + +#### Environment variables + +Environment variables can be accessed using the variable `$env` or using the function `getEnvironmentVariable` +in your Kriti templating. A simple example would be to use the environment variable `SNOWFLAKE_JDBC_URL` to set +the connection string for a Snowflake data connector: + +``` +{"jdbc_url": "$env["SNOWFLAKE_JDBC_URL"]"} +{"jdbc_url": "getEnvironmentVariable("SNOWFLAKE_JDBC_URL")"} +``` + +#### Session variables + +You can refer to session variables in the template using the variable `$session`. The session variable keys always contain +`x-hasura-*` as the prefix: + +``` +{{if (empty($session?['x-hasura-role'])) && (empty($session?['x-hasura-user-id']))}} + {"jdbc_url": "jdbc:snowflake://url.snowflakecomputing.com/?user=getEnvironmentVariable("DEFAULT_USER")&password=getEnvironmentVariable("DEFAULT_PASS")&warehouse=warehouse&db=db&role=role&schema=schema"} +{{else}} + {"jdbc_url": "jdbc:snowflake://url.snowflakecomputing.com/?user={{$session['x-hasura-role']}}&password={{$session['x-hasura-user-id']}}&warehouse=warehouse&db=db&role=role&schema=schema"} +{{end}} +``` + +In this example, if the session variables `x-hasura-role` and `x-hasura-user-id` are both empty, then the graphql engine will use +the connection string with the `DEFAULT_USER` and the `DEFAULT_PASS` from our environment variables. +Otherwise, the graphql engine will use the values of `x-hasura-role` and `x-hasura-user-id` to generate the +connection string dynamically. + +#### Configuration variables + +Config variables can be accessed using the variable `$config`. This allows you to access any field from the +data connector's configuration fields. As a simple example you could setup your template to check the value +of the JDBC url configuration field as environment variable or fallback to using the value as the JDBC url. + +``` +{"jdbc_url":{{$env?[$config.jdbc_url] ?? $config.jdbc_url}}} +``` + + + + +In the Console, navigate to the `Data` tab and select your data connector. Then, under the `Advanced` tab you +will can set the template value. + + + + + + +You can add a _template_ for a data connector database by adding their config to the +`/metadata/databases/database.yaml` file: + +```yaml {5} +- name: snowflake + kind: snowflake + configuration: + template: | + {"jdbc_url": "getEnvironmentVariable("SNOWFLAKE_JDBC_URL")"} + timeout: null + value: {} +``` + +Apply the Metadata by running: + +```yaml +hasura metadata apply +``` + + + + +The _template_ via the [{{:connector_name}}_add_source](/api-reference/metadata-api/source.mdx) Metadata API. + +```http {10} +POST /v1/metadata HTTP/1.1 +Content-Type: application/json +X-Hasura-Role: admin + +{ + "type": "snowflake_add_source", + "args": { + "name": "db", + "configuration": { + "template": "{\"jdbc_url\": \"getEnvironmentVariable(\"SNOWFLAKE_JDBC_URL\")\"}", + "timeout": null, + "value": {} + } + } +} +``` + + + diff --git a/docs/static/img/databases/data-connector-config.png b/docs/static/img/databases/data-connector-config.png new file mode 100644 index 00000000000..ae28254e078 Binary files /dev/null and b/docs/static/img/databases/data-connector-config.png differ