Initial advanced config for data connectors

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10091
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Rob Dominguez <24390149+robertjdominguez@users.noreply.github.com>
GitOrigin-RevId: a74bb822338b5f738884958fd4abaebf8d3a1925

docs/docs/api-reference/kriti-templating.mdx

@@ -45,6 +45,7 @@ At present, Kriti templating is available for:
 - [Event Triggers REST Connectors](/event-triggers/rest-connectors.mdx)
 - [Cron Triggers REST Connectors](/scheduled-triggers/create-cron-trigger.mdx#rest-connectors)
 - [Dynamic Database Connection Routing](databases/database-config/dynamic-db-connection.mdx#connection-template)
+- [Data Connector Config](databases/database-config/data-connector-config.mdx)
 
 ### Example
 

docs/docs/databases/database-config/data-connector-config.mdx

@@ -0,0 +1,228 @@
+---
+description: Hasura Cloud and Hasura Enterprise data connector configuration
+title: 'Cloud and Enterprise Edition: Data connector configuration'
+keywords:
+  - hasura
+  - docs
+  - cloud
+  - enterprise
+  - configuration
+  - data connector
+  - data connector configuration
+  - connections
+sidebar_label: Data Connector Config
+sidebar_position: 4
+---
+
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+import Thumbnail from '@site/src/components/Thumbnail';
+import HeadingIcon from '@site/src/components/HeadingIcon';
+import ProductBadge from '@site/src/components/ProductBadge';
+
+# Data Connector Config
+
+<ProductBadge ce free standard pro ee self />
+
+## Introduction
+
+The configuration of [data connectors](/databases/data-connectors/index.mdx) depends on the specific data connector
+you are connecting to. But, two options are available that are common amongst all data
+connectors. Those two options are:
+
+1. [Timeout](#timeout)
+2. [Template](#template)
+
+:::info This configuration only applies to Hasura Data Connector Agents 
+
+Please note that this only applies to [data connectors](/databases/data-connectors/index.mdx). If you are using a
+native database connection then this configuration does not apply. You can learn more about connecting to a
+native database [here](/databases/overview.mdx#supported-databases).
+
+:::
+
+### Timeout
+
+The timeout setting is related to how long the graphql engine will wait for a response from the
+data connector agent before throwing an error. This value defaults to 30 seconds.
+
+As an example, if it takes more than 30 seconds to introspect the schema of a data connector, then it 
+is likely you may be connecting to a slow or latent data source. In this case, you may want to increase
+the timeout value to allow for the introspection to complete.
+
+Timeout has three available setting options:
+
+1. Seconds `seconds` `{ "seconds": 60 }`
+2. Milliseconds `milliseconds` `{ "milliseconds": 60000 }`
+3. Microseconds `microseconds` `{ "microseconds": 60000000 }`
+
+<Tabs groupId="timeout-setting" className="timeout-settings-tabs">
+<TabItem value="console" label="Console">
+
+:::info Console only supports `seconds` timeout
+The Console only gives you the option to use `seconds`. If you need to use `milliseconds` or `microseconds` you
+will need to use CLI or the API.
+:::
+
+In the Console, navigate to the `Data` tab and select your data connector. Then, under the `Advanced` tab you
+will can set the timeout value.
+
+<Thumbnail
+  src="/img/databases/data-connector-config.png"
+  alt="Data connector advanced config"
+/>
+
+</TabItem>
+<TabItem value="cli" label="CLI">
+
+You can add a _timeout_ for a data connector database by adding their config to the
+`/metadata/databases/database.yaml` file. In the example below, we're using
+Snowflake:
+
+```yaml {5}
+- name: snowflake
+  kind: snowflake
+  configuration:
+    template: null
+    timeout: { "seconds": 120 }
+    value: {
+      "jdbc_url": "jdbc:snowflake://url.snowflakecomputing.com/?user=user&password=password&warehouse=warehouse&db=db&role=role&schema=schema"
+    }
+```
+
+</TabItem>
+<TabItem value="api" label="API">
+
+The _timeout_ via the [{{connector_name}}_add_source](/api-reference/metadata-api/source.mdx) Metadata API. In the example below, we're using
+Snowflake:
+
+```http {11}
+POST /v1/metadata HTTP/1.1
+Content-Type: application/json
+X-Hasura-Role: admin
+
+{
+  "type": "snowflake_add_source",
+  "args": {
+    "name": "db",
+    "configuration": {
+      "template": null,
+      "timeout": { "seconds": 120 },
+      "value": {
+        "jdbc_url": "jdbc:snowflake://url.snowflakecomputing.com/?user=user&password=password&warehouse=warehouse&db=db&role=role&schema=schema"
+      }
+    }
+  }
+}
+```
+
+</TabItem>
+</Tabs>
+
+### Template
+
+You can use the template field to either set connection strings from `ENV` variables or implement more complex logic
+for a connection string based on some other available variables. Currently, the template for data connectors
+has access to:
+
+- [Environment variables](#environment-variables)
+- [Session variables](#session-variables)
+- [Configuration variables](#configuration-variables)
+
+#### Environment variables
+
+Environment variables can be accessed using the variable `$env` or using the function `getEnvironmentVariable`
+in your Kriti templating. A simple example would be to use the environment variable `SNOWFLAKE_JDBC_URL` to set
+the connection string for a Snowflake data connector:
+
+```
+{"jdbc_url": "$env["SNOWFLAKE_JDBC_URL"]"}
+{"jdbc_url": "getEnvironmentVariable("SNOWFLAKE_JDBC_URL")"}
+```
+
+#### Session variables
+
+You can refer to session variables in the template using the variable `$session`. The session variable keys always contain
+`x-hasura-*` as the prefix:
+
+```
+{{if (empty($session?['x-hasura-role'])) && (empty($session?['x-hasura-user-id']))}}
+  {"jdbc_url": "jdbc:snowflake://url.snowflakecomputing.com/?user=getEnvironmentVariable("DEFAULT_USER")&password=getEnvironmentVariable("DEFAULT_PASS")&warehouse=warehouse&db=db&role=role&schema=schema"}
+{{else}}
+  {"jdbc_url": "jdbc:snowflake://url.snowflakecomputing.com/?user={{$session['x-hasura-role']}}&password={{$session['x-hasura-user-id']}}&warehouse=warehouse&db=db&role=role&schema=schema"}
+{{end}}
+```
+
+In this example, if the session variables `x-hasura-role` and `x-hasura-user-id` are both empty, then the graphql engine will use
+the connection string with the `DEFAULT_USER` and the `DEFAULT_PASS` from our environment variables.
+Otherwise, the graphql engine will use the values of `x-hasura-role` and `x-hasura-user-id` to generate the
+connection string dynamically.
+
+#### Configuration variables
+
+Config variables can be accessed using the variable `$config`. This allows you to access any field from the
+data connector's configuration fields. As a simple example you could setup your template to check the value
+of the JDBC url configuration field as environment variable or fallback to using the value as the JDBC url.
+
+```
+{"jdbc_url":{{$env?[$config.jdbc_url] ?? $config.jdbc_url}}}
+```
+
+<Tabs groupId="timeout-setting" className="timeout-settings-tabs">
+<TabItem value="console" label="Console">
+
+In the Console, navigate to the `Data` tab and select your data connector. Then, under the `Advanced` tab you
+will can set the template value.
+
+<Thumbnail
+  src="/img/databases/data-connector-config.png"
+  alt="Data connector advanced config"
+/>
+
+</TabItem>
+<TabItem value="cli" label="CLI">
+
+You can add a _template_ for a data connector database by adding their config to the
+`/metadata/databases/database.yaml` file:
+
+```yaml {5}
+- name: snowflake
+  kind: snowflake
+  configuration:
+    template: |
+      {"jdbc_url": "getEnvironmentVariable("SNOWFLAKE_JDBC_URL")"}
+    timeout: null
+    value: {}
+```
+
+Apply the Metadata by running:
+
+```yaml
+hasura metadata apply
+```
+
+</TabItem>
+<TabItem value="api" label="API">
+
+The _template_ via the [{{:connector_name}}_add_source](/api-reference/metadata-api/source.mdx) Metadata API.
+
+```http {10}
+POST /v1/metadata HTTP/1.1
+Content-Type: application/json
+X-Hasura-Role: admin
+
+{
+  "type": "snowflake_add_source",
+  "args": {
+    "name": "db",
+    "configuration": {
+      "template": "{\"jdbc_url\": \"getEnvironmentVariable(\"SNOWFLAKE_JDBC_URL\")\"}",
+      "timeout": null,
+      "value": {}
+    }
+  }
+}
+```
+
+</TabItem>
+</Tabs>