From 8b0ae5fcb09060af78d1709b95128182175fb694 Mon Sep 17 00:00:00 2001 From: Rob Dominguez Date: Thu, 25 Apr 2024 08:33:55 -0500 Subject: [PATCH] Docs: Turn OTEL into a directory [DOCS-1987]: https://hasurahq.atlassian.net/browse/DOCS-1987?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10775 Co-authored-by: Toan Nguyen <1615675+hgiasac@users.noreply.github.com> GitOrigin-RevId: 8f17e6d8b0097965835708d32e217035ff872afd --- .../observability/cloud/azure-monitor.mdx | 4 +- docs/docs/observability/cloud/datadog.mdx | 4 +- docs/docs/observability/cloud/index.mdx | 6 +- docs/docs/observability/cloud/newrelic.mdx | 4 +- docs/docs/observability/cloud/prometheus.mdx | 4 +- .../enterprise-edition/index.mdx | 2 +- .../enterprise-edition/prometheus/index.mdx | 20 +++--- .../observability-best-practices.mdx | 14 ++-- .../opentelemetry/_category_.json | 5 ++ .../opentelemetry/data-connectors.mdx | 68 +++++++++++++++++++ .../graphql-engine.mdx} | 8 +-- .../observability/opentelemetry/index.mdx | 31 +++++++++ 12 files changed, 137 insertions(+), 33 deletions(-) create mode 100644 docs/docs/observability/opentelemetry/_category_.json create mode 100644 docs/docs/observability/opentelemetry/data-connectors.mdx rename docs/docs/observability/{opentelemetry.mdx => opentelemetry/graphql-engine.mdx} (98%) create mode 100644 docs/docs/observability/opentelemetry/index.mdx diff --git a/docs/docs/observability/cloud/azure-monitor.mdx b/docs/docs/observability/cloud/azure-monitor.mdx index a1ad7c1051d..739ff2addfd 100644 --- a/docs/docs/observability/cloud/azure-monitor.mdx +++ b/docs/docs/observability/cloud/azure-monitor.mdx @@ -36,8 +36,8 @@ above. :::tip More metrics on OpenTelemetry exporter -Try the new [OpenTelemetry exporter](/observability/opentelemetry.mdx) to get advanced metrics and traces to connect -with Azure Monitor. This integration will export the metrics as detailed [here](#view-metrics). +Try the new [OpenTelemetry exporter](/observability/opentelemetry/index.mdx) to get advanced metrics and traces to +connect with Azure Monitor. This integration will export the metrics as detailed [here](#view-metrics). ::: diff --git a/docs/docs/observability/cloud/datadog.mdx b/docs/docs/observability/cloud/datadog.mdx index 8c7901cab90..ed3449d2abb 100644 --- a/docs/docs/observability/cloud/datadog.mdx +++ b/docs/docs/observability/cloud/datadog.mdx @@ -34,8 +34,8 @@ For Hasura Cloud projects, the Datadog Integration is only available on the `Sta :::tip More metrics on OpenTelemetry exporter -Try the new [OpenTelemetry exporter](/observability/opentelemetry.mdx) to get advanced metrics and traces to connect -with Datadog. This integration will export the metrics as detailed [here](#view-metrics). +Try the new [OpenTelemetry exporter](/observability/opentelemetry/index.mdx) to get advanced metrics and traces to +connect with Datadog. This integration will export the metrics as detailed [here](#view-metrics). ::: diff --git a/docs/docs/observability/cloud/index.mdx b/docs/docs/observability/cloud/index.mdx index 536d71f8241..b372a4e839a 100644 --- a/docs/docs/observability/cloud/index.mdx +++ b/docs/docs/observability/cloud/index.mdx @@ -37,12 +37,12 @@ choice: - [New Relic](newrelic.mdx) - [Azure monitor](azure-monitor.mdx) - [Prometheus](prometheus.mdx) -- [OpenTelemetry](/observability/opentelemetry.mdx) +- [OpenTelemetry](/observability/opentelemetry/index.mdx) :::tip More metrics on OpenTelemetry exporter -Try the new [OpenTelemetry exporter](/observability/opentelemetry.mdx) to get advanced metrics and traces to connect -with the tool of your choice. This integration will export the metrics as detailed +Try the new [OpenTelemetry exporter](/observability/opentelemetry/index.mdx) to get advanced metrics and traces to +connect with the tool of your choice. This integration will export the metrics as detailed [here](/observability/cloud/prometheus.mdx#metrics-details). ::: diff --git a/docs/docs/observability/cloud/newrelic.mdx b/docs/docs/observability/cloud/newrelic.mdx index 97661f55a97..5b04ce2348d 100644 --- a/docs/docs/observability/cloud/newrelic.mdx +++ b/docs/docs/observability/cloud/newrelic.mdx @@ -35,8 +35,8 @@ For Hasura Cloud projects, the New Relic Integration is only available on the `S :::tip More metrics on OpenTelemetry exporter -Try the new [OpenTelemetry exporter](/observability/opentelemetry.mdx) to get advanced metrics and traces to connect -with New Relic. This integration will export the metrics as detailed [here](#view-metrics). +Try the new [OpenTelemetry exporter](/observability/opentelemetry/index.mdx) to get advanced metrics and traces to +connect with New Relic. This integration will export the metrics as detailed [here](#view-metrics). ::: diff --git a/docs/docs/observability/cloud/prometheus.mdx b/docs/docs/observability/cloud/prometheus.mdx index 2194fb09339..f2b7ee7c703 100644 --- a/docs/docs/observability/cloud/prometheus.mdx +++ b/docs/docs/observability/cloud/prometheus.mdx @@ -36,8 +36,8 @@ above. :::tip More metrics on OpenTelemetry exporter -Try the new [OpenTelemetry exporter](/observability/opentelemetry.mdx) to get advanced metrics and traces to connect -with Prometheus. This integration will export the metrics as detailed +Try the new [OpenTelemetry exporter](/observability/opentelemetry/index.mdx) to get advanced metrics and traces to +connect with Prometheus. This integration will export the metrics as detailed [here](/observability/cloud/prometheus.mdx#metrics-details). ::: diff --git a/docs/docs/observability/enterprise-edition/index.mdx b/docs/docs/observability/enterprise-edition/index.mdx index 6a9b2843ae0..a59c8bfd799 100644 --- a/docs/docs/observability/enterprise-edition/index.mdx +++ b/docs/docs/observability/enterprise-edition/index.mdx @@ -34,4 +34,4 @@ Check out the following guides on how to export telemetry data from Hasura Cloud choice: - [Prometheus](prometheus/index.mdx) -- [OpenTelemetry](/observability/opentelemetry.mdx) +- [OpenTelemetry](/observability/opentelemetry/index.mdx) diff --git a/docs/docs/observability/enterprise-edition/prometheus/index.mdx b/docs/docs/observability/enterprise-edition/prometheus/index.mdx index 9aedaaf240d..ace26ebca59 100644 --- a/docs/docs/observability/enterprise-edition/prometheus/index.mdx +++ b/docs/docs/observability/enterprise-edition/prometheus/index.mdx @@ -23,20 +23,20 @@ import ProductBadge from '@site/src/components/ProductBadge'; :::tip More metrics on OpenTelemetry exporter -Try the new [OpenTelemetry exporter](/observability/opentelemetry.mdx) to get advanced metrics and traces to connect -with Prometheus. This integration will export the metrics as detailed +Try the new [OpenTelemetry exporter](/observability/opentelemetry/index.mdx) to get advanced metrics and traces to +connect with Prometheus. This integration will export the metrics as detailed [here](/observability/enterprise-edition/prometheus/metrics.mdx). ::: ## Overview -In this section, you'll find information on how to integrate [Prometheus](https://prometheus.io/) with Hasura -Enterprise edition: +In this section, you'll find information on how to integrate [Prometheus](https://prometheus.io/) with Hasura Enterprise +edition: -- [Integrate Prometheus with Hasura EE and build a Grafana Dashboard](integrate-prometheus-grafana.mdx): - Configure Prometheus integration with Hasura Enterprise Edition. -- [Pre-built dashboards](pre-built-dashboards.mdx): Learn about pre-built dashboards available - for Hasura Enterprise Edition. -- [Available metrics](metrics.mdx): Learn about metrics available to monitor the health, - performance and reliability of the Hasura GraphQL Engine. +- [Integrate Prometheus with Hasura EE and build a Grafana Dashboard](integrate-prometheus-grafana.mdx): Configure + Prometheus integration with Hasura Enterprise Edition. +- [Pre-built dashboards](pre-built-dashboards.mdx): Learn about pre-built dashboards available for Hasura Enterprise + Edition. +- [Available metrics](metrics.mdx): Learn about metrics available to monitor the health, performance and reliability of + the Hasura GraphQL Engine. diff --git a/docs/docs/observability/observability-best-practices.mdx b/docs/docs/observability/observability-best-practices.mdx index 756ca2af8da..837ca466ca7 100644 --- a/docs/docs/observability/observability-best-practices.mdx +++ b/docs/docs/observability/observability-best-practices.mdx @@ -90,8 +90,8 @@ to be exported to your observability platform using the appropriate log drivers. #### Metrics via Prometheus -You can export metrics of your Hasura Enterprise project to Prometheus easily via enabling the `metrics` API. You can find -more information on this [here](/observability/enterprise-edition/prometheus/integrate-prometheus-grafana.mdx). +You can export metrics of your Hasura Enterprise project to Prometheus easily via enabling the `metrics` API. You can +find more information on this [here](/observability/enterprise-edition/prometheus/integrate-prometheus-grafana.mdx). For security reasons, the metrics endpoint should not be leaked to the internet. Or if unavoidable, the Prometheus secret should be confidential to prevent misuse. @@ -101,13 +101,13 @@ visualize Golden signal metrics that you will love for real-time monitoring. #### Metrics via OpenTelemetry -Hasura Enterprise is open-telemetry compliant and can export metrics to third-party observability -platforms which support OpenTelemetry. Check out [the OpenTelemetry page](/observability/opentelemetry.mdx) for more information. +Hasura Enterprise is open-telemetry compliant and can export metrics to third-party observability platforms which +support OpenTelemetry. Check out [the OpenTelemetry page](/observability/opentelemetry/index.mdx) for more information. #### Distributed traces Hasura Enterprise also can export distributed traces via OpenTelemetry. Read more at -[here](/observability/opentelemetry.mdx) for more information. +[here](/observability/opentelemetry/index.mdx) for more information. ## Database observability @@ -130,8 +130,8 @@ be implemented: [Query Tags](/observability/query-tags.mdx) are SQL comments that consist of `key=value` pairs that are appended to generated SQL statements. When you issue a query or mutation with query tags, the generated SQL has some extra -information. Database analytics tools can use that information (metadata) to analyze DB load and track -or monitor performance. +information. Database analytics tools can use that information (metadata) to analyze DB load and track or monitor +performance. ### Using Query Tags and **pganalyze** diff --git a/docs/docs/observability/opentelemetry/_category_.json b/docs/docs/observability/opentelemetry/_category_.json new file mode 100644 index 00000000000..d0d712e2bda --- /dev/null +++ b/docs/docs/observability/opentelemetry/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "OpenTelemetry", + "position": 6, + "className": "cloud-and-enterprise-icon" +} diff --git a/docs/docs/observability/opentelemetry/data-connectors.mdx b/docs/docs/observability/opentelemetry/data-connectors.mdx new file mode 100644 index 00000000000..059e35bae07 --- /dev/null +++ b/docs/docs/observability/opentelemetry/data-connectors.mdx @@ -0,0 +1,68 @@ +--- +sidebar_label: Data Connectors +description: Traces via OpenTelemetry for Hasura GraphQL Engine Data Connectors in Enterprise Edition +title: 'Data Connector Traces via OpenTelemetry' +sidebar_class_name: beta-icon +keywords: + - hasura + - docs + - enterprise + - opentelemetry + - open telemetry + - traces +sidebar_position: 3 +--- + +import Tabs from '@theme/Tabs'; +import TabItem from '@theme/TabItem'; +import Thumbnail from '@site/src/components/Thumbnail'; +import ProductBadge from '@site/src/components/ProductBadge'; + +# Export Traces to OpenTelemetry Compliant Receiver from Hasura Data Connectors + +
+ +
Beta
+
+ +## Introduction + +Hasura Data Connectors can export and link internal OpenTelemetry data with GraphQL Engine to visualize the full +timeline of a GraphQL request. + +:::info Note + +At present, data connectors only support traces via OpenTelemetry. + +::: + +## Configuration + +You can easily configure OpenTelemetry for your data connectors using environment variables. + +### GraphQL Data Connector + +The GraphQL Data Connector — which power sources like Athena, MariaDB, MySQL, Oracle, Redshift, and Snowflake — is built +on top of [Quarkus](https://quarkus.io/) with an OpenTelemetry extension. Therefore, the configuration properties of the +connector also follow that of the extension: + +| Environment Variable | Type | Default | Description | +| -------------------------------------- | ------------------------------ | ------- | ----------------------------------------------------------------------------------------------- | +| `QUARKUS_OTEL_EXPORTER_OTLP_ENDPOINT ` | string | | The endpoint of the OpenTelemetry receiver | +| `QUARKUS_OTEL_EXPORTER_OTLP_PROTOCOL ` | enum (`grpc`, `http/protobuf`) | `grpc` | The protocol used to exchange data between the client and the server | +| `QUARKUS_OTEL_EXPORTER_OTLP_HEADERS ` | string | | Key-value pairs to be used as headers associated with requests. i.e.: `key1=value1,key2=value2` | +| `QUARKUS_OTEL_RESOURCE_ATTRIBUTES ` | string | | Specify resource attributes in the following format: `key1=val1,key2=val2` | + +See the complete list of configuration reference +[here](https://quarkus.io/version/main/guides/opentelemetry#configuration-reference). + +### Mongo Data Connector + +The Mongo Data Connector powers MongoDB connections and, thus, has a different configuration: + +| Environment Variable | Type | Default | Description | +| ---------------------------------------- | --------------------------------------------------------- | --------------------- | -------------------------------------------------------------------- | +| `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT ` | string | http://localhost:4318 | The endpoint of the OpenTelemetry receiver | +| `OTEL_EXPORTER_OTLP_TRACES_PROTOCOL ` | enum (`grpc`, `http/protobuf`) | `http/protobuf` | The protocol used to exchange data between the client and the server | +| `OTEL_SERVICE_NAME ` | string | | Name of the service | +| `OTEL_PROPAGATORS ` | enum (`b3`, `b3multi`, `tracecontext`, `baggage`, `none`) | `tracecontext` | Comma-separate list of propagator schemes to use | diff --git a/docs/docs/observability/opentelemetry.mdx b/docs/docs/observability/opentelemetry/graphql-engine.mdx similarity index 98% rename from docs/docs/observability/opentelemetry.mdx rename to docs/docs/observability/opentelemetry/graphql-engine.mdx index a9002623747..b89440fb155 100644 --- a/docs/docs/observability/opentelemetry.mdx +++ b/docs/docs/observability/opentelemetry/graphql-engine.mdx @@ -1,7 +1,7 @@ --- -sidebar_label: OpenTelemetry -description: Traces via OpenTelemetry for Hasura Enterprise Edition -title: 'Traces via OpenTelemetry' +sidebar_label: GraphQL Engine +description: Traces via OpenTelemetry for Hasura GraphQL Engine Enterprise Edition +title: 'GraphQL Engine Traces via OpenTelemetry' sidebar_class_name: beta-icon keywords: - hasura @@ -10,7 +10,7 @@ keywords: - opentelemetry - open telemetry - traces -sidebar_position: 6 +sidebar_position: 2 --- import Tabs from '@theme/Tabs'; diff --git a/docs/docs/observability/opentelemetry/index.mdx b/docs/docs/observability/opentelemetry/index.mdx new file mode 100644 index 00000000000..d5429757a57 --- /dev/null +++ b/docs/docs/observability/opentelemetry/index.mdx @@ -0,0 +1,31 @@ +--- +sidebar_label: OpenTelemetry +description: Traces via OpenTelemetry for Hasura GraphQL Engine and Data Connectors for Enterprise Edition +title: 'GraphQL Engine and Data Connector Traces via OpenTelemetry' +sidebar_class_name: beta-icon +keywords: + - hasura + - docs + - enterprise + - opentelemetry + - open telemetry + - traces +sidebar_position: 1 +--- + +# OpenTelemetry for Observability + +## Introduction + +OpenTelemetry provides a robust framework for gathering traces and metrics, allowing you to optimize performance and +monitor your applications effectively. In this section, we will explore how to configure and utilize OpenTelemetry to +export crucial telemetry data for both the Hasura GraphQL Engine and its associated data connectors, ensuring you have +comprehensive insights to maintain and improve your system's operations. + +Whether you are looking to fine-tune your deployments or troubleshoot issues, our guide will help you leverage +OpenTelemetry's capabilities to meet your observability needs. + +## Options + +- [Export OTEL information for Hasura GraphQL Engine](/observability/opentelemetry/graphql-engine.mdx) +- [Export OTEL information for data connectors](/observability/opentelemetry/data-connectors.mdx)