graphql-engine/rfcs/pg-client-certs.md

This document outlines the current implementation of postgres client certs on Main and the changes we plan on making to it.

Cert Storage

Client Cert data is currently stored in Vault with the Tenant Config:

{
  "adminSecret": "my_secret",
  "certificates": {
    "test_db": {
      "backend": "postgres",
      "client-key-password": "my_password", (OPTIONAL)
      "client-cert": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURpekNDQW5PZ0F3SU...",
      "client-key": "LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcFFJQkFBS0NB...",
      "server-ca": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURmekNDQW1lZ0F3SU..."
    }
  }
  ...
}

Cert sets are key’d by the connection’s Source name. This name can be changed via the console and makes this implementation rather brittle. We would have to ensure that Vault is updated any time the Source name is changed in Console.

Cert Injection

This forever loop is known as the multitenant lifecycle loop. It handles the setup and teardown of tenant instances. We poll for config data from the Controller and pass the old and new configs into the applyConfig subroutine where we align on the configs to determine if we are creating, updating, or deleting a tenant.

When creating or updating a tenant, we fetch the tenant config and the client certs from Vault. We immediately write the certs to disk and then hold onto the filepaths for injecting into the Graphql Engine instance.

Per Tenant, cert paths are stored as a Map Text (Map Text Text). The outer map is from source name to cert map, and the inner map is cert-type (server-ca, client-key, client-cert) to cert-filepath.

After writing the cert data to disk and generating the cert path map, we spawn the pro server instance with the cert path map in the ProServeCtx. The construction of the final connection strings occurs in getSourceResolver.

First we check for cert map data for the given postgres source. Then if there is no cert data for the source we return the base url from the PostgresSourceConnInfo. Otherwise we attempt to parse the URI and concat on the cert query params. If the parse fails then there was either a bad connection string or it used the key-val format. In either case we log the parse failure and return the unmodified connection string. The actual parsing and concatenation happens in addCertsToURI. We use Network.URI.parseURI to do the parsing.

Cert Cleanup

We delete certificate data from disk in two cases:

1. In applyConfig we delete all certificate data from disk when a Tenant is taken down.
2. When updating a Tenant we also delete all certificate data before fetching new data from Vault. It would be preferable to just erase the delta.

Proposed new behavior

https://github.com/hasura/graphql-engine-mono/issues/1143

We want to use Env Vars to correlate certificate data to postgres sources rather than using Source Name. We want to allow users to add sources with client certs via the metadata api:

{
  "type": "pg_add_source",
  "args": {
    "name": "pg1",
    "configuration": {
      "connection_info": {
        "database_url": {
           "from_env": "<DB_URL_ENV_VAR>"
         },
        "ssl_configuration": {
          "sslmode": "verify-ca",
          "sslrootcert": {
            "from_env": "<SSL_ROOT_CERT_ENV_VAR>"
          },
          "sslcert": {
            "from_env": "<SSL_CERT_ENV_VAR>"
          },
          "sslkey": {
            "from_env": "<SSL_KEY_ENV_VAR>"
          },
          "sslpassword": {
            "from_env": "<SSL_PASSWORD_ENV_VAR>"
          }
        }
      }
    }
  }
}

Connection Strings with client certificates will be constructed by the Metadata API using URL Templating. Eg.,

postgres://postgres:password@localhost?sslmode={{SSL_MODE}}&sslkey={{CLIENT_KEY}&sslcert={{CLIENT_CERT}}&sslrootcert={{SERVER_CA}}

Implementation plan for new behavior

We don’t have access to metadata until we launch the Engine instance. So we will defer writing cert files to disk until we are generating the connection pool.

In getSourceResolver we will fetch the metadata and lookup all cert env vars and use a hash of the cert data to check for the file on disk. Cert files will be stored under a tenant specific file path such as ~/.postgres/<tenant-id>/<cert-hash>. Any certs not present will be written to disk at this point. We can then construct a Map EnvVar FilePath to use with renderURLTemplate to perform the connection string interpolation.

We will handle cleanup in the multitenant lifecycle loop by deleting the tenant cert folder.