View Source Plausible.S3 (Plausible v0.0.1)

Helper functions for S3 exports/imports.

Summary

Functions

Chunks and uploads Zip archive to the provided S3 destination.

Returns the pre-configured S3 bucket for CSV exports.

Returns access_key_id and secret_access_key to be used by ClickHouse during imports from S3.

Presigns an upload for an imported file.

Returns the pre-configured S3 bucket for CSV imports.

Functions

Link to this function

export_upload_multipart(stream, s3_bucket, s3_path, filename, config_overrides \\ [])

View Source
@spec export_upload_multipart(
  Enumerable.t(),
  String.t(),
  Path.t(),
  String.t(),
  keyword()
) ::
  :uri_string.uri_string()

Chunks and uploads Zip archive to the provided S3 destination.

Returns a presigned URL to download the exported Zip archive from S3. The URL expires in 24 hours.

In the current implementation the bucket always goes into the path component.

@spec exports_bucket() :: String.t()

Returns the pre-configured S3 bucket for CSV exports.

config :plausible, Plausible.S3,
  exports_bucket: System.fetch_env!("S3_EXPORTS_BUCKET")

Example:

iex> exports_bucket()
"test-exports"
Link to this function

import_clickhouse_credentials()

View Source
@spec import_clickhouse_credentials() :: %{
  access_key_id: String.t(),
  secret_access_key: String.t()
}

Returns access_key_id and secret_access_key to be used by ClickHouse during imports from S3.

Example:

iex> import_clickhouse_credentials()
%{access_key_id: "minioadmin", secret_access_key: "minioadmin"}
Link to this function

import_presign_upload(site_id, filename)

View Source

Presigns an upload for an imported file.

In the current implementation the bucket always goes into the path component.

Example:

iex> %{
...>   s3_url:  "http://localhost:10000/test-imports/123/imported_browsers.csv",
...>   presigned_url: "http://localhost:10000/test-imports/123/imported_browsers.csv?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=minioadmin" <> _
...> } = import_presign_upload(_site_id = 123, _filename = "imported_browsers.csv")
@spec imports_bucket() :: String.t()

Returns the pre-configured S3 bucket for CSV imports.

config :plausible, Plausible.S3,
  imports_bucket: System.fetch_env!("S3_IMPORTS_BUCKET")

Example:

iex> imports_bucket()
"test-imports"