View Source Plausible.Imported.CSVImporter (Plausible v0.0.1)

CSV importer from either S3 for which it uses ClickHouse s3 table function or from local storage for which it uses input function.

Summary

Functions

Extracts min/max date range from a list of uploads.

Extracts the table name from the provided filename.

Returns local directory for CSV imports storage.

Extracts table name and min/max dates from the filename.

Checks if the provided filename conforms to the expected format.

Functions

@spec date_range([String.t() | %{required(String.t()) => String.t()}, ...]) ::
  Date.Range.t() | nil

Extracts min/max date range from a list of uploads.

Examples:

iex> date_range([
...>   %{"filename" => "imported_devices_20190101_20210101.csv"},
...>   "pages_20200101_20220101.csv"
...> ])
Date.range(~D[2019-01-01], ~D[2022-01-01])

iex> date_range([])
nil
@spec extract_table(String.t()) :: String.t()

Extracts the table name from the provided filename.

Raises if the filename doesn't conform to the expected format.

Examples:

iex> extract_table("my_data.csv")
** (ArgumentError) invalid filename

iex> extract_table("imported_devices_00010101_20250101.csv")
"imported_devices"

iex> extract_table("devices_00010101_20250101.csv")
"imported_devices"

Returns local directory for CSV imports storage.

Builds upon $PERSISTENT_CACHE_DIR (if set) and falls back to /tmp

Examples:

iex> local_dir = local_dir(_site_id = 37)
iex> String.ends_with?(local_dir, "/plausible-imports/37")
true
Link to this function

new_import(site, user, opts)

View Source
@spec new_import(Plausible.Site.t(), Plausible.Auth.User.t(), Keyword.t()) ::
  {:ok, Oban.Job.t()} | {:error, Ecto.Changeset.t() | :import_in_progress}
Link to this function

parse_filename!(filename)

View Source
@spec parse_filename!(String.t()) ::
  {table :: String.t(), start_date :: Date.t(), end_date :: Date.t()}

Extracts table name and min/max dates from the filename.

Examples:

iex> parse_filename!("my_data.csv")
** (ArgumentError) invalid filename

iex> parse_filename!("imported_devices_00010101_20250101.csv")
{"imported_devices", ~D[0001-01-01], ~D[2025-01-01]}

iex> parse_filename!("devices_00010101_20250101.csv")
{"imported_devices", ~D[0001-01-01], ~D[2025-01-01]}
Link to this function

valid_filename?(filename)

View Source
@spec valid_filename?(String.t()) :: boolean()

Checks if the provided filename conforms to the expected format.

Examples:

iex> valid_filename?("my_data.csv")
false

iex> valid_filename?("imported_devices_00010101_20250101.csv")
true

iex> valid_filename?("devices_00010101_20250101.csv")
true