View Source Plausible.Imported.CSVImporter (Plausible v0.0.1)
CSV importer from S3 that uses ClickHouse s3 table function.
Summary
Functions
Extracts min/max date range from a list of uploads.
Extracts the table name from the provided filename.
Extracts table name and min/max dates from the filename.
Checks if the provided filename conforms to the expected format.
Functions
@spec date_range([String.t() | %{required(String.t()) => String.t()}, ...]) :: Date.Range.t() | nil
Extracts min/max date range from a list of uploads.
Examples:
iex> date_range([
...> %{"filename" => "imported_devices_20190101_20210101.csv"},
...> "imported_pages_20200101_20220101.csv"
...> ])
Date.range(~D[2019-01-01], ~D[2022-01-01])
iex> date_range([])
nil
Extracts the table name from the provided filename.
Raises if the filename doesn't conform to the expected format.
Examples:
iex> extract_table("my_data.csv")
** (ArgumentError) invalid filename
iex> extract_table("imported_devices_00010101_20250101.csv")
"imported_devices"
@spec new_import(Plausible.Site.t(), Plausible.Auth.User.t(), Keyword.t()) :: {:ok, Oban.Job.t()} | {:error, Ecto.Changeset.t()}
@spec parse_filename!(String.t()) :: {table :: String.t(), start_date :: Date.t(), end_date :: Date.t()}
Extracts table name and min/max dates from the filename.
Examples:
iex> parse_filename!("my_data.csv")
** (ArgumentError) invalid filename
iex> parse_filename!("imported_devices_00010101_20250101.csv")
{"imported_devices", ~D[0001-01-01], ~D[2025-01-01]}
Checks if the provided filename conforms to the expected format.
Examples:
iex> valid_filename?("my_data.csv")
false
iex> valid_filename?("imported_devices_00010101_20250101.csv")
true