* Reapply "Local CSV exports/imports and S3/UI updates (#3989)" (#3995)
This reverts commit aee69e44c8.
* remove unused functions
* eh, that one was actually used
* ugh, they were both used
---------
Co-authored-by: ruslandoga <67764432+ruslandoga@users.noreply.github.com>
* local CSV exports/imports and S3 updates
* credo
* dialyzer
* refactor input columns
* fix ci minio/clickhouse tests
* Update lib/plausible_web/live/csv_export.ex
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* fix date range filter in export_pages_q and process only pageviews
* remove toTimeZone(zero_timestamp) note
* use SiteImport.pending(), SiteImport.importing()
* escape [SiteImport.pending(), SiteImport.importing()]
* use random s3 keys for imports to avoid collisions (sometimes makes the upload get stuck)
* clamp import date ranges
* site is already in assigns
* recompute cutoff date each time
* use toDate(timestamp[, timezone]) shortcut
* show alreats on export cancel/delete and extract hint into a component
* switch to Imported.clamp_dates/4
* reprocess tables when imports are added
* recompute cutoff_date on each call
* actually use clamped_date_range on submit
* add warning message
* add expiry rules to buckets in make minio
* add site_id to imports notifications and use it in csv_importer
* try/catch safer
* return :ok
* date range is not available when no uploads
* improve ui and warning messages
* use Generic.notice
* fix flaky exports test
* begin tests
* Improve `Importer` notification payload shape
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* ui
* fix redirect link
* improve make minio
* use implicit button form for csv export
* add exports_bucket helper
* read S3_EXPORTS_BUCKET
* supply s3_bucket in export_csv job args
* make plausible_minio use unprotected port
* move s3_csv_export queue to base queues
* Update lib/plausible_web/controllers/site_controller.ex
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* add csv importer
* make table validation explicit
* update some docs
* improve docs
* add minio container to ci
* more tests
* eh
* continue
* add passing test
* add failing test
* add config test
* add minio to Makefile
* testcontainers
* remove extra whitespace
* explain the implementation a bit
* account for async deletes in tests
* bounces is UInt32
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Include gelocation DB download in the development workflow
* Make sure `tls_certificate_check` is started ASAP
This prevents `:application_either_not_started_or_not_ready` errors
on application startup.
* Mark Makefile targets as PHONY
By default Make assumes the targets are files,
in this case none of them are.
* chore(docker): improve repeat contributions workflow
* This change adds two new commands to gracefully stop and remove the Postgres and Clickhouse docker containers. To do so, it also gives them a recognizable name.
* Additionally, the Postgres container is updated to use a named volume for its data. This lower friction for repeat contributions where one would otherwise sign up and activate their accounts again and again each time.
* Apple M1 compatibility for local development
* Patch upgrade for Python to a version that is built without errors
* Use a new community image for clickhouse that's ARM64 compatible
* Update contribution docs
* chore(make): convenience commands for installation & dev server
* docs: easier development env instructions
* docs: add note about docker volumes
* docs: detail pre-commit configuration
* style: eslint and prettier changes
* Allow for passing custom classes to fade-in
* style: eslint & prettier for the details button component
* style: react lifecycle methods to come first
* docs: add instructions to disable pre-commit
* style: devices components
* Move render methods to the last (together) in the order list
* Remove unused component import
* React lifecycle to come first before our own methods
* General styling and eslint changes
* Cleaner renderContent method using switch/case (fixes an eslint error as well!)
* Cleaner renderPill method with proper spacing + removing uncessary else
* style: more eslint/prettier for pages components
* Use newer Fragment syntax
* Remove unnecessary else statement
* Use backtick strings for concatenating strings
* Remove unnecessary space
* Remove unused imports and variable declarations
* Bunch render methods together as last in the order list
* fix: details button to drop to the bottom naturally on smaller screens
This _mostly_ fixes one of the issues being tracked on #972, titled "Details button issue on Firefox specifically"
* refactor: reduce usage of our CSS class in favor of tailwind's util classes
* refactor: remove our css classes in favor of Tailwind's util classes
* This change adds two new commands to gracefully stop and remove the Postgres and Clickhouse docker containers. To do so, it also gives them a recognizable name.
* Additionally, the Postgres container is updated to use a named volume for its data. This lower friction for repeat contributions where one would otherwise sign up and activate their accounts again and again each time.
* Update contribution guidelines to include dependency installation
Now mentions the required system dependencies, and the step for installing Elixir deps
Erlang is used by the :gettext dependency, and ecto.create will fail if it is not present
* Make Clickhouse and PostgreSQL containers run in detached mode
This ensures that they run in the background so that they are present during the follow steps
* task: Add CONTRIBUTING.md with development setup instructions
* Add --prefix assets to npm install instructions
Co-authored-by: Dennis Tel <dennis@aiden.cx>
