Commit Graph

156 Commits

Author SHA1 Message Date
ruslandoga
8ee4827fea
Run migrations in order across repos (#4466)
* order migrations

* migrate in 'streaks'

* cleanup

* docs

* tests

* add comment

* changelog

* continue

* run migrations tests in ci

* add pending_streams cmd

* fix ci

* add docs

* fix test

* simplify test

* only test v2+

* fix test

* return async: true

* cleanup

* add 'no pending migration'

* drop migrate/0 and pending_migrations/0

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-09-09 11:27:11 +02:00
ruslandoga
b6859352cb
Fix tzdata (#4539)
* fix tzdata 2024b

* 2024b is buggy, use 2024a instead

* fix comma
2024-09-06 10:06:38 +02:00
Cenk Kücük
c1b217dc54
use GH runner instead of buildjet to overcome docker hub rate limits (#4470) 2024-08-27 11:06:12 +02:00
Artur Pata
83b7a7ec53 Update Typescript config and add typecheck to CI 2024-08-21 12:40:22 +03:00
ruslandoga
0328453e40
use ERL_FLAGS=+JMsingle true in ghcr.io images (#4450)
Co-authored-by: Cenk Kücük <cenk@plausible.io>
2024-08-19 13:30:55 +02:00
Cenk Kücük
3b485afc91
escape newlines in commit message (#4449)
* test description
2024-08-17 01:15:06 +02:00
Cenk Kücük
7f175a86e3
do not push private images to dockerhub anymore (#4447) 2024-08-17 01:08:54 +02:00
Cenk Kücük
1e31632a8a
Try-emojis (#4448)
* Try emojis for notification

* update msg
2024-08-17 01:00:19 +02:00
Cenk Kücük
4b6d8b6859
Send notification to chat when docker build was successful (#4446) 2024-08-17 00:46:30 +02:00
Artur Pata
8af3f7332a
Add testing framework (#4440)
* Add testing framework

* Test query period picking behaviour
2024-08-15 12:27:22 +03:00
Cenk Kücük
9c1e198d05
validate that all checks passed (#4438) 2024-08-13 15:56:51 +02:00
ruslandoga
13a95c879a
bump cache version (#4339) 2024-07-11 11:37:57 +03:00
Cenk Kücük
5c146ad30e
Update build-private-images-ghcr.yml (#4280) 2024-06-26 16:52:37 +02:00
Cenk Kücük
6f2e1242af
Add preview env URL to PRs (#4273) 2024-06-25 01:14:23 +02:00
dependabot[bot]
c09789b36f
Bump docker/build-push-action from 5 to 6 (#4264) 2024-06-24 10:41:56 +02:00
Cenk Kücük
62cd18e5e7
Add additional docker image tag (#4197)
* Update build-private-images-ghcr.yml
2024-06-06 16:49:08 +02:00
dependabot[bot]
a0ffda06f4
Bump actions/checkout from 2 to 4 (#4173)
Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-06 12:37:33 +02:00
hq1
0381bc6121
Use codespell (#4145)
* Fix typos

* Add .codespellignore

* Add codespell GH action
2024-05-27 10:52:01 +02:00
Cenk Kücük
39cf8c4179
Upgrade Clickhouse version in CI (#4062)
* Update elixir.yml

* Fix error

See https://github.com/ClickHouse/ClickHouse/issues/62431#issuecomment-2090283419

* fix order to pass test

* Revert "fix order to pass test"

This reverts commit d79fbeb297.

* Update Makefile

Update CH version
2024-05-13 21:53:29 +02:00
hq1
d6824de1ad
Rename internal build symbols (#3942)
* Rename internal build symbols

* Rename remaining + add `on_ce` macro

cc @ruslandoga
2024-04-29 08:05:33 +02:00
dependabot[bot]
805b5acbc7
Bump peaceiris/actions-gh-pages from 3 to 4 (#4007)
Bumps [peaceiris/actions-gh-pages](https://github.com/peaceiris/actions-gh-pages) from 3 to 4.
- [Release notes](https://github.com/peaceiris/actions-gh-pages/releases)
- [Changelog](https://github.com/peaceiris/actions-gh-pages/blob/main/CHANGELOG.md)
- [Commits](https://github.com/peaceiris/actions-gh-pages/compare/v3...v4)

---
updated-dependencies:
- dependency-name: peaceiris/actions-gh-pages
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-24 00:23:48 +02:00
Adrian Gruntkowski
c1c03b729c
Reapply "Local CSV exports/imports and S3/UI updates (#3989)" (#3995) (#3996)
* Reapply "Local CSV exports/imports and S3/UI updates (#3989)" (#3995)

This reverts commit aee69e44c8.

* remove unused functions

* eh, that one was actually used

* ugh, they were both used

---------

Co-authored-by: ruslandoga <67764432+ruslandoga@users.noreply.github.com>
2024-04-11 09:15:01 +02:00
Adrian Gruntkowski
aee69e44c8
Revert "Local CSV exports/imports and S3/UI updates (#3989)" (#3995)
This reverts commit 1a0cb52f95.
2024-04-09 21:26:23 +02:00
ruslandoga
1a0cb52f95
Local CSV exports/imports and S3/UI updates (#3989)
* local CSV exports/imports and S3 updates

* credo

* dialyzer

* refactor input columns

* fix ci minio/clickhouse tests

* Update lib/plausible_web/live/csv_export.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* fix date range filter in export_pages_q and process only pageviews

* remove toTimeZone(zero_timestamp) note

* use SiteImport.pending(), SiteImport.importing()

* escape [SiteImport.pending(), SiteImport.importing()]

* use random s3 keys for imports to avoid collisions (sometimes makes the upload get stuck)

* clamp import date ranges

* site is already in assigns

* recompute cutoff date each time

* use toDate(timestamp[, timezone]) shortcut

* show alreats on export cancel/delete and extract hint into a component

* switch to Imported.clamp_dates/4

* reprocess tables when imports are added

* recompute cutoff_date on each call

* actually use clamped_date_range on submit

* add warning message

* add expiry rules to buckets in make minio

* add site_id to imports notifications and use it in csv_importer

* try/catch safer

* return :ok

* date range is not available when no uploads

* improve ui and warning messages

* use Generic.notice

* fix flaky exports test

* begin tests

* Improve `Importer` notification payload shape

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-04-09 20:59:48 +02:00
Karl-Aksel Puulmann
3115c6e7a8
Reducing JOINs in queries (#3966)
* Move experimental_session_count? logic to within query object

* WIP new querying system for deciding what tables to query

* both -> either

* Include sample_percent in both tables

* Remove a hanging TODO

* Allow filtering by visit props on event queries if flag is on

* Make default sessions join more conditional

* Simplify events_join_sessions?

* Add some TODOs

* Fix assignment

* Handle entry/exit page visit props separately from props stored in events table

* Update test which created sessions/events differently from everyone else

* Make query_events private

* Dont filter by session properties on events table if querying sessions and joining in events

* Handle visits, pageviews, events and visitors metrics from other table

* both -> either

* events, pageviews are strictly event metrics

* Add support for (plain) breakdowns deciding which table to use

* Run tests with experimental_reduced_joins as a separate job

Also refactor which tests are run with postgres:15 to reduce number of jobs

* moduledocs for TableDecider

* Fix matrix

* Custom build name

* Move TEST_EXPERIMENTAL_REDUCED_JOINS check

* Handle percentage separately from other metrics

* Remove debug code

* TableDecider tests

* both => sample_percent

* Improve naming

* Simplify code

* Breakdowns retain old behavior if getting metric visitors

* Unify behavior of entry/exit page hostnames with rest

* Fix test naming
2024-04-04 13:54:23 +03:00
Karl-Aksel Puulmann
604bf88451
Add github action to validate whether migrations and app change at the same time (#3945) 2024-03-26 10:29:55 +02:00
ruslandoga
4ae00540dc
Split Elixir CI jobs (#3898)
* split elixir tests

* more understandable matrix

* add unused deps in mix.lock check

* get all deps for static checks
2024-03-18 09:17:04 +01:00
ruslandoga
83a46fb47e
Dockerfile and co updates (#3811)
* wip

* more env, setup user before packages to avoid them picking our uid

* make entrypoint.sh executable by all again

* apparently it's a best practice to have executables be owned by root

* make entrypoint executable in COPY

* stop writing to /app, write to /tmp if needed

* fewer changes

* system user

* keep same style for multiline commands

* fewer changes

* add changelog entry

* fix group assignment for plausible user

* use gid=999

* no home

* no home

* add gecos

* add plausible user to nogroup instead of creating a custom one

* eh

* fewer changes

* fewer changes

* fewer changes

* use PERSISTENT_CACHE_DIR instead of STORAGE_DIR

* ignore more

* cleanup

* remove hex timeout env var

* use ERL_FLAGS=+JMsingle true in public builds

* fallback to /tmp and nest under /tzdata_data/ for tzdata

---------

Co-authored-by: Cenk Kücük <cenk@plausible.io>
2024-03-08 14:33:55 +00:00
Cenk Kücük
fecd665c19
Push private images to GHCR (#3859) 2024-03-04 13:20:18 +00:00
ruslandoga
f3423aefec
Add csv importer (#3795)
* add csv importer

* make table validation explicit

* update some docs

* improve docs

* add minio container to ci

* more tests

* eh

* continue

* add passing test

* add failing test

* add config test

* add minio to Makefile

* testcontainers

* remove extra whitespace

* explain the implementation a bit

* account for async deletes in tests

* bounces is UInt32

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-02-27 14:19:09 +01:00
Cenk Kücük
cc867396b2
Update elixir.yml (#3831)
Only test against the last 2 major versions
2024-02-26 14:10:57 +00:00
ruslandoga
bea8ceeffb
use newer postgres versions in the ci (#3821) 2024-02-26 11:36:11 +00:00
Cenk Kücük
bdaf80c05d
Publish community edition under plausible/community-edition on GHCR and dockerhub (#3815) 2024-02-23 11:17:45 +00:00
Adrian Gruntkowski
e96a52c588
Revert "push docker images to ghcr.io (#3787)" (#3790)
This reverts commit c430274d7e.
2024-02-15 20:54:28 +01:00
Cenk Kücük
c430274d7e
push docker images to ghcr.io (#3787) 2024-02-15 12:42:25 +00:00
dependabot[bot]
c8b25347ed
Bump buildjet/cache from 3 to 4 (#3753)
Bumps [buildjet/cache](https://github.com/buildjet/cache) from 3 to 4.
- [Changelog](https://github.com/BuildJet/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/buildjet/cache/compare/v3...v4)

---
updated-dependencies:
- dependency-name: buildjet/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Cenk Kücük <cenk@plausible.io>
2024-02-07 16:23:54 +00:00
dependabot[bot]
3c9062a1dd
Bump actions/cache from 3 to 4 (#3734)
Bumps [actions/cache](https://github.com/actions/cache) from 3 to 4.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Cenk Kücük <cenk@plausible.io>
2024-01-29 14:07:47 +00:00
Adam Rutkowski
00ef386356 Revert "Bump actions/cache from 3 to 4 (#3707)"
This reverts commit 100e89370f.
2024-01-27 10:32:48 +01:00
dependabot[bot]
100e89370f
Bump actions/cache from 3 to 4 (#3707)
Bumps [actions/cache](https://github.com/actions/cache) from 3 to 4.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Cenk Kücük <cenk@plausible.io>
2024-01-26 12:45:11 +00:00
Karl-Aksel Puulmann
e2eb861799
Github Actions: Run tests on all PRs (#3709)
Chained PRs (like https://github.com/plausible/analytics/pull/3708)
don't currently run tests. This PR fixes that.
2024-01-23 09:32:20 +02:00
hq1
32456d4348
Upgrade stack (deps, elixir 1.16, OTP 26.2.1) (#3678)
* Bump asdf erlang & elixir

* Bump erlang/elixir in the dockerfile

* Remove Oban.Stager config as per https://github.com/sorentwo/oban/blob/main/guides/upgrading/v2.14.md

* Configure Oban for tests as per https://github.com/sorentwo/oban/blob/main/guides/upgrading/v2.14.md

* Mark sampling hint clause with unsafe fragment

Any other/recommended way to do that? cc @ruslandoga

* Address String.slice/2 deprecation

* Update deps

* Address 0.0 matching warning

* Make funnel settings work

although this is probably not the best way to do it.
Needs revisiting, not sure what broke it - could not find
any breaking changes in related dependencies.

* Migrate oban as per https://github.com/sorentwo/oban/blob/main/guides/upgrading/v2.17.md

* Update credo

* Bump CI cache

* Use `Bypass.pass` to avoid exit shutdown message
2024-01-11 08:48:04 +01:00
dependabot[bot]
a769447dcc
Bump actions/github-script from 6 to 7 (#3534)
Bumps [actions/github-script](https://github.com/actions/github-script) from 6 to 7.
- [Release notes](https://github.com/actions/github-script/releases)
- [Commits](https://github.com/actions/github-script/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/github-script
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-03 15:30:41 +00:00
hq1
3905e27d40
Pin https://github.com/docker/metadata-action/ (#3575)
Ref: https://github.com/docker/metadata-action/pull/352#issuecomment-1833733738
2023-11-30 13:56:40 +01:00
hq1
bd7deb5631
Update dockerfile for build options (#3567)
* Update dockerfile for build options

* Start permanent on small

* Default to small

* Default to small fixup
2023-11-30 12:39:34 +01:00
hq1
b9ec38038c
Add small build option (#3536)
* Update applications

* Clone community config

* Move modules to experimental dir

* Update runtime config

* Apply first set of compile-time conditionals

* Move funnel schemas to experimental

* Make funnel schema-less build compile

* Use experimental/lib for elixir code

* Move JS funnels to experimental

* Clean up conditional rendering

* Tidy up the pipeline

* Make two builds pass tests without warnings

* Reuse existing dotenvs

* Do a bunch of renames

* Clean up naming

* Run secondary CI

* Update router

* Remove RewriteFunnelDupes migration

Tests were disabled already and it was a one-off shot

* Fixup quota mixins

* Add moduledoc

* Change MIX_ENV for seconary test run

* Skip crm on small

* !fixup

* Exclude flags pipeline

* Update lib/plausible_web/controllers/stats_controller.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2023-11-20 12:52:20 +01:00
dependabot[bot]
dc5ab2e407
Bump hashicorp/setup-terraform from 2 to 3 (#3489)
Bumps [hashicorp/setup-terraform](https://github.com/hashicorp/setup-terraform) from 2 to 3.
- [Release notes](https://github.com/hashicorp/setup-terraform/releases)
- [Changelog](https://github.com/hashicorp/setup-terraform/blob/main/CHANGELOG.md)
- [Commits](https://github.com/hashicorp/setup-terraform/compare/v2...v3)

---
updated-dependencies:
- dependency-name: hashicorp/setup-terraform
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-11-16 01:07:21 +01:00
Uku Taht
cfe81d6d3f
Simplify asset building (#3459)
* Add tailwind and esbuild

* Remove unused images

* Move unprocessed assets to priv directory

* Fix applyTheme script

* Remove autoprefixer

* Update bundlemon

* Remove babel config

* Revert "Remove autoprefixer"

This reverts commit fc60c31c73.

* Make dashboard react file work

* Fix app.css imports

* Remove autoprefixer

* Add back in robots.txt

* Go back to css/ and js/ folders as opposed to assets/

* Bundle embed.host.js and embed.content.js

* Add components folder to live reload paths

* Remove bundlemon

* Use mix assets task in Dockerfil

* Add assets setup to CONTRIBUTING.md
2023-11-01 16:27:29 +02:00
dependabot[bot]
b298e0e1b5
Bump actions/setup-node from 3 to 4 (#3468)
Bumps [actions/setup-node](https://github.com/actions/setup-node) from 3 to 4.
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](https://github.com/actions/setup-node/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-31 14:58:18 -03:00
hq1
117eef000d
Upgrade Erlang/Elixir stack (#3454)
* Bump deps

* Bump stack

* Fix deprecation warnings

* Fix VCR cassettes mismatch due to OTP-18414

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Format & fix flaky tests

* Handle raw IPv4 hostnames; test public suffix TLD

* Configure locus db cache_dir

So that maxmind unavailability doesn't affect
application startup. PERSISTENT_CACHE_DIR env var is used
to point locus at the GeoIP DB file.

* WIP: Remove ExVCR

* Fix test env config

* Fixup exvcr

* Remove exvcr from deps

* Add convert script

* Remove exvcr cassettes

* Remove convert script

* Rename test

* Update moduledoc

* Update dockerfile

* Bump CI cache

* Tag more slow tests, why not?

* Use charlist for locus cache option

* Pin nodejs

* Merge google tests, make them async

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2023-10-24 10:33:48 +02:00
Marko Saric
ff66e9b1e4
changing the link to the feedback board 2023-10-17 16:06:56 +02:00