Commit Graph

2050 Commits

Author SHA1 Message Date
hq1
71ef0bd043
Clean up after V2 migration (#2868)
* Clean up after V2 migration

This PR removes all the leftovers and alternative code
branching after v2 migration.

The self-hosted release is being drafted at:

https://github.com/plausible/hosting/issues/68

Refs:
  - https://github.com/plausible/analytics/pull/2865
  - https://github.com/plausible/analytics/pull/2825
  - https://github.com/plausible/analytics/pull/2780

* !fixup
2023-04-24 12:17:57 +02:00
ruslandoga
adcce15632
Make self-hosted data migration easier (#2865)
* default to v2

* allow N defaults in data migration prompt and custom messages

* join domains lookup

* remove duplicate test runs from ci (both are v2)
2023-04-21 09:33:57 +02:00
Marko Saric
586bbeb8ac
Adding a note to contact us for more API calls (#2858)
this should make it clearer to people that they can get more API capacity when they encounter the "Too many API requests" error
2023-04-18 16:38:32 +02:00
hq1
b8e96b948f
Stop using Double in cache store tests (#2854) 2023-04-17 09:24:45 +01:00
dependabot[bot]
1e0610037e
Bump eslint from 8.35.0 to 8.38.0 in /tracker (#2836)
Bumps [eslint](https://github.com/eslint/eslint) from 8.35.0 to 8.38.0.
- [Release notes](https://github.com/eslint/eslint/releases)
- [Changelog](https://github.com/eslint/eslint/blob/main/CHANGELOG.md)
- [Commits](https://github.com/eslint/eslint/compare/v8.35.0...v8.38.0)

---
updated-dependencies:
- dependency-name: eslint
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-17 09:12:05 +01:00
Vini Brasil
3d656ae95b
Match day of the week feature for comparisons (#2822)
* Add support for `match_day_of_week?` back-end option

* Add match day of week input to comparison input

* Default match day of the week to true
2023-04-13 14:01:54 +01:00
hq1
825a754976
Make ingest threshold configurable (#2845)
* Make ingest threshold configurable

* Credo
2023-04-13 13:52:54 +02:00
dependabot[bot]
ef73816bb5
Bump fun_with_flags_ui from 0.8.0 to 0.8.1 (#2837)
Bumps [fun_with_flags_ui](https://github.com/tompave/fun_with_flags_ui) from 0.8.0 to 0.8.1.
- [Release notes](https://github.com/tompave/fun_with_flags_ui/releases)
- [Changelog](https://github.com/tompave/fun_with_flags_ui/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tompave/fun_with_flags_ui/compare/v0.8.0...v0.8.1)

---
updated-dependencies:
- dependency-name: fun_with_flags_ui
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-13 12:42:26 +02:00
hq1
b9c2110472
V2 migration tweaks for self hosted release (#2825)
* Get rid of PASS_V2_SCHEMA_MIGRATION

* Use in-memory domain lookup + regular table settings

* Remove faulty date arithmetic + prev part calculation

* Set V2_MIGRATION_DONE in Mix.env == :dev

* Mute credo
2023-04-13 12:09:39 +02:00
Vini Brasil
9f036d3915
Show detailed comparison dates in top stats (#2804)
* Pass comparison_query as argument to top stats functions

* Add comparison values and dates to top stats API

* Display comparison dates and values in top stats

* Rename function to renderPercentageComparison

* Create component to do conditional rendering

* DRY date range formatting function

---------

Co-authored-by: Uku Taht <uku.taht@gmail.com>
2023-04-11 09:59:10 -03:00
hq1
c04e9286dd
Associate goals with sites, not domains (#2828)
* Revert "Rephrase error message"

This reverts commit f624443a96.

* Revert "Temporarily disable goal creation"

This reverts commit a091635b9d.

* Update ecto schema

* Make sure goal operations are per site

* Update tests

* Split postgres migrations
2023-04-10 10:51:36 +02:00
hq1
3cb089eab4
Migrate and freeze goals creation (#2833)
* Full migration (to be submitted separately)

* Do not remove `Goal.domain` just yet

* Do not make Goal.site not nullable just yet

* Temporarily disable goal creation

* Rephrase error message

* Add down migration
2023-04-10 10:29:10 +02:00
dependabot[bot]
df67f7c378
Bump json5 from 1.0.1 to 1.0.2 in /assets (#2569)
Bumps [json5](https://github.com/json5/json5) from 1.0.1 to 1.0.2.
- [Release notes](https://github.com/json5/json5/releases)
- [Changelog](https://github.com/json5/json5/blob/main/CHANGELOG.md)
- [Commits](https://github.com/json5/json5/compare/v1.0.1...v1.0.2)

---
updated-dependencies:
- dependency-name: json5
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-07 11:15:08 -03:00
dependabot[bot]
2df7d25df3
Bump ex_doc from 0.28.5 to 0.29.4 (#2832)
Bumps [ex_doc](https://github.com/elixir-lang/ex_doc) from 0.28.5 to 0.29.4.
- [Release notes](https://github.com/elixir-lang/ex_doc/releases)
- [Changelog](https://github.com/elixir-lang/ex_doc/blob/main/CHANGELOG.md)
- [Commits](https://github.com/elixir-lang/ex_doc/compare/v0.28.5...v0.29.4)

---
updated-dependencies:
- dependency-name: ex_doc
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-07 11:12:45 -03:00
dependabot[bot]
b3b02421a0
Bump @playwright/test from 1.31.2 to 1.32.2 in /tracker (#2815)
Bumps [@playwright/test](https://github.com/Microsoft/playwright) from 1.31.2 to 1.32.2.
- [Release notes](https://github.com/Microsoft/playwright/releases)
- [Commits](https://github.com/Microsoft/playwright/compare/v1.31.2...v1.32.2)

---
updated-dependencies:
- dependency-name: "@playwright/test"
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-07 11:08:58 -03:00
Uku Taht
e672ea66ec
Frontend timezone issues (#2810)
* Use dayjs with UTC plugin for date-formatter

* Remove 'toHuman' function

* Use dayjs dates in Datepicker component

* Use util/date.js functions in date formatter

* Remove `fromJSDate`

* Use formatISO instead of raw dayjs formatter
2023-04-07 09:56:02 -03:00
hq1
154ce3a44c
Split clickhouse repos - making the main one read only (#2826)
* Split clickhouse pools into readonly/import deletions

* Remove CRM site transfers

* Initialize ImportDeletionRepo

* Put ImportDeletionRepo to use
2023-04-06 12:45:36 +02:00
hq1
509c549c01
Update ua_inspector database (#2827)
* Update `ua_inspector` database

* Add changelog entry
2023-04-06 12:45:19 +02:00
dependabot[bot]
d069d0a164
Bump hammer from 6.0.0 to 6.1.0 (#2824)
Bumps [hammer](https://github.com/ExHammer/hammer) from 6.0.0 to 6.1.0.
- [Release notes](https://github.com/ExHammer/hammer/releases)
- [Changelog](https://github.com/ExHammer/hammer/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ExHammer/hammer/compare/v6.0.0...v6.1.0)

---
updated-dependencies:
- dependency-name: hammer
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-06 10:41:11 +02:00
RobertJoonas
7204e470a8
Filter modal refactor (#2806)
* Refactor: FilterModal -> RegularFilterModal

* Rename filter.js to regular-filter-modal.js

* Refactor: extract FilterTypeSelector component

* fix undefined order in test

* classNames function style improvement
2023-04-05 16:20:35 +01:00
dependabot[bot]
4edf126196
Bump bcrypt_elixir from 2.3.1 to 3.0.1 (#2391)
Bumps [bcrypt_elixir](https://github.com/riverrun/bcrypt_elixir) from 2.3.1 to 3.0.1.
- [Release notes](https://github.com/riverrun/bcrypt_elixir/releases)
- [Changelog](https://github.com/riverrun/bcrypt_elixir/blob/master/CHANGELOG.md)
- [Commits](https://github.com/riverrun/bcrypt_elixir/commits)

---
updated-dependencies:
- dependency-name: bcrypt_elixir
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-05 12:00:13 +02:00
Uku Taht
71e4d228c0
Escape pipes in goal names (#2814)
* Escape pipes in goal names

* Apply suggestions from code review

Use ~S sigil for better readability

Co-authored-by: Vini Brasil <vini@hey.com>

---------

Co-authored-by: Vini Brasil <vini@hey.com>
2023-04-05 12:59:25 +03:00
ruslandoga
b646652071
add transport opts to clickhouse repos (#2783) 2023-04-05 11:58:55 +02:00
hq1
406ea6f0cd
Render an error on site creation with domain=domain_changed_from (#2821) 2023-04-05 11:57:20 +02:00
hq1
ffc96205f6
Make general settings consistent with the rest (#2819) 2023-04-04 12:52:34 +02:00
hq1
1d01328287
Allow domain change (#2803)
* Migration (PR: https://github.com/plausible/analytics/pull/2802)

* Implement Site.Domain interface allowing change and expiry

* Fixup seeds so they work with V2_MIGRATION_DONE=1

* Update Sites.Cache so it's capable of multi-keyed lookups

* Implement worker handling domain change expiration

* Implement domain change UI

* Implement transition period for public APIs

* Exclude v2 tests in primary test run

* Update lib/plausible_web/controllers/site_controller.ex

Co-authored-by: Vini Brasil <vini@hey.com>

* Update lib/plausible_web/controllers/site_controller.ex

Co-authored-by: Vini Brasil <vini@hey.com>

* Update moduledoc

* Update changelog

* Remove remnant from previous implementation attempt

* !fixup

* !fixup

* Implement domain change via Sites API

cc @ukutaht

* Update CHANGELOG

* Credo

* !fixup commit missing tests

* Allow continuous domain change within the same site

---------

Co-authored-by: Vini Brasil <vini@hey.com>
2023-04-04 10:55:12 +02:00
hq1
5ca53a70be
Start :ssl for migrations (#2818)
* Start :ssl for migrations

* Start :ch in migrate.sh
2023-04-04 10:47:31 +02:00
hq1
94a86a17eb
Migration: enable domain changes (upcoming feature) (#2802)
* Migration: enable domain changes (upcoming feature)

* Pass domain check within the same site
2023-04-04 10:12:08 +02:00
hq1
70a85c3f4c
Revert "Temporarily disable site creation (#2813)" (#2817)
This reverts commit b626329ec5.
2023-04-04 08:23:31 +02:00
Uku Taht
b626329ec5
Temporarily disable site creation (#2813)
* Temporarily disable site creation

* Fix linter

* Disabled message wording
2023-04-04 08:14:42 +02:00
dependabot[bot]
ee92595815
Bump loader-utils from 1.4.1 to 1.4.2 in /assets (#2447)
Bumps [loader-utils](https://github.com/webpack/loader-utils) from 1.4.1 to 1.4.2.
- [Release notes](https://github.com/webpack/loader-utils/releases)
- [Changelog](https://github.com/webpack/loader-utils/blob/v1.4.2/CHANGELOG.md)
- [Commits](https://github.com/webpack/loader-utils/compare/v1.4.1...v1.4.2)

---
updated-dependencies:
- dependency-name: loader-utils
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-30 14:15:06 -03:00
dependabot[bot]
3a9f528724
Bump eslint-plugin-playwright from 0.10.0 to 0.12.0 in /tracker (#2722)
Bumps [eslint-plugin-playwright](https://github.com/playwright-community/eslint-plugin-playwright) from 0.10.0 to 0.12.0.
- [Release notes](https://github.com/playwright-community/eslint-plugin-playwright/releases)
- [Commits](https://github.com/playwright-community/eslint-plugin-playwright/compare/v0.10.0...v0.12.0)

---
updated-dependencies:
- dependency-name: eslint-plugin-playwright
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-30 14:11:22 -03:00
dependabot[bot]
e71e3717b3
Bump @playwright/test from 1.31.1 to 1.31.2 in /tracker (#2721)
Bumps [@playwright/test](https://github.com/Microsoft/playwright) from 1.31.1 to 1.31.2.
- [Release notes](https://github.com/Microsoft/playwright/releases)
- [Commits](https://github.com/Microsoft/playwright/compare/v1.31.1...v1.31.2)

---
updated-dependencies:
- dependency-name: "@playwright/test"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-03-30 13:11:27 -03:00
Uku Taht
9a4c4f46bb
Remove empty filters from suggestions query (#2801) 2023-03-29 09:45:58 -03:00
Uku Taht
70f1c6538d
Fix open state in combobox (#2797)
* Fix open state in combobox

* Improve filter suggestions when using the `contains` filter

* Support contains filter type for utm filter group

* Remove console.log
2023-03-28 20:06:37 +02:00
ruslandoga
66be99b38a
update chto for missing source in telemetry fix (#2787) 2023-03-28 07:06:03 +02:00
hq1
e8509203a2
Update location dependency (faster startup) (#2793)
* Update location dependency (faster startup)

* Update with https://github.com/plausible/location/pull/6
2023-03-27 16:56:34 +02:00
RobertJoonas
bc835895a2
Rename dimensions script to pageview-props (#2786)
* rename dimensions script to pageview-props

* commit mix.lock change
2023-03-27 17:50:15 +03:00
Vini Brasil
5fe32b59cd
Control comparison datepicker using React state (#2784)
This commit improves the datepicker state management. Due to a bug, we
needed to use setTimeout to open Flatpickr. This change replaces the
setTimeout workaround by using a proper React state.
2023-03-27 11:03:14 -03:00
Uku Taht
46048e50f7
Support multiple filters - frontend (#2773)
* Wrap Plausible.Stats.Filters with unit tests

* Parse `member` filter type

* Support for `member` filter in `aggregate_time_on_page`

* Support `not_member` filter type

* Support `matches_member` and `not_matches_member` filters

* Extract util module for React filters

* Implement Combobox from scratch with no libs

* Support multple filter clauses in combobox

* Don't use browser / os in version label

* Show highlighted option in combobox

* WIP

* Fix location filters outside filter modal

* Align open/close behaviour with react-select

* Styling updates for combobox

* Add support for wildcards in Combobox

* Implement keybindings for combobox

* Allow free choice inputs in combobox

* Rename 'Save filter' -> Apply filter

* Remove TODO comment

* Clean up some rebase mistakes

* Rename `allowWildcard` -> `freeChoice`

* Dark mode fixes

* Remove hint from filter modal

* Escape pipe character in filter modal

* Do not allow selecting duplicate options in combobox

* Escape brackets in `page_regex/1`

* Fix disabled style in dark mode

* Add regex fallback for safari

* Show no matches found when visibleOptions is empty

* Disable enter key when no visible options

* Do not submit empty form fields

* Remove unnecessary setOpen(true)
2023-03-27 16:51:31 +03:00
hq1
d2f2c69387
Conditionally support switching between v1 and v2 clickhouse schemas (#2780)
* Remove ClickhouseSetup module

This has been an implicit point of contact to many
tests. From now on the goal is for each test to maintain
its own, isolated setup so that no accidental clashes
and implicit assumptions are relied upon.

* Implement v2 schema check

An environment variable V2_MIGRATION_DONE acts like
a feature flag, switching plausible from using old events/sessions
schemas to v2 schemas introduced by NumericIDs migration.

* Run both test suites sequentially

While the code for v1 and v2 schemas must be kept still,
we will from now on run tests against both code paths.
Secondary test run will set V2_MIGRATION_DONE=1 variable,
thus making all `Plausible.v2?()` checks return `true'.

* Remove unused function

This is a remnant from the short period when
we would check for existing events before allowing
creating a new site.

* Update test setups/factories with v2 migration check

* Make GateKeeper return site id along with :allow

* Make Billing module check for v2 schema

* Make ingestion aware of v2 schema

* Disable site transfers for when v2 is live

In a separate changeset we will implement simplified
site transfer for when v2 migration is complete.
The new transfer will only rename the site domain in postgres
and keep track of the original site prior to the transfer
so we keep an ingestion grace period until the customers
redeploy their scripting.

* Make Stats base queries aware of v2 schema switch

* Update breakdown with v2 conditionals

* Update pageview local start with v2 check

* Update current visitoris with v2 check

* Update stats controller with v2 checks

* Update external controller with v2 checks

* Update remaining tests with proper fixtures

* Rewrite redundant assignment

* Remove unused alias

* Mute credo, this is not the right time

* Add test_helper prompt

* Fetch priv dir so it works with a release

* Fetch distinct partitions only

* Don't limit inspect output for partitions

* Ensure SQL is printed to IO

* Remove redundant domain fixture
2023-03-27 13:52:42 +02:00
Uku Taht
a75d0b35b0
Fix Timex.total_offset blowing up during clock changes (#2788)
* Fix Timex.total_offset blowing up during clock changes

* Format large numbers with underscore in tests

Co-authored-by: Adam <hq@mtod.org>

---------

Co-authored-by: Adam <hq@mtod.org>
2023-03-24 10:41:01 -03:00
Adam
6637751a5e
Implement Numeric IDs migration (#2762)
* Implement Numeric IDs migration

* Fix typo

* Mute credo for now

* Improve configurability and add stop_t

* Adjust to Ch/Chto only

* Fix opts key for dictionary password

* Add regular ecto migration with numeric ids v2 schemas (#2768)

* Add regular ecto migration

* Fix typo

* Update priv/ingest_repo/migrations/20230320094327_create_v2_schemas.exs

Co-authored-by: Vini Brasil <vini@hey.com>

* Implement v2 events/sessions schema modules (#2777)

* Implement v2 events/sessions schema modules

* Clean up session schemas

---------

Co-authored-by: Vini Brasil <vini@hey.com>

* Update moduledocs

---------

Co-authored-by: Vini Brasil <vini@hey.com>
2023-03-23 09:47:41 +01:00
ruslandoga
a129bf82e0
use github: plausible/chto for chto (#2782)
* use github: plausible/chto for chto

* use chto with correct ch github reference (plausible/ch)
2023-03-22 14:56:57 +01:00
Vini Brasil
87a63fe28a
Custom comparison mode (#2753)
This commit implements the custom comparison mode using a date range.
2023-03-22 09:31:44 -03:00
Marko Saric
9f71bb3921
Updating password reset copy (#2775) 2023-03-22 11:38:30 +02:00
Adam
baa04be191
Remove obsolete migration config (#2769)
The compilation warning advises to use `Application.compile_env/3`
however there is no `:url` environment setting under `:plausible` key
anyway, so we might as well get rid of it completely.
2023-03-22 11:32:24 +02:00
Uku Taht
a154a4214b
Support multiple filters - backend (#2749)
* Wrap Plausible.Stats.Filters with unit tests

* Parse `member` filter type

* Support escaped | in member filter

* Support for `member` filter in `aggregate_time_on_page`

* Add support for `member` filter type on goals

* Disable Credo warning

* Support `not_member` filter type

* Disable credo for `query_sessions`

* Support `matches_member` and `not_matches_member` filters

* Disable Credo for `Filters.filter_value/2`

* Support `matches_member` and `not_matches_member` for goal filter

* Support for contains_member and friends

* Updates for new chto driver

* make top_stats_test.exs:203 pass (#2779)

---------

Co-authored-by: ruslandoga <rusl@n-do.ga>
2023-03-22 11:01:25 +02:00
Adam
6d79ca5093
Switch to new clickhouse adapter (ch/chto) (#2733)
* another clickhouse adapter

* don't restore stats_removal.ex

* fix events main-graph error (#2746)

* update ch, chto

* update chto again (#2759)

* Stop treating page filter as an entry_page filter (#2752)

* remove dead code

* stop treating page filter as entry page filter in breakdown queries

* stop treating page filter as entry page filter in aggregate queries

* stop treating page filter as entry page filter in timeseries queries

* mix format

* update changelog

* break code down to smaller functions to keep credo happy

* remove unused functions

* make CSV export return only conversions with goal filter (#2760)

* make CSV export return only conversions with goal filter

* update changelog

* update elixir version in mix.exs (#2742)

* revert admin.ex changes (#2776)

---------

Co-authored-by: ruslandoga <67764432+ruslandoga@users.noreply.github.com>
Co-authored-by: ruslandoga <rusl@n-do.ga>
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
2023-03-21 09:55:59 +01:00
Adam
736e6e385c
Capture no_cache sentry messages together (#2772) 2023-03-20 14:03:05 +01:00