* Wrap Plausible.Stats.Filters with unit tests
* Parse `member` filter type
* Support escaped | in member filter
* Support for `member` filter in `aggregate_time_on_page`
* Add support for `member` filter type on goals
* Disable Credo warning
* Support `not_member` filter type
* Disable credo for `query_sessions`
* Support `matches_member` and `not_matches_member` filters
* Disable Credo for `Filters.filter_value/2`
* Support `matches_member` and `not_matches_member` for goal filter
* Support for contains_member and friends
* Updates for new chto driver
* make top_stats_test.exs:203 pass (#2779)
---------
Co-authored-by: ruslandoga <rusl@n-do.ga>
This pull request adds support for multiple comparison modes, changes the comparison checkbox to a combobox, and implements the year over year comparison mode. The feature is still behind a feature flag.
Co-authored-by: Uku Taht <uku.taht@gmail.com>
* globally rename 'pages_per_visit' to 'views_per_visit'
* change the order of top stats
* rename 'Visits' to 'Total visits' in the UI
* add views_per_visit to UI
* put the new metric under a feature flag
* add new metric to CSV export under feature flag
* mix format
* use only one feature flag
* refactor metric validation
* link to the correct docs section
* add the new metric to aggregate API
* explicitly remove __internal_visits
The overall metric selection is well defined by
`Plausible.Stats.Timeseries.empty_row/2`. The only metric that needs
to be removed from the timeseries response is __internal_visits.
This commit also moves the `remove_internal_visits_metric` function to a
new Util module to be used by both breakdown and timeseries.
* add the new metric to timeseries API
* mix format
* add moduledoc to keep credo happy
* convert pages_per_visit to string straight away
* do rounding in db query
* query # of visits with sum(sign) instead
* stop converting pages_per_visit to string
* Use user-agent instead of screen_width to get device type
Co-authored-by: eriknakata <erik.nakata5@gmail.com>
* Fix credo
* Log on unhandled UAInspector device type
* Make 'browser' the default tab in devices report
* Remove device tooltip
* Remove screen_width from ingestion completely
* Remove browserstack harness, run playwright directly
* Select meta key based on OS platform
* Run CI tests in parallel
* Improve device match readability
* Add changelog
---------
Co-authored-by: eriknakata <erik.nakata5@gmail.com>
* Clickhouse migration: add ingest_counters table
* Configure ingest counters per MIX_ENV
* Emit telemetry for ingest events with rich metadata
* Allow building Request.t() with fake now() - for testing purposes
* Use clickhousex branch where session_id is assigned to each connection
* Add helper function for getting site id via cache
* Add Ecto schema for `ingest_counters` table
* Implement metrics buffer
* Implement buffering handler for `Plausible.Ingestion.Event` telemetry
* Implement periodic metrics aggregation
* Update counters docs
* Add toStartOfMinute() to ordering key
* Reset the sync connection state in `after` clause
* Flush counters on app termination
* Use separate Repo with async settings enabled at config level
* Switch to clickhouse_settings repo root config key
* Add AsyncInsertRepo module
* Add visits metric and make it graphable
* include visits metric in csv export (visitors.csv)
* put visits under a feature flag (CSV export)
* feature flag for displaying visits on the dashboard
* fix formatting
* add visits metric to top stats (fix)
* fix imported_test to expect visits metric included
* fix formatting
* Use HeadlessUI for search select box
* Remove downshift from package.json
* More consistent API for Combobox component
* Combine toFilterType and valueWithoutPrefix into a single function
* Rename MyCombobox -> PlausibleCombobox
* Update webpack-cli
* Disable cache for build
* Revert "Disable cache for build"
This reverts commit aa130541f8.
* Disable cache for build
* Update webpack dependencies
* Remove glob from webpack config
* Webpack is required by package.json
* Require autoprefixer in postcss config
* Revert build changes
* Fix styling for dark mode
* Reject unknown imported cities from queries
This commit fixes a bug where the city report returned `N/A` entries.
The functions that build imported data queries were using SQL
`COALESCE`, assuming city data is `NULL` when unknown, when actually its
unknown value is `0`.
This commit addresses the problem using SQL `NULLIF` combined with the
previous `COALESCE` call. With this change both `0` and `NULL` are
treated as unknown.
Since 1cb07efe6d cities can be `NULL`, but
previously we saved `0` as unknown.
Closes#1960
* Add entry to CHANGELOG
* Ignore cyclomatic complexity Credo check
This commit adds city data to imported records from Google Analytics. The
current implementation sets city to 0 because GA does not use the GeoNames
database.
Google Analytics Reporting API uses [Geographical IDs](https://developers.google.com/analytics/devguides/collection/protocol/v1/geoid)
to identify cities and countries. Plausible uses
[GeoNames](https://geonames.org/) and I couldn't find databases corelating the
two.
Fortunately, GA also returns the city name and this commit uses the city name
and the country ISO code to find the Geoname ID. To avoid making expensive ETS
searches, I created another ETS table in the Location library that uses
{country, city} as a key.
Related PR: https://github.com/plausible/location/pull/3
* Configure ingest repo access/pool size
If I'm not mistaken 3 is a sane default, the only
inserts we're doing are:
- session buffer dump
- events buffer dump
- GA import dump
And all are serializable within their scopes?
* Add IngestRepo
* Start IngestRepo
* Use IngestRepo for inserts
* Annotate ClickhouseRepo as read_only
So no insert* functions are expanded
* Update moduledoc
* rename alias
* Fix default env var value so it can be casted
* Use IngestRepo for migrations
* Set default ingest pool size from 3 to 5
in case conns are restarting or else...
* Ensure all Repo prometheus metrics are collected
* Make Device section components aware of (not set)
So that no extra sub-filters are possible when the unset
top item is selected.
* Support '(not set)' in breakdown/filters
* Update expectations for export tests
* Add extra tests for returning/filtering by '(not set)'
* Add changelog entry
* Remove ListReport conditional render
* Prevent redundant sub-filters
* Fix filter text rendering
---------
Co-authored-by: Uku Taht <uku.taht@gmail.com>
* Move Endpoint errors setup to common config
* Implement naive Sentry link resolver
* Implement error report e-mail
* Delete static sentry script
* Implement user feedback form on server errors
* Re-arrange pipe
* Use Sentry.Config.dsn() where applicable
* Fix typo
* Use Map.replace/3
Some changes to be more consistent with the emails we send. Also "valid" subscription rather than "active" subscription fits better for the different cases where this screen is shown
* fix subquery for sessions in base_event_query/2
As the 'sessions' table is using the CollapsingMergeTree engine, we have
to select session_id's distinctively. Otherwise we will get multiple rows
(with sign -1 and 1) as long as the background merge hasn't happened.
* update changelog
* use GROUP BY instead of SELECT DISTINCT
* remove comma
* Fingerprint DBConnection.ConnectionError in Sentry
* Check events before creating a site
* Enable sites limit screen
* Remove debugging remnant
* Fix buggy assertions
This wasn't doing what expected:
iex(1)> Repo.exists?(Plausible.Site, domain: "foo")
[debug] QUERY OK source="sites" db=0.6ms idle=1906.2ms
SELECT TRUE FROM "sites" AS s0 LIMIT 1 []
* Encapsulate check to satisfy credo
* Use less technically involved error message
* Bring back e-mail to the limit error message
This PR replaces geolix with locus to simplify self-hosted setup. locus can auto-update maxmind dbs which are recommended for self-hosters if they want city-level geolocation. locus is also a bit faster.
This PR also uses a test mmdb file from https://github.com/maxmind/MaxMind-DB for e2e geolocation tests without stubs.
* Add Checkly Terraform config
* add deployment workflow
* use pagerduty instead of email for notifications
* use terraform cloud backend
* update variable declaration
* rename checkly check group
* update syntax
* test trigger
* Revert "test trigger"
This reverts commit 333e82beac.
* run a single job at a time
Co-authored-by: Cenk Kücük <c@cenk.me>
* Fix breakdown API pagination when using event metrics
This commit fixes a bug where the subsequent breakdown API pages had
the same items as the first page. The fix sorts the underlying
ClickHouse query by timestamp, keeping the same order between requests,
as we use OFFSET/LIMIT pagination.
* Fix repeated results assertion
* Add different ORDER BY to each breakdown property
* Ignore XX and T1 countries
* Add fallback if country_code=nil
* Lookup city overrides directly in CityOverrides module
* Changelog
* Add empty moduledoc
* Remove redundant comment
