* add metric validation + support in aggregate
* add a test ensuring comparison works
* disallow time_on_page with a goal filter
* Return time_on_page as `nil` from aggregate API
In case time_on_page cannot be calculated, we'll return it as `nil` from
the Stats API.
This is to make the behaviour consistent between breakdown and aggregate
endpoints. As for the UI, we'll still continue to report time_on_page as
0 - not changing any UI behaviour as discussed with Marko.
* add tests for time_on_page in event:page breakdown
* update changelog
* invalidate time_on_page with event:name filter
* add the ability to only query time_on_page in page breakdown
We'll need the visitors metric to get the list of pages to calculate the
time_on_page for.
* Dependencies: swap Cachex for ConCache
* Implement Cache adapter wrapping ConCache
* Implement cache stats tracker, for metrics
* Use Cache.Adapter in Plausible.Cache
Marking the test as not slow anymore
* Use Cache Adapter when tracking sessions
* Use Cache Adapter for UA parsing
* Rename child identifiers - cachex is obsolete now
* Test stats tracking
* Update grafana metrics
* Put all caches under common child specification
* Try less
* Shorten the function delegation path
* changes to the in-app footer
changes to the in-app footer to focus on product usage links rather than marketing pages
* Format
---------
Co-authored-by: Marko Saric <34340819+metmarkosaric@users.noreply.github.com>
* wip
* more env, setup user before packages to avoid them picking our uid
* make entrypoint.sh executable by all again
* apparently it's a best practice to have executables be owned by root
* make entrypoint executable in COPY
* stop writing to /app, write to /tmp if needed
* fewer changes
* system user
* keep same style for multiline commands
* fewer changes
* add changelog entry
* fix group assignment for plausible user
* use gid=999
* no home
* no home
* add gecos
* add plausible user to nogroup instead of creating a custom one
* eh
* fewer changes
* fewer changes
* fewer changes
* use PERSISTENT_CACHE_DIR instead of STORAGE_DIR
* ignore more
* cleanup
* remove hex timeout env var
* use ERL_FLAGS=+JMsingle true in public builds
* fallback to /tmp and nest under /tzdata_data/ for tzdata
---------
Co-authored-by: Cenk Kücük <cenk@plausible.io>
This can cause issues as 1 day after a deploy the tzdata list is updated
from 2021 to 2024. Selecting a timezone thats not supported in 2021 can
then break dashboards temporarily following a deploy.
This can be reverted after https://github.com/plausible/analytics/pull/3811
* WIP mutation to populate event session columns
* Remove duplication
* report errors, allow_nondeterministic_updates
* use right columns
* Update existing columns instead of session_* ones
* Make dialyzer happy
* Fix issue with passing pre-existing params in
* Logger -> IO.puts
* Use IngestRepo.config for connection settings
* Make dictionary options configurable
* Move allow_nondeterministic_mutations to within the migration
* Solve credo warning about too deep nesting
* Missed logger call
* Pattern matching in function head
* Remove `add_percentage`, calculate percentages in clickhouse queries
This simplifies querying logic and avoids doing extra queries and avoids
race conditions.
* Remove special none handling from breakdowns, handling percentages correctly
* Add (failing) test showing expected add_percentage behavior for user making multiple sessions
* Update add_percentage behavior to use separate subqueries
* Refactor: Explicitly add field names to INSERT
This avoids issues when code schema is out of sync with real schema
* Dont write session parameters to events
These would only be stored on first event anyways. Work remains to be done
on tests which have their own helper
* Remove writes to country_code in a test
* Remove old columns from being accessible in elixir code
* Update most tests to use new way of adding session props to events
* Update testing harness
* Update stats controller test
* Update for shield rules
* update breakdown tests
* Fix typing of state for dialyzer
* Drop support for old session attributes code
* Update remaining tests
* cond -> if
This migration will noop in staging/production as it already has been run. It also leaves
behind a backup table that initially takes no extra space but will need to be cleaned up
manually
* Create a stub of site settings section for imports and exports
* Use legacy site import indication to determine UA import handling
* Add provisional logos for upcoming import sources
* Stub basics of import page
* Add very rudimentary support for multiple UA imports
* Implement imports list as live view
* Add support for opening LV modal from backend and closing from frontend
* Introduce notion of themes to `button` and `button_link` components
* Add confirmation modal on deleting import
* Swap GA4 logo
* Implement disabled state support for `button_link` component
* Disable export and non-implemented import sources
* Use native starts start date for upper boundary of import time range
* Ensure integrations view uses legacy UA import flow
* Remove unnecessary preload in SiteController
* Remove unnecessary exception for legacy imports
* Move API controller stats tests under PlausibleWeb
* Test listing imports
* Add test for explicit listener setup
* Add tests for legacy flag state in UA importer
* Add test for purging legacy import data
* Add tests for `Sites.native_stats_start_date`
* Test forgetting imports
* Add `Stats.Clickhouse.imported_pageview_counts/1` and fix test flakiness
* Show page view counts on imports list
* Add tests for static imports and exports view
* Adjust button look slightly
* Use `case` instead of `cond`
* Make feature flag customisable per site
* Fix buttons and empty state styling
* Add another import to seeds
* Use JS confirm dialog instead of modal for deletion confirmations
* Revert "Add support for opening LV modal from backend and closing from frontend"
This reverts commit 260e6c753032b451542e24be9edc2118790b5a00.
* Default `legacy` to false when inserting new import jobs
* Drop `method` attribute from `button_link` and `unstyled_link` components
* add csv importer
* make table validation explicit
* update some docs
* improve docs
* add minio container to ci
* more tests
* eh
* continue
* add passing test
* add failing test
* add config test
* add minio to Makefile
* testcontainers
* remove extra whitespace
* explain the implementation a bit
* account for async deletes in tests
* bounces is UInt32
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Migration: add country rules
* Add CountryRule schema
* Implement CountryRule cache
* Add country rules context interface
* Start country rules cache
* Lookup country rules on ingestion
* Remove :shields feature flag from test helpers
* Add nested sidebar menu for Shields
* Fix typo
* IP Rules: hide description on mobile view
* Prepare SiteController to handle multiple shield types
* Seed some country shield
* Implement LV for country rules
* Remove "YOU" indicator from country rules
* Fix small build
* Format
* Update typespecs
* Make docs link point at /countries
* Fix flash on top of modal for Safari
* Build the rule struct with site_id provided up-front
* Clarify why we're messaging the ComboBox component
* Re-open combobox suggestions after pressing Escape
* Update changelog
* Fix font size in country table cells
* Pass `added_by` via rule add options
* Display site's timezone timestamps in rule tooltips
* Display formatted timestamps in site's timezone
And simplify+test Timezone module; an input timestamp converted
to UTC can never be ambiguous.
* Remove no-op atom
* Display the maximum number of rules when reached
* Improve readability of remove button tests
* Credo
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
Follow-up to https://github.com/plausible/analytics/pull/3719
The previous behavior was predicated on:
- Allowing a single custom property filter
- Allowing breaking down only by the chosen custom property filter
Now these restrictions are removed the previous auto-switching just becomes annoying
* Replace footer text
* add COPYING.txt file
* Add new logos
* Use new logos in all layouts
* New logos
* Check license key on startup
* Bypass license check when Mix.env == :dev
* Use new logos with smaller wordmarks
* Add generic logo_path/1 function
* Use new favicons everywhere
* Bypass license check in test env
* Use sha256 for license key hash
* Mix.env -> config_env()
* Use Mix.evn at compile time rather than runtime
* Mix format
---------
Co-authored-by: Uku Taht <uku.taht@gmail.com>
* improve test
* add os to os_version breakdown
...and add operating_system_versions.csv to the CSV export
* fix conversion rate for os_version breakdown
* update changelog
* fix existing CSV tests
* use case instead of cond
* Add data migration for moving to VersionedCollapsingMergeTree
This has been tested locally and partially on staging. Still requires a bit of work to verify.
Verification query:
```
SELECT main._partition_id, tmp.count, main.count
FROM (
SELECT _partition_id, count() AS count
FROM sessions_v2_tmp_versioned
GROUP BY _partition_id
) AS tmp
FULL OUTER JOIN (
SELECT _partition_id, count() AS count
FROM sessions_v2
GROUP BY _partition_id
) AS main
ON (tmp._partition_id == main._partition_id)
ORDER BY main._partition_id
```
* Add an early exit to migration
* cluster? extract common code
* Add `GET /capabilities` to Plugins API
It aims to:
- help the client verify the data-domain the token is associated with
- list all the features available for the site's owner
(and therefore determine availability of the subset of those for the current
Plugins API caller)
The endpoint does not require authentication, in the sense that it'll
always respond with 200 OK. However when the token is provided,
a verification lookup is made.
* Remove IO.inspect() call
* Credo
* Aesthetics
* s/send_resp/send_error/
* Call preload just once
This will makes it impossible to store session attributes on events.
Looking at production data also revealed no cases where these updates
are effective.
