Commit Graph

331 Commits

Author SHA1 Message Date
Karl-Aksel Puulmann
9af498833e
Channels: backfill utm_medium based on click_param_id (#4833)
* Backfill utm_medium

Follow-up to https://github.com/plausible/analytics/pull/4817

* Update backfill
2024-11-19 08:12:39 +00:00
Artur Pata
d187e59049
Fix routes not opening that take dynamic route params when the param contains forward slash (#4824) 2024-11-14 22:20:06 +00:00
MorgeMoensch
fb45e64539
Add missing question mark on login page (#4783)
* Fix typo on login form to unify login and register

* Update CHANGELOG.md
2024-11-06 16:10:34 +00:00
Karl-Aksel Puulmann
af5eb720db
Release breakdown_comparisons_ui flag (#4774) 2024-11-05 09:13:22 +00:00
RobertJoonas
0f93328b6f
Fix custom property value filter suggestions (#4720)
* new endpoint for custom prop value filter suggestions

* add test coverage for searching prop suggestions

* plug the new endpoint into FE

* update changelog

* format JS

* Update CHANGELOG.md

Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>

---------

Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>
2024-10-23 12:24:23 +00:00
ruslandoga
5fec52ab36
Release v2.1.4 (#4660) 2024-10-09 07:45:17 +00:00
ruslandoga
f28a1b8086
Log Oban errors (and still report them to Sentry) (#4657)
* log oban errors in CE

* update changelog

* not just ce

* add comment
2024-10-08 10:26:35 +00:00
hq1
2359cb920c
Account settings w sidebar (#4654)
* Outline /settings/v2 fundamentals

* Add setting tiles stubs

* Bootstrap name change

* Bootstrap theme change

* Bootstrap security settings

* Use table component for listing sessions

* Disable current e-mail field

* Implement Danger Zone

* Deal with compilation warnings

* Implement "Subscription" section

* Implement invoices list

* Fix invoices empty state & add API keys

* Fix headings in Subscription section

* Fix API keys mobile view

* Fix subscription boxes width

* Fix formatting

* Move tests for settings WIP

* Adjust remaining tests and router placement

Include docs links in tiles, where applicable.

* Fix remaining routes and remove dead code

* Fix route in a live view where no @conn is available

* Update mobile view settings picker

* Format

* Fix subscription section headings

* Fix account e-mail on dark mode

* Delete unused template

* Fix mobile setting section picker

* Optimize Login Management tile for mobile

* Update invoices section with docs link

* Update copy

* Remove trailing dots from (sub)titles

* Fix CSV export padding for "exporting" state

* Align subscription status to the right

* Fix failing test

* Fix subscription status alignment once again

* Improve subscription mobile view a little

* Fixup test compilation 🙈

* Add extra margin to subscription status box

* Make cancel button in 2FA modals expand in mobile view

* Stats API only

* Capitalize "Current session" indicator

* Show "Show More" invoices button only when there's >12

* tiny change

* Update changelog

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
Co-authored-by: Marko Saric <34340819+metmarkosaric@users.noreply.github.com>
2024-10-08 08:30:01 +00:00
Karl-Aksel Puulmann
4224f9d187
Fix revenue metrics for custom named goals (#4653)
Since https://github.com/plausible/analytics/pull/4415, revenue metrics
have not been correctly displayed on the dashboard if the goal name has
been renamed.
2024-10-07 08:18:46 +00:00
hq1
6940281d66
Settings password reset (#4649)
* Enable exceptions when revoking all user sessions

* Add `User` changeset for changing password

* Make button in `2fa_input` component optional

* Implement password change from User Settings

* Add tests

* Fix 2FA modal cancel button formatting

* Update changelog

* Don't pass redundant params to `render_settings` and clean up code a bit

* Render one error per field in password reset form

* Refactor inline form 2FA validation

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-10-03 06:39:32 +00:00
Adrian Gruntkowski
714c036757
Implement listing user sessions in user settings (#4588)
* Implement listing user sessions in user settings

* Make copy adjustments (h/t @metmarkosaric)

* Make warning button text color more consistent across user settings

* Add tests for `UserAuth.revoke_user_session/2`

* Test and improve `Auth.UserSessions`

* Test and improve controller actions

* Update CHANGELOG.md
2024-09-30 11:03:08 +00:00
Karl-Aksel Puulmann
7f361cf018
Make countries/regions/cities tooltips work (#4580)
Previously these tooltips said `Add filter: Country is undefined`
because new labels were yet to be in `query`.
2024-09-30 06:56:20 +00:00
ruslandoga
d2e374ac8c
release v2.1.3 (#4630) 2024-09-26 11:48:18 +00:00
ruslandoga
00dffb579c
release v2.1.2 (#4483) 2024-09-25 11:26:16 +00:00
Artur Pata
9fcb76d8e6
Allow sorting breakdown lists by some metrics (#4513)
* On the dashboard, allow Metrics (aka column configurations) to
  * specify width they need
  * specify if they are sortable
* Default sort order for breakdown endpoints is hardcoded on the dashboard (needed to show what column the report is sorted by)
* Unifies Google Keywords modal with other breakdowns
2024-09-12 14:49:43 +03:00
Karl-Aksel Puulmann
dd1021eff3
Limit docs query API, return 401 for API routes (#4570)
* Limit docs query API, return 401 for API routes

* Changelog entry

* Use 404 with changed error message

* Update changelog
2024-09-12 14:11:22 +03:00
ruslandoga
2180ab4764
Auto HTTPS in CE (#4491)
* auto https

* changelog

* enable auto-tls only when http_port is 80

* make erlang ssl less verbose

* simplify config

* simplify config

* fix ce_dev typo

* fix another typo

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-09-10 14:15:43 +02:00
ruslandoga
8ee4827fea
Run migrations in order across repos (#4466)
* order migrations

* migrate in 'streaks'

* cleanup

* docs

* tests

* add comment

* changelog

* continue

* run migrations tests in ci

* add pending_streams cmd

* fix ci

* add docs

* fix test

* simplify test

* only test v2+

* fix test

* return async: true

* cleanup

* add 'no pending migration'

* drop migrate/0 and pending_migrations/0

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-09-09 11:27:11 +02:00
Karl-Aksel Puulmann
604dde99fd
APIv2: Regex operations, consistent operators (#4488)
* Rename matches/does_not_match filters internally

These have never been exposed to the frontend/user directly, only via
APIv1 filtering syntax. As such we are free to rename these without
breaking things

* Rename function arguments for consistency, simplify

* Add support for `match`/`not_match` operators for query apiv2

These match the string against a regular expression, as defined in
https://github.com/google/re2/wiki/Syntax

* not_match -> match_not

* does_not_contain -> contains_not

Note that for backwards compatibility:
- Browser handles does_not_contain in URL
- Backend will handle does_not_contain in queries for a day where we will remove it for better autocompletion

* not_matches_wildcard -> matches_wildcard_not

* prettier

* match -> matches

* Fix and test fix for matches_wildcard against prop when prop is missing

* Custom properties support for matches/matches_not

* Restore contains_not

* Test contains and contains_not behavior for custom properties
2024-09-09 10:05:24 +03:00
ruslandoga
67d7c6522c
Change SMTP defaults (#4538)
* default smtp config updates

* tests

* improve SMTP_MIDDLEBOX_COMP_MODE decoding

* changelog

* format

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-09-06 10:31:23 +02:00
ruslandoga
e7ac60c067
Return domain-less cookies in CE (#4482) 2024-09-05 12:05:40 +02:00
Karl-Aksel Puulmann
059b5e0cdd
Dont include imports for time:hour and time:minute dimensions (#4504)
* Dont include imports for time:hour and time:minute dimensions

Also include more information about import warnings in query results

* Update lib/plausible/stats/query_result.ex

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>

* Revert patch

* Imported disabled graph notice (#4522)

* add explicit skip_imported_reason for unsupported interval

* stop returning information about imports from main_graph

* return warning about interval in Stats API Timeseries

* display warning bubble about interval too short for imported data

* update changelog

* improve styling of the exclamation circle icon

* return tuple from timeseries instead of map

* rename variable

* Update CHANGELOG.md

---------

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
2024-09-04 10:53:00 +03:00
ruslandoga
47c2aec2e8
Update bamboo_mua to fix headers issue (#4474)
* update bamboo_mua to fix headers issue

* changelog

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-09-03 14:09:57 +02:00
ruslandoga
6c7f123d2c
Add better assets compression to CE (#4476)
* add better assets compression to ce

* changelog

* fix conditional compilation

* improve changelog message

* don't need zstd
2024-09-03 13:54:11 +02:00
ruslandoga
4a36148f9b
Better Google error messages in CE (#4485) 2024-09-03 13:44:34 +02:00
Adrian Gruntkowski
373d4dd665
Implement token-based sessions (#4463)
* Turn `Plausible.Auth.UserSession` into full schema

* Implement token based sessions and use them as default

* Ignore expired user sessions during retrieval from DB

* Implement plug bumping user session last used and timeout timestamps

* Implement Oban worker removing expired user sessions with grace period

* Implement legacy session conversion on touch, when applicable

* Update `UserAuth` moduledoc

* Extend `UserAuth` tests to account for db-backed session tokens

* Update CHANGELOG

* Add tests for `UserSessionTouch` plug

* Add test for `CleanUserSessions` worker

* Add logging of legacy session retrievals

* Use single update permitting stale records  when touching user session

* Don't fetch session and user for external API endpoints (/api/event too)

* Refactor `Users.with_subscription/1` and expose helper query

* Skip fetching session in legacy `SessionTimeoutPlug`

* Rely on user session assign from `AuthContext` in `SentryContext`

* Silence legacy session warnings in `UserSessionTouchTest`

* Rely on session assign from `AuthPlug` in `SuperAdminOnlyPlug`

* Change `UserAuth` to get session, user and last subscription in one go

* Avoid refetching user session in `AuthorizeSiteAccess` plug

* Fix code formatting

* Refactor `UserAuth.get_user_token/1` (h/t @aerosol)

* Remove bogus empty opts from `scope` declarations in router

* Only touch session once an hour and keep `user.last_seen` in sync

* Bring back logging of legacy token use
2024-09-03 11:34:37 +02:00
Karl-Aksel Puulmann
f18d004584
Fix property filter suggestions 500 error when property hasn't been selected (#4503)
Broken by APIv2 refactors.

Relevant sentry error: https://sentry.plausible.io/organizations/plausible/issues/448/?query=is%3Aunresolved&referrer=issue-stream&statsPeriod=14d&stream_index=4
2024-09-03 10:37:17 +03:00
ruslandoga
19ecd3d0ee
Update Postgrex SSL config (#4460)
* update postgrex config

* enable ssl only if DATABASE_CACERTFILE is set

* update tests

* changelog

---------

Co-authored-by: Cenk Kücük <cenk@plausible.io>
2024-08-30 11:11:36 +01:00
Uku Taht
2e89a50188
Map lowercase tagged sources to capitalized form during ingestion (#4417)
* Map lowercase tagged sources to capitalized form during ingestion

* @moduledoc

* Add changlog entry
2024-08-27 14:03:15 +03:00
Artur Pata
8af3f7332a
Add testing framework (#4440)
* Add testing framework

* Test query period picking behaviour
2024-08-15 12:27:22 +03:00
Karl-Aksel Puulmann
4967960278
Populate log_comment with debug information, /debug/clickhouse route (#4435)
* Set log_comment with request information

* CRMAuthPlug -> SuperAdminOnlyPlug

* Super basic debug view

* Handle clustered setups

* Changelog entry

* Cleanup

* fragment trick to use ecto querying, filtering

* Move clustered_table? function to IngestRepo module

* Format

* More resilient user_id getting in helper
2024-08-14 12:33:36 +03:00
RobertJoonas
d831a4e337
Enable search for Countries, Cities, and Regions Details views (#4436)
* enable search for locations details views

* update changelog
2024-08-13 12:40:32 +03:00
hq1
cc769dfb3d
Edit goals with display names (#4415)
* Update Goal schema

* Equip ComboBox with the ability of JS selection callbacks

* Update factory so display_name is always present

* Extend Goals context interface

* Update seeds

Also farming unsuspecting BEAM programmers for better
sample page paths :)

* Update ComboBox test

* Unify error message color class with helpers seen elsewhere

* Use goal.display_name where applicable

* Implement LiveView extensions for editing goals

* Sprinkle display name in external stats controller tests

* Format

* Fix goal list mobile view

* Update lib/plausible_web/live/goal_settings/list.ex

Co-authored-by: Artur Pata <artur.pata@gmail.com>

* Update lib/plausible_web/live/goal_settings/form.ex

Co-authored-by: Artur Pata <artur.pata@gmail.com>

* Update the APIs: plugins and external

* Update test so the intent is clearer

* Format

* Update CHANGELOG

* Simplify form tabs tests

* Revert "Format"

This reverts commit c1647b5307.

* Fixup format commit that went too far

* ComboBox: select the input contents on first focus

* Update lib/plausible/goal/schema.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Update lib/plausible/goals/goals.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Update lib/plausible_web/live/goal_settings/form.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Pass form goal instead of just ID

* Make tab component dumber

* Extract separate render functions for edit and create forms

* Update test to account for extracted forms

* Inline goal get query

* Extract revenue goal settings to a component and avoid computing assigns in flight

* Make LV modal preload optional

* Disable preload for goal settings form modal

* Get rid of phash component ID hack

* For another render after render_submit when testing goal updates

* Fix LV preload option

* Enable preload back for goals modal for now

* Make formatter happy

* Implement support for preopening of LV modal

* Preopen goals modal to avoid feedback gap on loading edited goal

* Remove `console.log` call from modal JS

* Clean up display name input IDs

* Make revenue settings functional on first edit again

* Display names: 2nd stage migration

* Update migration with data backfill

---------

Co-authored-by: Artur Pata <artur.pata@gmail.com>
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-08-09 11:12:00 +02:00
RobertJoonas
1f8662438d
Add Details modals for all Devices reports (#4419)
* return concatenated browser names and versions directly from the API

* return concatenated os names and versions directly from the API

* add detailed views for all device reports

* extract put_combined_name_with_version function

* return only version under the name key when detailed=true

* update changelog

* add more metrics into detailed views of Devices reports

* split up different devices modals into separate files
2024-08-02 15:50:22 +03:00
Uku Taht
2f9ca72fd6
Expose goal contains filter in dashboard (#4412)
* Expose goal contains filter in dashboard

* CHANGELOG
2024-07-31 16:49:21 +03:00
Uku Taht
e4ef3f2888
Add changelog for searching goals (#4407) 2024-07-31 10:26:11 +03:00
RobertJoonas
6ed4f3ad69
update changelog (#4383) 2024-07-23 12:06:32 +03:00
hq1
1488683d26
Implement UI to edit existing funnels (#4369)
* Update tailwind config

Turns out `extra` wasn't scanned properly with
previous wildcards. I don't care any more, it's not slow.

* Add interface for updating funnels

* Extend ComboBox with the ability to preselect an initial value

* Implement editing funnels UI

* Update CHANGELOG

* s/add_funnel/setup_funnel

* modal width 2/5 => 2/3

* Let's not make the list disappear on modal pop-up

* Fix the damn modal width again

* Watch extra dir

* Format

* Remove commented code

The test is implemented elsewhere

* Track funnel modified to drop default selection

* Fix screen size adoption and format large numbers

* Preserve currency so that string casting includes it

* Format

* Fix ComboBox attribute for preselected option
2024-07-17 22:04:36 +02:00
ruslandoga
b988364203
Improve PG unix sockets configuration (#4202)
* improve pg unix sockets configuration

* changelog

* don't use test env in config tests
2024-07-15 13:05:31 +02:00
ruslandoga
8a77968c34
Update bamboo_mua to disable MX lookups when relay is used (#4350)
* update bamboo_mua to disable mx lookups when relay is used

* changelog
2024-07-15 12:03:06 +02:00
ruslandoga
14d5a8028f
Don't require TOTP vault key in CE (#4317) 2024-07-15 12:02:31 +02:00
hq1
d56bb2b4d5
Implement traffic drop notifications (#4300)
* Expose current visitors 12h aggregate

* Remove unused site association

* Distinct drop/spike notification factories

* Rename modules accordingly + implement drop handling

* Rename periodic oban service

* Implement drop email

* Rest of the owl

* Update changelog

* Update moduledoc

* Update moduledoc

* Min threshold to 1

* Threshold 1

* Remove merge artifact

* Put panel behind a feature flag

* Format
2024-07-11 14:55:18 +02:00
Karl-Aksel Puulmann
ef2e277f89
fix(suggestions): Properly provide suggestions (#4332)
* fix(suggestions): Properly provide suggestions

Previously, if a filter was applied then filter modal suggestions would
_always_ apply the previous suggestion. Now when say applying a `page`
filter suggestions we remove the previous `page` filter.

* Improve readability
2024-07-10 12:06:16 +03:00
RobertJoonas
8c6234604a
Modal Search Continued (#4326)
* update changelog

* remove redundant noop

* capitalize changelog entry
2024-07-09 16:31:32 +03:00
Karl-Aksel Puulmann
a181f3eab3
APIv2: TimeSeries using QueryBuilder, release experimental_session_count (#4305)
* Move fragments module under Plausible.Stats.SQL

* Introduce select_merge_as macro

This simplifies some select_merge calls

* Simplify select_join_fields

* Remove a needless dynamic

* wrap_select_columns macro

* Move metrics from base.ex to expression.ex

* Move WhereBuilder under Plausible.Stats.SQL

* Moduledoc

* Improved macros

* Wrap more code

* select_merge_as more

* Move defp to the end

* include.time_labels parsing

* include.time_labels in result

Note that the previous implementation of the labels from TimeSeries.ex was broken

* Apply consistent function in imports and timeseries.ex

* Remove boilerplate

* WIP: Limited support for timeseries-with-querybuilder

* time:week dimension

* cleanup: property -> dimension

* Make querying with time series work

* Refactor: Move special metrics (percentage, conversion rate) to own module

* Explicitly format datetimes

* Consistent include_imported in special metrics

* Solve week-related crash

* conversion_rate hacking

* Keep include_imported consistent after splitting the query

* Simplify do_decide_tables

* Handle time dimensions in imports cleaner

* Allow time dimensions in custom property queries

* time:week handling continued

* cast_revenue_metrics_to_money

* fix `full_intervals` support

* Handle minute/realtime graphs

* experimental_session_count? with timeseries

This becomes required as we try to include visits from sessions by default

* Support hourly data in imports

* Update bounce_rate in more csv tests

* Update some time-series query tests

* Fix for meta.warning being included incorrectly

* Simplify imported.ex

* experimental_session_count flag removal

* moduledoc

* Split interval and time modules
2024-07-09 14:25:02 +03:00
Adrian Gruntkowski
2f2602e316
Stop listing UA views in Google data import flow (#4309)
* Stop listing UA views in Google data import flow

* Update CHANGELOG

* Trigger preview deploy
2024-07-04 13:08:32 +02:00
Karl-Aksel Puulmann
0594478add
APIv2: Replace breakdown module with QueryBuilder (#4293)
* Revert "Revert "APIv2: Replace breakdown module with QueryBuilder (#4283)" (#4292)"

This reverts commit ef5e0e0382.

* Allow querying events and pageviews from sessions table

This is not strictly accurate, especially with shorter time frames, but
is useful for a fallback mechanism. I'll figure out something around
shorter time frames in the future.

See also: https://github.com/plausible/analytics/pull/4292

* Only query events and pageviews in legacy breakdowns
2024-07-01 12:50:01 +03:00
Karl-Aksel Puulmann
ef5e0e0382
Revert "APIv2: Replace breakdown module with QueryBuilder (#4283)" (#4292)
This reverts commit 7dd12d1dd6.
2024-07-01 10:50:44 +03:00
Karl-Aksel Puulmann
7dd12d1dd6
APIv2: Replace breakdown module with QueryBuilder (#4283)
* WIP: Breakdown using QueryBuilder

* Revert "Remove problematic test"

This reverts commit b442bb5d1f.

* Get more breakdown tests passing

* Preload goals, sort when dealing with time_on_page

* Handle conversion_rate in breakdowns

* Simplify ordering by using selected_as consistently for dimensions

* Get breakdown tests passing

* Strings to atoms in keys for StatsController.transform_keys calls to work

* Handle revenue metrics removal

* Add test for nil-removal case

* Include percentage metric

* Fix and test with imported locations

* Fixup time-on-page

* Fix country/region automatic filters

* Handle multiple imports (os/browser version) in importsv2

* Filter goals

* Default to ordering by page as well

* Calculate conversion rate on sessions if needed

* Order by event dimensions - handles event:page special case

* Update tests

* Update more tests, handle goal=0 case in imports

* Handle event:goal breakdowns correctly with filters

* Revenue to money

* Improved table deciding

* Also update event:page filters on event:page breakdown

* bounce_rate to 0

Previous behavior relied on two queries being made - new query leads to 0 naturally

* Update pagination test

* dont count non-pageviews as path goal completions

* Make revenue logic breakdown-specific

Its hard to fit into the new schema and likely needs a rethink for apiv2

* Retain previous behavior for TimeSeries module

* Get GA4 test passing

Most failures are related to ordering, pageviews shouldnt be read off of sessions

* Clean up old methods

* Simplify imported.ex

* Dont crash on garbage filters

* Reflect ordering-related change in test

* Fix test data

* Update table_decider

* Re-simplify get_revenue_tracking_currency

* Revert revenue changes

* Use Query.set

* Remove a TODO

* csv importer: no pageviews

Pageviews were incorrectly fetched from sessions table before, causing issues

* csv importer tweaking

* Remove use Plausible

* to_existing_atom
2024-07-01 09:03:33 +03:00
ruslandoga
2214c43b34
happy tcp (#4245)
* happy tcp

* changelog

* update mix.lock

---------

Co-authored-by: hq1 <hq@mtod.org>
2024-06-24 13:29:55 +02:00