Commit Graph

12 Commits

Author SHA1 Message Date
Karl-Aksel Puulmann
141eea88ff
APIv2: Revenue metrics (#4659)
* WIP: Start refactoring revenue metrics

* Hacks to make things work

* Remove old revenue code, remove revenue metrics if needed

* Update query_optimizer docs

* Minor fixes

* Add tests around average/total revenue when non-revenue goal filtering going on

* Optimize, calculate filters as expected (OR-ing clauses)

* Revenue: Handle cases where revenue metrics should not be returned or nil

* Expose revenue metrics in internal schema, add tests

* Docstring

* Remove TODO

* Typegen

* Solve warnings

* Remove nesting

* ce_test fix

* Tag tests as ee_only

* Fix: When filtering by revenue goal and no conversions, return 0.0 instead of nil

* More straight-forward preloading logic
2024-10-09 10:18:48 +00:00
Karl-Aksel Puulmann
5ad743c8d3
APIv2: Comparisons for breakdowns, timeseries, time_on_page (#4647)
* Refactor comparisons to a new options format

Prerequisite for APIv2 comparison work

* Experiment with default include deduplication

* WIP

Oops, breaks `include.total_rows`

* WIP

* Refactor breakdown.ex

* Pagination fix: dont paginate split subqueries

* Timeseries tests pass

* Aggregate tests use QueryExecutor

* Simplify QueryExecutor

* Handle legacy time-on-page metric in query_executor.ex

No behavioral changes

* Remove keep_requested_metrics

* Clean up imports

* Refactor aggregate.ex to be more straight-forward in output format building

* top stats: compute comparison via apiv2

* Minor cleanups

* WIP: Pipelines

* WIP: refactor for code cleanliness

* QueryExecutor to QueryRunner

* Make compilable

* Comparisons for timeseries works

Except for comparisons where comparison window is bigger than source query window

* Add special case for timeseries

* JSON schema tests for comparisons

* Test comparisons with the new API

* comparison date range parsing improvement

* Make comparisons api internal-only

* typegen

* credo

* Different schemata

* get_comparison_query

* Add comment on timeseries result format

* comparisons typegen

* Percent change for revenue metrics fix

* Use defstruct for query_runner over map

* Remove preloading atoms
2024-10-08 10:13:04 +00:00
Artur Pata
82a15884ad
Automatically generate Typescript types for v2 API query schema (#4574)
* Generate types from query schema

* Flip the query schema so private is static

* Ensure private schema stays private

* Refactor comment, json schema utils
2024-09-18 11:01:20 +00:00
Uku Taht
7a77ebf9bf
Add feature-flagged channels UI (#4585)
* Add feature-flagged channels UI

* Implement channels modal

* Channel -> Channels tab
2024-09-18 08:34:12 +00:00
Karl-Aksel Puulmann
ef57502854
APIv2: Implement pagination and include.total_rows (#4575)
Offset-based pagination is used to make sure Looker integration
is able to work as efficiently as possible. To know how many
requests users need to do `include.total_rows` option was added.
2024-09-12 15:51:18 +03:00
Karl-Aksel Puulmann
bd11b4cf67
APIv2: Standard iso8601 timestamps, operate on UTC (#4563)
* query.date_range is now in UTC instead of user timezone

This simplifies things down the line and fixes several bugs where
query.date_range is cast to naivedatetime for ecto purposes

Many places still remain broken:
- comparison queries
- `to_date_range` calls

* Make default_for_date_range not care about time zones

* Make timezone parameter mandatory for to_date_range

* Simplify utc_date_range, update legacy query builder

* Fix more cases where query date range is needed

* query.date_range -> query.utc_time_range

* Query.date_range/1 function

* ensure_include_imported update

* Clean up send_email_report
2024-09-11 09:21:59 +03:00
Artur Pata
52b94842c0
Assert filters are tuples, simplify schema (#4541) 2024-09-10 18:01:42 +03:00
Karl-Aksel Puulmann
e8d544c841
Remove does_not_contain support (#4564)
It only needed to be live until users have reload. This has been live
for >24h.
2024-09-10 15:38:04 +03:00
Karl-Aksel Puulmann
604dde99fd
APIv2: Regex operations, consistent operators (#4488)
* Rename matches/does_not_match filters internally

These have never been exposed to the frontend/user directly, only via
APIv1 filtering syntax. As such we are free to rename these without
breaking things

* Rename function arguments for consistency, simplify

* Add support for `match`/`not_match` operators for query apiv2

These match the string against a regular expression, as defined in
https://github.com/google/re2/wiki/Syntax

* not_match -> match_not

* does_not_contain -> contains_not

Note that for backwards compatibility:
- Browser handles does_not_contain in URL
- Backend will handle does_not_contain in queries for a day where we will remove it for better autocompletion

* not_matches_wildcard -> matches_wildcard_not

* prettier

* match -> matches

* Fix and test fix for matches_wildcard against prop when prop is missing

* Custom properties support for matches/matches_not

* Restore contains_not

* Test contains and contains_not behavior for custom properties
2024-09-09 10:05:24 +03:00
Karl-Aksel Puulmann
8fa3a83129
APIv2: and/or/not support (#4480)
* First approximation of AND/OR/NOT support

Broken by this:
- Goal filtering
- Table deciding
- Imports

* TableDecider handle nesting

* Query.remove_top_level_filters

* Plausible.Stats.Imported.SQL.Expression

* Handle AND/OR/NOT with imported data, create Plausible.Stats.Imported.SQL.WhereBuilder

* Add parser validations for event:goal, event:hostname and event:props:x filters top level constraints

* Move module around

* Query.get_filter -> Filters.filtering_on_dimension? in some callsites

* Filters.get_toplevel_filter

* TableDecider.sessions_join_events?, remove old method

* Transforming filters in query_optimizer

* Query API tests for and/or/not

* Reorder parser steps

* Post-merge test fixups

* Solve merge issue

* Simplify filtering_on_dimension?

* Update transformer code

* dimensions_used_in_filters min_depth option, simplify parser validations

* rename_dimensions_used_in_filter

* fix rename_dimensions_used_in_filter

* Rename a test
2024-09-04 15:44:03 +03:00
RobertJoonas
f04c47f881
Support realtime periods in API v2 (#4469)
* add realtime date_ranges into the private API schema

This commit starts parsing date ranges into a new NaiveDateTimeRange
struct, rather than a simple Date.Range.

* transform realtime labels into negative integers + test

* move schema type argument to last position in helper functions

* allow passing a date param + tests

* Update test/plausible/stats/query_parser_test.exs

Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>

* Update test/plausible/stats/query_parser_test.exs

Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>

* Update test/plausible/stats/query_parser_test.exs

Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>

* Update test/plausible/stats/query_parser_test.exs

Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>

* keep test file structure consistent

* Turn NaiveDateTimeRange into DateTimeRange

* change 'now' field from NaiveDateTime to DateTime in v2 query

* fix minute interval labels + add missing tests

* return query_result.date_range as iso8601 timestamps with timezone

* allow timestamps with tz as date_range arguments in API v2

* delete Plausible.Timezones.to_utc_datetime

* simplify returning comparison periods

* add comment about realtime not supported in comparisons

* pass only now instead of test_opts

* drop redundant else branch

* separate tests

* stick to a single check_date_range function in tests

* fix credo error

---------

Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>
2024-09-02 12:56:58 +03:00
Karl-Aksel Puulmann
9c71161eab
APIv2: JSON schema validation, separate internal and public API validation (#4464)
* Restore `date` internal parameter, validate via json schema

* Improved error formatting from json schema, get most tests passing

* Handle internal overrides to JSON schema

* Parsing tests all pass

* Remove some repeated code, enforce length/uniqueness in schema

* Explicit separation between internal and public API validation

* Mark file as external_resource

* map_join

* Update query tests

* Update query tests

* Serve schema under an /api/docs/query/schema.json endpoint

* dotify errors
2024-08-26 14:01:27 +03:00