Commit Graph

18 Commits

Author SHA1 Message Date
Karl-Aksel Puulmann
1cb7982cd9
Filtering by multiple custom properties (#3719)
* WIP: PropFilterRow

* Get multi-behavior working

* Render multiple prop filters in one

* Modal reads from query string correctly

* Backend support for multiple custom property filters

* Add backend tests for multiple custom property filters

* Disable already selected options in property keys

We can't allow choosing the same property multiple times without changing the request
params, which we decided against

* Allow choosing any property under Behaviors > Custom props even if custom prop filter applied

This was a limitation (I believe) introduced by using ARRAY JOINs to query custom properties

* CHANGELOG.md

* Solve credo warning about too deep nesting

* Update assets/js/dashboard/stats/modals/prop-filter-modal.js

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>

* Refactor internal function for clarity

* Add another step -> Add another

* Solve 500 error

* Separate boxes per property filter

* Retain other filters in props table

* removeFilter behavior for props

* matches_member support for custom props

* filter_suggestions for prop keys should account for prop filter

* find over filter

* refactor appliedFilters

* FILTER_TYPES => FILTER_OPERATIONS

* Make add another link not wrap the whole page

* Unique keys

---------

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
2024-02-12 09:03:57 +02:00
hq1
117eef000d
Upgrade Erlang/Elixir stack (#3454)
* Bump deps

* Bump stack

* Fix deprecation warnings

* Fix VCR cassettes mismatch due to OTP-18414

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Format & fix flaky tests

* Handle raw IPv4 hostnames; test public suffix TLD

* Configure locus db cache_dir

So that maxmind unavailability doesn't affect
application startup. PERSISTENT_CACHE_DIR env var is used
to point locus at the GeoIP DB file.

* WIP: Remove ExVCR

* Fix test env config

* Fixup exvcr

* Remove exvcr from deps

* Add convert script

* Remove exvcr cassettes

* Remove convert script

* Rename test

* Update moduledoc

* Update dockerfile

* Bump CI cache

* Tag more slow tests, why not?

* Use charlist for locus cache option

* Pin nodejs

* Merge google tests, make them async

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2023-10-24 10:33:48 +02:00
Vini Brasil
8c3b541971
Allow internal prop keys in queries (#3231)
* Remove `Plausible.Sites.set_allowed_event_props/2` function

This commit removes the `Plausible.Sites.set_allowed_event_props/2`
function in favor of `Plausible.Props.allow/2`.

* Allow internal prop keys in queries

This commit allows internal prop keys by default in queries. I decided
not to include internal keys in the database, because they might change
and we'd need to migrate it again.

* remove redundant 'props.' from behaviours/index.js

---------

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
2023-08-05 15:13:16 +01:00
RobertJoonas
51cad4c0ec
Props into CSV export (#3167)
* move set_allowed_event_props into context module

* extract parse_csv fn

* add custom_props.csv to CSV export (under FF)

* add test for custom prop export with a prop filter

* add test for goal filtered scenario

* mix format

* fix flaky test

* extract breakdown_custom_prop_values function

* Revert "fix flaky test"

This reverts commit 256b9bb2e7. The flaky
test was already fixed in 86557b1878

* make set_allowed_event_props a bang function

* test that all files are included in the CSV export
2023-07-24 15:27:44 +01:00
RobertJoonas
db359c2a6f
Filter out garbage props by site.allowed_event_props (#2907)
* add migration

* add not null constraint

* remove not null constraint and default value

* add site schema change

* filter garbage props for prop_key suggestions

* filter garbage props returned by StatsController.conversions/2

* add crm action to set prop allowlist

* use nullable allowed_event_props field instead

* filter garbage props in db query instead

* implement suggestion dealing with string input

* refactor the allowlist condition in db query

* improve test
2023-05-16 11:59:56 +02:00
hq1
6a831d9898
Harden API input validation (#2869)
* Reject invalid country codes early

* Validate all date parameters early

* Ensure request JSON parses to a map

* Re-arrange function calls

* Extend malicious filter validation
2023-05-04 14:09:43 +02:00
RobertJoonas
caeff41932
Prop filter modal (#2841)
* Add PropFilterModal (only UI)

* small variable refactor

* allow selecting prop value filter type

* allow selecting only one prop_key

* allow selecting many prop_values only when prop_key selected

* handle submitting filter

* get applied filters from query + remove option

* change prop filter label format

* support member and not_member filter types for pageview props

* show (none) value in filter suggestions

* refactor zip_results/4 and remove unused code

* fix displaying (none) values in goals section prop breakdown

* remove unnecessary functionality

* fix bug: returning prop names for goal :member filter

* fix bug: submitting regular filter modal with Enter key

* bugfix: disallow opening prop filter modal when feature flag disabled

* mix format

* break selected combobox values into multiple lines

* fix useEffect behavior for focusing on prop_key field

* support submitting prop filter with Enter key

* refactor getFormState in PropFilterModal

* separate fetchPropKey and fetchPropValue functions

* Allow querying props for pageview goals

* Make the internal props API only return a list of props (not map)
* Separate function for fetching all props in Stats API goal breakdown (this returns a map as before)

* ditch state for keeping search bar visible

* group by event_name in db query
2023-04-27 14:09:33 +03:00
RobertJoonas
7204e470a8
Filter modal refactor (#2806)
* Refactor: FilterModal -> RegularFilterModal

* Rename filter.js to regular-filter-modal.js

* Refactor: extract FilterTypeSelector component

* fix undefined order in test

* classNames function style improvement
2023-04-05 16:20:35 +01:00
hq1
d2f2c69387
Conditionally support switching between v1 and v2 clickhouse schemas (#2780)
* Remove ClickhouseSetup module

This has been an implicit point of contact to many
tests. From now on the goal is for each test to maintain
its own, isolated setup so that no accidental clashes
and implicit assumptions are relied upon.

* Implement v2 schema check

An environment variable V2_MIGRATION_DONE acts like
a feature flag, switching plausible from using old events/sessions
schemas to v2 schemas introduced by NumericIDs migration.

* Run both test suites sequentially

While the code for v1 and v2 schemas must be kept still,
we will from now on run tests against both code paths.
Secondary test run will set V2_MIGRATION_DONE=1 variable,
thus making all `Plausible.v2?()` checks return `true'.

* Remove unused function

This is a remnant from the short period when
we would check for existing events before allowing
creating a new site.

* Update test setups/factories with v2 migration check

* Make GateKeeper return site id along with :allow

* Make Billing module check for v2 schema

* Make ingestion aware of v2 schema

* Disable site transfers for when v2 is live

In a separate changeset we will implement simplified
site transfer for when v2 migration is complete.
The new transfer will only rename the site domain in postgres
and keep track of the original site prior to the transfer
so we keep an ingestion grace period until the customers
redeploy their scripting.

* Make Stats base queries aware of v2 schema switch

* Update breakdown with v2 conditionals

* Update pageview local start with v2 check

* Update current visitoris with v2 check

* Update stats controller with v2 checks

* Update external controller with v2 checks

* Update remaining tests with proper fixtures

* Rewrite redundant assignment

* Remove unused alias

* Mute credo, this is not the right time

* Add test_helper prompt

* Fetch priv dir so it works with a release

* Fetch distinct partitions only

* Don't limit inspect output for partitions

* Ensure SQL is printed to IO

* Remove redundant domain fixture
2023-03-27 13:52:42 +02:00
Uku Taht
531dfb114b
Refactor: Use HeadlessUI for search-select component (#2676)
* Use HeadlessUI for search select box

* Remove downshift from package.json

* More consistent API for Combobox component

* Combine toFilterType and valueWithoutPrefix into a single function

* Rename MyCombobox -> PlausibleCombobox

* Update webpack-cli

* Disable cache for build

* Revert "Disable cache for build"

This reverts commit aa130541f8.

* Disable cache for build

* Update webpack dependencies

* Remove glob from webpack config

* Webpack is required by package.json

* Require autoprefixer in postcss config

* Revert build changes

* Fix styling for dark mode
2023-02-20 11:10:11 +02:00
Vini Brasil
9c98a3f2e8
Add API support for intervals (#2417) 2022-11-14 18:41:51 -03:00
Adam Rutkowski
0fa6b688af
Google APIs integration improvements (#2358)
* Make TestUtils module available in all tests

* Add macros patching the application env in tests

Unfortunately a lot of existing functionality relies on
certain application env setup. This isn't ideal because
the app config is a shared state that prevents us from
running the tests in parallel.

Those macros encapsulate setting up new env for test purposes
and make sure the changes are reverted when the test finishes.

* Allow passing request opts to HTTPClient.post/4

We need this to swap custom request building in
Google Analytics import.

* Unify errors when listing sites

* React: propagate backend error messages if available

* React: catch API errors in Search Terms component

* Propagate google API errors on referrer drilldown

* Handle verified properties errors in SC settings

* Add missing tests for SC settings controller

* Unify errors for fetching search analytics queries (list stats)

* Unify errors refreshing Google Auth Token

* Test fetch_stats/3 errors and replace Double with Mox

* Fixup makrup

* s/class/className

* Simplify Search Terms display in case of errors

* Fix warnings
2022-10-24 09:34:02 +02:00
Adam Rutkowski
ec90a264b4
Improve user input validation (#2291)
* Make pagination params parsing ignore bad input

* Remove unused binding

* Don't crash on filter parse error

* Sanitize input date on internal stats API

* Revert Query module changes (ref 55645734)

* Implement simplistic input date validation in stats controller

* Mute bad request logging
2022-10-11 14:42:14 +02:00
RobertJoonas
40275b64d4
Pageview custom dimensions (#1816)
* added custom dimension filtering tests for pages

* first filter UI in place

* pages, entry pages and exit pages can be filtered by pageview props

* added tests for expected filtering behaviour

* fix dimension filter for sources + tests

* added is_not filtering functionality

* fixed formatting

* fixed admin_test

* added (none) as filter value + is_not filter type in UI

* added prefilling applied filter values and some UI tweaks

* added fetch options

* Make prop suggestions work with `props` filter

* Fix test

* Track login state internally

* Add CHANGELOG entry

Co-authored-by: Uku Taht <uku.taht@gmail.com>
2022-04-21 11:47:15 +03:00
Uku Taht
1dba113e2f
[Draft] Improve location translations (#1526)
* WIP

* Use location library for search suggestions

* Remove unused code

* Remove Countries completely

* Fix tests
2021-12-13 12:03:27 +02:00
Uku Taht
05bf43c1be
City level location data (#1449)
* Merge branch 'plausible_master'

* Add City level details

* Add City level details

* Use ISO codes instead of geoname_id for subdivisions

* Add easier way to configure geolocation database

* Add workflow for dev branch

* Correct clickhouse migration

* Translate subdivision names

* Translate city names

* WIP

* Region and country filters

* Fix region filter

* Remove region_name when removing region filter

* Add modals for regions and cities

* Remove dead code

* WIP

* Revert "WIP"

This reverts commit 3202bf2fe9.

* Feature flag to hide cities when deployed

* Add changelog entry

* Remove unused code

* Remove unused variables

* Fix test

Co-authored-by: AymanTerra <aymanterra@yahoo.com>
2021-11-23 11:39:09 +02:00
Uku Taht
c68b88c24b
Upgrade otp (#1313)
* update tool versions to latest

* Upgrades erlang and elixir, format project

Co-authored-by: happysalada <raphael@megzari.com>
2021-09-09 11:17:24 +03:00
Vignesh Joglekar
30ac901141
Adds manual, editable, auto-suggested filters, and negated&globbed path-based filters (#1121)
* Adds manual-filters + friends

commit 308192044d726e9a6f7406e333048ab3407aa260
Merge: 39287ab a299fab
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri Jun 11 03:40:55 2021 -0500

    Merge branch 'manual-filters' into manual-filters-2

commit a299fab1fb
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri Jun 11 02:20:26 2021 -0500

    Changes to split and pattern matched function for time_on_page

commit 10f10c933a
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri Jun 11 01:53:18 2021 -0500

    Fixes a couple of minor UX issues

commit f2e5ce8eb3
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri Jun 11 01:49:10 2021 -0500

    Fixes time on page for globbed and negated page paths

commit bb18af6526
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu Jun 10 05:24:05 2021 -0500

    Close to finalized version of updated version

    Just needs some additional testing + potentially code cleanup

commit d0b7bfe3da
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri May 28 04:21:21 2021 -0500

    Real Dialyzer Fix

commit 296a76af34
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri May 28 03:44:29 2021 -0500

    Dialyzer fix

commit 91f3b44017
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 17:11:24 2021 -0500

    Changelog

commit e041f75745
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 17:09:52 2021 -0500

    Formatting

commit f689642204
Merge: e00929b 4ff25f6
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 17:08:47 2021 -0500

    Merge branch 'master' into manual-filters

commit e00929b1c5
Merge: 83887c4 806975e
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 17:01:48 2021 -0500

    Merge branch 'manual-filters' of github.com:Vigasaurus/plausible-analytics into manual-filters

commit 83887c49cc
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 16:59:32 2021 -0500

    Adds tests for suggestions, formats goals suggestion query

commit 1cb7732a08
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 16:36:31 2021 -0500

    Adds goals as auto-complete capable filter

commit 4ca39cc406
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 16:08:22 2021 -0500

    95% completed auto-complete setup

    Still needs:
    - tests
    - goals added as filter

commit 22d38c4cd8
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 03:47:19 2021 -0500

    80% of auto-complete filters progress

    Still needs
    - countries and screen into new format
    - re-style dropdown and background
    - drop debounce time
    - tests

commit 806975ede9
Merge: 81c5e05 1a93542
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Tue May 25 15:28:21 2021 -0500

    Merge branch 'master' into manual-filters

commit 81c5e05760
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Tue May 25 15:21:03 2021 -0500

    Makes colorings on top bar elements consistent

commit fa7f6c2268
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Tue May 25 14:58:25 2021 -0500

    Makes requested changes, adds different version of filter button

commit 7dc65b9b7a
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Sat May 22 04:29:01 2021 -0500

    Changelog

commit c684f1c76a
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Sat May 22 04:26:14 2021 -0500

    Various UI Improvements

    - Makes edit buttons full-length & properly sized
    - Adds remove filter button in edit menu

commit a632e7a8ee
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Sat May 22 03:11:50 2021 -0500

    Adds tests for exclusions and wildcards

commit eb91a7942b
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Sat May 22 03:02:23 2021 -0500

    Fixes editing UX on list view

commit 6209d72aee
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri May 21 04:01:17 2021 -0500

    Bugfix in realtime view, formatting

commit 007d44df38
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri May 21 03:23:16 2021 -0500

    Second pass - mostly everything user-facing is done

    Still needs:
    - Tests
    - Potentially negating other filters
    - Potentially some code cleanup

commit cb7b5b9fbd
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri May 21 01:49:52 2021 -0500

    First pass on manual filter & path regex/negated filters

    Still needs:
    - Form structure on filter modal
    - Edit filter button
    - Filter dropdown UI improvement
    - Filter modal mount data collection
    - Tests
    - Potentially negating other filters

* Fixes an issue between the updated React version and Flatpickr not showing up

Reverting to v16 was the simplest fix without diving into the issue upstream with Flatpickr

* Resolves comments on UI and code style

Co-authored-by: Uku Taht <Uku.taht@gmail.com>
2021-06-21 14:42:16 +03:00