Commit Graph

208 Commits

Author SHA1 Message Date
Adam Rutkowski
8a2df294f5
Replace empty OS/screen size/Browser with (not set) in UI/API/exports (#2646)
* Make Device section components aware of (not set)

So that no extra sub-filters are possible when the unset
top item is selected.

* Support '(not set)' in breakdown/filters

* Update expectations for export tests

* Add extra tests for returning/filtering by '(not set)'

* Add changelog entry

* Remove ListReport conditional render

* Prevent redundant sub-filters

* Fix filter text rendering

---------

Co-authored-by: Uku Taht <uku.taht@gmail.com>
2023-02-07 12:47:54 +01:00
Marko Saric
4ea519fd11
Some changes to the "site taken" copy (#2647)
* Some changes to the "site taken" copy

* Update site.ex

* Fix tests

---------

Co-authored-by: Vinicius Brasil <vini@hey.com>
2023-02-06 15:33:45 -03:00
Adam Rutkowski
482249ffba
Better error message on site creation error (#2644)
* Ensure clear error message on site creation error

* Update colons in API error messages
2023-02-06 12:20:48 +01:00
RobertJoonas
bd0de97521
Fix returning more pageviews with a visit property filter (#2612)
* fix subquery for sessions in base_event_query/2

As the 'sessions' table is using the CollapsingMergeTree engine, we have
to select session_id's distinctively. Otherwise we will get multiple rows
(with sign -1 and 1) as long as the background merge hasn't happened.

* update changelog

* use GROUP BY instead of SELECT DISTINCT

* remove comma
2023-01-23 12:14:27 +02:00
ruslandoga
166748dcf2
Replace Geolix with Locus (#2362)
This PR replaces geolix with locus to simplify self-hosted setup. locus can auto-update maxmind dbs which are recommended for self-hosters if they want city-level geolocation. locus is also a bit faster.

This PR also uses a test mmdb file from https://github.com/maxmind/MaxMind-DB for e2e geolocation tests without stubs.
2023-01-17 12:05:09 -03:00
RobertJoonas
13a05b4e1e
Change metric labels for realtime with goal filter (#2560)
* change metric labels for realtime with goal filter

* changelog

* fix formatting
2023-01-09 09:31:55 +02:00
Vini Brasil
4503895d0a
Fix breakdown API pagination when using event metrics (#2562)
* Fix breakdown API pagination when using event metrics

This commit fixes a bug where the subsequent breakdown API pages had
the same items as the first page. The fix sorts the underlying
ClickHouse query by timestamp, keeping the same order between requests,
as we use OFFSET/LIMIT pagination.

* Fix repeated results assertion

* Add different ORDER BY to each breakdown property
2023-01-04 22:14:40 -03:00
Uku Taht
1785653b1e
Ignore unknown countries (#2556)
* Ignore XX and T1 countries

* Add fallback if country_code=nil

* Lookup city overrides directly in CityOverrides module

* Changelog

* Add empty moduledoc

* Remove redundant comment
2023-01-03 10:35:23 -03:00
Adam Rutkowski
5de43b758d
Run tests in async mode where applicable (#2542)
* Set pg pool size for MIX_ENV=test

* Include slow tests in CI run

* Exclude slow tests by default

* Mark tests slow/async where applicable

* Restructure captcha mocks

* Revert async where env is relied upon

* Add --max-failures=1 to CI run

* Set warnings as errors

* Disable async where various mocks are used

* Revert "Disable async where various mocks are used"

This reverts commit 2446b72a29.

* Disable async for test using vcr
2022-12-26 10:20:29 -03:00
Vini Brasil
0e87b489f6
Return empty list when breaking down by event:page without events (#2530)
* Return empty list when breaking down by event:page without events

This commit fixes a bug with pagination where breaking down by event:page
would always return results despite pagination.

Closes #2255

* Update CHANGELOG.md
2022-12-19 13:49:03 +01:00
Uku Taht
a37b3433d7
Always show direct traffic in sources reports (#2531)
* Remove show_noref behaviour

Removes query param show_noref which was used from React to control
whether to show Direct / None traffic or not. The show_noref behaviour
was untested previously.

Closes #2523

* Add changelog entry

* Fix tests

* Removed files I did not mean to check in :)
2022-12-19 13:40:21 +01:00
Adam Rutkowski
467317f24e
Fix: application startup, event timestamps (#2479)
### Changes

This PR:
- pushes PromEx to the bottom of supervision stack to avoid Endpoint
instrumentation failure
- ensures the site cache is ready by exposing it through the health
check endpoint
- fixes event timestamps being calculated at compile time, with
regression unit and integration tests

### Tests
- [x] Automated tests have been added
- [ ] This PR does not require tests

### Changelog
- [ ] Entry has been added to changelog
- [x] This PR does not make a user-facing change

### Documentation
- [ ] [Docs](https://github.com/plausible/docs) have been updated
- [x] This change does not need a documentation update

### Dark mode
- [ ] The UI has been tested both in dark and light mode
- [x] This PR does not change the UI
2022-11-29 10:46:49 +01:00
Adam Rutkowski
356575ef78
Gatekeep ingestion pipeline (#2472)
* Update Sites.Cache

So it's now capable of refreshing most recent sites.
Refreshing a single site is no longer wanted.

* Introduce Warmer.RecentlyUpdated

This is Sites Cache warmer that runs only for
most recently updated sites every 30s.

* Validate Request creation early

* Rename RateLimiter to GateKeeper and introduce detailed policies

* Update events API tests - a provisioned site is now required

* Update events ingestion tests

* Make limits visible in CRM Sites index

* Hard-deprecate DOMAIN_BLACKLIST

* Remove unnecessary clause

* Fix typo

* Explicitly delegate Warmer.All

* GateKeeper.allwoance => GateKeeper.check

* Instrument Sites.Cache measurments

* Update send_pageview task to output response headers

* Instrument ingestion pipeline

* Credo

* Make event telemetry test a sync case

* Simplify Request.uri/hostname handling

* Use embedded schema, apply action and rely on get_field
2022-11-28 15:50:55 +01:00
Vignesh Joglekar
497a52c10a
Add support for intervals (#1574)
Co-authored-by: Vini Brasil <vini@hey.com>
Co-authored-by: Vignesh Joglekar <hey@vigneshjoglekar.com>
2022-11-22 09:50:58 -03:00
Vini Brasil
9c98a3f2e8
Add API support for intervals (#2417) 2022-11-14 18:41:51 -03:00
Adam Rutkowski
0fa6b688af
Google APIs integration improvements (#2358)
* Make TestUtils module available in all tests

* Add macros patching the application env in tests

Unfortunately a lot of existing functionality relies on
certain application env setup. This isn't ideal because
the app config is a shared state that prevents us from
running the tests in parallel.

Those macros encapsulate setting up new env for test purposes
and make sure the changes are reverted when the test finishes.

* Allow passing request opts to HTTPClient.post/4

We need this to swap custom request building in
Google Analytics import.

* Unify errors when listing sites

* React: propagate backend error messages if available

* React: catch API errors in Search Terms component

* Propagate google API errors on referrer drilldown

* Handle verified properties errors in SC settings

* Add missing tests for SC settings controller

* Unify errors for fetching search analytics queries (list stats)

* Unify errors refreshing Google Auth Token

* Test fetch_stats/3 errors and replace Double with Mox

* Fixup makrup

* s/class/className

* Simplify Search Terms display in case of errors

* Fix warnings
2022-10-24 09:34:02 +02:00
Adam Rutkowski
ec90a264b4
Improve user input validation (#2291)
* Make pagination params parsing ignore bad input

* Remove unused binding

* Don't crash on filter parse error

* Sanitize input date on internal stats API

* Revert Query module changes (ref 55645734)

* Implement simplistic input date validation in stats controller

* Mute bad request logging
2022-10-11 14:42:14 +02:00
Adam Rutkowski
523b114682
Allow super-admin Stats API access for locked sites (#2308) 2022-10-05 12:04:37 +02:00
Adam Rutkowski
bf2b6880c5
Block Stats API for locked sites (#2302)
* Extract Sites.locked? predicate

* Lock Stats API when dashboard is locked

* Tidy tests

* Don't pollute application env from tests

* Add changelog entry

* Revert "Add changelog entry"

This reverts commit 76346074f9.
2022-10-04 15:34:45 +03:00
Uku Taht
12a513f3cd
Remove Mimic dependency (#2280) 2022-10-03 11:41:51 -03:00
Vinicius Brasil
7489290d11
International Domain Names (IDN) Support (#2034)
* Accept letters from non-Latin alphabets in domain names

* Replace static URLs with Router functions in settings_visibility

* Beautify dashboard URL in visibility tab

* Add IDN support to CHANGELOG
2022-09-28 10:42:15 +03:00
RobertJoonas
155e274150
Stats API: allow escaping | literal character with \| when filtering (#2266)
* add separate module for filter parsing

* add tests for filter parser

* allow escaping pipe character in filter value

* add documentation and doctests

* do not remove escape chars from wildcard values

* changelog update

* change the parse_filters/1 function argument
2022-09-26 16:20:08 +03:00
Uku Taht
e16e357dd2
Fix shared link with bad auth (#2225)
* Render 404 when shared link cannot be found

* Add documentation for StatsController and shared link rendering

* Refactor shared_link/2 for more clarity

* Add changelog entry

* Use mermaid graph for sequence diagram

* Use more accurate return value in sequence diagram

* Refactor Ecto query to be more idiomatic

* Remove order dependence in test

* Restore backwards compatibility for older shared links

* Add changelog entry
2022-09-20 15:37:18 +03:00
Uku Taht
669091f2ef
Ignore unknown country in imported data (#2247) 2022-09-20 15:02:14 +03:00
Uku Taht
b239f73a6d
Ignore unknown country code (#2223)
* Ignore unknown country code

* Add changelog entry
2022-09-16 11:02:39 +03:00
RobertJoonas
0194d57b38
add upper bound to limit parameter (#2226)
* add upper bound to limit parameter

* add more strict validation for limit parameter + 2 tests

* update changelog
2022-09-16 10:21:44 +03:00
Vinicius Brasil
a10d44a0d7
Refactor event struct creation function (#2098)
* Replace Ingestion.Request headers with user_agent

* Replace generic Ingestion.Request params with specific fields

* Refactor event building function into small functions

* Move Plausible.Ingestion to Plausible.Ingestion.Event

* Add option to override event fields while building

* Rename Ingestion.Request meta to props

* Replace UTM-specific fields with generic query_params

* Remove Map.from_struct/1 call from ingestion pipeline

* Remove stash options from ingestion
2022-08-16 14:43:10 +03:00
RobertJoonas
a058cf6240
added a test and changed hashmode condition (#2082) 2022-08-09 14:31:37 +03:00
Vinicius Brasil
b415ebe776
Fix geolocation subdivision pattern matching (#2063)
* Fix geolocation subdivision pattern matching

This commit fixes a bug where regions were not being saved. This was
caused because Geolix response was returning an additional
`:geolocation` map key. It also adds a test case for this.

Closes #2033

* Add geolocation database to .gitignore
2022-07-28 15:59:39 +03:00
Vinicius Brasil
b5ea6ae3dc
Keep user filter when listing cities, countries, and regions stats (#2030)
This commit fixes a bug where location filters were filtering stats but
not the locations list. This was caused by a `Map.put/3` call that
overrides the user filter. This commit rollbacks 5b57143273
changes and removes the `Map.put/3` call.

Closes #1982
2022-07-25 12:19:38 +03:00
Uku Taht
292a419473
Add feature flag to block event ingest (#1991)
Event ingest can be blocked using the flag `block_event_ingest`
2022-07-06 12:25:40 +03:00
Uku Taht
3e5695408a
Use new Session.CacheStore in favour of Session.Store (#1934)
* Remove Session.Store in favour of Session.CacheStore

* Add CHANGELOG entry

* Use appropriate enum function
2022-06-06 10:44:33 +03:00
Kurt McAlpine
b92889aaf0
Add API to retrieve site by domain (#1942) 2022-06-03 10:37:14 +03:00
Uku Taht
102ff1885e Trim trailing whitespace from pathname 2022-06-02 11:34:10 +03:00
Uku Taht
cd8df23e5e Do not rely on insertion order in tests 2022-05-06 10:20:35 +03:00
Uku Taht
e4816e8f64 Update Elixir version 2022-05-02 11:56:00 +03:00
Uku Taht
02abbda06d Ignore nil custom prop as well 2022-04-29 11:50:31 +03:00
Uku Taht
c92c548ca8 Do not error on empty string custom prop value 2022-04-29 11:13:36 +03:00
Uku Taht
5e415c2420 Add entry_props back in 2022-04-22 10:58:02 +03:00
Uku Taht
8fb4f3f886 Revert entry props 2022-04-21 19:22:38 +03:00
RobertJoonas
40275b64d4
Pageview custom dimensions (#1816)
* added custom dimension filtering tests for pages

* first filter UI in place

* pages, entry pages and exit pages can be filtered by pageview props

* added tests for expected filtering behaviour

* fix dimension filter for sources + tests

* added is_not filtering functionality

* fixed formatting

* fixed admin_test

* added (none) as filter value + is_not filter type in UI

* added prefilling applied filter values and some UI tweaks

* added fetch options

* Make prop suggestions work with `props` filter

* Fix test

* Track login state internally

* Add CHANGELOG entry

Co-authored-by: Uku Taht <uku.taht@gmail.com>
2022-04-21 11:47:15 +03:00
Uku Taht
66aceda9e9 Fix main graph selection with imported data 2022-04-19 10:50:50 +03:00
Andrea Mazzarella
1128ff4bfa
Update the internal /sites api to paginate results (#1824)
* Update the internal /sites api to paginate results and adapts site-switcher to it

* Update the Changelog

* Format internal controller

* Remove the `+ Add Site` link from the site-switcher in the dashboard

* Change camel to snake case and replace imports with fully qualified calls

* Remove trailing comma from site-switcher
2022-04-18 12:32:01 +03:00
Vignesh Joglekar
3b97ecdc62
Adds Main Graph Metric Selection (#1364)
* First pass bringing in previous graph improvements, and comparsion context

* Swaps issue template to new issue form syntax

* Indentation update

* Indentation update?

* More indentation

* Intendation is hard

* Finalized indentation?

* Github indentation

* Missing fields

* Formatting changes

* Checkbox changes

* Uses new timeseries API, various UI improvements, descopes conversions, ToP from graphing

* Fixes Mobile UI Issues

* Improves point detection and display on hover

* Fixes & adds tests for updated main-graph API route

* Changelog

* Changes to better metric option declaration & minor UI/default fixes

* Fixes top stat tooltips showing unformatted numbers for special (non-rounded) top stats

* Formatting

* Fixes regression with dashed portion not stopping at present_index

* Removes comparison + lint

* Improves top stat active style

* Removes comparison tests

* Splits out tooltip and top stats

Still needs:
- Tests
- Potentially more cleanup

* Adds/moves tests for top stats

* Formatting

* Updates metric LS key, removes console log

* Various fixes + cleanup

* Makes tooltip position & style more consistent

* Fixes test (returns import status on both main graph & top stats)

* Fixes interaction with month dateFormatter

* Fixes edge case tooltip behavior

It was simpler than I thought :/

* Make the entire top stat clickable

* Minor UI improvements

* Fixes another tooltip visibility edge case + cleans up boolean algebra

Co-authored-by: Uku Taht <Uku.taht@gmail.com>
2022-04-13 10:38:47 +03:00
Kian-Meng Ang
70cb3eed54
Fix typos (#1829) 2022-04-11 20:42:40 +03:00
Uku Taht
a359271af2 Fix more tests 2022-04-06 11:34:54 +03:00
Uku Taht
7f58e6be4e Add Year to Date option in datepicker 2022-03-31 13:52:48 +03:00
Uku Taht
8467979051 Add All Time option to datepicker 2022-03-31 11:00:20 +03:00
Andrea Mazzarella
3c93a2d91b
1421 contains filter (#1799)
* small refactors and adds 'contains' to modal

* supports contains filter in the backend

* moves entry and exit page under the page filter

* prettier

* updates the CHANGELOG

* undo package-lock changes

* fixes formatting for elixir

* renames unused parameter to _

* Update changelog

* Use uppercase for constants and update type/prefix lookup
2022-03-29 08:39:16 +03:00
RobertJoonas
e5cf800d4a
Improve stats api (#1759)
* can now query events metric when breaking down on a visit prop

* updated changelog

* fixed formatting

* fixed a bug - querying for bounce rate does not return visits anymore

* removed the changelog update
2022-03-24 16:16:51 +02:00