Commit Graph

2927 Commits

Author SHA1 Message Date
RobertJoonas
14cf6501c3
Refactor: Move from class to function components in React (#4198)
* Remove dead code (ComparisonContext)

The state of ComparisonContext is never read. We are storing all the
necessary state either in localStorage or in the `query` object.

* turn Filters into a fn comp

* turn Dashboard and CurrentVisitors into fn components

* turn Realtime into a fn comp

* Turn QueryLink into a fn comp

Also fix the "staticContext" console error when opening the period picker.
We don't need to pass all props into the Link component.

* extract a WRAPSTATE object
2024-06-10 13:03:39 +03:00
RobertJoonas
7b11b71dcf
Minor refactor: Remove skip_refresh_imported_opts from query modifying functions (#4204)
* Remove skip_refresh_imported_opts from query modifying functions

+ add test coverage for timeseries conversion rate.

* format

* improve comment
2024-06-10 13:03:12 +03:00
hq1
92d5092518
Verification improvements next batch (#4203)
* Upgrade req

* Fetch body even on non-200 statuses

Turns out sites, will happily go with 307 and no location
header for example. In which case saying we can't reach it
is an overstatement. After all we should let the headless
check determine the rest.

* Update callback status interpretation cases

* Update tests

* Fix flaky test

* Update expectation
2024-06-10 10:40:19 +02:00
ruslandoga
22ea56a779
increase api_keys.hourly_request_limit in CE (#4200)
* increase api_keys.hourly_request_limit in CE

* changelog

* add tests
2024-06-10 07:21:58 +02:00
hq1
823f7a8800
Plugins API: Get/Create Funnels (#4192)
* Bootstrap OpenAPI Funnel schemas

* Implement Plugins API Funnel view

* Allow casting funnel step directly from `%Goal{}`

* Check feature availability on funnel creation

just like it's done when inserting goals

* Implement Plugins API context module for Funnels

* Implement GET/PUT funnels via Plugins API

* Fix typo

* A rare event in which dialyzer found an actual bug, wow!

* Format

* Wrap creation request with a root `funnel` key

* Format

* Extract common funnel get query

* Remove redundant tag

* Refactor queries a bit
2024-06-07 13:10:01 +02:00
Cenk Kücük
62cd18e5e7
Add additional docker image tag (#4197)
* Update build-private-images-ghcr.yml
2024-06-06 16:49:08 +02:00
dependabot[bot]
a0ffda06f4
Bump actions/checkout from 2 to 4 (#4173)
Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-06 12:37:33 +02:00
RobertJoonas
b49bbbc8bb
fix with-imported-switch for safari (#4194) 2024-06-06 13:04:00 +03:00
RobertJoonas
2a529b4eaa
Fix with imported switch + warning bubble renders (#4187)
* prioritize other skip_imported_reasons over not requested

* return with_imported_switch info from top stats

* fix the with-imported-switch component

* Refactor ImportedQueryUnsupportedWarning

* do not render warning bubble when imported data not requested

* stop displaying warning bubble before the API response is received

* fix tab switch pills jumping due to bubble height

* drop loading condition of ImportedQueryUnsupportedWarning for Funnels

* add explicit check for realtime when rendering the bubble

* remove unused JS imports

* rename snake_case to camelCase
2024-06-06 11:50:54 +03:00
Adrian Gruntkowski
fd81ef2105
Include imported page paths in pageview goal suggestions (#4193) 2024-06-06 10:40:48 +02:00
ruslandoga
b096057eef
Release v2.1.1 (#4191) 2024-06-06 10:26:50 +02:00
ruslandoga
97e0255a2b
Wrap data migration using with_repo to ensure ClickhouseRepo is running (#4190)
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-06-06 09:39:14 +02:00
ruslandoga
ebabf75cfe
add missing changelog (#4186) 2024-06-05 14:08:19 +03:00
Adrian Gruntkowski
a764938928
Fix handling filtering by imported special properties (#4179)
* Handle custom prop queries separately when deciding imported table

* Add Query level tests
2024-06-04 13:06:31 +02:00
Karl-Aksel Puulmann
272fdb81d0
Fix parsing = in jsonurl (#4185)
* Fix parsing = in jsonurl

This works around a bug in the jsonurl library where the literal causes a syntaxerror.
The urlencoded = however is handled without errors

* Changelog
2024-06-04 13:06:19 +02:00
hq1
c22687172a
Remove redundant dot (#4178) 2024-06-03 14:54:18 +02:00
hq1
5fd4547382
More verification tweaks (#4177)
* Stop reporting multiple snippets in case manual extension is in use

* Try detecting CookieBot/GTM case
2024-06-03 13:49:04 +02:00
RobertJoonas
1d3b068233
Implement filtering for imported data (#4118)
* move imported.ex to imported subfolder

* move constructing base imported query into a separate module

* Implement imported table deciding and filtering

+ tests for pages, entry_pages, exit_pages and common filter types

* add top stats test with country filter

* add timeseries test

* Drop bounce_rate and time_on_page from imported & page-filtered Top Stats

* rename field returned by top stats

* turn pages into a fn comp

* Move dashboard API results under a results key

...and also return the skip_imported_reason to the frontend to be used
for displaying warnings.

* extend ListReport component with an optional afterFetchData prop

* turn Devices into a fn comp

* add not_requested as a skip_imported_reason

* display warning icons in the dashboard

* Implement filtering suggestions and translate filter fields for imported

* WIP

* Improve and cover filtering suggestions with tests

* Rename imported suggestions query helpers

* fix screen size breakdown with screen size filter

* support filtering by the same suggestion property

* support location filters when fetching location suggestions

* support filtering by multiple props from the same table

* Implement filtering by goals

* Make views per visit metric work for import entry and exit pages

* Get rid of circular dependencies between Stats.Imported and Stats.Imported.Base

* Clean up Query struct manipulation in Breakdown

* Rename helper function for clarity

* Automatically refresh query struct state after modifications

* Shutup credo

* display imported warning bubble in prop breakdown section

* Render warning bubble for funnels whenever imported data is in the view

* Transform any operator on respective goal filters

* Fix percentage and conversion_rate calculation in presence of custom props

* add tests for for combining page and pageview goal filters

* add skip_refresh option to query tweaking functions

* add imported CR support for timeseries

* still show url breakdown when special goal + url in filter

* rename Query.refresh

* use flat_map instead of map and concat

* fix darkmode color

* Handle invalid imported region codes in suggestions gracefully

* Add an entry to CHANGELOG.md

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-06-03 12:29:08 +02:00
ruslandoga
7cd9beac8f
Fix registration in ce (#4165) 2024-06-03 09:48:33 +02:00
ruslandoga
8588776df2
Add newline to csv importer insert sql (#4172)
* add newline to csv importer insert sql

* add test

* changelog

* fix test typo

* fix test typo x2
2024-06-03 09:48:16 +02:00
ruslandoga
cb074d835b
Fix VersionedSessions migration (#4162) 2024-06-03 09:47:44 +02:00
hq1
12ddec6b77
Minor verification improvements (#4175)
* Handle long domains in verification recommendations

* Accept 200 callback status from WP sites
2024-06-03 09:24:52 +02:00
RobertJoonas
57e28f0da6
Make React frontend capable of reading API results under a results key (#4159)
* Make React frontend capable of reading API results under a results key

* remove redundant double negation

* Update assets/js/dashboard/stats/modals/props.js

add length key back

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-05-31 09:01:15 +01:00
ruslandoga
dc7243ff2e
Populate site_imports from sites.imported_data in CE (#4155)
* populate site_imports from sites.imported_data in CE

* not dry_run?

* Wrap data migration using `with_repo` to ensure ClickhouseRepo is running

* Do not create funnels in seeds when running CE

* Remove migrator wrapper

* Explicitly raise on rollback attempt

* Put a warning in data migration scripts which are put in migrations

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-05-29 17:37:21 +02:00
ruslandoga
55dfe4cea7
Make Plausible.ce? and Plausible.ee? into functions (#4158)
* make Plausible.ce? and Plausible.ee? into functions

* Fix phash and predicate order to make dialyzer trick working again

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-05-29 14:55:25 +02:00
ruslandoga
79349730a0
update mua (#4157) 2024-05-29 08:51:46 +02:00
hq1
bb7b0d9f94
Tidy up verification code a little (#4156) 2024-05-29 08:51:10 +02:00
hq1
9d77e970ef
Redo site verification UI (#4153)
* Redo site verification UI

* format
2024-05-28 13:43:14 +02:00
Karl-Aksel Puulmann
e5c3eabe31
Fix linking in dashboard sources/utm details modal (#4151)
This was broken by a recent refactor and the links did not filter properly
2024-05-28 13:28:56 +02:00
hq1
d9cf037866
Handle no snippet WordPress case (#4152) 2024-05-28 09:37:01 +02:00
RobertJoonas
4d7a67d294
Fix setting with_imported to false explicitly in the URL (#4148) 2024-05-27 15:27:23 +02:00
hq1
78c5762761
Verification: prioritize domain mismatch over multiple snippets (#4146) 2024-05-27 10:52:22 +02:00
hq1
0381bc6121
Use codespell (#4145)
* Fix typos

* Add .codespellignore

* Add codespell GH action
2024-05-27 10:52:01 +02:00
hq1
c15ce1fa15
Include docs anchors in diagnostic message links (#4144)
* Include docs anchors in diagnostic message links

* format

* One more
2024-05-27 08:49:42 +02:00
ruslandoga
2431d785a6
Remove duplicate clause in filter parser (#4137) 2024-05-24 16:28:52 +02:00
ruslandoga
a139e8295c
Fix warning in migration (#4133) 2024-05-24 16:26:25 +02:00
hq1
206a6543f6
First pass: extract verification errors (#4138)
* First pass: extract verification errors

We like to make them easier to maintain.
An update fixing documentation URLs will follow.

* Update moduledoc
2024-05-24 16:23:25 +02:00
hq1
a7b69719c1
More fixes to the verification process (#4136)
* More fixes to the verification process:
 - extend the list of known attributes
 - follow redirect chain host to verify data-domain difference
 - display the actual data-domain in error message, not URL

* format

* fix test

* remove redundant pattern matching

* Update lib/plausible/verification/checks/snippet.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-05-24 12:45:31 +02:00
hq1
ee61094023
Verification improvements (#4132)
* Simplify browserless script and catch errors

* Improve fetching body:
    - follow up on max. 4 redirects
    - rely on Req default timeouts (wait much longer to account for slow
      sites)

* Improve installation check:
  - rely on Req default timeouts, wait longer
  - log service errors as warnings
  - use stealth mode to bypass captchas

* Stop cutting off body too large

* Improve diagnostics for known failures

* Another round of diagnostic improvements

* Format

* Add a test for callback status 500
2024-05-24 10:40:59 +02:00
hq1
c81cb16933
Snippet integration verification (#4106)
* Allow running browserless.io locally

* Compile tailwind classes based on extra/ too

* Add browserless runtime configuration

* Ignore verification events on ingestion

* Improve extracting HTML text in tests

* Update dependencies

- Floki will be used on production to parse site contents
- Req will be used to handle redundant stuff like retrying etc.

* Add shuttle SVG to generic components

Later on we'll use it to indicate verification errors

* Connect live socket & allow skipping awaiting the first pageview

* Connect live socket in general settings

* Implement verification checks & diagnostics

* Stub remote services with Req for testing

* Change snippet screen copy

* Update tracker script, so that:

1. headless browsers aren't ignored if `window.__plausible` is defined
2. callback optionally supplies the event response HTTP status

This will be later used to check whether the server acknowledged
the verification event.

* Implement LiveView verification UI

* Embed the verification UIs into settings and onboarding

* Implement browserless puppeteer verification script

It:
 - tries to visit the site
 - defines window.__plausible, so the tracker doesn't ignore test events
 - sends a verification event and instruments the callback
 - awaits the callback to fire and returns the result

* Improve diagnostics for CSP

Only report CSP error if the snippet is already found

* Put verification behind a feature flag/env setting

* Contact Us hint only for Enterprise Edition

* For headless code, use JS context instead of EEx interpolation

* Update diagnostics test with WordPress scenarios

* Shorten exception/throw interception

* Rename test

* Tidy up

* Bust URL always on headless check

* Update moduledoc

* Detect official Plausible WordPress Plugin

and act accordingly on diagnostics interoperation

* Stop using 'rating' in favour of 'interpretation'

* Only report CSP error if no proxy is likely

* Update CHANGELOG

* Allow event-* attributes on snippet elements

* Improve naive GTM detection, not to confuse it with GA4

* Update lib/plausible/verification.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Update test/plausible/site/verification/checks_test.exs

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* s/perform_wrapped/perform_safe

* Update lib/plausible/verification/checks/installation.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Remove garbage

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-05-23 15:00:50 +02:00
ruslandoga
5881f1c1bf
release v2.1.0 (#4124) 2024-05-23 10:00:41 +02:00
Karl-Aksel Puulmann
850cc30e7a
Fix opening a filter modal (#4123)
Follow-up to #4117

Opening a utm modal crashed if there was an existing page filter. Bug
struck in during a refactoring.
2024-05-23 10:54:08 +03:00
Karl-Aksel Puulmann
ab1416ba10
Fix special goal conversions (#4122)
Follow-up to https://github.com/plausible/analytics/pull/4117

Special goals (e.g. 404, Outbound link click) always reported (none) as
results due to a function returning the wrong value.
2024-05-23 10:53:54 +03:00
ruslandoga
9687fa786d
remove Plausible Analytics metions from CE (#4121)
* remove Plausible Analytics metions from CE

* update tests

* still mention Plausible Analytics on landing page
2024-05-23 09:43:01 +02:00
Karl-Aksel Puulmann
23a6431adb
Frontend: Use jsonurl, update filtering (#4117)
* Start refactoring, supporting jsonurl

* WIP: Listing of applied filters works

* WIP: Remove some unneeded hackery

* WIP: Handle country labels better

* WIP: Handle multiple filters better

* WIP: Serializing filters for API queries

* Split modal code into 3

* Merge code paths for prop and other filter modals

* Get suggestions working again

* Display prop filters properly

* Handle re-opening a props modal properly

* Better label handling

* Better linking to filter modals

* Remove unneeded component

* Standardize how we update query more

* Use updatedQuery to remove more usecases of URLSearchParams

* Dont export toFilterQuery

* Custom toString for PlausibleSearchParams

* Fix props suggestions/filtering

* Refactor isFilteringOnFixedValue

* Improved encoding - goals now work again

* fix a typo

* Handle more cases where query.filters[ is used

* Fix locations tab changing behavior

* Fix for `setQuery` not to double up ?

* Handle goal filters properly now

* Delete dead code

* Update special goals handling

* Update <ListReport /> linking

* Show labeled values in list of filters

* Updae Props component handling of storage keys

* re-add special case handling in devices view

* Fix modal-related typo

* Get updatedQuery callsites working

* Update location modals linking

* Update props details model linking logic

* Switch back tab from props when removing goal filter

* Remove query.filters usage from within <Referrers /> component

* Private escapeFilterValue

* Fix sources/index.js

* Legacy redirect logic

* Update comment

* Disabled options in props modal

* Update escaping and is_not operator

* Restore `false` search property handling meaning unset

* changelog

* Fix filtering after clicking on a map

* FilterOperatorSelector

* replaceFilterByPrefix

* Improve naming for filter modals/groups
2024-05-22 11:01:41 +03:00
hq1
cd4d1d06bb
Rename Plugins API Token(s) to Plugin Token(s) (#4075)
Deployment needs to be coordinated with WordPress plugin
update. Reasoning: there's too much confusion coming
from the use of the "API" terminology, leading to wrong
keys being generated by users.
2024-05-22 07:45:09 +02:00
Adrian Gruntkowski
2d23db2982
Return warning when imported stats are skipped in Stats API due to unsupported query (#4116) 2024-05-21 11:45:18 +02:00
Adrian Gruntkowski
7876802d05
Improve CSV importer tests and visit duration query for imported data (#4114)
* Tidy up breakdown helper functions in CSV importer tests

* Fix a typo

* Extend city breakdown tests in CSV importer test suite

* Make visit duration computation consistent across native and imported data
2024-05-21 09:24:10 +02:00
Uku Taht
b2860d1602
Fix unselecting filter from filter modal (#4109) 2024-05-21 09:23:57 +02:00
Adrian Gruntkowski
756d9c28ce
Improve test coverage and fix stats API and dashboard CSV export (#4110)
* Fix broken hostname property and handle missing imported metrics gracefully

* Add test for CSV export of imported data

* Add extra coverage for property and metric combox which were failing

* Compute visit duration and bounce rate for exit pages in imported data

* Drop support for breaking down by `event:hostname` property for now
2024-05-20 16:11:35 +01:00