* Remove `Plausible.Sites.set_allowed_event_props/2` function
This commit removes the `Plausible.Sites.set_allowed_event_props/2`
function in favor of `Plausible.Props.allow/2`.
* Allow internal prop keys in queries
This commit allows internal prop keys by default in queries. I decided
not to include internal keys in the database, because they might change
and we'd need to migrate it again.
* remove redundant 'props.' from behaviours/index.js
---------
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
This commit removes the allowed props list from the data attribute
passed to React. The list was used for checking whether props configured
or not. This commit adds a new `hasProps` attribute, and removes the
list.
* fix conversions fetchData limit to the right number
* make Goal Conversions metric column width match Properties
* add more top-margin to Details links in ListReport
* add special handling for url prop in Conversions
* change 'Details' button to 'More' when no additional details
* fix CI
* breakdown by 'path' prop instead for 404 pages
* change the 'MORE' button back to 'DETAILS'
But keep the option of not showing the button when there are less
than 9 list entries.
* capture special props in an object instead (review suggestion)
* Add cloaked link to special goals list
---------
Co-authored-by: Vinicius Brasil <vini@hey.com>
* rename conversions.js to deprecated-conversions.js
* add a new Conversions component and switch by props FF
* add a Details view to the new Conversions component
* allow querying conversions with pagination
...and keep the current behaviour for DeprecatedConversions, always returning
page=1 and limit=100
* hide some columns on mobile
* prevent ListReport first column header overflow
* extract add_exit_rate function
* Change internal API metric names for /entry-pages & /exit-pages
* `unique_entrances` -> `visitors`,
* `total_entrances` -> `visits`,
* `unique_exits` -> `visitors`,
* `total_exits` -> `visits`,
This is just a consistency improvement - the `visitors` metric always means
one thing and there's no need to call it different for entry pages internally.
This commit does not change any noticable behavior. The UI labels are kept
the same and the column headers in the CSV export will also remain the same.
* Change internal API metric names for /conversions
* `unique_conversions` -> `visitors`,
* `total_conversions` -> `events`,
* return revenue metrics from /custom-prop-values (backend)
* be more explicit about which metric is plotted with Bar
* validate that ListReport input metrics actually exist in the API response
* display revenue metrics in the dashboard Properties section
* limit the number of columns shown on mobile
* add revenue metrics to Properties > Details
* review suggestions
* define hiddenOnMobile per metric instead of keeping the first 3
* rewrite if-else block
---------
Co-authored-by: Vini Brasil <vini@hey.com>
* extract add_exit_rate function
* Change internal API metric names for /entry-pages & /exit-pages
* `unique_entrances` -> `visitors`,
* `total_entrances` -> `visits`,
* `unique_exits` -> `visitors`,
* `total_exits` -> `visits`,
This is just a consistency improvement - the `visitors` metric always means
one thing and there's no need to call it different for entry pages internally.
This commit does not change any noticable behavior. The UI labels are kept
the same and the column headers in the CSV export will also remain the same.
* Change internal API metric names for /conversions
* `unique_conversions` -> `visitors`,
* `total_conversions` -> `events`,
* make (none) value in custom prop breakdown add +1 to pagination limit
This is needed because the (none) value is always added to the
breakdown_results **after** fetching those from ClickHouse.
* only include (none) values on the first page of results
* fix percentage metric calculation for paginated results
Instead of summing up the number of visitors from the breakdown results
to get the total, we have to make a separate query to `Stats.aggregate`.
Otherwise, the percentages for each results page will wrongly add up to 100%.
Since imported data for aggregated visitors and other properties (such as
browsers, OSs, etc) live in different tables, we have to tweak the tests to
also include the same number of visitors in the `imported_visitors` table.
* add details view for props
* changelog
* exclude imported data from total
* Add creatable option to ComboBox
This commit changes the ComboBox component to allow a `creatable`
option. This option enables users to create new options along with
choosing existing options.
* Test ComboBox class parameter
* Use display_value instead of input
* Change scroll block to nearest to prevent glitches
* add the props section in behaviors
* update listReport when keyLabel (=propKey) changes
* make column min-width configurable and increase for props
* add rendering condition to limit container height
* fix filter link
* fix tests
* disable clear for single-option combobox
* improve single-option combobox styling
* fix fetchPropKeyOptions fn update on query change
* BUGFIX: searching for prop_values in property filter modal
* change the order of funnels and props section pickers
* change props section Bar color from gray to light-red
* remove disabled options from combobox dropdown (multi & single)
* display percentage metric values without a % sign
* change metric labels in goal filter view to Visitors and Events
* fix realtime update timer
* refactor SourceList to use ListReport
* refactor SourceList into a fn comp
* change referrer-drilldown API response format and remove dead code
* use ListReport in referrer-list
* fix CI
* fix flaky test
* remove IO.inspect
* Move money cast function to Stats.Util
* Add revenue metrics to main graph API
* Plot revenue metrics on the main graph
* Return plain numbers instead of a map
* Fix Credo issues
* Fix canMetricBeGraphed function
* Revert canMetricBeGraphed function changes
* refactor ListReport
Pass a 'getFilterFor' function instead of a 'filter' object to ListReport
* Keep the statsBoxClass in one place only
* add classname prop to MoreLink
* define metric structs as ListReport inputs
* Fix a bug
If the query changes, we also want to reset the eventListener function.
Otherwise we keep calling an outdated function that fetches old data.
* fix CI
* PoC: evaluate funnel on step selection
* Adjust test to actually _select_ steps
* Throttle and test ephemeral funnel evaluation
* Format compact percentages server-side
At this point it's pointless to duplicate this;
all the primitives exist in the API in case the client
wants a different representation.
* Dynamically set :domain for the live socket
Ref: https://github.com/phoenixframework/phoenix_live_view/pull/2715
* Make runtime config raise before the user ends up in a reconnect loop
* hall of shame: remove console.info remnant
* Check origin on live websocket
* Get rid of single pipe
* Add Funnel react component
assets/js/dashboard/stats/behaviours/funnel.js - restored from:
98a76cbd Remove console.info calls
d94db99d Convert Funnel class component into a functional one
028036ad Review comments
3067a940 Stop doing maths in react
73407cc3 Fix error handling when local storage gets corrupted
e8c6fc52 Format numbers on funnel labels
c815709f Reorganize component responsibility
7a88fe44 Outline basic error handling
94caed7c Chart styling updates
4514608a Add percentages to funnel
d622c32d Add funnel picker
Co-authored-by: Uku Taht <uku.taht@gmail.com>
* Pass funnels list to react via data-funnels
* Implement Funnels react API
lib/plausible_web/controllers/api/stats_controller.ex - restored from:
f36ad234 Adjust to Plausible.Stats interface
9b532273 Test funnel stats controller
028036ad Review comments
bea3725f Remove IO.inspect
7a88fe44 Outline basic error handling
c8ae3eaf Move Funnels to StatsController and use base query
667cf222 Put private functions at the bottom
* Tweak funnel presentation
* Handle errors at the top
* Do not register DataLabels plugin globally
or else all the existing charts are affected
* Calculate drop-off percentage evaluating funnels
* Tweak dark mode + implement nicer tooltips
* Make currently selected funnel bold in the picker
* Count user_ids not session_ids when evaluating funnels
So if a visitor goes:
1. Start session
2. Complete funnel step 1
3. Inactive for 30 minutes
4. Complete funnel step 2
We would not be able to track this funnel completion because of the session timeout.
We like to o measure this as funnel completion even though the session expired in the middle.
cc @ukutaht
* Add extra properties to the funnels API
cc @ukutaht
* Improve tooltips so that step to data is rendered
* Change tooltip number formatting
* Remove debugging remnants
* Quick & dirty mobile view
* Fix mobile view: tweak dark mode & funnel switching
* Ignore DOMException: the operation was aborted
Otherwise this sometimes flashes the space shuttle
screen when navigating quickly via a keyboard.
* Format percentages on the main chart
* Close missing tag 🙈
* Revert "Close missing tag 🙈"
This reverts commit 9c2f970e22fd7e2980503242b414f42ce8bce1d2.
* Use jsx to render funnel tooltip
To get markup validated via lsp mostly...
* Fixup: s/class/className
* Fix className interpolation
* Add a ruler to the tooltip
* Tweak funnel chart style
* Fix font distortion issue on chart/canvas labels
* s/class/className
* Put "Set up funnels" link behind a feature flag
* Refactor internal selection storage
Getting ready for live funnel evaluation
* Don't try to connect LV socket if there's no CRSF token set up
This is perfectly okay for some of the templates/layouts.
* Fix up funnel creation typespecs
Unfortunately we can't define a type with literal string keys,
hence this must suffice.
* Use uniq over count/distinct
* Revert JSX in tooltips
Ref: https://github.com/plausible/analytics/pull/3066#discussion_r1241891155
* Remove the extra query for counting all visitors
cc @ukutaht
* Add premium notice
---------
Co-authored-by: Uku Taht <uku.taht@gmail.com>
This commit adds revenue data to top stats. Average and total are displayed when filtering by a revenue goal (or many if they have the same currency set).
* Update formatter config
* Install LiveView JS integration & hooks
* Temporarily update endpoint/session config
* Optionally allow preloading funnels for goals
* Site controller
* Implement funnel settings
lib/plausible_web/live/funnel_settings/combo_box.ex - restored from:
054de6e2 Fix the tab/blur bug again
20da4c89 Rename InputPicker to ComboBox
lib/plausible_web/live/funnel_settings/form.ex - restored from:
9bedda3b Remove potential FIXME
20da4c89 Rename InputPicker to ComboBox
028036ad Review comments
aea4ebc4 Access Funnel min/max steps via the __using__/1 macro
0dde27fd Remove inspect call
eed588a7 Start testing the funnel editor
0e95228b Extract funnel settings test module
7b16ace5 Leverage aplinejs to deal with the tyranny
8dc6a3e7 wip
cf228630 wip
30a43fd1 wip
89f10ecb wip
950a18d9 Dirty funnel save
298a6a53 wip
7690d50f wip
639c6238 fixup
aa59adeb wip
ff75c00b wip
lib/plausible_web/live/funnel_settings/list.ex - restored from:
4eae122c Fix data-confirm attr interpolation
51f0397d Implement deleting funnels
1f6fe25d Add number of steps to funnels list
298a6a53 wip
ff75c00b wip
test/plausible_web/live/funnel_settings/funnel_settings/combo_box_test.exs - restored from:
20da4c89 Rename InputPicker to ComboBox
test/plausible_web/live/funnel_settings/funnel_settings_test.exs - restored from:
34822ff4 Bootstrap InputPicker tests
lib/plausible_web/live/funnel_settings.ex - restored from:
028036ad Review comments
acd9c4f2 Prepare ephemeral funnel definitions so that users can test funnels
51f0397d Implement deleting funnels
0e95228b Extract funnel settings test module
8dc6a3e7 wip
89f10ecb wip
950a18d9 Dirty funnel save
298a6a53 wip
aa59adeb wip
ff75c00b wip
test/plausible_web/controllers/error_report_controller_test.exs - restored from:
34822ff4 Bootstrap InputPicker tests
test/support/html.ex - restored from:
0a53979d Improve InputPicker tests - include AlpineJS assertions
34822ff4 Bootstrap InputPicker tests
lib/plausible_web/views/layout_view.ex - restored from:
b490403b !ifxup
lib/plausible_web/templates/site/settings_funnels.html.eex - restored from:
51f0397d Implement deleting funnels
ea1315f3 Test funnels list in settings
7b16ace5 Leverage aplinejs to deal with the tyranny
ff75c00b wip
4da25c35 Fixup
lib/plausible_web/templates/layout/app.html.eex - restored from:
ff75c00b wip
* Add funnel settings route
* Warn about funnels deletion when deleting goals
lib/plausible_web/templates/site/settings_goals.html.eex - restored from:
fdd9bcd0 Fixup
f1e6364d Merge remote-tracking branch 'origin/master' into funnels-rebase
9d0b7c6d Fix markup error
4a4ddbdc Optionally preload funnels for goals and stub funnel-goal deletion
ebdc4333 Extend the prompt in case of funnel-goal deletion
639c6238 fixup
aa59adeb wip
* Split new JS LiveView additions
* Put funnels behind a feature flag
* Integrate dashboard feature toggle
* Update signing salt for live view
* Update moduledocs
* Update live reloader config
* Use Phoenix.HTML.Safe for goal names
* Workaround to get flashes working in embedded liveview
* Keep feature toggles idempotent, rename property to setting
We'll still retain the ability to flip bools on a lower level.
* Update moduledocs
* Make live flash disappear after 5s
* Tailwind: purge .heex files too
* Update docs link
* Add live components to tailwind purge config
* Update another flaky test
Ref f0bdf872
cc @vinibrsl
* Fix combobox input length w/ WebKit
* Intoduce generic notice component
* Revert "Fix combobox input length w/ WebKit"
This reverts commit 3c653a6d85d5000167631e10ef45a93c13b41ed1.
* Fix combobox input length on webkit
* Make whole combobox item clickable, not only text
* Fix glitch moving Save button on activation
* Tweak dark mode
* Show funnel form without waiting for funnel name input
* Tweak dark mode
* Include static Phoenix components in tailwind purge
* Tune funnels form into a liveview of its own
This is so that ComboBoxes can publish their selections
and unavailable choices can be propagated to other siblings.
* Push less data over websocket
* Undo Lsp/formatter race condition
* Fixup typespecs
* Bust CI cache
* copy relevant files from b2ace16540
* make it work and set site.funnels to empty list
* make Behaviours a functional component
* add UI for a setup hint and always display conversions by default
* cherry-pick migration commit
* update site schema with new fields
* backend: implement disable-feature action
* switch between tabs in the behaviors section
Introduces template components to build props and funnels on. Both
only show a setup notice atm, and both are behind feature flags.
* extend API for disabling props and funnels
* render feature setup note directly from the Behaviours component
* fix UI behavior when features are hidden
* update setup notices
* add conversions feature switch to Site Settings > Goals
* mix format
* remove IO.inspect
* change setup notice - use buttons + popup confirmation
* optimize for light mode
* restrict access to setup notices
* some styling improvements
* allow super-admins to enable/disable features
* only show conversions (last 30min) in realtime mode
* use shorter display names for tabs
* optimize for mobile screens
* note about sending custom events
* changelog update + fix CI
* change HTTP verb for the disable-feature action
* change UI label for show/hide goals
This commit brings back percentage arrows to default view. These were
removed by the recent comparisons work, and we've had reports of people
missing it. As we haven't noticed any difference in performance, I've
decided to revert that change.
This commit updates the imported 'G' icon, so that it appears only when
querying or comparing a range with imported data. By hiding the icon in
all other cases, users will see it only when they can actually click on
it.
Previously, the comparison line could sometimes be displayed with a
different scale than the main plot due to the maxTicksLimit option not
being fixed and changing based on the data. This commit fixes the issue
by setting the maxTicksLimit value to 8 in the graph settings instead of
for each individual axis.
Additionally, a callback that attempted to dynamically change the Y axis
ticks while resizing has been removed. As the graph has a fixed height,
dynamically resizing the Y axis no longer makes sense.
* Add PropFilterModal (only UI)
* small variable refactor
* allow selecting prop value filter type
* allow selecting only one prop_key
* allow selecting many prop_values only when prop_key selected
* handle submitting filter
* get applied filters from query + remove option
* change prop filter label format
* support member and not_member filter types for pageview props
* show (none) value in filter suggestions
* refactor zip_results/4 and remove unused code
* fix displaying (none) values in goals section prop breakdown
* remove unnecessary functionality
* fix bug: returning prop names for goal :member filter
* fix bug: submitting regular filter modal with Enter key
* bugfix: disallow opening prop filter modal when feature flag disabled
* mix format
* break selected combobox values into multiple lines
* fix useEffect behavior for focusing on prop_key field
* support submitting prop filter with Enter key
* refactor getFormState in PropFilterModal
* separate fetchPropKey and fetchPropValue functions
* Allow querying props for pageview goals
* Make the internal props API only return a list of props (not map)
* Separate function for fetching all props in Stats API goal breakdown (this returns a map as before)
* ditch state for keeping search bar visible
* group by event_name in db query
This pull request removes the comparisons feature flag, making it visible to all users. It also removes percentages from top stats in default view, leaving those for the comparison view only.
* Fix date picker shifting day bug
This commit fixes a bug where picking a specific date in custom range
takes you to the stats of the day before.
This was caused because Flatpickr returns a Date instance, and we were
trying to parse this Date instance into day.js using the `utc` function,
assuming it was a ISO-8601 string.
This commit fixes it by casting Date into dayjs using the `dayjs`
function instead of `utc`.
* Fix invalid date Flatpickr bug
This commit fixes a bug where the Flatpickr was not setting the minDate
correctly based on statsBegin. This was failing because parseUTCDate is
now returning a day.js instance, and Flatpickr cannot handle that type.
* Hide dayjs usage in util/date
* Pass comparison_query as argument to top stats functions
* Add comparison values and dates to top stats API
* Display comparison dates and values in top stats
* Rename function to renderPercentageComparison
* Create component to do conditional rendering
* DRY date range formatting function
---------
Co-authored-by: Uku Taht <uku.taht@gmail.com>
* Use dayjs with UTC plugin for date-formatter
* Remove 'toHuman' function
* Use dayjs dates in Datepicker component
* Use util/date.js functions in date formatter
* Remove `fromJSDate`
* Use formatISO instead of raw dayjs formatter
* Escape pipes in goal names
* Apply suggestions from code review
Use ~S sigil for better readability
Co-authored-by: Vini Brasil <vini@hey.com>
---------
Co-authored-by: Vini Brasil <vini@hey.com>
* Fix open state in combobox
* Improve filter suggestions when using the `contains` filter
* Support contains filter type for utm filter group
* Remove console.log
This commit improves the datepicker state management. Due to a bug, we
needed to use setTimeout to open Flatpickr. This change replaces the
setTimeout workaround by using a proper React state.
* Wrap Plausible.Stats.Filters with unit tests
* Parse `member` filter type
* Support for `member` filter in `aggregate_time_on_page`
* Support `not_member` filter type
* Support `matches_member` and `not_matches_member` filters
* Extract util module for React filters
* Implement Combobox from scratch with no libs
* Support multple filter clauses in combobox
* Don't use browser / os in version label
* Show highlighted option in combobox
* WIP
* Fix location filters outside filter modal
* Align open/close behaviour with react-select
* Styling updates for combobox
* Add support for wildcards in Combobox
* Implement keybindings for combobox
* Allow free choice inputs in combobox
* Rename 'Save filter' -> Apply filter
* Remove TODO comment
* Clean up some rebase mistakes
* Rename `allowWildcard` -> `freeChoice`
* Dark mode fixes
* Remove hint from filter modal
* Escape pipe character in filter modal
* Do not allow selecting duplicate options in combobox
* Escape brackets in `page_regex/1`
* Fix disabled style in dark mode
* Add regex fallback for safari
* Show no matches found when visibleOptions is empty
* Disable enter key when no visible options
* Do not submit empty form fields
* Remove unnecessary setOpen(true)
This pull request enhances the navigation for comparisons by implementing the following changes:
* Hides the comparison input behind the period picker, making the top bar less busy.
* Adds a key binding for the comparison input (assigned to x) alongside arrow navigation.
* Persists the selected comparison mode in Local Storage by domain, resulting in its persistence between refreshes.
This pull request adds support for multiple comparison modes, changes the comparison checkbox to a combobox, and implements the year over year comparison mode. The feature is still behind a feature flag.
Co-authored-by: Uku Taht <uku.taht@gmail.com>
* globally rename 'pages_per_visit' to 'views_per_visit'
* change the order of top stats
* rename 'Visits' to 'Total visits' in the UI
* add views_per_visit to UI
* put the new metric under a feature flag
* add new metric to CSV export under feature flag
* mix format
* use only one feature flag
* Use user-agent instead of screen_width to get device type
Co-authored-by: eriknakata <erik.nakata5@gmail.com>
* Fix credo
* Log on unhandled UAInspector device type
* Make 'browser' the default tab in devices report
* Remove device tooltip
* Remove screen_width from ingestion completely
* Remove browserstack harness, run playwright directly
* Select meta key based on OS platform
* Run CI tests in parallel
* Improve device match readability
* Add changelog
---------
Co-authored-by: eriknakata <erik.nakata5@gmail.com>
* Add visits metric and make it graphable
* include visits metric in csv export (visitors.csv)
* put visits under a feature flag (CSV export)
* feature flag for displaying visits on the dashboard
* fix formatting
* add visits metric to top stats (fix)
* fix imported_test to expect visits metric included
* fix formatting
* Revert "Fix graph element z-index (#2698)"
This reverts commit 35ab26308d.
* Revert "Fix interval selector z-index (#2696)"
This reverts commit c307057386.
* Set graph z-index to zero
* Add as={Fragment} to Transition component
Previously, the graph element was overlapping with other UI elements
such as the period input. This commit fixes this issue by setting the
z-index of the graph element to 0, which moves it behind other elements
in the stacking order.
* Refactor graph dataset building function
This commit refactors the graph dataset building functions into three
separate functions, one for each plot: main plot, comparison plot, and
dashed plot.
* Merge default graph dataset options
This commit moves default options from graph building functions to the
main function.
* Remove truncateToPresentIndex function
* Include dashed part when comparing periods
This commit fixes a bug where some period + interval combinations were
not working properly with comparisons. This is because `present_index`
is not available in every period, e.g. Last Month.
* Use HeadlessUI for search select box
* Remove downshift from package.json
* More consistent API for Combobox component
* Combine toFilterType and valueWithoutPrefix into a single function
* Rename MyCombobox -> PlausibleCombobox
* Update webpack-cli
* Disable cache for build
* Revert "Disable cache for build"
This reverts commit aa130541f8.
* Disable cache for build
* Update webpack dependencies
* Remove glob from webpack config
* Webpack is required by package.json
* Require autoprefixer in postcss config
* Revert build changes
* Fix styling for dark mode
* Position the graph component relatively
* lose graph container height flicker (resized canvas)
* rename variable
* prevent flicker in the loading process
* remove unused import
* disable cursor-pointer class for minute interval
* change var names and loading state conditions
* refactor showLoader condition
* fix UI bug with interval change
set `graphData` to `null` so that the component would fade out when
loader becomes visible. Before, the spinner was just displayed on top of
the graph.
* Add year to X axis of multi-year graph
* Remove hardcoded condition for rendering year string, render if multiple years present in graph view
* Apply new argument contract to dateFormatter usage in graph-util.js
* Better defensive runtime type guard when deriving hasMultipleYears
* Remove console log
* Remove unrelated JSDoc change
---------
Co-authored-by: Vini Brasil <vini@hey.com>
This commit adds support for comparing the actual showed period on the main graph with the previous one. This is a first pass and it's hidden under a feature flag because it's not feature complete yet as we want to support other comparison modes.
* Make Device section components aware of (not set)
So that no extra sub-filters are possible when the unset
top item is selected.
* Support '(not set)' in breakdown/filters
* Update expectations for export tests
* Add extra tests for returning/filtering by '(not set)'
* Add changelog entry
* Remove ListReport conditional render
* Prevent redundant sub-filters
* Fix filter text rendering
---------
Co-authored-by: Uku Taht <uku.taht@gmail.com>
* Remove the ability to collpase the top graph
This commit removes the ability to collapse the top graph. The graph
collapsed whenever `metric` was falsy. I removed all related code to
that. Metric now defaults to visitors.
We want to add new items to top stats, and this commit will make it
easier to change it. Also, there's currently a bug where top stats is
randomly collapsing, which should be fixed by this commit.
* Refactor graph and top stats loading state
The graph loading state shows and hides the graph conditionally
depending on whether the data is loaded, loading or refreshing.
The current code is a bit difficult to read because its big
conditionals. This commit refactors the loading state making it easier
to read.
This commit also fixes a bug where the graph wasn't fading out when
changing metrics.
* Move Endpoint errors setup to common config
* Implement naive Sentry link resolver
* Implement error report e-mail
* Delete static sentry script
* Implement user feedback form on server errors
* Re-arrange pipe
* Use Sentry.Config.dsn() where applicable
* Fix typo
* Use Map.replace/3
* extract maybeUpdateMetric function
* reset graph metric after fetching top stats
* fix bug
Also make sure we do not allow 'metric' to become undefined anymore
* keep graph hidden when empty metric saved
* reset metric in setState callback
* pass function not call
* extract blinkingDot function
* position pulsating-circle with tailwind instead
* remove unused function
* extract renderStatName function
* display seconds since last realtime update
Adds a 'Last updated X seconds ago' label to the Current Visitors tooltip.
* small refactor: avoid duplication of this.props and this.state
* show the 'last updated ...' tooltip in historical
* changelog update
* use className utility function
* add a new realtime-update-timer module
* hook to the new 'tick' event in ListReport for auto-updates
This commit fixes the bug where all reports using the `ListReport` component did not
auto-update in realtime mode. Those reports are:
- Pages (Top / Entry / Exit)
- Locations (Countries / Regions / Cities)
- Devices (Screen Sizes / Browsers + versions / OS-s + versions)
* fetch data for ListReports only when scrolled into view
* refactor fetching data in ListReport
* refer to one source of truth for utm tags
* make the 'All' tab in Sources auto-update
* make all UTM tabs in Sources auto-update
* fetch UTM data only when scrolled into view
* auto-update Referrers with the new timer
* auto-update google search terms
* auto-update Conversions
* make countries map auto-update
* auto-update visitor-graph and top stats with new timer
* use new tick event for current visitors (in Historical)
* remove the old timer class
* update changelog
* Visual improvements to automatic realtime updates (#2532)
* minor consistency fix for text color in dark mode
* use FlipMove in goal conversions report
* use FlipMove in ListReports
* set main graph and top stats loading state correctly
* refactor isIntervalValid function
* enforce intervals are valid when set and stored
* remove duplicate data fetching on interval change
Fetching new data is handled by the `fetchGraphData` callback in `updateInterval`
* refactor updateMetric function
* make it clearer why 'metric' can be a faulty value
* extract 'query' and 'site' variables from 'this.props'
* reset interval state only when period is changed
The 'maybeRollbackInterval' function was also used to fetch data. This commit replaces
all those function calls with 'fetchGraphData' which better describes the actual behavior.
We should only worry about rolling back the interval if 'query.period' has changed.
This commit also stops the graph from flickering when it is updated in realtime.
* update names of two variables
* remove unnecessary negation
* make collapsed graph state more explicit
* consider stored invalid intervals when graph mounts
* fix not showing loading spinner regression
* remove interval state from VisitorGraph (#2540)
* Realtime prop breakdown (#2535)
* disable load more in realtime mode
* extract doFetch function
* separate fetchPropBreakdown and fetchNextPage functions
* subscribe for auto-updates in realtime
* improve readability with function name changes
* Remove show_noref behaviour
Removes query param show_noref which was used from React to control
whether to show Direct / None traffic or not. The show_noref behaviour
was untested previously.
Closes#2523
* Add changelog entry
* Fix tests
* Removed files I did not mean to check in :)
This commit lints JavaScript files, removes commented-out code, and
moves 3 constants to `graph-util.js`. This is part of the Intervals
work, in order to make the original pull request smaller and release
this feature incrementally.
Related: #1574
Co-authored-by: Vignesh Joglekar <hey@vigneshjoglekar.com>
* Make TestUtils module available in all tests
* Add macros patching the application env in tests
Unfortunately a lot of existing functionality relies on
certain application env setup. This isn't ideal because
the app config is a shared state that prevents us from
running the tests in parallel.
Those macros encapsulate setting up new env for test purposes
and make sure the changes are reverted when the test finishes.
* Allow passing request opts to HTTPClient.post/4
We need this to swap custom request building in
Google Analytics import.
* Unify errors when listing sites
* React: propagate backend error messages if available
* React: catch API errors in Search Terms component
* Propagate google API errors on referrer drilldown
* Handle verified properties errors in SC settings
* Add missing tests for SC settings controller
* Unify errors for fetching search analytics queries (list stats)
* Unify errors refreshing Google Auth Token
* Test fetch_stats/3 errors and replace Double with Mox
* Fixup makrup
* s/class/className
* Simplify Search Terms display in case of errors
* Fix warnings
* Overrides content-type for SVG favicons
* Organize favicon rendering
Make sure the placeholder icon is always requested from
/favicon/sources/placeholder
* Run prettier on site-switcher.js
* Yak Shave: upgrade Heroicons to 2.0
* Use HeroIcons instead of custom svg
* Update lib/plausible_web/plugs/favicon.ex
Co-authored-by: Adam Rutkowski <hq@mtod.org>
Co-authored-by: Adam Rutkowski <hq@mtod.org>
* Replace current tooltip with Popper.js
* Merge tooltip and title for top stats
* Format bounce rate and visit duration numbers in tooltip
* Add 'width=manual' mode for embed
* Add changelog entry
* Use helper function canMetricBeGraphed
This commit fixes a commit when the 24-hour clock was displayed like
`09 Uhr:00` or `06 h:00` for some locales. It keeps appending the `:00`
string but removes any non-numeric characters first.
As the whole project is AGPL v3 licensed, this just reflects that in
the package.json. This is not changing the tracker JS license.
Signed-off-by: Bhavin Gandhi <bhavin7392@gmail.com>
* added custom dimension filtering tests for pages
* first filter UI in place
* pages, entry pages and exit pages can be filtered by pageview props
* added tests for expected filtering behaviour
* fix dimension filter for sources + tests
* added is_not filtering functionality
* fixed formatting
* fixed admin_test
* added (none) as filter value + is_not filter type in UI
* added prefilling applied filter values and some UI tweaks
* added fetch options
* Make prop suggestions work with `props` filter
* Fix test
* Track login state internally
* Add CHANGELOG entry
Co-authored-by: Uku Taht <uku.taht@gmail.com>
* Update the internal /sites api to paginate results and adapts site-switcher to it
* Update the Changelog
* Format internal controller
* Remove the `+ Add Site` link from the site-switcher in the dashboard
* Change camel to snake case and replace imports with fully qualified calls
* Remove trailing comma from site-switcher
* First pass bringing in previous graph improvements, and comparsion context
* Swaps issue template to new issue form syntax
* Indentation update
* Indentation update?
* More indentation
* Intendation is hard
* Finalized indentation?
* Github indentation
* Missing fields
* Formatting changes
* Checkbox changes
* Uses new timeseries API, various UI improvements, descopes conversions, ToP from graphing
* Fixes Mobile UI Issues
* Improves point detection and display on hover
* Fixes & adds tests for updated main-graph API route
* Changelog
* Changes to better metric option declaration & minor UI/default fixes
* Fixes top stat tooltips showing unformatted numbers for special (non-rounded) top stats
* Formatting
* Fixes regression with dashed portion not stopping at present_index
* Removes comparison + lint
* Improves top stat active style
* Removes comparison tests
* Splits out tooltip and top stats
Still needs:
- Tests
- Potentially more cleanup
* Adds/moves tests for top stats
* Formatting
* Updates metric LS key, removes console log
* Various fixes + cleanup
* Makes tooltip position & style more consistent
* Fixes test (returns import status on both main graph & top stats)
* Fixes interaction with month dateFormatter
* Fixes edge case tooltip behavior
It was simpler than I thought :/
* Make the entire top stat clickable
* Minor UI improvements
* Fixes another tooltip visibility edge case + cleans up boolean algebra
Co-authored-by: Uku Taht <Uku.taht@gmail.com>
* small refactors and adds 'contains' to modal
* supports contains filter in the backend
* moves entry and exit page under the page filter
* prettier
* updates the CHANGELOG
* undo package-lock changes
* fixes formatting for elixir
* renames unused parameter to _
* Update changelog
* Use uppercase for constants and update type/prefix lookup
* Add has_imported_stats boolean to Site
* Add Google Analytics import panel to general settings
* Get GA profiles to display in import settings panel
* Add import_from_google method as entrypoint to import data
* Add imported_visitors table
* Remove conflicting code from migration
* Import visitors data into clickhouse database
* Pass another dataset to main graph for rendering in red
This adds another entry to the JSON data returned via the main graph API
called `imported_plot`, which is similar to `plot` in form but will be
completed with previously imported data. Currently it simply returns
the values from `plot` / 2. The data is rendered in the main graph in
red without fill, and without an indicator for the present. Rationale:
imported data will not continue to grow so there is no projection
forward, only backwards.
* Hook imported GA data to dashboard timeseries plot
* Add settings option to forget imported data
* Import sources from google analytics
* Merge imported sources when queried
* Merge imported source data native data when querying sources
* Start converting metrics to atoms so they can be subqueried
This changes "visitors" and in some places "sources" to atoms. This does
not change the behaviour of the functions - the tests all pass unchanged
following this commit. This is necessary as joining subqueries requires
that the keys in `select` statements be atoms and not strings.
* Convery GA (direct) source to empty string
* Import utm campaign and utm medium from GA
* format
* Import all data types from GA into new tables
* Handle large amounts of more data more safely
* Fix some mistakes in tables
* Make GA requests in chunks of 5 queries
* Only display imported timeseries when there is no filter
* Correctly show last 30 minutes timeseries when 'realtime'
* Add with_imported key to Query struct
* Account for injected :is_not filter on sources from dashboard
* Also add tentative imported_utm_sources table
This needs a bit more work on the google import side, as GA do not
report sources and utm sources as distinct things.
* Return imported data to dashboard for rest of Sources panel
This extends the merge_imported function definition for sources to
utm_sources, utm_mediums and utm_campaigns too. This appears to be
working on the DB side but something is incomplete on the client side.
* Clear imported stats from all tables when requested
* Merge entry pages and exit pages from imported data into unfiltered dashboard view
This requires converting the `"visits"` and `"visit_duration"` metrics
to atoms so that they can be used in ecto subqueries.
* Display imported devices, browsers and OSs on dashboard
* Display imported country data on dashboard
* Add more metrics to entries/exits for modals
* make sure data is returned via API with correct keys
* Import regions and cities from GA
* Capitalize device upon import to match native data
* Leave query limits/offsets until after possibly joining with imported data
* Also import timeOnPage and pageviews for pages from GA
* imported_countries -> imported_locations
* Get timeOnPage and pageviews for pages from GA
These are needed for the pages modal, and for calculating exit rates for
exit pages.
* Add indicator to dashboard when imported data is being used
* Don't show imported data as separately line on main graph
* "bounce_rate" -> :bounce_rate, so it works in subqueries
* Drop imported browser and OS versions
These are not needed.
* Toggle displaying imported data by clicking indicator
* Parse referrers with RefInspector
- Use 'ga:fullReferrer' instead of 'ga:source'. This provides the actual
referrer host + path, whereas 'ga:source' includes utm_mediums and
other values when relevant.
- 'ga:fullReferror' does however include search engine names directly,
so they are manually checked for as RefInspector won't pick up on
these.
* Keep imported data indicator on dashboard and strikethrough when hidden
* Add unlink google button to import panel
* Rename some GA browsers and OSes to plausible versions
* Get main top pages and exit pages panels working correctly with imported data
* mix format
* Fetch time_on_pages for imported data when needed
* entry pages need to fetch bounces from GA
* "sample_percent" -> :sample_percent as only atoms can be used in subqueries
* Calculate bounce_rate for joined native and imported data for top pages modal
* Flip some query bindings around to be less misleading
* Fixup entry page modal visit durations
* mix format
* Fetch bounces and visit_duration for sources from GA
* add more source metrics used for data in modals
* Make sources modals display correct values
* imported_visitors: bounce_rate -> bounces, avg_visit_duration -> visit_duration
* Merge imported data into aggregate stats
* Reformat top graph side icons
* Ensure sample_percent is yielded from aggregate data
* filter event_props should be strings
* Hide imported data from frontend when using filter
* Fix existing tests
* fix tests
* Fix imported indicator appearing when filtering
* comma needed, lost when rebasing
* Import utm_terms and utm_content from GA
* Merge imported utm_term and utm_content
* Rename imported Countries data as Locations
* Set imported city schema field to int
* Remove utm_terms and utm_content when clearing imported
* Clean locations import from Google Analytics
- Country and region should be set to "" when GA provides "(not set)"
- City should be set to 0 for "unknown", as we cannot reliably import
city data from GA.
* Display imported region and city in dashboard
* os -> operating_system in some parts of code
The inconsistency of using os in some places and operating_system in
others causes trouble with subqueries and joins for the native and
imported data, which would require additional logic to account for. The
simplest solution is the just use a consistent word for all uses. This
doesn't make any user-facing or database changes.
* to_atom -> to_existing_atom
* format
* "events" metric -> :events
* ignore imported data when "events" in metrics
* update "bounce_rate"
* atomise some more metrics from new city and region api
* atomise some more metrics for email handlers
* "conversion_rate" -> :conversion_rate during csv export
* Move imported data stats code to own module
* Move imported timeseries function to Stats.Imported
* Use Timex.parse to import dates from GA
* has_imported_stats -> imported_source
* "time_on_page" -> :time_on_page
* Convert imported GA data to UTC
* Clean up GA request code a bit
There was some weird logic here with two separate lists that really
ought to be together, so this merges those.
* Fail sooner if GA timezone can't be identified
* Link imported tables to site by id
* imported_utm_content -> imported_utm_contents
* Imported GA from all of time
* Reorganise GA data fetch logic
- Fetch data from the start of time (2005)
- Check whether no data was fetched, and if so, inform user and don't
consider data to be imported.
* Clarify removal of "visits" data when it isn't in metrics
* Apply location filters from API
This makes it consistent with the sources etc which filter out 'Direct /
None' on the API side. These filters are used by both the native and
imported data handling code, which would otherwise both duplicate the
filters in their `where` clauses.
* Do not use changeset for setting site.imported_source
* Add all metrics to all dimensions
* Run GA import in the background
* Send email when GA import completes
* Add handler to insert imported data into tests and imported_browsers_factory
* Add remaining import data test factories
* Add imported location data to test
* Test main graph with imported data
* Add imported data to operating systems tests
* Add imported data to pages tests
* Add imported data to entry pages tests
* Add imported data to exit pages tests
* Add imported data to devices tests
* Add imported data to sources tests
* Add imported data to UTM tests
* Add new test module for the data import step
* Test import of sources GA data
* Test import of utm_mediums GA data
* Test import of utm_campaigns GA data
* Add tests for UTM terms
* Add tests for UTM contents
* Add test for importing pages and entry pages data from GA
* Add test for importing exit page data
* Fix module file name typo
* Add test for importing location data from GA
* Add test for importing devices data from GA
* Add test for importing browsers data from GA
* Add test for importing OS data from GA
* Paginate GA requests to download all data
* Bump clickhouse_ecto version
* Move RefInspector wrapper function into module
* Drop timezone transform on import
* Order imported by side_id then date
* More strings -> atoms
Also changes a conditional to be a bit nicer
* Remove parallelisation of data import
* Split sources and UTM sources from fetched GA data
GA has only a "source" dimension and no "UTM source" dimension. Instead
it returns these combined. The logic herein to tease these apart is:
1. "(direct)" -> it's a direct source
2. if the source is a domain -> it's a source
3. "google" -> it's from adwords; let's make this a UTM source "adwords"
4. else -> just a UTM source
* Keep prop names in queries as strings
* fix typo
* Fix import
* Insert data to clickhouse in batches
* Fix link when removing imported data
* Merge source tables
* Import hostname as well as pathname
* Record start and end time of imported data
* Track import progress
* Fix month interval with imported data
* Do not JOIN when imported date range has no overlap
* Fix time on page using exits
Co-authored-by: mcol <mcol@posteo.net>
* Change `Country` to `Location` in the Add filter dropdown
This filter allows to select region and city too
* Rename country filter group to Location
* Use search icon for filters
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
* Change `Country` to `Location` in the Add filter dropdown
This filter allows to select region and city too
* Rename country filter group to Location
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
* chore(docker): improve repeat contributions workflow
* This change adds two new commands to gracefully stop and remove the Postgres and Clickhouse docker containers. To do so, it also gives them a recognizable name.
* Additionally, the Postgres container is updated to use a named volume for its data. This lower friction for repeat contributions where one would otherwise sign up and activate their accounts again and again each time.
* fix(ui): site switcher should toggle, instead of just opening
* feat/wip: public site toggle dropdown
* fix(ux): toggling the site switcher & closing from anywhere
* Do not react to key presses for guest users
* Cleanup duplicate code
* Fix logic left in for local testing
* Add a changelog entry for guest dropdown toggle
* Add `utm_content` and `utm_term`.
Support `utm_content` and `utm_term` as requested in #515.
* Add dropdown for UTM options
* Remove utm_content and term from filter modal for now
Co-authored-by: Blender Defender <defenderblender@gmail.com>
* chore(docker): improve repeat contributions workflow
* This change adds two new commands to gracefully stop and remove the Postgres and Clickhouse docker containers. To do so, it also gives them a recognizable name.
* Additionally, the Postgres container is updated to use a named volume for its data. This lower friction for repeat contributions where one would otherwise sign up and activate their accounts again and again each time.
* fix(ui): site switcher should toggle, instead of just opening
* fix(ux): toggling the site switcher & closing from anywhere
* chore(docker): improve repeat contributions workflow
* This change adds two new commands to gracefully stop and remove the Postgres and Clickhouse docker containers. To do so, it also gives them a recognizable name.
* Additionally, the Postgres container is updated to use a named volume for its data. This lower friction for repeat contributions where one would otherwise sign up and activate their accounts again and again each time.
* fix(ui): pill text to use text decoration instead of border
Support is pretty good including Safari 12+ without a prefix.
* docs(changelog): entry for pill text fix
