Commit Graph

323 Commits

Author SHA1 Message Date
Matt Colligan
945bf82c72
Fix conversions metrics (#1447)
f576fa2 should have updated the conversion metric names so that
`unique_conversions` and `total_conversions` are the two metrics
returned by the conversions API. This updates those so that the CSV
export outputs the correct data.
2021-11-05 15:18:59 +02:00
Uku Taht
76f6a00bec Add honeycomb 2021-11-05 14:58:57 +02:00
Matt Colligan
f576fa2a2c
Improvements to CSV export (#1427)
* Add details=True to export API parameters

This makes the ZIP export add `%{"details" => "True"}` to the query's
`params` when fetching data internally for packaging in the ZIP.

This adds bounce_rate and time_on_page to the data in pages.csv, and
bounce_rate and visit_duration to sources.csv.

* Make API return data with consistent names

Some of the data types returned via the JSON or CSV API use inconsistent
naming, and some have redundant name changes (i.e. count -> visitors ->
count). This makes these all consistent and removes the redundancy.

This addresses #1426, fixes some of the CSV headers, and unifies the
JSON and CSV return data labels.

* Update changelog

* Test should use Timex.shift, not relative time

* Return full country names in CSV export

This also replaces the " character with ' in two country names, as those
are the characters used in the names, yielding a more predictable and
'correct' output.

* Fetch CSV exported data concurrently

* Use spinner to indicate when export has started

* Use 300 as default number of brekadown entries for export

Higher numbers (e.g. 1000) seem to cause clickhouse errors when there
many pages to request. It is unclear what is causing the error, as
clickhouse returns an "unknown" error code and an empty error message.
2021-11-04 14:20:39 +02:00
Matt Colligan
8138a6e3a5
Start putting together means to export all aggregate data (#1387)
* Export all dashboard data in zip

This packages all data currently visible in the dashboard into
individual CSVs and downloads them together in a zip.

* Also export conversions in zip of CSVs

* Update export test with zip file response

* Add zip file download to changelog
2021-10-26 15:54:50 +02:00
Uku Taht
710f6f568a Ignores invalid query param part 2021-10-25 10:02:38 +02:00
Uku Taht
6dd9b0bb2f Do not error when bad custom prop value is supplied 2021-10-22 15:48:59 +02:00
Uku Taht
950bd012d5 Simplify geolix database config 2021-10-18 15:49:55 +02:00
Uku Taht
f77c37800f Use safe version of Cachex.fetch
Fixes #905
2021-10-18 11:27:52 +02:00
Vignesh Joglekar
0c982a8670
Adds list-based country report option (#1381)
* Adds list option for countries report

* Changelog

* Renames Geo to Locations

* Lint

Co-authored-by: Uku Taht <Uku.taht@gmail.com>
2021-10-14 10:55:43 +02:00
Vignesh Joglekar
7e9d83d62e
Adds pagination for custom props & fixes an issue with errant (none) return values (#1382)
* Adds pagination for custom props, fixes errant (none) being returned in the middle of a dataset

* Formatting

* Fixes errant (none) value correctly

* Changelog

* Adds tests
2021-10-14 09:45:47 +02:00
Uku Taht
015dcfa051 HOTFIX: add domain blacklist 2021-09-30 09:45:52 +02:00
Uku Taht
bd7de59a9c Show total visitors when filtered for goal 2021-09-29 13:28:29 +02:00
Uku Taht
a722d02e63 Return 400 when domain is missing 2021-09-24 13:28:10 +02:00
Uku Taht
5be38a57fa Do not return prop names when they are not used 2021-09-24 11:03:20 +02:00
Uku Taht
2f3bee800d Return more information in /api/event response
Fixes #1246
2021-09-24 09:38:23 +02:00
Uku Taht
d93fe8197e Accept chrome extension URLs 2021-09-22 11:20:11 +02:00
Uku Taht
c913d2fbb9 Fix referrer drilldown for Google
Fixes #1341
2021-09-21 10:53:21 +02:00
Ro Savage
b3bc796d50
Add conversion_rate to sources api and source table (#1299)
* Add conversion_rate to sources api and source table

* Remove percentageFormatter

* Update source tests to include conversionat rate

* Add CR to detals modal

* Correct formatting with linter

* Add change log

* Add CR to Pages, Device and Countries panels
2021-09-20 16:17:11 +02:00
Uku Taht
3b4db64b43 Mix format 2021-09-16 10:22:54 +02:00
Uku Taht
999a477b5d Fix get_root_domain for "(none)" 2021-09-16 10:17:14 +02:00
RobertJoonas
4742e05ab1
Allow URLs without hostname setting it to (none) (#1335) 2021-09-16 10:05:43 +02:00
Uku Taht
7e772151e3 Adjust tests 2021-09-10 12:55:59 +03:00
Uku Taht
bd797c201f Mix format 2021-09-10 12:38:03 +03:00
Uku Taht
7cb148a4f1 Use only root domain in user id generation 2021-09-10 12:35:00 +03:00
Uku Taht
c68b88c24b
Upgrade otp (#1313)
* update tool versions to latest

* Upgrades erlang and elixir, format project

Co-authored-by: happysalada <raphael@megzari.com>
2021-09-09 11:17:24 +03:00
Uku Taht
08afc2ae71 Timeout db queries in 10 seconds instead of 5 2021-09-06 14:40:15 +03:00
Uku Taht
01412d090f Fix sampling in JOIN queries 2021-08-31 11:13:09 +03:00
Uku Taht
7bc40a63fe Remove unused variable 2021-08-26 14:00:52 +03:00
Uku Taht
7394a5376a Fix custom props being limited to 9 2021-08-25 11:16:20 +03:00
Uku Taht
a443094228 Show more than 9 goals 2021-08-23 11:28:19 +03:00
Uku Taht
0fe053e106 Fix realtime dashboard 2021-08-19 15:41:08 +03:00
Uku Taht
63b7abad58 Disable sampling in the API 2021-08-19 13:13:13 +03:00
Uku Taht
f9ea3cda6b Fix custom props in conversions 2021-08-19 11:03:41 +03:00
Uku Taht
a9b16fa6c6 Fix exit rate when filtering for page 2021-08-19 10:32:03 +03:00
Uku Taht
cff071bdf3 Remove sampling notice from realtime view 2021-08-18 15:24:44 +03:00
Uku Taht
49214eaff8 Fix exit pages when using a goal filter 2021-08-18 14:49:39 +03:00
Uku Taht
03e46d2ed5 Apply filters correctly to suggestions 2021-08-18 12:33:08 +03:00
Uku Taht
d9ac8ccdb9 Fix samplign in main graph and top pages 2021-08-18 12:29:00 +03:00
Uku Taht
669866a16b
Finish stats module (#1248)
* Fix small inconsitencies in stats module

* Format
2021-08-17 15:21:12 +03:00
Uku Taht
4fef567623 Fix goal filter 2021-08-17 11:01:00 +03:00
Uku Taht
ecbc68f05b Fix entry pages 2021-08-16 11:58:36 +03:00
Uku Taht
c3c6c082d1 Fix present index 2021-08-13 11:28:51 +03:00
Uku Taht
39f5bc6afb Make sure Direct / None is not showed in small reports 2021-08-04 10:50:23 +03:00
Uku Taht
7d1b1214e2 Implement dashboard with new stats module 2021-08-02 15:46:30 +03:00
Vignesh Joglekar
30ac901141
Adds manual, editable, auto-suggested filters, and negated&globbed path-based filters (#1121)
* Adds manual-filters + friends

commit 308192044d726e9a6f7406e333048ab3407aa260
Merge: 39287ab a299fab
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri Jun 11 03:40:55 2021 -0500

    Merge branch 'manual-filters' into manual-filters-2

commit a299fab1fb
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri Jun 11 02:20:26 2021 -0500

    Changes to split and pattern matched function for time_on_page

commit 10f10c933a
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri Jun 11 01:53:18 2021 -0500

    Fixes a couple of minor UX issues

commit f2e5ce8eb3
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri Jun 11 01:49:10 2021 -0500

    Fixes time on page for globbed and negated page paths

commit bb18af6526
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu Jun 10 05:24:05 2021 -0500

    Close to finalized version of updated version

    Just needs some additional testing + potentially code cleanup

commit d0b7bfe3da
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri May 28 04:21:21 2021 -0500

    Real Dialyzer Fix

commit 296a76af34
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri May 28 03:44:29 2021 -0500

    Dialyzer fix

commit 91f3b44017
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 17:11:24 2021 -0500

    Changelog

commit e041f75745
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 17:09:52 2021 -0500

    Formatting

commit f689642204
Merge: e00929b 4ff25f6
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 17:08:47 2021 -0500

    Merge branch 'master' into manual-filters

commit e00929b1c5
Merge: 83887c4 806975e
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 17:01:48 2021 -0500

    Merge branch 'manual-filters' of github.com:Vigasaurus/plausible-analytics into manual-filters

commit 83887c49cc
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 16:59:32 2021 -0500

    Adds tests for suggestions, formats goals suggestion query

commit 1cb7732a08
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 16:36:31 2021 -0500

    Adds goals as auto-complete capable filter

commit 4ca39cc406
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 16:08:22 2021 -0500

    95% completed auto-complete setup

    Still needs:
    - tests
    - goals added as filter

commit 22d38c4cd8
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 03:47:19 2021 -0500

    80% of auto-complete filters progress

    Still needs
    - countries and screen into new format
    - re-style dropdown and background
    - drop debounce time
    - tests

commit 806975ede9
Merge: 81c5e05 1a93542
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Tue May 25 15:28:21 2021 -0500

    Merge branch 'master' into manual-filters

commit 81c5e05760
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Tue May 25 15:21:03 2021 -0500

    Makes colorings on top bar elements consistent

commit fa7f6c2268
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Tue May 25 14:58:25 2021 -0500

    Makes requested changes, adds different version of filter button

commit 7dc65b9b7a
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Sat May 22 04:29:01 2021 -0500

    Changelog

commit c684f1c76a
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Sat May 22 04:26:14 2021 -0500

    Various UI Improvements

    - Makes edit buttons full-length & properly sized
    - Adds remove filter button in edit menu

commit a632e7a8ee
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Sat May 22 03:11:50 2021 -0500

    Adds tests for exclusions and wildcards

commit eb91a7942b
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Sat May 22 03:02:23 2021 -0500

    Fixes editing UX on list view

commit 6209d72aee
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri May 21 04:01:17 2021 -0500

    Bugfix in realtime view, formatting

commit 007d44df38
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri May 21 03:23:16 2021 -0500

    Second pass - mostly everything user-facing is done

    Still needs:
    - Tests
    - Potentially negating other filters
    - Potentially some code cleanup

commit cb7b5b9fbd
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri May 21 01:49:52 2021 -0500

    First pass on manual filter & path regex/negated filters

    Still needs:
    - Form structure on filter modal
    - Edit filter button
    - Filter dropdown UI improvement
    - Filter modal mount data collection
    - Tests
    - Potentially negating other filters

* Fixes an issue between the updated React version and Flatpickr not showing up

Reverting to v16 was the simplest fix without diving into the issue upstream with Flatpickr

* Resolves comments on UI and code style

Co-authored-by: Uku Taht <Uku.taht@gmail.com>
2021-06-21 14:42:16 +03:00
Uku Taht
e71de6dc1f
Invitations (#1122)
* Invite existing user to a site

* Add invitation flow for non-existing users

* Accept and reject invitations

* Use invitation flow for existing users

* Locking mechanism for sites

* Authorization for site settings

* Show usage based on site ownership

* Add ability to remove members from a site

* Do not show settings link to viewer roles

* Ability to remove invitations

* Remove `Plausible.Sites.count_for/1`

* Fix tests

* Do not show the trial banner after the trial

* Correct trial emails

* Transfer ownership

* Send invitation email to existing user

* Add invitation email flows

* Add plug for role-based authorization

* Rename AuthorizeStatsPlug -> AuthorizeSiteAccess

* Add email flow for ownership transfer

* Fix URLs in emails

* Fix small copy issues

* Make 'People' its own section in site settings

* Notify user via email if their access has been removed

* Check site lock status when invitation is accepted

* Check lock status when user subscribes

* Make sure only admins and owners can create shared links

* Changelog

* Add LockSites to daily cron

* Clean invitations after 48 hours

* Add notices about expiry

* Add invitation expired page

* Add doc link
2021-06-16 15:00:07 +03:00
Uku Taht
9f16603bfb Accept application/json in /api/event 2021-06-02 15:10:44 +03:00
Uku Taht
1a93542cd7 Add rate limit to API requests 2021-05-25 11:58:49 +03:00
Vignesh Joglekar
5f42141e08
Adds time on page as a top stat when a page filter is active (#1051)
* Adds time on page as a top stat when a page filter is active

* Removes dangling debug statement
2021-05-19 10:21:43 +03:00
Vignesh Joglekar
41e4690116
Adds Time on Page metric to Top Pages report (#1007)
* First pass

Needs more testing & potentially cleanup

* Fixes tests, error handling

* Formatting

* Removes broken test

* Fixes inconsistent test

This was due to Clickhouse setup not inserting the sessions with the exact same timestamp consistently and making the test inconsistent

* Combines `include` param, asyncs time_on_page and bounce_rate

* Fixes CH error when no pageviews exist in period

* Format

* Changelog

* Increases await timeout to accomodate larger data sets

* Improves handling of timeout behavior

* Fixes session-based filtering on time on page queries

* Formatting

* Removes old forced entry page modal from top pages report
2021-05-18 15:14:33 +03:00
Uku Taht
7f9057818d Protect the API endpoint from creating too many sites 2021-05-05 10:39:24 +03:00
Uku Taht
119b9514b2 Add limit of 20 sites 2021-05-05 10:30:05 +03:00
Uku Taht
0799138622 Add nil check to hashing 2021-05-03 17:06:36 +03:00
Uku Taht
5c759dbfed Clean up Ecto changeset code for events 2021-04-27 11:38:56 +03:00
Uku Taht
f185d3736e Add hostname to user id hash 2021-04-27 11:10:37 +03:00
Uku Taht
5c77ec3f5d
Support multiple data-domain values (#968)
* Support multiple `data-domain` values

* Add changelog entry
2021-04-27 10:26:44 +03:00
Uku Taht
b107717774
Additional metrics for timeseries API call and CSV export (#952)
* Add `metrics` option to timeseries call

* Add more metrics to csv export

* Add changelog entry
2021-04-23 15:27:50 +03:00
Uku Taht
80c62cfdb2 Fix bad request call 2021-04-15 15:35:15 +03:00
Uku Taht
70f0657327 Update shared link API 2021-04-15 11:38:44 +03:00
Uku Taht
80abfa6ef6 Add validations 2021-04-14 15:04:25 +03:00
Uku Taht
3ed964b45b Add API endpoints for site and shared link creation 2021-04-14 11:45:45 +03:00
Uku Taht
4cbebe8ac9 Remove old Sentry code 2021-03-31 11:38:14 +03:00
Uku Taht
01538c9907
WIP: add lazy loading for dashboard (#859)
* WIP: add lazy loading for dashboard

* Improve error handling

* WIP

* Implement lazy loading for most reports

* Add lazy loading to conversions
2021-03-25 11:55:15 +02:00
Uku Taht
c4c0bb2095 Various formatting improvements
* Format numbers over 1B
* Show full number when hovering
2021-03-24 12:08:05 +02:00
Uku Taht
3eb2f6d7a8
Add support for sampling (#875)
* Add support for sampling

* Use correct mix package

* Add notice about data sampling
2021-03-24 11:19:10 +02:00
Uku Taht
432ad2a606
Stats API improvements (#826)
* Show hourly data for a day

* Fix aggregations with long periods and comaprisons enabled

* Return Direct / None instead of empty string when missing source

* Fix/Align breakdown query with current queries in the UI

* Show correct number of days for 7d and 30d

* Wrap results of aggregate call in `results`

* Allow breakdown by event:name

* Ensure event:name filter works in all contexts

* WIP

* Add support for custom event props
2021-03-18 11:47:57 +02:00
Uku Taht
0ceb539a22 Reduce errors sent to Sentry 2021-03-03 11:36:39 +02:00
Vignesh Joglekar
ff32218bd0
Adds entry and exit pages to Top Pages module (#712)
* Initial Pass

* Adds support for page visits counting by referrer

* Includes goal selection in entry and exit computation

* Adds goal-based entry and exit page stats, formatting, code cleanup

* Changelog

* Format

* Exit rate, visit duration, updated tests

* I keep forgetting to format :/

* Tests, last time

* Fixes double counting, exit rate >100%, relevant tests

* Fixes exit pages on filter and goal states

* Adds entry and exit filters, fixes various bugs

* Fixes discussed issues

* Format

* Fixes impossible case in tests

Originally, there were only 2 pageviews for `test-site.com`,`/` on `2019-01-01`, but that doesn't make sense when there were 3 sessions that exited on the same site/date.

* Format

* Removes boolean function parameter in favor of separate function

* Adds support for queries that use `page` filter as `entry-page`

* Format

* Makes  loader/title interaction in sources report consistent
2021-02-26 11:02:37 +02:00
Uku Taht
a29ac44434
Api improvements (#723)
* Require date parameter with custom period

* Validate format of the `date` param with custom period

* Return proper status codes for auth failures

* Add breakdown endpoint

* Fix pagination

* Add API validations for breakdown call

* Change the breakdown endpoint to return an object instead of an array

* Remove change to aggregate call

* Change timeseries call
2021-02-22 10:21:25 +02:00
Uku Taht
77628d04c9
Validate API query params (#703)
* Add validations for `date`, `period` and `metrics` variables

* Validate interval query parameter
2021-02-10 16:27:58 +02:00
Uku Taht
5acb5b7039
Stats API (#679)
* WIP

* Add ability to filter by anything

* Add API keys

* Add version to api endpoint

* Fix API test route

* Fix API tests

* Allow 'date' parameter in '6mo' and '12mo'

* Rename session -> visit in API filters

* Filter expressions in the API

* Implement filters in aggregate call

* Add `compare` option to aggregate call

* Add way to manage API keys through the UI

* Authenticate with API key

* Use API key in tests
2021-02-05 11:23:30 +02:00
Uku Taht
cb3e401dce Revert "Add Comparison Line and Improved Tooltip to Visitors Graph (#624)"
This reverts commit 979d2526d5.
2021-01-21 14:42:55 +02:00
Vignesh Joglekar
979d2526d5
Add Comparison Line and Improved Tooltip to Visitors Graph (#624)
* First pass

* Title Section of Tooltip+Formatting

* Cleanup, Formatting, Rest of Tooltip

* Fixes bug in "month" calculate_plot

Bug: where calculate_plot on "month" didn't correctly use the end date in the date range, and instead always used the current time - making requesting past chunks of 6 or 12 month data impossible

* Cleanup + Comments

* Cleanup+Percentage Formatting Changes

* Fixes Realtime Tooltip

* Prevent FE Crash

* Minor format, code cleanup

* Removes erroneous changes

* Github UI is bad at indenting
2021-01-20 15:49:39 +02:00
Oliver Kriska
ae42b86792
Dialyzer and Credo checks (#558)
* Checks

 - added Dialyzer
 - fixed Dialyzer errors
 - added Dialyzer check to GitHub Actions with cache
 - added Credo
 - fixed Credo Warnings
 - added Credo Warnings check to GitHub Actions with cache
 - added compile warnings check to GitHub Actions
 - reformated GitHub Actions YAML

* Dialyzer

 - allow it in test env

* Dialyzer

 - fixed test env
 - renamed GitHub actions steps

* AppSignal

 - upgraded deprecated version
 - Upgraded:
  appsignal 2.0.5 => 2.0.7
  certifi 2.5.2 => 2.5.3
  hackney 1.16.0 => 1.17.0
  idna 6.0.1 => 6.1.1
  parse_trans 3.3.0 => 3.3.1
  unicode_util_compat 0.5.0 => 0.7.0 (minor)

* Credo

 - fixed CRM plug
2021-01-07 15:16:04 +02:00
Uku Taht
4715c88308 Remove warning 2021-01-05 15:17:50 +02:00
Uku Taht
4ba5fb5ead
Add cache for user-agent strings (#547) 2021-01-04 16:38:56 +02:00
Uku Taht
81c12884cd
Add elixir action (#526)
* Add elixir action

* Format the codebase

* Add postgresql

* Postgres config

* Run postgres on localhost

* Add clickhouse to CI
2020-12-29 15:17:27 +02:00
Uku Taht
9f1eac005e Fixes calls to Query.shift_back/2 2020-12-23 11:18:45 +02:00
Vignesh Joglekar
e2599ffcc1
Changes "This Month" view to Month-to-date for current month (#491)
* Initial MTD

* Adds FE support

* Removes broken Day to Time, debug statements

* Changelog

* Cleans up conditional
2020-12-23 10:42:22 +02:00
Vignesh Joglekar
f776c6bb30
Improves realtime top stats and conversion rate calculation (#500)
* Fixes goal conversion rate when period is realtime

* Adds unique visitors (30m) to realtime top stats

* Changelog

Co-authored-by: Uku Taht <Uku.taht@gmail.com>
2020-12-22 15:54:41 +02:00
Uku Taht
5dcb329744
Fix sessions bug (#496)
* Add sitekey to captcha

* Remove appsignal decorators

* Fix session write buffer
2020-12-21 14:37:39 +02:00
Vignesh Joglekar
d3f7c94ebe
Adds "Load More" functionality to pages modal (#480)
* Initial Feature Commit

* Changelog
2020-12-17 11:35:27 +02:00
Uku Taht
3142e8e190 Add Appsignal instrumentation 2020-12-14 16:06:43 +02:00
Uku Taht
d27dad7299 Count Opera Mobile as Opera 2020-11-23 11:09:21 +02:00
Uku Taht
19a9e314fc Accept anything in prop value as long as it can to_string 2020-11-18 14:53:43 +02:00
Uku Taht
b0b5d92f6e Fix URL decoding 2020-11-17 14:40:22 +02:00
Uku Taht
d0d7b823f8
Browser and OS version (#397)
* Collect browser and OS version

* Display browser version

* Show operating system versions

* Device categorization

* Treat headless chrome like a bot

* Ignore events from automated browsers

* Only take major and minor of the version

* Add tests

* Add CHANGELOG entry

* Add changelog entry for bots

* Store empty value as browser when unknown
2020-11-10 15:18:59 +02:00
Uku Taht
0a7684f3bc Mix format 2020-11-03 11:20:11 +02:00
Uku Taht
b816d92020 Show conversion rates in conversions report 2020-10-30 11:26:16 +02:00
Uku Taht
0b6e645b44
Rename metadata -> properties (#390) 2020-10-30 10:49:41 +02:00
Uku Taht
40900c7653
Ability to add event metadata (#381)
* Ability to add event metadata

* Close Dropdown on outside click

* Show (none) value in metadata breakdown

* Allow filtering for metadata key/val pairs

* Use correct clickhouse_ecto

* Better naming for meta filter

* Add tests

* Add changelog entry

* Remove change made for testing
2020-10-28 11:09:04 +02:00
Uku Taht
c533562eaa
Show correct stats when filtered for goal (#374)
* Show proper stats when filtered for goal

* Add changelog entry
2020-10-20 11:24:20 +03:00
Uku Taht
8d9667a949
Show utm_medium, utm_source, and utm_campaign in sources modal (#321)
* Show utm_medium, utm_source, and utm_campaign in sources modal

* Allow filtering by UTM tags

* Integrate filters with URL bar

* Add CHANGELOG entry

* Refresh modal when filter changes

* Remove Direct / None from campaign results

* Add UTM tabs to top sources report

* Add pagination

* Remove dropdown from top sources popup

* Fix bug in clickhouse_ecto

* Remove referrers_for_goal

* Make sure UTM tags work OK with goals

* Make source tab selection sticky

* Consistent styling for devices and source tabs

* Add noref in realtime to utm tabs

* Fix tests
2020-09-28 11:29:24 +03:00
Uku Taht
122dd94327
Store utm_medium, utm_source, and utm_campaign (#318) 2020-09-18 11:37:32 +03:00
Uku Taht
4b36bb7138
Use clickhouse_ecto for db connection (#317)
* Use clickhouse-ecto for stats

* Use clickhouse ecto instead of low-level clickhousex

* Remove defaults from event schema

* Remove all references to Clickhousex

* Document configuration change

* Ensure createdb and migrations can be run in a release

* Remove config added for debug

* Update plausible_variables.sample.env
2020-09-17 16:36:01 +03:00
Uku Taht
49f153c380
Paginate referrers (#313)
* Add pagination for referrers

* Fix top referrers for goal
2020-09-10 10:42:43 +03:00
Uku Taht
4aa4dfdcaf
Hash mode (#299)
* Build tracker with hash mode

* Extract hash fragment in hash mode

* Serve new hash-based tracker
2020-08-25 10:56:36 +03:00
Uku Taht
d30f17c807
Extract utm_source on the server side (#298)
* Extract utm_source on the server side

* Build tracker scripts
2020-08-24 16:32:24 +03:00
Uku Taht
1519359bbf
Makes the tracker and its payload smaller (#296) 2020-08-24 15:16:07 +03:00
Uku Taht
76896f7cee
Show total conversions in addition to uniques (#294) 2020-08-20 14:57:49 +03:00
Uku Taht
01ef17abb0
Make sure all future URLs and sources are decoded (#292) 2020-08-19 15:02:16 +03:00
Uku Taht
b749889279 Rename Active visitors -> Current visitors
For consistency
2020-08-18 14:27:54 +03:00
Uku Taht
503dc60b30 Rename Total visitors -> Unique visitors 2020-08-17 16:35:55 +03:00
Uku Taht
ff64fccb18
Add site switcher (#281)
* WIP

* Add site switcher

* Run ci

* Highlight active site

* Do not show sites dropdown for logged out users

* Add site switcher to realtime view

* Remove blue border when logged out
2020-08-13 13:45:18 +03:00
Uku Taht
3a1c9e67cd Add ability to block certain IP addresses 2020-08-11 11:04:26 +03:00
Uku Taht
0a2a2656aa
Page drilldown (#276)
* WIP

* Add filter option for page

* Use JOIN to get correct top numbers

* Filter google search terms by page

* Allow filtering by page from modal

* More robust base url for google search console integration

* Fix tests
2020-08-10 16:16:12 +03:00
Uku Taht
f781c5647c
Direct traffic improvements (#268)
* Direct traffic improvements

* Make sure an empty icon shows always

* Enable direct traffic drilldown
2020-08-05 14:45:20 +03:00
Uku Taht
99bc20a8b3
Referrer drilldown (#252)
* Referrer drilldown

* Render external links for referrer drilldown modal

* Rename top pages to entry pages

* Add google keywords to referrer drilldown

* Fix link in source modal

* Remove back link from referrer drilldown

* Fix tests

* Mix format

* Remove unused function

* Get top stats from the events table

* Revert "Get top stats from the events table"

This reverts commit 67384e2d35.

* Fix pages modal title

* Improve pages report in referrer drilldown

* Fix label in referrer drilldown
2020-07-30 11:18:28 +03:00
Uku Taht
d266cfd60a
Remove comparison line (#232)
* Fix svg attributes in React

* Remove comparison line on visitor graph
2020-07-21 10:30:03 +03:00
Chandra Tungathurthi
f7b37fe9ea
Selhosted version Improvements and additional features (#209)
* first commit with test and compile job

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding 'prepare' stage

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated ci script to include "test" compile phase

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding environment variables for connecting to postgresql

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated ci config for postgres

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* using non-alpine version of elixir

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* re-using the 'compile' artifacts and added explict env variables for testing

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removing redundant deps fetching from common code

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting using mix.format -- beware no-code changes!

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* added release config

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding consistent env variable for Database

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* more cleaning up of environment variables

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding releases config for enabling releases

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* cleaning up env configs

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Cleaned up config and prepared config for releases

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated CI script with new config for test

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added Dockerfile for creating production docker image

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding "docker" build job yay!

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* using non-slim version of debian and installing webpack

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding overlays for migrations on releases

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* restricting the docker built to master branch only

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* typo fix

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding "Hosting.md" to explain hosting instructions

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removed the default comments

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added documentation related to env variables

* updated documentation and fixed typo

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated documentation

* Bumping up elixir version as `overlays` are only supported in latest version

read release notes: https://github.com/elixir-lang/elixir/releases/tag/v1.10.0

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding tarball assembly during release

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated HOSTING.md

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added support for db migration

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* minor corrections

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* initializing admin user

Admin user has been added in the "migration" phase. A default user is automatically created in the process. One can provide the related env variables, else a new one will be automatically created for you.

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Initial base domain update - phase#1

These changes are only meant for correct operating it under self-hosting. There are many other cosmetic changes, that require updates to email, site and other places where the original website and author is used.

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Using dedicated config variable `base_domain` instead

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding base_domain to releases config

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removing the dedicated config "base_domain", relying on endpoint host

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Removed the usage of "Mix" in code!

It is bad practice to use "mix" module inside the code as in actual release this module is unavailable. Replacing this with a config environment variable

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added support for SMTP via Bamboo Smtp Adapter

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Capturing SMTP errors via Sentry

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Minor updates

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding junit formatter -- useful for generating test reports

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding documentation for default user

* Resolve "Gitlab Adoption: Add supported services in "Security & Compliance""

* bumping up the debian version to fix issues

fixing some vulnerabilities identified by the scanning tools

* More updates for self-hosting

Changes in most of the places to suit self-hosting. Although, there are some which have been left-off.

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* quick-dirty-fix!

* bumping up the db connect timeout

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* bumping up the db connect timeout

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* bumping up the db connect timeout

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* bumping up timeout - skipping MRs :-/

* removing restrictions on watching for changes

this stuff isn't working

* Update HOSTING.md

* renamed the module name

* reverting formatting-whitespace changes

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* reverting the name to release

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding docker-compose.yml and related instructions

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* using `plausible_url` instead of assuming `https`

this is because, it is much to test in local dev machines and in most cases there's already a layer above which is capable for `https` termination and http -> https upgrade

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* WIP: merging changes from upstream

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* wip: more changes

* Pushing in changes from upstream

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* changes to ci for testing

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* cleaning up and finishing clickhouse integration

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updating readme with hosting details

* removing deleted files from upstream

* minor config adjustments

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting changes

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* changing the connection strategy for clickhouse during release

since clickhouse integration doesn't have an ecto support, we need to prepare the db _before_ the clickhouse migration. One workaround is to connect to a default db on init and then create a db

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting

* cleanup and added separated migration to setup

* Big improvements to selfhosting

- added ability for disabling
  - authentication completely
  - registration
  - landing page

- formatting cleanups

* Big improvements to selfhosting

- added ability for disabling
  - authentication completely
  - registration
  - landing page

- formatting cleanups

* changing smtp auth  to optional

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removed stale templates and permanently removed landing page

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removed stale templates and permanently removed landing page

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removed stale templates and permanently removed landing page

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* WIP

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* fixes form upstream merge

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* added disabling subscription for selfhosted version

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated doc

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Remove reference to file that doesn't exist

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* do not show direct traffic if there's no data

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* addressing PR comments

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
2020-07-21 09:58:00 +03:00
Uku Taht
15f6594ddb Add healthcheck endpoint 2020-07-20 10:34:35 +03:00
Uku Taht
7e391fe05e
Visit duration (#227)
* Show average visit duration

* Better format for duration

* Clean up the code for visit duration
2020-07-17 11:09:54 +03:00
Uku Taht
4d55c4657a
Rotate salts on a daily basis (#224)
* Add daily salt rotation for user ids

* Rotate salts on a cron schedule daily

* Keep old salt fo 48 hours

* Double write for plausible.io to test salt rotation

* Rotate salts for every site
2020-07-15 11:47:24 +03:00
Uku Taht
232298d327
Realtime dashboard (#212)
* Auto-updating dashboard with realtime info

* Remove extra route

* Draw list of countries next to the map

* Nice animations

* Do not show bounce rates in realtime modals

* Update countries and devices in realtime

* Remove unused component

* Show total pageviews in the last 30 minutes

* Show proper labels

* Remove unnecessary z-index

* Fix label for main graph

* Fix compiler warnings

* Add tests

* Fix copy pluralizations

* Fix copy in countries modal

* Real-time -> Realtime

* Looser test assertion

* Show last 30 minutes conversions on realtime report

* Remove EventTarget API because it doesn't work on Safari

* Get referrer drilldown from sessions table

* Fix failing tests
2020-07-14 16:52:26 +03:00
Chandra Tungathurthi
517d4db262
Selfhosted version improvements (#89)
* first commit with test and compile job

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding 'prepare' stage

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated ci script to include "test" compile phase

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding environment variables for connecting to postgresql

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated ci config for postgres

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* using non-alpine version of elixir

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* re-using the 'compile' artifacts and added explict env variables for testing

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removing redundant deps fetching from common code

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting using mix.format -- beware no-code changes!

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* added release config

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding consistent env variable for Database

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* more cleaning up of environment variables

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding releases config for enabling releases

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* cleaning up env configs

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Cleaned up config and prepared config for releases

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated CI script with new config for test

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added Dockerfile for creating production docker image

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding "docker" build job yay!

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* using non-slim version of debian and installing webpack

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding overlays for migrations on releases

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* restricting the docker built to master branch only

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* typo fix

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding "Hosting.md" to explain hosting instructions

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removed the default comments

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added documentation related to env variables

* updated documentation and fixed typo

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated documentation

* Bumping up elixir version as `overlays` are only supported in latest version

read release notes: https://github.com/elixir-lang/elixir/releases/tag/v1.10.0

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding tarball assembly during release

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated HOSTING.md

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added support for db migration

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* minor corrections

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* initializing admin user

Admin user has been added in the "migration" phase. A default user is automatically created in the process. One can provide the related env variables, else a new one will be automatically created for you.

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Initial base domain update - phase#1

These changes are only meant for correct operating it under self-hosting. There are many other cosmetic changes, that require updates to email, site and other places where the original website and author is used.

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Using dedicated config variable `base_domain` instead

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding base_domain to releases config

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removing the dedicated config "base_domain", relying on endpoint host

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Removed the usage of "Mix" in code!

It is bad practice to use "mix" module inside the code as in actual release this module is unavailable. Replacing this with a config environment variable

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added support for SMTP via Bamboo Smtp Adapter

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Capturing SMTP errors via Sentry

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Minor updates

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding junit formatter -- useful for generating test reports

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding documentation for default user

* Resolve "Gitlab Adoption: Add supported services in "Security & Compliance""

* bumping up the debian version to fix issues

fixing some vulnerabilities identified by the scanning tools

* More updates for self-hosting

Changes in most of the places to suit self-hosting. Although, there are some which have been left-off.

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* quick-dirty-fix!

* bumping up the db connect timeout

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* bumping up the db connect timeout

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* bumping up the db connect timeout

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* bumping up timeout - skipping MRs :-/

* removing restrictions on watching for changes

this stuff isn't working

* Update HOSTING.md

* renamed the module name

* reverting formatting-whitespace changes

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* reverting the name to release

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding docker-compose.yml and related instructions

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* using `plausible_url` instead of assuming `https`

this is because, it is much to test in local dev machines and in most cases there's already a layer above which is capable for `https` termination and http -> https upgrade

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* WIP: merging changes from upstream

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* wip: more changes

* Pushing in changes from upstream

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* changes to ci for testing

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* cleaning up and finishing clickhouse integration

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updating readme with hosting details

* removing deleted files from upstream

* minor config adjustments

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting changes

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* changing the connection strategy for clickhouse during release

since clickhouse integration doesn't have an ecto support, we need to prepare the db _before_ the clickhouse migration. One workaround is to connect to a default db on init and then create a db

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting

* cleanup and added separated migration to setup

* Big improvements to selfhosting

- added ability for disabling
  - authentication completely
  - registration
  - landing page

- formatting cleanups

* Big improvements to selfhosting

- added ability for disabling
  - authentication completely
  - registration
  - landing page

- formatting cleanups

* changing smtp auth  to optional

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removed stale templates and permanently removed landing page

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removed stale templates and permanently removed landing page

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
2020-07-02 11:21:11 +03:00
Uku Taht
77e8276673 Ignore malformed referrer URL 2020-06-25 17:54:11 +03:00
Uku Taht
192bc4e956 Add extra context in Sentry 2020-06-16 16:21:08 +03:00
Uku Taht
c42f8ea5c6 Guard for nil in result country 2020-06-12 18:44:16 +03:00
Uku Taht
06e2a9e9c7 Use consistent database ID 2020-06-12 11:15:19 +03:00
Uku Taht
a12cfa8a8f
Maxmind (#80)
* Add maxmind Geolite2 database

* Look up the actual country from MaxMind

* Fix test for country lookup
2020-06-12 09:51:45 +03:00
Uku Taht
1c9ad7e778 Remove initial referrer from Clickhouse 2020-06-10 10:23:57 +03:00
Chandra Tungathurthi
b4b7532f07
Formatting only changes - No code change (#75)
* first commit with test and compile job

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding 'prepare' stage

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated ci script to include "test" compile phase

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding environment variables for connecting to postgresql

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated ci config for postgres

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* using non-alpine version of elixir

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* re-using the 'compile' artifacts and added explict env variables for testing

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removing redundant deps fetching from common code

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting using mix.format -- beware no-code changes!

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* added release config

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding consistent env variable for Database

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* more cleaning up of environment variables

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding releases config for enabling releases

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* cleaning up env configs

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Cleaned up config and prepared config for releases

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated CI script with new config for test

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added Dockerfile for creating production docker image

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding "docker" build job yay!

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* using non-slim version of debian and installing webpack

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding overlays for migrations on releases

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* restricting the docker built to master branch only

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* typo fix

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding "Hosting.md" to explain hosting instructions

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removed the default comments

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added documentation related to env variables

* updated documentation and fixed typo

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated documentation

* Bumping up elixir version as `overlays` are only supported in latest version

read release notes: https://github.com/elixir-lang/elixir/releases/tag/v1.10.0

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding tarball assembly during release

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated HOSTING.md

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added support for db migration

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* minor corrections

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* initializing admin user

Admin user has been added in the "migration" phase. A default user is automatically created in the process. One can provide the related env variables, else a new one will be automatically created for you.

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Initial base domain update - phase#1

These changes are only meant for correct operating it under self-hosting. There are many other cosmetic changes, that require updates to email, site and other places where the original website and author is used.

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Using dedicated config variable `base_domain` instead

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding base_domain to releases config

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removing the dedicated config "base_domain", relying on endpoint host

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Removed the usage of "Mix" in code!

It is bad practice to use "mix" module inside the code as in actual release this module is unavailable. Replacing this with a config environment variable

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added support for SMTP via Bamboo Smtp Adapter

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Capturing SMTP errors via Sentry

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Minor updates

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding junit formatter -- useful for generating test reports

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding documentation for default user

* Resolve "Gitlab Adoption: Add supported services in "Security & Compliance""

* bumping up the debian version to fix issues

fixing some vulnerabilities identified by the scanning tools

* More updates for self-hosting

Changes in most of the places to suit self-hosting. Although, there are some which have been left-off.

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* quick-dirty-fix!

* bumping up the db connect timeout

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* bumping up the db connect timeout

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* bumping up the db connect timeout

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* bumping up timeout - skipping MRs :-/

* removing restrictions on watching for changes

this stuff isn't working

* Update HOSTING.md

* renamed the module name

* reverting formatting-whitespace changes

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* reverting the name to release

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding docker-compose.yml and related instructions

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* using `plausible_url` instead of assuming `https`

this is because, it is much to test in local dev machines and in most cases there's already a layer above which is capable for `https` termination and http -> https upgrade

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* WIP: merging changes from upstream

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* wip: more changes

* Pushing in changes from upstream

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* changes to ci for testing

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* cleaning up and finishing clickhouse integration

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updating readme with hosting details

* removing deleted files from upstream

* minor config adjustments

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting changes

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
2020-06-08 10:35:13 +03:00
Uku Taht
933cff6fe0
Remove stats from postgres (#74)
* Remove stats code related to postgres events and sessions

* Use Clickhouse events in fetch tweets worker
2020-06-05 16:14:17 +03:00
Uku Taht
390a4577b5 Fetch config options at runtime 2020-06-05 11:15:26 +03:00
Uku Taht
a228c6162f Default to empty pathname 2020-05-25 12:25:41 +03:00
Uku Taht
b31c0114c5 Use Clickhouse everywhere 2020-05-22 12:33:17 +03:00
Uku Taht
578523170f Query via clickhouse 2020-05-21 18:59:07 +03:00
Uku Taht
0ccb893c04 Ingest with new CH schema 2020-05-21 13:36:53 +03:00
Uku Taht
d94a8dbc7e
Clickhouse (#66)
* Get stats from clickhosue

* Pull stats from clickhouse

* Use correct Query namespace

* Use Clickhouse in unit tests

* Use Clickhouse in stats controller tests

* Use fixtures for unit tests

* Add Clickhouse to travis

* Use Clickhouse session store for sessions

* Add garbage collection to session store

* Reload session state from Clickhouse on server restart

* Query from sessions table

* Trap exits in event write buffer

* Run hydration without starting the whole app

* Make session length 30 minutes

* Revert changes to fingerprint schema

* Remove clickhouse from fingerprint sessions

* Flush buffers before shutdown

* Use old stats when merging

* Remove old session schema

* Fix tests with CH sessions

* Add has_pageviews? to Stats

* Use CH in staging

* Update schema

* Fix test setup
2020-05-21 13:03:39 +03:00
Uku Taht
d1b703f007
Clickhouse (#63)
* Get stats from clickhosue

* Pull stats from clickhouse

* Use correct Query namespace

* Use Clickhouse in unit tests

* Use Clickhouse in stats controller tests

* Use fixtures for unit tests

* Add Clickhouse to travis

* Use Clickhouse session store for sessions

* Add garbage collection to session store

* Reload session state from Clickhouse on server restart

* Query from sessions table

* Trap exits in event write buffer

* Run hydration without starting the whole app

* Make session length 30 minutes

* Revert changes to fingerprint schema

* Remove clickhouse from fingerprint sessions

* Flush buffers before shutdown

* Use old stats when merging

* Remove old session schema

* Fix tests with CH sessions

* Add has_pageviews? to Stats
2020-05-18 12:44:52 +03:00
Uku Taht
3faec27c82 Fix clickhouse buffer inserts 2020-05-07 14:53:06 +03:00
Uku Taht
5d1dc0d27c
Clickhouse DB (#62)
* WIP

* Get all stats from Clickhouse

* Use https dependency

* Do not namespace db tables in hydrate task

* Update hydration task

* Ingest data to clickhouse

* Double-write to both Clickhouse and Postgres

* Add test setup

* Keep old stats module

* Prepare for live ingestion test
2020-05-07 14:28:41 +03:00
Uku Taht
4163d2a9ba
Authentication updates (#54)
* Use separate cookie for shared link auth

* Remove CSRF protection from login and register pages

* Add AuthorizeStats Plug

* Fix tests
2020-04-14 14:04:35 +03:00
Uku Taht
2b50c6cf2a Use correct headers 2020-04-02 13:43:06 +03:00
Uku Taht
d3c770a6da Remove log 2020-04-02 13:38:01 +03:00
Uku Taht
d8936888e6 Inspect headers 2020-04-02 13:32:36 +03:00
Uku Taht
e14001664c Fix tests 2020-03-31 15:01:27 +03:00
Uku Taht
594b085467
Analytics without using cookies (#44)
* Use fingerprints instead of user_id

* Fix tests
2020-03-24 10:50:16 +02:00
Uku Taht
cc5722050a
Show favicons for referrers (#39) 2020-03-03 14:32:32 +02:00
Uku Taht
90a2c67825 Remove debug code 2020-03-02 10:48:11 +02:00
Uku Taht
c0f686b3d0 Fix trailing slash in referrer 2020-03-02 10:47:25 +02:00
Uku Taht
36c08b989d Debug bad referrer 2020-03-02 09:54:00 +02:00
Uku Taht
7d5443396f Use fingerprint 2020-02-27 12:04:13 +02:00
Uku Taht
aaa9c67e47
Add fingerprint sessions (#36) 2020-02-27 11:46:48 +02:00
Uku Taht
5495e53454
Backfill fingerprints for returning users (#33) 2020-02-12 14:42:34 +02:00
Uku Taht
ed3110a6ea
Introduce initial referrer and initial referrer source (#32)
* Introduce initial referrer and initial referrer source

* Add test
2020-02-12 11:11:02 +02:00
Uku Taht
7bb5d98956 Remove raw fingerprint 2020-02-11 11:02:44 +02:00
Uku Taht
759c21fff0 Use cloudflare header for fingerprint 2020-02-11 10:52:59 +02:00
Uku Taht
16b2dbf023 Store raw fingerprint for testing 2020-02-11 10:10:53 +02:00
Uku Taht
158dd3f822 Strip www from domain 2020-02-10 16:07:34 +02:00
Uku Taht
0875d57b59 Track fingerprint 2020-02-10 16:00:19 +02:00
Uku Taht
8a4d0a91d1
WIP: New UI updates (#19)
* WIP: New UI idea

* Remove hover styling if things are not clickable

* Improve mobile view

* Remove dead code

* Fade in tabs

* Update landing page with new style

* Fix countries test

* Fix alignment in conversions report
2020-02-10 15:17:00 +02:00
Uku Taht
3679a7351f Remove www from domain 2020-02-05 13:53:57 +02:00
Uku Taht
7dbbc8ba22
Configurable site id (#30)
* Use site id instead of hostname for events

* Use site id in domain status check

* Revert change to tracking module

* Catch more places where link generation needed updating

* Rename site_id to domain

* Drop hostname index from events
2020-02-04 15:44:13 +02:00
Uku Taht
423db302f7 Only count new visitors in referrer report 2020-01-16 15:40:06 +02:00
Uku Taht
254d34d0a4 Trim trailing slash from referrer 2020-01-08 10:53:08 +02:00
Uku Taht
29ae3a2c21 Show bounce rate for referrers and pages 2020-01-07 14:53:04 +02:00
Uku Taht
df1ba2cd82 Remove session length from the UI 2020-01-06 15:31:43 +02:00
Uku Taht
980275092d Show bounce rate and session length 2020-01-06 14:36:55 +02:00
Uku Taht
ddac5685a2 Add unload event to the tracking script 2019-12-18 16:06:01 +08:00
Uku Taht
f7d752988a Add sessions 2019-12-18 15:23:53 +08:00
Uku Taht
5ee2d908f2 Show conversion rate in the top stats when filtered for goal 2019-11-25 17:17:18 +08:00
Uku Taht
3b533fae21 Compare graph when filtering for goal 2019-11-25 11:37:50 +08:00
Uku Taht
c540c70e87 Show conversion rate when filtering for goal 2019-11-22 16:37:44 +08:00
Uku Taht
f71f98a521 Test Google referrer drilldown 2019-11-20 16:42:45 +08:00
Uku Taht
96abfc4ea1 Click on goal to see conversions for it 2019-11-20 13:48:27 +08:00
Uku Taht
e8f20e67cc
React (#17)
* Load dashboard with react

* Rename stast2 to dashboard

* Save timeframe on the frontend

* Implement current visitors

* Implement comparisons

* React to route changes

* Add modals

* Show number of visitors on hover

* Show 'Today' for today

* Add 30 days

* Show referrer drilldown

* Arrow keys to go back and forward

* Improve comparisons UI

* Fix dropdown when clicking on it

* Verify API access in a memoized fashion

* Test API access

* Test stats through controller

* Move map formatting from stats controller to stats

* Remove unused code

* Remove dead code from query

* Remove dead code from stats templates

* Add stats view test back in

* Render modal inside the modal component

* Implement google search terms

* Add explanation for screen sizes

* Separate dashboard JS from the app js
2019-11-19 12:30:42 +08:00
Uku Taht
d687663b80 Use new /event endpoint for pageviews 2019-10-31 11:49:46 +08:00
Uku Taht
3ea64de42b Add name to events 2019-10-25 14:06:11 +08:00
Uku Taht
0c453be08c Rename pageviews to events 2019-10-24 14:58:17 +08:00
Uku Taht
71edf2c86d Remove unused fields from pageviews 2019-10-15 15:34:41 +08:00
Uku Taht
d2855210de Fix bug with screen size 2019-09-04 10:50:56 +01:00
Uku Taht
779d64e19a Initial commit 2019-09-02 12:29:19 +01:00