Commit Graph

1956 Commits

Author SHA1 Message Date
Uku Taht
f10c67e0e7 Add custom.plausible.io to checkly monitoring 2023-01-09 17:59:44 +02:00
RobertJoonas
13a05b4e1e
Change metric labels for realtime with goal filter (#2560)
* change metric labels for realtime with goal filter

* changelog

* fix formatting
2023-01-09 09:31:55 +02:00
Uku Taht
1772ddff17
Add Checkly (#2520)
* Add Checkly Terraform config

* add deployment workflow

* use pagerduty instead of email for notifications

* use terraform cloud backend

* update variable declaration

* rename checkly check group

* update syntax

* test trigger

* Revert "test trigger"

This reverts commit 333e82beac.

* run a single job at a time

Co-authored-by: Cenk Kücük <c@cenk.me>
2023-01-05 20:25:54 +01:00
Uku Taht
51fc9c0945
Refine parallel application-side joins (#2564)
* Consolidate task timeouts, incrase to 15s

* Use Task.async_stream for parallel Clickhouse queries

* Propagate Opentelemetry context to child Task process
2023-01-05 12:34:01 +01:00
David Janda
f949278d39
Convert list elements to button elements for stats tabs in order to enable keyboard navigation (#2526) 2023-01-05 11:45:19 +02:00
Adam Rutkowski
89da676215
Update clickhousex (start using Mint and proper pool management) (#2502)
* Pin clickhousex to @ruslandoga's rebase branch

* Bump lockfile

* Use HTTP pings

* Update lockfile

* Pin to master
2023-01-05 09:38:52 +01:00
Vini Brasil
4503895d0a
Fix breakdown API pagination when using event metrics (#2562)
* Fix breakdown API pagination when using event metrics

This commit fixes a bug where the subsequent breakdown API pages had
the same items as the first page. The fix sorts the underlying
ClickHouse query by timestamp, keeping the same order between requests,
as we use OFFSET/LIMIT pagination.

* Fix repeated results assertion

* Add different ORDER BY to each breakdown property
2023-01-04 22:14:40 -03:00
Adam Rutkowski
44afbaa2f9
Wrap paddle actions with DB transactions (#2558)
* Wrap paddle actions with DB transactions

* Bow to credo

* Make the paddle transactions crash when applicable

* s/changeset/details
2023-01-03 16:14:50 +01:00
Uku Taht
1785653b1e
Ignore unknown countries (#2556)
* Ignore XX and T1 countries

* Add fallback if country_code=nil

* Lookup city overrides directly in CityOverrides module

* Changelog

* Add empty moduledoc

* Remove redundant comment
2023-01-03 10:35:23 -03:00
RobertJoonas
47e21121db
Realtime dashboard improvements (#2445)
* add a new realtime-update-timer module

* hook to the new 'tick' event in ListReport for auto-updates

This commit fixes the bug where all reports using the `ListReport` component did not
auto-update in realtime mode. Those reports are:

- Pages (Top / Entry / Exit)
- Locations (Countries / Regions / Cities)
- Devices (Screen Sizes / Browsers + versions / OS-s + versions)

* fetch data for ListReports only when scrolled into view

* refactor fetching data in ListReport

* refer to one source of truth for utm tags

* make the 'All' tab in Sources auto-update

* make all UTM tabs in Sources auto-update

* fetch UTM data only when scrolled into view

* auto-update Referrers with the new timer

* auto-update google search terms

* auto-update Conversions

* make countries map auto-update

* auto-update visitor-graph and top stats with new timer

* use new tick event for current visitors (in Historical)

* remove the old timer class

* update changelog

* Visual improvements to automatic realtime updates (#2532)

* minor consistency fix for text color in dark mode

* use FlipMove in goal conversions report

* use FlipMove in ListReports

* set main graph and top stats loading state correctly

* refactor isIntervalValid function

* enforce intervals are valid when set and stored

* remove duplicate data fetching on interval change

Fetching new data is handled by the `fetchGraphData` callback in `updateInterval`

* refactor updateMetric function

* make it clearer why 'metric' can be a faulty value

* extract 'query' and 'site' variables from 'this.props'

* reset interval state only when period is changed

The 'maybeRollbackInterval' function was also used to fetch data. This commit replaces
all those function calls with 'fetchGraphData' which better describes the actual behavior.
We should only worry about rolling back the interval if 'query.period' has changed.

This commit also stops the graph from flickering when it is updated in realtime.

* update names of two variables

* remove unnecessary negation

* make collapsed graph state more explicit

* consider stored invalid intervals when graph mounts

* fix not showing loading spinner regression

* remove interval state from VisitorGraph (#2540)

* Realtime prop breakdown (#2535)

* disable load more in realtime mode

* extract doFetch function

* separate fetchPropBreakdown and fetchNextPage functions

* subscribe for auto-updates in realtime

* improve readability with function name changes
2023-01-02 17:42:57 +02:00
Vini Brasil
22e2ae1844
Add OTEL tracing to breakdown functions (#2544)
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
2023-01-02 11:53:02 -03:00
Vini Brasil
b6d30019ef
Cascade delete sent_renewal_notifications table when user is deleted (#2549)
* Cascade delete sent_renewal_notifications table when user is deleted

This commit fixes a bug when deleting a user would trigger a constraint
error.

* Update CHANGELOG.md
2023-01-02 11:46:18 -03:00
Vini Brasil
10a9767912
Remove ingestion requests and Oban queries from OTEL (#2543) 2022-12-27 11:14:36 +01:00
Adam Rutkowski
5de43b758d
Run tests in async mode where applicable (#2542)
* Set pg pool size for MIX_ENV=test

* Include slow tests in CI run

* Exclude slow tests by default

* Mark tests slow/async where applicable

* Restructure captcha mocks

* Revert async where env is relied upon

* Add --max-failures=1 to CI run

* Set warnings as errors

* Disable async where various mocks are used

* Revert "Disable async where various mocks are used"

This reverts commit 2446b72a29.

* Disable async for test using vcr
2022-12-26 10:20:29 -03:00
Vini Brasil
5152e8d416
Reject events with long URIs and data URIs (#2536)
* Reject events with data URIs

* Reject events with URIs longer than 2,000 characters

* Update CHANGELOG.md
2022-12-21 15:53:04 +02:00
Marko Saric
ecd3ff276b
Some changes to the copy to keep it consistent with the main site 2022-12-21 12:32:48 +01:00
Vini Brasil
0e87b489f6
Return empty list when breaking down by event:page without events (#2530)
* Return empty list when breaking down by event:page without events

This commit fixes a bug with pagination where breaking down by event:page
would always return results despite pagination.

Closes #2255

* Update CHANGELOG.md
2022-12-19 13:49:03 +01:00
Uku Taht
a37b3433d7
Always show direct traffic in sources reports (#2531)
* Remove show_noref behaviour

Removes query param show_noref which was used from React to control
whether to show Direct / None traffic or not. The show_noref behaviour
was untested previously.

Closes #2523

* Add changelog entry

* Fix tests

* Removed files I did not mean to check in :)
2022-12-19 13:40:21 +01:00
Uku Taht
8f20bf0ab2
Add metric ID to HTML (#2518) 2022-12-19 12:21:15 +01:00
Vini Brasil
478e0c6990
Handle GA authentication errors (#2505) 2022-12-07 23:32:14 -03:00
ruslandoga
f95e67bec8
Release v1.5.1 (#2504)
### Changes

Preparations for `v1.5.1` release.

### Tests
- [x] This PR does not require tests

### Changelog
- [x] Entry has been added to changelog

### Documentation
- [x] This change does not need a documentation update

### Dark mode
- [x] This PR does not change the UI
2022-12-06 15:39:09 +02:00
ruslandoga
138e7c06d6
add BUILD_METADATA fallback when parsing (#2503)
### Changes

This PR adds a fallback to empty build metadata when BUILD_METADATA
contains invalid JSON.

Example `warning` log for `BUILD_METADATA={...}`:

```
20:57:57.872 [warning] failed to parse $BUILD_METADATA, reason: ** (Jason.DecodeError) unexpected byte at position 1: 0x2E (".")
```

Fixes https://github.com/plausible/analytics/issues/2491

### Tests
- [x] This PR does not require tests

### Changelog
- [ ] Entry has been added to changelog

### Documentation
- [x] This change does not need a documentation update

### Dark mode
- [x] This PR does not change the UI
2022-12-05 17:59:16 +02:00
ruslandoga
f9ab2650b3
Release v1.5.0 (#2490)
### Changes

Preparing for `v1.5.0` release.

### Tests
- [x] This PR does not require tests

### Changelog
- [x] Entry has been added to changelog

### Documentation
- [ ] [Docs](https://github.com/plausible/docs) have been updated

### Dark mode
- [x] This PR does not change the UI
2022-12-02 16:57:59 +02:00
Vini Brasil
9b32dace3e
Add Content-Security-Policy to favicon Plug (#2489) 2022-12-02 06:33:24 -03:00
Serafeim Papastefanos
fd6ebf7598
Trim the value in case it contains newlines (#2480)
### Changes

Hello friends, I'm trying to self-host plausible on my server and have
observed that if I add a file named (for example) `secrets/LISTEN_IP` to
configure the bind IP I'll get the following error:

```
Compiling 1 file (.ex)
** (RuntimeError) Invalid LISTEN_IP '172.19.130.83
' error: :einval
    /home/serafeim/plausible/config/runtime.exs:20: (file)
```

(notice that I get the same error when configuring the DATABASE_URL or
whatever comes first)

It seems that for whatever reason the File.read! will insert a newline
at the end of the file. Notice that I have double checked the file with
vim and *it does not contain a newline at the end* or at least I don't
know *how* to make the file not have a newline at the end.

Thus I'm adding a `String.trim()` there to fix that thing; even if I am
doing something wrong and my files *do* contain a newline at the end,
adding a `String.trim()` there definite won't hurt :)

Below you'll find a checklist. For each item on the list, check one
option and delete the other.

### Tests
- [ ] Automated tests have been added
- [x] This PR does not require tests

### Changelog
- [ ] Entry has been added to changelog
- [x] This PR does not make a user-facing change

### Documentation
- [ ] [Docs](https://github.com/plausible/docs) have been updated
- [x] This change does not need a documentation update

### Dark mode
- [ ] The UI has been tested both in dark and light mode
- [x] This PR does not change the UI
2022-11-30 10:12:48 -03:00
Vini Brasil
96182da57d
Update bamboo_postmark to fork repository (#2477)
This commit updates mix.exs to resolve bamboo_postmark to our fork. The
fork encodes names with quotes when building e-mails, adding support for
special names with commas and quotes. Related to
plausible/bamboo_postmark#1.

Closes #1885
2022-11-29 17:48:44 +02:00
Vini Brasil
90786cb3bc
Remove intervals feature flag (#2474)
This commit removes the intervals feature flag, as this feature has
already been released to 100% of the users.
2022-11-29 10:12:36 -03:00
Adam Rutkowski
467317f24e
Fix: application startup, event timestamps (#2479)
### Changes

This PR:
- pushes PromEx to the bottom of supervision stack to avoid Endpoint
instrumentation failure
- ensures the site cache is ready by exposing it through the health
check endpoint
- fixes event timestamps being calculated at compile time, with
regression unit and integration tests

### Tests
- [x] Automated tests have been added
- [ ] This PR does not require tests

### Changelog
- [ ] Entry has been added to changelog
- [x] This PR does not make a user-facing change

### Documentation
- [ ] [Docs](https://github.com/plausible/docs) have been updated
- [x] This change does not need a documentation update

### Dark mode
- [ ] The UI has been tested both in dark and light mode
- [x] This PR does not change the UI
2022-11-29 10:46:49 +01:00
Adam Rutkowski
356575ef78
Gatekeep ingestion pipeline (#2472)
* Update Sites.Cache

So it's now capable of refreshing most recent sites.
Refreshing a single site is no longer wanted.

* Introduce Warmer.RecentlyUpdated

This is Sites Cache warmer that runs only for
most recently updated sites every 30s.

* Validate Request creation early

* Rename RateLimiter to GateKeeper and introduce detailed policies

* Update events API tests - a provisioned site is now required

* Update events ingestion tests

* Make limits visible in CRM Sites index

* Hard-deprecate DOMAIN_BLACKLIST

* Remove unnecessary clause

* Fix typo

* Explicitly delegate Warmer.All

* GateKeeper.allwoance => GateKeeper.check

* Instrument Sites.Cache measurments

* Update send_pageview task to output response headers

* Instrument ingestion pipeline

* Credo

* Make event telemetry test a sync case

* Simplify Request.uri/hostname handling

* Use embedded schema, apply action and rely on get_field
2022-11-28 15:50:55 +01:00
Vini Brasil
e2563acf36
Standardise dropdown icon margins (#2473) 2022-11-25 09:12:52 -03:00
Vini Brasil
b9367941f0
Add more data to pageview seeds (#2471) 2022-11-25 12:53:33 +02:00
Vini Brasil
ffbfb14714
Pass interval param to CSV export (#2469)
This commit adds the interval param to the CSV export client-side
request.
2022-11-25 10:53:22 +02:00
Vini Brasil
32a38fac7a
Add keyboard navigation to intervals (#2470)
* Create function to handle keybindings

* Add <i> keybinding to intervals
2022-11-25 10:51:43 +02:00
ruslandoga
77cca04417
fix ownership transfer when is_selfhost=true (#2455)
* fix ownership transfer when is_selfhost=true

* add changelog entry
2022-11-24 16:18:29 +02:00
Vini Brasil
994e7d09de
Parse event URL and domain in Plausible.Ingestion.Request (#2351)
* Parse event URL in Plausible.Ingestion.Request

* Parse event domain in Plausible.Ingestion.Request

* Rework ingestion pipeline processing (#2462)

* Rework ingestion pipeline processing

So that Request can have multiple domains and
based on that each event is processed uniformly.
The build_and_buffer/1 function now returns an
accumulator with all the dropped/buffered events
for further inspection.

* Reduce function complexity

* Don't chain struct fields to check for an empty host

* Separate referrer and utm tags

* Fix up `with` clause, credo was right cc @vinibrsl

Co-authored-by: Adam Rutkowski <hq@mtod.org>
2022-11-23 14:05:44 +01:00
Adam Rutkowski
8b5ae9baaa
Migration: index Sites.updated_at (#2467) 2022-11-23 14:26:50 +02:00
Vini Brasil
75fcd8d611
Fix container is not a DOM Element error (#2463) 2022-11-22 11:21:05 -03:00
Vignesh Joglekar
497a52c10a
Add support for intervals (#1574)
Co-authored-by: Vini Brasil <vini@hey.com>
Co-authored-by: Vignesh Joglekar <hey@vigneshjoglekar.com>
2022-11-22 09:50:58 -03:00
Adam Rutkowski
ed9e03ae14
Outline basic rate-limiting interface (#2452)
* Outline basic rate-limiting interface

* Fixup typespecs

* Address empty cache race condition on refresh (#2457)

Ref https://github.com/plausible/analytics/pull/2444/files#r1023751222

* Fix up test case: make sure the cache is refreshed
2022-11-21 15:54:47 +01:00
RobertJoonas
135471c32e
Add tagged-events script extension (#2333)
Adds a new script extension that allows tracking interactions with specific HTML elements on a website. For example - to track link clicks on one specific `<a>` element, you can tag it like this:

```html
<a href=... class="plausible-event-name=<your_event_name>">
```

And you can also tag the link with custom property names and values:

```html
<a href=... class="plausible-event-name=<your_event_name> plausible-event-<your_custom_prop>=<your_value>">
```

Tagging a link as above will send a custom event with the given name and props, if a `click` or `auxclick` browser event happens, and targets the link element.

The tracking behavior is somewhat different based on the HTML element type:  

- `<a>` 
  - triggers on `click` and `auxclick` events
  - intercepts navigation based on the same rules as `outbound-links` and `file-downloads`
- `<form>`
  - triggers on `submit` event
  - always intercepts navigation (calls `form.submit()` after preventing default and sending the Plausible event)
- other (`<img>`, `<button>`, `<span>`, `<div>`, `<h2>`, etc ...)
  - triggers on `click` and `auxclick` events
  - does not prevent default to intercept possible navigation. Simply calls Plausible with the event name and props read from the element class list.
2022-11-21 16:17:44 +02:00
Vini Brasil
3bedf9281c
Seed database with pageviews (#2449)
* Seed database with pageviews

This commit adds basic support for database seeding useful for testing,
especially dashboard changes, like intervals.

It creates two years of pageviews with random timestamps. There is lot
of room for improvement, such as adding sources, entry pages,
geolocation, devices, custom events, but this already helps us with
testing.

* Update CONTRIBUTING.md file
2022-11-17 21:46:42 -03:00
Marko Saric
1a9eb094c8
New screenshot and updates to the copy 2022-11-17 16:55:17 +01:00
Adam Rutkowski
e522a2d7c1
Stop relying on npm-bin (hard deprecated) (#2446) 2022-11-16 16:35:08 +02:00
Adam Rutkowski
3fe6a95696
Update sites cache implementation (#2444)
* Allow refreshing a single site cache + clear cache on prefill

* Reorganize Site.Cache tests with describe blocks

* Tidy up Cache tests

* Make sure the cache is cleared on (p)re-fill

* Allow process name customization in Cache.Warmer

* s/Cache.prefill/Cache.refresh

* Unify Cache refresh instrumentation

* Apply credo suggestion: change `with` to `case`

* Update typespecs to pass dialyzer
2022-11-16 15:04:53 +01:00
Adam Rutkowski
457a558471
Kick off sites by domain cache implementation (#2434)
* Implement sites by domain caching interface + warmer

* Add test

* Implement hit rate interface

* Add moduledocs

* Fix up typespec

* s/warmer/warmer_fn

* Extract measure_duration/2

* Fix up typespec

* Log errors and return nil on cache internal errors

* Fix up non-existing cache test

* Retrieve specific db columns when pre-filling the cache

* Reduce the subset of fields retrieved from the DB

See 63f3c6233d (r89871536)
2022-11-16 10:06:23 +01:00
Vini Brasil
bea227d71b
Clean graph JavaScript code (#2441)
This commit lints JavaScript files, removes commented-out code, and
moves 3 constants to `graph-util.js`. This is part of the Intervals
work, in order to make the original pull request smaller and release
this feature incrementally.

Related: #1574

Co-authored-by: Vignesh Joglekar <hey@vigneshjoglekar.com>
2022-11-15 11:38:39 -03:00
Vini Brasil
9c98a3f2e8
Add API support for intervals (#2417) 2022-11-14 18:41:51 -03:00
RobertJoonas
47bf003c29
Merge pull request #2431 from plausible/fix-email-links
Fix email links
2022-11-10 16:45:43 +02:00
Robert
061d041213 small addition to spike_notification email 2022-11-10 16:08:16 +02:00
Robert
27e8d8db3c remove redundant test 2022-11-10 15:55:33 +02:00