Commit Graph

1738 Commits

Author SHA1 Message Date
Adrian Gruntkowski
5bf59d1d8a
Implement adjusting imported date range to actual and existing stats (#3943)
* Implement adjusting imported date range to actual and existing stats

* Drop redundant prefix from import list entries

* Make pageview numbers in imports list formatted for readability

* Test and improve date range cropping

* DRY UA and GA4 stats start and end date API calls

* Extend UA/GA import controller tests and improve error handling

* refactor finding longest open range without existing data

* Fix typo in test description

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>

* Rename `open_ranges` to `free_ranges`

---------

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
2024-03-28 09:32:41 +01:00
ruslandoga
c263df5805
CSV imports (UI) (#3845)
* add basic ui

* remove TODO

* credo

* allow folder upload

* redirect external

* mention folder, use folder icon for file picker

* back to multiple file upload

* mention zip

* escape dots in archive filename
2024-03-26 12:55:14 +01:00
hq1
b31433a7bf
Ensure all the react container attributes are strings (#3948) 2024-03-26 11:01:59 +01:00
hq1
edf70d14b6
Use sessionStorage for "dashboard first launch" banner tracking (#3892)
* Use sessionStorage for offer e-mail report banner tracking

Keeping it within the cookie is problematic, as the banners don't
expire and overflow the cookie with data when enough new sites
are added.

Ref https://github.com/plausible/analytics/issues/3762

* Update changelog

* Extract a component

* Make is_dbip evaluate to quoted boolean
2024-03-26 09:49:15 +01:00
hq1
7523abe93e
Add metrics to ingestion pipeline (#3927)
* Add metrics to ingestion pipeline

* Format

* Format

* Update buckets

* Credo
2024-03-26 09:42:48 +01:00
Karl-Aksel Puulmann
4af7019011
Ignore sessions without entry/exit pages when breaking down entry/exit pages (#3933)
* Ignore sessions without entry/exit pages when breaking down entry/exit pages

* Update stats controller tests to have more realistic test data (pageview followed by event)
2024-03-26 09:01:07 +02:00
hq1
2fae0146a4
Reapply 3918 (#3940)
* Reapply "Pages shield (#3918)"

This reverts commit 33b5c10654.

* Make the FF check work against the site actor
2024-03-25 10:36:22 +01:00
hq1
9989ce6927
Migration for 3918 (#3939)
* Revert "Pages shield (#3918)"

This reverts commit 53f94a9f82.

* Migration: Shield page rules
2024-03-25 10:19:50 +01:00
hq1
53f94a9f82
Pages shield (#3918)
* Migration: Shield page rules

* Add Ecto schema for Page Rules

* Add Page Rule cache

* Fix typo

* BTW: Use already imported function

* Extend Shields context interface + split existing tests

* Ingestion: filter matching patches + refactor shield actions

* Add LV section for adding Page Rules

* Validate max page path length

* Put Pages Shield behind a feature flag

* Update CHANGELOG

* Update docs link anchor

As per https://github.com/plausible/docs/pull/477

* Update lib/plausible/shields.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Update lib/plausible_web/live/shields/page_rules.ex

Co-authored-by: ruslandoga <doga.ruslan@gmail.com>

* Update lib/plausible_web/live/shields/page_rules.ex

Co-authored-by: ruslandoga <doga.ruslan@gmail.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
Co-authored-by: ruslandoga <doga.ruslan@gmail.com>
2024-03-25 09:48:56 +01:00
Adrian Gruntkowski
ba5b80a8c0
Add label to site imports and populate it (#3914) 2024-03-22 11:17:02 +01:00
RobertJoonas
d6e1e8bebd
Put total conversions on the graph + goal-filtered CSV export improvements (#3929)
* Add validation for the events metric in main_graph

* Test the already existing events metric support in main-graph

* Put total conversions on the graph

* extract main_graph_csv function (refactor only)

* add total_conversions and conversion_rate to goal-filtered visitors.csv

* update changelog
2024-03-22 09:35:23 +00:00
Uku Taht
fd879eeb16
Store referrers from android apps (#3715)
* Store referrers from android apps

* Add test for unknown referrer protocol

* Store android referrer protocol
2024-03-21 17:45:34 +02:00
RobertJoonas
c32779a3e5
Timeseries for conversion rate (#3919)
* add conversion rate to Stats API timeseries

* make sure CR can be queried as the only metric

* add a test asserting zeros are returned

* add tests for filtering by other properties at the same time

* Remove unnecessary validation of params

1. It doesn't make to validate `interval` (and its granularity) in all
   endpoints. It's only relevant for the main graph.

2. The plug (renamed to `date_validation_plug`) already makes sure that
   the dates are validated. No need to call the same function again in
   Top Stats and Funnel endpoints.

* add metric validation to main graph

* Add tests for main graph API

* put conversion rate on the graph

* update changelog

* Add revenue metrics into metrics.ex

* make fn private

* avoid setting graph metric to visitors in goal-filtered view
2024-03-21 13:58:00 +00:00
Adrian Gruntkowski
d6e81670e4
Unify UA and GA4 import flow into one (#3888)
* Unify GA4 and UA import flow into one

* Clean up property and view data retrieval via Google HTTP APIs

* Turn `Map.get` into `Map.fetch!` in API response processing code

* Bump list account summaries page size limit to max of 200

* Show only views in legacy flow and fix legacy redirect after import start

* Move google analytics import actions tests to a separate module

* Extend Google Analytics controller tests

* DRY up `property?` predicate (h/t @RobertJoonas)
2024-03-21 11:37:10 +01:00
ruslandoga
5f9465614b
Include domain and dates in zip archive filename (#3921)
* include domain and dates in zip archive filename

* adapt to comments
2024-03-21 11:35:42 +01:00
Karl-Aksel Puulmann
32ab138301
Fix issue with name clash (#3925)
Unexpectedly, table.name caused a name clash after CR refactor, so using a unique name
for the output column

Sentry issue: https://sentry.plausible.io/organizations/sentry/issues/5612
2024-03-21 10:11:29 +00:00
Karl-Aksel Puulmann
c219652dae
Re-apply Move conversion_rate logic from elixir to clickhouse (#3924)
* Revert "Revert "Move conversion_rate logic from elixir to clickhouse (#3887)"…"

This reverts commit 253fb5d67d.

* Fix issue with missing columns

The issue came from refactoring event:goal UNION ALL logic and trying to move
name select from first to last. If any other tables were joined, the incorrect
item would be used as an array index, causing this issue.

Added a relevant test.
2024-03-21 10:48:41 +02:00
Karl-Aksel Puulmann
253fb5d67d
Revert "Move conversion_rate logic from elixir to clickhouse (#3887)" (#3923)
This reverts commit 1909743b90.
2024-03-21 09:53:31 +02:00
Karl-Aksel Puulmann
1909743b90
Move conversion_rate logic from elixir to clickhouse (#3887)
* Separate out query building from pagination/execution logic.

* Refactor pageview_goals breakdown query, removing index column from results

* Remove zip_columns logic

* Use common pagination util

* Do everything in a single query for breakdowns for goals

* Order in DB

* Make sure column order is identical

* Calculate CR within the goal breakdown query

* Calculate CR for property breakdowns

* WIP: Calculate group CR

* CR with order_by

* Compatibility fix

* Import Ecto.Query and cleanup

* handle total_visitors the same way as add_percentage

* Handle conversion_rate in aggregate.ex

* Solve rebase fail

* Simplify maybe_add_group_conversion_rate

* Add conversion_rate defaults to 0 test

* Add test for conversion_rate should not be calculated with imported data (failing here and on master)

* Dont include imported data when breakdown by prop or goal

* Remove revenue_nils
2024-03-21 09:38:44 +02:00
ruslandoga
64f35bf8f6
make robots.txt restrictive by default (#3905)
* make robots.txt restrictive by default

* add note about cloud robots.txt
2024-03-19 12:07:30 +01:00
ruslandoga
279e89c693
CSV imports (no UI) (#3895)
* encode/decode date range in filenames

* Update lib/plausible/imported/csv_importer.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Update lib/plausible/imported/csv_importer.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* drop unused functions

* send failure email if there is no data to export

* use PlausibleWeb.Email.mailer_email_from()

* ensure we get dates from minmax date query

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-03-19 12:06:47 +01:00
ruslandoga
dfcc8d794a
Add space after logo on the index page (#3825) 2024-03-19 12:02:30 +01:00
Karl-Aksel Puulmann
02d2256483
Set exit_page only on pageviews (#3870)
* Set exit_page only on pageviews

* Update tests

* Update entry_page on first pageview

* Update CHANGELOG.md
2024-03-18 11:11:15 +02:00
ruslandoga
07b714a143
Update Sentry (#3843)
* update Sentry

* Sentry.HTTPClient.child_spec is now optional

* Sentry.EventFilter is deprecated

* update sentry to 10.2.0

* fix dialyzer warnings
2024-03-18 10:10:20 +01:00
ruslandoga
8c0e94f72c
Fix manual embed width (#3910)
* fix manual embed width

* changelog
2024-03-18 09:37:42 +01:00
ruslandoga
5e74b1cf74
CSV exports (UI) (#3875)
* ui

* fix redirect link

* improve make minio

* use implicit button form for csv export

* add exports_bucket helper

* read S3_EXPORTS_BUCKET

* supply s3_bucket in export_csv job args

* make plausible_minio use unprotected port

* move s3_csv_export queue to base queues

* Update lib/plausible_web/controllers/site_controller.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-03-18 08:52:57 +01:00
hq1
59afa20955
Reapply #3878 + bugfix hit rate tracking (#3891)
* Reapply "Replace caching engine (#3878)" (#3883)

This reverts commit c5881cdc6d.

* Ensure hit rate is tracked on `get_or_store`

* Remove :wx and :observer

* Remove unused deps

* Use `:set` table type
2024-03-14 08:06:12 +01:00
RobertJoonas
e8f3946dde
Fix division by zero in imported queries (#3890)
* prevent division by 0 in merge_imported queries

* Revert "fix bounce_rate change bug (#3886)"

This reverts commit 6eef32a8ff.

After 02aa0b2, we can keep on assuming that bounce rate is always numeric.
2024-03-13 10:37:14 +00:00
Adrian Gruntkowski
4d7d88cfec
Implement basics of GA4 import (#3851)
* Implement LV date input using flatpickr

* Implement basics of GA4 import (very dirty WIP)

* Split Google HTTP API into UA and GA4 specific parts

* Add a quick way to record GA4 API responses

* Add first GA4 import fixtures with GA4 Data API responses

* Extract GA4 and UA specific logic form Google API

* Extract UA and GA4 specific actions to distinct controllers

* Add integration test for GA4 importer

* Update GA4 fixtures

* Test GA4 API

* Add debug logging and fix paginating through API results in in GA4 import

* Revert "Implement LV date input using flatpickr"

This reverts commit c696f8ee39d5702f27015c09a4f079ca124cc7bb.

* Fix note
2024-03-12 18:08:25 +01:00
ruslandoga
5a3072ca21
CSV exports (no UI) (#3836)
* csv exports

* use ex_unit's tmp_dir
2024-03-12 17:27:27 +01:00
RobertJoonas
6eef32a8ff
fix bounce_rate change bug (#3886) 2024-03-12 10:46:23 +00:00
RobertJoonas
7641c66a2b
Stats api time on page (#3858)
* add metric validation + support in aggregate

* add a test ensuring comparison works

* disallow time_on_page with a goal filter

* Return time_on_page as `nil` from aggregate API

In case time_on_page cannot be calculated, we'll return it as `nil` from
the Stats API.

This is to make the behaviour consistent between breakdown and aggregate
endpoints. As for the UI, we'll still continue to report time_on_page as
0 - not changing any UI behaviour as discussed with Marko.

* add tests for time_on_page in event:page breakdown

* update changelog

* invalidate time_on_page with event:name filter

* add the ability to only query time_on_page in page breakdown

We'll need the visitors metric to get the list of pages to calculate the
time_on_page for.
2024-03-12 10:00:32 +00:00
hq1
c5881cdc6d
Revert "Replace caching engine (#3878)" (#3883)
This reverts commit 437a3350ff.
2024-03-12 08:30:16 +01:00
Marko Saric
ca7edd47cb
removing target="_blank" from blog link (#3882) 2024-03-12 08:16:39 +01:00
hq1
437a3350ff
Replace caching engine (#3878)
* Dependencies: swap Cachex for ConCache

* Implement Cache adapter wrapping ConCache

* Implement cache stats tracker, for metrics

* Use Cache.Adapter in Plausible.Cache

Marking the test as not slow anymore

* Use Cache Adapter when tracking sessions

* Use Cache Adapter for UA parsing

* Rename child identifiers - cachex is obsolete now

* Test stats tracking

* Update grafana metrics

* Put all caches under common child specification

* Try less

* Shorten the function delegation path
2024-03-12 07:58:12 +01:00
hq1
bb8a272a36
Fix margins in help guides section (#3881) 2024-03-11 14:52:16 +01:00
hq1
b68e3aa6a2
changes to the in-app footer (clean) (#3880)
* changes to the in-app footer

changes to the in-app footer to focus on product usage links rather than marketing pages

* Format

---------

Co-authored-by: Marko Saric <34340819+metmarkosaric@users.noreply.github.com>
2024-03-11 14:39:18 +01:00
ruslandoga
83a46fb47e
Dockerfile and co updates (#3811)
* wip

* more env, setup user before packages to avoid them picking our uid

* make entrypoint.sh executable by all again

* apparently it's a best practice to have executables be owned by root

* make entrypoint executable in COPY

* stop writing to /app, write to /tmp if needed

* fewer changes

* system user

* keep same style for multiline commands

* fewer changes

* add changelog entry

* fix group assignment for plausible user

* use gid=999

* no home

* no home

* add gecos

* add plausible user to nogroup instead of creating a custom one

* eh

* fewer changes

* fewer changes

* fewer changes

* use PERSISTENT_CACHE_DIR instead of STORAGE_DIR

* ignore more

* cleanup

* remove hex timeout env var

* use ERL_FLAGS=+JMsingle true in public builds

* fallback to /tmp and nest under /tzdata_data/ for tzdata

---------

Co-authored-by: Cenk Kücük <cenk@plausible.io>
2024-03-08 14:33:55 +00:00
Karl-Aksel Puulmann
a9d3c03782
Validate the same metric isnt queried multiple times in external stats API (#3871)
* Validate the same metric isnt queried multiple times in external stats API

Issue: https://3.basecamp.com/5308029/buckets/35611491/card_tables/cards/7161347855

* Changelog entry

* Make credo happy
2024-03-08 10:46:18 +02:00
Karl-Aksel Puulmann
26d41ddbb9
Mutation to populate event session columns (#3844)
* WIP mutation to populate event session columns

* Remove duplication

* report errors, allow_nondeterministic_updates

* use right columns

* Update existing columns instead of session_* ones

* Make dialyzer happy

* Fix issue with passing pre-existing params in

* Logger -> IO.puts

* Use IngestRepo.config for connection settings

* Make dictionary options configurable

* Move allow_nondeterministic_mutations to within the migration

* Solve credo warning about too deep nesting

* Missed logger call

* Pattern matching in function head
2024-03-08 09:27:24 +02:00
Karl-Aksel Puulmann
c6d98397a8
Move add_percentage logic into clickhouse (#3854)
* Remove `add_percentage`, calculate percentages in clickhouse queries

This simplifies querying logic and avoids doing extra queries and avoids
race conditions.

* Remove special none handling from breakdowns, handling percentages correctly

* Add (failing) test showing expected add_percentage behavior for user making multiple sessions

* Update add_percentage behavior to use separate subqueries
2024-03-06 11:08:25 +02:00
Karl-Aksel Puulmann
c60a2faee4
Write event table session columns (#3865)
* Write event table session columns

* Update testing factory rig
2024-03-06 10:59:24 +02:00
Karl-Aksel Puulmann
d5048fd6b4
Stop writing session properties into events table (#3800)
* Refactor: Explicitly add field names to INSERT

This avoids issues when code schema is out of sync with real schema

* Dont write session parameters to events

These would only be stored on first event anyways. Work remains to be done
on tests which have their own helper

* Remove writes to country_code in a test

* Remove old columns from being accessible in elixir code

* Update most tests to use new way of adding session props to events

* Update testing harness

* Update stats controller test

* Update for shield rules

* update breakdown tests

* Fix typing of state for dialyzer

* Drop support for old session attributes code

* Update remaining tests

* cond -> if
2024-03-01 10:53:56 +02:00
hq1
5eb9d724e5
Plugins API: mark data_domain as nullable in capabilities schema (#3840)
* Plugins API: mark data_domain as nullable in capabilities schema

* fixup
2024-02-28 10:26:01 +01:00
Adrian Gruntkowski
39aa81a16f
Implement UI for multiple imports (#3727)
* Create a stub of site settings section for imports and exports

* Use legacy site import indication to determine UA import handling

* Add provisional logos for upcoming import sources

* Stub basics of import page

* Add very rudimentary support for multiple UA imports

* Implement imports list as live view

* Add support for opening LV modal from backend and closing from frontend

* Introduce notion of themes to `button` and `button_link` components

* Add confirmation modal on deleting import

* Swap GA4 logo

* Implement disabled state support for `button_link` component

* Disable export and non-implemented import sources

* Use native starts start date for upper boundary of import time range

* Ensure integrations view uses legacy UA import flow

* Remove unnecessary preload in SiteController

* Remove unnecessary exception for legacy imports

* Move API controller stats tests under PlausibleWeb

* Test listing imports

* Add test for explicit listener setup

* Add tests for legacy flag state in UA importer

* Add test for purging legacy import data

* Add tests for `Sites.native_stats_start_date`

* Test forgetting imports

* Add `Stats.Clickhouse.imported_pageview_counts/1` and fix test flakiness

* Show page view counts on imports list

* Add tests for static imports and exports view

* Adjust button look slightly

* Use `case` instead of `cond`

* Make feature flag customisable per site

* Fix buttons and empty state styling

* Add another import to seeds

* Use JS confirm dialog instead of modal for deletion confirmations

* Revert "Add support for opening LV modal from backend and closing from frontend"

This reverts commit 260e6c753032b451542e24be9edc2118790b5a00.

* Default `legacy` to false when inserting new import jobs

* Drop `method` attribute from `button_link` and `unstyled_link` components
2024-02-28 09:34:04 +01:00
Marko Saric
0fe5682f91
Change the CTA to upgrade now rather than contact (#3837) 2024-02-28 08:17:35 +01:00
hq1
316148e788
Fix settings options mobile (#3838)
* Fix settings options on mobile view

* Fix settings options selected item on mobile

* Format
2024-02-27 17:48:09 +01:00
ruslandoga
f3423aefec
Add csv importer (#3795)
* add csv importer

* make table validation explicit

* update some docs

* improve docs

* add minio container to ci

* more tests

* eh

* continue

* add passing test

* add failing test

* add config test

* add minio to Makefile

* testcontainers

* remove extra whitespace

* explain the implementation a bit

* account for async deletes in tests

* bounces is UInt32

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-02-27 14:19:09 +01:00
hq1
f1b6a672d4
Fix added_by not saving on adding country rules (#3835)
* Fix `added_by` not saving on adding country rules

* Format

* Remove dupe attr
2024-02-27 12:36:16 +01:00
hq1
518cdb3307
Shield: Country Rules (#3828)
* Migration: add country rules

* Add CountryRule schema

* Implement CountryRule cache

* Add country rules context interface

* Start country rules cache

* Lookup country rules on ingestion

* Remove :shields feature flag from test helpers

* Add nested sidebar menu for Shields

* Fix typo

* IP Rules: hide description on mobile view

* Prepare SiteController to handle multiple shield types

* Seed some country shield

* Implement LV for country rules

* Remove "YOU" indicator from country rules

* Fix small build

* Format

* Update typespecs

* Make docs link point at /countries

* Fix flash on top of modal for Safari

* Build the rule struct with site_id provided up-front

* Clarify why we're messaging the ComboBox component

* Re-open combobox suggestions after pressing Escape

* Update changelog

* Fix font size in country table cells

* Pass `added_by` via rule add options

* Display site's timezone timestamps in rule tooltips

* Display formatted timestamps in site's timezone

And simplify+test Timezone module; an input timestamp converted
to UTC can never be ambiguous.

* Remove no-op atom

* Display the maximum number of rules when reached

* Improve readability of remove button tests

* Credo

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-02-27 12:08:13 +01:00
Cenk Kücük
8a8df685a7
Update country code for anonymising VPN services (#3766)
* Update country code for anonymising VPN services

* Update to latest
2024-02-27 09:42:41 +00:00
RobertJoonas
dd428430f5
Query imported data for views_per_visit (#3830)
* query imported data for views_per_visit

* changelog update
2024-02-26 15:54:49 +00:00
hq1
c822f8de1e
Hide user name overlay on mobile (#3829) 2024-02-26 10:58:32 +01:00
Cenk Kücük
0f0f604d2e
Branding stuff (#3816)
* Replace footer text

* add COPYING.txt file

* Add new logos

* Use new logos in all layouts

* New logos

* Check license key on startup

* Bypass license check when Mix.env == :dev

* Use new logos with smaller wordmarks

* Add generic logo_path/1 function

* Use new favicons everywhere

* Bypass license check in test env

* Use sha256 for license key hash

* Mix.env -> config_env()

* Use Mix.evn at compile time rather than runtime

* Mix format

---------

Co-authored-by: Uku Taht <uku.taht@gmail.com>
2024-02-23 13:35:22 +02:00
hq1
bca617d0bf
String.to_atom/1 is safe to do at compile-time (#3813)
otherwise random compile-time errors may occur
2024-02-22 19:51:21 +00:00
RobertJoonas
52f584efa9
Group os_version by os (#3806)
* improve test

* add os to os_version breakdown

...and add operating_system_versions.csv to the CSV export

* fix conversion rate for os_version breakdown

* update changelog

* fix existing CSV tests

* use case instead of cond
2024-02-22 15:58:50 +00:00
Karl-Aksel Puulmann
8ba2b934b3
Add data migration for moving sessions_v2 table to VersionedCollapsingMergeTree (#3802)
* Add data migration for moving to VersionedCollapsingMergeTree

This has been tested locally and partially on staging. Still requires a bit of work to verify.

Verification query:

```
SELECT main._partition_id, tmp.count, main.count
FROM (
SELECT _partition_id, count() AS count
FROM sessions_v2_tmp_versioned
GROUP BY _partition_id
) AS tmp
FULL OUTER JOIN (
SELECT _partition_id, count() AS count
FROM sessions_v2
GROUP BY _partition_id
) AS main
ON (tmp._partition_id == main._partition_id)
ORDER BY main._partition_id
```

* Add an early exit to migration

* cluster? extract common code
2024-02-22 09:54:39 +02:00
RobertJoonas
d74b1d5e60
Reapply conversion rate into Stats API + bugfixes (#3805)
* Revert "Revert api conversion rate (#3789)"

This reverts commit 8e8790dd30.

* fix browser_version CR breakdown bug

* changelog bugfix

* inspect data structures before sending to sentry
2024-02-21 15:53:05 +00:00
hq1
6035618213
Add GET /capabilities to Plugins API (#3808)
* Add `GET /capabilities` to Plugins API

It aims to:

 - help the client verify the data-domain the token is associated with
 - list all the features available for the site's owner
   (and therefore determine availability of the subset of those for the current
   Plugins API caller)

The endpoint does not require authentication, in the sense that it'll
always respond with 200 OK. However when the token is provided,
a verification lookup is made.

* Remove IO.inspect() call

* Credo

* Aesthetics

* s/send_resp/send_error/

* Call preload just once
2024-02-21 12:41:56 +01:00
Karl-Aksel Puulmann
5cdca6f408
Remove code that updates session browser/geo/OS attributes (#3796)
This will makes it impossible to store session attributes on events.

Looking at production data also revealed no cases where these updates
are effective.
2024-02-20 09:35:12 +02:00
hq1
eceac8afd5
Allow inviting users who are members already (#3797)
* Allow e-mail exclusion in team members quota

* Exclude invitee from quota on invitation create

* Enable invitation submission but report errors on quota violation

* Use a single interface for team members quota

* Check the `Keyword.validate/2` result

* Update test/plausible_web/controllers/site/membership_controller_test.exs

Co-authored-by: Uku Taht <Uku.taht@gmail.com>

---------

Co-authored-by: Uku Taht <Uku.taht@gmail.com>
2024-02-19 12:12:31 +01:00
RobertJoonas
8e8790dd30
Revert api conversion rate (#3789)
* Revert "Unify percentage change for CR and bounce_rate (#3781)"

This reverts commit a6b1a6ebc7.

* Revert "Bring Stats API up to speed: Add `conversion_rate` to Aggregate and Breakdown (#3739)"

This reverts commit 672d682e95.
2024-02-15 17:43:35 +00:00
RobertJoonas
a6b1a6ebc7
Unify percentage change for CR and bounce_rate (#3781)
* Fix conversion rate change calculation

The change in conversion rate should be calculated similar to bounce rate.
For example, an increase of 25% -> 50% should not be a 100% change, but
a 25% change instead.

* Use the same comparison function in Stats API and dashboard API

This commit fixes a bug where the percentage change reported by the Stats
API is different from the one returned by the internal dashboard API.

* changelog update
2024-02-15 12:10:08 +00:00
hq1
926de4dd10
Experimental session count (#3786)
* WIP

* Allow `experimetnal_session_count` request serialization

* Extend `Plausible.Stats.Query` with `experimental_session_count` flag

* Add `FunWithFlags` actor implementation for `Site`

* Change the way sessions are retrieved

* Remove redundant test

* Format

* Update the test

---------

Co-authored-by: Uku Taht <uku.taht@gmail.com>
2024-02-15 12:21:07 +01:00
RobertJoonas
672d682e95
Bring Stats API up to speed: Add conversion_rate to Aggregate and Breakdown (#3739)
* disable event metric with include_imported in every case

* add missing test for metric validation

* refactor metric validation functions

* implement conversion_rate metric validation

* move calculate_cr function into Stats.Util

* Refactor: Move aggregate CR logic into Stats.aggregate

* define atoms to exist

* Ensure that CR does not depend on visitors being queried

If 'visitors' are already queried, we'll use that value. Otherwise we'll
need to make another query to fetch it.

* confirm Stats API aggregate supports CR (tests only)

* small refactor

This is the only 'event_property' left after pattern matching on all
others in the function clauses defined above.

* Make it possible to optionally query conversion_rate

...in breakdown queries (excluding goal and custom prop breakdown)

* A little refactor asking for revenue metrics

1. The `@revenue_metrics` module attribute is an empty list on full build
   anyway
2. We don't need to query for revenue metrics if there are no revenue goals
   returned in the given query (even if revenue goals exist in site.goals)
3. Revenue metrics are already dropped in prop breakdown without a goal
   filter via (get_revenue_tracking_currency/3)

* Make it possible to optionally query conversion_rate (continuation)

... also from a custom prop and goal breakdown

* Frontend adjustments to the Locations report

* Display conversion rate in Regions and Cities (ListReport view)
* Display total conversions, conversions (visitors), and CR in the
  "Details" modals of Countries, Regions, and Cities
* Move the percentage into a separate column in the Countries details table

* confirm Stats API breakdown supports conversion_rate (tests only)

* small refactor: extract maybe_add_time_on_page function

* Make it possible to query cr alone

... (without the visitors metric). Already supported in aggregate, this
commit only implements it for the breakdown API.

* Reuse Stats.Util helper functions from b02db88 for aggregate API

We can follow the same logic as with breakdown for manually adding
`visitors` into the metrics list and taking it out of the response
later on.

That way we don't have to make another query, e.g. in a case where
only pageviews and conversion rate is queried. Also keeps things
consistent.

* changelog update

* fix test after resolving merge conflict

* Use explicit string->atom mapping instead of casting

* alias Util module instead of importing it

* use Enum.empty instead of Enum.any

* improve readability

* rename special_metrics to computed_metrics and explain with a comment

* rename visitors_without_event_filters to total_visitors

* keep a single function for removing unwanted metrics

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-02-15 09:18:57 +00:00
ruslandoga
423f72a0ad
safe(r) Oban telemetry (#3743)
* safe telemetry

* use try/catch instead

* add some tests

* cleanup tests

---------

Co-authored-by: hq1 <hq@mtod.org>
2024-02-14 10:12:03 +01:00
Adrian Gruntkowski
f8b4d5066a
Add multiple imports per site (#3724)
* Clean up references to no longer active `google_analytics_imports` Oban queue

* Stub CSV importer

* Add SiteImport schema

* Rename `Plausible.Imported` module file to match module name

* Add `import_id` column to `Imported.*` CH schemas

* Implement Importer behavior and manage imports state using new entities

* Implement importer callbacks and maintain site.imported_data for UA

* Keep imports in sync when forgetting all imports

* Scope imported data queries to completed import IDs

* Mark newly imported data with respective import ID

* Clean up Importer implementation a bit

* Test querying legacy and new imported data

* Send Oban notifications on import worker failure too

* Fix checking for forgettable imports and remove redundant function

* Fix UA integration test

* Change site import source to atom enum and add source label

* Add typespecs and reduce repetition in `Plausible.Imported`

* Improve documentation and typespecs

* Add test for purging particular import

* Switch email notification templates depending on import source

* Document running import synchronously

* Fix UA importer args parsing and ensure it's covered by tests

* Clear `site.stats_start_date` on complete import to force recalculation

* Test Oban notifications (h/t @ruslandoga)

* Purge stats on import failure right away to reduce a chance of leaving debris behind

* Fix typos

Co-authored-by: hq1 <hq@mtod.org>

* Fix another typo

* Refactor fetching earliest import and earliest stats start date

* Use `Date.after?` instead of `Timex.after?`

* Cache import data in site virtual fields and limit queried imports to 5

* Ensure always current `stats_start_date` is used

* Work around broken typespec in Timex

* Make `SiteController.forget_imported` action idempotent

* Discard irrecoverably failed import tasks

* Use macros for site import statuses

There's also a fix ensuring only complete imports are considered
where relevant - couldn't isolate it as it was in a common hunk

* Use `import_id` as worker job uniqueness criterion

* Do not load imported stats data in plugins API context

---------

Co-authored-by: hq1 <hq@mtod.org>
2024-02-14 09:32:36 +01:00
Marko Saric
05adf5d83c
copy changes to the CTA on the top of a new dashboard (#3783)
* Update stats.html.eex

* Update stats.html.eex
2024-02-13 15:43:49 +01:00
Karl-Aksel Puulmann
f5e15941de
Add migration for minmax indexes on session timestamps (#3772) 2024-02-13 10:35:24 +02:00
Karl-Aksel Puulmann
f3509f2a17
Refactor spike detection top sources query (#3770)
* ORDER BY referrer_source for spikes job

This is more consistent with the rest of the queries

* Refactor top_sources -> top_sources_for_spike

* Remove more dead code

* Remove unused arguments

* Remove unused select arguments

* Add a test to top_sources_for_spike
2024-02-13 08:28:32 +02:00
Karl-Aksel Puulmann
d1fe184cb7
Remove ORDER BY min(session.start) (#3771)
This adds a needless performance overhead. Note that all queries already
sort by the _name_ of the breakdown value besides the session start.
2024-02-13 08:28:23 +02:00
hq1
494ffbd622
Shields behind a FF (#3780) 2024-02-12 15:22:06 +01:00
hq1
99fe03701e
IP Block List (#3761)
* Add Ecto.Network dependency

* Migration: Add ip block list table

* If Cachex errors out, mark the cache as not ready

* Add IPRule schema

* Seed IPRules

* Add Shields context module

* Implement IPRuleCache

* Start IPRuleCache

* Drop blocklisted IPs on ingestion

* Cosmetic rename

* Add settings sidebar item

* Consider IPRuleCache readiness on health checks

* Fix typo

* Implement IP blocklist live view

* Update moduledocs

* Extend contextual module tests

* Convert IPRules LiveView into LiveComponent

* Keep live flashes on the tabs view

* Update changelog

* Format

* Credo

* Remove garbage

* Update drop reason typespecs

* Update typespecs for cache keys

* Keep track of who added a rule and when

* Test if adding via LV prefills the updated_by tooltip

* Update ecto_network dependency

* s/updated_by/added_by

* s/drop_blocklist_ip/drop_shield_rule_ip

* Add docs link

* s/Updated/Added
2024-02-12 14:55:20 +01:00
hq1
6a2d7fc0f5
Merge Plugins.API.Router into main one (#3767)
* Merge `Plugins.API.Router` into main one

In order to get grafana metrics reported
See: https://github.com/akoutmos/prom_ex/issues/224

* Format
2024-02-12 10:44:32 +01:00
hq1
f5129f1b0d
Turn Revenue Goals into Custom Events if the plan doesn't support them (#3768)
* Turn Revenue Goals into Custom Events if the plan runs out

* Tag test with full_build
2024-02-12 10:43:54 +01:00
Karl-Aksel Puulmann
499be9f0b3
Add more custom properties to be tracked (#3778)
We just released support for multiple custom property filters, but our demo/live setup
does not have many useful custom properties. This PR adds a few more to allow
better slicing-and-dicing.
2024-02-12 11:11:19 +02:00
Karl-Aksel Puulmann
1cb7982cd9
Filtering by multiple custom properties (#3719)
* WIP: PropFilterRow

* Get multi-behavior working

* Render multiple prop filters in one

* Modal reads from query string correctly

* Backend support for multiple custom property filters

* Add backend tests for multiple custom property filters

* Disable already selected options in property keys

We can't allow choosing the same property multiple times without changing the request
params, which we decided against

* Allow choosing any property under Behaviors > Custom props even if custom prop filter applied

This was a limitation (I believe) introduced by using ARRAY JOINs to query custom properties

* CHANGELOG.md

* Solve credo warning about too deep nesting

* Update assets/js/dashboard/stats/modals/prop-filter-modal.js

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>

* Refactor internal function for clarity

* Add another step -> Add another

* Solve 500 error

* Separate boxes per property filter

* Retain other filters in props table

* removeFilter behavior for props

* matches_member support for custom props

* filter_suggestions for prop keys should account for prop filter

* find over filter

* refactor appliedFilters

* FILTER_TYPES => FILTER_OPERATIONS

* Make add another link not wrap the whole page

* Unique keys

---------

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
2024-02-12 09:03:57 +02:00
hq1
9f2f894421
Fix flashing on reapplying theme (#3760)
* Fix flashing on reapplying theme

This seem to be making a difference on staging

* Add `blocking="rendering"` attribute
2024-02-07 09:32:30 +01:00
hq1
9ad5ecebc2
Improve print dashboard css (#3754)
* Improve print dashboard CSS

* More improvements
2024-02-07 08:57:39 +01:00
Marko Saric
03e1224693
improvements to the email copy (#3756)
* Update yearly_expiration_notification.html.eex

* Update yearly_renewal_notification.html.eex
2024-02-05 18:35:40 +01:00
RobertJoonas
51f0e406a0
improve stats api upgrade required error message (#3747) 2024-02-01 17:00:09 +00:00
Adrian Gruntkowski
3738cd9578
Parse referrer filter value when passed separately (#3742)
* Parse referrer filter value when passed separately

* Remove unnecessary test setup
2024-02-01 10:27:11 +01:00
hq1
e28793a45a
Make hello@plausible.io clickable (#3746)
* Make `hello@plausible.io` clickable

* Format

* ...
2024-02-01 09:43:21 +01:00
hq1
9fb4ea0c3e
Bugfix: available features for expired trials, no subscriptions (#3740)
* Reorganize how subscriptions/trials are evaluated

* Bugfix: expired trial+no subscriptions should not have access to extra features

* Make self-hosted users always on trial

* Seed secondary user with password

* Format

* Fix docs

* Fix small_test run

* Run the test only on full_build

* More tweaks to small builds

* Allow [Goals] for expired trials with no subscription
2024-02-01 08:15:04 +01:00
hq1
931161f693
Generalize Cache/Warmers - extract only specifics to Site.Cache (#3716)
* Simplify caches config

The intervals were proven to never change really,
and previous shape made them not very discoverable either.
We'll look into a more declarative setup.

* Aim for more declarative supervision tree setup

* Generalize `Plausible.Cache`

This makes the `Site.Cache` module implement the specifics,
leaving the common bits reusable for upcoming cache processes.

* Generalize `Plausible.Cache` warmers

* Fix typos
2024-01-30 10:11:29 +01:00
hq1
18ddf1e7a0
Re enable accept traffic until notifications (#3737)
* Revert "Remove 0-sized queue"

This reverts commit 6d229b3f70.

* Revert "Temporarily remove notification cron worker (#3732)"

This reverts commit 447e3929cb.

* Revert "Temporarily disable accept_traffic_until actions (#3730)"

This reverts commit 58afe3376f.

* Send accept_traffic_until notifications at 8 UTC rather than 5

* Add option for dry-run

* Expose dry_run in a dedicated function
2024-01-29 13:37:30 +01:00
RobertJoonas
04b9067591
Add event:goal property to Stats API (#3718)
* add tests for filtering by goal in timeseries and aggregate

* refactor filter parsing

* stop returning custom props in event:goal breakdown

* test breaking down wildcard pageview goals

* extract filter utils

* parse more goal filter options

* add passing tests for new filter types

* do not allow querying session metrics with a goal filter

* remove unused page_match property

* test that non-configured goals are not returned in breakdown

* enforce filtered goals configured

* update changelog

* Allow simple filtering by revenue goals

This does not mean that revenue metrics are supported. If a revenue goal
is filtered by, we treat it like a simple custom event goal in the API.

* use List.wrap
2024-01-29 10:16:47 +00:00
hq1
712f60f37f
Update accept_traffic_until on payment succeeded notification (#3731)
* Update `accept_traffic_until` on payment succeeded notification

* full build only
2024-01-27 10:12:15 +01:00
hq1
58afe3376f
Temporarily disable accept_traffic_until actions (#3730)
* Temporarily disable accept_traffic_until actions

* Remove commented code
2024-01-27 10:11:36 +01:00
hq1
0fa7f3c1e1
Make subsequent query boundaries idempotent (#3723)
* Make subsequent query boundaries idempotent

This should prevent time-sensitive race conditions
on subsequent `Plausible.Stats.*` calls for the same query.
Not sure if there are more places where now() is
dynamic relative to query life cycle, but it's a start.

* Format

* Update lib/plausible/stats/query.ex

Co-authored-by: Wojtek Mach <wojtekmach@users.noreply.github.com>

* Match on `Query.now` in a test

---------

Co-authored-by: Wojtek Mach <wojtekmach@users.noreply.github.com>
2024-01-25 08:59:03 +01:00
Adrian Gruntkowski
822483c37c
Extract Universal Analytics import logic (#3700)
* Move imported tables schemas to separate modules outside Google ns

* Move buffer for imports to Imported ns

* fix schema newlines

* Extract UA import processing and persistence

* Decouple analytics worker implementation from UA

* Rename env variable for import buffer size

* Preserve old import queue until release
2024-01-23 10:24:08 +01:00
Karl-Aksel Puulmann
11c5d3b251
Remove ARRAY JOINs (#3708)
* Use array indexing instead of left joins

This has multiple benefits:
- Allows us to more easily add support for multiple custom property
  filters
- Performance: Increased performance of some queries by ~20%

Work left to do:
- Check tests coverage
- Create macros for repeated fragments
- Quote and refer to right tables in fragments/macros

* Use macros to clean up code

* Use existing atoms to avoid memory leaks
2024-01-23 09:32:49 +02:00
RobertJoonas
041b0ab779
trace aggregate and timeseries queries too (#3712) 2024-01-22 16:03:07 +00:00
hq1
36a8b36382
CombBox bugfix: stale spinner on creatable options selection (#3713) 2024-01-22 15:32:11 +01:00
RobertJoonas
6ca4eaf5e2
Improve query tracing (#3704)
* Enable breakdown property tracing for event:goal

This is currently overwritten by event:name. The queries for event:name
will still be tracked if queried specifically.

* Add filters tracing to Query.trace/1

* make sure breakdown metrics are sorted and joined

* check opts skip tracing directly without passing it
2024-01-22 12:40:08 +00:00
hq1
8965ac40a7
Decorate site settings sidebar with heroicons (#3711)
* Decorate site settings sidebar with heroicons

* Capitalize "Danger Zone"

* Capitalize "Danger Zone"
2024-01-22 12:14:47 +01:00
Karl-Aksel Puulmann
1b95433d6e
Support contains/matches for custom properties (#3687)
* Allow `matches` operator to work in BE for custom props

Note: No FE support yet, needs further testing

* feat: allow choosing `contains` for property filters in the UI

* no autocomplete on prop values if `contains` for consistency

* CHANGELOG.md

* Fix: Handle (none) property in property breakdowns when using matching

When matching we should always exclude (none)
2024-01-22 12:23:59 +02:00
hq1
0a124e69e7
Actually enforce proper WordPress spelling (#3710) 2024-01-22 09:32:47 +01:00
Marko Saric
fb80847d7e
capital P in WordPress (#3703) 2024-01-22 09:19:22 +01:00
Uku Taht
a5e0619053
Escape special regex characters (#3634)
* Escape special regex characters

* Add changelog entry
2024-01-17 11:32:36 +02:00
Cenk Kücük
44fca0c95d
Prioritize X-Plausible-IP header over everything else (#3689) 2024-01-15 17:15:56 +00:00
RobertJoonas
0a29679afa
Stop expecting trial_expiry_date to exist on upgrade (#3688)
* stop expecting trial_expiry_date to exist on upgrade

* add test
2024-01-15 16:24:22 +00:00
RobertJoonas
d3094ffdb7
Prevent upgrades that would leave the customer locked (#3683)
* Remove allowance_required field from grace_period

Since we are now preventing customers from subscribing to a plan that
does not accommodate their pageview usage, there is no need for an extra
check on removing the grace period after a successful upgrade.

This extra check is the reason why the automatic unlocks have recently
failed in several cases.

* refactor outgrown subscription notices

* make a test actually test the described functionality

* Apply greater pageview allowance margin only for trial upgrades

...in order to prevent cancelled or paused subscriptions from subscribing
to plans that would still leave their account locked.

* Mark the entire ChoosePlanTest module full build only

* remove account locking guide

This is irrelevant for self-hosters, and the internal knowledge base is
a better place for this document. Moved it there.

* refactor Keyword get clause

* add a pattern matching assertion in code
2024-01-15 14:59:56 +00:00
RobertJoonas
7536f0285c
Fix weekly interval bug (#3686)
* Fix a bug

This commit fixes a bug where the timeseries weekly interval query for a
month returned 0 visitors for the first week.

This happened because the start of the month defining the first week was
missing the UTC -> site.timezone conversion.

* Update test/plausible_web/controllers/api/stats_controller/main_graph_test.exs

Remove dot from test description

Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>

---------

Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>
2024-01-15 14:39:27 +00:00
Adrian Gruntkowski
eaa7020230
Latency-friendly LV modal implementation and Goals form refactor (#3649)
* Make modal for goal settings trigger without BE roundtrip

* Turn goal form into a live component and extract modal into a wrapper

* Further extract modal component and handle reset action

* Make ComboBox selection callback more flexible

* Add rudimentary loading state to dialog

* Make form unaware of being put inside a modal

* Make modal a live component and completely reset contents on open server-side

* Try to avoid race condition

* Fix race condition

* Remove unnecessary conditional on socket assigns

* Add typespecs and fix formatting

* Make goals form high latency friendly

* Fix tests to account for goal settings form becoming live component

* Fix goal settings form live component declaration

* Add documentation for modal

* Fix small build test

* Fix typo

Co-authored-by: hq1 <hq@mtod.org>

* Revert no longer necessary test changes from 46f65d9

* Fix and clean up modal styling

* Keep focus on dialog when open and show only spinner on backdrop when loading

* Adjust corners and shadows and implement open/close transitions

* Lock body scroll when modal is open

* Make modal top-aligned again to avoid jumping around on variable content height

---------

Co-authored-by: hq1 <hq@mtod.org>
2024-01-15 11:39:30 +01:00
Adrian Gruntkowski
954b70c79b
Run SiteLocker right after site ownership transfer (#3684)
* Run SiteLocker right after site ownership transfer

* Put site unlocking behind full build flag

* Tag regression tests as full build only
2024-01-15 11:39:04 +01:00
hq1
32456d4348
Upgrade stack (deps, elixir 1.16, OTP 26.2.1) (#3678)
* Bump asdf erlang & elixir

* Bump erlang/elixir in the dockerfile

* Remove Oban.Stager config as per https://github.com/sorentwo/oban/blob/main/guides/upgrading/v2.14.md

* Configure Oban for tests as per https://github.com/sorentwo/oban/blob/main/guides/upgrading/v2.14.md

* Mark sampling hint clause with unsafe fragment

Any other/recommended way to do that? cc @ruslandoga

* Address String.slice/2 deprecation

* Update deps

* Address 0.0 matching warning

* Make funnel settings work

although this is probably not the best way to do it.
Needs revisiting, not sure what broke it - could not find
any breaking changes in related dependencies.

* Migrate oban as per https://github.com/sorentwo/oban/blob/main/guides/upgrading/v2.17.md

* Update credo

* Bump CI cache

* Use `Bypass.pass` to avoid exit shutdown message
2024-01-11 08:48:04 +01:00
hq1
c1a1d697a4
Partially revert #3661 - just keep the real errors wrapped, but don't display anything to the user (#3677)
* Revert "Remove unused RocketIcon"

This reverts commit c5e8d0c172.

* Revert "Display either hash or actual error message"

This reverts commit 0c091ab35f.

* Revert "Use ApiErrorNotice in funnels"

This reverts commit 5929de248e.

* Revert "Don't render "No data yet" when there's a NetworkError for example"

This reverts commit 70bee07632.

* Revert "Show the sinking shuttle notice whenever an API error occurs"

This reverts commit 9a62c8af2b.

* Revert "Add Hahash dependency"

This reverts commit b94207ea0a.

* Remove support hash
2024-01-09 17:17:42 +01:00
RobertJoonas
403f559b35
Limit Custom Properties access in Stats API (#3670)
* add new function and doc for allowed_props

* limit props access in Stats API queries

* use dot syntax instead
2024-01-09 14:58:10 +00:00
hq1
24a8aa2821
Reapply sentry context (#3675)
* Reapply "Sentry context in live views (#3672)"

This reverts commit 5449fead160064b8a0081c458cc5dcd34399eb0b.

* Make sure `:selection_made` is handled in `GoalSettings.Form`

That was a bit unepexcted.. normally `handle_info` is injected
by the LiveView use macro and it discards any message gracefully.
After switching to `use PlausibleWeb, :live_view` we're also
using `PlausibleWeb.Live.Flash` that happens to inject its own receive
clause for closing the flash. Which then renders the original,
overridable, `handle_info` catch-all obsolete.

* Update LV SentryContext only on connected sockets

(first mount already has the right context coming from Sentry plug)

* Make sure Live.ChoosePlan passes `current_user_id` session key
2024-01-09 12:28:31 +01:00
hq1
c87b165aef
Revert "Sentry context in live views (#3672)" (#3673)
This reverts commit 9bb2dc00d0.
2024-01-08 17:55:08 +01:00
hq1
9bb2dc00d0
Sentry context in live views (#3672)
* Add common LiveView macro to PlausibleWeb

* Keep peer data, URI and UA in /live websocket metadata

* Use new PlausibleWeb macro in existing LiveViews

* Implement adding some basic Sentry context `on_mount`

* Format

* Use macro in Live.FunnelSettings

* Update FunnelSettings.Form
2024-01-08 15:08:47 +01:00
hq1
dc8210dd84
Use tags rather than context to annotate support hashes in Sentry (#3668) 2024-01-04 15:24:09 +01:00
hq1
9cb44291f7
Rate limit e-mail changes (#3667) 2024-01-04 14:34:57 +01:00
hq1
5c8f39ac4c
Prevent crashes on ambiguity when casting timezones (#3663)
* Fall back to UTC when timezone gap found

* bugfix 1

* bugfix 2

* Apply safe tz conversion to all cases where it's done on a ts other than now

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-01-04 14:24:08 +01:00
hq1
72b4e05bbf
Always enable e-mail verification on full build (#3666)
* Always enable e-mail verification on full build

* s/change/set

* Update lib/plausible/auth/user.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-01-04 10:14:25 +01:00
hq1
4569ac09c1
Don't leak internal server errors, use support hash (#3661)
* Add Hahash dependency

* Don't leak internal server error details to the user

* Show the sinking shuttle notice whenever an API error occurs

* Don't render "No data yet" when there's a NetworkError for example

* Use ApiErrorNotice in funnels

* Display either hash or actual error message

The reason "internal-server-error" doesn't work well as a fallback
hash is that e.g. `NetworkError when attempting to fetch resource`
might be completely at client's fault. In such cases it's better
to display the whole thing still.

* Remove unused RocketIcon
2024-01-04 10:13:37 +01:00
Vinicius Brasil
9e5d63ed96
Display premium features tabs on the dashboard (#3646)
* Display premium features tabs on the dashboard

This commit makes the funnels and props tab on the dashboard visible
even when the site owner's plan does not have access to that feature.
This is to raise awareness of those features, and if the site owner does
not want to see that tab, they can still click "Hide this report" to
hide it.

Previously, when the plan did not support a feature, the feature module
`enabled/1` function returned `false`, regardless of the
`sites.feature_enabled` toggle. This commit creates a new function
called `opted_out/1` to differentiate access from explicitly opting out
a feature.

* Remove unused data-conversions-enabled attribute

* DRY Plausible.Billing.Feature.check_availability/1 function

* Allow opting out features the user doesn't have access to

* add upgrade CTA to FeatureSetupNotice

* fix JS linting errors

* simplify notice.js

* fix behaviour when deleting funnels saved to localstorage

In case some other funnel exists, we will use that as the default
selected one. If not, a feature setup notice will be displayed again.

---------

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
2024-01-03 11:32:21 +00:00
RobertJoonas
155a7a56a7
Fix goal filter inconsistency (#3658)
* remove unused code

* add e.name == "pageview" condition to pageview goals

This fixes the weird behavior where filtering by a pageview goal would
also return custom events and vice versa.

* update changelog
2024-01-03 11:31:52 +00:00
hq1
b6a2acb57a
Update accept_traffic_until notifications (#3665)
* Update accept_traffic_until notifications:

  - extend has_stats to 48h
  - ensure the user is properly greeted

* Clarify the structure passed to email template function
2024-01-03 11:19:25 +01:00
hq1
21bbd3835a
Disallow funny business on timezone entry (#3662)
* Disallow funny business on timezone entry

* Add external API test
2024-01-02 14:38:02 +01:00
Marko Saric
7efa253e3f
Add a note about site settings being available on the locked screen (#3650)
* Add a note about site settings being available on the locked screen

some people are not aware that they can access site settings when the dashboard is locked. i've tried to add that to the note here as it should improve the user experience and minimize the number of people reaching out to support

* Adjust markup a bit

* Convert view to HEEx and apply predefined components

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-01-02 14:27:25 +01:00
RobertJoonas
1342135908
Add custom_props.csv back to CSV export for Growth plans (#3656)
* restore special props access for growth plans

* format

* refactor if condition

Co-authored-by: hq1 <hq@mtod.org>

* format

---------

Co-authored-by: hq1 <hq@mtod.org>
2024-01-02 12:31:08 +00:00
hq1
8be9397199
Skip breakdown of imported pages with pageviews=0 (#3655)
* Skip breakdown of imported pages with pageviews=0

* Fixup

* Prove good impoted records are merged
2024-01-02 13:19:04 +01:00
hq1
73923404b0
Improve approaching accept_traffic_until e-mail (#3660)
* Improve approaching accept_traffic_until e-mail

* fixup

* format
2024-01-02 13:18:51 +01:00
hq1
f755b20569
Lock traffic notifications (#3641)
* Update communication

* Remove an unreachable function (mistyped)

* [migration] Make accept_traffic_until a date

* Fix typo

* Set `accept_traffic_until` when creating a site

* Update sites `accept_traffic_until` on subscription change

* Add a note to yearly cancellation notification

* Rephrase annual e-mail for clarity

* Pass the small build test

* Add email notifications

* Fixup

* Implement `accept_traffic_until` notification worker

* Fixup - no need to test this for small build

* Update moduledoc

* Move moduletag

* s/sent_at/sent_on

* Use WHERE NOT EXISTS instead of LEFT JOIN

* Use upsert when tracking notifications sent

* Store sent marker before actually sending notification

* Prefer to keep `accept_traffic_until` on the user record

This gives us a single source of truth, addresses cases like
ownership transparently, simplifies the code and enables CRM toggles.
The only downside is that there's another join performed in the
Sites.Cache full refresh - in this case, small refreshes are
skipped - but this is fine, since the traffic will be let in
anyway.

* Expose `accepted_traffic_until` in the CRM

* Update lib/plausible/auth/user.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Preload owner in CRM

* Use the offset parameter in trial over e-mail contents

* Format

* Harden cache test

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2023-12-28 08:42:27 +01:00
Adrian Gruntkowski
66d4b3a966
Fix SiteLocker for case of user without trial expiry date but with active subscription (#3652) 2023-12-28 08:42:10 +01:00
RobertJoonas
d2270f3c35
Small bugfix + refactor email reports (#3642)
* use more convenient testing functions

* do not display + sign with 0% change in emails

* Rename module/file/function names

before, `weekly_report` was also used for monthly reports and that was a
bit confusing to read in code.

* Refactor send_email_report.ex

This commit improves readability by refactoring the code into smaller
functions and reducing the number of arguments given to functions.

But more importantly, it stops making duplicate stats queries for every
email recipient by moving the queries out of the for loop.

* Refactor: move querying logic out of the worker module

and merge all stats information under a single `stats` assign.
2023-12-21 12:56:06 +00:00
Cenk Kücük
147597cd98
Drop events when classified as datacenter IP (#3647) 2023-12-21 07:49:00 +00:00
Adrian Gruntkowski
9d97dc1912
Move limit enforcement to accepting site ownership transfer (#3612)
* Move limit enforcement to accepting site ownerhsip transfer

* enforce pageview limit on ownership transfer accept

* Refactor plan limit check logic

* Extract `ensure_can_take_ownership` to `Invitations` context and refactor

* Improve styling of exceeded limits notice in invitation dialog and disable button

* styling improvements to notice

* make transfer_ownership return transfer to self error

* do not allow transferring to user without active subscription WIP

* Add missing typespec and improve existing ones

* Fix formatting

* Explicitly label direct match on function argument for clarity

* Slightly refactor `CreateInvitation.bulk_transfer_ownership_direct`

* Exclude quota enforcement tests from small build test suite

* Remove unused return type from `invite_error()` union type

* Do not block plan upgrade when there's pending ownership transfer

* Don't block and only warn about missing features on transfer

* Remove `x-init` attribute used for debugging

* Add tests for `Quota.monthly_pageview_usage/2`

* Test and improve site admin ownership transfer actions

* Extend tests for `AcceptInvitation.transfer_ownership`

* Test transfer ownership controller level accept action error cases

* Test choosing plan by user without sites but with a pending ownership transfer

* Test invitation x-data in sites LV

* Remove sitelocker trigger in invitation acceptance code and simplify logic

* Add Quota test for `user.allow_next_upgrade_override` being set

* ignore pageview limit only when subscribing to plan

* Use sandbox Paddle instance for staging

* Use sandbox paddle key for staging and dev

---------

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
2023-12-20 14:56:49 +00:00
RobertJoonas
22ecbe7bc7
Refactor: Split up the choose_plan LV code (#3637)
* move format_price to Plausible.Billing

* move PlausibleWeb.Components.Billing file to subfolder

* extract new Notice module

* rename test file and module name

* move growth_grandfathered notice to notice.ex

* extract a PlanBenefits module

* extract PlanBox component

* extract PageviewSlider component

* fix plan benefits text color
2023-12-15 13:59:16 -03:00
RobertJoonas
77ca5abbc8
fix theme applying based on the 'theme' query param (#3639) 2023-12-15 13:59:00 -03:00
Vinicius Brasil
7f51928338
Remove business tier feature flag (#3632)
* Remove business tier feature flag

This commit removes all code branches related to the business tier
feature flag, as we're not flipping this flag off anymore. It also
removes unused routes, e.g. /billing/change-plan and /billing/upgrade

* remove unused billing templates

* refactor with clause to case instead

* assert on the url in email tests

---------

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
2023-12-14 11:25:46 -03:00
ruslandoga
6639d6af63
Flush Clickhouse buffers faster by encoding early (#3623)
* encode early

* rename: CLICKHOUSE_MAX_BUFFER_SIZE -> CLICKHOUSE_MAX_BUFFER_SIZE_BYTES

* deprecate CLICKHOUSE_MAX_BUFFER_SIZE

* cleanup

---------

Co-authored-by: hq1 <hq@mtod.org>
2023-12-14 14:21:27 +01:00
Uku Taht
17ba44e8db
Fix darkmode flash (#3625)
* Inline theme script to layout

* Extract theme_script as Heex component

* Remove unused function

* Reference assigns directly

* Clean up current user

* Clean up theme javascript

* Use new theme script in focus.html layout

* Use `assigns` instead of `@conn.assigns` in `focus.html.heex` template too

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2023-12-14 10:57:02 +02:00
RobertJoonas
ab54a66a43
Fix class in styled_link and remove existing uses of class (#3631)
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2023-12-13 15:29:13 +00:00
hq1
5a01624f03
Fix link styling regression introduced via #3572 (#3629)
* Fix link styling regression introduced via #3572

* Justify external links in user dropdown

* Format

* Use assigns.new_tab to justify justification 🤡
2023-12-13 15:58:21 +01:00
hq1
8d85c38047
Add :payment_required policy to GateKeeper (#3628)
* Migration: Add `accept_traffic_until` to "sites"

* Drop traffic from sites where `accept_traffic_until` is passed
2023-12-13 14:52:32 +01:00
RobertJoonas
686ffa6ef3
Clean up legacy trials code + displaying limits (#3620)
* remove unused functionality

We can now safely delete all the logic around users who are on trial and
signed up *before* the business tiers release, since that's no longer
possible

* add monthly_pageview_limit fn clause that takes a user

* make Quota.site_limit return enterprise site limit

...not `:unlimited`, as we still need to display it
in the account settings.

* make `team_member_limit/1` return :unlimited on small_build

* improve team_member_usage/1 function doc

* stop displaying unlimited symbol in usage section

These unlimited limits include:

* `monthly_pageview_limit` for trials
* `team_member_limit` for old enterprise plans
* `site limit` for old accounts (before 2021-05-05)

* format

* small refactor case clause + move mod vars

* review suggestions
2023-12-13 10:47:50 +00:00
Uku Taht
677b4f2e79
Use dropdown component in /sites page (#3572)
* Use dropdown component in sites page

* Add x- attributes to globals

* Update tests

* Prevent default on click

* Fix compile warning

* Add conditional rendering back in

* Remove $id magic
2023-12-13 10:59:20 +02:00
RobertJoonas
e85ee417ab
Prevent invited users without sites from subscribing (#3600)
* prevent invited users without sites from subscribing

* always setup a site in choose_plan_test context

* change trial_expiry_date=nil condition to no sites owned
2023-12-12 15:48:33 +00:00
hq1
8ba851d27f
Protect against custom props payloads breaking event insertions (#3617)
* Protect against custom props payloads breaking event insertions

* Use two traversals instead of three

* Reorganize invalid props filtering
2023-12-12 15:43:11 +01:00
hq1
b96319321c
Add Browser/Version breakdown to CSV (#3599)
* Add csv for browser versions

* add changelog

* Fixup - ensure the dashboard browser version breakdown still works

* Update CHANGELOG

---------

Co-authored-by: Ekaterina Krivich <ekaterinak@heathmont.net>
2023-12-12 14:39:08 +01:00
Vinicius Brasil
45a763ab2b
Add data retention to the choose plan page (#3605)
* Add data retention field to plans

* Display data retention as benefit when choosing plan

* Split fields in two module attributes

* Remove extra whitespace

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2023-12-12 08:19:54 -03:00
hq1
7b5c20db17
Revert "experiment in improving throughput of write buffers (#3511)" (#3614)
This reverts commit 16bfb11701.
2023-12-11 16:44:32 +01:00
Adam Rutkowski
69579272d6 Revert "dump buffer on invalid insert (#3613)"
This reverts commit 7a4cafac16.
2023-12-11 16:31:03 +01:00
ruslandoga
7a4cafac16
dump buffer on invalid insert (#3613)
* dump buffer on invalid insert

* dump to PERSISTENT_CACHE_DIR

* fix warning

* ensure dir is not nil
2023-12-11 15:56:18 +01:00
ruslandoga
16bfb11701
experiment in improving throughput of write buffers (#3511)
* wip

* fewer length/1 calls

* encode early

* cleanup

* add micro benchmark

* extract types at compilation

* remove specs to make credo happier

* cleanup is_bounce

* cleanup :D

* add eprof to buffer insert+flush benchmark

* add ci/bench.yml

* rm state.schema from write buffer

* rm noisy logs

* rm benches

* add :source to insert

* make CLICKHOUSE_MAX_BUFFER_SIZE mean bytes

---------

Co-authored-by: hq1 <hq@mtod.org>
2023-12-11 13:28:49 +01:00
Vinicius Brasil
e4230db2d9
Improve subscription status checking code (#3598)
* Improve subscription status checking code

This commit improves the subscription status checking in code, and
creates convinience functions to work with it, including nil-checking
and in?/2 function.

* Change in?/2 to macro
2023-12-07 09:05:07 -03:00
Adrian Gruntkowski
12f786810c
Remove two_factor feature flag (#3597)
* Remove `two_factor` feature flag

* Avoid compilation warnings in small build
2023-12-07 11:22:40 +01:00
ruslandoga
0eedf9aa98
rm Hammer (#3571) 2023-12-06 15:07:37 +01:00
Marko Saric
0b00762591
Changes to the emails as discussed (#3540)
* Update over_limit.html.eex

* Update dashboard_locked.html.eex

* Update dashboard_locked.html.eex

* Update over_limit.html.eex

* Update dashboard_locked.html.eex

* fix tests

* stop querying owned_site_ids three times

... when querying for billing cycles. Adds an optional `owned_site_ids`
argument to the `usage_cycle` function.

* add penultimate billing cycle info to emails

This commit also refactors some code and adds unit tests to email templates

* use delimit_integer instead of large_number_format

... to display usage with exact numbers such as 1,099,999 instead of 1M

* add penultimate cycle date ranges and linebreaks

---------

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
2023-12-06 12:02:22 +00:00
hq1
615b6aef7d
Plugins API exentsions (custom props, bulk goal delete, goal creation => ListResponse always) (#3593)
* End polymorphic response in goals create

Being part of the v3 spec, this isn't well
(or at all) supported by OpenAPI generators.

Always respond with `Goal.ListResponse`

* Implement `PUT /custom_props`

* Implement bulk `DELETE /goals`

* Expose API for (bulk-)disabling custom props

* Add controller typespecs

* Delegate list wrapping to `Plausible.API.*`
2023-12-06 12:33:33 +01:00
Adrian Gruntkowski
da0fa6c355
Implement UI for 2FA setup and verification (#3541)
* Add 2FA actions to `AuthController`

* Hook up new `AuthController` actions to router

* Add `qr_code` to project dependencies

* Implement generic `qr_code` component rendering SVG QR code from text

* Implement enabled and disabled 2FA setting state in user settings view

* Implement view for initiating 2FA setup

* Implement view for verifying 2FA setup

* Implement view for rendering generated 2FA recovery codes

* Implement view for verifying 2FA code

* Implement view for verifying 2FA recovery code

* Improve `input_with_clipboard` component

* Improve view for initiating 2FA setup

* Improve verify 2FA setup view

* Implement `verify_2fa_input` component

* Improve view for verifying 2FA setup

* Improve view rendering generated 2FA recovery codes

* Use `verify_2fa_input` component in verify 2FA view

* Do not render PA contact on self-hosted instances

* Improve flash message phrasing on generated recovery codes

* Add byline with a warning to disable 2FA modal

* Extract modal to component and move 2FA components to dedicated module

* First pass on loading state for "generate new codes"

* Adjust modal button logic

* Fix button in verify_2fa_input component

* Use button component in activate view

* Implement wait states for recovery code related actions properly

* Apply rate limiting to 2FA verification

* Log failed 2FA code input attempts

* Add ability to trust device and skip 2FA for 30 days

* Improve styling in dark mode

* Fix waiting state under Chrome and Safari

* Delete trust cookie when disabling 2FA

* Put 2FA behind a feature flag

* Extract 2FA cookie deletion

* ff fixup

* Improve session management during 2FA login

* Extract part of 2FA controller logic to a separate module and clean up a bit

* Clear 2FA user session when rate limit hit

* Add id to form in verify 2FA setup view

* Add controller tests for 2FA actions and login action

* Update CHANGELOG.md

* Use `full_build?()` instead of `@is_selfhost` removed after rebase

* Update `Auth.TOTP` moduledoc

* Add TOTP token management and make `TOTP.enable` more test-friendly

* Use TOTP token for device trust feature

* Use zero-deps `eqrcode` instead of deps-heavy `qr_code`

* Improve flash messages copy

Co-authored-by: hq1 <hq@mtod.org>

* Make one more copy improvement

Co-authored-by: hq1 <hq@mtod.org>

* Fix copy in remaining spots

* Change redirect after login to accept URLs from #3560 (h/t @aerosol)

* Add tests checking handling login_dest on login and 2FA verification

* Fix regression in email activation form submit button behavior

* Rename `PlausibleWeb.TwoFactor` -> `PlausibleWeb.TwoFactor.Session`

* Move `qr_code` component under `Components.TwoFactor`

* Set domain and secure options for new cookies

---------

Co-authored-by: hq1 <hq@mtod.org>
2023-12-06 12:01:19 +01:00
RobertJoonas
4566e6b530
New admin route for displaying usage (#3577)
* add a new crm usage route for admins

* add a test for admin route authorization

* add full_build_only tag
2023-12-06 10:07:07 +00:00
Adrian Gruntkowski
d42c6927d5
Use autocomplete=new-password for shared link password field (#3589) 2023-12-04 15:27:56 +01:00
Uku Taht
032823e112
Add last_bill_date to new subscriptions (#3588)
* Add last_bill_date to new subscriptions

* Remove leftover test code

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>

---------

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
2023-12-04 14:31:33 +02:00
Uku Taht
44d71c8c0e
Fix domains that start with UTF character (#3560)
* Avoid redirect in site settings

* Fix unicode in SiteController existing tests

* Fix various tests

* Add CHANGELOG

* Make sure test site is example.com

* Use Route helpers in site_controller

* Fix UTF redirect in change domain submit action

* Fix UTF site domain in reset stats action
2023-12-04 14:22:17 +02:00
hq1
74675f0cbc
Move dogfooding to full build (#3576)
* Move dogfooding to `extra/`

* Remove unused view functions
2023-12-04 10:30:20 +01:00
Vinicius Brasil
5e6033e261
Move input range bubble to JavaScript (#3569)
* Move input range bubble to JavaScript

This commit switches the input range bubble on the choose plan page from
LiveView to JavaScript. The reason for this change is the input range
is a regular HTML input rendered by the browser, not LV, therefore
bubble was not in sync when sliding the input.

* Apply suggestions from code review
2023-11-30 15:02:36 -03:00
hq1
7bf1e2a6ed
Reapply "Define a better monthly pageview usage (#3564)" (#3574)
This reverts commit c739b8878d.
2023-11-30 13:30:04 +01:00
hq1
5278c23965
Revert and extract migration (#3573)
* Revert "Define a better monthly pageview usage (#3564)"

This reverts commit 57188a402a.

* Extract migration from 57188a402a/priv/repo/migrations/20231129103158_add_allow_next_upgrade_override_to_users.exs
2023-11-30 13:19:25 +01:00
RobertJoonas
57188a402a
Define a better monthly pageview usage (#3564)
* refactor asking for the monthly pageview usage

* add tests for usage and limits section in account settings

* display pageview usage per billing cycle for active subscribers

* disable cycle tabs if no usage

* make current billing cycle whole

...instead of capping it at today's date

* run queries for different cycles concurrently

* fix linebreak bug

* add calculate usage action into CRM

* change some names of assigns

* block subscribing to a plan by pageview usage

Depending on whether the customer has already subscribed or not, checking
their pageview usage is different:

* If they're not subscribed yet, we allow them to subscribe to a plan If
  it their last 30 days usage does not exceed the plan pageview limit by
  more than 15% (30% for when subscribing to a 10k plan)

* For existing subscribers, we'll use the exact same mechanism that we're
  using for locking sites - the last two billing cycles usage. If both
  cycles exceed the plan limit by more than 10% - we don't allow them to
  subscribe to the plan

* apply credo suggestion

* prevent highlight bar overflow

* move disabled classes to button element

* optimize for darkmode

* unify link and text styling on the same horizontal line

'Upgrade' & 'Update billing details' links + billing interval text were
positioned on the same line. The font size was similar, but not the same

* improve exceeded_limits function readability

* Refactor some tests and remove code duplication

* override allow upgrade when limits exceeded

In cases where limits are exceeded, we can set the boolean flag
`allow_next_upgrade_override` to `true` in the CRM. This will allow
the user to upgrade to any plan they want. After they've upgraded or
changed their plan - the flag will automatically reset to `false`.

* only apply upgrade override for exceeded pageview limit

* fix tests on the CI

* make current_cycle usage always displayed by default

* make pageview allowance margin more clear

* add comment
2023-11-30 11:50:44 +00:00
Vinicius Brasil
8dfaad56f0
Match choose plan styling with landing page (#3568)
* Match choose page styling with landing page

Related: https://github.com/plausible/website/pull/507

* Fix tests

* Apply suggestions from code review
2023-11-29 16:14:36 -03:00
hq1
a4b9c3b8ba
Remove custom domains support + update build options (#3559)
* Disable super-admin checks on small build

* Mute a test writing to stdout

* Move sampling outside of small build

* Convert waiting_first_pageview to heex and stop relying on env vars

* Set site limit unlimited on small build

* Stop relying on app env to get trial expiry

* Remove custom domains - including migration

* Remove is_selfhosted from layout view

* Quota fixup

* Stop relying on app env for self hosted registration

* Stop relying on app env for pass reset success

* Apply on_trial? check only on full build

* Update templates relying on app env

* Adjusts auth controller tests for small build

* Trial fixup

* Fixup

* Stop relying on app env

* Rest of the fsckn owl

* Update typespecs

* Fix dialyzer warning

* Remove unused module

* Credo + format

* GeoIP is not, for full build

* Use `small_build?()` where applicable

* Implement bypassing FirstLaunchPlug without insertions

* Get Marko's patch de58a18a85

* Test is-dbip=false presence

* Fix typespec

* Remove future hardcodes

* Handle `nil` from `Plausible.Geo.database_type()`

* Remove XXX marker

* Use one typespec for two clauses

* Introduce `MIX_ENV=small_dev`

* Revert "Use one typespec for two clauses"

This reverts commit 8d8cd21764.
2023-11-29 11:04:54 +01:00
Adrian Gruntkowski
86b9d1680d
Simplify TOTP core logic and add email notifications (#3563)
* Modify API of `Auth.TOTP` to work with simplified flow

* Send email when 2FA is enabled and disabled

* Add tests for `initiated?/1`

* Add tests for email submission and improve recipient address composition

* Fix email tags

* Rename email templates to HEEx

* Fix formatting in email templates
2023-11-28 15:29:55 +01:00
RobertJoonas
ff2c3346d2
Bugfix: Allow breakdown by internally used prop keys for Growth plans (#3562)
* fix bug - allow internally used prop key breakdown for Growth plans

* use case instead of with
2023-11-28 09:30:35 +00:00
Marko Saric
de58a18a85
Keeping footers consistent 2023-11-27 21:52:57 +01:00
hq1
0f577a998c
Update credo config + extras (#3547)
* Disable CyclomaticComplexity checks

* Move AuhtorizeSitesAPI plug under extra
2023-11-22 20:48:32 +01:00
hq1
88e1d9dc28
Small build updates (#3546)
* Sites API

* Extract Revenue react api helpers

* !fixup

* Extract JS Money module to /extra

* Extract Revenue full build extras (tests pass for full)

* Update MIX_ENV=small mix test

* Remove dead code

* Add moduledocs

* Add credo config

* Trick dialyzer

* DRY revenue metrics

* Use more concise version of on_full_build macro

* Disable credo check
2023-11-22 15:34:47 +01:00
Vinicius Brasil
af0b97e68a
Move Mix.Task.DumpPlans to Plausible.Release (#3544)
This commit moves the dump_plans Mix task to the Plausible.Release
module so it can run in production.

Usage: `./bin/plausible eval Plausible.Release.dump_plans`
2023-11-21 12:22:17 -03:00
Vinicius Brasil
b35096bbc8
Dump plan information to PostgreSQL (#3543)
* Use Ecto.Schema for casting plans from JSON files

* Dump plans to internal database table
2023-11-21 11:25:54 -03:00
Adrian Gruntkowski
65cc8980e0
Implement core logic for TOTP support (#3525)
* Add `nimble_totp`, `cloak` and `cloak_ecto` to project dependencies

* Setup Cloak-based secrets vault and create a dedicated Ecto type

* Add `totp_enabled|secret|last_used_at` fields to `User` schema

* Implement schema and stateless logic for TOTP recovery codes

* Implement core logic of TOTP auth

* Fix typos and improve style of doc comments

Co-authored-by: hq1 <hq@mtod.org>

* Fix moduledoc alignment

* Use more compact conditional expression

Co-authored-by: hq1 <hq@mtod.org>

* Disambiguate `I` as `7` when generating recovery codes (h/t @hq1)

* Fix a typo in runtime config error message

---------

Co-authored-by: hq1 <hq@mtod.org>
2023-11-20 14:04:48 +01:00
hq1
b9ec38038c
Add small build option (#3536)
* Update applications

* Clone community config

* Move modules to experimental dir

* Update runtime config

* Apply first set of compile-time conditionals

* Move funnel schemas to experimental

* Make funnel schema-less build compile

* Use experimental/lib for elixir code

* Move JS funnels to experimental

* Clean up conditional rendering

* Tidy up the pipeline

* Make two builds pass tests without warnings

* Reuse existing dotenvs

* Do a bunch of renames

* Clean up naming

* Run secondary CI

* Update router

* Remove RewriteFunnelDupes migration

Tests were disabled already and it was a one-off shot

* Fixup quota mixins

* Add moduledoc

* Change MIX_ENV for seconary test run

* Skip crm on small

* !fixup

* Exclude flags pipeline

* Update lib/plausible_web/controllers/stats_controller.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2023-11-20 12:52:20 +01:00
Adrian Gruntkowski
9b0a6e6e7d
Improve LiveView flash messages (#3523)
* Cancel existing flash timer before setting a new one

* Abstract flash logic across live views

* Render flash messages one after another
2023-11-20 11:31:56 +01:00
RobertJoonas
02a1271ee4
Bugfix site transfers (#3531)
* Bugfix: allow ownership transfers when premium features enabled but not used

Fields like `props_enabled` and `funnels_enabled` are true by default,
and these fields do not indicate whether the user/site is actually using
these features or not.

* allow site transfers if they will be at limit after transfer

* small refactor
2023-11-17 17:43:41 +00:00
Vinicius Brasil
0175158e81
List only premium features on the CRM (#3530) 2023-11-17 14:42:31 -03:00
Cenk Kücük
f4d7ba70e8
drop IPs classified as datacenter (#3522)
* drop IPs classified as datacenter

* process event and increment counter instead of dropping
2023-11-17 15:34:20 +01:00
Márton Salomváry
555eb25d20
Allow custom event timeseries in stats API (#3505)
* Allow custom event timeseries in stats API

* Fix linting error

---------

Co-authored-by: Uku Taht <Uku.taht@gmail.com>
2023-11-17 07:37:56 -03:00
Vinicius Brasil
cfaa5be8f4
Configurable limits for enterprise plans (#3527) 2023-11-16 21:57:14 -03:00
RobertJoonas
d66322e12d
Limit grandfathering to *active* subscribers and trials (#3524)
* refactor asserting plan generation in plans_test.exs

* stop grandfathering old expired trials

For users who registered before the business tiers release, we want to
offer a chance to subscribe to a grandfathered plan. However, if they
let their trial expire and don't subscribe in the next 10 days, they'll
lose that opportunity.

* stop grandfathering expired subscriptions

* remove default title and icon from Generic.notice

* fix bug with dismissable notice

classList is null when dismissable_id is not given

* alias Plausible.Auth.User

* Refactor Generic.notice component

Make it easy to apply different colors

* move subscription_cancelled_notice across the app

And remove from user settings > subscription box. Also, include a note
about losing grandfathered status when letting the subscription expire.

* allow full width in Generic.notice

* use Generic.notice for subscription_past_due_notice

* use Generic.notice for subscription_paused_notice

* prevent two notices clashing into each other with gap-y-2

* define attrs for phx components

* optimize for light mode

* make subscription cancelled notice dismissable

but if it's dismiss, show it in the place where it was before in the
account settings > subscription box

* make function private

* replace function doc with regular comment to avoid compile warning

* use array for classnames

Co-authored-by: Vinicius Brasil <vini@hey.com>

* fix typos in function doc

---------

Co-authored-by: Vinicius Brasil <vini@hey.com>
2023-11-16 15:40:50 +00:00
ruslandoga
13055aafc0
use windows in 'time on page' queries (#3446)
* use windows in 'time on page' query

* add imported time on page back

* join imported_pages instead of extra query

* don't forget aggregated time on page!

* eh?

* no need for separate window clause

* use dynamic_filter_condition in aggregate_time_on_page

* it's avg, not total

* fix dynamic_filter_condition call

* tests pass

* fewer changes

* fewer changes + feature flag

* base quickfix

* fewer changes

* fewer changes?

* fewer changes!

* replace coalesce with if(empty(),etc)

* make window_aggregate_time_on_page return same result as neighbor_

* breakdown as well

* add failing test

* more info

* more info

* format example

* add failing aggregate.ex test

* fix breakdown.ex windowing time_on_page calculation

* fix aggregate.ex windowing time_on_page calculation
2023-11-16 11:03:36 +02:00
Vinicius Brasil
e541f1a55f
Hide premium feature notice for trials (#3509) 2023-11-14 09:40:04 -03:00
RobertJoonas
af979d02c7
Bugfix: Allow cancelled subscriptions to subscribe (#3517)
* extract set_slider function in choose_plan_test

* allow cancelled subscriptions to subscribe even before expired
2023-11-14 08:40:20 +01:00
RobertJoonas
fdf1462c04
Notice across the app about Funnels and Revenue goals private preview end (#3510)
* change upgrade CTA notice message for enterprise and business plans

* add dismissable option to Generic.notice

* more general notice about losing premium features in X days

* save notice dismissed per user

... more than one Plausible account can use the same device, so we should
scope the fact that the notice has been dismissed by user id.

* fix bug applying classes to Generic.notice

* apply shadow to the new notice on light mode

* use Heroicons.x_mark instead of raw SVG

* use Enum.filter instead of list comprehension
2023-11-13 16:19:58 +00:00
Adrian Gruntkowski
3ba57a04fc
Make OpenTelemetry sampler ratio configurable via env (#3514) 2023-11-13 14:57:51 +01:00
hq1
c64ab6b6ee
Fix bug showing invitation popups (#3513)
Apparently using the `<p>` element breaks x-show/styles cascading.
Swapping it with `<div>` renders `x-show` correctly including
children elements.
2023-11-13 11:31:46 +01:00
Adrian Gruntkowski
f464ceae88
Implement pinned sites (#3469)
* Revert "Remove site pins for now"

This reverts commit 5eccf4eaf6.

* Implement basic site pin schema level logic within user specific preferences

* Add vertical ellipsis menu markup

* Implement basic changesets for user preferences

* Implement pin toggling

* Try to fix pin sorting

* Implement pin toggling in LV

* Adjust moduledocs for new schema(s)

* Remove unnecessary `distinct` from query

* Use `button` for pin/unpin action

* Generalize preference setting

* Rename schema and fields for clarity

* Rename `list_type` -> `entry_type`

* Safeguard setting options

* Test `set_option/4` and `toggle_pin/2`

* Add test for listing pinned sites via `Sites.list`

* Disallow pinning sites outside page explicitly

* Test pinning in LV

* Test conditional rendering of site settings in /sites

* Remove unnecessary TODO comment

* Safeguard `Sites.set_option/4` against invalid user/site combo

* Handle pinned sites in dashboard site picker

* Clear flashes upon (un)pinning sites

* Update CHANGELOG

* Prevent blinking of hamburger menu items on first paint

* Highlight hamburger handle on hover in /sites

* Start showing hotkeys in site picker again

* Sort pinned sites in the order they were pinned

* Update sites list order immediately after pin/unpin toggle

* Refactor and split `Sites.list/3`, extracting `Sites.list_with_invitations/3`

* Cap number of pinned sites at 9 per user

* First pass on visual indication of site cards (dis)appearing

* Apply ellipsis gradient+shadow on card hover

* Fix responsive padding of site cards

* Sort by invitations first, pinned sites second and then the rest

* Revert "Apply ellipsis gradient+shadow on card hover"

This reverts commit 0608796612639030ccbb12df639709f78edc1434.

* Apply more subtle hover effect on the ellipsis menu

* Make error and success flash LV boxes use separate component containers

* Promote `pinned_at` in table migration to a column

* Switch logic to using `pinned_at` as a standard schema field

* Refactor `Sites.list*` getting rid of subquery (h/t @ukutaht)

* Remove migration which is already merged upstream

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
2023-11-13 09:08:26 +01:00
RobertJoonas
26d9e16d7d
Fixing Business Tier related bugs (#3504)
* fix underlined blank space in upgrade link

* Add :if clause to site limit notice

* Change test description

* Move growth? and trial? conditionals to CTA function

* Improve wording in billing notices

---------

Co-authored-by: Vinicius Brasil <vini@hey.com>
2023-11-10 11:03:02 -03:00
hq1
af87a63cab
Fix disabled invitation form on business tier (#3508)
ref https://github.com/plausible/analytics/pull/3493
2023-11-10 10:07:30 +02:00
Adrian Gruntkowski
2d2ea1a2a3
Implement trace ID ratio-based sampling for OpenTelemetry (#3499)
* Implement trace ID ratio-based sampling for OpenTelemetry

* Make `decide` more resilient
2023-11-09 12:24:50 +01:00
Adrian Gruntkowski
8698b8770b
Create site_user_preferences table and apply LV fix (#3506)
* Create `site_user_preferences` table

* Do not trigger querying on filter text change when there's no change
2023-11-09 12:24:41 +01:00
Vini Brasil
b5000cc247
Fix private preview notice typo (#3503)
* Fix private preview notice typo

* fix link to choose-plan instead of upgrade

* do not display premium feature notice if legacy feature access

* register trial message

* change enterprise contact us copy

* fix test

---------

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
2023-11-08 16:57:20 +00:00
Vini Brasil
2578391be2
Display private preview for non business accounts (#3502)
* Display private preview for non business accounts

* Update lib/plausible_web/components/billing.ex

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>

---------

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
2023-11-08 11:59:03 -03:00
RobertJoonas
7036332db2
Business tier bug smash (#3493)
* price formatting

* fix space underlined

* add a redirect from :upgrade to :choose_plan

* Add premium feature notice to revenue goals

* add the number of API request to plan benefits

* be more explicit about Stats API requests

* start linking to the new upgrade page if FF enabled

* add redirect to :upgrade_to_enterprise_plan from :choose_plan

This commit intends to keep the existing behaviour of redirecting users
directly to the enterprise upgrade page when they click on an upgrade
link in an email and have an enterprise plan configured.

Hence, we can also simplify the /settings template and only link to
'/billing/choose-plan', even for enterprise plans.

* deprecate /billing/change-plan route based on FF

* Move new site template to HEEX

* Move new team member invitation template to HEEX

* Standardize exceeded limits error message

* Fix failing tests

* Limit Stats API access to unlimited trials

---------

Co-authored-by: Vinicius Brasil <vini@hey.com>
2023-11-08 10:24:30 -03:00
Vini Brasil
c9bf5827e9
Prepare business tier for release (#3464)
* Change limits for trials

* Keep legacy trial limits for users that registered before the business tier

* Change private preview notice for release

* Run formatter

* Add countdown to private preview notice
2023-11-08 09:51:34 -03:00
hq1
38b1834b3f
Tidy up openapi schema (#3498)
* Tidy up OpenAPI schema

* Remove `items` from object schema
2023-11-08 11:51:37 +01:00
Uku Taht
058d8cc6c9
Extract button component (#3474)
* Add button component

* Use new button in settings screen

* Use button component in registration screens

* Use new button component for Billing.upgrade_link

* Separate .button and .button_link

* Add attr definiton for disabled

* Fix funnels test
2023-11-08 11:40:07 +02:00
RobertJoonas
f977351ae2
Fix upgrade page for legacy trials + bug fixes (#3486)
* allow using Stats API and Props for free_10k subscriptions

* return v3 plans for legacy trials

* do not display grandfathering notice for legacy trials

* set a more accurate BT release date

* fix bug on dev env

Allow the `find/1` function to find sandbox plans

* add error handling and tests for change_plan_preview

* fix feature warning bug

* fix credo warnings

* fix tests

* set BT release date further into the future

* rename function and some vars

* bugfix with limit exceeding

* fix test
2023-11-06 14:01:55 +00:00
hq1
a8fa05706b
Add loading state to site cards (#3483)
* Show loading state for mini plots

* Dark mode

* Don't render no change in green

* Fix loading placeholders for mobile/desktop view

* Fixup test

* Make loading state background color for light theme one step brighter

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2023-11-06 10:17:15 +01:00