Commit Graph

3144 Commits

Author SHA1 Message Date
Adrian Gruntkowski
c0a8aa025c
Clean up guest invitations selectively after accepting (#4778) 2024-11-06 08:56:48 +00:00
Adrian Gruntkowski
1e1a4ab35b
Get rid of memberships prosthetic for Teams.Sites.list_with_invitations (#4764)
* Get rid of memberships prosthetic for `Teams.Sites.list_with_invitations`

* Fix query in newly introduced teams backfill script
2024-11-05 09:44:27 +00:00
Karl-Aksel Puulmann
af5eb720db
Release breakdown_comparisons_ui flag (#4774) 2024-11-05 09:13:22 +00:00
Adrian Gruntkowski
dac9e3a0f0
Add removing guest memberships with mismatched team to backfill script (#4776) 2024-11-05 09:05:52 +00:00
Karl-Aksel Puulmann
1048967ac5
Channels: Write click_id_param column (#4704)
* Add migration for click_id_source

* click_id_param

* Write click_id_source column

* Add columns to schema

* click_id_param
2024-11-05 08:04:04 +00:00
ruslandoga
0ec8ac6d7c
remove @plausible.io contacts from CE (#4766) 2024-11-05 07:37:21 +00:00
ruslandoga
bef8b33087
update ecto_ch to fewer parens version (#4775) 2024-11-05 07:36:45 +00:00
Karl-Aksel Puulmann
4e10efe723
Channels: click_id_param column (#4703)
* Add migration for click_id_source

* click_id_param
2024-11-05 07:03:00 +00:00
Uku Taht
9d06d45e45
New remap sources migration that's case insensitive (#4771) 2024-11-04 09:47:01 +00:00
Artur Pata
bfa01bff2a
Limit maximum date to today in calendars (#4746)
* Limit maximum date to today in calendars

* Add test

* Make calendar max dates sensitive to site timezone
2024-11-04 09:23:36 +00:00
ruslandoga
40f28ed151
rm Timex.diff/3 (#4695) 2024-11-04 09:18:04 +00:00
Artur Pata
13ad279820
Add second line for Filters (if saved_segments enabled) (#4729) 2024-11-04 08:22:48 +00:00
Adrian Gruntkowski
5d9e94770d
Run full test suite against read_team_schemas FF on CI (#4763)
* Run all tests with `read_team_schemas` flag up and down on CI

Replacing the existing setup for `experimental_reduced_joins` which
wasn't actively used and will be removed some time in the future.
This way we don't make CI run longer than necessary.

* Add one more test case for listing invitations on /sites LV

* Intentionally reintroduce regression to test CI

* Revert "Intentionally reintroduce regression to test CI"

This reverts commit cdb83c1494.

* Run tests for `experimental_reduced_joins` together with `read_team_schemas`
2024-10-31 13:17:31 +00:00
Marko Saric
71b41cb430
copy change to be consistent with the main site (#4699)
* copy change to be consistent with the main site

* Update README.md
2024-10-31 11:37:57 +00:00
Karl-Aksel Puulmann
b16bd91600
Hard-code sample rate based on fractional_hardcoded_sample_rate flag (#4762)
* Hard-code sample rate based on fractional_hardcoded_sample_rate flag

We found cases where using a numeric sample rate would cause issues when
joining two tables due to different _sample_factor. Ref: https://3.basecamp.com/5308029/buckets/26383192/card_tables/cards/7973456592#__recording_7978780711

The proper fix is to use fractional sample rates everywhere, but this is
a whole project due to not wanting to sample small sites. For now, hard-code
sample rate for specific sites having the issue while we work on a
larger fix.

* is_number
2024-10-31 10:25:22 +00:00
Adrian Gruntkowski
7ff1a16ae3
Fix sites list for needs to upgrade condition check (#4758) 2024-10-31 09:20:29 +00:00
Adrian Gruntkowski
fc92ed5018
Switch Sites.list[_with_invitations] to use teams schemas (#4747)
* Simplify "needs_to_upgrade" check in Sites LV

* Use new schema for listing sites (WIP)

* wip: trying to filter out redundant site entries

* Move site listing logic around and continue fixing tests

* Fix Plausible.SiteTest

Expectations in some tests were adjusted. For instance,
we do not show invitations for sites where user is already
a member or owner.

* 1st stab at high-level factories covering old/new models

* Switch live /sites tests to transitional factories

* Improve prosthetics and make the remaining /sites tests pass

* Put new sites list logic behind a feature flag

* Fix listing pinned sites

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
2024-10-31 08:53:02 +00:00
Uku Taht
a1b1b84963
Remap sources migration (#4751) 2024-10-31 08:07:50 +00:00
Uku Taht
c3a06caa97
Channel and source data updates (#4599)
* Channel and source data updates

* Update source mappings for migration

* Fix codespell

Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>

* Update lib/plausible/ingestion/acquisition.ex

Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>

* Standardize access to utm params

* Add wikipedia as "known" source

* Move custom sources to json file

* Add some advertising utm_sources

* Move source mapping logic to refinspector file

* Rename PlausibleWeb.RefInspector -> Plausible.Ingestion.Source

* Move mapping overrides to custom_sources.json

* More robust detection of paid sources

* Add missing utm_sources to migration

* Codespell

* Add moduledoc for Plausible.Ingestion.Source

* Fix dialyzer

* Remove migration

* Add more custom favicons

* Re-generate referrer favicons file

* Add doctest for sources

---------

Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>
2024-10-30 13:41:51 +00:00
Karl-Aksel Puulmann
62fb285b71
Comparisons: Fix exports issue (#4756)
* Increase max_parser_depth setting

This is currently causing issues in production with exports w/ comparisons, but
would blow up in other cases with sufficient data.

The reason this is now blowing up is that we create a large AND/OR condition
for breakdown comparisons. ecto-ch nests each conditional in its separate parentheses
causing us to reach the default 1000 max_parser_depth limit

We can improve this in a couple of ways:
- Not add unneccessary parentheses in ecto-ch
- Adding a new operator to make the AND/OR nesting require less conditionals, using
arrays and tuples instead

* Don't query comparisons for exports

csv exports don't actually output any comparison data, so querying for
it just causes issues.
2024-10-30 13:40:26 +00:00
Adrian Gruntkowski
dd79ad3c22
Fix further team sync discrepancies (#4755)
* Sync removing and rejecting invitations and site transfers

* Sync invitation/transfer ID with each new invite

* Accept only subset of guest invitations on sync

* Pass test

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
2024-10-30 11:20:33 +00:00
Karl-Aksel Puulmann
b5d48a7347
Breakdown comparisons: Tooltip UX, arrows (#4719)
* Revert "Revert "Remove no change icon, better alignment""

This reverts commit a69d0a14a7fe038fe110907571240ad99a2ba6a4.

* different arrows

* Render graph tooltips in react

* Use ChangeArrow in graph tooltips

* Only display a single date if viewing a single date

* Tests for breakdown responses with labels

* Simplify code a bit

* strokeClass

* refactor ChangeArrow

* Play with stroke widths

* jest tests

* Silence logging in jest tests

* not variable stroke width

* prettier

* Conversion rate widths

* Update a test

* Trim trailing dates for label when comparing running month

* Reformat file

* Update expect.toHaveTextContent for multiline

* Update date range labelling to be more consistent with the frontend

* reformat
2024-10-30 10:19:47 +00:00
Adrian Gruntkowski
dc7e019d58
Adjust GA4 import timeout and page size to reduce risk of 502s and timeouts (#4753) 2024-10-29 11:24:01 +00:00
RobertJoonas
ad8f49b38d
fix dashboard crash (#4754) 2024-10-29 10:59:58 +00:00
RobertJoonas
37116a2b12
Add tests for the pageleave script (#4744)
* move util function to util module

* move playwright config file to tracker dir root

* update Playwright + add gitignores

* Playwright: enable reuseExistingServer (non-CI env)

* store tracker src copies to avoid re-compilation in dev env

* test pageleave on simple navigation

* fix test util fn

* rename local_test npm script

* make test util able to expect multiple requests

* test pageleaves in SPAs

* test pageleave with manual URL

* test pageleave not sent in manual when pageview not triggered

* extend util fn to refute event requests

* test pageleaves not sent in excluded hash pages

* store hashes instead of file copies to detect /tracker/src changes

* drop async

* speed up test suite
2024-10-28 15:30:03 +00:00
RobertJoonas
1a6fd1dc15
Remove unused controller action (#4748)
* remove unused controller action

* remove unused alias
2024-10-28 14:33:53 +00:00
Adrian Gruntkowski
a3ab3c9061
Ensure upserted invitations and memberships are up to date on conflict (#4749) 2024-10-28 13:54:57 +00:00
Adrian Gruntkowski
27ac3b6b5b
Address team syncing discrepancies (#4739)
* Clean site transfers after 48 hours

* Sync accepting site transfers and invitations within transaction

* Add dry run mode to teams backfill and make it a default

* Extend invitation clean worker tests
2024-10-28 10:11:47 +00:00
RobertJoonas
b49fc82c2b
add size reporter GH action (#4745) 2024-10-28 09:59:16 +00:00
hq1
8bd32b68c9
Bring back premium feature notices in tiles (#4742)
* Lost tapes: bring back premium feature notices in tiles

* Ensure that e-mail CTA is only presented to business/enterprise customers

* Format

* Fix typespecs

* Add t() type to EnterprisePlan

* Reduce Plan.t() definition

Found no reason for it to be there.
2024-10-28 09:07:18 +00:00
Adrian Gruntkowski
0404522b5a
Sync team consistently on site creation (#4736) 2024-10-24 18:44:28 +00:00
Adrian Gruntkowski
2ec9e32557
Extend team consistency checks and improve syncing team and memberships (#4735)
* Extend team consistency checks

* Sync team against user right after creating the site

* Prune orphaned team guest memberships and invitations on site removal
2024-10-24 13:38:58 +00:00
hq1
c1a6501ec1
Delete teams along with users on DELETE /me (#4734)
* Delete teams along with users on DELETE /me

* Delete orphaned teams during backfill

* Format
2024-10-24 10:24:12 +00:00
Adrian Gruntkowski
5dbe1870e0
Implement teams consistency check and improve teams backfill (#4733)
* Implement team consistency check script

* Improve backfill teams sync check

* Fix team grace period sync
2024-10-24 10:03:22 +00:00
Karl-Aksel Puulmann
35851e9084
Comparisons: Only render comparisons in top stats when both query and data supports it (#4732)
* Comparisons: Only render comparisons in top stats when both query and data support it

This fixes a bug from navigation:
1. Comparison disabled, choose a non-realtime period (e.g. press T for "Last 30 days")
2. Enable comparison mode (press X)
3. Choose the realtime period (press R)
4. Choose a non-realtime period again (press T)

Kudos to robert for discovering the repro case.

* Fix conditional

Broken when fixing another bug d727ba5ed5 (diff-f2361637bc87773faced883d7560491e4612b7581f4748f03241821f4ff8f6feL166)
2024-10-24 08:27:55 +00:00
Karl-Aksel Puulmann
a59f12733d
Fix filter transformations (#4727)
Previously the code would use the incorrect operator and lead to nesting
2024-10-23 15:33:34 +00:00
hq1
1234cbf2c1
Enable teams sync (#4726) 2024-10-23 12:25:46 +00:00
RobertJoonas
0f93328b6f
Fix custom property value filter suggestions (#4720)
* new endpoint for custom prop value filter suggestions

* add test coverage for searching prop suggestions

* plug the new endpoint into FE

* update changelog

* format JS

* Update CHANGELOG.md

Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>

---------

Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>
2024-10-23 12:24:23 +00:00
Adrian Gruntkowski
6695f2d7a0
Fix backfilling teams for users without sites but with subscriptions (#4725) 2024-10-23 11:59:33 +00:00
Adrian Gruntkowski
7d6f10f0cb
Ensure team is present before use in sync logic (#4709)
* Ensure team is present before use in sync logic

* Ensure teams backfill works against partially assigned sites

* Associate site with team on creation

* Associate site with team on sync

* Reuse alias

* Add tests for invitation creation sync

* Move team assertions to a helper module

* Format

* Test team creation on site creation via Sites context module

* Add tests for teams sync on subscription changes

* Tag tests

* Test grace period start syncing up with teams

* Test grace period manual lock sycning w/ teams

* Test grace period end sycing up w/ teams

* Test clearing grace period sync with teams

* Update moduledoc

* Fix missing preloads and wrong result pattern matching in sync logic

* Test sync on accepting invites and site transfers

* Test sync on membership role update and member removal

* transfer async fix WIP

* Stop privisioning team in site factory

* Remove unused relationship from Site schema

* Ensure consistent parsing of `passthrough` from Paddle webhook

* Update team passthrough notification tests & logic

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
2024-10-23 11:29:15 +00:00
Karl-Aksel Puulmann
7794ff1417
Fix bounce_rate metric causing 500s with comparisons (#4723) 2024-10-23 10:18:23 +00:00
Karl-Aksel Puulmann
296b3e53d9
Make time-on-page ungraphable (#4721)
This was marked as graphable in FE but not in BE previously. After removing the BE check,
this started throwing errors:

https://sentry.plausible.io/organizations/plausible/issues/626/?project=2&query=is%3Aunresolved+issue.priority%3A%5Bhigh%2C+medium%5D&referrer=issue-stream&statsPeriod=14d&stream_index=0
2024-10-23 09:56:47 +00:00
Karl-Aksel Puulmann
d648505a9b
Fix money values on graphs and conversions graph tooltips (#4717)
* Fix conversions graph tooltips

* Show revenue value instead of `undefined` in graph

This broke due to the unification of formatters. Not sure if this is the best fix
2024-10-22 12:43:49 +00:00
Karl-Aksel Puulmann
d727ba5ed5
Restore average/total revenue to top stats (#4716)
* Restore average/total revenue to top stats

These werent being graphed due to incorrect conditional

* prefer ternary
2024-10-22 11:47:29 +00:00
Adrian Gruntkowski
be83b7829d
Improve GA4 import error handling (#4714)
* Log warnings on HTTP GA4 import errors and add Sentry context on socket errors

* Treat socket and 5xx errors as recoverable (up for retry in ~1hr) in GA4 imports
2024-10-22 10:57:29 +00:00
Karl-Aksel Puulmann
472f4f181c
Comparisons pagination fixes (#4697)
* Add filter clauses for each main result filter

This handles the case where main query has a limit and results change. Doesnt handle metrics like percentage.

* Fix percentage calculations by ignoring breakdown-related filters in totals queries

* Refactor comparisons test suite

* Move comparisons logic to comparisons module

* New route for internal query tests

Only to be used in testing

* Support comparison queries with imports/breakdowns

* time dimension predicate extraction

* Clean up a test

* Update docstring

* Update route test

* fix a typo
2024-10-22 10:23:40 +00:00
Karl-Aksel Puulmann
aec0318c3b
Dashboard: show comparison for breakdowns (#4692)
* Comparisons: Move code to LegacyQueryBuilder

* WIP: Return comparison results to frontend

* refactor: remove useless param

* Different result format

* Pass object to metric.renderValue

* remove dead code

* Fixup response format

* Comparison in a tooltip

Not perfect at all, but a good start. Problems arise with money etc.

* Simple change arrow

* Extract metric entry to ts

* popper attempt WIP

* Slightly nicer content

* Solve warning

* Unified changeArrow in app

* Remove needless spanning

* Always set `graph_metric` in top stats.

FE already has business logic around whether a given metric is graphable

* Remove dead code

* Move Money module under dashboard utils, keep in build

* change <Metric /> definition to take in a `formatter` and store default formatters in another (typed) const

* Use standard system for formatting numbers

* Arrows only in table

* remove dead import

* Inline renderValue

* Render metric name in tooltip

* numberFormatter -> numberShortFormatter

* numberShortFormatter update

* Separate long/short formatters

* Use long vs short formatters

* Put column name into tooltip

* Slightly improved label handling for percentages, conversion rate

* Improved boundary handling in tooltip.js

* Iterate tooltips, no tooltip for - revenue

* Update top stats tests after graph_metric change

* Change revenue metrics stats API return structure

Conversion now happens earlier in query pipeline, we return float for comparison purposes

* useQueryContext in a component

* graph_metric for current visitors to fix realtime view

* No tooltips if fully - row

* renderValue as a proper function

* Simplify MetricEntry

* Use common const

* tooltip to typescript

* More explicit return structure

* metric-entry -> metric-value

* Restore some files

* ChangeArrow

* Restore MoreLink

* Fix typing in MoreLink

* <MetricValue />

* Tests for MetricValue and ChangeArrow

* details modal fixups

* re-add space between arrow and percentage

* Solve stylelint issues

* Update test

* Format

* Add flag `breakdown_comparisons_ui`

* reformat

* Remove no change icon, better alignment

* Revert "Remove no change icon, better alignment"

This reverts commit a8d62b6383.

* number-formatter.ts

* numberLongFormatter refactor

* useMemo dependency

* Handle nulls/undefined in top stats

---------

Co-authored-by: Uku Taht <uku.taht@gmail.com>
2024-10-22 09:02:13 +00:00
ruslandoga
0e4e614d4c
Remove eex artifact (#4712)
fixes #4710
2024-10-22 07:20:39 +00:00
Adrian Gruntkowski
3023d322d4
Limit resource usage in teams backfill data migration (#4708) 2024-10-21 08:52:07 +00:00
hq1
4bb8b44cad
Fix docs URL slug (#4707) 2024-10-21 08:27:34 +00:00