analytics/lib
Karl-Aksel Puulmann 32a16a72fa
Speed up calculating monthly usage for users with many many sites (#4101)
* Delete dead code

* Speed up calculating usage for users with many many sites

Currently, the settings page time outs for a user with 14k sites.

This PR speeds things up by:
1. Doing the work in parallel (max 10 queries at once)
2. Increasing chunking size (300 -> 1000)

Note that the query is relatively lightweight on clickhouse - running
these queries manually takes ~70ms. If this becomes slow we can also
introduce a PROJECTION to speed up the calculation, but this wasn't a
bottleneck currently.

On chunking size:
ClickHouse can handle even 10k site_ids in a single query fast if run
via clickhouse-client , but running the same query via ecto_ch it becomes
really slow (60ms vs 1s).

Not sure if this is a driver, serialization or networking issue.
2024-05-13 13:36:31 +03:00
..
mix/tasks Filter by hostnames (#3963) 2024-04-04 10:48:30 +02:00
plausible Speed up calculating monthly usage for users with many many sites (#4101) 2024-05-13 13:36:31 +03:00
plausible_web Handle s3 timeout on settings page (#4036) 2024-05-13 10:17:27 +02:00
workers Handle s3 timeout on settings page (#4036) 2024-05-13 10:17:27 +02:00
oban_error_reporter.ex Implement resumable GA4 imports to work around rate limiting (#4049) 2024-04-30 18:06:18 +02:00
plausible_release.ex Rename internal build symbols (#3942) 2024-04-29 08:05:33 +02:00
plausible_web.ex Merge Plugins.API.Router into main one (#3767) 2024-02-12 10:44:32 +01:00
plausible.ex Rename internal build symbols (#3942) 2024-04-29 08:05:33 +02:00
sentry_filter.ex Update Sentry (#3843) 2024-03-18 10:10:20 +01:00