Commit Graph

125 Commits

Author SHA1 Message Date
hq1
f4f4d70adb
Only check usage for EPs by _current_ subscription (#4929) 2024-12-19 10:58:20 +00:00
Adrian Gruntkowski
729a32e610
Teams writes switch (#4883)
* Comment out legacy fields and relationships

* WIP

* WIP 2

* WIP 3

* wip

* Remove teams backfill and consistency check scripts

* WIP 3

* Fix CheckUsage tests

* Update billing/subscription tests

* WIP 4

* Make site transfer fail if some invitation already exists

* Fixup: do symmetric invitation/site transfer check

* Update UI bugs: make listing sites/inviting admins work like before

* Fix Sites test

* Fix external sites controller test

* Fix live sites tests

* Fix props availability lookup

* Fix site controller tests

* Fix billing controller tests

* WIP - accept invitation tests

* Another round of test fixes + invitations logic bugs

* users_test -> teams_test

* Update registration via invitation

Here, we still rely on "polymorphic" invitation structures,
hence the "unified by id" helper.

For now, it'll remain local unless we discover it's
needed in the broader `Teams.Invitations` context.

cc @zoldar

* Yet another round of test and bugfixes along the way

* Include team in site setup success e-mail

* Fix send_site_setup_emails worker

* Fixed almost all tests except CRM ones

* Update enterprise plan admin test

* Fix CRM + remaining tests

* Address credo warnings (modulo one FIXME)

* Remove last FIXME and rephrase the invitation test case description

* Set Team fields via User CRM transparently

* Map user reference in Enterprise Plan CRM via team owner

* Fix resource actions in user CRM

* Get rid of warning when opening create form in API keys CRM

* Stop emitting warnings when editing Enterprise Plans via CRM

* Tests: Bump await_clickhouse_count interval

* Remove XXX marker

* Fix register from invitation link in email sent for ownership transfer

* Simplify fetching all pending site ownership site IDs

* Remove commented out schema fields

* Remove unused functions

* Address flakiness in ingest counter tests

* Remove unused `Teams.Sites.create`

* Don't restart trial on team with subscription when creating site

* Account for cases of legacy teams with empty trial expiry date

* Revert "Address flakiness in ingest counter tests"

This reverts commit 60dc1e4115.

* Fix flaky ingest counters tests under load

* Attempt 2

* Pre-emptively hardcode site ids in sampling cache test

to avoid supplying the same IDs alongside with counters test,
that inserts through another repo (async).
what we're observing is, clickhouse not summing mergetree columns fast
enough, even though we wait quite a bit.

* Fix ingest counter tests by accounting for delayed summation

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
2024-12-16 11:11:14 +00:00
Adrian Gruntkowski
3afec60d98
Remove team adapters (#4877)
* wip

* wip

* 🍌

* WIP

* Draw the rest of the owl (well, almost)

* Remove obsolete unlimited trial logic

* Remove `allow_next_upgrade_override?` adapter

* Remove `Teams` adapter

* Remove /sites adapters

* Remove `Sites` adapter

* Remove `change_plan` adapter

* Fix up CE test

* Remove adapter for Billing.latest_enterprise_plan_with_price(s)

* Remove adapter for `Billing.has_active_subscription?`

* Remove adapter for `Billing.active_subscription_for`

* Remove remaining billing adapter

* Remove all_pending_transfers

* Remove `get_owner` adapter

* Remove `has_sites?` and `owns_sites?` adapters

* Remove `Ownership` adapter

* Remove `check_invitation_permissions` adapter

* Remove `check_team_member_limit` adapter

* Remove `ensure_transfer_valid` adapter

* Remove Invitations adapter

* Remove sole teams adapter

* Make dialyzer happy

* Consolidate `Billing.features_usage` definition

* Remove unused `Quota.Usage`

* Remove remains of `read_team_schema` FF and reduce number of CI passes

* Bang up the condition

* Include pending invitations when querying `has_sites?`

* Fix and improve conditional expression in `plan_box` component

* Update comments

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
2024-12-05 09:02:09 +00:00
Adrian Gruntkowski
b86c2e715f
Switch reads to Teams schemas across the rest of the app (#4860)
* Pre-emptively introduce `site.team_owner` relation

* Drop null constraint on user_id from subscriptions and enterprise_plans

* Temporarily remove populating old schemas in Teams.Test

* Point to site.owner via new schema

* Switch more reads to teams schema WIP

* Fix AuhtorizeSiteAccess test

There's no need to translate `admin`<->`editor` here,
the redundancy is inlined wherever the plug is initialized.

* Fix regions test

* Fix main graph test

* Fix authorization test

* Try to rely on team for subscription/plans where applicable

* Test fixes

* Fix plans test

* Prep for CheckUsage changes

* Skip remaining CheckUsage tests for now

* Fix user deletion to account for team relations

* Fix HelpScout tests

* 💀 Modify ingestion to read team schemas

* Made all tests green except skipped ones

* Mute warnings about transferring site with no order

By making artificial site membership struct,
when reading data off team membership schema.

* Fix site removal test case

* Re-enable locked site tests, that don't have to rely on `SiteLocker`

* Format

* Revert "Mute warnings about transferring site with no order"

This reverts commit 0e45f8c9d9.

* Re-enable old models and fix remaining tests

* Use new factories in a long running minio test

* FIXME->TODO

* Fix remaining tests in legacy mode (no FF raised)

* oof

cc @zoldar

* Add missing definitions of editor role in FE code

* Remove no longer relevant comment about roles

* Fix JS formatting

* Always prioritize site transfers over memberships in sites list

* Fix misaligned "Reject" invitation button

* Fix site pinning when user is guest in multiple sites in team

* Fix subscription settings controller tests

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
2024-12-02 12:20:16 +00:00
Adrian Gruntkowski
95471c0085
Switch AcceptInvitation ops to read from team schemas behind FF (#4847)
* Move `bulk_transfer_ownership_direct` under `AcceptInvitation`

* [WIP] Switch ownership transfer operations to read from team schemas behind FF

* Fix usage test regression

* Semantics - current user; ownership is not necessarily involved

* Perform remaining read via adapter; remove obsolete test

* Properly list site with pending site transfer while being guest on a team

* Account for pending site transfers in Settings > People list

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
2024-11-26 08:47:12 +00:00
Adrian Gruntkowski
380dc00d1a
Switch on teams schema in choose plan view behind FF (#4838)
* Switch on teams schema in choose plan view behind FF

* Proxy via Read adapter where applicable for billing context

* Proxy remaining plan-related functions

* Switch enterprise_configured?/1 tests to use the adapter

* Format

* Update SiteLocker tests

* Actually use `has_active_subscription?/1` billing adapter

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
2024-11-20 08:35:18 +00:00
Adrian Gruntkowski
27ac3b6b5b
Address team syncing discrepancies (#4739)
* Clean site transfers after 48 hours

* Sync accepting site transfers and invitations within transaction

* Add dry run mode to teams backfill and make it a default

* Extend invitation clean worker tests
2024-10-28 10:11:47 +00:00
Adrian Gruntkowski
7d6f10f0cb
Ensure team is present before use in sync logic (#4709)
* Ensure team is present before use in sync logic

* Ensure teams backfill works against partially assigned sites

* Associate site with team on creation

* Associate site with team on sync

* Reuse alias

* Add tests for invitation creation sync

* Move team assertions to a helper module

* Format

* Test team creation on site creation via Sites context module

* Add tests for teams sync on subscription changes

* Tag tests

* Test grace period start syncing up with teams

* Test grace period manual lock sycning w/ teams

* Test grace period end sycing up w/ teams

* Test clearing grace period sync with teams

* Update moduledoc

* Fix missing preloads and wrong result pattern matching in sync logic

* Test sync on accepting invites and site transfers

* Test sync on membership role update and member removal

* transfer async fix WIP

* Stop privisioning team in site factory

* Remove unused relationship from Site schema

* Ensure consistent parsing of `passthrough` from Paddle webhook

* Update team passthrough notification tests & logic

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
2024-10-23 11:29:15 +00:00
Adrian Gruntkowski
17b12ddaeb
Implement basics of Teams (#4658)
* Extend schemas with new fields and relationships for teams

* Implement listing sites and sites with invitations with teams

* Implement creating invitations with teams

* Implement accepting invites with teams

* Add `Teams.SiteTransfer` schema

* Implement creating ownership transfers

* Implement accepting site transfer between teams

* Make results shapes from `Teams.Memberships` role functions more consistent

* Remove :team relation from ApiKey schema

* Pass and provision team on subscription creation

* Pass and provision team on enterprise plan creation

* Implement creating site for a team

* Keep team in sync during legacy ownership transfer and invitations

* Resolve conflict in `Teams.get_or_create` without transaction

* Abstract `GracePeriod` manipulation behind `Plausible.Users`

* Put `User.start_trial` behind `Plausible.Users` API

* Sync team fields on user update, if team exists

* Sync cleaning invitations, updating and removing members

* Transfer invitations too

* Implement backfill script

* Allow separate pg repo for backfill script

* Rollback purposefully at the end

* Update backfill script with parallel processing

* Use `IS DISTINCT FROM` when comparing nullable fields

* Handle no teams to backfill case gracefully when reporting

* Parallelize guest memberships backfill

* Remove transaction wrapping and query timeouts

* Make team sync check more granular and fix formatting

* Wrap single team backfill in a transatction for consistent restarts

* Make invitation and site transfer backfills preserve invitation ID

* Update migration repo config for easier dev access

* Backfill teams for users with subscriptions without sites

* Log timestamps

* Put teams sync behind a compile-time flag

* Keep timestamps in sync and fix subscriptions backfill

* Fix formatting

* Make credo happy

* Don't `use Plausible.Migration` to avoid dialyzer complaining

None of the tooling from there is used anywhere and `@repo` can
be defined directly in the migration script.

* Drop SSL workarounds in the backfill script

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
2024-10-21 07:35:23 +00:00
hq1
70997abfc7
Reduce Phoenix.HTML usage (#4675)
* Turn .eex templates into .heex

* Add new compile-time presets to `PlausibleWeb`

* Fix remaining templates

* Update static components

* Update live components

* Update live views

* Update rest of the owl

* Update mjml template

* Format

* Format

* Revert MJML stuff, it's coupled with EEx

* yawn at test

* Get rid of `FormHelpers` module

* Ensure YOU label shows up first on IP rules list

* Update lib/plausible_web/templates/email/welcome_email.html.heex

Co-authored-by: Artur Pata <artur.pata@gmail.com>

* Fix create site email link

* Fix server error markup (and turn thanks into heex)

* Format

---------

Co-authored-by: Artur Pata <artur.pata@gmail.com>
2024-10-15 03:55:14 +00:00
Adrian Gruntkowski
6981972617
Make invitations cleanup worker rely on UTC time (#4610) 2024-09-25 12:44:55 +00:00
hq1
3251b5bb33
Check usage for users with active/cancelled/past due subscriptions (#4607)
* Check usage for users with active/cancelled/past due subscriptions

So that over limit e-mails and grace period is handled
regardless - they're still subscribers.

* Ensure current subscription is fetched, for an ongoing plan

* Add extra test

* Revert "Ensure current subscription is fetched, for an ongoing plan"

This reverts commit 9ddc7bdccb.

* Fixup test

* Reapply "Ensure current subscription is fetched, for an ongoing plan"

This reverts commit 7ab537954b.

* Do a subscription preload and assert ID match

* Use inner lateral join to fetch subscriptions
2024-09-25 11:27:12 +00:00
Adrian Gruntkowski
373d4dd665
Implement token-based sessions (#4463)
* Turn `Plausible.Auth.UserSession` into full schema

* Implement token based sessions and use them as default

* Ignore expired user sessions during retrieval from DB

* Implement plug bumping user session last used and timeout timestamps

* Implement Oban worker removing expired user sessions with grace period

* Implement legacy session conversion on touch, when applicable

* Update `UserAuth` moduledoc

* Extend `UserAuth` tests to account for db-backed session tokens

* Update CHANGELOG

* Add tests for `UserSessionTouch` plug

* Add test for `CleanUserSessions` worker

* Add logging of legacy session retrievals

* Use single update permitting stale records  when touching user session

* Don't fetch session and user for external API endpoints (/api/event too)

* Refactor `Users.with_subscription/1` and expose helper query

* Skip fetching session in legacy `SessionTimeoutPlug`

* Rely on user session assign from `AuthContext` in `SentryContext`

* Silence legacy session warnings in `UserSessionTouchTest`

* Rely on session assign from `AuthPlug` in `SuperAdminOnlyPlug`

* Change `UserAuth` to get session, user and last subscription in one go

* Avoid refetching user session in `AuthorizeSiteAccess` plug

* Fix code formatting

* Refactor `UserAuth.get_user_token/1` (h/t @aerosol)

* Remove bogus empty opts from `scope` declarations in router

* Only touch session once an hour and keep `user.last_seen` in sync

* Bring back logging of legacy token use
2024-09-03 11:34:37 +02:00
hq1
e3af1a317d
Onboarding improvements (#4459)
* Migration: add installation meta

* Update site schema with installation meta

* Remove VERIFICATION_ENABLED env var

* Add context API to create/remove special goals

* Add context api to update installation meta

* Remove verification enabled check

* Update new progress flow definitions

* Update generic components

* Remove internal /status API

* Implement installation live view

* Update traffic change notifier link

* Update verification, no more modal

* Update routes

* Remove focus.html - will unify everything under app layout

* Fix broken link

* Update templates with focus_box mostly

* Update controller tests

* Update controllers and stop using the focus layout

* copy changes

* Update verification.ex

* Remove dead template

* Update settings_general.html.heex

* Update copy in tests

* Update installation.ex

* Remove dangling dot

* Fix link

* Update installation.ex

* Update installation.ex

* Better tooltips?

* Simpler labels

* Revert "Simpler labels"

This reverts commit 797560ef82f2067458b03b884be5aecc8fdc72bc.

* Add copy to clipboard link and fix snippet's dark mode

* Offer installation detection skip only if ws connected

* Put COPY link at the bottom with background

* Make tooltips link to docs

* Fix cherry-pick gone wrong

* Hide tooltips on mobile screens

* WIP: 404 tracking wizard

* Revert "WIP: 404 tracking wizard"

This reverts commit a9c9c79bbd.

* Update lib/plausible_web/live/components/verification.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Update lib/plausible_web/live/installation.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Use current_user from socket.assigns

* Update lib/plausible_web/live/installation.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Use current_user from socket.assigns

* Use conn.private to steer verification tests

* Drop non-sticky tooltip in favour of component parametrization

Co-authored-by: Artur Pata <artur.pata@gmail.com>

* Reapply "WIP: 404 tracking wizard"

This reverts commit 3ba81671d7.

* Fix installation tests including 404 tracking

* Fixup the tooltip component

* Format

* Update installation.ex

* Put flash whenever installation option changes

* Use last known installation type on domain change

* Extract user flow definition to provide compile-time checks

* See if this helps running CE migrations successfully

* Use `styled_link` on registration/login views

* Don't crash when there's no conn.private carried over

* Format

* Push "Determining installation type" message a bit lower

* Use links and footer lists uniformly

This commit introduces a `<.focus_list/>` component
for rendering focus box footer links with colored
discs. It also equips generic link components
with the ability of sending non-GET requests
along with CSRF token, so we can apply uniform
styling and stop using legacy Phoenix link tags.

cc @zoldar @apata

* ws 👾

* Render more descriptive flashes on script config change

---------

Co-authored-by: Marko Saric <34340819+metmarkosaric@users.noreply.github.com>
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
Co-authored-by: Artur Pata <artur.pata@gmail.com>
2024-09-02 12:49:54 +02:00
Uku Taht
b868042581
Refactor: remove transform_filters function from imported base (#4409) 2024-07-31 15:36:25 +03:00
Uku Taht
49bb57f601
Fix email report sentry errors (#4381)
* Fix missing email param on email unsubscribe

* Guard against email report being deleted
2024-07-23 12:05:42 +03:00
hq1
d56bb2b4d5
Implement traffic drop notifications (#4300)
* Expose current visitors 12h aggregate

* Remove unused site association

* Distinct drop/spike notification factories

* Rename modules accordingly + implement drop handling

* Rename periodic oban service

* Implement drop email

* Rest of the owl

* Update changelog

* Update moduledoc

* Update moduledoc

* Min threshold to 1

* Threshold 1

* Remove merge artifact

* Put panel behind a feature flag

* Format
2024-07-11 14:55:18 +02:00
RobertJoonas
dd1d74ccb7
Always recommend a suitable plan on the choose-plan page (#4222)
* pull last_bill_date from paddle sandbox in mix task

* move cycle usage checks to Quota module

* move quota.ex to a subfolder

* split up Quota module

* set choose-plan pageview slider according to usage

* silence credo
2024-06-17 09:25:46 +03:00
hq1
4603aa7a51
Reset grace period if usage drops within the last cycle (#4212)
So that `LockSites` job unlocks the site,
encountering no grace period and no upgrade needed.
2024-06-11 15:49:31 +02:00
Karl-Aksel Puulmann
0a883f10e7
Refactor: Use common current_visitors code (#4071)
* Use common module for counting current visitors in external stats controller

* Refactor spike notifier, remove now-dead code
2024-05-07 15:03:37 +03:00
ruslandoga
972dd5d150
redirect to s3 url when downloading exports (#4002)
* redirect to s3 url

* use new on_ee macro, reduce wait time for email to five seconds
2024-05-02 19:53:12 +01:00
hq1
d6824de1ad
Rename internal build symbols (#3942)
* Rename internal build symbols

* Rename remaining + add `on_ce` macro

cc @ruslandoga
2024-04-29 08:05:33 +02:00
Adrian Gruntkowski
9bae3ccce3
Improve UI/UX of imports view and GA import flow (#4017)
* Add runtime config option for enabled/disabling csv imports and exports

* Use the new option to toggle rendering exports UI

* Disable import buttons when at maximum imports or when option disabled for CSV

* Improve forms for GA import flow

* Add test for maximum imports reached

* Remove "Changed your mind?" prefixing back button

* Hide UA imports in Integrations when `imports_exports` flag is enabled

* Implement `csv_imports_exports` feature flag

* Revert "Add runtime config option for enabled/disabling csv imports and exports"

This reverts commit e30f202dd3.

* Send import notification email only to the user who ran the import

* Improve rendering of disabled button state

* Put import status heroicon in front of import label
2024-04-18 12:12:48 +02:00
Adrian Gruntkowski
c1c03b729c
Reapply "Local CSV exports/imports and S3/UI updates (#3989)" (#3995) (#3996)
* Reapply "Local CSV exports/imports and S3/UI updates (#3989)" (#3995)

This reverts commit aee69e44c8.

* remove unused functions

* eh, that one was actually used

* ugh, they were both used

---------

Co-authored-by: ruslandoga <67764432+ruslandoga@users.noreply.github.com>
2024-04-11 09:15:01 +02:00
Adrian Gruntkowski
aee69e44c8
Revert "Local CSV exports/imports and S3/UI updates (#3989)" (#3995)
This reverts commit 1a0cb52f95.
2024-04-09 21:26:23 +02:00
ruslandoga
1a0cb52f95
Local CSV exports/imports and S3/UI updates (#3989)
* local CSV exports/imports and S3 updates

* credo

* dialyzer

* refactor input columns

* fix ci minio/clickhouse tests

* Update lib/plausible_web/live/csv_export.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* fix date range filter in export_pages_q and process only pageviews

* remove toTimeZone(zero_timestamp) note

* use SiteImport.pending(), SiteImport.importing()

* escape [SiteImport.pending(), SiteImport.importing()]

* use random s3 keys for imports to avoid collisions (sometimes makes the upload get stuck)

* clamp import date ranges

* site is already in assigns

* recompute cutoff date each time

* use toDate(timestamp[, timezone]) shortcut

* show alreats on export cancel/delete and extract hint into a component

* switch to Imported.clamp_dates/4

* reprocess tables when imports are added

* recompute cutoff_date on each call

* actually use clamped_date_range on submit

* add warning message

* add expiry rules to buckets in make minio

* add site_id to imports notifications and use it in csv_importer

* try/catch safer

* return :ok

* date range is not available when no uploads

* improve ui and warning messages

* use Generic.notice

* fix flaky exports test

* begin tests

* Improve `Importer` notification payload shape

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2024-04-09 20:59:48 +02:00
Karl-Aksel Puulmann
a6d4786959
Worker to clean site data from ClickHouse (#3959)
* Create a worker to clean clickhouse deleted sites data

The plan is to run this weekly, but going to trigger it manually the first few times on cloud

* Make asserting count more reliable

* credo

* PR feedback

* Fixes
2024-04-08 12:26:38 +03:00
Karl-Aksel Puulmann
8d977e0f76
Tests: session properties without the prefixes (#3863)
* Undo event session attributes renaming

* Rename session_ attributes in tests
2024-03-05 12:44:33 +02:00
Karl-Aksel Puulmann
d5048fd6b4
Stop writing session properties into events table (#3800)
* Refactor: Explicitly add field names to INSERT

This avoids issues when code schema is out of sync with real schema

* Dont write session parameters to events

These would only be stored on first event anyways. Work remains to be done
on tests which have their own helper

* Remove writes to country_code in a test

* Remove old columns from being accessible in elixir code

* Update most tests to use new way of adding session props to events

* Update testing harness

* Update stats controller test

* Update for shield rules

* update breakdown tests

* Fix typing of state for dialyzer

* Drop support for old session attributes code

* Update remaining tests

* cond -> if
2024-03-01 10:53:56 +02:00
Adrian Gruntkowski
39aa81a16f
Implement UI for multiple imports (#3727)
* Create a stub of site settings section for imports and exports

* Use legacy site import indication to determine UA import handling

* Add provisional logos for upcoming import sources

* Stub basics of import page

* Add very rudimentary support for multiple UA imports

* Implement imports list as live view

* Add support for opening LV modal from backend and closing from frontend

* Introduce notion of themes to `button` and `button_link` components

* Add confirmation modal on deleting import

* Swap GA4 logo

* Implement disabled state support for `button_link` component

* Disable export and non-implemented import sources

* Use native starts start date for upper boundary of import time range

* Ensure integrations view uses legacy UA import flow

* Remove unnecessary preload in SiteController

* Remove unnecessary exception for legacy imports

* Move API controller stats tests under PlausibleWeb

* Test listing imports

* Add test for explicit listener setup

* Add tests for legacy flag state in UA importer

* Add test for purging legacy import data

* Add tests for `Sites.native_stats_start_date`

* Test forgetting imports

* Add `Stats.Clickhouse.imported_pageview_counts/1` and fix test flakiness

* Show page view counts on imports list

* Add tests for static imports and exports view

* Adjust button look slightly

* Use `case` instead of `cond`

* Make feature flag customisable per site

* Fix buttons and empty state styling

* Add another import to seeds

* Use JS confirm dialog instead of modal for deletion confirmations

* Revert "Add support for opening LV modal from backend and closing from frontend"

This reverts commit 260e6c753032b451542e24be9edc2118790b5a00.

* Default `legacy` to false when inserting new import jobs

* Drop `method` attribute from `button_link` and `unstyled_link` components
2024-02-28 09:34:04 +01:00
ruslandoga
423f72a0ad
safe(r) Oban telemetry (#3743)
* safe telemetry

* use try/catch instead

* add some tests

* cleanup tests

---------

Co-authored-by: hq1 <hq@mtod.org>
2024-02-14 10:12:03 +01:00
Adrian Gruntkowski
f8b4d5066a
Add multiple imports per site (#3724)
* Clean up references to no longer active `google_analytics_imports` Oban queue

* Stub CSV importer

* Add SiteImport schema

* Rename `Plausible.Imported` module file to match module name

* Add `import_id` column to `Imported.*` CH schemas

* Implement Importer behavior and manage imports state using new entities

* Implement importer callbacks and maintain site.imported_data for UA

* Keep imports in sync when forgetting all imports

* Scope imported data queries to completed import IDs

* Mark newly imported data with respective import ID

* Clean up Importer implementation a bit

* Test querying legacy and new imported data

* Send Oban notifications on import worker failure too

* Fix checking for forgettable imports and remove redundant function

* Fix UA integration test

* Change site import source to atom enum and add source label

* Add typespecs and reduce repetition in `Plausible.Imported`

* Improve documentation and typespecs

* Add test for purging particular import

* Switch email notification templates depending on import source

* Document running import synchronously

* Fix UA importer args parsing and ensure it's covered by tests

* Clear `site.stats_start_date` on complete import to force recalculation

* Test Oban notifications (h/t @ruslandoga)

* Purge stats on import failure right away to reduce a chance of leaving debris behind

* Fix typos

Co-authored-by: hq1 <hq@mtod.org>

* Fix another typo

* Refactor fetching earliest import and earliest stats start date

* Use `Date.after?` instead of `Timex.after?`

* Cache import data in site virtual fields and limit queried imports to 5

* Ensure always current `stats_start_date` is used

* Work around broken typespec in Timex

* Make `SiteController.forget_imported` action idempotent

* Discard irrecoverably failed import tasks

* Use macros for site import statuses

There's also a fix ensuring only complete imports are considered
where relevant - couldn't isolate it as it was in a common hunk

* Use `import_id` as worker job uniqueness criterion

* Do not load imported stats data in plugins API context

---------

Co-authored-by: hq1 <hq@mtod.org>
2024-02-14 09:32:36 +01:00
Karl-Aksel Puulmann
f3509f2a17
Refactor spike detection top sources query (#3770)
* ORDER BY referrer_source for spikes job

This is more consistent with the rest of the queries

* Refactor top_sources -> top_sources_for_spike

* Remove more dead code

* Remove unused arguments

* Remove unused select arguments

* Add a test to top_sources_for_spike
2024-02-13 08:28:32 +02:00
Adrian Gruntkowski
822483c37c
Extract Universal Analytics import logic (#3700)
* Move imported tables schemas to separate modules outside Google ns

* Move buffer for imports to Imported ns

* fix schema newlines

* Extract UA import processing and persistence

* Decouple analytics worker implementation from UA

* Rename env variable for import buffer size

* Preserve old import queue until release
2024-01-23 10:24:08 +01:00
RobertJoonas
d3094ffdb7
Prevent upgrades that would leave the customer locked (#3683)
* Remove allowance_required field from grace_period

Since we are now preventing customers from subscribing to a plan that
does not accommodate their pageview usage, there is no need for an extra
check on removing the grace period after a successful upgrade.

This extra check is the reason why the automatic unlocks have recently
failed in several cases.

* refactor outgrown subscription notices

* make a test actually test the described functionality

* Apply greater pageview allowance margin only for trial upgrades

...in order to prevent cancelled or paused subscriptions from subscribing
to plans that would still leave their account locked.

* Mark the entire ChoosePlanTest module full build only

* remove account locking guide

This is irrelevant for self-hosters, and the internal knowledge base is
a better place for this document. Moved it there.

* refactor Keyword get clause

* add a pattern matching assertion in code
2024-01-15 14:59:56 +00:00
hq1
b6a2acb57a
Update accept_traffic_until notifications (#3665)
* Update accept_traffic_until notifications:

  - extend has_stats to 48h
  - ensure the user is properly greeted

* Clarify the structure passed to email template function
2024-01-03 11:19:25 +01:00
hq1
f755b20569
Lock traffic notifications (#3641)
* Update communication

* Remove an unreachable function (mistyped)

* [migration] Make accept_traffic_until a date

* Fix typo

* Set `accept_traffic_until` when creating a site

* Update sites `accept_traffic_until` on subscription change

* Add a note to yearly cancellation notification

* Rephrase annual e-mail for clarity

* Pass the small build test

* Add email notifications

* Fixup

* Implement `accept_traffic_until` notification worker

* Fixup - no need to test this for small build

* Update moduledoc

* Move moduletag

* s/sent_at/sent_on

* Use WHERE NOT EXISTS instead of LEFT JOIN

* Use upsert when tracking notifications sent

* Store sent marker before actually sending notification

* Prefer to keep `accept_traffic_until` on the user record

This gives us a single source of truth, addresses cases like
ownership transparently, simplifies the code and enables CRM toggles.
The only downside is that there's another join performed in the
Sites.Cache full refresh - in this case, small refreshes are
skipped - but this is fine, since the traffic will be let in
anyway.

* Expose `accepted_traffic_until` in the CRM

* Update lib/plausible/auth/user.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Preload owner in CRM

* Use the offset parameter in trial over e-mail contents

* Format

* Harden cache test

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2023-12-28 08:42:27 +01:00
RobertJoonas
d2270f3c35
Small bugfix + refactor email reports (#3642)
* use more convenient testing functions

* do not display + sign with 0% change in emails

* Rename module/file/function names

before, `weekly_report` was also used for monthly reports and that was a
bit confusing to read in code.

* Refactor send_email_report.ex

This commit improves readability by refactoring the code into smaller
functions and reducing the number of arguments given to functions.

But more importantly, it stops making duplicate stats queries for every
email recipient by moving the queries out of the for loop.

* Refactor: move querying logic out of the worker module

and merge all stats information under a single `stats` assign.
2023-12-21 12:56:06 +00:00
Marko Saric
0b00762591
Changes to the emails as discussed (#3540)
* Update over_limit.html.eex

* Update dashboard_locked.html.eex

* Update dashboard_locked.html.eex

* Update over_limit.html.eex

* Update dashboard_locked.html.eex

* fix tests

* stop querying owned_site_ids three times

... when querying for billing cycles. Adds an optional `owned_site_ids`
argument to the `usage_cycle` function.

* add penultimate billing cycle info to emails

This commit also refactors some code and adds unit tests to email templates

* use delimit_integer instead of large_number_format

... to display usage with exact numbers such as 1,099,999 instead of 1M

* add penultimate cycle date ranges and linebreaks

---------

Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
2023-12-06 12:02:22 +00:00
hq1
7bf1e2a6ed
Reapply "Define a better monthly pageview usage (#3564)" (#3574)
This reverts commit c739b8878d.
2023-11-30 13:30:04 +01:00
hq1
5278c23965
Revert and extract migration (#3573)
* Revert "Define a better monthly pageview usage (#3564)"

This reverts commit 57188a402a.

* Extract migration from 57188a402a/priv/repo/migrations/20231129103158_add_allow_next_upgrade_override_to_users.exs
2023-11-30 13:19:25 +01:00
RobertJoonas
57188a402a
Define a better monthly pageview usage (#3564)
* refactor asking for the monthly pageview usage

* add tests for usage and limits section in account settings

* display pageview usage per billing cycle for active subscribers

* disable cycle tabs if no usage

* make current billing cycle whole

...instead of capping it at today's date

* run queries for different cycles concurrently

* fix linebreak bug

* add calculate usage action into CRM

* change some names of assigns

* block subscribing to a plan by pageview usage

Depending on whether the customer has already subscribed or not, checking
their pageview usage is different:

* If they're not subscribed yet, we allow them to subscribe to a plan If
  it their last 30 days usage does not exceed the plan pageview limit by
  more than 15% (30% for when subscribing to a 10k plan)

* For existing subscribers, we'll use the exact same mechanism that we're
  using for locking sites - the last two billing cycles usage. If both
  cycles exceed the plan limit by more than 10% - we don't allow them to
  subscribe to the plan

* apply credo suggestion

* prevent highlight bar overflow

* move disabled classes to button element

* optimize for darkmode

* unify link and text styling on the same horizontal line

'Upgrade' & 'Update billing details' links + billing interval text were
positioned on the same line. The font size was similar, but not the same

* improve exceeded_limits function readability

* Refactor some tests and remove code duplication

* override allow upgrade when limits exceeded

In cases where limits are exceeded, we can set the boolean flag
`allow_next_upgrade_override` to `true` in the CRM. This will allow
the user to upgrade to any plan they want. After they've upgraded or
changed their plan - the flag will automatically reset to `false`.

* only apply upgrade override for exceeded pageview limit

* fix tests on the CI

* make current_cycle usage always displayed by default

* make pageview allowance margin more clear

* add comment
2023-11-30 11:50:44 +00:00
RobertJoonas
df44f549d8
Recommending a plan (#3476)
* use a different article in the email copies

... for recommending a plan, since the user can choose between Growth
and Business.

* small refactoring improvement

Rename `Plans.available_plans_with_prices` to `Plans.available_plans_for`,
taking an optional `with_prices` argument.

* highlight recommended tier for trial users on the ugprade page

* review suggestion
2023-11-02 14:46:14 +00:00
Adrian Gruntkowski
70c001099d
Improve and simplify email verification codes generation (#3407)
* Refactor email verification codes generation to avoid predictability

* Improve `Site.Memberships.any?` slightly

* Update tests

* Fix seeds

* Use `expired?` predicate for checking verification code validity in tests

* Store verification code as string in database to avoid unnecessary int casting
2023-10-16 13:21:18 +02:00
RobertJoonas
3d2f356ba7
Refactor enterprise plan upgrade and change-plan actions (#3397)
* rename enterprise?/1 function

* change link text to Upgrade when subscription deleted

* extract paddle_button and paddle_script components

* create a new upgrade-to-enterprise-plan page

* extract upgrade_link component

* rename function

* link to enterprise plan upgrade page from settings

...if the user has an enterprise plan configured

* fetch enterprise plan price on the new page

* add change_enterprise_plan functionality on the new page

* render existing change_enterprise_plan_contact_us.html

...when subscribed to latest configured enterprise plan

* rename vars and extract resumable? fn

* remove dead billing route

* small test refactor: extract convenience fn

* add tests for...

...restricting paused and past_due subscription access to the new
enterprise plan page.

1. redirect to /settings from the controller action
2. hiding the change-plan link from the user settings

* implement redirect to /settings

* hide the enterprise upgrade/change-plan link

* add tests for a deleted enterprise subscription

* plug in the new controller action and delete dead code

* optimize for dark mode

* fix compile warning

* credo fix

* display N/A instead of crash when price nil

* change subscription.status type to Ecto.Enum

Also, create a new `Subscription.Status` module that exposes macros to
return the used atom values (prevent typos at compiletime).

* fix bug (@conn not available anymore)

* use Routes.billing_path where applicable

* add a status() type

* silence credo

* refactor suggestion from review

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Remove the __using__ macro from Subscription.Status

... instead be explicit about requires and aliases and also order
the use, import, require, and alias clauses according to
https://github.com/christopheradams/elixir_style_guide#module-attribute-ordering

* drop the virtual Enteprise 'price_per_interval' field

* apply review suggestion to make the code more DRY

* use dot syntax to fetch current user in new controller actions

* fix formatting

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2023-10-10 20:35:17 +03:00
hq1
29d0f82182
Apply unified capitalization to buttons and titles (#3321) 2023-09-05 09:43:01 -03:00
Vini Brasil
48b02c1396
Avoid suggesting plan for trials switching to enterprise plans (#3282)
This commit fixes a bug where the SendTrialNotifications job tried to
suggest a plan for users switching to enterprise plans, resulting in the
exception below:

```
UndefinedFunctionError: function :enterprise.volume/0 is undefined (module :enterprise is not available)
  Module "enterprise", in :enterprise.volume/0
  File "lib/plausible_web/templates/email/trial_upgrade_email.html.eex", line 5, in PlausibleWeb.EmailView."trial_upgrade_email.html"/1
  File "lib/phoenix_view.ex", line 381, in Phoenix.View.render_within/3
  File "lib/phoenix_view.ex", line 557, in Phoenix.View.render_to_iodata/3
  File "lib/phoenix_view.ex", line 564, in Phoenix.View.render_to_string/3
  File "lib/bamboo_phoenix.ex", line 291, in Bamboo.Phoenix.render_text_or_html_email/1
  File "lib/workers/send_trial_notifications.ex", line 67, in Plausible.Workers.SendTrialNotifications.send_today_reminder/1
  File "lib/workers/send_trial_notifications.ex", line 36, in anonymous fn/2 in Plausible.Workers.SendTrialNotifications.perform/1
```
2023-08-21 09:01:50 -03:00
hq1
9f0e7d54f8
Fix flaky log assertion (#3199) 2023-07-27 12:51:54 +02:00
hq1
e6996ccfda
Fix weekly/monthly report e-mails (MJML) (#3150)
* Render weekly/monthly reports with MJML

* Update changelog

* Fix APP-2NA (#3119)
2023-07-17 16:59:01 +02:00
hq1
9debcdee24
Fix APP-2NA (#3119) 2023-07-11 10:47:14 +02:00