* Create a stub of site settings section for imports and exports
* Use legacy site import indication to determine UA import handling
* Add provisional logos for upcoming import sources
* Stub basics of import page
* Add very rudimentary support for multiple UA imports
* Implement imports list as live view
* Add support for opening LV modal from backend and closing from frontend
* Introduce notion of themes to `button` and `button_link` components
* Add confirmation modal on deleting import
* Swap GA4 logo
* Implement disabled state support for `button_link` component
* Disable export and non-implemented import sources
* Use native starts start date for upper boundary of import time range
* Ensure integrations view uses legacy UA import flow
* Remove unnecessary preload in SiteController
* Remove unnecessary exception for legacy imports
* Move API controller stats tests under PlausibleWeb
* Test listing imports
* Add test for explicit listener setup
* Add tests for legacy flag state in UA importer
* Add test for purging legacy import data
* Add tests for `Sites.native_stats_start_date`
* Test forgetting imports
* Add `Stats.Clickhouse.imported_pageview_counts/1` and fix test flakiness
* Show page view counts on imports list
* Add tests for static imports and exports view
* Adjust button look slightly
* Use `case` instead of `cond`
* Make feature flag customisable per site
* Fix buttons and empty state styling
* Add another import to seeds
* Use JS confirm dialog instead of modal for deletion confirmations
* Revert "Add support for opening LV modal from backend and closing from frontend"
This reverts commit 260e6c753032b451542e24be9edc2118790b5a00.
* Default `legacy` to false when inserting new import jobs
* Drop `method` attribute from `button_link` and `unstyled_link` components
* add csv importer
* make table validation explicit
* update some docs
* improve docs
* add minio container to ci
* more tests
* eh
* continue
* add passing test
* add failing test
* add config test
* add minio to Makefile
* testcontainers
* remove extra whitespace
* explain the implementation a bit
* account for async deletes in tests
* bounces is UInt32
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Migration: add country rules
* Add CountryRule schema
* Implement CountryRule cache
* Add country rules context interface
* Start country rules cache
* Lookup country rules on ingestion
* Remove :shields feature flag from test helpers
* Add nested sidebar menu for Shields
* Fix typo
* IP Rules: hide description on mobile view
* Prepare SiteController to handle multiple shield types
* Seed some country shield
* Implement LV for country rules
* Remove "YOU" indicator from country rules
* Fix small build
* Format
* Update typespecs
* Make docs link point at /countries
* Fix flash on top of modal for Safari
* Build the rule struct with site_id provided up-front
* Clarify why we're messaging the ComboBox component
* Re-open combobox suggestions after pressing Escape
* Update changelog
* Fix font size in country table cells
* Pass `added_by` via rule add options
* Display site's timezone timestamps in rule tooltips
* Display formatted timestamps in site's timezone
And simplify+test Timezone module; an input timestamp converted
to UTC can never be ambiguous.
* Remove no-op atom
* Display the maximum number of rules when reached
* Improve readability of remove button tests
* Credo
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* improve test
* add os to os_version breakdown
...and add operating_system_versions.csv to the CSV export
* fix conversion rate for os_version breakdown
* update changelog
* fix existing CSV tests
* use case instead of cond
* Add `GET /capabilities` to Plugins API
It aims to:
- help the client verify the data-domain the token is associated with
- list all the features available for the site's owner
(and therefore determine availability of the subset of those for the current
Plugins API caller)
The endpoint does not require authentication, in the sense that it'll
always respond with 200 OK. However when the token is provided,
a verification lookup is made.
* Remove IO.inspect() call
* Credo
* Aesthetics
* s/send_resp/send_error/
* Call preload just once
This will makes it impossible to store session attributes on events.
Looking at production data also revealed no cases where these updates
are effective.
* Allow e-mail exclusion in team members quota
* Exclude invitee from quota on invitation create
* Enable invitation submission but report errors on quota violation
* Use a single interface for team members quota
* Check the `Keyword.validate/2` result
* Update test/plausible_web/controllers/site/membership_controller_test.exs
Co-authored-by: Uku Taht <Uku.taht@gmail.com>
---------
Co-authored-by: Uku Taht <Uku.taht@gmail.com>
* Revert "Unify percentage change for CR and bounce_rate (#3781)"
This reverts commit a6b1a6ebc7.
* Revert "Bring Stats API up to speed: Add `conversion_rate` to Aggregate and Breakdown (#3739)"
This reverts commit 672d682e95.
* Fix conversion rate change calculation
The change in conversion rate should be calculated similar to bounce rate.
For example, an increase of 25% -> 50% should not be a 100% change, but
a 25% change instead.
* Use the same comparison function in Stats API and dashboard API
This commit fixes a bug where the percentage change reported by the Stats
API is different from the one returned by the internal dashboard API.
* changelog update
* WIP
* Allow `experimetnal_session_count` request serialization
* Extend `Plausible.Stats.Query` with `experimental_session_count` flag
* Add `FunWithFlags` actor implementation for `Site`
* Change the way sessions are retrieved
* Remove redundant test
* Format
* Update the test
---------
Co-authored-by: Uku Taht <uku.taht@gmail.com>
* disable event metric with include_imported in every case
* add missing test for metric validation
* refactor metric validation functions
* implement conversion_rate metric validation
* move calculate_cr function into Stats.Util
* Refactor: Move aggregate CR logic into Stats.aggregate
* define atoms to exist
* Ensure that CR does not depend on visitors being queried
If 'visitors' are already queried, we'll use that value. Otherwise we'll
need to make another query to fetch it.
* confirm Stats API aggregate supports CR (tests only)
* small refactor
This is the only 'event_property' left after pattern matching on all
others in the function clauses defined above.
* Make it possible to optionally query conversion_rate
...in breakdown queries (excluding goal and custom prop breakdown)
* A little refactor asking for revenue metrics
1. The `@revenue_metrics` module attribute is an empty list on full build
anyway
2. We don't need to query for revenue metrics if there are no revenue goals
returned in the given query (even if revenue goals exist in site.goals)
3. Revenue metrics are already dropped in prop breakdown without a goal
filter via (get_revenue_tracking_currency/3)
* Make it possible to optionally query conversion_rate (continuation)
... also from a custom prop and goal breakdown
* Frontend adjustments to the Locations report
* Display conversion rate in Regions and Cities (ListReport view)
* Display total conversions, conversions (visitors), and CR in the
"Details" modals of Countries, Regions, and Cities
* Move the percentage into a separate column in the Countries details table
* confirm Stats API breakdown supports conversion_rate (tests only)
* small refactor: extract maybe_add_time_on_page function
* Make it possible to query cr alone
... (without the visitors metric). Already supported in aggregate, this
commit only implements it for the breakdown API.
* Reuse Stats.Util helper functions from b02db88 for aggregate API
We can follow the same logic as with breakdown for manually adding
`visitors` into the metrics list and taking it out of the response
later on.
That way we don't have to make another query, e.g. in a case where
only pageviews and conversion rate is queried. Also keeps things
consistent.
* changelog update
* fix test after resolving merge conflict
* Use explicit string->atom mapping instead of casting
* alias Util module instead of importing it
* use Enum.empty instead of Enum.any
* improve readability
* rename special_metrics to computed_metrics and explain with a comment
* rename visitors_without_event_filters to total_visitors
* keep a single function for removing unwanted metrics
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Clean up references to no longer active `google_analytics_imports` Oban queue
* Stub CSV importer
* Add SiteImport schema
* Rename `Plausible.Imported` module file to match module name
* Add `import_id` column to `Imported.*` CH schemas
* Implement Importer behavior and manage imports state using new entities
* Implement importer callbacks and maintain site.imported_data for UA
* Keep imports in sync when forgetting all imports
* Scope imported data queries to completed import IDs
* Mark newly imported data with respective import ID
* Clean up Importer implementation a bit
* Test querying legacy and new imported data
* Send Oban notifications on import worker failure too
* Fix checking for forgettable imports and remove redundant function
* Fix UA integration test
* Change site import source to atom enum and add source label
* Add typespecs and reduce repetition in `Plausible.Imported`
* Improve documentation and typespecs
* Add test for purging particular import
* Switch email notification templates depending on import source
* Document running import synchronously
* Fix UA importer args parsing and ensure it's covered by tests
* Clear `site.stats_start_date` on complete import to force recalculation
* Test Oban notifications (h/t @ruslandoga)
* Purge stats on import failure right away to reduce a chance of leaving debris behind
* Fix typos
Co-authored-by: hq1 <hq@mtod.org>
* Fix another typo
* Refactor fetching earliest import and earliest stats start date
* Use `Date.after?` instead of `Timex.after?`
* Cache import data in site virtual fields and limit queried imports to 5
* Ensure always current `stats_start_date` is used
* Work around broken typespec in Timex
* Make `SiteController.forget_imported` action idempotent
* Discard irrecoverably failed import tasks
* Use macros for site import statuses
There's also a fix ensuring only complete imports are considered
where relevant - couldn't isolate it as it was in a common hunk
* Use `import_id` as worker job uniqueness criterion
* Do not load imported stats data in plugins API context
---------
Co-authored-by: hq1 <hq@mtod.org>
* ORDER BY referrer_source for spikes job
This is more consistent with the rest of the queries
* Refactor top_sources -> top_sources_for_spike
* Remove more dead code
* Remove unused arguments
* Remove unused select arguments
* Add a test to top_sources_for_spike
* Add Ecto.Network dependency
* Migration: Add ip block list table
* If Cachex errors out, mark the cache as not ready
* Add IPRule schema
* Seed IPRules
* Add Shields context module
* Implement IPRuleCache
* Start IPRuleCache
* Drop blocklisted IPs on ingestion
* Cosmetic rename
* Add settings sidebar item
* Consider IPRuleCache readiness on health checks
* Fix typo
* Implement IP blocklist live view
* Update moduledocs
* Extend contextual module tests
* Convert IPRules LiveView into LiveComponent
* Keep live flashes on the tabs view
* Update changelog
* Format
* Credo
* Remove garbage
* Update drop reason typespecs
* Update typespecs for cache keys
* Keep track of who added a rule and when
* Test if adding via LV prefills the updated_by tooltip
* Update ecto_network dependency
* s/updated_by/added_by
* s/drop_blocklist_ip/drop_shield_rule_ip
* Add docs link
* s/Updated/Added
* WIP: PropFilterRow
* Get multi-behavior working
* Render multiple prop filters in one
* Modal reads from query string correctly
* Backend support for multiple custom property filters
* Add backend tests for multiple custom property filters
* Disable already selected options in property keys
We can't allow choosing the same property multiple times without changing the request
params, which we decided against
* Allow choosing any property under Behaviors > Custom props even if custom prop filter applied
This was a limitation (I believe) introduced by using ARRAY JOINs to query custom properties
* CHANGELOG.md
* Solve credo warning about too deep nesting
* Update assets/js/dashboard/stats/modals/prop-filter-modal.js
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
* Refactor internal function for clarity
* Add another step -> Add another
* Solve 500 error
* Separate boxes per property filter
* Retain other filters in props table
* removeFilter behavior for props
* matches_member support for custom props
* filter_suggestions for prop keys should account for prop filter
* find over filter
* refactor appliedFilters
* FILTER_TYPES => FILTER_OPERATIONS
* Make add another link not wrap the whole page
* Unique keys
---------
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
* Reorganize how subscriptions/trials are evaluated
* Bugfix: expired trial+no subscriptions should not have access to extra features
* Make self-hosted users always on trial
* Seed secondary user with password
* Format
* Fix docs
* Fix small_test run
* Run the test only on full_build
* More tweaks to small builds
* Allow [Goals] for expired trials with no subscription
* Simplify caches config
The intervals were proven to never change really,
and previous shape made them not very discoverable either.
We'll look into a more declarative setup.
* Aim for more declarative supervision tree setup
* Generalize `Plausible.Cache`
This makes the `Site.Cache` module implement the specifics,
leaving the common bits reusable for upcoming cache processes.
* Generalize `Plausible.Cache` warmers
* Fix typos
* Revert "Remove 0-sized queue"
This reverts commit 6d229b3f70.
* Revert "Temporarily remove notification cron worker (#3732)"
This reverts commit 447e3929cb.
* Revert "Temporarily disable accept_traffic_until actions (#3730)"
This reverts commit 58afe3376f.
* Send accept_traffic_until notifications at 8 UTC rather than 5
* Add option for dry-run
* Expose dry_run in a dedicated function
* add tests for filtering by goal in timeseries and aggregate
* refactor filter parsing
* stop returning custom props in event:goal breakdown
* test breaking down wildcard pageview goals
* extract filter utils
* parse more goal filter options
* add passing tests for new filter types
* do not allow querying session metrics with a goal filter
* remove unused page_match property
* test that non-configured goals are not returned in breakdown
* enforce filtered goals configured
* update changelog
* Allow simple filtering by revenue goals
This does not mean that revenue metrics are supported. If a revenue goal
is filtered by, we treat it like a simple custom event goal in the API.
* use List.wrap
* Make subsequent query boundaries idempotent
This should prevent time-sensitive race conditions
on subsequent `Plausible.Stats.*` calls for the same query.
Not sure if there are more places where now() is
dynamic relative to query life cycle, but it's a start.
* Format
* Update lib/plausible/stats/query.ex
Co-authored-by: Wojtek Mach <wojtekmach@users.noreply.github.com>
* Match on `Query.now` in a test
---------
Co-authored-by: Wojtek Mach <wojtekmach@users.noreply.github.com>
* Move imported tables schemas to separate modules outside Google ns
* Move buffer for imports to Imported ns
* fix schema newlines
* Extract UA import processing and persistence
* Decouple analytics worker implementation from UA
* Rename env variable for import buffer size
* Preserve old import queue until release
* Allow `matches` operator to work in BE for custom props
Note: No FE support yet, needs further testing
* feat: allow choosing `contains` for property filters in the UI
* no autocomplete on prop values if `contains` for consistency
* CHANGELOG.md
* Fix: Handle (none) property in property breakdowns when using matching
When matching we should always exclude (none)
* Remove allowance_required field from grace_period
Since we are now preventing customers from subscribing to a plan that
does not accommodate their pageview usage, there is no need for an extra
check on removing the grace period after a successful upgrade.
This extra check is the reason why the automatic unlocks have recently
failed in several cases.
* refactor outgrown subscription notices
* make a test actually test the described functionality
* Apply greater pageview allowance margin only for trial upgrades
...in order to prevent cancelled or paused subscriptions from subscribing
to plans that would still leave their account locked.
* Mark the entire ChoosePlanTest module full build only
* remove account locking guide
This is irrelevant for self-hosters, and the internal knowledge base is
a better place for this document. Moved it there.
* refactor Keyword get clause
* add a pattern matching assertion in code
* Fix a bug
This commit fixes a bug where the timeseries weekly interval query for a
month returned 0 visitors for the first week.
This happened because the start of the month defining the first week was
missing the UTC -> site.timezone conversion.
* Update test/plausible_web/controllers/api/stats_controller/main_graph_test.exs
Remove dot from test description
Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>
---------
Co-authored-by: Karl-Aksel Puulmann <macobo@users.noreply.github.com>
* Make modal for goal settings trigger without BE roundtrip
* Turn goal form into a live component and extract modal into a wrapper
* Further extract modal component and handle reset action
* Make ComboBox selection callback more flexible
* Add rudimentary loading state to dialog
* Make form unaware of being put inside a modal
* Make modal a live component and completely reset contents on open server-side
* Try to avoid race condition
* Fix race condition
* Remove unnecessary conditional on socket assigns
* Add typespecs and fix formatting
* Make goals form high latency friendly
* Fix tests to account for goal settings form becoming live component
* Fix goal settings form live component declaration
* Add documentation for modal
* Fix small build test
* Fix typo
Co-authored-by: hq1 <hq@mtod.org>
* Revert no longer necessary test changes from 46f65d9
* Fix and clean up modal styling
* Keep focus on dialog when open and show only spinner on backdrop when loading
* Adjust corners and shadows and implement open/close transitions
* Lock body scroll when modal is open
* Make modal top-aligned again to avoid jumping around on variable content height
---------
Co-authored-by: hq1 <hq@mtod.org>
* Reapply "Sentry context in live views (#3672)"
This reverts commit 5449fead160064b8a0081c458cc5dcd34399eb0b.
* Make sure `:selection_made` is handled in `GoalSettings.Form`
That was a bit unepexcted.. normally `handle_info` is injected
by the LiveView use macro and it discards any message gracefully.
After switching to `use PlausibleWeb, :live_view` we're also
using `PlausibleWeb.Live.Flash` that happens to inject its own receive
clause for closing the flash. Which then renders the original,
overridable, `handle_info` catch-all obsolete.
* Update LV SentryContext only on connected sockets
(first mount already has the right context coming from Sentry plug)
* Make sure Live.ChoosePlan passes `current_user_id` session key
* Add common LiveView macro to PlausibleWeb
* Keep peer data, URI and UA in /live websocket metadata
* Use new PlausibleWeb macro in existing LiveViews
* Implement adding some basic Sentry context `on_mount`
* Format
* Use macro in Live.FunnelSettings
* Update FunnelSettings.Form
* Fall back to UTC when timezone gap found
* bugfix 1
* bugfix 2
* Apply safe tz conversion to all cases where it's done on a ts other than now
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Display premium features tabs on the dashboard
This commit makes the funnels and props tab on the dashboard visible
even when the site owner's plan does not have access to that feature.
This is to raise awareness of those features, and if the site owner does
not want to see that tab, they can still click "Hide this report" to
hide it.
Previously, when the plan did not support a feature, the feature module
`enabled/1` function returned `false`, regardless of the
`sites.feature_enabled` toggle. This commit creates a new function
called `opted_out/1` to differentiate access from explicitly opting out
a feature.
* Remove unused data-conversions-enabled attribute
* DRY Plausible.Billing.Feature.check_availability/1 function
* Allow opting out features the user doesn't have access to
* add upgrade CTA to FeatureSetupNotice
* fix JS linting errors
* simplify notice.js
* fix behaviour when deleting funnels saved to localstorage
In case some other funnel exists, we will use that as the default
selected one. If not, a feature setup notice will be displayed again.
---------
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
* remove unused code
* add e.name == "pageview" condition to pageview goals
This fixes the weird behavior where filtering by a pageview goal would
also return custom events and vice versa.
* update changelog
* Update accept_traffic_until notifications:
- extend has_stats to 48h
- ensure the user is properly greeted
* Clarify the structure passed to email template function
* restore special props access for growth plans
* format
* refactor if condition
Co-authored-by: hq1 <hq@mtod.org>
* format
---------
Co-authored-by: hq1 <hq@mtod.org>
* Update communication
* Remove an unreachable function (mistyped)
* [migration] Make accept_traffic_until a date
* Fix typo
* Set `accept_traffic_until` when creating a site
* Update sites `accept_traffic_until` on subscription change
* Add a note to yearly cancellation notification
* Rephrase annual e-mail for clarity
* Pass the small build test
* Add email notifications
* Fixup
* Implement `accept_traffic_until` notification worker
* Fixup - no need to test this for small build
* Update moduledoc
* Move moduletag
* s/sent_at/sent_on
* Use WHERE NOT EXISTS instead of LEFT JOIN
* Use upsert when tracking notifications sent
* Store sent marker before actually sending notification
* Prefer to keep `accept_traffic_until` on the user record
This gives us a single source of truth, addresses cases like
ownership transparently, simplifies the code and enables CRM toggles.
The only downside is that there's another join performed in the
Sites.Cache full refresh - in this case, small refreshes are
skipped - but this is fine, since the traffic will be let in
anyway.
* Expose `accepted_traffic_until` in the CRM
* Update lib/plausible/auth/user.ex
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Preload owner in CRM
* Use the offset parameter in trial over e-mail contents
* Format
* Harden cache test
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* use more convenient testing functions
* do not display + sign with 0% change in emails
* Rename module/file/function names
before, `weekly_report` was also used for monthly reports and that was a
bit confusing to read in code.
* Refactor send_email_report.ex
This commit improves readability by refactoring the code into smaller
functions and reducing the number of arguments given to functions.
But more importantly, it stops making duplicate stats queries for every
email recipient by moving the queries out of the for loop.
* Refactor: move querying logic out of the worker module
and merge all stats information under a single `stats` assign.
* Move limit enforcement to accepting site ownerhsip transfer
* enforce pageview limit on ownership transfer accept
* Refactor plan limit check logic
* Extract `ensure_can_take_ownership` to `Invitations` context and refactor
* Improve styling of exceeded limits notice in invitation dialog and disable button
* styling improvements to notice
* make transfer_ownership return transfer to self error
* do not allow transferring to user without active subscription WIP
* Add missing typespec and improve existing ones
* Fix formatting
* Explicitly label direct match on function argument for clarity
* Slightly refactor `CreateInvitation.bulk_transfer_ownership_direct`
* Exclude quota enforcement tests from small build test suite
* Remove unused return type from `invite_error()` union type
* Do not block plan upgrade when there's pending ownership transfer
* Don't block and only warn about missing features on transfer
* Remove `x-init` attribute used for debugging
* Add tests for `Quota.monthly_pageview_usage/2`
* Test and improve site admin ownership transfer actions
* Extend tests for `AcceptInvitation.transfer_ownership`
* Test transfer ownership controller level accept action error cases
* Test choosing plan by user without sites but with a pending ownership transfer
* Test invitation x-data in sites LV
* Remove sitelocker trigger in invitation acceptance code and simplify logic
* Add Quota test for `user.allow_next_upgrade_override` being set
* ignore pageview limit only when subscribing to plan
* Use sandbox Paddle instance for staging
* Use sandbox paddle key for staging and dev
---------
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
* move format_price to Plausible.Billing
* move PlausibleWeb.Components.Billing file to subfolder
* extract new Notice module
* rename test file and module name
* move growth_grandfathered notice to notice.ex
* extract a PlanBenefits module
* extract PlanBox component
* extract PageviewSlider component
* fix plan benefits text color
* Remove business tier feature flag
This commit removes all code branches related to the business tier
feature flag, as we're not flipping this flag off anymore. It also
removes unused routes, e.g. /billing/change-plan and /billing/upgrade
* remove unused billing templates
* refactor with clause to case instead
* assert on the url in email tests
---------
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
* remove unused functionality
We can now safely delete all the logic around users who are on trial and
signed up *before* the business tiers release, since that's no longer
possible
* add monthly_pageview_limit fn clause that takes a user
* make Quota.site_limit return enterprise site limit
...not `:unlimited`, as we still need to display it
in the account settings.
* make `team_member_limit/1` return :unlimited on small_build
* improve team_member_usage/1 function doc
* stop displaying unlimited symbol in usage section
These unlimited limits include:
* `monthly_pageview_limit` for trials
* `team_member_limit` for old enterprise plans
* `site limit` for old accounts (before 2021-05-05)
* format
* small refactor case clause + move mod vars
* review suggestions
* prevent invited users without sites from subscribing
* always setup a site in choose_plan_test context
* change trial_expiry_date=nil condition to no sites owned
* Add data retention field to plans
* Display data retention as benefit when choosing plan
* Split fields in two module attributes
* Remove extra whitespace
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Improve subscription status checking code
This commit improves the subscription status checking in code, and
creates convinience functions to work with it, including nil-checking
and in?/2 function.
* Change in?/2 to macro
* Update over_limit.html.eex
* Update dashboard_locked.html.eex
* Update dashboard_locked.html.eex
* Update over_limit.html.eex
* Update dashboard_locked.html.eex
* fix tests
* stop querying owned_site_ids three times
... when querying for billing cycles. Adds an optional `owned_site_ids`
argument to the `usage_cycle` function.
* add penultimate billing cycle info to emails
This commit also refactors some code and adds unit tests to email templates
* use delimit_integer instead of large_number_format
... to display usage with exact numbers such as 1,099,999 instead of 1M
* add penultimate cycle date ranges and linebreaks
---------
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
* End polymorphic response in goals create
Being part of the v3 spec, this isn't well
(or at all) supported by OpenAPI generators.
Always respond with `Goal.ListResponse`
* Implement `PUT /custom_props`
* Implement bulk `DELETE /goals`
* Expose API for (bulk-)disabling custom props
* Add controller typespecs
* Delegate list wrapping to `Plausible.API.*`
* Add 2FA actions to `AuthController`
* Hook up new `AuthController` actions to router
* Add `qr_code` to project dependencies
* Implement generic `qr_code` component rendering SVG QR code from text
* Implement enabled and disabled 2FA setting state in user settings view
* Implement view for initiating 2FA setup
* Implement view for verifying 2FA setup
* Implement view for rendering generated 2FA recovery codes
* Implement view for verifying 2FA code
* Implement view for verifying 2FA recovery code
* Improve `input_with_clipboard` component
* Improve view for initiating 2FA setup
* Improve verify 2FA setup view
* Implement `verify_2fa_input` component
* Improve view for verifying 2FA setup
* Improve view rendering generated 2FA recovery codes
* Use `verify_2fa_input` component in verify 2FA view
* Do not render PA contact on self-hosted instances
* Improve flash message phrasing on generated recovery codes
* Add byline with a warning to disable 2FA modal
* Extract modal to component and move 2FA components to dedicated module
* First pass on loading state for "generate new codes"
* Adjust modal button logic
* Fix button in verify_2fa_input component
* Use button component in activate view
* Implement wait states for recovery code related actions properly
* Apply rate limiting to 2FA verification
* Log failed 2FA code input attempts
* Add ability to trust device and skip 2FA for 30 days
* Improve styling in dark mode
* Fix waiting state under Chrome and Safari
* Delete trust cookie when disabling 2FA
* Put 2FA behind a feature flag
* Extract 2FA cookie deletion
* ff fixup
* Improve session management during 2FA login
* Extract part of 2FA controller logic to a separate module and clean up a bit
* Clear 2FA user session when rate limit hit
* Add id to form in verify 2FA setup view
* Add controller tests for 2FA actions and login action
* Update CHANGELOG.md
* Use `full_build?()` instead of `@is_selfhost` removed after rebase
* Update `Auth.TOTP` moduledoc
* Add TOTP token management and make `TOTP.enable` more test-friendly
* Use TOTP token for device trust feature
* Use zero-deps `eqrcode` instead of deps-heavy `qr_code`
* Improve flash messages copy
Co-authored-by: hq1 <hq@mtod.org>
* Make one more copy improvement
Co-authored-by: hq1 <hq@mtod.org>
* Fix copy in remaining spots
* Change redirect after login to accept URLs from #3560 (h/t @aerosol)
* Add tests checking handling login_dest on login and 2FA verification
* Fix regression in email activation form submit button behavior
* Rename `PlausibleWeb.TwoFactor` -> `PlausibleWeb.TwoFactor.Session`
* Move `qr_code` component under `Components.TwoFactor`
* Set domain and secure options for new cookies
---------
Co-authored-by: hq1 <hq@mtod.org>
* Avoid redirect in site settings
* Fix unicode in SiteController existing tests
* Fix various tests
* Add CHANGELOG
* Make sure test site is example.com
* Use Route helpers in site_controller
* Fix UTF redirect in change domain submit action
* Fix UTF site domain in reset stats action
* refactor asking for the monthly pageview usage
* add tests for usage and limits section in account settings
* display pageview usage per billing cycle for active subscribers
* disable cycle tabs if no usage
* make current billing cycle whole
...instead of capping it at today's date
* run queries for different cycles concurrently
* fix linebreak bug
* add calculate usage action into CRM
* change some names of assigns
* block subscribing to a plan by pageview usage
Depending on whether the customer has already subscribed or not, checking
their pageview usage is different:
* If they're not subscribed yet, we allow them to subscribe to a plan If
it their last 30 days usage does not exceed the plan pageview limit by
more than 15% (30% for when subscribing to a 10k plan)
* For existing subscribers, we'll use the exact same mechanism that we're
using for locking sites - the last two billing cycles usage. If both
cycles exceed the plan limit by more than 10% - we don't allow them to
subscribe to the plan
* apply credo suggestion
* prevent highlight bar overflow
* move disabled classes to button element
* optimize for darkmode
* unify link and text styling on the same horizontal line
'Upgrade' & 'Update billing details' links + billing interval text were
positioned on the same line. The font size was similar, but not the same
* improve exceeded_limits function readability
* Refactor some tests and remove code duplication
* override allow upgrade when limits exceeded
In cases where limits are exceeded, we can set the boolean flag
`allow_next_upgrade_override` to `true` in the CRM. This will allow
the user to upgrade to any plan they want. After they've upgraded or
changed their plan - the flag will automatically reset to `false`.
* only apply upgrade override for exceeded pageview limit
* fix tests on the CI
* make current_cycle usage always displayed by default
* make pageview allowance margin more clear
* add comment
* Disable super-admin checks on small build
* Mute a test writing to stdout
* Move sampling outside of small build
* Convert waiting_first_pageview to heex and stop relying on env vars
* Set site limit unlimited on small build
* Stop relying on app env to get trial expiry
* Remove custom domains - including migration
* Remove is_selfhosted from layout view
* Quota fixup
* Stop relying on app env for self hosted registration
* Stop relying on app env for pass reset success
* Apply on_trial? check only on full build
* Update templates relying on app env
* Adjusts auth controller tests for small build
* Trial fixup
* Fixup
* Stop relying on app env
* Rest of the fsckn owl
* Update typespecs
* Fix dialyzer warning
* Remove unused module
* Credo + format
* GeoIP is not, for full build
* Use `small_build?()` where applicable
* Implement bypassing FirstLaunchPlug without insertions
* Get Marko's patch de58a18a85
* Test is-dbip=false presence
* Fix typespec
* Remove future hardcodes
* Handle `nil` from `Plausible.Geo.database_type()`
* Remove XXX marker
* Use one typespec for two clauses
* Introduce `MIX_ENV=small_dev`
* Revert "Use one typespec for two clauses"
This reverts commit 8d8cd21764.
* Modify API of `Auth.TOTP` to work with simplified flow
* Send email when 2FA is enabled and disabled
* Add tests for `initiated?/1`
* Add tests for email submission and improve recipient address composition
* Fix email tags
* Rename email templates to HEEx
* Fix formatting in email templates
This commit moves the dump_plans Mix task to the Plausible.Release
module so it can run in production.
Usage: `./bin/plausible eval Plausible.Release.dump_plans`
* Update applications
* Clone community config
* Move modules to experimental dir
* Update runtime config
* Apply first set of compile-time conditionals
* Move funnel schemas to experimental
* Make funnel schema-less build compile
* Use experimental/lib for elixir code
* Move JS funnels to experimental
* Clean up conditional rendering
* Tidy up the pipeline
* Make two builds pass tests without warnings
* Reuse existing dotenvs
* Do a bunch of renames
* Clean up naming
* Run secondary CI
* Update router
* Remove RewriteFunnelDupes migration
Tests were disabled already and it was a one-off shot
* Fixup quota mixins
* Add moduledoc
* Change MIX_ENV for seconary test run
* Skip crm on small
* !fixup
* Exclude flags pipeline
* Update lib/plausible_web/controllers/stats_controller.ex
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Bugfix: allow ownership transfers when premium features enabled but not used
Fields like `props_enabled` and `funnels_enabled` are true by default,
and these fields do not indicate whether the user/site is actually using
these features or not.
* allow site transfers if they will be at limit after transfer
* small refactor
* refactor asserting plan generation in plans_test.exs
* stop grandfathering old expired trials
For users who registered before the business tiers release, we want to
offer a chance to subscribe to a grandfathered plan. However, if they
let their trial expire and don't subscribe in the next 10 days, they'll
lose that opportunity.
* stop grandfathering expired subscriptions
* remove default title and icon from Generic.notice
* fix bug with dismissable notice
classList is null when dismissable_id is not given
* alias Plausible.Auth.User
* Refactor Generic.notice component
Make it easy to apply different colors
* move subscription_cancelled_notice across the app
And remove from user settings > subscription box. Also, include a note
about losing grandfathered status when letting the subscription expire.
* allow full width in Generic.notice
* use Generic.notice for subscription_past_due_notice
* use Generic.notice for subscription_paused_notice
* prevent two notices clashing into each other with gap-y-2
* define attrs for phx components
* optimize for light mode
* make subscription cancelled notice dismissable
but if it's dismiss, show it in the place where it was before in the
account settings > subscription box
* make function private
* replace function doc with regular comment to avoid compile warning
* use array for classnames
Co-authored-by: Vinicius Brasil <vini@hey.com>
* fix typos in function doc
---------
Co-authored-by: Vinicius Brasil <vini@hey.com>
* use windows in 'time on page' query
* add imported time on page back
* join imported_pages instead of extra query
* don't forget aggregated time on page!
* eh?
* no need for separate window clause
* use dynamic_filter_condition in aggregate_time_on_page
* it's avg, not total
* fix dynamic_filter_condition call
* tests pass
* fewer changes
* fewer changes + feature flag
* base quickfix
* fewer changes
* fewer changes?
* fewer changes!
* replace coalesce with if(empty(),etc)
* make window_aggregate_time_on_page return same result as neighbor_
* breakdown as well
* add failing test
* more info
* more info
* format example
* add failing aggregate.ex test
* fix breakdown.ex windowing time_on_page calculation
* fix aggregate.ex windowing time_on_page calculation
* change upgrade CTA notice message for enterprise and business plans
* add dismissable option to Generic.notice
* more general notice about losing premium features in X days
* save notice dismissed per user
... more than one Plausible account can use the same device, so we should
scope the fact that the notice has been dismissed by user id.
* fix bug applying classes to Generic.notice
* apply shadow to the new notice on light mode
* use Heroicons.x_mark instead of raw SVG
* use Enum.filter instead of list comprehension
* Revert "Remove site pins for now"
This reverts commit 5eccf4eaf6.
* Implement basic site pin schema level logic within user specific preferences
* Add vertical ellipsis menu markup
* Implement basic changesets for user preferences
* Implement pin toggling
* Try to fix pin sorting
* Implement pin toggling in LV
* Adjust moduledocs for new schema(s)
* Remove unnecessary `distinct` from query
* Use `button` for pin/unpin action
* Generalize preference setting
* Rename schema and fields for clarity
* Rename `list_type` -> `entry_type`
* Safeguard setting options
* Test `set_option/4` and `toggle_pin/2`
* Add test for listing pinned sites via `Sites.list`
* Disallow pinning sites outside page explicitly
* Test pinning in LV
* Test conditional rendering of site settings in /sites
* Remove unnecessary TODO comment
* Safeguard `Sites.set_option/4` against invalid user/site combo
* Handle pinned sites in dashboard site picker
* Clear flashes upon (un)pinning sites
* Update CHANGELOG
* Prevent blinking of hamburger menu items on first paint
* Highlight hamburger handle on hover in /sites
* Start showing hotkeys in site picker again
* Sort pinned sites in the order they were pinned
* Update sites list order immediately after pin/unpin toggle
* Refactor and split `Sites.list/3`, extracting `Sites.list_with_invitations/3`
* Cap number of pinned sites at 9 per user
* First pass on visual indication of site cards (dis)appearing
* Apply ellipsis gradient+shadow on card hover
* Fix responsive padding of site cards
* Sort by invitations first, pinned sites second and then the rest
* Revert "Apply ellipsis gradient+shadow on card hover"
This reverts commit 0608796612639030ccbb12df639709f78edc1434.
* Apply more subtle hover effect on the ellipsis menu
* Make error and success flash LV boxes use separate component containers
* Promote `pinned_at` in table migration to a column
* Switch logic to using `pinned_at` as a standard schema field
* Refactor `Sites.list*` getting rid of subquery (h/t @ukutaht)
* Remove migration which is already merged upstream
---------
Co-authored-by: Adam Rutkowski <hq@mtod.org>
* fix underlined blank space in upgrade link
* Add :if clause to site limit notice
* Change test description
* Move growth? and trial? conditionals to CTA function
* Improve wording in billing notices
---------
Co-authored-by: Vinicius Brasil <vini@hey.com>
* Fix private preview notice typo
* fix link to choose-plan instead of upgrade
* do not display premium feature notice if legacy feature access
* register trial message
* change enterprise contact us copy
* fix test
---------
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
* price formatting
* fix space underlined
* add a redirect from :upgrade to :choose_plan
* Add premium feature notice to revenue goals
* add the number of API request to plan benefits
* be more explicit about Stats API requests
* start linking to the new upgrade page if FF enabled
* add redirect to :upgrade_to_enterprise_plan from :choose_plan
This commit intends to keep the existing behaviour of redirecting users
directly to the enterprise upgrade page when they click on an upgrade
link in an email and have an enterprise plan configured.
Hence, we can also simplify the /settings template and only link to
'/billing/choose-plan', even for enterprise plans.
* deprecate /billing/change-plan route based on FF
* Move new site template to HEEX
* Move new team member invitation template to HEEX
* Standardize exceeded limits error message
* Fix failing tests
* Limit Stats API access to unlimited trials
---------
Co-authored-by: Vinicius Brasil <vini@hey.com>
* Change limits for trials
* Keep legacy trial limits for users that registered before the business tier
* Change private preview notice for release
* Run formatter
* Add countdown to private preview notice
* Add button component
* Use new button in settings screen
* Use button component in registration screens
* Use new button component for Billing.upgrade_link
* Separate .button and .button_link
* Add attr definiton for disabled
* Fix funnels test
* allow using Stats API and Props for free_10k subscriptions
* return v3 plans for legacy trials
* do not display grandfathering notice for legacy trials
* set a more accurate BT release date
* fix bug on dev env
Allow the `find/1` function to find sandbox plans
* add error handling and tests for change_plan_preview
* fix feature warning bug
* fix credo warnings
* fix tests
* set BT release date further into the future
* rename function and some vars
* bugfix with limit exceeding
* fix test
* Show loading state for mini plots
* Dark mode
* Don't render no change in green
* Fix loading placeholders for mobile/desktop view
* Fixup test
* Make loading state background color for light theme one step brighter
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* use a different article in the email copies
... for recommending a plan, since the user can choose between Growth
and Business.
* small refactoring improvement
Rename `Plans.available_plans_with_prices` to `Plans.available_plans_for`,
taking an optional `with_prices` argument.
* highlight recommended tier for trial users on the ugprade page
* review suggestion
* Implement complete basics of LV sites
* Reimplement everything in LV except pagination
* Implement basic search capability
* PoC: plot visitors on sites index
* Add rudimentary clipped gradient in minicharts
* Fix clipping gradient, define once
* Format
* Add moduledoc to visitors component
* Move paginator helpers to the top core namespace
* Fix typespec of `Plausible.Sites.list`
* Split sites component into subcomponents
* Add function to uniformly calculate 24h intervals
and visitor totals across multiple sites.
* Integrate batch 24h interval query with plots on sites view
* Don't confuse heex compiler with alpine @ shorthands
* Make linear gradient svg definition truly invisible
* Implement basic pagination
* Extract `site_stats` from site and invitation cards
* Improve pagination
* Tweak css
* Improve filtering on pagination and make WSS fail graceful
* Test `last_24h_visitors_hourly_intervals/2`
* Replace /sites with LV implementation
* Add debounce to search filter
* Fix typespecs
* Fix styling
* Fix mini graph scaling factor calculation
* Fix search consuming itself
* Minimal tweaks to the plots
* Fixup
* Remove magic numbers from the plot
* Create `site_pins` table
* Add `SitePin` schema
* Implement listing invitations, sites and pins in a single query
* Add FIXME note
* Remove site pins for now
* Add tests for `Plausible.Sites.list/3`
* Add a couple more tests to sites dead view
* Remove unnecessary FIXME
* Add LV tests for Sites
* Calculate and display 24h visitors change
* Render the change in bold
* Add clarfying comment on virtual field in `Site` schema
* Remove unnecessary function from Invitations API
* Remove unused list opt from type definition in `Sites`
* Improve joins in list query slightly
* Add comment on manually computing sites list total
* Start searching from a singly character in domain field
* Add typespec to `last_24h_visitors_hourly_intervals`
* Extend moduledoc in visitors component
* Simplify loading sites in LV
* Simplify assigns in LV
* Add missing group for shadow under site card
* Make invitation modal render
* Make HTML in sites LV semantically correct
* Remove autofocus and focus search on `/`
* Remove shadow from search input
* Make search cancel on escape
* Fix tests relying on outdated HTML structure
* Make visitor chart color scheme consistent with dashboard chart
* Update styling of trend labels
* Fix empty state and improve search blur/focus handling
* Use live navigation for pagination
* Implement spinner on load from search
* Remove unused `Plausible.Stats.Clickhouse.last_24h_visitors/1`
* Calculate uniques correctly across hour boundaries
* Swap inlined svg for Heroicons component in invitation modal
* Add order by to base query in 24h hourly intervals
* Revert "Add order by to base query in 24h hourly intervals"
This reverts commit a6be5e3026.
* Query clickhouse 24h visitors only on second mount
* Remove redundant sign from percentage change when negative
* Switch to offset-based pagination
- offset seems easier to deal with for when actions on
paginated list will be performed such as site pinning;
tracking cursor data makes some entries disappear in
edge cases. The data set is still fairly small and
static, even for large customers.
- we're removing Phoenix.Pagination as it doesn't really
fir any use case, and it was only used to limit the number
of sites in the site picker
- site picker is now limited to 9 sites (future: pinned
sites will be prioritized there)
- no need to re-query for total count any more
- BTW, the old /sites template was removed
* Refine the plot queries; Tests pass snapshot
* Add PromEx plugin for LiveView
* Fix tiny plot cut-off at the top
---------
Co-authored-by: Adam Rutkowski <hq@mtod.org>
* Warn user about Stats API when downgrading
* Update test/plausible/billing/quota_test.exs
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
---------
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
* fix the styling of the red text notice under checkout link
* avoid some code repetition
* simplify rendering the change_plan_link
* refactor disabling checkout link and showing disabled message
* disable change plan and upgrade link when exceeding pageview limit
* disable checkout when exceeding team member limit
* disable checkout when site limit exceeded
* extract checkout related code in a separate function
* stick to a single order of features
* losing features warning
* fix back link from change-plan-preview
* create Quota.exceeded_limits function
* restrict subscribing with exceeded limits on the API level too
* use with instead of case
Co-authored-by: Vini Brasil <vini@hey.com>
* use :map type instead of :any for user
Co-authored-by: Vini Brasil <vini@hey.com>
* create Quota.usage function
---------
Co-authored-by: Vini Brasil <vini@hey.com>
* replace unlisted plans with legacy plans
...and add a legacy plan that has an existing subscriber in Paddle. All
legacy plans are considered generation 1 - meaning that when a user on
one of these plans and they'll go to the upgrade page, then the listed
plans will be v1 for Growth and v3 for Business.
* remove redundant plans_sandbox function
* remove the unused 'scope' argument from Plans.find
* remove unused plan
* add basic test coverage for legacy plans
* add another plan with an existing active subscriber (fix another bug)
* Bump deps
* Bump stack
* Fix deprecation warnings
* Fix VCR cassettes mismatch due to OTP-18414
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Format & fix flaky tests
* Handle raw IPv4 hostnames; test public suffix TLD
* Configure locus db cache_dir
So that maxmind unavailability doesn't affect
application startup. PERSISTENT_CACHE_DIR env var is used
to point locus at the GeoIP DB file.
* WIP: Remove ExVCR
* Fix test env config
* Fixup exvcr
* Remove exvcr from deps
* Add convert script
* Remove exvcr cassettes
* Remove convert script
* Rename test
* Update moduledoc
* Update dockerfile
* Bump CI cache
* Tag more slow tests, why not?
* Use charlist for locus cache option
* Pin nodejs
* Merge google tests, make them async
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* change team member limits for new v4 plans
* duplicate business plans with unlimited team members
We need to do this because we want grandfathered users to have unlimited
team members on business plans as well. Otherwise we'd have to build
overrides on the subscription level when checking the limit.
* refactor generating plan structs
* move Plan module into a separate file
* remove not needed conditions
* add generation field to plans
* sync the sanbox plan limits and features with plan generations
* implement displaying plan benefits
* add grandfathering notice
* plug in the real v3 business plan IDs
* optimize N/A text color for darkmode
* use String.to_existing_atom instead
Co-authored-by: Vini Brasil <vini@hey.com>
* Remove the unnecessary part of a comment
Co-authored-by: Vini Brasil <vini@hey.com>
* make the Plan.new function simpler
* use exlamation marks
---------
Co-authored-by: Vini Brasil <vini@hey.com>
* Migration: track last seen usage for Plugins API Tokens
* Track and interpret Token.last_seen_at
* Display last used
* Order tokens by inserted date, rather than UUID :clown:
* s/Last seen/Last used in the UI
* Test for "Last used" column presence
* Fix table layout for very long descriptions
* Update lib/plausible/plugins/api/tokens.ex
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Update lib/plausible/plugins/api/token.ex
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Update test/plausible/plugins/api/token_test.exs
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* s/last_seen_at/last_used_at
* Update lib/plausible_web/live/plugins/api/settings.ex
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* fixup
* Document reasoning behind 5m windows
* s/last_seen/last_used
* Mute credo
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* fix text color in dark mode
* rename a function
* use aliases in quota.ex
* rename a function
* make Goals a similar feature to others but with a free option
* rename a function
* mix format
* Read feature status from Billing.Feature instead of %Site{}
This commit changes data attributes passed to React. Previously the
controller read feature statuses directly from the %Site{} schema. The
Billing.Feature context is aware of the user plan and the features
available.
* Limit funnels internal API based on site owner plan
* Limit props internal API based on site owner plan
* Use site factory in QueryTest
* Limit custom property filter based on site owner plan
* Limit revenue goals queries based on site owner plan
* Update styling for Choose Plan page
* Make it look good on mobile
* Update tests
* Remove unnecessary assign
* Optimize for dark mode
* Change order or interval picker and slider on mobile
* Format
---------
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
* Add SSO link with signed JWT token
* Falls back to Nolt URL without SSO if token cannot be generated
* Add profile image (gravatar) to Nolt SSO link
* Improve navbar dropdown
* Add 'contact support' link to nav dropdown
* Add CSS rule to prevent horizontal jumps
* Dark mode styling
* Close dropdown when link is clicked
* Clarify links in dropdown
* Clarify CSS comment
* Use Alpine.data() over window
* Rename suggestions_dropdown -> combo-box
* Mix format
* Make logout link look good on dark mode
* Use proxy for gravatar
* Do not use Gravatar proxy in self-hosted
* Changelog
* Add Github Repo link to nav dropdown
* Make dialyzer happy
* Add proxy for Gravatar
* Update assets/css/app.css
Co-authored-by: hq1 <hq@mtod.org>
* Update lib/plausible_web/controllers/avatar_controller.ex
Co-authored-by: hq1 <hq@mtod.org>
* Fix alpine <> Liveview integration
---------
Co-authored-by: hq1 <hq@mtod.org>
* Refactor email verification codes generation to avoid predictability
* Improve `Site.Memberships.any?` slightly
* Update tests
* Fix seeds
* Use `expired?` predicate for checking verification code validity in tests
* Store verification code as string in database to avoid unnecessary int casting
* Include ApiKey functions in Auth context
* Make feature notice work without %Site{}
Previously the extra feature notice required a %Site{} in order to check
the owner plan. However, not every feature is scoped by site, for
example the Stats API. For features like this, a %User{} is required,
and not a %Site{}.
This commit replaces the `:site` param with `:billable_user`, which is
common to both site and user-scoped features.
* Add stats_api to the list of extra features
* Limit API Key creation based on user plan
This pull request implements limits to funnels, revenue goals and custom props based on the site owner plan. It extends the current "premium feature" notice to account for the new plans, trials and the on-going private preview. Stats API is not in the context of this pull request, but will be implemented likewise.
* Implement PoC for email reverification flow on update
* Improve user settings form and email change validation
* Expose `previous_email` in Kaffy CRM
* Improve plugs setup and remove dead action from AuthController
* Fix seeds
* Extract predicate query functions from AuthController
* Add tests
* Update CHANGELOG.md
* Rename `has_any_sites?` to `Memberships.any?` and `has_any_memberships?`
* Improve flash message on cancelling email change
* Cover one more test case for email update
* rename enterprise?/1 function
* change link text to Upgrade when subscription deleted
* extract paddle_button and paddle_script components
* create a new upgrade-to-enterprise-plan page
* extract upgrade_link component
* rename function
* link to enterprise plan upgrade page from settings
...if the user has an enterprise plan configured
* fetch enterprise plan price on the new page
* add change_enterprise_plan functionality on the new page
* render existing change_enterprise_plan_contact_us.html
...when subscribed to latest configured enterprise plan
* rename vars and extract resumable? fn
* remove dead billing route
* small test refactor: extract convenience fn
* add tests for...
...restricting paused and past_due subscription access to the new
enterprise plan page.
1. redirect to /settings from the controller action
2. hiding the change-plan link from the user settings
* implement redirect to /settings
* hide the enterprise upgrade/change-plan link
* add tests for a deleted enterprise subscription
* plug in the new controller action and delete dead code
* optimize for dark mode
* fix compile warning
* credo fix
* display N/A instead of crash when price nil
* change subscription.status type to Ecto.Enum
Also, create a new `Subscription.Status` module that exposes macros to
return the used atom values (prevent typos at compiletime).
* fix bug (@conn not available anymore)
* use Routes.billing_path where applicable
* add a status() type
* silence credo
* refactor suggestion from review
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Remove the __using__ macro from Subscription.Status
... instead be explicit about requires and aliases and also order
the use, import, require, and alias clauses according to
https://github.com/christopheradams/elixir_style_guide#module-attribute-ordering
* drop the virtual Enteprise 'price_per_interval' field
* apply review suggestion to make the code more DRY
* use dot syntax to fetch current user in new controller actions
* fix formatting
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* add a new upgrade page liveview behind a FF
* Create plans_v4.json file
* Add the upgrade page UI template and some basic functionalities
* different content based on subscription plan existing or not
* pageview slider
* monthly/yearly switch
* fix tests
* split into 2 separate functions
* rename variables
* implement volume slider + read default interval/volume from plan
* organize choose-plan.ex better
* remove unused vars from tests
* make monthly_cost and yearly_cost nil by default
The actual prices for all plans are stored in Paddle. We don't need to
keep the duplicates in the JSON files.
* add fetch_prices/1 to PaddleApi
* make v4 business ID's differ from growth ones
* render actual price information from plans
...and make the prices in both growth and business plan boxes change
dynamically when the pageview slider or interval is changed.
* highlight current subscription plan box
* add test describe block for business tier subscription
* connect to live socket only on the specific LV page using focus.html
* only wrap the input slider inside the form
* little readability improvement
* add v4 team_member_limits (after rebase with master)
* extract monthly_quota_box function in user_settings
When the business_tier FF is enabled, this section is different and
links to the new upgrade page.
* document subscription statuses
* change _notice.html.eex to .heex
* extract subscription status notice components
* add failed payment notices to upgrade page
* create class_of_element/2 convenience function for testing
* add cancel_subscription mix task
* implement checkout buttons
* mix format
* get all available plans with prices through plans.ex
* use more suitable function for fetching usage
* avoid double db lookups on mount
* rename variable
* separate functions for getting plan by product_id vs subscription
* separate subscription status docs into context module
* consider cancelled subscriptions
* default volume by usage if no subscription plan
* add enterprise-level volume option to slider
* optimize for darkmode
* UI improvements
* display 2 months free notice for yearly billing
* VAT excluded notice
* note about having a business subscription in user settings
* make the page pop and fit plans on screen on first render
* optimize for mobile and remove background containers
* change default price tag to simply 'N/A'
* fix tests
* Change Paddle.js integration to use JavaScript directly
* rename many variables
* allow users on v1 and v2 plan subscribe to 20M and 50M tiers
* add a test for two months free label
* make it work with a free_10k subscription
* small test improvement and formatting
* change other upgrade link in user settings if FF enabled
* dialyzer
* fix typo
* add test for free_10k user
* silence credo
* mix format
* credo - add moduledoc
* credo - another moduledoc
* handle calls to sentry on the api level
* refactor getting regular subscription plan for LiveView
* post review code style tweaks
* remove unused aliases
* credo - add @moduledoc false to Subscriptions
* crash in cancel_subscription task when Repo update fails
* readability improvements (review suggestions)
* add comment about 'external_resource' module attr
---------
Co-authored-by: Vinicius Brasil <vini@hey.com>
This change addresses two problems:
* controller action crashing missing "token" param - it's handled gracefully
now and will not pollute Sentry anymore with http://sentry.plausible.io/organizations/sentry/issues/4319
* LiveView receives email extracted from token on initial page load instead
of reverifying token on every re-mount (which can happen when somebody
leaves form open for an extended period of time; rare but happens and
needlessly pollutes Sentry as well)
* Make membership creation and role updates more explicit in terms of changesets
* Extract invitation accept flow logic and refactor it slightly
* Improve acceptation logic
* Update moduledoc
* Improve SiteLocker API and add typespecs
* Stop naming function not returning a boolean like a predicate
* Refactor rest of invitation actions and safeguard against rogue requests
* Update code docs slightly
* Extend `Billing.check_needs_update/1` tests
* Parametrize selfhost flag and toggle SiteLocker logic on it
* Add tests for newly extracted services
* Add test case and a fix for locking site on grace period ended
* Make invitation controller tests async as there's no more env patching
* Add test cases for self-invites and fix one bug
* Add and refactor tests for rejecting and removing invitations
* Prevent issuing ownership transfer to existing owner
* Improve name of the test
* Improve `Billing.check_needs_to_upgrade/1` return value
* Improve `Billing.SiteLocker.update_sites_for/1` and its tests
* Fix typos
Co-authored-by: hq1 <hq@mtod.org>
* Make invitation removal and rejection resilient to races
---------
Co-authored-by: hq1 <hq@mtod.org>
* Update depenedencies: OpenAPISpex + cursor based pagination
* Update formatter config
* Add internal server error implementation
* Test errors
* Implement pagination interface
* Implement Plugins API module macros
* Implement Public API base URI
(to be used with path helpers once called from within
forwarded router's scope)
* Implement OpenAPI specs + schemas
* Implement Shared Links context module
* Add pagination and error views
* Add Shared Link view
* Implement Shared Link controller
* Expose SharedLink.t() spec
* Implement separate router for the Plugins API
* Update moduledocs
* Always wrap resource objects with `data`
* Update moduledoc
* Use https://github.com/open-api-spex/open_api_spex/pull/425
due to https://github.com/open-api-spex/open_api_spex/issues/92
* Rely on BASE_URL for swagger-ui server definition
* Fixup goals migration
* Migrate broken goals before deleting dupes
* Remove bypassing test rate limiting for which there's none anyway
* Move the context module under `Plausible.` namespace
* Bring back conn assignment to PluginsAPICase template
* Update test/plausible_web/plugins/api/controllers/shared_links_test.exs
Co-authored-by: Uku Taht <Uku.taht@gmail.com>
* Update renamed aliases
* Seed static token for development purposes
* Delegate Plugins API 500s to a familiar shape
* Simplify with statement
---------
Co-authored-by: Uku Taht <Uku.taht@gmail.com>
* Implement Plugins API Token schema
* Work with domain change grace period
* Do not cast internal data, extend schema with hints
* Implement Plugins API authorization
* Test no authorization header passed
* Preload authorized site
* Fixup typespecs
* Add zxcvbn dependency
* Change password length range requirement from 6-64 to 12-128
* Reimplement register form in LV
* Implement server-side check for password strength
* Add rudimentary strength meter
* Make password input with strength a separate component and improve it
* Fix existing tests to provide strong enough password
* Apply formatting
* Replace existing registration form with new one
* Hide built-in label in `.input` component when none provided
* Crop password to first 32 chars for analysis by zxcvbn
* Add tests for new form components
* Integrate hCaptcha into LV
* Fix existing AuthController tests
* Add tests for Live.RegisterForm
* Hide strength meter when password input is empty
* Randomize client IP in headers during tests to avoid hitting rate limit
* Apply auxilliary formatting fixes to AuthController
* Integrate registration from invitation into LV registration logic
* Fix existing password set and reset forms
* Make `password_length_hint` component more customizable
* Optimize `Auth.User.set_password/2`
* Remove unnecessary attribute from registration form
* Move password set and reset forms to LV
* Add tests for SetPasswordForm LV component
* Add tests for password checks in `Auth.User`
* Document code a bit
* Implement simpler approach to hCaptcha integration
* Update CHANGELOG.md
* Improve consistency of color scheme
* Introduce debounce across all text inputs in registration and password forms
* Fix email input background in register form
* Ensure only single error is rendered for empty password confirmation case
* Remove `/password` form entirely in favor of preferred password reset
* Remove unnecessary `router` option from `live_render` calls
* Make expensive assigns in LV with `assign_new` (h/t @aerosol)
* Accept passwords longer than 32 bytes uniformly as very strong
* Avoid displaying blank error side by side with weak password error
* Make register actions handle errors gracefully
* Render only a single piece of feedback to reduce noise
* Make register and password reset forms pw manager friendly (h/t @cnkk)
* Move registration forms to live routes
* Delete no longer used deadviews
* Adjust registration form in accordance to changes in #3290
* Reintroduce dogfood page path for invitation form from #3290
* Use alternative approach to submitting plausible metrics from LV form
* Rename metrics events and extend tests to account for them
* Enforce goals unique
* Remove unnecessary alias
* Skip tests that can no longer run anymore
To run, make sure the migration from
priv/repo/migrations/20230914071245_goals_unique.exs
is rolled back.
* Use separate transactions for the migration
* Update priv/repo/migrations/20230914071245_goals_unique.exs
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Add `RewriteFunnelDupes` data fix
To rewrite funnels referencing goals whose names
are the same. This enables us to enforce all goals
within the site unique later on.
* Credo
* Update priv/data_migrations/FunnelDupeGoals/sql/list-funnels-with-dupe-goal-ids.sql.eex
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Revert unrelated changes
* Remove dead code
* Update lib/plausible/data_migration/rewrite_funnel_dupes.ex
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Update lib/plausible/data_migration/rewrite_funnel_dupes.ex
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Rewrite goals for which both event_name and page_path exist.
This might've happened due to the old form only hiding form
fields from the viewport, allowing to submit both values in
certain cases when switching tabs. The tabs behaviour has
been changed as of #3293 but no proper constraints existed.
* Add new check constraint to the goals schema
* Use NOT VALID option for adding the CHECK constraint
* Skip wrapping migration in a single transaction
* Add hint to creatable ComboBoxes without suggestions available
* Load external resources once in funnel settings
* Load external resources once in goal settings
* Make Custom Props Settings UI match Goal Settings
* Remove unnecessary goals query
This should be done only once in the live view
* Remove funnels feature flag
* fixup
* Make the modal scrollable
* By default, focus first suggestion for creatables
* Update StaticSearch
So it's capable of casting custom data structures
into weighted items. Missing tests added.
* Add Search + modal to funnel settings
* Add sample props to seeds
* Load all suggestions asynchronously, unless `Mix.env == :test`
* ComboBox: Fix inconsistent suggestions
We require "Create ..." element to be only focused
when there are no suggestions available.
This causes some issues, depending on the state,
the least focusable index might be either 0 ("Create...")
or 1. This patch addresses all the quirks with focus.
* Fix ComboBox max results message
So that AlpineJS doesn't think it's a focusable
option.
* Keep the state up to date when changing props
* Add hint to creatable ComboBoxes without suggestions available
* Load external resources once in funnel settings
* Load external resources once in goal settings
* Make Custom Props Settings UI match Goal Settings
* Remove unnecessary goals query
This should be done only once in the live view
* Remove funnels feature flag
* fixup
* Make the modal scrollable
* By default, focus first suggestion for creatables
* Add sample props to seeds
* Load all suggestions asynchronously, unless `Mix.env == :test`
* ComboBox: Fix inconsistent suggestions
We require "Create ..." element to be only focused
when there are no suggestions available.
This causes some issues, depending on the state,
the least focusable index might be either 0 ("Create...")
or 1. This patch addresses all the quirks with focus.
* Fix ComboBox max results message
So that AlpineJS doesn't think it's a focusable
option.
* Keep the state up to date when changing props
* Fixup site_id
* Fix typo
* fixup
* Add hint to creatable ComboBoxes without suggestions available
* Load external resources once in funnel settings
* Load external resources once in goal settings
* Make Custom Props Settings UI match Goal Settings
* Remove unnecessary goals query
This should be done only once in the live view
* Remove funnels feature flag
* fixup
* Make the modal scrollable
* By default, focus first suggestion for creatables
* Add sample props to seeds
* Load all suggestions asynchronously, unless `Mix.env == :test`
* ComboBox: Fix inconsistent suggestions
We require "Create ..." element to be only focused
when there are no suggestions available.
This causes some issues, depending on the state,
the least focusable index might be either 0 ("Create...")
or 1. This patch addresses all the quirks with focus.
* Fix ComboBox max results message
So that AlpineJS doesn't think it's a focusable
option.
* Keep the state up to date when changing props
* Update seeds with sensible prop names
* Make escape work for closing combobox suggestions
Co-authored-by: Uku Taht <Uku.taht@gmail.com>
* Revert "Make escape work for closing combobox suggestions"
This reverts commit 306866d2a1.
@ukutaht unfortunately this makes it impossible to select
an suggestion.
* Revert "Revert "Make escape work for closing combobox suggestions""
This reverts commit 4844857812.
* Make ESC great again
* Improve readability
---------
Co-authored-by: Uku Taht <Uku.taht@gmail.com>
* Allow admins to initiate ownership transfer from the CRM
* Add stronger assertion for bulk invite action
* Fix compile warning
* Move bulk transfer logic to Sites module
* Replaces unused variables with _
* Add typespec for `bulk_transfer_ownership`
* Extract from keywordlist options instead of matching
* Fix and extend bulk transfer tests
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Filter out empty entries when listing stats for UTM props
* Update test fixtures removing noref entries in UTM CSV stat exports
* Update external API tests to account for lack of noref records for UTM stats
* Filter out entries with empty UTM props from imported GA stats
* Remove unreachable GA utm_source dim clause in imported stats logic
* Move inline functions to module
* Extend invite/4 for ownership transfers
* Verify inviter has sufficient permissions
* Ensure ownership transfers don't count as team member
This commit changes the team member usage query to exclude ownership
transfer invitations. Previously, when an ownership transfer was
pending, the team member usage was incremented.
* Draw attention to payment notice when transferring ownership
* Remove duplicate mail sending from membership_controller
---------
Co-authored-by: Uku Taht <uku.taht@gmail.com>
* Refactor MembershipController.invite_member/2
This commit refactors the controller action used for creating new
invitations. It moves the code to Plausible.Sites.invite/4 and replaces
`ifs` and `cases` with `with`.
* Add team_member_limit to plan definition
* Create usage and limits functions for team members
* Apply team member limit when inviting new users
* Add team members to Usage & Limits section
* Change invite function to receive email address instead of %User{}
* Wrap invite function in a DB transaction
* Remove unnecessary joins from team member usage query
* Replace UNION ALL with UNION to remove duplicates
* Add Heroicons dependency
* Add name_of/1 html helper
Currently with Floki there's no way to query for
`[name=foo[some]]` selector
* Update changelog
* Make goal deletion possible with only goal id
* Remove stale goal controllers
* Improve ComboBox component
- make sure the list options are always of the parent input width
- allow passing a suggestion function instead of a module
* Stale fixup
* Update routes
* Use the new goals route in funnel settings
* Use a function in the funnel combo
* Use function in the props combo
* Remove old goals form
* Implement new goal settings
* Update moduledoc
* Fix revenue switch in dark mode
* Connect live socket on goal settings page
* Fixup
* Use Heroicons.trash icon
* Tweak goals search input
* Remove unused alias
* Fix search/button alignment
* Fix backspace icon alignment
* Delegate :superadmin check to get_for_user/3
I'll do props settings separately, it's work in progress
in a branch on top of this one already. cc @ukutaht
* Rename socket assigns
* Fixup to 5c9f58e
* Fixup
* Render ComboBox suggestions asynchronously
This commit:
- prevents redundant work by checking the socket connection
- allows passing no options to the ComboBox component,
so that when combined with the `async` option, the options
are asynchronously initialized post-render
- allows updating the suggestions asynchronously with the
`async` option set to `true` - helpful in case of DB
queries used for suggestions
* Update tests
* Throttle comboboxes
* Update tests
* Dim the search input
* Use debounce=200 in ComboBox component
* Move creatable option to the top
* Ensure there's always a leading slash for goals
* Test pageview goals with leading / missing
* Make the modal scrollable on small viewports
* Update user_settings template to HEEX
* Move site_limit function to Billing.Quota
* Create Billing.Quota.site_usage function
* Move monthly_pageview_limit function to Billing.Quota
* Create Billing.Quota.monthly_pageview_usage function
* Add "Usage & Limits" section to user settings page
* Apply suggestions from code review
* Inflate sampled results in Stats.Funnel.funnel/3
* Add test for sampling
* Update test/plausible/funnels_test.exs
Co-authored-by: hq1 <hq@mtod.org>
* Ignore unused binding
* Use bang version of hash function
---------
Co-authored-by: hq1 <hq@mtod.org>
* Add new priority email template?
* Allow priority email to be sent with not layout
* Use priority email template for priority transactional emails
* Test for postmark message stream
This commit fixes a bug where the SendTrialNotifications job tried to
suggest a plan for users switching to enterprise plans, resulting in the
exception below:
```
UndefinedFunctionError: function :enterprise.volume/0 is undefined (module :enterprise is not available)
Module "enterprise", in :enterprise.volume/0
File "lib/plausible_web/templates/email/trial_upgrade_email.html.eex", line 5, in PlausibleWeb.EmailView."trial_upgrade_email.html"/1
File "lib/phoenix_view.ex", line 381, in Phoenix.View.render_within/3
File "lib/phoenix_view.ex", line 557, in Phoenix.View.render_to_iodata/3
File "lib/phoenix_view.ex", line 564, in Phoenix.View.render_to_string/3
File "lib/bamboo_phoenix.ex", line 291, in Bamboo.Phoenix.render_text_or_html_email/1
File "lib/workers/send_trial_notifications.ex", line 67, in Plausible.Workers.SendTrialNotifications.send_today_reminder/1
File "lib/workers/send_trial_notifications.ex", line 36, in anonymous fn/2 in Plausible.Workers.SendTrialNotifications.perform/1
```
This pull request introduces a series of improvements to Plausible.Billing.Plans, including:
* Tag the JSON file with the plan version
* Rename the JSON field limit to monthly_pageview_limit
* Move site_limit function to Billing.Plans
* Refactor subscription_interval, allowance and site_limit functions
* Remove unused AnalyzePlans task
This commit introduces a series of improvements on the Plans module including function renaming, documentation and readability. This is the groundwork for billing plans.
There should be no actual changes with this commit, therefore no changes in tests either.
* Update interval dropdown based on custom range selection
* Add `monthsBetweenDates` function
* Reassign interval for custom "period" based on local storage
* Generate intervals by period based on stats_start_date
* Account for "custom" and "all" period intervals dynamically
* Remove `Plausible.Sites.set_allowed_event_props/2` function
This commit removes the `Plausible.Sites.set_allowed_event_props/2`
function in favor of `Plausible.Props.allow/2`.
* Allow internal prop keys in queries
This commit allows internal prop keys by default in queries. I decided
not to include internal keys in the database, because they might change
and we'd need to migrate it again.
* remove redundant 'props.' from behaviours/index.js
---------
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
This commit removes the allowed props list from the data attribute
passed to React. The list was used for checking whether props configured
or not. This commit adds a new `hasProps` attribute, and removes the
list.
* Remove url prop key from suggestions
This commit prevents the `url` prop key from being suggested by the
props settings page. This prop is used internally for file downloads and
outbound link clicks, and doesn't need to be manually allowed.
* Document Props module
* extract add_exit_rate function
* Change internal API metric names for /entry-pages & /exit-pages
* `unique_entrances` -> `visitors`,
* `total_entrances` -> `visits`,
* `unique_exits` -> `visitors`,
* `total_exits` -> `visits`,
This is just a consistency improvement - the `visitors` metric always means
one thing and there's no need to call it different for entry pages internally.
This commit does not change any noticable behavior. The UI labels are kept
the same and the column headers in the CSV export will also remain the same.
* Change internal API metric names for /conversions
* `unique_conversions` -> `visitors`,
* `total_conversions` -> `events`,
* return revenue metrics from /custom-prop-values (backend)
* be more explicit about which metric is plotted with Bar
* validate that ListReport input metrics actually exist in the API response
* display revenue metrics in the dashboard Properties section
* limit the number of columns shown on mobile
* add revenue metrics to Properties > Details
* review suggestions
* define hiddenOnMobile per metric instead of keeping the first 3
* rewrite if-else block
---------
Co-authored-by: Vini Brasil <vini@hey.com>
* extract add_exit_rate function
* Change internal API metric names for /entry-pages & /exit-pages
* `unique_entrances` -> `visitors`,
* `total_entrances` -> `visits`,
* `unique_exits` -> `visitors`,
* `total_exits` -> `visits`,
This is just a consistency improvement - the `visitors` metric always means
one thing and there's no need to call it different for entry pages internally.
This commit does not change any noticable behavior. The UI labels are kept
the same and the column headers in the CSV export will also remain the same.
* Change internal API metric names for /conversions
* `unique_conversions` -> `visitors`,
* `total_conversions` -> `events`,
* make (none) value in custom prop breakdown add +1 to pagination limit
This is needed because the (none) value is always added to the
breakdown_results **after** fetching those from ClickHouse.
* only include (none) values on the first page of results
* fix percentage metric calculation for paginated results
Instead of summing up the number of visitors from the breakdown results
to get the total, we have to make a separate query to `Stats.aggregate`.
Otherwise, the percentages for each results page will wrongly add up to 100%.
Since imported data for aggregated visitors and other properties (such as
browsers, OSs, etc) live in different tables, we have to tweak the tests to
also include the same number of visitors in the `imported_visitors` table.
* add details view for props
* changelog
* exclude imported data from total
Previously, the interval was always overwritten with the default value, which is determined based on the age of the site.
It should not be overwritten when the interval is explicitly passed by the user
* move set_allowed_event_props into context module
* extract parse_csv fn
* add custom_props.csv to CSV export (under FF)
* add test for custom prop export with a prop filter
* add test for goal filtered scenario
* mix format
* fix flaky test
* extract breakdown_custom_prop_values function
* Revert "fix flaky test"
This reverts commit 256b9bb2e7. The flaky
test was already fixed in 86557b1878
* make set_allowed_event_props a bang function
* test that all files are included in the CSV export
This commit makes static search more strict by rejecting matches with a score less than 0.6. Here's an example of suggestion that was matching with a 0.5 score that should not be suggested. This makes the suggestion list smaller and more reasonable.
Co-authored by: Robert Joonas <robertjoonas16@gmail.com>
* Add creatable option to ComboBox
This commit changes the ComboBox component to allow a `creatable`
option. This option enables users to create new options along with
choosing existing options.
* Test ComboBox class parameter
* Use display_value instead of input
* Change scroll block to nearest to prevent glitches
* refactor SourceList to use ListReport
* refactor SourceList into a fn comp
* change referrer-drilldown API response format and remove dead code
* use ListReport in referrer-list
* fix CI
* fix flaky test
* remove IO.inspect
* copy prop_breakdown tests into a separate file
* add a new endpoint for custom props
Duplicate existing goal prop breakdown as a building base for the
new endpoint
* stick to original metric names + CR definitions
We currently use CR with two different definitions, which is inconsistent
and should be changed. This commit just documents the difference for the
time being.
* basic prop breakdown without goal filter
* increase % metric precision to one decimal place
* add some tests without goal filter
* silence credo for TODO comment
* use events metric instead of pageviews
* review feedback
* inline add_cr_a instead
* Move money cast function to Stats.Util
* Add revenue metrics to main graph API
* Plot revenue metrics on the main graph
* Return plain numbers instead of a map
* Fix Credo issues
* Fix canMetricBeGraphed function
* Revert canMetricBeGraphed function changes
* Add noindex,nofollow to dashboard pages
* Implement NoRobots plug
* Enable NoRobots plug in the router
* Fixup internal route
* Fix double slash in the router
* Add special bot treatment to plausible.io live demo page
* Revert aggressive protection with agent detection
* PoC: evaluate funnel on step selection
* Adjust test to actually _select_ steps
* Throttle and test ephemeral funnel evaluation
* Format compact percentages server-side
At this point it's pointless to duplicate this;
all the primitives exist in the API in case the client
wants a different representation.
* Add Funnel react component
assets/js/dashboard/stats/behaviours/funnel.js - restored from:
98a76cbd Remove console.info calls
d94db99d Convert Funnel class component into a functional one
028036ad Review comments
3067a940 Stop doing maths in react
73407cc3 Fix error handling when local storage gets corrupted
e8c6fc52 Format numbers on funnel labels
c815709f Reorganize component responsibility
7a88fe44 Outline basic error handling
94caed7c Chart styling updates
4514608a Add percentages to funnel
d622c32d Add funnel picker
Co-authored-by: Uku Taht <uku.taht@gmail.com>
* Pass funnels list to react via data-funnels
* Implement Funnels react API
lib/plausible_web/controllers/api/stats_controller.ex - restored from:
f36ad234 Adjust to Plausible.Stats interface
9b532273 Test funnel stats controller
028036ad Review comments
bea3725f Remove IO.inspect
7a88fe44 Outline basic error handling
c8ae3eaf Move Funnels to StatsController and use base query
667cf222 Put private functions at the bottom
* Tweak funnel presentation
* Handle errors at the top
* Do not register DataLabels plugin globally
or else all the existing charts are affected
* Calculate drop-off percentage evaluating funnels
* Tweak dark mode + implement nicer tooltips
* Make currently selected funnel bold in the picker
* Count user_ids not session_ids when evaluating funnels
So if a visitor goes:
1. Start session
2. Complete funnel step 1
3. Inactive for 30 minutes
4. Complete funnel step 2
We would not be able to track this funnel completion because of the session timeout.
We like to o measure this as funnel completion even though the session expired in the middle.
cc @ukutaht
* Add extra properties to the funnels API
cc @ukutaht
* Improve tooltips so that step to data is rendered
* Change tooltip number formatting
* Remove debugging remnants
* Quick & dirty mobile view
* Fix mobile view: tweak dark mode & funnel switching
* Ignore DOMException: the operation was aborted
Otherwise this sometimes flashes the space shuttle
screen when navigating quickly via a keyboard.
* Format percentages on the main chart
* Close missing tag 🙈
* Revert "Close missing tag 🙈"
This reverts commit 9c2f970e22fd7e2980503242b414f42ce8bce1d2.
* Use jsx to render funnel tooltip
To get markup validated via lsp mostly...
* Fixup: s/class/className
* Fix className interpolation
* Add a ruler to the tooltip
* Tweak funnel chart style
* Fix font distortion issue on chart/canvas labels
* s/class/className
* Put "Set up funnels" link behind a feature flag
* Refactor internal selection storage
Getting ready for live funnel evaluation
* Don't try to connect LV socket if there's no CRSF token set up
This is perfectly okay for some of the templates/layouts.
* Fix up funnel creation typespecs
Unfortunately we can't define a type with literal string keys,
hence this must suffice.
* Use uniq over count/distinct
* Revert JSX in tooltips
Ref: https://github.com/plausible/analytics/pull/3066#discussion_r1241891155
* Remove the extra query for counting all visitors
cc @ukutaht
* Add premium notice
---------
Co-authored-by: Uku Taht <uku.taht@gmail.com>
This pull request changes the UI for revenue goals, replaces the select box with a toggle, and adds a notice. It also polishes up the goal creation UI in general to have the same font sizes and weights compared to the rest of the page.
This commit adds revenue data to top stats. Average and total are displayed when filtering by a revenue goal (or many if they have the same currency set).
* Update formatter config
* Install LiveView JS integration & hooks
* Temporarily update endpoint/session config
* Optionally allow preloading funnels for goals
* Site controller
* Implement funnel settings
lib/plausible_web/live/funnel_settings/combo_box.ex - restored from:
054de6e2 Fix the tab/blur bug again
20da4c89 Rename InputPicker to ComboBox
lib/plausible_web/live/funnel_settings/form.ex - restored from:
9bedda3b Remove potential FIXME
20da4c89 Rename InputPicker to ComboBox
028036ad Review comments
aea4ebc4 Access Funnel min/max steps via the __using__/1 macro
0dde27fd Remove inspect call
eed588a7 Start testing the funnel editor
0e95228b Extract funnel settings test module
7b16ace5 Leverage aplinejs to deal with the tyranny
8dc6a3e7 wip
cf228630 wip
30a43fd1 wip
89f10ecb wip
950a18d9 Dirty funnel save
298a6a53 wip
7690d50f wip
639c6238 fixup
aa59adeb wip
ff75c00b wip
lib/plausible_web/live/funnel_settings/list.ex - restored from:
4eae122c Fix data-confirm attr interpolation
51f0397d Implement deleting funnels
1f6fe25d Add number of steps to funnels list
298a6a53 wip
ff75c00b wip
test/plausible_web/live/funnel_settings/funnel_settings/combo_box_test.exs - restored from:
20da4c89 Rename InputPicker to ComboBox
test/plausible_web/live/funnel_settings/funnel_settings_test.exs - restored from:
34822ff4 Bootstrap InputPicker tests
lib/plausible_web/live/funnel_settings.ex - restored from:
028036ad Review comments
acd9c4f2 Prepare ephemeral funnel definitions so that users can test funnels
51f0397d Implement deleting funnels
0e95228b Extract funnel settings test module
8dc6a3e7 wip
89f10ecb wip
950a18d9 Dirty funnel save
298a6a53 wip
aa59adeb wip
ff75c00b wip
test/plausible_web/controllers/error_report_controller_test.exs - restored from:
34822ff4 Bootstrap InputPicker tests
test/support/html.ex - restored from:
0a53979d Improve InputPicker tests - include AlpineJS assertions
34822ff4 Bootstrap InputPicker tests
lib/plausible_web/views/layout_view.ex - restored from:
b490403b !ifxup
lib/plausible_web/templates/site/settings_funnels.html.eex - restored from:
51f0397d Implement deleting funnels
ea1315f3 Test funnels list in settings
7b16ace5 Leverage aplinejs to deal with the tyranny
ff75c00b wip
4da25c35 Fixup
lib/plausible_web/templates/layout/app.html.eex - restored from:
ff75c00b wip
* Add funnel settings route
* Warn about funnels deletion when deleting goals
lib/plausible_web/templates/site/settings_goals.html.eex - restored from:
fdd9bcd0 Fixup
f1e6364d Merge remote-tracking branch 'origin/master' into funnels-rebase
9d0b7c6d Fix markup error
4a4ddbdc Optionally preload funnels for goals and stub funnel-goal deletion
ebdc4333 Extend the prompt in case of funnel-goal deletion
639c6238 fixup
aa59adeb wip
* Split new JS LiveView additions
* Put funnels behind a feature flag
* Integrate dashboard feature toggle
* Update signing salt for live view
* Update moduledocs
* Update live reloader config
* Use Phoenix.HTML.Safe for goal names
* Workaround to get flashes working in embedded liveview
* Keep feature toggles idempotent, rename property to setting
We'll still retain the ability to flip bools on a lower level.
* Update moduledocs
* Make live flash disappear after 5s
* Tailwind: purge .heex files too
* Update docs link
* Add live components to tailwind purge config
* Update another flaky test
Ref f0bdf872
cc @vinibrsl
* Fix combobox input length w/ WebKit
* Intoduce generic notice component
* Revert "Fix combobox input length w/ WebKit"
This reverts commit 3c653a6d85d5000167631e10ef45a93c13b41ed1.
* Fix combobox input length on webkit
* Make whole combobox item clickable, not only text
* Fix glitch moving Save button on activation
* Tweak dark mode
* Show funnel form without waiting for funnel name input
* Tweak dark mode
* Include static Phoenix components in tailwind purge
* Tune funnels form into a liveview of its own
This is so that ComboBoxes can publish their selections
and unavailable choices can be propagated to other siblings.
* Push less data over websocket
* Undo Lsp/formatter race condition
* Fixup typespecs
* Bust CI cache
This commit fixes a bug where performing a breakdown by goal with
revenue metrics would return data for both event goals and pageview
goals. However, revenue data is only applicable for event goals as they
hold currency data.
To fix the bug, this commit removes revenue metrics from the pageview
breakdown query.
This commit fixes a bug where timeseries queries - such as the main
graph - would fail with imported data in realtime mode. Realtime mode
`date` field is not actually a date, but minutes from now. This would
cause the imported tables join to fail with this error:
```
(Ch.Error Code: 53. DB::Exception: Can't infer common type for joined
columns: date: Int64 at left, s1.date: Date at right. There is no
supertype for types Int64, Date because some of them are
Date/Date32/DateTime/DateTime64 and some of them are not.
(TYPE_MISMATCH)
```
This commit removes imported data from realtime queries, as it doesn't
make sense to include it. Imported data does not have time precision,
and would only appear in the first day the data was imported anyways.
* Pass cached site struct down the ingestion pipeline
Revenue goals need the cached site struct during ingestion to get the
goals name and currency. This cache lookup is not necessary as
`GateKeeper.check/1`, which is called first in the ingestion pipeline,
could already return the site struct from the cache.
This commit changes `GateKeeper.check/1` to return the site struct
instead of the site ID. Moreover, this commit changes the ingestion
pipeline to avoid calling the sites cache twice.
Related: https://github.com/plausible/analytics/pull/2957#discussion_r1203921549
* Remove revenue_goals unnecessary fallback
* Change duplicate child_id in cache test
* Remove revenue goal condition from cache query
* Remove Plausible.DataCase.reload/1
* copy relevant files from b2ace16540
* make it work and set site.funnels to empty list
* make Behaviours a functional component
* add UI for a setup hint and always display conversions by default
* cherry-pick migration commit
* update site schema with new fields
* backend: implement disable-feature action
* switch between tabs in the behaviors section
Introduces template components to build props and funnels on. Both
only show a setup notice atm, and both are behind feature flags.
* extend API for disabling props and funnels
* render feature setup note directly from the Behaviours component
* fix UI behavior when features are hidden
* update setup notices
* add conversions feature switch to Site Settings > Goals
* mix format
* remove IO.inspect
* change setup notice - use buttons + popup confirmation
* optimize for light mode
* restrict access to setup notices
* some styling improvements
* allow super-admins to enable/disable features
* only show conversions (last 30min) in realtime mode
* use shorter display names for tabs
* optimize for mobile screens
* note about sending custom events
* changelog update + fix CI
* change HTTP verb for the disable-feature action
* change UI label for show/hide goals
* Add revenue fields to ClickHouse events
This commit adds 4 fields to the ClickHouse events_v2 table:
* `revenue_source_amount` and `revenue_source_currency` store revenue in
the original currency sent during ingestion
* `revenue_reporting_amount` and `revenue_reporting_currency` store
revenue in a common currency to perform calculations, and this
currency is defined by the user when setting up the goal
The type of amount fields is `Nullable(Decimal64(3))`. That covers all
fiat currencies and allows us to store huge amounts. Even though
ClickHouse does not suggest using `Nullable`, this is a good use case,
because otherwise additional work would have to be done to
differentiate missing values from real zeroes.
I ran a benchmark with the data pattern we expect in production, where
we have more missing values than real decimals. I created 100 million
records where 90% of decimals are missing. The difference between the
tables in storage is just 0.4Mb.
* Add revenue parameter to Events API
This commit adds support for sending revenue data in ingestion using the
`revenue` parameter - aliased to `$`.
* Add revenue parameter to mix send_pageview
* Add average and total revenue to breakdown queries
* Add revenue goal option to goal creation
This commit adds a currency field to the goals form. Goals that have a
currency set are now revenue goals, and are cached with sites to later
be used during ingestion.
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
* Enable feature flag in tests
---------
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
* Escape domain when constructing favicon URL
A domain may include a slash, and in that case the domain must be
escaped, before it is used as an attribute for the image tag.
* match with 'conn.request_path' instead + test
---------
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
* default to source_query.include_imported before true
This fixes a bug where a Stats API aggregate query was trying
to query current period *without imported data* but previous
period *with imported data*.
* remove __internal_visits at a better time
As soon as we have made the db query, we don't need it anymore
* disallow events metric with imported data
* update changelog
* keep default in a better place
* remove unused option
* update test description
Co-authored-by: Vini Brasil <vini@hey.com>
---------
Co-authored-by: Vini Brasil <vini@hey.com>
This pull request adds constraints on user input values during ingestion. The purpose of this PR is to filter out unrealistic or invalid inputs. More comments about each limit inline.
* add migration
* add not null constraint
* remove not null constraint and default value
* add site schema change
* filter garbage props for prop_key suggestions
* filter garbage props returned by StatsController.conversions/2
* add crm action to set prop allowlist
* use nullable allowed_event_props field instead
* filter garbage props in db query instead
* implement suggestion dealing with string input
* refactor the allowlist condition in db query
* improve test
* Simplify Phoenix error template
* Test tracking script is not included in error pages
* Test tracking script is not rendered in error templates
* Rename error layout and remove unnecessary HTML boilerplate
* Add layout setting to errors rendered without exceptions
* Add skip_plausible_tracking option to more pages
* upgrade phoenix
Co-authored-by: Vini Brasil <vini@hey.com>
* fix a test (flash message)
The flash message in focus.html.eex was not covered by any test. This
commit fixes also fixes that.
* change function name
* remove unnecessary formatter and format
* update CI cache
* fix dialyzer error
---------
Co-authored-by: Vini Brasil <vini@hey.com>
* Reject invalid country codes early
* Validate all date parameters early
* Ensure request JSON parses to a map
* Re-arrange function calls
* Extend malicious filter validation
* add entry page filter condition to bounce rate
* Fix select_merge with dynamic
* fix tests and add one test
* generalize page_filter_condition function
* use dynamic_filter_condition for session filters too
* disable views_per_visit with a page filter
* update changelog
* disable credo for complex function
---------
Co-authored-by: Uku Taht <uku.taht@gmail.com>
* Add PropFilterModal (only UI)
* small variable refactor
* allow selecting prop value filter type
* allow selecting only one prop_key
* allow selecting many prop_values only when prop_key selected
* handle submitting filter
* get applied filters from query + remove option
* change prop filter label format
* support member and not_member filter types for pageview props
* show (none) value in filter suggestions
* refactor zip_results/4 and remove unused code
* fix displaying (none) values in goals section prop breakdown
* remove unnecessary functionality
* fix bug: returning prop names for goal :member filter
* fix bug: submitting regular filter modal with Enter key
* bugfix: disallow opening prop filter modal when feature flag disabled
* mix format
* break selected combobox values into multiple lines
* fix useEffect behavior for focusing on prop_key field
* support submitting prop filter with Enter key
* refactor getFormState in PropFilterModal
* separate fetchPropKey and fetchPropValue functions
* Allow querying props for pageview goals
* Make the internal props API only return a list of props (not map)
* Separate function for fetching all props in Stats API goal breakdown (this returns a map as before)
* ditch state for keeping search bar visible
* group by event_name in db query
This pull request removes the comparisons feature flag, making it visible to all users. It also removes percentages from top stats in default view, leaving those for the comparison view only.
This commit fixes a bug where comparisons would not work with imported data. This is because comparisons copies the source query and modifies the dates only, where it should reevaluate if there is imported data for the new comparison date range.
Closes#2870
Co-authored-by: Adam <hq@mtod.org>
* Pass comparison_query as argument to top stats functions
* Add comparison values and dates to top stats API
* Display comparison dates and values in top stats
* Rename function to renderPercentageComparison
* Create component to do conditional rendering
* DRY date range formatting function
---------
Co-authored-by: Uku Taht <uku.taht@gmail.com>
* Full migration (to be submitted separately)
* Do not remove `Goal.domain` just yet
* Do not make Goal.site not nullable just yet
* Temporarily disable goal creation
* Rephrase error message
* Add down migration
* Escape pipes in goal names
* Apply suggestions from code review
Use ~S sigil for better readability
Co-authored-by: Vini Brasil <vini@hey.com>
---------
Co-authored-by: Vini Brasil <vini@hey.com>
* Migration (PR: https://github.com/plausible/analytics/pull/2802)
* Implement Site.Domain interface allowing change and expiry
* Fixup seeds so they work with V2_MIGRATION_DONE=1
* Update Sites.Cache so it's capable of multi-keyed lookups
* Implement worker handling domain change expiration
* Implement domain change UI
* Implement transition period for public APIs
* Exclude v2 tests in primary test run
* Update lib/plausible_web/controllers/site_controller.ex
Co-authored-by: Vini Brasil <vini@hey.com>
* Update lib/plausible_web/controllers/site_controller.ex
Co-authored-by: Vini Brasil <vini@hey.com>
* Update moduledoc
* Update changelog
* Remove remnant from previous implementation attempt
* !fixup
* !fixup
* Implement domain change via Sites API
cc @ukutaht
* Update CHANGELOG
* Credo
* !fixup commit missing tests
* Allow continuous domain change within the same site
---------
Co-authored-by: Vini Brasil <vini@hey.com>
* Wrap Plausible.Stats.Filters with unit tests
* Parse `member` filter type
* Support for `member` filter in `aggregate_time_on_page`
* Support `not_member` filter type
* Support `matches_member` and `not_matches_member` filters
* Extract util module for React filters
* Implement Combobox from scratch with no libs
* Support multple filter clauses in combobox
* Don't use browser / os in version label
* Show highlighted option in combobox
* WIP
* Fix location filters outside filter modal
* Align open/close behaviour with react-select
* Styling updates for combobox
* Add support for wildcards in Combobox
* Implement keybindings for combobox
* Allow free choice inputs in combobox
* Rename 'Save filter' -> Apply filter
* Remove TODO comment
* Clean up some rebase mistakes
* Rename `allowWildcard` -> `freeChoice`
* Dark mode fixes
* Remove hint from filter modal
* Escape pipe character in filter modal
* Do not allow selecting duplicate options in combobox
* Escape brackets in `page_regex/1`
* Fix disabled style in dark mode
* Add regex fallback for safari
* Show no matches found when visibleOptions is empty
* Disable enter key when no visible options
* Do not submit empty form fields
* Remove unnecessary setOpen(true)
* Remove ClickhouseSetup module
This has been an implicit point of contact to many
tests. From now on the goal is for each test to maintain
its own, isolated setup so that no accidental clashes
and implicit assumptions are relied upon.
* Implement v2 schema check
An environment variable V2_MIGRATION_DONE acts like
a feature flag, switching plausible from using old events/sessions
schemas to v2 schemas introduced by NumericIDs migration.
* Run both test suites sequentially
While the code for v1 and v2 schemas must be kept still,
we will from now on run tests against both code paths.
Secondary test run will set V2_MIGRATION_DONE=1 variable,
thus making all `Plausible.v2?()` checks return `true'.
* Remove unused function
This is a remnant from the short period when
we would check for existing events before allowing
creating a new site.
* Update test setups/factories with v2 migration check
* Make GateKeeper return site id along with :allow
* Make Billing module check for v2 schema
* Make ingestion aware of v2 schema
* Disable site transfers for when v2 is live
In a separate changeset we will implement simplified
site transfer for when v2 migration is complete.
The new transfer will only rename the site domain in postgres
and keep track of the original site prior to the transfer
so we keep an ingestion grace period until the customers
redeploy their scripting.
* Make Stats base queries aware of v2 schema switch
* Update breakdown with v2 conditionals
* Update pageview local start with v2 check
* Update current visitoris with v2 check
* Update stats controller with v2 checks
* Update external controller with v2 checks
* Update remaining tests with proper fixtures
* Rewrite redundant assignment
* Remove unused alias
* Mute credo, this is not the right time
* Add test_helper prompt
* Fetch priv dir so it works with a release
* Fetch distinct partitions only
* Don't limit inspect output for partitions
* Ensure SQL is printed to IO
* Remove redundant domain fixture
* Fix Timex.total_offset blowing up during clock changes
* Format large numbers with underscore in tests
Co-authored-by: Adam <hq@mtod.org>
---------
Co-authored-by: Adam <hq@mtod.org>
* Wrap Plausible.Stats.Filters with unit tests
* Parse `member` filter type
* Support escaped | in member filter
* Support for `member` filter in `aggregate_time_on_page`
* Add support for `member` filter type on goals
* Disable Credo warning
* Support `not_member` filter type
* Disable credo for `query_sessions`
* Support `matches_member` and `not_matches_member` filters
* Disable Credo for `Filters.filter_value/2`
* Support `matches_member` and `not_matches_member` for goal filter
* Support for contains_member and friends
* Updates for new chto driver
* make top_stats_test.exs:203 pass (#2779)
---------
Co-authored-by: ruslandoga <rusl@n-do.ga>
This pull request adds support for multiple comparison modes, changes the comparison checkbox to a combobox, and implements the year over year comparison mode. The feature is still behind a feature flag.
Co-authored-by: Uku Taht <uku.taht@gmail.com>
* globally rename 'pages_per_visit' to 'views_per_visit'
* change the order of top stats
* rename 'Visits' to 'Total visits' in the UI
* add views_per_visit to UI
* put the new metric under a feature flag
* add new metric to CSV export under feature flag
* mix format
* use only one feature flag
* refactor metric validation
* link to the correct docs section
* add the new metric to aggregate API
* explicitly remove __internal_visits
The overall metric selection is well defined by
`Plausible.Stats.Timeseries.empty_row/2`. The only metric that needs
to be removed from the timeseries response is __internal_visits.
This commit also moves the `remove_internal_visits_metric` function to a
new Util module to be used by both breakdown and timeseries.
* add the new metric to timeseries API
* mix format
* add moduledoc to keep credo happy
* convert pages_per_visit to string straight away
* do rounding in db query
* query # of visits with sum(sign) instead
* stop converting pages_per_visit to string
* Use user-agent instead of screen_width to get device type
Co-authored-by: eriknakata <erik.nakata5@gmail.com>
* Fix credo
* Log on unhandled UAInspector device type
* Make 'browser' the default tab in devices report
* Remove device tooltip
* Remove screen_width from ingestion completely
* Remove browserstack harness, run playwright directly
* Select meta key based on OS platform
* Run CI tests in parallel
* Improve device match readability
* Add changelog
---------
Co-authored-by: eriknakata <erik.nakata5@gmail.com>
* Clickhouse migration: add ingest_counters table
* Configure ingest counters per MIX_ENV
* Emit telemetry for ingest events with rich metadata
* Allow building Request.t() with fake now() - for testing purposes
* Use clickhousex branch where session_id is assigned to each connection
* Add helper function for getting site id via cache
* Add Ecto schema for `ingest_counters` table
* Implement metrics buffer
* Implement buffering handler for `Plausible.Ingestion.Event` telemetry
* Implement periodic metrics aggregation
* Update counters docs
* Add toStartOfMinute() to ordering key
* Reset the sync connection state in `after` clause
* Flush counters on app termination
* Use separate Repo with async settings enabled at config level
* Switch to clickhouse_settings repo root config key
* Add AsyncInsertRepo module
* Add visits metric and make it graphable
* include visits metric in csv export (visitors.csv)
* put visits under a feature flag (CSV export)
* feature flag for displaying visits on the dashboard
* fix formatting
* add visits metric to top stats (fix)
* fix imported_test to expect visits metric included
* fix formatting
* Use HeadlessUI for search select box
* Remove downshift from package.json
* More consistent API for Combobox component
* Combine toFilterType and valueWithoutPrefix into a single function
* Rename MyCombobox -> PlausibleCombobox
* Update webpack-cli
* Disable cache for build
* Revert "Disable cache for build"
This reverts commit aa130541f8.
* Disable cache for build
* Update webpack dependencies
* Remove glob from webpack config
* Webpack is required by package.json
* Require autoprefixer in postcss config
* Revert build changes
* Fix styling for dark mode
* Reject unknown imported cities from queries
This commit fixes a bug where the city report returned `N/A` entries.
The functions that build imported data queries were using SQL
`COALESCE`, assuming city data is `NULL` when unknown, when actually its
unknown value is `0`.
This commit addresses the problem using SQL `NULLIF` combined with the
previous `COALESCE` call. With this change both `0` and `NULL` are
treated as unknown.
Since 1cb07efe6d cities can be `NULL`, but
previously we saved `0` as unknown.
Closes#1960
* Add entry to CHANGELOG
* Ignore cyclomatic complexity Credo check
This commit adds city data to imported records from Google Analytics. The
current implementation sets city to 0 because GA does not use the GeoNames
database.
Google Analytics Reporting API uses [Geographical IDs](https://developers.google.com/analytics/devguides/collection/protocol/v1/geoid)
to identify cities and countries. Plausible uses
[GeoNames](https://geonames.org/) and I couldn't find databases corelating the
two.
Fortunately, GA also returns the city name and this commit uses the city name
and the country ISO code to find the Geoname ID. To avoid making expensive ETS
searches, I created another ETS table in the Location library that uses
{country, city} as a key.
Related PR: https://github.com/plausible/location/pull/3
* Configure ingest repo access/pool size
If I'm not mistaken 3 is a sane default, the only
inserts we're doing are:
- session buffer dump
- events buffer dump
- GA import dump
And all are serializable within their scopes?
* Add IngestRepo
* Start IngestRepo
* Use IngestRepo for inserts
* Annotate ClickhouseRepo as read_only
So no insert* functions are expanded
* Update moduledoc
* rename alias
* Fix default env var value so it can be casted
* Use IngestRepo for migrations
* Set default ingest pool size from 3 to 5
in case conns are restarting or else...
* Ensure all Repo prometheus metrics are collected
* Make Device section components aware of (not set)
So that no extra sub-filters are possible when the unset
top item is selected.
* Support '(not set)' in breakdown/filters
* Update expectations for export tests
* Add extra tests for returning/filtering by '(not set)'
* Add changelog entry
* Remove ListReport conditional render
* Prevent redundant sub-filters
* Fix filter text rendering
---------
Co-authored-by: Uku Taht <uku.taht@gmail.com>
* Move Endpoint errors setup to common config
* Implement naive Sentry link resolver
* Implement error report e-mail
* Delete static sentry script
* Implement user feedback form on server errors
* Re-arrange pipe
* Use Sentry.Config.dsn() where applicable
* Fix typo
* Use Map.replace/3
Some changes to be more consistent with the emails we send. Also "valid" subscription rather than "active" subscription fits better for the different cases where this screen is shown
* fix subquery for sessions in base_event_query/2
As the 'sessions' table is using the CollapsingMergeTree engine, we have
to select session_id's distinctively. Otherwise we will get multiple rows
(with sign -1 and 1) as long as the background merge hasn't happened.
* update changelog
* use GROUP BY instead of SELECT DISTINCT
* remove comma
* Fingerprint DBConnection.ConnectionError in Sentry
* Check events before creating a site
* Enable sites limit screen
* Remove debugging remnant
* Fix buggy assertions
This wasn't doing what expected:
iex(1)> Repo.exists?(Plausible.Site, domain: "foo")
[debug] QUERY OK source="sites" db=0.6ms idle=1906.2ms
SELECT TRUE FROM "sites" AS s0 LIMIT 1 []
* Encapsulate check to satisfy credo
* Use less technically involved error message
* Bring back e-mail to the limit error message
This PR replaces geolix with locus to simplify self-hosted setup. locus can auto-update maxmind dbs which are recommended for self-hosters if they want city-level geolocation. locus is also a bit faster.
This PR also uses a test mmdb file from https://github.com/maxmind/MaxMind-DB for e2e geolocation tests without stubs.
* Add Checkly Terraform config
* add deployment workflow
* use pagerduty instead of email for notifications
* use terraform cloud backend
* update variable declaration
* rename checkly check group
* update syntax
* test trigger
* Revert "test trigger"
This reverts commit 333e82beac.
* run a single job at a time
Co-authored-by: Cenk Kücük <c@cenk.me>
* Fix breakdown API pagination when using event metrics
This commit fixes a bug where the subsequent breakdown API pages had
the same items as the first page. The fix sorts the underlying
ClickHouse query by timestamp, keeping the same order between requests,
as we use OFFSET/LIMIT pagination.
* Fix repeated results assertion
* Add different ORDER BY to each breakdown property
* Ignore XX and T1 countries
* Add fallback if country_code=nil
* Lookup city overrides directly in CityOverrides module
* Changelog
* Add empty moduledoc
* Remove redundant comment
* Cascade delete sent_renewal_notifications table when user is deleted
This commit fixes a bug when deleting a user would trigger a constraint
error.
* Update CHANGELOG.md
* Set pg pool size for MIX_ENV=test
* Include slow tests in CI run
* Exclude slow tests by default
* Mark tests slow/async where applicable
* Restructure captcha mocks
* Revert async where env is relied upon
* Add --max-failures=1 to CI run
* Set warnings as errors
* Disable async where various mocks are used
* Revert "Disable async where various mocks are used"
This reverts commit 2446b72a29.
* Disable async for test using vcr
* Return empty list when breaking down by event:page without events
This commit fixes a bug with pagination where breaking down by event:page
would always return results despite pagination.
Closes#2255
* Update CHANGELOG.md
* Remove show_noref behaviour
Removes query param show_noref which was used from React to control
whether to show Direct / None traffic or not. The show_noref behaviour
was untested previously.
Closes#2523
* Add changelog entry
* Fix tests
* Removed files I did not mean to check in :)
### Changes
This PR:
- pushes PromEx to the bottom of supervision stack to avoid Endpoint
instrumentation failure
- ensures the site cache is ready by exposing it through the health
check endpoint
- fixes event timestamps being calculated at compile time, with
regression unit and integration tests
### Tests
- [x] Automated tests have been added
- [ ] This PR does not require tests
### Changelog
- [ ] Entry has been added to changelog
- [x] This PR does not make a user-facing change
### Documentation
- [ ] [Docs](https://github.com/plausible/docs) have been updated
- [x] This change does not need a documentation update
### Dark mode
- [ ] The UI has been tested both in dark and light mode
- [x] This PR does not change the UI
* Update Sites.Cache
So it's now capable of refreshing most recent sites.
Refreshing a single site is no longer wanted.
* Introduce Warmer.RecentlyUpdated
This is Sites Cache warmer that runs only for
most recently updated sites every 30s.
* Validate Request creation early
* Rename RateLimiter to GateKeeper and introduce detailed policies
* Update events API tests - a provisioned site is now required
* Update events ingestion tests
* Make limits visible in CRM Sites index
* Hard-deprecate DOMAIN_BLACKLIST
* Remove unnecessary clause
* Fix typo
* Explicitly delegate Warmer.All
* GateKeeper.allwoance => GateKeeper.check
* Instrument Sites.Cache measurments
* Update send_pageview task to output response headers
* Instrument ingestion pipeline
* Credo
* Make event telemetry test a sync case
* Simplify Request.uri/hostname handling
* Use embedded schema, apply action and rely on get_field
* Parse event URL in Plausible.Ingestion.Request
* Parse event domain in Plausible.Ingestion.Request
* Rework ingestion pipeline processing (#2462)
* Rework ingestion pipeline processing
So that Request can have multiple domains and
based on that each event is processed uniformly.
The build_and_buffer/1 function now returns an
accumulator with all the dropped/buffered events
for further inspection.
* Reduce function complexity
* Don't chain struct fields to check for an empty host
* Separate referrer and utm tags
* Fix up `with` clause, credo was right cc @vinibrsl
Co-authored-by: Adam Rutkowski <hq@mtod.org>
* Seed database with pageviews
This commit adds basic support for database seeding useful for testing,
especially dashboard changes, like intervals.
It creates two years of pageviews with random timestamps. There is lot
of room for improvement, such as adding sources, entry pages,
geolocation, devices, custom events, but this already helps us with
testing.
* Update CONTRIBUTING.md file
* Allow refreshing a single site cache + clear cache on prefill
* Reorganize Site.Cache tests with describe blocks
* Tidy up Cache tests
* Make sure the cache is cleared on (p)re-fill
* Allow process name customization in Cache.Warmer
* s/Cache.prefill/Cache.refresh
* Unify Cache refresh instrumentation
* Apply credo suggestion: change `with` to `case`
* Update typespecs to pass dialyzer
* Implement sites by domain caching interface + warmer
* Add test
* Implement hit rate interface
* Add moduledocs
* Fix up typespec
* s/warmer/warmer_fn
* Extract measure_duration/2
* Fix up typespec
* Log errors and return nil on cache internal errors
* Fix up non-existing cache test
* Retrieve specific db columns when pre-filling the cache
* Reduce the subset of fields retrieved from the DB
See 63f3c6233d (r89871536)
