* Migration: add country rules
* Add CountryRule schema
* Implement CountryRule cache
* Add country rules context interface
* Start country rules cache
* Lookup country rules on ingestion
* Remove :shields feature flag from test helpers
* Add nested sidebar menu for Shields
* Fix typo
* IP Rules: hide description on mobile view
* Prepare SiteController to handle multiple shield types
* Seed some country shield
* Implement LV for country rules
* Remove "YOU" indicator from country rules
* Fix small build
* Format
* Update typespecs
* Make docs link point at /countries
* Fix flash on top of modal for Safari
* Build the rule struct with site_id provided up-front
* Clarify why we're messaging the ComboBox component
* Re-open combobox suggestions after pressing Escape
* Update changelog
* Fix font size in country table cells
* Pass `added_by` via rule add options
* Display site's timezone timestamps in rule tooltips
* Display formatted timestamps in site's timezone
And simplify+test Timezone module; an input timestamp converted
to UTC can never be ambiguous.
* Remove no-op atom
* Display the maximum number of rules when reached
* Improve readability of remove button tests
* Credo
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Replace footer text
* add COPYING.txt file
* Add new logos
* Use new logos in all layouts
* New logos
* Check license key on startup
* Bypass license check when Mix.env == :dev
* Use new logos with smaller wordmarks
* Add generic logo_path/1 function
* Use new favicons everywhere
* Bypass license check in test env
* Use sha256 for license key hash
* Mix.env -> config_env()
* Use Mix.evn at compile time rather than runtime
* Mix format
---------
Co-authored-by: Uku Taht <uku.taht@gmail.com>
* Add data migration for moving to VersionedCollapsingMergeTree
This has been tested locally and partially on staging. Still requires a bit of work to verify.
Verification query:
```
SELECT main._partition_id, tmp.count, main.count
FROM (
SELECT _partition_id, count() AS count
FROM sessions_v2_tmp_versioned
GROUP BY _partition_id
) AS tmp
FULL OUTER JOIN (
SELECT _partition_id, count() AS count
FROM sessions_v2
GROUP BY _partition_id
) AS main
ON (tmp._partition_id == main._partition_id)
ORDER BY main._partition_id
```
* Add an early exit to migration
* cluster? extract common code
* Add Ecto.Network dependency
* Migration: Add ip block list table
* If Cachex errors out, mark the cache as not ready
* Add IPRule schema
* Seed IPRules
* Add Shields context module
* Implement IPRuleCache
* Start IPRuleCache
* Drop blocklisted IPs on ingestion
* Cosmetic rename
* Add settings sidebar item
* Consider IPRuleCache readiness on health checks
* Fix typo
* Implement IP blocklist live view
* Update moduledocs
* Extend contextual module tests
* Convert IPRules LiveView into LiveComponent
* Keep live flashes on the tabs view
* Update changelog
* Format
* Credo
* Remove garbage
* Update drop reason typespecs
* Update typespecs for cache keys
* Keep track of who added a rule and when
* Test if adding via LV prefills the updated_by tooltip
* Update ecto_network dependency
* s/updated_by/added_by
* s/drop_blocklist_ip/drop_shield_rule_ip
* Add docs link
* s/Updated/Added
* Reorganize how subscriptions/trials are evaluated
* Bugfix: expired trial+no subscriptions should not have access to extra features
* Make self-hosted users always on trial
* Seed secondary user with password
* Format
* Fix docs
* Fix small_test run
* Run the test only on full_build
* More tweaks to small builds
* Allow [Goals] for expired trials with no subscription
* Move limit enforcement to accepting site ownerhsip transfer
* enforce pageview limit on ownership transfer accept
* Refactor plan limit check logic
* Extract `ensure_can_take_ownership` to `Invitations` context and refactor
* Improve styling of exceeded limits notice in invitation dialog and disable button
* styling improvements to notice
* make transfer_ownership return transfer to self error
* do not allow transferring to user without active subscription WIP
* Add missing typespec and improve existing ones
* Fix formatting
* Explicitly label direct match on function argument for clarity
* Slightly refactor `CreateInvitation.bulk_transfer_ownership_direct`
* Exclude quota enforcement tests from small build test suite
* Remove unused return type from `invite_error()` union type
* Do not block plan upgrade when there's pending ownership transfer
* Don't block and only warn about missing features on transfer
* Remove `x-init` attribute used for debugging
* Add tests for `Quota.monthly_pageview_usage/2`
* Test and improve site admin ownership transfer actions
* Extend tests for `AcceptInvitation.transfer_ownership`
* Test transfer ownership controller level accept action error cases
* Test choosing plan by user without sites but with a pending ownership transfer
* Test invitation x-data in sites LV
* Remove sitelocker trigger in invitation acceptance code and simplify logic
* Add Quota test for `user.allow_next_upgrade_override` being set
* ignore pageview limit only when subscribing to plan
* Use sandbox Paddle instance for staging
* Use sandbox paddle key for staging and dev
---------
Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
* Add data retention field to plans
* Display data retention as benefit when choosing plan
* Split fields in two module attributes
* Remove extra whitespace
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Add 2FA actions to `AuthController`
* Hook up new `AuthController` actions to router
* Add `qr_code` to project dependencies
* Implement generic `qr_code` component rendering SVG QR code from text
* Implement enabled and disabled 2FA setting state in user settings view
* Implement view for initiating 2FA setup
* Implement view for verifying 2FA setup
* Implement view for rendering generated 2FA recovery codes
* Implement view for verifying 2FA code
* Implement view for verifying 2FA recovery code
* Improve `input_with_clipboard` component
* Improve view for initiating 2FA setup
* Improve verify 2FA setup view
* Implement `verify_2fa_input` component
* Improve view for verifying 2FA setup
* Improve view rendering generated 2FA recovery codes
* Use `verify_2fa_input` component in verify 2FA view
* Do not render PA contact on self-hosted instances
* Improve flash message phrasing on generated recovery codes
* Add byline with a warning to disable 2FA modal
* Extract modal to component and move 2FA components to dedicated module
* First pass on loading state for "generate new codes"
* Adjust modal button logic
* Fix button in verify_2fa_input component
* Use button component in activate view
* Implement wait states for recovery code related actions properly
* Apply rate limiting to 2FA verification
* Log failed 2FA code input attempts
* Add ability to trust device and skip 2FA for 30 days
* Improve styling in dark mode
* Fix waiting state under Chrome and Safari
* Delete trust cookie when disabling 2FA
* Put 2FA behind a feature flag
* Extract 2FA cookie deletion
* ff fixup
* Improve session management during 2FA login
* Extract part of 2FA controller logic to a separate module and clean up a bit
* Clear 2FA user session when rate limit hit
* Add id to form in verify 2FA setup view
* Add controller tests for 2FA actions and login action
* Update CHANGELOG.md
* Use `full_build?()` instead of `@is_selfhost` removed after rebase
* Update `Auth.TOTP` moduledoc
* Add TOTP token management and make `TOTP.enable` more test-friendly
* Use TOTP token for device trust feature
* Use zero-deps `eqrcode` instead of deps-heavy `qr_code`
* Improve flash messages copy
Co-authored-by: hq1 <hq@mtod.org>
* Make one more copy improvement
Co-authored-by: hq1 <hq@mtod.org>
* Fix copy in remaining spots
* Change redirect after login to accept URLs from #3560 (h/t @aerosol)
* Add tests checking handling login_dest on login and 2FA verification
* Fix regression in email activation form submit button behavior
* Rename `PlausibleWeb.TwoFactor` -> `PlausibleWeb.TwoFactor.Session`
* Move `qr_code` component under `Components.TwoFactor`
* Set domain and secure options for new cookies
---------
Co-authored-by: hq1 <hq@mtod.org>
* refactor asking for the monthly pageview usage
* add tests for usage and limits section in account settings
* display pageview usage per billing cycle for active subscribers
* disable cycle tabs if no usage
* make current billing cycle whole
...instead of capping it at today's date
* run queries for different cycles concurrently
* fix linebreak bug
* add calculate usage action into CRM
* change some names of assigns
* block subscribing to a plan by pageview usage
Depending on whether the customer has already subscribed or not, checking
their pageview usage is different:
* If they're not subscribed yet, we allow them to subscribe to a plan If
it their last 30 days usage does not exceed the plan pageview limit by
more than 15% (30% for when subscribing to a 10k plan)
* For existing subscribers, we'll use the exact same mechanism that we're
using for locking sites - the last two billing cycles usage. If both
cycles exceed the plan limit by more than 10% - we don't allow them to
subscribe to the plan
* apply credo suggestion
* prevent highlight bar overflow
* move disabled classes to button element
* optimize for darkmode
* unify link and text styling on the same horizontal line
'Upgrade' & 'Update billing details' links + billing interval text were
positioned on the same line. The font size was similar, but not the same
* improve exceeded_limits function readability
* Refactor some tests and remove code duplication
* override allow upgrade when limits exceeded
In cases where limits are exceeded, we can set the boolean flag
`allow_next_upgrade_override` to `true` in the CRM. This will allow
the user to upgrade to any plan they want. After they've upgraded or
changed their plan - the flag will automatically reset to `false`.
* only apply upgrade override for exceeded pageview limit
* fix tests on the CI
* make current_cycle usage always displayed by default
* make pageview allowance margin more clear
* add comment
* Disable super-admin checks on small build
* Mute a test writing to stdout
* Move sampling outside of small build
* Convert waiting_first_pageview to heex and stop relying on env vars
* Set site limit unlimited on small build
* Stop relying on app env to get trial expiry
* Remove custom domains - including migration
* Remove is_selfhosted from layout view
* Quota fixup
* Stop relying on app env for self hosted registration
* Stop relying on app env for pass reset success
* Apply on_trial? check only on full build
* Update templates relying on app env
* Adjusts auth controller tests for small build
* Trial fixup
* Fixup
* Stop relying on app env
* Rest of the fsckn owl
* Update typespecs
* Fix dialyzer warning
* Remove unused module
* Credo + format
* GeoIP is not, for full build
* Use `small_build?()` where applicable
* Implement bypassing FirstLaunchPlug without insertions
* Get Marko's patch de58a18a85
* Test is-dbip=false presence
* Fix typespec
* Remove future hardcodes
* Handle `nil` from `Plausible.Geo.database_type()`
* Remove XXX marker
* Use one typespec for two clauses
* Introduce `MIX_ENV=small_dev`
* Revert "Use one typespec for two clauses"
This reverts commit 8d8cd21764.
* replace unlisted plans with legacy plans
...and add a legacy plan that has an existing subscriber in Paddle. All
legacy plans are considered generation 1 - meaning that when a user on
one of these plans and they'll go to the upgrade page, then the listed
plans will be v1 for Growth and v3 for Business.
* remove redundant plans_sandbox function
* remove the unused 'scope' argument from Plans.find
* remove unused plan
* add basic test coverage for legacy plans
* add another plan with an existing active subscriber (fix another bug)
* change team member limits for new v4 plans
* duplicate business plans with unlimited team members
We need to do this because we want grandfathered users to have unlimited
team members on business plans as well. Otherwise we'd have to build
overrides on the subscription level when checking the limit.
* refactor generating plan structs
* move Plan module into a separate file
* remove not needed conditions
* add generation field to plans
* sync the sanbox plan limits and features with plan generations
* implement displaying plan benefits
* add grandfathering notice
* plug in the real v3 business plan IDs
* optimize N/A text color for darkmode
* use String.to_existing_atom instead
Co-authored-by: Vini Brasil <vini@hey.com>
* Remove the unnecessary part of a comment
Co-authored-by: Vini Brasil <vini@hey.com>
* make the Plan.new function simpler
* use exlamation marks
---------
Co-authored-by: Vini Brasil <vini@hey.com>
* Migration: track last seen usage for Plugins API Tokens
* Track and interpret Token.last_seen_at
* Display last used
* Order tokens by inserted date, rather than UUID :clown:
* s/Last seen/Last used in the UI
* Test for "Last used" column presence
* Fix table layout for very long descriptions
* Update lib/plausible/plugins/api/tokens.ex
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Update lib/plausible/plugins/api/token.ex
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Update test/plausible/plugins/api/token_test.exs
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* s/last_seen_at/last_used_at
* Update lib/plausible_web/live/plugins/api/settings.ex
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* fixup
* Document reasoning behind 5m windows
* s/last_seen/last_used
* Mute credo
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* fix text color in dark mode
* rename a function
* use aliases in quota.ex
* rename a function
* make Goals a similar feature to others but with a free option
* rename a function
* mix format
* Refactor email verification codes generation to avoid predictability
* Improve `Site.Memberships.any?` slightly
* Update tests
* Fix seeds
* Use `expired?` predicate for checking verification code validity in tests
* Store verification code as string in database to avoid unnecessary int casting
* Include ApiKey functions in Auth context
* Make feature notice work without %Site{}
Previously the extra feature notice required a %Site{} in order to check
the owner plan. However, not every feature is scoped by site, for
example the Stats API. For features like this, a %User{} is required,
and not a %Site{}.
This commit replaces the `:site` param with `:billable_user`, which is
common to both site and user-scoped features.
* Add stats_api to the list of extra features
* Limit API Key creation based on user plan
This pull request implements limits to funnels, revenue goals and custom props based on the site owner plan. It extends the current "premium feature" notice to account for the new plans, trials and the on-going private preview. Stats API is not in the context of this pull request, but will be implemented likewise.
* Implement PoC for email reverification flow on update
* Improve user settings form and email change validation
* Expose `previous_email` in Kaffy CRM
* Improve plugs setup and remove dead action from AuthController
* Fix seeds
* Extract predicate query functions from AuthController
* Add tests
* Update CHANGELOG.md
* Rename `has_any_sites?` to `Memberships.any?` and `has_any_memberships?`
* Improve flash message on cancelling email change
* Cover one more test case for email update
* Make FixBrokenGoals migration idempotent
The migration in question was renamed in order to fix order of executing migrations when run
from the ground up (via https://github.com/plausible/analytics/pull/3378).
As a side effect, it's executed again on databases that had it applied earlier, with
a different timestamp prefix.
As this migration is safe to run multiple times, it was modified to make forward
migration work gracefully when constraint already exists.
* Add `pending-migrations.sh` release script
* add a new upgrade page liveview behind a FF
* Create plans_v4.json file
* Add the upgrade page UI template and some basic functionalities
* different content based on subscription plan existing or not
* pageview slider
* monthly/yearly switch
* fix tests
* split into 2 separate functions
* rename variables
* implement volume slider + read default interval/volume from plan
* organize choose-plan.ex better
* remove unused vars from tests
* make monthly_cost and yearly_cost nil by default
The actual prices for all plans are stored in Paddle. We don't need to
keep the duplicates in the JSON files.
* add fetch_prices/1 to PaddleApi
* make v4 business ID's differ from growth ones
* render actual price information from plans
...and make the prices in both growth and business plan boxes change
dynamically when the pageview slider or interval is changed.
* highlight current subscription plan box
* add test describe block for business tier subscription
* connect to live socket only on the specific LV page using focus.html
* only wrap the input slider inside the form
* little readability improvement
* add v4 team_member_limits (after rebase with master)
* extract monthly_quota_box function in user_settings
When the business_tier FF is enabled, this section is different and
links to the new upgrade page.
* document subscription statuses
* change _notice.html.eex to .heex
* extract subscription status notice components
* add failed payment notices to upgrade page
* create class_of_element/2 convenience function for testing
* add cancel_subscription mix task
* implement checkout buttons
* mix format
* get all available plans with prices through plans.ex
* use more suitable function for fetching usage
* avoid double db lookups on mount
* rename variable
* separate functions for getting plan by product_id vs subscription
* separate subscription status docs into context module
* consider cancelled subscriptions
* default volume by usage if no subscription plan
* add enterprise-level volume option to slider
* optimize for darkmode
* UI improvements
* display 2 months free notice for yearly billing
* VAT excluded notice
* note about having a business subscription in user settings
* make the page pop and fit plans on screen on first render
* optimize for mobile and remove background containers
* change default price tag to simply 'N/A'
* fix tests
* Change Paddle.js integration to use JavaScript directly
* rename many variables
* allow users on v1 and v2 plan subscribe to 20M and 50M tiers
* add a test for two months free label
* make it work with a free_10k subscription
* small test improvement and formatting
* change other upgrade link in user settings if FF enabled
* dialyzer
* fix typo
* add test for free_10k user
* silence credo
* mix format
* credo - add moduledoc
* credo - another moduledoc
* handle calls to sentry on the api level
* refactor getting regular subscription plan for LiveView
* post review code style tweaks
* remove unused aliases
* credo - add @moduledoc false to Subscriptions
* crash in cancel_subscription task when Repo update fails
* readability improvements (review suggestions)
* add comment about 'external_resource' module attr
---------
Co-authored-by: Vinicius Brasil <vini@hey.com>
* Update depenedencies: OpenAPISpex + cursor based pagination
* Update formatter config
* Add internal server error implementation
* Test errors
* Implement pagination interface
* Implement Plugins API module macros
* Implement Public API base URI
(to be used with path helpers once called from within
forwarded router's scope)
* Implement OpenAPI specs + schemas
* Implement Shared Links context module
* Add pagination and error views
* Add Shared Link view
* Implement Shared Link controller
* Expose SharedLink.t() spec
* Implement separate router for the Plugins API
* Update moduledocs
* Always wrap resource objects with `data`
* Update moduledoc
* Use https://github.com/open-api-spex/open_api_spex/pull/425
due to https://github.com/open-api-spex/open_api_spex/issues/92
* Rely on BASE_URL for swagger-ui server definition
* Fixup goals migration
* Migrate broken goals before deleting dupes
* Remove bypassing test rate limiting for which there's none anyway
* Move the context module under `Plausible.` namespace
* Bring back conn assignment to PluginsAPICase template
* Update test/plausible_web/plugins/api/controllers/shared_links_test.exs
Co-authored-by: Uku Taht <Uku.taht@gmail.com>
* Update renamed aliases
* Seed static token for development purposes
* Delegate Plugins API 500s to a familiar shape
* Simplify with statement
---------
Co-authored-by: Uku Taht <Uku.taht@gmail.com>
* Enforce goals unique
* Remove unnecessary alias
* Skip tests that can no longer run anymore
To run, make sure the migration from
priv/repo/migrations/20230914071245_goals_unique.exs
is rolled back.
* Use separate transactions for the migration
* Update priv/repo/migrations/20230914071245_goals_unique.exs
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Add `RewriteFunnelDupes` data fix
To rewrite funnels referencing goals whose names
are the same. This enables us to enforce all goals
within the site unique later on.
* Credo
* Update priv/data_migrations/FunnelDupeGoals/sql/list-funnels-with-dupe-goal-ids.sql.eex
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Revert unrelated changes
* Remove dead code
* Update lib/plausible/data_migration/rewrite_funnel_dupes.ex
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Update lib/plausible/data_migration/rewrite_funnel_dupes.ex
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
---------
Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
* Rewrite goals for which both event_name and page_path exist.
This might've happened due to the old form only hiding form
fields from the viewport, allowing to submit both values in
certain cases when switching tabs. The tabs behaviour has
been changed as of #3293 but no proper constraints existed.
* Add new check constraint to the goals schema
* Use NOT VALID option for adding the CHECK constraint
* Skip wrapping migration in a single transaction
* Add hint to creatable ComboBoxes without suggestions available
* Load external resources once in funnel settings
* Load external resources once in goal settings
* Make Custom Props Settings UI match Goal Settings
* Remove unnecessary goals query
This should be done only once in the live view
* Remove funnels feature flag
* fixup
* Make the modal scrollable
* By default, focus first suggestion for creatables
* Add sample props to seeds
* Load all suggestions asynchronously, unless `Mix.env == :test`
* ComboBox: Fix inconsistent suggestions
We require "Create ..." element to be only focused
when there are no suggestions available.
This causes some issues, depending on the state,
the least focusable index might be either 0 ("Create...")
or 1. This patch addresses all the quirks with focus.
* Fix ComboBox max results message
So that AlpineJS doesn't think it's a focusable
option.
* Keep the state up to date when changing props
* Update seeds with sensible prop names
* Make escape work for closing combobox suggestions
Co-authored-by: Uku Taht <Uku.taht@gmail.com>
* Revert "Make escape work for closing combobox suggestions"
This reverts commit 306866d2a1.
@ukutaht unfortunately this makes it impossible to select
an suggestion.
* Revert "Revert "Make escape work for closing combobox suggestions""
This reverts commit 4844857812.
* Make ESC great again
* Improve readability
---------
Co-authored-by: Uku Taht <Uku.taht@gmail.com>
* Refactor MembershipController.invite_member/2
This commit refactors the controller action used for creating new
invitations. It moves the code to Plausible.Sites.invite/4 and replaces
`ifs` and `cases` with `with`.
* Add team_member_limit to plan definition
* Create usage and limits functions for team members
* Apply team member limit when inviting new users
* Add team members to Usage & Limits section
* Change invite function to receive email address instead of %User{}
* Wrap invite function in a DB transaction
* Remove unnecessary joins from team member usage query
* Replace UNION ALL with UNION to remove duplicates
This pull request introduces a series of improvements to Plausible.Billing.Plans, including:
* Tag the JSON file with the plan version
* Rename the JSON field limit to monthly_pageview_limit
* Move site_limit function to Billing.Plans
* Refactor subscription_interval, allowance and site_limit functions
* Remove unused AnalyzePlans task
This commit introduces a series of improvements on the Plans module including function renaming, documentation and readability. This is the groundwork for billing plans.
There should be no actual changes with this commit, therefore no changes in tests either.
This commit adds revenue data to top stats. Average and total are displayed when filtering by a revenue goal (or many if they have the same currency set).
This pull request removes the comparisons feature flag, making it visible to all users. It also removes percentages from top stats in default view, leaving those for the comparison view only.
This commit fixes a bug where comparisons would not work with imported data. This is because comparisons copies the source query and modifies the dates only, where it should reevaluate if there is imported data for the new comparison date range.
Closes#2870
Co-authored-by: Adam <hq@mtod.org>
* default to v2
* allow N defaults in data migration prompt and custom messages
* join domains lookup
* remove duplicate test runs from ci (both are v2)
* Get rid of PASS_V2_SCHEMA_MIGRATION
* Use in-memory domain lookup + regular table settings
* Remove faulty date arithmetic + prev part calculation
* Set V2_MIGRATION_DONE in Mix.env == :dev
* Mute credo
* Full migration (to be submitted separately)
* Do not remove `Goal.domain` just yet
* Do not make Goal.site not nullable just yet
* Temporarily disable goal creation
* Rephrase error message
* Add down migration
* Migration (PR: https://github.com/plausible/analytics/pull/2802)
* Implement Site.Domain interface allowing change and expiry
* Fixup seeds so they work with V2_MIGRATION_DONE=1
* Update Sites.Cache so it's capable of multi-keyed lookups
* Implement worker handling domain change expiration
* Implement domain change UI
* Implement transition period for public APIs
* Exclude v2 tests in primary test run
* Update lib/plausible_web/controllers/site_controller.ex
Co-authored-by: Vini Brasil <vini@hey.com>
* Update lib/plausible_web/controllers/site_controller.ex
Co-authored-by: Vini Brasil <vini@hey.com>
* Update moduledoc
* Update changelog
* Remove remnant from previous implementation attempt
* !fixup
* !fixup
* Implement domain change via Sites API
cc @ukutaht
* Update CHANGELOG
* Credo
* !fixup commit missing tests
* Allow continuous domain change within the same site
---------
Co-authored-by: Vini Brasil <vini@hey.com>
* Remove ClickhouseSetup module
This has been an implicit point of contact to many
tests. From now on the goal is for each test to maintain
its own, isolated setup so that no accidental clashes
and implicit assumptions are relied upon.
* Implement v2 schema check
An environment variable V2_MIGRATION_DONE acts like
a feature flag, switching plausible from using old events/sessions
schemas to v2 schemas introduced by NumericIDs migration.
* Run both test suites sequentially
While the code for v1 and v2 schemas must be kept still,
we will from now on run tests against both code paths.
Secondary test run will set V2_MIGRATION_DONE=1 variable,
thus making all `Plausible.v2?()` checks return `true'.
* Remove unused function
This is a remnant from the short period when
we would check for existing events before allowing
creating a new site.
* Update test setups/factories with v2 migration check
* Make GateKeeper return site id along with :allow
* Make Billing module check for v2 schema
* Make ingestion aware of v2 schema
* Disable site transfers for when v2 is live
In a separate changeset we will implement simplified
site transfer for when v2 migration is complete.
The new transfer will only rename the site domain in postgres
and keep track of the original site prior to the transfer
so we keep an ingestion grace period until the customers
redeploy their scripting.
* Make Stats base queries aware of v2 schema switch
* Update breakdown with v2 conditionals
* Update pageview local start with v2 check
* Update current visitoris with v2 check
* Update stats controller with v2 checks
* Update external controller with v2 checks
* Update remaining tests with proper fixtures
* Rewrite redundant assignment
* Remove unused alias
* Mute credo, this is not the right time
* Add test_helper prompt
* Fetch priv dir so it works with a release
* Fetch distinct partitions only
* Don't limit inspect output for partitions
* Ensure SQL is printed to IO
* Remove redundant domain fixture
The compilation warning advises to use `Application.compile_env/3`
however there is no `:url` environment setting under `:plausible` key
anyway, so we might as well get rid of it completely.
This commit moves a migration that was created in `clickhouse_repo/` to
its correct folder, `ingest_repo/`. The migration was created in
1cb07efe6d prior the read/write
separation.
This commit adds city data to imported records from Google Analytics. The
current implementation sets city to 0 because GA does not use the GeoNames
database.
Google Analytics Reporting API uses [Geographical IDs](https://developers.google.com/analytics/devguides/collection/protocol/v1/geoid)
to identify cities and countries. Plausible uses
[GeoNames](https://geonames.org/) and I couldn't find databases corelating the
two.
Fortunately, GA also returns the city name and this commit uses the city name
and the country ISO code to find the Geoname ID. To avoid making expensive ETS
searches, I created another ETS table in the Location library that uses
{country, city} as a key.
Related PR: https://github.com/plausible/location/pull/3
* Configure ingest repo access/pool size
If I'm not mistaken 3 is a sane default, the only
inserts we're doing are:
- session buffer dump
- events buffer dump
- GA import dump
And all are serializable within their scopes?
* Add IngestRepo
* Start IngestRepo
* Use IngestRepo for inserts
* Annotate ClickhouseRepo as read_only
So no insert* functions are expanded
* Update moduledoc
* rename alias
* Fix default env var value so it can be casted
* Use IngestRepo for migrations
* Set default ingest pool size from 3 to 5
in case conns are restarting or else...
* Ensure all Repo prometheus metrics are collected
* Cascade delete sent_renewal_notifications table when user is deleted
This commit fixes a bug when deleting a user would trigger a constraint
error.
* Update CHANGELOG.md
