Commit Graph

155 Commits

Author SHA1 Message Date
ruslandoga
0eedf9aa98
rm Hammer (#3571) 2023-12-06 15:07:37 +01:00
Adrian Gruntkowski
da0fa6c355
Implement UI for 2FA setup and verification (#3541)
* Add 2FA actions to `AuthController`

* Hook up new `AuthController` actions to router

* Add `qr_code` to project dependencies

* Implement generic `qr_code` component rendering SVG QR code from text

* Implement enabled and disabled 2FA setting state in user settings view

* Implement view for initiating 2FA setup

* Implement view for verifying 2FA setup

* Implement view for rendering generated 2FA recovery codes

* Implement view for verifying 2FA code

* Implement view for verifying 2FA recovery code

* Improve `input_with_clipboard` component

* Improve view for initiating 2FA setup

* Improve verify 2FA setup view

* Implement `verify_2fa_input` component

* Improve view for verifying 2FA setup

* Improve view rendering generated 2FA recovery codes

* Use `verify_2fa_input` component in verify 2FA view

* Do not render PA contact on self-hosted instances

* Improve flash message phrasing on generated recovery codes

* Add byline with a warning to disable 2FA modal

* Extract modal to component and move 2FA components to dedicated module

* First pass on loading state for "generate new codes"

* Adjust modal button logic

* Fix button in verify_2fa_input component

* Use button component in activate view

* Implement wait states for recovery code related actions properly

* Apply rate limiting to 2FA verification

* Log failed 2FA code input attempts

* Add ability to trust device and skip 2FA for 30 days

* Improve styling in dark mode

* Fix waiting state under Chrome and Safari

* Delete trust cookie when disabling 2FA

* Put 2FA behind a feature flag

* Extract 2FA cookie deletion

* ff fixup

* Improve session management during 2FA login

* Extract part of 2FA controller logic to a separate module and clean up a bit

* Clear 2FA user session when rate limit hit

* Add id to form in verify 2FA setup view

* Add controller tests for 2FA actions and login action

* Update CHANGELOG.md

* Use `full_build?()` instead of `@is_selfhost` removed after rebase

* Update `Auth.TOTP` moduledoc

* Add TOTP token management and make `TOTP.enable` more test-friendly

* Use TOTP token for device trust feature

* Use zero-deps `eqrcode` instead of deps-heavy `qr_code`

* Improve flash messages copy

Co-authored-by: hq1 <hq@mtod.org>

* Make one more copy improvement

Co-authored-by: hq1 <hq@mtod.org>

* Fix copy in remaining spots

* Change redirect after login to accept URLs from #3560 (h/t @aerosol)

* Add tests checking handling login_dest on login and 2FA verification

* Fix regression in email activation form submit button behavior

* Rename `PlausibleWeb.TwoFactor` -> `PlausibleWeb.TwoFactor.Session`

* Move `qr_code` component under `Components.TwoFactor`

* Set domain and secure options for new cookies

---------

Co-authored-by: hq1 <hq@mtod.org>
2023-12-06 12:01:19 +01:00
Adrian Gruntkowski
65cc8980e0
Implement core logic for TOTP support (#3525)
* Add `nimble_totp`, `cloak` and `cloak_ecto` to project dependencies

* Setup Cloak-based secrets vault and create a dedicated Ecto type

* Add `totp_enabled|secret|last_used_at` fields to `User` schema

* Implement schema and stateless logic for TOTP recovery codes

* Implement core logic of TOTP auth

* Fix typos and improve style of doc comments

Co-authored-by: hq1 <hq@mtod.org>

* Fix moduledoc alignment

* Use more compact conditional expression

Co-authored-by: hq1 <hq@mtod.org>

* Disambiguate `I` as `7` when generating recovery codes (h/t @hq1)

* Fix a typo in runtime config error message

---------

Co-authored-by: hq1 <hq@mtod.org>
2023-11-20 14:04:48 +01:00
ruslandoga
4586e091a5
Update PromEx to fix a memory leak in benchmarks (#3532) 2023-11-20 09:00:22 +01:00
Vinicius Brasil
cfaa5be8f4
Configurable limits for enterprise plans (#3527) 2023-11-16 21:57:14 -03:00
ruslandoga
f2a8809fd1
unlock unused deps (#3518) 2023-11-14 09:30:33 +01:00
hq1
671904aefd
Enable JSON log formatting (#3512)
* Enable JSON log formatting

if `LOG_FORMAT=json` is set, the app will start logging
JSON-formatted messages. By the way, the Repo :loggers configration
was removed since it's been already deprecated by Ecto v3
(https://hexdocs.pm/ecto/changelog.html#deprecations-4).

* Update changelog
2023-11-13 14:53:11 +01:00
Adrian Gruntkowski
07cab27fef
Implement new sites view (#3463)
* Implement complete basics of LV sites

* Reimplement everything in LV except pagination

* Implement basic search capability

* PoC: plot visitors on sites index

* Add rudimentary clipped gradient in minicharts

* Fix clipping gradient, define once

* Format

* Add moduledoc to visitors component

* Move paginator helpers to the top core namespace

* Fix typespec of `Plausible.Sites.list`

* Split sites component into subcomponents

* Add function to uniformly calculate 24h intervals
and visitor totals across multiple sites.

* Integrate batch 24h interval query with plots on sites view

* Don't confuse heex compiler with alpine @ shorthands

* Make linear gradient svg definition truly invisible

* Implement basic pagination

* Extract `site_stats` from site and invitation cards

* Improve pagination

* Tweak css

* Improve filtering on pagination and make WSS fail graceful

* Test `last_24h_visitors_hourly_intervals/2`

* Replace /sites with LV implementation

* Add debounce to search filter

* Fix typespecs

* Fix styling

* Fix mini graph scaling factor calculation

* Fix search consuming itself

* Minimal tweaks to the plots

* Fixup

* Remove magic numbers from the plot

* Create `site_pins` table

* Add `SitePin` schema

* Implement listing invitations, sites and pins in a single query

* Add FIXME note

* Remove site pins for now

* Add tests for `Plausible.Sites.list/3`

* Add a couple more tests to sites dead view

* Remove unnecessary FIXME

* Add LV tests for Sites

* Calculate and display 24h visitors change

* Render the change in bold

* Add clarfying comment on virtual field in `Site` schema

* Remove unnecessary function from Invitations API

* Remove unused list opt from type definition in `Sites`

* Improve joins in list query slightly

* Add comment on manually computing sites list total

* Start searching from a singly character in domain field

* Add typespec to `last_24h_visitors_hourly_intervals`

* Extend moduledoc in visitors component

* Simplify loading sites in LV

* Simplify assigns in LV

* Add missing group for shadow under site card

* Make invitation modal render

* Make HTML in sites LV semantically correct

* Remove autofocus and focus search on `/`

* Remove shadow from search input

* Make search cancel on escape

* Fix tests relying on outdated HTML structure

* Make visitor chart color scheme consistent with dashboard chart

* Update styling of trend labels

* Fix empty state and improve search blur/focus handling

* Use live navigation for pagination

* Implement spinner on load from search

* Remove unused `Plausible.Stats.Clickhouse.last_24h_visitors/1`

* Calculate uniques correctly across hour boundaries

* Swap inlined svg for Heroicons component in invitation modal

* Add order by to base query in 24h hourly intervals

* Revert "Add order by to base query in 24h hourly intervals"

This reverts commit a6be5e3026.

* Query clickhouse 24h visitors only on second mount

* Remove redundant sign from percentage change when negative

* Switch to offset-based pagination

  - offset seems easier to deal with for when actions on
    paginated list will be performed such as site pinning;
    tracking cursor data makes some entries disappear in
    edge cases. The data set is still fairly small and
    static, even for large customers.
  - we're removing Phoenix.Pagination as it doesn't really
    fir any use case, and it was only used to limit the number
    of sites in the site picker
  - site picker is now limited to 9 sites (future: pinned
    sites will be prioritized there)
  - no need to re-query for total count any more
  - BTW, the old /sites template was removed

* Refine the plot queries; Tests pass snapshot

* Add PromEx plugin for LiveView

* Fix tiny plot cut-off at the top

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
2023-11-02 13:18:11 +01:00
Uku Taht
cfe81d6d3f
Simplify asset building (#3459)
* Add tailwind and esbuild

* Remove unused images

* Move unprocessed assets to priv directory

* Fix applyTheme script

* Remove autoprefixer

* Update bundlemon

* Remove babel config

* Revert "Remove autoprefixer"

This reverts commit fc60c31c73.

* Make dashboard react file work

* Fix app.css imports

* Remove autoprefixer

* Add back in robots.txt

* Go back to css/ and js/ folders as opposed to assets/

* Bundle embed.host.js and embed.content.js

* Add components folder to live reload paths

* Remove bundlemon

* Use mix assets task in Dockerfil

* Add assets setup to CONTRIBUTING.md
2023-11-01 16:27:29 +02:00
hq1
117eef000d
Upgrade Erlang/Elixir stack (#3454)
* Bump deps

* Bump stack

* Fix deprecation warnings

* Fix VCR cassettes mismatch due to OTP-18414

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

* Format & fix flaky tests

* Handle raw IPv4 hostnames; test public suffix TLD

* Configure locus db cache_dir

So that maxmind unavailability doesn't affect
application startup. PERSISTENT_CACHE_DIR env var is used
to point locus at the GeoIP DB file.

* WIP: Remove ExVCR

* Fix test env config

* Fixup exvcr

* Remove exvcr from deps

* Add convert script

* Remove exvcr cassettes

* Remove convert script

* Rename test

* Update moduledoc

* Update dockerfile

* Bump CI cache

* Tag more slow tests, why not?

* Use charlist for locus cache option

* Pin nodejs

* Merge google tests, make them async

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
2023-10-24 10:33:48 +02:00
Uku Taht
97b24c0492
Nolt sso (along with a better nav dropdown) (#3395)
* Add SSO link with signed JWT token

* Falls back to Nolt URL without SSO if token cannot be generated

* Add profile image (gravatar) to Nolt SSO link

* Improve navbar dropdown

* Add 'contact support' link to nav dropdown

* Add CSS rule to prevent horizontal jumps

* Dark mode styling

* Close dropdown when link is clicked

* Clarify links in dropdown

* Clarify CSS comment

* Use Alpine.data() over window

* Rename suggestions_dropdown -> combo-box

* Mix format

* Make logout link look good on dark mode

* Use proxy for gravatar

* Do not use Gravatar proxy in self-hosted

* Changelog

* Add Github Repo link to nav dropdown

* Make dialyzer happy

* Add proxy for Gravatar

* Update assets/css/app.css

Co-authored-by: hq1 <hq@mtod.org>

* Update lib/plausible_web/controllers/avatar_controller.ex

Co-authored-by: hq1 <hq@mtod.org>

* Fix alpine <> Liveview integration

---------

Co-authored-by: hq1 <hq@mtod.org>
2023-10-17 12:01:27 +03:00
hq1
082ec91c63
OpenAPI: first pass on Plugins API - Shared Links (#3378)
* Update depenedencies: OpenAPISpex + cursor based pagination

* Update formatter config

* Add internal server error implementation

* Test errors

* Implement pagination interface

* Implement Plugins API module macros

* Implement Public API base URI

(to be used with path helpers once called from within
forwarded router's scope)

* Implement OpenAPI specs + schemas

* Implement Shared Links context module

* Add pagination and error views

* Add Shared Link view

* Implement Shared Link controller

* Expose SharedLink.t() spec

* Implement separate router for the Plugins API

* Update moduledocs

* Always wrap resource objects with `data`

* Update moduledoc

* Use https://github.com/open-api-spex/open_api_spex/pull/425

due to https://github.com/open-api-spex/open_api_spex/issues/92

* Rely on BASE_URL for swagger-ui server definition

* Fixup goals migration

* Migrate broken goals before deleting dupes

* Remove bypassing test rate limiting for which there's none anyway

* Move the context module under `Plausible.` namespace

* Bring back conn assignment to PluginsAPICase template

* Update test/plausible_web/plugins/api/controllers/shared_links_test.exs

Co-authored-by: Uku Taht <Uku.taht@gmail.com>

* Update renamed aliases

* Seed static token for development purposes

* Delegate Plugins API 500s to a familiar shape

* Simplify with statement

---------

Co-authored-by: Uku Taht <Uku.taht@gmail.com>
2023-10-02 11:18:49 +02:00
Adrian Gruntkowski
51c1138d0d
Implement better user password validation (#3344)
* Add zxcvbn dependency

* Change password length range requirement from 6-64 to 12-128

* Reimplement register form in LV

* Implement server-side check for password strength

* Add rudimentary strength meter

* Make password input with strength a separate component and improve it

* Fix existing tests to provide strong enough password

* Apply formatting

* Replace existing registration form with new one

* Hide built-in label in `.input` component when none provided

* Crop password to first 32 chars for analysis by zxcvbn

* Add tests for new form components

* Integrate hCaptcha into LV

* Fix existing AuthController tests

* Add tests for Live.RegisterForm

* Hide strength meter when password input is empty

* Randomize client IP in headers during tests to avoid hitting rate limit

* Apply auxilliary formatting fixes to AuthController

* Integrate registration from invitation into LV registration logic

* Fix existing password set and reset forms

* Make `password_length_hint` component more customizable

* Optimize `Auth.User.set_password/2`

* Remove unnecessary attribute from registration form

* Move password set and reset forms to LV

* Add tests for SetPasswordForm LV component

* Add tests for password checks in `Auth.User`

* Document code a bit

* Implement simpler approach to hCaptcha integration

* Update CHANGELOG.md

* Improve consistency of color scheme

* Introduce debounce across all text inputs in registration and password forms

* Fix email input background in register form

* Ensure only single error is rendered for empty password confirmation case

* Remove `/password` form entirely in favor of preferred password reset

* Remove unnecessary `router` option from `live_render` calls

* Make expensive assigns in LV with `assign_new` (h/t @aerosol)

* Accept passwords longer than 32 bytes uniformly as very strong

* Avoid displaying blank error side by side with weak password error

* Make register actions handle errors gracefully

* Render only a single piece of feedback to reduce noise

* Make register and password reset forms pw manager friendly (h/t @cnkk)

* Move registration forms to live routes

* Delete no longer used deadviews

* Adjust registration form in accordance to changes in #3290

* Reintroduce dogfood page path for invitation form from #3290

* Use alternative approach to submitting plausible metrics from LV form

* Rename metrics events and extend tests to account for them
2023-09-25 10:27:29 +02:00
ruslandoga
c3910ae201
log mailer errors (#3336)
* log mailer errors

* update sentry to fix logger metadata bug

* update changelog
2023-09-21 13:06:28 +03:00
hq1
b3ff695797
Improve goal settings UX (#3293)
* Add Heroicons dependency

* Add name_of/1 html helper

Currently with Floki there's no way to query for
`[name=foo[some]]` selector

* Update changelog

* Make goal deletion possible with only goal id

* Remove stale goal controllers

* Improve ComboBox component

- make sure the list options are always of the parent input width
- allow passing a suggestion function instead of a module

* Stale fixup

* Update routes

* Use the new goals route in funnel settings

* Use a function in the funnel combo

* Use function in the props combo

* Remove old goals form

* Implement new goal settings

* Update moduledoc

* Fix revenue switch in dark mode

* Connect live socket on goal settings page

* Fixup

* Use Heroicons.trash icon

* Tweak goals search input

* Remove unused alias

* Fix search/button alignment

* Fix backspace icon alignment

* Delegate :superadmin check to get_for_user/3

I'll do props settings separately, it's work in progress
in a branch on top of this one already. cc @ukutaht

* Rename socket assigns

* Fixup to 5c9f58e

* Fixup

* Render ComboBox suggestions asynchronously

This commit:
  - prevents redundant work by checking the socket connection
  - allows passing no options to the ComboBox component,
    so that when combined with the `async` option, the options
    are asynchronously initialized post-render
  - allows updating the suggestions asynchronously with the
    `async` option set to `true` - helpful in case of DB
    queries used for suggestions

* Update tests

* Throttle comboboxes

* Update tests

* Dim the search input

* Use debounce=200 in ComboBox component

* Move creatable option to the top

* Ensure there's always a leading slash for goals

* Test pageview goals with leading / missing

* Make the modal scrollable on small viewports
2023-09-04 13:44:22 +02:00
hq1
20d31a8d0d
Bump ua_inspector/matomo db (#3263)
* Bump ua_inspector/matomo db

* Update changelog

* Bump matomo to master

* Pin matomo sha
2023-08-14 16:03:12 +02:00
dependabot[bot]
66bbef36bb
Bump timex from 3.7.8 to 3.7.11 (#3198)
Bumps [timex](https://github.com/bitwalker/timex) from 3.7.8 to 3.7.11.
- [Release notes](https://github.com/bitwalker/timex/releases)
- [Changelog](https://github.com/bitwalker/timex/blob/main/CHANGELOG.md)
- [Commits](https://github.com/bitwalker/timex/commits)

---
updated-dependencies:
- dependency-name: timex
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-27 14:27:41 +02:00
dependabot[bot]
eb3222f840
Bump phoenix_ecto from 4.4.0 to 4.4.2 (#3194)
Bumps [phoenix_ecto](https://github.com/phoenixframework/phoenix_ecto) from 4.4.0 to 4.4.2.
- [Changelog](https://github.com/phoenixframework/phoenix_ecto/blob/main/CHANGELOG.md)
- [Commits](https://github.com/phoenixframework/phoenix_ecto/compare/v4.4.0...v4.4.2)

---
updated-dependencies:
- dependency-name: phoenix_ecto
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-26 10:43:47 +02:00
dependabot[bot]
15daa0222f
Bump ex_doc from 0.29.4 to 0.30.3 (#3187)
Bumps [ex_doc](https://github.com/elixir-lang/ex_doc) from 0.29.4 to 0.30.3.
- [Changelog](https://github.com/elixir-lang/ex_doc/blob/main/CHANGELOG.md)
- [Commits](https://github.com/elixir-lang/ex_doc/compare/v0.29.4...v0.30.3)

---
updated-dependencies:
- dependency-name: ex_doc
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-25 09:57:12 +02:00
dependabot[bot]
79274480aa
Bump ex_money from 5.12.4 to 5.15.0 (#3177)
Bumps [ex_money](https://github.com/kipcole9/money) from 5.12.4 to 5.15.0.
- [Release notes](https://github.com/kipcole9/money/releases)
- [Changelog](https://github.com/kipcole9/money/blob/master/CHANGELOG.md)
- [Commits](https://github.com/kipcole9/money/compare/v5.12.4...v5.15.0)

---
updated-dependencies:
- dependency-name: ex_money
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Vini Brasil <vini@hey.com>
2023-07-24 10:31:09 +01:00
hq1
e6996ccfda
Fix weekly/monthly report e-mails (MJML) (#3150)
* Render weekly/monthly reports with MJML

* Update changelog

* Fix APP-2NA (#3119)
2023-07-17 16:59:01 +02:00
dependabot[bot]
19914112c4
Bump floki from 0.32.1 to 0.34.3 (#3135)
Bumps [floki](https://github.com/philss/floki) from 0.32.1 to 0.34.3.
- [Release notes](https://github.com/philss/floki/releases)
- [Changelog](https://github.com/philss/floki/blob/main/CHANGELOG.md)
- [Commits](https://github.com/philss/floki/compare/v0.32.1...v0.34.3)

---
updated-dependencies:
- dependency-name: floki
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-13 13:19:40 +01:00
dependabot[bot]
8058a06d2d
Bump phoenix_live_view from 0.19.0 to 0.19.4 (#3131)
Bumps [phoenix_live_view](https://github.com/phoenixframework/phoenix_live_view) from 0.19.0 to 0.19.4.
- [Changelog](https://github.com/phoenixframework/phoenix_live_view/blob/main/CHANGELOG.md)
- [Commits](https://github.com/phoenixframework/phoenix_live_view/compare/v0.19.0...v0.19.4)

---
updated-dependencies:
- dependency-name: phoenix_live_view
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-12 10:24:27 +02:00
dependabot[bot]
4615866c13
Bump phoenix_live_reload from 1.3.3 to 1.4.1 (#2849)
Bumps [phoenix_live_reload](https://github.com/phoenixframework/phoenix_live_reload) from 1.3.3 to 1.4.1.
- [Release notes](https://github.com/phoenixframework/phoenix_live_reload/releases)
- [Changelog](https://github.com/phoenixframework/phoenix_live_reload/blob/main/CHANGELOG.md)
- [Commits](https://github.com/phoenixframework/phoenix_live_reload/compare/v1.3.3...v1.4.1)

---
updated-dependencies:
- dependency-name: phoenix_live_reload
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-11 13:59:31 +02:00
ruslandoga
7e2627ae4c
array join (#3021) 2023-07-11 12:43:53 +02:00
Vini Brasil
e4d4f7d954
Revenue tracking: Ingestion and breakdown queries (#2957)
* Add revenue fields to ClickHouse events

This commit adds 4 fields to the ClickHouse events_v2 table:

* `revenue_source_amount` and `revenue_source_currency` store revenue in
  the original currency sent during ingestion

* `revenue_reporting_amount` and `revenue_reporting_currency` store
  revenue in a common currency to perform calculations, and this
  currency is defined by the user when setting up the goal

The type of amount fields is `Nullable(Decimal64(3))`. That covers all
fiat currencies and allows us to store huge amounts. Even though
ClickHouse does not suggest using `Nullable`, this is a good use case,
because otherwise additional work would have to be done to
differentiate missing values from real zeroes.

I ran a benchmark with the data pattern we expect in production, where
we have more missing values than real decimals. I created 100 million
records where 90% of decimals are missing. The difference between the
tables in storage is just 0.4Mb.

* Add revenue parameter to Events API

This commit adds support for sending revenue data in ingestion using the
`revenue` parameter - aliased to `$`.

* Add revenue parameter to mix send_pageview

* Add average and total revenue to breakdown queries
2023-06-12 18:29:17 +01:00
hq1
49ca6b859a
Prepare base for the funnel PRs merge (#2993)
* Update dependencies as per the funnels branch

Co-authored-by: Uku Taht <uku.taht@gmail.com>

* Mute sigil_E deprecation errors

* Use Phoenix.Component in PlausbileWeb

* Add tooltip to react/no-unknown-property whitelist

---------

Co-authored-by: Uku Taht <uku.taht@gmail.com>
Co-authored-by: Vinicius Brasil <vini@hey.com>
2023-06-01 15:26:23 +02:00
ruslandoga
447ed12155
use ecto_ch (#2890)
* use ecto_ch

* use Ch type in BoolUInt8

* update ecto_ch and ch to latest versions
2023-05-25 09:26:50 +02:00
Vini Brasil
10d9e3b083
Revenue tracking: Add currency field to goal creation (#2948)
* Add revenue goal option to goal creation

This commit adds a currency field to the goals form. Goals that have a
currency set are now revenue goals, and are cached with sites to later
be used during ingestion.

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>

* Enable feature flag in tests

---------

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
2023-05-23 12:08:09 +02:00
RobertJoonas
d161c1be0b
Upgrade phoenix (#2902)
* upgrade phoenix

Co-authored-by: Vini Brasil <vini@hey.com>

* fix a test (flash message)

The flash message in focus.html.eex was not covered by any test. This
commit fixes also fixes that.

* change function name

* remove unnecessary formatter and format

* update CI cache

* fix dialyzer error

---------

Co-authored-by: Vini Brasil <vini@hey.com>
2023-05-09 11:51:35 +03:00
dependabot[bot]
ef73816bb5
Bump fun_with_flags_ui from 0.8.0 to 0.8.1 (#2837)
Bumps [fun_with_flags_ui](https://github.com/tompave/fun_with_flags_ui) from 0.8.0 to 0.8.1.
- [Release notes](https://github.com/tompave/fun_with_flags_ui/releases)
- [Changelog](https://github.com/tompave/fun_with_flags_ui/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tompave/fun_with_flags_ui/compare/v0.8.0...v0.8.1)

---
updated-dependencies:
- dependency-name: fun_with_flags_ui
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-13 12:42:26 +02:00
dependabot[bot]
2df7d25df3
Bump ex_doc from 0.28.5 to 0.29.4 (#2832)
Bumps [ex_doc](https://github.com/elixir-lang/ex_doc) from 0.28.5 to 0.29.4.
- [Release notes](https://github.com/elixir-lang/ex_doc/releases)
- [Changelog](https://github.com/elixir-lang/ex_doc/blob/main/CHANGELOG.md)
- [Commits](https://github.com/elixir-lang/ex_doc/compare/v0.28.5...v0.29.4)

---
updated-dependencies:
- dependency-name: ex_doc
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-07 11:12:45 -03:00
hq1
509c549c01
Update ua_inspector database (#2827)
* Update `ua_inspector` database

* Add changelog entry
2023-04-06 12:45:19 +02:00
dependabot[bot]
d069d0a164
Bump hammer from 6.0.0 to 6.1.0 (#2824)
Bumps [hammer](https://github.com/ExHammer/hammer) from 6.0.0 to 6.1.0.
- [Release notes](https://github.com/ExHammer/hammer/releases)
- [Changelog](https://github.com/ExHammer/hammer/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ExHammer/hammer/compare/v6.0.0...v6.1.0)

---
updated-dependencies:
- dependency-name: hammer
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-06 10:41:11 +02:00
dependabot[bot]
4edf126196
Bump bcrypt_elixir from 2.3.1 to 3.0.1 (#2391)
Bumps [bcrypt_elixir](https://github.com/riverrun/bcrypt_elixir) from 2.3.1 to 3.0.1.
- [Release notes](https://github.com/riverrun/bcrypt_elixir/releases)
- [Changelog](https://github.com/riverrun/bcrypt_elixir/blob/master/CHANGELOG.md)
- [Commits](https://github.com/riverrun/bcrypt_elixir/commits)

---
updated-dependencies:
- dependency-name: bcrypt_elixir
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-04-05 12:00:13 +02:00
ruslandoga
66be99b38a
update chto for missing source in telemetry fix (#2787) 2023-03-28 07:06:03 +02:00
hq1
e8509203a2
Update location dependency (faster startup) (#2793)
* Update location dependency (faster startup)

* Update with https://github.com/plausible/location/pull/6
2023-03-27 16:56:34 +02:00
hq1
d2f2c69387
Conditionally support switching between v1 and v2 clickhouse schemas (#2780)
* Remove ClickhouseSetup module

This has been an implicit point of contact to many
tests. From now on the goal is for each test to maintain
its own, isolated setup so that no accidental clashes
and implicit assumptions are relied upon.

* Implement v2 schema check

An environment variable V2_MIGRATION_DONE acts like
a feature flag, switching plausible from using old events/sessions
schemas to v2 schemas introduced by NumericIDs migration.

* Run both test suites sequentially

While the code for v1 and v2 schemas must be kept still,
we will from now on run tests against both code paths.
Secondary test run will set V2_MIGRATION_DONE=1 variable,
thus making all `Plausible.v2?()` checks return `true'.

* Remove unused function

This is a remnant from the short period when
we would check for existing events before allowing
creating a new site.

* Update test setups/factories with v2 migration check

* Make GateKeeper return site id along with :allow

* Make Billing module check for v2 schema

* Make ingestion aware of v2 schema

* Disable site transfers for when v2 is live

In a separate changeset we will implement simplified
site transfer for when v2 migration is complete.
The new transfer will only rename the site domain in postgres
and keep track of the original site prior to the transfer
so we keep an ingestion grace period until the customers
redeploy their scripting.

* Make Stats base queries aware of v2 schema switch

* Update breakdown with v2 conditionals

* Update pageview local start with v2 check

* Update current visitoris with v2 check

* Update stats controller with v2 checks

* Update external controller with v2 checks

* Update remaining tests with proper fixtures

* Rewrite redundant assignment

* Remove unused alias

* Mute credo, this is not the right time

* Add test_helper prompt

* Fetch priv dir so it works with a release

* Fetch distinct partitions only

* Don't limit inspect output for partitions

* Ensure SQL is printed to IO

* Remove redundant domain fixture
2023-03-27 13:52:42 +02:00
ruslandoga
a129bf82e0
use github: plausible/chto for chto (#2782)
* use github: plausible/chto for chto

* use chto with correct ch github reference (plausible/ch)
2023-03-22 14:56:57 +01:00
Adam
6d79ca5093
Switch to new clickhouse adapter (ch/chto) (#2733)
* another clickhouse adapter

* don't restore stats_removal.ex

* fix events main-graph error (#2746)

* update ch, chto

* update chto again (#2759)

* Stop treating page filter as an entry_page filter (#2752)

* remove dead code

* stop treating page filter as entry page filter in breakdown queries

* stop treating page filter as entry page filter in aggregate queries

* stop treating page filter as entry page filter in timeseries queries

* mix format

* update changelog

* break code down to smaller functions to keep credo happy

* remove unused functions

* make CSV export return only conversions with goal filter (#2760)

* make CSV export return only conversions with goal filter

* update changelog

* update elixir version in mix.exs (#2742)

* revert admin.ex changes (#2776)

---------

Co-authored-by: ruslandoga <67764432+ruslandoga@users.noreply.github.com>
Co-authored-by: ruslandoga <rusl@n-do.ga>
Co-authored-by: RobertJoonas <56999674+RobertJoonas@users.noreply.github.com>
2023-03-21 09:55:59 +01:00
Adam
de0f68a647
Pin merged deps after #2693 (#2710) 2023-02-27 15:23:05 +01:00
Adam Rutkowski
867dad6da7
Implement ingest counters (#2693)
* Clickhouse migration: add ingest_counters table

* Configure ingest counters per MIX_ENV

* Emit telemetry for ingest events with rich metadata

* Allow building Request.t() with fake now() - for testing purposes

* Use clickhousex branch where session_id is assigned to each connection

* Add helper function for getting site id via cache

* Add Ecto schema for `ingest_counters` table

* Implement metrics buffer

* Implement buffering handler for `Plausible.Ingestion.Event` telemetry

* Implement periodic metrics aggregation

* Update counters docs

* Add toStartOfMinute() to ordering key

* Reset the sync connection state in `after` clause

* Flush counters on app termination

* Use separate Repo with async settings enabled at config level

* Switch to clickhouse_settings repo root config key

* Add AsyncInsertRepo module
2023-02-23 14:34:24 +01:00
Vini Brasil
1cb07efe6d
Save city name when importing from GA (#2608)
This commit adds city data to imported records from Google Analytics. The
current implementation sets city to 0 because GA does not use the GeoNames
database.

Google Analytics Reporting API uses [Geographical IDs](https://developers.google.com/analytics/devguides/collection/protocol/v1/geoid)
to identify cities and countries. Plausible uses
[GeoNames](https://geonames.org/) and I couldn't find databases corelating the
two.

Fortunately, GA also returns the city name and this commit uses the city name
and the country ISO code to find the Geoname ID. To avoid making expensive ETS
searches, I created another ETS table in the Location library that uses
{country, city} as a key.

Related PR: https://github.com/plausible/location/pull/3
2023-02-14 09:32:18 -03:00
Adam Rutkowski
912c9374d4
Bump clickhousex (per-query timeout override) (#2645)
https://github.com/plausible/clickhousex/pull/21
2023-02-06 12:30:39 +01:00
ruslandoga
166748dcf2
Replace Geolix with Locus (#2362)
This PR replaces geolix with locus to simplify self-hosted setup. locus can auto-update maxmind dbs which are recommended for self-hosters if they want city-level geolocation. locus is also a bit faster.

This PR also uses a test mmdb file from https://github.com/maxmind/MaxMind-DB for e2e geolocation tests without stubs.
2023-01-17 12:05:09 -03:00
Adam Rutkowski
97a3ce72e4
Update clickhousex & fingerprint sentry errors (#2587)
* Revert "Revert "Fingerprint Clickhousex.Error exceptions in Sentry (#2578)" (#2586)"

This reverts commit a78e07236a.

* Bump clickhousex to include https://github.com/plausible/clickhousex/pull/20
2023-01-12 17:22:45 +01:00
Adam Rutkowski
a78e07236a
Revert "Fingerprint Clickhousex.Error exceptions in Sentry (#2578)" (#2586)
This reverts commit 2428a2bf36.
2023-01-12 06:49:41 +01:00
Adam Rutkowski
2428a2bf36
Fingerprint Clickhousex.Error exceptions in Sentry (#2578)
* Fingerprint Clickhousex.Error exceptions in Sentry

* Update clickhousex
2023-01-11 15:19:07 +01:00
ruslandoga
a6e75ada7b
update ecto to include db url scheme (#2570)
* update ecto to include db url scheme

* update clickhousex

* update changelog
2023-01-10 13:02:28 +01:00
Adam Rutkowski
89da676215
Update clickhousex (start using Mint and proper pool management) (#2502)
* Pin clickhousex to @ruslandoga's rebase branch

* Bump lockfile

* Use HTTP pings

* Update lockfile

* Pin to master
2023-01-05 09:38:52 +01:00