analytics/lib/plausible_web/router.ex

287 lines
12 KiB
Elixir
Raw Normal View History

2019-09-02 14:29:19 +03:00
defmodule PlausibleWeb.Router do
use PlausibleWeb, :router
import Phoenix.LiveView.Router
2019-09-02 14:29:19 +03:00
@two_weeks_in_seconds 60 * 60 * 24 * 14
pipeline :browser do
plug :accepts, ["html"]
plug :fetch_session
2023-05-15 15:27:42 +03:00
plug :fetch_live_flash
2019-09-02 14:29:19 +03:00
plug :put_secure_browser_headers
plug PlausibleWeb.FirstLaunchPlug, redirect_to: "/register"
2019-09-02 14:29:19 +03:00
plug PlausibleWeb.SessionTimeoutPlug, timeout_after_seconds: @two_weeks_in_seconds
plug PlausibleWeb.AuthPlug
plug PlausibleWeb.LastSeenPlug
end
pipeline :shared_link do
plug :accepts, ["html"]
plug :put_secure_browser_headers
end
pipeline :csrf do
plug :protect_from_forgery
end
2019-09-02 14:29:19 +03:00
pipeline :api do
2019-12-03 12:42:17 +03:00
plug :accepts, ["json"]
2019-09-02 14:29:19 +03:00
plug :fetch_session
plug PlausibleWeb.AuthPlug
end
pipeline :internal_stats_api do
plug :accepts, ["json"]
plug :fetch_session
plug PlausibleWeb.AuthorizeSiteAccess
end
pipeline :public_api do
plug :accepts, ["json"]
end
pipeline :flags do
2022-04-21 10:54:08 +03:00
plug :accepts, ["html"]
plug :put_secure_browser_headers
plug :fetch_session
plug PlausibleWeb.CRMAuthPlug
2022-04-21 10:54:08 +03:00
end
if Mix.env() == :dev do
2019-09-02 14:29:19 +03:00
forward "/sent-emails", Bamboo.SentEmailViewerPlug
end
2021-01-07 11:42:45 +03:00
use Kaffy.Routes, scope: "/crm", pipe_through: [PlausibleWeb.CRMAuthPlug]
scope path: "/flags" do
pipe_through :flags
forward "/", FunWithFlags.UI.Router, namespace: "flags"
2022-04-21 10:54:08 +03:00
end
scope "/api/stats", PlausibleWeb.Api do
pipe_through :internal_stats_api
get "/:domain/current-visitors", StatsController, :current_visitors
get "/:domain/main-graph", StatsController, :main_graph
Adds Main Graph Metric Selection (#1364) * First pass bringing in previous graph improvements, and comparsion context * Swaps issue template to new issue form syntax * Indentation update * Indentation update? * More indentation * Intendation is hard * Finalized indentation? * Github indentation * Missing fields * Formatting changes * Checkbox changes * Uses new timeseries API, various UI improvements, descopes conversions, ToP from graphing * Fixes Mobile UI Issues * Improves point detection and display on hover * Fixes & adds tests for updated main-graph API route * Changelog * Changes to better metric option declaration & minor UI/default fixes * Fixes top stat tooltips showing unformatted numbers for special (non-rounded) top stats * Formatting * Fixes regression with dashed portion not stopping at present_index * Removes comparison + lint * Improves top stat active style * Removes comparison tests * Splits out tooltip and top stats Still needs: - Tests - Potentially more cleanup * Adds/moves tests for top stats * Formatting * Updates metric LS key, removes console log * Various fixes + cleanup * Makes tooltip position & style more consistent * Fixes test (returns import status on both main graph & top stats) * Fixes interaction with month dateFormatter * Fixes edge case tooltip behavior It was simpler than I thought :/ * Make the entire top stat clickable * Minor UI improvements * Fixes another tooltip visibility edge case + cleans up boolean algebra Co-authored-by: Uku Taht <Uku.taht@gmail.com>
2022-04-13 10:38:47 +03:00
get "/:domain/top-stats", StatsController, :top_stats
get "/:domain/sources", StatsController, :sources
get "/:domain/utm_mediums", StatsController, :utm_mediums
get "/:domain/utm_sources", StatsController, :utm_sources
get "/:domain/utm_campaigns", StatsController, :utm_campaigns
get "/:domain/utm_contents", StatsController, :utm_contents
get "/:domain/utm_terms", StatsController, :utm_terms
get "/:domain/referrers/:referrer", StatsController, :referrer_drilldown
get "/:domain/pages", StatsController, :pages
get "/:domain/entry-pages", StatsController, :entry_pages
get "/:domain/exit-pages", StatsController, :exit_pages
get "/:domain/countries", StatsController, :countries
get "/:domain/regions", StatsController, :regions
get "/:domain/cities", StatsController, :cities
get "/:domain/browsers", StatsController, :browsers
get "/:domain/browser-versions", StatsController, :browser_versions
get "/:domain/operating-systems", StatsController, :operating_systems
get "/:domain/operating-system-versions", StatsController, :operating_system_versions
get "/:domain/screen-sizes", StatsController, :screen_sizes
get "/:domain/conversions", StatsController, :conversions
2020-10-30 11:49:41 +03:00
get "/:domain/property/:prop_name", StatsController, :prop_breakdown
Adds manual, editable, auto-suggested filters, and negated&globbed path-based filters (#1121) * Adds manual-filters + friends commit 308192044d726e9a6f7406e333048ab3407aa260 Merge: 39287ab a299fab Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Fri Jun 11 03:40:55 2021 -0500 Merge branch 'manual-filters' into manual-filters-2 commit a299fab1fbe98ec04ea3dfbfbd175a2af5e73804 Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Fri Jun 11 02:20:26 2021 -0500 Changes to split and pattern matched function for time_on_page commit 10f10c933af56a3482747f818bd580a8a1903ae2 Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Fri Jun 11 01:53:18 2021 -0500 Fixes a couple of minor UX issues commit f2e5ce8eb33460cb2d1dc9ac0bd0d4ad0a20aa31 Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Fri Jun 11 01:49:10 2021 -0500 Fixes time on page for globbed and negated page paths commit bb18af6526068860e4090dc91d02455e515f1238 Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Thu Jun 10 05:24:05 2021 -0500 Close to finalized version of updated version Just needs some additional testing + potentially code cleanup commit d0b7bfe3dacb4c611e6857bed308b5954aa2cc1e Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Fri May 28 04:21:21 2021 -0500 Real Dialyzer Fix commit 296a76af34ff7e16c83e5b5bf62f0547dc30ece9 Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Fri May 28 03:44:29 2021 -0500 Dialyzer fix commit 91f3b44017afb3838bbb4de8ec9f3ba4d39709bc Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Thu May 27 17:11:24 2021 -0500 Changelog commit e041f75745ab57b0d3c00d4a9ff7f8a490b27c17 Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Thu May 27 17:09:52 2021 -0500 Formatting commit f689642204b02f6544dc072b95c9c22e180b9e87 Merge: e00929b 4ff25f6 Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Thu May 27 17:08:47 2021 -0500 Merge branch 'master' into manual-filters commit e00929b1c51411fe4bf939266d3aeb6f67dcb2ab Merge: 83887c4 806975e Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Thu May 27 17:01:48 2021 -0500 Merge branch 'manual-filters' of github.com:Vigasaurus/plausible-analytics into manual-filters commit 83887c49ccc7fbc50ba1766bfa74c77994ef1900 Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Thu May 27 16:59:32 2021 -0500 Adds tests for suggestions, formats goals suggestion query commit 1cb7732a08a50eddb81166fb796bf5731a0c5aba Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Thu May 27 16:36:31 2021 -0500 Adds goals as auto-complete capable filter commit 4ca39cc406777794bab7586116115031c6d8a82a Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Thu May 27 16:08:22 2021 -0500 95% completed auto-complete setup Still needs: - tests - goals added as filter commit 22d38c4cd83dd1d72ec435e1b8cf6fc3eb70d957 Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Thu May 27 03:47:19 2021 -0500 80% of auto-complete filters progress Still needs - countries and screen into new format - re-style dropdown and background - drop debounce time - tests commit 806975ede9ebcbaa03ce888c41f20da866413697 Merge: 81c5e05 1a93542 Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Tue May 25 15:28:21 2021 -0500 Merge branch 'master' into manual-filters commit 81c5e05760bf7f5d6fc05290e662ab61f9a4b73b Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Tue May 25 15:21:03 2021 -0500 Makes colorings on top bar elements consistent commit fa7f6c22680f5f00b89c7ea99032b409a5f8bbd2 Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Tue May 25 14:58:25 2021 -0500 Makes requested changes, adds different version of filter button commit 7dc65b9b7a5a9678499a7c404109e710ae7da36d Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Sat May 22 04:29:01 2021 -0500 Changelog commit c684f1c76a2599324c19ca32de10b2f2849fd5e9 Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Sat May 22 04:26:14 2021 -0500 Various UI Improvements - Makes edit buttons full-length & properly sized - Adds remove filter button in edit menu commit a632e7a8eea4a9897052cb08d3f7605403c227f7 Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Sat May 22 03:11:50 2021 -0500 Adds tests for exclusions and wildcards commit eb91a7942b662d412511a4c47501a9b6d7f2d625 Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Sat May 22 03:02:23 2021 -0500 Fixes editing UX on list view commit 6209d72aeede6aa8ce4f8e009d2ea168bb8f77a7 Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Fri May 21 04:01:17 2021 -0500 Bugfix in realtime view, formatting commit 007d44df38dba254154b3450aacec9dd253ccd95 Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Fri May 21 03:23:16 2021 -0500 Second pass - mostly everything user-facing is done Still needs: - Tests - Potentially negating other filters - Potentially some code cleanup commit cb7b5b9fbd83f618cf4ebbe6582a9202b4989033 Author: Vignesh Joglekar <rexvigasaurus@gmail.com> Date: Fri May 21 01:49:52 2021 -0500 First pass on manual filter & path regex/negated filters Still needs: - Form structure on filter modal - Edit filter button - Filter dropdown UI improvement - Filter modal mount data collection - Tests - Potentially negating other filters * Fixes an issue between the updated React version and Flatpickr not showing up Reverting to v16 was the simplest fix without diving into the issue upstream with Flatpickr * Resolves comments on UI and code style Co-authored-by: Uku Taht <Uku.taht@gmail.com>
2021-06-21 14:42:16 +03:00
get "/:domain/suggestions/:filter_name", StatsController, :filter_suggestions
get "/:domain/funnels/:id", StatsController, :funnel
end
scope "/api/v1/stats", PlausibleWeb.Api do
pipe_through [:public_api, PlausibleWeb.AuthorizeStatsApiPlug]
get "/realtime/visitors", ExternalStatsController, :realtime_visitors
get "/aggregate", ExternalStatsController, :aggregate
get "/breakdown", ExternalStatsController, :breakdown
get "/timeseries", ExternalStatsController, :timeseries
end
scope "/api/v1/sites", PlausibleWeb.Api do
pipe_through [:public_api, PlausibleWeb.AuthorizeSitesApiPlug]
post "/", ExternalSitesController, :create_site
2021-04-15 11:38:44 +03:00
put "/shared-links", ExternalSitesController, :find_or_create_shared_link
put "/goals", ExternalSitesController, :find_or_create_goal
delete "/goals/:goal_id", ExternalSitesController, :delete_goal
get "/:site_id", ExternalSitesController, :get_site
put "/:site_id", ExternalSitesController, :update_site
delete "/:site_id", ExternalSitesController, :delete_site
end
2019-09-02 14:29:19 +03:00
scope "/api", PlausibleWeb do
pipe_through :api
2019-10-31 06:49:46 +03:00
post "/event", Api.ExternalController, :event
2019-09-02 14:29:19 +03:00
get "/error", Api.ExternalController, :error
2020-07-20 10:34:35 +03:00
get "/health", Api.ExternalController, :health
2022-05-27 16:01:42 +03:00
get "/system", Api.ExternalController, :info
2019-09-02 14:29:19 +03:00
post "/paddle/webhook", Api.PaddleController, :webhook
get "/:domain/status", Api.InternalController, :domain_status
get "/sites", Api.InternalController, :sites
2019-09-02 14:29:19 +03:00
end
scope "/", PlausibleWeb do
pipe_through [:browser, :csrf]
2019-09-02 14:29:19 +03:00
get "/register", AuthController, :register_form
post "/register", AuthController, :register
get "/register/invitation/:invitation_id", AuthController, :register_from_invitation_form
post "/register/invitation/:invitation_id", AuthController, :register_from_invitation
get "/activate", AuthController, :activate_form
post "/activate/request-code", AuthController, :request_activation_code
post "/activate", AuthController, :activate
2019-09-02 14:29:19 +03:00
get "/login", AuthController, :login_form
post "/login", AuthController, :login
get "/password/request-reset", AuthController, :password_reset_request_form
post "/password/request-reset", AuthController, :password_reset_request
get "/password/reset", AuthController, :password_reset_form
post "/password/reset", AuthController, :password_reset
post "/error_report", ErrorReportController, :submit_error_report
end
scope "/", PlausibleWeb do
pipe_through [:shared_link]
get "/share/:domain", StatsController, :shared_link
post "/share/:slug/authenticate", StatsController, :authenticate_shared_link
end
scope "/", PlausibleWeb do
pipe_through [:browser, :csrf]
2019-09-02 14:29:19 +03:00
get "/password", AuthController, :password_form
post "/password", AuthController, :set_password
get "/logout", AuthController, :logout
2019-09-02 14:29:19 +03:00
get "/settings", AuthController, :user_settings
put "/settings", AuthController, :save_settings
delete "/me", AuthController, :delete_me
get "/settings/api-keys/new", AuthController, :new_api_key
post "/settings/api-keys", AuthController, :create_api_key
delete "/settings/api-keys/:id", AuthController, :delete_api_key
2019-09-02 14:29:19 +03:00
get "/auth/google/callback", AuthController, :google_auth_callback
get "/", PageController, :index
get "/billing/change-plan", BillingController, :change_plan_form
get "/billing/change-plan/preview/:plan_id", BillingController, :change_plan_preview
post "/billing/change-plan/:new_plan_id", BillingController, :change_plan
2019-09-02 14:29:19 +03:00
get "/billing/upgrade", BillingController, :upgrade
2021-03-16 13:03:50 +03:00
get "/billing/upgrade/:plan_id", BillingController, :upgrade_to_plan
2021-10-20 17:49:11 +03:00
get "/billing/upgrade/enterprise/:plan_id", BillingController, :upgrade_enterprise_plan
get "/billing/change-plan/enterprise/:plan_id", BillingController, :change_enterprise_plan
get "/billing/upgrade-success", BillingController, :upgrade_success
get "/billing/subscription/ping", BillingController, :ping_subscription
2019-09-02 14:29:19 +03:00
2020-03-31 16:16:21 +03:00
get "/sites", SiteController, :index
2019-09-02 14:29:19 +03:00
get "/sites/new", SiteController, :new
post "/sites", SiteController, :create_site
get "/sites/:website/change-domain", SiteController, :change_domain
put "/sites/:website/change-domain", SiteController, :change_domain_submit
get "/:website/change-domain-snippet", SiteController, :add_snippet_after_domain_change
2019-09-02 14:29:19 +03:00
post "/sites/:website/make-public", SiteController, :make_public
post "/sites/:website/make-private", SiteController, :make_private
2019-09-09 14:37:57 +03:00
post "/sites/:website/weekly-report/enable", SiteController, :enable_weekly_report
post "/sites/:website/weekly-report/disable", SiteController, :disable_weekly_report
post "/sites/:website/weekly-report/recipients", SiteController, :add_weekly_report_recipient
Formatting only changes - No code change (#75) * first commit with test and compile job Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * adding 'prepare' stage Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * updated ci script to include "test" compile phase Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * adding environment variables for connecting to postgresql Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * updated ci config for postgres Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * using non-alpine version of elixir Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * re-using the 'compile' artifacts and added explict env variables for testing Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * removing redundant deps fetching from common code Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * formatting using mix.format -- beware no-code changes! Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * added release config Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * adding consistent env variable for Database Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * more cleaning up of environment variables Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Adding releases config for enabling releases Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * cleaning up env configs Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Cleaned up config and prepared config for releases Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * updated CI script with new config for test Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Added Dockerfile for creating production docker image Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Adding "docker" build job yay! Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * using non-slim version of debian and installing webpack Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Adding overlays for migrations on releases Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * restricting the docker built to master branch only Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * typo fix Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * adding "Hosting.md" to explain hosting instructions Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * removed the default comments Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Added documentation related to env variables * updated documentation and fixed typo Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * updated documentation * Bumping up elixir version as `overlays` are only supported in latest version read release notes: https://github.com/elixir-lang/elixir/releases/tag/v1.10.0 Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Adding tarball assembly during release Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * updated HOSTING.md Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Added support for db migration Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * minor corrections Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * initializing admin user Admin user has been added in the "migration" phase. A default user is automatically created in the process. One can provide the related env variables, else a new one will be automatically created for you. Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Initial base domain update - phase#1 These changes are only meant for correct operating it under self-hosting. There are many other cosmetic changes, that require updates to email, site and other places where the original website and author is used. Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Using dedicated config variable `base_domain` instead Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * adding base_domain to releases config Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * removing the dedicated config "base_domain", relying on endpoint host Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Removed the usage of "Mix" in code! It is bad practice to use "mix" module inside the code as in actual release this module is unavailable. Replacing this with a config environment variable Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Added support for SMTP via Bamboo Smtp Adapter Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Capturing SMTP errors via Sentry Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Minor updates Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Adding junit formatter -- useful for generating test reports Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * adding documentation for default user * Resolve "Gitlab Adoption: Add supported services in "Security & Compliance"" * bumping up the debian version to fix issues fixing some vulnerabilities identified by the scanning tools * More updates for self-hosting Changes in most of the places to suit self-hosting. Although, there are some which have been left-off. Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * quick-dirty-fix! * bumping up the db connect timeout Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * bumping up the db connect timeout Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * bumping up the db connect timeout Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * bumping up timeout - skipping MRs :-/ * removing restrictions on watching for changes this stuff isn't working * Update HOSTING.md * renamed the module name * reverting formatting-whitespace changes Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * reverting the name to release Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * adding docker-compose.yml and related instructions Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * using `plausible_url` instead of assuming `https` this is because, it is much to test in local dev machines and in most cases there's already a layer above which is capable for `https` termination and http -> https upgrade Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * WIP: merging changes from upstream Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * wip: more changes * Pushing in changes from upstream Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * changes to ci for testing Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * cleaning up and finishing clickhouse integration Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * updating readme with hosting details * removing deleted files from upstream * minor config adjustments Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * formatting changes Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
2020-06-08 10:35:13 +03:00
delete "/sites/:website/weekly-report/recipients/:recipient",
SiteController,
:remove_weekly_report_recipient
2019-09-09 14:37:57 +03:00
post "/sites/:website/monthly-report/enable", SiteController, :enable_monthly_report
post "/sites/:website/monthly-report/disable", SiteController, :disable_monthly_report
Formatting only changes - No code change (#75) * first commit with test and compile job Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * adding 'prepare' stage Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * updated ci script to include "test" compile phase Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * adding environment variables for connecting to postgresql Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * updated ci config for postgres Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * using non-alpine version of elixir Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * re-using the 'compile' artifacts and added explict env variables for testing Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * removing redundant deps fetching from common code Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * formatting using mix.format -- beware no-code changes! Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * added release config Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * adding consistent env variable for Database Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * more cleaning up of environment variables Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Adding releases config for enabling releases Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * cleaning up env configs Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Cleaned up config and prepared config for releases Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * updated CI script with new config for test Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Added Dockerfile for creating production docker image Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Adding "docker" build job yay! Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * using non-slim version of debian and installing webpack Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Adding overlays for migrations on releases Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * restricting the docker built to master branch only Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * typo fix Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * adding "Hosting.md" to explain hosting instructions Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * removed the default comments Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Added documentation related to env variables * updated documentation and fixed typo Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * updated documentation * Bumping up elixir version as `overlays` are only supported in latest version read release notes: https://github.com/elixir-lang/elixir/releases/tag/v1.10.0 Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Adding tarball assembly during release Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * updated HOSTING.md Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Added support for db migration Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * minor corrections Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * initializing admin user Admin user has been added in the "migration" phase. A default user is automatically created in the process. One can provide the related env variables, else a new one will be automatically created for you. Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Initial base domain update - phase#1 These changes are only meant for correct operating it under self-hosting. There are many other cosmetic changes, that require updates to email, site and other places where the original website and author is used. Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Using dedicated config variable `base_domain` instead Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * adding base_domain to releases config Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * removing the dedicated config "base_domain", relying on endpoint host Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Removed the usage of "Mix" in code! It is bad practice to use "mix" module inside the code as in actual release this module is unavailable. Replacing this with a config environment variable Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Added support for SMTP via Bamboo Smtp Adapter Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Capturing SMTP errors via Sentry Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Minor updates Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * Adding junit formatter -- useful for generating test reports Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * adding documentation for default user * Resolve "Gitlab Adoption: Add supported services in "Security & Compliance"" * bumping up the debian version to fix issues fixing some vulnerabilities identified by the scanning tools * More updates for self-hosting Changes in most of the places to suit self-hosting. Although, there are some which have been left-off. Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * quick-dirty-fix! * bumping up the db connect timeout Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * bumping up the db connect timeout Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * bumping up the db connect timeout Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * bumping up timeout - skipping MRs :-/ * removing restrictions on watching for changes this stuff isn't working * Update HOSTING.md * renamed the module name * reverting formatting-whitespace changes Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * reverting the name to release Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * adding docker-compose.yml and related instructions Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * using `plausible_url` instead of assuming `https` this is because, it is much to test in local dev machines and in most cases there's already a layer above which is capable for `https` termination and http -> https upgrade Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * WIP: merging changes from upstream Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * wip: more changes * Pushing in changes from upstream Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * changes to ci for testing Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * cleaning up and finishing clickhouse integration Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * updating readme with hosting details * removing deleted files from upstream * minor config adjustments Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me> * formatting changes Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
2020-06-08 10:35:13 +03:00
post "/sites/:website/monthly-report/recipients",
SiteController,
:add_monthly_report_recipient
delete "/sites/:website/monthly-report/recipients/:recipient",
SiteController,
:remove_monthly_report_recipient
post "/sites/:website/spike-notification/enable", SiteController, :enable_spike_notification
post "/sites/:website/spike-notification/disable", SiteController, :disable_spike_notification
put "/sites/:website/spike-notification", SiteController, :update_spike_notification
post "/sites/:website/spike-notification/recipients",
SiteController,
:add_spike_notification_recipient
delete "/sites/:website/spike-notification/recipients/:recipient",
SiteController,
:remove_spike_notification_recipient
get "/sites/:website/shared-links/new", SiteController, :new_shared_link
post "/sites/:website/shared-links", SiteController, :create_shared_link
get "/sites/:website/shared-links/:slug/edit", SiteController, :edit_shared_link
put "/sites/:website/shared-links/:slug", SiteController, :update_shared_link
delete "/sites/:website/shared-links/:slug", SiteController, :delete_shared_link
2020-06-30 11:00:19 +03:00
delete "/sites/:website/custom-domains/:id", SiteController, :delete_custom_domain
get "/sites/:website/memberships/invite", Site.MembershipController, :invite_member_form
post "/sites/:website/memberships/invite", Site.MembershipController, :invite_member
post "/sites/invitations/:invitation_id/accept", InvitationController, :accept_invitation
post "/sites/invitations/:invitation_id/reject", InvitationController, :reject_invitation
delete "/sites/:website/invitations/:invitation_id", InvitationController, :remove_invitation
get "/sites/:website/transfer-ownership", Site.MembershipController, :transfer_ownership_form
post "/sites/:website/transfer-ownership", Site.MembershipController, :transfer_ownership
put "/sites/:website/memberships/:id/role/:new_role", Site.MembershipController, :update_role
delete "/sites/:website/memberships/:id", Site.MembershipController, :remove_member
get "/sites/:website/weekly-report/unsubscribe", UnsubscribeController, :weekly_report
get "/sites/:website/monthly-report/unsubscribe", UnsubscribeController, :monthly_report
2019-09-02 14:29:19 +03:00
get "/:website/snippet", SiteController, :add_snippet
get "/:website/settings", SiteController, :settings
2020-11-06 14:30:38 +03:00
get "/:website/settings/general", SiteController, :settings_general
get "/:website/settings/people", SiteController, :settings_people
2020-11-19 15:57:55 +03:00
get "/:website/settings/visibility", SiteController, :settings_visibility
2020-11-06 14:30:38 +03:00
get "/:website/settings/goals", SiteController, :settings_goals
2023-04-18 16:09:43 +03:00
2023-05-15 15:27:42 +03:00
get "/:website/settings/funnels", SiteController, :settings_funnels
# post "/:website/settings/funnels", SiteController, :save_funnel
2023-05-16 17:55:28 +03:00
# live "/:website/settings/funnel2", Live.FunnelSettings
2023-04-18 16:09:43 +03:00
2020-11-16 16:38:44 +03:00
get "/:website/settings/search-console", SiteController, :settings_search_console
get "/:website/settings/email-reports", SiteController, :settings_email_reports
get "/:website/settings/custom-domain", SiteController, :settings_custom_domain
2020-11-19 15:57:55 +03:00
get "/:website/settings/danger-zone", SiteController, :settings_danger_zone
2019-10-31 08:39:51 +03:00
get "/:website/goals/new", SiteController, :new_goal
post "/:website/goals", SiteController, :create_goal
2019-10-31 09:20:45 +03:00
delete "/:website/goals/:id", SiteController, :delete_goal
2019-09-02 14:29:19 +03:00
put "/:website/settings", SiteController, :update_settings
put "/:website/settings/google", SiteController, :update_google_auth
[Continued] Google Analytics import (#1753) * Add has_imported_stats boolean to Site * Add Google Analytics import panel to general settings * Get GA profiles to display in import settings panel * Add import_from_google method as entrypoint to import data * Add imported_visitors table * Remove conflicting code from migration * Import visitors data into clickhouse database * Pass another dataset to main graph for rendering in red This adds another entry to the JSON data returned via the main graph API called `imported_plot`, which is similar to `plot` in form but will be completed with previously imported data. Currently it simply returns the values from `plot` / 2. The data is rendered in the main graph in red without fill, and without an indicator for the present. Rationale: imported data will not continue to grow so there is no projection forward, only backwards. * Hook imported GA data to dashboard timeseries plot * Add settings option to forget imported data * Import sources from google analytics * Merge imported sources when queried * Merge imported source data native data when querying sources * Start converting metrics to atoms so they can be subqueried This changes "visitors" and in some places "sources" to atoms. This does not change the behaviour of the functions - the tests all pass unchanged following this commit. This is necessary as joining subqueries requires that the keys in `select` statements be atoms and not strings. * Convery GA (direct) source to empty string * Import utm campaign and utm medium from GA * format * Import all data types from GA into new tables * Handle large amounts of more data more safely * Fix some mistakes in tables * Make GA requests in chunks of 5 queries * Only display imported timeseries when there is no filter * Correctly show last 30 minutes timeseries when 'realtime' * Add with_imported key to Query struct * Account for injected :is_not filter on sources from dashboard * Also add tentative imported_utm_sources table This needs a bit more work on the google import side, as GA do not report sources and utm sources as distinct things. * Return imported data to dashboard for rest of Sources panel This extends the merge_imported function definition for sources to utm_sources, utm_mediums and utm_campaigns too. This appears to be working on the DB side but something is incomplete on the client side. * Clear imported stats from all tables when requested * Merge entry pages and exit pages from imported data into unfiltered dashboard view This requires converting the `"visits"` and `"visit_duration"` metrics to atoms so that they can be used in ecto subqueries. * Display imported devices, browsers and OSs on dashboard * Display imported country data on dashboard * Add more metrics to entries/exits for modals * make sure data is returned via API with correct keys * Import regions and cities from GA * Capitalize device upon import to match native data * Leave query limits/offsets until after possibly joining with imported data * Also import timeOnPage and pageviews for pages from GA * imported_countries -> imported_locations * Get timeOnPage and pageviews for pages from GA These are needed for the pages modal, and for calculating exit rates for exit pages. * Add indicator to dashboard when imported data is being used * Don't show imported data as separately line on main graph * "bounce_rate" -> :bounce_rate, so it works in subqueries * Drop imported browser and OS versions These are not needed. * Toggle displaying imported data by clicking indicator * Parse referrers with RefInspector - Use 'ga:fullReferrer' instead of 'ga:source'. This provides the actual referrer host + path, whereas 'ga:source' includes utm_mediums and other values when relevant. - 'ga:fullReferror' does however include search engine names directly, so they are manually checked for as RefInspector won't pick up on these. * Keep imported data indicator on dashboard and strikethrough when hidden * Add unlink google button to import panel * Rename some GA browsers and OSes to plausible versions * Get main top pages and exit pages panels working correctly with imported data * mix format * Fetch time_on_pages for imported data when needed * entry pages need to fetch bounces from GA * "sample_percent" -> :sample_percent as only atoms can be used in subqueries * Calculate bounce_rate for joined native and imported data for top pages modal * Flip some query bindings around to be less misleading * Fixup entry page modal visit durations * mix format * Fetch bounces and visit_duration for sources from GA * add more source metrics used for data in modals * Make sources modals display correct values * imported_visitors: bounce_rate -> bounces, avg_visit_duration -> visit_duration * Merge imported data into aggregate stats * Reformat top graph side icons * Ensure sample_percent is yielded from aggregate data * filter event_props should be strings * Hide imported data from frontend when using filter * Fix existing tests * fix tests * Fix imported indicator appearing when filtering * comma needed, lost when rebasing * Import utm_terms and utm_content from GA * Merge imported utm_term and utm_content * Rename imported Countries data as Locations * Set imported city schema field to int * Remove utm_terms and utm_content when clearing imported * Clean locations import from Google Analytics - Country and region should be set to "" when GA provides "(not set)" - City should be set to 0 for "unknown", as we cannot reliably import city data from GA. * Display imported region and city in dashboard * os -> operating_system in some parts of code The inconsistency of using os in some places and operating_system in others causes trouble with subqueries and joins for the native and imported data, which would require additional logic to account for. The simplest solution is the just use a consistent word for all uses. This doesn't make any user-facing or database changes. * to_atom -> to_existing_atom * format * "events" metric -> :events * ignore imported data when "events" in metrics * update "bounce_rate" * atomise some more metrics from new city and region api * atomise some more metrics for email handlers * "conversion_rate" -> :conversion_rate during csv export * Move imported data stats code to own module * Move imported timeseries function to Stats.Imported * Use Timex.parse to import dates from GA * has_imported_stats -> imported_source * "time_on_page" -> :time_on_page * Convert imported GA data to UTC * Clean up GA request code a bit There was some weird logic here with two separate lists that really ought to be together, so this merges those. * Fail sooner if GA timezone can't be identified * Link imported tables to site by id * imported_utm_content -> imported_utm_contents * Imported GA from all of time * Reorganise GA data fetch logic - Fetch data from the start of time (2005) - Check whether no data was fetched, and if so, inform user and don't consider data to be imported. * Clarify removal of "visits" data when it isn't in metrics * Apply location filters from API This makes it consistent with the sources etc which filter out 'Direct / None' on the API side. These filters are used by both the native and imported data handling code, which would otherwise both duplicate the filters in their `where` clauses. * Do not use changeset for setting site.imported_source * Add all metrics to all dimensions * Run GA import in the background * Send email when GA import completes * Add handler to insert imported data into tests and imported_browsers_factory * Add remaining import data test factories * Add imported location data to test * Test main graph with imported data * Add imported data to operating systems tests * Add imported data to pages tests * Add imported data to entry pages tests * Add imported data to exit pages tests * Add imported data to devices tests * Add imported data to sources tests * Add imported data to UTM tests * Add new test module for the data import step * Test import of sources GA data * Test import of utm_mediums GA data * Test import of utm_campaigns GA data * Add tests for UTM terms * Add tests for UTM contents * Add test for importing pages and entry pages data from GA * Add test for importing exit page data * Fix module file name typo * Add test for importing location data from GA * Add test for importing devices data from GA * Add test for importing browsers data from GA * Add test for importing OS data from GA * Paginate GA requests to download all data * Bump clickhouse_ecto version * Move RefInspector wrapper function into module * Drop timezone transform on import * Order imported by side_id then date * More strings -> atoms Also changes a conditional to be a bit nicer * Remove parallelisation of data import * Split sources and UTM sources from fetched GA data GA has only a "source" dimension and no "UTM source" dimension. Instead it returns these combined. The logic herein to tease these apart is: 1. "(direct)" -> it's a direct source 2. if the source is a domain -> it's a source 3. "google" -> it's from adwords; let's make this a UTM source "adwords" 4. else -> just a UTM source * Keep prop names in queries as strings * fix typo * Fix import * Insert data to clickhouse in batches * Fix link when removing imported data * Merge source tables * Import hostname as well as pathname * Record start and end time of imported data * Track import progress * Fix month interval with imported data * Do not JOIN when imported date range has no overlap * Fix time on page using exits Co-authored-by: mcol <mcol@posteo.net>
2022-03-11 00:04:59 +03:00
delete "/:website/settings/google-search", SiteController, :delete_google_auth
delete "/:website/settings/google-import", SiteController, :delete_google_auth
2019-09-02 14:29:19 +03:00
delete "/:website", SiteController, :delete_site
delete "/:website/stats", SiteController, :reset_stats
2019-09-02 14:29:19 +03:00
2022-03-24 12:49:45 +03:00
get "/:website/import/google-analytics/view-id",
SiteController,
:import_from_google_view_id_form
post "/:website/import/google-analytics/view-id", SiteController, :import_from_google_view_id
2022-03-24 12:49:45 +03:00
get "/:website/import/google-analytics/user-metric",
SiteController,
:import_from_google_user_metric_notice
get "/:website/import/google-analytics/confirm", SiteController, :import_from_google_confirm
[Continued] Google Analytics import (#1753) * Add has_imported_stats boolean to Site * Add Google Analytics import panel to general settings * Get GA profiles to display in import settings panel * Add import_from_google method as entrypoint to import data * Add imported_visitors table * Remove conflicting code from migration * Import visitors data into clickhouse database * Pass another dataset to main graph for rendering in red This adds another entry to the JSON data returned via the main graph API called `imported_plot`, which is similar to `plot` in form but will be completed with previously imported data. Currently it simply returns the values from `plot` / 2. The data is rendered in the main graph in red without fill, and without an indicator for the present. Rationale: imported data will not continue to grow so there is no projection forward, only backwards. * Hook imported GA data to dashboard timeseries plot * Add settings option to forget imported data * Import sources from google analytics * Merge imported sources when queried * Merge imported source data native data when querying sources * Start converting metrics to atoms so they can be subqueried This changes "visitors" and in some places "sources" to atoms. This does not change the behaviour of the functions - the tests all pass unchanged following this commit. This is necessary as joining subqueries requires that the keys in `select` statements be atoms and not strings. * Convery GA (direct) source to empty string * Import utm campaign and utm medium from GA * format * Import all data types from GA into new tables * Handle large amounts of more data more safely * Fix some mistakes in tables * Make GA requests in chunks of 5 queries * Only display imported timeseries when there is no filter * Correctly show last 30 minutes timeseries when 'realtime' * Add with_imported key to Query struct * Account for injected :is_not filter on sources from dashboard * Also add tentative imported_utm_sources table This needs a bit more work on the google import side, as GA do not report sources and utm sources as distinct things. * Return imported data to dashboard for rest of Sources panel This extends the merge_imported function definition for sources to utm_sources, utm_mediums and utm_campaigns too. This appears to be working on the DB side but something is incomplete on the client side. * Clear imported stats from all tables when requested * Merge entry pages and exit pages from imported data into unfiltered dashboard view This requires converting the `"visits"` and `"visit_duration"` metrics to atoms so that they can be used in ecto subqueries. * Display imported devices, browsers and OSs on dashboard * Display imported country data on dashboard * Add more metrics to entries/exits for modals * make sure data is returned via API with correct keys * Import regions and cities from GA * Capitalize device upon import to match native data * Leave query limits/offsets until after possibly joining with imported data * Also import timeOnPage and pageviews for pages from GA * imported_countries -> imported_locations * Get timeOnPage and pageviews for pages from GA These are needed for the pages modal, and for calculating exit rates for exit pages. * Add indicator to dashboard when imported data is being used * Don't show imported data as separately line on main graph * "bounce_rate" -> :bounce_rate, so it works in subqueries * Drop imported browser and OS versions These are not needed. * Toggle displaying imported data by clicking indicator * Parse referrers with RefInspector - Use 'ga:fullReferrer' instead of 'ga:source'. This provides the actual referrer host + path, whereas 'ga:source' includes utm_mediums and other values when relevant. - 'ga:fullReferror' does however include search engine names directly, so they are manually checked for as RefInspector won't pick up on these. * Keep imported data indicator on dashboard and strikethrough when hidden * Add unlink google button to import panel * Rename some GA browsers and OSes to plausible versions * Get main top pages and exit pages panels working correctly with imported data * mix format * Fetch time_on_pages for imported data when needed * entry pages need to fetch bounces from GA * "sample_percent" -> :sample_percent as only atoms can be used in subqueries * Calculate bounce_rate for joined native and imported data for top pages modal * Flip some query bindings around to be less misleading * Fixup entry page modal visit durations * mix format * Fetch bounces and visit_duration for sources from GA * add more source metrics used for data in modals * Make sources modals display correct values * imported_visitors: bounce_rate -> bounces, avg_visit_duration -> visit_duration * Merge imported data into aggregate stats * Reformat top graph side icons * Ensure sample_percent is yielded from aggregate data * filter event_props should be strings * Hide imported data from frontend when using filter * Fix existing tests * fix tests * Fix imported indicator appearing when filtering * comma needed, lost when rebasing * Import utm_terms and utm_content from GA * Merge imported utm_term and utm_content * Rename imported Countries data as Locations * Set imported city schema field to int * Remove utm_terms and utm_content when clearing imported * Clean locations import from Google Analytics - Country and region should be set to "" when GA provides "(not set)" - City should be set to 0 for "unknown", as we cannot reliably import city data from GA. * Display imported region and city in dashboard * os -> operating_system in some parts of code The inconsistency of using os in some places and operating_system in others causes trouble with subqueries and joins for the native and imported data, which would require additional logic to account for. The simplest solution is the just use a consistent word for all uses. This doesn't make any user-facing or database changes. * to_atom -> to_existing_atom * format * "events" metric -> :events * ignore imported data when "events" in metrics * update "bounce_rate" * atomise some more metrics from new city and region api * atomise some more metrics for email handlers * "conversion_rate" -> :conversion_rate during csv export * Move imported data stats code to own module * Move imported timeseries function to Stats.Imported * Use Timex.parse to import dates from GA * has_imported_stats -> imported_source * "time_on_page" -> :time_on_page * Convert imported GA data to UTC * Clean up GA request code a bit There was some weird logic here with two separate lists that really ought to be together, so this merges those. * Fail sooner if GA timezone can't be identified * Link imported tables to site by id * imported_utm_content -> imported_utm_contents * Imported GA from all of time * Reorganise GA data fetch logic - Fetch data from the start of time (2005) - Check whether no data was fetched, and if so, inform user and don't consider data to be imported. * Clarify removal of "visits" data when it isn't in metrics * Apply location filters from API This makes it consistent with the sources etc which filter out 'Direct / None' on the API side. These filters are used by both the native and imported data handling code, which would otherwise both duplicate the filters in their `where` clauses. * Do not use changeset for setting site.imported_source * Add all metrics to all dimensions * Run GA import in the background * Send email when GA import completes * Add handler to insert imported data into tests and imported_browsers_factory * Add remaining import data test factories * Add imported location data to test * Test main graph with imported data * Add imported data to operating systems tests * Add imported data to pages tests * Add imported data to entry pages tests * Add imported data to exit pages tests * Add imported data to devices tests * Add imported data to sources tests * Add imported data to UTM tests * Add new test module for the data import step * Test import of sources GA data * Test import of utm_mediums GA data * Test import of utm_campaigns GA data * Add tests for UTM terms * Add tests for UTM contents * Add test for importing pages and entry pages data from GA * Add test for importing exit page data * Fix module file name typo * Add test for importing location data from GA * Add test for importing devices data from GA * Add test for importing browsers data from GA * Add test for importing OS data from GA * Paginate GA requests to download all data * Bump clickhouse_ecto version * Move RefInspector wrapper function into module * Drop timezone transform on import * Order imported by side_id then date * More strings -> atoms Also changes a conditional to be a bit nicer * Remove parallelisation of data import * Split sources and UTM sources from fetched GA data GA has only a "source" dimension and no "UTM source" dimension. Instead it returns these combined. The logic herein to tease these apart is: 1. "(direct)" -> it's a direct source 2. if the source is a domain -> it's a source 3. "google" -> it's from adwords; let's make this a UTM source "adwords" 4. else -> just a UTM source * Keep prop names in queries as strings * fix typo * Fix import * Insert data to clickhouse in batches * Fix link when removing imported data * Merge source tables * Import hostname as well as pathname * Record start and end time of imported data * Track import progress * Fix month interval with imported data * Do not JOIN when imported date range has no overlap * Fix time on page using exits Co-authored-by: mcol <mcol@posteo.net>
2022-03-11 00:04:59 +03:00
post "/:website/settings/google-import", SiteController, :import_from_google
delete "/:website/settings/forget-imported", SiteController, :forget_imported
2022-03-21 13:47:27 +03:00
get "/:domain/export", StatsController, :csv_export
get "/:domain/*path", StatsController, :stats
2019-09-02 14:29:19 +03:00
end
end