Commit Graph

29 Commits

Author SHA1 Message Date
Anon Ray
046a783a14
server: pass http and websocket request to logging context (#5470)
* pass request body to logging context in all cases

* add message size logging on the websocket API

  this is required by graphql-engine-pro/#416

* message size logging on websocket API

  As we need to log all messages recieved/sent by the websocket server,
  it makes sense to log them as part of the websocket server event logs.
  Previously message recieved were logged inside the onMessage handler,
  and messages sent were logged only for "data" messages (as a server event log)

* fix review comments

Co-authored-by: Phil Freeman <phil@hasura.io>
2020-07-29 20:18:36 +05:30
Phil Freeman
0dddbe9e9d
Add MonadTrace and MonadExecuteQuery abstractions (#5383)
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
2020-07-15 16:10:48 +05:30
Lyndon Maydwell
24592a516b
Pass environment variables around as a data structure, via @sordina (#5374)
* Pass environment variables around as a data structure, via @sordina

* Resolving build error

* Adding Environment passing note to changelog

* Removing references to ILTPollerLog as this seems to have been reintroduced from a bad merge

* removing commented-out imports

* Language pragmas already set by project

* Linking async thread

* Apply suggestions from code review

Use `runQueryTx` instead of `runLazyTx` for queries.

* remove the non-user facing entry in the changelog

Co-authored-by: Phil Freeman <paf31@cantab.net>
Co-authored-by: Phil Freeman <phil@hasura.io>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
2020-07-14 12:00:58 -07:00
Anon Ray
a7a60c2dfe
server: changes catalog initialization and logging for pro customization (#5139)
* new typeclass to abstract the logic of QueryLog-ing

* abstract the logic of logging websocket-server logs

  introduce a MonadWSLog typeclass

* move catalog initialization to init step

  expose a helper function to migrate catalog
  create schema cache in initialiseCtx

* expose various modules and functions for pro
2020-06-19 12:12:32 +05:30
Anon Ray
0cf4cbc5c6
server: refactor GQL execution check and config API (#5094)
Co-authored-by: Vamshi Surabhi <vamshi@hasura.io>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
2020-06-16 20:53:06 +05:30
Brandon Simmons
ff62d5e0bf Migrate to GHC 8.10, upgrade dependencies. Closes #4517
This also seems to squash a stubborn space leak we see with
subscriptions (linking to canonical #3388 for reference).

This may also fix some of the "Unexpected exception" websockets
exceptions we are now surfacing (see e.g. #4344)

Also: dev.sh: fix hpc reporting

Initial work on this done by Vamshi.
2020-05-13 19:13:02 -04:00
Rakesh Emmadi
d52bfcda4e
backend only insert permissions (rfc #4120) (#4224)
* move user info related code to Hasura.User module

* the RFC #4120 implementation; insert permissions with admin secret

* revert back to old RoleName based schema maps

An attempt made to avoid duplication of schema contexts in types
if any role doesn't possess any admin secret specific schema

* fix compile errors in haskell test

* keep 'user_vars' for session variables in http-logs

* no-op refacto

* tests for admin only inserts

* update docs for admin only inserts

* updated CHANGELOG.md

* default behaviour when admin secret is not set

* fix x-hasura-role to X-Hasura-Role in pytests

* introduce effective timeout in actions async tests

* update docs for admin-secret not configured case

* Update docs/graphql/manual/api-reference/schema-metadata-api/permission.rst

Co-Authored-By: Marion Schleifer <marion@hasura.io>

* Apply suggestions from code review

Co-Authored-By: Marion Schleifer <marion@hasura.io>

* a complete iteration

backend insert permissions accessable via 'x-hasura-backend-privilege'
session variable

* console changes for backend-only permissions

* provide tooltip id; update labels and tooltips;

* requested changes

* requested changes

- remove className from Toggle component
- use appropriate function name (capitalizeFirstChar -> capitalize)

* use toggle props from definitelyTyped

* fix accidental commit

* Revert "introduce effective timeout in actions async tests"

This reverts commit b7a59c19d6.

* generate complete schema for both 'default' and 'backend' sessions

* Apply suggestions from code review

Co-Authored-By: Marion Schleifer <marion@hasura.io>

* remove unnecessary import, export Toggle as is

* update session variable in tooltip

* 'x-hasura-use-backend-only-permissions' variable to switch

* update help texts

* update docs

* update docs

* update console help text

* regenerate package-lock

* serve no backend schema when backend_only: false and header set to true

- Few type name refactor as suggested by @0x777

* update CHANGELOG.md

* Update CHANGELOG.md

* Update CHANGELOG.md

* fix a merge bug where a certain entity didn't get removed

Co-authored-by: Marion Schleifer <marion@hasura.io>
Co-authored-by: Rishichandra Wawhal <rishi@hasura.io>
Co-authored-by: rikinsk <rikin.kachhia@gmail.com>
Co-authored-by: Tirumarai Selvan <tiru@hasura.io>
2020-04-24 14:40:53 +05:30
Antoine Leblanc
5b54f9d766
server: add support for webhook connection expiration (#4196)
* add expiry time to webhook user info

This also adds an optional message to webhook errors: if we fail to
parse an expiry time, we will log a warning with the parse error.

* refactored Auth

This change had one main goal: put in common all expiry time
extraction code between the JWT and WebHook parts of the
code. Furthermore, this change also moves all WebHook specific code to
its own module, similarly to what is done for JWT.

* Remove dependency on string-conversions in favor of text-conversions

string-conversions silently uses UTF8 instead of being explicit about
it, and it uses lenientDecode when decoding ByteStrings when it’s
usually better to reject invalid UTF8 input outright. text-conversions
solves both those problems.

Co-authored-by: Alexis King <lexi.lambda@gmail.com>
2020-04-02 19:00:13 -05:00
Brandon Simmons
f506cb397f Log time spent consuming request bytes from socket. Closes #3547
Add a new 'request_read_time' to the logging record.

We also use 'withElapsedTime' to get proper monotonic clock for
'query_execution_time' as well.
2020-02-03 18:50:10 -06:00
Brandon Simmons
58ef316118 Add request timings and count histograms to telemetry. Closes #3552
We upload a set of accumulating timers and counters to track service
time for different types of operations, across several dimensions (e.g.
did we hit the plan cache, was a remote involved, etc.)

Also...

Standardize on DiffTime as a standard duration type, and try to use it
consistently.

See discussion here:
https://github.com/hasura/graphql-engine/pull/3584#pullrequestreview-340679369

It should be possible to overwrite that module so the new threadDelay
sticks per the pattern in #3705 blocked on #3558

Rename the Control.Concurrent.Extended.threadDelay to `sleep` since a
naive use with a literal argument would be very bad!

We catch a bug in 'computeTimeDiff'.

Add convenient 'Read' instances to the time unit utility types. Make
'Second' a newtype to support this.
2020-02-03 18:50:10 -06:00
Alexis King
5b969208c6 Use arrows instead of monads to define the schema cache construction 2020-01-08 16:43:06 -06:00
nizar-m
c0038a74a5 Allow request body to be logged even with successful queries (#3529) 2019-12-16 15:02:05 -06:00
Anon Ray
490b639981 refactor some internal components (#3414) 2019-11-26 17:44:21 +05:30
Toan Nguyen
0c74839934 add raw query field for error http log (close #2963) (#3020) 2019-10-07 13:04:33 -05:00
Rakesh Emmadi
8a0615ff70 add gzip brotli compression to http responses (close #2674) (#2751) 2019-09-19 18:24:40 +05:30
Anon Ray
8f1e7dbc8d breaking: server logging changes (close #507, close #2171) (#1835) 2019-07-11 05:37:06 +00:00
Rakesh Emmadi
39f43bdd59 log inconsistent metadata objects only when there are any (close #2053) (#2058) 2019-04-29 11:52:48 +05:30
Rakesh Emmadi
e32f5a1fb1 sync metadata cache across multiple instances connected to same db (closes #1182) (#1574)
1. Haskel library `pg-client-hs` has been updated to expose a function that helps listen to `postgres` notifications over a `channel` in this [PR](https://github.com/hasura/pg-client-hs/pull/5)
2. The server records an event in a table `hdb_catalog.hdb_cache_update_event` whenever any `/v1/query` (that changes metadata) is requested. A trigger notifies a `cache update` event via `hasura_cache_update` channel
3. The server runs two concurrent threads namely `listener` and `processor`. The `listener` thread listens to events on `hasura_cache_update` channel and pushed into a `Queue`. The `processor` thread fetches events from that `Queue` and processes it. Thus server rebuilds schema cache from database and updates.
2019-03-12 11:16:27 +05:30
Rakesh Emmadi
4d9d1505dd improve startup logging, close #1236 (#1258) 2019-01-02 16:54:17 +05:30
Vamshi Surabhi
ec8b2c80b5
refactor to remove warnings especially with orphan instances (#1163)
* remove phase one/two distinction and hdbquery typeclass

* move extensions to default-extensions

* switch to LazyTx which only acquires a connection if needed

* move defns from TH module into Ops module

* remove tojson orphan instance for http exception

* remove orphan instance for dmlp1

* getTopLevelNodes will not throw any exceptions
2018-12-13 12:56:15 +05:30
Rakesh Emmadi
3ea20bc4d7 allow authentication webhook with POST (close #1138) (#1147) 2018-12-03 16:49:08 +05:30
Vamshi Surabhi
199531cbd9
http and websocket logs now correctly have user information, closes #849 (#850) 2018-10-25 15:07:57 +05:30
Anon Ray
1a0af29920 remove 's' unit from query_execution_time in logs (close #509) (#553)
Removes the seconds unit (trailing `s`) from `query_execution_time` in logs.

- [x] Server

It was a string before, changed to double.

- [x] Bug fix (non-breaking change which fixes an issue)

Docs should mention the type/unit of `query_execution_time` is numeric/double.
2018-09-27 18:02:21 +05:30
Anon Ray
75090d51b9 jwt config now takes a jwk url (close #465) (#527)
JWT config now takes an optional jwk_url parameter (which points to published JWK Set). This is useful for providers who rotate their JWK Set.

Optional jwk_url parameter is taken. The published JWK set under that URL should be in standard JWK format (tools.ietf.org/html/rfc7517#section-4.8).

If the response contains an Expires header, the JWK set is automatically refreshed.
2018-09-27 16:52:49 +05:30
Rakesh Emmadi
0a3f68a6eb allow selectively updating columns on a conflict during insert (fix #342)
* fix primary key changing on upsert, fix #342

* add 'update_columns' in 'on_conflict' object, consider 'allowUpsert'

* 'ConflictCtx' type should respect upsert cases

* validation for not null fields in an object
2018-09-04 19:09:48 +05:30
Rakesh Emmadi
8ef2692eb7 improve logs on webhook errors (closes #238, #242) (#243)
* logging for webhook IO exceptions, fix log request for errors, fix #238

* log status code and response in case of any error for webhook
2018-08-03 14:13:35 +05:30
Vamshi Surabhi
e3f960da96 initial support for livequeries (#176)
fix #59
2018-07-20 12:52:46 +05:30
Rakesh Emmadi
5efa366b49 add 'on_conflict' argument to insert mutation (closes #105)
* add 'on_conflict' condition to allow upsert mutation, closes #105

* check for empty unique or primary key constraints

* add 'on_conflict' condition test cases and introspection test case

* update 'conflict_action' enum values' description
2018-07-17 18:53:23 +05:30
Vamshi Surabhi
530027cf20 move raven into graphql-engine repo 2018-06-28 00:32:00 +05:30