Rewrite GraphQL schema generation and query parsing (close #2801) (#4111)
Aka “the PDV refactor.” History is preserved on the branch 2801-graphql-schema-parser-refactor.
* [skip ci] remove stale benchmark commit from commit_diff
* [skip ci] Check for root field name conflicts between remotes
* [skip ci] Additionally check for conflicts between remotes and DB
* [skip ci] Check for conflicts in schema when tracking a table
* [skip ci] Fix equality checking in GraphQL AST
* server: fix mishandling of GeoJSON inputs in subscriptions (fix #3239) (#4551)
* Add support for multiple top-level fields in a subscription to improve testability of subscriptions
* Add an internal flag to enable multiple subscriptions
* Add missing call to withConstructorFn in live queries (fix #3239)
Co-authored-by: Alexis King <lexi.lambda@gmail.com>
* Scheduled triggers (close #1914) (#3553)
server: add scheduled triggers
Co-authored-by: Alexis King <lexi.lambda@gmail.com>
Co-authored-by: Marion Schleifer <marion@hasura.io>
Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
Co-authored-by: Aleksandra Sikora <ola.zxcvbnm@gmail.com>
* dev.sh: bump version due to addition of croniter python dependency
* server: fix an introspection query caching issue (fix #4547) (#4661)
Introspection queries accept variables, but we need to make sure to
also touch the variables that we ignore, so that an introspection
query is marked not reusable if we are not able to build a correct
query plan for it.
A better solution here would be to deal with such unused variables
correctly, so that more introspection queries become reusable.
An even better solution would be to type-safely track *how* to reuse
which variables, rather than to split the reusage marking from the
planning.
Co-authored-by: Tirumarai Selvan <tiru@hasura.io>
* flush log buffer on exception in mkWaiApp ( fix #4772 ) (#4801)
* flush log buffer on exception in mkWaiApp
* add comment to explain the introduced change
* add changelog
* allow logging details of a live query polling thread (#4959)
* changes for poller-log
add various multiplexed query info in poller-log
* minor cleanup, also fixes a bug which will return duplicate data
* Live query poller stats can now be logged
This also removes in-memory stats that are collected about batched
query execution as the log lines when piped into an monitoring tool
will give us better insights.
* allow poller-log to be configurable
* log minimal information in the livequery-poller-log
Other information can be retrieved from /dev/subscriptions/extended
* fix few review comments
* avoid marshalling and unmarshalling from ByteString to EncJSON
* separate out SubscriberId and SubscriberMetadata
Co-authored-by: Anon Ray <rayanon004@gmail.com>
* Don't compile in developer APIs by default
* Tighten up handling of admin secret, more docs
Store the admin secret only as a hash to prevent leaking the secret
inadvertently, and to prevent timing attacks on the secret.
NOTE: best practice for stored user passwords is a function with a
tunable cost like bcrypt, but our threat model is quite different (even
if we thought we could reasonably protect the secret from an attacker
who could read arbitrary regions of memory), and bcrypt is far too slow
(by design) to perform on each request. We'd have to rely on our
(technically savvy) users to choose high entropy passwords in any case.
Referencing #4736
* server/docs: add instructions to fix loss of float precision in PostgreSQL <= 11 (#5187)
This adds a server flag, --pg-connection-options, that can be used to set a PostgreSQL connection parameter, extra_float_digits, that needs to be used to avoid loss of data on older versions of PostgreSQL, which have odd default behavior when returning float values. (fixes #5092)
* [skip ci] Add new commits from master to the commit diff
* [skip ci] serve default directives (skip & include) over introspection
* [skip ci] Update non-Haskell assets with the version on master
* server: refactor GQL execution check and config API (#5094)
Co-authored-by: Vamshi Surabhi <vamshi@hasura.io>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
* [skip ci] fix js issues in tests by pinning dependencies version
* [skip ci] bump graphql version
* [skip ci] Add note about memory usage
* generalize query execution logic on Postgres (#5110)
* generalize PGExecCtx to support specialized functions for various operations
* fix tests compilation
* allow customising PGExecCtx when starting the web server
* server: changes catalog initialization and logging for pro customization (#5139)
* new typeclass to abstract the logic of QueryLog-ing
* abstract the logic of logging websocket-server logs
introduce a MonadWSLog typeclass
* move catalog initialization to init step
expose a helper function to migrate catalog
create schema cache in initialiseCtx
* expose various modules and functions for pro
* [skip ci] cosmetic change
* [skip ci] fix test calling a mutation that does not exist
* [skip ci] minor text change
* [skip ci] refactored input values
* [skip ci] remove VString Origin
* server: fix updating of headers behaviour in the update cron trigger API and create future events immediately (#5151)
* server: fix bug to update headers in an existing cron trigger and create future events
Co-authored-by: Tirumarai Selvan <tiru@hasura.io>
* Lower stack chunk size in RTS to reduce thread STACK memory (closes #5190)
This reduces memory consumption for new idle subscriptions significantly
(see linked ticket).
The hypothesis is: we fork a lot of threads per websocket, and some of
these use slightly more than the initial 1K stack size, so the first
overflow balloons to 32K, when significantly less is required.
However: running with `+RTS -K1K -xc` did not seem to show evidence of
any overflows! So it's a mystery why this improves things.
GHC should probably also be doubling the stack buffer at each overflow
or doing something even smarter; the knobs we have aren't so helpful.
* [skip ci] fix todo and schema generation for aggregate fields
* 5087 libpq pool leak (#5089)
Shrink libpq buffers to 1MB before returning connection to pool. Closes #5087
See: https://github.com/hasura/pg-client-hs/pull/19
Also related: #3388 #4077
* bump pg-client-hs version (fixes a build issue on some environments) (#5267)
* do not use prepared statements for mutations
* server: unlock scheduled events on graceful shutdown (#4928)
* Fix buggy parsing of new --conn-lifetime flag in 2b0e3774
* [skip ci] remove cherry-picked commit from commit_diff.txt
* server: include additional fields in scheduled trigger webhook payload (#5262)
* include scheduled triggers metadata in the webhook body
Co-authored-by: Tirumarai Selvan <tiru@hasura.io>
* server: call the webhook asynchronously in event triggers (#5352)
* server: call the webhook asynchronosly in event triggers
* Expose all modules in Cabal file (#5371)
* [skip ci] update commit_diff.txt
* [skip ci] fix cast exp parser & few TODOs
* [skip ci] fix remote fields arguments
* [skip ci] fix few more TODO, no-op refactor, move resolve/action.hs to execute/action.hs
* Pass environment variables around as a data structure, via @sordina (#5374)
* Pass environment variables around as a data structure, via @sordina
* Resolving build error
* Adding Environment passing note to changelog
* Removing references to ILTPollerLog as this seems to have been reintroduced from a bad merge
* removing commented-out imports
* Language pragmas already set by project
* Linking async thread
* Apply suggestions from code review
Use `runQueryTx` instead of `runLazyTx` for queries.
* remove the non-user facing entry in the changelog
Co-authored-by: Phil Freeman <paf31@cantab.net>
Co-authored-by: Phil Freeman <phil@hasura.io>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
* [skip ci] fix: restrict remote relationship field generation for hasura queries
* [skip ci] no-op refactor; move insert execution code from schema parser module
* server: call the webhook asynchronously in event triggers (#5352)
* server: call the webhook asynchronosly in event triggers
* Expose all modules in Cabal file (#5371)
* [skip ci] update commit_diff.txt
* Pass environment variables around as a data structure, via @sordina (#5374)
* Pass environment variables around as a data structure, via @sordina
* Resolving build error
* Adding Environment passing note to changelog
* Removing references to ILTPollerLog as this seems to have been reintroduced from a bad merge
* removing commented-out imports
* Language pragmas already set by project
* Linking async thread
* Apply suggestions from code review
Use `runQueryTx` instead of `runLazyTx` for queries.
* remove the non-user facing entry in the changelog
Co-authored-by: Phil Freeman <paf31@cantab.net>
Co-authored-by: Phil Freeman <phil@hasura.io>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
* [skip ci] implement header checking
Probably closes #14 and #3659.
* server: refactor 'pollQuery' to have a hook to process 'PollDetails' (#5391)
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
* update pg-client (#5421)
* [skip ci] update commit_diff
* Fix latency buckets for telemetry data
These must have gotten messed up during a refactor. As a consequence
almost all samples received so far fall into the single erroneous 0 to
1K seconds (originally supposed to be 1ms?) bucket.
I also re-thought what the numbers should be, but these are still
arbitrary and might want adjusting in the future.
* [skip ci] include the latest commit compared against master in commit_diff
* [skip ci] include new commits from master in commit_diff
* [skip ci] improve description generation
* [skip ci] sort all introspect arrays
* [skip ci] allow parsers to specify error codes
* [skip ci] fix integer and float parsing error code
* [skip ci] scalar from json errors are now parse errors
* [skip ci] fixed negative integer error message and code
* [skip ci] Re-fix nullability in relationships
* [skip ci] no-op refactor and removed couple of FIXMEs
* [skip ci] uncomment code in 'deleteMetadataObject'
* [skip ci] Fix re-fix of nullability for relationships
* [skip ci] fix default arguments error code
* [skip ci] updated test error message
!!! WARNING !!!
Since all fields accept `null`, they all are technically optional in
the new schema. Meaning there's no such thing as a missing mandatory
field anymore: a field that doesn't have a default value, and which
therefore isn't labelled as "optional" in the schema, will be assumed
to be null if it's missing, meaning it isn't possible anymore to have
an error for a missing mandatory field. The only possible error is now
when a optional positional argument is omitted but is not the last
positional argument.
* [skip ci] cleanup of int scalar parser
* [skip ci] retro-compatibility of offset as string
* [skip ci] Remove commit from commit_diff.txt
Although strictly speaking we don't know if this will work correctly in PDV
if we would implement query plan caching, the fact is that in the theoretical
case that we would have the same issue in PDV, it would probably apply not just
to introspection, and the fix would be written completely differently. So this
old commit is of no value to us other than the heads-up "make sure query plan
caching works correctly even in the presence of unused variables", which is
already part of the test suite.
* Add MonadTrace and MonadExecuteQuery abstractions (#5383)
* [skip ci] Fix accumulation of input object types
Just like object types, interface types, and union types, we have to avoid
circularities when collecting input types from the GraphQL AST.
Additionally, this fixes equality checks for input object types (whose fields
are unordered, and hence should be compared as sets) and enum types (ditto).
* [skip ci] fix fragment error path
* [skip ci] fix node error code
* [skip ci] fix paths in insert queries
* [skip ci] fix path in objects
* [skip ci] manually alter node id path for consistency
* [skip ci] more node error fixups
* [skip ci] one last relay error message fix
* [skip ci] update commit_diff
* Propagate the trace context to event triggers (#5409)
* Propagate the trace context to event triggers
* Handle missing trace and span IDs
* Store trace context as one LOCAL
* Add migrations
* Documentation
* changelog
* Fix warnings
* Respond to code review suggestions
* Respond to code review
* Undo changelog
* Update CHANGELOG.md
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
* server: log request/response sizes for event triggers (#5463)
* server: log request/response sizes for event triggers
event triggers (and scheduled triggers) now have request/response size
in their logs.
* add changelog entry
* Tracing: Simplify HTTP traced request (#5451)
Remove the Inversion of Control (SuspendRequest) and simplify
the tracing of HTTP Requests.
Co-authored-by: Phil Freeman <phil@hasura.io>
* Attach request ID as tracing metadata (#5456)
* Propagate the trace context to event triggers
* Handle missing trace and span IDs
* Store trace context as one LOCAL
* Add migrations
* Documentation
* Include the request ID as trace metadata
* changelog
* Fix warnings
* Respond to code review suggestions
* Respond to code review
* Undo changelog
* Update CHANGELOG.md
* Typo
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
* server: add logging for action handlers (#5471)
* server: add logging for action handlers
* add changelog entry
* change action-handler log type from internal to non-internal
* fix action-handler-log name
* server: pass http and websocket request to logging context (#5470)
* pass request body to logging context in all cases
* add message size logging on the websocket API
this is required by graphql-engine-pro/#416
* message size logging on websocket API
As we need to log all messages recieved/sent by the websocket server,
it makes sense to log them as part of the websocket server event logs.
Previously message recieved were logged inside the onMessage handler,
and messages sent were logged only for "data" messages (as a server event log)
* fix review comments
Co-authored-by: Phil Freeman <phil@hasura.io>
* server: stop eventing subsystem threads when shutting down (#5479)
* server: stop eventing subsystem threads when shutting down
* Apply suggestions from code review
Co-authored-by: Karthikeyan Chinnakonda <chkarthikeyan95@gmail.com>
Co-authored-by: Phil Freeman <phil@hasura.io>
Co-authored-by: Phil Freeman <paf31@cantab.net>
Co-authored-by: Karthikeyan Chinnakonda <chkarthikeyan95@gmail.com>
* [skip ci] update commit_diff with new commits added in master
* Bugfix to support 0-size HASURA_GRAPHQL_QUERY_PLAN_CACHE_SIZE
Also some minor refactoring of bounded cache module:
- the maxBound check in `trim` was confusing and unnecessary
- consequently trim was unnecessary for lookupPure
Also add some basic tests
* Support only the bounded cache, with default HASURA_GRAPHQL_QUERY_PLAN_CACHE_SIZE of 4000. Closes #5363
* [skip ci] remove merge commit from commit_diff
* server: Fix compiler warning caused by GHC upgrade (#5489)
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
* [skip ci] update all non server code from master
* [skip ci] aligned object field error message with master
* [skip ci] fix remaining undefined?
* [skip ci] remove unused import
* [skip ci] revert to previous error message, fix tests
* Move nullableType/nonNullableType to Schema.hs
These are functions on Types, not on Parsers.
* [skip ci] fix setup to fix backend only test
the order in which permission checks are performed on the branch is
slightly different than on master, resulting in a slightly different
error if there are no other mutations the user has access to. By
adding update permissions, we go back to the expected case.
* [skip ci] fix insert geojson tests to reflect new paths
* [skip ci] fix enum test for better error message
* [skip ci] fix header test for better error message
* [skip ci] fix fragment cycle test for better error message
* [skip ci] fix error message for type mismatch
* [skip ci] fix variable path in test
* [skip ci] adjust tests after bug fix
* [skip ci] more tests fixing
* Add hdb_catalog.current_setting abstraction for reading Hasura settings
As the comment in the function’s definition explains, this is needed to
work around an awkward Postgres behavior.
* [skip ci] Update CONTRIBUTING.md to mention Node setup for Python tests
* [skip ci] Add missing Python tests env var to CONTRIBUTING.md
* [skip ci] fix order of result when subscription is run with multiple nodes
* [skip ci] no-op refactor: fix a warning in Internal/Parser.hs
* [skip ci] throw error when a subscription contains remote joins
* [skip ci] Enable easier profiling by hiding AssertNF behind a flag
In order to compile a profiling build, run:
$ cabal new-build -f profiling --enable-profiling
* [skip ci] Fix two warnings
We used to lookup the objects that implement a given interface by filtering all
objects in the schema document. However, one of the tests expects us to
generate a warning if the provided `implements` field of an introspection query
specifies an object not implementing some interface. So we use that field
instead.
* [skip ci] Fix warnings by commenting out query plan caching
* [skip ci] improve masking/commenting query caching related code & few warning fixes
* [skip ci] Fixed compiler warnings in graphql-parser-hs
* Sync non-Haskell assets with master
* [skip ci] add a test inserting invalid GraphQL but valid JSON value in a jsonb column
* [skip ci] Avoid converting to/from Map
* [skip ci] Apply some hlint suggestions
* [skip ci] remove redundant constraints from buildLiveQueryPlan and explainGQLQuery
* [skip ci] add NOTEs about missing Tracing constraints in PDV from master
* Remove -fdefer-typed-holes, fix warnings
* Update cabal.project.freeze
* Limit GHC’s heap size to 8GB in CI to avoid the OOM killer
* Commit package-lock.json for Python tests’ remote schema server
* restrict env variables start with HASURA_GRAPHQL_ for headers configuration in actions, event triggers & remote schemas (#5519)
* restrict env variables start with HASURA_GRAPHQL_ for headers definition in actions & event triggers
* update CHANGELOG.md
* Apply suggestions from code review
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
* add test for table_by_pk node when roles doesn't have permission to PK
* [skip ci] fix introspection query if any enum column present in primary key (fix #5200) (#5522)
* [skip ci] test case fix for a6450e126bc2d98bcfd3791501986e4627ce6c6f
* [skip ci] add tests to agg queries when role doesn't have access to any cols
* fix backend test
* Simplify subscription execution
* [skip ci] add test to check if required headers are present while querying
* Suppose, table B is related to table A and to query B certain headers are
necessary, then the test checks that we are throwing error when the header
is not set when B is queried through A
* fix mutations not checking for view mutability
* [skip ci] add variable type checking and corresponding tests
* [skip ci] add test to check if update headers are present while doing an upsert
* [skip ci] add positive counterparts to some of the negative permission tests
* fix args missing their description in introspect
* [skip ci] Remove unused function; insert missing markNotReusable call
* [skip ci] Add a Note about InputValue
* [skip ci] Delete LegacySchema/ 🎉
* [skip ci] Delete GraphQL/{Resolve,Validate}/ 🎉
* [skip ci] Delete top-level Resolve/Validate modules; tidy .cabal file
* [skip ci] Delete LegacySchema top-level module
Somehow I missed this one.
* fix input value to json
* [skip ci] elaborate on JSON objects in GraphQL
* [skip ci] add missing file
* [skip ci] add a test with subscription containing remote joins
* add a test with remote joins in mutation output
* [skip ci] Add some comments to Schema/Mutation.hs
* [skip ci] Remove no longer needed code from RemoteServer.hs
* [skip ci] Use a helper function to generate conflict clause parsers
* [skip ci] fix type checker error in fields with default value
* capitalize the header keys in select_articles_without_required_headers
* Somehow, this was the reason the tests were failing. I have no idea, why!
* [skip ci] Add a long Note about optional fields and nullability
* Improve comments a bit; simplify Schema/Common.hs a bit
* [skip ci] full implementation of 5.8.5 type checking.
* [skip ci] fix validation test teardown
* [skip ci] fix schema stitching test
* fix remote schema ignoring enum nullability
* [skip ci] fix fieldOptional to not discard nullability
* revert nullability of use_spheroid
* fix comment
* add required remote fields with arguments for tests
* [skip ci] add missing docstrings
* [skip ci] fixed description of remote fields
* [skip ci] change docstring for consistency
* fix several schema inconsistencies
* revert behaviour change in function arguments parsing
* fix remaining nullability issues in new schema
* minor no-op refactor; use isListType from graphql-parser-hs
* use nullability of remote schema node, while creating a Remote reln
* fix 'ID' input coercing & action 'ID' type relationship mapping
* include ASTs in MonadExecuteQuery
* needed for PRO code-base
* Delete code for "interfaces implementing ifaces" (draft GraphQL spec)
Previously I started writing some code that adds support for a future GraphQL
feature where interfaces may themselves be sub-types of other interfaces.
However, this code was incomplete, and partially incorrect. So this commit
deletes support for that entirely.
* Ignore a remote schema test during the upgrade/downgrade test
The PDV refactor does a better job at exposing a minimal set of types through
introspection. In particular, not every type that is present in a remote schema
is re-exposed by Hasura. The test
test_schema_stitching.py::TestRemoteSchemaBasic::test_introspection assumed that
all types were re-exposed, which is not required for GraphQL compatibility, in
order to test some aspect of our support for remote schemas.
So while this particular test has been updated on PDV, the PDV branch now does
not pass the old test, which we argue to be incorrect. Hence this test is
disabled while we await a release, after which we can re-enable it.
This also re-enables a test that was previously disabled for similar, though
unrelated, reasons.
* add haddock documentation to the action's field parsers
* Deslecting some tests in server-upgrade
Some tests with current build are failing on server upgrade
which it should not. The response is more accurate than
what it was.
Also the upgrade tests were not throwing errors when the test is
expected to return an error, but succeeds. The test framework is
patched to catch this case.
* [skip ci] Add a long Note about interfaces and object types
* send the response headers back to client after running a query
* Deselect a few more tests during upgrade/downgrade test
* Update commit_diff.txt
* change log kind from db_migrate to catalog_migrate (#5531)
* Show method and complete URI in traced HTTP calls (#5525)
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
* restrict env variables start with HASURA_GRAPHQL_ for headers configuration in actions, event triggers & remote schemas (#5519)
* restrict env variables start with HASURA_GRAPHQL_ for headers definition in actions & event triggers
* update CHANGELOG.md
* Apply suggestions from code review
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
* fix introspection query if any enum column present in primary key (fix #5200) (#5522)
* Fix telemetry reporting of transport (websocket was reported as http)
* add log kinds in cli-migrations image (#5529)
* add log kinds in cli-migrations image
* give hint to resolve timeout error
* minor changes and CHANGELOG
* server: set hasura.tracecontext in RQL mutations [#5542] (#5555)
* server: set hasura.tracecontext in RQL mutations [#5542]
* Update test suite
Co-authored-by: Tirumarai Selvan <tiru@hasura.io>
* Add bulldozer auto-merge and -update configuration
We still need to add the github app (as of time of opening this PR)
Afterwards devs should be able to allow bulldozer to automatically
"update" the branch, merging in parent when it changes, as well as
automatically merge when all checks pass.
This is opt-in by adding the `auto-update-auto-merge` label to the PR.
* Remove 'bulldozer' config, try 'kodiak' for auto-merge
see: https://github.com/chdsbd/kodiak
The main issue that bit us was not being able to auto update forked
branches, also:
https://github.com/palantir/bulldozer/issues/66
https://github.com/palantir/bulldozer/issues/145
* Cherry-picked all commits
* [skip ci] Slightly improve formatting
* Revert "fix introspection query if any enum column present in primary key (fix #5200) (#5522)"
This reverts commit 0f9a5afa59a88f6824f4d63d58db246a5ba3fb03.
This undoes a cherry-pick of 34288e1eb5f2c5dad9e6d1e05453dd52397dc970 that was
already done previously in a6450e126bc2d98bcfd3791501986e4627ce6c6f, and
subsequently fixed for PDV in 70e89dc250f8ddc6e2b7930bbe2b3eeaa6dbe1db
* Do a small bit of tidying in Hasura.GraphQL.Parser.Collect
* Fix cherry-picking work
Some previous cherry-picks ended up modifying code that is commented out
* [skip ci] clarified comment regarding insert representation
* [skip ci] removed obsolete todos
* cosmetic change
* fix action error message
* [skip ci] remove obsolete comment
* [skip ci] synchronize stylish haskell extensions list
* use previously defined scalar names in parsers rather than ad-hoc literals
* Apply most syntax hlint hints.
* Clarify comment on update mutation.
* [skip ci] Clarify what fields should be specified for objects
* Update "_inc" description.
* Use record types rather than tuples fo IntrospectionResult and ParsedIntrospection
* Get rid of checkFieldNamesUnique (use Data.List.Extended.duplicates)
* Throw more errors when collecting query root names
* [skip ci] clean column parser comment
* Remove dead code inserted in ab65b39
* avoid converting to non-empty list where not needed
* add note and TODO about the disabled checks in PDV
* minor refactor in remoteField' function
* Unify two getObject methods
* Nitpicks in Remote.hs
* Update CHANGELOG.md
* Revert "Unify two getObject methods"
This reverts commit bd6bb40355b3d189a46c0312eb52225e18be57b3.
We do need two different getObject functions as the corresponding error message is different
* Fix error message in Remote.hs
* Update CHANGELOG.md
Co-authored-by: Auke Booij <auke@tulcod.com>
* Apply suggested Changelog fix.
Co-authored-by: Auke Booij <auke@tulcod.com>
* Fix typo in Changelog.
* [skip ci] Update changelog.
* reuse type names to avoid duplication
* Fix Hashable instance for Definition
The presence of `Maybe Unique`, and an optional description, as part of
`Definition`s, means that `Definition`s that are considered `Eq`ual may get
different hashes. This can happen, for instance, when one object is memoized
but another is not.
* [skip ci] Update commit_diff.txt
* Bump parser version.
* Bump freeze file after changes in parser.
* [skip ci] Incorporate commits from master
* Fix developer flag in server/cabal.project.freeze
Co-authored-by: Auke Booij <auke@tulcod.com>
* Deselect a changed ENUM test for upgrade/downgrade CI
* Deselect test here as well
* [skip ci] remove dead code
* Disable more tests for upgrade/downgrade
* Fix which test gets deselected
* Revert "Add hdb_catalog.current_setting abstraction for reading Hasura settings"
This reverts commit 66e85ab9fbd56cca2c28a80201f6604fbe811b85.
* Remove circular reference in cabal.project.freeze
Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
Co-authored-by: Auke Booij <auke@hasura.io>
Co-authored-by: Tirumarai Selvan <tiru@hasura.io>
Co-authored-by: Marion Schleifer <marion@hasura.io>
Co-authored-by: Aleksandra Sikora <ola.zxcvbnm@gmail.com>
Co-authored-by: Brandon Simmons <brandon.m.simmons@gmail.com>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
Co-authored-by: Anon Ray <rayanon004@gmail.com>
Co-authored-by: rakeshkky <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Anon Ray <ecthiender@users.noreply.github.com>
Co-authored-by: Vamshi Surabhi <vamshi@hasura.io>
Co-authored-by: Antoine Leblanc <antoine@hasura.io>
Co-authored-by: Brandon Simmons <brandon@hasura.io>
Co-authored-by: Phil Freeman <phil@hasura.io>
Co-authored-by: Lyndon Maydwell <lyndon@sordina.net>
Co-authored-by: Phil Freeman <paf31@cantab.net>
Co-authored-by: Naveen Naidu <naveennaidu479@gmail.com>
Co-authored-by: Karthikeyan Chinnakonda <chkarthikeyan95@gmail.com>
Co-authored-by: Nizar Malangadan <nizar-m@users.noreply.github.com>
Co-authored-by: Antoine Leblanc <crucuny@gmail.com>
Co-authored-by: Auke Booij <auke@tulcod.com>
2020-08-21 20:27:01 +03:00
{- # LANGUAGE CPP # -}
2018-06-27 16:11:32 +03:00
2021-11-04 19:08:33 +03:00
module Hasura.Server.App
2022-07-24 00:18:01 +03:00
( APIResp ( JSONResp , RawResp ) ,
2021-11-04 19:08:33 +03:00
ConsoleRenderer ( .. ) ,
2022-12-07 14:28:58 +03:00
MonadVersionAPIWithExtraData ( .. ) ,
2021-11-04 19:08:33 +03:00
Handler ,
HandlerCtx ( hcReqHeaders , hcServerCtx , hcUser ) ,
HasuraApp ( HasuraApp ) ,
2023-01-06 12:33:13 +03:00
Loggers ( .. ) ,
2021-11-04 19:08:33 +03:00
MonadConfigApiHandler ( .. ) ,
MonadMetadataApiAuthorization ( .. ) ,
2023-01-06 12:33:13 +03:00
ServerCtx ( .. ) ,
2021-11-04 19:08:33 +03:00
boolToText ,
configApiGetHandler ,
isAdminSecretSet ,
mkGetHandler ,
mkSpockAction ,
mkWaiApp ,
onlyAdmin ,
renderHtmlTemplate ,
)
where
2018-06-27 16:11:32 +03:00
2020-06-19 09:42:32 +03:00
import Control.Concurrent.Async.Lifted.Safe qualified as LA
2023-01-06 12:33:13 +03:00
import Control.Concurrent.STM qualified as STM
2020-10-27 16:53:49 +03:00
import Control.Exception ( IOException , try )
import Control.Monad.Stateless
import Control.Monad.Trans.Control ( MonadBaseControl )
2020-07-15 13:40:48 +03:00
import Control.Monad.Trans.Control qualified as MTC
2020-10-27 16:53:49 +03:00
import Data.Aeson hiding ( json )
2021-01-29 04:02:34 +03:00
import Data.Aeson qualified as J
2022-06-08 18:31:28 +03:00
import Data.Aeson.Key qualified as K
import Data.Aeson.KeyMap qualified as KM
2022-12-07 14:28:58 +03:00
import Data.Aeson.Types qualified as J
2020-07-15 13:40:48 +03:00
import Data.ByteString.Char8 qualified as B8
2020-06-19 09:42:32 +03:00
import Data.ByteString.Lazy qualified as BL
import Data.CaseInsensitive qualified as CI
2020-07-14 22:00:58 +03:00
import Data.Environment qualified as Env
2020-06-19 09:42:32 +03:00
import Data.HashMap.Strict qualified as M
import Data.HashSet qualified as S
2020-10-27 16:53:49 +03:00
import Data.String ( fromString )
2020-06-19 09:42:32 +03:00
import Data.Text qualified as T
2021-06-08 17:28:00 +03:00
import Data.Text.Conversions ( convertText )
2020-12-28 15:56:00 +03:00
import Data.Text.Extended
2021-06-08 17:28:00 +03:00
import Data.Text.Lazy qualified as LT
import Data.Text.Lazy.Encoding qualified as TL
2023-01-06 12:33:13 +03:00
import Database.PG.Query qualified as PG
2021-04-13 20:32:29 +03:00
import GHC.Stats.Extended qualified as RTS
2022-06-29 10:42:51 +03:00
import Hasura.Backends.DataConnector.API ( openApiSchema )
2020-10-27 16:53:49 +03:00
import Hasura.Backends.Postgres.Execute.Types
2020-06-19 09:42:32 +03:00
import Hasura.Base.Error
import Hasura.EncJSON
import Hasura.GraphQL.Execute qualified as E
import Hasura.GraphQL.Execute.Backend qualified as EB
2022-03-21 13:39:49 +03:00
import Hasura.GraphQL.Execute.Subscription.State qualified as ES
2021-02-20 16:45:49 +03:00
import Hasura.GraphQL.Explain qualified as GE
2020-10-27 16:53:49 +03:00
import Hasura.GraphQL.Logging ( MonadQueryLog )
2022-07-12 17:00:15 +03:00
import Hasura.GraphQL.Schema.NamingCase
2022-07-14 20:57:28 +03:00
import Hasura.GraphQL.Schema.Options qualified as Options
2020-10-27 16:53:49 +03:00
import Hasura.GraphQL.Transport.HTTP qualified as GH
import Hasura.GraphQL.Transport.HTTP.Protocol qualified as GH
2021-08-24 19:25:12 +03:00
import Hasura.GraphQL.Transport.WSServerApp qualified as WS
2020-10-27 16:53:49 +03:00
import Hasura.GraphQL.Transport.WebSocket.Server qualified as WS
2020-04-24 10:55:51 +03:00
import Hasura.HTTP
2020-10-27 16:53:49 +03:00
import Hasura.Logging qualified as L
2020-12-14 07:30:19 +03:00
import Hasura.Metadata.Class
Rewrite GraphQL schema generation and query parsing (close #2801) (#4111)
Aka “the PDV refactor.” History is preserved on the branch 2801-graphql-schema-parser-refactor.
* [skip ci] remove stale benchmark commit from commit_diff
* [skip ci] Check for root field name conflicts between remotes
* [skip ci] Additionally check for conflicts between remotes and DB
* [skip ci] Check for conflicts in schema when tracking a table
* [skip ci] Fix equality checking in GraphQL AST
* server: fix mishandling of GeoJSON inputs in subscriptions (fix #3239) (#4551)
* Add support for multiple top-level fields in a subscription to improve testability of subscriptions
* Add an internal flag to enable multiple subscriptions
* Add missing call to withConstructorFn in live queries (fix #3239)
Co-authored-by: Alexis King <lexi.lambda@gmail.com>
* Scheduled triggers (close #1914) (#3553)
server: add scheduled triggers
Co-authored-by: Alexis King <lexi.lambda@gmail.com>
Co-authored-by: Marion Schleifer <marion@hasura.io>
Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
Co-authored-by: Aleksandra Sikora <ola.zxcvbnm@gmail.com>
* dev.sh: bump version due to addition of croniter python dependency
* server: fix an introspection query caching issue (fix #4547) (#4661)
Introspection queries accept variables, but we need to make sure to
also touch the variables that we ignore, so that an introspection
query is marked not reusable if we are not able to build a correct
query plan for it.
A better solution here would be to deal with such unused variables
correctly, so that more introspection queries become reusable.
An even better solution would be to type-safely track *how* to reuse
which variables, rather than to split the reusage marking from the
planning.
Co-authored-by: Tirumarai Selvan <tiru@hasura.io>
* flush log buffer on exception in mkWaiApp ( fix #4772 ) (#4801)
* flush log buffer on exception in mkWaiApp
* add comment to explain the introduced change
* add changelog
* allow logging details of a live query polling thread (#4959)
* changes for poller-log
add various multiplexed query info in poller-log
* minor cleanup, also fixes a bug which will return duplicate data
* Live query poller stats can now be logged
This also removes in-memory stats that are collected about batched
query execution as the log lines when piped into an monitoring tool
will give us better insights.
* allow poller-log to be configurable
* log minimal information in the livequery-poller-log
Other information can be retrieved from /dev/subscriptions/extended
* fix few review comments
* avoid marshalling and unmarshalling from ByteString to EncJSON
* separate out SubscriberId and SubscriberMetadata
Co-authored-by: Anon Ray <rayanon004@gmail.com>
* Don't compile in developer APIs by default
* Tighten up handling of admin secret, more docs
Store the admin secret only as a hash to prevent leaking the secret
inadvertently, and to prevent timing attacks on the secret.
NOTE: best practice for stored user passwords is a function with a
tunable cost like bcrypt, but our threat model is quite different (even
if we thought we could reasonably protect the secret from an attacker
who could read arbitrary regions of memory), and bcrypt is far too slow
(by design) to perform on each request. We'd have to rely on our
(technically savvy) users to choose high entropy passwords in any case.
Referencing #4736
* server/docs: add instructions to fix loss of float precision in PostgreSQL <= 11 (#5187)
This adds a server flag, --pg-connection-options, that can be used to set a PostgreSQL connection parameter, extra_float_digits, that needs to be used to avoid loss of data on older versions of PostgreSQL, which have odd default behavior when returning float values. (fixes #5092)
* [skip ci] Add new commits from master to the commit diff
* [skip ci] serve default directives (skip & include) over introspection
* [skip ci] Update non-Haskell assets with the version on master
* server: refactor GQL execution check and config API (#5094)
Co-authored-by: Vamshi Surabhi <vamshi@hasura.io>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
* [skip ci] fix js issues in tests by pinning dependencies version
* [skip ci] bump graphql version
* [skip ci] Add note about memory usage
* generalize query execution logic on Postgres (#5110)
* generalize PGExecCtx to support specialized functions for various operations
* fix tests compilation
* allow customising PGExecCtx when starting the web server
* server: changes catalog initialization and logging for pro customization (#5139)
* new typeclass to abstract the logic of QueryLog-ing
* abstract the logic of logging websocket-server logs
introduce a MonadWSLog typeclass
* move catalog initialization to init step
expose a helper function to migrate catalog
create schema cache in initialiseCtx
* expose various modules and functions for pro
* [skip ci] cosmetic change
* [skip ci] fix test calling a mutation that does not exist
* [skip ci] minor text change
* [skip ci] refactored input values
* [skip ci] remove VString Origin
* server: fix updating of headers behaviour in the update cron trigger API and create future events immediately (#5151)
* server: fix bug to update headers in an existing cron trigger and create future events
Co-authored-by: Tirumarai Selvan <tiru@hasura.io>
* Lower stack chunk size in RTS to reduce thread STACK memory (closes #5190)
This reduces memory consumption for new idle subscriptions significantly
(see linked ticket).
The hypothesis is: we fork a lot of threads per websocket, and some of
these use slightly more than the initial 1K stack size, so the first
overflow balloons to 32K, when significantly less is required.
However: running with `+RTS -K1K -xc` did not seem to show evidence of
any overflows! So it's a mystery why this improves things.
GHC should probably also be doubling the stack buffer at each overflow
or doing something even smarter; the knobs we have aren't so helpful.
* [skip ci] fix todo and schema generation for aggregate fields
* 5087 libpq pool leak (#5089)
Shrink libpq buffers to 1MB before returning connection to pool. Closes #5087
See: https://github.com/hasura/pg-client-hs/pull/19
Also related: #3388 #4077
* bump pg-client-hs version (fixes a build issue on some environments) (#5267)
* do not use prepared statements for mutations
* server: unlock scheduled events on graceful shutdown (#4928)
* Fix buggy parsing of new --conn-lifetime flag in 2b0e3774
* [skip ci] remove cherry-picked commit from commit_diff.txt
* server: include additional fields in scheduled trigger webhook payload (#5262)
* include scheduled triggers metadata in the webhook body
Co-authored-by: Tirumarai Selvan <tiru@hasura.io>
* server: call the webhook asynchronously in event triggers (#5352)
* server: call the webhook asynchronosly in event triggers
* Expose all modules in Cabal file (#5371)
* [skip ci] update commit_diff.txt
* [skip ci] fix cast exp parser & few TODOs
* [skip ci] fix remote fields arguments
* [skip ci] fix few more TODO, no-op refactor, move resolve/action.hs to execute/action.hs
* Pass environment variables around as a data structure, via @sordina (#5374)
* Pass environment variables around as a data structure, via @sordina
* Resolving build error
* Adding Environment passing note to changelog
* Removing references to ILTPollerLog as this seems to have been reintroduced from a bad merge
* removing commented-out imports
* Language pragmas already set by project
* Linking async thread
* Apply suggestions from code review
Use `runQueryTx` instead of `runLazyTx` for queries.
* remove the non-user facing entry in the changelog
Co-authored-by: Phil Freeman <paf31@cantab.net>
Co-authored-by: Phil Freeman <phil@hasura.io>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
* [skip ci] fix: restrict remote relationship field generation for hasura queries
* [skip ci] no-op refactor; move insert execution code from schema parser module
* server: call the webhook asynchronously in event triggers (#5352)
* server: call the webhook asynchronosly in event triggers
* Expose all modules in Cabal file (#5371)
* [skip ci] update commit_diff.txt
* Pass environment variables around as a data structure, via @sordina (#5374)
* Pass environment variables around as a data structure, via @sordina
* Resolving build error
* Adding Environment passing note to changelog
* Removing references to ILTPollerLog as this seems to have been reintroduced from a bad merge
* removing commented-out imports
* Language pragmas already set by project
* Linking async thread
* Apply suggestions from code review
Use `runQueryTx` instead of `runLazyTx` for queries.
* remove the non-user facing entry in the changelog
Co-authored-by: Phil Freeman <paf31@cantab.net>
Co-authored-by: Phil Freeman <phil@hasura.io>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
* [skip ci] implement header checking
Probably closes #14 and #3659.
* server: refactor 'pollQuery' to have a hook to process 'PollDetails' (#5391)
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
* update pg-client (#5421)
* [skip ci] update commit_diff
* Fix latency buckets for telemetry data
These must have gotten messed up during a refactor. As a consequence
almost all samples received so far fall into the single erroneous 0 to
1K seconds (originally supposed to be 1ms?) bucket.
I also re-thought what the numbers should be, but these are still
arbitrary and might want adjusting in the future.
* [skip ci] include the latest commit compared against master in commit_diff
* [skip ci] include new commits from master in commit_diff
* [skip ci] improve description generation
* [skip ci] sort all introspect arrays
* [skip ci] allow parsers to specify error codes
* [skip ci] fix integer and float parsing error code
* [skip ci] scalar from json errors are now parse errors
* [skip ci] fixed negative integer error message and code
* [skip ci] Re-fix nullability in relationships
* [skip ci] no-op refactor and removed couple of FIXMEs
* [skip ci] uncomment code in 'deleteMetadataObject'
* [skip ci] Fix re-fix of nullability for relationships
* [skip ci] fix default arguments error code
* [skip ci] updated test error message
!!! WARNING !!!
Since all fields accept `null`, they all are technically optional in
the new schema. Meaning there's no such thing as a missing mandatory
field anymore: a field that doesn't have a default value, and which
therefore isn't labelled as "optional" in the schema, will be assumed
to be null if it's missing, meaning it isn't possible anymore to have
an error for a missing mandatory field. The only possible error is now
when a optional positional argument is omitted but is not the last
positional argument.
* [skip ci] cleanup of int scalar parser
* [skip ci] retro-compatibility of offset as string
* [skip ci] Remove commit from commit_diff.txt
Although strictly speaking we don't know if this will work correctly in PDV
if we would implement query plan caching, the fact is that in the theoretical
case that we would have the same issue in PDV, it would probably apply not just
to introspection, and the fix would be written completely differently. So this
old commit is of no value to us other than the heads-up "make sure query plan
caching works correctly even in the presence of unused variables", which is
already part of the test suite.
* Add MonadTrace and MonadExecuteQuery abstractions (#5383)
* [skip ci] Fix accumulation of input object types
Just like object types, interface types, and union types, we have to avoid
circularities when collecting input types from the GraphQL AST.
Additionally, this fixes equality checks for input object types (whose fields
are unordered, and hence should be compared as sets) and enum types (ditto).
* [skip ci] fix fragment error path
* [skip ci] fix node error code
* [skip ci] fix paths in insert queries
* [skip ci] fix path in objects
* [skip ci] manually alter node id path for consistency
* [skip ci] more node error fixups
* [skip ci] one last relay error message fix
* [skip ci] update commit_diff
* Propagate the trace context to event triggers (#5409)
* Propagate the trace context to event triggers
* Handle missing trace and span IDs
* Store trace context as one LOCAL
* Add migrations
* Documentation
* changelog
* Fix warnings
* Respond to code review suggestions
* Respond to code review
* Undo changelog
* Update CHANGELOG.md
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
* server: log request/response sizes for event triggers (#5463)
* server: log request/response sizes for event triggers
event triggers (and scheduled triggers) now have request/response size
in their logs.
* add changelog entry
* Tracing: Simplify HTTP traced request (#5451)
Remove the Inversion of Control (SuspendRequest) and simplify
the tracing of HTTP Requests.
Co-authored-by: Phil Freeman <phil@hasura.io>
* Attach request ID as tracing metadata (#5456)
* Propagate the trace context to event triggers
* Handle missing trace and span IDs
* Store trace context as one LOCAL
* Add migrations
* Documentation
* Include the request ID as trace metadata
* changelog
* Fix warnings
* Respond to code review suggestions
* Respond to code review
* Undo changelog
* Update CHANGELOG.md
* Typo
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
* server: add logging for action handlers (#5471)
* server: add logging for action handlers
* add changelog entry
* change action-handler log type from internal to non-internal
* fix action-handler-log name
* server: pass http and websocket request to logging context (#5470)
* pass request body to logging context in all cases
* add message size logging on the websocket API
this is required by graphql-engine-pro/#416
* message size logging on websocket API
As we need to log all messages recieved/sent by the websocket server,
it makes sense to log them as part of the websocket server event logs.
Previously message recieved were logged inside the onMessage handler,
and messages sent were logged only for "data" messages (as a server event log)
* fix review comments
Co-authored-by: Phil Freeman <phil@hasura.io>
* server: stop eventing subsystem threads when shutting down (#5479)
* server: stop eventing subsystem threads when shutting down
* Apply suggestions from code review
Co-authored-by: Karthikeyan Chinnakonda <chkarthikeyan95@gmail.com>
Co-authored-by: Phil Freeman <phil@hasura.io>
Co-authored-by: Phil Freeman <paf31@cantab.net>
Co-authored-by: Karthikeyan Chinnakonda <chkarthikeyan95@gmail.com>
* [skip ci] update commit_diff with new commits added in master
* Bugfix to support 0-size HASURA_GRAPHQL_QUERY_PLAN_CACHE_SIZE
Also some minor refactoring of bounded cache module:
- the maxBound check in `trim` was confusing and unnecessary
- consequently trim was unnecessary for lookupPure
Also add some basic tests
* Support only the bounded cache, with default HASURA_GRAPHQL_QUERY_PLAN_CACHE_SIZE of 4000. Closes #5363
* [skip ci] remove merge commit from commit_diff
* server: Fix compiler warning caused by GHC upgrade (#5489)
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
* [skip ci] update all non server code from master
* [skip ci] aligned object field error message with master
* [skip ci] fix remaining undefined?
* [skip ci] remove unused import
* [skip ci] revert to previous error message, fix tests
* Move nullableType/nonNullableType to Schema.hs
These are functions on Types, not on Parsers.
* [skip ci] fix setup to fix backend only test
the order in which permission checks are performed on the branch is
slightly different than on master, resulting in a slightly different
error if there are no other mutations the user has access to. By
adding update permissions, we go back to the expected case.
* [skip ci] fix insert geojson tests to reflect new paths
* [skip ci] fix enum test for better error message
* [skip ci] fix header test for better error message
* [skip ci] fix fragment cycle test for better error message
* [skip ci] fix error message for type mismatch
* [skip ci] fix variable path in test
* [skip ci] adjust tests after bug fix
* [skip ci] more tests fixing
* Add hdb_catalog.current_setting abstraction for reading Hasura settings
As the comment in the function’s definition explains, this is needed to
work around an awkward Postgres behavior.
* [skip ci] Update CONTRIBUTING.md to mention Node setup for Python tests
* [skip ci] Add missing Python tests env var to CONTRIBUTING.md
* [skip ci] fix order of result when subscription is run with multiple nodes
* [skip ci] no-op refactor: fix a warning in Internal/Parser.hs
* [skip ci] throw error when a subscription contains remote joins
* [skip ci] Enable easier profiling by hiding AssertNF behind a flag
In order to compile a profiling build, run:
$ cabal new-build -f profiling --enable-profiling
* [skip ci] Fix two warnings
We used to lookup the objects that implement a given interface by filtering all
objects in the schema document. However, one of the tests expects us to
generate a warning if the provided `implements` field of an introspection query
specifies an object not implementing some interface. So we use that field
instead.
* [skip ci] Fix warnings by commenting out query plan caching
* [skip ci] improve masking/commenting query caching related code & few warning fixes
* [skip ci] Fixed compiler warnings in graphql-parser-hs
* Sync non-Haskell assets with master
* [skip ci] add a test inserting invalid GraphQL but valid JSON value in a jsonb column
* [skip ci] Avoid converting to/from Map
* [skip ci] Apply some hlint suggestions
* [skip ci] remove redundant constraints from buildLiveQueryPlan and explainGQLQuery
* [skip ci] add NOTEs about missing Tracing constraints in PDV from master
* Remove -fdefer-typed-holes, fix warnings
* Update cabal.project.freeze
* Limit GHC’s heap size to 8GB in CI to avoid the OOM killer
* Commit package-lock.json for Python tests’ remote schema server
* restrict env variables start with HASURA_GRAPHQL_ for headers configuration in actions, event triggers & remote schemas (#5519)
* restrict env variables start with HASURA_GRAPHQL_ for headers definition in actions & event triggers
* update CHANGELOG.md
* Apply suggestions from code review
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
* add test for table_by_pk node when roles doesn't have permission to PK
* [skip ci] fix introspection query if any enum column present in primary key (fix #5200) (#5522)
* [skip ci] test case fix for a6450e126bc2d98bcfd3791501986e4627ce6c6f
* [skip ci] add tests to agg queries when role doesn't have access to any cols
* fix backend test
* Simplify subscription execution
* [skip ci] add test to check if required headers are present while querying
* Suppose, table B is related to table A and to query B certain headers are
necessary, then the test checks that we are throwing error when the header
is not set when B is queried through A
* fix mutations not checking for view mutability
* [skip ci] add variable type checking and corresponding tests
* [skip ci] add test to check if update headers are present while doing an upsert
* [skip ci] add positive counterparts to some of the negative permission tests
* fix args missing their description in introspect
* [skip ci] Remove unused function; insert missing markNotReusable call
* [skip ci] Add a Note about InputValue
* [skip ci] Delete LegacySchema/ 🎉
* [skip ci] Delete GraphQL/{Resolve,Validate}/ 🎉
* [skip ci] Delete top-level Resolve/Validate modules; tidy .cabal file
* [skip ci] Delete LegacySchema top-level module
Somehow I missed this one.
* fix input value to json
* [skip ci] elaborate on JSON objects in GraphQL
* [skip ci] add missing file
* [skip ci] add a test with subscription containing remote joins
* add a test with remote joins in mutation output
* [skip ci] Add some comments to Schema/Mutation.hs
* [skip ci] Remove no longer needed code from RemoteServer.hs
* [skip ci] Use a helper function to generate conflict clause parsers
* [skip ci] fix type checker error in fields with default value
* capitalize the header keys in select_articles_without_required_headers
* Somehow, this was the reason the tests were failing. I have no idea, why!
* [skip ci] Add a long Note about optional fields and nullability
* Improve comments a bit; simplify Schema/Common.hs a bit
* [skip ci] full implementation of 5.8.5 type checking.
* [skip ci] fix validation test teardown
* [skip ci] fix schema stitching test
* fix remote schema ignoring enum nullability
* [skip ci] fix fieldOptional to not discard nullability
* revert nullability of use_spheroid
* fix comment
* add required remote fields with arguments for tests
* [skip ci] add missing docstrings
* [skip ci] fixed description of remote fields
* [skip ci] change docstring for consistency
* fix several schema inconsistencies
* revert behaviour change in function arguments parsing
* fix remaining nullability issues in new schema
* minor no-op refactor; use isListType from graphql-parser-hs
* use nullability of remote schema node, while creating a Remote reln
* fix 'ID' input coercing & action 'ID' type relationship mapping
* include ASTs in MonadExecuteQuery
* needed for PRO code-base
* Delete code for "interfaces implementing ifaces" (draft GraphQL spec)
Previously I started writing some code that adds support for a future GraphQL
feature where interfaces may themselves be sub-types of other interfaces.
However, this code was incomplete, and partially incorrect. So this commit
deletes support for that entirely.
* Ignore a remote schema test during the upgrade/downgrade test
The PDV refactor does a better job at exposing a minimal set of types through
introspection. In particular, not every type that is present in a remote schema
is re-exposed by Hasura. The test
test_schema_stitching.py::TestRemoteSchemaBasic::test_introspection assumed that
all types were re-exposed, which is not required for GraphQL compatibility, in
order to test some aspect of our support for remote schemas.
So while this particular test has been updated on PDV, the PDV branch now does
not pass the old test, which we argue to be incorrect. Hence this test is
disabled while we await a release, after which we can re-enable it.
This also re-enables a test that was previously disabled for similar, though
unrelated, reasons.
* add haddock documentation to the action's field parsers
* Deslecting some tests in server-upgrade
Some tests with current build are failing on server upgrade
which it should not. The response is more accurate than
what it was.
Also the upgrade tests were not throwing errors when the test is
expected to return an error, but succeeds. The test framework is
patched to catch this case.
* [skip ci] Add a long Note about interfaces and object types
* send the response headers back to client after running a query
* Deselect a few more tests during upgrade/downgrade test
* Update commit_diff.txt
* change log kind from db_migrate to catalog_migrate (#5531)
* Show method and complete URI in traced HTTP calls (#5525)
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
* restrict env variables start with HASURA_GRAPHQL_ for headers configuration in actions, event triggers & remote schemas (#5519)
* restrict env variables start with HASURA_GRAPHQL_ for headers definition in actions & event triggers
* update CHANGELOG.md
* Apply suggestions from code review
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
* fix introspection query if any enum column present in primary key (fix #5200) (#5522)
* Fix telemetry reporting of transport (websocket was reported as http)
* add log kinds in cli-migrations image (#5529)
* add log kinds in cli-migrations image
* give hint to resolve timeout error
* minor changes and CHANGELOG
* server: set hasura.tracecontext in RQL mutations [#5542] (#5555)
* server: set hasura.tracecontext in RQL mutations [#5542]
* Update test suite
Co-authored-by: Tirumarai Selvan <tiru@hasura.io>
* Add bulldozer auto-merge and -update configuration
We still need to add the github app (as of time of opening this PR)
Afterwards devs should be able to allow bulldozer to automatically
"update" the branch, merging in parent when it changes, as well as
automatically merge when all checks pass.
This is opt-in by adding the `auto-update-auto-merge` label to the PR.
* Remove 'bulldozer' config, try 'kodiak' for auto-merge
see: https://github.com/chdsbd/kodiak
The main issue that bit us was not being able to auto update forked
branches, also:
https://github.com/palantir/bulldozer/issues/66
https://github.com/palantir/bulldozer/issues/145
* Cherry-picked all commits
* [skip ci] Slightly improve formatting
* Revert "fix introspection query if any enum column present in primary key (fix #5200) (#5522)"
This reverts commit 0f9a5afa59a88f6824f4d63d58db246a5ba3fb03.
This undoes a cherry-pick of 34288e1eb5f2c5dad9e6d1e05453dd52397dc970 that was
already done previously in a6450e126bc2d98bcfd3791501986e4627ce6c6f, and
subsequently fixed for PDV in 70e89dc250f8ddc6e2b7930bbe2b3eeaa6dbe1db
* Do a small bit of tidying in Hasura.GraphQL.Parser.Collect
* Fix cherry-picking work
Some previous cherry-picks ended up modifying code that is commented out
* [skip ci] clarified comment regarding insert representation
* [skip ci] removed obsolete todos
* cosmetic change
* fix action error message
* [skip ci] remove obsolete comment
* [skip ci] synchronize stylish haskell extensions list
* use previously defined scalar names in parsers rather than ad-hoc literals
* Apply most syntax hlint hints.
* Clarify comment on update mutation.
* [skip ci] Clarify what fields should be specified for objects
* Update "_inc" description.
* Use record types rather than tuples fo IntrospectionResult and ParsedIntrospection
* Get rid of checkFieldNamesUnique (use Data.List.Extended.duplicates)
* Throw more errors when collecting query root names
* [skip ci] clean column parser comment
* Remove dead code inserted in ab65b39
* avoid converting to non-empty list where not needed
* add note and TODO about the disabled checks in PDV
* minor refactor in remoteField' function
* Unify two getObject methods
* Nitpicks in Remote.hs
* Update CHANGELOG.md
* Revert "Unify two getObject methods"
This reverts commit bd6bb40355b3d189a46c0312eb52225e18be57b3.
We do need two different getObject functions as the corresponding error message is different
* Fix error message in Remote.hs
* Update CHANGELOG.md
Co-authored-by: Auke Booij <auke@tulcod.com>
* Apply suggested Changelog fix.
Co-authored-by: Auke Booij <auke@tulcod.com>
* Fix typo in Changelog.
* [skip ci] Update changelog.
* reuse type names to avoid duplication
* Fix Hashable instance for Definition
The presence of `Maybe Unique`, and an optional description, as part of
`Definition`s, means that `Definition`s that are considered `Eq`ual may get
different hashes. This can happen, for instance, when one object is memoized
but another is not.
* [skip ci] Update commit_diff.txt
* Bump parser version.
* Bump freeze file after changes in parser.
* [skip ci] Incorporate commits from master
* Fix developer flag in server/cabal.project.freeze
Co-authored-by: Auke Booij <auke@tulcod.com>
* Deselect a changed ENUM test for upgrade/downgrade CI
* Deselect test here as well
* [skip ci] remove dead code
* Disable more tests for upgrade/downgrade
* Fix which test gets deselected
* Revert "Add hdb_catalog.current_setting abstraction for reading Hasura settings"
This reverts commit 66e85ab9fbd56cca2c28a80201f6604fbe811b85.
* Remove circular reference in cabal.project.freeze
Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
Co-authored-by: Auke Booij <auke@hasura.io>
Co-authored-by: Tirumarai Selvan <tiru@hasura.io>
Co-authored-by: Marion Schleifer <marion@hasura.io>
Co-authored-by: Aleksandra Sikora <ola.zxcvbnm@gmail.com>
Co-authored-by: Brandon Simmons <brandon.m.simmons@gmail.com>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
Co-authored-by: Anon Ray <rayanon004@gmail.com>
Co-authored-by: rakeshkky <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Anon Ray <ecthiender@users.noreply.github.com>
Co-authored-by: Vamshi Surabhi <vamshi@hasura.io>
Co-authored-by: Antoine Leblanc <antoine@hasura.io>
Co-authored-by: Brandon Simmons <brandon@hasura.io>
Co-authored-by: Phil Freeman <phil@hasura.io>
Co-authored-by: Lyndon Maydwell <lyndon@sordina.net>
Co-authored-by: Phil Freeman <paf31@cantab.net>
Co-authored-by: Naveen Naidu <naveennaidu479@gmail.com>
Co-authored-by: Karthikeyan Chinnakonda <chkarthikeyan95@gmail.com>
Co-authored-by: Nizar Malangadan <nizar-m@users.noreply.github.com>
Co-authored-by: Antoine Leblanc <crucuny@gmail.com>
Co-authored-by: Auke Booij <auke@tulcod.com>
2020-08-21 20:27:01 +03:00
import Hasura.Prelude hiding ( get , put )
2022-09-09 11:26:44 +03:00
import Hasura.RQL.DDL.EventTrigger ( MonadEventLogCleanup )
2019-08-14 02:34:37 +03:00
import Hasura.RQL.DDL.Schema
2022-04-27 16:57:28 +03:00
import Hasura.RQL.Types.Common
2022-03-13 10:40:06 +03:00
import Hasura.RQL.Types.Endpoint as EP
2022-10-20 15:45:31 +03:00
import Hasura.RQL.Types.Metadata ( MetadataDefaults )
2022-04-27 16:57:28 +03:00
import Hasura.RQL.Types.SchemaCache
import Hasura.RQL.Types.Source
import Hasura.SQL.Backend
2020-07-15 13:40:48 +03:00
import Hasura.Server.API.Config ( runGetConfig )
2020-12-14 07:30:19 +03:00
import Hasura.Server.API.Metadata
2020-10-27 16:53:49 +03:00
import Hasura.Server.API.PGDump qualified as PGD
2018-07-03 18:34:25 +03:00
import Hasura.Server.API.Query
2021-01-07 12:04:22 +03:00
import Hasura.Server.API.V2Query qualified as V2Q
2021-03-13 17:40:50 +03:00
import Hasura.Server.Auth ( AuthMode ( .. ) , UserAuthentication ( .. ) )
2020-06-19 09:42:32 +03:00
import Hasura.Server.Compression
2021-01-07 12:04:22 +03:00
import Hasura.Server.Cors
2020-04-24 10:55:51 +03:00
import Hasura.Server.Init
2021-09-29 19:20:06 +03:00
import Hasura.Server.Limits
2020-06-19 09:42:32 +03:00
import Hasura.Server.Logging
import Hasura.Server.Metrics ( ServerMetrics )
import Hasura.Server.Middleware ( corsMiddleware )
Rewrite OpenAPI
### Description
This PR rewrites OpenAPI to be more idiomatic. Some noteworthy changes:
- we accumulate all required information during the Analyze phase, to avoid having to do a single lookup in the schema cache during the OpenAPI generation phase (we now only need the schema cache as input to run the analysis)
- we no longer build intermediary endpoint information and aggregate it, we directly build the the `PathItem` for each endpoint; additionally, that means we no longer have to assume that different methods have the same metadata
- we no longer have to first declare types, then craft references: we do everything in one step
- we now properly deal with nullability by treating "typeName" and "typeName!" as different
- we add a bunch of additional fields in the generated "schema", such as title
- we do now support enum values in both input and output positions
- checking whether the request body is required is now performed on the fly rather than by introspecting the generated schema
- the methods in the file are sorted by topic
### Controversial point
However, this PR creates some additional complexity, that we might not want to keep. The main complexity is _knot-tying_: to avoid lookups when generating the OpenAPI, it builds an actual graph of input types, which means that we need something similar to (but simpler than) `MonadSchema`, to avoid infinite recursions when analyzing the input types of a query. To do this, this PR introduces `CircularT`, a lesser `SchemaT` that aims at avoiding ever having to reinvent this particular wheel ever again.
### Remaining work
- [x] fix existing tests (they are all failing due to some of the schema changes)
- [ ] add tests to cover the new features:
- [x] tests for `CircularT`
- [ ] tests for enums in output schemas
- [x] extract / document `CircularT` if we wish to keep it
- [x] add more comments to `OpenAPI`
- [x] have a second look at `buildVariableSchema`
- [x] fix all missing diagnostics in `Analyze`
- [x] add a Changelog entry?
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4654
Co-authored-by: David Overton <7734777+dmoverton@users.noreply.github.com>
GitOrigin-RevId: f4a9191f22dfcc1dccefd6a52f5c586b6ad17172
2022-06-30 15:55:56 +03:00
import Hasura.Server.OpenAPI ( buildOpenAPI )
2022-07-24 00:18:01 +03:00
import Hasura.Server.Prometheus ( PrometheusMetrics )
2019-09-19 15:54:40 +03:00
import Hasura.Server.Rest
2022-03-09 01:59:28 +03:00
import Hasura.Server.SchemaCacheRef
( SchemaCacheRef ,
getSchemaCache ,
readSchemaCacheRef ,
withSchemaCacheUpdate ,
)
2019-02-14 08:58:38 +03:00
import Hasura.Server.Types
import Hasura.Server.Utils
2018-07-03 18:34:25 +03:00
import Hasura.Server.Version
2019-09-19 15:54:40 +03:00
import Hasura.Session
2023-01-06 12:33:13 +03:00
import Hasura.ShutdownLatch
2018-06-27 16:11:32 +03:00
import Hasura.Tracing qualified as Tracing
2021-08-06 00:07:17 +03:00
import Network.HTTP.Client qualified as HTTP
2020-11-25 13:56:44 +03:00
import Network.HTTP.Types qualified as HTTP
2018-06-27 16:11:32 +03:00
import Network.Mime ( defaultMimeLookup )
2021-07-29 11:29:12 +03:00
import Network.Wai.Extended qualified as Wai
import Network.Wai.Handler.WebSockets.Custom qualified as WSC
2020-06-19 09:42:32 +03:00
import Network.WebSockets qualified as WS
2020-10-27 16:53:49 +03:00
import System.FilePath ( joinPath , takeFileName )
2021-08-06 02:37:02 +03:00
import System.Mem ( performMajorGC )
2020-06-19 09:42:32 +03:00
import System.Metrics qualified as EKG
2021-07-29 11:29:12 +03:00
import System.Metrics.Json qualified as EKG
import Text.Mustache qualified as M
2020-10-27 16:53:49 +03:00
import Web.Spock.Core ( ( <//> ) )
2020-06-19 09:42:32 +03:00
import Web.Spock.Core qualified as Spock
2021-09-24 01:56:37 +03:00
2018-06-27 16:11:32 +03:00
data ServerCtx = ServerCtx
2023-01-06 12:33:13 +03:00
{ scLoggers :: ! Loggers ,
2020-04-24 10:55:51 +03:00
scCacheRef :: ! SchemaCacheRef ,
scAuthMode :: ! AuthMode ,
scManager :: ! HTTP . Manager ,
scSQLGenCtx :: ! SQLGenCtx ,
scEnabledAPIs :: ! ( S . HashSet API ) ,
scInstanceId :: ! InstanceId ,
2022-03-21 13:39:49 +03:00
scSubscriptionState :: ! ES . SubscriptionsState ,
2020-04-24 10:55:51 +03:00
scEnableAllowlist :: ! Bool ,
scResponseInternalErrorsConfig :: ! ResponseInternalErrorsConfig ,
2020-07-14 22:00:58 +03:00
scEnvironment :: ! Env . Environment ,
2022-07-14 20:57:28 +03:00
scRemoteSchemaPermsCtx :: ! Options . RemoteSchemaPermissions ,
scFunctionPermsCtx :: ! Options . InferFunctionPermissions ,
2022-04-28 23:55:13 +03:00
scEnableMaintenanceMode :: ! ( MaintenanceMode () ) ,
[Preview] Inherited roles for postgres read queries
fixes #3868
docker image - `hasura/graphql-engine:inherited-roles-preview-48b73a2de`
Note:
To be able to use the inherited roles feature, the graphql-engine should be started with the env variable `HASURA_GRAPHQL_EXPERIMENTAL_FEATURES` set to `inherited_roles`.
Introduction
------------
This PR implements the idea of multiple roles as presented in this [paper](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/FGALanguageICDE07.pdf). The multiple roles feature in this PR can be used via inherited roles. An inherited role is a role which can be created by combining multiple singular roles. For example, if there are two roles `author` and `editor` configured in the graphql-engine, then we can create a inherited role with the name of `combined_author_editor` role which will combine the select permissions of the `author` and `editor` roles and then make GraphQL queries using the `combined_author_editor`.
How are select permissions of different roles are combined?
------------------------------------------------------------
A select permission includes 5 things:
1. Columns accessible to the role
2. Row selection filter
3. Limit
4. Allow aggregation
5. Scalar computed fields accessible to the role
Suppose there are two roles, `role1` gives access to the `address` column with row filter `P1` and `role2` gives access to both the `address` and the `phone` column with row filter `P2` and we create a new role `combined_roles` which combines `role1` and `role2`.
Let's say the following GraphQL query is queried with the `combined_roles` role.
```graphql
query {
employees {
address
phone
}
}
```
This will translate to the following SQL query:
```sql
select
(case when (P1 or P2) then address else null end) as address,
(case when P2 then phone else null end) as phone
from employee
where (P1 or P2)
```
The other parameters of the select permission will be combined in the following manner:
1. Limit - Minimum of the limits will be the limit of the inherited role
2. Allow aggregations - If any of the role allows aggregation, then the inherited role will allow aggregation
3. Scalar computed fields - same as table column fields, as in the above example
APIs for inherited roles:
----------------------
1. `add_inherited_role`
`add_inherited_role` is the [metadata API](https://hasura.io/docs/1.0/graphql/core/api-reference/index.html#schema-metadata-api) to create a new inherited role. It accepts two arguments
`role_name`: the name of the inherited role to be added (String)
`role_set`: list of roles that need to be combined (Array of Strings)
Example:
```json
{
"type": "add_inherited_role",
"args": {
"role_name":"combined_user",
"role_set":[
"user",
"user1"
]
}
}
```
After adding the inherited role, the inherited role can be used like single roles like earlier
Note:
An inherited role can only be created with non-inherited/singular roles.
2. `drop_inherited_role`
The `drop_inherited_role` API accepts the name of the inherited role and drops it from the metadata. It accepts a single argument:
`role_name`: name of the inherited role to be dropped
Example:
```json
{
"type": "drop_inherited_role",
"args": {
"role_name":"combined_user"
}
}
```
Metadata
---------
The derived roles metadata will be included under the `experimental_features` key while exporting the metadata.
```json
{
"experimental_features": {
"derived_roles": [
{
"role_name": "manager_is_employee_too",
"role_set": [
"employee",
"manager"
]
}
]
}
}
```
Scope
------
Only postgres queries and subscriptions are supported in this PR.
Important points:
-----------------
1. All columns exposed to an inherited role will be marked as `nullable`, this is done so that cell value nullification can be done.
TODOs
-------
- [ ] Tests
- [ ] Test a GraphQL query running with a inherited role without enabling inherited roles in experimental features
- [] Tests for aggregate queries, limit, computed fields, functions, subscriptions (?)
- [ ] Introspection test with a inherited role (nullability changes in a inherited role)
- [ ] Docs
- [ ] Changelog
Co-authored-by: Vamshi Surabhi <6562944+0x777@users.noreply.github.com>
GitOrigin-RevId: 3b8ee1e11f5ceca80fe294f8c074d42fbccfec63
2021-03-08 14:14:13 +03:00
scExperimentalFeatures :: ! ( S . HashSet ExperimentalFeature ) ,
2022-04-11 20:49:25 +03:00
scLoggingSettings :: ! LoggingSettings ,
2021-12-08 09:26:46 +03:00
scEventingMode :: ! EventingMode ,
2022-05-26 14:54:30 +03:00
scEnableReadOnlyMode :: ! ReadOnlyMode ,
2022-07-24 00:18:01 +03:00
scDefaultNamingConvention :: ! ( Maybe NamingCase ) ,
2023-01-06 12:33:13 +03:00
scServerMetrics :: ! ServerMetrics ,
2022-12-22 22:47:17 +03:00
scMetadataDefaults :: ! MetadataDefaults ,
2023-01-06 12:33:13 +03:00
scEnabledLogTypes :: HashSet ( L . EngineLogType L . Hasura ) ,
scMetadataDbPool :: PG . PGPool ,
scShutdownLatch :: ShutdownLatch ,
scMetaVersionRef :: STM . TMVar MetadataResourceVersion ,
scPrometheusMetrics :: PrometheusMetrics ,
2023-01-23 16:35:48 +03:00
scTraceSamplingPolicy :: Tracing . SamplingPolicy ,
scCheckFeatureFlag :: ! ( FeatureFlag -> IO Bool )
2023-01-06 12:33:13 +03:00
}
-- | Collection of the LoggerCtx, the regular Logger and the PGLogger
-- TODO (from master): better naming?
data Loggers = Loggers
{ _lsLoggerCtx :: ! ( L . LoggerCtx L . Hasura ) ,
_lsLogger :: ! ( L . Logger L . Hasura ) ,
_lsPgLogger :: ! PG . PGLogger
2018-06-27 16:11:32 +03:00
}
data HandlerCtx = HandlerCtx
2020-06-16 18:23:06 +03:00
{ hcServerCtx :: ! ServerCtx ,
hcUser :: ! UserInfo ,
hcReqHeaders :: ! [ HTTP . Header ] ,
hcRequestId :: ! RequestId ,
hcSourceIpAddress :: ! Wai . IpAddress
2018-06-27 16:11:32 +03:00
}
2020-12-14 07:30:19 +03:00
type Handler m = ReaderT HandlerCtx ( MetadataStorageT m )
2018-06-27 16:11:32 +03:00
2019-04-30 11:34:08 +03:00
data APIResp
2019-06-04 13:10:28 +03:00
= JSONResp ! ( HttpResponse EncJSON )
2019-07-11 08:37:06 +03:00
| RawResp ! ( HttpResponse BL . ByteString )
2019-04-30 11:34:08 +03:00
2021-02-03 10:10:39 +03:00
-- | API request handlers for different endpoints
data APIHandler m a where
-- | A simple GET request
2022-12-15 10:48:18 +03:00
AHGet :: ! ( Handler m ( HttpLogGraphQLInfo , APIResp ) ) -> APIHandler m void
2021-02-03 10:10:39 +03:00
-- | A simple POST request that expects a request body from which an 'a' can be extracted
2022-12-15 10:48:18 +03:00
AHPost :: ! ( a -> Handler m ( HttpLogGraphQLInfo , APIResp ) ) -> APIHandler m a
2021-02-03 10:10:39 +03:00
-- | A general GraphQL request (query or mutation) for which the content of the query
-- is made available to the handler for authentication.
-- This is a more specific version of the 'AHPost' constructor.
2022-12-15 10:48:18 +03:00
AHGraphQLRequest :: ! ( GH . ReqsText -> Handler m ( HttpLogGraphQLInfo , APIResp ) ) -> APIHandler m GH . ReqsText
2019-07-11 08:37:06 +03:00
2020-10-27 16:53:49 +03:00
boolToText :: Bool -> Text
2019-11-26 15:14:21 +03:00
boolToText = bool " false " " true "
2020-10-27 16:53:49 +03:00
isAdminSecretSet :: AuthMode -> Text
2019-11-26 15:14:21 +03:00
isAdminSecretSet AMNoAuth = boolToText False
isAdminSecretSet _ = boolToText True
2022-12-15 10:48:18 +03:00
mkGetHandler :: Handler m ( HttpLogGraphQLInfo , APIResp ) -> APIHandler m ()
2019-07-11 08:37:06 +03:00
mkGetHandler = AHGet
2022-12-15 10:48:18 +03:00
mkPostHandler :: ( a -> Handler m ( HttpLogGraphQLInfo , APIResp ) ) -> APIHandler m a
2019-07-11 08:37:06 +03:00
mkPostHandler = AHPost
2022-12-15 10:48:18 +03:00
mkGQLRequestHandler :: ( GH . ReqsText -> Handler m ( HttpLogGraphQLInfo , APIResp ) ) -> APIHandler m GH . ReqsText
2021-02-03 10:10:39 +03:00
mkGQLRequestHandler = AHGraphQLRequest
2019-11-26 15:14:21 +03:00
mkAPIRespHandler :: ( Functor m ) => ( a -> Handler m ( HttpResponse EncJSON ) ) -> ( a -> Handler m APIResp )
2019-07-11 08:37:06 +03:00
mkAPIRespHandler = ( fmap . fmap ) JSONResp
2019-04-30 11:34:08 +03:00
2021-04-06 20:52:55 +03:00
mkGQLAPIRespHandler ::
( Functor m ) =>
( a -> Handler m ( b , ( HttpResponse EncJSON ) ) ) ->
( a -> Handler m ( b , APIResp ) )
mkGQLAPIRespHandler = ( fmap . fmap . fmap ) JSONResp
2019-02-28 16:53:03 +03:00
isMetadataEnabled :: ServerCtx -> Bool
isMetadataEnabled sc = S . member METADATA $ scEnabledAPIs sc
isGraphQLEnabled :: ServerCtx -> Bool
isGraphQLEnabled sc = S . member GRAPHQL $ scEnabledAPIs sc
2019-04-30 11:34:08 +03:00
isPGDumpEnabled :: ServerCtx -> Bool
isPGDumpEnabled sc = S . member PGDUMP $ scEnabledAPIs sc
2019-06-11 16:29:03 +03:00
isConfigEnabled :: ServerCtx -> Bool
isConfigEnabled sc = S . member CONFIG $ scEnabledAPIs sc
2019-04-30 08:15:23 +03:00
isDeveloperAPIEnabled :: ServerCtx -> Bool
isDeveloperAPIEnabled sc = S . member DEVELOPER $ scEnabledAPIs sc
2018-07-20 10:22:46 +03:00
-- {-# SCC parseBody #-}
2021-07-05 21:47:09 +03:00
parseBody :: ( FromJSON a , MonadError QErr m ) => BL . ByteString -> m ( Value , a )
2019-07-11 08:37:06 +03:00
parseBody reqBody =
case eitherDecode' reqBody of
Left e -> throw400 InvalidJSON ( T . pack e )
2021-07-05 21:47:09 +03:00
Right jVal -> ( jVal , ) <$> decodeValue jVal
2018-06-27 16:11:32 +03:00
2020-12-14 07:30:19 +03:00
onlyAdmin :: ( MonadError QErr m , MonadReader HandlerCtx m ) => m ()
2018-06-27 16:11:32 +03:00
onlyAdmin = do
2020-04-24 12:10:53 +03:00
uRole <- asks ( _uiRole . hcUser )
[Preview] Inherited roles for postgres read queries
fixes #3868
docker image - `hasura/graphql-engine:inherited-roles-preview-48b73a2de`
Note:
To be able to use the inherited roles feature, the graphql-engine should be started with the env variable `HASURA_GRAPHQL_EXPERIMENTAL_FEATURES` set to `inherited_roles`.
Introduction
------------
This PR implements the idea of multiple roles as presented in this [paper](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/FGALanguageICDE07.pdf). The multiple roles feature in this PR can be used via inherited roles. An inherited role is a role which can be created by combining multiple singular roles. For example, if there are two roles `author` and `editor` configured in the graphql-engine, then we can create a inherited role with the name of `combined_author_editor` role which will combine the select permissions of the `author` and `editor` roles and then make GraphQL queries using the `combined_author_editor`.
How are select permissions of different roles are combined?
------------------------------------------------------------
A select permission includes 5 things:
1. Columns accessible to the role
2. Row selection filter
3. Limit
4. Allow aggregation
5. Scalar computed fields accessible to the role
Suppose there are two roles, `role1` gives access to the `address` column with row filter `P1` and `role2` gives access to both the `address` and the `phone` column with row filter `P2` and we create a new role `combined_roles` which combines `role1` and `role2`.
Let's say the following GraphQL query is queried with the `combined_roles` role.
```graphql
query {
employees {
address
phone
}
}
```
This will translate to the following SQL query:
```sql
select
(case when (P1 or P2) then address else null end) as address,
(case when P2 then phone else null end) as phone
from employee
where (P1 or P2)
```
The other parameters of the select permission will be combined in the following manner:
1. Limit - Minimum of the limits will be the limit of the inherited role
2. Allow aggregations - If any of the role allows aggregation, then the inherited role will allow aggregation
3. Scalar computed fields - same as table column fields, as in the above example
APIs for inherited roles:
----------------------
1. `add_inherited_role`
`add_inherited_role` is the [metadata API](https://hasura.io/docs/1.0/graphql/core/api-reference/index.html#schema-metadata-api) to create a new inherited role. It accepts two arguments
`role_name`: the name of the inherited role to be added (String)
`role_set`: list of roles that need to be combined (Array of Strings)
Example:
```json
{
"type": "add_inherited_role",
"args": {
"role_name":"combined_user",
"role_set":[
"user",
"user1"
]
}
}
```
After adding the inherited role, the inherited role can be used like single roles like earlier
Note:
An inherited role can only be created with non-inherited/singular roles.
2. `drop_inherited_role`
The `drop_inherited_role` API accepts the name of the inherited role and drops it from the metadata. It accepts a single argument:
`role_name`: name of the inherited role to be dropped
Example:
```json
{
"type": "drop_inherited_role",
"args": {
"role_name":"combined_user"
}
}
```
Metadata
---------
The derived roles metadata will be included under the `experimental_features` key while exporting the metadata.
```json
{
"experimental_features": {
"derived_roles": [
{
"role_name": "manager_is_employee_too",
"role_set": [
"employee",
"manager"
]
}
]
}
}
```
Scope
------
Only postgres queries and subscriptions are supported in this PR.
Important points:
-----------------
1. All columns exposed to an inherited role will be marked as `nullable`, this is done so that cell value nullification can be done.
TODOs
-------
- [ ] Tests
- [ ] Test a GraphQL query running with a inherited role without enabling inherited roles in experimental features
- [] Tests for aggregate queries, limit, computed fields, functions, subscriptions (?)
- [ ] Introspection test with a inherited role (nullability changes in a inherited role)
- [ ] Docs
- [ ] Changelog
Co-authored-by: Vamshi Surabhi <6562944+0x777@users.noreply.github.com>
GitOrigin-RevId: 3b8ee1e11f5ceca80fe294f8c074d42fbccfec63
2021-03-08 14:14:13 +03:00
unless ( uRole == adminRoleName ) $
2018-06-27 16:11:32 +03:00
throw400 AccessDenied " You have to be an admin to access this endpoint "
2020-03-20 09:46:45 +03:00
setHeader :: MonadIO m => HTTP . Header -> Spock . ActionT m ()
setHeader ( headerName , headerValue ) =
Spock . setHeader ( bsToTxt $ CI . original headerName ) ( bsToTxt headerValue )
2019-11-26 15:14:21 +03:00
-- | Typeclass representing the metadata API authorization effect
2021-01-07 12:04:22 +03:00
class ( Monad m ) => MonadMetadataApiAuthorization m where
authorizeV1QueryApi ::
2021-10-13 19:38:56 +03:00
RQLQuery -> HandlerCtx -> m ( Either QErr () )
2020-12-14 07:30:19 +03:00
2021-01-07 12:04:22 +03:00
authorizeV1MetadataApi ::
2021-10-13 19:38:56 +03:00
RQLMetadata -> HandlerCtx -> m ( Either QErr () )
2020-12-14 07:30:19 +03:00
2021-01-07 12:04:22 +03:00
authorizeV2QueryApi ::
2021-10-13 19:38:56 +03:00
V2Q . RQLQuery -> HandlerCtx -> m ( Either QErr () )
2020-06-16 18:23:06 +03:00
2021-01-07 12:04:22 +03:00
instance MonadMetadataApiAuthorization m => MonadMetadataApiAuthorization ( ReaderT r m ) where
authorizeV1QueryApi q hc = lift $ authorizeV1QueryApi q hc
authorizeV1MetadataApi q hc = lift $ authorizeV1MetadataApi q hc
authorizeV2QueryApi q hc = lift $ authorizeV2QueryApi q hc
instance MonadMetadataApiAuthorization m => MonadMetadataApiAuthorization ( MetadataStorageT m ) where
authorizeV1QueryApi q hc = lift $ authorizeV1QueryApi q hc
authorizeV1MetadataApi q hc = lift $ authorizeV1MetadataApi q hc
authorizeV2QueryApi q hc = lift $ authorizeV2QueryApi q hc
instance MonadMetadataApiAuthorization m => MonadMetadataApiAuthorization ( Tracing . TraceT m ) where
authorizeV1QueryApi q hc = lift $ authorizeV1QueryApi q hc
authorizeV1MetadataApi q hc = lift $ authorizeV1MetadataApi q hc
authorizeV2QueryApi q hc = lift $ authorizeV2QueryApi q hc
2020-07-15 13:40:48 +03:00
2020-06-16 18:23:06 +03:00
-- | The config API (/v1alpha1/config) handler
class Monad m => MonadConfigApiHandler m where
runConfigApiHandler ::
ServerCtx ->
-- | console assets directory
Maybe Text ->
Spock . SpockCtxT () m ()
2019-06-04 13:10:28 +03:00
2020-07-15 13:40:48 +03:00
-- instance (MonadIO m, UserAuthentication m, HttpLog m, Tracing.HasReporter m) => MonadConfigApiHandler (Tracing.TraceT m) where
-- runConfigApiHandler = configApiGetHandler
mapActionT ::
( Monad m , Monad n ) =>
( m ( MTC . StT ( Spock . ActionCtxT () ) a ) -> n ( MTC . StT ( Spock . ActionCtxT () ) a ) ) ->
Spock . ActionT m a ->
Spock . ActionT n a
2020-10-28 19:40:33 +03:00
mapActionT f tma = MTC . restoreT . pure =<< MTC . liftWith ( \ run -> f ( run tma ) )
2020-07-15 13:40:48 +03:00
2018-06-27 16:11:32 +03:00
mkSpockAction ::
2022-12-15 10:48:18 +03:00
forall m a .
( MonadIO m ,
MonadBaseControl IO m ,
FromJSON a ,
UserAuthentication ( Tracing . TraceT m ) ,
HttpLog m ,
Tracing . HasReporter m ,
HasResourceLimits m
) =>
2019-11-26 15:14:21 +03:00
ServerCtx ->
-- | `QErr` JSON encoder function
2019-05-10 09:05:11 +03:00
( Bool -> QErr -> Value ) ->
2019-11-26 15:14:21 +03:00
-- | `QErr` modifier
2021-07-05 12:45:31 +03:00
( QErr -> QErr ) ->
2020-07-15 13:40:48 +03:00
APIHandler ( Tracing . TraceT m ) a ->
2019-11-26 15:14:21 +03:00
Spock . ActionT m ()
2021-07-05 21:47:09 +03:00
mkSpockAction serverCtx @ ServerCtx { .. } qErrEncoder qErrModifier apiHandler = do
req <- Spock . request
let origHeaders = Wai . requestHeaders req
ipAddress = Wai . getSourceFromFallback req
pathInfo = Wai . rawPathInfo req
-- Bytes are actually read from the socket here. Time this.
( ioWaitTime , reqBody ) <- withElapsedTime $ liftIO $ Wai . strictRequestBody req
( requestId , headers ) <- getRequestId origHeaders
2022-09-20 05:48:21 +03:00
tracingCtx <- liftIO $ Tracing . extractB3HttpContext headers
2021-09-29 19:20:06 +03:00
handlerLimit <- lift askHTTPHandlerLimit
2021-07-05 21:47:09 +03:00
let runTraceT ::
2022-12-15 10:48:18 +03:00
forall m1 a1 .
( MonadIO m1 , Tracing . HasReporter m1 ) =>
Tracing . TraceT m1 a1 ->
m1 a1
2022-12-22 22:47:17 +03:00
runTraceT = do
( maybe Tracing . runTraceT Tracing . runTraceTInContext tracingCtx )
scTraceSamplingPolicy
2021-07-05 21:47:09 +03:00
( fromString ( B8 . unpack pathInfo ) )
2021-09-24 01:56:37 +03:00
2021-07-05 21:47:09 +03:00
runHandler ::
2022-12-15 10:48:18 +03:00
MonadBaseControl IO m2 =>
2021-07-05 21:47:09 +03:00
HandlerCtx ->
2022-12-15 10:48:18 +03:00
ReaderT HandlerCtx ( MetadataStorageT m2 ) a2 ->
m2 ( Either QErr a2 )
2021-09-29 19:20:06 +03:00
runHandler handlerCtx handler =
runMetadataStorageT $ flip runReaderT handlerCtx $ runResourceLimits handlerLimit $ handler
2021-07-05 21:47:09 +03:00
getInfo parsedRequest = do
2023-01-06 12:33:13 +03:00
authenticationResp <- lift ( resolveUserInfo ( _lsLogger scLoggers ) scManager headers scAuthMode parsedRequest )
2022-12-15 10:48:18 +03:00
authInfo <- onLeft authenticationResp ( logErrorAndResp Nothing requestId req ( reqBody , Nothing ) False origHeaders ( ExtraUserInfo Nothing ) . qErrModifier )
let ( userInfo , _ , authHeaders , extraUserInfo ) = authInfo
2021-07-05 21:47:09 +03:00
pure
( userInfo ,
2021-11-09 15:00:21 +03:00
authHeaders ,
2021-07-05 21:47:09 +03:00
HandlerCtx serverCtx userInfo headers requestId ipAddress ,
2022-12-15 10:48:18 +03:00
shouldIncludeInternal ( _uiRole userInfo ) scResponseInternalErrorsConfig ,
extraUserInfo
2021-07-05 21:47:09 +03:00
)
mapActionT runTraceT $ do
-- Add the request ID to the tracing metadata so that we
-- can correlate requests and traces
lift $ Tracing . attachMetadata [ ( " request_id " , unRequestId requestId ) ]
2022-12-15 10:48:18 +03:00
( serviceTime , ( result , userInfo , authHeaders , includeInternal , queryJSON , extraUserInfo ) ) <- withElapsedTime $ case apiHandler of
2021-07-05 21:47:09 +03:00
-- in the case of a simple get/post we don't have to send the webhook anything
AHGet handler -> do
2022-12-15 10:48:18 +03:00
( userInfo , authHeaders , handlerState , includeInternal , extraUserInfo ) <- getInfo Nothing
2021-07-05 21:47:09 +03:00
res <- lift $ runHandler handlerState handler
2022-12-15 10:48:18 +03:00
pure ( res , userInfo , authHeaders , includeInternal , Nothing , extraUserInfo )
2021-07-05 21:47:09 +03:00
AHPost handler -> do
2022-12-15 10:48:18 +03:00
( userInfo , authHeaders , handlerState , includeInternal , extraUserInfo ) <- getInfo Nothing
2021-07-05 21:47:09 +03:00
( queryJSON , parsedReq ) <-
2022-12-15 10:48:18 +03:00
runExcept ( parseBody reqBody ) ` onLeft ` \ e -> do
logErrorAndResp ( Just userInfo ) requestId req ( reqBody , Nothing ) includeInternal origHeaders extraUserInfo ( qErrModifier e )
2021-07-05 21:47:09 +03:00
res <- lift $ runHandler handlerState $ handler parsedReq
2022-12-15 10:48:18 +03:00
pure ( res , userInfo , authHeaders , includeInternal , Just queryJSON , extraUserInfo )
2021-07-05 21:47:09 +03:00
-- in this case we parse the request _first_ and then send the request to the webhook for auth
AHGraphQLRequest handler -> do
( queryJSON , parsedReq ) <-
2021-10-28 21:42:50 +03:00
runExcept ( parseBody reqBody ) ` onLeft ` \ e -> do
-- if the request fails to parse, call the webhook without a request body
-- TODO should we signal this to the webhook somehow?
2022-12-15 10:48:18 +03:00
( userInfo , _ , _ , _ , extraUserInfo ) <- getInfo Nothing
logErrorAndResp ( Just userInfo ) requestId req ( reqBody , Nothing ) False origHeaders extraUserInfo ( qErrModifier e )
( userInfo , authHeaders , handlerState , includeInternal , extraUserInfo ) <- getInfo ( Just parsedReq )
2021-11-09 15:00:21 +03:00
2021-07-05 21:47:09 +03:00
res <- lift $ runHandler handlerState $ handler parsedReq
2022-12-15 10:48:18 +03:00
pure ( res , userInfo , authHeaders , includeInternal , Just queryJSON , extraUserInfo )
2021-07-05 21:47:09 +03:00
-- apply the error modifier
let modResult = fmapL qErrModifier result
-- log and return result
case modResult of
Left err ->
2022-12-15 10:48:18 +03:00
logErrorAndResp ( Just userInfo ) requestId req ( reqBody , queryJSON ) includeInternal headers extraUserInfo err
Right ( httpLogGraphQLInfo , res ) -> do
let httpLogMetadata = buildHttpLogMetadata @ m httpLogGraphQLInfo extraUserInfo
logSuccessAndResp ( Just userInfo ) requestId req ( reqBody , queryJSON ) res ( Just ( ioWaitTime , serviceTime ) ) origHeaders authHeaders httpLogMetadata
2021-07-05 21:47:09 +03:00
where
logErrorAndResp ::
2022-12-15 10:48:18 +03:00
forall m3 a3 ctx .
( MonadIO m3 , HttpLog m3 ) =>
2021-07-05 21:47:09 +03:00
Maybe UserInfo ->
RequestId ->
Wai . Request ->
( BL . ByteString , Maybe Value ) ->
Bool ->
[ HTTP . Header ] ->
2022-12-15 10:48:18 +03:00
ExtraUserInfo ->
2021-07-05 21:47:09 +03:00
QErr ->
2022-12-15 10:48:18 +03:00
Spock . ActionCtxT ctx m3 a3
logErrorAndResp userInfo reqId waiReq req includeInternal headers extraUserInfo qErr = do
let httpLogMetadata = buildHttpLogMetadata @ m3 emptyHttpLogGraphQLInfo extraUserInfo
2023-01-06 12:33:13 +03:00
lift $ logHttpError ( _lsLogger scLoggers ) scLoggingSettings userInfo reqId waiReq req qErr headers httpLogMetadata
2021-07-05 21:47:09 +03:00
Spock . setStatus $ qeStatus qErr
Spock . json $ qErrEncoder includeInternal qErr
2021-11-09 15:00:21 +03:00
logSuccessAndResp userInfo reqId waiReq req result qTime reqHeaders authHdrs httpLoggingMetadata = do
2021-07-05 21:47:09 +03:00
let ( respBytes , respHeaders ) = case result of
JSONResp ( HttpResponse encJson h ) -> ( encJToLBS encJson , pure jsonHeader <> h )
RawResp ( HttpResponse rawBytes h ) -> ( rawBytes , h )
2022-12-09 09:06:48 +03:00
( compressedResp , encodingType ) = compressResponse ( Wai . requestHeaders waiReq ) respBytes
encodingHeader = maybeToList ( contentEncodingHeader <$> encodingType )
2021-07-05 21:47:09 +03:00
reqIdHeader = ( requestIdHeader , txtToBs $ unRequestId reqId )
2021-11-09 15:00:21 +03:00
allRespHeaders = pure reqIdHeader <> encodingHeader <> respHeaders <> authHdrs
2023-01-06 12:33:13 +03:00
lift $ logHttpSuccess ( _lsLogger scLoggers ) scLoggingSettings userInfo reqId waiReq req respBytes compressedResp qTime encodingType reqHeaders httpLoggingMetadata
2021-07-05 21:47:09 +03:00
mapM_ setHeader allRespHeaders
Spock . lazyBytes compressedResp
2020-03-20 09:46:45 +03:00
v1QueryHandler ::
2021-10-13 19:38:56 +03:00
( MonadIO m ,
2021-01-07 12:04:22 +03:00
MonadBaseControl IO m ,
MonadMetadataApiAuthorization m ,
Tracing . MonadTrace m ,
2021-07-29 11:29:12 +03:00
MonadReader HandlerCtx m ,
MonadMetadataStorage m ,
MonadResolveSource m ,
2022-09-15 14:45:14 +03:00
EB . MonadQueryTags m ,
MonadEventLogCleanup m
2020-07-14 22:00:58 +03:00
) =>
RQLQuery ->
2020-12-14 07:30:19 +03:00
m ( HttpResponse EncJSON )
2018-06-27 16:11:32 +03:00
v1QueryHandler query = do
2021-01-07 12:04:22 +03:00
( liftEitherM . authorizeV1QueryApi query ) =<< ask
2020-07-14 22:00:58 +03:00
scRef <- asks ( scCacheRef . hcServerCtx )
2023-01-06 12:33:13 +03:00
logger <- asks ( _lsLogger . scLoggers . hcServerCtx )
2022-03-09 01:59:28 +03:00
res <- bool ( fst <$> ( action logger ) ) ( withSchemaCacheUpdate scRef logger Nothing ( action logger ) ) $ queryModifiesSchemaCache query
2020-03-20 09:46:45 +03:00
return $ HttpResponse res []
2018-06-27 16:11:32 +03:00
where
2021-09-09 14:54:19 +03:00
action logger = do
2021-01-29 08:48:17 +03:00
userInfo <- asks hcUser
scRef <- asks ( scCacheRef . hcServerCtx )
2022-10-20 15:45:31 +03:00
metadataDefaults <- asks ( scMetadataDefaults . hcServerCtx )
2022-03-09 01:59:28 +03:00
schemaCache <- liftIO $ fst <$> readSchemaCacheRef scRef
2021-01-29 08:48:17 +03:00
httpMgr <- asks ( scManager . hcServerCtx )
sqlGenCtx <- asks ( scSQLGenCtx . hcServerCtx )
instanceId <- asks ( scInstanceId . hcServerCtx )
env <- asks ( scEnvironment . hcServerCtx )
2020-12-21 12:11:37 +03:00
remoteSchemaPermsCtx <- asks ( scRemoteSchemaPermsCtx . hcServerCtx )
2021-01-29 08:48:17 +03:00
functionPermsCtx <- asks ( scFunctionPermsCtx . hcServerCtx )
2021-02-18 19:46:14 +03:00
maintenanceMode <- asks ( scEnableMaintenanceMode . hcServerCtx )
[Preview] Inherited roles for postgres read queries
fixes #3868
docker image - `hasura/graphql-engine:inherited-roles-preview-48b73a2de`
Note:
To be able to use the inherited roles feature, the graphql-engine should be started with the env variable `HASURA_GRAPHQL_EXPERIMENTAL_FEATURES` set to `inherited_roles`.
Introduction
------------
This PR implements the idea of multiple roles as presented in this [paper](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/FGALanguageICDE07.pdf). The multiple roles feature in this PR can be used via inherited roles. An inherited role is a role which can be created by combining multiple singular roles. For example, if there are two roles `author` and `editor` configured in the graphql-engine, then we can create a inherited role with the name of `combined_author_editor` role which will combine the select permissions of the `author` and `editor` roles and then make GraphQL queries using the `combined_author_editor`.
How are select permissions of different roles are combined?
------------------------------------------------------------
A select permission includes 5 things:
1. Columns accessible to the role
2. Row selection filter
3. Limit
4. Allow aggregation
5. Scalar computed fields accessible to the role
Suppose there are two roles, `role1` gives access to the `address` column with row filter `P1` and `role2` gives access to both the `address` and the `phone` column with row filter `P2` and we create a new role `combined_roles` which combines `role1` and `role2`.
Let's say the following GraphQL query is queried with the `combined_roles` role.
```graphql
query {
employees {
address
phone
}
}
```
This will translate to the following SQL query:
```sql
select
(case when (P1 or P2) then address else null end) as address,
(case when P2 then phone else null end) as phone
from employee
where (P1 or P2)
```
The other parameters of the select permission will be combined in the following manner:
1. Limit - Minimum of the limits will be the limit of the inherited role
2. Allow aggregations - If any of the role allows aggregation, then the inherited role will allow aggregation
3. Scalar computed fields - same as table column fields, as in the above example
APIs for inherited roles:
----------------------
1. `add_inherited_role`
`add_inherited_role` is the [metadata API](https://hasura.io/docs/1.0/graphql/core/api-reference/index.html#schema-metadata-api) to create a new inherited role. It accepts two arguments
`role_name`: the name of the inherited role to be added (String)
`role_set`: list of roles that need to be combined (Array of Strings)
Example:
```json
{
"type": "add_inherited_role",
"args": {
"role_name":"combined_user",
"role_set":[
"user",
"user1"
]
}
}
```
After adding the inherited role, the inherited role can be used like single roles like earlier
Note:
An inherited role can only be created with non-inherited/singular roles.
2. `drop_inherited_role`
The `drop_inherited_role` API accepts the name of the inherited role and drops it from the metadata. It accepts a single argument:
`role_name`: name of the inherited role to be dropped
Example:
```json
{
"type": "drop_inherited_role",
"args": {
"role_name":"combined_user"
}
}
```
Metadata
---------
The derived roles metadata will be included under the `experimental_features` key while exporting the metadata.
```json
{
"experimental_features": {
"derived_roles": [
{
"role_name": "manager_is_employee_too",
"role_set": [
"employee",
"manager"
]
}
]
}
}
```
Scope
------
Only postgres queries and subscriptions are supported in this PR.
Important points:
-----------------
1. All columns exposed to an inherited role will be marked as `nullable`, this is done so that cell value nullification can be done.
TODOs
-------
- [ ] Tests
- [ ] Test a GraphQL query running with a inherited role without enabling inherited roles in experimental features
- [] Tests for aggregate queries, limit, computed fields, functions, subscriptions (?)
- [ ] Introspection test with a inherited role (nullability changes in a inherited role)
- [ ] Docs
- [ ] Changelog
Co-authored-by: Vamshi Surabhi <6562944+0x777@users.noreply.github.com>
GitOrigin-RevId: 3b8ee1e11f5ceca80fe294f8c074d42fbccfec63
2021-03-08 14:14:13 +03:00
experimentalFeatures <- asks ( scExperimentalFeatures . hcServerCtx )
2021-11-30 15:31:27 +03:00
eventingMode <- asks ( scEventingMode . hcServerCtx )
2021-12-08 09:26:46 +03:00
readOnlyMode <- asks ( scEnableReadOnlyMode . hcServerCtx )
2022-05-26 14:54:30 +03:00
defaultNamingCase <- asks ( scDefaultNamingConvention . hcServerCtx )
2023-01-23 16:35:48 +03:00
checkFeatureFlag <- asks ( scCheckFeatureFlag . hcServerCtx )
2022-04-22 22:53:12 +03:00
let serverConfigCtx =
ServerConfigCtx
functionPermsCtx
remoteSchemaPermsCtx
sqlGenCtx
maintenanceMode
experimentalFeatures
eventingMode
readOnlyMode
2022-05-26 14:54:30 +03:00
defaultNamingCase
2022-10-20 15:45:31 +03:00
metadataDefaults
2023-01-23 16:35:48 +03:00
checkFeatureFlag
2021-09-09 14:54:19 +03:00
runQuery
env
logger
instanceId
userInfo
schemaCache
httpMgr
2021-02-18 19:46:14 +03:00
serverConfigCtx
query
2021-09-24 01:56:37 +03:00
2021-01-07 12:04:22 +03:00
v1MetadataHandler ::
2021-10-13 19:38:56 +03:00
( MonadIO m ,
2021-01-07 12:04:22 +03:00
MonadBaseControl IO m ,
MonadReader HandlerCtx m ,
Tracing . MonadTrace m ,
MonadMetadataStorage m ,
MonadResolveSource m ,
2022-09-09 11:26:44 +03:00
MonadMetadataApiAuthorization m ,
MonadEventLogCleanup m
2021-01-07 12:04:22 +03:00
) =>
RQLMetadata ->
m ( HttpResponse EncJSON )
2022-08-19 00:56:47 +03:00
v1MetadataHandler query = Tracing . trace " Metadata " $ do
2021-01-07 12:04:22 +03:00
( liftEitherM . authorizeV1MetadataApi query ) =<< ask
2021-01-29 08:48:17 +03:00
userInfo <- asks hcUser
scRef <- asks ( scCacheRef . hcServerCtx )
2022-03-09 01:59:28 +03:00
schemaCache <- liftIO $ fst <$> readSchemaCacheRef scRef
2021-01-29 08:48:17 +03:00
httpMgr <- asks ( scManager . hcServerCtx )
2021-12-08 09:26:46 +03:00
_sccSQLGenCtx <- asks ( scSQLGenCtx . hcServerCtx )
2021-01-29 08:48:17 +03:00
env <- asks ( scEnvironment . hcServerCtx )
instanceId <- asks ( scInstanceId . hcServerCtx )
2023-01-06 12:33:13 +03:00
logger <- asks ( _lsLogger . scLoggers . hcServerCtx )
2021-12-08 09:26:46 +03:00
_sccRemoteSchemaPermsCtx <- asks ( scRemoteSchemaPermsCtx . hcServerCtx )
_sccFunctionPermsCtx <- asks ( scFunctionPermsCtx . hcServerCtx )
_sccExperimentalFeatures <- asks ( scExperimentalFeatures . hcServerCtx )
_sccMaintenanceMode <- asks ( scEnableMaintenanceMode . hcServerCtx )
_sccEventingMode <- asks ( scEventingMode . hcServerCtx )
_sccReadOnlyMode <- asks ( scEnableReadOnlyMode . hcServerCtx )
2022-05-26 14:54:30 +03:00
_sccDefaultNamingConvention <- asks ( scDefaultNamingConvention . hcServerCtx )
2022-10-20 15:45:31 +03:00
_sccMetadataDefaults <- asks ( scMetadataDefaults . hcServerCtx )
2023-01-23 16:35:48 +03:00
_sccCheckFeatureFlag <- asks ( scCheckFeatureFlag . hcServerCtx )
2021-12-08 09:26:46 +03:00
let serverConfigCtx = ServerConfigCtx { .. }
2021-07-27 08:41:16 +03:00
r <-
2022-03-09 01:59:28 +03:00
withSchemaCacheUpdate
2021-07-27 08:41:16 +03:00
scRef
2021-07-27 18:14:12 +03:00
logger
2021-09-24 01:56:37 +03:00
Nothing
2021-07-27 08:41:16 +03:00
$ runMetadataQuery
2021-09-24 01:56:37 +03:00
env
logger
2021-07-27 08:41:16 +03:00
instanceId
userInfo
httpMgr
serverConfigCtx
schemaCache
query
2021-01-07 12:04:22 +03:00
pure $ HttpResponse r []
v2QueryHandler ::
2021-10-13 19:38:56 +03:00
( MonadIO m ,
2021-01-07 12:04:22 +03:00
MonadBaseControl IO m ,
MonadMetadataApiAuthorization m ,
Tracing . MonadTrace m ,
MonadReader HandlerCtx m ,
MonadMetadataStorage m ,
MonadResolveSource m ,
2021-07-29 11:29:12 +03:00
EB . MonadQueryTags m
2021-01-07 12:04:22 +03:00
) =>
V2Q . RQLQuery ->
m ( HttpResponse EncJSON )
2022-08-19 00:56:47 +03:00
v2QueryHandler query = Tracing . trace " v2 Query " $ do
2021-01-07 12:04:22 +03:00
( liftEitherM . authorizeV2QueryApi query ) =<< ask
scRef <- asks ( scCacheRef . hcServerCtx )
2023-01-06 12:33:13 +03:00
logger <- asks ( _lsLogger . scLoggers . hcServerCtx )
2021-07-27 08:41:16 +03:00
res <-
2022-03-09 01:59:28 +03:00
bool ( fst <$> dbAction ) ( withSchemaCacheUpdate scRef logger Nothing dbAction ) $
2021-01-07 12:04:22 +03:00
V2Q . queryModifiesSchema query
return $ HttpResponse res []
where
-- Hit postgres
dbAction = do
userInfo <- asks hcUser
scRef <- asks ( scCacheRef . hcServerCtx )
2022-03-09 01:59:28 +03:00
schemaCache <- liftIO $ fst <$> readSchemaCacheRef scRef
2021-01-07 12:04:22 +03:00
httpMgr <- asks ( scManager . hcServerCtx )
sqlGenCtx <- asks ( scSQLGenCtx . hcServerCtx )
instanceId <- asks ( scInstanceId . hcServerCtx )
env <- asks ( scEnvironment . hcServerCtx )
remoteSchemaPermsCtx <- asks ( scRemoteSchemaPermsCtx . hcServerCtx )
[Preview] Inherited roles for postgres read queries
fixes #3868
docker image - `hasura/graphql-engine:inherited-roles-preview-48b73a2de`
Note:
To be able to use the inherited roles feature, the graphql-engine should be started with the env variable `HASURA_GRAPHQL_EXPERIMENTAL_FEATURES` set to `inherited_roles`.
Introduction
------------
This PR implements the idea of multiple roles as presented in this [paper](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/FGALanguageICDE07.pdf). The multiple roles feature in this PR can be used via inherited roles. An inherited role is a role which can be created by combining multiple singular roles. For example, if there are two roles `author` and `editor` configured in the graphql-engine, then we can create a inherited role with the name of `combined_author_editor` role which will combine the select permissions of the `author` and `editor` roles and then make GraphQL queries using the `combined_author_editor`.
How are select permissions of different roles are combined?
------------------------------------------------------------
A select permission includes 5 things:
1. Columns accessible to the role
2. Row selection filter
3. Limit
4. Allow aggregation
5. Scalar computed fields accessible to the role
Suppose there are two roles, `role1` gives access to the `address` column with row filter `P1` and `role2` gives access to both the `address` and the `phone` column with row filter `P2` and we create a new role `combined_roles` which combines `role1` and `role2`.
Let's say the following GraphQL query is queried with the `combined_roles` role.
```graphql
query {
employees {
address
phone
}
}
```
This will translate to the following SQL query:
```sql
select
(case when (P1 or P2) then address else null end) as address,
(case when P2 then phone else null end) as phone
from employee
where (P1 or P2)
```
The other parameters of the select permission will be combined in the following manner:
1. Limit - Minimum of the limits will be the limit of the inherited role
2. Allow aggregations - If any of the role allows aggregation, then the inherited role will allow aggregation
3. Scalar computed fields - same as table column fields, as in the above example
APIs for inherited roles:
----------------------
1. `add_inherited_role`
`add_inherited_role` is the [metadata API](https://hasura.io/docs/1.0/graphql/core/api-reference/index.html#schema-metadata-api) to create a new inherited role. It accepts two arguments
`role_name`: the name of the inherited role to be added (String)
`role_set`: list of roles that need to be combined (Array of Strings)
Example:
```json
{
"type": "add_inherited_role",
"args": {
"role_name":"combined_user",
"role_set":[
"user",
"user1"
]
}
}
```
After adding the inherited role, the inherited role can be used like single roles like earlier
Note:
An inherited role can only be created with non-inherited/singular roles.
2. `drop_inherited_role`
The `drop_inherited_role` API accepts the name of the inherited role and drops it from the metadata. It accepts a single argument:
`role_name`: name of the inherited role to be dropped
Example:
```json
{
"type": "drop_inherited_role",
"args": {
"role_name":"combined_user"
}
}
```
Metadata
---------
The derived roles metadata will be included under the `experimental_features` key while exporting the metadata.
```json
{
"experimental_features": {
"derived_roles": [
{
"role_name": "manager_is_employee_too",
"role_set": [
"employee",
"manager"
]
}
]
}
}
```
Scope
------
Only postgres queries and subscriptions are supported in this PR.
Important points:
-----------------
1. All columns exposed to an inherited role will be marked as `nullable`, this is done so that cell value nullification can be done.
TODOs
-------
- [ ] Tests
- [ ] Test a GraphQL query running with a inherited role without enabling inherited roles in experimental features
- [] Tests for aggregate queries, limit, computed fields, functions, subscriptions (?)
- [ ] Introspection test with a inherited role (nullability changes in a inherited role)
- [ ] Docs
- [ ] Changelog
Co-authored-by: Vamshi Surabhi <6562944+0x777@users.noreply.github.com>
GitOrigin-RevId: 3b8ee1e11f5ceca80fe294f8c074d42fbccfec63
2021-03-08 14:14:13 +03:00
experimentalFeatures <- asks ( scExperimentalFeatures . hcServerCtx )
2021-01-29 08:48:17 +03:00
functionPermsCtx <- asks ( scFunctionPermsCtx . hcServerCtx )
2021-02-18 19:46:14 +03:00
maintenanceMode <- asks ( scEnableMaintenanceMode . hcServerCtx )
2021-11-30 15:31:27 +03:00
eventingMode <- asks ( scEventingMode . hcServerCtx )
2021-12-08 09:26:46 +03:00
readOnlyMode <- asks ( scEnableReadOnlyMode . hcServerCtx )
2022-05-26 14:54:30 +03:00
defaultNamingCase <- asks ( scDefaultNamingConvention . hcServerCtx )
2022-10-20 15:45:31 +03:00
defaultMetadata <- asks ( scMetadataDefaults . hcServerCtx )
2023-01-23 16:35:48 +03:00
checkFeatureFlag <- asks ( scCheckFeatureFlag . hcServerCtx )
2022-04-22 22:53:12 +03:00
let serverConfigCtx =
ServerConfigCtx
functionPermsCtx
remoteSchemaPermsCtx
sqlGenCtx
maintenanceMode
experimentalFeatures
eventingMode
readOnlyMode
2022-05-26 14:54:30 +03:00
defaultNamingCase
2022-10-20 15:45:31 +03:00
defaultMetadata
2023-01-23 16:35:48 +03:00
checkFeatureFlag
2022-04-22 22:53:12 +03:00
2021-02-18 19:46:14 +03:00
V2Q . runQuery env instanceId userInfo schemaCache httpMgr serverConfigCtx query
2021-01-07 12:04:22 +03:00
2020-04-24 10:55:51 +03:00
v1Alpha1GQHandler ::
2021-10-13 19:38:56 +03:00
( MonadIO m ,
2020-12-28 15:56:00 +03:00
MonadBaseControl IO m ,
2020-07-29 16:30:29 +03:00
E . MonadGQLExecutionCheck m ,
MonadQueryLog m ,
Tracing . MonadTrace m ,
GH . MonadExecuteQuery m ,
2020-12-14 07:30:19 +03:00
MonadError QErr m ,
MonadReader HandlerCtx m ,
MonadMetadataStorage ( MetadataStorageT m ) ,
2021-09-29 19:20:06 +03:00
EB . MonadQueryTags m ,
HasResourceLimits m
2020-07-29 16:30:29 +03:00
) =>
2021-09-15 11:29:34 +03:00
E . GraphQLQueryType ->
GH . GQLBatchedReqs ( GH . GQLReq GH . GQLQueryText ) ->
2022-12-15 10:48:18 +03:00
m ( HttpLogGraphQLInfo , HttpResponse EncJSON )
2020-06-08 15:13:01 +03:00
v1Alpha1GQHandler queryType query = do
2020-07-14 22:00:58 +03:00
userInfo <- asks hcUser
reqHeaders <- asks hcReqHeaders
ipAddress <- asks hcSourceIpAddress
requestId <- asks hcRequestId
2023-01-06 12:33:13 +03:00
logger <- asks ( _lsLogger . scLoggers . hcServerCtx )
2020-06-16 18:23:06 +03:00
responseErrorsConfig <- asks ( scResponseInternalErrorsConfig . hcServerCtx )
2020-07-14 22:00:58 +03:00
env <- asks ( scEnvironment . hcServerCtx )
2021-12-08 09:26:46 +03:00
execCtx <- mkExecutionContext
2020-07-14 22:00:58 +03:00
2020-06-08 15:13:01 +03:00
flip runReaderT execCtx $
2020-07-29 16:30:29 +03:00
GH . runGQBatched env logger requestId responseErrorsConfig userInfo ipAddress reqHeaders queryType query
2019-07-11 08:37:06 +03:00
2021-01-29 04:02:34 +03:00
mkExecutionContext ::
( MonadIO m ,
MonadReader HandlerCtx m
) =>
m E . ExecutionCtx
mkExecutionContext = do
manager <- asks ( scManager . hcServerCtx )
scRef <- asks ( scCacheRef . hcServerCtx )
2022-03-09 01:59:28 +03:00
( sc , scVer ) <- liftIO $ readSchemaCacheRef scRef
2021-01-29 04:02:34 +03:00
sqlGenCtx <- asks ( scSQLGenCtx . hcServerCtx )
enableAL <- asks ( scEnableAllowlist . hcServerCtx )
2023-01-06 12:33:13 +03:00
logger <- asks ( _lsLogger . scLoggers . hcServerCtx )
2021-12-08 09:26:46 +03:00
readOnlyMode <- asks ( scEnableReadOnlyMode . hcServerCtx )
2022-07-24 00:18:01 +03:00
prometheusMetrics <- asks ( scPrometheusMetrics . hcServerCtx )
pure $ E . ExecutionCtx logger sqlGenCtx ( lastBuiltSchemaCache sc ) scVer manager enableAL readOnlyMode prometheusMetrics
2021-01-29 04:02:34 +03:00
2019-07-11 08:37:06 +03:00
v1GQHandler ::
2021-10-13 19:38:56 +03:00
( MonadIO m ,
2020-12-28 15:56:00 +03:00
MonadBaseControl IO m ,
2020-07-29 16:30:29 +03:00
E . MonadGQLExecutionCheck m ,
MonadQueryLog m ,
Tracing . MonadTrace m ,
GH . MonadExecuteQuery m ,
2020-12-14 07:30:19 +03:00
MonadError QErr m ,
MonadReader HandlerCtx m ,
MonadMetadataStorage ( MetadataStorageT m ) ,
2021-09-29 19:20:06 +03:00
EB . MonadQueryTags m ,
HasResourceLimits m
2020-07-29 16:30:29 +03:00
) =>
2021-09-15 11:29:34 +03:00
GH . GQLBatchedReqs ( GH . GQLReq GH . GQLQueryText ) ->
2022-12-15 10:48:18 +03:00
m ( HttpLogGraphQLInfo , HttpResponse EncJSON )
2020-06-08 15:13:01 +03:00
v1GQHandler = v1Alpha1GQHandler E . QueryHasura
v1GQRelayHandler ::
2021-10-13 19:38:56 +03:00
( MonadIO m ,
2020-12-28 15:56:00 +03:00
MonadBaseControl IO m ,
2020-07-29 16:30:29 +03:00
E . MonadGQLExecutionCheck m ,
MonadQueryLog m ,
Tracing . MonadTrace m ,
GH . MonadExecuteQuery m ,
2020-12-14 07:30:19 +03:00
MonadError QErr m ,
MonadReader HandlerCtx m ,
MonadMetadataStorage ( MetadataStorageT m ) ,
2021-09-29 19:20:06 +03:00
EB . MonadQueryTags m ,
HasResourceLimits m
2020-07-29 16:30:29 +03:00
) =>
2021-09-15 11:29:34 +03:00
GH . GQLBatchedReqs ( GH . GQLReq GH . GQLQueryText ) ->
2022-12-15 10:48:18 +03:00
m ( HttpLogGraphQLInfo , HttpResponse EncJSON )
2020-06-08 15:13:01 +03:00
v1GQRelayHandler = v1Alpha1GQHandler E . QueryRelay
2019-05-10 09:05:11 +03:00
2020-06-16 18:23:06 +03:00
gqlExplainHandler ::
2020-12-14 07:30:19 +03:00
forall m .
( MonadIO m ,
2020-12-28 15:56:00 +03:00
MonadBaseControl IO m ,
2020-12-14 07:30:19 +03:00
MonadError QErr m ,
MonadReader HandlerCtx m ,
2021-03-31 13:39:01 +03:00
MonadMetadataStorage ( MetadataStorageT m ) ,
2021-07-29 11:29:12 +03:00
EB . MonadQueryTags m
2020-12-14 07:30:19 +03:00
) =>
2020-07-14 22:00:58 +03:00
GE . GQLExplain ->
2020-12-14 07:30:19 +03:00
m ( HttpResponse EncJSON )
2018-10-19 05:15:28 +03:00
gqlExplainHandler query = do
onlyAdmin
2021-09-15 23:45:49 +03:00
scRef <- asks ( scCacheRef . hcServerCtx )
2022-03-09 01:59:28 +03:00
sc <- liftIO $ getSchemaCache scRef
2020-12-28 15:56:00 +03:00
res <- GE . explainGQLQuery sc query
2020-03-20 09:46:45 +03:00
return $ HttpResponse res []
2018-06-27 16:11:32 +03:00
2020-12-14 07:30:19 +03:00
v1Alpha1PGDumpHandler :: ( MonadIO m , MonadError QErr m , MonadReader HandlerCtx m ) => PGD . PGDumpReqBody -> m APIResp
2019-04-30 11:34:08 +03:00
v1Alpha1PGDumpHandler b = do
onlyAdmin
2021-01-20 03:31:53 +03:00
scRef <- asks ( scCacheRef . hcServerCtx )
2022-03-09 01:59:28 +03:00
sc <- liftIO $ getSchemaCache scRef
2021-02-23 20:37:27 +03:00
let sources = scSources sc
2021-01-20 03:31:53 +03:00
sourceName = PGD . prbSource b
2021-04-22 00:44:37 +03:00
sourceConfig = unsafeSourceConfiguration @ ( 'Postgres 'Vanilla ) =<< M . lookup sourceName sources
2021-01-20 03:31:53 +03:00
ci <-
fmap _pscConnInfo sourceConfig
` onNothing ` throw400 NotFound ( " source " <> sourceName <<> " not found " )
2019-04-30 11:34:08 +03:00
output <- PGD . execPGDump b ci
2020-03-20 09:46:45 +03:00
return $ RawResp $ HttpResponse output [ sqlHeader ]
2019-05-16 10:45:29 +03:00
2019-11-26 15:14:21 +03:00
consoleAssetsHandler ::
( MonadIO m , HttpLog m ) =>
L . Logger L . Hasura ->
2022-04-11 20:49:25 +03:00
LoggingSettings ->
2019-11-26 15:14:21 +03:00
Text ->
FilePath ->
Spock . ActionT m ()
2022-04-11 20:49:25 +03:00
consoleAssetsHandler logger loggingSettings dir path = do
2019-11-26 15:14:21 +03:00
req <- Spock . request
let reqHeaders = Wai . requestHeaders req
2019-05-16 10:45:29 +03:00
-- '..' in paths need not be handed as it is resolved in the url by
-- spock's routing. we get the expanded path.
eFileContents <-
liftIO $
try $
BL . readFile $
joinPath [ T . unpack dir , path ]
2019-11-26 15:14:21 +03:00
either ( onError reqHeaders ) onSuccess eFileContents
2019-05-16 10:45:29 +03:00
where
2019-05-16 14:28:51 +03:00
onSuccess c = do
2020-03-20 09:46:45 +03:00
mapM_ setHeader headers
2019-11-26 15:14:21 +03:00
Spock . lazyBytes c
onError :: ( MonadIO m , HttpLog m ) => [ HTTP . Header ] -> IOException -> Spock . ActionT m ()
2022-04-11 20:49:25 +03:00
onError hdrs = raiseGenericApiError logger loggingSettings hdrs . err404 NotFound . tshow
2019-05-16 10:45:29 +03:00
fn = T . pack $ takeFileName path
-- set gzip header if the filename ends with .gz
( fileName , encHeader ) = case T . stripSuffix " .gz " fn of
Just v -> ( v , [ gzipHeader ] )
Nothing -> ( fn , [] )
2020-03-20 09:46:45 +03:00
mimeType = defaultMimeLookup fileName
2019-05-16 10:45:29 +03:00
headers = ( " Content-Type " , mimeType ) : encHeader
2019-11-26 15:14:21 +03:00
class ( Monad m ) => ConsoleRenderer m where
2022-09-27 17:50:32 +03:00
renderConsole :: Text -> AuthMode -> Bool -> Maybe Text -> Maybe Text -> m ( Either String Text )
2019-11-26 15:14:21 +03:00
2020-07-15 13:40:48 +03:00
instance ConsoleRenderer m => ConsoleRenderer ( Tracing . TraceT m ) where
2022-09-27 17:50:32 +03:00
renderConsole a b c d e = lift $ renderConsole a b c d e
2020-07-15 13:40:48 +03:00
2022-12-07 14:28:58 +03:00
-- Type class to get any extra [Pair] for the version API
class ( Monad m ) => MonadVersionAPIWithExtraData m where
getExtraDataForVersionAPI :: m [ J . Pair ]
2019-11-26 15:14:21 +03:00
renderHtmlTemplate :: M . Template -> Value -> Either String Text
renderHtmlTemplate template jVal =
2019-05-16 14:28:51 +03:00
bool ( Left errMsg ) ( Right res ) $ null errs
2019-05-16 10:45:29 +03:00
where
2019-11-26 15:14:21 +03:00
errMsg = " template rendering failed: " ++ show errs
( errs , res ) = M . checkedSubstitute template jVal
2020-06-16 18:23:06 +03:00
-- | Default implementation of the 'MonadConfigApiHandler'
configApiGetHandler ::
2021-07-05 12:45:31 +03:00
forall m .
2021-10-13 19:38:56 +03:00
( MonadIO m , MonadBaseControl IO m , UserAuthentication ( Tracing . TraceT m ) , HttpLog m , Tracing . HasReporter m , HasResourceLimits m ) =>
2020-06-16 18:23:06 +03:00
ServerCtx ->
Maybe Text ->
Spock . SpockCtxT () m ()
configApiGetHandler serverCtx @ ServerCtx { .. } consoleAssetsDir =
Spock . get " v1alpha1/config " $
mkSpockAction serverCtx encodeQErr id $
mkGetHandler $ do
onlyAdmin
2021-03-04 19:21:27 +03:00
let res =
runGetConfig
scFunctionPermsCtx
scRemoteSchemaPermsCtx
scAuthMode
scEnableAllowlist
2022-03-21 13:39:49 +03:00
( ES . _ssLiveQueryOptions $ scSubscriptionState )
2022-04-22 22:53:12 +03:00
( ES . _ssStreamQueryOptions $ scSubscriptionState )
2021-03-09 21:45:43 +03:00
consoleAssetsDir
scExperimentalFeatures
2022-09-27 09:48:21 +03:00
scEnabledAPIs
2022-11-21 08:03:37 +03:00
scDefaultNamingConvention
2022-12-15 10:48:18 +03:00
return ( emptyHttpLogGraphQLInfo , JSONResp $ HttpResponse ( encJFromJValue res ) [] )
2021-09-24 01:56:37 +03:00
2019-09-09 23:26:04 +03:00
data HasuraApp = HasuraApp
2021-03-31 13:39:01 +03:00
{ _hapApplication :: ! Wai . Application ,
_hapSchemaRef :: ! SchemaCacheRef ,
2022-03-21 13:39:49 +03:00
_hapAsyncActionSubscriptionState :: ! ES . AsyncActionSubscriptionState ,
2021-03-31 13:39:01 +03:00
_hapShutdownWsServer :: ! ( IO () )
2019-09-09 23:26:04 +03:00
}
2020-07-14 22:00:58 +03:00
-- TODO: Put Env into ServerCtx?
2018-07-20 10:22:46 +03:00
mkWaiApp ::
2019-12-11 04:04:49 +03:00
forall m .
2021-10-13 19:38:56 +03:00
( MonadIO m ,
Rewrite OpenAPI
### Description
This PR rewrites OpenAPI to be more idiomatic. Some noteworthy changes:
- we accumulate all required information during the Analyze phase, to avoid having to do a single lookup in the schema cache during the OpenAPI generation phase (we now only need the schema cache as input to run the analysis)
- we no longer build intermediary endpoint information and aggregate it, we directly build the the `PathItem` for each endpoint; additionally, that means we no longer have to assume that different methods have the same metadata
- we no longer have to first declare types, then craft references: we do everything in one step
- we now properly deal with nullability by treating "typeName" and "typeName!" as different
- we add a bunch of additional fields in the generated "schema", such as title
- we do now support enum values in both input and output positions
- checking whether the request body is required is now performed on the fly rather than by introspecting the generated schema
- the methods in the file are sorted by topic
### Controversial point
However, this PR creates some additional complexity, that we might not want to keep. The main complexity is _knot-tying_: to avoid lookups when generating the OpenAPI, it builds an actual graph of input types, which means that we need something similar to (but simpler than) `MonadSchema`, to avoid infinite recursions when analyzing the input types of a query. To do this, this PR introduces `CircularT`, a lesser `SchemaT` that aims at avoiding ever having to reinvent this particular wheel ever again.
### Remaining work
- [x] fix existing tests (they are all failing due to some of the schema changes)
- [ ] add tests to cover the new features:
- [x] tests for `CircularT`
- [ ] tests for enums in output schemas
- [x] extract / document `CircularT` if we wish to keep it
- [x] add more comments to `OpenAPI`
- [x] have a second look at `buildVariableSchema`
- [x] fix all missing diagnostics in `Analyze`
- [x] add a Changelog entry?
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4654
Co-authored-by: David Overton <7734777+dmoverton@users.noreply.github.com>
GitOrigin-RevId: f4a9191f22dfcc1dccefd6a52f5c586b6ad17172
2022-06-30 15:55:56 +03:00
MonadFix m ,
2019-11-26 15:14:21 +03:00
MonadStateless IO m ,
2020-06-16 18:23:06 +03:00
LA . Forall ( LA . Pure m ) ,
2019-11-26 15:14:21 +03:00
ConsoleRenderer m ,
2022-12-07 14:28:58 +03:00
MonadVersionAPIWithExtraData m ,
2019-11-26 15:14:21 +03:00
HttpLog m ,
2020-07-15 13:40:48 +03:00
UserAuthentication ( Tracing . TraceT m ) ,
2021-01-07 12:04:22 +03:00
MonadMetadataApiAuthorization m ,
2020-06-16 18:23:06 +03:00
E . MonadGQLExecutionCheck m ,
MonadConfigApiHandler m ,
2020-07-14 22:00:58 +03:00
MonadQueryLog m ,
2020-06-19 09:42:32 +03:00
WS . MonadWSLog m ,
2020-07-15 13:40:48 +03:00
Tracing . HasReporter m ,
GH . MonadExecuteQuery m ,
2020-12-03 07:06:22 +03:00
HasResourceLimits m ,
2020-12-14 07:30:19 +03:00
MonadMetadataStorage ( MetadataStorageT m ) ,
2020-12-28 15:56:00 +03:00
MonadResolveSource m ,
2022-09-09 11:26:44 +03:00
EB . MonadQueryTags m ,
MonadEventLogCleanup m
2021-02-13 03:05:23 +03:00
) =>
( ServerCtx -> Spock . SpockT m () ) ->
2020-07-14 22:00:58 +03:00
-- | Set of environment variables for reference in UIs
2019-11-28 12:03:14 +03:00
Env . Environment ->
2019-05-16 09:13:25 +03:00
CorsConfig ->
2020-07-14 22:00:58 +03:00
-- | is console enabled - TODO: better type
2019-05-16 10:45:29 +03:00
Bool ->
2020-07-14 22:00:58 +03:00
-- | filepath to the console static assets directory - TODO: better type
2019-05-16 09:13:25 +03:00
Maybe Text ->
2022-09-27 17:50:32 +03:00
-- | DSN for console sentry integration
Maybe Text ->
2020-07-14 22:00:58 +03:00
-- | is telemetry enabled
2019-05-16 09:13:25 +03:00
Bool ->
2021-02-11 20:54:25 +03:00
SchemaCacheRef ->
2020-10-12 12:14:23 +03:00
WS . ConnectionOptions ->
2020-11-12 12:25:48 +03:00
KeepAliveDelay ->
2021-07-13 15:23:30 +03:00
S . HashSet ( L . EngineLogType L . Hasura ) ->
2023-01-06 12:33:13 +03:00
ServerCtx ->
2021-08-24 19:25:12 +03:00
WSConnectionInitTimeout ->
2023-01-06 12:33:13 +03:00
EKG . Store EKG . EmptyMetrics ->
2019-11-26 15:14:21 +03:00
m HasuraApp
2021-02-13 03:05:23 +03:00
mkWaiApp
setupHook
env
corsCfg
enableConsole
consoleAssetsDir
2022-09-27 17:50:32 +03:00
consoleSentryDsn
server: remove remnants of query plan caching (fix #1795)
Query plan caching was introduced by - I believe - hasura/graphql-engine#1934 in order to reduce the query response latency. During the development of PDV in hasura/graphql-engine#4111, it was found out that the new architecture (for which query plan caching wasn't implemented) performed comparably to the pre-PDV architecture with caching. Hence, it was decided to leave query plan caching until some day in the future when it was deemed necessary.
Well, we're in the future now, and there still isn't a convincing argument for query plan caching. So the time has come to remove some references to query plan caching from the codebase. For the most part, any code being removed would probably not be very well suited to the post-PDV architecture of query execution, so arguably not much is lost.
Apart from simplifying the code, this PR will contribute towards making the GraphQL schema generation more modular, testable, and easier to profile. I'd like to eventually work towards a situation in which it's easy to generate a GraphQL schema parser *in isolation*, without being connected to a database, and then parse a GraphQL query *in isolation*, without even listening any HTTP port. It is important that both of these operations can be examined in detail, and in isolation, since they are two major performance bottlenecks, as well as phases where many important upcoming features hook into.
Implementation
The following have been removed:
- The entirety of `server/src-lib/Hasura/GraphQL/Execute/Plan.hs`
- The core phases of query parsing and execution no longer have any references to query plan caching. Note that this is not to be confused with query *response* caching, which is not affected by this PR. This includes removal of the types:
- - `Opaque`, which is replaced by a tuple. Note that the old implementation was broken and did not adequately hide the constructors.
- - `QueryReusability` (and the `markNotReusable` method). Notably, the implementation of the `ParseT` monad now consists of two, rather than three, monad transformers.
- Cache-related tests (in `server/src-test/Hasura/CacheBoundedSpec.hs`) have been removed .
- References to query plan caching in the documentation.
- The `planCacheOptions` in the `TenantConfig` type class was removed. However, during parsing, unrecognized fields in the YAML config get ignored, so this does not cause a breaking change. (Confirmed manually, as well as in consultation with @sordina.)
- The metrics no longer send cache hit/miss messages.
There are a few places in which one can still find references to query plan caching:
- We still accept the `--query-plan-cache-size` command-line option for backwards compatibility. The `HASURA_QUERY_PLAN_CACHE_SIZE` environment variable is not read.
https://github.com/hasura/graphql-engine-mono/pull/1815
GitOrigin-RevId: 17d92b254ec093c62a7dfeec478658ede0813eb7
2021-07-27 14:51:52 +03:00
enableTelemetry
2021-04-29 13:31:09 +03:00
schemaCacheRef
2021-08-24 19:25:12 +03:00
connectionOptions
keepAliveDelay
enabledLogTypes
2023-01-06 12:33:13 +03:00
serverCtx @ ServerCtx { .. }
2022-04-11 20:49:25 +03:00
wsConnInitTimeout
2023-01-06 12:33:13 +03:00
ekgStore = do
2022-03-09 01:59:28 +03:00
let getSchemaCache' = first lastBuiltSchemaCache <$> readSchemaCacheRef schemaCacheRef
2018-07-20 10:22:46 +03:00
2019-04-17 12:48:41 +03:00
let corsPolicy = mkDefaultCorsPolicy corsCfg
2022-04-22 22:53:12 +03:00
2021-08-24 19:25:12 +03:00
wsServerEnv <-
WS . createWSServerEnv
2023-01-06 12:33:13 +03:00
( _lsLogger scLoggers )
scSubscriptionState
2022-03-09 01:59:28 +03:00
getSchemaCache'
2023-01-06 12:33:13 +03:00
scManager
2021-08-24 19:25:12 +03:00
corsPolicy
2023-01-06 12:33:13 +03:00
scSQLGenCtx
scEnableReadOnlyMode
scEnableAllowlist
2021-08-24 19:25:12 +03:00
keepAliveDelay
2023-01-06 12:33:13 +03:00
scServerMetrics
scPrometheusMetrics
scTraceSamplingPolicy
2019-07-10 15:01:52 +03:00
2019-11-26 15:14:21 +03:00
spockApp <- liftWithStateless $ \ lowerIO ->
2020-06-16 18:23:06 +03:00
Spock . spockAsApp $
Spock . spockT lowerIO $
2023-01-06 12:33:13 +03:00
httpApp setupHook corsCfg serverCtx enableConsole consoleAssetsDir consoleSentryDsn enableTelemetry ekgStore
2019-03-04 10:46:53 +03:00
2023-01-06 12:33:13 +03:00
let wsServerApp = WS . createWSServerApp env enabledLogTypes scAuthMode wsServerEnv wsConnInitTimeout -- TODO: Lyndon: Can we pass environment through wsServerEnv?
2019-09-09 23:26:04 +03:00
stopWSServer = WS . stopWSServerApp wsServerEnv
2019-12-11 04:04:49 +03:00
waiApp <- liftWithStateless $ \ lowerIO ->
2020-10-12 12:14:23 +03:00
pure $ WSC . websocketsOr connectionOptions ( \ ip conn -> lowerIO $ wsServerApp ip conn ) spockApp
2019-12-11 04:04:49 +03:00
2023-01-06 12:33:13 +03:00
return $ HasuraApp waiApp schemaCacheRef ( ES . _ssAsyncActions scSubscriptionState ) stopWSServer
2021-02-11 20:54:25 +03:00
2019-11-26 15:14:21 +03:00
httpApp ::
2021-07-05 12:45:31 +03:00
forall m .
2021-10-13 19:38:56 +03:00
( MonadIO m ,
Rewrite OpenAPI
### Description
This PR rewrites OpenAPI to be more idiomatic. Some noteworthy changes:
- we accumulate all required information during the Analyze phase, to avoid having to do a single lookup in the schema cache during the OpenAPI generation phase (we now only need the schema cache as input to run the analysis)
- we no longer build intermediary endpoint information and aggregate it, we directly build the the `PathItem` for each endpoint; additionally, that means we no longer have to assume that different methods have the same metadata
- we no longer have to first declare types, then craft references: we do everything in one step
- we now properly deal with nullability by treating "typeName" and "typeName!" as different
- we add a bunch of additional fields in the generated "schema", such as title
- we do now support enum values in both input and output positions
- checking whether the request body is required is now performed on the fly rather than by introspecting the generated schema
- the methods in the file are sorted by topic
### Controversial point
However, this PR creates some additional complexity, that we might not want to keep. The main complexity is _knot-tying_: to avoid lookups when generating the OpenAPI, it builds an actual graph of input types, which means that we need something similar to (but simpler than) `MonadSchema`, to avoid infinite recursions when analyzing the input types of a query. To do this, this PR introduces `CircularT`, a lesser `SchemaT` that aims at avoiding ever having to reinvent this particular wheel ever again.
### Remaining work
- [x] fix existing tests (they are all failing due to some of the schema changes)
- [ ] add tests to cover the new features:
- [x] tests for `CircularT`
- [ ] tests for enums in output schemas
- [x] extract / document `CircularT` if we wish to keep it
- [x] add more comments to `OpenAPI`
- [x] have a second look at `buildVariableSchema`
- [x] fix all missing diagnostics in `Analyze`
- [x] add a Changelog entry?
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4654
Co-authored-by: David Overton <7734777+dmoverton@users.noreply.github.com>
GitOrigin-RevId: f4a9191f22dfcc1dccefd6a52f5c586b6ad17172
2022-06-30 15:55:56 +03:00
MonadFix m ,
2020-06-16 18:23:06 +03:00
MonadBaseControl IO m ,
ConsoleRenderer m ,
2022-12-07 14:28:58 +03:00
MonadVersionAPIWithExtraData m ,
2020-06-16 18:23:06 +03:00
HttpLog m ,
2020-07-15 13:40:48 +03:00
UserAuthentication ( Tracing . TraceT m ) ,
2021-01-07 12:04:22 +03:00
MonadMetadataApiAuthorization m ,
2020-06-16 18:23:06 +03:00
E . MonadGQLExecutionCheck m ,
MonadConfigApiHandler m ,
2020-06-19 09:42:32 +03:00
MonadQueryLog m ,
2020-07-15 13:40:48 +03:00
Tracing . HasReporter m ,
GH . MonadExecuteQuery m ,
2020-12-14 07:30:19 +03:00
MonadMetadataStorage ( MetadataStorageT m ) ,
2020-12-03 07:06:22 +03:00
HasResourceLimits m ,
2020-12-28 15:56:00 +03:00
MonadResolveSource m ,
2022-09-09 11:26:44 +03:00
EB . MonadQueryTags m ,
MonadEventLogCleanup m
2020-06-16 18:23:06 +03:00
) =>
2021-02-13 03:05:23 +03:00
( ServerCtx -> Spock . SpockT m () ) ->
CorsConfig ->
2019-11-26 15:14:21 +03:00
ServerCtx ->
Bool ->
Maybe Text ->
2022-09-27 17:50:32 +03:00
Maybe Text ->
2019-11-26 15:14:21 +03:00
Bool ->
2023-01-06 12:33:13 +03:00
EKG . Store EKG . EmptyMetrics ->
2019-11-26 15:14:21 +03:00
Spock . SpockT m ()
2023-01-06 12:33:13 +03:00
httpApp setupHook corsCfg serverCtx enableConsole consoleAssetsDir consoleSentryDsn enableTelemetry ekgStore = do
2021-02-13 03:05:23 +03:00
-- Additional spock action to run
setupHook serverCtx
2021-09-24 01:56:37 +03:00
2018-06-27 16:11:32 +03:00
-- cors middleware
2019-02-14 08:58:38 +03:00
unless ( isCorsDisabled corsCfg ) $
2019-11-26 15:14:21 +03:00
Spock . middleware $
corsMiddleware ( mkDefaultCorsPolicy corsCfg )
2021-09-24 01:56:37 +03:00
2018-06-29 14:05:09 +03:00
-- API Console and Root Dir
2019-07-08 08:51:41 +03:00
when ( enableConsole && enableMetadata ) serveApiConsole
2021-09-24 01:56:37 +03:00
2023-01-19 23:07:09 +03:00
-- Local console assets for server and CLI consoles
serveApiConsoleAssets
2021-06-08 17:28:00 +03:00
-- Health check endpoint with logs
let healthzAction = do
2021-11-17 20:58:34 +03:00
let errorMsg = " ERROR "
runMetadataStorageT checkMetadataStorageHealth >>= \ case
Left err -> do
-- error running the health check
logError err
Spock . setStatus HTTP . status500 >> Spock . text errorMsg
2022-06-07 14:23:16 +03:00
Right _ -> do
2021-11-17 20:58:34 +03:00
-- healthy
2022-03-09 01:59:28 +03:00
sc <- liftIO $ getSchemaCache $ scCacheRef serverCtx
2021-11-17 20:58:34 +03:00
let responseText =
if null ( scInconsistentObjs sc )
then " OK "
else " WARN: inconsistent objects in schema "
2021-06-08 17:28:00 +03:00
logSuccess responseText
Spock . setStatus HTTP . status200 >> Spock . text ( LT . toStrict responseText )
2021-09-24 01:56:37 +03:00
2021-06-08 17:28:00 +03:00
Spock . get " healthz " healthzAction
2021-09-24 01:56:37 +03:00
2021-07-05 12:45:31 +03:00
-- This is an alternative to `healthz` (See issue #6958)
2021-06-08 17:28:00 +03:00
Spock . get " hasura/healthz " healthzAction
2021-09-24 01:56:37 +03:00
2019-11-26 15:14:21 +03:00
Spock . get " v1/version " $ do
2021-06-08 17:28:00 +03:00
logSuccess $ " version: " <> convertText currentVersion
2022-12-07 14:28:58 +03:00
extraData <- lift $ getExtraDataForVersionAPI
2020-03-20 09:46:45 +03:00
setHeader jsonHeader
2022-12-07 14:28:58 +03:00
Spock . lazyBytes $ encode $ object $ [ " version " .= currentVersion ] <> extraData
2021-09-24 01:56:37 +03:00
2021-01-29 04:02:34 +03:00
let customEndpointHandler ::
2021-07-05 12:45:31 +03:00
forall n .
2021-10-13 19:38:56 +03:00
( MonadIO n ,
2021-07-05 12:45:31 +03:00
MonadBaseControl IO n ,
E . MonadGQLExecutionCheck n ,
MonadQueryLog n ,
GH . MonadExecuteQuery n ,
MonadMetadataStorage ( MetadataStorageT n ) ,
2021-09-29 19:20:06 +03:00
EB . MonadQueryTags n ,
HasResourceLimits n
2021-01-29 04:02:34 +03:00
) =>
RestRequest Spock . SpockMethod ->
2022-12-15 10:48:18 +03:00
Handler ( Tracing . TraceT n ) ( HttpLogGraphQLInfo , APIResp )
2021-01-29 04:02:34 +03:00
customEndpointHandler restReq = do
scRef <- asks ( scCacheRef . hcServerCtx )
2022-03-09 01:59:28 +03:00
endpoints <- liftIO $ scEndpoints <$> getSchemaCache scRef
2021-01-29 04:02:34 +03:00
execCtx <- mkExecutionContext
env <- asks ( scEnvironment . hcServerCtx )
requestId <- asks hcRequestId
userInfo <- asks hcUser
reqHeaders <- asks hcReqHeaders
ipAddress <- asks hcSourceIpAddress
req <-
restReq & traverse \ case
2022-03-13 10:40:06 +03:00
Spock . MethodStandard ( Spock . HttpMethod m ) -> case m of
Spock . GET -> pure EP . GET
Spock . POST -> pure EP . POST
Spock . PUT -> pure EP . PUT
Spock . DELETE -> pure EP . DELETE
Spock . PATCH -> pure EP . PATCH
other -> throw400 BadRequest $ " Method " <> tshow other <> " not supported. "
2021-01-29 04:02:34 +03:00
_ -> throw400 BadRequest $ " Nonstandard method not allowed for REST endpoints "
2021-04-06 20:52:55 +03:00
fmap JSONResp <$> runCustomEndpoint env execCtx requestId userInfo reqHeaders ipAddress req endpoints
2021-01-29 04:02:34 +03:00
-- See Issue #291 for discussion around restified feature
Spock . hookRouteAll ( " api " <//> " rest " <//> Spock . wildcard ) $ \ wildcard -> do
queryParams <- Spock . params
body <- Spock . body
method <- Spock . reqMethod
-- This is where we decode the json encoded body args. They
-- are treated as if they came from query arguments, but allow
-- us to pass non-scalar values.
let bodyParams = case J . decodeStrict body of
2022-06-08 18:31:28 +03:00
Just ( J . Object o ) -> map ( first K . toText ) $ KM . toList o
2021-01-29 04:02:34 +03:00
_ -> []
allParams = fmap Left <$> queryParams <|> fmap Right <$> bodyParams
2021-04-06 20:52:55 +03:00
spockAction encodeQErr id $ do
2021-01-29 04:02:34 +03:00
-- TODO: Are we actually able to use mkGetHandler in this situation? POST handler seems to do some work that we might want to avoid.
mkGetHandler $ customEndpointHandler ( RestRequest wildcard method allParams )
2021-04-06 20:52:55 +03:00
2021-01-29 04:02:34 +03:00
when enableMetadata $ do
Spock . post " v1/graphql/explain " gqlExplainAction
2019-02-28 16:53:03 +03:00
Spock . post " v1alpha1/graphql/explain " gqlExplainAction
2018-06-27 16:11:32 +03:00
2021-04-06 20:52:55 +03:00
Spock . post " v1/query " $
spockAction encodeQErr id $ do
2022-12-15 10:48:18 +03:00
mkPostHandler $ fmap ( emptyHttpLogGraphQLInfo , ) <$> mkAPIRespHandler v1QueryHandler
2018-06-27 16:11:32 +03:00
2021-01-07 12:04:22 +03:00
Spock . post " v1/metadata " $
spockAction encodeQErr id $
2021-07-05 12:45:31 +03:00
mkPostHandler $
2022-12-15 10:48:18 +03:00
fmap ( emptyHttpLogGraphQLInfo , ) <$> mkAPIRespHandler v1MetadataHandler
2021-01-07 12:04:22 +03:00
Spock . post " v2/query " $
spockAction encodeQErr id $
2021-07-05 12:45:31 +03:00
mkPostHandler $
2022-12-15 10:48:18 +03:00
fmap ( emptyHttpLogGraphQLInfo , ) <$> mkAPIRespHandler v2QueryHandler
2021-01-07 12:04:22 +03:00
2019-04-30 11:34:08 +03:00
when enablePGDump $
2019-11-26 15:14:21 +03:00
Spock . post " v1alpha1/pg_dump " $
spockAction encodeQErr id $
2021-07-05 12:45:31 +03:00
mkPostHandler $
2022-12-15 10:48:18 +03:00
fmap ( emptyHttpLogGraphQLInfo , ) <$> v1Alpha1PGDumpHandler
2019-04-30 11:34:08 +03:00
2020-06-16 18:23:06 +03:00
when enableConfig $ runConfigApiHandler serverCtx consoleAssetsDir
2019-06-11 16:29:03 +03:00
2019-02-28 16:53:03 +03:00
when enableGraphQL $ do
2019-11-26 15:14:21 +03:00
Spock . post " v1alpha1/graphql " $
spockAction GH . encodeGQErr id $
2021-04-06 20:52:55 +03:00
mkGQLRequestHandler $
mkGQLAPIRespHandler $
v1Alpha1GQHandler E . QueryHasura
2018-06-27 16:11:32 +03:00
2019-11-26 15:14:21 +03:00
Spock . post " v1/graphql " $
spockAction GH . encodeGQErr allMod200 $
2021-04-06 20:52:55 +03:00
mkGQLRequestHandler $
2023-01-06 12:33:13 +03:00
mkGQLAPIRespHandler $
v1GQHandler
2019-05-10 09:05:11 +03:00
2020-07-03 09:30:35 +03:00
Spock . post " v1beta1/relay " $
spockAction GH . encodeGQErr allMod200 $
2021-04-06 20:52:55 +03:00
mkGQLRequestHandler $
mkGQLAPIRespHandler $
v1GQRelayHandler
2020-06-08 15:13:01 +03:00
2021-04-13 20:32:29 +03:00
-- This exposes some simple RTS stats when we run with `+RTS -T`. We want
-- this to be available even when developer APIs are not compiled in, to
-- support benchmarking.
-- See: https://hackage.haskell.org/package/base/docs/GHC-Stats.html
exposeRtsStats <- liftIO RTS . getRTSStatsEnabled
when exposeRtsStats $ do
2021-04-14 23:31:04 +03:00
Spock . get " dev/rts_stats " $ do
2021-08-06 02:37:02 +03:00
-- This ensures the live_bytes and other counters from GCDetails are fresh:
liftIO performMajorGC
2021-04-14 23:31:04 +03:00
stats <- liftIO RTS . getRTSStats
Spock . json stats
2021-09-24 01:56:37 +03:00
2019-04-30 08:15:23 +03:00
when ( isDeveloperAPIEnabled serverCtx ) $ do
2019-11-26 15:14:21 +03:00
Spock . get " dev/ekg " $
spockAction encodeQErr id $
2019-07-11 08:37:06 +03:00
mkGetHandler $ do
2019-07-10 15:01:52 +03:00
onlyAdmin
2023-01-06 12:33:13 +03:00
respJ <- liftIO $ EKG . sampleAll ekgStore
2022-12-15 10:48:18 +03:00
return ( emptyHttpLogGraphQLInfo , JSONResp $ HttpResponse ( encJFromJValue $ EKG . sampleToJson respJ ) [] )
server: remove remnants of query plan caching (fix #1795)
Query plan caching was introduced by - I believe - hasura/graphql-engine#1934 in order to reduce the query response latency. During the development of PDV in hasura/graphql-engine#4111, it was found out that the new architecture (for which query plan caching wasn't implemented) performed comparably to the pre-PDV architecture with caching. Hence, it was decided to leave query plan caching until some day in the future when it was deemed necessary.
Well, we're in the future now, and there still isn't a convincing argument for query plan caching. So the time has come to remove some references to query plan caching from the codebase. For the most part, any code being removed would probably not be very well suited to the post-PDV architecture of query execution, so arguably not much is lost.
Apart from simplifying the code, this PR will contribute towards making the GraphQL schema generation more modular, testable, and easier to profile. I'd like to eventually work towards a situation in which it's easy to generate a GraphQL schema parser *in isolation*, without being connected to a database, and then parse a GraphQL query *in isolation*, without even listening any HTTP port. It is important that both of these operations can be examined in detail, and in isolation, since they are two major performance bottlenecks, as well as phases where many important upcoming features hook into.
Implementation
The following have been removed:
- The entirety of `server/src-lib/Hasura/GraphQL/Execute/Plan.hs`
- The core phases of query parsing and execution no longer have any references to query plan caching. Note that this is not to be confused with query *response* caching, which is not affected by this PR. This includes removal of the types:
- - `Opaque`, which is replaced by a tuple. Note that the old implementation was broken and did not adequately hide the constructors.
- - `QueryReusability` (and the `markNotReusable` method). Notably, the implementation of the `ParseT` monad now consists of two, rather than three, monad transformers.
- Cache-related tests (in `server/src-test/Hasura/CacheBoundedSpec.hs`) have been removed .
- References to query plan caching in the documentation.
- The `planCacheOptions` in the `TenantConfig` type class was removed. However, during parsing, unrecognized fields in the YAML config get ignored, so this does not cause a breaking change. (Confirmed manually, as well as in consultation with @sordina.)
- The metrics no longer send cache hit/miss messages.
There are a few places in which one can still find references to query plan caching:
- We still accept the `--query-plan-cache-size` command-line option for backwards compatibility. The `HASURA_QUERY_PLAN_CACHE_SIZE` environment variable is not read.
https://github.com/hasura/graphql-engine-mono/pull/1815
GitOrigin-RevId: 17d92b254ec093c62a7dfeec478658ede0813eb7
2021-07-27 14:51:52 +03:00
-- This deprecated endpoint used to show the query plan cache pre-PDV.
-- Eventually this endpoint can be removed.
2019-11-26 15:14:21 +03:00
Spock . get " dev/plan_cache " $
spockAction encodeQErr id $
2019-07-11 08:37:06 +03:00
mkGetHandler $ do
2019-04-30 11:34:08 +03:00
onlyAdmin
2022-12-15 10:48:18 +03:00
return ( emptyHttpLogGraphQLInfo , JSONResp $ HttpResponse ( encJFromJValue J . Null ) [] )
2019-11-26 15:14:21 +03:00
Spock . get " dev/subscriptions " $
spockAction encodeQErr id $
2019-07-11 08:37:06 +03:00
mkGetHandler $ do
2019-04-30 11:34:08 +03:00
onlyAdmin
2022-03-21 13:39:49 +03:00
respJ <- liftIO $ ES . dumpSubscriptionsState False $ scSubscriptionState serverCtx
2022-12-15 10:48:18 +03:00
return ( emptyHttpLogGraphQLInfo , JSONResp $ HttpResponse ( encJFromJValue respJ ) [] )
2019-11-26 15:14:21 +03:00
Spock . get " dev/subscriptions/extended " $
spockAction encodeQErr id $
2019-07-11 08:37:06 +03:00
mkGetHandler $ do
2019-04-30 11:34:08 +03:00
onlyAdmin
2022-03-21 13:39:49 +03:00
respJ <- liftIO $ ES . dumpSubscriptionsState True $ scSubscriptionState serverCtx
2022-12-15 10:48:18 +03:00
return ( emptyHttpLogGraphQLInfo , JSONResp $ HttpResponse ( encJFromJValue respJ ) [] )
2022-05-02 08:03:12 +03:00
Spock . get " dev/dataconnector/schema " $
2022-03-31 07:45:03 +03:00
spockAction encodeQErr id $
mkGetHandler $ do
onlyAdmin
2022-12-15 10:48:18 +03:00
return ( emptyHttpLogGraphQLInfo , JSONResp $ HttpResponse ( encJFromJValue openApiSchema ) [] )
2021-10-06 10:15:14 +03:00
Spock . get " api/swagger/json " $
spockAction encodeQErr id $
mkGetHandler $ do
onlyAdmin
2022-03-09 01:59:28 +03:00
sc <- liftIO $ getSchemaCache $ scCacheRef serverCtx
Rewrite OpenAPI
### Description
This PR rewrites OpenAPI to be more idiomatic. Some noteworthy changes:
- we accumulate all required information during the Analyze phase, to avoid having to do a single lookup in the schema cache during the OpenAPI generation phase (we now only need the schema cache as input to run the analysis)
- we no longer build intermediary endpoint information and aggregate it, we directly build the the `PathItem` for each endpoint; additionally, that means we no longer have to assume that different methods have the same metadata
- we no longer have to first declare types, then craft references: we do everything in one step
- we now properly deal with nullability by treating "typeName" and "typeName!" as different
- we add a bunch of additional fields in the generated "schema", such as title
- we do now support enum values in both input and output positions
- checking whether the request body is required is now performed on the fly rather than by introspecting the generated schema
- the methods in the file are sorted by topic
### Controversial point
However, this PR creates some additional complexity, that we might not want to keep. The main complexity is _knot-tying_: to avoid lookups when generating the OpenAPI, it builds an actual graph of input types, which means that we need something similar to (but simpler than) `MonadSchema`, to avoid infinite recursions when analyzing the input types of a query. To do this, this PR introduces `CircularT`, a lesser `SchemaT` that aims at avoiding ever having to reinvent this particular wheel ever again.
### Remaining work
- [x] fix existing tests (they are all failing due to some of the schema changes)
- [ ] add tests to cover the new features:
- [x] tests for `CircularT`
- [ ] tests for enums in output schemas
- [x] extract / document `CircularT` if we wish to keep it
- [x] add more comments to `OpenAPI`
- [x] have a second look at `buildVariableSchema`
- [x] fix all missing diagnostics in `Analyze`
- [x] add a Changelog entry?
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4654
Co-authored-by: David Overton <7734777+dmoverton@users.noreply.github.com>
GitOrigin-RevId: f4a9191f22dfcc1dccefd6a52f5c586b6ad17172
2022-06-30 15:55:56 +03:00
json <- buildOpenAPI sc
2022-12-15 10:48:18 +03:00
return ( emptyHttpLogGraphQLInfo , JSONResp $ HttpResponse ( encJFromJValue json ) [] )
2018-06-27 16:11:32 +03:00
2019-11-26 15:14:21 +03:00
forM_ [ Spock . GET , Spock . POST ] $ \ m -> Spock . hookAny m $ \ _ -> do
req <- Spock . request
let headers = Wai . requestHeaders req
2021-02-11 20:54:25 +03:00
qErr = err404 NotFound " resource does not exist "
2022-04-11 20:49:25 +03:00
raiseGenericApiError logger ( scLoggingSettings serverCtx ) headers qErr
2018-06-27 16:11:32 +03:00
where
2023-01-06 12:33:13 +03:00
logger = ( _lsLogger . scLoggers ) serverCtx
2019-07-11 08:37:06 +03:00
2021-06-08 17:28:00 +03:00
logSuccess msg = do
req <- Spock . request
reqBody <- liftIO $ Wai . strictRequestBody req
let headers = Wai . requestHeaders req
blMsg = TL . encodeUtf8 msg
( reqId , _newHeaders ) <- getRequestId headers
lift $
2022-04-11 20:49:25 +03:00
logHttpSuccess logger ( scLoggingSettings serverCtx ) Nothing reqId req ( reqBody , Nothing ) blMsg blMsg Nothing Nothing headers ( emptyHttpLogMetadata @ m )
2021-06-08 17:28:00 +03:00
2021-11-17 20:58:34 +03:00
logError err = do
2021-06-08 17:28:00 +03:00
req <- Spock . request
reqBody <- liftIO $ Wai . strictRequestBody req
let headers = Wai . requestHeaders req
( reqId , _newHeaders ) <- getRequestId headers
lift $
2022-12-15 10:48:18 +03:00
logHttpError logger ( scLoggingSettings serverCtx ) Nothing reqId req ( reqBody , Nothing ) err headers ( emptyHttpLogMetadata @ m )
2021-06-08 17:28:00 +03:00
2019-11-26 15:14:21 +03:00
spockAction ::
2021-07-05 12:45:31 +03:00
forall a n .
2021-07-05 21:47:09 +03:00
( FromJSON a , MonadIO n , MonadBaseControl IO n , UserAuthentication ( Tracing . TraceT n ) , HttpLog n , Tracing . HasReporter n , HasResourceLimits n ) =>
2020-04-24 10:55:51 +03:00
( Bool -> QErr -> Value ) ->
2021-07-05 12:45:31 +03:00
( QErr -> QErr ) ->
APIHandler ( Tracing . TraceT n ) a ->
Spock . ActionT n ()
2022-04-11 20:49:25 +03:00
spockAction qErrEncoder qErrModifier apiHandler = mkSpockAction serverCtx qErrEncoder qErrModifier apiHandler
2019-11-26 15:14:21 +03:00
2019-05-10 09:05:11 +03:00
-- all graphql errors should be of type 200
2020-07-14 22:00:58 +03:00
allMod200 qe = qe { qeStatus = HTTP . status200 }
2021-04-06 20:52:55 +03:00
gqlExplainAction = do
spockAction encodeQErr id $
mkPostHandler $
2022-12-15 10:48:18 +03:00
fmap ( emptyHttpLogGraphQLInfo , ) <$> mkAPIRespHandler gqlExplainHandler
2020-07-14 22:00:58 +03:00
enableGraphQL = isGraphQLEnabled serverCtx
enableMetadata = isMetadataEnabled serverCtx
enablePGDump = isPGDumpEnabled serverCtx
enableConfig = isConfigEnabled serverCtx
2019-07-11 08:37:06 +03:00
2018-12-18 12:39:01 +03:00
serveApiConsole = do
2019-05-16 10:45:29 +03:00
-- redirect / to /console
2019-11-26 15:14:21 +03:00
Spock . get Spock . root $ Spock . redirect " console "
2019-05-16 10:45:29 +03:00
-- serve console html
2019-11-26 15:14:21 +03:00
Spock . get ( " console " <//> Spock . wildcard ) $ \ path -> do
req <- Spock . request
let headers = Wai . requestHeaders req
2021-02-11 20:54:25 +03:00
authMode = scAuthMode serverCtx
2022-09-27 17:50:32 +03:00
consoleHtml <- lift $ renderConsole path authMode enableTelemetry consoleAssetsDir consoleSentryDsn
2022-04-11 20:49:25 +03:00
either ( raiseGenericApiError logger ( scLoggingSettings serverCtx ) headers . internalError . T . pack ) Spock . html consoleHtml
2019-11-26 15:14:21 +03:00
2023-01-19 23:07:09 +03:00
serveApiConsoleAssets = do
-- serve static files if consoleAssetsDir is set
for_ consoleAssetsDir $ \ dir ->
Spock . get ( " console/assets " <//> Spock . wildcard ) $ \ path -> do
consoleAssetsHandler logger ( scLoggingSettings serverCtx ) dir ( T . unpack path )
2019-11-26 15:14:21 +03:00
raiseGenericApiError ::
2022-12-15 10:48:18 +03:00
forall m .
2019-11-26 15:14:21 +03:00
( MonadIO m , HttpLog m ) =>
L . Logger L . Hasura ->
2022-04-11 20:49:25 +03:00
LoggingSettings ->
2019-11-26 15:14:21 +03:00
[ HTTP . Header ] ->
QErr ->
Spock . ActionT m ()
2022-04-11 20:49:25 +03:00
raiseGenericApiError logger loggingSetting headers qErr = do
2019-11-26 15:14:21 +03:00
req <- Spock . request
reqBody <- liftIO $ Wai . strictRequestBody req
2021-05-19 17:07:04 +03:00
( reqId , _newHeaders ) <- getRequestId $ Wai . requestHeaders req
2022-12-15 10:48:18 +03:00
lift $ logHttpError logger loggingSetting Nothing reqId req ( reqBody , Nothing ) qErr headers ( emptyHttpLogMetadata @ m )
2020-03-20 09:46:45 +03:00
setHeader jsonHeader
2019-11-26 15:14:21 +03:00
Spock . setStatus $ qeStatus qErr
Spock . lazyBytes $ encode qErr