2021-08-05 00:23:33 +03:00
|
|
|
{-# LANGUAGE DeriveAnyClass #-}
|
|
|
|
|
2020-10-07 13:23:17 +03:00
|
|
|
module Hasura.GraphQL.Execute.Remote
|
|
|
|
( buildExecStepRemote,
|
Enable remote joins from remote schemas in the execution engine.
### Description
This PR adds the ability to perform remote joins from remote schemas in the engine. To do so, we alter the definition of an `ExecutionStep` targeting a remote schema: the `ExecStepRemote` constructor now expects a `Maybe RemoteJoins`. This new argument is used when processing the execution step, in the transport layer (either `Transport.HTTP` or `Transport.WebSocket`).
For this `Maybe RemoteJoins` to be extracted from a parsed query, this PR also extends the `Execute.RemoteJoin.Collect` module, to implement "collection" from a selection set. Not only do those new functions extract the remote joins, but they also apply all necessary transformations to the selection sets (such as inserting the necessary "phantom" fields used as join keys).
Finally in `Execute.RemoteJoin.Join`, we make two changes. First, we now always look for nested remote joins, regardless of whether the join we just performed went to a source or a remote schema; and second we adapt our join tree logic according to the special cases that were added to deal with remote server edge cases.
Additionally, this PR refactors / cleans / documents `Execute.RemoteJoin.RemoteServer`. This is not required as part of this change and could be moved to a separate PR if needed (a similar cleanup of `Join` is done independently in #3894). It also introduces a draft of a new documentation page for this project, that will be refined in the release PR that ships the feature (either #3069 or a copy of it).
While this PR extends the engine, it doesn't plug such relationships in the schema, meaning that, as of this PR, the new code paths in `Join` are technically unreachable. Adding the corresponding schema code and, ultimately, enabling the metadata API will be done in subsequent PRs.
### Keeping track of concrete type names
The main change this PR makes to the existing `Join` code is to handle a new reserved field we sometimes use when targeting remote servers: the `__hasura_internal_typename` field. In short, a GraphQL selection set can sometimes "branch" based on the concrete "runtime type" of the object on which the selection happens:
```graphql
query {
author(id: 53478) {
... on Writer {
name
articles {
title
}
}
... on Artist {
name
articles {
title
}
}
}
}
```
If both of those `articles` are remote joins, we need to be able, when we get the answer, to differentiate between the two different cases. We do this by asking for `__typename`, to be able to decide if we're in the `Writer` or the `Artist` branch of the query.
To avoid further processing / customization of results, we only insert this `__hasura_internal_typename: __typename` field in the query in the case of unions of interfaces AND if we have the guarantee that we will processing the request as part of the remote joins "folding": that is, if there's any remote join in this branch in the tree. Otherwise, we don't insert the field, and we leave that part of the response untouched.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3810
GitOrigin-RevId: 89aaf16274d68e26ad3730b80c2d2fdc2896b96c
2022-03-09 06:17:28 +03:00
|
|
|
getVariableDefinitionAndValue,
|
2020-12-21 12:11:37 +03:00
|
|
|
resolveRemoteVariable,
|
|
|
|
resolveRemoteField,
|
2021-05-24 23:12:53 +03:00
|
|
|
runVariableCache,
|
2020-10-07 13:23:17 +03:00
|
|
|
)
|
|
|
|
where
|
2021-09-24 01:56:37 +03:00
|
|
|
|
2020-12-21 12:11:37 +03:00
|
|
|
import Data.Aeson qualified as J
|
2023-04-26 18:42:13 +03:00
|
|
|
import Data.HashMap.Strict qualified as HashMap
|
2020-12-28 15:56:00 +03:00
|
|
|
import Data.HashSet qualified as Set
|
2020-12-21 12:11:37 +03:00
|
|
|
import Data.Text qualified as T
|
|
|
|
import Data.Text.Extended
|
2021-05-11 18:18:31 +03:00
|
|
|
import Hasura.Base.Error
|
2021-06-11 06:26:50 +03:00
|
|
|
import Hasura.GraphQL.Execute.Backend
|
Enable remote joins from remote schemas in the execution engine.
### Description
This PR adds the ability to perform remote joins from remote schemas in the engine. To do so, we alter the definition of an `ExecutionStep` targeting a remote schema: the `ExecStepRemote` constructor now expects a `Maybe RemoteJoins`. This new argument is used when processing the execution step, in the transport layer (either `Transport.HTTP` or `Transport.WebSocket`).
For this `Maybe RemoteJoins` to be extracted from a parsed query, this PR also extends the `Execute.RemoteJoin.Collect` module, to implement "collection" from a selection set. Not only do those new functions extract the remote joins, but they also apply all necessary transformations to the selection sets (such as inserting the necessary "phantom" fields used as join keys).
Finally in `Execute.RemoteJoin.Join`, we make two changes. First, we now always look for nested remote joins, regardless of whether the join we just performed went to a source or a remote schema; and second we adapt our join tree logic according to the special cases that were added to deal with remote server edge cases.
Additionally, this PR refactors / cleans / documents `Execute.RemoteJoin.RemoteServer`. This is not required as part of this change and could be moved to a separate PR if needed (a similar cleanup of `Join` is done independently in #3894). It also introduces a draft of a new documentation page for this project, that will be refined in the release PR that ships the feature (either #3069 or a copy of it).
While this PR extends the engine, it doesn't plug such relationships in the schema, meaning that, as of this PR, the new code paths in `Join` are technically unreachable. Adding the corresponding schema code and, ultimately, enabling the metadata API will be done in subsequent PRs.
### Keeping track of concrete type names
The main change this PR makes to the existing `Join` code is to handle a new reserved field we sometimes use when targeting remote servers: the `__hasura_internal_typename` field. In short, a GraphQL selection set can sometimes "branch" based on the concrete "runtime type" of the object on which the selection happens:
```graphql
query {
author(id: 53478) {
... on Writer {
name
articles {
title
}
}
... on Artist {
name
articles {
title
}
}
}
}
```
If both of those `articles` are remote joins, we need to be able, when we get the answer, to differentiate between the two different cases. We do this by asking for `__typename`, to be able to decide if we're in the `Writer` or the `Artist` branch of the query.
To avoid further processing / customization of results, we only insert this `__hasura_internal_typename: __typename` field in the query in the case of unions of interfaces AND if we have the guarantee that we will processing the request as part of the remote joins "folding": that is, if there's any remote join in this branch in the tree. Otherwise, we don't insert the field, and we leave that part of the response untouched.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3810
GitOrigin-RevId: 89aaf16274d68e26ad3730b80c2d2fdc2896b96c
2022-03-09 06:17:28 +03:00
|
|
|
import Hasura.GraphQL.Execute.RemoteJoin.Types (RemoteJoins)
|
2020-12-21 12:11:37 +03:00
|
|
|
import Hasura.GraphQL.Parser
|
2021-02-20 16:45:49 +03:00
|
|
|
import Hasura.GraphQL.Transport.HTTP.Protocol
|
2020-10-07 13:23:17 +03:00
|
|
|
import Hasura.GraphQL.Transport.HTTP.Protocol qualified as GH
|
2021-02-20 16:45:49 +03:00
|
|
|
import Hasura.Prelude
|
Enable remote joins from remote schemas in the execution engine.
### Description
This PR adds the ability to perform remote joins from remote schemas in the engine. To do so, we alter the definition of an `ExecutionStep` targeting a remote schema: the `ExecStepRemote` constructor now expects a `Maybe RemoteJoins`. This new argument is used when processing the execution step, in the transport layer (either `Transport.HTTP` or `Transport.WebSocket`).
For this `Maybe RemoteJoins` to be extracted from a parsed query, this PR also extends the `Execute.RemoteJoin.Collect` module, to implement "collection" from a selection set. Not only do those new functions extract the remote joins, but they also apply all necessary transformations to the selection sets (such as inserting the necessary "phantom" fields used as join keys).
Finally in `Execute.RemoteJoin.Join`, we make two changes. First, we now always look for nested remote joins, regardless of whether the join we just performed went to a source or a remote schema; and second we adapt our join tree logic according to the special cases that were added to deal with remote server edge cases.
Additionally, this PR refactors / cleans / documents `Execute.RemoteJoin.RemoteServer`. This is not required as part of this change and could be moved to a separate PR if needed (a similar cleanup of `Join` is done independently in #3894). It also introduces a draft of a new documentation page for this project, that will be refined in the release PR that ships the feature (either #3069 or a copy of it).
While this PR extends the engine, it doesn't plug such relationships in the schema, meaning that, as of this PR, the new code paths in `Join` are technically unreachable. Adding the corresponding schema code and, ultimately, enabling the metadata API will be done in subsequent PRs.
### Keeping track of concrete type names
The main change this PR makes to the existing `Join` code is to handle a new reserved field we sometimes use when targeting remote servers: the `__hasura_internal_typename` field. In short, a GraphQL selection set can sometimes "branch" based on the concrete "runtime type" of the object on which the selection happens:
```graphql
query {
author(id: 53478) {
... on Writer {
name
articles {
title
}
}
... on Artist {
name
articles {
title
}
}
}
}
```
If both of those `articles` are remote joins, we need to be able, when we get the answer, to differentiate between the two different cases. We do this by asking for `__typename`, to be able to decide if we're in the `Writer` or the `Artist` branch of the query.
To avoid further processing / customization of results, we only insert this `__hasura_internal_typename: __typename` field in the query in the case of unions of interfaces AND if we have the guarantee that we will processing the request as part of the remote joins "folding": that is, if there's any remote join in this branch in the tree. Otherwise, we don't insert the field, and we leave that part of the response untouched.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3810
GitOrigin-RevId: 89aaf16274d68e26ad3730b80c2d2fdc2896b96c
2022-03-09 06:17:28 +03:00
|
|
|
import Hasura.RQL.IR.RemoteSchema qualified as IR
|
2022-04-27 16:57:28 +03:00
|
|
|
import Hasura.RQL.Types.Relationships.Remote
|
|
|
|
import Hasura.RQL.Types.ResultCustomization
|
scaffolding for remote-schemas module
The main aim of the PR is:
1. To set up a module structure for 'remote-schemas' package.
2. Move parts by the remote schema codebase into the new module structure to validate it.
## Notes to the reviewer
Why a PR with large-ish diff?
1. We've been making progress on the MM project but we don't yet know long it is going to take us to get to the first milestone. To understand this better, we need to figure out the unknowns as soon as possible. Hence I've taken a stab at the first two items in the [end-state](https://gist.github.com/0x777/ca2bdc4284d21c3eec153b51dea255c9) document to figure out the unknowns. Unsurprisingly, there are a bunch of issues that we haven't discussed earlier. These are documented in the 'open questions' section.
1. The diff is large but that is only code moved around and I've added a section that documents how things are moved. In addition, there are fair number of PR comments to help with the review process.
## Changes in the PR
### Module structure
Sets up the module structure as follows:
```
Hasura/
RemoteSchema/
Metadata/
Types.hs
SchemaCache/
Types.hs
Permission.hs
RemoteRelationship.hs
Build.hs
MetadataAPI/
Types.hs
Execute.hs
```
### 1. Types representing metadata are moved
Types that capture metadata information (currently scattered across several RQL modules) are moved into `Hasura.RemoteSchema.Metadata.Types`.
- This new module only depends on very 'core' modules such as
`Hasura.Session` for the notion of roles and `Hasura.Incremental` for `Cacheable` typeclass.
- The requirement on database modules is avoided by generalizing the remote schemas metadata to accept an arbitrary 'r' for a remote relationship
definition.
### 2. SchemaCache related types and build logic have been moved
Types that represent remote schemas information in SchemaCache are moved into `Hasura.RemoteSchema.SchemaCache.Types`.
Similar to `H.RS.Metadata.Types`, this module depends on 'core' modules except for `Hasura.GraphQL.Parser.Variable`. It has something to do with remote relationships but I haven't spent time looking into it. The validation of 'remote relationships to remote schema' is also something that needs to be looked at.
Rips out the logic that builds remote schema's SchemaCache information from the monolithic `buildSchemaCacheRule` and moves it into `Hasura.RemoteSchema.SchemaCache.Build`. Further, the `.SchemaCache.Permission` and `.SchemaCache.RemoteRelationship` have been created from existing modules that capture schema cache building logic for those two components.
This was a fair amount of work. On main, currently remote schema's SchemaCache information is built in two phases - in the first phase, 'permissions' and 'remote relationships' are ignored and in the second phase they are filled in.
While remote relationships can only be resolved after partially resolving sources and other remote schemas, the same isn't true for permissions. Further, most of the work that is done to resolve remote relationships can be moved to the first phase so that the second phase can be a very simple traversal.
This is the approach that was taken - resolve permissions and as much as remote relationships information in the first phase.
### 3. Metadata APIs related types and build logic have been moved
The types that represent remote schema related metadata APIs and the execution logic have been moved to `Hasura.RemoteSchema.MetadataAPI.Types` and `.Execute` modules respectively.
## Open questions:
1. `Hasura.RemoteSchema.Metadata.Types` is so called because I was hoping that all of the metadata related APIs of remote schema can be brought in at `Hasura.RemoteSchema.Metadata.API`. However, as metadata APIs depended on functions from `SchemaCache` module (see [1](https://github.com/hasura/graphql-engine-mono/blob/ceba6d62264603ee5d279814677b29bcc43ecaea/server/src-lib/Hasura/RQL/DDL/RemoteSchema.hs#L55) and [2](https://github.com/hasura/graphql-engine-mono/blob/ceba6d62264603ee5d279814677b29bcc43ecaea/server/src-lib/Hasura/RQL/DDL/RemoteSchema.hs#L91), it made more sense to create a separate top-level module for `MetadataAPI`s.
Maybe we can just have `Hasura.RemoteSchema.Metadata` and get rid of the extra nesting or have `Hasura.RemoteSchema.Metadata.{Core,Permission,RemoteRelationship}` if we want to break them down further.
1. `buildRemoteSchemas` in `H.RS.SchemaCache.Build` has the following type:
```haskell
buildRemoteSchemas ::
( ArrowChoice arr,
Inc.ArrowDistribute arr,
ArrowWriter (Seq CollectedInfo) arr,
Inc.ArrowCache m arr,
MonadIO m,
HasHttpManagerM m,
Inc.Cacheable remoteRelationshipDefinition,
ToJSON remoteRelationshipDefinition,
MonadError QErr m
) =>
Env.Environment ->
( (Inc.Dependency (HashMap RemoteSchemaName Inc.InvalidationKey), OrderedRoles),
[RemoteSchemaMetadataG remoteRelationshipDefinition]
)
`arr` HashMap RemoteSchemaName (PartiallyResolvedRemoteSchemaCtxG remoteRelationshipDefinition, MetadataObject)
```
Note the dependence on `CollectedInfo` which is defined as
```haskell
data CollectedInfo
= CIInconsistency InconsistentMetadata
| CIDependency
MetadataObject
-- ^ for error reporting on missing dependencies
SchemaObjId
SchemaDependency
deriving (Eq)
```
this pretty much means that remote schemas is dependent on types from databases, actions, ....
How do we fix this? Maybe introduce a typeclass such as `ArrowCollectRemoteSchemaDependencies` which is defined in `Hasura.RemoteSchema` and then implemented in graphql-engine?
1. The dependency on `buildSchemaCacheFor` in `.MetadataAPI.Execute` which has the following signature:
```haskell
buildSchemaCacheFor ::
(QErrM m, CacheRWM m, MetadataM m) =>
MetadataObjId ->
MetadataModifier ->
```
This can be easily resolved if we restrict what the metadata APIs are allowed to do. Currently, they operate in an unfettered access to modify SchemaCache (the `CacheRWM` constraint):
```haskell
runAddRemoteSchema ::
( QErrM m,
CacheRWM m,
MonadIO m,
HasHttpManagerM m,
MetadataM m,
Tracing.MonadTrace m
) =>
Env.Environment ->
AddRemoteSchemaQuery ->
m EncJSON
```
This should instead be changed to restrict remote schema APIs to only modify remote schema metadata (but has access to the remote schemas part of the schema cache), this dependency is completely removed.
```haskell
runAddRemoteSchema ::
( QErrM m,
MonadIO m,
HasHttpManagerM m,
MonadReader RemoteSchemasSchemaCache m,
MonadState RemoteSchemaMetadata m,
Tracing.MonadTrace m
) =>
Env.Environment ->
AddRemoteSchemaQuery ->
m RemoteSchemeMetadataObjId
```
The idea is that the core graphql-engine would call these functions and then call
`buildSchemaCacheFor`.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6291
GitOrigin-RevId: 51357148c6404afe70219afa71bd1d59bdf4ffc6
2022-10-21 06:13:07 +03:00
|
|
|
import Hasura.RemoteSchema.SchemaCache
|
2020-12-21 12:11:37 +03:00
|
|
|
import Hasura.Session
|
2020-10-07 13:23:17 +03:00
|
|
|
import Language.GraphQL.Draft.Syntax qualified as G
|
2021-02-12 06:04:09 +03:00
|
|
|
|
Enable remote joins from remote schemas in the execution engine.
### Description
This PR adds the ability to perform remote joins from remote schemas in the engine. To do so, we alter the definition of an `ExecutionStep` targeting a remote schema: the `ExecStepRemote` constructor now expects a `Maybe RemoteJoins`. This new argument is used when processing the execution step, in the transport layer (either `Transport.HTTP` or `Transport.WebSocket`).
For this `Maybe RemoteJoins` to be extracted from a parsed query, this PR also extends the `Execute.RemoteJoin.Collect` module, to implement "collection" from a selection set. Not only do those new functions extract the remote joins, but they also apply all necessary transformations to the selection sets (such as inserting the necessary "phantom" fields used as join keys).
Finally in `Execute.RemoteJoin.Join`, we make two changes. First, we now always look for nested remote joins, regardless of whether the join we just performed went to a source or a remote schema; and second we adapt our join tree logic according to the special cases that were added to deal with remote server edge cases.
Additionally, this PR refactors / cleans / documents `Execute.RemoteJoin.RemoteServer`. This is not required as part of this change and could be moved to a separate PR if needed (a similar cleanup of `Join` is done independently in #3894). It also introduces a draft of a new documentation page for this project, that will be refined in the release PR that ships the feature (either #3069 or a copy of it).
While this PR extends the engine, it doesn't plug such relationships in the schema, meaning that, as of this PR, the new code paths in `Join` are technically unreachable. Adding the corresponding schema code and, ultimately, enabling the metadata API will be done in subsequent PRs.
### Keeping track of concrete type names
The main change this PR makes to the existing `Join` code is to handle a new reserved field we sometimes use when targeting remote servers: the `__hasura_internal_typename` field. In short, a GraphQL selection set can sometimes "branch" based on the concrete "runtime type" of the object on which the selection happens:
```graphql
query {
author(id: 53478) {
... on Writer {
name
articles {
title
}
}
... on Artist {
name
articles {
title
}
}
}
}
```
If both of those `articles` are remote joins, we need to be able, when we get the answer, to differentiate between the two different cases. We do this by asking for `__typename`, to be able to decide if we're in the `Writer` or the `Artist` branch of the query.
To avoid further processing / customization of results, we only insert this `__hasura_internal_typename: __typename` field in the query in the case of unions of interfaces AND if we have the guarantee that we will processing the request as part of the remote joins "folding": that is, if there's any remote join in this branch in the tree. Otherwise, we don't insert the field, and we leave that part of the response untouched.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3810
GitOrigin-RevId: 89aaf16274d68e26ad3730b80c2d2fdc2896b96c
2022-03-09 06:17:28 +03:00
|
|
|
getVariableDefinitionAndValue :: Variable -> (G.VariableDefinition, (G.Name, J.Value))
|
|
|
|
getVariableDefinitionAndValue var@(Variable varInfo gType varValue) =
|
2020-12-21 12:11:37 +03:00
|
|
|
(varDefn, (varName, varJSONValue))
|
|
|
|
where
|
|
|
|
varName = getName var
|
|
|
|
|
|
|
|
varDefn = G.VariableDefinition varName gType defaultVal
|
|
|
|
|
|
|
|
defaultVal =
|
|
|
|
case varInfo of
|
2020-12-28 15:56:00 +03:00
|
|
|
VIRequired _ -> Nothing
|
2020-12-21 12:11:37 +03:00
|
|
|
VIOptional _ val -> Just val
|
|
|
|
|
|
|
|
varJSONValue =
|
|
|
|
case varValue of
|
2020-12-28 15:56:00 +03:00
|
|
|
JSONValue v -> v
|
2020-12-21 12:11:37 +03:00
|
|
|
GraphQLValue val -> graphQLValueToJSON val
|
|
|
|
|
|
|
|
unresolveVariables ::
|
|
|
|
forall fragments.
|
|
|
|
Functor fragments =>
|
|
|
|
G.SelectionSet fragments Variable ->
|
|
|
|
G.SelectionSet fragments G.Name
|
|
|
|
unresolveVariables =
|
|
|
|
fmap (fmap (getName . vInfo))
|
|
|
|
|
2020-10-07 13:23:17 +03:00
|
|
|
collectVariables ::
|
|
|
|
forall fragments var.
|
2022-11-15 14:25:04 +03:00
|
|
|
(Foldable fragments, Hashable var) =>
|
2020-10-07 13:23:17 +03:00
|
|
|
G.SelectionSet fragments var ->
|
|
|
|
Set.HashSet var
|
|
|
|
collectVariables =
|
|
|
|
Set.unions . fmap (foldMap Set.singleton)
|
|
|
|
|
|
|
|
buildExecStepRemote ::
|
2021-02-20 16:45:49 +03:00
|
|
|
RemoteSchemaInfo ->
|
2021-10-29 17:42:07 +03:00
|
|
|
ResultCustomizer ->
|
2020-10-07 13:23:17 +03:00
|
|
|
G.OperationType ->
|
Enable remote joins from remote schemas in the execution engine.
### Description
This PR adds the ability to perform remote joins from remote schemas in the engine. To do so, we alter the definition of an `ExecutionStep` targeting a remote schema: the `ExecStepRemote` constructor now expects a `Maybe RemoteJoins`. This new argument is used when processing the execution step, in the transport layer (either `Transport.HTTP` or `Transport.WebSocket`).
For this `Maybe RemoteJoins` to be extracted from a parsed query, this PR also extends the `Execute.RemoteJoin.Collect` module, to implement "collection" from a selection set. Not only do those new functions extract the remote joins, but they also apply all necessary transformations to the selection sets (such as inserting the necessary "phantom" fields used as join keys).
Finally in `Execute.RemoteJoin.Join`, we make two changes. First, we now always look for nested remote joins, regardless of whether the join we just performed went to a source or a remote schema; and second we adapt our join tree logic according to the special cases that were added to deal with remote server edge cases.
Additionally, this PR refactors / cleans / documents `Execute.RemoteJoin.RemoteServer`. This is not required as part of this change and could be moved to a separate PR if needed (a similar cleanup of `Join` is done independently in #3894). It also introduces a draft of a new documentation page for this project, that will be refined in the release PR that ships the feature (either #3069 or a copy of it).
While this PR extends the engine, it doesn't plug such relationships in the schema, meaning that, as of this PR, the new code paths in `Join` are technically unreachable. Adding the corresponding schema code and, ultimately, enabling the metadata API will be done in subsequent PRs.
### Keeping track of concrete type names
The main change this PR makes to the existing `Join` code is to handle a new reserved field we sometimes use when targeting remote servers: the `__hasura_internal_typename` field. In short, a GraphQL selection set can sometimes "branch" based on the concrete "runtime type" of the object on which the selection happens:
```graphql
query {
author(id: 53478) {
... on Writer {
name
articles {
title
}
}
... on Artist {
name
articles {
title
}
}
}
}
```
If both of those `articles` are remote joins, we need to be able, when we get the answer, to differentiate between the two different cases. We do this by asking for `__typename`, to be able to decide if we're in the `Writer` or the `Artist` branch of the query.
To avoid further processing / customization of results, we only insert this `__hasura_internal_typename: __typename` field in the query in the case of unions of interfaces AND if we have the guarantee that we will processing the request as part of the remote joins "folding": that is, if there's any remote join in this branch in the tree. Otherwise, we don't insert the field, and we leave that part of the response untouched.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3810
GitOrigin-RevId: 89aaf16274d68e26ad3730b80c2d2fdc2896b96c
2022-03-09 06:17:28 +03:00
|
|
|
IR.GraphQLField Void Variable ->
|
|
|
|
Maybe RemoteJoins ->
|
2022-02-16 10:54:21 +03:00
|
|
|
Maybe OperationName ->
|
2021-02-20 16:45:49 +03:00
|
|
|
ExecutionStep
|
Enable remote joins from remote schemas in the execution engine.
### Description
This PR adds the ability to perform remote joins from remote schemas in the engine. To do so, we alter the definition of an `ExecutionStep` targeting a remote schema: the `ExecStepRemote` constructor now expects a `Maybe RemoteJoins`. This new argument is used when processing the execution step, in the transport layer (either `Transport.HTTP` or `Transport.WebSocket`).
For this `Maybe RemoteJoins` to be extracted from a parsed query, this PR also extends the `Execute.RemoteJoin.Collect` module, to implement "collection" from a selection set. Not only do those new functions extract the remote joins, but they also apply all necessary transformations to the selection sets (such as inserting the necessary "phantom" fields used as join keys).
Finally in `Execute.RemoteJoin.Join`, we make two changes. First, we now always look for nested remote joins, regardless of whether the join we just performed went to a source or a remote schema; and second we adapt our join tree logic according to the special cases that were added to deal with remote server edge cases.
Additionally, this PR refactors / cleans / documents `Execute.RemoteJoin.RemoteServer`. This is not required as part of this change and could be moved to a separate PR if needed (a similar cleanup of `Join` is done independently in #3894). It also introduces a draft of a new documentation page for this project, that will be refined in the release PR that ships the feature (either #3069 or a copy of it).
While this PR extends the engine, it doesn't plug such relationships in the schema, meaning that, as of this PR, the new code paths in `Join` are technically unreachable. Adding the corresponding schema code and, ultimately, enabling the metadata API will be done in subsequent PRs.
### Keeping track of concrete type names
The main change this PR makes to the existing `Join` code is to handle a new reserved field we sometimes use when targeting remote servers: the `__hasura_internal_typename` field. In short, a GraphQL selection set can sometimes "branch" based on the concrete "runtime type" of the object on which the selection happens:
```graphql
query {
author(id: 53478) {
... on Writer {
name
articles {
title
}
}
... on Artist {
name
articles {
title
}
}
}
}
```
If both of those `articles` are remote joins, we need to be able, when we get the answer, to differentiate between the two different cases. We do this by asking for `__typename`, to be able to decide if we're in the `Writer` or the `Artist` branch of the query.
To avoid further processing / customization of results, we only insert this `__hasura_internal_typename: __typename` field in the query in the case of unions of interfaces AND if we have the guarantee that we will processing the request as part of the remote joins "folding": that is, if there's any remote join in this branch in the tree. Otherwise, we don't insert the field, and we leave that part of the response untouched.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3810
GitOrigin-RevId: 89aaf16274d68e26ad3730b80c2d2fdc2896b96c
2022-03-09 06:17:28 +03:00
|
|
|
buildExecStepRemote remoteSchemaInfo resultCustomizer tp rootField remoteJoins operationName =
|
|
|
|
let selSet = [G.SelectionField $ IR.convertGraphQLField rootField]
|
|
|
|
unresolvedSelSet = unresolveVariables selSet
|
|
|
|
allVars = map getVariableDefinitionAndValue $ Set.toList $ collectVariables selSet
|
2023-04-26 18:42:13 +03:00
|
|
|
varValues = HashMap.fromList $ map snd allVars
|
|
|
|
varValsM = bool (Just varValues) Nothing $ HashMap.null varValues
|
2020-12-21 12:11:37 +03:00
|
|
|
varDefs = map fst allVars
|
2022-02-16 10:54:21 +03:00
|
|
|
_grQuery = G.TypedOperationDefinition tp (_unOperationName <$> operationName) varDefs [] unresolvedSelSet
|
2020-12-21 12:11:37 +03:00
|
|
|
_grVariables = varValsM
|
2022-02-16 10:54:21 +03:00
|
|
|
_grOperationName = operationName
|
Enable remote joins from remote schemas in the execution engine.
### Description
This PR adds the ability to perform remote joins from remote schemas in the engine. To do so, we alter the definition of an `ExecutionStep` targeting a remote schema: the `ExecStepRemote` constructor now expects a `Maybe RemoteJoins`. This new argument is used when processing the execution step, in the transport layer (either `Transport.HTTP` or `Transport.WebSocket`).
For this `Maybe RemoteJoins` to be extracted from a parsed query, this PR also extends the `Execute.RemoteJoin.Collect` module, to implement "collection" from a selection set. Not only do those new functions extract the remote joins, but they also apply all necessary transformations to the selection sets (such as inserting the necessary "phantom" fields used as join keys).
Finally in `Execute.RemoteJoin.Join`, we make two changes. First, we now always look for nested remote joins, regardless of whether the join we just performed went to a source or a remote schema; and second we adapt our join tree logic according to the special cases that were added to deal with remote server edge cases.
Additionally, this PR refactors / cleans / documents `Execute.RemoteJoin.RemoteServer`. This is not required as part of this change and could be moved to a separate PR if needed (a similar cleanup of `Join` is done independently in #3894). It also introduces a draft of a new documentation page for this project, that will be refined in the release PR that ships the feature (either #3069 or a copy of it).
While this PR extends the engine, it doesn't plug such relationships in the schema, meaning that, as of this PR, the new code paths in `Join` are technically unreachable. Adding the corresponding schema code and, ultimately, enabling the metadata API will be done in subsequent PRs.
### Keeping track of concrete type names
The main change this PR makes to the existing `Join` code is to handle a new reserved field we sometimes use when targeting remote servers: the `__hasura_internal_typename` field. In short, a GraphQL selection set can sometimes "branch" based on the concrete "runtime type" of the object on which the selection happens:
```graphql
query {
author(id: 53478) {
... on Writer {
name
articles {
title
}
}
... on Artist {
name
articles {
title
}
}
}
}
```
If both of those `articles` are remote joins, we need to be able, when we get the answer, to differentiate between the two different cases. We do this by asking for `__typename`, to be able to decide if we're in the `Writer` or the `Artist` branch of the query.
To avoid further processing / customization of results, we only insert this `__hasura_internal_typename: __typename` field in the query in the case of unions of interfaces AND if we have the guarantee that we will processing the request as part of the remote joins "folding": that is, if there's any remote join in this branch in the tree. Otherwise, we don't insert the field, and we leave that part of the response untouched.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3810
GitOrigin-RevId: 89aaf16274d68e26ad3730b80c2d2fdc2896b96c
2022-03-09 06:17:28 +03:00
|
|
|
in ExecStepRemote remoteSchemaInfo resultCustomizer GH.GQLReq {..} remoteJoins
|
2020-12-21 12:11:37 +03:00
|
|
|
|
2021-08-05 00:23:33 +03:00
|
|
|
-- | Association between keys uniquely identifying some remote JSON variable and
|
|
|
|
-- an 'Int' identifier that will be used to construct a valid variable name to
|
|
|
|
-- be used in a GraphQL query.
|
|
|
|
newtype RemoteJSONVariableMap
|
|
|
|
= RemoteJSONVariableMap (HashMap RemoteJSONVariableKey Int)
|
|
|
|
deriving newtype (Eq, Monoid, Semigroup)
|
|
|
|
|
|
|
|
-- | A unique identifier for some remote JSON variable whose name will need to
|
|
|
|
-- be substituted when constructing a GraphQL query.
|
|
|
|
--
|
|
|
|
-- For a detailed explanation of this behavior, see the following comment:
|
|
|
|
-- https://github.com/hasura/graphql-engine/issues/7170#issuecomment-880838970
|
|
|
|
data RemoteJSONVariableKey = RemoteJSONVariableKey !G.GType !J.Value
|
|
|
|
deriving stock (Eq, Generic)
|
|
|
|
deriving anyclass (Hashable)
|
2020-12-21 12:11:37 +03:00
|
|
|
|
2021-08-05 00:23:33 +03:00
|
|
|
-- | Resolves a `RemoteSchemaVariable` into a GraphQL `Variable`.
|
|
|
|
--
|
|
|
|
-- A `RemoteSchemaVariable` can either be a query variable (i.e. a variable
|
|
|
|
-- provided in the query) or it can be a `SessionPresetVariable` (in which case
|
|
|
|
-- we look up the value of the session variable and coerce it into the
|
|
|
|
-- appropriate type and then construct the GraphQL 'Variable').
|
2020-12-21 12:11:37 +03:00
|
|
|
--
|
2021-08-05 00:23:33 +03:00
|
|
|
-- NOTE: The session variable preset is a hard preset (i.e. if the session
|
|
|
|
-- variable doesn't exist, an error will be thrown).
|
2020-12-21 12:11:37 +03:00
|
|
|
--
|
2021-08-05 00:23:33 +03:00
|
|
|
-- The name of the GraphQL variable generated will be a GraphQL-ized version of
|
|
|
|
-- the session variable (i.e. '-' will be replaced with '_'), since session
|
|
|
|
-- variables are not valid GraphQL names.
|
2020-12-21 12:11:37 +03:00
|
|
|
--
|
2021-08-05 00:23:33 +03:00
|
|
|
-- Additionally, we need to handle partially traversed JSON values; likewise, we
|
|
|
|
-- create a new variable out of thin air.
|
2020-12-21 12:11:37 +03:00
|
|
|
--
|
2021-05-24 23:12:53 +03:00
|
|
|
-- For example, considering the following schema for a role:
|
|
|
|
--
|
|
|
|
-- input UserName {
|
|
|
|
-- firstName : String! @preset(value:"Foo")
|
|
|
|
-- lastName : String!
|
|
|
|
-- }
|
|
|
|
--
|
2020-12-21 12:11:37 +03:00
|
|
|
-- type Query {
|
2021-05-24 23:12:53 +03:00
|
|
|
-- user(
|
|
|
|
-- user_id: Int! @preset(value:"x-hasura-user-id")
|
|
|
|
-- user_name: UserName!
|
|
|
|
-- ): User
|
2020-12-21 12:11:37 +03:00
|
|
|
-- }
|
|
|
|
--
|
2021-05-24 23:12:53 +03:00
|
|
|
-- and the incoming query to the graphql-engine is:
|
2020-12-21 12:11:37 +03:00
|
|
|
--
|
2021-05-24 23:12:53 +03:00
|
|
|
-- query($foo: UserName!) {
|
|
|
|
-- user(user_name: $foo) { id name }
|
2020-12-21 12:11:37 +03:00
|
|
|
-- }
|
|
|
|
--
|
2021-05-24 23:12:53 +03:00
|
|
|
-- with variables:
|
|
|
|
--
|
|
|
|
-- { "foo": {"lastName": "Bar"} }
|
2020-12-21 12:11:37 +03:00
|
|
|
--
|
2021-05-24 23:12:53 +03:00
|
|
|
--
|
2021-08-05 00:23:33 +03:00
|
|
|
-- After resolving the session argument presets, the query that will be sent to
|
|
|
|
-- the remote server will be:
|
2021-05-24 23:12:53 +03:00
|
|
|
--
|
|
|
|
-- query ($x_hasura_user_id: Int!, $hasura_json_var_1: String!) {
|
|
|
|
-- user (user_id: $x_hasura_user_id, user_name: {firstName: "Foo", lastName: $hasura_json_var_1}) {
|
|
|
|
-- id
|
|
|
|
-- name
|
2020-12-21 12:11:37 +03:00
|
|
|
-- }
|
2021-05-24 23:12:53 +03:00
|
|
|
-- }
|
2020-12-21 12:11:37 +03:00
|
|
|
resolveRemoteVariable ::
|
|
|
|
(MonadError QErr m) =>
|
|
|
|
UserInfo ->
|
|
|
|
RemoteSchemaVariable ->
|
2021-08-05 00:23:33 +03:00
|
|
|
StateT RemoteJSONVariableMap m Variable
|
2020-12-21 12:11:37 +03:00
|
|
|
resolveRemoteVariable userInfo = \case
|
|
|
|
SessionPresetVariable sessionVar typeName presetInfo -> do
|
|
|
|
sessionVarVal <-
|
|
|
|
onNothing (getSessionVariableValue sessionVar $ _uiSession userInfo) $
|
|
|
|
throw400 NotFound $
|
|
|
|
sessionVar <<> " session variable expected, but not found"
|
2022-01-27 18:12:38 +03:00
|
|
|
varName <-
|
|
|
|
sessionVariableToGraphQLName sessionVar
|
|
|
|
`onNothing` throw500 ("'" <> sessionVariableToText sessionVar <> "' cannot be made into a valid GraphQL name")
|
2020-12-21 12:11:37 +03:00
|
|
|
coercedValue <-
|
|
|
|
case presetInfo of
|
|
|
|
SessionArgumentPresetScalar ->
|
|
|
|
case G.unName typeName of
|
|
|
|
"Int" ->
|
|
|
|
case readMaybe $ T.unpack sessionVarVal of
|
|
|
|
Nothing -> throw400 CoercionError $ sessionVarVal <<> " cannot be coerced into an Int value"
|
|
|
|
Just i -> pure $ G.VInt i
|
|
|
|
"Boolean" ->
|
|
|
|
if
|
|
|
|
| sessionVarVal `elem` ["true", "false"] ->
|
|
|
|
pure $ G.VBoolean $ "true" == sessionVarVal
|
|
|
|
| otherwise ->
|
|
|
|
throw400 CoercionError $ sessionVarVal <<> " cannot be coerced into a Boolean value"
|
|
|
|
"Float" ->
|
|
|
|
case readMaybe $ T.unpack sessionVarVal of
|
|
|
|
Nothing ->
|
|
|
|
throw400 CoercionError $ sessionVarVal <<> " cannot be coerced into a Float value"
|
|
|
|
Just i -> pure $ G.VFloat i
|
|
|
|
-- The `String`,`ID` and the default case all use the same code. But,
|
|
|
|
-- it will be better to not merge all of them into the default case
|
|
|
|
-- because it will be helpful to know how all the built-in scalars
|
|
|
|
-- are handled
|
|
|
|
"String" -> pure $ G.VString sessionVarVal
|
|
|
|
"ID" -> pure $ G.VString sessionVarVal
|
|
|
|
-- When we encounter a custom scalar, we just pass it as a string
|
|
|
|
_ -> pure $ G.VString sessionVarVal
|
|
|
|
SessionArgumentPresetEnum enumVals -> do
|
|
|
|
sessionVarEnumVal <-
|
|
|
|
G.EnumValue
|
|
|
|
<$> onNothing
|
|
|
|
(G.mkName sessionVarVal)
|
|
|
|
(throw400 CoercionError $ sessionVarVal <<> " is not a valid GraphQL name")
|
|
|
|
case sessionVarEnumVal `Set.member` enumVals of
|
|
|
|
True -> pure $ G.VEnum sessionVarEnumVal
|
|
|
|
False -> throw400 CoercionError $ sessionVarEnumVal <<> " is not one of the valid enum values"
|
|
|
|
-- nullability is false, because we treat presets as hard presets
|
|
|
|
let variableGType = G.TypeNamed (G.Nullability False) typeName
|
|
|
|
pure $ Variable (VIRequired varName) variableGType (GraphQLValue coercedValue)
|
2021-05-24 23:12:53 +03:00
|
|
|
RemoteJSONValue gtype jsonValue -> do
|
2021-08-05 00:23:33 +03:00
|
|
|
let key = RemoteJSONVariableKey gtype jsonValue
|
|
|
|
varMap <- gets coerce
|
|
|
|
index <-
|
2023-04-26 18:42:13 +03:00
|
|
|
HashMap.lookup key varMap `onNothing` do
|
|
|
|
let i = HashMap.size varMap + 1
|
|
|
|
put . coerce $ HashMap.insert key i varMap
|
2021-05-24 23:12:53 +03:00
|
|
|
pure i
|
2022-01-27 18:12:38 +03:00
|
|
|
-- This should never fail.
|
|
|
|
let varText = "hasura_json_var_" <> tshow index
|
|
|
|
varName <-
|
|
|
|
G.mkName varText
|
|
|
|
`onNothing` throw500 ("'" <> varText <> "' is not a valid GraphQL name")
|
2021-05-24 23:12:53 +03:00
|
|
|
pure $ Variable (VIRequired varName) gtype $ JSONValue jsonValue
|
2020-12-21 12:11:37 +03:00
|
|
|
QueryVariable variable -> pure variable
|
|
|
|
|
2021-08-05 00:23:33 +03:00
|
|
|
-- | TODO: Documentation.
|
2020-12-21 12:11:37 +03:00
|
|
|
resolveRemoteField ::
|
2021-11-30 03:37:14 +03:00
|
|
|
(MonadError QErr m) =>
|
2020-12-21 12:11:37 +03:00
|
|
|
UserInfo ->
|
Enable remote joins from remote schemas in the execution engine.
### Description
This PR adds the ability to perform remote joins from remote schemas in the engine. To do so, we alter the definition of an `ExecutionStep` targeting a remote schema: the `ExecStepRemote` constructor now expects a `Maybe RemoteJoins`. This new argument is used when processing the execution step, in the transport layer (either `Transport.HTTP` or `Transport.WebSocket`).
For this `Maybe RemoteJoins` to be extracted from a parsed query, this PR also extends the `Execute.RemoteJoin.Collect` module, to implement "collection" from a selection set. Not only do those new functions extract the remote joins, but they also apply all necessary transformations to the selection sets (such as inserting the necessary "phantom" fields used as join keys).
Finally in `Execute.RemoteJoin.Join`, we make two changes. First, we now always look for nested remote joins, regardless of whether the join we just performed went to a source or a remote schema; and second we adapt our join tree logic according to the special cases that were added to deal with remote server edge cases.
Additionally, this PR refactors / cleans / documents `Execute.RemoteJoin.RemoteServer`. This is not required as part of this change and could be moved to a separate PR if needed (a similar cleanup of `Join` is done independently in #3894). It also introduces a draft of a new documentation page for this project, that will be refined in the release PR that ships the feature (either #3069 or a copy of it).
While this PR extends the engine, it doesn't plug such relationships in the schema, meaning that, as of this PR, the new code paths in `Join` are technically unreachable. Adding the corresponding schema code and, ultimately, enabling the metadata API will be done in subsequent PRs.
### Keeping track of concrete type names
The main change this PR makes to the existing `Join` code is to handle a new reserved field we sometimes use when targeting remote servers: the `__hasura_internal_typename` field. In short, a GraphQL selection set can sometimes "branch" based on the concrete "runtime type" of the object on which the selection happens:
```graphql
query {
author(id: 53478) {
... on Writer {
name
articles {
title
}
}
... on Artist {
name
articles {
title
}
}
}
}
```
If both of those `articles` are remote joins, we need to be able, when we get the answer, to differentiate between the two different cases. We do this by asking for `__typename`, to be able to decide if we're in the `Writer` or the `Artist` branch of the query.
To avoid further processing / customization of results, we only insert this `__hasura_internal_typename: __typename` field in the query in the case of unions of interfaces AND if we have the guarantee that we will processing the request as part of the remote joins "folding": that is, if there's any remote join in this branch in the tree. Otherwise, we don't insert the field, and we leave that part of the response untouched.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3810
GitOrigin-RevId: 89aaf16274d68e26ad3730b80c2d2fdc2896b96c
2022-03-09 06:17:28 +03:00
|
|
|
IR.RemoteSchemaRootField r RemoteSchemaVariable ->
|
|
|
|
StateT RemoteJSONVariableMap m (IR.RemoteSchemaRootField r Variable)
|
2020-12-21 12:11:37 +03:00
|
|
|
resolveRemoteField userInfo = traverse (resolveRemoteVariable userInfo)
|
2021-05-24 23:12:53 +03:00
|
|
|
|
2021-08-05 00:23:33 +03:00
|
|
|
-- | TODO: Documentation.
|
2021-05-24 23:12:53 +03:00
|
|
|
runVariableCache ::
|
|
|
|
Monad m =>
|
2021-08-05 00:23:33 +03:00
|
|
|
StateT RemoteJSONVariableMap m a ->
|
2021-05-24 23:12:53 +03:00
|
|
|
m a
|
|
|
|
runVariableCache = flip evalStateT mempty
|