graphql-engine/server/src-lib/Hasura/GraphQL/Schema/RemoteRelationship.hs
Vamshi Surabhi a01d1188f2 scaffolding for remote-schemas module
The main aim of the PR is:

1. To set up a module structure for 'remote-schemas' package.
2. Move parts by the remote schema codebase into the new module structure to validate it.

## Notes to the reviewer

Why a PR with large-ish diff?

1. We've been making progress on the MM project but we don't yet know long it is going to take us to get to the first milestone. To understand this better, we need to figure out the unknowns as soon as possible. Hence I've taken a stab at the first two items in the [end-state](https://gist.github.com/0x777/ca2bdc4284d21c3eec153b51dea255c9) document to figure out the unknowns. Unsurprisingly, there are a bunch of issues that we haven't discussed earlier. These are documented in the 'open questions' section.

1. The diff is large but that is only code moved around and I've added a section that documents how things are moved. In addition, there are fair number of PR comments to help with the review process.

## Changes in the PR

### Module structure

Sets up the module structure as follows:

```
Hasura/
  RemoteSchema/
    Metadata/
      Types.hs
    SchemaCache/
      Types.hs
      Permission.hs
      RemoteRelationship.hs
      Build.hs
    MetadataAPI/
      Types.hs
      Execute.hs
```

### 1. Types representing metadata are moved

Types that capture metadata information (currently scattered across several RQL modules) are moved into `Hasura.RemoteSchema.Metadata.Types`.

- This new module only depends on very 'core' modules such as
  `Hasura.Session` for the notion of roles and `Hasura.Incremental` for `Cacheable` typeclass.

- The requirement on database modules is avoided by generalizing the remote schemas metadata to accept an arbitrary 'r' for a remote relationship
  definition.

### 2. SchemaCache related types and build logic have been moved

Types that represent remote schemas information in SchemaCache are moved into `Hasura.RemoteSchema.SchemaCache.Types`.

Similar to `H.RS.Metadata.Types`, this module depends on 'core' modules except for `Hasura.GraphQL.Parser.Variable`. It has something to do with remote relationships but I haven't spent time looking into it. The validation of 'remote relationships to remote schema' is also something that needs to be looked at.

Rips out the logic that builds remote schema's SchemaCache information from the monolithic `buildSchemaCacheRule` and moves it into `Hasura.RemoteSchema.SchemaCache.Build`. Further, the `.SchemaCache.Permission` and `.SchemaCache.RemoteRelationship` have been created from existing modules that capture schema cache building logic for those two components.

This was a fair amount of work. On main, currently remote schema's SchemaCache information is built in two phases - in the first phase, 'permissions' and 'remote relationships' are ignored and in the second phase they are filled in.

While remote relationships can only be resolved after partially resolving sources and other remote schemas, the same isn't true for permissions. Further, most of the work that is done to resolve remote relationships can be moved to the first phase so that the second phase can be a very simple traversal.

This is the approach that was taken - resolve permissions and as much as remote relationships information in the first phase.

### 3. Metadata APIs related types and build logic have been moved

The types that represent remote schema related metadata APIs and the execution logic have been moved to `Hasura.RemoteSchema.MetadataAPI.Types` and `.Execute` modules respectively.

## Open questions:

1. `Hasura.RemoteSchema.Metadata.Types` is so called because I was hoping that all of the metadata related APIs of remote schema can be brought in at `Hasura.RemoteSchema.Metadata.API`. However, as metadata APIs depended on functions from `SchemaCache` module (see [1](ceba6d6226/server/src-lib/Hasura/RQL/DDL/RemoteSchema.hs (L55)) and [2](ceba6d6226/server/src-lib/Hasura/RQL/DDL/RemoteSchema.hs (L91)), it made more sense to create a separate top-level module for `MetadataAPI`s.

   Maybe we can just have `Hasura.RemoteSchema.Metadata` and get rid of the extra nesting or have `Hasura.RemoteSchema.Metadata.{Core,Permission,RemoteRelationship}` if we want to break them down further.

1. `buildRemoteSchemas` in `H.RS.SchemaCache.Build` has the following type:

   ```haskell
   buildRemoteSchemas ::
     ( ArrowChoice arr,
       Inc.ArrowDistribute arr,
       ArrowWriter (Seq CollectedInfo) arr,
       Inc.ArrowCache m arr,
       MonadIO m,
       HasHttpManagerM m,
       Inc.Cacheable remoteRelationshipDefinition,
       ToJSON remoteRelationshipDefinition,
       MonadError QErr m
     ) =>
     Env.Environment ->
     ( (Inc.Dependency (HashMap RemoteSchemaName Inc.InvalidationKey), OrderedRoles),
       [RemoteSchemaMetadataG remoteRelationshipDefinition]
     )
       `arr` HashMap RemoteSchemaName (PartiallyResolvedRemoteSchemaCtxG remoteRelationshipDefinition, MetadataObject)
   ```

   Note the dependence on `CollectedInfo` which is defined as

   ```haskell
   data CollectedInfo
     = CIInconsistency InconsistentMetadata
     | CIDependency
         MetadataObject
         -- ^ for error reporting on missing dependencies
         SchemaObjId
         SchemaDependency
     deriving (Eq)
   ```

   this pretty much means that remote schemas is dependent on types from databases, actions, ....

   How do we fix this? Maybe introduce a typeclass such as `ArrowCollectRemoteSchemaDependencies` which is defined in `Hasura.RemoteSchema` and then implemented in graphql-engine?

1. The dependency on `buildSchemaCacheFor` in `.MetadataAPI.Execute` which has the following signature:

   ```haskell
   buildSchemaCacheFor ::
     (QErrM m, CacheRWM m, MetadataM m) =>
     MetadataObjId ->
     MetadataModifier ->
   ```

   This can be easily resolved if we restrict what the metadata APIs are allowed to do. Currently, they operate in an unfettered access to modify SchemaCache (the `CacheRWM` constraint):

   ```haskell
   runAddRemoteSchema ::
     ( QErrM m,
       CacheRWM m,
       MonadIO m,
       HasHttpManagerM m,
       MetadataM m,
       Tracing.MonadTrace m
     ) =>
     Env.Environment ->
     AddRemoteSchemaQuery ->
     m EncJSON
   ```

   This should instead be changed to restrict remote schema APIs to only modify remote schema metadata (but has access to the remote schemas part of the schema cache), this dependency is completely removed.

   ```haskell
   runAddRemoteSchema ::
     ( QErrM m,
       MonadIO m,
       HasHttpManagerM m,
       MonadReader RemoteSchemasSchemaCache m,
       MonadState RemoteSchemaMetadata m,
       Tracing.MonadTrace m
     ) =>
     Env.Environment ->
     AddRemoteSchemaQuery ->
     m RemoteSchemeMetadataObjId
   ```

   The idea is that the core graphql-engine would call these functions and then call
   `buildSchemaCacheFor`.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6291
GitOrigin-RevId: 51357148c6404afe70219afa71bd1d59bdf4ffc6
2022-10-21 03:15:04 +00:00

227 lines
11 KiB
Haskell

module Hasura.GraphQL.Schema.RemoteRelationship
( remoteRelationshipField,
)
where
import Control.Lens
import Data.Has
import Data.HashMap.Strict.Extended qualified as Map
import Data.List.NonEmpty qualified as NE
import Data.Text.Casing qualified as C
import Data.Text.Extended
import Hasura.Base.Error
import Hasura.GraphQL.Schema.Backend
import Hasura.GraphQL.Schema.Common
import Hasura.GraphQL.Schema.Instances ()
import Hasura.GraphQL.Schema.Options
import Hasura.GraphQL.Schema.Parser (FieldParser, MonadMemoize)
import Hasura.GraphQL.Schema.Parser qualified as P
import Hasura.GraphQL.Schema.Remote
import Hasura.GraphQL.Schema.Select
import Hasura.GraphQL.Schema.Table
import Hasura.Name qualified as Name
import Hasura.Prelude
import Hasura.RQL.IR qualified as IR
import Hasura.RQL.Types.Common (FieldName, RelType (..), relNameToTxt)
import Hasura.RQL.Types.Relationships.Remote
import Hasura.RQL.Types.ResultCustomization
import Hasura.RQL.Types.SchemaCache hiding (askTableInfo)
import Hasura.RQL.Types.Source
import Hasura.RQL.Types.SourceCustomization
import Hasura.RemoteSchema.Metadata
import Hasura.RemoteSchema.SchemaCache
import Hasura.RemoteSchema.SchemaCache qualified as Remote
import Hasura.SQL.AnyBackend
import Hasura.Session
import Language.GraphQL.Draft.Syntax qualified as G
-- | Remote relationship field parsers
remoteRelationshipField ::
SchemaContext ->
SchemaOptions ->
SourceCache ->
RemoteSchemaMap ->
RemoteSchemaPermissions ->
RemoteRelationshipParserBuilder
remoteRelationshipField schemaContext schemaOptions sourceCache remoteSchemaCache remoteSchemaPermissions = RemoteRelationshipParserBuilder
\RemoteFieldInfo {..} -> do
queryType <- retrieve scSchemaKind
-- Remote relationships aren't currently supported in Relay, due to type conflicts, and
-- introspection issues such as https://github.com/hasura/graphql-engine/issues/5144.
if not $ isHasuraSchema queryType
then pure Nothing
else case _rfiRHS of
RFISource anyRemoteSourceFieldInfo ->
-- see Note [SchemaT and stacking]
lift $
runSourceSchema schemaContext schemaOptions $
dispatchAnyBackendWithTwoConstraints @BackendSchema @BackendTableSelectSchema
anyRemoteSourceFieldInfo
\remoteSourceFieldInfo -> do
fields <- remoteRelationshipToSourceField sourceCache remoteSourceFieldInfo
pure $ Just $ fmap (IR.RemoteSourceField . mkAnyBackend) <$> fields
RFISchema remoteSchema ->
-- see Note [SchemaT and stacking]
lift $ runRemoteSchema schemaContext do
fields <- remoteRelationshipToSchemaField remoteSchemaCache remoteSchemaPermissions _rfiLHS remoteSchema
pure $ fmap (pure . fmap IR.RemoteSchemaField) fields
-- | Parser(s) for remote relationship fields to a remote schema
remoteRelationshipToSchemaField ::
forall r m n lhsJoinField.
(MonadBuildRemoteSchema r m n) =>
RemoteSchemaMap ->
RemoteSchemaPermissions ->
Map.HashMap FieldName lhsJoinField ->
RemoteSchemaFieldInfo ->
SchemaT r m (Maybe (FieldParser n (IR.RemoteSchemaSelect (IR.RemoteRelationshipField IR.UnpreparedValue))))
remoteRelationshipToSchemaField remoteSchemaCache remoteSchemaPermissions lhsFields RemoteSchemaFieldInfo {..} = runMaybeT do
roleName <- retrieve scRole
remoteSchemaContext <-
Map.lookup _rrfiRemoteSchemaName remoteSchemaCache
`onNothing` throw500 ("invalid remote schema name: " <>> _rrfiRemoteSchemaName)
introspection <- hoistMaybe $ getIntrospectionResult remoteSchemaPermissions roleName remoteSchemaContext
let remoteSchemaRelationships = _rscRemoteRelationships remoteSchemaContext
roleIntrospection = irDoc introspection
remoteSchemaRoot = irQueryRoot introspection
remoteSchemaCustomizer = rsCustomizer $ _rscInfo remoteSchemaContext
RemoteSchemaIntrospection typeDefns = roleIntrospection
let hasuraFieldNames = Map.keysSet lhsFields
relationshipDef = ToSchemaRelationshipDef _rrfiRemoteSchemaName hasuraFieldNames _rrfiRemoteFields
(newInpValDefns :: [G.TypeDefinition [G.Name] RemoteSchemaInputValueDefinition], remoteFieldParamMap) <-
if roleName == adminRoleName
then do
-- we don't validate the remote relationship when the role is admin
-- because it's already been validated, when the remote relationship
-- was created
pure (_rrfiInputValueDefinitions, _rrfiParamMap)
else do
(_, roleRemoteField) <-
afold @(Either _) $
-- TODO: this really needs to go way, we shouldn't be doing
-- validation when building parsers
validateToSchemaRelationship relationshipDef _rrfiLHSIdentifier _rrfiName (_rrfiRemoteSchema, introspection) lhsFields
pure (Remote._rrfiInputValueDefinitions roleRemoteField, Remote._rrfiParamMap roleRemoteField)
let -- add the new input value definitions created by the remote relationship
-- to the existing schema introspection of the role
remoteRelationshipIntrospection = RemoteSchemaIntrospection $ typeDefns <> Map.fromListOn getTypeName newInpValDefns
fieldName <- textToName $ relNameToTxt _rrfiName
-- This selection set parser, should be of the remote node's selection set parser, which comes
-- from the fieldCall
let fieldCalls = unRemoteFields _rrfiRemoteFields
nestedFieldType <- lift $ lookupNestedFieldType remoteSchemaRoot roleIntrospection fieldCalls
let typeName = G.getBaseType nestedFieldType
fieldTypeDefinition <-
onNothing (lookupType roleIntrospection typeName)
-- the below case will never happen because we get the type name
-- from the schema document itself i.e. if a field exists for the
-- given role, then it's return type also must exist
$
throw500 $ "unexpected: " <> typeName <<> " not found "
-- These are the arguments that are given by the user while executing a query
let remoteFieldUserArguments = map snd $ Map.toList remoteFieldParamMap
remoteFld <-
withRemoteSchemaCustomization remoteSchemaCustomizer $
lift $
P.wrapFieldParser nestedFieldType
<$> remoteField remoteRelationshipIntrospection remoteSchemaRelationships remoteSchemaRoot fieldName Nothing remoteFieldUserArguments fieldTypeDefinition
pure $
remoteFld
`P.bindField` \fld@IR.GraphQLField {IR._fArguments = args, IR._fSelectionSet = selSet, IR._fName = fname} -> do
let remoteArgs =
Map.toList args <&> \(argName, argVal) -> IR.RemoteFieldArgument argName $ P.GraphQLValue argVal
let resultCustomizer =
applyFieldCalls fieldCalls $
applyAliasMapping (singletonAliasMapping fname (fcName $ NE.last fieldCalls)) $
makeResultCustomizer remoteSchemaCustomizer fld
pure $
IR.RemoteSchemaSelect
{ IR._rselArgs = remoteArgs,
IR._rselResultCustomizer = resultCustomizer,
IR._rselSelection = selSet,
IR._rselFieldCall = fieldCalls,
IR._rselRemoteSchema = _rrfiRemoteSchema
}
where
-- Apply parent field calls so that the result customizer modifies the nested field
applyFieldCalls :: NonEmpty FieldCall -> ResultCustomizer -> ResultCustomizer
applyFieldCalls fieldCalls resultCustomizer =
foldr (modifyFieldByName . fcName) resultCustomizer $ NE.init fieldCalls
lookupNestedFieldType' ::
(MonadMemoize m, MonadError QErr m) =>
G.Name ->
RemoteSchemaIntrospection ->
FieldCall ->
SchemaT r m G.GType
lookupNestedFieldType' parentTypeName remoteSchemaIntrospection (FieldCall fcName _) =
case lookupObject remoteSchemaIntrospection parentTypeName of
Nothing -> throw400 RemoteSchemaError $ "object with name " <> parentTypeName <<> " not found"
Just G.ObjectTypeDefinition {..} ->
case find ((== fcName) . G._fldName) _otdFieldsDefinition of
Nothing -> throw400 RemoteSchemaError $ "field with name " <> fcName <<> " not found"
Just G.FieldDefinition {..} -> pure _fldType
lookupNestedFieldType ::
(MonadMemoize m, MonadError QErr m) =>
G.Name ->
RemoteSchemaIntrospection ->
NonEmpty FieldCall ->
SchemaT r m G.GType
lookupNestedFieldType parentTypeName remoteSchemaIntrospection (fieldCall :| rest) = do
fieldType <- lookupNestedFieldType' parentTypeName remoteSchemaIntrospection fieldCall
case NE.nonEmpty rest of
Nothing -> pure fieldType
Just rest' -> do
lookupNestedFieldType (G.getBaseType fieldType) remoteSchemaIntrospection rest'
-- | Parser(s) for remote relationship fields to a database table.
-- Note that when the target is a database table, an array relationship
-- declaration would have the '_aggregate' field in addition to the array
-- relationship field, hence [FieldParser ...] instead of 'FieldParser'
remoteRelationshipToSourceField ::
forall r m n tgt.
( MonadBuildSourceSchema r m n,
BackendSchema tgt,
BackendTableSelectSchema tgt
) =>
SourceCache ->
RemoteSourceFieldInfo tgt ->
SchemaT r m [FieldParser n (IR.RemoteSourceSelect (IR.RemoteRelationshipField IR.UnpreparedValue) IR.UnpreparedValue tgt)]
remoteRelationshipToSourceField sourceCache RemoteSourceFieldInfo {..} = do
roleName <- retrieve scRole
sourceInfo <-
onNothing (unsafeSourceInfo @tgt =<< Map.lookup _rsfiSource sourceCache) $
throw500 $ "source not found " <> dquote _rsfiSource
withSourceCustomization (_siCustomization sourceInfo) do
tCase <- asks getter
tableInfo <- askTableInfo sourceInfo _rsfiTable
fieldName <- textToName $ relNameToTxt _rsfiName
case tableSelectPermissions @tgt roleName tableInfo of
Nothing -> pure []
Just tablePerms -> do
parsers <- case _rsfiType of
ObjRel -> do
selectionSetParserM <- tableSelectionSet sourceInfo tableInfo
pure $ case selectionSetParserM of
Nothing -> []
Just selectionSetParser ->
pure $
P.subselection_ fieldName Nothing selectionSetParser <&> \fields ->
IR.SourceRelationshipObject $
IR.AnnObjectSelectG fields _rsfiTable $ IR._tpFilter $ tablePermissionsInfo tablePerms
ArrRel -> do
let aggFieldName = applyFieldNameCaseIdentifier tCase $ C.fromAutogeneratedTuple (fieldName, [G.convertNameToSuffix Name._aggregate])
selectionSetParser <- selectTable sourceInfo tableInfo fieldName Nothing
aggSelectionSetParser <- selectTableAggregate sourceInfo tableInfo aggFieldName Nothing
pure $
catMaybes
[ selectionSetParser <&> fmap IR.SourceRelationshipArray,
aggSelectionSetParser <&> fmap IR.SourceRelationshipArrayAggregate
]
pure $
parsers <&> fmap \select ->
IR.RemoteSourceSelect _rsfiSource _rsfiSourceConfig select _rsfiMapping