graphql-engine/server/src-lib/Hasura/RQL/DML/Update.hs
David Overton e5f88d8039 Nested array support for Data Connectors Backend and MongoDB
## Description

This change adds support for querying into nested arrays in Data Connector agents that support such a concept (currently MongoDB).

### DC API changes

- New API type `ColumnType` which allows representing the type of a "column" as either a scalar type, an object reference or an array of `ColumnType`s. This recursive definition allows arbitrary nesting of arrays of types.
- The `type` fields in the API types `ColumnInfo` and `ColumnInsertSchema` now take a `ColumnType` instead of a `ScalarType`.
- To ensure backwards compatibility, a `ColumnType` representing a scalar serialises and deserialises to the same representation as `ScalarType`.
- In queries, the `Field` type now has a new constructor `NestedArrayField`. This contains a nested `Field` along with optional `limit`, `offset`, `where` and `order_by` arguments. (These optional arguments are not yet used by either HGE or the MongoDB agent.)

### MongoDB Haskell agent changes

- The `/schema` endpoint will now recognise arrays within the JSON validation schema and generate corresponding arrays in the DC schema.
- The `/query` endpoint will now handle `NestedArrayField`s within queries (although it does not yet handle `limit`, `offset`, `where` and `order_by`).

### HGE server changes

- The `Backend` type class adds a new type family `XNestedArrays b` to enable nested arrays on a per-backend basis (currently enabled only for the `DataConnector` backend.
- Within `RawColumnInfo` the column type is now represented by a new type `RawColumnType b` which mirrors the shape of the DC API `ColumnType`, but uses `XNestedObjects b` and `XNestedArrays b` type families to allow turning nested object and array supports on or off for a particular backend. In the `DataConnector` backend `API.CustomType` is converted into `RawColumnInfo 'DataConnector` while building the schema.
- In the next stage of schema building, the `RawColumnInfo` is converted into a `StructuredColumnInfo` which allows us to represent the three different types of columns: scalar, object and array. TODO: the `StructuredColumnInfo` looks very similar to the Logical Model types. The main difference is that it uses the `XNestedObjects` and `XNestedArrays` type families. We should be able to combine these two representations.
- The `StructuredColumnInfo` is then placed into a `FIColumn` `FieldInfo`. This involved some refactoring of `FieldInfo` as I had previously split out `FINestedObject` into a separate constructor. However it works out better to represent all "column" fields (i.e. scalar, object and array) using `FIColumn` as this make it easier to implement permission checking correctly. This is the reason the `StructuredColumnInfo` was needed.
- Next, the `FieldInfo` are used to generate `FieldParser`s. We add a new constructor to `AnnFieldG` for `AFNestedArray`. An `AFNestedArray` field parser can contain either a simple array selection or an array aggregate. Simple array `FieldParsers` are currently limited to subfield selection. We will add support for limit, offset, where and order_by in a future PR. We also don't yet generate array aggregate `FieldParsers.
- The new `AFNestedArray` field is handled by the `QueryPlan` module in the `DataConnector` backend. There we generate an `API.NestedArrayField` from the AFNestedArray. We also handle nested arrays when reshaping the response from the DC agent.

## Limitations

- Support for limit, offset, filter (where) and order_by is not yet fully implemented, although it should not be hard to add this
- Support for aggregations on nested arrays is not yet fully implemented
- Permissions involving nested arrays (and objects) not yet implemented
- This should be integrated with Logical Model types, but that will happen in a separate PR

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9149
GitOrigin-RevId: 0e7b71a994fc1d2ca1ef73bfe7b96e95b5328531
2023-05-24 08:02:43 +00:00

242 lines
7.2 KiB
Haskell

module Hasura.RQL.DML.Update
( runUpdate,
)
where
import Control.Lens ((^?))
import Control.Monad.Trans.Control (MonadBaseControl)
import Data.Aeson.Types
import Data.HashMap.Strict qualified as HashMap
import Data.Sequence qualified as DS
import Data.Text.Extended
import Database.PG.Query qualified as PG
import Hasura.Backends.Postgres.Connection
import Hasura.Backends.Postgres.Execute.Mutation
import Hasura.Backends.Postgres.SQL.DML qualified as S
import Hasura.Backends.Postgres.SQL.Types
import Hasura.Backends.Postgres.Translate.Returning
import Hasura.Backends.Postgres.Types.Table
import Hasura.Backends.Postgres.Types.Update
import Hasura.Base.Error
import Hasura.EncJSON
import Hasura.Prelude
import Hasura.QueryTags
import Hasura.RQL.DML.Internal
import Hasura.RQL.DML.Types
import Hasura.RQL.IR.BoolExp
import Hasura.RQL.IR.Update
import Hasura.RQL.IR.Update.Batch
import Hasura.RQL.Types.BackendType
import Hasura.RQL.Types.Column
import Hasura.RQL.Types.Common
import Hasura.RQL.Types.Metadata
import Hasura.RQL.Types.Permission
import Hasura.RQL.Types.SchemaCache
import Hasura.SQL.Types
import Hasura.Session
import Hasura.Table.Cache
import Hasura.Tracing qualified as Tracing
convInc ::
(QErrM m) =>
ValueParser ('Postgres 'Vanilla) m S.SQLExp ->
PGCol ->
ColumnType ('Postgres 'Vanilla) ->
Value ->
m (PGCol, S.SQLExp)
convInc f col colType val = do
prepExp <- f (CollectableTypeScalar colType) val
return (col, S.SEOpApp S.incOp [S.mkSIdenExp col, prepExp])
convMul ::
(QErrM m) =>
ValueParser ('Postgres 'Vanilla) m S.SQLExp ->
PGCol ->
ColumnType ('Postgres 'Vanilla) ->
Value ->
m (PGCol, S.SQLExp)
convMul f col colType val = do
prepExp <- f (CollectableTypeScalar colType) val
return (col, S.SEOpApp S.mulOp [S.mkSIdenExp col, prepExp])
convSet ::
(QErrM m) =>
ValueParser ('Postgres 'Vanilla) m S.SQLExp ->
PGCol ->
ColumnType ('Postgres 'Vanilla) ->
Value ->
m (PGCol, S.SQLExp)
convSet f col colType val = do
prepExp <- f (CollectableTypeScalar colType) val
return (col, prepExp)
convDefault :: (Monad m) => PGCol -> ColumnType ('Postgres 'Vanilla) -> () -> m (PGCol, S.SQLExp)
convDefault col _ _ = return (col, S.SEUnsafe "DEFAULT")
convOp ::
(UserInfoM m, QErrM m) =>
FieldInfoMap (FieldInfo ('Postgres 'Vanilla)) ->
[PGCol] ->
UpdPermInfo ('Postgres 'Vanilla) ->
[(PGCol, a)] ->
(PGCol -> ColumnType ('Postgres 'Vanilla) -> a -> m (PGCol, S.SQLExp)) ->
m [(PGCol, S.SQLExp)]
convOp fieldInfoMap preSetCols updPerm objs conv =
forM objs $ \(pgCol, a) -> do
-- if column has predefined value then throw error
when (pgCol `elem` preSetCols) $ throwNotUpdErr pgCol
checkPermOnCol PTUpdate allowedCols pgCol
colType <- askColumnType fieldInfoMap pgCol relWhenPgErr
res <- conv pgCol colType a
-- build a set expression's entry
withPathK (getPGColTxt pgCol) $ return res
where
allowedCols = upiCols updPerm
relWhenPgErr = "relationships can't be updated"
throwNotUpdErr c = do
roleName <- _uiRole <$> askUserInfo
throw400 NotSupported $
"column "
<> c <<> " is not updatable"
<> " for role "
<> roleName <<> "; its value is predefined in permission"
validateUpdateQueryWith ::
(UserInfoM m, QErrM m, TableInfoRM ('Postgres 'Vanilla) m) =>
SessionVariableBuilder m ->
ValueParser ('Postgres 'Vanilla) m S.SQLExp ->
UpdateQuery ->
m (AnnotatedUpdate ('Postgres 'Vanilla))
validateUpdateQueryWith sessVarBldr prepValBldr uq = do
let tableName = uqTable uq
tableInfo <- withPathK "table" $ askTableInfoSource tableName
let coreInfo = _tiCoreInfo tableInfo
-- If it is view then check if it is updatable
mutableView
tableName
viIsUpdatable
(_tciViewInfo coreInfo)
"updatable"
-- Check if the role has update permissions
updPerm <- askUpdPermInfo tableInfo
-- Check if all dependent headers are present
validateHeaders $ upiRequiredHeaders updPerm
-- Check if select is allowed
selPerm <-
modifyErr (<> selNecessaryMsg) $
askSelPermInfo tableInfo
let fieldInfoMap = _tciFieldInfoMap coreInfo
allCols = mapMaybe (^? _SCIScalarColumn) $ getCols fieldInfoMap
preSetObj = upiSet updPerm
preSetCols = HashMap.keys preSetObj
-- convert the object to SQL set expression
setItems <-
withPathK "$set" $
convOp fieldInfoMap preSetCols updPerm (HashMap.toList $ uqSet uq) $
convSet prepValBldr
incItems <-
withPathK "$inc" $
convOp fieldInfoMap preSetCols updPerm (HashMap.toList $ uqInc uq) $
convInc prepValBldr
mulItems <-
withPathK "$mul" $
convOp fieldInfoMap preSetCols updPerm (HashMap.toList $ uqMul uq) $
convMul prepValBldr
defItems <-
withPathK "$default" $
convOp fieldInfoMap preSetCols updPerm ((,()) <$> uqDefault uq) convDefault
-- convert the returning cols into sql returing exp
mAnnRetCols <- forM mRetCols $ \retCols ->
withPathK "returning" $ checkRetCols fieldInfoMap selPerm retCols
resolvedPreSetItems <-
HashMap.toList
<$> mapM (convPartialSQLExp sessVarBldr) preSetObj
let setExpItems =
resolvedPreSetItems
++ setItems
++ incItems
++ mulItems
++ defItems
when (null setExpItems) $
throw400 UnexpectedPayload "atleast one of $set, $inc, $mul has to be present"
-- convert the where clause
annSQLBoolExp <-
withPathK "where" $
convBoolExp fieldInfoMap selPerm (uqWhere uq) sessVarBldr fieldInfoMap prepValBldr
resolvedUpdFltr <-
convAnnBoolExpPartialSQL sessVarBldr $
upiFilter updPerm
resolvedUpdCheck <-
fromMaybe gBoolExpTrue
<$> traverse
(convAnnBoolExpPartialSQL sessVarBldr)
(upiCheck updPerm)
return $
AnnotatedUpdateG
tableName
resolvedUpdFltr
resolvedUpdCheck
( SingleBatch $
UpdateBatch
(HashMap.fromList $ fmap UpdateSet <$> setExpItems)
annSQLBoolExp
)
(mkDefaultMutFlds mAnnRetCols)
allCols
Nothing
where
mRetCols = uqReturning uq
selNecessaryMsg =
"; \"update\" is only allowed if the role "
<> "has \"select\" permission as \"where\" can't be used "
<> "without \"select\" permission on the table"
validateUpdateQuery ::
(QErrM m, UserInfoM m, CacheRM m) =>
UpdateQuery ->
m (AnnotatedUpdate ('Postgres 'Vanilla), DS.Seq PG.PrepArg)
validateUpdateQuery query = do
let source = uqSource query
tableCache :: TableCache ('Postgres 'Vanilla) <- fold <$> askTableCache source
flip runTableCacheRT tableCache $
runDMLP1T $
validateUpdateQueryWith sessVarFromCurrentSetting (valueParserWithCollectableType binRHSBuilder) query
runUpdate ::
forall m.
( QErrM m,
UserInfoM m,
CacheRM m,
MonadBaseControl IO m,
MonadIO m,
Tracing.MonadTrace m,
MetadataM m
) =>
SQLGenCtx ->
UpdateQuery ->
m EncJSON
runUpdate sqlGen q = do
sourceConfig <- askSourceConfig @('Postgres 'Vanilla) (uqSource q)
userInfo <- askUserInfo
let strfyNum = stringifyNum sqlGen
validateUpdateQuery q
>>= runTxWithCtx (_pscExecCtx sourceConfig) (Tx PG.ReadWrite Nothing) LegacyRQLQuery
. flip runReaderT emptyQueryTagsComment
. execUpdateQuery strfyNum Nothing userInfo