2023-04-19 12:03:36 +03:00
|
|
|
module Hasura.LogicalModel.Common
|
2023-02-15 20:55:06 +03:00
|
|
|
( toFieldInfo,
|
2023-04-17 14:30:10 +03:00
|
|
|
columnsFromFields,
|
2023-04-25 14:58:15 +03:00
|
|
|
logicalModelFieldsToFieldInfo,
|
2023-02-15 20:55:06 +03:00
|
|
|
)
|
|
|
|
where
|
|
|
|
|
2023-04-25 14:58:15 +03:00
|
|
|
import Data.Bifunctor (bimap)
|
2023-04-26 18:42:13 +03:00
|
|
|
import Data.HashMap.Strict qualified as HashMap
|
2023-04-27 10:41:55 +03:00
|
|
|
import Data.HashMap.Strict.InsOrd qualified as InsOrdHashMap
|
2023-02-15 20:55:06 +03:00
|
|
|
import Data.Text.Extended (ToTxt (toTxt))
|
2023-04-28 19:14:39 +03:00
|
|
|
import Hasura.LogicalModel.NullableScalarType (NullableScalarType (..))
|
2023-05-19 19:12:51 +03:00
|
|
|
import Hasura.LogicalModel.Types (LogicalModelField (..), LogicalModelType (..), LogicalModelTypeScalar (..))
|
2023-02-15 20:55:06 +03:00
|
|
|
import Hasura.Prelude
|
|
|
|
import Hasura.RQL.Types.Backend (Backend (..))
|
Nested array support for Data Connectors Backend and MongoDB
## Description
This change adds support for querying into nested arrays in Data Connector agents that support such a concept (currently MongoDB).
### DC API changes
- New API type `ColumnType` which allows representing the type of a "column" as either a scalar type, an object reference or an array of `ColumnType`s. This recursive definition allows arbitrary nesting of arrays of types.
- The `type` fields in the API types `ColumnInfo` and `ColumnInsertSchema` now take a `ColumnType` instead of a `ScalarType`.
- To ensure backwards compatibility, a `ColumnType` representing a scalar serialises and deserialises to the same representation as `ScalarType`.
- In queries, the `Field` type now has a new constructor `NestedArrayField`. This contains a nested `Field` along with optional `limit`, `offset`, `where` and `order_by` arguments. (These optional arguments are not yet used by either HGE or the MongoDB agent.)
### MongoDB Haskell agent changes
- The `/schema` endpoint will now recognise arrays within the JSON validation schema and generate corresponding arrays in the DC schema.
- The `/query` endpoint will now handle `NestedArrayField`s within queries (although it does not yet handle `limit`, `offset`, `where` and `order_by`).
### HGE server changes
- The `Backend` type class adds a new type family `XNestedArrays b` to enable nested arrays on a per-backend basis (currently enabled only for the `DataConnector` backend.
- Within `RawColumnInfo` the column type is now represented by a new type `RawColumnType b` which mirrors the shape of the DC API `ColumnType`, but uses `XNestedObjects b` and `XNestedArrays b` type families to allow turning nested object and array supports on or off for a particular backend. In the `DataConnector` backend `API.CustomType` is converted into `RawColumnInfo 'DataConnector` while building the schema.
- In the next stage of schema building, the `RawColumnInfo` is converted into a `StructuredColumnInfo` which allows us to represent the three different types of columns: scalar, object and array. TODO: the `StructuredColumnInfo` looks very similar to the Logical Model types. The main difference is that it uses the `XNestedObjects` and `XNestedArrays` type families. We should be able to combine these two representations.
- The `StructuredColumnInfo` is then placed into a `FIColumn` `FieldInfo`. This involved some refactoring of `FieldInfo` as I had previously split out `FINestedObject` into a separate constructor. However it works out better to represent all "column" fields (i.e. scalar, object and array) using `FIColumn` as this make it easier to implement permission checking correctly. This is the reason the `StructuredColumnInfo` was needed.
- Next, the `FieldInfo` are used to generate `FieldParser`s. We add a new constructor to `AnnFieldG` for `AFNestedArray`. An `AFNestedArray` field parser can contain either a simple array selection or an array aggregate. Simple array `FieldParsers` are currently limited to subfield selection. We will add support for limit, offset, where and order_by in a future PR. We also don't yet generate array aggregate `FieldParsers.
- The new `AFNestedArray` field is handled by the `QueryPlan` module in the `DataConnector` backend. There we generate an `API.NestedArrayField` from the AFNestedArray. We also handle nested arrays when reshaping the response from the DC agent.
## Limitations
- Support for limit, offset, filter (where) and order_by is not yet fully implemented, although it should not be hard to add this
- Support for aggregations on nested arrays is not yet fully implemented
- Permissions involving nested arrays (and objects) not yet implemented
- This should be integrated with Logical Model types, but that will happen in a separate PR
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9149
GitOrigin-RevId: 0e7b71a994fc1d2ca1ef73bfe7b96e95b5328531
2023-05-24 11:00:59 +03:00
|
|
|
import Hasura.RQL.Types.Column (ColumnInfo (..), ColumnMutability (..), ColumnType (..), StructuredColumnInfo (..), fromCol)
|
2023-05-17 11:53:31 +03:00
|
|
|
import Hasura.Table.Cache (FieldInfo (..), FieldInfoMap)
|
2023-02-15 20:55:06 +03:00
|
|
|
import Language.GraphQL.Draft.Syntax qualified as G
|
|
|
|
|
2023-04-17 14:30:10 +03:00
|
|
|
columnsFromFields ::
|
2023-04-27 10:41:55 +03:00
|
|
|
InsOrdHashMap.InsOrdHashMap k (LogicalModelField b) ->
|
|
|
|
InsOrdHashMap.InsOrdHashMap k (NullableScalarType b)
|
2023-04-17 14:30:10 +03:00
|
|
|
columnsFromFields =
|
2023-04-27 10:41:55 +03:00
|
|
|
InsOrdHashMap.mapMaybe
|
2023-04-17 14:30:10 +03:00
|
|
|
( \case
|
2023-05-19 19:12:51 +03:00
|
|
|
LogicalModelField
|
|
|
|
{ lmfType =
|
|
|
|
LogicalModelTypeScalar
|
|
|
|
( LogicalModelTypeScalarC
|
|
|
|
{ lmtsScalar = nstType,
|
|
|
|
lmtsNullable = nstNullable
|
|
|
|
}
|
|
|
|
),
|
2023-04-19 12:03:36 +03:00
|
|
|
lmfDescription = nstDescription
|
2023-04-17 14:30:10 +03:00
|
|
|
} ->
|
|
|
|
Just (NullableScalarType {..})
|
|
|
|
_ -> Nothing
|
|
|
|
)
|
|
|
|
|
2023-04-27 10:41:55 +03:00
|
|
|
toFieldInfo :: forall b. (Backend b) => InsOrdHashMap.InsOrdHashMap (Column b) (NullableScalarType b) -> Maybe [FieldInfo b]
|
2023-04-04 15:45:20 +03:00
|
|
|
toFieldInfo fields =
|
2023-02-15 20:55:06 +03:00
|
|
|
traverseWithIndex
|
2023-04-19 12:03:36 +03:00
|
|
|
(\i -> fmap FIColumn . logicalModelToColumnInfo i)
|
2023-04-27 10:41:55 +03:00
|
|
|
(InsOrdHashMap.toList fields)
|
2023-02-15 20:55:06 +03:00
|
|
|
|
2023-04-25 14:58:15 +03:00
|
|
|
traverseWithIndex :: (Applicative m) => (Int -> aa -> m bb) -> [aa] -> m [bb]
|
|
|
|
traverseWithIndex f = zipWithM f [0 ..]
|
|
|
|
|
Nested array support for Data Connectors Backend and MongoDB
## Description
This change adds support for querying into nested arrays in Data Connector agents that support such a concept (currently MongoDB).
### DC API changes
- New API type `ColumnType` which allows representing the type of a "column" as either a scalar type, an object reference or an array of `ColumnType`s. This recursive definition allows arbitrary nesting of arrays of types.
- The `type` fields in the API types `ColumnInfo` and `ColumnInsertSchema` now take a `ColumnType` instead of a `ScalarType`.
- To ensure backwards compatibility, a `ColumnType` representing a scalar serialises and deserialises to the same representation as `ScalarType`.
- In queries, the `Field` type now has a new constructor `NestedArrayField`. This contains a nested `Field` along with optional `limit`, `offset`, `where` and `order_by` arguments. (These optional arguments are not yet used by either HGE or the MongoDB agent.)
### MongoDB Haskell agent changes
- The `/schema` endpoint will now recognise arrays within the JSON validation schema and generate corresponding arrays in the DC schema.
- The `/query` endpoint will now handle `NestedArrayField`s within queries (although it does not yet handle `limit`, `offset`, `where` and `order_by`).
### HGE server changes
- The `Backend` type class adds a new type family `XNestedArrays b` to enable nested arrays on a per-backend basis (currently enabled only for the `DataConnector` backend.
- Within `RawColumnInfo` the column type is now represented by a new type `RawColumnType b` which mirrors the shape of the DC API `ColumnType`, but uses `XNestedObjects b` and `XNestedArrays b` type families to allow turning nested object and array supports on or off for a particular backend. In the `DataConnector` backend `API.CustomType` is converted into `RawColumnInfo 'DataConnector` while building the schema.
- In the next stage of schema building, the `RawColumnInfo` is converted into a `StructuredColumnInfo` which allows us to represent the three different types of columns: scalar, object and array. TODO: the `StructuredColumnInfo` looks very similar to the Logical Model types. The main difference is that it uses the `XNestedObjects` and `XNestedArrays` type families. We should be able to combine these two representations.
- The `StructuredColumnInfo` is then placed into a `FIColumn` `FieldInfo`. This involved some refactoring of `FieldInfo` as I had previously split out `FINestedObject` into a separate constructor. However it works out better to represent all "column" fields (i.e. scalar, object and array) using `FIColumn` as this make it easier to implement permission checking correctly. This is the reason the `StructuredColumnInfo` was needed.
- Next, the `FieldInfo` are used to generate `FieldParser`s. We add a new constructor to `AnnFieldG` for `AFNestedArray`. An `AFNestedArray` field parser can contain either a simple array selection or an array aggregate. Simple array `FieldParsers` are currently limited to subfield selection. We will add support for limit, offset, where and order_by in a future PR. We also don't yet generate array aggregate `FieldParsers.
- The new `AFNestedArray` field is handled by the `QueryPlan` module in the `DataConnector` backend. There we generate an `API.NestedArrayField` from the AFNestedArray. We also handle nested arrays when reshaping the response from the DC agent.
## Limitations
- Support for limit, offset, filter (where) and order_by is not yet fully implemented, although it should not be hard to add this
- Support for aggregations on nested arrays is not yet fully implemented
- Permissions involving nested arrays (and objects) not yet implemented
- This should be integrated with Logical Model types, but that will happen in a separate PR
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9149
GitOrigin-RevId: 0e7b71a994fc1d2ca1ef73bfe7b96e95b5328531
2023-05-24 11:00:59 +03:00
|
|
|
logicalModelToColumnInfo :: forall b. (Backend b) => Int -> (Column b, NullableScalarType b) -> Maybe (StructuredColumnInfo b)
|
2023-04-25 14:58:15 +03:00
|
|
|
logicalModelToColumnInfo i (column, NullableScalarType {..}) = do
|
|
|
|
name <- G.mkName (toTxt column)
|
2023-05-24 16:51:56 +03:00
|
|
|
pure
|
|
|
|
$
|
Nested array support for Data Connectors Backend and MongoDB
## Description
This change adds support for querying into nested arrays in Data Connector agents that support such a concept (currently MongoDB).
### DC API changes
- New API type `ColumnType` which allows representing the type of a "column" as either a scalar type, an object reference or an array of `ColumnType`s. This recursive definition allows arbitrary nesting of arrays of types.
- The `type` fields in the API types `ColumnInfo` and `ColumnInsertSchema` now take a `ColumnType` instead of a `ScalarType`.
- To ensure backwards compatibility, a `ColumnType` representing a scalar serialises and deserialises to the same representation as `ScalarType`.
- In queries, the `Field` type now has a new constructor `NestedArrayField`. This contains a nested `Field` along with optional `limit`, `offset`, `where` and `order_by` arguments. (These optional arguments are not yet used by either HGE or the MongoDB agent.)
### MongoDB Haskell agent changes
- The `/schema` endpoint will now recognise arrays within the JSON validation schema and generate corresponding arrays in the DC schema.
- The `/query` endpoint will now handle `NestedArrayField`s within queries (although it does not yet handle `limit`, `offset`, `where` and `order_by`).
### HGE server changes
- The `Backend` type class adds a new type family `XNestedArrays b` to enable nested arrays on a per-backend basis (currently enabled only for the `DataConnector` backend.
- Within `RawColumnInfo` the column type is now represented by a new type `RawColumnType b` which mirrors the shape of the DC API `ColumnType`, but uses `XNestedObjects b` and `XNestedArrays b` type families to allow turning nested object and array supports on or off for a particular backend. In the `DataConnector` backend `API.CustomType` is converted into `RawColumnInfo 'DataConnector` while building the schema.
- In the next stage of schema building, the `RawColumnInfo` is converted into a `StructuredColumnInfo` which allows us to represent the three different types of columns: scalar, object and array. TODO: the `StructuredColumnInfo` looks very similar to the Logical Model types. The main difference is that it uses the `XNestedObjects` and `XNestedArrays` type families. We should be able to combine these two representations.
- The `StructuredColumnInfo` is then placed into a `FIColumn` `FieldInfo`. This involved some refactoring of `FieldInfo` as I had previously split out `FINestedObject` into a separate constructor. However it works out better to represent all "column" fields (i.e. scalar, object and array) using `FIColumn` as this make it easier to implement permission checking correctly. This is the reason the `StructuredColumnInfo` was needed.
- Next, the `FieldInfo` are used to generate `FieldParser`s. We add a new constructor to `AnnFieldG` for `AFNestedArray`. An `AFNestedArray` field parser can contain either a simple array selection or an array aggregate. Simple array `FieldParsers` are currently limited to subfield selection. We will add support for limit, offset, where and order_by in a future PR. We also don't yet generate array aggregate `FieldParsers.
- The new `AFNestedArray` field is handled by the `QueryPlan` module in the `DataConnector` backend. There we generate an `API.NestedArrayField` from the AFNestedArray. We also handle nested arrays when reshaping the response from the DC agent.
## Limitations
- Support for limit, offset, filter (where) and order_by is not yet fully implemented, although it should not be hard to add this
- Support for aggregations on nested arrays is not yet fully implemented
- Permissions involving nested arrays (and objects) not yet implemented
- This should be integrated with Logical Model types, but that will happen in a separate PR
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9149
GitOrigin-RevId: 0e7b71a994fc1d2ca1ef73bfe7b96e95b5328531
2023-05-24 11:00:59 +03:00
|
|
|
-- TODO(dmoverton): handle object and array columns
|
2023-05-24 16:51:56 +03:00
|
|
|
SCIScalarColumn
|
|
|
|
$ ColumnInfo
|
|
|
|
{ ciColumn = column,
|
|
|
|
ciName = name,
|
|
|
|
ciPosition = i,
|
|
|
|
ciType = ColumnScalar nstType,
|
|
|
|
ciIsNullable = nstNullable,
|
|
|
|
ciDescription = G.Description <$> nstDescription,
|
|
|
|
ciMutability = ColumnMutability {_cmIsInsertable = False, _cmIsUpdatable = False}
|
|
|
|
}
|
2023-04-25 14:58:15 +03:00
|
|
|
|
|
|
|
logicalModelFieldsToFieldInfo ::
|
|
|
|
forall b.
|
|
|
|
(Backend b) =>
|
2023-04-27 10:41:55 +03:00
|
|
|
InsOrdHashMap.InsOrdHashMap (Column b) (LogicalModelField b) ->
|
2023-04-25 14:58:15 +03:00
|
|
|
FieldInfoMap (FieldInfo b)
|
|
|
|
logicalModelFieldsToFieldInfo =
|
2023-04-26 18:42:13 +03:00
|
|
|
HashMap.fromList
|
2023-04-25 14:58:15 +03:00
|
|
|
. fmap (bimap (fromCol @b) FIColumn)
|
|
|
|
. fromMaybe mempty
|
|
|
|
. traverseWithIndex
|
|
|
|
(\i (column, lmf) -> (,) column <$> logicalModelToColumnInfo i (column, lmf))
|
2023-04-27 10:41:55 +03:00
|
|
|
. InsOrdHashMap.toList
|
2023-04-25 14:58:15 +03:00
|
|
|
. columnsFromFields
|