Commit Graph

51 Commits

Author SHA1 Message Date
Tom Harding
4885a3fd9a Import J, not Aeson, A, JSON, Yaml...
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8948
GitOrigin-RevId: d70c4a50b94ffe7d42a1fb1017051d351f236acc
2023-04-26 17:30:24 +00:00
Daniel Chambers
91c88b2109 Properly return schema information about views and PK mutability from the Super Connector
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8786
GitOrigin-RevId: b7cd0dd5736a408a269819ea76dfbe53455060ee
2023-04-18 17:24:51 +00:00
David Overton
346804fc67 Support nested object fields in DC API and use this to implement nest…
## Description

This change adds support for nested object fields in HGE IR and Schema Cache, the Data Connectors backend and API, and the MongoDB agent.

### Data Connector API changes

- The `/schema` endpoint response now includes an optional set of GraphQL type definitions. Table column types can refer to these definitions by name.
- Queries can now include a new field type `object` which contains a column name and a nested query. This allows querying into a nested object within a field.

### MongoDB agent changes

- Add support for querying into nested documents using the new `object` field type.

### HGE changes

- The `Backend` type class has a new type family `XNestedObjects b` which controls whether or not a backend supports querying into nested objects. This is currently enabled only for the `DataConnector` backend.
- For backends that support nested objects, the `FieldInfo` type gets a new constructor `FINestedObject`, and the `AnnFieldG` type gets a new constructor `AFNestedObject`.
- If the DC `/schema` endpoint returns any custom GraphQL type definitions they are stored in the `TableInfo` for each table in the source.
- During schema cache building, the function `addNonColumnFields` will check whether any column types match custom GraphQL object types stored in the `TableInfo`. If so, they are converted into `FINestedObject` instead of `FIColumn` in the `FieldInfoMap`.
- When building the `FieldParser`s from `FieldInfo` (function `fieldSelection`) any `FINestedObject` fields are converted into nested object parsers returning `AFNestedObject`.
- The `DataConnector` query planner converts `AFNestedObject` fields into `object` field types in the query sent to the agent.

## Limitations

### HGE not yet implemented:
- Support for nested arrays
- Support for nested objects/arrays in mutations
- Support for nested objects/arrays in order-by
- Support for filters (`where`) in nested objects/arrays
- Support for adding custom GraphQL types via track table metadata API
- Support for interface and union types
- Tests for nested objects

### Mongo agent not yet implemented:

- Generate nested object types from validation schema
- Support for aggregates
- Support for order-by
- Configure agent port
- Build agent in CI
- Agent tests for nested objects and MongoDB agent

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7844
GitOrigin-RevId: aec9ec1e4216293286a68f9b1af6f3f5317db423
2023-04-11 01:30:37 +00:00
Rishichandra Wawhal
c6d65508b2 [feature branch] EE Lite Trials
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8208
Co-authored-by: awjchen <13142944+awjchen@users.noreply.github.com>
Co-authored-by: Vijay Prasanna <11921040+vijayprasanna13@users.noreply.github.com>
Co-authored-by: Toan Nguyen  <1615675+hgiasac@users.noreply.github.com>
Co-authored-by: Abhijeet Khangarot <26903230+abhi40308@users.noreply.github.com>
Co-authored-by: Solomon <24038+solomon-b@users.noreply.github.com>
Co-authored-by: gneeri <10553562+gneeri@users.noreply.github.com>
GitOrigin-RevId: 454ee0dea636da77e43810edb2f427137027956c
2023-04-05 08:59:09 +00:00
Daniel Chambers
fde4c0fae5 Enhance insert table schema with extra info and remove need for cached schema usage in Super Connector mutations
[GDC-643]: https://hasurahq.atlassian.net/browse/GDC-643?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8560
GitOrigin-RevId: 0e965da447eb6a5acf5a0f291f9e205882630e6e
2023-04-05 02:23:20 +00:00
Daniel Chambers
8acd447803 Fix non-determinism in expected ordering in agent test suite test
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8493
GitOrigin-RevId: 210b6beb991d806e9a3e4cc901ee922ede0e3c84
2023-03-27 02:22:51 +00:00
Brandon Martin
63eabc2374 Add dataset support to Oracle in the Super Connector
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8469
Co-authored-by: Daniel Chambers <1214352+daniel-chambers@users.noreply.github.com>
GitOrigin-RevId: 6f2aebc150d7c93490e6da3ada5c282dd3d9a56c
2023-03-24 14:35:30 +00:00
Daniel Chambers
e9c697aaa9 Fix "limit" from permissions being incorrectly applied to aggregates in Data Connectors
[GDC-1064]: https://hasurahq.atlassian.net/browse/GDC-1064?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8345
GitOrigin-RevId: 9ef91944ba6408e1030a47de58f3271145433ee2
2023-03-20 04:02:23 +00:00
Daniel Chambers
0b63b0b6b1 Fixed queries with no column fields and only introspection fields in Data Connectors
[GDC-1063]: https://hasurahq.atlassian.net/browse/GDC-1063?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8249
GitOrigin-RevId: 9d6dec3b4ef38fd7972bd0b9d3cfdfece1dcdcb5
2023-03-15 05:16:32 +00:00
Daniel Chambers
dc83352863 Support for using Data Connectors as the target of remote relationships
[GDC-487]: https://hasurahq.atlassian.net/browse/GDC-487?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[GDC-488]: https://hasurahq.atlassian.net/browse/GDC-488?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8182
GitOrigin-RevId: 712953666e1a5ea07500f1e1aed669f27650f7a4
2023-03-07 01:33:26 +00:00
Daniel Chambers
81ffaafe6f Fixed unstable ordering in Data Connector agent test
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8143
GitOrigin-RevId: 4926e95f327123e8925340fed35c24687bd8189e
2023-03-01 01:30:44 +00:00
Daniel Chambers
6b19069c37 Only load the schema once for all schema tests in the Data Connector agent test suite
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8142
GitOrigin-RevId: 1e2340fa5670a9c11e424e85045dc18f5286cff7
2023-02-28 13:52:30 +00:00
Daniel Chambers
6f50261274 Additional tests for mutations in the Data Connector agent test suite
[GDC-719]: https://hasurahq.atlassian.net/browse/GDC-719?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8080
GitOrigin-RevId: 2e35dd72723e0e9fc63426dc90b1a89aedbe4774
2023-02-23 13:54:37 +00:00
Daniel Chambers
190e429eef Add tests for delete mutations to the Data Connector agent test suite
[GDC-721]: https://hasurahq.atlassian.net/browse/GDC-721?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8067
GitOrigin-RevId: bf5b7e7c20c6011d438888be6017331def7063bf
2023-02-22 23:41:47 +00:00
Daniel Chambers
72e319e486 Add tests for update mutations to the Data Connector agent test suite
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8054
GitOrigin-RevId: 6869e8938e7a70128770f959bf34e3a291074d14
2023-02-22 04:19:16 +00:00
Daniel Chambers
ac475fa02e Added weird table edge-case insert mutation tests to agent test suite
[GDC-719]: https://hasurahq.atlassian.net/browse/GDC-719?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7964
GitOrigin-RevId: d3077aafd29f3b5553e00bf6bba4c1ff9966485d
2023-02-16 07:51:36 +00:00
Lyndon Maydwell
630bf277c1 SQLite Mutations
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7573
Co-authored-by: Daniel Chambers <1214352+daniel-chambers@users.noreply.github.com>
GitOrigin-RevId: 7807368faaa4bac5c0726c2dab041f8180a3fc31
2023-02-14 11:20:36 +00:00
Daniel Chambers
2ced1f2676 More tests for insert mutations for Data Connector agents
[GDC-719]: https://hasurahq.atlassian.net/browse/GDC-719?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7901
GitOrigin-RevId: b7f28c4b9604c0ca78c528deb2923e8519fae56a
2023-02-13 01:03:18 +00:00
Daniel Chambers
0d5badba09 Initial tests for insert mutations for Data Connector agents
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7875
GitOrigin-RevId: 793501685124e41edfd83c6be4a2a9c7ae7f9267
2023-02-08 08:36:58 +00:00
David Overton
98af923df5 MongoDB Data Connector prototype agent
Basic MongoDB agent. This is intended as a starting point for playing with nested documents in a MongoDB back end. Currently supports basic queries with projections, where expressions, limit and offset. No support for joins, aggregates or mutations.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7840
GitOrigin-RevId: 3f03b8416c95acf2b68da1db56cbe36a513a4bde
2023-02-07 02:57:06 +00:00
Daniel Chambers
2e8fc69394 Adds an agent test for ordering by a single column aggregate function
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7820
GitOrigin-RevId: 8455c2ba032173ade474f2744b43d1259f88c9b2
2023-02-06 23:45:56 +00:00
Daniel Chambers
7a4bde9652 Added result_type property to SingleColumnAggregate
[GDC-756]: https://hasurahq.atlassian.net/browse/GDC-756?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7793
GitOrigin-RevId: b8201615984a71a9f41bed4195e2435e362cf383
2023-02-06 04:20:17 +00:00
Daniel Chambers
67bc017cf3 Gardening: Enable and fix warnings in dc-api package
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7765
GitOrigin-RevId: 7dd6b1c065cc68ab413736ca49e5957a7697566b
2023-02-02 05:48:25 +00:00
Daniel Chambers
b012f2ebc7 Use Dataset Clones for all SQLite tests
[GDC-718]: https://hasurahq.atlassian.net/browse/GDC-718?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7743
GitOrigin-RevId: 6c3577c1d4ffd2212a72b6e1a24e0e384f2db046
2023-02-02 04:27:57 +00:00
Daniel Chambers
0abfc97df7 Implemented datasets support in Data Connector agent test suite
[GDC-718]: https://hasurahq.atlassian.net/browse/GDC-718?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7654
GitOrigin-RevId: 1d036cd39f717fce1ce0176103e40a1309ded3af
2023-01-30 07:00:26 +00:00
Daniel Chambers
354f7593d9 Custom update column operator support for Data Connectors [GDC-688]
[GDC-688]: https://hasurahq.atlassian.net/browse/GDC-688?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7548
GitOrigin-RevId: 861638d6cc69803776640b50ffe1646b3cf0a7db
2023-01-19 04:22:58 +00:00
Daniel Chambers
c104ffdb51 Fix SQLite agent including built-in comparison operators as custom operators
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7565
GitOrigin-RevId: 83a8868a80a124aee05b8dd42a4ffa15489891ae
2023-01-18 07:36:00 +00:00
David Overton
a9f77acb32 Remove builtin scalar types from Data Connector backend
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7167
Co-authored-by: Daniel Chambers <1214352+daniel-chambers@users.noreply.github.com>
GitOrigin-RevId: 926e7282b908e3a9669ac39d625aa54971e11c37
2023-01-11 02:37:26 +00:00
Lyndon Maydwell
e9dcbf6491 Adding custom scalar types to SQLite DC Agent - GDC-610
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6783
Co-authored-by: David Overton <7734777+dmoverton@users.noreply.github.com>
Co-authored-by: Daniel Chambers <1214352+daniel-chambers@users.noreply.github.com>
GitOrigin-RevId: ee53c20b0090b6b3c88792ddc98b8287872fe0f3
2022-12-08 06:50:08 +00:00
Daniel Chambers
c14fd3ba4c Add mutability properties to the Data Connector schema API [GDC-664]
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7190
GitOrigin-RevId: ce602b5e5cc5aee8716ff3f7a036b18b3bf47188
2022-12-08 02:07:01 +00:00
Daniel Chambers
354ab2f7aa Use Sandwich test framework for Data Connector agent test suite
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7133
GitOrigin-RevId: 625dde70d1c594dcdfd377efac78710174a74efc
2022-12-05 02:16:01 +00:00
Daniel Chambers
7e9f5bdd96 Fixed filtering not being applied across object relations in SQLite agent
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6937
GitOrigin-RevId: 655e9be98059a1e86566b975ae646b49d67fa06e
2022-11-18 08:15:50 +00:00
Brandon Martin
da611aaa58 [GDC] Add explicit ordering to distinct, paginated, and filtered test
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6932
GitOrigin-RevId: 42cd06ac91935c2a86820df9125681018d895770
2022-11-18 01:00:23 +00:00
Daniel Chambers
bf11489491 Fix ambiguous ordering in Data Connector agent test
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6915
GitOrigin-RevId: f75bd88e84a7e71cf4ced459349644c22f7240de
2022-11-17 02:27:12 +00:00
Brandon Martin
1e6cf5f133 super-connector: More coalesce defaults for rows/aggregates
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6861
Co-authored-by: Daniel Chambers <1214352+daniel-chambers@users.noreply.github.com>
GitOrigin-RevId: 3a3efd89196e619ae7491d3fc95c19f8c5156f2e
2022-11-15 20:53:55 +00:00
Brandon Martin
22e6d7afcd server: Add Uppercase option to tests-dc-api
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6863
GitOrigin-RevId: 6e4950786cebedcbfcff26fed691a6b6bf0c1aa3
2022-11-14 23:08:26 +00:00
Daniel Chambers
431311e74e Add support for ordering by related tables and aggregates to the SQLite agent [GDC-313]
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6773
GitOrigin-RevId: 9cca50511aa9e8ef7c4e2a20f280be0fa51aab85
2022-11-09 06:06:58 +00:00
Daniel Chambers
381c81c806 Added new agent test to check ordering in deeply nested array relationships [GDC-605]
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6721
GitOrigin-RevId: e0478bef0a8f5fffba9ef032be323cadc0f85116
2022-11-04 12:27:35 +00:00
Daniel Chambers
723c91bef4 Fixed unstable ordering issues in data connector agent test suite
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6687
GitOrigin-RevId: 278a6073923981683cb75cfb922f284e0960c324
2022-11-04 04:36:47 +00:00
Samir Talwar
342391f39d Upgrade Ormolu to v0.5.
This upgrades the version of Ormolu required by the HGE repository to v0.5.0.1, and reformats all code accordingly.

Ormolu v0.5 reformats code that uses infix operators. This is mostly useful, adding newlines and indentation to make it clear which operators are applied first, but in some cases, it's unpleasant. To make this easier on the eyes, I had to do the following:

* Add a few fixity declarations (search for `infix`)
* Add parentheses to make precedence clear, allowing Ormolu to keep everything on one line
* Rename `relevantEq` to `(==~)` in #6651 and set it to `infix 4`
* Add a few _.ormolu_ files (thanks to @hallettj for helping me get started), mostly for Autodocodec operators that don't have explicit fixity declarations

In general, I think these changes are quite reasonable. They mostly affect indentation.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6675
GitOrigin-RevId: cd47d87f1d089fb0bc9dcbbe7798dbceedcd7d83
2022-11-02 20:55:13 +00:00
Lyndon Maydwell
4b968b17e0 ConfigSpec Data Connector Agent Test
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6616
GitOrigin-RevId: dd5582cd6b61b7412d931035c043daa5fbb55c71
2022-11-01 21:38:12 +00:00
David Overton
9921823915 GDC-189 custom aggregations
>

## Description
->

This PR allows DC agents to define custom aggregate functions for their scalar types.

### Related Issues
->

GDC-189

### Solution and Design
>

We added a new property `aggregate_functions` to the scalar types capabilities. This allows the agent author to specify a set of aggregate functions supported by each scalar type, along with the function's result type.

During GraphQL schema generation, the custom aggregate functions are available via a new method `getCustomAggregateOperators` on the `Backend` type class.
Custom functions are merged with the builtin aggregate functions when building GraphQL schemas for table aggregate fields and for `order_by` operators on array relations.

### Steps to test and verify
>

• Codec tests for aggregate function capabilities have been added to the unit tests.
• Some custom aggregate operators have been added to the reference agent and are used in a new test in `api-tests`.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6199
GitOrigin-RevId: e9c0d1617af93847c1493671fdbb794f573bde0c
2022-10-27 00:44:06 +00:00
Daniel Chambers
792ef3117d Export Chinook dataset to JSON lines for Athena consumption
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6489
GitOrigin-RevId: 36a068179d595f3d838995fb057b8ee590d1fd02
2022-10-24 02:10:51 +00:00
Daniel Chambers
84b84a78e1 Add column scalar type information Data Connector query API request [GDC-493]
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6433
GitOrigin-RevId: c5f5e0c2e25c6820d9f73a1e90699cf18dc4cd47
2022-10-20 03:24:53 +00:00
Lyndon Maydwell
d54bb30d3b Structured Error Protocol for Data Connectors Agents - GDW-137
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6061
Co-authored-by: Vishnu Bharathi <4211715+scriptnull@users.noreply.github.com>
GitOrigin-RevId: 855d96378030f4e01b0c74b00e20e592e51e7a49
2022-10-11 00:26:24 +00:00
Daniel Chambers
8369cac3bd Data Connector agent data schema capabilities [GDC-479]
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6268
GitOrigin-RevId: 4ec29566d3c2ab2144dad8055b4442a4027915ec
2022-10-10 06:59:30 +00:00
Daniel Chambers
c862c64b33 Break down Data Connector agent schema tests into finer grained, more accurate, tests [GDC-479]
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6225
GitOrigin-RevId: 8eaadcc14cf4bc90a22450e924daf8cc340ed2ea
2022-10-10 02:25:09 +00:00
Daniel Chambers
d713cad315 Add support for lowercase table and column names in the Data Connector agent tests [GDC-463]
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6179
GitOrigin-RevId: 6455e5809ea06b86dfbcddf781ee105088ca78f9
2022-10-05 23:25:00 +00:00
Auke Booij
05b3a64e8f Clean up Hasura.Prelude a bit
- Remove `onJust` in favor of the more general `for_`
- Remove `withJust` which was used only once
- Remove `hashNub` in favor of `Ord`-based `uniques`
- Simplify some of the implementations in `Hasura.Prelude`
- Add `hlint` hint from `maybe True` to `all`, and `maybe False` to `any`

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6173
GitOrigin-RevId: 2c6ebbe2d04f60071d2a53a2d43c6d62dbc4b84e
2022-10-03 21:50:53 +00:00
Daniel Chambers
6af3fa2d78 Add the ability to test against tables in configurable schemas in Data Connector Agent test suite
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6083
GitOrigin-RevId: 5a7ee37ed3a62224a062dbaac7d322dfd95b83c4
2022-09-27 23:42:32 +00:00