Commit Graph

488 Commits

Author SHA1 Message Date
Daniel Harvey
8e6ec8b60d feat(server): native Postgres array support
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9673
GitOrigin-RevId: 2dfbc07acf9da879910acfc9885bb7138b81e883
2023-06-28 08:47:59 +00:00
kodiakhq[bot]
d4e368324d chore(tooling): import ekg-prometheus into the monorepo
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9526
Co-authored-by: awjchen <13142944+awjchen@users.noreply.github.com>
Co-authored-by: paritosh-08 <85472423+paritosh-08@users.noreply.github.com>
GitOrigin-RevId: 131739ab8e68165453fd47d1eafcc7957ec6f411
2023-06-27 18:37:11 +00:00
Samir Talwar
699ba62def server: Rename "error-message" to "hasura-error-message".
There is already a library in Hackage called "error-message".

This breaks the Nix build as we end up overriding that library, which causes a change to any of its dependents.

Outside Nix, Shadowing isn't strictly a problem, but it's prudent to avoid it in case we accidentally end up with the other library in edge cases.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9679
GitOrigin-RevId: e3f54fd4e5eafec9b0a00128767daad0c8aeab72
2023-06-27 16:51:06 +00:00
Tom Harding
87abdf17a1 Allow __typename in BigQuery aggregate queries
[NDAT-532]: https://hasurahq.atlassian.net/browse/NDAT-532?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9677
GitOrigin-RevId: aae8a57461588c8ef8ace943ac7077445c05c4a5
2023-06-27 10:29:13 +00:00
David Overton
37560dde8f Dmoverton/schemaless capability
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9667
GitOrigin-RevId: 41b3928e976703633437ef14624ec00fb0a8498d
2023-06-27 05:49:03 +00:00
Philip Lykke Carlsen
d95435ad16 fix: Restore function of optimize_permission_filters
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9629
GitOrigin-RevId: 10584252fd97889c68813a21ac817238b0847a40
2023-06-26 15:43:00 +00:00
Naveen Naidu
4796a9dd69 server/postgres: insert input validation [experimental]
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9620
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
GitOrigin-RevId: 345c3f763f8dd3397c999c5967af39192b944640
2023-06-25 13:48:10 +00:00
Philip Lykke Carlsen
dcbe839f3a test: test-harness improvements
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9656
GitOrigin-RevId: 5c300851668050f6964076ceb385fef56bd9aca7
2023-06-23 19:37:51 +00:00
Puru Gupta
f2fe9cfe3b server: add support for header resolution from env vars
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9509
GitOrigin-RevId: 818f747422c5444fcb55419729ad58d74b890d52
2023-06-23 08:39:28 +00:00
David Overton
3e83370c96 Fix parsing of scalar types in the DataConnector backend
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9596
GitOrigin-RevId: 0161814a6678b67187147fcfa18d20523e480a9d
2023-06-21 06:50:51 +00:00
pranshi06
3124c93673 server: configurable header length limit for warp
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9363
GitOrigin-RevId: 8dd1e6a5f51d86ed20123c7ed7d42906fe5251bb
2023-06-19 12:34:35 +00:00
Daniel Harvey
98fdeda54e fix(server): skip insert when no values provided for SQLServer
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9564
GitOrigin-RevId: 7d95ee428b8da7e246d9f1040f04d8abe359b667
2023-06-16 08:48:48 +00:00
Gil Mizrahi
d1c1a992f5 include the response in the test-harness missing content-type assertion
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9562
GitOrigin-RevId: 2c30c85c2a7d97d2ae604f8f133f1beefbc2cacc
2023-06-15 13:42:04 +00:00
David Overton
dd34fb85ab Be more lenient about missing values in data connector response
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9532
GitOrigin-RevId: f237a237fe5453338b71ead98fca38bb037676b0
2023-06-15 01:33:26 +00:00
Daniel Harvey
32fd3b4be0 chore(api-tests): enable Jaeger tracing in all Native Database test setups
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9549
GitOrigin-RevId: c3a583bec1d02e23bd340b646ab6c7c1a72f931b
2023-06-14 17:19:06 +00:00
Philip Lykke Carlsen
5d76f9cabf Feat: No feature flags for Native Queries and Stored Procedures
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9541
GitOrigin-RevId: 81a33de3a35405c6e45cb5bad160ad00871745eb
2023-06-14 13:21:37 +00:00
Daniel Chambers
331e4e3d5d Stop swallowing data connector connection errors during schema cache building
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9533
GitOrigin-RevId: ce81cd4b7449db4f9f62caa98c888641aa27b3c7
2023-06-14 09:45:29 +00:00
Samir Talwar
e16e70760a server/upgrade-tests: Avoid Docker host networking.
Instead, we update the metadata to point to the correct database URL each time we start HGE.

This means we can run the tests on macOS (and any other environment where Docker runs inside a VM).

I was hoping this would solve the issues we're seeing on CI too, but no such luck.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9528
GitOrigin-RevId: 32cbf65430462b4b0e59f5d644260f7c26c7135c
2023-06-13 22:30:26 +00:00
Daniel Harvey
d49c476176 chore(postgres-agent): implement dataset API
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9326
GitOrigin-RevId: 55f3e63dfcbc79474800b4a3dac00e0e9c82871f
2023-06-13 18:10:39 +00:00
pranshi06
45f2d8f52d server: forward null value to the remote schema when field type is nullable
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9460
Co-authored-by: Auke Booij <164426+abooij@users.noreply.github.com>
GitOrigin-RevId: 9449a679d66817d7bf9f401009a638b57cb58695
2023-06-13 15:04:40 +00:00
hasura-bot
0a418040c3 feat(server): optional args GQL argument
GITHUB_PR_NUMBER: 9497
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/9497

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8340
Co-authored-by: Wsevolod <31807671+kraftwerk28@users.noreply.github.com>
Co-authored-by: Daniel Harvey <4729125+danieljharvey@users.noreply.github.com>
GitOrigin-RevId: e436b171d88991b814cc6c7b3ed47df75795ac64
2023-06-13 14:16:40 +00:00
Gil Mizrahi
9458893bc1 test stored procedures dont work in oss
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9517
GitOrigin-RevId: a33f9a0f0ad2ae82715269256fe997ad98bb1b72
2023-06-13 10:42:33 +00:00
paritosh-08
14b4de37f7 server: close subscriptions (with 1012) on metadata change
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9159
GitOrigin-RevId: 5ab25ef57782c126de8852f3546809feb66a9d44
2023-06-13 09:24:19 +00:00
Tom Harding
d864fff13f Move query validation into the schema cache building code
[NDAT-707]: https://hasurahq.atlassian.net/browse/NDAT-707?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9510
Co-authored-by: Gil Mizrahi <8547573+soupi@users.noreply.github.com>
GitOrigin-RevId: 11474498c75bfe1eb1173fed1073cc0206d290a1
2023-06-12 14:57:41 +00:00
Daniel Harvey
be252cddf7 chore(ci-info): fix various compiler warnings
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9461
GitOrigin-RevId: 2160bd10c5464fad34bb5b9057beaa718c58a2ae
2023-06-08 12:25:22 +00:00
kodiakhq[bot]
6f3359ae96 chore(tooling): import ekg-json into the monorepo
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9424
Co-authored-by: Johan Tibell <17277+tibbe@users.noreply.github.com>
Co-authored-by: Oleg Grenrus <51087+phadej@users.noreply.github.com>
Co-authored-by: Mikhail Glushenkov <47439+23Skidoo@users.noreply.github.com>
Co-authored-by: Eric Conlon <37287+ejconlon@users.noreply.github.com>
Co-authored-by: Brandon Simmons <210815+jberryman@users.noreply.github.com>
Co-authored-by: awjchen <13142944+awjchen@users.noreply.github.com>
Co-authored-by: Tom Harding <6302310+i-am-tom@users.noreply.github.com>
GitOrigin-RevId: f493865321da664cef45e5f56f33bb00ae03c713
2023-06-07 08:38:48 +00:00
Daniel Harvey
0c38a693ac chore(ci): Use hlint github action
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9434
GitOrigin-RevId: 2205da4c6384b89d31f830a5119514154019566d
2023-06-06 16:00:22 +00:00
Brandon Simmons
440a8664de server: Upgrade the HGE server to GHC 9.4.5
Upgrade to GHC 9.4.5, and update any tests.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8954
Co-authored-by: Mohd Bilal <24944223+m-Bilal@users.noreply.github.com>
Co-authored-by: Samir Talwar <47582+SamirTalwar@users.noreply.github.com>
Co-authored-by: Philip Lykke Carlsen <358550+plcplc@users.noreply.github.com>
GitOrigin-RevId: 5261126777cb478567ea471c4bf5441bc345ea0d
2023-06-06 13:29:30 +00:00
Daniel Harvey
0aeb48992e chore(testing): index all queries for Native Query tests by backend type
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9416
GitOrigin-RevId: 7b10bd1f8e2cbb98ec86646d9f54b18ad42bf85c
2023-06-05 13:06:03 +00:00
Gil Mizrahi
d8a4d254f6 chore(tests): add a flag to trace the graphql commands sent to the server during a test.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9409
GitOrigin-RevId: 46d0c48b519bf32ddb9869a8a7af81aad302532b
2023-06-05 08:22:53 +00:00
Lyndon Maydwell
6487a44546 Adding function permissions to UDFs for Dataconnectors
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9349
GitOrigin-RevId: 0b71d06a13ad78a55a205a5755260b060525d340
2023-06-05 05:29:53 +00:00
Daniel Chambers
e2345fb3bb Fixes to HTTP request recording in the Data Connector agent test suite
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9394
GitOrigin-RevId: 873dc4e8ec744013f8303d197a91ec499e6a9c66
2023-06-02 14:34:49 +00:00
Tom Harding
b124c943a9 chore(server): allow recursive native queries
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9373
Co-authored-by: Gil Mizrahi <8547573+soupi@users.noreply.github.com>
Co-authored-by: Daniel Harvey <4729125+danieljharvey@users.noreply.github.com>
GitOrigin-RevId: b998da0e2725f687a00c2c6c993435e44df4f70f
2023-06-02 10:37:55 +00:00
Samir Talwar
dd46aa6715 server: Preserve ordering when possible, and sort when it's not.
When upgrading to GHC v9.4, we noticed a number of failures because the sort order of HashMaps has changed. With this changeset, I am endeavoring to mitigate this now and in the future.

This makes one of two changes in a few areas where we depend on the sort order of elements in a `HashMap`:

  1. the ordering of the request is preserved with `InsOrdHashMap`, or
  2. we sort the data after retrieving it.

Fortunately, we do not do this anywhere where we _must_ preserve order; it's "just" descriptions, error messages, and OpenAPI metadata. The main problem is that tests are likely to fail each time we upgrade GHC (or whatever is providing the hash seed).

[NDAT-705]: https://hasurahq.atlassian.net/browse/NDAT-705?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9390
GitOrigin-RevId: 84503e029b44094edbbc298651744bc2843c15f3
2023-06-02 09:31:26 +00:00
Daniel Chambers
d421cdb726 Put the message property first when encoding error JSON
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9379
GitOrigin-RevId: 385035810190786f54a1db86e3a2e4a2c3bd5dee
2023-06-02 05:29:50 +00:00
Daniel Harvey
0034ae8892 chore(server): use new version of sandwich
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9386
Co-authored-by: Daniel Chambers <1214352+daniel-chambers@users.noreply.github.com>
GitOrigin-RevId: 008312c7550025cb44d515fa3d01796175b60e29
2023-06-02 02:00:41 +00:00
kodiakhq[bot]
34e172d576 chore(tooling): import ci-info-hs into monorepo
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9383
Co-authored-by: Shahidh K Muhammed <4124733+shahidhk@users.noreply.github.com>
Co-authored-by: Ajeet D'Souza <1777663+ajeetdsouza@users.noreply.github.com>
Co-authored-by: Brandon Simmons <210815+jberryman@users.noreply.github.com>
Co-authored-by: Vamshi Surabhi <6562944+0x777@users.noreply.github.com>
Co-authored-by: Auke Booij <164426+abooij@users.noreply.github.com>
Co-authored-by: Matthew Pickering <1216657+mpickering@users.noreply.github.com>
Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
GitOrigin-RevId: a7a210e2237f5adc63302b18403616c7c0aa39ca
2023-06-01 12:51:39 +00:00
Gil Mizrahi
b87506e87c chore(tests): output hge stderr in api-tests
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9384
GitOrigin-RevId: 5d78b59e12bf870a3cba81829dcf4d3e0d2aebf9
2023-06-01 11:43:38 +00:00
Gil Mizrahi
3ce1b414b4 NativeQueries: validate that there are no unused or undeclare arguments
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9357
GitOrigin-RevId: 9d0ac74c8a2780c6d1b3e7769c3d4775bacb2d8d
2023-06-01 09:15:48 +00:00
Philip Lykke Carlsen
df10ccf310 fix(pg): Support aliased fields for Native Query object relations
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9374
GitOrigin-RevId: 073e51480e2be97e32938fa61a3547374e04ead8
2023-05-31 15:42:50 +00:00
Daniel Harvey
7d011644ac chore(server): allow non-nullable NQ -> NQ object relationships
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9353
GitOrigin-RevId: 9da95675d224f3045509592c2515b7afa2500d00
2023-05-31 10:34:28 +00:00
Daniel Harvey
9de3db4729 chore(server): respect logical model array nullability
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9303
GitOrigin-RevId: 4d38a125a6889e2b0be34905e21a78036a77881e
2023-05-31 00:15:47 +00:00
Daniel Harvey
39396c50b8 feature(server): comparable computed field aggregations for Postgres
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9300
GitOrigin-RevId: 92e4afaf2c029304e9ddb1d9541aa78c2b9b3027
2023-05-25 12:55:29 +00:00
Tom Harding
3492484034 Implement get_table_info for Postgres, Citus, and Cockroach
[NDAT-621]: https://hasurahq.atlassian.net/browse/NDAT-621?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9201
GitOrigin-RevId: 91886f5f8d125d49c35647cfbfc3c6621d932761
2023-05-25 09:32:12 +00:00
Daniel Chambers
7e4c24aea0 Fix WebSockets api-tests breaking for Data Connector backends
[GDC-1024]: https://hasurahq.atlassian.net/browse/GDC-1024?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9291
GitOrigin-RevId: f7a2306268fc1a22744dd9393ac17648551434b0
2023-05-25 07:06:28 +00:00
Tom Harding
e0c0043e76 Upgrade Ormolu to 0.7.0.0
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9284
GitOrigin-RevId: 2f2cf2ad01900a54e4bdb970205ac0ef313c7e00
2023-05-24 13:53:53 +00:00
David Overton
e5f88d8039 Nested array support for Data Connectors Backend and MongoDB
## Description

This change adds support for querying into nested arrays in Data Connector agents that support such a concept (currently MongoDB).

### DC API changes

- New API type `ColumnType` which allows representing the type of a "column" as either a scalar type, an object reference or an array of `ColumnType`s. This recursive definition allows arbitrary nesting of arrays of types.
- The `type` fields in the API types `ColumnInfo` and `ColumnInsertSchema` now take a `ColumnType` instead of a `ScalarType`.
- To ensure backwards compatibility, a `ColumnType` representing a scalar serialises and deserialises to the same representation as `ScalarType`.
- In queries, the `Field` type now has a new constructor `NestedArrayField`. This contains a nested `Field` along with optional `limit`, `offset`, `where` and `order_by` arguments. (These optional arguments are not yet used by either HGE or the MongoDB agent.)

### MongoDB Haskell agent changes

- The `/schema` endpoint will now recognise arrays within the JSON validation schema and generate corresponding arrays in the DC schema.
- The `/query` endpoint will now handle `NestedArrayField`s within queries (although it does not yet handle `limit`, `offset`, `where` and `order_by`).

### HGE server changes

- The `Backend` type class adds a new type family `XNestedArrays b` to enable nested arrays on a per-backend basis (currently enabled only for the `DataConnector` backend.
- Within `RawColumnInfo` the column type is now represented by a new type `RawColumnType b` which mirrors the shape of the DC API `ColumnType`, but uses `XNestedObjects b` and `XNestedArrays b` type families to allow turning nested object and array supports on or off for a particular backend. In the `DataConnector` backend `API.CustomType` is converted into `RawColumnInfo 'DataConnector` while building the schema.
- In the next stage of schema building, the `RawColumnInfo` is converted into a `StructuredColumnInfo` which allows us to represent the three different types of columns: scalar, object and array. TODO: the `StructuredColumnInfo` looks very similar to the Logical Model types. The main difference is that it uses the `XNestedObjects` and `XNestedArrays` type families. We should be able to combine these two representations.
- The `StructuredColumnInfo` is then placed into a `FIColumn` `FieldInfo`. This involved some refactoring of `FieldInfo` as I had previously split out `FINestedObject` into a separate constructor. However it works out better to represent all "column" fields (i.e. scalar, object and array) using `FIColumn` as this make it easier to implement permission checking correctly. This is the reason the `StructuredColumnInfo` was needed.
- Next, the `FieldInfo` are used to generate `FieldParser`s. We add a new constructor to `AnnFieldG` for `AFNestedArray`. An `AFNestedArray` field parser can contain either a simple array selection or an array aggregate. Simple array `FieldParsers` are currently limited to subfield selection. We will add support for limit, offset, where and order_by in a future PR. We also don't yet generate array aggregate `FieldParsers.
- The new `AFNestedArray` field is handled by the `QueryPlan` module in the `DataConnector` backend. There we generate an `API.NestedArrayField` from the AFNestedArray. We also handle nested arrays when reshaping the response from the DC agent.

## Limitations

- Support for limit, offset, filter (where) and order_by is not yet fully implemented, although it should not be hard to add this
- Support for aggregations on nested arrays is not yet fully implemented
- Permissions involving nested arrays (and objects) not yet implemented
- This should be integrated with Logical Model types, but that will happen in a separate PR

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9149
GitOrigin-RevId: 0e7b71a994fc1d2ca1ef73bfe7b96e95b5328531
2023-05-24 08:02:43 +00:00
Daniel Chambers
f77b6aaa1d Data Connectors: Support using array literals in permission checks
[GDC-868]: https://hasurahq.atlassian.net/browse/GDC-868?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9238
GitOrigin-RevId: 93593f79a4a6a92eec1d0dc10fd8e5bb21bd123e
2023-05-24 04:42:02 +00:00
Lyndon Maydwell
4a38055e9e Adding default table return type to fibs functions in UDF API tests
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9270
GitOrigin-RevId: a08e991e60893aa18b370652fc993b437797ed3c
2023-05-23 23:23:54 +00:00
Tom Harding
ec63ea6ed0 Use the backend-agnostic get_table_info for GDCs
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9186
GitOrigin-RevId: 08f0772139f155918cbc136392cf9869db4c6332
2023-05-23 16:49:45 +00:00