Commit Graph

6719 Commits

Author SHA1 Message Date
David Overton
346804fc67 Support nested object fields in DC API and use this to implement nest…
## Description

This change adds support for nested object fields in HGE IR and Schema Cache, the Data Connectors backend and API, and the MongoDB agent.

### Data Connector API changes

- The `/schema` endpoint response now includes an optional set of GraphQL type definitions. Table column types can refer to these definitions by name.
- Queries can now include a new field type `object` which contains a column name and a nested query. This allows querying into a nested object within a field.

### MongoDB agent changes

- Add support for querying into nested documents using the new `object` field type.

### HGE changes

- The `Backend` type class has a new type family `XNestedObjects b` which controls whether or not a backend supports querying into nested objects. This is currently enabled only for the `DataConnector` backend.
- For backends that support nested objects, the `FieldInfo` type gets a new constructor `FINestedObject`, and the `AnnFieldG` type gets a new constructor `AFNestedObject`.
- If the DC `/schema` endpoint returns any custom GraphQL type definitions they are stored in the `TableInfo` for each table in the source.
- During schema cache building, the function `addNonColumnFields` will check whether any column types match custom GraphQL object types stored in the `TableInfo`. If so, they are converted into `FINestedObject` instead of `FIColumn` in the `FieldInfoMap`.
- When building the `FieldParser`s from `FieldInfo` (function `fieldSelection`) any `FINestedObject` fields are converted into nested object parsers returning `AFNestedObject`.
- The `DataConnector` query planner converts `AFNestedObject` fields into `object` field types in the query sent to the agent.

## Limitations

### HGE not yet implemented:
- Support for nested arrays
- Support for nested objects/arrays in mutations
- Support for nested objects/arrays in order-by
- Support for filters (`where`) in nested objects/arrays
- Support for adding custom GraphQL types via track table metadata API
- Support for interface and union types
- Tests for nested objects

### Mongo agent not yet implemented:

- Generate nested object types from validation schema
- Support for aggregates
- Support for order-by
- Configure agent port
- Build agent in CI
- Agent tests for nested objects and MongoDB agent

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7844
GitOrigin-RevId: aec9ec1e4216293286a68f9b1af6f3f5317db423
2023-04-11 01:30:37 +00:00
Sean Park-Ross
a1cd2b8176 Docs: Websocket Auth Clarification
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8695
GitOrigin-RevId: c19c121083e768d805d2adad38f7027ecd134957
2023-04-10 15:33:44 +00:00
Daniel Harvey
99b0139275 chore(server): variable renaming around custom return type permissions
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8624
Co-authored-by: Gil Mizrahi <8547573+soupi@users.noreply.github.com>
GitOrigin-RevId: 901db6df865ce0433399731f878c6f3a10cf6e87
2023-04-10 14:17:20 +00:00
Puru Gupta
c54697b958 server: enable remote schema to remote schema relationship for unauthenticated context
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8562
GitOrigin-RevId: 2b1e2241fd2dbe379d5f9e40224e1245f1188475
2023-04-10 13:35:38 +00:00
Karthikeyan Chinnakonda
ddf34d9ffa Miscellaneous scheduled triggers fixes
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8618
GitOrigin-RevId: f9e3eb0d3d8b2ba36344b9ce0739c6a51e77f59e
2023-04-10 12:27:21 +00:00
Rakesh Emmadi
b18c86c7f7 docs: document the strict parameter in healthz API reference
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8693
GitOrigin-RevId: 9201279658a474da7624ab439345ef62fbc56fc4
2023-04-10 11:45:48 +00:00
Vijay Prasanna
33bea64ed1 console (fix): Remove cloud/pro only check for dynamic db routing
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8661
Co-authored-by: Sooraj <8408875+soorajshankar@users.noreply.github.com>
GitOrigin-RevId: 9216c4a04e49a35b13ace6aa0312a0fe4665ab87
2023-04-10 10:57:07 +00:00
Toan Nguyen
5715fe4931 console: remove the response_size > 0 filter in the Operations table
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8656
GitOrigin-RevId: c6d160a75dfe87dae8f870e9bc0a2113bb0f846d
2023-04-07 12:03:08 +00:00
Vishnu Bharathi
eabbb917ed ci: tag release v2.22.1
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8681
GitOrigin-RevId: 79762246e1115f6495d9f4b06d2ec9f43254b5e2
2023-04-07 11:15:45 +00:00
Naveen Naidu
2814735a20 server: fix dirty read of schema cache
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8677
GitOrigin-RevId: 615bfb4dc8d22a46a87ceb76c5f89d608e88f97d
2023-04-07 09:23:03 +00:00
Daniel Chambers
2e443401f3 Update super connector version in Oracle compose file
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8674
GitOrigin-RevId: 985bf9249e63756d082748b0fafc8ead18916ec7
2023-04-07 02:16:08 +00:00
Daniel Chambers
a93fb4f11a Update versions of HGE in super connector-based compose files used by docs
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8673
GitOrigin-RevId: f9372eb69398cc058f81606d46f5250c77240890
2023-04-07 01:37:10 +00:00
Daniele Cammareri
9140d2c83b doc: add validation for dynamic routing
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8669
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
GitOrigin-RevId: a1fe971ae0b62ea90da6f4a1ac303293fa7b8a44
2023-04-06 18:21:59 +00:00
Varun Dey
5addb8cfca console: Use case experiment
[GT-555]: https://hasurahq.atlassian.net/browse/GT-555?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8621
Co-authored-by: Rikin Kachhia <54616969+rikinsk@users.noreply.github.com>
GitOrigin-RevId: f7d192f637948eebdf157d7ee6e9834da42bd3a9
2023-04-06 17:35:47 +00:00
Daniele Cammareri
904803c09d console: add connection template validation for dynamic routing
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8527
GitOrigin-RevId: 8906b92e4ab97e92547220fada146707d8085e57
2023-04-06 16:27:57 +00:00
Antoine Leblanc
0476331136 small cleanups of pro's init
### Description

(This PR is better reviewed commit by commit.)

This PR is an aggregation of small incremental changes to Pro's init:
- it deletes some dead code,
- it starts reorganizing the code of that file by sections, similar to OSS' init,
- it extracts and cleans up license key cache init (groups several blocks of code in one separate function)
- makes some changes to a service class to reduce the dependency on `_acAppStateRef`

This PR is a first step: our goal is to move the schema cache build _in_ the app monad, in order to achieve #8344. To do so, we will need to remove `_acAppStateRef` from Pro's `AppContext`. There are two different paths we can take from here, which is why i cut this PR here:
- the first would be to change the different instances we implement on `AppM` to take as an argument the parts of the schema cache they depend on, rather than reading them from `_acAppStateRef`; as of this PR, `MetricsConfig` is the only such field;
- the second would be to apply the same strategy we already used for the TLSAllowList, and use a `IORef` that can be updated after the schema cache is built; this change would have a smaller footprint, but introduces one new `IORef` per such field, which feels something we don't want to generalize

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8655
GitOrigin-RevId: 809697d460bdb5c83ef7d30a2e835f589bcd80a6
2023-04-06 15:37:15 +00:00
Rikin Kachhia
ed9f120241 docs: add cloud use case based onboarding guides
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8578
Co-authored-by: Abhijeet Khangarot <26903230+abhi40308@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Aaysha <109507451+aayshasura@users.noreply.github.com>
Co-authored-by: nevermore <31686586+OjasWadhwani@users.noreply.github.com>
Co-authored-by: Priya Sharma <25654753+PriyaSharma45@users.noreply.github.com>
Co-authored-by: Vaishnavi <85152989+vaishnavigvs@users.noreply.github.com>
Co-authored-by: Rob Dominguez <24390149+robertjdominguez@users.noreply.github.com>
GitOrigin-RevId: a2f5154c1dece4513c8b86849a5e54cbf3560cbb
2023-04-06 14:45:21 +00:00
Julian@Hasura
ec0fec5166 console: Fix empty columns on mysql permissions builder
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8654
GitOrigin-RevId: 40a29888d94856792b454b73c052a4bd1b6c4dc4
2023-04-06 13:38:13 +00:00
Sean Park-Ross
32fd3e04be Docs: Zero downtime env var updates admonition
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8642
GitOrigin-RevId: 43489870696ec745e885d466e45554c48f814ba7
2023-04-06 12:54:55 +00:00
Tirumarai Selvan
8b49d28ea8 Rename title for dynamic routing and other edits
- Renames feature to Dynamic Routing or Dynamic Routing for Databases (as appropriate)
- Adds a heading for How it works

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8663
Co-authored-by: Rob Dominguez <24390149+robertjdominguez@users.noreply.github.com>
GitOrigin-RevId: dea48043ce490cd4cda7d37e8a2e1633945596fb
2023-04-06 12:20:19 +00:00
Samir Talwar
6d79b7dfaf make: Add run-oss-server, and associated frontend targets.
`make run-oss-server` will now build the Console for OSS ("CE"), start PostgreSQL, and run a simply-configured server.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8652
GitOrigin-RevId: cada25e66fe216867ff331d597764e64aaf2dc19
2023-04-06 08:53:13 +00:00
Stefano Magni
ab02ed21f6 chore(console): Mention another source of flakiness in the testing best practices
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8660
GitOrigin-RevId: 07f38ad31a256b55b570aad72d127698dbb50042
2023-04-06 08:03:54 +00:00
Rob Dominguez
281f23f8b6 docs: update ee trial instructions
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8653
GitOrigin-RevId: d4a3a2762d347584cdf8c160e866c6987f5b4e30
2023-04-06 03:14:38 +00:00
paritosh-08
774075a709 server: add code field to warning object in metadata API responses
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8601
GitOrigin-RevId: 7034c0a04b87c44731680917aa3946084b705959
2023-04-06 02:41:24 +00:00
Tom Harding
3cef692dd7 feature(server): Get _all_ source tables
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8649
GitOrigin-RevId: 4bb9311d1d7ab4e8ee641bde5df2babcb1b8b306
2023-04-05 20:16:14 +00:00
Philip Lykke Carlsen
41a5092b97 refactor(tests): Support componentised Postgres fixtures, port SimpleObjectSpec
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8607
GitOrigin-RevId: b068f7308fa8fa4b626aa2f9352da3965b03fa34
2023-04-05 18:05:37 +00:00
Rishichandra Wawhal
660ba821c6 README, docs: add Telemetry note
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8648
GitOrigin-RevId: 0678a0099c11f948ffaf327b6df02538e41834db
2023-04-05 17:23:15 +00:00
Tom Harding
7c1345b4ba hotfix(console): Prefix get_source_tables in the console
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8647
GitOrigin-RevId: 212dfe78345a6f2b59ccfeb655fe6212e60a1cad
2023-04-05 16:38:29 +00:00
Daniel Harvey
9c83934869 chore(tooling): add make repl-postgres command
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8638
GitOrigin-RevId: 8cbc8317a661e2bb43e965be5c32696d03d394bd
2023-04-05 13:24:04 +00:00
Erik Magnusson
215c447dad Erik/permission/quickfixes
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8639
GitOrigin-RevId: bc8d53d358fa340abc9ee4ad51c8ee949a2556cd
2023-04-05 13:01:39 +00:00
Vishnu Bharathi
60afce2b94 ci: tag release v2.23.0-beta.1
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8641
GitOrigin-RevId: e4395099602829a41d66c47e2c821b7a8912acef
2023-04-05 12:27:36 +00:00
Vaishnavi
91c3a4ec09 EE Trial Docs
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8432
Co-authored-by: Manas Agarwal <5352361+manasag@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Rob Dominguez <24390149+robertjdominguez@users.noreply.github.com>
Co-authored-by: Rishichandra Wawhal <27274869+wawhal@users.noreply.github.com>
GitOrigin-RevId: cc03d4602057f0032abad2a4c0f915e68024f3b9
2023-04-05 11:40:55 +00:00
Antoine Leblanc
d00f4ef497 Always flush the loggers without explicitly relying on onException
### Description

This PR makes use of the fact that the `LoggerSet` is created in a `ManagedT` context to guarantee that we flush it when the `ManagedT` context terminates, which is guaranteed even when an exception is thrown since `allocate` is implemented using `bracket`.

This allows us to remove several manual calls to `flip onException flushLoggers`, and avoids having to manually perform that check everywhere.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8629
GitOrigin-RevId: 47657b910dc1bb7cfc0594f02e6cce9893ae3439
2023-04-05 11:07:14 +00:00
Rishichandra Wawhal
c6d65508b2 [feature branch] EE Lite Trials
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8208
Co-authored-by: awjchen <13142944+awjchen@users.noreply.github.com>
Co-authored-by: Vijay Prasanna <11921040+vijayprasanna13@users.noreply.github.com>
Co-authored-by: Toan Nguyen  <1615675+hgiasac@users.noreply.github.com>
Co-authored-by: Abhijeet Khangarot <26903230+abhi40308@users.noreply.github.com>
Co-authored-by: Solomon <24038+solomon-b@users.noreply.github.com>
Co-authored-by: gneeri <10553562+gneeri@users.noreply.github.com>
GitOrigin-RevId: 454ee0dea636da77e43810edb2f427137027956c
2023-04-05 08:59:09 +00:00
Nicolas Inchauspe
f2b76b7da7 console: replace bootstrap modal with radix one
[PLAT-511]: https://hasurahq.atlassian.net/browse/PLAT-511?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8622
GitOrigin-RevId: 8387bad11c7529ded8a89c551f5858f3f1ee0c27
2023-04-05 08:24:06 +00:00
Varun Choudhary
6bf0a49cd6 console: e2e snapshot testing for remote schema
This PR adds e2e snapshot testing for remote schema.
We are testing it in 2 ways -

1. testing the shortest possible path to create RS and then modifying it to the longest possible path
2. testing the longest possible path to create RS and then modifying it to shortest possible path
3. [ see doc](https://docs.google.com/document/d/1NjZsvd6xOq90lNMai8nKBaWl-LqBWrkiZCHSQ7wusdE/edit) for more detail

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8383
GitOrigin-RevId: d045219ca7db00e797492e9d61efb25283c1b3b3
2023-04-05 07:42:53 +00:00
Daniel Chambers
fde4c0fae5 Enhance insert table schema with extra info and remove need for cached schema usage in Super Connector mutations
[GDC-643]: https://hasurahq.atlassian.net/browse/GDC-643?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8560
GitOrigin-RevId: 0e965da447eb6a5acf5a0f291f9e205882630e6e
2023-04-05 02:23:20 +00:00
awjchen
a4db4c225f docs: enterprise edition telemetry
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8575
Co-authored-by: Manas Agarwal <5352361+manasag@users.noreply.github.com>
Co-authored-by: Rob Dominguez <24390149+robertjdominguez@users.noreply.github.com>
GitOrigin-RevId: 83afe631cf0cb20a0d26fc28b1151f1e30376101
2023-04-04 22:37:56 +00:00
Antoine Leblanc
e6a8bcb1c9 Make flag value a part of the cache's dynamic config.
### Description

If i understand correctly, the _value_ of a feature flag **can** change at runtime. Since this has an impact on how the schema cache is built, this PR makes the value of the flag a part of the cache's "dynamic" config, therefore making it an arrow-ish argument for the incremental framework. It also removes `CheckFeatureFlag` from the static config, now obsolete.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8631
GitOrigin-RevId: 2ca898e4193e8552c40d2b21a819d3dd414601fe
2023-04-04 19:39:39 +00:00
Varun Choudhary
18e0ae0783 console: fix flaky test for event trigger
This PR fixes the flaky test, discussed [here ](https://hasurahq.slack.com/archives/C04E6U4TUM6/p1680545040680069?thread_ts=1680544673.299789&cid=C04E6U4TUM6)

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8613
GitOrigin-RevId: 79deee9bd4e1abfcde19cc848ce1de2b5bf248a5
2023-04-04 16:35:06 +00:00
Antoine Leblanc
306162f477 Remove ServerConfigCtx.
### Description

This PR removes `ServerConfigCtx` and `HasServerConfigCtx`. Instead, it favours different approaches:
- when the code was only using one field, it passes that field explicitly (usually `SQLGenCtx` or `CheckFeatureFlag`)
- when the code was using several fields, but in only one function, it inlines
- for the cache build, it introduces `CacheStaticConfig` and `CacheDynamicConfig`, which are subsets of `AppEnv` and `AppContext` respectively

The main goal of this is to help with the modularization of the engine: as `ServerConfigCtx` had fields whose types were imported from several unrelated parts of the engine, using it tied together parts of the engine that should not be aware of one another (such as tying together `Hasura.LogicalModel` and `Hasura.GraphQL.Schema`).

The bulk of this PR is a change to the cache build, as a follow up to #8509: instead of giving the entire `ServerConfigCtx` as a incremental rule argument, we only give the new `CacheDynamicConfig` struct, which has fewer fields. The other required fields, that were coming from the `AppEnv`, are now given via the `HasCacheStaticConfig` constraint, which is a "subset" of `HasAppEnv`.

(Some further work could include moving `StringifyNumbers` out of `GraphQL.Schema.Options`, given how it is used all across the codebase, including in `RQL.DML`.)

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8513
GitOrigin-RevId: 818cbcd71494e3cd946b06adbb02ca328a8a298e
2023-04-04 16:01:42 +00:00
Tom Harding
794690f30c Implement get_source_tables command for all backends
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8619
GitOrigin-RevId: 5e1b2c11775d801a77dc6d92de4c5abc1c9b8c60
2023-04-04 15:27:19 +00:00
Sean Park-Ross
9c779a64bf docs/sean/small-fixes-03
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8536
Co-authored-by: Rob Dominguez <24390149+robertjdominguez@users.noreply.github.com>
GitOrigin-RevId: 422c938f13d850caa22b01b867d48f515e1f7038
2023-04-04 14:49:52 +00:00
Gil Mizrahi
352d8ff09c validate logical models custom return type against postgres
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8563
GitOrigin-RevId: 462a608e0e90d1923bc1d735257f4506825f5db1
2023-04-04 14:02:59 +00:00
Puru Gupta
8a6a8459ab server: zero-downtime env vars update on cloud
[GS-232]: https://hasurahq.atlassian.net/browse/GS-232?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8475
Co-authored-by: Anon Ray <616387+ecthiender@users.noreply.github.com>
Co-authored-by: Naveen Naidu <30195193+Naveenaidu@users.noreply.github.com>
GitOrigin-RevId: fb47f02f4e0d6d889c0348abc50fd46382f1bd5e
2023-04-04 13:26:21 +00:00
Daniel Harvey
9c99bcb6f8 chore(server): move perms to custom return types
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8604
GitOrigin-RevId: ff024429b3f06e4867334665f35d4bd404a85cba
2023-04-04 12:46:51 +00:00
Julian@Hasura
71a00338ff console: Mongo Schema Form DSF-205
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8557
GitOrigin-RevId: 292043907095e37c7bb99725a754fb7da9f71743
2023-04-04 12:10:27 +00:00
Antoine Leblanc
0177510b8f Fix haskell error in example code block
Fix trivial error in documentation.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8620
GitOrigin-RevId: 1b8410ae213b1e5f0954bdf4e957d29f1a668696
2023-04-04 11:28:35 +00:00
Antoine Leblanc
e9ae080d28 Add ManagedT deep dive.
### Description

This small documentation PR explains why and how we use `ManagedT`, and how it solves problems we would have with `Managed`.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8615
GitOrigin-RevId: 5b7710a8cb6373fefb69bb4e9e8eda389c14c2da
2023-04-04 10:34:58 +00:00
Tom Harding
4854d3ca21 Add extremely basic support for BigQuery JSON fields
[NDAT-501]: https://hasurahq.atlassian.net/browse/NDAT-501?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8614
GitOrigin-RevId: 24530130499174a1a02e0fb4510859fcfeed207d
2023-04-04 09:45:17 +00:00