Commit Graph

213 Commits

Author SHA1 Message Date
Philip Lykke Carlsen
30fbdf2e83 feat(tests): Add support for heartbeat monitoring
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9212
GitOrigin-RevId: 38491ce24ae766533b55e0b402b883d3066618fe
2023-05-22 08:04:03 +00:00
Daniel Harvey
a73a009031 chore(server): more expressive logical models
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9197
GitOrigin-RevId: 3471c68d4e59310bdb62ecb3694ab30a2094916e
2023-05-19 16:14:20 +00:00
Lyndon Maydwell
cb8e6feb2e Adding UDF (user-defined-functions) support to Data Connectors - GDC-820
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8121
GitOrigin-RevId: ceb3e29e330bba294061f85c1f75700974d01452
2023-05-19 04:48:46 +00:00
Gil Mizrahi
db5370bb62 test-harness: if server fails to start, throwTo main
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9206
GitOrigin-RevId: 0663121db8ee546799a4de6deae861740614afc5
2023-05-18 10:02:15 +00:00
Daniel Chambers
605f0281dc Add untrack_tables metadata API
[GDC-1198]: https://hasurahq.atlassian.net/browse/GDC-1198?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9181
GitOrigin-RevId: 0ff4acae990476f93d11829173e0e4f31e8db04e
2023-05-17 15:31:56 +00:00
Philip Lykke Carlsen
3c7fa7cca1 feat(tests-harness): Support api-tests using postgres-agent data connector
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9188
Co-authored-by: Daniel Harvey <4729125+danieljharvey@users.noreply.github.com>
GitOrigin-RevId: 3eda64bcfcaebfb3fcef6783a88a1a4d36dade43
2023-05-17 14:44:06 +00:00
Philip Lykke Carlsen
e73f997284 feat(test-harness): Maintain pools of hge processes
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9158
GitOrigin-RevId: 243e2d771cd8c31852b8b1959b006f27440f079d
2023-05-16 10:05:12 +00:00
Daniel Chambers
c664e1fbba Add track_tables metadata API
[GDC-1198]: https://hasurahq.atlassian.net/browse/GDC-1198?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9120
GitOrigin-RevId: bb86acaf8b06529a34113a80c3fe2409c3d69a63
2023-05-16 04:05:02 +00:00
Philip Lykke Carlsen
53841c98c9 refactor(test-harness): Make admin secret injectable
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9147
GitOrigin-RevId: e12138ad59fb690261e5b8e474d959623bce0826
2023-05-15 18:27:52 +00:00
Samir Talwar
bc8c7406df Remove native MySQL dependencies and code wherever I found it.
This includes:

  - CI scripts
  - Cabal dependency lists
  - Dockerfiles
  - Docker Compose files
  - server test harness code

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9066
GitOrigin-RevId: 509d7931693c4755509d5b8e5bcf50b42cc24769
2023-05-05 22:20:55 +00:00
Daniel Harvey
285a200a87 chore(server): delete MySQL native backend
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9050
GitOrigin-RevId: 1515fb0efdb1baa05ffe3ff7cf6f230acd0cde29
2023-05-05 10:32:56 +00:00
Daniel Harvey
1181ca78a3 feature(server): metadata for native query -> native query object relationships
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9026
GitOrigin-RevId: 5a0528ed48f6e0d2e9a3e7f5d3090a0208dad3e1
2023-05-04 14:33:06 +00:00
Gil Mizrahi
e8d2d4f364 Refer to a stored procedure by name in commands interface
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9014
GitOrigin-RevId: 64f51811770d88ad6d5b298fc4c71170ffd7182c
2023-05-02 13:32:01 +00:00
Daniel Harvey
cb00535693 chore(server): add link-type to Logical Model references
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9004
GitOrigin-RevId: f361ca5f3b82995096f9289f8e90c7c20920a744
2023-04-28 16:48:19 +00:00
Gil Mizrahi
930df62de7 Stored procedures api commands
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8975
GitOrigin-RevId: c71a9f74bf01bb8c0bc8c8cd4b744b530d99476a
2023-04-28 13:38:34 +00:00
Samir Talwar
e3b46b78a9 server/upgrade-tests: Tests that ensure that HGE upgrades successfully.
These tests ensure that upgrading HGE preserves the GraphQL schema.

They do this by running two different versions of HGE against the same metadata, and ensuring that the GraphQL schema doesn't change.

We might find that in the future, we make an additive change that makes these tests fail. Improving the tests to allow for this is left as an exercise to whoever triggers it. (Sorry.)

Currently, we do this with:

  * an empty database (zero tracked relations)
  * the Chinook dataset
  * the "huge schema" dataset

The base version of HGE tested against can be overridden with an option. The version must be available on Docker Hub.

Further information is in the Haddock documentation.

[NDAT-627]: https://hasurahq.atlassian.net/browse/NDAT-627?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8982
GitOrigin-RevId: 97b4deda1e6fe1db33ce35db02e12c6acc6c29e3
2023-04-28 12:54:51 +00:00
Daniel Harvey
a0b1d69392 chore(server): make native queries pro-only
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8926
GitOrigin-RevId: 68d2091c97ed80e86f851d656749eab865f73ec4
2023-04-27 14:31:53 +00:00
Tom Harding
b6799f0882 Import InsOrdHashMap, not OMap, OM, Map, HM, ...
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8946
GitOrigin-RevId: 434e7c335bc69119020dd35761c7d4539bc51ff8
2023-04-27 07:43:22 +00:00
Tom Harding
4885a3fd9a Import J, not Aeson, A, JSON, Yaml...
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8948
GitOrigin-RevId: d70c4a50b94ffe7d42a1fb1017051d351f236acc
2023-04-26 17:30:24 +00:00
Daniel Chambers
cb80942e4f Disable use of local relationships in metadata if Data Connector agents don't declare relationships capability
[GDC-489]: https://hasurahq.atlassian.net/browse/GDC-489?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8862
GitOrigin-RevId: cfff2d9c688185e00f4a830dbdbe173dba4ce7f0
2023-04-26 06:49:36 +00:00
Samir Talwar
be67b0db59 server: Make HLint pass, by hook or by crook.
This fixes the simple HLint warnings, and adds a few suppressions to avoid noise.

The suppressions don't really solve the problems, but I think the warnings here are quite benign and I'm uncomfortable with how likely I would be to introduce a bug during refactoring.

In the case of _pg-client_ and _resource-pool_, we can't use the recommended functions anyway, and there doesn't seem to be a way to tell HLint to ignore entire packages.

I have updated the `make` targets to only fail if errors or warnings are found, not suggestions. This brings it in line with the CI job.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8910
GitOrigin-RevId: 596277b4ae5833876fc3f43875208c1279518a59
2023-04-25 14:03:02 +00:00
Tom Harding
f8ae944dbc Move Hasura.GraphQL.Schema.Options to Hasura.RQL.Types.Options
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8877
GitOrigin-RevId: 8be82f60a57cd9582d6980a6dea2f34c7b0c13c1
2023-04-24 15:18:56 +00:00
Tom Harding
99c1d99ecb chore(server): break out Hasura.Base into own package
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8850
Co-authored-by: Daniel Harvey <4729125+danieljharvey@users.noreply.github.com>
GitOrigin-RevId: 47310479d66adc3baf27ca9d2dbda7addea6bb3d
2023-04-21 14:26:15 +00:00
Tom Harding
3151dd0074 Extract hasura-extras from graphql-engine
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8848
Co-authored-by: Daniel Harvey <4729125+danieljharvey@users.noreply.github.com>
GitOrigin-RevId: 4a667e2760f866e57e0e7d804815153571b413e6
2023-04-21 13:15:27 +00:00
Philip Lykke Carlsen
983fc2ad47 Rename "Custom Return Types" → "Logical Models"
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8800
GitOrigin-RevId: e5e10f31c6cc8953a8ee947441a7f80b0e9b5e5e
2023-04-19 09:05:25 +00:00
Daniel Harvey
2fd3f91398 chore(server): metadata changes for array relationships for Native Queries
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8670
GitOrigin-RevId: c23e23e3cf48013ab76fc2fa98c8b8b800c6cee6
2023-04-17 11:31:59 +00:00
Philip Lykke Carlsen
0346224444 Rename "Logical Models" → "Native Queries"
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8769
GitOrigin-RevId: 66f2cbfb620d641e672a4074554d9d324a18c591
2023-04-13 16:12:20 +00:00
Daniel Chambers
56db8ec358 Data Connectors: Fix track_table not working if the table was created after schema cache was built
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8732
GitOrigin-RevId: e11a0ef6979d1d58a0b39dcd8fff48d446d3420f
2023-04-13 01:30:50 +00:00
David Overton
346804fc67 Support nested object fields in DC API and use this to implement nest…
## Description

This change adds support for nested object fields in HGE IR and Schema Cache, the Data Connectors backend and API, and the MongoDB agent.

### Data Connector API changes

- The `/schema` endpoint response now includes an optional set of GraphQL type definitions. Table column types can refer to these definitions by name.
- Queries can now include a new field type `object` which contains a column name and a nested query. This allows querying into a nested object within a field.

### MongoDB agent changes

- Add support for querying into nested documents using the new `object` field type.

### HGE changes

- The `Backend` type class has a new type family `XNestedObjects b` which controls whether or not a backend supports querying into nested objects. This is currently enabled only for the `DataConnector` backend.
- For backends that support nested objects, the `FieldInfo` type gets a new constructor `FINestedObject`, and the `AnnFieldG` type gets a new constructor `AFNestedObject`.
- If the DC `/schema` endpoint returns any custom GraphQL type definitions they are stored in the `TableInfo` for each table in the source.
- During schema cache building, the function `addNonColumnFields` will check whether any column types match custom GraphQL object types stored in the `TableInfo`. If so, they are converted into `FINestedObject` instead of `FIColumn` in the `FieldInfoMap`.
- When building the `FieldParser`s from `FieldInfo` (function `fieldSelection`) any `FINestedObject` fields are converted into nested object parsers returning `AFNestedObject`.
- The `DataConnector` query planner converts `AFNestedObject` fields into `object` field types in the query sent to the agent.

## Limitations

### HGE not yet implemented:
- Support for nested arrays
- Support for nested objects/arrays in mutations
- Support for nested objects/arrays in order-by
- Support for filters (`where`) in nested objects/arrays
- Support for adding custom GraphQL types via track table metadata API
- Support for interface and union types
- Tests for nested objects

### Mongo agent not yet implemented:

- Generate nested object types from validation schema
- Support for aggregates
- Support for order-by
- Configure agent port
- Build agent in CI
- Agent tests for nested objects and MongoDB agent

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7844
GitOrigin-RevId: aec9ec1e4216293286a68f9b1af6f3f5317db423
2023-04-11 01:30:37 +00:00
Tom Harding
3cef692dd7 feature(server): Get _all_ source tables
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8649
GitOrigin-RevId: 4bb9311d1d7ab4e8ee641bde5df2babcb1b8b306
2023-04-05 20:16:14 +00:00
Philip Lykke Carlsen
41a5092b97 refactor(tests): Support componentised Postgres fixtures, port SimpleObjectSpec
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8607
GitOrigin-RevId: b068f7308fa8fa4b626aa2f9352da3965b03fa34
2023-04-05 18:05:37 +00:00
Rishichandra Wawhal
c6d65508b2 [feature branch] EE Lite Trials
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8208
Co-authored-by: awjchen <13142944+awjchen@users.noreply.github.com>
Co-authored-by: Vijay Prasanna <11921040+vijayprasanna13@users.noreply.github.com>
Co-authored-by: Toan Nguyen  <1615675+hgiasac@users.noreply.github.com>
Co-authored-by: Abhijeet Khangarot <26903230+abhi40308@users.noreply.github.com>
Co-authored-by: Solomon <24038+solomon-b@users.noreply.github.com>
Co-authored-by: gneeri <10553562+gneeri@users.noreply.github.com>
GitOrigin-RevId: 454ee0dea636da77e43810edb2f427137027956c
2023-04-05 08:59:09 +00:00
Daniel Chambers
fde4c0fae5 Enhance insert table schema with extra info and remove need for cached schema usage in Super Connector mutations
[GDC-643]: https://hasurahq.atlassian.net/browse/GDC-643?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8560
GitOrigin-RevId: 0e965da447eb6a5acf5a0f291f9e205882630e6e
2023-04-05 02:23:20 +00:00
Tom Harding
794690f30c Implement get_source_tables command for all backends
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8619
GitOrigin-RevId: 5e1b2c11775d801a77dc6d92de4c5abc1c9b8c60
2023-04-04 15:27:19 +00:00
Gil Mizrahi
352d8ff09c validate logical models custom return type against postgres
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8563
GitOrigin-RevId: 462a608e0e90d1923bc1d735257f4506825f5db1
2023-04-04 14:02:59 +00:00
Daniel Harvey
79682e0598 chore(server): move custom types out of logical models
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8565
Co-authored-by: Tom Harding <6302310+i-am-tom@users.noreply.github.com>
GitOrigin-RevId: 38bf56cc420a6c818a9ca7d6f846f5018535c808
2023-03-31 15:35:13 +00:00
Daniel Harvey
4905dc1675 chore(tests): pass BigQuery service account env var through to HGE
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8532
GitOrigin-RevId: 68e4c846e23369f476fa672b987d6769fad43cbf
2023-03-30 17:59:01 +00:00
Daniel Harvey
7227e96278 feature(server): custom return types as discreet metadata entity
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8556
Co-authored-by: Tom Harding <6302310+i-am-tom@users.noreply.github.com>
GitOrigin-RevId: b7dcbcf378279c3bf4c8d223174b90c2cb4b9e53
2023-03-30 15:15:11 +00:00
Philip Lykke Carlsen
ed813655f6 refactor(tests): late binding of shouldReturnYaml
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8558
GitOrigin-RevId: 5044f4c726b71760fdbb05b71a1fc79e4eaa8a31
2023-03-30 09:05:04 +00:00
Philip Lykke Carlsen
e40e89d2d7 refactor(test-harness): Split up and move Harness.Test.Schema
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8545
GitOrigin-RevId: 9069737c23a286b3cefab59714064beac1465295
2023-03-29 17:40:33 +00:00
Naveen Naidu
4e3dbed938 server: revert changes to created_at column of 'hdb_catalog.event_log'
This PR reverts the following two commits:
1. https://github.com/hasura/graphql-engine-mono/pull/8287
2. https://github.com/hasura/graphql-engine-mono/pull/8467

We are undoing a migration that was done on `hdb_catalog.event_log` table which was done in d4ae6a517da63f2f43567dc16fda135b3cd1d7e6 . And as such, users who were using event triggers on that version will come across the error:
```json
{"detail":{"info":{"code":"not-supported","error":"Expected source catalog version <= 3, but the current version is 4","path":"$"},"kind":"catalog_migrate"},"level":"error","timestamp":"2023-03-28T10:17:24.289+0530","type":"startup"}
{"code":"not-supported","error":"Expected source catalog version <= 3, but the current version is 4","path":"$"}
```
To fix these errors please run the following SQL on the source where event triggers were created on:
```
UPDATE hdb_catalog.hdb_source_catalog_version SET version = 3, upgraded_on= NOW();
ALTER table hdb_catalog.event_log ALTER COLUMN created_at SET DEFAULT NOW();
ALTER table hdb_catalog.event_invocation_logs ALTER COLUMN created_at SET DEFAULT NOW();
```

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8534
GitOrigin-RevId: b6bbcce0163c8beed80619d3cea056e643b8c180
2023-03-29 10:33:30 +00:00
Gil Mizrahi
4b33b17f93 chore(ci): show bigquery error details when requested
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8525
GitOrigin-RevId: 882f3ee0dbe7776b15d53a66851b2a57b41fad73
2023-03-29 09:03:47 +00:00
Philip Lykke Carlsen
926d5ecdb0 feat: Handle logical models in metadata when the feature is disabled
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8443
GitOrigin-RevId: 4df0f62979dc78103b786b6f0f8ffcde6def0739
2023-03-28 11:57:11 +00:00
Daniel Harvey
2fb455b813 chore(tests): increase BigQuery test retries
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8522
GitOrigin-RevId: c412a45ceddd18caed7e5b505cfe4163ec9beb88
2023-03-28 11:20:21 +00:00
Daniel Harvey
2d9c8299c2 feature(server): add SQLServer logical models
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8388
GitOrigin-RevId: cbf813d1114cb03816003ba73788d33ac37f1473
2023-03-27 16:56:05 +00:00
Daniel Harvey
305eebb3ab feature(server): add Logical Models support for CockroachDB
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8500
GitOrigin-RevId: 0205bd207843f50ad4fa12ad6e6c07d1f2740912
2023-03-27 14:00:07 +00:00
Antoine Leblanc
dd635fbbc7 Build the first schema cache in the app monad.
### Description

This PR is one more step towards #8344: it splits `initializeContext` into two parts: the first one builds the `AppEnv`, and the second builds the `AppContext` and its corresponding `AppStateRef`. Splitting it allows us to run the first schema cache build _in the app monad_, which was one of the steps required to remove `CacheBuild`, and ultimately to achieve #8344, which in turn is a blocker for several projects.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8445
GitOrigin-RevId: 0ef9eeec0c941b915da505cc9b1c60f36b108a56
2023-03-27 10:27:25 +00:00
Tom Harding
fe140ee791 Remove Fixture.Options argument from api-tests
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8480
GitOrigin-RevId: a6dd5c6c2cc6274b7bd1694cf1100efa48e6bdbd
2023-03-24 13:59:35 +00:00
Antoine Leblanc
068b5e4523 Rename and clean CE app monad.
### Description

This small PR is on top of #8440. It continues the cleanup of `Hasura/App` by grouping together all instances on the app monad, renaming the app monad, and removing its `m` parameter.

See [this commit](cb6ecba3d4) to see the diff in isolation.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8441
GitOrigin-RevId: 7abb6524a160bcb4f75e729e726ce2af69643998
2023-03-23 15:52:48 +00:00
Antoine Leblanc
d7309b811e Partial cleanup of Hasura/App, refactor of GlobalCtx
### Description

As part of another project (the continuation of #8421), i have started a cleanup of `Hasura.App`, focusing on deleting old code and grouping together things that belong together. This quickly grew into a refactor of `GlobalCtx`, now renamed into `BasicConnectionInfo`. This small refactor adds comments, and aims at making clear what the purpose of those types and functions is.

Furthermore, it also changes the way the default postgres connection info is created, by making that part of the process of creating the `BasicConnectionInfo`, to deduplicate similar effort across different files.

This is expected to be a no-op.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8440
GitOrigin-RevId: 412c5b1905f629beb9c6cd262b9798cb31c93bdb
2023-03-22 21:41:53 +00:00