The simplification will allow us to avoid a few `MonadError QErr m` constsraints in the future - this effect can be created locally instead of reusing a global one.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8729
GitOrigin-RevId: 851e28b1f5bfe4c47da43fa324714a941ef25c57
## Description
This change adds support for nested object fields in HGE IR and Schema Cache, the Data Connectors backend and API, and the MongoDB agent.
### Data Connector API changes
- The `/schema` endpoint response now includes an optional set of GraphQL type definitions. Table column types can refer to these definitions by name.
- Queries can now include a new field type `object` which contains a column name and a nested query. This allows querying into a nested object within a field.
### MongoDB agent changes
- Add support for querying into nested documents using the new `object` field type.
### HGE changes
- The `Backend` type class has a new type family `XNestedObjects b` which controls whether or not a backend supports querying into nested objects. This is currently enabled only for the `DataConnector` backend.
- For backends that support nested objects, the `FieldInfo` type gets a new constructor `FINestedObject`, and the `AnnFieldG` type gets a new constructor `AFNestedObject`.
- If the DC `/schema` endpoint returns any custom GraphQL type definitions they are stored in the `TableInfo` for each table in the source.
- During schema cache building, the function `addNonColumnFields` will check whether any column types match custom GraphQL object types stored in the `TableInfo`. If so, they are converted into `FINestedObject` instead of `FIColumn` in the `FieldInfoMap`.
- When building the `FieldParser`s from `FieldInfo` (function `fieldSelection`) any `FINestedObject` fields are converted into nested object parsers returning `AFNestedObject`.
- The `DataConnector` query planner converts `AFNestedObject` fields into `object` field types in the query sent to the agent.
## Limitations
### HGE not yet implemented:
- Support for nested arrays
- Support for nested objects/arrays in mutations
- Support for nested objects/arrays in order-by
- Support for filters (`where`) in nested objects/arrays
- Support for adding custom GraphQL types via track table metadata API
- Support for interface and union types
- Tests for nested objects
### Mongo agent not yet implemented:
- Generate nested object types from validation schema
- Support for aggregates
- Support for order-by
- Configure agent port
- Build agent in CI
- Agent tests for nested objects and MongoDB agent
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7844
GitOrigin-RevId: aec9ec1e4216293286a68f9b1af6f3f5317db423
### Description
(This PR is better reviewed commit by commit.)
This PR is an aggregation of small incremental changes to Pro's init:
- it deletes some dead code,
- it starts reorganizing the code of that file by sections, similar to OSS' init,
- it extracts and cleans up license key cache init (groups several blocks of code in one separate function)
- makes some changes to a service class to reduce the dependency on `_acAppStateRef`
This PR is a first step: our goal is to move the schema cache build _in_ the app monad, in order to achieve #8344. To do so, we will need to remove `_acAppStateRef` from Pro's `AppContext`. There are two different paths we can take from here, which is why i cut this PR here:
- the first would be to change the different instances we implement on `AppM` to take as an argument the parts of the schema cache they depend on, rather than reading them from `_acAppStateRef`; as of this PR, `MetricsConfig` is the only such field;
- the second would be to apply the same strategy we already used for the TLSAllowList, and use a `IORef` that can be updated after the schema cache is built; this change would have a smaller footprint, but introduces one new `IORef` per such field, which feels something we don't want to generalize
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8655
GitOrigin-RevId: 809697d460bdb5c83ef7d30a2e835f589bcd80a6
### Description
If i understand correctly, the _value_ of a feature flag **can** change at runtime. Since this has an impact on how the schema cache is built, this PR makes the value of the flag a part of the cache's "dynamic" config, therefore making it an arrow-ish argument for the incremental framework. It also removes `CheckFeatureFlag` from the static config, now obsolete.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8631
GitOrigin-RevId: 2ca898e4193e8552c40d2b21a819d3dd414601fe
### Description
This PR removes `ServerConfigCtx` and `HasServerConfigCtx`. Instead, it favours different approaches:
- when the code was only using one field, it passes that field explicitly (usually `SQLGenCtx` or `CheckFeatureFlag`)
- when the code was using several fields, but in only one function, it inlines
- for the cache build, it introduces `CacheStaticConfig` and `CacheDynamicConfig`, which are subsets of `AppEnv` and `AppContext` respectively
The main goal of this is to help with the modularization of the engine: as `ServerConfigCtx` had fields whose types were imported from several unrelated parts of the engine, using it tied together parts of the engine that should not be aware of one another (such as tying together `Hasura.LogicalModel` and `Hasura.GraphQL.Schema`).
The bulk of this PR is a change to the cache build, as a follow up to #8509: instead of giving the entire `ServerConfigCtx` as a incremental rule argument, we only give the new `CacheDynamicConfig` struct, which has fewer fields. The other required fields, that were coming from the `AppEnv`, are now given via the `HasCacheStaticConfig` constraint, which is a "subset" of `HasAppEnv`.
(Some further work could include moving `StringifyNumbers` out of `GraphQL.Schema.Options`, given how it is used all across the codebase, including in `RQL.DML`.)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8513
GitOrigin-RevId: 818cbcd71494e3cd946b06adbb02ca328a8a298e
- Remove `MonadMetadataStorageQueryAPI` which was only implemented by a default implementation
- Introduce `TransT` which can be used to easily derive `lift`ing implementations for `MonadBlaBlaBla` classes
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8579
GitOrigin-RevId: 4f804fda7e2de5c9d75ee4df269f500ebd46b8c9
Also add a `default` implementation for `MonadQueryTags`.
This avoids a bunch of imports on `Hasura.GraphQL.Execute.Backend` which is a big module with lots of (transitive) dependencies.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8571
GitOrigin-RevId: 8ecca452721b77953e6d088c79d8d6f003f2996f
This PR reverts the following two commits:
1. https://github.com/hasura/graphql-engine-mono/pull/8287
2. https://github.com/hasura/graphql-engine-mono/pull/8467
We are undoing a migration that was done on `hdb_catalog.event_log` table which was done in d4ae6a517da63f2f43567dc16fda135b3cd1d7e6 . And as such, users who were using event triggers on that version will come across the error:
```json
{"detail":{"info":{"code":"not-supported","error":"Expected source catalog version <= 3, but the current version is 4","path":"$"},"kind":"catalog_migrate"},"level":"error","timestamp":"2023-03-28T10:17:24.289+0530","type":"startup"}
{"code":"not-supported","error":"Expected source catalog version <= 3, but the current version is 4","path":"$"}
```
To fix these errors please run the following SQL on the source where event triggers were created on:
```
UPDATE hdb_catalog.hdb_source_catalog_version SET version = 3, upgraded_on= NOW();
ALTER table hdb_catalog.event_log ALTER COLUMN created_at SET DEFAULT NOW();
ALTER table hdb_catalog.event_invocation_logs ALTER COLUMN created_at SET DEFAULT NOW();
```
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8534
GitOrigin-RevId: b6bbcce0163c8beed80619d3cea056e643b8c180
## Description
This PR is a incremental step towards achieving the goal of #8344. It is a less ambitious version of #8484.
This PR removes all references to `HasServerConfigCtx` from the cache build and removes `ServerConfigCtx` from `CacheBuildParams`, making `ServerConfigCtx` an argument being passed around manually instead. This has several benefits: by making it an arrow argument, we now properly integrate the fields that change over time in the dependency framework, as they should be, and we can clean up some of the top-level app code.
## Implementation
In practice, this PR introduces a `HasServerConfigCtx` instance for `CacheRWT`, the monad we use to build the cache, so we can retrieve the `ServerConfigCtx` in the implementation of `CacheRWM`. This contributes to reducing the amount of `HasServerConfigCtx` in the code: we can remove `SchemaUpdateT` altogether, and we can remove the `HasServerConfigCtx` instance of `Handler`. This makes `HasServerConfigCtx` almost **an implementation detail of the Metadata API**.
This first step is enough to achieve the goal of #8344: we can now build the schema cache in the app monad, since we no longer rely on `HasServerConfigCtx` to build it.
## Drawbacks
This PR does not attempt to remove the use of `ServerConfigCtx` itself in the schema cache build: doing so would make this PR much much bigger. Ideally, to avoid having all the static fields given as arrow-ish arguments to the cache, we could depend on `HasAppEnv` in the cache build, and use `AppContext` as an arrow argument. But making the cache build depend on the full `AppEnv` and `AppContext` creates a lot of circular imports; and since removing `ServerConfigCtx` itself isn't required to achieve #8344, this PR keeps it wholesale and defers cleaning it to a future PR.
A negative consequence of this is that we need an `Eq` instance on `ServerConfigCtx`, and that instance is inelegant.
## Future work
There are several further steps we can take in parallel after this is merged. First, again, we can make a new version of #8344, removing `CacheBuild`, FINALLY. As for `ServerConfigCtx`, we can split it / rename it to make ad-hoc structures. If it turns out that `ServerConfigCtx` is only ever used for the schema cache build, we could split it between `CacheBuildEnv` and `CacheBuildContext`, which will be subsets of `AppEnv` and `AppContext`, avoiding import loops.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8509
GitOrigin-RevId: 01b37cc3fd3490d6b117701e22fc4ac88b62b6b5
## Description
This PR does several different things that happen to overlap; the most important being:
- it removes `RunT`: it was redundant in places where we already had `Handler`, and only used in one other place, `SchemaUpdate`, for which a local `SchemaUpdateT` is more than enough;
- it reduces the number of places where we create a `ServerConfigCtx`, since now `HasServerConfigCtx` can be implemented directly by `SchemaUpdateT` and `Handler` based on the full `AppContext`;
- it drastically reduces the number of arguments we pass around in the app init code, by introducing `HasAppEnv`;
- it simplifies `HandlerCtx` to reduce duplication
In doing so, this changes paves the way towards removing `ServerConfigCtx`, since there are only very few places where we construct it: we can now introduce smaller classes than `HasServerConfigCtx`, that expose only a relevant subset of fields, and implement them where we now implement `HasServerConfigCtx`.
This PR is loosely based on ideas in #8337, that are no longer applicable due to the changes introduced in #8159. A challenge of this PR was the postgres tests, which were running in `PGMetadataStorageAppT CacheBuild` 🙀
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8392
GitOrigin-RevId: b90c1359066d20dbea329c87762ccdd1217b4d69
- Derive a few `instance`s
- Delete some dead code (methods and types)
- Delete some `INLINE` pragmas that are unlikely to have a big effect
- Monomorphize Postgres `LISTEN` code to avoid effect juggling
- Generalize some methods in `pg-client` so that others can be simplified
- Handle errors differently for `TxET` to deduplicate code
- Use `hoist` instead of specialized combinators such as `mapActionT`
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8130
GitOrigin-RevId: bc1e908b6c0869f440a214a76744e92d40fea1e6
This PR is on top of #7789.
### Description
This PR entirely rewrites the API of the Tracing library, to make `interpTraceT` a thing of the past. Before this change, we ran traces by sticking a `TraceT` on top of whatever we were doing. This had several major drawbacks:
- we were carrying a bunch of `TraceT` across the codebase, and the entire codebase had to know about it
- we needed to carry a second class constraint around (`HasReporterM`) to be able to run all of those traces
- we kept having to do stack rewriting with `interpTraceT`, which went from inconvenient to horrible
- we had to declare several behavioral instances on `TraceT m`
This PR rewrite all of `Tracing` using a more conventional model: there is ONE `TraceT` at the bottom of the stack, and there is an associated class constraint `MonadTrace`: any part of the code that happens to satisfy `MonadTrace` is able to create new traces. We NEVER have to do stack rewriting, `interpTraceT` is gone, and `TraceT` and `Reporter` become implementation details that 99% of the code is blissfully unaware of: code that needs to do tracing only needs to declare that the monad in which it operates implements `MonadTrace`.
In doing so, this PR revealed **several bugs in the codebase**: places where we were expecting to trace something, but due to the default instance of `HasReporterM IO` we would actually not do anything. This PR also splits the code of `Tracing` in more byte-sized modules, with the goal of potentially moving to `server/lib` down the line.
### Remaining work
This PR is a draft; what's left to do is:
- [x] make Pro compile; i haven't updated `HasuraPro/Main` yet
- [x] document Tracing by writing a note that explains how to use the library, and the meaning of "reporter", "trace" and "span", as well as the pitfalls
- [x] discuss some of the trade-offs in the implementation, which is why i'm opening this PR already despite it not fully building yet
- [x] it depends on #7789 being merged first
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7791
GitOrigin-RevId: cadd32d039134c93ddbf364599a2f4dd988adea8
