Commit Graph

92 Commits

Author SHA1 Message Date
Daniel Harvey
599b4abdc5 Fix boolean expression type operator mapping (#767)
<!-- The PR description should answer 2 (maybe 3) important questions:
-->

### What

A scalar `BooleanExpressionType` lets us give operators fancy names.
This PR makes them actually work.

Functional no-op as this feature is behind a feature flag.

<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->

<!-- Consider: do we need to add a changelog entry? -->

### How

<!-- How is it trying to accomplish it (what are the implementation
steps)? -->

V3_GIT_ORIGIN_REV_ID: b49ef95d3d6672a1e27371fa5f4df63acd0849fc
2024-06-27 11:23:58 +00:00
Samir Talwar
00db411c6e Remove ModelsGraphqlOutput::graphql_types, as it's not used. (#769)
### What

This removes an unused field. The remaining struct had just one field
and is private to the crate, so I converted it to a type alias.

### How

In order to make `rustc` highlight the issue, I reduced the number of
types we export as `pub`, changing some to `pub(crate)`.

Then I just deleted the field once the warning showed up.

V3_GIT_ORIGIN_REV_ID: 7a26e99f062ed0f2c7449e1f57bc76068f059afb
2024-06-27 09:13:21 +00:00
Samir Talwar
eb1dbffbd1 Use SubgraphIdentifier in the metadata accessor, and SmolStr internally. (#768)
### What

Using `String` everywhere to represent subgraph identifiers is
definitely going to cause problems at some point. We can avoid this by
using `open_dds::identifier::SubgraphIdentifier` instead.

`Identifier` and `SubgraphIdentifier` were wrappers around `String`. I
have also changed them to wrap `SmolStr` instead, for better performance
as they're cloned a lot.

### How

First of all, I hid the innards of `Identifier` and
`SubgraphIdentifier`, instead exposing certain behaviors as methods.
Mostly, this means exposing `as_str()` and `to_string()`.

Then I replaced the internals with `SmolStr`. I had to change one place
where `RefCast` was used, but that was pretty much it.

Finally, I switched out `String` for `SubgraphIdentifier` in
`QualifiedObject`. This necessitated a couple of constants for the two
"magic" subgraphs, `__globals` and `__unknown_namespace`, but was
otherwise fine.

I didn't touch `Qualified` for now.

V3_GIT_ORIGIN_REV_ID: 28664609c3173b181c3789093cb9796896642eb7
2024-06-26 22:04:22 +00:00
Samir Talwar
5d619a540f Replace lazy_static with stdlib equivalents. (#758)
### What

The `lazy_static` macro is poorly maintained, fairly bloated, and has
been mostly superseded by
[`OnceLock`](https://doc.rust-lang.org/stable/std/sync/struct.OnceLock.html)
in the stdlib.

### How

1. I turned a couple of `static ref` values into `const`, sometimes by
creating `const fn` equivalents to other functions.
2. I inlined static behavior to construct a JSON pointer into some
tests, where we don't care too much about losing a few milliseconds.
3. For the rest, I replaced `lazy_static` with a `static OnceLock` and a
call to `OnceLock::get_or_init`.

V3_GIT_ORIGIN_REV_ID: 18e4150a5fb24fe71f6ed77fe6178b7942405aa3
2024-06-26 12:45:41 +00:00
Daniel Chambers
82c0c65bd0 Enable aggregate relationships by default (#765)
NOTE: This PR is stacked on #756 and should be shipped after that is
merged.

This PR enables the existing aggregate relationships work (see #725,
#731, #756) by default by removing the experimental flag it used to be
disabled behind.

The new OpenDD schema changes that were added are also unhidden so that
they are visible in the OpenDD JSON Schema.

V3_GIT_ORIGIN_REV_ID: cfd86d8a9ea61887ccf0f1a5d08bdcc3dda59cdc
2024-06-26 10:49:10 +00:00
Daniel Chambers
0624a7553d Aggregate relationships GraphQL schema and execution (#756)
## Description
This PR implements the GraphQL schema and execution for aggregate
relationships.

In the `schema` crate, the new `model_aggregate_relationship_field`
function handles generating schema for ModelAggregateTarget
relationships. It mostly delegates the meat of its implementation to
reused logic; some refactoring has occurred to make this possible.
This involved changes in `select_many`, `select_aggregate` and
`model_arguments`. The creation of the model arguments field argument
now exists in `model_arguments` and is reused by `select_many` and
`select_aggregate`. The creation of all aggregate field arguments is now
in `select_aggregate::generate_select_aggregate_arguments`, and is then
reused when generating the aggregate relationship field. That field is
annotated with the new `RelationshipToModelAggregate` annotation.

In the `execute` crate, the logic around generating an the aggregate
selection IR was moved from `select_aggregate` into `model_selection`.
This was so it can be reused by the logic in `relationship` that now
uses it to generate an aggregate selection when encountering an
`RelationshipToModelAggregate` field.
Inside `relationship` some rearranging was done so that
`build_local_model_relationship` and `build_remote_relationship` could
work with either a normal model selection IR or the new aggregate
selection IR. The necessitated moving the creation of that IR outside
those functions into the caller, so the different callers can create
different IR (normal vs aggregate IR). This also reduced code
duplication.

New tests have been added to `engine` that cover aggregate relationships
and also remote joined aggregate relationships.

This PR also corrects two bugs in metadata resolve revealed by new
testing:
* The filter input field name in `GraphqlConfig` must be specified if
using an aggregate relationship
* The filter input type name defined on a `Model` must be specified if
that model is the target of an aggregate relationship. Conversely, the
filter input type name can be specified if the `Model` itself doesn't
define an aggregate, but is still involved in a aggregate relationship
(this previously produced an error).

This PR completes the feature, but it is still hidden behind the
experimental flag. There will be a follow up PR to remove that and
expose the functionality by default.

JIRA: [V3ENGINE-160](https://hasurahq.atlassian.net/browse/V3ENGINE-160)

[V3ENGINE-160]:
https://hasurahq.atlassian.net/browse/V3ENGINE-160?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

V3_GIT_ORIGIN_REV_ID: d499371906f7af71a4017c7c3ae75b7693cd3fa7
2024-06-26 09:10:33 +00:00
Samir Talwar
63ac02bc07 Take snapshots of passing and failing metadata resolution. (#763)
### What

In order to more easily monitor and review changes to metadata
resolution, this introduces snapshot testing for both successful and
failing calls to `resolve`. I used [Insta](https://insta.rs/) for this.

### How

For tests of the failure case, we already had a text file with the
expected error, so I have turned those files into snapshot files. I
wrote a small script to move the files rather than deleting and
recreating them so I could guarantee that the contents have not changed.
(Unfortunately, Git's diff doesn't always recognise the move as a move
because Insta has added a header.)

For tests of the successful case, I added a line to snapshot the
metadata rather than discarding it.

I also rewrote the tests to use `insta::glob` so we could get rid of
`test_each`.

V3_GIT_ORIGIN_REV_ID: 41bef4cf77bddb8d20d7c101df52ae149e8b0476
2024-06-26 08:15:29 +00:00
Samir Talwar
8baa50010e Make the internal metadata-resolve flags copyable. (#762)
### What

This just means we can drop a bunch of references and pass by value.

### How

`#[derive(Clone, Copy)]` and fixing lints.

V3_GIT_ORIGIN_REV_ID: e15d323f8232755294d1f7a2c70ccf0de8a1632f
2024-06-26 02:27:41 +00:00
Vamshi Surabhi
8af78227a0 Experimental SQL interface (#742)
Adds a very experimental SQL interface to v3-engine for GenAI use cases.

---------

Co-authored-by: Abhinav Gupta <127770473+abhinav-hasura@users.noreply.github.com>
Co-authored-by: Gil Mizrahi <gil@gilmi.net>
Co-authored-by: Anon Ray <ecthiender@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: 077779ec4e7843abdffdac1ed6aa655210649b93
2024-06-25 18:46:39 +00:00
Daniel Harvey
cedd4e56c6 Use new boolean expressions in arguments (#753)
<!-- Thank you for submitting this PR! :) -->

## Description

Adding a test for this and fixing a few missing parts. Mostly threading
`boolean_expression_types` everywhere and adding them to our type
lookups. Behind a feature flag so this is a functional no-op.

V3_GIT_ORIGIN_REV_ID: 5fd6d5b9e06f0216e770b0715c59c0479881017f
2024-06-25 14:35:07 +00:00
Daniel Harvey
30ea20f055 Check field type against boolean expression type (#754)
<!-- Thank you for submitting this PR! :) -->

## Description

Previously we didn't check whether a boolean expression over strings was
used on a `String` field or a `User` field. Now we look up the types and
actually find out.

Functional no-op as behind a feature flag.

V3_GIT_ORIGIN_REV_ID: 8b7e94c4b873c49e206caa84e24c0d17c049c899
2024-06-25 11:13:27 +00:00
Rakesh Emmadi
e8ec700d70 Improvements to query usage analytics data shape (#715)
This PR introduces the following changes to query usage analytics data
shape:
- The `name` field in `RelationshipUsage` is just `RelationshipName`
without `Qualified` wrapper. The `source` is already qualified, and the
same qualification applies to `name`.
- The `used` for both field and input field is a list. A field can use
multiple opendd objects at a time.
- Example: A root field can use `Model` and `Permission` (with both
filter and argument presets).
- The permission usage now revamped to express available permissions in
the opendd
  - Filter predicate - provides lists of fields and relationships
  - Field presets - provides a list of fields involved
  - Argument presets - provides a list of arguments involved
- The `GqlFieldArgument` is dropped in favor of `GqlInputField`.
- Opendd object usage is not specified for `GqlFieldArgument`. An input
argument with object type can have field presets permission. It is
replaced with `GqlInputField` to allow specifying the permission usage.

This PR also includes JSON schema for the data shape with a golden test
to verify.
V3_GIT_ORIGIN_REV_ID: f0bf9ba201471af367ef5027bc2c8b9f915994ac
2024-06-25 07:57:15 +00:00
Anon Ray
d96bb22844 forward headers from commands as response headers (#707)
## Description

According to NDC headers pass-through spec, commands can include headers
in their responses, which are forwarded as response headers by the
engine to the client. This PR implements it.

V3_GIT_ORIGIN_REV_ID: 4fe458db02c5dd51f4674e4e013312f8e179c087
2024-06-25 07:27:05 +00:00
Anon Ray
3b033331d5 remove redundant Result type in get_underlying_named_type (#755)
## Description

Remove redundant `Result` type in
`ndc_validation::get_underlying_named_type`.

Functional no-op.

V3_GIT_ORIGIN_REV_ID: 0addf72d7f32e290c4687fbf7da4ffae74943fbf
2024-06-25 07:07:57 +00:00
Samir Talwar
1378730d43 Remove redundant clones. (#752)
I noticed a few extra calls to `.clone()` while working on an unrelated
refactor. I want to remove them for brevity and simplicity; I don't
expect a performance improvement.

This turns on the Clippy warning `redundant_clone`, which detects
unnecessary calls to `.clone()` (and `.to_string()`).

It is an unstable warning and so might reports some false positives. If
we find any, we can suppress the warning there.

V3_GIT_ORIGIN_REV_ID: a713f29cf862d6f4cb40300105c6b9f96df00676
2024-06-24 14:09:02 +00:00
Daniel Harvey
e43c6b5421 Ensure boolean expression relationship is local (#750)
<!-- Thank you for submitting this PR! :) -->

## Description

When using boolean expression types on models, we have to check that any
relationships they define are local, as we currently do not support
remote predicates. This adds these checks in the `models_graphql` stage,
once we know about a) models and their sources b) boolean expressions c)
relationships.

Behind a feature flag, so strictly a no-op.

V3_GIT_ORIGIN_REV_ID: 70b2e4b316f5b8d57fa06d5492cccdddca0aaf1c
2024-06-24 12:13:02 +00:00
Daniel Harvey
c64baf6418 Move models_graphql later in the resolve pipeline (#748)
<!-- Thank you for submitting this PR! :) -->

## Description

We need to run this step after the `relationships` stage, so we can
lookup object relationships when validating boolean expression
relationships.

We move aggregation checks back to `models`, as they are also used in
the `relationships` step and we don't want circular dep hell.

Functional no-op.

V3_GIT_ORIGIN_REV_ID: 20fd777b0bc68c2bffc6febb6389a1a955ab2f16
2024-06-24 10:22:33 +00:00
Daniel Harvey
995339bbf6 Split models resolve stage (#741)
<!-- Thank you for submitting this PR! :) -->

## Description

To check filter expression types properly, we need the resolved `source`
for all models. Previously we did not have this, as we did all model
resolving in one go. This splits that into stages, the existing `models`
resolves models and their sources, and then `models_graphql` resolves a
Model's filter expressions, aggregations and graphql schema. This means
we can inspect the model sources during the `models_graphql` in the
actual PR we want to do here.

It's massive, but it's a functional no-op, I promise.

V3_GIT_ORIGIN_REV_ID: 6c8fbf0763fb1daa209a4c84855609e8e9372e8b
2024-06-24 02:46:25 +00:00
Daniel Chambers
acf357c687 Refactor object_type_fields and RelationshipTarget (#737)
## Description
This PR refactors the `output_type::object_type_fields` function, which
is a monster, and splits out the handling of relationship fields into a
separate function. This function then itself is broken down into
separate functions that each handle one of the different types of
relationship fields.

This is particularly necessary now, as logic to handle
ModelAggregateRelationshipTargets is about to get added and that would
only grow the monster function more.

The `RelationshipTarget` `metadata_resolve` type has had its enum
variant structs broken out into separate struct types so that they can
be passed around separate to the enum. This makes it easier to write
functions that deal with each enum variant separately (like
`output_type::object_type_fields`).

This PR causes no behavioural changes.

V3_GIT_ORIGIN_REV_ID: 4707f8b07154bd4d503f78392a3d2d8976eab339
2024-06-21 12:09:41 +00:00
Daniel Harvey
99954495d5 Obey is_null and logical_operators (#733)
<!-- Thank you for submitting this PR! :) -->

## Description

Previously we ignored `isNull` and `logicalOperators`. Now we obey
`isNull` for scalars, and `logicalOperators` for objects.

Both `isNull` on objects and `logicalOperators` on scalars don't exist
as features, so will create follow up tickets for these.

No new tests, but I was able to make both things fail by disabling the
appropriate options:

<img width="655" alt="Screenshot 2024-06-19 at 16 59 33"
src="https://github.com/hasura/v3-engine/assets/4729125/47eb342d-49c1-4ec8-8bfa-5e3f2d286928">

<img width="758" alt="Screenshot 2024-06-19 at 16 30 08"
src="https://github.com/hasura/v3-engine/assets/4729125/5722407c-e12e-429a-ab3a-fc221a8addfa">

Forgive me, I also broke up the `stages/boolean_expressions/object`
module into a few pieces to make this easier to think about.

This is behind a feature flag, no op.

V3_GIT_ORIGIN_REV_ID: b1273d8a9655a21adedc7c7a04a96e0cfe98cc98
2024-06-21 10:01:34 +00:00
Daniel Chambers
b1ee3ae1b8 Metadata resolve for aggregates over relationships (#731)
## Description
This PR continues on from #725 and adds the metadata resolve logic to
validate the usage of aggregates applied to relationships. The metadata
resolve logic is gated behind a new `enable_aggregate_relationships`
flag and disabled by default.

The new resolve logic lives in
`crates/metadata-resolve/src/stages/relationships/mod.rs`, however, most
of the actual logic that validates the usage of the aggregate expression
with the relationship has been reused from the model aggregate resolve
code. Subsequently, that code
(`crates/metadata-resolve/src/stages/models/aggregation.rs`) was
refactored to return a distinct error type
`ModelAggregateExpressionError` that can be composed with the
`RelationshipError` type, so the same errors can be returned with the
additional context of the relationship they were found in.

New metadata resolve error tests have been added in
`crates/metadata-resolve/tests/failing/aggregate_expression_in_relationship/*`.
Also, two new passing metadata resolve tests have been added to cover
the happy case of root field and relationship aggregate expressions:
`crates/metadata-resolve/tests/passing/aggregate_expressions/*`

JIRA: [V3ENGINE-160](https://hasurahq.atlassian.net/browse/V3ENGINE-160)

[V3ENGINE-160]:
https://hasurahq.atlassian.net/browse/V3ENGINE-160?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

V3_GIT_ORIGIN_REV_ID: 8cd7040cc9277940641d5ac239d7b34f8c1462c5
2024-06-21 01:08:54 +00:00
Samir Talwar
783aec942d Move all dependency versions into the workspace file. (#734)
This keeps versions in one place so we can more easily ensure we upgrade
crates together.

V3_GIT_ORIGIN_REV_ID: 6a929bb6196c19a1f66a768585b669127035e9be
2024-06-20 12:25:55 +00:00
Daniel Harvey
1717e9ddaf Validate BooleanExpressionTypes when they are used by models (#730)
<!-- Thank you for submitting this PR! :) -->

## Description

We cannot validate `BooleanExpressionType`s much until they are used.
This adds checks for when they are used as model `where` clauses. We
ensure the data connector in question is allowed to use nested filtering
(if we try and do any), and ensure that every scalar field has mappings
for the appropriate data connector.

Hidden behind a feature flag, so technically a no-op.

V3_GIT_ORIGIN_REV_ID: ab54f913157954e8197798febafc70012bfad9ad
2024-06-19 11:14:24 +00:00
paritosh-08
2da4637baa add field arguments in schema and execution (#695)
### TODO
- [x] Validate the presence of arguments in the data connector schema
- [x] Add test for IR
- [x] Add test using custom connector

## Description

JIRA: https://hasurahq.atlassian.net/browse/V3ENGINE-148

This is part 2 of the field arguments.

This will add the field arguments to the graphql schema and also handles
the execution part.

![image](https://github.com/hasura/v3-engine/assets/85472423/c8f55d70-6539-42ef-ab5a-91b74e5699e2)

## Changelog

- Add a changelog entry (in the "Changelog entry" section below) if the
changes
  in this PR have any user-facing impact. See
[changelog
guide](https://github.com/hasura/graphql-engine-mono/wiki/Changelog-Guide).
- If no changelog is required ignore/remove this section and add a
  `no-changelog-required` label to the PR.

### Product

_(Select all products this will be available in)_

- [x] community-edition
- [x] cloud
<!-- product : end : DO NOT REMOVE -->

### Type

<!-- See changelog structure:
https://github.com/hasura/graphql-engine-mono/wiki/Changelog-Guide#structure-of-our-changelog
-->

_(Select only one. In case of multiple, choose the most appropriate)_

- [ ] highlight
- [x] enhancement
- [ ] bugfix
- [ ] behaviour-change
- [ ] performance-enhancement
- [ ] security-fix
<!-- type : end : DO NOT REMOVE -->

### Changelog entry

<!--
  - Add a user understandable changelog entry
- Include all details needed to understand the change. Try including
links to docs or issues if relevant
  - For Highlights start with a H4 heading (#### <entry title>)
  - Get the changelog entry reviewed by your team
-->

Add support for field arguments

<!-- changelog-entry : end : DO NOT REMOVE -->

<!-- changelog : end : DO NOT REMOVE -->

---------

Co-authored-by: Anon Ray <ecthiender@users.noreply.github.com>
Co-authored-by: Samir Talwar <samir.talwar@hasura.io>
V3_GIT_ORIGIN_REV_ID: ea06b896feaca30419f81cd070fd4cfd608ef35b
2024-06-19 08:15:09 +00:00
Daniel Harvey
cf17d0a414 Tidy models metadata-resolve (#729)
<!-- Thank you for submitting this PR! :) -->

## Description

Doing some work on the `models` section and it's a bit big and
confusing, so doing a tidy-up to get my head around it all. Splits the
large module into separate files (`ordering`, `source`, `graphql`,
`filter` etc).

Only actual changes are:
- Move filter resolving to after model source is checked, so we can
refer to it. This will be useful for the thing I actually want to do.
- Make steps like `graphql` and `source` return the values they create
rather than mutating `Model` directly.
- Move some filter checks into `filter` rather than `source`, now we are
able to. This logic was all over the place.

Functional no-op.

V3_GIT_ORIGIN_REV_ID: 540db16d34590ed9e481c36d251ab5e1886f2c81
2024-06-18 14:13:02 +00:00
Daniel Harvey
e14908e083 Use file globs in metadata-resolve tests (#728)
<!-- Thank you for submitting this PR! :) -->

## Description

Because we globbed for folders, we had to put a test in the root of each
folder which is messy. If we glob for files instead, like the `failing`
tests do, we can have a better time. Functional no-op.

V3_GIT_ORIGIN_REV_ID: 381f5a44d9ae6100198bd28a286493bc3a2ba98e
2024-06-18 12:19:07 +00:00
Samir Talwar
5ad33bd6a5 Avoid returning a Result when it is unnecessary. (#726)
Return a `T` instead of a `Result<T, E>` when we never return an error
(`E`) case.

I also enabled some more warnings. `unnecessary_box_returns` has been
suppressed where appropriate, and `unused_async` doesn't seem to be
violated anywhere any more.

I got rid of some calls to `.unwrap()` too.

V3_GIT_ORIGIN_REV_ID: 015ebd05978cf8c2d87474a90e0cd4333779a761
2024-06-18 12:03:04 +00:00
Daniel Harvey
7968975c95 Comparable relationships in BooleanExpressionTypes (#717)
<!-- Thank you for submitting this PR! :) -->

## Description

This implements local relationships in the new `BooleanExpressionType`.
We copy all the existing tests for this and reimplement them using the
new types.

Things currently missing:
- a `BooleanExpressionType` can be specified to overwrite the target
model's `where` clause when defining. Currently we ignore this, there is
a follow up ticket for this:
https://hasurahq.atlassian.net/browse/V3ENGINE-209

This work is behind a feature flag, so this is a no-op for the user.

V3_GIT_ORIGIN_REV_ID: 6ae87e1ffb2ad0ebd27c6097e8ea87aca1a4c299
2024-06-18 10:25:04 +00:00
dependabot[bot]
9cf56ca622 Bump serde_json from 1.0.116 to 1.0.117 (#722)
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.116 to
1.0.117.

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: 274e7a3dc975113e95ff3b10ce6b73385f0b6ecc
2024-06-17 05:13:47 +00:00
Daniel Harvey
8dae5ae742 Add example boolean expression type metadata for an Elasticsearch range (#699)
<!-- Thank you for submitting this PR! :) -->

## Description

This PR adds a test with example metadata for how to map to something
like an Elasticsearch [Range
query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-range-query.html).
Functional no-op.

V3_GIT_ORIGIN_REV_ID: 727e1706aab75957214b8cb13876eb23b843ba29
2024-06-13 08:37:32 +00:00
Daniel Chambers
962ae79e99 Move AggregateExpression resolve tests from engine tests to metadata-resolve tests (#714)
## Description
This PR moves all the tests that were added to
`crates/engine/tests/validate_metadata_artifacts/aggregate_expressions`
to the metadata-resolve `metadata_golden_tests`.

All the `error.txt` files got renamed to `expected_error.txt` and the
`metadata is not consistent:` bit on the front of the error got trimmed
off.

I had to change the way that `test_failing_metadata` works so that
instead of looking for directories under `failing/` it looks for
`metadata.json` files under `failing/`. This is because not every
directory has a test in it, some directories are just used for grouping
(eg `failing/aggregate_expressions/`). This doesn't change what tests
get run. The only side effect is that every test name has `_metadata`
suffixed to it (the filename). 😢 `test-each` doesn't appear to offer a
way to disable this behaviour.

V3_GIT_ORIGIN_REV_ID: dece7cd3bb4623c51050d12471d2c0990fd69bfd
2024-06-13 08:20:21 +00:00
Samir Talwar
5df3e6fac4 Inline format arguments into the format string when possible. (#703)
V3_GIT_ORIGIN_REV_ID: 50d6a12eefbfcc6b217d226759856e957fac0f4b
2024-06-12 11:25:38 +00:00
Daniel Chambers
81ac867d16 Aggregates Root Field - Part 2: Metadata Resolve (#684)
This is Part 2 in a stacked PR set that delivers aggregate root field
support.
* Part 1: OpenDD: https://github.com/hasura/v3-engine/pull/683
* Part 3: GraphQL API: https://github.com/hasura/v3-engine/pull/685

JIRA: [V3ENGINE-159](https://hasurahq.atlassian.net/browse/V3ENGINE-159)

## Description
This PR implements the metadata resolve phase of the engine and adds
support for resolving `AggregateExpression`s and validates their use
when linked to a `Model`.

The bulk of the changes can be found in:
* `crates/metadata-resolve/src/stages/aggregates/*` - This is where the
`AggregateExpression`s are resolved
* `crates/metadata-resolve/src/stages/models/mod.rs` - This is where we
validate the `AggregateExpression` specified for use in the model is
actually compatible with the model and its data connector

The `ndc-spec` version used has been lifted to the latest version that
adds support for aggregates over nested objects
(https://github.com/hasura/ndc-spec/pull/144). This necessitated changes
in the Custom Connector, but actual functionality to implement
aggregation over nested objects is implemented in Part 3.

There are also some changes in
`crates/metadata-resolve/src/types/subgraph.rs` where the `Display`
trait for the various `Qualified<T>`, `QualifiedTypeReference`, etc
types has been reworked so that they print more cleanly, with the
subgraph being put outside the type syntax, and array types getting
formatted correctly. For example, previous an array of Varchars would
have printed as `Varchar (in subgraph default)!`, now it properly
formats as `[Varchar!]! (in subgraph default)`.
This was necessary to make useful error messages using these types.

A tonne of tests have been added in
`crates/engine/tests/validate_metadata_artifacts/aggregate_expressions`
to test every error condition of the metadata resolve process.

[V3ENGINE-159]:
https://hasurahq.atlassian.net/browse/V3ENGINE-159?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

V3_GIT_ORIGIN_REV_ID: ffd859127a3f1560707f06ef01906c9d1b183d31
2024-06-12 08:30:25 +00:00
Daniel Chambers
e07197e1be Aggregates Root Field - Part 1: OpenDD (#683)
This is Part 1 in a stacked PR set that delivers aggregate root field
support.
* Part 2: Metadata Resolve: https://github.com/hasura/v3-engine/pull/684
* Part 3: GraphQL API: https://github.com/hasura/v3-engine/pull/685

JIRA: [V3ENGINE-159](https://hasurahq.atlassian.net/browse/V3ENGINE-159)

## Description
This PR is the first cut in implementing aggregates functionality for
v3, as defined in the [Aggregates and Grouping
RFC](https://github.com/hasura/v3-engine/blob/main/rfcs/aggregations.md).

This PR modifies the Open DD schema and adds the `AggregateExpression`
metadata object. It also modifies the `Model` and `GraphqlConfig` types
and adds new fields that are required to link in aggregates
functionality. For more information, please see the below RFC links.

This PR implements only the minimum properties required to get a
aggregates root fields in the GraphQL API. There is more defined in the
RFC that will be implemented in later iterations.

RFC links:
*
[AggregateExpression](https://github.com/hasura/v3-engine/blob/main/rfcs/aggregations.md#aggregateexpression-new)
*
[Model](https://github.com/hasura/v3-engine/blob/main/rfcs/aggregations.md#model--object-type)
*
[GraphqlConfig](https://github.com/hasura/v3-engine/blob/main/rfcs/aggregations.md#graphql-config)

## Changelog

### Product

_(Select all products this will be available in)_

- [x] community-edition
- [x] cloud
<!-- product : end : DO NOT REMOVE -->

### Type

- [ ] highlight
- [x] enhancement
- [ ] bugfix
- [ ] behaviour-change
- [ ] performance-enhancement
- [ ] security-fix
<!-- type : end : DO NOT REMOVE -->

### Changelog entry

Support for aggregate GraphQL root fields via the new
AggregateExpression OpenDD type and its use on the Model type

<!-- changelog-entry : end : DO NOT REMOVE -->

<!-- changelog : end : DO NOT REMOVE -->

[V3ENGINE-159]:
https://hasurahq.atlassian.net/browse/V3ENGINE-159?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

V3_GIT_ORIGIN_REV_ID: 6969ee73036a011e3026b58545c0286a65749067
2024-06-12 08:12:33 +00:00
paritosh-08
9f3b8370c1 make FieldMapping enum again (#705)
<!-- Thank you for submitting this PR! :) -->

## Description

<!--
  Questions to consider answering:
  1. What user-facing changes are being made?
2. What are issues related to this PR? (Consider adding `(close
#<issue-no>)` to the PR title)
  3. What is the conceptual design behind this PR?
  4. How can this PR be tested/verified?
  5. Does the PR have limitations?
  6. Does the PR introduce breaking changes?
-->

This PR reverts the changes to `FieldMapping` and introduces the
`argument_mapping` in `ColumnMapping` instead.

V3_GIT_ORIGIN_REV_ID: badca2416b1a1b82d1a596aa60bf7a39b3b6152a
2024-06-12 02:16:36 +00:00
paritosh-08
a3ea579006 add field arguments in open dd metadata (#694)
<!-- Thank you for submitting this PR! :) -->

## Description

<!--
  Questions to consider answering:
  1. What user-facing changes are being made?
2. What are issues related to this PR? (Consider adding `(close
#<issue-no>)` to the PR title)
  3. What is the conceptual design behind this PR?
  4. How can this PR be tested/verified?
  5. Does the PR have limitations?
  6. Does the PR introduce breaking changes?
-->

JIRA: https://hasurahq.atlassian.net/browse/V3ENGINE-148

This is part 1 of supporting field arguments in the engine.
Part 2 is here: https://github.com/hasura/v3-engine/pull/695

It adds the OpenDD types according to the RFC and resolves them.

V3_GIT_ORIGIN_REV_ID: f7736c968489e35c9133f2c6e276d05baf84d4be
2024-06-11 16:31:47 +00:00
Samir Talwar
0c6d6a67d2 Semicolons, everywhere! (#700)
If a function doesn't return a value, terminate with a semicolon.

I also moved `implicit_hasher` and `return_self_not_must_use` to the
"definitely keep disabling this" list, and installed
[Bacon](https://dystroy.org/bacon/) in the Nix shell to make it easier
to run Clippy.

V3_GIT_ORIGIN_REV_ID: ffb17b42d982518aec433a1676dba0a0dd0ad95d
2024-06-11 15:33:32 +00:00
Daniel Harvey
d34c419929 Allow nested metadata-resolve tests (#697)
<!-- Thank you for submitting this PR! :) -->

## Description

We have a set of tests in `metadata-resolve` that uses globs to find
files in either `passing` or `failing` folders and check them. This
allows those files to live in nested folders for tidyness. Functional
no-op.

V3_GIT_ORIGIN_REV_ID: 22d42e986676a110f455c59c3df9e6946c5d996c
2024-06-11 14:11:10 +00:00
Daniel Harvey
75ced29d11 Resolve nested object boolean expressions (#680)
<!-- Thank you for submitting this PR! :) -->

## Description

This adds the ability to describe nested object boolean expressions,
which become `fieldPath` items in the generated
`ndc_models::ComparisonTarget::Column` items. This allows us to describe
filtering a `User` based on some element in their nested `address` field
(like `postcode`, for example).

Like the other `BooleanExpressionType` work, this remains behind a
feature flag so should make no user-facing changes.

It is also missing a whole heap of metadata resolve checks, going to
follow with these after doing the happy path to unblock other work.

V3_GIT_ORIGIN_REV_ID: c89e2942a651d349fca97706affcf40d91afeefb
2024-06-11 12:57:21 +00:00
dependabot[bot]
05e13fe566 Bump serde from 1.0.202 to 1.0.203 (#687)
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.202 to 1.0.203.

V3_GIT_ORIGIN_REV_ID: 81b23526c9a289d401260350b8169907eb322bbf
2024-06-10 06:26:06 +00:00
Anon Ray
8bc5c01961 Argument presets for DataConnectorLink Pt. 2 (#675)
## Description

This PR implements argument presets for `DataConnectorLink`, which can
be used to forward request headers as function/procedure arguments to
NDC. This PR implements the execution part.

**Note**: response header forwarding is not implemented yet.

V3_GIT_ORIGIN_REV_ID: ff69129aee3e3052367ca42acdec3922cbc2cb0c
2024-06-06 16:04:28 +00:00
Anon Ray
90444710cb Argument presets for DataConnectorLink Pt. 1 (#667)
## Description

This PR implements argument presets and response headers for
`DataConnectorLink`, which can be used to forward request headers as
function/procedure arguments to NDC, and also forward response headers
from NDC back to the client. This PR implements only the OpenDD changes
and metadata resolving part.

V3_GIT_ORIGIN_REV_ID: 2b7f20af7c2f192c0103553bccb000114dcba916
2024-06-06 14:55:09 +00:00
Daniel Harvey
58f939e605 Use OperatorMappings in boolean expressions (#674)
Previously we required an operator called `_eq` to be called `_eq` in
all data connectors that used it.

Now we can give them nice names, and map each one to a different
operator name per data connector.

V3_GIT_ORIGIN_REV_ID: 1d2f678e25d932e5ea45d34999cf5709ef0038b4
2024-06-06 11:31:36 +00:00
Rakesh Emmadi
49e0c79d9e avoid infinite recursion in building annotations from input types (#676)
## Description
JIRA: https://hasurahq.atlassian.net/browse/V3ENGINE-180

Fix an edge case where the function
`build_annotations_from_input_object_type_permissions` produce infinite
recursions
when an input object type contains fields with self-referential types.

Note: This PR doesn't contain any tests. They'll be added in a separate
PR ([JIRA](https://hasurahq.atlassian.net/browse/V3ENGINE-188)). This
fix needs to be done quickly and merged as other teams (graphql
connector) are blocked by this.
V3_GIT_ORIGIN_REV_ID: 4cc09cb96871804d4daf83d7e49a6f4529c1943d
2024-06-06 08:16:33 +00:00
Daniel Harvey
2cd613077a Make very happy path BooleanExpressionType work (#673)
<!-- Thank you for submitting this PR! :) -->

## Description

This makes the absolute happiest path `BooleanExpressionType` work.

Caveats:

- We ignore operator mappings, so we assume that the OpenDD operator
name (ie, `_eq`) is the same as the data connector's name for it.

- We have not tried to implement `comparisonRelationships` in any way
yet.

- Have not tried nested objects yet.

All functionality here is behind a feature flag so has no user-facing
impact.

V3_GIT_ORIGIN_REV_ID: 5edb706c6d8b03b9fa59433ce24f05f37a69729a
2024-06-05 16:18:28 +00:00
Anon Ray
52cfca7f91 fix: ndc validation of command (#669)
## Description

There is a bug in NDC validation of command where it returns early
without validating. This is because we perform the NDC validation right
after resolving the command source, but haven't attached the "resolved
command source" to the command yet. As a result the command's source is
`None`, and the validation does an early return.

This PR fixes it by taking the command source directly. If there is no
source to resolve, we don't have to perform any NDC validation of the
command.

Note: this is not the case with models. Model's source resolving
attaches the "resolved source", then performs the validation.

## Changelog

- Add a changelog entry (in the "Changelog entry" section below) if the
changes
  in this PR have any user-facing impact. See
[changelog
guide](https://github.com/hasura/graphql-engine-mono/wiki/Changelog-Guide).
- If no changelog is required ignore/remove this section and add a
  `no-changelog-required` label to the PR.

### Product

_(Select all products this will be available in)_

- [x] community-edition
- [x] cloud
<!-- product : end : DO NOT REMOVE -->

### Type

<!-- See changelog structure:
https://github.com/hasura/graphql-engine-mono/wiki/Changelog-Guide#structure-of-our-changelog
-->

_(Select only one. In case of multiple, choose the most appropriate)_

- [ ] highlight
- [ ] enhancement
- [x] bugfix
- [ ] behaviour-change
- [ ] performance-enhancement
- [ ] security-fix
<!-- type : end : DO NOT REMOVE -->

### Changelog entry

<!--
  - Add a user understandable changelog entry
- Include all details needed to understand the change. Try including
links to docs or issues if relevant
  - For Highlights start with a H4 heading (#### <entry title>)
  - Get the changelog entry reviewed by your team
-->

Fix skipping of NDC validation of commands due to wrong assumption.

<!-- changelog-entry : end : DO NOT REMOVE -->

<!-- changelog : end : DO NOT REMOVE -->

V3_GIT_ORIGIN_REV_ID: 5b9f4c922ee83394f1b6fe840cd934b6e138e2e9
2024-06-05 15:40:31 +00:00
Daniel Harvey
b314d89ff0 Use newtypes for operator names (#672)
<!-- Thank you for submitting this PR! :) -->

## Description

We were using `String` for these, and making some assumptions about NDCs
and OpenDD operator names being the same. This uses newtypes and
documents where these assumptions are being made. Functional no-op.

V3_GIT_ORIGIN_REV_ID: 195e55db6fadf48c956684130e23f647eaa17fb1
2024-06-05 14:46:05 +00:00
Samir Talwar
ca830c05fb Be more explicit about clones, and remove a few. (#671)
Calling `.to_owned()` on a reference, `.to_vec()` on a vector reference,
etc. are just synonyms for `.clone()` which are less explicit about
cloning. Let's be explicit.

This also removes some unnecessary clones.

V3_GIT_ORIGIN_REV_ID: 1bc00c4106f0346303d73e4268c89030c0ce93fc
2024-06-05 13:26:33 +00:00
Anon Ray
f62f95f30c gardening: tidy up data connector resolve types (#668)
## Description

Tiny PR to add some comments and clarifications to the data connector
resolve types.

Also, rearranges the types and functions to make the flow clearer.

V3_GIT_ORIGIN_REV_ID: db5b18cbd7c79670c9dfc68c80950ccf5105f9ce
2024-06-05 10:34:57 +00:00
Anon Ray
8064c292b5 upgrade to ndc v0.1.3 (#629)
Upgrade engine to ndc v0.1.3.

Also updates custom connector.

---------

Co-authored-by: Samir Talwar <samir.talwar@hasura.io>
Co-authored-by: Gil Mizrahi <gil@hasura.io>
V3_GIT_ORIGIN_REV_ID: 80959d4702ef785b7809d6432e092b6477ef88c1
2024-06-05 10:16:11 +00:00