Commit Graph

34 Commits

Author SHA1 Message Date
Daniel Chambers
e44589931c Reimplement most OpenDD newtypes using SmolStr and a macro (#797)
This PR replaces most of the string newtypes in the open-dds crate with
new implementations based on `SmolStr`, using a macro adapted from the
macro used in ndc-models. `SmolStr` usage should result in less heap
allocations and the macro ensures all the newtypes get all the same
trait implementations (such as various `From` and `Borrow` instances)
and helper impl functions (such as `as_str`).

The new macro, `str_newtype`, creates a newtype wrapper around `SmolStr`
or another type (typically `Identifier`), and writes out all the various
trait implementations. It also takes a documentation string which
ensures that every newtype has a description in JSON Schema.

The changes in the downstream crates are just adjusting to use the new
newtypes.

Also:
* `QueryGraphqlConfig` used a lot of String typed-properties; these have
been changed for the appropriate `GraphQlTypeName` and
`GraphQlFieldName` types.
* metadata-resolve had a duplicate newtype called
`ConnectorArgumentName`, which duplicated open-dds's
`DataConnectorArgumentName`. This has been removed and replaced.
* A new newtype called `CollectionName` has been added in open-dds to
represent the name of the ndc collection that a model points to. This
was previously just a string.
V3_GIT_ORIGIN_REV_ID: d5a33756ef0e110b2255478358a38e42e6ff89a2
2024-07-05 10:16:33 +00:00
Daniel Chambers
862ad8e0dc Update ndc-models to latest version which includes new name newtypes (#791)
This PR updated ndc-models to the latest version on main. This version
is still a 0.1.x version, but it now includes all the [new
newtypes](https://github.com/hasura/ndc-spec/pull/156) that wrap
previously stringly-typed things. For example, `ArgumentName`,
`FieldName`, etc.

This pervades across the entire engine, but thankfully the changes are
mostly mechanical repetitive changes. Usually you will see conversions
from `String`-typed variables into the newtypes using this sort of form:
`FieldName::from(string.as_str())`, which is the most efficient way
copying the value (the str slice is copied). Or you will see usages of
the newtype as a raw string by `.as_str()`-ing it. Converting the
newtypes into a String can be done with `.into()` if owned, but if
referenced `.as_str().to_owned()` performs the clone and type
conversion.

Other changes:
* A few minor instances of `ok_or()` usages (or similar) have been
converted into lazy error construction variants (eg `ok_or_else()`)

V3_GIT_ORIGIN_REV_ID: 64a371ae6197ef3be98a6f7cdc4052d654a43da0
2024-07-04 08:58:12 +00:00
Daniel Chambers
e380876823 Handle multiple ndc-model versions during query planning and execution (#790)
The PR adds code that handles multiple versions of the ndc-models during
the execution pipeline. Depending on whether a connector supports `v01`
or `v02` models, a different set of types is used to send and receive
the http requests.

However, the engine internally still uses the latest (v02) models inside
its IR. Unfortunately, it was going to be quite traumatic to prevent the
engine from using ndc models inside the IR and during response
processing and remote joins. This means that the engine generates v02
requests, and these are downgraded into v01 requests. v01 responses are
upgraded to v02 responses before being processed by the engine.

The ndc client (`execute::ndc::client`) now only takes new wrapper enum
types (`execute::ndc::types`) that split between v01 or v02
requests/responses. Every place that carries an ndc request now carries
this type instead, which allows it to carry either a v01 or a v02
request.

When ndc requests are created during planning, all creation goes via the
new `execute::plan::ndc_request` module. This inspects the connector's
supported version, creates the necessary request, and if needed,
downgrades it to v01.

When ndc responses are read during planning or during remote joins, they
are upgraded to v02 via helper functions defined on the types in
`execute::ndc::types`.

The upgrade/downgrade code is located in `execute::ndc::migration`. Keep
in mind the "v02" types are currently the same as the "v01" types so the
migration code is not doing much. This will change as the v02 types are
modified.

However, this approach has its drawbacks. One is that it prevents
changes to the ndc types [like
this](https://github.com/hasura/ndc-spec/pull/158) without a fair bit of
pain (see
[comment](https://github.com/hasura/ndc-spec/pull/158#issuecomment-2202127094)).
Another is that the downgrade code can fail at runtime and it is not
immediately obvious to developers using new, but unused, v02 features
that their new feature would fail on v01, because that mapping to v01
has already been written. Another is that we're paying some (small,
probably?) performance cost by upgrading/downgrading types because we
need to rebuild data structures.

Also:
* `execute::ndc::response` has been merged into `execute::ndc::client`,
since it was inextricably linked.
V3_GIT_ORIGIN_REV_ID: f3f36736b52058323d789c378fed06af566f39a3
2024-07-03 08:32:37 +00:00
Anon Ray
c1b9592f6b respect relationship comparison capability when generating relationship fields in model filter (#789)
### What

When generating GraphQL schema for relationship fields in model filter,
engine ignores `relation_comparisons` capability of the data connector.
Engine would generate schema for data connectors which don't have this
capability. This PR fixes that.

### How

While generating fields for the filter input type, take the relationship
capabilities into account.

The `ObjectBooleanExpressionType` and `BooleanExpressionType` objects
are quite different, hence their schema generation part is also
different, and is split in two different functions
(`build_comparable_relationships_schema`, and
`build_new_comparable_relationships_schema`). Added checking of
relationship comparison capability in both the functions.

V3_GIT_ORIGIN_REV_ID: dce2b88f7792e01e5bb390ecdb580e223ec80f01
2024-07-02 07:49:10 +00:00
Daniel Harvey
0123c558f1 Enable BooleanExpressionTypes (#783)
<!-- The PR description should answer 2 (maybe 3) important questions:
-->

### What

# BooleanExpressionType

A new metadata kind `BooleanExpressionType` can now be defined. These
can be used in place of `ObjectBooleanExpressionType` and
`DataConnectorScalarRepresentation`, and allow more granular control of
comparison operators and how they are used.

The old metadata types still work, but will eventually be deprecated.

```yaml
kind: BooleanExpressionType
version: v1
definition:
  name: album_bool_exp
  operand:
    object:
      type: Album
      comparableFields:
        - fieldName: AlbumId
          booleanExpressionType: pg_int_comparison_exp
        - fieldName: ArtistId
          booleanExpressionType: pg_int_comparison_exp_with_is_null
        - field: Address
          booleanExpressionType: address_bool_exp
      comparableRelationships:
        - relationshipName: Artist
          booleanExpressionType: artist_bool_exp
  logicalOperators:
    enable: true
  isNull:
    enable: true
  graphql:
    typeName: app_album_bool_exp
```

```yaml
kind: BooleanExpressionType
version: v1
definition:
  name: pg_int_comparison_exp
  operand:
    scalar:
      type: Int
      comparisonOperators:
        - name: equals
          argumentType: String!
        - name: _in
          argumentType: [String!]!
      dataConnectorOperatorMapping:
        - dataConnectorName: postgres_db
          dataConnectorScalarType: String
          operatorMapping:
            equals: _eq
  logicalOperators:
    enable: true
  isNull:
    enable: true
  graphql:
    typeName: app_postgres_int_bool_exp
```

<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->

<!-- Consider: do we need to add a changelog entry? -->

### How

Remove feature flag, unhide JsonSchema items, fix a few missing bits of
JsonSchema the tests didn't warn us about before.

<!-- How is it trying to accomplish it (what are the implementation
steps)? -->

V3_GIT_ORIGIN_REV_ID: dd3055d926fdeb7446cd57085679f2492a4358a1
2024-07-01 15:29:14 +00:00
Rakesh Emmadi
9e98ed2310 Implement query usage analytics (#713)
JIRA: https://hasurahq.atlassian.net/browse/V3ENGINE-176

This PR implements usage analytics
([RFC](https://docs.google.com/document/d/1k8_ZniiejxHhD0SphtEJ2mg6MoJON0dkk0q4b5WZWDE/edit))
for a GraphQL query when a request is made. The analytics data JSON is
attached to the span, whose details are provided below:
Span name - `execute`
Attribute - `internal.query_usage_analytics = <JSON-string>` OR
`internal.query_usage_analytics_error = <String>`

Refs:
- [Link](https://github.com/hasura/v3-engine/pull/606) to the PR, where
data types for usage analytics were added.
- [Link](https://github.com/hasura/v3-engine/pull/715) to the PR, where
some improvements are made to the analytics data types.

Note: Changelog entry is added.
V3_GIT_ORIGIN_REV_ID: f4089f3354b638533ff72fbec72951e883c31100
2024-07-01 12:10:12 +00:00
Daniel Chambers
a72d45d165 Add support for multiple versions of ndc-spec to Open DD and metadata-resolve (#781)
This PR introduces support for multiple versions of the ndc-spec by
adding a new `VersionedSchemaAndCapabilities` enum variant under the
`DataConnectorLink` in OpenDD. This allows the capture of both ndc
v0.1.* and v0.2.* schema and capabilities.

This is achieved by referencing the `ndc-models` crate twice, once for
`v0.1.4` and once for the first commit after `v0.1.4`. That commit was
chosen to avoid actual v0.2.0 breaking changes for now, while we lay in
this multiple version support plumbing. Future PRs will use a newer
commit and adopt the breaking changes where necessary. The
`VersionedSchemaAndCapabilities::V02` variant uses the the v0.2
reference of `ndc-models`.

Then, during metadata resolve, when we resolve the
`DataConnectorContext` from `DataConnectorLink`, we perform a migration
of v0.1 types to v0.2 types and store and use the v0.2 types during
metadata resolve. This migration is performed in the new module
`ndc_migration`. We also record the `NdcVersion` (either `V01` or `V02`)
in the `DataConnectorLink`. The `execute` crate will need to use this to
determine which version to send to the connector at runtime (to be
implemented in a future PR).

The new changes to OpenDD are hidden from the JSON Schema via a new
`UnstableFeatures` flag, and the use of the new variant is gated behind
it in metadata resolve, since we don't yet support it upstream in the
`execute` crate.

V3_GIT_ORIGIN_REV_ID: d6d8a768ea3537c0b5e620799e94d3dd1e529526
2024-06-28 11:57:41 +00:00
David Overton
77ddaf360e Open DD changes for OrderByExpression (#780)
<!-- The PR description should answer 2 (maybe 3) important questions:
-->

### What

Adds new Open DD type `OrderByExpression` and defines `ModelV2` type, as
described in
https://github.com/hasura/v3-engine/blob/main/rfcs/open-dd-expression-type-changes.md.

### How

- Added new types `OrderByExpression`, `ModelV2` and
`ModelGraphQlDefinitionV2` to the `open-dds` crate.
- Added new `UnstableFeatures` flag `enable_order_by_expressions` to
`metadata-resolve`. This is not yet used, as `metadata-resolve` does not
yet use the new Open DD types.

---------

Co-authored-by: Daniel Harvey <danieljamesharvey@gmail.com>
V3_GIT_ORIGIN_REV_ID: e861e88f293f1b380e6346de8490b96cec6f65bb
2024-06-28 11:26:30 +00:00
Daniel Harvey
bd3bc77808 Add scalar boolean expression types to data_connector_scalar_types (#778)
<!-- The PR description should answer 2 (maybe 3) important questions:
-->

### What

Changed tests to use the new `BooleanExpressionType` until something
broke. Fortunately something broke - we need to add the scalar boolean
expression types to the data connector scalar types for various lookups.

<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->

<!-- Consider: do we need to add a changelog entry? -->

### How

Split scalar boolean expression type resolve into own step so it's
available earlier in the pipeline.

Loop through them and add them to the `data_connector_scalar_types`
outputs.

<!-- How is it trying to accomplish it (what are the implementation
steps)? -->

V3_GIT_ORIGIN_REV_ID: dbd8969c3e9e9d8db1d4a34e93aefc34bdf31421
2024-06-28 10:35:44 +00:00
Daniel Harvey
f87561921c Validate boolean expression types when used as arguments (#777)
<!-- The PR description should answer 2 (maybe 3) important questions:
-->

### What

In https://github.com/hasura/v3-engine/pull/750 and
https://github.com/hasura/v3-engine/pull/754 we added a number of checks
for boolean expression types that can be run once we know the data
source they will be used against.

Previously these checks were only used for model `where` clauses, this
pull request moves them into a shared folder and also checks them when
they are used as model or command arguments.

<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->

<!-- Consider: do we need to add a changelog entry? -->

### How

Add a new `arguments` resolve step. It doesn't fit inside the usual
`commands` or `models` step as we need the data sources for both
validated, plus access to all the resolved `relationships` outputs.

<!-- How is it trying to accomplish it (what are the implementation
steps)? -->

V3_GIT_ORIGIN_REV_ID: f713659962e3f20b2c85f287b6c362fb52ffa1ed
2024-06-27 18:44:40 +00:00
Samir Talwar
291d564d82 Support for partial supergraphs in metadata resolution. (#772)
### What

This adds a flag, `--partial-supergraph`, which instructs the metadata
resolver to prune relationships to unknown subgraphs rather than failing
to resolve.

### How

The flag gets passed through as
`metadata_resolve::configuration::Configuration`, and known subgraphs
are now tracked in `MetadataAccessor`. If the flag is set and a
relationship target refers to an unknown subgraph, we return an empty
list of relationships instead of failing.

Some test infrastructure has been added to set configuration flags per
test.

V3_GIT_ORIGIN_REV_ID: 6f0de2442a3bfc7c7a4c48e3dc7296dc1538cd67
2024-06-27 14:17:31 +00:00
Daniel Chambers
82c0c65bd0 Enable aggregate relationships by default (#765)
NOTE: This PR is stacked on #756 and should be shipped after that is
merged.

This PR enables the existing aggregate relationships work (see #725,
#731, #756) by default by removing the experimental flag it used to be
disabled behind.

The new OpenDD schema changes that were added are also unhidden so that
they are visible in the OpenDD JSON Schema.

V3_GIT_ORIGIN_REV_ID: cfd86d8a9ea61887ccf0f1a5d08bdcc3dda59cdc
2024-06-26 10:49:10 +00:00
Daniel Chambers
0624a7553d Aggregate relationships GraphQL schema and execution (#756)
## Description
This PR implements the GraphQL schema and execution for aggregate
relationships.

In the `schema` crate, the new `model_aggregate_relationship_field`
function handles generating schema for ModelAggregateTarget
relationships. It mostly delegates the meat of its implementation to
reused logic; some refactoring has occurred to make this possible.
This involved changes in `select_many`, `select_aggregate` and
`model_arguments`. The creation of the model arguments field argument
now exists in `model_arguments` and is reused by `select_many` and
`select_aggregate`. The creation of all aggregate field arguments is now
in `select_aggregate::generate_select_aggregate_arguments`, and is then
reused when generating the aggregate relationship field. That field is
annotated with the new `RelationshipToModelAggregate` annotation.

In the `execute` crate, the logic around generating an the aggregate
selection IR was moved from `select_aggregate` into `model_selection`.
This was so it can be reused by the logic in `relationship` that now
uses it to generate an aggregate selection when encountering an
`RelationshipToModelAggregate` field.
Inside `relationship` some rearranging was done so that
`build_local_model_relationship` and `build_remote_relationship` could
work with either a normal model selection IR or the new aggregate
selection IR. The necessitated moving the creation of that IR outside
those functions into the caller, so the different callers can create
different IR (normal vs aggregate IR). This also reduced code
duplication.

New tests have been added to `engine` that cover aggregate relationships
and also remote joined aggregate relationships.

This PR also corrects two bugs in metadata resolve revealed by new
testing:
* The filter input field name in `GraphqlConfig` must be specified if
using an aggregate relationship
* The filter input type name defined on a `Model` must be specified if
that model is the target of an aggregate relationship. Conversely, the
filter input type name can be specified if the `Model` itself doesn't
define an aggregate, but is still involved in a aggregate relationship
(this previously produced an error).

This PR completes the feature, but it is still hidden behind the
experimental flag. There will be a follow up PR to remove that and
expose the functionality by default.

JIRA: [V3ENGINE-160](https://hasurahq.atlassian.net/browse/V3ENGINE-160)

[V3ENGINE-160]:
https://hasurahq.atlassian.net/browse/V3ENGINE-160?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

V3_GIT_ORIGIN_REV_ID: d499371906f7af71a4017c7c3ae75b7693cd3fa7
2024-06-26 09:10:33 +00:00
Samir Talwar
63ac02bc07 Take snapshots of passing and failing metadata resolution. (#763)
### What

In order to more easily monitor and review changes to metadata
resolution, this introduces snapshot testing for both successful and
failing calls to `resolve`. I used [Insta](https://insta.rs/) for this.

### How

For tests of the failure case, we already had a text file with the
expected error, so I have turned those files into snapshot files. I
wrote a small script to move the files rather than deleting and
recreating them so I could guarantee that the contents have not changed.
(Unfortunately, Git's diff doesn't always recognise the move as a move
because Insta has added a header.)

For tests of the successful case, I added a line to snapshot the
metadata rather than discarding it.

I also rewrote the tests to use `insta::glob` so we could get rid of
`test_each`.

V3_GIT_ORIGIN_REV_ID: 41bef4cf77bddb8d20d7c101df52ae149e8b0476
2024-06-26 08:15:29 +00:00
Samir Talwar
8baa50010e Make the internal metadata-resolve flags copyable. (#762)
### What

This just means we can drop a bunch of references and pass by value.

### How

`#[derive(Clone, Copy)]` and fixing lints.

V3_GIT_ORIGIN_REV_ID: e15d323f8232755294d1f7a2c70ccf0de8a1632f
2024-06-26 02:27:41 +00:00
Daniel Harvey
cedd4e56c6 Use new boolean expressions in arguments (#753)
<!-- Thank you for submitting this PR! :) -->

## Description

Adding a test for this and fixing a few missing parts. Mostly threading
`boolean_expression_types` everywhere and adding them to our type
lookups. Behind a feature flag so this is a functional no-op.

V3_GIT_ORIGIN_REV_ID: 5fd6d5b9e06f0216e770b0715c59c0479881017f
2024-06-25 14:35:07 +00:00
Daniel Harvey
30ea20f055 Check field type against boolean expression type (#754)
<!-- Thank you for submitting this PR! :) -->

## Description

Previously we didn't check whether a boolean expression over strings was
used on a `String` field or a `User` field. Now we look up the types and
actually find out.

Functional no-op as behind a feature flag.

V3_GIT_ORIGIN_REV_ID: 8b7e94c4b873c49e206caa84e24c0d17c049c899
2024-06-25 11:13:27 +00:00
Daniel Harvey
e43c6b5421 Ensure boolean expression relationship is local (#750)
<!-- Thank you for submitting this PR! :) -->

## Description

When using boolean expression types on models, we have to check that any
relationships they define are local, as we currently do not support
remote predicates. This adds these checks in the `models_graphql` stage,
once we know about a) models and their sources b) boolean expressions c)
relationships.

Behind a feature flag, so strictly a no-op.

V3_GIT_ORIGIN_REV_ID: 70b2e4b316f5b8d57fa06d5492cccdddca0aaf1c
2024-06-24 12:13:02 +00:00
Daniel Chambers
b1ee3ae1b8 Metadata resolve for aggregates over relationships (#731)
## Description
This PR continues on from #725 and adds the metadata resolve logic to
validate the usage of aggregates applied to relationships. The metadata
resolve logic is gated behind a new `enable_aggregate_relationships`
flag and disabled by default.

The new resolve logic lives in
`crates/metadata-resolve/src/stages/relationships/mod.rs`, however, most
of the actual logic that validates the usage of the aggregate expression
with the relationship has been reused from the model aggregate resolve
code. Subsequently, that code
(`crates/metadata-resolve/src/stages/models/aggregation.rs`) was
refactored to return a distinct error type
`ModelAggregateExpressionError` that can be composed with the
`RelationshipError` type, so the same errors can be returned with the
additional context of the relationship they were found in.

New metadata resolve error tests have been added in
`crates/metadata-resolve/tests/failing/aggregate_expression_in_relationship/*`.
Also, two new passing metadata resolve tests have been added to cover
the happy case of root field and relationship aggregate expressions:
`crates/metadata-resolve/tests/passing/aggregate_expressions/*`

JIRA: [V3ENGINE-160](https://hasurahq.atlassian.net/browse/V3ENGINE-160)

[V3ENGINE-160]:
https://hasurahq.atlassian.net/browse/V3ENGINE-160?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

V3_GIT_ORIGIN_REV_ID: 8cd7040cc9277940641d5ac239d7b34f8c1462c5
2024-06-21 01:08:54 +00:00
Daniel Harvey
1717e9ddaf Validate BooleanExpressionTypes when they are used by models (#730)
<!-- Thank you for submitting this PR! :) -->

## Description

We cannot validate `BooleanExpressionType`s much until they are used.
This adds checks for when they are used as model `where` clauses. We
ensure the data connector in question is allowed to use nested filtering
(if we try and do any), and ensure that every scalar field has mappings
for the appropriate data connector.

Hidden behind a feature flag, so technically a no-op.

V3_GIT_ORIGIN_REV_ID: ab54f913157954e8197798febafc70012bfad9ad
2024-06-19 11:14:24 +00:00
Daniel Harvey
e14908e083 Use file globs in metadata-resolve tests (#728)
<!-- Thank you for submitting this PR! :) -->

## Description

Because we globbed for folders, we had to put a test in the root of each
folder which is messy. If we glob for files instead, like the `failing`
tests do, we can have a better time. Functional no-op.

V3_GIT_ORIGIN_REV_ID: 381f5a44d9ae6100198bd28a286493bc3a2ba98e
2024-06-18 12:19:07 +00:00
Samir Talwar
5ad33bd6a5 Avoid returning a Result when it is unnecessary. (#726)
Return a `T` instead of a `Result<T, E>` when we never return an error
(`E`) case.

I also enabled some more warnings. `unnecessary_box_returns` has been
suppressed where appropriate, and `unused_async` doesn't seem to be
violated anywhere any more.

I got rid of some calls to `.unwrap()` too.

V3_GIT_ORIGIN_REV_ID: 015ebd05978cf8c2d87474a90e0cd4333779a761
2024-06-18 12:03:04 +00:00
Daniel Harvey
8dae5ae742 Add example boolean expression type metadata for an Elasticsearch range (#699)
<!-- Thank you for submitting this PR! :) -->

## Description

This PR adds a test with example metadata for how to map to something
like an Elasticsearch [Range
query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-range-query.html).
Functional no-op.

V3_GIT_ORIGIN_REV_ID: 727e1706aab75957214b8cb13876eb23b843ba29
2024-06-13 08:37:32 +00:00
Daniel Chambers
962ae79e99 Move AggregateExpression resolve tests from engine tests to metadata-resolve tests (#714)
## Description
This PR moves all the tests that were added to
`crates/engine/tests/validate_metadata_artifacts/aggregate_expressions`
to the metadata-resolve `metadata_golden_tests`.

All the `error.txt` files got renamed to `expected_error.txt` and the
`metadata is not consistent:` bit on the front of the error got trimmed
off.

I had to change the way that `test_failing_metadata` works so that
instead of looking for directories under `failing/` it looks for
`metadata.json` files under `failing/`. This is because not every
directory has a test in it, some directories are just used for grouping
(eg `failing/aggregate_expressions/`). This doesn't change what tests
get run. The only side effect is that every test name has `_metadata`
suffixed to it (the filename). 😢 `test-each` doesn't appear to offer a
way to disable this behaviour.

V3_GIT_ORIGIN_REV_ID: dece7cd3bb4623c51050d12471d2c0990fd69bfd
2024-06-13 08:20:21 +00:00
Daniel Harvey
d34c419929 Allow nested metadata-resolve tests (#697)
<!-- Thank you for submitting this PR! :) -->

## Description

We have a set of tests in `metadata-resolve` that uses globs to find
files in either `passing` or `failing` folders and check them. This
allows those files to live in nested folders for tidyness. Functional
no-op.

V3_GIT_ORIGIN_REV_ID: 22d42e986676a110f455c59c3df9e6946c5d996c
2024-06-11 14:11:10 +00:00
Daniel Harvey
75ced29d11 Resolve nested object boolean expressions (#680)
<!-- Thank you for submitting this PR! :) -->

## Description

This adds the ability to describe nested object boolean expressions,
which become `fieldPath` items in the generated
`ndc_models::ComparisonTarget::Column` items. This allows us to describe
filtering a `User` based on some element in their nested `address` field
(like `postcode`, for example).

Like the other `BooleanExpressionType` work, this remains behind a
feature flag so should make no user-facing changes.

It is also missing a whole heap of metadata resolve checks, going to
follow with these after doing the happy path to unblock other work.

V3_GIT_ORIGIN_REV_ID: c89e2942a651d349fca97706affcf40d91afeefb
2024-06-11 12:57:21 +00:00
Anon Ray
52cfca7f91 fix: ndc validation of command (#669)
## Description

There is a bug in NDC validation of command where it returns early
without validating. This is because we perform the NDC validation right
after resolving the command source, but haven't attached the "resolved
command source" to the command yet. As a result the command's source is
`None`, and the validation does an early return.

This PR fixes it by taking the command source directly. If there is no
source to resolve, we don't have to perform any NDC validation of the
command.

Note: this is not the case with models. Model's source resolving
attaches the "resolved source", then performs the validation.

## Changelog

- Add a changelog entry (in the "Changelog entry" section below) if the
changes
  in this PR have any user-facing impact. See
[changelog
guide](https://github.com/hasura/graphql-engine-mono/wiki/Changelog-Guide).
- If no changelog is required ignore/remove this section and add a
  `no-changelog-required` label to the PR.

### Product

_(Select all products this will be available in)_

- [x] community-edition
- [x] cloud
<!-- product : end : DO NOT REMOVE -->

### Type

<!-- See changelog structure:
https://github.com/hasura/graphql-engine-mono/wiki/Changelog-Guide#structure-of-our-changelog
-->

_(Select only one. In case of multiple, choose the most appropriate)_

- [ ] highlight
- [ ] enhancement
- [x] bugfix
- [ ] behaviour-change
- [ ] performance-enhancement
- [ ] security-fix
<!-- type : end : DO NOT REMOVE -->

### Changelog entry

<!--
  - Add a user understandable changelog entry
- Include all details needed to understand the change. Try including
links to docs or issues if relevant
  - For Highlights start with a H4 heading (#### <entry title>)
  - Get the changelog entry reviewed by your team
-->

Fix skipping of NDC validation of commands due to wrong assumption.

<!-- changelog-entry : end : DO NOT REMOVE -->

<!-- changelog : end : DO NOT REMOVE -->

V3_GIT_ORIGIN_REV_ID: 5b9f4c922ee83394f1b6fe840cd934b6e138e2e9
2024-06-05 15:40:31 +00:00
Daniel Harvey
5fd6b6c03a Resolve boolean_expression_type metadata (#630)
<!-- Thank you for submitting this PR! :) -->

## Description

This somewhat resolves the `BooleanExpressionType` metadata type.

There is a lot that is missing, but everything is behind a feature so I
want to merge what is here before moving onto getting it working with
the later parts of the pipeline. We'll start with a lot of duplication
and then work to remove as much as we can.

Functional no-op for users, the feature flag is not exposed to the
actual binary.

How to review this PR:

- Check that any changes to existing code are cosmetic and naming-only
to ensure this is a no-op
- Look through the new resolve step and compare it to the
[RFC](https://github.com/hasura/v3-engine/blob/main/rfcs/open-dd-expression-type-changes.md)
and see if it is doing roughly what you might expect.
- Don't get too bogged down in missing things / style stuff, there are
definitely missing things and bad style.

V3_GIT_ORIGIN_REV_ID: 24e3b3e72e62d0094db3b461cb8c6d359982755d
2024-06-03 13:35:49 +00:00
Samir Talwar
4c3bcb882e Lint tests too, and suppress some warnings. (#622)
`test_each::path` always gives us a `PathBuf` even if we don't want one,
so we need to suppress the associated warning.

V3_GIT_ORIGIN_REV_ID: 4f2bb29122df6979ad378227fb88e4632d3551f7
2024-05-28 08:51:37 +00:00
Daniel Chambers
a132759dac Fixed arbitrary ordering of directions in GraphqlConfig causing error (#625)
## Description
If the user specified their orderings in their GraphqlConfig in a
different order, they would be presented with an error.

```yaml
enumTypeNames:
  - directions: # Reversed ordering of this list
      - Desc
      - Asc
    typeName: OrderBy
```

> ERR Code=opendds-validation Message="invalid metadata: error building
schema: unable to build schema: metadata is not consistent: invalid
directions: Desc,Asc defined in orderByInput of GraphqlConfig ,
currenlty there is no support for partial directions. Please specify a
type which has both 'asc' and 'desc' directions"
7:56PM ERR Supergraph Build failed.

This has been fixed.

## Changelog

### Product

_(Select all products this will be available in)_

- [x] community-edition
- [x] cloud
<!-- product : end : DO NOT REMOVE -->

### Type

<!-- See changelog structure:
https://github.com/hasura/graphql-engine-mono/wiki/Changelog-Guide#structure-of-our-changelog
-->

_(Select only one. In case of multiple, choose the most appropriate)_

- [ ] highlight
- [ ] enhancement
- [x] bugfix
- [ ] behaviour-change
- [ ] performance-enhancement
- [ ] security-fix
<!-- type : end : DO NOT REMOVE -->

### Changelog entry

<!--
  - Add a user understandable changelog entry
- Include all details needed to understand the change. Try including
links to docs or issues if relevant
  - For Highlights start with a H4 heading (#### <entry title>)
  - Get the changelog entry reviewed by your team
-->

Fixed arbitrary ordering of directions in GraphqlConfig causing invalid
error

<!-- changelog-entry : end : DO NOT REMOVE -->

<!-- changelog : end : DO NOT REMOVE -->

V3_GIT_ORIGIN_REV_ID: b27a0dd3b5c665f54ea58a40af8b2b1bfb0d2434
2024-05-28 05:43:48 +00:00
Tom Harding
e4c5d7a752 Add the SubgraphIdentifier type (#613)
## Description

Parallel to #611, in service of removing supergraph config, we'd like to
map all current supergraph config into a generated subgraph. To ensure
that we don't end up with an unfortunate naming collision, I've decided
to refine the definition of a subgraph identifier to exclude any string
starting with `__internal`, which we can then have as our "internal
namespace". This change introduces the new type, and updates the
"unknown_namespace" subgraph to come under this umbrella.

This change should be a no-op to any user who hasn't used a subgraph
name prefixed with `__internal`.

<!--
  Questions to consider answering:
  1. What user-facing changes are being made?
2. What are issues related to this PR? (Consider adding `(close
#<issue-no>)` to the PR title)
  3. What is the conceptual design behind this PR?
  4. How can this PR be tested/verified?
  5. Does the PR have limitations?
  6. Does the PR introduce breaking changes?
-->

## Changelog

User-defined subgraphs may no longer be given names that start with
`__internal`, as this will be reserved for internally generated
subgraphs.

- Add a changelog entry (in the "Changelog entry" section below) if the
changes
  in this PR have any user-facing impact. See
[changelog
guide](https://github.com/hasura/graphql-engine-mono/wiki/Changelog-Guide).
- If no changelog is required ignore/remove this section and add a
  `no-changelog-required` label to the PR.

### Product

_(Select all products this will be available in)_

- [x] community-edition
- [x] cloud
<!-- product : end : DO NOT REMOVE -->

### Type

<!-- See changelog structure:
https://github.com/hasura/graphql-engine-mono/wiki/Changelog-Guide#structure-of-our-changelog
-->

_(Select only one. In case of multiple, choose the most appropriate)_

- [ ] highlight
- [ ] enhancement
- [ ] bugfix
- [x] behaviour-change
- [ ] performance-enhancement
- [ ] security-fix
<!-- type : end : DO NOT REMOVE -->

### Changelog entry

<!--
  - Add a user understandable changelog entry
- Include all details needed to understand the change. Try including
links to docs or issues if relevant
  - For Highlights start with a H4 heading (#### <entry title>)
  - Get the changelog entry reviewed by your team
-->

User-defined subgraphs may no longer have names that start with
`__internal`.

<!-- changelog-entry : end : DO NOT REMOVE -->

<!-- changelog : end : DO NOT REMOVE -->

V3_GIT_ORIGIN_REV_ID: a6df1bba7b1f2cda4593654c15d168f25b7134b2
2024-05-27 10:06:42 +00:00
Tom Harding
da62dfb2b0 Allow supergraph config to appear in subgraphs (#611)
## Description

As per [APG-113](https://hasurahq.atlassian.net/browse/APG-113), we
would like to remove the separate concept of supergraphs. This is the
first step: we allow the supergraph config to appear in subgraphs. The
next step will be to remove the non-subgraph version of it.

This PR updates the subgraph enum type to allow for supergraph config
objects, but does nothing else. This means it should be a no-op for
current users, but will be forward-compatible with the eventual goal of
the epic. We also throw in a makeshift test framework that we can
probably replace with Insta at a later date.

<!--
  Questions to consider answering:
  1. What user-facing changes are being made?
2. What are issues related to this PR? (Consider adding `(close
#<issue-no>)` to the PR title)
  3. What is the conceptual design behind this PR?
  4. How can this PR be tested/verified?
  5. Does the PR have limitations?
  6. Does the PR introduce breaking changes?
-->

## Changelog

- Add a changelog entry (in the "Changelog entry" section below) if the
changes
  in this PR have any user-facing impact. See
[changelog
guide](https://github.com/hasura/graphql-engine-mono/wiki/Changelog-Guide).
- If no changelog is required ignore/remove this section and add a
  `no-changelog-required` label to the PR.

### Product

_(Select all products this will be available in)_

- [x] community-edition
- [x] cloud
<!-- product : end : DO NOT REMOVE -->

### Type

<!-- See changelog structure:
https://github.com/hasura/graphql-engine-mono/wiki/Changelog-Guide#structure-of-our-changelog
-->

_(Select only one. In case of multiple, choose the most appropriate)_

- [ ] highlight
- [ ] enhancement
- [ ] bugfix
- [x] behaviour-change
- [ ] performance-enhancement
- [ ] security-fix
<!-- type : end : DO NOT REMOVE -->

### Changelog entry

<!--
  - Add a user understandable changelog entry
- Include all details needed to understand the change. Try including
links to docs or issues if relevant
  - For Highlights start with a H4 heading (#### <entry title>)
  - Get the changelog entry reviewed by your team
-->

Supergraph configuration (`CompatibilityConfig`, `AuthConfig`, and
`GraphqlConfig`) may now be defined in any subgraph. Note that
supergraph configuration may still only be defined once for any given
supergraph.

<!-- changelog-entry : end : DO NOT REMOVE -->

<!-- changelog : end : DO NOT REMOVE -->

[APG-113]:
https://hasurahq.atlassian.net/browse/APG-113?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

V3_GIT_ORIGIN_REV_ID: a1b9911c4abb332fe37914efd1c3afbbd554a5d0
2024-05-24 08:59:55 +00:00
Daniel Harvey
951f994bec Fix minor formatting (#602)
<!-- Thank you for submitting this PR! :) -->

## Description

We have a formatting check job but it was not a required check so it got
missed, this fixes everything and will make the check required in
Github. Functional no-op.

V3_GIT_ORIGIN_REV_ID: 514478f2a48482ca34a860fedcfe6185b29c1dc3
2024-05-21 11:33:06 +00:00
Daniel Harvey
b138cd2017 OpenDD types for new boolean expressions (#599)
<!-- Thank you for submitting this PR! :) -->

## Description

We have a new `BooleanExpressionType` metadata kind. This adds it, tests
it can be parsed, but hides it from generated metadata and throws an
error if one is actually used in the engine.

V3_GIT_ORIGIN_REV_ID: 036b5fd9c32475d1c5a5e5e6321fb736fe6caefa
2024-05-21 09:28:53 +00:00