Commit Graph

8580 Commits

Author SHA1 Message Date
Samir Talwar
a8d1002175 server: Factor out authentication-related code into a new namespace, Hasura.Authentication.
In preparation for tightening up the various ways in which we construct and work with session variables, I am trying to move the behavior into the same module(s) as the data types, so that we can avoid exposing the internals of data structures in favor of smart constructors and conversions.

The session variable code was split between `Hasura.RQL.Types.Roles`, `Hasura.RQL.Types.Session`, and `Hasura.Session`, with the first two containing most of the data structures (and some logic) and the latter containing the rest of the logic. These files do not interact with the rest of `Hasura.RQL`, though they are depended upon by that namespace.

I have refactored these files into a new namespace, `Hasura.Authentication`. It now looks like this:

1. Role types are now in `Hasura.Authentication.Role`.
2. Header constants were moved from `Hasura.Server.Utils` to `Hasura.Authentication.Headers` (plural) to avoid cycles.
3. Header logic was moved from various places into `Hasura.Authentication.Header` (singular) for the same reason.
4. Session variable types and logic live together in `Hasura.Authentication.Session`.
5. User info types and logic live together in `Hasura.Authentication.User`.

This new structure is cycle-free and generally avoids importing the rest of the code, which means we should be able to start pruning the list of exports and locking down session variable construction.

No behavior was changed in this changeset.

The majority of changes are to the imports in a number of files; everything depends on these things. By splitting into multiple files, we also reduce the surface area of an individual import, which was a pleasant side-effect of this work.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10960
GitOrigin-RevId: 7cb962c06483cd9b92b80432aed5cabecb465cda
2024-07-29 06:02:56 +00:00
Tom Harding
4e6394355d Return correct type when an update_many has no arguments
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10954
GitOrigin-RevId: 214163f4f4764ea66194bf26bfe202a807771e90
2024-07-26 16:19:18 +00:00
ashwiniag
e34620a5f3 ci: tag release v2.40.3
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10956
GitOrigin-RevId: caf79b11499d6575cd2971ece2f0ef9e4103b511
2024-07-26 14:41:08 +00:00
Daniel Harvey
cc2373a6ad Add generated Elasticsearch schema to range test (#894)
<!-- The PR description should answer 2 (maybe 3) important questions:
-->

### What

We've had issues with `metadata-resolve` rejecting Elasticsearch schema
output, so adding said output to this test. Appears to work fine, so
merging it for further discussion and to improve the test case.

<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->

<!-- Consider: do we need to add a changelog entry? -->

### How

Add elastic search schema to test.

<!-- How is it trying to accomplish it (what are the implementation
steps)? -->

V3_GIT_ORIGIN_REV_ID: ea7c39ca7ab07fc18abd08eb822d2d56fc152ae6
2024-07-26 14:17:54 +00:00
dependabot[bot]
a54614431d Bump async-graphql-parser from 7.0.6 to 7.0.7 (#865)
Bumps
[async-graphql-parser](https://github.com/async-graphql/async-graphql)
from 7.0.6 to 7.0.7.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/async-graphql/async-graphql/blob/master/CHANGELOG.md">async-graphql-parser's
changelog</a>.</em></p>
<blockquote>
<h1>[7.0.7] 2024-07-14</h1>
<ul>
<li>Support raw values from serde_json <a
href="https://redirect.github.com/async-graphql/async-graphql/pull/1554">#1554</a></li>
<li>The custom directive <code>ARGUMENT_DEFINITION</code> is not being
output at the appropriate location in SDL <a
href="https://redirect.github.com/async-graphql/async-graphql/pull/1559">#1559</a></li>
<li>Support for JSON extended representations of BSON ObjectId and Uuid
<a
href="https://redirect.github.com/async-graphql/async-graphql/pull/1542">#1542</a></li>
<li>feat: get directives from SelectionField <a
href="https://redirect.github.com/async-graphql/async-graphql/pull/1548">#1548</a></li>
<li>Support Directives on Subscriptions <a
href="https://redirect.github.com/async-graphql/async-graphql/pull/1500">#1500</a></li>
<li>fix subscription err typo <a
href="https://redirect.github.com/async-graphql/async-graphql/pull/1556">#1556</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/async-graphql/async-graphql/commits">compare
view</a></li>
</ul>
</details>
<br />

[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=async-graphql-parser&package-manager=cargo&previous-version=7.0.6&new-version=7.0.7)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: 8c0d1a222a57c86cedf2f0870f6cccb0a861a3e6
2024-07-26 11:04:16 +00:00
Daniel Harvey
8bd439362b Update changelog links (#886)
<!-- The PR description should answer 2 (maybe 3) important questions:
-->

### What

Forgot to do this in last PR.

<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->

<!-- Consider: do we need to add a changelog entry? -->

### How

<!-- How is it trying to accomplish it (what are the implementation
steps)? -->

V3_GIT_ORIGIN_REV_ID: 24ec7278a63379f252d452b8ec627c934e1c534c
2024-07-26 10:43:47 +00:00
Tom Harding
0080e32e3c Remove empty subscription results
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10952
GitOrigin-RevId: 5581a416323877a92aa25a96398a62a88581c165
2024-07-26 09:08:48 +00:00
Anon Ray
72289171aa rename NdcFieldName to NdcFieldAlias (#882)
### What

We introduced a newtype around the NDC field alias, but we called it
`NdcFieldName`. While in reality it is the alias of the field requested
in the query.

This PR changes the name to `NdcFieldAlias`.

This is a no-op change

V3_GIT_ORIGIN_REV_ID: 8e892c29860e93243a200b6a6291fd0a32cc6fe3
2024-07-26 08:10:15 +00:00
Philip Lykke Carlsen
4f6bde1fee Enable use of Otel baggage via tracing-util crate (#888)
### What

Part of the point of the `tracing-util` crate is to centrally enforce
usage of a single version of opentelemetry libraries.

Previously we added some support for relaying baggage, but not actually
for defining it. This PR exposes the crates and types necessary to add
baggage to the context.

V3_GIT_ORIGIN_REV_ID: 107ec652d4e812f31bbfaa362cedf44b25dc3c39
2024-07-26 05:44:58 +00:00
Daniel Harvey
3357f970e9 Remove old Docker based building stuff (#876)
<!-- The PR description should answer 2 (maybe 3) important questions:
-->

### What

We have a bunch of local development infra for building the engine
inside a Docker container. This is helpful for Buildkite which doesn't
come with stuff like `cargo` preinstalled. We've not using Buildkite
anymore, let's remove it.

V3_GIT_ORIGIN_REV_ID: b4b7679aab5b14081288df25d139944f160a61fe
2024-07-25 16:16:49 +00:00
Daniel Harvey
42768bab3a Implement NoAuth mode in AuthConfig (#877)
<!-- The PR description should answer 2 (maybe 3) important questions:
-->

### What

We'd like to make it simpler to try out DDN, by starting with a mode
that uses no auth.

<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->

<!-- Consider: do we need to add a changelog entry? -->

### How

Add a `NoAuth` `AuthConfig` mode that is configured thus:

```json
"noAuth": {
  "role": "admin",
  "sessionVariables": {
    "x-hasura-user-id": "1"
  }
}
```

Given the above config:

- If no `x-hasura-role` is sent with a request, we run it as `admin`.
- If a `x-hasura-role` header is sent and it's `admin`, it continues to
work
- If any other `x-hasura-role` header is sent, an error will happen.
- All other headers are ignored, and we always set `x-hasura-user-id` to
1

<!-- How is it trying to accomplish it (what are the implementation
steps)? -->

V3_GIT_ORIGIN_REV_ID: dddcfbee9c3a31e84dfc8013de32e3a9bf31943d
2024-07-25 15:49:26 +00:00
Daniel Chambers
f84c2f3695 Validate that the capabilities version matches the DataConnectorLink schema version (#880)
This PR adds validation code to `metadata_resolve` that prevents someone
from putting a schema/capabilities from the wrong NDC version into the
DataConnectorLink while specifying a different schema version in the
DataConnectorLink. For example:

```
kind: DataConnectorLink
version: v1
definition:
  name: data_connector
  schema:
    version: v0.2
    schema: {}
    capabilities:
      version: 0.1.5 # Not allowed for version v0.2!
      capabilities: {}
```

This PR has two commits. One is a refactor where we rearrange the
DataConnectorError types so that the name of the data connector is
captured centrally in `NamedDataConnectorError`, so that it doesn't have
to be passed around and included in every error manually. The other is
the validation changes to `metadata_resolve`.

Completes APIPG-705

V3_GIT_ORIGIN_REV_ID: baed571f36f4cbed824ca546128f5df360d5b298
2024-07-25 14:31:28 +00:00
Daniel Harvey
1cd8e7f599 Remove benchmarks (#887)
<!-- The PR description should answer 2 (maybe 3) important questions:
-->

### What

These take ages to run, are slowing development down and not offering
the value they should. We should be benchmarking engine, but not like
this.

<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->

<!-- Consider: do we need to add a changelog entry? -->

### How

Remove benchmarks CI job from Buildkite.

<!-- How is it trying to accomplish it (what are the implementation
steps)? -->

V3_GIT_ORIGIN_REV_ID: 30a2c9d5f6ba09f5319a07fe394db8becaa16b8e
2024-07-25 13:53:36 +00:00
Daniel Chambers
8e8b9839a9 Make tests run over both the ndc v0.1.x and v0.2.x custom connectors (#879)
This PR updates as many tests as possible that use the custom connector
so that the tests run over two versions of the custom connector:
1. The custom connector in the repo, which currently speaks `ndc_models`
v0.2.x
2. The custom connector from the past (commit ), which is the last
version to speak `ndc_models` v0.1.x

This helps us test both the NDC v0.1.x and v0.2.x code paths. When the
postgres connector upgrades to v0.2.x, we can use the same approach as
in this PR to get the tests to run over multiple versions of the
postgres connector too, for much better coverage. This approach with the
custom connector will become less useful over time as the v0.1.x
connector is not updated and will diverge in data from the v0.2.x
connector. The postgres connector is likely to be longer-lasting, as it
is more stable.

The basic test used for `execute` integration tests is
`test_execution_expectation` (in `crates/engine/tests/common.rs`) and it
has been extended into a version called
`test_execution_expectation_for_multiple_ndc_versions` that takes
metadata on a per NDC version basis and then runs the test multiple
times, once for each NDC version. This allows one to swap out the
DataConnectorLink involved in the test to a different one that points at
either the v0.1.x or v0.2.x versions of the connector. The assertion is
that both connectors should produce the same results, even if they talk
a different version of the NDC protocol. As each version runs, we
`println!` the version so that if the test fails you can look in stdout
for the test and see which one was executing when it failed.

Tests that use the custom connector now use
`test_execution_expectation_for_multiple_ndc_versions` and run across
both connector versions. Some tests were unable to be used across both
version as the data between the two versions has changed. Some tests
were modified to avoid the changed data so as to support running across
both versions. Any tests that use `test_execution_expectation_legacy`
don't run across both versions because those tests aren't backed by the
same test implementation as
`test_execution_expectation_for_multiple_ndc_versions`.

Unfortunately the custom connector doesn't use the standard connector
SDK, so it doesn't support `HASURA_CONNECTOR_PORT`. This means that the
old connector is stuck on 8101. To work around this, I've moved the
current connector port to 8102 instead. Technically we might be able to
use docker to remap the ports, but then this binds us into always
running the connectors in docker in order to move their ports around, so
I avoided that approach.

Completes APIPG-703

V3_GIT_ORIGIN_REV_ID: fb0e410ddbee0ea699815388bc63584d6ff5dd70
2024-07-25 13:32:01 +00:00
Daniel Harvey
291df666a6 [changelog] release v2024.07.25 (#885)
Get ready for `v2024.07.25` release, updating changelog.

V3_GIT_ORIGIN_REV_ID: 4561b318ae234323c53bb8acb8b45d90aede78ab
2024-07-25 12:41:15 +00:00
Daniel Harvey
cb380da086 Pass TraceContextResponsePropagator to set_text_map_propagator (#884)
<!-- The PR description should answer 2 (maybe 3) important questions:
-->

### What

In a recent engine change, we changed some of our trace context mapping
to use the shared settings consistently. However, we needed to make sure
we included `TraceContextResponsePropagator`, which returns the
`traceresponse` header.

Request from console after this fix:

<img width="810" alt="Screenshot 2024-07-25 at 11 58 30"
src="https://github.com/user-attachments/assets/c8e73c56-87fd-49da-a887-f91cdb6d607a">

<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->

<!-- Consider: do we need to add a changelog entry? -->

### How

Adds `TraceContextResponsePropagator` to the global set of text map
propagators.

<!-- How is it trying to accomplish it (what are the implementation
steps)? -->

V3_GIT_ORIGIN_REV_ID: 48df6a6fe55e78a48f1dc6bf82304199a0a7e248
2024-07-25 11:32:06 +00:00
Tom Harding
4320eb0b71 Add some more Show instances
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10950
GitOrigin-RevId: 20fee6351ee4a8bceabddf12bee50c3101ad78fa
2024-07-25 10:24:39 +00:00
Anon Ray
fd734e061d human-readable NDC relationship name in NDC IR (#881)
<!-- The PR description should answer 2 (maybe 3) important questions:
-->

### What

NDC query request expects relationship names which are unique across the
query.

Previously, we would generate relationship name of the form -

```
[{\"subgraph\":\"connector_2\",\"name\":\"Album\"},\"Tracks\"]
```

This works, but is harder to read while debugging. This PR changes it to
have a human-readable name like -

```
connector_2___Album__Tracks
```

This is a no-op change, apart from the relationship names in NDC query
requests.

### How

Instead of json-ifying the data structure in a tuple, create a formatted
string.

V3_GIT_ORIGIN_REV_ID: 3fea3bf56f1688bc1cade1ea2b3ed6eb60509cac
2024-07-25 08:10:58 +00:00
hasura-bot
0ddf3fd740 Update upgrade-v3.mdx
GITHUB_PR_NUMBER: 10474
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/10474

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10948
Co-authored-by: Konstantinos Gallis <43298282+kongallis@users.noreply.github.com>
GitOrigin-RevId: 386d4197248c660287c18ac906c3ae6b9bcecccc
2024-07-24 14:19:04 +00:00
Rakesh Emmadi
bf1fd4dbd9 [changelog] release v2024.07.24 (#875)
Update changelog for release v2024.07.24

V3_GIT_ORIGIN_REV_ID: 7a87941774635dd1fb0e98ac406a88908fa55ba4
2024-07-24 14:03:01 +00:00
Daniel Harvey
66e847bc46 Move "test" job to Github Actions (#872)
<!-- The PR description should answer 2 (maybe 3) important questions:
-->

### What

We've had our CI mixed between Github and Buildkite for a while, it's
time to commit. First step is moving the "tests" step to Github Actions.

<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->

<!-- Consider: do we need to add a changelog entry? -->

### How

This PR:

- Moves the `test` step to Github Actions
- Creates a new `custom_connector.Dockerfile` which builds custom
connector only, more quickly.
- Changes the metadata tests to use `localhost` instead of their Docker
internal names (ie `custom_connector` or `postgres_connector`) - this is
because the tests are being run from outside Docker now
- Removes the `test` Buildkite step

It does not:

- Remove the code coverage or benchmarks steps from Buildkite
- Tidy up `justfile` or Dockerfiles

<!-- How is it trying to accomplish it (what are the implementation
steps)? -->

---------

Co-authored-by: Philip Lykke Carlsen <plcplc@gmail.com>
V3_GIT_ORIGIN_REV_ID: a67534ebc1634a24b48d2620c45003221852e199
2024-07-24 13:41:40 +00:00
Daniel Harvey
35a1fcad15 Remove code coverage CI job (#874)
<!-- The PR description should answer 2 (maybe 3) important questions:
-->

### What

Our CI times are slow, thus merging is slow, and I do not feel this job
pulls it weight.

<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->

<!-- Consider: do we need to add a changelog entry? -->

### How

Remove the code coverage job from Buildkite, delete all associated
files.

<!-- How is it trying to accomplish it (what are the implementation
steps)? -->

V3_GIT_ORIGIN_REV_ID: 7bbde842f9f405920d1e559b8fef88078cc8d564
2024-07-24 13:22:31 +00:00
Daniel Chambers
dacb229d10 sql crate now executes via plan and decouples from NDC types (#873)
Previously the `sql` crate generated a v02 ndc query request and then
downgraded it to v01 if necessary. This is fragile in that its easy to
use v02 ndc features and then get v01 downgrade errors, plus the
downgrade logic is extensive and tedious.

This PR refactors the `sql` crate so that it generates `ir` and `plan`
types and eventually creates `ResolvedQueryExecutionPlan` (rather than
ndc_models types), and then the ResolvedQueryExecutionPlan is
transformed into the appropriate ndc version in the same fashion as the
main engine execute code does it. This eliminates all the downgrade
logic and simplifies things.

Unfortunately, ndc's `QueryRequest` could not just simply be replaced
with `QueryExecutionPlan` on `sql`'s `NDCQuery` and `NDCPushDown`,
because it involves lifetime parameters which are incompatible with the
datafusion framework types. Instead, the individual components of a
query are kept on `NDCQuery` and `NDCPushDown`, and these are eventually
assembled into a `ResolvedQueryExecutionPlan` at a place where the
lifetime parameters are workable. In some sense this is clearer, as one
can now see where each individual part of the query is actually created
and relevant, instead of copying around and mutating a `QueryRequest`.

Completes
https://linear.app/hasura/issue/APIPG-702/implement-separate-logic-that-maps-engine-types-to-ndc-models-types-on

V3_GIT_ORIGIN_REV_ID: c4a9226c1b1addcfe5cd0bca783f1b65ab3ada38
2024-07-24 11:37:44 +00:00
Daniel Chambers
00fa5c42ba Refactor to prevent unresolved queries from being sent as ndc requests (#871)
~~Note: this PR is stacked on #845.~~ Rebased on main

This PR refactors the `execute::plan::types` further to make a clear
distinction between unresolved and resolved states. An "unresolved"
state refers to one in which remote predicates have not been computed
into local predicates. A "resolved" state is after this process is
performed and remote predicates are eliminated.

Previously, unresolved types could be passed to
`execute::plan::ndc_request` and they would fail at runtime due to the
presence of unresolved remote predicates. Now, this is impossible due to
a type-level distinction between unresolved and resolve states.

This distinction is made by type-parameterizing all
`execute::plan::types` that involve a predicate so that the predicate
type is parameterized out. Then, an `Unresolved` type alias is created
that sets the predicate type to
`execute::ir::filter::expression::Expression` (which contains remote
predicates) and a `Resolved` type alias is created that uses
`ResolvedFilterExpression` instead (which does not contain remote
predicates).

For example, for `QueryNode`, we now have:

```rust
pub struct QueryNode<'s, TFilterExpression> {
    ...
    pub predicate: Option<TFilterExpression>,
    ...
}
```

And then the two aliases are:

```rust
pub type UnresolvedQueryNode<'s> = QueryNode<'s, ir::filter::expression::Expression<'s>>;
pub type ResolvedQueryNode<'s> = QueryNode<'s, ResolvedFilterExpression>;
```

Subsequently, `plan::ndc_request` only deals with `Resolved` types.

This is mostly just type-fiddling, but one place some logic moved around
is in with the old `plan::types::FilterExpression`. This was mostly a
functional duplicate of `ir::filter::execute::Expression` except that it
had a "planned" remote predicate variant in it. In order to reduce the
number of types (so we didn't need `UnresolvedFilterExpression` and
`ResolvedFilterExpression`), this type has been repurposed into
`ResolvedFilterExpression` and no longer deals with remote predicates.
Instead, `ir::filter::execute::Expression` is resolved into a
`ResolvedFilterExpression` and the planning of the remote predicate is
done at that time, just before it is resolved. This works fine, since an
entirely new ndc query is performed in order to resolve the predicate,
so planning that can be deferred until then and it doesn't need to be
done at the same time as the main query.

Part of
https://linear.app/hasura/issue/APIPG-702/implement-separate-logic-that-maps-engine-types-to-ndc-models-types-on

V3_GIT_ORIGIN_REV_ID: 3ec89efbaa7b543fad6a100e2739bcc74b1d567f
2024-07-24 09:55:39 +00:00
Samir Talwar
934cb3e25e server: Remove headers from error details.
In rare cases, sensitive headers were written to the GraphQL Engine logs. These cases are now handled correctly to avoid logging sensitive information.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10946
GitOrigin-RevId: a9d66532245789a16cf91936d53bbfce19d272a4
2024-07-24 09:32:01 +00:00
Daniel Chambers
d409e52f7d Remove NDC types from ir and plan and perform separate plan->NDC mapping per NDC version (#845)
This PR removes usages of ndc_models types from the `ir` types and the
`plan::types` in the `execute` crate. This is done to isolate the IR and
planning logic from the specific version of NDC that needs to be used to
talk to the specific data connector. Once planning is done, the
`plan::types` are mapped into `ndc_models` types via
`plan::ndc_request::v01` and `plan::ndc_request::v02`. Those two modules
contain code that maps `plan::types` to a specific ndc_models version.
This code is entirely separate per version.

Now, when the `plan::types` are `resolve`d, they don't return NDC types.
Now they return themselves, but modified to remove any remote
predicates. After `resolve` the types are in a state that they can be
converted into a single NDC request via `plan::ndc_request`.

Next steps for a future PR: `QueryExecutionPlan` (etc) needs to type
parameterize out the `FilterExpression` type so that after `resolve`, a
new `QueryExecutionPlan` is returned that uses another FilterExpression
type that does not have remote predicates in it. This will make
resolving typesafe and prevent accidentally using `plan::ndc_request` on
unresolved `QueryExecutionPlan`s.

The `sql` crate also needs to stop talking ndc directly and probably
talk `plan::types` instead. Right now it writes ndc v0.2.0 and then
downgrades it to v0.1.0 if necessary, which is awful. Talking
`plan::types` is currently not possible due to lifetime parameters used
on the `plan::types`. These may be able to be removed from the resolved
variant of those types as mentioned above.

Part of
https://linear.app/hasura/issue/APIPG-702/implement-separate-logic-that-maps-engine-types-to-ndc-models-types-on

V3_GIT_ORIGIN_REV_ID: b536009ea784d1486a2ece2262e0ce9d0f937ef0
2024-07-24 07:28:50 +00:00
Daniel Harvey
f2a12f492c Fix manually passed boolean expression arguments (#869)
<!-- The PR description should answer 2 (maybe 3) important questions:
-->

### What

When a boolean expression is passed as an argument it was not being
translated into an `ndc_models::Expression` and so queries failed.

<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->

<!-- Consider: do we need to add a changelog entry? -->

### How

Look up the type of an argument in `metadata-resolve`, and mark the
`ArgumentInfo` with a new `ArgumentKind`. Then in IR step we use that to
work out whether to turn the argument to JSON as before, or translate it
into an `Expression` type that will eventually be turned into an
`ndc_models::Expression`.

<!-- How is it trying to accomplish it (what are the implementation
steps)? -->

V3_GIT_ORIGIN_REV_ID: 4da3ce0ae04895c33de2b6bdb6fff1018c39b3ad
2024-07-23 13:36:18 +00:00
Philip Lykke Carlsen
cdfa09c6ad Enable baggage propagation (#868)
### What

This PR enables the use of Opentelemetry Baggage. Every bit of baggage
is then replicated on every span.

The current implementation does not actually set any baggage itself - it
only relays and outputs what it's getting.

Crafting a request with a `baggage` header set:

![image](https://github.com/user-attachments/assets/f2974398-370a-4e8c-8761-692cfc5682f6)

Has it propagated (here, to `dev-auth-webhook`) and stamped onto every
span:

![image](https://github.com/user-attachments/assets/6661c41f-56be-4edd-9027-e88eb816f1e7)

### How

This PR actually makes the engine and auth-hook use the globally
specified propagators (before they would only use an obsucre, concrete,
re-exported one from opentelemetry_contrib), and adds the
`BaggagePropagator` to the list.

It also adds a `SpanProcessor` which outputs the baggage as span
attributes. Currently it outputs all Baggage entries, but can be made
more specific in the future if we want to treat Baggage differently.

V3_GIT_ORIGIN_REV_ID: 3b5b8604b624c0b90c192e68b3b57fab7ca9b63e
2024-07-23 11:44:54 +00:00
Daniel Harvey
8e79e53b63 Raise warnings during metadata-resolve (#859)
<!-- The PR description should answer 2 (maybe 3) important questions:
-->

### What

Sometimes our builds succeed, but we'd like to tell the user how they
could do better. This implements the simplest possible warnings system.

<img width="957" alt="Screenshot 2024-07-18 at 16 06 49"
src="https://github.com/user-attachments/assets/ff91d221-667a-43f9-bc8a-51bf4574a7b8">

<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->

<!-- Consider: do we need to add a changelog entry? -->

### How

Warnings are printed to stdout on `v3-engine` startup, and will be
returned to the CLI via `v3-metadata-build-service`. The diff is mostly
updated snapshot tests as we're returning more from
`metadata-resolve::resolve` now.

<!-- How is it trying to accomplish it (what are the implementation
steps)? -->

V3_GIT_ORIGIN_REV_ID: d01520e53f49d9b594e94a4531b6a86e749875c3
2024-07-23 10:01:21 +00:00
Anon Ray
9eaabd116f fix: generate relationship definition for nested selection (#855)
### What

Previously, while generating relationship definitions for NDC, we would
ignore columns with nested selection.

This PR fixes that.

Closes https://hasurahq.atlassian.net/browse/V3ENGINE-247

### How

While matching on `FieldSelection::Column`, don't ignore it. Check if it
contains nested selection, if it does, call
`collect_relationships_from_nested_selection`

V3_GIT_ORIGIN_REV_ID: 9db94744d8e2d35f8430bded07209ef519175205
2024-07-23 08:12:51 +00:00
Vamshi Surabhi
a6dbc7bb1d Do not replace the existing analyzer rules (#870)
<!-- The PR description should answer 2 (maybe 3) important questions:
-->

### What

This PR fixes issues in the SQL layer where the following queries would
fail:

1. `select count(*) from "Track"`
2. `select * from "Track" where id = 1`

<!-- Consider: do we need to add a changelog entry? -->

### How

These were failing because the built-in analyzer rules that rewrite
`count(*)` and type-cast expressions weren't firing.
`with_analyzer_rules` replaces the analyzer rules of a session context
with the given list. We want our analyzer rule to be fired in addition
to the built-in analyzer rules.

Tests are being worked on in a separate PR.

V3_GIT_ORIGIN_REV_ID: 42231f97b5b28d9b7eeff0c3e592cb43ff7d952f
2024-07-23 06:04:38 +00:00
hasura-bot
dea939923d Update observability.mdx
GITHUB_PR_NUMBER: 10108
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/10108

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10636
Co-authored-by: Valentin Hervieu <2678610+ValentinH@users.noreply.github.com>
GitOrigin-RevId: 956c8bc17bd9f5cfebc912678b572a38dbad9070
2024-07-22 10:07:02 +00:00
dependabot[bot]
3ec5dbf39a Bump tokio from 1.38.0 to 1.38.1 (#863)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.38.0 to 1.38.1.

V3_GIT_ORIGIN_REV_ID: 917209ce6aeb733d92e4232fc7183c153020e276
2024-07-22 08:11:20 +00:00
dependabot[bot]
48428dd4af Bump darling from 0.20.9 to 0.20.10 (#862)
Bumps [darling](https://github.com/TedDriggs/darling) from 0.20.9 to 0.20.10.

V3_GIT_ORIGIN_REV_ID: c5ced551c5a019f7416ac70d367197a41bafa7b7
2024-07-22 07:40:57 +00:00
dependabot[bot]
e095ea67dd Bump openssl from 0.10.64 to 0.10.66 (#864)
Bumps [openssl](https://github.com/sfackler/rust-openssl) from 0.10.64 to 0.10.66.

V3_GIT_ORIGIN_REV_ID: b3d1e21b504122bb1a9e2332a60ee4b3f04781aa
2024-07-22 07:10:56 +00:00
dependabot[bot]
13e39c7231 Bump syn from 2.0.71 to 2.0.72 (#861)
Bumps [syn](https://github.com/dtolnay/syn) from 2.0.71 to 2.0.72.

V3_GIT_ORIGIN_REV_ID: fcdab429e082ee272ee4011a27894dc73b14646e
2024-07-22 06:40:10 +00:00
Rob Dominguez
4244b0d87b Docs: Add keywords to support page
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10940
GitOrigin-RevId: 7b8f4897512b047cb4ec8cd940e58d2dd7e9cb5c
2024-07-19 13:34:18 +00:00
Sean Park-Ross
7d6395fc67 Docs: Update banner text pointing to DDN
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10943
GitOrigin-RevId: 73e8e7867b1f4459891fb5594eb020ef06c8dcc1
2024-07-19 13:26:50 +00:00
Tom Harding
3f15dfd1f5 Don't set columns to NULL when nullable variables are omitted
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10921
GitOrigin-RevId: 01779e0d02d4de1ca56d84f3d11f5e7513b87676
2024-07-19 11:59:52 +00:00
Daniel Harvey
407c5b9ece Rename one of the ArgumentPreset types to DataConnectorArgumentPreset (#860)
<!-- The PR description should answer 2 (maybe 3) important questions:
-->

### What

We're having issues with our deduplication of names in JSONSchema. We
would like to fix this, but in the short term, this renames a
conflicting object to avoid this quickly.

<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->

<!-- Consider: do we need to add a changelog entry? -->

### How

Rename `ArgumentPreset` in `open_dds::data_connectors` to
`DataConnectorArgumentPreset`.

<!-- How is it trying to accomplish it (what are the implementation
steps)? -->

V3_GIT_ORIGIN_REV_ID: e3eafeffe8ba4d513f9d0a09a623f101650247ea
2024-07-19 09:10:29 +00:00
Samir Talwar
b8d7c6039b Use a test PostgreSQL image based on Debian, not Alpine.
Alpine is often slower than Debian (because musl is sometimes slower than glibc) and not how anyone actually deploys PostgreSQL in production.

Most notably, some floating-point computations result in slightly different values, and Debian ships with better support for different locales so sorting text (e.g. with `ORDER BY`) behaves differently.

Let's test against an environment that people are likely to actually use.

As a result, there are slight changes to the results of PostGIS computations in a couple of test cases.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10942
GitOrigin-RevId: 4caed19def23a372fc3930c409514b1c9b385026
2024-07-19 08:18:01 +00:00
paritosh-08
5a598875bb update changelog for v2024.07.18 (#854)
<!-- The PR description should answer 2 (maybe 3) important questions:
-->

### What

update changelog for new release

### How

:)

V3_GIT_ORIGIN_REV_ID: 6620f9923f393d190f0f2fab1aced1cff4d6aec0
2024-07-18 13:55:28 +00:00
Daniel Harvey
34d1ac55fe Return AggregateExpressionError from aggregates stage (#843)
<!-- The PR description should answer 2 (maybe 3) important questions:
-->

### What

This stage already has it's own error type, but it returns the larger
`Error` type, so let's return the more specific type instead.

<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->

<!-- Consider: do we need to add a changelog entry? -->

### How

Changing return types mostly. Functional no-op.

<!-- How is it trying to accomplish it (what are the implementation
steps)? -->

V3_GIT_ORIGIN_REV_ID: 2aae1f06775db6d88c34b1d3c1779396e0ba410e
2024-07-18 12:01:40 +00:00
Daniel Harvey
b7bf7fe677 Move data connector scalar types error into own type (#839)
<!-- The PR description should answer 2 (maybe 3) important questions:
-->

### What

More breaking down the big error type, this time we sort the
`data_connector_scalar_types` stage. Functional no-op.

<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->

<!-- Consider: do we need to add a changelog entry? -->

### How

Move error cases into a smaller enum.

<!-- How is it trying to accomplish it (what are the implementation
steps)? -->

V3_GIT_ORIGIN_REV_ID: 50b699f3a77594deb27a6cc6ab8dd61752404daf
2024-07-18 10:59:42 +00:00
Rakesh Emmadi
561b908342 Implement remote relationship predicates in where filter clause (#761)
Now, users can filter their queries using remote relationships in the
filter predicate. Users need to provide the relationships for comparison
in `comparableRelationships` field of the newer `BooleanExpressionType`
opendd metadata.

Minimal Algorithm:
```
Relationship: ARemoteB => Model_A -> Model_B (remote NDC)
Column Mapping: (A_column_1, B_column_1), (A_column_2, B_column_2).

query:
  Model_A:
    where: ARemoteB: {B_column_3: {_eq: value}}

Step 1: Fetch RHS column values (in mapping) from remote target model
  SELECT B_column_1, B_column_2 from model_b_collection WHERE B_column_3 = value;
yields  the following rows
[
  [(B_column_1, b_value_1), (B_column_2, b_value_2)],
  [(B_column_1, b_value_11), (B_column_2, b_value_22)],
]

Step 2: Using above rows the generate LHS column filter for Model_A query.

SELECT <fields> from model_a_collection WHERE
  ((A_column_1 = b_value_1) AND (A_column_2 = b_value_2))
 OR ((A_column_1 = b_value_11) AND (A_column_2 = b_value_22))

The above comparison is equivalent to
WHERE
  (A_column_1, A_column_2) IN ((b_value_1, b_value_11), (b_value_2, b_value_22))

```

Sample query:
```graphql
query MyQuery {
 Track(
    where: {
      _or: [
        { AlbumRemote: { Artist: { ArtistId: { _eq: 2 } } } }
        { TrackId: { _eq: 3 } }
      ]
    }
  ) {
    TrackId
    AlbumRemote {
      Artist {
        ArtistId
        Name
      }
    }
  }
}
```
In the query above, `AlbumRemote` is a remote relationship which targets
a model backed by a different data connector.

V3_GIT_ORIGIN_REV_ID: 7aa76fcae83e1f22de460f1eef5648fb7a35c047
2024-07-18 07:47:04 +00:00
Daniel Harvey
342ff1fce6 Return BooleanExpressionError from object-boolean-expressions stage (#844)
<!-- The PR description should answer 2 (maybe 3) important questions:
-->

### What

Much like this change, but for the `object_boolean_expressions` stage:
https://github.com/hasura/v3-engine/pull/843

<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->

<!-- Consider: do we need to add a changelog entry? -->

### How

<!-- How is it trying to accomplish it (what are the implementation
steps)? -->

V3_GIT_ORIGIN_REV_ID: 85da185b45bac429b754b0b92419f378a59fb536
2024-07-17 18:23:14 +00:00
Samir Talwar
9cad4dea99 server/tests-py: More useful messages when logging tests fail.
It's useful to know what caused the failure when parsing logs.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10938
GitOrigin-RevId: 53f3036625644a04bc5fd3afb150262515cfec29
2024-07-17 14:47:08 +00:00
Samir Talwar
b2ac4d82bc Remove the version from Docker Compose files.
Recent versions of Docker Compose no longer support this, instead just printing a warning.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10937
GitOrigin-RevId: 97a82968c48f5c09d6cbe74d8ea7386979e46e7a
2024-07-17 13:49:05 +00:00
Daniel Chambers
455724dd07 Fixed command targeted relationships not using data connector argument names (#841)
### What
This PR fixes an issue where relationships that target commands do not
correctly use the data connector's argument name when making the ndc
request. Instead, they use the OpenDD argument name, which is incorrect.

For metadata where the OpenDD argument name is the same as the data
connector's argument name, the code works but only coincidentally.

### How
I've updated an existing test to change the name of the command argument
to be different from the data connector's argument name. This test
failed but is now fixed by this PR, which simply looks up the name of
the data connector argument name and uses that instead.

V3_GIT_ORIGIN_REV_ID: 71f1e812174c7bb9922792523129e4bcdce911ed
2024-07-17 08:37:18 +00:00
Rob Dominguez
9f6ac66746 Bug Bash: Fix typo (#838)
### What

This closes hasura/graphql-engine#10433 and closes
hasura/v3-engine-multitenant#917

[DOCS-2225](https://hasurahq.atlassian.net/browse/DOCS-2225)

[DOCS-2225]:
https://hasurahq.atlassian.net/browse/DOCS-2225?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

V3_GIT_ORIGIN_REV_ID: 8c9d161af0737ecfee973d0083198617e21329de
2024-07-16 12:59:22 +00:00