<!-- The PR description should answer 2 (maybe 3) important questions:
-->
Closes:
https://linear.app/hasura/issue/APIPG-397/support-remote-relationship-predicates-in-permission-filters
### What
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
Allow defining permission filters with remote relationships in their
predicates.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
- Lift metadata resolve restriction for remote relationships in
permission predicates
- Abstract out the remote relationship resolving logic, in query filter,
into a new function and re-use it while resolving permission filters.
- Tests:
- A metadata build test to check the presence of essential equal
operator on source fields in relationship mapping.
- Ported all `select_many/relationship_predicate/`* tests to a new
`select_many/remote_relationship_predicate/*` with appropriate metadata
changes.
---------
Co-authored-by: Anon Ray <ecthiender@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: 9c496ecdc9829ed626354ef85e776e1afcb0dfc7
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
This pull request optimizes the `DistinctComparisons` struct in the
codebase to improve the performance of storing and checking for distinct
comparison predicates in remote relationship comparison expressions.
<!-- Consider: do we need to add a changelog entry? -->
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
- Replaced `Vec` with `IndexSet`: Changed the data structure used in the
DistinctComparisons struct from `Vec` to `IndexSet` to leverage the
average O(1) complexity for contains and insert operations provided by
`IndexSet`.
- Updated push Method: Modified the push method to use `IndexSet`'s
insert method directly, which simplifies the code and improves
performance.
**Performance Improvement:**
query:
```graphql
query RemoteRelationship {
Album(where: {TracksRemote: {Name: {_ilike: "%B%"}}}) {
Title
}
}
```
The `TracksRemote` predicate query yields 723 non-distinct results,
which reduce to 266 unique results after deduplication.
Benchmark used: [graphql-bench](https://github.com/hasura/graphql-bench)
configuration: autocannon - Requests Per Second strategy (50 rps) - 10
seconds duration.
Results:
- Before Optimization:
Average Latency: 38.99 ms
- After Optimization:
Average Latency: 23.32 ms
- Percentage Decrease in Latency: Approximately 40%
V3_GIT_ORIGIN_REV_ID: 17a7160b7229eb3a2fde93273d5cf05102f9b4bd
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
Update architecture doc to include `ir` crate. Functional no-op.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 32f47529c97c7cfe188f098f2024e159e5ab33cc
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
`execute` is now the biggest `crate` in engine and does a lot, let's
split it into it's constituent steps.
Functional no-op.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Split out `ir` crate from the `execute` crate. Replace export of entire
modules with that of specific types / functions. Therefore, consumers
outside the crate talk about `ir::CommandInfo` rather than
`ir::command::CommandInfo`. There is no need for other crates to know
about the internal structure of this crate.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 47553aec63e80af7f95e659a170a2685e9ac2ce3
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
The `plan/type.rs` has become large and overwhelmed. This PR refactors
its code and removes it.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
- Move code from `plan/types.rs` into old `arguments.rs`, `filter.rs`
and new `field.rs`, `query.rs`, `mutation.rs`.
- Delete `plan/types.rs`
- Refactor code in other modules to accommodate new changes.
V3_GIT_ORIGIN_REV_ID: 0e294ca8fb4bf1d8622806f5c8b72a2bb01ccdaf
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
These checks are breaking artifact generation, so disabling them until
we can find a safer way to introduce them.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: ae97c87720b67384127122ed0220383036c87bbf
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
Making this match engine.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Replace `V4` with `V6`
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: e86d118b96d41407a292f9ad4132b8ab6d06454f
In certain circumstances, all HTTP headers were included in the set of session variables. This has now been reduced to only the session variables. This change affects a number of areas, including the HTTP logs, JWT claims, rate limiting, and accessing session variables in Kriti code.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10964
GitOrigin-RevId: fc573cd89f7c0f1a17b9e8a56396f31e70669650
### What
Telemetry-baggage is propagated via headers from incoming requests to a
service and relayed when the service itself calls another service.
However, when a service is open to the public it may not want just
anyone to be able to pass it baggage.
This PR adds the ability to configure the policy towards baggage
relaying in the tracing-util crate.
### How
When the argument `initialize_tracing(..., propagate_caller_baggage =
false)` we add to the globally defined text map propagator a derived
version of the `BaggagePropagator` which cannot extract baggage from
incoming requests, only inject its own context baggage into outgoing
requests.
V3_GIT_ORIGIN_REV_ID: af9a51c20a8fe7ae2085e8218a4f1d5e01b26ae1
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
This allows object types to be used as arguments for comparison
operators. This is useful for Elasticsearch's `range` operator, which
allows passing an object like `{ gt: 1, lt: 100 }` to an `integer` field
in order to filter items that are greater than `1` and less than `100`.
This PR has the nice side effect of dropping the requirement to use
information from scalar `BooleanExpressionType`s in place of
`DataConnectorScalarTypes`, which we only required because we were not
looking up the comparable operator information in scalar boolean
expression types correctly.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Previously, when using `ObjectBooleanExpressionType` and
`DataConnectorScalarRepresentation`, we had no information about the
argument types of comparison operators (ie, what values should I pass to
`_eq`?), and so inferred this by looking up the comparison operator in
the data connector schema, then looking for a
`DataConnectorScalarRepresentation` that tells us what OpenDD type that
maps to.
Now, with `BooleanExpressionType`, we have this information provided in
OpenDD itself:
```yaml
kind: BooleanExpressionType
version: v1
definition:
name: Int_comparison_exp
operand:
scalar:
type: Int
comparisonOperators:
- name: _eq
argumentType: Int! # This is an OpenDD type
- name: _within
argumentType: WithinInput!
- name: _in
argumentType: "[Int!]!"
```
Now we look up this information properly, as well as tightening up some
validation around relationships that was making us fall back to the old
way of doing things where the user had failed to provide a
`comparableRelationship` entry.
This means
a) we can actually use object types as comparable operator types
b) scalar boolean expression types aren't used outside the world of
boolean expressions, which is a lot easier to reason about.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: ad5896c7f3dbf89a38e7a11ca9ae855a197211e3
Bumps [env_logger](https://github.com/rust-cli/env_logger) from 0.11.3
to 0.11.5.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/rust-cli/env_logger/releases">env_logger's
releases</a>.</em></p>
<blockquote>
<h2>v0.11.5</h2>
<h2>[0.11.5] - 2024-07-25</h2>
<h2>v0.11.4</h2>
<h2>[0.11.4] - 2024-07-23</h2>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/rust-cli/env_logger/blob/main/CHANGELOG.md">env_logger's
changelog</a>.</em></p>
<blockquote>
<h2>[0.11.5] - 2024-07-25</h2>
<h2>[0.11.4] - 2024-07-23</h2>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="73bb418802"><code>73bb418</code></a>
chore: Release</li>
<li><a
href="57f1bb7aae"><code>57f1bb7</code></a>
chore: Release</li>
<li><a
href="ba2748f48c"><code>ba2748f</code></a>
chore: Release</li>
<li><a
href="4dd976b01e"><code>4dd976b</code></a>
Merge pull request <a
href="https://redirect.github.com/rust-cli/env_logger/issues/324">#324</a>
from Maximkaaa/try_parse</li>
<li><a
href="05aacb93c5"><code>05aacb9</code></a>
feat: Add Builder::try_parse method</li>
<li><a
href="faf5b3e499"><code>faf5b3e</code></a>
chore: Tests for <code>parse_spec</code> error messages</li>
<li><a
href="c42511cc3c"><code>c42511c</code></a>
refactor: Return errors from <code>parse_spec</code></li>
<li><a
href="0e25d9e993"><code>0e25d9e</code></a>
chore(deps): Update Rust Stable to v1.79 (<a
href="https://redirect.github.com/rust-cli/env_logger/issues/322">#322</a>)</li>
<li><a
href="e1640c493d"><code>e1640c4</code></a>
Merge pull request <a
href="https://redirect.github.com/rust-cli/env_logger/issues/320">#320</a>
from epage/template</li>
<li><a
href="f8acfee686"><code>f8acfee</code></a>
chore: Update from _rust/main template</li>
<li>Additional commits viewable in <a
href="https://github.com/rust-cli/env_logger/compare/v0.11.3...v0.11.5">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: 1e051c353b7c5ff1e2ab6aaa43a5231858de517a
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.62 to
1.0.63.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/dtolnay/thiserror/releases">thiserror's
releases</a>.</em></p>
<blockquote>
<h2>1.0.63</h2>
<ul>
<li>Documentation improvements</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="915c75e8a3"><code>915c75e</code></a>
Release 1.0.63</li>
<li><a
href="3d5ec25464"><code>3d5ec25</code></a>
Merge pull request <a
href="https://redirect.github.com/dtolnay/thiserror/issues/312">#312</a>
from dtolnay/backtracedoc</li>
<li><a
href="de8a1e54d4"><code>de8a1e5</code></a>
Update documentation of #[from] and #[backtrace] attributes</li>
<li>See full diff in <a
href="https://github.com/dtolnay/thiserror/compare/1.0.62...1.0.63">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: 7c28562af5d512f1be5543cd6f90141136eaf80e
Bumps [clap](https://github.com/clap-rs/clap) from 4.5.9 to 4.5.11.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/clap-rs/clap/releases">clap's
releases</a>.</em></p>
<blockquote>
<h2>v4.5.10</h2>
<h2>[4.5.10] - 2024-07-23</h2>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/clap-rs/clap/blob/master/CHANGELOG.md">clap's
changelog</a>.</em></p>
<blockquote>
<h2>[4.5.11] - 2024-07-25</h2>
<h2>[4.5.10] - 2024-07-23</h2>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="349ed304cf"><code>349ed30</code></a>
chore: Release</li>
<li><a
href="68ed9ff02d"><code>68ed9ff</code></a>
docs: Update changelog</li>
<li><a
href="afecaa4f9a"><code>afecaa4</code></a>
Merge pull request <a
href="https://redirect.github.com/clap-rs/clap/issues/5576">#5576</a>
from shannmu/option_value_new</li>
<li><a
href="4a00677024"><code>4a00677</code></a>
chore: Release</li>
<li><a
href="36c849b5bb"><code>36c849b</code></a>
refactor(clap_complete): Reuse code about shortflag parsing</li>
<li><a
href="fc479ba625"><code>fc479ba</code></a>
feat(clap_complete): Add support for <code>-fbar</code> and
<code>-f=bar</code> completion</li>
<li><a
href="48a23f5c94"><code>48a23f5</code></a>
test(clap_complete): Add test cases for <code>-fbar</code> and
<code>-f=bar</code> completion</li>
<li><a
href="6b18d7725c"><code>6b18d77</code></a>
docs(derive): Acknowledge Vec<Vec<!-- raw HTML omitted -->></li>
<li>See full diff in <a
href="https://github.com/clap-rs/clap/compare/clap_complete-v4.5.9...clap_complete-v4.5.11">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: e10319e4cd77b3518672bc325fc2e924167a45c9
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
We have decided to remove the role emulation feature from engine
altogether.
More details in the RFC -
https://docs.google.com/document/d/1tlS9pqRzLEotLXN_dhjFOeIgbH6zmejOdZTbkkPD-aM/edit
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: e7cb765df5afac6c6d6a05a572a832ce9910cc0b
In preparation for tightening up the various ways in which we construct and work with session variables, I am trying to move the behavior into the same module(s) as the data types, so that we can avoid exposing the internals of data structures in favor of smart constructors and conversions.
The session variable code was split between `Hasura.RQL.Types.Roles`, `Hasura.RQL.Types.Session`, and `Hasura.Session`, with the first two containing most of the data structures (and some logic) and the latter containing the rest of the logic. These files do not interact with the rest of `Hasura.RQL`, though they are depended upon by that namespace.
I have refactored these files into a new namespace, `Hasura.Authentication`. It now looks like this:
1. Role types are now in `Hasura.Authentication.Role`.
2. Header constants were moved from `Hasura.Server.Utils` to `Hasura.Authentication.Headers` (plural) to avoid cycles.
3. Header logic was moved from various places into `Hasura.Authentication.Header` (singular) for the same reason.
4. Session variable types and logic live together in `Hasura.Authentication.Session`.
5. User info types and logic live together in `Hasura.Authentication.User`.
This new structure is cycle-free and generally avoids importing the rest of the code, which means we should be able to start pruning the list of exports and locking down session variable construction.
No behavior was changed in this changeset.
The majority of changes are to the imports in a number of files; everything depends on these things. By splitting into multiple files, we also reduce the surface area of an individual import, which was a pleasant side-effect of this work.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10960
GitOrigin-RevId: 7cb962c06483cd9b92b80432aed5cabecb465cda
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
We've had issues with `metadata-resolve` rejecting Elasticsearch schema
output, so adding said output to this test. Appears to work fine, so
merging it for further discussion and to improve the test case.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Add elastic search schema to test.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: ea7c39ca7ab07fc18abd08eb822d2d56fc152ae6
Bumps
[async-graphql-parser](https://github.com/async-graphql/async-graphql)
from 7.0.6 to 7.0.7.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/async-graphql/async-graphql/blob/master/CHANGELOG.md">async-graphql-parser's
changelog</a>.</em></p>
<blockquote>
<h1>[7.0.7] 2024-07-14</h1>
<ul>
<li>Support raw values from serde_json <a
href="https://redirect.github.com/async-graphql/async-graphql/pull/1554">#1554</a></li>
<li>The custom directive <code>ARGUMENT_DEFINITION</code> is not being
output at the appropriate location in SDL <a
href="https://redirect.github.com/async-graphql/async-graphql/pull/1559">#1559</a></li>
<li>Support for JSON extended representations of BSON ObjectId and Uuid
<a
href="https://redirect.github.com/async-graphql/async-graphql/pull/1542">#1542</a></li>
<li>feat: get directives from SelectionField <a
href="https://redirect.github.com/async-graphql/async-graphql/pull/1548">#1548</a></li>
<li>Support Directives on Subscriptions <a
href="https://redirect.github.com/async-graphql/async-graphql/pull/1500">#1500</a></li>
<li>fix subscription err typo <a
href="https://redirect.github.com/async-graphql/async-graphql/pull/1556">#1556</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/async-graphql/async-graphql/commits">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: 8c0d1a222a57c86cedf2f0870f6cccb0a861a3e6
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
Forgot to do this in last PR.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 24ec7278a63379f252d452b8ec627c934e1c534c
### What
We introduced a newtype around the NDC field alias, but we called it
`NdcFieldName`. While in reality it is the alias of the field requested
in the query.
This PR changes the name to `NdcFieldAlias`.
This is a no-op change
V3_GIT_ORIGIN_REV_ID: 8e892c29860e93243a200b6a6291fd0a32cc6fe3
### What
Part of the point of the `tracing-util` crate is to centrally enforce
usage of a single version of opentelemetry libraries.
Previously we added some support for relaying baggage, but not actually
for defining it. This PR exposes the crates and types necessary to add
baggage to the context.
V3_GIT_ORIGIN_REV_ID: 107ec652d4e812f31bbfaa362cedf44b25dc3c39
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
We have a bunch of local development infra for building the engine
inside a Docker container. This is helpful for Buildkite which doesn't
come with stuff like `cargo` preinstalled. We've not using Buildkite
anymore, let's remove it.
V3_GIT_ORIGIN_REV_ID: b4b7679aab5b14081288df25d139944f160a61fe
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
We'd like to make it simpler to try out DDN, by starting with a mode
that uses no auth.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Add a `NoAuth` `AuthConfig` mode that is configured thus:
```json
"noAuth": {
"role": "admin",
"sessionVariables": {
"x-hasura-user-id": "1"
}
}
```
Given the above config:
- If no `x-hasura-role` is sent with a request, we run it as `admin`.
- If a `x-hasura-role` header is sent and it's `admin`, it continues to
work
- If any other `x-hasura-role` header is sent, an error will happen.
- All other headers are ignored, and we always set `x-hasura-user-id` to
1
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: dddcfbee9c3a31e84dfc8013de32e3a9bf31943d
This PR adds validation code to `metadata_resolve` that prevents someone
from putting a schema/capabilities from the wrong NDC version into the
DataConnectorLink while specifying a different schema version in the
DataConnectorLink. For example:
```
kind: DataConnectorLink
version: v1
definition:
name: data_connector
schema:
version: v0.2
schema: {}
capabilities:
version: 0.1.5 # Not allowed for version v0.2!
capabilities: {}
```
This PR has two commits. One is a refactor where we rearrange the
DataConnectorError types so that the name of the data connector is
captured centrally in `NamedDataConnectorError`, so that it doesn't have
to be passed around and included in every error manually. The other is
the validation changes to `metadata_resolve`.
Completes APIPG-705
V3_GIT_ORIGIN_REV_ID: baed571f36f4cbed824ca546128f5df360d5b298
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
These take ages to run, are slowing development down and not offering
the value they should. We should be benchmarking engine, but not like
this.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Remove benchmarks CI job from Buildkite.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 30a2c9d5f6ba09f5319a07fe394db8becaa16b8e
This PR updates as many tests as possible that use the custom connector
so that the tests run over two versions of the custom connector:
1. The custom connector in the repo, which currently speaks `ndc_models`
v0.2.x
2. The custom connector from the past (commit ), which is the last
version to speak `ndc_models` v0.1.x
This helps us test both the NDC v0.1.x and v0.2.x code paths. When the
postgres connector upgrades to v0.2.x, we can use the same approach as
in this PR to get the tests to run over multiple versions of the
postgres connector too, for much better coverage. This approach with the
custom connector will become less useful over time as the v0.1.x
connector is not updated and will diverge in data from the v0.2.x
connector. The postgres connector is likely to be longer-lasting, as it
is more stable.
The basic test used for `execute` integration tests is
`test_execution_expectation` (in `crates/engine/tests/common.rs`) and it
has been extended into a version called
`test_execution_expectation_for_multiple_ndc_versions` that takes
metadata on a per NDC version basis and then runs the test multiple
times, once for each NDC version. This allows one to swap out the
DataConnectorLink involved in the test to a different one that points at
either the v0.1.x or v0.2.x versions of the connector. The assertion is
that both connectors should produce the same results, even if they talk
a different version of the NDC protocol. As each version runs, we
`println!` the version so that if the test fails you can look in stdout
for the test and see which one was executing when it failed.
Tests that use the custom connector now use
`test_execution_expectation_for_multiple_ndc_versions` and run across
both connector versions. Some tests were unable to be used across both
version as the data between the two versions has changed. Some tests
were modified to avoid the changed data so as to support running across
both versions. Any tests that use `test_execution_expectation_legacy`
don't run across both versions because those tests aren't backed by the
same test implementation as
`test_execution_expectation_for_multiple_ndc_versions`.
Unfortunately the custom connector doesn't use the standard connector
SDK, so it doesn't support `HASURA_CONNECTOR_PORT`. This means that the
old connector is stuck on 8101. To work around this, I've moved the
current connector port to 8102 instead. Technically we might be able to
use docker to remap the ports, but then this binds us into always
running the connectors in docker in order to move their ports around, so
I avoided that approach.
Completes APIPG-703
V3_GIT_ORIGIN_REV_ID: fb0e410ddbee0ea699815388bc63584d6ff5dd70
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
In a recent engine change, we changed some of our trace context mapping
to use the shared settings consistently. However, we needed to make sure
we included `TraceContextResponsePropagator`, which returns the
`traceresponse` header.
Request from console after this fix:
<img width="810" alt="Screenshot 2024-07-25 at 11 58 30"
src="https://github.com/user-attachments/assets/c8e73c56-87fd-49da-a887-f91cdb6d607a">
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Adds `TraceContextResponsePropagator` to the global set of text map
propagators.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 48df6a6fe55e78a48f1dc6bf82304199a0a7e248
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
NDC query request expects relationship names which are unique across the
query.
Previously, we would generate relationship name of the form -
```
[{\"subgraph\":\"connector_2\",\"name\":\"Album\"},\"Tracks\"]
```
This works, but is harder to read while debugging. This PR changes it to
have a human-readable name like -
```
connector_2___Album__Tracks
```
This is a no-op change, apart from the relationship names in NDC query
requests.
### How
Instead of json-ifying the data structure in a tuple, create a formatted
string.
V3_GIT_ORIGIN_REV_ID: 3fea3bf56f1688bc1cade1ea2b3ed6eb60509cac
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
We've had our CI mixed between Github and Buildkite for a while, it's
time to commit. First step is moving the "tests" step to Github Actions.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
This PR:
- Moves the `test` step to Github Actions
- Creates a new `custom_connector.Dockerfile` which builds custom
connector only, more quickly.
- Changes the metadata tests to use `localhost` instead of their Docker
internal names (ie `custom_connector` or `postgres_connector`) - this is
because the tests are being run from outside Docker now
- Removes the `test` Buildkite step
It does not:
- Remove the code coverage or benchmarks steps from Buildkite
- Tidy up `justfile` or Dockerfiles
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
---------
Co-authored-by: Philip Lykke Carlsen <plcplc@gmail.com>
V3_GIT_ORIGIN_REV_ID: a67534ebc1634a24b48d2620c45003221852e199
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
Our CI times are slow, thus merging is slow, and I do not feel this job
pulls it weight.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Remove the code coverage job from Buildkite, delete all associated
files.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 7bbde842f9f405920d1e559b8fef88078cc8d564
Previously the `sql` crate generated a v02 ndc query request and then
downgraded it to v01 if necessary. This is fragile in that its easy to
use v02 ndc features and then get v01 downgrade errors, plus the
downgrade logic is extensive and tedious.
This PR refactors the `sql` crate so that it generates `ir` and `plan`
types and eventually creates `ResolvedQueryExecutionPlan` (rather than
ndc_models types), and then the ResolvedQueryExecutionPlan is
transformed into the appropriate ndc version in the same fashion as the
main engine execute code does it. This eliminates all the downgrade
logic and simplifies things.
Unfortunately, ndc's `QueryRequest` could not just simply be replaced
with `QueryExecutionPlan` on `sql`'s `NDCQuery` and `NDCPushDown`,
because it involves lifetime parameters which are incompatible with the
datafusion framework types. Instead, the individual components of a
query are kept on `NDCQuery` and `NDCPushDown`, and these are eventually
assembled into a `ResolvedQueryExecutionPlan` at a place where the
lifetime parameters are workable. In some sense this is clearer, as one
can now see where each individual part of the query is actually created
and relevant, instead of copying around and mutating a `QueryRequest`.
Completes
https://linear.app/hasura/issue/APIPG-702/implement-separate-logic-that-maps-engine-types-to-ndc-models-types-on
V3_GIT_ORIGIN_REV_ID: c4a9226c1b1addcfe5cd0bca783f1b65ab3ada38
~~Note: this PR is stacked on #845.~~ Rebased on main
This PR refactors the `execute::plan::types` further to make a clear
distinction between unresolved and resolved states. An "unresolved"
state refers to one in which remote predicates have not been computed
into local predicates. A "resolved" state is after this process is
performed and remote predicates are eliminated.
Previously, unresolved types could be passed to
`execute::plan::ndc_request` and they would fail at runtime due to the
presence of unresolved remote predicates. Now, this is impossible due to
a type-level distinction between unresolved and resolve states.
This distinction is made by type-parameterizing all
`execute::plan::types` that involve a predicate so that the predicate
type is parameterized out. Then, an `Unresolved` type alias is created
that sets the predicate type to
`execute::ir::filter::expression::Expression` (which contains remote
predicates) and a `Resolved` type alias is created that uses
`ResolvedFilterExpression` instead (which does not contain remote
predicates).
For example, for `QueryNode`, we now have:
```rust
pub struct QueryNode<'s, TFilterExpression> {
...
pub predicate: Option<TFilterExpression>,
...
}
```
And then the two aliases are:
```rust
pub type UnresolvedQueryNode<'s> = QueryNode<'s, ir::filter::expression::Expression<'s>>;
pub type ResolvedQueryNode<'s> = QueryNode<'s, ResolvedFilterExpression>;
```
Subsequently, `plan::ndc_request` only deals with `Resolved` types.
This is mostly just type-fiddling, but one place some logic moved around
is in with the old `plan::types::FilterExpression`. This was mostly a
functional duplicate of `ir::filter::execute::Expression` except that it
had a "planned" remote predicate variant in it. In order to reduce the
number of types (so we didn't need `UnresolvedFilterExpression` and
`ResolvedFilterExpression`), this type has been repurposed into
`ResolvedFilterExpression` and no longer deals with remote predicates.
Instead, `ir::filter::execute::Expression` is resolved into a
`ResolvedFilterExpression` and the planning of the remote predicate is
done at that time, just before it is resolved. This works fine, since an
entirely new ndc query is performed in order to resolve the predicate,
so planning that can be deferred until then and it doesn't need to be
done at the same time as the main query.
Part of
https://linear.app/hasura/issue/APIPG-702/implement-separate-logic-that-maps-engine-types-to-ndc-models-types-on
V3_GIT_ORIGIN_REV_ID: 3ec89efbaa7b543fad6a100e2739bcc74b1d567f
In rare cases, sensitive headers were written to the GraphQL Engine logs. These cases are now handled correctly to avoid logging sensitive information.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10946
GitOrigin-RevId: a9d66532245789a16cf91936d53bbfce19d272a4
This PR removes usages of ndc_models types from the `ir` types and the
`plan::types` in the `execute` crate. This is done to isolate the IR and
planning logic from the specific version of NDC that needs to be used to
talk to the specific data connector. Once planning is done, the
`plan::types` are mapped into `ndc_models` types via
`plan::ndc_request::v01` and `plan::ndc_request::v02`. Those two modules
contain code that maps `plan::types` to a specific ndc_models version.
This code is entirely separate per version.
Now, when the `plan::types` are `resolve`d, they don't return NDC types.
Now they return themselves, but modified to remove any remote
predicates. After `resolve` the types are in a state that they can be
converted into a single NDC request via `plan::ndc_request`.
Next steps for a future PR: `QueryExecutionPlan` (etc) needs to type
parameterize out the `FilterExpression` type so that after `resolve`, a
new `QueryExecutionPlan` is returned that uses another FilterExpression
type that does not have remote predicates in it. This will make
resolving typesafe and prevent accidentally using `plan::ndc_request` on
unresolved `QueryExecutionPlan`s.
The `sql` crate also needs to stop talking ndc directly and probably
talk `plan::types` instead. Right now it writes ndc v0.2.0 and then
downgrades it to v0.1.0 if necessary, which is awful. Talking
`plan::types` is currently not possible due to lifetime parameters used
on the `plan::types`. These may be able to be removed from the resolved
variant of those types as mentioned above.
Part of
https://linear.app/hasura/issue/APIPG-702/implement-separate-logic-that-maps-engine-types-to-ndc-models-types-on
V3_GIT_ORIGIN_REV_ID: b536009ea784d1486a2ece2262e0ce9d0f937ef0
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
When a boolean expression is passed as an argument it was not being
translated into an `ndc_models::Expression` and so queries failed.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Look up the type of an argument in `metadata-resolve`, and mark the
`ArgumentInfo` with a new `ArgumentKind`. Then in IR step we use that to
work out whether to turn the argument to JSON as before, or translate it
into an `Expression` type that will eventually be turned into an
`ndc_models::Expression`.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 4da3ce0ae04895c33de2b6bdb6fff1018c39b3ad
### What
This PR enables the use of Opentelemetry Baggage. Every bit of baggage
is then replicated on every span.
The current implementation does not actually set any baggage itself - it
only relays and outputs what it's getting.
Crafting a request with a `baggage` header set:
Has it propagated (here, to `dev-auth-webhook`) and stamped onto every
span:
### How
This PR actually makes the engine and auth-hook use the globally
specified propagators (before they would only use an obsucre, concrete,
re-exported one from opentelemetry_contrib), and adds the
`BaggagePropagator` to the list.
It also adds a `SpanProcessor` which outputs the baggage as span
attributes. Currently it outputs all Baggage entries, but can be made
more specific in the future if we want to treat Baggage differently.
V3_GIT_ORIGIN_REV_ID: 3b5b8604b624c0b90c192e68b3b57fab7ca9b63e
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
Sometimes our builds succeed, but we'd like to tell the user how they
could do better. This implements the simplest possible warnings system.
<img width="957" alt="Screenshot 2024-07-18 at 16 06 49"
src="https://github.com/user-attachments/assets/ff91d221-667a-43f9-bc8a-51bf4574a7b8">
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Warnings are printed to stdout on `v3-engine` startup, and will be
returned to the CLI via `v3-metadata-build-service`. The diff is mostly
updated snapshot tests as we're returning more from
`metadata-resolve::resolve` now.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: d01520e53f49d9b594e94a4531b6a86e749875c3
### What
Previously, while generating relationship definitions for NDC, we would
ignore columns with nested selection.
This PR fixes that.
Closes https://hasurahq.atlassian.net/browse/V3ENGINE-247
### How
While matching on `FieldSelection::Column`, don't ignore it. Check if it
contains nested selection, if it does, call
`collect_relationships_from_nested_selection`
V3_GIT_ORIGIN_REV_ID: 9db94744d8e2d35f8430bded07209ef519175205
