<!-- The PR description should answer 2 important questions: -->
### What
Add schemars title and description to `AuthConfig` enum variants.
### How
Use new `opendd` `json_schema` options.
---------
Co-authored-by: Daniel Harvey <danieljamesharvey@gmail.com>
V3_GIT_ORIGIN_REV_ID: 8896357dc8c400101587e36dfb14f807cbdc4bc0
<!-- The PR description should answer 2 important questions: -->
### What
Even though aggregates are not pushed down right now, this makes sure
they work at the datafusion layer and verifies the plans.
### How
Another group of SQL test files
V3_GIT_ORIGIN_REV_ID: a2a29d6f6347bb1b028c06313e98ed16bb7172a0
<!-- The PR description should answer 2 important questions: -->
### What
Supports calling functions from the SQL interface
### How
Similar to models.
V3_GIT_ORIGIN_REV_ID: 2958aeacdfd31bae0e4353cb7e20e627c84931b5
<!-- The PR description should answer 2 important questions: -->
### What
Adds tests for select, filter and order by.
### How
Reuses the existing test framework. There is a giant `metadata.json`
file which is used for all tests, based on Postgres. Each test is a
folder with a `query.sql` file, an `expected.json` for expected output,
and a `plan.json` for the expected explain output.
V3_GIT_ORIGIN_REV_ID: 7f5c134c5d3cf47e5f2ffa305f24f4274ccd545e
<!-- The PR description should answer 2 important questions: -->
### What
Push down the `_lt`, `_lte`, `_gt`, `_gte` operators
### How
By matching operators with the same name on the NDC side, for now, until
we have additional NDC operator meanings that we can use.
V3_GIT_ORIGIN_REV_ID: 4341490a3cdbb62e9fe90c10279527716687545d
<!-- The PR description should answer 2 important questions: -->
### What
Implements a new optimizer pass which pushes sort stages inside
`ModelQuery` stages.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 2820f88003aec376b71605c0f753d7b50825ddad
### What
Push down the following SQL predicates to NDC via OpenDD IR:
- Logical operators AND, OR and NOT
- Equality operator and inequality operator
TODO:
- [ ] Comparison operators
- [x] `IS NULL` and `IS NOT NULL`
- [ ] Validate operators actually exist in the OpenDD metadata and NDC
mappings
- [ ] Use the actual OpenDD operators instead of the stand-in `_eq` and
`_neq` operators
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
### How
- `plan/filter.rs` implements two functions `can_pushdown_filter` and
`pushdown_filter` (which translates to OpenDD IR)
- `planner/filter.rs` translates OpenDD IR to NDC IR for execution.
V3_GIT_ORIGIN_REV_ID: ee2c38bf2c36292710becbabfaca51ccae983a2f
### What
Update dependencies in preparation for some cloud work, and move
dependency versions to the workspace.
### How
```
$ cargo update
```
V3_GIT_ORIGIN_REV_ID: a42ba0c8a1cb376280f9771abd7d1f4f7bf19b84
<!-- The PR description should answer 2 important questions: -->
### What
This is a no-op change. Moves model related planning code from the
top-level to `planner` submodule. This is in preparation for commands
implementation.
V3_GIT_ORIGIN_REV_ID: 97a73ced40dadf168efc1147e52ef4f36103bc50
### What
We now support cloud-only crates, which are not open-sourced.
### How
Anything in `crates/cloud` will not be synced with the _graphql-engine_
repository.
In order to facilitate this, we generate and commit a
Cargo.toml/Cargo.lock pair with the cloud-only sections removed. We also
transform the justfile to remove this code.
This includes only a test repository, to ensure that nothing private is
synced.
When this is merged, it should not result in a commit to the
_graphql-engine_ repository.
V3_GIT_ORIGIN_REV_ID: 038839acdf3a97da05bbd4b6278171cc12e7cd71
<!-- The PR description should answer 2 important questions: -->
### What
Introduce the `json_schema` attribute for enum variants to specify
schema metadata for adjacently tagged (`as_versioned_with_definition`)
enums. For untagged and internally tagged enums, metadata is inherited
from the corresponding variant type.
Also, update the `README.md` in `opendd-derive` crate.
### How
Update types and generate metadata expression for adjacent tagged enums.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 5c7dff5c7d7f4f44f0a27e26f1d8941fb861c346
<!-- The PR description should answer 2 important questions: -->
### What
Remove the projection pushdown optimization. `datafusion` already
optimizes this to the correct NDC IR.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 6e0034e4d920c39b70667f5f521341069a5c53de
<!-- The PR description should answer 2 important questions: -->
### What
By default, `rustc` splits crates up and builds in parallel. This is
faster but misses some optimisations between these sections. Let's
reduce it to increase runtime performance.
### How
Add settings to `release` profile in `Cargo.toml`.
<img width="776" alt="Screenshot 2024-08-13 at 12 59 46"
src="https://github.com/user-attachments/assets/a03389dc-80ba-4723-8ca3-36af50846324">
V3_GIT_ORIGIN_REV_ID: 44fa511024140b680b30c9abfaa48034bf0845a9
<!-- The PR description should answer 2 important questions: -->
### What
We cannot support having relationship comparisons in nested object field
filters of a boolean expression. NDC does not support this natively
([slack
thread](https://hasurahq.slack.com/archives/C05HND0F6LB/p1722845028319099)).
This PR adds a metadata build check for this case and reject such
boolean expressions.
Tests in the PR is partially based on
https://github.com/hasura/v3-engine/pull/935.
**Note:** This might break older builds having relationship comparisons
in the nested object field filter expressions. This is a rare scenario.
We will check through our schema-diff job for any failing builds. If
there are significant, we might hold this PR. Full context in this slack
thread -
https://hasurahq.slack.com/archives/C06P2U8U55G/p1723121764332539.
### How
- Add a metadata build check for boolean expressions that restricts them
to have nested object filters with relationship comparisons.
- Raise GraphQL API runtime internal error while building the filter IR
when a relationship comparison found within a nested field filter.
---------
Co-authored-by: Daniel Chambers <daniel@hasura.io>
V3_GIT_ORIGIN_REV_ID: 86dce0b784c77e57bb1888125dba5b599e40f741
<!-- The PR description should answer 2 important questions: -->
### What
We started using `mimalloc` allocator in MBS a while ago with good
results, let's use it here too.
Once `v3-engine-multitenant` is merged we should use it there too.
### How
Import crate, switch it on in engine binary and in benchmarks.
<img width="845" alt="Screenshot 2024-08-13 at 10 10 49"
src="https://github.com/user-attachments/assets/dc872668-1633-468a-86d3-51fca5be68bf">
V3_GIT_ORIGIN_REV_ID: ebad91bb57964477d0f227e341c7bd12d54f0f68
<!-- The PR description should answer 2 important questions: -->
### What
Breaking down the big `Error` type more. This creates `ModelsError`.
Functional no-op.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
### How
Mostly moving things and making types more specific. In one case, making
a type more general (added comments on how to resolve this in future)
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 79b14085ca1111489621b8b7d3490492a4c30ead
<!-- The PR description should answer 2 important questions: -->
### What
Updates datafusion dependency to `41`.
### How
Fixes for breaking changes.
V3_GIT_ORIGIN_REV_ID: cc957f6c7ff9b3c004cc5e0cbb48387c1f963b7a
### What
Introspection queries (on 'hasura' schema) would fail when there is no
data in the underlying tables.
### How
A more robust 'MemTable' with a comprehensive set of tests is introduced
which shouldn't run into these issues.
V3_GIT_ORIGIN_REV_ID: e09de03e8d093fb4348514cfed6b6dc1d9b0b0c8
<!-- The PR description should answer 2 important questions: -->
### What
- Add columns with nested fields to the SQL schema
- Alias nested fields appropriately in order to support them for query
execution
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
### How
- Translate OpenDD types to Arrow types during schema generation
(`to_arrow_type`)
- Generate `NestedField` structures during planning to prepare data in
the right format during execution (`fields_for`)
V3_GIT_ORIGIN_REV_ID: d37d2eade2fd5c0f08861c1bbc6368a88299b0f3
<!-- The PR description should answer 2 important questions: -->
### What
Renables Github benchmarking after we removed it in
https://github.com/hasura/v3-engine/pull/819
### How
Tell Criterion to only sample for 5 seconds each time to stop each
benchmark going on forever. This makes the whole run take a reasonable
10 minutes.
V3_GIT_ORIGIN_REV_ID: 364be6490f4f4b21877849daf1f734fa51ecf542
<!-- The PR description should answer 2 important questions: -->
### What
Upgrade to [Rust
1.80.0](https://blog.rust-lang.org/2024/07/25/Rust-1.80.0.html)
### How
Update `rust-toolchain.yaml` and Dockerfiles, fix warnings.
V3_GIT_ORIGIN_REV_ID: ba797e1aba6b9623a921734473a6b70a2a38c8b7
<!-- The PR description should answer 2 important questions: -->
### What
When querying a table with no data through SQL would result in an error.
### How
Instead of returning a `RecordBatch`, arrow_json's implementation
returns an `Option<RecordBatch>`, we now account for `None`.
V3_GIT_ORIGIN_REV_ID: 459440e82aeb1b2faa009405e025fc024497d5b4
Bumps [syn](https://github.com/dtolnay/syn) from 2.0.72 to 2.0.74.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/dtolnay/syn/releases">syn's
releases</a>.</em></p>
<blockquote>
<h2>2.0.74</h2>
<ul>
<li>Fix <em>"temporary is dropped and runs the destructor for type
`impl Iterator`"</em> regression affecting certain use of
<code>Generics</code> iterator methods (<a
href="https://redirect.github.com/dtolnay/syn/issues/1719">#1719</a>)</li>
</ul>
<h2>2.0.73</h2>
<ul>
<li>Support parsing unnamed C varargs within function pointer types (<a
href="https://redirect.github.com/dtolnay/syn/issues/1711">#1711</a>)</li>
<li>Improve synthesized error message on unexpected tokens at the end of
the expected contents of a delimited group (<a
href="https://redirect.github.com/dtolnay/syn/issues/1713">#1713</a>)</li>
<li>Support parsing unstable tail call syntax (<a
href="https://redirect.github.com/dtolnay/syn/issues/1714">#1714</a>, <a
href="https://redirect.github.com/rust-lang/rust/issues/112788">rust-lang/rust#112788</a>)</li>
<li>Add <a
href="https://docs.rs/syn/2.0.73/syn/enum.Fields.html#method.members"><code>Fields::members</code></a>
iterator (<a
href="https://redirect.github.com/dtolnay/syn/issues/1716">#1716</a>,
thanks <a
href="https://github.com/Fancyflame"><code>@Fancyflame</code></a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="b15ae28409"><code>b15ae28</code></a>
Release 2.0.74</li>
<li><a
href="6cf2344440"><code>6cf2344</code></a>
Merge pull request <a
href="https://redirect.github.com/dtolnay/syn/issues/1719">#1719</a>
from dtolnay/iterators</li>
<li><a
href="2955ac55b7"><code>2955ac5</code></a>
Expose non-impl-Trait iterator return types</li>
<li><a
href="7dc05a5643"><code>7dc05a5</code></a>
Add regression test for issue 1718</li>
<li><a
href="857942e411"><code>857942e</code></a>
Revert "Define Fields::members iterator type privately inside
method"</li>
<li><a
href="65ec30d659"><code>65ec30d</code></a>
Revert "Embed Generics iterator implementations into method
bodies"</li>
<li><a
href="b5a5a8c177"><code>b5a5a8c</code></a>
Release 2.0.73</li>
<li><a
href="8cdb5c734d"><code>8cdb5c7</code></a>
Add example of using Fields::members iterator</li>
<li><a
href="2afdc12674"><code>2afdc12</code></a>
Remove redundancy of Generics::split_for_impl cfg with type's cfg</li>
<li><a
href="64b0dc8d0d"><code>64b0dc8</code></a>
Combine Generics impl blocks into one</li>
<li>Additional commits viewable in <a
href="https://github.com/dtolnay/syn/compare/2.0.72...2.0.74">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: 79522dd492c62fe9ae9260e6553847845859d7ce
Bumps [indexmap](https://github.com/indexmap-rs/indexmap) from 2.2.6 to
2.3.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/indexmap-rs/indexmap/blob/master/RELEASES.md">indexmap's
changelog</a>.</em></p>
<blockquote>
<h2>2.3.0</h2>
<ul>
<li>Added trait <code>MutableEntryKey</code> for opt-in mutable access
to map entry keys.</li>
<li>Added method <code>MutableKeys::iter_mut2</code> for opt-in mutable
iteration of map
keys and values.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="22c0b4e0f3"><code>22c0b4e</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/335">#335</a>
from epage/mut</li>
<li><a
href="39f7cc097a"><code>39f7cc0</code></a>
Release 2.3.0</li>
<li><a
href="6049d518a0"><code>6049d51</code></a>
feat(map): Add MutableKeys::iter_mut2</li>
<li><a
href="65c3c46e37"><code>65c3c46</code></a>
feat(map): Add MutableEntryKey</li>
<li><a
href="7f7d39f734"><code>7f7d39f</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/332">#332</a>
from waywardmonkeys/missing-indentation-in-doc-comment</li>
<li><a
href="8222a59a85"><code>8222a59</code></a>
Fix missing indentation in doc comment.</li>
<li><a
href="1a71dde63d"><code>1a71dde</code></a>
Merge pull request <a
href="https://redirect.github.com/indexmap-rs/indexmap/issues/327">#327</a>
from waywardmonkeys/dep-update-dev-dep-itertools</li>
<li><a
href="ac2a8a5a40"><code>ac2a8a5</code></a>
deps(dev): Update <code>itertools</code></li>
<li>See full diff in <a
href="https://github.com/indexmap-rs/indexmap/compare/2.2.6...2.3.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: a1bcf216fdc94b327c7f078046478f40d0eed4e8
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.122 to
1.0.124.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/serde-rs/json/releases">serde_json's
releases</a>.</em></p>
<blockquote>
<h2>v1.0.124</h2>
<ul>
<li>Fix a bug in processing string escapes in big-endian architectures
(<a
href="https://redirect.github.com/serde-rs/json/issues/1173">#1173</a>,
thanks <a
href="https://github.com/purplesyringa"><code>@purplesyringa</code></a>)</li>
</ul>
<h2>v1.0.123</h2>
<ul>
<li>Optimize string parsing by applying SIMD-within-a-register: 30.3%
improvement on <a
href="https://github.com/miloyip/nativejson-benchmark/blob/v1.0.0/data/twitter.json">twitter.json</a>
from 613 MB/s to 799 MB/s (<a
href="https://redirect.github.com/serde-rs/json/issues/1161">#1161</a>,
thanks <a
href="https://github.com/purplesyringa"><code>@purplesyringa</code></a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="cf771a0471"><code>cf771a0</code></a>
Release 1.0.124</li>
<li><a
href="8b314a77bf"><code>8b314a7</code></a>
Merge pull request <a
href="https://redirect.github.com/serde-rs/json/issues/1173">#1173</a>
from iex-rs/fix-big-endian</li>
<li><a
href="8eba7863b1"><code>8eba786</code></a>
Fix skip_to_escape on BE architectures</li>
<li><a
href="2cab07e686"><code>2cab07e</code></a>
Release 1.0.123</li>
<li><a
href="346189a524"><code>346189a</code></a>
Fix needless_borrow clippy lint in new control character test</li>
<li><a
href="859ead8e6d"><code>859ead8</code></a>
Merge pull request <a
href="https://redirect.github.com/serde-rs/json/issues/1161">#1161</a>
from iex-rs/vectorized-string-parsing</li>
<li><a
href="e43da5ee0e"><code>e43da5e</code></a>
Immediately bail-out on empty strings</li>
<li><a
href="8389d8a112"><code>8389d8a</code></a>
Don't run the slow algorithm from the beginning</li>
<li><a
href="1f0dcf791a"><code>1f0dcf7</code></a>
Allow clippy::items_after_statements</li>
<li><a
href="a95d6df9d0"><code>a95d6df</code></a>
Big endian support</li>
<li>Additional commits viewable in <a
href="https://github.com/serde-rs/json/compare/v1.0.122...v1.0.124">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: bf3ef8f9c7404cc0dafb1e4d4dd5fed896061971
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.204 to
1.0.206.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/serde-rs/serde/releases">serde's
releases</a>.</em></p>
<blockquote>
<h2>v1.0.206</h2>
<ul>
<li>Improve support for <code>flatten</code> attribute inside of enums
(<a
href="https://redirect.github.com/serde-rs/serde/issues/2567">#2567</a>,
thanks <a
href="https://github.com/Mingun"><code>@Mingun</code></a>)</li>
</ul>
<h2>v1.0.205</h2>
<ul>
<li>Use serialize_entry instead of serialize_key + serialize_value when
serialize flattened newtype enum variants (<a
href="https://redirect.github.com/serde-rs/serde/issues/2785">#2785</a>,
thanks <a
href="https://github.com/Mingun"><code>@Mingun</code></a>)</li>
<li>Avoid triggering a collection_is_never_read lint in the
deserialization of enums containing flattened fields (<a
href="https://redirect.github.com/serde-rs/serde/issues/2791">#2791</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="85c73ef8de"><code>85c73ef</code></a>
Release 1.0.206</li>
<li><a
href="5ba1796a7e"><code>5ba1796</code></a>
Resolve doc_markdown pedantic lint on regression test function</li>
<li><a
href="e52b7b380f"><code>e52b7b3</code></a>
Touch up PR 2567</li>
<li><a
href="84c7419652"><code>84c7419</code></a>
Merge pull request <a
href="https://redirect.github.com/serde-rs/serde/issues/2794">#2794</a>
from dtolnay/neverread</li>
<li><a
href="536221b1f9"><code>536221b</code></a>
Temporarily ignore collection_is_never_read on
FlattenSkipDeserializing</li>
<li><a
href="fc55ac70d3"><code>fc55ac7</code></a>
Merge pull request <a
href="https://redirect.github.com/serde-rs/serde/issues/2567">#2567</a>
from Mingun/fix-2565</li>
<li><a
href="2afe5b4ef9"><code>2afe5b4</code></a>
Add regression test for issue <a
href="https://redirect.github.com/serde-rs/serde/issues/2792">#2792</a></li>
<li><a
href="b4ec2595c9"><code>b4ec259</code></a>
Correctly process flatten fields in enum variants</li>
<li><a
href="c3ac7b675a"><code>c3ac7b6</code></a>
Add regression test for issue <a
href="https://redirect.github.com/serde-rs/serde/issues/1904">#1904</a></li>
<li><a
href="24614e44bf"><code>24614e4</code></a>
Add regression test for issue <a
href="https://redirect.github.com/serde-rs/serde/issues/2565">#2565</a></li>
<li>Additional commits viewable in <a
href="https://github.com/serde-rs/serde/compare/v1.0.204...v1.0.206">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: 50cd5a67d87a37060b2197f1355699d5701df396
<!-- The PR description should answer 2 important questions: -->
### What
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
Just an improvement to the middleware plugin
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
We now take a nonempty list of plugins. This ensures that we only do
things if we have plugins.
V3_GIT_ORIGIN_REV_ID: c8fb548f763cdefe3526c67d7c801104ad5c527a
<!-- The PR description should answer 2 important questions: -->
### What
We want to store the pre-plugins as artifacts. For this, we need to get
the list of pre-plugins while building the artifacts.
### How
This can be achieved by extracting the pre-plugins during the build
step.
---------
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: 64e8697d90092acad0cb8a338becb7868af78350
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
This PR introduces a new OpenDD object of kind `LifecyclePluginHook`. An
example
```json
{
"kind": "LifecyclePluginHook",
"version": "v1",
"definition": {
"pre": "parse",
"name": "test",
"url": "http://localhost:8787",
"config": {
"request": {
"headers": {
"additional": {
"hasura-m-auth": {
"value": "zZkhKqFjqXR4g5MZCsJUZCnhCcoPyZ"
}
}
},
"session": {},
"rawRequest": {
"query": {},
"variables": {}
}
}
}
}
}
```
The plugin configs (only pre-parse plugins for now) are stored in the
engine state and used wherever required.
### How
We have added the OpenDD object.
V3_GIT_ORIGIN_REV_ID: aa02315362e5fc9a36b63ead48909e1baa92779f
### What
Add a check for presence of descriptions in JSON schema. Add missing
descriptions for types.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
- Extend the `validate_root_json_schema` utility with a check for
descriptions.
- Add a doc comment for types that need a description in their JSON
schema.
---------
Co-authored-by: Abhinav Gupta <127770473+abhinav-hasura@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: 5c411aa0cf33ac1fde076c29020edd4957fbc27c
### What
Try not to pollute the CI cache with the wrong thing.
### How
1. Remove the package selector as we don't care about production builds
here any more.
2. Tell the test build to not save the cache so there is no write
contention.
3. Downgrade `mockito` to v1.4 reduce the size of the cache, because
`v1.5` depends on `http` v1.
V3_GIT_ORIGIN_REV_ID: 2109c8c7db5d80e3b2c29d2949423e8faebd10b2
<!-- The PR description should answer 2 important questions: -->
### What
Resolve a TODO from boolean expressions stage.
There is a small chance this cause a build error - will check the schema
diff tests before merging.
### How
Use `store_new_graphql_type` on the graphql types of boolean
expressions.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 0498f8f3480ef2d5bbdbb57a0e50cae3bbdef868
<!-- The PR description should answer 2 important questions: -->
### What
Avoid some unnecessary cloning during metadata resolve.
### How
Some data structures, such as `graphql_types` which are updated by
various stages of metadata resolve, were being unnecessarily cloned.
Instead of cloning from an immutable reference, we now move the value
into each stage and return it as part of the stage output.
V3_GIT_ORIGIN_REV_ID: 067698c3e004c70165fb0a8190542115a9f6cfb6
### What
- Added `AuthConfig` v2 config example in
`static/auth/auth_config_v2.json`
- Moved exisiting `auth_config.json` to `static/auth/`
- Removed unused `pre_plugins.json`
If one wants to start the engine with a v2 of AuthConfig,
`static/auth/auth_config_v2.json` can be used.
V3_GIT_ORIGIN_REV_ID: 471f8ae43ab02c2182457804a24b8445bb41f06c
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
- Add metadata resolver for the new `OrderByExpression` metadata object
kind.
- When resolving a v1 `Model`, generate an `OrderByExpression` based on
the model's `orderable_fields`.
### How
`OrderByExpression` was added to OpenDD in
https://github.com/hasura/v3-engine/pull/780. This PR adds metadata
resolvers for this new metadata object kind. The OpenDD changes involved
a new version v2 of `Model` where `orderableFields` was moved out of the
out of the model definition. The model now references an
`OrderByExpression` instead.
To retain backwards compatibility with `Model` v1, the model metadata
resolver now extracts the `orderableFields` from a v1 model and uses
them to create a new `OrderByExpression` for internal use by the
resolved metadata.
V3_GIT_ORIGIN_REV_ID: a7dbafe860e586efdb2e03c23020a067011c57a1
Specifically, DataConnectorColumnName and DataConnectorName which are
wrappers on `SmolStr` and hence cheap to clone.
We want to use `Plan` as a physical node in the sql layer but given
datafusion's architecture, a physical node cannot contain references.
This is a small PR towards this effort.
V3_GIT_ORIGIN_REV_ID: 284dcfb4e8e7ce83705b415611c22e8a6e25e4be
### What
Added all relevant W3C and Zipkin/B3 trace response headers, to exposed
headers for CORS.
The headers list (as pointed out by Samir) -
For Zipkin/B3:
- `X-B3-TraceId`
- `X-B3-SpanId`
- `X-B3-ParentSpanId`
- `X-B3-Sampled`
For W3C:
- `traceparent`
- `tracestate`
This is generally useful for any client accessing the API to retrieve
tracing information.
### How
Created a constant array of relevant header names. And initialize the
CORS middleware, with these as exposed headers.
V3_GIT_ORIGIN_REV_ID: c7aaf2507b03e1897971ca6cd2bbaa06b08dfa52
<!-- The PR description should answer 2 important questions: -->
### What
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
Fields involved in the relationship's inner predicates were incorrectly
reported as fields of the root model. This PR resolves the issue. Also,
fixes the predicates inside `And` or `Or` are not reported.
Note: Changelog not required, as query usage analytics are Hasura
internal and hidden from users.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
- Use `for` loop instead of `Iterator map` to avoid confusion around the
execution of lambda passed to the `map` function (more context in this
[slack](https://hasurahq.slack.com/archives/C04PUMV4X16/p1722871834852519)
thread)
- Introduce a new struct, to report predicate relationship fields, that
has a field to report its inner filter predicate usage.
V3_GIT_ORIGIN_REV_ID: 9ca23e6005ccb09f2321a2ae30ef575f99e84e06
- `DataConnectorAggregationFunctionName` and `AggregateFunctionName` now
use `str_newtype`.
- All usages of `String`s for subgraph names are removed.
(This is part of a larger effort to remove references in
`execute::plan::QueryPlan`).
V3_GIT_ORIGIN_REV_ID: d51f0a2335e8dabbc9efdad1d1efff285ddb74c3
<!-- The PR description should answer 2 important questions: -->
### What
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
When reporting query usage analytics, mention whether a field is
deprecated.
Note: The changelog is not required, as the usage analytics are for
Hasura internal use.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
OpenDd allows marking an ObjectType's field as deprecated with an
optional reason. Plumb the deprecation context to the input/output
schema annotation. Report the field usage with a deprecated boolean
field.
V3_GIT_ORIGIN_REV_ID: 430cdcf3e1ff0c43812caecb8d06a64b729665be
### What
Add a flag `--export-traces-stdout` to log traces to stdout. Default is
disabled.
Command-line flag - `--export-traces-stdout`
Env var - `EXPORT_TRACES_STDOUT`
### How
Introduce a new command line flag. Make `initialize_tracing` accept a
`bool`, and setup the stdout exporter based on that.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: f39d6f863fd2bca65ad89f1cef4b077aa9eabc5b
Bumps [regex](https://github.com/rust-lang/regex) from 1.10.5 to 1.10.6.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/rust-lang/regex/blob/master/CHANGELOG.md">regex's
changelog</a>.</em></p>
<blockquote>
<h1>1.10.6 (2024-08-02)</h1>
<p>This is a new patch release with a fix for the <code>unstable</code>
crate feature that
enables <code>std::str::Pattern</code> trait integration.</p>
<p>Bug fixes:</p>
<ul>
<li>[BUG <a
href="https://redirect.github.com/rust-lang/regex/issues/1219">#1219</a>](<a
href="https://redirect.github.com/rust-lang/regex/pull/1219">rust-lang/regex#1219</a>):
Fix the <code>Pattern</code> trait implementation as a result of nightly
API breakage.</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="ab88aa5c68"><code>ab88aa5</code></a>
1.10.6</li>
<li><a
href="76f2d30d85"><code>76f2d30</code></a>
changelog: 1.10.6</li>
<li><a
href="2970d2940d"><code>2970d29</code></a>
unstable: fit <code>Pattern</code> trait implementation</li>
<li><a
href="8856fe36ac"><code>8856fe3</code></a>
regex-cli-0.2.1</li>
<li><a
href="c4c76a19b8"><code>c4c76a1</code></a>
regex-test-0.1.1</li>
<li><a
href="c2f9ca49a6"><code>c2f9ca4</code></a>
regex-test: bump toml dependency</li>
<li><a
href="1288b83af3"><code>1288b83</code></a>
regex-lite-0.1.6</li>
<li>See full diff in <a
href="https://github.com/rust-lang/regex/compare/1.10.5...1.10.6">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: 527f0e7680e48f2c97e6e36acc28fab7fcc82e2a
Bumps [mockito](https://github.com/lipanski/mockito) from 1.4.0 to
1.5.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/lipanski/mockito/releases">mockito's
releases</a>.</em></p>
<blockquote>
<h2>1.5.0</h2>
<ul>
<li><strong>[Breaking]</strong> <a
href="https://redirect.github.com/lipanski/mockito/pull/198">Upgrade</a>
to hyper v1</li>
</ul>
<p>Thanks to <a
href="https://github.com/tottoto"><code>@tottoto</code></a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="f1c3fe1b7f"><code>f1c3fe1</code></a>
Bump to 1.5.0</li>
<li><a
href="08f2fa322d"><code>08f2fa3</code></a>
Merge pull request <a
href="https://redirect.github.com/lipanski/mockito/issues/199">#199</a>
from tottoto/refactor-response-body</li>
<li><a
href="42e3efe734"><code>42e3efe</code></a>
Refactor response body</li>
<li><a
href="f477e54857"><code>f477e54</code></a>
Merge pull request <a
href="https://redirect.github.com/lipanski/mockito/issues/198">#198</a>
from tottoto/update-to-hyper-1</li>
<li><a
href="e8694ae991"><code>e8694ae</code></a>
Update to hyper 1</li>
<li><a
href="b152b76130"><code>b152b76</code></a>
Depend on some crate directly</li>
<li>See full diff in <a
href="https://github.com/lipanski/mockito/compare/1.4.0...1.5.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: db7b12bcbffe55aa820bedd72ba517d9f2152bf2
Bumps [clap](https://github.com/clap-rs/clap) from 4.5.11 to 4.5.13.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/clap-rs/clap/releases">clap's
releases</a>.</em></p>
<blockquote>
<h2>v4.5.13</h2>
<h2>[4.5.13] - 2024-07-31</h2>
<h3>Fixes</h3>
<ul>
<li><em>(derive)</em> Improve error message when
<code>#[flatten]</code>ing an optional <code>#[group(skip)]</code></li>
<li><em>(help)</em> Properly wrap long subcommand descriptions in
help</li>
</ul>
<h2>v4.5.12</h2>
<h2>[4.5.12] - 2024-07-31</h2>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/clap-rs/clap/blob/master/CHANGELOG.md">clap's
changelog</a>.</em></p>
<blockquote>
<h2>[4.5.13] - 2024-07-31</h2>
<h3>Fixes</h3>
<ul>
<li><em>(derive)</em> Improve error message when
<code>#[flatten]</code>ing an optional <code>#[group(skip)]</code></li>
<li><em>(help)</em> Properly wrap long subcommand descriptions in
help</li>
</ul>
<h2>[4.5.12] - 2024-07-31</h2>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="66b5a122ec"><code>66b5a12</code></a>
chore: Release</li>
<li><a
href="a83b724fc1"><code>a83b724</code></a>
docs: Update changelog</li>
<li><a
href="e19469f0e2"><code>e19469f</code></a>
Merge pull request <a
href="https://redirect.github.com/clap-rs/clap/issues/5615">#5615</a>
from epage/wrap</li>
<li><a
href="360bb3bc11"><code>360bb3b</code></a>
fix(help): Properly wrap subcommand about</li>
<li><a
href="58b5bd89ea"><code>58b5bd8</code></a>
test(help): Show bad subcommand wrapping</li>
<li><a
href="e1f6320730"><code>e1f6320</code></a>
Merge pull request <a
href="https://redirect.github.com/clap-rs/clap/issues/5614">#5614</a>
from epage/snap</li>
<li><a
href="7b5b61b021"><code>7b5b61b</code></a>
test(help): Switch to snapbox</li>
<li><a
href="1490e2434a"><code>1490e24</code></a>
chore: Update snapbox</li>
<li><a
href="6671d93500"><code>6671d93</code></a>
Merge pull request <a
href="https://redirect.github.com/clap-rs/clap/issues/5613">#5613</a>
from epage/flatten</li>
<li><a
href="13dfdb3870"><code>13dfdb3</code></a>
fix(derive): Improve flattening-skipped-group assert</li>
<li>Additional commits viewable in <a
href="https://github.com/clap-rs/clap/compare/clap_complete-v4.5.11...v4.5.13">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: 898ddf94dc4357f35e50b1599dedef672c18cd8a
Bumps [serde_with](https://github.com/jonasbb/serde_with) from 3.8.3 to
3.9.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/jonasbb/serde_with/releases">serde_with's
releases</a>.</em></p>
<blockquote>
<h2>serde_with v3.9.0</h2>
<h3>Added</h3>
<ul>
<li>
<p>Deserialize a map` and skip all elements failing to deserialize by <a
href="https://github.com/johnmave126"><code>@johnmave126</code></a> (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/763">#763</a>)</p>
<p><code>MapSkipError</code> acts like a map
(<code>HashMap</code>/<code>BTreeMap</code>), but keys or values that
fail to deserialize, like are ignored.</p>
<p>For formats with heterogeneously typed maps, we can collect only the
elements where both key and value are deserializable.
This is also useful in conjunction to <code>#[serde(flatten)]</code> to
ignore some entries when capturing additional fields.</p>
<pre lang="text"><code>// JSON
"value": {"0": "v0", "5":
"v5", "str": "str", "10": 2},
<p>// Rust
#[serde_as(as = "MapSkipError<DisplayFromStr, _>")]
value: BTreeMap<u32, String>,</p>
<p>// Only deserializes entries with a numerical key and a string value,
i.e.,
{0 => "v0", 5 => "v5"}
</code></pre></p>
</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="c3e489f4e5"><code>c3e489f</code></a>
Bump version to 3.9.0 (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/770">#770</a>)</li>
<li><a
href="57ad8778c4"><code>57ad877</code></a>
Bump version to 3.9.0</li>
<li><a
href="d038657903"><code>d038657</code></a>
Implement <code>MapSkipError</code>, analogous to
<code>VecSkipError</code>, but for map-like data ...</li>
<li><a
href="aaa0a29589"><code>aaa0a29</code></a>
Update serde_with/src/guide/serde_as_transformations.md</li>
<li><a
href="17dec1112b"><code>17dec11</code></a>
Add tests to make sure syntax errors are not suppressed.</li>
<li><a
href="97543d0c2d"><code>97543d0</code></a>
Implement MapSkipError, skipping un-deserializable entries.</li>
<li><a
href="bf6724d2c0"><code>bf6724d</code></a>
Move VecSkipError to a separate file, preparing for MapSkipError</li>
<li><a
href="31e9172cab"><code>31e9172</code></a>
Fix dead code warning by correcting a typo (<a
href="https://redirect.github.com/jonasbb/serde_with/issues/769">#769</a>)</li>
<li><a
href="f5b2626307"><code>f5b2626</code></a>
Fix dead code warnings in tests</li>
<li><a
href="4ad4a1b4e9"><code>4ad4a1b</code></a>
Fix dead code warning by correcting a typo</li>
<li>See full diff in <a
href="https://github.com/jonasbb/serde_with/compare/v3.8.3...v3.9.0">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: ec2c5b357a416492e2b8033138aa83075023089d
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
We would like to generate operation_name level metrics with execution
latency. Right now, the operation_name is part of validate span which
isn't really doing anything while `execute_query` is the parent span
which will represent the operation time.
<!-- Consider: do we need to add a changelog entry? -->
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
This PR adds `operation_name` to `execute_query` span
V3_GIT_ORIGIN_REV_ID: fc14d92c66b0245739d672b7570be1871243f241
<!-- The PR description should answer 2 important questions: -->
### What
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
Fixes a bug where queries with nested relationship selection and filter
predicates fail due to an issue with NDC relationship collection.
```graphql
query MyQuery {
Album {
AlbumId
Title
ArtistId
Tracks {
AlbumId
Name
TrackId
}
}
}
```
A selection permission defined on the `Tracks` model with a relationship
comparison in the predicate.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
- Previously, the collection of relationships occurred independently by
traversing through the IR AST. Consequently, during planning, the
collection of local relationships was explicitly ignored. This caused
confusion and resulted in the omission of relationship collectors when
planning nested selections for local relationships, leading to the
issue.
- In this PR, the independent collection of relationships is removed.
Instead, all NDC relationships for field selection, filter, and
permission predicates are now collected during planning. This unifies
the logic, and ensures consistency in achieving the same purpose.
V3_GIT_ORIGIN_REV_ID: cbd5bfef7a90a7d7602061a9c733ac54b764e0d3
<!-- The PR description should answer 2 important questions: -->
### What
Trying to understand what is going on here. Still no closer, but have
added a test and made some types more specific in order to clarify my
understanding.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
### How
Add some introspection tests for relationships with
`ObjectBooleanExpressionType`s to ensure they generate. Tried to make
relationship fields disappear to recreate build problems but could not.
Split `BooleanExpressionGraphqlConfig` and
`ObjectBooleanExpressionGraphqlConfig` to make sure we're not mixing
them up. We only want to use `BooleanExpressionGraphqlConfig` in
`metadata_resolve`, this ensures that.
Pushed some partiality in `schema/boolean_expressions.rs` out - a
function was `Option<inputs> -> Option<outputs>` and now it's `inputs ->
outputs`. We use `Option` a lot and it makes reasoning why something
hasn't been added to the schema difficult.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 893e6f32bfded14ea724be7eaedc519e264f4c01
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.120 to
1.0.122.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/serde-rs/json/releases">serde_json's
releases</a>.</em></p>
<blockquote>
<h2>v1.0.121</h2>
<ul>
<li>Optimize position search in error path (<a
href="https://redirect.github.com/serde-rs/json/issues/1160">#1160</a>,
thanks <a
href="https://github.com/purplesyringa"><code>@purplesyringa</code></a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="54381d6fee"><code>54381d6</code></a>
Release 1.0.122</li>
<li><a
href="16fb6e0b85"><code>16fb6e0</code></a>
Work around buggy rust-analyzer behavior</li>
<li><a
href="49d7d6626f"><code>49d7d66</code></a>
Merge pull request <a
href="https://redirect.github.com/serde-rs/json/issues/1166">#1166</a>
from dtolnay/allocvec</li>
<li><a
href="6827c7b3c5"><code>6827c7b</code></a>
Fix json! invocations when std prelude is not in scope</li>
<li><a
href="611b2a4fb6"><code>611b2a4</code></a>
Merge pull request <a
href="https://redirect.github.com/serde-rs/json/issues/1165">#1165</a>
from serde-rs/jsonmac</li>
<li><a
href="7633cb7f05"><code>7633cb7</code></a>
Eliminate local_inner_macros in favor of non-ident macro paths</li>
<li><a
href="eca2658a22"><code>eca2658</code></a>
Release 1.0.121</li>
<li><a
href="b0d678cfb4"><code>b0d678c</code></a>
Merge pull request <a
href="https://redirect.github.com/serde-rs/json/issues/1160">#1160</a>
from iex-rs/efficient-position</li>
<li><a
href="b1edc7d13f"><code>b1edc7d</code></a>
Optimize position search in error path</li>
<li><a
href="40dd7f5e86"><code>40dd7f5</code></a>
Merge pull request <a
href="https://redirect.github.com/serde-rs/json/issues/1159">#1159</a>
from iex-rs/fix-recursion</li>
<li>Additional commits viewable in <a
href="https://github.com/serde-rs/json/compare/v1.0.120...v1.0.122">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: 92ae4524a7ffab84c6b55097453b2f1484a0c747
This PR fixes the following bugs:
- Fixes a bug where models and commands were allowed even though they
did not define arguments to satisfy the underlying data connector
collection/function/procedure. **UPDATE:** This only raises a warning
rather than fails the build, because existing builds on staging and
production have this issue. This will need to be transitioned to an
error once the Compatibility Date plumbing is in place.
- Fixes a bug where argument presets set in the DataConnectorLink were
sent to every connector function/procedure regardless of whether the
function/procedure actually declared that argument
- Fixes a bug where argument presets set in the DataConnectorLink were
not sent to connector collections that backed Models
- Fixes a bug where the type of the argument name in the
DataConnectorLink's argument presets was incorrect in the Open DD
schema. It was `ArgumentName` but should have been
`DataConnectorArgumentName`
- Fixes a bug where the check to ensure that argument presets in the
DataConnectorLink does not overlap with arguments defined on
Models/Commands was comparing against the Model/Command argument name
not the data connector argument name
There are a number of changes that tighten things up in this PR.
Firstly, the custom connector is improved so that it rejects requests
with arguments of the wrong type or unexpected arguments. This causes
tests that should have been failing to actually fail. Then, new tests
have been added to metadata_resolve to cover the untested edge cases
around data connector link argument presets.
Then, metadata resolve is refactored so that the link argument presets
are validated and stored on each command/model source, rather than on
the DataConnectorLink. Extra validation has been added during this
process to fix the above bugs. Any irrelevant argument presets to the
particular command/model are dropped. Then, during execution, we read
the presets from the command/model source instead of from the
DataConnectorLink, which ensures we only send the appropriate arguments.
JIRA: [V3ENGINE-290](https://hasurahq.atlassian.net/browse/V3ENGINE-290)
Fixes
https://linear.app/hasura/issue/APIPG-676/dataconnectorlink-argument-presets-are-always-sent-regardless-of
[V3ENGINE-290]:
https://hasurahq.atlassian.net/browse/V3ENGINE-290?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
V3_GIT_ORIGIN_REV_ID: dd02e52e1ff224760c5f0ed6a73a1ae56779e1f1
The validation added in #880 validated that the version in the
DataConnectorLink's capabilities version matched the version specified
in the schema. Unfortunately, there are existing builds with invalid
capabilities versions that failed to parse. Subsequently the validation
was removed in #907 to fix staging the deploy that broke.
This is the unique set of errors found when deploying to staging:
```
error generating artifacts: schema build error: invalid metadata: The data connector myts (in subgraph app) has an error: The version specified in the capabilities ("") is an invalid version: empty string, expected a semver version
error generating artifacts: schema build error: invalid metadata: The data connector my_ts (in subgraph app) has an error: The version specified in the capabilities ("") is an invalid version: empty string, expected a semver version
error generating artifacts: schema build error: invalid metadata: The data connector mydbpg (in subgraph app) has an error: The version specified in the capabilities ("") is an invalid version: empty string, expected a semver version
error generating artifacts: schema build error: invalid metadata: The data connector chinook (in subgraph app) has an error: The version specified in the capabilities ("") is an invalid version: empty string, expected a semver version
error generating artifacts: schema build error: invalid metadata: The data connector clickhouse (in subgraph analytics) has an error: The version specified in the capabilities ("^0.1.1") is an invalid version: unexpected character '^' while parsing major version number
error generating artifacts: schema build error: invalid metadata: The data connector chinook_link (in subgraph app) has an error: The version specified in the capabilities ("") is an invalid version: empty string, expected a semver version
error generating artifacts: schema build error: invalid metadata: The data connector app_connector (in subgraph app) has an error: The version specified in the capabilities ("^0.1.1") is an invalid version: unexpected character '^' while parsing major version number
error generating artifacts: schema build error: invalid metadata: The data connector chinook (in subgraph app) has an error: The version specified in the capabilities ("^0.1.1") is an invalid version: unexpected character '^' while parsing major version number
error generating artifacts: schema build error: invalid metadata: The data connector nodejs (in subgraph app) has an error: The version specified in the capabilities ("") is an invalid version: empty string, expected a semver version
error generating artifacts: schema build error: invalid metadata: The data connector db (in subgraph app) has an error: The version specified in the capabilities ("*") is an invalid version: unexpected character '*' while parsing major version number
error generating artifacts: schema build error: invalid metadata: The data connector my_pg (in subgraph my_subgraph) has an error: The version specified in the capabilities ("") is an invalid version: empty string, expected a semver version
error generating artifacts: schema build error: invalid metadata: The data connector mypg (in subgraph myapp) has an error: The version specified in the capabilities ("") is an invalid version: empty string, expected a semver version
error generating artifacts: schema build error: invalid metadata: The data connector mypglink (in subgraph mysubgraph) has an error: The version specified in the capabilities ("") is an invalid version: empty string, expected a semver version
error generating artifacts: schema build error: invalid metadata: The data connector mypg (in subgraph app2) has an error: The version specified in the capabilities ("") is an invalid version: empty string, expected a semver version
error generating artifacts: schema build error: invalid metadata: The data connector test_connector (in subgraph app) has an error: The version specified in the capabilities ("") is an invalid version: empty string, expected a semver version
```
The invalid versions are: `""`, `"*"`, "^0.1.1"`.
This PR restores the version validation code, but for NDC v0.1.x
capabilities (the only supported version right now, v0.2.x is feature
flagged off), we now accept versions that fail to parse as a valid
semver, and instead we raise an issue that gets logged as a warning.
NDC v0.2.x capabilities retains the stricter behaviour and does not
accept dodgy a capabilities version. This is backwards compatible
because trying to use NDC v0.2.x right now produces a build error.
Fixes APIPG-736
V3_GIT_ORIGIN_REV_ID: 9e9bf99123bad31e8229e8ea29343eb8aaf9786d
Prior to this, on every request, a datafusion catalog provider was
created from the stored sql context. This PR reworks it so that this is
cheap and also more maintainable will fewer intermediate steps. There is
also some work done towards supporting table valued functions.
---------
Co-authored-by: Abhinav Gupta <127770473+abhinav-hasura@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: 8c30485366969d81d2a35760962e0383ed5e488c
<!-- The PR description should answer 2 important questions: -->
### What
When no `booleanExpressionType` is specified in a
`BooleanExpressionType` `comparableRelationship`, we fallback to
whatever is defined for the model. However, we were ignoring old style
`ObjectBooleanExpressionType`, meaning relationship fields were
disappearing.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
### How
Also match on `ModelExpressionType::ObjectBooleanExpressionType` when
looking up leaf boolean expressions for relationships.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 9a67b734679b8a1fe3d176a259ba579e127948b8
<!-- The PR description should answer 2 important questions: -->
### What
Change an error down to an issue / warning to unbreak builds.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
### How
Introduce `BooleanExpressionIssue`, move error value to it, emit this
instead. Later we'll turn this into an error based on compatibility
date.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: f0903cc04ea1cf328c9bf67a38d76fd670743679
<!-- The PR description should answer 2 important questions: -->
### What
We emit a warning suggesting users deprecate
`DataConnectorScalarRepresentation`, however it still has uses outside
boolean expressions, so let's not advise this until it is sensible
advice.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
### How
Remove a warning.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: a95a705d121396a09a9b626237999f032e650189
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
Closes:
https://linear.app/hasura/issue/APIPG-397/support-remote-relationship-predicates-in-permission-filters
### What
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
Allow defining permission filters with remote relationships in their
predicates.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
- Lift metadata resolve restriction for remote relationships in
permission predicates
- Abstract out the remote relationship resolving logic, in query filter,
into a new function and re-use it while resolving permission filters.
- Tests:
- A metadata build test to check the presence of essential equal
operator on source fields in relationship mapping.
- Ported all `select_many/relationship_predicate/`* tests to a new
`select_many/remote_relationship_predicate/*` with appropriate metadata
changes.
---------
Co-authored-by: Anon Ray <ecthiender@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: 9c496ecdc9829ed626354ef85e776e1afcb0dfc7
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
This pull request optimizes the `DistinctComparisons` struct in the
codebase to improve the performance of storing and checking for distinct
comparison predicates in remote relationship comparison expressions.
<!-- Consider: do we need to add a changelog entry? -->
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
- Replaced `Vec` with `IndexSet`: Changed the data structure used in the
DistinctComparisons struct from `Vec` to `IndexSet` to leverage the
average O(1) complexity for contains and insert operations provided by
`IndexSet`.
- Updated push Method: Modified the push method to use `IndexSet`'s
insert method directly, which simplifies the code and improves
performance.
**Performance Improvement:**
query:
```graphql
query RemoteRelationship {
Album(where: {TracksRemote: {Name: {_ilike: "%B%"}}}) {
Title
}
}
```
The `TracksRemote` predicate query yields 723 non-distinct results,
which reduce to 266 unique results after deduplication.
Benchmark used: [graphql-bench](https://github.com/hasura/graphql-bench)
configuration: autocannon - Requests Per Second strategy (50 rps) - 10
seconds duration.
Results:
- Before Optimization:
Average Latency: 38.99 ms
- After Optimization:
Average Latency: 23.32 ms
- Percentage Decrease in Latency: Approximately 40%
V3_GIT_ORIGIN_REV_ID: 17a7160b7229eb3a2fde93273d5cf05102f9b4bd
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
Update architecture doc to include `ir` crate. Functional no-op.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 32f47529c97c7cfe188f098f2024e159e5ab33cc
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
`execute` is now the biggest `crate` in engine and does a lot, let's
split it into it's constituent steps.
Functional no-op.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Split out `ir` crate from the `execute` crate. Replace export of entire
modules with that of specific types / functions. Therefore, consumers
outside the crate talk about `ir::CommandInfo` rather than
`ir::command::CommandInfo`. There is no need for other crates to know
about the internal structure of this crate.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 47553aec63e80af7f95e659a170a2685e9ac2ce3
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
The `plan/type.rs` has become large and overwhelmed. This PR refactors
its code and removes it.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
- Move code from `plan/types.rs` into old `arguments.rs`, `filter.rs`
and new `field.rs`, `query.rs`, `mutation.rs`.
- Delete `plan/types.rs`
- Refactor code in other modules to accommodate new changes.
V3_GIT_ORIGIN_REV_ID: 0e294ca8fb4bf1d8622806f5c8b72a2bb01ccdaf
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
These checks are breaking artifact generation, so disabling them until
we can find a safer way to introduce them.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: ae97c87720b67384127122ed0220383036c87bbf
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
Making this match engine.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Replace `V4` with `V6`
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: e86d118b96d41407a292f9ad4132b8ab6d06454f
### What
Telemetry-baggage is propagated via headers from incoming requests to a
service and relayed when the service itself calls another service.
However, when a service is open to the public it may not want just
anyone to be able to pass it baggage.
This PR adds the ability to configure the policy towards baggage
relaying in the tracing-util crate.
### How
When the argument `initialize_tracing(..., propagate_caller_baggage =
false)` we add to the globally defined text map propagator a derived
version of the `BaggagePropagator` which cannot extract baggage from
incoming requests, only inject its own context baggage into outgoing
requests.
V3_GIT_ORIGIN_REV_ID: af9a51c20a8fe7ae2085e8218a4f1d5e01b26ae1
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
This allows object types to be used as arguments for comparison
operators. This is useful for Elasticsearch's `range` operator, which
allows passing an object like `{ gt: 1, lt: 100 }` to an `integer` field
in order to filter items that are greater than `1` and less than `100`.
This PR has the nice side effect of dropping the requirement to use
information from scalar `BooleanExpressionType`s in place of
`DataConnectorScalarTypes`, which we only required because we were not
looking up the comparable operator information in scalar boolean
expression types correctly.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Previously, when using `ObjectBooleanExpressionType` and
`DataConnectorScalarRepresentation`, we had no information about the
argument types of comparison operators (ie, what values should I pass to
`_eq`?), and so inferred this by looking up the comparison operator in
the data connector schema, then looking for a
`DataConnectorScalarRepresentation` that tells us what OpenDD type that
maps to.
Now, with `BooleanExpressionType`, we have this information provided in
OpenDD itself:
```yaml
kind: BooleanExpressionType
version: v1
definition:
name: Int_comparison_exp
operand:
scalar:
type: Int
comparisonOperators:
- name: _eq
argumentType: Int! # This is an OpenDD type
- name: _within
argumentType: WithinInput!
- name: _in
argumentType: "[Int!]!"
```
Now we look up this information properly, as well as tightening up some
validation around relationships that was making us fall back to the old
way of doing things where the user had failed to provide a
`comparableRelationship` entry.
This means
a) we can actually use object types as comparable operator types
b) scalar boolean expression types aren't used outside the world of
boolean expressions, which is a lot easier to reason about.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: ad5896c7f3dbf89a38e7a11ca9ae855a197211e3
Bumps [env_logger](https://github.com/rust-cli/env_logger) from 0.11.3
to 0.11.5.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/rust-cli/env_logger/releases">env_logger's
releases</a>.</em></p>
<blockquote>
<h2>v0.11.5</h2>
<h2>[0.11.5] - 2024-07-25</h2>
<h2>v0.11.4</h2>
<h2>[0.11.4] - 2024-07-23</h2>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/rust-cli/env_logger/blob/main/CHANGELOG.md">env_logger's
changelog</a>.</em></p>
<blockquote>
<h2>[0.11.5] - 2024-07-25</h2>
<h2>[0.11.4] - 2024-07-23</h2>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="73bb418802"><code>73bb418</code></a>
chore: Release</li>
<li><a
href="57f1bb7aae"><code>57f1bb7</code></a>
chore: Release</li>
<li><a
href="ba2748f48c"><code>ba2748f</code></a>
chore: Release</li>
<li><a
href="4dd976b01e"><code>4dd976b</code></a>
Merge pull request <a
href="https://redirect.github.com/rust-cli/env_logger/issues/324">#324</a>
from Maximkaaa/try_parse</li>
<li><a
href="05aacb93c5"><code>05aacb9</code></a>
feat: Add Builder::try_parse method</li>
<li><a
href="faf5b3e499"><code>faf5b3e</code></a>
chore: Tests for <code>parse_spec</code> error messages</li>
<li><a
href="c42511cc3c"><code>c42511c</code></a>
refactor: Return errors from <code>parse_spec</code></li>
<li><a
href="0e25d9e993"><code>0e25d9e</code></a>
chore(deps): Update Rust Stable to v1.79 (<a
href="https://redirect.github.com/rust-cli/env_logger/issues/322">#322</a>)</li>
<li><a
href="e1640c493d"><code>e1640c4</code></a>
Merge pull request <a
href="https://redirect.github.com/rust-cli/env_logger/issues/320">#320</a>
from epage/template</li>
<li><a
href="f8acfee686"><code>f8acfee</code></a>
chore: Update from _rust/main template</li>
<li>Additional commits viewable in <a
href="https://github.com/rust-cli/env_logger/compare/v0.11.3...v0.11.5">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: 1e051c353b7c5ff1e2ab6aaa43a5231858de517a
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.62 to
1.0.63.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/dtolnay/thiserror/releases">thiserror's
releases</a>.</em></p>
<blockquote>
<h2>1.0.63</h2>
<ul>
<li>Documentation improvements</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="915c75e8a3"><code>915c75e</code></a>
Release 1.0.63</li>
<li><a
href="3d5ec25464"><code>3d5ec25</code></a>
Merge pull request <a
href="https://redirect.github.com/dtolnay/thiserror/issues/312">#312</a>
from dtolnay/backtracedoc</li>
<li><a
href="de8a1e54d4"><code>de8a1e5</code></a>
Update documentation of #[from] and #[backtrace] attributes</li>
<li>See full diff in <a
href="https://github.com/dtolnay/thiserror/compare/1.0.62...1.0.63">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: 7c28562af5d512f1be5543cd6f90141136eaf80e
Bumps [clap](https://github.com/clap-rs/clap) from 4.5.9 to 4.5.11.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/clap-rs/clap/releases">clap's
releases</a>.</em></p>
<blockquote>
<h2>v4.5.10</h2>
<h2>[4.5.10] - 2024-07-23</h2>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/clap-rs/clap/blob/master/CHANGELOG.md">clap's
changelog</a>.</em></p>
<blockquote>
<h2>[4.5.11] - 2024-07-25</h2>
<h2>[4.5.10] - 2024-07-23</h2>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="349ed304cf"><code>349ed30</code></a>
chore: Release</li>
<li><a
href="68ed9ff02d"><code>68ed9ff</code></a>
docs: Update changelog</li>
<li><a
href="afecaa4f9a"><code>afecaa4</code></a>
Merge pull request <a
href="https://redirect.github.com/clap-rs/clap/issues/5576">#5576</a>
from shannmu/option_value_new</li>
<li><a
href="4a00677024"><code>4a00677</code></a>
chore: Release</li>
<li><a
href="36c849b5bb"><code>36c849b</code></a>
refactor(clap_complete): Reuse code about shortflag parsing</li>
<li><a
href="fc479ba625"><code>fc479ba</code></a>
feat(clap_complete): Add support for <code>-fbar</code> and
<code>-f=bar</code> completion</li>
<li><a
href="48a23f5c94"><code>48a23f5</code></a>
test(clap_complete): Add test cases for <code>-fbar</code> and
<code>-f=bar</code> completion</li>
<li><a
href="6b18d7725c"><code>6b18d77</code></a>
docs(derive): Acknowledge Vec<Vec<!-- raw HTML omitted -->></li>
<li>See full diff in <a
href="https://github.com/clap-rs/clap/compare/clap_complete-v4.5.9...clap_complete-v4.5.11">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: e10319e4cd77b3518672bc325fc2e924167a45c9
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
We have decided to remove the role emulation feature from engine
altogether.
More details in the RFC -
https://docs.google.com/document/d/1tlS9pqRzLEotLXN_dhjFOeIgbH6zmejOdZTbkkPD-aM/edit
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: e7cb765df5afac6c6d6a05a572a832ce9910cc0b
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
We've had issues with `metadata-resolve` rejecting Elasticsearch schema
output, so adding said output to this test. Appears to work fine, so
merging it for further discussion and to improve the test case.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Add elastic search schema to test.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: ea7c39ca7ab07fc18abd08eb822d2d56fc152ae6
Bumps
[async-graphql-parser](https://github.com/async-graphql/async-graphql)
from 7.0.6 to 7.0.7.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/async-graphql/async-graphql/blob/master/CHANGELOG.md">async-graphql-parser's
changelog</a>.</em></p>
<blockquote>
<h1>[7.0.7] 2024-07-14</h1>
<ul>
<li>Support raw values from serde_json <a
href="https://redirect.github.com/async-graphql/async-graphql/pull/1554">#1554</a></li>
<li>The custom directive <code>ARGUMENT_DEFINITION</code> is not being
output at the appropriate location in SDL <a
href="https://redirect.github.com/async-graphql/async-graphql/pull/1559">#1559</a></li>
<li>Support for JSON extended representations of BSON ObjectId and Uuid
<a
href="https://redirect.github.com/async-graphql/async-graphql/pull/1542">#1542</a></li>
<li>feat: get directives from SelectionField <a
href="https://redirect.github.com/async-graphql/async-graphql/pull/1548">#1548</a></li>
<li>Support Directives on Subscriptions <a
href="https://redirect.github.com/async-graphql/async-graphql/pull/1500">#1500</a></li>
<li>fix subscription err typo <a
href="https://redirect.github.com/async-graphql/async-graphql/pull/1556">#1556</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/async-graphql/async-graphql/commits">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: 8c0d1a222a57c86cedf2f0870f6cccb0a861a3e6
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
Forgot to do this in last PR.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 24ec7278a63379f252d452b8ec627c934e1c534c
### What
We introduced a newtype around the NDC field alias, but we called it
`NdcFieldName`. While in reality it is the alias of the field requested
in the query.
This PR changes the name to `NdcFieldAlias`.
This is a no-op change
V3_GIT_ORIGIN_REV_ID: 8e892c29860e93243a200b6a6291fd0a32cc6fe3
### What
Part of the point of the `tracing-util` crate is to centrally enforce
usage of a single version of opentelemetry libraries.
Previously we added some support for relaying baggage, but not actually
for defining it. This PR exposes the crates and types necessary to add
baggage to the context.
V3_GIT_ORIGIN_REV_ID: 107ec652d4e812f31bbfaa362cedf44b25dc3c39
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
We have a bunch of local development infra for building the engine
inside a Docker container. This is helpful for Buildkite which doesn't
come with stuff like `cargo` preinstalled. We've not using Buildkite
anymore, let's remove it.
V3_GIT_ORIGIN_REV_ID: b4b7679aab5b14081288df25d139944f160a61fe
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
We'd like to make it simpler to try out DDN, by starting with a mode
that uses no auth.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Add a `NoAuth` `AuthConfig` mode that is configured thus:
```json
"noAuth": {
"role": "admin",
"sessionVariables": {
"x-hasura-user-id": "1"
}
}
```
Given the above config:
- If no `x-hasura-role` is sent with a request, we run it as `admin`.
- If a `x-hasura-role` header is sent and it's `admin`, it continues to
work
- If any other `x-hasura-role` header is sent, an error will happen.
- All other headers are ignored, and we always set `x-hasura-user-id` to
1
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: dddcfbee9c3a31e84dfc8013de32e3a9bf31943d
This PR adds validation code to `metadata_resolve` that prevents someone
from putting a schema/capabilities from the wrong NDC version into the
DataConnectorLink while specifying a different schema version in the
DataConnectorLink. For example:
```
kind: DataConnectorLink
version: v1
definition:
name: data_connector
schema:
version: v0.2
schema: {}
capabilities:
version: 0.1.5 # Not allowed for version v0.2!
capabilities: {}
```
This PR has two commits. One is a refactor where we rearrange the
DataConnectorError types so that the name of the data connector is
captured centrally in `NamedDataConnectorError`, so that it doesn't have
to be passed around and included in every error manually. The other is
the validation changes to `metadata_resolve`.
Completes APIPG-705
V3_GIT_ORIGIN_REV_ID: baed571f36f4cbed824ca546128f5df360d5b298
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
These take ages to run, are slowing development down and not offering
the value they should. We should be benchmarking engine, but not like
this.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Remove benchmarks CI job from Buildkite.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 30a2c9d5f6ba09f5319a07fe394db8becaa16b8e
This PR updates as many tests as possible that use the custom connector
so that the tests run over two versions of the custom connector:
1. The custom connector in the repo, which currently speaks `ndc_models`
v0.2.x
2. The custom connector from the past (commit ), which is the last
version to speak `ndc_models` v0.1.x
This helps us test both the NDC v0.1.x and v0.2.x code paths. When the
postgres connector upgrades to v0.2.x, we can use the same approach as
in this PR to get the tests to run over multiple versions of the
postgres connector too, for much better coverage. This approach with the
custom connector will become less useful over time as the v0.1.x
connector is not updated and will diverge in data from the v0.2.x
connector. The postgres connector is likely to be longer-lasting, as it
is more stable.
The basic test used for `execute` integration tests is
`test_execution_expectation` (in `crates/engine/tests/common.rs`) and it
has been extended into a version called
`test_execution_expectation_for_multiple_ndc_versions` that takes
metadata on a per NDC version basis and then runs the test multiple
times, once for each NDC version. This allows one to swap out the
DataConnectorLink involved in the test to a different one that points at
either the v0.1.x or v0.2.x versions of the connector. The assertion is
that both connectors should produce the same results, even if they talk
a different version of the NDC protocol. As each version runs, we
`println!` the version so that if the test fails you can look in stdout
for the test and see which one was executing when it failed.
Tests that use the custom connector now use
`test_execution_expectation_for_multiple_ndc_versions` and run across
both connector versions. Some tests were unable to be used across both
version as the data between the two versions has changed. Some tests
were modified to avoid the changed data so as to support running across
both versions. Any tests that use `test_execution_expectation_legacy`
don't run across both versions because those tests aren't backed by the
same test implementation as
`test_execution_expectation_for_multiple_ndc_versions`.
Unfortunately the custom connector doesn't use the standard connector
SDK, so it doesn't support `HASURA_CONNECTOR_PORT`. This means that the
old connector is stuck on 8101. To work around this, I've moved the
current connector port to 8102 instead. Technically we might be able to
use docker to remap the ports, but then this binds us into always
running the connectors in docker in order to move their ports around, so
I avoided that approach.
Completes APIPG-703
V3_GIT_ORIGIN_REV_ID: fb0e410ddbee0ea699815388bc63584d6ff5dd70
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
In a recent engine change, we changed some of our trace context mapping
to use the shared settings consistently. However, we needed to make sure
we included `TraceContextResponsePropagator`, which returns the
`traceresponse` header.
Request from console after this fix:
<img width="810" alt="Screenshot 2024-07-25 at 11 58 30"
src="https://github.com/user-attachments/assets/c8e73c56-87fd-49da-a887-f91cdb6d607a">
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Adds `TraceContextResponsePropagator` to the global set of text map
propagators.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 48df6a6fe55e78a48f1dc6bf82304199a0a7e248
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
NDC query request expects relationship names which are unique across the
query.
Previously, we would generate relationship name of the form -
```
[{\"subgraph\":\"connector_2\",\"name\":\"Album\"},\"Tracks\"]
```
This works, but is harder to read while debugging. This PR changes it to
have a human-readable name like -
```
connector_2___Album__Tracks
```
This is a no-op change, apart from the relationship names in NDC query
requests.
### How
Instead of json-ifying the data structure in a tuple, create a formatted
string.
V3_GIT_ORIGIN_REV_ID: 3fea3bf56f1688bc1cade1ea2b3ed6eb60509cac
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
We've had our CI mixed between Github and Buildkite for a while, it's
time to commit. First step is moving the "tests" step to Github Actions.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
This PR:
- Moves the `test` step to Github Actions
- Creates a new `custom_connector.Dockerfile` which builds custom
connector only, more quickly.
- Changes the metadata tests to use `localhost` instead of their Docker
internal names (ie `custom_connector` or `postgres_connector`) - this is
because the tests are being run from outside Docker now
- Removes the `test` Buildkite step
It does not:
- Remove the code coverage or benchmarks steps from Buildkite
- Tidy up `justfile` or Dockerfiles
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
---------
Co-authored-by: Philip Lykke Carlsen <plcplc@gmail.com>
V3_GIT_ORIGIN_REV_ID: a67534ebc1634a24b48d2620c45003221852e199
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
Our CI times are slow, thus merging is slow, and I do not feel this job
pulls it weight.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Remove the code coverage job from Buildkite, delete all associated
files.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 7bbde842f9f405920d1e559b8fef88078cc8d564
Previously the `sql` crate generated a v02 ndc query request and then
downgraded it to v01 if necessary. This is fragile in that its easy to
use v02 ndc features and then get v01 downgrade errors, plus the
downgrade logic is extensive and tedious.
This PR refactors the `sql` crate so that it generates `ir` and `plan`
types and eventually creates `ResolvedQueryExecutionPlan` (rather than
ndc_models types), and then the ResolvedQueryExecutionPlan is
transformed into the appropriate ndc version in the same fashion as the
main engine execute code does it. This eliminates all the downgrade
logic and simplifies things.
Unfortunately, ndc's `QueryRequest` could not just simply be replaced
with `QueryExecutionPlan` on `sql`'s `NDCQuery` and `NDCPushDown`,
because it involves lifetime parameters which are incompatible with the
datafusion framework types. Instead, the individual components of a
query are kept on `NDCQuery` and `NDCPushDown`, and these are eventually
assembled into a `ResolvedQueryExecutionPlan` at a place where the
lifetime parameters are workable. In some sense this is clearer, as one
can now see where each individual part of the query is actually created
and relevant, instead of copying around and mutating a `QueryRequest`.
Completes
https://linear.app/hasura/issue/APIPG-702/implement-separate-logic-that-maps-engine-types-to-ndc-models-types-on
V3_GIT_ORIGIN_REV_ID: c4a9226c1b1addcfe5cd0bca783f1b65ab3ada38
~~Note: this PR is stacked on #845.~~ Rebased on main
This PR refactors the `execute::plan::types` further to make a clear
distinction between unresolved and resolved states. An "unresolved"
state refers to one in which remote predicates have not been computed
into local predicates. A "resolved" state is after this process is
performed and remote predicates are eliminated.
Previously, unresolved types could be passed to
`execute::plan::ndc_request` and they would fail at runtime due to the
presence of unresolved remote predicates. Now, this is impossible due to
a type-level distinction between unresolved and resolve states.
This distinction is made by type-parameterizing all
`execute::plan::types` that involve a predicate so that the predicate
type is parameterized out. Then, an `Unresolved` type alias is created
that sets the predicate type to
`execute::ir::filter::expression::Expression` (which contains remote
predicates) and a `Resolved` type alias is created that uses
`ResolvedFilterExpression` instead (which does not contain remote
predicates).
For example, for `QueryNode`, we now have:
```rust
pub struct QueryNode<'s, TFilterExpression> {
...
pub predicate: Option<TFilterExpression>,
...
}
```
And then the two aliases are:
```rust
pub type UnresolvedQueryNode<'s> = QueryNode<'s, ir::filter::expression::Expression<'s>>;
pub type ResolvedQueryNode<'s> = QueryNode<'s, ResolvedFilterExpression>;
```
Subsequently, `plan::ndc_request` only deals with `Resolved` types.
This is mostly just type-fiddling, but one place some logic moved around
is in with the old `plan::types::FilterExpression`. This was mostly a
functional duplicate of `ir::filter::execute::Expression` except that it
had a "planned" remote predicate variant in it. In order to reduce the
number of types (so we didn't need `UnresolvedFilterExpression` and
`ResolvedFilterExpression`), this type has been repurposed into
`ResolvedFilterExpression` and no longer deals with remote predicates.
Instead, `ir::filter::execute::Expression` is resolved into a
`ResolvedFilterExpression` and the planning of the remote predicate is
done at that time, just before it is resolved. This works fine, since an
entirely new ndc query is performed in order to resolve the predicate,
so planning that can be deferred until then and it doesn't need to be
done at the same time as the main query.
Part of
https://linear.app/hasura/issue/APIPG-702/implement-separate-logic-that-maps-engine-types-to-ndc-models-types-on
V3_GIT_ORIGIN_REV_ID: 3ec89efbaa7b543fad6a100e2739bcc74b1d567f
This PR removes usages of ndc_models types from the `ir` types and the
`plan::types` in the `execute` crate. This is done to isolate the IR and
planning logic from the specific version of NDC that needs to be used to
talk to the specific data connector. Once planning is done, the
`plan::types` are mapped into `ndc_models` types via
`plan::ndc_request::v01` and `plan::ndc_request::v02`. Those two modules
contain code that maps `plan::types` to a specific ndc_models version.
This code is entirely separate per version.
Now, when the `plan::types` are `resolve`d, they don't return NDC types.
Now they return themselves, but modified to remove any remote
predicates. After `resolve` the types are in a state that they can be
converted into a single NDC request via `plan::ndc_request`.
Next steps for a future PR: `QueryExecutionPlan` (etc) needs to type
parameterize out the `FilterExpression` type so that after `resolve`, a
new `QueryExecutionPlan` is returned that uses another FilterExpression
type that does not have remote predicates in it. This will make
resolving typesafe and prevent accidentally using `plan::ndc_request` on
unresolved `QueryExecutionPlan`s.
The `sql` crate also needs to stop talking ndc directly and probably
talk `plan::types` instead. Right now it writes ndc v0.2.0 and then
downgrades it to v0.1.0 if necessary, which is awful. Talking
`plan::types` is currently not possible due to lifetime parameters used
on the `plan::types`. These may be able to be removed from the resolved
variant of those types as mentioned above.
Part of
https://linear.app/hasura/issue/APIPG-702/implement-separate-logic-that-maps-engine-types-to-ndc-models-types-on
V3_GIT_ORIGIN_REV_ID: b536009ea784d1486a2ece2262e0ce9d0f937ef0
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
When a boolean expression is passed as an argument it was not being
translated into an `ndc_models::Expression` and so queries failed.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Look up the type of an argument in `metadata-resolve`, and mark the
`ArgumentInfo` with a new `ArgumentKind`. Then in IR step we use that to
work out whether to turn the argument to JSON as before, or translate it
into an `Expression` type that will eventually be turned into an
`ndc_models::Expression`.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 4da3ce0ae04895c33de2b6bdb6fff1018c39b3ad
### What
This PR enables the use of Opentelemetry Baggage. Every bit of baggage
is then replicated on every span.
The current implementation does not actually set any baggage itself - it
only relays and outputs what it's getting.
Crafting a request with a `baggage` header set:
Has it propagated (here, to `dev-auth-webhook`) and stamped onto every
span:
### How
This PR actually makes the engine and auth-hook use the globally
specified propagators (before they would only use an obsucre, concrete,
re-exported one from opentelemetry_contrib), and adds the
`BaggagePropagator` to the list.
It also adds a `SpanProcessor` which outputs the baggage as span
attributes. Currently it outputs all Baggage entries, but can be made
more specific in the future if we want to treat Baggage differently.
V3_GIT_ORIGIN_REV_ID: 3b5b8604b624c0b90c192e68b3b57fab7ca9b63e
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
Sometimes our builds succeed, but we'd like to tell the user how they
could do better. This implements the simplest possible warnings system.
<img width="957" alt="Screenshot 2024-07-18 at 16 06 49"
src="https://github.com/user-attachments/assets/ff91d221-667a-43f9-bc8a-51bf4574a7b8">
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Warnings are printed to stdout on `v3-engine` startup, and will be
returned to the CLI via `v3-metadata-build-service`. The diff is mostly
updated snapshot tests as we're returning more from
`metadata-resolve::resolve` now.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: d01520e53f49d9b594e94a4531b6a86e749875c3
### What
Previously, while generating relationship definitions for NDC, we would
ignore columns with nested selection.
This PR fixes that.
Closes https://hasurahq.atlassian.net/browse/V3ENGINE-247
### How
While matching on `FieldSelection::Column`, don't ignore it. Check if it
contains nested selection, if it does, call
`collect_relationships_from_nested_selection`
V3_GIT_ORIGIN_REV_ID: 9db94744d8e2d35f8430bded07209ef519175205
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
This PR fixes issues in the SQL layer where the following queries would
fail:
1. `select count(*) from "Track"`
2. `select * from "Track" where id = 1`
<!-- Consider: do we need to add a changelog entry? -->
### How
These were failing because the built-in analyzer rules that rewrite
`count(*)` and type-cast expressions weren't firing.
`with_analyzer_rules` replaces the analyzer rules of a session context
with the given list. We want our analyzer rule to be fired in addition
to the built-in analyzer rules.
Tests are being worked on in a separate PR.
V3_GIT_ORIGIN_REV_ID: 42231f97b5b28d9b7eeff0c3e592cb43ff7d952f
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
We're having issues with our deduplication of names in JSONSchema. We
would like to fix this, but in the short term, this renames a
conflicting object to avoid this quickly.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Rename `ArgumentPreset` in `open_dds::data_connectors` to
`DataConnectorArgumentPreset`.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: e3eafeffe8ba4d513f9d0a09a623f101650247ea
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
update changelog for new release
### How
:)
V3_GIT_ORIGIN_REV_ID: 6620f9923f393d190f0f2fab1aced1cff4d6aec0
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
This stage already has it's own error type, but it returns the larger
`Error` type, so let's return the more specific type instead.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Changing return types mostly. Functional no-op.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 2aae1f06775db6d88c34b1d3c1779396e0ba410e
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
More breaking down the big error type, this time we sort the
`data_connector_scalar_types` stage. Functional no-op.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Move error cases into a smaller enum.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 50b699f3a77594deb27a6cc6ab8dd61752404daf
Now, users can filter their queries using remote relationships in the
filter predicate. Users need to provide the relationships for comparison
in `comparableRelationships` field of the newer `BooleanExpressionType`
opendd metadata.
Minimal Algorithm:
```
Relationship: ARemoteB => Model_A -> Model_B (remote NDC)
Column Mapping: (A_column_1, B_column_1), (A_column_2, B_column_2).
query:
Model_A:
where: ARemoteB: {B_column_3: {_eq: value}}
Step 1: Fetch RHS column values (in mapping) from remote target model
SELECT B_column_1, B_column_2 from model_b_collection WHERE B_column_3 = value;
yields the following rows
[
[(B_column_1, b_value_1), (B_column_2, b_value_2)],
[(B_column_1, b_value_11), (B_column_2, b_value_22)],
]
Step 2: Using above rows the generate LHS column filter for Model_A query.
SELECT <fields> from model_a_collection WHERE
((A_column_1 = b_value_1) AND (A_column_2 = b_value_2))
OR ((A_column_1 = b_value_11) AND (A_column_2 = b_value_22))
The above comparison is equivalent to
WHERE
(A_column_1, A_column_2) IN ((b_value_1, b_value_11), (b_value_2, b_value_22))
```
Sample query:
```graphql
query MyQuery {
Track(
where: {
_or: [
{ AlbumRemote: { Artist: { ArtistId: { _eq: 2 } } } }
{ TrackId: { _eq: 3 } }
]
}
) {
TrackId
AlbumRemote {
Artist {
ArtistId
Name
}
}
}
}
```
In the query above, `AlbumRemote` is a remote relationship which targets
a model backed by a different data connector.
V3_GIT_ORIGIN_REV_ID: 7aa76fcae83e1f22de460f1eef5648fb7a35c047
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
Much like this change, but for the `object_boolean_expressions` stage:
https://github.com/hasura/v3-engine/pull/843
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 85da185b45bac429b754b0b92419f378a59fb536
### What
This PR fixes an issue where relationships that target commands do not
correctly use the data connector's argument name when making the ndc
request. Instead, they use the OpenDD argument name, which is incorrect.
For metadata where the OpenDD argument name is the same as the data
connector's argument name, the code works but only coincidentally.
### How
I've updated an existing test to change the name of the command argument
to be different from the data connector's argument name. This test
failed but is now fixed by this PR, which simply looks up the name of
the data connector argument name and uses that instead.
V3_GIT_ORIGIN_REV_ID: 71f1e812174c7bb9922792523129e4bcdce911ed
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
Filtering on nested arrays doesn't work, let's make sure it's not
allowed for now.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Adding a check in `boolean_expression_types` stage.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
---------
Co-authored-by: Gil Mizrahi <gil@gilmi.net>
V3_GIT_ORIGIN_REV_ID: cc08e8c24098c1fea9b6e1ee61b82ade989dd29a
This PR adds true support for ndc_models v0.2.0 to v3-engine. Note that
v0.2.0 is not finalized yet, so we're pointing at v0.2.0-rc0. The
support still comes via the migration methodology, where v0.2.x ndc
models are downgraded to v0.1.x to support backwards compatibility. In
the future we want to remove this and have the engine generate the
different versioned ndc models separately instead of performing a
migration.
The ndc_models_v01 crate reference has been bumped to the official
v0.1.5 version, which brings the newtypes to the v0.1.x version. The
ndc_models crate reference is now on v0.2.0-rc0.
The custom connector has been updated to support ndc-spec v0.2.0. All
tests that talk to the custom connector have been updated with its
latest v0.2.0 schema/capabilities.
In `metadata_resolve` the v01->v02 schema/capabilities migration code
has been updated to handle the new v0.2.0 types. This includes inferring
v0.2.0 capabilities from what was possible in v0.1.x.
In `execution`, the migration code has been updated to deal with the new
v0.1.5 newtypes and v0.2.0 types. This means there are now cases where a
downgrade is impossible and produces an error (see `NdcDowngradeError`
in `execute::ndc::migration`). A bug has also been fixed where NDC
expressions in arguments were not being serialized to the correct NDC
version.
V3_GIT_ORIGIN_REV_ID: 5b4afcde64c307b2bd7c985c588d6c74d9623a0f
### What
Much like https://github.com/hasura/v3-engine/pull/824, we combine
relay-related errors into `RelayError`.
### How
Remove them from the big `Error` type.
---------
Co-authored-by: Daniel Chambers <daniel@hasura.io>
V3_GIT_ORIGIN_REV_ID: b26460c6aa4d622c6f5548e5cd294c7480acdca4
### What
Part of ongoing tidy up of errors, this splits out errors types for the
boolean expression stages.
### How
Remove things from `Error`, move files around. Functional no-op.
V3_GIT_ORIGIN_REV_ID: ccf1f29600a169a3787d744c7f60e79220aef8d2
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
To stop us being confused between `Error` type and `Error` trait.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Import `thiserror::Error` explicitly in place.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: b930480927b2c64537960cfb69f2b2b30921f4fd
This PR fixes the custom connector whose schema endpoint doesn't
actually return correct output (it was missing some
functions/procedures, etc). Then it updates all tests that actually talk
to the custom connector with the latest version of its
schema/capabilities in their DataConnectorLink.
This test update is done by a new script added to the justfile that
finds and patches all metadata json files and inserts the new schema and
capabilities after reading them from the custom connector running in
docker.
V3_GIT_ORIGIN_REV_ID: f1825a6f74ddcb6c01198fe4a41de6b4fc0bf533
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
As a treat, combine all Apollo errors into one enum.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Remove items from `ObjectTypesError` and `Error`.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 5a16a030b35372283490f3de7343fcfca2fadea5
### What
And don't set it except for the main application; tests and test
infrastructure does not care.
### How
We use the `VERSION` constant, populated from the `RELEASE_VERSION`
environment variable at build time.
It's now also optional so tests don't have to specify it.
V3_GIT_ORIGIN_REV_ID: 1bfc2efb060307cc9446bf07e944e107f0607ae0
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
Update changelogs for new release.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 7adfd3c2c53b912bb7c4604ebed93601a201c15e
This PR removes the usage of ndc_models in the field arguments IR and
uses the actual arguments IR instead. This change was missed in #810.
V3_GIT_ORIGIN_REV_ID: 414b4eda9724e7702b5a09ea2855b457d7ce88d6
### What
`ValueExpression` contained boolean expressions, and was used in places
where boolean expressions were not allowed.
### How
This creates a new type `ValueExpressionOrPredicate`, and uses it in the
places where boolean expressions are allowed.
---------
Co-authored-by: Daniel Chambers <daniel@hasura.io>
V3_GIT_ORIGIN_REV_ID: c0a07c5e0096aeb4369ca7d6b5147451d1ccd14d
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
Break down the big `Error` enum some more. This time, add all errors
from the `object_types` stage into the `ObjectTypesError` enum.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Moving code around. Functional no-op.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 154d4b40b21365c783d545a23cf18be623f2a3de
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
In the haste of breaking things up, realised we put the `relay` checks
in with the `apollo` ones where they are two separate things.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Split up `apollo` step into `relay` and `apollo` steps. Functional
no-op.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 8d02d3cb77b1248239be37ed61d0b87b9b734fdf
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
We pass around a lot of `BTreeMap` and `IndexMap` types. This has two
problems:
a) everytime we change the items inside, we have to manually update lots
of call sites
b) we can't attach useful behaviour to it
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
This adds some wrappers around `object_types`, and adds a `.get()`
function with a useful default error. This error only contains the
missing `type_name`, so the idea is that it would be wrapped in another
error that would provide more context. The hope is that we can get away
from one giant error enum, and instead have a set of smaller error types
that live along each resolving stage.
In isolation this PR isn't very interesting, as it's a tiny drop in the
ocean, so really it's here to say, "is this a thing we vaguely like?"
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 804a703d7155ce5b929937689d5649c6571357b0
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
Put all the errors that happen in the `data_connectors` stage into a new
type `DataConnectorError`.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Moving enum members to the new type. Functional no-op.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 2e50eb561ce6b7c7fac4de4b31c9957f7ee4696c
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
Much like https://github.com/hasura/v3-engine/pull/808, move an error to
the place it is thrown.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Move files around. Functional no-op.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: a61527b57662efada2c7d2049e12f103deec1e4e
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
We have a giant error file in `metadata-resolve`, and it's really
unclear what can go wrong where. Let's improve it in some small way.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Move `GraphqlConfigError` to the `graphql_config` stage, and make that
stage only return that kind of error.
Functional no-op.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
---------
Co-authored-by: Samir Talwar <samir.talwar@hasura.io>
V3_GIT_ORIGIN_REV_ID: 9aa58875a24d66e6945c9ead6434fedb124f1dd8
This PR removes the usage of `ndc_models` `Expression` types from the
IR. It then adds logic to map from the new IR `FilterExpression` types
to `ndc_models` types in the plan part of the code where IR ->
ndc_models mapping is performed. The new IR types and the change to the
IR creation logic is mostly in `crates/execute/src/ir/filter.rs`.
The new IR types have some optimisations applied to them. In particular,
some basic boolean expression simplification logic is applied when
`FilterExpression::mk_and` `mk_or` and `mk_not` are used. This strips
out redundant and/ors/nots resulting in simpler boolexps for connectors
to process. (This logic is similar to what GDC did in Hasura v2).
Unfortunately it turned out that arguments had JSON-serialized
`ndc_models` types embedded in them, in particular, where argument
presets had been used to set a BooleanExpression as an argument value.
The old code was simply creating an ndc_models::Expression and
serializing it directly to JSON. This has now been refactored to retain
the new IR `FilterExpression` until `plan` time, at which point it will
be mapped into the ndc type and serialized to JSON and embedded in the
argument value. The types change for this can be seen in
`crates/execute/src/ir/arguments.rs`, but the real logic is actually in
`crates/execute/src/ir/permissions.rs`, with the
`make_value_from_value_expression` and
`make_argument_from_value_expression` functions.
The mapping of expression and argument IR into `ndc_models` can be found
in `crates/execute/src/plan/common.rs`.
In `metadata_resolve`, some ndc_models types were removed and
non-ndc_model types were used instead. In particular
`ndc_models::ComparisonOperatorName` (swapped with
`DataConnectorOperatorName`) and `ndc_models::UnaryComparisonOperator`
(swapped with a new `UnaryComparisonOperator` enum type).
This PR is a functional no-op, other than the boolean expression
simplification.
V3_GIT_ORIGIN_REV_ID: 13805430fb2e2ca5653406a094d719138f0d5b51
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
Since these code comments end up in the docs, let's make sure we point
out these old types are no longer in favour.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Updating doc comments.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: a21190fbf2327e1ae265c76d1f467af063f1dd64
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
If the target data connector for a relationship does not have the
`foreach` capability, we threw an error, however we should allow this.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
Check data connector names for relationship target before throwing
capability error.
V3_GIT_ORIGIN_REV_ID: 157908f0077b7c3af67a77600e5f8c9fc65df7f2
Note: This PR is stacked on #797 and should be merged after it.
This PR removes the use of ndc_models types from the arguments and order
by IR. It then shifts the logic that maps the IR to the ndc_models into
the plan part of the code where IR -> ndc_model mapping is performed.
This will help isolate the IR from ndc_models and move us towards being
able to have multiple IR -> ndc_models mapping codes, one per supported
ndc version.
This is a functional no-op PR.
V3_GIT_ORIGIN_REV_ID: 66be868ff4c8185c6190537d570d88813cb7f410
This PR replaces most of the string newtypes in the open-dds crate with
new implementations based on `SmolStr`, using a macro adapted from the
macro used in ndc-models. `SmolStr` usage should result in less heap
allocations and the macro ensures all the newtypes get all the same
trait implementations (such as various `From` and `Borrow` instances)
and helper impl functions (such as `as_str`).
The new macro, `str_newtype`, creates a newtype wrapper around `SmolStr`
or another type (typically `Identifier`), and writes out all the various
trait implementations. It also takes a documentation string which
ensures that every newtype has a description in JSON Schema.
The changes in the downstream crates are just adjusting to use the new
newtypes.
Also:
* `QueryGraphqlConfig` used a lot of String typed-properties; these have
been changed for the appropriate `GraphQlTypeName` and
`GraphQlFieldName` types.
* metadata-resolve had a duplicate newtype called
`ConnectorArgumentName`, which duplicated open-dds's
`DataConnectorArgumentName`. This has been removed and replaced.
* A new newtype called `CollectionName` has been added in open-dds to
represent the name of the ndc collection that a model points to. This
was previously just a string.
V3_GIT_ORIGIN_REV_ID: d5a33756ef0e110b2255478358a38e42e6ff89a2
This PR just sorts the definitions in the Open DD JSON schema file, so
that when it changes, we can get better diffs because the types won't
move around inside the file.
V3_GIT_ORIGIN_REV_ID: 80221fae1b4d6e4b98b658f9ba2f19413abf7389
### What
This PR fixes a bug with variable nullability coercion. Specifically,
providing a non-null variable for a nullable field should work, as all
non-nullable variables can be used as nullable variables via "coercion".
Related issues:
* https://hasurahq.atlassian.net/browse/V3ENGINE-243
* https://github.com/hasura/v3-e2e-testing/pull/224 (the test for this
change)
### How
I think someone did something clever with macros, so I tried to unpack
the macro to make sense of it, and in the process, spotted the bug: it's
not that both the location must be nullable AND the variable not, it's
that _if_ the variable is nullable, the location must not be nullable.
V3_GIT_ORIGIN_REV_ID: 4f4ba3fe6898220bbba9ae2867233cd762bef1cb
Update the changelog for new version
---------
Co-authored-by: Anon Ray <ecthiender@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: b87f16ebaa1471f010ec461be097bcfd6648c99a
This PR updated ndc-models to the latest version on main. This version
is still a 0.1.x version, but it now includes all the [new
newtypes](https://github.com/hasura/ndc-spec/pull/156) that wrap
previously stringly-typed things. For example, `ArgumentName`,
`FieldName`, etc.
This pervades across the entire engine, but thankfully the changes are
mostly mechanical repetitive changes. Usually you will see conversions
from `String`-typed variables into the newtypes using this sort of form:
`FieldName::from(string.as_str())`, which is the most efficient way
copying the value (the str slice is copied). Or you will see usages of
the newtype as a raw string by `.as_str()`-ing it. Converting the
newtypes into a String can be done with `.into()` if owned, but if
referenced `.as_str().to_owned()` performs the clone and type
conversion.
Other changes:
* A few minor instances of `ok_or()` usages (or similar) have been
converted into lazy error construction variants (eg `ok_or_else()`)
V3_GIT_ORIGIN_REV_ID: 64a371ae6197ef3be98a6f7cdc4052d654a43da0
The PR adds code that handles multiple versions of the ndc-models during
the execution pipeline. Depending on whether a connector supports `v01`
or `v02` models, a different set of types is used to send and receive
the http requests.
However, the engine internally still uses the latest (v02) models inside
its IR. Unfortunately, it was going to be quite traumatic to prevent the
engine from using ndc models inside the IR and during response
processing and remote joins. This means that the engine generates v02
requests, and these are downgraded into v01 requests. v01 responses are
upgraded to v02 responses before being processed by the engine.
The ndc client (`execute::ndc::client`) now only takes new wrapper enum
types (`execute::ndc::types`) that split between v01 or v02
requests/responses. Every place that carries an ndc request now carries
this type instead, which allows it to carry either a v01 or a v02
request.
When ndc requests are created during planning, all creation goes via the
new `execute::plan::ndc_request` module. This inspects the connector's
supported version, creates the necessary request, and if needed,
downgrades it to v01.
When ndc responses are read during planning or during remote joins, they
are upgraded to v02 via helper functions defined on the types in
`execute::ndc::types`.
The upgrade/downgrade code is located in `execute::ndc::migration`. Keep
in mind the "v02" types are currently the same as the "v01" types so the
migration code is not doing much. This will change as the v02 types are
modified.
However, this approach has its drawbacks. One is that it prevents
changes to the ndc types [like
this](https://github.com/hasura/ndc-spec/pull/158) without a fair bit of
pain (see
[comment](https://github.com/hasura/ndc-spec/pull/158#issuecomment-2202127094)).
Another is that the downgrade code can fail at runtime and it is not
immediately obvious to developers using new, but unused, v02 features
that their new feature would fail on v01, because that mapping to v01
has already been written. Another is that we're paying some (small,
probably?) performance cost by upgrading/downgrading types because we
need to rebuild data structures.
Also:
* `execute::ndc::response` has been merged into `execute::ndc::client`,
since it was inextricably linked.
V3_GIT_ORIGIN_REV_ID: f3f36736b52058323d789c378fed06af566f39a3
### What
When generating GraphQL schema for relationship fields in model filter,
engine ignores `relation_comparisons` capability of the data connector.
Engine would generate schema for data connectors which don't have this
capability. This PR fixes that.
### How
While generating fields for the filter input type, take the relationship
capabilities into account.
The `ObjectBooleanExpressionType` and `BooleanExpressionType` objects
are quite different, hence their schema generation part is also
different, and is split in two different functions
(`build_comparable_relationships_schema`, and
`build_new_comparable_relationships_schema`). Added checking of
relationship comparison capability in both the functions.
V3_GIT_ORIGIN_REV_ID: dce2b88f7792e01e5bb390ecdb580e223ec80f01
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
# BooleanExpressionType
A new metadata kind `BooleanExpressionType` can now be defined. These
can be used in place of `ObjectBooleanExpressionType` and
`DataConnectorScalarRepresentation`, and allow more granular control of
comparison operators and how they are used.
The old metadata types still work, but will eventually be deprecated.
```yaml
kind: BooleanExpressionType
version: v1
definition:
name: album_bool_exp
operand:
object:
type: Album
comparableFields:
- fieldName: AlbumId
booleanExpressionType: pg_int_comparison_exp
- fieldName: ArtistId
booleanExpressionType: pg_int_comparison_exp_with_is_null
- field: Address
booleanExpressionType: address_bool_exp
comparableRelationships:
- relationshipName: Artist
booleanExpressionType: artist_bool_exp
logicalOperators:
enable: true
isNull:
enable: true
graphql:
typeName: app_album_bool_exp
```
```yaml
kind: BooleanExpressionType
version: v1
definition:
name: pg_int_comparison_exp
operand:
scalar:
type: Int
comparisonOperators:
- name: equals
argumentType: String!
- name: _in
argumentType: [String!]!
dataConnectorOperatorMapping:
- dataConnectorName: postgres_db
dataConnectorScalarType: String
operatorMapping:
equals: _eq
logicalOperators:
enable: true
isNull:
enable: true
graphql:
typeName: app_postgres_int_bool_exp
```
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Remove feature flag, unhide JsonSchema items, fix a few missing bits of
JsonSchema the tests didn't warn us about before.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: dd3055d926fdeb7446cd57085679f2492a4358a1
### What
This is a no-op refactor that involves unifying `get_base_type` and
`get_underlying_type_name` functions, whose motives are the same.
### How
Replace the name of erstwhile `get_base_type` fn with
`get_underlying_type_name`. Remove the latter. Update the rest of the
code to use the new function.
V3_GIT_ORIGIN_REV_ID: 0b0d999670641ba265fde153af9b43b4d865e215
### What
Update ndc-postgres configuration to v4, including the new
`mutationsVersion: "v4"`.
### How
- sed `s/experimental_/v2_/g`
- `ndc-postgres-cli upgrade`
- `ndc-postgres-cli update`
V3_GIT_ORIGIN_REV_ID: 67c608c2a3f1e232d5e0725fe8817672aa3dd627
This PR introduces support for multiple versions of the ndc-spec by
adding a new `VersionedSchemaAndCapabilities` enum variant under the
`DataConnectorLink` in OpenDD. This allows the capture of both ndc
v0.1.* and v0.2.* schema and capabilities.
This is achieved by referencing the `ndc-models` crate twice, once for
`v0.1.4` and once for the first commit after `v0.1.4`. That commit was
chosen to avoid actual v0.2.0 breaking changes for now, while we lay in
this multiple version support plumbing. Future PRs will use a newer
commit and adopt the breaking changes where necessary. The
`VersionedSchemaAndCapabilities::V02` variant uses the the v0.2
reference of `ndc-models`.
Then, during metadata resolve, when we resolve the
`DataConnectorContext` from `DataConnectorLink`, we perform a migration
of v0.1 types to v0.2 types and store and use the v0.2 types during
metadata resolve. This migration is performed in the new module
`ndc_migration`. We also record the `NdcVersion` (either `V01` or `V02`)
in the `DataConnectorLink`. The `execute` crate will need to use this to
determine which version to send to the connector at runtime (to be
implemented in a future PR).
The new changes to OpenDD are hidden from the JSON Schema via a new
`UnstableFeatures` flag, and the use of the new variant is gated behind
it in metadata resolve, since we don't yet support it upstream in the
`execute` crate.
V3_GIT_ORIGIN_REV_ID: d6d8a768ea3537c0b5e620799e94d3dd1e529526
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
Adds new Open DD type `OrderByExpression` and defines `ModelV2` type, as
described in
https://github.com/hasura/v3-engine/blob/main/rfcs/open-dd-expression-type-changes.md.
### How
- Added new types `OrderByExpression`, `ModelV2` and
`ModelGraphQlDefinitionV2` to the `open-dds` crate.
- Added new `UnstableFeatures` flag `enable_order_by_expressions` to
`metadata-resolve`. This is not yet used, as `metadata-resolve` does not
yet use the new Open DD types.
---------
Co-authored-by: Daniel Harvey <danieljamesharvey@gmail.com>
V3_GIT_ORIGIN_REV_ID: e861e88f293f1b380e6346de8490b96cec6f65bb
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
Changed tests to use the new `BooleanExpressionType` until something
broke. Fortunately something broke - we need to add the scalar boolean
expression types to the data connector scalar types for various lookups.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Split scalar boolean expression type resolve into own step so it's
available earlier in the pipeline.
Loop through them and add them to the `data_connector_scalar_types`
outputs.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: dbd8969c3e9e9d8db1d4a34e93aefc34bdf31421
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
Allow engine to connect to NDCs via HTTPS.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Add `cacert` to Docker image using Nix.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 52458920236f3868cc8daf18e140f8536d9bc674
This PR refactors the `DataConnectorContext` type in
`crates/metadata-resolve/src/stages/data_connectors/types.rs` to remove
the extra copy of scalar types it had and simplifies and removes the
nesting in the types.
`DataConnectorContext` used to have its own copy of `scalars` which
contained `ScalarTypeInfo`s. However, it already contains a more
complete copy of scalar types inside `inner.schema.scalar_types`! Turns
out the `scalars` copy (and `ScalarTypeInfo`) is unnecessary. The only
value it added was having the computed `ComparisonOperators` struct,
which is simply copied from there to where it really lives on
`ScalarTypeWithRepresentationInfo` during the
`data_connector_scalar_types` stage.
So I've moved `ComparisonOperators`, and the code that creates it, to
`data_connector_scalar_types` and deleted `ScalarTypeInfo`. This meant
that `DataConnectorContext` only contained `DataConnectorCoreInfo`
(pointless!), so I inlined `DataConnectorCoreInfo` into
`DataConnectorContext` to simplify things. This removed `.inner` calls
all through the code.
These changes help my work with supporting multiple `ndc-models`
versions because it simplifies the number of places we store and deal
with scalar types.
This PR is a functional no-op.
V3_GIT_ORIGIN_REV_ID: 3beb0b07abc7ff5cfaa7e9b60eb46aec94d7ec1a
<!-- Thank you for submitting this PR! :) -->
## Description
Upgrade Rust, as a treat. Functional no-op.
---------
Co-authored-by: Samir Talwar <samir@functional.computer>
V3_GIT_ORIGIN_REV_ID: 1e0014049e89b8658326c8d8f652df800c415526
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
In https://github.com/hasura/v3-engine/pull/750 and
https://github.com/hasura/v3-engine/pull/754 we added a number of checks
for boolean expression types that can be run once we know the data
source they will be used against.
Previously these checks were only used for model `where` clauses, this
pull request moves them into a shared folder and also checks them when
they are used as model or command arguments.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Add a new `arguments` resolve step. It doesn't fit inside the usual
`commands` or `models` step as we need the data sources for both
validated, plus access to all the resolved `relationships` outputs.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: f713659962e3f20b2c85f287b6c362fb52ffa1ed
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
Much like https://github.com/hasura/v3-engine/pull/774, we split the big
files into separate modules. Functional no-op.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Copy pasta, `just fix-local`, mostly.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: ba7e40057583f98df54573c72663a4a2d2c4a4ab
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
Tiny no-op PR to split the `command` stage of `metadata-resolve` into
smaller modules the same way `models` and others work.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
Copy and paste, run `just fix-local` to remove unrequired imports.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: ab2846be83ba0e948ea222b42eb4fd7ffa5b3523
### What
Output all traces to stdout.
---------
Co-authored-by: Daniel Harvey <danieljamesharvey@gmail.com>
V3_GIT_ORIGIN_REV_ID: 06330076ca305a331996530ddcd4d4c13d46bd95
### What
This adds a flag, `--partial-supergraph`, which instructs the metadata
resolver to prune relationships to unknown subgraphs rather than failing
to resolve.
### How
The flag gets passed through as
`metadata_resolve::configuration::Configuration`, and known subgraphs
are now tracked in `MetadataAccessor`. If the flag is set and a
relationship target refers to an unknown subgraph, we return an empty
list of relationships instead of failing.
Some test infrastructure has been added to set configuration flags per
test.
V3_GIT_ORIGIN_REV_ID: 6f0de2442a3bfc7c7a4c48e3dc7296dc1538cd67
### What
This PR adds the ability to include internal errors in API responses via
a command line argument `--expose-internal-errors`.
The default behavior remains not to show the contents of internal error
messages.
V3_GIT_ORIGIN_REV_ID: 11c47286d3fbceeda71df3a224853633aeea8902
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
A scalar `BooleanExpressionType` lets us give operators fancy names.
This PR makes them actually work.
Functional no-op as this feature is behind a feature flag.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: b49ef95d3d6672a1e27371fa5f4df63acd0849fc
### What
There was a report that using a different argument name other than
`headers` for `DataConnectorLink.argumentPresets` doesn't work. This PR
updates the test to use a different name, and this seems to work.
### How
Update the headers argument name in custom connector schema, and the
argument name in `DataConnectorLink.argumentPresets` metadata.json.
V3_GIT_ORIGIN_REV_ID: 91346c01573f666e5707c0f41f4635d689bf5b98
### What
This removes an unused field. The remaining struct had just one field
and is private to the crate, so I converted it to a type alias.
### How
In order to make `rustc` highlight the issue, I reduced the number of
types we export as `pub`, changing some to `pub(crate)`.
Then I just deleted the field once the warning showed up.
V3_GIT_ORIGIN_REV_ID: 7a26e99f062ed0f2c7449e1f57bc76068f059afb
This PR fixes the version of the NDC Spec that Open DD's JSON schema
references when it uses NDC's schema and capabilities types in the
`DataConnectorLink`. OpenDD references 0.1.4 of NDC, but uses 0.1.3 in
the JSON schema. This corrects that.
V3_GIT_ORIGIN_REV_ID: bdbb417b3227861dae7835f6d3bda0d1bf935ea7
### What
Using `String` everywhere to represent subgraph identifiers is
definitely going to cause problems at some point. We can avoid this by
using `open_dds::identifier::SubgraphIdentifier` instead.
`Identifier` and `SubgraphIdentifier` were wrappers around `String`. I
have also changed them to wrap `SmolStr` instead, for better performance
as they're cloned a lot.
### How
First of all, I hid the innards of `Identifier` and
`SubgraphIdentifier`, instead exposing certain behaviors as methods.
Mostly, this means exposing `as_str()` and `to_string()`.
Then I replaced the internals with `SmolStr`. I had to change one place
where `RefCast` was used, but that was pretty much it.
Finally, I switched out `String` for `SubgraphIdentifier` in
`QualifiedObject`. This necessitated a couple of constants for the two
"magic" subgraphs, `__globals` and `__unknown_namespace`, but was
otherwise fine.
I didn't touch `Qualified` for now.
V3_GIT_ORIGIN_REV_ID: 28664609c3173b181c3789093cb9796896642eb7
### What
The `lazy_static` macro is poorly maintained, fairly bloated, and has
been mostly superseded by
[`OnceLock`](https://doc.rust-lang.org/stable/std/sync/struct.OnceLock.html)
in the stdlib.
### How
1. I turned a couple of `static ref` values into `const`, sometimes by
creating `const fn` equivalents to other functions.
2. I inlined static behavior to construct a JSON pointer into some
tests, where we don't care too much about losing a few milliseconds.
3. For the rest, I replaced `lazy_static` with a `static OnceLock` and a
call to `OnceLock::get_or_init`.
V3_GIT_ORIGIN_REV_ID: 18e4150a5fb24fe71f6ed77fe6178b7942405aa3
NOTE: This PR is stacked on #756 and should be shipped after that is
merged.
This PR enables the existing aggregate relationships work (see #725,
#731, #756) by default by removing the experimental flag it used to be
disabled behind.
The new OpenDD schema changes that were added are also unhidden so that
they are visible in the OpenDD JSON Schema.
V3_GIT_ORIGIN_REV_ID: cfd86d8a9ea61887ccf0f1a5d08bdcc3dda59cdc
## Description
This PR implements the GraphQL schema and execution for aggregate
relationships.
In the `schema` crate, the new `model_aggregate_relationship_field`
function handles generating schema for ModelAggregateTarget
relationships. It mostly delegates the meat of its implementation to
reused logic; some refactoring has occurred to make this possible.
This involved changes in `select_many`, `select_aggregate` and
`model_arguments`. The creation of the model arguments field argument
now exists in `model_arguments` and is reused by `select_many` and
`select_aggregate`. The creation of all aggregate field arguments is now
in `select_aggregate::generate_select_aggregate_arguments`, and is then
reused when generating the aggregate relationship field. That field is
annotated with the new `RelationshipToModelAggregate` annotation.
In the `execute` crate, the logic around generating an the aggregate
selection IR was moved from `select_aggregate` into `model_selection`.
This was so it can be reused by the logic in `relationship` that now
uses it to generate an aggregate selection when encountering an
`RelationshipToModelAggregate` field.
Inside `relationship` some rearranging was done so that
`build_local_model_relationship` and `build_remote_relationship` could
work with either a normal model selection IR or the new aggregate
selection IR. The necessitated moving the creation of that IR outside
those functions into the caller, so the different callers can create
different IR (normal vs aggregate IR). This also reduced code
duplication.
New tests have been added to `engine` that cover aggregate relationships
and also remote joined aggregate relationships.
This PR also corrects two bugs in metadata resolve revealed by new
testing:
* The filter input field name in `GraphqlConfig` must be specified if
using an aggregate relationship
* The filter input type name defined on a `Model` must be specified if
that model is the target of an aggregate relationship. Conversely, the
filter input type name can be specified if the `Model` itself doesn't
define an aggregate, but is still involved in a aggregate relationship
(this previously produced an error).
This PR completes the feature, but it is still hidden behind the
experimental flag. There will be a follow up PR to remove that and
expose the functionality by default.
JIRA: [V3ENGINE-160](https://hasurahq.atlassian.net/browse/V3ENGINE-160)
[V3ENGINE-160]:
https://hasurahq.atlassian.net/browse/V3ENGINE-160?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
V3_GIT_ORIGIN_REV_ID: d499371906f7af71a4017c7c3ae75b7693cd3fa7
### What
In order to more easily monitor and review changes to metadata
resolution, this introduces snapshot testing for both successful and
failing calls to `resolve`. I used [Insta](https://insta.rs/) for this.
### How
For tests of the failure case, we already had a text file with the
expected error, so I have turned those files into snapshot files. I
wrote a small script to move the files rather than deleting and
recreating them so I could guarantee that the contents have not changed.
(Unfortunately, Git's diff doesn't always recognise the move as a move
because Insta has added a header.)
For tests of the successful case, I added a line to snapshot the
metadata rather than discarding it.
I also rewrote the tests to use `insta::glob` so we could get rid of
`test_each`.
V3_GIT_ORIGIN_REV_ID: 41bef4cf77bddb8d20d7c101df52ae149e8b0476
### What
Use `claims.jwt.hasura.io` instead of `https:~1~1hasura.io~1jwt~1claims`
to avoid the awkward escaping, as JWT claims namespace. Also, update the
tests to use the new namespace.
This is related to the recent auth docs rehaul
(https://github.com/hasura/v3-docs/pull/448#discussion_r1653368661)
V3_GIT_ORIGIN_REV_ID: 42526785ebb82f96c4f92bada054a62251c9fc7c
### What
This just means we can drop a bunch of references and pass by value.
### How
`#[derive(Clone, Copy)]` and fixing lints.
V3_GIT_ORIGIN_REV_ID: e15d323f8232755294d1f7a2c70ccf0de8a1632f
We shouldn't be replacing 'TableScan's for `information_schema` and
`hasura` schemas. Previously, we only had a check for `hasura` schema,
this PR now includes a check for `information_schema`. Queries on
`information_schema` will now work.
V3_GIT_ORIGIN_REV_ID: b25276556027b52ff940ddd3d094ea20f6fc7538
Adds a very experimental SQL interface to v3-engine for GenAI use cases.
---------
Co-authored-by: Abhinav Gupta <127770473+abhinav-hasura@users.noreply.github.com>
Co-authored-by: Gil Mizrahi <gil@gilmi.net>
Co-authored-by: Anon Ray <ecthiender@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: 077779ec4e7843abdffdac1ed6aa655210649b93
<!-- The PR description should answer 2 (maybe 3) important questions:
-->
### What
Update the interface of `GraphQLResponse`
- make some functions public
- added some helper functions that are used in v3-engine-multitenant
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 691ff8e3505f96ba9e9a2e8518023e4812319a05
<!-- Thank you for submitting this PR! :) -->
## Description
Adding a test for this and fixing a few missing parts. Mostly threading
`boolean_expression_types` everywhere and adding them to our type
lookups. Behind a feature flag so this is a functional no-op.
V3_GIT_ORIGIN_REV_ID: 5fd6d5b9e06f0216e770b0715c59c0479881017f
### What
I'm not sure why this is still hidden, but it shouldn't be.
### How
We remove the flag.
V3_GIT_ORIGIN_REV_ID: 4a73e380e0daebe3370a6561bcd4056a9013410a
<!-- Thank you for submitting this PR! :) -->
## Description
Previously we didn't check whether a boolean expression over strings was
used on a `String` field or a `User` field. Now we look up the types and
actually find out.
Functional no-op as behind a feature flag.
V3_GIT_ORIGIN_REV_ID: 8b7e94c4b873c49e206caa84e24c0d17c049c899
## What
This PR introduces a changelog file, `changelog.md`.
Any PR that is not simply a technical refactor should include a relevant
entry in this file.
Additionally we also simplify the pull request template. The template
used to contain a section for a changelog entry, which is now rendered
irrelevant.
V3_GIT_ORIGIN_REV_ID: 00881d86ffe87c4c0584b88b960837543dde34b7
This PR introduces the following changes to query usage analytics data
shape:
- The `name` field in `RelationshipUsage` is just `RelationshipName`
without `Qualified` wrapper. The `source` is already qualified, and the
same qualification applies to `name`.
- The `used` for both field and input field is a list. A field can use
multiple opendd objects at a time.
- Example: A root field can use `Model` and `Permission` (with both
filter and argument presets).
- The permission usage now revamped to express available permissions in
the opendd
- Filter predicate - provides lists of fields and relationships
- Field presets - provides a list of fields involved
- Argument presets - provides a list of arguments involved
- The `GqlFieldArgument` is dropped in favor of `GqlInputField`.
- Opendd object usage is not specified for `GqlFieldArgument`. An input
argument with object type can have field presets permission. It is
replaced with `GqlInputField` to allow specifying the permission usage.
This PR also includes JSON schema for the data shape with a golden test
to verify.
V3_GIT_ORIGIN_REV_ID: f0bf9ba201471af367ef5027bc2c8b9f915994ac
## Description
According to NDC headers pass-through spec, commands can include headers
in their responses, which are forwarded as response headers by the
engine to the client. This PR implements it.
V3_GIT_ORIGIN_REV_ID: 4fe458db02c5dd51f4674e4e013312f8e179c087
I noticed a few extra calls to `.clone()` while working on an unrelated
refactor. I want to remove them for brevity and simplicity; I don't
expect a performance improvement.
This turns on the Clippy warning `redundant_clone`, which detects
unnecessary calls to `.clone()` (and `.to_string()`).
It is an unstable warning and so might reports some false positives. If
we find any, we can suppress the warning there.
V3_GIT_ORIGIN_REV_ID: a713f29cf862d6f4cb40300105c6b9f96df00676
<!-- Thank you for submitting this PR! :) -->
## Description
When using boolean expression types on models, we have to check that any
relationships they define are local, as we currently do not support
remote predicates. This adds these checks in the `models_graphql` stage,
once we know about a) models and their sources b) boolean expressions c)
relationships.
Behind a feature flag, so strictly a no-op.
V3_GIT_ORIGIN_REV_ID: 70b2e4b316f5b8d57fa06d5492cccdddca0aaf1c
<!-- Thank you for submitting this PR! :) -->
## Description
A few debug lines slipped in recently, let's make `clippy` `warn` on
those, so they are kicked out by CI. Functional no-op.
V3_GIT_ORIGIN_REV_ID: 290f6de35f9315b68811eb5f15969fb0333e9d06
<!-- Thank you for submitting this PR! :) -->
## Description
We need to run this step after the `relationships` stage, so we can
lookup object relationships when validating boolean expression
relationships.
We move aggregation checks back to `models`, as they are also used in
the `relationships` step and we don't want circular dep hell.
Functional no-op.
V3_GIT_ORIGIN_REV_ID: 20fd777b0bc68c2bffc6febb6389a1a955ab2f16
<!-- Thank you for submitting this PR! :) -->
## Description
To check filter expression types properly, we need the resolved `source`
for all models. Previously we did not have this, as we did all model
resolving in one go. This splits that into stages, the existing `models`
resolves models and their sources, and then `models_graphql` resolves a
Model's filter expressions, aggregations and graphql schema. This means
we can inspect the model sources during the `models_graphql` in the
actual PR we want to do here.
It's massive, but it's a functional no-op, I promise.
V3_GIT_ORIGIN_REV_ID: 6c8fbf0763fb1daa209a4c84855609e8e9372e8b
## Description
Aggregate response reshaping is where the computed aggregates are
extracted from the NDC response and rewritten into the response GraphQL
shape (usually a nested object structure). Previously this reshaping was
precomputed and stored in the internal IR for the request and then
passed through to the response reshaping code. At the time, this seemed
smarter than re-walking the GraphQL annotation structure and
reinterpreting it to know how to reshape the response, when we already
do this at IR-creation time.
However, this was too smart for its own good. 😭 Unfortunately, the
necessary bits of the IR are not available when processing the an
aggregate response inside a relationship field. What _is_ available is
the GraphQL annotation structure. 🤦♂️
This PR reworks the aggregate response reshaping to solely use the
GraphQL annotation structure and removes the precomputation code and
related IR storage of it. This means the new reshaping code will be
usable for the upcoming aggregate relationships implementation. And...
it turns out that the new code actually cleaner and simpler than the old
"clever" code anyway.
This PR does not change any functionality.
V3_GIT_ORIGIN_REV_ID: 59e29cdac396c094a35c1203c02a3f27ba2c4c58
## Description
This PR refactors the `output_type::object_type_fields` function, which
is a monster, and splits out the handling of relationship fields into a
separate function. This function then itself is broken down into
separate functions that each handle one of the different types of
relationship fields.
This is particularly necessary now, as logic to handle
ModelAggregateRelationshipTargets is about to get added and that would
only grow the monster function more.
The `RelationshipTarget` `metadata_resolve` type has had its enum
variant structs broken out into separate struct types so that they can
be passed around separate to the enum. This makes it easier to write
functions that deal with each enum variant separately (like
`output_type::object_type_fields`).
This PR causes no behavioural changes.
V3_GIT_ORIGIN_REV_ID: 4707f8b07154bd4d503f78392a3d2d8976eab339
<!-- Thank you for submitting this PR! :) -->
## Description
When doing relationships across bool exps, by default we use the target
model's `where` clause bool exp. However, we allow the user to provide a
different bool exp instead, which we now pay attention to.
Behind a feature flag so a user-facing no-op.
V3_GIT_ORIGIN_REV_ID: b31157c455d56375c35db28f2418da9d41b68c46
<!-- Thank you for submitting this PR! :) -->
## Description
Previously we ignored `isNull` and `logicalOperators`. Now we obey
`isNull` for scalars, and `logicalOperators` for objects.
Both `isNull` on objects and `logicalOperators` on scalars don't exist
as features, so will create follow up tickets for these.
No new tests, but I was able to make both things fail by disabling the
appropriate options:
<img width="655" alt="Screenshot 2024-06-19 at 16 59 33"
src="https://github.com/hasura/v3-engine/assets/4729125/47eb342d-49c1-4ec8-8bfa-5e3f2d286928">
<img width="758" alt="Screenshot 2024-06-19 at 16 30 08"
src="https://github.com/hasura/v3-engine/assets/4729125/5722407c-e12e-429a-ab3a-fc221a8addfa">
Forgive me, I also broke up the `stages/boolean_expressions/object`
module into a few pieces to make this easier to think about.
This is behind a feature flag, no op.
V3_GIT_ORIGIN_REV_ID: b1273d8a9655a21adedc7c7a04a96e0cfe98cc98
<!-- Thank you for submitting this PR! :) -->
## Description
PR to implement filtering and sorting on nested fields in the custom
connector.
V3_GIT_ORIGIN_REV_ID: d112ff16beabeab1aea8d2bb24f51d22b461c8cd
## Description
This PR continues on from #725 and adds the metadata resolve logic to
validate the usage of aggregates applied to relationships. The metadata
resolve logic is gated behind a new `enable_aggregate_relationships`
flag and disabled by default.
The new resolve logic lives in
`crates/metadata-resolve/src/stages/relationships/mod.rs`, however, most
of the actual logic that validates the usage of the aggregate expression
with the relationship has been reused from the model aggregate resolve
code. Subsequently, that code
(`crates/metadata-resolve/src/stages/models/aggregation.rs`) was
refactored to return a distinct error type
`ModelAggregateExpressionError` that can be composed with the
`RelationshipError` type, so the same errors can be returned with the
additional context of the relationship they were found in.
New metadata resolve error tests have been added in
`crates/metadata-resolve/tests/failing/aggregate_expression_in_relationship/*`.
Also, two new passing metadata resolve tests have been added to cover
the happy case of root field and relationship aggregate expressions:
`crates/metadata-resolve/tests/passing/aggregate_expressions/*`
JIRA: [V3ENGINE-160](https://hasurahq.atlassian.net/browse/V3ENGINE-160)
[V3ENGINE-160]:
https://hasurahq.atlassian.net/browse/V3ENGINE-160?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
V3_GIT_ORIGIN_REV_ID: 8cd7040cc9277940641d5ac239d7b34f8c1462c5
This keeps versions in one place so we can more easily ensure we upgrade
crates together.
V3_GIT_ORIGIN_REV_ID: 6a929bb6196c19a1f66a768585b669127035e9be
<!-- Thank you for submitting this PR! :) -->
## Description
We cannot validate `BooleanExpressionType`s much until they are used.
This adds checks for when they are used as model `where` clauses. We
ensure the data connector in question is allowed to use nested filtering
(if we try and do any), and ensure that every scalar field has mappings
for the appropriate data connector.
Hidden behind a feature flag, so technically a no-op.
V3_GIT_ORIGIN_REV_ID: ab54f913157954e8197798febafc70012bfad9ad
### TODO
- [x] Validate the presence of arguments in the data connector schema
- [x] Add test for IR
- [x] Add test using custom connector
## Description
JIRA: https://hasurahq.atlassian.net/browse/V3ENGINE-148
This is part 2 of the field arguments.
This will add the field arguments to the graphql schema and also handles
the execution part.
## Changelog
- Add a changelog entry (in the "Changelog entry" section below) if the
changes
in this PR have any user-facing impact. See
[changelog
guide](https://github.com/hasura/graphql-engine-mono/wiki/Changelog-Guide).
- If no changelog is required ignore/remove this section and add a
`no-changelog-required` label to the PR.
### Product
_(Select all products this will be available in)_
- [x] community-edition
- [x] cloud
<!-- product : end : DO NOT REMOVE -->
### Type
<!-- See changelog structure:
https://github.com/hasura/graphql-engine-mono/wiki/Changelog-Guide#structure-of-our-changelog
-->
_(Select only one. In case of multiple, choose the most appropriate)_
- [ ] highlight
- [x] enhancement
- [ ] bugfix
- [ ] behaviour-change
- [ ] performance-enhancement
- [ ] security-fix
<!-- type : end : DO NOT REMOVE -->
### Changelog entry
<!--
- Add a user understandable changelog entry
- Include all details needed to understand the change. Try including
links to docs or issues if relevant
- For Highlights start with a H4 heading (#### <entry title>)
- Get the changelog entry reviewed by your team
-->
Add support for field arguments
<!-- changelog-entry : end : DO NOT REMOVE -->
<!-- changelog : end : DO NOT REMOVE -->
---------
Co-authored-by: Anon Ray <ecthiender@users.noreply.github.com>
Co-authored-by: Samir Talwar <samir.talwar@hasura.io>
V3_GIT_ORIGIN_REV_ID: ea06b896feaca30419f81cd070fd4cfd608ef35b
<!-- Thank you for submitting this PR! :) -->
## Description
Doing some work on the `models` section and it's a bit big and
confusing, so doing a tidy-up to get my head around it all. Splits the
large module into separate files (`ordering`, `source`, `graphql`,
`filter` etc).
Only actual changes are:
- Move filter resolving to after model source is checked, so we can
refer to it. This will be useful for the thing I actually want to do.
- Make steps like `graphql` and `source` return the values they create
rather than mutating `Model` directly.
- Move some filter checks into `filter` rather than `source`, now we are
able to. This logic was all over the place.
Functional no-op.
V3_GIT_ORIGIN_REV_ID: 540db16d34590ed9e481c36d251ab5e1886f2c81
<!-- Thank you for submitting this PR! :) -->
## Description
Because we globbed for folders, we had to put a test in the root of each
folder which is messy. If we glob for files instead, like the `failing`
tests do, we can have a better time. Functional no-op.
V3_GIT_ORIGIN_REV_ID: 381f5a44d9ae6100198bd28a286493bc3a2ba98e
Return a `T` instead of a `Result<T, E>` when we never return an error
(`E`) case.
I also enabled some more warnings. `unnecessary_box_returns` has been
suppressed where appropriate, and `unused_async` doesn't seem to be
violated anywhere any more.
I got rid of some calls to `.unwrap()` too.
V3_GIT_ORIGIN_REV_ID: 015ebd05978cf8c2d87474a90e0cd4333779a761
<!-- Thank you for submitting this PR! :) -->
## Description
This implements local relationships in the new `BooleanExpressionType`.
We copy all the existing tests for this and reimplement them using the
new types.
Things currently missing:
- a `BooleanExpressionType` can be specified to overwrite the target
model's `where` clause when defining. Currently we ignore this, there is
a follow up ticket for this:
https://hasurahq.atlassian.net/browse/V3ENGINE-209
This work is behind a feature flag, so this is a no-op for the user.
V3_GIT_ORIGIN_REV_ID: 6ae87e1ffb2ad0ebd27c6097e8ea87aca1a4c299
## Description
This PR adds the necessary new types and properties to OpenDD to support
aggregates over relationships. This follows the definition as laid out
in the [Aggregates
RFC](https://github.com/hasura/v3-engine/blob/main/rfcs/aggregations.md#relationship).
In order to achieve this incrementally, the changes have been made to
the OpenDD types but have been hidden from the generated JSON Schema
behind `#[opendd(hidden)]` macros. New functionality needed to be added
to the macro to support this; see the changes in opendds-derive crate.
There are also a couple of fixes to incorrect comments in the existing
aggregate expression OpenDD types.
A future PR will bring metadata resolve logic to validate the usage of
the new OpenDD properties and types.
V3_GIT_ORIGIN_REV_ID: 75de972cf8d2115248b3bf91f51f320e8c22b1f8
We pretend we handle directives but we simply set them to an empty map,
and then don't actually use them at all.
This is completely unused code that can be removed.
V3_GIT_ORIGIN_REV_ID: a86dc43acc2de2f2d3f78f0c8ebc53ce9f5bde8c
<!-- Thank you for submitting this PR! :) -->
## Description
Making the schema work for boolean expressions has been challenging as
they are no longer tied to a single data connector, so we cannot do any
checks of whether a relationships is local or remote. We defer this to
the IR step when the source data connector for a relationship is known,
so that we can generate schema for boolean expressions decoupled from
any concept of data connector.
Functional no-op.
V3_GIT_ORIGIN_REV_ID: d2923acedf92b031ba092bb83c515812c4d346f0
## Description
This PR moves all the tests that were added to
`crates/engine/tests/validate_metadata_artifacts/aggregate_expressions`
to the metadata-resolve `metadata_golden_tests`.
All the `error.txt` files got renamed to `expected_error.txt` and the
`metadata is not consistent:` bit on the front of the error got trimmed
off.
I had to change the way that `test_failing_metadata` works so that
instead of looking for directories under `failing/` it looks for
`metadata.json` files under `failing/`. This is because not every
directory has a test in it, some directories are just used for grouping
(eg `failing/aggregate_expressions/`). This doesn't change what tests
get run. The only side effect is that every test name has `_metadata`
suffixed to it (the filename). 😢 `test-each` doesn't appear to offer a
way to disable this behaviour.
V3_GIT_ORIGIN_REV_ID: dece7cd3bb4623c51050d12471d2c0990fd69bfd
Rules:
* The schema ID is set by the `id` property, falling back to `rename`.
* The schema ID is always prefixed by a URL base.
* The schema name is set by the `rename` property, falling back to `id`.
* The schema title is set by the `title` property, falling back to
`rename`, then `id`.
* If the name or title are automatically created from the ID, remove
characters that might choke a code generator, such as ' ' or '/'.
V3_GIT_ORIGIN_REV_ID: fac46a0849c2f36642daeca2971fdce7d253256a
This is Part 3 in a stacked PR set that delivers aggregate root field
support.
* Part 1: OpenDD: https://github.com/hasura/v3-engine/pull/683
* Part 2: Metadata Resolve: https://github.com/hasura/v3-engine/pull/684
JIRA: [V3ENGINE-159](https://hasurahq.atlassian.net/browse/V3ENGINE-159)
## Description
This PR implements the GraphQL API for aggregate root fields. The
GraphQL schema matches the design in the [Aggregate and Grouping
RFC](https://github.com/hasura/v3-engine/blob/main/rfcs/aggregations.md#aggregations-walkthrough).
### Schema Generation
The main new part of the GraphQL schema generation can be found in
`crates/schema/src/aggregates.rs`. This is where we generate the new
aggregate selection types. However, the root field generation can be
found in `crates/schema/src/query_root/select_aggregate.rs`.
The new `filter_input` type generation lives in
`crates/schema/src/model_filter_input.rs`. As this type effectively
encapsulates the existing field arguments used on the Select Many root
field, the code to generate them has moved into `model_filter_input.rs`
and `select_many.rs` simply reuses the functionality from there (without
actually using the filter input type!).
### IR
The main aggregates IR generation for the aggregate root field happens
in `crates/execute/src/ir/query_root/select_aggregate.rs`. It reads all
the input arguments to the root field and then kicks the selection logic
over to `model_aggregate_selection_ir` from
`crates/execute/src/ir/model_selection.rs`.
`crates/execute/src/ir/model_selection.rs` has received some refactoring
to facilitate that new `model_aggregate_selection_ir` function; it
mostly shares functionality with the existing `model_selection_ir`,
except instead of creating fields IR, it creates aggregates IR instead.
The actual reading of the aggregate selection happens in
`crates/execute/src/ir/aggregates.rs`.
The aggregates selection IR captures the nested JSON structure of the
aggregate selection, because NDC does not return aggregates in the same
nested JSON structure as the GraphQL request. NDC takes a flat list of
aggregate operations to run. This captured nested JSON structure is used
during response rewriting to convert NDC's flat list into the nested
structure that matches the GraphQL request.
The aggregate selection IR is placed onto `ModelSelection` alongside the
existing fields IR. Since both fields and aggregates can be put into the
one NDC request (even though they are not right now), this made sense.
They both translate onto one NDC `Query`. This necessitated making the
field selection optional on the `ModelSelection`
(`ModelSelection.selection`), since aggregate requests currently don't
use them.
### Planning
`crates/execute/src/plan/model_selection.rs` takes care of mapping the
aggregates into the NDC request from the generated IR.
There has been a new `ProcessResponseAs` variant added in
`crates/execute/src/plan.rs` to capture how to read and reshape an NDC
aggregates response. This is handled in
`crates/execute/src/process_response.rs` where the captured JSON
structure in the IR is used to restore NDC's flat aggregates list into
the required nested JSON output structure.
### Testing
The Custom Connector has been updated with functionality to allow
aggregates over nested object fields
(`crates/custom-connector/src/query.rs`).
New execution and introspection tests have been added to
`crates/engine/tests/execute/aggregates/` to test aggregates against
Postgres and the Custom Connector.
[V3ENGINE-159]:
https://hasurahq.atlassian.net/browse/V3ENGINE-159?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
V3_GIT_ORIGIN_REV_ID: ff47f13eaca70d10de21e102a6667110f8f8af40
This is Part 2 in a stacked PR set that delivers aggregate root field
support.
* Part 1: OpenDD: https://github.com/hasura/v3-engine/pull/683
* Part 3: GraphQL API: https://github.com/hasura/v3-engine/pull/685
JIRA: [V3ENGINE-159](https://hasurahq.atlassian.net/browse/V3ENGINE-159)
## Description
This PR implements the metadata resolve phase of the engine and adds
support for resolving `AggregateExpression`s and validates their use
when linked to a `Model`.
The bulk of the changes can be found in:
* `crates/metadata-resolve/src/stages/aggregates/*` - This is where the
`AggregateExpression`s are resolved
* `crates/metadata-resolve/src/stages/models/mod.rs` - This is where we
validate the `AggregateExpression` specified for use in the model is
actually compatible with the model and its data connector
The `ndc-spec` version used has been lifted to the latest version that
adds support for aggregates over nested objects
(https://github.com/hasura/ndc-spec/pull/144). This necessitated changes
in the Custom Connector, but actual functionality to implement
aggregation over nested objects is implemented in Part 3.
There are also some changes in
`crates/metadata-resolve/src/types/subgraph.rs` where the `Display`
trait for the various `Qualified<T>`, `QualifiedTypeReference`, etc
types has been reworked so that they print more cleanly, with the
subgraph being put outside the type syntax, and array types getting
formatted correctly. For example, previous an array of Varchars would
have printed as `Varchar (in subgraph default)!`, now it properly
formats as `[Varchar!]! (in subgraph default)`.
This was necessary to make useful error messages using these types.
A tonne of tests have been added in
`crates/engine/tests/validate_metadata_artifacts/aggregate_expressions`
to test every error condition of the metadata resolve process.
[V3ENGINE-159]:
https://hasurahq.atlassian.net/browse/V3ENGINE-159?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
V3_GIT_ORIGIN_REV_ID: ffd859127a3f1560707f06ef01906c9d1b183d31
This is a no-op refactor of IR related types. Introducing a new concrete
`IR` enum with `Mutation` and `Query` variants, where each contains a
map of root fields.
V3_GIT_ORIGIN_REV_ID: 76a197f5cf1efb34a8493c2b3356bea2bc2feb3c
<!-- Thank you for submitting this PR! :) -->
## Description
<!--
Questions to consider answering:
1. What user-facing changes are being made?
2. What are issues related to this PR? (Consider adding `(close
#<issue-no>)` to the PR title)
3. What is the conceptual design behind this PR?
4. How can this PR be tested/verified?
5. Does the PR have limitations?
6. Does the PR introduce breaking changes?
-->
This PR reverts the changes to `FieldMapping` and introduces the
`argument_mapping` in `ColumnMapping` instead.
V3_GIT_ORIGIN_REV_ID: badca2416b1a1b82d1a596aa60bf7a39b3b6152a
<!-- Thank you for submitting this PR! :) -->
## Description
It turns out that every usage of `Builder::allow_all_namespaced`
concerned itself with either introspection nodes or (in the GDS case)
`None`.
Combined with the fact that both the SDL and GDS implementations of
`SchemaContext::introspection_namespace_node` were trivial, the
`NamespacedNodeInfo` contained in
`Namespaced::AllowAll(S::NamespacedNodeInfo)` is always functionally
meaningless.
This PR removes the `NamespacedNodeInfo` from `Namespaced::AllowAll` as
well as `SchemaContext::introspection_namespace_node`, thus simplifying
the implementation.
<!--
Questions to consider answering:
1. What user-facing changes are being made?
2. What are issues related to this PR? (Consider adding `(close
#<issue-no>)` to the PR title)
3. What is the conceptual design behind this PR?
4. How can this PR be tested/verified?
5. Does the PR have limitations?
6. Does the PR introduce breaking changes?
-->
## Changelog
- Add a changelog entry (in the "Changelog entry" section below) if the
changes
in this PR have any user-facing impact. See
[changelog
guide](https://github.com/hasura/graphql-engine-mono/wiki/Changelog-Guide).
- If no changelog is required ignore/remove this section and add a
`no-changelog-required` label to the PR.
### Product
_(Select all products this will be available in)_
- [ ] community-edition
- [ ] cloud
<!-- product : end : DO NOT REMOVE -->
### Type
<!-- See changelog structure:
https://github.com/hasura/graphql-engine-mono/wiki/Changelog-Guide#structure-of-our-changelog
-->
_(Select only one. In case of multiple, choose the most appropriate)_
- [ ] highlight
- [ ] enhancement
- [ ] bugfix
- [ ] behaviour-change
- [ ] performance-enhancement
- [ ] security-fix
<!-- type : end : DO NOT REMOVE -->
### Changelog entry
<!--
- Add a user understandable changelog entry
- Include all details needed to understand the change. Try including
links to docs or issues if relevant
- For Highlights start with a H4 heading (#### <entry title>)
- Get the changelog entry reviewed by your team
-->
_Replace with changelog entry_
<!-- changelog-entry : end : DO NOT REMOVE -->
<!-- changelog : end : DO NOT REMOVE -->
V3_GIT_ORIGIN_REV_ID: 1525905488203ff07182aa700c9fff26512743ab
<!-- Thank you for submitting this PR! :) -->
## Description
<!--
Questions to consider answering:
1. What user-facing changes are being made?
2. What are issues related to this PR? (Consider adding `(close
#<issue-no>)` to the PR title)
3. What is the conceptual design behind this PR?
4. How can this PR be tested/verified?
5. Does the PR have limitations?
6. Does the PR introduce breaking changes?
-->
JIRA: https://hasurahq.atlassian.net/browse/V3ENGINE-148
This is part 1 of supporting field arguments in the engine.
Part 2 is here: https://github.com/hasura/v3-engine/pull/695
It adds the OpenDD types according to the RFC and resolves them.
V3_GIT_ORIGIN_REV_ID: f7736c968489e35c9133f2c6e276d05baf84d4be
<!-- Thank you for submitting this PR! :) -->
## Description
The level of nesting in this file makes it hard to follow, this breaks
the deepest nesting into a function. Functional no-op.
V3_GIT_ORIGIN_REV_ID: 03777bdae0c772754f5ca370dcc3d9577abc9c80
If a function doesn't return a value, terminate with a semicolon.
I also moved `implicit_hasher` and `return_self_not_must_use` to the
"definitely keep disabling this" list, and installed
[Bacon](https://dystroy.org/bacon/) in the Nix shell to make it easier
to run Clippy.
V3_GIT_ORIGIN_REV_ID: ffb17b42d982518aec433a1676dba0a0dd0ad95d
<!-- Thank you for submitting this PR! :) -->
## Description
Should have been removed in https://github.com/hasura/v3-engine/pull/680
V3_GIT_ORIGIN_REV_ID: 027d4bb88c32d400cf260677e2fef3802e479af4
<!-- Thank you for submitting this PR! :) -->
## Description
We have a set of tests in `metadata-resolve` that uses globs to find
files in either `passing` or `failing` folders and check them. This
allows those files to live in nested folders for tidyness. Functional
no-op.
V3_GIT_ORIGIN_REV_ID: 22d42e986676a110f455c59c3df9e6946c5d996c
It turns out that code generators based on JSON schemas don't always
play nicely with characters such as `' '` or `'/'`. To work around this,
when we use the schema name as a title, we replace any non-alphanumeric
character or underscore with an underscore (`'_'`).
V3_GIT_ORIGIN_REV_ID: 6af9a92c84c59294ff28bae27d7421627cde6a80
<!-- Thank you for submitting this PR! :) -->
## Description
This adds the ability to describe nested object boolean expressions,
which become `fieldPath` items in the generated
`ndc_models::ComparisonTarget::Column` items. This allows us to describe
filtering a `User` based on some element in their nested `address` field
(like `postcode`, for example).
Like the other `BooleanExpressionType` work, this remains behind a
feature flag so should make no user-facing changes.
It is also missing a whole heap of metadata resolve checks, going to
follow with these after doing the happy path to unblock other work.
V3_GIT_ORIGIN_REV_ID: c89e2942a651d349fca97706affcf40d91afeefb
<!-- Thank you for submitting this PR! :) -->
## Description
When we generate a JSON schema via `schemars`, we end up with duplicate
types in the schema, which get names like `ValueExpression2`, `Role2`,
and so on. This isn't ideal, and seems to arise for two reasons:
1. The type is polymorphic, and is monomorphised in two ways, and thus
the types can't be unified.
2. The type is monomorphic, but is used inside and outside of its home
module.
The first problem was fixed previously by splitting polymorphic types,
but the second has proven to be a bit more work. This PR finally solves
the problem by introducing a new library, `jsonschema-tidying`:
* First, we search the definitions within the JSON schema for any whose
names end in a number, such as `ValueExpression2` or `MetadataV2`.
* Then, we look for types whose names match everything up to the final
numeric digits, and discard any types for whom we can't find a match (so
we keep `ValueExpression2` because `ValueExpression` exists, but discard
`MetadataV2` because `MetadataV` does not).
* Next, we remove the duplicate definition from the definitions map,
potentially breaking links in both the schema _and_ the rest of the
definitions map.
* Finally, we traverse the entirety of the tree looking for any
references to the duplicate entry, and replace them with references to
the original entry.
This PR has no direct user-facing change, however it _will_ have an
effect on the docs generation code, which will hopefully result in
tidier documentation.
<!--
Questions to consider answering:
1. What user-facing changes are being made?
4. What are issues related to this PR? (Consider adding `(close
#<issue-no>)` to the PR title)
5. What is the conceptual design behind this PR?
6. How can this PR be tested/verified?
7. Does the PR have limitations?
8. Does the PR introduce breaking changes?
-->
## Changelog
- Add a changelog entry (in the "Changelog entry" section below) if the
changes
in this PR have any user-facing impact. See
[changelog
guide](https://github.com/hasura/graphql-engine-mono/wiki/Changelog-Guide).
- If no changelog is required ignore/remove this section and add a
`no-changelog-required` label to the PR.
### Product
_(Select all products this will be available in)_
- [ ] community-edition
- [ ] cloud
<!-- product : end : DO NOT REMOVE -->
### Type
<!-- See changelog structure:
https://github.com/hasura/graphql-engine-mono/wiki/Changelog-Guide#structure-of-our-changelog
-->
_(Select only one. In case of multiple, choose the most appropriate)_
- [ ] highlight
- [x] enhancement
- [x] bugfix
- [ ] behaviour-change
- [ ] performance-enhancement
- [ ] security-fix
<!-- type : end : DO NOT REMOVE -->
### Changelog entry
<!--
- Add a user understandable changelog entry
- Include all details needed to understand the change. Try including
links to docs or issues if relevant
- For Highlights start with a H4 heading (#### <entry title>)
- Get the changelog entry reviewed by your team
-->
_Replace with changelog entry_
<!-- changelog-entry : end : DO NOT REMOVE -->
<!-- changelog : end : DO NOT REMOVE -->
V3_GIT_ORIGIN_REV_ID: fe73acf7d9df0b9867852e673e53cb086e3725d3
<!-- Thank you for submitting this PR! :) -->
## Description
With the introduction of `NamespacedGetter` as the means deciding how to
interpret what it means to extract "namespaced" data from a schema it is
superfluous to thread along the "namespace" that lookups will be based
on.
This PR removes those namespace arguments.
As a nice side effect this removes the cases where we used to have to
supply a dummy role value when generating a role-agnostic schema.
<!--
Questions to consider answering:
1. What user-facing changes are being made?
2. What are issues related to this PR? (Consider adding `(close
#<issue-no>)` to the PR title)
3. What is the conceptual design behind this PR?
4. How can this PR be tested/verified?
5. Does the PR have limitations?
6. Does the PR introduce breaking changes?
-->
## Changelog
- Add a changelog entry (in the "Changelog entry" section below) if the
changes
in this PR have any user-facing impact. See
[changelog
guide](https://github.com/hasura/graphql-engine-mono/wiki/Changelog-Guide).
- If no changelog is required ignore/remove this section and add a
`no-changelog-required` label to the PR.
### Product
_(Select all products this will be available in)_
- [ ] community-edition
- [ ] cloud
<!-- product : end : DO NOT REMOVE -->
### Type
<!-- See changelog structure:
https://github.com/hasura/graphql-engine-mono/wiki/Changelog-Guide#structure-of-our-changelog
-->
_(Select only one. In case of multiple, choose the most appropriate)_
- [ ] highlight
- [ ] enhancement
- [ ] bugfix
- [ ] behaviour-change
- [ ] performance-enhancement
- [ ] security-fix
<!-- type : end : DO NOT REMOVE -->
### Changelog entry
<!--
- Add a user understandable changelog entry
- Include all details needed to understand the change. Try including
links to docs or issues if relevant
- For Highlights start with a H4 heading (#### <entry title>)
- Get the changelog entry reviewed by your team
-->
_Replace with changelog entry_
<!-- changelog-entry : end : DO NOT REMOVE -->
<!-- changelog : end : DO NOT REMOVE -->
V3_GIT_ORIGIN_REV_ID: 0aca2b3838f1b0d944b3c41ddd54d20dc74503a4
<!-- Thank you for submitting this PR! :) -->
## Description
We want to test `where` clauses over nested selects, so first let's get
a working test for selecting everything. Adds new test tables and
updates them in the test metadata etc. Functional no-op.
V3_GIT_ORIGIN_REV_ID: 8ebc6b27379b930eaf2220061c2ec58cd1650167
Add a test case for input type infinite recursion bug (https://github.com/hasura/v3-engine/pull/676). I confirmed the
test failure when the fix was removed.
V3_GIT_ORIGIN_REV_ID: b9d0a9b2a65328aa52f5ec057dcba6470c4e3956
