<!-- The PR description should answer 2 important questions: -->
### What
Parse offset/limit in the GraphQL -> OpenDD pipeline, and use it test
offset / limit. Functional no-op.
V3_GIT_ORIGIN_REV_ID: 068d14d08a4bde140f5633625a59137d3ff81d84
<!-- The PR description should answer 2 important questions: -->
### What
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
Earlier, for pre-response plugins, a span is emitted for each plugin
execution. This PR wraps all plugin executions in a new span that
extends the client headers as parent context.
Before:
After:
Note: `pre_response_plugins_execute` span name renamed to
`execute_all_pre_response_plugins` in later commits.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
Move execution of pre-response plugins in a task, into a new function.
Create a span that wraps a function that executes all plugins async.
V3_GIT_ORIGIN_REV_ID: a1febecf4b429e1441e4b53a9b32f39519a12ba9
<!-- The PR description should answer 2 important questions: -->
### What
Move the function/procedure planning from `sql` to the shared OpenDD IR
pipeline in `plan`. This should be a no-op for `sql`
### How
Move code, fix type errors.
V3_GIT_ORIGIN_REV_ID: 7da797ffedbc40a44692670679aa176817f2c65e
<!-- The PR description should answer 2 important questions: -->
### What
We need to bump `tonic` because of `cargo audit`, and to do so we need
to bump OpenTelemetry, and to do that we need to finally bump `http`,
and to do that...
### How
Follow the breadcrumb trail, fix the errors.
Checked, trace propagation still works:
<img width="1781" alt="Screenshot 2024-10-02 at 15 56 28"
src="https://github.com/user-attachments/assets/4c3f8763-e646-445d-84de-f507da15ad2e">
V3_GIT_ORIGIN_REV_ID: aa219e91375aa310dc656183948edc33f77cdfdc
<!-- The PR description should answer 2 important questions: -->
### What
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
Write tests to confirm websocket connection behavior in conjunction with
[graphl-ws](https://github.com/enisdenjo/graphql-ws/blob/master/PROTOCOL.md)
subprotocol.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
Test the websocket by spinning up a server in an async tokio task. Use
tokio-tungstenite for websocket client.
V3_GIT_ORIGIN_REV_ID: 32c19298b6a5b23649b22d8d820ef8d47ef1d293
<!-- The PR description should answer 2 important questions: -->
### What
Running `docker compose up` is a generally accepted way to try a project
out, and ours was broken.
This fixes ours so that running `docker compose up` builds engine and
runs it with a sample schema for `ndc-postgres`. This will allow users
to get a flavour for DDN as a consumer.
We also remove a bunch of outdated and frankly confusing stuff from the
readme, instead directing users to the docs.
V3_GIT_ORIGIN_REV_ID: 77c817d1738efafe4027c7d0da1aa21bcf78dd9d
<!-- The PR description should answer 2 important questions: -->
### What
Release `v2024-10-02`, let's do `v2024-10-02` instead.
### How
Update changelog.
V3_GIT_ORIGIN_REV_ID: a6fe7e0c81ea2d4b9cfe76322ee2586cd152b458
<!-- The PR description should answer 2 important questions: -->
### What
Previously when we used a string from a session variable as a passed
through header it would become double escaped like `\"the string\"`.
This is because we were encoding to JSON, going back to
`reqwest::header::HeaderValue` and then back to JSON. Instead, let's
stay in JSON.
### How
Instead we stay in JSON for the session variable encoding, and
explicitly do the `reqwest::header::HeaderValue` -> JSON encoding. We
take the string in `HeaderValue` and make it a
`serde_json::Value::String`.
V3_GIT_ORIGIN_REV_ID: 3fd4df7c4aac5eadedd2d533cdf8e20dfa718060
<!-- The PR description should answer 2 important questions: -->
### What
In #1168 we checked for duplicate command names. However this made a
project build fail, so we instead put this behind a flag and raise a
warning.
V3_GIT_ORIGIN_REV_ID: 41e3dcfda1d3f2da1e7ca672c9d20e0a3917d868
### What
This PR allows MBS errors to provide a larger "context". Our notion of
context is an ordered list of annotated paths. An example would be for a
name conflict, where we want to say `["this name was used here", "...
and also here"]`, so reporting two paths would be helpful.
Note that the actual format of `path` is a structured data type, not a
string. An example of the first instance introduced by this PR can be
seen here:
```json
{
"errors": [
{
"context": [
{
"message": "Data connector name given here",
"path": [
{ "Index": 11 },
{ "Key": "definition" },
{ "Key": "source" },
{ "Key": "dataConnectorName" }
]
}
],
"code": "opendds-validation",
"message": "invalid metadata: error building schema: invalid metadata: the source data connector b (in subgraph default) for model Artists (in subgraph default) has not been defined",
"path": null
}
]
}
```
The `path` key is already generated by other parts of the MBS process,
so we preserve it for now to avoid breaking changes.
### How
* A new crate, `error-context`, to hold the type of the context
structure.
* Otherwise essentially the same as the path mechanism.
V3_GIT_ORIGIN_REV_ID: 0bb529542ba0a7f6774c08fa6adb7d7c608d6c5b
<!-- The PR description should answer 2 important questions: -->
### What
Update changelog.
V3_GIT_ORIGIN_REV_ID: 5fd36aafebbad8321fb7ab95f8a7e46759441926
### What
This PR completes step 2 on the plan in #1153. With this PR, an attempt
to call MBS to build metadata containing a model referencing an unknown
data connector should result in a path to the model.
### How
#1179 introduced the polymorphic version of the error type introduced by
#1170. Unfortunately, due to Rust not having an apartness constraint, we
can't write `impl<T, S: From<T> From<WithContext<T>> for
WithContext<S>`, so we introduce the `coerce` function to allow us to
cast between `WithContext<ChildError>` and `WithContext<ParentError>`.
This means that we can return a `WithContext<ModelsError>` from the
models stage, coerce it to a `WithContext<Error>`, and propagate that
path throughout the system. This PR also extends the `path` function
inside MBS to detect these paths if present, so MBS API errors will
contain any available path.
V3_GIT_ORIGIN_REV_ID: a47c565b792fefb99294c9e21ba526424ddf7cb7
<!-- The PR description should answer 2 important questions: -->
### What
We used the old `source` key alias in doc examples, but we'd like to use
the newer `sourceType` instead.
### How
Change JSON example.
V3_GIT_ORIGIN_REV_ID: f3223fb9716a2cf277ea2e82891ae8bbcdcd9ee0
<!-- The PR description should answer 2 important questions: -->
### What
This PR makes the spans visible to the users.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: b39cc2496053f4eadaec2a260146c83e4ed5b24d
<!-- The PR description should answer 2 important questions: -->
### What
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
Create a new trace for each subscription poll and link the span. It is
done to avoid accumulation of polling spans in one trace. Also, move
waiting for poll interval to post execution.
### How
- Update `in_span_async_with_link` to `new_trace_async_with_link` which
signifies opening a new trace with a link.
- Wrap the body present `loop {}` in `new_trace_async_with_link` to emit
new trace for each poll.
- Capture poll exceptions in the trace.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: c984ef70e0e55a5edae12e3f3c0580a0065516cd
### What
When the engine is run in `PARTIAL_SUPERGRAPH` mode, any relationship
that targets an unknown subgraph should be silently dropped. This was
being done when resolving relationship navigation fields on the
`ObjectType`, but not for `comparableRelationships` on
`BooleanExpressionType`. In that case we were erroring as we tried to
resolve the relationship and couldn't find the target subgraph.
### How
The `relationships` metadata resolve step has been enhanced to capture
if a relationship is targeting an unknown subgraph. The
`object_boolean_expressions` step already used this, so it was tweaked
to skip relationships targeting unknown subgraphs.
The `object_relationships` step, however, did not use `relationships`
and instead went back to metadata_accessor for relationships. It then
had special logic that skipped unknown subgraph relationships. The step
has now been refactored to use `relationships` instead and the special
skipping logic has been discarded, and it now just uses the unknown
subgraph information from `relationships`.
In addition, the `object_relationships` step now _consumes_ the output
of the `type_permissions` step, rather than cloning it. This reduces
unnecessary cloning and makes sense since `object_relationships` is
simply a further enriched version of `type_permissions`.
The test of whether the source object type of a relationship actually
exists has also been moved from `object_relationships` to
`relationships`, and the index built by `relationships` has been
reordered to group by type first and then name, since that is more
useful, especially in the `object_relationships` step.
V3_GIT_ORIGIN_REV_ID: e5d18343f5ce24532a3258e88751bc3183692c50
<!-- The PR description should answer 2 important questions: -->
### What
Fixes:
https://linear.app/hasura/issue/ENG-1073/figure-out-whats-causing-the-slowdown-in-response-processing
This removes the accidentally-quadratic behavior of repeated
nested `from_value`/`to_value` calls where the assumption seems to have
been
that ndc model fields containing Value would be untouched (similar to
the behavior of aeson).
### How
We work with `Value` directly instead of using `serde` machinery to get
a `RowSet`.
#### Benchmarks
The large result query shows modest latency improvement:
A query with a response of the same size as above, but with deep nesting
to trigger quadratic behavior shows significant latency improvement
(~12ms to ~2ms for `process_response` span)
Small responses (common, historically) don't show any improvement, as
they don't exhibit the issue.
V3_GIT_ORIGIN_REV_ID: f709e4f8a370b46c53f533eef2fef7e772b5cb72
### What
To do this iteratively, we want to be able to make e.g. models throw
contextualised errors while other stages don't. To do this, we want to
update the models code to return its own version of `ErrorWithContext`
with its own logic for propagating paths. To avoid the mess, I've made
this polymorphic.
V3_GIT_ORIGIN_REV_ID: 8ff2082e3ac8eea908727cffac93fd9cf58ddcce
<!-- The PR description should answer 2 important questions: -->
### What
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
Add traces to `graphql-ws` crate. Since websockets are implemented by
running multiple parallel tasks, we are using span links to establish
relation between traces emitted by those tasks.
Refactor by moving code into new modules and updating function names and
types.
### How
- Define a new `in_span_async_with_link()` in tracing-util to enable
creating a span with a link.
- Wrap spans around necessary code sections with attaching required
attributes.
- Small no-op refactor in `common.rs` test file.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 33cd313b94421097f3c1bb1bcf165f04c1a9cd65
<!-- The PR description should answer 2 important questions: -->
### What
Add basic NDC execution testing for the new pipeline to the existing
`engine/tests/execution.rs` tests. Grabs rows from NDC and snapshots the
results, but good enough to eyeball and see things are somewhat working.
Somewhat interesting: we can't test `selectUnique` queries using the new
pipeline, because the primary key argument isn't an actual model
argument (and thus fails in the `argument_mappings` lookup). It's
actually taken as an argument in GraphQL, but turned into an additional
filter expression and `AND`ed onto whatever filters are there. Will need
to have a think about this, but not now.
Functional no-op.
### How
Copy the execution step from JSONAPI and use it in engine tests.
V3_GIT_ORIGIN_REV_ID: 5b1ea8c1f08048b563db694b618c8823e4139383
<!-- The PR description should answer 2 important questions: -->
### What
Another incremental step towards metadata-resolve paths. This PR wraps
the current error type with `ErrorWithContext`, a type that can
optionally contain a path. The PR should be a functional no-op as we
still ignore these paths when displaying the error; the next PR will
surface them in the error responses.
### How
We wrap the type and then do a _lot_ of hand-holding to get Rust
inference to work.
V3_GIT_ORIGIN_REV_ID: c1384cb1e8700c9b7e3aa643b7d0fbd4c8471098
<!-- The PR description should answer 2 important questions: -->
### What
Before we pull the command planning into `plan`, let's split the types
so the general and DataFusion stuff don't live in the same place.
### How
Move types, follow errors. Functional no-op.
V3_GIT_ORIGIN_REV_ID: bb4adbc6897a79f47be37d5ad1a13b7b8efb5e93
<!-- The PR description should answer 2 important questions: -->
### What
A JSONAPI request allows users to provide the fields they require like
so:
```
fields[Authors]=author_id,first_name
```
This PR implements the most basic form of this, along with some notes on
what the better version could be. This is motivated by allowing us to
test the OpenDD pipeline more than anything else.
### How
Loop through sparse field sets looking for the field we want to include,
reject it if not.
JSONAPI is behind a feature flag so is a functional no-op.
V3_GIT_ORIGIN_REV_ID: 27dbb6103c42910be71529351e7a2c9ce0abcb4a
<!-- The PR description should answer 2 important questions: -->
### What
This PR adds the pre-response plugins to the engine and engine
multitenant. We have also added a new flag/env var to get the
pre-response plugin artifacts:
- `PRE_RESPONSE_PLUGIN_CONFIG_ARTIFACT_URL_TEMPLATE`
A sample metadata for pre-response plugin is:
```json
{
"kind": "LifecyclePluginHook",
"version": "v1",
"definition": {
"name": "logging",
"url": {
"value": "http://localhost:5001/log"
},
"pre": "response",
"config": {
"request": {
"headers": {
"additional": {
"hasura-m-auth": {
"value": "your-strong-m-auth-key"
}
}
},
"session": {},
"rawRequest": {
"query": {},
"variables": {}
},
"rawResponse": {}
}
}
}
}
```
Please note that the execution of pre-response plugins happens on a new
tokio thread
([tokio::spawn](https://docs.rs/tokio/latest/tokio/task/fn.spawn.html))
which will run in the background. We will NOT wait for any of the task
to complete before sending the response to the user.
An example of the trace with 5 pre-response plugin calls:
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
We have added this in the plugin middleware.
V3_GIT_ORIGIN_REV_ID: 270d77c45d5e8cc85a5f281f59f981f210eb4547
Linear:
https://linear.app/hasura/issue/APIPG-395/query-usage-analytics-report-relationship-fields
### What
This PR adds mappings to the relationship usage analytics. The goal is
to improve the tracking of relationships between models in the system by
mapping relationship fields (source to target fields) and relationship
arguments, ensuring that analytics around these mappings can be
accurately captured.
No changelog entry is required as the change is not user-facing.
### How
The implementation introduces the following changes:
- Added `get_relationship_model_mappings` and
`get_relationship_command_mappings` functions to extract mappings of
relationship fields and arguments.
- Updated the `RelationshipUsage` structure to include the new
`mappings` field for both model and command mappings.
These changes affect various components handling relationships within
the analytics system, as seen in files such as
`query_usage_analytics.rs`, where mappings are now tracked across
different relationship types.
### Example analytics:
For the following query:
```graphql
query MyQuery {
ArticleMany {
# relationship to command
AuthorFromCommand {
id
}
# relationship to a model
Author {
id
}
}
}
```
The query usage analytics looks something like:
- For relationship to model:
```yaml
used:
- relationship:
name: article
source:
subgraph: default
name: commandArticle
target:
model:
model_name:
subgraph: default
name: Articles
opendd_type:
subgraph: default
name: article
relationship_type: Object
mapping:
- source_field: article_id
target_field: article_id
deprecated: false
deprecated_reason:
```
- For relationship to command:
```yaml
used:
- relationship:
name: AuthorFromCommand
source:
subgraph: default
name: article
target:
command:
command_name:
subgraph: default
name: get_author_by_id
opendd_type:
subgraph: default
name: commandAuthor
mapping:
- source_field: author_id
target_argument: author_id
deprecated: false
deprecated_reason:
```
V3_GIT_ORIGIN_REV_ID: e26d30af7a2b77156d1120bd1a51f8d44b9dc443
<!-- The PR description should answer 2 important questions: -->
### What
Much in the vein of https://github.com/hasura/v3-engine/pull/1166, we
move the model aggregate planning from `sql` to the `plan` crate. No
tests actually exercise this code in the OpenDD IR pipeline yet, perhaps
if we extend the GraphQL -> OpenDD IR pipeline we can put it under test.
### How
Move the code, fix the errors. Functional no-op.
V3_GIT_ORIGIN_REV_ID: 7beee0aec828296fefa24c975d4662a20aa0d2e5
Bumps [syn](https://github.com/dtolnay/syn) from 2.0.77 to 2.0.79.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/dtolnay/syn/releases">syn's
releases</a>.</em></p>
<blockquote>
<h2>2.0.79</h2>
<ul>
<li>Fix infinite loop on parsing chained ranges (<a
href="https://redirect.github.com/dtolnay/syn/issues/1741">#1741</a>)</li>
<li>Fix panic in parsing <code>use</code> items containing absolute
paths (<a
href="https://redirect.github.com/dtolnay/syn/issues/1742">#1742</a>)</li>
</ul>
<h2>2.0.78</h2>
<ul>
<li>Fix infinite loop on chained comparison (<a
href="https://redirect.github.com/dtolnay/syn/issues/1739">#1739</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="732e6e3940"><code>732e6e3</code></a>
Release 2.0.79</li>
<li><a
href="af63396422"><code>af63396</code></a>
Merge pull request <a
href="https://redirect.github.com/dtolnay/syn/issues/1742">#1742</a>
from dtolnay/usecrateroot</li>
<li><a
href="31e8632338"><code>31e8632</code></a>
Fix construction of UseGroup containing crate roots</li>
<li><a
href="037861ac3c"><code>037861a</code></a>
Merge pull request <a
href="https://redirect.github.com/dtolnay/syn/issues/1741">#1741</a>
from dtolnay/binoploop</li>
<li><a
href="8df4dd0fa4"><code>8df4dd0</code></a>
Force cursor to advance in parse_expr calls</li>
<li><a
href="09d020f5a1"><code>09d020f</code></a>
Release 2.0.78</li>
<li><a
href="7eaebfbb47"><code>7eaebfb</code></a>
Merge pull request <a
href="https://redirect.github.com/dtolnay/syn/issues/1739">#1739</a>
from dtolnay/chainedcomparison</li>
<li><a
href="b3d2886fc9"><code>b3d2886</code></a>
Fix infinite loop on chained comparison</li>
<li><a
href="3f3d0c57ac"><code>3f3d0c5</code></a>
Add regression test for issue 1738</li>
<li><a
href="346efaec55"><code>346efae</code></a>
Touch up PR 1737</li>
<li>Additional commits viewable in <a
href="https://github.com/dtolnay/syn/compare/2.0.77...2.0.79">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: bc56f0e5a76ae84d54e24ad11c786202a278dec8
Bumps [async-trait](https://github.com/dtolnay/async-trait) from 0.1.82
to 0.1.83.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/dtolnay/async-trait/releases">async-trait's
releases</a>.</em></p>
<blockquote>
<h2>0.1.83</h2>
<ul>
<li>Prevent needless_arbitrary_self_type lint being produced in
generated code (<a
href="https://redirect.github.com/dtolnay/async-trait/issues/278">#278</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="e6b4d471c6"><code>e6b4d47</code></a>
Release 0.1.83</li>
<li><a
href="bfe8d61766"><code>bfe8d61</code></a>
Merge pull request <a
href="https://redirect.github.com/dtolnay/async-trait/issues/278">#278</a>
from dtolnay/arbself</li>
<li><a
href="7bd974b3a4"><code>7bd974b</code></a>
Ignore needless_arbitrary_self_type clippy lint in generated code</li>
<li><a
href="b53b4e7554"><code>b53b4e7</code></a>
Add regression test for issue 277</li>
<li>See full diff in <a
href="https://github.com/dtolnay/async-trait/compare/0.1.82...0.1.83">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: f1b834b7b1078af67662dfd1d55a94c1ba5494f7
Bumps
[async-graphql-parser](https://github.com/async-graphql/async-graphql)
from 7.0.9 to 7.0.11.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/async-graphql/async-graphql/blob/master/CHANGELOG.md">async-graphql-parser's
changelog</a>.</em></p>
<blockquote>
<h1>[7.0.11] 2024-09-26</h1>
<ul>
<li>fix <a
href="https://redirect.github.com/async-graphql/async-graphql/issues/1598">#1598</a></li>
</ul>
<h1>[7.0.10] 2024-09-24</h1>
<ul>
<li>add <code>SchemeBuilder.limit_directives</code> method to set the
maximum number of directives on a single field.</li>
<li>remove needless ?Sized <a
href="https://redirect.github.com/async-graphql/async-graphql/pull/1593">#1593</a></li>
<li>fix: generate each variant description correctly. <a
href="https://redirect.github.com/async-graphql/async-graphql/pull/1589">#1589</a></li>
<li>Make <code>From<T></code> for [Error] set source <a
href="https://redirect.github.com/async-graphql/async-graphql/pull/1561">#1561</a></li>
<li>feat(graphiql): add support for WS connection params <a
href="https://redirect.github.com/async-graphql/async-graphql/pull/1597">#1597</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a
href="https://github.com/async-graphql/async-graphql/commits">compare
view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
</details>
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
V3_GIT_ORIGIN_REV_ID: 49257197190f6dd1a1a207d058245a582592a4d4
### What
In order to talk about artifacts in MBS, we pull in the entirety of
engine-multitenant, which pulls in the entirety of engine, which is
quite a big dependency tree. This PR factors that code out into a
separate package, `build-artifacts`, to lighten the MBS dependency
burden.
---------
Co-authored-by: Daniel Harvey <danieljamesharvey@gmail.com>
V3_GIT_ORIGIN_REV_ID: ad60badb5b793a80cf60cf4b040501eb6abc6bb2
### What
Previously, MBS would not complain if two commands had the same root
field name, and would instead just keep whichever was resolved last.
This has now been fixed.
### How
We do precisely what we do with all the other steps: check a running
list of graphql names that have already been used. We've added a test
that does indeed fail on `main`, which now works.
V3_GIT_ORIGIN_REV_ID: fdf8f73acc62abf315125636c0010b01f6cdb96b
<!-- The PR description should answer 2 important questions: -->
### What
We're building a new OpenDD IR pipeline. The `sql` crate already has a
lot of what we need, so let's take the model selection parts (ie, not
aggregates yet), pull them into the `plan` crate, and re-use them for
both `sql` and the `jsonapi` pipelines.
The broad idea here is that the shared `plan` will get incrementally
bigger, and `sql` will get smaller.
This is a functional no-op for `sql`, and slightly improves the WIP
JSONAPI pipeline as we enjoy better permission checks.
### How
- Copy model planning and helper functions from `sql` into `plan`
- Replace instances `DataFusionError` with a smaller local `PlanError`
- Fix JSONAPI to use these new `plan` functions
- Remove the code in `sql`, instead using the shared `plan` functions in
planning, mapping back into `DataFusionError` as appropriate.
V3_GIT_ORIGIN_REV_ID: 50314442b9b56f31d2b38a0cf6f104e265bc3886
### What
This will save me minutes of time a week.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: d441b07fcf69b30c59902198e025669df410346c
<!-- The PR description should answer 2 important questions: -->
### What
The references are making multiple frontends difficult to implement,
let's wrap them with `Arc` instead and have an easier time.
### How
Change the types, follow the errors. Functional no-op.
V3_GIT_ORIGIN_REV_ID: 8baea2bd6c0e56e8bfb1f899b8d15731eebfa976
### What
We'd like to use `NdcFieldAlias` in the `plan` crate, however because of
the ways deps between `graphql_ir` and `execute` work we cannot without
a cycle. Functional no-op.
### How
Create a new crate that depends on nothing for planning-related domain
types.
V3_GIT_ORIGIN_REV_ID: c441f2de2eba01bda59ce16e1e4b0e4d9f765d78
And bump other opentelemetry-* deps to compatible with 0.23
<!-- The PR description should answer 2 important questions: -->
### What
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
Need
[add_link](https://docs.rs/opentelemetry/0.23.0/opentelemetry/trace/trait.Span.html#tymethod.add_link)
API on Spans for tracing `graphql-ws` websockets code. It is available
opentelementry version `0.23` and above. This also bumps all the
dependents of the opentelemetry crate (`opentelemetry-*`)
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
By updating root `Cargo.toml`.
V3_GIT_ORIGIN_REV_ID: 87c1bedde35f5e8e91366aceb1379ab5d43b5b99
<!-- The PR description should answer 2 important questions: -->
### What
When we merged the PR that added `ResolveFilterExpressionContext`
(amongst other changes, sadly), the `Generate Query Plan` got slower.
Changing this to a reference to try and improve it. Locally run
benchmarks show this as mostly an improvement.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
Use reference to `http_context` inside `ResolveFilterExpressionContext`,
remove resulting `.clone()` calls.
V3_GIT_ORIGIN_REV_ID: b7e728cf4f376f7c69b83eab0d79a43c90ee265b
<!-- The PR description should answer 2 important questions: -->
### What
We are going to need tests for our OpenDD IR pipeline, and we are going
to need to convert GraphQL requests into OpenDD IR at some point, so
this makes the most basic `normalized_ast -> OpenDD IR QueryRequest`
pipeline and implements / tests it for the simplest possible query.
This only affects tests at this point, so is a functional no-op.
### How
This PR adds the most basic `normalized_ast -> OpenDD IR QueryRequest`
pipeline and implements / tests it for the simplest possible query.
```rust
enum TestOpenDDPipeline {
Skip,
GenerateOpenDDQuery,
GenerateExecutionPlan,
}
```
It adds a flag for each engine test to opt-in for testing with the new
pipeline. Currently one passes `GenerateOpenDDQuery`, and tests the
result against a snapshot, and the rest pass `Skip`.
The unblocks two following steps:
- we can improve the GraphQL -> OpenDD IR generation, enabling more
tests by passing `GenerateOpenDDQuery`
- once the main new `plan` pipeline generates the same types as the
existing `execute` crate, we can compare the old `execute::plan` with
the new one, and enable that per test by passing
`GenerateExecutionPlan`.
Once all the tests are passing `GenerateExecutionPlan` we can remove the
flag and we know we'll have parity in plan creation.
V3_GIT_ORIGIN_REV_ID: 607dfce77b68849c7fc66fc652e38182fa0c83ea
<!-- The PR description should answer 2 important questions: -->
### What
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
Use polling interval from OpenDD metadata instead of hard-coded 2
seconds.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
The interval value (in milliseconds) is already available in
subscription plan. Use it.
V3_GIT_ORIGIN_REV_ID: be99c3367b8d29b085fe536c26cd44ed601c7d91
<!-- The PR description should answer 2 important questions: -->
### What
If a model has arguments, but they are all provided by presets, then
previously we would require users to pass an empty `args: {}` argument
like this:
```graphql
query MyQuery
ActorsByMovieMany(args: {}) {
actor_id
movie_id
name
}
}
```
There is no need for this, so this PR loosens this restriction, by
providing a default empty value. This means users can also do the above
query with:
```graphql
query MyQuery
ActorsByMovieMany {
actor_id
movie_id
name
}
}
```
Because both versions now work, this is a non-breaking change.
### How
Instead of just looking at number of arguments in schema generation,
consider which have been prefilled and provide a default empty value if
there is nothing a user could pass anyway.
V3_GIT_ORIGIN_REV_ID: cf184e42a114df782e1480a8f19548dda31e5992
<!-- The PR description should answer 2 important questions: -->
### What
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
This PR enables "unit" testing for `execute_request` function from
`graphql-ws` crate which is responsible for executing
graphql operations. It is tested in conjunction with the
`graphql_frontend`'s `execute_query` by comparing responses from the
both.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
Briefly
```
http_response = graphql_frontend::execute_query
ws_response = graphql_ws::execute_request
compare(http_response, ws_response)
```
V3_GIT_ORIGIN_REV_ID: 371ac3de1136732d7dc2e88dbd093264a96a7d2e
<!-- The PR description should answer 2 important questions: -->
### What
We would like to show error paths for `metadata-resolve` so that
debugging these errors is a little less painful, both for us and end
users. To this end, #1147 introduced a type wrapper that would be
deserialised to contain its own JSON path, so we could then pass this
path to errors. This PR does precisely this for the
`UnknownModelDataConnector` error.
I chose this error because... it was the first one on the list, not for
any reason beyond that. Right now, this is an extremely simple case
whereby only one path is required, however other errors may need two
("name at path X conflicts with name at path Y", for example). This PR
also changes the default engine error stdout to show the `Debug`
instance rather than the `Display` instance, as the error path is
discarded by the `Display` instance. Unfortunately, we use `Display` for
both stdout and user responses, which is maybe something we'd want to
change eventually, but for now this means we can't just add the error
path to the `Display` instance.
### How
I started by making `Model` a `Spanned` element within the metadata
structure. I then added the `path` key to the resolved `Model` type. I
then found the first error type that included a model name, and added
the `path` key to that error variant. Then, I just did the wiring.
You'll note that this error doesn't _alway_ return a path because it
isn't always raised by a model-first code path, but this is probably the
first PR of many.
### Next steps
* Next step is to make the output a little neater, probably by creating
an actual structured error type (most likely a lot like `Spanned`, with
a `path` and a `value`). Then, we can use a `Display` instance again to
print this nicely in the stdout, but ignore the path in the MBS API
response.
* After that, the plan is to stop ignoring it in the MBS API response,
with a new key to hold an error path.
* Step three is to allow for errors to produce multiple error paths in a
list, hopefully such that they tell a story ("I found this... but then I
found this... and those two things conflict")
* Step four will be a wave of PRs that look quite similar to this one,
wiring up paths to as many errors as possible.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 2d8dda018055f65711e66b08aa15188b516e2ddc
<!-- The PR description should answer 2 important questions: -->
### What
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
- Introduce a brand new `graphql-ws` crate that implements
- WebSocket request handling by `WebSocketServer` struct
-
[graphql-ws](https://github.com/enisdenjo/graphql-ws/blob/master/PROTOCOL.md)
protocol and handles subscriptions in async tokio tasks.
- OSS engine now handles GraphQL websockets through `GET /graphql`
handshakes.
### How
Refer to added
[architecture.md](https://github.com/hasura/v3-engine/blob/rakeshkky/graphql-ws-crate/crates/graphql-ws/architecture.md)
file.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 3032cbe50267d3f0102c450be2749c09fb3992bf
<!-- The PR description should answer 2 important questions: -->
### What
This PR adds the `Spanned` type: an OpenDD wrapper that can be placed
inside the Metadata. It's basically a pair of the value and the path to
the value in the original metadata. This allows us to do things like
source maps.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
### How
It's almost exactly what @danieljharvey proposed originally.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
---------
Co-authored-by: Daniel Harvey <danieljamesharvey@gmail.com>
V3_GIT_ORIGIN_REV_ID: 4f037686b6981fffc4b0a8ac8f95c2f9c623af67
