### What
This PR adds resolving logic that can promote warnings (plural) to an
error based on the `ddn_flags::Flags`.
In addition it adds a flag `require_valid_ndc_v01_version`, which, when
set, promotes a `DataConnectorIssue::InvalidNdcV01Version` to an error.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 2d6b089f6237fea85c889243547f4deffa4c20ee
### What
Passing `Configuration` by reference signals intent (there is only ever
one configuration that is given at the toplevel which is never altered)
better than owning and copying values.
While in its current state it requires slightly more memory to pass a
reference to a `Configuration` rather than a value, this does not seem
like the thing to optimize for when it inhibits readability.
V3_GIT_ORIGIN_REV_ID: e5e05ad2e7ee41dfb49173ec9575de17552c63ae
<!-- The PR description should answer 2 important questions: -->
### What
Make the plugin execution trace user-facing.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
By changing the SpanVisibilty from `Internal` to `User`.
V3_GIT_ORIGIN_REV_ID: 9afe59c01ebb7549235705566516319a73fd5da1
<!-- The PR description should answer 2 important questions: -->
### What
We want to use `CompatibilityConfig` to configure turning warnings into
errors after a certain date, and to make sure we don't break old builds.
This allows passing a path to a optional compatibility config file to
engine and scaffolds how we'll map this config to metadata resolve
options.
### How
Add a flag to `v3-engine`, parse the file if flag is set. Test it by
adding static test file and using it with `just watch` and `just run`.
V3_GIT_ORIGIN_REV_ID: 972c67ae29905b9ce1bb57e150f4cfcfd6a069ef
### What
Upgrade to the latest version of Rust.
### How
I modified `rust-toolchain.toml` and ran `cargo update`.
V3_GIT_ORIGIN_REV_ID: 5fe675ee5dc568cd24de542c826ffd2d355f90ff
### What
Updates the version number to `3.0.0`, now we're officially at GA.
### How
I updated the version number, and also updated some dependencies. The
`Cargo.lock` file was not up to date.
V3_GIT_ORIGIN_REV_ID: 3e9b784c8ad856556193e56aee9fa607fa8ce19e
<!-- The PR description should answer 2 important questions: -->
### What
Add more types for validating compatibility configuration. Functional
no-op.
V3_GIT_ORIGIN_REV_ID: 56239149762bc170ce40a7f5c526cb8f3f49ec61
<!-- The PR description should answer 2 important questions: -->
### What
`v3-engine` needs to be able to read and use compatibility
configuration, so let's move it to a crate here.
### How
Move `CompatibilityConfig` and `CompatibilityDate`.
V3_GIT_ORIGIN_REV_ID: 2de4b22411d296d6a354e2856e685aea4b261637
<!-- The PR description should answer 2 important questions: -->
### What
1. Commands backed by ndc procedures can now be executed using in sql as
follows:
```sql
select * from command(args); -- args to be provided with struct syntax
```
2. There is an optional `disallow_mutations` field in the sql request
that defaults to `false`. When it is explicitly set to true, mutations
are disallowed. This is for pacha to get a confirmation for mutations.
### How
Most of the code is reused from ndc functions. There is a new
'NDCProcedurePushDown' physical node and the associated changes to a
mutation response from a connector.
V3_GIT_ORIGIN_REV_ID: 94913ab931290e0aa91ccd01173955da3aa1e423
<!-- The PR description should answer 2 important questions: -->
### What
The motivation was to allow introspecting field descriptions of object
types. To accomplish this, a fair number of things had to change. Prior
to this, we determined a model's and command's schema at the sql layer
by looking at its type mappings. When introspecting these models and
commands, if there are any nested fields, they are output as
'STRUCT<field1 type1, field2 type2>' without any descriptions.
With this change, we first build a catalog of types at the sql layer
from scalars and object types defined at opendd layer. These types are
then used when adding models and commands as opposed to looking at their
type mappings. This also changes how the type catalog is presented in
introspection. One can now query 'struct type's and their fields along
with their descriptions. Models and commands merely refer to these
struct types.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 550da03c84b33ca44851858ee2cb73a31674d3d0
<!-- The PR description should answer 2 important questions: -->
### What
Push down limits and offsets, including inside existing limit and offset
queries.
### How
Adds a new rewrite which pushes a limit/offset inside an `NDCQuery`, in
case it is not picked up by the sort pushdown or the table scan
pushdown.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 2dcf14ff9f605919abb047560681615b88e766e2
<!-- The PR description should answer 2 important questions: -->
### What
Same as with `ndc-postgres` and `ndc-postgres-multitenant`, we call a
`just` command in CI, rather than using an actions plugin.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 96daa602e4647fdcd641d79539df66a264914efc
### What
If `prettier` isn't installed globally, this means the format script
still works.
### How
If it's already installed, this is a no-op. Otherwise, this prompts the
user to download a temporary version of the package to run the command.
V3_GIT_ORIGIN_REV_ID: 6d10387a0a6d5cdc33b748c0533b7bb276d322ac
<!-- The PR description should answer 2 important questions: -->
### What
Add schemars title and description to `AuthConfig` enum variants.
### How
Use new `opendd` `json_schema` options.
---------
Co-authored-by: Daniel Harvey <danieljamesharvey@gmail.com>
V3_GIT_ORIGIN_REV_ID: 8896357dc8c400101587e36dfb14f807cbdc4bc0
<!-- The PR description should answer 2 important questions: -->
### What
Even though aggregates are not pushed down right now, this makes sure
they work at the datafusion layer and verifies the plans.
### How
Another group of SQL test files
V3_GIT_ORIGIN_REV_ID: a2a29d6f6347bb1b028c06313e98ed16bb7172a0
<!-- The PR description should answer 2 important questions: -->
### What
Supports calling functions from the SQL interface
### How
Similar to models.
V3_GIT_ORIGIN_REV_ID: 2958aeacdfd31bae0e4353cb7e20e627c84931b5
<!-- The PR description should answer 2 important questions: -->
### What
Adds tests for select, filter and order by.
### How
Reuses the existing test framework. There is a giant `metadata.json`
file which is used for all tests, based on Postgres. Each test is a
folder with a `query.sql` file, an `expected.json` for expected output,
and a `plan.json` for the expected explain output.
V3_GIT_ORIGIN_REV_ID: 7f5c134c5d3cf47e5f2ffa305f24f4274ccd545e
<!-- The PR description should answer 2 important questions: -->
### What
Push down the `_lt`, `_lte`, `_gt`, `_gte` operators
### How
By matching operators with the same name on the NDC side, for now, until
we have additional NDC operator meanings that we can use.
V3_GIT_ORIGIN_REV_ID: 4341490a3cdbb62e9fe90c10279527716687545d
<!-- The PR description should answer 2 important questions: -->
### What
Implements a new optimizer pass which pushes sort stages inside
`ModelQuery` stages.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 2820f88003aec376b71605c0f753d7b50825ddad
### What
Push down the following SQL predicates to NDC via OpenDD IR:
- Logical operators AND, OR and NOT
- Equality operator and inequality operator
TODO:
- [ ] Comparison operators
- [x] `IS NULL` and `IS NOT NULL`
- [ ] Validate operators actually exist in the OpenDD metadata and NDC
mappings
- [ ] Use the actual OpenDD operators instead of the stand-in `_eq` and
`_neq` operators
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
### How
- `plan/filter.rs` implements two functions `can_pushdown_filter` and
`pushdown_filter` (which translates to OpenDD IR)
- `planner/filter.rs` translates OpenDD IR to NDC IR for execution.
V3_GIT_ORIGIN_REV_ID: ee2c38bf2c36292710becbabfaca51ccae983a2f
### What
Update dependencies in preparation for some cloud work, and move
dependency versions to the workspace.
### How
```
$ cargo update
```
V3_GIT_ORIGIN_REV_ID: a42ba0c8a1cb376280f9771abd7d1f4f7bf19b84
<!-- The PR description should answer 2 important questions: -->
### What
This is a no-op change. Moves model related planning code from the
top-level to `planner` submodule. This is in preparation for commands
implementation.
V3_GIT_ORIGIN_REV_ID: 97a73ced40dadf168efc1147e52ef4f36103bc50
### What
We now support cloud-only crates, which are not open-sourced.
### How
Anything in `crates/cloud` will not be synced with the _graphql-engine_
repository.
In order to facilitate this, we generate and commit a
Cargo.toml/Cargo.lock pair with the cloud-only sections removed. We also
transform the justfile to remove this code.
This includes only a test repository, to ensure that nothing private is
synced.
When this is merged, it should not result in a commit to the
_graphql-engine_ repository.
V3_GIT_ORIGIN_REV_ID: 038839acdf3a97da05bbd4b6278171cc12e7cd71
<!-- The PR description should answer 2 important questions: -->
### What
Introduce the `json_schema` attribute for enum variants to specify
schema metadata for adjacently tagged (`as_versioned_with_definition`)
enums. For untagged and internally tagged enums, metadata is inherited
from the corresponding variant type.
Also, update the `README.md` in `opendd-derive` crate.
### How
Update types and generate metadata expression for adjacent tagged enums.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 5c7dff5c7d7f4f44f0a27e26f1d8941fb861c346
<!-- The PR description should answer 2 important questions: -->
### What
Remove the projection pushdown optimization. `datafusion` already
optimizes this to the correct NDC IR.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 6e0034e4d920c39b70667f5f521341069a5c53de
<!-- The PR description should answer 2 important questions: -->
### What
By default, `rustc` splits crates up and builds in parallel. This is
faster but misses some optimisations between these sections. Let's
reduce it to increase runtime performance.
### How
Add settings to `release` profile in `Cargo.toml`.
<img width="776" alt="Screenshot 2024-08-13 at 12 59 46"
src="https://github.com/user-attachments/assets/a03389dc-80ba-4723-8ca3-36af50846324">
V3_GIT_ORIGIN_REV_ID: 44fa511024140b680b30c9abfaa48034bf0845a9
<!-- The PR description should answer 2 important questions: -->
### What
We cannot support having relationship comparisons in nested object field
filters of a boolean expression. NDC does not support this natively
([slack
thread](https://hasurahq.slack.com/archives/C05HND0F6LB/p1722845028319099)).
This PR adds a metadata build check for this case and reject such
boolean expressions.
Tests in the PR is partially based on
https://github.com/hasura/v3-engine/pull/935.
**Note:** This might break older builds having relationship comparisons
in the nested object field filter expressions. This is a rare scenario.
We will check through our schema-diff job for any failing builds. If
there are significant, we might hold this PR. Full context in this slack
thread -
https://hasurahq.slack.com/archives/C06P2U8U55G/p1723121764332539.
### How
- Add a metadata build check for boolean expressions that restricts them
to have nested object filters with relationship comparisons.
- Raise GraphQL API runtime internal error while building the filter IR
when a relationship comparison found within a nested field filter.
---------
Co-authored-by: Daniel Chambers <daniel@hasura.io>
V3_GIT_ORIGIN_REV_ID: 86dce0b784c77e57bb1888125dba5b599e40f741
<!-- The PR description should answer 2 important questions: -->
### What
We started using `mimalloc` allocator in MBS a while ago with good
results, let's use it here too.
Once `v3-engine-multitenant` is merged we should use it there too.
### How
Import crate, switch it on in engine binary and in benchmarks.
<img width="845" alt="Screenshot 2024-08-13 at 10 10 49"
src="https://github.com/user-attachments/assets/dc872668-1633-468a-86d3-51fca5be68bf">
V3_GIT_ORIGIN_REV_ID: ebad91bb57964477d0f227e341c7bd12d54f0f68
<!-- The PR description should answer 2 important questions: -->
### What
Breaking down the big `Error` type more. This creates `ModelsError`.
Functional no-op.
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
### How
Mostly moving things and making types more specific. In one case, making
a type more general (added comments on how to resolve this in future)
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 79b14085ca1111489621b8b7d3490492a4c30ead
<!-- The PR description should answer 2 important questions: -->
### What
Updates datafusion dependency to `41`.
### How
Fixes for breaking changes.
V3_GIT_ORIGIN_REV_ID: cc957f6c7ff9b3c004cc5e0cbb48387c1f963b7a
### What
Introspection queries (on 'hasura' schema) would fail when there is no
data in the underlying tables.
### How
A more robust 'MemTable' with a comprehensive set of tests is introduced
which shouldn't run into these issues.
V3_GIT_ORIGIN_REV_ID: e09de03e8d093fb4348514cfed6b6dc1d9b0b0c8
<!-- The PR description should answer 2 important questions: -->
### What
- Add columns with nested fields to the SQL schema
- Alias nested fields appropriately in order to support them for query
execution
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
### How
- Translate OpenDD types to Arrow types during schema generation
(`to_arrow_type`)
- Generate `NestedField` structures during planning to prepare data in
the right format during execution (`fields_for`)
V3_GIT_ORIGIN_REV_ID: d37d2eade2fd5c0f08861c1bbc6368a88299b0f3
<!-- The PR description should answer 2 important questions: -->
### What
Renables Github benchmarking after we removed it in
https://github.com/hasura/v3-engine/pull/819
### How
Tell Criterion to only sample for 5 seconds each time to stop each
benchmark going on forever. This makes the whole run take a reasonable
10 minutes.
V3_GIT_ORIGIN_REV_ID: 364be6490f4f4b21877849daf1f734fa51ecf542
<!-- The PR description should answer 2 important questions: -->
### What
Upgrade to [Rust
1.80.0](https://blog.rust-lang.org/2024/07/25/Rust-1.80.0.html)
### How
Update `rust-toolchain.yaml` and Dockerfiles, fix warnings.
V3_GIT_ORIGIN_REV_ID: ba797e1aba6b9623a921734473a6b70a2a38c8b7
<!-- The PR description should answer 2 important questions: -->
### What
When querying a table with no data through SQL would result in an error.
### How
Instead of returning a `RecordBatch`, arrow_json's implementation
returns an `Option<RecordBatch>`, we now account for `None`.
V3_GIT_ORIGIN_REV_ID: 459440e82aeb1b2faa009405e025fc024497d5b4
