<!-- The PR description should answer 2 important questions: -->
### What
Our Nix build was building all the workspace crates as part of it's deps
step. This means when any library crate is changed, we throw away all
the caching, which isn't ideal. This filters the source files out of
those builds, so that we get more cache hits. We also move all Cargo
features into the workspace, which I've been meaning to do for ages, so
things are more consistent, and again, we get more cache hits generally.
V3_GIT_ORIGIN_REV_ID: c724b152692575edf6c00ab426e48ecca13aa998
### What
This PR updates the engine to use the NDC Spec v0.2.0-rc.1 version. This
is very likely to be the final RC before release.
### How
The `ndc_models` crate got updated, which then resulted in the schema
migration code in `metadata_resolve` being updated. This affected a lot
of test results because connectors that used deprecated type
representations got migrated to other representations, and if a type
representation was missing then JSON was used instead.
The NDC request-sending code in `execute` was updated to send the
`X-Hasura-NDC-Version` header depending on the version of request
getting sent.
The custom connector was updated to be compatible with the new NDC
0.2.0-rc.1 types. This resulted in the schema changing, so a lot of
tests that contained the connector's schema were updated.
---------
Co-authored-by: Daniel Harvey <danieljamesharvey@gmail.com>
V3_GIT_ORIGIN_REV_ID: b1c7081eb1ee6cffdead08328a857903102332c6
<!-- The PR description should answer 2 important questions: -->
### What
First iteration of JSONAPI filtering
What does it do?
- Field comparisons against values
- And
- Or
What doesn't it do
- Generate schema
- Use `BooleanExpressionType` to work out which operators are available
- Work for nested fields (which don't work anyway atm)
V3_GIT_ORIGIN_REV_ID: 71f490b2767275b5e57fa4bcf213f2339391c093
<!-- The PR description should answer 2 important questions: -->
### What
Accidentally made the fallthrough to show all fields too liberal, so our
"include this field?" function became const true. Now we check properly.
V3_GIT_ORIGIN_REV_ID: a56ce76e1cdf57ea17856b9835128f75b3cf18c3
<!-- The PR description should answer 2 important questions: -->
### What
Lets be a bit more granular about our spans for JSONAPI endpoint.
<img width="1508" alt="Screenshot 2024-10-29 at 16 46 32"
src="https://github.com/user-attachments/assets/97f67ae3-ac27-4dcd-84a4-083dc6ef5e67">
V3_GIT_ORIGIN_REV_ID: d5c0342c0c9dae2f50e78c7f529a54eeedb541a9
<!-- The PR description should answer 2 important questions: -->
### What
A few tweaks to make OpenAPI validators happy.
V3_GIT_ORIGIN_REV_ID: a1ff6c7180f1ee02d0e6f53c83aeeceea964e5d8
<!-- The PR description should answer 2 important questions: -->
### What
Release `OrderByExpression` and version 2 of `Model`. These have been
working behind feature flags for a while.
### How
Remove feature flags, remove `hidden` pragmas on `open-dds` metadata.
V3_GIT_ORIGIN_REV_ID: dfb7e9316171c11d2605dfbf01552db093d81d63
<!-- The PR description should answer 2 important questions: -->
### What
Adds response types for JSONAPI endpoints. They look OK!
<img width="733" alt="Screenshot 2024-10-23 at 17 48 05"
src="https://github.com/user-attachments/assets/cbd52523-3511-404a-9716-eed142847158">
V3_GIT_ORIGIN_REV_ID: 72050f3d0accf57c3c77e83aed4bd684da46786b
<!-- The PR description should answer 2 important questions: -->
### What
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 894ba81666bb441e71b8e204b89e5732ac4f1c83
<!-- The PR description should answer 2 important questions: -->
### What
<img width="731" alt="Screenshot 2024-10-17 at 16 39 07"
src="https://github.com/user-attachments/assets/a9e504e8-68f1-4c89-930d-2a5b00549dfc">
<img width="723" alt="Screenshot 2024-10-17 at 16 39 14"
src="https://github.com/user-attachments/assets/9bd76265-1726-4825-a25f-c4faac42990f">
This PR exposes a super basic OpenAPI 3.1 spec for the JSONAPI at
`/v1/rest/__schema`.
It includes:
- `page[limit]` and `page[offset]`
- `fields[ModelName]`
It does not include:
- `sorting`
- return types
- It also changes the JSONAPI to be served from `/v1/rest/` instead of
`/v1/jsonapi`.
The schema is generated depending on the user's role, so they'll only
see fields they'll allowed to.
### How
- Generate a `State` type for each `Role`
- Use `oas3` crate for types, manually create `Spec` type.
V3_GIT_ORIGIN_REV_ID: d15bb1dbc690795dcd3e2d8d245fd59e97653a56
I am sorry @rakeshkky , we will merge this again once we've done a
release, I promise.
Reverts hasura/v3-engine#1254
V3_GIT_ORIGIN_REV_ID: 74236d25d4e84658717531a55d87c8d3371b553c
<!-- The PR description should answer 2 important questions: -->
### What
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 3458ea1e5ebe2d7387a7e503f957e5d55f225599
Reverts hasura/v3-engine#1225
According to @danieljharvey, websockets don't currently work in
multitenant and we want to do a release using a commit after this one.
V3_GIT_ORIGIN_REV_ID: 2d9239ab3203d5acbedef1cd86644861a99c99b2
<!-- The PR description should answer 2 important questions: -->
### What
Wasn't sure what was implemented, added some tests, it's working.
This PR is stacked on top of
https://github.com/hasura/v3-engine/pull/1233
V3_GIT_ORIGIN_REV_ID: fbfce6dd8ce325c5a1659db2755a09c7728030f2
<!-- The PR description should answer 2 important questions: -->
### What
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
- Remove `enable-subscriptions` from unstable features.
- Expose the subscriptions related opendd metadata in the json schema
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
- Update unstable feature related types by dropping subscriptions.
- Remove `hidden=true` opendd attribute and other related on
subscription opendd metadata.
- Update json schema files
V3_GIT_ORIGIN_REV_ID: 0aa763f516d394aab2e375da0817d0e60228c9b2
<!-- The PR description should answer 2 important questions: -->
### What
Work so far has been mostly in one big file, let's split it up a bit so
we can see what is going on better.
### How
Move code around. No functional changes.
V3_GIT_ORIGIN_REV_ID: 7dbb2ea8d66de845a0e725edf61eede65cd2d994
<!-- The PR description should answer 2 important questions: -->
Fixes
- offset / limit didn't work - fixed by updating the `hasura/jsonapi`
library to use `offset` / `limit` instead of `number` / `page` - this
was a change in JSONAPI 1.1
- Render type names as `subgraph_typename`
- Send `OrderBy` as `None` when it contains no columns - `Some(vec![])`
was upsetting Mongo, so let's be more stringent.
No functional changes as feature is behind flag.
V3_GIT_ORIGIN_REV_ID: 419993a3acd955ef74872078d11769f564c7fc36
<!-- The PR description should answer 2 important questions: -->
### What
Previously we were doing the business of calculating which arguments
(and parts of arguments) were preset in the GraphQL `schema` crate. This
meant we would have to reimplement the logic for each backend. Now we
move it into `metadata-resolve` so the results can be shared by all
frontends.
### How
Move argument preset resolve into `metadata-resolve`. What's left in
`graphql-schema` is all the stuff relating to `Annotation`s of various
kinds.
A lot of the diff is changing `ModelWithPermissions` and
`CommandWithPermissions` to `ModelWithArgumentPresets` and
`CommandWithArgumentPresets` in crates after `metadata-resolve`.
Functional no-op.
V3_GIT_ORIGIN_REV_ID: b1b0983abb9f6282652c8689b02e0796026752f5
<!-- The PR description should answer 2 important questions: -->
### What
This PR adds two new operand types for `BooleanExpressionType` in
OpenDD: `objectAggregate` and `scalarAggregate`. These two operand types
are then validated in metadata resolve to ensure their correctness.
Their design is based on [this
RFC](https://github.com/hasura/v3-engine/pull/945#issuecomment-2325652794).
They are hidden from the OpenDD JSON Schema and their usage is blocked
by default via a new unstable feature flag.
They are also currently not "usable", as the metadata resolve code that
would use an `objectAggregate` operand-ed `BooleanExpressionType` to
satisfy an aggregate predicate over an array relationship does not
exist. This will be added in a future PR; this PR is big enough!
### How
A new unstable feature `enable_aggregation_predicates` has been added to
gate this feature behind while it is in development.
The new OpenDD types have been added to
`crates/open-dds/src/boolean_expression.rs`.
A new metadata resolve stage `aggregate_boolean_expressions` has been
added that specifically resolves these two new operand types. Most of
the new code changes in this PR exist in this new stage.
The output of this stage is composed into the `boolean_expressions`
stage afterwards, but other than just being consumed and included there,
it is unused for now. In a future PR it will be used when resolving
object-operanded `BooleanExpressionTypes`.
An exhaustive set of metadata resolve tests has been added to test all
the error, warning and passing cases for both object and scalar
aggregate operands
(`crates/metadata-resolve/tests/failing/aggregate_boolean_expressions/`
and
`crates/metadata-resolve/tests/passing/aggregate_boolean_expressions/`).
These files make up the vast majority of the file count in this PR.
V3_GIT_ORIGIN_REV_ID: e0e030b322d05212a0e2630acc9f4db7b05000c1
### What
Currently if you want v3-engine to accept metadata that is only a
partial supergraph (ie missing some subgraphs), you must set the
PARTIAL_SUPERGRAPH env var. Builds sent to MBS get sent to a special
endpoints `/validate/partial` and `/build/partial` that runs the engine
build process with that configuration option set.
This results in a terrible user experience for local builds, because MBS
will accept a partial supergraph and yield build artifacts, but
v3-engine will refuse to run them unless you set that env var.
This PR removes that env var and creates a new OpenDD flag called
`allow_partial_supergraph`. When MBS's `/validate/partial` endpoint is
used, that flag is set in the build artifacts. v3-engine then looks at
that flag to enable partial supergraph mode. This means a
`/build/partial` build via MBS just works when you run it locally via
v3-engine.
### How
`metadata_resolve`'s `Configuration.allow_unknown_subgraphs` has been
removed, and `metadata_resolve` now looks at OpenDD's new
`Flags.allow_partial_supergraph` instead.
In MBS, usage of `ValidationMode` (the enum that enables partial builds)
previously used to set `Configuration.allow_unknown_subgraphs`; now it
is used in `compute_open_dds_flags` in order to set
`Flags.allow_partial_supergraph`.
The existing metadata_resolve test that tested partial supergraphs has
been modified to use the flags rather than the removed configuration
option
(`crates/metadata-resolve/tests/passing/missing_subgraph_when_ignoring_unknown_subgraphs`).
A new `metadata_resolve` test has been added that checks that comparable
relationships in `BooleanExpressionType` properly respects the
`allow_partial_subgraphs` flag (this functionality was added in #1182).
V3_GIT_ORIGIN_REV_ID: 2c984eb791263a1fb0606c6c44a2a1ae4a5e7370
<!-- The PR description should answer 2 important questions: -->
### What
Move the function/procedure planning from `sql` to the shared OpenDD IR
pipeline in `plan`. This should be a no-op for `sql`
### How
Move code, fix type errors.
V3_GIT_ORIGIN_REV_ID: 7da797ffedbc40a44692670679aa176817f2c65e
<!-- The PR description should answer 2 important questions: -->
### What
We need to bump `tonic` because of `cargo audit`, and to do so we need
to bump OpenTelemetry, and to do that we need to finally bump `http`,
and to do that...
### How
Follow the breadcrumb trail, fix the errors.
Checked, trace propagation still works:
<img width="1781" alt="Screenshot 2024-10-02 at 15 56 28"
src="https://github.com/user-attachments/assets/4c3f8763-e646-445d-84de-f507da15ad2e">
V3_GIT_ORIGIN_REV_ID: aa219e91375aa310dc656183948edc33f77cdfdc
<!-- The PR description should answer 2 important questions: -->
### What
A JSONAPI request allows users to provide the fields they require like
so:
```
fields[Authors]=author_id,first_name
```
This PR implements the most basic form of this, along with some notes on
what the better version could be. This is motivated by allowing us to
test the OpenDD pipeline more than anything else.
### How
Loop through sparse field sets looking for the field we want to include,
reject it if not.
JSONAPI is behind a feature flag so is a functional no-op.
V3_GIT_ORIGIN_REV_ID: 27dbb6103c42910be71529351e7a2c9ce0abcb4a
<!-- The PR description should answer 2 important questions: -->
### What
We're building a new OpenDD IR pipeline. The `sql` crate already has a
lot of what we need, so let's take the model selection parts (ie, not
aggregates yet), pull them into the `plan` crate, and re-use them for
both `sql` and the `jsonapi` pipelines.
The broad idea here is that the shared `plan` will get incrementally
bigger, and `sql` will get smaller.
This is a functional no-op for `sql`, and slightly improves the WIP
JSONAPI pipeline as we enjoy better permission checks.
### How
- Copy model planning and helper functions from `sql` into `plan`
- Replace instances `DataFusionError` with a smaller local `PlanError`
- Fix JSONAPI to use these new `plan` functions
- Remove the code in `sql`, instead using the shared `plan` functions in
planning, mapping back into `DataFusionError` as appropriate.
V3_GIT_ORIGIN_REV_ID: 50314442b9b56f31d2b38a0cf6f104e265bc3886
<!-- The PR description should answer 2 important questions: -->
### What
<!-- What is this PR trying to accomplish (and why, if it's not
obvious)? -->
<!-- Consider: do we need to add a changelog entry? -->
<!-- Does this PR introduce new validation that might break old builds?
-->
<!-- Consider: do we need to put new checks behind a flag? -->
- Introduce a brand new `graphql-ws` crate that implements
- WebSocket request handling by `WebSocketServer` struct
-
[graphql-ws](https://github.com/enisdenjo/graphql-ws/blob/master/PROTOCOL.md)
protocol and handles subscriptions in async tokio tasks.
- OSS engine now handles GraphQL websockets through `GET /graphql`
handshakes.
### How
Refer to added
[architecture.md](https://github.com/hasura/v3-engine/blob/rakeshkky/graphql-ws-crate/crates/graphql-ws/architecture.md)
file.
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 3032cbe50267d3f0102c450be2749c09fb3992bf
<!-- The PR description should answer 2 important questions: -->
### What
We are creating a new build pipeline. This makes a new crate for it
called `plan` and puts the existing work from JSONAPI into it. JSONAPI
uses the new plan so we have a quick way of testing it works.
### How
Mostly moving code around. All behind feature flags, so functional
no-op.
V3_GIT_ORIGIN_REV_ID: af4901c93415c7ced2b0f537e600512d28fc5766
### What
We want to be able to reference the path within parsers even if the
parser succeeds. This allows us to do things like generate source spans
to help users make updates.
### How
We add `path` as a currently ignored argument to `deserialize`, and then
every time we want to decorate the error path, we replace that call with
a bidirectional decorator.
V3_GIT_ORIGIN_REV_ID: 18ddede2db84801f995b64f17bfb29893acb4657
<!-- The PR description should answer 2 important questions: -->
### What
Much like https://github.com/hasura/v3-engine/pull/1116, make clearer
what is and is not graphql-centric in engine by renaming `schema` to
`graphql-schema`.
### How
Moving file around, no functional changes.
V3_GIT_ORIGIN_REV_ID: ec06c33a964c16a53c1a4ed306de3fdccd2e8efc
<!-- The PR description should answer 2 important questions: -->
### What
Let's get to a very basic end-to-end pipeline. This introduces
(extremely-happy-path) planning, execution and response processing so
that we have a rough end to end API.
V3_GIT_ORIGIN_REV_ID: 57ce28eb5ad20f061e0ede37b0bf43fbaddaaf2a
<!-- The PR description should answer 2 important questions: -->
### What
Parse sorting and pagination params into OpenDD query IR.
Parsing filter expression will come in a follow-up PR.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 375d9c267effbc67e125365329f472e2f6ecc6b9
<!-- The PR description should answer 2 important questions: -->
### What
Initial, hacky attempt to parse URI parts into the OpenDD query request.
This is not very useful yet. As we don't have any way to execute a
OpenDD query yet.
### How
<!-- How is it trying to accomplish it (what are the implementation
steps)? -->
V3_GIT_ORIGIN_REV_ID: 546c8954d08ddf403993307bd77518f3863dbd33
<!-- The PR description should answer 2 important questions: -->
### What
Let's add a test!
(and check it still works 👍 )
<img width="405" alt="Screenshot 2024-09-10 at 13 05 01"
src="https://github.com/user-attachments/assets/6a49feaa-bd0b-4137-a9d5-a1e1336d9fa6">
### How
Glob a folder for test files, parse them and run them against the
functions.
V3_GIT_ORIGIN_REV_ID: cc3e0d8cfb6f5eaa58cf72dab97e7220a57e7673
<!-- The PR description should answer 2 important questions: -->
### What
Somewhere to put the `jsonapi` proof of concept.
### How
Empty crate
V3_GIT_ORIGIN_REV_ID: 5675b6fff57e54faa00f05630ae69f8d3471eb7f
