graphql-engine/v3/crates/sql
Vamshi Surabhi 91b4fc6ee6 Improves the type mapping layer to SQL (#988)
<!-- The PR description should answer 2 important questions: -->

### What

The motivation was to allow introspecting field descriptions of object
types. To accomplish this, a fair number of things had to change. Prior
to this, we determined a model's and command's schema at the sql layer
by looking at its type mappings. When introspecting these models and
commands, if there are any nested fields, they are output as
'STRUCT<field1 type1, field2 type2>' without any descriptions.

With this change, we first build a catalog of types at the sql layer
from scalars and object types defined at opendd layer. These types are
then used when adding models and commands as opposed to looking at their
type mappings. This also changes how the type catalog is presented in
introspection. One can now query 'struct type's and their fields along
with their descriptions. Models and commands merely refer to these
struct types.

### How

<!-- How is it trying to accomplish it (what are the implementation
steps)? -->

V3_GIT_ORIGIN_REV_ID: 550da03c84b33ca44851858ee2cb73a31674d3d0
2024-08-19 21:25:25 +00:00
..
src Improves the type mapping layer to SQL (#988) 2024-08-19 21:25:25 +00:00
Cargo.toml [PACHA-4] initial support for commands (#975) 2024-08-18 03:34:15 +00:00
readme.md Refactor SQL layer to use OpenDD query IR (#925) 2024-08-05 23:38:19 +00:00

SQL Interface

An experimental SQL interface over OpenDD models. This is mostly targeted at AI use cases for now - GenAI models are better at generating SQL queries than GraphQL queries.

This is implemented using the Apache DataFusion Query Engine by deriving the SQL metadata for datafusion from Open DDS metadata. As the implementation currently stands, once we get a LogicalPlan from datafusion we replace TableScans with NDC queries to the underlying connector. There is a rudimentary optimizer that pushes down projections to the opendd query so that we don't fetch all the columns of a collection.