hasura/graphql-engine
1
0
Fork 0
mirror of https://github.com/hasura/graphql-engine.git synced 2024-12-15 09:22:43 +03:00
Code Issues Projects Releases Wiki Activity
Blazing fast, instant realtime GraphQL APIs on your DB with fine grained access control, also trigger webhooks on database events.
5,379 Commits 138 Branches 441 Tags 4.4 GiB
TypeScript 39%
Haskell 30.7%
Rust 10.7%
JavaScript 7.1%
Go 4.3%
Other 8%
a01d1188f2
Go to file
Vamshi Surabhi a01d1188f2 scaffolding for remote-schemas module 
The main aim of the PR is:

1. To set up a module structure for 'remote-schemas' package.
2. Move parts by the remote schema codebase into the new module structure to validate it.

## Notes to the reviewer

Why a PR with large-ish diff?

1. We've been making progress on the MM project but we don't yet know long it is going to take us to get to the first milestone. To understand this better, we need to figure out the unknowns as soon as possible. Hence I've taken a stab at the first two items in the [end-state](https://gist.github.com/0x777/ca2bdc4284d21c3eec153b51dea255c9) document to figure out the unknowns. Unsurprisingly, there are a bunch of issues that we haven't discussed earlier. These are documented in the 'open questions' section.

1. The diff is large but that is only code moved around and I've added a section that documents how things are moved. In addition, there are fair number of PR comments to help with the review process.

## Changes in the PR

### Module structure

Sets up the module structure as follows:

```
Hasura/
  RemoteSchema/
    Metadata/
      Types.hs
    SchemaCache/
      Types.hs
      Permission.hs
      RemoteRelationship.hs
      Build.hs
    MetadataAPI/
      Types.hs
      Execute.hs
```

### 1. Types representing metadata are moved

Types that capture metadata information (currently scattered across several RQL modules) are moved into `Hasura.RemoteSchema.Metadata.Types`.

- This new module only depends on very 'core' modules such as
  `Hasura.Session` for the notion of roles and `Hasura.Incremental` for `Cacheable` typeclass.

- The requirement on database modules is avoided by generalizing the remote schemas metadata to accept an arbitrary 'r' for a remote relationship
  definition.

### 2. SchemaCache related types and build logic have been moved

Types that represent remote schemas information in SchemaCache are moved into `Hasura.RemoteSchema.SchemaCache.Types`.

Similar to `H.RS.Metadata.Types`, this module depends on 'core' modules except for `Hasura.GraphQL.Parser.Variable`. It has something to do with remote relationships but I haven't spent time looking into it. The validation of 'remote relationships to remote schema' is also something that needs to be looked at.

Rips out the logic that builds remote schema's SchemaCache information from the monolithic `buildSchemaCacheRule` and moves it into `Hasura.RemoteSchema.SchemaCache.Build`. Further, the `.SchemaCache.Permission` and `.SchemaCache.RemoteRelationship` have been created from existing modules that capture schema cache building logic for those two components.

This was a fair amount of work. On main, currently remote schema's SchemaCache information is built in two phases - in the first phase, 'permissions' and 'remote relationships' are ignored and in the second phase they are filled in.

While remote relationships can only be resolved after partially resolving sources and other remote schemas, the same isn't true for permissions. Further, most of the work that is done to resolve remote relationships can be moved to the first phase so that the second phase can be a very simple traversal.

This is the approach that was taken - resolve permissions and as much as remote relationships information in the first phase.

### 3. Metadata APIs related types and build logic have been moved

The types that represent remote schema related metadata APIs and the execution logic have been moved to `Hasura.RemoteSchema.MetadataAPI.Types` and `.Execute` modules respectively.

## Open questions:

1. `Hasura.RemoteSchema.Metadata.Types` is so called because I was hoping that all of the metadata related APIs of remote schema can be brought in at `Hasura.RemoteSchema.Metadata.API`. However, as metadata APIs depended on functions from `SchemaCache` module (see [1](ceba6d6226/server/src-lib/Hasura/RQL/DDL/RemoteSchema.hs (L55)) and [2](ceba6d6226/server/src-lib/Hasura/RQL/DDL/RemoteSchema.hs (L91)), it made more sense to create a separate top-level module for `MetadataAPI`s.

   Maybe we can just have `Hasura.RemoteSchema.Metadata` and get rid of the extra nesting or have `Hasura.RemoteSchema.Metadata.{Core,Permission,RemoteRelationship}` if we want to break them down further.

1. `buildRemoteSchemas` in `H.RS.SchemaCache.Build` has the following type:

   ```haskell
   buildRemoteSchemas ::
     ( ArrowChoice arr,
       Inc.ArrowDistribute arr,
       ArrowWriter (Seq CollectedInfo) arr,
       Inc.ArrowCache m arr,
       MonadIO m,
       HasHttpManagerM m,
       Inc.Cacheable remoteRelationshipDefinition,
       ToJSON remoteRelationshipDefinition,
       MonadError QErr m
     ) =>
     Env.Environment ->
     ( (Inc.Dependency (HashMap RemoteSchemaName Inc.InvalidationKey), OrderedRoles),
       [RemoteSchemaMetadataG remoteRelationshipDefinition]
     )
       `arr` HashMap RemoteSchemaName (PartiallyResolvedRemoteSchemaCtxG remoteRelationshipDefinition, MetadataObject)
   ```

   Note the dependence on `CollectedInfo` which is defined as

   ```haskell
   data CollectedInfo
     = CIInconsistency InconsistentMetadata
     | CIDependency
         MetadataObject
         -- ^ for error reporting on missing dependencies
         SchemaObjId
         SchemaDependency
     deriving (Eq)
   ```

   this pretty much means that remote schemas is dependent on types from databases, actions, ....

   How do we fix this? Maybe introduce a typeclass such as `ArrowCollectRemoteSchemaDependencies` which is defined in `Hasura.RemoteSchema` and then implemented in graphql-engine?

1. The dependency on `buildSchemaCacheFor` in `.MetadataAPI.Execute` which has the following signature:

   ```haskell
   buildSchemaCacheFor ::
     (QErrM m, CacheRWM m, MetadataM m) =>
     MetadataObjId ->
     MetadataModifier ->
   ```

   This can be easily resolved if we restrict what the metadata APIs are allowed to do. Currently, they operate in an unfettered access to modify SchemaCache (the `CacheRWM` constraint):

   ```haskell
   runAddRemoteSchema ::
     ( QErrM m,
       CacheRWM m,
       MonadIO m,
       HasHttpManagerM m,
       MetadataM m,
       Tracing.MonadTrace m
     ) =>
     Env.Environment ->
     AddRemoteSchemaQuery ->
     m EncJSON
   ```

   This should instead be changed to restrict remote schema APIs to only modify remote schema metadata (but has access to the remote schemas part of the schema cache), this dependency is completely removed.

   ```haskell
   runAddRemoteSchema ::
     ( QErrM m,
       MonadIO m,
       HasHttpManagerM m,
       MonadReader RemoteSchemasSchemaCache m,
       MonadState RemoteSchemaMetadata m,
       Tracing.MonadTrace m
     ) =>
     Env.Environment ->
     AddRemoteSchemaQuery ->
     m RemoteSchemeMetadataObjId
   ```

   The idea is that the core graphql-engine would call these functions and then call
   `buildSchemaCacheFor`.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6291
GitOrigin-RevId: 51357148c6404afe70219afa71bd1d59bdf4ffc6
2022-10-21 03:15:04 +00:00
.circleci server/tests-py: Start webhook.py inside the test harness. 2022-10-20 19:00:01 +00:00
.github Improve the tone of the OSS issue templates by removing questions 2022-10-19 02:21:46 +00:00
architecture streaming subscriptions architecture: update try it out instructions 2022-09-29 17:28:01 +00:00
assets updated readme gif 2022-02-24 09:38:54 +00:00
cabal Extract the test harness from the Hspec tests 2022-10-04 08:31:26 +00:00
cli cli: port commonmetadata package to use internal/errors package 2022-10-20 15:18:46 +00:00
cli-ext Upgrade all package-lock.json files to the v2 format. 2022-09-28 08:15:29 +00:00
community community: streaming chat app - add graphql query tab 2022-10-05 09:33:31 +00:00
console console: added cloned row values to payload if they exist 2022-10-20 14:36:50 +00:00
contrib/metadata-types contrib: Metadata V3 types + JSON Schema 2021-11-10 05:37:11 +00:00
dc-agents Add column scalar type information Data Connector query API request [GDC-493] 2022-10-20 03:24:53 +00:00
docker-compose tests: Docker Compose healthchecks 2022-10-17 13:20:31 +00:00
docs Update troubleshooting.mdx adding client-side error solution 2022-10-20 19:05:50 +00:00
frontend platform: rename codebases and re import code 2022-10-20 16:21:56 +00:00
install-manifests NDAT-164 CockroachDB 'Getting Started' Docs 2022-10-20 17:07:40 +00:00
nix packaging: Rebuild the base images to get PostgreSQL Client 15. 2022-10-18 13:27:24 +00:00
packaging packaging: Rebuild the base images to get PostgreSQL Client 15. 2022-10-18 13:27:24 +00:00
rfcs rfc: separate metadata API types from business logic types 2022-10-05 18:39:45 +00:00
scripts tests: Docker Compose healthchecks 2022-10-17 13:20:31 +00:00
server scaffolding for remote-schemas module 2022-10-21 03:15:04 +00:00
translations Malayalam(ml_IN) translation for SECURITY.md 2022-10-20 08:52:57 +00:00
.dockerignore Create a Docker image specifically for running the Python tests. 2022-09-07 14:29:42 +00:00
.envrc Push the Nix dependency management configuration to the OSS repository. 2022-10-11 15:33:22 +00:00
.envrc.local.example Nix: Upgrade to GHC 9.2.4. 2022-10-06 21:05:34 +00:00
.ghcversion tooling: bump .ghcversion to 9.2.4 2022-10-06 14:13:39 +00:00
.git-blame-ignore-revs Added a .git-blame-ignore-revs file 2021-10-27 18:17:16 +00:00
.gitignore server: address recent graphql-ws related bugs 2021-11-04 12:40:02 +00:00
.hlint.yaml Clean up Hasura.Prelude a bit 2022-10-03 21:50:53 +00:00
.kodiak.toml ci: sets include_pr_number kodiak config to false (#1569) 2021-06-14 15:21:56 +00:00
.nvmrc tooling: bump .nvmrc version to 16 2022-09-06 11:23:56 +00:00
.prettierignore Add a missing changelog entry for error message changes to v2.10.0. 2022-09-05 09:41:06 +00:00
cabal.project Update Kriti to 0.3.3 2022-10-20 00:59:23 +00:00
cabal.project.freeze Update Kriti to 0.3.3 2022-10-20 00:59:23 +00:00
CHANGELOG.md Kriti Documentation 2022-09-06 15:47:18 +00:00
code-of-conduct.md update code of conduct (#2886) 2019-09-16 14:07:52 +05:30
CONTRIBUTING.md community: update contrib guide (#5866) 2020-10-01 10:03:30 +02:00
docker-compose.yml Mutable Sqlite GDC Fixture - GDC-455 2022-10-17 22:53:14 +00:00
event-triggers.md docs: update the hasura sample apps links 2022-02-18 17:06:46 +00:00
flake.lock Push the Nix dependency management configuration to the OSS repository. 2022-10-11 15:33:22 +00:00
flake.nix Push the Nix dependency management configuration to the OSS repository. 2022-10-11 15:33:22 +00:00
LICENSE Change license for core GraphQL Engine to Apache 2.0 (#1821) 2019-03-19 16:23:36 +05:30
LICENSE-community add community boilerplates and examples (#430) 2018-09-13 12:00:07 +05:30
Makefile server: Use make to simplify tests-py venv generation. 2022-07-25 20:05:49 +00:00
package-lock.json console: Update Cypress to v10 and run the migration guide 2022-07-05 08:51:35 +00:00
README.md docs: update the hasura sample apps links 2022-02-18 17:06:46 +00:00
remote-schemas.md update docs link to avoid redirects 2021-03-01 18:51:18 +00:00
sample.hie.yaml Extract the test harness from the Hspec tests 2022-10-04 08:31:26 +00:00
SECURITY.md Malayalam(ml_IN) translation for SECURITY.md 2022-10-20 08:52:57 +00:00
shell.nix Push the Nix dependency management configuration to the OSS repository. 2022-10-11 15:33:22 +00:00
weeder.dhall remove homebrew NESeq, use vendored 2022-07-19 08:42:28 +00:00

README.md

Hasura GraphQL Engine

Latest release Docs

Hasura is an open source product that accelerates API development by 10x by giving you GraphQL or REST APIs with built in authorization on your data, instantly.

Read more at hasura.io and the docs.

Hasura GraphQL Engine Demo

Hasura GraphQL Engine Realtime Demo

Features

  • Make powerful queries: Built-in filtering, pagination, pattern search, bulk insert, update, delete mutations
  • Works with existing, live databases: Point it to an existing database to instantly get a ready-to-use GraphQL API
  • Realtime: Convert any GraphQL query to a live query by using subscriptions
  • Merge remote schemas: Access custom GraphQL schemas for business logic via a single GraphQL Engine endpoint. Read more.
  • Extend with Actions: Write REST APIs to extend Hasura’s schema with custom business logic.
  • Trigger webhooks or serverless functions: On Postgres insert/update/delete events (read more)
  • Scheduled Triggers: Execute custom business logic at specific points in time using a cron config or a one-off event.
  • Fine-grained access control: Dynamic access control that integrates with your auth system (eg: auth0, firebase-auth)
  • Admin UI & Migrations: Admin UI & Rails-inspired schema migrations
  • Supported Databases: Supports PostgreSQL (and its flavours), MS SQL Server and Big Query. Support for more databases coming soon.

Read more at hasura.io and the docs.

Table of contents

Table of Contents

Quickstart:

One-click deployment on Hasura Cloud

The fastest and easiest way to try Hasura out is via Hasura Cloud.

  1. Click on the following button to deploy GraphQL engine on Hasura Cloud including Postgres add-on or using an existing Postgres database:

    Deploy to Hasura Cloud

  2. Open the Hasura console

    Click on the button "Launch console" to open the Hasura console.

  3. Make your first GraphQL query

    Create a table and instantly run your first query. Follow this simple guide.

Other one-click deployment options

Check out the instructions for the following one-click deployment options:

Infra provider One-click link Additional information
Heroku Deploy to Heroku docs
DigitalOcean Deploy to DigitalOcean docs
Azure Deploy to Azure docs
Render Deploy to Render docs

Other deployment methods

For Docker-based deployment and advanced configuration options, see deployment guides or install manifests.

Architecture

The Hasura GraphQL Engine fronts a Postgres database instance and can accept GraphQL requests from your client apps. It can be configured to work with your existing auth system and can handle access control using field-level rules with dynamic variables from your auth system.

You can also merge remote GraphQL schemas and provide a unified GraphQL API.

Hasura GraphQL Engine architecture

Client-side tooling

Hasura works with any GraphQL client. See awesome-graphql for a list of clients. Our frontend tutorial series also have integrations with GraphQL clients for different frameworks.

Add business logic

GraphQL Engine provides easy-to-reason, scalable and performant methods for adding custom business logic to your backend:

Remote schemas

Add custom resolvers in a remote schema in addition to Hasura's database-based GraphQL schema. Ideal for use-cases like implementing a payment API, or querying data that is not in your database - read more.

Actions

Actions are a way to extend Hasura’s schema with custom business logic using custom queries and mutations. Actions can be added to Hasura to handle various use cases such as data validation, data enrichment from external sources and any other complex business logic - read more

Trigger webhooks on database events

Add asynchronous business logic that is triggered based on database events. Ideal for notifications, data-pipelines from Postgres or asynchronous processing - read more.

Derived data or data transformations

Transform data in Postgres or run business logic on it to derive another dataset that can be queried using GraphQL Engine - read more.

Demos

Check out all the example applications in the community/sample-apps directory.

Realtime applications

  • Group Chat application built with React, includes a typing indicator, online users & new message notifications.

  • Live location tracking app that shows a running vehicle changing current GPS coordinates moving on a map.

  • A realtime dashboard for data aggregations on continuously changing data.

Videos

Support & Troubleshooting

The documentation and community will help you troubleshoot most issues. If you have encountered a bug or need to get in touch with us, you can contact us using one of the following channels:

We are committed to fostering an open and welcoming environment in the community. Please see the Code of Conduct.

If you want to report a security issue, please read this.

Stay up to date

We release new features every month. Sign up for our newsletter by using the link below. We send newsletters only once a month. https://hasura.io/newsletter/

Contributing

Check out our contributing guide for more details.

Brand assets

Hasura brand assets (logos, the Hasura mascot, powered by badges etc.) can be found in the assets/brand folder. Feel free to use them in your application/website etc. We'd be thrilled if you add the "Powered by Hasura" badge to your applications built using Hasura. ❤️

<!-- For light backgrounds -->
<a href="https://hasura.io">
  <img width="150px" src="https://graphql-engine-cdn.hasura.io/img/powered_by_hasura_primary_darkbg.svg" />
</a>

<!-- For dark backgrounds -->
<a href="https://hasura.io">
  <img width="150px" src="https://graphql-engine-cdn.hasura.io/img/powered_by_hasura_primary_lightbg.svg" />
</a>

License

The core GraphQL Engine is available under the Apache License 2.0 (Apache-2.0).

All other contents (except those in server, cli and console directories) are available under the MIT License. This includes everything in the docs and community directories.

Translations

This readme is available in the following translations:

Translations for other files can be found here.