Commit Graph

269 Commits

Author SHA1 Message Date
Daniel Chambers
b9fb7d8720 Support composite primary keys for Data Connector [GDW-127]
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4926
GitOrigin-RevId: 2b6e5052f56a765e0b9a19345fcc4688d7e4700f
2022-07-01 12:21:17 +00:00
Samir Talwar
40617719ef server: Remove the Show instance from QErr and anything that touches it.
We only use these `Show` instances in error messages (where we call
`show` explicitly anyway) and test cases (in which Hspec requires `Show
a` for any `a` in an assertion).

This removes the instance in favor of a custom `showQErr` function
(which serializes the error to JSON). It is then used in certain error
message production which previously called `show` on a `QErr`.

There are two places where we serialize a QErr and then construct a new
QErr from the resulting string. Instead, we modify the existing QErr to
add extra information.

An orphan `Show QErr` instance is retained for tests so that we can have
nice test failure messages.

This is preparation for future changes in which the error message within
`QErr` will not be exposed directly, and therefore will not have a
`Show` instance. That said, it feels like a sensible kind of cleanup
anyway.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4897
GitOrigin-RevId: 8f79f7a356f0aea571156f39aefac242bf751f3a
2022-07-01 11:48:26 +00:00
Tom Harding
c90b8c4776 Add forkIO and threadDelay errors to HLint.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4682
GitOrigin-RevId: 35897df6793b41dbf54521a868c2d2daf7e268ea
2022-06-30 09:56:06 +00:00
Abby Sassel
9d93baa96f server/postgres: parameterise array variables in queries/subscriptions
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4776
Co-authored-by: Tom Harding <6302310+i-am-tom@users.noreply.github.com>
Co-authored-by: Brandon Martin <40686+codedmart@users.noreply.github.com>
GitOrigin-RevId: ca962e3bfaa28ef00c5d04273d6604348d7da1c8
2022-06-29 13:37:11 +00:00
Samir Talwar
5f38743f29 server: Encapsulate catalog versions in their own type.
This came about as I tried to add an instance over catalog versions and
found they were just simple integers most of the time (and in one case,
a float).

I think this change also clarifies how catalog versions work.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4864
GitOrigin-RevId: a6b7db86de564b71a8c2b602bee6a456b8e20d63
2022-06-29 08:19:33 +00:00
Daniel Chambers
59ffce9ac1 Generate TypeScript types for the Data Connector reference agent from the OpenAPI schema
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4855
GitOrigin-RevId: 4cc09d6706e67c69fbbedef72ff816365b9f7b4e
2022-06-29 07:43:52 +00:00
Auke Booij
8ccf7724ce server: Metadata origin for definitions (type parameter version v2)
The code that builds the GraphQL schema, and `buildGQLContext` in particular, is partial: not every value of `(ServerConfigCtx, GraphQLQueryType, SourceCache, HashMap RemoteSchemaName (RemoteSchemaCtx, MetadataObject), ActionCache, AnnotatedCustomTypes)` results in a valid GraphQL schema. When it fails, we want to be able to return better error messages than we currently do.

The key thing that is missing is a way to trace back GraphQL type information to their origin from the Hasura metadata. Currently, we have a number of correctness checks of our GraphQL schema. But these correctness checks only have access to pure GraphQL type information, and hence can only report errors in terms of that. Possibly the worst is the "conflicting definitions" error, which, in practice, can only be debugged by Hasura engineers. This is terrible DX for customers.

This PR allows us to print better error messages, by adding a field to the `Definition` type that traces the GraphQL type to its origin in the metadata. So the idea is simple: just add `MetadataObjId`, or `Maybe` that, or some other sum type of that, to `Definition`.

However, we want to avoid having to import a `Hasura.RQL` module from `Hasura.GraphQL.Parser`. So we instead define this additional field of `Definition` through a new type parameter, which is threaded through in `Hasura.GraphQL.Parser`. We then define type synonyms in `Hasura.GraphQL.Schema.Parser` that fill in this type parameter, so that it is not visible for the majority of the codebase.

The idea of associating metadata information to `Definition`s really comes to fruition when combined with hasura/graphql-engine-mono#4517. Their combination would allow us to use the API of fatal errors (just like the current `MonadError QErr`) to report _inconsistencies_ in the metadata. Such inconsistencies are then _automatically_ ignored. So no ad-hoc decisions need to be made on how to cut out inconsistent metadata from the GraphQL schema. This will allow us to report much better errors, as well as improve the likelihood of a successful HGE startup.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4770
Co-authored-by: Samir Talwar <47582+SamirTalwar@users.noreply.github.com>
GitOrigin-RevId: 728402b0cae83ae8e83463a826ceeb609001acae
2022-06-28 15:53:44 +00:00
Daniel Harvey
e4da80421e server: Change withPath to take a single pathItem in MonadParse
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4815
GitOrigin-RevId: d0d889754944cc8270bdee2fd21b0cf5dcad9175
2022-06-28 10:08:09 +00:00
Jesse Hallett
2f3ae93ab0 server: initial set of DTO types for metadata
This implements an initial set of DTO types that represent serialized metadata. These new types come with codecs using autodocodec which are used to derive both JSON serialization, and OpenAPI documentation. This ensures that we can automatically generate API documentation that is guaranteed to match JSON produced by the server.

For the moment the new types are not used for anything except to generate an early version of an OpenAPI document. Because this is early work the DTO types for each metadata format version list top-level properties only with placeholders for the types of each top-level property. This early iteration demonstrates using a sum type in Haskell that maps to a tagged union in OpenAPI (using the `version` field value as a tag).

This work is experimental and incomplete! Please do not incorporate the generated OpenAPI documentation into essential workflows at this time.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4801
GitOrigin-RevId: d2f110a6237b73520cdba24667333ef14e8cdd3d
2022-06-27 16:33:31 +00:00
Brandon Simmons
b704192268 server: GHC 9.2 changes compatible with 8.10 (#3550)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4841
Co-authored-by: awjchen <13142944+awjchen@users.noreply.github.com>
GitOrigin-RevId: ce47b1290fefb07f3f800c6c62120437c02086e5
2022-06-25 22:09:05 +00:00
Daniel Chambers
7ec5e79bd1 AVRelationship Support for Data Connectors [GDW-123]
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4735
GitOrigin-RevId: f23965a6c7ea8a0e6b25dbf9d3faeccec0ef6ec3
2022-06-24 06:59:52 +00:00
Antoine Leblanc
664633fc4b Remove the source cache from the schema context
### Description

This PR removes the source cache from the schema context. :)

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4830
GitOrigin-RevId: af312a22b70d042c5faf8d67c0cee254537be836
2022-06-23 11:43:17 +00:00
Samir Talwar
8db9b77c77 server: Reorganize quasi-quoted names.
Pretty much all quasi-quoted names in the server code base have ended up in `Hasura.GraphQL.Parser.Constants`. I'm now finding this unpleasant for two reasons:

1. I would like to factor out the parser code into its own Cabal package, and I don't want to have to expose all these names.
2. Most of them really have nothing to do with the parsers.

In order to remedy this, I have:

1. moved the names used by parser code to `Hasura.GraphQL.Parser.DirectiveName`, as they're all related to directives;
2. moved `Hasura.GraphQL.Parser.Constants` to `Hasura.Name`, changing the qualified import name from `G` to `Name`;
3. moved names only used in tests to the appropriate test case;
4. removed unused items from `Hasura.Name`; and
5. grouped related names.

Most of the changes are simply changing `G` to `Name`, which I find much more meaningful.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4777
GitOrigin-RevId: a77aa0aee137b2b5e6faec94495d3a9fbfa1348b
2022-06-23 09:15:31 +00:00
Daniel Harvey
88ace749bc server: Fix a bunch of HLint suggestions
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4738
GitOrigin-RevId: d0c0b13ac02ca80e51ae3d582f2e6917f76ad202
2022-06-21 11:12:42 +00:00
Gil Mizrahi
3e81ab3a57 server/postgres: test identifier uniqueness rewrite
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4693
GitOrigin-RevId: bd882835b86a37df125f79d5597826a15498fcaf
2022-06-17 10:44:18 +00:00
David Overton
5c7489cc23 Flesh out capabilities types
>

## Description ✍️
->

Fleshes out the Capabilities types used by Data Connector agents.

### Related Issues ✍
->

https://hasurahq.atlassian.net/browse/GDW-85

### Solution and Design ✍
>

Capabilities are divided into sections for queries, mutation, subscriptions, filtering and relationships:
```haskell
data Capabilities = Capabilities
  { cQueries :: Maybe QueryCapabilities,
    cMutations :: Maybe MutationCapabilities,
    cSubscriptions :: Maybe SubscriptionCapabilities,
    cFiltering :: Maybe FilteringCapabilities,
    cRelationships :: Maybe RelationshipCapabilities
  }
```

Each section is optional. If the section is absent it means the the related capabilities are not supported by the agent.
The types for each section can contain additional details about exactly what is supported. E.g.
```haskell
data QueryCapabilities = QueryCapabilities
  { qcSupportsPrimaryKeys :: Bool
  }
```

These are currently mostly empty, but will be filled in later.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4671
GitOrigin-RevId: 3d48570266bfce5e177a543a0ed6f63a7b450f0b
2022-06-16 08:31:07 +00:00
Tom Harding
e22eb1afea Weeding (2/?)
## Description

Following on from #4572, this removes more dead code as identified by Weeder. Comments and thoughts similarly welcome!

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4587
GitOrigin-RevId: 73aa6a5a2833ee41d29b71fcd0a72ed19822ca73
2022-06-09 16:40:49 +00:00
Brandon Simmons
6e8da71ece server: migrate to aeson-2 in preparation for ghc 9.2 upgrade
(Work here originally done by awjchen, rebased and fixed up for merge by
jberryman)

This is part of a merge train towards GHC 9.2 compatibility. The main
issue is the use of the new abstract `KeyMap` in 2.0. See:
https://hackage.haskell.org/package/aeson-2.0.3.0/changelog

Alex's original work is here:
#4305

BEHAVIOR CHANGE NOTE: This change causes a different arbitrary ordering
of serialized Json, for example during metadata export. CLI users care
about this in particular, and so we need to call it out as a _behavior
change_ as we did in v2.5.0. The good news though is that after this
change ordering should be more stable (alphabetical key order).

See: https://hasurahq.slack.com/archives/C01M20G1YRW/p1654012632634389

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4611
Co-authored-by: awjchen <13142944+awjchen@users.noreply.github.com>
GitOrigin-RevId: 700265162c782739b2bb88300ee3cda3819b2e87
2022-06-08 15:32:27 +00:00
Antoine Leblanc
eaba2e08d3 Clean Relay's code, break schema cycles, introduce Node ID V2
## Motivation

This PR rewrites most of Relay to achieve the following:
- ~~fix a bug in which the same node id could refer to two different tables in the schema~~
- remove one of the few remaining uses of the source cache in the schema building code

In doing so, it also:
- simplifies the `BackendSchema` class by removing `node` from it,
- makes it much easier for other backends to support Relay,
- documents, re-organizes, and clarifies the code.

## Description

This PR introduces a new `NodeId` version ~~, and adapts the Postgres code to always generate this V2 version~~. This new id contains the source name, in addition to the table name, in order to disambiguate similar table names across different sources (which is now possible with source customization). In doing so, it now explicitly handles that case for V1 node ids, and returns an explicit error message instead of running the risk of _silently returning the wrong information_.

Furthermore, it adapts `nodeField` to support multiple backends; most of the code was trivial to generalize, and as a result it lowers the cost of entry for other backends, that now only need to support `AFNodeId` in their translation layer.

Finally, it removes one more cycle in the schema building code, by using the same trick we used for remote relationships instead of using the memoization trick of #4576.

## Remaining work

- ~~[ ]write a Changelog entry~~
- ~~[x] adapt all tests that were asserting on an old node id~~

## Future work

This PR was adapted from its original form to avoid a breaking change: while it introduces a Node ID V2, we keep generating V1 IDs and the parser rejects V2 IDs. It will be easy to make the switch at a later data in a subsequent PR.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4593
GitOrigin-RevId: 88e5cb91e8b0646900547fa8c7c0e1463de267a1
2022-06-07 13:36:29 +00:00
Solomon
01f56735ac Feature/gdw 113
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4620
GitOrigin-RevId: f732fff4f3f2449ebc97f76522ee6ad11bcfbc68
2022-06-07 01:35:38 +00:00
David Overton
6586053296 Gdw 85/capabilities endpoint
>

## Description ✍️

- Creates a new `/capabilities` endpoint for the GDC agent API
- Removes capabilities from the `/schema` endpoint
- Removes the `/config-schema` endpoint and includes the `ConfigSchemaResponse` within the `CapabilitiesResponse`

### Related Issues ✍
->

https://hasurahq.atlassian.net/browse/GDW-85

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4478
GitOrigin-RevId: 426662ee9e751343d94207d439a5025df65d2de7
2022-06-03 08:08:23 +00:00
Daniel Chambers
1dbdb0236c server: Data Connector IR refactor, support for column boolean comparison expressions [GDW-114]
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4579
GitOrigin-RevId: 84da015123a53124488b88ce66bde26113625754
2022-06-02 02:07:57 +00:00
Auke Booij
13fc1c62d1 Move Hasura.GraphQL.Parser.Column to .RQL.IR. and .GraphQL.Schema.
This is a first step towards clarifying the role of `UnpreparedValue` as part of the IR. It certainly does not belong in the parser framework.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4588
GitOrigin-RevId: d1582a0b266729b79e00d31057178a4099168e6d
2022-05-30 22:07:57 +00:00
Antoine Leblanc
498442b1d3 Remove circular dependency in schema building code
### Description

The main goal of this PR is, as stated, to remove the circular dependency in the schema building code. This cycle arises from the existence of remote relationships: when we build the schema for a source A, a remote relationship might force us to jump to the schema of a source B, or some remote schema. As a result, we end up having to do a dispatch from a "leaf" of the schema, similar to the one done at the root. In turn, this forces us to carry along in the schema a lot of information required for that dispatch, AND it forces us to import the instances in scope, creating an import loop.

As discussed in #4489, this PR implements the "dependency injection" solution: we pass to the schema a function to call to do the dispatch, and to get a generated field for a remote relationship. That way, this function can be chosen at the root level, and the leaves need not be aware of the overall context.

This PR grew a bit bigger than that, however; in an attempt to try and remove the `SourceCache` from the schema altogether, it changed a lot of functions across the schema building code, to thread along the `SourceInfo b` of the source being built. This avoids having to do cache lookups within a given source. A few cases remain, such as relay, that we might try to tackle in a subsequent PR.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4557
GitOrigin-RevId: 9388e48372877520a72a9fd1677005df9f7b2d72
2022-05-27 17:22:38 +00:00
Evie Ciobanu
a8c0137f21 server: add infrastructure to write runit tests for update parsers
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4533
GitOrigin-RevId: d094149d6cbdeebe152c58032715bad725480d9b
2022-05-26 14:06:24 +00:00
paritosh-08
fd30fb343b server: naming conventions for auto generated fields
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3982
Co-authored-by: Brandon Simmons <210815+jberryman@users.noreply.github.com>
GitOrigin-RevId: f90b2e8f394e7bd69780f003d2d980475f104f42
2022-05-26 11:55:29 +00:00
Rakesh Emmadi
acde210fdc server/bigquery: generate graphql schema for table computed fields
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4460
GitOrigin-RevId: 7b772cd9fba6b612ad05eb1aca1fa13e6ae8556d
2022-05-25 10:25:38 +00:00
Puru Gupta
5501f686df server: do not serialize env vars in logs or errors: PR I - Remote Schemas
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4463
Co-authored-by: Naveen Naidu <30195193+Naveenaidu@users.noreply.github.com>
Co-authored-by: Brandon Simmons <210815+jberryman@users.noreply.github.com>
GitOrigin-RevId: 71c6824130d71312f5bd5ae94fc268c0544c6ca3
2022-05-21 09:05:59 +00:00
Daniel Chambers
97ac9cbcdc Data Connector Agent Configuration Schema [GDW-103]
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4450
GitOrigin-RevId: 2dce6e901c9734407d084d57166039fd94394279
2022-05-05 05:19:46 +00:00
Daniel Chambers
4f835623b1 Rename Data Wrapper to Data Connector [GDW-89]
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4400
GitOrigin-RevId: 2d90542f95ef4dad70c8dfe1ca3b4c1f3bdaa527
2022-05-02 05:04:07 +00:00
Solomon
db1c50affa Feature/gdw execute array literals GDW-82
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4319
GitOrigin-RevId: 20db2a1eb6e8521c4e572aa642c802129792468f
2022-04-28 01:53:06 +00:00
Antoine Leblanc
3cbcbd9291 Remove RQL/Types.hs
## Description

This PR removes `RQL.Types`, which was now only re-exporting a bunch of unrelated modules.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4363
GitOrigin-RevId: 894f29a19bff70b3dad8abc5d9858434d5065417
2022-04-27 13:58:47 +00:00
Antoine Leblanc
ea32b8bf82 Move HasServerConfigCtx to Hasura.Server.Types.
### Description

Small PR that moves code out of `RQL.Types.hs`. Specifically, it moves `HasServerConfigCtx` to where `ServerConfigCtx` is defined. This removes code from `RQL.Types`, makes the dependency on `Server.Types` more explicit, and will make some further cleanups easier.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4336
GitOrigin-RevId: 95bb3467d741763892c4e68a38760497157ba1aa
2022-04-22 14:51:00 +00:00
Puru Gupta
94539b0dfb multitenant: update shutdown logic
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4154
GitOrigin-RevId: e331b1c531b8e03102353652579b731ff4c09958
2022-04-19 08:50:17 +00:00
Solomon
c945b2d391 Replaces litName splices with name quasiquotes
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4267
GitOrigin-RevId: 2d93c35a7e34dbada3b72aabcae5fc2858bbfc29
2022-04-18 19:44:04 +00:00
paritosh-08
4e0a0a5804 server: add custom table name in EnumReference
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4177
GitOrigin-RevId: 569b4e450aabe11296b7a43fe4aca1883f221790
2022-04-13 10:08:21 +00:00
Daniel Chambers
38c41b2ae6 GDW Agent Test Suite [GDW-79]
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4175
GitOrigin-RevId: d37d7d131597af6b9cca6bd773c8dbbce8719ca5
2022-04-10 04:48:30 +00:00
Solomon
e9436c5d97 Initial GDW Query Generation
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4039
GitOrigin-RevId: ac01773c390876cd41733c8ebf7199d2d445d921
2022-04-08 06:49:40 +00:00
Vamshi Surabhi
bafd5be2b2 enable remote relationship metadata APIs on SQLServer
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4187
GitOrigin-RevId: 3e2a7734198df7758fc75610f4bcbd8e65a1c9de
2022-04-07 15:48:49 +00:00
Karthikeyan Chinnakonda
362aca9db3 server: streaming subscriptions execution (Incremental PR - 2)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4016
GitOrigin-RevId: 778300dd5ea094bc76b8f96c046313132863f832
2022-04-07 14:43:01 +00:00
Solomon
f3aac06dd2 Rewrite ir generators
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4094
GitOrigin-RevId: f592adbd686aa7cc78c094d5d4b88f4b62500d18
2022-04-05 22:09:40 +00:00
Robert
dfb72ecbad server: fix unexpected behaviour of waitForShutdown
With the current implementation, only the first call to `waitForShutdown` on a given
`ShutdownLatch` will return, while others will block (typically indefinitely). That's not
how one would expect a shutdown latch to work.

This isn't currently a concrete issue because we only wait once on each `ShutdownLatch`.
But in the context of #4154 we'll probably end up wanting to wait for shutdown from
multiple threads.

This adds a number of tests to verify the current behaviour, and adds a test for multiple
`waitForShutdown` calls that fails prior to the functional change.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4162
GitOrigin-RevId: 9a108858d11390b847404f30bc7b93c06fc3f966
2022-04-05 21:07:30 +00:00
Daniel Chambers
a70e4979ee Moved GDW API types into their own internal lib
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4120
GitOrigin-RevId: e7688fdc5a5621c0b760c9169ebf61ce2aea4913
2022-04-01 01:21:29 +00:00
David Overton
44577dab1b Add ToSchema instances to GDW API types
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4006
Co-authored-by: Daniel Chambers <1214352+daniel-chambers@users.noreply.github.com>
GitOrigin-RevId: 756ca0ed60865d0eb675562e8959f0d1839f9abe
2022-03-31 04:46:08 +00:00
jkachmar
adb648b429 server: Rework internal webhook request transform components
## Description

Some of the documentation/organizational changes I was putting into the suggestions for #3624 were a bit too convoluted for GitHub's suggestion interface, so I'm putting them here instead.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3910
Co-authored-by: Solomon <24038+solomon-b@users.noreply.github.com>
GitOrigin-RevId: 06e0cb08bd18e7f8b21452df0697cfd80bc56fde
2022-03-23 20:24:44 +00:00
jkachmar
647231b685 Yeet some default-extensions
Manually enables:
* EmptyCase
* ExistentialQuantification
* QuantifiedConstraints
* QuasiQuotes
* TemplateHaskell
* TypeFamilyDependencies

...in the following components:
* 'graphql-engine' library
* 'graphql-engine' 'src-test'
* 'graphql-engine' 'tests/integration'
* 'graphql-engine' tests-hspec'

Additionally, performs some light refactoring and documentation.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3991
GitOrigin-RevId: 514477d3466b01f60eca8935d0fef60dd0756838
2022-03-16 00:40:17 +00:00
Daniel Chambers
69501b2657 server: Use max-age when refreshing JWKs if must-revalidate is present
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3979
GitOrigin-RevId: 5f5ebfb25ff9729e34397084d86b0fe968099b25
2022-03-15 07:36:43 +00:00
Antoine Leblanc
376c7d48f1 Add remote relationships from remote schemas in schema parser generators
### Description

This PR extends the `RemoteSchema` parsers to also include remote relationships. This include a significant refactoring of the top level schema building blocks, since remote schemas can no longer be built in isolation: they have to be built within the same run of `MonadSchema` as the sources. It is originally taken from the changes in #3069 and was slightly adapted.

I highly recommend turning OFF whitespace in the Github UI for `Schema.hs`, since I've adjusted the indentation of two large functions.

### Warning

Given the lack of a feature flag, this PR technically **enables the feature**. While the metadata API is not plugged in, a savvy user could use `replace_metadata` to set a metadata that contains remote joins from remote schemas, and they would be enabled. Is this acceptable?

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3811
GitOrigin-RevId: a5b00f865cdb8890b0fc02b139c2ebd48929f138
2022-03-14 16:22:50 +00:00
Antoine Leblanc
bdd455473c Reduce the size of SelectSpec test to reduce memory usage.
### Description

Despite making sure only a small range was used for each value, this test can nonetheless result in an explosion of memory usage: it has twice in a row resulted in my poor laptop filling its swap in a matter of seconds, forcing me to kill the test or hard-reboot my machine.

This small PR lowers the high bound of the range used for the generator, to a value that seems to consistently allows the test to finish in a more constrained environment.

Additionally, it also sorts the tests alphabetically: it makes it much easier when scrolling through the output to find a specific test.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3942
GitOrigin-RevId: c7a786511fe82701fab29cf164dc3fbbe77f4262
2022-03-10 03:06:55 +00:00
Antoine Leblanc
85b8753fde Cleanup post #3810
### Description

#3810 was merged with comments still open; this small PR does a few minute clean-ups to address some remaining nits.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3941
GitOrigin-RevId: 3d15eb399828123640a73247b848bc4ddff02c38
2022-03-10 02:13:49 +00:00
Daniel Chambers
a8424c48a1 Customize column GraphQL schema descriptions
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3888
GitOrigin-RevId: 027c319a66671a44fc6e5506bdfc9d2c10a8569f
2022-03-09 06:35:46 +00:00
Antoine Leblanc
6e1761f8f9 Enable remote joins from remote schemas in the execution engine.
### Description

This PR adds the ability to perform remote joins from remote schemas in the engine. To do so, we alter the definition of an `ExecutionStep` targeting a remote schema: the `ExecStepRemote` constructor now expects a `Maybe RemoteJoins`. This new argument is used when processing the execution step, in the transport layer (either `Transport.HTTP` or `Transport.WebSocket`).

For this `Maybe RemoteJoins` to be extracted from a parsed query, this PR also extends the `Execute.RemoteJoin.Collect` module, to implement "collection" from a selection set. Not only do those new functions extract the remote joins, but they also apply all necessary transformations to the selection sets (such as inserting the necessary "phantom" fields used as join keys).

Finally in `Execute.RemoteJoin.Join`, we make two changes. First, we now always look for nested remote joins, regardless of whether the join we just performed went to a source or a remote schema; and second we adapt our join tree logic according to the special cases that were added to deal with remote server edge cases.

Additionally, this PR refactors / cleans / documents `Execute.RemoteJoin.RemoteServer`. This is not required as part of this change and could be moved to a separate PR if needed (a similar cleanup of `Join` is done independently in #3894). It also introduces a draft of a new documentation page for this project, that will be refined in the release PR that ships the feature (either #3069 or a copy of it).

While this PR extends the engine, it doesn't plug such relationships in the schema, meaning that, as of this PR, the new code paths in `Join` are technically unreachable. Adding the corresponding schema code and, ultimately, enabling the metadata API will be done in subsequent PRs.

### Keeping track of concrete type names

The main change this PR makes to the existing `Join` code is to handle a new reserved field we sometimes use when targeting remote servers: the `__hasura_internal_typename` field. In short, a GraphQL selection set can sometimes "branch" based on the concrete "runtime type" of the object on which the selection happens:

```graphql
query {
  author(id: 53478) {
    ... on Writer {
      name
      articles {
        title
      }
    }
    ... on Artist {
      name
      articles {
        title
      }
    }
  }
}
```

If both of those `articles` are remote joins, we need to be able, when we get the answer, to differentiate between the two different cases. We do this by asking for `__typename`, to be able to decide if we're in the `Writer` or the `Artist` branch of the query.

To avoid further processing / customization of results, we only insert this `__hasura_internal_typename: __typename` field in the query in the case of unions of interfaces AND if we have the guarantee that we will processing the request as part of the remote joins "folding": that is, if there's any remote join in this branch in the tree. Otherwise, we don't insert the field, and we leave that part of the response untouched.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3810
GitOrigin-RevId: 89aaf16274d68e26ad3730b80c2d2fdc2896b96c
2022-03-09 03:18:22 +00:00
David Overton
2792f515d4 Traverse variables in action remote joins
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3864
GitOrigin-RevId: 0fb624260db46474056ee323638d9be7d074b3fc
2022-03-08 08:23:20 +00:00
Solomon
d67d4e2310 Webhook Transform Cleanup / Refactor
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3624
GitOrigin-RevId: 849e6dd70d6fe3d84056a485b20928ff813881d4
2022-03-08 00:43:08 +00:00
Antoine Leblanc
0e3beb028d Extract generic containers from the codebase
### Description

There were several places in the codebase where we would either implement a generic container, or express the need for one. This PR extracts / creates all relevant containers, and adapts the relevant parts of the code to make use of said new generic containers. More specifically, it introduces the following modules:
- `Data.Set.Extended`, for new functions on `Data.Set`
- `Data.HashMap.Strict.Multi`, for hash maps that accept multiple values
- `Data.HashMap.Strict.NonEmpty`, for hash maps that can never be constructed as empty
- `Data.Trie`, for a generic implementation of a prefix tree

This PR makes use of those new containers in the following parts of the code:
- `Hasura.GraphQL.Execute.RemoteJoin.Types`
- `Hasura.RQL.Types.Endpoint*`

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3828
GitOrigin-RevId: e6c1b971bcb3f5ab66bc91d0fa4d0e9df7a0c6c6
2022-03-01 16:04:22 +00:00
Daniel Chambers
0f9f2192a0 server: Customize root field GraphQL schema descriptions
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3719
GitOrigin-RevId: b0a9bb6a0f65aac72ca95b66219eec16b2f5a0dd
2022-02-28 07:50:12 +00:00
Antoine Leblanc
a1886b3729 Generalize remote schemas IR
### Description

This PR is one further step towards remote joins from remote schemas. It introduces a custom partial AST to represent queries to remote schemas in the IR: we now need to augment what used to be a straightforward GraphQL AST with additional information for remote join fields.

This PR does the minimal amount of work to adjust the rest of the code accordingly, using `Void` in all places that expect a type representing remote relationships.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3794
GitOrigin-RevId: 33fc317731aace71f82ad158a1951ea93350d6cc
2022-02-25 20:38:46 +00:00
Philip Lykke Carlsen
b9ad721ea6 Reduce boolean blindness by promoting data type StringifyNumbers
I discovered and removed instances of Boolean Blindness about whether json numbers should be stringified or not.

Although quite far-reaching, this is a completely mechanical change and should have no observable impact outside the server code.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3763
GitOrigin-RevId: c588891afd8a6923a135c736f6581a43a2eddbc7
2022-02-23 20:19:24 +00:00
Auke Booij
b535257251 Avoid Arrows by interpreting monads
TL;DR
---

We go from this:
```haskell
  (|
    withRecordInconsistency
      ( (|
          modifyErrA
            ( do
                (info, dependencies) <- liftEitherA -< buildRelInfo relDef
                recordDependencies -< (metadataObject, schemaObject, dependencies)
                returnA -< info
            )
        |) (addTableContext @b table . addRelationshipContext)
      )
    |) metadataObject
```
to this:
```haskell
  withRecordInconsistencyM metadataObject $ do
    modifyErr (addTableContext @b table . addRelationshipContext) $ do
      (info, dependencies) <- liftEither $ buildRelInfo relDef
      recordDependenciesM metadataObject schemaObject dependencies
      return info
```

Background
---
We use Haskell's `Arrows` language extension to gain some syntactic sugar when working with `Arrow`s. `Arrow`s are a programming abstraction comparable to `Monad`s.

Unfortunately the syntactic sugar provided by this language extension is not very sweet.

This PR shows how we can sometimes avoid using `Arrow`s altogether, without loss of functionality or correctness. It is a demo of a technique that can be used to cut down the amount of `Arrows`-based code in our codebase by about half.

Approach
---

Although _in general_ not every `Monad` is an `Arrow`, specific `Arrow` instantiations are exactly as powerful as their `Monad` equivalents. Otherwise they wouldn't be very equivalent, would they?

Just like `liftEither` interprets the `Either e` monad into an arbitrary monad implementing `MonadError e`, we add `interpA` which interprets certain concrete monads such as `Writer w` into specific arrows, e.g. ones satisfying `ArrowWriter w`. This means that the part of the code that only uses such interpretable effects can be written _monadically_, and then used in _arrow_ constructions down the line.

This approach cannot be used for arrow effects which do not have a monadic equivalent. In our codebase, the only instance of this is `ArrowCache m`, implemented by the `Rule m` arrow. So code written with `ArrowCache m` in the context cannot be rewritten monadically using this technique.

See also
---
- #1827
- #2210

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3543
Co-authored-by: jkachmar <8461423+jkachmar@users.noreply.github.com>
GitOrigin-RevId: eb79619c95f7a571bce99bc144ce42ee65d08505
2022-02-22 18:09:50 +00:00
jkachmar
df4ca23a39 server: Splits QuickCheck extension and orphan instance modules
## Description

Hopefully this is relatively self-explanatory: this change splits the helper functions we've used to extend QuickCheck from the orphan instances and generators that we have defined for unit tests. These have now been placed in `Test.QuickCheck.Extended` and `Hasura.QuickCheck.Instances`, respectively.

This change also adds some documentation to the functions defined in `Test.QuickCheck.Extended` in the spirit of similar functions defined by `Test.QuickCheck`, itself.

### Motivation

We should adhere to the existing convention of constructing "extension modules" for common libraries separately from the code that takes advantage of these.

Alone, this wouldn't be a reason to split up `Hasura.Generators`, but we should **also** follow a convention of defining **all** orphan instances in modules whose names clearly indicate that they exist solely for the purpose of exporting these orphan instances (e.g. `Hasura.QuickCheck.Instances`).

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3747
GitOrigin-RevId: fb856a790b4a39163f81481d4f900fafb1797ea6
2022-02-22 15:33:37 +00:00
Solomon
d1ba271c3d Feature/removable request transform body and modified request transform API
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3475
GitOrigin-RevId: bc847b18d491fe4957a190f5d0fe2ae6e6719791
2022-02-17 04:37:18 +00:00
Robert
1ff3723ed8 server: assorted minor clean-up around HTTP managers
- consistent qualified imports
- less convoluted initialization of pro logging HTTP manager
- pass pro HTTP manager directly instead of via Has
- remove some dead healthcheck code

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3639
GitOrigin-RevId: dfa7b9c62d1842a07a8514cdb77f1ed86064fb06
2022-02-16 07:09:47 +00:00
Daniel Chambers
2c7a4e3a16 Customization of computed field GraphQL schema descriptions
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3615
GitOrigin-RevId: f51590d4cfc0412be9baa371353f9b9f3b908f84
2022-02-15 23:17:27 +00:00
Lyndon Maydwell
ff6aac31b8 Adding multiple jwt secrets (incorporating provenance requirements)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3173
Co-authored-by: Solomon <24038+solomon-b@users.noreply.github.com>
Co-authored-by: Daniel Chambers <1214352+daniel-chambers@users.noreply.github.com>
GitOrigin-RevId: 395a5d5854896f866b612895d6f41e29376c2caa
2022-02-13 23:34:39 +00:00
Swann Moreau
8bd34b4a51 server, pro: add support for per-role allowlists
spec: https://github.com/hasura/graphql-engine-mono/pull/2278

Briefly:
- extend metadata so that allowlist entries get a new scope field
- update `add_collection_to_allowlist` to accept this new scope field,
  and adds `update_scope_of_collection_in_allowlist` to change the scope
- scope can be global or role-based; a collection is available for every
  role if it is global, and available to every listed role if it is role-based
- graphql-engine-oss is aware of role-based allowlist metadata; collections
  with non-global scope are treated as if they weren't in the allowlist

To run the tests:
- `cabal run graphql-engine-tests -- unit --match Allowlist`
- py-tests against pro:
  - launch `graphql-engine-pro` with `HASURA_GRAPHQL_ADMIN_SECRET` and `HASURA_GRAPHQL_ENABLE_ALLOWLIST`
  - `pytest test_allowlist_queries.py --hge-urls=... --pg-urls=... --hge-key=... --test-allowlist-queries --pro-tests`

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2477
Co-authored-by: Anon Ray <616387+ecthiender@users.noreply.github.com>
Co-authored-by: Robert <132113+robx@users.noreply.github.com>
GitOrigin-RevId: 01f8026fbe59d8701e2de30986511a452fce1a99
2022-02-08 16:54:49 +00:00
Rakesh Emmadi
793aede022 server/mssql: improve database exception handling and better API errors
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3487
GitOrigin-RevId: d3f696072e8290b45c2f81509ce31cb5c13a4aef
2022-02-07 14:12:55 +00:00
Daniel Chambers
4d9417fac4 server: Refresh JWKs maximum once per second
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3429
GitOrigin-RevId: 123fe33f026a36282ee1137eeefd612191ff4844
2022-01-28 00:18:56 +00:00
Solomon
4b792abdcc Feature/webhook response transforms
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3074
Co-authored-by: David Overton <7734777+dmoverton@users.noreply.github.com>
Co-authored-by: Lyndon Maydwell <92299+sordina@users.noreply.github.com>
Co-authored-by: Tirumarai Selvan <8663570+tirumaraiselvan@users.noreply.github.com>
GitOrigin-RevId: 006c5c9b71cdca1c2f47962230e6189e09557fab
2022-01-19 04:47:36 +00:00
Chris Parks
1913f6c763 Avoid overlapping legacy/unified formats when parsing RemoteRelationshipDefinition (closes #3171)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3184
GitOrigin-RevId: 5827f065c918e4d6f7a89247b68d18bac264440f
2021-12-20 19:32:13 +00:00
Auke Booij
e805fc01f3 Avoid NonNullableType
This refactors the AST of GraphQL types so that we don't need `NonNullableType`.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3178
GitOrigin-RevId: 6513c8ea0a3cf4ad3ca7d8ef9ca996912fd5eedc
2021-12-20 17:03:28 +00:00
Auke Booij
2c3277e0a7 Refactor InputFieldInfo
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3143
GitOrigin-RevId: a374e07216f3f7d7dee61b222a2758d3ebeb98bb
2021-12-20 15:53:45 +00:00
Antoine Leblanc
bacadc30da Fix several issues with remote relationships.
## Remaining Work

- [x] changelog entry
- [x] more tests: `<backend>_delete_remote_relationship` is definitely untested
- [x] negative tests: we probably want to assert that there are some APIs we DON'T support
- [x] update the console to use the new API, if necessary
- [x] ~~adding the corresponding documentation for the API for other backends (only `pg_` was added here)~~
  - deferred to https://github.com/hasura/graphql-engine-mono/issues/3170
- [x] ~~deciding which backends should support this API~~
  - deferred to https://github.com/hasura/graphql-engine-mono/issues/3170
- [x] ~~deciding what to do about potentially overlapping schematic representations~~
  - ~~cf. https://github.com/hasura/graphql-engine-mono/pull/3157#issuecomment-995307624~~
  - deferred to https://github.com/hasura/graphql-engine-mono/issues/3171
- [x] ~~add more descriptive versioning information to some of the types that are changing in this PR~~
  -  cf. https://github.com/hasura/graphql-engine-mono/pull/3157#discussion_r769830920
  - deferred to https://github.com/hasura/graphql-engine-mono/issues/3172

## Description

This PR fixes several important issues wrt. the remote relationship API.

- it fixes a regression introduced by [#3124](https://github.com/hasura/graphql-engine-mono/pull/3124), which prevented `<backend>_create_remote_relationship` from accepting the old argument format (break of backwards compatibility, broke the console)
- it removes the command `create_remote_relationship` added to the v1/metadata API as a work-around as part of [#3124](https://github.com/hasura/graphql-engine-mono/pull/3124)
- it reverts the subsequent fix in the console: [#3149](https://github.com/hasura/graphql-engine-mono/pull/3149)

Furthermore, this PR also addresses two other issues:
- THE DOCUMENTATION OF THE METADATA API WAS WRONG, and documented `create_remote_relationship` instead of `<backend>_create_remote_relationship`: this PR fixes this by adding `pg_` everywhere, but does not attempt to add the corresponding documentation for other backends, partly because:
- `<backend>_delete_remote_relationship` WAS BROKEN ON NON-POSTGRES BACKENDS; it always expected an argument parameterized by Postgres.

As of main, the `<backend>_(create|update|delete)_remote_relationship` commands are supported on Postgres, Citus, BigQuery, but **NOT MSSQL**. I do not know if this is intentional or not, if it even should be publicized or not, and as a result this PR doesn't change this.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3157
Co-authored-by: jkachmar <8461423+jkachmar@users.noreply.github.com>
GitOrigin-RevId: 37e2f41522a9229a11c595574c3f4984317d652a
2021-12-16 20:29:19 +00:00
Anon Ray
4121c1dd3d Revert "Feature/multiple jwt secrets"
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3136
Co-authored-by: pranshi06 <85474619+pranshi06@users.noreply.github.com>
GitOrigin-RevId: aa41817e39f932f909067f2effca9d9973a5fb94
2021-12-14 14:29:52 +00:00
Vamshi Surabhi
0728a9e60e fixes remote relationships format in metadata (fixes graphql-engine-mono/issues/3108)
## Description

This PR fixes two issues:
  - in [#2903](https://github.com/hasura/graphql-engine-mono/pull/2903), we introduced a new metadata representation of remote relationships, which broke parsing a metadata blob containing an old-style db-to-rs remote relationship
  - in [#1179](https://github.com/hasura/graphql-engine-mono/pull/1179), we silently and mistakenly deprecated `create_remote_relationship` in favour of `<backend>_create_remote_relationship`

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3124
Co-authored-by: jkachmar <8461423+jkachmar@users.noreply.github.com>
Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
GitOrigin-RevId: 45481db7a8d42c7612e938707cd2d652c4c81bf8
2021-12-14 06:46:11 +00:00
Chris Parks
038a6113c3 Remove Identifier associated type family from Backend class (close #1758)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3096
GitOrigin-RevId: 03f218a76c5f333a77646a60c8c69767e2286ea6
2021-12-13 16:49:09 +00:00
pranshi06
dee86453ea server: fallback to unauthorised role when JWT is not found in cookie
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2882
GitOrigin-RevId: ad03fbd0572e00ffe7abea106388a4df4d12af2c
2021-12-08 18:29:29 +00:00
Solomon
f243760398 Feature/multiple jwt secrets
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2765
Co-authored-by: Philip Lykke Carlsen <358550+plcplc@users.noreply.github.com>
Co-authored-by: Chris Done <11019+chrisdone@users.noreply.github.com>
Co-authored-by: Aishwarya Rao <59638722+aishwaryarao712@users.noreply.github.com>
Co-authored-by: Tirumarai Selvan <8663570+tirumaraiselvan@users.noreply.github.com>
Co-authored-by: Karthikeyan Chinnakonda <15602904+codingkarthik@users.noreply.github.com>
Co-authored-by: Ikechukwu Eze <22247592+iykekings@users.noreply.github.com>
Co-authored-by: Brandon Simmons <210815+jberryman@users.noreply.github.com>
Co-authored-by: awjchen <13142944+awjchen@users.noreply.github.com>
Co-authored-by: Martin Mark <74692114+martin-hasura@users.noreply.github.com>
Co-authored-by: Abby Sassel <3883855+sassela@users.noreply.github.com>
Co-authored-by: Sooraj <8408875+soorajshankar@users.noreply.github.com>
Co-authored-by: Sameer Kolhar <6604943+kolharsam@users.noreply.github.com>
Co-authored-by: Vijay Prasanna <11921040+vijayprasanna13@users.noreply.github.com>
Co-authored-by: hasura-bot <30118761+hasura-bot@users.noreply.github.com>
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Sibi Prabakaran <737477+psibi@users.noreply.github.com>
Co-authored-by: Robert <132113+robx@users.noreply.github.com>
Co-authored-by: Vishnu Bharathi <4211715+scriptnull@users.noreply.github.com>
Co-authored-by: Praveen Durairaju <14110316+praveenweb@users.noreply.github.com>
Co-authored-by: Abhijeet Khangarot <26903230+abhi40308@users.noreply.github.com>
GitOrigin-RevId: b3b07c3b42da9f3daa450d6d9e5fbf0bf506c651
2021-12-02 05:36:46 +00:00
Auke Booij
caf9957aca Remove Unique from Definition
GraphQL types can refer to each other in a circular way. The PDV framework used to use values of type `Unique` to recognize two fragments of GraphQL schema as being the same instance. Internally, this is based on `Data.Unique` from the `base` package, which simply increases a counter on every creation of a `Unique` object.

**NB**: The `Unique` values are _not_ used for knot tying the schema combinators themselves (i.e. `Parser`s). The knot tying for `Parser`s is purely based on keys provided to `memoizeOn`. The `Unique` values are _only_ used to recognize two pieces of GraphQL _schema_ as being identical. Originally, the idea was that this would help us with a perfectly correct identification of GraphQL types. But this fully correct equality checking of GraphQL types was never implemented, and does not seem to be necessary to prevent bugs.

Specifically, these `Unique` values are stored as part of `data Definition a`, which specifies a part of our internal abstract syntax tree for the GraphQL types that we expose. The `Unique` values get initialized by the `SchemaT` effect.

In #2894 and #2895, we are experimenting with how (parts of) the GraphQL types can be hidden behind certain permission predicates. This would allow a single GraphQL schema in memory to serve all roles, implementing #2711. The permission predicates get evaluated at query parsing time when we know what role is doing a certain request, thus outputting the correct GraphQL types for that role.

If the approach of #2895 is followed, then the `Definition` objects, and thus the `Unique` values, would be hidden behind the permission predicates. Since the permission predicates are evaluated only after the schema is already supposed to be built, this means that the permission predicates would prevent us from initializing the `Unique` values, rendering them useless.

The simplest remedy to this is to remove our usage of `Unique` altogether from the GraphQL schema and schema combinators. It doesn't serve a functional purpose, doesn't prevent bugs, and requires extra bookkeeping.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2980
GitOrigin-RevId: 50d3f9e0b9fbf578ac49c8fc773ba64a94b1f43d
2021-12-01 16:21:35 +00:00
Evie Ciobanu
0a4194a1bc server: throw broken invariant on data loader error
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3010
GitOrigin-RevId: 3b09a7d61343d406d1ecc5d6aaab866564c9dff8
2021-12-01 12:50:38 +00:00
Philip Lykke Carlsen
4c1f3d0140 Adding ColumnMutability to column metadata
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2962
GitOrigin-RevId: ee2687e84842dc5dd647ce9aa017c6051556d0e5
2021-12-01 11:34:16 +00:00
Antoine Leblanc
f7071b3c93 Changed RemoteSchemaIntrospection's internal representation from a list to a hashmap.
### Description

This PR changes the internal representation of a parsed remote schema. We were still using a list of type definitions, meaning every time we were doing a type lookup we had to iterate through a linked list! 🙀 It was very noticeable on large schemas, that need to do a lot of lookups. This PR consequently changes the internal representation to a HashMap. Building the OneGraph schema on my machine now takes **23 seconds**, compared to **367 seconds** before this patch.

Some important points:
- ~~this PR removes a check for type duplication in remote schemas; it's unclear to me whether that's something we need to add back or not~~ (no longer true)
- this PR makes it obvious that we do not distinguish between "this remote schema is missing type X" and "this remote schema expects type X to be an object, but it's a scalar"; this PR doesn't change anything about it, but adds a comment where we could surface that error (see [2991](https://github.com/hasura/graphql-engine-mono/issues/2991))

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2963
GitOrigin-RevId: f5c96ad40f4e0afcf8cef635b4d64178111f98d3
2021-11-30 14:47:50 +00:00
David Overton
5bfce057c6 Refactor remote schema customization
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2771
GitOrigin-RevId: 0c90136f956df3f4552140e6ca3d2f4766f8b3f5
2021-11-30 00:38:27 +00:00
pranshi06
81bf0e5429 [pro, server] Multiple Admin Secrets
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2631
Co-authored-by: Solomon <24038+solomon-b@users.noreply.github.com>
GitOrigin-RevId: d89a38a9cd9ee7391ca5011b7a11c52e1668a31d
2021-11-17 07:33:33 +00:00
Evie Ciobanu
691b9233ce server: IR Selection basic generators
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2627
GitOrigin-RevId: a8a80bf90655db874d269efcf12e59eb9a46575c
2021-11-10 11:13:04 +00:00
Karthikeyan Chinnakonda
adb20d1d7c server: log DB locking queries during source catalog migration
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2676
GitOrigin-RevId: f01574a30d3a6bf95467ce69bb8b5e69ce4cc057
2021-11-09 14:22:44 +00:00
Puru Gupta
504f13725f server: forward auth webhook set-cookies header on response
>

High-Level TODO:

* [x] Code Changes
* [x] Tests
* [x] Check that pro/multitenant build ok
* [x] Documentation Changes
* [x] Updating this PR with full details
* [ ] Reviews
* [ ] Ensure code has all FIXMEs and TODOs addressed
* [x] Ensure no files are checked in mistakenly
* [x] Consider impact on console, cli, etc.

### Description
>
This PR adds support for adding set-cookie header on the response from the auth webhook. If the set-cookie header is sent by the webhook, it will be forwarded in the graphQL engine response.

Fixes a bug in test-server.sh: testing of get-webhook tests was done by POST method and vice versa. To fix, the parameters were swapped.

### Changelog

- [x] `CHANGELOG.md` is updated with user-facing content relevant to this PR.

### Affected components

- [x] Server
- [ ] Console
- [ ] CLI
- [x] Docs
- [ ] Community Content
- [ ] Build System
- [x] Tests
- [ ] Other (list it)

### Related Issues
->
Closes [#2269](https://github.com/hasura/graphql-engine/issues/2269)

### Solution and Design
>

### Steps to test and verify
>
Please refer to the docs to see how to send the set-cookie header from webhook.

### Limitations, known bugs & workarounds
>
- Support for only set-cookie header forwarding is added
- the value forwarded in the set-cookie header cannot be validated completely, the [Cookie](https://hackage.haskell.org/package/cookie) package has been used to parse the header value and any unnecessary information is stripped off before forwarding the header. The standard given in [RFC6265](https://datatracker.ietf.org/doc/html/rfc6265) has been followed for the Set-Cookie format.

### Server checklist

#### Catalog upgrade

Does this PR change Hasura Catalog version?
- [x] No
- [ ] Yes
  - [ ] Updated docs with SQL for downgrading the catalog

#### Metadata

Does this PR add a new Metadata feature?
- [x] No

#### GraphQL
- [x] No new GraphQL schema is generated
- [ ] New GraphQL schema is being generated:
   - [ ] New types and typenames are correlated

#### Breaking changes

- [x] No Breaking changes

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2538
Co-authored-by: Robert <132113+robx@users.noreply.github.com>
GitOrigin-RevId: d9047e997dd221b7ce4fef51911c3694037e7c3f
2021-11-09 12:01:31 +00:00
Brandon Simmons
b167120f96 server: add explicit export lists in OSS server and enforce with warning
We'll see if this improves compile times at all, but I think it's worth
doing as at least the most minimal form of module documentation.

This was accomplished by first compiling everything with
-ddump-minimal-imports, and then a bunch of scripting (with help from
ormolu)

**EDIT** it doesn't seem to improve CI compile times but the noise floor is high as it looks like we're not caching library dependencies anymore

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2730
GitOrigin-RevId: 667eb8de1e0f1af70420cbec90402922b8b84cb4
2021-11-04 16:09:38 +00:00
David Overton
aac64f2c81 Source typename customization (close graphql-engine#6974)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/1616
GitOrigin-RevId: f7eefd2367929209aa77895ea585e96a99a78d47
2021-10-29 14:43:14 +00:00
Solomon Bothwell
03aff6e82e Wrap string interpolations with double quotes on the backend
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2601
GitOrigin-RevId: 53c27dddd30968dd28325e3336721aa1a238815f
2021-10-21 13:32:49 +00:00
Robert
71af68e9e5 server: drop HasVersion implicit parameter (closes #2236)
The only real use was for the dubious multitenant option
--consoleAssetsVersion, which actually overrode not just
the assets version. I.e., as far as I can tell, if you pass
--consoleAssetsVersion to multitenant, that version will
also make it into e.g. HTTP client user agent headers as
the proper graphql-engine version.

I'm dropping that option, since it seems unused in production
and I don't want to go to the effort of fixing it, but am happy
to look into that if folks feels strongly that it should be
kept.

(Reason for attacking this is that I was looking into http
client things around blacklisting, and the versioning thing
is a bit painful around http client headers.)

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2458
GitOrigin-RevId: a02b05557124bdba9f65e96b3aa2746aeee03f4a
2021-10-13 16:39:58 +00:00
Solomon Bothwell
4e05bdcaec Feature/request transform string interpolation
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2443
Co-authored-by: Tirumarai Selvan <8663570+tirumaraiselvan@users.noreply.github.com>
GitOrigin-RevId: d7d68984d0ae1403bb414572e9704c01ed27deab
2021-09-29 08:14:29 +00:00
Robert
11a454c2d6 server, pro: actually reformat the code-base using ormolu
This commit applies ormolu to the whole Haskell code base by running `make format`.

For in-flight branches, simply merging changes from `main` will result in merge conflicts.
To avoid this, update your branch using the following instructions. Replace `<format-commit>`
by the hash of *this* commit.

$ git checkout my-feature-branch
$ git merge <format-commit>^    # and resolve conflicts normally
$ make format
$ git commit -a -m "reformat with ormolu"
$ git merge -s ours post-ormolu

https://github.com/hasura/graphql-engine-mono/pull/2404

GitOrigin-RevId: 75049f5c12f430c615eafb4c6b8e83e371e01c8e
2021-09-23 22:57:37 +00:00
Karthikeyan Chinnakonda
5f79b5f102 server: generalize the event triggers codepath for all backends
https://github.com/hasura/graphql-engine-mono/pull/2189

Co-authored-by: hasura-bot <30118761+hasura-bot@users.noreply.github.com>
Co-authored-by: Martin Mark <74692114+martin-hasura@users.noreply.github.com>
Co-authored-by: Vishnu Bharathi <4211715+scriptnull@users.noreply.github.com>
Co-authored-by: Sameer Kolhar <6604943+kolharsam@users.noreply.github.com>
Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
Co-authored-by: Matt Hardman <28978422+mattshardman@users.noreply.github.com>
Co-authored-by: Vijay Prasanna <11921040+vijayprasanna13@users.noreply.github.com>
Co-authored-by: Divi <32202683+imperfect-fourth@users.noreply.github.com>
GitOrigin-RevId: 97c71571656c6e0c57d06f2d38193833180901c0
2021-09-20 07:35:49 +00:00
Solomon Bothwell
af5ff07614 Request Transformations
https://github.com/hasura/graphql-engine-mono/pull/1984

Co-authored-by: jkachmar <8461423+jkachmar@users.noreply.github.com>
GitOrigin-RevId: 1767d6bdde48c156fe171b5a9b7e44d7f2eb4869
2021-09-16 11:03:57 +00:00
David Overton
7c77c81cf9 return original schema in introspect_remote_schema
https://github.com/hasura/graphql-engine-mono/pull/2364

GitOrigin-RevId: 99698aa977bf8ff85e29d67a0956ff1509cd30be
2021-09-16 09:07:18 +00:00
Robert
fe035125f4 server: drop LazyTxT newtype
This is a follow-up to #1959.

Today, I spent a while in review figuring out that a harmless PR change didn't do anything,
because it was moving from a `runLazy...` to something without the `Lazy`. So let's get
that source of confusion removed.

This should be a bit easier to review commit by commit, since some of the functions had
confusing names. (E.g. there was a misnamed `Migrate.Internal.runTx` before.)

The change should be a no-op.

https://github.com/hasura/graphql-engine-mono/pull/2335

GitOrigin-RevId: 0f284c4c0f814482d7827e7732a6d49e7735b302
2021-09-15 20:46:45 +00:00
Karthikeyan Chinnakonda
982b5a3d15 server: log operation details for each query in a batch query execution
https://github.com/hasura/graphql-engine-mono/pull/2306

GitOrigin-RevId: 066a02fc57711b1faad447e6e448e3e004376c74
2021-09-15 08:30:32 +00:00
Karthikeyan Chinnakonda
3247c8bd71 server: generalize event triggers - incremental PR 2
https://github.com/hasura/graphql-engine-mono/pull/2270

GitOrigin-RevId: d7644b25d3ee57ffa630de15ae692c1bfa03b4f6
2021-09-09 11:55:11 +00:00
Sameer Kolhar
f6af579619 server,docs,tests: add support for connection_parameters to pg_add_source API
https://github.com/hasura/graphql-engine-mono/pull/1690

GitOrigin-RevId: a7be66c9af3143b34133d197f7858ba22f442a41
2021-09-06 17:00:12 +00:00
Gil Mizrahi
5ca5e105a1 Fix telemetry unit test by sorting before comparing
https://github.com/hasura/graphql-engine-mono/pull/2068

GitOrigin-RevId: 62147b29c1899e43156e5f5d7f0fb67eda00647a
2021-08-11 15:36:30 +00:00
Karthikeyan Chinnakonda
06f5e4fb77 server: inherited roles for mutations, remote schemas, actions and custom functions
https://github.com/hasura/graphql-engine-mono/pull/1715

GitOrigin-RevId: 4818292cff8c3a5b264968e7032887a1e98b6f79
2021-08-09 10:21:05 +00:00
Lyndon Maydwell
d483109443 Revert "Disable TLS checks for actions services with self-signed certificates"
Reverts hasura/graphql-engine-mono#1595

https://github.com/hasura/graphql-engine-mono/pull/2036

GitOrigin-RevId: b32adde77b189c14eef0090866d58750d1481b50
2021-08-06 17:06:55 +00:00
jkachmar
4a83bb1834 Remote schema execution logic
https://github.com/hasura/graphql-engine-mono/pull/1995

Co-authored-by: David Overton <7734777+dmoverton@users.noreply.github.com>
GitOrigin-RevId: 178669089ec5e63b1f3da1d3ba0a9f8debbc108d
2021-08-06 13:40:37 +00:00
Antoine Leblanc
52d91e3e8d Update GraphQL Parser version to fix text encoding issue (fix #1965)
### A long tale about encoding

GraphQL has an [introspection system](http://spec.graphql.org/June2018/#sec-Introspection), which allows its schema to be introspected. This is what we use to introspect [remote schemas](41383e1f88/server/src-rsr/introspection.json). There is one place in the introspection where we might find GraphQL values: the default value of an argument.

```json
{
  "fields": [
    {
      "name": "echo",
      "args": [
        {
          "name": "msg",
          "defaultValue": "\"Hello\\nWorld!\""
        }
      ]
    }
  ]
}
```

Note that GraphQL's introspection is transport agnostic: the default value isn't returned as a JSON value, but as a _string-encoded GraphQL Value_. In this case, the value is the GraphQL String `"Hello\nWorld!"`. Embedded into a string, it is encoded as: `"\"Hello\\nWorld!\""`.

When we [parse that value](41383e1f88/server/src-lib/Hasura/GraphQL/RemoteServer.hs (L351)), we first extract that JSON string, to get its content, `"Hello\nWorld!"`, then use our [GraphQL Parser library](21c1ddfb41/src/Language/GraphQL/Draft/Parser.hs (L200)) to interpret this: we find the double quote, understand that the content is a String, unescape the backslashes, and end up with the desired string value: `['H', 'e', 'l', 'l', 'o', '\n', 'W', 'o', 'r', 'l', 'd', '!']`. This all works fine.

However, there was a bug in the _printer_ part of our parser library: when printing back a String value, we would not re-escape characters properly. In practice, this meant that the GraphQL String `"Hello\nWorld"` would be encoded in JSON as `"\"Hello\nWorld!\""`. Note how the `\n` is not properly double-escaped. This led to a variety of problems, as described in #1965:
- we would successfully parse a remote schema containing such characters in its default values, but then would print those erroneous JSON values in our introspection, which would _crash the console_
- we would inject those default values in queries sent to remote schemas, and print them wrong doing so, sending invalid values to remote schemas and getting errors in result

It turns out that this bug had been lurking in the code for a long time: I combed through the history of [the parser library](https://github.com/hasura/graphql-parser-hs), and as far as I can tell, this bug has always been there. So why was it never caught? After all, we do have [round trip tests](21c1ddfb41/test/Spec.hs (L52)) that print + parse arbitrary values and check that we get the same value as a result. They do use any arbitrary unicode character in their generated strings. So... that should have covered it, right?

Well... it turns out that [the tests were ignoring errors](7678066c49/test/Spec.hs (L45)), and would always return "SUCCESS" in CI, even if they failed... Furthermore, the sample size was small enough that, most of the time, _they would not hit such characters_. Running the tests locally on a loop, I only got errors ~10% of the time...

This was all fixed in hasura/graphql-parser-hs#44. This was probably one of Hasura's longest standing bugs? ^^'

### Description

This PR bumps the version of graphql-parser-hs in the engine, and switches some of our own arbitrary tests to use unicode characters in text rather than alphanumeric values. It turns out those tests were much better at hitting "bad" values, and that they consistently failed when generating arbitrary unicode characters.

https://github.com/hasura/graphql-engine-mono/pull/2031

GitOrigin-RevId: 54fa48270386a67336e5544351691619e0684559
2021-08-06 11:54:45 +00:00
Antoine Leblanc
2c0a8d818c Kill Arbitrary, take 2 (fix #1736)
### Description

A first PR, #1947, removed all the `Arbitrary` stuff from our codebase. But #1740, merged on the same day, added some tests relying on `Arbitrary`. In the merge process, some unneeded `Arbitrary` code got reintroduced.

This PR removes all `Arbitrary` stuff from `src-lib`, and cleans / refactor `Hasura.Generator` in `src-test` to only reduce it to the bare minimum amount of `Arbitrary` instances.

https://github.com/hasura/graphql-engine-mono/pull/1957

GitOrigin-RevId: 7e76009bb022205e3737fca45749411a266cc08c
2021-08-06 10:18:37 +00:00
Lyndon Maydwell
f6987ca4ff Disable TLS checks for actions services with self-signed certificates
https://github.com/hasura/graphql-engine-mono/pull/1595

GitOrigin-RevId: 3834e7d005bfaeaa7cc429c9d662d23b3d903f5c
2021-08-06 03:01:24 +00:00
jkachmar
c322af93f8 server: Uses GraphQL type in remote variable cache key
https://github.com/hasura/graphql-engine-mono/pull/1801

GitOrigin-RevId: 98843e422b2431849b675acdb318ffae2f492f18
2021-08-04 21:24:36 +00:00
David Overton
9be6d4cd97 Fix getCustomizer test case
https://github.com/hasura/graphql-engine-mono/pull/1986

GitOrigin-RevId: 2f94613a7f6afe89738aee0fc5fba0ab6b19c8da
2021-08-03 13:17:34 +00:00
David Overton
1abb1dee69 Remote Schema Customization take 2 using parser tranformations
https://github.com/hasura/graphql-engine-mono/pull/1740

GitOrigin-RevId: e807952058243a97f67cd9969fa434933a08652f
2021-07-30 11:33:59 +00:00
Antoine Leblanc
8adc8abf0f kill Arbitrary (fix #1736)
Delete all weird Arbitrary things from our codebase.

https://github.com/hasura/graphql-engine-mono/pull/1947

GitOrigin-RevId: b6d48db78d2ef4ac2fd232c684ff5039dc345fc4
2021-07-30 07:55:37 +00:00
Auke Booij
7bead93827 server: remove remnants of query plan caching (fix #1795)
Query plan caching was introduced by - I believe - hasura/graphql-engine#1934 in order to reduce the query response latency. During the development of PDV in hasura/graphql-engine#4111, it was found out that the new architecture (for which query plan caching wasn't implemented) performed comparably to the pre-PDV architecture with caching. Hence, it was decided to leave query plan caching until some day in the future when it was deemed necessary.

Well, we're in the future now, and there still isn't a convincing argument for query plan caching. So the time has come to remove some references to query plan caching from the codebase. For the most part, any code being removed would probably not be very well suited to the post-PDV architecture of query execution, so arguably not much is lost.

Apart from simplifying the code, this PR will contribute towards making the GraphQL schema generation more modular, testable, and easier to profile. I'd like to eventually work towards a situation in which it's easy to generate a GraphQL schema parser *in isolation*, without being connected to a database, and then parse a GraphQL query *in isolation*, without even listening any HTTP port. It is important that both of these operations can be examined in detail, and in isolation, since they are two major performance bottlenecks, as well as phases where many important upcoming features hook into.

Implementation

The following have been removed:
- The entirety of `server/src-lib/Hasura/GraphQL/Execute/Plan.hs`
- The core phases of query parsing and execution no longer have any references to query plan caching. Note that this is not to be confused with query *response* caching, which is not affected by this PR. This includes removal of the types:
- - `Opaque`, which is replaced by a tuple. Note that the old implementation was broken and did not adequately hide the constructors.
- - `QueryReusability` (and the `markNotReusable` method). Notably, the implementation of the `ParseT` monad now consists of two, rather than three, monad transformers.
- Cache-related tests (in `server/src-test/Hasura/CacheBoundedSpec.hs`) have been removed .
- References to query plan caching in the documentation.
- The `planCacheOptions` in the `TenantConfig` type class was removed. However, during parsing, unrecognized fields in the YAML config get ignored, so this does not cause a breaking change. (Confirmed manually, as well as in consultation with @sordina.)
- The metrics no longer send cache hit/miss messages.

There are a few places in which one can still find references to query plan caching:

- We still accept the `--query-plan-cache-size` command-line option for backwards compatibility. The `HASURA_QUERY_PLAN_CACHE_SIZE` environment variable is not read.

https://github.com/hasura/graphql-engine-mono/pull/1815

GitOrigin-RevId: 17d92b254ec093c62a7dfeec478658ede0813eb7
2021-07-27 11:52:43 +00:00
Vamshi Surabhi
e8e4f30dd6 server: support remote relationships on SQL Server and BigQuery (#1497)
Remote relationships are now supported on SQL Server and BigQuery. The major change though is the re-architecture of remote join execution logic. Prior to this PR, each backend is responsible for processing the remote relationships that are part of their AST.

This is not ideal as there is nothing specific about a remote join's execution that ties it to a backend. The only backend specific part is whether or not the specification of the remote relationship is valid (i.e, we'll need to validate whether the scalars are compatible).

The approach now changes to this:

1. Before delegating the AST to the backend, we traverse the AST, collect all the remote joins while modifying the AST to add necessary join fields where needed.

1. Once the remote joins are collected from the AST, the database call is made to fetch the response. The necessary data for the remote join(s) is collected from the database's response and one or more remote schema calls are constructed as necessary.

1. The remote schema calls are then executed and the data from the database and from the remote schemas is joined to produce the final response.

### Known issues

1. Ideally the traversal of the IR to collect remote joins should return an AST which does not include remote join fields. This operation can be type safe but isn't taken up as part of the PR.

1. There is a lot of code duplication between `Transport/HTTP.hs` and `Transport/Websocket.hs` which needs to be fixed ASAP. This too hasn't been taken up by this PR.

1. The type which represents the execution plan is only modified to handle our current remote joins and as such it will have to be changed to accommodate general remote joins.

1. Use of lenses would have reduced the boilerplate code to collect remote joins from the base AST.

1. The current remote join logic assumes that the join columns of a remote relationship appear with their names in the database response. This however is incorrect as they could be aliased. This can be taken up by anyone, I've left a comment in the code.

### Notes to the reviewers

I think it is best reviewed commit by commit.

1. The first one is very straight forward.

1. The second one refactors the remote join execution logic but other than moving things around, it doesn't change the user facing functionality.  This moves Postgres specific parts to `Backends/Postgres` module from `Execute`. Some IR related code to `Hasura.RQL.IR` module.  Simplifies various type class function signatures as a backend doesn't have to handle remote joins anymore

1. The third one fixes partial case matches that for some weird reason weren't shown as warnings before this refactor

1. The fourth one generalizes the validation logic of remote relationships and implements `scalarTypeGraphQLName` function on SQL Server and BigQuery which is used by the validation logic. This enables remote relationships on BigQuery and SQL Server.

https://github.com/hasura/graphql-engine-mono/pull/1497

GitOrigin-RevId: 77dd8eed326602b16e9a8496f52f46d22b795598
2021-06-11 03:27:39 +00:00
Vamshi Surabhi
96104ec1a8 Revert "remote schema typename customisation"
This reverts the remote schema type customisation and namespacing feature temporarily as we test for certain conditions.

GitOrigin-RevId: f8ee97233da4597f703970c3998664c03582d8e7
2021-06-10 09:57:16 +00:00
David Overton
4a69fdeb01 Dmoverton/5863 prefix namespacing
GitOrigin-RevId: 108e8b25e745cb4f74d143d316262049cef62b70
2021-06-09 22:42:05 +00:00
Karthikeyan Chinnakonda
72c24eea69 server: fix cron trigger bug of new events not getting generated when cron trigger is imported via metadata
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
GitOrigin-RevId: 1a9076e039ba2c8da898683c64ef988fb8512eae
2021-05-26 16:20:19 +00:00
Antoine Leblanc
6e95f761f5 server: rewrite remote input parsers to deal with partial variable expansion (fix hasura/graphql-engine#6656)
GitOrigin-RevId: e0b197a0fd1e259d43e6152b726b350c4d527a4b
2021-05-24 20:13:47 +00:00
Antoine Leblanc
5238bb8011 server: support for custom directives
Co-authored-by: Aravind K P <8335904+scriptonist@users.noreply.github.com>
GitOrigin-RevId: f11b3b2e964af4860c3bb0fd9efec6be54c2e88b
2021-05-20 10:03:50 +00:00
Antoine Leblanc
2152911e24 server: introduce Hasura.Base (take 2)
GitOrigin-RevId: 0dd10f1ccd338b1cf382ebff59b6ee7f209d39a1
2021-05-11 15:19:33 +00:00
kodiakhq[bot]
a935746e17 Integration test improvements (for speed/clarity), also increase polling interval for scheduled events
This claws back ~7min from integration tests (run serially, as with `dev.sh test --integration`
Further improvements would do well to focus on optimizing metadata operations, as `setup` dominates

GitOrigin-RevId: 76637d6fa953c2404627c4391447a05bf09355fa
2021-04-27 05:35:26 +00:00
Lyndon Maydwell
c737ce992d Schema-Sync Improvements for Cloud, Pro, and OSS - Version Checking for Metadata (#738)
Modifying schema-sync implementation to use polling for OSS/Pro. Invalidations are now propagated via the `hdb_catalog.hdb_schema_notifications` table in OSS/Pro. Pattern followed is now a Listener/Processor split with Cloud listening for changes via a LISTEN/NOTIFY channel and OSS polling for resource version changes in the metadata table. See issue #460 for more details.

GitOrigin-RevId: 48434426df02e006f4ec328c0d5cd5b30183db25
2021-04-06 03:25:53 +00:00
Vladimir Ciobanu
c08e6d108b server: allow GeoJSON to be passed for Geometry/Geography operators
GitOrigin-RevId: 51ca927b55d3d717da07447f67fd4d3c068a8357
2021-03-26 17:00:18 +00:00
Lyndon Maydwell
0c1016e065 Inconsistent metadata support for REST endpoints
Previously invalid REST endpoints would throw errors during schema cache build.

This PR changes the validation to instead add to the inconsistent metadata objects in order to allow use of `allow_inconsistent_metadata` with inconsistent REST endpoints.

All non-fatal endpoint definition errors are returned as inconsistent metadata warnings/errors depending on the use of `allow_inconsistent_metadata`. The endpoints with issues are then created and return informational runtime errors when they are called.

Console impact when creating endpoints is that error messages now refer to metadata inconsistencies rather than REST feature at the top level:

![image](https://user-images.githubusercontent.com/92299/109911843-ede9ec00-7cfe-11eb-9c55-7cf924d662a6.png)

<img width="969" alt="image" src="https://user-images.githubusercontent.com/92299/110258597-8336fa00-7ff7-11eb-872c-bfca945aa0e8.png">

Note: Conflicting endpoints generate one error per conflicting set of endpoints due to the implementation of `groupInconsistentMetadataById` and `imObjectIds`. This is done to ensure that error messages are terse, but may pose errors if there are some assumptions made surrounding `imObjectIds`.

Related to https://github.com/hasura/graphql-engine-mono/pull/473 (Allow Inconsistent Metadata (v2) #473 (Merged))

---

### Kodiak commit message

Changes the validation to use inconsistent metadata objects for REST endpoint issues.

#### Commit title

Inconsistent metadata for REST endpoints

GitOrigin-RevId: b9de971208e9bb0a319c57df8dace44cb115ff66
2021-03-10 05:26:10 +00:00
Karthikeyan Chinnakonda
92026b769f [Preview] Inherited roles for postgres read queries
fixes #3868

docker image - `hasura/graphql-engine:inherited-roles-preview-48b73a2de`

Note:

To be able to use the inherited roles feature, the graphql-engine should be started with the env variable `HASURA_GRAPHQL_EXPERIMENTAL_FEATURES` set to `inherited_roles`.

Introduction
------------

This PR implements the idea of multiple roles as presented in this [paper](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/FGALanguageICDE07.pdf). The multiple roles feature in this PR can be used via inherited roles. An inherited role is a role which can be created by combining multiple singular roles. For example, if there are two roles `author` and `editor` configured in the graphql-engine, then we can create a inherited role with the name of `combined_author_editor` role which will combine the select permissions of the `author` and `editor` roles and then make GraphQL queries using the `combined_author_editor`.

How are select permissions of different roles are combined?
------------------------------------------------------------

A select permission includes 5 things:

1. Columns accessible to the role
2. Row selection filter
3. Limit
4. Allow aggregation
5. Scalar computed fields accessible to the role

 Suppose there are two roles, `role1` gives access to the `address` column with row filter `P1` and `role2` gives access to both the `address` and the `phone` column with row filter `P2` and we create a new role `combined_roles` which combines `role1` and `role2`.

Let's say the following GraphQL query is queried with the `combined_roles` role.

```graphql
query {
   employees {
     address
     phone
   }
}
```

This will translate to the following SQL query:

```sql

 select
    (case when (P1 or P2) then address else null end) as address,
    (case when P2 then phone else null end) as phone
 from employee
 where (P1 or P2)
```

The other parameters of the select permission will be combined in the following manner:

1. Limit - Minimum of the limits will be the limit of the inherited role
2. Allow aggregations - If any of the role allows aggregation, then the inherited role will allow aggregation
3. Scalar computed fields - same as table column fields, as in the above example

APIs for inherited roles:
----------------------

1. `add_inherited_role`

`add_inherited_role` is the [metadata API](https://hasura.io/docs/1.0/graphql/core/api-reference/index.html#schema-metadata-api) to create a new inherited role. It accepts two arguments

`role_name`: the name of the inherited role to be added (String)
`role_set`: list of roles that need to be combined (Array of Strings)

Example:

```json
{
  "type": "add_inherited_role",
  "args": {
      "role_name":"combined_user",
      "role_set":[
          "user",
          "user1"
      ]
  }
}
```

After adding the inherited role, the inherited role can be used like single roles like earlier

Note:

An inherited role can only be created with non-inherited/singular roles.

2. `drop_inherited_role`

The `drop_inherited_role` API accepts the name of the inherited role and drops it from the metadata. It accepts a single argument:

`role_name`: name of the inherited role to be dropped

Example:

```json

{
  "type": "drop_inherited_role",
  "args": {
      "role_name":"combined_user"
  }
}
```

Metadata
---------

The derived roles metadata will be included under the `experimental_features` key while exporting the metadata.

```json
{
  "experimental_features": {
    "derived_roles": [
      {
        "role_name": "manager_is_employee_too",
        "role_set": [
          "employee",
          "manager"
        ]
      }
    ]
  }
}
```

Scope
------

Only postgres queries and subscriptions are supported in this PR.

Important points:
-----------------

1. All columns exposed to an inherited role will be marked as `nullable`, this is done so that cell value nullification can be done.

TODOs
-------

- [ ] Tests
   - [ ] Test a GraphQL query running with a inherited role without enabling inherited roles in experimental features
   - [] Tests for aggregate queries, limit, computed fields, functions, subscriptions (?)
   - [ ] Introspection test with a inherited role (nullability changes in a inherited role)
- [ ] Docs
- [ ] Changelog

Co-authored-by: Vamshi Surabhi <6562944+0x777@users.noreply.github.com>
GitOrigin-RevId: 3b8ee1e11f5ceca80fe294f8c074d42fbccfec63
2021-03-08 11:15:10 +00:00
Karthikeyan Chinnakonda
4211d27272 server: support reading JWT from Cookie header
GitOrigin-RevId: 1de90242d3c000361f87256c2dddce1677863231
2021-02-25 09:03:46 +00:00
Karthikeyan Chinnakonda
7be72d7bee server: support for maintenance mode in v1.4
Co-authored-by: Anon Ray <616387+ecthiender@users.noreply.github.com>
GitOrigin-RevId: 85f636b1bc8063689845da90e85cc480e87dca7e
2021-02-18 16:47:22 +00:00
Rakesh Emmadi
9ef603360c server: generalize schema cache building (#496)
Co-authored-by: Vamshi Surabhi <vamshi@hasura.io>
Co-authored-by: Vladimir Ciobanu <admin@cvlad.info>
Co-authored-by: Antoine Leblanc <antoine@hasura.io>
Co-authored-by: Stylish Haskell Bot <stylish-haskell@users.noreply.github.com>
GitOrigin-RevId: 9d631878037637f3ed2994b5d0525efd978f7b8f
2021-02-14 06:08:46 +00:00
Swann Moreau
c14dcd5792 pass gql requests into auth webhook POST body (#149)
* fix arg order in UserAuthentication instance [force ci]

* change the constructor name to AHGraphQLRequest

Co-authored-by: Stylish Haskell Bot <stylish-haskell@users.noreply.github.com>
Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
GitOrigin-RevId: fb3258f4a84efc6c730b0c6222ebd8cea1b91081
2021-02-03 07:11:39 +00:00
Karthikeyan Chinnakonda
10a3f9960d server: new function permissions layer
Co-authored-by: Rikin Kachhia <54616969+rikinsk@users.noreply.github.com>
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
GitOrigin-RevId: 35645121242294cb6bb500ea598e9a1f2ca67fa1
2021-01-29 05:49:09 +00:00
Lyndon Maydwell
0767333597 server: support restified versions of graphql queries (#303)
Restified GraphQL Endpoints feature.

GitOrigin-RevId: 3d6e589426ec21a60a915b47f579f0ac4934af45
2021-01-29 01:03:35 +00:00
Karthikeyan Chinnakonda
3020150274 server: allow mapping session variables to standard JWT claims
fixes https://github.com/hasura/graphql-engine/issues/6449

A while back we added [support for customizing JWT claims](https://github.com/hasura/graphql-engine/pull/3575) and this enabled to map a session variable to any value within the unregistered claims, but as reported in #6449 , users aren't able to map the `x-hasura-user-id` session variable to the `sub` standard JWT claim.

This PR fixes the above issue by allowing mapping session variables to standard JWT claims as well.

GitOrigin-RevId: d3e63d7580adac55eb212e0a1ecf7c33f5b3ac4b
2021-01-21 16:50:46 +00:00
Karthikeyan Chinnakonda
c14bcb6967 server: accept new config allowed_skew in JWT config to provide leeway in JWT expiry
fixes https://github.com/hasura/graphql-engine/issues/2109

This PR accepts a new config `allowed_skew` in the JWT config to provide for some leeway while comparing the JWT expiry time.

GitOrigin-RevId: ef50cf77d8e2780478685096ed13794b5c4c9de4
2021-01-13 08:39:18 +00:00
Antoine Leblanc
3948ca84da server: RQL code health
This PR is a combination of the following other PRs:
- #169: move HasHttpManager out of RQL.Types
- #170: move UserInfoM to Hasura.Session
- #179: delete dead code from RQL.Types
- #180: move event related code to EventTrigger

GitOrigin-RevId: d97608d7945f2c7a0a37e307369983653eb62eb1
2021-01-08 23:10:36 +00:00
Rakesh Emmadi
be62641f68 server: multi source metadata APIs (#217)
Co-authored-by: Aleksandra Sikora <aleksandra@hasura.io>
Co-authored-by: Anon Ray <ecthiender@users.noreply.github.com>
Co-authored-by: Vishnu Bharathi <vishnubharathi04@gmail.com>
Co-authored-by: Aleksandra Sikora <aleksandra@hasura.io>
Co-authored-by: Sameer Kolhar <kolhar730@gmail.com>
Co-authored-by: Aleksandra Sikora <ola.zxcvbnm@gmail.com>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
GitOrigin-RevId: 0dd1e4d58ab81f1b4ce24de2d3eab709c2755e6d
2021-01-07 09:05:19 +00:00
Rakesh Emmadi
29f2ddc289 server: support separate metadata database and server code setup for multi sources (#197)
This is an incremental PR towards https://github.com/hasura/graphql-engine/pull/5797

Co-authored-by: Anon Ray <ecthiender@users.noreply.github.com>
GitOrigin-RevId: a6cb8c239b2ff840a0095e78845f682af0e588a9
2020-12-28 12:56:55 +00:00
Phil Freeman
2dfbf99b41 server: simplify shutdown logic, improve resource management (#218) (#195)
* Remove unused ExitCode constructors

* Simplify shutdown logic

* Update server/src-lib/Hasura/App.hs

Co-authored-by: Brandon Simmons <brandon@hasura.io>

* WIP: fix zombie thread issue

* Use forkCodensity for the schema sync thread

* Use forkCodensity for the oauthTokenUpdateWorker

* Use forkCodensity for the schema update processor thread

* Add deprecation notice

* Logger threads use Codensity

* Add the MonadFix instance for Codensity to get log-sender thread logs

* Move outIdleGC out to the top level, WIP

* Update forkImmortal fuction for more logging info

* add back the idle GC to Pro

* setupAuth

* use ImmortalThreadLog

* Fix tests

* Add another finally block

* loud warnings

* Change log level

* hlint

* Finalize the logger in the correct place

* Add ManagedT

* Update server/src-lib/Hasura/Server/Auth.hs

Co-authored-by: Brandon Simmons <brandon@hasura.io>

* Comments etc.

Co-authored-by: Brandon Simmons <brandon@hasura.io>
Co-authored-by: Naveen Naidu <naveennaidu479@gmail.com>
GitOrigin-RevId: 156065c5c3ace0e13d1997daef6921cc2e9f641c
2020-12-21 18:56:57 +00:00
Karthikeyan Chinnakonda
39a4352569 Merge pull request #113 from hasura/karthikeyan/remote-schema-permissions
server: remote schema permissions
GitOrigin-RevId: 63b9717e30351676c9474bdfddd3ad1ee1409eea
2020-12-21 09:12:35 +00:00
Tirumarai Selvan
b544b87b9b Merge pull request #223 from hasura/jberryman/5863-prep-refactoring
GitOrigin-RevId: 71b1453edf4b93ffc16a15ea3c6057bb865b6606
2020-12-20 06:53:38 +00:00
Auke Booij
84f2991c3d server: schema cache generation generalization (#213)
Generalize TableCoreInfoRM, TableCoreCacheRT, some table metadata data types, generalize fromPGCol to fromCol, generalize some schema cache functions, prepare some enum schema cache code for generalization

GitOrigin-RevId: a65112bc1688e00fd707d27af087cb2585961da2
2020-12-17 11:38:15 +00:00
Rakesh Emmadi
a153e96309 server: expand metadata storage class with async actions and core metadata operations (#184)
An incremental PR towards https://github.com/hasura/graphql-engine/pull/5797
- Expands `MonadMetadataStorage` with operations related to async actions and setting/updating metadata

GitOrigin-RevId: 53386b7b2d007e162050b826d0708897f0b4c8f6
2020-12-14 04:31:20 +00:00
Rakesh Emmadi
a2cf9a53c2 server: move to storing metadata as a json blob (#115)
GitOrigin-RevId: 3d1a7618a4ec086c2d255549a6c15087201e9ab0
2020-12-08 14:23:28 +00:00
Vishnu Bharathi P
58c44f55dd Merge oss/master onto mono/main
GitOrigin-RevId: 1c8c4d60e033c8a0bc8b2beed24c5bceb7d4bcc8
2020-11-12 22:37:19 +05:30
Vishnu Bharathi P
666058ab7f oss: renames dot files and folders
GitOrigin-RevId: 540aeec3be091e1cfb7b05a988f50445534ed663
2020-11-12 22:37:19 +05:30
Auke Booij
3bcde3d4b8
server: metadata separation: reorganize metadata types (#6103)
https://github.com/hasura/graphql-engine/pull/6103
2020-11-03 18:01:33 +00:00
Auke Booij
0540b279db
server: make more use of hlint (#6059)
https://github.com/hasura/graphql-engine/pull/6059
2020-10-28 16:40:33 +00:00
Alexis King
bf466e3b63
server: Fix fine-grained incremental cache invalidation (fix #3759) (#6027)
This issue was very tricky to track down, but fortunately easy to fix.
The interaction here is subtle enough that it’s difficult to put into
English what would go wrong in what circumstances, but the new unit test
captures precisely that interaction to ensure it remains fixed.
2020-10-27 14:52:19 -05:00
Rakesh Emmadi
4ce6002af2
support customizing JWT claims (close #3485) (#3575)
* improve jsonpath parser to accept special characters and property tests for the same

* make the JWTClaimsMapValueG parametrizable

* add documentation in the JWT file

* modify processAuthZHeader

Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
Co-authored-by: Marion Schleifer <marion@hasura.io>
2020-08-31 22:10:01 +05:30
Brandon Simmons
1d4ec4eafb Support only the bounded cache, with default HASURA_GRAPHQL_QUERY_PLAN_CACHE_SIZE of 4000. Closes #5363 2020-07-28 19:02:44 -04:00
Brandon Simmons
3a6b2ec744 Bugfix to support 0-size HASURA_GRAPHQL_QUERY_PLAN_CACHE_SIZE
Also some minor refactoring of bounded cache module:
 - the maxBound check in `trim` was confusing and unnecessary
 - consequently trim was unnecessary for lookupPure

Also add some basic tests
2020-07-27 18:40:17 -04:00
Brandon Simmons
2eab6a89aa Fix latency buckets for telemetry data
These must have gotten messed up during a refactor. As a consequence
almost all samples received so far fall into the single erroneous 0 to
1K seconds (originally supposed to be 1ms?) bucket.

I also re-thought what the numbers should be, but these are still
arbitrary and might want adjusting in the future.
2020-07-22 12:29:38 -04:00
Phil Freeman
0dddbe9e9d
Add MonadTrace and MonadExecuteQuery abstractions (#5383)
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
2020-07-15 16:10:48 +05:30