Commit Graph

90 Commits

Author SHA1 Message Date
Daniel Chambers
59ffce9ac1 Generate TypeScript types for the Data Connector reference agent from the OpenAPI schema
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4855
GitOrigin-RevId: 4cc09d6706e67c69fbbedef72ff816365b9f7b4e
2022-06-29 07:43:52 +00:00
Brandon Simmons
80cc53aaac server: vendor used bits of 'ip' package to cut dependencies
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4704
GitOrigin-RevId: 14759b410e99a1a97aa01b1139d9d61292301aff
2022-06-20 23:16:51 +00:00
Daniel Chambers
0723925ffc Expose the entire official Chinook dataset through the DC Reference agent and use it in the tests
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4688
GitOrigin-RevId: f87b2e23be69aed3cc7f95e814f9426db174dc9b
2022-06-13 20:59:48 +00:00
Brandon Simmons
45af456fb3 dev: add new freeze file helper script: scripts/cabal-freeze-update.sh
This was reworked from an earlier version based on feedback from @solomon-b : https://github.com/hasura/graphql-engine-mono/pull/4618

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4646
GitOrigin-RevId: c54469a2d8466471e6d5c25a0cc80e26d94ce7c7
2022-06-08 21:14:10 +00:00
Brandon Simmons
6e8da71ece server: migrate to aeson-2 in preparation for ghc 9.2 upgrade
(Work here originally done by awjchen, rebased and fixed up for merge by
jberryman)

This is part of a merge train towards GHC 9.2 compatibility. The main
issue is the use of the new abstract `KeyMap` in 2.0. See:
https://hackage.haskell.org/package/aeson-2.0.3.0/changelog

Alex's original work is here:
#4305

BEHAVIOR CHANGE NOTE: This change causes a different arbitrary ordering
of serialized Json, for example during metadata export. CLI users care
about this in particular, and so we need to call it out as a _behavior
change_ as we did in v2.5.0. The good news though is that after this
change ordering should be more stable (alphabetical key order).

See: https://hasurahq.slack.com/archives/C01M20G1YRW/p1654012632634389

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4611
Co-authored-by: awjchen <13142944+awjchen@users.noreply.github.com>
GitOrigin-RevId: 700265162c782739b2bb88300ee3cda3819b2e87
2022-06-08 15:32:27 +00:00
paritosh-08
fd30fb343b server: naming conventions for auto generated fields
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3982
Co-authored-by: Brandon Simmons <210815+jberryman@users.noreply.github.com>
GitOrigin-RevId: f90b2e8f394e7bd69780f003d2d980475f104f42
2022-05-26 11:55:29 +00:00
Solomon
b4f89569c8 GDC: Integration Tests and Servant Agent
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4467
GitOrigin-RevId: 5e81d8581197c90ad2de9106e724c63d7592ae72
2022-05-11 06:15:27 +00:00
Daniel Chambers
4f835623b1 Rename Data Wrapper to Data Connector [GDW-89]
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4400
GitOrigin-RevId: 2d90542f95ef4dad70c8dfe1ca3b4c1f3bdaa527
2022-05-02 05:04:07 +00:00
jkachmar
d058dafae2 server: Update pg-client-hs and graphql-parser-hs
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4247
GitOrigin-RevId: b101466444d5c650c6886149537b1de2db4809ad
2022-04-12 01:23:52 +00:00
Daniel Chambers
38c41b2ae6 GDW Agent Test Suite [GDW-79]
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4175
GitOrigin-RevId: d37d7d131597af6b9cca6bd773c8dbbce8719ca5
2022-04-10 04:48:30 +00:00
David Overton
44577dab1b Add ToSchema instances to GDW API types
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4006
Co-authored-by: Daniel Chambers <1214352+daniel-chambers@users.noreply.github.com>
GitOrigin-RevId: 756ca0ed60865d0eb675562e8959f0d1839f9abe
2022-03-31 04:46:08 +00:00
Solomon
94331e23f5 GDW-15 Serializable Types
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3924
GitOrigin-RevId: 75b276edcd2d1f88bbdbed1b96b08708f9c68450
2022-03-16 04:13:08 +00:00
Robert
437ef45431 pro, multitenant: remove broken process-level ekg stats
Fixes #3364.

This removes code that's only there to collect some metrics that were apparently never collected in production anyway, as confirmed by this slack discussion: https://hasurahq.slack.com/archives/CTRL7L1PZ/p1645547435034329

> We don't ever ingest these statsd metrics to lux, nor on HC AFAIK.

We're keeping the command line options for the moment to allow lux time to stop passing them.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3365
GitOrigin-RevId: bc6aa2c18a7d3eb0779ed17494662df5b7b54a76
2022-03-07 16:54:19 +00:00
Puru Gupta
fcef6e5cb2 server: http ip blocklist (closes #2449)
## Description

This PR is in reference to #2449 (support IP blacklisting for multitenant)

*RFC Update: Add support for IPv6 blocking*

### Solution and Design

Using [http-client-restricted](https://hackage.haskell.org/package/http-client-restricted) package, we're creating the HTTP manager with restricting capabilities. The IPs can be supplied from the CLI arguments as `--ipv4BlocklistCidrs cidr1, cidr2...` or `--disableDefaultIPv4Blocklist` for a default IP list. The new manager will block all requests to the provided CIDRs.

We are extracting the error message string to show the end-user that given IP is blocked from being set as a webhook. There are 2 ways to extract the error message "connection to IP address is blocked". Given below are the responses from event trigger to a blocked IP for these implementations:
- 6d74fde316f61e246c861befcca5059d33972fa7 - We return the error message string as a HTTPErr(HOther) from `Hasura/Eventing/HTTP.hs`.
```
{
    "data": {
        "message": "blocked connection to private IP address "
    },
    "version": "2",
    "type": "client_error"
}
```

- 88e17456345cbb449a5ecd4877c84c9f319dbc25 - We case match on HTTPExceptionContent for InternaException in `Hasura/HTTP.hs` and extract the error message string from it. (this is implemented as it handles all the cases where pro engine makes webhook requests)
```
{
  "data": {
    "message": {
      "type": "http_exception",
      "message": "blocked connection to private IP address ",
      "request": {
        "secure": false,
        "path": "/webhook",
        "responseTimeout": "ResponseTimeoutMicro 60000000",
        "queryString": "",
        "method": "POST",
        "requestHeaders": {
          "Content-Type": "application/json",
          "X-B3-ParentSpanId": "5ae6573edb2a6b36",
          "X-B3-TraceId": "29ea7bd6de6ebb8f",
          "X-B3-SpanId": "303137d9f1d4f341",
          "User-Agent": "hasura-graphql-engine/cerebushttp-ip-blacklist-a793a0e41-dirty"
        },
        "host": "139.59.90.109",
        "port": 8000
      }
    }
  },
  "version": "2",
  "type": "client_error"
}
```

### Steps to test and verify
The restricted IPs can be used as webhooks in event triggers, and hasura will return an error message in reponse.

### Limitations, known bugs & workarounds
- The `http-client-restricted` has a needlessly complex interface, and puts effort into implementing proxy support which we don't want, so we've inlined a stripped down version.
- Performance constraint: As the blocking is checked for each request, if a long list of blocked CIDRs is supplied, iterating through all of them is not what we would prefer. Using trie is suggested to overcome this. (Added to RFC)
- Calls to Lux endpoints are inconsistent: We use either the http manager from the ProServeCtx which is unrestricted, or the http manager from the ServeCtx which is restricted (the latter through the instances for MonadMetadataApiAuthorization and UserAuthentication). (The failure scenario here would be: cloud sets PRO_ENDPOINT to something that resolves to an internal address, and then restricted requests to those endpoints fail, causing auth to fail on user requests. This is about HTTP requests to lux auth endpoints.)

## Changelog

-  `CHANGELOG.md` is updated with user-facing content relevant to this PR.

## Affected components

-  Server
-  Tests

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3186
Co-authored-by: Robert <132113+robx@users.noreply.github.com>
GitOrigin-RevId: 5bd2de2d028bc416b02c99e996c7bebce56fb1e7
2022-02-25 13:30:57 +00:00
paritosh-08
bc74046ab2 server: case insensitive lookup in session variable
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3663
GitOrigin-RevId: b36666d8849a23dad209f9921f140390c2b57496
2022-02-24 13:08:35 +00:00
Robert
de175c6e66 multitenant: remove stale multitenant-test (fixes #3652)
The tests haven't been passing for a while. They also seem of
limited value, and not worth the dependency footprint.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3722
GitOrigin-RevId: dd0283bf78b2a7ba646789a1a354d095ea3f54e4
2022-02-24 09:13:46 +00:00
Antoine Leblanc
5541ec011e Add remote relationships hspec tests
## Description

This PR adds the possibility for hspec tests to start a remote server with a custom schema, using the _morpheus_ library. In addition, it adds:
- X-to-DB object relationships tests
- X-to-DB array relationships tests
- X-to-RS relationships tests

For now, all those X are only postgres, but the tests are written in a way that will allow for it to easily be any other DB, or even remote schemas. The actual tests were taken mostly from #3069.

To achieve this, this PR heavily refactors the test harness. Most importantly: it generalizes the notion of a `Backend` to a notion of generic `Context`, allowing for contexts that are the unions of two backends, or of a backend and a remote schema.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3567
Co-authored-by: jkachmar <8461423+jkachmar@users.noreply.github.com>
GitOrigin-RevId: 623f700ba482743f94d3eaf659e6cfa22cd0dbc9
2022-02-18 13:36:31 +00:00
jkachmar
d50aae87a5 Updates cabal freeze file
#### TODO

- [x] fix `hashable >= 1.3.1` serialization ordering issue [^1]
  - `test_graphql_mutations.py::TestGraphQLMutateEnums` was failing
- [x] fix `unordered-containers` serialization ordering issue [^2]
  - `test_graphql_queries.py` was failing on Citus
- [ ] verify that no new failures have been introduced
- [ ] open issues to fix the above
  - identify test cases that "leak" implementation details by depending on `hashable` instance ordering
  - bump `hashable >= 1.3.1` and update test cases with new ordering OR modify them so that ordering is stable
  - bump `unordered-containers >= 0.2.15.0` and update test cases with new ordering OR modify them so that ordering is stable
    - one of the test cases was failing on string equality comparison for a generated Citus query
    - we probably don't want to _actually_ do this unless there are _very specific_ guarantees we want to make about generated query structure
---

Just what it says on the tin.

https://github.com/hasura/graphql-engine-mono/pull/3538 updated the freeze file a few weeks ago, but it looks like the index state hadn't been updated since December so a lot of stuff that had newer versions didn't get updated.

---

EDIT: I should add, the motivation for doing this in the first place is that `hspec > 2.8.4` now supports specifying filtering spec trees based on patterns provided by the `HSPEC_MATCH` environment variable.

For example, one could have a script that executes the following:
```
HSPEC_MATCH="PostgreSQL" \
  ghcid \
    --command \
      'cabal repl graphql-engine:test:tests-hspec \
         --repl-option -O0 \
         --repl-option -fobject-code' \
    --test "main"
```
...which will loop on typechecking the `tests-hspec` component, and then as soon as it passes (i.e. no warnings or errors) will run _only_ the `PostgreSQL` sub-components.

[^1]: `hashable >= 1.3.1.0` [updated its default salts](https://github.com/haskell-unordered-containers/hashable/pull/196), which [broke serialization ordering](https://github.com/haskell/aeson/issues/837)
[^2]: `unordered-containers >= 0.2.16.0` [introduced changes to some of its internal functions](https://hackage.haskell.org/package/unordered-containers-0.2.16.0/changelog) which seem like they could have affected serialization stability

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3672
GitOrigin-RevId: bbd1d48c73db4021913f0b5345b7315a8d6525d3
2022-02-18 05:32:08 +00:00
Solomon
6c7820caa0 Bumps Kriti Version
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3625
GitOrigin-RevId: a453254ac89b470a56189e6178b2a00cf628fe63
2022-02-15 22:32:38 +00:00
Gil Mizrahi
476eb5fe2b update odbc to 0.2.6
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3647
GitOrigin-RevId: 82fb189c2bff9dce1c1b2e8e9dd2be4d9bc7f527
2022-02-14 10:43:08 +00:00
Robert
4cb6f96e22 haskell: remove outdated dependencies from cabal.project.freeze
Result of running `cabal freeze` locally and then removing the flags on local targets.

The context is that I keep seeing unintelligible diffs to `cabal.project.freeze` in PRs that add dependencies. The idea being that we should keep `cabal.project.freeze` in a state where `cabal freeze` (or whichever other way you're meant to update the freeze file) doesn't cause unrelated changes.

See https://github.com/hasura/graphql-engine-mono/issues/3545 for a follow-up ticket primarily aiming at documenting how we update the freeze file.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3538
GitOrigin-RevId: 45a5dadcdfe29d5c831807dc49f877a2380162fc
2022-02-03 15:35:38 +00:00
Robert
5df40cc65b multitenant tests: reduce reliance on env vars, use fake cabal run
There's two major parts to this change:

1. cut down on environment variables needed to run the tests:
  - lux parameters that don't change are now in code
  - most remaining parameters have reasonable defaults
  - the only variable that is still required to be set is HASURA_MULTITENANT_SPEC_DB_URL
  - env.sh is no longer needed
2. find a better work-around for the problems running graphql-engine-multitenant
  via cabal run (https://github.com/haskell/cabal/issues/7914), by adding a shell
  script that implements a more correct version of cabal run.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3461
GitOrigin-RevId: 0939a79cc45cd3c1c103719552b12099678850dd
2022-01-24 18:15:25 +00:00
Evie Ciobanu
9ddb3ae106 server: fix building kriti-lang by using a branch fix
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3432
Co-authored-by: Robert <132113+robx@users.noreply.github.com>
GitOrigin-RevId: c067eee0a7dba32b79420ab078b2233d4fbfda75
2022-01-21 10:29:51 +00:00
Rakesh Emmadi
5419236afd server/mssql: update odbc to 0.2.5
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3386
GitOrigin-RevId: fe508c34f548d1a5653b1d4f484277d5b4e41167
2022-01-19 06:58:30 +00:00
Gil Mizrahi
ac8f9daaa9 Use hspec-discover for hspec-tests
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3328
GitOrigin-RevId: 5f1cfc43434d971083b32e992e40f348bbf75ecd
2022-01-13 21:14:53 +00:00
awjchen
1f196ea2fc server: update ekg forks
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3213
GitOrigin-RevId: 5ccc59012784542e7eac30e0277482cc85e8abaa
2021-12-27 17:01:46 +00:00
jkachmar
63cff8b731 Experimental backend adapter
This commit introduces an "experimental" backend adapter to the GraphQL Engine.

It defines a high-level interface which will eventually be used as the basis for implementing separate data source query generation & marshaling services that communicate with the GraphQL Engine Server via some protocol.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2684
Co-authored-by: awjchen <13142944+awjchen@users.noreply.github.com>
Co-authored-by: Chris Parks <592078+cdparks@users.noreply.github.com>
GitOrigin-RevId: 4463b682142ad6e069e223b88b14db511f634768
2021-12-22 00:11:28 +00:00
jkachmar
44fa624ee9 server: Bumps 'network' to a version that supports M1 ARM
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3111
GitOrigin-RevId: be76385679a49715daf0bbafa7cbf56a0b6b0b70
2021-12-13 08:13:05 +00:00
Chris Done
12335bce91 Add a test of tests in common via persistent #2872
Dupe of https://github.com/hasura/graphql-engine-mono/pull/2872 with branch renamed so it doesn't break a tool.

prev pr: https://github.com/hasura/graphql-engine-mono/pull/2921

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2922
Co-authored-by: Abby Sassel <3883855+sassela@users.noreply.github.com>
GitOrigin-RevId: 24af6a814dd106864a2fd3cb985edfe9a8ef5d61
2021-11-26 18:22:00 +00:00
Solomon
2bbc3e5d26 Bumps the Kriti Pin to 0.2.2
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2891
Co-authored-by: Lyndon Maydwell <92299+sordina@users.noreply.github.com>
GitOrigin-RevId: 2517fc3efdf6708177d6163db023c67a7f7e2ec5
2021-11-22 12:23:23 +00:00
jkachmar
576b07db28 ci,server: upgrades to GHC 8.10.7 and reworks server builder image
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2727
Co-authored-by: Vishnu Bharathi <4211715+scriptnull@users.noreply.github.com>
Co-authored-by: Swann Moreau <62569634+evertedsphere@users.noreply.github.com>
Co-authored-by: Robert <132113+robx@users.noreply.github.com>
GitOrigin-RevId: 0779ab7755fd4bdb7ed1551629b2c01d99c5ccf0
2021-11-15 13:22:39 +00:00
Solomon Bothwell
256b59f030 Updates kriti pin to 0.2.1
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2697
GitOrigin-RevId: 89705a3cc5a67c28cbf8d120df190fc4e42c24ff
2021-10-28 00:26:30 +00:00
awjchen
c49f913539 tests: Implement tests for DB-to-DB joins
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2645
GitOrigin-RevId: ab85f4911c5b3559ea7878e6682459fd8fc6a356
2021-10-21 03:40:17 +00:00
awjchen
67b4e1cc5b server/tests: initial work on a test suite for DB-to-DB joins
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2403
Co-authored-by: Kali Vara Purushotham Santhati <72007599+purush7@users.noreply.github.com>
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Naveen Naidu <30195193+Naveenaidu@users.noreply.github.com>
Co-authored-by: Divi <32202683+imperfect-fourth@users.noreply.github.com>
Co-authored-by: Rikin Kachhia <54616969+rikinsk@users.noreply.github.com>
GitOrigin-RevId: deaad7afa9df5d3e542cf699e57957de9254f347
2021-10-20 20:02:34 +00:00
Puru Gupta
8b60122b9e [server] add openapi support
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/1935
Co-authored-by: paritosh-08 <85472423+paritosh-08@users.noreply.github.com>
Co-authored-by: pranshi06 <85474619+pranshi06@users.noreply.github.com>
Co-authored-by: Lyndon Maydwell <92299+sordina@users.noreply.github.com>
GitOrigin-RevId: 3e43b84d4e9e181b405855704112b49467dafdf9
2021-10-06 07:16:13 +00:00
Lyndon Maydwell
1ae8bf00f7 Update Kriti to support field lookup by string literal in webhook transformations
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2482
GitOrigin-RevId: 4f5ea1c3b7989a17368ab995c6a45c7135253ba0
2021-10-01 06:24:06 +00:00
Solomon Bothwell
a7cc5df95d Fixes update_action transform bug and updates Kriti pin
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2399
GitOrigin-RevId: 009ee6ec2a3834951e464c232eb517d636ab940a
2021-09-28 01:38:24 +00:00
Solomon Bothwell
af5ff07614 Request Transformations
https://github.com/hasura/graphql-engine-mono/pull/1984

Co-authored-by: jkachmar <8461423+jkachmar@users.noreply.github.com>
GitOrigin-RevId: 1767d6bdde48c156fe171b5a9b7e44d7f2eb4869
2021-09-16 11:03:57 +00:00
Gil Mizrahi
904029aa7d Add fancier trace debugging functions to Hasura.Prelude
## Suggestion: Add fancier trace debugging functions to `Hasura.Prelude`

This PR adds two trace functions, `ltrace` and `ltraceM`, which use the `pretty-simple` package to `show` the input with nice formatting and colors for ease of reading (and comparing using diff tools such as `meld` or `vim-diff`).

I've also added warning pragmas to the functions, which means:
1. Traces will not be left in code, as CI builds with -Werror
2. Developers will have to change the `ghc-options` to `-Wwarn` in their `cabal.project.local` settings to use these functions

### Example

Usage:

```hs
selectFunctionAggregate ... = ... do
  ltraceM "functionInfo" function
  ...
```

Output to terminal looks like this:

<img width="524" alt="Screen Shot 2021-08-12 at 10 33 24" src="https://user-images.githubusercontent.com/8547573/129158878-4a5e96ba-30a5-452c-8f33-9eb4b2cc5e2a.png">

### Dependencies

Requires adding the following dependencies:
  - prettyprinter-ansi-terminal-1.1.2 (BSD2)
  - pretty-simple-4.0.0.0 (BSD3)

Question: what is the process for adding new dependencies? How does decisions on this matter happen?

https://github.com/hasura/graphql-engine-mono/pull/2075

GitOrigin-RevId: 490b0f0ca595da319b43e92e190ba50c0b132cd5
2021-08-12 19:33:56 +00:00
awjchen
046da3caca Combine oss and pro server configuration
https://github.com/hasura/graphql-engine-mono/pull/1983

GitOrigin-RevId: 91069bd5d1f9c5a166eae2317cf4435220b472b0
2021-08-11 04:19:27 +00:00