Commit Graph

77 Commits

Author SHA1 Message Date
pranshi06
1372a649df server: add support for ES-* ( ES256, ES384 and ES512) algorithms for signing the JWT
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9273
GitOrigin-RevId: e891a14e992e4345f5470e1e99dbfc21d9105c31
2023-06-08 09:27:42 +00:00
Samir Talwar
e24bcf2a39 server/tests-py: Never allocate the same port twice.
We seem to be getting flakes where we try and use the same port for two different servers. This is because in certain cases we cannot simply allocate the port dynamically, but have to decide it in advance, leading to a race condition.

We resolve this by keeping track of the ports we allocate when using this method, making sure we never allocate them twice. We also make sure we allocate from a different pool of ports to the usual dynamic port pool (typically above port 32768, and often above port 49152).

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8903
GitOrigin-RevId: 375a23867591a4566493dddbc550c58cf88ea392
2023-04-25 12:51:01 +00:00
Samir Talwar
6ed9f36125 Remove the server upgrade/downgrade tests.
These tests are intended to catch issues in upgrading HGE. However:

* the tests are very convoluted and hard to understand,
* we can only run a small subset of Python tests that don't mutate any data or metadata, and
* I have never seen them fail for a legitimate reason, but I've seen a lot of flakes.

While we do believe it's important to test that upgrades don't break the stored introspection, these tests don't seem to be doing that any more. I humbly request that we delete them now and either (a) figure out how to test this properly, or (b) just wait for v3, which does away with reintrospecting on server startup entirely.

[NDAT-259]: https://hasurahq.atlassian.net/browse/NDAT-259?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8844
GitOrigin-RevId: 528bc632fce377b7eff2026b832bd58586ac5a0b
2023-04-24 10:04:15 +00:00
Samir Talwar
26e03a07bb server/tests-py: Parallelize JWT tests.
This rewrites the JWT tests to generate and specify the secrets per test class, and to provide the server configuration to the HGE fixture.

It covers the tests in:

  - *test_jwt.py*
  - *test_jwt_claims_map.py*
  - *test_config_api.py*
  - *test_graphql_queries.py* (just a couple here)

This does reduce the number of code paths exercised with JWT, as we were previously running *all* tests with JWT tokens. However, this seems excessive; we don't need to tread every code path, just enough to ensure we handle the tokens appropriately. I believe that the test coverage in *test_jwt.py* does this well enough (though I'd prefer if we moved the coverage lower down in the stack as unit tests).

These tests were configured in multiple different ways by *test-server.sh*; this configuration is now moved to test subclasses within the various files. This results in a bit of duplication.

Unfortunately, the tests would ideally use parameterization rather than subclassing, but that doesn't work because of `hge_fixture_env`, which creates a "soft" dependency between the environment variables and `hge_server`. Parameterizing the former *should* force the latter to be recreated for each new set of environment variables, but `hge_server` isn't actually aware there's a dependency.

It currently looks like this adds lines of code; we'll more than make up for it when we delete the relevant lines from *test-server.sh*. I am not doing that here because I plan on deleting the whole file in a subsequent changeset.

[NDAT-538]: https://hasurahq.atlassian.net/browse/NDAT-538?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8803
GitOrigin-RevId: f7f2caa62de0b0a45e42964b69a8ae73d1575fe8
2023-04-19 10:30:21 +00:00
Samir Talwar
0ea8bd8df8 server/tests-py: Parallelize test_logging.py.
This adds the ability to capture logs to the HGE fixture, and uses this in test_logging.py to analyze the logs, instead of relying on a shell script redirecting the logs to a file.

We then inject the logs into the tests and parse the JSON. Because we're no longer reading a file, we need to do this in a separate thread, as we'll block on reading rather than the stream ending. (Once HGE stops, the stream will be closed.)

Some of the tests require a JWK server, so this has been extracted from test_jwk.py.

[NDAT-540]: https://hasurahq.atlassian.net/browse/NDAT-540?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8772
GitOrigin-RevId: 9413e714f1c42b8a0991d0d30c4358209fd30c0c
2023-04-17 13:09:34 +00:00
Samir Talwar
5677446ff3 server/tests-py: Run test_websocket_init_cookie.py in parallel.
This requires rewriting the test class to split it into 3, each specifying the correct environment variables for HGE.

It would be lovely to use parameterization rather than subclassing, but that doesn't work because of `hge_fixture_env`, which creates a "soft" dependency between the environment variables and `hge_server`. Parameterizing the former *should* force the latter to be recreated for each new set of environment variables, but `hge_server` isn't actually aware there's a dependency. See `TestParameterizedFixtures` in test_tests.py for more information.

[NDAT-539]: https://hasurahq.atlassian.net/browse/NDAT-539?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8747
GitOrigin-RevId: 878b2fc20f39f962a67cd950046a99c283cfc6fc
2023-04-12 16:26:22 +00:00
Samir Talwar
43f179688c server/tests-py: Run test_graphql_read_only_source.py in parallel.
This requires making it add its own source, rather than expecting one to be provided.

[NDAT-537]: https://hasurahq.atlassian.net/browse/NDAT-537?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8745
GitOrigin-RevId: 65c818e4f065e9f98e9cfd65c158eff5d768ae31
2023-04-12 14:37:21 +00:00
Samir Talwar
23f78a05b1 server/tests-py: Enable TLS tests in parallel mode on CI.
Part of [NDAT-257](https://hasurahq.atlassian.net/browse/NDAT-257).

* Run the TLS tests on CI.
* Run non-TLS tests even when a TLS certificate is provided; just ignore it.

[NDAT-257]: https://hasurahq.atlassian.net/browse/NDAT-257?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7806
GitOrigin-RevId: d705e2e5131a674070a98a893323a7b98a92f688
2023-02-08 14:42:27 +00:00
Samir Talwar
fa53a70631 server/tests-py: Run TestSubscriptionBasicNoAuth in the new mode.
This test _cannot_ run without an admin secret, so we need a new marker just for that.

This also fixes the Docker Compose file so that _run.sh_ works again. Apparently it was not used much.

[NDAT-257]: https://hasurahq.atlassian.net/browse/NDAT-257?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7831
GitOrigin-RevId: 1cb9449a20c7cbe56fbd1ba0f12337f056683624
2023-02-08 11:51:54 +00:00
Samir Talwar
204ec89c61 server/tests-py: Get all tests passing with separate HGE binaries.
This rewrites the last couple of Python tests that were failing when run with a separate HGE binary per test class. The changes are as follows:

1. The event triggers tests, naming conventions tests, and subscriptions tests all generate a new source DB per test, so can run in parallel.
2. The scheduled triggers tests use the correct URL for the trigger service when the port is generated randomly.
3. Whitespace and trailing commas are added to the scheduled triggers tests.
4. Support for SQL Server is added to _hge.py_ so the naming conventions test that runs on SQL Server passes. (The other SQL Server tests do not pass and we're not going to bother with them for now.)
5. Container names are fixed in _run.sh_.
6. _run.sh_ and _run-new.sh_ don't pull images explicitly as it's annoying when running tests a lot. If you want to pull the latest versions, just run `docker compose pull` from the _server/tests-py_ directory, or the root directory. (If you don't have the images at all, they'll still be pulled automatically.)

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7350
GitOrigin-RevId: db58f310f017b2a0884fcf61ccc56d15583f99bd
2022-12-21 15:56:41 +00:00
Philip Lykke Carlsen
bd2e2080f8 server/tests: Port TestGraphQLQueryFunctions
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7158
GitOrigin-RevId: a4b3ca8565df37905665303f96fd40ec4b9e76dd
2022-12-15 16:42:55 +00:00
Samir Talwar
beb20c7dab server/tests-py: Remove a check for PostgreSQL 10.
We no longer support this and therefore don't run tests against it.

This also refactors the code a little so it doesn't have to skip running a PostgreSQL-specific test against MS SQL Server.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7201
GitOrigin-RevId: 307c2ab0052162c012f7b1c55866b57f2fa6d9a6
2022-12-14 16:57:28 +00:00
Philip Lykke Carlsen
886302fd7e server/tests: Introuce pytest-to-haskell porting script
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7054
GitOrigin-RevId: 2c4597465da28d26993b347693813ce7d42962d3
2022-12-02 15:59:19 +00:00
Samir Talwar
7c452bfca1 server/tests-py: Spawn a new PostgreSQL schema ("database") per test.
When running using the "new" style (with a HGE binary, not a URL), a new PostgreSQL metadata and source database are created for each test. When we get this into CI, this should drastically reduce the flakiness.

I have also enabled parallelization by default when using `run-new.sh`. It's much faster.

I had to basically rewrite _server/tests-py/test_graphql_read_only_source.py_ so that it does two different things depending on how it's run. It's unfortunate, but it should eventually go away.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6879
GitOrigin-RevId: a121b9035f8da3e61a3e36d8b1fbc6ccae918fad
2022-11-15 19:08:53 +00:00
Samir Talwar
6a81bdd0b9 server/tests-py: Generate random ports for a few more helper services.
I missed these.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6481
GitOrigin-RevId: 8bc92c27c8de25bc3518be5b788e7b5c55867df8
2022-10-27 11:49:06 +00:00
Samir Talwar
133a9b7452 server/tests-py: Generate random ports for helper services.
If the tests are run with specific ports assigned to specific services,
set through the environment variables, we continue to use those ports.
We just don't hard-code them now, we pick them up from the environment
variables.

However, if the environment variables are not set, we generate a random
port for each service. This allows us to run multiple tests in parallel
in the future, independently.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6218
GitOrigin-RevId: 3d2a1880bf67544c848951888ce7b4fa1ba379dc
2022-10-21 17:34:21 +00:00
Samir Talwar
c2cb07f7e8 server/tests-py: Start webhook.py inside the test harness.
We use a helper service to start a webhook-based authentication service for some tests. This moves the initialization of the service out of _test-server.sh_ and into the Python test harness, as a fixture.

In order to do this, I had to make a few changes. The main deviation is that we no longer run _all_ tests against an HGE with this authentication service, just a few (those in _test_webhook.py_). Because this reduced coverage, I have added some more tests there, which actually cover some areas not exacerbated elsewhere (mainly trying to use webhook credentials to talk to an admin-only endpoint).

The webhook service can run both with and without TLS, and decide whether it's necessary to skip one of these based on the arguments passed and how HGE is started, according to the following logic:

* If a TLS CA certificate is passed in, it will run with TLS, otherwise it will skip it.
* If HGE was started externally and a TLS certificate is provided, it will skip running without TLS, as it will assume that HGE was configured to talk to a webhook over HTTPS.
* Some tests should only be run with TLS; this is marked with a `tls_webhook_server` marker.
* Some tests should only be run _without_ TLS; this is marked with a `no_tls_webhook_server` marker.

The actual parameterization of the webhook service configuration is done through test subclasses, because normal pytest parameterization doesn't work with the `hge_fixture_env` hack that we use. Because `hge_fixture_env` is not a sanctioned way of conveying data between fixtures (and, unfortunately, there isn't a sanctioned way of doing this when the fixtures in question may not know about each other directly), parameterizing the `webhook_server` fixture doesn't actually parameterize `hge_server` properly. Subclassing forces this to work correctly.

The certificate generation is moved to a Python fixture, so that we don't have to revoke the CA certificate for _test_webhook_insecure.py_; we can just generate a bogus certificate instead. The CA certificate is still generated in the _test-server.sh_ script, as it needs to be installed into the OS certificate store.

Interestingly, the CA certificate installation wasn't actually working, because the certificates were written to the wrong location. This didn't cause any failures, as we weren't actually testing this behavior. This is now fixed with the other changes.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6363
GitOrigin-RevId: 0f277d374daa64f657257ed2a4c2057c74b911db
2022-10-20 19:00:01 +00:00
Samir Talwar
0d4d7e6b1e server/tests-py: Start helper services in the test harness.
This makes it possible for the test harness to start the test JWK server and the test remote schema server.

In order to do this, we still generate the TLS certificates in the test script (because we need to install the generated CA certificate in the OS certificate store), and then pass the certificate and key paths into the test runner.

Because we are still using _test-server.sh_ for now, we don't use the JWK server fixture in that case, as HGE needs the JWK server to be up and running when it starts. Instead, we keep running it outside (for now).

This is also the case for the GraphQL server fixture when we are running the server upgrade/downgrade tests.

I have also refactored _graphql_server.py_ so there isn't a global `HGE_URLS` value, but instead the value is passed through.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6303
GitOrigin-RevId: 06f05ff674372dc5d632e55d68e661f5c7a17c10
2022-10-13 15:45:24 +00:00
Samir Talwar
e78dc7f4eb server/tests-py: Remove checks for PostgreSQL 9.x.
We no longer run tests against these PostgreSQL versions.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6320
GitOrigin-RevId: 175ec5bd5db5fe0a96733f948a10b141aa40ba09
2022-10-12 14:27:48 +00:00
Samir Talwar
3cb9bab9f1 server/tests-py: Provide the admin secret to the HGE server.
When we run the HGE server inside the test harness, it needs to run with
an admin secret for some tests to make sense. This tags each test that
requires an admin secret with `pytest.mark.admin_secret`, which then
generates a UUID and injects that into both the server and the test case
(if required).

It also simplifies the way the test harness picks up an existing admin
secret, allowing it to use the environment variable instead of requiring
it via a parameter.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6120
GitOrigin-RevId: 55c5b9e8c99bdad9c8304098444ddb9516749a2c
2022-09-29 17:20:07 +00:00
Samir Talwar
60f81023db server/tests-py: Run the auth hook inside the test harness.
This teaches `hge_server` how to run more tests, thanks to `hge_env`.

It also simplifies the logic a bit more.

I have also modified _run.sh_ and _docker-compose.yml_ so we can run multiple test suites, one after another.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6105
GitOrigin-RevId: eff009362eb6bb90c07cedaf96dfe6ec9336ff32
2022-09-29 10:44:03 +00:00
Samir Talwar
8cb2738cbe server/tests-py: Declaratively state the HGE environment variables.
This has two purposes:

* When running the Python integration tests against a running HGE instance, with `--hge-url`, it will check the environment variables available and actively skip the test if they aren't set. This replaces the previous ad-hoc skip behavior.
* More interestingly, when running against a binary with `--hge-bin`, the environment variables are passed through, which means different tests can run with different environment variables.

  On top of this, the various services we use for testing now also provide their own environment variables, rather than expecting a test script to do it.

In order to make this work, I also had to invert the dependency between various services and `hge_ctx`. I extracted a `pg_version` fixture to provide the PostgreSQL version, and now pass the `hge_url` and `hge_key` explicitly to `ActionsWebhookServer`.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6028
GitOrigin-RevId: 16d866741dba5887da1adf4e1ade8182ccc9d344
2022-09-28 09:21:02 +00:00
Samir Talwar
1a5aaae9cf server/tests-py: Add a --hge-bin argument to the Pytest runner.
This argument allows the user to specify how to run HGE, rather than starting it beforehand. The runner will start a new instance of HGE for each test class.

This does not provide isolation, as the database is still re-used, but it helps us get closer.

You can try it yourself by executing:

```
$ cabal build graphql-engine:exe:graphql-engine
$ ./server/tests-py/run-new.sh
```

This doesn't affect CI at all.

I also fixed a few warnings flagged by Pylance.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5881
GitOrigin-RevId: ea6f0fd631a2c278b2c6b50e9dbdd9d804ebc9d4
2022-09-15 12:31:12 +00:00
Samir Talwar
bff1309344 server/tests-py: Make HGECtxGQLServer a fixture.
Starting it and stopping it for the various tests that actually use it.

There are only a few.

This also removes some dead code and fixes warnings in _test_webhook_request_context.py_.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5846
GitOrigin-RevId: 7760467f9de7b1f9718e7482275c298eeaa3ad3a
2022-09-14 21:42:40 +00:00
Samir Talwar
38c94d8cda server/tests-py: Fix some warnings.
They're getting in the way of real problems.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5541
GitOrigin-RevId: 9bc99a0826715f6df77055cca79e428ad49742a8
2022-08-24 08:32:07 +00:00
Samir Talwar
a0176c215f server/tests-py: Set up postgis extensions using a fixture.
Let's put it in one place.

This is a precursor to moving database provisioning into the Python
integration tests.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5453
GitOrigin-RevId: 5920b0b1177d94496485fcb4e178b946534ee4eb
2022-08-15 14:30:42 +00:00
Samir Talwar
5df8419a4f server/tests-py: Use markers for backends instead of parameterized tests.
I'm trying to shore up the Python integration tests to make them more reliable. In doing so, I noticed this.

---

It feels a lot more sensible as we never run on more than one backend at a time.

This also removes the `check_file_exists` parameter from the setup functions; it never worked. It was always set to the result of a comparison between a backend name and a function, which was always `False`. Enabling it breaks things.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5254
GitOrigin-RevId: 8718ab21527c2ba0a7205d1c01ebaac1a10be844
2022-08-02 19:33:59 +00:00
Samir Talwar
987b55f981 server/tests-py: Reduce the number of locations we check the status code.
We have a lot of `assert st_code == 200` scattered about. This is a
problem because (a) it makes the code harder to parse and (b) the error
message is lacking; I have seen a few flaky tests which were impossible
to diagnose because I didn't know what the response _should_ be.

This reduces the number of places in which we perform this assertion
(moving most of them to `HGECtx.execute_query`), so that we can have a
better chance of seeing a useful error message on test failure.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4957
GitOrigin-RevId: 3ff388bccf49f96569aa6b7db85266a0c5ee27ea
2022-07-05 18:01:07 +00:00
Samir Talwar
75c01223a8 server: Use plain assertions in pytest.
When pytest rewrites assertions to make them more useful, it also
truncates long assertion messages in the middle, often obscuring the
actual error.

Disabling this allows us to see the full message, which should hopefully
allow us to see the actual error.

In order to support this, we need to make sure that custom assertion
messages include the actual assertion information, as this will no
longer be rendered by pytest.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4835
GitOrigin-RevId: de6839a3b40d0edc7cb96e46625eebca0aaf0c95
2022-06-25 00:04:16 +00:00
Karthikeyan Chinnakonda
2325755954 server: streaming subscriptions schema generation and tests (incremental PR - 3)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4259
Co-authored-by: Rikin Kachhia <54616969+rikinsk@users.noreply.github.com>
Co-authored-by: Brandon Simmons <210815+jberryman@users.noreply.github.com>
Co-authored-by: paritosh-08 <85472423+paritosh-08@users.noreply.github.com>
GitOrigin-RevId: 4d1b4ec3c01f3a839f4392d3b77950fc3ab30236
2022-04-22 19:54:11 +00:00
Anon Ray
e87433c2bb server: remove hdb_lib and add tests for read-only source
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3634
Co-authored-by: Philip Lykke Carlsen <358550+plcplc@users.noreply.github.com>
GitOrigin-RevId: 2db62a279496cd7e5dd57bdf02c3efa7b70042c8
2022-02-21 10:00:00 +00:00
paritosh-08
4d2a5dba51 server: fix multiple calls to database on startup
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3569
GitOrigin-RevId: 820d56a7af7faa05292ef65734d5d40fe3dd4641
2022-02-09 05:14:29 +00:00
Vamshi Surabhi
2e1be1d2c5 bump python test suite dependencies
This PR upgrades some of the pinned dependencies do not build with python 3.10 - cffi, ruamel, py. Further, it upgrades other packages where the effort is minimal.

For the reviewers: Please review it commit by commit.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3367
GitOrigin-RevId: c5401fe289d3185a79c4d382297f86fbde139825
2022-01-17 07:40:56 +00:00
pranshi06
dee86453ea server: fallback to unauthorised role when JWT is not found in cookie
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2882
GitOrigin-RevId: ad03fbd0572e00ffe7abea106388a4df4d12af2c
2021-12-08 18:29:29 +00:00
Puru Gupta
504f13725f server: forward auth webhook set-cookies header on response
>

High-Level TODO:

* [x] Code Changes
* [x] Tests
* [x] Check that pro/multitenant build ok
* [x] Documentation Changes
* [x] Updating this PR with full details
* [ ] Reviews
* [ ] Ensure code has all FIXMEs and TODOs addressed
* [x] Ensure no files are checked in mistakenly
* [x] Consider impact on console, cli, etc.

### Description
>
This PR adds support for adding set-cookie header on the response from the auth webhook. If the set-cookie header is sent by the webhook, it will be forwarded in the graphQL engine response.

Fixes a bug in test-server.sh: testing of get-webhook tests was done by POST method and vice versa. To fix, the parameters were swapped.

### Changelog

- [x] `CHANGELOG.md` is updated with user-facing content relevant to this PR.

### Affected components

- [x] Server
- [ ] Console
- [ ] CLI
- [x] Docs
- [ ] Community Content
- [ ] Build System
- [x] Tests
- [ ] Other (list it)

### Related Issues
->
Closes [#2269](https://github.com/hasura/graphql-engine/issues/2269)

### Solution and Design
>

### Steps to test and verify
>
Please refer to the docs to see how to send the set-cookie header from webhook.

### Limitations, known bugs & workarounds
>
- Support for only set-cookie header forwarding is added
- the value forwarded in the set-cookie header cannot be validated completely, the [Cookie](https://hackage.haskell.org/package/cookie) package has been used to parse the header value and any unnecessary information is stripped off before forwarding the header. The standard given in [RFC6265](https://datatracker.ietf.org/doc/html/rfc6265) has been followed for the Set-Cookie format.

### Server checklist

#### Catalog upgrade

Does this PR change Hasura Catalog version?
- [x] No
- [ ] Yes
  - [ ] Updated docs with SQL for downgrading the catalog

#### Metadata

Does this PR add a new Metadata feature?
- [x] No

#### GraphQL
- [x] No new GraphQL schema is generated
- [ ] New GraphQL schema is being generated:
   - [ ] New types and typenames are correlated

#### Breaking changes

- [x] No Breaking changes

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2538
Co-authored-by: Robert <132113+robx@users.noreply.github.com>
GitOrigin-RevId: d9047e997dd221b7ce4fef51911c3694037e7c3f
2021-11-09 12:01:31 +00:00
David Overton
aac64f2c81 Source typename customization (close graphql-engine#6974)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/1616
GitOrigin-RevId: f7eefd2367929209aa77895ea585e96a99a78d47
2021-10-29 14:43:14 +00:00
Abby Sassel
a9e3a1b4bd server/bigquery: improve test setup and documentation (close #2345)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2495
GitOrigin-RevId: ef250a0796200be6b75363b4275eb8a7ae780154
2021-10-12 17:59:44 +00:00
Rakesh Emmadi
09ae6af337 server/mssql: insert SQL generation and execution
>

### Description
>
Insert mutations for MSSQL backend. This PR implements execution logic.

### Changelog

- [x] `CHANGELOG.md` is updated with user-facing content relevant to this PR. If no changelog is required, then add the `no-changelog-required` label.

### Affected components

- [x] Server
- [x] Tests

### Related Issues
->
Close https://github.com/hasura/graphql-engine-mono/issues/2114

### Steps to test and verify
>
Track a MSSQL table and perform the generated insert mutation to test.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2248
Co-authored-by: Abby Sassel <3883855+sassela@users.noreply.github.com>
Co-authored-by: Philip Lykke Carlsen <358550+plcplc@users.noreply.github.com>
GitOrigin-RevId: 936f138c80d7a928180e6e7b0c4da64ecc1f7ebc
2021-10-01 12:53:15 +00:00
Antoine Leblanc
9bae641ac4 Always enable inherited roles tests
### Description

The inherited roles integration tests were behind a flag, and its corresponding fixture, presumably to avoid enabling the option globally. However, #2288 introduced a new test using inherited roles that was not gated behind the flag, which fails when run with `dev.sh`. However, that test works on CI... because inherited roles are globally enabled there.

Consequently, this PR:
- globally enables inherited roles in dev.sh
- removes the flag and the associated fixture

https://github.com/hasura/graphql-engine-mono/pull/2358

Co-authored-by: Vishnu Bharathi <4211715+scriptnull@users.noreply.github.com>
GitOrigin-RevId: ebfa6754873324bed15b2cc5e37ec2d8008e8f8d
2021-09-16 06:24:54 +00:00
Sameer Kolhar
edeb8c98fd server: support for graphql-ws protocol
https://github.com/hasura/graphql-engine-mono/pull/1655

Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Vijay Prasanna <11921040+vijayprasanna13@users.noreply.github.com>
Co-authored-by: hasura-bot <30118761+hasura-bot@users.noreply.github.com>
Co-authored-by: Brandon Simmons <210815+jberryman@users.noreply.github.com>
Co-authored-by: Varun Choudhary <68095256+Varun-Choudhary@users.noreply.github.com>
Co-authored-by: Divi <32202683+imperfect-fourth@users.noreply.github.com>
GitOrigin-RevId: 9db3902388fef06b94f9513255e2b5333bd23c3e
2021-08-24 16:26:12 +00:00
Karthikeyan Chinnakonda
06f5e4fb77 server: inherited roles for mutations, remote schemas, actions and custom functions
https://github.com/hasura/graphql-engine-mono/pull/1715

GitOrigin-RevId: 4818292cff8c3a5b264968e7032887a1e98b6f79
2021-08-09 10:21:05 +00:00
Solomon Bothwell
d88e2bbcce server: add tests ensuring the correct functioning of all endpoints based on user roles
https://github.com/hasura/graphql-engine-mono/pull/1625

Co-authored-by: Sameer Kolhar <6604943+kolharsam@users.noreply.github.com>
GitOrigin-RevId: 6b56efc838d2ed1acc44b2847161fde22d6aee17
2021-07-16 16:09:25 +00:00
Karthikeyan Chinnakonda
43973f80cf server tests: refactor the per_class_tests_db_state fixture
https://github.com/hasura/graphql-engine-mono/pull/1701

GitOrigin-RevId: 278b9e226cc7719f4c44bf9c9a802146a8a004f4
2021-07-01 09:21:45 +00:00
Abby Sassel
41908936cf server: improve backend-specific test setup
GitOrigin-RevId: c40393bcdd78feaba7e9490ce4ed78de19b5bfc5
2021-05-25 13:55:11 +00:00
Karthikeyan Chinnakonda
4702ba514a pro server: disable GraphQL schema introspection for specified roles
GitOrigin-RevId: 36ceb1852bc7779092cfb59c92035dd9635b2e3f
2021-05-05 12:26:23 +00:00
kodiakhq[bot]
a935746e17 Integration test improvements (for speed/clarity), also increase polling interval for scheduled events
This claws back ~7min from integration tests (run serially, as with `dev.sh test --integration`
Further improvements would do well to focus on optimizing metadata operations, as `setup` dominates

GitOrigin-RevId: 76637d6fa953c2404627c4391447a05bf09355fa
2021-04-27 05:35:26 +00:00
Abby Sassel
64d52f5fa3 server: generalize integration tests
Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
GitOrigin-RevId: 647edb8d293bf783c504b7e15ce02c56858b3b72
2021-03-11 18:18:41 +00:00
Karthikeyan Chinnakonda
92026b769f [Preview] Inherited roles for postgres read queries
fixes #3868

docker image - `hasura/graphql-engine:inherited-roles-preview-48b73a2de`

Note:

To be able to use the inherited roles feature, the graphql-engine should be started with the env variable `HASURA_GRAPHQL_EXPERIMENTAL_FEATURES` set to `inherited_roles`.

Introduction
------------

This PR implements the idea of multiple roles as presented in this [paper](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/FGALanguageICDE07.pdf). The multiple roles feature in this PR can be used via inherited roles. An inherited role is a role which can be created by combining multiple singular roles. For example, if there are two roles `author` and `editor` configured in the graphql-engine, then we can create a inherited role with the name of `combined_author_editor` role which will combine the select permissions of the `author` and `editor` roles and then make GraphQL queries using the `combined_author_editor`.

How are select permissions of different roles are combined?
------------------------------------------------------------

A select permission includes 5 things:

1. Columns accessible to the role
2. Row selection filter
3. Limit
4. Allow aggregation
5. Scalar computed fields accessible to the role

 Suppose there are two roles, `role1` gives access to the `address` column with row filter `P1` and `role2` gives access to both the `address` and the `phone` column with row filter `P2` and we create a new role `combined_roles` which combines `role1` and `role2`.

Let's say the following GraphQL query is queried with the `combined_roles` role.

```graphql
query {
   employees {
     address
     phone
   }
}
```

This will translate to the following SQL query:

```sql

 select
    (case when (P1 or P2) then address else null end) as address,
    (case when P2 then phone else null end) as phone
 from employee
 where (P1 or P2)
```

The other parameters of the select permission will be combined in the following manner:

1. Limit - Minimum of the limits will be the limit of the inherited role
2. Allow aggregations - If any of the role allows aggregation, then the inherited role will allow aggregation
3. Scalar computed fields - same as table column fields, as in the above example

APIs for inherited roles:
----------------------

1. `add_inherited_role`

`add_inherited_role` is the [metadata API](https://hasura.io/docs/1.0/graphql/core/api-reference/index.html#schema-metadata-api) to create a new inherited role. It accepts two arguments

`role_name`: the name of the inherited role to be added (String)
`role_set`: list of roles that need to be combined (Array of Strings)

Example:

```json
{
  "type": "add_inherited_role",
  "args": {
      "role_name":"combined_user",
      "role_set":[
          "user",
          "user1"
      ]
  }
}
```

After adding the inherited role, the inherited role can be used like single roles like earlier

Note:

An inherited role can only be created with non-inherited/singular roles.

2. `drop_inherited_role`

The `drop_inherited_role` API accepts the name of the inherited role and drops it from the metadata. It accepts a single argument:

`role_name`: name of the inherited role to be dropped

Example:

```json

{
  "type": "drop_inherited_role",
  "args": {
      "role_name":"combined_user"
  }
}
```

Metadata
---------

The derived roles metadata will be included under the `experimental_features` key while exporting the metadata.

```json
{
  "experimental_features": {
    "derived_roles": [
      {
        "role_name": "manager_is_employee_too",
        "role_set": [
          "employee",
          "manager"
        ]
      }
    ]
  }
}
```

Scope
------

Only postgres queries and subscriptions are supported in this PR.

Important points:
-----------------

1. All columns exposed to an inherited role will be marked as `nullable`, this is done so that cell value nullification can be done.

TODOs
-------

- [ ] Tests
   - [ ] Test a GraphQL query running with a inherited role without enabling inherited roles in experimental features
   - [] Tests for aggregate queries, limit, computed fields, functions, subscriptions (?)
   - [ ] Introspection test with a inherited role (nullability changes in a inherited role)
- [ ] Docs
- [ ] Changelog

Co-authored-by: Vamshi Surabhi <6562944+0x777@users.noreply.github.com>
GitOrigin-RevId: 3b8ee1e11f5ceca80fe294f8c074d42fbccfec63
2021-03-08 11:15:10 +00:00
Phil Freeman
7fffc11077 Caching, Rate Limiting, Metrics & Session Variable Improvements (#376)
* server: use a leaky bucket algorithm for bytes-per-second cache rate limiting

* Use evalsha properly

* Adds redis cache limit parameters to PoliciesConfig

* Loads Leaky Bucket Script On Server Start

* Adds more redis logging and moves cache update into lua script

* reverts setex in lua and adds notes

* Refactors cacheStore and adds max TTL and cache size limits

* Filter session vars in cache key

* WIP

* parens

* cache-clear-hander POC implementation

* cache-clear-hander POC implementation

* Pro projectId used as cache key

* POC working!

* prefixing query-response keys in redis

* Add cacheClearer to RedisScripts

* Partial implementation of cacheClearer from scripts record

* updating tests

* [automated] stylish-haskell commit

* Adds query look with up with metrics script

* Adds missing module and lua script from last commit

* Changes redis script module structure to match cache clearing branch

* minor change to lua script

* cleaning up cache clearing

* generalising JsonLog

* [automated] stylish-haskell commit

* Draft Cache Metrics Endpoint

* Adds Cache Metrics Handler

* Adds hook handler module

* Missed HandlerHook module in last commit

* glob

* Fixes redis mget bug

* Removes cache totals and changes dashes to colons in metric cache keys

* Adds query param to clear clear endpoint for deleting specific keys

* Adds query param to clear clear endpoint for deleting specific keys

* Cache Metrics on query families rather then queries

* Replace Set with nub

* Base16 Redis Hashes

* Query Family Redis Keys With Roles

* response headers for cache keys

* fixing bug in family key by excluding operation name; using hash for response header instead of entire key

* Adds query family to redis cache keys and cache clear endpoint

* Fixes queryfamily hash bug

* Moves cache endpoints to /pro

* Moved cache clear to POST

* Refactors cache clear function

* Fixes query family format bug

* Adds query cache tests and optional --redis-url flag to python test suite

* Adds session variable cache test

* Update pro changelog

* adding documentation for additional caching features

* more docs

* clearing up units of leaky bucket params

* Adds comments to leaky bucket script

* removes old todo

* Fixes session variable filtering to work with new query rootfield

* more advanced defaulting behaviour for bucket rate and capacity.

* Updates Docs

* Moves Role into QueryFamily hash

* Use Aeson for Cache Clear endpoint response

* Moves trace to bracket the leaky bucket script

* Misc review tweaks

* Adds sum type for cache clear query params

* Hardcodes RegisReplyLog log level

* Update docs/graphql/cloud/response-caching.rst

Co-authored-by: Phil Freeman <phil@hasura.io>

* new prose for rate limiting docs

* [automated] stylish-haskell commit

* make rootToSessVarPreds total

* [automated] stylish-haskell commit

* Fixes out of scope error

* Renamed _acRedis to _acCacheStore

Co-authored-by: Solomon Bothwell <ssbothwell@gmail.com>
Co-authored-by: Lyndon Maydwell <lyndon@sordina.net>
Co-authored-by: David Overton <david@hasura.io>
Co-authored-by: Stylish Haskell Bot <stylish-haskell@users.noreply.github.com>
Co-authored-by: Lyndon Maydwell <lyndon@hasura.io>
GitOrigin-RevId: dda5c1a3f902967b3d78310f950541a55fabb1b0
2021-02-13 00:06:18 +00:00
Swann Moreau
c14dcd5792 pass gql requests into auth webhook POST body (#149)
* fix arg order in UserAuthentication instance [force ci]

* change the constructor name to AHGraphQLRequest

Co-authored-by: Stylish Haskell Bot <stylish-haskell@users.noreply.github.com>
Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
GitOrigin-RevId: fb3258f4a84efc6c730b0c6222ebd8cea1b91081
2021-02-03 07:11:39 +00:00