Commit Graph

68 Commits

Author SHA1 Message Date
Samir Talwar
204ec89c61 server/tests-py: Get all tests passing with separate HGE binaries.
This rewrites the last couple of Python tests that were failing when run with a separate HGE binary per test class. The changes are as follows:

1. The event triggers tests, naming conventions tests, and subscriptions tests all generate a new source DB per test, so can run in parallel.
2. The scheduled triggers tests use the correct URL for the trigger service when the port is generated randomly.
3. Whitespace and trailing commas are added to the scheduled triggers tests.
4. Support for SQL Server is added to _hge.py_ so the naming conventions test that runs on SQL Server passes. (The other SQL Server tests do not pass and we're not going to bother with them for now.)
5. Container names are fixed in _run.sh_.
6. _run.sh_ and _run-new.sh_ don't pull images explicitly as it's annoying when running tests a lot. If you want to pull the latest versions, just run `docker compose pull` from the _server/tests-py_ directory, or the root directory. (If you don't have the images at all, they'll still be pulled automatically.)

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7350
GitOrigin-RevId: db58f310f017b2a0884fcf61ccc56d15583f99bd
2022-12-21 15:56:41 +00:00
Philip Lykke Carlsen
bd2e2080f8 server/tests: Port TestGraphQLQueryFunctions
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7158
GitOrigin-RevId: a4b3ca8565df37905665303f96fd40ec4b9e76dd
2022-12-15 16:42:55 +00:00
Samir Talwar
beb20c7dab server/tests-py: Remove a check for PostgreSQL 10.
We no longer support this and therefore don't run tests against it.

This also refactors the code a little so it doesn't have to skip running a PostgreSQL-specific test against MS SQL Server.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7201
GitOrigin-RevId: 307c2ab0052162c012f7b1c55866b57f2fa6d9a6
2022-12-14 16:57:28 +00:00
Philip Lykke Carlsen
886302fd7e server/tests: Introuce pytest-to-haskell porting script
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7054
GitOrigin-RevId: 2c4597465da28d26993b347693813ce7d42962d3
2022-12-02 15:59:19 +00:00
Samir Talwar
7c452bfca1 server/tests-py: Spawn a new PostgreSQL schema ("database") per test.
When running using the "new" style (with a HGE binary, not a URL), a new PostgreSQL metadata and source database are created for each test. When we get this into CI, this should drastically reduce the flakiness.

I have also enabled parallelization by default when using `run-new.sh`. It's much faster.

I had to basically rewrite _server/tests-py/test_graphql_read_only_source.py_ so that it does two different things depending on how it's run. It's unfortunate, but it should eventually go away.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6879
GitOrigin-RevId: a121b9035f8da3e61a3e36d8b1fbc6ccae918fad
2022-11-15 19:08:53 +00:00
Samir Talwar
6a81bdd0b9 server/tests-py: Generate random ports for a few more helper services.
I missed these.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6481
GitOrigin-RevId: 8bc92c27c8de25bc3518be5b788e7b5c55867df8
2022-10-27 11:49:06 +00:00
Samir Talwar
133a9b7452 server/tests-py: Generate random ports for helper services.
If the tests are run with specific ports assigned to specific services,
set through the environment variables, we continue to use those ports.
We just don't hard-code them now, we pick them up from the environment
variables.

However, if the environment variables are not set, we generate a random
port for each service. This allows us to run multiple tests in parallel
in the future, independently.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6218
GitOrigin-RevId: 3d2a1880bf67544c848951888ce7b4fa1ba379dc
2022-10-21 17:34:21 +00:00
Samir Talwar
c2cb07f7e8 server/tests-py: Start webhook.py inside the test harness.
We use a helper service to start a webhook-based authentication service for some tests. This moves the initialization of the service out of _test-server.sh_ and into the Python test harness, as a fixture.

In order to do this, I had to make a few changes. The main deviation is that we no longer run _all_ tests against an HGE with this authentication service, just a few (those in _test_webhook.py_). Because this reduced coverage, I have added some more tests there, which actually cover some areas not exacerbated elsewhere (mainly trying to use webhook credentials to talk to an admin-only endpoint).

The webhook service can run both with and without TLS, and decide whether it's necessary to skip one of these based on the arguments passed and how HGE is started, according to the following logic:

* If a TLS CA certificate is passed in, it will run with TLS, otherwise it will skip it.
* If HGE was started externally and a TLS certificate is provided, it will skip running without TLS, as it will assume that HGE was configured to talk to a webhook over HTTPS.
* Some tests should only be run with TLS; this is marked with a `tls_webhook_server` marker.
* Some tests should only be run _without_ TLS; this is marked with a `no_tls_webhook_server` marker.

The actual parameterization of the webhook service configuration is done through test subclasses, because normal pytest parameterization doesn't work with the `hge_fixture_env` hack that we use. Because `hge_fixture_env` is not a sanctioned way of conveying data between fixtures (and, unfortunately, there isn't a sanctioned way of doing this when the fixtures in question may not know about each other directly), parameterizing the `webhook_server` fixture doesn't actually parameterize `hge_server` properly. Subclassing forces this to work correctly.

The certificate generation is moved to a Python fixture, so that we don't have to revoke the CA certificate for _test_webhook_insecure.py_; we can just generate a bogus certificate instead. The CA certificate is still generated in the _test-server.sh_ script, as it needs to be installed into the OS certificate store.

Interestingly, the CA certificate installation wasn't actually working, because the certificates were written to the wrong location. This didn't cause any failures, as we weren't actually testing this behavior. This is now fixed with the other changes.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6363
GitOrigin-RevId: 0f277d374daa64f657257ed2a4c2057c74b911db
2022-10-20 19:00:01 +00:00
Samir Talwar
0d4d7e6b1e server/tests-py: Start helper services in the test harness.
This makes it possible for the test harness to start the test JWK server and the test remote schema server.

In order to do this, we still generate the TLS certificates in the test script (because we need to install the generated CA certificate in the OS certificate store), and then pass the certificate and key paths into the test runner.

Because we are still using _test-server.sh_ for now, we don't use the JWK server fixture in that case, as HGE needs the JWK server to be up and running when it starts. Instead, we keep running it outside (for now).

This is also the case for the GraphQL server fixture when we are running the server upgrade/downgrade tests.

I have also refactored _graphql_server.py_ so there isn't a global `HGE_URLS` value, but instead the value is passed through.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6303
GitOrigin-RevId: 06f05ff674372dc5d632e55d68e661f5c7a17c10
2022-10-13 15:45:24 +00:00
Samir Talwar
e78dc7f4eb server/tests-py: Remove checks for PostgreSQL 9.x.
We no longer run tests against these PostgreSQL versions.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6320
GitOrigin-RevId: 175ec5bd5db5fe0a96733f948a10b141aa40ba09
2022-10-12 14:27:48 +00:00
Samir Talwar
3cb9bab9f1 server/tests-py: Provide the admin secret to the HGE server.
When we run the HGE server inside the test harness, it needs to run with
an admin secret for some tests to make sense. This tags each test that
requires an admin secret with `pytest.mark.admin_secret`, which then
generates a UUID and injects that into both the server and the test case
(if required).

It also simplifies the way the test harness picks up an existing admin
secret, allowing it to use the environment variable instead of requiring
it via a parameter.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6120
GitOrigin-RevId: 55c5b9e8c99bdad9c8304098444ddb9516749a2c
2022-09-29 17:20:07 +00:00
Samir Talwar
60f81023db server/tests-py: Run the auth hook inside the test harness.
This teaches `hge_server` how to run more tests, thanks to `hge_env`.

It also simplifies the logic a bit more.

I have also modified _run.sh_ and _docker-compose.yml_ so we can run multiple test suites, one after another.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6105
GitOrigin-RevId: eff009362eb6bb90c07cedaf96dfe6ec9336ff32
2022-09-29 10:44:03 +00:00
Samir Talwar
8cb2738cbe server/tests-py: Declaratively state the HGE environment variables.
This has two purposes:

* When running the Python integration tests against a running HGE instance, with `--hge-url`, it will check the environment variables available and actively skip the test if they aren't set. This replaces the previous ad-hoc skip behavior.
* More interestingly, when running against a binary with `--hge-bin`, the environment variables are passed through, which means different tests can run with different environment variables.

  On top of this, the various services we use for testing now also provide their own environment variables, rather than expecting a test script to do it.

In order to make this work, I also had to invert the dependency between various services and `hge_ctx`. I extracted a `pg_version` fixture to provide the PostgreSQL version, and now pass the `hge_url` and `hge_key` explicitly to `ActionsWebhookServer`.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6028
GitOrigin-RevId: 16d866741dba5887da1adf4e1ade8182ccc9d344
2022-09-28 09:21:02 +00:00
Samir Talwar
1a5aaae9cf server/tests-py: Add a --hge-bin argument to the Pytest runner.
This argument allows the user to specify how to run HGE, rather than starting it beforehand. The runner will start a new instance of HGE for each test class.

This does not provide isolation, as the database is still re-used, but it helps us get closer.

You can try it yourself by executing:

```
$ cabal build graphql-engine:exe:graphql-engine
$ ./server/tests-py/run-new.sh
```

This doesn't affect CI at all.

I also fixed a few warnings flagged by Pylance.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5881
GitOrigin-RevId: ea6f0fd631a2c278b2c6b50e9dbdd9d804ebc9d4
2022-09-15 12:31:12 +00:00
Samir Talwar
bff1309344 server/tests-py: Make HGECtxGQLServer a fixture.
Starting it and stopping it for the various tests that actually use it.

There are only a few.

This also removes some dead code and fixes warnings in _test_webhook_request_context.py_.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5846
GitOrigin-RevId: 7760467f9de7b1f9718e7482275c298eeaa3ad3a
2022-09-14 21:42:40 +00:00
Samir Talwar
38c94d8cda server/tests-py: Fix some warnings.
They're getting in the way of real problems.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5541
GitOrigin-RevId: 9bc99a0826715f6df77055cca79e428ad49742a8
2022-08-24 08:32:07 +00:00
Samir Talwar
a0176c215f server/tests-py: Set up postgis extensions using a fixture.
Let's put it in one place.

This is a precursor to moving database provisioning into the Python
integration tests.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5453
GitOrigin-RevId: 5920b0b1177d94496485fcb4e178b946534ee4eb
2022-08-15 14:30:42 +00:00
Samir Talwar
5df8419a4f server/tests-py: Use markers for backends instead of parameterized tests.
I'm trying to shore up the Python integration tests to make them more reliable. In doing so, I noticed this.

---

It feels a lot more sensible as we never run on more than one backend at a time.

This also removes the `check_file_exists` parameter from the setup functions; it never worked. It was always set to the result of a comparison between a backend name and a function, which was always `False`. Enabling it breaks things.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5254
GitOrigin-RevId: 8718ab21527c2ba0a7205d1c01ebaac1a10be844
2022-08-02 19:33:59 +00:00
Samir Talwar
987b55f981 server/tests-py: Reduce the number of locations we check the status code.
We have a lot of `assert st_code == 200` scattered about. This is a
problem because (a) it makes the code harder to parse and (b) the error
message is lacking; I have seen a few flaky tests which were impossible
to diagnose because I didn't know what the response _should_ be.

This reduces the number of places in which we perform this assertion
(moving most of them to `HGECtx.execute_query`), so that we can have a
better chance of seeing a useful error message on test failure.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4957
GitOrigin-RevId: 3ff388bccf49f96569aa6b7db85266a0c5ee27ea
2022-07-05 18:01:07 +00:00
Samir Talwar
75c01223a8 server: Use plain assertions in pytest.
When pytest rewrites assertions to make them more useful, it also
truncates long assertion messages in the middle, often obscuring the
actual error.

Disabling this allows us to see the full message, which should hopefully
allow us to see the actual error.

In order to support this, we need to make sure that custom assertion
messages include the actual assertion information, as this will no
longer be rendered by pytest.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4835
GitOrigin-RevId: de6839a3b40d0edc7cb96e46625eebca0aaf0c95
2022-06-25 00:04:16 +00:00
Karthikeyan Chinnakonda
2325755954 server: streaming subscriptions schema generation and tests (incremental PR - 3)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4259
Co-authored-by: Rikin Kachhia <54616969+rikinsk@users.noreply.github.com>
Co-authored-by: Brandon Simmons <210815+jberryman@users.noreply.github.com>
Co-authored-by: paritosh-08 <85472423+paritosh-08@users.noreply.github.com>
GitOrigin-RevId: 4d1b4ec3c01f3a839f4392d3b77950fc3ab30236
2022-04-22 19:54:11 +00:00
Anon Ray
e87433c2bb server: remove hdb_lib and add tests for read-only source
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3634
Co-authored-by: Philip Lykke Carlsen <358550+plcplc@users.noreply.github.com>
GitOrigin-RevId: 2db62a279496cd7e5dd57bdf02c3efa7b70042c8
2022-02-21 10:00:00 +00:00
paritosh-08
4d2a5dba51 server: fix multiple calls to database on startup
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3569
GitOrigin-RevId: 820d56a7af7faa05292ef65734d5d40fe3dd4641
2022-02-09 05:14:29 +00:00
Vamshi Surabhi
2e1be1d2c5 bump python test suite dependencies
This PR upgrades some of the pinned dependencies do not build with python 3.10 - cffi, ruamel, py. Further, it upgrades other packages where the effort is minimal.

For the reviewers: Please review it commit by commit.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3367
GitOrigin-RevId: c5401fe289d3185a79c4d382297f86fbde139825
2022-01-17 07:40:56 +00:00
pranshi06
dee86453ea server: fallback to unauthorised role when JWT is not found in cookie
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2882
GitOrigin-RevId: ad03fbd0572e00ffe7abea106388a4df4d12af2c
2021-12-08 18:29:29 +00:00
Puru Gupta
504f13725f server: forward auth webhook set-cookies header on response
>

High-Level TODO:

* [x] Code Changes
* [x] Tests
* [x] Check that pro/multitenant build ok
* [x] Documentation Changes
* [x] Updating this PR with full details
* [ ] Reviews
* [ ] Ensure code has all FIXMEs and TODOs addressed
* [x] Ensure no files are checked in mistakenly
* [x] Consider impact on console, cli, etc.

### Description
>
This PR adds support for adding set-cookie header on the response from the auth webhook. If the set-cookie header is sent by the webhook, it will be forwarded in the graphQL engine response.

Fixes a bug in test-server.sh: testing of get-webhook tests was done by POST method and vice versa. To fix, the parameters were swapped.

### Changelog

- [x] `CHANGELOG.md` is updated with user-facing content relevant to this PR.

### Affected components

- [x] Server
- [ ] Console
- [ ] CLI
- [x] Docs
- [ ] Community Content
- [ ] Build System
- [x] Tests
- [ ] Other (list it)

### Related Issues
->
Closes [#2269](https://github.com/hasura/graphql-engine/issues/2269)

### Solution and Design
>

### Steps to test and verify
>
Please refer to the docs to see how to send the set-cookie header from webhook.

### Limitations, known bugs & workarounds
>
- Support for only set-cookie header forwarding is added
- the value forwarded in the set-cookie header cannot be validated completely, the [Cookie](https://hackage.haskell.org/package/cookie) package has been used to parse the header value and any unnecessary information is stripped off before forwarding the header. The standard given in [RFC6265](https://datatracker.ietf.org/doc/html/rfc6265) has been followed for the Set-Cookie format.

### Server checklist

#### Catalog upgrade

Does this PR change Hasura Catalog version?
- [x] No
- [ ] Yes
  - [ ] Updated docs with SQL for downgrading the catalog

#### Metadata

Does this PR add a new Metadata feature?
- [x] No

#### GraphQL
- [x] No new GraphQL schema is generated
- [ ] New GraphQL schema is being generated:
   - [ ] New types and typenames are correlated

#### Breaking changes

- [x] No Breaking changes

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2538
Co-authored-by: Robert <132113+robx@users.noreply.github.com>
GitOrigin-RevId: d9047e997dd221b7ce4fef51911c3694037e7c3f
2021-11-09 12:01:31 +00:00
David Overton
aac64f2c81 Source typename customization (close graphql-engine#6974)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/1616
GitOrigin-RevId: f7eefd2367929209aa77895ea585e96a99a78d47
2021-10-29 14:43:14 +00:00
Abby Sassel
a9e3a1b4bd server/bigquery: improve test setup and documentation (close #2345)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2495
GitOrigin-RevId: ef250a0796200be6b75363b4275eb8a7ae780154
2021-10-12 17:59:44 +00:00
Rakesh Emmadi
09ae6af337 server/mssql: insert SQL generation and execution
>

### Description
>
Insert mutations for MSSQL backend. This PR implements execution logic.

### Changelog

- [x] `CHANGELOG.md` is updated with user-facing content relevant to this PR. If no changelog is required, then add the `no-changelog-required` label.

### Affected components

- [x] Server
- [x] Tests

### Related Issues
->
Close https://github.com/hasura/graphql-engine-mono/issues/2114

### Steps to test and verify
>
Track a MSSQL table and perform the generated insert mutation to test.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2248
Co-authored-by: Abby Sassel <3883855+sassela@users.noreply.github.com>
Co-authored-by: Philip Lykke Carlsen <358550+plcplc@users.noreply.github.com>
GitOrigin-RevId: 936f138c80d7a928180e6e7b0c4da64ecc1f7ebc
2021-10-01 12:53:15 +00:00
Antoine Leblanc
9bae641ac4 Always enable inherited roles tests
### Description

The inherited roles integration tests were behind a flag, and its corresponding fixture, presumably to avoid enabling the option globally. However, #2288 introduced a new test using inherited roles that was not gated behind the flag, which fails when run with `dev.sh`. However, that test works on CI... because inherited roles are globally enabled there.

Consequently, this PR:
- globally enables inherited roles in dev.sh
- removes the flag and the associated fixture

https://github.com/hasura/graphql-engine-mono/pull/2358

Co-authored-by: Vishnu Bharathi <4211715+scriptnull@users.noreply.github.com>
GitOrigin-RevId: ebfa6754873324bed15b2cc5e37ec2d8008e8f8d
2021-09-16 06:24:54 +00:00
Sameer Kolhar
edeb8c98fd server: support for graphql-ws protocol
https://github.com/hasura/graphql-engine-mono/pull/1655

Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Vijay Prasanna <11921040+vijayprasanna13@users.noreply.github.com>
Co-authored-by: hasura-bot <30118761+hasura-bot@users.noreply.github.com>
Co-authored-by: Brandon Simmons <210815+jberryman@users.noreply.github.com>
Co-authored-by: Varun Choudhary <68095256+Varun-Choudhary@users.noreply.github.com>
Co-authored-by: Divi <32202683+imperfect-fourth@users.noreply.github.com>
GitOrigin-RevId: 9db3902388fef06b94f9513255e2b5333bd23c3e
2021-08-24 16:26:12 +00:00
Karthikeyan Chinnakonda
06f5e4fb77 server: inherited roles for mutations, remote schemas, actions and custom functions
https://github.com/hasura/graphql-engine-mono/pull/1715

GitOrigin-RevId: 4818292cff8c3a5b264968e7032887a1e98b6f79
2021-08-09 10:21:05 +00:00
Solomon Bothwell
d88e2bbcce server: add tests ensuring the correct functioning of all endpoints based on user roles
https://github.com/hasura/graphql-engine-mono/pull/1625

Co-authored-by: Sameer Kolhar <6604943+kolharsam@users.noreply.github.com>
GitOrigin-RevId: 6b56efc838d2ed1acc44b2847161fde22d6aee17
2021-07-16 16:09:25 +00:00
Karthikeyan Chinnakonda
43973f80cf server tests: refactor the per_class_tests_db_state fixture
https://github.com/hasura/graphql-engine-mono/pull/1701

GitOrigin-RevId: 278b9e226cc7719f4c44bf9c9a802146a8a004f4
2021-07-01 09:21:45 +00:00
Abby Sassel
41908936cf server: improve backend-specific test setup
GitOrigin-RevId: c40393bcdd78feaba7e9490ce4ed78de19b5bfc5
2021-05-25 13:55:11 +00:00
Karthikeyan Chinnakonda
4702ba514a pro server: disable GraphQL schema introspection for specified roles
GitOrigin-RevId: 36ceb1852bc7779092cfb59c92035dd9635b2e3f
2021-05-05 12:26:23 +00:00
kodiakhq[bot]
a935746e17 Integration test improvements (for speed/clarity), also increase polling interval for scheduled events
This claws back ~7min from integration tests (run serially, as with `dev.sh test --integration`
Further improvements would do well to focus on optimizing metadata operations, as `setup` dominates

GitOrigin-RevId: 76637d6fa953c2404627c4391447a05bf09355fa
2021-04-27 05:35:26 +00:00
Abby Sassel
64d52f5fa3 server: generalize integration tests
Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
GitOrigin-RevId: 647edb8d293bf783c504b7e15ce02c56858b3b72
2021-03-11 18:18:41 +00:00
Karthikeyan Chinnakonda
92026b769f [Preview] Inherited roles for postgres read queries
fixes #3868

docker image - `hasura/graphql-engine:inherited-roles-preview-48b73a2de`

Note:

To be able to use the inherited roles feature, the graphql-engine should be started with the env variable `HASURA_GRAPHQL_EXPERIMENTAL_FEATURES` set to `inherited_roles`.

Introduction
------------

This PR implements the idea of multiple roles as presented in this [paper](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/FGALanguageICDE07.pdf). The multiple roles feature in this PR can be used via inherited roles. An inherited role is a role which can be created by combining multiple singular roles. For example, if there are two roles `author` and `editor` configured in the graphql-engine, then we can create a inherited role with the name of `combined_author_editor` role which will combine the select permissions of the `author` and `editor` roles and then make GraphQL queries using the `combined_author_editor`.

How are select permissions of different roles are combined?
------------------------------------------------------------

A select permission includes 5 things:

1. Columns accessible to the role
2. Row selection filter
3. Limit
4. Allow aggregation
5. Scalar computed fields accessible to the role

 Suppose there are two roles, `role1` gives access to the `address` column with row filter `P1` and `role2` gives access to both the `address` and the `phone` column with row filter `P2` and we create a new role `combined_roles` which combines `role1` and `role2`.

Let's say the following GraphQL query is queried with the `combined_roles` role.

```graphql
query {
   employees {
     address
     phone
   }
}
```

This will translate to the following SQL query:

```sql

 select
    (case when (P1 or P2) then address else null end) as address,
    (case when P2 then phone else null end) as phone
 from employee
 where (P1 or P2)
```

The other parameters of the select permission will be combined in the following manner:

1. Limit - Minimum of the limits will be the limit of the inherited role
2. Allow aggregations - If any of the role allows aggregation, then the inherited role will allow aggregation
3. Scalar computed fields - same as table column fields, as in the above example

APIs for inherited roles:
----------------------

1. `add_inherited_role`

`add_inherited_role` is the [metadata API](https://hasura.io/docs/1.0/graphql/core/api-reference/index.html#schema-metadata-api) to create a new inherited role. It accepts two arguments

`role_name`: the name of the inherited role to be added (String)
`role_set`: list of roles that need to be combined (Array of Strings)

Example:

```json
{
  "type": "add_inherited_role",
  "args": {
      "role_name":"combined_user",
      "role_set":[
          "user",
          "user1"
      ]
  }
}
```

After adding the inherited role, the inherited role can be used like single roles like earlier

Note:

An inherited role can only be created with non-inherited/singular roles.

2. `drop_inherited_role`

The `drop_inherited_role` API accepts the name of the inherited role and drops it from the metadata. It accepts a single argument:

`role_name`: name of the inherited role to be dropped

Example:

```json

{
  "type": "drop_inherited_role",
  "args": {
      "role_name":"combined_user"
  }
}
```

Metadata
---------

The derived roles metadata will be included under the `experimental_features` key while exporting the metadata.

```json
{
  "experimental_features": {
    "derived_roles": [
      {
        "role_name": "manager_is_employee_too",
        "role_set": [
          "employee",
          "manager"
        ]
      }
    ]
  }
}
```

Scope
------

Only postgres queries and subscriptions are supported in this PR.

Important points:
-----------------

1. All columns exposed to an inherited role will be marked as `nullable`, this is done so that cell value nullification can be done.

TODOs
-------

- [ ] Tests
   - [ ] Test a GraphQL query running with a inherited role without enabling inherited roles in experimental features
   - [] Tests for aggregate queries, limit, computed fields, functions, subscriptions (?)
   - [ ] Introspection test with a inherited role (nullability changes in a inherited role)
- [ ] Docs
- [ ] Changelog

Co-authored-by: Vamshi Surabhi <6562944+0x777@users.noreply.github.com>
GitOrigin-RevId: 3b8ee1e11f5ceca80fe294f8c074d42fbccfec63
2021-03-08 11:15:10 +00:00
Phil Freeman
7fffc11077 Caching, Rate Limiting, Metrics & Session Variable Improvements (#376)
* server: use a leaky bucket algorithm for bytes-per-second cache rate limiting

* Use evalsha properly

* Adds redis cache limit parameters to PoliciesConfig

* Loads Leaky Bucket Script On Server Start

* Adds more redis logging and moves cache update into lua script

* reverts setex in lua and adds notes

* Refactors cacheStore and adds max TTL and cache size limits

* Filter session vars in cache key

* WIP

* parens

* cache-clear-hander POC implementation

* cache-clear-hander POC implementation

* Pro projectId used as cache key

* POC working!

* prefixing query-response keys in redis

* Add cacheClearer to RedisScripts

* Partial implementation of cacheClearer from scripts record

* updating tests

* [automated] stylish-haskell commit

* Adds query look with up with metrics script

* Adds missing module and lua script from last commit

* Changes redis script module structure to match cache clearing branch

* minor change to lua script

* cleaning up cache clearing

* generalising JsonLog

* [automated] stylish-haskell commit

* Draft Cache Metrics Endpoint

* Adds Cache Metrics Handler

* Adds hook handler module

* Missed HandlerHook module in last commit

* glob

* Fixes redis mget bug

* Removes cache totals and changes dashes to colons in metric cache keys

* Adds query param to clear clear endpoint for deleting specific keys

* Adds query param to clear clear endpoint for deleting specific keys

* Cache Metrics on query families rather then queries

* Replace Set with nub

* Base16 Redis Hashes

* Query Family Redis Keys With Roles

* response headers for cache keys

* fixing bug in family key by excluding operation name; using hash for response header instead of entire key

* Adds query family to redis cache keys and cache clear endpoint

* Fixes queryfamily hash bug

* Moves cache endpoints to /pro

* Moved cache clear to POST

* Refactors cache clear function

* Fixes query family format bug

* Adds query cache tests and optional --redis-url flag to python test suite

* Adds session variable cache test

* Update pro changelog

* adding documentation for additional caching features

* more docs

* clearing up units of leaky bucket params

* Adds comments to leaky bucket script

* removes old todo

* Fixes session variable filtering to work with new query rootfield

* more advanced defaulting behaviour for bucket rate and capacity.

* Updates Docs

* Moves Role into QueryFamily hash

* Use Aeson for Cache Clear endpoint response

* Moves trace to bracket the leaky bucket script

* Misc review tweaks

* Adds sum type for cache clear query params

* Hardcodes RegisReplyLog log level

* Update docs/graphql/cloud/response-caching.rst

Co-authored-by: Phil Freeman <phil@hasura.io>

* new prose for rate limiting docs

* [automated] stylish-haskell commit

* make rootToSessVarPreds total

* [automated] stylish-haskell commit

* Fixes out of scope error

* Renamed _acRedis to _acCacheStore

Co-authored-by: Solomon Bothwell <ssbothwell@gmail.com>
Co-authored-by: Lyndon Maydwell <lyndon@sordina.net>
Co-authored-by: David Overton <david@hasura.io>
Co-authored-by: Stylish Haskell Bot <stylish-haskell@users.noreply.github.com>
Co-authored-by: Lyndon Maydwell <lyndon@hasura.io>
GitOrigin-RevId: dda5c1a3f902967b3d78310f950541a55fabb1b0
2021-02-13 00:06:18 +00:00
Swann Moreau
c14dcd5792 pass gql requests into auth webhook POST body (#149)
* fix arg order in UserAuthentication instance [force ci]

* change the constructor name to AHGraphQLRequest

Co-authored-by: Stylish Haskell Bot <stylish-haskell@users.noreply.github.com>
Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
GitOrigin-RevId: fb3258f4a84efc6c730b0c6222ebd8cea1b91081
2021-02-03 07:11:39 +00:00
Karthikeyan Chinnakonda
10a3f9960d server: new function permissions layer
Co-authored-by: Rikin Kachhia <54616969+rikinsk@users.noreply.github.com>
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
GitOrigin-RevId: 35645121242294cb6bb500ea598e9a1f2ca67fa1
2021-01-29 05:49:09 +00:00
Karthikeyan Chinnakonda
39a4352569 Merge pull request #113 from hasura/karthikeyan/remote-schema-permissions
server: remote schema permissions
GitOrigin-RevId: 63b9717e30351676c9474bdfddd3ad1ee1409eea
2020-12-21 09:12:35 +00:00
Vamshi Surabhi
ff99b24031 fix a security issue with 1.2 2020-05-22 14:40:57 +05:30
Tirumarai Selvan
cc8e2ccc78
Scheduled triggers (close #1914) (#3553)
server: add scheduled triggers 

Co-authored-by: Alexis King <lexi.lambda@gmail.com>
Co-authored-by: Marion Schleifer <marion@hasura.io>
Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
Co-authored-by: Aleksandra Sikora <ola.zxcvbnm@gmail.com>
2020-05-13 18:03:16 +05:30
Rakesh Emmadi
fd6535b861
option to reload remote schemas in 'reload metadata' (fix #3792, #4117) (#4141)
* option to reload remote schemas in 'reload_metadata' API, fix #3792, #4117

* add tests

* update changelog

* update docs api reference for 'reload_metadata'

* send reload_remote_schemas: true with the reload_metadata query

* add reload remote schemas checkbox; minor refactor

* Add a Note about cache invalidation and inconsistent metadata objects

* Small pluralization agreement tweak in docs

* Remove duplicated line in CHANGELOG

* no-op refactor

Suggested by Alexis @lexi-lambda

* Update server/src-lib/Hasura/RQL/DDL/RemoteSchema.hs

As suggested by @lexi-lambda

Co-Authored-By: Alexis King <lexi.lambda@gmail.com>

* fix tests

* requested changes

* comment 'replaceMetadataToOrdJson' unit tests

Co-authored-by: Rishichandra Wawhal <rishi@hasura.io>
Co-authored-by: Alexis King <lexi.lambda@gmail.com>
Co-authored-by: Tirumarai Selvan <tiru@hasura.io>
2020-03-26 17:22:20 +05:30
Rakesh Emmadi
f80b69e931
few actions' fixes and improvements (fix #3977, #4061 & close #4021) (#4109)
* add 'ID' to default scalars for custom types, fix #4061

* preserve cookie headers from sync action webhook, close #4021

* validate action webhook response to conform to output type, fix #3977

* fix tests, don't run actions' tests on PG version < 10

* update CHANGELOG.md

* no-op refactor, use types from http-network more

Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
2020-03-20 12:16:45 +05:30
Vamshi Surabhi
b84db36ebb
allow custom mutations through actions (#3042)
* basic doc for actions

* custom_types, sync and async actions

* switch to graphql-parser-hs on github

* update docs

* metadata import/export

* webhook calls are now supported

* relationships in sync actions

* initialise.sql is now in sync with the migration file

* fix metadata tests

* allow specifying arguments of actions

* fix blacklist check on check_build_worthiness job

* track custom_types and actions related tables

* handlers are now triggered on async actions

* default to pgjson unless a field is involved in relationships, for generating definition list

* use 'true' for action filter for non admin role

* fix create_action_permission sql query

* drop permissions when dropping an action

* add a hdb_role view (and relationships) to fetch all roles in the system

* rename 'webhook' key in action definition to 'handler'

* allow templating actions wehook URLs with env vars

* add 'update_action' /v1/query type

* allow forwarding client headers by setting `forward_client_headers` in action definition

* add 'headers' configuration in action definition

* handle webhook error response based on status codes

* support array relationships for custom types

* implement single row mutation, see https://github.com/hasura/graphql-engine/issues/3731

* single row mutation: rename 'pk_columns' -> 'columns' and no-op refactor

* use top level primary key inputs for delete_by_pk & account select permissions for single row mutations

* use only REST semantics to resolve the webhook response

* use 'pk_columns' instead of 'columns' for update_by_pk input

* add python basic tests for single row mutations

* add action context (name) in webhook payload

* Async action response is accessible for non admin roles only if
  the request session vars equals to action's

* clean nulls, empty arrays for actions, custom types in export metadata

* async action mutation returns only the UUID of the action

* unit tests for URL template parser

* Basic sync actions python tests

* fix output in async query & add async tests

* add admin secret header in async actions python test

* document async action architecture in Resolve/Action.hs file

* support actions returning array of objects

* tests for list type response actions

* update docs with actions and custom types metadata API reference

* update actions python tests as per #f8e1330

Co-authored-by: Tirumarai Selvan <tirumarai.selvan@gmail.com>
Co-authored-by: Aravind Shankar <face11301@gmail.com>
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
2020-02-13 23:08:23 +05:30
Tirumarai Selvan
f8e133070b
run default tests in test_server_upgrade (#3718)
* run basic tests after upgrade

* terminate before specifying file in pytest cmd

* Move fixture definitions out of test classes

Previously we had abstract classes with the fixtures defined
in them. The test classes then inherits these super classes. This
is creating inheritence problems, especially when you want to just
inherit the tests in class, but not the fixtures. We have now moved
all those fixture definitions outside of the class (in conftest.py).
These fixtures are now used by the test classes when and where they
are required.

* Run pytests on server upgrade

Server upgrade tests are run by
  1) Run pytest with schema/metadata setup but do not do schema/metadata
teardown
  2) Upgrade the server
  3) Run pytest using the above schema and teardown at the end of the
tests
  4) Cleanup hasura metadata and start again with next set of tests

We have added options --skip-schema-setup and --skip-schema-teardown to
help running server upgrade tests.

While running the tests, we noticed that error codes and messages for
some of the tests have changed. So we have added another option to
pytest `--avoid-error-message-checks`. If this flag is set, and if
comparing expected and response message fails, and if the expected
response has an error message, Pytest will throw warnings instead of an
error.

* Use marks to specify server-upgrade tests

Not all tests can be run as serve upgrade tests, particularly those
which themselves change the schema. We introduce two pytest markers.
Marker allow_server_upgrade_test will add the test into the list of
server  upgrade  tests  that  can  be run. skip_server_upgrade_test
removes it from the list.

With this we have added tests for queries, mutations, and selected
event trigger and remote schema tests to the list of server upgrade
tests.

* Remove components not needed anymore

* Install curl

* Fix error in query validation

* Fix error in test_v1_queries.py

* install procps for server upgrade tests

* Use postgres image which has postgis installed

* set pager off with psql

* quote the bash variable WORKTREE_DIR

Co-authored-by: nizar-m <19857260+nizar-m@users.noreply.github.com>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
2020-02-13 14:44:02 +05:30
Anon Ray
afd6f30e72 read cache control header to refresh JWK (fix #3301) (#3446) 2019-12-03 14:56:59 -06:00