Commit Graph

294 Commits

Author SHA1 Message Date
Karthikeyan Chinnakonda
6bddaa7564 server: inherited roles improvements for select permissions only
https://github.com/hasura/graphql-engine-mono/pull/1539

GitOrigin-RevId: 7444973e9cc7e0598eeb1bd7951ad45dd37ec550
2021-07-16 21:19:58 +00:00
Solomon Bothwell
d88e2bbcce server: add tests ensuring the correct functioning of all endpoints based on user roles
https://github.com/hasura/graphql-engine-mono/pull/1625

Co-authored-by: Sameer Kolhar <6604943+kolharsam@users.noreply.github.com>
GitOrigin-RevId: 6b56efc838d2ed1acc44b2847161fde22d6aee17
2021-07-16 16:09:25 +00:00
Vishnu Bharathi
6e89653814 ci: uploads ARM64 CLI binaries during deploy
https://github.com/hasura/graphql-engine-mono/pull/1804

GitOrigin-RevId: 3348d15c8869d375bb7bf22bfc53a5d148d1568f
2021-07-16 04:20:01 +00:00
Abby Sassel
1c09e7c8cf cleanup: format .circleci/config.yml files
First up: please let me know if we're already using a YAML file formatter, which one, and I'll close this.

**Otherwise:**

This PR formats .circleci/config.yml files with https://github.com/redhat-developer/yaml-language-server formatter.

**As someone who** edits yaml files infrequently, but with a few default language servers active in my editor,
**I'd like to** edit the file without having to think about formatting or yaml idiosyncrasies
**So that** I can raise a PR and move on with my life 

### Before: incorrect formatting highlighted for 1/5 of `.circleci.config.yaml` file. Difficult to read/edit, and unable to use any common editor integrations
![Screenshot 2021-07-14 at 10 51 41](https://user-images.githubusercontent.com/3883855/125602289-6c0ce457-8a1f-43ea-96f4-1b29e440181a.png)

### After: writing yaml brings me joy now
![Screenshot 2021-07-14 at 11 21 42](https://user-images.githubusercontent.com/3883855/125606378-e6e4ee2a-c44a-40c4-bebe-3e7ab4cb98ae.png)

https://github.com/hasura/graphql-engine-mono/pull/1788

GitOrigin-RevId: d821f3ab5af758e4af3aa442afb0997d76e53c72
2021-07-14 15:24:13 +00:00
hasura-bot
f9e11f6bde server/bugfix: update server_builder_image tag
GITHUB_PR_NUMBER: 7235
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/7235

https://github.com/hasura/graphql-engine-mono/pull/1791

Co-authored-by: Abby Sassel <3883855+sassela@users.noreply.github.com>
GitOrigin-RevId: 2905c0d80341aaeec328460864d35c3a3bc96ee0
2021-07-14 13:14:17 +00:00
Vishnu Bharathi
8a6f3f7e9d cli: support ARM architecture
https://github.com/hasura/graphql-engine-mono/pull/1534

GitOrigin-RevId: 67723bb64e2505c665b04b5b0d691e5599843f87
2021-07-14 08:54:43 +00:00
Aniket Deshpande
3bdd777ec4 MySQL: Add mysql-client dependencies on CI and in .cabal file
https://github.com/hasura/graphql-engine-mono/pull/1514

Co-authored-by: Sibi Prabakaran <737477+psibi@users.noreply.github.com>
Co-authored-by: Chris Done <11019+chrisdone@users.noreply.github.com>
Co-authored-by: Abby Sassel <3883855+sassela@users.noreply.github.com>
Co-authored-by: Toan Nguyen  <1615675+hgiasac@users.noreply.github.com>
GitOrigin-RevId: 81fd23f439e8c3db8149fedae2855db1e84a60a3
2021-07-12 20:07:46 +00:00
Sooraj
effb221d97 console: fix test_oss_console ci errors
https://github.com/hasura/graphql-engine-mono/pull/1726

GitOrigin-RevId: ae87ca45a111fcdb25ba9c53c1bfd8650e1be78c
2021-07-12 10:40:50 +00:00
Aniket Deshpande
66f09eeaab MSSQL nodes aggregates & inherited roles
https://github.com/hasura/graphql-engine-mono/pull/1293

Co-authored-by: Chris Done <11019+chrisdone@users.noreply.github.com>
Co-authored-by: Abby Sassel <3883855+sassela@users.noreply.github.com>
GitOrigin-RevId: 776402dbbaf3d8166a62b1aaaf6abc7e584b3eb2
2021-07-08 20:50:09 +00:00
hasura-bot
35ab147dd9 server/citus: run tests in CI (OSS changes)
GITHUB_PR_NUMBER: 7161
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/7161

https://github.com/hasura/graphql-engine-mono/pull/1711

Co-authored-by: Abby Sassel <3883855+sassela@users.noreply.github.com>
GitOrigin-RevId: 7acbfdad71d9c26fb1bb015f66f844988927bdc8
2021-07-02 05:19:58 +00:00
Abby Sassel
5b7d949ef4 server/citus: run tests in CI
https://github.com/hasura/graphql-engine-mono/pull/1526

GitOrigin-RevId: 4fd4214a799bc0d91cd2d29b1ea444f80714f6bc
2021-07-01 14:41:12 +00:00
Brandon Simmons
f1b1dc109e tests: document server-upgrade-downgrade test
https://github.com/hasura/graphql-engine-mono/pull/1658

GitOrigin-RevId: c097de5758847ee01e5cb4f049a662338f6a4dda
2021-06-29 18:39:30 +00:00
Divi
bcb5baf63b ci: make deploy job dependent on cli, console test jobs
https://github.com/hasura/graphql-engine-mono/pull/1663

GitOrigin-RevId: 6731c33adcbfff9ba972fca226d9f3245496233c
2021-06-25 14:07:14 +00:00
kodiakhq[bot]
a11b4135fc Use exceptions to handle compatibility with pg 9.6 in 2.0 upgrade migration
https://github.com/hasura/graphql-engine-mono/pull/1651

Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Brandon Simmons <210815+jberryman@users.noreply.github.com>
GitOrigin-RevId: cdb4a33cbb1a136bc30e8f0dd59aaae584d3a636
2021-06-24 02:20:11 +00:00
Rakesh Emmadi
13bedf5821 server/postgres: fix resetting the metadata catalog version to 43 while initializing postgres source with 1.0 catalog (#1645)
* fix resetting the catalog version to 43 on migration from 1.0 to 2.0

* ci: remove applying patch in test_oss_server_upgrade job

* make the 43 to 46th migrations idempotent

* Set missing HASURA_GRAPHQL_EVENTS_HTTP_POOL_SIZE=8 in upgrade_test

It's not clear why this wasn't caught in CI.

* ci: disable one component of event backpressure test

Co-authored-by: Vishnu Bharathi P <vishnubharathi04@gmail.com>
Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
Co-authored-by: Brandon Simmons <brandon@hasura.io>
GitOrigin-RevId: c74c6425266a99165c6beecc3e4f7c34e6884d4d
2021-06-23 18:01:09 +00:00
Abby Sassel
b6b1354a2b server/mssql: Fix sqlcmd symlink Dockerfile
https://github.com/hasura/graphql-engine-mono/pull/1578

GitOrigin-RevId: 97d793b42b029cd51a8a96699ca4a67f1463d349
2021-06-15 19:45:08 +00:00
hasura-bot
6d8a441e64 ci: renames mssql test job (#1567)
GITHUB_PR_NUMBER: 7059
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/7059

https://github.com/hasura/graphql-engine-mono/pull/1567

Co-authored-by: Vishnu Bharathi <4211715+scriptnull@users.noreply.github.com>
GitOrigin-RevId: 12d4cfdb8aee13644c0b9eed56d152e74335dd24
2021-06-14 14:41:20 +00:00
Abby Sassel
34de64bcdf server/mssql: run tests in CI (#1452)
https://github.com/hasura/graphql-engine-mono/pull/1452

GitOrigin-RevId: 9fd345f7e5c24ebc3f47bdd0b15f84c9b8766f0c
2021-06-14 07:31:42 +00:00
Aravind K P
d8198a8bad cli,ci: testsuite enhancements
- remove `HASURA_TEST_CLI_HGE_DOCKER_TAG` & `HASURA_TEST_CLI_HGE_DOCKER_REPO` env variables
- add `HASURA_TEST_CLI_HGE_DOCKER_IMAGE` environment variable to configure hge image used in tests
- add template test project directories
- add helper functions to manipulate the template projects in individual tests
- add config v2 tests

Co-authored-by: Kali Vara Purushotham Santhati <72007599+purush7@users.noreply.github.com>
GitOrigin-RevId: 009a74c042861ff0a8dec2b06002e55de3a8a629
2021-06-03 13:27:24 +00:00
Karthikeyan Chinnakonda
7d630e8147 build: use the debian buster version in the docker image
GitOrigin-RevId: 51077f8b5d0c5bdf06a6560444f89d74a3660336
2021-05-27 07:50:15 +00:00
hasura-bot
07adb7b273 ci: update server builder to correct version
GITHUB_PR_NUMBER: 6945
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/6945

Co-authored-by: Vishnu Bharathi <4211715+scriptnull@users.noreply.github.com>
GitOrigin-RevId: b0c20d6b249f1659ef076881a3aeb2f84b96eaf7
2021-05-18 12:16:34 +00:00
Vishnu Bharathi
ebc01c2378 cli, ci: enables docker based tests
Co-authored-by: Aravind K P <8335904+scriptonist@users.noreply.github.com>
GitOrigin-RevId: d7a53e512d296bdd4b2119bb1d0b89e30da7d476
2021-05-17 00:30:08 +00:00
Aravind K P
2d126342f8 ci: move "install deps" in server test CI jobs to server image builder
GitOrigin-RevId: 8e4388a856b7bae4abcd3c9db289826e89128f8b
2021-05-12 15:25:30 +00:00
kodiakhq[bot]
a935746e17 Integration test improvements (for speed/clarity), also increase polling interval for scheduled events
This claws back ~7min from integration tests (run serially, as with `dev.sh test --integration`
Further improvements would do well to focus on optimizing metadata operations, as `setup` dominates

GitOrigin-RevId: 76637d6fa953c2404627c4391447a05bf09355fa
2021-04-27 05:35:26 +00:00
Vishnu Bharathi
7821b34a2c ci,console,pro-console: build pro console using local oss console
Co-authored-by: Aleksandra Sikora <9019397+beerose@users.noreply.github.com>
GitOrigin-RevId: f9a918422a0d22bc1775fabedb603c3311673afb
2021-04-12 18:50:02 +00:00
Aravind K P
6d4e0266c9 ci: refactor build steps of cli
GitOrigin-RevId: 918276b6883fe15b2be93fae2dd260acc959ff69
2021-04-08 12:51:07 +00:00
Sooraj
9130fa8b8f console/ci: add unit tests
GitOrigin-RevId: 01c9328a298dc6e53b910fa05d7587614c324f20
2021-04-07 13:28:13 +00:00
Aravind K P
1cc3a66fef cli-migrations,ci: add config v3 image
GitOrigin-RevId: ee04d046fd70355495fb18976ebfcca00454c0da
2021-04-06 09:59:16 +00:00
Aleksandra Sikora
dc17de8571 console/ci: add missing env var
GitOrigin-RevId: 15621e6174a339c4741970f9ce9d60a1ae93afa5
2021-03-24 20:58:15 +00:00
Ikechukwu Eze
81f527a7ee console/ci: upgrade terser and drop large resource class for console build
GitOrigin-RevId: 1e352fa17b7561cfe618bd3cb9f0f75be3e95c73
2021-03-24 19:22:41 +00:00
hasura-bot
e0ec670b05 ci: fix missing postgres envs in test_server (#949)
Co-authored-by: Vishnu Bharathi P <vishnubharathi04@gmail.com>
GITHUB_PR_NUMBER: 6708
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/6708

Co-authored-by: Vishnu Bharathi P <vishnubharathi04@gmail.com>
GitOrigin-RevId: ef035813e9dd4f5e61e62889068222d49bbaa350
2021-03-24 14:38:18 +00:00
Vishnu Bharathi P
d6c4f95e27 ci: use large resource class for build_console job
GitOrigin-RevId: cc9f1d18ad455bf16372bc5ba0e2c705224338b9
2021-03-18 08:44:39 +00:00
Abhijeet Singh Khangarot
1c1198c93b console: remote schema permissions bug fixes
Co-authored-by: Sooraj <8408875+soorajshankar@users.noreply.github.com>
GitOrigin-RevId: 78c4e144ea32ab1376f36272fab0204c2bd88ef0
2021-03-17 15:57:36 +00:00
hasura-bot
2099208e4b cli: merges cli-ext with cli
GITHUB_PR_NUMBER: 6640
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/6640

Co-authored-by: Vishnu Bharathi <4211715+scriptnull@users.noreply.github.com>
Co-authored-by: Aravind K P <8335904+scriptonist@users.noreply.github.com>
GitOrigin-RevId: 362d82d8ee18afd7efa63e9c8912f6c5444f82dd
2021-03-17 15:10:03 +00:00
kodiakhq[bot]
c8934d0b57 circleci: Use backport of Matt Pickering's memory reclamation RTS patch in CI
A few cleanups in CI, and use Matt's patch for better idle memory reclamation.

See Swann's report here: https://docs.google.com/document/d/1K1ofxYCIDdO75_UOAqJHn3CeMNE78MrJIJR-GG9iEKg/edit
And patch here: 29d4df0f5e

GitOrigin-RevId: ecdd4cbdd4244a8abf022078525c1843bf2a4aea
2021-03-12 02:09:21 +00:00
Karthikeyan Chinnakonda
92026b769f [Preview] Inherited roles for postgres read queries
fixes #3868

docker image - `hasura/graphql-engine:inherited-roles-preview-48b73a2de`

Note:

To be able to use the inherited roles feature, the graphql-engine should be started with the env variable `HASURA_GRAPHQL_EXPERIMENTAL_FEATURES` set to `inherited_roles`.

Introduction
------------

This PR implements the idea of multiple roles as presented in this [paper](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/FGALanguageICDE07.pdf). The multiple roles feature in this PR can be used via inherited roles. An inherited role is a role which can be created by combining multiple singular roles. For example, if there are two roles `author` and `editor` configured in the graphql-engine, then we can create a inherited role with the name of `combined_author_editor` role which will combine the select permissions of the `author` and `editor` roles and then make GraphQL queries using the `combined_author_editor`.

How are select permissions of different roles are combined?
------------------------------------------------------------

A select permission includes 5 things:

1. Columns accessible to the role
2. Row selection filter
3. Limit
4. Allow aggregation
5. Scalar computed fields accessible to the role

 Suppose there are two roles, `role1` gives access to the `address` column with row filter `P1` and `role2` gives access to both the `address` and the `phone` column with row filter `P2` and we create a new role `combined_roles` which combines `role1` and `role2`.

Let's say the following GraphQL query is queried with the `combined_roles` role.

```graphql
query {
   employees {
     address
     phone
   }
}
```

This will translate to the following SQL query:

```sql

 select
    (case when (P1 or P2) then address else null end) as address,
    (case when P2 then phone else null end) as phone
 from employee
 where (P1 or P2)
```

The other parameters of the select permission will be combined in the following manner:

1. Limit - Minimum of the limits will be the limit of the inherited role
2. Allow aggregations - If any of the role allows aggregation, then the inherited role will allow aggregation
3. Scalar computed fields - same as table column fields, as in the above example

APIs for inherited roles:
----------------------

1. `add_inherited_role`

`add_inherited_role` is the [metadata API](https://hasura.io/docs/1.0/graphql/core/api-reference/index.html#schema-metadata-api) to create a new inherited role. It accepts two arguments

`role_name`: the name of the inherited role to be added (String)
`role_set`: list of roles that need to be combined (Array of Strings)

Example:

```json
{
  "type": "add_inherited_role",
  "args": {
      "role_name":"combined_user",
      "role_set":[
          "user",
          "user1"
      ]
  }
}
```

After adding the inherited role, the inherited role can be used like single roles like earlier

Note:

An inherited role can only be created with non-inherited/singular roles.

2. `drop_inherited_role`

The `drop_inherited_role` API accepts the name of the inherited role and drops it from the metadata. It accepts a single argument:

`role_name`: name of the inherited role to be dropped

Example:

```json

{
  "type": "drop_inherited_role",
  "args": {
      "role_name":"combined_user"
  }
}
```

Metadata
---------

The derived roles metadata will be included under the `experimental_features` key while exporting the metadata.

```json
{
  "experimental_features": {
    "derived_roles": [
      {
        "role_name": "manager_is_employee_too",
        "role_set": [
          "employee",
          "manager"
        ]
      }
    ]
  }
}
```

Scope
------

Only postgres queries and subscriptions are supported in this PR.

Important points:
-----------------

1. All columns exposed to an inherited role will be marked as `nullable`, this is done so that cell value nullification can be done.

TODOs
-------

- [ ] Tests
   - [ ] Test a GraphQL query running with a inherited role without enabling inherited roles in experimental features
   - [] Tests for aggregate queries, limit, computed fields, functions, subscriptions (?)
   - [ ] Introspection test with a inherited role (nullability changes in a inherited role)
- [ ] Docs
- [ ] Changelog

Co-authored-by: Vamshi Surabhi <6562944+0x777@users.noreply.github.com>
GitOrigin-RevId: 3b8ee1e11f5ceca80fe294f8c074d42fbccfec63
2021-03-08 11:15:10 +00:00
Karthikeyan Chinnakonda
d2418383bf server: add https remote schema tests
GitOrigin-RevId: 9e9e66248f956174dc8197adeb7a48da12e53dac
2021-03-02 16:52:47 +00:00
Karthikeyan Chinnakonda
4211d27272 server: support reading JWT from Cookie header
GitOrigin-RevId: 1de90242d3c000361f87256c2dddce1677863231
2021-02-25 09:03:46 +00:00
Aravind K P
65ea2f4c1b ci: update cli and console builder images
Co-authored-by: Rishichandra Wawhal <27274869+wawhal@users.noreply.github.com>
GitOrigin-RevId: 1c14fc13a8ce9569a721dba1d9bd67c45292ce06
2021-02-24 07:42:42 +00:00
Vladimir Ciobanu
281cb771ff server: add MSSQL support
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
Co-authored-by: Vamshi Surabhi <6562944+0x777@users.noreply.github.com>
Co-authored-by: Aravind K P <8335904+scriptonist@users.noreply.github.com>
GitOrigin-RevId: 699c453b9692e1b822f393f23ff5e6db4e010d57
2021-02-23 17:38:36 +00:00
Aravind K P
730ac1aed4 ci: stop building and pushing cli-migrations-v1 image in deploy step (#602)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
GitOrigin-RevId: 2e0b6f39a953116273a6dcaa59c06371844519f2
2021-02-19 11:49:02 +00:00
Karthikeyan Chinnakonda
77b3bb5a20 server: run function permissions tests in CI
GitOrigin-RevId: 750b44d0f30df506eb8f0a8eec42fcd1a7d51599
2021-02-18 08:27:00 +00:00
Aravind K P
a0995d1b30 cli: add client package, statestore implementations, deprecate config v1
GitOrigin-RevId: 5a1d9cb772ac62603f2543bfe6c01a95c0a035c6
2021-02-17 04:21:11 +00:00
Vishnu Bharathi
341837b652 ci,cli-ext: adds pkg as dev dependency and updates build instructions
Fixes https://github.com/hasura/graphql-engine/issues/6426

GitOrigin-RevId: 913a9e1de4bac7c96cbeab139e7593ee2ddc578b
2021-02-08 08:04:00 +00:00
Swann Moreau
c14dcd5792 pass gql requests into auth webhook POST body (#149)
* fix arg order in UserAuthentication instance [force ci]

* change the constructor name to AHGraphQLRequest

Co-authored-by: Stylish Haskell Bot <stylish-haskell@users.noreply.github.com>
Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
GitOrigin-RevId: fb3258f4a84efc6c730b0c6222ebd8cea1b91081
2021-02-03 07:11:39 +00:00
Karthikeyan Chinnakonda
10a3f9960d server: new function permissions layer
Co-authored-by: Rikin Kachhia <54616969+rikinsk@users.noreply.github.com>
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
GitOrigin-RevId: 35645121242294cb6bb500ea598e9a1f2ca67fa1
2021-01-29 05:49:09 +00:00
Rakesh Emmadi
ff3c58f230 ci: fix oss test server upgrade ci test when upgrading from 1.3.3
GitOrigin-RevId: a5b663ea5b079212c063f15bc67559f5a234f96f
2021-01-28 11:40:24 +00:00
Swann Moreau
d4b474061f server: split server test suite to run in parallel on circleci (#309)
Co-authored-by: rakeshkky <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Vishnu Bharathi P <vishnubharathi04@gmail.com>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
Co-authored-by: Nizar Malangadan <nizar-m@users.noreply.github.com>
Co-authored-by: nizar-m <19857260+nizar-m@users.noreply.github.com>
GitOrigin-RevId: b8a1e9c3ba4797bad8dbb2daa75d23d4ad163546
2021-01-25 08:24:13 +00:00
Aleksandra Sikora
b4fd2e3f9f console and cli: run console in CLI mode for tests, misc CLI fixes (#368)
GitOrigin-RevId: 4563444387a4c9d02b5429729fe30bc87763288f
2021-01-20 20:18:44 +00:00
Rakesh Emmadi
21fb33d709 fix oss server upgrade ci test from 1.3.3 stable release (#378)
GitOrigin-RevId: 1bcd4bb1e1f741ff44219dc1b559af5514c6df91
2021-01-20 12:00:00 +00:00
Swann Moreau
c995cffbb5 server: misc ci improvements (#372)
* split ~/.cabal/{packages,store} caches

* use xlarge for {oss_,}build_server

GitOrigin-RevId: e37619e85459b3d05ebcbf93447b02d10b8b4841
2021-01-19 11:43:36 +00:00
hasura-bot
a398d3b190 cli: support datasources
GITHUB_PR_NUMBER: 6111
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/6111

Co-authored-by: Aravind K P <8335904+scriptonist@users.noreply.github.com>
GitOrigin-RevId: 1f6517acfacb58c566bb5e48f74ea0dfa5c6f063
2021-01-18 17:12:09 +00:00
Karthikeyan Chinnakonda
c14bcb6967 server: accept new config allowed_skew in JWT config to provide leeway in JWT expiry
fixes https://github.com/hasura/graphql-engine/issues/2109

This PR accepts a new config `allowed_skew` in the JWT config to provide for some leeway while comparing the JWT expiry time.

GitOrigin-RevId: ef50cf77d8e2780478685096ed13794b5c4c9de4
2021-01-13 08:39:18 +00:00
Rakesh Emmadi
be62641f68 server: multi source metadata APIs (#217)
Co-authored-by: Aleksandra Sikora <aleksandra@hasura.io>
Co-authored-by: Anon Ray <ecthiender@users.noreply.github.com>
Co-authored-by: Vishnu Bharathi <vishnubharathi04@gmail.com>
Co-authored-by: Aleksandra Sikora <aleksandra@hasura.io>
Co-authored-by: Sameer Kolhar <kolhar730@gmail.com>
Co-authored-by: Aleksandra Sikora <ola.zxcvbnm@gmail.com>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
GitOrigin-RevId: 0dd1e4d58ab81f1b4ce24de2d3eab709c2755e6d
2021-01-07 09:05:19 +00:00
Karthikeyan Chinnakonda
39a4352569 Merge pull request #113 from hasura/karthikeyan/remote-schema-permissions
server: remote schema permissions
GitOrigin-RevId: 63b9717e30351676c9474bdfddd3ad1ee1409eea
2020-12-21 09:12:35 +00:00
Rakesh Emmadi
a2cf9a53c2 server: move to storing metadata as a json blob (#115)
GitOrigin-RevId: 3d1a7618a4ec086c2d255549a6c15087201e9ab0
2020-12-08 14:23:28 +00:00
Vishnu Bharathi
4f76b9fee2 Fixes circleci error caused on force push to branches
* adds fallback for invalid CIRCLE_COMPARE_URL

* ci: separates ciignore script for mono and oss repos

GitOrigin-RevId: c473c289b8ad6695c589942c27c2369a5d0cd4c7
2020-12-07 07:45:24 +00:00
hasura-bot
f6bd354b40 server: hasura on PG v13 (#125)
GitOrigin-RevId: 00fd91c250bcf3dc7ee638e3b152e0dab7281de7
2020-12-01 12:22:42 +00:00
Vishnu Bharathi
db88fff890 enables OSS CI tests for monorepo
* init oss ci tests in monorepo

* uses circle env to set base branch

* change ref name to filter_only_vtags

* install from local artifact

* updates comment in ciignore script

* adds oss_all_server_tests_pass job

Co-authored-by: scriptonist <aravindkp@hasura.io>
GitOrigin-RevId: 681b73c7c8331950d84fdd86111ab4297bb60725
2020-11-24 17:01:04 +00:00
hasura-bot
d1ef6e66c9 server: add missing remote schema webhook env var in server-upgrade script (#63)
Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
GITHUB_PR_NUMBER: 6221
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/6221

Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
GitOrigin-RevId: cbff9c7cf037ac45f30af6e9d215d46fac662758
2020-11-23 05:27:24 +00:00
Vishnu Bharathi P
58c44f55dd Merge oss/master onto mono/main
GitOrigin-RevId: 1c8c4d60e033c8a0bc8b2beed24c5bceb7d4bcc8
2020-11-12 22:37:19 +05:30
Vishnu Bharathi P
666058ab7f oss: renames dot files and folders
GitOrigin-RevId: 540aeec3be091e1cfb7b05a988f50445534ed663
2020-11-12 22:37:19 +05:30
Aravind K P
153054bed9
ci: bump cli builder version (close #5462) (#6140)
https://github.com/hasura/graphql-engine/pull/6140
2020-11-06 06:54:52 +00:00
Karthikeyan Chinnakonda
3ea611f9fd
Server: Validate remote schema queries (#5938)
* [skip ci] use the args while making the fieldParser

* modify the execution part of the remote queries

* parse union queries deeply

* add test for remote schema field validation

* add tests for validating remote query arguments


Co-authored-by: Auke Booij <auke@hasura.io>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2020-10-13 14:03:11 +05:30
Karthikeyan Chinnakonda
92ef504c9e
Server: add URL templating for event triggers and remote schemas (#5760)
* add url templating for event triggers


Co-authored-by: Brandon Simmons <brandon.m.simmons@gmail.com>
2020-09-10 15:00:34 +05:30
Rakesh Emmadi
4ce6002af2
support customizing JWT claims (close #3485) (#3575)
* improve jsonpath parser to accept special characters and property tests for the same

* make the JWTClaimsMapValueG parametrizable

* add documentation in the JWT file

* modify processAuthZHeader

Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
Co-authored-by: Marion Schleifer <marion@hasura.io>
2020-08-31 22:10:01 +05:30
Alexis King
8c29f15b4d
server: Bump GHC version to 8.10.2 (#5659) 2020-08-26 23:16:23 -05:00
Alexis King
7e970177c1
Rewrite GraphQL schema generation and query parsing (close #2801) (#4111)
Aka “the PDV refactor.” History is preserved on the branch 2801-graphql-schema-parser-refactor.

* [skip ci] remove stale benchmark commit from commit_diff

* [skip ci] Check for root field name conflicts between remotes

* [skip ci] Additionally check for conflicts between remotes and DB

* [skip ci] Check for conflicts in schema when tracking a table

* [skip ci] Fix equality checking in GraphQL AST

* server: fix mishandling of GeoJSON inputs in subscriptions (fix #3239) (#4551)

* Add support for multiple top-level fields in a subscription to improve testability of subscriptions

* Add an internal flag to enable multiple subscriptions

* Add missing call to withConstructorFn in live queries (fix #3239)

Co-authored-by: Alexis King <lexi.lambda@gmail.com>

* Scheduled triggers (close #1914) (#3553)

server: add scheduled triggers

Co-authored-by: Alexis King <lexi.lambda@gmail.com>
Co-authored-by: Marion Schleifer <marion@hasura.io>
Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
Co-authored-by: Aleksandra Sikora <ola.zxcvbnm@gmail.com>

* dev.sh: bump version due to addition of croniter python dependency

* server: fix an introspection query caching issue (fix #4547) (#4661)

Introspection queries accept variables, but we need to make sure to
also touch the variables that we ignore, so that an introspection
query is marked not reusable if we are not able to build a correct
query plan for it.

A better solution here would be to deal with such unused variables
correctly, so that more introspection queries become reusable.

An even better solution would be to type-safely track *how* to reuse
which variables, rather than to split the reusage marking from the
planning.

Co-authored-by: Tirumarai Selvan <tiru@hasura.io>

* flush log buffer on exception in mkWaiApp ( fix #4772 ) (#4801)

* flush log buffer on exception in mkWaiApp

* add comment to explain the introduced change

* add changelog

* allow logging details of a live query polling thread (#4959)

* changes for poller-log

add various multiplexed query info in poller-log

* minor cleanup, also fixes a bug which will return duplicate data

* Live query poller stats can now be logged

This also removes in-memory stats that are collected about batched
query execution as the log lines when piped into an monitoring tool
will give us better insights.

* allow poller-log to be configurable

* log minimal information in the livequery-poller-log

Other information can be retrieved from /dev/subscriptions/extended

* fix few review comments

* avoid marshalling and unmarshalling from ByteString to EncJSON

* separate out SubscriberId and SubscriberMetadata

Co-authored-by: Anon Ray <rayanon004@gmail.com>

* Don't compile in developer APIs by default

* Tighten up handling of admin secret, more docs

Store the admin secret only as a hash to prevent leaking the secret
inadvertently, and to prevent timing attacks on the secret.

NOTE: best practice for stored user passwords is a function with a
tunable cost like bcrypt, but our threat model is quite different (even
if we thought we could reasonably protect the secret from an attacker
who could read arbitrary regions of memory), and bcrypt is far too slow
(by design) to perform on each request. We'd have to rely on our
(technically savvy) users to choose high entropy passwords in any case.

Referencing #4736

* server/docs: add instructions to fix loss of float precision in PostgreSQL <= 11 (#5187)

This adds a server flag, --pg-connection-options, that can be used to set a PostgreSQL connection parameter, extra_float_digits, that needs to be used to avoid loss of data on older versions of PostgreSQL, which have odd default behavior when returning float values. (fixes #5092)

* [skip ci] Add new commits from master to the commit diff

* [skip ci] serve default directives (skip & include) over introspection

* [skip ci] Update non-Haskell assets with the version on master

* server: refactor GQL execution check and config API (#5094)

Co-authored-by: Vamshi Surabhi <vamshi@hasura.io>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>

* [skip ci] fix js issues in tests by pinning dependencies version

* [skip ci] bump graphql version

* [skip ci] Add note about memory usage

* generalize query execution logic on Postgres (#5110)

* generalize PGExecCtx to support specialized functions for various operations

* fix tests compilation

* allow customising PGExecCtx when starting the web server

* server: changes catalog initialization and logging for pro customization (#5139)

* new typeclass to abstract the logic of QueryLog-ing

* abstract the logic of logging websocket-server logs

  introduce a MonadWSLog typeclass

* move catalog initialization to init step

  expose a helper function to migrate catalog
  create schema cache in initialiseCtx

* expose various modules and functions for pro

* [skip ci] cosmetic change

* [skip ci] fix test calling a mutation that does not exist

* [skip ci] minor text change

* [skip ci] refactored input values

* [skip ci] remove VString Origin

* server: fix updating of headers behaviour in the update cron trigger API and create future events immediately (#5151)

* server: fix bug to update headers in an existing cron trigger and create future events

Co-authored-by: Tirumarai Selvan <tiru@hasura.io>

* Lower stack chunk size in RTS to reduce thread STACK memory (closes #5190)

This reduces memory consumption for new idle subscriptions significantly
(see linked ticket).

The hypothesis is: we fork a lot of threads per websocket, and some of
these use slightly more than the initial 1K stack size, so the first
overflow balloons to 32K, when significantly less is required.

However: running with `+RTS -K1K -xc` did not seem to show evidence of
any overflows! So it's a mystery why this improves things.

GHC should probably also be doubling the stack buffer at each overflow
or doing something even smarter; the knobs we have aren't so helpful.

* [skip ci] fix todo and schema generation for aggregate fields

* 5087 libpq pool leak (#5089)

Shrink libpq buffers to 1MB before returning connection to pool. Closes #5087

See: https://github.com/hasura/pg-client-hs/pull/19

Also related: #3388 #4077

* bump pg-client-hs version (fixes a build issue on some environments) (#5267)

* do not use prepared statements for mutations

* server: unlock scheduled events on graceful shutdown (#4928)

* Fix buggy parsing of new --conn-lifetime flag in 2b0e3774

* [skip ci] remove cherry-picked commit from commit_diff.txt

* server: include additional fields in scheduled trigger webhook payload (#5262)

* include scheduled triggers metadata in the webhook body

Co-authored-by: Tirumarai Selvan <tiru@hasura.io>

* server: call the webhook asynchronously in event triggers (#5352)

* server: call the webhook asynchronosly in event triggers

* Expose all modules in Cabal file (#5371)

* [skip ci] update commit_diff.txt

* [skip ci] fix cast exp parser & few TODOs

* [skip ci] fix remote fields arguments

* [skip ci] fix few more TODO, no-op refactor, move resolve/action.hs to execute/action.hs

* Pass environment variables around as a data structure, via @sordina (#5374)

* Pass environment variables around as a data structure, via @sordina

* Resolving build error

* Adding Environment passing note to changelog

* Removing references to ILTPollerLog as this seems to have been reintroduced from a bad merge

* removing commented-out imports

* Language pragmas already set by project

* Linking async thread

* Apply suggestions from code review

Use `runQueryTx` instead of `runLazyTx` for queries.

* remove the non-user facing entry in the changelog

Co-authored-by: Phil Freeman <paf31@cantab.net>
Co-authored-by: Phil Freeman <phil@hasura.io>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>

* [skip ci] fix: restrict remote relationship field generation for hasura queries

* [skip ci] no-op refactor; move insert execution code from schema parser module

* server: call the webhook asynchronously in event triggers (#5352)

* server: call the webhook asynchronosly in event triggers

* Expose all modules in Cabal file (#5371)

* [skip ci] update commit_diff.txt

* Pass environment variables around as a data structure, via @sordina (#5374)

* Pass environment variables around as a data structure, via @sordina

* Resolving build error

* Adding Environment passing note to changelog

* Removing references to ILTPollerLog as this seems to have been reintroduced from a bad merge

* removing commented-out imports

* Language pragmas already set by project

* Linking async thread

* Apply suggestions from code review

Use `runQueryTx` instead of `runLazyTx` for queries.

* remove the non-user facing entry in the changelog

Co-authored-by: Phil Freeman <paf31@cantab.net>
Co-authored-by: Phil Freeman <phil@hasura.io>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>

* [skip ci] implement header checking

Probably closes #14 and #3659.

* server: refactor 'pollQuery' to have a hook to process 'PollDetails' (#5391)

Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>

* update pg-client (#5421)

* [skip ci] update commit_diff

* Fix latency buckets for telemetry data

These must have gotten messed up during a refactor. As a consequence
almost all samples received so far fall into the single erroneous 0 to
1K seconds (originally supposed to be 1ms?) bucket.

I also re-thought what the numbers should be, but these are still
arbitrary and might want adjusting in the future.

* [skip ci] include the latest commit compared against master in commit_diff

* [skip ci] include new commits from master in commit_diff

* [skip ci] improve description generation

* [skip ci] sort all introspect arrays

* [skip ci] allow parsers to specify error codes

* [skip ci] fix integer and float parsing error code

* [skip ci] scalar from json errors are now parse errors

* [skip ci] fixed negative integer error message and code

* [skip ci] Re-fix nullability in relationships

* [skip ci] no-op refactor and removed couple of FIXMEs

* [skip ci] uncomment code in 'deleteMetadataObject'

* [skip ci] Fix re-fix of nullability for relationships

* [skip ci] fix default arguments error code

* [skip ci] updated test error message

!!! WARNING !!!
Since all fields accept `null`, they all are technically optional in
the new schema. Meaning there's no such thing as a missing mandatory
field anymore: a field that doesn't have a default value, and which
therefore isn't labelled as "optional" in the schema, will be assumed
to be null if it's missing, meaning it isn't possible anymore to have
an error for a missing mandatory field. The only possible error is now
when a optional positional argument is omitted but is not the last
positional argument.

* [skip ci] cleanup of int scalar parser

* [skip ci] retro-compatibility of offset as string

* [skip ci] Remove commit from commit_diff.txt

Although strictly speaking we don't know if this will work correctly in PDV
if we would implement query plan caching, the fact is that in the theoretical
case that we would have the same issue in PDV, it would probably apply not just
to introspection, and the fix would be written completely differently.  So this
old commit is of no value to us other than the heads-up "make sure query plan
caching works correctly even in the presence of unused variables", which is
already part of the test suite.

* Add MonadTrace and MonadExecuteQuery abstractions (#5383)

* [skip ci] Fix accumulation of input object types

Just like object types, interface types, and union types, we have to avoid
circularities when collecting input types from the GraphQL AST.

Additionally, this fixes equality checks for input object types (whose fields
are unordered, and hence should be compared as sets) and enum types (ditto).

* [skip ci] fix fragment error path

* [skip ci] fix node error code

* [skip ci] fix paths in insert queries

* [skip ci] fix path in objects

* [skip ci] manually alter node id path for consistency

* [skip ci] more node error fixups

* [skip ci] one last relay error message fix

* [skip ci] update commit_diff

* Propagate the trace context to event triggers (#5409)

* Propagate the trace context to event triggers

* Handle missing trace and span IDs

* Store trace context as one LOCAL

* Add migrations

* Documentation

* changelog

* Fix warnings

* Respond to code review suggestions

* Respond to code review

* Undo changelog

* Update CHANGELOG.md

Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>

* server: log request/response sizes for event triggers (#5463)

* server: log request/response sizes for event triggers

  event triggers (and scheduled triggers) now have request/response size
  in their logs.

* add changelog entry

* Tracing: Simplify HTTP traced request (#5451)

Remove the Inversion of Control (SuspendRequest) and simplify
the tracing of HTTP Requests.

Co-authored-by: Phil Freeman <phil@hasura.io>

* Attach request ID as tracing metadata (#5456)

* Propagate the trace context to event triggers

* Handle missing trace and span IDs

* Store trace context as one LOCAL

* Add migrations

* Documentation

* Include the request ID as trace metadata

* changelog

* Fix warnings

* Respond to code review suggestions

* Respond to code review

* Undo changelog

* Update CHANGELOG.md

* Typo

Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>

* server: add logging for action handlers (#5471)

* server: add logging for action handlers

* add changelog entry

* change action-handler log type from internal to non-internal

* fix action-handler-log name

* server: pass http and websocket request to logging context (#5470)

* pass request body to logging context in all cases

* add message size logging on the websocket API

  this is required by graphql-engine-pro/#416

* message size logging on websocket API

  As we need to log all messages recieved/sent by the websocket server,
  it makes sense to log them as part of the websocket server event logs.
  Previously message recieved were logged inside the onMessage handler,
  and messages sent were logged only for "data" messages (as a server event log)

* fix review comments

Co-authored-by: Phil Freeman <phil@hasura.io>

* server: stop eventing subsystem threads when shutting down (#5479)

* server: stop eventing subsystem threads when shutting down

* Apply suggestions from code review

Co-authored-by: Karthikeyan Chinnakonda <chkarthikeyan95@gmail.com>

Co-authored-by: Phil Freeman <phil@hasura.io>
Co-authored-by: Phil Freeman <paf31@cantab.net>
Co-authored-by: Karthikeyan Chinnakonda <chkarthikeyan95@gmail.com>

* [skip ci] update commit_diff with new commits added in master

* Bugfix to support 0-size HASURA_GRAPHQL_QUERY_PLAN_CACHE_SIZE

Also some minor refactoring of bounded cache module:
 - the maxBound check in `trim` was confusing and unnecessary
 - consequently trim was unnecessary for lookupPure

Also add some basic tests

* Support only the bounded cache, with default HASURA_GRAPHQL_QUERY_PLAN_CACHE_SIZE of 4000. Closes #5363

* [skip ci] remove merge commit from commit_diff

* server: Fix compiler warning caused by GHC upgrade (#5489)

Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>

* [skip ci] update all non server code from master

* [skip ci] aligned object field error message with master

* [skip ci] fix remaining undefined?

* [skip ci] remove unused import

* [skip ci] revert to previous error message, fix tests

* Move nullableType/nonNullableType to Schema.hs

These are functions on Types, not on Parsers.

* [skip ci] fix setup to fix backend only test

the order in which permission checks are performed on the branch is
slightly different than on master, resulting in a slightly different
error if there are no other mutations the user has access to. By
adding update permissions, we go back to the expected case.

* [skip ci] fix insert geojson tests to reflect new paths

* [skip ci] fix enum test for better error message

* [skip ci] fix header test for better error message

* [skip ci] fix fragment cycle test for better error message

* [skip ci] fix error message for type mismatch

* [skip ci] fix variable path in test

* [skip ci] adjust tests after bug fix

* [skip ci] more tests fixing

* Add hdb_catalog.current_setting abstraction for reading Hasura settings

As the comment in the function’s definition explains, this is needed to
work around an awkward Postgres behavior.

* [skip ci] Update CONTRIBUTING.md to mention Node setup for Python tests

* [skip ci] Add missing Python tests env var to CONTRIBUTING.md

* [skip ci] fix order of result when subscription is run with multiple nodes

* [skip ci] no-op refactor: fix a warning in Internal/Parser.hs

* [skip ci] throw error when a subscription contains remote joins

* [skip ci] Enable easier profiling by hiding AssertNF behind a flag

In order to compile a profiling build, run:

$ cabal new-build -f profiling --enable-profiling

* [skip ci] Fix two warnings

We used to lookup the objects that implement a given interface by filtering all
objects in the schema document.  However, one of the tests expects us to
generate a warning if the provided `implements` field of an introspection query
specifies an object not implementing some interface.  So we use that field
instead.

* [skip ci] Fix warnings by commenting out query plan caching

* [skip ci] improve masking/commenting query caching related code & few warning fixes

* [skip ci] Fixed compiler warnings in graphql-parser-hs

* Sync non-Haskell assets with master

* [skip ci] add a test inserting invalid GraphQL but valid JSON value in a jsonb column

* [skip ci] Avoid converting to/from Map

* [skip ci] Apply some hlint suggestions

* [skip ci] remove redundant constraints from buildLiveQueryPlan and explainGQLQuery

* [skip ci] add NOTEs about missing Tracing constraints in PDV from master

* Remove -fdefer-typed-holes, fix warnings

* Update cabal.project.freeze

* Limit GHC’s heap size to 8GB in CI to avoid the OOM killer

* Commit package-lock.json for Python tests’ remote schema server

* restrict env variables start with HASURA_GRAPHQL_ for headers configuration in actions, event triggers & remote schemas (#5519)

* restrict env variables start with HASURA_GRAPHQL_ for headers definition in actions & event triggers

* update CHANGELOG.md

* Apply suggestions from code review

Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>

* add test for table_by_pk node when roles doesn't have permission to PK

* [skip ci] fix introspection query if any enum column present in primary key (fix #5200) (#5522)

* [skip ci] test case fix for a6450e126b

* [skip ci] add tests to agg queries when role doesn't have access to any cols

* fix backend test

* Simplify subscription execution

* [skip ci] add test to check if required headers are present while querying

* Suppose, table B is related to table A and to query B certain headers are
  necessary, then the test checks that we are throwing error when the header
  is not set when B is queried through A

* fix mutations not checking for view mutability

* [skip ci] add variable type checking and corresponding tests

* [skip ci] add test to check if update headers are present while doing an upsert

* [skip ci] add positive counterparts to some of the negative permission tests

* fix args missing their description in introspect

* [skip ci] Remove unused function; insert missing markNotReusable call

* [skip ci] Add a Note about InputValue

* [skip ci] Delete LegacySchema/ 🎉

* [skip ci] Delete GraphQL/{Resolve,Validate}/ 🎉

* [skip ci] Delete top-level Resolve/Validate modules; tidy .cabal file

* [skip ci] Delete LegacySchema top-level module

Somehow I missed this one.

* fix input value to json

* [skip ci] elaborate on JSON objects in GraphQL

* [skip ci] add missing file

* [skip ci] add a test with subscription containing remote joins

* add a test with remote joins in mutation output

* [skip ci] Add some comments to Schema/Mutation.hs

* [skip ci] Remove no longer needed code from RemoteServer.hs

* [skip ci] Use a helper function to generate conflict clause parsers

* [skip ci] fix type checker error in fields with default value

* capitalize the header keys in select_articles_without_required_headers

* Somehow, this was the reason the tests were failing. I have no idea, why!

* [skip ci] Add a long Note about optional fields and nullability

* Improve comments a bit; simplify Schema/Common.hs a bit

* [skip ci] full implementation of 5.8.5 type checking.

* [skip ci] fix validation test teardown

* [skip ci] fix schema stitching test

* fix remote schema ignoring enum nullability

* [skip ci] fix fieldOptional to not discard nullability

* revert nullability of use_spheroid

* fix comment

* add required remote fields with arguments for tests

* [skip ci] add missing docstrings

* [skip ci] fixed description of remote fields

* [skip ci] change docstring for consistency

* fix several schema inconsistencies

* revert behaviour change in function arguments parsing

* fix remaining nullability issues in new schema

* minor no-op refactor; use isListType from graphql-parser-hs

* use nullability of remote schema node, while creating a Remote reln

* fix 'ID' input coercing & action 'ID' type relationship mapping

* include ASTs in MonadExecuteQuery

* needed for PRO code-base

* Delete code for "interfaces implementing ifaces" (draft GraphQL spec)

Previously I started writing some code that adds support for a future GraphQL
feature where interfaces may themselves be sub-types of other interfaces.
However, this code was incomplete, and partially incorrect.  So this commit
deletes support for that entirely.

* Ignore a remote schema test during the upgrade/downgrade test

The PDV refactor does a better job at exposing a minimal set of types through
introspection.  In particular, not every type that is present in a remote schema
is re-exposed by Hasura.  The test
test_schema_stitching.py::TestRemoteSchemaBasic::test_introspection assumed that
all types were re-exposed, which is not required for GraphQL compatibility, in
order to test some aspect of our support for remote schemas.

So while this particular test has been updated on PDV, the PDV branch now does
not pass the old test, which we argue to be incorrect.  Hence this test is
disabled while we await a release, after which we can re-enable it.

This also re-enables a test that was previously disabled for similar, though
unrelated, reasons.

* add haddock documentation to the action's field parsers

* Deslecting some tests in server-upgrade

Some tests with current build are failing on server upgrade
which it should not. The response is more accurate than
what it was.

Also the upgrade tests were not throwing errors when the test is
expected to return an error, but succeeds. The test framework is
patched to catch this case.

* [skip ci] Add a long Note about interfaces and object types

* send the response headers back to client after running a query

* Deselect a few more tests during upgrade/downgrade test

* Update commit_diff.txt

* change log kind from db_migrate to catalog_migrate (#5531)

* Show method and complete URI in traced HTTP calls (#5525)

Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>

* restrict env variables start with HASURA_GRAPHQL_ for headers configuration in actions, event triggers & remote schemas (#5519)

* restrict env variables start with HASURA_GRAPHQL_ for headers definition in actions & event triggers

* update CHANGELOG.md

* Apply suggestions from code review

Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>

* fix introspection query if any enum column present in primary key (fix #5200) (#5522)

* Fix telemetry reporting of transport (websocket was reported as http)

* add log kinds in cli-migrations image (#5529)

* add log kinds in cli-migrations image

* give hint to resolve timeout error

* minor changes and CHANGELOG

* server: set hasura.tracecontext in RQL mutations [#5542] (#5555)

* server: set hasura.tracecontext in RQL mutations [#5542]

* Update test suite

Co-authored-by: Tirumarai Selvan <tiru@hasura.io>

* Add bulldozer auto-merge and -update configuration

We still need to add the github app (as of time of opening this PR)

Afterwards devs should be able to allow bulldozer to automatically
"update" the branch, merging in parent when it changes, as well as
automatically merge when all checks pass.

This is opt-in by adding the `auto-update-auto-merge` label to the PR.

* Remove 'bulldozer' config, try 'kodiak' for auto-merge

see: https://github.com/chdsbd/kodiak

The main issue that bit us was not being able to auto update forked
branches, also:
https://github.com/palantir/bulldozer/issues/66
https://github.com/palantir/bulldozer/issues/145

* Cherry-picked all commits

* [skip ci] Slightly improve formatting

* Revert "fix introspection query if any enum column present in primary key (fix #5200) (#5522)"

This reverts commit 0f9a5afa59.

This undoes a cherry-pick of 34288e1eb5 that was
already done previously in a6450e126b, and
subsequently fixed for PDV in 70e89dc250

* Do a small bit of tidying in Hasura.GraphQL.Parser.Collect

* Fix cherry-picking work

Some previous cherry-picks ended up modifying code that is commented out

* [skip ci] clarified comment regarding insert representation

* [skip ci] removed obsolete todos

* cosmetic change

* fix action error message

* [skip ci] remove obsolete comment

* [skip ci] synchronize stylish haskell extensions list

* use previously defined scalar names in parsers rather than ad-hoc literals

* Apply most syntax hlint hints.

* Clarify comment on update mutation.

* [skip ci] Clarify what fields should be specified for objects

* Update "_inc" description.

* Use record types rather than tuples fo IntrospectionResult and ParsedIntrospection

* Get rid of checkFieldNamesUnique (use Data.List.Extended.duplicates)

* Throw more errors when collecting query root names

* [skip ci] clean column parser comment

* Remove dead code inserted in ab65b39

* avoid converting to non-empty list where not needed

* add note and TODO about the disabled checks in PDV

* minor refactor in remoteField' function

* Unify two getObject methods

* Nitpicks in Remote.hs

* Update CHANGELOG.md

* Revert "Unify two getObject methods"

This reverts commit bd6bb40355.

We do need two different getObject functions as the corresponding error message is different

* Fix error message in Remote.hs

* Update CHANGELOG.md

Co-authored-by: Auke Booij <auke@tulcod.com>

* Apply suggested Changelog fix.

Co-authored-by: Auke Booij <auke@tulcod.com>

* Fix typo in Changelog.

* [skip ci] Update changelog.

* reuse type names to avoid duplication

* Fix Hashable instance for Definition

The presence of `Maybe Unique`, and an optional description, as part of
`Definition`s, means that `Definition`s that are considered `Eq`ual may get
different hashes.  This can happen, for instance, when one object is memoized
but another is not.

* [skip ci] Update commit_diff.txt

* Bump parser version.

* Bump freeze file after changes in parser.

* [skip ci] Incorporate commits from master

* Fix developer flag in server/cabal.project.freeze

Co-authored-by: Auke Booij <auke@tulcod.com>

* Deselect a changed ENUM test for upgrade/downgrade CI

* Deselect test here as well

* [skip ci] remove dead code

* Disable more tests for upgrade/downgrade

* Fix which test gets deselected

* Revert "Add hdb_catalog.current_setting abstraction for reading Hasura settings"

This reverts commit 66e85ab9fb.

* Remove circular reference in cabal.project.freeze

Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
Co-authored-by: Auke Booij <auke@hasura.io>
Co-authored-by: Tirumarai Selvan <tiru@hasura.io>
Co-authored-by: Marion Schleifer <marion@hasura.io>
Co-authored-by: Aleksandra Sikora <ola.zxcvbnm@gmail.com>
Co-authored-by: Brandon Simmons <brandon.m.simmons@gmail.com>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
Co-authored-by: Anon Ray <rayanon004@gmail.com>
Co-authored-by: rakeshkky <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Anon Ray <ecthiender@users.noreply.github.com>
Co-authored-by: Vamshi Surabhi <vamshi@hasura.io>
Co-authored-by: Antoine Leblanc <antoine@hasura.io>
Co-authored-by: Brandon Simmons <brandon@hasura.io>
Co-authored-by: Phil Freeman <phil@hasura.io>
Co-authored-by: Lyndon Maydwell <lyndon@sordina.net>
Co-authored-by: Phil Freeman <paf31@cantab.net>
Co-authored-by: Naveen Naidu <naveennaidu479@gmail.com>
Co-authored-by: Karthikeyan Chinnakonda <chkarthikeyan95@gmail.com>
Co-authored-by: Nizar Malangadan <nizar-m@users.noreply.github.com>
Co-authored-by: Antoine Leblanc <crucuny@gmail.com>
Co-authored-by: Auke Booij <auke@tulcod.com>
2020-08-21 12:27:01 -05:00
Aravind
1e3eb035d3
cli: fix tests failing due to a env var not being set in CI (#5015) 2020-06-11 08:35:41 +05:30
Brandon Simmons
5de647b926 Don't compile in developer APIs by default 2020-06-08 10:21:14 +05:30
Aravind
177556f441
cli: fix tests failing in local dev environment (#4729) 2020-06-03 14:01:40 +05:30
Aleksandra Sikora
45c8699e23
console: dependencies cleanup (#4705) 2020-06-02 13:44:45 +05:30
Tirumarai Selvan
c0d2bc6653
Remote Joins: Create relationships across database and remote schemas (#2392)
add remote joins: Create relationships across database and remote schemas (#2392)

Co-authored-by: Aleksandra Sikora <ola.zxcvbnm@gmail.com>

Co-authored-by: Chris Done <chrisdone@gmail.com>
Co-authored-by: Chris Done <github@chrisdone.com>
Co-authored-by: wawhal <rishichandra.wawhal@gmail.com>
Co-authored-by: Aravind Shankar <aravind@hasura.io>
Co-authored-by: Brandon Simmons <brandon.m.simmons@gmail.com>
Co-authored-by: Rishichandra Wawhal <rishi@hasura.io>
Co-authored-by: Brandon Simmons <brandon@hasura.io>
Co-authored-by: nizar-m <19857260+nizar-m@users.noreply.github.com>
Co-authored-by: Praveen Durairaju <praveend.web@gmail.com>
Co-authored-by: rakeshkky <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Anon Ray <rayanon004@gmail.com>
Co-authored-by: Shahidh K Muhammed <shahidh@hasura.io>
Co-authored-by: soorajshankar <soorajshankar@users.noreply.github.com>
Co-authored-by: Sooraj Sanker <sooraj@Soorajs-MacBook-Pro.local>
Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
Co-authored-by: Aleksandra Sikora <ola.zxcvbnm@gmail.com>
2020-05-27 20:32:58 +05:30
Vamshi Surabhi
ff99b24031 fix a security issue with 1.2 2020-05-22 14:40:57 +05:30
Brandon Simmons
ff62d5e0bf Migrate to GHC 8.10, upgrade dependencies. Closes #4517
This also seems to squash a stubborn space leak we see with
subscriptions (linking to canonical #3388 for reference).

This may also fix some of the "Unexpected exception" websockets
exceptions we are now surfacing (see e.g. #4344)

Also: dev.sh: fix hpc reporting

Initial work on this done by Vamshi.
2020-05-13 19:13:02 -04:00
Tirumarai Selvan
cc8e2ccc78
Scheduled triggers (close #1914) (#3553)
server: add scheduled triggers 

Co-authored-by: Alexis King <lexi.lambda@gmail.com>
Co-authored-by: Marion Schleifer <marion@hasura.io>
Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
Co-authored-by: Aleksandra Sikora <ola.zxcvbnm@gmail.com>
2020-05-13 18:03:16 +05:30
Aravind Shankar
ab97dee4b0
add build_image as required job for test_and_build_cli_migrations (#4615) 2020-04-30 12:21:56 +05:30
Auke Booij
ad882a93e2
tests: tear down numerics values (fix #4602) (#4604)
Also temporarily disable part of the CI to avoid it becoming confused about tests for/from old versions
2020-04-29 16:56:16 +02:00
Antoine Leblanc
66e18b7dee
console: support materialized views (close #91) (#4270) 2020-04-24 15:13:42 +05:30
Aravind Shankar
455b78552d
cli(test): refactor integration test metadata directory (#4491) 2020-04-22 16:58:02 +05:30
Aravind Shankar
2a693be553
cli(test): fix cli init test to use initDir from subtest group (#4490) 2020-04-22 15:54:24 +05:30
Aravind Shankar
0513fd39bf
ci: add force flag while creating pull request using hub (#4431) 2020-04-22 15:53:47 +05:30
Karthikeyan Chinnakonda
ced6275ff0
add additional tests for testing claims_namespace_path in JWT tokens (#4481)
* add additional tests for testing claims_namespace_path in JWT tokens

- add tests for at root level and at a nested level

* modify the JWT tests

* combine the claims_namespace_path tests together in test-server.sh

* change the order of the claims_namespace_path tests

* change the order of the claims_namespace_path tests
2020-04-21 14:54:35 +05:30
Rikin Kachhia
3c9d1f2330
ci: skip server tests in PRs if there are no server changes (#4412) 2020-04-17 13:18:40 -05:00
Karthikeyan Chinnakonda
a26bc80496
accept a new argument claims_namespace_path in JWT config (#4365)
* add new optional field `claims_namespace_path` in JWT config

* return value when empty array is found in executeJSONPath

* update the docs related to claims_namespace_path

* improve encodeJSONPath, add property tests for parseJSONPath

* throw error if both claims_namespace_path and claims_namespace are set

* refactor the Data.Parser.JsonPath to Data.Parser.JSONPathSpec

* update the JWT docs

Co-Authored-By: Marion Schleifer <marion@hasura.io>

Co-authored-by: Marion Schleifer <marion@hasura.io>
Co-authored-by: rakeshkky <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Tirumarai Selvan <tirumarai.selvan@gmail.com>
2020-04-16 12:15:21 +05:30
Aravind Shankar
1ada6627b0
ci: add tests for cli-migrations image (#4396)
- Created new job test_and_build_cli_migrations which runs after test_and_build_cli
- Build the cli-migrations and cli-migrations-v2 and save the images as tar image.
- Run the test defined in each workflow v1 and v2.
- Load the image that was built earlier in deploy step
2020-04-15 07:26:00 +05:30
Aravind Shankar
491a50b1e9
cli: link cli-ext plugin with cli version (close #4105) (#4280) 2020-04-13 08:46:06 +05:30
Tirumarai Selvan
76fbe90b60
type is not required for jwk_url in JWT config (#4334)
* type is not required for jwk_url

* remove type from JWTConfig

* Omit type field in JWTConfig serialization if jwk_url is provided

* remove type from jwk_url test suite

* add changelog

* fix docs with new format

Co-authored-by: Alexis King <lexi.lambda@gmail.com>
2020-04-10 19:25:59 +05:30
nizar-m
0e666a9203
Fix catalog version for v1.1.1 (close #4354) (#4355)
* Fix catalog version for v1.1.1

* Remove entries of removed tables from hdb_catalog

While downgrading catalog version from 32 -> 31, not removing entries
in hdb_table and hdb_relationship for the tables that are removed in
the downgrade, results in incosistent schema, when the server with
downgraded version is started. This should probably be handled in
a better fashion.

With the change in this commit, the server is able to successfully
start with downgraded catalog version 31.

* Test downgrade command along with upgrade tests
2020-04-09 19:57:59 -05:00
Aravind Shankar
69090427b9
ci: use hub instead of ghr for releases (#4241) 2020-04-02 07:12:18 +05:30
Aravind Shankar
863910fed2
ci: fix console hot-fix releases (#4208) 2020-04-01 18:26:54 +05:30
Shahidh K Muhammed
5bc95d2b7a
add workflow automations, update changelog (#4079)
Co-authored-by: Aravind Shankar <aravind@hasura.io>
2020-03-29 12:01:30 +05:30
Aravind Shankar
c0902a57d3 fix merge conflict 2020-03-25 18:32:48 +05:30
Aravind Shankar
0a0dfd4768
ci: fix deploy script to upload github release assets (#4103) 2020-03-25 16:20:59 +05:30
Aravind Shankar
68a8fada61 fix function name 2020-03-25 15:56:43 +05:30
Aravind Shankar
d36cc16150 fix deploy script 2020-03-19 17:58:29 +05:30
Aravind Shankar
315f19738c add support for cli-migrations-v2 2020-03-19 17:51:53 +05:30
Brandon Simmons
2d3313e643
Try again to fix flaky TestEventFlood test (#4088)
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
2020-03-18 22:45:41 +05:30
Aravind Shankar
bca029d00e
ci: fix ciignore script to ignore certain directories (#4086) 2020-03-13 13:55:47 +05:30
nizar-m
db724f719d
Server upgrade tests: Do not fail it no tests were collected (#4071)
Co-authored-by: Nizar Malangadan <nizar-m@users.noreply.github.com>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
2020-03-10 12:31:26 +05:30
Shahidh K Muhammed
b87063dc05
build: rename file to adhere to windows rules (close #4002) (#4003) 2020-03-03 12:33:21 +05:30
Aravind Shankar
21e6dd7759
cli: bug fixes related to actions (#3951) 2020-02-25 20:28:28 +05:30
Aravind Shankar
89b6ce86ca
cli: misc fixes related to actions (#3942) 2020-02-25 12:16:11 +05:30
Aravind Shankar
bb63d7e60e
cli: allow managing actions (#3859)
Co-authored-by: Rishichandra Wawhal <rishichandra.wawhal@gmail.com>
Co-authored-by: Rikin Kachhia <54616969+rikinsk@users.noreply.github.com>
Co-authored-by: Aravind <aravindkp@outlook.in>
Co-authored-by: Anon Ray <ecthiender@users.noreply.github.com>
Co-authored-by: Shahidh K Muhammed <muhammedshahid.k@gmail.com>
2020-02-24 21:44:46 +05:30
Vamshi Surabhi
b84db36ebb
allow custom mutations through actions (#3042)
* basic doc for actions

* custom_types, sync and async actions

* switch to graphql-parser-hs on github

* update docs

* metadata import/export

* webhook calls are now supported

* relationships in sync actions

* initialise.sql is now in sync with the migration file

* fix metadata tests

* allow specifying arguments of actions

* fix blacklist check on check_build_worthiness job

* track custom_types and actions related tables

* handlers are now triggered on async actions

* default to pgjson unless a field is involved in relationships, for generating definition list

* use 'true' for action filter for non admin role

* fix create_action_permission sql query

* drop permissions when dropping an action

* add a hdb_role view (and relationships) to fetch all roles in the system

* rename 'webhook' key in action definition to 'handler'

* allow templating actions wehook URLs with env vars

* add 'update_action' /v1/query type

* allow forwarding client headers by setting `forward_client_headers` in action definition

* add 'headers' configuration in action definition

* handle webhook error response based on status codes

* support array relationships for custom types

* implement single row mutation, see https://github.com/hasura/graphql-engine/issues/3731

* single row mutation: rename 'pk_columns' -> 'columns' and no-op refactor

* use top level primary key inputs for delete_by_pk & account select permissions for single row mutations

* use only REST semantics to resolve the webhook response

* use 'pk_columns' instead of 'columns' for update_by_pk input

* add python basic tests for single row mutations

* add action context (name) in webhook payload

* Async action response is accessible for non admin roles only if
  the request session vars equals to action's

* clean nulls, empty arrays for actions, custom types in export metadata

* async action mutation returns only the UUID of the action

* unit tests for URL template parser

* Basic sync actions python tests

* fix output in async query & add async tests

* add admin secret header in async actions python test

* document async action architecture in Resolve/Action.hs file

* support actions returning array of objects

* tests for list type response actions

* update docs with actions and custom types metadata API reference

* update actions python tests as per #f8e1330

Co-authored-by: Tirumarai Selvan <tirumarai.selvan@gmail.com>
Co-authored-by: Aravind Shankar <face11301@gmail.com>
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
2020-02-13 23:08:23 +05:30
Tirumarai Selvan
f8e133070b
run default tests in test_server_upgrade (#3718)
* run basic tests after upgrade

* terminate before specifying file in pytest cmd

* Move fixture definitions out of test classes

Previously we had abstract classes with the fixtures defined
in them. The test classes then inherits these super classes. This
is creating inheritence problems, especially when you want to just
inherit the tests in class, but not the fixtures. We have now moved
all those fixture definitions outside of the class (in conftest.py).
These fixtures are now used by the test classes when and where they
are required.

* Run pytests on server upgrade

Server upgrade tests are run by
  1) Run pytest with schema/metadata setup but do not do schema/metadata
teardown
  2) Upgrade the server
  3) Run pytest using the above schema and teardown at the end of the
tests
  4) Cleanup hasura metadata and start again with next set of tests

We have added options --skip-schema-setup and --skip-schema-teardown to
help running server upgrade tests.

While running the tests, we noticed that error codes and messages for
some of the tests have changed. So we have added another option to
pytest `--avoid-error-message-checks`. If this flag is set, and if
comparing expected and response message fails, and if the expected
response has an error message, Pytest will throw warnings instead of an
error.

* Use marks to specify server-upgrade tests

Not all tests can be run as serve upgrade tests, particularly those
which themselves change the schema. We introduce two pytest markers.
Marker allow_server_upgrade_test will add the test into the list of
server  upgrade  tests  that  can  be run. skip_server_upgrade_test
removes it from the list.

With this we have added tests for queries, mutations, and selected
event trigger and remote schema tests to the list of server upgrade
tests.

* Remove components not needed anymore

* Install curl

* Fix error in query validation

* Fix error in test_v1_queries.py

* install procps for server upgrade tests

* Use postgres image which has postgis installed

* set pager off with psql

* quote the bash variable WORKTREE_DIR

Co-authored-by: nizar-m <19857260+nizar-m@users.noreply.github.com>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
2020-02-13 14:44:02 +05:30
Anon Ray
c6e34baad8
fix parsing JWK expiry time from headers on startup (fix #3655) (#3779) 2020-02-05 12:37:30 +05:30
Aravind Shankar
c00a3d47ff circleci: update workflow id (#3745) 2020-01-20 15:34:00 +05:30
Brandon Simmons
1dd63a9386 Switch from stack to cabal-install for building server code (close #3280) (#3558)
Co-authored-by: Alexis King <lexi.lambda@gmail.com>
2020-01-17 16:07:15 -06:00
Shahidh K Muhammed
9c3e9ff4be
ci: push the latest assets only on stable releases (close #3714) (#3716)
[skip ci]
2020-01-16 11:35:35 +05:30
Tirumarai Selvan
66beb02f7b remove brotli from packaging and CI images (#3673)
* Revert "add brotli shared lib to packager image (#2924)"
This reverts commit 79414cb225.
* remove unneeded dockerfile
* remove brotli traces in ci images and compression test
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
2020-01-10 15:15:47 -06:00
Alexis King
27997107ab Add caching for recreating event trigger functions 2020-01-08 16:45:46 -06:00
Alexis King
447bac74e7 Split up Hasura.RQL.DDL.Schema.Cache
This should hopefully improve compile times by avoiding the need to
specialize everything at once.
2020-01-08 16:45:46 -06:00
Alexis King
f2963a25c3 Add Hasura.Incremental, a library for incremental builds 2020-01-08 16:43:06 -06:00
Phil Freeman
b3e0595a9d Test with postgis 3.0.0 (#3519) 2019-12-18 19:32:15 -06:00
Rakesh Emmadi
421a182f64 export metadata without nulls, empty arrays & default values (#3393)
* export metadata without nulls, empty arrays
* property tests for 'ReplaceMetadata' using QuickCheck
-> Derive Arbitrary class for 'ReplaceMetadata' dependant types

* reduce property test cases number to 30
QuickCheck generates the `ReplaceMetadata` value really large
for higher number test cases. Encoded JSON for such values is large and
consumes more memory. Thus, CI is giving up while running property
tests.

* circle-ci: Add property tests as saperate job
* add no command mode to tests
* add yaml.v2 to go mod
* remove indirect comment for yaml.v2 dependency
2019-12-14 00:47:38 -06:00
Ashish Rao
3d3e987c2a cli(build): migrate to go modules (close #3476) (#3493) 2019-12-12 11:07:26 +05:30
Anon Ray
afd6f30e72 read cache control header to refresh JWK (fix #3301) (#3446) 2019-12-03 14:56:59 -06:00
nizar-m
9faadec005 remove usage of deprecated 'pytest.config' (#3434)
pytest is now at version 5.3
2019-11-29 10:44:26 +05:30
Anon Ray
490b639981 refactor some internal components (#3414) 2019-11-26 17:44:21 +05:30
Alexis King
d6a649095c
Build server in CI with optimizations, even in pull requests (#3240) 2019-10-30 11:07:57 -05:00
Shahidh K Muhammed
31e0225230
add tests with postgres 12 (close #2749) (#3102) 2019-10-26 11:09:57 +05:30
Rikin Kachhia
8000652a5d
fix console test environment (#3192) 2019-10-22 11:28:35 +05:30
Rakesh Emmadi
6d92e4f9db save permissions, relationships and collections in catalog with 'is_system_defined' explicitly (#3165)
* save permissions, relationships and collections in catalog with 'is_system_defined'
* Use common stanzas in the .cabal file
* Refactor migration code into lib instead of exe
* Add new server test suite that exercises migrations
* Make graphql-engine clean succeed even if the schema does not exist
2019-10-21 11:01:05 -05:00
Alexis King
e01008413e
Track variable uses within query validation for caching (fix #3097) (#3135)
This fix is a little ugly, but it’s the only simple solution without a
significant refactoring that restructures the relationship between
GraphQL/Validate and GraphQL/Resolve. The ugliness should go away if we
implement something like #2801.
2019-10-16 09:33:34 -05:00
nizar-m
44da458c81 fix hpc combine error (close #2946) (#2947)
* Fix hpc combine error

* Do not perform ciignore

* xfail test jsonb_has_all

* Bring back ciignore

* Refer jsonb_has_all xfaul to the corresponding issue in graphql-engine-internal
2019-10-02 12:06:27 +05:30
Rakesh Emmadi
8a0615ff70 add gzip brotli compression to http responses (close #2674) (#2751) 2019-09-19 18:24:40 +05:30
Alexis King
ed26da59a6 Add support for GraphQL enum types via enum table references
These changes also add a new type, PGColumnType, between PGColInfo and
PGScalarType, and they process PGRawColumnType values into PGColumnType
values during schema cache generation.
2019-08-26 00:54:56 -05:00
José Lorenzo Rodríguez
c7a2320456 Implemented graceful shutdown for HTTP requests (close #2698) (#2717)
* Listens for SIGTERM as the termination signal
* Stops accepting new connections once the signal is received
* Waits for all connections to be drained, before shutting down
* Forcefully kills all pending connections after 30 seconds

Currently this does not send a close message to websocket clients, I'd
like to submit that change as a separate pull request, but at least this
solve my biggest concern which is not getting confirmation for mutations
while restarting the server.
2019-08-26 00:31:27 -05:00
Shahidh K Muhammed
3042fd175e
build latest.cli-migrations image on release (close #2591) (#2707) 2019-08-10 12:39:27 +00:00
Vamshi Surabhi
66a2f33fa9 use all available cpu cores by default (#2511)
Use all available cores by default. sets -N rts flag by default. Can be disabled by
setting the `GHCRTS` env variable to `-N1`.
2019-07-12 05:25:15 +00:00
Anon Ray
f2f14e727b Merge pull request from GHSA-2j98-fw5g-j43v
* fix bug in audience check while verifying JWT

  - previously the check was converting the audience type into a string
  and then comparing with the conf value. all audience types (as it is a
  string or URI) will convert to plain strings
  - use the Audience type from the jose library for comparing

* add docs for audience

* add issuer check as well

* docs minor syntax fix

* skip audience check if not given in conf

* minor docs update

* qualify import jose library
2019-07-11 09:58:39 +00:00
Anon Ray
8f1e7dbc8d breaking: server logging changes (close #507, close #2171) (#1835) 2019-07-11 05:37:06 +00:00
Shahidh K Muhammed
76ceb707f4
bundle console assets into server (close #516, close #521, close #2130) (#2192)
This PR builds console static assets into the server docker image at `/srv/console-assets`. When env var `HASURA_GRAPHQL_CONSOLE_ASSETS_DIR=/srv/console-assets` or flag `--console-assets-dir=/srv/console-assets` is set on the server, the files in this directory are served at `/console/assets/*`.

The console html template will have a variable called `cdnAssets: false` when this flag is set and it loads assets from server itself instead of CDN.

The assets are moved to a new bucket with a new naming scheme:

```
graphql-engine-cdn.hasura.io/console/assets/
   /common/{}
   /versioned/<version/{}
   /channel/<channel>/<version>/{}
```

Console served by CLI will still load assets from CDN - will fix that in the next release.
2019-05-16 13:15:29 +05:30
Rakesh Emmadi
0b210cc245 support allow-list for graphql queries (closes #989) (#2075) 2019-05-16 11:43:25 +05:30
Rakesh Emmadi
204cd3514b optimise server startup time (close #1430) (#2120)
1. Reuses postgres connections during startup which reduces the overhead of opening and closing connections. 
2. Faster schema cache building. This is done by fetching all the required data in a single sql statement.
2019-05-08 13:06:42 +05:30
Shahidh K Muhammed
71cf017197 add an api to dump postgres schema (close #1939) (#1967) 2019-04-30 14:04:08 +05:30
nizar-m
cadc41b09d Use UTF-8 encoding for stack hpc report generation (#1974) 2019-04-10 14:41:50 +05:30
nizar-m
a40bf10b9f run graphql tests on both http and websocket; add parallelism (close #1868) (#1921)
Examples 
1)  `
pytest --hge-urls "http://127.0.0.1:8080" --pg-urls "postgresql://admin@127.0.0.1:5432/hge_tests" -vv
`
2)  `pytest --hge-urls "http://127.0.0.1:8080"   "http://127.0.0.1:8081" --pg-urls "postgresql://admin@127.0.0.1:5432/hge_tests"  "postgresql://admin@127.0.0.1:5432/hge_tests2" -vv
`
### Solution and Design
<!-- How is this issue solved/fixed? What is the design? -->
<!-- It's better if we elaborate -->
#### Reducing execution time of tests
- The Schema setup and teardown, which were earlier done per test method, usually takes around 1 sec. 
- For mutations, the model has now been changed to only do schema setup and teardown once per test class.
-  A data setup and teardown will be done once per test instead (usually takes ~10ms).
- For the test class to get this behaviour, one can can extend the class `DefaultTestMutations`. 
    - The function  `dir()` should be define which returns the location of the configuration folder.
    - Inside the configuration folder, there should be 
        - Files `<conf_dir>/schema_setup.yaml` and `<conf_dir>/schema_teardown.yaml`, which has the metadata query executed during schema setup and teardown respectively
        - Files named `<conf_dir>/values_setup.yaml` and `<conf_dir>/values_teardown.yaml`. These files are executed to setup and remove data from the tables respectively. 

#### Running Graphql queries on both http and websockets
- Each GraphQL query/mutation is run on the both HTTP and websocket protocols
- Pytests test parameterisation is used to achieve this
- The errors over websockets are slightly different from that on HTTP
   - The code takes care of converting the errors in HTTP to errors in websockets

#### Parallel executation of tests.
- The plugin pytest-xdist helps in running tests on parallel workers.
- We are using this plugin to group tests by file and run on different workers.
- Parallel test worker processes operate on separate postgres databases(and separate graphql-engines connected to these databases). Thus tests on one worker will not affect the tests on the other worker.
- With two workers, this decreases execution times by half, as the tests on event triggers usually takes a long time, but does not consume much CPU.
2019-04-08 12:52:38 +05:30
Shahidh K Muhammed
2274e2748b Revert "reduce server build time on ci (#1836)"
This reverts commit cbd6bb1f15.
2019-03-25 09:49:23 +05:30
Anon Ray
cbd6bb1f15 reduce server build time on ci (#1836)
Due to a misconfiguration in the caching logic, a server was building dependencies on every build. This fixes it to reduce server build time.
2019-03-22 18:11:15 +05:30
Vamshi Surabhi
c7346fd55a bump stackage to lts 13 and refer to hasura's pg-client-hs (#1747) 2019-03-14 20:25:33 +05:30
Rakesh Emmadi
e32f5a1fb1 sync metadata cache across multiple instances connected to same db (closes #1182) (#1574)
1. Haskel library `pg-client-hs` has been updated to expose a function that helps listen to `postgres` notifications over a `channel` in this [PR](https://github.com/hasura/pg-client-hs/pull/5)
2. The server records an event in a table `hdb_catalog.hdb_cache_update_event` whenever any `/v1/query` (that changes metadata) is requested. A trigger notifies a `cache update` event via `hasura_cache_update` channel
3. The server runs two concurrent threads namely `listener` and `processor`. The `listener` thread listens to events on `hasura_cache_update` channel and pushed into a `Queue`. The `processor` thread fetches events from that `Queue` and processes it. Thus server rebuilds schema cache from database and updates.
2019-03-12 11:16:27 +05:30
Shahidh K Muhammed
a83adf6503 skip ciignore check for release branches 2019-03-07 21:47:27 +05:30
Shahidh K Muhammed
a1089c1ec5
support ciignore for master builds too (#1708) 2019-03-07 17:48:06 +05:30
Shahidh K Muhammed
17183caed4
adds a check to see if the jobs should be run or not (close #1161) (#1705)
CircleCI jobs are run for any PR that is submitted to the repo. This PR adds a check to decide whether the job should be run or not.

Figured out that CircleCI has a way to gracefully terminate a job:
```
circleci-agent step halt
```

A `.ciignore` file is ran against all the changes in the PR to decide whether the PR should be built or not. If the answer comes out as `no`, a file is written at `/buid/skip_job.txt`. This is done in the `check_build_worthiness` step.

All further jobs, in the beginning, looks for this file and gracefully terminates the job if this file is present. The directory is passed down to the jobs as the workspace.

```yaml
  skip_job_on_ciignore: &skip_job_on_ciignore
    run: |
      if [ -f /build/skip_job.txt ]; then
        echo "halting job due to /build/skip_job.txt"
        circleci-agent step halt
      fi
```

ref: https://support.circleci.com/hc/en-us/articles/360015562253-Conditionally-end-a-running-job-gracefully

There are some known issues on jobs that are run when PR is merged to master, need to address them after this PR is merged.
2019-03-07 11:28:03 +05:30
Anon Ray
02d80c9ac6 read cookie while initialising websocket connection (fix #1660) (#1668)
* read cookie while initialising websocket connection (fix #1660)

* add tests for cookie on websocket init

* fix logic for tests

* enforce cors, and flag to force read cookie when cors disabled

  - as browsers don't enforce SOP on websockets, we enforce CORS policy
  on websocket handshake
  - if CORS is disabled, by default cookie is not read (because XSS
  risk!). Add special flag to force override this behaviour

* add log and forward origin header to webhook

  - add log notice when cors is disabled, and cookie is not read on
  websocket handshake
  - forward origin header to webhook in POST mode. So that when CORS is
  disabled, webhook can also enforce CORS independently.

* add docs, and forward all client headers to webhook
2019-03-04 13:16:53 +05:30
Rakesh Emmadi
377290a058 breaking: correct (de)serialisation of postgres numeric types in json (fix #1523) (#1662) 2019-03-01 17:15:04 +05:30
nizar-m
1fa66dc622 add option to disable metadata and graphql apis (close #1088) (#1650) 2019-02-28 19:23:03 +05:30
Shahidh K Muhammed
fdd5784bf7
test server upgrade from last release (close #570) (#1667) 2019-02-27 20:42:47 +05:30