Commit Graph

1001 Commits

Author SHA1 Message Date
Antoine Leblanc
405b29c82d server: generate backend-related code with TemplateHaskell
GitOrigin-RevId: 251b8dbeb5181e42cca0369abf54bfc2179ad3d8
2021-03-18 20:35:23 +00:00
Vladimir Ciobanu
f0dd71db59 server: mssql_add_source: allow connection strings from environment
GitOrigin-RevId: d51e14bdfbb57b83a453f5954397792cbe81455b
2021-03-18 18:33:48 +00:00
Antoine Leblanc
1abe5d4902 server: combine instance imports
GitOrigin-RevId: 4deab805ceef1202e5a2d04f0568fa33df3ff791
2021-03-18 17:40:45 +00:00
Karthikeyan Chinnakonda
395de58c42 server: fix inherited_roles issue when some of the underlying roles don't have permissions configured
GitOrigin-RevId: 771491ec0cf6c0d860ef374ed67b426220fa2b96
2021-03-18 15:29:25 +00:00
Karthikeyan Chinnakonda
478c01ab9b server: support tracking functions having only base type arguments
GitOrigin-RevId: e8939ad25212fc0965e0912fc88721a0cdff844e
2021-03-18 12:50:52 +00:00
Tirumarai Selvan
e56e606ddb tag release v2.0.0-alpha.5 (#890)
GitOrigin-RevId: 63d7ffdb732360b96b3dc5ee670b4bb9cf1ef613
2021-03-18 06:30:20 +00:00
Abby Sassel
89d838b85a server: split BoolExpBasic tests into common and backend-specific classes
GitOrigin-RevId: 9e62260881de63e5f89007bb1d6607254625cda7
2021-03-17 10:57:06 +00:00
Vladimir Ciobanu
91710bba58 server: use relative paths in TH splices
While debugging issues with HLS, Reed Mullanix noticed that we don't use relative paths. This leads to problems when using HLS + Emacs due to a bug in `lsp-mode` which prevents it from finding the correct project root.

However, it is still a good practice to use relative paths in TH for other reasons, including being able to import these modules in GHCI.

This PR should make it so HLS-1.0 & emacs provide type inference, imports, etc., in all modules in our codebase.

GitOrigin-RevId: 5f53b9a7ccf46df1ea7be94ff0a5c6ec861f4ead
2021-03-16 17:36:39 +00:00
Abby Sassel
881cc55e0e server: split TestGraphQLQueryBasic into common and backend-specific test classes
GitOrigin-RevId: 9e99047f7ab83f06c94c4741c7c0591eb2ca35e8
2021-03-16 16:42:20 +00:00
Naveen Naidu
babb05451a multitenant: change default connection pool params
GitOrigin-RevId: 44d005c2d8e941d955f4654f21da091e5c1520ae
2021-03-16 15:29:43 +00:00
Karthikeyan Chinnakonda
d1a2fabb26 server: fix the FromJSON instance of ExperimentalFeature
GitOrigin-RevId: 4d3c296021bfc26295632fb4b331c1fe0c902734
2021-03-16 11:49:00 +00:00
Solomon Bothwell
e4b669e56b Traverses RFAction RootField cnstr to accumulate session variables
GitOrigin-RevId: 148bc50c59820254dc40b8951e43abadcac2c25b
2021-03-16 06:13:31 +00:00
Vladimir Ciobanu
da8f6981d4 server: reduce the number of backend dispatches
Fixes https://github.com/hasura/graphql-engine-mono/issues/712

Main point of interest: the `Hasura.SQL.Backend` module.

This PR creates an `Exists` type indexed by indexed type and packed constraint while hiding all of its complexity by not exporting the constructor.

Existential constructors/types which are no longer (directly) existential:
- [X] BackendSourceInfo :: BackendSourceInfo
- [x] BackendSourceMetadata :: BackendSourceMetadata
- [x] MOSourceObjId :: MetadatObjId
- [x] SOSourceObj :: SchemaObjId
- [x] RFDB :: RootField
- [x] LQP :: LiveQueryPlan
- [x] ExecutionStep :: ExecStepDB

This PR also removes ALL usages of `Typeable.cast` from our codebase. We still need to derive `Typeable` in a few places in order to be able to derive `Data` in one place. I have not dug deeper to see why this is needed.

GitOrigin-RevId: bb47e957192e4bb0af4c4116aee7bb92f7983445
2021-03-15 13:03:55 +00:00
Karthikeyan Chinnakonda
6093010a6e server: fix issue of parsing input objects in remote schema queries
GitOrigin-RevId: 98c62cab836aff2778dbfedd6535450989090a30
2021-03-15 06:32:07 +00:00
Antoine Leblanc
a544fe9822 server: generalized LogQueryLog (fix #708)
GitOrigin-RevId: 0e18e0f5caaa6889804debf7111ef9aa99e080b5
2021-03-13 14:41:44 +00:00
Abby Sassel
64d52f5fa3 server: generalize integration tests
Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
GitOrigin-RevId: 647edb8d293bf783c504b7e15ce02c56858b3b72
2021-03-11 18:18:41 +00:00
Abby Sassel
660b442943 server: misc. dev.sh improvements
Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
GitOrigin-RevId: 4c93b49a370afb98747aabc1c977494297432402
2021-03-10 22:19:14 +00:00
Karthikeyan Chinnakonda
84d55cf771 docs: add a guide on writing python tests
Co-authored-by: Rikin Kachhia <54616969+rikinsk@users.noreply.github.com>
GitOrigin-RevId: 37528597f947a62b4c119cad8feb5428b904a00b
2021-03-10 17:30:41 +00:00
Tirumarai Selvan
692e374c94 tag release v2.0.0-alpha.4
GitOrigin-RevId: fdde115ba4442795494566f2d6a2aacbf50f0632
2021-03-10 11:48:44 +00:00
Anon Ray
d16bc4fe46 server: fix handling of nullable object relationships
GitOrigin-RevId: 3214e77ed06a07b6c1293d59daa088649a881e78
2021-03-10 08:56:03 +00:00
Rakesh Emmadi
2b0c990d50 server: remove Action.hs-boot file
GitOrigin-RevId: 65267a7b6343a12b690f7710785fdfa30873e479
2021-03-10 07:27:19 +00:00
Lyndon Maydwell
0c1016e065 Inconsistent metadata support for REST endpoints
Previously invalid REST endpoints would throw errors during schema cache build.

This PR changes the validation to instead add to the inconsistent metadata objects in order to allow use of `allow_inconsistent_metadata` with inconsistent REST endpoints.

All non-fatal endpoint definition errors are returned as inconsistent metadata warnings/errors depending on the use of `allow_inconsistent_metadata`. The endpoints with issues are then created and return informational runtime errors when they are called.

Console impact when creating endpoints is that error messages now refer to metadata inconsistencies rather than REST feature at the top level:

![image](https://user-images.githubusercontent.com/92299/109911843-ede9ec00-7cfe-11eb-9c55-7cf924d662a6.png)

<img width="969" alt="image" src="https://user-images.githubusercontent.com/92299/110258597-8336fa00-7ff7-11eb-872c-bfca945aa0e8.png">

Note: Conflicting endpoints generate one error per conflicting set of endpoints due to the implementation of `groupInconsistentMetadataById` and `imObjectIds`. This is done to ensure that error messages are terse, but may pose errors if there are some assumptions made surrounding `imObjectIds`.

Related to https://github.com/hasura/graphql-engine-mono/pull/473 (Allow Inconsistent Metadata (v2) #473 (Merged))

---

### Kodiak commit message

Changes the validation to use inconsistent metadata objects for REST endpoint issues.

#### Commit title

Inconsistent metadata for REST endpoints

GitOrigin-RevId: b9de971208e9bb0a319c57df8dace44cb115ff66
2021-03-10 05:26:10 +00:00
Karthikeyan Chinnakonda
71beba0c54 server: add experimental features in the alpha config API
GitOrigin-RevId: b4841e3dbea8340f8fad638359f3b1cb28188db6
2021-03-09 18:46:42 +00:00
Karthikeyan Chinnakonda
0a7d634326 server: fix issue when remote relationship col has a custom GQL name
GitOrigin-RevId: ca45383020d0e8f80afb445ab8bb240cc1505ea9
2021-03-09 09:25:29 +00:00
Karthikeyan Chinnakonda
92026b769f [Preview] Inherited roles for postgres read queries
fixes #3868

docker image - `hasura/graphql-engine:inherited-roles-preview-48b73a2de`

Note:

To be able to use the inherited roles feature, the graphql-engine should be started with the env variable `HASURA_GRAPHQL_EXPERIMENTAL_FEATURES` set to `inherited_roles`.

Introduction
------------

This PR implements the idea of multiple roles as presented in this [paper](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/FGALanguageICDE07.pdf). The multiple roles feature in this PR can be used via inherited roles. An inherited role is a role which can be created by combining multiple singular roles. For example, if there are two roles `author` and `editor` configured in the graphql-engine, then we can create a inherited role with the name of `combined_author_editor` role which will combine the select permissions of the `author` and `editor` roles and then make GraphQL queries using the `combined_author_editor`.

How are select permissions of different roles are combined?
------------------------------------------------------------

A select permission includes 5 things:

1. Columns accessible to the role
2. Row selection filter
3. Limit
4. Allow aggregation
5. Scalar computed fields accessible to the role

 Suppose there are two roles, `role1` gives access to the `address` column with row filter `P1` and `role2` gives access to both the `address` and the `phone` column with row filter `P2` and we create a new role `combined_roles` which combines `role1` and `role2`.

Let's say the following GraphQL query is queried with the `combined_roles` role.

```graphql
query {
   employees {
     address
     phone
   }
}
```

This will translate to the following SQL query:

```sql

 select
    (case when (P1 or P2) then address else null end) as address,
    (case when P2 then phone else null end) as phone
 from employee
 where (P1 or P2)
```

The other parameters of the select permission will be combined in the following manner:

1. Limit - Minimum of the limits will be the limit of the inherited role
2. Allow aggregations - If any of the role allows aggregation, then the inherited role will allow aggregation
3. Scalar computed fields - same as table column fields, as in the above example

APIs for inherited roles:
----------------------

1. `add_inherited_role`

`add_inherited_role` is the [metadata API](https://hasura.io/docs/1.0/graphql/core/api-reference/index.html#schema-metadata-api) to create a new inherited role. It accepts two arguments

`role_name`: the name of the inherited role to be added (String)
`role_set`: list of roles that need to be combined (Array of Strings)

Example:

```json
{
  "type": "add_inherited_role",
  "args": {
      "role_name":"combined_user",
      "role_set":[
          "user",
          "user1"
      ]
  }
}
```

After adding the inherited role, the inherited role can be used like single roles like earlier

Note:

An inherited role can only be created with non-inherited/singular roles.

2. `drop_inherited_role`

The `drop_inherited_role` API accepts the name of the inherited role and drops it from the metadata. It accepts a single argument:

`role_name`: name of the inherited role to be dropped

Example:

```json

{
  "type": "drop_inherited_role",
  "args": {
      "role_name":"combined_user"
  }
}
```

Metadata
---------

The derived roles metadata will be included under the `experimental_features` key while exporting the metadata.

```json
{
  "experimental_features": {
    "derived_roles": [
      {
        "role_name": "manager_is_employee_too",
        "role_set": [
          "employee",
          "manager"
        ]
      }
    ]
  }
}
```

Scope
------

Only postgres queries and subscriptions are supported in this PR.

Important points:
-----------------

1. All columns exposed to an inherited role will be marked as `nullable`, this is done so that cell value nullification can be done.

TODOs
-------

- [ ] Tests
   - [ ] Test a GraphQL query running with a inherited role without enabling inherited roles in experimental features
   - [] Tests for aggregate queries, limit, computed fields, functions, subscriptions (?)
   - [ ] Introspection test with a inherited role (nullability changes in a inherited role)
- [ ] Docs
- [ ] Changelog

Co-authored-by: Vamshi Surabhi <6562944+0x777@users.noreply.github.com>
GitOrigin-RevId: 3b8ee1e11f5ceca80fe294f8c074d42fbccfec63
2021-03-08 11:15:10 +00:00
Naveen Naidu
f55df228fe Server: Add Event Trigger Metrics
- [x] **Event Triggers Metrics**
  - [x] Distribution of size of event trigger fetches / Number of events fetched in the last `event trigger fetch`
  - [x] Event Triggers: Number of event trigger HTTP workers in process
  - [x]  Event Triggers: Avg event trigger lock time (if an event has been fetched but not processed because http worker is not free)

#### Sample response
The metrics can be viewed from the `/dev/ekg` endpoint

```json
{
"num_events_fetched":{
      "max":0,
      "mean":0,
      "count":1,
      "min":0,
      "variance":null,
      "type":"d",
      "sum":0
   },
 "num_event_trigger_http_workers":{
      "type":"g",
      "val":0
   },
"event_lock_time":{
      "max":0,
      "mean":0,
      "count":0,
      "min":0,
      "variance":0,
      "type":"d",
      "sum":0
   },
```

#### Todo
- [ ]  Group similar metrics together (Eg: Group all the metrics related to Event trigger, How do we do it??)

Closes: https://github.com/hasura/graphql-engine-mono/issues/202
GitOrigin-RevId: bada11d871272b04c8a09d006d9d037a8464a472
2021-03-07 05:25:24 +00:00
Antoine Leblanc
5a1f71b3bc server/mssql: fix parsing of text column values
GitOrigin-RevId: c4692cdc254af074f70cdceb78c7dc6211e7e084
2021-03-05 18:33:16 +00:00
Sameer Kolhar
78785ae513 server: fix fromJSON for RemoteSchemaPermsCtx
GitOrigin-RevId: a0a6d50a39ffbc7fab0a303b50c3e2c485954773
2021-03-05 13:26:57 +00:00
Rakesh Emmadi
edf5a0661e server: support mssql views
GitOrigin-RevId: 50e2fc1d64a67a9284ae62d2401ca6f9627ebdab
2021-03-05 10:53:38 +00:00
Sameer Kolhar
642c950827 server: fix ToJSON instance of RemoteSchemaPermsCtx
GitOrigin-RevId: 7e8ffa747bb645ce6b606ef38a16120f420832bf
2021-03-05 07:47:00 +00:00
Sameer Kolhar
ee56b741d8 server: add is_remote_schema_permissions_enabled to config API
GitOrigin-RevId: 7129d0fba25edc2450c74140335df9398154ce52
2021-03-04 16:22:31 +00:00
Tirumarai Selvan
d1d8cc0a40 tag release v2.0.0-alpha.3 (#790)
GitOrigin-RevId: 7b61e8190d07ebe45ff96ee5209992dfc3c53b93
2021-03-04 14:30:32 +00:00
hasura-bot
da3c9f2c53 minor typo fixes (#165)
Co-authored-by: Colby Thomas <coloradocolby@gmail.com>
GITHUB_PR_NUMBER: 6311
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/6311

Co-authored-by: Colby Thomas <coloradocolby@gmail.com>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
GitOrigin-RevId: 946665878fb9199d067ad1bd981de9525e86a0ae
2021-03-04 06:36:15 +00:00
Vladimir Ciobanu
d5ff1acf2d better handling for one-to-one relationships
Co-authored-by: Rikin Kachhia <54616969+rikinsk@users.noreply.github.com>
GitOrigin-RevId: 1bb5bc0c4ac8109ee1d20563d23cf98e0906a483
2021-03-03 13:02:59 +00:00
Karthikeyan Chinnakonda
15ed0cf536 server: document the remote joins architecture
GitOrigin-RevId: f142908fa3dd3b0cb8887521c639be4a017e606a
2021-03-03 11:47:13 +00:00
Vladimir Ciobanu
d4d02c08d8 update ci-info-hs for cabal 3.4 (#747)
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
GitOrigin-RevId: 29bed5510b971572bf545e588065bbe1b5743d2c
2021-03-03 07:02:42 +00:00
David Overton
456b6d5664 Tests for metadata resource version
GitOrigin-RevId: c1db1355ef8c6617db8179c8c50ba2bd91f7062f
2021-03-03 04:41:02 +00:00
Karthikeyan Chinnakonda
d2418383bf server: add https remote schema tests
GitOrigin-RevId: 9e9e66248f956174dc8197adeb7a48da12e53dac
2021-03-02 16:52:47 +00:00
Karthikeyan Chinnakonda
f2724c97b4 build: fix the packaging of the static console assets
GitOrigin-RevId: 01a6e4f15366476f01ca355f1013712e24bb4a3b
2021-03-02 13:23:53 +00:00
David Overton
e4c8d560c5 Fix/missing source error
GitOrigin-RevId: 074f4eb2d13a2c8ab002a355c6734eed257482d3
2021-03-02 04:27:29 +00:00
Vladimir Ciobanu
88ce387919 docs: add note about recovering existentially quantified type info
This should be merged after #686

GitOrigin-RevId: a372ca475a9c4a182b4819310634d4c3985225ab
2021-03-01 22:31:55 +00:00
Vladimir Ciobanu
ddbc497506 docs: add note on existentials and references to it
Added a note on existentials. I plan to create a subsequent PR with a note on how we use the singletons trick to recover the type inside an existential.

GitOrigin-RevId: 1f227d859dcc384b4ac7e103053f643f879827d1
2021-03-01 21:38:50 +00:00
Abby Sassel
01cb59919f docs: changelog & server documentation improvements
GitOrigin-RevId: e6a5e464acb5492ab7a592362fff2bac6528dfaa
2021-03-01 20:53:43 +00:00
Praveen Durairaju
0ff26100bf update docs link to avoid redirects
GitOrigin-RevId: 1f2a1d21bfb9b2908d56305fa2dfb69270deafdf
2021-03-01 18:51:18 +00:00
Vamshi Surabhi
e9f85ce6e6 mssql: connection pooling
GitOrigin-RevId: c1a6509f19a903724ce2b770ae23cbd925b537f8
2021-02-25 18:16:52 +00:00
Abby Sassel
fb1a0d286d server: support ltree operators (close #625)
GitOrigin-RevId: fb6a3eb8cbe4604789938bcbc78916fbcd1af515
2021-02-25 11:06:49 +00:00
Antoine Leblanc
f55bbe96ef server: fix MSSQL returning malformed JSON on empty select
GitOrigin-RevId: b0b9f68dbdb2b015e4c0640670a2e4cf978fe5fb
2021-02-25 09:58:24 +00:00
Karthikeyan Chinnakonda
4211d27272 server: support reading JWT from Cookie header
GitOrigin-RevId: 1de90242d3c000361f87256c2dddce1677863231
2021-02-25 09:03:46 +00:00
Vamshi Surabhi
74a58fb66c mssql: basic support for geography and geometry types
GitOrigin-RevId: 97bbf0e8d53b00c9f4365b145adb19bdcd769ceb
2021-02-24 12:53:15 +00:00
Lyndon Maydwell
b944d598e0 Adding check for multiple query definitions for rest endpoints
When adding a rest endpoint that references a query with multiple definitions, throw an error.

Needed a second PR for this as Kodiak merged the prior PR before this commit was added.

---

### Kodiak commit message

Adding check for multiple query definitions for rest endpoints.

GitOrigin-RevId: 6e3977a210e29f143b61282fbb37c93eb5b9d73c
2021-02-24 10:13:11 +00:00
Vamshi Surabhi
bde1a4d2f2 bump odbc version (#660)
GitOrigin-RevId: 43873267167dd742d3a6baeb5e1434e2a968a4ca
2021-02-24 07:47:07 +00:00
Lyndon Maydwell
08da0c63b6 REST Endpoints - Prohibit Invalid slashes, duplicate variables, non-singular query definitions, subscriptions
Resolves Issues:

* https://github.com/hasura/graphql-engine-mono/issues/658 - Invalid Slashes
* https://github.com/hasura/graphql-engine-mono/issues/628 - Subscriptions

Implementation:

* Moved some logic from Endpoint.hs to allow reuse of splitting url into PathSegments.
* Additional validation steps alongside checking for overlapping routes
* Logging potential misuse of GET for mutations

Future Work:

* [ ] GET is allowed for mutations (Ignore/Log warning for Now)
* [ ] Add to scInconsistentObjs rather than throwing error
  * Add information to scInconsistentObjs instead of raising errors directly.

TODO:

* [x] Duplicate variable segments with the same name in the location should not be allowed
* [x] We should throw an error on trailing and leading slashes and URLs which contain empty segments
* [x] Endpoints can be created using subscriptions. But the error only shows at the time of the query
* [x] Tests

---

### Kodiak commit message

Prohibit Invalid slashes, duplicate variables, subscriptions for REST endpoints.

GitOrigin-RevId: 86c0d4af97984c8afd02699e6071e9c1658710b8
2021-02-24 04:31:05 +00:00
Antoine Leblanc
7527039f08 server: fix build
GitOrigin-RevId: 8ba687fce6892ac32182311cccc1d4d149049d62
2021-02-23 21:43:56 +00:00
Vladimir Ciobanu
281cb771ff server: add MSSQL support
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
Co-authored-by: Vamshi Surabhi <6562944+0x777@users.noreply.github.com>
Co-authored-by: Aravind K P <8335904+scriptonist@users.noreply.github.com>
GitOrigin-RevId: 699c453b9692e1b822f393f23ff5e6db4e010d57
2021-02-23 17:38:36 +00:00
Anon Ray
cad013c039 multitenant: fix cloud schema files
GitOrigin-RevId: 99f04df3654ab83f64489960b5f7ecf5f86c9a99
2021-02-22 16:03:07 +00:00
Naveen Naidu
4fa34e1d5e server: Apply migrations when catalog version < 43
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
GitOrigin-RevId: 4fa1a04f4924002faac1eca2e5092dfa8767fdb9
2021-02-22 15:00:24 +00:00
Abby Sassel
b00009f4bb server: fix issue with queries on character column types (close #6217)
GitOrigin-RevId: 81eaa0db63656b1d37b9b71736d4309af902cc47
2021-02-22 10:21:21 +00:00
Anon Ray
0dfe83f931 server: optimize resolving source
GitOrigin-RevId: e0513a714b5e3f6d730b0470e1836ed44badd3c0
2021-02-22 07:53:36 +00:00
Antoine Leblanc
377425ff2d server: generalize subscriptions
GitOrigin-RevId: 464e80abf151032dc50eaf6cf8dafc5e7cfa51cd
2021-02-20 13:46:43 +00:00
David Overton
7e79f8dc64 Fix migrations
Provides queries for fetching and inserting metadata into that database that do not assume there is a `resource_version` column. This means that will work when migrating to/from older versions.

Co-authored-by: Lyndon Maydwell <92299+sordina@users.noreply.github.com>
GitOrigin-RevId: dac636d530524082c5a13ae0f016a2d4ced16f7f
2021-02-19 08:47:10 +00:00
Lyndon Maydwell
59c01786fe Optimistic Metadata Locking
Add optimistic concurrency control to the ‘replace_metadata’ call.

Prevents users from submitting out-of-date metadata to metadata-mutating APIs.

See https://github.com/hasura/graphql-engine-mono/issues/472 for details.

GitOrigin-RevId: 5f220f347a3eba288a9098b01e9913ffd7e38166
2021-02-19 02:40:23 +00:00
Karthikeyan Chinnakonda
7be72d7bee server: support for maintenance mode in v1.4
Co-authored-by: Anon Ray <616387+ecthiender@users.noreply.github.com>
GitOrigin-RevId: 85f636b1bc8063689845da90e85cc480e87dca7e
2021-02-18 16:47:22 +00:00
Anon Ray
1091203ced multitenant: fix issues with cloud schema sync
GitOrigin-RevId: 7749fe71a1174ac8dbfb79734584c60d882f07b7
2021-02-18 15:12:26 +00:00
Karthikeyan Chinnakonda
77b3bb5a20 server: run function permissions tests in CI
GitOrigin-RevId: 750b44d0f30df506eb8f0a8eec42fcd1a7d51599
2021-02-18 08:27:00 +00:00
Lyndon Maydwell
0d67b5cb53 Allow Inconsistent Metadata (v2) (#473)
Allow Inconsistent Metadata

GitOrigin-RevId: 7b7e1fdccbeaa1b7dccf66ff22c4b40f8cf5bb27
2021-02-16 08:10:21 +00:00
Vladimir Ciobanu
7290a57308 rename 'Connection Timeout' to 'Pool Timeout'
GitOrigin-RevId: f2f03a368b372d83bd9c9e71e2069310ba7e6f09
2021-02-16 05:19:49 +00:00
Vladimir Ciobanu
83c74ac133 connection acquisition timeout for oss and pro
Co-authored-by: Vamshi Surabhi <6562944+0x777@users.noreply.github.com>
GitOrigin-RevId: 673452c1fed2494b437b5fdd1f195e48b10f7634
2021-02-15 13:33:21 +00:00
Rakesh Emmadi
9ef603360c server: generalize schema cache building (#496)
Co-authored-by: Vamshi Surabhi <vamshi@hasura.io>
Co-authored-by: Vladimir Ciobanu <admin@cvlad.info>
Co-authored-by: Antoine Leblanc <antoine@hasura.io>
Co-authored-by: Stylish Haskell Bot <stylish-haskell@users.noreply.github.com>
GitOrigin-RevId: 9d631878037637f3ed2994b5d0525efd978f7b8f
2021-02-14 06:08:46 +00:00
Phil Freeman
7fffc11077 Caching, Rate Limiting, Metrics & Session Variable Improvements (#376)
* server: use a leaky bucket algorithm for bytes-per-second cache rate limiting

* Use evalsha properly

* Adds redis cache limit parameters to PoliciesConfig

* Loads Leaky Bucket Script On Server Start

* Adds more redis logging and moves cache update into lua script

* reverts setex in lua and adds notes

* Refactors cacheStore and adds max TTL and cache size limits

* Filter session vars in cache key

* WIP

* parens

* cache-clear-hander POC implementation

* cache-clear-hander POC implementation

* Pro projectId used as cache key

* POC working!

* prefixing query-response keys in redis

* Add cacheClearer to RedisScripts

* Partial implementation of cacheClearer from scripts record

* updating tests

* [automated] stylish-haskell commit

* Adds query look with up with metrics script

* Adds missing module and lua script from last commit

* Changes redis script module structure to match cache clearing branch

* minor change to lua script

* cleaning up cache clearing

* generalising JsonLog

* [automated] stylish-haskell commit

* Draft Cache Metrics Endpoint

* Adds Cache Metrics Handler

* Adds hook handler module

* Missed HandlerHook module in last commit

* glob

* Fixes redis mget bug

* Removes cache totals and changes dashes to colons in metric cache keys

* Adds query param to clear clear endpoint for deleting specific keys

* Adds query param to clear clear endpoint for deleting specific keys

* Cache Metrics on query families rather then queries

* Replace Set with nub

* Base16 Redis Hashes

* Query Family Redis Keys With Roles

* response headers for cache keys

* fixing bug in family key by excluding operation name; using hash for response header instead of entire key

* Adds query family to redis cache keys and cache clear endpoint

* Fixes queryfamily hash bug

* Moves cache endpoints to /pro

* Moved cache clear to POST

* Refactors cache clear function

* Fixes query family format bug

* Adds query cache tests and optional --redis-url flag to python test suite

* Adds session variable cache test

* Update pro changelog

* adding documentation for additional caching features

* more docs

* clearing up units of leaky bucket params

* Adds comments to leaky bucket script

* removes old todo

* Fixes session variable filtering to work with new query rootfield

* more advanced defaulting behaviour for bucket rate and capacity.

* Updates Docs

* Moves Role into QueryFamily hash

* Use Aeson for Cache Clear endpoint response

* Moves trace to bracket the leaky bucket script

* Misc review tweaks

* Adds sum type for cache clear query params

* Hardcodes RegisReplyLog log level

* Update docs/graphql/cloud/response-caching.rst

Co-authored-by: Phil Freeman <phil@hasura.io>

* new prose for rate limiting docs

* [automated] stylish-haskell commit

* make rootToSessVarPreds total

* [automated] stylish-haskell commit

* Fixes out of scope error

* Renamed _acRedis to _acCacheStore

Co-authored-by: Solomon Bothwell <ssbothwell@gmail.com>
Co-authored-by: Lyndon Maydwell <lyndon@sordina.net>
Co-authored-by: David Overton <david@hasura.io>
Co-authored-by: Stylish Haskell Bot <stylish-haskell@users.noreply.github.com>
Co-authored-by: Lyndon Maydwell <lyndon@hasura.io>
GitOrigin-RevId: dda5c1a3f902967b3d78310f950541a55fabb1b0
2021-02-13 00:06:18 +00:00
Sameer Kolhar
8655e6fd3a console: function permissions UI (#413)
GitOrigin-RevId: ccdfab19751b0d238a4ebcec59ba73a798103ca9
2021-02-12 17:02:41 +00:00
Karthikeyan Chinnakonda
8a68cc6650 server: don't expose remote schema mutations in relay
GitOrigin-RevId: 21c126bfd46a22d3b0e64d57e92d78ea3c4df9ab
2021-02-12 13:29:19 +00:00
Antoine Leblanc
1ca4034697 server: generalize execution of queries and mutations
GitOrigin-RevId: aff477a0849d4667f2f6dc6804d6ca2982e53f95
2021-02-12 03:05:05 +00:00
Antoine Leblanc
5b0fc9a365 server: update cabal dependencies
GitOrigin-RevId: a30b1dc80455059330cdae7c9084eb70552d54d0
2021-02-12 02:59:00 +00:00
Matthew Pickering
35b81f39e9 Memory performance improvements from Cherre (#518)
* Stop shutdown handler retaining the whole serveCtx

This might look like quite a strange way to write the function but it's
the only way I could get GHC to not capture `serveCtx` in the shutdown
handler.

Fixes the metadata issue in #344

* Force argumentNames

The arguments list is often empty so we end up with a lot of duplicate
thunks if this value is not forced.

* Increase sharing in nullableType and nonNullableType

The previous definitions would lead to increased allocation as it would
destory any previously created sharing. The new definition only allocate
a fresh constructor if the value is changed.

* Add memoization for field parsers

It was observed in #344 that many parsers were not being memoised which
led to an increase in memory usage. This patch generalisation memoisation so
that it works for FieldParsers as well as normal Parsers.

There can still be substantial improvement made by also memoising
InputFieldParsers but that is left for future work.

Co-authored-by: Antoine Leblanc <antoine@hasura.io>

* [automated] stylish-haskell commit

* changelog

Co-authored-by: Phil Freeman <paf31@cantab.net>
Co-authored-by: Antoine Leblanc <antoine@hasura.io>
Co-authored-by: Stylish Haskell Bot <stylish-haskell@users.noreply.github.com>
Co-authored-by: Phil Freeman <phil@hasura.io>
GitOrigin-RevId: 36255f77a47cf283ea61df9d6a4f9138d4e5834c
2021-02-12 01:34:56 +00:00
Anon Ray
06b599b747 server: multitenant metadata storage
The metadata storage implementation for graphql-engine-multitenant.

- It uses a centralized PG database to store metadata of all tenants (instead of per tenant database)
- Similarly, it uses a single schema-sync listener thread per MT worker (instead of listener thread per tenant) (PS: although, the processor thread is spawned per tenant)
- 2 new flags are introduced - `--metadataDatabaseUrl` and (optional) `--metadataDatabaseRetries`

Internally, a "metadata mode" is introduced to indicate an external/managed store vs a store managed by each pro-server.

To run :
- obtain the schema file (located at `pro/server/res/cloud/metadata_db_schema.sql`)
- apply the schema on a PG database
- set the `--metadataDatabaseUrl` flag to point to the above database
- run the MT executable

The schema (and its migrations) for the metadata db is managed outside the MT worker.

### New metadata

The following is the new portion of `Metadata` added :

```yaml
version: 3
metrics_config:
  analyze_query_variables: true
  analyze_response_body: false
api_limits:
  disabled: false
  depth_limit:
    global: 5
    per_role:
      user: 7
      editor: 9
  rate_limit:
    per_role:
      user:
        unique_params:
        - x-hasura-user-id
        - x-hasura-team-id
        max_reqs_per_min: 20
    global:
      unique_params: IP
      max_reqs_per_min: 10
```

- In Pro, the code around fetching/updating/syncing pro-config is removed
- That also means, `hdb_pro_catalog` for keeping the config cache is not required. Hence the `hdb_pro_catalog` is also removed
- The required config comes from metadata / schema cache

### New Metadata APIs

- `set_api_limits`
- `remove_api_limits`
- `set_metrics_config`
- `remove_metrics_config`

#### `set_api_limits`

```yaml
type: set_api_limits
args:
  disabled: false
  depth_limit:
    global: 5
    per_role:
      user: 7
      editor: 9
  rate_limit:
    per_role:
      anonymous:
         max_reqs_per_min: 10
         unique_params: "ip"
      editor:
        max_reqs_per_min: 30
        unique_params:
        - x-hasura-user-id
      user:
        unique_params:
        - x-hasura-user-id
        - x-hasura-team-id
        max_reqs_per_min: 20
    global:
      unique_params: IP
      max_reqs_per_min: 10
```

#### `remove_api_limits`

```yaml
type: remove_api_limits
args: {}
```

#### `set_metrics_config`

```yaml
type: set_metrics_config
args:
  analyze_query_variables: true
  analyze_response_body: false
```

#### `remove_metrics_config`

```yaml
type: remove_metrics_config
args: {}
```

#### TODO
- [x] on-prem pro implementation for `MonadMetadataStorage`
- [x] move the project config from Lux to pro metadata (PR: #379)
- [ ] console changes for pro config/api limits, subscription workers (cc @soorajshankar @beerose)
- [x] address other minor TODOs
  - [x] TxIso for `MonadSourceResolver`
  - [x] enable EKG connection pool metrics
  - [x] add logging of connection info when sources are added?
  - [x] confirm if the `buildReason` for schema cache is correct
- [ ] testing
- [x] 1.3 -> 1.4 cloud migration script (#465; PR: #508)
  - [x] one-time migration of existing metadata from users' db to centralized PG
  - [x] one-time migration of pro project config + api limits + regression tests from metrics API  to metadata
- [ ] integrate with infra team (WIP - cc @hgiasac)
  - [x] benchmark with 1000+ tenants + each tenant making read/update metadata query every second (PR: https://github.com/hasura/graphql-engine-mono/pull/411)
  - [ ] benchmark with few tenants having large metadata (100+ tables etc.)
  - [ ] when user moves regions (https://github.com/hasura/lux/issues/1717)
    - [ ] metadata has to be migrated from one regional PG to another
    - [ ] migrate metrics data as well ?
      - [ ] operation logs
      - [ ] regression test runs

- [ ] find a way to share the schema files with the infra team

Co-authored-by: Naveen Naidu <30195193+Naveenaidu@users.noreply.github.com>
GitOrigin-RevId: 39e8361f2c0e96e0f9e8f8fb45e6cc14857f31f1
2021-02-11 17:55:21 +00:00
Karthikeyan Chinnakonda
0cfad08993 server: function permissions changes
GitOrigin-RevId: b0d784dd23aacd7a62cf471dd33eca869e25c13e
2021-02-11 09:44:38 +00:00
Antoine Leblanc
9b38863c73 server: change RootField so that only the DB component is existential
GitOrigin-RevId: ff620b775ea9b1c8433255705254ed9d58eba290
2021-02-09 19:30:08 +00:00
Antoine Leblanc
7760ed2d98 server: generalize top-level build functions
GitOrigin-RevId: 5ace65f702b411bf75c84e949742c7d657382fab
2021-02-09 12:48:22 +00:00
Vladimir Ciobanu
6b7b68504a server: improve the UX around SourceInfo
GitOrigin-RevId: 47f7e9b7daa3f89771dd3883e9f6050274827e27
2021-02-08 16:31:17 +00:00
Antoine Leblanc
e449cf9990 server: finish generalizing mutations
GitOrigin-RevId: 645eefadee35f1642eee805e20161417ab38b949
2021-02-08 10:40:51 +00:00
Antoine Leblanc
83701fb63e server: changes to support other backends
GitOrigin-RevId: ec0ad47957ab6f9a0855623fffedb23924e7c75d
2021-02-03 16:25:17 +00:00
Karthikeyan Chinnakonda
312e56dfa6 server: support tracking of custom functions which return single row
GitOrigin-RevId: fbf99816a70e666bb9b02fca92a0ee7d419cdff8
2021-02-03 14:07:30 +00:00
Swann Moreau
c14dcd5792 pass gql requests into auth webhook POST body (#149)
* fix arg order in UserAuthentication instance [force ci]

* change the constructor name to AHGraphQLRequest

Co-authored-by: Stylish Haskell Bot <stylish-haskell@users.noreply.github.com>
Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
GitOrigin-RevId: fb3258f4a84efc6c730b0c6222ebd8cea1b91081
2021-02-03 07:11:39 +00:00
Karthikeyan Chinnakonda
94a886ee94 server: fix mutation functions not being exposed to admin when function permissions are inferred
GitOrigin-RevId: 7d8031341351bdbf25d4244ce57d4f9396dde437
2021-02-01 12:58:31 +00:00
Karthikeyan Chinnakonda
05f21a2768 server: fix the FromJSON instance of FunctionPermissionsCtx
GitOrigin-RevId: 506d654ef9cad95ffdddbf832a2779fc34e6bd00
2021-02-01 06:04:28 +00:00
Swann Moreau
0b294bab5c update resource limits on circleci [force ci]
GitOrigin-RevId: 0ed0c7d32400f0a940be7ad1b22a6b55b751cbb5
2021-01-29 14:36:07 +00:00
Tirumarai Selvan A
88c3d7c787 tag release v1.4.0-alpha.1
GitOrigin-RevId: f09f265175226f96faba53ccfe68349377523f98
2021-01-29 12:31:13 +00:00
Vladimir Ciobanu
1bda14960e update pg-client-hs to revert buffer shrinking
GitOrigin-RevId: f8c47c2551c09a9e9f0d57a68c983f580d796c00
2021-01-29 12:04:39 +00:00
Karthikeyan Chinnakonda
10a3f9960d server: new function permissions layer
Co-authored-by: Rikin Kachhia <54616969+rikinsk@users.noreply.github.com>
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
GitOrigin-RevId: 35645121242294cb6bb500ea598e9a1f2ca67fa1
2021-01-29 05:49:09 +00:00
Lyndon Maydwell
0767333597 server: support restified versions of graphql queries (#303)
Restified GraphQL Endpoints feature.

GitOrigin-RevId: 3d6e589426ec21a60a915b47f579f0ac4934af45
2021-01-29 01:03:35 +00:00
Rakesh Emmadi
ff3c58f230 ci: fix oss test server upgrade ci test when upgrading from 1.3.3
GitOrigin-RevId: a5b663ea5b079212c063f15bc67559f5a234f96f
2021-01-28 11:40:24 +00:00
Antoine Leblanc
353859db09 server: remove GraphQL.Utils
GitOrigin-RevId: 90639f9f3d263ccb0ce4e3b8b6e19ce784f4b25d
2021-01-26 13:14:35 +00:00
Antoine Leblanc
62a3bb0d9e Move text operator out of SQL.Types. (#406)
GitOrigin-RevId: 25adad545b302fd1dde25ea0365d492dfcdaa71d
2021-01-26 12:23:20 +00:00
Antoine Leblanc
6494229f54 server: generalize functions (#393)
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
GitOrigin-RevId: 5d2140152a2a18601c785ea80a7689cbe3bd277e
2021-01-25 10:13:54 +00:00
Swann Moreau
d4b474061f server: split server test suite to run in parallel on circleci (#309)
Co-authored-by: rakeshkky <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Vishnu Bharathi P <vishnubharathi04@gmail.com>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
Co-authored-by: Nizar Malangadan <nizar-m@users.noreply.github.com>
Co-authored-by: nizar-m <19857260+nizar-m@users.noreply.github.com>
GitOrigin-RevId: b8a1e9c3ba4797bad8dbb2daa75d23d4ad163546
2021-01-25 08:24:13 +00:00
Antoine Leblanc
591db631ea ci: run stylish haskell on all incoming PRs
### Description

This PR adds two new github actions: one in the OSS repo, one in the monorepo. The OSS action runs `stylish-haskell` on all files touched by the PR, and displays a warning for each file that was in need of a formatting. The monorepo action does the same thing, with a twist: if the branch is not a shadow copy of an OSS branch, we assume that it is a local branch, and simply push a new commit with the changes.

Furthermore, this PR upgrades our stylish-haskell config to add record formatting, as close as possible to our styleguide.

Both actions use the standard stylish-haskell, not our modified fork.

### Known limitation

The monorepo action does not handle forks: pushing to the branch will fail, and checking the branch out might fail too. This is probably acceptable since we don't use forks with the monorepo, but it wouldn't be hard to handle that gracefully.

GitOrigin-RevId: 814138c5b9826098e2e4ea192778fc0d93fbe390
2021-01-21 23:37:26 +00:00
Karthikeyan Chinnakonda
3020150274 server: allow mapping session variables to standard JWT claims
fixes https://github.com/hasura/graphql-engine/issues/6449

A while back we added [support for customizing JWT claims](https://github.com/hasura/graphql-engine/pull/3575) and this enabled to map a session variable to any value within the unregistered claims, but as reported in #6449 , users aren't able to map the `x-hasura-user-id` session variable to the `sub` standard JWT claim.

This PR fixes the above issue by allowing mapping session variables to standard JWT claims as well.

GitOrigin-RevId: d3e63d7580adac55eb212e0a1ecf7c33f5b3ac4b
2021-01-21 16:50:46 +00:00
Antoine Leblanc
4815fcd500 server: progress on generic metadata
This PR generalizes a bunch of metadata structures.

Most importantly, it changes `SourceCache` to hold existentially quantified values:
```
data BackendSourceInfo =
  forall b. Backend b => BackendSourceInfo (SourceInfo b)

type SourceCache = HashMap SourceName BackendSourceInfo
```

This changes a *lot* of things throughout the code. For now, all code using the schema cache explicitly casts sources to Postgres, meaning that if any non-Postgres `SourceInfo` makes it to the cache, it'll be ignored.

That means that after this PR is submitted, we can split work between two different aspects:
  - creating `SourceInfo` for other backends
  - handling those other sources down the line

GitOrigin-RevId: fb9ea00f32e840fc33c5467896fb1dfa5283ab42
2021-01-20 00:32:45 +00:00
hasura-bot
98ccd81704 Server: Remote relationships permissions
GITHUB_PR_NUMBER: 6125
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/6125

Co-authored-by: Karthikeyan Chinnakonda <15602904+codingkarthik@users.noreply.github.com>
GitOrigin-RevId: 53d0671e6335dad1af7cb00e3e05e7021a910673
2021-01-19 20:57:58 +00:00
hasura-bot
2c56254e5a server: simplify JSON instances
GITHUB_PR_NUMBER: 6152
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/6152

Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
GitOrigin-RevId: 6c94aef8c57e852b3d41b8355c09e64fce756a7c
2021-01-19 19:15:42 +00:00
hasura-bot
38fc4cb677 server: consistently log request_id at the same level
GITHUB_PR_NUMBER: 6244
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/6244

Co-authored-by: José Lorenzo Rodríguez <37621+lorenzo@users.noreply.github.com>
GitOrigin-RevId: fef22d98ac7be23ca21a63dc1c696fa7169253a4
2021-01-19 17:26:45 +00:00
Antoine Leblanc
e754190301 server: clean MaybeT usage, and introduce new hlint rules
### Description

Our Prelude provides the very convenient `hoistMaybe :: Maybe b -> MaybeT m b`. This PR adds hlint rules to replace uses of `MaybeT $ pure $ x` with the cleaner `hoistMaybe x`, and rules to specifically replace `MaybeT $ pure Nothing` with `empty`.

GitOrigin-RevId: 7254f4954e34e4d7ca972dc7c12073d3ab8cb0b8
2021-01-19 13:38:42 +00:00
Vladimir Ciobanu
6e752a7876 server: add type information to aggregates and stringify them (closes #5704)
Fixes https://github.com/hasura/graphql-engine/issues/5704 by checking, for aggregate fields whether we are handling a numeric aggregation.

This PR also adds type information to `ColFld` such that we know the type of the field.

This is the second attempt. See #319 for a less invasive approach. @nicuveo suggested type information might be useful, and since it wasn't hard to add, I think this version is better as well.

GitOrigin-RevId: aa6a259fd5debe9466df6302839ddbbd0ea659b5
2021-01-18 13:52:51 +00:00
Karthikeyan Chinnakonda
6c22132061 server: move remote schema permissions APIs to /v1/metadata
Earlier (pre catalog separation), the remote schema permissions were in `/v1/query`. This PR moves it to `/v1/metadata`.

GitOrigin-RevId: cb39d9df4cc2288f67231504e3a7909f2f8df4da
2021-01-18 10:39:29 +00:00
hasura-bot
513a3d0c19 Fix action relationship type and input arguments (closes #6402) (#284)
Co-authored-by: Antoine Leblanc <antoine@hasura.io>
GITHUB_PR_NUMBER: 6417
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/6417
GitOrigin-RevId: 37b67a4d04e0ed3b16fc5fc9bf025b24b1f1bf6e
2021-01-18 06:57:24 +00:00
Rakesh Emmadi
648488ce7a server: fix computed field comment is not being saved in metadata (#337)
GitOrigin-RevId: 7eb1c5b86740d884a2718cead6dab962a45283d1
2021-01-14 07:03:42 +00:00
Karthikeyan Chinnakonda
c14bcb6967 server: accept new config allowed_skew in JWT config to provide leeway in JWT expiry
fixes https://github.com/hasura/graphql-engine/issues/2109

This PR accepts a new config `allowed_skew` in the JWT config to provide for some leeway while comparing the JWT expiry time.

GitOrigin-RevId: ef50cf77d8e2780478685096ed13794b5c4c9de4
2021-01-13 08:39:18 +00:00
Lyndon Maydwell
ece4fb4bce Merge pull request #342 from hasura/naveen/move-serverCtx-to-Config
server: Move 'serverMetrics' to Config.hs for better visibility
GitOrigin-RevId: 4f2b40ec81d4d4bb662cc16693b993fb3606b64a
2021-01-13 06:28:12 +00:00
Karthikeyan Chinnakonda
f967a4b22e Server: fix query actions issue used with relationship and configured with permissions
Fixes https://github.com/hasura/graphql-engine/issues/6385

In the v1.3.4-beta.2 version, the SQL generated for a query action containing relationship and configured with permissions parsed the `x-hasura-user-id` session variable value through the `hasura.user` postgres setting instead of passing the session variables as an JSON object to the query as it was in v1.3.3.

This PR fixes the SQL generation to that of v1.3.3

Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
GitOrigin-RevId: 838ba812f89b51df7fcead81b9d3c2885dfa39b4
2021-01-12 12:04:21 +00:00
Karthikeyan Chinnakonda
7b83018c39 Server: fix issue of not being able to track tables with non-compliant GraphQL names (#313)
* server: use only the valid tables in the node parser

* add test

GitOrigin-RevId: 90edf001220c7d78eaab86ff565ee4311844b5d3
2021-01-12 08:13:58 +00:00
Antoine Leblanc
3948ca84da server: RQL code health
This PR is a combination of the following other PRs:
- #169: move HasHttpManager out of RQL.Types
- #170: move UserInfoM to Hasura.Session
- #179: delete dead code from RQL.Types
- #180: move event related code to EventTrigger

GitOrigin-RevId: d97608d7945f2c7a0a37e307369983653eb62eb1
2021-01-08 23:10:36 +00:00
hasura-bot
e44b47f498 ci: activate -Werror in dev.sh config
This PR was migrated from https://github.com/hasura/graphql-engine/pull/6171

---
### Description
A trivial change to make `dev.sh` use -Werror. This is what our CI uses, and it's annoying to have a PR rejected because of a warning that was missed at dev time. Since this is only for `dev.sh`, it won't impact most people's normal workflow.

Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
GitOrigin-RevId: 37c2f088c37326c244533a003bf9b8169448abc8
2021-01-08 21:32:34 +00:00
Antoine Leblanc
0d194724ee server: generalize update schema
### Description

This PR updates the graphql schema to be backend-agnostic. To do so, it also moves the definition of operators to `BackendSchema`, to be specified differently per backend.

GitOrigin-RevId: 35b9d2d1bff93fb68b872d6ab0d3d12ec12c1b93
2021-01-08 19:46:34 +00:00
Rakesh Emmadi
84b5153dc1 replace 'pg_set_table_custom_fields' with 'pg_set_table_customization' in /v1/metadata API (#321)
GitOrigin-RevId: ac770273630321eec224c436c09bd61134bf28c3
2021-01-08 07:42:46 +00:00
Rakesh Emmadi
be62641f68 server: multi source metadata APIs (#217)
Co-authored-by: Aleksandra Sikora <aleksandra@hasura.io>
Co-authored-by: Anon Ray <ecthiender@users.noreply.github.com>
Co-authored-by: Vishnu Bharathi <vishnubharathi04@gmail.com>
Co-authored-by: Aleksandra Sikora <aleksandra@hasura.io>
Co-authored-by: Sameer Kolhar <kolhar730@gmail.com>
Co-authored-by: Aleksandra Sikora <ola.zxcvbnm@gmail.com>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
GitOrigin-RevId: 0dd1e4d58ab81f1b4ce24de2d3eab709c2755e6d
2021-01-07 09:05:19 +00:00
Karthikeyan Chinnakonda
44347d2d74 server: template the schema and table names in the event trigger PG functions
Earlier, while creating the event trigger's internal postgres trigger, we used to get the name of the table from the `TG_TABLE_NAME` special trigger variable. Using this with normal tables works fine, but it breaks when the parent table is partitioned because we associate the ET configuration in the schema only with the original table (as it should be).

In this PR, we supply the table name and schema name through template variables instead of using `TG_TABLE_NAME` and `TG_TABLE_SCHEMA`, so that event triggers work with a partitioned table as well.

TODO:

- [x] Changelog
- [x] unit test (ET on partition table)

GitOrigin-RevId: 556376881a85525300dcf64da0611ee9ad387eb0
2021-01-06 20:22:34 +00:00
Swann Moreau
5bc0355bdd [server] coalesce multiple run_sql calls in tests (#270)
GitOrigin-RevId: abd7303aaf8e7a8739fd10574249aec450082ef8
2021-01-06 16:07:22 +00:00
Vladimir Ciobanu
f4c2bb6370 server: remove unused Catalog.hs module (#296)
GitOrigin-RevId: 473c32c2232573815776f657dc1daa67a314103b
2021-01-06 08:01:53 +00:00
Karthikeyan Chinnakonda
8a7e749a69 server: fix issue when a non-nullable remote schema field was added as nullable (#279)
GitOrigin-RevId: cf6df889d93bc2e4da47e4fbc6c112be99de977a
2021-01-05 14:47:52 +00:00
Rakesh Emmadi
29f2ddc289 server: support separate metadata database and server code setup for multi sources (#197)
This is an incremental PR towards https://github.com/hasura/graphql-engine/pull/5797

Co-authored-by: Anon Ray <ecthiender@users.noreply.github.com>
GitOrigin-RevId: a6cb8c239b2ff840a0095e78845f682af0e588a9
2020-12-28 12:56:55 +00:00
Karthikeyan Chinnakonda
ef33a61266 server: fix handling of empty array values in relationships of set_custom_types API (#219)
* server: fix handling of empty array values in relationships of set_custom_types

* add a test for dropping the relationship

GitOrigin-RevId: abff138b0021647a1cb6c6c041c2ba53c1ff9b77
2020-12-24 10:11:58 +00:00
Auke Booij
4bbd2fd4dc server: allow fragments to use variables (fix hasura/graphql-engine#6303) (#225)
GitOrigin-RevId: a60b6816212b749836d5d88f296e229e581ab452
2020-12-22 10:42:34 +00:00
Phil Freeman
2dfbf99b41 server: simplify shutdown logic, improve resource management (#218) (#195)
* Remove unused ExitCode constructors

* Simplify shutdown logic

* Update server/src-lib/Hasura/App.hs

Co-authored-by: Brandon Simmons <brandon@hasura.io>

* WIP: fix zombie thread issue

* Use forkCodensity for the schema sync thread

* Use forkCodensity for the oauthTokenUpdateWorker

* Use forkCodensity for the schema update processor thread

* Add deprecation notice

* Logger threads use Codensity

* Add the MonadFix instance for Codensity to get log-sender thread logs

* Move outIdleGC out to the top level, WIP

* Update forkImmortal fuction for more logging info

* add back the idle GC to Pro

* setupAuth

* use ImmortalThreadLog

* Fix tests

* Add another finally block

* loud warnings

* Change log level

* hlint

* Finalize the logger in the correct place

* Add ManagedT

* Update server/src-lib/Hasura/Server/Auth.hs

Co-authored-by: Brandon Simmons <brandon@hasura.io>

* Comments etc.

Co-authored-by: Brandon Simmons <brandon@hasura.io>
Co-authored-by: Naveen Naidu <naveennaidu479@gmail.com>
GitOrigin-RevId: 156065c5c3ace0e13d1997daef6921cc2e9f641c
2020-12-21 18:56:57 +00:00
Auke Booij
792de5ba7c server: remove bad hlint hint (#232)
GitOrigin-RevId: 14170853e6a273a0bf1f26eba3f61ad6ce5d1918
2020-12-21 13:30:16 +00:00
Karthikeyan Chinnakonda
39a4352569 Merge pull request #113 from hasura/karthikeyan/remote-schema-permissions
server: remote schema permissions
GitOrigin-RevId: 63b9717e30351676c9474bdfddd3ad1ee1409eea
2020-12-21 09:12:35 +00:00
Tirumarai Selvan
b544b87b9b Merge pull request #223 from hasura/jberryman/5863-prep-refactoring
GitOrigin-RevId: 71b1453edf4b93ffc16a15ea3c6057bb865b6606
2020-12-20 06:53:38 +00:00
Auke Booij
84f2991c3d server: schema cache generation generalization (#213)
Generalize TableCoreInfoRM, TableCoreCacheRT, some table metadata data types, generalize fromPGCol to fromCol, generalize some schema cache functions, prepare some enum schema cache code for generalization

GitOrigin-RevId: a65112bc1688e00fd707d27af087cb2585961da2
2020-12-17 11:38:15 +00:00
Rakesh Emmadi
a153e96309 server: expand metadata storage class with async actions and core metadata operations (#184)
An incremental PR towards https://github.com/hasura/graphql-engine/pull/5797
- Expands `MonadMetadataStorage` with operations related to async actions and setting/updating metadata

GitOrigin-RevId: 53386b7b2d007e162050b826d0708897f0b4c8f6
2020-12-14 04:31:20 +00:00
Antoine Leblanc
390585e6a2 Generalize Delete schema. (#193)
GitOrigin-RevId: ba01d7b5d1fa7c343daea7049b07e4cd25ed4a65
2020-12-10 16:59:43 +00:00
Rakesh Emmadi
a2cf9a53c2 server: move to storing metadata as a json blob (#115)
GitOrigin-RevId: 3d1a7618a4ec086c2d255549a6c15087201e9ab0
2020-12-08 14:23:28 +00:00
Auke Booij
5edca3a5c2 server: add tshow :: Show a => a -> Text, including hlint hint (#142)
GitOrigin-RevId: 627a55533831b51aa0d01f8a8d95e3d013744dc5
2020-12-08 12:38:31 +00:00
hasura-bot
920c6ea427 ci: add hlint escape hatch (#71)
* ci: add hlint escape hatch

Co-authored-by: Antoine Leblanc <antoine@hasura.io>
GITHUB_PR_NUMBER: 6164
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/6164

* Applied changes to new workflow files.

* Add missing label trigger for lint worklow

Co-authored-by: Antoine Leblanc <antoine@hasura.io>
GitOrigin-RevId: 3e22c301a9412f7f77978b35f29c38c7876ac13f
2020-12-07 12:10:38 +00:00
Auke Booij
9fb80d1230 server: remove some unused top-level definitions; some minor code cleanups (#141)
GitOrigin-RevId: 92002684c5d89791a05975e4b137d93599491ffa
2020-12-04 19:33:10 +00:00
Auke Booij
be7f34891c server: give stack traces when encountering conflicting type definitions (#150)
Since PDV, introspection queries are parsed by a certain kind of reflection where during the GraphQL schema generation, we collect all GraphQL types used during schema generation to generate answers to introspection queries. This has a great advantage, namely that we don't need to keep track of which types are being used in our schema, as this information is extracted after the fact.

But what happens when we encounter two types with the same name in the GraphQL schema? Well, they better be the same, otherwise we likely made a programming error. So what do we do when we *do* encounter a conflict? So far, we've been throwing a rather generic error message, namely `found conflicting definitions for <typename> when collecting types from the schema`. It does not specify what the conflict is, or how it arose. In fact, I'm a little bit hesitant to output more information about what the conflict is, because we support many different kinds of GraphQL types, and these can have disagreements in many different ways. It'd be a bit tiring (not to mention error-prone) to spell this out explicitly for all types. And, in any case, at the moment our equality checks for types is incorrect anyway, as we are avoiding implementing a certain recursive equality checking algorithm.

As it turns out, type conflicts arise not just due to programming errors, but also arise naturally under certain configurations. @codingkarthik encountered an interesting case recently where adding a specific remote and a single unrelated database table would result in a conflict in our Relay schema. It was not readily visible how this conflict arose: this took significant engineering effort.

This adds stack traces to type collection, so that we can inform the user where the type conflict is taking place. The origin of the above conflict can easily be spotted using this PR. Here's a sample error message:

```
Found conflicting definitions for "PageInfo". The definition at "mutation_root.UpdateUser.favourites.anime.edges.node.characters.pageInfo" differs from the the definition at "query_root.test_connection.pageInfo"
```

Co-authored-by: Antoine Leblanc <antoine@hasura.io>
GitOrigin-RevId: d4c01c243022d8570b3c057b168a61c3033244ff
2020-12-04 15:37:55 +00:00
hasura-bot
3451413d1a server: do not block catalog migration on inconsistent metadata (#139)
Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
GITHUB_PR_NUMBER: 6286
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/6286

Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
Co-authored-by: Tirumarai Selvan <tiru@hasura.io>
GitOrigin-RevId: 27bbfc960da87e4ad54bad5943782f28aa113874
2020-12-04 08:39:05 +00:00
hasura-bot
115f2cb621 server: don't memoize backend scalar type reps through WithScalarType (#136)
Co-authored-by: Auke Booij <auke@hasura.io>
GITHUB_PR_NUMBER: 6281
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/6281
GitOrigin-RevId: b7ab3352af21175f0065f1bc2304a1232f6a5580
2020-12-03 12:22:24 +00:00
Phil Freeman
20913c254d server: allocation limits (#82)
* server: allocation limits

* Move most of the allocation limits work into Pro

Co-authored-by: Brandon Simmons <brandon@hasura.io>

* code review changes

* fix build

Co-authored-by: Brandon Simmons <brandon@hasura.io>
GitOrigin-RevId: 035f15def48f4a0376cb2701eff9982c43a78e2d
2020-12-03 04:07:22 +00:00
Antoine Leblanc
3537e2acdc server: fix primary key mutations issue (closes #6255) (#129)
* Add update_by_pk test with more interesting permissions.

* Add new delete test with more interesting permissions.

* Fix byPk operations not providing all columns.

* Empty commit to unlock ci.

GitOrigin-RevId: eb1ca2cc4c59f421bc5fedf49bef91aa4bbdca94
2020-12-02 14:13:00 +00:00
Swann Moreau
e873c4e650 server: add metric for warp threads (#88)
* server: add ekg gauge to count number of warp threads currently alive

* Apply suggestions from code review

* Fix a weird impredicativity bug

* changelog

* Update server/src-lib/Hasura/App.hs

Co-authored-by: Brandon Simmons <brandon@hasura.io>

* Add hlint pragma :|

Co-authored-by: Phil Freeman <phil@hasura.io>
Co-authored-by: Phil Freeman <paf31@cantab.net>
Co-authored-by: Brandon Simmons <brandon@hasura.io>
GitOrigin-RevId: 02e5d85cd8bdf7b1ebbe3395c61d8cac3abc135f
2020-12-02 06:16:58 +00:00
Phil Freeman
1843643c74 Disallow caching for remote joins with forwarded headers (master) (#58)
GitOrigin-RevId: 76eb061534fd2a068965b8b22517a0729d9e3020
2020-12-01 23:24:24 +00:00
Brandon Simmons
bcf251a469 Add missing PROFILING CPP pragma
GitOrigin-RevId: b50acc16cd7194e5a778a34592d44cf21c969d52
2020-12-01 16:05:10 +00:00
Auke Booij
3c3ed55914 server: schema that grows (#105)
This PR makes a bunch of schema generation code in Hasura.GraphQL.Schema backend-agnostic, by moving the backend-specific parts into a new BackendSchema type class. This way, the schema generation code can be reused for other backends, simply by implementing new instances of the BackendSchema type class.

This work is now in a state where the schema generators are sufficiently generic to accept the implementation of a new backend. That means that we can start exposing MS SQL schema. Execution is not implemented yet, of course.
The branch currently does not support computed fields or Relay. This is, in a sense, intentional: computed field support is normally baked into the schema generation (through the fieldSelection schema generator), and so this branch shows a programming technique that allows us to expose certain GraphQL schema depending on backend support. We can write support for computed fields and Relay at a later stage.

Co-authored-by: Antoine Leblanc <antoine@hasura.io>
GitOrigin-RevId: df369fc3d189cbda1b931d31678e9450a6601314
2020-12-01 15:51:13 +00:00
hasura-bot
f6bd354b40 server: hasura on PG v13 (#125)
GitOrigin-RevId: 00fd91c250bcf3dc7ee638e3b152e0dab7281de7
2020-12-01 12:22:42 +00:00
hasura-bot
7b31ff99d1 Support Postgres POSIX regex operators (close #4317) (#119)
Co-authored-by: christophediprima <dipdipdip84@gmail.com>
Co-authored-by: dip <dipdipdip84@gmail.com>
Co-authored-by: Auke Booij <auke@hasura.io>
Co-authored-by: Antoine Leblanc <antoine@hasura.io>
GITHUB_PR_NUMBER: 6172
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/6172
GitOrigin-RevId: 5192d238b527cd21b6efb2f74e279ecc34756c29
2020-11-27 10:54:52 +00:00
Auke Booij
86aef93d31 server: remove a few unused Lift instances (#114)
GitOrigin-RevId: b5679e9579d226f7209b73ac966d29391c308c06
2020-11-26 13:58:07 +00:00
Auke Booij
350d6b3d43 server: make the UnpreparedValue less tightly bound to the backend (#108)
Co-authored-by: Antoine Leblanc <antoine@hasura.io>
GitOrigin-RevId: 90cb7e1cc97961523085ae7ae3d195549b901b1f
2020-11-25 16:08:27 +00:00
hasura-bot
64743cb189 server: all remaining IR changes (#75)
Co-authored-by: Antoine Leblanc <antoine@hasura.io>
GITHUB_PR_NUMBER: 6233
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/6233

Co-authored-by: Antoine Leblanc <antoine@hasura.io>
Co-authored-by: Auke Booij <auke@hasura.io>
GitOrigin-RevId: 268cdad529ad5d9bebeb5b881fda5413ea9d7715
2020-11-25 14:19:50 +00:00
hasura-bot
29925eb08d server: metadata storage abstraction for scheduled triggers
An incremental PR towards https://github.com/hasura/graphql-engine/pull/5797

* metadata storage abstraction for scheduled triggers

Co-authored-by: rakeshkky <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Auke Booij <auke@hasura.io>
GITHUB_PR_NUMBER: 6131
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/6131

* update pro server code

Co-authored-by: rakeshkky <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Auke Booij <auke@hasura.io>
GitOrigin-RevId: 17244a47b3e8633acf2492e0b0734b72025f0a09
2020-11-25 10:57:38 +00:00
Tirumarai Selvan A
6fe5daa702 fix migration to 1) account for long trigger names and 2) correct syntax for dropping function without arguments in pg < 10
GitOrigin-RevId: b5382db23dcf8c05a9936404b8ab303a336c8a44
2020-11-25 09:29:04 +00:00
hasura-bot
5ca8cd23a0 server: support remote relationship with ID scalar types (#92)
Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
GITHUB_PR_NUMBER: 6227
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/6227

Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
GitOrigin-RevId: 666ec37d570e46482e0f27db2c97c9e3f9c1f4d3
2020-11-24 12:20:22 +00:00