Commit Graph

1243 Commits

Author SHA1 Message Date
Rakesh Emmadi
04e041b8cb server: fix replace configuration in add source metadata API
>

### Description
>

### Changelog

- [x] `CHANGELOG.md` is updated with user-facing content relevant to this PR. If no changelog is required, then add the `no-changelog-required` label.

### Affected components

- [x] Server

### Related Issues
->
Fixes #1528

### Solution and Design
>
Only replace connection configuration instead of replacing entire metadata with empty one.

GitOrigin-RevId: f9a16dcc7b1219ec1af915bf083622fcb7dde69a
2021-06-09 12:14:24 +00:00
Sameer Kolhar
b83ba51fa3 server: add logs for version and healthz endpoints
GitOrigin-RevId: 9fdd953173643f88a64f44efc345a3a7da061ec1
2021-06-08 14:28:52 +00:00
Sameer Kolhar
e24abede99 server, multi-tenant: update error logging strategy on pro and multitenant
Co-authored-by: Lyndon Maydwell <92299+sordina@users.noreply.github.com>
GitOrigin-RevId: 7462d36488003bfdacb5566c7a0e9f273a937a0e
2021-06-08 12:56:22 +00:00
Philip Lykke Carlsen
79a80c3f0a MSSQL: Validate variables in subscriptions (close #1210)
GitOrigin-RevId: 5888621dfe44f5f710c2431515114ce0e8864e61
2021-06-08 03:51:15 +00:00
Karthikeyan Chinnakonda
cd509a0036 server: recreate event triggers while migrating source catalog
GitOrigin-RevId: 9a07fec31fcdbe6ef3b96bb61c95e0dd544e7f50
2021-06-07 13:58:16 +00:00
Vamshi Surabhi
355c3ff736 server: split Internal/Parser to avoid hs-boot files
This is a minor refactor (part of `Internal/Parser.hs` is moved into `Internal/Input.hs`) to remove `Collect.hs-boot` and `Directives.hs-boot` files. Without these changes:
1. Most changes would trigger recompilation from the modules with hs-boot files.
1. haskell-language-server fails for some reason in the presence of hs-boot files.

GitOrigin-RevId: 77a2e443417b449c5d7d9d418fc75fcdf076a9ae
2021-06-03 15:15:12 +00:00
Tirumarai Selvan
205a31de0c tag release v2.0.0-beta.1
GitOrigin-RevId: e015b7d72751b54684e2e74a645d0b58714ef102
2021-06-02 12:07:58 +00:00
Chris Done
31658eb874 Implement snakeCaseTableName
This implements the `snakeCaseTableName` method. It's part of the larger BigQuery PR at https://github.com/hasura/graphql-engine-mono/pull/1317, but @0x777 requested a smaller PR with this change.

GitOrigin-RevId: aeb93c3228bb6ba7293cd73b7a34b39c7ed6139a
2021-06-02 11:13:36 +00:00
kodiakhq[bot]
01d8a37706 server: fix asymptotics of event_log batch fetching
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
GitOrigin-RevId: 9b8afb502e94dd371a8596ccde97d023bd2344a9
2021-06-01 17:34:38 +00:00
Philip Lykke Carlsen
fc17132930 Revert splitting-out of variables in MSSQL
GitOrigin-RevId: 720c0e0c775c9418aea569d3f87875236fdcebfe
2021-06-01 10:05:26 +00:00
Karthikeyan Chinnakonda
f1f56ccf75 server: source initialization fix
GitOrigin-RevId: fcb94ca743a99ee3ffe30d40717bb1f0a13cf751
2021-05-31 13:54:59 +00:00
Philip Lykke Carlsen
125f37ea28 Server: Use a bespoke type for nullability instead of Bool
GitOrigin-RevId: a793c7a921174611d44b7e7cdc1dc43b132376fe
2021-05-28 10:40:15 +00:00
Swann Moreau
b825af9a96 server: fix error propagation in initCatalogForSource
GitOrigin-RevId: 679bca324dfa7728ebcff5862782f29f538bee20
2021-05-28 08:57:46 +00:00
Lyndon Maydwell
f3749f24b3 Fix for 6913 - Undefined ENV variable during a replace metadata throws on allow_inconsistent_metadata
GitOrigin-RevId: 46494f8784d21179e49d2034e1f68171cf146e49
2021-05-28 04:32:33 +00:00
Antoine Leblanc
30ef72bc93 server: better error message on column type mismatch
GitOrigin-RevId: d3d1ff4278e4ced37e90e950fd9043883f72d37f
2021-05-27 16:47:01 +00:00
Rakesh Emmadi
e43d0273e0 server: mssql: apply schema changes by mssql_run_sql DDL on metadata (fix #779)
Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
GitOrigin-RevId: 6905d5914c8a698445c0ef03d6a8303747701e1c
2021-05-27 15:07:10 +00:00
Antoine Leblanc
00ca310d21 server: do not do a "successful" noop when tracking a function in the wrong source
GitOrigin-RevId: 9eb78899acd0c60081b2a999a75c11b32f394ba7
2021-05-27 12:09:19 +00:00
Karthikeyan Chinnakonda
7d630e8147 build: use the debian buster version in the docker image
GitOrigin-RevId: 51077f8b5d0c5bdf06a6560444f89d74a3660336
2021-05-27 07:50:15 +00:00
Swann Moreau
390a1552a8 server: rename inDevelopmentMode -> devMode
GitOrigin-RevId: a99deb144f07f4b0bdf60430a12ad34962355d36
2021-05-27 06:52:14 +00:00
Karthikeyan Chinnakonda
72c24eea69 server: fix cron trigger bug of new events not getting generated when cron trigger is imported via metadata
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
GitOrigin-RevId: 1a9076e039ba2c8da898683c64ef988fb8512eae
2021-05-26 16:20:19 +00:00
Abby Sassel
06bfe650f0 server/mssql: subscription test cleanup post-fix
GitOrigin-RevId: b67e577b15be3c54f869bb3497b30fb0bb53ddf0
2021-05-26 09:06:57 +00:00
Swann Moreau
8e3e7501c7 server: do not collect metrics for sources that have been dropped
GitOrigin-RevId: ed439426527237c2907df77f45582dbe566aeb51
2021-05-26 07:41:28 +00:00
Antoine Leblanc
1674325eb2 server/mssql: fix order of clause in reselect query generation
GitOrigin-RevId: fb7a258059b767a275a5b337e1549f6b3be45499
2021-05-25 14:58:51 +00:00
Tirumarai Selvan
54b9e96702 update contributing.md with server dependencies
GitOrigin-RevId: 9c42625eea8d758e8aff3eca0af8cde48a500766
2021-05-25 14:00:33 +00:00
Abby Sassel
41908936cf server: improve backend-specific test setup
GitOrigin-RevId: c40393bcdd78feaba7e9490ce4ed78de19b5bfc5
2021-05-25 13:55:11 +00:00
Tirumarai Selvan
3bcf758e14 tag release v2.0.0-alpha.11
GitOrigin-RevId: cbc36c7b0385ad7e1ca7fcda8a364ee9b201b35d
2021-05-25 12:18:44 +00:00
Vladimir Ciobanu
5ad01a4bac server: Add test file to suite
GitOrigin-RevId: 6522f776c9b267d5c94be484d55f879e79d30c96
2021-05-25 11:37:14 +00:00
Naveen Naidu
12b57530f3 server: log warning for deprecated environment variables
Co-authored-by: Anon Ray <616387+ecthiender@users.noreply.github.com>
GitOrigin-RevId: e3a845d93d05545816aa9448aad5845b7247b679
2021-05-25 10:50:51 +00:00
Swann Moreau
d3611af58d server: lazy event catalog initialisation
event catalog:
- `hdb_catalog` is no longer automatically created
- catalog is initialised when the first event trigger is created
- catalog initialisation is done during the schema cache build, using `ArrowCache` so it is only run in response to a change to the set of event triggers

event queue:
- `processEventQueue` thread is prevented from starting when `HASURA_GRAPHQL_EVENTS_FETCH_INTERVAL=0`
- `processEventQueue` thread only processes sources for which at least one event trigger exists in some table in the source

Co-authored-by: Anon Ray <616387+ecthiender@users.noreply.github.com>
GitOrigin-RevId: 73f256465d62490cd2b59dcd074718679993d4fe
2021-05-25 06:51:03 +00:00
Antoine Leblanc
6e95f761f5 server: rewrite remote input parsers to deal with partial variable expansion (fix hasura/graphql-engine#6656)
GitOrigin-RevId: e0b197a0fd1e259d43e6152b726b350c4d527a4b
2021-05-24 20:13:47 +00:00
Anon Ray
53beef47ea server, console, docs: fix untrack_function metadata query bug
Co-authored-by: Sameer Kolhar <6604943+kolharsam@users.noreply.github.com>
GitOrigin-RevId: a68a6f8b5c5243372332bfec7d9cdcd3011c001f
2021-05-24 15:57:26 +00:00
Vladimir Ciobanu
0f8f4764c7 server: rename source
GitOrigin-RevId: 5f4a17941bfd447deb9663a4c250f149238c5f53
2021-05-24 13:15:02 +00:00
Abby Sassel
c9e7e10eaa server/bugfix: Include permission filter in the exists clause. Fixes #6931
GitOrigin-RevId: d3080dfa00c96afcf1254d83757a5e50a0726381
2021-05-24 09:09:50 +00:00
Anon Ray
63594e1828 server: fix referenced session vars for live queries when UDF has specified session argument
GitOrigin-RevId: 23efdf4d3b3902b5ce7ec14061faa53b18258e86
2021-05-24 07:34:33 +00:00
Chris Done
feb08d8998 Various MSSQL bugfixes/cleanups
GitOrigin-RevId: 673df70de7b7f9130482b7a28f860bbaffad44a4
2021-05-21 12:28:52 +00:00
Philip Lykke Carlsen
8cc358b023 Plc/feature/mssql logging variables
GitOrigin-RevId: b67a09cbc5bb8b21857c25a0827800ca1826fa1c
2021-05-21 11:38:23 +00:00
Antoine Leblanc
60a119ae22 server: also collect types from directives into the schema
GitOrigin-RevId: 0736570fd8781e66d558bde313d26ad62f7cf6ca
2021-05-21 04:55:32 +00:00
Abby Sassel
1afa4ac3cc server/citus: feature branch
Co-authored-by: Vladimir Ciobanu <1017953+vladciobanu@users.noreply.github.com>
Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
Co-authored-by: Ikechukwu Eze <22247592+iykekings@users.noreply.github.com>
Co-authored-by: Philip Lykke Carlsen <358550+plcplc@users.noreply.github.com>
Co-authored-by: Phil Freeman <630306+paf31@users.noreply.github.com>
GitOrigin-RevId: 1b964fe5f5f50380172cb702b6a328fed782b6b7
2021-05-21 02:47:51 +00:00
Solomon Bothwell
6d2d2a5826 Postgres Client Cert Update
Co-authored-by: Lyndon Maydwell <92299+sordina@users.noreply.github.com>
GitOrigin-RevId: 50fe785fbc0156fc7df3ad3c71c8aca7c0256318
2021-05-21 01:50:43 +00:00
Naveen Naidu
44f6f06e89 server: Add tests for the event trigger big int and geojson format payload bug fix
GitOrigin-RevId: 1e796b16dca54849334381e855b17b19609c8b7a
2021-05-20 12:27:35 +00:00
Antoine Leblanc
5238bb8011 server: support for custom directives
Co-authored-by: Aravind K P <8335904+scriptonist@users.noreply.github.com>
GitOrigin-RevId: f11b3b2e964af4860c3bb0fd9efec6be54c2e88b
2021-05-20 10:03:50 +00:00
Naveen Naidu
d8d4626ba3 server: fix bigint overflow, incorrect geojson format in event trigger payload
GitOrigin-RevId: f43f54f6851d366b60ff72a1fe9b89ca236c3d89
2021-05-20 07:24:20 +00:00
Karthikeyan Chinnakonda
9a99923fef server: support adding a tracked function as a computed field and vice versa
GitOrigin-RevId: 2f4db0b716d493ce245d9fa7e37e11382f18cd67
2021-05-20 06:29:24 +00:00
Antoine Leblanc
03c70c7b7d tests: skip test_pg_multisource_table_name_conflict if the setup isn't correct
GitOrigin-RevId: 4e658812eaac6534e6b6606f03e893449ad4b975
2021-05-19 21:06:54 +00:00
Antoine Leblanc
dfbdfca144 server: fix order of fields in introspection output
GitOrigin-RevId: edd6af755d46c3d13c8b7b5504f8d89701a3250f
2021-05-19 16:38:54 +00:00
Swann Moreau
a83194239e server: forward request id when not user-provided
GitOrigin-RevId: 78745dda0112321c809220fdcdeaf2adc6f9c3e5
2021-05-19 14:08:00 +00:00
Abby Sassel
3ba5d1e540 server: include more detail in inconsistent metadata error messages (fix #6684)
GitOrigin-RevId: 1a0d8fa2b465320e9ab4ea43259d0e8d92364473
2021-05-19 05:02:53 +00:00
David Overton
ddad668f07 Fix/custom table name
GitOrigin-RevId: 5004717ac7d9e848ca186a1cdf52e375547034bf
2021-05-18 13:37:27 +00:00
Antoine Leblanc
ecb90c47a1 server: fix action output fields return type (fix hasura/graphql-engine#6631)
GitOrigin-RevId: 9be5ac87fe69c55aafb6b5af918eb4deb97e27d5
2021-05-18 09:32:43 +00:00
Naveen Naidu
14f80c55ff EventTrigger: Fix num_event_fetched EKG metric
GitOrigin-RevId: 6ca047edb235f1b30986c08698410ce71f587849
2021-05-17 09:50:29 +00:00
Antoine Leblanc
4fce62d125 server: fix action poller being a bit too eager
GitOrigin-RevId: a767123e49aeb3ee472e9313a86b55a30db468e4
2021-05-17 06:00:02 +00:00
Karthikeyan Chinnakonda
ecb4b9d098 server: graceful shutdown for event and cron triggers and async action (II)
GitOrigin-RevId: 2090c3cc34f633c691b4e4ff9ae918a60c37ce26
2021-05-14 09:39:33 +00:00
Antoine Leblanc
27e68517e7 server: unify parsing Postgres scalars
GitOrigin-RevId: f2a938758cb82b432ec209b0fb453a1d8b206bee
2021-05-13 20:13:49 +00:00
Antoine Leblanc
08d605baca gardening: move RQL.Instances to Base and clean it
GitOrigin-RevId: 01fa4133a4002f891d213c1f913511ccfd3c1741
2021-05-13 13:18:33 +00:00
Swann Moreau
04a87970f4 server: sanitise event trigger logs (fix #1233)
- sanitise the response field in event trigger and scheduled trigger logs, removing the body and the headers
- flatten the log structure to include the event id at `$.detail.event_id` instead of `$.detail.context.event_id`

the log format changes as follows:
```diff
diff --git a/log b/log
index 96127e0..1fb909a 100644
--- a/log
+++ b/log
@@ -1,48 +1,15 @@
 {
   "detail": {
-    "context": {
-      "event_id": "b9d4e627-6029-43f2-9d46-31c532b07070"
-    },
+    "event_id": "adcc8dcd-2f32-4554-bd55-90c787aee137",
     "request": {
       "size": 416
     },
     "response": {
-      "body": "{\n  \"args\": {}, \n  \"data\": \"{\\\"event\\\":{\\\"session_variables\\\":{\\\"x-hasura-role\\\":\\\"admin\\\"},\\\"op\\\":\\\"INSERT\\\",\\\"data\\\":{\\\"old\\\":null,\\\"new\\\":{\\\"name\\\":\\\"someotheranimal\\\",\\\"id\\\":3}},\\\"trace_context\\\":{\\\"trace_id\\\":\\\"e8237c03de151634\\\",\\\"span_id\\\":\\\"8c5f8952e9e06da8\\\"}},\\\"created_at\\\":\\\"2021-05-06T07:52:58.796611Z\\\",\\\"id>
-      "headers": [
-        {
-          "name": "Date",
-          "value": "Thu, 06 May 2021 07:53:00 GMT"
-        },
-        {
-          "name": "Content-Type",
-          "value": "application/json"
-        },
-        {
-          "name": "Content-Length",
-          "value": "1692"
-        },
-        {
-          "name": "Connection",
-          "value": "keep-alive"
-        },
-        {
-          "name": "Server",
-          "value": "gunicorn/19.9.0"
-        },
-        {
-          "name": "Access-Control-Allow-Origin",
-          "value": "*"
-        },
-        {
-          "name": "Access-Control-Allow-Credentials",
-          "value": "true"
-        }
-      ],
-      "size": 1692,
+      "size": 1719,
       "status": 200
     }
   },
   "level": "info",
-  "timestamp": "2021-05-06T13:23:00.376+0530",
+  "timestamp": "2021-05-06T13:25:14.481+0530",
   "type": "event-trigger"
 }
```

GitOrigin-RevId: d9622de366737da04dc6d9ff73238be16ec9305e
2021-05-12 12:10:53 +00:00
Naveen Naidu
27aa5b4781 pro: restrict GQL operations based on node limits
Co-authored-by: Rikin Kachhia <54616969+rikinsk@users.noreply.github.com>
GitOrigin-RevId: 49268fcd36dd15e10c7b7c5fe91af93cd65cb4e2
2021-05-12 06:25:45 +00:00
Antoine Leblanc
4c8753afca server: fix single char SQL generation (fixes hasura/graphql-engine#6814)
GitOrigin-RevId: cbbec459cf7b214d50a14e5ef09f2567356e6481
2021-05-11 23:28:30 +00:00
Antoine Leblanc
2152911e24 server: introduce Hasura.Base (take 2)
GitOrigin-RevId: 0dd10f1ccd338b1cf382ebff59b6ee7f209d39a1
2021-05-11 15:19:33 +00:00
Chris Done
56c094b299 Support query explain for MSSQL (fixes #1024)
Co-authored-by: Abby Sassel <3883855+sassela@users.noreply.github.com>
Co-authored-by: Vamshi Surabhi <6562944+0x777@users.noreply.github.com>
GitOrigin-RevId: 5a57d7570884a5469a947742e4ab9290a0cff55f
2021-05-11 10:05:39 +00:00
Abby Sassel
1e45580e7c server/mssql: fix subscriptions (fixes #6887)
GitOrigin-RevId: 0d0184bd910400ea3a48a650fc61391fa8fd5518
2021-05-10 10:18:45 +00:00
Antoine Leblanc
007bd338a6 server: remove unnecessary text decorations in BigQuery explain
GitOrigin-RevId: 55101cb3164490f52ef7c7be4c720d351e26b3d4
2021-05-07 11:41:31 +00:00
Karthikeyan Chinnakonda
d7ccf81526 server: fix bug when executing a custom function with a row type argument
GitOrigin-RevId: d3a1fa8c9f6b420f4c88eb60c92019c8d0fdc58d
2021-05-06 09:19:49 +00:00
Lyndon Maydwell
3fad5f6678 REST endpoint uuid param bugfix
GitOrigin-RevId: 56066b145576c1f536c420be3e537e8313e61099
2021-05-06 06:44:17 +00:00
hasura-bot
e5b8ae8639 server: support custom URI schemes in CORS config (fix #5818)
GITHUB_PR_NUMBER: 5940
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/5940

Co-authored-by: Carl Patenaude-Poulin <1834919+lilred@users.noreply.github.com>
Co-authored-by: Brandon Simmons <210815+jberryman@users.noreply.github.com>
GitOrigin-RevId: 8060a18a49baeb32e607f2695f16d6867aa1f78e
2021-05-06 00:53:14 +00:00
Antoine Leblanc
d22d26b39c server: fix BigQuery's hack for explain.
GitOrigin-RevId: 6668e16dee1a6bb0b666cfe0b4727f2c4f3181fb
2021-05-05 16:20:54 +00:00
Karthikeyan Chinnakonda
4702ba514a pro server: disable GraphQL schema introspection for specified roles
GitOrigin-RevId: 36ceb1852bc7779092cfb59c92035dd9635b2e3f
2021-05-05 12:26:23 +00:00
Anon Ray
3adbbcd219 server: add EKG metrics for active subscriptions
GitOrigin-RevId: 348245eebf5c3ddaf975707a23125e1bb057fe34
2021-05-05 04:25:07 +00:00
Antoine Leblanc
fe22aa5407 ci: improve hlint messages
GitOrigin-RevId: 7547c2a08161989885b4e6c382c1e42cec809857
2021-05-04 21:52:55 +00:00
Antoine Leblanc
34fc92ea8c server: fix validation of query variables (fixes hasura/graphql-engine#6867)
GitOrigin-RevId: 395bf986e0b823f9f18fb9f204e2056b2d2fd682
2021-05-04 17:58:43 +00:00
Rakesh Emmadi
d362e0726e server/mssql: fix MSSQL table metadata SQL, return empty array for no rows
GitOrigin-RevId: ea73e71365150916e1f517f29b812983770644e5
2021-05-03 12:08:11 +00:00
Tirumarai Selvan
eb38eee63a tag release v2.0.0-alpha.10 (#1256)
GitOrigin-RevId: afe16e57e03cf2b790d47f8e44bbb63ae4405f8e
2021-04-29 11:17:50 +00:00
Anon Ray
7550761853 server: add active websocket connections to server metrics
GitOrigin-RevId: 5177d1c90ffd48c7e28f8ffb726753cb55c89b7f
2021-04-29 10:32:15 +00:00
Brandon Simmons
9c9bb43a53 server: restore proper batching behavior in event trigger processing (#1237)
This essentially restores the original code from c425b554b8
(https://github.com/hasura/graphql-engine/pull/4013). Prior to this
commit we would slurp messages as fast as possible from the database
(one thing c425b55 fixed).

Another thing broken as a consequence of the same logic was the
removeEventFromLockedEvents logic which unlocks in-flight events
(breaking at-least-once delivery)

Some archeology, post-c425b55:

- cc8e2ccc erroneously attempted to refactor using `bracket`, resulting
  in the same slurp-all-events behavior (since we don't ever wait for
  processEvent to complete)
- at some point event processing within a batch is made serial, this
  reported as a bug. See: https://github.com/hasura/graphql-engine/issues/5189
- in 0ef52292b5 (which I approved...) an `async` is added, again
  causing the same issue...

GitOrigin-RevId: d8cbaab385267a4c3f1f173e268a385265980fb1
2021-04-29 04:02:05 +00:00
Lyndon Maydwell
507d3aac2c Change schema sync env var to HASURA_GRAPHQL_SCHEMA_SYNC_POLL_INTERVAL
GitOrigin-RevId: 13ed047c9ff391e212c8a50dd13dd0adad666a9e
2021-04-28 22:27:21 +00:00
Swann Moreau
c834629afb server: fix regression in query-log generation
Co-authored-by: Anon Ray <616387+ecthiender@users.noreply.github.com>
GitOrigin-RevId: 74cc6bad2274dc1587f4d3b38e444857d079a4ac
2021-04-28 17:39:02 +00:00
Karthikeyan Chinnakonda
aca8964fdc server: make postgres related ENV vars source specific
Co-authored-by: Tirumarai Selvan <8663570+tirumaraiselvan@users.noreply.github.com>
Co-authored-by: Vishnu Bharathi <4211715+scriptnull@users.noreply.github.com>
GitOrigin-RevId: 8ec3db00f00e9c28bf2dc0f47bd312a656c61a69
2021-04-28 16:50:14 +00:00
Vamshi Surabhi
1c668dac0e server/mssql: fix aggregations
Aggregate fields - both the table and the relationship fields haven't been enabled till now. This PR fixes enables the aggregate fields and fixes an issue with root aggregate fields.

Co-authored-by: Chris Done <11019+chrisdone@users.noreply.github.com>
GitOrigin-RevId: a6d7ed9b45cda6af659a57576a8623c725a7372f
2021-04-28 11:06:26 +00:00
Karthikeyan Chinnakonda
bc1e131717 server: accept new env var HASURA_GRAPHQL_EVENTS_FETCH_BATCH_SIZE
GitOrigin-RevId: 62c463d3ee754ce9f05ba09afa8cd74ca807a96c
2021-04-27 17:23:46 +00:00
Vladimir Ciobanu
f50f61ab6a server: fix one-to-one relationships
GitOrigin-RevId: e19f4dec04abbdcf4589d24031705e5c57ea1fbb
2021-04-27 13:45:45 +00:00
kodiakhq[bot]
a935746e17 Integration test improvements (for speed/clarity), also increase polling interval for scheduled events
This claws back ~7min from integration tests (run serially, as with `dev.sh test --integration`
Further improvements would do well to focus on optimizing metadata operations, as `setup` dominates

GitOrigin-RevId: 76637d6fa953c2404627c4391447a05bf09355fa
2021-04-27 05:35:26 +00:00
Solomon Bothwell
1001e16060 Limits when runMetadataQuery bumps the resource version
GitOrigin-RevId: ce5bf96572e4b4325e1e1f26350ac10e6c90abc3
2021-04-27 04:23:28 +00:00
Solomon Bothwell
787232f798 Adds SIGINT handler
GitOrigin-RevId: 9df663ee62f0f10ee0145af1e485cb6896866d5a
2021-04-24 05:38:55 +00:00
Chris Done
53fc5617cf Feature/bigquery python tests
Co-authored-by: Aniket Deshpande <922486+aniketd@users.noreply.github.com>
GitOrigin-RevId: c425f84f1c9b8e38ebbfa509b6fa9298e023f386
2021-04-22 11:32:55 +00:00
Antoine Leblanc
5283eebf75 server: fix on_conflict missing when no column update permissions
GitOrigin-RevId: 34dd9f648ca1e268274b6244c48c9e9710c4477d
2021-04-22 10:27:35 +00:00
Antoine Leblanc
ba70ca427a server: switch to a sub-backend approach
GitOrigin-RevId: 660126d5f65620fb58a3ffcbed564e9e35f59938
2021-04-21 21:45:32 +00:00
Karthikeyan Chinnakonda
d09b00040f server: V2 maintenance mode fixes
GitOrigin-RevId: dcd69fb217a4e976523f1bc8e7b2693111091002
2021-04-21 10:56:13 +00:00
Karthikeyan Chinnakonda
7be8003a4b server: limit of an inherited role will be the max of the limit of the limits of the roles
GitOrigin-RevId: 1373f80a691047a9c69b3705e3aafa9244f673b0
2021-04-21 09:13:15 +00:00
Tirumarai Selvan
0359e8dbf0 tag release v2.0.0-alpha.9 (#1196)
GitOrigin-RevId: 801cb0a1ab4f69434734beeda3abfca373115e72
2021-04-21 08:22:39 +00:00
Phil Freeman
e4d78a45cc [server] Fix migration for v46
GitOrigin-RevId: 72464c6b9f92388247b6727a520ecb7c89d78aa5
2021-04-21 06:49:31 +00:00
Antoine Leblanc
84ed74aba1 server: fix empty remote input objects (fixes #6029, #6703)
GitOrigin-RevId: 3c474ee85b5d1271abfc8848e29ae1d3be28ff63
2021-04-21 01:17:01 +00:00
Abby Sassel
703928de9f mssql: support query multiplexing in subscriptions
GitOrigin-RevId: 757ceba2c1cdb1107ce0b0e41d2e70ac795d0d73
2021-04-20 16:58:13 +00:00
Karthikeyan Chinnakonda
58cf521c62 server: support relationship column comparison with root table columns
GitOrigin-RevId: 9d9bcb82a997c1f060df5ba0726c29d934e70d71
2021-04-19 12:17:07 +00:00
Lyndon Maydwell
33f6ffecec Generating query-log for cached requests
GitOrigin-RevId: fe620d0860854b257ab4901ad301646bacd15e0d
2021-04-19 01:23:01 +00:00
Tirumarai Selvan
2425607a18 tag release v2.0.0-alpha.8 (#1159)
GitOrigin-RevId: c883bbbef79532ce97b0b25177843f13bb47c184
2021-04-16 13:39:06 +00:00
Vladimir Ciobanu
e8dfeb107b server: query API now requires admin (#1156)
GitOrigin-RevId: a0b45c965805f324e9abfe85ba96dd0d9753713b
2021-04-16 13:27:07 +00:00
Antoine Leblanc
dd1192ca2c server: cleanup of language extensions [gardening]
GitOrigin-RevId: d862c724715cb8a4c2f37d2e0e525d12c46b18eb
2021-04-16 06:55:56 +00:00
Karthikeyan Chinnakonda
2d9959b174 server: add unique constraints to the cron events table (#1131)
GitOrigin-RevId: 3939001a90de041d588c4133ecc1b28b35734d72
2021-04-15 11:21:58 +00:00
Brandon Simmons
8222651c4b fixup 3e9746d5e: /dev/rts_stats should sample on each GET
oops...

GitOrigin-RevId: 3e0a50c4ec2846784d47bcd8d0b62dc6976cbe39
2021-04-14 20:32:01 +00:00
Rakesh Emmadi
f7c37b007a server: use_prepared_statements option in add_pg_source metadata API
GitOrigin-RevId: 7b5af6d1cafc0b95fc86354293b3c3a4669e8bd2
2021-04-14 17:52:17 +00:00
Philip Carlsen
4cb224afd8 Adding support for ST_3DDWithin and ST_3DIntersects
Co-authored-by: Rikin Kachhia <54616969+rikinsk@users.noreply.github.com>
GitOrigin-RevId: de31fa68fd7dc8ffa0c4e4b89b727a5e14b0525e
2021-04-14 13:03:11 +00:00
Naveen Naidu
f6fbf20b80 Server: Format Tracing values to string
GitOrigin-RevId: eaa5aba0bc283f36870910671e3336aca0ea8222
2021-04-14 10:40:51 +00:00
Phil Freeman
3a3da164d7 [server] fix invocation log queries leaking data across tenants
GitOrigin-RevId: ceb568bc3fe05934db28fa7e3f897815bf7af414
2021-04-14 01:24:45 +00:00
kodiakhq[bot]
1b378ae6e8 Add a new /dev/rts_stats endpoint, enabled when '+RTS -T'
GitOrigin-RevId: e347bc2f66bad814516662e34bbd8322c214be25
2021-04-13 17:33:38 +00:00
Abby Sassel
791f136d09 server/mssql: Plug in the Explain API
Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
GitOrigin-RevId: 069055449f6a005f433ee80a4f92b962f21aa714
2021-04-13 11:11:23 +00:00
Sameer Kolhar
b274a2d240 server: disable caching for actions with forward client headers enabled
Co-authored-by: Lyndon Maydwell <92299+sordina@users.noreply.github.com>
Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
Co-authored-by: Abby Sassel <3883855+sassela@users.noreply.github.com>
Co-authored-by: hasura-bot <30118761+hasura-bot@users.noreply.github.com>
Co-authored-by: Rikin Kachhia <54616969+rikinsk@users.noreply.github.com>
Co-authored-by: Ikechukwu Eze <22247592+iykekings@users.noreply.github.com>
Co-authored-by: Aleksandra Sikora <9019397+beerose@users.noreply.github.com>
Co-authored-by: Rishichandra Wawhal <27274869+wawhal@users.noreply.github.com>
Co-authored-by: Naveen Naidu <30195193+Naveenaidu@users.noreply.github.com>
Co-authored-by: Vishnu Bharathi <4211715+scriptnull@users.noreply.github.com>
GitOrigin-RevId: c9a8be3cb607f7767e9d6791717106adf123e3a8
2021-04-13 07:01:34 +00:00
Brandon Simmons
17dc201fef re-Enable HASURA_GRAPHQL_PG_CONN_LIFETIME by default with a default o…
GitOrigin-RevId: 192fc4b80c6bf359334f013af87e20d7de9598ce
2021-04-13 02:55:59 +00:00
Solomon Bothwell
52c2b0384e Adds support for client postgres certificates to multitenant
Co-authored-by: Lyndon Maydwell <92299+sordina@users.noreply.github.com>
GitOrigin-RevId: ca64b51fca128edc215102e773b813be68c9251f
2021-04-13 00:16:34 +00:00
Chris Done
f7a202a363 BigQuery Feature Branch
This will implement BigQuery support.

Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
Co-authored-by: Sibi Prabakaran <737477+psibi@users.noreply.github.com>
Co-authored-by: Aniket Deshpande <922486+aniketd@users.noreply.github.com>
Co-authored-by: Vamshi Surabhi <6562944+0x777@users.noreply.github.com>
GitOrigin-RevId: 1a6ffaf34233e13e8125a5c908eaa7e32d65007b
2021-04-12 10:19:20 +00:00
Swann Moreau
e3f53cbe73 pro-server: add db and pool connection acquire time metrics
Co-authored-by: Anon Ray <616387+ecthiender@users.noreply.github.com>
GitOrigin-RevId: 7624de50d612796dc35b67630765e6cb4206b071
2021-04-08 09:51:43 +00:00
Antoine Leblanc
1a4aad4ba1 server: introduce option to revert to v1 boolean collapse behaviour
GitOrigin-RevId: af6c944270301d8b17618a706ab328a28c0e51dc
2021-04-08 08:26:18 +00:00
Lyndon Maydwell
bec6e4c1c8 Disable schema sync with an interval of 0 instead of an explicit flag
Removing `schemaSyncDisable` flag and interpreting `schemaPollInterval` of `0` as disabling schema sync.

This change brings the convention in line with how action and other intervals are used to disable processes.

There is an opportunity to abstract the notion of an optional interval similar to how actions uses `AsyncActionsFetchInterval`.

This can be used for the following fields of ServeOptions, with RawServeOptions having a milliseconds value where `0` is interpreted as disable.

OptionalInterval:

```
-- | Sleep time interval for activities
data OptionalInterval
  = Skip -- ^ No polling
  | Interval !Milliseconds -- ^ Interval time
  deriving (Show, Eq)
```

ServeOptions:

```
data ServeOptions impl
  = ServeOptions
  {
  ...
  , soEventsFetchInterval           :: !OptionalInterval
  , soAsyncActionsFetchInterval     :: !OptionalInterval
  , soSchemaPollInterval            :: !OptionalInterval
  ...
  }
```

Rather than encoding a `Maybe OptionalInterval` in RawServeOptions, instead a `Maybe Milliseconds` can be used to more directly express the input format, with the ServeOptions constructor interpreting `0` as `Skip`.

Current inconsistencies:

* `soEventsFetchInterval` has no value interpreted as disabling the fetches
* `soAsyncActionsFetchInterval` uses an `OptionalInterval` analog in `RawServeOptions` instead of `Milliseconds`
* `soSchemaPollInterval` currently uses `Milliseconds` directly in `ServeOptions`

---

### Kodiak commit message
Information used by [Kodiak bot](https://kodiakhq.com/) while merging this PR.

#### Commit title
Same as the title of this pull request

GitOrigin-RevId: 3cda1656ae39ae95ba142512ed4e123d6ffeb7fe
2021-04-07 10:01:00 +00:00
Karthikeyan Chinnakonda
ce4794cdcc pro-server: log parameterized query hash for GraphQL queries
GitOrigin-RevId: e804d0b3397a0ae8df05225be43de264babd3de3
2021-04-06 17:53:55 +00:00
Tirumarai Selvan
8be6f55e86 tag release v2.0.0-alpha.7 (#1050)
GitOrigin-RevId: 372e700ee837ef01a18fab3749c0ab9636e4bb88
2021-04-06 11:06:47 +00:00
Naveen Naidu
78827b2f72 server: revert max connection pool settings default to 50
GitOrigin-RevId: 7860c48a88b171b68aee58b0e1932aa679faba14
2021-04-06 08:56:55 +00:00
Lyndon Maydwell
c737ce992d Schema-Sync Improvements for Cloud, Pro, and OSS - Version Checking for Metadata (#738)
Modifying schema-sync implementation to use polling for OSS/Pro. Invalidations are now propagated via the `hdb_catalog.hdb_schema_notifications` table in OSS/Pro. Pattern followed is now a Listener/Processor split with Cloud listening for changes via a LISTEN/NOTIFY channel and OSS polling for resource version changes in the metadata table. See issue #460 for more details.

GitOrigin-RevId: 48434426df02e006f4ec328c0d5cd5b30183db25
2021-04-06 03:25:53 +00:00
Antoine Leblanc
71ae144aa6 server: implement transaction workaround
GitOrigin-RevId: 404f8bbdefb8a99b31fac75e4253daad81c8af90
2021-04-01 20:41:41 +00:00
Rikin Kachhia
e3d8a721ee server: add "SET check_function_bodies=false" in pg_dump clean output
GitOrigin-RevId: f0526f9666c3986d8e3b374556859043f10dbe8d
2021-03-31 14:49:38 +00:00
Antoine Leblanc
60da76ed89 server/mssql: fix foreign keys being wrongfully coalesced
GitOrigin-RevId: 0cfef7176e751d17ea709814ae912574511ca79a
2021-03-31 13:49:31 +00:00
Rakesh Emmadi
362230e8d0 server: async action query subscription
Multi source support had limited the availability of async action queries in subscriptions. This PR
adds support for async action query subscriptions with new implementation. Also addresses https://github.com/hasura/graphql-engine/issues/6460.

GitOrigin-RevId: 5ddc321073d224f287dc4b86ce2239ff55190b36
2021-03-31 10:40:15 +00:00
Karthikeyan Chinnakonda
3e880be55f server: efficient polling for cron triggers and async actions
GitOrigin-RevId: 210df9e6e660b90fb74e930a8909129edba4e7a3
2021-03-30 12:58:40 +00:00
Rakesh Emmadi
7798b9c870 server: allow replacing configuration in add source metadata API (#956)
GitOrigin-RevId: a797d6f1f1e568eeab6acd752d3404a30b890461
2021-03-30 10:10:23 +00:00
Vladimir Ciobanu
c08e6d108b server: allow GeoJSON to be passed for Geometry/Geography operators
GitOrigin-RevId: 51ca927b55d3d717da07447f67fd4d3c068a8357
2021-03-26 17:00:18 +00:00
kodiakhq[bot]
e5b2525d9f Prune unused dependencies, sort dependencies list
GitOrigin-RevId: f6727f11a35925e7b627ecccafa34133dd9e8ee7
2021-03-26 10:45:47 +00:00
Abby Sassel
33a710f4c7 server/docs: Add MSSQL testing steps for those running pytest manually
GitOrigin-RevId: 36eaab863c7bea552a394bd6c36cedb1aa877f50
2021-03-26 09:49:19 +00:00
Antoine Leblanc
d8c56a40f6 server: split boolean operators between core and backend-specific
GitOrigin-RevId: f1291946a1122220e82371676d88867fd7b2b7c4
2021-03-25 17:51:08 +00:00
Antoine Leblanc
9052518da2 Revert #893
GitOrigin-RevId: 055e7c2761ceae028d4fefc282475199714e6efe
2021-03-25 16:38:47 +00:00
Rakesh Emmadi
d0048897cf server: use_prepared_statements option in add_pg_source metadata API (#893)
* add `use_prepared_statements` option to add_pg_source API

* update CHANGELOG.md

* change default value for 'use_prepared_statements' to False

* update CHANGELOG.md

GitOrigin-RevId: 6f5b90991f4a8c03a51e5829d2650771bb0e29c1
2021-03-25 12:54:00 +00:00
Naveen Naidu
bade27c1cc server: emit max connections reached error from postgres
Co-authored-by: Anon Ray <616387+ecthiender@users.noreply.github.com>
GitOrigin-RevId: 26ea3bcb7a2b247f26c9b032b23ac6d68d40b4aa
2021-03-25 07:42:49 +00:00
Sameer Kolhar
a73b6479b1 server/console/docs: add a comment field for actions
####  CHANGELOG (done)

#### PR description

This PR hopes to resolve https://github.com/hasura/graphql-engine/issues/4662.
The main work done in this PR is to add an comment field on the add actions page. Prior to this the `actionsDescription` was used as the comment for actions.

#### Updated UI

![image](https://user-images.githubusercontent.com/6604943/111342362-64dc9880-86a0-11eb-8922-36b72bf100cb.png)

Co-authored-by: Aleksandra Sikora <9019397+beerose@users.noreply.github.com>
GitOrigin-RevId: 5f31b27d66c27bc6c1b0fc5044f0ac13ecc11fa0
2021-03-24 20:11:48 +00:00
Sameer Kolhar
3507699aed console: team console code improvements
GitOrigin-RevId: 03c011b419cb8085633432c030e730953b23f526
2021-03-24 18:04:59 +00:00
Abby Sassel
9f0bcabf23 server/mssql: better support for spatial data operators (fixes #789)
GitOrigin-RevId: b3d59a5c351e747370b82d086fe13d5d8d178d05
2021-03-24 13:44:44 +00:00
Tirumarai Selvan
ce37e7b1f1 tag release v2.0.0-alpha.6 (#947)
GitOrigin-RevId: a214f5e6ba20a0975e5f8a6c2478950f7455549f
2021-03-24 11:26:22 +00:00
David Overton
ad8df00c48 Tests/multisource
GitOrigin-RevId: 603d51d65dc4ff8a64c8d1abd19845bf6a316bde
2021-03-24 06:45:33 +00:00
Antoine Leblanc
9672cc9b32 sever: clean extension types
GitOrigin-RevId: 43f48f90dc96bec87b94a69ca1825511ec9b7507
2021-03-24 02:56:36 +00:00
Karthikeyan Chinnakonda
d236391af8 server: remove restriction of tracking only base function argument types
GitOrigin-RevId: 48feb18f1075bb8fe6a1b27a950198127eaaf3da
2021-03-23 16:04:11 +00:00
Antoine Leblanc
7aa49b8993 ci: improve compilation times by building a dynamic executable
GitOrigin-RevId: ef56e5df7d3c18f07e7f83edadc0cbe5a3b247a6
2021-03-22 21:43:43 +00:00
Vladimir Ciobanu
6dcb3dfc8d server: mssql: fix ToTxt TableName instance
GitOrigin-RevId: e5c1cceb753477c801c4dac570f9f9cfd56f36f0
2021-03-22 20:07:33 +00:00
Rakesh Emmadi
97549ccc8e server: fix action output type generation (#850)
* fix action output type generation

* update CHANGELOG.md

GitOrigin-RevId: 558ff640f5fe5dd5ebf15a3c063f1b989a5fb28a
2021-03-22 10:57:46 +00:00
Abby Sassel
7a75a08dfa server/mssql: belatedly address code review feedback
GitOrigin-RevId: faf3a646fd1067f2e0ed417b97871cd5173ca02e
2021-03-19 16:40:44 +00:00
Antoine Leblanc
f1183b8ae3 server: memoize recursive action input arguments
GitOrigin-RevId: 0dc11f95953f6ad8dfcd82a11f105442042291f8
2021-03-19 14:47:58 +00:00
Abby Sassel
6436ac06e0 server/mssql: support N/IN and N/LIKE operators
GitOrigin-RevId: 6f897b384a4469425d46b1bedf015ecbd1600fcc
2021-03-19 12:43:14 +00:00
Antoine Leblanc
b7e964952f server: move backendTag to its own class.
GitOrigin-RevId: 4d22215611079f01c5f31ab3adc996ec7e5e390e
2021-03-19 11:00:37 +00:00
Antoine Leblanc
405b29c82d server: generate backend-related code with TemplateHaskell
GitOrigin-RevId: 251b8dbeb5181e42cca0369abf54bfc2179ad3d8
2021-03-18 20:35:23 +00:00
Vladimir Ciobanu
f0dd71db59 server: mssql_add_source: allow connection strings from environment
GitOrigin-RevId: d51e14bdfbb57b83a453f5954397792cbe81455b
2021-03-18 18:33:48 +00:00
Antoine Leblanc
1abe5d4902 server: combine instance imports
GitOrigin-RevId: 4deab805ceef1202e5a2d04f0568fa33df3ff791
2021-03-18 17:40:45 +00:00
Karthikeyan Chinnakonda
395de58c42 server: fix inherited_roles issue when some of the underlying roles don't have permissions configured
GitOrigin-RevId: 771491ec0cf6c0d860ef374ed67b426220fa2b96
2021-03-18 15:29:25 +00:00
Karthikeyan Chinnakonda
478c01ab9b server: support tracking functions having only base type arguments
GitOrigin-RevId: e8939ad25212fc0965e0912fc88721a0cdff844e
2021-03-18 12:50:52 +00:00
Tirumarai Selvan
e56e606ddb tag release v2.0.0-alpha.5 (#890)
GitOrigin-RevId: 63d7ffdb732360b96b3dc5ee670b4bb9cf1ef613
2021-03-18 06:30:20 +00:00
Abby Sassel
89d838b85a server: split BoolExpBasic tests into common and backend-specific classes
GitOrigin-RevId: 9e62260881de63e5f89007bb1d6607254625cda7
2021-03-17 10:57:06 +00:00
Vladimir Ciobanu
91710bba58 server: use relative paths in TH splices
While debugging issues with HLS, Reed Mullanix noticed that we don't use relative paths. This leads to problems when using HLS + Emacs due to a bug in `lsp-mode` which prevents it from finding the correct project root.

However, it is still a good practice to use relative paths in TH for other reasons, including being able to import these modules in GHCI.

This PR should make it so HLS-1.0 & emacs provide type inference, imports, etc., in all modules in our codebase.

GitOrigin-RevId: 5f53b9a7ccf46df1ea7be94ff0a5c6ec861f4ead
2021-03-16 17:36:39 +00:00
Abby Sassel
881cc55e0e server: split TestGraphQLQueryBasic into common and backend-specific test classes
GitOrigin-RevId: 9e99047f7ab83f06c94c4741c7c0591eb2ca35e8
2021-03-16 16:42:20 +00:00
Naveen Naidu
babb05451a multitenant: change default connection pool params
GitOrigin-RevId: 44d005c2d8e941d955f4654f21da091e5c1520ae
2021-03-16 15:29:43 +00:00
Karthikeyan Chinnakonda
d1a2fabb26 server: fix the FromJSON instance of ExperimentalFeature
GitOrigin-RevId: 4d3c296021bfc26295632fb4b331c1fe0c902734
2021-03-16 11:49:00 +00:00
Solomon Bothwell
e4b669e56b Traverses RFAction RootField cnstr to accumulate session variables
GitOrigin-RevId: 148bc50c59820254dc40b8951e43abadcac2c25b
2021-03-16 06:13:31 +00:00
Vladimir Ciobanu
da8f6981d4 server: reduce the number of backend dispatches
Fixes https://github.com/hasura/graphql-engine-mono/issues/712

Main point of interest: the `Hasura.SQL.Backend` module.

This PR creates an `Exists` type indexed by indexed type and packed constraint while hiding all of its complexity by not exporting the constructor.

Existential constructors/types which are no longer (directly) existential:
- [X] BackendSourceInfo :: BackendSourceInfo
- [x] BackendSourceMetadata :: BackendSourceMetadata
- [x] MOSourceObjId :: MetadatObjId
- [x] SOSourceObj :: SchemaObjId
- [x] RFDB :: RootField
- [x] LQP :: LiveQueryPlan
- [x] ExecutionStep :: ExecStepDB

This PR also removes ALL usages of `Typeable.cast` from our codebase. We still need to derive `Typeable` in a few places in order to be able to derive `Data` in one place. I have not dug deeper to see why this is needed.

GitOrigin-RevId: bb47e957192e4bb0af4c4116aee7bb92f7983445
2021-03-15 13:03:55 +00:00
Karthikeyan Chinnakonda
6093010a6e server: fix issue of parsing input objects in remote schema queries
GitOrigin-RevId: 98c62cab836aff2778dbfedd6535450989090a30
2021-03-15 06:32:07 +00:00
Antoine Leblanc
a544fe9822 server: generalized LogQueryLog (fix #708)
GitOrigin-RevId: 0e18e0f5caaa6889804debf7111ef9aa99e080b5
2021-03-13 14:41:44 +00:00
Abby Sassel
64d52f5fa3 server: generalize integration tests
Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
GitOrigin-RevId: 647edb8d293bf783c504b7e15ce02c56858b3b72
2021-03-11 18:18:41 +00:00
Abby Sassel
660b442943 server: misc. dev.sh improvements
Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
GitOrigin-RevId: 4c93b49a370afb98747aabc1c977494297432402
2021-03-10 22:19:14 +00:00
Karthikeyan Chinnakonda
84d55cf771 docs: add a guide on writing python tests
Co-authored-by: Rikin Kachhia <54616969+rikinsk@users.noreply.github.com>
GitOrigin-RevId: 37528597f947a62b4c119cad8feb5428b904a00b
2021-03-10 17:30:41 +00:00
Tirumarai Selvan
692e374c94 tag release v2.0.0-alpha.4
GitOrigin-RevId: fdde115ba4442795494566f2d6a2aacbf50f0632
2021-03-10 11:48:44 +00:00
Anon Ray
d16bc4fe46 server: fix handling of nullable object relationships
GitOrigin-RevId: 3214e77ed06a07b6c1293d59daa088649a881e78
2021-03-10 08:56:03 +00:00
Rakesh Emmadi
2b0c990d50 server: remove Action.hs-boot file
GitOrigin-RevId: 65267a7b6343a12b690f7710785fdfa30873e479
2021-03-10 07:27:19 +00:00
Lyndon Maydwell
0c1016e065 Inconsistent metadata support for REST endpoints
Previously invalid REST endpoints would throw errors during schema cache build.

This PR changes the validation to instead add to the inconsistent metadata objects in order to allow use of `allow_inconsistent_metadata` with inconsistent REST endpoints.

All non-fatal endpoint definition errors are returned as inconsistent metadata warnings/errors depending on the use of `allow_inconsistent_metadata`. The endpoints with issues are then created and return informational runtime errors when they are called.

Console impact when creating endpoints is that error messages now refer to metadata inconsistencies rather than REST feature at the top level:

![image](https://user-images.githubusercontent.com/92299/109911843-ede9ec00-7cfe-11eb-9c55-7cf924d662a6.png)

<img width="969" alt="image" src="https://user-images.githubusercontent.com/92299/110258597-8336fa00-7ff7-11eb-872c-bfca945aa0e8.png">

Note: Conflicting endpoints generate one error per conflicting set of endpoints due to the implementation of `groupInconsistentMetadataById` and `imObjectIds`. This is done to ensure that error messages are terse, but may pose errors if there are some assumptions made surrounding `imObjectIds`.

Related to https://github.com/hasura/graphql-engine-mono/pull/473 (Allow Inconsistent Metadata (v2) #473 (Merged))

---

### Kodiak commit message

Changes the validation to use inconsistent metadata objects for REST endpoint issues.

#### Commit title

Inconsistent metadata for REST endpoints

GitOrigin-RevId: b9de971208e9bb0a319c57df8dace44cb115ff66
2021-03-10 05:26:10 +00:00
Karthikeyan Chinnakonda
71beba0c54 server: add experimental features in the alpha config API
GitOrigin-RevId: b4841e3dbea8340f8fad638359f3b1cb28188db6
2021-03-09 18:46:42 +00:00
Karthikeyan Chinnakonda
0a7d634326 server: fix issue when remote relationship col has a custom GQL name
GitOrigin-RevId: ca45383020d0e8f80afb445ab8bb240cc1505ea9
2021-03-09 09:25:29 +00:00
Karthikeyan Chinnakonda
92026b769f [Preview] Inherited roles for postgres read queries
fixes #3868

docker image - `hasura/graphql-engine:inherited-roles-preview-48b73a2de`

Note:

To be able to use the inherited roles feature, the graphql-engine should be started with the env variable `HASURA_GRAPHQL_EXPERIMENTAL_FEATURES` set to `inherited_roles`.

Introduction
------------

This PR implements the idea of multiple roles as presented in this [paper](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/FGALanguageICDE07.pdf). The multiple roles feature in this PR can be used via inherited roles. An inherited role is a role which can be created by combining multiple singular roles. For example, if there are two roles `author` and `editor` configured in the graphql-engine, then we can create a inherited role with the name of `combined_author_editor` role which will combine the select permissions of the `author` and `editor` roles and then make GraphQL queries using the `combined_author_editor`.

How are select permissions of different roles are combined?
------------------------------------------------------------

A select permission includes 5 things:

1. Columns accessible to the role
2. Row selection filter
3. Limit
4. Allow aggregation
5. Scalar computed fields accessible to the role

 Suppose there are two roles, `role1` gives access to the `address` column with row filter `P1` and `role2` gives access to both the `address` and the `phone` column with row filter `P2` and we create a new role `combined_roles` which combines `role1` and `role2`.

Let's say the following GraphQL query is queried with the `combined_roles` role.

```graphql
query {
   employees {
     address
     phone
   }
}
```

This will translate to the following SQL query:

```sql

 select
    (case when (P1 or P2) then address else null end) as address,
    (case when P2 then phone else null end) as phone
 from employee
 where (P1 or P2)
```

The other parameters of the select permission will be combined in the following manner:

1. Limit - Minimum of the limits will be the limit of the inherited role
2. Allow aggregations - If any of the role allows aggregation, then the inherited role will allow aggregation
3. Scalar computed fields - same as table column fields, as in the above example

APIs for inherited roles:
----------------------

1. `add_inherited_role`

`add_inherited_role` is the [metadata API](https://hasura.io/docs/1.0/graphql/core/api-reference/index.html#schema-metadata-api) to create a new inherited role. It accepts two arguments

`role_name`: the name of the inherited role to be added (String)
`role_set`: list of roles that need to be combined (Array of Strings)

Example:

```json
{
  "type": "add_inherited_role",
  "args": {
      "role_name":"combined_user",
      "role_set":[
          "user",
          "user1"
      ]
  }
}
```

After adding the inherited role, the inherited role can be used like single roles like earlier

Note:

An inherited role can only be created with non-inherited/singular roles.

2. `drop_inherited_role`

The `drop_inherited_role` API accepts the name of the inherited role and drops it from the metadata. It accepts a single argument:

`role_name`: name of the inherited role to be dropped

Example:

```json

{
  "type": "drop_inherited_role",
  "args": {
      "role_name":"combined_user"
  }
}
```

Metadata
---------

The derived roles metadata will be included under the `experimental_features` key while exporting the metadata.

```json
{
  "experimental_features": {
    "derived_roles": [
      {
        "role_name": "manager_is_employee_too",
        "role_set": [
          "employee",
          "manager"
        ]
      }
    ]
  }
}
```

Scope
------

Only postgres queries and subscriptions are supported in this PR.

Important points:
-----------------

1. All columns exposed to an inherited role will be marked as `nullable`, this is done so that cell value nullification can be done.

TODOs
-------

- [ ] Tests
   - [ ] Test a GraphQL query running with a inherited role without enabling inherited roles in experimental features
   - [] Tests for aggregate queries, limit, computed fields, functions, subscriptions (?)
   - [ ] Introspection test with a inherited role (nullability changes in a inherited role)
- [ ] Docs
- [ ] Changelog

Co-authored-by: Vamshi Surabhi <6562944+0x777@users.noreply.github.com>
GitOrigin-RevId: 3b8ee1e11f5ceca80fe294f8c074d42fbccfec63
2021-03-08 11:15:10 +00:00
Naveen Naidu
f55df228fe Server: Add Event Trigger Metrics
- [x] **Event Triggers Metrics**
  - [x] Distribution of size of event trigger fetches / Number of events fetched in the last `event trigger fetch`
  - [x] Event Triggers: Number of event trigger HTTP workers in process
  - [x]  Event Triggers: Avg event trigger lock time (if an event has been fetched but not processed because http worker is not free)

#### Sample response
The metrics can be viewed from the `/dev/ekg` endpoint

```json
{
"num_events_fetched":{
      "max":0,
      "mean":0,
      "count":1,
      "min":0,
      "variance":null,
      "type":"d",
      "sum":0
   },
 "num_event_trigger_http_workers":{
      "type":"g",
      "val":0
   },
"event_lock_time":{
      "max":0,
      "mean":0,
      "count":0,
      "min":0,
      "variance":0,
      "type":"d",
      "sum":0
   },
```

#### Todo
- [ ]  Group similar metrics together (Eg: Group all the metrics related to Event trigger, How do we do it??)

Closes: https://github.com/hasura/graphql-engine-mono/issues/202
GitOrigin-RevId: bada11d871272b04c8a09d006d9d037a8464a472
2021-03-07 05:25:24 +00:00
Antoine Leblanc
5a1f71b3bc server/mssql: fix parsing of text column values
GitOrigin-RevId: c4692cdc254af074f70cdceb78c7dc6211e7e084
2021-03-05 18:33:16 +00:00
Sameer Kolhar
78785ae513 server: fix fromJSON for RemoteSchemaPermsCtx
GitOrigin-RevId: a0a6d50a39ffbc7fab0a303b50c3e2c485954773
2021-03-05 13:26:57 +00:00
Rakesh Emmadi
edf5a0661e server: support mssql views
GitOrigin-RevId: 50e2fc1d64a67a9284ae62d2401ca6f9627ebdab
2021-03-05 10:53:38 +00:00
Sameer Kolhar
642c950827 server: fix ToJSON instance of RemoteSchemaPermsCtx
GitOrigin-RevId: 7e8ffa747bb645ce6b606ef38a16120f420832bf
2021-03-05 07:47:00 +00:00
Sameer Kolhar
ee56b741d8 server: add is_remote_schema_permissions_enabled to config API
GitOrigin-RevId: 7129d0fba25edc2450c74140335df9398154ce52
2021-03-04 16:22:31 +00:00
Tirumarai Selvan
d1d8cc0a40 tag release v2.0.0-alpha.3 (#790)
GitOrigin-RevId: 7b61e8190d07ebe45ff96ee5209992dfc3c53b93
2021-03-04 14:30:32 +00:00
hasura-bot
da3c9f2c53 minor typo fixes (#165)
Co-authored-by: Colby Thomas <coloradocolby@gmail.com>
GITHUB_PR_NUMBER: 6311
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/6311

Co-authored-by: Colby Thomas <coloradocolby@gmail.com>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
GitOrigin-RevId: 946665878fb9199d067ad1bd981de9525e86a0ae
2021-03-04 06:36:15 +00:00
Vladimir Ciobanu
d5ff1acf2d better handling for one-to-one relationships
Co-authored-by: Rikin Kachhia <54616969+rikinsk@users.noreply.github.com>
GitOrigin-RevId: 1bb5bc0c4ac8109ee1d20563d23cf98e0906a483
2021-03-03 13:02:59 +00:00
Karthikeyan Chinnakonda
15ed0cf536 server: document the remote joins architecture
GitOrigin-RevId: f142908fa3dd3b0cb8887521c639be4a017e606a
2021-03-03 11:47:13 +00:00
Vladimir Ciobanu
d4d02c08d8 update ci-info-hs for cabal 3.4 (#747)
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
GitOrigin-RevId: 29bed5510b971572bf545e588065bbe1b5743d2c
2021-03-03 07:02:42 +00:00
David Overton
456b6d5664 Tests for metadata resource version
GitOrigin-RevId: c1db1355ef8c6617db8179c8c50ba2bd91f7062f
2021-03-03 04:41:02 +00:00
Karthikeyan Chinnakonda
d2418383bf server: add https remote schema tests
GitOrigin-RevId: 9e9e66248f956174dc8197adeb7a48da12e53dac
2021-03-02 16:52:47 +00:00
Karthikeyan Chinnakonda
f2724c97b4 build: fix the packaging of the static console assets
GitOrigin-RevId: 01a6e4f15366476f01ca355f1013712e24bb4a3b
2021-03-02 13:23:53 +00:00
David Overton
e4c8d560c5 Fix/missing source error
GitOrigin-RevId: 074f4eb2d13a2c8ab002a355c6734eed257482d3
2021-03-02 04:27:29 +00:00
Vladimir Ciobanu
88ce387919 docs: add note about recovering existentially quantified type info
This should be merged after #686

GitOrigin-RevId: a372ca475a9c4a182b4819310634d4c3985225ab
2021-03-01 22:31:55 +00:00
Vladimir Ciobanu
ddbc497506 docs: add note on existentials and references to it
Added a note on existentials. I plan to create a subsequent PR with a note on how we use the singletons trick to recover the type inside an existential.

GitOrigin-RevId: 1f227d859dcc384b4ac7e103053f643f879827d1
2021-03-01 21:38:50 +00:00
Abby Sassel
01cb59919f docs: changelog & server documentation improvements
GitOrigin-RevId: e6a5e464acb5492ab7a592362fff2bac6528dfaa
2021-03-01 20:53:43 +00:00
Praveen Durairaju
0ff26100bf update docs link to avoid redirects
GitOrigin-RevId: 1f2a1d21bfb9b2908d56305fa2dfb69270deafdf
2021-03-01 18:51:18 +00:00
Vamshi Surabhi
e9f85ce6e6 mssql: connection pooling
GitOrigin-RevId: c1a6509f19a903724ce2b770ae23cbd925b537f8
2021-02-25 18:16:52 +00:00
Abby Sassel
fb1a0d286d server: support ltree operators (close #625)
GitOrigin-RevId: fb6a3eb8cbe4604789938bcbc78916fbcd1af515
2021-02-25 11:06:49 +00:00
Antoine Leblanc
f55bbe96ef server: fix MSSQL returning malformed JSON on empty select
GitOrigin-RevId: b0b9f68dbdb2b015e4c0640670a2e4cf978fe5fb
2021-02-25 09:58:24 +00:00
Karthikeyan Chinnakonda
4211d27272 server: support reading JWT from Cookie header
GitOrigin-RevId: 1de90242d3c000361f87256c2dddce1677863231
2021-02-25 09:03:46 +00:00
Vamshi Surabhi
74a58fb66c mssql: basic support for geography and geometry types
GitOrigin-RevId: 97bbf0e8d53b00c9f4365b145adb19bdcd769ceb
2021-02-24 12:53:15 +00:00
Lyndon Maydwell
b944d598e0 Adding check for multiple query definitions for rest endpoints
When adding a rest endpoint that references a query with multiple definitions, throw an error.

Needed a second PR for this as Kodiak merged the prior PR before this commit was added.

---

### Kodiak commit message

Adding check for multiple query definitions for rest endpoints.

GitOrigin-RevId: 6e3977a210e29f143b61282fbb37c93eb5b9d73c
2021-02-24 10:13:11 +00:00
Vamshi Surabhi
bde1a4d2f2 bump odbc version (#660)
GitOrigin-RevId: 43873267167dd742d3a6baeb5e1434e2a968a4ca
2021-02-24 07:47:07 +00:00
Lyndon Maydwell
08da0c63b6 REST Endpoints - Prohibit Invalid slashes, duplicate variables, non-singular query definitions, subscriptions
Resolves Issues:

* https://github.com/hasura/graphql-engine-mono/issues/658 - Invalid Slashes
* https://github.com/hasura/graphql-engine-mono/issues/628 - Subscriptions

Implementation:

* Moved some logic from Endpoint.hs to allow reuse of splitting url into PathSegments.
* Additional validation steps alongside checking for overlapping routes
* Logging potential misuse of GET for mutations

Future Work:

* [ ] GET is allowed for mutations (Ignore/Log warning for Now)
* [ ] Add to scInconsistentObjs rather than throwing error
  * Add information to scInconsistentObjs instead of raising errors directly.

TODO:

* [x] Duplicate variable segments with the same name in the location should not be allowed
* [x] We should throw an error on trailing and leading slashes and URLs which contain empty segments
* [x] Endpoints can be created using subscriptions. But the error only shows at the time of the query
* [x] Tests

---

### Kodiak commit message

Prohibit Invalid slashes, duplicate variables, subscriptions for REST endpoints.

GitOrigin-RevId: 86c0d4af97984c8afd02699e6071e9c1658710b8
2021-02-24 04:31:05 +00:00
Antoine Leblanc
7527039f08 server: fix build
GitOrigin-RevId: 8ba687fce6892ac32182311cccc1d4d149049d62
2021-02-23 21:43:56 +00:00
Vladimir Ciobanu
281cb771ff server: add MSSQL support
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
Co-authored-by: Vamshi Surabhi <6562944+0x777@users.noreply.github.com>
Co-authored-by: Aravind K P <8335904+scriptonist@users.noreply.github.com>
GitOrigin-RevId: 699c453b9692e1b822f393f23ff5e6db4e010d57
2021-02-23 17:38:36 +00:00
Anon Ray
cad013c039 multitenant: fix cloud schema files
GitOrigin-RevId: 99f04df3654ab83f64489960b5f7ecf5f86c9a99
2021-02-22 16:03:07 +00:00
Naveen Naidu
4fa34e1d5e server: Apply migrations when catalog version < 43
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
GitOrigin-RevId: 4fa1a04f4924002faac1eca2e5092dfa8767fdb9
2021-02-22 15:00:24 +00:00
Abby Sassel
b00009f4bb server: fix issue with queries on character column types (close #6217)
GitOrigin-RevId: 81eaa0db63656b1d37b9b71736d4309af902cc47
2021-02-22 10:21:21 +00:00
Anon Ray
0dfe83f931 server: optimize resolving source
GitOrigin-RevId: e0513a714b5e3f6d730b0470e1836ed44badd3c0
2021-02-22 07:53:36 +00:00
Antoine Leblanc
377425ff2d server: generalize subscriptions
GitOrigin-RevId: 464e80abf151032dc50eaf6cf8dafc5e7cfa51cd
2021-02-20 13:46:43 +00:00
David Overton
7e79f8dc64 Fix migrations
Provides queries for fetching and inserting metadata into that database that do not assume there is a `resource_version` column. This means that will work when migrating to/from older versions.

Co-authored-by: Lyndon Maydwell <92299+sordina@users.noreply.github.com>
GitOrigin-RevId: dac636d530524082c5a13ae0f016a2d4ced16f7f
2021-02-19 08:47:10 +00:00
Lyndon Maydwell
59c01786fe Optimistic Metadata Locking
Add optimistic concurrency control to the ‘replace_metadata’ call.

Prevents users from submitting out-of-date metadata to metadata-mutating APIs.

See https://github.com/hasura/graphql-engine-mono/issues/472 for details.

GitOrigin-RevId: 5f220f347a3eba288a9098b01e9913ffd7e38166
2021-02-19 02:40:23 +00:00
Karthikeyan Chinnakonda
7be72d7bee server: support for maintenance mode in v1.4
Co-authored-by: Anon Ray <616387+ecthiender@users.noreply.github.com>
GitOrigin-RevId: 85f636b1bc8063689845da90e85cc480e87dca7e
2021-02-18 16:47:22 +00:00
Anon Ray
1091203ced multitenant: fix issues with cloud schema sync
GitOrigin-RevId: 7749fe71a1174ac8dbfb79734584c60d882f07b7
2021-02-18 15:12:26 +00:00
Karthikeyan Chinnakonda
77b3bb5a20 server: run function permissions tests in CI
GitOrigin-RevId: 750b44d0f30df506eb8f0a8eec42fcd1a7d51599
2021-02-18 08:27:00 +00:00
Lyndon Maydwell
0d67b5cb53 Allow Inconsistent Metadata (v2) (#473)
Allow Inconsistent Metadata

GitOrigin-RevId: 7b7e1fdccbeaa1b7dccf66ff22c4b40f8cf5bb27
2021-02-16 08:10:21 +00:00
Vladimir Ciobanu
7290a57308 rename 'Connection Timeout' to 'Pool Timeout'
GitOrigin-RevId: f2f03a368b372d83bd9c9e71e2069310ba7e6f09
2021-02-16 05:19:49 +00:00
Vladimir Ciobanu
83c74ac133 connection acquisition timeout for oss and pro
Co-authored-by: Vamshi Surabhi <6562944+0x777@users.noreply.github.com>
GitOrigin-RevId: 673452c1fed2494b437b5fdd1f195e48b10f7634
2021-02-15 13:33:21 +00:00
Rakesh Emmadi
9ef603360c server: generalize schema cache building (#496)
Co-authored-by: Vamshi Surabhi <vamshi@hasura.io>
Co-authored-by: Vladimir Ciobanu <admin@cvlad.info>
Co-authored-by: Antoine Leblanc <antoine@hasura.io>
Co-authored-by: Stylish Haskell Bot <stylish-haskell@users.noreply.github.com>
GitOrigin-RevId: 9d631878037637f3ed2994b5d0525efd978f7b8f
2021-02-14 06:08:46 +00:00
Phil Freeman
7fffc11077 Caching, Rate Limiting, Metrics & Session Variable Improvements (#376)
* server: use a leaky bucket algorithm for bytes-per-second cache rate limiting

* Use evalsha properly

* Adds redis cache limit parameters to PoliciesConfig

* Loads Leaky Bucket Script On Server Start

* Adds more redis logging and moves cache update into lua script

* reverts setex in lua and adds notes

* Refactors cacheStore and adds max TTL and cache size limits

* Filter session vars in cache key

* WIP

* parens

* cache-clear-hander POC implementation

* cache-clear-hander POC implementation

* Pro projectId used as cache key

* POC working!

* prefixing query-response keys in redis

* Add cacheClearer to RedisScripts

* Partial implementation of cacheClearer from scripts record

* updating tests

* [automated] stylish-haskell commit

* Adds query look with up with metrics script

* Adds missing module and lua script from last commit

* Changes redis script module structure to match cache clearing branch

* minor change to lua script

* cleaning up cache clearing

* generalising JsonLog

* [automated] stylish-haskell commit

* Draft Cache Metrics Endpoint

* Adds Cache Metrics Handler

* Adds hook handler module

* Missed HandlerHook module in last commit

* glob

* Fixes redis mget bug

* Removes cache totals and changes dashes to colons in metric cache keys

* Adds query param to clear clear endpoint for deleting specific keys

* Adds query param to clear clear endpoint for deleting specific keys

* Cache Metrics on query families rather then queries

* Replace Set with nub

* Base16 Redis Hashes

* Query Family Redis Keys With Roles

* response headers for cache keys

* fixing bug in family key by excluding operation name; using hash for response header instead of entire key

* Adds query family to redis cache keys and cache clear endpoint

* Fixes queryfamily hash bug

* Moves cache endpoints to /pro

* Moved cache clear to POST

* Refactors cache clear function

* Fixes query family format bug

* Adds query cache tests and optional --redis-url flag to python test suite

* Adds session variable cache test

* Update pro changelog

* adding documentation for additional caching features

* more docs

* clearing up units of leaky bucket params

* Adds comments to leaky bucket script

* removes old todo

* Fixes session variable filtering to work with new query rootfield

* more advanced defaulting behaviour for bucket rate and capacity.

* Updates Docs

* Moves Role into QueryFamily hash

* Use Aeson for Cache Clear endpoint response

* Moves trace to bracket the leaky bucket script

* Misc review tweaks

* Adds sum type for cache clear query params

* Hardcodes RegisReplyLog log level

* Update docs/graphql/cloud/response-caching.rst

Co-authored-by: Phil Freeman <phil@hasura.io>

* new prose for rate limiting docs

* [automated] stylish-haskell commit

* make rootToSessVarPreds total

* [automated] stylish-haskell commit

* Fixes out of scope error

* Renamed _acRedis to _acCacheStore

Co-authored-by: Solomon Bothwell <ssbothwell@gmail.com>
Co-authored-by: Lyndon Maydwell <lyndon@sordina.net>
Co-authored-by: David Overton <david@hasura.io>
Co-authored-by: Stylish Haskell Bot <stylish-haskell@users.noreply.github.com>
Co-authored-by: Lyndon Maydwell <lyndon@hasura.io>
GitOrigin-RevId: dda5c1a3f902967b3d78310f950541a55fabb1b0
2021-02-13 00:06:18 +00:00
Sameer Kolhar
8655e6fd3a console: function permissions UI (#413)
GitOrigin-RevId: ccdfab19751b0d238a4ebcec59ba73a798103ca9
2021-02-12 17:02:41 +00:00
Karthikeyan Chinnakonda
8a68cc6650 server: don't expose remote schema mutations in relay
GitOrigin-RevId: 21c126bfd46a22d3b0e64d57e92d78ea3c4df9ab
2021-02-12 13:29:19 +00:00
Antoine Leblanc
1ca4034697 server: generalize execution of queries and mutations
GitOrigin-RevId: aff477a0849d4667f2f6dc6804d6ca2982e53f95
2021-02-12 03:05:05 +00:00
Antoine Leblanc
5b0fc9a365 server: update cabal dependencies
GitOrigin-RevId: a30b1dc80455059330cdae7c9084eb70552d54d0
2021-02-12 02:59:00 +00:00
Matthew Pickering
35b81f39e9 Memory performance improvements from Cherre (#518)
* Stop shutdown handler retaining the whole serveCtx

This might look like quite a strange way to write the function but it's
the only way I could get GHC to not capture `serveCtx` in the shutdown
handler.

Fixes the metadata issue in #344

* Force argumentNames

The arguments list is often empty so we end up with a lot of duplicate
thunks if this value is not forced.

* Increase sharing in nullableType and nonNullableType

The previous definitions would lead to increased allocation as it would
destory any previously created sharing. The new definition only allocate
a fresh constructor if the value is changed.

* Add memoization for field parsers

It was observed in #344 that many parsers were not being memoised which
led to an increase in memory usage. This patch generalisation memoisation so
that it works for FieldParsers as well as normal Parsers.

There can still be substantial improvement made by also memoising
InputFieldParsers but that is left for future work.

Co-authored-by: Antoine Leblanc <antoine@hasura.io>

* [automated] stylish-haskell commit

* changelog

Co-authored-by: Phil Freeman <paf31@cantab.net>
Co-authored-by: Antoine Leblanc <antoine@hasura.io>
Co-authored-by: Stylish Haskell Bot <stylish-haskell@users.noreply.github.com>
Co-authored-by: Phil Freeman <phil@hasura.io>
GitOrigin-RevId: 36255f77a47cf283ea61df9d6a4f9138d4e5834c
2021-02-12 01:34:56 +00:00
Anon Ray
06b599b747 server: multitenant metadata storage
The metadata storage implementation for graphql-engine-multitenant.

- It uses a centralized PG database to store metadata of all tenants (instead of per tenant database)
- Similarly, it uses a single schema-sync listener thread per MT worker (instead of listener thread per tenant) (PS: although, the processor thread is spawned per tenant)
- 2 new flags are introduced - `--metadataDatabaseUrl` and (optional) `--metadataDatabaseRetries`

Internally, a "metadata mode" is introduced to indicate an external/managed store vs a store managed by each pro-server.

To run :
- obtain the schema file (located at `pro/server/res/cloud/metadata_db_schema.sql`)
- apply the schema on a PG database
- set the `--metadataDatabaseUrl` flag to point to the above database
- run the MT executable

The schema (and its migrations) for the metadata db is managed outside the MT worker.

### New metadata

The following is the new portion of `Metadata` added :

```yaml
version: 3
metrics_config:
  analyze_query_variables: true
  analyze_response_body: false
api_limits:
  disabled: false
  depth_limit:
    global: 5
    per_role:
      user: 7
      editor: 9
  rate_limit:
    per_role:
      user:
        unique_params:
        - x-hasura-user-id
        - x-hasura-team-id
        max_reqs_per_min: 20
    global:
      unique_params: IP
      max_reqs_per_min: 10
```

- In Pro, the code around fetching/updating/syncing pro-config is removed
- That also means, `hdb_pro_catalog` for keeping the config cache is not required. Hence the `hdb_pro_catalog` is also removed
- The required config comes from metadata / schema cache

### New Metadata APIs

- `set_api_limits`
- `remove_api_limits`
- `set_metrics_config`
- `remove_metrics_config`

#### `set_api_limits`

```yaml
type: set_api_limits
args:
  disabled: false
  depth_limit:
    global: 5
    per_role:
      user: 7
      editor: 9
  rate_limit:
    per_role:
      anonymous:
         max_reqs_per_min: 10
         unique_params: "ip"
      editor:
        max_reqs_per_min: 30
        unique_params:
        - x-hasura-user-id
      user:
        unique_params:
        - x-hasura-user-id
        - x-hasura-team-id
        max_reqs_per_min: 20
    global:
      unique_params: IP
      max_reqs_per_min: 10
```

#### `remove_api_limits`

```yaml
type: remove_api_limits
args: {}
```

#### `set_metrics_config`

```yaml
type: set_metrics_config
args:
  analyze_query_variables: true
  analyze_response_body: false
```

#### `remove_metrics_config`

```yaml
type: remove_metrics_config
args: {}
```

#### TODO
- [x] on-prem pro implementation for `MonadMetadataStorage`
- [x] move the project config from Lux to pro metadata (PR: #379)
- [ ] console changes for pro config/api limits, subscription workers (cc @soorajshankar @beerose)
- [x] address other minor TODOs
  - [x] TxIso for `MonadSourceResolver`
  - [x] enable EKG connection pool metrics
  - [x] add logging of connection info when sources are added?
  - [x] confirm if the `buildReason` for schema cache is correct
- [ ] testing
- [x] 1.3 -> 1.4 cloud migration script (#465; PR: #508)
  - [x] one-time migration of existing metadata from users' db to centralized PG
  - [x] one-time migration of pro project config + api limits + regression tests from metrics API  to metadata
- [ ] integrate with infra team (WIP - cc @hgiasac)
  - [x] benchmark with 1000+ tenants + each tenant making read/update metadata query every second (PR: https://github.com/hasura/graphql-engine-mono/pull/411)
  - [ ] benchmark with few tenants having large metadata (100+ tables etc.)
  - [ ] when user moves regions (https://github.com/hasura/lux/issues/1717)
    - [ ] metadata has to be migrated from one regional PG to another
    - [ ] migrate metrics data as well ?
      - [ ] operation logs
      - [ ] regression test runs

- [ ] find a way to share the schema files with the infra team

Co-authored-by: Naveen Naidu <30195193+Naveenaidu@users.noreply.github.com>
GitOrigin-RevId: 39e8361f2c0e96e0f9e8f8fb45e6cc14857f31f1
2021-02-11 17:55:21 +00:00
Karthikeyan Chinnakonda
0cfad08993 server: function permissions changes
GitOrigin-RevId: b0d784dd23aacd7a62cf471dd33eca869e25c13e
2021-02-11 09:44:38 +00:00
Antoine Leblanc
9b38863c73 server: change RootField so that only the DB component is existential
GitOrigin-RevId: ff620b775ea9b1c8433255705254ed9d58eba290
2021-02-09 19:30:08 +00:00
Antoine Leblanc
7760ed2d98 server: generalize top-level build functions
GitOrigin-RevId: 5ace65f702b411bf75c84e949742c7d657382fab
2021-02-09 12:48:22 +00:00
Vladimir Ciobanu
6b7b68504a server: improve the UX around SourceInfo
GitOrigin-RevId: 47f7e9b7daa3f89771dd3883e9f6050274827e27
2021-02-08 16:31:17 +00:00
Antoine Leblanc
e449cf9990 server: finish generalizing mutations
GitOrigin-RevId: 645eefadee35f1642eee805e20161417ab38b949
2021-02-08 10:40:51 +00:00
Antoine Leblanc
83701fb63e server: changes to support other backends
GitOrigin-RevId: ec0ad47957ab6f9a0855623fffedb23924e7c75d
2021-02-03 16:25:17 +00:00
Karthikeyan Chinnakonda
312e56dfa6 server: support tracking of custom functions which return single row
GitOrigin-RevId: fbf99816a70e666bb9b02fca92a0ee7d419cdff8
2021-02-03 14:07:30 +00:00
Swann Moreau
c14dcd5792 pass gql requests into auth webhook POST body (#149)
* fix arg order in UserAuthentication instance [force ci]

* change the constructor name to AHGraphQLRequest

Co-authored-by: Stylish Haskell Bot <stylish-haskell@users.noreply.github.com>
Co-authored-by: Karthikeyan Chinnakonda <karthikeyan@hasura.io>
GitOrigin-RevId: fb3258f4a84efc6c730b0c6222ebd8cea1b91081
2021-02-03 07:11:39 +00:00
Karthikeyan Chinnakonda
94a886ee94 server: fix mutation functions not being exposed to admin when function permissions are inferred
GitOrigin-RevId: 7d8031341351bdbf25d4244ce57d4f9396dde437
2021-02-01 12:58:31 +00:00
Karthikeyan Chinnakonda
05f21a2768 server: fix the FromJSON instance of FunctionPermissionsCtx
GitOrigin-RevId: 506d654ef9cad95ffdddbf832a2779fc34e6bd00
2021-02-01 06:04:28 +00:00
Swann Moreau
0b294bab5c update resource limits on circleci [force ci]
GitOrigin-RevId: 0ed0c7d32400f0a940be7ad1b22a6b55b751cbb5
2021-01-29 14:36:07 +00:00
Tirumarai Selvan A
88c3d7c787 tag release v1.4.0-alpha.1
GitOrigin-RevId: f09f265175226f96faba53ccfe68349377523f98
2021-01-29 12:31:13 +00:00
Vladimir Ciobanu
1bda14960e update pg-client-hs to revert buffer shrinking
GitOrigin-RevId: f8c47c2551c09a9e9f0d57a68c983f580d796c00
2021-01-29 12:04:39 +00:00
Karthikeyan Chinnakonda
10a3f9960d server: new function permissions layer
Co-authored-by: Rikin Kachhia <54616969+rikinsk@users.noreply.github.com>
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
GitOrigin-RevId: 35645121242294cb6bb500ea598e9a1f2ca67fa1
2021-01-29 05:49:09 +00:00
Lyndon Maydwell
0767333597 server: support restified versions of graphql queries (#303)
Restified GraphQL Endpoints feature.

GitOrigin-RevId: 3d6e589426ec21a60a915b47f579f0ac4934af45
2021-01-29 01:03:35 +00:00
Rakesh Emmadi
ff3c58f230 ci: fix oss test server upgrade ci test when upgrading from 1.3.3
GitOrigin-RevId: a5b663ea5b079212c063f15bc67559f5a234f96f
2021-01-28 11:40:24 +00:00
Antoine Leblanc
353859db09 server: remove GraphQL.Utils
GitOrigin-RevId: 90639f9f3d263ccb0ce4e3b8b6e19ce784f4b25d
2021-01-26 13:14:35 +00:00
Antoine Leblanc
62a3bb0d9e Move text operator out of SQL.Types. (#406)
GitOrigin-RevId: 25adad545b302fd1dde25ea0365d492dfcdaa71d
2021-01-26 12:23:20 +00:00
Antoine Leblanc
6494229f54 server: generalize functions (#393)
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
GitOrigin-RevId: 5d2140152a2a18601c785ea80a7689cbe3bd277e
2021-01-25 10:13:54 +00:00
Swann Moreau
d4b474061f server: split server test suite to run in parallel on circleci (#309)
Co-authored-by: rakeshkky <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Vishnu Bharathi P <vishnubharathi04@gmail.com>
Co-authored-by: Vamshi Surabhi <0x777@users.noreply.github.com>
Co-authored-by: Nizar Malangadan <nizar-m@users.noreply.github.com>
Co-authored-by: nizar-m <19857260+nizar-m@users.noreply.github.com>
GitOrigin-RevId: b8a1e9c3ba4797bad8dbb2daa75d23d4ad163546
2021-01-25 08:24:13 +00:00
Antoine Leblanc
591db631ea ci: run stylish haskell on all incoming PRs
### Description

This PR adds two new github actions: one in the OSS repo, one in the monorepo. The OSS action runs `stylish-haskell` on all files touched by the PR, and displays a warning for each file that was in need of a formatting. The monorepo action does the same thing, with a twist: if the branch is not a shadow copy of an OSS branch, we assume that it is a local branch, and simply push a new commit with the changes.

Furthermore, this PR upgrades our stylish-haskell config to add record formatting, as close as possible to our styleguide.

Both actions use the standard stylish-haskell, not our modified fork.

### Known limitation

The monorepo action does not handle forks: pushing to the branch will fail, and checking the branch out might fail too. This is probably acceptable since we don't use forks with the monorepo, but it wouldn't be hard to handle that gracefully.

GitOrigin-RevId: 814138c5b9826098e2e4ea192778fc0d93fbe390
2021-01-21 23:37:26 +00:00
Karthikeyan Chinnakonda
3020150274 server: allow mapping session variables to standard JWT claims
fixes https://github.com/hasura/graphql-engine/issues/6449

A while back we added [support for customizing JWT claims](https://github.com/hasura/graphql-engine/pull/3575) and this enabled to map a session variable to any value within the unregistered claims, but as reported in #6449 , users aren't able to map the `x-hasura-user-id` session variable to the `sub` standard JWT claim.

This PR fixes the above issue by allowing mapping session variables to standard JWT claims as well.

GitOrigin-RevId: d3e63d7580adac55eb212e0a1ecf7c33f5b3ac4b
2021-01-21 16:50:46 +00:00
Antoine Leblanc
4815fcd500 server: progress on generic metadata
This PR generalizes a bunch of metadata structures.

Most importantly, it changes `SourceCache` to hold existentially quantified values:
```
data BackendSourceInfo =
  forall b. Backend b => BackendSourceInfo (SourceInfo b)

type SourceCache = HashMap SourceName BackendSourceInfo
```

This changes a *lot* of things throughout the code. For now, all code using the schema cache explicitly casts sources to Postgres, meaning that if any non-Postgres `SourceInfo` makes it to the cache, it'll be ignored.

That means that after this PR is submitted, we can split work between two different aspects:
  - creating `SourceInfo` for other backends
  - handling those other sources down the line

GitOrigin-RevId: fb9ea00f32e840fc33c5467896fb1dfa5283ab42
2021-01-20 00:32:45 +00:00
hasura-bot
98ccd81704 Server: Remote relationships permissions
GITHUB_PR_NUMBER: 6125
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/6125

Co-authored-by: Karthikeyan Chinnakonda <15602904+codingkarthik@users.noreply.github.com>
GitOrigin-RevId: 53d0671e6335dad1af7cb00e3e05e7021a910673
2021-01-19 20:57:58 +00:00
hasura-bot
2c56254e5a server: simplify JSON instances
GITHUB_PR_NUMBER: 6152
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/6152

Co-authored-by: Antoine Leblanc <1618949+nicuveo@users.noreply.github.com>
GitOrigin-RevId: 6c94aef8c57e852b3d41b8355c09e64fce756a7c
2021-01-19 19:15:42 +00:00
hasura-bot
38fc4cb677 server: consistently log request_id at the same level
GITHUB_PR_NUMBER: 6244
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/6244

Co-authored-by: José Lorenzo Rodríguez <37621+lorenzo@users.noreply.github.com>
GitOrigin-RevId: fef22d98ac7be23ca21a63dc1c696fa7169253a4
2021-01-19 17:26:45 +00:00
Antoine Leblanc
e754190301 server: clean MaybeT usage, and introduce new hlint rules
### Description

Our Prelude provides the very convenient `hoistMaybe :: Maybe b -> MaybeT m b`. This PR adds hlint rules to replace uses of `MaybeT $ pure $ x` with the cleaner `hoistMaybe x`, and rules to specifically replace `MaybeT $ pure Nothing` with `empty`.

GitOrigin-RevId: 7254f4954e34e4d7ca972dc7c12073d3ab8cb0b8
2021-01-19 13:38:42 +00:00
Vladimir Ciobanu
6e752a7876 server: add type information to aggregates and stringify them (closes #5704)
Fixes https://github.com/hasura/graphql-engine/issues/5704 by checking, for aggregate fields whether we are handling a numeric aggregation.

This PR also adds type information to `ColFld` such that we know the type of the field.

This is the second attempt. See #319 for a less invasive approach. @nicuveo suggested type information might be useful, and since it wasn't hard to add, I think this version is better as well.

GitOrigin-RevId: aa6a259fd5debe9466df6302839ddbbd0ea659b5
2021-01-18 13:52:51 +00:00
Karthikeyan Chinnakonda
6c22132061 server: move remote schema permissions APIs to /v1/metadata
Earlier (pre catalog separation), the remote schema permissions were in `/v1/query`. This PR moves it to `/v1/metadata`.

GitOrigin-RevId: cb39d9df4cc2288f67231504e3a7909f2f8df4da
2021-01-18 10:39:29 +00:00
hasura-bot
513a3d0c19 Fix action relationship type and input arguments (closes #6402) (#284)
Co-authored-by: Antoine Leblanc <antoine@hasura.io>
GITHUB_PR_NUMBER: 6417
GITHUB_PR_URL: https://github.com/hasura/graphql-engine/pull/6417
GitOrigin-RevId: 37b67a4d04e0ed3b16fc5fc9bf025b24b1f1bf6e
2021-01-18 06:57:24 +00:00
Rakesh Emmadi
648488ce7a server: fix computed field comment is not being saved in metadata (#337)
GitOrigin-RevId: 7eb1c5b86740d884a2718cead6dab962a45283d1
2021-01-14 07:03:42 +00:00
Karthikeyan Chinnakonda
c14bcb6967 server: accept new config allowed_skew in JWT config to provide leeway in JWT expiry
fixes https://github.com/hasura/graphql-engine/issues/2109

This PR accepts a new config `allowed_skew` in the JWT config to provide for some leeway while comparing the JWT expiry time.

GitOrigin-RevId: ef50cf77d8e2780478685096ed13794b5c4c9de4
2021-01-13 08:39:18 +00:00
Lyndon Maydwell
ece4fb4bce Merge pull request #342 from hasura/naveen/move-serverCtx-to-Config
server: Move 'serverMetrics' to Config.hs for better visibility
GitOrigin-RevId: 4f2b40ec81d4d4bb662cc16693b993fb3606b64a
2021-01-13 06:28:12 +00:00