Commit Graph

225 Commits

Author SHA1 Message Date
Samir Talwar
397e95fc6a Upgrade Citus to v11.3.0 in tests.
Upgrades Citus to v11.3.0 in tests.

This breaks an assumption made by the tests for the `get_source_tables` metadata API, in which data is expected to be ordered. We fix it by explicitly ordering rather than relying on the goodwill of the database.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9039
GitOrigin-RevId: ee86db7e1c264d5009bb0203ac2f3fb2cda7b39f
2023-05-04 11:40:10 +00:00
Gil Mizrahi
e010781098 chore(refactor): rename NativeQueryArgumentName to ArgumentName
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9012
GitOrigin-RevId: fa45f7a01f4fa1f3f1bb6c3057f32839bab453ca
2023-05-02 12:55:57 +00:00
Tom Harding
b6799f0882 Import InsOrdHashMap, not OMap, OM, Map, HM, ...
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8946
GitOrigin-RevId: 434e7c335bc69119020dd35761c7d4539bc51ff8
2023-04-27 07:43:22 +00:00
kodiakhq[bot]
bb8f6a76cc PLAT-438: improve traces from OpenTelemetry
[PLAT-438](https://hasurahq.atlassian.net/browse/PLAT-438)

After this change the top level trace is is annotated with `request_id` and `parameterized_query_hash`. It looks like this,  in jeager:

![image](https://user-images.githubusercontent.com/210815/233206107-73682a85-3306-4d1d-8a6c-148890dc1a38.png)

 bulk queries should output a separate `parameterized_query_hash` for each query span

...after the last two commits, things look like this (remove pointless spans, more attributes):

![image](https://user-images.githubusercontent.com/210815/233476532-47521f35-2cdf-4f7d-af13-39c7ce15c9ec.png)

[PLAT-438]: https://hasurahq.atlassian.net/browse/PLAT-438?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8824
GitOrigin-RevId: 0fd105c879161587d41b729b9bac968e92efae95
2023-04-26 16:21:29 +00:00
Tom Harding
7e334e08a4 Import HashMap, not HM, Map, M...
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8947
GitOrigin-RevId: 18e52c928e1df535579e2077b4af6c2ce92bdcef
2023-04-26 15:43:44 +00:00
Daniel Chambers
cb80942e4f Disable use of local relationships in metadata if Data Connector agents don't declare relationships capability
[GDC-489]: https://hasurahq.atlassian.net/browse/GDC-489?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8862
GitOrigin-RevId: cfff2d9c688185e00f4a830dbdbe173dba4ce7f0
2023-04-26 06:49:36 +00:00
Daniel Harvey
ea5c92acae chore(server): move Hasura.SQL.Backend to Hasura.RQL.Types.BackendType
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8876
GitOrigin-RevId: abfc18eeef96a1f3593bfe823adab4d161161333
2023-04-24 18:37:33 +00:00
Tom Harding
f8ae944dbc Move Hasura.GraphQL.Schema.Options to Hasura.RQL.Types.Options
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8877
GitOrigin-RevId: 8be82f60a57cd9582d6980a6dea2f34c7b0c13c1
2023-04-24 15:18:56 +00:00
Tom Harding
f80369573c Move Hasura.SQL.Tag into RQL.Types
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8871
Co-authored-by: Daniel Harvey <4729125+danieljharvey@users.noreply.github.com>
GitOrigin-RevId: f69ff31018d6a68a3221d75cc1e2ef467bc46e12
2023-04-24 12:27:27 +00:00
Philip Lykke Carlsen
983fc2ad47 Rename "Custom Return Types" → "Logical Models"
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8800
GitOrigin-RevId: e5e10f31c6cc8953a8ee947441a7f80b0e9b5e5e
2023-04-19 09:05:25 +00:00
Gil Mizrahi
5ed9fcc0a3 bugfix(NativeQueries): fix validation failed on name with uppercase on postgres
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8790
GitOrigin-RevId: f3b9b041fd96c6087ff4bd3379ef6283befcdc54
2023-04-18 16:09:45 +00:00
Daniel Harvey
eab64a8ca3 chore(server): replace Metadata.hs-boot with 3 orphan instances
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8740
GitOrigin-RevId: 801f03045b9981b07f8c0abfdeb2722e715e63cd
2023-04-17 12:11:21 +00:00
Daniel Harvey
2fd3f91398 chore(server): metadata changes for array relationships for Native Queries
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8670
GitOrigin-RevId: c23e23e3cf48013ab76fc2fa98c8b8b800c6cee6
2023-04-17 11:31:59 +00:00
Philip Lykke Carlsen
0346224444 Rename "Logical Models" → "Native Queries"
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8769
GitOrigin-RevId: 66f2cbfb620d641e672a4074554d9d324a18c591
2023-04-13 16:12:20 +00:00
Daniel Chambers
56db8ec358 Data Connectors: Fix track_table not working if the table was created after schema cache was built
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8732
GitOrigin-RevId: e11a0ef6979d1d58a0b39dcd8fff48d446d3420f
2023-04-13 01:30:50 +00:00
Tom Harding
af0b56332d De-duplicate prepared logical model arguments
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8723
GitOrigin-RevId: 5df1a05c47852d7371ef323bb0df2ac9d1749243
2023-04-12 09:05:48 +00:00
Tom Harding
3cef692dd7 feature(server): Get _all_ source tables
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8649
GitOrigin-RevId: 4bb9311d1d7ab4e8ee641bde5df2babcb1b8b306
2023-04-05 20:16:14 +00:00
Rishichandra Wawhal
c6d65508b2 [feature branch] EE Lite Trials
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8208
Co-authored-by: awjchen <13142944+awjchen@users.noreply.github.com>
Co-authored-by: Vijay Prasanna <11921040+vijayprasanna13@users.noreply.github.com>
Co-authored-by: Toan Nguyen  <1615675+hgiasac@users.noreply.github.com>
Co-authored-by: Abhijeet Khangarot <26903230+abhi40308@users.noreply.github.com>
Co-authored-by: Solomon <24038+solomon-b@users.noreply.github.com>
Co-authored-by: gneeri <10553562+gneeri@users.noreply.github.com>
GitOrigin-RevId: 454ee0dea636da77e43810edb2f427137027956c
2023-04-05 08:59:09 +00:00
Gil Mizrahi
352d8ff09c validate logical models custom return type against postgres
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8563
GitOrigin-RevId: 462a608e0e90d1923bc1d735257f4506825f5db1
2023-04-04 14:02:59 +00:00
Daniel Harvey
9c99bcb6f8 chore(server): move perms to custom return types
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8604
GitOrigin-RevId: ff024429b3f06e4867334665f35d4bd404a85cba
2023-04-04 12:46:51 +00:00
Daniel Harvey
cb9f822fc0 chore(server): move user-defined-functions into Hasura.Function.*
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8595
GitOrigin-RevId: b3b7dec8504fe4daf324125e4017fa7d1945b3ca
2023-04-03 10:20:20 +00:00
Tom Harding
7e06b30e4d Add metadata commands for custom return type permissions
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8587
Co-authored-by: Daniel Harvey <4729125+danieljharvey@users.noreply.github.com>
GitOrigin-RevId: 660f2eda9cf1c7c612d66745064b3998c77804e0
2023-03-31 16:29:24 +00:00
Daniel Harvey
79682e0598 chore(server): move custom types out of logical models
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8565
Co-authored-by: Tom Harding <6302310+i-am-tom@users.noreply.github.com>
GitOrigin-RevId: 38bf56cc420a6c818a9ca7d6f846f5018535c808
2023-03-31 15:35:13 +00:00
Daniel Harvey
7227e96278 feature(server): custom return types as discreet metadata entity
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8556
Co-authored-by: Tom Harding <6302310+i-am-tom@users.noreply.github.com>
GitOrigin-RevId: b7dcbcf378279c3bf4c8d223174b90c2cb4b9e53
2023-03-30 15:15:11 +00:00
Daniel Harvey
305eebb3ab feature(server): add Logical Models support for CockroachDB
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8500
GitOrigin-RevId: 0205bd207843f50ad4fa12ad6e6c07d1f2740912
2023-03-27 14:00:07 +00:00
Daniel Harvey
7e437fc32b feature(server): ordered columns in Logical Model return type
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8434
Co-authored-by: Philip Lykke Carlsen <358550+plcplc@users.noreply.github.com>
GitOrigin-RevId: 4ebd569bef868e01b15583e4af90d583d6713da7
2023-03-22 16:35:56 +00:00
awjchen
4f6240fb04 server: refactor ConnectionTemplateResolver
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8381
GitOrigin-RevId: 5286c384bb8c739170c90960a9b4a324a629c43b
2023-03-20 21:56:20 +00:00
Daniel Harvey
8d92c48c3c chore(server): logical models in Citus
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8356
GitOrigin-RevId: 98cb2dcaebe45b98d506279bc7a363b8d2696cd8
2023-03-17 11:28:31 +00:00
Gil Mizrahi
021e769235 Logical models: check that specified named columns are returned from query
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8312
GitOrigin-RevId: fc7c7a14115c0b228abc2f48d3cfac51d2852277
2023-03-16 10:45:46 +00:00
Daniel Harvey
95f5553af6 chore(server): split new statistics log from QueryLog
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8326
GitOrigin-RevId: 02ee652302de5328e63054a6448dca10de7b5c1b
2023-03-15 13:06:47 +00:00
Rakesh Emmadi
9b6d403809 multitenant: enhancements to resize pools logs
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8190
GitOrigin-RevId: 34ffbd2b79033850c3652d58fbf6b119e94c92ca
2023-03-15 10:30:48 +00:00
Tom Harding
2124fa0f08 feature(server): make execution statistics available through logging
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8286
Co-authored-by: Daniel Harvey <4729125+danieljharvey@users.noreply.github.com>
GitOrigin-RevId: 72de592c08778649693d8ff0a0555b16fb28c4bd
2023-03-14 11:33:45 +00:00
Philip Lykke Carlsen
1bad75e3d8 feat(logical models): Add validation of types and parameters
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8267
GitOrigin-RevId: 827852edf074eb72c95ac379fe3ac5d095fd26ab
2023-03-13 18:15:27 +00:00
Antoine Leblanc
cf531b05cb Rewrite Tracing to allow for only one TraceT in the entire stack.
This PR is on top of #7789.

### Description

This PR entirely rewrites the API of the Tracing library, to make `interpTraceT` a thing of the past. Before this change, we ran traces by sticking a `TraceT` on top of whatever we were doing. This had several major drawbacks:
- we were carrying a bunch of `TraceT` across the codebase, and the entire codebase had to know about it
- we needed to carry a second class constraint around (`HasReporterM`) to be able to run all of those traces
- we kept having to do stack rewriting with `interpTraceT`, which went from inconvenient to horrible
- we had to declare several behavioral instances on `TraceT m`

This PR rewrite all of `Tracing` using a more conventional model: there is ONE `TraceT` at the bottom of the stack, and there is an associated class constraint `MonadTrace`: any part of the code that happens to satisfy `MonadTrace` is able to create new traces. We NEVER have to do stack rewriting, `interpTraceT` is gone, and `TraceT` and `Reporter` become  implementation details that 99% of the code is blissfully unaware of: code that needs to do tracing only needs to declare that the monad in which it operates implements `MonadTrace`.

In doing so, this PR revealed **several bugs in the codebase**: places where we were expecting to trace something, but due to the default instance of `HasReporterM IO` we would actually not do anything. This PR also splits the code of `Tracing` in more byte-sized modules, with the goal of potentially moving to `server/lib` down the line.

### Remaining work

This PR is a draft; what's left to do is:
- [x] make Pro compile; i haven't updated `HasuraPro/Main` yet
- [x] document Tracing by writing a note that explains how to use the library, and the meaning of "reporter", "trace" and "span", as well as the pitfalls
- [x] discuss some of the trade-offs in the implementation, which is why i'm opening this PR already despite it not fully building yet
- [x] it depends on #7789 being merged first

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7791
GitOrigin-RevId: cadd32d039134c93ddbf364599a2f4dd988adea8
2023-03-13 17:38:39 +00:00
Daniel Chambers
1727b5236a Block adding remote relationships where they are not supported by the Data Connector agent
[GDC-1015]: https://hasurahq.atlassian.net/browse/GDC-1015?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8232
GitOrigin-RevId: 0cc3b7b1f17b2e6d4cdfa713b1581357de62f359
2023-03-08 06:01:04 +00:00
Daniel Chambers
dc83352863 Support for using Data Connectors as the target of remote relationships
[GDC-487]: https://hasurahq.atlassian.net/browse/GDC-487?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
[GDC-488]: https://hasurahq.atlassian.net/browse/GDC-488?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8182
GitOrigin-RevId: 712953666e1a5ea07500f1e1aed669f27650f7a4
2023-03-07 01:33:26 +00:00
Daniel Harvey
ba5753e0cb feature(server): select permissions for Logical Models
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8049
Co-authored-by: Tom Harding <6302310+i-am-tom@users.noreply.github.com>
GitOrigin-RevId: 0e51ebfbf01e408f9a7a343edb5b3d9a7183140a
2023-03-06 16:40:48 +00:00
Tom Harding
1574125f10 Separate the metadata and cache representations of logical models
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8176
GitOrigin-RevId: a1ead98ea9d07b30ee09298e7f27a139d87711fa
2023-03-02 16:04:18 +00:00
Gil Mizrahi
60bd355a99 test: logimo validation prepared statement is deallocated
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8175
GitOrigin-RevId: 35ec7e69b616ebd7a4a30760b1698bd379e07d74
2023-03-02 15:30:35 +00:00
Gil Mizrahi
625e41cd77 rename naqi to logimo part 3 - data types
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8068
GitOrigin-RevId: 435527a98e645ed69c9be484ff0bd21af8181d69
2023-02-22 13:46:54 +00:00
Gil Mizrahi
2b0e9ea14c rename NativeQuery module hierarchy to LogicalModel
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8063
GitOrigin-RevId: 0ed0cee7fb1d77f166770caae21cc05d5dd30b75
2023-02-22 09:23:54 +00:00
Gil Mizrahi
b761add3c4 rename naqi to logimo part 1
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8037
GitOrigin-RevId: 4839410ba836d4b69fd1ee30875bca9b46decc66
2023-02-21 13:46:44 +00:00
Daniel Harvey
3b42e704dd [server] Custom types prototype
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7807
Co-authored-by: Tom Harding <6302310+i-am-tom@users.noreply.github.com>
GitOrigin-RevId: ee3c644b92aa71a236d247a0cfc5deb7846f91c2
2023-02-15 17:56:58 +00:00
Tom Harding
565176c155 Remove NativeQueryMetadata
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7980
GitOrigin-RevId: caadc019cc1f7824a65de10b20415ff736b1aafb
2023-02-15 16:27:47 +00:00
Philip Lykke Carlsen
fed36dadc7 Fix: Unable to use environment variable for CockroachDB connection string in Cloud
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7883
GitOrigin-RevId: b1f515c4de3b5f345991a0fad1152f42169f9d0d
2023-02-14 12:15:58 +00:00
Antoine Leblanc
dec8579db8 Allow backend execution to happen on the base app monad.
### Description

Each Backend executes queries against the database in a slightly different stack: Postgres uses its own `TXeT`, MSSQL uses a variant of it, BigQuery is simply in `ExceptT QErr IO`... To accommodate those variations, we had originally introduced an `ExecutionMonad b` type family in `BackendExecute`, allowing each backend to describe its own stack. It was then up to that backend's `BackendTransport` instance to implement running said stack, and converting the result back into our main app monad.

However, this was not without complications: `TraceT` is one of them: as it usually needs to be on the top of the stack, converting from one stack to the other implies the use `interpTraceT`, which is quite monstrous. Furthermore, as part of the Entitlement Services work, we're trying to move to a "Services" architecture in which the entire engine runs in one base monad, that delegates features and dependencies to monad constraints; and as a result we'd like to minimize the number of different monad stacks we have to maintain and translate from and to in the codebase.

To improve things, this PR changes `ExecutionMonad b` from an _absolute_ stack to a _relative_ one: i.e.: what needs to be stacked on top of our base monad for the execution. In `Transport`, we then only need to pop the top of the stack, and voila. This greatly simplifies the implementation of the backends, as there's no longer any need to do any stack transformation: MySQL's implementation becomes a `runIdentityT`! This also removes most mentions of `TraceT` from the execution code since it's no longer required: we can rely on the base monad's existing `MonadTrace` constraint.

To continue encapsulating monadic actions in `DBStepInfo` and avoid threading a bunch of `forall` all over the place, this PR introduces a small local helper: `OnBaseMonad`. One only downside of all this is that this requires adding `MonadBaseControl IO m` constraint all over the place: previously, we would run directly on `IO` and lift, and would therefore not need to bring that constraint all the way.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7789
GitOrigin-RevId: e9b2e431c5c47fa9851abf87545c0415ff6d1a12
2023-02-09 14:40:04 +00:00
Tom Harding
89bbb24c54 Monomorphise NativeQueryName
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7865
GitOrigin-RevId: 6c26edeb40f3b523053ffc45220a3c1046585de3
2023-02-08 15:48:12 +00:00
Gil Mizrahi
f90373b1ca NaQI - validate against the database - infra and check syntax
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7778
GitOrigin-RevId: c8366ad7261d9c120dc0612bedb96fa5ae59eda3
2023-02-03 16:29:22 +00:00
Daniel Harvey
d32e734310 [server] Pass parameters to native queries
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7727
Co-authored-by: Gil Mizrahi <8547573+soupi@users.noreply.github.com>
GitOrigin-RevId: ddca3d962c05024913ce6ec4c6b4ea4a0f4a6a8c
2023-02-03 11:16:29 +00:00
Daniel Harvey
4418d294f9 [server] parse native query in metadata call
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7731
Co-authored-by: Gil Mizrahi <8547573+soupi@users.noreply.github.com>
GitOrigin-RevId: 96d60c72da05970f5b34f310f9fe71d9f67387a1
2023-02-01 08:46:19 +00:00