Commit Graph

2649 Commits

Author SHA1 Message Date
Philip Lykke Carlsen
1af8d53c6f refactor(hlint): Demote Environment-from-OS to a suggestion
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8092
GitOrigin-RevId: 042aec936638fcb7890ecaa8cb368ed442100904
2023-02-23 21:44:44 +00:00
Puru Gupta
f45928b03b server: centralize various application state (introducing AppContext and AppEnv)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7905
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Naveen Naidu <30195193+Naveenaidu@users.noreply.github.com>
Co-authored-by: Anon Ray <616387+ecthiender@users.noreply.github.com>
GitOrigin-RevId: 74ce763b266dc053c10888767d5b4a0d9692508a
2023-02-23 14:45:24 +00:00
Daniel Chambers
6f50261274 Additional tests for mutations in the Data Connector agent test suite
[GDC-719]: https://hasurahq.atlassian.net/browse/GDC-719?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8080
GitOrigin-RevId: 2e35dd72723e0e9fc63426dc90b1a89aedbe4774
2023-02-23 13:54:37 +00:00
ananya-2410
36db970a34 ci: tag release v2.20.0-beta.1
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8081
GitOrigin-RevId: 0e13f833e881a5acf648f43ce16358a13945f5c3
2023-02-23 06:52:12 +00:00
Daniel Chambers
190e429eef Add tests for delete mutations to the Data Connector agent test suite
[GDC-721]: https://hasurahq.atlassian.net/browse/GDC-721?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8067
GitOrigin-RevId: bf5b7e7c20c6011d438888be6017331def7063bf
2023-02-22 23:41:47 +00:00
Antoine Leblanc
6e574f1bbe harmonize network manager handling
## Description

### I want to speak to the `Manager`

Oh boy. This PR is both fairly straightforward and overreaching, so let's break it down.

For most network access, we need a [`HTTP.Manager`](https://hackage.haskell.org/package/http-client-0.1.0.0/docs/Network-HTTP-Client-Manager.html). It is created only once, at the top level, when starting the engine, and is then threaded through the application to wherever we need to make a network call. As of main, the way we do this is not standardized: most of the GraphQL execution code passes it "manually" as a function argument throughout the code. We also have a custom monad constraint, `HasHttpManagerM`, that describes a monad's ability to provide a manager. And, finally, several parts of the code store the manager in some kind of argument structure, such as `RunT`'s `RunCtx`.

This PR's first goal is to harmonize all of this: we always create the manager at the root, and we already have it when we do our very first `runReaderT`. Wouldn't it make sense for the rest of the code to not manually pass it anywhere, to not store it anywhere, but to always rely on the current monad providing it? This is, in short, what this PR does: it implements a constraint on the base monads, so that they provide the manager, and removes most explicit passing from the code.

### First come, first served

One way this PR goes a tiny bit further than "just" doing the aforementioned harmonization is that it starts the process of implementing the "Services oriented architecture" roughly outlined in this [draft document](https://docs.google.com/document/d/1FAigqrST0juU1WcT4HIxJxe1iEBwTuBZodTaeUvsKqQ/edit?usp=sharing). Instead of using the existing `HasHTTPManagerM`, this PR revamps it into the `ProvidesNetwork` service.

The idea is, again, that we should make all "external" dependencies of the engine, all things that the core of the engine doesn't care about, a "service". This allows us to define clear APIs for features, to choose different implementations based on which version of the engine we're running, harmonizes our many scattered monadic constraints... Which is why this service is called "Network": we can refine it, moving forward, to be the constraint that defines how all network communication is to operate, instead of relying on disparate classes constraint or hardcoded decisions. A comment in the code clarifies this intent.

### Side-effects? In my Haskell?

This PR also unavoidably touches some other aspects of the codebase. One such example: it introduces `Hasura.App.AppContext`, named after `HasuraPro.Context.AppContext`: a name for the reader structure at the base level. It also transforms `Handler` from a type alias to a newtype, as `Handler` is where we actually enforce HTTP limits; but without `Handler` being a distinct type, any code path could simply do a `runExceptT $ runReader` and forget to enforce them.

(As a rule of thumb, i am starting to consider any straggling `runReaderT` or `runExceptT` as a code smell: we should not stack / unstack monads haphazardly, and every layer should be an opaque `newtype` with a corresponding run function.)

## Further work

In several places, i have left TODOs when i have encountered things that suggest that we should do further unrelated cleanups. I'll write down the follow-up steps, either in the aforementioned document or on slack. But, in short, at a glance, in approximate order, we could:

- delete `ExecutionCtx` as it is only a subset of `ServerCtx`, and remove one more `runReaderT` call
- delete `ServerConfigCtx` as it is only a subset of `ServerCtx`, and remove it from `RunCtx`
- remove `ServerCtx` from `HandlerCtx`, and make it part of `AppContext`, or even make it the `AppContext` altogether (since, at least for the OSS version, `AppContext` is there again only a subset)
- remove `CacheBuildParams` and `CacheBuild` altogether, as they're just a distinct stack that is a `ReaderT` on top of `IO` that contains, you guessed it, the same thing as `ServerCtx`
- move `RunT` out of `RQL.Types` and rename it, since after the previous cleanups **it only contains `UserInfo`**; it could be bundled with the authentication service, made a small implementation detail in `Hasura.Server.Auth`
-  rename `PGMetadaStorageT` to something a bit more accurate, such as `App`, and enforce its IO base

This would significantly simply our complex stack. From there, or in parallel, we can start moving existing dependencies as Services. For the purpose of supporting read replicas entitlement, we could move `MonadResolveSource` to a `SourceResolver` service, as attempted in #7653, and transform `UserAuthenticationM` into a `Authentication` service.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7736
GitOrigin-RevId: 68cce710eb9e7d752bda1ba0c49541d24df8209f
2023-02-22 15:55:54 +00:00
Gil Mizrahi
625e41cd77 rename naqi to logimo part 3 - data types
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8068
GitOrigin-RevId: 435527a98e645ed69c9be484ff0bd21af8181d69
2023-02-22 13:46:54 +00:00
Gil Mizrahi
2b0e9ea14c rename NativeQuery module hierarchy to LogicalModel
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8063
GitOrigin-RevId: 0ed0cee7fb1d77f166770caae21cc05d5dd30b75
2023-02-22 09:23:54 +00:00
Daniel Chambers
72e319e486 Add tests for update mutations to the Data Connector agent test suite
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8054
GitOrigin-RevId: 6869e8938e7a70128770f959bf34e3a291074d14
2023-02-22 04:19:16 +00:00
Gil Mizrahi
b761add3c4 rename naqi to logimo part 1
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8037
GitOrigin-RevId: 4839410ba836d4b69fd1ee30875bca9b46decc66
2023-02-21 13:46:44 +00:00
Krushan Bauva
2b1d5be10a server: catch TimeoutThread and InvalidRequest exceptions in websocket connections
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8014
GitOrigin-RevId: eef2d1ce99ec87bb239584c3ca4a9f056131b5cc
2023-02-21 11:55:54 +00:00
Rakesh Emmadi
39f3e90be8 server: accept "strict" param in healthz API
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8009
Co-authored-by: Sean Park-Ross <94021366+seanparkross@users.noreply.github.com>
GitOrigin-RevId: f535a84dc8aa35f60aaeab15ec5b54ba6f80ef6b
2023-02-21 09:13:44 +00:00
Jesse Hallett
37100138b6 server: codecs for rest endpoint definitions in metadata
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8030
GitOrigin-RevId: 0980502c535ab308e2458a98cebfd166fa55f8b6
2023-02-20 18:37:55 +00:00
Philip Lykke Carlsen
e979a39f6d chore: Fix all outstanding hlint hints
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8042
GitOrigin-RevId: 87c718fa7b09f375ea0e7c2465788ac49f575290
2023-02-20 17:43:28 +00:00
Jesse Hallett
99a982d4dd server: codecs for the metadata Action type
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7898
GitOrigin-RevId: 4211854f656ab02bb058b2d0831637477b3d4767
2023-02-20 17:10:19 +00:00
paritosh-08
17c3d25b1b server: warning in replace_metadata API
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7575
GitOrigin-RevId: 0171514056f725cb7a86dc7856eb57c674937c83
2023-02-20 14:20:38 +00:00
Philip Lykke Carlsen
384bb1b9d4 refactor(server): Add HLint rules discouraging getEnvironment
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7959
GitOrigin-RevId: c3189553a7f419a46510463dad7985f2953ef3c3
2023-02-20 13:45:24 +00:00
Daniel Harvey
2720f0a25a chore(server): store Native Queries as ordered map
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8027
GitOrigin-RevId: bfcf4001d82970e25e0a85e6cd720b2b3a52a5be
2023-02-20 11:48:16 +00:00
Jesse Hallett
8640346b4f server: codecs for cron triggers
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8029
GitOrigin-RevId: c26308782618c760921a0eff3e4bd87702bd3eb2
2023-02-20 02:52:00 +00:00
Solomon
316c2cc4e1 Remove suffix from source kind display name
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7788
GitOrigin-RevId: 17eea4c8de746daab120d8bd1e7fce43ac6618bf
2023-02-17 23:18:54 +00:00
Gil Mizrahi
0054a6feca add basic naqi telemetry
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8026
GitOrigin-RevId: d82fa3ce10a40b9f8ead0e40400295a17eb66d25
2023-02-17 13:58:48 +00:00
Gil Mizrahi
f57786a22d remove .ToSource.hs.swp
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8025
GitOrigin-RevId: d041fa09cf2faacb12a4b5e3bdd238fdde8d1957
2023-02-17 10:19:38 +00:00
Solomon
fd0ba0013d Adds Agent availability check to dc_add_agent metadata action
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7792
GitOrigin-RevId: 5ba1669b27078a45a4efe2ba736908a4929a2a50
2023-02-17 01:30:51 +00:00
Philip Lykke Carlsen
ecd0e7c3f4 fix, Sql Server: Properly quote delimited identifiers
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7929
Co-authored-by: Gil Mizrahi <8547573+soupi@users.noreply.github.com>
GitOrigin-RevId: 4ed630dcf6851a92980571c8041273512c69c622
2023-02-16 11:20:11 +00:00
Daniel Chambers
ac475fa02e Added weird table edge-case insert mutation tests to agent test suite
[GDC-719]: https://hasurahq.atlassian.net/browse/GDC-719?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7964
GitOrigin-RevId: d3077aafd29f3b5553e00bf6bba4c1ff9966485d
2023-02-16 07:51:36 +00:00
Solomon
b3553ac88d GDC Mysql integration tests
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7603
Co-authored-by: Daniel Chambers <1214352+daniel-chambers@users.noreply.github.com>
GitOrigin-RevId: 651bc2fb59a46449e2a4325a4c7af57592e53845
2023-02-16 06:30:51 +00:00
Jesse Hallett
b235e94e28 ci: fix an error in a recently-added buildkite test script
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8001
GitOrigin-RevId: 12f7d66772eb6d9b011b7a77a85544ffeb6ec270
2023-02-15 21:07:52 +00:00
Daniel Harvey
3b42e704dd [server] Custom types prototype
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7807
Co-authored-by: Tom Harding <6302310+i-am-tom@users.noreply.github.com>
GitOrigin-RevId: ee3c644b92aa71a236d247a0cfc5deb7846f91c2
2023-02-15 17:56:58 +00:00
Tom Harding
565176c155 Remove NativeQueryMetadata
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7980
GitOrigin-RevId: caadc019cc1f7824a65de10b20415ff736b1aafb
2023-02-15 16:27:47 +00:00
Vishnu Bharathi
3eb1be0acb ci: tag release v2.19.0
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7996
GitOrigin-RevId: 45b08009c4e669c2b20c473bb7bdf896160b6e10
2023-02-15 13:11:28 +00:00
Varun Choudhary
ea76e01b0f console: support suffix for tls allow list
[GS-392]: https://hasurahq.atlassian.net/browse/GS-392?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7745
Co-authored-by: Krushan Bauva <31391329+krushanbauva@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
GitOrigin-RevId: 72c9a2eef1cfaaefa30a7d53764b1c3688b38ae7
2023-02-15 10:26:41 +00:00
Gil Mizrahi
7a3c1489ad set omit_request_id to true by default
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7978
GitOrigin-RevId: 7e359979b304ff63b8e582e460c7668f8207fe75
2023-02-15 09:53:19 +00:00
awjchen
2df8589097 server: fix race condition in metadata update
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7984
GitOrigin-RevId: edd4a581a5b785e7a09b2d4c8b2a5c6ccc7f8b63
2023-02-15 06:12:59 +00:00
Vijay Prasanna
9900027a8e feature (console): remove the GDC experimental feature flag
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7942
Co-authored-by: Nicolas Beaussart <7281023+beaussan@users.noreply.github.com>
Co-authored-by: Julian <843342+okjulian@users.noreply.github.com>
Co-authored-by: Brandon Martin <40686+codedmart@users.noreply.github.com>
GitOrigin-RevId: 643c02d36f71b3e25060ddbb723bf57f31dd4318
2023-02-15 04:52:54 +00:00
Gil Mizrahi
20e72ddb29 reject untrack naqi when naqi does not exist
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7975
GitOrigin-RevId: e56385e7396c3980d78b53acc6e9b3656fd9909d
2023-02-14 14:40:15 +00:00
Philip Lykke Carlsen
fed36dadc7 Fix: Unable to use environment variable for CockroachDB connection string in Cloud
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7883
GitOrigin-RevId: b1f515c4de3b5f345991a0fad1152f42169f9d0d
2023-02-14 12:15:58 +00:00
Lyndon Maydwell
630bf277c1 SQLite Mutations
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7573
Co-authored-by: Daniel Chambers <1214352+daniel-chambers@users.noreply.github.com>
GitOrigin-RevId: 7807368faaa4bac5c0726c2dab041f8180a3fc31
2023-02-14 11:20:36 +00:00
Samir Talwar
2df5b80063 server/tests-py: Run in parallel on all PostgreSQL versions in CI.
If this works, we can start knocking out running in the old, sequential mode.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7909
GitOrigin-RevId: 4cda5b52b3c385dd3286ba1f58d5cd7c62b87314
2023-02-13 23:39:28 +00:00
Auke Booij
334660d906 server: refactor FromJSON FromIntrospection instances
- Inline a few instances to avoid code duplication
- Use `(<$>)` to avoid `let`
- Improve error reporting when types of invalid kind are specified in `possibleTypes` or `interfaces`.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7540
GitOrigin-RevId: 954fb710f94a275daff938b9a6e58765c4286d0c
2023-02-13 07:50:12 +00:00
Karthikeyan Chinnakonda
958a5e1a33 Include cron trigger's name in the error message
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7914
GitOrigin-RevId: 7bce2788ac154e7673402022414ab1fff653340d
2023-02-13 07:11:57 +00:00
Daniel Chambers
2ced1f2676 More tests for insert mutations for Data Connector agents
[GDC-719]: https://hasurahq.atlassian.net/browse/GDC-719?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7901
GitOrigin-RevId: b7f28c4b9604c0ca78c528deb2923e8519fae56a
2023-02-13 01:03:18 +00:00
Lyndon Maydwell
5a3f10363d Fixing omit_tracked argument to suggest_relationships API to be more granular
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7900
GitOrigin-RevId: 2116875b866ffc79ce5e3604751c62b8ac9c37e3
2023-02-10 01:53:30 +00:00
Antoine Leblanc
dec8579db8 Allow backend execution to happen on the base app monad.
### Description

Each Backend executes queries against the database in a slightly different stack: Postgres uses its own `TXeT`, MSSQL uses a variant of it, BigQuery is simply in `ExceptT QErr IO`... To accommodate those variations, we had originally introduced an `ExecutionMonad b` type family in `BackendExecute`, allowing each backend to describe its own stack. It was then up to that backend's `BackendTransport` instance to implement running said stack, and converting the result back into our main app monad.

However, this was not without complications: `TraceT` is one of them: as it usually needs to be on the top of the stack, converting from one stack to the other implies the use `interpTraceT`, which is quite monstrous. Furthermore, as part of the Entitlement Services work, we're trying to move to a "Services" architecture in which the entire engine runs in one base monad, that delegates features and dependencies to monad constraints; and as a result we'd like to minimize the number of different monad stacks we have to maintain and translate from and to in the codebase.

To improve things, this PR changes `ExecutionMonad b` from an _absolute_ stack to a _relative_ one: i.e.: what needs to be stacked on top of our base monad for the execution. In `Transport`, we then only need to pop the top of the stack, and voila. This greatly simplifies the implementation of the backends, as there's no longer any need to do any stack transformation: MySQL's implementation becomes a `runIdentityT`! This also removes most mentions of `TraceT` from the execution code since it's no longer required: we can rely on the base monad's existing `MonadTrace` constraint.

To continue encapsulating monadic actions in `DBStepInfo` and avoid threading a bunch of `forall` all over the place, this PR introduces a small local helper: `OnBaseMonad`. One only downside of all this is that this requires adding `MonadBaseControl IO m` constraint all over the place: previously, we would run directly on `IO` and lift, and would therefore not need to bring that constraint all the way.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7789
GitOrigin-RevId: e9b2e431c5c47fa9851abf87545c0415ff6d1a12
2023-02-09 14:40:04 +00:00
ananya-2410
c681771192 ci: tag release v2.19.0-beta.1
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7911
GitOrigin-RevId: 154a508058b5f3dc8c4901fb6a22ab00009adc19
2023-02-09 12:24:28 +00:00
Auke Booij
8e0e778cd5 server/tests: ensure that all responses have a Content-Type header
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7850
GitOrigin-RevId: 5e3ec3907177bd35974d5dd9a990dca399904139
2023-02-09 10:28:19 +00:00
kodiakhq[bot]
b01d3a45de server: dev build modes
Add some configurations for modern profiling modes, and integration into dev.sh

These require cabal 3.8 due to the use of `import`

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7671
GitOrigin-RevId: f793f64105cfd99fb51b247fa8bc050f6d4bd23e
2023-02-08 22:43:00 +00:00
Tom Harding
89bbb24c54 Monomorphise NativeQueryName
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7865
GitOrigin-RevId: 6c26edeb40f3b523053ffc45220a3c1046585de3
2023-02-08 15:48:12 +00:00
Samir Talwar
23f78a05b1 server/tests-py: Enable TLS tests in parallel mode on CI.
Part of [NDAT-257](https://hasurahq.atlassian.net/browse/NDAT-257).

* Run the TLS tests on CI.
* Run non-TLS tests even when a TLS certificate is provided; just ignore it.

[NDAT-257]: https://hasurahq.atlassian.net/browse/NDAT-257?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7806
GitOrigin-RevId: d705e2e5131a674070a98a893323a7b98a92f688
2023-02-08 14:42:27 +00:00
Samir Talwar
fa53a70631 server/tests-py: Run TestSubscriptionBasicNoAuth in the new mode.
This test _cannot_ run without an admin secret, so we need a new marker just for that.

This also fixes the Docker Compose file so that _run.sh_ works again. Apparently it was not used much.

[NDAT-257]: https://hasurahq.atlassian.net/browse/NDAT-257?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7831
GitOrigin-RevId: 1cb9449a20c7cbe56fbd1ba0f12337f056683624
2023-02-08 11:51:54 +00:00
Daniel Chambers
0d5badba09 Initial tests for insert mutations for Data Connector agents
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7875
GitOrigin-RevId: 793501685124e41edfd83c6be4a2a9c7ae7f9267
2023-02-08 08:36:58 +00:00