Commit Graph

167 Commits

Author SHA1 Message Date
Samir Talwar
9c88a3a16f Only schedule cleanup for event triggers if any exist.
Users were seeing spurious log entries saying that cleanup could not be scheduled for BigQuery sources as event triggers are not supported.

This makes the error go away by enforcing non-emptyness on the list of triggers, which means we cannot call the function throwing the error, as there will never be any event triggers for which to schedule cleanup.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10987
GitOrigin-RevId: ce9bd6a340bbc8f45f4c74cf9c69f65c4ee91bf5
2024-08-07 13:56:58 +00:00
Samir Talwar
a8d1002175 server: Factor out authentication-related code into a new namespace, Hasura.Authentication.
In preparation for tightening up the various ways in which we construct and work with session variables, I am trying to move the behavior into the same module(s) as the data types, so that we can avoid exposing the internals of data structures in favor of smart constructors and conversions.

The session variable code was split between `Hasura.RQL.Types.Roles`, `Hasura.RQL.Types.Session`, and `Hasura.Session`, with the first two containing most of the data structures (and some logic) and the latter containing the rest of the logic. These files do not interact with the rest of `Hasura.RQL`, though they are depended upon by that namespace.

I have refactored these files into a new namespace, `Hasura.Authentication`. It now looks like this:

1. Role types are now in `Hasura.Authentication.Role`.
2. Header constants were moved from `Hasura.Server.Utils` to `Hasura.Authentication.Headers` (plural) to avoid cycles.
3. Header logic was moved from various places into `Hasura.Authentication.Header` (singular) for the same reason.
4. Session variable types and logic live together in `Hasura.Authentication.Session`.
5. User info types and logic live together in `Hasura.Authentication.User`.

This new structure is cycle-free and generally avoids importing the rest of the code, which means we should be able to start pruning the list of exports and locking down session variable construction.

No behavior was changed in this changeset.

The majority of changes are to the imports in a number of files; everything depends on these things. By splitting into multiple files, we also reduce the surface area of an individual import, which was a pleasant side-effect of this work.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10960
GitOrigin-RevId: 7cb962c06483cd9b92b80432aed5cabecb465cda
2024-07-29 06:02:56 +00:00
kodiakhq[bot]
dd3bbef2fe Jberryman/ghc 9.6.4
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10615
GitOrigin-RevId: 8a45c92bec8444d5ec0aed125f4aa96d9c684728
2024-01-23 19:47:53 +00:00
kodiakhq[bot]
07bad7c498 INFRA-832: logs OTLP export
https://hasurahq.atlassian.net/browse/INFRA-832

Foundational work already merged:
- #10171
- 0184ba8bfcae9b
- 699317ffd061d3

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10238
Co-authored-by: Varun Choudhary <68095256+Varun-Choudhary@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Toan Nguyen  <1615675+hgiasac@users.noreply.github.com>
GitOrigin-RevId: d88c6a1aafe74e7393873aacc61e6fce3bc7c068
2023-09-13 16:49:55 +00:00
Toan Nguyen
f915c7d1a2 server: support w3c traceparent context
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10218
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Rob Dominguez <24390149+robertjdominguez@users.noreply.github.com>
GitOrigin-RevId: d3dbea6220fd2127ab76c0a240fc4725ca5d6aac
2023-09-13 13:42:30 +00:00
pranshi06
75f0629c5d server: change log level to error for triggers in Cloud
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9873
GitOrigin-RevId: 703a16da479d35908a9a8c2862884d11a3135731
2023-08-10 10:04:05 +00:00
pranshi06
da582dda33 server: add log line for Event Trigger timeout errors
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9847
GitOrigin-RevId: 2cff5d3bca1e04b57cbe054e774e97e543cb7ce5
2023-07-13 09:53:27 +00:00
pranshi06
d32ae8ea05 server: update HTTP exception errors in log line for Triggers
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9738
GitOrigin-RevId: cece764a6fa087958675077c8420e9478d419e8e
2023-07-06 07:38:09 +00:00
Puru Gupta
f2fe9cfe3b server: add support for header resolution from env vars
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9509
GitOrigin-RevId: 818f747422c5444fcb55419729ad58d74b890d52
2023-06-23 08:39:28 +00:00
Tom Harding
e0c0043e76 Upgrade Ormolu to 0.7.0.0
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9284
GitOrigin-RevId: 2f2cf2ad01900a54e4bdb970205ac0ef313c7e00
2023-05-24 13:53:53 +00:00
Krushan Bauva
e3df24507d server: add dynamic labels trigger_name and source_name to existing event trigger metrics
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9265
GitOrigin-RevId: 6fb6504f1a476ea6c8b810e067770920757e8dc6
2023-05-24 13:23:43 +00:00
Daniel Harvey
8f4692d871 chore(server): move table related things to Hasura.Table.*
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9174
GitOrigin-RevId: d440647ac04b9c1717ecf22a2dbfb8c5f22b7c7a
2023-05-17 08:55:32 +00:00
Daniel Harvey
0fce0099b8 chore(server): use generics over TH in Hasura.Eventing
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9143
GitOrigin-RevId: ed11588db37f5336698fc33f50066e75fa7f73a1
2023-05-16 15:53:37 +00:00
pranshi06
0e6811d9df server: add HTTP response status in the Kriti context for Actions
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9125
GitOrigin-RevId: d0d35d1f9fa5ec04b42132007e2905087349fbd6
2023-05-15 16:35:22 +00:00
Krushan Bauva
ca0f007bc3 server: add new metric hasura_events_fetched_per_batch
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9121
Co-authored-by: Rob Dominguez <24390149+robertjdominguez@users.noreply.github.com>
GitOrigin-RevId: 3b68c1ec192c55f1fb765a8887271276e22f7bdd
2023-05-15 15:06:39 +00:00
pranshi06
2011f4cf37 server: time related modelling in Event Triggers for Postgres sources
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8846
GitOrigin-RevId: f9d99e3368f5722a8dbfc8aa037cd1f3971c4b98
2023-05-11 09:01:29 +00:00
paritosh-08
ae49a3f9db server: add the env var for toggling metric granularity
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9034
GitOrigin-RevId: 24a4ae932b7467a53060025e2772edbebaf94b5b
2023-05-11 05:21:16 +00:00
Daniel Harvey
285a200a87 chore(server): delete MySQL native backend
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9050
GitOrigin-RevId: 1515fb0efdb1baa05ffe3ff7cf6f230acd0cde29
2023-05-05 10:32:56 +00:00
Tom Harding
7e334e08a4 Import HashMap, not HM, Map, M...
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8947
GitOrigin-RevId: 18e52c928e1df535579e2077b4af6c2ce92bdcef
2023-04-26 15:43:44 +00:00
Tom Harding
74f282ad16 chore(server): remove Eventing instances from RQL.Types
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8909
GitOrigin-RevId: 302a9583c3364d507dfe106111ffcabac0b1dbf5
2023-04-25 17:18:30 +00:00
Daniel Harvey
ea5c92acae chore(server): move Hasura.SQL.Backend to Hasura.RQL.Types.BackendType
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8876
GitOrigin-RevId: abfc18eeef96a1f3593bfe823adab4d161161333
2023-04-24 18:37:33 +00:00
Daniel Harvey
8cf134dad1 Split Hasura.RQL.DDL.Headers to put types in Hasura.RQL.Types.Headers
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8873
GitOrigin-RevId: 566cb4271f0eb27e6688c2e0fbc26711bdf8baa9
2023-04-24 16:45:55 +00:00
Puru Gupta
44d9987e92 server: process event triggers with a timeout
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8823
GitOrigin-RevId: 4a38ef993fffe018ae16e232f2abb5d30c604855
2023-04-24 09:28:49 +00:00
Daniel Chambers
4ccfc3490b Satisfy get_table_info from cached DBObjectsIntrospection for Data Connectors
[GDC-643]: https://hasurahq.atlassian.net/browse/GDC-643?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8761
GitOrigin-RevId: 6b8e15fc35f7f463d86c865c880779794f0f28b8
2023-04-18 05:37:34 +00:00
Philip Lykke Carlsen
0346224444 Rename "Logical Models" → "Native Queries"
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8769
GitOrigin-RevId: 66f2cbfb620d641e672a4074554d9d324a18c591
2023-04-13 16:12:20 +00:00
Karthikeyan Chinnakonda
ddf34d9ffa Miscellaneous scheduled triggers fixes
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8618
GitOrigin-RevId: f9e3eb0d3d8b2ba36344b9ce0739c6a51e77f59e
2023-04-10 12:27:21 +00:00
Anon Ray
5a81eaa9b6 server: core changes for zero-downtime env vars update on cloud
[GS-232]: https://hasurahq.atlassian.net/browse/GS-232?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7207
Co-authored-by: pranshi06 <85474619+pranshi06@users.noreply.github.com>
Co-authored-by: Rakesh Emmadi <12475069+rakeshkky@users.noreply.github.com>
Co-authored-by: Puru Gupta <32328846+purugupta99@users.noreply.github.com>
Co-authored-by: Naveen Naidu <30195193+Naveenaidu@users.noreply.github.com>
GitOrigin-RevId: 90a771036da5275cd277f3daaf410381955c69de
2023-03-30 16:33:39 +00:00
Daniel Harvey
7227e96278 feature(server): custom return types as discreet metadata entity
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8556
Co-authored-by: Tom Harding <6302310+i-am-tom@users.noreply.github.com>
GitOrigin-RevId: b7dcbcf378279c3bf4c8d223174b90c2cb4b9e53
2023-03-30 15:15:11 +00:00
paritosh-08
e317c1a53f server: add new metrics for scheduled triggers
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8401
Co-authored-by: Rob Dominguez <24390149+robertjdominguez@users.noreply.github.com>
GitOrigin-RevId: 0cdec9e7b5e9251bf7c8b710b7552d065d62e195
2023-03-30 05:52:54 +00:00
paritosh-08
b7bae6dfec server: add new metric for event trigger observability
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8380
Co-authored-by: Rob Dominguez <24390149+robertjdominguez@users.noreply.github.com>
GitOrigin-RevId: df7d5c53668fc84e7d70d471e29774136f5d560a
2023-03-29 16:48:19 +00:00
Karthikeyan Chinnakonda
f5ffeeba52 Process scheduled events with a timeout
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8542
GitOrigin-RevId: f6c9300af3d038d17848ac0c001a6044dbafa464
2023-03-29 11:26:12 +00:00
Naveen Naidu
4e3dbed938 server: revert changes to created_at column of 'hdb_catalog.event_log'
This PR reverts the following two commits:
1. https://github.com/hasura/graphql-engine-mono/pull/8287
2. https://github.com/hasura/graphql-engine-mono/pull/8467

We are undoing a migration that was done on `hdb_catalog.event_log` table which was done in d4ae6a517da63f2f43567dc16fda135b3cd1d7e6 . And as such, users who were using event triggers on that version will come across the error:
```json
{"detail":{"info":{"code":"not-supported","error":"Expected source catalog version <= 3, but the current version is 4","path":"$"},"kind":"catalog_migrate"},"level":"error","timestamp":"2023-03-28T10:17:24.289+0530","type":"startup"}
{"code":"not-supported","error":"Expected source catalog version <= 3, but the current version is 4","path":"$"}
```
To fix these errors please run the following SQL on the source where event triggers were created on:
```
UPDATE hdb_catalog.hdb_source_catalog_version SET version = 3, upgraded_on= NOW();
ALTER table hdb_catalog.event_log ALTER COLUMN created_at SET DEFAULT NOW();
ALTER table hdb_catalog.event_invocation_logs ALTER COLUMN created_at SET DEFAULT NOW();
```

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8534
GitOrigin-RevId: b6bbcce0163c8beed80619d3cea056e643b8c180
2023-03-29 10:33:30 +00:00
pranshi06
f194c47156 server: fix value of created_at for postgres event triggers
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8287
Co-authored-by: Karthikeyan Chinnakonda <15602904+codingkarthik@users.noreply.github.com>
GitOrigin-RevId: d4ae6a517da63f2f43567dc16fda135b3cd1d7e6
2023-03-22 07:32:42 +00:00
Solomon
cca1a92399 Simplify Transformable Requests Module
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8400
GitOrigin-RevId: 10728012c3d74e178c34b926e13d3627d514ce17
2023-03-22 00:01:07 +00:00
pranshi06
ea691b3c06 server: improve logging around 'Internal Exception' errors
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8369
GitOrigin-RevId: 22160a2c8a17d571ceeda0bf3e9b672bea31b8f9
2023-03-21 12:00:00 +00:00
Puru Gupta
c437a42f6d server: rename SchemaCacheRef to AppStateRef and add AppContext to it
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8159
Co-authored-by: Naveen Naidu <30195193+Naveenaidu@users.noreply.github.com>
Co-authored-by: Anon Ray <616387+ecthiender@users.noreply.github.com>
GitOrigin-RevId: a57f6dc8b3e992d86490e5c51508827f00151dfe
2023-03-17 10:30:38 +00:00
pranshi06
b4157f58e8 server: fix scheduled events not showing up on console when the payload construction is failed
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8154
GitOrigin-RevId: ce3e4df13a55a260bc7b684818181c7ec156632c
2023-03-14 12:28:49 +00:00
Rakesh Emmadi
0c3244c2c0 multitenant: switch schema sync listener to polling mechanism
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8146
GitOrigin-RevId: 7fe82899ca8ef9eba526fe4b298b0ffbce32be82
2023-03-14 04:39:34 +00:00
Antoine Leblanc
cf531b05cb Rewrite Tracing to allow for only one TraceT in the entire stack.
This PR is on top of #7789.

### Description

This PR entirely rewrites the API of the Tracing library, to make `interpTraceT` a thing of the past. Before this change, we ran traces by sticking a `TraceT` on top of whatever we were doing. This had several major drawbacks:
- we were carrying a bunch of `TraceT` across the codebase, and the entire codebase had to know about it
- we needed to carry a second class constraint around (`HasReporterM`) to be able to run all of those traces
- we kept having to do stack rewriting with `interpTraceT`, which went from inconvenient to horrible
- we had to declare several behavioral instances on `TraceT m`

This PR rewrite all of `Tracing` using a more conventional model: there is ONE `TraceT` at the bottom of the stack, and there is an associated class constraint `MonadTrace`: any part of the code that happens to satisfy `MonadTrace` is able to create new traces. We NEVER have to do stack rewriting, `interpTraceT` is gone, and `TraceT` and `Reporter` become  implementation details that 99% of the code is blissfully unaware of: code that needs to do tracing only needs to declare that the monad in which it operates implements `MonadTrace`.

In doing so, this PR revealed **several bugs in the codebase**: places where we were expecting to trace something, but due to the default instance of `HasReporterM IO` we would actually not do anything. This PR also splits the code of `Tracing` in more byte-sized modules, with the goal of potentially moving to `server/lib` down the line.

### Remaining work

This PR is a draft; what's left to do is:
- [x] make Pro compile; i haven't updated `HasuraPro/Main` yet
- [x] document Tracing by writing a note that explains how to use the library, and the meaning of "reporter", "trace" and "span", as well as the pitfalls
- [x] discuss some of the trade-offs in the implementation, which is why i'm opening this PR already despite it not fully building yet
- [x] it depends on #7789 being merged first

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7791
GitOrigin-RevId: cadd32d039134c93ddbf364599a2f4dd988adea8
2023-03-13 17:38:39 +00:00
Gil Mizrahi
7872be0e82 feature(server): support subscriptions in logical models
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8076
GitOrigin-RevId: 84a3e89d97bdb81c02803b644f417dfe51834405
2023-02-28 11:18:43 +00:00
Philip Lykke Carlsen
e979a39f6d chore: Fix all outstanding hlint hints
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8042
GitOrigin-RevId: 87c718fa7b09f375ea0e7c2465788ac49f575290
2023-02-20 17:43:28 +00:00
Karthikeyan Chinnakonda
958a5e1a33 Include cron trigger's name in the error message
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7914
GitOrigin-RevId: 7bce2788ac154e7673402022414ab1fff653340d
2023-02-13 07:11:57 +00:00
Rakesh Emmadi
e6c5a9505b server: log when cron triggers are fetched from database for generating events
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7725
Co-authored-by: Rob Dominguez <24390149+robertjdominguez@users.noreply.github.com>
GitOrigin-RevId: 8dd2b07c769392e057400c5e60e40e7fc56ac26f
2023-02-07 12:23:42 +00:00
Antoine Leblanc
e99f9a2f57 Remove MetadataStorageT, clean up error handling.
## Description

This PR removes `MetadataStorageT`, and cleans up all top-level error handling. In short: this PR changes `MonadMetadataStorage` to explicitly return a bunch of `Either QErr a`, instead of relying on the stack providing a `MonadError QErr`. Since we implement that class on the base monad *below any ExceptT*, this removes a lot of very complicated instances that make assumptions about the shape of the stack.

On the back of this, we can remove several layers of ExceptT from the core of the code, including the one in `RunT`, which allows us to remove several instances of `liftEitherM . runExceptT`.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7689
GitOrigin-RevId: 97d600154d690f58c0b93fb4cc2d30fd383fd8b8
2023-02-03 01:05:09 +00:00
Rakesh Emmadi
ab99215828 server: log when events of scheduled & event triggers are fetched from database for delivery
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7651
Co-authored-by: Sean Park-Ross <94021366+seanparkross@users.noreply.github.com>
GitOrigin-RevId: 3149dae41c6a8a16caa1d44fbc0606451d6d2cfa
2023-01-30 06:08:08 +00:00
awjchen
12fdac004f server: fix tracing bug where some errors prevent spans from being emitted
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7450
GitOrigin-RevId: 23f6c9cfea8e7ca64b39866d15d2e6187aaaa0d9
2023-01-25 03:38:21 +00:00
Daniel Harvey
06b284cf33 [server] metadata API for native access
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7476
Co-authored-by: Tom Harding <6302310+i-am-tom@users.noreply.github.com>
GitOrigin-RevId: 781c29666e92004dc82918c2292fdacc27fded4c
2023-01-16 17:21:22 +00:00
awjchen
1ec5efd5d3 server: replicate log-based data transfer metrics as prometheus metrics
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7354
GitOrigin-RevId: 3f49b8ebba515b42a9d7b22e83e6f39d9d6087c6
2022-12-28 03:49:33 +00:00
awjchen
ee78e32c6e server: implement trace sampling
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7300
GitOrigin-RevId: d96d7fa5aaf0c1e71d1c4c0fa8f0162abce39e18
2022-12-22 19:48:51 +00:00
paritosh-08
3d4c66041f server: add event_processing_time metric
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6987
GitOrigin-RevId: 4f5948a225b18e2db59572482d2a608117fbb7d0
2022-12-06 15:11:04 +00:00