Commit Graph

110 Commits

Author SHA1 Message Date
Krushan Bauva
ca0f007bc3 server: add new metric hasura_events_fetched_per_batch
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9121
Co-authored-by: Rob Dominguez <24390149+robertjdominguez@users.noreply.github.com>
GitOrigin-RevId: 3b68c1ec192c55f1fb765a8887271276e22f7bdd
2023-05-15 15:06:39 +00:00
pranshi06
2011f4cf37 server: time related modelling in Event Triggers for Postgres sources
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8846
GitOrigin-RevId: f9d99e3368f5722a8dbfc8aa037cd1f3971c4b98
2023-05-11 09:01:29 +00:00
paritosh-08
ae49a3f9db server: add the env var for toggling metric granularity
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9034
GitOrigin-RevId: 24a4ae932b7467a53060025e2772edbebaf94b5b
2023-05-11 05:21:16 +00:00
Tom Harding
7e334e08a4 Import HashMap, not HM, Map, M...
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8947
GitOrigin-RevId: 18e52c928e1df535579e2077b4af6c2ce92bdcef
2023-04-26 15:43:44 +00:00
Tom Harding
74f282ad16 chore(server): remove Eventing instances from RQL.Types
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8909
GitOrigin-RevId: 302a9583c3364d507dfe106111ffcabac0b1dbf5
2023-04-25 17:18:30 +00:00
Daniel Harvey
ea5c92acae chore(server): move Hasura.SQL.Backend to Hasura.RQL.Types.BackendType
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8876
GitOrigin-RevId: abfc18eeef96a1f3593bfe823adab4d161161333
2023-04-24 18:37:33 +00:00
Daniel Harvey
8cf134dad1 Split Hasura.RQL.DDL.Headers to put types in Hasura.RQL.Types.Headers
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8873
GitOrigin-RevId: 566cb4271f0eb27e6688c2e0fbc26711bdf8baa9
2023-04-24 16:45:55 +00:00
Puru Gupta
44d9987e92 server: process event triggers with a timeout
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8823
GitOrigin-RevId: 4a38ef993fffe018ae16e232f2abb5d30c604855
2023-04-24 09:28:49 +00:00
Daniel Chambers
4ccfc3490b Satisfy get_table_info from cached DBObjectsIntrospection for Data Connectors
[GDC-643]: https://hasurahq.atlassian.net/browse/GDC-643?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8761
GitOrigin-RevId: 6b8e15fc35f7f463d86c865c880779794f0f28b8
2023-04-18 05:37:34 +00:00
Philip Lykke Carlsen
0346224444 Rename "Logical Models" → "Native Queries"
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8769
GitOrigin-RevId: 66f2cbfb620d641e672a4074554d9d324a18c591
2023-04-13 16:12:20 +00:00
Daniel Harvey
7227e96278 feature(server): custom return types as discreet metadata entity
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8556
Co-authored-by: Tom Harding <6302310+i-am-tom@users.noreply.github.com>
GitOrigin-RevId: b7dcbcf378279c3bf4c8d223174b90c2cb4b9e53
2023-03-30 15:15:11 +00:00
paritosh-08
b7bae6dfec server: add new metric for event trigger observability
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8380
Co-authored-by: Rob Dominguez <24390149+robertjdominguez@users.noreply.github.com>
GitOrigin-RevId: df7d5c53668fc84e7d70d471e29774136f5d560a
2023-03-29 16:48:19 +00:00
Naveen Naidu
4e3dbed938 server: revert changes to created_at column of 'hdb_catalog.event_log'
This PR reverts the following two commits:
1. https://github.com/hasura/graphql-engine-mono/pull/8287
2. https://github.com/hasura/graphql-engine-mono/pull/8467

We are undoing a migration that was done on `hdb_catalog.event_log` table which was done in d4ae6a517da63f2f43567dc16fda135b3cd1d7e6 . And as such, users who were using event triggers on that version will come across the error:
```json
{"detail":{"info":{"code":"not-supported","error":"Expected source catalog version <= 3, but the current version is 4","path":"$"},"kind":"catalog_migrate"},"level":"error","timestamp":"2023-03-28T10:17:24.289+0530","type":"startup"}
{"code":"not-supported","error":"Expected source catalog version <= 3, but the current version is 4","path":"$"}
```
To fix these errors please run the following SQL on the source where event triggers were created on:
```
UPDATE hdb_catalog.hdb_source_catalog_version SET version = 3, upgraded_on= NOW();
ALTER table hdb_catalog.event_log ALTER COLUMN created_at SET DEFAULT NOW();
ALTER table hdb_catalog.event_invocation_logs ALTER COLUMN created_at SET DEFAULT NOW();
```

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8534
GitOrigin-RevId: b6bbcce0163c8beed80619d3cea056e643b8c180
2023-03-29 10:33:30 +00:00
pranshi06
f194c47156 server: fix value of created_at for postgres event triggers
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8287
Co-authored-by: Karthikeyan Chinnakonda <15602904+codingkarthik@users.noreply.github.com>
GitOrigin-RevId: d4ae6a517da63f2f43567dc16fda135b3cd1d7e6
2023-03-22 07:32:42 +00:00
Solomon
cca1a92399 Simplify Transformable Requests Module
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8400
GitOrigin-RevId: 10728012c3d74e178c34b926e13d3627d514ce17
2023-03-22 00:01:07 +00:00
pranshi06
ea691b3c06 server: improve logging around 'Internal Exception' errors
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8369
GitOrigin-RevId: 22160a2c8a17d571ceeda0bf3e9b672bea31b8f9
2023-03-21 12:00:00 +00:00
Puru Gupta
c437a42f6d server: rename SchemaCacheRef to AppStateRef and add AppContext to it
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8159
Co-authored-by: Naveen Naidu <30195193+Naveenaidu@users.noreply.github.com>
Co-authored-by: Anon Ray <616387+ecthiender@users.noreply.github.com>
GitOrigin-RevId: a57f6dc8b3e992d86490e5c51508827f00151dfe
2023-03-17 10:30:38 +00:00
Rakesh Emmadi
0c3244c2c0 multitenant: switch schema sync listener to polling mechanism
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8146
GitOrigin-RevId: 7fe82899ca8ef9eba526fe4b298b0ffbce32be82
2023-03-14 04:39:34 +00:00
Antoine Leblanc
cf531b05cb Rewrite Tracing to allow for only one TraceT in the entire stack.
This PR is on top of #7789.

### Description

This PR entirely rewrites the API of the Tracing library, to make `interpTraceT` a thing of the past. Before this change, we ran traces by sticking a `TraceT` on top of whatever we were doing. This had several major drawbacks:
- we were carrying a bunch of `TraceT` across the codebase, and the entire codebase had to know about it
- we needed to carry a second class constraint around (`HasReporterM`) to be able to run all of those traces
- we kept having to do stack rewriting with `interpTraceT`, which went from inconvenient to horrible
- we had to declare several behavioral instances on `TraceT m`

This PR rewrite all of `Tracing` using a more conventional model: there is ONE `TraceT` at the bottom of the stack, and there is an associated class constraint `MonadTrace`: any part of the code that happens to satisfy `MonadTrace` is able to create new traces. We NEVER have to do stack rewriting, `interpTraceT` is gone, and `TraceT` and `Reporter` become  implementation details that 99% of the code is blissfully unaware of: code that needs to do tracing only needs to declare that the monad in which it operates implements `MonadTrace`.

In doing so, this PR revealed **several bugs in the codebase**: places where we were expecting to trace something, but due to the default instance of `HasReporterM IO` we would actually not do anything. This PR also splits the code of `Tracing` in more byte-sized modules, with the goal of potentially moving to `server/lib` down the line.

### Remaining work

This PR is a draft; what's left to do is:
- [x] make Pro compile; i haven't updated `HasuraPro/Main` yet
- [x] document Tracing by writing a note that explains how to use the library, and the meaning of "reporter", "trace" and "span", as well as the pitfalls
- [x] discuss some of the trade-offs in the implementation, which is why i'm opening this PR already despite it not fully building yet
- [x] it depends on #7789 being merged first

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7791
GitOrigin-RevId: cadd32d039134c93ddbf364599a2f4dd988adea8
2023-03-13 17:38:39 +00:00
Philip Lykke Carlsen
e979a39f6d chore: Fix all outstanding hlint hints
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8042
GitOrigin-RevId: 87c718fa7b09f375ea0e7c2465788ac49f575290
2023-02-20 17:43:28 +00:00
Rakesh Emmadi
ab99215828 server: log when events of scheduled & event triggers are fetched from database for delivery
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7651
Co-authored-by: Sean Park-Ross <94021366+seanparkross@users.noreply.github.com>
GitOrigin-RevId: 3149dae41c6a8a16caa1d44fbc0606451d6d2cfa
2023-01-30 06:08:08 +00:00
awjchen
12fdac004f server: fix tracing bug where some errors prevent spans from being emitted
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7450
GitOrigin-RevId: 23f6c9cfea8e7ca64b39866d15d2e6187aaaa0d9
2023-01-25 03:38:21 +00:00
Daniel Harvey
06b284cf33 [server] metadata API for native access
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7476
Co-authored-by: Tom Harding <6302310+i-am-tom@users.noreply.github.com>
GitOrigin-RevId: 781c29666e92004dc82918c2292fdacc27fded4c
2023-01-16 17:21:22 +00:00
awjchen
1ec5efd5d3 server: replicate log-based data transfer metrics as prometheus metrics
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7354
GitOrigin-RevId: 3f49b8ebba515b42a9d7b22e83e6f39d9d6087c6
2022-12-28 03:49:33 +00:00
awjchen
ee78e32c6e server: implement trace sampling
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/7300
GitOrigin-RevId: d96d7fa5aaf0c1e71d1c4c0fa8f0162abce39e18
2022-12-22 19:48:51 +00:00
paritosh-08
3d4c66041f server: add event_processing_time metric
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6987
GitOrigin-RevId: 4f5948a225b18e2db59572482d2a608117fbb7d0
2022-12-06 15:11:04 +00:00
paritosh-08
00005020a6 server: rename event trigger metric hasura_event_processing_time_seconds to hasura_event_webhook_processing_time_seconds
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6950
GitOrigin-RevId: a32ce3f167eb6bb5c48645a614288c043c252d26
2022-11-21 07:49:28 +00:00
paritosh-08
5be4d37240 server,pro: add metrics related to event trigger runtime
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6825
GitOrigin-RevId: 913e1bd21912db781d69f579aee943faf6ce3a11
2022-11-16 17:12:48 +00:00
Samir Talwar
342391f39d Upgrade Ormolu to v0.5.
This upgrades the version of Ormolu required by the HGE repository to v0.5.0.1, and reformats all code accordingly.

Ormolu v0.5 reformats code that uses infix operators. This is mostly useful, adding newlines and indentation to make it clear which operators are applied first, but in some cases, it's unpleasant. To make this easier on the eyes, I had to do the following:

* Add a few fixity declarations (search for `infix`)
* Add parentheses to make precedence clear, allowing Ormolu to keep everything on one line
* Rename `relevantEq` to `(==~)` in #6651 and set it to `infix 4`
* Add a few _.ormolu_ files (thanks to @hallettj for helping me get started), mostly for Autodocodec operators that don't have explicit fixity declarations

In general, I think these changes are quite reasonable. They mostly affect indentation.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/6675
GitOrigin-RevId: cd47d87f1d089fb0bc9dcbbe7798dbceedcd7d83
2022-11-02 20:55:13 +00:00
Tom Harding
e71496efa5 Replace Hasura.RQL.Types.Numeric with refined
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5913
GitOrigin-RevId: 96e218229a08dfbc5a598d709be1ee2083d41ec6
2022-09-21 18:03:04 +00:00
paritosh-08
ab71adc3a0 server, pro: event trigger auto cleanup (increment 3)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5823
Co-authored-by: pranshi06 <85474619+pranshi06@users.noreply.github.com>
Co-authored-by: Puru Gupta <32328846+purugupta99@users.noreply.github.com>
Co-authored-by: Karthikeyan Chinnakonda <15602904+codingkarthik@users.noreply.github.com>
GitOrigin-RevId: 3d4af905d5df4c05107ad46ef29f4c0a567ff754
2022-09-15 11:46:22 +00:00
Solomon
418d0e2cc2 Use NonNegative types for arg/env parsing
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5305
GitOrigin-RevId: 8274ca8c4217d15939fa2a78664a2be486426bf7
2022-08-17 01:09:19 +00:00
awjchen
1a4e3ae001 server: add event trigger metrics for Prometheus
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5346
GitOrigin-RevId: 0db925f85e06e99b636995887c0e19a49fe9be2d
2022-08-15 05:34:04 +00:00
Solomon
3855efa9a5 Move soEventsHttpPoolSize and soEventsFetchInterval defaulting into arg parser
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5284
GitOrigin-RevId: f2302b3219f3aa9ebde623b4efa7f3ee8a97c031
2022-08-10 19:18:11 +00:00
Solomon
07e4a49e5e Consolidate Arg/Env Parser Defaults and cleanup arg merging code
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5262
GitOrigin-RevId: 2501e44a8fe0a57b13141c307d2dd44b028052b6
2022-08-05 00:29:52 +00:00
Samir Talwar
40617719ef server: Remove the Show instance from QErr and anything that touches it.
We only use these `Show` instances in error messages (where we call
`show` explicitly anyway) and test cases (in which Hspec requires `Show
a` for any `a` in an assertion).

This removes the instance in favor of a custom `showQErr` function
(which serializes the error to JSON). It is then used in certain error
message production which previously called `show` on a `QErr`.

There are two places where we serialize a QErr and then construct a new
QErr from the resulting string. Instead, we modify the existing QErr to
add extra information.

An orphan `Show QErr` instance is retained for tests so that we can have
nice test failure messages.

This is preparation for future changes in which the error message within
`QErr` will not be exposed directly, and therefore will not have a
`Show` instance. That said, it feels like a sensible kind of cleanup
anyway.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4897
GitOrigin-RevId: 8f79f7a356f0aea571156f39aefac242bf751f3a
2022-07-01 11:48:26 +00:00
Daniel Harvey
03c93ad96f server: Refactor TByteString to SerializableBlob
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4736
GitOrigin-RevId: 446832b00c24205c2237666ac9d954277c5fd05d
2022-06-17 09:57:38 +00:00
Puru Gupta
ebc8cd5134 server: do not serialize env vars in logs or errors: PR III - Event Triggers
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4521
Co-authored-by: Naveen Naidu <30195193+Naveenaidu@users.noreply.github.com>
Co-authored-by: Brandon Simmons <210815+jberryman@users.noreply.github.com>
GitOrigin-RevId: c7702e8f3da466dc108936328de30ceb87df258a
2022-06-05 20:28:13 +00:00
Naveen Naidu
222419527f server: refactor MaintenanceMode datatype
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4298
GitOrigin-RevId: 0b88d040bd4b24c61c4ae91e18dc6edf67b3672e
2022-04-28 20:56:11 +00:00
Antoine Leblanc
3cbcbd9291 Remove RQL/Types.hs
## Description

This PR removes `RQL.Types`, which was now only re-exporting a bunch of unrelated modules.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/4363
GitOrigin-RevId: 894f29a19bff70b3dad8abc5d9858434d5065417
2022-04-27 13:58:47 +00:00
jkachmar
647231b685 Yeet some default-extensions
Manually enables:
* EmptyCase
* ExistentialQuantification
* QuantifiedConstraints
* QuasiQuotes
* TemplateHaskell
* TypeFamilyDependencies

...in the following components:
* 'graphql-engine' library
* 'graphql-engine' 'src-test'
* 'graphql-engine' 'tests/integration'
* 'graphql-engine' tests-hspec'

Additionally, performs some light refactoring and documentation.

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3991
GitOrigin-RevId: 514477d3466b01f60eca8935d0fef60dd0756838
2022-03-16 00:40:17 +00:00
Karthikeyan Chinnakonda
a2da867fc8 Fetch only events of the ETs which exist in the metadata
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3857
GitOrigin-RevId: 1ee501116103e2be49d4538c80f87c11c066af8e
2022-03-08 09:06:32 +00:00
Solomon
d67d4e2310 Webhook Transform Cleanup / Refactor
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3624
GitOrigin-RevId: 849e6dd70d6fe3d84056a485b20928ff813881d4
2022-03-08 00:43:08 +00:00
Solomon
4b792abdcc Feature/webhook response transforms
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3074
Co-authored-by: David Overton <7734777+dmoverton@users.noreply.github.com>
Co-authored-by: Lyndon Maydwell <92299+sordina@users.noreply.github.com>
Co-authored-by: Tirumarai Selvan <8663570+tirumaraiselvan@users.noreply.github.com>
GitOrigin-RevId: 006c5c9b71cdca1c2f47962230e6189e09557fab
2022-01-19 04:47:36 +00:00
Solomon
0faa92337d Use transformed request body in event invocation logs
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3057
GitOrigin-RevId: 7e001830f97c81289553e37c35098ccaeadbdf22
2021-12-06 22:40:31 +00:00
David Overton
aac64f2c81 Source typename customization (close graphql-engine#6974)
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/1616
GitOrigin-RevId: f7eefd2367929209aa77895ea585e96a99a78d47
2021-10-29 14:43:14 +00:00
Robert
71af68e9e5 server: drop HasVersion implicit parameter (closes #2236)
The only real use was for the dubious multitenant option
--consoleAssetsVersion, which actually overrode not just
the assets version. I.e., as far as I can tell, if you pass
--consoleAssetsVersion to multitenant, that version will
also make it into e.g. HTTP client user agent headers as
the proper graphql-engine version.

I'm dropping that option, since it seems unused in production
and I don't want to go to the effort of fixing it, but am happy
to look into that if folks feels strongly that it should be
kept.

(Reason for attacking this is that I was looking into http
client things around blacklisting, and the versioning thing
is a bit painful around http client headers.)

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2458
GitOrigin-RevId: a02b05557124bdba9f65e96b3aa2746aeee03f4a
2021-10-13 16:39:58 +00:00
Solomon Bothwell
4e05bdcaec Feature/request transform string interpolation
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/2443
Co-authored-by: Tirumarai Selvan <8663570+tirumaraiselvan@users.noreply.github.com>
GitOrigin-RevId: d7d68984d0ae1403bb414572e9704c01ed27deab
2021-09-29 08:14:29 +00:00
Robert
11a454c2d6 server, pro: actually reformat the code-base using ormolu
This commit applies ormolu to the whole Haskell code base by running `make format`.

For in-flight branches, simply merging changes from `main` will result in merge conflicts.
To avoid this, update your branch using the following instructions. Replace `<format-commit>`
by the hash of *this* commit.

$ git checkout my-feature-branch
$ git merge <format-commit>^    # and resolve conflicts normally
$ make format
$ git commit -a -m "reformat with ormolu"
$ git merge -s ours post-ormolu

https://github.com/hasura/graphql-engine-mono/pull/2404

GitOrigin-RevId: 75049f5c12f430c615eafb4c6b8e83e371e01c8e
2021-09-23 22:57:37 +00:00
Naveen Naidu
3d95c67748 server/query-tags: implement new metadata specification and API
https://github.com/hasura/graphql-engine-mono/pull/2203

GitOrigin-RevId: 6169fcce0c6dccaec055bf8e723e9aa7c19a7f0c
2021-09-23 12:38:56 +00:00