…arger ones
…the goal being to save on data transfer costs, libdeflate being much faster than zlib for larger inputs and at higher compression levels. A few notes:
In last month...
- 95% of response bodies > 20kB compress below 32% (with zlib level 1)
- The 10% of responses > 20kB comprise 75% egress traffic to clients
- libdeflate at level 6 is comparable in performance to zlib level 1, and twice as fast as zlib level 6
- We expect compressing 20kB+ response bodies at level 6 to reduce data transfer to clients by 25% or so (although this is difficult to predict accurately)
The new libdeflate bindings used here also need review: https://github.com/hasura/libdeflate-hs
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/10341
GitOrigin-RevId: bc7b19e0024e442d85ac0b34995610edbab13bd6
A fix for #9447
Before this change, for the workload in the ticket, we saw a memory stay at around 4 GB an hour after the workload ended. after this change we see both a lower memory high watermark, and also see memory come back to baseline at the end as we expect (after the `HASURA_GRAPHQL_PG_CONN_LIFETIME` and a subsequent `_idleStaleReaperThread` interval elapses). But note the numbers given on any given machine may be slightly different, since the behavior depends on the version of glibc and processor speed (Since the test case depends on the server being overloaded).
This might also help some users commenting in https://github.com/hasura/graphql-engine/issues/9592
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/9823
GitOrigin-RevId: 5650aa42d10d46c418c21686983a982d69011884
Upgrade to GHC 9.4.5, and update any tests.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/8954
Co-authored-by: Mohd Bilal <24944223+m-Bilal@users.noreply.github.com>
Co-authored-by: Samir Talwar <47582+SamirTalwar@users.noreply.github.com>
Co-authored-by: Philip Lykke Carlsen <358550+plcplc@users.noreply.github.com>
GitOrigin-RevId: 5261126777cb478567ea471c4bf5441bc345ea0d
This removes string interpolation from quasiquoted literals. We only use
this in one place and it's totally unnecessary.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/5750
GitOrigin-RevId: 3493a11db6347332e7e3721a7dca616947505be6
They're tied to old-style cabal, and graphql-engine doesn't build with
those anymore anyhow.
Also removes a cabal.project Cabal version constraint related to Setup.hs.
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3847
GitOrigin-RevId: 715e836cd2ddf7982143ba0ab8821f87a5eac0b1
## Description
This PR is in reference to #2449 (support IP blacklisting for multitenant)
*RFC Update: Add support for IPv6 blocking*
### Solution and Design
Using [http-client-restricted](https://hackage.haskell.org/package/http-client-restricted) package, we're creating the HTTP manager with restricting capabilities. The IPs can be supplied from the CLI arguments as `--ipv4BlocklistCidrs cidr1, cidr2...` or `--disableDefaultIPv4Blocklist` for a default IP list. The new manager will block all requests to the provided CIDRs.
We are extracting the error message string to show the end-user that given IP is blocked from being set as a webhook. There are 2 ways to extract the error message "connection to IP address is blocked". Given below are the responses from event trigger to a blocked IP for these implementations:
- 6d74fde316f61e246c861befcca5059d33972fa7 - We return the error message string as a HTTPErr(HOther) from `Hasura/Eventing/HTTP.hs`.
```
{
"data": {
"message": "blocked connection to private IP address "
},
"version": "2",
"type": "client_error"
}
```
- 88e17456345cbb449a5ecd4877c84c9f319dbc25 - We case match on HTTPExceptionContent for InternaException in `Hasura/HTTP.hs` and extract the error message string from it. (this is implemented as it handles all the cases where pro engine makes webhook requests)
```
{
"data": {
"message": {
"type": "http_exception",
"message": "blocked connection to private IP address ",
"request": {
"secure": false,
"path": "/webhook",
"responseTimeout": "ResponseTimeoutMicro 60000000",
"queryString": "",
"method": "POST",
"requestHeaders": {
"Content-Type": "application/json",
"X-B3-ParentSpanId": "5ae6573edb2a6b36",
"X-B3-TraceId": "29ea7bd6de6ebb8f",
"X-B3-SpanId": "303137d9f1d4f341",
"User-Agent": "hasura-graphql-engine/cerebushttp-ip-blacklist-a793a0e41-dirty"
},
"host": "139.59.90.109",
"port": 8000
}
}
},
"version": "2",
"type": "client_error"
}
```
### Steps to test and verify
The restricted IPs can be used as webhooks in event triggers, and hasura will return an error message in reponse.
### Limitations, known bugs & workarounds
- The `http-client-restricted` has a needlessly complex interface, and puts effort into implementing proxy support which we don't want, so we've inlined a stripped down version.
- Performance constraint: As the blocking is checked for each request, if a long list of blocked CIDRs is supplied, iterating through all of them is not what we would prefer. Using trie is suggested to overcome this. (Added to RFC)
- Calls to Lux endpoints are inconsistent: We use either the http manager from the ProServeCtx which is unrestricted, or the http manager from the ServeCtx which is restricted (the latter through the instances for MonadMetadataApiAuthorization and UserAuthentication). (The failure scenario here would be: cloud sets PRO_ENDPOINT to something that resolves to an internal address, and then restricted requests to those endpoints fail, causing auth to fail on user requests. This is about HTTP requests to lux auth endpoints.)
## Changelog
- ✅ `CHANGELOG.md` is updated with user-facing content relevant to this PR.
## Affected components
- ✅ Server
- ✅ Tests
PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3186
Co-authored-by: Robert <132113+robx@users.noreply.github.com>
GitOrigin-RevId: 5bd2de2d028bc416b02c99e996c7bebce56fb1e7
