graphql-engine/server/src-lib/Hasura/HTTP.hs
Puru Gupta fcef6e5cb2 server: http ip blocklist (closes #2449)
## Description

This PR is in reference to #2449 (support IP blacklisting for multitenant)

*RFC Update: Add support for IPv6 blocking*

### Solution and Design

Using [http-client-restricted](https://hackage.haskell.org/package/http-client-restricted) package, we're creating the HTTP manager with restricting capabilities. The IPs can be supplied from the CLI arguments as `--ipv4BlocklistCidrs cidr1, cidr2...` or `--disableDefaultIPv4Blocklist` for a default IP list. The new manager will block all requests to the provided CIDRs.

We are extracting the error message string to show the end-user that given IP is blocked from being set as a webhook. There are 2 ways to extract the error message "connection to IP address is blocked". Given below are the responses from event trigger to a blocked IP for these implementations:
- 6d74fde316f61e246c861befcca5059d33972fa7 - We return the error message string as a HTTPErr(HOther) from `Hasura/Eventing/HTTP.hs`.
```
{
    "data": {
        "message": "blocked connection to private IP address "
    },
    "version": "2",
    "type": "client_error"
}
```

- 88e17456345cbb449a5ecd4877c84c9f319dbc25 - We case match on HTTPExceptionContent for InternaException in `Hasura/HTTP.hs` and extract the error message string from it. (this is implemented as it handles all the cases where pro engine makes webhook requests)
```
{
  "data": {
    "message": {
      "type": "http_exception",
      "message": "blocked connection to private IP address ",
      "request": {
        "secure": false,
        "path": "/webhook",
        "responseTimeout": "ResponseTimeoutMicro 60000000",
        "queryString": "",
        "method": "POST",
        "requestHeaders": {
          "Content-Type": "application/json",
          "X-B3-ParentSpanId": "5ae6573edb2a6b36",
          "X-B3-TraceId": "29ea7bd6de6ebb8f",
          "X-B3-SpanId": "303137d9f1d4f341",
          "User-Agent": "hasura-graphql-engine/cerebushttp-ip-blacklist-a793a0e41-dirty"
        },
        "host": "139.59.90.109",
        "port": 8000
      }
    }
  },
  "version": "2",
  "type": "client_error"
}
```

### Steps to test and verify
The restricted IPs can be used as webhooks in event triggers, and hasura will return an error message in reponse.

### Limitations, known bugs & workarounds
- The `http-client-restricted` has a needlessly complex interface, and puts effort into implementing proxy support which we don't want, so we've inlined a stripped down version.
- Performance constraint: As the blocking is checked for each request, if a long list of blocked CIDRs is supplied, iterating through all of them is not what we would prefer. Using trie is suggested to overcome this. (Added to RFC)
- Calls to Lux endpoints are inconsistent: We use either the http manager from the ProServeCtx which is unrestricted, or the http manager from the ServeCtx which is restricted (the latter through the instances for MonadMetadataApiAuthorization and UserAuthentication). (The failure scenario here would be: cloud sets PRO_ENDPOINT to something that resolves to an internal address, and then restricted requests to those endpoints fail, causing auth to fail on user requests. This is about HTTP requests to lux auth endpoints.)

## Changelog

-  `CHANGELOG.md` is updated with user-facing content relevant to this PR.

## Affected components

-  Server
-  Tests

PR-URL: https://github.com/hasura/graphql-engine-mono/pull/3186
Co-authored-by: Robert <132113+robx@users.noreply.github.com>
GitOrigin-RevId: 5bd2de2d028bc416b02c99e996c7bebce56fb1e7
2022-02-25 13:30:57 +00:00

141 lines
5.5 KiB
Haskell

module Hasura.HTTP
( wreqOptions,
HttpException (..),
hdrsToText,
addDefaultHeaders,
defaultHeaders,
HttpResponse (..),
addHttpResponseHeaders,
getHTTPExceptionStatus,
serializeHTTPExceptionMessage,
)
where
import Control.Exception (fromException)
import Control.Lens hiding ((.=))
import Data.Aeson qualified as J
import Data.CaseInsensitive (original)
import Data.HashMap.Strict qualified as M
import Data.Text qualified as T
import Data.Text.Conversions (UTF8 (..), convertText)
import Data.Text.Encoding qualified as TE
import Hasura.Prelude
import Hasura.Server.Utils (redactSensitiveHeader)
import Hasura.Server.Version (currentVersion)
import Network.HTTP.Client qualified as HTTP
import Network.HTTP.Client.Restricted qualified as Restricted
import Network.HTTP.Types qualified as HTTP
import Network.Wreq qualified as Wreq
hdrsToText :: [HTTP.Header] -> [(Text, Text)]
hdrsToText hdrs =
[ (bsToTxt $ original hdrName, bsToTxt hdrVal)
| (hdrName, hdrVal) <- hdrs
]
wreqOptions :: HTTP.Manager -> [HTTP.Header] -> Wreq.Options
wreqOptions manager hdrs =
Wreq.defaults
& Wreq.headers .~ addDefaultHeaders hdrs
& Wreq.checkResponse ?~ (\_ _ -> return ())
& Wreq.manager .~ Right manager
-- Adds defaults headers overwriting any existing ones
addDefaultHeaders :: [HTTP.Header] -> [HTTP.Header]
addDefaultHeaders hdrs = defaultHeaders <> rmDefaultHeaders hdrs
where
rmDefaultHeaders = filter (not . isDefaultHeader)
isDefaultHeader :: HTTP.Header -> Bool
isDefaultHeader (hdrName, _) = hdrName `elem` map fst defaultHeaders
defaultHeaders :: [HTTP.Header]
defaultHeaders = [contentType, userAgent]
where
contentType = ("Content-Type", "application/json")
userAgent =
( "User-Agent",
"hasura-graphql-engine/" <> unUTF8 (convertText currentVersion)
)
newtype HttpException = HttpException
{unHttpException :: HTTP.HttpException}
deriving (Show)
getHTTPExceptionStatus :: HttpException -> Maybe Int
getHTTPExceptionStatus = \case
(HttpException (HTTP.HttpExceptionRequest _ httpExceptionContent)) ->
case httpExceptionContent of
HTTP.StatusCodeException response _ -> Just $ HTTP.statusCode $ HTTP.responseStatus response
HTTP.ProxyConnectException _ _ status -> Just $ HTTP.statusCode status
_ -> Nothing
(HttpException (HTTP.InvalidUrlException _ _)) -> Nothing
serializeHTTPExceptionMessage :: HttpException -> Text
serializeHTTPExceptionMessage (HttpException (HTTP.HttpExceptionRequest _ httpExceptionContent)) =
case httpExceptionContent of
HTTP.StatusCodeException _ _ -> "unexpected"
HTTP.TooManyRedirects _ -> "Too many redirects"
HTTP.OverlongHeaders -> "Overlong headers"
HTTP.ResponseTimeout -> "Response timeout"
HTTP.ConnectionTimeout -> "Connection timeout"
HTTP.ConnectionFailure _ -> "Connection failure"
HTTP.InvalidStatusLine _ -> "Invalid HTTP Status Line"
HTTP.InternalException err -> case fromException err of
Just (Restricted.ConnectionRestricted _ _) -> "Blocked connection to private IP address"
Nothing -> "Internal Exception"
HTTP.ProxyConnectException _ _ _ -> "Proxy connection exception"
HTTP.NoResponseDataReceived -> "No response data received"
HTTP.TlsNotSupported -> "TLS not supported"
HTTP.InvalidDestinationHost _ -> "Invalid destination host"
HTTP.InvalidHeader _ -> "Invalid Header"
HTTP.InvalidRequestHeader _ -> "Invalid Request Header"
HTTP.WrongRequestBodyStreamSize _ _ -> "Wrong request body stream size"
HTTP.ResponseBodyTooShort _ _ -> "Response body too short"
HTTP.InvalidChunkHeaders -> "Invalid chunk headers"
HTTP.IncompleteHeaders -> "Incomplete headers"
_ -> "unexpected"
serializeHTTPExceptionMessage (HttpException (HTTP.InvalidUrlException url reason)) = T.pack $ "URL: " <> url <> " is invalid because " <> reason
encodeHTTPRequestJSON :: HTTP.Request -> J.Value
encodeHTTPRequestJSON request =
J.Object $
M.fromList
[ ("host", J.toJSON $ TE.decodeUtf8 $ HTTP.host request),
("port", J.toJSON $ HTTP.port request),
("secure", J.toJSON $ HTTP.secure request),
("requestHeaders", J.toJSON $ M.fromList $ hdrsToText $ map redactSensitiveHeader $ HTTP.requestHeaders request),
("path", J.toJSON $ TE.decodeUtf8 $ HTTP.path request),
("queryString", J.toJSON $ TE.decodeUtf8 $ HTTP.queryString request),
("method", J.toJSON $ TE.decodeUtf8 $ HTTP.method request),
("responseTimeout", J.String $ tshow $ HTTP.responseTimeout request)
]
instance J.ToJSON HttpException where
toJSON httpException =
case httpException of
(HttpException (HTTP.InvalidUrlException _ e)) ->
J.object
[ "type" J..= ("invalid_url" :: Text),
"message" J..= e
]
(HttpException (HTTP.HttpExceptionRequest req _)) ->
let statusMaybe = getHTTPExceptionStatus httpException
exceptionContent = serializeHTTPExceptionMessage httpException
reqJSON = encodeHTTPRequestJSON req
in J.object $
[ "type" J..= ("http_exception" :: Text),
"message" J..= exceptionContent,
"request" J..= reqJSON
]
<> maybe mempty (\status -> ["status" J..= status]) statusMaybe
data HttpResponse a = HttpResponse
{ _hrBody :: !a,
_hrHeaders :: !HTTP.ResponseHeaders
}
deriving (Functor, Foldable, Traversable)
addHttpResponseHeaders :: HTTP.ResponseHeaders -> HttpResponse a -> HttpResponse a
addHttpResponseHeaders newHeaders (HttpResponse b h) = HttpResponse b (newHeaders <> h)