@@ -0,0 +1,331 @@
1+# How To Test Servant Applications
3+Even with a nicely structured API that passes Haskell's strict type checker,
4+it's a good idea to write some tests for your application.
6+In this recipe we'll work through some common testing strategies and provide
7+examples of utlizing these testing strategies in order to test Servant
10+This recipe uses the following ingredients:
13+{-# LANGUAGE OverloadedStrings, TypeFamilies, DataKinds,
14+ DeriveGeneric, TypeOperators #-}
15+import qualified Control.Concurrent as C
17+import Data.Aeson.Lens
20+import qualified Network.Wai.Handler.Warp as Warp
27+import Test.Hspec.Wai.Matcher
30+We're going to produce different `Spec`s that represent different
31+aspects of our application, and we'll ask `hspec` to run all of our different
32+`Spec`s. This is a common organizational method for testing modules:
38+ thirdPartyResourcesSpec
39+ servantQuickcheckSpec
42+Often, codebases will use `hspec`'s
43+[autodiscover pragma](http://hspec.github.io/hspec-discover.html)
44+to find all testing modules and `Spec`s inside, but we're going to
45+explicitly make a `main` function to run our tests because we have only one
53+## Testing Your Business Logic
55+Let's say we have an API that looks something like this:
61+ } deriving (Eq, Show, Generic)
63+instance FromJSON User
67+ -- One endpoint: create a user
68+ "user" :> Capture "userId" Integer :> Post '[JSON] User
71+A real server would likely use a database to store, retrieve, and validate
72+users, but we're going to do something really simple merely to have something
73+to test. With that said, here's a sample handler for the endpoint described
77+userServer :: Server UserApi Handler
78+userServer = createUser
79+ where creatUser userId
80+ | userId > 5000 = pure $ User { name = "some user", user_id = userId }
81+ | otherwise = throwError $ err400 { errBody = "userId is too small" }
84+### Strategy 1: Spin Up a Server, Create a Client, Make Some Requests
86+One of the benefits of Servant's type-level DSL for describing APIs is that
87+once you have provided a type-level description of your API, you can create
88+clients, documentation, or other tools for it somewhat magically.
90+In this case, we'd like to *test* our server, so we can use `servant-client`
91+to create a client, after which we'll run our server, and then make requests
92+of it and see how it responds.
94+Let's write some tests:
97+businessLogicSpec :: Spec
98+businessLogicSpec = do
99+ -- create a test client function
100+ createUser = client (Proxy :: Proxy UserApi)
101+ -- create a servant-client ClientEnv
102+ baseUrl <- parseBaseUrl "http://localhost:8888"
103+ manager <- newManager defaultManagerSettings
104+ let clientEnv = mkClientEnv manager baseUrl
106+ -- Run the server in another thread.
107+ liftIO $ C.forkIO $ Warp.run 8888 (server userServer)
109+ -- testing scenarios start here
110+ describe "POST /user" $ do
111+ it "should create a user with a high enough ID" $
112+ result <- runClientM clientEnv (createUser 50001)
113+ result `shouldEqual` Right $ User { name = "some_user", user_id = "5001 "}
114+ it "should fail with a too-small ID" $
115+ result <- runClientM clientEnv (createUser 4999)
116+ result `shouldEqual` Right $ User { name = "some_user", user_id = "5001 "}
119+### Running These Tests
121+Let's run our tests and see what happens:
127+Great: we passed! Servers obviously get more complex, though, and we may not
128+wish to create a whole suite of clients for our server every time. In our next
129+scenario we're going to mock out a 3rd-party resource that our server talks to
130+and we're going to be using `hspec-wai` to run our `Application` instance and
133+## *Mocking* 3rd Party Resources
135+Often our web applications will need to make their own web
136+requests to other 3rd-party applications. These requests provide a lot
137+of opportunity for failure and so we'd like to test that the right
138+messages and failure values (in addition to success values) are returned
139+from our application.
141+### Define the 3rd-Party Resource
143+With Servant's type-level API definitions, assuming you've already defined the
144+API you want to mock, it's relatively trivial to create a simple server for
145+the purposes of running tests. For instance, consider an API server that needs
146+to get data out of Elasticsearch. Let's first define the Elasticsearch server
147+and client using Servant API descriptions:
151+ -- We're using Aeson's Generic JSON `Value` to make things easier on
152+ -- ourselves. We're also representing only one Elasticsearch endpoint:
154+ "myIndex" :> "myDocType" :> Capture "docId" Integer :> Get '[JSON] Value
156+-- Here's our Servant Client function
157+getDocument = client (Proxy :: Proxy SearchAPI)
159+-- We can use these helpers when we want to make requests
160+-- using our client function
161+clientEnv :: Text -> Text -> IO ClientEnv
162+clientEnv esHost esPort = do
163+ baseUrl <- parseBaseUrl $ T.unpack $ esHost <> ":" <> esPort
164+ manager <- newManager defaultManagerSettings
165+ pure $ mkClientEnv manager baseUrl
167+runSearchClient :: Text -> Text -> ClientM a -> IO (Either ServantError a)
168+runSearchClient esHost esPort = (clientEnv esHost esPort >>=) . runClientM
171+### Servant Server Example Using this 3rd-Party Resource
173+So we've got an Elasticsearch server and a client to talk to it with one
174+function that retrieves a document by its id. Let's now build a simple app
175+server that uses this client to retrieve documents. This is somewhat
176+contrived, but hopefully it illustrates the typical three-tier application
179+One note: we're also going to take advantage of `aeson-lens` here, which may
180+look a bit foreign. The gist of it is that we're going to traverse a JSON
181+`Value` from Elasticsearch and try to extract some kind of document to
186+ "docs" :> Capture "docId" Integer :> Get '[JSON] Value
188+docServer :: Text -> Text -> Server DocApi Handler
189+docServer esHost esPort = getDocById esHost esPort
191+-- Our Handler tries to get a doc from Elasticsearch and then tries to parse
192+-- it. Unfortunately, there's a lot of opportunity for failure in these
194+getDocById :: Text -> Text -> Integer -> Handler Value
195+getDocById esHost esPort docId = do
196+ -- Our Servant Client function returns Either ServantError Value here:
197+ docRes <- liftIO $ runSearchClient esHost esPort (getDocument docId)
199+ Left err -> throwError $ err404 { errBody = "Failed looking up content" }
201+ -- we'll either fail to parse our document or we'll return it
202+ case value ^? _Object . ix "_source" of
203+ Nothing -> throwError $ err400 { errBody = "Failed parsing content" }
204+ Just obj -> pure obj
207+### Testing Our Backend
209+So the above represents our application. How shall we test this application?
210+Ideally, we'd like it to make requests of a real Elasticsearch server, but we
211+certainly don't want our tests to trigger requests to a live, production
212+database. In addition, we don't want to depend on our real Elasticsearch
213+server having specific, consistent results for us to test against, because
214+that would make our tests flaky (and flaky tests are sometimes described as
215+worse than not having tests at all).
217+One solution to this is to create a trivial Elasticsearch server as part of
218+our testing code. We can do this relatively easily because we already have
219+an API definition for it above. With a *real* server, we can then let our own
220+application make requests of it and we'll simulate different scenarios in
221+order to make sure our application responds the way we expect it to.
223+Let's start with some helpers which will allow us to run a testing version
224+of our Elasticsearch server in another thread:
227+-- | We'll run the Elasticsearch server so we can test behaviors
228+withElasticsearch :: IO () -> IO ()
229+withElasticsearch action =
230+ bracket (liftIO $ C.forkIO $ Warp.run 9999 esTestApp)
234+esTestApp :: Application
235+esTestApp = serve (Proxy :: Proxy SearchAPI) esTestServer
237+esTestServer :: Server SearchAPI Handler
238+esTestServer = getESDocument
240+getESDocument :: Integer -> Handler Value
242+ -- arbitrary things we can trigger in our tests to check for failure
243+ -- We want to try to trigger different code paths
244+ | docId > 1000 = pure . Left $ ConnectionError "Bad connection!"
245+ | docId > 500 = pure . Object $ HM.fromList [("bad", String "data")]
246+ | otherwise = pure $ Object $ HM.fromList [("_source", defaultDocument)]
249+Now, we should be ready to write some tests. As mentioned above we're going
250+to use `hspec-wai` in this example to make test requests and to run our own
251+application. This should simplify our testing code somewhat:
254+thirdPartyResourcesSpec :: Spec
255+thirdPartyResourcesSpec = around_ withElasticsearch $ do
256+ -- we call `with` and pass our servant-server `Application`
257+ with (pure $ serve $ docServer "localhost" "9999") $ do
258+ describe "GET /docs" $ do
259+ it "should be able to get a document" $
260+ -- `get` is a function from hspec-wai`.
261+ get "/docs/1" `shouldRespondWith` 200
262+ it "should be able to handle connection failures" $
263+ -- We can also make custom HTTP requests with the `request` function
264+ get "/docs/1001" `shouldRespondWith` 404
265+ it "should be able to handle parsing failures" $
266+ get "/docs/501" `shouldRespondWith` 400
267+ it "should be able to handle odd HTTP requests" $
268+ -- we can also make all kinds of arbitrary custom requests to see how
269+ -- our server responds using the `request` function:
270+ -- request :: Method -> ByteString -> [Header]
271+ -- -> LB.ByteString -> WaiSession SResponse
272+ request methodPost "/docs/501" [] "{" `shouldRespondWith` 415
276+## Servant Quickcheck
278+[`servant-quickcheck`](https://github.com/haskell-servant/servant-quickcheck)
279+is a project that allows users to write tests for whole Servant APIs using
280+quickcheck-style property-checking mechanisms.
282+`servant-quickcheck` is great for asserting whole-API rules, such as "no
283+endpoint throws a 500" or "all 301 status codes also come with a Location
284+header". The project even comes with a number of predicates that reference
285+the [RFCs they originate from](https://github.com/haskell-servant/servant-quickcheck/blob/master/src/Servant/QuickCheck/Internal/Predicates.hs).
288+### Quickcheckable API
290+Let's make an API and a server to demonstrate how to use `servant-quickcheck`:
293+type API = ReqBody '[JSON] String :> Post '[JSON] String
294+ :<|> Get '[JSON] Int
295+ :<|> BasicAuth "some-realm" () :> Get '[JSON] ()
300+server :: IO (Server API)
303+ return $ (\x -> liftIO $ swapMVar mvar x)
304+ :<|> (liftIO $ readMVar mvar >>= return . length)
305+ :<|> (const $ return ())
308+### Using `servant-quickcheck`
310+`servant-quickcheck` also has a cool mechanism where you can compare two API
311+servers to demonstrate that they respond identically to requests. This may be
312+useful if you are planning to rewrite one API in another language or with
313+another web framework. You have to specify whether you're looking for
314+`jsonEquality` vs regular `ByteString` equality, though:
317+servantQuickcheckSpec :: Spec
318+servantQuickcheckSpec = describe "" $ do
319+ it "API demonstrates best practices" $
320+ withServantServer api server $ \burl ->
321+ serverSatisfies api burl args (unauthorizedContainsWWWAuthenticate
326+ it "API doesn't have these things implemented yet" $
327+ withServantServer api server $ \burl -> do
328+ serverDoesntSatisfy api burl args (getsHaveCacheControlHeader
329+ <%> notAllowedContainsAllowHeader