git.iko.soy/typeable/bloodhound
on 10671dc help sign in
typeable/bloodhound/README.org
339 lines 10 kB history raw
1* Bloodhound
3#+CAPTION: Bloodhound (dog)
4[[./bloodhound.jpg]]
6* Elasticsearch client and query DSL for Haskell
8** Why?
10Because you're tired of obnoxious errors like [this](http://i.imgur.com/FKtZYIP.png) and want types to guide your use of the API.
12** Stability
14Bloodhound is alpha at the moment. The library works fine, but I don't want to mislead anyone into thinking the API is final or stable. I wouldn't call the library "complete" or representative of everything you can do in Elasticsearch but being compared to clients in other languages the story here so far is good.
16* Examples
18** Index Operations
20*** Create Index
22#+BEGIN_SRC haskell
24-- Formatted for use in ghci, so there are "let"s in front of the decls.
26-- if you see :{ and :}, they're so you can copy-paste
27-- the multi-line examples into your ghci REPL.
29:set -XDeriveGeneric
30import Database.Bloodhound.Client
31import Data.Aeson
32import Data.Either (Either(..))
33import Data.Maybe (fromJust)
34import Data.Time.Calendar (Day(..))
35import Data.Time.Clock (secondsToDiffTime, UTCTime(..))
36import Data.Text (Text)
37import GHC.Generics (Generic)
38import Network.HTTP.Conduit
39import qualified Network.HTTP.Types.Status as NHTS
41-- no trailing slashes in servers, library handles building the path.
42let testServer = (Server "http://localhost:9200")
43let testIndex = IndexName "twitter"
44let testMapping = MappingName "tweet"
46-- defaultIndexSettings is exported by Database.Bloodhound.Client as well.
47let defaultIndexSettings = IndexSettings (ShardCount 3) (ReplicaCount 2)
49-- createIndex returns IO Reply
51-- response :: Reply, Reply is a synonym for Network.HTTP.Conduit.Response
52response <- createIndex testServer defaultIndexSettings testIndex
54#+END_SRC
56*** Delete Index
58#+BEGIN_SRC haskell
60-- response :: Reply
61response <- deleteIndex testServer testIndex
63-- print response if it was a success
64Response {responseStatus = Status {statusCode = 200, statusMessage = "OK"}
65 , responseVersion = HTTP/1.1
66 , responseHeaders = [("Content-Type", "application/json; charset=UTF-8")
67 , ("Content-Length", "21")]
68 , responseBody = "{\"acknowledged\":true}"
69 , responseCookieJar = CJ {expose = []}
70 , responseClose' = ResponseClose}
72-- if the index to be deleted didn't exist anyway
73Response {responseStatus = Status {statusCode = 404, statusMessage = "Not Found"}
74 , responseVersion = HTTP/1.1
75 , responseHeaders = [("Content-Type", "application/json; charset=UTF-8")
76 , ("Content-Length","65")]
77 , responseBody = "{\"error\":\"IndexMissingException[[twitter] missing]\",\"status\":404}"
78 , responseCookieJar = CJ {expose = []}
79 , responseClose' = ResponseClose}
81#+END_SRC
83*** Refresh Index
85**** Note, you *have* to do this if you expect to read what you just wrote
87#+BEGIN_SRC haskell
89resp <- refreshIndex testServer testIndex
91-- print resp on success
92Response {responseStatus = Status {statusCode = 200, statusMessage = "OK"}
93 , responseVersion = HTTP/1.1
94 , responseHeaders = [("Content-Type", "application/json; charset=UTF-8")
95 , ("Content-Length","50")]
96 , responseBody = "{\"_shards\":{\"total\":10,\"successful\":5,\"failed\":0}}"
97 , responseCookieJar = CJ {expose = []}
98 , responseClose' = ResponseClose}
100#+END_SRC
102** Mapping Operations
104*** Create Mapping
106#+BEGIN_SRC haskell
108-- don't forget imports and the like at the top.
110data TweetMapping = TweetMapping deriving (Eq, Show)
112-- I know writing the JSON manually sucks.
113-- I don't have a proper data type for Mappings yet.
114-- Let me know if this is something you need.
116:{
117instance ToJSON TweetMapping where
118 toJSON TweetMapping =
119 object ["tweet" .=
120 object ["properties" .=
121 object ["location" .=
122 object ["type" .= ("geo_point" :: Text)]]]]
123:}
125resp <- createMapping testServer testIndex testMapping TweetMapping
127#+END_SRC
129*** Delete Mapping
131#+BEGIN_SRC haskell
133resp <- deleteMapping testServer testIndex testMapping
135#+END_SRC
137** Document Operations
139*** Indexing Documents
141#+BEGIN_SRC haskell
143-- don't forget the imports and derive generic setting for ghci
144-- at the beginning of the examples.
146:{
147data Location = Location { lat :: Double
148 , lon :: Double } deriving (Eq, Generic, Show)
150data Tweet = Tweet { user :: Text
151 , postDate :: UTCTime
152 , message :: Text
153 , age :: Int
154 , location :: Location } deriving (Eq, Generic, Show)
156exampleTweet = Tweet { user = "bitemyapp"
157 , postDate = UTCTime
158 (ModifiedJulianDay 55000)
159 (secondsToDiffTime 10)
160 , message = "Use haskell!"
161 , age = 10000
162 , location = Location 40.12 (-71.34) }
164-- automagic (generic) derivation of instances because we're lazy.
165instance ToJSON Tweet
166instance FromJSON Tweet
167instance ToJSON Location
168instance FromJSON Location
169:}
171-- Should be able to toJSON and encode the data structures like this:
172-- λ> toJSON $ Location 10.0 10.0
173-- Object fromList [("lat",Number 10.0),("lon",Number 10.0)]
174-- λ> encode $ Location 10.0 10.0
175-- "{\"lat\":10,\"lon\":10}"
177resp <- indexDocument testServer testIndex testMapping exampleTweet (DocId "1")
179-- print resp on success
180Response {responseStatus =
181 Status {statusCode = 200, statusMessage = "OK"}
182 , responseVersion = HTTP/1.1, responseHeaders =
183 [("Content-Type","application/json; charset=UTF-8"),
184 ("Content-Length","75")]
185 , responseBody = "{\"_index\":\"twitter\",\"_type\":\"tweet\",\"_id\":\"1\",\"_version\":2,\"created\":false}"
186 , responseCookieJar = CJ {expose = []}, responseClose' = ResponseClose}
188#+END_SRC
190*** Deleting Documents
192#+BEGIN_SRC haskell
194resp <- deleteDocument testServer testIndex testMapping (DocId "1")
196#+END_SRC
198*** Getting Documents
200#+BEGIN_SRC haskell
202-- n.b., you'll need the earlier imports. responseBody is from http-conduit
204resp <- getDocument testServer testIndex testMapping (DocId "1")
206-- responseBody :: Response body -> body
207let body = responseBody resp
209-- you have two options, you use decode and just get Maybe (EsResult Tweet)
210-- or you can use eitherDecode and get Either String (EsResult Tweet)
212let maybeResult = decode body :: Maybe (EsResult Tweet)
213-- the explicit typing is so Aeson knows how to parse the JSON.
215-- use either if you want to know why something failed to parse.
216-- (string errors, sadly)
217let eitherResult = decode body :: Either String (EsResult Tweet)
219-- print eitherResult should look like:
220Right (EsResult {_index = "twitter"
221 , _type = "tweet"
222 , _id = "1"
223 , _version = 2
224 , found = Just True
225 , _source = Tweet {user = "bitemyapp"
226 , postDate = 2009-06-18 00:00:10 UTC
227 , message = "Use haskell!"
228 , age = 10000
229 , location = Location {lat = 40.12, lon = -71.34}}})
231-- _source in EsResult is parametric, we dispatch the type by passing in what we expect (Tweet) as a parameter to EsResult.
233-- use the _source record accessor to get at your document
234λ> fmap _source result
235Right (Tweet {user = "bitemyapp"
236 , postDate = 2009-06-18 00:00:10 UTC
237 , message = "Use haskell!"
238 , age = 10000
239 , location = Location {lat = 40.12, lon = -71.34}})
241#+END_SRC
243** Search
245*** Querying
247**** Term Query
249#+BEGIN_SRC haskell
251-- exported by the Client module, just defaults some stuff.
252-- mkSearch :: Maybe Query -> Maybe Filter -> Search
253-- mkSearch query filter = Search query filter Nothing False 0 10
255let query = TermQuery (Term "user" "bitemyapp") Nothing
257-- AND'ing identity filter with itself and then tacking it onto a query
258-- search should be a null-operation. I include it for the sake of example.
259-- <||> (or/plus) should make it into a search that returns everything.
261let filter = IdentityFilter <&&> IdentityFilter
263-- constructing the search object the searchByIndex function dispatches on.
264let search = mkSearch (Just query) (Just filter)
266-- you can also searchByType and specify the mapping name.
267reply <- searchByIndex testServer testIndex search
269let result = eitherDecode (responseBody reply) :: Either String (SearchResult Tweet)
271λ> fmap (hits . searchHits) result
272Right [Hit {hitIndex = IndexName "twitter"
273 , hitType = MappingName "tweet"
274 , hitDocId = DocId "1"
275 , hitScore = 0.30685282
276 , hitSource = Tweet {user = "bitemyapp"
277 , postDate = 2009-06-18 00:00:10 UTC
278 , message = "Use haskell!"
279 , age = 10000
280 , location = Location {lat = 40.12, lon = -71.34}}}]
282#+END_SRC
284*** Filtering
287* Possible future functionality
289** Node discovery and failover
291Might require TCP support.
293** Support for TCP access to Elasticsearch
295Pretend to be a transport client?
297** Bulk cluster-join merge
299Might require making a lucene index on disk with the appropriate format.
301** GeoShapeFilter
303http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-geo-shape-filter.html
305** Geohash cell filter
307http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-geohash-cell-filter.html
309** HasChild Filter
311http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-has-child-filter.html
313** HasParent Filter
315http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-has-parent-filter.html
317** Indices Filter
319http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-indices-filter.html
321** Query Filter
323http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-filter.html
325** Script based sorting
327http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-sort.html#_script_based_sorting
329** Runtime checking for cycles in data structures
331check for n > 1 occurrences in DFS:
333http://hackage.haskell.org/package/stable-maps-0.0.5/docs/System-Mem-StableName-Dynamic.html
335http://hackage.haskell.org/package/stable-maps-0.0.5/docs/System-Mem-StableName-Dynamic-Map.html
337* Photo Origin
339Photo from HA! Designs: https://www.flickr.com/photos/hadesigns/