git.iko.soy/typeable/bloodhound
on 1aea945 help sign in
typeable/bloodhound/README.org
258 lines 8.3 kB history raw
1* Bloodhound
3#+CAPTION: Bloodhound (dog)
4[[./bloodhound.jpg]]
6* Elasticsearch client and query DSL for Haskell
8** Why?
10Because you're tired of obnoxious errors like [this](http://i.imgur.com/FKtZYIP.png) and want types to guide your use of the API.
12** Stability
14Bloodhound is alpha at the moment. The library works fine, but I don't want to mislead anyone into thinking the API is final or stable. I wouldn't call the library "complete" or representative of everything you can do in Elasticsearch but being compared to clients in other languages the story here so far is good.
16* Examples
18** Index Operations
20*** Create Index
22#+BEGIN_SRC haskell
24-- Formatted for use in ghci, so there are "let"s in front of the decls.
26-- :set -XDeriveGeneric
27import Database.Bloodhound.Client
28import Data.Aeson
29import Data.Either (Either(..))
30import Data.Maybe (fromJust)
31import Data.Time.Calendar (Day(..))
32import Data.Time.Clock (secondsToDiffTime, UTCTime(..))
33import Data.Text (Text)
34import GHC.Generics (Generic)
35import Network.HTTP.Conduit
36import qualified Network.HTTP.Types.Status as NHTS
38-- no trailing slashes in servers, library handles building the path.
39let testServer = (Server "http://localhost:9200")
40let testIndex = IndexName "twitter"
41let testMapping = MappingName "tweet"
43-- defaultIndexSettings is exported by Database.Bloodhound.Client as well.
44let defaultIndexSettings = IndexSettings (ShardCount 3) (ReplicaCount 2)
46-- createIndex returns IO Reply
48-- response :: Reply, Reply is a synonym for Network.HTTP.Conduit.Response
49response <- createIndex testServer defaultIndexSettings testIndex
51#+END_SRC
53*** Delete Index
55#+BEGIN_SRC haskell
57-- response :: Reply
58response <- deleteIndex testServer testIndex
60-- print response if it was a success
61Response {responseStatus = Status {statusCode = 200, statusMessage = "OK"}, responseVersion = HTTP/1.1, responseHeaders = [("Content-Type","application/json; charset=UTF-8"),("Content-Length","21")], responseBody = "{\"acknowledged\":true}", responseCookieJar = CJ {expose = []}, responseClose' = ResponseClose}
63-- if the index to be deleted didn't exist anyway
64Response {responseStatus = Status {statusCode = 404, statusMessage = "Not Found"}, responseVersion = HTTP/1.1, responseHeaders = [("Content-Type","application/json; charset=UTF-8"),("Content-Length","65")], responseBody = "{\"error\":\"IndexMissingException[[twitter] missing]\",\"status\":404}", responseCookieJar = CJ {expose = []}, responseClose' = ResponseClose}
66#+END_SRC
68*** Refresh Index
70**** Note, you *have* to do this if you expect to read what you just wrote
72#+BEGIN_SRC haskell
74resp <- refreshIndex testServer testIndex
76-- print resp on success
77Response {responseStatus = Status {statusCode = 200, statusMessage = "OK"}, responseVersion = HTTP/1.1, responseHeaders = [("Content-Type","application/json; charset=UTF-8"),("Content-Length","50")], responseBody = "{\"_shards\":{\"total\":10,\"successful\":5,\"failed\":0}}", responseCookieJar = CJ {expose = []}, responseClose' = ResponseClose}
79#+END_SRC
81** Mapping Operations
83*** Create Mapping
85#+BEGIN_SRC haskell
87-- don't forget imports and the like at the top.
89data TweetMapping = TweetMapping deriving (Eq, Show)
91-- I know writing the JSON directly/manually sucks.
92-- I don't have a real data type for Mappings yet.
93-- Let me know if this is something you need.
95:{
96instance ToJSON TweetMapping where
97 toJSON TweetMapping =
98 object ["tweet" .=
99 object ["properties" .=
100 object ["location" .=
101 object ["type" .= ("geo_point" :: Text)]]]]
102:}
104resp <- createMapping testServer testIndex testMapping TweetMapping
106#+END_SRC
108*** Delete Mapping
110#+BEGIN_SRC haskell
112resp <- deleteMapping testServer testIndex testMapping
114#+END_SRC
116** Document Operations
118*** Indexing Documents
120#+BEGIN_SRC haskell
122-- don't forget the imports and derive generic setting for ghci at the beginning of the examples.
124:{
125data Location = Location { lat :: Double
126 , lon :: Double } deriving (Eq, Generic, Show)
128data Tweet = Tweet { user :: Text
129 , postDate :: UTCTime
130 , message :: Text
131 , age :: Int
132 , location :: Location } deriving (Eq, Generic, Show)
134exampleTweet = Tweet { user = "bitemyapp"
135 , postDate = UTCTime
136 (ModifiedJulianDay 55000)
137 (secondsToDiffTime 10)
138 , message = "Use haskell!"
139 , age = 10000
140 , location = Location 40.12 (-71.34) }
142-- automagic (generic) derivation of instances because we're lazy.
143instance ToJSON Tweet
144instance FromJSON Tweet
145instance ToJSON Location
146instance FromJSON Location
147:}
149-- Should be able to toJSON and encode the data structures like this:
150-- λ> toJSON $ Location 10.0 10.0
151-- Object fromList [("lat",Number 10.0),("lon",Number 10.0)]
152-- λ> encode $ Location 10.0 10.0
153-- "{\"lat\":10,\"lon\":10}"
155resp <- indexDocument testServer testIndex testMapping exampleTweet (DocId "1")
157-- print resp on success
158Response {responseStatus = Status {statusCode = 200, statusMessage = "OK"}, responseVersion = HTTP/1.1, responseHeaders = [("Content-Type","application/json; charset=UTF-8"),("Content-Length","75")], responseBody = "{\"_index\":\"twitter\",\"_type\":\"tweet\",\"_id\":\"1\",\"_version\":2,\"created\":false}", responseCookieJar = CJ {expose = []}, responseClose' = ResponseClose}
160#+END_SRC
162*** Deleting Documents
164#+BEGIN_SRC haskell
166resp <- deleteDocument testServer testIndex testMapping (DocId "1")
168#+END_SRC
170*** Getting Documents
172#+BEGIN_SRC haskell
174-- n.b., you'll need the earlier imports. responseBody is from http-conduit
176resp <- getDocument testServer testIndex testMapping (DocId "1")
178-- responseBody :: Response body -> body
179let body = responseBody resp
181-- you have two options, you use decode and just get Maybe (EsResult Tweet)
182-- or you can use eitherDecode and get Either String (EsResult Tweet)
184let maybeResult = decode body :: Maybe (EsResult Tweet)
185-- the explicit typing is so Aeson knows how to parse the JSON.
187-- use either if you want to know why something failed to parse.
188-- (string errors, sadly)
189let eitherResult = decode body :: Either String (EsResult Tweet)
191-- print eitherResult should look like:
192Right (EsResult {_index = "twitter", _type = "tweet", _id = "1", _version = 2, found = Just True, _source = Tweet {user = "bitemyapp", postDate = 2009-06-18 00:00:10 UTC, message = "Use haskell!", age = 10000, location = Location {lat = 40.12, lon = -71.34}}})
194-- _source in EsResult is parametric, we dispatch the type by passing in what we expect (Tweet) as a parameter to EsResult.
196#+END_SRC
198** Search
200*** Querying
203*** Filtering
206* Possible future functionality
208** Node discovery and failover
210Might require TCP support.
212** Support for TCP access to Elasticsearch
214Pretend to be a transport client?
216** Bulk cluster-join merge
218Might require making a lucene index on disk with the appropriate format.
220** GeoShapeFilter
222http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-geo-shape-filter.html
224** Geohash cell filter
226http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-geohash-cell-filter.html
228** HasChild Filter
230http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-has-child-filter.html
232** HasParent Filter
234http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-has-parent-filter.html
236** Indices Filter
238http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-indices-filter.html
240** Query Filter
242http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-filter.html
244** Script based sorting
246http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-sort.html#_script_based_sorting
248** Runtime checking for cycles in data structures
250check for n > 1 occurrences in DFS:
252http://hackage.haskell.org/package/stable-maps-0.0.5/docs/System-Mem-StableName-Dynamic.html
254http://hackage.haskell.org/package/stable-maps-0.0.5/docs/System-Mem-StableName-Dynamic-Map.html
256* Photo Origin
258[Photo from HA! Designs](https://www.flickr.com/photos/hadesigns/)