git.iko.soy/typeable/attoparsec
on 0.9.1.1 help sign in
typeable/attoparsec/doc/attoparsec-rewired-2.md
93 lines 4.7 kB history raw
1In my
2[first of this pair of articles](http://www.serpentine.com/blog/2010/03/03/whats-in-a-parsing-library-1/),
3I laid out some of the qualities I've been looking for in a parsing
4library.
6Before I dive back into detail, I want to show off some numbers. The
7new Attoparsec code is _fast_.
9![Performance](http://chart.apis.google.com/chart?cht=bvs&chs=340x200&chd=t:260,0,0,0,0|0,471,0,0,0|0,0,17037,0,0|0,0,0,23753,0|0,0,0,0,36753&chds=0,40000&chco=4D89F9|894DF9|F94D89|4DB989|1969FD&chdl=260+ms:+http_parser|471+ms:+Attoparsec|17037+ms:+Parsec+3+CPS|23753+ms:+Lazy+Parsec+3+CPS|36753+ms:+Parsec+3&chxt=y&chxl=0:||10|20|30|40&chtt=Time+to+parse+45,668|HTTP+GET+requests)
11What did I benchmark? I captured some real HTTP GET requests from a
12live public web server, averaging 431 bytes per request. I chucked
13them into a file, and measured the time needed to parse the entire
14contents of the file with the following libraries:
16* Ryan Dahl's [http-parser](http://github.com/ry/http-parser) library,
17 which is 1,672 lines of hand-rolled C craziness. Its heritage seems
18 to be closely based on the Ragel-generated parser used by Mongrel.
19 This library is a fair approximation to about as fast as you can
20 get, since it's been tuned for just one purpose. I wrote a small,
21 but reasonably realistic,
22 [driver program](http://bitbucket.org/bos/attoparsec/src/tip/examples/rfc2616.c)
23 to wire it up to file-based data, adding another 210 lines of code.
25* An Attoparsec-based
26 [HTTP request parser](http://bitbucket.org/bos/attoparsec/src/tip/examples/RFC2616.hs),
27 54 lines long, with about 30 lines of
28 [driver program](http://bitbucket.org/bos/attoparsec/src/tip/examples/TestRFC2616.hs).
29 (Attoparsec itself is about 900 lines of code.)
31* Several
32 [Parsec-3-based parsers](http://bitbucket.org/bos/attoparsec/src/tip/examples/Parsec_RFC2616.hs),
33 which are almost identical in length to the Attoparsec-based
34 version.
36The Parsec 3 parsers come in three varieties:
38* The fastest uses a patch that Antoine Latter wrote to switch Parsec
39 3's internal machinery over to using continuation passing style
40 (CPS). This parser uses `ByteString` for input, and reads the
41 entire 18.8MB file in one chunk.
43* Next is the same parser, using lazy `ByteString` I/O to read the
44 file in 64KB chunks. This costs about 50% in performance, but is
45 almost mandatory to maintain a sensible footprint on large inputs.
47* In last place is the official version of Parsec 3, reading the input
48 in one chunk. (Reading lazily still costs an additional 50%, but I
49 didn't want to further clutter the chart with more big numbers.)
51What's interesting to me is that the tiny Attoparsec-based parser,
52which is more or less a transliteration of the relevant parts of
53[RFC 2616](http://www.w3.org/Protocols/rfc2616/rfc2616.html), is so
54fast.
56I went back and remeasured performance of the Attoparsec and C parsers
57on a larger data set (295,568 URLs), and got these numbers:
59* Attoparsec: 2.889 seconds, or 102,308 requests/second
61* C: 1.614 seconds, or 183,128 requests/second
63That clocks the Attoparsec-based parser at about 56% the speed of the
64C parser. Not bad, given that it's about 3.2% the number of lines of
65code!
67Of course there are tradeoffs involved here.
69* Parsec 3 emits much more friendly error messages, and can handle
70 many different input types. Attoparsec, being aimed at
71 plumbing-oriented network protocols, considers friendly error
72 messages to not be worth the effort, and is specialised to the
73 arrays of bytes you get straight off the network.
75* Parsec 3 requires all of its input to be available when the parser
76 is run (either in one large chunk or via lazy I/O). If Attoparsec
77 has insufficient data to return a complete result, it hands back a
78 continuation that you provide extra data to. This eliminates the
79 need for lazy I/O and any additional buffering, and makes for a
80 beautiful, pure API that doesn't care what its input source is.
82The memory footprint of the Attoparsec-based parser is small: it will
83run in 568KB of heap on my 64-bit laptop. The smallest heap size that
84the Parsec 3 parser can survive in isn't all that much larger: with
85lazily read input, it will run in a 750KB heap.
87Overall, this is yet another instance where a little careful attention
88to performance yields very exciting results. Personally, I'd be quite
89happy to trade a 97% reduction in code size for such a small
90performance hit, especially given the clarity, ease of use, and
91flexibility of the resulting code. (The `http_parser` API is frankly
92not so much fun to use, even though I completely understand the
93motivation behind it.)