@@ -0,0 +1,88 @@
1+My goal in working on the
2+[new GHC I/O manager](http://www.serpentine.com/blog/2010/01/22/new-ghc-io-manager-first-benchmark-numbers/)
3+has been to get the Haskell network stack into a state where it could
4+be used to attack high-performance and scalable networking problems,
5+domains in which it has historically been weak.
6+
7+While it's encouraging to have an excellent networking stack (Johan
8+and I now have this thoroughly in hand), the next thing I'd look for
9+is libraries to help build networked applications. One of the
10+fundamental things that such apps need to do well is parse data, be it
11+received from the network or read from files.
12+
13+The Haskell parsing library of first resort has for years been
14+[Parsec](http://www.haskell.org/haskellwiki/Parsec). While other
15+capable libraries exist
16+(e.g. [polyparse](http://hackage.haskell.org/package/polyparse) and
17+[uu-parsinglib](http://hackage.haskell.org/package/uu-parsinglib)),
18+they don't appear to see much use.
19+
20+As appealing as Parsec's API is, it has a few problems:
21+
22+* Parsec 2 is slow, and it has high memory overhead, due to its use of
23+ Haskell's `String` type for tokens. Parsec 3 can use the more
24+ efficient `ByteString` type (which is in any case much more
25+ appropriate for networked applications that deal in octets), but it
26+ achieves this flexibility at the cost of being even slower than
27+ Parsec 2.
28+
29+* Parsec's API demands that all of a parser's input be available at
30+ once. People usually work around this by feeding a Parsec parser
31+ with lazily read data, but lazy I/O is at odds with my goal of
32+ writing solid networked code.
33+
34+What properties should a parsing library for networked applications
35+ideally possess? There are a few obvious desiderata that have been
36+well known for years. For example, it's important to have an appealing
37+API and programming model. Parsec squarely fits this desire.
38+
39+Performance is also a big consideration. Ideally, a parsing library
40+would be fast enough that you wouldn't feel any real need for either
41+of the following:
42+
43+* A few weeks to write an insane hand-bummed parser.
44+
45+* Mechanical parser generators or lexers
46+ (e.g. [happy](http://www.haskell.org/happy/) or
47+ [alex](http://www.haskell.org/alex/)).
48+
49+There are some additional important constraints on a realistic
50+library: it must fit well into a highly concurrent networked world
51+full of unreliable, hostile and incompetent clients.
52+
53+* High concurrency levels demand a low per-connection memory
54+ footprint.
55+
56+* The need to cope with poorly behaved clients requires that
57+ applications must be able to throttle connections that are too busy,
58+ or kill connections that are too slow or attempting to consume too
59+ many server resources. A good parsing library will not get in the
60+ way of these needs.
61+
62+A few years ago, I made a few half-hearted attempts to write a
63+specialised version of Parsec, which I eventually named
64+[Attoparsec](http://hackage.haskell.org/package/attoparsec).
65+
66+I began with a stripped-down Parsec that was specialised to accept
67+`ByteString` input. I then extended the API to allow a parser to
68+consume small chunks of input at a time.
69+
70+Because I wasn't using Attoparsec "in anger" at the time, I made sure
71+that my library worked (more or less), but I was not measuring its
72+performance.
73+
74+In late January of this year, I began to think about using Attoparsec
75+as the parser for a simple HTTP server that I could use to benchmark
76+our new GHC I/O manager code. Clearly, I'd want the parser to perform
77+well, or it would distort my numbers rather badly.
78+
79+By coincidence, [John MacFarlane](http://johnmacfarlane.net/) emailed
80+me around the same time, with disturbing findings: he'd tried
81+Attoparsec, and found its performance to be *terrible*! In fact, it
82+was 4 to 20 times _slower_ than plain Parsec with his experimental
83+parser and test data. Clearly, I had some hard work to look forward
84+to.
85+
86+Happily, that work is now almost complete, and I am pleased with the
87+results. In the next post, I'll have some details of what this all
88+entails.