@@ -25,7 +25,9 @@ Imperative languages (such as C and Python) perform IO by calling a function in
26In this example and others to follow, the task will be to read a filename from the command line, read a URL out of that file, and fetch the contents at that URL.
26In this example and others to follow, the task will be to read a filename from the command line, read a URL out of that file, and fetch the contents at that URL.
28− output = fetchUrl(stripWhitespace(readFile(cliInput())))
29+output = fetchUrl(stripWhitespace(readFile(cliInput())))
30For readability, intermediate variables would be a good idea, but nothing stops you from nesting IO this way.
32For readability, intermediate variables would be a good idea, but nothing stops you from nesting IO this way.
@@ -33,9 +35,11 @@ For readability, intermediate variables would be a good idea, but nothing stops
34Languages that use explicit monads for IO (such as Haskell) perform IO by running IO behind a bind operation, which I'll notate with `<-`. Thus, while you can't put IO _anywhere_, the general structure is a sequence of IO operations.
36Languages that use explicit monads for IO (such as Haskell) perform IO by running IO behind a bind operation, which I'll notate with `<-`. Thus, while you can't put IO _anywhere_, the general structure is a sequence of IO operations.
36− fileName <- cliInput()
39+fileName <- cliInput()
37− url <- readFile(fileName)
40+url <- readFile(fileName)
38− output <- fetchUrl(stripWhitespace(url))
41+output <- fetchUrl(stripWhitespace(url))
40[Tackling the Awkward Squad](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/07/mark.pdf) is the canonical description of monadic IO, and does a great job of explaining how the Haskell community decided to use this as their primary IO style.
44[Tackling the Awkward Squad](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/07/mark.pdf) is the canonical description of monadic IO, and does a great job of explaining how the Haskell community decided to use this as their primary IO style.
@@ -43,22 +47,24 @@ Languages that use explicit monads for IO (such as Haskell) perform IO by runnin
44Languages that use explicit state machines for IO (such as Elm) perform IO by accepting events and producing a list of effects.
48Languages that use explicit state machines for IO (such as Elm) perform IO by accepting events and producing a list of effects.
46− event :: Initialize
51+event :: Initialize
47− | CLIInput text
52+ | CLIInput text
48− | FileInput text
53+ | FileInput text
49− | HTTPResponse body
54+ | HTTPResponse body
50− effect :: CLIInput
55+effect :: CLIInput
51− | ReadFile fileName
56+ | ReadFile fileName
52− | FetchUrl url
57+ | FetchUrl url
53− | Print text
58+ | Print text
54− main(event) {
59+main(event) {
55− switch event {
60+ switch event {
56− Initialize -> [CLIInput ~]
61+ Initialize -> [CLIInput ~]
57− CLIinput fileName -> [ReadFile fileName]
62+ CLIinput fileName -> [ReadFile fileName]
58− FileInput url -> [FetchUrl stripWhitespace(url)]
63+ FileInput url -> [FetchUrl stripWhitespace(url)]
59− HTTPResponse body -> [Print body]
64+ HTTPResponse body -> [Print body]
63The effects produced are not functions but data.
69The effects produced are not functions but data.
@@ -70,39 +76,45 @@ Let's look at another type of program: a simple HTTP server that you can give te
71Imperative:
77Imperative:
73− text = "no text entered yet"
80+text = "no text entered yet"
74− while (true)
81+while (true)
75− cli = readCli()
82+ cli = readCli()
76− if (cli != Null)
83+ if (cli != Null)
77− text = cli
84+ text = cli
78− request = readHttp()
85+ request = readHttp()
79− if (request != Null)
86+ if (request != Null)
80− respond(request,text)
87+ respond(request,text)
84− loop("no text entered yet")
93+loop("no text entered yet")
88− loop :: text -> IO Null
97+loop :: text -> IO Null
90− input <- readInput()
99+ input <- readInput()
91− switch input
100+ switch input
92− CLIInput newText -> loop(newText)
101+ CLIInput newText -> loop(newText)
93− HTTPRequest request ->
102+ HTTPRequest request ->
94− _ <- respond(request,text)
103+ _ <- respond(request,text)
95− loop(text)
104+ loop(text)
97State machine:
107State machine:
99− text = "no text entered yet"
110+text = "no text entered yet"
100− main(text,event) {
111+main(text,event) {
101− switch event {
112+ switch event {
102− CLIInput newText -> [newText, Null]
113+ CLIInput newText -> [newText, Null]
103− HTTPRequest request -> [text, HTTPResponse text]
114+ HTTPRequest request -> [text, HTTPResponse text]
107Again, note the response in the state machine is data, not a function.
119Again, note the response in the state machine is data, not a function.
@@ -155,16 +167,16 @@ However, Arvo is a single-level store — all of its state is permanent. This is
155Further, Arvo and the vanes rarely perform long sequences of IO. Generally, they do one thing and emit some effects, similar to the HTTP server example above. Sometimes they "pass through" a request. For example, the Eyre vane passes HTTP requests to userspace agents or threads. In this case, there is a sequence of IO, which in monadic code would be:
167Further, Arvo and the vanes rarely perform long sequences of IO. Generally, they do one thing and emit some effects, similar to the HTTP server example above. Sometimes they "pass through" a request. For example, the Eyre vane passes HTTP requests to userspace agents or threads. In this case, there is a sequence of IO, which in monadic code would be:
163− request <- receiveRequest()
175+ request <- receiveRequest()
164− agent = lookupAgent(request)
176+ agent = lookupAgent(request)
165− response <- callAgent(agent, request)
177+ response <- callAgent(agent, request)
166− _ <- sendResponse(request, response)
178+ _ <- sendResponse(request, response)
170This is clear code to read, but it leaves open the question of how to respond during the callAgent() function. If another request comes in, can we handle it in parallel? If we need to upgrade ourselves before the agent responds, can we make sure to properly handle the response?
182This is clear code to read, but it leaves open the question of how to respond during the callAgent() function. If another request comes in, can we handle it in parallel? If we need to upgrade ourselves before the agent responds, can we make sure to properly handle the response?
@@ -174,16 +186,16 @@ The naive solution in monadic code is to say that you can only handle one reques
174In a state machine, the naive solution is:
186In a state machine, the naive solution is:
177− state :: (map request agent)
189+state :: (map request agent)
178− main(state, event) {
190+main(state, event) {
179− switch event {
191+ switch event {
180− HTTPRequest request ->
192+ HTTPRequest request ->
181− agent = lookupAgent(request)
193+ agent = lookupAgent(request)
182− [put(state, request, agent), CallAgent agent request]
194+ [put(state, request, agent), CallAgent agent request]
183− AgentResponse request response ->
195+ AgentResponse request response ->
184− [del(state, request), HTTPRespond request response]
196+ [del(state, request), HTTPRespond request response]
189Upgrading this is trivial, since the state (map of requests to outstanding calls to agents) is explicit. It also trivially handles concurrent requests.
201Upgrading this is trivial, since the state (map of requests to outstanding calls to agents) is explicit. It also trivially handles concurrent requests.
@@ -201,61 +213,61 @@ However, userspace sometimes needs to perform long sequences of IO. Let's look a
204− topStories <- fetch(topStoriesUrl)
216+topStories <- fetch(topStoriesUrl)
205− loop(topStories)
217+loop(topStories)
209− loop :: topStories -> IO Null
221+loop :: topStories -> IO Null
210− if topStories is Null
222+ if topStories is Null
211− return Null
223+ return Null
212− comments <- retryLoop(5,head(topStories))
224+ comments <- retryLoop(5,head(topStories))
213− otherComments <- loop(tail(topStories))
225+ otherComments <- loop(tail(topStories))
214− append(comments,otherComments)
226+ append(comments,otherComments)
216− retryLoop :: [n story] -> IO Comments
228+retryLoop :: [n story] -> IO Comments
217− comments <- fetchComments(story)
229+ comments <- fetchComments(story)
218− if comments is HTTPError:
230+ if comments is HTTPError:
219− if n == 0:
231+ if n == 0:
222− retryLoop(n-1, story)
234+ retryLoop(n-1, story)
225State machine:
237State machine:
228− state :: Initial
240+state :: Initial
229− | FetchingTop
241+ | FetchingTop
230− | FetchingStory comments stories retries
242+ | FetchingStory comments stories retries
231− | Done comments
243+ | Done comments
233− main(state, event) {
245+main(state, event) {
234− switch event {
246+ switch event {
235− Initialize ->
247+ Initialize ->
236− assert state == Initial
248+ assert state == Initial
237− [FetchingTop, Fetch topStoriesUrl]
249+ [FetchingTop, Fetch topStoriesUrl]
238− HttpResponse response ->
250+ HttpResponse response ->
239− switch state {
251+ switch state {
240− FetchingTop ->
252+ FetchingTop ->
241− [FetchingStory Null stories 5, FetchComments head(stories)]
253+ [FetchingStory Null stories 5, FetchComments head(stories)]
242− FetchingStory comments stories retries ->
254+ FetchingStory comments stories retries ->
243− if response is HTTPError:
255+ if response is HTTPError:
244− if retries == 0:
256+ if retries == 0:
245− [Initial, Null]
257+ [Initial, Null]
247− [FetchingStory comments stories retries-1,
259+ [FetchingStory comments stories retries-1,
248− FetchComments head(stories)]
260+ FetchComments head(stories)]
250− comments = append(response,comments)
262+ comments = append(response,comments)
251− if tail(stories) is Null:
263+ if tail(stories) is Null:
252− [Done comments, Null]
264+ [Done comments, Null]
254− [FetchingStory append(comments,response) tail(stories) 5,
266+ [FetchingStory append(comments,response) tail(stories) 5,
255− FetchComments head(tail(stories))]
267+ FetchComments head(tail(stories))]
261With practice, the state machine version of this can be written correctly. However, (1) it's verbose, (2) the control flow jumps all over the place, and (3) in practice it's challenging to get exactly right. This is still a small example, and it gets much worse as the length and complexity of the IO sequence grows.
273With practice, the state machine version of this can be written correctly. However, (1) it's verbose, (2) the control flow jumps all over the place, and (3) in practice it's challenging to get exactly right. This is still a small example, and it gets much worse as the length and complexity of the IO sequence grows.
@@ -285,7 +297,7 @@ We're in a bit of a bind, though: Jael has to be a state machine. The solution i
285This thread has a definite purpose: given the most recent block number we already know about, fetch all the PKI transactions since then. The function signature is:
297This thread has a definite purpose: given the most recent block number we already know about, fetch all the PKI transactions since then. The function signature is:
288− syncEthereum oldBlockNumber -> IO [newBlockNumber, newTxs]
300+syncEthereum oldBlockNumber -> IO [newBlockNumber, newTxs]
291Jael runs this thread every five minutes and, if it succeeds, then we update our block number and PKI state. If the thread fails, gets stuck, or we need to upgrade it, we just kill the old thread and start a new one. Again, a few extra HTTP requests don’t matter.
303Jael runs this thread every five minutes and, if it succeeds, then we update our block number and PKI state. If the thread fails, gets stuck, or we need to upgrade it, we just kill the old thread and start a new one. Again, a few extra HTTP requests don’t matter.
@@ -293,24 +305,24 @@ Jael runs this thread every five minutes and, if it succeeds, then we update our
293From Jael's perspective, it looks something like:
305From Jael's perspective, it looks something like:
296− state :: State block txs outstandingThread
308+state :: State block txs outstandingThread
297− main(state,event) {
309+main(state,event) {
298− switch event {
310+ switch event {
299− Initialize -> [State 0 Null Null, StartTimer (now + five minutes)]
311+ Initialize -> [State 0 Null Null, StartTimer (now + five minutes)]
300− TimerFired ->
312+ TimerFired ->
301− effects = If outstandingThread is Null
313+ effects = If outstandingThread is Null
302− then Null
314+ then Null
303− else [Kill outstandingThread]
315+ else [Kill outstandingThread]
304− newId = newThreadId()
316+ newId = newThreadId()
305− [State txs newId,
317+ [State txs newId,
306− append(effects,
318+ append(effects,
307− StartThread newId syncEthereum(block),
319+ StartThread newId syncEthereum(block),
308− StartTimer (now + five minutes))]
320+ StartTimer (now + five minutes))]
309− ThreadFinished Failure -> [State block txs Null, Null]
321+ ThreadFinished Failure -> [State block txs Null, Null]
310− ThreadFinished Success newBlockNumber newTxs ->
322+ ThreadFinished Success newBlockNumber newTxs ->
311− [State newBlockNumber append(txs,newTxs) Null, Null]
323+ [State newBlockNumber append(txs,newTxs) Null, Null]
316In other words, from Jael's perspective, it's just a single IO event that encapsulates arbitrarily complex IO in the thread. A single IO event is easy enough, and we can maintain all our permanency and robustness guarantees, because we know the thread will have one of three results: success, failure, or it hangs. If we set a timeout, hang becomes failure, so there's only two possible results: success or failure. If we handle both of those correctly, we've handled all the possible IO errors that could have occurred.
328In other words, from Jael's perspective, it's just a single IO event that encapsulates arbitrarily complex IO in the thread. A single IO event is easy enough, and we can maintain all our permanency and robustness guarantees, because we know the thread will have one of three results: success, failure, or it hangs. If we set a timeout, hang becomes failure, so there's only two possible results: success or failure. If we handle both of those correctly, we've handled all the possible IO errors that could have occurred.