@@ -1,10 +1,9 @@
1−-- | This module provides the low-level API to running Cloud Haskell on
1+-- | This module provides the API for running Cloud Haskell on Microsoft Azure
2−-- Microsoft Azure virtual machines (<http://www.windowsazure.com>). Virtual
2+-- virtual machines (<http://www.windowsazure.com>). Virtual machines within an
3−-- machines within an Azure cloud service can talk to each other directly using
3+-- Azure cloud service can talk to each other directly using standard Cloud
4−-- standard Cloud Haskell primitives (using TCP/IP under the hood); to talk to
4+-- Haskell primitives (using TCP/IP under the hood); to talk to the remote
5−-- the remote machines from your local machine you can use the primitives
5+-- machines from your local machine you can use the primitives provided in this
6−-- provided in this module (which use ssh under the hood). It looks something
6+-- module (which use ssh under the hood). It looks something like
10-- > ( ` )_
9-- > ( ` )_
@@ -19,11 +18,6 @@
19-- > | | Local machine
18-- > | | Local machine
22−-- In this module we concentrate on the link between the local machine to the
23−-- remote machines. "Control.Distributed.Process.Backend.Azure.Process" provides
24−-- a higher-level interface that can be used in code that runs on the remote
27-- /NOTE/: It is unfortunate that the local machine cannot talk to the remote
21-- /NOTE/: It is unfortunate that the local machine cannot talk to the remote
28-- machine using the standard Cloud Haskell primitives. In an ideal world, we
22-- machine using the standard Cloud Haskell primitives. In an ideal world, we
29-- could just start a Cloud Haskell node on the local machine, too.
23-- could just start a Cloud Haskell node on the local machine, too.
@@ -66,6 +60,17 @@
66-- /Connect to Existing Virtual Machine/ instead and then select the first VM
60-- /Connect to Existing Virtual Machine/ instead and then select the first VM
67-- you created.
61-- you created.
63+-- Once your virtual machines have been set up, you have to make sure that the
64+-- user you created when you created the VM can ssh from any virtual machine to
65+-- any other using public key authentication. Moreover, you have to make sure
66+-- that @libssh2@ is installed; if you are using the Ubuntu image we recommend
67+-- you can install @libssh2@ using
69+-- > sudo apt-get install libssh2-1
71+-- (TODO: if you don't install libssh2 things will break without a clear error
69-- In these notes, we assume three virtual machines called @CHDemo1@,
74-- In these notes, we assume three virtual machines called @CHDemo1@,
70-- @CHDemo2@, and @CHDemo3@, all part of the @CloudHaskellDemo@ cloud service.
75-- @CHDemo2@, and @CHDemo3@, all part of the @CloudHaskellDemo@ cloud service.
@@ -136,6 +141,274 @@
136-- (here @CloudHaskellDemo.cloudapp.net@) will also resolve to this (V)IP
141-- (here @CloudHaskellDemo.cloudapp.net@) will also resolve to this (V)IP
137-- address. To login to individual machines (through SSH) you will need to use
142-- address. To login to individual machines (through SSH) you will need to use
138-- the specific port mentioned under INPUT ENDPOINTS.
143-- the specific port mentioned under INPUT ENDPOINTS.
145+-- [Overview of the API]
147+-- The Azure 'Backend' provides low-level functionality for interacting with
148+-- Azure virtual machines. 'findVMs' finds all currently available virtual
149+-- machines; 'copyToVM' copies the executable to a specified VM (recall that
150+-- all VMs, as well as the local machine, are assumed to run the same OS so
151+-- that they can all run the same binary), and 'checkMD5' checks the MD5 hash
152+-- of the executable on a remote machine.
154+-- 'callOnVM' and 'spawnOnVM' deal with setting up Cloud Haskell nodes.
155+-- 'spawnOnVM' takes a virtual machine and a port number, as well as a
156+-- @RemoteProcess ()@, starts the executable on the remote node, sets up a new
157+-- Cloud Haskell node, and then runs the specified process. The Cloud Haskell
158+-- node will be shut down when the given process terminates. 'RemoteProcess' is
161+-- > type RemoteProcess a = Closure (Backend -> Process a)
163+-- (If you don't know what a 'Closure' is you should read
164+-- "Control.Distributed.Process.Closure".) 'spawnOnVM' terminates once the
165+-- Cloud Haskell node has been set up.
167+-- 'callOnVM' is similar to 'spawnOnVM', but it takes a /pair/ of processes:
168+-- one to run on the remote host (on a newly created Cloud Haskell node), and
169+-- one to run on the local machine. In this case, the new Cloud Haskell node
170+-- will be terminated when the /local/ process terminates. 'callOnVM' is useful
171+-- because the remote process and the local process can communicate through a
172+-- set of primitives provided in this module ('localSend', 'localExpect', and
173+-- 'remoteSend' -- there is no 'remoteExpect'; instead the remote process can
174+-- use the standard Cloud Haskell 'expect' primitive).
176+-- [First Example: Echo]
178+-- The @echo@ demo starts a new Cloud Haskell node, waits for input from the
179+-- user on the local machine, sends this to the remote machine. The remote
180+-- machine will echo this back; the local machine will wait for the echo, show
181+-- the echo, and repeat.
183+-- Before you can try it you will first need to copy the executable to the remote server:
185+-- > cloud-haskell-azure-echo install \
186+-- > --subscription-id <<your subscription ID> \
187+-- > --certificate /path/to/credentials.x509 \
188+-- > --private /path/to/credentials.private \
189+-- > --user <<remote username>> \
190+-- > --cloud-service <<Cloud Service name>> \
191+-- > --virtual-machine <<Virtual Machine name>>
193+-- (If you leave out the @--virtual-machine@ argument the binary will be copied
194+-- to every virtual machine in the specified cloud service). Once installed,
195+-- you can run it as follows:
197+-- > cloud-haskell-azure-echo run \
198+-- > --subscription-id <<your subscription ID> \
199+-- > --certificate /path/to/credentials.x509 \
200+-- > --private /path/to/credentials.private \
201+-- > --user <<remote username>> \
202+-- > --cloud-service <<Cloud Service name>> \
203+-- > --virtual-machine <<Virtual Machine name>> \
206+-- > CHDemo1: # Everything I type will be echoed back
207+-- > Echo: Everything I type will be echoed back
208+-- > # Until I enter a blank line
209+-- > Echo: Until I enter a blank line
212+-- "Control.Distributed.Process.Backend.Azure.GenericMain" provides a generic
213+-- main function that you can to structure your code. It provides command line
214+-- arguments such as the ones we saw in section /Testing the Setup/, it
215+-- initializes the Azure backend, and it takes care of running your code on the
216+-- remote machines. You don't have to use 'genericMain' if you prefer not to,
217+-- but then it will be your own responsibility to initialize Azure and to make
218+-- sure that your executable does the right thing when it's invoked on the
219+-- remote node. In these notes we will assume that you will use 'genericMain'.
221+-- The full @echo@ demo is
223+-- > {-# LANGUAGE TemplateHaskell #-}
225+-- > import System.IO (hFlush, stdout)
226+-- > import Control.Monad (unless, forever)
227+-- > import Control.Monad.IO.Class (liftIO)
228+-- > import Control.Distributed.Process (Process, expect)
229+-- > import Control.Distributed.Process.Closure (remotable, mkClosure)
230+-- > import Control.Distributed.Process.Backend.Azure
232+-- > , ProcessPair(..)
239+-- > import Control.Distributed.Process.Backend.Azure.GenericMain (genericMain)
241+-- > echoRemote :: () -> Backend -> Process ()
242+-- > echoRemote () _backend = forever $ do
244+-- > remoteSend (str :: String)
246+-- > remotable ['echoRemote]
248+-- > echoLocal :: LocalProcess ()
250+-- > str <- liftIO $ putStr "# " >> hFlush stdout >> getLine
251+-- > unless (null str) $ do
253+-- > liftIO $ putStr "Echo: " >> hFlush stdout
254+-- > echo <- localExpect
255+-- > liftIO $ putStrLn echo
259+-- > main = genericMain __remoteTable callable spawnable
261+-- > callable :: String -> IO (ProcessPair ())
262+-- > callable "echo" = return $ ProcessPair ($(mkClosure 'echoRemote) ()) echoLocal
263+-- > callable _ = error "callable: unknown"
265+-- > spawnable :: String -> IO (RemoteProcess ())
266+-- > spawnable _ = error "spawnable: unknown"
268+-- 'genericMain' expects three arguments: the first is the standard
269+-- '__remoteTable' argument familiar from
270+-- "Control.Distributed.Process.Closure"; the second and third should map
271+-- strings to process pairs (for use with 'callOnVM') or remote processes (for
272+-- use with 'spawnOnVM') respectively.
274+-- When you invoke the @echo@ demo with @--closure echo@ the 'genericMain'
275+-- function calls 'callOnVM' with the process pair consisting of 'echoRemote'
276+-- and 'echoLocal'. Hopefully the definition of these two functions is
279+-- [Second Example: Ping]
281+-- The second example differs from the @echo@ demo in that it uses both
282+-- 'callable' ('callOnVM') and 'spawnable' ('spawnOnVM'). It uses the latter to
283+-- install a ping server which keeps running in the background; it uses the
284+-- former to run a ping client which sends a request to the ping server and
285+-- outputs the response. As with the @echo@ server, we must first copy the
288+-- > cloud-haskell-azure-ping install \
289+-- > --subscription-id <<your subscription ID> \
290+-- > --certificate /path/to/credentials.x509 \
291+-- > --private /path/to/credentials.private \
292+-- > --user <<remote username>> \
293+-- > --cloud-service <<Cloud Service name>> \
298+-- Now we can start the ping server on every virtual machine in the cloud
299+-- service (to install it to a single virtual machine only, pass the
300+-- @--virtual-machine@ argument):
302+-- > cloud-haskell-azure-ping run \
303+-- > --subscription-id <<your subscription ID> \
304+-- > --certificate /path/to/credentials.x509 \
305+-- > --private /path/to/credentials.private \
306+-- > --user <<remote username>> \
307+-- > --cloud-service <<Cloud Service name>> \
309+-- > --closure server
315+-- Finally, we can run the ping client:
317+-- > cloud-haskell-azure-ping run
318+-- > --subscription-id <<your subscription ID> \
319+-- > --certificate /path/to/credentials.x509 \
320+-- > --private /path/to/credentials.private \
321+-- > --user <<remote username>> \
322+-- > --cloud-service <<Cloud Service name>> \
324+-- > --closure client
325+-- > CHDemo3: Ping server at pid://10.119.182.127:8080:0:2 ok
326+-- > CHDemo2: Ping server at pid://10.59.238.125:8080:0:2 ok
327+-- > CHDemo1: Ping server at pid://10.59.224.122:8080:0:2 ok
329+-- Note that we must pass a different port number, because the client will run
330+-- within its own Cloud Haskell instance.
332+-- The code for the ping server is similar to the echo server, but demonstrates
333+-- both 'callable' and 'spawnable' and shows one way to discover nodes.
335+-- > {-# LANGUAGE TemplateHaskell #-}
337+-- > import Data.Binary (encode, decode)
338+-- > import Control.Monad (forever)
339+-- > import Control.Monad.IO.Class (liftIO)
340+-- > import Control.Exception (try, IOException)
341+-- > import Control.Distributed.Process
349+-- > , ProcessMonitorNotification(..)
351+-- > import Control.Distributed.Process.Closure (remotable, mkClosure)
352+-- > import Control.Distributed.Process.Backend.Azure
354+-- > , ProcessPair(..)
360+-- > import Control.Distributed.Process.Backend.Azure.GenericMain (genericMain)
361+-- > import qualified Data.ByteString.Lazy as BSL (readFile, writeFile)
363+-- > pingServer :: () -> Backend -> Process ()
364+-- > pingServer () _backend = do
365+-- > us <- getSelfPid
366+-- > liftIO $ BSL.writeFile "pingServer.pid" (encode us)
371+-- > pingClientRemote :: () -> Backend -> Process ()
372+-- > pingClientRemote () _backend = do
373+-- > mPingServerEnc <- liftIO $ try (BSL.readFile "pingServer.pid")
374+-- > case mPingServerEnc of
376+-- > remoteSend $ "Ping server not found: " ++ show (err :: IOException)
377+-- > Right pingServerEnc -> do
378+-- > let pingServerPid = decode pingServerEnc
379+-- > pid <- getSelfPid
380+-- > _ref <- monitor pingServerPid
381+-- > send pingServerPid pid
382+-- > gotReply <- receiveWait
383+-- > [ match (\() -> return True)
384+-- > , match (\(ProcessMonitorNotification {}) -> return False)
387+-- > then remoteSend $ "Ping server at " ++ show pingServerPid ++ " ok"
388+-- > else remoteSend $ "Ping server at " ++ show pingServerPid ++ " failure"
390+-- > remotable ['pingClientRemote, 'pingServer]
392+-- > pingClientLocal :: LocalProcess ()
393+-- > pingClientLocal = localExpect >>= liftIO . putStrLn
396+-- > main = genericMain __remoteTable callable spawnable
398+-- > callable :: String -> IO (ProcessPair ())
399+-- > callable "client" = return $ ProcessPair ($(mkClosure 'pingClientRemote) ()) pingClientLocal
400+-- > callable _ = error "callable: unknown"
402+-- > spawnable :: String -> IO (RemoteProcess ())
403+-- > spawnable "server" = return $ ($(mkClosure 'pingServer) ())
404+-- > spawnable _ = error "spawnable: unknown"
406+-- The ping server stores its PID in a file, which the ping client attempts to
407+-- read. This kind of pattern is typical, and is provided for in the high-level API.
409+-- [Using the High-Level API]
411+-- TODO: Does not yet exist.
139module Control.Distributed.Process.Backend.Azure
412module Control.Distributed.Process.Backend.Azure
140 ( -- * Initialization
413 ( -- * Initialization
141 Backend(..)
414 Backend(..)