git.iko.soy/distributed-process
on master help sign in
Avoid racy blocking behavior in closeLocalNode.

When a process dies, it tries to acquire the localState mvar while masking exceptions uninterruptibly. In turn, closeLocalNode tries to kill processes while holding the localState mvar, thus causing a deadlock. This patch kills processes only after refilling the mvar.

commit1e1636ff49695e51c7e03a46ebc0a22c0cdbad25 authorFacundo Domínguez <facundo.dominguez@tweag.io> dateTue 13 Oct 2015, 15:53 (11 years ago) parent35e2788
1 file changed+13−9
src/Control/Distributed/Process/Node.hs+13−9
src/Control/Distributed/Process/Node.hs+13−9
@@ -326,16 +326,20 @@ startServiceProcesses node = do
326326-- | Force-close a local node, killing all processes on that node.
327327closeLocalNode :: LocalNode -> IO ()
328328closeLocalNode node = do
329− modifyMVar_ (localState node) $ \st -> case st of
329+ -- Kill processes after refilling the mvar. Otherwise, there is potential for
330+ -- deadlock as a dying process tries to get the mvar while masking exceptions
331+ -- uninterruptibly.
332+ join $ modifyMVar (localState node) $ \st -> case st of
330333 LocalNodeValid vst -> do
331− forM_ (vst ^. localProcesses) $ \lproc ->
332− -- Semantics of 'throwTo' guarantee that target thread will get
333− -- delivered an exception. Therefore, target thread will be killed
334− -- eventually and that's as good as we can do. No need to wait for
335− -- thread to actually finish dying.
336− killThread (processThread lproc)
337− return LocalNodeClosed
338− LocalNodeClosed -> return LocalNodeClosed
334+ return ( LocalNodeClosed
335+ , forM_ (vst ^. localProcesses) $ \lproc ->
336+ -- Semantics of 'throwTo' guarantee that target thread will get
337+ -- delivered an exception. Therefore, target thread will be
338+ -- killed eventually and that's as good as we can do. No need
339+ -- to wait for thread to actually finish dying.
340+ killThread (processThread lproc)
341+ )
342+ LocalNodeClosed -> return (LocalNodeClosed, return ())
339343 -- This call will have the effect of shutting down the NC as well (see
340344 -- 'createBareLocalNode').
341345 NT.closeEndPoint (localEndPoint node)