@@ -51,19 +51,58 @@
5151-- to the agent's event sinks from both sources, so an agent need only concern
5252-- itself with how to respond to its inputs.
5353--
54+-- Some agents may wish to prioritise messages from their mailbox over traffic
55+-- on the management event bus, or vice versa. The 'mxReceive' and
56+-- 'mxReceiveChan' API calls do this for the mailbox and event bus,
57+-- respectively. The /prioritisation/ these APIs offer is simply that the chosen
58+-- data stream will be checked first. No blocking will occur if the chosen
59+-- (prioritised) source is devoid of input messages, instead the agent handling
60+-- code will revert to switching between the alternatives in /round-robin/ as
61+-- usual. This switching (and prioritisation) will take place a limited number
62+-- of times, after which - in order to avoid over-utilising system
63+-- resources - the agent will revert to blocking on the event bus (and ignoring
64+-- its mailbox).
65+--
66+-- Prioritisation then, is a /hint/ about the preference of data source from
67+-- which the next input should be chosen. No guarantee can be made that the
68+-- chosen source will in fact be selected at runtime.
69+--
70+-- [Management API Semantics]
71+--
72+-- The management API provides /no guarantees whatsoever/, viz:
73+--
74+-- * The ordering of messages delivered to the event bus.
75+--
76+-- * The order in which agents will be executed.
77+--
78+-- * Whether messages will be taken from the mailbox first, or the event bus.
79+--
5480-- [Management Data API]
5581--
5682-- Both management agents and clients of the API have access to a variety of
57−-- data storage capabilities, to facilitate publishing and consuming information
58−-- obtained by agents. Each agent is assigned its own data table, which acts
59−-- as a shared map, where the keys are @String@s and the values are
60−-- @Serializable@ datum of whatever type the agent or its clients stores.
83+-- data storage capabilities, to facilitate publishing and consuming useful
84+-- system information. Agents maintain their own internal state privately (via a
85+-- state transformer - see 'mxGetLocal' et al), however it is possible for
86+-- agents to share additional data with each other (and the outside world)
87+-- using /data tables/.
88+--
89+-- Each agent is assigned its own data table, which acts as a shared map, where
90+-- the keys are @String@s and the values are @Serializable@ datum of whatever
91+-- type the agent or its clients stores.
92+--
93+-- Because an agent's /data table/ stores its values in raw 'Message' format,
94+-- it works effectively as an /un-typed dictionary/, into which data of varying
95+-- types can be fed and later retrieved. The upside of this is that different
96+-- keys can be mapped to various types without any additional work on the part
97+-- of the developer. The downside is that the code reading these values must
98+-- know in advance what type(s) to expect, and the API provides no additional
99+-- support for handling that.
61100--
62101-- Publishing is accomplished using the 'mxPublish' and 'mxSet' APIs, whilst
63102-- querying and deletion are handled by 'mxGet', 'mxClear', 'mxPurgeTable' and
64103-- 'mxDropTable' respectively.
65104--
66−-- When management agents terminate, their tables are left in memory despite the
105+-- When a management agent terminates, their tables are left in memory despite
67106-- termination, such that an agent may resume its role (by restarting) or have
68107-- its 'MxAgentId' taken over by another subsequent agent, leaving the data
69108-- originally captured in place.
@@ -89,18 +128,78 @@
89128-- will obtain data from the event bus and process' mailbox in a round robbin
90129-- fashion, i.e., one after the other, changing each time.
91130--
92−-- [Architecture Overview]
93−--
94−-- /Management Extensions/ can be used by infrastructure or application code,
95−-- to provide access to runtime instrumentation data, expose control planes
96−-- and/or provide remote access to external clients.
131+-- [Example Code]
132+--
133+-- What follows is a grossly over-simplified example of a management agent that
134+-- provides a basic name monitoring facility. Whenever a process name is
135+-- registered or unregistered, clients are informed of the fact.
136+--
137+-- > -- simple notification data type
138+-- >
139+-- > data Registration = Reg { added :: Bool
140+-- > , procId :: ProcessId
141+-- > , name :: String
142+-- > }
143+-- >
144+-- > -- start a /name monitoring agent/
145+-- > nameMonitorAgent = do
146+-- > mxAgent (MxAgentId "name-monitor") Set.empty [
147+-- > (mxSink $ \(pid :: ProcessId) -> do
148+-- > mxUpdateState $ Set.insert pid
149+-- > mxReady)
150+-- > , (mxSink $
151+-- > let act =
152+-- > case ev of
153+-- > (MxRegistered p n) -> notify True n p
154+-- > (MxUnRegistered p n) -> notify False n p
155+-- > _ -> return ()
156+-- > act >> mxReady)
157+-- > ]
158+-- > where
159+-- > notify a n p = do
160+-- > Foldable.mapM_ (liftMX . deliver (Reg a n p)) =<< mxGetLocal
161+-- >
162+--
163+-- The client interface (for sending their pid) can take one of two forms:
164+--
165+-- > monitorNames = getSelfPid >>= nsend "name-monitor"
166+-- > monitorNames2 = getSelfPid >>= mxNotify
167+--
168+-- For some real-world examples, see the distributed-process-platform package.
169+--
170+-- [Performance, Stablity and Scalability]
171+--
172+-- /Management Agents/ offer numerous advantages over regular processes:
173+-- broadcast communication with them can have a lower latency, they offer
174+-- simplified messgage (i.e., input type) handling and they have access to
175+-- internal system information that would be otherwise unobtainable.
176+--
177+-- Do not be tempted to implement everything (e.g., the kitchen sink) using the
178+-- management API though. There are overheads associated with management agents
179+-- which is why they're presented as tools for consuming low level system
180+-- information, instead of as /application level/ development tools.
181+--
182+-- Agents that rely heavily on a busy mailbox can cause the management event
183+-- bus to backlog un-GC'ed data, leading to increased heap space. Producers that
184+-- do not take care to avoid passing unevaluated thunks to the API can crash
185+-- /all/ the agents in the system. Agents are not monitored or managed in any
186+-- way, and those that crash will not be restarted.
187+--
188+-- The management event bus can receive a great deal of traffic. Every time
189+-- a message is sent and/or received, an event is passed to the agent controller
190+-- and broadcast to all agents (plus the trace controller, if tracing is enabled
191+-- for the node). This is already a significant overhead - though profiling and
192+-- benchmarks have demonstrated that it does not adversely affect performance
193+-- if few agents are installed. Agents will typically use more cycles than plain
194+-- processes, since they perform additional work: selecting input data from both
195+-- the event bus /and/ their own mailboxes, plus searching through the set of
196+-- event sinks (for each agent) to determine the right handler for the event.
197+--
198+-- Each management agent requires not only its own @Process@ (in which the agent
199+-- code is run), but also a peer process that provides its /data table/. These
200+-- data tables also have to be coordinated and manaaged on each agent's behalf.
97201--
98−-- The management infrastructure is broken down into three components, which
99−-- roughly correspond to the modules in which they're implemented:
100−--
101−-- 1. @Agent@ - registration and definition of /Management Agents/
102−-- 2. @Table@ - provides an API for publishing /instrumentation data/
103−-- 3. @Trace@ - provides an API for tracing/debugging capabilities
202+-- [Architecture Overview]
104203--
105204-- The architecture of the management event bus is internal and subject to
106205-- change without prior notice. The description that follows is provided for
@@ -131,7 +230,7 @@
131230-- The management control plane is driven by a shared broadcast channel, which
132231-- is written to by the agent controller and subscribed to by all agent
133232-- processes. Agents are spawned as regular processes, whose primary
134−-- implementation (i.e., their "server loop") is responsible for consuming
233+-- implementation (i.e., /server loop/) is responsible for consuming
135234-- messages from both the broadcast channel and their own mailbox. Once
136235-- consumed, messages are applied to the agent's /event sinks/ until one
137236-- matches the input, at which point it is applied and the loop continues.
@@ -241,9 +340,8 @@ mxNotify msg = do
241340
242341-- | Publish an arbitrary @Message@ as a property in the management database.
243342--
244−-- For publishing @Serializable@ data, use 'mxSet' instead. For publishing
245−-- properties efficiently within an agent, use 'mxAgentPublish' instead,
246−-- since it will be more efficient in that context.
343+-- For publishing @Serializable@ data, use 'mxSet' instead.
344+--
247345mxPublish :: MxAgentId -> String -> Message -> Process ()
248346mxPublish a k v = Table.set k v (Table.MxForAgent a)
249347