@@ -1,8 +1,8 @@
11# Talk architecture
22
3−This document is complemented by talk's source code. Definitions of data structures will be provided where useful. Descriptions of processes will be accompanied by diagrams describing this flow in code arms. [@TODO? For more detailed explanations, see the implementation overview at the end of this document.]
3+This document is complemented by talk's source code, but doesn't require it. Definitions of data structures will be provided where useful. Descriptions of processes will be accompanied by diagrams describing this flow in code arms. For an overview of the different cores and their most important arms, see the end of this document.
44
5−***Current implementation*** *remarks mention where the [current implementation](https://github.com/Fang-/arvo/tree/talk-split) differs from what is described, and why changing that to match this document would be desirable.*
5+>**Current implementation** remarks mention where the [current implementation](https://github.com/Fang-/arvo/tree/talk-split) differs from what is described, and why changing that to match this document would be desirable.
66
77
88## Overview
@@ -11,7 +11,7 @@ At the time of writing, talk is Urbit's biggest user-facing application, and fre
1111
1212Official Urbit terminology calls these the `guardian` (like a unix daemon) and `agent`. You may also view them as your personal server and client. To be more semantically appropriate for talk, we're calling them `broker` and `reader` respectively.
1313
14−Everyone (per identity) has a single talk broker that does all the heavy lifting. Connected to it can be any number of readers, whose task it is to present the user with an interface for interacting with talk. Brokers and readers will only interact with each other if they are in the same team. That is, if they belong to the same identity, with moons counting as they planets.
14+Everyone (per identity) has a single talk broker that does all the heavy lifting. Connected to it can be any number of readers, whose task it is to present the user with an interface for interacting with talk. Brokers and readers will only interact with each other if they are in the same team. That is, if they belong to the same identity, with moons counting as their planets.
1515
1616### Partners
1717
@@ -26,7 +26,7 @@ A `partner` is either a `station` (a "chat room" hosted on a ship, represented a
2626
2727The broker is responsible for all primary talk functionality, everything that happens "under the hood". It can send, receive and store messages, keeps track of presence lists and configurations, and manages the stations you've created.
2828
29−Note: *You may see `lowdown`s as output in some of the arm flows. These are sent to readers. Their purpose is described later in this document.*
29+> **Note**: You may see `lowdown`s as output in some of the arm flows. These are sent to readers. Their purpose is described later in this document.
3030
3131### General messaging flow (broker to broker)
3232
@@ -53,6 +53,24 @@ When a broker gets a diff ("subscription poke") with a `%grams` report, it adds
5353
5454
5555
56+```
57+++ diff-talk-report ::
58+ ++ ra-diff-talk-report :: sends report to its appropriate story.
59+ ++ pa-diff-talk-report :: processes a report for a story.
60+ ++ pa-lesson :: learns a list of messages.
61+++ poke-talk-command ::
62+ ++ ra-apply :: applies a talk command.
63+ ++ ra-think :: consumes a list of messages.
64+ ++ ra-consume :: conducts a message to each partner in audience.
65+ ++ ra-conduct :: records a message or send it.
66+ ++ ra-transmit :: sends a message to a partner.
67+ ++ ra-record :: stores a message in a story.
68+ ++ pa-learn :: either adds or modifies a message.
69+ ++ pa-append :: adds a message to the story.
70+ ++ pa-revise :: modifies a message in the story.
71+ ++ pa-refresh :: sends messages to all interested subscribers.
72+```
73+
5674### Stories (broker to broker)
5775
5876```
@@ -86,12 +104,26 @@ To create a station, you send a `%design` command to your broker, containing the
86104
87105When people subscribe to a station it sends a `%peer` to the station's ship's broker. In the subscription path, the name of the station is specified, as well as the range of the subscription. For example, a subscriber may only be interested in messages up to a month from now.
88106
89−***Current implementation*** *always gets you messages from up to a day old, and all messages after that. Giving the user the option to specify the range of the subscription should prove useful to future applications of the talk system. The same can be said for making a range enforceable on a per-station basis.*
107+>**Current implementation** always gets you messages from up to a day old, and all messages after that. Giving the user the option to specify the range of the subscription should prove useful to future applications of the talk system. The same can be said for making a range enforceable on a per-station basis.
90108
91109When a broker gets peered, it checks the source for read permissions on the specified story before adding it to the story's list of subscribers. When this happens, the broker sends a bunch of different reports to the new subscriber to bring them up to speed on configuration, presences and messages of the station.
92110
93111
94112
113+```
114+++ poke-talk-command ::
115+ ++ ra-apply :: applies a talk command.
116+ ++ ra-config :: (re)configures a story.
117+ ++ pa-reform :: changes the configuration of the story.
118+++ peer ::
119+ ++ ra-subscribe :: adds a subscriber to a story, sends them info.
120+ ++ pa-report-cabal :: sends subscribers a report with configuration.
121+ ++ pa-notify :: changes a ship's presence in the story.
122+ ++ pa-report-group :: sends subscribers a report with presences.
123+ ++ pa-first-grams :: determines the backlog of messages to send.
124+ ++ pa-start :: sends a range of messages.
125+```
126+
95127#### Hearing stories
96128
97129```
@@ -111,21 +143,32 @@ When subscribing, no stories are created. A story is the one true canonical sour
111143
112144Every instance of a talk broker comes with a default mailbox. For planets, this is their `%porch`. It can be written to by everyone, but only read by its owner. The mailbox is used to store *all* incoming messages from *all* subscriptions. This means that if you are subscribed to a station you host, messages will be stored in both the station's story *and* your mailbox. (This is not a problem, and actually a good thing. It provides clear separation between subscriptions and stations themselves and makes it simple to not be subscribed to your own stations.)
113145
114−***Current implementation*** *still draws this line a bit unclearly, because of how the mailbox itself is also a regular story, and the storage of relevant data being as described below. The effort for changing this is not worth the small gains in clarity, however, and may even end up increasing clutter. (You'd almost think a broker-host and broker-guest would be appropriate, but likely not.)*
146+>**Current implementation** still draws this line a bit unclearly, because of how the mailbox itself is also a regular story, and the storage of relevant data being as described below. The effort for changing this is not worth the small gains in clarity, however, and may even end up increasing clutter. (You'd almost think a broker-host and broker-guest would be appropriate, but likely not.)
115147
116148Presence lists and configurations of stations you've joined are stored in maps in the broker's general state. These are updated whenever a `%cabal` or `%group` `report` is received. Such updates are sent to all subscribers of a story whenever changes occur.
117149
118−***Current implementation*** *doesn't store presence and config of local stations in the maps, instead keeping this strictly in the stories themselves. Also adding local story presences and configs to the global maps would help simplify communication with readers and readers themselves. (See also "Reader" below.)*
119−***Current implementation*** *includes remote presences and configs in `%cabal` and `%group` reports. Leaving those out should not hinder functionality. "But federation?" No party in a federation scenario should care where presence/config originated.*
150+>**Current implementation** doesn't store presence and config of local stations in the maps, instead keeping this strictly in the stories themselves. Also adding local story presences and configs to the global maps would help simplify communication with readers and readers themselves. (See also "Reader" below.)
151+
152+>**Current implementation** includes remote presences and configs in `%cabal` and `%group` reports. Leaving those out should not hinder functionality. "But federation?" No party in a federation scenario should care where presence/config originated.
120153
121154
122155
156+```
157+++ diff-talk-report ::
158+ ++ ra-diff-talk-report :: applies a report to the story it's intended for.
159+ ++ pa-diff-talk-report :: processes a report.
160+ ++ pa-cabal :: stores remote configuration.
161+ ++ pa-report-cabal :: sends subscribers a report with configuration.
162+ ++ pa-remind :: stores remote presence.
163+ ++ pa-report-group :: sends subscribers a report with presences.
164+```
165+
123166
124167## Reader
125168
126169A reader is an application used for interfacing with a talk broker. As such, it communicates solely with the broker of the identity it is associated with. No other brokers. No other readers. That is all left up to the broker itself.
127170
128−***Current implementation*** *still stores different tales (subsets of stories). Ideally, and taking the changes described below into account, readers should be able to get away with a single list of messages and maps for presences and configurations. Practically, you could say it's like subscribing to the broker's mailbox and the changes made to broker's presence and config lists. This means that the `man/knot` that's still passed around to cores in the reader has become mostly meaningless, and should be removed.*
171+>**Current implementation** still stores different tales (subsets of stories). Ideally, and taking the changes described below into account, readers should be able to get away with a single list of messages and maps for presences and configurations. Practically, you could say it's like subscribing to the broker's mailbox and the changes made to broker's presence and config lists. This means that the `man/knot` that's still passed around to cores in the reader has become mostly meaningless, and should be removed.
129172
130173### Staying informed (broker to reader)
131174
@@ -140,11 +183,27 @@ A reader is an application used for interfacing with a talk broker. As such, it
140183
141184As silently illustrated in the flow diagrams shown earlier in this document, `lowdown`s may get sent whenever something about a story changes: new or changed messages (`%grams`), presence (`%precs`) or configuration (`%confs`). Those lowdowns contain exclusively the changes compared to what the reader knows. When it first boots up, this is all state of all the broker's subscriptions. Afterwards, this is just whatever change the broker recently recorded.
142185
143−***Current implementation*** *has many more lowdowns, which include different lowdowns for local and remote station changes. Taking the changes described above into account (ditching tales for simpler storage) would make it possible to simplify reader state and the lowdowns that update it considerably.*
144−***Current implementation*** *sends much more than just the change for most lowdowns, often sending the entire new state. This places the burden of calculating the differences with the reader, even though the broker had that information readily available. Sending the change alone would free the reader from the required logic, and potentially allow the reader to stop mirroring broker state altogether. Reports might also be updated to behave this way. If needed, ++scry arms could be added to the broker for (hopefully infrequent) state access.*
186+>**Current implementation** has many more lowdowns, which include different lowdowns for local and remote station changes. Taking the changes described above into account (ditching tales for simpler storage) would make it possible to simplify reader state and the lowdowns that update it considerably.
187+
188+>**Current implementation** sends much more than just the change for most lowdowns, often sending the entire new state. This places the burden of calculating the differences with the reader, even though the broker had that information readily available. Sending the change alone would free the reader from the required logic, and potentially allow the reader to stop mirroring broker state altogether. Reports might also be updated to behave this way. If needed, ++scry arms could be added to the broker for (hopefully infrequent) state access.
145189
146190
147191
192+```
193+++ diff-talk-lowdown
194+ ++ ra-lowdown :: processes a lowdown.
195+ ++ ra-low-precs :: applies changes from a presence lowdown.
196+ ++ sh-low-precs :: displays changes from a presence lowdown.
197+ ++ ra-low-confs :: applies changes from a configuration lowdown.
198+ ++ sh-low-confs :: displays changes from a configuration lowdown.
199+ ++ ra-low-grams :: applies changes from a messages lowdown.
200+ ++ sh-low-grams :: displays changes from a messages lowdown.
201+ ++ pa-lesson :: learns a list of messages.
202+ ++ pa-learn :: either adds or modifies a message.
203+ ++ pa-append :: adds a message to the story.
204+ ++ pa-revise :: modifies a message in the story.
205+```
206+
148207### Changing shared UI (reader to broker to reader)
149208
150209```
@@ -166,19 +225,29 @@ On top of that, readers are in a good position for determining a user's status,
166225When a reader wants something done, it sends an `update` to its broker, containing the change that needs to happen. The reader applies the change to its own state before sending a `lowdown` describing it to all readers. Even the reader that requested the change gets informed, so it knows for sure that its change went through.
167226In the case of a `%status` update, we follow a similar flow similar to when we receive a `%group` `report`.
168227
169−***Current implementation*** *doesn't deal with status updates very well. You want to be able to be able to set your status per partner, but it's currently set by knot, which means it only works for local stations. What's more, there's no `command` for telling a foreign station you want to change your status, so unless you own the station your change won't be propagated.*
170−***Current implementation*** *also considers a `status` to be both `presence` and `human`. This same `human` structure is also used in storing local nicknames, however. All this needs some work for clarity and simplicity. (What's a "true name"? Demonology?) Allowing users to set per-station handles is probably fine, but this needs to actually be implemented.*
228+>**Current implementation** doesn't deal with status updates very well. You want to be able to be able to set your status per partner, but it's currently set by knot, which means it only works for local stations. What's more, there's no `command` for telling a foreign station you want to change your status, so unless you own the station your change won't be propagated.
229+
230+>**Current implementation** also considers a `status` to be both `presence` and `human`. This same `human` structure is also used in storing local nicknames, however. All this needs some work for clarity and simplicity. (What's a "true name"? Demonology?) Allowing users to set per-station handles is probably fine, but this needs to actually be implemented.
171231
172232
173233
234+```
235+++ poke-talk-update ::
236+ ++ ra-update :: processes an update.
237+ ++ pa-notify :: changes a ship's presence in the story.
238+ ++ pa-report-group :: sends subscribers a report with presences.
239+ ++ ra-inform :: sends a lowdown with changes to all readers.
240+++ diff-talk-lowdown ::
241+ ++ ra-lowdown :: processes a lowdown.
242+ ++ ra-low-names :: applies new or changed local identities.
243+ ++ ra-low-glyfs :: applies new or changed glyph bindings.
244+```
174245
175246## Implementation overview
176247
177−@TODO per core: task/subject, important arms, general flow description
178−
179248We're not going to be explaining talk's usage of cores here. ~~Please see the "core shenanigans" document for that.~~
180249
181−The different cores are described below. Renaming (of both cores and their arms) is going to happen.
250+The different cores used by the talk applications are described below. Renaming (of both cores and their arms) is going to happen.
182251
183252### `++ra` transactions (broker and reader)
184253
@@ -203,7 +272,7 @@ It modifies state and produces `command`s, `report`s and `lowdown`s.
203272
204273Used for modifying stories.
205274
206−***Current implementation*** *still has a `++pa` core in the reader, for dealing with tales. All it does is add messages to the specified tale. When ditching tales, this functionality should just get moved into the reader's `++ra` core.*
275+>**Current implementation** still has a `++pa` core in the reader, for dealing with tales. All it does is add messages to the specified tale. When ditching tales, this functionality should just get moved into the reader's `++ra` core.
207276
208277Important arms include:
209278* `++pa-diff-talk-report` for processing `report`s.
@@ -248,4 +317,3 @@ Used in telegram rendering. Can turn telegrams into their full representations,
248317## Notes
249318
250319* Federation might be implemented as a new subscription type, where multiple brokers subscribe to a station, mirror its state, and (after processing) relay commands/reports to the other federation nodes.
251−* ...