@@ -1,8 +1,8 @@
1# Talk architecture
1# Talk architecture
3−This document is complemented by talk's source code. Definitions of data structures will be provided where useful. Descriptions of processes will be accompanied by diagrams describing this flow in code arms. [@TODO? For more detailed explanations, see the implementation overview at the end of this document.]
3+This document is complemented by talk's source code, but doesn't require it. Definitions of data structures will be provided where useful. Descriptions of processes will be accompanied by diagrams describing this flow in code arms. For an overview of the different cores and their most important arms, see the end of this document.
5−***Current implementation*** *remarks mention where the [current implementation](https://github.com/Fang-/arvo/tree/talk-split) differs from what is described, and why changing that to match this document would be desirable.*
5+>**Current implementation** remarks mention where the [current implementation](https://github.com/Fang-/arvo/tree/talk-split) differs from what is described, and why changing that to match this document would be desirable.
@@ -11,7 +11,7 @@ At the time of writing, talk is Urbit's biggest user-facing application, and fre
12Official Urbit terminology calls these the `guardian` (like a unix daemon) and `agent`. You may also view them as your personal server and client. To be more semantically appropriate for talk, we're calling them `broker` and `reader` respectively.
12Official Urbit terminology calls these the `guardian` (like a unix daemon) and `agent`. You may also view them as your personal server and client. To be more semantically appropriate for talk, we're calling them `broker` and `reader` respectively.
14−Everyone (per identity) has a single talk broker that does all the heavy lifting. Connected to it can be any number of readers, whose task it is to present the user with an interface for interacting with talk. Brokers and readers will only interact with each other if they are in the same team. That is, if they belong to the same identity, with moons counting as they planets.
14+Everyone (per identity) has a single talk broker that does all the heavy lifting. Connected to it can be any number of readers, whose task it is to present the user with an interface for interacting with talk. Brokers and readers will only interact with each other if they are in the same team. That is, if they belong to the same identity, with moons counting as their planets.
16### Partners
16### Partners
@@ -26,7 +26,7 @@ A `partner` is either a `station` (a "chat room" hosted on a ship, represented a
27The broker is responsible for all primary talk functionality, everything that happens "under the hood". It can send, receive and store messages, keeps track of presence lists and configurations, and manages the stations you've created.
27The broker is responsible for all primary talk functionality, everything that happens "under the hood". It can send, receive and store messages, keeps track of presence lists and configurations, and manages the stations you've created.
29−Note: *You may see `lowdown`s as output in some of the arm flows. These are sent to readers. Their purpose is described later in this document.*
29+> **Note**: You may see `lowdown`s as output in some of the arm flows. These are sent to readers. Their purpose is described later in this document.
31### General messaging flow (broker to broker)
31### General messaging flow (broker to broker)
@@ -53,6 +53,24 @@ When a broker gets a diff ("subscription poke") with a `%grams` report, it adds
54
54
57+++ diff-talk-report ::
58+ ++ ra-diff-talk-report :: sends report to its appropriate story.
59+ ++ pa-diff-talk-report :: processes a report for a story.
60+ ++ pa-lesson :: learns a list of messages.
61+++ poke-talk-command ::
62+ ++ ra-apply :: applies a talk command.
63+ ++ ra-think :: consumes a list of messages.
64+ ++ ra-consume :: conducts a message to each partner in audience.
65+ ++ ra-conduct :: records a message or send it.
66+ ++ ra-transmit :: sends a message to a partner.
67+ ++ ra-record :: stores a message in a story.
68+ ++ pa-learn :: either adds or modifies a message.
69+ ++ pa-append :: adds a message to the story.
70+ ++ pa-revise :: modifies a message in the story.
71+ ++ pa-refresh :: sends messages to all interested subscribers.
56### Stories (broker to broker)
74### Stories (broker to broker)
@@ -86,12 +104,26 @@ To create a station, you send a `%design` command to your broker, containing the
87When people subscribe to a station it sends a `%peer` to the station's ship's broker. In the subscription path, the name of the station is specified, as well as the range of the subscription. For example, a subscriber may only be interested in messages up to a month from now.
105When people subscribe to a station it sends a `%peer` to the station's ship's broker. In the subscription path, the name of the station is specified, as well as the range of the subscription. For example, a subscriber may only be interested in messages up to a month from now.
89−***Current implementation*** *always gets you messages from up to a day old, and all messages after that. Giving the user the option to specify the range of the subscription should prove useful to future applications of the talk system. The same can be said for making a range enforceable on a per-station basis.*
107+>**Current implementation** always gets you messages from up to a day old, and all messages after that. Giving the user the option to specify the range of the subscription should prove useful to future applications of the talk system. The same can be said for making a range enforceable on a per-station basis.
91When a broker gets peered, it checks the source for read permissions on the specified story before adding it to the story's list of subscribers. When this happens, the broker sends a bunch of different reports to the new subscriber to bring them up to speed on configuration, presences and messages of the station.
109When a broker gets peered, it checks the source for read permissions on the specified story before adding it to the story's list of subscribers. When this happens, the broker sends a bunch of different reports to the new subscriber to bring them up to speed on configuration, presences and messages of the station.
93
111
114+++ poke-talk-command ::
115+ ++ ra-apply :: applies a talk command.
116+ ++ ra-config :: (re)configures a story.
117+ ++ pa-reform :: changes the configuration of the story.
119+ ++ ra-subscribe :: adds a subscriber to a story, sends them info.
120+ ++ pa-report-cabal :: sends subscribers a report with configuration.
121+ ++ pa-notify :: changes a ship's presence in the story.
122+ ++ pa-report-group :: sends subscribers a report with presences.
123+ ++ pa-first-grams :: determines the backlog of messages to send.
124+ ++ pa-start :: sends a range of messages.
95#### Hearing stories
127#### Hearing stories
@@ -111,21 +143,32 @@ When subscribing, no stories are created. A story is the one true canonical sour
112Every instance of a talk broker comes with a default mailbox. For planets, this is their `%porch`. It can be written to by everyone, but only read by its owner. The mailbox is used to store *all* incoming messages from *all* subscriptions. This means that if you are subscribed to a station you host, messages will be stored in both the station's story *and* your mailbox. (This is not a problem, and actually a good thing. It provides clear separation between subscriptions and stations themselves and makes it simple to not be subscribed to your own stations.)
144Every instance of a talk broker comes with a default mailbox. For planets, this is their `%porch`. It can be written to by everyone, but only read by its owner. The mailbox is used to store *all* incoming messages from *all* subscriptions. This means that if you are subscribed to a station you host, messages will be stored in both the station's story *and* your mailbox. (This is not a problem, and actually a good thing. It provides clear separation between subscriptions and stations themselves and makes it simple to not be subscribed to your own stations.)
114−***Current implementation*** *still draws this line a bit unclearly, because of how the mailbox itself is also a regular story, and the storage of relevant data being as described below. The effort for changing this is not worth the small gains in clarity, however, and may even end up increasing clutter. (You'd almost think a broker-host and broker-guest would be appropriate, but likely not.)*
146+>**Current implementation** still draws this line a bit unclearly, because of how the mailbox itself is also a regular story, and the storage of relevant data being as described below. The effort for changing this is not worth the small gains in clarity, however, and may even end up increasing clutter. (You'd almost think a broker-host and broker-guest would be appropriate, but likely not.)
116Presence lists and configurations of stations you've joined are stored in maps in the broker's general state. These are updated whenever a `%cabal` or `%group` `report` is received. Such updates are sent to all subscribers of a story whenever changes occur.
148Presence lists and configurations of stations you've joined are stored in maps in the broker's general state. These are updated whenever a `%cabal` or `%group` `report` is received. Such updates are sent to all subscribers of a story whenever changes occur.
118−***Current implementation*** *doesn't store presence and config of local stations in the maps, instead keeping this strictly in the stories themselves. Also adding local story presences and configs to the global maps would help simplify communication with readers and readers themselves. (See also "Reader" below.)*
150+>**Current implementation** doesn't store presence and config of local stations in the maps, instead keeping this strictly in the stories themselves. Also adding local story presences and configs to the global maps would help simplify communication with readers and readers themselves. (See also "Reader" below.)
119−***Current implementation*** *includes remote presences and configs in `%cabal` and `%group` reports. Leaving those out should not hinder functionality. "But federation?" No party in a federation scenario should care where presence/config originated.*
152+>**Current implementation** includes remote presences and configs in `%cabal` and `%group` reports. Leaving those out should not hinder functionality. "But federation?" No party in a federation scenario should care where presence/config originated.
121
154
157+++ diff-talk-report ::
158+ ++ ra-diff-talk-report :: applies a report to the story it's intended for.
159+ ++ pa-diff-talk-report :: processes a report.
160+ ++ pa-cabal :: stores remote configuration.
161+ ++ pa-report-cabal :: sends subscribers a report with configuration.
162+ ++ pa-remind :: stores remote presence.
163+ ++ pa-report-group :: sends subscribers a report with presences.
126A reader is an application used for interfacing with a talk broker. As such, it communicates solely with the broker of the identity it is associated with. No other brokers. No other readers. That is all left up to the broker itself.
169A reader is an application used for interfacing with a talk broker. As such, it communicates solely with the broker of the identity it is associated with. No other brokers. No other readers. That is all left up to the broker itself.
128−***Current implementation*** *still stores different tales (subsets of stories). Ideally, and taking the changes described below into account, readers should be able to get away with a single list of messages and maps for presences and configurations. Practically, you could say it's like subscribing to the broker's mailbox and the changes made to broker's presence and config lists. This means that the `man/knot` that's still passed around to cores in the reader has become mostly meaningless, and should be removed.*
171+>**Current implementation** still stores different tales (subsets of stories). Ideally, and taking the changes described below into account, readers should be able to get away with a single list of messages and maps for presences and configurations. Practically, you could say it's like subscribing to the broker's mailbox and the changes made to broker's presence and config lists. This means that the `man/knot` that's still passed around to cores in the reader has become mostly meaningless, and should be removed.
130### Staying informed (broker to reader)
173### Staying informed (broker to reader)
@@ -140,11 +183,27 @@ A reader is an application used for interfacing with a talk broker. As such, it
141As silently illustrated in the flow diagrams shown earlier in this document, `lowdown`s may get sent whenever something about a story changes: new or changed messages (`%grams`), presence (`%precs`) or configuration (`%confs`). Those lowdowns contain exclusively the changes compared to what the reader knows. When it first boots up, this is all state of all the broker's subscriptions. Afterwards, this is just whatever change the broker recently recorded.
184As silently illustrated in the flow diagrams shown earlier in this document, `lowdown`s may get sent whenever something about a story changes: new or changed messages (`%grams`), presence (`%precs`) or configuration (`%confs`). Those lowdowns contain exclusively the changes compared to what the reader knows. When it first boots up, this is all state of all the broker's subscriptions. Afterwards, this is just whatever change the broker recently recorded.
143−***Current implementation*** *has many more lowdowns, which include different lowdowns for local and remote station changes. Taking the changes described above into account (ditching tales for simpler storage) would make it possible to simplify reader state and the lowdowns that update it considerably.*
186+>**Current implementation** has many more lowdowns, which include different lowdowns for local and remote station changes. Taking the changes described above into account (ditching tales for simpler storage) would make it possible to simplify reader state and the lowdowns that update it considerably.
144−***Current implementation*** *sends much more than just the change for most lowdowns, often sending the entire new state. This places the burden of calculating the differences with the reader, even though the broker had that information readily available. Sending the change alone would free the reader from the required logic, and potentially allow the reader to stop mirroring broker state altogether. Reports might also be updated to behave this way. If needed, ++scry arms could be added to the broker for (hopefully infrequent) state access.*
188+>**Current implementation** sends much more than just the change for most lowdowns, often sending the entire new state. This places the burden of calculating the differences with the reader, even though the broker had that information readily available. Sending the change alone would free the reader from the required logic, and potentially allow the reader to stop mirroring broker state altogether. Reports might also be updated to behave this way. If needed, ++scry arms could be added to the broker for (hopefully infrequent) state access.
146
190
194+ ++ ra-lowdown :: processes a lowdown.
195+ ++ ra-low-precs :: applies changes from a presence lowdown.
196+ ++ sh-low-precs :: displays changes from a presence lowdown.
197+ ++ ra-low-confs :: applies changes from a configuration lowdown.
198+ ++ sh-low-confs :: displays changes from a configuration lowdown.
199+ ++ ra-low-grams :: applies changes from a messages lowdown.
200+ ++ sh-low-grams :: displays changes from a messages lowdown.
201+ ++ pa-lesson :: learns a list of messages.
202+ ++ pa-learn :: either adds or modifies a message.
203+ ++ pa-append :: adds a message to the story.
204+ ++ pa-revise :: modifies a message in the story.
148### Changing shared UI (reader to broker to reader)
207### Changing shared UI (reader to broker to reader)
@@ -166,19 +225,29 @@ On top of that, readers are in a good position for determining a user's status,
166When a reader wants something done, it sends an `update` to its broker, containing the change that needs to happen. The reader applies the change to its own state before sending a `lowdown` describing it to all readers. Even the reader that requested the change gets informed, so it knows for sure that its change went through.
225When a reader wants something done, it sends an `update` to its broker, containing the change that needs to happen. The reader applies the change to its own state before sending a `lowdown` describing it to all readers. Even the reader that requested the change gets informed, so it knows for sure that its change went through.
167In the case of a `%status` update, we follow a similar flow similar to when we receive a `%group` `report`.
226In the case of a `%status` update, we follow a similar flow similar to when we receive a `%group` `report`.
169−***Current implementation*** *doesn't deal with status updates very well. You want to be able to be able to set your status per partner, but it's currently set by knot, which means it only works for local stations. What's more, there's no `command` for telling a foreign station you want to change your status, so unless you own the station your change won't be propagated.*
228+>**Current implementation** doesn't deal with status updates very well. You want to be able to be able to set your status per partner, but it's currently set by knot, which means it only works for local stations. What's more, there's no `command` for telling a foreign station you want to change your status, so unless you own the station your change won't be propagated.
170−***Current implementation*** *also considers a `status` to be both `presence` and `human`. This same `human` structure is also used in storing local nicknames, however. All this needs some work for clarity and simplicity. (What's a "true name"? Demonology?) Allowing users to set per-station handles is probably fine, but this needs to actually be implemented.*
230+>**Current implementation** also considers a `status` to be both `presence` and `human`. This same `human` structure is also used in storing local nicknames, however. All this needs some work for clarity and simplicity. (What's a "true name"? Demonology?) Allowing users to set per-station handles is probably fine, but this needs to actually be implemented.
172
232
235+++ poke-talk-update ::
236+ ++ ra-update :: processes an update.
237+ ++ pa-notify :: changes a ship's presence in the story.
238+ ++ pa-report-group :: sends subscribers a report with presences.
239+ ++ ra-inform :: sends a lowdown with changes to all readers.
240+++ diff-talk-lowdown ::
241+ ++ ra-lowdown :: processes a lowdown.
242+ ++ ra-low-names :: applies new or changed local identities.
243+ ++ ra-low-glyfs :: applies new or changed glyph bindings.
175## Implementation overview
246## Implementation overview
177−@TODO per core: task/subject, important arms, general flow description
179We're not going to be explaining talk's usage of cores here. ~~Please see the "core shenanigans" document for that.~~
248We're not going to be explaining talk's usage of cores here. ~~Please see the "core shenanigans" document for that.~~
181−The different cores are described below. Renaming (of both cores and their arms) is going to happen.
250+The different cores used by the talk applications are described below. Renaming (of both cores and their arms) is going to happen.
183### `++ra` transactions (broker and reader)
252### `++ra` transactions (broker and reader)
@@ -203,7 +272,7 @@ It modifies state and produces `command`s, `report`s and `lowdown`s.
204Used for modifying stories.
273Used for modifying stories.
206−***Current implementation*** *still has a `++pa` core in the reader, for dealing with tales. All it does is add messages to the specified tale. When ditching tales, this functionality should just get moved into the reader's `++ra` core.*
275+>**Current implementation** still has a `++pa` core in the reader, for dealing with tales. All it does is add messages to the specified tale. When ditching tales, this functionality should just get moved into the reader's `++ra` core.
208Important arms include:
277Important arms include:
209* `++pa-diff-talk-report` for processing `report`s.
278* `++pa-diff-talk-report` for processing `report`s.
@@ -248,4 +317,3 @@ Used in telegram rendering. Can turn telegrams into their full representations,
250* Federation might be implemented as a new subscription type, where multiple brokers subscribe to a station, mirror its state, and (after processing) relay commands/reports to the other federation nodes.
319* Federation might be implemented as a new subscription type, where multiple brokers subscribe to a station, mirror its state, and (after processing) relay commands/reports to the other federation nodes.