@@ -0,0 +1,189 @@
1+# Talk
2+## Architecture
3+
4+##### notes to self
5+
6+Justify everything!
7+
8+Include proposed changes:
9+* mirror config and state for local stories into remotes, allowing for:
10+ * less different lowdowns,
11+ * no more config/locals in tales in reader,
12+ * potentially no config/locals in stories, simplifying and clarifying.
13+* something to do with the mailbox story, also see above.
14+* something to allow federation?
15+
16+
17+This document is complemented by talk's source code. Definitions of data structures will be provided where useful. Descriptions of processes will be accompanied by diagrams describing this flow in code arms. [@TODO? For more detailed explanations, see the implementation overview at the end of this document.]
18+
19+***Current implementation*** *remarks mention where the [current implementation](https://github.com/Fang-/arvo/tree/talk-split) differs from what is described, and why changing that to match this document would be desirable.*
20+
21+### Overview
22+
23+Talk has been split into two, because x y z.
24+Official Urbit terminology calls these the `guardian` (like a unix daemon) and `agent`. To be more semantically appropriate for talk, we're calling them `broker` and `reader` respectively.
25+
26+#### Partners
27+
28+```
29+++ partner (each station passport) :: interlocutor
30+```
31+
32+A `partner` is either a `station` (a "chat room" hosted on a ship, represented as `~ship-name/station-name`) or a `passport`, which represents a non-station target. An example of a passport is a Twitter handle. Since passports aren't yet fully implemented for any kind of usage, we'll ignore them for now.
33+
34+
35+### Broker
36+
37+The broker is responsible for all primary talk functionality, everything that happens "under the hood". It can send, receive and store messages, keeps track of presence lists and configurations, and manages the stations you've created.
38+
39+Note: *You may see `lowdown`s as output in some of the arm flows. These are sent to readers. Their purpose is described later in this document.*
40+
41+#### General messaging flow (broker to broker)
42+
43+```
44+++ command :: effect on party
45+ $% {$publish (list thought)} :: originate
46+ {$review (list thought)} :: deliver
47+ :: ... ::
48+ == ::
49+++ thought (trel serial audience statement) :: which whom what
50+++ report :: talk update
51+ $% {$grams (pair @ud (list telegram))} :: beginning thoughts
52+ :: ... ::
53+ == ::
54+```
55+
56+Posting a message is done by poking your own broker with a `%publish` `command`, which prompts the broker to verifies that command originated from someone in its `team` (its own ship, or one of its moons), and to actually publish it.
57+
58+Having received and verified the command to publish a message, your broker sends a `%review` `command` to the brokers of all members of the `audience`. That is, all ships of the stations in the audience. (Again, messaging to other kinds of partners isn't implemented yet.)
59+
60+Upon getting poked with a command to review a message, a broker seeks out the appropriate story and (after checking if the sender has write permissions) adds the message to the story. Doing this causes a `%grams` `report` to be sent to everyone that has subscribed to that story.
61+
62+When a broker gets a diff ("subscription poke") with a `%grams` report, it adds all messages in it to the appropriate story.
63+
64+
65+
66+#### Stories (broker to broker)
67+
68+```
69+++ story :: wire content
70+ $: count/@ud :: (lent grams)
71+ grams/(list telegram) :: all history
72+ locals/atlas :: local presence
73+ sequence/(map partner @ud) :: partners heard
74+ shape/config :: configuration
75+ known/(map serial @ud) :: messages heard
76+ followers/(map bone river) :: subscribers
77+ == ::
78+```
79+
80+A `story` is the structure in which a station's full state is stored. Its the canonical source for its messages, configuration, presence and subscribers.
81+
82+##### Telling stories
83+
84+```
85+++ house :: broker state
86+ $: stories/(map knot story) :: conversations
87+ :: ... ::
88+ == ::
89+++ command :: effect on party
90+ $% {$design (pair knot (unit config))} :: configure+destroy
91+ :: ... ::
92+ == ::
93+```
94+
95+To create a station, you send a `%design` command to your broker, containing the name of the station and its initial configuration. After verifying you are in the broker's team, it sets the configuration for a story with the specified name, creating that story if it doesn't yet exist.
96+
97+When people subscribe to a station it sends a `%peer` to the station's ship's broker. In the subscription path, the name of the station is specified, as well as the range of the subscription. For example, a subscriber may only be interested in messages up to a month from now.
98+***Current implementation*** *always gets you messages from up to a day old, and all messages after that. Giving the user the option to specify the range of the subscription should prove useful to future applications of the talk system. The same can be said for making a range enforceable on a per-station basis.*
99+
100+When a broker gets peered, it checks the source for read permissions on the specified story before adding it to the story's list of subscribers. When this happens, the broker sends a bunch of different reports to the new subscriber to bring them up to speed on configuration, presences and messages of the station.
101+
102+
103+
104+##### Hearing stories
105+
106+```
107+++ house :: broker state
108+ $: remotes/(map partner atlas) :: remote presence
109+ mirrors/(map station config) :: remote config
110+ :: ... ::
111+ == ::
112+++ report :: talk update
113+ $% {$cabal config} :: config neighborhood
114+ {$group atlas} :: presence
115+ :: ... ::
116+ == ::
117+```
118+
119+When subscribing, no stories are created. A story is the one true canonical source of a station, there can only be one. So where does a subscriber keep a station's messages and other data?
120+
121+Every instance of a talk broker comes with a default mailbox. For planets, this is their `%porch`. It can be written to by everyone, but only read by its owner. The mailbox is used to store *all* incoming messages from *all* subscriptions. This means that if you are subscribed to a station you host, messages will be stored in both the station's story *and* your mailbox.
122+
123+Presence lists and configurations of stations you've joined are stored in maps in the broker's general state. These are updated whenever a `%cabal` or `%group` `report` is received. Such updates are sent to all subscribers of a story whenever changes occur.
124+***Current implementation*** *doesn't store presence and config of local stations in the maps, instead keeping this strictly in the stories themselves. Also adding local story presences and configs to the global maps would help simplify communication with readers and readers themselves. (See also "Reader" below.)*
125+***Current implementation*** *includes remote presences and configs in `%cabal` and `%group` reports. Leaving those out should not hinder functionality. "But federation?" No party in a federation scenario should care where presence/config originated.*
126+
127+
128+
129+### Reader
130+
131+A reader is an application used for interfacing with a talk broker. As such, it communicates solely with the broker of the identity it is associated with. No other brokers. No other readers. That is all left up to the broker itself.
132+
133+***Current implementation*** *still stores different tales (subsets of stories). Ideally, and taking the changes described below into account, readers should be able to get away with a single list of messages and maps for presences and configurations. Practically, you could say it's like subscribing to the broker's mailbox and the changes made to broker's presence and config lists.*
134+
135+#### Staying informed (broker to reader)
136+
137+```
138+++ lowdown :: reader update
139+ $% {$confs (map station (unit config))} :: changed config
140+ {$precs (map partner atlas)} :: changed presence
141+ {$grams (pair knot (pair @ud (list telegram)))} :: new grams
142+ :: ... ::
143+ == ::
144+```
145+
146+As silently illustrated in the flow diagrams shown earlier in this document, `lowdown`s may get sent whenever something about a story changes: new or changed messages (`%grams`), presence (`%precs`) or configuration (`%confs`). Those lowdowns contain exclusively the changes compared to what the reader knows. When it first boots up, this is all state of all the broker's subscriptions. Afterwards, this is just whatever change the broker recently recorded.
147+***Current implementation*** *has many more lowdowns, which include different lowdowns for local and remote station changes. Taking the changes described above into account (ditching tales for simpler storage) would make it possible to simplify reader state and the lowdowns that update it considerably.*
148+***Current implementation*** *sends much more than just the change for most lowdowns, often sending the entire new state. This places the burden of calculating the differences with the reader, even though the broker had that information readily available. Sending the change alone would free the reader from the required logic, and potentially allow the reader to stop mirroring broker state altogether. Reports might also be updated to behave this way. If needed, ++scry arms could be added to the broker for (hopefully infrequent) state access.*
149+
150+
151+
152+#### Changing shared UI (reader to broker to reader)
153+
154+```
155+++ update :: change shared state
156+ $% {$status (pair (set knot) status)} :: our status update
157+ {$human (pair ship human)} :: new identity
158+ {$bind (pair char (set partner))} :: bind a glyph
159+ == ::
160+++ lowdown :: reader update
161+ $% {$glyfs (jug char (set partner))} :: new bindings
162+ {$names (map ship (unit human))} :: new identities
163+ :: ... ::
164+ == ::
165+```
166+@TODO maybe update the above molds a bit to better fit the "send only diffs" design?
167+@TODO status per partner, not per story, right?
168+
169+@TODO
170+Sometimes when a user changes something about their UI, we want that to synchronize across all readers. This includes nicknames and glyph bindings. Also, status changes.
171+Broker receives these updates, processes them, then propagates to all readers.
172+
173+
174+
175+### Implementation overview
176+
177+@TODO per core: task/subject, important arms, general flow description
178+
179+We're not going to be explaining talk's usage of cores here. ~~Please see the "core shenanigans" document for that.~~
180+
181+#### `++ra` transactions
182+#### `++pa` stories/tales
183+#### `++sh` shell
184+#### `++sn` station rendering
185+#### `++ta` partner rendering
186+#### `++te` audience rendering
187+#### `++tr` telegram rendering
188+
189+@TODO insert/link to arm flow diagrams