@@ -1,1335 +0,0 @@
5−aliases = ["docs/learn/arvo/hall/"]
7−> Last major revision of this tutorial: 2017. Some of the information and Hoon syntax here may
11−Hall is the Urbit messaging and notifications protocol. This document details Hall's architecture, its interfaces, and the new Gall model it's built to target.
15−- [Hall Architecture](#hall-architecture)
17−- [The Hall Interface](#the-hall-interface)
19−- [The New Gall Model](#the-new-gall-model)
23−This document is complemented by Hall's source code, but doesn't require it. Definitions of data structures, code snippets and diagrams will be provided where useful.
25−Hall's implementation is structured according to the new Gall model. Familiarity with its concepts is assumed. See [the new Gall model](#the-new-gall-model).
29−Talk was Urbit's first big user-facing application. It continues to enjoy a prominent role in the Urbit landscape, but now does so as two separate applications.
31−The messaging parts of Talk have been separated from its user-interface parts. What we ended up with is a shiny new generic messaging bus, and the chat interface we all know and love. The messaging bus, which should now prove useful to many different applications, will be named `Hall`. Applications that use it are referred to as **clients**. One such application, as you might have guessed, is `:talk`.
34−## Structures & functionality
39−++ circle {hos/ship nom/term} :: native target
42−A `circle` is essentially a named collection of messages created by and hosted on a ship's Hall, usually represented as `~ship-name/circle-name`. Most of Hall revolves around doing things with circles.
46−When we subscribe to a circle, the primary thing we're interested in is its messages. Message data itself isn't that complicated, but a fair amount of metadata comes into play when actually sending a message. Let's work our way up, starting at the contents.
49−++ speech :> content body
50− $% {$lin pat/? msg/cord} :< no/@ text line
51− {$url url/purf:eyre} :< parsed url
52− {$exp exp/cord res/(list tank)} :< hoon line
53− {$ire top/serial sep/speech} :< in reply to
54− {$fat tac/attache sep/speech} :< attachment
55− {$app app/term sep/speech} :< app message
56− {$inv inv/? cir/circle} :< inv/ban for circle
60−At the heart of every message lies a `speech` that describes the message body. There's a large number of different speech types, from simple text messages to parsed URLs, Hoon expressions and more.
63−++ audience (set circle) :< destinations
64−++ serial @uvH :< unique identifier
65−++ thought :> inner message
66− $: uid/serial :< unique identifier
67− aud/audience :< destinations
69− sep/speech :< content
73−A `speech` is always accompanied by various bits of metadata. We include a unique identifier, a message timestamp and an `audience`, the set of all intended recipients of the message.
76−++ telegram {aut/ship thought} :< whose message
77−++ envelope {num/@ud gam/telegram} :< outward message
80−Finally, before sending the message over the wire, we add on the message's original author. In most cases, we also include a sequence number, the index of the message in the originating circle's list of messages. This can be a useful point of reference for clients.
82−### Participant metadata
84−Messages aren't the only thing a subscription gets us. We're also kept up to date with relevant metadata.
87−++ crowd {loc/group rem/(map partner group)} :< our & srcs presences
88−++ group (map ship status) :< presence map
89−++ status {pec/presence man/human} :< participant
90−++ presence :> status type
96−++ human :> human identifier
97− $: han/(unit cord) :< handle
98− tru/(unit truename) :< true name
100−++ truename {fir/cord mid/(unit cord) las/cord} :< real-life name
103−`status` is user-set metadata that describes, well, the status of users in a circle. This encompasses their `presence`, which shows their activity, and their `human` identity, which includes their display handle.
105−For reasons we'll discover shortly, circles keep track of both their own `group` and those of the circles they're subscribed to. `crowd` encapsulates this.
110−++ lobby {loc/config rem/(map circle config)} :< our & srcs configs
111−++ config :> circle config
112− $: src/(set circle) :< active sources
113− cap/cord :< description
114− fit/filter :< message rules
115− con/control :< restrictions
117−++ filter :> content filters
118− $: cas/? :< dis/allow capitals
119− utf/? :< dis/allow non-ascii
121−++ control {sec/security ses/(set ship)} :< access control
124−Another part of metadata we get from circle subscriptions is configurations. Again, circles want to remember their own configuration, as well as those of their subscriptions. But why, precisely?
126−The `config` structure contains `src`, a set of partners. These indicate the different sources a circle is currently pulling content from. This allows circles to aggregate messages from multiple places. In doing so, it also receives metadata from those places, hence why we have structures for storing "remote" presences and configurations alongside local ones.
128−Aside from that, `config` contains a description for the circle, which is exactly what it sounds like. It also specifies a `filter`, which are content formatting rules all messages passing through this circle are made to adhere to.
130−Lastly, there's a `control` structure that contains both a security mode and a list of ships, which is either a white- or blacklist depending on the aforementioned mode. There are four such modes available.
133−++ security :> security mode
134− $? $channel :< blacklist
135− $village :< whitelist
136− $journal :< pub r, whitelist w
137− $mailbox :< our r, blacklist w
141−A `channel` is publicly readable and writable, with a blacklist for blocking.
142−A `village` is privately readable and writable, with a whitelist for inviting.
143−A `journal` is publicly readable and privately writable, with a whitelist for authors.
144−A `mailbox` is readable by its owner and publicly writable, with a blacklist for blocking.
148−To see how that all ties together, we're going to take a look at Hall's state.
151−++ state :> hall state
152− $: stories/(map term story) :< conversations
155−++ story :> wire content
156− $: grams/(list telegram) :< all messages
157− locals/group :< local presence
158− remotes/(map circle group) :< remote presence
159− shape/config :< configuration
160− mirrors/(map circle config) :< remote config
165−Stories are the primary driver behind Hall. They are the structures that are used to power circles, which we can now say are named stories hosted on ships.
167−With the configuration described above in mind, we can try and imagine the things we can do with stories. Knowing that we can subscribe them to any number of sources, they can function as central hubs for our messaging, aggregate specific kinds of data feeds, or simply accept whatever messages get sent to it like a regular old chatroom.
171−Upon initial startup, Hall creates a default story, a mailbox named `inbox`. This mailbox is the primary target for anyone and anything that wants to reach its owner. Applications can use it to send notifications and other information to the user (Hall itself does this as well), and users can use it to send direct messages to each other.
173−Applications that use Hall, especially clients, are encouraged to use the default mailbox as the primary messaging hub. This way, users can easily switch between different applications without "losing" their subscriptions, message backlog, etc.
175−As an example, Talk operates like this, serving as an interface for reading from and managing the user's mailbox. Local stories are subscribed to through the `inbox`, which ends up containing all messages the user receives.
177−### Public membership
179−To aid in discoverability of circles, it is possible for users to make their participation in any given circle public. This data can then be used by things like profile widgets, or read out directly by other users.
182−## Interfaces for applications
184−Applications can interact with Hall in two complementary ways: they can tell it what actions to perform, and they can subscribe to its state and the changes made to it.
188−Hall can be commanded by poking it with `action`s.
191−++ action :> user action
192− $% :: circle configuration ::
193− {$create nom/term des/cord sec/security} :< create circle
194− {$delete nom/term why/(unit cord)} :< delete + announce
195− {$depict nom/term des/cord} :< change description
196− {$filter nom/term fit/filter} :< change message rules
197− {$permit nom/term inv/? sis/(set ship)} :< invite/banish
198− {$source nom/term sub/? srs/(set source)} :< un/sub to/from src
200− {$convey tos/(list thought)} :< post exact
201− {$phrase aud/audience ses/(list speech)} :< post easy
202− :: personal metadata ::
203− {$notify aud/audience pes/(unit presence)} :< our presence update
204− {$naming aud/audience man/human} :< our name update
205− :: changing shared ui ::
206− {$glyph gyf/char aud/audience bin/?} :< un/bind a glyph
207− {$nick who/ship nic/cord} :< new identity
209− {$public add/? cir/circle} :< show/hide membership
213−The largest part of these actions concern themselves with managing local circles. `nom` is always the name used to identify the relevant story. To disambiguate between "add" and "delete" type actions (for permissions and subscriptions), a loob `?` is used.
215−To send messages, two interfaces are available. `%convey` lets you specify all details of the messages, including its timestamp, serial, and fully assembled audience. `%phrase`, on the other hand, takes care of that for you, allowing you to specify just the target partners and message contents.
217−`%notify` and `%naming` are useful for setting your own presence and nickname in a circle respectively, so others can see if you're active and what to call you by.
219−"Shared UI" encompasses UI data that should be consistent across applications. For example, if the user sets a local nickname for an identity, they expect to see that nickname regardless of the application they're currently using. The same goes for glyph bindings, for easy audience targeting.
221−### Queries, prizes and rumors
223−To receive data from Hall, applications will have to query it. There are two paths that are useful for clients to query. Let's look at them and their results.
226−++ query :> query paths
227− $% {$client $~} :< shared ui state
228− $: $circle :> story query
229− nom/naem :< circle name
230− wat/(set circle-data) :< data to get
231− ran/range :< query duration
235−++ circle-data :> kinds of circle data
236− $? $grams :< messages
237− $group-l :< local presence
238− $group-r :< remote presences
239− $config-l :< local config
240− $config-r :< remote configs
242−++ range :> inclusive msg range
243− %- unit :< ~ means everything
244− $: hed/place :< start of range
245− tal/(unit place) :< opt end of range
247−++ place :> range indicators
248− $% {$da @da} :< date
249− {$ud @ud} :< message number
251−++ prize :> query result
252− $% $: $client :< /client
253− gys/(jug char (set partner)) :< glyph bindings
254− nis/(map ship cord) :< nicknames
256− {$circle burden} :< /circle
259−++ burden :< full story state
260− $: gaz/(list telegram) :< all messages
261− cos/lobby :< loc & rem configs
262− pes/crowd :< loc & rem presences
266−To be clear, the paths for those queries are `/client` and `/circle/[name]/[data]/[start]/[end]`, where data is at least one `circle-data`, and `start` and `end` are optional and can be either a date or a message number.
268−`/client` queries produce shared UI state. That is, glyph bindings and nicknames.
269−`/circle` queries produce the entire public state of the requested story, including remotes.
271−Any changes to the results of these queries are communicated via the following `rumor`s. It's a fairly long list of potential changes,
274−++ rumor :< query result change
275− $% {$client rum/rumor-client} :< /client
276− {$circle rum/rumor-story} :< /circle
279−++ rumor-client :< changed ui state
280− $% {$glyph diff-glyph} :< un/bound glyph
281− {$nick diff-nick} :< changed nickname
283−++ diff-glyph {bin/? gyf/char aud/audience} :< un/bound glyph
284−++ diff-nick {who/ship nic/cord} :< changed nickname
285−++ diff-story :> story change
286− $% {$new cof/config} :< new story
287− {$config cir/circle dif/diff-config} :< new/changed config
288− {$status cir/circle who/ship dif/diff-status} :< new/changed status
289− {$remove $~} :< removed story
292−++ rumor-story ::> story rumor
293− $? diff-story ::< both in & outward
294− $% {$gram nev/envelope} ::< new/changed msgs
296−++ diff-config :> config change
297− $% {$full cof/config} :< set w/o side-effects
298− {$source add/? src/source} :< add/rem sources
299− {$caption cap/cord} :< changed description
300− {$filter fit/filter} :< changed filter
301− {$secure sec/security} :< changed security
302− {$permit add/? sis/(set ship)} :< add/rem to b/w-list
303− {$remove $~} :< removed config
305−++ diff-status :> status change
306− $% {$full sat/status} :< fully changed status
307− {$presence pec/presence} :< changed presence
308− {$human dif/diff-human} :< changed name
309− {$remove $~} :< removed status
311−++ diff-human :> name change
312− $% {$full man/human} :< fully changed name
313− {$handle han/(unit cord)} :< changed handle
314− {$true tru/(unit truename)} :< changed true name
318−These are, as enforced by Gall, the changes as they happen. This makes it possible for very minimal Hall clients to be implemented that only display a stream of changes, rather than keeping state themselves.
321−## Communication between Halls
323−Halls can communicate with other Halls by sending commands and subscription updates.
327−To request changes to a story, Halls can send `command`s.
330−++ command :> effect on story
331− $% {$publish tos/(list thought)} :< deliver
332− {$present nos/(set term) dif/diff-status} :< status update
337−A `%publish` command contains messages. These are sent to foreign Halls to be published to their stories.
339−The `%present` command is used for sending status updates about our ship.
343−Halls can query other halls. Here's the peer move we send when we open a query on a foreign circle:
348− /[our-circle]/[host]/[query-path] :< rumor path
349− [host %hall] :< query target
350− /circle/[their-circle]/[data]/[start]/[end] :< query path
354−Again, the rumor path is what our Hall uses to identify what query a received message originates from. Our circle is specified so that we know what story is interested in the changes we get.
356−The query path is slightly more interesting. Of course it specifies the name of their story we want to subscribe our story to, but also a "start" and "end". These can (optionally) be used to specify the range of messages we want to get from our query. Once that range has passed, we stop receiving updates.
358−### Federation implementation
360−Along the way so far, we've skipped some parts of structures. Most of those relate to federation. Let's see what we missed.
363−++ query :> query paths
364− $% {$burden who/ship} :< duties to share
365− {$report $~} :< duty reports
368−++ prize :> query result
369− $% {$burden sos/(map term burden)} :< /burden
372−++ rumor :< query result change
373− $% {$burden nom/term rum/rumor-story} :< /burden
377− $% {$bear bur/burden} :< new inherited story
380−++ burden :> full story state
381− $: gaz/(list telegram) :< all messages
382− cos/lobby :< loc & rem configs
383− pes/crowd :< loc & rem presences
387−We briefly touched upon how federation works on the higher level. On the lower, this is how things go down.
389−1. When Hall boots on a star or galaxy, it starts querying its parent's `/burden` path. (Galaxies query ~zod.)
390−2. Upon receiving that query, the parent sends a `%burden` prize containing all state of its channels (fully public circles) to the child. It also subscribes to `/report` on that child.
391−3. When a new story gets created on the parent, its children get a `%burden` rumor with a `%bear` story diff, and they create a local copy of that channel.
392−4. When something about a parent's story changes, a `%burden` rumor is sent to all its children (because they're querying `/burden`). The children apply this change to their local version of the story.
393−5. When a child's story has a `%grams` or `%status` change, a `%burden` rumor is sent to the parent (because they're querying `/report`). The parent applies this change to their local version of the story, and #4 happens.
396−## Hall implementation
398−Now that we know what gets sent over the wire and why, let's see what happens whenever such events happen. We won't be covering everything, but enough to illustrate the general flow.
400−First though, it's useful to understand how Hall's code is structured. It consists of primary cores, designed to work optimally with Gall.
402−- `++ta` is the transaction core. For every event that happens, be it a poke, peer or diff, the transaction core gets put to work to process it. Once it's done, it produces a list of deltas that describe how our state needs to be modified.
403−- `++da` is the delta application core. It is used to apply `++ta`'s deltas to our application state, and produce a list of side-effects.
405−Both those cores also have a core within themselves dedicated solely to dealing with stories, `++so` and `++sa` respectively.
407−Once all work in an engine core has finished, its `-done` arm is called to produce the result of all the core's computation.
409−In the diagrams that follow, regular lines indicate flows that always happen. Dashed lines indicate flows that are traversed if the described condition is met.
410−Below the diagrams you will find brief explanations of the involved arms. In a particularly bright future, these might be available on-hover instead.
411−For brevity, utility arms that aren't a direct and important part of the flow have been omitted.
413−(You will find that the origins still use old Gall arms, with some glue code in between. At the time of writing, new Gall has not yet been implemented, so this will have to do.)
415−We will only display the delta generation parts of the flow. Delta application is usually implemented simply enough to deduce what it does from looking at the label.
419−Hall-to-Hall subscriptions happen when a source gets added to a story. This is done with a `%source` action, and results in a `%peer` move being sent (prompted by a `%story %follow` delta).
420−Upon receiving a valid peer on a `/circle` path, the subscribing ship is added to that circle's presence map.
422−
425−++ poke-talk-action :< we got poked with an action.
426− ++ ta-action :< applies an action.
427− ++ ta-config :< (re)configures a story.
429−++ peer :< we got peered.
430− ++ g-query :< resolves query to prize.
431− ++ ta-subscribe :< act upon subscription.
432− ++ so-attend :< adds a ship's status to the story.
437−Once a query has opened, Hall will receive updates on it, rumors. The changes describes in these rumors are applied via the following flow.
439−
442−++ diff-talk-rumor :< we got a query update.
443− ++ ta-hear :< applies a rumor to the story it's intended for.
444− ++ so-hear :< applies a %circle rumor to the story.
445− ++ so-config-full :< splits a %full diff-config into separate changes.
446− ++ so-bear :< accept burden, assimilate into the story's state.
447− ++ so-lesson :< sends a report with presences.
452−To send messages, the user sends a `%convey` or `%phrase` action, resulting in a `%publish` command being sent to the involved partners. Receiving a `%publish` command causes its messages to be added to the story through the creation of a `%story %grams` delta.
454−
457−++ poke-talk-action :< we got poked with an action.
458− ++ ta-action :< applies an action.
460−++ poke-talk-command :< we get poked with a command.
461− ++ ta-apply :< applies a command.
462− ++ ta-think :< consumes each message in the given list.
463− ++ ta-consume :< conducts a message to each partner in audience.
464− ++ ta-conduct :< records or sends a message.
465− ++ ta-transmit :< sends a message to a partner.
466− ++ ta-record :< stores a message in a story.
467− ++ so-learn :< either adds or modifies a message.
473−Hall is neither complete nor perfect. There are features that need to be implemented, varying from small quality of life changes to broad-reaching functionality.
475−### Features and functionality
477−To turn Talk into a [sustainable social platform](https://urbit.org/fora/posts/~2017.4.26..18.00.25..b93c~/), it needs a number of things.
479−We want to have some kind of discoverability for circles. This could be realized by making it possible for users to add friends, who would have their Hall subscribe to a list of circles the added friends are in. The existing public membership functionality can be leveraged for this.
481−Eventually, content self-moderation might need to be implemented. Users would flag their content if it is potentially offensive. Others would easily be able to filter out content that might offend them.
482−This functionality brings its own large sets of challenges that need to be tackled.
484−Additionally, federation might be put to broader use. Knowing that a circle is federated could help fall back to alternative hosts in case the one Hall originally subscribed to is unavailable. Implementation-wise, care would need to be taken for this to not get ugly. Not to mention, when/how would availability of a host be verified?
486−Other changes that might have a fair impact on the functioning of Hall are un/read states for messages, and being able to use moons to allow your friends to chat with you in a local channel.
488−More minor functionalities that need to be implemented include:
489−- Extended permissions management. Being able to black-/whitelist entire ship classes.
490−- Improved presence functionality. Actually using the presence system, and sending "typing", "idle", etc. statuses.
493−And, of course, there's a lot that the Talk client can improve on as well.
498−To gain a more thorough understanding of Hall's inner workings, take a look at its source code. It comes with inline documentation.
499−[On Github.](https://github.com/urbit/urbit/blob/master/pkg/arvo/app/hall.hoon)
501−To see an expansive example of a Hall client, take a look at the code of Talk. It, too, comes with inline documentation.
502−[On Github.](https://github.com/urbit/urbit/blob/master/pkg/arvo/app/talk.hoon)
506−This document describes the different interfaces Hall provides and the data that is accessible and modifiable through them. Knowledge of the Urbit application model (including [new gall](#the-new-gall-model)) and [Hall's architecture](#hall-architecture) is assumed.
508−While the structures here are given in Hoon, they match fairly closely to their JSON equivalent. Most important to note is that `$%({$x y/z})` becomes accessible as `json.x.y`.
512−Queries are the paths you pass into `%peer` moves. Internally, they get translated to a `++query` structure for easier handling. We'll be giving examples of valid query paths alongside the structures themselves.
516−To be able to keep certain UI elements like glyphs and local nicknames consistent across different Hall clients, they can query Hall for the current UI state.
520− $% {$client $~} :< shared ui state
528−:> all shared ui state
534−Contains a map of glyphs and the audiences that they map to, as well as a map of ships and their locally set nicknames.
537−++ prize :> query result
538− $% {$client prize-client} :< /client
541−++ prize-client :> shared ui state
542− $: gys/(jug char audience) :< glyph bindings
543− nis/(map ship nick) :< local nicknames
549−Contains either a bound or unbound glyph and its target, or a ship with its new nickname. A nickname of `''` means the associated ship no longer has a nickname set for it.
552−++ rumor :> query result change
553− $% {$client rum/rumor-client} :< /client
556−++ rumor-client :< changed ui state
557− $% {$glyph diff-glyph} :< un/bound glyph
558− {$nick diff-nick} :< changed nickname
560−++ diff-glyph {bin/? gyf/char aud/audience} :< un/bound glyph
561−++ diff-nick {who/ship nic/nick} :< changed nickname
567−To aid in circle discoverability, users can add circles to their "public membership" list. This can then be queried for by, for example, a profile page.
571− $% {$public $~} :< public memberships
579−:> all public memberships
585−Contains the set of circles the user has on their public list.
588−++ prize :> query result
589− $% {$public cis/(set circle)} :< /public
596−Contains a circle that was either added or removed from the public list.
599−++ rumor :> query result change
600− $% {$public add/? cir/circle} :< /public
607−To allow a circle owner to inspect who is currently subscribed to their stories, they can issue a query to retrieve subscription data.
611− $% {$peers nom/naem} :< readers of story
616−Query paths are structured as follows:
625−:> peers for circle %urbit-help
631−Contains a map of ships and the different queries they currently have active for the selected story.
634−++ prize :> query result
635− $% {$peers pes/(jar ship query)} :< /peers
642−Contains a ship and a query, and a flag to indicate whether that subscription has started or ended.
645−++ rumor :> query result change
646− $% {$peers add/? who/ship qer/query} :< /peers
653−Circle queries allow for the retrieving of data from stories. Their messages, configuration, and presences can all be accessed. Since this is a lot of data, there are lots of possibilities for filtering it built in to the query itself.
656−A quick refresher on the difference between "local" and "remote" presence and configuration: "local" means it pertains to the circle itself; "remote" means it pertains to one of its configured sources. The latter is primarily useful to clients when using a circle for aggregation, like the `%inbox`.
660− $% $: $circle :> story query
661− nom/naem :< circle name
662− wer/(unit circle) :< from source
663− wat/(set circle-data) :< data to get
664− ran/range :< query duration
668−++ circle-data :> kinds of circle data
669− $? $grams :< messages
670− $group-l :< local presence
671− $group-r :< remote presences
672− $config-l :< local config
673− $config-r :< remote configs
675−++ range :> inclusive msg range
676− %- unit :< ~ means everything
677− $: hed/place :< start of range
678− tal/(unit place) :< opt end of range
680−++ place :> range indicators
681− $% {$da @da} :< date
682− {$ud @ud} :< message number
686−Query paths are structured as follows:
689−/circle/[circle-name]/(from-circle)/[what/data]/(range-start(/range-end))
690−(from-circle) : optional message source, ~ship/circle
691−[what/data] : one or more of grams, group, group-l, group-r,
692− : config, config-l, config-r, combined using /
693−(range) : an optional range with an optional end, its points denoted
694− : in either message number (@ud) or date (@da)
700−:> get all messages from circle %urbit-help
701−/circle/urbit-help/grams
702−:> get all messages, all presences and local configs from %urbit-help
703−/circle/urbit-help/grams/group/config-l
704−:> get all messages %urbit-help has heard from its source ~zod/fora
705−/circle/urbit-help/~zod/fora/grams
706−:> get the first 100 messages from %urbit-help
707−/circle/urbit-help/grams/0/99
708−:> get the first 100 messages %urbit-help has heard from its source ~zod/fora
709−/circle/urbit-help/~zod/fora/grams/0/99
710−:> get all messages from %urbit-help, starting now
711−/circle/urbit-help/grams/group/(scow %da now)
712−:> get all messages from %urbit-help, starting now, ending at the 100th message
713−/circle/urbit-help/grams/(scow %da now)/99
714−:> get local presences from %urbit-help for a week
715−/circle/urbit-help/group-l/(scow %da now)/(scow %da (add now ~d7))
720−Contains (where applicable) messages in envelopes (with message numbers), as well as local and remote configurations and presences.
723−++ prize :> query result
724− $% {$circle package} :< /circle
727−++ package :> story state
728− $: nes/(list envelope) :< messages
729− cos/lobby :< loc & rem configs
730− pes/crowd :< loc & rem presences
736−Contains a detailed change description of the data relevant to the query that changed.
738−Messages are wrapped in envelopes to include their sequence number, and note the source they were heard from. Configuration and status changes specify the circle they apply to.
741−++ rumor :> query result change
742− $% {$circle rum/rumor-story} :< /circle
745−++ rumor-story :> story rumor
746− $% {$new cof/config} :< new story
747− {$gram src/circle nev/envelope} :< new/changed message
748− {$config cir/circle dif/diff-config} :< new/changed config
749− {$status cir/circle who/ship dif/diff-status} :< new/changed status
750− {$remove $~} :< removed story
752−++ diff-config :> config change
753− $% {$full cof/config} :< set w/o side-effects
754− {$source add/? src/source} :< add/rem sources
755− {$caption cap/cord} :< changed description
756− {$filter fit/filter} :< changed filter
757− {$secure sec/security} :< changed security
758− {$permit add/? sis/(set ship)} :< add/rem to b/w-list
759− {$remove $~} :< removed config
761−++ diff-status :> status change
762− $% {$full sat/status} :< fully changed status
763− {$presence pec/presence} :< changed presence
764− {$human dif/diff-human} :< changed name
765− {$remove $~} :< removed status
767−++ diff-human :> name change
768− $% {$full man/human} :< fully changed name
769− {$handle han/(unit cord)} :< changed handle
770− {$true tru/(unit truename)} :< changed true name
777−Actions can be sent by poking Hall with data marked as `%hall-action`. Actions are used for all user operations. If an error or other unexpected behavior occurs while executing an action, Hall notifies the user by sending an `%app` message to their `%inbox`.
779−### Circle configuration
781−Since all of these apply to a specific circle, they all specify a name `nom` of the circle to operate on.
784−++ action :> user action
785− $% {$create nom/naem des/cord sec/security} :< create circle
786− {$delete nom/naem why/(unit cord)} :< delete + announce
787− {$depict nom/naem des/cord} :< change description
788− {$filter nom/naem fit/filter} :< change message rules
789− {$permit nom/naem inv/? sis/(set ship)} :< invite/banish
790− {$source nom/naem sub/? srs/(set source)} :< un/sub to/from src
795−`%create`: Creates a circle with description `des` and security mode `sec`. If this mode is a whitelist, the user is automatically added to it.
797−`%delete`: Deletes the circle. If a reason `why` is provided, posts that as the last message to the circle before deleting it.
799−`%depict`: Set the description of the circle to `des`.
801−`%filter`: Set the filter (message sanitation rules) for the circle to `fit`.
803−`%permit`: Either invite or banish ships to/from the circle, modifying the access control list accordingly. Regardless of whether this actually makes any changes, sends an `%inv` message to the involved ships' `%inbox`es.
805−`%source`: Add or remove sources to/from the circle, un/subscribing it to/from the `grams`, `group-l` and `config-l` each one.
809−There are two interfaces for telling Hall to send a message. The first takes entire `thought`s, the second only `speech`es and the audience to send them to.
812−++ action :> user action
813− $% {$convey tos/(list thought)} :< post exact
814− {$phrase aud/audience ses/(list speech)} :< post easy
819−`%convey`: Sends the thoughts to their audiences.
821−`%phrase`: Turns the speeches into thoughts by applying sane defaults for metadata (auto-generated `uid`, `now` for `wen`), and then sends those thoughts.
823−### Personal metadata
825−These concern the presence users have in circles they are participating in.
828−++ action :> user action
829− $% {$notify aud/audience pes/(unit presence)} :< our presence update
830− {$naming aud/audience man/human} :< our name update
835−`%notify`: Sets the user's presence in the audience circles to `pes`. A good client will automatically set these based on the user's activity. (`%talk` on typing, `%idle` on idle, `%gone` on sign-off.)
837−`%naming`: Sets the user's name (handle and real name) for the given audience. Good clients can display these in place of ship names.
839−### Changing shared UI
841−When the user makes any changes to shared UI elements (elements that should persist between clients), this has to be communicated to Hall.
844−++ action :> user action
845− $% {$glyph gyf/char aud/audience bin/?} :< un/bind a glyph
846− {$nick who/ship nic/nick} :< new identity
851−`%glyph`: Adds or removes a binding of a glyph to an audience.
853−`%nick`: Sets a local nickname for a ship. An empty nickname `''` means the ship has no nickname.
855−### Miscellaneous changes
858−++ action :> user action
859− $% {$public add/? cir/circle} :< show/hide membership
864−`%public`: Adds or removes a circle to/from the user's public membership list.
868−This document is complemented by the source code of applications that use the new Gall model (like [Hall](https://github.com/urbit/urbit/blob/master/pkg/arvo/app/hall.hoon)), but doesn't require it. Code snippets will be provided where useful. Some knowledge of Hoon and the functioning of Hoon apps is assumed.
870−New `%gall` has not yet fully solidified. As such, its structure and naming are tentative.
872−## Renewing a pillar of Arvo
874−When you write an app, Gall is responsible for making sure its event arms like `++poke` get called when they need to. In essence, it provides developers with a consistent interface for hooking their app up to the operating system.
876−It's not doing a lot to make sure this is done in a structured way, though, which results in apps (especially larger ones) feeling like big, tangled messes. Something needs to be done about this.
880−Imagine a `++poke`, `++diff`, or even a `++peer` arm. In current implementations, you'll often see their product defined through cast as the following:
882−Or (sadly) more commonly:
883−`^- {(list move) _+>.$}`
885−These arms are producing both side-effects and new state. To do this, their code has to simultaneously figure out "how does this poke/diff/whatever change our state?" and "what side-effects does this change have?"
887−You'll find that the logic relating to these two often becomes tangled, making the flow of the whole more difficult to understand. And the side-effects more often than not relate to subscriptions, for which the developer may be writing a lot of boilerplate code. Gall currently does not help in either of these places.
889−> The main significant cost of software development is the cost of untangling what the computer is doing, and or is supposed to be doing, over and over again in your head. Any way of lowering this untangling cost is extremely welcome.
895−Let's imagine a world in which Gall **does** help, in both of those places.
897−It would be wonderful if Gall could take care of all standard subscription management logic for you, and direct the flow of code as it concludes is appropriate. Applications, then, would have to help Gall by providing the different parts of this flow, and providing the checks that allow Gall to make the right decisions. This takes some work out of the developer's hands and results in more structured application code.
899−If we expand on that a bit, then we can even have Gall help us in untangling state changes from side effects. We do so by separating state changes into two phases: **analyzing** what changes need to be made, and **applying** those changes.
901−But where do the side-effects go, then? We need to realize there's two kinds: side-effects that are subscription updates, and side-effects that aren't. The latter get produces by applying state changes, while the former get integrated into the Gall flow we described above.
903−Since subscriptions are essentially queries to an app, requesting a specific part of its data/state, we can deduce subscription updates from state changes. We merely provide the logic, and Gall chains it all together. Having Gall support all this allows us to cleanly and structurally untangle state change analysis, application, and subscription logic into their own arms.
905−For trivial apps, this might result in some arms with trivial code. Though it may feel like writing cruft, it makes understanding the app just as simple as it actually as. For more complex apps, all logic is no longer a big chunk of "what do we do when x happens?" Rather, it's a few smaller chunks, like "x happened, how does that change our state?" and "if our state changes like y, what does that mean for subscription z?" These are precisely the kinds of questions that application developers should be asked.
908−## As seen in new Gall
910−Of course, for the above to be made into reality, Gall will need to see some changes. Let's outline the new structures and the arms they get passed between.
914−This wouldn't be Urbit if we didn't have cool terminology for people to learn. These ones are (for the most part) semantically sensible, and you might already see how they relate to the story above.
916−`brain`: application state.
917−`delta`: change to application state.
918−`opera`: side-effect, operation, move.
919−`query`: request (to an app) for data on a specific path.
920−`prize`: query result.
921−`rumor`: change to a query result.
923−These should be fairly straightforward. Like packets are to Urbits, `delta`s are to `brain`s and `rumor`s are to `prize`s. If one diligently updates the `prize` they received using all the `rumor`s that are relevant to it, they will always have the same `prize` as if they queried for it all over again.
927−This also wouldn't be Urbit if we couldn't boil that down to a few simple pseudocode functions. Let's try, shall we?
930−++ bake |= {brain delta} ^- {brain opera}
931−++ peek |= {brain query} ^- (unit (unit prize))
932−++ feel |= {query delta} ^- (unit rumor)
933−++ gain |= {prize rumor} ^- prize
935−:: for any given brain, delta and query:
936−.= (peek brain:(bake brain delta) query)
937− (gain rumor:(feel query delta) prize:(peek brain query))
940−There. In the first three arms we already have the bulk of new Gall! `++gain` won't be part of it, since most applications don't need to keep track of `prize`s directly. Those that do may still implement and hook it up themselves though.
942−Of course, these aren't all of the new Gall arms. Most important to still mention is `++leak`, which takes a `ship` and `query` and checks if the former has permission to ask for the latter.
944−There's also `++look` for asynchronous reads, `++hear` for subscription updates (rumors), `++fail` for dealing with process errors, `++cope` for dealing with transaction results, and `++pour` for dealing with responses from Arvo.
945−`++prep`, `++poke` and `++pull` continue to function as they do right now.
947−For a more verbose specification of all of these, see the new Gall spec.
952−To help solidify this and see what this would looks like in the wild, let's make a very simple example app. It's a counter that can go up and down, and can be queried for its value, or whether or not its value is a multiple of x. First we write all the structures that will support our app, and then we implement the arms that operate on them.
953−(The app code in its entirety can be found [here](example).)
957−Below is the application state, and the deltas we'll be using to modify it.
961−++ brain {num/@ud $~} :< application state
962−++ delta :> state change
963− $% {$increment $~} :< +1
964− {$decrement $~} :< -1
968−Fairly simple, right? We store a number, and every change either in- or decrements it.
972−Let's also define the queries we'll be supporting, their results, and the changes to their results. Instead of only allowing people to query the current number, let's make it a tad more interesting by allowing them to query whether or not the current number is a multiple of something.
975−++ query :> valid queries
976− $% {$number $~} :< current number
977− {$mul-of val/@ud} :< is num multiple of?
979−++ prize :> query results
980− $% {$number num/@ud} :< /number
981− {$mul-of mul/?} :< /mul-of
983−++ rumor :> query result changes
984− $% {$number delta} :< /number
985− {$mul-of mul/?} :< /mul-of
990−You'll see the `rumor` for a `/number` query just contains a delta. In this specific case, a change in state maps directly to a change to `prize`. For `/mul-of` queries, things aren't that simple.
992−### Responding to queries
994−Now, let's implement all the arms that will be called as things start happening. First, we want to implement `++leak`, which checks if a given `ship` is allowed to make a certain `query`. In this case, we're fine with everyone checking out our application's data.
999−++ leak :> read permission
1000− |= {who/ship qer/query}
1002− & :: everyone's allowed
1005−Next, let's create the `prize`s for the folks that make it through `++leak`. We'll be producing a `(unit (unit prize))` so we can potentially say `~` for "unavailable" and `[~ ~]` for "invalid query."
1008−++ peek :> synchronous read
1010− ^- (unit (unit prize))
1012− $number ``[%number num]
1013− $mul-of ?: =(0 mul.qer) [~ ~]
1014− ``[%mul-of =(0 (mod num mul.qer))]
1019−### Updating state & query results
1021−Our app can be queried! If they're interested enough, they'll want to receive updates on that whenever relevant state changes. First, let's see how state changes happen. We still have regular old `++poke` arms, let's use that to prompt our application to change its state.
1024−++ poke-loob :> regular old poke
1028− ?: inc [%increment ~]
1031−++ bake :> apply delta to state
1036− $increment +>(num +(num))
1037− $decrement +>(num (dec num))
1041−`delta`s get redirected into `++bake` so that our state can get changed as described. After this happens the same `delta`s get, for each active query, routed into `++feel`. That then figures out what the `rumor` relevant to the query is, if applicable.
1044−++ feel :> delta to rumor
1045− |= {qer/query del/delta}
1052− :: since we only want to send a rumor if result
1053− :: changed, we need to deduce the old state from
1054− :: the current state and the delta. depends on the
1055− :: fact that state changes before ++feel. funky!
1056− :: we could, of course, store mul/? in state, and
1057− :: make a delta for it, but should we need to?
1062− $increment (dec num)
1066− =/ new =(0 (mod num mul.qer))
1072−As we saw earlier, for `/number` queries the `delta`s match one-on-one with the relevant `rumor`s. For `/mul-of` queries we do a little bit of work to see if anything actually changed. No need to send a `rumor` if it hasn't.
1077−- Where event arms currently get passed a `wire`, and the new Gall described above passes them a custom `query` structure, the original new Gall spec gave them `(list coin)`, a "pre-parsed wire". These structures (eg `[%$ p=[p=~.ud q=123]]`) are uncomfortable to work with. Even with a "coin list parsing library" it still doesn't feel ideal. Because of custom structures being easier to work with than lists of `++knot`s or `++coin`s, writing a `++path-to-query` arm by hand is a small cost for huge gains.
1079−- Why do `prize` units mean what they do? One would expect `~` to mean "invalid" and `[~ ~]` to mean "unavailable" as opposed to the other way around.
1081−- The original new Gall spec mentions output lists (like `(list delta)` etc.) to be in reverse order. This was done because adding to the head of a list (`[item list]`) is easier than appending to the tail of a list (`(weld list [item ~])` or other). While it's technically easier to produce a list in reverse order, semantically it's yet another thing to keep in mind. Luckily, most of the "add to x, produce all of it later" code is set-and-forget, so probably not a huge deal. But do we really fear a few stray `++flop`s that much?
1083−- There's some weirdness with relation to relying on old state in `++feel` to determine whether a query result actually changed. Just relying on state in general for delta-to-rumor conversion may or may not violate the new Gall spec.
1085−- The original new Gall spec specifies `++leak` as taking a `(unit (set ship))`, where `~` means public. Is there a use case for checking more than just a single ship?
1087−- Gall should probably cache `++feel` results during a single pass, to avoid the cycles of generating what it can be certain of is the same `(unit rumor)`. (But then for eg `/circle/nom/13` and `/circle/nom/14` it would still recalculate. Better than nothing.)
1090−## The original new-Gall spec
1097−|* $: :> vinyl: historical state (including version)
1098− :> brain: working state of the application (not including version)
1099− :> delta: grain of change across all state
1100− :> prize: (pair mark noun) for namespace value
1101− :> rumor: (pair mark noun) for namespace diff
1102− :> opera: (pair bone card) for operation (old ++move)
1111−|_ $: :> ops: pending operations, in reverse order
1112− :> ego: current state
1118−++ bake :< apply delta
1119− |= $: :> del: change
1123− :> core after change (including operations)
1127−++ cope :< transaction result
1128− |= $: :> weg: forward identity
1129− :> het: success or error report
1134− :> actions in reverse order
1139−++ fail :< process error
1140− |= $: :> why: error dump
1144− :> actions in reverse order
1150− |= $: :> del: change
1151− :> pex: preparsed path, inside-first
1156− :> query updates in reverse order
1161−++ hear :< subscription update
1162− |= $: :> weg: forward identity
1166− :> actions in reverse order
1171−++ pull :< subscription cancel
1172− |= $: :> weg: forward identity
1173− :> het: error report, if any
1178− :> actions in reverse order
1183−++ leak :< check access
1184− |= $: :> lec: leakset (~ means public)
1185− :> pex: preparsed path, inside-first
1187− lec/(unit (set ship))
1190− :> if path `pex` is visible to ships in `lec`
1194−++ look :< asynchronous read
1195− |= $: :> pex: preparsed path, inside-first
1199− :> actions in reverse order
1203−++ prep :< load system
1210−++ peek :< synchronous read
1211− |= $: :> pex: preparsed path, inside-first
1215− :> value at `pec`; ~ for unavailable, [~ ~] for invalid
1217− ^- (unit (unit prize))
1220−++ poke :< generic poke
1221− |= $: :> ost: opaque cause
1222− :> msg: message with mark and vase
1227− :> actions in reverse order
1232−++ pour :< arvo response
1233− |= $: :> weg: forward identity
1234− :> sin: response card
1239− :> actions in reverse order
1246−## A new-Gall example app
1250−:: new gall example, may or may not compile.
1253−++ query :> valid queries
1254− $% {$number $~} :< current number
1255− {$mul-of val/@ud} :< is num multiple of?
1257−++ prize :> query results
1258− $% {$number num/@ud} :< /number
1259− {$mul-of mul/?} :< /mul-of
1261−++ rumor :> query result changes
1262− $% {$number delta} :< /number
1263− {$mul-of mul/?} ::TODO or just flip? :< /mul-of
1265−:: the above would ordinarily be placed in sur/, ::
1266−:: since other apps might want to use it. ::
1267−++ brain {num/@ud $~} :< application state
1268−++ delta :> state change
1269− $% {$increment $~} :< +1
1270− {$decrement $~} :< -1
1276−++ leak :> read permission
1277− |= {who/ship qer/query}
1279− & :: everyone's allowed
1281−++ peek :> synchronous read
1283− ^- (unit (unit prize))
1285− $number ``[%number num]
1286− $mul-of ?: =(0 mul.qer) [~ ~]
1287− ``[%mul-of =(0 (mod num mul.qer))]
1290−++ poke-loob :> regular old poke
1294− ?: inc [%increment ~]
1297−++ bake :> apply delta to state
1302− $increment +>(num +(num))
1303− $decrement +>(num (dec num))
1306−++ feel :> delta to rumor
1307− |= {qer/query del/delta}
1314− :: since we only want to send a rumor if result
1315− :: changed, we need to deduce the old state from
1316− :: the current state and the delta. depends on the
1317− :: fact that state changes before ++feel. funky!
1318− :: we could, of course, store mul/? in state, and
1319− :: make a delta for it, but should we need to?
1324− $increment (dec num)
1328− =/ new =(0 (mod num mul.qer))
1333−:: not pictured: ++path-to-query