@@ -1,36 +1,59 @@
11
2−# The Gall Delta Model
2+# The new Gall model
33
4−This document is complemented by the source code of applications that use the new gall model (like hall), but doesn't require it. Code snippets and diagrams will be provided where useful. Some knowledge of the functioning of Hoon apps is assumed.
4+This document is complemented by the source code of applications that use the new gall model (like hall), but doesn't require it. Code snippets and diagrams will be provided where useful. Some knowledge of Hoon and the functioning of Hoon apps is assumed.
55
6−Naming of functions and molds in this document is tentative.
6+New gall has not yet fully solidified. As such, structure and naming are tentative to change.
77
88## Table of contents
99
1010@TODO
1111
12−## Introduction
12+## Renewing a pillar of Arvo
13+
14+When you write an app, gall is responsible for making sure its event arms like `++poke` get called when they need to. In essence, it provides developers with a consistent interface for hooking their app up to the operating system.
15+It's not doing a lot to make sure this is done in a structured way though, which results in apps (especially larger ones) feeling like big tangled messes. Something needs to be done about this.
16+
17+### Problems
1318
1419Imagine a `++poke`, `++diff`, or even a `++peer` arm. In current implementations, you'll often see their product defined through cast as the following:
1520`^- (quip move +>)`
1621Or (sadly) more commonly:
1722`^- {(list move) _+>.$}`
1823
19−These arms are producing both side-effects and new state. To do this, the application has to simultaneously figure out "how does this poke/diff/whatever change our state?" and "what side-effects does this change have?". You'll find that the logic relating to these two often becomes tangled, making the flow of the whole more difficult to understand. Gall currently does not help here.
24+These arms are producing both side-effects and new state. To do this, their code has to simultaneously figure out "how does this poke/diff/whatever change our state?" and "what side-effects does this change have?".
25+You'll find that the logic relating to these two often becomes tangled, making the flow of the whole more difficult to understand. And the side-effects more often than not relate to subscriptions, for which the developer may be writing a lot of boilerplate code. Gall currently does not help in either of these places.
26+
27+> The main significant cost of software development is the cost of
28+untangling what the computer is doing, and or is supposed to be doing,
29+over and over again in your head. Any way of lowering this untangling
30+cost is extremely welcome.
31+
32+-- *~sorreg-namtyv*
33+
34+### Solutions
2035
21−@TODO better segue
22−What if we could separate these two, untangling state changes from side effects? And if we plucked state changes apart into *calculating* and *applying* the change, wouldn't we be able to deduce side-effects from calculated changes? Enter New Gall.
36+Let's imagine a world in which gall *does* help, in both those places.
2337
24−@TODO better argumentation, lay out advantages of change, etc.
38+It would be wonderful if gall could take care of all standard subscription management logic for you, and direct the flow of code as it concludes is appropriate.
39+Applications, then, would have to help gall by providing the different parts of this flow, and the checks that allow gall to make the right decisions. This takes some work out of the developer's hands and results in more structured application code.
2540
26−State gets changed, subscriptions need to be updated to reflect that, etc.
41+If we expand on that a bit we can even have gall help us in untangling state changes from side effects. We do so by separating state changes into two phases: *analyzing* what changes need to be made, and *applying* those changes.
42+But where do the side-effects go then? We need to realize there's two kinds: side-effects that are subscription updates, and those that aren't. The latter get produces by applying state changes, while the former get integrated into the gall flow we described above.
43+Since subscriptions are essentially queries to an app, requesting a specific part of its data/state, we can deduce subscription updates from state changes. We merely provide the logic, and gall chains it all together.
2744
28−Queries: we're just getting data (and maybe staying up to date on it)!
45+Having gall support all this allows us to cleanly and structurally untangle state change analysis, application, and subscription logic into their own arms.
46+For trivial apps, this might result in some arms with trivial code. Though it may feel like writing cruft, it makes understanding the app just as simple as it actually as.
47+For more complex apps, all logic is no longer a big chunk of "what do we do when x happens?", but rather a couple of smaller chunks, like "x happened, how does that change our state?" and "if our state changes like y, what does that mean for subscription z?" These are precisely the kinds of questions that application developers should be asked.
2948
3049
31−## Structure
50+## As seen in new gall
3251
33−This wouldn't be Urbit if we had some cool new terminology for people to learn. These aren't super obscure though, don't worry.
52+Of course for the above to be made into reality, gall will need to see some changes. Let's outline the new structures and the arms they get passed between.
53+
54+### Structures
55+
56+This wouldn't be Urbit if we had some cool new terminology for people to learn. These ones are (for the most part) semantically sensible though, and you might already see how these relate to the story above.
3457
3558`brain`: application state.
3659`delta`: change to application state.
@@ -39,30 +62,38 @@ This wouldn't be Urbit if we had some cool new terminology for people to learn.
3962`prize`: query result.
4063`rumor`: change to a query result.
4164
42−These should be fairly straightforward. The idea of `brain`+`delta` and `prize`+`rumor` is comparable to Urbit's "state is a function of facts" way of thinking. If I diligently update the `prize` I got using the `rumor`s, then I won't ever need to `query` again to pull down the full correct `prize`.
65+These should be fairly straightforward. Like packets are to Urbits, `delta`s are to `brain`s and `rumor`s are to `prize`s. If one diligently updates the `prize` they received using all the `rumor`s that are relevant to it, they will always have the same `prize` as if they queried for it all over again.
4366
44−This also wouldn't be Urbit if we couldn't boil that down to a few simple functions. Let's try, shall we?
67+### Arms
68+
69+This also wouldn't be Urbit if we couldn't boil that down to a few simple pseudocode functions. Let's try, shall we?
4570
4671```
4772++ bake |= {brain delta} ^- {brain opera}
4873++ peek |= {brain query} ^- (unit (unit prize))
4974++ feel |= {query delta} ^- (unit rumor)
5075++ gain |= {prize rumor} ^- prize
76+
77+:: for any given brain, delta and query:
78+.= (peek brain:(bake brain delta) query)
79+ (gain rumor:(feel query delta) prize:(peek brain query))
5180```
5281
53−The first three are part of new gall. `++gain` usually won't be implemented in that exact way, but needs to be possible still, to illustrate our functional definitions. The following two statements should be equal, for any given brain, query and delta:
82+There, in the first three arms we already have the bulk of new gall! `++gain` won't be part of it, since most applications don't need to keep track of `prize`s directly. Those that do may still implement and hook it up themselves though.
5483
55−```
56−(peek brain:(bake brain delta) query)
57−(gain rumor:(feel query delta) (peek brain query))
58−```
84+Of course, these aren't all of the new gall arms. Most important to still mention is `++leak`, which takes a `ship` and `query` and checks if the former has permission to ask for the latter.
85+There's also `++look` for asynchronous reads, `++hear` for subscription updates (rumors), `++fail` for dealing with process errors, `++cope` for dealing with transaction results, and `++pour` for dealing with responses from arvo.
86+`++prep`, `++poke` and `++pull` continue to function as they do right now.
87+For a more verbose specification of all of these, see the new gall spec.
5988
6089
6190## By example
6291
92+To help solidify this and see what this would looks like in the wild, let's make a very simple example app. It's a counter that can go up and down, and can be queried for its value, or whether or not its value is a multiple of x. First we write all the structures that will support our app, and then we implement the arms that operate on them.
93+(The app code in its entirety can be found [here](gall-eg.hoon).)
94+
6395### State & deltas
6496
65−To help solidify this, let's look at a very simple example app. We'll be making a counter that can go up and down. (The app code in its entirety can be found [here](gall-eg.hoon).)
6697Below is the application state, and the deltas we'll be using to modify it.
6798
6899```
@@ -98,7 +129,7 @@ Let's also define the queries we'll be supporting, their results, and the change
98129
99130You'll see the `rumor` for a `/number` query just contains a delta. In this specific case, a change in state maps directly to a change to `prize`. For `/mul-of` queries, things aren't that simple.
100131
101−### Functionality
132+### Responding to queries
102133
103134Now, let's implement all the arms that will be called as things start happening. First, we want to implement `++leak`, which checks if a given `ship` is allowed to make a certain `query`. In this case, we're fine with everyone checking out our application's data.
104135
@@ -124,7 +155,9 @@ Next, let's create the `prize`s for the folks that make it through `++leak`. We'
124155 ==
125156```
126157
127−Our app has been queried! If they're interested enough, they'll want to receive updates on that whenever relevant state changes. First, let's see how state changes happen. We still have regular old `++poke` arms, let's use that to prompt our application to change its state.
158+### Updating state & query results
159+
160+Our app can get queried! If they're interested enough, they'll want to receive updates on that whenever relevant state changes. First, let's see how state changes happen. We still have regular old `++poke` arms, let's use that to prompt our application to change its state.
128161
129162```
130163++ poke-loob :> regular old poke
@@ -159,6 +192,8 @@ Our app has been queried! If they're interested enough, they'll want to receive
159192 :: changed, we need to deduce the old state from
160193 :: the current state and the delta. depends on the
161194 :: fact that state changes before ++feel. funky!
195+ :: we could, of course, store mul/? in state, and
196+ :: make a delta for it, but should we need to?
162197 =/ old
163198 .= 0
164199 %+ mod
@@ -176,26 +211,24 @@ Our app has been queried! If they're interested enough, they'll want to receive
176211As we saw earlier, for `/number` queries the `delta`s match one-on-one with the relevant `rumor`s. For `/mul-of` queries we do a little bit of work to see if anything actually changed. No need to send a `rumor` if it hasn't.
177212
178213
179−## Under the hood
180−
181−These arms get chained together by gall. `++cope`(?) can be implemented to accept `prize`s. `++hear`(?) can be implemented to accept `rumor`s.
182−@TODO probably show examples?
183−
184−One thing that wasn't shown was a `++path-to-query` arm. This is responsible for converting a query path to a query object as defined in the application's structures.
214+## Criticism
185215
216+* Where event arms currently get passed a `wire`, and the new gall described above passes them a custom `query` structure, the original new gall spec gave them `(list coin)`, a "pre-parsed wire". These structures (eg `[%$ p=[p=~.ud q=123]]`) are uncomfortable to work with. Even with a "coin list parsing library" it still doesn't feel ideal.
217+Because of custom structures being easier to work with than lists of knots or coins, writing a `++path-to-query` arm by hand is a small cost for huge gains.
186218
187−## Criticism
219+* Why do `prize` units mean what they do? Would have expected `~` to mean "invalid" and `[~ ~]` to mean "unavailable" as opposed to the other way around.
188220
189−`coin`s are a pain to work with. The "coin list parsing library" makes this a bit easier, but ideally we'd just define a `++path-to-query` arm and do everything as described above.
221+* The original new gall spec mentions output lists (like `(list delta)` etc.) to be in reverse order. This was done because adding to the head of a list (`[item list]`) is easier than appending to the tail of a list (`(weld list [item ~])` or other).
222+While it's technically easier to produce a list in reverse order, semantically it's yet another thing to keep in mind. Luckily, most of the "add to x, produce all of it later" code is set-and-forget, so probably not a huge deal, but do we really fear a few stray `++flop`s that much?
190223
191−Why do prize units mean what they do? Would've expected it the other way around.
224+* There's some weirdness with relation to relying on old state in `++feel` to determine whether a query result actually changed. Just relying on state in general for delta-to-rumor conversion may or may not violate the new gall spec.
192225
193−Not sure if I'm a fan of "in reverse order". It's technically easier, but semantically yet another thing to keep in mind. Luckily, most of the "add to x, produce all of it later" code is set-and-forget, so probably not a huge deal.
226+* The original new gall spec specifies `++leak` as taking a `(unit (set ship))`, where `~` means public. Is there a use case for checking more than just a single ship?
194227
195−`++bake` should just produce `(quip move +>)`. why would we want to be forced to keep operas in state? just to avoid [~ +>] over +>?
228+* Gall should probably cache `++feel` results during a single pass, to avoid the cycles of generating what it can be certain of is the same `(unit rumor)`. (But then for eg `/circle/nom/13` and `/circle/nom/14` it would still recalculate. Better than nothing.)
196229
197−weirdness wrt relying on (old) state in ++feel
198230
199−gall should probably cache `++feel` results during a single pass. (but then for eg /circle/nom/13 and /circle/nom/14 it would still recalculate. better than nothing though.)
231+## Further reading
200232
201−Lots of cruft for small applications to bother with. (Though I admit it does feel like structured cruft, at least.)
233+@TODO original gall-spec.hoon / heed.hoon
234+@TODO proposed gall-spec.hoon / heed.hoon