@@ -0,0 +1,268 @@
1+---
2+date: ~2016.10.6
3+type: post
4+title: Urbit monthly update: "hard tech" and the personal ledger
5+author: Curtis Yarvin
6+layout: urbit,post
7+navmode: navbar
8+navdpad: false
9+navselect: blog
10+navpath: /
11+navhome: /
12+navclass: urbit
13+---
14+<br /><br />
15+
16+September's monthly update slipped into October! But don't
17+worry, it's a good one.
18+
19+## "Hard tech"
20+
21+In [this amusing
22+talk](http://www.themacro.com/articles/2016/09/chad-rigetti-on-building-quantum-computers/),
23+Chad Rigetti defines "hard tech" as "solving problems you don't
24+even know are solvable." Rigetti (who is not only named "Chad",
25+but looks like he should be named "Chad") is building a quantum
26+computer, which is definitely "hard tech." More power to him!
27+
28+But software can be "hard," too. Is Urbit "hard tech"?
29+
30+The correct answer at this point is: no, but it *used* to be hard
31+tech. Now it's actually real, if not exactly easy. The Urbit
32+network hasn't crashed once this summer, though we did one
33+intentional breach (right before delivering the stars). I'm
34+still kind of amazed by this.
35+
36+But we still need to get to the point where (a) the whole network
37+can't crash ever, (b) you can put your data in an urbit and be
38+pretty sure it'll live forever, (c) Urbit works well as a social
39+network, and therefore (d) we can distribute planets to ordinary
40+human beings and not have them be like, "gah, what is this?"
41+
42+If we had the resources of Microsoft or Google, or even of
43+Mozilla, we could solve these problems with a brute-force
44+human-wave attack. We could just debug the system into
45+existence. It must be nice.
46+
47+But not. In some ways it's essential to Urbit's mission that
48+there isn't a giant army behind it. It means that the only way
49+Urbit can succeed, or even survive, is to be excellent. That's
50+the right attitude for a stack designed for the 2020s. Not that
51+we can wait until the 2020s to get (a) through (d). Just that we
52+don't want our children and grandchildren to curse the day this
53+stack was born. Unix, the Internet and the Web gained the world
54+and lost their souls -- we don't have that option.
55+
56+Given this attitude, the thing that terrified me most of all
57+about this summer's crowdsale is that the most important parts of
58+Arvo -- the network and the PKI -- simply didn't exist in any
59+viable sense. Yes, the present `%ames` network does net work.
60+There is even in theory a sort of PKI, although no one has ever
61+even tried to revoke a key. It might be secure -- who knows?
62+
63+More concretely, the existing `%ames` vane (kernel module) contains
64+both the protocol and the PKI, mixed together in a single giant
65+state machine of the sort I've learned never to build again.
66+There is no perfect code anywhere in the present production Arvo,
67+but the only body of *dead-end* code -- which needs replacement,
68+not documentation and maintenance -- is `%ames`.
69+
70+Unfortunately, networking and key management is the most
71+important part of Urbit. In a sense it *is* Urbit. So having it
72+not exist, except as a design in our heads, was not a good thing.
73+
74+While documenting, evangelizing and in general growing Urbit is
75+vastly important, what's even more important is finishing it --
76+without compromising its elegance. Elegance is the only thing
77+Urbit has. It's not like there aren't other operating systems.
78+
79+So the first thing I did after the crowdsale was to disappear
80+into a hole and take care of this technical debt. This was not
81+"hard tech" in the sense that I knew it was solvable. I didn't
82+know it was solvable *well*, though. And it has to be solved
83+*well*, because otherwise who cares.
84+
85+## Check out our new, improved cc-release
86+
87+I'm delighted to report that the new, rewritten `%ames` network,
88+and the all-new `%jael` PKI, are now at code complete. (Code
89+complete means "compiling but not tested," which is actually
90+saying a lot in a typed functional language.) They've been
91+checked in to a new `cc-release` branch on
92+`github.com/urbit/arvo`.
93+
94+I think these two vanes are definitely the best code in Urbit.
95+There's still a bit of fundamental work in Arvo (implementing the
96+boot sequence and event security mechanisms described in the
97+whitepaper), but this is not "hard tech" -- the designs are
98+clear and surprises seem unlikely.
99+
100+## %xmas: the new network
101+
102+New `%ames` (currently `arvo/xmas.hoon`, since old `%ames` is still in
103+the build) has had all the key management stripped out; it's just
104+a network vane. It's also no longer a single giant state
105+machine, but a system of five simple 200-line state machines.
106+
107+It was wonderful to discover that Hoon lets you write purely
108+functional programs with what are essentially global variables.
109+Specifically, nested cores generalize across global variables,
110+local variables, and objects, in a way I find technically quite
111+lovely if I do say so myself. On the other hand, Hoon ain't
112+magic, and it's not like we've solved any of the
113+software-engineering problems inherent in a 2000-line state
114+machine based on global variables. So, nice to kill that.
115+
116+Don't look at new `%ames` just yet. It was done in July and
117+August. It needs a few touchups to fit the `%jael` interface.
118+And it needs to be brought up to `%jael`'s quality standards.
119+
120+## %jael: reference code, PKI, and personal ledger
121+
122+`%jael` (`/arvo/jael.hoon`), the PKI, is what I'm really excited
123+about. It's what you should take a look at if you know any Hoon
124+or even if you don't. There's several really important things
125+about `%jael`.
126+
127+The first is that it's actually reference-quality code. Or so I
128+believe. It's literally intended to serve as a reference style
129+for what all production-quality Hoon should look like. If you're
130+reading the code and you see anything that isn't absolutely
131+perfect, whether it's cosmetic or structural or anything in
132+between, file a PR. `%jael` is fully commented -- if you think
133+it needs more comments, or fewer, file a PR...
134+
135+Needless to say, a reference style will be an enormous help in
136+managing community contributions. It's not that, for now at
137+least, we won't accept patches in the reference `%jael` style.
138+But we'll always accept patches to bring those patches up to
139+reference. For the moment, the code will just define the style
140+by analogy, but it'll eventually spawn an actual document.
141+
142+If you're feeling ambitious and want to change the reference
143+style itself, your change is best submitted as a patch to `%jael`
144+itself. And *really* ambitious people can create their own
145+alternative Hoon styles -- hopefully defined at the same same
146+level of precision.
147+
148+Second, `%jael` actually implements the Urbit PKI as designed,
149+right down to unusual but politically critical features like
150+escaping a bad parent. Since the code is readable and heavily
151+commented, it can and will serve as the best possible reference
152+for the PKI's semantics.
153+
154+Both `%ames` and `%jael` are designed for the new and correct
155+single-homed urbit model, where one pier is one ship. In the new
156+world, to boot your planet, you use your ticket to generate a
157+symmetric key that authenticates you to your parent. When you
158+share your public key with your parent, over that initial secure
159+channel, it notices that it hasn't signed it yet and shares its
160+signature back to you.
161+
162+Third, `%jael` isn't just a PKI -- it's a generalized system for
163+formal promises. The whole point of a PKI is that entities can
164+make secure statements to each other. Of course, every `:talk`
165+message is a signed statement, but the really important messages
166+are *promises* -- such as, for instance, "Tlon owes you 3 stars."
167+
168+(Right now, the stars we sold in the crowdsale are recorded in the
169+state of a higher-level application (`:womb`). (We have an
170+encrypted offline backup, of course.) This is fine at a certain
171+level, but it's a bit rickety and not really up to Urbit's
172+standards of architectural elegance. Promises and secrets are
173+fundamental state and need to be handled at a fundamental level.)
174+
175+In a secure distributed social network, people need to be able to
176+make formal promises to each other. Essentially, every ship is
177+an *accounting entity*. It records its liabilities, which are
178+the promises it's made to others, and its assets, which are the
179+promises others have made to it. When I update the promises I've
180+made to you, I send you an updated statement.
181+
182+All kinds of promises fit this mold. For instance, while you
183+don't usually think of a symmetric key as a promise from one
184+entity to another, it is. It's a promise from ship A to ship B,
185+that B can identify itself to A by this secret.
186+
187+So `%jael` will, among other glorious things, essentially allow
188+every galaxy to serve as an exchange for star invitations, and
189+any star to serve as an exchange for planet invitations. Mere
190+payments are just the start of what you can do with this. And of
191+course, one of our big near-term goals is to make first Urbit
192+invitations, then (once the crypto is truly final) actual ships,
193+tradeable on "regular" cryptocurrency exchanges.
194+
195+One of my views about Bitcoin is that its decentralized,
196+trust-free ledger isn't really the most important thing about it.
197+Humans build trust quite easily. For instance, Bitcoin doesn't
198+even enable trust-free payments -- a payment is always an
199+exchange for other good or service. Do I provide the good or
200+service before, or after, the Bitcoin payment? Either way,
201+trust is unavoidable.
202+
203+The important thing about Bitcoin is just that it enables users
204+to manage their obligations to each other digitally, without
205+interacting with a 20th-century asset management system made of
206+paper and magnetic tape. Giving every user their own asset
207+management ledger is another small revolution in that direction.
208+And if I'm defining and issuing my own digital assets, why do I
209+need to put them on a global blockchain? You're still trusting
210+me to actually live up to my promises.
211+
212+Last but not least, `%jael` solves one of Urbit's most important
213+and hardest problems: how do I back up my urbit? I know I said
214+this was impossible, but I lied.
215+
216+Urbit uses persistent sessions, which means a pair of ships uses
217+the same session sequence numbers for life. Since every packet
218+is a transaction, when you send an acknowledgment, you mean it.
219+The upside is that we get exactly-once messaging, which if you've
220+ever done distributed programming is a very, very good thing.
221+The downside is that if you lose your state, there's no way to
222+recover it, since you don't know what sequence number to send
223+next. (See under: [Two Generals'
224+Problem](https://en.wikipedia.org/wiki/Two_Generals%27_Problem)).
225+
226+In the new world, the Unix layer will subscribe to a stream of
227+`%jael` changes which contain all certificate and rights actions.
228+We'll save this separately from the rest of the pier state, and
229+it's small and trivial to back up.
230+
231+When restoring, if you have all your private keys but have lost
232+all other state (obviously if you don't have the latest private
233+key, you're hosed), you sign a new version of your will which
234+contains a "brain damage" bit. When this will is propagated to
235+your neighbors, they learn that you're brain-damaged and reset
236+their sequence numbers -- and forget you know anything they told
237+you, treating you as a new stranger.
238+
239+At the application level, all your subscriptions get closed.
240+Apps already have to deal with this. Forward messages shouldn't
241+involve any state coupling; that's what subscriptions are for.
242+In case there is some coupling anyway, you'll get an event that
243+"ship X lost his mind, forget anything you know about him."
244+
245+Essentially, we're cheating the fact that you can't have
246+transient sessions and [exactly-once
247+delivery](http://bravenewgeek.com/you-cannot-have-exactly-once-delivery/)
248+by letting the abstraction leak in this special case, and putting
249+the responsibility for fixing it on the brain-damaged ship. It
250+may even leak up to the user. But dude, you're the one that let
251+your brain fall out of your skull.
252+
253+Another thing I like about this solution (and about `%jael` in
254+general) is that it solves the problem of how you use someone
255+else's secrets, if you manage to steal them. There's no point in
256+security through technical difficulty -- if you have the latest
257+private key for a ship, you're the captain now. You shouldn't
258+have to hack something up to make this work.
259+
260+## Keep in touch / office hours
261+
262+`:talk` is always a great place to come and ask questions.
263+Someone from Tlon will be available in `:talk` from 12PM - 3PM
264+PDT each weekday. And [our forum](https://urbit.org/fora) is a
265+great place to post questions.
266+
267+For anything else you can always get in touch with us via
268+[`urbit@urbit.org`](mailto:urbit@urbit.org).