@@ -0,0 +1,279 @@
1+---
2+date: ~2016.8.30
3+type: post
4+title: Urbit monthly update (August)
5+author: Curtis Yarvin
6+layout: urbit,post
7+comments: true
8+navmode: navbar
9+navdpad: false
10+navselect: blog
11+navpath: /
12+navhome: /
13+navclass: urbit
14+---
15+<br /><br />
16+
17+Here's August's monthly letter from Tlon, only slightly late.
18+We've been doing a lot of planning and, unfortunately, a little
19+bit of research.
20+
21+We hope we're done with this research stuff. There is really
22+nothing more unpleasant than CS research. But the protocol and
23+crypto stated (`%ames` and `%jael`) just needed another round of
24+basic architecture. This is fortunately now done. We think.
25+
26+Also, we have more to say than we expected. The concept of an
27+"update" only makes sense when everyone already knows what's
28+going on. The first update is just the status quo.
29+
30+We'll be issuing *four* public letters in the next month or two:
31+these infrastructure milestones; an application/user-experience
32+MVP strategy; a documentation/education plan; and an upbeat,
33+super-motivating corporate PR narrative suitable for television.
34+You can skip the last one if you like.
35+
36+## Infrastructure milestones
37+
38+By "infrastructure" we roughly mean everything in `/===/arvo,`
39+ie, the kernel and vanes.
40+
41+Infrastructure development in the presently foreseeable future is
42+a system of four milestones, the first of which we've hit. We
43+have a reasonably good start on the second.
44+
45+The first milestone (August 15) was "first inception" (FI), where
46+we claimed our true and final urbits.
47+
48+The next milestone is a "continuity candidate" (CC), a release
49+where it's at least possible that we'll need no further
50+continuity breaches and/or reinstalls. CC requires a nontrivial
51+amount of further engineering work.
52+
53+Sometime after that comes "final continuity" (FC), where we
54+explicitly renounce continuity breaches. CC to FC is a testing
55+milestone and is satisfied by collective judgment.
56+
57+The last milestone is "security freeze" (SF), where we declare
58+the PKI final and issue live keys.
59+
60+## FI: first inception
61+
62+The outcome of the FI milestone was a working network, with
63+multiple federated stars and final `@p` names. (Modulo some
64+justified complaints about the `lut` suffix.) Federation
65+probably means we can invite a lot more people to Urbit and it'll
66+keep working, but let's still be a little cautious.
67+
68+The FI keys are not final, but the cryptosystem is (though there
69+may be still be bugs in our implementation).
70+
71+The Urbit network works and is an adequate, if still quite buggy,
72+social network, with two modes of social communication: chat and
73+long-form text with comments.
74+
75+People can learn to program in Hoon and even write %gall apps,
76+but the documentation is lacking and involves wide gaps, which
77+must be filled by reverse engineering or public assistance.
78+Fortunately the Tlon people are reputed to be quite helpful.
79+
80+At least one breach is certain. All data in your FI urbits will
81+be destroyed! In the breach to the CC release, we will port the
82+address allocation registry (the urbit.org womb) manually to the
83+new network.
84+
85+If you have been spawning planets from your stars, you'll need to
86+follow our procedures, which we'll post before the next breach.
87+Emergency breaches may also happen. We recommend against
88+spawning at the moment, but of course your star is yours.
89+
90+Data in FI urbits should also be considered completely insecure.
91+(In fact, the urbit process probably should be run in a VM if
92+there is any other important data on the machine, since we have
93+not done any real security analysis and network data is used.)
94+
95+## CC/FC: continuity
96+
97+The CC release at least attempts to maintain continuity. What is
98+continuity?
99+
100+Continuity is when we stop rebooting the universe. You can leave
101+a continuous urbit turned off on a USB key in your attic for 10
102+years. When you activate it on in 2026, it will probably spend
103+several days upgrading itself, but it will work.
104+
105+CC is just a candidate. FC is when we actually believe we've
106+achieved continuity.
107+
108+Here are some features we've lumped into the CC/FC bag. This
109+list may expand, but hopefully won't contract.
110+
111+Urbit is a reliable datastore. For instance, as a developer, you
112+should edit your source files in the Urbit filesystem `%clay`,
113+then back them up with `git` via a one-way mirror. (Most people
114+now edit outside Urbit and copy trees in.) Of course, `%gall`
115+application state you create in a FC urbit can and should also
116+last forever, via state adapters.
117+
118+Urbit has a reliable storage system that can use a cluster log
119+like Kafka, does not lose data even on Linux, encrypts the image
120+as well as the log, and allows log pruning.
121+
122+Urbit handles all crashes correctly on all platforms. A stack
123+overflow or interrupt always produces a usable trace.
124+
125+Urbit boots with the proper, beautiful, functional, boot sequence
126+defined in the whitepaper. It's single-homed -- designed to
127+operate one ship per kernel -- and uses the event security model
128+from the whitepaper.
129+
130+Urbit has a working end-to-end "permanent networking" message
131+system with in-order, exactly-once delivery and end-to-end acks,
132+and perfect causal consistency over the wire with no head-of-line
133+blocking between different ducts.
134+
135+Urbit no longer mixes cryptosystem code with networking code.
136+Secret storage moves into a separate `%jael` vane. `%jael` will
137+hold public, private and symmetric Urbit keys, foreign API keys,
138+tokens, passwords, etc; and other capabilities, like Urbit
139+invitations.
140+
141+Urbit has a communication recovery procedure so that a
142+brain-damaged ship can resume communication after it loses
143+information, even if all it knows is its own identity and key.
144+Only-once delivery is violated, but the onus for recovery is
145+placed on the damaged ship rather than its healthy neighbor.
146+
147+Urbit has have a FUSE-oriented filesystem interface to the
148+local OS, rather than using libuv filesystem monitoring.
149+
150+Urbit has a mark system that supports mark containers, and
151+handles mark evolution properly in both the network and the
152+filesystem.
153+
154+Urbit handles library versioning properly in `%ford`, never using
155+`=>` to load libraries, using aliases correctly to allow multiple
156+conflicting versions in a dependency graph.
157+
158+Urbit has a stable, extensible but backward-compatible I/O
159+protocol between Arvo and vere (the Unix process). Any future
160+Arvo will work on an old vere.
161+
162+Urbit uses Murmur32 rather than FNV as a short hash (mug).
163+
164+Urbit is still not a cryptographically secure system, and doesn't
165+allow true cryptographic ownership transfers. The cryptographic
166+transfer features exist, but using them is a bad idea until the
167+keys are final.
168+
169+(For right now, everyone who issues urbits to others needs to at
170+least track ownership centrally, ideally with a mirror database
171+outside Urbit. We could not stress this more stressfully!)
172+
173+Urbit has received a thorough performance and scaling audit, and
174+there are no trivial efficiency problems to solve.
175+
176+The Urbit console can output colored text and control a 2D ASCII
177+text array (curses style). The web console (not web dojo) works
178+with some keylag.
179+
180+All Urbit vanes are thoroughly documented/specified with an
181+overview document and inline comments.
182+
183+## SF: security freeze
184+
185+The SF release will render Urbit a "live" cryptographic system.
186+It is unnecessary to point out that all subsystems in the
187+security chain must be audited, and operational procedures for
188+stars and galaxies need to be standardized.
189+
190+(Yes, the SF release needs to invalidate the current
191+cryptographic structure, even if it hasn't been compromised.
192+Since it's not secure, we don't know it hasn't been compromised.)
193+
194+However, the SF release keys will be instantiated via network
195+update, which is essentially "god mode." New keys will be
196+distributed to old key holders, and any disparities between
197+cryptographic and centralized ownership records will be resolved
198+in favor of the latter. Before SF, no one can gain anything by
199+stealing a key or transferring it without authorization.
200+
201+A software update can of course do anything to the PKI. After
202+SF, at least, global Arvo updates (which ideally can become rarer
203+and rarer) will need to be approved through a constitutional,
204+republican governance process.
205+
206+After the SF release, we have no control over cryptographic
207+transfers, but the Arvo code itself may penalize long wills to
208+minimize velocity and keep Urbit from acting as a currency.
209+(This is essentially a Tobin tax.)
210+
211+The SF release needs to run the actual Arvo interpreter, with
212+jets, in a completely sandboxed subprocess with no I/O privilege.
213+Sandboxing jets, or even a safe jet language, should also be
214+investigated. Network jet update must also be possible.
215+
216+An SF urbit can issue and manage children without any problem.
217+It can be hosted easily in a commercial data centre. The
218+interpreter can migrate images securely, automatically and
219+efficiently to a different hosting service.
220+
221+An SF urbit will have a framework for DoS defence, blocking and
222+banning, and other nuisances of the modern world. There will be
223+at least the beginning of a global reputation system.
224+
225+## CC: work overview
226+
227+The one subsystem that needs a complete rewrite is `%ames` (the
228+network). The `%jael` vane is completely new. Fortunately, the
229+new `%ames` vane compiles, and the `%jael` interface is defined
230+(and not that `%jael` can be complicated).
231+
232+The mug replacement work is also done, but needs to be debugged
233+(which is no easy task).
234+
235+Applying the canonical boot event system sounds like a giant
236+change, but it's actually pretty straightforward -- same for
237+single-homing. Arvo proper is only a few hundred lines of Hoon.
238+
239+The event system in `vere` needs to be replaced, but we have a
240+module built by a contractor with both the local FS and Kafka.
241+
242+All the vanes besides `%ames` need documentation and a general
243+tuneup, but this should not rise to the level of a rewrite for
244+any other vane.
245+
246+## Community involvement
247+
248+Urbit's most serious problem right now is that many more people
249+are interested in it than are actually working on it. This
250+reflects its heritage as a basement project.
251+
252+Our most important work item remains opening up the codebase so
253+that community development can happen. We'll start turning the
254+CC tasks into community operations as fast as possible.
255+
256+But this is a separate topic. It fits under a discussion of
257+Tlon's corporate plans and operations, which we'll post later.
258+
259+## Contributors
260+
261+The contributors have been amazing. We need to make more use of
262+their talents, which are growing daily. And we need to do a
263+better job of chronicling their deeds! But for now, github
264+remembers.
265+
266+## Keep in touch / office hours
267+
268+`:talk` is always a great place to come and ask questions. Going
269+forward someone from Tlon will be available in `:talk` from 12PM - 3PM
270+PDT each weekday.
271+
272+Post to our [new forum](https://urbit.org/fora)! Don't be
273+afraid, the water's fine! Actually there are a few glitches,
274+but it's hosted on Urbit (and also backed up). For the less
275+daring, [r/urbit](https://www.reddit.com/r/urbit) is a reliable
276+old standby.
277+
278+For anything else you can always get in touch with us via
279+[`urbit@urbit.org`](mailto:urbit@urbit.org).