on master help sign in
New posts.
af872d2C. Guy Yarvin
main/blog/blockchain.mdnew file+616
@@ -0,0 +1,616 @@
1+# Why Urbit (probably) doesn't need a blockchain
2+
3+TLDR: Urbit (probably) doesn't need a blockchain, because the
4+Urbit address-space PKI is a special case of a consensus ledger
5+(itself a special case of Byzantine agreement).
6+
7+Special cases are often easier to solve than general problems.
8+The "New Jersey" or "worse-is-better" school of systems design
9+emphasizes solving any problem with the weakest possible tools.
10+
11+## The worse-is-better school
12+
13+Culturally, Urbit is best described as a functional stack built in
14+the "New Jersey" school of system software design, as described
15+in Richard Gabriel's famous ["worse is
16+better"](https://en.wikipedia.org/wiki/Worse_is_better) essay.
17+
18+There are two kinds of mindsets when approaching hard technical
19+problems. The "MIT" mindset, more typical among mathematicians
20+and scientists, wants to attack the problem as its simplest and
21+broadest case, producing the most general and powerful solution
22+available. In short: better-is-better.
23+
24+The "New Jersey" worse-is-better mentality, more typical among
25+programmers and engineers, wants to do everything possible to
26+*not* attack the problem. Can I produce a serviceable
27+approximation to a true solution? Do I actually have to solve
28+the general problem, or can I reduce it to an easier special
29+case? Can I change the product so it doesn't have this problem?
30+
31+Ideally, anyone building anything has both these mindsets in
32+their toolbox. But it's worth noting that (a) Jersey designs
33+(meaning the Bell Labs school that produced Unix, C, and Go) tend
34+to beat MIT designs in the marketplace, and (b) there are very
35+few pure-functional systems built with the true Jersey spirit.
36+
37+## Why Urbit (probably) doesn't need a blockchain
38+
39+Nakamoto consensus is an example of better-is-better. The
40+blockchain design is actually more powerful than it needs to be.
41+It's not just a trustless decentralized ledger; it's a trustless
42+decentralized database and (as Ethereum has realized; but this
43+idea is clearly present in Bitcoin's scripting language) a
44+general-purpose, zero-trust decentralized computer.
45+
46+Again, Urbit is not a general-purpose decentralized database or
47+consensus computer. It's a special case of the former, which is
48+a special case of the latter. Urbit is nothing but a digital
49+title chain for network addresses. Our belief is that the
50+special case is much easier than the general case, which is why
51+Urbit (probably) doesn't need a blockchain.
52+
53+The rest of this essay describes what makes this special case
54+special, and how we can use that to not need full Nakamoto
55+consensus, or any kind of general-purpose Byzantine agreement.
56+
57+## The double-sell problem
58+
59+Urbit has the same basic "double-spend" issue as Bitcoin. As
60+Satoshi puts it: "The problem of course is the payee can't verify
61+that one of the owners did not double-spend the coin." (Satoshi's
62+grammar is usually better than this.) But Urbit is not a
63+currency, so we say "double-selling."
64+
65+"Double-selling" means signing over an Urbit address (*ship*) to
66+two different parties at the same time, presumably with the
67+intent to sell it to both and skip town with the loot.
68+
69+Again, the logical problem is the same. Bitcoin attacks it as a
70+broad theoretical breakthrough: better-is-better. Urbit attacks
71+it as a narrow special case: worse-is-better.
72+
73+Intuitively, Urbit's answer is that a real-estate transaction is
74+substantively a very different thing from a credit-card payment.
75+No real-estate agent ever has ever leaned over the desk, quickly
76+grabbed *both* the check and the keys, and run laughing out
77+the door.
78+
79+To understand more rigorously how Urbit is a special case, let's
80+take a quick look at the Urbit address space and PKI.
81+
82+## The Urbit address space and PKI
83+
84+(NB: this description differs in minor ways from the running
85+code. Where they differ, the description is right. Urbit is not
86+at present secure: use for entertainment purpose only.)
87+
88+An Urbit ship is an unsigned integer that serves four purposes: a
89+cryptographic identity, a packet routing address, a stateful
90+virtual computer, and a unique, human-memorable name.
91+
92+Each ship is owned by a private key. To transfer a ship to a new
93+keypair, sign the new public key with the old private key. This
94+revokes the old key and activates the new key. Propagating this
95+update globally makes the revocation global. This is essentially
96+the digital title system a smart 12-year-old would design.
97+
98+### Ships and wills
99+
100+Ships are 128-bit numbers classified into tiers by bitwidth. An
101+8-bit ship (a number less than 256) is a *galaxy*. A 16-bit ship
102+is a *star*. A 32-bit ship is a *planet*. A 64-bit ship is a
103+*moon*. A 128-bit ship is a *comet*.
104+
105+The chain of signatures for each ship is called a *will*. Unlike
106+most certificate systems, there is no metadata in an Urbit will.
107+Nothing is signed but the public-key fingerprint. The length of
108+the will is the keypair version that owns the ship. This version
109+is called its *life*. Each new signed fingerprint in the will is
110+a *deed*.
111+
112+Who gets to create a ship? Comets are fingerprints of a random
113+keypair. Moons, planets and stars are created by their *prefix*,
114+the ship whose bits are the bottom half of their formal width.
115+For instance, the star `0xbeef` is the prefix of the planet
116+`0xdeadbeef`. Galaxies have their fingerprints hardcoded in the
117+official kernel.
118+
119+Ship numbers are not generally written in hex. They're written
120+in Hoon's `@p` syntax, a phonemic base-256. `@p` also permutes
121+within tiers, to superficially conceal prefix structure; for
122+instance, `0xdead.beef` is `~firbyr-napbes`, while `0xcafe.beef`
123+is `~nissep-sopnub`.
124+
125+### Parent hierarchy
126+
127+The prefix of a moon, planet, or star also becomes its *parent*.
128+The parent of a ship is its natural locus of federated services.
129+The most basic such service is to mediate P2P routing, as a
130+so-called "supernode." But the set of potential federated
131+services is unbounded.
132+
133+For moons, each new key update is signed by the parent. Planets
134+and stars sign their own updates, making them independent. To
135+prevent denial-of-service attacks by parents against children,
136+stars and planets can *exit*, choosing a new parent. An exit
137+deed is a key update also signed by the new parent.
138+
139+The full public cryptographic record of a ship is its *dossier*.
140+The dossier is the set of ship-will pairs, including all parents
141+and prefixes recursively, needed to validate the ship's current
142+key. A ship can process a dossier from an untrusted source.
143+
144+The full set of ship-will pairs a ship knows is its *cabinet*.
145+So the crucial question for PKI propagation is whether a
146+submitted dossier is accepted into a ship's cabinet.
147+
148+### Propagation race
149+
150+The rule is simple: any deeds in the dossier which are new (not
151+already in the cabinet) must not use keys superseded in the
152+cabinet. Otherwise revocation doesn't work. But revocation
153+affects only dossiers received in the future; nothing already in
154+the cabinet can be revoked.
155+
156+Intuitively, this algorithm creates a propagation race. Life `2`
157+of `~firbyr-napbes` revokes life `1` of `~firbyr-napbes`, so
158+propagating life `2` to the whole network ensures that anyone who
159+steals the private key for life `1` can't use it in any way --
160+whether to launch children, or to sign a new, bogus life `2`.
161+
162+## Solving the special case
163+
164+So how can we solve the double-sell problem, for the special case
165+of the system designed above?
166+
167+### Double-spend relaying in Bitcoin
168+
169+The first thing we need to notice about our special case is that
170+*even Bitcoin*, not considered as a general consensus problem but
171+just as a decentralized payment rail, isn't that far from *not*
172+needing a blockchain.
173+
174+As the Karame ["Two Bitcoins at the Price of One"](http://eprint.iacr.org/2012/248.pdf)
175+paper points out:
176+
177+> An efficient countermeasure to combat double-spending on fast
178+Bitcoin payments would be for Bitcoin peers to propagate alerts
179+whenever they receive two or more transactions that share common
180+inputs and different outputs.
181+
182+"Fast Bitcoin payments" means zero-confirmation, ie, Bitcoin
183+without the blockchain. These double-spend alerts have never
184+been integrated into Bitcoin Core, despite frequent proposals.
185+Given that Bitcoin *does* have a blockchain, it seems quite
186+sensible for purists to shun hacks that make zero-conf
187+transactions somewhat more secure. Also, double-spend alerts in
188+Bitcoin are a DoS vulnerability; Bitcoin has fees to control the
189+cost of propagating transactions, but who pays for an alert?
190+
191+In principle, however, we could imagine a version of Bitcoin
192+without a blockchain, in which the mempool was the ledger, and
193+there was an alerting and/or conflict model to detect and resolve
194+double spends. An accepted transaction is simply one that
195+propagates to one or more randomly chosen observer nodes.
196+
197+This design, for the problem that Bitcoin is solving, intuitively
198+feels weak and a little hacky. The Karame paper is not actually
199+advocating for removing the blockchain from Bitcoin. At the same
200+time, low-value zero-confirmation payments are commonly used.
201+
202+One way to think about security is to flip the tables and think
203+of the defender as the attacker, and vice versa. Just imagine
204+you have some moral right to double spend. The rest of the world
205+is unfairly trying to stop you. Can you defend yourself?
206+
207+In the context of a gossip network that shares all the things, a
208+procedure that involves telling one person one thing, and another
209+person another conflicting thing, and not getting "hacked" by the
210+unfair world which believes there should be only one truth, is
211+inherently quite fragile. And where the attack is fragile, the
212+defense is robust.
213+
214+Most of the security in any digital title system comes from the
215+PKI itself. The double-spend problem, the general issue of
216+Byzantine failures in information propagation, is already a
217+corner case. So perhaps we can solve it as a corner case.
218+
219+Again: *even Bitcoin* isn't that far from being able to build
220+a practical decentralized ledger without a blockchain. The same
221+gossip model that broadcasts nonconflicting transactions is
222+not at all incapable of reacting to conflicting transactions.
223+We're not really recommending that Bitcoin ditch the blockchain,
224+but this gives us hope with our fundamentally easier problem.
225+
226+## Various specific specialnesses
227+
228+What are the differences between the Urbit double-sell problem
229+and the Bitcoin double-spend problem? Is our special case
230+really, truly, special?
231+
232+In worse-is-better mode, we want every special-case advantage we
233+can get. Every difference we find is one we can use as a
234+leverage point.
235+
236+### Minimal transaction interdependency
237+
238+The first major technical difference between the Urbit title
239+history and the Bitcoin blockchain is that Bitcoin and its
240+relatives are designed to secure a complete, interdependent,
241+collective transaction history. Except for the parent-child
242+relationship, Urbit wills are independent of each other.
243+
244+Bitcoin's use of UTXOs rather than balances reduces
245+interdependency somewhat. But the validity of each transaction
246+is still dependent on all the transactions that created its
247+inputs. (Ethereum drops the UTXO model, whose value in reducing
248+dependencies may not match its cost in complexity.)
249+
250+Urbit is not a payment system or a shared ledger. Bitcoins are
251+divisible, but Urbit ships aren't. An Urbit will containing
252+its parent chain is self-validating and has no dependencies.
253+There is no collective history. There is no Merkle tree. There
254+is no chain, with or without blocks.
255+
256+This simplification doesn't eliminate the double-sell problem in
257+theory, but makes it much more tractable in practice. The set of
258+past transactions (key transfers) that a new transaction could
259+conflict with is much smaller.
260+
261+### Trust hierarchy and governance structure
262+
263+The second major technical difference is that Bitcoin is a flat,
264+trustless, fully decentralized system. Urbit has a hierarchical,
265+polycentric governance system which is a trust hierarchy.
266+
267+Now and for quite some time, urbits default to automatically
268+accepting OS updates from their parent, making this trust
269+effectively absolute. This is not a novelty -- all "evergreen"
270+browsers now work this way. You trust your browser quite a bit,
271+which means you trust your browser vendor.
272+
273+And since few Bitcoin node operators code their own daemons, the
274+same dependency on a source distribution exists in Bitcoin. It's
275+informal and off-chain, though, which is not really a good thing.
276+(Urbit is like if Bitcoin Core hotpatched itself automatically by
277+publishing source-code diffs on the blockchain.)
278+
279+To be on the safe side, we should think of Urbit as a centralized
280+system until the OS update that turns off automatic updates is
281+shipped. This of course requires a certain measure of
282+perfection, which cannot in any way be predicted.
283+
284+After Arvo proper becomes "deciduous" (apps will of course always
285+be remotely hotpatched), child-to-parent trust should diminish to
286+the power of the parent to deny service to the child. This will
287+always be present without some major redesign.
288+
289+But it's opposed by the equal and opposite power of the child to
290+switch to a different parent, creating a competitive market in
291+governance. This should deliver service at the highest quality
292+and lowest price, fortifying trust with incentives. The power
293+of a restaurant owner to deny you service does not give the food
294+industry the power to starve you to death.
295+
296+And if no parent wants your business, with 2^16 stars who could
297+accept it, you probably shouldn't be on the network at all.
298+Urbit, like a real society, is designed to be a network of
299+incentives that holds everyone in productive tension. We want
300+everyone to be free, and no one to be unaccountable.
301+
302+For example: a parent which extends its will also needs to send
303+updated signatures to its children and former children. Either
304+will otherwise be unable to propagate their dossiers to ships
305+whose cabinets have the updated parent will, *even if some other
306+parent adopts them*.
307+
308+But Urbit is a collectively governed and updatable system. A
309+star or galaxy which denies service to its own children will (a)
310+get a bad reputation, (b) drive its children to other parents,
311+and (c) force the network to hardcode a ship-specific exception
312+into the propagation algorithm, overriding stale-key revocation.
313+
314+In theory, a parent can extort its children by denying service.
315+In practice, since Urbit is self-governing, not ungoverned, and
316+its government can right this wrong with a source-code patch,
317+the extortion won't work. So it won't happen. So the act of
318+government won't be necessary, and won't happen either.
319+
320+Of course a perfectly decentralized system would be freer. But
321+worse, as usual, is better. And since we are paying for this
322+worseness, imperfect decentralization, we'd like to get all the
323+mileage out of it that we can.
324+
325+Again, Urbit, not being a trustless system, does not have to
326+solve the trustless consensus problem -- although we still very
327+much want to minimize and bound the delegation of trust. This is
328+an obvious relaxation of Bitcoin's hard theoretical problem.
329+
330+### Routing: security through topology
331+
332+Please don't laugh. Security through routing topology isn't a
333+joke -- unless firewalls are a joke. Cisco sells too many
334+firewalls for them to be a joke.
335+
336+In Bitcoin we may have all sorts of reasons to ask what ship
337+holds what coins. In Urbit, when we ask what key controls an
338+ship, we are either sending or receiving a message. Is our
339+outgoing message safely sealed to the current owner? Is our
340+incoming message properly signed by the current owner?
341+
342+When the parent also forwards all packets until a P2P link is
343+established (Urbit forwarding resolves to STUN if either party
344+has full cone NAT, TURN if not), we can design in all kinds of
345+interactions between routing and certificate management.
346+
347+In particular, a parent which is a single source of truth in some
348+sense is a very attractive design element. Again, we're paying
349+for our imperfect decentralization. We might as well use it.
350+
351+For example: I want to send a packet to ship X. Two parties,
352+A and B, are competing for ownership of X. A is the rightful
353+owner. I am secure against B if either of these conditions are
354+true: my packet can only be decoded by A, or my packet is not
355+sent to B. Both would be ideal, of course. Either will do.
356+
357+### Latency and friction tolerance
358+
359+Confirmed Bitcoin payments aren't especially low-latency by
360+payment standards. They're reasonably low-friction by payment
361+standards. But fees are certainly a thing.
362+
363+Urbit is digital real estate, not digital money. When we look at
364+the standards of latency and friction in actual real estate --
365+where a transaction that takes a week is super-fast, a 5% payment
366+overhead is normal, and a third-party intermediary is standard --
367+we see a much, much weaker set of requirements.
368+
369+Obviously a week and a 5% overhead are incredibly sucky levels of
370+latency and friction. The point is that even if this is the only
371+level of performance Urbit can achieve, that's not necessarily
372+the end of the world for Urbit.
373+
374+For instance: if we wait a week for a double-sell alert, and we
375+don't see one, either the gossip network is totally hosed (to a
376+point we'd surely find out about), or there wasn't a double-sell.
377+This stuff is just not that hard.
378+
379+(One little-made point about Bitcoin, by the way, is that Bitcoin
380+enables trust-free *payments*, but not trust-free *exchanges*.
381+The seller still needs to deliver the goods either before or
382+after the payment; in either case, there is trust. Only Ethereum
383+can solve this problem -- and only for digital exchanges.)
384+
385+## We can afford to be patient
386+
387+With a better-is-better, general-purpose solution, one size fits
388+all until the end of time. With a worse-is-better, special-case
389+solution, the details of that case are critical.
390+
391+But we actually don't know these details. One rule of the
392+worse-is-better school: never solve a problem before you have to.
393+You don't know all the details, and you need those details. (Of
394+course, with the better-is-better approach, the details don't
395+matter at all.)
396+
397+Is there such a thing as a transitive adjective? Grammarians are
398+unclear. But technically, the adjective "secure" is always an
399+error. Nothing is "secure." "Secure against X" is the correct
400+usage, where X is some threat model.
401+
402+Urbit is an insecure test network with no sensitive data on it.
403+This will remain true for a while. No Urbit ship has ever
404+been transferred cryptographically. This will also remain the
405+status quo for quite a while.
406+
407+We *don't know the threat model*. We *can't solve the problem*.
408+It's as simple as that. The best Urbit can do, at the present
409+time, is map out the ballpark of solutions.
410+
411+And we have time to proceed patiently. One way to think about
412+the timing of cryptographic decentralization is to look at the
413+DNS. It's clear that now, a decentralized DNS with cryptographic
414+domain-name titles would be extremely desirable. It was never
415+designed to happen. It will probably never happen.
416+
417+But if we look at when cryptographic DNS with allodial title
418+*should* have happened, it's probably sometime in the mid-90s.
419+Ie, after over a decade of production use of the Domain Name
420+System. In short: there's no need to hurry here.
421+
422+## A map of the solution space
423+
424+But we still need a map of the worse-is-better ballpark. Let's
425+look at some options, from least secure to most.
426+
427+The essential question we can't answer, without a better sense of
428+the threat model, is how aggressively we need to propagate wills.
429+We especially are not sure about the threat model of malicious
430+stars (16-bit ships) and/or galaxies (8-bit ships).
431+
432+We need to know what kinds of double-sell attacks are plausible.
433+Since we don't know this yet, we need not one propagation design,
434+but a range of designs -- stretching all the way up to a full
435+blockchain.
436+
437+### Acceptance model
438+
439+What doesn't change is the will *acceptance* model. A ship will
440+always accept a new will if (a) the new will extends the old
441+will, (b) the parent signature on the initial public key is at
442+least as recent as the ship's current version of that parent.
443+
444+The parent clause implies that when a parent extends its will, it
445+needs to send updated initial signatures to all the children it's
446+launched. Otherwise, their own future updates will not be
447+accepted by ships that have seen the parent's update.
448+
449+This is not a big deal in the world of parent responsibilities.
450+But like many such responsibilities, it lets a badly operated
451+parent deny service to its children.
452+
453+A ship accepts all wills, new or updated, and stores them
454+permanently. If this is abused trivially as a DoS vector, Urbit
455+will mitigate it by the usual technique of degrading service to
456+non-scarce ship tiers. Scarcity rules.
457+
458+### Level 0: passive propagation only
459+
460+The Ames protocol can piggyback a will on any packet. Each ship
461+also permanently records which other ships hold which version of
462+its own will. An update is automatically sent when needed.
463+
464+With level 0 propagation, all key exchanges in Ames are passive
465+and invisible to the programmer. When the sender has no will or
466+symmetric key for the receiver, the packet is signed, bundled
467+with the will, and sent as cleartext.
468+
469+Signed cleartext? If you have no will for a peer, you've never
470+talked to this ship before. Your first words to a stranger
471+probably won't be super secret. If this isn't true, the
472+programmer has to send an innocuous message first and wait for it
473+to be acknowledged. Kicking a rare corner case up to the
474+programmer is a classic worse-is-better move.
475+
476+With level 0 propagation only (the current implementation,
477+because we don't even care about key updates yet), an updated
478+does not effectively revoke the old key. If Alice updates to a
479+new key, Mallory steals the old key, then Mallory talks to Bob
480+before Alice does, Mallory can impersonate Alice.
481+
482+So not only does level 0 not solve double-sell, it's also quite
483+insecure. But it's still fine for now (because Urbit isn't
484+really decentralized yet).
485+
486+### Level 1: neighbor-only propagation with parent filtering
487+
488+The list of other ships which hold your will, essentially the
489+list of all ships you've talked to, is your list of *neighbors*.
490+You essentially have permanent, if perhaps very sporadic,
491+conversations with all your cryptographic neighbors. Symmetric
492+keys time out fast, but wills are held forever.
493+
494+By proactively pinging all your neighbors when you extend your
495+will, you prevent anyone who finds the old private key (ideally
496+you update when your old key is stale, not when it's actually
497+been *stolen*) from jumping into these conversations.
498+
499+What about non-neighbors? We add a second feature: a parent will
500+only forward a packet if its sender matches the current will. So
501+updating your parent (who is always your neighbor) transforms it
502+
503+There are two cases: where Alice's parent Trent is also Bob's
504+parent, and where Bob has a separate parent Walter. In the first
505+case, the expected packet flow is that Alice will send the packet
506+to Trent, who sends it to Bob. In the second case, Trent sends
507+it to Walter, who sends it to Bob.
508+
509+In either flow, Bob or Walter is sure that the packet came
510+through Trent. Since Trent is Alice's parent and hence Alice's
511+neighbor, and hence is updated, Trent will forward no packets
512+from Mallory. So Mallory has no way to reach Bob.
513+
514+Non-neighborly packets not matching this flow can't be accepted.
515+Suppose Mallory gets Bob's public IP address directly? If Bob
516+gets a non-forwarded packet marked "Alice," but Alice is not his
517+neighbor, that's super weird. He must reject the packet.
518+
519+Level 1 is nice because it provides a reasonable level of
520+security, but it's still extremely simple. Not only does it not
521+use a blockchain, it doesn't even need a gossip flood!
522+
523+And level 1 propagation also does a competent worse-is-better job
524+on the double-sell problem. Trent simply becomes the single
525+source of truth. The buyer of Alice needs to check that Trent,
526+her parent, has Alice's latest will from the buyer's perspective.
527+If Alice's seller propagates a different fork to other ships,
528+this damages Alice's ability to communicate, but produces no
529+obvious gain for the seller.
530+
531+Level 1 puts a lot of security weight on the parent. If Trent
532+misbehaves, for example by not filtering Mallory's packets, Alice
533+can be impersonated. And its solution to the double-sell problem
534+is adequate, but not ideal. Level 1 is the worse-is-better sweet
535+spot for a young Urbit, but will probably give way to level 2.
536+
537+### Level 2: broadcast all wills
538+
539+We can't say definitively that level 1 will be inadequate. But
540+intuitively, it feels slightly weak. We will probably need a
541+gossip protocol.
542+
543+The only ships that can be transferred are galaxies, stars and
544+planets; if all updates of all wills of all these major ships are
545+broadcast, we hardly have a scaling problem like Bitcoin's. Even
546+if changing the lock on your front door counted as a real-estate
547+transaction, there are a lot fewer real-estate transactions than
548+credit-card payments.
549+
550+Moreover, conflict detection (double-sell alerts) is a natural
551+part of any gossip protocol. Gossip should be between stars, not
552+directly between planets; horizontal gossip, vertical
553+distribution. Star-to-planet distribution may still be limited.
554+
555+Especially since stars are somewhat trusted, ganking a gossip
556+protocol with random update edges is incredibly hard and requires
557+a pretty preposterous conspiracy. And flooding all planet wills
558+does a lot to keep stars honest. It would be incredibly stupid
559+for a star to try a double-sell attack against its own planets,
560+but code is always more powerful than mere incentives.
561+
562+Finally, the exit model isn't fully baked yet, but it will
563+probably have more aggressive propagation needs. What's clear is
564+that an exit statement is another type of will entry, involving a
565+signature both from the previous key and the new parent. You can
566+only escape to another ship of your parent's class that agrees to
567+adopt you.
568+
569+In general, because escaping need to work for routing (without
570+the cooperation of the former parent), wills containing an exit
571+statement (a planet moving to a new star, or a star moving to a
572+new galaxy) exit) probably need to be flooded globally among all
573+8-bit galaxies and 16-bit stars. Exits should be rare, so they
574+shouldn't present much of a scaling problem.
575+
576+### Level 3: full blockchain
577+
578+Obviously a blockchain can solve Urbit's problem. Probably the
579+right thing to do would be to use the Bitcoin blockchain, and
580+confirm the current Urbit key version with a "colored coin" of
581+some kind. (This should not be confused with the orthogonal
582+idea of representing Urbit *invitations* as colored coins.)
583+
584+This involves making all Urbit ships into Bitcoin SPV nodes at
585+the least. Not the worst fate in the world, but also a cost.
586+Alternatively, Urbit could have its own "altcoin" blockchain,
587+with actual Urbit-specific mining.
588+
589+Again, given that level 2 seems like it should work quite well,
590+it's hard to see why level 3 propagation would be needed. But we
591+can't really know at this point. There's a reason Urbit can
592+update itself over the air. That's why we say: Urbit (probably)
593+doesn't need a blockchain. If it does, we can retrofit it.
594+
595+## Retrospective note
596+
597+This essay has presented the Urbit cryptographic title as a
598+low-power, stripped-down, worse-is-better variation of Nakamoto
599+consensus. That's clearly the right way to introduce it in 2016.
600+
601+Actually, though, this part of Urbit was designed before the
602+Bitcoin paper was published. There were other academic
603+approaches to Byzantine consensus, but certainly none validated
604+on a large scale as Nakamoto consensus has been. It never seemed
605+appropriate to treat cryptographic titles to network addresses as
606+a case of generalized Byzantine consensus, and without the
607+existence of Bitcoin there was no temptation in this direction.
608+
609+In a sense the secret of PKIs is that PKIs are easy. What's hard
610+is adapting an existing identity and authorization structure into
611+a PKI. If you build the simplest possible PKI, then force the
612+human processes to fit themselves to the protocol instead of vice
613+versa, you have a much easier job. Similarly, if you are not
614+adding PKI to an existing network, but building the network and
615+the PKI as one system, many simplifications are possible. This
616+is once again the classic worse-is-better method.
main/blog/bytes/0.mdnew file+253
@@ -0,0 +1,253 @@
1+# Urbyte 0: setup, nouns
2+
3+This is the first post in our new Urbyte series.
4+
5+The Urbyte concept is that you, an ordinary programmer, not a
6+kernel hacker or a Haskell magician (not that there's anything
7+wrong with kernel hackers or Haskell magicians) can learn Hoon,
8+Urbit's programming language.
9+
10+We're going to test this daring idea by teaching you Hoon, in
11+easily digestible, byte-size chunks. Each post in the "Urbytes"
12+series is designed to take about 15 minutes to consume. (Maybe
13+10, if you're a kernel hacker or a Haskell magician.)
14+
15+We'll assume no knowledge of Urbit except for previous lessons.
16+Ideally you have some basic familiarity with programming and can
17+operate the Unix command line.
18+
19+We strongly encourage you to type in all the examples in each
20+Urbyte. Urbytes are short, but not meant to be skimmed.
21+
22+## Install
23+
24+First, [install Urbit](https://urbit.org/posts/install) on any
25+Mac or Unix machine. On Windows, make a virtual Linux machine
26+using VirtualBox or a similar tool.
27+
28+While you're downloading, read about Urbit ships:
29+
30+## Ships
31+
32+In the Urbyte exercises you'll be running what we call a "fake
33+`~zod`." A string starting with `~`, like `~zod`, is what Urbit
34+calls a *ship*.
35+
36+A ship is one number which means four things: a network address,
37+a cryptographic identity, a memorable name, and a single virtual
38+computer. We write this number in a base-256 format, where each
39+digit is a syllable.
40+
41+`~zod` is ship zero, making it a very important ship indeed.
42+Your "fakezod" thinks it's `~zod`, but no one can talk to it.
43+
44+This is not the true Urbit experience, of course. But it's
45+ideal if all you want to do is play around with Hoon. If you
46+have a real urbit ship, of course, you can use that instead.
47+
48+## Create and restart
49+
50+To create a fakezod, run:
51+
52+```
53+$ urbit -FI ~zod -c myzod
54+```
55+
56+This will take about 30 seconds and spin out a bunch of boot
57+messages. At the end, you'll see:
58+
59+```
60+ames: on localhost, UDP 31337.
61+http: live (insecure, public) on 8080
62+http: live ("secure", public) on 8443
63+http: live (insecure, loopback) on 12321
64+~zod:dojo>
65+```
66+
67+To make sure everything is working, type `(add 2 2)` at the
68+prompt, then hit return. Your screen now shows:
69+
70+```
71+ames: on localhost, UDP 31337.
72+http: live (insecure, public) on 8080
73+http: live ("secure", public) on 8443
74+http: live (insecure, loopback) on 12321
75+> (add 2 2)
76+4
77+~zod:dojo>
78+```
79+
80+That's super awesome! You made your first noun! We'll make a
81+few more in a second. First, quit Urbit with `^D`:
82+
83+```
84+> (add 2 2)
85+4
86+~zod:dojo>
87+$
88+```
89+
90+You now have an Urbit image, or *pier*, in the `myzod` directory.
91+Right now the only thing in your pier is Urbit's system files:
92+
93+```
94+$ ls -a myzod
95+./ ../ .urb/
96+```
97+
98+Restarting uses almost the same command, but without `-c`. There
99+are also fewer startup messages.
100+
101+```
102+$ urbit -FI ~zod myzod
103+[...]
104+ames: on localhost, UDP 31337.
105+http: live (insecure, public) on 8080
106+http: live ("secure", public) on 8443
107+http: live (insecure, loopback) on 12321
108+~zod:dojo>
109+```
110+
111+Note also that Urbit is an ACID database. You can stop it
112+politely with `^D`, abruptly with `^Z`, even violently with `kill
113+-9`. None of these should cause Urbit to lose data. If you do
114+manage to corrupt a pier, it's a bug; please let us know.
115+
116+## Let's make a noun
117+
118+We already made a noun. Let's make another. (We won't show the
119+`~zod:dojo> ` prompt in future, just the echoed command.)
120+
121+```
122+~zod:dojo> 42
123+```
124+
125+You'll see:
126+
127+```
128+> 42
129+42
130+~zod:dojo>
131+```
132+
133+## Understanding nouns
134+
135+A value in Urbit is called a *noun*. A noun is either an *atom*
136+(an unsigned integer of any size) or a *cell* (an ordered pair of
137+any two nouns).
138+
139+In a sentence, a noun is a binary tree whose leaves are numbers.
140+If you know Lisp, a noun is a minimalist S-expression. A noun
141+can't have cycles (pointer loops). An implementation can use
142+dags (shared subtrees), but the programmer can't detect them
143+(there's no pointer equality test).
144+
145+The absence of graph cycles makes nouns much easier to save in
146+persistent storage, send over the network, manage memory for
147+(Urbit has no tracing garbage collector), etc.
148+
149+## Let's make some atoms
150+
151+So one kind of atom is a number. And the syntax (or at least one
152+syntax) for a number is a decimal, like `42`. Seems legit.
153+
154+But just to be sure we understand this decimal thing, let's make
155+some atoms:
156+
157+```
158+> 4
159+4
160+```
161+
162+Seems pretty straightforward.
163+
164+```
165+> 42
166+42
167+```
168+
169+Why not try it again to make sure?
170+
171+```
172+> 420
173+420
174+```
175+
176+Can we add another zero?
177+
178+```
179+~zod:dojo> 420
180+```
181+
182+Wat? When you tried to type that next zero and turn `420` into
183+`4200`, Urbit actually *deleted the `0` and beeped at you.*
184+
185+But isn't an atom an unsigned integer of any size? It is.
186+But the syntax is actually `4.200`:
187+
188+```
189+> 4.200
190+4.200
191+> (mul 42 100)
192+4.200
193+```
194+
195+Just think of it as "4,200", written the German way with a dot.
196+
197+Why? Long decimals are unreadable. Human beings know this, so
198+we group them in threes. For some historical reason, programming
199+languages never copied this marvelous usability innovation.
200+
201+English notation for decimals is more common than German
202+notation. Unfortunately, dot is URL-safe and comma isn't, and
203+it's nice to have a regular syntax for atoms that's URL-safe.
204+
205+As for why Urbit deleted your zero: it parses your command line
206+as you type, and rejects any characters after the parser stops.
207+
208+## Let's make some cells
209+
210+There's not much mystery about cells. A cell has a head and a
211+tail, and puts brackets around them:
212+
213+```
214+> [42 420]
215+[42 420]
216+```
217+
218+`42` is the *head* and `420` is the *tail* of the cell. Cell
219+syntax nests to the right:
220+
221+```
222+> [4 [42 420]]
223+[4 42 420]
224+
225+> [4 42 420]
226+[4 42 420]
227+
228+> [[4 42] 420]
229+[[4 42] 420]
230+```
231+
232+If you look at these nouns as binary trees, you see
233+
234+```
235+ [4 [42 420]] [[4 42] 420]
236+ . .
237+ / \ / \
238+ 4 . . 420
239+ / \ / \
240+ 42 420 4 42
241+```
242+
243+Each number is an atom; each dot is a cell.
244+
245+## Questions and/or exercises
246+
247+And we're done with our very first Urbyte! Here's a question to
248+think about. These exercises are never required, but they may
249+be fun and/or interesting.
250+
251+The question today is: what is the right way to represent common
252+data structures as nouns? A linked list? A string? A *signed*
253+integer? A table or associative array?
main/blog/bytes/1.mdnew file+288
@@ -0,0 +1,288 @@
1+# Urbyte 1: atoms, auras, types
2+
3+Thanks for reading Urbyte 0! In Urbyte 1, we'll look a little
4+more deeply at atoms and other simple noun types.
5+
6+## The atom question
7+
8+It seems limiting to have only one kind of atom, the unsigned
9+decimal. How are signed integers represented? Symbols?
10+Floating-point values? Dates? IP addresses? Even just hex?
11+
12+The Lisp S-expression, with cons cells and atoms, is very similar
13+to Hoon's nouns. But languages in the Lisp family mark each atom
14+with a dynamic tag that defines the atom's meaning. These tags
15+are effectively dynamic types.
16+
17+Hoon also tracks the meaning of atoms. But Hoon is statically
18+type, so an atom description is obviously a type. An atomic type
19+in Hoon is called an *aura*. The aura syntax is just an ASCII
20+string starting with `@`, like `@ud` for unsigned decimal.
21+
22+## Examples
23+
24+With the dojo command `? expression`, you tell the dojo to print
25+both the product of the expression, and the product's type.
26+
27+Let's type in some aura examples:
28+
29+### Unsigned integers
30+
31+```
32+> 42
33+42
34+
35+> ? 42
36+ @ud
37+42
38+
39+> ? 0x42
40+ @ux
41+0x42
42+
43+> ? `@ud`0x42
44+ @ud
45+66
46+
47+> ? (add 0x42 42)
48+ @
49+108
50+```
51+
52+So decimal is aura `@ud`, hexadecimal is `@ux`. Note the `@ud`
53+cast syntax, which lets us change the type of an atom, and
54+converting hex `0x42` to a decimal.
55+
56+An aura is soft type. The programmer can always insist on how
57+to interpret an atom. Hoon doesn't have "dependent types"; and
58+can't enforce any constraints the aura puts on the atom's value.
59+
60+### Signed integers
61+
62+```
63+> -7
64+-7
65+
66+> ? -7
67+ @sd
68+
69+> --7
70+--7
71+
72+> ? --7
73+ @sd
74+--7
75+
76+> ? (sum:si -7 --7)
77+ @s
78+--0
79+
80+> `@ud`-7
81+13
82+
83+> `@ud`--7
84+14
85+```
86+
87+`--7` means "positive 7." `+7` might have been better, but `+`
88+is not URL-safe.
89+
90+With unlimited atom width, traditional sign extension makes no
91+sense. The sign bit is the low bit. Even atoms are positive,
92+odd atoms are negative.
93+
94+Hoon needs `--` to distinguish positive signed numbers because
95+Hoon's type system doesn't feed back into its parser. The
96+expression itself needs to control whether it's `@sd` or `@ud`.
97+
98+And Hoon has no overloading or type detection; we need to use a
99+different function to add signed integers. We'll always choose
100+slightly clunkier syntax in exchange for much simpler semantics.
101+
102+### Symbols
103+
104+```
105+> ?
106+ @t
107+'Ürbit'
108+
109+> ? `@ux`'Ürbit'
110+ @ux
111+0x7469.6272.9cc3
112+
113+> ? `@ux`'foo'
114+ @ux
115+0x6f.6f66
116+```
117+
118+The `@t` aura is UTF-8 text, with the first byte low. Again,
119+Hoon doesn't actually have the power to check statically that
120+your UTF-8 is valid. Auras don't tell you what an atom is, just
121+what you want it to be.
122+
123+### Dates, floats, IP addresses...
124+
125+```
126+> ? now
127+ @da
128+~2016.7.11..01.32.04..2c0d
129+
130+> ? @~h6
131+ @dr
132+~h6
133+
134+> `@ux`~h6
135+ @ux
136+0x5460.0000.0000.0000.0000
137+
138+> ? (add ~h6 now)
139+ @
140+170.141.184.502.236.089.414.812.327.632.537.911.296
141+
142+> `@da`(add ~h6 now)
143+~2016.7.11..07.32.28..842f
144+
145+> ? .3.14
146+ @rs
147+.3.14
148+
149+> ? .127.0.0.1
150+ @if
151+.127.0.0.1
152+```
153+
154+`@da` is a 128-bit absolute date (64 bits for fractions of a
155+second, 64 bits for seconds); `@dr` is a relative date; `@rs` is
156+a single-precision IEEE real; `@if` is an IPv4 address. This is
157+not all Hoon's aura syntax, but it's a good range of examples.
158+
159+`\`@da\~(add ~h6 now)` is a good example of the use and limits of
160+the aura system. Unlike the signed-integer sum, offsetting a
161+date can use plain atomic addition. But the product of `add` is
162+a flavorless atom with no aura, so we have to cast it back to a
163+date aura.
164+
165+## Auras explained
166+
167+Does an aura represent the units of the atom, its syntax, its
168+semantics, its constraints?
169+
170+It can mean any or all of these. Programmers can even use their
171+own aura strings which nothing in Hoon recognizes. The set of
172+syntaxes that Hoon knows how to parse and print is fixed, but
173+there are plenty of reasons to introduce user-defined auras.
174+
175+If your program uses both fortnights and furlongs, the aura
176+system will not parse a fortnight syntax or print furlongs in the
177+dojo. But it will keep you from accidentally confusing the two,
178+which is probably most of what you want.
179+
180+Hoon does interpret the string in one way: auras specialize to
181+the right. For example, `@u` is any unsigned integer; `@ux` is
182+an unsigned integer that likes to be printed as hex.
183+
184+Hoon's typechecker lets you go up or down the specialization
185+ladder, but complains if you try to go across. For example,
186+`@u` can silently convert into `@ux` or `@ux` into `@u`, but
187+turning `@ud` into `@ux`, `@ux` into `@da`, etc, requires an
188+explicit cast.
189+
190+## Constants, symbols, loobeans and null
191+
192+An atom type isn't just an aura. There's another choice: the
193+type can be *general* and contain all atoms (like the types we've
194+seen above), or *constant* and contain just one atom.
195+
196+A type describes a set of values. While auras may informally
197+describe a format for which not all atoms are valid, any atom can
198+be cast into the aura whether valid or invalid.
199+
200+In a constant, the type contains just one atom. Hoon actually
201+enforces this restriction.
202+
203+The regular syntax for an atomic constant is just to put `%` on
204+front of the constant. Like:
205+
206+```
207+> ? 42
208+ @ud
209+42
210+
211+> ? %42
212+ $42
213+42
214+
215+> ? %~h6
216+ $~h6
217+~h6
218+```
219+
220+The `$` on the type indicates the constant. Note that constants
221+still have auras; `~h6` (6 hours) is still a `@dr`.
222+
223+## Common irregulars
224+
225+There are a few special forms worth learning early: symbols,
226+null, and loobeans:
227+
228+```
229+> ? %foo
230+ $foo
231+%foo
232+
233+> `@ux`%foo
234+0x6f.6f66
235+
236+> .n
237+.n
238+
239+> |
240+.n
241+
242+> .y
243+.y
244+
245+> &
246+.y
247+
248+> `@ud`&
249+0
250+
251+> ~
252+~
253+
254+> `@ud`~
255+0
256+```
257+
258+`%symbol` is a constant with aura `@tas` (text, ASCII, symbol).
259+The symbol rules, which are enforced by the Hoon parser but not
260+(as we've discussed) the type system: a symbol is lowercase with
261+infix hyphens ("kebab-case").
262+
263+Loobeans are not true and false, but yes (`.y` or `&`) and no
264+(`.n` or `|`). This is because *yes is zero and no is one*.
265+This makes sense at a certain mathematical level, though it
266+was probably a mistake.
267+
268+`~` is Hoon's nil: constant `0`, aura `@n`.
269+
270+## Cells and types
271+
272+Naturally, all these atoms fit neatly into cells, and retain
273+their types within the cells.
274+
275+```
276+> ? [~h6 [.3.14 %foo] 0xdead.beef]
277+ {@dr {@rs $foo} @ux}
278+[~h6 [.3.14 %foo] 0xdead.beef]
279+
280+> `*`[~h6 [.3.14 %foo] 0xdead.beef]
281+[398.449.671.992.126.314.905.600 [1.078.523.331 7.303.014] 3.735.928.559]
282+```
283+
284+Another fun trick is to cast any noun to the `*` type, ie,
285+generic noun. We then know nothing about the noun, so we print
286+all its atoms as decimals.
287+
288+That's all for today! Please wait for the next fun Urbyte.
main/blog/bytes/2.mdnew file+38
@@ -0,0 +1,38 @@
1+
2+## The dojo
3+
4+You're playing in the Urbit REPL/shell (`:dojo`). (If you
5+accidentally press `^X`, it will put you into the other default
6+app, `:talk`; press `^X` again to switch back.)
7+
8+The dojo actually has its own command language, which is a
9+superset of Hoon. Any Hoon expression is a dojo command that
10+prints the product of the expression. But there are other dojo
11+commands.
12+
13+### Setting variables
14+
15+Another command sets a dojo variable. Type
16+
17+```
18+> =a 42
19+```
20+
21+You've set the dojo variable `a` to the value `42`. The dojo
22+puts this binding in the Hoon subject for expressions to use.
23+(We'll see how this works in a later Urbyte.) So you can write:
24+
25+```
26+> a
27+42
28+
29+> [4 a]
30+[4 42]
31+```
32+
33+To unbind `a`, just omit the value:
34+
35+```
36+> =a
37+```
38+