@@ -1,2347 +0,0 @@
1−---
2−sort: 0
3−title: Urbit whitepaper
4−---
5−
6−Urbit: an operating function
7−============================
8−
9−<div class="warning">This is Urbit whitepaper DRAFT 40K. Some small details
10−remain at variance with the codebase.</div>
11−
12−Abstract
13−========
14−
15−Urbit is a clean-slate, full-stack redesign of system software.
16−In 25K lines of code, it's a packet protocol, a pure functional
17−language, a deterministic OS, an ACID database, a versioned
18−filesystem, a web server and a global PKI. Urbit runs on a frozen
19−combinator interpreter specified in 200 words; the rest of the
20−stack upgrades itself over its own network.
21−
22−Architecturally, Urbit is an opaque computing and communication
23−layer above Unix and the Internet. To the user, it's a new
24−decentralized network where you own and control your own
25−general-purpose personal server, or "planet." A planet is not a
26−new way to host your old apps; it's a different experience.
27−
28−Download
29−--------
30−[As `.md`](https://storage.googleapis.com/urbit-extra/preview-1/whitepaper.md)
31−[As `.pdf`](https://storage.googleapis.com/urbit-extra/preview-1/whitepaper.pdf)
32−
33−<toc></toc>
34−
35−<div id="toc">
36−
37−Objective
38−=========
39−
40−How can we put users back in control of their own computing?
41−
42−Most people still have a general-purpose home computer, but it's
43−atrophying into a client. Their critical data is all in the
44−cloud. Technically, of course, that's ideal. Data centers are
45−pretty good at being data centers.
46−
47−But in the cloud, all users have is a herd of special-purpose
48−appliances, not one of which is a general-purpose computer. Do
49−users want their own general-purpose personal cloud computer? If
50−so, why don't they have one now? How might we change this?
51−
52−Conventional cloud computing, the way the cloud works now, is
53−"1:n". One application, hosted on one logical server by its own
54−developer, serves "n" users. Each account on each application is
55−one "personal appliance" - a special-purpose computer, completely
56−under the developer's control.
57−
58−Personal cloud computing, the way we wish the cloud worked, is
59−"n:1": each user has one logical server, which runs "n"
60−independent applications. This general-purpose computer is a
61−"personal server," completely under the user's control.
62−
63−"Personal server" is a phrase only a marketing department could
64−love. We prefer to say: your *planet*. Your planet is your
65−digital identity, your network address, your filesystem and your
66−application server. Every byte on it is yours; every instruction
67−it runs is under your control.
68−
69−Most people should park their planets in the cloud, because the
70−cloud works better. But a planet is not a planet unless it's
71−independent. A host without contractual guarantees of absolute
72−privacy and unconditional migration is not a host, but a trap.
73−The paranoid and those with global adversaries should migrate to
74−their own closets while home computing remains legal.
75−
76−But wait: isn't "planet" just a fancy word for a self-hosted
77−server? Who wants to self-host? Why would anyone want to be their
78−own sysadmin?
79−
80−*Managing* your own computing is a cost, not a benefit. Your
81−planet should be as easy as possible to manage. (It certainly
82−should be easier than herding your "n" personal appliances.) But
83−the benefit of "n:1" is *controlling* your own computing.
84−
85−The "n:1" cloud is not a better way to implement your existing
86−user experience. It's a different relationship between human and
87−computer. An owner is not just another customer. A single-family
88−home is not just another motel room. Control matters. A lot.
89−
90−Perhaps this seems abstract. It's hard to imagine the "n:1" world
91−before it exists. Let's try a thought-experiment: adding
92−impossible levels of ownership to today's "1:n" ecosystem.
93−
94−Take the web apps you use today. Imagine you trust them
95−completely. Imagine any app can use any data from any other app,
96−just because both accounts are you. Imagine you can "sidegrade"
97−any app by moving its data safely to a compatible competitor.
98−Imagine all your data is organized into a personal namespace, and
99−you can compute your own functions on that namespace.
100−
101−In this world, no app developer has any way to hold your data
102−hostage. Forget how this works technically (it doesn't). How does
103−it change the apps?
104−
105−Actually, the rules of this thought-experiment world are so
106−different that *few of the same apps exist*. Other people's apps
107−are fundamentally different from your own apps. They're not
108−"yours" because you developed them -- they're your apps because
109−you can fire the developer without any pain point. You are not a
110−hostage, so the power dynamic changes. Which changes the app.
111−
112−For example: with other people's apps, when you want to shop on
113−the Internets, you point your browser at amazon.com or use the
114−Google bar as a full-text store. With your own apps, you're more
115−likely to point your browser at your own shopping assistant. This
116−program, which *works entirely for you and is not slipping anyone
117−else a cut*, uses APIs to sync inventory data and send purchase
118−orders.
119−
120−Could you write this web app today? Sure. It would be a store.
121−The difference between apps you control and apps you don't is the
122−difference between a shopping assistant and a store. It would be
123−absurd if a shopping assistant paid its developer a percentage of
124−all transactions. It would be absurd if a store didn't. The
125−general task is the same, but every detail is different.
126−
127−Ultimately, the planet is a different user experience because you
128−trust the computer more. A program running on someone else's
129−computer can promise it's working only for you. This promise is
130−generally false and you can't enforce it. When a program on your
131−computer makes the same promise, it's generally true and you can
132−enforce it. Control changes the solution because control produces
133−trust and trust changes the problem.
134−
135−Could we actually add this level of user sovereignty to the "1:n"
136−cloud? It'd take a lot of engineers, a lot of lawyers, and a
137−whole lot of standards conferences. Or we could just build
138−ourselves some planets.
139−
140−Obstacles
141−=========
142−
143−If this is such a great product, why can't you already buy it?
144−
145−In 2015, general-purpose cloud servers are easily available. But
146−they are industrial tools, not personal computers. Most users
147−(consumer and enterprise) use personal appliances. They choose
148−"1:n" over "n:1". Why does "1:n" beat "n:1" in the real world?
149−
150−Perhaps "1:n" is just better. Your herd of developer-hosted
151−appliances is just a better product than a self-hosted planet. Or
152−at least, this is what the market seems to be telling us.
153−
154−Actually, the market is telling us something more specific. It's
155−saying: the appliance herd is a better product than a self-hosted
156−*Unix server on the Internet*.
157−
158−The market hasn't invalidated the abstract idea of the planet.
159−It's invalidated the concrete product of the planet *we can
160−actually build on the system software we actually have*.
161−
162−In 1978, a computer was a VAX. A VAX cost \$50K and was the size
163−of a fridge. By 1988, it would cost \$5K and fit on your desk.
164−But if a computer is a VAX, however small or cheap, there is no
165−such thing as a PC. And if a planet is an AWS box, there is no
166−such thing as a planet.
167−
168−The system software stack that 2015 inherited -- two '70s
169−designs, Unix and the Internet -- remains a viable platform for
170−"1:n" industrial servers. Maybe it's not a viable platform for
171−"n:1" personal servers? Just as VAX/VMS was not a viable
172−operating system for the PC?
173−
174−But what would a viable platform for a personal server look like?
175−What exactly is this stack? If it's not a Unix server on the
176−Internet, it's an X server on network Y. What are X and Y? Do
177−they exist?
178−
179−Clearly not. So all we have to replace is Unix and the Internet.
180−In other words, all we have to replace is everything. Is this an
181−obstacle, or an opportunity?
182−
183−A clean-slate redesign seems like the obvious path to the levels
184−of simplicity we'll need in a viable planet. Moreover, it's
185−actually easier to redesign Unix and the Internet than Unix *or*
186−the Internet. Computing and communication are not separate
187−concerns; if we design the network and OS as one system, we avoid
188−all kinds of duplications and impedance mismatches.
189−
190−And if not now, when? Will there ever be a clean-slate redesign
191−of '70s system software? A significant problem demands an
192−ambitious solution. Intuitively, clean-slate computing and the
193−personal cloud feel like a fit. Let's see if we can make the
194−details work. We may never get another chance to do it right.
195−
196−Principles of platform reform
197−-----------------------------
198−
199−To review: we don't have planets because our antique system
200−software stack, Unix and the Internet, is a lousy platform on
201−which to build a planet. If we care about the product, we need to
202−start with a new platform.
203−
204−How can we replace Unix and the Internet? We can't. But we can
205−tile over them, like cheap bathroom contractors. We can use the
206−Unix/Internet, or "classical layer," as an implementation
207−substrate for our new layer. A platform on top of a platform.
208−Under your Brazilian slate, there's pink Color Tile from 1976.
209−
210−"Tiling over" is the normal way we replace system software. The
211−Internet originally ran over phone circuits; under your
212−transatlantic TCP packets, there's an ATM switch from 1996. And
213−of course, under your browser there's an OS from 1976.
214−
215−After a good job of tiling over, the obsolete layer can sometimes
216−be removed. In some cases it's useless, but would cost too much
217−to rip out. In some cases it's still used -- a Mac doesn't (yet)
218−boot to Safari. But arguably, that's because the browser platform
219−is anything but a perfect tiling job.
220−
221−One property the browser got right was total opacity. The old
222−platform implements the new platform, but can't be visible
223−through it. If web apps could make Unix system calls or use Unix
224−libraries, there would be no such thing as web apps.
225−
226−(In fact, one easy way to think of a planet is as "the browser
227−for the server side." The browser is one universal client that
228−hosts "n" independent client applications; the planet is one
229−universal server that hosts "n" independent server applications.)
230−
231−And the bathroom remains a bathroom. The new platform does the
232−same *general* job as the old one. So to justify the reform, it
233−has to be *much* better at its new, specific job. For instance,
234−the browser is easily two orders of magnitude better than Unix at
235−installing untrusted transient applications (ie, web pages).
236−
237−Abstract targets
238−----------------
239−
240−Again, we have two problems: Unix and the Internet. What about
241−each do we need to fix? What exactly is wrong with the classical
242−layer? What qualities should our replacement have?
243−
244−Since we're doing a clean-slate design, it's a mistake to focus
245−too much on fixing the flaws of the old platform. The correct
246−question is: what is the right way to build a system software
247−stack? Besides cosmetic details like character sets, this
248−exercise should yield the same results on Mars as Earth.
249−
250−But we come from Earth and will probably find ourselves making
251−normal Earth mistakes. So we can at least express our abstract
252−design goals in normal Earth terms.
253−
254−### A simpler OS
255−
256−One common reaction to the personal-server proposition: "my
257−mother is not a Linux system administrator." Neither is mine. She
258−does own an iPhone, however. Which is also a general-purpose
259−computer. A usability target: a planet should be as easy to
260−manage as an iPhone.
261−
262−(To go into way too much detail: on a planet with the usability
263−of iOS, the user as administrator has four system configuration
264−tasks for the casual or newbie user: (a) deciding which
265−applications to run; (b) checking resource usage; (c) configuring
266−API authentication to your existing Web apps (your planet wants
267−to access, rip or even sync your silo data); and (d) maintaining
268−a reputation database (your friends, enemies, etc). (a) and (b)
269−exist even on iOS; (c) and (d) are inherent in any planet; (d) is
270−common on the Web, and (c) not that rare.)
271−
272−Could a planet just run iOS? The complexity of managing a
273−computer approximates the complexity of its state. iOS is a
274−client OS. Its apps are not much more than webpages. There is
275−much more state on a server; it is much more valuable, and much
276−more intricate, and much more durable.
277−
278−(Moreover, an iOS app is a black box. It's running on your
279−physical hardware, but you don't have much more control over it
280−than if it was remote. iOS apps are not designed to share data
281−with each other or with user-level computing tools; there is no
282−visible filesystem, shell, etc. This is not quite the ownership
283−experience -- it's easy to get locked in to a black box.)
284−
285−And while an Apple product is a good benchmark for any usability
286−goal, it's the exception that proves the rule for architecture.
287−iOS is Unix, after all -- Unix with a billion-dollar makeover.
288−Unix is not a turd, and Cupertino could probably polish it even
289−if it was.
290−
291−Simplicity is the only viable path to usability for a new
292−platform. It's not sufficient, but it is necessary. The computer
293−feels simple to the user not because it's presenting an illusion
294−of simplicity, but because it really is simple.
295−
296−While there is no precise way to measure simplicity, a good proxy
297−is lines of code -- or, to be slightly more sophisticated,
298−compressed code size. Technical simplicity is not actually
299−usability, just a force multiplier in the fight for usability.
300−But usability proper can only be assessed once the UI is
301−complete, and the UI is the top layer by definition. So we assume
302−this equation and target simplicity.
303−
304−### A sane network
305−
306−When we look at the reasons we can't have a nice planet, Unix is
307−a small part of the problem. The main problem is the Internet.
308−
309−There's a reason what we call "social networks" on the Internet
310−are actually centralized systems -- social *servers*. For a "1:n"
311−application, social integration - communication between two users
312−of the same application - is trivial. Two users are two rows in
313−the same database.
314−
315−When we shift to a "n:1" model, this same integration becomes a
316−distributed systems problem. If we're building a tictactoe app in
317−a "1:n" design, our game is a single data structure in which
318−moves are side effects. If we're building the same app on a
319−network of "n:1" model, our game is a distributed system in which
320−moves are network messages.
321−
322−Building and managing distributed Internet systems is not easy.
323−It's nontrivial to build and manage a centralized API. Deploying
324−a new global peer-to-peer protocol is a serious endeavor.
325−
326−But this is what we have to do for our tictactoe app. We have to
327−build, for instance, some kind of identity model - because who
328−are you playing against, an IP address? To play tictactoe, you
329−find yourself building your own distributed social network.
330−
331−While there are certainly asocial apps that a planet can run,
332−it's not clear that an asocial planet is a viable product. If the
333−cost of building a distributed social service isn't close to the
334−cost of a building its centralized equivalent, the design isn't
335−really successful.
336−
337−Fortunately, while there are problems in "n:1" services that no
338−platform can solve for you (mostly around consistency) there are
339−set of problems with "1:n" services that no platform can solve
340−for you (mostly around scaling). Scaling problems in "n:1" social
341−services only arise when you're Justin Bieber and have a million
342−friends, ie, a rare case even in a mature network. Mr. Bieber can
343−probably afford a very nice computer.
344−
345−### Concrete requirements
346−
347−Here are some major features we think any adequate planet needs.
348−They're obviously all features of Urbit.
349−
350−#### Repeatablе computing
351−
352−Any non-portable planet is locked in to its host. That's bad. You
353−can have all the legal guarantees you like of unconditional
354−migration. Freedom means nothing if there's nowhere to run to.
355−Some of today's silos are happy to give you a tarball of your own
356−data, but what would you do with it?
357−
358−The strongest way to ensure portability is a deterministic,
359−frozen, non-extensible execution model. Every host runs exactly
360−the same computation on the same image and input, for all time.
361−When you move that image, the only thing it notices is that its
362−IP address has changed.
363−
364−We could imagine a planet with an unfrozen spec, which had some
365−kind of backward-compatible upgrade process. But with a frozen
366−spec, there is no state outside the planet itself, no input not
367−input to the planet itself, and no way of building a planet on
368−one host that another host can't compute correctly.
369−
370−Of course every computer is deterministic at the CPU level, but
371−CPU-level determinism can't in practice record and replay its
372−computation history. A computer which is deterministic at the
373−semantic level can. Call it "repeatable computing."
374−
375−#### Orthogonal persistence
376−
377−It's unclear why we'd expose the transience semantics of the
378−hardware memory hierarchy to either the programmer or the user.
379−When we do so, we develop two different models for managing data:
380−"programming languages" and "databases." Mapping between these
381−models, eg "ORM," is the quintessence of boilerplate.
382−
383−A simple pattern for orthogonal persistence without a separate
384−database is "prevalence": a checkpoint plus a log of events since
385−the checkpoint. Every event is an ACID transaction. In fact, most
386−databases use this pattern internally, but their state transition
387−function is not a general-purpose interpreter.
388−
389−#### Identity
390−
391−In the classical layer, hosts, usernames, IP addresses, domain
392−names and public keys are all separate concepts. A planet has one
393−routable, memorable, cryptographically independent identity which
394−serves all these purposes.
395−
396−The "Zooko's triangle" impossibility result tells us we can't
397−build a secure, meaningful, decentralized identity system. Rather
398−than surrendering one of these three goals, the planet can
399−retreat a little bit on two of them. It can be memorable, but not
400−meaningful; it can start as a centralized system, but
401−decentralize itself over time. 100-80-70 is often preferable to
402−100-100-0.
403−
404−#### A simple typed functional language
405−
406−Given the level of integration we're expecting in this design,
407−it's silly to think we could get away without a new language.
408−There's no room in the case for glue. Every part has to fit.
409−
410−The main obstacle to functional language adoption is that
411−functional programming is math, and most human beings are really
412−bad at math. Even most programmers are bad at math. Their
413−intuition of computation is mechanical, not mathematical.
414−
415−A pure, higher-order, typed, strict language with mechanical
416−intuition and no mathematical roots seems best positioned to
417−defeat this obstacle. Its inference algorithm should be almost
418−but not quite as strong as Hindley-Milner unification, perhaps
419−inferring "forward but not backward."
420−
421−We'd also like two other features from our types. One, a type
422−should define a subset of values against a generic data model,
423−the way a DTD defines a set of XML values. Two, defining a type
424−should mean defining an executable function, whose range is the
425−type, that verifies or normalizes a generic value. Why these
426−features? See the next section...
427−
428−#### High-level communication
429−
430−A planet could certainly use a network type descriptor that was
431−like a MIME type, if a MIME type was an executable specification
432−and could validate incoming content automatically. After ORM,
433−manual data validation must be the second leading cause of
434−boilerplate. If we have a language in which a type is also a
435−validator, the automatic validation problem seems solvable. We
436−can get to something very like a typed RPC.
437−
438−Exactly-once message delivery semantics are impossible unless the
439−endpoints have orthogonal persistence. Then they're easy: use a
440−single persistent session with monotonically increasing sequence
441−numbers. It's nice not worrying about idempotence.
442−
443−With an integrated OS and protocol, it's possible to fully
444−implement the end-to-end principle, and unify the application
445−result code with the packet acknowledgment -- rather than having
446−three different layers of error code. Not only is every packet a
447−transaction at the system level; every message is a transaction
448−at the application level. Applications can even withhold an
449−acknowledgment to signal internal congestion, sending the other
450−end into exponential backoff.
451−
452−It's also nice to support simple higher-level protocol patterns
453−like publish/subscribe. We don't want every application to have
454−to implement its own subscription queue backpressure. Also, if we
455−can automatically validate content types, perhaps we can also
456−also diff and patch them, making remote syncing practical.
457−
458−The web will be around for a while. It would be great to have a
459−web server that let a browser page with an SSO login authenticate
460−itself as another planet, translating messages into JSON. This
461−way, any distributed application is also a web application.
462−
463−Finally, a planet is also a great web *client*. There is lots of
464−interesting data behind HTTP APIs. A core mission of a planet is
465−collecting and maintaining the secrets the user needs to manage
466−any off-planet data. (Of course, eventually this data should come
467−home, but it may take a while.) The planet's operating system
468−should serve as client-side middleware that mediates the API
469−authentication process, letting the programmer program against
470−the private web as if it was public, using site-specific auth
471−drivers with user-configured secrets.
472−
473−#### Global namespace
474−
475−The URL taught us that any global identity scheme is the root of
476−a global namespace. But the URL also made a big mistake:
477−mutability.
478−
479−A global namespace is cool. An immutable (referentially
480−transparent) global namespace, which can use any data in the
481−universe as if it was a constant, is really cool. It's even
482−cooler if, in case your data hasn't been created yet, your
483−calculation can block waiting for it. Coolest of all is when the
484−data you get back is typed, and you can use it in your typed
485−functional language just like dereferencing a pointer.
486−
487−Of course, an immutable namespace should be a distributed version
488−control system. If we want typed data, it needs to be a typed
489−DVCS, clearly again with type-specific diff and patch. Also, our
490−DVCS (which already needs a subscription mechanism to support
491−blocking) should be very good at reactive syncing and mirroring.
492−
493−#### Semantic drivers
494−
495−One unattractive feature of a pure interpreter is that it exacts
496−an inescapable performance tax -- since an interpreter is always
497−slower than native code. This violates the prime directive of OS
498−architecture: the programmer must never pay for any service that
499−doesn't get used. Impure interpreters partly solve this problem
500−with a foreign-function interface, which lets programmers move
501−inner loops into native code and also make system calls. An FFI
502−is obviously unacceptable in a deterministic computer.
503−
504−A pure alternative to the FFI is a semantic registry in which
505−functions, system or application, can declare their semantics in
506−a global namespace. A smart interpreter can recognize these
507−hints, match them to a checksum of known good code, and run a
508−native driver that executes the function efficiently. This
509−separates policy (pure algorithm as executable specification)
510−from mechanism (native code or even hardware).
511−
512−#### Repository-driven updates
513−
514−A planet owner is far too busy to manually update applications.
515−They have to update themselves.
516−
517−Clearly the right way to execute an application is to run it
518−directly from the version-control system, and reload it when a
519−new version triggers any build dependencies. But in a planet, the
520−user is not normally the developer. To install an app is to
521−mirror the developer's repository. The developer's commit starts
522−the global update process by updating mirror subscriptions.
523−
524−Of course, data structures may change, requiring the developer to
525−include an adapter function that converts the old state. (Given
526−orthogonal persistence, rebooting the app is not an option.)
527−
528−Added fun is provided by the fact that apps use the content types
529−defined in their own repository, and these types may change.
530−Ideally these changes are backward compatible, but an updated
531−planet can still find itself sending an un-updated planet
532−messages that don't validate. This race condition should block
533−until the update has fully propagated, but not throw an error up
534−to the user level -- because no user has done anything wrong.
535−
536−### Why not a planet built on JS or JVM?
537−
538−Many programmers might accept our reasoning at the OS level, but
539−get stuck on Urbit's decision not to reuse existing languages or
540−interpreters. Why not JS, JVM, Scheme, Haskell...? The planet is
541−isolated from the old layer, but can't it reuse mature designs?
542−
543−One easy answer is that, if we're going to be replacing Unix and
544−the Internet, or at least tiling over them, rewriting a bit of
545−code is a small price to pay for doing it right. Even learning a
546−new programming language is a small price to pay. And an
547−essential aspect of "doing it right" is a system of components
548−that fit together perfectly; we need all the simplicity wins we
549−can get.
550−
551−But these are big, hand-waving arguments. It takes more than this
552−kind of rhetoric to justify reinventing the wheel. Let's look at
553−a few details, trying not to get ahead of ourselves.
554−
555−In the list above, only JS and the JVM were ever designed to be
556−isolated. The others are tools for making POSIX system calls.
557−Isolation in JS and the JVM is a client thing. It is quite far
558−from clear what "node.js with browser-style isolation" would even
559−mean. And who still uses Java applets?
560−
561−Let's take a closer look at the JS/JVM options - not as the only
562−interpreters in the world, just as good examples. Here are some
563−problems we'd need them to solve, but they don't solve - not, at
564−least, out of the box.
565−
566−First: repeatability. JS and the JVM are not frozen, but warm;
567−they release new versions with backward compatibility. This means
568−they have "current version" state outside the planet proper. Not
569−lethal but not good, either.
570−
571−When pure, JS and then JVM are at least nominally deterministic,
572−but they are also used mainly on transient data. It's not clear
573−that the the actual implementations and specifications are built
574−for the lifecycle of a planet - which must never miscompute a
575−single bit. (ECC servers are definitely recommended.)
576−
577−Second: orthogonal persistence. Historically, successful OP
578−systems are very rare. Designing the language and OS as one unit
579−seems intuitively required.
580−
581−One design decision that helps enormously with OP is an acyclic
582−data model. Acyclic data structures are enormously easier to
583−serialize, to specify and validate, and of course to
584−garbage-collect. Acyclic databases are far more common than
585−cyclic ("network" or "object") databases. Cyclic languages are
586−more common than acyclic languages -- but pure functional
587−languages are acyclic, so we know acyclic programming can work.
588−
589−(It's worth mentioning existing image-oriented execution
590−environments - like Smalltalk and its descendants, or even the
591−Lisp machine family. These architectures (surely Urbit's closest
592−relatives) could in theory be adapted to use as orthogonally
593−persistent databases, but in practice are not designed for it.
594−For one thing, they're all cyclic. More broadly, the assumption
595−that the image is a GUI client in RAM is deeply ingrained.)
596−
597−Third: since a planet is a server and a server is a real OS, its
598−interpreter should be able to efficiently virtualize itself.
599−There are two kinds of interpreter: the kind that can run an
600−instance of itself as a VM, and the kind that can't.
601−
602−JS can almost virtualize itself with `eval`, but `eval` is a toy.
603−(One of those fun but dangerous toys -- like lawn darts.) And
604−while it's not at all the same thing, the JVM can load applets --
605−or at least, in 1997 it could...
606−
607−(To use some Urbit concepts we haven't met yet: with semantic
608−drivers (which don't exist in JS or the JVM, although asm.js is a
609−sort of substitute), we don't even abandon all the world's JS or
610−JVM code by using Urbit. Rather, we can implement the JS or JVM
611−specifications in Hoon, then jet-propel them with practical Unix
612−JS or JVM engines, letting us run JS or Java libraries.)
613−
614−Could we address any or all of these problems in the context of
615−JS, the JVM or any other existing interpreter? We could. This
616−does not seem likely to produce results either better or sooner
617−than building the right thing from scratch.
618−
619−Definition
620−==========
621−
622−An operating function (OF) is a logical computer whose state is a
623−fixed function of its input history:
624−
625− V(I) => T
626−
627−where `T` is the state, `V` is the fixed function, `I` is the
628−list of input events from first to last.
629−
630−Intuitively, what the computer knows is a function of what it's
631−heard. If the `V` function is identical on every computer for all
632−time forever, all computers which hear the same events will learn
633−the same state from them.
634−
635−Is this function a protocol, an OS, or a database? All three.
636−Like an OS, `V` specifies the semantics of a computer. Like a
637−protocol, `V` specifies the semantics of its input stream. And
638−like a database, `V` interprets a store of persistent state.
639−
640−Advantages
641−----------
642−
643−This is a very abstract description, which doesn't make it easy
644−to see what an OF is useful for.
645−
646−Two concrete advantages of any practical OF (not just Urbit):
647−
648−### Orthogonal persistencе
649−
650−An OF is inherently a single-level store. `V(I) => T` is an
651−equation for the lifecycle of a computer. It doesn't make any
652−distinction between transient RAM and persistent disk.
653−
654−The word "database" has always conflated two concepts: data that
655−isn't transient; structures specialized for search. A substantial
656−percentage of global programmer-hours are spent on translating
657−data between memory and disk formats.
658−
659−Every practical modern database computes something like
660−`V(I) => T`. `I` is the transaction log. An OF is an ACID
661−database whose history function (log to state) is a
662−general-purpose computer. A transaction is an event, an event is
663−a transaction.
664−
665−(Obviously a practical OF, though still defined as `V(I) => T`,
666−does not recompute its entire event history on every event.
667−Rather, it computes some incremental iterator `U` that can
668−compute `U(E T0) => T1`, where `E` is each event in `I`.
669−
670−In plain English, the next state `T1` is a function `U` of the
671−latest event `E` and the current state `T0`. We can assume that
672−some kind of incrementality will fall out of any reasonable `V`.)
673−
674−In any reasonable OF, you can write a persistent application by
675−leaving your data in the structures you use it from. Of course,
676−if you want to use specialized search structures, no one is
677−stopping you.
678−
679−### Repeatable computing
680−
681−Every computer is deterministic at the CPU level, in the sense
682−that the CPU has a manual which is exactly right. But it is not
683−actually practical to check the CPU's work.
684−
685−"Repeatable computing" is high-level determinism. It's actually
686−practical to audit the validity of a repeatable computer by
687−re-executing its computation history. For an OF, the history is
688−the event log.
689−
690−Not every urbit is a bank; not every urbit has to store its full
691−log. Still, the precision of a repeatable computer also affects
692−the user experience. (It's appalling, for instance, how
693−comfortable the modern user is with browser hangs and crashes.)
694−
695−Requirements
696−------------
697−
698−`V` can't be updated for the lifecycle of the computer. Or, since
699−`V` is also a protocol, for the lifetime of the network.
700−
701−So `V` needs to be perfect, which means it needs to be small.
702−It's also easier to eradicate ambiguity in a small definition.
703−
704−But we're designing a general-purpose computer which is
705−programmed by humans. Thus `V` is a programming language in some
706−sense. Few practical languages are small, simple or perfect.
707−
708−`V` is a practical interpreter and should be reasonably fast. But
709−`V` is also a (nonpreemptive) operating system. One universal
710−feature of an OS is the power to virtualize user-level code. So
711−we need a small, simple, perfect interpreter which can
712−efficiently virtualize itself.
713−
714−`V` is also a system of axioms in the mathematical sense. Axioms
715−are always stated informally. This statement succeeds iff it
716−accurately communicates the same axioms to every competent
717−reader. Compressing the document produces a rough metric of
718−information content: the complexity of `V`.
719−
720−(It's possible to cheat on this test. For instance, we could
721−design a simple `V` that only executes another interpreter, `W`.
722−`W`, the only program written in V's language, is simply encoded
723−in the first event. Which could be quite a large event.
724−
725−This design achieves the precision of `V`, but not its stability.
726−For stability, the true extent of `V` is the semantic kernel that
727−the computer can't replace during its lifetime from events in the
728−input history. Thus `V` properly includes `W`.)
729−
730−There are no existing interpreters that ace all these tests, so
731−Urbit uses its own. Our `V` is defined in 350 bytes gzipped.
732−
733−Nouns
734−-----
735−
736−A value in Urbit is a *noun*. A noun is an *atom* or a *cell*. An
737−atom is an unsigned integer of any size. A cell is an ordered
738−pair of nouns.
739−
740−In the system equation `V(I) => T`, `T` is a noun, `I` is a list
741−of nouns - where a list is either `0` or a cell `[item list]`.
742−The `V` function is defined below.
743−
744−Nock
745−----
746−
747−Nock or `N` is a combinator interpreter on nouns. It's specified
748−by these pseudocode reduction rules:
749−
750− Nock(a) *a
751− [a b c] [a [b c]]
752−
753− ?[a b] 0
754− ?a 1
755− +[a b] +[a b]
756− +a 1 + a
757− =[a a] 0
758− =[a b] 1
759− =a =a
760−
761− /[1 a] a
762− /[2 a b] a
763− /[3 a b] b
764− /[(a + a) b] /[2 /[a b]]
765− /[(a + a + 1) b] /[3 /[a b]]
766− /a /a
767−
768− *[a [b c] d] [*[a b c] *[a d]]
769−
770− *[a 0 b] /[b a]
771− *[a 1 b] b
772− *[a 2 b c] *[*[a b] *[a c]]
773− *[a 3 b] ?*[a b]
774− *[a 4 b] +*[a b]
775− *[a 5 b] =*[a b]
776−
777− *[a 6 b c d] *[a 2 [0 1] 2 [1 c d] [1 0] 2 [1 2 3] [1 0] 4 4 b]
778− *[a 7 b c] *[a 2 b 1 c]
779− *[a 8 b c] *[a 7 [[7 [0 1] b] 0 1] c]
780− *[a 9 b c] *[a 7 c 2 [0 1] 0 b]
781− *[a 10 [b c] d] *[a 8 c 7 [0 3] d]
782− *[a 10 b c] *[a c]
783−
784− *a *a
785−
786−Note that operators 6-10 are macros and not formally essential.
787−Note also that Nock reduces invalid reductions to self, thus
788−specifying nontermination (bottom).
789−
790−A valid Nock reduction takes a cell `[subject formula]`. The
791−formula defines a function that operates on the subject.
792−
793−A valid formula is always a cell. If the head of the formula is a
794−cell, Nock reduces head and tail separately and produces the cell
795−of their products ("autocons"). If the head is an atom, it's an
796−operator from `0` to `10`.
797−
798−In operators `3`, `4`, and `5`, the formula's tail is another
799−formula, whose product is the input to an axiomatic function. `3`
800−tests atom/cell, `4` increments, `5` tests equality.
801−
802−In operator `0`, the tail is a "leg" (subtree) address in the
803−subject. Leg `1` is the root, `2n` the left child of `n`, `2n+1`
804−the right child.
805−
806−In operator `1`, the tail is produced as a constant. In operator
807−`2`, the tail is a formula, producing a `[subject formula]` cell
808−which Nock reduces again (rendering the system Turing complete).
809−
810−In the macro department, `6` is if-then-else; `7` is function
811−composition; `8` is a stack push; `9` is a function call; `10` is
812−a hint.
813−
814−Surprisingly, Nock is quite a practical interpreter, although it
815−poses unusual implementation challenges. For instance, the only
816−arithmetic operation is increment (see the implementation issues
817−section for how this is practical). Another atypical feature is
818−that Nock can neither test pointer equality, nor create cycles.
819−
820−The fixed function
821−------------------
822−
823−So is `V` just Nock? Almost. Where `N` is Nock, `V` is:
824−
825− V(I) == N(I [2 [0 3] [0 2]])
826−
827−If Nock was not `[subject formula]` but `[formula subject]`, not
828−`N(S F)` but `N(F S)`, V would be Nock.
829−
830−In either case, the head `I0` of `I` is the boot formula; the
831−tail is the boot subject. Intuitively: to interpret the event
832−history, treat the first event as a program; run that program on
833−the rest of history.
834−
835−More abstractly: the event sequence `I` begins with a *boot
836−sequence* of non-uniform events, which do strange things like
837−executing each other. Once this sequence completes, we end up in
838−a *main sequence* of uniform events (starting at `I5`), which
839−actually look and act like actual input.
840−
841−`I0`: The lifecycle formula
842−---------------------------
843−
844−`I0` is special because we run it in theory but not in practice.
845−Urbit is both a logical definition and a practical interpreter.
846−Sometimes there's tension between these goals. But even in
847−practice, we check that `I0` is present and correct.
848−
849−`I0` is a nontrivial Nock formula. Let's skip ahead and write it
850−in our high-level language, Hoon:
851−
852− => [load rest]=.
853− =+ [main step]=.*(rest load)
854− |- ?~ main step
855− $(main +.main, step (step -.main))
856−
857−Or in pseudocode:
858−
859− from pair $load and $rest
860− let pair $main and $step be:
861− the result of Nock $load run on $rest
862− loop
863− if $main is empty
864− return $step
865− else
866− continue with
867− $main set to tail of $main,
868− $step set to:
869− call function $step on the head of $main
870−
871−In actual Nock (functional machine code, essentially):
872−
873− [8 [2 [0 3] [0 2]] 8 [ 1 6 [5 [1 0] 0 12] [0 13] 9 2
874− [0 2] [[0 25] 8 [0 13] 9 2 [0 4] [0 56] 0 11] 0 7] 9 2 0 1]
875−
876−`I0` is given the sequence of events from `I1` on. It takes `I1`,
877−the boot loader, and runs it against the rest of the sequence.
878−`I1` consumes `I2`, `I3`, and `I4`, and then it produces the true
879−initial state function `step` and the rest of the events from
880−`I5` on.
881−
882−`I0` then continues on to the main sequence, iteratively
883−computing `V(I)` by calling the `step` function over and over on
884−each event. Each call to `step` produces the next state,
885−incorporating the changes resulting from the current event.
886−
887−`I1`: the language formula
888−--------------------------
889−
890−The language formula, `I1`, is another nontrivial Nock formula.
891−Its job is to compile `I3`, the Hoon compiler as source, with
892−`I2`, the Hoon compiler as a Nock formula.
893−
894−If `I3`, compiled with `I2`, equals `I2`, `I1` then uses `I2` to
895−load `I4`, the Arvo source. The main event sequence begins with
896−`I5`. The only events which are Nock formulas are `I0`, `I1`, and
897−`I2`; the only events which are Hoon source are `I3` and `I4`.
898−
899−`I1`, in pseudo-Hoon:
900−
901− => [fab=- src=+< arv=+>- seq=+>+]
902− =+ ken=(fab [[%atom %ud] 164] src)
903− ?> =(+>.fab +.ken)
904− [seq (fab ken arv)]
905−
906−In pseudocode:
907−
908− with tuple as [$fab, $src, $arv, $seq], // these are I2 I3 I4 and I5...
909− let $ken be:
910− call $fab on type-and-value "uint 164" and Hoon $src
911− assert environment from $fab is equal to the value from $ken
912− return pair of the remaining $seq and:
913− call $fab on environment $ken and Hoon $arv
914−
915−In actual Nock:
916−
917− [7 [0 2] 8 [8 [0 2] 9 2 [0 4] [7 [0 3] [1 [1.836.020.833
918− 25.717] 164] 0 6] 0 11] 6 [5 [0 27] 0 5] [[0 31] 8 [0 6]
919− 9 2 [0 4] [7 [0 3] [0 2] 0 30] 0 11] 0 0]
920−
921−(`25.717`? Urbit uses the German dot notation for large atoms.)
922−
923−We compile the compiler source, check that it produces the same
924−compiler formula, and then compile the OS with it.
925−
926−Besides `I0` and `I1`, there's no Nock code in the boot sequence
927−that doesn't ship with Hoon source.
928−
929−`I2` and `I3`: the Hoon compiler
930−--------------------------------
931−
932−Again, `I2` is the Hoon compiler (including basic libraries) as a
933−Nock formula. `I3` is the same compiler as source.
934−
935−Hoon is a strict, typed, higher-order pure functional language
936−which compiles itself to Nock. It avoids category theory and
937−aspires to a relatively mechanical, concrete style.
938−
939−The Hoon compiler works with three kinds of noun: `nock`, a Nock
940−formula; `twig`, a Hoon AST; and `span`, a Hoon type, which
941−defines semantics and constraints on some set of nouns.
942−
943−There are two major parts of the compiler: the front end, which
944−parses a source file (as a text atom) to a twig; and the back
945−end, `ut`, which accepts a subject span and a twig, and compiles
946−them down to a product span and a Nock formula.
947−
948−### Back end and type system
949−
950−The Hoon back end, (`ut`), about 1700 lines of Hoon, performs
951−type inference and (Nock) code generation. The main method of
952−`ut` is `mint`, with signature `$+([span twig] [span Nock])`
953−(i.e., in pseudocode, `mint(span, twig) -> [span Nock]`).
954−
955−Just as a Nock formula executes against a subject noun, a Hoon
956−expression is always compiled against a subject span. From this
957−`[span twig]`, `mint` produces a cell `[span Nock]` . The Nock
958−formula generates a useful product from any noun in the subject
959−span. The span describes the set of product nouns.
960−
961−Type inference in Hoon uses only forward tracing, not unification
962−(tracing backward) as in Hindley-Milner (Haskell, ML). Hoon needs
963−more user annotation than a unification language, but it's easier
964−for the programmer to follow what the algorithm is doing - just
965−because Hoon's algorithm isn't as smart.
966−
967−But the Hoon type system can solve most of the same problems as
968−Haskell's, notably including typeclasses / genericity. For
969−instance, it can infer the type of an AST by compiling a grammar
970−in the form of an LL combinator parser (like Hoon's own grammar).
971−
972−The Hoon type system, slightly simplified for this overview:
973−
974− ++ span $| $? %noun
975− %void
976− ==
977− $% [%atom p=cord]
978− [%cell p=span q=span]
979− [%core p=span q=(map term twig)]
980− [%cube p=noun q=span]
981− [%face p=term q=span]
982− [%fork p=span q=span]
983− [%hold p=span q=twig]
984− ==
985−
986−In pseudocode:
987−
988− define $span as one of:
989− 'noun'
990− 'void'
991− 'atom' with text aura
992− 'cell' of $span and $span
993− 'core' of environment $span and map of name to code AST
994− 'cube' of constant value and $span
995− 'face' of name wrapping $span
996− 'fork' of $span and alternate $span
997− 'hold' of subject $span and continuation AST
998−
999−(The syntax `%string` means a *cord*, or atom with ASCII bytes in
1000−LSB first order. Eg, `%foo` is also 0x6f.6f66 or 7.303.014.)
1001−
1002−The simplest spans are `%noun`, the set of all nouns, and
1003−`%void`, the empty set.
1004−
1005−Otherwise, a span is one of `%atom`, `%cell`, `%core`, `%cube`,
1006−`%face`, `%fork` or `%hold`, each of which is parameterized with
1007−additional data.
1008−
1009−An `%atom` is any atom, plus an *aura* cord that describes both
1010−what logical units the atom represents, and how to print it. For
1011−instance, `%ud` is an unsigned decimal, `%ta` is ASCII text,
1012−`%da` is a 128-bit Urbit date. `%n` is nil (`~`).
1013−
1014−A `%cell` is a recursively typed cell. `%cube` is a constant.
1015−`%face` wraps another span in a name for symbolic access. `%fork`
1016−is the union of two spans.
1017−
1018−A `%core` is a combination of code and data - Hoon's version of
1019−an object. It's a cell `[(map term formula) payload]`, i.e. a set
1020−of named formulas (essentially "computed attributes") and a
1021−"payload", which includes the context the core is defined in.
1022−Each arm uses the whole core as its subject.
1023−
1024−One common core pattern is the *gate*, Hoon's version of a
1025−lambda. A gate is a core with a single formula, and whose payload
1026−is a cell `[arguments context]`. To call the function, replace
1027−the formal arguments with your actual argument, then apply. The
1028−context contains any data or code the function may need.
1029−
1030−(A core is not exactly an object, nor a map of names to formulas
1031−a vtable. The formulas are computed attributes. Only a formula
1032−that produces a gate is the equivalent of a method.)
1033−
1034−### Syntax, text
1035−
1036−Hoon's front end (`vast`) is, at 1100 lines, quite a gnarly
1037−grammar. It's probably the most inelegant section of Urbit.
1038−
1039−Hoon is a keyword-free, ideographic language. All alphabetic
1040−strings are user text; all punctuation is combinator syntax. The
1041−*content* of your code is always alphabetic; the *structure* is
1042−always punctuation. And there are only 80 columns per line.
1043−
1044−A programming language is a UI for programmers. UIs should be
1045−actually usable, not apparently usable. Some UI tasks are harder
1046−than they appear; others are easier. Forming associative memories
1047−is easier than it looks.
1048−
1049−Languages do need to be read out loud, and the conventional names
1050−for punctuation are clumsy. So Hoon replaces them:
1051−
1052− ace [1 space] gal < pel (
1053− bar | gap [>1 space, nl] per )
1054− bas \ gar > sel [
1055− buc $ hax # sem ;
1056− cab _ hep - ser ]
1057− cen % kel { soq '
1058− col : ker } tar *
1059− com , ket ^ tec `
1060− doq " lus + tis =
1061− dot . pam & wut ?
1062− fas / pat @ zap !
1063− sig ~
1064−
1065−For example, `%=` sounds like "centis" rather than "percent
1066−equals." Since even a silent reader will subvocalize, the length
1067−and complexity of the sound is a tax on reading the code.
1068−
1069−A few digraphs also have irregular sounds:
1070−
1071− == stet
1072− -- shed
1073− ++ slus
1074− -> lark
1075− -< lush
1076− +> dark
1077− +< dush
1078−
1079−Hoon defines over 100 digraph ideograms (like `|=`). Ideograms
1080−(or *runes*) have consistent internal structure; for instance,
1081−every rune with the `|` prefix produces a core. There are no
1082−user-defined runes. These facts considerably mitigate the
1083−memorization task.
1084−
1085−Finally, although it's not syntax, style merits a few words.
1086−While names in Hoon code can have any internal convention the
1087−programmer wants (as long as it's lowercase ASCII plus hyphen),
1088−one convention we use a lot: face names which are random
1089−three-letter syllables, arm names which are four-letter random
1090−Scrabble words.
1091−
1092−Again, forming associative memories is easier than it looks.
1093−Random names quickly associate to their meanings, even without
1094−any mnemonic. A good example is the use of Greek letters in math.
1095−As with math variables, consistency helps too - use the same name
1096−for the same concept everywhere.
1097−
1098−This "lapidary" convention is appropriate for code that's
1099−nontrivial, but still clean and relatively short. But code can't
1100−be clean unless it's solving a clean problem. For dirty problems,
1101−long informative names and/or kebab-case are better. For trivial
1102−problems (like defining `add`), we use an even more austere
1103−"hyperlapidary" style with one-letter names.
1104−
1105−### Twig structure
1106−
1107−A Hoon expression compiles to a `twig`, or AST node. A twig is
1108−always a cell.
1109−
1110−Like Nock formulas, Hoon twigs "autocons." A twig whose head is a
1111−cell is a pair of twigs, which compiles to a pair of formulas,
1112−which is a formula producing a pair. ("autocons" is an homage to
1113−Lisp, whose `(cons a (cons b c))` becomes Urbit's `[a b c]`.)
1114−
1115−Otherwise, a twig is a tagged union. Its tag is always a rune,
1116−stored by its phonetic name as a cord. Because it's normal for
1117−31-bit numbers to be direct (thus, somewhat more efficient), we
1118−drop the vowels. So `=+` is not `%tislus`, but `%tsls`.
1119−
1120−There are too many kinds of twig to enumerate here. Most are
1121−implemented as macros; only about 25 are directly translated to
1122−Nock. There is nothing sophisticated about twigs - given the
1123−syntax and type system, probably anyone would design the same
1124−twig system.
1125−
1126−### Syntax geometry
1127−
1128−Just visually, one nice feature of imperative languages is the
1129−distinction between statements, which want to grow vertically
1130−down the page, and expressions, which prefer to grow
1131−horizontally.
1132−
1133−Most functional languages, inherently lacking the concept of
1134−statements, have a visual problem: their expressions grow best
1135−horizontally, and often besiege the right margin.
1136−
1137−Hoon has two syntax modes: "tall," which grows vertically, and
1138−"wide," which grows horizontally. Tall code can contain wide
1139−code, but not vice versa. In general, any twig can be written in
1140−either mode.
1141−
1142−Consider the twig `[%tsls a b]`, where `a` and `b` are twigs.
1143−`=+` defines a variables, roughly equivalent to Lisp's `let`. `a`
1144−defines the variable for use in `b`. In wide mode it looks like
1145−an expression: `=+(a b)`. In tall mode, the separator is two
1146−spaces or a newline, without parentheses:
1147−
1148− =+ a
1149− b
1150−
1151−which is the same code, but looks like a "statement." The
1152−deindentation is reasonable because even though `b` is, according
1153−to the AST, "inside" the `=+`, it's more natural to see it
1154−separated out.
1155−
1156−For twigs with a constant number of components, like
1157−`[%tsls a b]`, tall mode relies on the parser to stop itself.
1158−With a variable fanout we need a `==` terminator:
1159−
1160− :* a
1161− b
1162− ==
1163−
1164−The indentation rules of constant-fanout twigs are unusual.
1165−Consider the stem `?:`, which happens to have the exact same
1166−semantics as C `?:` (if-then-else). Where in wide form we'd write
1167−`?:(a b c)`, in tall form the convention is
1168−
1169− ?: a
1170− b
1171− c
1172−
1173−This is *backstep* indentation. Its motivation: defend the right
1174−margin. Ideally, `c` above is a longer code block than `a` or
1175−`b`. And in `c`, we have not lost any margin at all. Ideally, we
1176−can write an arbitrarily tall and complex twig that always grows
1177−vertically and never has a margin problem. Backstep indentation
1178−is tail call optimization for syntax.
1179−
1180−Finally, the wide form with rune and parentheses (like `=+(a b)`)
1181−is the *normal* wide form. Hoon has a healthy variety of
1182−*irregular* wide forms, for which no principles at all apply. But
1183−using normal forms everywhere would get quite cumbersome.
1184−
1185−`I4`: Arvo
1186−----------
1187−
1188−`I4` is the source code for Arvo: about 600 lines of Hoon. This
1189−is excessive for what Arvo does and can probably be tightened. Of
1190−course, it does not include the kernel modules (or *vanes*).
1191−
1192−`I4` formally produces the kernel step function, `step` for `I0`.
1193−But in a practical computer, we don't run `I0`, and there are
1194−other things we want to do besides `step` it. Inside `step` is
1195−the Arvo core (leg 7, context), which does the actual work.
1196−
1197−Arvo's arms are `keep`, `peek`, `poke`, `wish`, `load`.
1198−
1199−`wish` compiles a Hoon string. `keep` asks Arvo when it wants to
1200−be woken up next. `peek` exposes the Arvo namespace. These three
1201−are formally unnecessary, but useful in practice for the Unix
1202−process that runs Arvo.
1203−
1204−The essential Arvo arm is `poke`, which produces a gate which
1205−takes as arguments the current time and an event. (Urbit time is
1206−a 128-bit atom, starting before the universe and ending after it,
1207−with 64 bits for subseconds and 64 bits for seconds, ignoring any
1208−post-2015 leap seconds.)
1209−
1210−The product of the `poke` gate is a cell `[action-list Arvo]`. So
1211−Arvo, poked with the time and an event, produces a list of
1212−actions and a new Arvo state.
1213−
1214−Finally, `load` is used to replace the Arvo core. When we want to
1215−change Hoon or Arvo proper, we build a new core with our new
1216−tools, then start it by passing `load` the old core's state.
1217−(State structures in the new core need not match the old, so the
1218−new core may need adapter functions.) `load` produces its new
1219−core wrapped in the canonical outer `step` function.
1220−
1221−Since the language certainly needs to be able to change, the
1222−calling convention to the next generation is a Nock convention.
1223−`load` is leg `54` of the core battery, producing a gate whose
1224−formal argument at leg `6` is replaced with the Arvo state, then
1225−applied to the formula at leg `2`. Of course, future cores can
1226−use a different upgrade convention. Nothing in Arvo or Urbit is
1227−technically frozen, except for Nock.
1228−
1229−State
1230−-----
1231−
1232−Arvo has three pieces of dynamic state: its network address, or
1233−*plot*; an entropy pool; the kernel modules, or *vanes*.
1234−
1235−Configuration overview
1236−----------------------
1237−
1238−The first main-sequence event, `I5`, is `[%init plot]`, with some
1239−unique Urbit address (*plot*) whose secrets this urbit controls.
1240−The plot, an atom under 2\^128, is both a routing address and a
1241−cryptographic name. Every Urbit event log starts by setting its
1242−own unique and permanent plot.
1243−
1244−From `I6` we begin to install the vanes: kernel modules. Vanes
1245−are installed (or reloaded) by a `[%veer label code]` event. Vane
1246−labels by convention are cords with zero or one letter.
1247−
1248−The vane named `%$` (zero, the empty string) is the standard
1249−library. This contains library functions that aren't needed in
1250−the Hoon kernel, but are commonly useful at a higher level.
1251−
1252−The standard library is compiled with the Hoon kernel (`I2`) as
1253−subject. The other vanes are compiled with the standard library
1254−(which now includes the kernel) as subject. Vane `%a` handles
1255−networking; `%c`, storage; `%f`, build logic; `%g`, application
1256−lifecycle; `%e`, http interface; `%d`, terminal interface; `%b`,
1257−timers; and `%j`, storage of secrets.
1258−
1259−Mechanics
1260−---------
1261−
1262−Vanes are stored as a cell of a noun with its span. At least in
1263−Urbit, there is no such thing as a dynamic type system - only a
1264−static type system executed at runtime.
1265−
1266−Storing vanes with their spans has two benefits. One, it lets us
1267−type-check internal event dispatch. Two, it lets us do typed
1268−upgrades when we replace a vane. It makes routine dispatch
1269−operations incur compiler cost, but this caches well.
1270−
1271−Input and output
1272−----------------
1273−
1274−From the Unix process's perspective, Arvo consumes events (which
1275−Unix generates) and produces actions (which Unix executes). The
1276−most common event is hearing a UDP packet; the most common action
1277−is sending a UDP packet. But Arvo also interacts (still in an
1278−event-driven pattern) with other Unix services, including HTTP,
1279−the console, and the filesystem.
1280−
1281−Events and actions share the `ovum` structure. An ovum is a cell
1282−`[wire card]`. A wire is a path - a list of cords, like
1283−`[%foo %bar %baz ~]`, usually written with the irregular syntax
1284−`/foo/bar/baz`. A card is a tagged union of all the possible
1285−types of events and effects.
1286−
1287−A wire is a *cause*. For example, if the event is an HTTP request
1288−`%thus`, the wire will contain (printed to cords) the server port
1289−that received the request, the IP and port of the caller, and the
1290−socket file descriptor.
1291−
1292−The data in the wire is opaque to Urbit. But an Urbit `poke`, in
1293−response to this event or to a later one, can answer the request
1294−with an action `%this` - an HTTP response. Unix parses the action
1295−wire it sent originally and responds on the right socket file
1296−descriptor.
1297−
1298−Moves and ducts
1299−---------------
1300−
1301−Event systems are renowned for "callback hell". As a purely
1302−functional environment, Arvo can't use callbacks with shared
1303−state mutation, a la node. But this is not the only common
1304−pitfall in the event landscape.
1305−
1306−A single-threaded, nonpreemptive event dispatcher, like node or
1307−Arvo, is analogous to a multithreaded preemptive scheduler in
1308−many ways. In particular, there's a well-known duality between
1309−event flow and control flow.
1310−
1311−One disadvantage of many event systems is unstructured event
1312−flow, often amounting to "event spaghetti". Indeed, the
1313−control-flow dual of an unstructured event system is `goto`.
1314−
1315−Arvo is a structured event system whose dual is a call stack. An
1316−internal Arvo event is called a "move". A move has a duct, which
1317−is a list a wires, each of which is analagous to a stack frame.
1318−Moves come in two kinds: a `%pass` move calls upward, pushing a
1319−frame to the duct, while a `%give` move returns a result
1320−downward, popping a frame off the duct.
1321−
1322−`%pass` contains a target vane name; a wire, which is a return
1323−pointer that defines this move's cause within the source vane;
1324−and a card, the event data. `%give` contains just the card.
1325−
1326−The product of a vane event handler is a cell
1327−`[moves new-state]`, a list of moves and the vane's new state.
1328−
1329−On receiving a Unix event, Arvo turns it into a `%pass` by
1330−picking a vane from the Unix wire (which is otherwise opaque),
1331−then pushes it on a move stack.
1332−
1333−The Arvo main loop pops a move off the move stack, dispatches it,
1334−replaces the result vane state, and pushes the new moves on the
1335−stack. The Unix event terminates when the stack is empty.
1336−
1337−To dispatch a `%pass` move sent by vane `%x`, to vane `%y`, with
1338−wire `/foo/bar`, duct `old-duct`, and card `new-card`, we pass
1339−`[[/x/foo/bar old-duct] new-card]`, a `[duct card]` cell, to the
1340−`call` method on vane `%y`. In other words, we push the given
1341−wire (adding the vane name onto the front) on to the old duct to
1342−create the new duct, and then pass that along with the card to
1343−the other vane.
1344−
1345−To dispatch a `%give` move returned by vane `%x`, we check if the
1346−duct is only one wire deep. If so, return the card as an action
1347−to Unix. If not, pull the original calling vane from the top wire
1348−on the duct (by destructuring the duct as
1349−`[[vane wire] plot-duct]`), and call the `take` method on `vane`
1350−with `[plot-duct wire card]`.
1351−
1352−Intuitively, a pass is a service request and a give is a service
1353−response. The wire contains the information (normalized to a list
1354−of strings) that the caller needs to route and process the
1355−service result. The effect always gets back to its cause.
1356−
1357−A good example of this mechanism is any internal service which is
1358−asynchronous, responding to a request in a different *system*
1359−event than its cause - perhaps many seconds later. For instance,
1360−the `%c` vane can subscribe to future versions of a file.
1361−
1362−When `%c` gets its service request, it saves it with the duct in
1363−a table keyed by the subscription path. When the future version
1364−is saved on this path, the response is sent on the duct it was
1365−received from. Again, the effect gets back to its cause.
1366−Depending on the request, there may even be multiple responses to
1367−the same request, on the same duct.
1368−
1369−In practice, the structures above are slightly simplified - Arvo
1370−manages both vanes and moves as vases, `[span noun]` cells. Every
1371−dispatch is type-checked.
1372−
1373−One card structure that Arvo detects and automatically unwraps is
1374−`[%meta vase]` - where the vase is the vase of a card. `%meta`
1375−can be stacked up indefinitely. The result is that vanes
1376−themselves can operate internally at the vase level - dynamically
1377−executing code just as Arvo itself does.
1378−
1379−The `%g` vane uses `%meta` to expose the vane mechanism to
1380−user-level applications. The same pattern, a core which is an
1381−event transceiver, is repeated across four layers: Unix, Arvo,
1382−the `%g` vane, and the `:dojo` shell application.
1383−
1384−Event security
1385−--------------
1386−
1387−Arvo is a "single-homed" OS which defines one plot, usually with
1388−the label `our`, as its identity for life. All vanes are fully
1389−trusted by `our`. But internal security still becomes an issue
1390−when we execute user-level code which is not fully trusted, or
1391−work with data which is not trusted.
1392−
1393−Our security model is causal, like our event system. Every event
1394−is a cause and an effect. Event security is all about *who/whom*:
1395−*who* (other than us) caused this event; *whom* (other than us)
1396−it will affect. `our` is always authorized to do everything and
1397−hear anything, so strangers alone are tracked.
1398−
1399−Every event has a security `mask` with two fields `who` and
1400−`hum`, each a `(unit (set plot))`. (A `unit` is the equivalent of
1401−Haskell's `Maybe` - `(unit x`) is either `[~ x]` or `~`, where
1402−`~` is nil.)
1403−
1404−If `who` is `~`, nil, anyone else could have caused this move --
1405−in other words, it's completely untrusted. If `who` is `[~ ~]`,
1406−the empty set, no one else caused this move -- it's completely
1407−trusted. Otherwise, the move is "tainted" by anyone in the set.
1408−
1409−If `hum` is `~`, nil, anyone else can be affected by this move --
1410−in other words, it's completely unfiltered. If `hum` is `[~ ~]`,
1411−the empty set, no one else can hear the move -- it's completely
1412−private. Otherwise, the move is permitted to "leak" to anyone in
1413−the set.
1414−
1415−Obviously, in most moves the security mask is propagated from
1416−cause to effect without changes. It's the exceptions that keep
1417−security exciting.
1418−
1419−Namespace
1420−---------
1421−
1422−Besides `call` and `take`, each vane exports a `scry` gate whose
1423−argument is a `path` - a list of strings, like a wire.
1424−
1425−`scry` implements a global monotonic namespace - one that (a)
1426−never changes its mind, for the lifetime of the urbit; and (b)
1427−never conflicts across well-behaved urbits.
1428−
1429−This invariant is semantic - it's not enforced by the type
1430−system. Getting it right is up to the vane's developer.
1431−Installing kernel modules is always a high-trust operation.
1432−
1433−The product of the `scry` gate is `(unit (unit value))`.
1434−
1435−So `scry` can produce `~`, meaning the path is not yet bound;
1436−`[~ ~]`, meaning the path is bound to empty; or `[~ ~ value]`, an
1437−actual value is bound. This value is of the form
1438−`[mark span noun]`, where `span` is the type of the noun and
1439−`mark` is a higher-level "type" label best compared to a MIME
1440−type or filename extension. Marks are discussed under the `%f`
1441−vane.
1442−
1443−Vane activation and namespace invariant
1444−---------------------------------------
1445−
1446−The vane that Arvo stores is not the core that exports `call`
1447−etc, but a gate that produces the core. The argument to this gate
1448−is a cell `[@da $+(path (unit (unit value)))]`. Note that
1449−`$+(path (unit (unit value)))` is just the function signature of
1450−the `scry` namespace.
1451−
1452−So, the head of the argument is the current time; the tail is the
1453−system namespace. The general namespace is constructed from the
1454−`scry` method of the vanes. The first letter of the head of the
1455−path identifies the vane; the rest of the head, with the rest of
1456−the path, is passed to the vane. The vane core is activated just
1457−this way for all its methods, including `scry` itself.
1458−
1459−This answers the question of how to expose dynamic state without
1460−violating our referential transparency invariants. If we require
1461−the query to include exactly the current time (which is
1462−guaranteed to change between events) in the path, then we are
1463−able to respond with the current state of the dynamic data. If
1464−the path doesn't contain the current time, then fail with
1465−`[~ ~]`. This technique is only needed, of course, when you don't
1466−know what the state was in the past.
1467−
1468−Additionally, since each vane knows the plot it's on, a `scry`
1469−query with the plot in the path is guaranteed to be globally
1470−unique.
1471−
1472−But vanes do not have to rely on these safe methods to maintain
1473−monotonicity - for instance, the `%c` revision-control vane binds
1474−paths around the network and across history.
1475−
1476−### A tour of the vanes
1477−
1478−The vanes are separately documented, because that's the entire
1479−point of vanes. Let's take a quick tour through the system as it
1480−currently stands, however.
1481−
1482−### `%a` `%ames`: networking
1483−
1484−`%a` is the network vane, currently `%ames` (2000 lines). `%ames`
1485−implements an encrypted P2P network over UDP packets.
1486−
1487−As a vane, `%ames` provides a simple `%pass` service: send a
1488−message to another plot. The message is an arbitrary
1489−`[wire noun]` cell: a message channel and a message body. `%ames`
1490−will respond with a `%give` that returns either success or
1491−failure and an error dump.
1492−
1493−`%ames` messages maintain causal integrity across the network.
1494−The sender does not send the actual duct that caused the message,
1495−of course, but an opaque atom mapped to it. This opaque `bone`,
1496−plus the channel itself, are the "socket" in the RFC sense.
1497−
1498−Messages are ordered within this socket and delivered exactly
1499−once. (Yes, as normally defined this is impossible. Urbit can do
1500−exactly-once because `%ames` messaging is not a tool to build a
1501−consistency layer, but a tool built on the consistency layer.
1502−Thus, peers can have sequence numbers that are never reset.)
1503−
1504−More subtly, local and remote causes treated are exactly the same
1505−way, improving the sense of network abstraction. CORBA taught us
1506−that it's not possible to abstract RPC over local function calls
1507−without producing a leaky abstraction. The Arvo structured event
1508−model is designed to abstract over messages.
1509−
1510−One unusual feature is that `%ames` sends end-to-end acks.
1511−Acknowledging the last packet received in a message acknowledges
1512−that the message itself has been fully accepted. There is no
1513−separate message-level result code. Like its vane interface,
1514−`%ames` acks are binary: a positive ack has no data, a negative
1515−ack has a dump. Thus, Arvo is transactional at the message level
1516−as well as the packet/event level.
1517−
1518−Another `%ames` technique is dropping packets. For instance, it
1519−is always a mistake to respond to a packet with bad encryption:
1520−it enables timing attacks. Any packet the receiver doesn't
1521−understand should be dropped. Either the sender is deranged or
1522−malicious, or the receiver will receive a future upgrade that
1523−makes a later retransmission of this packet make sense.
1524−
1525−Messages are encrypted in a simple PKI (currently RSA, soon
1526−curve/ed25519) and AES keys are exchanged. Certificate transfer
1527−and key exchange are aggressively piggybacked, so the first
1528−packet to a stranger is always signed but not encrypted. The idea
1529−is that if you've never talked to someone before, probably the
1530−first thing you say is "hello." Which is not interesting for any
1531−realistic attacker. In the rare cases where this isn't true, it's
1532−trivial for the application to work around. (See the network
1533−architecture section for more about the PKI approach.)
1534−
1535−(For packet geeks only: `%ames` implements both "STUN" and "TURN"
1536−styles of peer-to-peer NAT traversal, using parent plots (see
1537−below) as supernodes. If the sending plot lacks a current IP/port
1538−(Urbit uses random ports) for the destination, it forwards across
1539−the parent hierarchy. As we forward a packet, we attach the
1540−sender's address, in case we have a full-cone NAT that can do
1541−STUN.
1542−
1543−When these indirect addresses are received, they are regarded
1544−with suspicion, and duplicate packets are sent - one forwarded,
1545−one direct. Once a direct packet is received, the indirect
1546−address is dropped. Forwarding up the hierarchy always succeeds,
1547−because galaxies (again, see below) are bound into the DNS at
1548−`galaxy.urbit.org`.)
1549−
1550−### `%c` `%clay`: filesystem
1551−
1552−`%c` is the filesystem, currently `%clay` (3000 lines). `%clay`
1553−is a sort of simplified, reactive, typed `git`.
1554−
1555−The `%clay` filesystem uses a uniform inode structure, `ankh`. An
1556−ankh contains: a Merkle hash of this subtree, a set of named
1557−children, and possibly file data. If present, the file data is
1558−typed and marked (again, see the `%f` vane for more about marked
1559−data).
1560−
1561−A path within the tree always contains the plot and desk
1562−(lightweight branch) where the file is located, which version of
1563−the file to use, and what path within the desk the file is at.
1564−
1565−Paths can be versioned in three ways: by change number (for
1566−changes within this desk); by date; or by label.
1567−
1568−Where `git` has multiple parallel states, `%clay` uses separate
1569−desks. For instance, where `git` creates a special merge state,
1570−`%clay` just uses a normal scratch desk. Don't detach your head,
1571−merge an old version to a scratch desk. Don't have explicit
1572−commit messages, edit a log file. `%clay` is a RISC `git`.
1573−
1574−`%clay` is a typed filesystem; not only does it save a type with
1575−each file, but it uses `mark` services from `%f` to perform typed
1576−diff and patch operations that match the mark types. Obviously,
1577−Hunt-McIlroy line diff only works for text files. `%clay` can
1578−store and revise arbitrary custom data structures, not a
1579−traditional revision-control feature.
1580−
1581−The `scry` namespace exported by clay uses the mode feature to
1582−export different views of the filesystem. For instance,
1583−`[%cx plot desk version path]`, which passes `%x` as the mode to
1584−clay's `scry`, produces the file (if any) at that path. `%y`
1585−produces the directory, essentially - the whole ankh with its
1586−subtrees trimmed. `%z` produces the whole ankh.
1587−
1588−`%clay` is reactive; it exports a subscription interface, with
1589−both local and network access. A simple use is waiting for a file
1590−or version which has not yet been committed.
1591−
1592−A more complex use of `%clay` subscription is synchronization,
1593−which is actually managed at the user level within a `%gall`
1594−application (`:hood`). It's straightforward to set up complex,
1595−multistep or even cyclical chains of change propagation.
1596−Intuitively, `git` only pulls; `%clay` both pulls and pushes.
1597−
1598−Finally, the easiest way to use `%clay` is to edit files directly
1599−from Unix. `%clay` can mount and synchronize subtrees in the
1600−Urbit home directory. There are two kinds of mount: export and
1601−sync. While an export mount marks the Unix files read-only, sync
1602−mounts support "dropbox" style direct manipulation. The Unix
1603−process watches the subtree with `inotify()` or equivalent, and
1604−generates filesystem change events automagically. A desk which is
1605−synced with Unix is like your `git` working tree; a merge from
1606−this working desk to a more stable desk is like a `git` commit.
1607−
1608−Access control in `%clay` is an attribute of the desk, and is
1609−either a blacklist or whitelist of plots. Creating and
1610−synchronizing desks is easy enough that the right way to create
1611−complex access patterns is to make a desk for the access pattern,
1612−and sync only what you want to share into it.
1613−
1614−### `%f` `%ford`: builder
1615−
1616−`%f` is the functional build system, currently `%ford` (1800
1617−lines). It might be compared to `make` et al, but loosely.
1618−
1619−`%ford` builds things. It builds applications, resources, content
1620−type conversions, filter pipelines, more or less any functional
1621−computation specified at a high level. Also, it caches repeated
1622−computations and exports a dependency tracker, so that `%ford`
1623−users know when they need to rebuild.
1624−
1625−`%ford` does all its building and execution in the virtual Nock
1626−interpreter, `mock`. `mock` is a Nock interpreter written in
1627−Hoon, and of course executed in Nock. (See the implementation
1628−issues section for how this is practical.)
1629−
1630−`mock` gives us two extra affordances. One, when code compiled
1631−with tracing hints crashes deterministically, we get a
1632−deterministic stack trace. Two, we add a magic operator `11`
1633−(`.^` in Hoon) which dereferences the global namespace. This is
1634−of course referentially transparent, and `mock` remains a
1635−functional superset of Nock.
1636−
1637−`%ford` is passed one card, `[%exec silk]`, which specifies a
1638−computation - essentially, a makefile as a noun. The `silk`
1639−structure is too large to enumerate here, but some highlights:
1640−
1641−The simplest task of `%ford` is building code, by loading sources
1642−and resources from `%clay`. For kernel components, a source file
1643−is just parsed directly into a `twig`. `%ford` can do a lot of
1644−work before it gets to the `twig`.
1645−
1646−The Hoon parser for a `%ford` build actually parses an extended
1647−build language wrapped around Hoon. Keeping the build
1648−instructions and the source in one file precludes a variety of
1649−exciting build mishaps.
1650−
1651−For any interesting piece of code, we want to include structures,
1652−libraries, and data resources. Some of these requests should be
1653−in the same `beak` (plot/desk/version) as the code we're
1654−building; some are external code, locked to an external version.
1655−
1656−`%ford` loads code from `%clay` with a *role* string that
1657−specifies its use, and becomes the head of the spur. (Ie, the
1658−first path item after plot, desk and version.) Roles: structures
1659−are in `/sur`, libraries in `/lib`, marks in `/mar`, applications
1660−in `/app`, generators in `/gen`, fabricators in `/fab`, filters
1661−in `/tip`, and anything not a Hoon file in `/doc`.
1662−
1663−Data resources are especially interesting, because we often want
1664−to compile whole directory trees of resources into a single noun,
1665−a typed constant from the point of view of the programmer. Also,
1666−requests to `%clay` may block - `%ford` will have to delay giving
1667−its result, and wait until a subscription for the result returns.
1668−
1669−Moreover, for any interesting build, the builder needs to track
1670−dependencies. With every build, `%ford` exports a dependency key
1671−that its customers can subscribe to, so that they get a
1672−notification when a dependency changes. For example, `%gall` uses
1673−this feature to auto-reload applications.
1674−
1675−Finally, both for loading internal resources in a build and as a
1676−vane service (notably used by the web vane `%eyre`), `%ford`
1677−defines a functional namespace which is mapped over the static
1678−document space in `/doc`.
1679−
1680−If we request a resource from `%ford` that does not correspond to
1681−a file in `/doc`, we search in `/fab` for the longest matching
1682−prefix of the path. The fabricator receives as its sole argument
1683−that part of the path not part of the matching prefix.
1684−
1685−Thus, if we search for `/a/b/c/d`, and there's no `/doc/a/b/c/d`,
1686−then we first check for `/fab/a/b/c/d`, then `/fab/a/b/c`, and so
1687−on. If we find, for example, a `/fab/a/b`, then we run that
1688−fabricator with the argument `/c/d`. Thus, a fabricator can
1689−present an arbitrary virtual document tree.
1690−
1691−The `%ford` vane also exports its namespace through `scry`, but
1692−`scry` has no way to block if a computation waits. It will just
1693−produce `~`, binding unknown.
1694−
1695−Another major `%ford` feature is the mark system. While the Hoon
1696−type system works very well, it does not fill the niche that MIME
1697−types fill on the Internets. Marks are like MIME types, if MIME
1698−types came with with executable specifications which defined how
1699−to validate, convert, diff, and patch content objects.
1700−
1701−The mark `%foo` is simply the core built from `/mar/foo`. If
1702−there is no `/mar/foo`, all semantics are defaulted - `%foo` is
1703−treated as `%noun`. If there is a core, it can tell `%ford` how
1704−to validate/convert from other marks (validation is always
1705−equivalent to a conversion from `%noun`, and seldom difficult,
1706−since every Hoon structure is defined as a fixpoint normalizer);
1707−convert *to* other marks; patch, diff, and merge. Obviously
1708−`%clay` uses these revision control operations, but anyone can.
1709−
1710−For complex conversions, `%ford` has a simple graph analyzer that
1711−can convert, or at least try to convert, any source mark to any
1712−target mark.
1713−
1714−There are two ford request modes, `%c` and `%r` - cooked and raw.
1715−A cooked request, `%fc` in `scry`, has a target mark and converts
1716−the result to it. A raw request, `%fr`, returns whatever `%ford`
1717−finds easiest to make.
1718−
1719−Generators and filters, `/gen` and `/tip`, are actually not used
1720−by any other vane at present, but only by the user-level `:dojo`
1721−app, which constructs dataflow calculations in Unix pipe style.
1722−Applications, `/app`, are used only by the `%g` vane.
1723−
1724−### `%g` `%gall`: application driver
1725−
1726−`%g` is the application system, currently `%gall` (1300 lines)
1727−`%gall` is half process table and half systemd, sort of.
1728−
1729−`%gall` runs user-level applications much as Arvo runs vanes. Why
1730−this extra layer? Vanes are kernel components written by kernel
1731−hackers, for whom expressiveness matters more than convenience. A
1732−bad kernel module can disable the OS; a bad application has to be
1733−firewalled.
1734−
1735−But the basic design of an application and a vane is the same: a
1736−stateful core that's called with events and produces moves. Only
1737−the details differ, and there are too many to cover here.
1738−
1739−One example is that `%gall` apps don't work with Arvo ducts
1740−directly, but opaque integers that `%gall` maps to ducts; the
1741−consequences of a bad app making a bad duct would be odd and
1742−potentially debilitating, so we don't let them touch ducts
1743−directly. Also, `%gall` does not like to crash, so to execute
1744−user-level code it uses the virtual interpeter `mock`. At the
1745−user level, the right way to signal an error is to crash and let
1746−`%gall` pick up the pieces - in a message handler, for instance,
1747−this returns the crash dump in a negative ack.
1748−
1749−`%gall` uses `%ford` to build cores and track dependencies. Like
1750−vanes, `%gall` applications update when new code is available,
1751−sometimes using adapter functions to translate old state types to
1752−new ones. Unlike Arvo, `%gall` triggers updates automatically
1753−when a dependency changes.
1754−
1755−Other than convenience and sanity checks, the principal value add
1756−of `%gall` is its inter-application messaging protocol. `%gall`
1757−messaging supports two communication patterns: a one-way message
1758−with a success/error result (`%poke`), and classic publish and
1759−subscribe (`%peer`).
1760−
1761−`%peer` is designed for synchronizing state. The subscriber
1762−specifies a path which defines an application resource, and
1763−receives a stream of `%full` and `%diff` cards, total and
1764−incremental updates. (A simple "get" request is a special case in
1765−which there is a single full and no diffs.) Backpressure breaks
1766−the subscription if the queue of undelivered updates grows too
1767−deep. Broken subscriptions should not cause user-level errors;
1768−the user should see an error only if the subscription can't be
1769−reopened.
1770−
1771−Recall that `%ames` routes messages which are arbitrary nouns
1772−between vanes on different urbits. `%gall` uses these `%ames`
1773−messages for remote urbits, and direct moves for local IPC. Thus,
1774−from the app's perspective, messaging an app on another Urbit is
1775−the same as messaging another app on the same Urbit. The
1776−abstraction does not leak.
1777−
1778−Messages are untyped at the `%ames` level, but all `%gall`
1779−messages carry a mark and are validated by the receiver. Marks
1780−are not absolute and timeless - `%ford` needs a `beak` (plot,
1781−desk, version) to load the mark source.
1782−
1783−When a message fails to validate, the packet is dropped silently.
1784−This puts the sender into exponential backoff retransmission. The
1785−general cause of a message that doesn't validate is that the
1786−sender's mark has received a backward-compatible update (mark
1787−updates that aren't backward compatible are a bad idea - use a
1788−new name), and the receiver doesn't have this update yet. The
1789−retransmitted packet will be processed correctly once the update
1790−has propagated.
1791−
1792−With this mechanism, Urbit can update a distributed system (such
1793−as our own `:talk` network), upgrading both applications and
1794−protocols, silently without user intervention or notification.
1795−The general pattern of application distribution is that the app
1796−executes from a local desk autosynced to a remote urbit, which
1797−performs the function of an app store or distro. As this server
1798−fills its subscriptions, a period of network heterogeneity is
1799−inevitable; and so is transient unavailability, as new versions
1800−try to talk to old ones. But it resolves without hard errors as
1801−all clients are updated.
1802−
1803−### `%e` `%eyre`: web server/client
1804−
1805−`%e` is the web system, currently `%eyre` (1600 lines).
1806−
1807−`%eyre` has three purposes. First, it is an HTTP and HTTPS
1808−client - or rather, it interfaces via actions/events to HTTP
1809−client handlers in the Unix layer.
1810−
1811−Second, `%eyre` serves the `%ford` namespace over HTTP. The URL
1812−extension is parsed as a mark, but the default mark for requests
1813−is `urb`, which creates HTML and injects an autoupdate script
1814−that long-polls on the dependencies, reloading the page in the
1815−background when they change.
1816−
1817−The normal way of publishing static or functional content in
1818−Urbit is to rely on `%ford` for format translation. Most static
1819−content is in markdown, the `%md` mark. Dynamic content is best
1820−generated with the `sail` syntax subsystem in Hoon, which is is
1821−essentially an integrated template language that reduces to XML.
1822−
1823−Third, `%eyre` acts as a client for `%gall` apps, translating the
1824−`%gall` message flow into JSON over HTTP. `%poke` requests become
1825−POST requests, `%peer` becomes a long-poll stream. Our `urb.js`
1826−framework is a client-side wrapper for these requests.
1827−
1828−The abstraction does not leak. On the server side, all it takes
1829−to support web clients is ensuring that outbound marks print to
1830−`%json` and inbound marks parse from `%json`. (The standard
1831−library includes JSON tools.) The `%gall` application does not
1832−even know it's dealing with a web client, all it sees are
1833−messages and subscriptions, just like it would receive from
1834−another Urbit app.
1835−
1836−The dataflow pattern of `%gall` subscriptions is ideal for React
1837−and similar "one-way data binding" client frameworks.
1838−
1839−Of course, `%gall` apps expect to be talking to an urbit plot, so
1840−web clients need to (a) identify themselves and (b) talk over
1841−HTTPS. `%eyre` contains a single sign-on (SSO) flow that
1842−authenticates an urbit user either to her own server or her
1843−friends'.
1844−
1845−Ideally, an urbit named `~plot` is DNSed to `plot.urbit.org`. If
1846−you use your urbit through the web, you'll have an insecure
1847−cookie on `*.urbit.org`. Other urbits read this and drive the SSO
1848−flow; if you're `~tasfyn-partyv`, you can log in as yourself to
1849−`~talsur-todres.urbit.org`. The SSO confirmation message is sent
1850−directly as an `%ames` message between the `%eyre` vanes on the
1851−respective urbits.
1852−
1853−Urbit also has an nginx configuration and node cache server,
1854−which (a) let a relatively slow Urbit server drive reasonably
1855−high request bandwidth, and (b) serve HTTPS by proxy.
1856−
1857−Note that while external caching helps `%ford` style functional
1858−publishing, it does not help `%gall` style clients, which use
1859−server resources directly. Urbit is a personal server; it can
1860−handle an HN avalanche on your blog, but it's not designed to
1861−power your new viral startup.
1862−
1863−### `%j` `%jael`: secret storage
1864−
1865−`%j`, currently `%jael` (200 lines), saves secrets in a tree.
1866−Types of secrets that belong in `%jael`: Urbit private keys,
1867−Urbit symmetric keys, web API user keys and/or passwords, web API
1868−consumer (application) keys.
1869−
1870−`%jael` has no fixed schema and is simply a classic tree
1871−registry. Besides a simpler security and type model, the main
1872−difference between secrets and ordinary `%clay` data is that
1873−secrets expire - sometimes automatically, sometimes manually.
1874−When another vane uses a `%jael` secret, it must register to
1875−receive an expiration notice.
1876−
1877−### `%d` `%dill`: console and Unix
1878−
1879−`%d`, currently `%dill` (450 lines) handles the terminal and
1880−miscellaneous Unix interfaces, mainly for initialization and
1881−debugging.
1882−
1883−The console terminal abstraction, implemented directly with
1884−`terminfo` in raw mode, gives Urbit random-access control over
1885−the input line. Keystrokes are not echoed automatically. Output
1886−lines are write-only and appear above the input line.
1887−
1888−`%dill` also starts default applications, measures system memory
1889−consumption, dumps error traces, etc.
1890−
1891−### `%b` `%behn`: timers
1892−
1893−`%b`, currently `%behn` (250 lines), is a timer vane that
1894−provides a simple subscription service to other vanes.
1895−
1896−Base applications
1897−-----------------
1898−
1899−Arvo ships with three major default applications: `:hood` (1600
1900−lines), `:dojo` (700 lines), and `:talk` (1600 lines).
1901−
1902−### `:dojo`: a shell
1903−
1904−`:dojo` is a shell for ad-hoc computation. A dojo command is a
1905−classic filter pipeline, with sources, filters, and sinks.
1906−
1907−Sources can be shell variables, `%clay` files, immediate
1908−expressions, source files in `/gen`, or uploads (Unix files in
1909−`~/urbit/$plot/.urb/get`). There are three kinds of generator:
1910−simple expressions, dialog cores, and web scrapers. Generators
1911−are executed by `%ford` through `mock`, and can read the Urbit
1912−namespace via virtual operator `11`.
1913−
1914−Urbit does not use short-lived applications like Unix commands. A
1915−`%gall` application is a Unix daemon. A generator is not like a
1916−Unix process at all; it cannot send moves. A dialog generator,
1917−waiting for user input, does not talk to the console; it tells
1918−`:dojo` what to say to the console. A web scraper does not talk
1919−to `%eyre`; it tells `:dojo` what resources it needs (GET only).
1920−This is POLA (principle of least authority); it makes the command
1921−line less powerful and thus less scary.
1922−
1923−`:dojo` filters are immediate expressions or `/tip` source files.
1924−Sinks are console output, shell variables, `%clay`, or downloads
1925−(Unix files in `~/urbit/$plot/.urb/put`). (The upload/download
1926−mechanism is a console feature, not related to `%clay` sync -
1927−think of browser uploading and downloading.)
1928−
1929−### `:talk`: a communication bus
1930−
1931−`:talk` is a distributed application for sharing *telegrams*, or
1932−typed social messages. Currently we use `:talk` as a simple chat
1933−service, but telegram distribution is independent of type.
1934−
1935−Every urbit running `:talk` can create any number of "stations."
1936−Every telegram has a unique id and a target audience, to which
1937−its sender uploads it. But `:talk` stations also subscribe to
1938−each other to construct feeds.
1939−
1940−Mutually subscribing stations will mirror, though not preserving
1941−message order. A `:talk` station is not a decentralized entity;
1942−but urbits can use subscription patterns to federate into
1943−"global" stations - very much as in NNTP (Usenet).
1944−
1945−Stations have a flexible admission control model, with a blanket
1946−policy mode and an exception list, that lets them serve as
1947−"mailboxes" (public write), "journals" (public read), "channels"
1948−(public read/write), or "chambers" (private read/write).
1949−
1950−Subscribers are also notified of presence and configuration
1951−changes, typing indicator, etc. Also, telegrams contain a
1952−"flavor", which senders can use to indicate categories of content
1953−that other readers may prefer not to see.
1954−
1955−`:talk` is usable both a command-line application and a reactive
1956−web client.
1957−
1958−### `:hood`: a system daemon
1959−
1960−`:hood` is a classic userspace system daemon, like Unix `init`.
1961−It's a system component, but it's in userspace because it can be.
1962−
1963−`:hood` is actually a compound application made of three
1964−libraries, `/helm`, `/drum`, and `/kiln`. `/helm` manages the
1965−PKI; `/drum` multiplexes the console; `/kiln` controls `%clay`
1966−sync.
1967−
1968−`/drum` routes console activity over `%gall` messages, and can
1969−connect to both local and foreign command-line interfaces - ssh,
1970−essentially.
1971−
1972−One pattern in Urbit console input is that an application doesn't
1973−just parse its input after a line is is entered, but on each
1974−character. This way, we can reject or correct syntax errors as
1975−they happen. It's a much more pleasant user experience.
1976−
1977−But since both sides of a conversation (the user and the
1978−application) are making changes to a single shared state (the
1979−input line), we have a reconciliation problem. The Urbit console
1980−protocol uses operational transformation (like Google Wave or
1981−`git replot`) for eventual consistency.
1982−
1983−`/helm` manages public keys and (in future) hosted urbits. See
1984−the network architecture section below.
1985−
1986−`/kiln` implements mirroring and synchronization. Any desk can
1987−mirror any other desk (given permission). Mirrors can form
1988−cycles -- a two-way mirror is synchronization.
1989−
1990−Implementation issues
1991−---------------------
1992−
1993−We've left a couple of knotty implementation issues unresolved up
1994−until now. Let's resolve them.
1995−
1996−### Jets
1997−
1998−How can an interpreter whose only arithmetic operator is
1999−increment compute efficiently? For instance, the only way to
2000−decrement `n` is to count up to `n - 1`, which is O(n).
2001−
2002−Obviously, the solution is: a sufficiently smart optimizer.
2003−
2004−A sufficiently smart optimizer doesn't need to optimize every
2005−Nock formula that could calculate a decrement function. It only
2006−needs to optimize one: the one we actually run.
2007−
2008−The only one we run is the one compiled from the decrement
2009−function `dec` in the Hoon standard library. So there's no sense
2010−in which our sufficiently smart optimizer needs to *analyze* Nock
2011−formulas to see if they're decrement formulas. It only needs to
2012−*recognize* the standard `dec`.
2013−
2014−The easiest way to do this is for the standard `dec` to declare,
2015−with a hint (Nock `10`), in some formalized way, that it is a
2016−decrement formula. The interpreter implementation can check this
2017−assertion by simple comparison - it knows what formula the
2018−standard `dec` compiles to. Our sufficiently smart optimizer
2019−isn't very smart at all!
2020−
2021−The C module that implements the efficient decrement is called a
2022−"jet." The jet system should not be confused with an FFI: a jet
2023−has *no* reason to make system calls, and should never be used to
2024−produce side effects. Additionally, a jet is incorrect unless it
2025−accurately duplicates an executable specification (the Hoon
2026−code). Achieving jet correctness is difficult, but we can
2027−spot-check it easily by running both soft and hard versions.
2028−
2029−Jets separate mechanism and policy in Nock execution. Except for
2030−perceived performance, neither programmer nor user has any
2031−control over whether any formula is jet-propelled. A jet can be
2032−seen as a sort of "software device driver," although invisible
2033−integration of exotic hardware (like FPGAs) is another use case.
2034−And jets do not have to be correlated with built-in or low-level
2035−functionality; for instance, Urbit has a markdown parser jet.
2036−
2037−Jets are of course the main fail vector for both computational
2038−correctness and security intrusion. Fortunately, jets don't make
2039−system calls, so sandboxing policy issues are trivial, but the
2040−sandbox transition needs to be very low-latency. Another approach
2041−would be a safe systems language, such as Rust.
2042−
2043−The correctness issue is more interesting, because errors happen.
2044−They are especially likely to happen early in Urbit's history. A
2045−common scenario will be that the host audits an urbit by
2046−re-executing all the events, and produces a different state. In
2047−this case, the urbit must become a "bastard" - logically
2048−instantiated at the current state. The event log is logically
2049−discarded as meaningless. Hosting a bastard urbit is not a huge
2050−problem, but if you have one you want to know.
2051−
2052−In the long run, jet correctness is an excellent target problem
2053−for fuzz testers. A sophisticated implementation might even put
2054−10% of runtime into automatic jet testing. While it's always hard
2055−for implementors to match a specification perfectly, it's much
2056−easier with an executable specification that's only one or two
2057−orders of magnitude slower.
2058−
2059−### Event planning
2060−
2061−How do we actually process Urbit events? If Urbit is a database,
2062−how does our database execute and maintain consistency in
2063−practice, either on a personal computer or a normal cloud server?
2064−How does orthogonal persistence work? Can we use multiple cores?
2065−
2066−An event is a transaction. A transaction either completes or
2067−doesn't, and we can't execute its side effects until we have
2068−committed it. For instance, if an incoming packet causes an
2069−outgoing packet, we can't send the outgoing packet until we've
2070−committed the incoming one to durable storage.
2071−
2072−Unfortunately, saving even a kilobyte of durable storage on a
2073−modern PC, with full write-through sync, can take 50 to 100ms.
2074−Solid-state storage improves this, but the whole machine is just
2075−not designed for low-latency persistence.
2076−
2077−In the cloud the situation is better. We can treat consensus on a
2078−nontrivial Raft cluster in a data center as persistence, even
2079−though the data never leaves RAM. Urbit is highly intolerant of
2080−computation error, for obvious reasons, and should be run in an
2081−EMP shielded data center on ECC memory.
2082−
2083−There are a number of open-source reliable log processors that
2084−work quite well. We use Apache Kafka.
2085−
2086−With either logging approach, the physical architecture of an
2087−Urbit implementation is clear. The urbit is stored in two forms:
2088−an event log, and a checkpoint image (saved periodically, always
2089−between events). The log can be pruned at the last reliably
2090−recorded checkpoint, or not. This design (sometimes called
2091−"prevalence") is widely used and supported by common tools.
2092−
2093−The checkpointing process is much easier because Urbit has no
2094−cycles and needs no tracing garbage collector. Without careful
2095−tuning, a tracing GC tends to turn all memory available to it
2096−into randomly allocated memory soup. The Urbit interpreter uses a
2097−region system with a copy step to deallocate the whole region
2098−used for processing each event.
2099−
2100−Events don't always succeed. How does a functional operating
2101−system deal with an infinite loop? The loop has to be
2102−interrupted, as always. How this happens depends on the event. If
2103−the user causes an infinite loop from a local console event, it's
2104−up to the user to interrupt it with \^C. Network packets have a
2105−timeout, currently a minute.
2106−
2107−When execution fails, we want a stack trace. But Urbit is a
2108−deterministic computer and the stack trace of an interrupt is
2109−inherently nondeterministic. How do we square this circle?
2110−
2111−Urbit is deterministic, but it's a function of its input. If an
2112−event crashes, we don't record the event in `I`. Instead, we
2113−record a `%crud` card that contains the stack trace and the
2114−failed event. To Urbit, this is simply another external event.
2115−
2116−Processing of `%crud` depends on the event that caused it. For
2117−keyboard input, we print the error to the screen. For a packet,
2118−we send a negative ack on the packet, with the trace. For
2119−instance, if you're at the console of urbit `A` and logged in
2120−over the network to `B`, and you run an infinite loop, the `B`
2121−event loop will time out; the network console message that `A`
2122−sent to `B` will return a negative ack; the console application
2123−on `A` will learn that its message failed, and print the trace.
2124−
2125−Finally, the possibilities of aggressive optimization in event
2126−execution haven't been explored. Formally, Urbit is a serial
2127−computer - but it's probable that a sophisticated, mature
2128−implementation would find a lot of ways to cheat. As always, a
2129−logically simple system is the easiest system to hack for speed.
2130−
2131−Network and PKI architecture
2132−----------------------------
2133−
2134−Two more questions we've left unanswered: how you get your urbit
2135−plot, and how packets get from one plot to another.
2136−
2137−Again, a plot is both a digital identity and a routing address.
2138−Imagine IPv4 if you owned your own address, converted it to a
2139−string that sounded like a foreign name, and used it as your
2140−Internet handle.
2141−
2142−Bases are parsed and printed with the `%p` aura, which is
2143−designed to make them as human-memorable as possible. `%p` uses
2144−phonemic plot-256 and renders smaller plots as shorter strings:
2145−
2146− 8 bits galaxy ~syd
2147− 16 bits star ~delsym
2148− 32 bits planet ~mighex-forfem
2149− 64 bits moon ~dabnev-nisseb-nomlec-sormug
2150− 128 bits comet ~satnet-rinsyr-silsec-navhut--bacnec-todmeb-sarseb-pagmul
2151−
2152−Of course, not everyone who thinks he's Napoleon is. For the
2153−urbit to actually send and receive messages under the plot it
2154−assigns itself in the `%init` event (`I5`), later events must
2155−convey the secrets that authenticate it. In most cases this means
2156−a public key, though some urbits are booted with a symmetric
2157−session key that only works with the parent plot.
2158−
2159−How do you get a plot? Comets are hashes of a random public key.
2160−"Civil" non-comets are issued by their parent plot - the
2161−numerical prefix in the next rank up: moons by planet, planets by
2162−star, stars by galaxy. The fingerprints of the initial galactic
2163−public keys (currently test keys) are hardcoded in `%ames`.
2164−
2165−The comet network is fully decentralized and "free as in beer,"
2166−so unlikely to be useful. Any network with disposable identities
2167−can only be an antisocial network, because disposable identities
2168−cannot be assigned default positive reputation. Perhaps comets
2169−with nontrivial hashcash in their plots could be an exception,
2170−but nonmemorable plots remain a serious handicap.
2171−
2172−The civil network is "semi-decentralized" - a centralized system
2173−designed to evolve into a decentralized one. Urbit does not use a
2174−blockchain - address space is digital land, not digital money.
2175−
2176−The initial public key of a civil plot is signed by its parent.
2177−But galaxies, stars and planets are independent once they've been
2178−created - they sign their own updates. (Moons are dependent;
2179−their planet signs updates.)
2180−
2181−The certificate or `will` is a signing chain; only the last key
2182−is valid; longer wills displace their prefixes. Thus update is
2183−revocation, and revocation is a pinning race. To securely update
2184−a key is to ensure that the new will reaches any peer before any
2185−messages signed by an attacker who has stolen the old key.
2186−
2187−The engineering details of this race depend on the actual threat
2188−model, which is hard to anticipate. If two competing parties have
2189−the same secret, no algorithm can tell who is in the right. Who
2190−pins first should win, and there will always be a time window
2191−during which the loser can cause problems. Aggressively flooding
2192−and expiring certificates reduces this window; caching and lazy
2193−distribution expands it. There is no tradeoff-free solution.
2194−
2195−Broadly, the design difference between Urbit and a blockchain
2196−network is that blockchains are "trust superconductors" - they
2197−eliminate any dependency on social, political or economic trust.
2198−Urbit is a "trust conductor" - engineered to minimize, but not
2199−eliminate, dependencies on trust.
2200−
2201−For instance, bitcoin prevents double-spending with global mining
2202−costs in multiple dollars per transaction (as of 2015). Trusted
2203−transaction intermediation is an easily implemented service whose
2204−cost is a tiny fraction of this. And the transaction velocity of
2205−money is high; transactions in land are far more rare.
2206−
2207−Another trust engineering problem in Urbit is the relationship
2208−between a plot and its parent hierarchy. The hierarchy provides a
2209−variety of services, starting with peer-to-peer routing. But
2210−children need a way to escape from bad parents.
2211−
2212−Urbit's escape principle: (a) any planet can depend on either its
2213−star's services, or its galaxy's; (b) any star can migrate to any
2214−galaxy; (c) stars and galaxies should be independently and
2215−transparently owned.
2216−
2217−Intuitively, an ordinary user is a planet, whose governor is its
2218−star, and whose appeals court is its galaxy. Since a star or
2219−galaxy should have significant reputation capital, it has an
2220−economic incentive to follow the rules. The escape system is a
2221−backup. And the comet network is a backup to the backup.
2222−
2223−From a privacy perspective, a planet is a personal server; a star
2224−or galaxy is a public service. Planet ownership should be private
2225−and secret; star or galaxy ownership should be public and
2226−transparent. Since, for the foreseeable future, individual
2227−planets have negligible economic value, Urbit is not a practical
2228−money-laundering tool. This is a feature, not a bug.
2229−
2230−Finally, a general principle of both repositories and republics
2231−is that the easier it is (technically) to fork the system, the
2232−harder it is (politically) to fork. Anyone could copy Urbit and
2233−replace the galactic fingerprint block. Anyone can also fork the
2234−DNS. If the DNS was mismanaged badly enough, someone could and
2235−would; since it's competently managed, everyone can't and won't.
2236−Forkability is an automatic governance corrector.
2237−
2238−From personal server to digital republic
2239−----------------------------------------
2240−
2241−Republics? Any global system needs a political design. Urbit is
2242−designed as a *digital republic*.
2243−
2244−The word *republic* is from Latin, "res publica" or "public
2245−thing." The essence of republican goverment is its
2246−*constitutional* quality - government by law, not people.
2247−
2248−A decentralized network defined by a deterministic interpreter
2249−comes as close to a digital constitution as we can imagine. Urbit
2250−is not the only member of this set - bitcoin is another; ethereum
2251−is even a general-purpose computer.
2252−
2253−The "law" of bitcoin and ethereum is self-enforcing; the "law" of
2254−Urbit is not. Urbit is not a blockchain, and no urbit can assume
2255−that any other urbit is computing the Nock function correctly.
2256−
2257−But at least the distinction between correct and incorrect
2258−computing is defined. In this sense, Nock is Urbit's
2259−constitution. It's not self-enforcing like Ethereum, but it's
2260−exponentially more efficient.
2261−
2262−Non-self-verifying rules are useful, too. Defining correctness is
2263−not enforcing correctness. But Urbit doesn't seek to eliminate
2264−its dependency on conventional trust. Correctness precisely
2265−defined is easily enforced with social tools: either the
2266−computation is tampered with or it isn't.
2267−
2268−Conclusion
2269−==========
2270−
2271−On the bottom, Urbit is an equation. In the middle it's an
2272−operating system. On the top it's a civilization -- or at least,
2273−a design for one.
2274−
2275−When we step back and look at that civilization, what we see
2276−isn't that surprising. It's the present that the past expected.
2277−The Internet is what the Arpanet of 1985 became; Urbit is what
2278−the Arpanet of 1985 wanted to become.
2279−
2280−In 1985 it seemed completely natural and inevitable that, by
2281−2015, everyone in the world would have a network computer. Our
2282−files, our programs, our communication would all go through it.
2283−
2284−When we got a video call, our computer would pick up. When we had
2285−to pay a bill, our computer would pay it. When we wanted to
2286−listen to a song, we'd play a music file on our computer. When we
2287−wanted to share a spreadsheet, our computer would talk to someone
2288−else's computer. What could be more obvious? How else would it
2289−work?
2290−
2291−(We didn't anticipate that this computer would live in a data
2292−center, not on our desk. But we didn't appreciate how easily a
2293−fast network can separate the server from its UI.)
2294−
2295−2015 has better chips, better wires, better screens. We know what
2296−the personal cloud appliance does with this infrastructure. We
2297−can imagine our 1985 future ported to it. But the most
2298−interesting thing about our planets: we don't know what the world
2299−will do with them.
2300−
2301−There's a qualitative difference between a personal appliance and
2302−a personal server; it's the difference between a social "network"
2303−and a social network. A true planet needs to work very hard to
2304−make social programming easier. Still, distributed social
2305−applications and centralized social applications are just
2306−different. A car is not a horseless carriage.
2307−
2308−We know one thing about the whole network: by default, a social
2309−"network" is a monarchy. It has one corporate dictator, its
2310−developer. By default, a true network is a republic; its users
2311−govern it. And more important: a distributed community cannot
2312−coerce its users. Perhaps there are cases where monarchy is more
2313−efficient and effective -- but freedom is a product people want.
2314−
2315−But no product is inevitable. Will we even get there? Will Urbit,
2316−or any planet, or any personal cloud server, actually happen?
2317−
2318−It depends. The future doesn't just happen. It happens,
2319−sometimes. When people work together to make it happen.
2320−Otherwise, it doesn't.
2321−
2322−The Internet didn't scale into an open, high-trust network of
2323−personal servers. It scaled into a low-trust network that we use
2324−as a better modem -- to talk to walled-garden servers that are
2325−better AOLs. We wish it wasn't this way. It is this way.
2326−
2327−If we want the network of the future, even the future of 1985,
2328−someone has to build it. Once someone's built it, someone else
2329−has to move there. Otherwise, it won't happen.
2330−
2331−And for those early explorers, the facilities will be rustic. Not
2332−at all what you're used to. More 1985 than 2015, even. These
2333−"better AOLs", the modern online services that are our personal
2334−appliances, are pretty plush in the feature department.
2335−
2336−Could they just win? Easily. Quite easily. It's a little painful
2337−herding multiple appliances, but the problem is easily solved by
2338−a few more corporate mergers. We could all just have one big
2339−appliance. If nothing changes, we will.
2340−
2341−To build a new world, we need the right equation and the right
2342−pioneers. Is Urbit the right equation? We think so. Check our
2343−work, please. Are you the right pioneer? History isn't over --
2344−it's barely gotten started.
2345−
2346−</div>
2347−