@@ -0,0 +1,128 @@
1+---
2+date: ~2017.5.10
3+type: post
4+title: Toward a frozen operating system
5+preview: Is it possible to freeze an entire OS?
6+author: Curtis Yarvin + Galen Wolfe-Pauly
7+layout: urbit,post
8+navmode: navbar
9+navdpad: false
10+navselect: blog
11+navpath: /
12+navhome: /
13+navclass: urbit
14+---
15+
16+<br /><br />
17+
18+Is it possible to freeze an entire OS, so that its codebase never changes and never has to change? Is the proposition even meaningful? Is it practical? And if so, why would we want a frozen OS?
19+
20+Think about it intuitively. There's a reason we call an OS a "platform." It's a piece of permanent infrastructure. Paul Graham wrote a [great essay](http://www.paulgraham.com/hundred.html) about the "hundred-year language," but the language is only one part of any complete platform. And in infrastructure terms, a hundred years isn't that long.
21+
22+Why not think of structures that have lasted *thousands* of years? An aqueduct, a pyramid, a Parthenon? Why set our sights lower? Why isn't it time to aim for perfect and timeless code? If not now, when?
23+
24+## Scope of the problem
25+
26+More prosaically, a frozen OS just means your OS never changes. A frozen OS never gets any updates. Not because it can't update itself, just because it doesn't need to.
27+
28+Is this even a meaningful concept? A frozen OS? First we'll have to decide what "OS" even means. Certainly, if "OS" means "the code that your hardware boots into," hardware always changes and so will the OS.
29+
30+A broader, more useful definition: the "OS" is *an opaque model of a persistent general-purpose computer*. This definition allows for arbitrary levels of virtualization. Linux on a droplet is an OS. A browser is an OS, because it's persistent (sorta) and opaque. But `node.js` is not an OS, because it's not opaque: it exports Linux system calls.
31+
32+Within this opaque model, we have to decide what's "system" and "user" software. It clearly makes no sense to freeze userspace. Let's use the [trusted codebase](https://en.wikipedia.org/wiki/Trusted_computing_base): code is system software, if compromising it would compromise the whole system.
33+
34+## Infinite maturity
35+
36+It's important to remember that just because a system is designed to achieve frozenness, doesn't mean it's useless until it’s frozen. It doesn't even mean it's immature. It just means that in the very long term, the system’s maturity goals are infinite.
37+
38+Systems designed to diverge infinitely (with normal X.X.X semantic versioning), even those designed to converge infinitely (as with [Knuth versioning](http://www.tex.ac.uk/FAQ-TeXfuture.html)), also achieve maturity, often great maturity, without being quite frozen.
39+
40+For example, C is a very mature language. C has long since mastered backward compatibility. We can be sure that any C program which compiles on today's correct C compilers will compile on any future correct C compiler.
41+
42+But, since C is not frozen, we can't be sure that today's C compilers will compile any future C program. So C has not yet achieved perfect *forward* compatibility. By definition, a frozen system is compatible both forward and backward. So C is mature, but it could still get more mature.
43+
44+The point of a frozen OS is to shoot for *infinite* maturity. Obviously a frozen platform is compatible forever in both directions.
45+
46+Infinite maturity does exist in the real world. But only in a few simple file formats and protocols — IP, JSON, XML. And not all this maturity is the result of perfection. Some is just the accident of a missing update mechanism.
47+
48+But IP, JSON, XML... this is how stable, how taken for granted, system software should be. And can be. Certainly, any JSON parser today will parse any JSON document made in the future.
49+
50+The dream of absolute zero is an OS whose core components are so simple, so boring, and so mechanically straightforward that we can imagine them becoming finished and never changing. We need to freeze not just a format or a protocol, but a language, a kernel, a standard library. To succeed, we need to flush out each quirk, complexity, and creeping feature, until the system is so boring and obvious that no one even wants to change it.
51+
52+## Kelvin versioning
53+
54+"Kelvin versioning" is one way to track a project designed to be frozen.
55+
56+In Kelvin versioning, a version is an integer in degrees Kelvin. Absolute zero is frozen — no further updates are possible. If your Kelvin versions don't track your actual progress, you run out of integers. This danger is a concrete incentive for the project to track its own progress accurately.
57+
58+(Kelvin versioning isn't useful only at lifecycle scale. It can be used for any development process with an irreversible end — as build numbers for a release, for example.)
59+
60+## The palm-tree dependency model
61+
62+A whole OS can't have a single Kelvin number. It has to consist of multiple components which depend on each other. What's the right dependency model for the parts of a frozen OS?
63+
64+One way to think about dependencies across a whole OS is the "palm-tree model." Fundamentally, there are two kinds of components in a system: "exclusive" components (where we have to choose one version of the code), and "inclusive" ones (where different users can see different versions).
65+
66+The exclusive components are the trunk of the palm tree: a stack of layers, each of which has one active copy. For instance, you are only running one kernel at a time. Different apps cannot use different versions of the kernel.
67+
68+The inclusive components are the fronds, branching out in any direction they want. Not only can two apps use different versions of the same library, in many situations *one* app needs to be able to use two different versions of the same library.
69+
70+## The palm trunk: telescoping Kelvins
71+
72+Any subsystem that serves as any kind of platform is exclusive, since anything that runs on top of the platform has to choose one version of the platform to run on. Since layers are good, this creates a natural stack which can get quite tall — and include components which are technically in userspace.
73+
74+The right way for this trunk to approach absolute zero is to "telescope" its Kelvin versions. The rules of telescoping are simple:
75+
76+1. If tool B sits on platform A, either both A and B must be at absolute zero, or B must be warmer than A.
77+
78+2. Whenever the temperature of A (the platform) declines, the temperature of B (the tool) must also decline.
79+
80+3. B must state the version of A it was developed against. A, when loading B, must state its own current version, and the warmest version of itself with which it's backward-compatible.
81+
82+Of course, if B itself is a platform on which some higher-level tool C depends, it must follow the same constraints recursively.
83+
84+The important effect of obeying rule 2: in a tall trunk, it's sufficient to state your dependency on the platform directly below you. Suppose C sits on B, which sits on A and exports features of A. C can state only the version of B it's written against, because when A chills, B must also chill.
85+
86+If A could chill without B chilling, we could modify C to use new features of A. Then our new C would silently fail to work on an A-B stack which had not received the A update.
87+
88+## The palm fronds: total flexibility
89+
90+The goal of the trunk is to get to the fronds. In the fronds, no one should ever worry about dependency conflicts. And nothing is ever frozen, or has to be frozen.
91+
92+The right way to solve dependencies in userspace is: within an organization, use a monorepo and link relative to its head. Between organizations, use pinned version numbers that are as immutable as possible. Avoid programming environments that create exclusion problems in userspace — for instance, that compel a single process to choose one version of a library when a diamond dependency exists.
93+
94+There is almost never any good reason to automatically update dependencies with fresh versions of foreign code. In the case of a security update, the maintainer of an app which depends on a patched library needs to catch the patch and propagate the update in a new version of the app. An app shouldn’t be security-critical to begin with, and propagating updates is a basic function of a modern OS. Also, if the app is security-critical, it could have its own bugs, so it needs a maintainer and an update pathway anyway.
95+
96+## Urbit: status and prospects
97+
98+Yes, we do believe it's possible to write a freezable OS. In fact, we wrote one. And it works. It's not yet frozen (except at the bottom layer), but it's getting there.
99+
100+Urbit, a clean-slate functional OS, has been under development since 2002. It took a decade (as a one-man project) just to build a prototype of a freezable OS. The result in 2017 (now with a small company) is a telescoping Kelvin stack of layers from VM to UI.
101+
102+At the bottom (the VM Nock), this stack is slightly warmer than liquid helium (5K). It's easy for Nock to get this cold, because it's defined in a page of axioms that gzips to 340 bytes. We are in no hurry to declare zero, but any change at this point would be both very unlikely and very difficult to implement.
103+
104+Urbit's typed functional language, Hoon, which compiles itself to Nock, is closing in on liquid krypton at 143K. Hoon is defined in about 5,000 lines of Hoon, or 10,000 if you count the deep standard library. Nock will never need to change; Hoon is perfectly serviceable but has some well-known rough spots.
105+
106+Above Hoon, the stack rapidly gets much warmer. Arvo is an event-driven microkernel that's 1,000 lines of Hoon. Above this are another 5,000 lines of system library, then 12,000 lines of Arvo modules: a peer-to-peer network, a PKI and private ledger, a revision-control system, an HTTP gateway, a functional build system, and an application sandbox. A trivial working system needs another 5,000 lines for basic chat and shell apps.
107+
108+We haven't had meaningful Kelvin numbers for the Arvo layer, but this release will change that. About a third of Arvo has been or will be rewritten in the last year; we'll assign Kelvins once this code (which now compiles) is tested into existence, largely based on how close the last compiling draft was to the first working draft. But you'll certainly be able to cook eggs on it.
109+
110+(Once again, not being frozen doesn't mean not working. Until all these layers are finished, we'll be updating them over the air. To be exact, Urbit distributes its own updates through subscriptions in its own revision-control system.)
111+
112+The actual status of Urbit is that in 2017, we hope to make our last discontinuous update — meaning that instead of your Urbit node updating itself silently, you destroy and recreate it. Urbit works today, but it isn't really alive until today's Urbit data lives forever.
113+
114+Even committing to continuity is a significant step toward the frozen OS, because we're committing to never making another change that the present platform can't install. Fortunately, the only layer that has to stay frozen for this is Nock.
115+
116+## The style of convergent system software
117+
118+We're obviously in no position to declare victory at any layer above Nock, but aiming at absolute zero creates a system design style we think works very well — even if we never get to zero.
119+
120+Deep, onion-like layering is essential. A thin layer has no room to grow. A good example of this principle is the difference between Urbit and Lisp machines. Both Nock and Lisp are very simple axiomatic definitions of computing. But practical Lisp systems expand by extending the model, whereas Urbit layers over a frozen axiom system.
121+
122+Software that expands by extension is inevitable. But we need to be able to kick it up to userspace. System software should be concerned with the fundamental problems of computing, which don't change. The few cases in which they do change — like crypto algorithms — are special cases, easily encapsulated. And even these changes can be expected to slow down in the long term.
123+
124+Perhaps the best example of the convergent style in existing, mature system software is the relational database. The genius of the relational model is that it's designed to never grow hair. Even SQL, with its COBOL-era syntax, is essentially invulnerable to any significant non-cosmetic improvement.
125+
126+The lesson: when we see a system that seems to want to expand indefinitely, we can and should look for a layer division within this system, separating an inner layer that tends to converge from an outer layer that naturally diverges. Layering is what distinguishes true system software from the giant balls of mud that all deep codebases want to become.
127+
128+Are all balls of mud, or at least all the muddy parts of an OS, susceptible to this transformation? We think so. But we'll see.