@@ -0,0 +1,167 @@
1+---
2+date: ~2017.1.30
3+type: post
4+title: Urbit in 2017
5+preview: In 2017, we're ready to start making the damn thing useful.
6+author: Curtis Yarvin + Galen Wolfe-Pauly
7+layout: urbit,post
8+navmode: navbar
9+navdpad: false
10+navselect: blog
11+navpath: /
12+navhome: /
13+navclass: urbit
14+---
15+<br /><br />
16+
17+ > TLDR: A nontrivially useful Urbit demands nontrivial work in two areas: systems maturity and user experience. We've actually done quite a bit of this work already, so we're pretty excited.
18+
19+In 2013 Urbit was intriguing. By 2015 it had become interesting. In 2016 it was actually exciting (if not always in a good way!). In 2017, we're ready to start making the damn thing useful. Or, at the very least, fun.
20+
21+Our `cc-release` stability upgrade is code-complete and in testing. This infrastructure upgrade makes it possible for us to finally think seriously about the user experience of Urbit. Our all-new UX is both a social network and aggregator, powered by the world's cutest command line. It isn't built yet, but we're finally confident that it can be built and remain stable.
22+
23+System software is boring (at least when done right). So we’ll start with the UI. (Or UX?) Urbit is certainly an *experience*, and can't help but be *designed*.
24+
25+Galen will cover the design, and Curtis will show up later to talk about infrastructure.
26+
27+## Design
28+
29+Galen is the person around here who thinks about design, ‘product’ and things like that. Galen went to architecture school. He talks like it. We’ll be getting more specific about what this product will look like in future posts. For now, we recorded Galen secretly and came away with these notes:
30+
31+### Notes on designing a system
32+
33+- *UX*? As a kid it was just *design* or maybe *web design*. At some point I remember people talking about *user interface* or *UI*. But I truly don't understand where the *X* came from. It's like Monster Energy got involved in the branding.
34+
35+- An *experience* sounds like the kind of thing that's going to be littered with ads, or is going to manipulate me in some way. Something I passively consume. Not something I actively control. So I prefer *interface* to *experience*, and I think you should too.
36+
37+#### Computing is simple
38+
39+- Urbit has to be just as usable as any widely used app in 2017. For a personal server, the usability bar can't be Unix. A Unix server is an industrial tool. An industrial tool isn’t a consumer product. You can attach a blade to the end of a drill and use it as a blender, but it’s not quite as nice as the Vitamix.
40+
41+- What is an app, anyway? It's *shared computing*. Everyone's data is one data structure, in one program, on one server, owned by one corporation. With *one* UI for everyone to do *one* thing.
42+
43+- There isn't one big corporation that writes *all* the code, so everyone's lives are spread across a bunch of different apps. In turn, you have to track all these silly little accounts, suffer through separate ‘experience’-es and figure out how to move your data around.
44+
45+- And there’s no alternative. The cloud computing services we depend on aren’t accessible through some other means.
46+
47+- The phrase ‘general-purpose computing’ sounds scary, but it shouldn’t be. ‘Computing’ doesn’t necessarily mean writing code. As far as I’m concerned we’re computing while we’re browsing Facebook. Or talking to Alexa.
48+
49+- We use computers to talk to people, store data, send money, share and collaborate on documents, and so on. Not what you would have used your Apple II for, but computing nonetheless.
50+
51+- So what’s a general-purpose computer today? The closest thing e can point to is WeChat. It’s like a bad copy of a mature Urbit: it’s a general-purpose personal server that’s really simple to use.
52+
53+- Everyone has probably heard of WeChat, but if you don’t know much about it you should really watch [this NYT video](https://www.nytimes.com/video/technology/100000004574648/china-internet-wechat.html).
54+
55+- From a distance WeChat is great. Everyone in China loves the one-app, one-identity, all-my-own-stuff user experience. It’s clearly better than being forced to switch apps constantly.
56+
57+- But WeChat is just another giant shared-computing service. It's made out of the usual 2010s server stack mess. The visual and interaction design looks pretty dated. And the infrastructure is just the normal 2010s server stack mess. We can learn a lot from the idea of WeChat, but not from the implementation.
58+
59+#### Start with the whole system
60+
61+- Shared computing may have its problems, but at least it has an ethos. Almost anyone can copy and paste their way to an app on the app store.
62+
63+- Urbit isn’t an app though: it’s a personal server. What's the standard UI for a personal server? It doesn’t exist.
64+
65+- Like a web app, your urbit runs in a data center. You log into it with your browser. Other than that, where do you even start? A `bash` prompt isn’t what we’re looking for.
66+
67+- But wait, is the idea of a command line bad UI? Or is the Unix command line just bad UI? We now know ordinary people can use command lines: they use chatbots.
68+
69+- The apps that we use all do quite a lot to abstract away the complexity of the command line. Instead of a CLI we have something like a 'profile'.
70+
71+- Your Urbit is a real computer. It would be a waste to abstract this reality away. But, if we can't succeed in making that reality simple enough for an ordinary person to manage — we have failed. iOS is almost a real computer. WeChat is almost a real computer.
72+
73+- We have to get all the way there: design a complete system that an ordinary person can control on their own.
74+
75+- Perhaps it's an antiquated view, but we do still believe that computers have the potential to be powerful creative tools.
76+
77+- The most important bottleneck in achieving this goal is winning the trust of the user. The user can't get lost or break the system, nor can they be prevented from exploring its furthest reaches.
78+
79+#### Reinventing the command line
80+
81+- As Neal Stephenson [tells us](http://cristal.inria.fr/~weis/info/commandline.html), the command line is the natural UI for any computer. It's especially the natural UI for a *network* computer, like a personal server. A command line scales way better than a GUI to a large selection of actions. And a network computer can do almost infinitely many things.
82+
83+- We tried to put a simplified "for dummies" GUI on top of our real OS, and it was awful. It's like a bad Linux desktop GUI. You feel instantly that you've got a choice between something limited and lame, and something mysterious and scary. So we threw those designs out, in search of something better.
84+
85+- The true command line is generally thought of as something mysterious and scary. Somewhere that you can do things to irreparably damage your computer. It's powerful, but also dangerous.
86+
87+- This tradeoff — between control and safety — seems wrong. Perhaps the command line is actually just another artifact of 70s computing. The core idea is good, but the design needs to be updated dramatically.
88+
89+- `bash` is a bad UX because there's no easy way to control its power. It's also quite difficult to learn how to use. At the other end of the spectrum is the Chatbot. Chatbots are weak because they don't scale well to any problem that isn't trivial.
90+
91+- So what does a great command line look like? It's syntactic, not free-form. It can't be a constant test of your ability to memorize the phonebook. It's also not tethered to a piece of industrial equipment.
92+
93+#### We're already headed in this direction
94+
95+- Urbit already has the beginnings of a great command line. We parse your input with an actual grammar and correct errors (or erase them, at least) as you type.
96+
97+- What's missing is help. A great command line needs to reach out and hug you in every way it can. It needs to assume you don't know anything at all about the system. It's there for you to solve your problems, but also to help you explore.
98+
99+- There's actually one exotic Unix shell that does it right: [`fish`](https://en.wikipedia.org/wiki/Friendly_interactive_shell). `fish` jumps through some pretty amazing hoops, like parsing manpages, to give you context and completion.
100+
101+- It's a lot easier for Urbit because we're designing the system from scratch. From a technical standpoint, all we have to do is add contextual help to our existing parser framework. An extended rule can parse a string, or tell the user what it wants to parse. In simple terms: we have to make the `:dojo` (our shell) much friendlier.
102+
103+- The `:dojo` is quite powerful, and exposes a lot of surface area for people to build small modules and programs. And, since we have a global revision-controlled filesystem, it’s pretty easy to share that code. The problem is, it’s just way too difficult to figure out how this system works.
104+
105+- It’s important to note that we’re not talking about ‘developers’ trying to tackle well known engineering problems. Urbit is meant to make building things on the web *fun* again. Welding a giant single-purpose app on top would me a mistake primarily because it’s inevitably kind of narrow and boring.
106+
107+- The `:dojo` shell is built around the concept of a *generator*, a program that builds a noun. The idea is that how you build the noun is orthogonal to what you do with it. You could print it, send it as a command, etc. There are several kinds of generators -- simple scripts, prompting dialogs, HTTP scrapers, etc.
108+
109+- A generator is initialized with command-line arguments and options, in the classic Unix style (except typed). In the classic Unix style, Urbit at present provides no help at all in constructing a command line. You just have to know.
110+
111+- The new `:dojo` will let the user build a command line with a helpful prompting dialog. Call it a meta-generator, a generator for a generator command. If you know the command, you can just type it. If not, you step through a simple form to build the command. The `:dojo` shows you all your options, explains them and helps you complete them.
112+
113+- Building a command is just a case of data entry. In a command-line world, data entry is always serial. You answer questions serially, one at a time.
114+
115+- A great Urbit console also has to be accessible both from a browser and a Unix terminal. That means a prompt needs to tell the console if it could be a radio button, even though a terminal can't have a radio button. We'll improve the interactivity of our terminal a little, but we're not rewriting Lynx.
116+
117+- So, what do you do with this fancy command line? Command everything -- from your urbit to the world. Installing an app is a command. Publishing a tweet is a command. Paying someone in bitcoin is a command.
118+
119+- Your urbit can show you all the things you can do, and help you do them. If there's something you'd like it to do, build it!
120+
121+- A great command line alone is not all that Urbit needs. But it feels like the right central paradigm for interacting with your Urbit. We'll get into the specifics in a future post.
122+
123+## Stability, maturity, performance and documentation
124+
125+Okay, here's Curtis:
126+
127+We're actually doing pretty well on this. Despite there being all kinds of unacceptably broken things about Urbit 2016, the network has actually stayed up since August, and we haven't had an unintentional continuity breach since June.
128+
129+Of course, it helps that we don't ask anyone to *use* Urbit in earnest. And ships still do sink. Since an urbit is an identity, sinking your planet is a horrendous experience. You'll never bond with your second planet the way you did with your first -- never mind any *data* you may have lost.
130+
131+Our upcoming `cc-release` is the last planned breaching release. With certain exceptions discussed below, all major parts are now (January 2017) code complete or better.
132+
133+This includes the rewritten network stack (`%ames`), code complete but untested; the new secret vault and promise tracker (`%jael`), mostly tested; and the new `u3_pier` event execution framework, with worker processes and real two-phase commit, smoke-tested. We've also completely restructured `hoon.hoon` and `zuse.hoon`, which now are at least cosmetically acceptable.
134+
135+This is not to say that the `cc-release` Urbit won't in many ways embarrass us as engineers. It will. But your ships won't randomly sink, your secrets won't randomly leak, and your datas won't randomly rot. Or if they do, it's an implementation bug, not a design flaw.
136+
137+There are still a few cracks we need to fill in. The main one: some top-level adjustments to the Arvo event system. The boot sequence now executes correctly, but the actual boot events need to be totally refactored, as does the rather critical Unix-Arvo interface. The long-promised security mask and voltage flags need to be added, and vanes adjusted to use them. Causal tracking of execution crashes needs to actually work. At the user level, `:talk` needs to be split into a user agent and a daemon, and the user-level `%kiln`, the "systemd of Urbit," needs a rethink to make upgrade logic state-triggered, not edge-triggered.
138+
139+The console and the dojo need some work to make our user experience real. One way to view a high-usability command line is that a command, if you don't have the esoteric knowledge to fill it in by just typing the syntax, is essentially a form. This form needs to work both in the browser and the terminal, which isn't trivial. Browser and terminal both need a simple multiscreen navigation model, which we don't have yet.
140+
141+There's also a bunch of Unix-level work to do, mainly in converting the old "single-player mode" console-bound Urbit into a real system service. Urbit works better as a daemon which runs one or more worker processes, and talks to client consoles over a Unix socket. A real architecture actually makes it practical for star owners to host the planets they issue, for example.
142+
143+Finally, because we never forget the wisdom of the sages, our last act before asking people to use Urbit will be the long awaited performance [war]. The new event system now at least releases outgoing events before the input queue is clear! You never know what optimization will achieve before you make a serious effort, but I expect great things from the [war].
144+
145+We haven't touched the documentation since the first half of 2016. As always, maturity has to come before documentation. Premature documentation is almost as dangerous as premature optimization.
146+
147+## Notable contributions
148+
149+Some notable contributions of late:
150+
151+- `~ponnys-podfer` wrote a MUD
152+- `~palfun-foslup` created a surprisingly capable talkbot
153+- `~master-morzod` reworked the console in both the Hoon and C layers, adding color and cleaning up a bunch of stuff.
154+
155+Anyone who contributes at this point deserves extreme praise for the ability to work in an immature, often broken system. We look forward to being ready for normal programmers who aren't intrepid jungle explorers.
156+
157+## Next crowdsale
158+
159+We'll do another crowdsale in the spring. We understand that the last sale left a lot of people unable to participate — and our first priority is making sure those who are excited about Urbit can own some real estate. We’ll announce our plans on how this might happen soon. If you’re interested, make sure you’re on our [mailing list]().
160+
161+## Future updates
162+
163+We’re going to change up the way we write these updates a bit. Obviously, we slipped a few months in getting this one out. Longer updates can be great, but we need to spend as much time as possible building stuff and writing code.
164+
165+Going forward we’re going to switch to commit-message style updates on a monthly basis, with longer updates arriving sporadically. Maker DAO does a great job of these shorter updates ([here](https://blog.makerdao.com/2016/12/14/maker-update-925/) is an example).
166+
167+If you're ever curious to check in or hear what's going on, don't hesitate to get in touch. You can always chat with us on [urbit.org/stream](https://urbit.org/stream), or by booting your own Urbit. [Twitter](https://twitter.com/urbit_) or email ([urbit@urbit.org](mailto:urbit@urbit.org)) are also perfectly fine.