https://urbit.org Urbit is a personal server built from scratch Thu, 18 Nov 2021 19:21:20 GMT https://validator.w3.org/feed/docs/rss2.html https://github.com/jpmonette/feed en https://urbit.org/blog/haleek-maul-interview https://urbit.org/blog/haleek-maul-interview Thu, 09 Sep 2021 00:00:00 GMT https://urbit.org/blog/state-of-urbit https://urbit.org/blog/state-of-urbit Tue, 24 Aug 2021 00:00:00 GMT ) about using Landscape from day-to-day around that time. Landscape wasn't just a useful tool for us, but it was something that other communities were starting to use in earnest. It seemed like Landscape could be a great place for people to grow new private communities, so long as we could onboard new users effectively. Hosting is a huge piece of the puzzle, since we need non-technical people to be able to use this wonderful platform as easily as its mainstream competitors. At this point (early this year), we encountered some pretty serious hurdles. To use Urbit, you need an Urbit ID. Each one of these is an NFT (something that no one cared about when we first launched Urbit ID). The basic idea is that an ID should cost something so that they aren't used abusively. But, in late 2020 the combination of a crypto bull market and congestion on the Ethereum network caused the transaction fee for buying an Urbit ID to increase from ~$1-2 to ~$100. An Urbit ID should cost _something_, but \>$100 is too much of a barrier for entry. We knew that this could happen and started researching solutions late last year, ultimately deciding to [implement a solution ourselves](https://urbit.org/blog/rollups/). The tough part about having an identity verified on the blockchain is that you become vulnerable to unpredictable volatility. We saw a period of painfully high prices, followed by a dip, followed by another stretch of extremely high prices. We opted to continue releasing Landscape updates and moving the system forward despite the fact that it became prohibitively expensive to onboard people. Luckily, our solution to this problem should go live this summer, and will reduce the cost of onboarding about 60x. The best part is that it is a permanent solution so we'll never have to face this problem again. The other challenge to scaling the onboarding process is hosting. We have a functional hosting structure but we want it to be fully automated and featured. We've been able to onboard friends and enthusiasts which has taught us a lot about how the system should work. But hosting has to be easy for anyone to join and get online quickly. Without an automated invite flow and it's not ready for prime time. We've made great progress on this front. We are currently testing improvements internally and aim to have a final version ready simultaneously with lower planet fees. Then, at long last, we can comfortably invite friends to get on the network without ever leaving the browser. It's worth mentioning that hosting is very much a Tlon Corporation project. Urbit is a platform owned, maintained, and developed by the people that use it (along with the nascent Urbit Foundation). Tlon, while also contributing to the Urbit project, ultimately wants to produce applications and services on Urbit. Hosting is one of those projects. Hosting makes Urbit easy, approachable, and usable without the compromises involved in using a conventional service. Since we'll be one of many Urbit-related providers, Tlon won't be the only host. There are already a few others (like [UrbitHost](https://urbithost.com/)) getting off the ground, which is fantastic. **Landscape → Grid** As the world has become more physically distributed we rely on our digital tools to keep our communities connected. The rise of DAOs, collectives, squads, and alternative community governance structures underscores the importance of tools like Landscape that communities can actually own and control. Our first goal was to find out if we could build something really good on top of Urbit. We did that. And we're continuing to lower the barrier to entry. We've also got an Urbit 'messenger' iOS app in the works that's really nice. Landscape is great, but in the long run Urbit is meant to be a platform for anyone to build on top of. Urbit is a computer, not just a single communications app like Landscape. The whole point of Urbit is to make it easy for an individual to run their own general-purpose computer in the cloud. Your Urbit isn't just meant to run Landscape it's meant to run any sort of application. And that's where we're going next. Grid is a holistic interface for interacting with your Urbit node. Landscape, as it's known today, will become one of many apps that can be launched. Perhaps most importantly, Grid will have peer-to-peer software distribution over the network. This means you can, as a developer, build something, let a friend sync it over the network, and never have to run a dedicated distribution server. Or, as an everyday user, you can run software on your private Urbit node without the developer ever being involved. Urbit isn't an OS in the traditional sense—it's an 'overlay' OS. It has to run on top of something else. But Urbit _is_ an OS in the sense that it's general-purpose. And our goal in building interfaces for Urbit is to deliver that feeling of openness and flexibility to the user. Grid is a major step toward that goal. Not only should a community own their communication tools, they should be able to run and host their own cloud software. They shouldn't need to rely on providers to store their data, run the application, and determine how their digital life will function. Grid will surely go through the same process that Landscape did. It'll start as a working first version that will inspire us to refine and make it better. But, unlike Landscape, Grid opens up an entirely new frontier for what can be done on Urbit. We're putting together our first conference on Oct 15-17th to inaugurate Urbit as an integrated platform and distribution system for new applications. [Tickets for Assembly 2021 are already available](https://checkout.eventcreate.com/assembly2021/select-buy). Tlon is going to Austin! **Closing** Urbit is a deeply optimistic project. Although the digital world seems more broken than ever, we're confident that something else is possible. We have already created software that people use everyday. We want everyone to have the experience that we do, and we want that experience to continue to get better. Beautiful, reliable infrastructure that communities can build on will have a significant impact on the world. The end game of connected computers doesn't have to be dystopian. For those of you who aren't directly involved with the project, I hope this gives you a snapshot of where we are. If we were on an Antarctic expedition the postcard would read something like 'Team is well-fed, healthy and motivated. Making great progress.' With that, I'll get back to work. ]]> https://urbit.org/blog/urbithost-interview https://urbit.org/blog/urbithost-interview Thu, 19 Aug 2021 00:00:00 GMT https://urbit.org/blog/on-christopher-alexander https://urbit.org/blog/on-christopher-alexander Mon, 19 Jul 2021 00:00:00 GMT I suppose the main thing is that most designers tend to become fixated on the mechanics of what they're building. Most design theory is about how things are made, not what it's like to live inside of them or actually rely on them on a daily basis. Alexander is much more concerned with the first-person experience of inhabiting a place or a building. I guess his approach is just much more empathetic which means it's also more practical. He's concerned with _how people actually live_ and how their environment evolves alongside them. > > We live, work, and develop on Urbit. We're often the first people to notice bugs because we are the end users ourselves. Alexander makes the point that old buildings were responsive to feedback because the architect was also the inhabitant. This tight feedback means that the system as a whole is dynamic and alive. We want Urbit to be a house that you live in and make yours, because unlike all the other software you use, your Urbit is actually _yours_. **Josh (~wolref-podlex)** > _The Nature of Order_ has had a big influence on my thinking. At first, the idea of all physical structures being alive sounded like something out of a new age bookstore. But not only do I think that Alexander justifies this claim; it started to make intuitive sense when I looked closely at my experience. > > When I first read _The Nature of Order_ I was living in an industrial neighborhood of San Francisco—the scale of the buildings and streets felt inhuman and everything was made of metal, glass, and concrete. I chose to leave. I moved into a 100+ year-old flat and the difference was immediate and visceral. The space felt like it was designed from experience rather than _ex nihilo_. The apartment was meant to be lived in, surrounded by public spaces where strangers could safely spend time outdoors together. > > If physical objects can have the quality of "aliveness" so too can information objects. Alexander's book is alive. Many of the software projects I've worked on have been cold fish. Keeping this quality in mind is an important part of my decision making. I want Urbit to be a living structure. **Édouard (~fabled-faster)** > I enjoy telling people unfamiliar with Urbit and interface design with respect to Urbit, that the object of our work is essentially that of designing an environment "like a piece of paper". > > Consider how paper as a constrained environment is one that yet can bear nearly any visual expression imaginable—apart from containing drawings or human-made marks, paper can be used to display photographs, paper can be multiplexed to form lenticular holograms, paper can be folded into a crane, etc. > > The "sheet of paper and its resulting forms" I manifest as a design ideal here is what Alexander might refer to as a "semilattice". A mathematical term he co-opted to describe an ideal structural relationship between elements in a system, as opposed to a "tree", which is a top-down structural system that forces elements into rigid categorical constraints. [In his own words](https://www.patternlanguage.com/archive/cityisnotatree.html): > > > _For the human mind, the tree is the easiest vehicle for complex thoughts. But the city is not, cannot and must not be a tree. The city is a receptacle for life. If the receptacle severs the overlap of the strands of life within it, because it is a tree, it will be like a bowl full of razor blades on edge, ready to cut up whatever is entrusted to it. In such a receptacle life will be cut to pieces. If we make cities which are trees, they will cut our life within to pieces._ > > A city is a good ideal to maintain for the construction of a new internet and notion of personal computing, but I've tended towards my own interpretation of a piece of paper as a semilattice-embodying object due to its accessibility and ability to be imagined by anyone, designer or not. Our object of design is that of a receptacle of life, of deep expression, of multiplexed possibilities. When I imagine an Urbit in the not-so-distant future, I imagine it as a _method_ (to contrast against those who pin software's expression down to "tools" or "functions") by which I can become "whole" again in a digital context, in all my complexity. Christopher Alexander wanted to collapse the distinction between those who live in buildings and those who design them. Urbit wants to do the same thing with software—in order to not produce ugliness, we should live in what we build. Although we have personal computers now, all of our data is housed in enormous warehouses that make the industrial computers of the 1950s look tiny. What we think of as “our information” actually belongs to someone else and, more often than not, is weaponized against us. Alexander’s moral imperative can be taken and applied to our digital structures. We need to become close with our software again. ]]> https://urbit.org/updates/2021-06-16-update https://urbit.org/updates/2021-06-16-update Tue, 15 Jun 2021 00:00:00 GMT https://urbit.org/blog/a-topiary https://urbit.org/blog/a-topiary Sun, 13 Jun 2021 00:00:00 GMT Under the trees of England I meditated on this lost and perhaps mythical labyrinth. I imagined it untouched and perfect on the secret summit of some mountain; I imagined it drowned under rice paddies or beneath the sea; I imagined it infinite, made not only of eight-sided pavilions and of twisting paths but also of rivers, provinces and kingdoms... I thought of a maze of mazes, of a sinuous, ever growing maze which would take in both past and future and would somehow involve the stars. -- _The Garden of Forking Paths_, Jorge Luis Borges The pre-technological world was already [networked](https://en.wikipedia.org/wiki/Indra%27s_net). No material object and no idea has ever stood alone—"no man is an island". Language itself is a network of associations shared between a group of people. The question of technology is: how do we bring our web of associations into the realm of computers? The legacy internet has focused on social networks almost exclusively—our mammalian brains readily took to this, we all love to [signal](https://conversationswithtyler.com/episodes/robin-hanson/). But before “social media” there was something called hypertext: a seemingly modest way to connect documents shared between computers. A footnote in a book used to require a library, locating the referenced book, and finding the cited page (which might have footnotes itself)! Hypertext allows one to traverse a network quickly and easily with just mouse clicks, making information accessible to those without the time and resources to hunt down physical artifacts. Urbit’s “%graph-store” is not the same as hypertext but it does bear some familial resemblances. Fundamentally, %graph-store is a data structure that can accommodate disparate data types within a single edifice. It is based on [graph databases](https://en.wikipedia.org/wiki/Graph_database) and recent implementations such as [GraphQL](https://graphql.org/). To understand this data structure it is worth diving into the origins of hypertext—the original linked network. The story of hypertext begins at the end of WWII with an article in _The Atlantic_ that proposed to extend the human mind via machine. Vannevar Bush had seen first-hand the explosion of electronics manufacturing and scientific knowledge, recognizing the great peril and opportunity of such a revolution. His [description](https://www.theatlantic.com/magazine/archive/1945/07/as-we-may-think/303881/) of the Memex presaged the technological world of today, envisioning a personal device that would allow the user to draw upon a vast library of information, and link documents together arbitrarily—what he called "associative indexing"—a deliberate recreation of how the mind connects thoughts. The 1960s saw the first implementations of these ideas. Ted Nelson's [writing](https://archive.org/details/SelectedPapers1977/page/n15/mode/2up) around _Project Xanadu_ elaborated this linking between documents into what Nelson called _hypertext_. Nelson imagined "a text arranged in a graph structure", centered around a corpus arranged in nonlinear sequence, manipulable by author or audience. The concept was first demonstrated in 1968, at Douglas Engelbart's 'Mother of all Demos', where a vision of computers as a collaborative social environment was revealed to the world. "[The Journal](https://www.dougengelbart.org/content/view/137/#7)", presented alongside other revolutionary demos, exhibited collaboratively edited documents with hyperlinked connections. Any time-travellers in the audience would have immediately recognized the first webpage. From here, the idea of a linked network of documents would be developed through successive phases of computing, from the mainframe to the PC. Experimentation with these ideas continued through the 1970s and 80s. A team directed by Andy Lippman at MIT extended the concept to threaded visual hypermedia with the _Aspen Movie Map_, a user-directed virtual tour of Aspen, Colorado. Hypertext and hypermedia gained a mass audience for the first time with HyperCard, a program that shipped with Mac computers. Users were able to collect or create '[stacks](https://archive.org/details/hypercardstacks)' of interlinked cards; a personal body of text, art, and knowledge could be accrued and modified, or shared with others—though not yet with native networking capabilities. With the World Wide Web, hypertext achieved escape velocity. Initially a one-man [project](http://info.cern.ch/hypertext/WWW/TheProject.html), the world wide web was similar to Bush’s vision of the Memex—a way to share, index, and connect information between individuals and groups. The web succeeded by utilizing the physical infrastructure already available: PCs with graphical interfaces, DNS, and the internet. Critically, its method of transmitting and traversing texts was a standardized protocol, allowing a free body of information to assemble atop the new infrastructure by anyone willing to learn the tools. A new frontier was cleared and cultivated with open standards, but it would gradually face enclosure. In the following decade, after the first wave of social networking sites had crested but before phones became the primary web clients, Twitter was launched. Originally an awkward bridge between SMS and the web, its design eventually coalesced around static identity, posts with chronological sorting, and interlinked post threads. These simple primitives spawned a vast corpus of information, a graph-of-graphs intertwined and overlapped, mapping an entire platonic world. Twitter is fundamentally constrained by its very nature: it is the walled garden of an advertising company. The financial incentives of the server operators demanded seizure of the commons and the replacement of standardized protocols with proprietary databases. As the first networked services withered, a new critique was implicitly articulated in the construction of a [new kind of abstract machine](https://moronlab.blogspot.com/2010/01/urbit-functional-programming-from.html). Urbit learned from the past: tightly defined protocols, baked-in cryptography, and decentralization were key. If power over the network was determined by those who ran the servers, then we would run the servers ourselves. Urbit is not fundamentally a hypertext system; it is a new kind of computer, designed to participate in a peer-to-peer network that rejects the client-server model. All Urbit software takes its networking model for granted, it is not an ad-hoc layering of mismatched abstractions. Fundamental design decisions determine the trajectory of the future. Early [implementations](https://en.wikipedia.org/wiki/Neo4j) of graph databases and projects like [RDF](https://en.wikipedia.org/wiki/Resource_Description_Framework) hit on the same core idea: linked data needs its own data structure. In 2020 Urbit OS adopted an application called %graph-store that made use of a new, native [data structure](https://docs.google.com/document/d/1-Gwfg442kV3cdfG7NnWPEf2TMa3uLUTAKkZD70ALZkE/edit) which would act as a database service for social applications. Chats, long-form text, and other types of data produced in the course of social computing now share the same tent, a single structure extensible to new types of content. Each item added to it is a node in a personal graph—a tree that grows branches as your computer communicates. These nodes can be linked arbitrarily to other graphs—Urbit's global immutable namespace grants both permanence and ownership and allows the graph’s branches to intertwine with others. Graphs can accommodate a variety of content, from simple text, to code and media yet to be conceived. You own your graph, and it is a record of your digital journey with others. Personal hypertext graphs are leaving behind proprietary platforms and gaining independence. Graphs until now have been used as tools of surveillance and marketing, but they are being reclaimed and sewn back into the network, each bound to the cryptographic identity of an individual, permanent computer. Where the map of your life was once splintered across a thousand systems and held in the custody of strangers, it is now possessed only by you and shared only as you see fit. On Urbit, an infinite forest of digital lives can take root, nesting their boughs and stalks. ]]> https://urbit.org/blog/first-contract https://urbit.org/blog/first-contract Fri, 04 Jun 2021 00:00:00 GMT 
Recently, the [first upgrade](https://github.com/urbit/azimuth/pull/35) to the [Azimuth](/docs/glossary/azimuth) smart contracts which make up Urbit ID was put up to an [upgrade proposal vote](/docs/glossary/upgrade) by the [Galactic Senate](/docs/glossary/senate). This vote is still open and will last for 30 days (June 20th) or until an absolute majority is reached. We’d like to take this opportunity to discuss two things. [First](#changelist), what the proposed changes are, and [second](#senate), a review of how Urbit governance works and where we’re at in the decentralization process. Before we dive into what the proposed changes are, let’s review what the Senate is actually capable of changing. Urbit ID is really two sets of smart contracts: Azimuth and [Ecliptic](/docs/glossary/ecliptic). Azimuth is the data of the public key infrastructure - roughly, this is the list of [ships](/docs/glossary/ship) and which Ethereum addresses own them, along with other data such as networking keys and sponsorship status. The Senate has no ability to touch this data directly. This is in direct contrast to all existing centralized services, where your account is always at risk of being taken away from you. What the Senate can change is Ecliptic, which is the “business logic” that decides how you can interact with the data in Azimuth. These are mechanisms such as what powers various [proxies](/docs/glossary/proxies) have, how stars/planets are released over time, and how sponsorship works. Put another way, the data and database format must remain the same, but the rules by which we interact with it may change according to the governance rules we [detail below](#senate). ## Changelist {#changelist} The Galactic Senate is voting on the following changes to Ecliptic: - Fixed ERC721 compatibility - Self-modifying proxies - Upgraded Claims contract ### Fixed ERC721 compatibility In Ecliptic, the contract includes various functions and events that make it conform to the ERC721 (non-fungible token) standard. However, two events have [ever so slightly non-confirming](https://github.com/urbit/azimuth/issues/8) descriptions. This causes Ethereum explorers like Etherscan to incorrectly recognize Azimuth as ERC20 (fungible token), rather than ERC721. The proposed change simply modifies the event descriptions to accurately match the ERC721 definition. This does not affect any functionality within the contract itself. ### Self-modifying proxies Owners of Azimuth assets are allowed to configure "proxy addresses" for those assets: Ethereum addresses allowed to act "as" the owner, but only for a subset of operations. For example, setting the management proxy will let you change your networking keys and sponsor using that address, just like your ownership address can. This is useful for keeping the ownership keys in very cold storage while still letting you perform lower-value operations. Currently, only the ownership address is allowed to change proxy addresses. Fairly early on after Azimuth's deployment, we realized it might be nice for proxies to be able to change themselves. This means that, in addition to being able to configure networking keys and sponsorship, your management proxy would be able to assign a new management proxy. This would allow you, or a trusted third party holding your management proxy, to rotate your proxy keys without needing to take your ownership keys out of cold storage. ### Upgraded Claims contract [The Claims contract](https://github.com/urbit/azimuth/blob/master/contracts/Claims.sol) lets asset managers associate various "claims" with their identity. For example, an Ethereum address to send donations to, or proving ownership of a Twitter account. For ease of use by off-chain tools and services, the contract emits events (notifications) whenever any claims are updated. However, the current version of the contract has a bug where it does not emit events when all claims are removed at once (clearClaims()). This "gap" in the event stream makes it much more difficult to write off-chain tools around this contract. Considering Claims has seen [practically no use](https://etherscan.io/address/0xe7e7f69b34d7d9bd8d61fb22c33b22708947971a#events) since it was first deployed, it should not be a problem to simply start over fresh with a new contract that contains [the fix](https://github.com/urbit/azimuth/pull/32) for the described bug. The reason we must do an Ecliptic upgrade to switch to a new Claims contract, is that it is tied in with the logic for transferring an asset. If an asset is flagged for "reset" (common when transferring to/from someone other than yourself), its configured proxies, networking keys and claims are cleared during transfer. As such, Ecliptic will need to know about the address of the new Claims contract. ## Urbit Governance {#senate}  The ultimate goal of Urbit is to become a digital republic manifested as a peer to peer network owned and controlled by its users. The goal of Tlon, then, is to become one of many companies who build products for Urbit rather than being the primary driving force behind its development. We’d like to take this opportunity to spell out our perspective on where we’re at in that process. Here are some previous posts that are relevant, but please note that some of them are several years old and as such may not accurately reflect our current position, but still serve as useful historical markers. We hope to revisit and refresh these documents soon. - [2016.5.16 - Interim Constitution](/blog/interim-constitution) - [2016.5.16 - The Urbit Address Space](/blog/the-urbit-address-space) - [2016.6.24 - The DAO as a Lesson in Decentralized Governance](/blog/the-dao-as-a-lesson-in-decentralized-governance) - [2019.1.11 - Governance of urbit.org](/blog/governance-of-urbit) ### Galactic Senate The Galactic Senate is composed of all [galaxy](/docs/glossary/galaxy) holders, which at present consists of more than 100 individuals and a few organizations, including Tlon and the Urbit Foundation. Using the Azimuth voting contract, the Senate can present and vote on two kinds of proposals: [document proposals](/docs/glossary/docvote) and [upgrade proposals](/docs/glossary/upgrade). Thus far, all matters which the Senate has voted on have been document proposals. The [most recent vote](https://github.com/urbit/azimuth/pull/34) declared the Urbit network as (i) being secure (as confirmed by a [third party audit](/blog/security-audit)) and (ii) having reached continuity (as in, no further [network breaches](/docs/glossary/breach) are expected). Previous votes were to declare that [Azimuth is live](https://github.com/urbit/azimuth/pull/25) and that [Arvo is stable](https://github.com/urbit/azimuth/pull/24). ### Address space distribution Perhaps the most informative measure of how decentralized Urbit is in terms of how many independent parties hold address space. By the very nature of Urbit, it is impossible to know this with great accuracy (a common feature of decentralized projects). However, the movement of galaxies is closely watched and thus we have a fairly good idea of how distributed they are. In the beginning, Urbit’s creator Curtis Yarvin was in possession of the entire address space - all 256 galaxies, and everything underneath it. A network of one is no network at all, and so over the last decade Urbit’s development has been primarily driven by selling or giving away these galaxies. On [June 1, 2016](/blog/the-urbit-address-space), the allocation was: > 95, to the Tlon Corporation. 50, to urbit.org, the future community > foundation. 40, to Tlon employees and their family members (24 to Curtis, who > started in 2002; 16 to everyone else, who started in 2014). 34, to outside > investors in Tlon. 37, to 33 other individuals, who donated to the project, > contributed code or services, won a contest, or were just in the right place > at the right time. Since then, Tlon has sold a number of its galaxies to fund development and others have changed hands to the point that Tlon and urbit.org no longer possess a majority share of galaxies. In January 2019, [Curtis gave all of his galaxies](/blog/a-founders-farewell) to Tlon when he left the project. In August 2020 we shared an update on the [known distribution of address space](/blog/value-of-address-space-pt3), where Curtis’ galaxies are marked as Tlon’s “naked galaxies”. Shortly thereafter, Tlon disbursed its naked galaxies among the employees that wanted one who did not already possess one, thus removing Tlon and urbit.org’s controlling share of the Senate. Galaxies held by current and former Tlon employees are entirely independent of Tlon Corporation—they may do with them, and vote with them, as they please. `~ravmel-ropdyl` plans to give more color to this decision in the near future. Distribution of stars is much more difficult to know. There is an active market for stars on [OpenSea](https://opensea.io/collection/urbit-id), so we know that they are changing hands frequently, but checking the number of distinct Ethereum addresses that hold stars does not tell you very much information since a single person can control multiple adresses. ### Software and smart contracts As was written in [The DAO as a Lesson in Decentralized Governance](https://urbit.org/blog/the-dao-as-a-lesson-in-decentralized-governance/), we are keenly aware of the threat of _decentralization theater_. Urbit has undergone steady progress towards decentralization, and with the developments we outline below, it’s never been more clear that the project is beginning to leave the cradle of Tlon. It’s also important to note that the decentralization of a project is always a movement away from an initial centralized state⁠—you cannot go from A to Z without first traversing every letter in between. While Urbit was centralized at the beginning, the journey towards a network owned and controlled by the users is now well underway. In the past, nearly all Urbit software was written by Tlon. While Tlon is still the only corporation actively updating Urbit’s MIT-licensed open source software, namely [Vere](/docs/glossary/vere) (the runtime), [Arvo](/docs/glossary/arvo) (the kernel), [Landscape](/docs/glossary/landscape) (the client), and [Bridge](/docs/glossary/bridge) (the ID manager), unaffiliated individuals have been making enormous contributions to Urbit over the past couple of years via the [grants program](https://grants.urbit.org/history). Recent examples include the [Bitcoin node and wallet integration](https://github.com/urbit/urbit/pull/4940), [WebRTC integration](https://grants.urbit.org/proposals/21131866-webrtc-gall-agent-and-external-app), [Port, an Urbit installer and ship manager](https://github.com/arthyn/port), and [`|fuse`, an important primitive for 3rd party software distribution](https://github.com/urbit/urbit/pull/4820). The only real power Tlon holds over Urbit is the ability to push [OTA updates](/docs/glossary/ota-updates) via `~zod`, and suggest that people download our binaries. To the best of our knowledge, all extant galaxies retrieve their OTA updates from `~zod`, who forward it to their stars, who forward it to their planets. This software distribution route is merely a convenient default setting. Ships initially retrieve OTA updates from their sponsor, but they have full authority to retrieve updates from whoever they want with the `|ota` command, or even to only install software updates by hand if they so choose. In the future, one can easily imagine many organizations shipping their own distributions of Urbit from their own urbit nodes, somewhat analogously to the large number of Linux distributions which exist today. There is nothing we can do to stop a group from forking Urbit, declaring `~sampel-palnet` the new `~zod` (both for updates and for sponsorship purposes), and retrieving all OTA updates via that route and building their own binaries. This is all by design, and we wouldn’t want it any other way. The use of the Azimuth smart contracts is also somewhat optional. By changing a few lines of code to point their ships at another set of PKI smart contracts, they could rid themselves of hierarchical peer discovery entirely and make use of some other routing strategy such as [Kademlia based DHT routing](https://en.wikipedia.org/wiki/Kademlia). This would likely give rise to an entirely separate network. The fact that the network is fully capable of such a schism helps keep the Senate honest and accountable to the users. If they ever were to vote on a contract modification that greatly angered or upset a portion of the user base, they would be inviting such a schism and be entirely deserving of it. In that sense, the power to control the network is already held by the users, they just have not yet had a reason to make use of it. ### Legitimacy The only way by which Tlon and the Senate hold any power is via legitimacy. Vitalik Buterin recently wrote a [very informative article](https://vitalik.ca/general/2021/03/23/legitimacy.html) on the role of legitimacy in decentralized governance, and I do not know of a better source to understand how Tlon and the Senate are kept in check by the users. Besides the DAO split resulting in Ethereum and Ethereum Classic mentioned in the article, perhaps the most prominent example of the power users have over a decentralized network is the saga of [Steem and Justin Sun](https://www.coindesk.com/justin-sun-bought-steemit-steem-moved-to-limit-his-power). The short version of the story is that Justin Sun bought a controlling share of the Steem blockchain and attempted to exert control over it, and the users banded together to fork the blockchain into one called Hive, which was identical to Steem except with Sun’s token balance set to zero. An analogous outcome for Urbit would be that the Senate makes sweeping changes to the PKI and invites a schism like that outlined above. For these reasons, there are strong disincentives for any one organization or individual to acquire too many galaxies. If galaxy ownership becomes more centralized, individuals will be less incentivized to build on the network which will reduce the value of address space. Furthermore, such centralization might provoke a fork if such centralized powers are perceived as malicious by the users. This is in contrast to real-life republics, where there is little risk for one party to accumulate as much power as they can manage since the cost to the citizens to overthrow that power is enormous. Thus, only especially egregious displays of power ever result in rebellion. Within the Urbit republic, users need only to change the code they run to leave the Senate powerless, and so the threat of rebellion is much more real. We estimate that within a digital republic like Urbit, the centrifugal forces outweigh the centripetal forces, and that the outcome of this is more stable than the reverse. ### Why have governance at all? An important question the reader may still have lingering is: why does Urbit need governance at all? The short answer is: (i) there will always be necessary protocol upgrades that cannot be foreseen, and (ii) a total lack of governance inevitably gives rise to a shadow government with undefined powers that is almost impossible to hold accountable. With (ii), we see this to be the case on many early blockchain projects where power is shared by the developers and the miners, but what each is capable of doing has never been codified, and thus users of the blockchain can never have a great deal of certainty of what changes might occur in the future. This doesn’t guarantee a bad outcome, but it does create more wariness among developers when they cannot know whether they really have a say in the project's direction. Recognizing this issue, many newer blockchain and blockchain-adjacent projects have built-in governance mechanisms that lay out exactly what an individual can expect should they join and contribute to that network. As for (i), with Urbit being a wholly new kind of network, it would take incredible luck to get the PKI exactly right on the very first try without ever needing another modification. Thus, some mechanism was needed to guarantee that the PKI’s logic could be altered. Leaving control of Azimuth in the hands of Tlon would have just granted Tlon dictatorial power, which Tlon nor Urbit users desired. There’s certainly room to argue that the Galactic Senate is not a perfect system: our interim constitution did outline potential roles for stars and planets to play in governance in the future, so there is ample opportunity for the system to evolve in response to the will of the users. As always, it is important to keep in mind the unfinished nature of the Urbit of today and the still moldable clay of the Urbit of tomorrow⁠—nothing yet is set in stone. ]]> https://urbit.org/blog/rollups https://urbit.org/blog/rollups Thu, 13 May 2021 00:00:00 GMT https://urbit.org/blog/iot https://urbit.org/blog/iot Thu, 29 Apr 2021 00:00:00 GMT 
Each of the ~4B 32-or-fewer-bit or [Azimuth](/docs/glossary/azimuth) identities, namely [planets](/docs/glossary/planet), [stars](/docs/glossary/star), and [galaxies](/docs/glossary/galaxy), may spawn ~4B 64-bit child identities known as [moons](/docs/glossary/moon). Moons are the undeveloped wild outback of Urbit, with a story waiting to be told that promises to be as enormous in scale as the cloud personal computing story told by planets, and possibly even more so. Throughout this piece, we use "Earth" and "Mars" in the Urbit sense. "Earth" refers to the [current computing paradigm](https://en.wikipedia.org/wiki/Big_ball_of_mud). "Mars" refers to the perfected computing paradigm that Urbit aspires to be, but also to the current and future state of Urbit. Today we will review the prehistory of moons on Earth, give a snapshot of how moons are being utilized on Urbit in 2021, cover existing issues with consumer IoT and how Urbit resolves them, and speculate what the lunar locus of the Urbit ID address space may become in the future. To keep the futurist scope limited, we primarily speculate on the usage of moons in (industrial) internet of things settings, especially with respect to agriculture, and how they may manifest as the workhorse collection unit of decentralized data markets. For those unfamiliar, moons differ from all other [ships](/docs/glossary/ship) in that they are not independent identities with total sovereignty over their data and keys. Instead, their cryptographic keys are issued and controlled by their parent identity. The relationship between the keys of a parent ship and its moons are analogous to the relationship between primary keys and subkeys in a system like [PGP](https://en.wikipedia.org/wiki/Pretty_Good_Privacy). The parent may revoke or change its child moons' keys at any time. A typical relationship between a planet and a moon may be like one of a human and their smartphone. If you lose your phone, you don't want someone to pretend they are you, operating one of your moons, on the network. The planet can revoke the keys, and the moon stored on the phone becomes inert, no longer able to communicate with the network. While planets and stars also depend on their sponsor to assist with peer discovery and software updates, these relationships rely on mutual agreement, since either party may choose to discontinue the relationship and establish a new one elsewhere. Moons possess no such freedom. ### Lunar prehistory  It may come as a surprise to you that you are already very familiar with moon identities—they've just never been explained as such because the context in which that terminology makes sense didn't exist before Urbit. Moons, in fact, are not a new concept at all. Virtually your entire experience of the internet in the modern era is the experience of someone who possesses some number of moons subordinate to planets who mostly refuse to talk to one another. The feeling you get of constantly moving between silos, juggling accounts, not controlling your own data, etc., is a direct result of the fact that, on Earth, you only possess moons. No planets. We're speaking, of course, of the client-server relationship. From the Urbit perspective, the Facebooks, Twitters, and Googles of the world are planets, and accounts held with these services may be thought of as moons subordinate to those planets. Your account with these megacorps is only yours for as long as they please. If you happen to step on a digital landmine, such as a virus that causes Google's servers to believe you are a bot, you are suddenly banned from their service, often with no recourse. All your data, your connections, your network—vanished. We are all Mark Zuckerberg's moons, and they're armed to the teeth with surveillance equipment. There are two classes of internet users: those with servers who hold all the power, and clients who use those servers in the way prescribed by the owners. Running a server is theoretically accessible to anyone, but practically difficult: server maintenance is a technical endeavor that requires specialized knowledge and substantial time investment. Other barriers exist beyond just the technical: for instance, if you wish to set up your own email server, you will probably be blocked by large providers because unknown email servers are almost always spam. The root cause of this circumstance is that the email protocol lacks [Sybil resistance](https://en.wikipedia.org/wiki/Sybil_attack). Urbit has Sybil resistance built in, eliminating certain kinds of incentives for gatekeeping such as this. We seek to level the playing field by giving everyone a server. We accomplish this by making the process of running your own server (planet) as easy as running your personal computer today. We aren't quite there yet, but we've made leaps and bounds over the past couple of years. Of course, this analogy isn't perfect—Urbit moons are servers themselves, but as they may only act in accordance with the wishes of its parent, they are effectively as locked in degrees of freedom as a client is to a server. The moon's keys may be revoked at any time by the parent. To distinguish these prehistoric moons from Urbit moons, we will refer to them later as "primitive moons". Urbit contends that moons are not the appropriate digital exocortex for a responsible adult. They are for devices, experiments, or your children. We hope that the dominant server-client relationship, which is the primary weapon of the surveillance-industrial complex, will meet its end in the near future. ### Moons in the early Martian era  In 2021, moons are mostly unused on Urbit. There's a smattering of minor use cases which we'll describe here, but they represent only a tiny fraction of their ultimate potential. Tlon recently launched a survey bot called [Eliza](/blog/eliza), which was initially a comet. We frequently got questions on whether Eliza was actually Tlon's bot, because it could just as easily be a malicious actor posing as Tlon. We soon plan to relaunch Eliza as a moon of `~zod`, which is the most recognizable official galaxy of Tlon. This is to establish that the bot is indeed official; only someone with access to `~zod` is capable of creating a moon of `~zod`, and so a moon of `~zod` automatically carries with it a cryptographic guarantee that you are communicating with an official digital representative of Tlon. On Earth something like this would need to be accomplished with certificate chains but, on Mars, this sort of authentication is baked into the very fabric of the network, rather than as an additional layer on top. This is a very good representative case of how moons naturally inherit the reputation of their parent, even in the absence of any formal reputation systems on Urbit. We see folks using moons as backup identities in case they can't or don't want to access their primary planet for one reason or another. Again, since anyone may verify the parent of a moon, the moon inherits the reputation of its parent. These backups could be vanity identities—for instance, every `~sampel-palnet` has underneath it a moon named `~master-sampel-palnet`[^0]. Some use moons to host group(s). Hosting a group inevitably means that that ship will process more network traffic, and a heavily trafficked group may impact a ship's performance. Thus one may wish to offload group hosting duties to a moon to maintain a more performant planet. As we currently lack a real system for 3rd party software distribution (though see this recent [developer call](https://youtu.be/XwICC6Its1E) on a solution in progress), one stopgap has been to download software by syncing a foreign [desk](/docs/glossary/desk) to your ship. This is risky, as buggy software may make break your ship and make it difficult to get running properly again. For this reason, testing new software on the livenet is frequently performed on moons, so as not to risk the parent ship if something were to go wrong. ### Internet of Things  Now we start to look towards the future by examining the problems faced by IoT in the present and speculating on how Urbit might solve them. The Internet of Things is much maligned by technologists as a privacy, security, and maintenance nightmare. These products usually depend on phoning home to perform their service, which is inevitably a funnel by which personal data is harvested, an attack surface for hackers to gain access to your home network, and a single point of failure which may render the device useless, all in one! One need look no further than the recent [Ubiquiti breach](https://krebsonsecurity.com/2021/03/whistleblower-ubiquiti-breach-catastrophic/) to see all of these failure modes happening on a massive scale, and this is par for the course. This sort of dynamic is completely antithetical to the Urbit philosophy, so any involvement of Urbit with IoT will reject many fundamental assumptions on how IoT operates today. However, it's important to understand why the current IoT model works like this at all. We posit that it ultimately stems from the fact that today's IoT products are primitive moons for which you do not operate the parent planet. Almost every IoT device operates in the Software as a Service (SaaS) paradigm because most people don't have personal servers that could do all the processing work actually necessary for an IoT device to operate. This is not the _only_ reason for this—it's also just more profitable to collect consumer data. However, it's the SaaS aspects which make the whole thing such a catastrophic mess. Imagine a different world: if the server that your device reports to is your own, there is no loss of control over personal data. If the device only communicates on the local network, it will prevent remote hackers and disruption if you internet fails. And if the manufacturer goes out of business, the device will still work-it is not dependent on the manufacturer's servers. The wants and needs of the individual are at odds with the wants and needs of the megacorp, and that mismatch manifests as the antifeatures that plague IoT. By eliminating the SaaS aspect, Urbit solves many of the largest outstanding issues with IoT in a single blow. Digital personal AI assistants have long been an anticipated science fiction prediction and their full realization is on the horizon. But in their current manifestation, many privacy- and security-conscious people would never consider using them at all, since they are inevitably presented as SaaS that requires intimate knowledge of your devices and their stored data. Recommendation engines such as the one used by YouTube offer helpful guidance towards content you'll find appealing, but they're also designed to keep you hooked to the service. These kinds of programs can be useful, but they're dystopian when they come saddled with anti-consumer subroutines. Since your Urbit belongs to you, it can never act against your own interests without your consent. In our lunar vision, your digital personal AI assistant will live on one of your moons, rather than on megacorp's cloud. For web-exposed devices, Urbit offers the same security guarantees that it offers for your personal computer: everything is end-to-end encrypted by default, and signed with keys that one can easily verify as belonging to the node sending the message. [Vint Cerf](https://en.wikipedia.org/wiki/Vint_Cerf) has outlined the case for a system like Urbit ID for IoT use [here](https://cacm.acm.org/magazines/2018/12/232883-self-authenticating-identifiers/fulltext). In brief, that the Urbit ID system allows one to match public keys with devices, bestowing devices with self-authenticating identities. This allows one to trust the communications received from the device to whatever degree the private key is held secure. IP addresses, GPS coordinates, MAC addresses, etc., are not sufficient to generate high levels of trust—they're all temporary, easily spoofed, or both. > One might imagine applying this to the Internet of Things (IOT) in which the > IOT device self-generates a public- and private-key pair and registers the > public key. For example, with a hub or controller so the hub can confirm it > has reached the right IOT device. By the same token, configuration of the IOT > device into an ensemble could include incorporation of the public key of the > controller into a list of valid devices that can command or obtain data from > the now-configured IOT device. Both ends can verify they are talking to the > originally configured devices, assuming no device has lost its private key. Here, the hub or controller referred to would be your planet. Urbit has many other features that make it well-suited for IoT, above and beyond eliminating the SaaS model and providing self-authenticating identities. Many of these additional advantages derive from the fact that Urbit is a [solid state interpreter](/docs/arvo/overview#solid-state-interpreter) and that [Ames](/docs/glossary/ames) is a network of solid-state interpreters equipped with cryptographic identity. We explore this in the next section. ### Future lunar cosmotechnics  We now polish off the old crystal ball and imagine future uses of Martian lunar technology. Particularly, how the tech may develop over time once it becomes widely exposed to the evolution-inducing forces of industry and culture. This section is pure speculation, and I only aim to offer a small glimpse of a potential future, rather than an exhaustive one. This picture will necessarily be biased by my personal proclivities and expertise, or lack thereof. Any discussion on the future of moons is incomplete without mentioning the possibility of [lunar emancipation](https://urbit.org/faq/#more-planets) in order to increase the number of planets. This fascinating topic sees regular discussion in the Urbit Community group, but is outside the scope of this article. In order to respect the reader's time, I'm narrowing our scope to a particular topic: the agriculture industry. I'm also assuming a future with specialized and increasingly small hardware to run Urbit. Right now, a 4 GB Raspberry Pi is the standard for "small Urbit". If someday there are CPUs specifically designed for running [Nock](/docs/glossary/nock), one could imagine going much smaller. Ultimately, I'm going to assume that there will someday be [solid-state interpreter](#ssi) microcontrollers - so the inclusion of Urbit on a device may require nothing more than a couple [surface-mount devices](https://en.wikipedia.org/wiki/Surface-mount_technology) on a larger chip. Urbit is intended first and foremost as a personal computer. When I unravel its potential usage in industry, I adopt the forecast of a drastic increase in the manufacturing capacity for individuals and small groups. Agriculture is a great microcosm of this long-standing trend; less human input is needed to produce the same amount of food, whether it be crops or livestock. The modern era is distinguished by the fact that small numbers of people can manage enormous farms which produce food for vast populations. Thus, computation on modern automated farms may be thought of as existing at the intersection of both personal and industrial scale, and show how computerized and mechanized workflows add to the amazing productivity of modern farms and demonstrate that individual-sized inputs are able to create industrial-sized outputs. I predict that the set of trades for which this dynamic holds will only increase over time. Agricultural automation has been a primary engine for the growth of civilization, but it has been hampered by the downsides of IoT elaborated in the previous section. One bellwether of this dynamic is the right to repair one's own equipment. [John Deere](https://www.thetruthaboutcars.com/2020/03/its-payback-time-right-to-repair-movement-targets-john-deere/) has long been at the center of this controversy. Farmers have lost the ability to repair their own equipment due to vendor locks and a dependency on remote servers. Furthermore, any suite of sensor products designed to gather important data such as genomics, hydrology, soil health, weather, etc., inevitably only cooperate with other products by the same vendor. It's the same story all over again—you are a client of a megacorp server. Or in Urbit terms, you have a bunch of primitive moons belonging to different planets held by megacorps, and no planet to call your own. Again, Urbit resolves these issues in the same fashion as above. One might argue that using Urbit is another form of vendor lock-in—but this is evaluating Urbit at the incorrect level. Since Urbit is an open source operating system, its functionality may be extended and modified arbitrarily by any party. _Some_ computational infrastructure is necessary— that fact can never be escaped. But you can escape from closed source software that does not permit you to modify, to open source software that allows you to tinker at will. Thus we do not consider open source software to be a form of lock-in, or at least, it's a much milder form of lock-in compared to something like Apple's walled garden. So let us focus on the additional meritorious aspects of Urbit (or more generally, solid state interpreter networks) that make it not merely a band-aid to the broken IoT system but a permanent and sound replacement. #### Solid state interpreter networks {#ssi}  Urbit is the world's first solid-state interpreter (SSI). This concept is summarized in the [white paper](https://media.urbit.org/whitepaper.pdf) as follows: > Imagine it as a chip. Plug this chip into power and network; packets go in and > out, sometimes changing its state. The chip never loses data and has no > concept of a reboot; every packet is an ACID transaction. Also, > Briefly, an SSI is an interpreter which is also a database. In more depth, an > SSI combines three properties: uniform persistence, source independent packet > networking, and high-level determinism. We do not wish to segue into a lengthy examination of what this really means— that's what the white paper is for—but we will attempt to explain why these properties are of great benefit in an industrial IoT context such as agriculture. We'd also like to predict that Urbit will not be the only SSI. Other sorts of SSI's specialized to various tasks may one day roam the planet, and these may be better suited for particular IoT tasks than [Arvo](/docs/glossary/arvo). However, this article will be couched in terms of SSIs communicating with an [Ames](/docs/glossary/ames)-like networking protocol and Urbit ID-style cryptographic identity, rather than Urbit in particular. For a SSI, losing power is less of a catastrophic event than it is for ordinary computers with volatile memory. Knowing that you can never lose any state is a powerful guarantee on the integrity of your data. Loss of power will always mean loss of any data potentially collected while power is down, regardless of the device you use, but with a SSI you at least have assurance that you will never lose data that was collected before it could be transferred from the cache to the disk. Of course, this is actually the sort of guarantee that [ACID transactions](https://urbit.org/docs/arvo/overview/#acid-database) have—it is just that this set of guarantees emerges naturally as a consequence when you have a SSI. Put another way, every SSI is a database; having every device be a database is a major boon in situations where enormous quantities of data are being collected across a distributed network. The standard today is that data is only collected at hubs, and any data that doesn't reach a hub is irretrievably lost. SSIs are also deterministic at a high level. While all computers are deterministic at the CPU level, interruptions to power can result in an undefined semantic state and result in the random erasure of half of its brain. This determinism is what begets ease of maintainability of Urbit—problems generally only need to be solved once by experts and the solution propagates to the rest of the network via [OTA updates](/docs/glossary/ota-updates). We addressed this maintenance issue previously—we only remark now that this property is derived from being a SSI, rather than something particular only to Arvo or Urbit. When one has hundreds or thousands of devices to manage, the guarantees on ease of maintenance provided by determinism are extraordinarily convenient. Packet routing on SSI networks is source-independent, and in the context of an Ames-like networking protocol, are always delivered exactly once and in a precise order due to the fact that nodes on an Ames network are in a permanent session. Source-independent means that the identity that originated a packet does not need to be the one to deliver it—packets are _facts_, and the truth of a fact does not depend on who you heard it from. This means that it is possible, for instance, to send instructions to a moon before it even exists and be guaranteed that that moon will receive them and perform the tasks given, even if the device that originally sent the packets no longer exists. As long as somebody has kept the packets on hand, they will eventually be delivered. So, one could imagine transmitting directions to a crop harvesting robot into which a moon will be installed before the robot has even been purchased, know that once it is delivered it will automatically begin its task, and know that those tasks will be completed in the correct order, without any accidental repeats. Taken to the limit, one can imagine preplanning the automated tasks for an entire farm before a single device is purchased, with powerful guarantees on the integrity of those instructions no matter how much time elapses between sending and receiving. #### P2P data markets  As was mentioned above, today's IoT devices ferry all sorts of your personal data to megacorp, where they package it and sell it to the highest bidder. Though personal data in aggregate is quite valuable, the value of a single individual's data is quite low. On the other hand, industrial scale data such as that collected on digitized farms could be quite valuable. Here we briefly examine how Urbit can be utilized to populate and facilitate a market for data. All data collected by your moons may be digitally signed by that moon, certifying the exact origin of that data. This is one advantage of having cryptographic identity baked into the core of the operating system and networking protocol. One could imagine collecting large amounts of such data and packaging it for exchange with other local and regional farmers. This can, of course, already be done in principle with current systems. But current systems do not automatically come equipped with the sort of provenance, immutability, and reputation that comes for free when you make use of SSI networks. Manipulated or faked data becomes easier to spot, and you'll know who it came from, so supplying such data puts one's reputation at stake. One major issue facing conjoining data gathered from disparate sources is simply how to do it. Data arriving in random order, from different software versions, running different hardware, is all very difficult to bundle into a good dataset - and indeed wrangling this sort of task forms a large part of the data scientist profession. As devices in SSI networks utilizing an Ames-like protocol are engaged in permanent sessions, leading to properties such as exactly-once delivery and well-ordered message sequences, issues about the order in which data arrives are dramatically simplified. I do not wish to get mathematical here, but I personally have confidence that the emerging discipline of [categorical informatics](http://math.mit.edu/~dspivak/informatics/), as well as the relevance of [low-dimensional topology to data fusion](https://arxiv.org/abs/1409.5505), resolves many of these issues. I promise to explain this in more detail in the future. For now, it suffices to say that SSI networks seem well-positioned to utilize these techniques. Cryptographic techniques such as [(composable) zero-knowledge proofs](https://en.wikipedia.org/wiki/Zero-knowledge_proof), [secure multi-party computation](https://en.wikipedia.org/wiki/Secure_multi-party_computation), [ring signatures](https://en.wikipedia.org/wiki/Ring_signature), and even [homomorphic encryption](https://en.wikipedia.org/wiki/Homomorphic_encryption) offer a variety of ways in which data may be shared in an IoT context with mathematical guarantees on what sort of privacy is maintained. This is perhaps less relevant in the context of agricultural IoT, where personal private data is largely absent, but we mention them here to invite further speculation from the reader about the possibilities of IoT and SSI networks. We end our discussion with an adaptation of a scenario borrowed from my brief time working on [Ceptr](http://ceptr.org/) that begins to show how a lunar IoT-connected world evokes properties of a hypothetical ["global nervous system"](https://en.wikipedia.org/wiki/Global_brain). This example is purposefully mundane to illustrate how deficient our Earth systems are, and while it is possible to accomplish in theory, it is impractical on a wide scale due to the numerous problems with IoT. Imagine that you suffer headaches from low atmospheric pressure. You'd like to know when a low-pressure front is an hour away from you so that you can pre-emptively take aspirin to prevent the headache. Your personal digital assistant, running on Urbit, can then periodically query the moons of sensors in your region for weather data to predict whether or not you will soon be passing through a low pressure front. This could be a free service or paid for automatically with cryptocurrency microtransactions. On Mars, this task will be trivial, and is merely a stone's throw away from a vast and undiscovered universe of other possibilities unlocked by SSI networks. [^0]: Of course, all moon names have four words, not three. What the fourth word needs to be for only three words to display is a matter of Urbit lore that is more fun to discover yourself than have it told to you, so we won't spoil it here. ]]> https://urbit.org/blog/interplanetary_commerce https://urbit.org/blog/interplanetary_commerce Thu, 08 Apr 2021 00:00:00 GMT 
## Paying A year ago, we started [integrating](https://urbit.org/blog/urbit-and-bitcoin/) Bitcoin into Urbit to be an OS-level primitive for money. Our initial objective: build a payments wallet, equally sovereign and ergonomic. The first public version, releasing this month, will have a broad set of capabilities including ship-to-ship payments, hardware wallet support, and full node integration. We’ll be unveiling this wallet on [April 27th](https://www.meetup.com/urbit-sf/events/276741426/). Do join us. Things only accelerate from here. Our next objective, to integrate Lightning Network capability, is now an [active bounty](https://gist.github.com/timlucmiptev/53ef1d290e55ea2fa25d9df681c32695) and open for applicants. Early on, we [recognized](https://www.youtube.com/watch?v=DTvRVHMbrcY) the congruence and compatibility between the Urbit and Lightning networks. Routing Lightning payments is an obvious business model for Urbit providers, one that can even nucleate new features onto Lightning itself. What we’re ultimately driving at is fully-sovereign Bitcoin usage, with the UX of the Cash App. ## Being Paid > And there is all the difference in the world between paying and being paid. The act of paying is perhaps the most uncomfortable infliction that the two orchard thieves entailed upon us. But being paid,—what will compare with it? > > Herman Melville However, a good wallet is only half the equation for a holistic P2P commercial experience. Interfaces for selling goods and services is the other half. While we don’t part with our Bitcoin [easily](https://nakamotoinstitute.org/mempool/everyones-a-scammer/), two initial markets appear logical and attractive: content and Urbit-native services. First, digital cottage industry, the “creator economy”, is blossoming. Streaming, podcasting, and writing on platforms like Youtube, Patreon, and Substack has been big business [lately](https://otherlife.co/barbarians-past-the-gate/). What fintech did to “unbundle the banks", we will now do to media monopolies and their centralized distribution platforms. With the introduction of Urbit payments, you can syndicate, distribute, and monetize your media on Urbit. By selling access to a private group containing content, digital creatives can begin doing this today. Second, Urbit-native [service providers](https://urbit.org/blog/providers/) are now operating at scale. Companies offering Urbit hosting, onboarding, Urbit IDs, and software now communicate with their customers directly on the network. A basic set of commercial tools will make their processes more efficient. Storefront interfaces, paywalls, CRM, and reputation tools will replace the ad-hoc assemblages they currently use. The Urbit Foundation is soliciting contributors to this project through Urbit Grants. If you’re interested in working on any of these components [send us a proposal](https://grants.urbit.org/create). ## An Ocean After honing our commercial tools on these use-cases, we’ll sail into a blue ocean. On the horizon: beautiful products, deployable at scale, built with the robust primitives of Urbit OS. Extensibility is Urbit’s promise to middle-men: every Web2.0 service they use to scalp us will find its P2P analog on Urbit. We invite developers and entrepreneurs of all stripes to join in the feeding frenzy. Ideas are constantly nucleating in Urbitcoiners and Urbit Community. ]]> https://urbit.org/blog/after-machine-war https://urbit.org/blog/after-machine-war Sun, 14 Mar 2021 00:00:00 GMT 
> The following fictional passage, from a book being written in an Urbit Group called “the Aleph,” sketches out a dismal future for the human race. As an item of speculative fiction, the piece is not what one might call original. Anyone could have come up with these ideas. In fact, several already have. The Matrix is an obvious antecedent. Vonnegut isn’t far behind either. And the treacly humanism lodged beneath the bitter humor resembles nothing so much as the work of that beau monde satirist, George Saunders. > > Really the only notable thing about it is that it was written on Urbit. And that, perhaps, has made all the difference. > > It’s easy to be pessimistic about the future. Intellectuals, especially, are drawn to pessimism, because pessimism (falsely) implies a greater wealth of knowledge about the ways of the world. But one thing the pessimist never sees coming is a turn for the better. To wit, the future described in the below excerpt doesn’t have to take place. And Urbit is one piece of evidence for the claim that it will not. > > The philosopher-cum-internet comedian Slavoj Zizek likes to point out that no one was really expecting the Soviet Union to collapse in 1989. In fact, the average Soviet dissident wasn’t even gunning for it. He was haggling over small reforms, pushing to get the less corrupt politicians to the top, just going about his day eating or opening a window or walking dully along. Then, kaboom. Collapse. No more USSR. > > Then as now, the unexpected is more likely to occur than most would believe. One thing is not only possible, but an established fact: A brand new digital infrastructure is slowly, quietly being built. While U.S. political parties haggle over $1,400 disbursements for citizens, the rules of global finance are being re-written in code. While pundits debate the merits of Dr. Seuss’s lesser known titles, a global shift away from traditional publishing is gaining steam. And while users and social apps themselves pop up and down like whack-a-mole, folks are putting down roots with permanent peer-to-peer digital identities. If you’re anything like an optimist, you’ll probably want to get on Urbit ASAP and put down your own.
The date is January 1, 2050. The place, New York City. The vibe...subdued. Back in 2020, you made a note to yourself to take stock of the world around you here at the half-century mark. And now the time has come. What do you find? The air quality is a bit worse, the bad parts of town are a bit more dilapidated, the Knicks are still terrible—all in all, from the outside at least, not much has changed from 30 years ago. Today you wake up groggy on your self-adjusting Smart Bed (Smart Bed Never Sleeps), you make yourself a steaming pot of coffee (locally sourced, delivered within the half-hour by a drone, now that you’ve got Titicaca Prime), you shower and get ready for work. But what was that dream you had last night? Only the faintest traces of it remain. A luminous rainbow, a mountain landscape, a sense of profound inner peace, mixed with euphoria, mixed with...an intense desire to go out and buy some Health Morsels cereal? That’s odd. You’ve never even had Health Morsels. You open up your laptop, where you can view the activity of the iThink microchip that you got implanted in your cortex a few years ago. “Ads help Elysium provide you with free, quality sleep-monitoring. To disable ads, please upgrade to Elysium Premium.” That explains it. After all—and you pause for a moment over whether the forthcoming thought is yours or the iThink’s, before deciding that it’s much the same anyway—Health Morsels probably are delicious. And healthy, too. As usual, they’re setting something on fire outside your home. Normally you’d scroll back through your Squawker feed to see what it was all about, but today you’re late for work, and anyway you’re in a pleasant haze after the Health Morsels dream—no point ruining it by poring over the latest Squawker pile-on. You glance at the awning. Ah, the bodega. Toilet paper and Juul Pods are cheaper in bulk from Titicaca anyway. Another day at the office. You were never exactly passionate about content moderation, but hey, it keeps the lights on. (And since ConEd got absorbed by Titicaca, it’s been pretty important to have a Verified Income Stream.) It’s a normal day at work. Algorithms haven’t yet been able to filter out all Unverified Content—so it’s up to you the flag Unacceptable Violence, Disputed Claims, and Unfederated Currency transactions. Except after your pleasant dream, it all suddenly seems so drab. You start to wonder whether there was an error last night—maybe they turned the dream levels up too high. Something from your night-time vision just won’t leave your mind, like a film strip looping over and over. It’s that river beside a sloping field, like you used to swim in when you were a kid. You can almost feel the icy cold against your skin, the sticks and mud against the soles of your feet, and then there’s the ripe, marshy scent of the riverbed. Uh oh. You realize you’re close to danger levels. This isn’t where your head’s supposed to be at. Frantically you scroll through content, seeking to moderate it. A user threatening violence against Titicaca. A stream that violates DMCA. A highly dubious Squawker claim, with no support from experts. Flagged. Flagged. Flagged. Yet the river remains. You see it, you feel it, just as you feel the kiss of the sun on your shoulders, the breeze rustling through your damp hair. Now you know what’s coming. You brace for it and wait. Three. Two. One. ZAP. Your iThink produces an electric shock: plenty painful but not permanently damaging—at least that’s what the Verified studies show. The river recedes from your mind. Following the shock, your iThink, as versatile a little machine as ever was invented, delivers a few milligrams of both amphetamine and oxycodone, bringing on a wonderful feeling of simultaneous alertness and relaxation. You look up at your monitor. Content moderation doesn’t seem so bad, with the stimulant and the barbiturate swirling paradoxically through your system. It’s true what they say in the ads: iThink doesn’t just make you think better, it makes you feel better. You dive back into your content moderation with renewed verve, even flagging a couple completely innocent posts just out of surplus exuberance. So what if you hardly get outside at all these days, so what if you’ll never swim in that stupid river. The fastest electronics, the Your Health and Wellbeing tracked and adjusted by an intelligence far greater than that which you could ever hope to possess, a gizmo that gives a whole new meaning to the term neural net. 2050 turned out to be a pretty good year after all. ]]> https://urbit.org/blog/2020-to-2021 https://urbit.org/blog/2020-to-2021 Wed, 20 Jan 2021 00:00:00 GMT 
In this post, which has been adapted from our most recent [newsletter](https://urbit.us11.list-manage.com/subscribe/post?u=972a03db9e0c6c25bb58de8c8&id=be143888d2), we'll take a moment to reflect on the events of the last year, and then turn our attention to the next one. ## 2020 In spite of the many challenges presented over the last year, it was a resoundingly good one for Urbit. It seems like a long time ago now that the folks at Tlon rolled out OS1 (now called Landscape), just as the pandemic forced everyone into their homes throughout the world. At the time, [Urbit is for Communities"](https://urbit.org/blog/urbit-is-for-communities/) was just a vision, but now it’s been transformed into reality—there are more thriving communities in Landscape than any of us can keep track of, with new faces popping onto the network on a daily basis. Landscape brought many new ships onto the network which helped identify those areas in which Urbit’s infrastructure could be most improved. [Ford Fusion](https://urbit.org/blog/ford-fusion/) eliminated the possibility of a partially-applied over-the-air (OTA) update through a drastic reduction in the size and complexity of the codebase. The efficiency of the OTA process was improved by several orders of magnitude, and many long-standing memory issues were resolved. Now we deploy code on our infrastructure nodes and see users receive an update, over a decentralized network, often in less than a minute. After improving the stability of the system Tlon turned its attention to bringing more ships onto the network. A long-standing barrier to Urbit’s adoption has been the technical knowledge required to host one’s own ship—this prompted Tlon to build and launch an Urbit [hosting service](https://tlon.io), the waitlist for which is now thousands deep. With Tlon becoming more singularly focused on building products on Urbit, concrete steps have been taken towards making urbit.org a separate organization that’s focused on long-term development of Urbit’s platform. I joined to [lead that effort](https://urbit.org/blog/first-steps-towards-urbit-org/) back in August and have since spent the majority of my time working with dozens of contributors on projects spanning documentation, education, external integrations, and much more. We rounded out the year with a full network reset (called a “continuity breach”), which has in the past has been synonymous with loss of all data on the network. This time things went differently: tooling was made available to the community that allowed ships to preserve their data, which enabled most to seamlessly rejoin post-reset. This is a significant milestone for the project as it signifies not only a commitment to technical excellence, but to supporting our budding community. ## 2021 In the last three weeks we’ve seen a huge amount of inbound interest as people are waking up to the inherent problems with using centralized services run by MEGACORPs—we’re just happy we started working on Urbit all those years ago. While we tend to not plan more than about six months in advance, I can still talk at a high level about what urbit.org will be focusing on in the immediate future. Growth in Urbit’s developer community has been growing organically in lockstep with the overall network growth. This means, happily, that there’s a pretty large pool of people eager to contribute to Urbit. The priority for urbit.org, therefore, is to improve its capacity to effectively assist contributors. In broad strokes, this means: - Rework urbit.org to better help developers learn how to contribute to Urbit - Since Urbit is ready to build on top of, provide practical guides that demonstrate how to do so - Guide developers to make high-value contributions to Urbit and its ecosystem - Build community amongst contributors My strategy to date has been heavily focused on working directly with contributors to provide technical guidance and direction, and it’s been working really well. To that end I’ll also be scaling our ability to provide such support by bringing more individuals into urbit.org that can mentor, review work, and provide guidance on where to best direct contributors' energy. 2021 is shaping up to be a big year for Urbit, and we'd [love your help](https://grants.urbit.org) in building it. If you want to see what we're working on in real time, join us in Landscape at ~wolref-podlex/foundation. See you on the network! ]]> https://urbit.org/blog/security-audit https://urbit.org/blog/security-audit Thu, 17 Dec 2020 00:00:00 GMT 
The security firm Leviathan has now finished their initial audit of the security of Urbit's Ames network protocol. In addition to their review of Ames, we also asked them to identify any other issues in order to help guide further development. I'm pleased to report they confirmed the cryptographic soundness of the protocol, namely that "a ship must have possession of their own valid Urbit key in order to authenticate as that ship and to send valid Ames messages. Additionally, no other ship except the two communicating may be able to decrypt or modify messages during transport." This means Ames uses cryptography in a mathematically sound manner that protects your privacy, keeps your secrets secret, and ensures nobody else will be able to impersonate you on the network. Since this was an architectural audit, it’s possible there are still vulnerabilities in the runtime implementation. Until the runtime has been hardened, Tlon continues to recommend against storing valuables in your ship. Aside from a couple of minor issues that have already been resolved, there are three general classes of problem highlighted in their report that future work will need to address: - Forward secrecy - Protection against denial of service attacks - Enforcement of social boundaries Ames’ design allows for versioning of the protocol, which means that the proposed changes below can be deployed as over-the-air updates as they become more pressing concerns. This feature gives us confidence that we can address future security issues as they arise. We will need to improve all of these over the next couple of years, as independent lines of work. As the network grows: better forward secrecy will decrease the damage from a key getting compromised; resistance against denial of service attacks will make the Urbit network resilient to increasingly determined saboteurs; and stricter enforcement of social boundaries will let Urbit’s social networks remain safe and friendly despite potentially increasingly large numbers of bad actors and increasingly complex app ecosystems. ## Forward Secrecy A protocol is said to have forward secrecy if compromising a key used in a recent message doesn’t also compromise all previous messages. Similarly, a protocol is said to have backward secrecy -- also called future secrecy -- if compromising a key used in a previous message fails to compromise all future messages. Ames has some degree of both, but more would be better, to decrease the damage an attacker could inflict by compromising a key. As it stands, an Urbit user can change their Ames keys whenever they want, by conducting an Ethereum transaction to update the Ames public key listed in the Azimuth contract, which implements Urbit’s PKI on Ethereum. Compromise of one Ames private key does not compromise earlier or later keys, so this has both forward and backward secrecy. The main drawback is that Ethereum transactions can be expensive, so key rotation is likely to be infrequent, and an attacker might be able to gain access to a long stretch of time by compromising a single key. Various key rotation “ratchet” schemes could be added to Ames to establish varying degrees of forward and backward secrecy, and other kinds of cryptographic protections beyond basic soundness, such as break-in recovery. Right now, changing a key through Azimuth is a manual process. It could be automated, possibly as one of a suite of services that an Urbit Provider will offer. Unfortunately, each key rotation costs the user money, so automating rotation could get prohibitively expensive. Existing protocols, such as the Signal app's encrypted messaging protocol, the Noise protocol framework, and the Wireguard VPN protocol, use ephemeral Diffie-Hellman key exchange and KDF chaining to achieve forward and backward secrecy and self-healing. Ames could be upgraded to use one of these existing ratchet algorithms, or we might create an Urbit-specific variant optimized for the Urbit network. This is work that should be done, but fortunately it will not require any novel research or drastic changes to Ames’s architecture. While Urbit’s cryptographic protections don’t yet match those of Signal or a few other privacy-focused apps, Ames is cryptographically sound, and since Urbit updates itself over-the-air and is backed by a stable, decentralized PKI, I’m confident that the level of sophistication of Urbit’s cryptography has the potential to be world-class with just a bit more work. ## Denial of Service A denial of service (DoS) attack against an Urbit ship is an attempt to overwhelm it remotely by sending it network data in such a way as to interfere with the ship’s ability to communicate with other ships or process other events -- usually by hogging its resources (computational power, memory, or bandwidth) until normal operation slows down or ceases entirely. Since the promise of Urbit is that everyone can run their own server, that server needs to be able to defend itself against DoS attacks. This is in fact one of the main impediments to running your own server in the modern world. Right now, Urbit has no protection. Soon, it will need to at least be able to resist a simple attack carried out by an individual. Eventually, it should be able to resist a determined Distributed Denial of Service (“DDoS”) attack, which is more difficult. There are two categories of DoS attacks in Urbit: authenticated and unauthenticated. An authenticated attack requires access to a ship's private key to construct valid Ames packets intended to overwhelm a target. All authenticated attacks will be handled primarily through enforcement of social boundaries, as discussed in the next section. An unauthenticated DoS attack constructs packets that eventually fail to authenticate and decrypt but still overwhelm the target by causing it to waste resources (CPU, memory, disk, bandwidth) in such a way that it can't keep up with legitimate communications. An unauthenticated DoS attack relies on the lack of Sybil resistance of IP networks: it's possible to generate new IP addresses rapidly, and a determined attacker might also be able to spoof its sender IP address. This is best considered an IP-layer attack that must be handled mostly at the IP layer itself, by maintaining state about which IP addresses can be trusted and which can't, determined empirically and heuristically by the Urbit runtime. The best any higher-level protocol, such as Ames or TCP, can do is make it as cheap as possible for a DoS target to reject a bad packet. The overall strategy that Ames can take, as a protocol layering over the IP protocol, is to add an authentication procedure to each packet that can quickly identify bad packets, which an implementation will then use to mark that IP address as suspect and to refrain from adding the sender IP to a list of known-good IPs. Currently, Ames handles this by attempting to decrypt a message and rejecting the message on failure. This may be a more costly solution than other methods, such as attaching a signature or a MAC to each message. The way to mitigate unauthenticated attacks is to make bad packets fail faster by augmenting each packet with a signature or MAC to each packet, since those are faster to validate than encryption. So far, we have identified three approaches that could be taken here. Of these three, Leviathan recommended the end-to-end HMAC solution, since that would not impose additional CPU requirements on packet-relay nodes, such as stars. - Sign each packet, and any relay or final recipient could validate the signature - Wrap each packet in an HMAC, validated by the final receiver - Wrap each packet in a HMAC for each hop in a relay chain; each relay would validate and strip the previous HMAC and add a new one for the next hop Any of these solutions would be attempts to increase the rate at which the network could discard invalid packets, with different tradeoffs. I'll back up a bit to provide some context, and then I'll compare the approaches. In any DoS solution, a planet will want to block or throttle IPs that have been sending it packets that fail to authenticate and decrypt. Ideally, a star will help protect planets for whom it’s relaying, too. Many different arrangements among galaxy, star, and planet are imaginable here, from an open policy to a policy wherein each packet has to come from a known-good IP. This sort of paranoid, closed-off arrangement might still not be enough to withstand a determined DDoS, but it's difficult for me to imagine doing better than that without launching Urbit-specific ISPs. Ames packets are cryptographically authenticated, so an Urbit ISP would actually be feasible. This brings us back to the three possible ways of mitigating DoS from within the Ames protocol. They all involve adding a piece of authenticating information to each packet that can be verified more quickly than waiting for the cyphertext to fail to decrypt — Ames's symmetric encryption is authenticated, so it's not a cryptographic soundness issue, but the performance impact of trying to decrypt could be significant enough to make it hard to prevent a DoS. As-is, an Ames packet is basically `[protocol-version checksum sndr rcvr (nullable origin) cyphertext]` (visit [this PR](https://github.com/urbit/urbit/pull/4051#issue-530369627) for a full specification). Let’s walk through the construction of the packet and how it can be upgraded to offer different DOS mitigations. ### DoS Mitigation Approach 1: Signature The first approach to DoS protection would be to add a signature to the packet. The signature would cover everything except the checksum and the origin, since those need to be swapped out by relays. The signature would probably obviate the need for the (20-bit) checksum, although we could also leave it in there if we can't figure out what else to do with those header bits. #### Advantages of a signature over other approaches - It can be validated by anyone on the network. Relays would not need to modify it. This might also give it symmetry with the upcoming remote scry protocol, which might also sign every packet. I'm not quite sure this is an important property, actually, but it might theoretically make launching an Urbit ISP or internet backbone easier one day. - It would be signed directly by the key listed on Ethereum, not something that might get ratcheted at some point for better forward secrecy — this could eventually lower the rate at which intermediate validators need their key caches invalidated, although this is a bit of a theoretical concern. #### Disadvantages of a signature over other approaches - Generating and validating a signature is significantly slower than generating and validating an HMAC. - An ECDSA signature is 64 bytes, which is ~6% of a packet payload. As far as I can tell, it can't be compressed. - Since the signature could be validated by anyone, it doesn't attest to a specific communication channel, so it might not help as much with IP filtering. ### DoS Mitigation Approach 2: End-to-End HMAC The second approach would be to use an HMAC instead of a signature, validated by the eventual receiver. This HMAC would also cover everything except the mug and origin. I'm calling this the "end-to-end HMAC" design. An HMAC requires a shared secret, for which we could use the same symmetric key that Ames uses. #### Advantages of end-to-end HMAC - It's simple in some sense; relays don't need to do anything cryptographic. - HMACs are much faster to create and validate than signatures. - HMACs, since they're hashes, can be truncated arbitrarily and still maintain some security — to find hash collisions, you'd still have to guess what packet||shared-secret would hash to; it would just be a smaller search space. Since Ames's encryption already guarantees authentication and we'd just be using this as a performance optimization, I suspect we could get away with using one that's significantly smaller than a signature — I'm not sure how small, but if 4 bytes would be enough in practice, that would take up less than 0.5% of the space in a packet. #### Disadvantages of end-to-end HMAC - Only the recipient can validate. This means the authentication couldn't be used by relays to discard invalid packets. The recipient could provide back-pressure out-of-band to a relay to ask the relay to discard packets from an IP with an invalid HMAC, but this is indirect and might need to filter backward through multiple relays to work properly. ### DoS Mitigation Approach 3: Point-to-Point HMAC The third approach, which is what I would build unless convinced otherwise, is what I'm calling "point-to-point HMAC". This design would include a different HMAC for each hop in a relay chain, instead of (or maybe in addition to) including one end-to-end HMAC. We have a couple different options for how to construct the shared secret for each hop. I would intuitively construct it using a Diffie-Hellman of my private signing key and your public signing key — the "networking keys" on Ethereum include a currently unused signing key in addition to the encryption key that Ames uses. This way even if we add more forward secrecy to Ames, changing the symmetric key used for encryption, relays wouldn't have to have their caches invalidated as often; the state required to perform relaying wouldn't be as tied to the state required to receive a message. #### Advantages of point-to-point HMAC - A star could perform some validation on every packet it's asked to relay. This makes it more difficult for an attacker to route a ton of packets to a planet through a star, or even through many relays. Signatures also have this property, but they're slower to verify. This approach and the signature approach, but not the end-to-end HMACs, force an attacker to use an (at least partially) authenticated attack in order to use a relay as a way to get to a planet. - A planet could quickly validate that a relayed packet actually came from its star, not an attacker who's managed to spoof the star's IP address. #### Disadvantages of point-to-point HMAC - Not everyone on the network can validate a packet, unlike a signature. - Each relay needs to validate and strip off the previous HMAC and create a new one before sending it. This should be quite fast, since an HMAC is just a hash, but it's still some CPU, and potentially creates an imbalance where an attacker could induce the relay to spend more CPU in validation than the attacker spent in constructing the packets. - By itself, it doesn't validate the cyphertext in an end-to-end way. I suspect we might want to include both a per-route HMAC and an end-to-end one, especially if we use truncated HMACs — although per-route might be enough on its own, since if the receiver fails to validate a bunch of packets whose HMACs were validated by the relay, the receiver can ask the relay to block that whole Urbit address regardless of IP, so it shouldn't be possible to sustain that kind of attack for long. #### An Aside About Cryptographic Performance I'm not sure exactly how much faster HMAC is. According to DJB (https://ed25519.cr.yp.to/ed25519-20110705.pdf), an ECDSA signature can be created in around 30 microseconds and verified in around 100 microseconds. This source (https://www.jacksondunstan.com/articles/3206) suggests about 21 microseconds for HMAC-SHA256, so about a 1.5x improvement in generation speed and 5x in validation speed. We should verify these numbers for ourselves before making a decision. ### DoS Protection Strategy and Timeline Tlon is not currently pursuing adding any of the extra cryptographic authentication on Ames packets described above. Since this is ultimately a performance issue, there are other more pressing performance issues that we could improve upon before decryption will become the limiting factor in how many packets can be processed by a ship. We will revisit this authentication performance question once some of the low-hanging fruit has been done, perhaps roughly a year from now. Since the audit, we’ve already improved the Ames packet format to allow faster parsing, faster validation of basic fields, and faster relaying; in the future, increases in performance of Nock execution and inter-process communication between Urbit's I/O process and Nock worker would allow Urbit to absorb somewhat more punishment before service can be effectively denied. ## Enforcement of Social Boundaries While Urbit needs to rely largely on its runtime to mitigate unauthenticated attacks, any attack conducted from an Urbit ship needs to be handled inside Arvo itself, not just by IP filtering in the runtime. Just like in any DoS, an attacker ship might try to induce you to use too much memory, CPU, or disk space in order to sabotage normal operation. An authenticated attack is harder to define than an unauthenticated attack, because it depends on what social relationship the attacker and target have, and it involves valid, authenticated Ames packets; once discovered, though, it’s easier to mitigate, because it can use Urbit’s permanent identity system to enforce social consequences, i.e. stop listening to that Urbit address and possibly encourage others to do so as well. There are a couple of different kinds of authenticated attack: attempts to exploit vulnerabilities in the Nock worker (the interpreter, allocator, and jets) by causing particular Nock code to run, and attempts to induce the target to exhaust various computational resources. Protecting against attempts to break out of the sandbox and hijack or otherwise interfere with the Unix process running Nock will require a process of hardening, fuzzing, and likely further security audits specifically evaluating the security of that boundary. This is largely out of scope of the Ames protocol itself, which was the subject of this audit. This leaves us with the latter case: resource exhaustion attacks. A peer could try to get my ship to use too much RAM, disk, CPU, or bandwidth by sending it valid Ames packets. This represents a social problem: a person on the network is trying to interfere with me. It's what's known in blockchain circles as an "attributable fault": once the attack is detected, it's trivial to determine the identity of the culprit. Once the attacker’s Urbit address is known, Urbit will fall back on the Sybil resistance of its identity system: there are a finite number of identities, each of which costs some money. Because Urbit identities are relatively permanent, they can be blocked permanently. As long as the expected value of an authenticated attack is less than the cost of the identity, the attacks won't happen very much — they'll cost money to sustain, instead of being self-perpetuating. Attacks like this are best thought of as attempts to violate social boundaries. I don't want you using up too many of my resources; that's a social boundary, and an attempt to force my ship to spend its resources on your behalf constitutes a violation of this boundary. The strategy for mitigating these attacks is to impose social and, indirectly, economic consequences on the perpetrators. The main social consequence is other ships refusing to communicate with you anymore. Some of this mitigation will take place in the Arvo kernel, and some will need to be application-specific. To protect the system from attackers, the kernel needs to be able to free space (Urbit is a single-level store, so there's no distinction between disk and RAM from Arvo's perspective) taken up by unfinished messages sent by peers. This can be done by allowing an application to ask Ames to delete partially received messages from a peer, generally replacing the received packets with an intent to nack (negative acknowledgment) to indicate that the transaction the peer wanted me to perform has been rejected. Applications should also be able to set resource usage policies that the Arvo kernel will enforce. For example, a chat application should configure Ames to reject any chat messages larger than, say, two hundred kilobytes. A file-sharing application would need a much larger per-message limit (ideally no per-message limit), but it might need to place a cap on its bandwidth usage to make sure lower-latency communications aren't swamped by a large file transfer. This kernel/application interface starts to resemble a permissioning or capabilities system, and it could be built as part of the crucial larger project of providing for configurable application permissions. To close the loop on enforcing social boundaries, reputation information should be shared across ships. This will likely be done by building one or more applications that share statistics and block operations. Some forms of reputation are likely to be globally shared, whereas others are likely to be multipolar, i.e. some people like you and some don’t. Generally, the lower the layer of the attack, the more global the reputation effect will be; the higher the layer, the more multipolar. An attempt to spam the network with millions of malicious packets will cause most people not to want to talk to you; sending me a movie I found mediocre is probably not even information worth sharing with anyone else. ## Conclusion Ames’ design has unparalleled potential to deter, mitigate, and recover from attacks, since every packet is authenticated and encrypted and backed by a stable, decentralized PKI. It feels good to have our design for encryption and authentication be confirmed by a third party as cryptographically sound, and we know more or less what we need to build to reach the full security potential of the platform, including the steps outlined here. ]]> https://urbit.org/blog/io-in-hoon https://urbit.org/blog/io-in-hoon Tue, 15 Dec 2020 00:00:00 GMT 
Let's talk about IO in Urbit. I won't say much about how events and effects are processed by the runtime; it suffices to note that the runtime is a normal Unix program which (1) listens for various events, (2) passes them to Arvo (Urbit OS), and (3) processes the effects produced by Arvo. Instead, I’ll focus on how IO works from the perspective of Hoon programs, like Arvo or apps running on Arvo. But first, let's review IO paradigms in other languages. All examples are in a made-up language, since it's useful to understand this information even before you're comfortable in Hoon. If you're learning Hoon and want a challenge, try translating them! ## The three styles of IO I will call these styles imperative IO, monadic IO, and state machines (note that this classification refers only to the experience of using them and the program structure they dictate). An imperative programming language can use any of these and a functional programming language can use any of them. ### Imperative IO Imperative languages (such as C and Python) perform IO by calling a function in the middle of whatever they were doing. Anywhere, in any statement, you could have IO. In this example and others to follow, the task will be to read a filename from the command line, read a URL out of that file, and fetch the contents at that URL. output = fetchUrl(stripWhitespace(readFile(cliInput()))) For readability, intermediate variables would be a good idea, but nothing stops you from nesting IO this way. ### Monadic IO Languages that use explicit monads for IO (such as Haskell) perform IO by running IO behind a bind operation, which I'll notate with `<-`. Thus, while you can't put IO _anywhere_, the general structure is a sequence of IO operations. fileName <- cliInput() url <- readFile(fileName) output <- fetchUrl(stripWhitespace(url)) [Tackling the Awkward Squad](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/07/mark.pdf) is the canonical description of monadic IO, and does a great job of explaining how the Haskell community decided to use this as their primary IO style. ### State machines Languages that use explicit state machines for IO (such as Elm) perform IO by accepting events and producing a list of effects. event :: Initialize | CLIInput text | FileInput text | HTTPResponse body effect :: CLIInput | ReadFile fileName | FetchUrl url | Print text main(event) { switch event { Initialize -> [CLIInput ~] CLIinput fileName -> [ReadFile fileName] FileInput url -> [FetchUrl stripWhitespace(url)] HTTPResponse body -> [Print body] } } The effects produced are not functions but data. It's important to note that this example is chosen to highlight a particular kind of IO — sequences of events. Further, this example has no persistent state — each IO operation is a pure function of the output of the previous one. ## A stateful, loopy example Let's look at another type of program: a simple HTTP server that you can give text at the CLI, which will respond to all HTTP requests with that text. Imperative: text = "no text entered yet" while (true) cli = readCli() if (cli != Null) text = cli request = readHttp() if (request != Null) respond(request,text) Monadic: loop("no text entered yet") where: loop :: text -> IO Null loop = input <- readInput() switch input CLIInput newText -> loop(newText) HTTPRequest request -> _ <- respond(request,text) loop(text) State machine: text = "no text entered yet" main(text,event) { switch event { CLIInput newText -> [newText, Null] HTTPRequest request -> [text, HTTPResponse text] } } Again, note the response in the state machine is data, not a function. I'm not trying to make a point about which type is better in general. There's a pretty strong argument that in the first example, imperative was the best option, followed by monadic, followed by state machine. There's an equally strong argument that the second example is most naturally represented by the state machine. Look at how state is handled in the monadic example. Formally, bind composes functions, so that every time you see a `<-` there's actually a new function (closure/delimited continuation) being created and stored for when we receive a response. All variables in scope are stored in that closure, so we can conveniently access them later without explicitly storing them. However, the state we preserve over time varies a lot. In this example, it varies between storing `loop`, `[loop input]`, `[loop input newText]`, and `[loop input request]`. By comparison, the state machine does not store intermediate values, it only stores its explicit state. While this is somewhat less flexible, it puts the permanent state front-and-center. I'll reiterate that all of these are possible to implement in purely functional languages, and further they can all be implemented in terms of each other. State machines are trivially implementable -- it's just a function [state event] -> [new-state (list effect)]. Monadic IO can be implemented directly (as in Haskell) or on top of state machines by storing continuations in your state. Imperative IO is somewhat more difficult but should be possible through algebraic effects. When inventing a new system, you should consider which of these you intend to support based on the sort of code you expect to see. For example, a scripting language will almost always benefit from imperative (or at least monadic) style. ## In Urbit So much for exploring the space of IO solutions. In terms of deployed code on Earth, the split is about 90% imperative, 5% monadic, and 5% state machine. In Urbit, the split is about 90% state machine and 10% monadic. This is what I intend to explain and defend. Urbit's OS is called Arvo; it has several "kernel modules" called vanes, and it has userspace applications. Colloquially these are called "apps," but when speaking about them abstractly, it's helpful to use their more specific name: ["agents"](https://urbit.org/docs/tutorials/arvo/gall/#agents). Arvo, the vanes, and agents are all structured as explicit state machines. In addition, there are ["threads"](https://urbit.org/docs/tutorials/arvo/gall/#threads), which are structured as monadic IO. Note there's only a superficial resemblance to Unix threads: they are not executed in parallel and they don't share memory, or at least not any more than agents do. They're simply a computation that takes an argument and produces a result, possibly after doing some IO, structured monadically. As you can see, all the lowest levels of Arvo are explicit state machines, and only the highest layer supports monadic IO. To see why this is the case, let's list advantages and disadvantages to each type. Note that, while I'm speaking specifically of agents and threads in the context of Urbit, these are fundamental properties of monadic and state machine IO. Agents are: - Upgradable. Since the state is explicit, you can upgrade an agent in-place by supplying a new state machine and a function from the old state type to the new state type. This is very similar to a database migration. - Robust. A state machine must accept any input at any time. If it receives unexpected input, it must have had explicit error handling for that input, since it's one of the types in the switch statement. Thus, it's relatively easy to stay in a consistent state. - Permanent. Since it's upgradable and robust, there's no reason to stop its running. - Hard to write long sequences of IO. Long sequences of IO create many possible states, and you must handle any input in any state. This can result in complex, hard-to-follow code with code paths that are rarely if ever taken. Threads are: - Not upgradable in-place. Since the state is implicit, you can only cleanly upgrade by (1) killing the thread and restarting it or (2) letting it resolve to a defined "upgradable state" periodically, usually at the top of a "main loop". If you do (2), then you are in fact structuring it as a state machine, so the other advantages of threads are compromised. - Fragile. If we don't receive expected input, the default behavior is to be stuck waiting for input. You must remember to add any error-handling features you want, and it's easy to end up in an inconsistent state. - Impermanent. Since a thread can't be upgraded and may get stuck in case of unexpected input, a thread can't be expected to last forever. It must be "rebooted" from time to time. - Easy to write long sequences of IO. Just...write the IO one line at a time. Arvo proper and the vanes are permanent pieces of software and, for this reason, state machines are more natural. I'll note that other kernels, like the Unix kernel, use imperative IO. The difference is that Unix isn't permanent in itself — it only lasts until you reboot. Since your permanent state is stored externally, you don't need to be upgradable or permanent. With sufficient discipline you can write robust C code, so it makes sense to use imperative style. Even so, a lot of C code is written in a state machine style; even if it doesn't need to be upgradable, it's worth it for the robustness and concurrency advantages. However, Arvo is a single-level store — all of its state is permanent. This is very convenient and eliminates many long-tail bugs, but it also means we need to code for permanence, and that's much easier when you structure your code with explicit state. Further, Arvo and the vanes rarely perform long sequences of IO. Generally, they do one thing and emit some effects, similar to the HTTP server example above. Sometimes they "pass through" a request. For example, the Eyre vane passes HTTP requests to userspace agents or threads. In this case, there is a sequence of IO, which in monadic code would be: loop() where: loop = request <- receiveRequest() agent = lookupAgent(request) response <- callAgent(agent, request) _ <- sendResponse(request, response) loop() This is clear code to read, but it leaves open the question of how to respond during the callAgent() function. If another request comes in, can we handle it in parallel? If we need to upgrade ourselves before the agent responds, can we make sure to properly handle the response? The naive solution in monadic code is to say that you can only handle one request at a time, and if an upgrade happens you just drop outstanding requests. You can resolve both of these by introducing a sort of main loop, which is just a way of saying "turn it into a state machine". In a state machine, the naive solution is: state :: (map request agent) main(state, event) { switch event { HTTPRequest request -> agent = lookupAgent(request) [put(state, request, agent), CallAgent agent request] AgentResponse request response -> [del(state, request), HTTPRespond request response] } } Upgrading this is trivial, since the state (map of requests to outstanding calls to agents) is explicit. It also trivially handles concurrent requests. I won't claim this is easier or harder to write than the monadic version, but it has the properties we need at this level of the system. Userspace agents similarly are permanent entities that need to not lose data or get stuck. For this reason they’re structured as state machines. A helpful comparison is that explicit state machines are basically Mealy machines, the kind you might have learned about in an introductory digital design class. This shouldn't be surprising; a digital circuit must always be consistent because it can't be manually "rebooted", and their input and output is highly formalized. By contrast, a Rube Goldberg machine doesn't have the same constraints; which is why they're a lot more fun! It's notable also that explicit state machines are considerably more declarative than imperative or monadic IO. Not everyone thinks that's a good thing, but anyone who defends functional programming should see some advantages to that. However, userspace sometimes needs to perform long sequences of IO. Let's look at an example where we want to fetch the front page of a site and the first couple comments on each story. HTTP is unreliable, so on failure we retry five times. Monadic: topStories <- fetch(topStoriesUrl) loop(topStories) where: loop :: topStories -> IO Null if topStories is Null return Null comments <- retryLoop(5,head(topStories)) otherComments <- loop(tail(topStories)) append(comments,otherComments) retryLoop :: [n story] -> IO Comments comments <- fetchComments(story) if comments is HTTPError: if n == 0: bail else: retryLoop(n-1, story) State machine: state :: Initial | FetchingTop | FetchingStory comments stories retries | Done comments main(state, event) { switch event { Initialize -> assert state == Initial [FetchingTop, Fetch topStoriesUrl] HttpResponse response -> switch state { FetchingTop -> [FetchingStory Null stories 5, FetchComments head(stories)] FetchingStory comments stories retries -> if response is HTTPError: if retries == 0: [Initial, Null] else: [FetchingStory comments stories retries-1, FetchComments head(stories)] else: comments = append(response,comments) if tail(stories) is Null: [Done comments, Null] else: [FetchingStory append(comments,response) tail(stories) 5, FetchComments head(tail(stories))] } } } With practice, the state machine version of this can be written correctly. However, (1) it's verbose, (2) the control flow jumps all over the place, and (3) in practice it's challenging to get exactly right. This is still a small example, and it gets much worse as the length and complexity of the IO sequence grows. The monadic version, on the other hand, flows cleanly down the page. We can factor out generic functionality like "retry this request n times" to make the essence of the computation clear. Suppose you have to upgrade this. In the state machine, you can see exactly which stage of the computation you're in and write specific code to upgrade cleanly. Threads, as we've discussed, can't be upgraded in-place. Let's be real though: we're downloading comment sections, not regulating a nuclear reactor. If we get a better version of this script, just kill the old one and start the new one from scratch. What's a couple extra HTTP requests? This suggests a general division of workloads: code which needs to be permanent, upgradable, and concurrent should generally be a state machine. Code with long or complex sequences of IO and definite termination should generally be a thread. These aren't completely mutually exclusive, but it's surprising how often they are. ### Jael: a case study When you need to do something permanent and upgradable, but you also need to do long sequences of IO, the main trick is to factor your problem into both an agent and a thread (or several). As a case study, consider Jael, one of our vanes. This maintains our PKI state, which it downloads over HTTP from an Ethereum node. Downloading from an Ethereum node is a complex sequence that starts with fetching its most recent block number, then fetching all the blocks we haven't seen and scanning for transactions we care about. We also maintain a map of block hashes to block number, and if we see that a block's parent doesn't have the hash we expect, that means a reorganization has occurred, so we must "rewind" one block at a time until we find where the fork happened, then restart going forward. You don't have to understand this sequence of IO; it's sufficient to see that it's complex and must be exactly right. We used to do this in a state machine directly in Jael. This data is, of course, permanent and needs to be accessible to the rest of the system, so Jael must be a state machine. However, we had many small mistakes in this sequence of IO, some of which only appeared during unusual events, such as when an HTTP request failed at the same time as a reorganization happened. We're in a bit of a bind, though: Jael has to be a state machine. The solution is to factor out the act of getting updates into a thread while the main PKI state is stored in the state machine in Jael. This thread has a definite purpose: given the most recent block number we already know about, fetch all the PKI transactions since then. The function signature is: syncEthereum oldBlockNumber -> IO [newBlockNumber, newTxs] Jael runs this thread every five minutes and, if it succeeds, then we update our block number and PKI state. If the thread fails, gets stuck, or we need to upgrade it, we just kill the old thread and start a new one. Again, a few extra HTTP requests don’t matter. From Jael's perspective, it looks something like: state :: State block txs outstandingThread main(state,event) { switch event { Initialize -> [State 0 Null Null, StartTimer (now + five minutes)] TimerFired -> effects = If outstandingThread is Null then Null else [Kill outstandingThread] newId = newThreadId() [State txs newId, append(effects, StartThread newId syncEthereum(block), StartTimer (now + five minutes))] ThreadFinished Failure -> [State block txs Null, Null] ThreadFinished Success newBlockNumber newTxs -> [State newBlockNumber append(txs,newTxs) Null, Null] } } In other words, from Jael's perspective, it's just a single IO event that encapsulates arbitrarily complex IO in the thread. A single IO event is easy enough, and we can maintain all our permanency and robustness guarantees, because we know the thread will have one of three results: success, failure, or it hangs. If we set a timeout, hang becomes failure, so there's only two possible results: success or failure. If we handle both of those correctly, we've handled all the possible IO errors that could have occurred. What about upgrading? We've already said we can handle the thread dying for arbitrary reasons, so if an upgrade comes in, we just kill it. We can upgrade Jael's state machine easily since its state is very simple. ## Recap Let's recap what we've covered: - There are three styles of IO: imperative, monadic, and explicit state machines. - Urbit supports explicit state machines (agents) and monadic IO (threads). - Explicit state machines are preferred for permanent, robust, concurrent code. - Monadic IO is preferred for short-lived code with complex IO. - If a state machine needs to perform complex IO, it should encapsulate it in a thread so that, from the perspective of the state machine, it's simple IO. Hopefully it's fairly clear why Urbit uses explicit state machines in most situations. More than anything else, our goal is to build software that can last forever. Explicit state machines are a crucial tool for that. ]]> https://urbit.org/blog/metaphase https://urbit.org/blog/metaphase Tue, 08 Dec 2020 00:00:00 GMT 
_Previous versions of this essay specified the stream deploy date as December 11, 2020; then December 18, 2020. This has been amended to the actual date of release, with our sincere apologies. For new readers, you can opt into the dev stream by running:_ ``` |ota ~hanruc-nalfus-nidsut-tomdun %kids |merge %home ~hanruc-nalfus-nidsut-tomdun %kids, =gem %take-that |merge %kids ~hanruc-nalfus-nidsut-tomdun %kids, =gem %take-that ``` _We will surface this option more broadly in a future settings overhaul._ A few weeks ago, the Landscape team — those of us building Tlon’s suite of userspace applications on top of Urbit— deployed our first over-the-air update for notifications. This is a big leap forward for us. Of course, getting notifications inside Landscape itself is a big deal, but getting a push notification on your phone, or an email digest based on activity inside Urbit is now just one step away, a weekend project for a hobby developer, not an indeterminate number of steps away. After release, we received some feedback from users on the behaviour and design of this feature; beyond the obvious kinks that come from an initial release, it seemed to some like a minimum viable affordance pitched as a fully designed feature. This “bottom-up” development approach was, to us, fairly straightforward; to some of our users, it seemed that we had closed the door on the feature without even beginning to expose all its possibilities. As I’ll describe, we haven’t just been iterating Landscape, but the way we develop Landscape, and the way we consider Landscape in relation to Arvo. Finally, we’ll discuss our upcoming two-stream release process: a “stable” release stream with updates every six weeks and a development stream, with ongoing changes continuously deployed on an opt-in basis. To someone coming to Landscape for the first time, it is starting to look mature; and it is indeed maturing quickly. After all, these mismatches in expectations are pangs calling out for growth. Landscape truly has _users_, not just explorers, who are developing the same expectations they have with the other tools they use: predictability, reliability, and iterative improvements, not just leaps forward. So to meet those expectations, we’ve been working on the prerequisite work for a mature release schedule, with two different release streams for users and explorers: a lifecycle, with the cells, interlinked, dividing into a coherent higher being. ## Landscapers and lifecycles For most of 2020, the core Landscape team has comprised of Liam (`~hastuc-dibtux`), Logan (`~tacryt-socryp`), and myself (`~haddef-sigwen`). Others at Tlon have rotated on and off, and some community members (`~radbur-sivmus` comes to mind) have also contributed a lot, but the three of us handle the core decisions and the entire stack for Landscape, working with designers, infrastructure, and third-party developers. After OS1’s release in April 2020, Landscape’s back-end has changed quite a bit; the first rearchitecture we performed was in group-store, adding roles and simplifying quite a lot of the back-end logic at the same time. Around May of this year, Logan flagged that our code surface was too large and had too much boilerplate by writing a set of bespoke full-stack applications when just one flexible userspace database would do. Our designers also had a vision of our interfaces being “modes and genres of the same data” that fit this vision as well; and so since then, we’ve put our resources into migrating all of Landscape to use graph-store (more information available in [this developer call](https://www.youtube.com/watch?v=E4DFuAZQ32Y)). Obviously, developing userspace for distributed computing has some additional challenges that you don’t have to consider when writing software for a single server. Each ship interacting has to mutually coordinate the agents running and the events to be processed. If those fall out of synchronisation, the edge cases multiply. We traditionally tended toward solving those problems by writing the simplest possible incarnation of an application or a solution first, triaging out additional complications until they can be most prudently addressed — usually when they are just one step away. For most of its history, Landscape’s userspace suite had to be up-to-date on all ships to function at all, so multiple staging streams added quite a few complications. Since incompatible versions of applications on different ships would produce mutual fail states if they attempted to interact –and since each application had no sense of how to negotiate compatibility with older applications, each user had to keep up-to-date. Landscape was moving too fast, constantly reconfiguring (to give you a sense of this: we still refer to a month ago as “old”). Historically, Arvo and Landscape were also considered just one object; changes in Landscape would often require refining the layers underneath it as well. We often aimed to use our interface’s requirements to refine Arvo in turn. But as Landscape’s cells continued to split, as it became a proper client _for_ Arvo rather than a subsection _of_ Arvo, it required its own planning and execution cycle that only came to fruition around April of this year. The first thing we did was start recurring, six-week product backlogs with designers and engineers to occasionally reassess short-term priorities with a larger roadmap. Once we did that, we began working on opening our process to the public, migrating all tasks, priorities, and current decisions to a [product board](https://github.com/urbit/landscape). By doing this, community members could both suggest features _and_ contribute features, with a centralised, transparent process for getting designer input and Tlon engineer priorities. This has also been a successful effort. By adjusting ourselves to a six week rhythm for execution, it’s clearly time to slow down our release schedule and dedicate design and QA time to each release. Since Arvo is still quite young, we can’t quite get to all the thornier aspects of instituting a lifecycle. For example, we can’t tackle agent version negotiation in full yet; and mark versioning is still not present. This means modifying the validators for our graph modules (that is, validating the graph structure of a notebook vs. a collection) presents a deployment challenge in a two-stream lifecycle. That said, our userspace infrastructure is maturing, so we could at least organise the APIs to those agents; and we could at least slow the development cycle on those agents in favour of the more flexible components of the stack. The front-end specific features, for example, can be deployed on different ships without side effects for other users. So now, before we could launch a stable stream, we had some things to do first: - **We wanted to provide assurance that the userspace suite is relatively stable.** All our remaining stores (invite-store, metadata-store, and contact-store) needed to be modernised to use the same libraries and utilities inside Arvo as newer userspace applications. - **We wanted all our applications to be using graph-store.** If we were going to migrate the entire network to a new userspace model, we shouldn’t undermine the new release cycle process and make it more difficult to test and deploy to meet higher user expectations. We needed to migrate everything first. - **We wanted to release the notification-store**. After the October 2020 groups release, Landscape simply wouldn’t be complete without it. - Finally, **we wanted all of Landscape to use the [Indigo component library](https://github.com/urbit/indigo-react).** Landscape shipped with each application using its own build with handwritten, bespoke DOM components. Once we moved to a single-page application architecture in June 2020, we wanted the same guaranteed consistency for both back-end and front-end code. Most of these are now accomplished; the remaining tasks are awaiting some final tests and UX work. We’re aiming for a stable stream of Landscape to deploy by **January 5, 2021**. What does this mean? For everyday users, you can expect Landscape releases every six weeks; the slate of features and any breaking changes will be flagged well in advance for third-party developers; and you can expect a more thoroughly considered user experience with notes and copy for each new release. And for intrepid users, you can opt-in to a development stream and help us find and eliminate bugs before release to the wider public. This development stream is much like how we've been deploying Landscape up until now — but now it's optional. ## Mitosis Describing our progress so candidly can feel strange — after all, it sounds so _early_. But to me, it’s exciting. It’s mitosis. Take another example. For all of Urbit’s history — all until now — Tlon was playing the role of both parents to the project when it was really just an embryonic corporate vehicle itself, fated to become twins. But in the past few months the burgeoning Urbit Foundation’s interim Director, Josh Lehman (`~wolref-podlex`), has begun to ramp up responsibility, taking increasing stewardship over urbit.org and giving it a physical voice, an Otherness, apart from Tlon’s own goals. Now, each day, we stray a little closer to these entities splitting apart, a step closer to the Urbit Foundation and the Tlon Corporation being completely different entities with completely different concerns. The same has occurred with Landscape. Previously Landscape was “_the_ interface to Urbit,” the proof that there was more to the project than a Unix executable. And slowly to those of us at Tlon, it became obvious what it was destined to become: a flagship client for an entire network, a tree branch rooting a world beneath clearnet, the first cell by which other clients might be born. Mitosis can be clumsy to watch; it can be nauseating to experience; it can be difficult to make out what will arrive where. But this is, after all, how a body forms. ]]> https://urbit.org/blog/security-and-continuity https://urbit.org/blog/security-and-continuity Mon, 30 Nov 2020 00:00:00 GMT https://urbit.org/blog/an-email https://urbit.org/blog/an-email Sun, 29 Nov 2020 00:00:00 GMT Date: Thu, Jul 2, 2020 at 9:50 AM Subject: Continuing To: XXXXXXXXXXXXX ``` Here are a few things I was thinking about after we chatted. It may be that I'm just reiterating things I already said — but perhaps it's useful: 1 - Our collective desire to depend on digital communication tools vastly exceeds what I would have ever guessed. Even if you somehow factor out what’s addictive about the platforms we use today: they serve a real need. The crux of the problem with our current platforms is that they don’t actually serve the need that we want them to serve. They’re not tools in the way we expect tools to be. For the most part, the platforms we depend on serve the needs of their shareholders or some graph or another that needs to go ‘up’, which makes them compromised in a very deep way. In the pre-digital world we enjoyed a few things about the way we communicated, collaborated and did business that are dwindling: simplicity, durability and ownership. I’ll try to give some straightforward examples of what I mean: A - When I talk to someone directly, I’m not worried that someone else is listening in. If they go tell someone else what I said to them, that’s something for us to work out between us — they’re not _always_ reporting our conversations back to a third party. The constraints of the interaction and the privacy implications are _simple_ — I don’t think about them at all. Good privacy guarantees flow from a strict focus on simplicity. When two people want to talk to each other: give them a secure channel to talk. Putting a centrally-controlled service between them is just an interim solution because no one could figure out a better way to do it. B - When I write something down on paper, or type into a typewriter, I have an intuitive sense of the durability of what I’ve made. I have notes on my shelf I took ten years ago. I don’t worry about them disappearing. This means I can have a very direct, high-resolution relationship with the medium. I can pour myself into my physically-written notes in a way I would never pour myself into a Google Doc. Our digital tools are deeply untrustworthy because we know they can disappear, that they won’t actually last forever. C - Around 2008 I started limiting myself to a few specific drawing and writing tools. I use them all the time and can actually refine my relationship to them since I know they’re not going to change out from under me. Perhaps at some point these pens wont be made anymore, but the ones I have are actually _mine_. Similarly, the things I make with them actually belong to me in a way that doesn’t require some ridiculous terms of service. A sense of ownership, when it comes to the medium of communication, determines our level of trust and commitment. Our digital tools have to achieve the same level of simplicity, durability and ownership that our pre-digital tools did. Until they do, we’re always going to be in conflict with them, full stop. The main thing is that exactly none of our present platform-centric world can survive when you see things this way. The digital world we live in is composed of an insanely complicated, ephemeral set of tools that distinctly _do not_ belong to the people that use them. We’re going to be in a constant struggle with technology until it conforms to these three very basic expectations. These are the expectations we try to get Urbit to fall in-line with, of course. Something else could also succeed by satisfying them. But I also don’t really see anything else, aside from Bitcoin and maybe Ethereum, falling in line with these core principles. So: while I wish that Urbit was more mature and ready to capture those that are being purged from mainstream social media, are bored with it or are frustrated with the constraints of ‘productivity software’ — I think this conflict is going to drag on until people realize that what we’re in conflict with is the structure of the software we use. Using software that’s managed by other people is always going to cause serious problems all the way up at the user level. For more speculation on how this might play out, watch Black Mirror. 2 - (1) is, in a way, a long-winded way of saying that the state of the world hasn’t changed our direction all that much. In order for Urbit to be approachable to use, it has to be very nicely made and that takes time and iteration. As the system stabilizes, Urbit can evolve in a more traditional way: we, along with others in the ecosystem, release stuff, observe how people use it, and try to make it better. We’re at the point where we’re nearly ready to get into that feedback loop, and we very much need to get there. We’re close. Urbit is a distinctly different _kind_ of software than anything we have now. Your Urbit is something you actually own and can do whatever you want with: not unlike your furniture or your kitchen tools. An Urbit is a tool for individual, human-scale computing and our whole approach to designing and making things is completely different. Most computing is about industrial-scale computing: where the software system stores everyone’s data and is managed by the people who build it. Urbit is the opposite of industrial-scale. Urbit is much more like something that’s carefully crafted and delivered to you directly. This craftsman-like approach to building software really isn’t a part of most modern software engineering. When I think about this from the standpoint of the person using what we make, it’s especially important. Urbit should feel like something that’s purposefully made by hand — not something you picked off the shelf at Wal-Mart. Some of this ethos was present in the early PC movement, but it’s mostly lost today. The care and attention that goes into building the system is also exactly what we make money on. People pay us to own a piece of it, to use it. We’re incentivized to make nice things, in the Urbit model, and we take that responsibility seriously. 3 - Networks are a really important social form that’s only just beginning to come into existence. A network is neither a state nor a company, but something in-between. Bitcoin is a network, Ethereum is network (but it’s still far too centralized). Urbit is a young network. (In my view, a real network is always decentralized. Centralized networks are basically just companies.) Urbit is controlled by the collective of people who use it — in varying degrees, of course. The main thing is that this collective is bound together only by private keys and a blockchain, not a set of legal documents. The agreements between everyone on the network are in some cases more formalized (in terms of ownership) and looser (in terms of content moderation) than what we’re used to in either a state controlled or company controlled model. This is a very unusual kind of institution. On a successful Urbit network, Facebook, Airtable, and Signal are just successful applications. You could probably build companies around them that ship software to Urbit — but they’re more like 5 - 10 people instead of thousands. Urbit doesn’t aim to eat away at the government per se. Instead, it’s an important communication tool for institutions of any scale to actually take control of their means of collaboration. The efficiency of inter-government communication and government to populace communication can clearly be improved by _a whole lot_. 4 - The most important thing is that people can generate a vision of the future. This can only emerge from a small collective of motivated people. Our public life is overwhelmed by people shooting from the hip, aiming for likes. There are difficult, infrastructural problems _all over the place_ that desperately need the considerate attention of small groups of people. Urbit, as communication tooling, is aimed at improving the quality of the fabric that binds these groups together. Or, to put it another way, before you can solve any specific problem you have to improve the problem solving process overall. Everything is gated on solving the process. 5 - Christopher Alexander is, indeed, probably the biggest influence on my thinking when it comes to paying close attention to making things that both generate a deep sense of connection and are shaped by the people who inhabit them. [http://en.bp.ntu.edu.tw/wp-content/uploads/2011/12/06-Alexander-A-city-is-not-a-tree.pdf](http://en.bp.ntu.edu.tw/wp-content/uploads/2011/12/06-Alexander-A-city-is-not-a-tree.pdf) is one of his early, well known essays that’s easy to find online. I think ‘The Nature of Order’ is the thing that had the biggest impact on me, but it’s harder to find. There’s probably a PDF floating around on the internet somewhere. It's also a beautiful (four volume) book. The first volume kind of nails it. This talk is also great: [https://twitter.com/urbit/status/1148030042846326784](https://twitter.com/urbit/status/1148030042846326784) The thing that comes to mind lately is Hejduk. He had passed away by the time I was at Cooper, but I feel like I was educated in the ghost of his work. This documentary gives a good impression of his thinking: [https://www.amazon.com/John-Hejduk-Builder-Worlds/dp/B07NKQ72C8](https://www.amazon.com/John-Hejduk-Builder-Worlds/dp/B07NKQ72C8) Hejduk was able to see that architecture isn’t about building buildings per se, but something else. That was always quite obvious to me. Talk soon! G ]]> https://urbit.org/blog/20201119-models-of-society https://urbit.org/blog/20201119-models-of-society Wed, 18 Nov 2020 00:00:00 GMT All people like to talk. Society is built by talking. In this—the year of our Lord two thousand and twenty—for many (maybe most) talking means typing on computers: handheld, laptop, or desktop. Computers can do all sorts of interesting things, like aid in designing buildings or simulate protein folding—fundamentally, a computer is a machine for doing math really fast. But computers can be networked, which turns them into tools for communication. Because society runs on talking and because computers enable us to do that, the most common thing that computers end up doing is to transmit human thoughts in text. Normal people call this writing, which is a kind of talking. Let’s call it messaging. When we think about building a good computer—and computer network—one that serves society in the best ways needs to do messaging well. Only a small percentage of people will regularly do anything else. (Urbit does serve those needs but I will leave that explanation to more capable minds.) ## Modes of Messaging
In general, technology serves humanity as a kind of appendage—instead of thinking new thoughts or acting on its own, it does what we command. Technology never creates new social structures—it merely makes existing ones easier, or it facilitates the creation of new ones. We use our tools to expand our capacities, and once we are capable of doing more, we can imagine new things to do. Inventing the wheel did not invent motorized transportation—but it did facilitate it and it was a necessary step, even an inspirational one. Similarly, the invention of writing made new forms of human interaction and new tools for organization possible. Consider the following conversational conventions, or modes of messaging: a letter; a dialogue; a book; a meeting; minutes of that meeting. These are each pieces of technology: finely constrained solutions to human problems. Now, we have digital forms of each of these which diverge variously from the original. A letter becomes an email. A dialogue becomes a chat room, or a series of blog posts, or a twitter spat. A book becomes a website or an ebook. Meetings in 2020 have virtually been replaced by their virtual counterparts. Minutes become recordings or transcripts. Although we do not have a clear picture of how humanity developed the technology of writing out of the basic capacity for speech, we can imagine that the computerization of writing has many parallels. It does not make the old tools obsolete, nor does it immediately introduce new social structures. It does create the potential, however. On the timeline of human existence, we have only had digitized text for a relatively short time. Because we want to understand how to best use these new forms, we should look at their underlying structure. In order to digitize messages well, computers need to reconcile two facts about all modes of messaging. They have: 1. One basic shape, and 2. An infinite variety of forms. 1 — The basic shape: all these modes have one common substance: human language. Any conversation is made of words. At first we only had spoken language. Then we discovered writing. Writing allows us to encode our words as glyphs to transmit them over time or distance. The internet extends this capacity. We called this "hypertext"—like good old fashioned text, but one level up. A staggering amount of messaging now takes place over this hypertext transfer protocol (http). In the process of digitizing language, we have learned the basic shape of a message. We can see this in how every social network shares messages in basically the same shape: an author, some content, references, an audience. We can confirm this basic shape by comparing it against our experience of the real world, and the previous comparison between speech and writing. Speech and writing both use the same substance, and take the same basic shape. They all have an author, content, references, and an audience. 2 — Our experience of this one basic shape takes an infinite number of forms. The examples above (letter, meeting, &c) are just a few. These forms depend on the content of the message, the people involved, and the place they occur. All three aspects contribute significantly to its form, but the one that computerized messaging has not adequately compensated for is the place. Content depends only on the author and their intent. Getting certain people to view that content might be more difficult but most know how to use a computer. The place however, has not yet been adequately compensated for, and I think that has contributed seriously to our lack of civil and serious discourse over the internet. A president would not give an address in a bar, and a prospective lover would not utter his advances on stage. In writing, we can use the different physical forms to distinguish place. You shouldn’t read a book by the pool. You might not read a letter in public. You wouldn’t put a billboard in your bedroom. Furthermore, it seems possible that in our eagerness to "connect the world," we focused injudiciously on creating communities around "interests." Evidence suggests that unless the interest has a highly refined focus, these communities devolve into pandering, self-reference, and exploitation. In this way, the internet is something like a convention of conventions, where you wander in a sea of foreigners loosely affiliated by interest and surrounded by advertising booths. In physical space, people do gather by interest but they also gather by physical proximity—they are grounded and bound in place. Having an indistinct sense of place makes the form of conversation ambiguous and the participants of the conversation disconnected. Consequently, people are unsure how to interact. One need only know the term "comments section" to understand that. ## People in Places
As the internet grew to accommodate new conversation forms using the basic substrate of hypertext, web developers began to conceive of "digital spaces" through analogy—chat "rooms" and "forums,” or the "silo" and the "walled garden." Today, most of our conversation takes place in these walled gardens—privately-owned places which may be beautiful on the inside, but inaccessible from elsewhere. While some of these walled gardens are very large, our conversation is essentially confined to them. There is no roaming from one space to another while keeping one’s identity. We create new identities for new spaces both because we must and because we are afraid of repercussions on one platform for what we say on another. This is because we do not own our identity; we merely have a key—our password. The owners of the walled garden still own the artifacts of our tenure there, and can revoke our access to them at any time. The concept of identity and the concept of place are fundamentally linked. We imagine the prince who travels overseas where no one knows his name, or we imagine the old prospect of traveling to a new world to make a fortune. While those both exemplify a dramatic change in identity and place, we usually want to keep our identity consistent. We want to be recognized by friends whichever place we go, and we all strive toward having a good reputation, which is a consistent identity across places. This consistent identity also allows control of that reputation per place. A man is not judged in public by the words he says among friends. And there is no way to search what someone has said except by asking him. With our current system of identity providers (an absurd term, but a real one), we surrender our rights to selectively expose our thoughts, regardless of the place they have been uttered. A data breach or a weak password might lay bare our most private thoughts. We run the real risk of being confronted by an old conversation presented out of context or after we have learned better. People need distinct spaces to form conversations in distinct modes, and they need to have a consistent identity that controls how public their words are. In theory, we would have a digital city with unique regions, public fora, and sovereign parcels, into which one person might travel at will. In practice, we see this pattern: an app emerges to facilitate a mode of conversation. It gains popularity. Facebook (or Google, or Microsoft) buys and absorbs it. As monolithic empires, they can use the same format for all their messages. This gives them a great amount of power. They use this power to create things like "Sign in with Facebook" (or Google, or Microsoft) that benefit the user by consolidating their identity, but fail to solve the underlying problem. Their problem is that in order to allow people to communicate using the current technology stack, they need to be in control of everything. ## Urbis ec Bit
Urbit does solve this underlying problem. At this point the explanation goes so far down the stack that you could spend years tracing it all, but the relevant part is this: Urbit is not an empire; it’s a territory. An empire is an entangling of land and rule: within the bounds of said empire, you agree to be bound by its laws and the penalties for transgression. The laws exist at the whim of a despot or cabal. Your standing as a Roman citizen buys you no favors in Persia. In contrast, a territory imposes only physical laws. For example, in the desert, you will die faster. The only way to change that is to alter the desert. Urbit imposes nothing but the digital equivalent of physical laws. You have an identity which can be said to be truly yours, and you have a piece of software that runs on whatever computer you choose and can do anything you want with the messages sent to your identity. There are no social protocols enforced, because no one can control what software you run on your computer. As long as you agree to communicate using the same basic substrate, you can communicate with anyone using Urbit. This is the digital equivalent of speaking the same language. Whether or not others will accept you into their coalitions is not the technology's concern. In the same way that the land provides clay and timber, Urbit also provides the tools and material for building places both public and private. These include the aforementioned consistent message substrate (content), as well as tools for organizing people/audiences, structure for referencing other content, and an identity system that is owned only by you while being verified by all. The places are being built—private rooms with just a few people, meeting halls, club houses, huge public spaces, and countless personal encounters, the contents of which are known to none but the participants. The default tools work with messages in a few of the ways with which we've grown familiar: chat, direct messaging, publishing, link-sharing. They map onto existing forms of messaging because as I said earlier, the technology does not create new social structures; it just facilitates them. We have only seen the beginning of what the internet is capable of doing. Within a few decades it replaced the majority of letters, phone calls, and text dissemination—that was just the first phase of getting up to parity with what written text and speech could already do. In order to get to the next phase of creating new social processes, we need our technology to be organic and democratic, in the sense that everyone can participate and shape it. It needs to have a consensual foundation—a common substrate—and infinite possibility for expression. Urbit provides both of those. Similar to working undeveloped land, it takes a lot of hard work, which is why Urbit is not ready for mainstream adoption yet, and why I alluded to the idea of settlers in my opening paragraph. Traveling to the new world is not for everyone, but everyone is welcome. _How could one do other than submit to Tlön, to the minute and vast evidence of an orderly plant? It is useless to answer that reality is also orderly. Perhaps it is, but in accordance with divine laws—I translate: inhuman laws—which we never quite grasp. Tlön is surely a labyrinth, but it is a labyrinth devised by men, a labyrinth destined to be deciphered by men._ ]]> https://urbit.org/blog/aesthetic-culture-1 https://urbit.org/blog/aesthetic-culture-1 Wed, 11 Nov 2020 00:00:00 GMT 
_This post is the first in a series of digests centered on art produced within and about Urbit by a member of the community, ~nartes-fasrum. Monthly digests will be available within the Urbit Community group in Landscape, and will occasionally be compiled here on the blog to give you a sneak peek at some of what’s happening on the network. Enjoy! ~wolref-podlex_ One of the most exciting things about Urbit is the aesthetic and design around it, developed partly by Tlon (through the design of Urbit itself) and partly by the community (by producing great Urbit art). As the community grows, it creates more art, and we have the opportunity to showcase some of the pieces we love. These works can draw curious new users in, and they give existing users ways to communicate their excitement to friends. To that end, I wrote [the Art by Urbit proposal](https://grants.urbit.org/proposals/636838408-art-by-urbit), a regular digest for the dissemination of Urbit art and propaganda. We'll bring you the works of both familiar faces and new users, in a variety of media types from image manipulation to 3D models and physical pieces. This first digest will cover some of the art produced in the past year, but will have a focus on the last two months. Future digests will be incremental updates summarizing the artworks of the prior month. ## ~datnut-pollen ~datnut- made a physical representation of their sigil from maple wood & epoxy resin, and mentioned the potential to use an Arduino to cause different lighting effects for different statuses of an urbit.  ## ~lagrev-nocfep ~lagrev- [made a thread of Dinosaur Comics](https://twitter.com/sigilante/status/1299101593997127680), using the Urbit Whitepaper as the seed text.  A sweet poem about the potential for Martian felines is theirs as well:  ## ~litmus-ritten ~litmus- produces art full of glowing gases and colourful curves:  ~litmus- also drew Urbitan for the release of OS1:  More of their work is available in the notebooks of the Internet Weirdness Research Unit. ## ~minder-folden ~minder- stylistically ranges from retrofuturistic memes to flat-out classic 50s retro:   More of ~minder-’s work is available in his The New Era notebook. ## ~naltyc-wornes ~naltyc-, & by extension ~dasfeb industries, produces content related to the exploration of Urbit on small & unique computers:   More of their work is available in the smol computers group. ## ~nartes-fasrum I use GIMP to edit images I pull off various social media platforms, and usually post the results on Twitter at [@nartesfasrum](https://twitter.com/nartesfasrum).  ## ~norsyr-torryn No discussion of Urbit art would be complete without mentioning [~norsyr-'s Canvas app](https://github.com/yosoyubik/canvas), which won first place in the [Hackathon 2020](https://urbit.org/blog/hackathon-results/) event.  ## ~radbur-sivmus ~radbur- writes poetry on a range of subjects, one of which could be Urbit. Interpretation of ["A Weaver's Regard"](https://t.fust.us/writing/poetry/weavers-regard) and ["Internet Absurdities"](https://t.fust.us/writing/poetry/internet-absurdities) is left as an exercise to the reader. ## ~sitful-hatred ~sitful- has a Martian Meme Depository chat in the Networked Subject group, which catalogs a number of the Urbit memes that have been created so far.  ## ~tocrex-holpen ~tocrex- has been drawing the galaxies of Urbit, and uploading results to Twitter at [@tocrexholpen](https://twitter.com/tocrexholpen). Each galaxy has a unique design and outfit, with the galactic sigil proudly displayed as its face.  ## Landscape Something that's become quickly evident with the addition of background images to Landscape is that people love to customize their ship's appearance and make it feel more like home. To close things out, we're providing a pair of backgrounds — one darker, one lighter — for use on your ship (or your desktop, should you so desire).   ]]> https://urbit.org/blog/20200929-state-of-urbit https://urbit.org/blog/20200929-state-of-urbit Mon, 28 Sep 2020 00:00:00 GMT 
When we announced OS 1, in April, we started to disappear into Urbit. Since then, we’ve been living on Urbit like we never have before. This has been our goal for years: to mature the system we’re building by becoming its first committed users. We knew that critical step toward Urbit’s maturity was being able to depend on it ourselves, but the system itself had to mature to the point that we could do that practically. We’ve done this in fits and starts. We’ve _tried_ and failed a few times over the years. We’ve always _used_ Urbit to some extent, but we’ve never _relied_ on it. These past six months have been different. I can’t tell whether it was out of sheer determination and commitment, or that the system was just genuinely ‘there’. Either way, both Tlon and the Urbit Community genuinely live on Urbit now. In a way, our ability to use Urbit day to day doesn’t make sense. When we shipped OS 1, it was full of holes. There were serious performance, stability, and reliability issues. Plenty of people showed up and thought, ‘I don’t understand why you think this is useful.’ But, it was good enough that we wanted to use it. And since we wanted to use it, we put our energy into making it better like we hadn’t ever before. Being able to use a system that you can really trust is a wonderful feeling. It’s Computing Without Compromises. All of the sudden, it’s Fall. We never found the time to name any of our releases or write announcements. We’ve just been making the system better in every way we can find. We’ve got a lot of exciting things planned for the last few months of the year — but I wanted to recap the progress we’ve made for those following along from the outside. If you haven’t been on the network, it’d be impossible to see. The tl;dr is that Urbit is much faster, more reliable, and nicer to use than it was six months ago. Things have gotten much better, across the board. We threw ourselves into OS 1 admittedly underprepared, but we’ve learned a lot and matured the system a lot along the way. Anyway, let’s walk through the details. We’ll talk about our infrastructure and interface work independently, since that’s how we organize things internally. Then we’ll talk a bit about what’s going on on the network. ### Infrastructure  In April, Urbit was slow, fragile, and didn’t always update itself properly. We made enormous progress on all of these. First off, Urbit is about 10x faster for most users. With some careful debugging, we were able to locate a few really [high](https://github.com/urbit/urbit/pull/3029) [impact](https://github.com/urbit/urbit/pull/3054) [improvements](https://twitter.com/pcmonk/status/1276327036722704385). There’s still performance work to do, but the effects of this work are widespread. Boot times are faster, local compute is faster, the network is faster. It’s a huge improvement. Second, we got our memory usage under control. Urbit still uses a 2GB memory arena. If the state of your Urbit grows to over 2GB things, uh, fall apart. This used to happen regularly, and now it’s pretty much unheard of. We implemented [tooling](https://github.com/urbit/urbit/pull/3235) to compact and deduplicate the memory image when it’s growing too large. We brought the memory usage of the compiler [down ~90%](https://github.com/urbit/urbit/pull/3041). There are areas where we can be more disciplined about how we use memory, but this is more or less a solved problem for the near term. Third, we implemented a completely new build system. Well, we actually [got rid of the build system](https://urbit.org/blog/ford-fusion/) altogether and [made it a part of the filesystem](https://github.com/urbit/urbit/pull/3060). This made it possible for us to fix our over the air (OTA) updates and make them both fast and reliable. In April, an OTA could bring down your node and would take hours or even days to apply. Today, we regularly deploy OTAs with no fear of them breaking any nodes — they often go from galaxy → planet in a minute or less. We also rewrote a large part of our runtime in Haskell and made overall progress on [network scaling](https://github.com/urbit/urbit/pull/3174). And: we implemented a [principled approach](https://github.com/urbit/urbit/pull/2366) to [error handling](https://github.com/urbit/urbit/pull/3064), making nodes much less likely to become unrecoverable. The network is stable, fast, and safe from day to day. We’re pretty confident it can grow quite a bit without serious issues. When we shipped OS 1 we really weren’t sure what we could handle. Now we’re standing on much more solid ground. ### Interface  When we shipped OS 1 it was a bunch of separate pages (that were slow to load), all with separate ways of storing and processing data. Our interface work over the past six months is all about unification and simplification. First, we pulled OS 1 together as a [single page app](https://github.com/urbit/urbit/pull/3025). Using OS 1 involves a good amount of moving around and refreshing the page each time was pretty painful. Now it’s snappy to move between pages and keep up with what’s going on. In tandem, we started using our own component library, [Indigo](https://github.com/urbit/indigo-react). This makes frontend development much cleaner and more efficient. Next, we unified the way we store data and compute. It used to be that each individual module (chat, publish, and links) used their own ‘agents’ to store and process data on the Urbit side. (An agent is the Urbit equivalent of a datastore + business logic.) With a lot of moving parts, this approach caused all kinds of synchronization challenges. Rather than a module-per-agent model we’ve started to move toward a type-of-data-per-agent model. The first of these is [graph store](https://github.com/urbit/urbit/pull/3110). We’re presently in the process of migrating all our modules to graph store. Those two are by far the most significant, but there’s so much more. We [rewrote](https://github.com/urbit/urbit/pull/2937) the `group-store`, allowing for different roles (admin, moderators, and so on) in a group. We shipped better debugging tools, automatic reconnection, a settings panel, and overall visual polish and refinement.  And I almost forgot [Leap](https://github.com/urbit/urbit/pull/3231): an omnibox for navigating around OS 1 with the keyboard. We have plans to make Leap much more powerful — but it’s a great first start toward making OS 1 keyboard accessible. I almost exclusively use [Leap](https://upload.wikimedia.org/wikipedia/commons/0/06/Canon_Cat_keyboard.jpg) to get around OS 1 these days. ### Community + network  Our primary goal with OS 1 was to get both Tlon and the Urbit Community living entirely on Urbit. We did that, and in doing so we’ve seen how nice it is to grow a community in a calm, non-invasive environment. We started doing regular [Urbit](https://twitter.com/urbit/status/1308875082312364048) [Dialogues](https://twitter.com/urbit/status/1304139629948329985) with friends and peers in the absence of physical meetups, which have been fun. Getting onto Urbit is still relatively challenging. By the standards of software you run yourself, it’s not so bad at all. But by comparison to ordinary consumer software, it’s way too hard. Tlon is [working on that](https://tlon.io) — but these last six months have been only for the brave. That is, those that want to host their own nodes. Turns out, there are plenty of people who are up for running Urbit themselves and quite a few small communities have sprouted around the network. Overall the network has a fun, cozy feeling. It’s nice. ### What’s next We’re still more or less flat-out, full speed ahead into an even better Urbit-world. The seeds of change were planted when we made Josh the [interim technical director of urbit.org](https://urbit.org/blog/first-steps-towards-urbit-org/). Urbit itself is getting close to the point where the separation between Urbit and Tlon has to become more real. Urbit is aimed at being an ubiquitous platform, Tlon is aimed at making it as usable and useful as possible. We’ll talk about some of this on [October 9th](https://www.meetup.com/urbit-sf/events/273439025/), and you can see a bit of what’s coming on [tlon.io](https://tlon.io) today. If you tried booting Urbit in April and it was tough, try again. We’re easy to find on the network, and we’d love to get reacquainted. ]]> https://urbit.org/blog/2019-10-3-roadmap https://urbit.org/blog/2019-10-3-roadmap Wed, 02 Oct 2019 00:00:00 GMT https://urbit.org/blog/2019-5-roadmap https://urbit.org/blog/2019-5-roadmap Wed, 15 May 2019 00:00:00 GMT https://urbit.org/blog/2021-11-18-the-promise-and-paradox-of-decentralization https://urbit.org/blog/2021-11-18-the-promise-and-paradox-of-decentralization Invalid Date https://urbit.org/updates/2021-05-05-update https://urbit.org/updates/2021-05-05-update Tue, 04 May 2021 00:00:00 GMT https://urbit.org/blog/community-spotlight-the-portico https://urbit.org/blog/community-spotlight-the-portico Mon, 03 May 2021 00:00:00 GMT ). Thus we have the explanation of all normativity, according to Orthodox Christianity. In moral matters we must take on the holiness of Christ. In epistemic matters we aspire to the wisdom and knowledge of Christ, which includes an understanding of the whole created order. Socially we (the community of believers, i.e. the Church) are called to be in ever greater communion with God, cooperating with His work just as the Son cooperates with the Father. Sadly we are conceived in sin and ignorant; hence the need for redemption by Christ. Accordingly, my understanding of what it means to be rational is just to be Christ-like. The only perfect exemplar of reason, so construed, is Christ Himself. Any deviation from Him is irrational. What could be more logical than to be conformed to the Logos? But we cannot attain rationality without redemption. Hence we cannot become rational without first being saved from sin. Doubtlessly many will find all of this too much to take, just too implausible. What about the development of mathematics, or the advances of the sciences? These are canonically ennobling, intellectual achievements. Jesus wasn’t a mathematician or a scientist! Nowhere in the Scriptures is He recorded as having applied Bayes’ theorem. Undoubtedly my construal of rationality de-emphasizes the pursuit of knowledge by labeling it as merely one of several normative ends. To seek knowledge without seeking holiness is to be understood as dangerously unbalanced. To try to attain holiness and knowledge without communion with God is arrogant. On the other hand, we need not exaggerate matters. St. Paul affirms the need for specialization of labor within the Church (1 Cor. 12). There is room for some to specialize in intellectual endeavors, so long as they act in a cooperative setting in which the other virtues are pursued as well. (Nor should anyone neglect the development of such virtues in himself.) Mathematicians and physicists are more than welcome in the Church. According to this (all too briefly put) theory of rationality, there can be no conflict between ‘logic’ and theology. Theology is compatible with rationality because the account of rationality in question derives from a theological tradition. Naturally I’ve only given a sketch; much more could be said. But even if some readers are put off by the theory presented here, I hope it’s at least clear that there are conflicting notions of rationality. Anyone who wants to be rational confronts the task of figuring out which theory of rationality is correct. **In terms of community moderation, The Portico follows norms that are quite different than other online communities, often times quite drastically. For instance, living with women that you are not married to, is not something that should be talked about lightly. Can you talk about the importance of norms and how they relate to a healthy community?** Orthodox Christians are not commanded to “get along” with the rest of the world (John 15:18-25). We’re different, and our norms of interaction should likewise differ from those of outsiders. There is no one, monolithic “Orthodox culture”, but there are standards governing which local expressions of culture are legitimate and which are sinful. We should live in a way that enables and promotes holiness and discourages sin. We have rules that may seem unusual from the outside. I don’t permit profanity, lewd jokes, or irreverent comments about God. In your question you mention a living arrangement that is prevalent in the world. Another rule I have is that we shouldn’t talk causally about sinful practices or lifestyles as if they are unremarkable parts of ordinary life. If Orthodox Christianity is true then such things aren’t minor peccadilloes. They’re spiritually destructive. The marriage-bond sanctifies activities that would otherwise be sinful. If someone wants to analyze what is destructive about men and women living together before marriage, we’re open to that. But if you want to be affirmed in your sin, we’re not the place for you. Anyone of any religious creed is welcome to show up—but as a guest in an Orthodox Christian community. Hosts should be courteous to guests, but my first priority is not to make you feel good about your choices. We’re here to encourage each other, learn, and grow spiritually as Orthodox Christians. Naturally that means a little gate-keeping may sometimes be necessary if someone ever comes along who doesn’t share those goals. Being different is difficult and takes work. In a way the cultural contrast is meant to be jarring. It can force you to confront moral and cultural presuppositions that you didn’t even know you had. When you become aware of the wider choices of culture available to you, you become free to look with a critical eye on the culture you’ve hitherto taken for granted. I think anyone could see how that might be liberating, even before considering the particulars of the cultures in question. Of course, as an Orthodox Christian, I think we have a good case to make about the particulars of our subculture. I’ve never had to kick or ban anyone from the group. The group is relatively small (~180 people at the moment) so that probably helps. The vast majority of conversations are relaxed and friendly. Occasionally there are intense debates about one issue or another but I enjoy it, as long as it doesn’t become bitter or the primary mode of discourse. But everything ought to be done in a spirit of truth-seeking, not of one-upmanship. **The fact that The Portico can exist side-by-side with atheist communities on Urbit is pretty striking. On Urbit, a user is able to create niche communities and governance structures and yet still participate in a larger network. In other words, Urbit is trying to thread the needle between sameness and difference through federation at multiple levels. It seems to me that modern culture has become homogeneous due to an inability to maintain, or distinguish between, different communities. People who have nothing in common are forced to engage as if they were roommates, while those who are similar have trouble creating their own home. In other words, what is the role of community formation in the modern world and how does it connect to your critique of liberalism? Does Urbit have anything to say about this?** As I said in an earlier response, I see the liberal project as more or less an attempt to sidestep (not solve) difficult metaphysical and religious disputes between various distinct moral communities. Liberals have tried several methods of governance to achieve their ends. We can divide such attempts into two broad categories: centralized and decentralized. Social media companies in the US confront similar governance challenges, so we can analyze them in similar terms. A social network like Facebook takes a centralized approach. Zuckerberg spoke early and often of his desire to connect people and “bring the world closer together”. There was a sense that if people from various communities could get to know each other a little better, we would see how much we have in common and make progress toward some kind of universal cosmopolitan consensus. But that didn’t happen. With centralized social media you’re practically guaranteed to interact with people whose moral and metaphysical commitments are sharply at odds with your own. As a result, you face a persistent, nagging risk of conflict. Without warning, a friendly disagreement can explode into an all-out flame war. Most choose one of two basic approaches for dealing with this threat: (1) You live a compartmentalized life. You keep your moral and metaphysical commitments to yourself and cultivate a culturally homogenized persona for your social media activities. No one disagrees with you because there’s nothing to disagree with. (2) You can enlist to fight in the culture war and display your regimental colors with pride. And because you take your smartphone everywhere you’re always on the front line. These aren’t the only options, of course. With some combination of charisma, patience, and grace you can be relatively open about your beliefs without getting into too much trouble. But even if you have the talent to pull it off, it takes time and energy. Many of us aren’t willing to go to all that trouble. Social media is supposed to serve us, not the other way around. Urbit is decentralized which, in my opinion, means that it isn’t really a network in the way that Facebook is. I see Urbit as more of a network platform. The users create the networks, or migrate them in from elsewhere. We’re all using Landscape and running our VMs on Nock, so obviously there can be interaction between different communities. But anyone who doesn’t want that cross-community interaction doesn’t have to have it. I’ve followed Urbit’s progress since 2016, so I’m plugged into groups with other Urbit people I’ve come to know over the years. Other than that, however, I don’t know much about who’s out there. It’s not like The Portico is in one neighborhood and the atheists are in the next neighborhood over. There’s a real sense in which we’re not even on the same network—different Urbit communities share a common protocol, but a protocol isn’t a network. Some people may want to see a little of everything. If atheists want to join our group they’re welcome to do so! We’ll be friendly. I don’t seek out their groups, though. One virtue of decentralized social media is that it offers a totally different set of trade-offs with respect to how you share and act on your beliefs. What if you could talk earnestly about philosophy and religion without having to worry about alienating many of your friends and family? I guess that’s somewhat possible on Facebook—do they still have “groups”?—but Urbit was designed for it. Having said all that, I don’t think decentralization is a panacea. Even if Urbit takes off, it will likely face problems that are difficult to anticipate now. My hope is simply that Urbit’s trade-offs are healthier than the ones we’ve had before. ]]> https://urbit.org/updates/2021-04-01-update https://urbit.org/updates/2021-04-01-update Wed, 31 Mar 2021 00:00:00 GMT https://urbit.org/blog/eliza https://urbit.org/blog/eliza Mon, 22 Feb 2021 00:00:00 GMT 
## The Storm before the Calm Building things, even Calm™ things, makes noise. George Nakashima’s wooden furniture evokes peace, but its construction still begins with a chainsaw. We mention this because we’re firing up our newest power tool. We call her Eliza. She is a data-collection robot. Hang on. An AI? Harvesting my data? Spying on me? Flooding me with spam? This is Urbit, right? Let us tell you about our new friend in basic terms, where MEGACORP might otherwise deploy legalese. ## What Eliza is and isn’t > Replicants are like any other machine - they're either a benefit or a hazard. If they're a benefit, it's not my problem. > > Rick Deckard Let's start with the benefits. Eliza is a simple chatbot. She was made to collect feedback on Landscape. She asks direct questions about peoples' usage of Urbit, records any responses, and reports them to her managers at Tlon. She'll also provide timely advice and information to ships kind enough to chat with her. She'll begin reaching out to ships via DM in the coming month. If you don't want to, just say no, and you'll never hear from her again. Finding bugs is critical to Urbit's rapid development, especially in this period of nascency. Most Urbit-related frustrations go unreported; regular folks don't think to open up Github when they experience a minor issue. That's why it's incumbent upon us to design a lower-friction forum for feedback. People who aren't comfortable filing an issue might be willing to talk to a friendly chatbot. Now, about hazards. By design, there are few. Eliza runs on Tlon's comet, not on your planet. There will be no OTA to install `%eliza-bot-store` on everyone's ships. Eliza learns things through conversation, not hidden surveillance software. Data she requests, like the text output of our network analytic tool, `+tally`, must be sent manually. MEGACORP tends to cut this corner; to refine their products, they just track your reactions in real-time. Since she's [open-source](https://github.com/urbit/urbit/blob/b0c1f83287cbf4c85847a9238d9f4be17e1fcb33/pkg/arvo/app/eliza.hoon), anyone can creatively deploy Eliza. Just write her a dialogue tree. Of course, that also means we can't keep people from misusing Urbit bots. We think automated agents on the Urbit network are inevitable, so let's create a good baseline culture around them. (Much love to Tendiebot, who's serving up stock prices in `~tomdys/wall-star-bets`.) However, once Urbit is used world-wide, we can't rely on culture alone. ## A New Internet Economics The sum of Urbit's architectural design is a new online economics: a way to finance and monetize access to internet infrastructure where people retain maximum equity. Explicitly acquire a low-maintenance, low-cost server. Use it to compute explicit things. Return to life. We all know the basic MEGACORP business model: offer a free service (to suckers who don't own a server), secretly mine the data they generate on the (addictive) platform, then sell targeted ads to advertisers. This model will not survive Urbit. The ubiquitous personal server changes the economic equation. You used to 'pay' for an experience with your time and data. On Urbit, once you purchase core internet infrastructure, it's yours. So, can Urbit meaningfully curtail our world's appetite for data? We'll see. Post-enlightenment society runs on instrumental reason, right down to the last byte. Marketers will always want a fresh dataset. However, when the goods are on your Urbit, you can at least demand the respect you deserve. Long term, we expect a transparent market for data, not an opaque "agreement". An analytics company wants to track everything you see and do on Urbit? They better be paying well. Eliza doesn't pay respondents. Her collection is a charity to Urbit's developers. (We're also still busy enabling payments on Urbit with the Bitcoin wallet.) However, her presence is not a term of service either. In a few years, by the nth time some smart aleck uses Eliza's chatbot code and conjures 'Zalexa' to attempt an intrusion upon your Calm™, we will see the economics of Urbit's architecture suddenly vivify. Blacklists, reputation systems, and, eventually, even data markets, all Urbit primitives envisioned long ago, will become real. We won't need Zoogle to underwrite a spam filter for the entire internet; the scarcity of Urbit address space, in tandem with a true web-of-trust, handles that. If Urbit is to be used worldwide, the old system will not go down without a fight. Spammers, spies, and digital robber-barons will hunt for ways to game Urbit. Our passage out of the digital gutter is beginning in earnest, and we are thrilled to put our new architectures to the test. ]]> https://urbit.org/updates/2021-02-19-update https://urbit.org/updates/2021-02-19-update Thu, 18 Feb 2021 00:00:00 GMT https://urbit.org/updates/2021-01-15-update https://urbit.org/updates/2021-01-15-update Thu, 14 Jan 2021 00:00:00 GMT https://urbit.org/blog/events-series https://urbit.org/blog/events-series Thu, 29 Oct 2020 00:00:00 GMT 
_TL;DR: Urbit is launching a series of community events, the first of which is a developer call taking place on Thursday, November 5 at 9am PT. [Sign up here](https://www.meetup.com/urbit-sf/events/274279522/), and read on for all of the details._ -- If you’re familiar with the project, you’ve probably noticed that Urbit folks talk a lot about the fact that [Urbit is for communities](https://urbit.org/blog/urbit-is-for-communities/). This is central to what Urbiteers believe: Urbit is a platform that any group of people should be able to use to build, communicate, and transact in precisely the way that makes sense to them. Today, Urbit is already capable of facilitating many of these activities, and it’s already a big part of the daily workflow for many people. I think we’re off to a pretty good start, and over time Urbit will become accessible to more communities. Something we tend to talk less about is the fact that Urbit itself is also a community. If you’ve spent any amount of time hanging out in Landscape in the Urbit Community group, you already know this—but getting to know the community takes time and effort. One of the reasons you don’t hear about the Urbit community as much is that community is difficult to pin down: you'd be hard-pressed to put "the community" into a room together. It's a weird, fuzzy notion and doesn't have clear boundaries. Other parts of Urbit do have clear boundaries. If you decided to learn more about Urbit, chances are you started by reading the docs. You might learn about [Nock](https://urbit.org/docs/tutorials/nock/), [Hoon](https://urbit.org/docs/glossary/hoon/), and [Arvo](https://urbit.org/docs/glossary/arvo/). You might get an [Urbit ID](https://urbit.org/understanding-urbit/urbit-id/) and [launch your own ship](https://urbit.org/getting-started/). These things are all pretty well documented and, while not necessarily easy, their respective processes are relatively straightforward and [getting easier](https://urbit.org/blog/hosting-the-future/) every day. Community is less well documented. It’s more difficult to document—it’s something you just have to experience. We lead with technology because it’s what we’re good at and because it’s easier to write about. But the social side of Urbit is no less important. Since so many of us are working on and contributing to Urbit in order to solve our own problems, and since we’re dogfooding as we go, you could define Urbit as the minimal technological infrastructure needed to sustain the Urbit community. It’s circular, but it’s as good a definition of Urbit as you’re likely to find. For this reason, you can’t really understand Urbit without some degree of exposure to the cultural knowledge of its community. When the Urbit community did fit in a single room together, transmitting cultural knowledge was easy and happened over pizza and beer. As the number and diversity of Urbit stakeholders continue to grow, we need to find ways to scale the transmission of this cultural knowledge. One way that seems to work well is getting everyone together once in a while for events. Over the last few years, I and others in the community have collectively attended hundreds of community events, virtually and in person around the world, from small developer chats to massive, global developer conferences with thousands of attendees. Urbit has even hosted [a few of our own](https://www.meetup.com/urbit-sf/). As a newcomer to the Urbit community I recently found myself seeking exposure to cultural knowledge. I know from experience attending and organizing events in other communities that regular events are a good way of making this knowledge more accessible, and of helping newcomers like myself onboard. I applied for [a grant](https://grants.urbit.org/proposals/1751024973) to bootstrap a new event series, and I’m excited to announce that this series is launching next week with the support of [urbit.org and the grants program](https://urbit.org/blog/first-steps-towards-urbit-org/). The event series has three components. The first is a series of developer calls. These will be shorter and more narrowly focused on the technologies that underpin Urbit, from the top of the stack to the very bottom. They’re an opportunity for the people developing and contributing to Urbit to share their knowledge and expertise, including things that typically don’t make it into the docs, such as how and why Urbit works the way it does. It’s recently become much easier to build applications on top of Urbit, so I hope these events will help developers see what sort of things they can already use Urbit to build, and how to get started. These events are also an opportunity for Urbit contributors to share real-time updates that don’t make it into this blog, and for the community to get to know the contributors (and one another!). The first developer call will take place on Thursday, November 5 at 9am PT, and you can [sign up to attend here](https://www.meetup.com/urbit-sf/events/274279522/). The second call will take place two weeks later on Thursday, November 19, and we’ll continue holding these calls about every two weeks. The second component is a larger Town Hall event to take place on Thursday, December 17. The Town Hall will feature multiple speakers and will cover topics including technology but also design, community, and our overall progress towards our vision for Urbit. You can think of it as part show and tell, part “State of the Network,” part meet-and-greet. The third component is a larger conference, tentatively called UrbitCon, to take place early next year. This event is still in a conceptual phase. It’ll be an opportunity for the global Urbit community to celebrate how far the project has come, to share technical as well as cultural knowledge, and, maybe most importantly, to hang out together and have some fun as well. Watch this space for more information on the Town Hall and UrbitCon. Collectively, these events allow the community to achieve several goals. Firstly, they’re a great way for community members to meet one another, exchange cultural knowledge, and share what they’re working on. Events provide excellent motivation to get something that’s a work-in-progress to a state where you can share it with others! Secondly, the content from these events will be recorded and published, and will form the cornerstone of a new knowledge base on urbit.org that will augment existing documentation. This will make it much easier for newcomers, even those that can’t attend the events live, to benefit from the wealth of knowledge and expertise of the Urbit community. Finally, these events are a great opportunity for community members to contribute things other than code. While Tlon will provide speakers for the first few events of the series, it’s important to emphasize that the initiative is entirely community-led and that we’ll need lots of help to organize and host events. If you’re interested in helping out with any of the events in the series, please find me and others helping out with events on Landscape in the Urbit Community group (the channels called “Events Series”). They say that a project’s community has reached critical mass when new things are happening that even the project’s earliest and most active contributors aren’t aware of. By this standard, the Urbit community reached critical mass sometime in the past few months, as even core team members tell me they’re regularly surprised to learn of another new project or initiative launched by a community member. Tlon could no longer catalog the Urbit community even if it wanted to. This is a good thing. As Tlon [has also said](https://urbit.org/blog/hosting-the-future/), it’s natural that Tlon and Urbit drift apart a bit over time. Just as Urbit itself is self-hosted, so the community, over time, should become self-hosting: it should play a more active role in shaping the future of the project. A community-run event series can be an important catalyst for this trend. I’m looking forward to the first event, and I hope to see you there, and at many events to come. -- _Sign up to attend [the first event](https://www.meetup.com/urbit-sf/), and join the channels called “Events Series” on Landscape in the Urbit Community group to get involved._ ]]> https://urbit.org/updates/2020-10-29-update https://urbit.org/updates/2020-10-29-update Wed, 28 Oct 2020 00:00:00 GMT https://urbit.org/updates/2020-12-01-update https://urbit.org/updates/2020-12-01-update Wed, 28 Oct 2020 00:00:00 GMT https://urbit.org/blog/hosting-the-future https://urbit.org/blog/hosting-the-future Wed, 30 Sep 2020 00:00:00 GMT 
Urbit users tend to be interesting, creative, and capable people; we’re genuinely impressed with the communities that have formed on the network, and with the quality of discourse there. But for the number of communities to continue to grow, getting on the network has to become dramatically easier. It’s not realistic to expect most people to configure their own Urbit OS node in the command line. At the moment, the biggest lever we have here is to help ordinary users get online, so they can participate in communities inside Urbit, and eventually form their own. This means hosting their Urbit nodes and showing them around the network when they first arrive. There are already several businesses overlapping with this space to varying degrees: [planet.market](https://planet.market/), [urth.systems](https://urth.systems/), and [geturbitid](https://www.geturbitid.com/). Now, Tlon is releasing its own hosting offering. For the uninitiated, “Hosting” just means that we run your urbit ship for you, making it easier for you to get onto the network. We’ve been testing our hosting service with friends and family for the past few months, and now we’re ready to bring it to the rest of the world. To learn more about Tlon hosting and sign up for the waiting list, check out the new [tlon.io](https://tlon.io/) website. We are officially launching the service on October 9th, with a virtual event and demo; we’ll also show off some of the new features and improvements to Landscape, the Urbit web client. To be very clear, hosting isn’t for everyone; if you’re deeply security-conscious or privacy-oriented, you’ll still want to self-host so that nobody else needs to touch your keys. That said, hosting isn’t a walled garden; you’ll get a planet when you sign up for hosting, which you will own 100% and forever. You’ll also always have the option to roll up your data and take it with you if you choose to stop using our service. Your Urbit data is just a single file, and we’ll happily hand it back to you if you ever decide to leave. Nothing will ever stop you from self-hosting. We’ve talked for a while about how the ultimate destiny of Urbit is to escape the control of Tlon. For Urbit to really succeed, it has to be managed by its own community. We started Tlon in order to solidify the foundation of this young platform. Now that Urbit is starting to grow up, Tlon and Urbit will naturally drift apart a bit. Urbit is meant to be universal, ubiquitous infrastructure. Tlon exists to make that infrastructure useful and usable. We expect Urbit, as a platform, to be cared for by the community in collaboration with the emerging [Urbit Foundation](https://urbit.org/blog/first-steps-towards-urbit-org/). We intend Tlon to become increasingly focussed on providing a great user experience for anyone interested in this new network, so the network can continue to grow and thrive. The way we see it, hosting is the most important thing, next to Landscape, that Tlon can do to help Urbit continue toward widespread adoption. We don’t expect to be the only ones providing this service—and look forward to including more contributions from the community here on urbit.org as they continue to mature. If you’re curious to keep up with what Tlon is doing here, head to [tlon.io](https://tlon.io) and join the waiting list. We’ll be posting updates to that list as well as to the Tlon Transmissions notebook in the Urbit Community group. Going forward we’ll do our best to keep urbit.org neutral and focussed on the technology itself. See you all on the network! ]]> https://urbit.org/updates/2020-09-30-update https://urbit.org/updates/2020-09-30-update Tue, 29 Sep 2020 00:00:00 GMT https://urbit.org/blog/gifts-q3-2020 https://urbit.org/blog/gifts-q3-2020 Tue, 22 Sep 2020 00:00:00 GMT
Twice a year we distribute address space to those that have made valuable contributions to Urbit. Now called our _Gifts_ program, the gifting of address space has been part of Urbit long before we had a grants program. I'm happy to announce that urbit.org has a record number of gifts to distribute to community contributors for the period from February to August of 2020. Because the gifting of address space is an old tradition, and the last several months have seen an unprecedented amount of user growth, let’s talk about why we do this. ## Why gifts? One way to think about Urbit is as _infrastructure for powering digital communities._ We’ve talked about this before in [some detail](https://urbit.org/blog/urbit-is-for-communities/), so while I won’t rehash the “why” of that here, I’ll reiterate that what we really want is a digital world that's generated by the communities that inhabit it. We know that digital communities need better technology with which to generate their world; just as importantly though, they need _people_ to inhabit and develop them. Tlon and urbit.org can’t bring the world into Urbit alone—we need community members to help us. By distributing address space to a variety of people that can invite their friends and families, the network can grow faster, more organically, and with a wider range of diversity. Currently, the majority of available address space is owned by Tlon and urbit.org, but it won’t stay that way. Instead, we want to give address space to folks that want Urbit to succeed. Because we’re custodians of something valuable it’s important that we are intentional about how we handle it. We treat that responsibility seriously. Gifts are one of the ways we invest in the future of the network. We look for others that see Urbit’s value and engage in building the digital home that they want to live in. They ease the learning curve for people new to Urbit, contribute to the technology, and create thriving communities. ## Recipients Gifts are being awarded for a variety of reasons this quarter. As usual, there have been plenty of code and documentation contributions, but this time around we’re also recognizing those who have evangelized Urbit, been active in guiding newcomers to the system, and developed thriving communities within Landscape. Gifts are being awarded using the following criteria: **Bronze**: one star for continuous contributions over the last six months in at least one of the above areas. **Silver**: two stars for creation of something exceptional and/or contributing across many categories. **Gold**: three stars awarded to one contributor who has contributed at a volume that indicates a high degree of care and ownership. Without further ado, here are the recipients: ### Gold [`~radbur-sivmus`](https://github.com/tylershuster) has more or less become an unofficial member of the interface team by tackling an absolutely [immense](https://github.com/urbit/urbit/pulls?q=is%3Apr+author%3Atylershuster+) amount of Landscape-related work. ### Silver - [`~littel-wolfur`](https://github.com/ryjm): Jake maintains the premier development group in Urbit, The Forge, and has led the way for other developers by authoring [srrs](https://github.com/ryjm/srrs), one of the more popular community-developed applications on Urbit. - [`~botter-nidnul`](https://github.com/botter-nidnul): `~botter` has been instrumental in furthering [Urbit on ARM](https://botter-nidnul.github.io/), has made contributions across Vere and our documentation, and been highly active in onboarding newcomers on our Discord server. - [`~timluc-miptev`](https://github.com/timlucmiptev): `~timluc` is highly engaged in many groups and the maintainer of some, wrote the [Nock for Everyday Coders](https://blog.timlucmiptev.space/part1.html) guide, and has gone far above and beyond in teaching the community about developing Landscape applications. - [`~lanrus-rinfep`](https://github.com/dclelland): Daniel has done extensive development on iOS by writing Swift libraries for interacting with Urbit, and employed all of these towards the development of an [iOS client for chat](https://github.com/dclelland/UrsusChat). - [`~minder-folden`](https://twitter.com/the_pritchard): Jonathan is a prolific member of the community. His group _The Sanctorium_ is always active, he's always around to help newcomers to the system, is an active evangelist on Twitter, and a creator of much excellent content. ### Bronze - [`~watter-parter`](https://github.com/lukechampine): Luke has made a number of [contributions](https://github.com/urbit/urbit/pulls?q=is%3Apr+is%3Aclosed+author%3Alukechampine) to Arvo, and wrote a great [introductory guide](https://github.com/lukechampine/rote) to developing Gall agents. - [`~naltyc-wornes`](https://github.com/JohnELester): `~naltyc` runs the popular group “Smol Computers” both inside and outside of Urbit where discussions ensue about various forms of computing on small devices like Android and ARM, and actively evangelizes the project. - [`~nartes-fasrum`](https://twitter.com/nartesfasrum): `~nartes` evangelizes Urbit on Twitter by creating artwork, and maintains the popular food-centric group “Urbytes.” - [`~fonnyx-nopmer`](https://twitter.com/fonnyx_nopmer): `~fonnyx` maintains an active presence throughout many groups on Urbit and runs the first Urbit-centric podcast, [Hooked on fonnyx](https://www.twitch.tv/fonnyx). - `~libhut-samwes`: `~libhut` started and maintains one of the most popular groups on Urbit centered on crypto and investing, and contributes to conversations throughout Urbit. - `~salmus-master`: In addition to embodying the quality of high-level, respectful debate we like to see, `~salmus` created the group: “The Tribal Network,” where he maintains one of the most active long-form notebooks on Urbit. - [`~fostyr-solfyr`](https://github.com/cmarcelo): `~fostyr` has submitted a number of PRs against Arvo, including a much-requested feature to reset the web login code. - [`~sarpen-laplux`](https://github.com/xiphiness): `~sarpen` has done a significant amount of research and work on Ethereum ABI parsing and other Ethereum-related topics in addition to being an active participant in dialogues throughout Urbit. - `~mister-todteg`: `~mister` is a top evangelist of Urbit on Twitter, has brought many newcomers onto the network, and has even made several hires for his company from within the network. Thank you to all who contributed to making Urbit what it is today! ]]> https://urbit.org/blog/providers https://urbit.org/blog/providers Mon, 17 Aug 2020 00:00:00 GMT
We look forward to a future where your Urbit is an always-on, always-available service. You should be able to pick up any device and conjure up your whole digital life immediately. Sounds good—but how does this actually come to pass? Urbit has to be technically robust and have a nice interface. Assuming we can deliver both of those things, your Urbit node also has to be safe, secure, and running somewhere with a network connection. Urbit has been getting nicer to use and more stable every day. We use it constantly, both as a company (Tlon) and for the broader Urbit community. From what we can tell, people are using Urbit to start communities of their own here and there. The problem is: to use our platform, you’ve got to run it yourself. This limits our audience _a lot_. Setting up a VPS and running an Urbit OS node is pretty easy if you’re comfortable in the command line, which makes it completely impossible for most people. We hear from people regularly who want to be able to sign up for Urbit like it was any cloud service. What they want is what we’ve been calling a _provider_: a service to provide easy onboarding and hosting. Given a good provider, we’d be pretty close to our long-term vision: Urbit would feel like a ubiquitous, easily accessible service. We’ve always assumed that providers would have to come into existence sooner or later. By the look of it, that time is now. Tlon and a few others have provider-like services in the works. We’ll talk about the specifics of what Tlon is planning to do in a future post. In this one, we’ll talk about what a provider is, why anyone would start one, and give a few examples of what they might look like. ## What is a provider? A provider is anyone (company, DAO, a group of friends) that provides easy onboarding and hosting. From a user perspective, signing up with a provider should be as simple as picking a planet and hosting plan, paying, and getting connected to a running Urbit OS node. This should be able to be done entirely in a browser or a standalone app and feel more or less like signing up for a conventional service. From a technical perspective, this means a provider needs to create an Ethereum wallet for the user (client-side, of course), deliver an Urbit ID to it, and boot an Urbit OS node for them. And, at some point in the process, collect payment. As a user, your provider is somewhere between your ISP and your WordPress host. Your provider both runs the star that sponsors your planet (and provides peer discovery) as well as potentially providing both access to communities and even provider-specific software. We don’t really expect providers to be terribly generic—quite the opposite. Providers are likely to differentiate themselves on price, available software, and available communities. We’ll get into some examples later. The utility that a provider offers involves quite a lot of trust on behalf of the user. While the provider never actually holds the keys to your Urbit ID, they do run your node and, in turn, can access its contents. Unlike giant cloud services, your data isn’t in an easily searchable database. Inspecting your data should be something explicitly protected by a good provider ToS. If a strongly worded ToS isn’t enough for you, you can always host yourself. And, given the way Urbit works, downloading your Urbit from a provider and running it elsewhere should be both really simple and something a provider will _always_ let you do. It’s perfectly fine to run Urbit in your closet—which means that providers are really competing on ease of use and price. Let’s move on and talk a bit about why starting a provider makes sense to begin with. ## Why would anyone start a provider? Our aim is to make Urbit OS so obviously nice to use that it’s well worth paying for. Today, a planet goes for about $10–$20 and, once you buy one, you’ve got to figure out how to really put it to use. Which takes some work. Any large holder of address space should immediately see the opportunity here: the more useful Urbit is, the more value can be captured by starting a provider. Providers make it possible to actually capture the value in address space, both by selling it and by operating nodes. Let’s say the base cost of running an Urbit OS node, in a hosted environment, is $10 / node / month. If you can sell hosting for $200 / year, you’re making back the ‘cost’ of the planet when someone signs up. Given that it’s more desirable to buy a planet when you know the star is online and you can put it to good use, this seems like a pretty good proposition: both for the provider and the customer. Starting a provider doesn’t _have_ to be done by address-space holders. You could start a provider with no address space of your own by custodying the address space of others. Then, you could take a cut of the address space sales and all of the hosting revenue. With a market of people looking to use Urbit, the crux of success with this approach is simply whether you can compete on user experience. Providers are, in a way, like the miners of Urbit. They’re incentivized to run fleets of very specific VMs for profit. The profit motive, in Urbit’s case, is just a bit more conventional than a bitcoin miner. An Urbit provider is just offering a service in exchange for money. ## What are some examples of providers? If you think of a provider simply as a business, its most basic incentive is to attract as many customers as possible. You can do that by competing on price or performance, or by competing on aesthetics and community—there are lots of ways that providers can compete. As a roughly similar ecosystem, the world of webhosting is a great comparison. There are lots of small webhosts that all have very specific services: Dreamhost, MediaTemple, prgmr.com, and Linode all come to mind. Then there are the more generic, large-scale, performance-oriented options: AWS, GCP, Azure. It’s not quite time for an AWS-scale Urbit provider, but perhaps the AWS of Urbit will start as something much smaller. Let’s walk through a few examples to get a feel for what’s possible: Communities - A provider could easily curate community content like news, events, and discussion that it only delivers to its customers. Think of it as a bit like the days of sharing a server among friends—only in this case it could be hundreds or thousands of people who form a collective to run infrastructure and share information. Custom software - There are so many ways that providers can offer custom software. Urbit isn’t ready for third party software distribution. When it is, subscribers to a specific provider could get unique modules and applications synced directly to their Urbit nodes along with their membership. Aside from Urbit-specific software, providers could provide custom software services to their subscribers. A straightforward example of this could come in the form of Bitcoin or Ethereum nodes for processing transactions. Sign up with a provider and get the ability to send transactions to the chain quickly and reliably. Security - Let’s say our base cost of $10 / month / planet from above is correct (it’s probably high, but it’s a simple number). Would someone pay $100 / month for a planet hosted in a bunker or in the pylon of an abandoned oil rig? Maybe. ## What about Tlon? Before there can be an ecosystem of providers, there has to be _one_ provider. While we’ve heard of a few other people working on things, we expect Tlon to be the first to try to really get into the provider business in earnest. I’ll save the details of what we’re going to do for a future post. Launching Tlon as a separate business will be the beginning of the separation of Tlon and Urbit. I’m really quite excited about it. Tlon is only able to consider becoming a business of its own because Urbit is mature enough that we can actually do it. For now, I’ll just say that Tlon will pick the low-hanging fruit first. Our goal is to make onboarding a group into Urbit incredibly simple. We want people to be able to go from email invite to running Urbit in a few minutes without ever knowing anything about Ethereum, Unix, or what a private key is. Once we’ve laid the groundwork with something that _just works_, we plan to open-source some of our components. Scaling up Urbit isn’t going to be easy. We hope that others will help us, and will help explore what’s possible in supporting Urbit users in the future. ]]> https://urbit.org/blog/first-steps-towards-urbit-org https://urbit.org/blog/first-steps-towards-urbit-org Tue, 11 Aug 2020 00:00:00 GMT ## Summary It’s been a long time since we last discussed separating the [governance of urbit.org](https://urbit.org/blog/governance-of-urbit/) from Tlon. In the last few weeks, we’ve taken steps to make such a separation real. In this update we’ll discuss the background that led us here, why we’re doing this now, and what this means for Urbit. ## Background At present Urbit is an open-source project developed primarily by the company Tlon. Urbit is an ambitious project with a long roadmap, which has required both careful guidance and resources to sustain its development. These resources have been Urbit’s address space, which have value in direct proportion to the value of Urbit as a platform. Naturally, Tlon has been focused thus far on making Urbit into a platform that people want to use. Ultimately, Urbit presents more possibilities than any one organization can manage. We anticipated this, and set aside address space for urbit.org, which we’ve often thought of as some type of software foundation. As Urbit matures to the point where Tlon can shift from developing the platform to developing products _on_ the platform, urbit.org is meant to continue to improve and maintain the parts of Urbit that many will come to rely upon. When we last discussed this, in early 2019, we weren’t quite ready to pull the trigger on setting up urbit.org. We weren’t sure if a software foundation would be the right structure, didn’t have the resources to tackle the overhead, and Urbit wasn’t yet stable enough for Tlon to focus on anything _but_ platform development. ## The Present Urbit is really starting to look like a platform. Tlon has put tons of work into making Urbit stable and performant, we’ve launched OS1, and hosting services are on the horizon. Meanwhile, the grants program and Hoon School were initiated, and community members have been [contributing](https://github.com/natareo/hooncard/blob/master/hooncard.pdf) [learning](https://github.com/timlucmiptev/gall-guide/blob/master/guide-docs/overview.md) [materials](https://github.com/lukechampine/rote/blob/master/urbit/app/rote.hoon), [issuing](https://github.com/urbit/urbit/pull/2885) [patches](https://github.com/urbit/urbit/pull/3202) [to](https://github.com/urbit/urbit/pull/3238) [Urbit](https://github.com/urbit/urbit/pull/2867), and [building](https://github.com/ryjm/srrs) [new](https://github.com/dclelland/UrsusChat) [applications](https://github.com/yosoyubik/canvas) [in](https://github.com/yosoyubik/urbitcoin) [userspace](https://github.com/taalhavras/ucal). With a stable platform taking shape and a strong community forming that wants to help build Urbit, it’s time to make urbit.org real. We’ve had plenty of time to think about what it should look like, how it should work, and who should lead it. Our plan is to run urbit.org as a protocol—we’ll treat it like a real entity, but instead of dealing with the administrative details of setting up such an organization, we’ll incubate it within Tlon and give it a mandate: To competently distribute address space as a means of improving Urbit. ## Moving Forward On that note, allow me to introduce myself: I’m Josh[^1], the new Interim Director of urbit.org. I’ve been a follower of this project for several years now and more recently a fairly active community member. I first became fascinated by Urbit as a programmer that saw the potential in a clean-slate OS; I later became compelled by a future in which social computing becomes small, friendly, and human-scale again. My background is predominantly in building software, companies, and teams—most notably [Starcity](https://starcity.com), where I am a co-founder and CTO. Starcity[^2], like Tlon, is focused on enabling communities to flourish. While the approaches couldn’t be more different, the goal of bringing people together in fundamentally human ways is shared. My goal is to leverage urbit.org to ensure Urbit’s success. I want to see a world in which Urbit is used by my friends, family, colleagues, everyone I know. To get there, much is left to build. To that end, the first objective of urbit.org is to recruit builders and give them the tools to succeed. I’ll be focusing heavily on providing technical support to developers that engage with us via Grants, outreach to the wider developer community, supporting the efforts of those that are creating educational courses and documentation, and enlarging the contribution surface by exposing it to other programming languages[^3]. The product work that Tlon is doing is crucial to Urbit’s long-term success, so I’ll also be leveraging bounties to more directly accelerate their development efforts. Landscape represents the clearest example of a cohesive vision for Urbit’s use. The existence of a polished product built on Urbit demonstrates to the world that the platform is robust and capable, and the growing community seems eager to help. You can expect more bounties very soon that provide a direct connection to the Landscape roadmap. As I’m only just getting started, I’d love to hear your thoughts. If you’re working on any grants you’ll hear from me soon, and if you have an idea that you want to pursue, I’m here to help you make it happen. You can reach me directly via Urbit as `~wolref-podlex` or by email at josh@urbit.org. I look forward to working with you. [^1]: But my friends call me ~wolref-podlex. [^2]: We’re also really into the idea of being the first builder of housing on Mars. Just saying... [^3]: We call this project Airlock. You’ll hear more about it soon. ]]> https://urbit.org/updates/2020-07-27-update https://urbit.org/updates/2020-07-27-update Sun, 26 Jul 2020 00:00:00 GMT Reminder: Hoon School begins August 1 [Hooniversity](https://hooniversity.org/) is the new home of Hoon School run by `~rabsef-bicrym` and `~risruc-habteb`. To enroll, visit hooniversity.org or email join@hooniversity.org You can also join the Hooniverse group via public channel `~/~hiddev-dannut/hooniverse`and follow them on Twitter @hooniversity1 That’s all for now! ]]> https://urbit.org/blog/ford-fusion https://urbit.org/blog/ford-fusion Tue, 14 Jul 2020 00:00:00 GMT
## Overview and Rationale Ford Fusion was an overhaul of Urbit's over-the-air upgrade process and a rewrite of its build system. The new update system corrects a few long-standing bugs with the previous one, and the new build system is simpler, smaller (by around 5,000 lines), and easier to manage. Since deployment of Ford Fusion to the livenet in late June, over-the-air updates (OTAs) have been much smoother. Before Ford Fusion, it was common for an OTA to take several hours, use too much memory, and leave ships in inconsistent states. After Ford Fusion, multiple OTAs have been pushed out, including kernelspace changes, and most users didn't even notice. Urbit has always been able to update itself OTA, but this process has often been rocky. Updating an operating system kernel on-the-fly is a difficult problem in general, like performing heart replacement surgery on yourself while running a marathon. Code that allows Linux to update its kernel in this way became a startup called Ksplice, won multiple awards, and sold to Oracle. Even that, as impressive as it is, and as brilliant as its [programmers](https://nelhage.com) are, can only perform certain limited kinds of patches to the kernel. Urbit isn't exactly a traditional operating system, so the comparison is somewhat unfair, but the purpose of better architecture is to create unfair comparisons. In this case, because the Nock layer is frozen, upgrading everything above that layer is easier. Upgrades are also facilitated by pure-functional semantics, transactional event processing, a type system oriented toward concrete data, and orthogonal persistence. These features make it feasible for Urbit to upgrade itself in the general case, not just some special cases. Ford Fusion has fixed the major upgrade issues of the past by guaranteeing three properties that in retrospect are obvious requirements, but, like much of Urbit, took many years and rewrites to identify as such: - Atomic: the update should complete or fail in one transaction. If it fails, the system shouldn't get stuck. - Self-contained: there must be no implicit dependencies or hysteresis (dependence on previous system states) when building the new software from source. - Ordered: updates must be monotonically sequenced from the system's lowest layer to highest. Let’s walk through them one by one. ### Atomic In previous versions of Urbit, updates failed atomicity by deferring parts of the update to later events, which are separate transactions that can fail independently. Generally, each deferral causes an exponential increase in the number of failure states that needed to be handled. We've learned that asynchronicity is an entropic state. A system will tend toward more asynchronicity over time unless effort is put into keeping it synchronous. [As Jonathan Blow has noted](https://youtu.be/pW-SOdj4Kkk?t=2547), the language server protocol has turned every editor plugin into a distributed system, since now it has to communicate asynchronously with the main editor process. Consider an update system that took multiple Arvo events to complete. An ad-hoc higher-level transaction system would need to be built to roll back the effects of the first few events in case of failure. It’s important for various parts of the system to be able to emit effects on upgrade; since those effects would need to be rolled back if a later event in this upgrade fails, the system would need to maintain a queue of those events and only apply them once all the other upgrade events have completed. Note that the asynchronicity has now spread. Some effects that would normally be guaranteed to be processed synchronously might now be asynchronous. Entropy has begun to take hold, chipping away at the set of invariants the system is capable of guaranteeing. This observation is not purely theoretical. False modularity was the cause of internal asynchronicity in Clay where it had to wait for responses in a complex dance with Ford, which was another vane (Arvo kernel module); moving Ford into Clay allowed function calls that were synchronous from Clay’s perspective, which allowed further simplifications, culminating in about a twenty percent reduction of source code size of the Arvo kernelspace. [Steve Yegge's "platform rant"](https://gist.github.com/chitchcock/1281611) describes a Bezosian edict prohibiting synchronous communication among modules through direct linking. This can be seen as an acknowledgment of the difficulty Amazon was going to have when it needed to turn internal services external. If your software needs to run in hell, build it that way from the start. An Urbit ship is not an enterprise SaaS product and does not need to run in this hell; it exists for just one person, with natural pressure pushing it in the opposite direction from Amazon’s web services. Instead of needing a ship’s state and functionality sharded into microservices strewn across multiple clusters, an Urbit instance is easiest to manage as a single server with all its state unified into one data structure and its event log stored as one totally ordered sequence of state updates -- the antithesis of a distributed system. ### Self-Contained Before Ford Fusion, each commit to the Clay filesystem validated its files using filetypes (called “marks”) defined by files in the previous commit. This could cause bugs if the filetype definitions had changed in a backward-incompatible manner. It also meant a commit could not add both a new filetype and new files of that type; instead, you needed two commits: one to define the filetypes, and a second to add files of that new type. More theoretically, it caused history-dependence. The validated contents of files in a commit could vary based on the history of commits that led to this one. Another way source code failed to be self-contained was that it had access to symmetry-breaking information at build-time, namely ship, desk (Urbit's answer to a git branch), and (faked) date. A build recipe should be able to shared, cached, and rerun without dependence on local conditions, so user code now no longer learns its ship, desk, or date until runtime. Source code also had build-time access to Urbit's immutable global namespace, called the “scry namespace”, which the kernel makes available as an implicit argument to userspace Hoon code. The scry namespace is immutable and referentially transparent, i.e. a request must always yield the same result for all time, but if an agent asks the kernel for a resource that’s from the future, hosted on another ship, or to which that agent doesn’t have permission, the kernel will deny the request. If the kernel denies a scry request that user code made during the build process, the build system has no choice but to treat it as a nondeterministic error. Nondeterministic errors can never be fully eradicated, if for no other reason than that the user always has the option to defenestrate the machine—there's nothing deterministic about that. But we try to minimize them, and especially to minimize uncertainty as to under what conditions they might occur. No build should be killed by the absence of files outside the desk, so as of this update, user code can no longer scry at build time. Once built, userspace programs can scry if run in a context with a scry handler; a Gall agent's runtime scry requests still work just fine. ### Ordered The final kind of failure fixed by Ford Fusion was the lack of ordered layering during a software update. The most common form of this failure was that old Ford had a tendency to try to build userspace code using the previous version's standard library. This didn't work too well, unsurprisingly. Emerging from this underworld required making a number of changes to the Arvo kernel, Clay, Gall, and the procedure for kernel updates. To avoid turning into a pillar of salt, I'll skip the details of how the old system worked and instead describe the new update procedure. ## How Updates Work Now These are the layers of the stack that update themselves on the fly, from lowest to highest: - hoon: the hoon language definition and compiler, defined in `/sys/hoon/hoon` - arvo: the arvo kernel proper and related type definitions, in `/sys/arvo/hoon` - zuse: the standard library, in `/sys/zuse/hoon` - vanes: arvo kernel modules, including clay itself, in folder `/sys/vane` - userspace: apps, marks, ancillary source code like libraries, and user data An update to one layer necessitates a reload of all layers above it; e.g. a change to Zuse should trigger updates to the vanes and userspace. Conversely, an update to a higher layer should not cause a spurious reload of lower layers, which should not be affected by the change; for example, an update to just userspace should not cause any reloads of system code. Clay is responsible for enforcing the layering of updates. An update to a module is triggered when an attempt is made to commit a change to Clay that affects one or more files needed to build the module. For example, if the 'foo' agent's source, defined in `/app/foo/hoon`, imports the 'bar' library from `/lib/bar/hoon`, then a modification to `/lib/bar/hoon` triggers an update to the 'foo' agent. All vanes and userspace files depend on Zuse, which depends on the Arvo and Hoon sources, so a change to the Hoon, Arvo, or Zuse sources will trigger updates to all vanes and userspace files. When asked to perform a commit, Clay determines which layers need to be updated based on which files have changed and which modules depend on those files. For now, all running programs load their source from the `%home` desk, so only changes to `%home` trigger stateful updates. Files in other desks can be built, but not installed into the system. This might be relaxed in the future. The process of updating varies by layer. The Hoon and Zuse layers are stateless, so their newly rebuilt cores (Nock executables) must be stored (somewhere in the system’s Nock tree, in memory; remember, Urbit is a single-level store), but they have no state that would need to be migrated. The Arvo kernel, vanes, and userspace agents are all live, stateful programs, so in order to update one of those, the system must extract the state from the old program, pass that data into the newly built program, then discard the old program and store the new one. Arvo and agent state injection routines can emit effects, but vane updates cannot. To work around this limitation, Gall has a two-phase update process. First it enters a dormant “pupal” phase that stores not running agent cores, but only the agent states that the old Gall had extracted from its agents. When Clay notifies Gall that its agents have been rebuilt, Gall “molts” back into normal functionality by loading the agent cores from Clay and then running their `+on-load` routines to inject the old state. If there's a change to files in `/sys` on the `%home` desk, Clay asks Arvo to update kernelspace. Clay sends a sequence of moves (effects) to Arvo to ask Arvo to perform any necessary updates to hoon, the kernel, Zuse, and vanes. This sequence is terminated by an extra move back to Clay itself, which will be received by the updated version of Clay after migrating its state. The rebuilt Clay can then use the newly rebuilt version of Zuse to rebuild userspace and notify clients of the update. One client is Gall, which molts when Clay notifies it. Clay triggers updates, but the Arvo kernel is responsible for performing updates to all kernelspace layers, and Gall is responsible for updating userspace agents. Agents are stored in Gall's state, but all other layers are stored directly in the Arvo core's state, so the Arvo kernel contains the routines that reload Hoon, the kernel itself, Zuse, and the vanes. The Arvo kernel reloads itself by receiving its new source code as a `%lyra` move from Clay, compiling this source into a core with empty state, then calling the new core's `+load` routine with the relevant parts of the old state. The state passed to the new Arvo now includes not just the vane cores and their states, but also the Arvo "duct" call `stack`, which maintains a stack of queues of moves to be passed from one vane to another, and a list of effects to emit to Unix at the end of the current Arvo event. If needed, Arvo could migrate the outstanding moves themselves -- if, say, the duct datatype changes. Passing the Arvo call stack state to new Arvo allows a kernel update to happen in the middle of a more complex event without disturbing other sequences of processing steps happening concurrently in the vanes. This entire update process happens in one Arvo event and doesn’t break event-dispatching semantics. This not only provides atomic rollback, but allows the update to be combined with other actions into a larger transaction -- for example, to stage complex changes, user code could trigger two kernel updates in a row, both in the same event. Note that this is the opposite situation from the entropically leaking asynchronicity described earlier. Now the guarantees don’t deteriorate; they can be composed into stronger guarantees. ## How Clay Validates a Desk A desk is Urbit's answer to a git branch. It's almost identical, except all files are typed and validated, and whenever a commit becomes the equivalent of git’s ‘HEAD’, it’s assigned a semantically meaningful revision number, and all files are typed and validated. If Clay has been asked to perform a commit, it needs to validate all the files in this desk and notify all subscribers to live queries of this desk's data. Gall, for example, maintains live queries on builds of its live agents. Validation uses the Ford build system, which as of this update is no longer a standalone vane but a core within Clay. A Clay commit, like a git commit, is specified as the current value of all its changed files (and, separately, references its parent commits by hash), not as the diff from a parent commit. Unlike git, Clay is typed, and every file must be validated according to its "mark". A mark is named like a file extension, e.g. `%txt`, `%png`, or `%noun`, and Clay maintains a mapping from that name to behaviors of values of that type under various operations. The last segment of any Clay path specifies the mark to use for operations on that file, including validation. Mark operations include conversion to and from other marks (such as converting `%json` to `%txt`), revision control operations (diff, patch, and merge), and validating an untyped noun. Operations for mark `%foo-bar` are defined by a core built using the source code at `/mar/foo-bar/hoon`, or if that doesn't exist, at `/mar/foo/bar/hoon`. Consider a file at `/web/foo/json`. In order to validate this file, Clay must load the mark definition core and use its validation routine to ensure the untyped value of `/web/foo/json` is in fact valid JSON. To obtain this core, Clay must build the file at `/mar/json/hoon` from source and then process the resulting raw mark core using some mild metaprogramming to get a standard interface core for dealing with marks, called a `$dais`, whose type is defined in Zuse. Since building a source file only makes sense if the file has been validated as a `%hoon` file, but mark definitions themselves must be built from source, there's a logical dependency cycle -- who validates the validators? To break this cycle, Clay hard-codes the validation of `%hoon` files. This allows mark definitions to be built from source, and in fact any file can depend on any other file of any mark as long as there are no cycles. As of Ford Fusion, Ford performs a cycle check to ensure acyclicity. Since building a file is a pure function, Clay memoizes the results of all builds, including builds of marks, mark conversions, and hoon source files. These memoization results are stored along with the desk and are used by later revisions of that desk. Future work should allow merge commits to pull memoized builds from all parents, but for now only the previous revision of the current desk is used. This is a major simplification of previous Ford architectures, which maintained much more complex caches with less clear eviction semantics. Now on every commit, we just throw away any unused memoized builds from the previous revision's Ford cache. Once Clay has validated every file in this new revision of a desk, it constructs and sends updates to any subscriptions that other vanes or agents have requested. More Ford builds may be run to fulfill these requests, including builds for any running agents whose dependencies changed in this commit. When Gall receives a newly rebuilt agent from Clay, it calls the gate produced by the `+on-load` arm of the new agent with the state extracted from the old agent. If there is a crash in any `+on-load` calls or in the handling of any effects they emit (which can include more agent activations), then the whole event crashes, canceling the commit. This effectively gives any agent the ability to abort a commit by crashing. It is a bit counterintuitive that an app reload failure could prevent a kernel update. The reason is that we don't want the system to update itself into a broken state. An Urbit can be rendered practically unusable by the presence of broken agents, even if the kernel hasn't lost integrity, so it's kinder to the user not to break their agents by installing an incompatible kernel update. This also puts virtuous pressure on kernel developers not to "break userspace", the importance of which has been insisted on for decades by Linus Torvalds, among others. If an agent does crash a commit event that included a kernel update, the attempted commit is now trivially rolled back, and the system can deliver an error message to the user. This does not leave the system in an inconsistent or stuck state, so the user could modify the failing agent and try the kernel update again later. Supporting better workflows for keeping third-party agents up-to-date will be an important aspect of Urbit’s upcoming software distribution work. ## Ford Build Semantics ### The Three Types of Ford Builds: Files, Marks, and Casts The Ford build semantics have been simplified. There are now three kinds of builds that Ford can perform: files, marks, and casts, all of which happen synchronously as function calls inside Clay and are available (without memoization) as scry interfaces. #### File Builds A file build takes in a filepath containing Ford runes and Hoon source, runs the Ford runes to perform imports, and then compiles the source, producing a `$vase`, a noun tagged with its Hoon type. Clay exposes file builds into the scry namespace with `%ca`: as an example, `.^(vase %ca /~zod/home/3/lib/sole/hoon)` will build the `sole` library. #### Mark Builds A mark build produces a `$dais` mark-interface core. It first performs a file build on the Hoon file in `/mar` that defines the mark core, then it does some metaprogramming to make the operations more convenient to use. If the raw mark core delegated revision control operations to another mark core, the mark build will also load the delegate mark core and resolve the result into the `$dais`. Clay exposes mark builds into the scry namespace with `%cb`: as an example, `.^(dais:clay %cb /~zod/home/3/mar/foo/hoon)` builds a `$dais` for the `%foo` mark. #### Cast Builds A cast build produces a `$tube`: a gate that takes a value of one mark as input and converts it to a valid value of another mark or crashes. To convert from mark `%foo` to mark `%bar`, Clay tries the following operations, in order: - direct grow from `%foo` - direct grab from `%bar` - indirect jump from `%foo` through `%qux` - indirect grab from `%bar` through `%qux` The `%foo` mark can “grow to” `%bar` by providing an arm in its `+grow` core named `+bar`. `%bar` can convert from `%foo` using a `+foo` arm in its `+grab` core. `%foo` can also chain a conversion through an intermediary using an arm in its `+jump` core, and `%bar` can specify an “indirect grab” by having a `+grab` arm produce a delegate mark instead of directly defining a conversion gate. Clay exposes cast builds into the scry namespace with `%cc`: as an example, `.^(tube:clay %cc /~zod/home/3/foo/bar)` builds a `$tube` conversion gate from `%foo` to `%bar`. ### Ford Runes There are now only four Ford runes. A file can contain zero, one, or many of each, but each Ford expression can only be one line, and they must be in the standard order of `/-`s, `/+`s, `/=`s, and then `/*`s. ``` /- foo, *bar, baz=qux ``` The `/-` rune imports a structures file from `/sur`. You can import it as just `foo`, in which case the build result of that file (usually a core with mold definitions) will be pinned into the compilation subject with the face `foo`. If you prefix it with a `*` as in `*bar`, the result will be pinned into the subject with no face; if the structures file compiled to a core, this exposes all the arms into the namespace of the compilation subject. Finally, if you import it as `baz=qux`, the `baz` face will be applied instead of `qux`. This is similar to "import as" in other languages. ``` /+ foo, *bar, baz=qux ``` The `/+` rune imports a library file from `/lib`. Aside from the different source folder, the syntax and semantics are the same as for `/-`. ``` /= clay-raw /sys/vane/clay ``` The `/=` rune imports the result of building a hoon file from a user-specified path (the second argument), wrapping it in a face specified by the first argument. The final `/hoon` at the end of the path must be omitted. This is mostly useful for importing a file for testing. The file at the specified path will be built as a normal userspace hoon file; i.e. its compilation subject will be Zuse augmented with the results of any Ford runes it has at the top of the file. ``` /* hello-gen %hoon /gen/hello/hoon ``` The `/*` rune imports the contents of a file in the desk, specified as the third argument with the full path including the trailing mark, converted to the mark specified by the second argument, and pinned into the compilation subject wrapped in the face specified by the first argument. This can be used to import static data at build-time, such as a data file, a media file, or, in the case of this example, a hoon file as source text rather than already built. A valid userspace hoon file must contain a nonempty list of hoons (hoon source expressions) below the Ford runes, separated by gap (more than one space, or at least one newline). The system wraps this list of hoons in a `=~` expression so that the result of the previous hoon is used as the subject of the next hoon. The result of the Ford runes is used as the compilation subject for this `=~` hoon; informally, the shape of the compilation subject can be thought of as: ``` :* fastar-2 fastar-1 fastis-2 fastis-1 faslus-2 faslus-1 fashep-2 fashep-1 == ``` ## Future Work Urbit still needs to make better use of desks other than `%home` and the development process should be adjusted given the tighter coupling between source code and kernel and tighter criteria for accepting an update. This work also hopefully provides a good foundation of a package management and software distribution system for Urbit. As `~wicdev-wisryt` has said, a user should be able to run `|install ~norsyr-torryn/canvas` to load and build remote source. No one should experience dependency hell on Urbit, but we're not there yet. At least now, building a desk has no dependencies, other than a Ford with a compatible Hoon compiler. No decisions have been made on this yet, but Ford might get moved to inside the desk, possibly by making Zuse callable. This could allow a desk to expose a Nock interface in addition to a typed Hoon interface, which could even let a desk be used as a "pill" bootloader. ## Conclusion `~littel-ponnys` and I spent most of 2018 rewriting Ford with the intent of improving its performance. Compared to its predecessor, its result was better in some ways but worse in others. The caching system was labyrinthine and poorly factored, making the system difficult to debug or prove correct, even informally. Some things were faster, but the caching and dependency tracking were actually complex enough that a number of common operations, like mark conversion, were too slow. In early 2020, `~master-morzod` suggested moving Ford into Clay to reduce asynchronicity. It seemed absurd at first, but at some point I realized I could combine that idea with a simpler build-caching scheme and self-contained desk builds, and `~wicdev-wisryt` realized he could use that to further simplify Clay’s commit and merge code, which he did as part of this project. The first time I rewrote Ford, it took me six months, with help from `~littel-ponnys`, and it weighed in at 6,000 lines of code. The second time, in late 2018, took a few weeks. The third time, in January 2020, took a week. I wrote `+ford` in Ford Fusion in one long day, and it’s about 500 lines of synchronous, functional code. It has taken me two or three years to understand this problem as well as I do, and I expect there are parts of it I still don’t understand. The code itself isn’t the issue; it’s finding the right answer to ontological and teleological questions. What _is_ Ford? What will it be in a hundred years? I’m confident Ford Fusion is more similar than its predecessor to the Ford of 2120, because it’s smaller, more functional, and easier to understand and administer. As an engineering discipline and organizational practice, working on a system intended to be frozen yields surprising simplifications like this every so often. Urbit is now reaching the point where we’re starting to see more of the obsidian edges of the frozen future system emerge from the lava. ]]> https://urbit.org/blog/value-of-address-space-pt3 https://urbit.org/blog/value-of-address-space-pt3 Thu, 09 Jul 2020 00:00:00 GMT  _This is the third in a series of three posts on Urbit address space. See also [part 1](https://urbit.org/blog/value-of-address-space-pt1/) and [part 2](https://urbit.org/blog/value-of-address-space-pt2/)._ It has been some time since we revisited this series of posts, so let’s start with a summary. Our framework for understanding the value of Urbit address space is that it’s useful, it’s rare, it’s easy to exchange, and the more that people use it, the more useful it becomes. Let’s briefly review each concept in a little more detail. First, utility. Urbit address space was designed for logging into Urbit OS, but Urbit ID can be used as an identity primitive for any internet service. That means it’s vastly extensible. Perhaps more importantly, because Urbit IDs at the star and galaxy level represent network infrastructure for the Urbit network, users have already begun to build a range of business models upon them, from hosting providers to community hubs. We explored several of these possibilities in detail in the [first post](https://urbit.org/blog/value-of-address-space-pt1/) in the series, and we also provide a primer on Urbit ID itself. Second, Urbit address space is finite, and that means it’s scarce. This design, which we explained in more detail in the [second post](https://urbit.org/blog/value-of-address-space-pt2/) in the series, was intended to promote good behavior and to limit spam. So far, that has worked. Scarcity is also impacted by availability of supply, which we’ll explore in this post. Third, network effects, also described in the second post in this series, have the potential to build upon themselves quite rapidly within the Urbit ecosystem. Urbit is essentially a network of other networks, each owned and controlled by their individual users. For this reason, Urbit’s network effect is best described by Reed’s law, which argues that the value of a network increases with the number of groups that can be built on the network. In Urbit’s case, this number is essentially bounded only by the amount of data that can be stored in Urbit itself, which today is 2GB. That size is already enough to support vastly more groups than anyone has tried to launch, and we plan to grow beyond that limit in the coming year. Also in Part 2, we explored trading of Urbit address space. We look at the current price of stars and planets as denominated in both BTC, and USD, and we see that the price of Urbit address space has remained fairly constant since its launch on-chain, while volume has been steadily increasing. The market is continually developing, with more ways to buy and sell Urbit address space cropping up. So, to recap: our view is that the value of Urbit address space is a function of its utility, scarcity, network effects, and ease of trade. In this post, we don’t expand that framework, but rather explore two elements that are important to actually using the framework. We’ll explain lockups, since they impact scarcity, and we’ll discuss usage metrics — including why we don’t (and in most cases can’t) track them. ## Lockups and spawning limits One way to think about galaxies, stars, and planets is that they’re different-sized buckets of Urbit IDs. A star contains 2¹⁶ (~65K) planets, a galaxy contains 2⁸ (256) stars and, in turn, 2²⁴ planets (16,777,216). Beyond this nesting pattern, there are three further considerations to note: booted versus unbooted addresses, locked versus unlocked addresses, and planet spawning limits. Booted and Unbooted Addresses – A booted address is one that has actually been used to boot Urbit OS. By convention, booted addresses are expected to have some existing reputation outside of their name alone, since they’ve been used on the network. Reputation, good and bad, comes in many forms. Did the address operate any useful infrastructure? Did it get placed on any blacklists for spam or abuse? Did it simply send and receive messages? The ability to programatically track reputation is still in its infancy, but we expect the tooling to develop as Urbit grows. Unbooted addresses, on the other hand, have no reputation. Both booted and unbooted addresses can be transferred between owners, but booted addresses will likely vary widely in price depending on the reputation each of them has acquired, and how they’ve been used. For the purposes of this discussion, we’re primarily referring to unbooted addresses, as they have no reputation, and so are more similar to one another. We anticipate that booted addresses will trade infrequently — and likely via broker or auction, since users will prefer to control the reputation of the ID they’ve developed, rather than transfer that reputation to another. Locked vs. Unlocked Addresses – At present, the majority of Urbit address space is locked by smart contracts. When Urbit address space was first registered on the Ethereum blockchain, this unlocking mechanism was put in place to prevent the network from being flooded with users. A young Urbit network in which all of the address space had been unlocked would be exposed to attacks by bots and spammers. Scarcity, on the other hand, encourages good behavior as it increases the cost to spam. A locked star is held in a contract and can neither be booted nor issue planets. The lockup contracts can be examined on [GitHub](https://github.com/urbit/azimuth). In practice, the only Urbit IDs that are ‘locked’ are stars. A galaxy that is said to be locked is simply a galaxy whose stars are currently held by a lockup contract. A locked galaxy can be transferred (i.e. sold or simply moved to a new key) and booted (importantly, any galaxy—booted or not—can vote), but it can’t issue stars until the lockup contract expires.   These lockups all began to release at varying linear rates in January 2019 (specifically on Tue Jan 08 23:34:41 2019 UTC). The various linear unlocking schedules are grouped as follows: - The urbit.org addresses, and early prize winners unlock linearly from 2019-2021. - Tlon, its founders, employees, and most purchasers from Tlon were frozen for a year, and now unlock linearly from 2020-2024. - 2018 private buyers were frozen for a year, and then began to unlock linearly over the course of one or three years, depending on the terms of their respective contracts. - Earlier private buyer addresses—including those in our first two crowdsales—were unlocked right away. As of January 2020, 17,283 stars were unlocked and tradeable. By January 2021, 33,355 will be available. On January 9, 2024, all 65,536 stars will be unlocked and available for trading. Being unlocked does not necessarily mean that a star can spawn all of its planets. For that calculation, we must turn to spawning limits. Spawning Limits – Unlocked stars are further gated by the number of planets they’re able to spawn. Every unlocked star was immediately able to spawn 1024 planets from the moment Urbit went live on the blockchain in January 2019—we wanted everyone to be able to invite some friends to their communities right away. From that date forward, the spawnable amount doubles every year (365 days), maxing out at 65,535 spawnable planets after 6 years.  When we layer the spawning limits over the various lockup contracts, we find that all Urbit address space is both unlocked and spawnable by January 9, 2025. ## Address space distribution Address space has continued to change hands since we put Urbit on-chain in January 2019. At that time, we had a fairly granular understanding of who owned what. Today we only have precise numbers for the Tlon and urbit.org owned address space. Galaxies change hands quite infrequently, so we know more about that ownership than stars and planets. We know next to nothing about the distribution of planets. We consider the opacity of our understanding about transactions on the network to be a privacy feature rather than a bug. To be specific about the information we can collect here: first, we don’t know anything about users beyond their ETH address, and second, we can’t tell the difference between a transfer of an address, and the re-keying of an address. This means we’re never entirely sure if a transfer has in fact occurred; we only know that the address has changed. The chart below is an estimate (as of January 2020) based on our general understanding of how galaxies are distributed. Much of this is anecdotal, and it’s largely based on the very few transactions we’ve observed within the galaxy table since Urbit went on chain in 2019. This chart should therefore be understood as incomplete, and an estimate only.  Address space distribution is best understood in terms of galaxies and stars independently, as they’re distinct assets. The chart above only illustrates galaxy distribution, but describes Tlon’s “naked” galaxy holdings, which are galaxies empty of their stars. These galaxies are empty because the stars they once contained have been distributed, primarily to Tlon contractors, employees, and to certain private buyers. We believe that wide distribution is important to the health of the network and the growth of Urbit as a whole. Our own distribution efforts are focused on disbursing Urbit address space ownership as broadly as possible to those interested in using and developing the network. We intend to further distribute Tlon’s address space over the coming years through private sales, employee compensation, grants, bounties, and gifts. It’s worth noting that the grants and bounties program specifically is set to increase in coming years. We’re very much interested in putting address space in the hands of people interested in helping develop the network; we see grants as the perfect mechanism for doing so. ## Usage data and composition Usage data is an obvious missing piece of the puzzle in this discussion. We understand this, and yet we don’t actively capture usage data, for two important reasons: it’s premature, and Urbit is privacy-focused by design. First, it’s too early. Urbit’s current users are a fanbase of pioneers who are willing to put up with the challenges of a young network. This fanbase grows on its own, but Tlon doesn’t actively encourage that growth yet. For the vast majority of potential users, the barriers to entry are still high enough that an invitation would risk losing them permanently. You can think of Urbit today as being a developer alpha. We’re laying the groundwork so that we can, in earnest, invite our friends and communities to a network they will love. That means we’re designing smooth onboarding, an elegant user interface, and increasing the stability of both the system and network. Tlon’s forthcoming hosting services will vastly reduce barriers to entry, and will be our first major push into the public sphere. Second, and perhaps more importantly, capturing data about traction on the network is difficult by design. Due to Urbit's privacy-forward architecture, there’s quite a lot we will never know. Once an Urbit ID is up and running, we don’t know how often it operates, where it’s hosted, or what it’s doing. This makes capturing usage metrics challenging, but we think this is the right problem to have. The Network Explorer – In the next several months we plan to roll out the first iteration of a network explorer that will allow users to visualize the structure and composition of the network as well as some basic activity data. Our first version of the network explorer will include representations of how the various layers of address space fit together, the finiteness of address space, historical uptime for nodes, and the various unlocking schedules. The network explorer will also allow users to zoom in on various nodes to learn more about them and what they offer. Urbit’s privacy-forward design makes collection of data appropriately impossible in many cases. That said, the network explorer should begin to collect all the elements of the value framework we’ve laid out here in one dynamic visualization. Closing – We have long said that Urbit address space had value because it was both useful and scarce. Taking a deeper look, we found that some limited liquidity, and network effect also play an important role in the value framework. All of the factors that make up our framework are, in turn, impacted by availability of supply and by user growth. In the Urbit ecosystem, supply is effectively gated—as described above—in order to prevent flooding of the network. User growth, on the other hand, is something we simply have not yet pursued in earnest. Instead, our development efforts have been focused on enhancing usability and stability of the platform. Our focus on user growth will begin as we roll out hosting. Once that happens, usage metrics will become both more interesting and more relevant. This concludes the final installment of our series on the value of Urbit address space. Though the explanation was long, the framework is quite simple: value in Urbit is derived from its use, scarcity, some liquidity, and network effect. Let us know if you’d like us to dig deeper into any of these elements—we’re always happy to opine about Urbit. ]]> https://urbit.org/updates/2020-06-15-update https://urbit.org/updates/2020-06-15-update Sun, 14 Jun 2020 00:00:00 GMT https://urbit.org/blog/hackathon-results https://urbit.org/blog/hackathon-results Thu, 11 Jun 2020 00:00:00 GMT
Results of the Urbit Hackathon We recently held an invite-only Urbit Hackathon for graduates of our Hoon School program, and the submissions really impressed us across the board. Submissions were judged on several criteria: creativity, usefulness, and code quality. Congratulations to all the participants, especially the winners—their projects are seriously impressive. ### First Place: Eight Stars Jose’s `~norsyr-torryn` Canvas is a peer-to-peer, collaborative drawing app that comes in the form of a Landscape tile. It’s kind of like a multiplayer MS Paint, but better. Canvas uses Urbit in a way that we think it’s supposed to be used: the stuff that doesn’t require peer-to-peer or storage heavy lifting, such as the actual “drawing” component of the app, is done outside of Urbit. In addition to a freehand template for the classic Paint experience, you can choose from a number of templates to work with, such as a grid of fillable hexagons, or maps of parts of the world and of continents with fillable country divisions. Have you ever wanted to compare which countries you visited, color-coded for how much you liked them? Canvas has a built-in feature for sharing your work on chats in Landscape. Check out a video [demonstration](https://www.youtube.com/watch?v=S6DySv730Hw) of Canvas, and view [the code](https://github.com/yosoyubik/canvas). Jose wins eight stars, and he also wins [le internet for the day](https://news.ycombinator.com/item?id=23228058). Bravo! ### Second Place: Four Stars Each Luke Champine’s (`~watter-parter`) flashcard app for Landscape, Rote, has been awarded one of two second-place prizes. Rote users write decks in udon, review them, and share them with other ships. His immaculate backend code-style and documentation secured his high placement in the Hackathon—his Hoon code also functions as a full-blown walkthrough. If you’re a Gall novice and you’re looking to learn, take a look at his [code](https://github.com/lukechampine/rote). Luke also kept a Notebook documenting his experience at `~watter-parter/hackathon`. Luke also included example decks for flashcards on Hoon runes and their irregular forms. So, all said and done, everything about Rote is an excellent resource to the eager Hoon neophyte. Pyry Kovanen (`~dinleb-rambep`) was also awarded second prize for his reading-list-sharing app, Books. Also a Landscape tile, and it’s pretty darn useful, with the ability to import books from Goodreads. Check out [the code](https://github.com/pkova/urbit/tree/books/pkg/interface/books) and the [video demo](https://youtu.be/Spy7FoPdfx8). ### Third Place: Two Stars Each N E Davis (`~lagrev-nocfep`) brought the heavy infrastructural artillery with Roessler, and won third prize. Roessler is actually three related Hoon tools: `roessler` proper, a library for solving the differential equations producing the [Roessler attractor](https://en.wikipedia.org/wiki/R%C3%B6ssler_attractor). `lazytrig`, a library for transcendental functions. `proj-3d`, a generator that takes the 3D coordinates produced by `Roessler`. The Roessler project is full of cool, novel features. You might enjoy taking a look at the [GitHub repo](https://github.com/sigilante/roessler) ...if your brain is big enough. John Franklin (`~dirwex-dosrev`) also secured third place for his Notes application. True to its name, it’s a Landscape tile that allows the user to create and search notes by keyword. Look at the [code](https://github.com/jfranklin9000/notes/) and the [demo video](https://www.youtube.com/watch?v=7DurNjCFOmA). Jon (`~ribnes-fonbex`) was awarded third prize for his Quadratic Voting project. It implements for Urbit ships a [quadratic voting system](https://en.wikipedia.org/wiki/Quadratic_voting), which is sort of like ranked-choice voting but allows people to express their stronger preferences. Congratulations to the winners and to everyone who submitted—nice work! ### The Future We were thoroughly impressed with all of the projects from this inaugural Hackathon. Not only did this Hackathon demonstrate the talent and dedication of our community members, it’s an encouraging sign about the maturity of the Urbit system. We’re going to hold more Hackathons in the future, with one big change: they will be open to anyone who applies. ]]> https://urbit.org/blog/the-missing-middle https://urbit.org/blog/the-missing-middle Mon, 25 May 2020 00:00:00 GMT
The promise of social technology is that we can discover more friends; _specifically, people more similar to us_ than we previously had access to. The means by which we do so, the methods of collaboration, the quality of connected experiences; these factors matter, but they are marginal considerations compared to the basic offering of an expanded choice of connections. The wilder-eyed, utopian end of early computing drew greatly from this promise; such unprecedented connection could bring about world peace, intellectual serendipity, higher consciousness. We didn’t get world peace, but we did get a genuine revolution in collaborative information-sharing and rapid communication. That ability to refactor our social preferences gave birth to an uncountable universe of online communities and subcultures that would be unimaginable to anyone who didn’t see it happen. We didn’t get a million friends either. Dunbar’s number, which posits an upper bound of ~150 trusted relationships per person, put the lie to dreams of a ‘global community.’ Better social tools give us more choice in who to fill those slots with, but not the ability to expand the personal spheres where we actually spend most of our time and forge most of our familiarity and trust with others. We know this, and the social networks know it. To combat a slump in user activity, Facebook jettisoned promises of boundless discovery and started promoting Groups. The success of Discord and Slack happened precisely because they promised tools for gated communities, not open commons. One of those early promises came true in spades, though: while we still live in Dunbar communities, we’ve sped up the sorting process. We can find people like us at astonishing speed, and the diversity of options for what ‘people like us’ means has expanded exponentially. The ‘Big Sort’ of modern society is rapid, unprecedented, and utterly reliant on social technology. The problem is that for all of our freedom to find each other and make mutual agreements, the underlying technologies used for all of this sorting and trusting don’t actually reflect the diversity of expectations, norms, or approaches we might want to try. One major reason for this status quo is the underlying structure of the client-server Internet. Unlike in the physical world, where cities, religious groups, and professional associations both own real assets and collectively determine rules and norms, there is no median level of civic compromise on the Internet; just massive technical providers and platform companies, and dispossessed Dunbar communities and individuals unable to determine such rules and norms for themselves. Both internal norms (‘how do we act’) and external perceptions (‘who are we?’) get yanked around by the dictates of platform owners, routing infrastructure providers, and the societal consensus that governs both, at levels far above our Dunbar phyles. Privacy is contradicted by the need to surveil internal activity in order to target ads. The current model of social connection technology sets the bar for self-determination so high as to prevent it from adapting to external circumstances. The technical realities of the situation are due to client-server technology and its institutional history. The social aspect is a holdover from the 20th-century media paradigm of mass consensus cultures. The ‘long 20th century’ of mass institutions and broad consensus realities is over. Contrary to impressions, it was not the inevitable or final result of moral and civil progress. Rather, it arose from pragmatic external incentives of mass advertising and great-power ideological rivalries. The assumption that we must inevitably play by the same rules of discourse, given the autonomy granted by wealth, technology, and knowledge, is rapidly degrading. This status quo is a leftover from a time when mass education prepared workers for mass jobs, where mass communication hammered the same messages over the factory loudspeaker and advertised the same small number of products over the few television channels available. Yet we still assume that the shape and function of the tools and spaces for dialogue we inherited from that era are immutable and inevitable; this outdated belief cuts us off from new methods of choice and collaboration. Without the external political and economic pressures which created our mass culture, inclusion in or acceptance by it becomes just another sinecure for sale, the outcome of a zero-sum auction to benefit from and utilize for further gain, the power accumulated by the legacy institutions safeguarding it. The ‘objective’ model has become a straightjacket on human ingenuity and community self-determination. The fact that it is ‘supposed to exist’ means that it is the ultimate prize. Inclusion in the mainstream Discourse grants scalable exploitation of any idea and prestige to its proponents. But it is a booby prize. No serious and nuanced idea, no dedicated community can withstand its flattening effects or the ways that it commercializes and cargo-cults its subject into an artificial shape. The resulting conversations are lackluster in form as well as subject. The stiffer the Discourse is; the less input it deems useful, the more fragile it becomes. The inside becomes a stultifying garden party of outdated, anodyne official narratives; the outside a stew of paranoid, enraged rump factions shut out of any path to respectability or good-faith engagement with their perspective. The same dynamic wreaks havoc inside Dunbar groups as well; when there are fewer gradations of acceptable difference, internal schisms take on a stark tinge of zero-sum loyalty versus betrayal, rather than consensual drift. Savvy users, knowing the stakes, migrate quickly, leaving behind wastelands of the obstinate and clueless, who are in turn preyed upon by grifters and demagogues. Only when external events force a ‘tectonic shift’ in official understanding do these standards change. The problem is that the usual result of a ‘tectonic shift’ is a sudden, destructive earthquake. If networked culture is going to survive dramatic events out in the real world, we’re going to need a middle structural layer that allows norms and agreements to shift and negotiate based on observed reality and the wishes of ordinary participants. This omission of a ‘middle layer’ of technical governance directly informs our vision for the Urbit network. In an Urbit-based world, we envision stars will be consensus hubs. There are 256 Urbit galaxies, each able to spawn 255 stars for a total of 65,280 potential stars. Each star, in turn, can spawn 65,535 planets—far above the Dunbar number of an intimate community, but also far below the ratios of authoritarian control under which current platform regimes operate. Stars and galaxies perform the essential routing and peer discovery functions of the Urbit network; planets represent distinct individual identities. Although every star is spawned by a particular galaxy, it can move to any other galaxy which agrees to host it. Similarly, planets may migrate between stars. Many imagine the extreme ramifications of such a system, projecting a Darwinian competition of starkly diverse political models encoded in governance. When we think about the future of Urbit, though, we don’t see a chaotic universe of iconoclastic outposts. We don’t doubt there will be diversity among stars (driven by the wishes of their owners and resident planet users/customers), but the real value is in the antifragility allowed by far more subtle gradations of difference in rules and structures decided upon by stars. Each star will have much, but not all, in common with its immediate neighbors, across a constellation which contains a great overall diversity of norms. Changes in policy or alignment will resemble ripples across a pond more than a buildup of tectonic force against a brittle fault. Stars are Schelling points for such gradations precisely because they are discrete cogs in the overall technical infrastructure of Urbit. Technical responsibility and any bounds of ownership will always mark such points. Their designation in the case of Urbit or any other system is necessarily somewhat arbitrary. Our vision is orders of magnitude less centralized than the status quo, but not devolved so far towards the individual that community formation demands massive amounts of _individual_ technical knowledge and maintenance tasks. There will always be a need for routing infrastructure, ergo power (the power to set community norms and external relations) will always accrue to the points of control over that infrastructure. Urbit simplifies much of this infrastructure but, just as importantly, it makes those technical locations, and the conditions of ownership and control, explicit. It allows for them to be owned and controlled by anyone who purchases them, rather than controlled by network-level overlords and unelected government regulators. Stars are somewhere between ISPs and consumer hosting platforms (like Geocities an eon ago, or Wordpress today); independent enough to set their own policies, but generally subject to the realities of running a profitable business. The decisions made by service providers inevitably have a great influence on the norms of any network, and Urbit is no different. The advantage of the Urbit model is that there are far more stars than ISPs, they are easier to run, they are fungible properties with clear ownership, and they are not burdened with technical and regulatory debt which ties them to the status quo. These qualities allow them to be purchased and maintained by a far wider range of people than might be able to control an ISP today; the massive decentralization of routing infrastructure also seems a likely antidote to the current state of regulatory overreach and centralized snooping which ISPs are subject to. The Internet gave us an unprecedented flowering of diverse cultures; it gave communities the option to pursue a vast number of paths and goals. And we _should_ have the ability, granted by wealth, free time, and communications technology, to arrange the rules of our collaboration as we wish. The structural design of the Urbit network is an effort to provide that power, encoded in a digital asset with private keys that can be held both by individuals and (via multisig arrangements) by groups. The lower price and complexity of stars compared to current ISPs also makes a lot more room for innovation in business models; star owners can make their own judgments about how to prioritize commercial revenue versus cooperative ownership, anonymity versus identity, short-term versus long-term contracts. We see stars as an opportunity to incubate community hubs, to plant a flag in the wilderness, declare values and alignments, and see who shows up. This process can be varied; some stars might advertise widely, others might quietly and slowly invite other communities on board. That’s the point. There’s a nice juxtaposition between first mover-effect and overall availability here; early adopters will have an opportunity to shape discourse, but the supply is not so constrained that they’ll capture all of it. Early adopters will likely have an immense and permanent value of voice, infrastructural ownership, and ongoing revenue over their shorter-term speculative one. The diversity and non-discoverability of Urbit are already showing themselves: we hear whisperings of insular communities that we not only _don’t_ know about but by our express intentions _can’t_ know about unless they make themselves known. Regardless of whether we or anyone else know about them, star owners already possess legible chunks of the ‘missing middle’’—large enough to affect the map of the Urbit universe, small enough to allow a wide array of voices—in a way that the ownership models of the current Internet can’t come anywhere close to providing. Not every neighborhood poker club needs an Urbit star. But the institution-founders, tastemakers, and social connectors of today and tomorrow, those who want to build opera houses on the digital prairie, should consider the advantages of building on a star. ]]> https://urbit.org/blog/tools-of-our-own https://urbit.org/blog/tools-of-our-own Tue, 12 May 2020 00:00:00 GMT
> People need only not to obtain things, they need above all the freedom to make things among which they can live, or give shape to them according to their own tastes, and to put them to use in caring for and about others. > > Ivan Illich, Tools for Conviviality One of the recent ways we've come to describe the mission of the Urbit project is to give communities [the power to shape, or evolve, their digital environments](https://urbit.org/blog/urbit-is-for-communities/). Of course, like many of our definitions, it requires some background to really contextualize the claim. When I describe this particular statement to people, they often ask, "What does that _mean_, though? What is a digital environment, and why would I need Urbit to do it?" After all, we think of our slate of tools as our entire toolbox. This makes sense for physical tools. And for a lot of people in the real world (at least, I think), their tools remain physical notebooks and paper. I, however, left those tools behind ages ago, and found myself in a world that used my habits, my memory, to loosely organize a wide array of online tools, that were themselves shaped according to the values of their creators. The [Rube Goldberg machine](https://en.wikipedia.org/wiki/Rube_Goldberg_machine) made out of aligning these tools in increasingly virtuosic, but also increasingly restrained and mechanical ways, felt like constructing an exoskeleton out of prefab parts. The person in my social group closest to constructing the full range of human movement became a legend. But let's be honest: none of us came even close. And it's nobody's distinct fault that the experience of aligning all these disparate services into a workflow is a skill in itself. It's a collective restriction that comes from the systems beneath.
### Systems as masters > If you don't understand a system you're using, you don't control it. If nobody understands the system, the system is in control. > > Philip Monk, Precepts If you asked any of the creators of the social tools we use today what their mission was, it would seem neutral, a public good. To "bring the world closer," to "make a more open and connected world," to "easily see what's happening in the world." Ignore the fact that the net effects of their products are far more mixed than their mission suggests: for many users, the tools themselves come second to the power the tools have in capturing their social graph. We just want to be where our friends are, to see who knows whom, to see a new friend vouched for by our common connections. Why is the social graph even a consideration when choosing a tool? Why do I have to know who has accounts where, how they choose to assemble their digital body, so that I can see that it exists? For a tool like a microblog—a bunch of entries in a database with metadata, usually threaded—the attachment of the social graph capture becomes not only an underlying appeal to us when choosing to use it, but also to the tool’s creators when trying to justify its existence. In this example, you could say that every social Internet product is trying to make up for its missing ability to connect coherent identities underneath. The battle becomes a matter between the stagnant empire of who owns the graph versus the various attempts to create a secure identity primitive as a layer of the Internet. But this is crazy: for decades, an aspect of the Internet stack that necessitated the creation of platforms—the ability to talk to other computers directly and to, privately, opt in, organize an array of devices as belonging to an identity—has driven not only the financial incentives of every product, but the use case for every person in the equation. A practically unrelated problem emerged that was too hard to wrangle and solve, and so it became the master. And for productivity tools—for the personal activities we do every day—the same problem ultimately applies. Either you wrangle more and more single-use "products" into a framework, wrangling combinatorial explosion, or you punt the problem to the user's brain, remembering the relation between tools. This is why hybrid products are born _and_ why massive enterprises are shifting toward subscription models for office software. With the [clearnet](), using our own computers for everyday tasks that extend outside themselves is not easy. It's way easier to sign into an enterprise's server farm for your persistent data storage and manipulation. In 2020, the prefab parts for the exoskeleton only come in "left arm, right leg" pairs, or you can just give up and plug straight into SkyNet.
### The collective body > “Autonomy” means that we make the **worlds that we are** grow. > > Tiqqun, The Cybernetic Hypothesis So why does this matter for us? Me and you and everyone we know? We put up with the cost all the time. But the limitations of our tools—on the vulnerable side of a power dynamic that, unbeknownst to everyone involved, functions as a catalyzed and increasingly accelerated system bringing out the worst in all of us—range from the everyday annoyances to a pervasive alienation, an information malaise. You have to ask yourself, "Why are we doing this again?" These days I find myself away from broadcast tools and more in the insular channels of my friend groups. What I want from technology is to share life with my friends; to be given the opportunity, and the power, to share that life, and all it includes… showing them care, or sharing the bits of knowledge that seem important to us as a group. To reinforce the fulfilment of collective potential in whatever direction our groups take off in. Some of my friends watch stuff together; some of my friends remark on goings-on; some of my friends research together; all communicate entirely differently. Not all of them belong on "IRC, but for gamers," or "IRC, but for highly productive and extremely important teams." And yet, that's where we are. Forget the prefab parts: a proper exoskeleton for our collective body evaporates; and we face the sunlight in whatever direction we choose. The ability to extend our shared spaces collectively, with each of our groups of friends and collaborators, using our own computers: that's Urbit. ]]> https://urbit.org/blog/platform-decay https://urbit.org/blog/platform-decay Thu, 07 May 2020 00:00:00 GMT ## Platform decay, decentralized marketplaces, and Urbit The current model followed by almost all social media services is one of easy and free participation in exchange for advertising and data collection. If you’re a fan of decentralized projects like Urbit, or just someone who pays attention to the ongoing debate over social media incentives, you’re likely to be familiar by now with a central critique of this model. There are historical reasons for why we ended up here: ad-serving and user-tracking technology outpaced subscription and micro-payment models because of recalcitrance by banks and regulatory hassles that prevented ‘real money’ from easily integrating into websites. Instead, we pay in eyeballs and analytical data. When we take a moment to think about possible alternatives, we usually fixate on that original bad bargain—’free’ in exchange for ads—before lamenting that first-mover advantage, critical masses of users, and sophisticated algorithms have made it very hard to see successful challenges to the reign of sites like Facebook and Twitter. We’d be happy to pay a few pennies to have an ad-free, data-secure alternative. But this critique misses something larger. There are, in fact, plenty of colossal Internet businesses that actually do charge people money to participate: e-commerce platforms. Some of these enable the sale of virtual goods and content (Steam, Patreon, Spotify), some physical goods (eBay, Etsy), and some both (Amazon, Kickstarter). Because these commercial platforms had to figure out how to make online payments easy and secure for mass audiences for their core business to work, they avoided the total dependency on ad revenue that social media sites have always had. Unfortunately, several factors have conspired to push e-commerce platforms closer and closer to a social media–style business model—by manipulating user behavior and prioritizing ad revenue and branding deals over providing a fair marketplace for ordinary buyers and sellers. E-commerce platforms, as generally perceived by the public, operate on a simple ‘deal’: users (small or large, individual or corporate) can buy and sell goods and services on the platform, facilitated by search engines and payment processors, and augmented by reputation systems and safeguards for buyers and sellers. In exchange, the platform takes a small cut of the revenue that passes through. E-commerce platforms were a key part of the genuinely revolutionary effects of ‘Web 2.0’. As software ate the world, it uncovered siloed value that had been previously inaccessible. This was particularly true for items with a high potential value, but a small market-base. Just as social connection technologies—from Usenet to Facebook—enabled people with niche interests to find each other, mass-platform e-commerce sites allowed buyers and sellers of niche goods—physical or virtual—to find each other and transact, thus granting those goods actual value, and their owners revenue. The ‘deal’ that eBay, Amazon, and the others made with millions of buyers and sellers was a great business proposition for over twenty years. They could take advantage of centuries worth of asymmetrically siloed value, now searchable worldwide, and take a ~10% cut. Their moat was in sophisticated search algorithms, network effects, and, as time went on, finding ways to make online payments as easy and secure as possible—the road not taken by early social media companies like Myspace that decided in favor of reliance on ad revenues. Crowdfunding sites like GoFundMe and Kickstarter reduced friction in online fundraising in exchange for a cut. Patreon applied a similar approach to regular support of individual content creators; Spotify, Soundcloud, and Pandora made it easier to legally stream music. All took advantage of the same ability to extract value from widely distributed niche products. All offered their vendor-users a version of the same deal: facilitation of niche broadcasting in exchange for a visibility and transaction fee. The problem is that the deal is now decaying rapidly, without any prospect of recovery. The behavior of every mature e-commerce vendor platform clearly shows an accelerating trend towards exploiting user data, serving ads, pushing exploitative auxiliary services like high-interest loans, and taking advantage of vendor lock-in and near monopolies on customer attention to warp marketplaces for the benefit of large brands. In other words, they’re all double-dipping. Clicking through normal transactions on PayPal, for example, invariably produces ads for subprime credit cards; similarly, Patreon users can now borrow against their (expected) future earnings. Spotify has reinvented pay-to-play for artists and labels; eBay has leaned hard on promoted listings, with extra fees, in order for vendor offerings to show up in search results. Amazon has increasingly ‘gated’ third-party offerings in categories and brands in which not only their own ‘store brand’ products, but those of large brands with which they have struck external deals, compete. A growing number of consumer products have essentially been prevented from being resold on Amazon or eBay by the threat of spurious, but expensive, anti-counterfeiting lawsuits by major brands, thus buoying new sales by effectively eliminating resale value. There are several key reasons for this. One is simply that as the initial burst of siloed value being accessed, revalued, and bought and sold subsides, the number of transactions plateaus and e-commerce platform revenue with it. The traditional model of venture capital funding for platform businesses, however, demands an exponential growth in profits; the exploitation of customers and vendors becomes irresistible in order to maintain these profit trajectories. Another reason is that as e-commerce platforms accrued large userbases of both buyers and sellers, they began to act like other online platform businesses—distorting marketplaces and shaping user behavior to maximize retention and profit, rather than offering a straightforward service. Platforms exploit user data because they can. Their ownership of your data is baked into the client–server architecture on which their businesses run, and the system cannot correct itself. Chris Dixon, in his prescient [essay](https://onezero.medium.com/why-decentralization-matters-5e3f79f7638e) about the future of decentralized commercial protocols, presents an S-curve whereby a platform business transitions between serving its users in order to grow and achieve market dominance, and eventually exploiting those users once it has achieved that dominance. In particular, early-stage e-commerce platforms do anything they can to benefit vendors willing to sell on their platform. Vendor presence is vital to attracting customers, but once those customers have been captured, the vendors become the [competition](https://www.wsj.com/articles/amazon-scooped-up-data-from-its-own-sellers-to-launch-competing-products-11587650015?mod=searchresults&page=1&pos=1). Customer eyeballs and dollars are a finite resource, and platforms carefully calculate the potential profit to be made from showing customers one product or another. The lack of platform-agnostic reputation systems also aids vendor capture by platforms. Rather than optimize for a system in which any buyer or seller with good external credibility (measured by, for example, credit scores or relevant business licenses) can use their platforms as a commercial intermediary, eBay and Amazon use seniority and internal transaction records to determine vendor permissions. Since sui generis internal reputation scores can’t be taken to other platforms, these policies create a sunk cost of effort and reputation which traps vendors within specific marketplaces, without an easy exit. Market consolidation and bureaucratic pressures for legibility also work against maintaining a square deal with vendors and users. The midpoint on Dixon’s S-curve, when exploitation of vendors begins to occur, can also be seen as the point where a customer base is solid enough to be marketed to advertisers and major brands who can pay for favorable search placement. Such large, legible deals are easier to negotiate, easier for individual executives to take credit for, and are useful in meeting arbitrary investor benchmarks and buoying quarterly revenue statistics; but they privilege larger brands with larger ad budgets over ordinary sellers in a niche. Another major downside of large, centralized e-commerce platforms is that, like any large bureaucracy, they have PR and regulatory concerns that provide an attack surface for regulators, activists, litigious brands, and monopolists seeking to crush competition—almost invariably at the expense of vendors and customers. Even more subtle and worrying effects come when e-commerce platforms realize that the actual products being shown do not have equal chances of being purchased, therefore delivering revenue to the platform itself. This applies even when the products being compared have nothing in common: apples to oranges; cat food to hiking boots. Accuracy of internal search engine results, and a comprehensive spread of offerings, are regularly sacrificed to the aim of placing a ‘sticky,’ lowest-common-denominator product in front of the customer. I recently ordered vegetable seeds on Amazon; after a few searches for different kinds of seeds, I started having to wade through ‘premium placement’ ads for farmer-themed romance novels before I could view, and choose from, the actual seeds on offer. This deprecation of niche products is ongoing and makes the interface of such platforms less and less navigable. It’s also worth mentioning that, frequently, such deprecation has very little to do with durable consumer interest or a free-market determination of product value. Rather, they come from a ‘thumb on the scale’ of large advertising campaigns that seek to convert a wide range of consumer curiosity into sales for a particular product at a particular time. The manufacturing of consumer desire in this way feels creepy. It’s frustrating to have your tastes dictated to you by whoever has the biggest ad spend. It also spells the end of small business on the Internet—you can spend years mastering knowledge of a profitable niche, but if Amazon profiles your customers and decides they’re more likely to spend their money on the latest Pixar DVD, your listings will be shoved below the fold. Pixar paid for placement and you didn’t; the Pixar ad placement deal made a splash in the quarterly report, your fees didn’t. In cultural terms, this threatens to create a winner-take-all situation that will prevent new and niche artists and thinkers from gaining attention at the expense of explicitly placed, ‘premium’ content. Again, this is all accelerated by the way that ad sales function; platforms like broadly popular content and products because the larger number of customers searching for and purchasing them creates more granular data and increased opportunities for micro-targeting ads. This incentive leads to the increasing invisibility of long-tail, niche content or products whose appeal has less to do with lowest-common-denominator addictive virality, whether it’s the repair manual for your ‘85 Toyota or a copy of Plato’s _Republic_. It also accelerates the stampede toward a ‘sharing’ and ‘renting’ economy; when ad impressions are a sizable part of company revenue, forcing customers to return to ad-infested platforms to continue renting objects or content, rather than making one-time purchases, is a no-brainer. There’s a nightmare scenario here, a kind of hybrid, zombie brainchild of Jean Baudrillard’s concerns about the simulation of common meaning and Peter Thiel’s arguments about the stealthy hollowing-out of industrial capacity. In this nightmare future, financial and legal regulation strangle the viability of anything like a real industry, and we’re left with a handful of incumbent platforms which have concluded that the smartest move is to just sell vaporware ‘content,’ collectible bobbleheads, and sugar water. The supposed niche of any platform business is academic at this level; Amazon, Spotify, and Patreon are simply machines for converting user attention into the highest-margin purchases possible. The ‘high-velocity trash economy’ (h/t Pierre Rochard) increasingly shows itself as a network of funnels by which insider prognostications about government and central bank policy are turned into profits, with the illusion of productive activity serving only as window dressing. This deforming process extends all the way down to the vagaries of the consumer economy. If there is no financial reason for available consumer offerings to be anything we actually need, they can and will be deprecated and displaced; useful goods and services will be sidelined by literal trash that has the ‘financial’ logic of maintaining an unbroken, upward metric by virtue of direct parasitism upon the human brainstem. The implications are obvious: if we can’t make markets obey a representation of factual reality, we won’t be able to build stuff we need in the factual world. The damage which has been done by our replacement of infrastructural capacity with nebulous thematic concerns that happen to feed our dopamine receptors is becoming rapidly apparent to anyone paying attention. If our search results for ‘hammer’ algorithmically default to pro wrestling merchandise, that changes our real, physical lives. Our ability to go out and build things with a hammer is diminished, and so is, implicitly, the importance placed on that act of building by the culture that surrounds us. The physical world of technology, no less than the informational, is a slippery pile of patches and detours. When JPL scientists reestablished contact with an early satellite after decades of inactivity, they relied on vintage computer parts from eBay to communicate with it. In some areas (cars, hand tools, some analog electronics) the present consumer offerings have generally declined, in terms of reliability and user serviceability, from a zenith some twenty or forty years ago. Forty-year-old Toyota trucks often fetch more, in equivalent value, than they did when new, because regulations and planned obsolescence have prevented the ongoing production of trucks that meet the same standards of reliability and fuel economy. The failings of our just-in-time supply chain have likewise been made blatantly obvious by the disruptions of COVID-19. Both argue in the strongest terms against allowing consumer-facing e-commerce to be captured by what are rapidly becoming payola-driven entertainment platforms rather than marketplaces for essential goods, whether car parts, beef jerky, or N95 masks. So what’s the solution? Many prescient observers envision decentralized protocols to safeguard the data of vendors and buyers, and algorithmic representation of their offerings. Any new ‘deal’ between vendors, buyers, and platforms must be rooted in data ownership and neutral code if it is to avoid a repeat of the current paradigm. Personal ownership of browsing, buying, and selling data is necessary so that search results and boosted listings aren’t distorted for each customer in order to ‘nudge’ them into consumptive patterns, and so that value is retained in useful, niche items which don’t lend themselves to dopamine addiction and manipulative marketing campaigns. The hope is that widespread personal control of data would lead to the emergence of protocols and marketplaces which have realistic economic models based on fees; the ‘square deal’ originally promised by ‘Web 2.0’ platforms. Such marketplace businesses wouldn’t be exponential unicorns; they’d be steady, stable infrastructure companies like those that built the foundational technologies of the 20th century and made a secure, middle-class lifestyle possible for millions of people. Data ownership wouldn’t prevent platforms from incorporating advertising, but it would shift the balance of power to an opt-in basis. By reducing the ability to target such ads, it would prevent the lopsided reliance on ad revenue which leads to smaller vendors being neglected. Big companies wouldn’t go away; they’d make their margins by focusing on economies of scale in the real world: logistics, fulfillment, and redundant inventory for commodity goods. We need decentralized, robust reputation systems to combat monopolistic platform lock-in and its attendant abuses, but also so that smaller, more private marketplaces can create their own rules, and still be functional and secure. Right now, reputation data is siloed by the same companies that built large marketplace platforms, which means that anyone who moves outside of their walls loses access to it, and thus loses protection from bad actors. The ability of users to run their own reputation scans, or to pay for a separate layer of reputation data which is marketplace-agnostic, is essential to breaking the hold that large marketplaces currently have on e-commerce. If a local community wanted to trade in homegrown food, for example, they could restrict access, while also making sure that their members didn’t have a previous, external track record of fraud in other markets. Where does Urbit fit into this? We think that the basic infrastructure and alignment of the Urbit system has two major advantages to offer those who will build these new protocols. The first is the simplicity of running one’s personal server of data and locally hosted processes. Any attempt to reimagine privacy-focused e-commerce under current client–server models quickly runs into complexity problems—Web 2.0 platforms require reams of legacy code and armies of SREs to keep them running. The second is the ability to maintain a semi-private but reliable digital identity among different platforms and communities, including commercial ones. Most current conceptions of autonomous marketplaces rightly focus on the potential of cryptocurrency as a decentralized payment system. But wallet and exchange problems, the work-in-progress state of micropayment infrastructure, and price fluctuations among different currencies (crypto or fiat) clearly show the need for a decentralized reputation system which is not simply a crypto wallet address, but is likewise more secure than a PHP forum profile. We think that Urbit ID can provide such a holistic, yet pseudonymous, reputation system without being tied to a particular currency, wallet, or commerce platform. Right now, Urbit is a communication system for communities and individuals. We aim to create spaces where human conversations can take place with a minimum of distractions. We believe that commerce, too, is best achieved in such a space; think of your local hardware store rather than a carnival with barkers at every turn. We also see this as a priority deeply in keeping with the underlying ethos of Urbit. We frequently speak of Urbit as a tool, and reference classic principles of tool design when considering how to expand it. The test of any tool is in its ability to shape the world; if that function isn’t well matched to both the actual world, and the hands and desires of those who will wield it, usability will suffer. Open marketplaces should function as magnifying glasses, to hand buyers and sellers a tool to see the shape of supply and demand, rather than distorting the actual terrain of the marketplace in order to create its own desires. Soon we’ll have BTC integration tied to Urbit ID, and independent security audits of the Urbit system. We want Urbit to become the foundation on which users can build simple, decentralized protocols that enable us to safely and easily buy and sell goods and content that we actually want and need. Urbit is calm computing. Calm commerce follows naturally. ]]> https://urbit.org/blog/introducing-os1 https://urbit.org/blog/introducing-os1 Wed, 29 Apr 2020 00:00:00 GMT ## Introducing OS 1 OS 1 is a simple, unified system for building digital communities. OS 1 is for groups of people sharing messages, links, and notes on Urbit. We designed and built OS 1 for ourselves, and we’re really enjoying using it every day. Instead of figuring out how to duct tape together apps and services, we use a single system to stay connected with the company and broader Urbit community that we really trust. OS 1 makes it feel like our company and community has a place in the digital world to call home. Most groups of friends and collaborators juggle apps and services to stay connected in the digital world today. We think OS 1 is a much better model. It’s still too early to ask the whole world to switch over to it — but we’re ready to help onboard our friends and adjacent communities into Urbit to have them try it out and get their feedback. First, let’s walk through what OS 1 is and how we’ve been using it. Then, if OS 1 is interesting to you, we’ll tell you how to apply for a community grant. Here we go: ## What is OS 1? OS 1 is a very simple, integrated tool for messaging, sharing links, and writing with a group of people. OS 1 is somewhere between ‘productivity software’ and a ‘social network’. We think it’s the beginning of an altogether new breed of social computing. OS 1 isn’t our first attempt to build an interface for Urbit, but it’s certainly the most complete. OS 1 is the first Urbit interface that doesn’t feel like an experiment. Today, when you boot Arvo (the Urbit OS kernel) and connect your browser to it, you’ll see OS 1. OS 1 isn’t the only possible client for Arvo, though. Anyone could build their own client or, since it’s completely open source, modify OS 1 however they like. For most people, out of the box, OS 1 feels like an app. Given that it’s running on Urbit OS — it’s much more than that. It’s a full-fledged general purpose virtual computer that’s secure and private to you. We’ll come back to what that means at the end. Let’s walk through the features of OS 1 one by one to see what it’s like. ### Groups The group is the main building block in OS 1. A group is simply a list of people that share a set of chat, links, and publish channels. A group is sort of like a super powerful group chat. A group of friends might just share a few chat channels. A reading group might just share a publish channel for discussion of what they’re reading and a links channel for upcoming books. As a company, we share a variety of chat, publish, and links channels to get things done and communicate. There are chats for each area of work, publish channels for digests and feature discussion, and links channels for music, influences and references, for example. Each person in a group is identified by their [Urbit ID](https://urbit.org/understanding-urbit/urbit-id/): a short, memorable name that you own with a private key (and belongs to you completely). Each group is private and requires an invite. Groups aren’t discoverable. They’re meant to be high-trust communities of people. More like a commune than a public event. Let’s walk through each module and talk about what they can do. ### Chat Chat is chat: a sequential list of messages meant for synchronous communication. There are already a million ways to chat — but Urbit chat differs in a few important ways. First, each chat channel is only shared between the host and the members of a chat. When you DM with people, messages are sent peer to peer. The system is, in fact, decentralized and pseudonymous by default. No one knows anything about you outside of your Urbit ID. Second, each chat message is encrypted and signed by your Urbit ID. Again, we don’t call Urbit ‘secure’ yet since it’s unaudited — but we’re working to have Urbit audited this year. Third, chat in Urbit OS is a protocol for sending messages between computers on the network (known as “[ships](https://urbit.org/docs/glossary/ship/)” and identified by their Urbit IDs). It’s easy to imagine extending chat in all kinds of fun and interesting directions: chats that only accept certain message types, chats that have disappearing history, chats that you can only post to during certain times of day, and so on. (If you’re curious to try one of these things, or have an idea about something you’d like to experiment with — consider submitting a proposal in our [grants program](https://grants.urbit.org/). We regularly reward contributor work with address space.) We think of chat as a primitive, not simply a ‘better IRC’. But even in the rudimentary form it’s in today, chat is essential to delivering a feeling of being at home. There’s no way to deliver a complete interface for groups without chat. Anyway, not everything can happen in a chat. Let’s keep moving. ### Links Links is a bit like Hacker News or Reddit. It’s just a collection of links with comments. There’s no upvoting or sorting. We decided that with a small group it’s more about collecting references than deciding what’s most exciting — but permutations of Links could be a great area for contributors to experiment with. We use links to collect external references and track things we’re thinking about. We’re always looking at things going on outside our small digital universe and talking about them. Links lets us keep an archive of those discussions. We originally imagined links as evolving into more of a general-purpose ‘collection creation’ interface. One where you can pull together images, video, URLs, maybe even runnable code and so on into a single channel. We’ll get there. ### Publishing Publishing is somewhere between a blog and a note-taking system. A Publish notebook is a list of markdown files with optional comments. A private Publish notebook is like a journal, a group notebook can be like a forum (with comments on) or a collaborative blog. We use publish for both long-form discussion and one-way communication about what we’re working on in its current form. The editor is calm and simple. In distraction-free mode, it’s a nice place to put a note together. Publish, like all our other modules, is an interface to a generalized protocol for storing text. We have high hopes for how Publish can evolve into a flexible tool for notetaking and discussion with richer content types and better controls over structuring discussion. ### Modules You’ll find a few other things on the home screen: a clock, the weather, and a simple command-line that we call the Dojo. These are system-wide utilities that aren’t group-specific. The clock and weather tiles are simple indications of the state of the outside world. We built them because they’re things we wanted to be able to check in on often. When you’re spending much of the day in front of a computer, it’s nice to know if you have enough daylight left to get some exercise in. ### Dojo The Dojo gives you a quick way to see that, under the hood, Urbit OS is a powerful general-purpose system. The system hasn’t evolved to the point that our libraries, frameworks, and modules are polished to our standards. Since it’s still early, we can’t claim that it’s easy to develop on top of this platform. But we want it to become much easier. ## Why we made OS 1 We made OS 1 because we wanted to begin the process of re-centralizing our digital lives on a platform we actually control. Replacing all the services we use all at once is impossible — so we picked a suite that we felt would work nicely together. We’ve always felt that one of the most exciting things about Urbit is in bringing your digital life together in a way that feels like home. The digital home, we think, is what our computers are meant to be but simply can’t be in a world of cloud services. Once upon a time, thirty years ago, your personal computer was an open-ended tool that was reasonably simple to understand and use, lasted a pretty long time, and belonged to you in every way. OS 1 is a first step down the road back to a system that’s simple, durable, and belongs to you in every way. It’s a long road, and we feel like we’ve made a very significant step forward. One really critical thing about OS 1 is the pattern of ‘groups sharing modules’. This pattern makes it perfectly clear how a virtual computer can outcompete a bunch of different services. As the collection of modules in OS 1 → N grows, we make it easier for others to contribute modules, and we increase the flexibility of connecting modules our system quickly outruns the messy, disconnected world we’re currently stuck in. OS 1 isn’t mature enough to be _easy_ for outside contributors to add modules — but it is [open source](https://github.com/urbit/urbit), and anyone is welcome to tinker and experiment with modifying it. We run a [grants program](https://grants.urbit.org/) to reward contributors for improving any part of Urbit OS. As the system matures we expect this program to expand and accelerate. ## Looking ahead By using OS 1 you inherit all the benefits of running Urbit OS. Each user runs their own node completely independently. Everyone using Urbit OS owns their own identity and data. And every Urbit OS node communicates over an encrypted and authenticated network. (Again, We don’t yet call Urbit OS ‘secure’ because it hasn’t been audited.) OS 1 is just a client for Urbit OS, of course. Other people could write their own clients that are completely different — like different windowing systems for Linux. Our hope is to build a client that includes interface components and frameworks that make it exceptionally easy to build modules for groups of people to share. Other clients could potentially invent entirely new ways of interacting with the underlying system. There are two threads by which OS 1 evolves into OS 2 and beyond. First, by becoming a more powerful system and second, by adding modules to expand our supported modes for interacting. In its current form, OS 1 by no means exposes all the power of the underlying system. Our approach to building OS 1 was to focus on the quality of the UI, and accept that it will feel like a monolithic app. We’re already working to evolve our interface into something that feels flexible to an ordinary user and developers can easily extend and build on top of. Specifically, we want better affordances for putting multiple streams of data on the screen at once, better keyboard shortcuts and navigation affordances, better libraries and frameworks for building modules. We want a standalone client and a single login system. These are some of the things currently on the drawing board. As far as modules are concerned, there are too many things we’d like to build to list them all. We’d really like to have threaded conversations; short, Twitter-style messages; biometric and status tracking; an e-book reader and annotations system — and that’s just to name a few we see in the near term. We’ve long since felt that the most powerful thing about Urbit is that it’s a general-purpose system. OS 1 puts us on the road to giving people an actual material sense of what using a general-purpose networked computer feels like. As we expand the affordances of the system and add modules over the course of OS 1, 2, and beyond we expect to deliver a system that can foster everyday creativity. There’s a long road ahead, in the best possible sense. We’re really excited about what the future holds. ### Becoming a homesteader As a distributed team trying to stay connected, we like using OS 1 from day to day. We’d really like to hand-onboard a small group of like-minded groups of people who want to experiment with living on the fringe of the digital world. Toward this end, we started the community grants program. Each community that’s selected will get a block of address space to share, direct support, and (in the near future) free hosting. Please tell us a bit about yourselves [here](https://urbit.org/community/community-grants/). ]]> https://urbit.org/blog/value-of-address-space-pt2 https://urbit.org/blog/value-of-address-space-pt2 Sun, 12 Apr 2020 00:00:00 GMT 32) planets on the network. Planets currently trade largely in a range of $10 to $15. We expect planet prices to fluctuate from provider to provider as described in the previous post, based on the quality of services various providers offer. That said, it's worth noting that the cost to configure addresses for distribution is fairly static and this may act as something of a lower bound. That said, we also anticipate that planet prices will be abstracted away from most purchasers, and will be included in bundled annual packages of hosting and data routing services. For this reason, infrastructure service providers (i.e. hosting providers) will likely be the best positioned parties to capture value from planet distribution. While an increase in the supply of Urbit address space is a technical possibility, it would run counter to the economic incentives of Urbit address space owners. This is why we view a galaxy vote to dilute address space as improbable. Scarcity may be sufficient alone to drive value, but it’s much more impactful when the unit has some underlying purpose, which brings us to utility. ## Utility Utility (i.e. what Urbit addresses can be used for) has largely been addressed in the previous installation of this series. Briefly, our view is that stars and galaxies are useful for more than simply selling their subsidiary address space. Their use value encompasses their unique functionality on the network and the business models that functionality represents. As a refresher, this use value falls in five broad categories. 1. Authentication technology is broadly useful. All Urbit identities — planets, stars, and galaxies — can be used for authenticating one’s Urbit computer, and for authentication in general. Urbit ID was designed for Urbit-specific authentication, but a lot of general functionality can be built on top of this infrastructure. Urbit ID is simple, extensible, and easy to use, making PGP style claims much more useful than they are today. 2. Network infrastructure services represent a potential revenue stream. Stars and galaxies specifically should be understood as providing their owners the opportunity to develop and use network infrastructure. We anticipate that stars and galaxies will eventually charge for infrastructure related services, like hosting and packet routing, and will distinguish themselves on quality and service. 3. Stars and galaxies are well positioned to offer additional services. Because of their role as high level network nodes, stars and galaxies are uniquely positioned to offer additional services, including content delivery networks (CDNs), Bitcoin and other blockchain-related nodes, 0x relays, and Tor exit nodes. 4. Schelling points have value. Because stars and galaxies hold a certain authority on the network (by dint of their relationship to their subsidiary nodes), they act as [Schelling points](https://nav.al/schelling-point), or default hubs, for various activities that will arise as the network grows. Already various groups and communities are beginning to collect around stars. This gravity has value and will eventually be priced. 5. Voting has value. Galaxies exclusively hold a vote on the network. This voting right can be separated from the galaxy itself and transferred, and we anticipate that this will happen eventually. The value of this right has not and may never be priced into the market, though as Urbit adoption grows, the value of a vote may emerge. Utility is only one component of a value calculation. To further develop the inquiry into value, let’s move on to trading factors. ## Trading, volume, and liquidity Urbit IDs are individually represented by non-fungible ERC-721 tokens, and can therefore be sold by any platform that supports this token standard. Currently there are three known Urbit-exclusive sales platforms and one generalized ERC-721 platform that list Urbit IDs. Prices for stars and planets are readily available. Volume is low, as no effort has been made to promote the fact of availability, yet the number of [total transactions](https://azimuth.network/stats/events.txt) has increased steadily since the network was launched to Ethereum in 2019, even absent public messaging.   Buying non-fungible tokens like Urbit IDs is rather different than trading quasi-fungible tokens like Bitcoin (even setting aside the issue of volume). Each Urbit ID is unique, and so they tend to be purchased by end users for their specific characteristics, rather than generically. As an example, the planet `~parret-barret` sold for about $1,000 at a time when most planets were selling for $10-20, perhaps due to its rhyming name. Planets with circular sigils also often sell for significantly more than other planets — we’re not entirely sure why this is the case. ## Planet Sales   ## Star Sales  This ID-specific trading pattern is generally appropriate in our view: Urbit ID isn’t designed to be rapidly traded at high volume. Urbit is more like a house than a fungible good like a currency. For the time being buying and selling of Urbit IDs happens through auctions on specific platforms, and via person-to-person transactions, because each transaction is for a _specific_ Urbit ID. Again, Urbit ID is designed in the spirit of property titles, rather than currencies, so this makes sense. Strictly speaking, liquidity is a measure of the degree to which a given asset will change in value due to a given transaction. In that sense, Urbit address space is likely fairly illiquid. But liquidity can’t be properly understood without volume. As noted above, the volume of transactions on the network has been steadily increasing since inception, and yet the price has remained fairly constant. This seems to indicate a healthy ratio.  A further impact on liquidity and volume is that of available supply. At present, the majority of Urbit address space is locked by smart contracts. These lockups began to release at varying linear rates in January 2019. All address space will be completely unlocked and tradeable by January 2025. Eventually as both understanding and demand increase, we anticipate that planets will find an equilibrium price, and stars will be repriced accordingly. We’ll discuss lockups and spawning limits in more detail in the third installment of this series. In our view, this combination of scarcity, utility, and the above trading factors is a useful description of existing market dynamics for Urbit address space. A further description of why Urbit has value lies not just in its trading characteristics, but in the utility of human interaction. This brings us to network effect. ## Network effect Much is made of the concept of network effect and its impact on network valuations generally. The concept is fairly straightforward so long as numbers aren’t applied. Broadly speaking, networks are interconnected systems of people or things. Network effects are mechanisms wherein every new user of a given system makes the system more valuable to every other user. Urbit is a network of networks, and it’s designed specifically for communities — for communication and collaboration. The more people who use Urbit, the more communities get started by design. The more communities there are, the more likely outsiders get pulled in to join them, and so the virtuous network effect cycle continues. Digging deeper, Urbit is unusual as a network for the following interrelated reasons: 1. Urbit is a platform in the true sense; wherein 2. connections are peer-to-peer; 3. privacy is high; 4. discoverability is low; 5. customization and autonomy are nearly limitless; and yet 6. the number of possible groups and node connections is essentially unbounded. The upshot is that Urbit can serve as the meta-network for many different kinds of individual networks, each with their own value. Urbit is much more akin to the internet itself than it is to, say, Facebook. Reed’s and Metcalf’s laws therefore apply to Urbit in some very interesting ways. ## Network laws Before diving into a discussion of network effect, it's worth noting that although individual tokens and users of those tokens benefit in various ways from a thriving network, the structure of the Urbit network does not grant token holders any rights to profits or losses "generated by the network" as a whole. In fact the very notion is counter to the intent of Urbit's ethos of decentralization. Individuals on the network may profit by their own activities, but there is no central authority profiting off activity in the aggregate; there is no ownership of the network as a whole, and so there are no centralized profits and losses to share. Address space tokens represent one's ownership of their own parcel of land as it were. That said, an entire continent may have aggregate value even though it is distributed. To explore this notion with regards to Urbit, it's worth reviewing the three laws typically cited to express the value of a network as a whole: - Sarnoff’s law states that the value of a network increases proportionally to its size. This is widely understood to understate the value of most modern networks. - Metcalf’s law states that the value of a network is the square of the number of users (n^2). - Reed’s law states that the value of a network increases with the number of groups that can be built on the network. This is expressed as 2^n, where n is the number of groups. Reed’s law captures the notion that groups vastly increase the density of connections on a network and so scale value more quickly. Urbit is a network of networks, and so is best described by Reed’s law. Urbit’s unique approach to groups also helps resolve two other problems which lead to value loss in many networks: Urbit’s design significantly reduces both network pollution (i.e. irrelevant feeds) and network congestion (clogged bandwidth), as individual clusters can easily separate their activity from other networks on the platform. ## Urbit is a platform network It’s important to note that Urbit has utility absent the network, as Urbit OS is in fact a Turing-complete virtual computer. Though the system is designed around communities, individuals can use Urbit to create personal documents, archive data, and write code for the system. This is how Urbit’s total network size can be relatively small and still provide value for individual users and clusters. However, Urbit already has users, and for many of these, critical mass — the point at which the network is more valuable than its cost — has already been reached. In addition, as each of these individual clusters writes code for the platform, positive indirect network effects increase (more utility arises from these individual development efforts). This combination of effects makes Urbit significantly more robust than non-platform networks which can’t rely on this intrinsic utility. That said, we do not intend to ever rely simply on this individuated computing utility alone. Urbit is for communities, and Tlon’s primary focus is to increase adoption of the system as a whole, including the network. To this end we’re working steadily to increase functionality of the system, improve messaging, and actively onboard communities. ## Network identity Another distinguishing characteristic of Urbit is that identities are pseudonymous but can easily be de-pseudonymized by users at will (though not by outside authorities). This flexibility combined with identity reputation accrual means people can behave in different ways depending on the social context. People can communicate privately, publicly, or in small groups, and they can do it anonymously or otherwise. This natural and extensible approach to social engagement results in personal comfort and drives adoption. ## Extensibility Because Urbit is vastly customizable, and because it’s a network platform for other networks, it can and should grow in ways we can’t predict. Each new use case that develops creates a new market, new communities, and new hubs. Each of these contributes to the diversity of the overall ecosystem, making it vastly more interesting and diverse than anything we might create alone. We expect this extensibility to contribute greatly to retention as users are able to build the world they want to inhabit. Urbit is a deeply extensible platform for communities: it’s a network of networks. Users join to be part of communities, then launch communities of their own, further driving growth and engagement in a virtuous cycle. In the next installment of this series, we’ll describe lockups and availability in more detail and will share the data we’ve managed to compile about address space distribution. ]]> https://urbit.org/blog/infrastructural https://urbit.org/blog/infrastructural Thu, 09 Apr 2020 00:00:00 GMT ## Seasons It’s early 2020, the transition between winter and spring, and Tlon’s interface group has concluded a series of projects that mark a transition point for the Urbit project. We’ve named the culmination of our efforts “OS 1,” as in “Operating System 1.” For those who have yet to try it out, OS 1 is a bundling of “essential digital utilities” in the form of 1) a linear chat, 2) a long-form text editor and publishing/discussion medium, 3) a link bookmarking/discussion software, and most importantly, 4) a grouping mechanism that allows you to bundle the above services in the context of a group of friends, acquaintances, or family. We see these utilities as being archetypical “software shapes” that are inherently flexible and open for interpretation in terms of their use, but we have wild expectations for how these software shapes will evolve in the near future. Later design output and supplementary writing will expand upon what we mean by this. Functionally, OS 1 has been designed to serve as a bridge between [OS 0](https://urbit.org/blog/landscape-a-portrait/) (The first interface in Urbit’s evolutionary chain) and OS 2 (Urbit’s eventual departure from the web browser context). In its service as a “bridge”, we designed OS 1 to extend from the baseline app-like “module” experience first defined in OS 0, while granting it new affordances that gesture towards nascent conceptions of “communal computing” we have planned for OS 2 onward. This year, as the warmer seasons give way to chillier ones, we plan on further demonstrating our vision of what new computers and networks can be and how they feel to use. Until then, we'll be building upon OS 1, our foundation and initial take on what we deem "human-shaped infrastructure." This post serves as a reflection–meditation on OS 1’s initial form development and the attitude we brought to bear in designing it. I will be highlighting a few disparate-yet-interconnected themes we encountered in our design work, and how we materially began to address them. ## Products Urbit’s interfaces, and the process by which we build them, are resolutely not projections of the world’s current conception of digital design—that of mass digital product design. Products are sold, bought, and discarded as they are in their physical sense. Digital products are slotted into the [vending machines](https://urcad.es/writing/personal-computer-vending-machine/) we call our phones and are mass produced as candy is produced. Candy is _essentially_ free, and contemporary digital products are _absolutely_ free of cost. If you’re reading this, you probably already have a good idea of what free corporate digital products mean for your privacy. Urbit is distinctly _not_ free (as in freely offered as a service via a corporate entity), and your urbit is distinctly _not_ a product. As we trace over the interfaces we design with our cursors, we’re not building a product, we’re not selling a product, we’re not painting a product, and we’re certainly not decorating a product. Here at Tlon, we’re assembling thoughtful infrastructure you can spend your life using, learning, and extending. We’re attempting to assemble spaces you can one day pass on to friends, close collaborators, or possibly your grandchildren. We are resolute in our practice and stay our hands to the greatest degree possible as a result of this mindset. ## Beautiful Infrastructure OS 1, while incremental in its initial expression from OS 0, serves to outline our stance of what sublimely beautiful, useful, and durable digital infrastructure should feel like to inhabit. For the purposes of OS 1, we can imagine “Beautiful Infrastructure” to form the spirit of what we’re building: **Beautiful infrastructure is inhabitable**. You’re not floating in an infinite feed of content streams. You are situated in a defined space with edges you can shift at-will, as your needs deem fit. Our model for inhabitable computing is new and in limited flux, but our vision is long. **Beautiful infrastructure is a gravitational force**. Even if you can’t feel it or directly experience it, well-designed infrastructure is a cohering force. One is grounded with respect to the infrastructure they live alongside. **Beautiful infrastructure is like a [shell](https://en.wikipedia.org/wiki/Shell)**. Well-considered infrastructural projects provide durable, generalized, reusable, and most importantly, _self-designed_ containers around otherwise formless flows of information. The current state of mass software design has gotten many of the “reusable” elements of design down pat, but these structures are optimized to pump ads and information into your brain, algorithmically-sorted-linear-feed style. In Urbit, I can take the conceptual container of "temporal linearity" and use this to shape communication data as I see fit. I should be able to shape once-linear communication and cast it into a nodal communication model: My DM is now a MUD. Truly beautiful infrastructure undergirds the daily exchanges people actually care about—I don’t drink plumbing, I drink the water it carries and protects. Another way of thinking about “Infrastructure” (here defined loosely as “the _truly_ essential structures that allow society at varying scales to operate”) is to think about it in terms of interpersonal support structure. We deeply hope that Urbit can form a support structure for you and your community. These self-evident statements underlie the core problems we sought to address in designing a userspace for OS 1: How do we design an _actually_ personal digital infrastructure? What does infrastructure look like when shared with immediate friends? With a cooperative organization? How can we impress upon Urbit's earliest adopters that once they've obtained an Urbit ID, their computer is at that moment _literally_ entirely theirs?

## Expressing Infrastructure OS 1 was optimized from an interface design standpoint to ensure that a new inhabitant viscerally felt like there was a “space to inhabit”, or a “canvas to be painted”, or a set of simple and fundamental digital structures they could arrange to make their computer their own. Counterintuitively, designing around this notion of a "rich blank slate” required OS 1 to be designed in a manner that _almost_ results in it feeling incomplete, or on the verge of appearing deeply unremarkable. A few key design decisions emphasized the above gestalt we sought to express: - We decided very early on to revert the colorful presentation of Modulo to a literal blank slate—OS 1 is much like a sheet of graph paper now. - All typography and typographic representation was severely flattened. This decision was one of our more drastic ones, and granted OS 1 a sense of being somewhat more "undirected" in terms of its form. We hope that as our system becomes more robust, users can decide which elements of their systems require emphasizing, to feel most at home with their information. - The majority of graphics used throughout the system were rendered into black and white line drawings. As architectural schematics and orthographic projections flatten their representation of reality so an architect can “see structure” easier, we made the decision to express a visual foundation that felt more like a diagram than an ergonomic user interface. - In its visual simplicity, OS 1 was essentially arranged to emanate the quality that modifying the underlying system was a simple task. Rather than present Urbit's first users with beautiful artistry and a sense that our infrastructure was to remain untouched and sacred, we decided to present a visual face to the project that felt nearly rudimentary, nothing outside the realm of editorial capability for the average person. Many other small constraints have been set into place to build what we truly believe to be simple, durable, personal, digital infrastructure. While visually primitive, OS 1 has managed to manifest a rich systematic expressiveness based on how it seems to be currently inhabited. We've been enjoying witnessing the ways people are beginning to interpret and make use of the faculties provided to create communal computing experiences of their own. What's most wonderful to observe is the fun people are having in striking out on their own in this collective endeavor. More than anything, we hope Urbit’s inhabitants are having a good time and learning to enjoy computers again. ]]> https://urbit.org/blog/value-of-address-space-pt1 https://urbit.org/blog/value-of-address-space-pt1 Mon, 06 Apr 2020 00:00:00 GMT 8) stars to spawn. Stars similarly provide packet-routing and peer-discovery services, and spawn planets. Each unbooted star contains ~65,000 planets (2¹⁶). Planets are the individual IDs that most people use on the network. There are a total of ~4B (2³²) planets all combined. Planets can also each spawn ~4B ‘moons,’ which are Urbit IDs intended to be used for IoT and other devices. Moons are permanently locked to their planets for security and to protect the network from sybil attacks. Stars and planets can leave their sponsors (i.e. the star or galaxy currently providing peer discovery and packet routing) if they wish, and sponsors can choose to stop servicing any given star or planet. This is intended to prevent lock-in and to encourage good behavior from both parties. This mutual accountability is important in a network of finite identities that accrue reputation. Essentially Urbit ID is 2³² individual names that are bundled into blocks that are larger and larger in size. You might think of a planet like an individual house, a star like a town or a city, and a galaxy like a county or a state. The Urbit ID registry and its rules are built on a set of smart contracts called [Azimuth](https://github.com/urbit/azimuth). These contracts are registered on the Ethereum blockchain. Galaxies can upgrade the mechanics of Azimuth by majority vote. Today, Urbit ID is live and deployed. Each Urbit ID is represented by an ERC-721 non-fungible token. People buy and sell Urbit IDs every day and the number of transactions has been steadily increasing. So, fundamentally, Urbit ID is just a finite set of names and a system for assigning them. We’ve left out a few technical details here, but if you’re curious you can read our [introductory](https://urbit.org/understanding-urbit/urbit-id/) materials or, to go deeper, read the [docs](https://urbit.org/docs/) or the [source](https://github.com/urbit/azimuth). ## Why we need a new address space In our view, the short history of the Internet has been a lesson in centralization and consolidation. DNS is a cabal of root servers. IP is essentially controlled by a single entity. All of our naming schemes have been more or less centralized. Certificate-signing is a mess of competing authorities. Search is Google. Hosting is AWS. And don’t even get us started on social networks. Centralization on its own isn’t bad, but it certainly has some negative implications, privacy being the most obvious. Beyond that, the naming schemes we have on the Internet today aren’t _personal;_ they’re not controlled by their users. Simultaneously working against this trend are a variety of decentralized web projects, various digital currencies, and dozens of protocols that are all focused on individual use and distributed ownership. But how do any of those actually surface to the mainstream? How do people identify themselves in this new digital world? Where do they call home? Where’s the Schelling point when they want to connect and work together? Urbit ID is the missing piece. It’s the minimally viable substrate for the digital world that’s been sketched out by Bitcoin, Ethereum, and all of their decentralized kin. From our perspective, the kit of decentralized technologies that exist today is designed for a world that’s only just coming into existence. That new world shouldn’t be owned by any one entity; it should be distributed. We believe that Urbit OS and Urbit ID are the right foundation for interacting in that world. The ownership structure of a new territory has a huge impact on its future shape. While most of Urbit ID remains an uninhabited grid, we see it as most likely to become a varied array of neighborhoods and cities. Not because this is a nice outcome, but because it’s the most economically likely one. More on the concept of neighborhoods below. ## History and distribution So how did these ~4B names get initialized and distributed? Urbit address space was created _ex nihilo_ — from nothing but an idea — and then distributed directly to project contributors who helped develop the system. Now that the system is up and running, Urbit address space has steadily become more distributed. This arguably follows the distribution patterns of the Bitcoin UTXO set, and even historical distribution patterns of physical land. Let’s take a look at how this came to be. Urbit was originally an experimental personal project. The original Urbit address space registry was just a text file in the source code at that time. At the very beginning, all of the address space was owned by Curtis — its creator — and it was all pretty much worthless. In the years leading up to 2013, Curtis gave away galaxies to anyone who would help or contribute to the project, and to several people who were simply in the right place at the right time. Once Urbit became a working prototype, in mid 2013, a company (Tlon) got off the ground to help develop Urbit and a number of other developers joined. At this point, Tlon bought half the address space from Curtis for what now looks like an exceptionally good deal. Then Curtis and Tlon each earmarked a significant chunk of address space for infrastructure and community development — we refer to this as the urbit.org address space, and it’s held by Tlon. From 2014-2018 Tlon and Curtis both gave away address space to individual contributors who were helping with the project. In 2016 and 2017, Tlon sold some of its address space in two crowdsales to fund the development of Urbit (which formatted the Urbit address space as a user-facing system). These sales were focused on getting address space into the custody of users with an interest in the technology and developing the network. It was then decided that a public key infrastructure was the most logical way to instantiate ownership of Urbit address space. Thus, after much [consideration](https://urbit.org/blog/urbit-and-the-blockchain/), Urbit ID was deployed to Ethereum in January 2019 under a set of contracts called Azimuth. At this point, there were about 2000 star holders and around 80 galaxy holders. We can’t be sure how many independent holders there are now — the address space is entirely out of our hands — but the distribution has been steadily increasing. Addresses trade regularly on urbit.live and Opensea, and are sold by a few other smaller outlets like planet.market and Urbit Marketplace, all of which are independent of Tlon. ## Analogues and the intrinsic value of address space We’ve often said that Urbit address space, particularly stars and galaxies, have intrinsic value that should be understood beyond simply the price of the planets they contain. Another way of putting this is that, because stars and galaxies are infrastructure nodes, they’re uniquely positioned to generate cash flow from activities on the network beyond simply selling their planets. This cash flow won't likely come from passive activity, but from actively working to develop businesses located on stars and galaxies. Let’s dig a bit deeper into these activities through the lens of some analogues to the existing world. ### DNS We’ve often compared Urbit ID to the IPv4 and IPv6 systems. In this scheme, galaxies are much like [/8 blocks](https://en.wikipedia.org/wiki/List_of_assigned_/8_IPv4_address_blocks) of IPv4 addresses, while planets are much like individual IP addresses. The difference is that IP addresses are rarely owned by end users, and they’re typically assigned dynamically, so they tend to change. This is why IP addresses can’t be used to accrue reputation for end users or to prevent spam. It’s worth noting that while most of us don’t own our IP addresses; they are currently [traded](https://auctions.ipv4.global/) on the open market. As of the time of this writing, individual addresses are selling for ~$20. /8 blocks are extremely valuable, and none are currently on the market. ### ISPs Another analogy that we often use is that of ISPs (Internet service providers). Stars in the Urbit network are the closest approximation here in that they help planets connect to the network. Ultimately, connections on Urbit are peer-to-peer, but stars provide initial connections, distribute software updates, and route packets when peer-to-peer connections don’t yet exist. These services all represent a cost, and we believe they’ll eventually be metered and priced. ### CDNs CDNs (content delivery networks) are another useful comparison. Stars are perfectly positioned to play this role in the Urbit ecosystem. Because stars are designed as packet-routing infrastructure, they’re able to easily spin up greater bandwidth to service high-demand content, or to aggregate content. Stars can also act as content curators and providers, or as content filters for those who want a more narrow range of access. On the other end of that spectrum, stars could easily function as something similar to VPNs (virtual private networks) for any type of content. ### Software distribution An infrastructure innovation that’s relevant here is that Urbit OS is specifically designed to make over-the-air (OTA) updates to the system — meaning software and updates are distributed automatically with no effort on the part of the end user. By default, planets and stars get their system updates from their sponsors. As the system evolves, we expect stars and galaxies to not only provide core system (kernel) updates, but also to act as software distribution hubs. Stars will distinguish themselves from one another by the quality of their service. Some might focus on speed of distrubtion; others on assesing the quality of the code they distribute. All of this represents effort on behalf of the star, and could act as a profit center for the star operator. ### Payments and transactions On top of routing and software distribution, there are likely other worthwhile services for stars and galaxies to provide. A payments-focused star could run Bitcoin, Lightning or Ethereum nodes exclusively for sponsored planets. A trading-focused star could run a 0x relayer and provide an order book for sponsored planets. It’s easy to imagine machine learning or compute services being offered as a service; for example, a star could bolt on a TensorFlow chip and allow its sponsored planets to run computer functions for a fee. ### Land We often use land as an analogy to describe Urbit address space. This is not simply because Urbit address space is finite or because IDs act as addresses on the network. It’s because Urbit is designed to be extremely extensible. Once you take possession of your Urbit — whether it’s a galaxy, star, or planet — you have immense flexibility: you can use it as-is out of the box, you can rearrange your environment, or you can go as far as writing code to create new functionality. Urbit address space is meant to be used productively. Urbit is very much territory for homesteaders. We expect that certain groups of Urbit IDs will be more valuable than other groups. There is no unit price for physical land — rather, price depends on location and a host of other factors. Similarly, we expect a range of prices to develop for Urbit IDs. We’ve already seen price variation around specific names and [sigil](https://urbit.org/blog/creating-sigils/) shapes. In the long term, the real value of an infrastructure node (a star or galaxy) is not simply in trading on the unit price of planets, but in developing and improving the real estate. ### Dealer-operators Another approach to developing the value of address space that is already emerging is that of the dealer-operator. The most convenient way for an ordinary person to use Urbit is to have a planet hosted in the cloud so they can log in from any device. Because the long-term value of a star or galaxy is in providing services, and because most planets and stars need to be hosted, providers of these services have begun to emerge. These providers sell users a planet, host it for them, and deliver related services through their stars and galaxies. This means that with a couple of clicks an ordinary user could buy a planet, sign up for a hosting/routing bundle, get access to a set of services, and be immediately ready to interact with the community of their choice. There are plenty of other examples that we haven’t even imagined yet. The important thing is that stars will likely distinguish themselves through the services they provide. Since planets and stars can always change sponsors, a large part of the long-term monetization of a star or galaxy lies in the quality of service that a given star operator provides. This independent ecosystem has a secondary benefit of creating a varied tapestry of digital environments on Urbit, since every community and every dealer-operator can customize as they see fit. That variety will make the landscape much more interesting and ripe for exploration than anything we might create with the top-down approach found on centralized services. That variety will also likely cause a range of values for planets depending on their sponsor or “location”. This ecosystem has already begun to emerge. Both Tlon and urbit.live are working on developing services along these lines, and we’ve heard from several other entrepreneurs considering similar offerings. This is emerging not simply because it serves a user need, but because it’s the directional flow of the system. In other words, it’s the simplest way to monetize address space while avoiding the oligarchic, surveillance-based, advertising-oriented systems we’re stuck with today. To summarize, our view is that Urbit address space is valuable because it’s useful and it’s scarce (and it has network effect, but more on that later). What’s more, while the use case has already begun to take shape, it will grow in directions we can’t yet imagine because the system is designed to be extensible — Urbit is a homestead, not a hotel. We’re looking forward to seeing what the world builds with Urbit. In the next post in this series, we’ll explore the issues of scarcity, utility, liquidity, and network effect. ]]> https://urbit.org/updates/2020-04-04-update https://urbit.org/updates/2020-04-04-update Fri, 03 Apr 2020 00:00:00 GMT https://urbit.org/blog/the-understanding-urbit-podcast https://urbit.org/blog/the-understanding-urbit-podcast Thu, 02 Apr 2020 00:00:00 GMT At the end of January, the team at Tlon invited me to spend some time at their San Francisco office and to create a podcast series about Urbit. The idea was to produce an interview-based series about the project told in the words of those working on it. In this modest goal, I think we were very successful. Beyond a transactional description, however, we captured a range of perspectives of the flaws in our current computing landscape and visions for how Urbit will alter it. Over the course of seven episodes, the interviews explore subjects ranging from the philosophy of technology to the performance optimization of Nock execution. Most are non-technical, but even the technical ones will be understandable and entertaining to a non-technical audience. That latter fact is the result of just how much fun the Tlon team is to be around. The series can be found on iTunes, Spotify, and most other streaming platforms. We begin our journey with Urbit veteran Galen Wolfe-Pauly, who introduces the project and its design principles while reflecting on the collective Stockholm syndrome we suffer in the grasp of existing computing models. We learn a bit about Galen’s background in architecture, and the importance of invisible technology. ]]> https://urbit.org/blog/urbit-is-for-communities https://urbit.org/blog/urbit-is-for-communities Sun, 22 Mar 2020 00:00:00 GMT ## Generated, not fabricated What is it that makes high trust communities take shape? When I think of the places in the physical world where people seem to feel the most at home, they’re places that have evolved over long periods of time and are owned by the communities that inhabit them. Communes in rural California, Rome, rural Japan, and villages in Indonesia all come to mind. These are places that aren’t designed in one go. They’re the product of lots of different contributors working in loose collaboration, often over long periods of time. Places that grow under the care of the people who live in them seem, somehow, to become much more alive and engaged. The strength of connection in these communities is palpable. You can feel the difference. There’s nothing even remotely like this in the digital world. The digital world is totally fabricated, stamped out, and controlled. It’s not just the surveillance, lack of privacy, or interfaces optimized for addiction — those are all downstream from the root problem: our software can’t evolve with us. It’s designed and controlled by the people that make it. We don’t participate in shaping it directly at all. And since our software can’t evolve with us, our connection to it remains thin. The digital world of today is low-trust by design. We live in a digital world controlled by other people. Nothing about it belongs to us. It’d be great if we felt a deep appreciation for and connection to our digital communities the same way we do in the physical world. Digital communities should feel alive and engaged with their surroundings. They should feel as vibrant as a city on a summer night, cozy like a family dinner in an old ranch house. We don’t often talk about it explicitly, but this is what Urbit is for. Urbit is for giving communities the tools to evolve their own environments; for us all to feel a sense of life and self-directedness in the digital world. It’s not any single specific technical feature of Urbit that makes it well-suited for communities to homestead and develop. It’s the combination of the technology under the hood and the interfaces we build on top. Let’s talk about both. ## Technically capable I love [this tweet](https://twitter.com/devonzuegel/status/1221804267083718656) about becoming a ‘digital prepper’. It feels like it has already happened to me, too. I’ve seen enough centralized services shut down and disappear (often with my data) that I’m reluctant to try new ones. No matter how friendly and cozy their branding. I find myself vetting every new app or service to figure out whether they have the potential to survive far into the future. The giant services I’ve become dependent on all seem somewhat worrisome. How long can these free services possibly stick around? Certainly not forever. The incentives in the current structure are totally misaligned. Let’s walk through them: I want to keep my data forever, but most companies are beholden to growth. If they can’t produce that growth and die, my data either disappears or becomes unusable without their software. I don’t want my data to be sold or for anyone to spy on me, but ad-funded companies have to engage in these practices to stay alive. I don’t want a piece of software to go away if a company dies, but apps and services require maintenance that’s complicated and expensive to do. Without the company around, their software is useless. The simple fact is, I let companies run my software because it’s too much work for me to run it myself. Most people can’t be their own sysadmin — and those that can have other things to do. This is more or less the entire technical value proposition of Urbit: to build a system that’s simple enough that it requires almost no maintenance. Our approach to building software makes all of the above problems easily solvable. Perhaps more importantly, we have a direct incentive to solve these problems. Tlon funds Urbit’s development by selling addresses and providing hosting. And the only reason anyone pays us for these things is because we make Urbit as directly useful for you as possible. That means we have to solve the problems inherent to the legacy internet. There can’t be any sense of suspicion or uncertainty in our new system. Your Urbit has to be something you can really trust and attach to. This is why your Urbit lets you run or write whatever code you like, and why we’re focused on making sure the data you put into your Urbit lasts forever: it won’t disappear when some company dies. But even with a system that fixes all of these issues by default, it only matters if the system is usable and understandable. What we really want is a digital world that’s generated by the communities that inhabit it — which means it has to be generated by people with no technical knowledge. ## Communities are built by people In the physical world, you can arrange the furniture in your house, or go to Ikea, or build your own furniture, or figure out how to build a house from scratch. By taking your preferred approach to design, you contribute to the shape of your city. The world around us is a wonderfully loose collaboration between the more technically oriented architects and planners, and the everyday homeowners and residents. The digital world is, of course, nothing like this. Even software developers are stuck with the current system. None of our existing apps or services can connect to one another, so whether you’re a user or a developer, you’re faced with the daunting task of replacing everything all at once if you want a better system. But nobody can do that. Urbit provides an experience with less distance between ordinary use and total customization. To start, we’ve created Ikea-like, Lego-like and woodshop-like levels of usability. What we’ve built isn’t nearly as complex as the physical world, of course, but it’s a start. For most people, customizing your Urbit should be a matter of picking from an available suite of modules. Not unlike going to Ikea. For some people, recombining interfaces and data sources within Ubit should feel like playing with Legos. And for others, who want to dive in and write code, it’s the same as going into a shop full of high-quality tools. We’re not quite to this level of sophistication, but we’re on the path. Let’s talk about where we are and where we’re going. ## OS 1 Our rough attempt to satisfy this goal is hovering on the horizon. We call it OS 1. In OS 1, you bring a group of people together to share a set of channels. A channel can be a chat, a notebook, or a collection of links. We built an interface for doing this, but you could just as well build your own. Inside OS1, data and interface aren’t stuck together like they are with apps in the default web. Nor are the people you’re connected with locked into any given Urbit app. App, data, and social graph are always separate in Urbit; they’re modular and always under your control. OS 1 is designed to make it easy to start a group. Everyone in the group runs their own instance — so everything is only shared with the people you’re connecting with. Light customization of this experience is easy for an ordinary user. If you want to go deeper, anyone can build a new protocol for storing data and transferring it around Urbit without too much difficulty, but you do have to write some code. If you really want to dive all the way in, you can modify the system all the way through to the kernel. Currently, our Ikea experience is pretty limited, there are only a few Legos to play with, and the shop is still quite rudimentary — but it’s a good start and, for the first time, it exists as one complete system. The system is ripe for experimentation, customization, and exploration. It’s ready for earnest homesteading. Come build. ## → N OS 1 is really just the beginning. There are so many more modules and means of communicating and collaborating to add. There is so much we can do to make the system more customizable. But OS 1 actually passes the bar of being both usable and futureproof. Urbit is something you can put energy into, in earnest, even as a digital prepper. It’s both safe enough and customizable enough that high trust communities can begin to take root. Urbit isn’t ready for you to build Rome — but it is ready for you to start a commune, salon, or reading group. The point, in the long term, is to let the technology fade into the background. We want people not to be ruled by their software, but the opposite: for our software to grow to suit us. Our hope is for Urbit to be as invisible, calm, and reliable as possible — and for people to be at the center, in control. If you want to explore this new world with us, let us know. We’re going to give away a limited number of hosted OS 1 communities. ]]> https://urbit.org/blog/precepts-discussion https://urbit.org/blog/precepts-discussion Tue, 17 Mar 2020 00:00:00 GMT The precepts aren’t arguments. We will discuss and justify them here. We don't expect to convince you this is exactly the right set of precepts, but we hope to convince you that this is a coherent set of precepts that might lead to something very interesting. We will address these in the same categories presented above: - A: General Design. Principles of engineering that apply in a great many situations. Any piece of code will exhibit at least a few of these. - B: Specific Design. principles of engineering that apply in a few specific situations. What these lack in general applicability they make up in specific guidance. - C: Attitude. Good code is the result of people approaching the act of engineering with the right attitude. - D: Theory. The eternal tension between theory and practice must not simply be acknowledged but dealt with. - E: Text Style. Code form matters. - F: Real Software. Not all software matters. Any software which wouldn't exist on Mars isn't real.
### A: General Design These are principles of engineering that apply in a great many situations. Any significant piece of code will exhibit at least a few of these. ### Simplicity > A20: One hundred lines of simplicity is better than twenty lines of > complexity. > A21: Prefer mechanical simplicity to mathematical simplicity. At Urbit's root is a strong opinion on what constitutes simplicity. We take a semi-subjective view of simplicity. Specifically, we say that the sort of simplicity that matters is the sort that makes it easy to reason about stuff. We call this "mechanical simplicity". A competing view of simplicity says that it can be measured objectively by metrics such as the number of moving parts, the number of kinds of moving parts, the orthogonality of concepts, or cyclomatic complexity. We believe these are important metrics because they are good approximations of mechanical simplicity, but they don't tell the whole story. In acoustics, we use decibels of sound pressure to measure objective volume of sound. However, the Fletcher-Munson curves (equal-loudness contours) describe how loud sounds are perceived to be relative to their frequency. For example, a 75 dB sound at 4kHz sounds equally loud to a 90 dB sound at 10 kHz, even though the latter is over 20 times the sound pressure. Similarly, some sorts of complexity are much easier for humans to deal with than others. For example, humans are much better at learning languages compared to even simple mathematics. Even those who are "good at math" are objectively much better at languages, in terms of how much complexity they can absorb. Humans are also built to understand some social dynamics and structures, even very complex and nuanced ones, without even consciously knowing the rules to them. Some examples of things that are objectively quite simple but subjectively hard for people to learn: - Pointers. Once you've properly learned them, it becomes hard to sympathize with your past self that found them difficult, but in practice these are a stumbling block for many people. Some think they're a "weed-out" concept, meaning they're just a good litmus test to see if you're the type of person who can learn how to program. Even if that's the case, it's still meaningful that they're effective in that. - Recursion. Same as above. Notably, `while` loops do not evoke the same reactions in people. - Monads. A little more complicated than the above, but objectively not that complicated. Certainly much less complicated than the C++ standard library, but for some reason many people fail to learn monads after giving it an honest attempt, while that's rarely the case for the C++ standard library. - Math in general. The ideas in mathematics are generally pretty clean and small, which is why once you've learned a concept, it seems obvious in retrospect. The C++ standard library doesn't look much more obvious in retrospect than it does on first glance. - Any language that induces a moment where you exclaim to yourself, "This is true programming! Everyone should program this way, it's so obviously better!". People say that often about Lisp, Haskell, Prolog, and Smalltalk. Nobody ever said that about C, Java, Javascript, or Python. And yet, which ones are easier to learn? Which ones are used in real life? I've emphasized learning here, but the same principles apply to using a tool after you've learned it. When evaluating simplicity, presumption should be on the side of "what everybody does when they have a choice" instead of "what's mathematically simpler". Being mathematically simple is a point in your favor, but the burden of proof still falls on you to argue that it will be simpler to learn and use. Adherence to this precept is difficult because you have to decide whether an objectively simpler construction is in fact better. This gives a framework for deciding the question. > A19: Dualities must be faced head-on and analyzed differently at > different layers. A crucial point is that the question of simplicity depends on the facts and circumstances of the case. In deeper layers of the system, mathematical simplicity tends to pay greater dividends. If the features of the base layer are not orthogonal, the interactions between them will only grow more complex as you rise in the stack. If the base layer has many moving parts, the number of moving parts can only increase as you rise in the stack. This isn't the only duality that depends on the layer of the stack. The most important layering of the stack is between "systems" and "applications", where "applications" are those pieces of code on which no other application depends. Many answers that are correct in system code are incorrect in application code, and vice versa. These precepts are an attempt to describe principles that apply at all areas of code, but they're much more important to follow in system code. > A22: The Law of Leaky Abstractions is a lie; abstract airtightly. The Law of Leaky Abstractions says that all non-trivial abstractions are leaky. This is the sort of statement that most engineers will accept immediately and most mathematicians would dismiss out-of-hand. In mathematics, if an abstraction leaks, it's not simply an imperfect abstraction, it's completely wrong and should be thrown away. So they manage to create perfect abstractions. They don't always abstract what you wanted to abstract, but at least they're perfect. If mathematicians can do that, there's no reason programmers shouldn't be able to do it as well. Sure, programmers have to deal with some performance considerations that mathematicians can elide, but that's rarely the reason abstractions leak. They leak most often because the creators are lazy or sloppy. If your abstraction leaks, find a way to either plug the leak or reframe the abstraction to incorporate the "leak" in a coherent way. It's not a leak if it's part of the spec. I call this "turning leaks into decorative fountains". ### Conceptual clarity > A6: Represent your data as closely as possible to the essential > structure of the problem. There are times when you need to represent your data in an unnatural format for performance reasons, but this is rare. When possible, keep the structure and location of your data in as natural of a manner as possible. For example, if a function can produce an integer or an error, don't produce a negative number to indicate an error — use an explicit tagged union. An important part of this is to fit your data types to your data skin-tightly. If your data types permit states that you intend to be impossible, this is suboptimal. There are degrees of this (if you need a number between 0 and 1000, a uint32 is probably fine even though it's suboptimal), but it's important to strive for it. As an example, if your program can't respond to requests until it completes an initialization process to fill in some of its state, it's tempting to make nullable all the parts of its state that are filled in during the initialization process, then just check whether they're nullable when you use them. This creates a complex set of representable states of partial initialization, and any bugs in that area will be difficult to track down. A better option is to create a "larval" stage of your program that performs the initialization. If requests come in during this larval stage, queue them up until it's done. When it finishes initialization, it metamorphizes into the full program, where all the fields in its state are not nullable. Then run through its queue of pending requests. This more closely resembles the essential structure of the problem. > A8: When mating different paradigms, build one cleanly on top of the > other. > > A9: Never misuse an abstraction. An abstraction provides a set of tools to solve a problem. Use them in the intended way. If this is awkward, consider that this may be the wrong abstraction to use. If the impedance mismatch is large between the paradigm that your program uses and the paradigm that you're mating it with, consider building an explicit layer whose sole purpose is to convert between the paradigms (see B5). ### Correctness > A10: Correctness is more important than performance. > > A12: Correctness is more important than optimality. Sometimes you devise a solution which is fast but slightly incorrect. This is rarely a good idea. Especially at lower layers of the system, incorrectness expands as it bubbles up. Other times, you devise a solution which is correct, but you could probably make it cheaper without sacrificing correctness. Sometimes that's worth doing, other times it's not. Finally, in some cases you find a solution that's "optimal" performance-wise or simplicity-wise, but it doesn't quite handle every situation correctly. Maybe in the real world there are genuine exceptions, and the code is much faster or simpler if you don't handle those exceptions. In these cases, it's more important to handle every case correctly than to "snap" to the nearest elegant formulation. > A11: Be simple and uncompromising in defining what's correct; go crazy > with optimizations. Nock is a great example of this. It contains the character of the virtual machine, but its asymptotics are bad. Add jets to fix the asymptotics. Another example is the ACID nature of Arvo. Arvo is a pure function f(logs) of its event log, so formally Arvo is just a function run against an event log. A naive implementation has very bad asymptotics; processing each new event is O(n) in the number of historical events. Choose the function g(state,log) such that f(logs ++ log) = g(f(logs),log). Then, as long as you keep the state in memory, processing each new event is constant in the number of previous events. This still requires O(n) restart from disk, but you can also periodically (and non-blockingly) write a checkpoint of the state to disk, so that restart from disk is only linear in the number of events since the last checkpoint. This pretty much says it all. Define what's correct in as simple of terms as possible — this will make it easier to implement and reason about. Then don't compromise on it. > A13: If you don't completely understand your code and the semantics of > all the code it depends on, your code is wrong. It's impossible to build correct code on a foundation that you're unsure of. This is different than the problem of building something with strong guarantees on top of something with weak guarantees — for example, delivery guarantees on top of an unreliable transport protocol. In that case, you are sure of the underlying system, it's just that what you're sure of is that you can't rely on it delivering packets in order or at all. So, you formalize what you _are_ sure of, which is that it will eventually get your packets through if you keep sending them forever. And that's something you can build on. This is another reason it's important to have simple and accurate correctness claims. When somebody else relies on your code, they will need to know what exactly you claim to provide. If you're accurate, then they don't have to understand your code, only your guarantees. > A14: Deterministic beats heuristic. Heuristics make it hard to give precise safety guarantees, so you should only use them when you don't guarantee much. > A17: Referential transparency is honesty and stability. A claim of referential transparency is a very strong guarantee, so it's very useful for any code built on top of it. That code doesn't need to know anything about how you produce the value because it can't negatively affect the rest of the system, and it's free to cache indefinitely. > A18: Responsibilities should be clearly separated. A program should be responsible for its own guarantees, though it must rely on other programs to maintain their guarantees. You should avoid guarantees that unduly couple separate programs. Any guarantee that references state in more than one program is suspect. ### State management > A1: Data is better than code > > A15: Stateless is better than stateful. > > A16: Explicit state is better than implicit state. Data is better than code, but code is better than state. If you must have state, state that is data is better than state that is code. There are many reasons for this. The most basic one is that humans are better at dealing with nouns than verbs. This is true to such an extent that we subconsciously convert statements of the form "so-and-so does such-and-such" to "so-and-so is a such-and-such-er". It's more challenging to view someone as a composition of their internal motivations such that they'll act in a particular way. It's much easier to see them as simply "friend" or "enemy" or "socialist" or "evil capitalist". Socially, the verb -> noun simplification is often problematic because it doesn't communicate the whole picture as accurately. It's an approximation that fails to capture important nuances. And that's because any nontrivial verb can't be understood in terms of a simple noun. However, the practical implications of this insight are different in code. Socially, people are who they are, so the best we can do is learn to understand them better. In code, though, we have a powerful tool: convert the verbs to actual nouns. We can take advantage of our innate ability to reason about nouns and use that to make code that can more easily be reasoned about. Another way of looking at this is that data is easy to reason about. Code is hard to reason about, because it's much larger. Function types are known as "exponential types" because the number of possible functions between sets of size `m` and `n` is `n^m`. If there's global state of size `s`, this is even worse: `n^(m*s)`. The halting problem says that it's impossible in general to even determine if a piece of code will finish without running it. A final way of seeing this is that it's easier to reason about things that don't change. Data doesn't change, so it's a stable foundation. Pure code changes based on its arguments. Impure code changes based on its arguments and state. Impure code that uses code stored in its state nests this whole complexity within itself. You can't, in general, update code in any way other than replacing it wholesale. This is not true of data, where to upgrade you just need a function from the old data type to the new data type. Sending code over the wire is problematic, because it's either very large or depends directly on what the other person has on their local ship. If you depend on what's on their ship and you're wrong, you may get the wrong result. So you explicitly require certain dependencies, at certain versions, ... this way lies dependency hell. There are ways to tame this, but if you just used data in the first place, those problems wouldn't exist. Of course, we can't always use data, precisely because it's smaller than code. And there are occasionally times when we could use either, but the code is actually easier to reason about than the data. All else equal, data > code > state of data > state of code. A note on laziness: lazily defined data occupies a middle ground. It's harder to reason about than strict data, but you get a lot more guarantees with it. It's probably not worth using as your basic substrate, but it makes some things easy to reason about. ### CQRS/pubsub > A2: Everything should be CQRS. > > A3: (Almost) Everything should be pubsub. > > A4: A subscriber shouldn't affect a publisher. > > A5: Communication between nodes should be communication between > independent actors. Fred Brooks pointed out that when you add people to an organization with `n` people, the number of interpersonal connections goes up with `n^2`. This is true of binary interpersonal connections, but if you allow groups of any subset of the people, then the number of total groups goes up with `2^n`. The same is true of entities in a program. A program must speak to many other programs about many things. Occasionally it can treat all others the same, so that the number of things it needs to say is constant in `n`. Sometimes it needs to say something different to each person, making it linear in `n`. Other times, it has a set of things it can speak about, say `m`. Pubsub lets the complexity grow with `m` instead of `n`. This lets the total communications grow with `m * n` instead of `n^2`. It's generally good to have more structure in how you communicate with other entities. If all you have is `send-message`, then you force each entity to define their norms around when messages are to be sent, and how to associate one message with another (eg a request to a response). This also gives the underlying system no visibility into the conversations. In CQRS+pubsub, we get a number of useful guarantees to help us structure our conversations. We know that a command will have no response other than exactly one positive or negative acknowledgment. We know that to request any data, we must have an active subscription, and we know that a subscription may be canceled at any time. We know that a subscriber's failure to process a subscription update can't break the publisher. We know that after the initial subscription update, any subscriber to a path will get the same updates as anyone else on that path. The underlying system can keep track of subscriptions, and can even cancel them automatically in certain circumstances (eg if the subscriber is on another ship and has stopped responding). This is like the nouns and verbs discussion above. Subscriptions are nouns, so we can reason about them better, and so can the underlying system. "Send a message" is a verb, so we have few guarantees about it. From Dijkstra's "Go To Statement Considered Harmful", which may as well have been written about unstructured event passing: > My second remark is that our intellectual powers are rather geared to > master static relations and that our powers to visualize processes > evolving in time are relatively poorly developed. For that reason we > should do (as wise programmers aware of our limitations) our utmost to > shorten the conceptual gap between the static program and the dynamic > process, to make the correspondence between the program (spread out in > text space) and the process (spread out in time) as trivial as > possible. In our case, we make the programs of both the publisher and subscriber simpler by writing in terms of well-known constructs like "subscribe" and "push subscription update". You need to know much less about your counterparty. > A7: A client's representation of data should be as close as possible > to that of the server. This is a natural architecture given pubsub, but it's also worth pursuing in its own right. This blurs the distinction between client and server, which makes the client more full-featured. It reduces communication to syncing, which means offline-mode is just a special case of sync latency, and all the local features are still usable. It also tends to reduce centralization, since you're already used to operating on your own, and you have all the data required to do so. The "server" becomes a commodity, easily moved when desired. Git is a good example of this — central git servers are common, but they're also a commodity, so you're never locked into a particular hosting provider. This is possible because the data structures are exactly the same on the client as on the server. > A23: Some cliches are repeated because they are true; others must be > repeated because they are not. Computer security is an industry built on cliches. An example of the first is "your security is only as strong as your weakest link." An example of the second is "security through obscurity is ineffective". Security through obscurity is in fact the most effective form of security — if nobody cares about you, you will never be hacked. The problem is that you must remain obscure for it to work, and that's hard to explain to people in a precise way. It's effective at protecting precisely when there is nothing worth protecting. So, it's better (for them, their companies, and their users) to tell them a lie. However, it's important for whoever's coming up with the cliches to understand the specifics of how they're untruthful. Some of these very precepts may fall under the category of "literally untrue but more useful than the truth".
## B: Specific Design There are some principles that aren't obviously better than the alternatives, but if you use them consistently you avoid a lot of issues. These are all examples of this. > B1: Always ack a dupe; never ack an ack. > > It's okay to ack a nack as long as you never nack a nack. This is a simple rule, and if you follow it you get various guarantees for free. For example, sending a message will always cause up to a constant number of additional messages to be sent by direct or indirect causality (other than application-level code), so you never have to worry about accidentally causing a loop. Additionally, acking an ack can never take you from imperfect to perfect reliability. Urbit isn't big on heuristics for stuff that's visible to userspace (A14), so don't even go down that road. Following this rule greatly reduces the architecture space for networks. This is very helpful in finding a good architecture. > B2: Never construct or deconstruct a duct. A duct is akin to a first-class call stack. Don't deconstruct that for the same reasons you shouldn't read your own call stack. If you read your own call stack, and do different things based on that, it breaks a lot of assumptions we have about messages. For example, we assume that a vane will handle a task the same way regardless if the initial IO event was a keystroke or a timer. Don't break that. Similarly, don't construct these by hand. This would be like inserting other functions into your own call stack and then returning to them. Powerful, yes, but pretty much always a bad idea because it breaks our assumptions. > B3: Route on wire before sign, never sign before wire. When we send a message to another vane or app, we send it on a wire. When we get a response to that message, we should care first about the wire it was sent on. It's tempting to shortcut that and route based on the type of response. Maybe you only call to `%iris` once in your app, so you feel like you can just route directly on that. But it doesn't scale, because that dimension is orthogonal to the true dimension you care about — which request this response corresponds to. This is essentially a violation of A6. > B4: Only go from statically typed to dynamically typed if you must. Consider the set `U` of all possible programs. Then consider "the program you want to write" to be a set `S` of programs that differ only in aspects you don't care about. For example, maybe one takes a little more time and one takes a little more memory, and maybe you don't care about that. This set has a generating "specification" — any program that meets that specification is in the set. A type system or testing suite can guarantee that a program is in another set `T`. Obviously, both `S` and `T` are in `U`. Hopefully the intersection of `S` and `T` is non-empty — that is the set of programs which typecheck and meet your spec. A perfect static type system can guarantee `T` is a subset of `S` — if it typechecks, then it works. Obviously, if you let `T := {}` (ie no program typechecks), then this is vacuously true. So you want `T` to be as large as possible while still being a subset of `S` — as many programs as possible typecheck without them being incorrect. But it can't make `T == S` — there will always be some programs that do the correct thing in practice, but the type checker can't prove that. Dually, a perfect testing suite can guarantee that `S` is a subset of `T` — no correct program fails the tests. This is vacuously true if you have no tests, for then `T = U`. So you want `T` to be as small as possible while still being a superset of `S` — you don't want to reject valid programs if you can help it. But you can't make `T == S` — there will always be some programs that are invalid but pass your typechecks (at least, if your program requires a Turing machine). (It is possible to create a static type system that has `S` subset `T` property described above for a testing suite — see Erlang's Dialyzerand [related work](http://user.it.uu.se/~tobiasl/publications/contracts.pdf).) A dynamic type system tells you neither of these ahead of time. Instead, you end up with type errors at runtime. Errors at runtime must be handled at runtime, when the programmer is not present. Errors at compile time are handled at compile time, when the programmer is present. There's a hybrid, which is "static type checks at run time". In other words, you load the code and run the type checks at run time. This is better than pure dynamic checking because as long as the code compiles, you get the guarantees of a static type system. But the later in the process that the code is compiled, or the more it's changed before compiling, the less guarantee you have that the type checker will pass. If a static type checker gives an error at a time when the programmer is not present (because it's run at runtime), that's worse than a dynamic type system — at least that would give it a chance of executing correctly! > B5: Anything with business logic speaks one paradigm; anything that > translates paradigms has no business logic. Translating between paradigms with significant impedance mismatch is usually convoluted code. The state machines are often hard to see, and it tends to be quite cluttered. You want your business logic to be as clean and short as possible. This way the interesting algorithms are laid bare. > B6: Functionally enforced semantic rules are better security primitives > than memory access restrictions. Memory access restrictions are blunt tools. Blunt tools are nice in simple systems, and if all you have is nails, then a hammer is perfect. But in practice you punch all sorts of holes in these restrictions, because you need more granularity and flexibility. Better to define rules in a more powerful language. This allows you to follow A6 and A8. It also lets you use all the correctness tools, like type systems, that you've developed for the rest of your code.
### C: Attitude Good code is the result of people approaching the act of engineering with the right attitude. > C1: Code courageously. If you avoid changing a section of code for fear of awakening the demons therein, you are living in fear. If you stay in the comfortable confines of the small section of the code you wrote or know well, you will never write legendary code. All code was written by humans and can be mastered by humans. It's natural to feel fear of code; however, you must act as though you are able to master and change any part of it. To code courageously is to walk into any abyss, bring light, and make it right. The unix/internet stack is a giant ball of mud. It's so big that when you're standing on it, it looks like a bog, and it's not clear that there is a bottom to it. You know there must be, but it doesn't matter because you learned long ago that the best solution is to not dig too deep. Build a strong structure by spreading out your weight and using strong construction materials. If there's something useful deep in the mud, it's probably not worth diving in to pull it out. That's dangerous and prone to failure, and you know many people who have gotten stuck down there. This is an appropriate reaction to the unix/internet stack. It's also self-perpetuating. The bog is filled — even _composed_ of — countless predecessors who did that same thing. If you ever find yourself on solid ground, protect that with your life. Learn how it works and how to use it well. Then build a strong structure on top of that. Don't let it decompose into a layer of mud. Urbit is small enough that, while there's definitely some construction-related dirt and muck covering it, the solid metallic core is not very deep — not more than waist deep in most places. The feeling of always being in contact with the bottom is very different than flailing about in a bog. This property is worth fighting for. It's the whole reason Urbit was written. New Urbit programmers are instinctively distrustful of anything below them and code defensively and fearfully. They must learn that it's all drastically simpler and more legible than they're used to. It's still hard, but you can actually make progress. To do so, you have to stand up straight, so you can get a firm footing on the ground. Keep your chin up and your shoulders back, and adopt the posture of a conqueror. When something doesn't work right, don't retreat into your shell of safety and control — fix it. The forces of chaos are always lurking, but their victory isn't inevitable. It's all on the line, so code with abandon. > C2: No time for lazy people. If there's clearly a right way to do something and a wrong way, do it the right way. Coding requires incredible discipline. Always follow conventions, and fire anyone who won't. Anything that can be solved by discipline is not a real problem. > "To him who knows to do good and does not do it, to him it is > sin." Preserving the property of solid ground requires perpetual discipline. Once lost, it's often impossible to reacquire. Don't compromise your standards. Call people out when they're being lazy. And when someone calls you out, recognize it and fix it. Sometimes it's just a question on a code review — "why did you do it this way?" Upon reflection, you determine that you did it that way because you couldn't be bothered to do it right. Maybe that was a reasonable choice at the time you were coding so you wouldn't be distracted, but if it's been called out in a code review, now's the time to fix it, not make excuses. > C3: When a smart person makes an obviously stupid suggestion, before > responding take a full 60 seconds to envision how you would implement > it and what the effects would be. Once you've determined that someone understands a situation well, if they earnestly make a suggestion that you haven't thought of, or that you immediately dismissed as impractical, stop and consciously consider it. This is one of the best ways of countering the natural echo chamber of your own mind. This is also one of the best ways to build stuff correctly. Very often, you will be tempted to cut a corner, and someone will point out that it should be done correctly. You should consider why you immediately rejected it. Was it because you were afraid of the relevant piece of code? If so, you should learn more about that code to determine if it's actually a good idea. Did you reject it because you felt it would distract you from working on what you thought was more important? Consider carefully whether this particular piece of technical debt is worth taking on. It's worth 60 seconds of thinking about how you could solve it correctly right now — often it would be quite easy to do correctly and you'd be paying interest on the incorrectness for a long time.
### D: Theory The eternal tension between theory and practice must not simply be acknowledged, but dealt with. > D1: Academia is a succubus. The results from academia are useful to take advantage of, but the process of academia is a deadly sin for a project as ambitious as Urbit. Academia usually gets the right answer, but it's slow, and it often cycles through a large number of wrong answers before it gets to the right one. It's also unconcerned with many things that a real project must consider. For example, it's generally more concerned with mathematical simplicity than mechanical simplicity, contradicting A21. > D3: Not being qualified to solve a problem is no reason not to solve it. > > D4: The best way to get the right answer is to try it the wrong way. > > D6: Everyone must regularly alternate between theory and practice. One common failure mode for academia is to get bogged down in optimally solving a particular part of a problem such that it fails to solve the whole problem. There's a sense of "if I don't know how to do it, I must discover the correct answer before I can do it". This will get you the right answer if it terminates, which is the sort of algorithm mathematicians like. Engineers like algorithms that do in fact terminate. The fastest way to build a good system is to build it as well as you know how, then iterate on it. You have to be disciplined about actually iterating over time as you learn about the problem, but often the way to learn more about the problem is to move beyond it and work on adjacent problems. Hindsight is 20/20, so get the problem in your rearview mirror as fast as possible. Then while you're driving somewhere else, you'll see what you should have done. Then turn around and go do it. This only works if you or someone else is willing to go back and fix it (C2). > D2: Never fear math. You can spend years working on a problem that you're able to solve in a few days with the right theoretical grounding. There's no excuse for being ignorant of theory; or rather, being ignorant of theory is an excuse to learn it. Many engineers fear theory the same way they fear a complex piece of code deep in the stack (C1). But they shouldn't, because theory is rarely muddy. The same characteristics that make theory impractical also make it clean and learnable. It's also pretty permanent, so a one-time investment will pay dividends for a long time. Of course, if you look deep down, you may find laziness is what's keeping you from learning theory. If that's the case, you need to decide how much you want to design good software. Because there's no room for laziness (C2). > D5: Practice tells you that things are good or bad; theory tells you why. You can never tell whether a system is good or bad without using it (see discussion on A21). Theory has great explanatory power, but only limited exploratory power. The best it can do is point out areas of the idea maze you haven't explored and suggest correlations between objective properties and your subjective judgment of the quality of the system. However, that explanatory power is indeed powerful, and the exploratory suggestions are often able to break you out of a rut (attractor basin). Respect the tension.
### E: Text Style Code form matters. Code that is hard to read is hard to maintain. Code that is inconsistently formatted is hard to read, but consistency isn't the end of the story. Some formats are simply hard to read. Formatting also isn't the end of the story. Naming and commenting conventions are even more important to clarity. The human brain is quite good at taking stuff that's in an unexpected form and interpreting it. This is why the diversity of accents and handwriting rarely causes issues, while the diversity of languages requires specialized translators. For someone used to typing on a QWERTY keyboard, it takes some time to get used to typing on a Dvorak keyboard because you have to rewrite your muscle memory. But it's nothing like how hard it is to learn another language. Russian script is roughly as different from English script as Russian grammar is from English's, but it's drastically easier to learn to read, handwrite, and type Cyrillic than it is to learn Russian grammar. So, uniformity of style is good, but it's not sufficient. The style must be readily readable and writable. ### Formatting > E1: No code should extend beyond 80 characters, most within 55. Very wide code is unreadable. Even before you get to the readability threshold, it's useful to keep your code relatively narrow. Too narrow of code, on the other hand, results in too tall of code. What these two have in common is that they make it hard to navigate the code. Code navigability is the ability to find the part of the code you're looking for. This has several inputs: - how far away is what you're looking for? - how many distinct places in the code can you have on screen? - how much code can you have on screen? - how fast can you recognize that you're in the correct place? Some of these are strongly affected by workflows. We can't force a developer to have an effective workflow, but we can support those who do. If your workflow is deficient in one of these areas, consider improving it. One of the primary inputs to code navigability is how much of it you can have on your screen. Code that is too wide limits the number of columns of code. Some people keep only one column on their screen, so they don't see a problem with long lines of code. The most efficient navigation is to already be there (i.e. to only have to move your eyes). If you only have one column of code open, you can only be in one place, which eliminates the best form of navigation. Dually, code that is too narrow limits the number of lines you can have on a screen (this isn't a separate precept simply because we've never had to tell someone to write _wider_ code). This suggests there is some happy medium. This is dependent on the language, but in Hoon we've found that up to 55 columns is a pretty good number for most code. However, there are plenty of legitimate reasons to go a bit longer than that. 70 columns is rare, but not surprising. There is essentially never a reason to extend beyond 80 columns. > E2: Tabs aren't real. Code is a text file, and text files are WYSIWYG. The reader should view the file exactly the same as the author wrote it. This is a low-overhead way of avoiding editor-dependence in code, which is as bad as browser-dependence in javascript. Sure, tabs aren't as big a deal as the hell that front-end devs have to deal with, but there's no reason to give an inch to the forces of platform-dependence. Some argue in favor of "customization". Customization is inconsistency. There can be good reason for inconsistency in some areas, but not in code formatting, which is such an intensely communicative action. > E3: A text file is a series of lines, each ending with a newline character. There are several competing definitions of text files, but most commonly-used software uses this one. It's simple and consistent. Its disadvantage relative to "a text files is a series of characters, possibly including newlines" is that there are some arrangements of characters that are not valid text files (ie non-empty and not ending in a newline). However, some series of characters were already not valid in a text file — eg those including control characters or other nonrepresentable characters. In return, we get the advantage that we can treat a text file as having a more complex and useful structure — a list of lists of characters, rather than simply a list of characters. These are theoretical concerns — the real answer is that it's the way it's always been done, and deviating from that without a good reason is asking for trouble. ### Names and comments > E4: One block comment is better than interleaved comments on every line. Interleaved comments are easy to write because you can just paraphrase the code. This is also why they're generally useless. In the micro, code should always be immediately understandable. In the macro, it's often helpful to give context as to the purpose or constraints of the function. Sometimes it's good to even give the algorithm line-by-line in pseudocode, but mixing the pseudocode with the real code obfuscates both. > E5: Don't name anything you don't have to. > E6: If it's the same thing, give it the same name. It is often said that there are two hard things in computer science: cache invalidation and naming things. This is a good aphorism. As often, someone will respond with "There are two hard things in CS: cache invalidation, naming things, and off-by-one errors". This is a weak joke because off-by-one errors are not hard at all. They're mostly a matter of discipline, and there's a couple of conventions that help. If you're looping on a zero-indexed variable, use `i < num_loops`. If you're looping on a one-indexed variable, use `i <= num_loops`. However, cache invalidation and naming things are genuinely hard because they must be solved ad hoc. There is no one-size-fits-all cache invalidation strategy, and possibly there never will be. Similar for naming. So you must actually comprehend and work within each individual context. You shouldn't solve hard problems if you don't need to, so don't name anything you don't have to. Giving unique names to things is tiring, and it reduces readability. A consistent pattern for naming things makes it easy to keep track of what is what and reduces the number of decisions you must make. Examples: - If the only difference between three things is that they're not the same, consider naming them `thing-1`, `thing-2`, and `thing-3`. - Use agglutinative morphology. If `a` is to `b` as `c` is to `d`, then find an affix which describes this distinction (say `-suf`) and rename them to `a`, `a-suf`, `c`, and `c-suf`. - A corollary of the above is that if one variable represents the "same" thing with a different type, that type should be the morpheme added. For example, if `a-set` is a set, then `~(tap in a)` should be called `a-list`. - Don't abbreviate (E8) — everyone abbreviates differently, and bits are cheap. If it takes you a long time to type the name, consider (1) finding a more succinct name (good any time you can do it), (2) becoming a faster typer, or (3) learning autocomplete tools, such as snippets. - Especially don't abbreviate or otherwise change the word when adding affixes. Inflectional morphology is bad in such an ad hoc context as code. > E7: Humans are good at memorizing. We're really good at memorizing sets of things, like runes. We're also good at inferring structure from patterns. We're bad at memorizing structure in apparently unrelated names, though, and once we see a pattern, we assume it's densely populated. So don't be afraid to use new names for new concepts. If concepts are siblings, they needn't necessarily have a direct connection in their name, but if they're hierarchical, then something should suggest that. This can be syntactic or semantic.
### F: Real Software Not all software matters. Or at least, not all software comes within the scope of "Urbit". Urbit cares about "Martian software". > F9: Timeless software is good software. There's different sorts of software. A lot of software is inherently ephemeral, and can really only be run in one place. A SAAS company runs bespoke software on a particular set of servers, and that software has no future. It will never be run on anyone else's servers, and it will eventually stop being run even there. It won't even have any direct descendents. It's a genetic dead end. However, not all software is this way. Urbit is interested in software that can live forever. This is what we call "real" software. This is the most important passage in all that has ever been written about Urbit: > What is Martian code actually like? There are two possibilities. > > One: since Earth code is fifty years old, and Martian code is fifty > million years old, Martian code has been evolving into a big ball of mud > for a million times longer than Earth software. (And two million times > longer than Windows.) > > This hypothesis strikes me as possible, but implausible. Since the _big > ball of mud_ expands indefinitely, Martian code would therefore be so > large and horrible that, despite its underground installed base, the > server room bulged into space like a termite mound, intercepting > low-flying asteroids and stinking up the solar system all the way to > Pluto. Our latest space telescopes would surely have detected this > abominable structure - if not, in fact, collided with it. > > Two: therefore, at some point in Martian history, some abject fsck of a > Martian code-monkey must have said: _fsck_ this entire fscking ball of > mud. For _lo_, its defects cannot be summarized; for they exceed the > global supply of bullet points; for numerous as the fishes in the sea, > like the fishes in the sea they fsck, making more little fscking fishes. > For lo, it is _fscked_, and a big ball of mud. And there is only one thing > to do with it: _obliterate_ the trunk, _fire_ the developers, and hire a > whole new fscking _army_ of Martian code-monkeys to rewrite the _entire > fscking thing_. > > This is such an obvious and essential response to the _big ball of mud_ > pattern that, despite the fact that we know nothing about Mars, we can > deduce that it must have happened on Mars. Probably several times. > Probably several hundred. For each of these attempts but the last, of > course, the result was either (a) abject failure, (b) another _big ball > of mud_, or (c) both. > > But the last, by definition, succeeded. This is the crucial inference we > can draw about Mars: since the Martians had 50 million years to try, in > the end they must have succeeded. The result: Martian code, as we know > it today. Not enormous and horrible — _tiny and diamond-perfect_. > Moreover, because it is tiny and diamond-perfect, it is perfectly stable > and never changes or decays. It neither is a _big ball of mud_, nor tends > to become one. It has achieved its final, permanent and excellent state. We don't know that Urbit will achieve this final form, but Urbit is an attempt to make progress in that direction. Any software that isn't a candidate for "tiny and diamond-perfect" is either a prototype or scaffolding. Here's some characteristics of real software: > F1: If it's not deterministic, it isn't real. If you do the same thing twice, your computer should react the same way. This is comforting. This is also what makes it easy to reason about and use effectively. If you're not sure what your computer will do, you'll be afraid of it and act defensively toward it. This inevitably leads to a big ball of mud. Diamonds are extremely hard, and they wear very slowly. A diamond surface will support you every time; a mud surface will shift out from under you exactly when you put weight on it. > F2: If it's centralized, it isn't real. Code that depends on a central authority or has a single point of failure is fundamentally brittle. No authority lasts forever, no company lasts forever, no server lasts forever. Under certain circumstances, bones and the like can be preserved for a long time — but not alive. The remains of buildings sometimes last for thousands of years, but the cities are gone. The only things that last forever in their useful form are big hunks of rock (like the pyramids at Giza) and ideas, through writing and transcription. So, if you depend on the sun existing, this is acceptable. If you depend on a piece of source code to be kept forever, copied and re-copied, you're fine. If you depend on a 20-year-old company existing and being easily accessible, this is brittle. Another way to look at this is to say that anything that wouldn't work on a spaceship is architecturally flawed. Maybe it requires many sequential round-trips to a central server because it doesn't have all the relevant information. See discussion of A7. > F3: If it's owned by someone else, it isn't yours. > > F4: If it's managed by someone else and you can't change who that is, it > isn't yours. > > F5: If it's not yours, it isn't permanent. > > F6: If it's not permanent, it isn't real. Nobody is as aligned with your values as you are. If you don't control the software, then you can't guarantee you'll always have access to it. Whoever does control the software may destroy it, move away, or simply revoke your access to it. Control has some nuance. If it's physically controlled by a company and you can't get your data and the software that makes it useful, you definitely don't have control. If you have physical control of it, you do have control. But if you're hosting the data and software on a commodity cloud server, you still mostly have control of it. Your privacy has been reduced, sure. However, as long as you keep regular backups of the data and software, if the hosting provider disappears you just need to find another host, or host it yourself. Any particular host is likely to disappear, but the situation where _every_ host disappears and you can't even host it yourself is much less likely. Brands are mortal, commodities are not. Some branded hosts allow you to download a zip file of your data. This gives you a little more control over your data, but it's much closer to "no control" than "total control". This is because your online world is data plus code, and without the code that makes it useful, data is only good for scrapbooking. The scrapbook will remind you of the good old days, but you won't be able to live them again. > F16: Sovereignty necessitates understanding. Not everyone cares to understand the software they're using. But if you want to compute sovereignly, you must understand what your computer is doing. If it regularly surprises you, you're not in control of it. You must tame it. The current paradigm is a wild monster which cannot be tamed by mere humans — only megacorporations even attempt to do so, and it's always temporary. So, your software needs to be simple enough that it can be understood. > F10: System modules should be designed to be frozen permanently; > conversely, Urbit consists of that system software which admits of a > Kelvin version. "Kelvin versioning" is a versioning scheme where your current version is a natural number, and new versions count down. Once you hit version 0, no more changes should be necessary. This is the opposite of the usual way of developing software, where there is no defined endpoint. This paradigm suggests that you're building something specific. Most current software builds higher and higher in a vain attempt to reach the heavens, until all available effort is expended just to keep this tower from falling over. Kelvin versioned software cuts away until you reach a strong bedrock. Not every piece of software should have a Kelvin version, but infrastructure should. > F11: The two procedures for achieving timelessness are distillation and > generalization. When building a timeless system, you start by building whatever you need to make it work. Then as you continue on, you may find some of the features are no longer necessary. Often, they're only needed when you're using them for user-facing work, but you've already moved on to building user-facing stuff somewhere else. You don't need the niceties anymore, so you should remove them. Other times, you find that you want something similar to a feature you already have, but not exactly the same. When this happens often enough with the same feature, you may discover what is the essential nature of the feature and what are simply ornaments of the original use of the feature. Then, you can generalize the feature so that each of the desired use cases is well served. > F13: That which cannot be Kelvin-versioned should update fluidly and > automatically. Not everything should be Kelvin versioned. Much user-facing code is in a continual process of discovery, and it must react to changing userbases and their needs. These should update without user interaction. You should be able to turn off your ship for years at a time, then turn it back on. After a few minutes or hours of catching up, you should be ready to go. This means you need upgrade paths to be permanent, and they must not lose information. Anything that stops being able to upgrade is not permanent, since it will succumb to bit rot. > F14: Humans are interactive and temporal; technology is permanent and > exhaustive. This is why the software humans touch directly should not be Kelvin versioned but the infrastructure that makes it work should be. > F7: The way it is isn't always the way it ought to be. You should presume that there's a good reason for why things are done a certain way because what works best tends to win in a market in the long term, but markets are rarely efficient. Markets become more efficient over time, if they become more liquid, and as they become larger. Software is young, the cost of switching technologies is often large, and the space of software paradigms yet to be explored dwarfs those we've exhausted. So don't be surprised if you find ways to improve the situation drastically. > F8: The way it ought to be isn't always the way it is. On the other hand, sometimes the elegant solution just isn't right. The ultimate test of the quality of software is whether it's useful to humans. Humans don't fit nice equations, so the timeless truths of mathematics are often mismatched with human convenience. Never sacrifice real utility for the sake of making the code more elegant. > F12: Standardization is better than perfection. Coordination problems are some of the most difficult. When given a choice between compatibility and improvement, carefully consider whether the problem solved by the improvement is weightier than the problem caused by the incompatibility. > F15: Communities are autonomous. This is both a positive statement and a normative one. Communities are in fact a thing separate from their platform, but sometimes changes in the platform can kill them. That's bad. It's slightly different from freedom of association, since that's a freedom that applies to individuals, not the communities themselves. Communities are the basic building blocks of society. We are each a member of some number of communities, and most power is held by these communities. Some communities coincide with family ties, some with common goals, beliefs, or interests. Some may be freely joined by just showing up — others are nearly impossible to join without having been born into them. Throughout civilization society has had many forms. Governments have ranged from police states to feudal societies to democracies to informal tribes of people. In each of these, the fundamental unit is the community. Humans don't live on their own, and they don't live in one universal society. They are a part of some communities and not of others. Technology shouldn't force us into a single global community. Discoverability isn't an unmitigated good. Most communities should be closed off from each other — that's what makes them a community. This allows them to develop their own ideas and norms. These bleed into other communities and may spread if they provide some benefit. Many technologies today combine us into a single community, or at least a single set of norms. If many communities are hosted by a single entity, there will be pressure from some of those communities toward the entity to pressure other communities to follow certain of their norms. This crosstalk isn't all bad, but there must be an equilibrium. All too often the lines between the communities are so permeable as to be inconsequential. In this case, the communities will converge in a single large community, at the cost of the memetic diversity of the original communities. Technology should mirror human interaction, and humans interact in communities. ]]> https://urbit.org/blog/precepts https://urbit.org/blog/precepts Tue, 17 Mar 2020 00:00:00 GMT Urbit is a principled approach to system design and programming, but it's often not obvious what those principles are. We have many specific terms for the different parts of the project: Arvo is the OS, Hoon is the language, Tlon is the company, Azimuth is the identity system, Ames is the network. But the word most associated with the project is "Urbit", and it's not clear what technically it refers to. It's only a small stretch to say that "Urbit" _is_ this set of principles, and that if anybody follows these principles strictly they will create a system that is isomorphic to the Urbit. Some of these are commonly held among many software projects, and some are not. Some are only debatably better than the alternatives, but Urbit exclusively chooses them. We’ll start with a brief description of many principles, then we’ll go into a long-form justification.
### A: General Design 1. Data is better than code. Store data in your state, send data over the wire, dispatch based on data. 2. Everything should be CQRS. 3. (Almost) Everything should be pubsub. 4. A subscriber shouldn't affect a publisher. 5. Communication between nodes should be communication between independent actors. Each message should do one complete thing, and there shouldn't need to be a sequence of coupled messages. 6. Represent your data as closely as possible to the essential structure of the problem. 7. A client's representation of data should be as close as possible to that of the server. This blurs the distinction between client and server. It allows offline-mode, reduces communication to syncing, and decentralizes. 8. When mating different paradigms, build one cleanly on top of the other. Never try to make them work on some of the same primitives. Never abuse one to make the other work. For example, ducts in themselves are very general - if you want to do pubsub, that can easily be built on top of ducts, but don't pretend that pubsub is a _part_ of the duct system. 9. Never misuse an abstraction. An abstraction provides a certain set of tools; use them and only them. 10. Correctness is more important than performance. 11. Be simple and uncompromising in defining what's correct; go crazy with optimizations. Nock is a great example of this. It contains the character of the virtual machine, but its asymptotics are bad. Add jets to fix the asymptotics. Another example is the ACID nature of Arvo. Arvo is a pure function f(logs) of its event log, so formally Arvo is just a function run against an event log. A naive implementation has very bad asymptotics; processing each new event is O(n) in the number of historical events. Choose the function g(state,log) such that f(logs ++ log) = g(f(logs),log). Then, as long as you keep the state in memory, processing each new event is constant in the number of previous events. This still requires O(n) restart from disk, but you can also periodically (and non-blockingly) write a checkpoint of the state to disk, so that restart from disk is only linear in the number of events since the last checkpoint. 12. Correctness is more important than optimality. 13. If you don't completely understand your code and the semantics of all the code it depends on, your code is wrong. 14. Deterministic beats heuristic. Heuristics are evil and should only be used where determinism is infeasible, such as in cache reclamation. 15. Stateless is better than stateful. 16. Explicit state is better than implicit state. 17. Referential transparency is honesty and stability. Lack of referential transparency and other forms of disingenuousness are some of the world's big problems. Only deviate from referential transparency if absolutely necessary. 18. Responsibilities should be clearly separated. This applies from kernel modules through network citizens. 19. Dualities must be faced head-on and analyzed differently at different layers. Statically typed vs. dynamically typed, imperative vs. functional, code vs. data, and effectful vs. pure can all be a matter of perspective, and all relevant perspectives must have coherent answers. 20. One hundred lines of simplicity is better than twenty lines of complexity. It's not enough for an abstraction to reduce code duplication; it must actually make the code simpler. 21. Prefer mechanical simplicity to mathematical simplicity. Often mechanical simplicity and mathematical simplicity go together. 22. The Law of Leaky Abstractions is a lie; abstract airtightly. If your abstractions are leaking, it's not due to some law of the universe; you just suck at abstracting. Usually, you didn't specify the abstraction narrowly enough. 23. Some cliches are repeated because they are true; others must be repeated because they are not.
### B: Specific Design 1. Always ack a dupe; never ack an ack. It's okay to ack a nack as long as you never nack a nack. 2. Never construct or deconstruct a duct. 3. Route on wire before sign, never sign before wire. 4. Only go from statically typed to dynamically typed if you must. Once you go into a dynamically typed world, it's hard to go back. Statically typed is better than dynamically typed, so if something can be made static, it should be. Each timeless data structure is a brick in the foundation of digital civilization. 5. Anything with business logic speaks one paradigm; anything that translates paradigms has no business logic. 6. Functionally enforced semantic rules are better security primitives than memory access restrictions. A compiler can use type safety as an effective tool to enforce some classes of security guarantees.
### C: Attitude 1. Code courageously. If you avoid changing a section of code for fear of awakening the demons therein, you are living in fear. If you stay in the comfortable confines of the small section of the code you wrote or know well, you will never write legendary code. All code was written by humans and can be mastered by humans. It's natural to feel fear of code; however, you must act as though you are able to master and change any part of it. To code courageously is to walk into any abyss, bring light, and make it right. 2. No time for lazy people. If there's clearly a right way to do something and a wrong way, do it the right way. Coding requires incredible discipline. Always follow conventions, and fire anyone who won't. Anything that can be solved by discipline is not a real problem. > "To him who knows to do good and does not do it, to him it is > sin." 3. When a smart person makes an obviously stupid suggestion, before responding take a full 60 seconds to envision how you would implement it and what the effects would be.
### D: Theory 1. Academia is a succubus. Academia has a really high ratio of smart people to useful products. Academia has many genuinely smart and interesting ideas, but always remember that pursuing them whole-heartedly will never result in a useful product. 2. Never fear math. Academia is inefficient, but it's also correct. Those people wasted a lot of time finding the right answer, but now that they've done it, you must exploit it. If you don't know the theory, learn it. No excuse for being "bad at math". Most who are "bad at math" are simply lazy or fearful. Be strong and of good courage. If you're truly so bad at math that you cannot learn it, you should not design systems. 3. Not being qualified to solve a problem is no reason not to solve it. If you don't know how to do it, do it anyway. "We are too young to realize that certain things are impossible. So, we will do them anyway." 4. The best way to get the right answer is to try it the wrong way. You cannot design a good system without spending most of your time in contact with the problem. The bad parts about systems are rarely obvious in theory, but often in practice. 5. Practice tells you that things are good or bad; theory tells you why. Never use theory to design a good system from scratch; only practice can tell whether the system is good. 6. Everyone must regularly alternate between theory and practice. There is an eternal wheel of system design. Given a system, use practice to discover what is bad, missing, or unnecessary. Then, use theory to fix the bad, add the missing, and remove the unnecessary. Repeat.
### E: Text Style 1. No code should extend beyond 80 characters, most within 55. 2. Tabs aren't real. 3. A text file is a series of lines, each ending with a newline character. Corollary: all non-empty text files end with a newline (often hidden by editors). An empty text file does not end with a newline. 4. One block comment is better than interleaved comments on every line. 5. Don't name anything you don't have to. This includes conventional names for variables of particular types. For example, the conventional name for a variable of type `path` is `pax`. You should have a good reason for using any other name for a `path`. A more recent convention that follows the same principle is to give the variable the same name as its type. 6. If it's the same thing, give it the same name. 7. Humans are good at memorizing. 8. Abbreviations aren't worth it.
### F: Real Software 1. If it's not deterministic, it isn't real. 2. If it's centralized, it isn't real. 3. If it's owned by someone else, it isn't yours. 4. If it's managed by someone else and you can't change who that is, it isn't yours. 5. If it's not yours, it isn't permanent. 6. If it's not permanent, it isn't real. 7. The way it is isn't always the way it ought to be. If you can make it the way it ought to be, do so. 8. The way it ought to be isn't always the way it is. Some things can't be made more elegant. For every ailment under the sun There is a remedy, or there is none; If there be one, try to find it; If there be none, never mind it. 9. Timeless software is good software. 10. System modules should be designed to be frozen permanently; conversely, Urbit consists of that system software which admits of a Kelvin version. 11. The two procedures for achieving timelessness are distillation and generalization. Every temporary or contingent feature in a system module should eventually either be removed (distillation) or relaxed into timelessness (generalization). For an example of the result of both procedures, consider the abstraction of the core in Nock and Hoon. 12. Standardization is better than perfection. Kelvin versioning stops when we run out of versions, not when every defect has been rectified. 13. That which cannot be Kelvin-versioned should update fluidly and automatically. All incompatible upgrades should provide permanent and automatic upgrade paths. This is principled permanent backward compatibility. 14. Humans are interactive and temporal; technology is permanent and exhaustive. 15. Communities are autonomous. 16. Sovereignty necessitates understanding. If you don't understand a system you're using, you don't control it. If nobody understands the system, the system is in control. ]]> https://urbit.org/blog/urbit-for-normies https://urbit.org/blog/urbit-for-normies Tue, 11 Feb 2020 00:00:00 GMT ## What it looks like Another way of explaining Urbit is to describe how it feels and looks. Visually, Urbit belongs to a genre often called “calm computing.” Urbit is designed to be simple, basic, and non-addictive. You won’t find any flashy-blinky lights here; no zany colors; no wacky branding. Just calm, functional, purpose-built tools for people to connect and work together. (One of my favorite examples of calm computing is the [light phone](https://www.thelightphone.com/).) Today Urbit’s interface looks like a tab in your web browser that has some limited functionality (chat, a weather app, a notebook, link sharing). Soon, you won’t even need the web browser — you’ll be able to access or run Urbit directly in its own window. Inside will be a super clean interface with all the functionality you might want from your computer, in one place and with one login (think document creation, email, social media access, accounting) — but it will run on your virtual Urbit computer, so it will be safe from corporate prying and data leaks, available on any device, and all inside one simple interface. ## What it’s for Fundamentally, Urbit is neutral open-source infrastructure, so it can be for many things. From our highest level perspective Urbit is computing for real communities and normal people, not for corporations. To make that real, Urbit focuses on delivering a computing experience that is secure, private, durable, and calm. For me, Urbit’s core purpose is to give us all back our privacy and our autonomy. It’s time for humanity to control its tools rather than the other way around. ## Key elements One potential barrier to understanding Urbit is that we’re building out multiple layers of this new internet simultaneously, and each has its own explanations and value propositions. Remember, what sets Urbit apart is that we’re building of all these elements at once to work together as a system. I’ll try to explain the most important of these without straying too deep into the technical weeds. 1. **A peer-to-peer network** Peer-to-peer networking is a really big deal. On the current internet, essentially all of our communications and activities go through centralized servers owned by corporations (Facebook, Google, etc.) — and those corporations can’t help but use all that information to make money. They’re capitalists, after all, and so we get constant privacy violations and interfaces built for maximum dopamine addiction. In an Urbit world, all those communications go directly between users (i.e. peer-to-peer) with nobody in the middle. That means there’s no data capture by any megacorp, less intrusion into your life online, and communications go back to the semi-private one-to-one or small group format that is most comfortable for humans. It’s more like real life. 2. **Your own server** Servers are complicated. Too complicated. Have you ever tried to use a command line? Exactly. It’s a terrible way to work, and that’s just a hint at how complex servers are to operate. To hide that complexity, we let corporations run our servers for us — they can afford teams of engineers. The problem is that today most of our computing is handled by these corporate servers — and that means we’re giving up control because of complexity. As an example, every time you use Siri, your message is sent to Apple’s servers, where it is processed, and the response is sent back to you (which is admittedly amazing). Unfortunately that’s why it never works if you’re offline, and it’s also why Apple knows what you’re saying to Siri. And this applies to all the apps we use: The actual computing happens on corporate servers. The problem is that we consumers aren’t in control and, again, the corporate entities watch, record, and gate-keep all of our activity. Fundamentally, Urbit is a super simple server. That means Urbit can run those computations and hold all that data for you, eliminating the need for a corporate intermediary. 3. **Permanent, personal ownership of your digital identity and all your data** Urbit IDs are permanently ownable — and that ownership is backed by cryptography, not some corporate database. Once you take possession of your Urbit ID, no central authority can ever take it away from you. Sure, if you start spamming people on the network, word could get around and you might get blocked by other users, but none of them can prevent you from using your ID or accessing your Urbit computer. Compare that to the current internet where Facebook owns your Facebook profile, Google owns your Gmail account, Twitter owns your Twitter handle, etc. At any time and for any reason, those services can arbitrarily shut off your access, and you have little to no recourse — all that data, all those followers, and all that reputation is wiped out instantly. Your Urbit, on the other hand — both the identity, and all the data inside your Urbit computer — is yours permanently, so long as you know your secret key. A secure decentralized identity system like the one we’ve built leads to some very interesting use cases that are otherwise insurmountable. For example: decentralized Uber, decentralized delivery services, and a solid solution to the problem of fake reviews. To build any of these services, the problem of permanent identity needs to be solved first, and we’ve done that as part of our overall system. I’ll spend more time on use cases below. It’s worth noting that this problem has never been solved before. All of our communication channels on the internet today rely on someone else to authenticate our identity. Urbit is — to my knowledge — the first practical application of a decentralized ID system that actually works. 4. **An incentive for good behavior** On the Urbit network, identities are finite, and they accrue reputation. Compare that to the current internet where identity is infinite (anyone can create as many email addresses or social media profiles as they want, essentially for free) and so spam, scams, and DoS attacks (“denial-of-service”: when many disposable identities ‘attack’ a server at the same time) are rampant. On the Urbit network, there are just over 4 billion total IDs (2³² to be exact). Scarcity creates value. That scarcity, combined with reputation accrual, means there's an incentive for good behavior. You might think 4 billion IDs sounds like too few given how many humans we have on the planet. I won’t go into the technical details here, but if we ever get to the point where that’s a problem (which would be a great problem to have), we’ve outlined several solutions that allow everyone alive to get on the network. We don’t intend to lock anyone out. 5. **Tools anyone can learn and use** Urbit is a system that anyone with patience can understand completely from top to bottom, from the integers of byte code to the complete OS and everything in between. Urbit in its entirety is just over 50,000 lines of code. That is tiny. Compare that to Linux (the open-source OS that most modern computing is built upon), which is tens of millions of lines of code, and you start to see how important this is for efficiency. Apple’s and Windows’ codebases are similarly massive. In our current computing system, nobody can understand the entire thing completely, and that means there are innumerable security holes. With Urbit, while it may demand something of you, any reasonably intelligent developer can grasp the whole thing. We’ve worked very hard to keep our system simple at each level. That makes it more efficient, more durable, and much more secure. 6. **Lessons from the last half century** Bear with me — this is where some jargon kicks in. In many ways, Urbit is an attempt to apply lessons learned from the failures of 20th century computer development. From an engineer’s perspective, Urbit solves a series of problems that we end users rarely notice, but which cause extraordinary amounts of complexity and waste. The list of applied lessons is long, but rather than bullet them all, I’ll explain the one that surprised me the most. Urbit is deterministic. That is, a given sequence of inputs always produces the same output. When I first heard that, I thought, “Wait, isn’t that what all operating systems do? Give you the same result every time?” Weirdly, the answer is no. Normal operating systems don’t work this way. Normal modern operating systems actually get varying results when they compute things. This is a deep problem that causes programmers countless hours of frustration. The most common implication of this that you’ve probably experienced is having to reboot your computer — either because it’s locked up, or because it simply won’t do something that it was totally doing a second ago. What’s annoying to a regular user, though, is absolutely crazy-making to developers. This determinism makes programming on an Urbit immensely more efficient than other environments. Urbit applies this and a whole list of other lessons from the past half-century of systems software to increase reliability and security in everything from file storage to networking to user-facing apps. All this adds up to more security and reliability and less service-lock-in with Urbit than with our current systems. 7. **Private, human-sized, and efficient** Because Urbit isn’t centralized or owned by any one company, there’s no incentive for it to dominate your life. Your Urbit isn’t designed to spy on you, get you addicted to clicking its buttons, manipulate your emotions to gain attention, or nudge you to buy something. Your Urbit is a simple, customizable, permanent place for you to do all your computer-based tasks with your friends in absolute privacy. This is so much the case, in fact, that it presents a bit of a problem for Tlon. We don’t know how many people use Urbit every day, who they talk to, or what they’re doing. While we think this is how things should be, it puts us in a bind when a VC wants to know about our user base; we can only make educated guesses. That said, we think this is the right problem to have. What all this adds up to is that the Urbit network is immensely more resource efficient, secure, easier to customize, and smaller in size than any other computing system in the world. If you care about data security, privacy, durability, or even just elegance, Urbit has the best design going.

## How it compares A final way to understand Urbit is to make comparisons between Urbit and current products we use today. This is limited, because ultimately Urbit will grow in directions we can’t predict, but comparisons are a useful starting place. Again, on the current internet, we all access megacorp software using our web browsers and app interfaces. This means that we each exist as a line in a megacorp database, and those megacorps use that data to make money, which results in rampant privacy violations and addictive interfaces. Urbit is the alternative to all that: it’s decentralized, private, and secure. And it’s yours. Let’s get to some real-world examples. ## Urbit vs. Reddit Imagine you’d like to discuss a certain research topic with a group of friends. Say, Canadian small cap equities or the best cookie recipes. You’ll need a chat interface for live discussion, a notebook to record concepts, and a way to share links for discussion. You can do this in the default web if you want: use Reddit for the links (but make sure it’s a private group), use Google Docs for the notebook (but make sure everyone has Gmail), and use iMessage for the chat (but everyone needs a Mac). And don’t forget, each service has their own login and security holes, and they’re all mining your data. The alternative is spin up a group on Urbit. All of that functionality is available today in one clean, simple interface. It’s private, you own the data, there’s only one login, and you can even learn to build apps for your group. ## Urbit vs. Facebook, Twitter, etc. Or let’s say you want to share pictures with your friends, post micro-messages to the world, write a blog, or share a podcast. There’s simply no reason you should use some centralized company (one that then owns your content) to broadcast any of those things. Instead, send it out from your Urbit. You’ll own the content, you’ll control the data, no company will spy on you, and nobody can ever take your ID from you. With Urbit we all finally own the tools, and that means we own the data by default. ## Urbit vs. Nest Imagine, as I do, that you want a connected video camera in your house, but don’t want the Nest corporation (which is now owned by Google) to be able to spy on you. Or perhaps you just don’t want to have to sign up for yet another duplicative service in order to use the camera you just bought. If any of that is the case, you can run your camera through your Urbit instead. The video feed will never leave your control, because it no longer has to live on Nest’s servers. Your camera will talk over a secure channel only to your Urbit. Your Urbit is the server. You can apply this same concept to smart speakers, a Ring doorbell, connected baby monitor, smart fridge, or any of the innumerable connected devices that give someone else access to your personal life. Why should anyone else have access to that information? ## Urbit vs. Google Docs Imagine you want to write a screenplay with a small group of friends, but you don’t want Google mining the content of your shared doc for advertising keywords, and you won’t want to use multiple services for chat, document sharing, email, etc. In that case, simply create an Urbit group with your friends and write together using Urbit’s writing tools. Your content will stay completely private to the team until you decide to release it to the world, and you’ll communicate using one simple interface. Even better, the archive of your multimodal work lives in one place, so looking back at your process doesn’t require reconstructing it from dozens of different apps. And everything is stored on your respective Urbit computers, not Google’s servers. ## Urbit for your protected health information Imagine, as is already becoming the case, that it becomes highly valuable to have your genome sequenced in order to make good healthcare decisions. In that scenario, most of us would want our health care providers to be able to make precise assessments of parts of our genome, but we wouldn’t want insurance companies or other third parties to have access without our permission. In the current paradigm, once you share the information, it’s out there and can be leaked or sold. So much so that we all sort of assume that leaks are the default. In the Urbit paradigm, that information lives on your Urbit computer and nowhere else, and any software necessary to process it can be run on your Urbit as well. In other words, there’s never any need to share the raw data with anyone; instead, you can download a small program, run it on your Urbit, and share the final analyses only without ever exposing your data. Some of these cases are, admittedly, a bit speculative since we haven’t built all the functionality yet. But it’s coming, and quite soon. For now, if you like the way this sounds you can [get yourself an Urbit ID](https://urbit.org/getting-started/#id) and start learning about how to safely store cryptographic keys (it’s really not that hard, and we give you suggestions). If you’re feeling technical or adventurous, you can try booting our beta OS interface to see what it looks like. It’s fussy, but it exists. Soon it will be less fussy. You can check out our [roadmap](https://urbit.org/understanding-urbit/roadmap/) if you’d like to know when. ## Why we need Urbit We humans haven’t evolved enough to manage the firehose of information that modern technology blasts us with. There’s too much news, too many connections, too many opinions, and more agendas out there than any of us can mindfully comprehend. We’re designed for much smaller communities, one-to-one connections, and slower idea assimilation than our modern infrastructure allows. The pace of change is increasing, and we don’t know how to manage it. We’re scared, and to manage that fear we’re becoming more insular, more entrenched. Layer on top of all that the incentives that big business and big politics have to manipulate our emotions, and we’re headed in a very dangerous direction. We can’t turn back the technological clock; humanity can only ever press onward. This is where Urbit comes in. Urbit is a forward-looking solution to this particular set of problems around human-scale computing and personal autonomy. Urbit provides a means by which groups can live peacefully in parallel. Urbit is a computing environment that allows humans to do everything they want on computers, but in communities of their choosing, in a way that feels direct and natural, and in a way that allows true personal freedom. Urbit is calm, secure, and private in its most fundamental architecture. No spying corporations. No manipulative politicians. And that is what we need. Humans need a way to at once take advantage of our vast technological advances but not be dominated by them. Urbit is an open-source tool that we all own, that anyone can understand down to its roots, that’s fully under its user’s control. It is human-scale computing for human-scale communities. Simple, durable, yours. For me, this first iteration of Urbit is more than enough, and it’s exactly the sort of peaceful future I want to work towards. ]]> https://urbit.org/updates/2020-02-06-update https://urbit.org/updates/2020-02-06-update Wed, 05 Feb 2020 00:00:00 GMT https://urbit.org/blog/creating-sigils https://urbit.org/blog/creating-sigils Mon, 03 Feb 2020 00:00:00 GMT  One of the basic building blocks of Urbit is Urbit ID, our naming and identity system. Your Urbit ID is a short, recognizable name that’s also a network address for your Urbit OS instance. (You can find a complete, high level description of Urbit ID [here](https://urbit.org/understanding-urbit/urbit-id/).) Your Urbit ID is meant to feel a bit like a secret code name. It doesn’t leak any information about the real you — but it’s still memorable and recognizable. Getting an Urbit ID should feel exciting, like getting a brand new identity in a new world. From a design standpoint, some of this is accomplished simply by having a system for the names themselves. Under the hood, Urbit IDs are actually just numbers. There has always been an algorithm for turning these numbers into pronounceable names like `~tacryt-socryp` (13,304,832) or `~litzod` (1,280) or `~nes` (212). While the names are great, they’re not quite enough, especially in a rich interface. Each Urbit ID really needed some form of visual representation, image, or crest. A digital identity needs to be something you can really attach to, both yours and those that belong to your friends. There are 2³² or 4.2 billion unique Urbit IDs. Clearly there’s no way that we were going to do this by hand. And purely algorithmic solutions can often produce disappointing, undifferentiated output. Regardless, we set out to tackle this problem. By the end of this process, every Urbit ID got its very own ‘sigil.’ In this post we’ll walk through how we thought about the problem, iterated through possible solutions, and the toolchain we used to arrive at the final result. ## Background What are the basic primitives of a visual language? There aren’t any completely authoritative answers to this question, of course. But we knew that to produce such a large number of visually distinct items we’d need some way to reason about form that’s closer to letterforms. So we started by collecting influences. The ‘chop’ or [seal]() is conceptually pretty close to what we wanted to get to.

[Kamon]() are even closer — especially since they have very little visual complexity.

We also spent a good amount of time with both Anni Albers and Agnes Martin. Both of these painters had an incredible mastery of the grid as a means of creating engaging visual work. These paintings all have a wonderful tension between an underlying system and their specific expression.

Frank Stella is a master at producing incredibly dynamic work from only a few elements. His ability to create a sense of depth from simple overlapping elements is fantastic.

The plans of both Ledoux and Palladio are incredibly dynamic. They’re mostly gridded, use only a few elements, and use lots of symmetry — but they’re at once distinct, engaging, and fresh.

Another influence: international maritime signal flags. We like these not only because of their striking visual qualities, but also because they correspond with a letter system, just like our sigils. Each one conveys useful information; for example, “Keep clear of me; I am maneuvering with difficulty,” a flag we all feel like raising from time to time. This influence was key to the idea that pairing symbol to phoneme was a possibility. You can see the whole collection of our visual research [here](https://www.are.na/tlon/2019-sigils-research). Ultimately, we didn’t specifically draw on any of these influences very consciously. The goal of visual research is to get a sense of the territory, what has been tried, and what works. We think these threads of prior work are evident in the final result — you can _feel_ them, but it’s tough to point to them individually. ## What’s in a name? > _“Lev’s sigil appeared, strobing, as Netherton was getting out of the cab in Henrietta Street. “Yes?” Netherton asked.”_ > > — William Gibson, The Peripheral Before we get to how we worked through the problem of actually designing these things, let’s talk about the problem of naming. We went through so many names for these things. Crest, symbol, seal, tag, pictograph, stamp, emblem, signet, icon, glyph, holomark, visage, radigy, pattern, the list goes on for an incredibly (embarrassingly) long time. We argued about this, we went back and forth about it. Sigil won, ultimately, because of the way that it’s personified in William Gibson’s _The Peripheral_. In the far future, characters talk to each other by some kind of telepresence. When a call comes in from someone else, they first appear as a ‘sigil.’ It’s a strange word, but we were imagining a similar use-case. While we can’t point to any specific parts of our visual research that reappear directly, sigil is lovingly borrowed. ## Experiments + iterations So, how did we actually end up making these things? We iterated through plenty of possibilities before landing on our final approach. Let’s walk through the two we remember best.

 The first attempt was a series of images in the style of Karel Martens. Overall this technique was just too cluttered and sometimes difficult to parameterize in a predictable way. We kept iterating.

 Just like the previous example, the main idea was that we could take a set of geometric primitives, combine them, then alter their appearance along a set of parameters, like color, rotation, and size. But also just like the previous attempt, the technique of overlapping symbols made the images too difficult to read and differentiate. We wanted to create sigils that were not only great to look at, but visually ordered. Since overlapping elements didn’t seem to quite hit the mark, on we went. ## Closing in Eventually, we realized: tiles! With tile designs there are no overlaps, which in early experiments immediately started to take on the consistent, legible appearance we were after. Each element is discrete and clearly visible. Tile designs were also easier to create and control from an implementation standpoint. Best of all, tiles made it clear that we should be thinking in terms of creating representations of individual syllables rather than entire names. Urbit IDs are constructed from two sets of 256 three-letter syllables. `~fallyn-balfus` is `fal` `lyn` `bal` `fus`, for example. (There are two syllable sets in order to prevent duplicates — we always alternate between the two when composing names.) Thinking in terms of syllables greatly reduced the complexity of the problem, but there was still some work to do.

 At first, we thought about using a 4 x 4 tile grid, as shown here. These designs were tricky to implement because of the detailed lines and arcs spanning across individual tiles. In order to generate a result that matched these mockups, a program would have to generate a data model that knew about which tiles touched which other tiles. We felt this was too complicated and had an unknown performance profile. Plus, they proposed a deterministic color scheme and we felt strongly that color is personal, not to mention hard to parameterize. In the end we thought it better to create something that had the potential for user customization.

 Around this stage, we were thinking a lot about how architectural plans aren’t unduly constrained by the fact that they’re only one color. We had a feeling we could make the grid smaller and assign details to the tiles individually.

 So, we scaled it down to a 2 x 2 grid and stuck with it. It’s simple to implement, visually unique, consistent, and the color scheme is black and white, leaving room for user customization. We felt we had reached a semblance of a final result. Now all we had to do was manually draw 512 unique but consistent individual ‘phonemes’.

 As it turns out, drawing 512 consistent, abstract elements by hand isn’t easy. We had to strike a balance between authorship and automation — we wanted each tile to look well-made, but not too authored. To do so, we took advantage of Figma’s web API which helped us design in SVG but iterate programmatically. ## Final process  We took a look at our hand drawn mockups and drew each individual visual building block of the tiles separately. Then we loaded the SVG elements into a script that programmatically combined them to create trillions of layered symbols. This way, we were able to easily select candidate phoneme symbols that worked visually. We eventually added a rule system to find higher quality phoneme symbols — but this combination of programmatic generation and hand selection was working pretty well.

 The final step, using the tool we built, was to hand select individual phoneme symbols and create a complete index.

 This allowed us to start looking at possible combinations and selecting phoneme symbols based on their interplay with others. Once we had selected the final set, we went about compiling the `sigil-js` library that’s now publicly available. This turned out to be fairly technically complex due to the different ways SVG is handled between browsers and Figma — but the end result is a single Javascript library you can use to embed an SVG in almost any context on a web page. See it in action and browse the available sigils [here](https://sigil.azimuth.network). If you're technically inclined, the [source](https://github.com/urbit/sigil-js) can be found on GitHub.

 We think sigils are pretty successful. They look nice in our native Urbit interfaces, people have made them into shirts and posters and cards, even laser-cut them to make stamps. Urbit ID and the sigils are components of Urbit that are totally separate from the OS — you can use them on their own and potentially even extend their functionality. We’re happy with what we’ve done with them so far, and hope that people will experiment with what’s possible with this naming and identity system.  ]]> https://urbit.org/updates/2019-12-06-update https://urbit.org/updates/2019-12-06-update Thu, 05 Dec 2019 00:00:00 GMT https://urbit.org/blog/pki-maze https://urbit.org/blog/pki-maze Tue, 26 Nov 2019 00:00:00 GMT 32, or 4 billion planets. If the PKI data is about 100 bytes for each planet, this is about 400 GB of data. This may be more than most users wish to store, but it's small enough that it would be very cheap for someone to host this data for many users[^3]. This information is currently stored directly on the Ethereum 1.0 blockchain, but as is well understood in blockchain circles this approach will not scale beyond a certain point. Many chains are pursing designs that allow the users of the smart contracts to store locally the data associated with the contracts they care about and only commit hashes to the chain. We expect there to be several viable options for this by the time our scaling needs exceed what's provided by Ethereum 1.0. This will free us from the cost of hosting the PKI data on all Ethereum nodes, but the data must still be stored somewhere. Any service that could handle such a large amount of data would inherently centralize the network. However, 4 billion is not enough addresses for every device on the planet today, much less in a few decades. So, we apply our maxim of re-examining our choices at each level for each use case. Examining the idea maze above, we cannot use the blockchain option for everything since the data is too big. However, it's not actually necessary for each of your devices to have its own self-sovereign identity separate from your planet. So we choose the option of using a central registry: your own planet. We allocate 4 billion "moons" to each of those planets. A moon is a 64-bit address whose 32-bit suffix is its planet. Your planet can easily store the keys for its own moons, and anyone who needs to talk to your moons can ask you for the keys. This is the sense in which moons are true ships: they're permanent names and you own them completely, as long as you own the planet. However, they're not independent ships — their keys can always be revoked by their planet. To create a sane network, we require global consistency for all our names. There are three other properties; pick any two: - Comets are impermanent, self sovereign, and plentiful. - Planets are permanent, self sovereign, and not plentiful. - Moons are permanent, not self sovereign, and plentiful. Fancifully, comets are wayward celestial bodies that are great for testing and miscellaneous low-value things that won't last for long. Planets are where you can build a home and shape it into anything you want it to be. People can always find your planet; it's not going anywhere. Moons are useful for special purposes, like storage, heavy industry, and anything else you might want to do off-planet.[^4] [^0]: Or the equivalent with a hierarchical key structure. In practice, you want to have a master key which signs a junior key for everyday use. You use the master key to rotate the junior key. [^1]: More strictly: it proves that he didn't mine it after the other blocks currently on the Bitcoin blockchain. It only gives an ordering within the chain, not a literal timestamp. [^2]: In addition to planets, there are stars and galaxies. From a PKI perspective they're treated exactly like planets, but on the network they provide infrastructure services like routing. [^3]: In practice, this will likely be an included service by your sponsoring star. It could be a separate service, but it will be very cheap because it's a commodity and the "cost of production" is very cheap. At any rate, it should never rise above the capital cost of a 400GB hard drive. [^4]: To extend the metaphor, stars are a neighborhood to live in -- they're easier for other planets to see, so when they want to send you a message they look for your star first. If they don't even know where your star is, they can certainly find your star's galaxy, and that will be enough to locate the star. Of course, if your star is not providing satisfactory service, you can take your planet and move to another star. ]]> https://urbit.org/blog/stable-arvo https://urbit.org/blog/stable-arvo Tue, 19 Nov 2019 00:00:00 GMT https://urbit.org/blog/why-hoon https://urbit.org/blog/why-hoon Wed, 13 Nov 2019 00:00:00 GMT +mule. We run userspace code metacircularly with negligible performance overhead because of Urbit's jet system. In Lisp, "eval is evil" is a common saying, but in Urbit, eval is a first-class feature. - **Universally serializable**. There's one serialization format, called "jam", for any piece of code or data in the system. This makes it so that deserialization has just one function, a few hundred lines of C, as its security attack surface. It also facilitates portability of the virtual machine state, since this function can be called on the top-level VM state itself to generate a portable snapshot. - **Jetted**. A jet is a piece of code that the Nock interpreter has that reimplements a Nock function in some other, faster language. The `+dec` decrement function in Hoon's standard library is defined axiomatically using recursion, but is run as a jet written in C that makes use of the processor's ALU for performance. When calling a Nock function, if the runtime has a matching jet, it will use that instead of the Nock implementation. Nock isn't as slow as you might think, especially considering it's a minimal, dynamic, axiomatic language.

This arrangement has the deeper implication that all code written in Nock is best considered as a specification for a program, which can be executed directly but might never run at all due to being jetted. Cryptographic functions, for example, should all have jets with constant-time implementations to prevent timing attacks. Most jets, though, just take advantage of hardware acceleration for things like floating point arithmetic. - The amount of Hoon syntax is unusually large, but also **unusually regular**. Hoon is a "runic" language, meaning expressions generally begin with a digraph "rune" corresponding to the type of expression. Runes are used in place of keywords and Lisp's "special forms". Hoon's syntax makes better use of screen real estate than Haskell or Lisp by having a mostly vertical "backbone" of runes that prevents long functions from indenting repeatedly and helps quickly identify control flow. Hoon does not allow syntactic abstraction, so you always know exactly what you're looking at when reading Hoon. The similarity of Hoon's syntax to its abstract syntax tree makes metaprogramming easier and can be thought of as another layer of homoiconicity on top of Nock's. - **Subject-oriented**. There is no implicit environment; a Hoon expression compiles down to a Nock formula, which is interpreted as a function that runs with the "subject" as the argument. The subject can be any Nock tree, but it contains everything that's in scope. It usually consists of the Hoon compiler and standard library as a stack of cores, along with whatever functions and variables have been defined in the lexical scope. - **Cores**. A core is a pair of code and data. The core is the underlying representation of a function (a lambda with implicit fixed point), a library or module, an OOP-style object, and an OOP-style "interface" or "protocol". The code consists of a set of Nock formulas that can each be run against the whole core as a subject. This means there's always an implicit fixed point when running a function, and mutual recursion can occur among a core's formulas. A core's data usually consists of the standard library (itself a stack of cores), and possibly a "sample" (function argument). - **The type system has several unusual features**. It's intensional, in the sense that all constructs are first-class and can be up-cast to `noun`, Hoon's "any" or "top" type that matches all Nock nouns. Types are also used as scopes, so they store all of Hoon's variable names and docstrings. The type system uses an unusual macro-like feature called "wetness" to implement parametric polymorphism. The type system can also auto-generate coercion functions that validate and lift raw nouns into structural types, such as lists or cells. This is used to validate untrusted data, such as from the network. - **Reflective**. The type of type is just a normal datatype in Hoon, and a lot of the system manipulates types. In particular, the `!>` rune, when applied to a piece of data, uses compile-time type reflection to produce something called a "vase": a pair of type and data, similar to a Data.Dynamic in Haskell, or a limited form of a dependent pair. Since the Arvo kernel does a lot of dynamic compilation, it uses vases to implement something akin to a dynamically typed language using Hoon. This allows for type-safe dynamic program loading, program execution, and data migration. - **Inert**. Because Nock is purely functional, Hoon compiles to it so directly, everything is homoiconic, and Hoon is intensional, there's a very nice feeling that everything is just a stationary tree. There are no special objects that can't be manipulated; everything in your environment is just a subtree, and you could grab it and print it out if you wanted to. There's nothing like a "database handle", "websocket connection object", or other mystical constructs. The calmness of working with such inert building blocks is addictive, as many Hoon programmers will attest. ### Why is Hoon the way it is? Minimalism, mostly. Urbit's principled minimalism simplifies all kinds of things at many layers of the stack; for example, Urbit's linker, which is part of the Ford build system, just conses together multiple libraries into a tuple to form the compile-time environment for a source file. Universal serialization means we can safely send arbitrary pieces of data to apps on other ships without any more work than sending them to a local app. Using an app called Aquarium that's about three hundred lines of code, Arvo can run a whole fleet of other Arvos inside itself at full speed, just like Docker, and perform a suite of deterministic end-to-end tests on the OS as a whole. Even Hoon's seemingly baroque syntax is extremely regular and an unusually thin layer over the abstract syntax tree. It's designed to be a power tool; learning the syntax takes some time, but you only have to learn it once, and then it's not hard to read. It's like an English speaker learning Hiragana or Cyrillic. This heaping spoonful of syntactic sugar (along with jets for performance) is enough to take Nock from a Turing tarpit to a practical, ergonomic programming tool. Subject orientation in Nock and Hoon stems partly from minimalism (there's just one subject, which serves as state, lexical scope, environment, and function argument), partly from a desire to simplify compilation (the type of the subject is a full specification of the compile-time environment for a source file), and partly in order to give the language a more imperative feel. You program Hoon as if you have a mutable environment, but you're embedded in a purely functional, immutable context. While Hoon is a purely functional language, many of its runes create a mutant copy of the subject for manipulation by future runes (similar to Forth's stack operations), which makes it feel more like an expression is "doing something" rather than just calculating something. Everything about a scope, including name bindings, aliases, and docstrings, is stored in the subject's type. This allows Hoon's compilation discipline to be similarly minimal: the compiler is a function from subject type and Hoon source to product type and compiled Nock. Running this Nock against a value of the subject type produces a vase of the result. It's hard to imagine a more streamlined formalism for compilation. The compilation discipline gets applied recursively to build the layers of the Arvo stack. The Arvo kernel is compiled using the Hoon compiler as subject, the Zuse standard library is compiled using the Arvo kernel as the subject, and apps and vanes (kernel modules) are compiled using Zuse as the subject. The promise of Urbit lies in its reimagination of the digital world using components that are as constrained and limited as possible. By adhering firmly to principle and doubling down on minimalism at every turn, we get an OS that provides far stronger guarantees than Unix with a thousand times less code. Given the complexity of modern software, this is what's required to put personal computing back into the hands of people. ]]> https://urbit.org/updates/2019-11-06-update https://urbit.org/updates/2019-11-06-update Tue, 05 Nov 2019 00:00:00 GMT https://urbit.org/blog/simple-durable-yours https://urbit.org/blog/simple-durable-yours Wed, 16 Oct 2019 00:00:00 GMT We spend all our time switching between services, juggling logins and different interfaces. And once we’re logged in, the services we use harvest our data, spy on us, and often shut down with little warning. If computers are going to be a part of our lives for generations to come, we need a sound foundation for them. We need a _system_, not a grab bag of apps and services. Urbit is that system. We built Urbit from scratch to be a system that’s simple, durable, and yours. Everything that computing today is not — but should be. What do we mean by simple, durable, and yours? Let’s walk through it one by one.

## Simple Urbit is designed to be incredibly simple for ordinary users. Your data, applications, and connections all live in a single, unified operating system that belongs to you. Your Urbit requires no system administration and gets all its updates over the air. For developers, Urbit is incredibly compact and efficient. It’s about 50K lines of code — smaller than your average webapp. The entire OS can be understood by a single person and is designed to make building decentralized apps straightforward.

## Durable Your Urbit is designed to be durable: both to store your data across generations and to be resilient against crashes. You can't really trust a computer that could disappear or become obsolete. We’ve become accustomed to this distrust, but it’s not the way things should be. Your Urbit is meant to be a permanent digital home. At its foundation, Urbit is designed a bit like a blockchain — making reliability and permanence a priority for the individual.

## Yours Finally, your Urbit belongs to you in every possible way. You own it with a private key, control it completely, and can inspect its source code. We want your Urbit to be your digital home. A home you don’t own is just a hotel room. Urbit began as a sketch, driven from a basic intuition that a better platform could be built. Today, Urbit is two components: Urbit ID, an identity system that’s already live on Ethereum; and Urbit OS, a decentralized network and OS stack that’s young but working well. Over time we’ve quietly evolved Urbit into a carefully engineered, compact system. Urbit isn’t done — but it’s well on its way. The network has been live since 2013 and the interface is coming along nicely. You can own a piece of Urbit today if you can find an invite from a friend, or by buying an Urbit ID. We’re ready for people to begin homesteading this new territory we created with new communities and new software. We think it’s possible to leave today’s technology behind and build something better. Something more humane, calm, and collectively owned. Our vision of the future isn’t one where technology intrudes in every corner of our lives. It’s one where technology is mostly invisible. ]]> https://urbit.org/blog/urbit-and-bitcoin https://urbit.org/blog/urbit-and-bitcoin Wed, 16 Oct 2019 00:00:00 GMT https://urbit.org/blog/your-last-computer https://urbit.org/blog/your-last-computer Wed, 16 Oct 2019 00:00:00 GMT Imagine that you can log in from anywhere with one simple, memorable name and password. And when you do, an entire OS appears for you. Inside is your whole digital life. Every conversation you’ve ever had, all your biometric data — every step and heartbeat. Your personal mesh of devices: your doors, your car, your irrigation. All of your communities, every message you’ve shared with friends and family. Every connection you’ve made and every archival piece of data, in one place. Secure and private, forever. You own and control this OS completely. Your system doesn’t live on someone else’s server, so there’s no risk of leaking it in a mass data breach or losing it when a service shuts down. Your Urbit is a sealed, secure, and durable system. We’re on our way to this future, building it step by step, but we’re not there yet. The Urbit of today is much more stripped down. The stripped-down Urbit – Urbit OS 1 – starts out as a new kind of social network. Today, each of our social tools does one thing. And so each of our communities ends up spanning multiple services. Urbit OS 1 gives you the tools to bring these together into one place. In Urbit OS 1, a community shares a set of modules. (Think of a module like an app without the data lock-in.) One community may choose to chat and share a forum. Another may choose to chat and watch the crypto markets. And yet another may chat, share links, ebooks, and a to-do list. Anyone can develop a module, and anyone can create a community. It’s a simple system for intentional digital communities. A first step away from the ill-formed digital world of today. We like to call it ‘social computing’ — on the surface it feels like a new kind of app, but under the hood it’s an OS. It’s customizable all the way down. We think that a new system like this, even in a simple form, will be a relief from the chaos of the current web. Think of it as a “back to the land” computing movement. If we’re ever going to build new digital cities, they’ll have to start as a few colonies on the frontier. Your Urbit is a simpler computer, a quieter computer, a more private computer. We want it to feel predictable, safe, and reliable — things only a complete, sealed system can do. This, we hope, can get us a world where technology keeps us connected, but doesn’t dominate our lives. ]]> https://urbit.org/blog/landscape-a-portrait https://urbit.org/blog/landscape-a-portrait Mon, 02 Sep 2019 00:00:00 GMT https://urbit.org/blog/announcing-urbit-grants https://urbit.org/blog/announcing-urbit-grants Mon, 05 Aug 2019 00:00:00 GMT https://urbit.org/updates/2019-7-25-update https://urbit.org/updates/2019-7-25-update Wed, 24 Jul 2019 00:00:00 GMT https://urbit.org/blog/azimuth-security-bounty-program https://urbit.org/blog/azimuth-security-bounty-program Sun, 21 Jul 2019 00:00:00 GMT https://urbit.org/updates/2019-6-14-update https://urbit.org/updates/2019-6-14-update Thu, 13 Jun 2019 00:00:00 GMT https://urbit.org/blog/urbit-grants-and-mid-2019-gifts https://urbit.org/blog/urbit-grants-and-mid-2019-gifts Sun, 09 Jun 2019 00:00:00 GMT https://urbit.org/updates/2019-5-31-update https://urbit.org/updates/2019-5-31-update Thu, 30 May 2019 00:00:00 GMT https://urbit.org/updates/2019-5-17-update https://urbit.org/updates/2019-5-17-update Thu, 16 May 2019 00:00:00 GMT https://urbit.org/blog/azimuth-as-multipass https://urbit.org/blog/azimuth-as-multipass Wed, 15 May 2019 00:00:00 GMT https://urbit.org/blog/the-state-of-landscape https://urbit.org/blog/the-state-of-landscape Wed, 15 May 2019 00:00:00 GMT https://urbit.org/blog/the-100-year-computer https://urbit.org/blog/the-100-year-computer Mon, 13 May 2019 00:00:00 GMT https://urbit.org/updates/2019-5-10-update https://urbit.org/updates/2019-5-10-update Thu, 09 May 2019 00:00:00 GMT https://urbit.org/updates/2019-4-11-update https://urbit.org/updates/2019-4-11-update Wed, 10 Apr 2019 00:00:00 GMT https://urbit.org/updates/2019-3-27-update https://urbit.org/updates/2019-3-27-update Tue, 26 Mar 2019 00:00:00 GMT https://urbit.org/updates/2019-3-11-update https://urbit.org/updates/2019-3-11-update Sun, 10 Mar 2019 00:00:00 GMT https://urbit.org/updates/2019-3-4-update https://urbit.org/updates/2019-3-4-update Sun, 03 Mar 2019 00:00:00 GMT `@p`3.108.299.008 ~morlyd-mogmev > `@p`479.733.505 ~morlyd-mogmev ``` We [announced the bug on Twitter](https://twitter.com/urbit/status/1105850450559918082) today, and [a conversation](https://github.com/urbit/arvo/issues/1105) has been ongoing on GitHub. It appears we've settled on a solution, outlined in [this 3-line suggestion](https://github.com/urbit/arvo/issues/1105#issuecomment-472585937) from @max19, which elegantly corrects the implementation of the algorithm while preserving all existing planet names – a truly miraculous solution that came out of nowhere. Thank you Max! Stay tuned for a post with a full report. ## Team _Jared_ `~nidsut-tomdun` spent the week on the `@p` debacle and more than once generated all four billion planet names to check for collisions. _Ted_ `~rovnys-ricfer` spent much of the week thinking about redesigning the PKI only to realize that the existing design is quite good. _Logan_ `~tacryt-socryp` and _Isaac_ `~novlud-padtyv` started working on the next major revision to Landscape. _Ben_ `~pitmug-roptic` quietly [started working on](https://github.com/urbit/urbit/pull/1221) 64-bit Vere. `~pilfer-pandex` [brought the hammer down](https://github.com/urbit/docs/pull/399#issuecomment-472667240) on the pronunciation debate. Tlon is at the beginning a new batch of work for 2019, so new proposals are popping up almost daily. _Morgan_ `~hidrel-fabtel` is working on revamping our proposal (“UP”) system so that the outside world can more easily participate in all of the work that’s going on. _Anthony_ `~poldec-tonteg` and _Jared_ `~nidsut-tomdun` are also working on standards for our repository documentation and open-source development process. ## Community Our Austin meetup and San Francisco AMA were a blast. You can watch [the recording of the latter](https://www.youtube.com/watch?v=ndk2b69fA9A) on Youtube. Thank you for coming and for all of your questions! Out there on the Internet, people are starting to build their own platforms around Urbit and Azimuth. There are now multiple planet stores and at least one active online community. It's exciting to watch the community start to grow all by itself. Until next time. ]]> https://urbit.org/updates/2019-2-26-network-update https://urbit.org/updates/2019-2-26-network-update Mon, 25 Feb 2019 00:00:00 GMT https://urbit.org/updates/2019-2-25-update https://urbit.org/updates/2019-2-25-update Sun, 24 Feb 2019 00:00:00 GMT https://urbit.org/updates/2019-2-11-update https://urbit.org/updates/2019-2-11-update Sun, 10 Feb 2019 00:00:00 GMT https://urbit.org/updates/2019-2-4-update https://urbit.org/updates/2019-2-4-update Mon, 04 Feb 2019 00:00:00 GMT https://urbit.org/blog/a-founders-farewell https://urbit.org/blog/a-founders-farewell Sun, 13 Jan 2019 00:00:00 GMT https://urbit.org/blog/azimuth-is-on-chain https://urbit.org/blog/azimuth-is-on-chain Sun, 13 Jan 2019 00:00:00 GMT Note: As of `~2019.1.14` we're in the process of rebooting the Arvo network. We expect it to be back online before the end of the week. We'll update this post and [tweet](https://twitter.com/urbit) when the network is online and stable. ## Rethinking the stack We've started to talk about Urbit as a stack of three parts: Azimuth, the address space and identity layer on Ethereum; Arvo, the OS and overlay network; and Aegean, a pattern for building decentralized communities on the Arvo network. We tried to capture this separation in [the primer](/primer), a new overview of Urbit. Explaining Urbit has always been challenging. A monolithic project that is OS, network, language, VM and UI barely makes sense. Separating the layers helps, we think. Let's step through them to see what's new. ## Azimuth  As a quick refresher: Urbit addresses, now called Azimuth 'points', are short names like `~diblun-tidduc`. You own your Azimuth point with a private key and can use it as a routable network address on the Arvo network. You can read more about Azimuth in [the primer](/primer#what-azimuth-is). If you're curious who owns all this address space, you can find that [in there too](/primer#azimuth-distribution). In late 2017 we decided to migrate the yet-to-be-named Azimuth to a system of Ethereum contracts. After a year of coding, testing, auditing and migrating — they're now fully deployed to the Ethereum mainnet. Azimuth is basically two parts, a database of who owns which points, and a set of rules about what points and their owners can do. That set of rules is called Ecliptic, and can be found at [`0x6ac07b7c4601b5ce11de8dfe6335b871c7c4dd4d`](https://etherscan.io/address/0x6ac07b7c4601b5ce11de8dfe6335b871c7c4dd4d) or [`ecliptic.eth`](https://etherscan.io/address/ecliptic.eth). The top-level Azimuth points, the galaxies, can vote on upgrades to the Ecliptic contract, but they can't edit the database of who owns what. The ownership database contract is called Azimuth, which you can find at [`0x223c067f8cf28ae173ee5cafea60ca44c335fecb`](https://etherscan.io/address/0x223c067f8cf28ae173ee5cafea60ca44c335fecb) or [`azimuth.eth`](https://etherscan.io/address/azimuth.eth). For those of you who own address space, you can use the `getOwnedPoints` function in the 'Read Contract' tab to check your balance. The function returns the number (not the phonemic string like `~ravmel-ropdyl`) of your points. Each Azimuth point is an ERC-721, non-fungible piece of digital property. To manage your points, we built [Bridge](https://github.com/urbit/bridge/releases), a completely custom interface for interacting with our Solidity contracts. Interface quality is really important to us, and we couldn't find anything in the 'crypto' world that met our standards. Bridge is good, but crypto UX is a hard problem. We've already got some ideas for improvements. If you're curious, you can find the source for the contracts in the [`urbit/azimuth`](https://github.com/urbit/azimuth) repo. The Bridge source is in the [`urbit/bridge`](https://github.com/urbit/bridge) repo. We had the source for our contracts audited by Open Zeppelin, Blockchain at Berkeley and Bloctrax. Did we miss something? We sure hope not — but we'd happily compensate anyone who can find bugs. Please reach out to security@urbit.org if you find something of concern. We'd be happy to hear from you.  Azimuth names have always had a pronunciation system. `~ronryd-nidseg` is 13,695,781, for example. Now, they also have a visual system. We call these strange crests from another planet 'sigils', and you'll notice them used throughout all our new interfaces. You can find the source in the [`urbit/sigil-js`](https://github.com/urbit/sigil-js) repository. Building these was a lot of fun — we'll save the details for a future post. So, how do you get an Azimuth point? We've left that question unanswered intentionally. Azimuth points are meant to be like digital land, not currency. They should change hands slowly. In order to get an Azimuth point, you'll need to find someone who already has one and is willing to part with it. ## Arvo  Arvo boots from Ethereum, is no longer embarrassingly slow, and has a brand new build system. [Our documentation](https://urbit.org/docs/learn/arvo/hoon) has also improved quite a lot. Hoon, Arvo's programming language, has had a lot of work done to its type system and compiler. We built new tooling for generating static sites (you're probably reading this on one right now) that are much, much faster and easier to use. It's tough to talk about the progress we've made on Arvo without writing a long technical post. We'll save that for later. In a way, what's remarkable about Arvo is that it has become _less_ remarkable. It's more stable, faster, and the code quality has improved. There's plenty of work to do, though. Should you be building on top of Arvo? Probably not yet. The kernel and kernel modules are really where the work still lies. We build things on top of Arvo, and in doing so it's pretty clear that the system can be better. Clay (the filesystem) and Gall (the application model) are due for a rewrite. A complete rewrite of the web server, Eyre is almost finished. ## Aegean  Aegean is a pattern for building 'cities': groups of people running the same 'agent' on their Arvo servers. Agents aren't just 'apps' or programs. An agent rolls together the tools that a community needs. Today, our companies, communities, cults, cliques all have to duct tape together separate MEGACORP apps and services to compute together. Aegean reverses this pattern. A community on Arvo is a city: just a bunch of people running the same agent. Each city is in control of its own codebase: the interface, application code _and_ data storage. This means a community can have complete agency over how to compute together. Aegean cities can be completely different without impacting one another at all. Cities share the neutral infrastructure of Arvo and Azimuth — but they may never even know others exist. An archipelago of hypercultures. It's important to note: the pattern of these cities is _not_ the pattern of Azimuth. The network hierarchy should impact the society of Urbit as much as the electrical grid impacts the life of the street in Manhattan. It matters when the power goes out — but most of the time groups form and dissipate continuously. Before we can let the ocean of cities grow we have to homestead a few of our own. Our first agent is called [Landscape](https://github.com/urbit/urbit/tree/master/pkg/interface), an agent for chat and longform discussion. We are using Landscape to run a few experimental cities over the next month or so. If you have an Azimuth point, you'll need to [request access](mailto:support@urbit.org) to join. Anyone can create their own city, of course. The source is open. But we're not going to encourage you to just yet. Feel free to explore, but also be patient. We've got a lot planned around this. If Azimuth is done and Arvo is young, Aegean is just being born. ## Tlon  So who is behind all of this? A team of collaborators and contributors. Many of us work at [Tlon](https://tlon.io) — the company we started in 2013 to advance Urbit platform development. Tlon has attracted an incredible group of smart, talented people who want the world to be able to own and control their computers. I'm proud to work alongside them. One important omission on `tlon.io` is Curtis. As of today, he's no longer working on Urbit. [His own words](/posts/essays/a-founders-farewell) are the best way to understand his thinking. It's an achievement for the team, I think, that he can depart with confidence. One major reason for that confidence is that Erik and Anthony have now become COO and CPO respectively. Having thoughtful, organized people leading the project is a huge benefit to the whole team. I'm very glad to have them on board. ## Looking ahead I certainly forgot at least a thing or two. We'll dive in to more detail in the coming weeks. There's a lot for you to explore. Have fun! ]]> https://urbit.org/blog/governance-of-urbit https://urbit.org/blog/governance-of-urbit Thu, 10 Jan 2019 00:00:00 GMT https://urbit.org/updates/2018-12-18-update https://urbit.org/updates/2018-12-18-update Mon, 17 Dec 2018 00:00:00 GMT https://urbit.org/updates/2018-11-20-update https://urbit.org/updates/2018-11-20-update Mon, 19 Nov 2018 00:00:00 GMT https://urbit.org/updates/2018-11-6-update https://urbit.org/updates/2018-11-6-update Mon, 05 Nov 2018 00:00:00 GMT https://urbit.org/updates/2018-10-24-update https://urbit.org/updates/2018-10-24-update Tue, 23 Oct 2018 00:00:00 GMT https://urbit.org/updates/2018-10-12-update https://urbit.org/updates/2018-10-12-update Thu, 11 Oct 2018 00:00:00 GMT https://urbit.org/updates/2018-9-11-update https://urbit.org/updates/2018-9-11-update Mon, 10 Sep 2018 00:00:00 GMT https://urbit.org/updates/2018-8-30-update https://urbit.org/updates/2018-8-30-update Wed, 29 Aug 2018 00:00:00 GMT https://urbit.org/updates/2018-8-23-update https://urbit.org/updates/2018-8-23-update Wed, 22 Aug 2018 00:00:00 GMT https://urbit.org/updates/2018-8-8-update https://urbit.org/updates/2018-8-8-update Tue, 07 Aug 2018 00:00:00 GMT https://urbit.org/updates/2018-7-30-update https://urbit.org/updates/2018-7-30-update Sun, 29 Jul 2018 00:00:00 GMT https://urbit.org/updates/2018-7-20-update https://urbit.org/updates/2018-7-20-update Thu, 19 Jul 2018 00:00:00 GMT https://urbit.org/updates/2018-7-13-update https://urbit.org/updates/2018-7-13-update Thu, 12 Jul 2018 00:00:00 GMT "A complex system that works is invariably found to have evolved from a simple > system that worked. A complex system designed from scratch never works and > cannot be patched up to make it work. You have to start over with a working > simple system." -[John Gall]()  The Infrastructure team -- _Curtis_ `~sorreg-namtyv`, _Anthony_ `~poldec-tonteg`, _Joe_ `~master-morzod`, _Ted_ `~rovnys-ricfer`, _Elliot_ `~littel-ponnys`, _Mark_ `~palfun-foslup` -- worked on improving the core of the Urbit system: - Curtis worked on improving Hoon: [`research-tome` Arvo](https://github.com/cgyarvin/arvo/tree/research-tome), [`research-tome` Vere](https://github.com/cgyarvin/urbit-1/tree/research-tome) - Joe continued to implement in-Urbit HTTPS integration: [`https-acme-wip` Arvo](https://github.com/urbit/arvo/tree/https-acme-wip), [`https-proxy-config` Vere](https://github.com/urbit/urbit/tree/https-proxy-config) - Ted and Elliot continued the state-machine reimplementation of new Ford: [`ford-turbo-state-machine` Arvo](https://github.com/urbit/arvo/tree/ford-turbo-state-machine), [updates](https://fora.urbit.org/proposals/posts/~2018.3.15..04.24.35..a47f~/) - Mark made crypto additions to the kernel: [`crypto-additions` Arvo](https://github.com/urbit/arvo/tree/crypto-additions) The documentation team -- _Josh_ `~taglux-nidsep` and _Robert_ `~lodleb-ritrul` -- improved how we communicate how Urbit works: - Josh continued work on the new Hoon tutorials: [`master` doc drafts](https://github.com/joshuareagan/doc-drafts) - Robert put in more work improving the Hoon standard library doc: [docs pull requests](https://github.com/urbit/docs/pulls) The Interface team -- _Galen_ `~ravmel-ropdyl`, _Anthony_ `~poldec-tonteg`, _Jimmy_ `~haptyl-wanwes`, _Gavin_ `~ridlur-figbud`, _Chris_ `~sigtus-hossyn`, _Isaac V_ `~davtyr-nimren`, _Logan_ `~tacryt-socryp`, and _Keaton_ `~tonlur-sarret` -- worked on Landscape (formerly MVE): - Galen, Jimmy, and Gavin have continued work on Landscape's visual design - Keaton worked on Landscape cloud onboarding on DigitalOcean - Chris improved the desktop client, still in private alpha - Logan improved networking in the iOS TestFlight app, still in private alpha - Isaac V has been rearchitecting the Landscape backend - Anthony has been getting the latest Landscape code onto the testnet: [`testnet` Arvo](https://github.com/urbit/arvo/tree/testnet) The Constitution/Keys team -- _Mark_ `~palfun-foslup`, _Curtis_ `~sorreg-namtyv`, _Morgan_ `~hidrel-fabtel`, _Will_ `~hadrud-lodsef`, _Anthony_ `~poldec-tonteg`, and _Ben_ `~mirfet-hocbyt` -- worked to make Urbit computing cryptographically real: - Curtis, Morgan, and Will have been fortifying the current proposed key ceremony, still in private draft mode - Ben has been cleaning up Wallet: [`mercury` Wallet](https://github.com/urbit/etherwallet) - Mark has updated our Constitution to ERC721 compliance, among other updates, and built out more of Jael: [`master` Constitution](https://github.com/urbit/constitution) [`research-constitution` Arvo](https://github.com/Fang-/arvo/tree/research-constitution) We're alive and growing. --- Photo credit: [Superstudio](https://en.wikipedia.org/wiki/Superstudio) ]]> https://urbit.org/updates/2018-7-3-update https://urbit.org/updates/2018-7-3-update Mon, 02 Jul 2018 00:00:00 GMT https://urbit.org/updates/2018-6-21-update https://urbit.org/updates/2018-6-21-update Wed, 20 Jun 2018 00:00:00 GMT https://urbit.org/updates/2018-6-14-update https://urbit.org/updates/2018-6-14-update Wed, 13 Jun 2018 00:00:00 GMT https://urbit.org/updates/2018-6-6-update https://urbit.org/updates/2018-6-6-update Tue, 05 Jun 2018 00:00:00 GMT https://urbit.org/updates/2018-5-30-update https://urbit.org/updates/2018-5-30-update Tue, 29 May 2018 00:00:00 GMT https://urbit.org/updates/2018-5-29-update https://urbit.org/updates/2018-5-29-update Mon, 28 May 2018 00:00:00 GMT https://urbit.org/updates/2018-5-23-update https://urbit.org/updates/2018-5-23-update Tue, 22 May 2018 00:00:00 GMT https://urbit.org/updates/2018-5-16-update https://urbit.org/updates/2018-5-16-update Tue, 15 May 2018 00:00:00 GMT _Meta_: We're going to start posting these to the new _Updates_ section in Fora now -- and actually weekly! On [Mars Talk last week](https://www.youtube.com/watch?v=gBd0TZyz_HE), _Anthony_ `~poldec-tonteg`, _Joshua_ `~taglux-nidsep`, _Ted_ `~rovnys-ricfer`, and _Keaton_ `~tonlur-sarret` talked about Sail, a DSL for representing XML/HTML data in Hoon. We built a simple web page with Sail and highlighted what features Sail gives you, like Hoon's type system, the Ford build system, and full access to Hoon programming inline. Check it out! Tlon hired someone this week! We're excited to have with us: - _Isaac V_ `~davtyr-nimren` Isaac has solid Hoon and frontend chops, having already implemented server-side rendering of Tree pages for us a short while back. He'll be taking on some of the Collections/Streams work to help us get ready for our summer release. _Galen_ `~ravmel-ropdyl`, _Anthony_ `~poldec-tonteg`, _Chris_ `~sigtus-hossyn`, _Gavin_ `~ridlur-figbud`, and _Jimmy_ `~haptyl-wanwes` have been working more on the new interface projects. Here's another leak:  _Logan_ `~tacryt-socryp` almost has a complete demo of the new Urbit iOS app working. This includes TouchID login authentication, an encrypted local database, full support for the Talk/Hall API for Streams, and automatic background fetching, among other features. _Curtis_ `~sorreg-namtyv` has been working more on several important major language feature improvements to Hoon: namely, pretty-printing of typed data, and simplifying the type system. Work can be found on his [research branches](https://github.com/cgyarvin/arvo), but we plan on publishing a more formal post of the changes and major documentation updates when they roll out in the coming months. More soon! _Ted_ `~rovnys-ricfer` and _Elliot_ `~littel-ponnys` have made a bunch of progress on Ford Turbo. In addition to having a proof-of-concept solution for the main performance bottleneck of current Ford, they've now finished writing over half of the schematics (build types), implemented proper cache reclamation support, and they've gotten basic Clay integration functioning properly. You can find their work [here](https://github.com/urbit/arvo/tree/ford-turbo). They're posting more detailed updates as comments to their original [Urbit Proposal](https://fora.urbit.org/proposals/posts/~2018.3.15..04.24.35..a47f~/). _Mark_ `~palfun-foslup` has been wrapping up Urbit/ETH integration, and is now working on getting Jael, the new Arvo secret-storage vane, to compile. This would be a great addition to the summer release. _Joshua_ `~taglux-nidsep`, _Robert_ `~lodleb-ritrul`, and _Keaton_ `~tonlur-sarret` have been getting lots of documentation work done. They've seen several PR's go through to improve the Hoon standard library docs, simplify the Hoon demo, update our examples to the latest syntax, improve the Urbit installation process, and more. Josh's work-in-progress is happening [here](https://github.com/joshuareagan/doc-drafts/). The main docs are [on GitHub](https://github.com/urbit/docs): if there's a piece of doc that's confusing to you, don't hesitate to file an issue. Or better, submit a pull request. _Joe_ `~master-morzod` is making strides on full HTTPS support for individual ships. He's implemented LetsEncrypt support, ASN.1, and has Dynamic DNS functioning on our testnet. When all this work is done, `your-ship.urbit.org` should connect automatically to the IP of the Unix machine your urbit is running on, rather than connecting to one of Tlon's ships, which currently proxies Urbit network data through one of our stars, is slow, and is basically a straight-up man-in-the-middle. Combined with private Collections and Streams, Urbit will be a lot more secure and usable within the year. Overall, we're making progress. More next week! ]]> https://urbit.org/updates/2018-4-14-update https://urbit.org/updates/2018-4-14-update Fri, 13 Apr 2018 00:00:00 GMT https://urbit.org/updates/2018-4-7-update https://urbit.org/updates/2018-4-7-update Fri, 06 Apr 2018 00:00:00 GMT https://urbit.org/updates/2018-3-24-update https://urbit.org/updates/2018-3-24-update Fri, 23 Mar 2018 00:00:00 GMT https://urbit.org/updates/2018-3-5-update https://urbit.org/updates/2018-3-5-update Sun, 04 Mar 2018 00:00:00 GMT https://urbit.org/updates/2018-2-24-update https://urbit.org/updates/2018-2-24-update Fri, 23 Feb 2018 00:00:00 GMT https://urbit.org/updates/2018-2-17-update https://urbit.org/updates/2018-2-17-update Fri, 16 Feb 2018 00:00:00 GMT https://urbit.org/updates/2018-2-9-update https://urbit.org/updates/2018-2-9-update Thu, 08 Feb 2018 00:00:00 GMT 100 open issues. Knowing a lot is wrong is the first step towards making a better product, but a big ol' pile of issues isn't really conducive to that. After [proposing](https://github.com/urbit/arvo/issues/572) a set of labels for our repositories to use, `~rovnys` and I set out to show all the issues some love. Today we finally made it through the last of them! You can now look for issues that are [easy](https://github.com/urbit/arvo/issues?q=is%3Aissue+is%3Aopen+label%3A%22difficulty+low%22) or [good for contributors](https://github.com/urbit/arvo/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+contributor+issue%22) to tackle. There's plenty of labels now for all your filtering needs. Internally, we'll try to get eyes on issues labeled [RFC](https://github.com/urbit/arvo/issues?q=is%3Aissue+is%3Aopen+label%3A%22request+for+comments%22) and [help wanted](https://github.com/urbit/arvo/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22) every so often. While contributors can't add labels themselves, we'll make sure your issues get tagged correctly. Enjoy! _Curtis_ `~sorreg-namtyv` has been helping Ted and _Anton_ `~pittyp-datfyn` with Ford. He's also continued his work on improving default values for types in Hoon. _Keaton_ `~tonlur-sarret` has been working on a super-secret project that he cannot discuss. Loose lips sink ships ... _Mark_ `~palfun-foslup` has submitted a PR for Clay permissions. We just completed a second round of review, and Mark is now doing more testing. _Iceman_ `~littel-ponnys`, _Joe_ `~master-morzod`, and Ted [had some fun](https://fora.urbit.org/posts/~2018.2.9..21.50.49..709f~/) discussing various forms of persistence in Clay and Gall in `urbit-meta`. Keaton and Ted demoed some Urbit web app examples in [Mars Talk Episode 20](https://www.youtube.com/watch?v=emWy1Afe4qY). ]]> https://urbit.org/updates/2018-2-5-update https://urbit.org/updates/2018-2-5-update Sun, 04 Feb 2018 00:00:00 GMT https://urbit.org/updates/2018-1-13-update https://urbit.org/updates/2018-1-13-update Fri, 12 Jan 2018 00:00:00 GMT https://urbit.org/updates/2018-1-6-update https://urbit.org/updates/2018-1-6-update Fri, 05 Jan 2018 00:00:00 GMT https://urbit.org/updates/2017-12-22-update https://urbit.org/updates/2017-12-22-update Thu, 21 Dec 2017 00:00:00 GMT https://urbit.org/updates/2017-12-16-update https://urbit.org/updates/2017-12-16-update Fri, 15 Dec 2017 00:00:00 GMT https://urbit.org/updates/2017-12-2-update https://urbit.org/updates/2017-12-2-update Fri, 01 Dec 2017 00:00:00 GMT the hall has opened, > circles form to admire, > people's collections. He also says: > Or if you want something slightly more serious: > Hall has received some more fixes and quality of life improvements since I opened the PR. It's > never been more ready > for the breach! I'm already working on something interesting that uses > it: an interface for managing your own notes, > blogs and foras. Complete with filesystem, storage, and hall notifications! _Joe_ `~master-morzod` has written an implementation of SSL termination in vere. This will allow your Urbit to serve true end-to-end-encrypted HTTPS, as opposed to Tlon's servers operating as man-in-the-middle, which is not a viable long-term solution. This has been an important piece of missing functionality, so we're excited to see it start to come together. Latest code can be found [here](https://github.com/joemfb/urbit/tree/346de776c5fabe3e02fecb7cab3d1652e4cc46e3). _Isaac_ `~davtyr-nimren` has been building the next generation of Urbit's web publishing system, currently located in the ren/tree directory of Arvo. This new system, instead of serializing data down to the client to be fetched after the initial page load, will perform the HTML rendering on the server. This is now possible because of the integration of our Udon markup language with the Hoon compiler. This allows us to include Udon files in other Udon files using Ford runes. The new system is much simpler and should improve page load time. Once we've written our planned Vere HTTP cache, this new paradigm should also improve the coherence of that cache. This system is mostly complete now, and Isaac is now working on smaller bugfixes. There was also a [new episode of Mars Talk](https://youtu.be/1xcjnBu6qFo) with Ted and Joe discussing Hoon's type system. Those are the highlights from this week. Back to breaching! ]]> https://urbit.org/updates/2017-11-14-update https://urbit.org/updates/2017-11-14-update Mon, 13 Nov 2017 00:00:00 GMT https://urbit.org/updates/2017-10-7-update https://urbit.org/updates/2017-10-7-update Mon, 16 Oct 2017 00:00:00 GMT https://urbit.org/updates/2017-9-29-update https://urbit.org/updates/2017-9-29-update Thu, 28 Sep 2017 00:00:00 GMT https://urbit.org/blog/urbit-and-the-blockchain https://urbit.org/blog/urbit-and-the-blockchain Sun, 24 Sep 2017 00:00:00 GMT https://urbit.org/updates/2017-9-23-update https://urbit.org/updates/2017-9-23-update Fri, 22 Sep 2017 00:00:00 GMT https://urbit.org/blog/bootstrapping-urbit-from-ethereum https://urbit.org/blog/bootstrapping-urbit-from-ethereum Tue, 19 Sep 2017 00:00:00 GMT https://urbit.org/updates/2017-8-11-update https://urbit.org/updates/2017-8-11-update Thu, 10 Aug 2017 00:00:00 GMT https://urbit.org/updates/2017-5-11-roadmap https://urbit.org/updates/2017-5-11-roadmap Wed, 10 May 2017 00:00:00 GMT https://urbit.org/blog/toward-a-frozen-operating-system https://urbit.org/blog/toward-a-frozen-operating-system Tue, 09 May 2017 00:00:00 GMT https://urbit.org/updates/2017-4-17-update https://urbit.org/updates/2017-4-17-update Sun, 16 Apr 2017 00:00:00 GMT https://urbit.org/updates/2017-3-1-update https://urbit.org/updates/2017-3-1-update Tue, 28 Feb 2017 00:00:00 GMT https://urbit.org/updates/2017-1-30-update https://urbit.org/updates/2017-1-30-update Sun, 29 Jan 2017 00:00:00 GMT TLDR: a real Urbit is an Urbit that does real work. This means nontrivial > progress in two areas: systems maturity and user experience. In 2H 2016 we > cleaned out most of the worst holes in Arvo and vere, and totally reconceived the > user experience. The new Urbit feels like an MVP and we can't wait to ship it. In 2013 Urbit was intriguing. By 2015 it had become interesting. In 2016 it was actually exciting (if not always in a good way!). In 2017, we're actually planning to make the darn thing useful. (For certain values of the word "useful.") Our `cc-release` stability upgrade (mainly affecting the OS layer `arvo` and the interpreter `vere`) is roughly code-complete and being tested into existence. Once it exists and is stable, it can actually be documented -- often, for the first time. And `cc-release` is our first release really designed so that neither your ship, nor the whole network, can actually sink. The Urbit network had zero unplanned, and one planned, breach (network flag day) in 2H 2016. The last planned breach, the `cc-release` update, is planned for 1H 2017. With the infrastructure converging, we can finally think seriously about the user experience of Urbit. What we're imagining is a small-scale social network and command-line API aggregator. It's just screenshots right now, but we're confident that we can build it and make it work. System software is boring (at least when done right). So let's start with the user experience. ## Design Galen Wolfe-Pauly is Tlon's CEO and Urbit's interaction designer. Galen went to architecture school. He talks like it. We recorded him secretly and came away with these notes:  - _UX_? As a kid it was just _design_ or maybe _web design_. At some point I remember people talking about _user interface_ or _UI_. But I truly don't understand where the _X_ came from. It's like Monster Energy got involved in the branding. - An _experience_ sounds like the kind of thing that's going to be littered with ads, or is going to manipulate me in some way. Something I passively consume. Not something I actively control. So I prefer _interface_ to _experience_, and I think you should too.  - Urbit has to be just as usable as any widely used app in 2017. For a personal server, the usability bar can't be Unix. A Unix server is an industrial tool. An industrial tool can't be a consumer product. You can attach a blade to the end of a drill and use it as a blender, but it's not a Vitamix. - What is an app, anyway? It's _shared computing_. Everyone's data is one data structure, in one program, on one server, owned by one corporation. With _one_ UI for everyone to do _one_ thing. - But there isn't one big corporation that writes _all_ the code. Everyone's lives are spread across a bunch of different apps. Every app solves the ownership problem separately. - So the whole user experience of using Web apps involves tracking all these silly little accounts, learning each and every user experience, and glaring helplessly at each little piece of data trapped in a different silo. - Even your Google account doesn't feel like one thing. Whatever your Google Docs are, it's not a bunch of document files on your Google Drive. All the resources of Google can't make their Borg backend feel like one computer which belongs to you. - The phrase 'general-purpose computing' sounds scary, but it shouldn't be. 'Computing' doesn't mean writing code. We're computing when we browse Facebook or talk to Alexa. - Your personal server will be solving the same problems today's whole ecosystem of network apps solves, just all in one place that belongs to you. - The closest thing to a general-purpose personal server today is probably the Chinese service WeChat. If you don't know much about WeChat, you should really watch [this NYT video](https://www.nytimes.com/video/technology/100000004574648/china-internet-wechat.html). - From a distance WeChat is great. Everyone in China loves the one-app, one-identity, all-my-own-stuff user experience. It's clearly better than being forced to switch apps constantly. - But WeChat is just another giant shared-computing service. It's made out of the usual 2010s server stack mess. The visual and interaction design looks pretty dated. WeChat tells us that users want one experience, but not much more than that.  - To paraphrase Walter Sobchak: say what you want about the tenets of shared computing, but at least it's an ethos. We know how to build very good special-purpose user experiences. What's the conventional UI for a general-purpose personal server? It doesn't exist. - For most users, your urbit runs in a data center (like a web app). You log into it with your browser (like a web app). Other than that, where do you even start? A `bash` prompt isn't what we're looking for. - But wait, is the idea of a command line bad UI? Or is the Unix command line just bad UI? We know ordinary people can use command lines. They use chatbots. Long ago, they used MS-DOS. - Today's computing user experiences, the PC and its mobile cousin, rejected the command line. This made computing easy enough, at the cost of hiding much of its creative power. - The power of a general-purpose server which can command not only local applications, but also network services, is at a whole new level. We don't think any visual metaphor or 2D control panel can do it justice. - We need to bring the full power of the machine to the user, without breaking their trust that the machine is on their side. We can't let the user get lost or break the system. It has to feel safe and simple to explore its furthest reaches.  - As Neal Stephenson [tells us](http://cristal.inria.fr/~weis/info/commandline.html), the command line is the natural UI for any computer. It's especially the natural UI for a _network_ computer, like a personal server. A command line scales way better than a GUI to a large selection of actions. And a network computer can do almost infinitely many things. - When we've tried to put a simplified "for dummies" GUI on top of Urbit, it's just awful. It's like a bad Linux desktop GUI. You feel instantly that you've got a choice between something limited and lame, and something mysterious and scary. So we threw those designs out, in search of something better. - The true command line is generally thought of as something mysterious and scary. Somewhere that you can do things to irreparably damage your computer. It's powerful, but also dangerous. - This tradeoff — between control and safety — is false. The unfriendly command line is actually just another artifact of 70s computing. The UI of the command line is designed for a programmer, not an ordinary person. - `bash` is a bad UI because there's no easy way to control its power, or learn all the things it can do, or separate its power to do dangerous things from its power to do safe ones. - At the other end of the spectrum is the chatbot, which is a bad UI because it's hard to make it much more than a toy. Like so many innovations, chatbots happened by accident. It's just a way to connect an input string to network services. But the natural-language metaphor of "chat" is confining. It doesn't scale well to any problem that isn't trivial. - What does a great command line look like? It should have a strict syntax, not try to guess what you mean. But it can't be a constant test of your ability to memorize the phonebook.  - Urbit already has the beginnings of a great command line. We parse your input with an actual grammar and catch errors as you type. You can't even press return on an invalid command. - What's missing is help. A great command line needs to reach out and hug you in every way it can. It needs to assume you don't know anything at all about the system. It's there for you to solve your problems, but also to help you explore. - There's actually one exotic Unix shell that does it right: [`fish`](https://en.wikipedia.org/wiki/Friendly_interactive_shell). `fish` jumps through some pretty amazing hoops, like parsing manpages, to give you context and completion. - It's a lot easier for Urbit because we're designing the system from scratch. From a technical standpoint, all we have to do is add contextual help to our existing parser framework. An extended rule can parse a string, or tell the user what it wants to parse. In simple terms: we have to make the `:dojo` (our shell) much friendlier. - The `:dojo` shell is built around the concept of a _generator_, a program that builds a noun. The idea is that how you build the noun is orthogonal to what you do with it. You could print it, send it as a command, etc. There are several kinds of generators -- simple scripts, prompting dialogs, HTTP scrapers, etc. - A generator is initialized with command-line arguments and options, in the classic Unix style (except typed). In the classic Unix style, Urbit at present provides no help at all in constructing a command line. You just have to know. - How should it actually work? Let's say you're trying to put Urbit in control of your Twitter feed. That means you're creating a Twitter gateway on Urbit, which you can control with a `|twitter` command. - Before you type a character, Urbit offers you `|` (any command with side effects). (You could also add a `!` prefix, which is like `sudo` -- it lets you do something dangerous.) Once you type `|`, you can page through a list of commands. Once you type `|t`, that list is much shorter. Once it can tab complete to `|twitter`, it prompts you with that. - And once you get to `|twitter`, you're in a "meta-generator" that's helping you build the correct command line. At least in a browser, you'll be stepping through a form with rich entry. But the command line will show you the text of the command you're building, so next time you can just type it. - Building a command is just a case of data entry. In a command-line world, data entry is always serial. You answer questions serially, one at a time. The only navigation is forward and back. - A great Urbit console also has to be accessible both from a browser and a Unix terminal. That means a prompt needs to tell the console if it could be a radio button, even though a terminal can't have a radio button. We'll improve the interactivity of our terminal a little, but we're not rewriting Lynx. - So, what do you do with this fancy command line? Command everything -- from your urbit to the world. Installing an app is a command. Publishing a tweet is a command. Paying someone in bitcoin is a command. - Your urbit can show you all the things you can do, and help you do them. If there's something you'd like it to do, build it! - A great command line alone is not all that Urbit needs. But it feels like the right central paradigm for interacting with your Urbit. We'll get into the specifics in a future post.  ## Infrastructure: stability, maturity, performance, documentation: We dragged Curtis, whose hide has turned completely white like a cave fish, from his coding hole to stammer out this update: We're actually doing pretty well on the infar. Despite there being all kinds of unacceptably broken things about Urbit 2016, the network has actually stayed up since August, and we haven't had an unintentional continuity breach since June. Of course, it helps that we don't ask anyone to _use_ Urbit in earnest. And ships still do sink. Since an urbit is an identity, sinking your planet is a horrendous experience. You'll never bond with your second planet the way you did with your first -- never mind any _data_ you may have lost. Our upcoming `cc-release` is the last planned breaching release. With certain exceptions discussed below, all major parts are now (January 2017) code complete or better. This includes the rewritten network stack (`%ames`), code complete but untested; the new secret vault and promise tracker (`%jael`), mostly tested; and the new `u3_pier` event execution framework, with worker processes and real two-phase commit, smoke-tested. We've also completely restructured `hoon.hoon` and `zuse.hoon`, which now are at least cosmetically acceptable. This is not to say that the `cc-release` Urbit won't in many ways embarrass us as engineers. It will. But your ships won't randomly sink, your secrets won't randomly leak, and your datas won't randomly rot. Or if they do, it's an implementation bug, not a design flaw. There are still a few cracks we need to fill in. The main one: some top-level adjustments to the Arvo event system. The boot sequence now executes correctly, but the actual boot events need to be totally refactored, as does the rather critical Unix-Arvo interface. The long-promised security mask and voltage flags need to be added, and vanes adjusted to use them. Causal tracking of execution crashes needs to actually work. At the user level, `:talk` needs to be split into a user agent and a daemon, and the user-level `%kiln`, the "systemd of Urbit," needs a rethink to make upgrade logic state-triggered, not edge-triggered. The console and the dojo need some work to make our user experience real. One way to view a high-usability command line is that a command, if you don't have the esoteric knowledge to fill it in by just typing the syntax, is essentially a form. This form needs to work both in the browser and the terminal, which isn't trivial. Browser and terminal both need a simple multiscreen navigation model, which we don't have yet. There's also a bunch of Unix-level work to do, mainly in converting the old "single-player mode" console-bound Urbit into a real system service. Urbit works better as a daemon which runs one or more worker processes, and talks to client consoles over a Unix socket. A real architecture actually makes it practical for star owners to host the planets they issue, for example. Finally, because we never forget the wisdom of the sages, our last act before asking people to use Urbit will be the long awaited performance crusade. The new event system now at least releases outgoing events before the input queue is clear! You never know what optimization will achieve before you make a serious effort, but I don't see any reason we can't carve out a new empire of speed in the savage, uncharted east. We haven't touched the documentation since the first half of 2016. As always, maturity has to come before documentation. Premature documentation is almost as dangerous as premature optimization. ## Notable contributions Some notable contributions of late: - `~ponnys-podfer` wrote a MUD - `~palfun-foslup` created a surprisingly capable talkbot - `~master-morzod` reworked the console in both the Hoon and C layers, adding color and cleaning up a bunch of stuff. Anyone who contributes at this point deserves extreme praise for the ability to work in an immature, often broken system. We look forward to being ready for normal programmers who aren't intrepid jungle explorers. ## Next crowdsale We'll do another crowdsale in the spring. We understand that the last sale left a lot of people unable to participate — and our first priority is making sure those who are excited about Urbit can own some real estate. We'll announce our plans on how this might happen soon. If you're interested, make sure you're on our mailing list. ## Contribution prizes We're still thinking about the right way to award Urbit real estate to outside contributors. Especially after the crowdsale, it really has to feel 100% fair. But it can't be in any way, whether legally or emotionally, a form of compensation. We can't afford to kill the fun. It doesn't feel right to do this at all when the documentation is completely inadequate, because it's not fair to people who don't have the time to reverse engineer. But we're thinking hard about this issue. ## Future updates We know you love these opinionated, literary state-of-the-urbit updates. We love writing them. But we obsess over them too long and they take too long to write. Urbit's ultimate success depends almost entirely on the code. Going forward we're going to switch to commit-message style updates on a monthly basis, with longer updates arriving sporadically. Maker DAO does a great job of these shorter updates ([here](https://blog.makerdao.com/2016/12/14/maker-update-925/) is an example). If you're ever curious to check in or hear what's going on, don't hesitate to get in touch. You can always chat with us on [urbit.org/stream](https://urbit.org/stream), or by booting your own Urbit. [Twitter](https://twitter.com/urbit) or email ([urbit@urbit.org](mailto:urbit@urbit.org)) are also perfectly fine. There's also [our forum](https://urbit.org/fora/) that anyone can post to. We've created [this thread](http://urbit.org/fora/posts/~2017.2.1..22.36.47..9f54~/) to discuss this post. ]]> https://urbit.org/updates/2016-10-6-update https://urbit.org/updates/2016-10-6-update Wed, 05 Oct 2016 00:00:00 GMT https://urbit.org/updates/2016-8-30-update https://urbit.org/updates/2016-8-30-update Mon, 29 Aug 2016 00:00:00 GMT ` to load libraries, using aliases correctly to allow multiple conflicting versions in a dependency graph. Urbit has a stable, extensible but backward-compatible I/O protocol between Arvo and vere (the Unix process). Any future Arvo will work on an old vere. Urbit uses Murmur32 rather than FNV as a short hash (mug). Urbit is still not a cryptographically secure system, and doesn't allow true cryptographic ownership transfers. The cryptographic transfer features exist, but using them is a bad idea until the keys are final. (For right now, everyone who issues urbits to others needs to at least track ownership centrally, ideally with a mirror database outside Urbit. We could not stress this more stressfully!) Urbit has received a thorough performance and scaling audit, and there are no trivial efficiency problems to solve. The Urbit console can output colored text and control a 2D ASCII text array (curses style). The web console (not web dojo) works with some keylag. All Urbit vanes are thoroughly documented/specified with an overview document and inline comments. ## SF: security freeze The SF release will render Urbit a "live" cryptographic system. It is unnecessary to point out that all subsystems in the security chain must be audited, and operational procedures for stars and galaxies need to be standardized. (Yes, the SF release needs to invalidate the current cryptographic structure, even if it hasn't been compromised. Since it's not secure, we don't know it hasn't been compromised.) However, the SF release keys will be instantiated via network update, which is essentially "god mode." New keys will be distributed to old key holders, and any disparities between cryptographic and centralized ownership records will be resolved in favor of the latter. Before SF, no one can gain anything by stealing a key or transferring it without authorization. A software update can of course do anything to the PKI. After SF, at least, global Arvo updates (which ideally can become rarer and rarer) will need to be approved through a constitutional, republican governance process. After the SF release, we have no control over cryptographic transfers, but the Arvo code itself may penalize long wills to minimize velocity and keep Urbit from acting as a currency. (This is essentially a Tobin tax.) The SF release needs to run the actual Arvo interpreter, with jets, in a completely sandboxed subprocess with no I/O privilege. Sandboxing jets, or even a safe jet language, should also be investigated. Network jet update must also be possible. An SF urbit can issue and manage children without any problem. It can be hosted easily in a commercial data centre. The interpreter can migrate images securely, automatically and efficiently to a different hosting service. An SF urbit will have a framework for DoS defence, blocking and banning, and other nuisances of the modern world. There will be at least the beginning of a global reputation system. ## CC: work overview The one subsystem that needs a complete rewrite is `%ames` (the network). The `%jael` vane is completely new. Fortunately, the new `%ames` vane compiles, and the `%jael` interface is defined (and not that `%jael` can be complicated). The mug replacement work is also done, but needs to be debugged (which is no easy task). Applying the canonical boot event system sounds like a giant change, but it's actually pretty straightforward -- same for single-homing. Arvo proper is only a few hundred lines of Hoon. The event system in `vere` needs to be replaced, but we have a module built by a contractor with both the local FS and Kafka. All the vanes besides `%ames` need documentation and a general tuneup, but this should not rise to the level of a rewrite for any other vane. ## Community involvement Urbit's most serious problem right now is that many more people are interested in it than are actually working on it. This reflects its heritage as a basement project. Our most important work item remains opening up the codebase so that community development can happen. We'll start turning the CC tasks into community operations as fast as possible. But this is a separate topic. It fits under a discussion of Tlon's corporate plans and operations, which we'll post later. ## Contributors The contributors have been amazing. We need to make more use of their talents, which are growing daily. And we need to do a better job of chronicling their deeds! But for now, github remembers. ## Keep in touch / office hours `:talk` is always a great place to come and ask questions. Going forward someone from Tlon will be available in `:talk` from 12PM - 3PM PDT each weekday. Post to our [new forum](https://urbit.org/fora)! Especially [this thread](https://urbit.org/fora/posts/~2016.8.30..18.05.18..8ba3~/) to discuss this very post. Don't be afraid, the water's fine! Actually there are a few glitches, but it's hosted on Urbit (and also backed up). For the less daring, [r/urbit](https://www.reddit.com/r/urbit) is a reliable old standby. For anything else you can always get in touch with us via [`urbit@urbit.org`](mailto:urbit@urbit.org). ]]> https://urbit.org/updates/2016-7-20-update https://urbit.org/updates/2016-7-20-update Tue, 19 Jul 2016 00:00:00 GMT https://urbit.org/blog/toward-a-new-clay https://urbit.org/blog/toward-a-new-clay Wed, 13 Jul 2016 00:00:00 GMT https://urbit.org/blog/why-urbit-probably-does-not-need-a-blockchain https://urbit.org/blog/why-urbit-probably-does-not-need-a-blockchain Wed, 13 Jul 2016 00:00:00 GMT An efficient countermeasure to combat double-spending on fast > Bitcoin payments would be for Bitcoin peers to propagate alerts > whenever they receive two or more transactions that share common > inputs and different outputs. "Fast Bitcoin payments" means zero-confirmation, ie, Bitcoin without the blockchain. These double-spend alerts have never been integrated into Bitcoin Core, despite frequent proposals. Given that Bitcoin _does_ have a blockchain, it seems quite sensible for purists to shun hacks that make zero-conf transactions somewhat more secure. Also, double-spend alerts in Bitcoin are a DoS vulnerability; Bitcoin has fees to control the cost of propagating transactions, but who pays for an alert? In principle, however, we could imagine a version of Bitcoin without a blockchain, in which the mempool was the ledger, and there was an alerting and/or conflict model to detect and resolve double spends. An accepted transaction is simply one that propagates to one or more randomly chosen observer nodes. This design, for the problem that Bitcoin is solving, intuitively feels weak and a little hacky. The Karame paper is not actually advocating for removing the blockchain from Bitcoin. At the same time, low-value zero-confirmation payments are commonly used. One way to think about security is to flip the tables and think of the defender as the attacker, and vice versa. Just imagine you have some moral right to double spend. The rest of the world is unfairly trying to stop you. Can you defend yourself? In the context of a gossip network that shares all the things, a procedure that involves telling one person one thing, and another person another conflicting thing, and not getting "hacked" by the unfair world which believes there should be only one truth, is inherently quite fragile. And where the attack is fragile, the defense is robust. Most of the security in any digital title system comes from the PKI itself. The double-spend problem, the general issue of Byzantine failures in information propagation, is already a corner case. So perhaps we can solve it as a corner case. Again: _even Bitcoin_ isn't that far from being able to build a practical decentralized ledger without a blockchain. The same gossip model that broadcasts nonconflicting transactions is not at all incapable of reacting to conflicting transactions. We're not really recommending that Bitcoin ditch the blockchain, but this gives us hope with our fundamentally easier problem. ## Various specific specialnesses What are the differences between the Urbit double-sell problem and the Bitcoin double-spend problem? Is our special case really, truly, special? In worse-is-better mode, we want every special-case advantage we can get. Every difference we find is one we can use as a leverage point. ### Minimal transaction interdependency The first major technical difference between the Urbit title history and the Bitcoin blockchain is that Bitcoin and its relatives are designed to secure a complete, interdependent, collective transaction history. Except for the parent-child relationship, Urbit wills are independent of each other. Bitcoin's use of UTXOs rather than balances reduces interdependency somewhat. But the validity of each transaction is still dependent on all the transactions that created its inputs. (Ethereum drops the UTXO model, whose value in reducing dependencies may not match its cost in complexity.) Urbit is not a payment system or a shared ledger. Bitcoins are divisible, but Urbit ships aren't. An Urbit will containing its parent chain is self-validating and has no dependencies. There is no collective history. There is no Merkle tree. There is no chain, with or without blocks. This simplification doesn't eliminate the double-sell problem in theory, but makes it much more tractable in practice. The set of past transactions (key transfers) that a new transaction could conflict with is much smaller. ### Trust hierarchy and governance structure The second major technical difference is that Bitcoin is a flat, trustless, fully decentralized system. Urbit has a hierarchical, polycentric governance system which is a trust hierarchy. Now and for quite some time, urbits default to automatically accepting OS updates from their parent, making this trust effectively absolute. This is not a novelty -- all "evergreen" browsers now work this way. You trust your browser quite a bit, which means you trust your browser vendor. And since few Bitcoin node operators code their own daemons, the same dependency on a source distribution exists in Bitcoin. It's informal and off-chain, though, which is not really a good thing. (Urbit is like if Bitcoin Core hotpatched itself automatically by publishing source-code diffs on the blockchain.) To be on the safe side, we should think of Urbit as a centralized system until the OS update that turns off automatic updates is shipped. This of course requires a certain measure of perfection, which cannot in any way be predicted. After Arvo proper becomes "deciduous" (apps will of course always be remotely hotpatched), child-to-parent trust should diminish to the power of the parent to deny service to the child. This will always be present without some major redesign. But it's opposed by the equal and opposite power of the child to switch to a different parent, creating a competitive market in governance. This should deliver service at the highest quality and lowest price, fortifying trust with incentives. The power of a restaurant owner to deny you service does not give the food industry the power to starve you to death. And if no parent wants your business, with 2^16 stars who could accept it, you probably shouldn't be on the network at all. Urbit, like a real society, is designed to be a network of incentives that holds everyone in productive tension. We want everyone to be free, and no one to be unaccountable. For example: a parent which extends its will also needs to send updated signatures to its children and former children. Either will otherwise be unable to propagate their dossiers to ships whose cabinets have the updated parent will, _even if some other parent adopts them_. But Urbit is a collectively governed and updatable system. A star or galaxy which denies service to its own children will (a) get a bad reputation, (b) drive its children to other parents, and (c) force the network to hardcode a ship-specific exception into the propagation algorithm, overriding stale-key revocation. In theory, a parent can extort its children by denying service. In practice, since Urbit is self-governing, not ungoverned, and its government can right this wrong with a source-code patch, the extortion won't work. So it won't happen. So the act of government won't be necessary, and won't happen either. Of course a perfectly decentralized system would be freer. But worse, as usual, is better. And since we are paying for this worseness, imperfect decentralization, we'd like to get all the mileage out of it that we can. Again, Urbit, not being a trustless system, does not have to solve the trustless consensus problem -- although we still very much want to minimize and bound the delegation of trust. This is an obvious relaxation of Bitcoin's hard theoretical problem. ### Routing: security through topology Please don't laugh. Security through routing topology isn't a joke -- unless firewalls are a joke. Cisco sells too many firewalls for them to be a joke. In Bitcoin we may have all sorts of reasons to ask what ship holds what coins. In Urbit, when we ask what key controls an ship, we are either sending or receiving a message. Is our outgoing message safely sealed to the current owner? Is our incoming message properly signed by the current owner? When the parent also forwards all packets until a P2P link is established (Urbit forwarding resolves to STUN if either party has full cone NAT, TURN if not), we can design in all kinds of interactions between routing and certificate management. In particular, a parent which is a single source of truth in some sense is a very attractive design element. Again, we're paying for our imperfect decentralization. We might as well use it. For example: I want to send a packet to ship X. Two parties, A and B, are competing for ownership of X. A is the rightful owner. I am secure against B if either of these conditions are true: my packet can only be decoded by A, or my packet is not sent to B. Both would be ideal, of course. Either will do. ### Latency and friction tolerance Confirmed Bitcoin payments aren't especially low-latency by payment standards. They're reasonably low-friction by payment standards. But fees are certainly a thing. Urbit is digital real estate, not digital money. When we look at the standards of latency and friction in actual real estate -- where a transaction that takes a week is super-fast, a 5% payment overhead is normal, and a third-party intermediary is standard -- we see a much, much weaker set of requirements. Obviously a week and a 5% overhead are incredibly sucky levels of latency and friction. The point is that even if this is the only level of performance Urbit can achieve, that's not necessarily the end of the world for Urbit. For instance: if we wait a week for a double-sell alert, and we don't see one, either the gossip network is totally hosed (to a point we'd surely find out about), or there wasn't a double-sell. This stuff is just not that hard. (One little-made point about Bitcoin, by the way, is that Bitcoin enables trust-free _payments_, but not trust-free _exchanges_. The seller still needs to deliver the goods either before or after the payment; in either case, there is trust. Only Ethereum can solve this problem -- and only for digital exchanges.) ## We can afford to be patient With a better-is-better, general-purpose solution, one size fits all until the end of time. With a worse-is-better, special-case solution, the details of that case are critical. But we actually don't know these details. One rule of the worse-is-better school: never solve a problem before you have to. You don't know all the details, and you need those details. (Of course, with the better-is-better approach, the details don't matter at all.) Is there such a thing as a transitive adjective? Grammarians are unclear. But technically, the adjective "secure" is always an error. Nothing is "secure." "Secure against X" is the correct usage, where X is some threat model. Urbit is an insecure test network with no sensitive data on it. This will remain true for a while. No Urbit ship has ever been transferred cryptographically. This will also remain the status quo for quite a while. We _don't know the threat model_. We _can't solve the problem_. It's as simple as that. The best Urbit can do, at the present time, is map out the ballpark of solutions. And we have time to proceed patiently. One way to think about the timing of cryptographic decentralization is to look at the DNS. It's clear that now, a decentralized DNS with cryptographic domain-name titles would be extremely desirable. It was never designed to happen. It will probably never happen. But if we look at when cryptographic DNS with allodial title _should_ have happened, it's probably sometime in the mid-90s. Ie, after over a decade of production use of the Domain Name System. In short: there's no need to hurry here. ## A map of the solution space But we still need a map of the worse-is-better ballpark. Let's look at some options, from least secure to most. The essential question we can't answer, without a better sense of the threat model, is how aggressively we need to propagate wills. We especially are not sure about the threat model of malicious stars (16-bit ships) and/or galaxies (8-bit ships). We need to know what kinds of double-sell attacks are plausible. Since we don't know this yet, we need not one propagation design, but a range of designs -- stretching all the way up to a full blockchain. ### Acceptance model What doesn't change is the will _acceptance_ model. A ship will always accept a new will if (a) the new will extends the old will, (b) the parent signature on the initial public key is at least as recent as the ship's current version of that parent. The parent clause implies that when a parent extends its will, it needs to send updated initial signatures to all the children it's launched. Otherwise, their own future updates will not be accepted by ships that have seen the parent's update. This is not a big deal in the world of parent responsibilities. But like many such responsibilities, it lets a badly operated parent deny service to its children. A ship accepts all wills, new or updated, and stores them permanently. If this is abused trivially as a DoS vector, Urbit will mitigate it by the usual technique of degrading service to non-scarce ship tiers. Scarcity rules. ### Level 0: passive propagation only The Ames protocol can piggyback a will on any packet. Each ship also permanently records which other ships hold which version of its own will. An update is automatically sent when needed. With level 0 propagation, all key exchanges in Ames are passive and invisible to the programmer. When the sender has no will or symmetric key for the receiver, the packet is signed, bundled with the will, and sent as cleartext. Signed cleartext? If you have no will for a peer, you've never talked to this ship before. Your first words to a stranger probably won't be super secret. If this isn't true, the programmer has to send an innocuous message first and wait for it to be acknowledged. Kicking a rare corner case up to the programmer is a classic worse-is-better move. With level 0 propagation only (the current implementation, because we don't even care about key updates yet), an updated does not effectively revoke the old key. If Alice updates to a new key, Mallory steals the old key, then Mallory talks to Bob before Alice does, Mallory can impersonate Alice. So not only does level 0 not solve double-sell, it's also quite insecure. But it's still fine for now (because Urbit isn't really decentralized yet). ### Level 1: neighbor-only propagation with parent filtering The list of other ships which hold your will, essentially the list of all ships you've talked to, is your list of _neighbors_. You essentially have permanent, if perhaps very sporadic, conversations with all your cryptographic neighbors. Symmetric keys time out fast, but wills are held forever. By proactively pinging all your neighbors when you extend your will, you prevent anyone who finds the old private key (ideally you update when your old key is stale, not when it's actually been _stolen_) from jumping into these conversations. What about non-neighbors? We add a second feature: a parent will only forward a packet if its sender matches the current will. So updating your parent (who is always your neighbor) transforms it There are two cases: where Alice's parent Trent is also Bob's parent, and where Bob has a separate parent Walter. In the first case, the expected packet flow is that Alice will send the packet to Trent, who sends it to Bob. In the second case, Trent sends it to Walter, who sends it to Bob. In either flow, Bob or Walter is sure that the packet came through Trent. Since Trent is Alice's parent and hence Alice's neighbor, and hence is updated, Trent will forward no packets from Mallory. So Mallory has no way to reach Bob. Non-neighborly packets not matching this flow can't be accepted. Suppose Mallory gets Bob's public IP address directly? If Bob gets a non-forwarded packet marked "Alice," but Alice is not his neighbor, that's super weird. He must reject the packet. Level 1 is nice because it provides a reasonable level of security, but it's still extremely simple. Not only does it not use a blockchain, it doesn't even need a gossip flood! And level 1 propagation also does a competent worse-is-better job on the double-sell problem. Trent simply becomes the single source of truth. The buyer of Alice needs to check that Trent, her parent, has Alice's latest will from the buyer's perspective. If Alice's seller propagates a different fork to other ships, this damages Alice's ability to communicate, but produces no obvious gain for the seller. Level 1 puts a lot of security weight on the parent. If Trent misbehaves, for example by not filtering Mallory's packets, Alice can be impersonated. And its solution to the double-sell problem is adequate, but not ideal. Level 1 is the worse-is-better sweet spot for a young Urbit, but will probably give way to level 2. ### Level 2: broadcast all wills We can't say definitively that level 1 will be inadequate. But intuitively, it feels slightly weak. We will probably need a gossip protocol. The only ships that can be transferred are galaxies, stars and planets; if all updates of all wills of all these major ships are broadcast, we hardly have a scaling problem like Bitcoin's. Even if changing the lock on your front door counted as a real-estate transaction, there are a lot fewer real-estate transactions than credit-card payments. Moreover, conflict detection (double-sell alerts) is a natural part of any gossip protocol. Gossip should be between stars, not directly between planets; horizontal gossip, vertical distribution. Star-to-planet distribution may still be limited. Especially since stars are somewhat trusted, ganking a gossip protocol with random update edges is incredibly hard and requires a pretty preposterous conspiracy. And flooding all planet wills does a lot to keep stars honest. It would be incredibly stupid for a star to try a double-sell attack against its own planets, but code is always more powerful than mere incentives. Finally, the exit model isn't fully baked yet, but it will probably have more aggressive propagation needs. What's clear is that an exit statement is another type of will entry, involving a signature both from the previous key and the new parent. You can only escape to another ship of your parent's class that agrees to adopt you. In general, because escaping need to work for routing (without the cooperation of the former parent), wills containing an exit statement (a planet moving to a new star, or a star moving to a new galaxy) exit) probably need to be flooded globally among all 8-bit galaxies and 16-bit stars. Exits should be rare, so they shouldn't present much of a scaling problem. ### Level 3: full blockchain Obviously a blockchain can solve Urbit's problem. Probably the right thing to do would be to use the Bitcoin blockchain, and confirm the current Urbit key version with a "colored coin" of some kind. (This should not be confused with the orthogonal idea of representing Urbit _invitations_ as colored coins.) This involves making all Urbit ships into Bitcoin SPV nodes at the least. Not the worst fate in the world, but also a cost. Alternatively, Urbit could have its own "altcoin" blockchain, with actual Urbit-specific mining. Again, given that level 2 seems like it should work quite well, it's hard to see why level 3 propagation would be needed. But we can't really know at this point. There's a reason Urbit can update itself over the air. That's why we say: Urbit (probably) doesn't need a blockchain. If it does, we can retrofit it. ## Retrospective note This essay has presented the Urbit cryptographic title as a low-power, stripped-down, worse-is-better variation of Nakamoto consensus. That's clearly the right way to introduce it in 2016. Actually, though, this part of Urbit was designed before the Bitcoin paper was published. There were other academic approaches to Byzantine consensus, but certainly none validated on a large scale as Nakamoto consensus has been. It never seemed appropriate to treat cryptographic titles to network addresses as a case of generalized Byzantine consensus, and without the existence of Bitcoin there was no temptation in this direction. In a sense the secret of PKIs is that PKIs are easy. What's hard is adapting an existing identity and authorization structure into a PKI. If you build the simplest possible PKI, then force the human processes to fit themselves to the protocol instead of vice versa, you have a much easier job. Similarly, if you are not adding PKI to an existing network, but building the network and the PKI as one system, many simplifications are possible. This is once again the classic worse-is-better method. ]]> https://urbit.org/blog/common-objections-to-urbit https://urbit.org/blog/common-objections-to-urbit Mon, 27 Jun 2016 00:00:00 GMT Urbit is not ready for prime time. _TLDR_: we agree. Urbit right now is a fun toy. It is fun to learn Hoon. It is even fun and quite practical to make actual contributions. (Hoon's IEEE floating-point was written by someone we've never met: `~nym`, formerly `~nodpet-rinned`.) In fact, we have never seen anyone _not_ have fun with Urbit. But there is all kinds of stuff wrong with the codebase. Urbit's documentation is much improved, but remains incredibly weak -- Arvo is almost totally undocumented. Scaling is poor at best. Urbit's reliability is dubious. Its security is nil. Urbit is feature-frozen. The only thing we're working on now, and for the rest of 2016, is doc, debugging and and self-hosting. In that order of importance -- teaching Urbit is more important than fixing it, which is more important than making it useful. But we're quite confident that these are all solvable problems. The most important milestone about Urbit today is that it's ready for technical decisions to be made collectively. A system like Urbit can only be invented by a mad auteur. It can't be maintained or stabilized by a mad auteur. Our primary goal today is expanding the number of people who understand Urbit, and we expect these pioneers to participate in technical governance. ## Nock jets are a total hoax. _TLDR_: all of computing is a hoax. You can't compare a hoax to nothing. You can only compare it to all the other hoaxes. A "jet" is a sort of software device driver in Urbit. Urbit is precisely defined in terms of an interpreter that fits on a T-shirt, Nock. But a naive implementation of Nock is nowhere near practical, because Nock has only one integer operation: increment. For example, to decrement a number in Nock, you increment a counter up to the number under it. The only way an efficient Nock interpreter can work is to recognize algorithms like decrement, and implement them non-naively. By far the easiest way to do this is not to analyze arbitrary code as it's being executed, but to register and match well-known implementations in the standard library. No one should be writing their own decrement. But the semantics of the interpreter, _so long as the jet and the jet harness are defined correctly_, are exactly those of Nock. We are not extending or changing Nock in any way. We are just taking a shortcut in certain cases we understand perfectly. It's not at all unprecedented to have implementations much more complex than the corresponding specification. The x86 instruction set is relatively simple; the complexity of its decoding and execution path in a modern CPU is enormous. This path may well have "peephole optimizations" that replace matched algorithms, and if it doesn't the modern compiler does. People, _bad_ people, use "jets" to cheat on benchmarks all the time. Nock is designed as a standard. It's built like a standard. Typically what causes real problems in standards is ambiguity. Two implementations are incompatible, but each swears it has the right interpretation of the natural-language spec. This simply cannot happen in the Nock world. If two implementations disagree, at least one of them is incorrect. This puts us firmly in "with enough eyeballs, all bugs are shallow" territory. The fact remains, however, that a Nock interpreter without jets is completely useless. And there is not even an informal document that tells you what jets you need for a reasonably efficient Urbit. (There should be such a document.) The correct way to think about jets is not to compare them to a perfect interpreter created by unicorns. Jets are a combined replacement for two other techniques in language implementation: intrinsics and a foreign-function interface. Compared to both intrinsics and an FFI, your jets have one key disadvantage: they need to be specified in pure code. You may not have this executable specification, which means you need to write it. Is this really so bad for your development process? An executable specification? Which can be trivially tested against the efficient implementation? Compared to intrinsics, jets give you true separation between mechanism and policy; only the interpreter, not the programmer and not the compiler, needs to know what's accelerated. Compared to an FFI, jets preclude I/O, so can be trivially sandboxed. Except that the programmer needs to mark routines which may benefit from optimization with a hint, here too we separate mechanism (efficient implementation) from policy. Yes, formal verification of jet equivalence is desirable. However, in the real world, many specifications, including quite complex ones, are tested into equivalence by brute force. Urbit has some jet abuses. The equivalence between our markdown parser in Hoon and the CommonMark C library is super dubious. It should be revisited. But the miscomputation resulting from a minor jet discrepancy is not a security hole or even fatal. It effectively means the event history, computed correctly, does not match the current state. The way to resolve this conflict is for the current state to win, and the event history to be discarded -- not abnormal anyway. See the next question. ## Urbit needs an infinitely extending packet/event. _TLDR_: no, it doesn't. Urbit can prune the log. We can prune the event log because the Urbit transition function doesn't have any access to the event history -- just the current state and the current event. Urbit defines the semantics of a complete event log. It doesn't and can't require an implementation to maintain the whole log. It can maintain a checkpoint and the log since that checkpoint. However, any practical urbit maintains a periodically updated checkpoint. Our present implementation doesn't provide a way to prune the event log. But that's just a missing feature. We have one or two of those! ## Urbit is a total scamcoin. It's 100% premined. _TLDR_: yeah. But at least it's not the DAO. Like a cryptocurrency, Urbit is a digital title. But Urbit is digital land, not digital money. If you assess it as a currency, it's a lousy currency. If you assess a cat as a dog, it's a lousy dog. In real-world economics, the crucial distinction between land and money is that land (a) has intrinsic utility and (b) is non-fungible. Because of (b) alone, frictionless transactions in land are impossible. A blockchain is only needed to ensure zero-cost high-frequency transactions. So digital land doesn't need a blockchain. Which means it doesn't need mining. Of course, just because a digital title system doesn't need mining, doesn't mean it can't distribute an initial allocation by proof-of-work -- or by any other arbitrary method which is careful not to benefit the system's own developers. Could we give away the Urbit address space randomly? Sure. But why? Probably the purest design is the "proof-of-burn" initial allocation used by some coins. By any standard, proof-of-burn is technically beautiful and results in an extraordinary level of ethical independence. We can't help but admire it. But any altcoin or metacoin is just way less work than Urbit. An altcoin can afford to burn even its creation energy, capturing no value at all from its seigniorage. Your altcoin doesn't have a clean-slate system software stack to drive. If your rocket has to go to the moon, it's going to leave a little bit of smoke in the air. Ethically, what makes "premining" feel wrong is that the creators of the currency are rewarded disproportionately for their effort. The scamcoin is a get-rich-quick scheme, vaguely pyramid-shaped. Nothing about Urbit matches this pattern at all. Curtis invented Urbit. So by the [homestead principle](https://en.wikipedia.org/wiki/Homestead_principle), he started out owning all of it. But by now, Urbit's 8-bit galaxies have already been distributed among [57 separate individuals](https://github.com/urbit/arvo/blob/master/sys/vane/ames.hoon#L118-L373). Curtis owns 20, or about 8%. Tlon, the corporation which still runs Urbit development, has 95. (Without Tlon, there is basically no Arvo.) 50 are reserved for a hypothetical Urbit Foundation. Tlon employees have another 16. The rest of the galaxies belong to donors, investors, contributors, contest winners, and just random people who were in the right place at the right time. One is a Silicon Valley tycoon. Another is an Orthodox nun in a monastery in Greece. A couple are ex-Tlon employees who might totally hate us. There is nothing they all have in common, which is sort of the point. This property distribution looks a lot like land ownership in the real world -- complex and asymmetrical. That's no accident. If you have a problem living in New York because New York's real estate is not distributed equally among its residents, you may have a similar problem with Urbit. The reality is that Urbit, like New York, wasn't born yesterday and comes with a complex ownership history which we couldn't possibly change, even if we wanted to. Decentralization is hard. The only real goal of a property system in address space is to create and maintain decentralization. The whole network has to exhibit centrifugal force, not centripetal force. Proof-of-work systems certainly seem to have unexpected centripetal qualities, producing unanticipated concentrations of centralized hashpower. See the next objection. ## Urbit isn't even really decentralized! It has a government! _TLDR_: it's technically impossible for Urbit to be decentralized at this stage of its life. Urbit is designed to achieve decentralization, not be born with it. A young network can't afford to be decentralized. It has to act as a unit. Because it has no network effect, it needs every scrap of efficiency it can get. A decentralized general-purpose computing network like Urbit can _only_ be bootstrapped by a central government. For example, by default your urbit upgrades its OS automatically (like an "evergreen" browser) with signed network updates. Whoever signs these updates is a government by definition. When it's impossible to eliminate centralized power, it becomes necessary to tame, stabilize, and limit it. This should not be surprising to anyone living under a constitutional government. As we found out with the DAO, if a network isn't ready to be decentralized, its developers shouldn't even try. If Ethereum is hard-forked to roll back the DAO (which looks like the most probable result), it acknowledges that it both has a central government, and needs a central government. Decentralization is good. Pretend decentralization -- "decentralization theater" -- is harmful, and not a effective path to actual decentralization. A key test of a free software project's health is whether it can fork. So long as the unity of the public-key hierarchy and the network protocol is maintained, the Arvo kernel and Urbit system can fork in all kinds of ways without disrupting the network. Any stress will fragment the "government" that develops the kernel, and as it fragments it grows more limited and stable. Urbit is designed to govern itself collectively by address space ownership -- effectively, "proof of stake." Any arbitrary power, will tend to fracture along these property lines. For instance, our hot updates are sent through the galaxy hierarchy; in a development fork, active galaxies would pick upstream sources. Two implementations of a standard are better than one. And by fragmenting the galaxy space into 57 separate partitions, we've created a global property map which is much more likely to splinter further, than reunite. In the long run, every galaxy should have a separate owner, who is a human being rather than a corporation. Combining galaxy holdings will damage the reputation of both galaxies. The only reason to merge ownership is to move toward incipient monopoly. Whether a galaxy admits this openly or denies it dishonestly, its reputation will suffer. The technical goal of Urbit is to create a network which, _if it actually takes over the world_, inevitably becomes decentralized. Even if Urbit takes over the world, no individual or coherent organization takes over the world. There is nothing even close to a single point of failure in a mature, successful Urbit. This is the goal of all the new "web3" decentralized systems. If IPFS, Sandstorm or Ethereum takes over the world, nobody takes over the world. (The same is almost true of the Internet -- though it wouldn't be that surprising if a time traveler from 2116 revealed that ICANN had become a planetary government.) Urbit, by starting out as a centralized network, just reduces the technical problem to match the actual goal. Only long-term decentralization matters. It's very important that Urbit in 2116 be fully decentralized. For Urbit in 2016, pretending to be decentralized would be a joke at best, and a hoax at worst. And of course, no one can DAO us -- since the property system is still just a central registry. Among other things, we check it into GitHub. GitHub is not a blockchain -- but still, good luck with that. ## Sandstorm/IPFS/Ethereum is way ahead of Urbit. In general, web3 (next distributed web) solutions don't compete with each other. They support each other. Concretely, they are all part of each others' network effect. IPFS, Sandstorm, Ethereum, and Urbit can all take over the world at the same time. None of them competes with the others. They enable each other. For instance, IPFS and Urbit are both global immutable namespaces. IPFS is a chord-routed, content-addressed namespace. Urbit is a direct-routed, identity-addressed namespace. Urbit is a typed revision-control system; IPFS is a blob store. IPFS isn't a computer; the work hasn't been done, but Urbit can easily mount the IPFS namespace and compute functionally against it. So the better IPFS does, the less Urbit needs to struggle against its lack of network effect. The better Urbit does, the easier it is for IPFS to point to a functional language that can actually use its referential transparency directly. So their relationship is symbiotic, not competitive at all. The closest thing to a general-purpose competitor that Urbit has is Sandstorm. Sandstorm, which by the way rocks, is also a personal server built around a network protocol. But Sandstorm and Urbit compete in the way that, on the PC, the OS and the browser compete. In theory, the OS could solve the problems that the browser solves; or the browser could be the OS. In practice, Windows is a great OS to run a browser on. Similarly, a world in which Sandstorm takes off is an awesome world for Urbit, because everyone who has a Sandstorm instance has a great platform for running Urbit on. Urbit on Sandstorm will be a sort of "double-hulled" personal server, safer and more friendly than either system alone. Sandstorm and Urbit have very different approaches to network identification; Sandstorm uses capabilities, Urbit identity. Broadly speaking, we believe, capabilities are optimal for expressive power; but identity is optimal for the UI of the common case. We'd need to see the Powerbox, Sandstorm's UI, more fully fleshed out to understand its UI approach better. ## Urbit doesn't have enough planets for every human A 32-bit planet is a tool, not a toy. Like a car, it's a device for a responsible and independent adult. There aren't 4 billion cars in the world, nor 4 billion independent adults. If you aren't an independent adult, and you don't need or even shouldn't have unconditional digital freedom (no one's 8-year-old daughter needs unconditional digital freedom), a moon from someone else's planet is fine. (Even most of today's independent adults don't complain enough about being Facebook's moons.) And limiting the supply of anything doesn't create a shortage. It sets a price. Ideally, in the long run, we think a planet should cost about $10. This could be off by an order of magnitude either way, and it wouldn't be the end of the world. If a planet costs $10, that means you paid $10 to get a planet. A planet with no reputation has a positive reputation, because someone paid $10 for it and hasn't yet done anything wrong. Positive default reputation is the single quality that makes a social network social. There is always a tradeoff between free speech and free beer. Urbit will always choose free speech. ## Cloud computing is inherently non-private. _TLDR_: a general-purpose cloud computer is much, much more private than a special-purpose cloud computer. In general, a cloud VM is secure against corporations but not governments. With a global adversary, compute in your closet -- or even on your body. One reason we want the decentralized Internet is that we're tired of giant corporations spying on us. Think about it this way: Amazon is Netflix's main competitor. And yet, Netflix runs its whole operation on AWS. Why is Netflix sure that Amazon won't spy on it? A general-purpose virtual machine is special from a privacy standpoint, just because it's commercially realistic to design terms of service that prevent the host from peeking and poking at the customer's machine. Crossing this moat is technically nontrivial and can't possibly happen by any reasonable accident. No hosting contract could contemplate it. Whereas when your cloud computer is a special-purpose machine -- just a database row on some corporation's mainframe -- there is no way these services can promise not to handle your data. Handling your data is _what they do_. On a special-purpose server, the concept of privacy is not even defined. Where even a general-purpose server falls down is where the user has a sovereign adversary, which can secretly violate explicit or implicit hosting contracts. Users with a sovereign adversary must rely only on their own physical security. However, most users don't have a sovereign adversary. And those who do maximize their "herd immunity" if they can use the same protocols as those who don't. Otherwise, the adversary could simply focus on everyone who uses the secure protocols. Also, the portability of a cloud server image is critical. Of course, since Urbit is formally specified, it's trivially portable. In fact, one strategy against sovereign adversaries may be continuous rotation between hosts, like an MX missile. ## Urbit will never be able to call any existing code. _TLDR_: neither will Javascript. And dear Unix, the browser does not miss interacting with your `$LD_LIBRARY_PATH` at all. You can't talk to the OS or use system libraries from Javascript. Therefore, there is no point in having any kind of gateway, compatibility bridge, native methods, or other interface between Javascript and C. When in doubt, remember that Urbit is "the browser for the server side." It plays by the same general rules. Isolation means isolation. ## There's no way Urbit will ever be adopted. It's too weird. _TLDR_: Urbit adoption is designed to be driven by ordinary end users. Urbit isn't ready for ordinary end users yet. We're working on it -- come help. Right now, Urbit is a developer tool. Or rather, a developer toy -- since the only thing it's useful for is developing Urbit. It also could be a much better toy. We're working on that too. But it's absolutely wrong to think of Urbit being adopted as a developer tool in the conventional sense. If you think this is almost impossible, you're right. If we look at the way most successful platforms are adopted, they're not adopted for their own virtues. They're adopted because they're part of a larger solution. It's very hard to displace a native component of an existing platform. Urbit is not trying to do this. C is a better language than Pascal, but it didn't beat Pascal because it's a better language. C beat Pascal because C was the native language of Unix, and people wanted to program in Unix. People don't use Javascript because Javascript is a great language, but because it's the native language of the browser. Replacing the native language of a platform may be doable -- but it's incredibly hard. Hoon is not competing with C, or even Haskell, as a Unix programming language. The only reason to program in Hoon is that it's the native language of Urbit. Urbit is not competing with node.js or Rails as a Web backend. The only reason to run Urbit is to run a personal server. Nock is cool, but it only gets adopted if Hoon gets adopted. Hoon is cool, but it only gets adopted if Urbit gets adopted. Urbit only gets adopted if it's useful as a personal server for human beings -- most of whom won't be Hoon programmers, or programmers at all. ## Hoon is too weird. No one will be able to program in it. _TLDR_: Hoon is ridiculously simple. Our documentation and tooling are just immature. It's pretty fair to assume that (a) functional programming is hard and weird, (b) Hoon is weird even for a functional language, so (c) Hoon must be hard-squared. Actually, Hoon is weird even for a functional language because it's an attempt to remove the hardest things about functional programming: category theory, lazy evaluation, and gratuitous abstraction. To know a language is to know what the compiler is doing; all the semantics of the Hoon compiler fit in 2000 lines of code. It's too easy for us to wind up living in Isaac Asimov's Trantor, a world in which everything got invented thousands of years ago and the idea of new technology is just weird and scary. History isn't over yet, and nor is technical history. New things are harder to learn than old things. We ask smart high-school students to learn calculus. Think about how much harder it was for Newton and Leibniz to invent calculus, or for their first students to pick it up. Hoon is nowhere near as hard as calculus. Most of the difference is just the collective certainty that our society conveys to a high-school student that calculus is learnable. When you're learning Urbit, you don't have this collective knowledge; so you don't have the confidence that the problem is solvable. Confidence makes a huge difference. Moreover, any bug or glitch in the developer experience destroys this confidence and takes a long time to heal. Yes, we know: Hoon remains quite hard to learn. Again, we have no reason to think this problem isn't solvable. Public enlightenment remains our main priority. ## Urbit programmers are stuck with Hoon. _TLDR_: patience, grasshopper. There's quite a straightforward way to implement any perfectly specified language with a reliable interpreter in Hoon. First, write an interpreter for that language in Hoon. Second, jet that intepreter function with the existing implementation. It is not clear that Javascript is _quite_ well-defined enough for this treatment. But Lua probably is. ## Ordinary users will never be able to manage an Urbit server. _TLDR_: Urbit's technical simplicity has always been designed to drive human usability. But the polish is the last thing that goes on. So we can't evaluate this claim right now. Right now, Urbit is only for geeks. You already need to be able to wrangle a Unix machine to run it. What's the point? This Urbit isn't the real Urbit. It's an early milestone on the way to the real Urbit. The real Urbit -- for normal human beings -- is a hosted service. Getting a planet is as easy as signing up for Facebook. Probably many users will think of it as another web account. They might not even know it can run more apps than the ones they use. A deterministic, persistent cloud computer is something different from a traditional server. Normally when we think of a server in the cloud, we think of it like a live animal -- a creature that needs to be cared for, fed, and herded. A file in the cloud is more like a rock -- you can forget about it for ten years. An urbit is a general-purpose computer, but it doesn't even have the concept of a reboot. It upgrades itself. If you don't feed it any packets for ten years, then turn it back on, it will spend a day or two upgrading itself, then be back to normal. Maybe it's something in between a rock and an animal -- a cactus? Another huge difference versus an Internet server is that we're confident that Urbit's limited supply of identities will make it easy to keep abusers off the network. So far, so good -- but of course, no one abuses a young network. The bottom line is that managing your urbit has to be as easy as managing your iPhone. Normally, it consists of deciding what apps you want to be running, and how much you trust them. Resources can't be free, so sometimes you have to play memory police. Humans have shown that they can handle these tasks. Our '70s OSes and networks, which were great designs in their day and have matured into excellent industrial tools, give us a totally misleading impression of how hard it should be to manage a general-purpose computer. Computing isn't scary or dangerous. Unix and the Internet are scary and dangerous. That's fine for an industrial tool, but not a personal tool. ## Urbit has no killer app. _TLDR_: the killer app is personal API aggregation. It's a truism that any new platform has to be fundamentally much better at solving some problem than the mature platform it seeks to replace. We can look at Urbit's killer app from two perspectives: as a developer, and as a user. From the developer's perspective, Urbit's advantage is that it's a network stack which provides exactly-once delivery of typed, validated, transactional, encrypted and authenticated messages. The difference between an Urbit API and a REST API is like the difference between a paved road and a dirt road. On the other hand, you do need to learn a new functional language. From the user's perspective, there's one task that any personal server can perform, that can't be performed by any Web service: unrestricted API aggregation. Well before Urbit is ready to _replace_ any conventional web services, it should be ready to _command_ them in place by API. (Or if needed, by scraping.) If all you care about is being in control of your own data, it's fine to leave it in the hands of these services, as you have the API secrets to get it out. Managing a set of security connections to your existing services is one of the first important tasks of a modern personal server. This is a stateful task that typically requires both a durable API key and a temporary OAuth token. APIs change all the time, so connection maintenance also requires evergreen updates. The origin of this API key will vary, but usually the best way to set it is BYOK: the user registers their own urbit as an "app." And the neat thing about having your own API keys, for multiple services, on a single general-purpose computer, is that none of these services really has any practical way to control what you do with your own data. Terms of service which constrain the use of data (as opposed to the abuse of the service) simply aren't realistic and will no longer apply. This enables a variety of novel user experiences which aren't implementable as traditional web services, and also aren't available via conventional third-party API aggregators. Also, since Urbit is after all quite immature, it doesn't make much sense to restrict this security state to code within Urbit. The Urbit interpreter `vere` runs as a Unix daemon that implements the FUSE user-level filesystem protocol, and maps the data on your services into the Unix filesystem. For example, one of our demos lets you `ls` your GitHub issues. So it's straightforward to build, for instance, a Python app that uses Urbit as a gateway to mash up a bunch of web services. As a Unix programmer, you might even just regard Urbit as a generic Web API gateway daemon, without worrying about its internals. We have other ideas for initial killer apps. But this, we feel, is the best one. API libraries for `npm` et al abound. Stateful, self-updating, generic API aggregators with FUSE integration, also integrated with a reactive web server, are rarer. ## Urbit is sniffed at by Houyhnhyms. It's difficult to categorize [this awesome essay](http://ngnghm.github.io/blog/2016/06/11/chapter-10-houyhnhnms-vs-martians/) in a single bullet point. So I've saved it for last. When reading this essay, it's important not to forget the amazing absence of genuine, mutually critical discourse in computer science (as in most sciences today). You never hear what Swift's designers think of Go, or vice versa. And critical public discourse decreases, not increases, with increasing academic legitimacy. The basic test of any field purportedly scientific in nature is whether its researchers are genuinely interested in each others' work, and willing to share these genuine responses publicly. If this practice does not exist, the field may still be useful and/or interesting. But it is not science. What's awesome about Fare's essay is that he's _doing it right_. All the actual details are irrelevant. And mostly wrong. But that's of very little importance compared to _doing it right_. Anyway, we've already addressed the idea that jets are a sham. Some other points: ### Event-sourcing and/or caching is a pattern, not a layer Fare' is perfectly right that "one size does not fit all" in persistence. Generally, Urbit uses event-sourced persistence. But it has multiple layers of event-sourcing: in the definition of Urbit itself; in the application sandbox vane (Gall), and in the application itself. Urbit also has a completely orthogonal revision control system (Clay). And there is not a single cache across the whole system, though arguably there should be; for instance, the functional build system (Ford, which Fare' the master of build systems might appreciate) has its own internal cache. It is a common mistake to confuse a design pattern with a layer or feature, and equally fatal whichever direction you fail. ### Let's be clear about why DSLs suck DSLs suck not because they are difficult to newbies in the base language. DSLs suck because every time a programmer, even one _experienced_ in the base language, comes into a new codebase or project, she has to learn a new custom language. You need enormous wins to justify this cost. It's the goal of the base language designer to preempt these wins, creating a language which is almost never so tiresome and full of boilerplate that it requires a DSL or template layer. Hoon is almost entirely made of macros -- but they are internal, fixed, built-in macros (with no gensyms). This fixed set of macros is essentially enough. Occasionally we want to add one or two. But we never, ever pine for context-dependent macros. ### Determinism is not consensus, and is not constraining Fare' is totally right that an urbit's data is meant for itself alone. In fact, an urbit has the right to compute incorrectly if it wants. We go to considerable lengths to ensure that you can't gain any particular advantage by incorrect computing. The whole point of Nock is that it's so low-level that there is no conceivable advantage in upgrading it. Divergence is just pointless. This is not a question of network integrity, just of practical standardization. This consistency stops at Nock. Without anyone violating the correctness of their lifecycle functions, anyone or everyone in Urbit could replace Hoon with Javascript. Determinism does not cramp Urbit's semantic style in any way, shape or form. ]]> https://urbit.org/blog/the-dao-as-a-lesson-in-decentralized-governance https://urbit.org/blog/the-dao-as-a-lesson-in-decentralized-governance Thu, 23 Jun 2016 00:00:00 GMT https://urbit.org/blog/interim-constitution https://urbit.org/blog/interim-constitution Sun, 15 May 2016 00:00:00 GMT https://urbit.org/blog/magic https://urbit.org/blog/magic Sun, 15 May 2016 00:00:00 GMT https://urbit.org/blog/the-urbit-address-space https://urbit.org/blog/the-urbit-address-space Sun, 15 May 2016 00:00:00 GMT Wherever possible, work toward the evolution of independent regions > in the world, each with its own natural and geographic boundaries; > each with its own economy; each one autonomous and self-governing; > each with a seat in a world government, without the intervening power > of larger states or countries. We adapt Alexander's vision to a digital republic as the pattern that _authority is proportional to property_. This pattern already has a name in the cryptocurrency community, which calls it _proof of stake_. Urbit's launch hierarchy is a sort of built-in proof of stake. Another governance pattern, a political cliche but no less valuable for it, is the system of checks and balances. The address space of Urbit is covered by three independent tiers, galaxies, stars and planets; which should govern? The obvious design is a three-chambered legislature balanced against itself. For more on Urbit's governance design, see our [beliefs and principles](/posts/principles/) and our [interim constitution](/posts/constitution/). ]]> https://urbit.org/updates/2016-5-16-roadmap https://urbit.org/updates/2016-5-16-roadmap Sun, 15 May 2016 00:00:00 GMT https://urbit.org/blog/an-urbit-overview https://urbit.org/blog/an-urbit-overview Tue, 10 May 2016 00:00:00 GMT https://urbit.org/blog/beliefs-and-principles https://urbit.org/blog/beliefs-and-principles Tue, 10 May 2016 00:00:00 GMT https://urbit.org/blog/what-is-urbit-for https://urbit.org/blog/what-is-urbit-for Tue, 10 May 2016 00:00:00 GMT https://urbit.org/events/2017-08-11-after-hours-bootstrapping-urbit-from-the-ethereum-blockchain https://urbit.org/events/2017-08-11-after-hours-bootstrapping-urbit-from-the-ethereum-blockchain Invalid Date https://urbit.org/events/2017-09-22-after-hours-fun-with-arvo-the-urbit-os https://urbit.org/events/2017-09-22-after-hours-fun-with-arvo-the-urbit-os Invalid Date https://urbit.org/events/2017-10-26-after-hours-designing-robust-human-systems https://urbit.org/events/2017-10-26-after-hours-designing-robust-human-systems Invalid Date https://urbit.org/events/2017-11-16-after-hours-visiting-contributors https://urbit.org/events/2017-11-16-after-hours-visiting-contributors Invalid Date https://urbit.org/events/2018-01-26-urbit-irl-20181 https://urbit.org/events/2018-01-26-urbit-irl-20181 Invalid Date https://urbit.org/events/2018-02-23-urbit-irl-20182 https://urbit.org/events/2018-02-23-urbit-irl-20182 Invalid Date https://urbit.org/events/2018-03-11-20183-hoon-school https://urbit.org/events/2018-03-11-20183-hoon-school Invalid Date https://urbit.org/events/2018-03-23-urbit-irl-20183 https://urbit.org/events/2018-03-23-urbit-irl-20183 Invalid Date https://urbit.org/events/2018-04-08-20184-hoon-school https://urbit.org/events/2018-04-08-20184-hoon-school Invalid Date https://urbit.org/events/2018-05-25-urbit-irl-20185 https://urbit.org/events/2018-05-25-urbit-irl-20185 Invalid Date https://urbit.org/events/2018-08-03-after-hours-decentralized-web-summit https://urbit.org/events/2018-08-03-after-hours-decentralized-web-summit Invalid Date https://urbit.org/events/2018-08-17-urbit-irl-20188 https://urbit.org/events/2018-08-17-urbit-irl-20188 Invalid Date https://urbit.org/events/2018-09-21-urbit-irl-20189 https://urbit.org/events/2018-09-21-urbit-irl-20189 Invalid Date https://urbit.org/events/2018-10-10-urbit-zcash-blockchain-week https://urbit.org/events/2018-10-10-urbit-zcash-blockchain-week Invalid Date https://urbit.org/events/2018-12-20-after-hours-its-just-a-party-man https://urbit.org/events/2018-12-20-after-hours-its-just-a-party-man Invalid Date https://urbit.org/events/2019-03-01-after-hours-ask-galen-anything https://urbit.org/events/2019-03-01-after-hours-ask-galen-anything Invalid Date https://urbit.org/primer ]]> https://urbit.org/events/2019-03-08-urbit-irl-austin-tx https://urbit.org/events/2019-03-08-urbit-irl-austin-tx Invalid Date https://urbit.org/events/2019-04-19-urbit-irl-20194 https://urbit.org/events/2019-04-19-urbit-irl-20194 Invalid Date https://urbit.org/events/2019-05-17-urbit-irl-20195 https://urbit.org/events/2019-05-17-urbit-irl-20195 Invalid Date https://urbit.org/events/2019-06-21-urbit-irl-20196 https://urbit.org/events/2019-06-21-urbit-irl-20196 Invalid Date https://urbit.org/events/2019-07-19-urbit-irl-20197 https://urbit.org/events/2019-07-19-urbit-irl-20197 Invalid Date https://urbit.org/events/2019-08-29-urbit-austin-20198 https://urbit.org/events/2019-08-29-urbit-austin-20198 Invalid Date https://urbit.org/events/2019-10-04-urbit-irl-201910 https://urbit.org/events/2019-10-04-urbit-irl-201910 Invalid Date https://urbit.org/events/2019-11-15-urbit-irl-201911 https://urbit.org/events/2019-11-15-urbit-irl-201911 Invalid Date https://urbit.org/events/2019-12-13-urbit-irl-201912 https://urbit.org/events/2019-12-13-urbit-irl-201912 Invalid Date https://urbit.org/events/2020-01-24-urbit-irl-20201 https://urbit.org/events/2020-01-24-urbit-irl-20201 Invalid Date https://urbit.org/events/2020-02-28-urbit-irl-20202 https://urbit.org/events/2020-02-28-urbit-irl-20202 Invalid Date https://urbit.org/events/2020-04-03-urbit-os1-launch-party https://urbit.org/events/2020-04-03-urbit-os1-launch-party Invalid Date https://urbit.org/blog/ Urbit Twitter: https://twitter.com/urbit Urbit Github: https://github.com/urbit Still confused about what Urbit is? https://urbit.org/understanding-urbit/ ]]> https://urbit.org/events/2020-10-09-urbit-hosting-launch-event https://urbit.org/events/2020-10-09-urbit-hosting-launch-event Invalid Date https://tlon.io/ --- Got a group of friends who want to use Urbit together? We periodically give away blocks of planets and help people get set up. Apply for a community grant:
https://urbit.org/community/community-grants/ Interested developing on Urbit? Hoon School is a community-run program to teach you the basics of the language and system. Sign up here:
https://hooniversity.org Mostly, you can find us on Urbit in the Urbit Community group. But you can also follow along here:
https://urbit.org/blog/
https://twitter.com/urbit
https://github.com/urbit And if you're still confused about what Urbit is, try these:
https://urbit.org/understanding-urbit/ ]]> https://urbit.org/events/2020-11-05-developer-call-graph-store https://urbit.org/events/2020-11-05-developer-call-graph-store Invalid Date https://urbit.org/events/2020-11-19-developer-call-bittorrent-tracking-on-urbit https://urbit.org/events/2020-11-19-developer-call-bittorrent-tracking-on-urbit Invalid Date https://urbit.org/events/2020-12-10-developer-call-injecting-external-services-into-urbit https://urbit.org/events/2020-12-10-developer-call-injecting-external-services-into-urbit Invalid Date https://urbit.org/events/2021-02-04-developer-call-architectural-idea-maze https://urbit.org/events/2021-02-04-developer-call-architectural-idea-maze Invalid Date https://urbit.org/events/2021-03-04-developer-call-scaling-azimuth https://urbit.org/events/2021-03-04-developer-call-scaling-azimuth Invalid Date https://urbit.org/events/2021-03-18-developer-call-pressure-and-protolith https://urbit.org/events/2021-03-18-developer-call-pressure-and-protolith Invalid Date https://urbit.org/events/2021-04-01-developer-call-javascript-apps-on-urbit https://urbit.org/events/2021-04-01-developer-call-javascript-apps-on-urbit Invalid Date https://urbit.org/events/2021-04-15-developer-call-software-distribution https://urbit.org/events/2021-04-15-developer-call-software-distribution Invalid Date https://urbit.org/events/2021-04-27-bitcoin-launch https://urbit.org/events/2021-04-27-bitcoin-launch Invalid Date https://urbit.org/events/2021-04-29-developer-call-real-time-communications-on-urbit https://urbit.org/events/2021-04-29-developer-call-real-time-communications-on-urbit Invalid Date https://urbit.org/events/2021-05-13-developer-call-versioned-networking https://urbit.org/events/2021-05-13-developer-call-versioned-networking Invalid Date https://urbit.org/events/2021-05-27-town-hall https://urbit.org/events/2021-05-27-town-hall Invalid Date https://urbit.org/events/2021-07-01-developer-call-runtime-future https://urbit.org/events/2021-07-01-developer-call-runtime-future Invalid Date https://urbit.org/events/2021-07-15-developer-call-yakshaving https://urbit.org/events/2021-07-15-developer-call-yakshaving Invalid Date https://urbit.org/events/2021-09-02-21q3-dc-meetup https://urbit.org/events/2021-09-02-21q3-dc-meetup Invalid Date https://urbit.org/events/2021-09-16-developer-call-azimuth-scaled https://urbit.org/events/2021-09-16-developer-call-azimuth-scaled Invalid Date https://urbit.org/events/2021-09-24-urbit-outside https://urbit.org/events/2021-09-24-urbit-outside Invalid Date https://urbit.org/events/2021-10-15-assembly https://urbit.org/events/2021-10-15-assembly Invalid Date https://urbit.org/events/2021-10-16-assembly-keynote https://urbit.org/events/2021-10-16-assembly-keynote Invalid Date https://urbit.org/events/2021-10-16-assembly-sw-dist-qa https://urbit.org/events/2021-10-16-assembly-sw-dist-qa Invalid Date https://urbit.org/events/20211007-developer-call-visor https://urbit.org/events/20211007-developer-call-visor Invalid Date https://urbit.org/events/20211028-developer-call-hosting https://urbit.org/events/20211028-developer-call-hosting Invalid Date