@@ -0,0 +1,69 @@
1++++
2+
3+title = "Seguro Prototype"
4+date = "2022-05-09"
5+
6+[taxonomies]
7+grant_type = ["Proposal"]
8+grant_category = ["Runtime"]
9+
10+[extra]
11+image = ""
12+description = ""
13+reward = "2 stars"
14+assignee = "~mastyr-bottec"
15+champion = "~master-morzod"
16+grant_id = "P0157"
17+completed = false
18+canceled = false
19+link = ""
20+
21++++
22+
23+# Seguro Specification and Prototype Proposal
24+
25+# Overview
26+Seguro improves Urbit's dependability by automatically replicating its event log across a set of machines in a cluster. The goal of this proposal is to specify and prototype such a system. This initial phase will be focused on writing a detailed technical specification, having it reviewed by Tlon engineering, accordingly revised, and ultimately approved for prototyping.
27+## Problem
28+The [Urbit runtime](https://github.com/urbit/urbit/tree/master/pkg/urbit/vere) was designed to host just a single Urbit instance running in a Unix process, with a single file volume attached for its event log and checkpoints. For individuals running only one or a handful of ships, this architecture is satisfactory. For providing Urbit to the world as a new, decentralized, peer-to-peer network of personal servers, the current implementation has inarguably succeeded.
29+However, for a quickly maturing platform which needs to scale to meet enormous demand, current Urbit technology is not suitable. Its low dependability is one element which continues to prevent the Urbit community from building truly resilient and scalable hosting services.
30+# Background
31+Urbit is simple, durable, yours. Seguro dials Urbit’s durability up to 11, making Urbit not only durable, but truly permanent. With Seguro’s event log replication, ship breaching will become unnecessary. Seguro is a major component in service of an Urbit that can scale the globe with dependability high enough to support critical workloads. Seguro “harbors” could be star-level services which provide ships a pre-configured place for their piers to be automatically backed up to and restored from, giving all pilots the benefits of Seguro without any additional operational overhead. In practice, user error would no longer be a reason to have to breach a ship.
32+
33+# Project Requirements
34+## Potential Architecture
35+
36+Seguro will utilize an off-the-shelf distributed database backend technology (like dqlite) to store and replicate a ship's event log across a number of replicas. To start, the Seguro prototype will be a separate repository and program, rather than an addition to vere. An analysis of existing off-the-shelf distributed database tools will be performed, published, and two of them will be selected for integration in the prototype.
37+## Cluster
38+A Seguro cluster will consist of a master and m replication slaves (user-specified). Urbit network UDP events will be received by an IP-level load balancer which will then replicate them to all the members of the cluster. Slaves will process these events and record them in their corresponding event logs but only the Seguro master will actually emit their corresponding side effects.
39+# Performance
40+## Configuration
41+Users will have the option to configure Seguro's degree of optimism, which would indicate the minimum number of healthy replicas which must process an event before emitting its side effect from the master. For example, if set to 0, the master will receive, process and emit side effects from an event without waiting for any slaves in the cluster to acknowledge their successful reception and processing of said event. If set to `m`, where `m` is the number of slaves in the cluster, the master will wait for all of the slaves to process the event before emitting its side effects. 0 is most performant, `m` is most durable. In other words, setting a value of 0 tells Seguro to be "optimistic" about event replications, and `m` tells Seguro to be the opposite.
42+## Event Log Batching
43+An event log batching system could be designed and implemented to improve the performance of Seguro by way of reducing per-event processing and replication overhead (if there is any).
44+## Snapshots & Event Log Truncation
45+Seguro's support of optimistic run-ahead as defined in the Configuration section depends on coupling segments of the log to binary versions (for error handling and crash recovery), which depends on "epochs" and/or event log truncation. There is a PR currently under review which implements this feature. This PR assumes existence of a local log that can be subdivided into epochs where each epoch is coupled to a particular snapshot. If the log is moved off of the module, and epochs/truncation are supported, there will need to be some way to make sure that the relevant snapshots are persisted with the same durability on slave machines.
46+# Previous Work
47+[Some effort](https://github.com/urbit/urbit/commit/cfeb35e37be63f96bb50fe1f60e2f59e35c07258) to support additional database technologies for event log storage was performed by Tlon in 2018. The work aimed at adding FoundationDB, RocksDB, LMDB and SQLite as storage options. Conceptually, adoption of distributed database systems for replicated/clustered event log storage was conceived and researched as far back as 2013.
48+# Open Questions
49+1. Should the ordering of (compute event, commit event, release effect) change? Why? How?
50+- Yes, but not as a part of this work. Impact should be small
51+2. What is the nature of database integration? Are we making a new kind of pipe where we control both sides and the remote talks to an arbitrary database, or does the database have a client SDK that gets pushed into the runtime?
52+- Client SDKs will be integrated into the runtime
53+3. What happens in case of failed nodes (master or slaves)?
54+- Masters: automatic leader replacement (promotion of slave to master) until no remaining healthy nodes exist
55+- Slaves: user warning at best
56+# About Me
57+I am Matthew LeVan (~mastyr-bottec), CTO of Wexpert Systems. I have a background in infrastructure engineering, worked extensively with cluster computing in my previous roles, and have been an Urbit enthusiast and mild contributor since 2016. At Wexpert, we are focused on building the core infrastructure to support scalable Urbit hosting on our own private servers.
58+# Milestones
59+Phase 1 - Implement a working external Seguro prototype
60+Deliverables:
61+- Brief analysis of database tools and selections with justification
62+- Writes/reads in and out of the Seguro cluster
63+- Compatibility with two databases
64+- Observed performance results
65+- Events recorded with continuous integer keys and arbitrarily sized values
66+ - Depending on database, either metadata records or compound keys must be used to support event values which overflow the database’s maximum size limit
67+
68+Payment: 2 stars
69+Expected Completion: June 2022