@@ -2,8 +2,8 @@
2title = "volt: Lightning on Urbit"
2title = "volt: Lightning on Urbit"
3date = 2021-04-02
3date = 2021-04-02
4[taxonomies]
4[taxonomies]
5−grant_type = ["bounties"]
5+grant_type = ["Bounty"]
6−grant_category = ["App Dev: Arvo"]
6+grant_category = ["App Dev", "Arvo"]
9description = "An Urbit-native Lightning client and integration with `lnd` on 'provider' ships."
9description = "An Urbit-native Lightning client and integration with `lnd` on 'provider' ships."
@@ -14,6 +14,7 @@ link = ""
16## Introduction
16## Introduction
17With the coming integration of the Bitcoin wallet, Urbit ships can now send money between each other and associate payments with other ships (and soon applications/access rights). This building block will let us feel out the possibilities of inter-ship payments, but it's a time-buying measure at most: as BTC's price increases, fees will also rise, making Bitcoin impractical for payments less than around $500.
18With the coming integration of the Bitcoin wallet, Urbit ships can now send money between each other and associate payments with other ships (and soon applications/access rights). This building block will let us feel out the possibilities of inter-ship payments, but it's a time-buying measure at most: as BTC's price increases, fees will also rise, making Bitcoin impractical for payments less than around $500.
19However, the Lightning Network, built on top of Bitcoin, is rapidly advancing in functionality. It presents a unique, non-custodial solution to the high fees problem, although usability is still an issue. Urbit is perfectly positioned to take advantage of Lightning's strong points while drastically improving its useability.
20However, the Lightning Network, built on top of Bitcoin, is rapidly advancing in functionality. It presents a unique, non-custodial solution to the high fees problem, although usability is still an issue. Urbit is perfectly positioned to take advantage of Lightning's strong points while drastically improving its useability.
@@ -21,12 +22,15 @@ However, the Lightning Network, built on top of Bitcoin, is rapidly advancing in
21Once Lightning is integrated into Urbit, and maintained over time, Urbit will be a complete solution for building applications and receiving money for them through native payment rails.
22Once Lightning is integrated into Urbit, and maintained over time, Urbit will be a complete solution for building applications and receiving money for them through native payment rails.
23## Lightning Primer
24## Lightning Primer
24−The implementer is not expected to understand Lightning prior, but will have to do significant work to get up to speed on the protocol. What follows is a quick overview of what that entails.
26+The implementer is not expected to understand Lightning prior, but will have to do significant work to get up to speed on the protocol. What follows is a quick overview of what that entails.
26−Lightning is a p2p network of nodes, which maintain "channels" between each other. Payments can then be routed from one peer to another by finding a path of channels with sufficient liquidity.
28+Lightning is a p2p network of nodes, which maintain "channels" between each other. Payments can then be routed from one peer to another by finding a path of channels with sufficient liquidity.
28### channels
30### channels
29A channel consists of:
32A channel consists of:
301. an on-chain Bitcoin transaction allocating funds to one side of the channel.
341. an on-chain Bitcoin transaction allocating funds to one side of the channel.
312. subsequent signed off-chain transactions re-allocating those initial funds, that can be posted to the blockchain in the event of unresponsiveness or malfeasance of the other party.
352. subsequent signed off-chain transactions re-allocating those initial funds, that can be posted to the blockchain in the event of unresponsiveness or malfeasance of the other party.
@@ -35,198 +39,242 @@ Peers open channels between each other in order to make payments in the network.
35Each channel maintains a set of commitment transactions over time, which are signed IOUs that can be enforced on-chain if necessary. These must be maintained and backed up in order to secure the channel.
39Each channel maintains a set of commitment transactions over time, which are signed IOUs that can be enforced on-chain if necessary. These must be maintained and backed up in order to secure the channel.
37### routing/payments
41### routing/payments
38Lightning payments follow a set sequence:
43Lightning payments follow a set sequence:
39−1. payee creates an invoice
45+1. payee creates an invoice
40− - optionally with route hints ("take this path to reach me, because maybe I'm not publicly listed")
47+- optionally with route hints ("take this path to reach me, because maybe I'm not publicly listed")
41− - contains the hash of a secret (the "preimage")
48+- contains the hash of a secret (the "preimage")
422. payee sends invoice to payer
502. payee sends invoice to payer
433. payer uses its channel graph to construct a route to payee that it thinks will work, using route hints if provided
513. payer uses its channel graph to construct a route to payee that it thinks will work, using route hints if provided
444. payer tries to send payment along that route. If it fails, payer updates its channel graph and retries
524. payer tries to send payment along that route. If it fails, payer updates its channel graph and retries
455. all hops along the route update their commitments to the new state
535. all hops along the route update their commitments to the new state
48When we say "sending payments" above, what we really mean is each hop constructing a commitment to pay the next hop if the next hop provides it the secret to the hash from the invoice in step 1.
57When we say "sending payments" above, what we really mean is each hop constructing a commitment to pay the next hop if the next hop provides it the secret to the hash from the invoice in step 1.
50−The contracts implementing this are called "hashed timelock contracts", or HTLCs. For more background, [this resource is recommended](https://github.com/lnbook/lnbook/blob/develop/routing.asciidoc). The basic concept is that once the final destination receives payment, it sends the secret to the prior hop, who uses it to get paid from the hop prior, and so on back to the original payer.
59+The contracts implementing this are called "hashed timelock contracts", or HTLCs. For more background, [this resource is recommended](https://github.com/lnbook/lnbook/blob/develop/routing.asciidoc). The basic concept is that once the final destination receives payment, it sends the secret to the prior hop, who uses it to get paid from the hop prior, and so on back to the original payer.
52### interceptors
61### interceptors
53Because Lightning uses onion routing, every hop along a payment path must use onion routing. In order to avoid implementing this for the last (inside Urbit) hop, we can use the [HTLC Interceptor](https://api.lightning.community/#htlcinterceptor) feature in the `lnd` RPC. This allows an `lnd` node to intercept an incoming payment request and inject its data into a new context, from which it can be handled.
63Because Lightning uses onion routing, every hop along a payment path must use onion routing. In order to avoid implementing this for the last (inside Urbit) hop, we can use the [HTLC Interceptor](https://api.lightning.community/#htlcinterceptor) feature in the `lnd` RPC. This allows an `lnd` node to intercept an incoming payment request and inject its data into a new context, from which it can be handled.
55### additional resources
65### additional resources
56−Additional background will be provided to the implementer by the Foundation as needed, and we can also arrange meetings with Lightning Labs for help as the project progresses. [Mastering the Lightning Network](https://github.com/lnbook/lnbook) is another good place to get up to speed.
67+Additional background will be provided to the implementer by the Foundation as needed, and we can also arrange meetings with Lightning Labs for help as the project progresses. [Mastering the Lightning Network](https://github.com/lnbook/lnbook) is another good place to get up to speed.
58## Lightning in Urbit
69## Lightning in Urbit
59−To speed and simplify implementation, we can use Urbit networking to provide full Lightning features without needing a full Lightning node (BOLT) implementation. In effect, we bridge "outside" Lightning to "inside Urbit", "interior" Lightning.
71+To speed and simplify implementation, we can use Urbit networking to provide full Lightning features without needing a full Lightning node (BOLT) implementation. In effect, we bridge "outside" Lightning to "inside Urbit", "interior" Lightning.
61Throughout the architecture, we refer to "interior" nodes: Urbit nodes running a Lightning implementation that do not network with the outside Lightning world.
73Throughout the architecture, we refer to "interior" nodes: Urbit nodes running a Lightning implementation that do not network with the outside Lightning world.
63### wins of this approach
75### wins of this approach
65−#### non-custodial *and* easy to use *and* powerful
77+#### non-custodial _and_ easy to use _and_ powerful
66−Current Lightning implementations force users to pick 2 of 3: non-custodial, easy, rich/powerful. Wallets like Breez are non-custodial and easy to use, but can only do payments. Sphinx chat is more powerful, but you are forced to choose between ease and custody--if you want control over your money, you need to configure your own Lightning node.
79+Current Lightning implementations force users to pick 2 of 3: non-custodial, easy, rich/powerful. Wallets like Breez are non-custodial and easy to use, but can only do payments. Sphinx chat is more powerful, but you are forced to choose between ease and custody--if you want control over your money, you need to configure your own Lightning node.
68Lightning on Urbit is able to achieve all 3 of these properties. The key is that we implement Lightning features inside an Urbit environment, which gives full access to Urbit's programmable ids and OS. We also can use Urbit's networking to link interior ships to providers running `lnd` and connecting to the outside Lightning network. As a result, `volt` will enable much richer Lightning experiences, across a full range of applications.
81Lightning on Urbit is able to achieve all 3 of these properties. The key is that we implement Lightning features inside an Urbit environment, which gives full access to Urbit's programmable ids and OS. We also can use Urbit's networking to link interior ships to providers running `lnd` and connecting to the outside Lightning network. As a result, `volt` will enable much richer Lightning experiences, across a full range of applications.
70#### leverages existing Urbit features
83#### leverages existing Urbit features
71−Lightning has some complex networking to implement, but because Urbit already has secure routing to named destinations in Ames, we can use that. This lets the work focus on implementing payment channels in Urbit.
85+Lightning has some complex networking to implement, but because Urbit already has secure routing to named destinations in Ames, we can use that. This lets the work focus on implementing payment channels in Urbit.
73#### clear upgrade paths
87#### clear upgrade paths
74The initial architecture does make one tradeoff: interior nodes are always one hop away from their providers, and so the provider knows how much money nodes are sending and receiving. There are two ways to handle that, one immediately possible, and the other doable after an upgrade:
89The initial architecture does make one tradeoff: interior nodes are always one hop away from their providers, and so the provider knows how much money nodes are sending and receiving. There are two ways to handle that, one immediately possible, and the other doable after an upgrade:
75−1. (ok, doable now) Nodes can upgrade to running their own providers. A busy node can configure its own `lnd` provider and connect to the Lightning network directly, while maintaining all of `volt`'s benefits in terms of integration of payments with Urbit. This is similar to the upgrade path that Sphinx chat provides, but still with non-custodial properties for users who don't take this path.
91+1. (ok, doable now) Nodes can upgrade to running their own providers. A busy node can configure its own `lnd` provider and connect to the Lightning network directly, while maintaining all of `volt`'s benefits in terms of integration of payments with Urbit. This is similar to the upgrade path that Sphinx chat provides, but still with non-custodial properties for users who don't take this path.
76−2. (better, doable in the future) We can implement Lightning's onion routing (not networking) in Urbit, allowing multi-hop paths within the Urbit network, with private payments *even for nodes not running lnd*.
92+2. (better, doable in the future) We can implement Lightning's onion routing (not networking) in Urbit, allowing multi-hop paths within the Urbit network, with private payments _even for nodes not running lnd_.
78### blockers
94### blockers
79Private key/hot wallet management inside Urbit. This is needed to sign commitment transactions. We can use dummy versions of this in development, and then figure out the best way to handle it when the rest is in place. We can't use cold wallet or external wallet signing, because commitment transactions need to be signed on demand, without manual user interaction.
96Private key/hot wallet management inside Urbit. This is needed to sign commitment transactions. We can use dummy versions of this in development, and then figure out the best way to handle it when the rest is in place. We can't use cold wallet or external wallet signing, because commitment transactions need to be signed on demand, without manual user interaction.
81## architecture
98## architecture
82`volt` consists of a “dumb” provider that bridges between `lnd` RPC calls and Urbit, and the “volt” agent that runs the Lightning protocol between interior Urbits.
100`volt` consists of a “dumb” provider that bridges between `lnd` RPC calls and Urbit, and the “volt” agent that runs the Lightning protocol between interior Urbits.
84### system diagram
102### system diagram
85
104
87### payment flows diagram
106### payment flows diagram
88
108
90### RFCs implemented
110### RFCs implemented
91We implement everything related to channels, HTLCs, and invoices. We leave out all the Lightning networking and routing elements.
112We implement everything related to channels, HTLCs, and invoices. We leave out all the Lightning networking and routing elements.
92[RFCs located here](https://github.com/lightningnetwork/lightning-rfc/blob/master/00-introduction.md)
113[RFCs located here](https://github.com/lightningnetwork/lightning-rfc/blob/master/00-introduction.md)
93−* RFC 02: channel opening, closing, HTLCs
115+- RFC 02: channel opening, closing, HTLCs
94−* RFC 03: commitment transactions
116+- RFC 03: commitment transactions
95−* RFC 05: on-chain transaction handling
117+- RFC 05: on-chain transaction handling
96−* RFC 11: invoices
118+- RFC 11: invoices
98### volt-provider lib
120### volt-provider lib
99Bridge between `lnd` or similar Lightning server running on the same machine as a Urbit. This could be implemented as a separate agent or as a library that `volt` uses. My recommendation would be to implement first as the latter, and split out to an agent if other agents could potentially be using it.
122Bridge between `lnd` or similar Lightning server running on the same machine as a Urbit. This could be implemented as a separate agent or as a library that `volt` uses. My recommendation would be to implement first as the latter, and split out to an agent if other agents could potentially be using it.
100−* Communicates with lnd over rpc
124+- Communicates with lnd over rpc
101−* passes HTLCs in and out
125+- passes HTLCs in and out
102−* accepts Hoon commands to open/close channels with outside nodes
126+- accepts Hoon commands to open/close channels with outside nodes
104#### heartbeat
128#### heartbeat
105−* checks for connectivity with lnd every N seconds
130+- checks for connectivity with lnd every N seconds
106−* updates local agent and subscribers with status
131+- updates local agent and subscribers with status
108#### incoming payments
133#### incoming payments
109−* intercepts the HTLC
135+- intercepts the HTLC
110−* injects the HTLC and metadata (next hop) into `volt`, where it's handled
136+- injects the HTLC and metadata (next hop) into `volt`, where it's handled
111−* receives a pre-image back and sends it back to the Lightning network (which RPC call?)
137+- receives a pre-image back and sends it back to the Lightning network (which RPC call?)
113#### outgoing payments
139#### outgoing payments
114−* receives outgoing-payment request from itself
141+- receives outgoing-payment request from itself
115−* finds an outgoing route, if one exists
142+- finds an outgoing route, if one exists
116−* constructs an HTLC with the interior node
143+- constructs an HTLC with the interior node
117−* sends the outgoing payment using RPC `SendPaymentV2` with params:
144+- sends the outgoing payment using RPC `SendPaymentV2` with params:
118 - invoice
145 - invoice
119 - route hints
146 - route hints
121−* when the preimage is received, inject back into `volt` to settle the HTLC
148+- when the preimage is received, inject back into `volt` to settle the HTLC
124Manager for interior Lightning in Urbit. `volt` handles everything related to channel management, HTLC creation and updating, and payment recording between ships.
152Manager for interior Lightning in Urbit. `volt` handles everything related to channel management, HTLC creation and updating, and payment recording between ships.
125−* connects to provider
154+- connects to provider
126−* provides Lightning pubkey to provider
155+- provides Lightning pubkey to provider
127−* maintains pubkey->Urbit ship mappings
156+- maintains pubkey->Urbit ship mappings
128−* maintains channel and commitment data
157+- maintains channel and commitment data
129−* maintains HTLCs
158+- maintains HTLCs
131#### channels
160#### channels
132−* open/close channels between provider nodes and others
162+- open/close channels between provider nodes and others
133−* monitor for fraud
163+- monitor for fraud
134−* serialize and encrypt commitments for backup
164+- serialize and encrypt commitments for backup
135−*note on backup*: commitments are critical for maintaining a channel's security, so we need functionality to back up that state to other Urbit ships whenever it changes, to avoid issues on breaches etc.
165+ _note on backup_: commitments are critical for maintaining a channel's security, so we need functionality to back up that state to other Urbit ships whenever it changes, to avoid issues on breaches etc.
137#### receive payment
167#### receive payment
138−* build invoice (with routing hints)
169+- build invoice (with routing hints)
139−* send invoice to payer (can be interior or exterior)
170+- send invoice to payer (can be interior or exterior)
140−* accept incoming HTLC from provider
171+- accept incoming HTLC from provider
141−* settle the HTLC if it's for us, fail otherwise
172+- settle the HTLC if it's for us, fail otherwise
142−* update channel commitments
173+- update channel commitments
143−* update payment history, optionally with ship metadata
174+- update payment history, optionally with ship metadata
145#### send payment
176#### send payment
146−* parse invoice from payee
178+- parse invoice from payee
147−* make sure that either
179+- make sure that either
148 - our node is our provider
180 - our node is our provider
149 - we have enough capacity to our provider
181 - we have enough capacity to our provider
150−* make a new HTLC commitment with provider (if remote provider)
182+- make a new HTLC commitment with provider (if remote provider)
151−* send payment request to provider
183+- send payment request to provider
152−* get result from provider and settle the HTLC
184+- get result from provider and settle the HTLC
154## user stories
186## user stories
155### provider
188### provider
156−As a provider, I can
190+As a provider, I can
157−* open/close channels with outside lightning nodes through the `lnd` RPC
192+- open/close channels with outside lightning nodes through the `lnd` RPC
158−* accept Urbit ships as clients and open/close channels with them
193+- accept Urbit ships as clients and open/close channels with them
159−* intercept incoming HTLC payments for interior Lightning on my ship to handle
194+- intercept incoming HTLC payments for interior Lightning on my ship to handle
160−* receive settled HTLCs from the interior and inject them back into lnd to settle my exterior
195+- receive settled HTLCs from the interior and inject them back into lnd to settle my exterior
162### Urbit ship
197### Urbit ship
163As an Urbit ship I can
199As an Urbit ship I can
164−* select a provider and connect to it
201+- select a provider and connect to it
165−* open/close channels with my provider
202+- open/close channels with my provider
166−* receive payments from Urbit users
203+- receive payments from Urbit users
167−* make payments to other Urbit ships
204+- make payments to other Urbit ships
168−* make and receive payments with outside Lightning nodes
205+- make and receive payments with outside Lightning nodes
169−* view my transaction history
206+- view my transaction history
171## weaknesses and future upgrades
208## weaknesses and future upgrades
172### weaknesses
210### weaknesses
173As designed, the provider will see incoming and outgoing transaction flow. The best solution for that is to run your own local Lightning node. Some of upgrades below would address this flaw.
212As designed, the provider will see incoming and outgoing transaction flow. The best solution for that is to run your own local Lightning node. Some of upgrades below would address this flaw.
175### upgrades
214### upgrades
176#### multi-provider support
216#### multi-provider support
177Allowing a ship to maintain channels with multiple providers would stop one provider from knowing all your transactions. This isn't super-complicated; it's just extra Hoon work that should probably be started soon after v1 is complete.
218Allowing a ship to maintain channels with multiple providers would stop one provider from knowing all your transactions. This isn't super-complicated; it's just extra Hoon work that should probably be started soon after v1 is complete.
179#### Urbit-native onion routing
220#### Urbit-native onion routing
180−It would be great for privacy if Urbit nodes could construct paths with multiple hops from the interior that could reach the exterior. The reverse is already doable (exterior payments reach multi-hop interior Urbit nodes).
222+It would be great for privacy if Urbit nodes could construct paths with multiple hops from the interior that could reach the exterior. The reverse is already doable (exterior payments reach multi-hop interior Urbit nodes).
182The multi-hops going out can’t be done until lnd implements ability to forward onion blobs to the wider network and inject the received preimage back in the RPC. This would also require us to be able to build onion routing (not terribly hard, but definitely adds scope).
224The multi-hops going out can’t be done until lnd implements ability to forward onion blobs to the wider network and inject the received preimage back in the RPC. This would also require us to be able to build onion routing (not terribly hard, but definitely adds scope).
184#### submarine swaps
226#### submarine swaps
185It is possible to re-balance channel liquidity (shifting more of the balance to one side of the channel or the other) by combining a Lightning payment with an on-chain transaction. This is outside the scope of this project, but could potentially be extremely valuable for merchants running businesses on Urbit.
228It is possible to re-balance channel liquidity (shifting more of the balance to one side of the channel or the other) by combining a Lightning payment with an on-chain transaction. This is outside the scope of this project, but could potentially be extremely valuable for merchants running businesses on Urbit.
187## Resources
230## Resources
188−* The libraries and full node interface provided by the `btc-wallet` project will be useful for parsing formats like `bech32` and for querying the state of the blockchain to monitor for cheating.
232+- The libraries and full node interface provided by the `btc-wallet` project will be useful for parsing formats like `bech32` and for querying the state of the blockchain to monitor for cheating.
189−* [Urbit Bitcoin Wallet](https://github.com/timlucmiptev/btc-agents)
233+- [Urbit Bitcoin Wallet](https://github.com/timlucmiptev/btc-agents)
190−* [Lightning RFC](https://github.com/lightningnetwork/lightning-rfc/blob/master/00-introduction.md)
234+- [Lightning RFC](https://github.com/lightningnetwork/lightning-rfc/blob/master/00-introduction.md)
191−* [lnd RPC API](https://api.lightning.community/)
235+- [lnd RPC API](https://api.lightning.community/)
193## contributor requirements
237## contributor requirements
194−* 4+ years of experience as a backend developer
239+- 4+ years of experience as a backend developer
195−* Full-time availablity is ideal
240+- Full-time availablity is ideal
196−* Some familiarity with Hoon a strong positive
241+- Some familiarity with Hoon a strong positive
197−* Some programming familiarity with Bitcoin a strong positive
242+- Some programming familiarity with Bitcoin a strong positive
198−* Attend regular (weekly or bi-weekly) checkin meetings with an urbit.org director to provide status updates
243+- Attend regular (weekly or bi-weekly) checkin meetings with an urbit.org director to provide status updates
200### working process
245### working process
201−* The contributor will work with the Foundation coordinator to on-board fully to Lightning and Bitcoin within Urbit.
202−* After initial onboarding, the contributor will do a weekly check-in call with the coordinator, more frequently if needed/desired.
203−* The contributor will fork [this Github repo](https://github.com/urbit/volt) and submit PRs to it to get feedback.
205−## Milestones
247+- The contributor will work with the Foundation coordinator to on-board fully to Lightning and Bitcoin within Urbit.
248+- After initial onboarding, the contributor will do a weekly check-in call with the coordinator, more frequently if needed/desired.
249+- The contributor will fork [this Github repo](https://github.com/urbit/volt) and submit PRs to it to get feedback.
208### transaction/invoice RFCs
253### transaction/invoice RFCs
210−* Implement RFCs 2, 3 and 11 from BOLT.
211−* Create the libs and types that further stages will depend on
257+- Implement RFCs 2, 3 and 11 from BOLT.
258+- Create the libs and types that further stages will depend on
214### on-chain transaction RFC
260### on-chain transaction RFC
216−* implement RFC 5, on-chain transaction handling
217−* test that channels can be opened and closed in the manner described there
264+- implement RFC 5, on-chain transaction handling
265+- test that channels can be opened and closed in the manner described there
220### provider libraries
267### provider libraries
222−* implement the `volt` provider that will talk to `lnd`
223−* create any configuration files needed to start `lnd` locally
271+- implement the `volt` provider that will talk to `lnd`
272+- create any configuration files needed to start `lnd` locally
226### `volt` agent and cleanup
274### `volt` agent and cleanup
228−* create the Gall agent to manage transactions
229−* connect it to the provider libs
230−* use the RFC libs to wire up all on-chain transactions
232−
278+- create the Gall agent to manage transactions
279+- connect it to the provider libs
280+- use the RFC libs to wire up all on-chain transactions