arch and remove shoe

ARCH.md

@@ -3,13 +3,13 @@
 ## Intro
 This architecture is, by Urbit standards, awkward. The awkwardness arises mainly from the asymmetry of full nodes: only a few nodes are providers/full nodes, and they have to keep remote clients updated as to the state of the blockchain. The system also requires providers to run a node side-by-side with their Urbit, although this can mostly be abstracted away as HTTP calls out.
 
-My goal in designing this was to isolate the architecture's awkwardness as much as possible to specific chokepoints, and to keep the non-provider portions as clean, simple state machine primitives.
+My goal in designing this was to isolate the architecture's awkwardness as much as possible to specific chokepoints, and to keep the non-provider portions as clean state machine primitives.
 
-System summary:
+System parts:
 - `btc-wallet-store`: holds wallets and watches their addresses
   * tracks whether a wallet has been scanned
   * generates receiving addresses and change addresses
-  * can take address input from any agent on its own ship
+  * can take address state input from any agent on its own ship
 - `btc-wallet-hook`: requests BTC state from provider and forwards it
   * subscribes to wallet-store for any address requests.
   * pokes wallet-store with new address info
@@ -17,37 +17,61 @@ System summary:
 - helper BTC libraries for address and transaction generation.
 
 ## btc-wallet-store
-Intentionally very limited in function. It is a primitive for tracking wallet state, including available addresses an existing/watched addresses.
+Intentionally very limited in function. It's a primitive for tracking wallet state, including available addresses an existing/watched addresses.
 
-It can take address inputs from any local source. This allows the possibility of creating import programs for pre-existing wallet data, or large amounts of wallet data.
+Addresses are put in a watchlist if they have UTXOs *or* have been previously used. "Used" means either existing in transactions already, or having been generated by the store's `gen-address` gate, which generates and watches a new address, and increments the next "free" address.
 
-Outgoing data:
+You add a wallet to the store by adding that wallet's xpub. The store scans that xpub's change and non-change addresses in batches, by sending the address batches to its subscribers and taking pokes back with info for each address.  The scan is done when store sees `max-gap` consecutive unused addresses.
+
+Any source on the ship can poke the store with address info. This allows the possibility of creating import programs for pre-existing wallet data, or large amounts of wallet data. Currently, the only program that interacts with it is `btc-wallet-hook`.
 
 Incoming data:
+- requested address info
+- unsolicited address updates (checked against watch list)
+- requests to generate and watch new addresses
+
+Outgoing data:
+- requests for address info
+- newly generated/watched addresses
 
 ## btc-wallet-hook
-I don't like the name "hook" here, but can't think of anything better atm. It's closer to a data source/manager on top of the wallet, potentially just one of many.
+I don't like the name "hook" here, but can't think of anything better atm. It's closer to a non-wallet-state manager on top of the wallet-store; potentially just one of many.
+
+It interfaces with hte 
 
 Outgoing data:
 
 Incoming data:
 
 Error conditions:
+Disconnected provider: when it receives a message that this is the case, it stops sending outgoing address info requests until the provider says it's back up. Once we receive a connected message, all pending requests are retried.
 
 ## btc-provider
-
+Layers on top of both BTC and ElectRS *TODO explain why the latter*
-Outgoing data:
 
 Incoming data:
 
+Outgoing data:
+
 Error conditions:
 
+## Resource Usage
+
+### Provider
+- machine: requires hard drive and processing to run a BTC full node and ElectRS indexer.
+- network: sends out *all* addresses in each new block to *all* clients
+
+### Wallet
+- machine: processes all addresses for each block to see whether they are being watched.
+- network: receives all addresses for each block
 
 ## Needed Extensions
 - Invoice generator that asks for addresses on behalf of ships and tracks whether they've made payments. `wallet-hook` could probably be renamed to `wallet-manager` and extended for this purpose.
+- `btc-provider` should push out address state in each block
+- `btc-wallet-store` should watch the next ~20 addresses in each wallet account, so that it can detect BTC sent to the wallet if the wallet is also managed/generates addresses in an outside-Urbit program.
 
-
+## Possible Improvements/Changes
-## Possible Improvements
+- Do away with `btc-wallet-hook` altogether in its current form, and instead make `btc-provider` both a server (as it is now) and also a client. Pros: conceptually clean; less between-agent data. Cons: complicates the otherwise simple provider codebase.
 - Multiple Providers
-- Gossip network for both pulling and pushing address updates
+- Gossip network for both pulling and pushing address updates to lower network usage on the providers.
 

app/btc-wallet-hook.hoon

@@ -13,7 +13,7 @@
 ::    none
 ::
 /-  *btc, *btc-wallet-hook, bws=btc-wallet-store, bp=btc-provider
-/+  shoe, dbug, default-agent, bwsl=btc-wallet-store
+/+  dbug, default-agent, bwsl=btc-wallet-store
 |%
 +$  versioned-state
     $%  state-0
@@ -26,32 +26,18 @@
       pend=back
   ==
 ::
-+$  card  card:shoe
++$  card  card:agent:gall
-+$  command
-  $?  %add-xpub
-  ==
-::
 --
 =|  state-0
 =*  state  -
 %-  agent:dbug
 ^-  agent:gall
-%-  (agent:shoe command)
-^-  (shoe:shoe command)
 =<
 |_  =bowl:gall
 +*  this      .
-    des   ~(. (default:shoe this command) bowl)
     def   ~(. (default-agent this %|) bowl)
     hc    ~(. +> bowl)
 ::
-++  command-parser  command-parser:des
-++  tab-list  tab-list:des
-++  can-connect  can-connect:des
-++  on-command  on-command:des
-++  on-connect  on-connect:des
-++  on-disconnect  on-disconnect:des
-::
 ++  on-init
   ^-  (quip card _this)
   ~&  >  '%btc-wallet-hook initialized'
@@ -110,12 +96,6 @@
 ++  on-fail   on-fail:def
 --
 |_  =bowl:gall
-++  handle-command
-  |=  comm=command
-  ^-  (quip card _state)
-  ~&  >  comm
-  `state
-::
 ++  handle-action
   |=  act=action
   ^-  (quip card _state)