Merge branch 'release/next-sys' of https://github.com/urbit/urbit into m/debug-dashboard

MAINTAINERS.md

@@ -1,5 +1,117 @@
 # Maintainers' Guide
 
+## Branch organization
+
+The essence of this branching scheme is that you create "release branches" of
+independently releasable units of work.  These can then be released by their
+maintainers when ready.
+
+### Master branch
+
+Master is what's released on the network.  Deployment instructions are in the
+next section, but tagged releases should always come from this branch.
+
+### Feature branches
+
+Anyone can create feature branches.  For those with commit access to
+urbit/urbit, you're welcome to create them in this repo; otherwise, fork the
+repo and create them there.
+
+Usually, new development should start from master, but if your work depends on
+work in another feature branch or release branch, start from there.
+
+If, after starting your work, you need changes that are in master, merge it into
+your branch.  If you need changes that are in a release branch or feature
+branch, merge it into your branch, but understand that your work now depends on
+that release branch, which means it won't be released until that one is
+released.
+
+### Release branches
+
+Release branches are code that is ready to release.  All release branch names
+should start with `release/`.
+
+All code must be reviewed before being pushed to a release branch.  Thus,
+feature branches should be PR'd against a release branch, not master.
+
+Create new release branches as needed.  You don't need a new one for every PR,
+since many changes are relatively small and can be merged together with little
+risk.  However, once you merge two branches, they're now coupled and will only
+be released together -- unless one of the underlying commits is separately put
+on a release branch.
+
+Here's a worked example.  The rule is to make however many branches are useful,
+and no more.  This example is not prescriptive, the developers making the
+changes may add, remove, or rename branches in this flow at will.
+
+Suppose you (plural, the dev community at large) complete some work in a
+userspace app, and you put it in `release/next-userspace`.  Separately, you make
+a small JS change.  If you PR it to `release/next-userspace`, then it will only
+be released at the same time as the app changes.  Maybe this is fine, or maybe
+you want this change to go out quickly, and the change in
+`release/next-userspace` is relatively risky, so you don't want to push it out
+on Friday afternoon.  In this case, put the change in another release branch,
+say `release/next-js`.  Now either can be released independently.
+
+Suppose you do further work that you want to PR to `release/next-userspace`, but
+it depends on your fixes in `release/next-js`.  Simply merge `release/next-js`
+into either your feature branch or `release/next-userspace` and PR your finished
+work to `release/next-userspace`.  Now there is a one-way coupling:
+`release/next-userspace` contains `release/next-js`, so releasing it will
+implicitly release `release/next-js`.  However, you can still release
+`release/next-js` independently.
+
+This scheme extends to other branches, like `release/next-kernel` or
+`release/os1.1` or `release/ford-fusion`.  Some branches may be long-lived and
+represent simply the "next" release of something, while others will have a
+definite lifetime that corresponds to development of a particular feature or
+numbered release.
+
+Since they are "done", release branches should be considered "public", in the
+sense that others may depend on them at will.  Thus, never rebase a release
+branch.
+
+When cutting a new release, you can filter branches with `git branch --list
+'release/*'` or by typing "release/" in the branch filter on Github.  This will
+give you the list of branches which have passed review and may be merged to
+master and released.  When choosing which branches to release, make sure you
+understand the risks of releasing them immediately.  If merging these produces
+nontrivial conflicts, consider asking the developers on those branches to merge
+between themselves.  In many cases a developer can do this directly, but if it's
+sufficiently nontrivial, this may be a reviewed PR of one release branch into
+another.
+
+### Non-OTAable release branches
+
+In some cases, work is completed which cannot be OTA'd as written.  For example,
+the code may lack state adapters, or it may not properly handle outstanding
+subscriptions.  It could also be code which is planned to be released only upon
+a breach (network-wide or rolling).
+
+In this case, the code may be PR'd to a `na-release/` branch.  All rules are the
+same as for release branches, except that the code does not need to apply
+cleanly to an existing ship.  If you later write state adapter or otherwise make
+it OTAable, then you may PR it to a release branch.
+
+### Other cases
+
+Outside contributors can generally target their PRs against master unless
+specifically instructed.  Maintainers should retarget those branches as
+appropriate.
+
+If a commit is not something that goes into a release (eg changes to README or
+CI), it may be committed straight to master.
+
+If a hotfix is urgent, it may be PR'd straight to master.  This should only be
+done if you reasonably expect that it will be released soon and before anything
+else is released.
+
+If a series of commits that you want to release is on a release branch, but you
+really don't want to release the whole branch, you must cherry-pick them onto
+another release branch.  Cherry-picking isn't ideal because those commits will
+be duplicated in the history, but it won't have any serious side effects.
+
+
 ## Hotfixes
 
 Here lies an informal guide for making hotfix releases and deploying them to
@@ -175,4 +287,3 @@ Post an announcement to urbit-dev.  The tag annotation, basically, is fine here
 -- I usually add the %base hash (for Urbit OS releases) and the release binary
 URLs (for Vere releases).  Check the urbit-dev archives for examples of these
 announcements.
-

bin/solid.pill

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ecfe53c9e00d8cba244c85eb0ec0ed3b4e8ac07737bb0c6a7aa57d6c52f52c85
+oid sha256:66c71e256923835bf9aee19f6bf7ea301f187230fe9cb4d7c695a98cb612019b
-size 15993675
+size 12884154

pkg/arvo/sys/vane/ames.hoon

@@ -1859,12 +1859,45 @@
           ==
         now
       ::
-      =/  =message-blob  (jam payload)
+      =/  =message-blob  (dedup-message (jam payload))
       =.  peer-core  (run-message-pump bone %memo message-blob)
       ::
       ?:  &(=(%boon valence) ?=(?(%dead %unborn) -.qos.peer-state))
         check-clog
       peer-core
+    ::  +dedup-message: replace with any existing copy of this message
+    ::
+    ++  dedup-message
+      |=  =message-blob
+      ^+  message-blob
+      ?:  (lte (met 13 message-blob) 1)
+        message-blob
+      =/  peers-l=(list [=ship =ship-state])  ~(tap by peers.ames-state)
+      |-  ^+  message-blob
+      =*  peer-loop  $
+      ?~  peers-l
+        message-blob
+      ?.  ?=(%known -.ship-state.i.peers-l)
+        peer-loop(peers-l t.peers-l)
+      =/  snd-l=(list [=bone =message-pump-state])
+        ~(tap by snd.ship-state.i.peers-l)
+      |-  ^+  message-blob
+      =*  bone-loop  $
+      ?~  snd-l
+        peer-loop(peers-l t.peers-l)
+      =/  blob-l=(list ^message-blob)
+        ~(tap to unsent-messages.message-pump-state.i.snd-l)
+      |-  ^+  message-blob
+      =*  blob-loop  $
+      ?^  blob-l
+        ?:  =(i.blob-l message-blob)
+          i.blob-l
+        blob-loop(blob-l t.blob-l)
+      ?~  unsent-fragments.message-pump-state.i.snd-l
+        bone-loop(snd-l t.snd-l)
+      ?:  =(message-blob fragment.i.unsent-fragments.message-pump-state.i.snd-l)
+        `@`fragment.i.unsent-fragments.message-pump-state.i.snd-l
+      bone-loop(snd-l t.snd-l)
     ::  +on-wake: handle timer expiration
     ::
     ++  on-wake
@@ -3017,20 +3050,24 @@
   (lte fragment-num.a fragment-num.b)
 ::  +split-message: split message into kilobyte-sized fragments
 ::
+::    We don't literally split it here since that would allocate many
+::    large atoms with no structural sharing.  Instead, each
+::    static-fragment has the entire message and a counter.  In
+::    +encrypt, we interpret this to get the actual fragment.
+::
 ++  split-message
   |=  [=message-num =message-blob]
   ^-  (list static-fragment)
   ::
-  =/  fragments=(list fragment)   (rip 13 message-blob)
+  =/  num-fragments=fragment-num  (met 13 message-blob)
-  =/  num-fragments=fragment-num  (lent fragments)
   =|  counter=@
   ::
   |-  ^-  (list static-fragment)
-  ?~  fragments  ~
+  ?:  (gte counter num-fragments)
+    ~
   ::
-  :-  [message-num num-fragments counter i.fragments]
+  :-  [message-num num-fragments counter `@`message-blob]
-  ::
+  $(counter +(counter))
-  $(fragments t.fragments, counter +(counter))
 ::  +assemble-fragments: concatenate fragments into a $message
 ::
 ++  assemble-fragments
@@ -3119,6 +3156,17 @@
   |=  [=symmetric-key plaintext=shut-packet]
   ^-  @
   ::
+  =.  meat.plaintext
+    ?.  ?&  ?=(%& -.meat.plaintext)
+            (gth (met 13 fragment.p.meat.plaintext) 1)
+        ==
+      meat.plaintext
+    %=    meat.plaintext
+        fragment.p
+      %^  end  13  1
+      %^  rsh  13  fragment-num.p.meat.plaintext
+      fragment.p.meat.plaintext
+    ==
   (en:crub:crypto symmetric-key (jam plaintext))
 ::  +decrypt: decrypt packet content to a $shut-packet or die
 ::

pkg/arvo/sys/vane/gall.hoon

@@ -55,7 +55,7 @@
 ++  state
   $:  :: state version
       ::
-      %4
+      %5
       :: agents by ship
       ::
       =agents
@@ -423,6 +423,12 @@
     =.  mo-core  (mo-untrack-ship ship)
     =.  mo-core  (mo-filter-queue ship)
     =/  agents=(list [name=term =running-agent])  ~(tap by running.agents.state)
+    =.  outstanding.agents.state
+      %-  malt
+      %+  skip  ~(tap by outstanding.agents.state)
+      |=  [[=wire duct] (qeu remote-request)]
+      =(/sys/way/(scot %p ship) (scag 3 wire))
+    ::
     |-  ^+  mo-core
     ?~  agents
       mo-core
@@ -1555,7 +1561,9 @@
         =.  ap-core
           =/  =tang
             ~[leaf+"subscribe wire not unique" >agent-name< >short-wire< >dock<]
-          %-  (slog >out=outgoing.subscribers.current-agent< tang)
+          =/  have
+            (~(got by outgoing.subscribers.current-agent) short-wire dock)
+          %-  (slog >out=have< tang)
           (ap-error %watch-not-unique tang)
         $(moves t.moves)
       =.  outgoing.subscribers.current-agent
@@ -1679,16 +1687,32 @@
   =?  all-state  ?=(%3 -.all-state)
     (state-3-to-4 all-state)
   ::
-  ?>  ?=(%4 -.all-state)
+  =?  all-state  ?=(%4 -.all-state)
+    (state-4-to-5 all-state)
+  ::
+  ?>  ?=(%5 -.all-state)
   gall-payload(state all-state)
   ::
   ::  +all-state: upgrade path
   ::
-  ++  all-state  $%(state-0 state-1 state-2 state-3 ^state)
+  ++  all-state  $%(state-0 state-1 state-2 state-3 state-4 ^state)
+  ::
+  ++  state-4-to-5
+    |=  =state-4
+    ^-  ^state
+    %=    state-4
+        -  %5
+        outstanding.agents  ~
+    ==
+  ::
+  ++  state-4
+    $:  %4
+        =agents
+    ==
   ::
   ++  state-3-to-4
     |=  =state-3
-    ^-  ^state
+    ^-  state-4
     %=    state-3
         -  %4
         outstanding.agents  ~

pkg/arvo/sys/zuse.hoon

@@ -6741,8 +6741,16 @@
       ;~  plug
         ;~(pfix (plus whit) name)
         ;~  pose
-          (ifix [;~(plug tis yel) yel] (star ;~(less yel escp)))
+          %+  ifix
-          (ifix [;~(plug tis say) say] (star ;~(less say escp)))
+            :_  yel
+            ;~(plug (ifix [. .]:(star whit) tis) yel)
+          (star ;~(less yel escp))
+        ::
+          %+  ifix
+            :_  say
+            ;~(plug (ifix [. .]:(star whit) tis) say)
+          (star ;~(less say escp))
+        ::
           (easy ~)
         ==
       ==