@@ -0,0 +1,162 @@
2+title = "Subscriptions"
7+Urbit relies heavily on subscriptions throughout the stack. We strongly
8+prefer reactive data flow compared to querying or polling, which is why
9+for example our equivalent to git natively supports subscriptions to its
12+We have many application-level styles of subscription, but we only have
13+three true subscription types in Arvo, named for the vane they're
14+implemented in: Clay subscriptions (filesystem), Gall subscriptions
15+(applications), and Jael subscriptions (Urbit ID info). They're at
16+three different points in the idea maze of subscriptions, and we'll
17+encounter each of them here.
19+For our purposes, a subscription is a stream of events from a publisher
20+to a subscriber, where (1) the publisher has indicated interest in that
21+stream, (2) any update sent to one subscriber is sent to all subscribers
22+on the same stream, and (3) when the publisher has a new update ready,
23+they send it immediately to their subscribers instead of waiting for a
28+### Exactly-once updates
30+Some subscriptions need the subscriber to receive each update separately
31+instead of collapsing them into a single diff. Sometimes a subscription
32+is really a sequence of commands, in which case there's no meaningful
33+way to collapse them. Similarly, software updates may be expected to be
38+Other times, it's more natural to think of a subscription as
39+communicating a data structure, where the only goal is to keep the data
40+structure on the subscriber's side up-to-date with what's on the
41+publisher's side. In this case, it's perfectly fine to collapse several
42+updates into one large update, or even to resend the entire data
43+structure. Partial updates exist only as an optimization.
45+For this sort of subscription, we commonly use terms like "diff" instead
46+of "message" or "update", and we say that we "apply a diff" instead of
47+"process an update". We also often identify the subscription with the
48+data structure itself.
52+Naively implemented, subscriptions pose a memory leak risk if the
53+subscriber can't receive updates as fast as the publisher wants to send
54+them. This commonly happens when the subscriber goes offline for an
55+extended period of time (possibly forever; especially for comets). It's
56+sufficient to bound the amount of memory dedicated to any particular
59+In a synchronous system, you don't have to worry about what happens if
60+one side or the other ceases to be available. All of our subscriptions,
61+though, can happen between ships.
65+For particularly low-volume subscriptions, it may be acceptable to
66+ignore the problem. Jael uses this approach, since the only time it's
67+used across ships is to update moon keys, which is very low-volume and
68+less than 1KB per update. If these subscriptions are used for more,
69+this strategy will have to be revisited.
73+A common solution to this problem is the venerable buffer. If more than
74+N updates need to be sent, start dropping updates. There are several
75+options for what to do when the buffer is full and the publisher wishes
76+to send another message:
78+- Drop the oldest message. Particularly suitable for systems where new
79+messages obsolete old ones. For example, if you just need the latest
80+ticker price for an asset, you could use a buffer size of one and
81+replaced it any time you have a new price.
83+- Don't add the new message to the buffer. Semantically this is usually
84+less helpful, but mechanically it can be simpler and reduce memory
85+churn, since the buffer stops changing and new messages are simply
88+- Drop the newest message and close the subscription. Unlike the
89+previous two, this is compatible with the principle of exactly-once
90+messaging. Specifically, it preserves the guarantee that you won't
91+receive two non-consecutive subscription updates, at the cost of making
92+you resubscribe if you fall behind.
94+Gall subscriptions use the last of these. The buffer is bounded by a
95+simple heuristic, and when a subscription exceeds that buffer, it is
96+closed and the app is expected to resubscribe when they come back
97+online. Note that even though the subscription is closed, messages in
98+flight cannot be removed until they've been acknowledged since Ames
99+guarantees exactly-once messaging, so those last few messages will be
100+stuck in memory until the subscriber comes back online.
102+Buffers are commonly bounded by size, but this puts constraints on the
103+types of messages may be sent. They may also be bounded in time: if the
104+subscriber stops taking messages for at least X time, the buffer is
105+full. Gall taks a hybrid approach; currently the heuristic is that the
106+buffer counts as full if it has at least five messages and it has been
107+30 seconds since a message was acknowledged.
111+Another scheme is to avoid multi-message subscriptions. Instead of
112+allowing the subscriber to subscribe to all future updates on a path,
113+allow them to "subscribe" to the *next* update only. When they get that
114+update, they subscribe to the new next update, so that they ultimately
117+Commonly this is implemented where each update is given a version
118+number, and every time you hear update N you request update N+1 (even
119+though you won't get an answer for an indeterminate period of time).
121+This is near the theoretical ideal for memory usage, since you'll never
122+queue up several messages to send to the subscriber.
124+It also encourages processing each update in order rather than zooming
125+forward from time to time. It's possible (if a little awkward) to
126+simulate this with buffers though, and it's possible to allow zooming
127+forward in the one-at-a-time scheme by having the requests be "most
128+recent after update N" instead of "next after update N".
130+This scheme is at its strongest, then, when you're already storing
131+intermediate states or updates, or when you don't care about
132+intermediate states and always send the most recent one. Otherwise,
133+buffers are a convenient place to temporarily store updates that you
134+don't want to store internally. Of course, if it's semantically
135+important that you give each update separately, buffers won't save you
136+from having to store the intermediate updates, since they may fill up.
138+Another conceptual advantage is that it has no notion of "connectivity".
139+It operates forever in a single mode. By contrast, buffer-based
140+solutions must keep track of whether the buffer is full and react to
141+that. This increases complexity in the system, and complexity is the
144+The biggest disadvantage of this system is that it takes much longer to
145+process many small updates. With buffers, the publisher could have a
146+dozen small updates in flight at the same time, while in this scheme
147+they would only send one at a time, waiting for a response between each
148+one. Losing this pipelining can be quite significant for some
149+applications, but for low-volume subscriptions (certainly anything that
150+updates less than once per minute) this will have no noticeable effect.
152+Clay uses this scheme. It's low-volume, it stores all previous states
153+anyway (since it's a revision-control system), and it's sometimes
154+important to get updates one at a time (for example when downloading OTA
155+software updates). Ships that send OTA updates often have many ships
156+subscribed to them, and the updates themselves may be large, so it's
157+important they don't queue up when ships inevitably disappear.
159+I tend to think most Gall subscriptions should follow this pattern as
160+well. Our most latency-sensitive subscriptions are chat messages, and
161+while I haven't tested it, I would guess that even those wouldn't be
162+affected by this scheme.