db20d15 · urbit.org · git.iko.soy

4template = "doc.html"

5+++

6

7−Urbit relies heavily on subscriptions throughout the stack. We strongly

7+Urbit relies heavily on subscriptions throughout the stack. We strongly

8prefer reactive data flow compared to querying or polling, which is why

9for example our equivalent to git natively supports subscriptions to its

10data.

12We have many application-level styles of subscription, but we only have

13three true subscription types in Arvo, named for the vane they're

14implemented in: Clay subscriptions (filesystem), Gall subscriptions

15−(applications), and Jael subscriptions (Urbit ID info). They're at

15+(applications), and Jael subscriptions (Urbit ID info). They're at

16three different points in the idea maze of subscriptions, and we'll

17encounter each of them here.

18

28### Exactly-once updates

29

30Some subscriptions need the subscriber to receive each update separately

31−instead of collapsing them into a single diff. Sometimes a subscription

31+instead of collapsing them into a single diff. Sometimes a subscription

32is really a sequence of commands, in which case there's no meaningful

33−way to collapse them. Similarly, software updates may be expected to be

33+way to collapse them. Similarly, software updates may be expected to be

34applied in order.

35

36### Syncing data

38Other times, it's more natural to think of a subscription as

39communicating a data structure, where the only goal is to keep the data

40structure on the subscriber's side up-to-date with what's on the

41−publisher's side. In this case, it's perfectly fine to collapse several

41+publisher's side. In this case, it's perfectly fine to collapse several

42updates into one large update, or even to resend the entire data

43−structure. Partial updates exist only as an optimization.

43+structure. Partial updates exist only as an optimization.

44

45For this sort of subscription, we commonly use terms like "diff" instead

46of "message" or "update", and we say that we "apply a diff" instead of

47−"process an update". We also often identify the subscription with the

47+"process an update". We also often identify the subscription with the

48data structure itself.

49

50## Memory pressure

51

52Naively implemented, subscriptions pose a memory leak risk if the

53subscriber can't receive updates as fast as the publisher wants to send

54−them. This commonly happens when the subscriber goes offline for an

54+them. This commonly happens when the subscriber goes offline for an

55−extended period of time (possibly forever; especially for comets). It's

55+extended period of time (possibly forever; especially for comets). It's

56sufficient to bound the amount of memory dedicated to any particular

57subscriber.

58

59In a synchronous system, you don't have to worry about what happens if

60−one side or the other ceases to be available. All of our subscriptions,

60+one side or the other ceases to be available. All of our subscriptions,

61though, can happen between ships.

62

63### Small data

64

65For particularly low-volume subscriptions, it may be acceptable to

66−ignore the problem. Jael uses this approach, since the only time it's

66+ignore the problem. Jael uses this approach, since the only time it's

67used across ships is to update moon keys, which is very low-volume and

68−less than 1KB per update. If these subscriptions are used for more,

68+less than 1KB per update. If these subscriptions are used for more,

69this strategy will have to be revisited.

70

71### Buffers

72

73−A common solution to this problem is the venerable buffer. If more than

73+A common solution to this problem is the venerable buffer. If more than

74−N updates need to be sent, start dropping updates. There are several

74+N updates need to be sent, start dropping updates. There are several

75options for what to do when the buffer is full and the publisher wishes

76to send another message:

77

78−- Drop the oldest message. Particularly suitable for systems where new

78+- Drop the oldest message. Particularly suitable for systems where new

79−messages obsolete old ones. For example, if you just need the latest

79+ messages obsolete old ones. For example, if you just need the latest

80−ticker price for an asset, you could use a buffer size of one and

80+ ticker price for an asset, you could use a buffer size of one and

81−replaced it any time you have a new price.

81+ replaced it any time you have a new price.

82

83−- Don't add the new message to the buffer. Semantically this is usually

83+- Don't add the new message to the buffer. Semantically this is usually

84−less helpful, but mechanically it can be simpler and reduce memory

84+ less helpful, but mechanically it can be simpler and reduce memory

85−churn, since the buffer stops changing and new messages are simply

85+ churn, since the buffer stops changing and new messages are simply

86−ignored.

86+ ignored.

87

88−- Drop the newest message and close the subscription. Unlike the

88+- Drop the newest message and close the subscription. Unlike the

89−previous two, this is compatible with the principle of exactly-once

89+ previous two, this is compatible with the principle of exactly-once

90−messaging. Specifically, it preserves the guarantee that you won't

90+ messaging. Specifically, it preserves the guarantee that you won't

91−receive two non-consecutive subscription updates, at the cost of making

91+ receive two non-consecutive subscription updates, at the cost of making

92−you resubscribe if you fall behind.

92+ you resubscribe if you fall behind.

93

94−Gall subscriptions use the last of these. The buffer is bounded by a

94+Gall subscriptions use the last of these. The buffer is bounded by a

95simple heuristic, and when a subscription exceeds that buffer, it is

96closed and the app is expected to resubscribe when they come back

97−online. Note that even though the subscription is closed, messages in

97+online. Note that even though the subscription is closed, messages in

98flight cannot be removed until they've been acknowledged since Ames

99guarantees exactly-once messaging, so those last few messages will be

100stuck in memory until the subscriber comes back online.

101

102Buffers are commonly bounded by size, but this puts constraints on the

103−types of messages may be sent. They may also be bounded in time: if the

103+types of messages may be sent. They may also be bounded in time: if the

104subscriber stops taking messages for at least X time, the buffer is

105−full. Gall taks a hybrid approach; currently the heuristic is that the

105+full. Gall taks a hybrid approach; currently the heuristic is that the

106buffer counts as full if it has at least five messages and it has been

10730 seconds since a message was acknowledged.

108

109### One at a time

110

111−Another scheme is to avoid multi-message subscriptions. Instead of

111+Another scheme is to avoid multi-message subscriptions. Instead of

112allowing the subscriber to subscribe to all future updates on a path,

113−allow them to "subscribe" to the *next* update only. When they get that

113+allow them to "subscribe" to the _next_ update only. When they get that

114update, they subscribe to the new next update, so that they ultimately

115get every message.

116

122queue up several messages to send to the subscriber.

123

124It also encourages processing each update in order rather than zooming

125−forward from time to time. It's possible (if a little awkward) to

125+forward from time to time. It's possible (if a little awkward) to

126simulate this with buffers though, and it's possible to allow zooming

127forward in the one-at-a-time scheme by having the requests be "most

128recent after update N" instead of "next after update N".

129

130This scheme is at its strongest, then, when you're already storing

131intermediate states or updates, or when you don't care about

132−intermediate states and always send the most recent one. Otherwise,

132+intermediate states and always send the most recent one. Otherwise,

133buffers are a convenient place to temporarily store updates that you

134−don't want to store internally. Of course, if it's semantically

134+don't want to store internally. Of course, if it's semantically

135important that you give each update separately, buffers won't save you

136from having to store the intermediate updates, since they may fill up.

137

138Another conceptual advantage is that it has no notion of "connectivity".

139−It operates forever in a single mode. By contrast, buffer-based

139+It operates forever in a single mode. By contrast, buffer-based

140solutions must keep track of whether the buffer is full and react to

141−that. This increases complexity in the system, and complexity is the

141+that. This increases complexity in the system, and complexity is the

142great evil.

143

144The biggest disadvantage of this system is that it takes much longer to

145−process many small updates. With buffers, the publisher could have a

145+process many small updates. With buffers, the publisher could have a

146dozen small updates in flight at the same time, while in this scheme

147they would only send one at a time, waiting for a response between each

148−one. Losing this pipelining can be quite significant for some

148+one. Losing this pipelining can be quite significant for some

149applications, but for low-volume subscriptions (certainly anything that

150updates less than once per minute) this will have no noticeable effect.

151

152−Clay uses this scheme. It's low-volume, it stores all previous states

152+Clay uses this scheme. It's low-volume, it stores all previous states

153anyway (since it's a revision-control system), and it's sometimes

154important to get updates one at a time (for example when downloading OTA

155−software updates). Ships that send OTA updates often have many ships

155+software updates). Ships that send OTA updates often have many ships

156subscribed to them, and the updates themselves may be large, so it's

157important they don't queue up when ships inevitably disappear.

158

159I tend to think most Gall subscriptions should follow this pattern as

160−well. Our most latency-sensitive subscriptions are chat messages, and

160+well. Our most latency-sensitive subscriptions are chat messages, and

161while I haven't tested it, I would guess that even those wouldn't be

162affected by this scheme.