4`%clay` is our filesystem.
6The first part of this will be reference documentation for the data
7types used by our filesystem. In fact, as a general guide, we recommend
8reading and attempting to understand the data structures used in any
9Hoon code before you try to read the code itself. Although complete
10understanding of the data structures is impossible without seeing them
11used in the code, an 80% understanding greatly clarifies the code. As
12another general guide, when reading Hoon, it rarely pays off to
13understand every line of code when it appears. Try to get the gist of
14it, and then move on. The next time you come back to it, it'll likely
17After a description of the data models, we'll give an overview of the
18interface that vanes and applications can use to interact with the
21Finally, we'll dive into the code and the algorithms themselves. You
27As you're reading through this section, remember you can always come
28back to this when you run into these types later on. You're not going to
29remember everything the first time through, but it is worth reading, or
30at least skimming, this so that you get a rough idea of how our state is
33The types that are certainly worth reading are `++raft`, `++room`,
34`++dome`, `++ankh`, `++rung`, `++rang`, `++blob`, `++yaki`, and `++nori`
35(possibly in that order). All in all, though, this section isn't too
36long, so many readers may wish to quickly read through all of it. If you
37get bored, though, just skip to the next section. You can always come
40### `++raft`, formal state
43 $: fat=(map ship room) :: domestic
44 hoy=(map ship rung) :: foreign
48This is the state of our vane. Anything that must be remembered between
49calls to clay is stored in this state.
51`fat` is the set of domestic servers. This stores all the information
52that is specfic to a particular ship on this pier. The keys to this map
53are the ships on the current pier. all the information that is specific
54to a particular foreign ship. The keys to this map are all the ships
55whose filesystems we have attempted to access through clay.
57`ran` is the store of all commits and deltas, keyed by hash. The is
58where all the "real" data we know is stored; the rest is "just
61### `++room`, filesystem per domestic ship
63 ++ room :: fs per ship
64 $: hun=duct :: terminal duct
65 hez=(unit duct) :: sync duch
66 dos=(map desk dojo) :: native desk
69This is the representation of the filesystem of a ship on our pier.
71`hun` is the duct we use to send messages to dill to display
72notifications of filesystem changes. Only `%note` gifts should be
73produced along this duct. This is set by the `%init` kiss.
75`hez`, if present, is the duct we use to send sync messages to unix so
76that they end up in the pier unix directory. Only `%ergo` gifts should
77be producd along this duct. This is set by `%into` and `%invo` kisses.
79`dos` is a well-known operating system released in 1981. It is also the
80set of desks on this ship, mapped to their data.
82### `++desk`, filesystem branch
84 ++ desk ,@tas :: ship desk case spur
86This is the name of a branch of the filesystem. The default desks are
87"arvo", "main", and "try". More may be created by simply referencing
88them. Desks have independent histories and states, and they may be
89merged into each other.
91### `++dojo`, domestic desk state
93 ++ dojo ,[p=cult q=dome] :: domestic desk state
95This is the all the data that is specific to a particular desk on a
96domestic ship. `p` is the set of subscribers to this desk and `q` is the
99### `++cult`, subscriptions
101 ++ cult (map duct rave) :: subscriptions
103This is the set of subscriptions to a particular desk. The keys are the
104ducts from where the subscriptions requests came. The results will be
105produced along these ducts. The values are a description of the
106requested information.
108### `++rave`, general subscription request
110 ++ rave :: general request
111 $% [& p=mood] :: single request
112 [| p=moat] :: change range
115This represents a subscription request for a desk. The request can be
116for either a single item in the desk or else for a range of changes on
119### `++rove`, stored general subscription request
121 ++ rove (each mood moot) :: stored request
123When we store a request, we store subscriptions with a little extra
124information so that we can determine whether new versions actually
125affect the path we're subscribed to.
127### `++mood`, single subscription request
129 ++ mood ,[p=care q=case r=path] :: request in desk
131This represents a request for the state of the desk at a particular
132commit, specfied by `q`. `p` specifies what kind of information is
133desired, and `r` specifies the path we are requesting.
135### `++moat`, range subscription request
137 ++ moat ,[p=case q=case r=path] :: change range
139This represents a request for all changes between `p` and `q` on path
140`r`. You will be notified when a change is made to the node referenced
141by the path or to any of its children.
143### `++moot`, stored range subscription request
145 ++ moot ,[p=case q=case r=path s=(map path lobe)] ::
147This is just a `++moat` plus a map of paths to lobes. This map
148represents the data at the node referenced by the path at case `p`, if
149we've gotten to that case (else null). We only send a notification along
150the subscription if the data at a new revision is different than it was.
152### `++care`, clay submode
154 ++ care ?(%u %v %w %x %y %z) :: clay submode
156This specifies what type of information is requested in a subscription
159`%u` requests the `++rang` at the current moment. Because this
160information is not stored for any moment other than the present, we
161crash if the `++case` is not a `%da` for now.
163`%v` requests the `++dome` at the specified commit.
165`%w` requests the revsion number of the desk.
167`%x` requests the file at a specified path at the specified commit. If
168there is no node at that path or if the node has no contents (that is,
169if `q:ankh` is null), then this produces null.
171`%y` requests a `++arch` of the specfied commit at the specified path.
173`%z` requests the `++ankh` of the specified commit at the specfied path.
175### `++arch`, shallow filesystem node
177 ++ arch ,[p=@uvI q=(unit ,@uvI) r=(map ,@ta ,~)] :: fundamental node
179This is analogous to `++ankh` except that the we have neither our
180contents nor the ankhs of our children. The other fields are exactly the
181same, so `p` is a hash of the associated ankh, `u.q`, if it exists, is a
182hash of the contents of this node, and the keys of `r` are the names of
183our children. `r` is a map to null rather than a set so that the
184ordering of the map will be equivalent to that of `r:ankh`, allowing
187### `++case`, specifying a commit
189 ++ case :: ship desk case spur
190 $% [%da p=@da] :: date
191 [%tas p=@tas] :: label
192 [%ud p=@ud] :: number
195A commit can be referred to in three ways: `%da` refers to the commit
196that was at the head on date `p`, `%tas` refers to the commit labeled
197`p`, and `%ud` refers to the commit numbered `p`. Note that since these
198all can be reduced down to a `%ud`, only numbered commits may be
199referenced with a `++case`.
201### `++dome`, desk data
203 ++ dome :: project state
204 $: ang=agon :: pedigree
207 hit=(map ,@ud tako) :: changes by id
208 lab=(map ,@tas ,@ud) :: labels
211This is the data that is actually stored in a desk.
213`ang` is unused and should be removed.
215`ank` is the current state of the desk. Thus, it is the state of the
216filesystem at revison `let`. The head of a desk is always a numbered
219`let` is the number of the most recently numbered commit. This is also
220the total number of numbered commits.
222`hit` is a map of numerical ids to hashes of commits. These hashes are
223mapped into their associated commits in `hut:rang`. In general, the keys
224of this map are exactly the numbers from 1 to `let`, with no gaps. Of
225course, when there are no numbered commits, `let` is 0, so `hit` is
226null. Additionally, each of the commits is an ancestor of every commit
227numbered greater than this one. Thus, each is a descendant of every
228commit numbered less than this one. Since it is true that the date in
229each commit (`t:yaki`) is no earlier than that of each of its parents,
230the numbered commits are totally ordered in the same way by both
231pedigree and date. Of course, not every commit is numbered. If that
232sounds too complicated to you, don't worry about it. It basically
233behaves exactly as you would expect.
235`lab` is a map of textual labels to numbered commits. Note that labels
236can only be applied to numbered commits. Labels must be unique across a
239### `++ankh`, filesystem node
241 ++ ankh :: fs node (new)
242 $: p=cash :: recursive hash
243 q=(unit ,[p=cash q=*]) :: file
244 r=(map ,@ta ankh) :: folders
247This is a single node in the filesystem. This may be file or a directory
248or both. In earth filesystems, a node is a file xor a directory. On
249mars, we're inclusive, so a node is a file ior a directory.
251`p` is a recursive hash that depends on the contents of the this file or
252directory and on any children.
254`q` is the contents of this file, if any. `p.q` is a hash of the
255contents while `q.q` is the data itself.
257`r` is the set of children of this node. In the case of a pure file,
258this is empty. The keys are the names of the children and the values
259are, recursively, the nodes themselves.
261### `++cash`, ankh hash
263 ++ cash ,@uvH :: ankh hash
265This is a 128-bit hash of an ankh. These are mostly stored within ankhs
266themselves, and they are used to check for changes in possibly-deep
269### `++rung`, filesystem per neighbor ship
271 ++ rung $: rus=(map desk rede) :: neighbor desks
274This is the filesystem of a neighbor ship. The keys to this map are all
275the desks we know about on their ship.
277### `++rede`, desk state
279 ++ rede :: universal project
280 $: lim=@da :: complete to
281 qyx=cult :: subscribers
282 ref=(unit rind) :: outgoing requests
283 dom=dome :: revision state
286This is our knowledge of the state of a desk, either foreign or
289`lim` is the date of the last full update. We only respond to requests
290for stuff before this time.
292`qyx` is the list of subscribers to this desk. For domestic desks, this
293is simply `p:dojo`, all subscribers to the desk, while in foreign desks
294this is all the subscribers from our ship to the foreign desk.
296`ref` is the request manager for the desk. For domestic desks, this is
297null since we handle requests ourselves.
299`dom` is the actual data in the desk.
301### `++rind`, request manager
303 ++ rind :: request manager
304 $: nix=@ud :: request index
305 bom=(map ,@ud ,[p=duct q=rave]) :: outstanding
306 fod=(map duct ,@ud) :: current requests
307 haw=(map mood (unit)) :: simple cache
310This is the request manager for a foreign desk.
312`nix` is one more than the index of the most recent request. Thus, it is
313the next available request number.
315`bom` is the set of outstanding requests. The keys of this map are some
316subset of the numbers between 0 and one less than `nix`. The members of
317the map are exactly those requests that have not yet been fully
320`fod` is the same set as `bom`, but from a different perspective. In
321particular, the values of `fod` are the same as the values of `bom`, and
322the `p` out of the values of `bom` are the same as the keys of `fod`.
323Thus, we can map ducts to their associated request number and `++rave`,
324and we can map numbers to their associated duct and `++rave`.
326`haw` is a map from simple requests to their values. This acts as a
327cache for requests that have already been made. Thus, the second request
328for a particular `++mood` is nearly instantaneous.
330### `++rang`, data store
332 ++ rang $: hut=(map tako yaki) ::
333 lat=(map lobe blob) ::
336This is a set of data keyed by hash. Thus, this is where the "real" data
337is stored, but it is only meaningful if we know the hash of what we're
340`hut` is a map from hashes to commits. We often get the hashes from
341`hit:dome`, which keys them by logical id. Not every commit has an id.
343`lat` is a map from hashes to the actual data. We often get the hashes
344from a `++yaki`, a commit, which references this map to get the data.
345There is no `++blob` in any `++yaki`. They are only accessible through
348### `++tako`, commit reference
350 ++ tako ,@ :: yaki ref
352This is a hash of a `++yaki`, a commit. These are most notably used as
353the keys in `hut:rang`, where they are associated with the actual
354`++yaki`, and as the values in `hit:dome`, where sequential ids are
355associated with these.
359 ++ yaki ,[p=(list tako) q=(map path lobe) r=tako t=@da] :: commit
361This is a single commit.
363`p` is a list of the hashes of the parents of this commit. In most
364cases, this will be a single commit, but in a merge there may be more
365parents. In theory, there may be an arbitrary number of parents, but in
366practice merges have exactly two parents. This may change in the future.
367For commit 1, there is no parent.
369`q` is a map of the paths on a desk to the data at that location. If you
370understand what a `++lobe` and a `++blob` is, then the type signature
371here tells the whole story.
373`r` is the hash associated with this commit.
375`t` is the date at which this commit was made.
377### `++lobe`, data reference
379 ++ lobe ,@ :: blob ref
381This is a hash of a `++blob`. These are most notably used in `lat:rang`,
382where they are associated with the actual `++blob`, and as the values in
383`q:yaki`, where paths are associated with their data in a commit.
387 ++ blob $% [%delta p=lobe q=lobe r=udon] :: delta on q
388 [%direct p=lobe q=* r=umph] ::
389 [%indirect p=lobe q=* r=udon s=lobe] ::
392This is a node of data. In every case, `p` is the hash of the blob.
394`%delta` is the case where we define the data by a delta on other data.
395In practice, the other data is always the previous commit, but nothing
396depends on this. `q` is the hash of the parent blob, and `r` is the
399`%direct` is the case where we simply have the data directly. `q` is the
400data itself, and `r` is any preprocessing instructions. These almost
401always come from the creation of a file.
403`%indirect` is both of the preceding cases at once. `q` is the direct
404data, `r` is the delta, and `s` is the parent blob. It should always be
405the case that applying `r` to `s` gives the same data as `q` directly
406(with the prepreprocessor instructions in `p.r`). This exists purely for
407performance reasons. This is unused, at the moment, but in general these
408should be created when there are a long line of changes so that we do
409not have to traverse the delta chain back to the creation of the file.
411### `++udon`, abstract delta
413 ++ udon :: abstract delta
414 $: p=umph :: preprocessor
416 $% [%a p=* q=*] :: trivial replace
417 [%b p=udal] :: atomic indel
418 [%c p=(urge)] :: list indel
419 [%d p=upas q=upas] :: tree edit
423This is an abstract change to a file. This is a superset of what would
424normally be called diffs. Diffs usually refer to changes in lines of
425text while we have the ability to do more interesting deltas on
426arbitrary data structures.
428`p` is any preprocessor instructions.
430`%a` refers to the trival delta of a complete replace of old data with
433`%b` refers to changes in an opaque atom on the block level. This has
434very limited usefulness, and is not used at the moment.
436`%c` refers to changes in a list of data. This is often lines of text,
437which is your classic diff. We, however, will work on any list of data.
439`%d` refers to changes in a tree of data. This is general enough to
440describe changes to any hoon noun, but often more special-purpose delta
441should be created for different content types. This is not used at the
442moment, and may in fact be unimplemented.
444### `++urge`, list change
446 ++ urge |*(a=_,* (list (unce a))) :: list change
448This is a parametrized type for list changes. For example, `(urge ,@t)`
449is a list change for lines of text.
451### `++unce`, change part of a list.
453 ++ unce |* a=_,* :: change part
454 $% [%& p=@ud] :: skip[copy]
455 [%| p=(list a) q=(list a)] :: p -> q[chunk]
458This is a single change in a list of elements of type `a`. For example,
459`(unce ,@t)` is a single change in a lines of text.
461`%&` means the next `p` lines are unchanged.
463`%|` means the lines `p` have changed to `q`.
465### `++umph`, preprocessing information
467 ++ umph :: change filter
468 $| $? %a :: no filter
472 $% [%d p=@ud] :: blocklist
475This space intentionally left undocumented. This stuff will change once
476we get a well-typed clay.
478### `++upas`, tree change
480 ++ upas :: tree change (%d)
481 $& [p=upas q=upas] :: cell
482 $% [%0 p=axis] :: copy old
483 [%1 p=*] :: insert new
484 [%2 p=axis q=udon] :: mutate!
487This space intentionally left undocumented. This stuff is not known to
488work, and will likely change when we get a well-typed clay. Also, this
489is not a complicated type; it is not difficult to work out the meaning.
491### `++nori`, repository action
493 ++ nori :: repository action
494 $% [& q=soba] :: delta
498This describes a change that we are asking clay to make to the desk.
499There are two kinds of changes that may be made: we can modify files or
500we can apply a label to a commit.
502In the `|` case, we will simply label the current commit with the given
503label. In the `&` case, we will apply the given changes.
507 ++ soba ,[p=cart q=(list ,[p=path q=miso])] :: delta
509This describes a set of changes to make to a desk. The `cart` is simply
510a pair of the old hash and the new hash of the desk. The list is a list
511of changes keyed by the file they're changing. Thus, the paths are paths
512to files to be changed while `miso` is a description of the change
515### `++miso`, ankh delta
517 ++ miso :: ankh delta
518 $% [%del p=*] :: delete
520 [%mut p=udon] :: mutate
523There are three kinds of changes that may be made to a node in a desk.
524We can insert a file, in which case `p` is the contents of the new file.
525We can delete a file, in which case `p` is the contents of the old file.
526Finally, we can mutate that file, in which case the `udon` describes the
527changes we are applying to the file.
529### `++mizu`, merged state
531 ++ mizu ,[p=@u q=(map ,@ud tako) r=rang] :: new state
533This is the input to the `%merg` kiss, which allows us to perform a
534merge. The `p` is the number of the new head commit. The `q` is a map
535from numbers to commit hashes. This is all the new numbered commits that
536are to be inserted. The keys to this should always be the numbers from
537`let.dom` plus one to `p`, inclusive. The `r` is the maps of all the new
538commits and data. Since these are merged into the current state, no old
539commits or data need be here.
541### `++riff`, request/desist
543 ++ riff ,[p=desk q=(unit rave)] :: request/desist
545This represents a request for data about a particular desk. If `q`
546contains a `rave`, then this opens a subscription to the desk for that
547data. If `q` is null, then this tells clay to cancel the subscription
550### `++riot`, response
552 ++ riot (unit rant) :: response/complete
554A riot is a response to a subscription. If null, the subscription has
555been completed, and no more responses will be sent. Otherwise, the
556`rant` is the produced data.
558### `++rant`, response data
560 ++ rant :: namespace binding
561 $: p=[p=care q=case r=@tas] :: clade release book
566This is the data at a particular node in the filesystem. `p.p` specifies
567the type of data that was requested (and is produced). `q.p` gives the
568specific version reported (since a range of versions may be requested in
569a subscription). `r.p` is the desk. `q` is the path to the filesystem
570node. `r` is the data itself (in the format specified by `p.p`).
572### `++nako`, subscription response data
574 ++ nako $: gar=(map ,@ud tako) :: new ids
576 lar=(set yaki) :: new commits
577 bar=(set blob) :: new content
580This is the data that is produced by a request for a range of revisions
581of a desk. This allows us to easily keep track of a remote repository --
582all the new information we need is contained in the `nako`.
584`gar` is a map of the revisions in the range to the hash of the commit
585at that revision. These hashes can be used with `hut:rang` to find the
588`let` is either the last revision number in the range or the most recent
589revision number, whichever is smaller.
591`lar` is the set of new commits, and `bar` is the set of new content.
596As with all vanes, there are exactly two ways to interact with clay.
597`%clay` exports a namespace accessible through `.^`, which is described
598above under `++care`. The primary way of interacting with clay, though,
599is by sending kisses and receiving gifts.
601 ++ gift :: out result <-$
602 $% [%ergo p=@p q=@tas r=@ud] :: version update
603 [%note p=@tD q=tank] :: debug message
604 [%writ p=riot] :: response
606 ++ kiss :: in request ->$
607 $% [%info p=@p q=@tas r=nori] :: internal edit
608 [%ingo p=@p q=@tas r=nori] :: internal noun edit
609 [%init p=@p] :: report install
610 [%into p=@p q=@tas r=nori] :: external edit
611 [%invo p=@p q=@tas r=nori] :: external noun edit
612 [%merg p=@p q=@tas r=mizu] :: internal change
613 [%wart p=sock q=@tas r=path s=*] :: network request
614 [%warp p=sock q=riff] :: file request
617There are only a small number of possible kisses, so it behooves us to
618describe each in detail.
620 $% [%info p=@p q=@tas r=nori] :: internal edit
622 [%into p=@p q=@tas r=nori] :: external edit
624These two kisses are nearly identical. At a high level, they apply
625changes to the filesystem. Whenever we add, remove, or edit a file, one
626of these cards is sent. The `p` is the ship whose filesystem we're
627trying to change, the `q` is the desk we're changing, and the `r` is the
628request change. For the format of the requested change, see the
629documentation for `++nori` above.
631When a file is changed in the unix filesystem, vere will send a `%into`
632kiss. This tells clay that the duct over which the kiss was sent is the
633duct that unix is listening on for changes. From within Arvo, though, we
634should never send a `%into` kiss. The `%info` kiss is exactly identical
635except it does not reset the duct.
637 [%ingo p=@p q=@tas r=nori] :: internal noun edit
639 [%invo p=@p q=@tas r=nori] :: external noun edit
641These kisses are currently identical to `%info` and `%into`, though this
642will not always be the case. The intent is for these kisses to allow
643typed changes to clay so that we may store typed data. This is currently
646 [%init p=@p] :: report install
648Init is called when a ship is started on our pier. This simply creates a
649default `room` to go into our `raft`. Essentially, this initializes the
650filesystem for a ship.
652 [%merg p=@p q=@tas r=mizu] :: internal change
654This is called to perform a merge. This is most visibly called by
655:update to update the filesystem of the current ship to that of its
656sein. The `p` and `q` are as in `%info`, and the `r` is the description
657of the merge. See `++mizu` above.
660`XX [%wake ~] :: timer activate XX`
662XX This card is sent by unix at the time specified by `++doze`. This
663time is XX usually the closest time specified in a subscription request.
664When `%wake` is XX called, we update our subscribers if there have been
667 [%wart p=sock q=@tas r=path s=*] :: network request
669This is a request that has come across the network for a particular
670file. When another ship asks for a file from us, that request comes to
671us in the form of a `%wart` kiss. This is handled by trivially turning
674 [%warp p=sock q=riff] :: file request
676This is a request for information about a particular desk. This is, in
677its most general form, a subscription, though in many cases it is the
678trivial case of a subscription -- a read. See `++riff` for the format of
681Lifecycle of a Local Read
682-------------------------
684There are two real types of interaction with a filesystem: you can read,
685and you can write. We'll describe each process, detailing both the flow
686of control followed by the kernel and the algorithms involved. The
687simpler case is that of the read, so we'll begin with that.
689When a vane or an application wishes to read a file from the filesystem,
690it sends a `%warp` kiss, as described above. Of course, you may request
691a file on another ship and, being a global filesystem, clay will happily
692produce it for you. That code pathway will be described in another
693section; here, we will restrict ourselves to examining the case of a
694read from a ship on our own pier.
696The kiss can request either a single version of a file node or a range
697of versions of a desk. Here, we'll deal only with a request for a single
700As in all vanes, a kiss enters clay via a call to `++call`. Scanning
701through the arm, we quickly see where `%warp` is handled.
703 ?: =(p.p.q.hic q.p.q.hic)
704 =+ une=(un p.p.q.hic now ruf)
705 =+ wex=(di:une p.q.q.hic)
709 (eave:wex hen u.q.q.q.hic)
712 [-.woo abet:(pish:une p.q.q.hic +.woo ran.wao)]
714We're following the familar patern of producing a list of moves and an
715updated state. In this case, the state is `++raft`.
717We first check to see if the sending and receiving ships are the same.
718If they're not, then this is a request for data on another ship. We
719describe that process later. Here, we discuss only the case of a local
722At a high level, the call to `++un` sets up the core for the domestic
723ship that contains the files we're looking for. The call to `++di` sets
724up the core for the particular desk we're referring to.
726After this, we perform the actual request. If there is no rave in the
727riff, then that means we are cancelling a request, so we call
728`++ease:de`. Otherwise, we start a subscription with `++eave:de`. We
729call `++abet:de` to resolve our various types of output into actual
730moves. We produce the moves we found above and the `++un` core resolved
731with `++pish:un` (putting the modified desk in the room) and `++abet:un`
732(putting the modified room in the raft).
734Much of this is fairly straightforward, so we'll only describe `++ease`,
735`++eave`, and `++abet:de`. Feel free to look up the code to the other
736steps -- it should be easy to follow.
738Although it's called last, it's usually worth examining `++abet` first,
739since it defines in what ways we can cause side effects. Let's do that,
740and also a few of the lines at the beginning of `++de`.
742 =| yel=(list ,[p=duct q=gift])
743 =| byn=(list ,[p=duct q=riot])
744 =| vag=(list ,[p=duct q=gift])
745 =| say=(list ,[p=duct q=path r=ship s=[p=@ud q=riff]])
746 =| tag=(list ,[p=duct q=path c=note])
749 ^- [(list move) rede]
753 |=([a=duct b=gift] [hun %give b])
756 |=([a=duct b=riot] [a %give [%writ b]])
759 |=([a=duct b=gift] [a %give b])
762 |= [a=duct b=path c=ship d=[p=@ud q=riff]]
764 [%pass b %a %want [who c] [%q %re p.q.d (scot %ud p.d) ~] q.d]
767 |=([a=duct b=path c=note] [a %pass b c])
770This is very simple code. We see there are exactly five different kinds
771of side effects we can generate.
773In `yel` we put gifts that we wish to be sent along the `hun:room` duct
774to dill. See the documentation for `++room` above. This is how we
775display messages to the terminal.
777In `byn` we put riots that we wish returned to subscribers. Recall that
778a riot is a response to a subscription. These are returned to our
779subscribers in the form of a `%writ` gift.
781In `vag` we put gifts along with the ducts on which to send them. This
782allows us to produce arbitrary gifts, but in practice this is only used
783to produce `%ergo` gifts.
785In `say` we put messages we wish to pass to ames. These messages are
786used to request information from clay on other piers. We must provide
787not only the duct and the request (the riff), but also the return path,
788the other ship to talk to, and the sequence number of the request.
790In `tag` we put arbitrary notes we wish to pass to other vanes. For now,
791the only notes we pass here are `%wait` and `%rest` to the timer vane.
793Now that we know what kinds of side effects we may have, we can jump
794into the handling of requests.
796 ++ ease :: release request
800 =+ rov=(~(got by qyx) hen)
801 =. qyx (~(del by qyx) hen)
802 (mabe rov (cury best hen))
803 =. qyx (~(del by qyx) hen)
805 =+ nux=(~(get by fod.u.ref) hen)
808 say [[hen [(scot %ud u.nux) ~] for [u.nux syd ~]] say]
809 fod.u.ref (~(del by fod.u.ref) hen)
810 bom.u.ref (~(del by bom.u.ref) u.nux)
813This is called when we're cancelling a subscription. For domestic desks,
814`ref` is null, so we're going to cancel any timer we might have created.
815We first delete the duct from our map of requests, and then we call
816`++mabe` with `++best` to send a `%rest` kiss to the timer vane if we
817have started a timer. We'll describe `++best` and `++mabe` momentarily.
819Although we said we're not going to talk about foreign requests yet,
820it's easy to see that for foreign desks, we cancel any outstanding
821requests for this duct and send a message over ames to the other ship
822telling them to cancel the subscription.
825 |= [hen=duct tym=@da]
826 %_(+> tag :_(tag [hen /tyme %t %rest tym]))
828This simply pushes a `%rest` note onto `tag`, from where it will be
829passed back to arvo to be handled. This cancels the timer at the given
830duct (with the given time).
832 ++ mabe :: maybe fire function
833 |* [rov=rove fun=$+(@da _+>.^$)]
840 ?. ?=(%da -.q.p.rov) ~
846 ?.((lth now p.p.mot) ~ [~ p.p.mot])
848 ?.((lth now p.q.mot) [~ now] [~ p.q.mot])
851This decides whether the given request can only be satsified in the
852future. In that case, we call the given function with the time in the
853future when we expect to have an update to give to this request. This is
854called with `++best` to cancel timers and with `++bait` to start them.
856For single requests, we have a time if the request is for a particular
857time (which is assumed to be in the future). For ranges of requests, we
858check both the start and end cases to see if they are time cases. If so,
859we choose the earlier time.
861If any of those give us a time, then we call the given funciton with the
864The more interesting case is, of course, when we're not cancelling a
865subscription but starting one.
868 |= [hen=duct rav=rave]
872 ?: &(=(p.p.rav %u) !=(p.q.p.rav now))
873 ~& [%clay-fail p.q.p.rav %now now]
880 (blab hen p.rav u.u.ver)
882There are two types of subscriptions -- either we're requesting a single
883file or we're requesting a range of versions of a desk. We'll dicuss the
886First, we check that we're not requesting the `rang` from any time other
887than the present. Since we don't store that information for any other
888time, we can't produce it in a referentially transparent manner for any
889time other than the present.
891Then, we try to read the requested `mood` `p.rav`. If we can't access
892the request data right now, we call `++duce` to put the request in our
893queue to be satisfied when the information becomes available.
895This case occurs when we make a request for a case whose (1) date is
896after the current date, (2) number is after the current number, or (3)
897label is not yet used.
899 ++ duce :: produce request
900 |= [hen=duct rov=rove]
902 =. qyx (~(put by qyx) hen rov)
904 (mabe rov (cury bait hen))
905 |- ^+ +>+.$ :: XX why?
908 ?. ?=([%& %v *] rav) rav
909 [%| [%ud let.dom] `case`q.p.rav r.p.rav]
912 say [[hen [(scot %ud inx) ~] for [inx syd ~ vaw]] say]
913 nix.u.ref +(nix.u.ref)
914 bom.u.ref (~(put by bom.u.ref) inx [hen vaw])
915 fod.u.ref (~(put by fod.u.ref) hen inx)
918The code for `++duce` is nearly the exact inverse of `++ease`, which in
919the case of a domestic desk is very simple -- we simply put the duct and
920rave into `qyx` and possibly start a timer with `++mabe` and `++bait`.
921Recall that `ref` is null for domestic desks and that `++mabe` fires the
922given function with the time we need to be woken up at, if we need to be
923woken up at a particular time.
926 |= [hen=duct tym=@da]
927 %_(+> tag :_(tag [hen /tyme %t %wait tym]))
929This sets an alarm by sending a `%wait` card with the given time to the
932Back in `++eave`, if `++aver` returned `[~ ~]`, then we cancel the
933subscription. This occurs when we make (1) a `%x` request for a file
934that does not exist, (2) a `%w` request with a case that is not a
935number, or (3) a `%w` request with a nonempty path. The `++blub` is
936exactly what you would expect it to be.
940 %_(+> byn [[hen ~] byn])
942We notify the duct that we're cancelling their subscription since it
945Otherwise, we have received the desired information, so we send it on to
946the subscriber with `++blab`.
948 ++ blab :: ship result
949 |= [hen=duct mun=mood dat=*]
951 +>(byn [[hen ~ [p.mun q.mun syd] r.mun dat] byn])
953The most interesting arm called in `++eave` is, of course, `++aver`,
954where we actually try to read the data.
959 ?: &(=(p.mun %u) !=(p.q.mun now)) :: prevent bad things
960 ~& [%clay-fail p.q.mun %now now]
962 =+ ezy=?~(ref ~ (~(get by haw.u.ref) mun))
964 =+ nao=(~(case-to-aeon ze lim dom ran) q.mun)
965 ?~(nao ~ [~ (~(read-at-aeon ze lim dom ran) u.nao mun)])
967We check immediately that we're not requesting the `rang` for any time
968other than the present.
970If this is a foreign desk, then we check our cache for the specific
971request. If either this is a domestic desk or we don't have the request
972in our cache, then we have to actually go read the data from our dome.
974We need to do two things. First, we try to find the number of the commit
975specified by the given case, and then we try to get the data there.
977Here, we jump into `arvo/zuse.hoon`, which is where much of the
978algorithmic code is stored, as opposed to the clay interface, which is
979stored in `arvo/clay.hoon`. We examine `++case-to-aeon:ze`.
981 ++ case-to-aeon :: case-to-aeon:ze
982 |= lok=case :: act count through
988 ?: =(0 let) [~ 0] :: avoid underflow
997 %tas (~(get by lab) p.lok)
998 %ud ?:((gth p.lok let) ~ [~ p.lok])
1001We handle each type of `case` differently. The latter two types are
1004If we're requesting a revision by label, then we simply look up the
1005requested label in `lab` from the given dome. If it exists, that is our
1006aeon; else we produce null, indicating the requested revision does not
1009If we're requesting a revision by number, we check if we've yet reached
1010that number. If so, we produce the number; else we produce null.
1012If we're requesting a revision by date, we check first if the date is in
1013the future, returning null if so. Else we start from the most recent
1014revision and scan backwards until we find the first revision committed
1015before that date, and we produce that. If we requested a date before any
1016revisions were committed, we produce `0`.
1018The definitions of `++aeon-to-tako` and `++tako-to-yaki` are trivial.
1020 ++ aeon-to-tako ~(got by hit)
1022 ++ tako-to-yaki ~(got by hut) :: grab yaki
1024We simply look up the aeon or tako in their respective maps (`hit` and
1027Assuming we got a valid version number, `++aver` calls
1028`++read-at-aeon:ze`, which reads the requested data at the given
1031 ++ read-at-aeon :: read-at-aeon:ze
1032 |= [oan=aeon mun=mood] :: seek and read
1034 ?: &(?=(%w p.mun) !?=(%ud -.q.mun)) :: NB only for speed
1036 (read:(rewind oan) mun)
1038If we're requesting the revision number with a case other than by
1039number, then we go ahead and just produce the number we were given.
1040Otherwise, we call `++rewind` to rewind our state to the given revision,
1041and then we call `++read` to get the requested information.
1043 ++ rewind :: rewind:ze
1044 |= oan=aeon :: rewind to aeon
1047 ?: (gth oan let) !! :: don't have version
1048 +>(ank (checkout-ankh q:(tako-to-yaki (aeon-to-tako oan))), let oan)
1050If we're already at the requested version, we do nothing. If we're
1051requesting a version later than our head, we are unable to comply.
1053Otherwise, we get the hash of the commit at the number, and from that we
1054get the commit itself (the yaki), which has the map of path to lobe that
1055represents a version of the filesystem. We call `++checkout-ankh` to
1056checkout the commit, and we replace `ank` in our context with the
1059 ++ checkout-ankh :: checkout-ankh:ze
1060 |= hat=(map path lobe) :: checkout commit
1063 %+ roll (~(tap by hat) ~)
1064 |= [[pat=path bar=lobe] ank=ankh]
1068 =+ zar=(lobe-to-noun bar)
1069 ank(q [~ (sham zar) zar])
1070 =+ nak=(~(get by r.ank) i.pat)
1072 r %+ ~(put by r.ank) i.pat
1073 $(pat t.pat, ank (fall nak _ankh))
1076Twice we call `++cosh`, which hashes a commit, updating `p` in an
1077`ankh`. Let's jump into that algorithm before we describe
1080 ++ cosh :: locally rehash
1081 |= ank=ankh :: NB v/unix.c
1082 ank(p rehash:(zu ank))
1084We simply replace `p` in the hash with the `cash` we get from a call to
1087 ++ zu !: :: filesystem
1088 |= ank=ankh :: filesystem state
1089 =| myz=(list ,[p=path q=miso]) :: changes in reverse
1090 =| ram=path :: reverse path into
1092 ++ rehash :: local rehash
1094 %+ mix ?~(q.ank 0 p.u.q.ank)
1099 (shaf %dash (mix axe (shaf %dush (mix p.n.r.ank p.q.n.r.ank))))
1100 $(r.ank l.r.ank, axe (peg axe 2))
1101 $(r.ank r.r.ank, axe (peg axe 3))
1104`++zu` is a core we set up with a particular filesystem node to traverse
1105a checkout of the filesystem and access the actual data inside it. One
1106of the things we can do with it is to create a recursive hash of the
1109In `++rehash`, if this node is a file, then we xor the remainder of the
1110hash with the hash of the contents of the file. The remainder of the
1111hash is `0` if we have no children, else we descend into our children.
1112Basically, we do a half SHA-256 of the xor of the axis of this child and
1113the half SHA-256 of the xor of the name of the child and the hash of the
1114child. This is done for each child and all the results are xored
1117Now we return to our discussion of `++checkout-ankh`.
1119We fold over every path in this version of the filesystem and create a
1120great ankh out of them. First, we call `++lobe-to-noun` to get the raw
1121data referred to be each lobe.
1123 ++ lobe-to-noun :: grab blob
1128This converts a lobe into the raw data it refers to by first getting the
1129blob with `++lobe-to-blob` and converting that into data with
1132 ++ lobe-to-blob ~(got by lat) :: grab blob
1134This just grabs the blob that the lobe refers to.
1136 ++ blob-to-noun :: grab blob
1139 %delta (lump r.p (lobe-to-noun q.p))
1144If we have either a direct or an indirect blob, then the data is stored
1145right in the blob. Otherwise, we have to reconstruct it from the diffs.
1146We do this by calling `++lump` on the diff in the blob with the data
1147obtained by recursively calling the parent of this blob.
1149 ++ lump :: apply patch
1152 ?+ p.don ~|(%unsupported !!)
1154 ?+ -.q.don ~|(%unsupported !!)
1156 %c (lurk ((hard (list)) src) p.q.don)
1157 %d (lure src p.q.don)
1161 =+ dst=(lore ((hard ,@) src))
1163 ?+ -.q.don ~|(%unsupported !!)
1164 %a ((hard (list ,@t)) q.q.don)
1165 %c (lurk dst p.q.don)
1169This is defined in `arvo/hoon.hoon` for historical reasons which are
1170likely no longer applicable. Since the `++umph` structure will likely
1171change we convert clay to be a typed filesystem, we'll only give a
1172high-level description of this process. If we have a `%a` udon, then
1173we're performing a trivial replace, so we produce simply `q.q.don`. If
1174we have a `%c` udon, then we're performing a list merge (as in, for
1175example, lines of text). The merge is performed by `++lurk`.
1177 ++ lurk :: apply list patch
1178 |* [hel=(list) rug=(urge)]
1187 hel (slag p.i.rug hel)
1188 war (weld (flop (scag p.i.rug hel)) war)
1194 hel =+ gur=(flop p.i.rug)
1197 ?>(&(?=(^ hel) =(i.gur i.hel)) $(hel t.hel, gur t.gur))
1198 war (weld q.i.rug war)
1202We accumulate our final result in `war`. If there's nothing more in our
1203list of merge instructions (unces), we just reverse `war` and produce
1204it. Otherwise, we process another unce. If the unce is of type `&`, then
1205we have `p.i.rug` lines of no changes, so we just copy them over from
1206`hel` to `war`. If the unice is of type `|`, then we verify that the
1207source lines (in `hel`) are what we expect them to be (`p.i.rug`),
1208crashing on failure. If they're good, then we append the new lines in
1209`q.i.rug` onto `war`.
1211And that's really it. List merges are pretty easy. Anyway, if you
1212recall, we were discussing `++checkout-ankh`.
1214 ++ checkout-ankh :: checkout-ankh:ze
1215 |= hat=(map path lobe) :: checkout commit
1218 %+ roll (~(tap by hat) ~)
1219 |= [[pat=path bar=lobe] ank=ankh]
1223 =+ zar=(lobe-to-noun bar)
1224 ank(q [~ (sham zar) zar])
1225 =+ nak=(~(get by r.ank) i.pat)
1227 r %+ ~(put by r.ank) i.pat
1228 $(pat t.pat, ank (fall nak _ankh))
1231If the path is null, then we calculate `zar`, the raw data at the path
1232`pat` in this version. We produce the given ankh with the correct data.
1234Otherwise, we try to get the child we're looking at from our parent
1235ankh. If it's already been created, this succeeds; otherwise, we simply
1236create a default blank ankh. We place ourselves in our parent after
1237recursively computing our children.
1239This algorithm really isn't that complicated, but it may not be
1240immediately obvious why it works. An example should clear everything up.
1242Suppose `hat` is a map of the following information.
1244 /greeting --> "customary upon meeting"
1245 /greeting/english --> "hello"
1246 /greeting/spanish --> "hola"
1247 /greeting/russian/short --> "привет"
1248 /greeting/russian/long --> "Здравствуйте"
1249 /farewell/russian --> "до свидания"
1251Furthermore, let's say that we process them in this order:
1254 /greeting/russian/short
1255 /greeting/russian/long
1260Then, the first path we process is `/greeting/english` . Since our path
1261is not null, we try to get `nak`, but because our ankh is blank at this
1262point it doesn't find anything. Thus, update our blank top-level ankh
1263with a child `%greeting`. and recurse with the blank `nak` to create the
1264ankh of the new child.
1266In the recursion, we our path is `/english` and our ankh is again blank.
1267We try to get the `english` child of our ankh, but this of course fails.
1268Thus, we update our blank `/greeting` ankh with a child `english`
1269produced by recursing.
1271Now our path is null, so we call `++lobe-to-noun` to get the actual
1272data, and we place it in the brand-new ankh.
1274Next, we process `/greeting/russian/short`. Since our path is not null,
1275we try to get the child named `%greeting`, which does exist since we
1276created it earlier. We put modify this child by recursing on it. Our
1277path is now `/russian/short`, so we look for a `%russian` child in our
1278`/greeting` ankh. This doesn't exist, so we add it by recursing. Our
1279path is now `/short`, so we look for a `%short` child in our
1280`/greeting/russian` ankh. This doesn't exist, so we add it by recursing.
1281Now our path is null, so we set the contents of this node to `"привет"`,
1282and we're done processing this path.
1284Next, we process `/greeting/russian/long`. This proceeds similarly to
1285the previous except that now the ankh for `/greeting/russian` already
1286exists, so we simply reuse it rather than creating a new one. Of course,
1287we still must create a new `/greeting/russian/long` ankh.
1289Next, we process `/greeting`. This ankh already exists, so after we've
1290recursed once, our path is null, and our ankh is not blank -- it already
1291has two children (and two grandchildren). We don't touch those, though,
1292since a node may be both a file and a directory. We just add the
1293contents of the file -- "customary upon meeting" -- to the existing
1296Next, we process `/greeting/spanish`. Of course, the `/greeting` ankh
1297already exists, but it doesn't have a `%spanish` child, so we create
1298that, taking care not to disturb the contents of the `/greeting` file.
1299We put "hola" into the ankh and call it good.
1301Finally, we process `/farewell/russian`. Here, the `/farewell` ankh
1302doesn't exist, so we create it. Clearly the newly-created ankh doesn't
1303have any children, so we have to add a `%russian` child, and in this
1304child we put our last content -- "до свидания".
1306We hope it's fairly obvious that the order we process the paths doesn't
1307affect the final ankh tree. The tree will be constructed in a very
1308different order depending on what order the paths come in, but the
1309resulting tree is independent of order.
1311At any rate, we were talking about something important, weren't we? If
1312you recall, that concludes our discussion of `++rewind`, which was
1313called from `++read-at-aeon`. In summary, `++rewind` returns a context
1314in which our current state is (very nearly) as it was when the specified
1315version of the desk was the head. This allows `++read-at-aeon` to call
1316`++read` to read the requested information.
1319 |= mun=mood :: read at point
1323 ?: &(?=(%w p.mun) !?=(%ud -.q.mun))
1330 ?^(r.mun ~ [~ [t.yak (forge-nori yak)]])
1331 ::?> ?=(^ hit) ?^(r.mun ~ [~ i.hit]) :: what do?? need [@da nori]
1332 (query(ank ank:(descend-path:(zu ank) r.mun)) p.mun)
1334If we're requesting the dome, then we just return that immediately.
1336If we're requesting the revision number of the desk and we're not
1337requesting it by number, then we just return the current number of this
1338desk. Note of course that this was really already handled in
1341If we're requesting a `%w` with a specific revision number, then we do
1342something or other with the commit there. It's kind of weird, and it
1343doesn't seem to work, so we'll ignore this case.
1345Otherwise, we descend into the ankh tree to the given path with
1346`++descend-path:zu`, and then we handle specific request in `++query`.
1348 ++ descend-path :: descend recursively
1351 ?~(way +> $(way t.way, +> (descend i.way)))
1353This is simple recursion down into the ankh tree. `++descend` descends
1354one level, so this will eventually get us down to the path we want.
1356 ++ descend :: descend
1359 =+ you=(~(get by r.ank) lol)
1360 +>.$(ram [lol ram], ank ?~(you [*cash ~ ~] u.you))
1362`ram` is the path that we're at, so to descend one level we push the
1363name of this level onto that path. We update the ankh with the correct
1364one at that path if it exists; else we create a blank one.
1366Once we've decscended to the correct level, we need to actually deal
1369 ++ query :: query:ze
1370 |= ren=?(%u %v %x %y %z) :: endpoint query
1373 %u [~ `rang`+<+>.query]
1374 %v [~ `dome`+<+<.query]
1375 %x ?~(q.ank ~ [~ q.u.q.ank])
1380Now that everything's set up, it's really easy. If they're requesting
1381the rang, dome, or ankh, we give it to them. If the contents of a file,
1382we give it to them if it is in fact a file. If the `arch`, then we
1383calculate it with `++as-arch`.
1385 ++ as-arch :: as-arch:ze
1386 ^- arch :: arch report
1388 ?~(q.ank ~ [~ p.u.q.ank])
1391 [[p.n.r.ank ~] $(r.ank l.r.ank) $(r.ank r.r.ank)]
1393This very simply strips out all the "real" data and returns just our own
1394hash, the hash of the file contents (if we're a file), and a map of the
1395names of our immediate children.
1397Lifecycle of a Local Subscription
1398---------------------------------
1400A subscription to a range of revisions of a desk initially follows the
1401same path that a single read does. In `++aver`, we checked the head of
1402the given rave. If the head was `&`, then it was a single request, so we
1403handled it above. If `|`, then we handle it with the following code.
1405 =+ nab=(~(case-to-aeon ze lim dom ran) p.p.rav)
1407 ?> =(~ (~(case-to-aeon ze lim dom ran) q.p.rav))
1408 (duce hen (rive rav))
1409 =+ huy=(~(case-to-aeon ze lim dom ran) q.p.rav)
1410 ?: &(?=(^ huy) |((lth u.huy u.nab) &(=(0 u.huy) =(0 u.nab))))
1412 =+ top=?~(huy let.dom u.huy)
1413 =+ sar=(~(lobes-at-path ze lim dom ran) u.nab r.p.rav)
1414 =+ ear=(~(lobes-at-path ze lim dom ran) top r.p.rav)
1417 =+ fud=(~(make-nako ze lim dom ran) u.nab top)
1418 (bleb hen u.nab fud)
1423 (duce hen `rove`[%| ptr q.p.rav r.p.rav ear])
1426Recall that `++case-to-aeon:ze` produces the revision number that a case
1427corresponds to, if it corresponds to any. If it doesn't yet correspond
1428to a revision, then it produces null.
1430Thus, we first check to see if we've even gotten to the beginning of the
1431range of revisions requested. If not, then we assert that we haven't yet
1432gotten to the end of the range either, because that would be really
1433strange. If not, then we immediately call `++duce`, which, if you
1434recall, for a local request, simply puts this duct and rove into our
1435cult `qyx`, so that we know who to respond to when the revision does
1438If we've already gotten to the first revision, then we can produce some
1439content immediately. If we've also gotten to the final revision, and
1440that revision is earlier than the start revision, then it's a bad
1441request and we call `++blub`, which tells the subscriber that his
1442subscription will not be satisfied.
1444Otherwise, we find the data at the given path at the beginning of the
1445subscription and at the last available revision in the subscription. If
1446they're the same, then we don't send a notification. Otherwise, we call
1447`++gack`, which creates the `++nako` we need to produce. We call
1448`++bleb` to actually produce the information.
1450If we already have the last requested revision, then we also tell the
1451subscriber with `++blub` that the subscription will receive no further
1454If there will be more revisions in the subscription, then we call
1455`++duce`, adding the duct to our subscribers. We modify the rove to
1456start at the next revision since we've already handled all the revisions
1459We glossed over the calls to `++lobes-at-path`, `++make-nako`, and
1460`++bleb`, so we'll get back to those right now. `++bleb` is simple, so
1461we'll start with that.
1463 ++ bleb :: ship sequence
1464 |= [hen=duct ins=@ud hip=nako]
1466 (blab hen [%w [%ud ins] ~] hip)
1468We're given a duct, the beginning revision number, and the nako that
1469contains the updates since that revision. We use `++blab` to produce
1470this result to the subscriber. The case is `%w` with a revision number
1471of the beginning of the subscription, and the data is the nako itself.
1473We call `++lobes-at-path:ze` to get the data at the particular path.
1475 ++ lobes-at-path :: lobes-at-path:ze
1476 |= [oan=aeon pax=path] :: data at path
1494At revision zero, the theoretical common revision between all
1495repositories, there is no data, so we produce null.
1497We get the list of paths (paired with their lobe) in the revision
1498referred to by the given number and we keep only those paths which begin
1499with `pax`. Converting to a map, we now have a map from the subpaths at
1500the given path to the hash of their data. This is simple and efficient
1501to calculate and compare to later revisions. This allows us to easily
1502tell if a node or its children have changed.
1504Finally, we will describe `++make-nako:ze`.
1506 ++ make-nako :: gack a through b
1508 ^- [(map aeon tako) aeon (set yaki) (set blob)]
1510 =- [(takos-to-yakis -<) (lobes-to-blobs ->)]
1511 %+ reachable-between-takos
1512 (~(get by hit) a) :: if a not found, a=0
1515 %- mo %+ skim (~(tap by hit) ~)
1517 &((gth p a) (lte p b))
1519We need to produce four things -- the numbers of the new commits, the
1520number of the latest commit, the new commits themselves, and the new
1523The first is fairly easy to produce. We simply go over our map of
1524numbered commits and produce all those numbered greater than `a` and not
1527The second is even easier to produce -- `b` is clearly our most recent
1530The third and fourth are slightly more interesting, though not too
1531terribly difficult. First, we call `++reachable-between-takos`.
1533 ++ reachable-between-takos
1534 |= [a=(unit tako) b=tako] :: pack a through b
1535 ^- [(set tako) (set lobe)]
1538 (reachable-takos r:(tako-to-yaki u.a))
1539 =+ yak=`yaki`(tako-to-yaki b)
1540 %+ new-lobes-takos (new-lobes ~ sar) :: get lobes
1541 |- ^- (set tako) :: walk onto sar
1542 ?: (~(has in sar) r.yak)
1544 =+ ber=`(set tako)`(~(put in `(set tako)`~) `tako`r.yak)
1548 |= [yek=tako bar=(set tako)]
1550 ?: (~(has in bar) yek) :: save some time
1553 ^$(yak (tako-to-yaki yek))
1555We take a possible starting commit and a definite ending commit, and we
1556produce the set of commits and the set of data between them.
1558We let `sar` be the set of commits reachable from `a`. If `a` is null,
1559then obviously no commits are reachable. Otherwise, we call
1560`++reachable-takos` to calculate this.
1562 ++ reachable-takos :: reachable
1563 |= p=tako :: XX slow
1565 =+ y=(tako-to-yaki p)
1566 =+ t=(~(put in _(set tako)) p)
1569 ?: (~(has in s) q) :: already done
1571 (~(uni in s) ^$(p q)) :: otherwise traverse
1573We very simply produce the set of the given tako plus its parents,
1576Back in `++reachable-between-takos`, we let `yak` be the yaki of `b`,
1577the ending commit. With this, we create a set that is the union of `sar`
1578and all takos reachable from `b`.
1580We pass `sar` into `++new-lobes` to get all the lobes referenced by any
1581tako referenced by `a`. The result is passed into `++new-lobes-takos` to
1582do the same, but not recomputing those in already calculated last
1583sentence. This produces the sets of takos and lobes we need.
1585 ++ new-lobes :: object hash set
1586 |= [b=(set lobe) a=(set tako)] :: that aren't in b
1588 %+ roll (~(tap in a) ~)
1589 |= [tak=tako bar=(set lobe)]
1591 =+ yak=(tako-to-yaki tak)
1592 %+ roll (~(tap by q.yak) ~)
1593 |= [[path lob=lobe] far=_bar]
1595 ?~ (~(has in b) lob) :: don't need
1597 =+ gar=(lobe-to-blob lob)
1599 %direct (~(put in far) lob)
1600 %delta (~(put in $(lob q.gar)) lob)
1601 %indirect (~(put in $(lob s.gar)) lob)
1604Here, we're creating a set of lobes referenced in a commit in `a`. We
1605start out with `b` as the initial set of lobes, so we don't need to
1606recompute any of the lobes referenced in there.
1608The algorithm is pretty simple, so we won't bore you with the details.
1609We simply traverse every commit in `a`, looking at every blob referenced
1610there, and, if it's not already in `b`, we add it to `b`. In the case of
1611a direct blob, we're done. For a delta or an indirect blob, we
1612recursively add every blob referenced within the blob.
1614 ++ new-lobes-takos :: garg & repack
1615 |= [b=(set lobe) a=(set tako)]
1616 ^- [(set tako) (set lobe)]
1619Here, we just update the set of lobes we're given with the commits we're
1620given and produce both sets.
1622This concludes our discussion of a local subscription.
1624Lifecycle of a Foreign Read or Subscription
1625-------------------------------------------
1627Foreign reads and subscriptions are handled in much the same way as
1628local ones. The interface is the same -- a vane or app sends a `%warp`
1629kiss with the request. The difference is simply that the `sock` refers
1632Thus, we start in the same place -- in `++call`, handling `%warp`.
1633However, since the two side of the `sock` are different, we follow a
1636 =+ wex=(do now p.q.hic p.q.q.hic ruf)
1640 abet:(eave:wex hen u.q.q.q.hic)
1641 [-.woo (posh q.p.q.hic p.q.q.hic +.woo ruf)]
1643If we compare this to how the local case was handled, we see that it's
1644not all that different. We use `++do` rather than `++un` and `++de` to
1645set up the core for the foreign ship. This gives us a `++de` core, so we
1646either cancel or begin the request by calling `++ease` or `++eave`,
1647exactly as in the local case. In either case, we call `++abet:de` to
1648resolve our various types of output into actual moves, as described in
1649the local case. Finally, we call `++posh` to update our raft, putting
1650the modified rung into the raft.
1652We'll first trace through `++do`.
1655 |= [now=@da [who=ship him=ship] syd=@tas ruf=raft]
1657 =+ rug=(~(get by hoy.ruf) him)
1660 =+ yit=(~(get by rus.rug) syd)
1661 ?^(yit u.yit `rede`[~2000.1.1 ~ [~ *rind] *dome])
1662 ((de now ~ ~) [who him] syd red ran.ruf)
1664If we already have a rung for this foreign ship, then we use that.
1665Otherwise, we create a new blank one. If we already have a rede in this
1666rung, then we use that, otherwise we create a blank one. An important
1667point to note here is that we let `ref` in the rede be `[~ *rind]`.
1668Recall, for domestic desks `ref` is null. We use this to distinguish
1669between foreign and domestic desks in `++de`.
1671With this information, we create a `++de` core as usual.
1673Although we've already covered `++ease` and `++eave`, we'll go through
1674them quickly again, highlighting the case of foreign request.
1676 ++ ease :: release request
1680 =+ rov=(~(got by qyx) hen)
1681 =. qyx (~(del by qyx) hen)
1682 (mabe rov (cury best hen))
1683 =. qyx (~(del by qyx) hen)
1685 =+ nux=(~(get by fod.u.ref) hen)
1688 say [[hen [(scot %ud u.nux) ~] for [u.nux syd ~]] say]
1689 fod.u.ref (~(del by fod.u.ref) hen)
1690 bom.u.ref (~(del by bom.u.ref) u.nux)
1693Here, we still remove the duct from our cult (we maintain a cult even
1694for foreign desks), but we also need to tell the foreign desk to cancel
1695our subscription. We do this by sending a request (by appending to
1696`say`, which gets resolved in `++abet:de` to a `%want` kiss to ames) to
1697the foreign ship to cancel the subscription. Since we don't anymore
1698expect a response on this duct, we remove it from `fod` and `bom`, which
1699are the maps between ducts, raves, and request sequence numbers.
1700Basically, we remove every trace of the subscription from our request
1703In the case of `++eave`, where we're creating a new request, everything
1704is exactly identical to the case of the local request except `++duce`.
1705We said that `++duce` simply puts the request into our cult. This is
1706true for a domestic request, but distinctly untrue for foreign requests.
1708 ++ duce :: produce request
1709 |= [hen=duct rov=rove]
1711 =. qyx (~(put by qyx) hen rov)
1713 |- ^+ +>+.$ :: XX why?
1716 ?. ?=([%& %v *] rav) rav
1717 [%| [%ud let.dom] `case`q.p.rav r.p.rav]
1720 say [[hen [(scot %ud inx) ~] for [inx syd ~ vaw]] say]
1721 nix.u.ref +(nix.u.ref)
1722 bom.u.ref (~(put by bom.u.ref) inx [hen vaw])
1723 fod.u.ref (~(put by fod.u.ref) hen inx)
1726If we have a request manager (i.e. this is a foreign desk), then we do
1727the approximate inverse of `++ease`. We create a rave out of the given
1728request and send it off to the foreign desk by putting it in `say`. Note
1729that the rave is created to request the information starting at the next
1730revision number. Since this is a new request, we put it into `fod` and
1731`bom` to associate the request with its duct and its sequence number.
1732Since we're using another sequence number, we must increment `nix`,
1733which represents the next available sequence number.
1735And that's really it for this side of the request. Requesting foreign
1736information isn't that hard. Let's see what it looks like on the other
1737side. When we get a request from another ship for information on our
1738ship, that comes to us in the form of a `%wart` from ames.
1740We handle a `%wart` in `++call`, right next to where we handle the
1745 =+ ryf=((hard riff) s.q.hic)
1748 :^ %pass [(scot %p p.p.q.hic) (scot %p q.p.q.hic) r.q.hic]
1750 [%warp [p.p.q.hic p.p.q.hic] ryf]
1753Every request we receive should be of type `riff`, so we coerce it into
1754that type. We just convert this into a new `%warp` kiss that we pass to
1755ourself. This gets handled like normal, as a local request. When the
1756request produces a value, it does so like normal as a `%writ`, which is
1757returned to `++take` along the path we just sent it on.
1761 =+ our=(need (slaw %p i.tea))
1762 =+ him=(need (slaw %p i.t.tea))
1765 [%pass ~ %a [%want [our him] [%r %re %c t.t.tea] p.+.q.hin]]
1768Since we encoded the ship we need to respond to in the path, we can just
1769pass our `%want` back to ames, so that we tell the requesting ship about
1772This comes back to the original ship as a `%waft` from ames, which comes
1773into `++take`, right next to where we handled `%writ`.
1777 =+ syd=(need (slaw %tas i.tea))
1778 =+ inx=(need (slaw %ud i.t.tea))
1781 (knit:(do now p.+.q.hin syd ruf) [inx ((hard riot) q.+.q.hin)])
1784 [-.zot (posh q.p.+.q.hin syd +.zot ruf)]
1785 [mos ..^$(ran.ruf ran.zat)] :: merge in new obj
1787This gets the desk and sequence number from the path the request was
1788sent over. This determines exactly which request is being responded to.
1789We call `++knit:de` to apply the changes to our local desk, and we call
1790`++wake` to update our subscribers. Then we call `++abet:de` and
1791`++posh` as normal (like in `++eave`).
1793We'll examine `++knit` and `++wake`, in that order.
1795 ++ knit :: external change
1796 |= [inx=@ud rot=riot]
1800 =+ ruv=(~(get by bom.u.ref) inx)
1802 => ?. |(?=(~ rot) ?=(& -.q.u.ruv)) .
1804 bom.u.ref (~(del by bom.u.ref) inx)
1805 fod.u.ref (~(del by fod.u.ref) p.u.ruv)
1808 =+ rav=`rave`q.u.ruv
1811 ?.(&(?=(| -.rav) ?=(%da -.q.p.rav)) lim `@da`p.q.p.rav)
1814 ?. ?=(& -.rav) haw.u.ref
1815 (~(put by haw.u.ref) p.rav ~)
1819 (~(put by haw.u.ref) [p.p.u.rot q.p.u.rot q.u.rot] ~ r.u.rot)
1820 ?. ?=(%w p.p.u.rot) +>+.$
1822 =+ nez=[%w [%ud let.dom] ~]
1823 =+ nex=(~(get by haw.u.ref) nez)
1825 ?~ u.nex +>+.^$ :: should never happen
1826 =. +>+.^$ =+ roo=(edis ((hard nako) u.u.nex))
1827 ?>(?=(^ ref.roo) roo)
1829 haw.u.ref (~(del by haw.u.ref) nez)
1832This is kind of a long gate, but don't worry, it's not bad at all.
1834First, we assert that we're not a domestic desk. That wouldn't make any
1837Since we have the sequence number of the request, we can get the duct
1838and rave from `bom` in our request manager. If we didn't actually
1839request this data (or the request was canceled before we got it), then
1842Else, we remove the request from `bom` and `fod` unless this was a
1843subscription request and we didn't receive a null riot (which would
1844indicate the last message on the subscription).
1846Now, if we received a null riot, then if this was a subscription request
1847by date, then we update `lim` in our request manager (representing the
1848latest time at which we have complete information for this desk) to the
1849date that was requested. If this was a single read request, then we put
1850the result in our simple cache `haw` to make future requests faster.
1853If we received actual data, then we put it into our cache `haw`. Unless
1854it's a `%w` request, we're done.
1856If it is a `%w` request, then we try to get the `%w` at our current head
1857from the cache. In general, that should be the thing we just put in a
1858moment ago, but that is not guaranteed. The most common case where this
1859is different is when we receive desk updates out of order. At any rate,
1860since we now have new information, we need to apply it to our local copy
1861of the desk. We do so in `++edis`, and then we remove the stuff we just
1862applied from the cache, since it's not really a true "single read", like
1863what should really be in the cache.
1865 ++ edis :: apply subscription
1869 hit.dom (~(uni by hit.dom) gar.nak)
1871 lat.ran %+ roll (~(tap in bar.nak) ~)
1873 |= [sar=blob yeb=(map lobe blob)]
1874 =+ zax=(blob-to-lobe sar)
1875 %+ ~(put by yeb) zax sar
1876 hut.ran %+ roll (~(tap in lar.nak) ~)
1878 |= [sar=yaki yeb=(map tako yaki)]
1879 %+ ~(put by yeb) r.sar sar
1882This shows, of course, exactly why nako is defined the way it is. To
1883become completely up to date with the foreign desk, we need to merge
1884`hit` with the foreign one so that we have all the revision numbers. We
1885update `let` so that we know which revision is the head.
1887We merge the new blobs in `lat`, keying them by their hash, which we get
1888from a call to `++blob-to-lobe`. Recall that the hash is stored in the
1889actual blob itself. We do the same thing to the new yakis, putting them
1890in `hut`, keyed by their hash.
1892Now, our local dome should be exactly the same as the foreign one.
1894This concludes our discussion of `++knit`. Now the changes have been
1895applied to our local copy of the desk, and we just need to update our
1896subscribers. We do so in `++wake:de`.
1898 ++ wake :: update subscribers
1900 =+ xiq=(~(tap by qyx) ~)
1901 =| xaq=(list ,[p=duct q=rove])
1904 ..wake(qyx (~(gas by *cult) xaq))
1907 =+ cas=?~(ref ~ (~(get by haw.u.ref) `mood`p.q.i.xiq))
1911 ..wake ?~ u.cas (blub p.i.xiq)
1912 (blab p.i.xiq p.q.i.xiq u.u.cas)
1914 =+ nao=(~(case-to-aeon ze lim dom ran) q.p.q.i.xiq)
1915 ?~ nao $(xiq t.xiq, xaq [i.xiq xaq])
1916 $(xiq t.xiq, ..wake (balk p.i.xiq u.nao p.q.i.xiq))
1919 =+ mot=`moot`p.q.i.xiq
1920 =+ nab=(~(case-to-aeon ze lim dom ran) p.mot)
1922 $(xiq t.xiq, xaq [i.xiq xaq])
1923 =+ huy=(~(case-to-aeon ze lim dom ran) q.mot)
1925 =+ ptr=[%ud +(let.dom)]
1928 xaq [[p.i.xiq [%| ptr q.mot r.mot s.mot]] xaq]
1930 (~(lobes-at-path ze lim dom ran) let.dom r.p.q.i.xiq)
1931 ?: =(s.p.q.i.xiq ear) ..wake
1932 =+ fud=(~(make-nako ze lim dom ran) u.nab let.dom)
1933 (bleb p.i.xiq let.dom fud)
1937 ..wake =- (blub:- p.i.xiq)
1939 (~(lobes-at-path ze lim dom ran) u.huy r.p.q.i.xiq)
1940 ?: =(s.p.q.i.xiq ear) (blub p.i.xiq)
1941 =+ fud=(~(make-nako ze lim dom ran) u.nab u.huy)
1942 (bleb p.i.xiq +(u.nab) fud)
1947This is even longer than `++knit`, but it's pretty similar to `++eave`.
1948We loop through each of our subscribers `xiq`, processing each in turn.
1949When we're done, we just put the remaining subscribers back in our
1952If the subscriber is a single read, then, if this is a foreign desk
1953(note that `++wake` is called from other arms, and not only on foreign
1954desks). Obviously, if we find an identical request there, then we can
1955produce the result immediately. Referential transparency for the win. We
1956produce the result with a call to `++blab`. If this is a foreign desk
1957but the result is not in the cache, then we produce `++blub` (canceled
1958subscription with no data) for reasons entirely opaque to me. Seriously,
1959it seems like we should wait until we get an actual response to the
1960request. If someone figures out why this is, let me know. At any rate,
1963If this is a domestic desk, then we check to see if the case exists yet.
1964If it doesn't, then we simply move on to the next subscriber, consing
1965this one onto `xaq` so that we can check again the next time around. If
1966it does exist, then we call `++balk` to fulfill the request and produce
1969`++balk` is very simple, so we'll describe it here before we get to the
1972 ++ balk :: read and send
1973 |= [hen=duct oan=@ud mun=mood]
1975 =+ vid=(~(read-at-aeon ze lim dom ran) oan mun)
1976 ?~ vid (blub hen) (blab hen mun u.vid)
1978We call `++read-at-aeon` on the given request and aeon. If you recall,
1979this processes a `mood` at a particular aeon and produces the result, if
1980there is one. If there is data at the requested location, then we
1981produce it with `++blab`. Else, we call `++blub` to notify the
1982subscriber that no data can ever come over this subscriptioin since it
1983is now impossible for there to ever be data for the given request.
1984Because referential transparency.
1986At any rate, back to `++wake`. If the given rave is a subscription
1987request, then we proceed similarly to how we do in `++eave`. We first
1988try to get the aeon referred to by the starting case. If it doesn't
1989exist yet, then we can't do anything interesting with this subscription,
1990so we move on to the next one.
1992Otherwise, we try to get the aeon referred to by the ending case. If it
1993doesn't exist yet, then we produce all the information we can. We call
1994`++lobes-at-path` at the given aeon and path to see if the requested
1995path has actually changed. If it hasn't, then we don't produce anything;
1996else, we produce the correct nako by calling `++bleb` on the result of
1997`++make-nako`, as in `++eave`. At any rate, we move on to the next
1998subscription, putting back into our cult the current subscription with a
1999new start case of the next aeon after the present.
2001If the aeon referred to by the ending case does exist, then we drop this
2002subscriber from our cult and satisfy its request immediately. This is
2003the same as before -- we check to see if the data at the path has
2004actually changed, producing it if it has; else, we call `++blub` since
2005no more data can be produced over this subscription.
2007This concludes our discussion of foreign requests.
keysesc to close
- j k
- move down, up
- enter
- open
- esc
- back, or up a level
- 1 2 3 4
- readme, files, commits, refs
- /
- filter
- y
- copy the clone command
- c
- compare the chosen commit with another (or shift-click both)
- ?
- show or hide this list