shrub/pkg/arvo/sys/vane/clay.hoon

5438 lines
165 KiB
Plaintext
Raw Normal View History

2016-11-24 07:25:07 +03:00
:: clay (4c), revision control
2019-02-02 00:46:09 +03:00
!:
2022-04-06 06:30:52 +03:00
:: The way to understand Clay is to take it section-by-section:
::
:: - Data structures. You *must* start here; make sure you understand
:: the entire contents of +raft.
::
:: - Individual reads. +aver is the entry point, follow it through
:: +read-at-aeon to understand each kind of read.
::
:: - Subscriptions. +wake is the center of this mechanism; nothing
:: else responds to subscriptions. +wake has no arguments, which means
:: every subscription response happens when something in Clay's *state*
:: has changed. No edge-triggered responses.
::
:: - Receiving foreign data. For individual requests, this is
:: +take-foreign-answer. For sync requests (%many, which is %sing %v
:: for a foreign desk), this is +foreign-update.
::
:: - Ford. +ford builds hoon files and gives files their types.
:: Read +build-file for the first, and +read-file is the second.
::
:: - Writing to a desk. Every write to a desk goes through +park, read
:: it thoroughly.
::
:: - Merges. Control flow starts at +start-merge, then +merge, but
:: everything is scaffolding for +merge-by-germ, which is the ideal of
:: a merge function: it takes two commits and a merge strategy and
:: produces a new commit.
2016-11-24 07:25:07 +03:00
::
2022-04-07 07:45:30 +03:00
:: - Tombstoning. This is in +tomb.
::
2016-11-24 07:25:07 +03:00
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::
:: Here are the structures. `++raft` is the formal arvo state. It's also
2022-04-06 06:30:52 +03:00
:: worth noting that many of the clay-related structures are defined in lull.
2016-11-24 07:25:07 +03:00
::
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
=/ bud
^~
=/ zuse !>(..zuse)
:* zuse=zuse
nave=(slap zuse !,(*hoon nave:clay))
cork=(slap zuse !,(*hoon cork))
same=(slap zuse !,(*hoon same))
mime=(slap zuse !,(*hoon mime))
cass=(slap zuse !,(*hoon cass:clay))
==
::
2020-12-06 11:38:37 +03:00
|= our=ship
=, clay
2016-11-24 07:25:07 +03:00
=> |%
+$ aeon @ud :: version number
2016-11-24 07:25:07 +03:00
::
:: Part of ++mery, representing the set of changes between the mergebase and
:: one of the desks being merged.
::
:: -- `new` is the set of files in the new desk and not in the mergebase.
:: -- `cal` is the set of changes in the new desk from the mergebase except
:: for any that are also in the other new desk.
:: -- `can` is the set of changes in the new desk from the mergebase and that
:: are also in the other new desk (potential conflicts).
:: -- `old` is the set of files in the mergebase and not in the new desk.
::
+$ cane
$: new=(map path lobe)
cal=(map path lobe)
can=(map path cage)
old=(map path ~)
2016-11-24 07:25:07 +03:00
==
::
:: Type of request.
::
2020-01-23 08:23:14 +03:00
:: %d produces a set of desks, %p gets file permissions, %t gets all paths
:: with the specified prefix, %u checks for existence, %v produces a ++dome
:: of all desk data, %w gets @ud and @da variants for the given case, %x
:: gets file contents, %y gets a directory listing, and %z gets a recursive
:: hash of the file contents and children.
2016-11-24 07:25:07 +03:00
::
:: ++ care ?(%d %p %t %u %v %w %x %y %z)
2016-11-24 07:25:07 +03:00
::
:: Keeps track of subscribers.
::
:: A map of requests to a set of all the subscribers who should be notified
:: when the request is filled/updated.
::
+$ cult (jug wove duct)
2016-11-24 07:25:07 +03:00
::
2021-05-02 21:15:03 +03:00
:: State for ongoing %fuse merges. `con` maintains the ordering,
:: `sto` stores the data needed to merge, and `bas` is the base
:: beak for the merge.
::
2021-04-24 00:33:49 +03:00
+$ melt [bas=beak con=(list [beak germ]) sto=(map beak (unit dome:clay))]
2016-11-24 07:25:07 +03:00
::
:: Domestic desk state.
::
:: Includes subscriber list, dome (desk content), possible commit state (for
:: local changes), possible merge state (for incoming merges), and permissions.
2016-11-24 07:25:07 +03:00
::
+$ dojo
$: qyx=cult :: subscribers
dom=dome :: desk state
per=regs :: read perms per path
pew=regs :: write perms per path
2021-04-20 06:46:46 +03:00
fiz=melt :: state for mega merges
2016-11-24 07:25:07 +03:00
==
::
:: Desk state.
::
:: Includes a checked-out ankh with current content, most recent version, map
:: of all version numbers to commit hashes (commits are in hut.rang), and map
:: of labels to version numbers.
::
2019-05-16 01:50:13 +03:00
:: `mim` is a cache of the content in the directories that are mounted
:: to unix. Often, we convert to/from mime without anything really
:: having changed; this lets us short-circuit that in some cases.
:: Whenever you give an `%ergo`, you must update this.
::
+$ dome
$: ank=ankh :: state
let=aeon :: top id
hit=(map aeon tako) :: versions by id
lab=(map @tas aeon) :: labels
tom=(map tako norm) :: tomb policies
nor=norm :: default policy
mim=(map path mime) :: mime cache
fod=ford-cache :: ford cache
2016-11-24 07:25:07 +03:00
== ::
::
:: Commit state.
::
:: -- `del` is the paths we're deleting.
:: -- `ink` is the insertions of hoon files (short-circuited for
:: bootstrapping).
:: -- `ins` is all the other insertions.
:: -- `dif` is the diffs in `dig` applied to their files.
:: -- `mut` is the diffs between `muc` and the original files.
::
+$ dork :: diff work
$: del=(list path) :: deletes
ink=(list (pair path cage)) :: hoon inserts
ins=(list (pair path cage)) :: inserts
dif=(list (trel path lobe cage)) :: changes
mut=(list (trel path lobe cage)) :: mutations
2016-11-24 07:25:07 +03:00
== ::
::
:: Over-the-wire backfill request
::
+$ fill
$% [%0 =desk =lobe]
==
::
2020-04-16 02:24:48 +03:00
:: Ford cache
::
+$ ford-cache
$: files=(map path [res=vase dez=(set [dir=? =path])])
naves=(map mark [res=vase dez=(set [dir=? =path])])
marks=(map mark [res=dais dez=(set [dir=? =path])])
casts=(map mars [res=vase dez=(set [dir=? =path])])
tubes=(map mars [res=tube dez=(set [dir=? =path])])
2020-06-01 19:56:21 +03:00
==
2020-04-16 02:24:48 +03:00
::
2016-11-24 07:25:07 +03:00
:: Hash of a blob, for lookup in the object store (lat.ran)
::
+$ lobe @uvI :: blob ref
2016-11-24 07:25:07 +03:00
::
:: New desk data.
::
:: Sent to other ships to update them about a particular desk. Includes a map
:: of all new aeons to hashes of their commits, the most recent aeon, and sets
:: of all new commits and data.
::
+$ nako :: subscription state
$: gar=(map aeon tako) :: new ids
let=aeon :: next id
lar=(set yaki) :: new commits
bar=(set plop) :: new content
2016-11-24 07:25:07 +03:00
== ::
::
2022-04-06 10:18:23 +03:00
::
2016-11-24 07:25:07 +03:00
:: Formal vane state.
::
2018-12-13 10:42:15 +03:00
:: -- `rom` is our domestic state.
2016-11-24 07:25:07 +03:00
:: -- `hoy` is a collection of foreign ships where we know something about
:: their clay.
:: -- `ran` is the object store.
:: -- `mon` is a collection of mount points (mount point name to urbit
:: location).
:: -- `hez` is the unix duct that %ergo's should be sent to.
2018-01-24 00:52:31 +03:00
:: -- `cez` is a collection of named permission groups.
:: -- `pud` is an update that's waiting on a kernel upgrade
2016-11-24 07:25:07 +03:00
::
+$ raft :: filesystem
2019-05-02 04:21:32 +03:00
$: rom=room :: domestic
hoy=(map ship rung) :: foreign
ran=rang :: hashes
mon=(map term beam) :: mount points
hez=(unit duct) :: sync duct
cez=(map @ta crew) :: permission groups
2020-04-30 03:38:24 +03:00
pud=(unit [=desk =yoki]) :: pending update
2016-11-24 07:25:07 +03:00
== ::
::
:: Object store.
::
:: Maps of commit hashes to commits and content hashes to content.
::
+$ rang ::
$: hut=(map tako yaki) ::
lat=(map lobe blob) ::
2016-11-24 07:25:07 +03:00
== ::
::
:: Unvalidated response to a request.
::
:: Like a +$rant, but with a page of data rather than a cage of it.
2016-11-24 07:25:07 +03:00
::
+$ rand :: unvalidated rant
$: p=[p=care q=case r=@tas] :: clade release book
q=path :: spur
r=page :: data
2016-11-24 07:25:07 +03:00
== ::
::
:: Generic desk state.
::
:: -- `lim` is the most recent date we're confident we have all the
:: information for. For local desks, this is always `now`. For foreign
:: desks, this is the last time we got a full update from the foreign
:: urbit.
:: -- `ref` is a possible request manager. For local desks, this is null.
:: For foreign desks, this keeps track of all pending foreign requests
:: plus a cache of the responses to previous requests.
:: -- `qyx` is the set of subscriptions, with listening ducts. These
:: subscriptions exist only until they've been filled.
:: -- `dom` is the actual state of the filetree. Since this is used almost
:: exclusively in `++ze`, we describe it there.
::
+$ rede :: universal project
$: lim=@da :: complete to
ref=(unit rind) :: outgoing requests
qyx=cult :: subscribers
dom=dome :: revision state
per=regs :: read perms per path
pew=regs :: write perms per path
2021-04-20 06:46:46 +03:00
fiz=melt :: domestic mega merges
2016-11-24 07:25:07 +03:00
== ::
::
:: Foreign request manager.
::
:: When we send a request to a foreign ship, we keep track of it in here. This
:: includes a request counter, a map of request numbers to requests, a reverse
:: map of requesters to request numbers, a simple cache of common %sing
:: requests, and a possible nako if we've received data from the other ship and
:: are in the process of validating it.
::
+$ rind :: request manager
$: nix=@ud :: request index
bom=(map @ud update-state) :: outstanding
fod=(map duct @ud) :: current requests
haw=(map mood (unit cage)) :: simple cache
== ::
::
:: Active downloads
::
+$ update-state
$: =duct
=rave
have=(map lobe blob)
need=[let=(list lobe) old=(list lobe)]
nako=(qeu (unit nako))
busy=_|
==
2016-11-24 07:25:07 +03:00
::
:: Domestic ship.
::
:: `hun` is the duct to dill, and `dos` is a collection of our desks.
::
+$ room :: fs per ship
$: hun=duct :: terminal duct
dos=(map desk dojo) :: native desk
2016-11-24 07:25:07 +03:00
== ::
::
:: Stored request.
::
:: Like a +$rave but with caches of current versions for %next and %many.
2016-11-24 07:25:07 +03:00
:: Generally used when we store a request in our state somewhere.
::
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
+$ cach (unit (unit cage)) :: cached result
+$ wove [for=(unit [=ship ver=@ud]) =rove] :: stored source + req
+$ rove :: stored request
2019-05-23 00:39:12 +03:00
$% [%sing =mood] :: single request
[%next =mood aeon=(unit aeon) =cach] :: next version of one
$: %mult :: next version of any
=mool :: original request
aeon=(unit aeon) :: checking for change
old-cach=(map [=care =path] cach) :: old version
new-cach=(map [=care =path] cach) :: new version
== ::
2019-05-23 00:39:12 +03:00
[%many track=? =moat lobes=(map path lobe)] :: change range
2016-11-24 07:25:07 +03:00
== ::
::
:: Foreign desk data.
::
+$ rung
$: rus=(map desk rede) :: neighbor desks
2019-02-02 00:46:09 +03:00
==
2016-11-24 07:25:07 +03:00
::
+$ card (wind note gift) :: local card
+$ move [p=duct q=card] :: local move
+$ note :: out request $->
$~ [%b %wait *@da] ::
$% $: %$ :: to arvo
$>(%what waif) ::
== ::
$: %a :: to %ames
$>(%plea task:ames) ::
== ::
$: %b :: to %behn
2020-05-21 11:10:39 +03:00
$> $? %drip ::
%rest ::
%wait ::
== ::
task:behn ::
== ::
$: %c :: to %clay
$> $? %info :: internal edit
%merg :: merge desks
2021-05-02 21:15:03 +03:00
%fuse :: merge many
2021-09-21 00:13:04 +03:00
%park ::
%perm ::
2020-04-25 03:44:29 +03:00
%pork ::
%warp ::
%werp ::
== ::
task ::
== ::
$: %d :: to %dill
$>(%flog task:dill) ::
== ::
$: %g :: to %gall
$> $? %deal
%jolt
==
task:gall
== ::
2019-08-07 01:42:37 +03:00
$: %j :: by %jael
$>(%public-keys task:jael) ::
== == ::
+$ riot (unit rant) :: response+complete
+$ sign :: in result $<-
2020-12-08 03:22:26 +03:00
$~ [%behn %wake ~] ::
$% $: %ames ::
$> $? %boon :: response
%done :: (n)ack
%lost :: lost boon
== ::
gift:ames ::
== ::
2020-12-08 03:22:26 +03:00
$: %behn ::
$% $>(%wake gift:behn) :: timer activate
$>(%writ gift) ::
== == ::
2020-12-08 03:22:26 +03:00
$: %clay ::
$> $? %mere ::
%note ::
%writ ::
== ::
gift ::
== ::
$: %gall
$> $? %onto
%unto
==
gift:gall
==
$: %jael ::
$>(%public-keys gift:jael) ::
== == ::
2016-11-24 07:25:07 +03:00
-- =>
~% %clay-utilities ..part ~
:: %utilities
::
|%
:: +sort-by-head: sorts alphabetically using the head of each element
::
++ sort-by-head
|=([a=(pair path *) b=(pair path *)] (aor p.a p.b))
::
2020-04-28 12:56:28 +03:00
:: By convention: paf == (weld pax pat)
::
++ mode-to-commit
|= [hat=(map path lobe) pax=path all=? mod=mode]
^- [deletes=(set path) changes=(map path cage)]
=/ deletes
%- silt
%+ turn
^- (list path)
%+ weld
^- (list path)
%+ murn mod
|= [pat=path mim=(unit mime)]
^- (unit path)
?^ mim
~
`pat
^- (list path)
?. all
~
=+ mad=(malt mod)
=+ len=(lent pax)
=/ descendants=(list path)
%+ turn
%+ skim ~(tap by hat)
|= [paf=path lob=lobe]
=(pax (scag len paf))
|= [paf=path lob=lobe]
(slag len paf)
%+ skim
descendants
|= pat=path
(~(has by mad) pat)
|= pat=path
(weld pax pat)
::
=/ changes
%- malt
%+ murn mod
|= [pat=path mim=(unit mime)]
^- (unit [path cage])
?~ mim
~
`[(weld pax pat) %mime !>(u.mim)]
::
[deletes changes]
2022-03-26 01:32:38 +03:00
::
:: Safely add items to an object store.
::
:: It's important that first argument is the old and second is the new.
2022-03-26 02:41:54 +03:00
:: The rule is that if one side has a tombstone, we want to use the
:: other side. Additionally, if the old one was %direct, we don't want
:: to overwrite that with a %delta, since that could create loops where
:: they didn't exist.
2022-03-26 01:32:38 +03:00
::
++ uni-blobs
|= [old=(map lobe blob) new=(map lobe blob)]
^- (map lobe blob)
%- (~(uno by old) new)
|= [=lobe o=blob n=blob]
2022-03-26 02:41:54 +03:00
^- blob
2022-03-26 01:32:38 +03:00
?- -.o
%direct o
%delta o
%dead n
==
::
++ blob-to-blub
|= =blob
^- blub
?- -.blob
%dead blob
%direct blob
%delta blob(q [%blub q.blob])
==
::
++ blub-to-blob
|= =blub
^- blob
?- -.blub
%dead blub
%direct blub
%delta blub(q q.q.blub)
==
-- =>
~% %clay + ~
2019-05-03 04:50:20 +03:00
|%
2019-05-23 08:35:09 +03:00
:: Printable form of a wove; useful for debugging
::
++ print-wove
|= =wove
:- for.wove
?- -.rove.wove
%sing [%sing mood.rove.wove]
%next [%next [mood aeon]:rove.wove]
%mult [%mult [mool aeon]:rove.wove]
%many [%many [track moat]:rove.wove]
==
::
:: Printable form of a cult; useful for debugging
::
++ print-cult
|= =cult
%+ turn ~(tap by cult)
|= [=wove ducts=(set duct)]
[ducts (print-wove wove)]
::
++ fusion
|%
:: +wrap: external wrapper
::
++ wrap
|* [* state:ford]
[+<- +<+>-] :: cache.state
::
++ with-face |=([face=@tas =vase] vase(p [%face face p.vase]))
++ with-faces
=| res=(unit vase)
|= vaz=(list [face=@tas =vase])
^- vase
?~ vaz (need res)
=/ faz (with-face i.vaz)
=. res `?~(res faz (slop faz u.res))
$(vaz t.vaz)
::
++ ford
=> |%
2020-04-16 23:30:55 +03:00
+$ build
$% [%file =path]
[%mark =mark]
2021-01-13 03:10:11 +03:00
[%dais =mark]
2020-04-16 23:30:55 +03:00
[%cast =mars]
2021-01-13 03:10:11 +03:00
[%tube =mars]
2020-04-16 23:30:55 +03:00
[%vale =path]
==
+$ state
$: baked=(map path cage)
cache=ford-cache
stack=(list (set [dir=? =path]))
2020-04-16 23:30:55 +03:00
cycle=(set build)
==
+$ args
2021-06-29 03:01:51 +03:00
$: =ankh
deletes=(set path)
changes=(map path (each page lobe))
file-store=(map lobe blob)
=ford-cache
==
--
|= args
:: nub: internal mutable state for this computation
::
=| nub=state
=. cache.nub ford-cache
|%
:: +pop-stack: pop build stack, copying deps downward
::
++ pop-stack
^- [(set [dir=? =path]) _stack.nub]
=^ top=(set [dir=? =path]) stack.nub stack.nub
=? stack.nub ?=(^ stack.nub)
stack.nub(i (~(uni in i.stack.nub) top))
[top stack.nub]
2021-02-24 20:59:07 +03:00
:: +read-file: retrieve marked, validated file contents at path
::
2021-02-24 20:59:07 +03:00
++ read-file
|= =path
^- [cage state]
2020-04-28 07:52:07 +03:00
~| %error-validating^path
?^ got=(~(get by baked.nub) path)
[u.got nub]
=; [res=cage bun=state]
=. nub bun
=. baked.nub (~(put by baked.nub) path res)
[res nub]
2020-04-16 23:30:55 +03:00
?: (~(has in cycle.nub) vale+path)
~|(cycle+vale+path^stack.nub !!)
=. cycle.nub (~(put in cycle.nub) vale+path)
2021-07-17 02:54:20 +03:00
::~> %slog.0^leaf/"ford: read file {(spud path)}"
?^ change=(~(get by changes) path)
=^ page nub
?: ?=(%& -.u.change)
[p.u.change nub]
~| %ugly-lobe^p.u.change^path
(lobe-to-page p.u.change)
2020-04-28 07:52:07 +03:00
=^ cage nub (validate-page path page)
[cage nub]
?< (~(has in deletes) path)
2020-06-20 10:45:53 +03:00
~| %file-not-found^path
2022-04-06 22:16:22 +03:00
:_(nub q:(need (~(get ^de ankh) path)))
2021-02-24 20:59:07 +03:00
:: +build-nave: build a statically typed mark core
::
2021-02-24 20:59:07 +03:00
++ build-nave
|= mak=mark
2021-01-13 23:20:58 +03:00
^- [vase state]
~| %error-building-mark^mak
2021-01-13 03:10:11 +03:00
?^ got=(~(get by naves.cache.nub) mak)
=? stack.nub ?=(^ stack.nub)
stack.nub(i (~(uni in i.stack.nub) dez.u.got))
[res.u.got nub]
2020-04-16 23:30:55 +03:00
?: (~(has in cycle.nub) mark+mak)
~|(cycle+mark+mak^stack.nub !!)
=. cycle.nub (~(put in cycle.nub) mark+mak)
=. stack.nub [~ stack.nub]
2021-01-13 03:10:11 +03:00
=; res=[=vase nub=state]
=. nub nub.res
=^ top stack.nub pop-stack
2021-01-13 03:10:11 +03:00
=. naves.cache.nub (~(put by naves.cache.nub) mak [vase.res top])
[vase.res nub]
2021-09-21 00:13:04 +03:00
:: ~> %slog.0^leaf/"ford: make mark {<mak>}"
=^ cor=vase nub (build-fit %mar mak)
2021-01-26 23:59:36 +03:00
=/ gad=vase (slap cor limb/%grad)
?@ q.gad
=+ !<(mok=mark gad)
2021-01-13 03:10:11 +03:00
=^ deg=vase nub $(mak mok)
2021-02-24 20:59:07 +03:00
=^ tub=vase nub (build-cast mak mok)
=^ but=vase nub (build-cast mok mak)
:_ nub
2021-01-13 03:10:11 +03:00
^- vase :: vase of nave
%+ slap
(with-faces deg+deg tub+tub but+but cor+cor nave+nave.bud ~)
2021-01-13 03:10:11 +03:00
!, *hoon
2021-01-27 06:51:52 +03:00
=/ typ _+<.cor
2021-02-22 22:35:57 +03:00
=/ dif diff:deg
2021-01-13 03:10:11 +03:00
^- (nave typ dif)
|%
++ diff
2021-01-13 03:10:11 +03:00
|= [old=typ new=typ]
^- dif
(diff:deg (tub old) (tub new))
++ form form:deg
++ join join:deg
++ mash mash:deg
++ pact
2021-01-13 03:10:11 +03:00
|= [v=typ d=dif]
^- typ
(but (pact:deg (tub v) d))
++ vale noun:grab:cor
--
:_ nub
2021-01-13 03:10:11 +03:00
^- vase :: vase of nave
%+ slap (slop (with-face cor+cor) zuse.bud)
2021-01-13 03:10:11 +03:00
!, *hoon
2021-01-26 23:59:36 +03:00
=/ typ _+<.cor
=/ dif _*diff:grad:cor
^- (nave:clay typ dif)
2021-01-13 03:10:11 +03:00
|%
++ diff |=([old=typ new=typ] (diff:~(grad cor old) new))
++ form form:grad:cor
2021-01-26 23:59:36 +03:00
++ join
|= [a=dif b=dif]
^- (unit (unit dif))
?: =(a b)
~
`(join:grad:cor a b)
2021-01-13 03:10:11 +03:00
++ mash
|= [a=[=ship =desk =dif] b=[=ship =desk =dif]]
^- (unit dif)
?: =(dif.a dif.b)
~
2021-01-27 06:51:52 +03:00
`(mash:grad:cor a b)
2021-01-13 03:10:11 +03:00
++ pact |=([v=typ d=dif] (pact:~(grad cor v) d))
++ vale noun:grab:cor
--
2021-02-24 20:59:07 +03:00
:: +build-dais: build a dynamically typed mark definition
2021-01-13 03:10:11 +03:00
::
2021-02-24 20:59:07 +03:00
++ build-dais
2021-01-13 03:10:11 +03:00
|= mak=mark
^- [dais state]
~| %error-building-dais^mak
?^ got=(~(get by marks.cache.nub) mak)
=? stack.nub ?=(^ stack.nub)
stack.nub(i (~(uni in i.stack.nub) dez.u.got))
[res.u.got nub]
?: (~(has in cycle.nub) dais+mak)
~|(cycle+dais+mak^stack.nub !!)
=. cycle.nub (~(put in cycle.nub) dais+mak)
=. stack.nub [~ stack.nub]
=; res=[=dais nub=state]
=. nub nub.res
=^ top stack.nub pop-stack
=. marks.cache.nub (~(put by marks.cache.nub) mak [dais.res top])
[dais.res nub]
2021-02-24 20:59:07 +03:00
=^ nav=vase nub (build-nave mak)
::~> %slog.0^leaf/"ford: make dais {<mak>}"
2021-01-13 03:10:11 +03:00
:_ nub
^- dais
=> [..zuse nav=nav]
|_ sam=vase
++ diff
|= new=vase
2021-01-13 03:10:11 +03:00
(slam (slap nav limb/%diff) (slop sam new))
++ form !<(mark (slap nav limb/%form))
++ join
|= [a=vase b=vase]
^- (unit (unit vase))
2021-01-13 03:10:11 +03:00
=/ res=vase (slam (slap nav limb/%join) (slop a b))
?~ q.res ~
?~ +.q.res [~ ~]
2021-01-26 23:59:36 +03:00
``(slap res !,(*hoon ?>(?=([~ ~ *] .) u.u)))
++ mash
|= [a=[=ship =desk diff=vase] b=[=ship =desk diff=vase]]
^- (unit vase)
2021-01-13 03:10:11 +03:00
=/ res=vase
%+ slam (slap nav limb/%mash)
%+ slop
:(slop =>([..zuse ship.a] !>(+)) =>([..zuse desk.a] !>(+)) diff.a)
:(slop =>([..zuse ship.b] !>(+)) =>([..zuse desk.b] !>(+)) diff.b)
2021-01-13 03:10:11 +03:00
?~ q.res
~
2021-01-26 23:59:36 +03:00
`(slap res !,(*hoon ?>((^ .) u)))
++ pact
|= diff=vase
2021-01-13 03:10:11 +03:00
(slam (slap nav limb/%pact) (slop sam diff))
++ vale
2021-03-06 00:53:35 +03:00
|: noun=q:(slap nav !,(*hoon *vale))
2021-01-13 03:10:11 +03:00
(slam (slap nav limb/%vale) noun/noun)
--
2021-02-24 20:59:07 +03:00
:: +build-cast: produce gate to convert mark .a to, statically typed
::
2021-02-24 20:59:07 +03:00
++ build-cast
|= [a=mark b=mark]
2021-01-13 03:10:11 +03:00
^- [vase state]
~| error-building-cast+[a b]
2020-04-28 07:52:07 +03:00
?: =([%mime %hoon] [a b])
:_(nub =>(..zuse !>(|=(m=mime q.q.m))))
?^ got=(~(get by casts.cache.nub) [a b])
=? stack.nub ?=(^ stack.nub)
stack.nub(i (~(uni in i.stack.nub) dez.u.got))
[res.u.got nub]
2020-04-16 23:30:55 +03:00
?: (~(has in cycle.nub) cast+[a b])
~|(cycle+cast+[a b]^stack.nub !!)
=. stack.nub [~ stack.nub]
2021-01-13 03:10:11 +03:00
=; res=[=vase nub=state]
=. nub nub.res
=^ top stack.nub pop-stack
2021-01-13 03:10:11 +03:00
=. casts.cache.nub (~(put by casts.cache.nub) [a b] [vase.res top])
[vase.res nub]
:: try +grow; is there a +grow core with a .b arm?
::
2021-09-21 00:13:04 +03:00
:: ~> %slog.0^leaf/"ford: make cast {<a>} -> {<b>}"
=^ old=vase nub (build-fit %mar a)
?: =/ ram (mule |.((slap old !,(*hoon grow))))
?: ?=(%| -.ram) %.n
2020-05-06 08:09:35 +03:00
=/ lab (mule |.((slob b p.p.ram)))
?: ?=(%| -.lab) %.n
p.lab
:: +grow core has .b arm; use that
::
:_ nub
2021-01-13 03:10:11 +03:00
%+ slap (with-faces cor+old ~)
^- hoon
2021-01-27 04:43:23 +03:00
:+ %brcl !,(*hoon v=+<.cor)
2021-01-13 03:10:11 +03:00
:+ %tsgl limb/b
!,(*hoon ~(grow cor v))
:: try direct +grab
::
=^ new=vase nub (build-fit %mar b)
2021-01-13 03:10:11 +03:00
=/ rab (mule |.((slap new tsgl/[limb/a limb/%grab])))
?: &(?=(%& -.rab) ?=(^ q.p.rab))
2021-01-13 03:10:11 +03:00
:_(nub p.rab)
:: try +jump
::
2021-01-13 03:10:11 +03:00
=/ jum (mule |.((slap old tsgl/[limb/b limb/%jump])))
?: ?=(%& -.jum)
(compose-casts a !<(mark p.jum) b)
?: ?=(%& -.rab)
(compose-casts a !<(mark p.rab) b)
?: ?=(%noun b)
:_(nub same.bud)
~|(no-cast-from+[a b] !!)
::
++ compose-casts
|= [x=mark y=mark z=mark]
2021-01-13 03:10:11 +03:00
^- [vase state]
2021-02-24 20:59:07 +03:00
=^ uno=vase nub (build-cast x y)
=^ dos=vase nub (build-cast y z)
2021-01-13 03:10:11 +03:00
:_ nub
2021-01-13 23:20:58 +03:00
%+ slap
(with-faces uno+uno dos+dos cork+=>([..zuse cork] !>(+)) ~)
2021-01-13 03:10:11 +03:00
!,(*hoon (cork uno dos))
2021-02-24 20:59:07 +03:00
:: +build-tube: produce a $tube mark conversion gate from .a to .b
2021-01-13 03:10:11 +03:00
::
2021-02-24 20:59:07 +03:00
++ build-tube
2021-01-13 03:10:11 +03:00
|= [a=mark b=mark]
^- [tube state]
2021-01-13 03:10:11 +03:00
~| error-building-tube+[a b]
?^ got=(~(get by tubes.cache.nub) [a b])
=? stack.nub ?=(^ stack.nub)
stack.nub(i (~(uni in i.stack.nub) dez.u.got))
[res.u.got nub]
?: (~(has in cycle.nub) tube+[a b])
~|(cycle+tube+[a b]^stack.nub !!)
=. stack.nub [~ stack.nub]
=; res=[=tube nub=state]
=. nub nub.res
=^ top stack.nub pop-stack
=. tubes.cache.nub (~(put by tubes.cache.nub) [a b] [tube.res top])
[tube.res nub]
2021-02-24 20:59:07 +03:00
=^ gat=vase nub (build-cast a b)
2021-09-21 00:13:04 +03:00
:: ~> %slog.0^leaf/"ford: make tube {<a>} -> {<b>}"
:_(nub =>([..zuse gat=gat] |=(v=vase (slam gat v))))
::
++ lobe-to-page
|= =lobe
^- [page state]
=/ =blob (~(got by file-store) lobe)
|- ^- [page state]
?- -.blob
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
%dead ~|(lobe-to-page-dead+lobe !!)
%direct [q.blob nub]
%delta
=/ [=parent=^lobe diff=page] [q r]:blob
=^ parent-page nub $(blob (~(got by file-store) parent-lobe))
=^ =cage nub (run-pact parent-page diff)
[[p q.q]:cage nub]
==
::
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
++ lobe-to-unit-page
|= =lobe
^- [(unit page) state]
=/ =blob (~(got by file-store) lobe)
|- ^- [(unit page) state]
?- -.blob
%dead [~ nub]
%direct [`q.blob nub]
%delta
=/ [=parent=^lobe diff=page] [q r]:blob
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
=^ parent-page nub $(blob (~(got by file-store) parent-lobe))
?~ parent-page
[~ nub]
=^ =cage nub (run-pact u.parent-page diff)
[`[p q.q]:cage nub]
==
::
2020-04-28 07:52:07 +03:00
++ validate-page
|= [=path =page]
^- [cage state]
2020-04-28 07:52:07 +03:00
~| validate-page-fail+path^from+p.page
=/ mak=mark (head (flop path))
?: =(mak p.page)
(page-to-cage page)
=^ [mark vax=vase] nub (page-to-cage page)
2021-02-24 20:59:07 +03:00
=^ =tube nub (build-tube p.page mak)
:_(nub [mak (tube vax)])
::
++ page-to-cage
|= =page
^- [cage state]
?: =(%hoon p.page)
:_(nub [%hoon -:!>(*@t) q.page])
2020-04-28 07:52:07 +03:00
?: =(%mime p.page)
:_(nub [%mime =>([..zuse ;;(mime q.page)] !>(+))])
2021-02-24 20:59:07 +03:00
=^ =dais nub (build-dais p.page)
:_(nub [p.page (vale:dais q.page)])
::
++ cast-path
|= [=path mak=mark]
^- [cage state]
=/ mok (head (flop path))
2020-04-18 05:50:03 +03:00
~| error-casting-path+[path mok mak]
2021-02-24 20:59:07 +03:00
=^ cag=cage nub (read-file path)
?: =(mok mak)
[cag nub]
2021-02-24 20:59:07 +03:00
=^ =tube nub (build-tube mok mak)
2020-04-18 05:50:03 +03:00
~| error-running-cast+[path mok mak]
:_(nub [mak (tube q.cag)])
::
++ run-pact
|= [old=page diff=page]
^- [cage state]
2020-06-25 06:26:04 +03:00
?: ?=(%hoon p.old)
=/ txt=wain (to-wain:format ;;(@t q.old))
=+ ;;(dif=(urge cord) q.diff)
=/ new=@t (of-wain:format (lurk:differ txt dif))
:_(nub [%hoon =>([..zuse new] !>(+))])
2021-02-24 20:59:07 +03:00
=^ dys=dais nub (build-dais p.old)
=^ syd=dais nub (build-dais p.diff)
2020-04-16 16:38:34 +03:00
:_(nub [p.old (~(pact dys (vale:dys q.old)) (vale:syd q.diff))])
::
2020-05-14 09:26:52 +03:00
++ prelude
|= =path
^- vase
2021-02-24 20:59:07 +03:00
=^ cag=cage nub (read-file path)
2020-05-14 09:26:52 +03:00
?> =(%hoon p.cag)
=/ tex=tape (trip !<(@t q.cag))
=/ =pile (parse-pile path tex)
=. hoon.pile !,(*hoon .)
=^ res=vase nub (run-pile pile)
res
::
++ build-dependency
|= dep=(each [dir=path fil=path] path)
^- [vase state]
=/ =path
?:(?=(%| -.dep) p.dep fil.p.dep)
2020-04-19 02:53:45 +03:00
~| %error-building^path
2021-02-24 20:59:07 +03:00
?^ got=(~(get by files.cache.nub) path)
=? stack.nub ?=(^ stack.nub)
stack.nub(i (~(uni in i.stack.nub) dez.u.got))
[res.u.got nub]
2020-04-16 23:30:55 +03:00
?: (~(has in cycle.nub) file+path)
~|(cycle+file+path^stack.nub !!)
=. cycle.nub (~(put in cycle.nub) file+path)
=. stack.nub
=- [(sy - ~) stack.nub]
?:(?=(%| -.dep) dep [& dir.p.dep])
2021-09-21 00:13:04 +03:00
:: ~> %slog.0^leaf/"ford: make file {(spud path)}"
2021-02-24 20:59:07 +03:00
=^ cag=cage nub (read-file path)
?> =(%hoon p.cag)
=/ tex=tape (trip !<(@t q.cag))
=/ =pile (parse-pile path tex)
2020-05-14 09:26:52 +03:00
=^ res=vase nub (run-pile pile)
=^ top stack.nub pop-stack
2021-02-24 20:59:07 +03:00
=. files.cache.nub (~(put by files.cache.nub) path [res top])
2020-05-14 09:26:52 +03:00
[res nub]
::
++ build-file
|= =path
(build-dependency |+path)
:: +build-directory: builds files in top level of a directory
::
:: this excludes files directly at /path/hoon,
:: instead only including files in the unix-style directory at /path,
:: such as /path/file/hoon, but not /path/more/file/hoon.
2020-05-14 09:26:52 +03:00
::
++ build-directory
|= =path
^- [(map @ta vase) state]
=/ fiz=(list @ta)
=/ nuk=_ankh (~(dip ^de ankh) path)
%+ murn
~(tap by dir.nuk)
|= [nom=@ta nak=_ankh]
?. ?=([~ [~ *] *] (~(get by dir.nak) %hoon)) ~
`nom
::
=| rez=(map @ta vase)
|-
?~ fiz
[rez nub]
=* nom=@ta i.fiz
=/ pax=^path (weld path nom %hoon ~)
=^ res nub (build-dependency &+[path pax])
$(fiz t.fiz, rez (~(put by rez) nom res))
::
2020-05-14 09:26:52 +03:00
++ run-pile
|= =pile
=/ sut=vase zuse.bud
2021-06-29 03:01:51 +03:00
=^ sut=vase nub (run-tauts sut %sur sur.pile)
2020-04-19 02:53:45 +03:00
=^ sut=vase nub (run-tauts sut %lib lib.pile)
=^ sut=vase nub (run-raw sut raw.pile)
=^ sut=vase nub (run-raz sut raz.pile)
=^ sut=vase nub (run-maz sut maz.pile)
=^ sut=vase nub (run-caz sut caz.pile)
2020-04-19 02:53:45 +03:00
=^ sut=vase nub (run-bar sut bar.pile)
2020-06-19 07:44:17 +03:00
=/ res=vase (road |.((slap sut hoon.pile)))
[res nub]
::
++ parse-pile
2020-04-19 02:53:45 +03:00
|= [pax=path tex=tape]
^- pile
2020-04-19 02:53:45 +03:00
=/ [=hair res=(unit [=pile =nail])] ((pile-rule pax) [1 1] tex)
?^ res pile.u.res
2020-06-29 18:01:23 +03:00
%- mean %- flop
2020-04-16 16:55:48 +03:00
=/ lyn p.hair
=/ col q.hair
2020-04-19 02:53:45 +03:00
:~ leaf+"syntax error at [{<lyn>} {<col>}] in {<pax>}"
2020-04-16 16:55:48 +03:00
leaf+(trip (snag (dec lyn) (to-wain:format (crip tex))))
leaf+(runt [(dec col) '-'] "^")
==
::
2020-04-19 02:53:45 +03:00
++ pile-rule
|= pax=path
%- full
%+ ifix
:_ gay
2020-04-16 15:56:06 +03:00
:: parse optional /? and ignore
::
;~(plug gay (punt ;~(plug fas wut gap dem gap)))
|^
;~ plug
%+ cook (bake zing (list (list taut)))
%+ rune hep
(most ;~(plug com gaw) taut-rule)
::
%+ cook (bake zing (list (list taut)))
%+ rune lus
(most ;~(plug com gaw) taut-rule)
::
%+ rune tis
;~(plug sym ;~(pfix gap stap))
::
%+ rune sig
;~((glue gap) sym wyde:vast stap)
::
%+ rune cen
;~(plug sym ;~(pfix gap ;~(pfix cen sym)))
::
%+ rune buc
;~ (glue gap)
sym
;~(pfix cen sym)
;~(pfix cen sym)
==
::
%+ rune tar
;~ (glue gap)
sym
;~(pfix cen sym)
;~(pfix stap)
2020-04-19 02:53:45 +03:00
==
::
%+ stag %tssg
(most gap tall:(vang & pax))
2020-04-19 02:53:45 +03:00
==
::
++ pant
|* fel=^rule
;~(pose fel (easy ~))
::
++ mast
|* [bus=^rule fel=^rule]
;~(sfix (more bus fel) bus)
::
++ rune
|* [bus=^rule fel=^rule]
%- pant
%+ mast gap
;~(pfix fas bus gap fel)
--
::
++ taut-rule
%+ cook |=(taut +<)
;~ pose
(stag ~ ;~(pfix tar sym))
;~(plug (stag ~ sym) ;~(pfix tis sym))
(cook |=(a=term [`a a]) sym)
==
::
++ run-tauts
|= [sut=vase wer=?(%lib %sur) taz=(list taut)]
^- [vase state]
?~ taz [sut nub]
=^ pin=vase nub (build-fit wer pax.i.taz)
=? p.pin ?=(^ face.i.taz) [%face u.face.i.taz p.pin]
$(sut (slop pin sut), taz t.taz)
::
++ run-raw
|= [sut=vase raw=(list [face=term =path])]
^- [vase state]
?~ raw [sut nub]
2020-04-19 02:53:45 +03:00
=^ pin=vase nub (build-file (snoc path.i.raw %hoon))
=. p.pin [%face face.i.raw p.pin]
$(sut (slop pin sut), raw t.raw)
::
++ run-raz
|= [sut=vase raz=(list [face=term =spec =path])]
^- [vase state]
?~ raz [sut nub]
=^ res=(map @ta vase) nub
(build-directory path.i.raz)
=; pin=vase
=. p.pin [%face face.i.raz p.pin]
$(sut (slop pin sut), raz t.raz)
::
=/ =type (~(play ut p.sut) [%kttr spec.i.raz])
:: ensure results nest in the specified type,
:: and produce a homogenous map containing that type.
::
:- %- ~(play ut p.sut)
[%kttr %make [%wing ~[%map]] ~[[%base %atom %ta] spec.i.raz]]
|-
?~ res ~
?. (~(nest ut type) | p.q.n.res)
~| [%nest-fail path.i.raz p.n.res]
!!
:- [p.n.res q.q.n.res]
[$(res l.res) $(res r.res)]
::
++ run-maz
|= [sut=vase maz=(list [face=term =mark])]
^- [vase state]
?~ maz [sut nub]
2021-02-24 20:59:07 +03:00
=^ pin=vase nub (build-nave mark.i.maz)
=. p.pin [%face face.i.maz p.pin]
$(sut (slop pin sut), maz t.maz)
::
++ run-caz
|= [sut=vase caz=(list [face=term =mars])]
^- [vase state]
?~ caz [sut nub]
2021-02-24 20:59:07 +03:00
=^ pin=vase nub (build-cast mars.i.caz)
=. p.pin [%face face.i.caz p.pin]
$(sut (slop pin sut), caz t.caz)
::
2020-04-19 02:53:45 +03:00
++ run-bar
|= [sut=vase bar=(list [face=term =mark =path])]
^- [vase state]
?~ bar [sut nub]
=^ =cage nub (cast-path [path mark]:i.bar)
=. p.q.cage [%face face.i.bar p.q.cage]
$(sut (slop q.cage sut), bar t.bar)
::
:: +build-fit: build file at path, maybe converting '-'s to '/'s in path
::
++ build-fit
|= [pre=@tas pax=@tas]
^- [vase state]
2020-04-28 03:39:18 +03:00
(build-file (fit-path pre pax))
::
:: +fit-path: find path, maybe converting '-'s to '/'s
::
:: Try '-' before '/', applied left-to-right through the path,
:: e.g. 'a-foo/bar' takes precedence over 'a/foo-bar'.
::
++ fit-path
|= [pre=@tas pax=@tas]
^- path
=/ paz (segments pax)
|- ^- path
2021-07-19 14:01:34 +03:00
?~ paz ~_(leaf/"clay: no files match /{(trip pre)}/{(trip pax)}/hoon" !!)
2020-04-28 03:39:18 +03:00
=/ pux=path pre^(snoc i.paz %hoon)
?: (~(has in deletes) pux)
$(paz t.paz)
?: (~(has by changes) pux)
pux
2022-04-06 22:16:22 +03:00
?^ (~(get ^de ankh) pux)
2020-04-28 03:39:18 +03:00
pux
$(paz t.paz)
--
--
2016-11-24 07:25:07 +03:00
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: section 4cA, filesystem logic
::
:: This core contains the main logic of clay. Besides `++ze`, this directly
:: contains the logic for commiting new revisions (local urbits), managing
:: and notifying subscribers (reactivity), and pulling and validating content
:: (remote urbits).
::
:: The state includes:
::
2018-12-13 10:42:15 +03:00
:: -- local urbit `our`
2016-11-24 07:25:07 +03:00
:: -- current time `now`
:: -- current duct `hen`
2019-02-02 00:46:09 +03:00
:: -- scry handler `ski`
2018-12-13 10:42:15 +03:00
:: -- all vane state `++raft` (rarely used, except for the object store)
2016-11-24 07:25:07 +03:00
:: -- target urbit `her`
:: -- target desk `syd`
::
:: For local desks, `our` == `her` is one of the urbits on our pier. For
:: foreign desks, `her` is the urbit the desk is on and `our` is the local
:: urbit that's managing the relationship with the foreign urbit. Don't mix
:: up those two, or there will be wailing and gnashing of teeth.
::
2018-12-13 10:42:15 +03:00
:: While setting up `++de`, we check if `our` == `her`. If so, we get
:: the desk information from `dos.rom`. Otherwise, we get the rung from
:: `hoy` and get the desk information from `rus` in there. In either case,
:: we normalize the desk information to a `++rede`, which is all the
:: desk-specific data that we utilize in `++de`. Because it's effectively
:: a part of the `++de` state, let's look at what we've got:
2016-11-24 07:25:07 +03:00
::
:: -- `lim` is the most recent date we're confident we have all the
:: information for. For local desks, this is always `now`. For foreign
:: desks, this is the last time we got a full update from the foreign
:: urbit.
:: -- `ref` is a possible request manager. For local desks, this is null.
:: For foreign desks, this keeps track of all pending foreign requests
:: plus a cache of the responses to previous requests.
:: -- `qyx` is the set of subscriptions, with listening ducts. These
:: subscriptions exist only until they've been filled.
:: -- `dom` is the actual state of the filetree. Since this is used almost
:: exclusively in `++ze`, we describe it there.
::
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
++ de :: per desk
2020-12-06 11:38:37 +03:00
|= [now=@da rof=roof hen=duct raft]
2018-12-13 10:42:15 +03:00
|= [her=ship syd=desk]
:: NB: ruf=raft crashes in the compiler
2018-12-13 10:42:15 +03:00
::
2020-12-06 11:38:37 +03:00
=* ruf |3.+6.^$
2018-12-13 10:42:15 +03:00
::
=/ [mow=(list move) hun=(unit duct) rede]
2018-12-13 10:42:15 +03:00
?. =(our her)
:: no duct, foreign +rede or default
::
2019-07-27 09:36:59 +03:00
:+ ?: (~(has by hoy.ruf) her)
~
2019-08-07 01:42:37 +03:00
[hun.rom.ruf %pass /sinks %j %public-keys (silt her ~)]~
2019-07-27 09:36:59 +03:00
~
=/ rus rus:(~(gut by hoy.ruf) her *rung)
%+ ~(gut by rus) syd
2021-04-24 00:33:49 +03:00
[lim=~2000.1.1 ref=`*rind qyx=~ dom=*dome per=~ pew=~ fiz=*melt]
2018-12-13 10:42:15 +03:00
:: administrative duct, domestic +rede
::
2019-07-27 09:36:59 +03:00
:+ ~ `hun.rom.ruf
=/ jod (~(gut by dos.rom.ruf) syd *dojo)
2021-04-20 06:46:46 +03:00
[lim=now ref=~ [qyx dom per pew fiz]:jod]
2018-12-13 10:42:15 +03:00
::
2019-07-27 09:36:59 +03:00
=* red=rede ->+
2016-11-24 07:25:07 +03:00
|%
++ abet :: resolve
2018-12-13 10:42:15 +03:00
^- [(list move) raft]
:- (flop mow)
?. =(our her)
:: save foreign +rede
::
=/ run (~(gut by hoy.ruf) her *rung)
2019-02-02 00:46:09 +03:00
=/ rug (~(put by rus.run) syd red)
ruf(hoy (~(put by hoy.ruf) her run(rus rug)))
2018-12-13 10:42:15 +03:00
:: save domestic +room
::
2019-05-03 04:50:20 +03:00
%= ruf
hun.rom (need hun)
2021-04-20 06:46:46 +03:00
dos.rom (~(put by dos.rom.ruf) syd [qyx dom per pew fiz]:red)
2019-05-03 04:50:20 +03:00
==
2016-11-24 07:25:07 +03:00
::
2019-05-03 04:50:20 +03:00
:: Handle `%sing` requests
2016-11-24 07:25:07 +03:00
::
2019-05-03 04:50:20 +03:00
++ aver
|= [for=(unit ship) mun=mood]
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
^- [(unit (unit cage)) ford-cache]
2019-05-03 04:50:20 +03:00
=+ ezy=?~(ref ~ (~(get by haw.u.ref) mun))
?^ ezy
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
[`u.ezy fod.dom.red]
2020-07-21 22:03:30 +03:00
?: ?=([%s [%ud *] %late *] mun)
:_ fod.dom.red
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
^- (unit (unit cage))
:+ ~ ~
2020-07-21 22:03:30 +03:00
^- cage
:- %cass
?~ let.dom
!>([0 *@da])
!>([let.dom t:(~(got by hut.ran) (~(got by hit.dom) let.dom))])
2019-05-23 00:39:12 +03:00
=+ nao=(case-to-aeon case.mun)
?~(nao [~ fod.dom.red] (read-at-aeon:ze for u.nao mun))
2016-11-24 07:25:07 +03:00
::
2019-05-03 04:50:20 +03:00
:: Queue a move.
::
2019-05-03 04:50:20 +03:00
++ emit
|= mof=move
2019-05-03 04:50:20 +03:00
%_(+> mow [mof mow])
2016-11-24 07:25:07 +03:00
::
2019-05-03 04:50:20 +03:00
:: Queue a list of moves
2016-11-24 07:25:07 +03:00
::
2019-05-03 04:50:20 +03:00
++ emil
|= mof=(list move)
2019-05-14 09:18:52 +03:00
%_(+> mow (weld (flop mof) mow))
2016-11-24 07:25:07 +03:00
::
:: Queue a list of moves, to be emitted before the rest
::
++ lime
|= mof=(list move)
%_(+> mow (weld mow (flop mof)))
::
2019-05-03 04:50:20 +03:00
:: Set timer.
2016-11-24 07:25:07 +03:00
::
2019-05-03 04:50:20 +03:00
++ bait
|= [hen=duct tym=@da]
(emit hen %pass /tyme/(scot %p her)/[syd] %b %wait tym)
2016-11-24 07:25:07 +03:00
::
2019-05-03 04:50:20 +03:00
:: Cancel timer.
2019-05-02 04:21:32 +03:00
::
2019-05-03 04:50:20 +03:00
++ best
|= [hen=duct tym=@da]
(emit hen %pass /tyme/(scot %p her)/[syd] %b %rest tym)
2019-05-03 04:50:20 +03:00
::
:: Give %writ, or slip a drip if foreign desk
2019-05-03 04:50:20 +03:00
::
++ writ
|= res=(unit [=mood =cage])
^- card
=/ =riot
?~ res
~
`[[care.mood case.mood syd] path.mood cage]:[u.res syd=syd]
?~ ref
[%give %writ riot]
[%slip %b %drip !>([%writ riot])]
2019-05-02 04:21:32 +03:00
::
2020-05-12 10:37:04 +03:00
++ case-to-date
|= =case
^- @da
:: if the case is already a date, use it.
::
?: ?=([%da *] case)
p.case
:: translate other cases to dates
::
=/ aey (case-to-aeon-before lim case)
?~ aey `@da`0
?: =(0 u.aey) `@da`0
t:(aeon-to-yaki:ze u.aey)
::
++ case-to-aeon (cury case-to-aeon-before lim)
::
:: Reduce a case to an aeon (version number)
::
:: We produce null if we can't yet reduce the case for whatever
:: resaon (usually either the time or aeon hasn't happened yet or
:: the label hasn't been created).
::
++ case-to-aeon-before
|= [lim=@da lok=case]
^- (unit aeon)
?- -.lok
%da
2020-05-12 10:37:04 +03:00
?: (gth p.lok lim) ~
|- ^- (unit aeon)
?: =(0 let.dom) [~ 0] :: avoid underflow
?: %+ gte p.lok
=< t
~| [%letdom let=let.dom hit=hit.dom hut=~(key by hut.ran)]
~| [%getdom (~(get by hit.dom) let.dom)]
%- aeon-to-yaki:ze
let.dom
[~ let.dom]
$(let.dom (dec let.dom))
::
%tas (~(get by lab.dom) p.lok)
%ud ?:((gth p.lok let.dom) ~ [~ p.lok])
2020-05-12 10:37:04 +03:00
==
2019-05-02 04:21:32 +03:00
::
2021-06-29 03:01:51 +03:00
++ static-ford-args [ank.dom ~ ~ lat.ran fod.dom]
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
:: Create a ford appropriate for the aeon
::
:: Don't forget to call +aeon-ford-cache!
::
++ aeon-ford
|= yon=aeon
%- ford:fusion
?: =(let.dom yon)
static-ford-args
=/ changes (~(run by q:(aeon-to-yaki:ze yon)) |=(=lobe |+lobe))
[*ankh ~ changes lat.ran *ford-cache]
:: Produce ford cache appropriate for the aeon
::
++ aeon-ford-cache
|* [yon=aeon res=* fod=ford-cache]
:- res
?: =(let.dom yon)
fod
fod.dom
2019-05-03 04:50:20 +03:00
:: Transfer a request to another ship's clay.
::
++ send-over-ames
2019-06-29 02:43:52 +03:00
|= [=duct =ship index=@ud =riff]
^+ +>
::
=/ =desk p.riff
=/ =wire /warp-index/(scot %p ship)/(scot %tas desk)/(scot %ud index)
=/ =path [%question desk (scot %ud index) ~]
(emit duct %pass wire %a %plea ship %c path `riff-any`[%1 riff])
2019-05-03 04:50:20 +03:00
::
++ foreign-capable
|= =rave
|^
?- -.rave
%many &
%sing (good-care care.mood.rave)
%next (good-care care.mood.rave)
%mult
%- ~(all in paths.mool.rave)
|= [=care =path]
(good-care care)
==
::
++ good-care
|= =care
(~(has in ^~((silt `(list ^care)`~[%u %w %x %y %z]))) care)
--
::
2019-05-03 04:50:20 +03:00
:: Create a request that cannot be filled immediately.
::
:: If it's a local request, we just put in in `qyx`, setting a timer if it's
:: waiting for a particular time. If it's a foreign request, we add it to
:: our request manager (ref, which is a ++rind) and make the request to the
:: foreign ship.
::
++ duce :: produce request
2019-05-13 23:53:46 +03:00
|= wov=wove
2019-05-03 04:50:20 +03:00
^+ +>
=. wov (dedupe wov)
=. qyx (~(put ju qyx) wov hen)
?~ ref
2019-05-23 00:39:12 +03:00
(run-if-future rove.wov |=(@da (bait hen +<)))
2019-05-14 08:20:46 +03:00
|- ^+ +>+.$
2019-05-23 00:39:12 +03:00
=/ =rave (rove-to-rave rove.wov)
=. rave
?. ?=([%sing %v *] rave) rave
[%many %| [%ud let.dom] case.mood.rave path.mood.rave]
::
?. (foreign-capable rave)
~|([%clay-bad-foreign-request-care rave] !!)
::
2019-05-03 04:50:20 +03:00
=+ inx=nix.u.ref
2019-05-14 08:20:46 +03:00
=. +>+.$
2019-05-03 04:50:20 +03:00
=< ?>(?=(^ ref) .)
2019-06-29 02:43:52 +03:00
(send-over-ames hen her inx syd `rave)
2019-05-14 08:20:46 +03:00
%= +>+.$
2019-05-03 04:50:20 +03:00
nix.u.ref +(nix.u.ref)
bom.u.ref (~(put by bom.u.ref) inx [hen rave ~ [~ ~] ~ |])
2019-05-03 04:50:20 +03:00
fod.u.ref (~(put by fod.u.ref) hen inx)
==
::
:: If a similar request exists, switch to the existing request.
::
:: "Similar" requests are those %next and %many requests which are the same
:: up to starting case, but we're already after the starting case. This
:: stacks later requests for something onto the same request so that they
:: all get filled at once.
::
++ dedupe :: find existing alias
|= wov=wove
2019-05-03 04:50:20 +03:00
^- wove
=; won=(unit wove) (fall won wov)
2019-05-23 00:39:12 +03:00
=* rov rove.wov
2019-05-03 04:50:20 +03:00
?- -.rov
%sing ~
%next
2019-05-23 00:39:12 +03:00
=+ aey=(case-to-aeon case.mood.rov)
2019-05-03 04:50:20 +03:00
?~ aey ~
%- ~(rep in ~(key by qyx))
|= [haw=wove res=(unit wove)]
2019-05-03 04:50:20 +03:00
?^ res res
2019-05-23 00:39:12 +03:00
?. =(for.wov for.haw) ~
=* hav rove.haw
2019-05-03 04:50:20 +03:00
=- ?:(- `haw ~)
?& ?=(%next -.hav)
2019-05-23 00:39:12 +03:00
=(mood.hav mood.rov(case case.mood.hav))
2019-05-03 04:50:20 +03:00
::
:: only a match if this request is before
:: or at our starting case.
2019-05-23 00:39:12 +03:00
=+ hay=(case-to-aeon case.mood.hav)
2019-05-03 04:50:20 +03:00
?~(hay | (lte u.hay u.aey))
2019-05-02 04:21:32 +03:00
==
2019-05-03 04:50:20 +03:00
::
%mult
2019-05-23 00:39:12 +03:00
=+ aey=(case-to-aeon case.mool.rov)
2019-05-03 04:50:20 +03:00
?~ aey ~
%- ~(rep in ~(key by qyx))
|= [haw=wove res=(unit wove)]
2019-05-03 04:50:20 +03:00
?^ res res
2019-05-23 00:39:12 +03:00
?. =(for.wov for.haw) ~
=* hav rove.haw
2019-05-03 04:50:20 +03:00
=- ?:(- `haw ~)
?& ?=(%mult -.hav)
2019-05-23 00:39:12 +03:00
=(mool.hav mool.rov(case case.mool.hav))
2019-05-03 04:50:20 +03:00
::
:: only a match if this request is before
:: or at our starting case, and it has been
:: tested at least that far.
2019-05-23 00:39:12 +03:00
=+ hay=(case-to-aeon case.mool.hav)
2019-05-03 04:50:20 +03:00
?& ?=(^ hay)
(lte u.hay u.aey)
2019-05-23 00:39:12 +03:00
?=(^ aeon.hav)
(gte u.aeon.hav u.aey)
2019-05-03 04:50:20 +03:00
==
2019-05-02 04:21:32 +03:00
==
2019-05-03 04:50:20 +03:00
::
%many
2019-05-23 00:39:12 +03:00
=+ aey=(case-to-aeon from.moat.rov)
2019-05-03 04:50:20 +03:00
?~ aey ~
%- ~(rep in ~(key by qyx))
|= [haw=wove res=(unit wove)]
2019-05-03 04:50:20 +03:00
?^ res res
2019-05-23 00:39:12 +03:00
?. =(for.wov for.haw) ~
=* hav rove.haw
2019-05-03 04:50:20 +03:00
=- ?:(- `haw ~)
?& ?=(%many -.hav)
2019-05-23 00:39:12 +03:00
=(hav rov(from.moat from.moat.hav))
2019-05-03 04:50:20 +03:00
::
:: only a match if this request is before
:: or at our starting case.
2019-05-23 00:39:12 +03:00
=+ hay=(case-to-aeon from.moat.hav)
2019-05-03 04:50:20 +03:00
?~(hay | (lte u.hay u.aey))
==
==
2016-11-24 07:25:07 +03:00
::
2022-04-07 09:16:26 +03:00
++ set-norm
|= =norm
=. nor.dom norm
..park
::
2021-11-15 17:07:24 +03:00
:: Attach label to aeon
::
++ label
|= [bel=@tas aey=(unit aeon)]
^+ ..park
=/ yon ?~(aey let.dom u.aey)
=/ yen (~(get by lab.dom) bel) :: existing aeon?
:: no existing aeon is bound to this label
::
?~ yen
=. lab.dom (~(put by lab.dom) bel yon)
..park
2022-03-26 02:41:54 +03:00
:: an aeon is bound to this label,
2021-11-15 17:07:24 +03:00
:: but it is the same as the existing one, so we no-op
::
?: =(u.yen yon)
~& "clay: tried to rebind existing label {<bel>} to equivalent aeon {<yon>}"
..park
2021-11-15 17:07:24 +03:00
:: an existing aeon bound to the label
:: that is distinct from the requested one.
:: rewriting would violate referential transparency
::
~| %tried-to-rewrite-existing-label
~| "requested aeon: {<yon>}, existing aeon: {<u.yen>}"
!!
::
2020-04-24 03:12:23 +03:00
:: Porcelain commit
::
++ info
|= [deletes=(set path) changes=(map path cage)]
2020-04-28 12:56:28 +03:00
^+ ..park
?: =(0 let.dom)
?> ?=(~ deletes)
=/ data=(map path (each page lobe))
(~(run by changes) |=(=cage &+[p q.q]:cage))
2020-04-30 03:38:24 +03:00
(park | &+[~ data] *rang)
::
2020-04-24 03:12:23 +03:00
=/ parent-tako=tako (aeon-to-tako:ze let.dom)
=/ data=(map path (each page lobe))
=/ parent-yaki (tako-to-yaki:ze parent-tako)
=/ after-deletes
%- ~(dif by q.parent-yaki)
(malt (turn ~(tap in deletes) |=(=path [path *lobe])))
=/ after=(map path (each page lobe))
(~(run by after-deletes) |=(=lobe |+lobe))
%- ~(uni by after)
^- (map path (each page lobe))
(~(run by changes) |=(=cage &+[p q.q]:cage))
::
=/ =yuki [~[parent-tako] data]
2020-04-30 03:38:24 +03:00
(park | &+yuki *rang)
2020-04-24 03:12:23 +03:00
::
2020-04-28 12:56:28 +03:00
:: Unix commit
::
++ into
|= [pax=path all=? mod=(list [pax=path mim=(unit mime)])]
^+ ..park
:: filter out unchanged, cached %mime values
::
=. mod
%+ skip mod
|= [pax=path mim=(unit mime)]
?~ mim
|
?~ mum=(~(get by mim.dom) pax)
|
:: TODO: check mimetype
::
=(q.u.mim q.u.mum)
=/ =yaki
?: =(0 let.dom)
*yaki
(~(got by hut.ran) (~(got by hit.dom) let.dom))
(info (mode-to-commit q.yaki pax all mod))
::
:: Plumbing commit
::
:: Guaranteed to finish in one event.
::
:: TODO: needs to check that head is ancestor of tako
:: TODO: needs to check tako in rang
:: TODO: needs to check that commit doesn't have same date
::
++ park
2020-04-30 00:50:30 +03:00
=/ check-sane |
|^
2020-04-30 03:38:24 +03:00
|= [updated=? =yoki =rang]
^+ ..park
=: hut.ran (~(uni by hut.rang) hut.ran)
2022-03-26 01:32:38 +03:00
lat.ran (uni-blobs lat.ran lat.rang)
==
2020-04-30 03:38:24 +03:00
=/ new-data=(map path (each page lobe))
?- -.yoki
%& q.p.yoki
%| (~(run by q.p.yoki) |=(=lobe |+lobe))
==
2021-06-29 04:22:55 +03:00
:: find desk kelvin
::
=/ kel=weft (get-kelvin yoki)
2021-06-29 04:22:55 +03:00
?. |(=(%base syd) =(kel [%zuse zuse]))
~>(%mean.|.(leaf/"clay: bad-kelvin, {<[need=zuse/zuse have=kel]>}") !!)
::
=/ old-yaki
2020-05-08 13:03:29 +03:00
?: =(0 let.dom)
*yaki
(aeon-to-yaki:ze let.dom)
=/ [deletes=(set path) changes=(map path (each page lobe))]
(get-changes q.old-yaki new-data)
~| [from=let.dom deletes=deletes changes=~(key by changes)]
::
:: promote ford cache
:: promote and fill in ankh
:: promote and fill in mime cache
::
2021-07-17 02:54:20 +03:00
=/ invalid (~(uni in deletes) ~(key by changes))
?: &(=(%base syd) !updated (~(any in invalid) is-kernel-path))
(sys-update yoki new-data)
2021-07-17 02:54:20 +03:00
::
2021-09-21 00:13:04 +03:00
~? (did-kernel-update invalid) %clay-kernel-updated
2021-07-17 02:54:20 +03:00
=? updated updated (did-kernel-update invalid)
=> ?. updated .
2021-09-21 00:13:04 +03:00
~>(%slog.0^leaf/"clay: rebuilding {<syd>} after kernel update" .)
2020-05-08 13:03:29 +03:00
:: clear caches if zuse reloaded
::
=. fod.dom
2021-06-29 03:01:51 +03:00
?: updated *ford-cache
2021-07-17 02:54:20 +03:00
(promote-ford fod.dom invalid)
2021-06-29 03:01:51 +03:00
=? ank.dom updated *ankh
=? changes updated (changes-for-upgrade q.old-yaki deletes changes)
::
2021-06-29 03:01:51 +03:00
=/ =args:ford:fusion [ank.dom deletes changes lat.ran fod.dom]
::
2021-06-29 03:01:51 +03:00
=^ change-cages ford-cache.args (checkout-changes args changes)
2020-04-30 00:50:30 +03:00
=/ sane-continuation (sane-changes changes change-cages)
=/ new-blobs=(map lobe blob)
%- malt
%+ turn ~(tap by change-cages)
|= [=path =lobe =cage]
[lobe %direct lobe [p q.q]:cage]
=/ data=(map path lobe)
2020-04-30 03:38:24 +03:00
%- ~(urn by new-data)
|= [=path value=(each page lobe)]
?- -.value
%| p.value
%& lobe:(~(got by change-cages) path)
==
:: if we didn't change the data and it's not a merge commit, abort
::
:: very important to keep all permanent changes below this point
::
?: &(=([r.old-yaki ~] p.p.yoki) =(data q.old-yaki))
..park
2020-05-01 05:32:48 +03:00
=/ =yaki
?- -.yoki
%& (make-yaki p.p.yoki data now)
%| ?> =(data q.p.yoki)
p.yoki
==
=: let.dom +(let.dom)
hit.dom (~(put by hit.dom) +(let.dom) r.yaki)
hut.ran (~(put by hut.ran) r.yaki yaki)
2022-03-26 01:32:38 +03:00
lat.ran (uni-blobs lat.ran new-blobs)
==
=. file-store.args lat.ran
::
=^ ankh ford-cache.args
(checkout-ankh args deletes change-cages ank.dom)
2020-04-30 00:50:30 +03:00
=/ null (sane-ankh sane-continuation ankh)
=. ankh.args ankh
=. ank.dom ankh
=^ mim ford-cache.args
(checkout-mime args deletes ~(key by changes))
=. mim.dom (apply-changes-to-mim mim.dom mim)
=. fod.dom ford-cache.args
=. ..park (emil (print q.old-yaki data))
wake:(ergo mim)
2021-08-26 19:32:52 +03:00
::
2021-07-17 02:54:20 +03:00
:: +is-kernel-path: should changing .pax cause a kernel or vane reload?
::
++ is-kernel-path |=(pax=path ?=([%sys *] pax))
::
2021-07-17 02:54:20 +03:00
++ did-kernel-update
|= invalid=(set path)
2022-03-26 02:41:54 +03:00
?. |(=(%base syd) &(=(%home syd)))
|
2021-07-17 02:54:20 +03:00
%- ~(any in invalid)
|=(p=path &((is-kernel-path p) !?=([%sys %vane *] p)))
2021-06-29 04:22:55 +03:00
:: +get-kelvin: read the desk's kernel version from /sys/kelvin
::
++ get-kelvin
|= =yoki
^- weft
2021-06-29 04:22:55 +03:00
|^ ?- -.yoki
%|
%- lobe-to-weft
2021-09-21 00:13:04 +03:00
~> %mean.(cat 3 'clay: missing /sys/kelvin on ' syd)
~| ~(key by q.p.yoki)
2021-06-29 04:22:55 +03:00
(~(got by q.p.yoki) /sys/kelvin)
::
%&
=/ fil=(each page lobe)
2021-09-21 00:13:04 +03:00
~> %mean.(cat 3 'clay: missing /sys/kelvin on ' syd)
~| ~(key by q.p.yoki)
2021-06-29 04:22:55 +03:00
(~(got by q.p.yoki) /sys/kelvin)
?- -.fil
%& (page-to-weft p.fil)
%| (lobe-to-weft p.fil)
2021-06-29 04:22:55 +03:00
==
==
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
::
++ lobe-to-weft
2021-06-29 04:22:55 +03:00
|= =lobe
^- weft
=/ =blob (lobe-to-blob:ze lobe)
=/ =page
?- -.blob
%direct q.blob
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
%delta r.blob
%dead ~|([%sys-kelvin-tombstoned syd] !!)
==
(page-to-weft page)
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
::
++ page-to-weft
|= =page
^- weft
?+ p.page ~|(clay-bad-kelvin-mark/p.page !!)
%kelvin ;;(weft q.page)
%mime
=+ ;;(=mime q.page)
!<(weft (slap !>(~) (ream q.q.mime)))
2021-06-29 04:22:55 +03:00
==
--
::
2020-04-17 00:05:19 +03:00
:: Find which files changed or were deleted
::
++ get-changes
|= [old=(map path lobe) new=(map path (each page lobe))]
2020-05-08 13:03:29 +03:00
^- [deletes=(set path) changes=(map path (each page lobe))]
=/ old=(map path (each page lobe))
(~(run by old) |=(=lobe |+lobe))
2020-04-17 00:05:19 +03:00
:* %- silt ^- (list path)
%+ murn ~(tap by (~(uni by old) new))
|= [=path *]
2020-04-17 00:05:19 +03:00
^- (unit ^path)
=/ a (~(get by new) path)
=/ b (~(get by old) path)
2020-04-17 00:05:19 +03:00
?: |(=(a b) !=(~ a))
~
`path
::
%- malt ^- (list [path (each page lobe)])
%+ murn ~(tap by (~(uni by old) new))
|= [=path *]
^- (unit [^path (each page lobe)])
=/ a (~(get by new) path)
=/ b (~(get by old) path)
?: |(=(a b) ?=(~ a))
2020-04-17 00:05:19 +03:00
~
`[path u.a]
2020-04-17 00:05:19 +03:00
==
2020-05-08 13:03:29 +03:00
:: Find all files for full desk rebuild
::
++ changes-for-upgrade
|= $: old=(map path lobe)
deletes=(set path)
changes=(map path (each page lobe))
==
^+ changes
=. old
%+ roll ~(tap in deletes)
|= [pax=path old=_old]
(~(del by old) pax)
=/ pre=_changes (~(run by old) |=(lob=lobe |+lob))
(~(uni by pre) changes)
2020-04-17 00:05:19 +03:00
::
:: Keep any parts of the ford cache whose dependencies didn't change
::
:: Make sure to invalidate any paths whose '-'s or '/'s could be
:: converted in an import; i.e. /mar, /lib, and /sur hoon files.
::
2021-07-17 02:54:20 +03:00
:: If anything in the kernel other than a vane updated,
:: clear the cache.
::
2020-04-16 02:24:48 +03:00
++ promote-ford
2021-07-17 02:54:20 +03:00
|= [=ford-cache invalid=(set path)]
^+ ford-cache
2021-07-17 02:54:20 +03:00
::
=. invalid
%- ~(gas in invalid)
%- zing
%+ turn ~(tap in invalid)
|= pax=path
^- (list path)
=/ xap=path (flop pax)
?. &(=(%hoon (head xap)) ?=([?(%mar %sur %lib) @ @ *] pax))
~
=- (turn - |=(suf=path [i.pax (snoc suf %hoon)]))
%- segments
%- crip
=/ xup (tail xap) :: lose %hoon extension
=/ pux (tail (flop xup)) :: lose static prefix
%+ turn (tail (spud pux)) :: lose leading '/'
|=(c=@tD `@tD`?:(=('/' c) '-' c)) :: convert '/' to '-'
::
2021-02-24 20:59:07 +03:00
:* ((invalidate path vase) files.ford-cache invalid)
2021-01-13 03:10:11 +03:00
((invalidate mark vase) naves.ford-cache invalid)
((invalidate mark dais) marks.ford-cache invalid)
2021-01-13 03:10:11 +03:00
((invalidate mars vase) casts.ford-cache invalid)
((invalidate mars tube) tubes.ford-cache invalid)
==
2020-04-16 02:24:48 +03:00
::
++ invalidate
|* [key=mold value=mold]
|= [cache=(map key [value dez=(set [dir=? =path])]) invalid=(set path)]
=/ builds=(list [key value dez=(set [dir=? =path])])
~(tap by cache)
2020-04-16 02:24:48 +03:00
|- ^+ cache
?~ builds
~
?: %- ~(any in dez.i.builds)
|= [dir=? =path]
?. dir (~(has in invalid) path)
=+ l=(lent path)
%- ~(any in invalid)
|= i=^path
&(=(path (scag l i)) ?=([@ %hoon ~] (slag l i)))
2020-04-16 02:24:48 +03:00
$(builds t.builds)
(~(put by $(builds t.builds)) i.builds)
::
++ page-to-cord
|= =page
^- @t
?+ p.page ~|([%sys-bad-mark p.page] !!)
%hoon ;;(@t q.page)
%mime q.q:;;(mime q.page)
==
::
++ lobe-to-cord
|= =lobe
^- @t
=- ?:(?=(%& -<) p.- (of-wain:format p.-))
|- ^- (each @t wain)
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
:: got:by ok since we must have a blob for anything at `let`
::
=/ =blob (lobe-to-blob:ze lobe)
?- -.blob
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
%dead ~|([%lobe-to-cord-tombstoned syd lobe] !!)
%direct [%& ;;(@t q.q.blob)]
%delta
:- %|
%+ lurk:differ
=- ?:(?=(%| -<) p.- (to-wain:format p.-))
$(lobe q.blob)
;;((urge cord) q.r.blob)
==
2020-06-01 19:56:21 +03:00
::
:: Updated q.yaki
::
++ checkout-changes
|= [=ford=args:ford:fusion changes=(map path (each page lobe))]
2021-02-24 22:43:35 +03:00
^- [(map path [=lobe =cage]) ford-cache]
%+ roll `(list [path (each page lobe)])`~(tap by changes)
|= $: [=path change=(each page lobe)]
[built=(map path [lobe cage]) cache=_ford-cache.ford-args]
==
^+ [built cache]
=. ford-cache.ford-args cache
=^ cage ford-cache.ford-args
2021-02-24 22:43:35 +03:00
:: ~> %slog.[0 leaf/"clay: validating {(spud path)}"]
%- wrap:fusion
2021-02-24 22:43:35 +03:00
(read-file:(ford:fusion ford-args) path)
=/ =lobe
2021-02-24 22:43:35 +03:00
?- -.change
%| p.change
2020-04-29 08:34:48 +03:00
:: Don't use p.change.i.cans because that's before casting to
:: the correct mark.
::
2020-05-12 10:37:04 +03:00
%& (page-to-lobe [p q.q]:cage)
==
2021-02-24 22:43:35 +03:00
[(~(put by built) path [lobe cage]) ford-cache.ford-args]
::
:: Update ankh
::
++ checkout-ankh
|= $: =ford=args:ford:fusion
deletes=(set path)
changes=(map path [lobe cage])
=ankh
==
^+ [ankh ford-cache.ford-args]
:: Delete
::
=. ankh
=/ dels ~(tap in deletes)
|- ^- ^ankh
=* outer-loop $
?~ dels
ankh
=. ankh
|- ^- ^ankh
=* inner-loop $
?~ i.dels
ankh(fil ~)
%= ankh
dir
%+ ~(put by dir.ankh) i.i.dels
%= inner-loop
i.dels t.i.dels
ankh (~(gut by dir.ankh) i.i.dels *^ankh)
==
==
outer-loop(dels t.dels)
:: Add/change
::
=/ cans=(list [=path =lobe =cage]) ~(tap by changes)
|- ^+ [ankh ford-cache.ford-args]
=* outer-loop $
?~ cans
[ankh ford-cache.ford-args]
=^ new-ankh ford-cache.ford-args
|- ^+ [ankh ford-cache.ford-args]
=* inner-loop $
?^ path.i.cans
=^ child-ankh ford-cache.ford-args
%= inner-loop
path.i.cans t.path.i.cans
ankh (~(gut by dir.ankh) i.path.i.cans *^ankh)
==
2020-04-29 06:33:12 +03:00
:- ankh(dir (~(put by dir.ankh) i.path.i.cans child-ankh))
ford-cache.ford-args
[ankh(fil `[lobe.i.cans cage.i.cans]) ford-cache.ford-args]
=. ankh new-ankh
outer-loop(cans t.cans)
::
2020-04-17 00:05:19 +03:00
:: Print notification to console
::
2020-03-25 03:24:19 +03:00
++ print
|= [old=(map path lobe) new=(map path lobe)]
2020-04-17 00:05:19 +03:00
^- (list move)
=/ [deletes=(set path) upserts=(map path (each page lobe))]
(get-changes old (~(run by new) |=(=lobe |+lobe)))
=/ upsert-set ~(key by upserts)
=/ old-set ~(key by old)
=/ changes=(set path) (~(int in upsert-set) old-set)
=/ additions=(set path) (~(dif in upsert-set) old-set)
2020-03-25 03:24:19 +03:00
?~ hun
2020-04-17 00:05:19 +03:00
~
2020-11-17 23:53:05 +03:00
?: (lte let.dom 1)
2020-04-29 04:13:30 +03:00
~
|^
;: weld
(paths-to-notes '-' deletes)
(paths-to-notes ':' changes)
(paths-to-notes '+' additions)
==
::
++ paths-to-notes
|= [prefix=@tD paths=(set path)]
%+ turn ~(tap in paths)
2020-04-17 00:05:19 +03:00
|= =path
[u.hun %give %note prefix (path-to-tank path)]
2020-04-17 00:05:19 +03:00
::
++ path-to-tank
|= =path
2020-11-17 23:53:05 +03:00
=/ pre=^path ~[(scot %p our) syd (scot %ud let.dom)]
2020-03-25 03:24:19 +03:00
:+ %rose ["/" "/" ~]
%+ turn (weld pre path)
|= a=cord
^- tank
?: ((sane %ta) a)
[%leaf (trip a)]
[%leaf (dash:us (trip a) '\'' ~)]
2020-04-17 00:05:19 +03:00
--
2020-04-25 03:44:29 +03:00
::
:: Check sanity
::
++ sane-changes
|= $: changes=(map path (each page lobe))
change-cages=(map path [lobe cage])
==
2020-04-29 08:34:48 +03:00
^- (unit [(map path [lobe cage]) args:ford:fusion])
2020-04-30 00:50:30 +03:00
?. check-sane
~
2020-04-29 08:34:48 +03:00
=/ tak=(unit tako) (~(get by hit.dom) let.dom)
?~ tak
~
=/ =yaki (~(got by hut.ran) u.tak)
:: Assert all blobs hash to their lobe
::
=/ foo
%- ~(urn by lat.ran)
|= [=lobe =blob]
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
?- -.blob
%dead ~
%delta ~
%direct
=/ actual-lobe=^lobe `@uv`(page-to-lobe q.blob)
~| [lobe p.blob actual-lobe]
?> &(=(lobe p.blob) =(lobe actual-lobe))
2020-04-29 08:34:48 +03:00
~
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
==
2020-04-29 08:34:48 +03:00
:: Assert we calculated the same change-cages w/o cache
::
:: ? remove deletes
2020-04-25 03:44:29 +03:00
::
=/ all-changes=(map path (each page lobe))
=/ original=(map path (each page lobe))
(~(run by q.yaki) |=(=lobe |+lobe))
(~(uni by original) changes)
2021-06-29 03:01:51 +03:00
=/ =args:ford:fusion [*ankh ~ all-changes lat.ran *ford-cache]
2020-04-25 03:44:29 +03:00
=^ all-change-cages ford-cache.args
(checkout-changes args all-changes)
=/ ccs=(list [=path =lobe =cage]) ~(tap by change-cages)
|- ^+ *sane-changes
?^ ccs
2020-04-29 08:34:48 +03:00
?. =(`[lobe cage]:i.ccs (~(get by all-change-cages) path.i.ccs))
~| not-same-cages+path.i.ccs
2020-04-25 03:44:29 +03:00
!!
$(ccs t.ccs)
2020-04-29 08:34:48 +03:00
`[all-change-cages args]
2020-04-25 03:44:29 +03:00
::
++ sane-ankh
2020-04-29 08:34:48 +03:00
|= $: $= cont
(unit [all-changes=(map path [lobe cage]) =ford=args:ford:fusion])
2020-04-25 03:44:29 +03:00
=test=ankh
==
2020-04-30 00:50:30 +03:00
?. check-sane
~
:: Assert all new lobes are reachable.
::
:: Needs to run after dome is updated
::
=/ tak=(unit tako) (~(get by hit.dom) let.dom)
?~ tak
~
=/ =yaki (~(got by hut.ran) u.tak)
=/ files=(list [=path =lobe]) ~(tap by q.yaki)
|- ^+ *sane-ankh
?^ files
?. (~(has by lat.ran) lobe.i.files)
~| missing-lobe=[path lobe]
!!
$(files t.files)
::
:: Assert we can rebuild the ankh
::
2020-04-29 08:34:48 +03:00
?~ cont
~
=+ u.cont
2020-04-25 03:44:29 +03:00
=^ ankh ford-cache.ford-args
2020-04-29 08:34:48 +03:00
(checkout-ankh ford-args ~ all-changes *ankh)
=| =path
|- ^- ~
=* loop $
=/ fil (bind fil.ankh |=([=lobe =cage] [lobe p.cage q.q.cage]))
=/ test (bind fil.ankh |=([=lobe =cage] [lobe p.cage q.q.cage]))
?. =(fil test)
~| [%not-same-file path ?=(~ fil.ankh) ?=(~ fil.test-ankh)]
~| ?~(fil.ankh ~ [[p p.q]:u.fil.ankh `@uv`(page-to-lobe [p q.q]:q.u.fil.ankh)])
~| ?~(fil.test-ankh ~ [[p p.q]:u.fil.test-ankh `@uv`(page-to-lobe [p q.q]:q.u.fil.test-ankh)])
2020-04-25 03:44:29 +03:00
!!
2020-04-29 08:34:48 +03:00
?. =(~(key by dir.ankh) ~(key by dir.test-ankh))
~| [%not-same-children path ~(key by dir.ankh) ~(key by dir.test-ankh)]
!!
=< ~
%+ turn ~(tap by dir.ankh)
|= [=@ta =child=^ankh]
~| sane-ankh=[path ta]
%= loop
path (snoc path ta)
ankh child-ankh
test-ankh (~(got by dir.test-ankh) ta)
==
2020-04-25 03:44:29 +03:00
::
:: Delay current update until sys update is complete
::
++ sys-update
|= $: =yoki
data=(map path (each page lobe))
==
2020-04-25 03:44:29 +03:00
^+ ..park
?> =(~ pud)
2020-04-30 03:38:24 +03:00
=. pud `[syd yoki]
2020-05-13 04:26:11 +03:00
|^ %. [hen %slip %c %pork ~]
emit:(pass-what files)
2020-04-25 03:44:29 +03:00
::
++ files
^- (list (pair path (cask)))
%+ murn
~(tap by data)
|= [pax=path dat=(each page lobe)]
^- (unit (pair path (cask)))
=/ xap (flop pax)
?> ?=(^ xap)
?. ?=(%hoon i.xap) ~
:^ ~ (flop t.xap) %hoon
2022-04-07 10:44:49 +03:00
~| pax=path
?- -.dat
%& (page-to-cord p.dat)
%| (lobe-to-cord p.dat)
==
2020-04-25 03:44:29 +03:00
::
++ pass-what
|= fil=(list (pair path (cask)))
2020-05-09 15:09:26 +03:00
^+ ..park
(emit hen %pass /what %$ what/fil)
2020-04-25 03:44:29 +03:00
--
--
::
2020-05-12 10:01:46 +03:00
:: We always say we're merging from 'ali' to 'bob'. The basic steps,
:: not all of which are always needed, are:
::
:: -- fetch ali's desk, async in case it's remote
:: -- diff ali's desk against the mergebase
:: -- diff bob's desk against the mergebase
:: -- merge the diffs
:: -- commit
::
++ start-merge
|= [=ali=ship =ali=desk =case =germ]
^+ ..start-merge
=/ =wire /merge/[syd]/(scot %p ali-ship)/[ali-desk]/[germ]
(emit hen %pass wire %c %warp ali-ship ali-desk `[%sing %v case /])
::
2021-04-20 06:46:46 +03:00
++ make-melt
2021-04-24 00:33:49 +03:00
|= [bas=beak con=(list [beak germ])]
2021-04-20 06:46:46 +03:00
^- melt
2021-04-24 00:33:49 +03:00
:+ bas con
2021-05-02 20:44:18 +03:00
%- ~(gas by *(map beak (unit dome:clay)))
:- [bas *(unit dome:clay)]
2021-04-24 00:33:49 +03:00
(turn con |=(a=[beak germ] [-.a *(unit dome:clay)]))
2021-04-20 06:46:46 +03:00
::
++ start-fuse
|= [bas=beak con=(list [beak germ])]
2021-04-20 06:46:46 +03:00
^+ ..start-fuse
=/ moves=(list move)
2021-05-04 02:29:28 +03:00
%+ turn
[[bas *germ] con]
|= [bec=beak germ]
^- move
=/ wir=wire /fuse/[syd]/(scot %p p.bec)/[q.bec]/(scot r.bec)
[hen %pass wir %c %warp p.bec q.bec `[%sing %v r.bec /]]
2021-05-02 20:44:18 +03:00
::
2021-05-02 21:15:03 +03:00
:: We also want to clear the state (fiz) associated with this
2021-04-20 06:46:46 +03:00
:: merge and print a warning if it's non trivial i.e. we're
:: starting a new fuse before the previous one terminated.
2021-05-02 20:44:18 +03:00
::
=/ err=tang
2021-05-04 02:29:28 +03:00
?~ con.fiz
~
=/ discarded=tang
%+ turn
~(tap in sto.fiz)
|= [k=beak v=(unit dome:clay)]
^- tank
=/ received=tape ?~(v "missing" "received")
2021-07-09 05:29:06 +03:00
leaf+"{<(en-beam k ~)>} {received}"
2021-05-04 02:29:28 +03:00
:_ discarded
leaf+"fusing into {<syd>} from {<bas>} {<con>} - overwriting prior fuse"
=. fiz (make-melt bas con)
2021-05-02 20:44:18 +03:00
((slog err) (emil moves))
2021-04-20 06:46:46 +03:00
::
++ take-fuse
2021-05-04 05:26:49 +03:00
|^
::
2021-04-24 00:33:49 +03:00
|= [bec=beak =riot]
2021-04-20 06:46:46 +03:00
^+ ..take-fuse
?~ riot
2021-05-02 20:44:18 +03:00
::
2021-05-02 21:15:03 +03:00
:: By setting fiz to *melt the merge is aborted - any further
2021-04-20 06:46:46 +03:00
:: responses we get for the merge will cause take-fuse to crash
2021-05-02 20:44:18 +03:00
::
2021-04-20 06:46:46 +03:00
=. fiz *melt
2021-07-09 05:29:06 +03:00
=/ msg=tape <(en-beam bec ~)>
((slog [leaf+"clay: fuse failed, missing {msg}"]~) ..take-fuse)
2021-06-22 03:43:49 +03:00
?. (~(has by sto.fiz) bec)
2021-07-09 05:29:06 +03:00
=/ msg=tape <(en-beam bec ~)>
((slog [leaf+"clay: got strange fuse response {<msg>}"]~) ..take-fuse)
2021-04-20 06:46:46 +03:00
=. fiz
2021-04-24 00:33:49 +03:00
:+ bas.fiz con.fiz
(~(put by sto.fiz) bec `!<(dome:clay q.r.u.riot))
2021-04-20 06:46:46 +03:00
=/ all-done=flag
2021-05-04 02:29:28 +03:00
%- ~(all by sto.fiz)
|= res=(unit dome:clay)
^- flag
!=(res ~)
2021-04-20 06:46:46 +03:00
?. all-done
..take-fuse
=| rag=rang
2021-05-04 05:26:49 +03:00
=/ clean-state ..take-fuse
2021-04-24 00:33:49 +03:00
=/ initial-dome=dome:clay (need (~(got by sto.fiz) bas.fiz))
2022-04-06 06:30:52 +03:00
=/ next-yaki=yaki
2021-05-04 05:26:49 +03:00
(~(got by hut.ran) (~(got by hit.initial-dome) let.initial-dome))
2021-04-20 06:46:46 +03:00
=/ parents=(list tako) ~[(~(got by hit.initial-dome) let.initial-dome)]
2021-04-24 00:33:49 +03:00
=/ merges con.fiz
2021-04-20 06:46:46 +03:00
|-
^+ ..take-fuse
?~ merges
2021-05-12 04:30:44 +03:00
=. ..take-fuse (done-fuse clean-state %& ~)
2022-04-06 06:30:52 +03:00
(park | [%| next-yaki(p (flop parents))] rag)
2021-04-20 06:46:46 +03:00
=/ [bec=beak g=germ] i.merges
2021-04-24 00:33:49 +03:00
=/ ali-dom=dome:clay (need (~(got by sto.fiz) bec))
2022-04-06 06:30:52 +03:00
=/ result (merge-helper p.bec q.bec g ali-dom `next-yaki)
2021-05-02 20:44:18 +03:00
?- -.result
2021-04-20 06:46:46 +03:00
%|
=/ failing-merge=tape "{<bec>} {<g>}"
(done-fuse clean-state %| %fuse-merge-failed leaf+failing-merge p.result)
2021-04-20 06:46:46 +03:00
::
%&
=/ merge-result=(unit merge-result) +.result
?~ merge-result
2021-05-02 21:04:03 +03:00
::
2021-05-02 21:15:03 +03:00
:: This merge was a no-op, just continue
2021-05-02 21:04:03 +03:00
::
2021-04-20 06:46:46 +03:00
$(merges t.merges)
2021-05-02 21:04:03 +03:00
?^ conflicts.u.merge-result
2021-05-02 20:44:18 +03:00
::
2021-05-02 21:15:03 +03:00
:: If there are merge conflicts send the error and abort the merge
2021-05-02 21:04:03 +03:00
::
2021-05-04 05:26:49 +03:00
(done-fuse clean-state %& conflicts.u.merge-result)
2021-05-02 21:04:03 +03:00
=/ merged-yaki=yaki
2022-04-06 06:30:52 +03:00
?- -.new.u.merge-result
%| +.new.u.merge-result
%&
::
:: Convert the yuki to yaki
::
=/ yuk=yuki +.new.u.merge-result
=/ lobes=(map path lobe)
%- ~(run by q.yuk)
|= val=(each page lobe)
^- lobe
?- -.val
%& (page-to-lobe +.val)
%| +.val
==
(make-yaki p.yuk lobes now)
==
2021-05-02 21:04:03 +03:00
%= $
2022-04-06 06:30:52 +03:00
next-yaki merged-yaki
merges t.merges
hut.ran (~(put by hut.ran) r.merged-yaki merged-yaki)
lat.rag (uni-blobs lat.rag lat.u.merge-result)
parents [(~(got by hit.ali-dom) let.ali-dom) parents]
2021-05-02 21:04:03 +03:00
==
2021-04-20 06:46:46 +03:00
==
2021-05-04 05:26:49 +03:00
:: +done-fuse: restore state after a fuse is attempted, whether it
:: succeeds or fails.
::
++ done-fuse
|= [to-restore=_..take-fuse result=(each (set path) (pair term tang))]
^+ ..take-fuse
=. fiz.to-restore *melt
(done:to-restore result)
--
2021-04-20 06:46:46 +03:00
::
++ done
|= result=(each (set path) (pair term tang))
^+ ..merge
(emit hen %give %mere result)
::
2020-05-12 10:01:46 +03:00
++ merge
|= [=ali=ship =ali=desk =germ =riot]
^+ ..merge
?~ riot
2021-04-20 06:46:46 +03:00
(done %| %ali-unavailable ~[>[ali-ship ali-desk germ]<])
2020-05-12 10:01:46 +03:00
=/ ali-dome=dome:clay !<(dome:clay q.r.u.riot)
2021-04-20 06:46:46 +03:00
=/ result=(each (unit merge-result) (pair term tang))
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
(merge-helper ali-ship ali-desk germ ali-dome ~)
2021-05-02 20:44:18 +03:00
?- -.result
2022-04-06 06:30:52 +03:00
%| (done %| +.result)
2021-04-20 06:46:46 +03:00
%&
=/ mr=(unit merge-result) +.result
?~ mr
(done %& ~)
=. ..merge (done %& conflicts.u.mr)
(park | new.u.mr ~ lat.u.mr)
==
::
+$ merge-result [conflicts=(set path) new=yoki lat=(map lobe blob)]
::
++ merge-helper
2022-04-06 06:30:52 +03:00
|= [=ali=ship =ali=desk =germ ali-dome=dome:clay next-yaki=(unit yaki)]
2021-04-20 06:46:46 +03:00
^- (each (unit merge-result) [term tang])
|^
^- (each (unit merge-result) [term tang])
2020-05-12 10:01:46 +03:00
=/ ali-yaki=yaki (~(got by hut.ran) (~(got by hit.ali-dome) let.ali-dome))
=/ bob-yaki=(unit yaki)
2022-04-06 06:30:52 +03:00
?~ next-yaki
2021-05-18 21:32:30 +03:00
?~ let.dom
~
(~(get by hut.ran) (~(got by hit.dom) let.dom))
2022-04-06 06:30:52 +03:00
next-yaki
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
=/ res (mule |.((merge-by-germ ali-yaki bob-yaki)))
?- -.res
%& &+p.res
%| |+merge-failed+p.res
==
2021-05-18 21:32:30 +03:00
::
2020-05-12 10:01:46 +03:00
++ merge-by-germ
|= [=ali=yaki bob-yaki=(unit yaki)]
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
^- (unit merge-result)
2020-05-12 10:01:46 +03:00
::
:: If this is an %init merge, we set the ali's commit to be
:: bob's.
::
?: ?=(%init germ)
?> ?=(~ bob-yaki)
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
`[conflicts=~ new=|+ali-yaki lat=~]
2020-05-12 10:01:46 +03:00
::
=/ bob-yaki (need bob-yaki)
|^
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
^- (unit merge-result)
2020-05-12 10:01:46 +03:00
?- germ
::
:: If this is a %only-this merge, we check to see if ali's and bob's
2021-04-20 06:46:46 +03:00
:: commits are the same, in which case we're done.
:: Otherwise, we create a new commit with bob's data plus ali and
:: bob as parents.
2020-05-12 10:01:46 +03:00
::
%only-this
2020-05-12 10:01:46 +03:00
?: =(r.ali-yaki r.bob-yaki)
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
~
:* ~
2020-05-12 10:01:46 +03:00
conflicts=~
new=&+[[r.bob-yaki r.ali-yaki ~] (to-yuki q.bob-yaki)]
lat=~
==
::
:: If this is a %only-that merge, we check to see if ali's and bob's
2020-05-12 10:01:46 +03:00
:: commits are the same, in which case we're done. Otherwise, we
:: create a new commit with ali's data plus ali and bob as
:: parents.
::
%only-that
2020-05-12 10:01:46 +03:00
?: =(r.ali-yaki r.bob-yaki)
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
~
:* ~
2020-05-12 10:01:46 +03:00
conflicts=~
new=&+[[r.bob-yaki r.ali-yaki ~] (to-yuki q.ali-yaki)]
lat=~
==
::
:: Create a merge commit with exactly the contents of the
:: destination desk except take any files from the source commit
:: which are not in the destination desk.
::
%take-this
?: =(r.ali-yaki r.bob-yaki)
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
~
=/ new-data (~(uni by q.ali-yaki) q.bob-yaki)
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
:* ~
conflicts=~
new=&+[[r.bob-yaki r.ali-yaki ~] (to-yuki new-data)]
lat=~
==
::
:: Create a merge commit with exactly the contents of the source
:: commit except preserve any files from the destination desk
:: which are not in the source commit.
::
%take-that
?: =(r.ali-yaki r.bob-yaki)
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
~
=/ new-data (~(uni by q.bob-yaki) q.ali-yaki)
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
:* ~
conflicts=~
new=&+[[r.bob-yaki r.ali-yaki ~] (to-yuki new-data)]
lat=~
==
::
2020-05-12 10:01:46 +03:00
:: If this is a %fine merge, we check to see if ali's and bob's
:: commits are the same, in which case we're done. Otherwise, we
:: check to see if ali's commit is in the ancestry of bob's, in
:: which case we're done. Otherwise, we check to see if bob's
:: commit is in the ancestry of ali's. If not, this is not a
:: fast-forward merge, so we error out. If it is, we add ali's
:: commit to bob's desk and checkout.
::
%fine
?: =(r.ali-yaki r.bob-yaki)
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
~
2020-05-12 10:37:04 +03:00
?: (~(has in (reachable-takos:ze r.bob-yaki)) r.ali-yaki)
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
~
2020-05-12 10:37:04 +03:00
?. (~(has in (reachable-takos:ze r.ali-yaki)) r.bob-yaki)
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
~_ %bad-fine-merge
~| "tried fast-forward but is not ancestor or descendant"
!!
`[conflicts=~ new=|+ali-yaki lat=~]
2020-05-12 10:01:46 +03:00
::
?(%meet %mate %meld %meet-this %meet-that)
2020-05-12 10:01:46 +03:00
?: =(r.ali-yaki r.bob-yaki)
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
~
2020-05-12 10:37:04 +03:00
?: (~(has in (reachable-takos:ze r.bob-yaki)) r.ali-yaki)
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
~
2020-05-12 10:37:04 +03:00
?: (~(has in (reachable-takos:ze r.ali-yaki)) r.bob-yaki)
2020-05-12 10:01:46 +03:00
$(germ %fine)
=/ merge-points (find-merge-points ali-yaki bob-yaki)
2020-05-12 10:01:46 +03:00
?~ merge-points
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
~_ %merge-no-merge-base
~| "consider a %this or %that merge to get a mergebase"
!!
2020-05-12 10:01:46 +03:00
=/ merge-point=yaki n.merge-points
?: ?=(?(%mate %meld) germ)
2020-05-12 10:01:46 +03:00
=/ ali-diffs=cane (diff-base ali-yaki bob-yaki merge-point)
=/ bob-diffs=cane (diff-base bob-yaki ali-yaki merge-point)
=/ bof=(map path (unit cage))
(merge-conflicts can.ali-diffs can.bob-diffs)
2020-05-14 03:47:19 +03:00
(build ali-yaki bob-yaki merge-point ali-diffs bob-diffs bof)
2020-05-12 10:01:46 +03:00
=/ ali-diffs=cane (calc-diffs ali-yaki merge-point)
=/ bob-diffs=cane (calc-diffs bob-yaki merge-point)
=/ both-diffs=(map path *)
%- %~ int by
%- ~(uni by `(map path *)`new.ali-diffs)
%- ~(uni by `(map path *)`cal.ali-diffs)
%- ~(uni by `(map path *)`can.ali-diffs)
`(map path *)`old.ali-diffs
%- ~(uni by `(map path *)`new.bob-diffs)
%- ~(uni by `(map path *)`cal.bob-diffs)
%- ~(uni by `(map path *)`can.bob-diffs)
`(map path *)`old.bob-diffs
?: &(?=(%meet germ) !=(~ both-diffs))
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
~_ %meet-conflict
~| [~(key by both-diffs) "consider a %mate merge"]
!!
=/ both-done=(map path lobe)
|^
?- germ
%meet ~
%meet-this (resolve (~(uni by new.bob-diffs) cal.bob-diffs))
%meet-that (resolve (~(uni by new.ali-diffs) cal.ali-diffs))
==
++ resolve
|= news=(map path lobe)
%- malt ^- (list [path lobe])
%+ murn ~(tap by both-diffs)
|= [=path *]
^- (unit [^path lobe])
=/ new (~(get by news) path)
?~ new
~
`[path u.new]
--
::
=/ deleted
%- ~(dif by (~(uni by old.ali-diffs) old.bob-diffs))
(~(run by both-done) |=(* ~))
2020-05-12 10:01:46 +03:00
=/ not-deleted=(map path lobe)
%+ roll ~(tap by deleted)
2020-05-12 10:01:46 +03:00
=< .(not-deleted q.merge-point)
|= [[pax=path ~] not-deleted=(map path lobe)]
(~(del by not-deleted) pax)
=/ hat=(map path lobe)
%- ~(uni by not-deleted)
%- ~(uni by new.ali-diffs)
%- ~(uni by new.bob-diffs)
%- ~(uni by cal.ali-diffs)
cal.bob-diffs
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
:* ~
2020-05-12 10:01:46 +03:00
conflicts=~
new=&+[[r.bob-yaki r.ali-yaki ~] (to-yuki hat)]
lat=~
==
==
::
++ to-yuki
|= m=(map path lobe)
^- (map path (each page lobe))
(~(run by m) |=(=lobe |+lobe))
::
:: The set of changes between the mergebase and one of the desks
:: being merged
::
:: -- `new` is the set of files in the new desk and not in the
:: mergebase.
:: -- `cal` is the set of changes in the new desk from the
:: mergebase except for any that are also in the other new desk.
:: -- `can` is the set of changes in the new desk from the
:: mergebase that are also in the other new desk (potential
:: conflicts).
:: -- `old` is the set of files in the mergebase and not in the
:: new desk.
::
+$ cane
$: new=(map path lobe)
cal=(map path lobe)
can=(map path cage)
old=(map path ~)
==
::
:: Calculate cane knowing there are no files changed by both
:: desks
::
++ calc-diffs
|= [hed=yaki bas=yaki]
^- cane
:* %- molt
%+ skip ~(tap by q.hed)
|= [pax=path lob=lobe]
(~(has by q.bas) pax)
::
%- molt
%+ skip ~(tap by q.hed)
|= [pax=path lob=lobe]
=+ (~(get by q.bas) pax)
|(=(~ -) =([~ lob] -))
::
~
::
%- malt ^- (list [path ~])
%+ murn ~(tap by q.bas)
|= [pax=path lob=lobe]
^- (unit (pair path ~))
?. =(~ (~(get by q.hed) pax))
~
`[pax ~]
==
::
:: Diff yak against bas where different from yuk
::
++ diff-base
|= [yak=yaki yuk=yaki bas=yaki]
^- cane
=/ new=(map path lobe)
%- malt
%+ skip ~(tap by q.yak)
|= [=path =lobe]
(~(has by q.bas) path)
::
=/ cal=(map path lobe)
%- malt ^- (list [path lobe])
%+ murn ~(tap by q.bas)
|= [pax=path lob=lobe]
^- (unit (pair path lobe))
=+ a=(~(get by q.yak) pax)
=+ b=(~(get by q.yuk) pax)
?. ?& ?=(^ a)
!=([~ lob] a)
=([~ lob] b)
==
~
`[pax +.a]
::
=/ can=(map path cage)
%- malt
%+ murn ~(tap by q.bas)
|= [=path =lobe]
^- (unit [^path cage])
=/ in-yak (~(get by q.yak) path)
?~ in-yak
~
?: =(lobe u.in-yak)
~
=/ in-yuk (~(get by q.yuk) path)
?~ in-yuk
~
?: =(lobe u.in-yuk)
~
?: =(u.in-yak u.in-yuk)
~
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
=/ cug=(unit cage) (diff-lobes lobe u.in-yak)
?~ cug
~_ %tombstoned-mergebase
~| path
~| "consider a 2-way merge such as %only-this or %only-that"
!!
`[path u.cug]
2020-05-12 10:01:46 +03:00
::
=/ old=(map path ~)
%- malt ^- (list [path ~])
%+ murn ~(tap by q.bas)
|= [pax=path lob=lobe]
?. =(~ (~(get by q.yak) pax))
~
(some pax ~)
::
[new cal can old]
::
:: These can/should save their caches
::
++ lobe-to-cage
|= =lobe
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
^- (unit cage)
=^ peg=(unit page) fod.dom
2020-05-12 10:01:46 +03:00
%- wrap:fusion
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
(lobe-to-unit-page:(ford:fusion static-ford-args) lobe)
?~ peg
~
2020-05-12 10:01:46 +03:00
=^ =cage fod.dom
%- wrap:fusion
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
(page-to-cage:(ford:fusion static-ford-args) u.peg)
`cage
2020-05-12 10:01:46 +03:00
::
++ get-dais
|= =mark
^- dais
=^ =dais fod.dom
%- wrap:fusion
2021-02-24 20:59:07 +03:00
(build-dais:(ford:fusion static-ford-args) mark)
2020-05-12 10:01:46 +03:00
dais
::
:: Diff two files on bob-desk
::
++ diff-lobes
|= [=a=lobe =b=lobe]
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
^- (unit cage)
2020-05-12 10:01:46 +03:00
=/ a-cage (lobe-to-cage a-lobe)
=/ b-cage (lobe-to-cage b-lobe)
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
?: |(?=(~ a-cage) ?=(~ b-cage))
~
?> =(p.u.a-cage p.u.b-cage)
=/ =dais (get-dais p.u.a-cage)
`[form:dais (~(diff dais q.u.a-cage) q.u.b-cage)]
2020-05-12 10:01:46 +03:00
::
:: Merge diffs that are on the same file.
::
++ merge-conflicts
|= [ali-conflicts=(map path cage) bob-conflicts=(map path cage)]
^- (map path (unit cage))
%- ~(urn by (~(int by ali-conflicts) bob-conflicts))
|= [=path *]
^- (unit cage)
=/ cal=cage (~(got by ali-conflicts) path)
=/ cob=cage (~(got by bob-conflicts) path)
=/ =mark
=+ (slag (dec (lent path)) path)
?~(- %$ i.-)
=/ =dais (get-dais mark)
2021-03-06 00:53:35 +03:00
=/ res=(unit (unit vase)) (~(join dais *vale:dais) q.cal q.cob)
2020-05-12 10:01:46 +03:00
?~ res
`[form:dais q.cob]
2020-05-12 10:01:46 +03:00
?~ u.res
~
`[form:dais u.u.res]
::
:: Apply the patches in bof to get the new merged content.
::
:: Gather all the changes between ali's and bob's commits and the
:: mergebase. This is similar to the %meet of ++merge, except
:: where they touch the same file, we use the merged versions.
::
++ build
|= $: ali=yaki
bob=yaki
bas=yaki
dal=cane
dob=cane
bof=(map path (unit cage))
==
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
^- (unit merge-result)
2020-05-12 10:01:46 +03:00
=/ both-patched=(map path cage)
%- malt
%+ murn ~(tap by bof)
|= [=path cay=(unit cage)]
^- (unit [^path cage])
?~ cay
~
:+ ~ path
=+ (~(get by q.bas) path)
?~ -
~| %mate-strange-diff-no-base
!!
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
:: +need ok because we would have crashed in +diff-base
::
=/ =cage ~|([%build-need path] (need (lobe-to-cage u.-)))
2020-05-12 10:01:46 +03:00
=/ =dais (get-dais p.cage)
?> =(p.u.cay form.dais)
:- p.cage
(~(pact dais q.cage) q.u.cay)
=/ con=(map path *) :: 2-change conflict
%- molt
%+ skim ~(tap by bof)
|=([pax=path cay=(unit cage)] ?=(~ cay))
=/ cab=(map path lobe) :: conflict base
%- ~(urn by con)
|= [pax=path *]
(~(got by q.bas) pax)
=. con :: change+del conflict
%- ~(uni by con)
%- malt ^- (list [path *])
%+ skim ~(tap by old.dal)
|= [pax=path ~]
?: (~(has by new.dob) pax)
~| %strange-add-and-del
!!
(~(has by can.dob) pax)
=. con :: change+del conflict
%- ~(uni by con)
%- malt ^- (list [path *])
%+ skim ~(tap by old.dob)
|= [pax=path ~]
?: (~(has by new.dal) pax)
~| %strange-del-and-add
!!
(~(has by can.dal) pax)
=. con :: add+add conflict
%- ~(uni by con)
%- malt ^- (list [path *])
%+ skip ~(tap by (~(int by new.dal) new.dob))
|= [pax=path *]
=((~(got by new.dal) pax) (~(got by new.dob) pax))
2020-05-14 03:47:19 +03:00
?: &(?=(%mate germ) ?=(^ con))
2020-07-01 01:24:42 +03:00
=+ (turn ~(tap by `(map path *)`con) |=([path *] >[+<-]<))
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
~_ %mate-conflict
~| (turn ~(tap by `(map path *)`con) |=([path *] +<-))
!!
2020-05-12 10:01:46 +03:00
=/ old=(map path lobe) :: oldies but goodies
%+ roll ~(tap by (~(uni by old.dal) old.dob))
=< .(old q.bob)
2020-05-12 10:01:46 +03:00
|= [[pax=path ~] old=(map path lobe)]
(~(del by old) pax)
=/ [hot=(map path lobe) lat=(map lobe blob)] :: new content
%+ roll ~(tap by both-patched)
|= [[pax=path cay=cage] hat=(map path lobe) lat=(map lobe blob)]
=/ =blob [%direct (page-to-lobe [p q.q]:cay) [p q.q]:cay]
:- (~(put by hat) pax p.blob)
?: (~(has by lat) p.blob)
lat
2022-03-26 01:32:38 +03:00
(uni-blobs lat (malt [p.blob blob] ~))
2020-05-12 10:01:46 +03:00
=/ hat=(map path lobe) :: all the content
%- ~(uni by old)
%- ~(uni by new.dal)
%- ~(uni by new.dob)
%- ~(uni by cal.dal)
%- ~(uni by cal.dob)
%- ~(uni by hot)
cab
=/ del=(map path ?)
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
(~(run by (~(uni by old.dal) old.dob)) |=(~ %|))
2020-05-12 10:01:46 +03:00
=/ new &+[[r.bob r.ali ~] (~(run by hat) |=(=lobe |+lobe))]
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
:* ~
2020-05-14 03:47:19 +03:00
(silt (turn ~(tap by con) head))
2020-05-12 10:01:46 +03:00
new
lat
==
--
--
::
:: Find the most recent common ancestor(s).
::
:: For performance, this depends on +reachable-takos being
:: memoized.
::
++ find-merge-points
|= [=ali=yaki =bob=yaki]
^- (set yaki)
:: Loop through ancestors breadth-first, lazily generating ancestry
::
=/ ali-takos (reachable-takos:ze r.ali-yaki)
:: Tako worklist
::
=/ takos=(qeu tako) [r.bob-yaki ~ ~]
:: Mergebase candidates. Have proven they're common ancestors, but
:: not that they're a most recent
::
=| bases=(set tako)
:: Takos we've already checked or are in our worklist
::
=| done=(set tako)
|- ^- (set yaki)
=* outer-loop $
:: If we've finished our worklist, convert to yakis and return
::
?: =(~ takos)
(silt (turn ~(tap in bases) ~(got by hut.ran)))
=^ =tako takos ~(get to takos)
=. done (~(put in done) tako)
:: If this is a common ancestor, stop recursing through our
:: parentage. Check if it's comparable to any existing candidate.
::
?: (~(has in ali-takos) tako)
=/ base-list ~(tap in bases)
|- ^- (set yaki)
=* bases-loop $
?~ base-list
:: Proven it's not an ancestor of any previous candidate.
:: Remove all ancestors of new candidate and add it to the
:: candidate list.
::
=. bases
=/ new-reachable (reachable-takos:ze tako)
(~(put in (~(dif in bases) new-reachable)) tako)
outer-loop
:: If it's an ancestor of another candidate, this is not most
:: recent, so skip and try next in worklist.
::
=/ base-reachable (reachable-takos:ze i.base-list)
?: (~(has in base-reachable) tako)
outer-loop
bases-loop(base-list t.base-list)
:: Append parents to list and recurse
::
=/ bob-yaki (~(got by hut.ran) tako)
=/ new-candidates (skip p.bob-yaki ~(has in done))
%_ outer-loop
done (~(gas in done) new-candidates)
takos (~(gas to takos) new-candidates)
==
::
2020-05-12 11:27:19 +03:00
:: Update mime cache
::
++ checkout-mime
|= $: =ford=args:ford:fusion
deletes=(set path)
changes=(set path)
==
^- [(map path (unit mime)) ford-cache]
=/ mim=(map path (unit mime))
=/ dels=(list path) ~(tap by deletes)
|- ^- (map path (unit mime))
?~ dels
~
(~(put by $(dels t.dels)) i.dels ~)
=/ cans=(list path) ~(tap by changes)
|- ^- [(map path (unit mime)) ford-cache]
?~ cans
[mim ford-cache.ford-args]
=^ cage ford-cache.ford-args
~| mime-cast-fail+i.cans
(wrap:fusion (cast-path:(ford:fusion ford-args) i.cans %mime))
=^ mim ford-cache.ford-args $(cans t.cans)
[(~(put by mim) i.cans `!<(mime q.cage)) ford-cache.ford-args]
::
:: Add or remove entries to the mime cache
::
++ apply-changes-to-mim
|= [mim=(map path mime) changes=(map path (unit mime))]
^- (map path mime)
=/ changes-l=(list [pax=path change=(unit mime)])
~(tap by changes)
|- ^- (map path mime)
?~ changes-l
mim
?~ change.i.changes-l
$(changes-l t.changes-l, mim (~(del by mim) pax.i.changes-l))
$(changes-l t.changes-l, mim (~(put by mim) [pax u.change]:i.changes-l))
::
:: Emit update to unix sync
::
++ ergo
|= mim=(map path (unit mime))
^+ ..park
=/ must (must-ergo her syd mon (turn ~(tap by mim) head))
%- emil
%+ turn ~(tap by must)
|= [pot=term len=@ud pak=(set path)]
:* (need hez) %give %ergo pot
%+ turn ~(tap in pak)
|= pax=path
[(slag len pax) (~(got by mim) pax)]
==
::
:: Output is a map of mount points to {length-of-mounted-path set-of-paths}.
::
++ must-ergo
|= [our=ship syd=desk mon=(map term beam) can=(list path)]
2020-05-12 11:27:19 +03:00
^- (map term (pair @ud (set path)))
%- malt ^- (list (trel term @ud (set path)))
%+ murn ~(tap by mon)
|= [nam=term bem=beam]
2020-05-12 11:27:19 +03:00
^- (unit (trel term @ud (set path)))
=- ?~(- ~ `[nam (lent s.bem) (silt `(list path)`-)])
%+ skim can
|= pax=path
&(=(p.bem our) =(q.bem syd) =(s.bem (scag (lent s.bem) pax)))
2020-05-12 11:27:19 +03:00
::
:: Mount a beam to unix
::
++ mount
|= [pot=term =case =spur]
^+ ..mount
=/ old-mon (~(get by mon) pot)
?^ old-mon
%- (slog >%already-mounted< >u.old-mon< ~)
..mount
=. mon (~(put by mon) pot [her syd case] spur)
=/ =yaki (~(got by hut.ran) (~(got by hit.dom) let.dom))
=/ changes (~(run by q.yaki) |=(=lobe |+lobe))
2021-06-29 03:01:51 +03:00
=/ =args:ford:fusion [ank.dom ~ changes lat.ran fod.dom]
2020-05-12 11:27:19 +03:00
=^ mim ford-cache.args
(checkout-mime args ~ ~(key by changes))
=. mim.dom (apply-changes-to-mim mim.dom mim)
=. fod.dom ford-cache.args
(ergo mim)
::
2019-05-03 04:50:20 +03:00
:: Set permissions for a node.
2016-11-24 07:25:07 +03:00
::
2019-05-03 04:50:20 +03:00
++ perm
|= [pax=path rit=rite]
2019-05-03 04:50:20 +03:00
^+ +>
=/ mis=(set @ta)
2019-05-03 04:50:20 +03:00
%+ roll
=- ~(tap in -)
?- -.rit
%r who:(fall red.rit *rule)
%w who:(fall wit.rit *rule)
%rw (~(uni in who:(fall red.rit *rule)) who:(fall wit.rit *rule))
2019-05-03 04:50:20 +03:00
==
|= [w=whom s=(set @ta)]
2019-05-03 04:50:20 +03:00
?: |(?=(%& -.w) (~(has by cez) p.w)) s
(~(put in s) p.w)
?^ mis
2019-07-16 02:11:22 +03:00
:: TODO remove this nasty hack
::
?. ?=([[%a *] *] hen)
+>.$
2019-06-29 02:43:52 +03:00
=- (emit hen %give %done `[%perm-fail [%leaf "No such group(s): {-}"]~])
2019-05-03 04:50:20 +03:00
%+ roll ~(tap in `(set @ta)`mis)
|= [g=@ta t=tape]
2019-05-03 04:50:20 +03:00
?~ t (trip g)
:(weld t ", " (trip g))
2019-07-16 02:11:22 +03:00
:: TODO remove this nasty hack
::
=< ?. ?=([[%a *] *] hen)
.
(emit hen %give %done ~)
::
2019-05-03 04:50:20 +03:00
?- -.rit
%r wake(per (put-perm per pax red.rit))
%w wake(pew (put-perm pew pax wit.rit))
%rw wake(per (put-perm per pax red.rit), pew (put-perm pew pax wit.rit))
2019-05-03 04:50:20 +03:00
==
2016-11-24 07:25:07 +03:00
::
2019-05-03 04:50:20 +03:00
++ put-perm
|= [pes=regs pax=path new=(unit rule)]
2019-05-03 04:50:20 +03:00
?~ new (~(del by pes) pax)
(~(put by pes) pax u.new)
2016-11-24 07:25:07 +03:00
::
2019-05-03 04:50:20 +03:00
:: Remove a group from all rules.
2016-11-24 07:25:07 +03:00
::
2019-05-03 04:50:20 +03:00
++ forget-crew
|= nom=@ta
2019-05-03 04:50:20 +03:00
%= +>
per (forget-crew-in nom per)
pew (forget-crew-in nom pew)
==
2016-11-24 07:25:07 +03:00
::
2019-05-03 04:50:20 +03:00
++ forget-crew-in
|= [nom=@ta pes=regs]
2019-05-03 04:50:20 +03:00
%- ~(run by pes)
|= r=rule
2019-05-03 04:50:20 +03:00
r(who (~(del in who.r) |+nom))
2016-11-24 07:25:07 +03:00
::
2019-05-03 04:50:20 +03:00
:: Cancel a request.
2016-11-24 07:25:07 +03:00
::
2019-05-03 04:50:20 +03:00
:: For local requests, we just remove it from `qyx`. For foreign requests,
:: we remove it from `ref` and tell the foreign ship to cancel as well.
2016-11-24 07:25:07 +03:00
::
2019-05-03 04:50:20 +03:00
++ cancel-request :: release request
^+ ..cancel-request
=^ wos=(list wove) qyx
:_ (~(run by qyx) |=(a=(set duct) (~(del in a) hen)))
2019-05-03 04:50:20 +03:00
%- ~(rep by qyx)
|= [[a=wove b=(set duct)] c=(list wove)]
2019-05-03 04:50:20 +03:00
?.((~(has in b) hen) c [a c])
::
2019-05-03 04:50:20 +03:00
?~ ref
=> .(ref `(unit rind)`ref)
?: =(~ wos) ..cancel-request :: TODO handle?
|- ^+ ..cancel-request
?~ wos ..cancel-request
=. ..cancel-request (run-if-future rove.i.wos |=(@da (best hen +<)))
$(wos t.wos)
::
?~ nux=(~(get by fod.u.ref) hen)
..cancel-request(ref `(unit rind)`ref)
2019-05-03 04:50:20 +03:00
=: fod.u.ref (~(del by fod.u.ref) hen)
bom.u.ref (~(del by bom.u.ref) u.nux)
==
%. [hen her u.nux [syd ~]]
send-over-ames(ref `(unit rind)`ref)
2016-11-24 07:25:07 +03:00
::
2019-05-03 04:50:20 +03:00
:: Handles a request.
2016-11-24 07:25:07 +03:00
::
2019-05-03 04:50:20 +03:00
:: `%sing` requests are handled by ++aver. `%next` requests are handled by
:: running ++aver at the given case, and then subsequent cases until we find
:: a case where the two results aren't equivalent. If it hasn't happened
:: yet, we wait. `%many` requests are handled by producing as much as we can
:: and then waiting if the subscription range extends into the future.
2016-11-24 07:25:07 +03:00
::
2019-05-03 04:50:20 +03:00
++ start-request
|= [for=(unit [ship @ud]) rav=rave]
2019-05-23 00:39:12 +03:00
^+ ..start-request
?: &(?=(^ for) !(foreign-capable rav))
~& [%bad-foreign-request-care from=for rav]
..start-request
=^ [new-sub=(unit rove) cards=(list card)] fod.dom
(try-fill-sub for (rave-to-rove rav))
=. ..start-request (send-cards cards [hen ~ ~])
2019-05-23 00:39:12 +03:00
?~ new-sub
..start-request
(duce for u.new-sub)
2016-11-24 07:25:07 +03:00
::
2019-05-17 03:24:10 +03:00
:: Called when a foreign ship answers one of our requests.
::
2020-05-13 08:53:23 +03:00
:: If it's a `%many` request, process in +take-foreign-update
2019-05-17 03:24:10 +03:00
::
:: After updating ref (our request manager), we handle %x, %w, and %y
:: responses. For %x, we call ++validate-x to validate the type of
:: the response. For %y, we coerce the result to an arch.
::
++ take-foreign-answer :: external change
|= [inx=@ud rut=(unit rand)]
^+ +>
?> ?=(^ ref)
=+ ruv=(~(get by bom.u.ref) inx)
?~ ruv +>.$
=/ rav=rave rave.u.ruv
2019-05-17 03:24:10 +03:00
?: ?=(%many -.rav)
abet:(apex:(foreign-update inx) rut)
2019-05-17 03:24:10 +03:00
?~ rut
:: nothing here, so cache that
::
%_ wake
haw.u.ref
?. ?=(%sing -.rav) haw.u.ref
2019-05-23 00:39:12 +03:00
(~(put by haw.u.ref) mood.rav ~)
2019-05-17 03:24:10 +03:00
==
2020-05-13 06:22:25 +03:00
|^
=/ result=(unit cage) (validate u.rut)
=/ =mood [p.p q.p q]:u.rut
=: haw.u.ref (~(put by haw.u.ref) mood result)
bom.u.ref (~(del by bom.u.ref) inx)
fod.u.ref (~(del by fod.u.ref) hen)
==
wake
:: something here, so validate
2019-05-17 03:24:10 +03:00
::
2020-05-13 06:22:25 +03:00
++ validate
|= =rand
^- (unit cage)
?- p.p.rand
%a ~| %no-big-ford-builds-across-network-for-now !!
%b ~| %i-guess-you-ought-to-build-your-own-marks !!
%c ~| %casts-should-be-compiled-on-your-own-ship !!
%d ~| %totally-temporary-error-please-replace-me !!
%e ~| %yes-naves-also-shouldnt-cross-the-network !!
%f ~| %even-static-casts-should-be-built-locally !!
%p ~| %requesting-foreign-permissions-is-invalid !!
%r ~| %no-cages-please-they-are-just-way-too-big !!
%s ~| %please-dont-get-your-takos-over-a-network !!
%t ~| %requesting-foreign-directory-is-vaporware !!
%v ~| %weird-shouldnt-get-v-request-from-network !!
%u `(validate-u r.rand)
%w `(validate-w r.rand)
%x (validate-x [p.p q.p q r]:rand)
%y `[p.r.rand !>(;;(arch q.r.rand))]
%z `(validate-z r.rand)
2020-05-13 06:22:25 +03:00
==
::
:: Make sure the incoming data is a %u response
::
++ validate-u
|= =page
^- cage
?> ?=(%flag p.page)
:- p.page
!> ;;(? q.page)
::
2020-05-13 06:22:25 +03:00
:: Make sure the incoming data is a %w response
::
++ validate-w
|= =page
^- cage
:- p.page
?+ p.page ~| %strange-w-over-nextwork !!
%cass !>(;;(cass q.page))
%null [[%atom %n ~] ~]
%nako !>(~|([%molding [&1 &2 &3]:q.page] ;;(nako q.page)))
2020-05-13 06:22:25 +03:00
==
::
:: Make sure that incoming data is of the mark it claims to be.
::
++ validate-x
|= [car=care cas=case pax=path peg=page]
^- (unit cage)
=/ vale-result
%- mule |.
%- wrap:fusion
2021-06-29 03:01:51 +03:00
:: Use %base's marks to validate, so we don't have to build the
:: foreign hoon/zuse
::
2021-06-29 03:01:51 +03:00
=/ args %*(static-ford-args . dom dom:(~(got by dos.rom) %base))
(page-to-cage:(ford:fusion args) peg)
2020-05-13 06:22:25 +03:00
?: ?=(%| -.vale-result)
%- (slog >%validate-x-failed< p.vale-result)
~
`-.p.vale-result
2020-05-23 00:32:32 +03:00
::
:: Make sure the incoming data is a %z response
::
++ validate-z
|= =page
^- cage
?> ?=(%uvi p.page)
:- p.page
!> ;;(@uvI q.page)
2020-05-13 06:22:25 +03:00
--
2019-05-17 03:24:10 +03:00
::
:: Respond to backfill request
2019-05-17 03:24:10 +03:00
::
:: Maybe should verify the requester is allowed to access this blob?
::
++ give-backfill
|= =lobe
^+ ..give-backfill
(emit hen %give %boon (blob-to-blub (lobe-to-blob:ze lobe)))
::
:: Ingest foreign update, requesting missing blobs if necessary
::
++ foreign-update
|= inx=@ud
2019-05-17 03:24:10 +03:00
?> ?=(^ ref)
=/ [sat=update-state lost=?]
=/ ruv (~(get by bom.u.ref) inx)
?~ ruv
~& [%clay-foreign-update-lost her syd inx]
[*update-state &]
[u.ruv |]
=/ done=? |
=. hen duct.sat
|%
++ abet
^+ ..foreign-update
?: lost
..foreign-update
?: done
=: bom.u.ref (~(del by bom.u.ref) inx)
fod.u.ref (~(del by fod.u.ref) hen)
==
=<(?>(?=(^ ref) .) wake)
=. bom.u.ref (~(put by bom.u.ref) inx sat)
..foreign-update
::
++ apex
|= rut=(unit rand)
^+ ..abet
?: lost ..abet
?~ rut
=. nako.sat (~(put to nako.sat) ~)
work
?> ?=(%nako p.r.u.rut)
=/ nako ;;(nako q.r.u.rut)
=/ missing (missing-blobs nako)
=. let.need.sat (welp let.need.sat ~(tap in let.missing))
=. old.need.sat (welp old.need.sat ~(tap in old.missing))
=. nako.sat (~(put to nako.sat) ~ nako)
work
2020-05-13 08:53:23 +03:00
::
++ missing-blobs
|= =nako
^- [let=(set lobe) old=(set lobe)]
=/ let-tako (~(got by gar.nako) let.nako)
=| let-lobes=(set lobe)
=| old-lobes=(set lobe)
=/ yakis ~(tap in lar.nako)
|- ^- [(set lobe) (set lobe)]
=* yaki-loop $
?~ yakis
[let-lobes old-lobes]
=/ lobes=(list [=path =lobe]) ~(tap by q.i.yakis)
|- ^- [(set lobe) (set lobe)]
=* blob-loop $
?~ lobes
yaki-loop(yakis t.yakis)
=/ lob (~(get by lat.ran) lobe.i.lobes)
?~ lob
=. old-lobes (~(put in old-lobes) lobe.i.lobes)
blob-loop(lobes t.lobes)
=? let-lobes &(=(let-tako r.i.yakis) ?=(%dead -.u.lob))
(~(put in let-lobes) lobe.i.lobes)
blob-loop(lobes t.lobes)
::
:: Receive backfill response
::
++ take-backfill
|= =blub
^+ ..abet
?: lost ..abet
=/ =blob (blub-to-blob blub)
?: ?& ?=(%dead -.blob)
(lien let.need.sat |=(=lobe =(p.blob lobe)))
==
%- (slog leaf+"clay: got dead backfill but need answer for {<p.blob>}" ~)
..abet
=? need.sat ?=(%delta -.blob)
?. (lien let.need.sat |=(=lobe =(p.blob lobe)))
?: ?& !(~(has by lat.ran) q.blob)
!(~(has by have.sat) q.blob)
==
[let.need.sat [q.blob old.need.sat]]
need.sat
=/ one (~(get by lat.ran) q.blob)
=/ two (~(get by have.sat) q.blob)
?: ?& ?| ?=(~ one)
?=(%dead -.u.one)
==
?| ?=(~ two)
?=(%dead -.u.two)
==
==
[[q.blob let.need.sat] old.need.sat]
need.sat
:: We can't put a blob in lat.ran if its parent isn't already
2022-03-26 01:32:38 +03:00
:: there.
::
=. ..abet
?: &(?=(%delta -.blob) !(~(has by lat.ran) q.blob))
2022-03-26 01:32:38 +03:00
..abet(have.sat (uni-blobs have.sat (malt [p.blob `^blob`blob] ~)))
..abet(lat.ran (uni-blobs lat.ran (malt [p.blob blob] ~)))
work(busy.sat |)
::
:: Fetch next blob
::
++ work
^+ ..abet
?: busy.sat
..abet
|- ^+ ..abet
?: =([~ ~] need.sat)
:: NB: if you change to release nakos as we get enough blobs
:: for them instead of all at the end, you *must* store the
:: `lim` that should be applied after the nako is complete and
:: not use the one in the rave, since that will apply to the
:: end of subscription.
::
2022-03-26 01:32:38 +03:00
=. lat.ran (uni-blobs lat.ran have.sat)
|- ^+ ..abet
?: =(~ nako.sat)
..abet
=^ next=(unit nako) nako.sat ~(get to nako.sat)
?~ next
..abet(done &)
=. ..abet (apply-foreign-update u.next)
=. ..foreign-update =<(?>(?=(^ ref) .) wake)
$
?~ let.need.sat
:: This is what removes an item from `need`. This happens every
:: time we take a backfill response, but it could happen more than
:: once if we somehow got this data in the meantime (maybe from
:: another desk updating concurrently, or a previous update on this
:: same desk).
::
?> ?=(^ old.need.sat)
?: ?| (~(has by lat.ran) i.old.need.sat)
(~(has by have.sat) i.old.need.sat)
==
$(old.need.sat t.old.need.sat)
(fetch i.old.need.sat)
=/ one (~(get by lat.ran) i.let.need.sat)
=/ two (~(get by have.sat) i.let.need.sat)
?: ?| ?& ?=(^ one)
!?=(%dead -.u.one)
==
?& ?=(^ two)
!?=(%dead -.u.two)
==
==
$(let.need.sat t.let.need.sat)
(fetch i.let.need.sat)
::
++ fetch
|= =lobe
^+ ..abet
=/ =fill [%0 syd lobe]
=/ =wire /back-index/(scot %p her)/[syd]/(scot %ud inx)
=/ =path [%backfill syd (scot %ud inx) ~]
=. ..foreign-update
=< ?>(?=(^ ref) .)
(emit hen %pass wire %a %plea her %c path fill)
..abet(busy.sat &)
2020-05-13 08:53:23 +03:00
::
:: When we get a %w foreign update, store this in our state.
::
:: We get the commits and blobs from the nako and add them to our
:: object store, then we update the map of aeons to commits and the
:: latest aeon.
::
++ apply-foreign-update
|= =nako
^+ ..abet
2020-05-13 08:53:23 +03:00
:: hit: updated commit-hashes by @ud case
:: nut: new commit-hash/commit pairs
:: hut: updated commits by hash
:: nat: new blob-hash/blob pairs
:: lat: updated blobs by hash
::
=/ hit (~(uni by hit.dom) gar.nako)
=/ nut (turn ~(tap in lar.nako) |=(=yaki [r.yaki yaki]))
=/ hut (~(uni by (malt nut)) hut.ran)
=/ nat (turn ~(tap in bar.nako) |=(=blub [p.blub (blub-to-blob blub)]))
2022-03-26 01:32:38 +03:00
=/ lat (uni-blobs lat.ran (malt nat))
2020-05-13 08:53:23 +03:00
:: traverse updated state and sanity check
::
=+ ~| :* %bad-foreign-update
[gar=gar.nako let=let.nako nut=(turn nut head) nat=(turn nat head)]
2020-05-13 08:53:23 +03:00
[hitdom=hit.dom letdom=let.dom]
==
?: =(0 let.nako)
~
=/ =aeon 1
|- ^- ~
=/ =tako
~| [%missing-aeon aeon] (~(got by hit) aeon)
=/ =yaki
~| [%missing-tako tako] (~(got by hut) tako)
=+ %+ turn
~(tap by q.yaki)
|= [=path =lobe]
~| [%missing-blob path lobe]
?> (~(has by lat) lobe)
~
?: =(let.nako aeon)
~
$(aeon +(aeon))
:: produce updated state
::
=/ =rave rave:(~(got by bom.u.ref) inx)
?> ?=(%many -.rave)
2020-05-13 08:53:23 +03:00
=: let.dom (max let.nako let.dom)
hit.dom hit
hut.ran hut
lat.ran lat
:: Is this correct? Seeems like it should only go to `to` if
:: we've gotten all the way to the end. Leaving this
:: behavior unchanged for now, but I believe it's wrong.
::
lim ?.(?=(%da -.to.moat.rave) lim p.to.moat.rave)
2020-05-13 08:53:23 +03:00
==
..abet
2020-05-13 08:53:23 +03:00
--
2019-05-17 03:24:10 +03:00
::
2019-05-23 00:39:12 +03:00
:: fire function if request is in future
::
++ run-if-future
|= [rov=rove fun=$-(@da _.)]
2016-11-24 07:25:07 +03:00
^+ +>.$
%+ fall
%+ bind
^- (unit @da)
?- -.rov
2019-05-23 00:39:12 +03:00
%sing
?. ?=(%da -.case.mood.rov) ~
`p.case.mood.rov
::
2019-05-23 00:39:12 +03:00
%next ~
%mult ~
%many
2016-12-02 22:34:07 +03:00
%^ hunt lth
?. ?=(%da -.from.moat.rov) ~
?. (lth now p.from.moat.rov) ~
[~ p.from.moat.rov]
2019-05-23 00:39:12 +03:00
?. ?=(%da -.to.moat.rov) ~
`(max now p.to.moat.rov)
2016-11-24 07:25:07 +03:00
==
fun
+>.$
::
2019-05-23 00:39:12 +03:00
++ rave-to-rove
|= rav=rave
2019-05-23 00:39:12 +03:00
^- rove
?- -.rav
%sing rav
%next [- mood ~ ~]:rav
%mult [- mool ~ ~ ~]:rav
%many [- track moat ~]:rav
==
::
++ rove-to-rave
|= rov=rove
2016-11-24 07:25:07 +03:00
^- rave
?- -.rov
2019-05-23 00:39:12 +03:00
%sing rov
%next [- mood]:rov
%mult [- mool]:rov
%many [- track moat]:rov
2016-11-24 07:25:07 +03:00
==
::
++ send-cards
|= [cards=(list card) ducts=(set duct)]
2019-05-23 00:39:12 +03:00
^+ ..wake
%- emil
%- zing
%+ turn cards
|= =card
%+ turn ~(tap by ducts)
|= =duct
[duct card]
2019-05-23 00:39:12 +03:00
::
:: Loop through open subscriptions and check if we can fill any of
:: them.
2016-11-24 07:25:07 +03:00
::
2019-05-23 00:39:12 +03:00
++ wake
2016-11-24 07:25:07 +03:00
^+ .
=/ subs=(list [=wove ducts=(set duct)]) ~(tap by qyx)
=| qux=cult
2016-11-24 07:25:07 +03:00
|- ^+ ..wake
?~ subs
..wake(qyx qux)
?: =(~ ducts.i.subs)
$(subs t.subs)
=^ [new-sub=(unit rove) cards=(list card)] fod.dom
(try-fill-sub wove.i.subs)
=. ..wake (send-cards cards ducts.i.subs)
=? qux ?=(^ new-sub)
=/ =wove [for.wove.i.subs u.new-sub]
%+ ~(put by qux) wove
(~(uni in ducts.i.subs) (fall (~(get by qux) wove) ~))
$(subs t.subs)
2019-05-23 00:39:12 +03:00
::
:: Try to fill a subscription
::
++ try-fill-sub
|= [far=(unit [=ship ver=@ud]) rov=rove]
^- [[(unit rove) (list card)] ford-cache]
=/ for=(unit ship) ?~(far ~ `ship.u.far)
?- -.rov
2019-05-23 00:39:12 +03:00
%sing
=/ cache-value=(unit (unit cage))
?~(ref ~ (~(get by haw.u.ref) mood.rov))
?^ cache-value
:: if we have a result in our cache, produce it
::
:_ fod.dom :- ~ :_ ~
(writ ?~(u.cache-value ~ `[mood.rov u.u.cache-value]))
2019-05-23 00:39:12 +03:00
:: else, check to see if rove is for an aeon we know
::
=/ aeon=(unit aeon) (case-to-aeon case.mood.rov)
?~ aeon
[[`rov ~] fod.dom]
2019-05-23 00:39:12 +03:00
:: we have the appropriate aeon, so read in the data
::
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
=^ value=(unit (unit cage)) fod.dom
2019-05-23 00:39:12 +03:00
(read-at-aeon:ze for u.aeon mood.rov)
?~ value
:: we don't have the data directly. how can we fetch it?
2019-05-23 00:39:12 +03:00
::
?: =(0 u.aeon)
~& [%clay-sing-indirect-data-0 `path`[syd '0' path.mood.rov]]
[[~ ~] fod.dom]
2019-05-23 00:39:12 +03:00
~& [%clay-sing-indirect-data desk=syd mood=mood.rov aeon=u.aeon]
[[`rov ~] fod.dom]
:: we have the data, so produce the results
2019-05-23 00:39:12 +03:00
::
:_ fod.dom :- ~ :_ ~
%- writ
?~ u.value
~
`[mood.rov u.u.value]
2016-11-24 07:25:07 +03:00
::
:: %next is just %mult with one path, so we pretend %next = %mult here.
2019-05-23 00:39:12 +03:00
::
?(%next %mult)
:: because %mult requests need to wait on multiple files for each
2019-05-23 00:39:12 +03:00
:: revision that needs to be checked for changes, we keep two
:: cache maps. {old} is the revision at {(dec aeon)}, {new} is
:: the revision at {aeon}. if we have no {aeon} yet, that means
:: it was still unknown last time we checked.
::
=* vor rov
|^
2019-05-23 00:39:12 +03:00
=/ rov=rove
?: ?=(%mult -.vor) vor
:* %mult
2019-05-23 00:39:12 +03:00
[case [[care path] ~ ~]]:mood.vor
aeon.vor
[[[care.mood.vor path.mood.vor] cach.vor] ~ ~]
~
==
2019-05-23 00:39:12 +03:00
?> ?=(%mult -.rov)
:: recurse here on next aeon if possible/needed.
::
|-
:: if we don't have an aeon yet, see if we have one now.
2019-05-23 00:39:12 +03:00
::
?~ aeon.rov
=/ aeon=(unit aeon) (case-to-aeon case.mool.rov)
:: if we still don't, wait.
2019-05-23 00:39:12 +03:00
::
?~ aeon [(store rov) fod.dom]
:: if we do, update the request and retry.
2019-05-23 00:39:12 +03:00
::
$(aeon.rov `+(u.aeon), old-cach.rov ~, new-cach.rov ~)
2018-01-22 18:01:00 +03:00
:: if old isn't complete, try filling in the gaps.
2019-05-23 00:39:12 +03:00
::
=^ o fod.dom
?: (complete old-cach.rov)
[old-cach.rov fod.dom]
2019-05-23 00:39:12 +03:00
(read-unknown mool.rov(case [%ud (dec u.aeon.rov)]) old-cach.rov)
=. old-cach.rov o
:: if the next aeon we want to compare is in the future, wait again.
2019-05-23 00:39:12 +03:00
::
=/ next-aeon=(unit aeon) (case-to-aeon [%ud u.aeon.rov])
?~ next-aeon [(store rov) fod.dom]
:: if new isn't complete, try filling in the gaps.
2019-05-23 00:39:12 +03:00
::
=^ n fod.dom
?: (complete new-cach.rov)
[new-cach.rov fod.dom]
2019-05-23 00:39:12 +03:00
(read-unknown mool.rov(case [%ud u.aeon.rov]) new-cach.rov)
=. new-cach.rov n
:: if new still isn't complete, wait again.
::
?. (complete new-cach.rov)
[(store rov) fod.dom]
:: if old not complete, give a result (possible false positive).
::
?: !(complete old-cach.rov)
:_ fod.dom
%- respond
%- malt
%+ murn ~(tap in paths.mool.rov)
|= [=care =path]
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
^- (unit [mood (unit cage)])
=/ cached (~(get by new-cach.rov) [care path])
?. ?=([~ ~ *] cached)
%- (slog 'clay: strange new-cache' >[care path cached]< ~)
~
`u=[[care [%ud let.dom] path] u.u.cached]
2019-05-23 00:39:12 +03:00
:: both complete, so check if anything has changed
::
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
=/ changes=(map mood (unit cage))
2019-05-23 00:39:12 +03:00
%+ roll ~(tap by old-cach.rov)
|= $: [[car=care pax=path] old-cach-value=cach]
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
changes=(map mood (unit cage))
2019-05-23 00:39:12 +03:00
==
=/ new-cach-value=cach (~(got by new-cach.rov) car pax)
?< |(?=(~ old-cach-value) ?=(~ new-cach-value))
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
=/ new-entry=(unit (pair mood (unit cage)))
2019-05-23 00:39:12 +03:00
=/ =mood [car [%ud u.aeon.rov] pax]
?~ u.new-cach-value
:: if new does not exist, always notify
::
`[mood ~]
2019-05-23 00:39:12 +03:00
?~ u.old-cach-value
:: added
::
`[mood `u.u.new-cach-value]
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
?: =([p q.q]:u.u.new-cach-value [p q.q]:u.u.old-cach-value)
2019-05-23 00:39:12 +03:00
:: unchanged
::
~
:: changed
::
`[mood `u.u.new-cach-value]
2019-05-23 00:39:12 +03:00
:: if changed, save the change
::
?~ new-entry
changes
(~(put by changes) u.new-entry)
:: if there are any changes, send response. if none, move on to
:: next aeon.
::
?^ changes [(respond changes) fod.dom]
2019-05-23 00:39:12 +03:00
$(u.aeon.rov +(u.aeon.rov), new-cach.rov ~)
::
:: check again later
::
2019-05-23 00:39:12 +03:00
++ store
|= rov=rove
^- [(unit rove) (list card)]
2019-05-23 00:39:12 +03:00
=/ new-rove=rove
?> ?=(%mult -.rov)
?: ?=(%mult -.vor) rov
?> ?=([* ~ ~] old-cach.rov)
=* one n.old-cach.rov
[%next [care.p.one case.mool.rov path.p.one] aeon.rov q.one]
[`new-rove ~]
::
2019-05-23 00:39:12 +03:00
:: send changes
::
++ respond
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
|= res=(map mood (unit cage))
^- [(unit rove) (list card)]
2019-05-23 00:39:12 +03:00
:- ~
?: ?=(%mult -.vor)
:_ ~
=/ moods ~(key by res)
=/ cas
?> ?=(^ moods)
[%da (case-to-date case.n.moods)]
=/ res
(~(run in moods) |=(m=mood [care.m path.m]))
=/ gift [%wris cas res]
?: ?=(^ ref)
[%slip %b %drip !>(gift)]
[%give gift]
2019-05-23 00:39:12 +03:00
?> ?=([* ~ ~] res)
:_ ~
%- writ
?~ q.n.res
~
`[p u.q]:n.res
::
2019-05-23 00:39:12 +03:00
:: no unknowns
::
++ complete
|= hav=(map (pair care path) cach)
?& ?=(^ hav)
(levy ~(tap by `(map (pair care path) cach)`hav) know)
==
::
2019-05-23 00:39:12 +03:00
:: know about file in cach
::
++ know |=([(pair care path) c=cach] ?=(^ c))
2019-05-23 00:39:12 +03:00
::
:: fill in the blanks
::
2019-05-23 00:39:12 +03:00
++ read-unknown
|= [=mool hav=(map (pair care path) cach)]
^- [_hav ford-cache]
2019-05-23 00:39:12 +03:00
=? hav ?=(~ hav)
%- malt ^- (list (pair (pair care path) cach))
%+ turn
~(tap in paths.mool)
|= [c=care p=path]
^- [[care path] cach]
[[c p] ~]
|- ^+ [hav fod.dom]
?~ hav [hav fod.dom]
=^ lef fod.dom $(hav l.hav)
=. l.hav lef
=^ rig fod.dom $(hav r.hav)
=. r.hav rig
=/ [[=care =path] =cach] n.hav
?^ cach
[hav fod.dom]
=^ q fod.dom (aver for care case.mool path)
=. q.n.hav q
[hav fod.dom]
--
2016-11-24 07:25:07 +03:00
::
2019-05-23 00:39:12 +03:00
%many
:_ fod.dom
2019-05-23 00:39:12 +03:00
=/ from-aeon (case-to-aeon from.moat.rov)
?~ from-aeon
:: haven't entered the relevant range, so do nothing
::
[`rov ~]
=/ to-aeon (case-to-aeon to.moat.rov)
=/ up-to ?~(to-aeon let.dom u.to-aeon)
=/ ver ?~(far %1 ver.u.far)
=/ new-lobes=(map path lobe)
(lobes-at-path:ze for up-to path.moat.rov)
=. from.moat.rov [%ud +(let.dom)]
=/ cards=(list card)
2019-05-23 00:39:12 +03:00
?: =(lobes.rov new-lobes)
:: if no changes, don't produce results
::
~
:: else changes, so produce them
::
=/ =cage
?: track.rov
[%null [%atom %n ~] ~]
[%nako !>((make-nako:ze ver u.from-aeon up-to))]
[(writ ~ [%w ud+let.dom /] cage) ~]
?~ to-aeon
:: we're in the middle of the range, so produce what we can,
:: but don't end the subscription
::
[`rov cards]
2019-05-23 00:39:12 +03:00
:: we're past the end of the range, so end subscription
::
[~ (snoc cards (writ ~))]
2016-11-24 07:25:07 +03:00
==
2019-05-23 00:39:12 +03:00
::
2016-11-24 07:25:07 +03:00
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
::
:: This core has no additional state, and the distinction exists purely for
:: documentation. The overarching theme is that `++de` directly contains
:: logic for metadata about the desk, while `++ze` is composed primarily
:: of helper functions for manipulating the desk state (`++dome`) itself.
:: Functions include:
::
:: -- converting between cases, commit hashes, commits, content hashes,
:: and content
:: -- creating commits and content and adding them to the tree
:: -- finding which data needs to be sent over the network to keep the
2018-02-08 00:28:33 +03:00
:: other urbit up-to-date
2016-11-24 07:25:07 +03:00
:: -- reading from the file tree through different `++care` options
:: -- the `++me` core for merging.
::
:: The dome is composed of the following:
::
:: -- `ank` is the ankh, which is the file data itself. An ankh is both
:: a possible file and a possible directory. An ankh has both:
:: -- `fil`, a possible file, stored as both a cage and its hash
:: -- `dir`, a map of @ta to more ankhs.
:: -- `let` is the number of the most recent revision.
:: -- `hit` is a map of revision numbers to commit hashes.
:: -- `lab` is a map of labels to revision numbers.
::
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
2019-05-14 08:20:46 +03:00
::
::
:: Other utility functions
::
2016-11-24 07:25:07 +03:00
++ ze
|%
:: These convert between aeon (version number), tako (commit hash), yaki
:: (commit data structure), lobe (content hash), and blob (content).
::
2016-11-24 07:25:07 +03:00
++ aeon-to-tako ~(got by hit.dom)
++ aeon-to-yaki |=(=aeon (tako-to-yaki (aeon-to-tako aeon)))
2016-11-24 07:25:07 +03:00
++ lobe-to-blob ~(got by lat.ran)
++ tako-to-yaki ~(got by hut.ran)
2020-05-12 10:37:04 +03:00
++ lobe-to-mark
|= a=lobe
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
^- (unit mark)
2020-05-12 10:37:04 +03:00
=> (lobe-to-blob a)
?- -
%delta `p.r
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
%direct `p.q
%dead ~
2020-05-12 10:37:04 +03:00
==
2016-11-24 07:25:07 +03:00
::
:: Gets a map of the data at the given path and all children of it.
::
++ lobes-at-path
|= [for=(unit ship) yon=aeon pax=path]
2016-11-24 07:25:07 +03:00
^- (map path lobe)
?: =(0 yon) ~
:: we use %z for the check because it looks at all child paths.
2019-05-23 04:15:57 +03:00
?. |(?=(~ for) (may-read u.for %z yon pax)) ~
2016-11-24 07:25:07 +03:00
%- malt
%+ skim
2018-04-04 21:15:10 +03:00
%~ tap by
2018-04-04 21:07:05 +03:00
=< q
%- aeon-to-yaki
yon
|= [p=path q=lobe]
2018-03-19 07:18:20 +03:00
?| ?=(~ pax)
?& !?=(~ p)
2016-11-24 07:25:07 +03:00
=(-.pax -.p)
$(p +.p, pax +.pax)
== ==
::
:: Creates a nako of all the changes between a and b.
::
++ make-nako
|= [ver=@ud a=aeon b=aeon]
2016-11-24 07:25:07 +03:00
^- nako
:+ ?> (lte b let.dom)
|-
?: =(b let.dom)
hit.dom
2021-04-20 06:46:46 +03:00
:: del everything after b
2016-11-24 07:25:07 +03:00
$(hit.dom (~(del by hit.dom) let.dom), let.dom (dec let.dom))
b
?: =(0 b)
[~ ~]
2021-04-20 06:46:46 +03:00
=/ excludes=(set tako)
=| acc=(set tako)
=/ lower=@ud 1
|-
:: a should be excluded, so wait until we're past it
?: (gte lower +(a))
2021-04-20 06:46:46 +03:00
acc
=/ res=(set tako) (reachable-takos (~(got by hit.dom) lower))
$(acc (~(uni in acc) res), lower +(lower))
=/ includes=(set tako)
=| acc=(set tako)
=/ upper=@ud b
|-
?: (lte upper a)
2021-04-20 06:46:46 +03:00
acc
=/ res=(set tako) (reachable-takos (~(got by hit.dom) upper))
$(acc (~(uni in acc) res), upper (dec upper))
[(~(run in (~(dif in includes) excludes)) tako-to-yaki) ~]
2020-05-12 10:37:04 +03:00
:: Traverse parentage and find all ancestor hashes
::
++ reachable-takos :: reachable
|= p=tako
2020-05-12 10:37:04 +03:00
^- (set tako)
~+
=| s=(set tako)
|- ^- (set tako)
=. s (~(put in s) p)
2020-05-12 10:37:04 +03:00
=+ y=(tako-to-yaki p)
|- ^- (set tako)
?~ p.y
s
?: (~(has in s) i.p.y)
$(p.y t.p.y)
=. s ^$(p i.p.y)
$(p.y t.p.y)
2020-05-12 10:37:04 +03:00
::
++ read-a
2021-02-24 20:44:08 +03:00
!.
|= [=aeon =path]
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
^- [(unit (unit cage)) ford-cache]
=^ =vase fod.dom
2021-07-23 14:03:36 +03:00
~_ leaf/"clay: %a build failed {<[syd aeon path]>}"
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
%+ aeon-ford-cache aeon
%- wrap:fusion
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
(build-file:(aeon-ford aeon) path)
:_(fod.dom [~ ~ %vase !>(vase)])
::
++ read-b
2021-02-24 20:44:08 +03:00
!.
|= [=aeon =path]
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
^- [(unit (unit cage)) ford-cache]
?. ?=([@ ~] path)
[[~ ~] fod.dom]
=^ =dais fod.dom
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
%+ aeon-ford-cache aeon
%- wrap:fusion
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
(build-dais:(aeon-ford aeon) i.path)
:_(fod.dom [~ ~ %dais !>(dais)])
::
++ read-c
2021-02-24 20:44:08 +03:00
!.
|= [=aeon =path]
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
^- [(unit (unit cage)) ford-cache]
?. ?=([@ @ ~] path)
[[~ ~] fod.dom]
=^ =tube fod.dom
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
%+ aeon-ford-cache aeon
%- wrap:fusion
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
(build-tube:(aeon-ford aeon) [i i.t]:path)
:_(fod.dom [~ ~ %tube !>(tube)])
::
++ read-e
!.
|= [=aeon =path]
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
^- [(unit (unit cage)) ford-cache]
?. ?=([@ ~] path)
[[~ ~] fod.dom]
=^ =vase fod.dom
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
%+ aeon-ford-cache aeon
%- wrap:fusion
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
(build-nave:(aeon-ford aeon) i.path)
:_(fod.dom [~ ~ %nave vase])
::
++ read-f
!.
|= [=aeon =path]
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
^- [(unit (unit cage)) ford-cache]
?. ?=([@ @ ~] path)
[[~ ~] fod.dom]
=^ =vase fod.dom
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
%+ aeon-ford-cache aeon
%- wrap:fusion
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
(build-cast:(aeon-ford aeon) [i i.t]:path)
:_(fod.dom [~ ~ %cast vase])
::
++ read-d
!.
|= [=aeon =path]
^- (unit (unit cage))
?. =(our her)
[~ ~]
?^ path
~&(%no-cd-path [~ ~])
[~ ~ %noun !>(~(key by dos.rom.ruf))]
::
2018-02-07 23:11:25 +03:00
:: Gets the permissions that apply to a particular node.
::
:: If the node has no permissions of its own, we use its parent's.
:: If no permissions have been set for the entire tree above the node,
:: we default to fully private (empty whitelist).
::
++ read-p
|= pax=path
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
^- (unit (unit cage))
=- [~ ~ %noun !>(-)]
2018-02-07 23:11:25 +03:00
:- (read-p-in pax per.red)
(read-p-in pax pew.red)
::
++ read-p-in
|= [pax=path pes=regs]
2018-02-07 23:11:25 +03:00
^- dict
=/ rul=(unit rule) (~(get by pes) pax)
2018-06-19 02:10:11 +03:00
?^ rul
:+ pax mod.u.rul
%- ~(rep in who.u.rul)
|= [w=whom out=(pair (set ship) (map @ta crew))]
?: ?=([%& @p] w)
[(~(put in p.out) +.w) q.out]
=/ cru=(unit crew) (~(get by cez.ruf) +.w)
2018-06-19 02:10:11 +03:00
?~ cru out
[p.out (~(put by q.out) +.w u.cru)]
?~ pax [/ %white ~ ~]
$(pax (scag (dec (lent pax)) `path`pax))
::
++ may-read
|= [who=ship car=care yon=aeon pax=path]
^- ?
?+ car
(allowed-by who pax per.red)
2018-02-07 23:11:25 +03:00
::
%p
2018-02-07 23:11:25 +03:00
=(who our)
::
?(%y %z)
=+ tak=(~(get by hit.dom) yon)
?~ tak |
=+ yak=(tako-to-yaki u.tak)
=+ len=(lent pax)
=- (levy ~(tap in -) |=(p=path (allowed-by who p per.red)))
%+ roll ~(tap in (~(del in ~(key by q.yak)) pax))
|= [p=path s=(set path)]
?. =(pax (scag len p)) s
%- ~(put in s)
?: ?=(%z car) p
(scag +(len) p)
==
::
++ may-write
|= [w=ship p=path]
(allowed-by w p pew.red)
::
++ allowed-by
|= [who=ship pax=path pes=regs]
^- ?
=/ rul=real rul:(read-p-in pax pes)
2018-06-20 20:12:26 +03:00
=/ in-list/?
?| (~(has in p.who.rul) who)
2018-06-20 01:27:16 +03:00
::
%- ~(rep by q.who.rul)
|= [[@ta cru=crew] out=_|]
?: out &
(~(has in cru) who)
==
2018-06-19 02:10:11 +03:00
?: =(%black mod.rul)
!in-list
in-list
2020-03-28 06:37:22 +03:00
:: +content-hash: get hash of contents (%cz hash)
::
++ content-hash
|= [=yaki pax=path]
^- @uvI
=+ len=(lent pax)
=/ descendants=(list (pair path lobe))
2020-03-28 06:37:22 +03:00
%+ turn
%+ skim ~(tap by (~(del by q.yaki) pax))
|= [paf=path lob=lobe]
2020-03-28 06:37:22 +03:00
=(pax (scag len paf))
|= [paf=path lob=lobe]
2020-03-28 06:37:22 +03:00
[(slag len paf) lob]
=+ us=(~(get by q.yaki) pax)
?: &(?=(~ descendants) ?=(~ us))
*@uvI
%+ roll
^- (list (pair path lobe))
[[~ ?~(us *lobe u.us)] descendants]
|=([[path lobe] @uvI] (shax (jam +<)))
2020-05-14 05:28:04 +03:00
:: +read-r: %x wrapped in a vase
::
++ read-r
|= [yon=aeon pax=path]
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
^- [(unit (unit cage)) ford-cache]
=^ x fod.dom (read-x yon pax)
:_ fod.dom
2020-05-14 05:28:04 +03:00
?~ x ~
?~ u.x [~ ~]
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
``[p.u.u.x !>(q.u.u.x)]
:: +read-s: produce yaki or blob for given tako or lobe
::
++ read-s
|= [yon=aeon pax=path]
2020-03-28 06:37:22 +03:00
^- (unit (unit cage))
?. ?=([?(%yaki %blob %hash %cage %open %late %base) * *] pax)
`~
2020-03-28 06:37:22 +03:00
?- i.pax
%yaki
=/ yak=(unit yaki) (~(get by hut.ran) (slav %uv i.t.pax))
?~ yak
~
``yaki+[-:!>(*yaki) u.yak]
2020-03-28 06:37:22 +03:00
::
%blob
=/ bol=(unit blob) (~(get by lat.ran) (slav %uv i.t.pax))
?~ bol
~
``blob+[-:!>(*blob) u.bol]
::
%hash
=/ yak=(unit yaki) (~(get by hut.ran) (slav %uv i.t.pax))
?~ yak
~
``uvi+[-:!>(*@uvI) (content-hash u.yak /)]
2020-05-01 05:32:48 +03:00
::
%cage
:: should save ford cache
::
=/ =lobe (slav %uv i.t.pax)
=^ =page fod.dom
%- wrap:fusion
(lobe-to-page:(ford:fusion static-ford-args) lobe)
2020-05-01 05:32:48 +03:00
=^ =cage fod.dom
%- wrap:fusion
(page-to-cage:(ford:fusion static-ford-args) page)
2020-05-01 05:32:48 +03:00
``cage+[-:!>(*^cage) cage]
2020-05-14 09:26:52 +03:00
::
%open
``open+!>(prelude:(ford:fusion static-ford-args))
2020-05-23 00:32:32 +03:00
::
2020-07-21 22:03:30 +03:00
%late !! :: handled in +aver
%base
?> ?=(^ t.t.pax)
:^ ~ ~ %uvs !>
^- (list @uv)
=/ him (slav %p i.t.pax)
2020-12-06 11:38:37 +03:00
=/ other dom:((de now rof hen ruf) him i.t.t.pax)
?: =(0 let.other)
~
=/ our-yaki (~(got by hut.ran) (~(got by hit.dom) yon))
=/ other-yaki (~(got by hut.ran) (~(got by hit.other) let.other))
%+ turn ~(tap in (find-merge-points other-yaki our-yaki))
|= =yaki
r.yaki
2020-03-28 06:37:22 +03:00
==
2018-08-22 22:56:46 +03:00
:: +read-t: produce the list of paths within a yaki with :pax as prefix
::
++ read-t
|= [yon=aeon pax=path]
^- (unit (unit [%file-list (hypo (list path))]))
:: if asked for version 0, produce an empty list of files
::
?: =(0 yon)
``[%file-list -:!>(*(list path)) *(list path)]
:: if asked for a future version, we don't have an answer
::
?~ tak=(~(get by hit.dom) yon)
~
:: look up the yaki snapshot based on the version
::
=/ yak=yaki (tako-to-yaki u.tak)
:: calculate the path length once outside the loop
::
=/ path-length (lent pax)
::
:^ ~ ~ %file-list
:- -:!>(*(list path))
^- (list path)
:: sort the matching paths alphabetically
::
=- (sort - aor)
:: traverse the filesystem, filtering for paths with :pax as prefix
::
%+ skim ~(tap in ~(key by q.yak))
|=(paf=path =(pax (scag path-length paf)))
::
2016-11-24 07:25:07 +03:00
:: Checks for existence of a node at an aeon.
::
:: This checks for existence of content at the node, and does *not* look
:: at any of its children.
::
++ read-u
|= [yon=aeon pax=path]
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
^- (unit (unit [%flag (hypo ?)]))
:: if asked for version 0, that never exists, so always give false
::
?: =(0 yon)
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
``[%flag -:!>(*?) |]
:: if asked for a future version, we don't have an answer
::
?~ tak=(~(get by hit.dom) yon)
2016-11-24 07:25:07 +03:00
~
:: look up the yaki snapshot based on the version
::
=/ yak=yaki (tako-to-yaki u.tak)
:: produce the result based on whether or not there's a file at :pax
::
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
``[%flag -:!>(*?) (~(has by q.yak) pax)]
2016-11-24 07:25:07 +03:00
::
:: Gets the dome (desk state) at a particular aeon.
::
:: For past aeons, we don't give an actual ankh in the dome, but the rest
:: of the data is legit. We also never send the mime cache over the wire.
2016-11-24 07:25:07 +03:00
::
++ read-v
|= [yon=aeon pax=path]
^- (unit (unit [%dome (hypo dome:clay)]))
2016-11-24 07:25:07 +03:00
?: (lth yon let.dom)
2020-06-25 07:13:31 +03:00
:* ~ ~ %dome -:!>(*dome:clay)
^- dome:clay
:* ank=`[[%ank-in-old-v-not-implemented *ankh] ~ ~]
let=yon
hit=(molt (skim ~(tap by hit.dom) |=([p=@ud *] (lte p yon))))
lab=(molt (skim ~(tap by lab.dom) |=([* p=@ud] (lte p yon))))
== ==
2016-11-24 07:25:07 +03:00
?: (gth yon let.dom)
~
``[%dome -:!>(*dome:clay) [ank let hit lab]:dom]
2016-11-24 07:25:07 +03:00
::
:: Gets all cases refering to the same revision as the given case.
::
:: For the %da case, we give just the canonical timestamp of the revision.
::
++ read-w
|= cas=case
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
^- (unit (unit cage))
2019-05-04 05:24:24 +03:00
=+ aey=(case-to-aeon cas)
?~ aey ~
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
=- [~ ~ %cass !>(-)]
^- cass
:- u.aey
?: =(0 u.aey) `@da`0
t:(aeon-to-yaki u.aey)
::
2020-05-12 10:37:04 +03:00
:: Get the data at a node.
2016-11-24 07:25:07 +03:00
::
2020-05-12 10:37:04 +03:00
:: If it's in our ankh (current state cache), we can just produce
:: the result. Otherwise, we've got to look up the node at the
:: aeon to get the content hash, use that to find the blob, and use
:: the blob to get the data. We also special-case the hoon mark
:: for bootstrapping purposes.
::
++ read-x
|= [yon=aeon pax=path]
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
^- [(unit (unit cage)) ford-cache]
2020-05-12 10:37:04 +03:00
?: =(0 yon)
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
[[~ ~] fod.dom]
2020-05-12 10:37:04 +03:00
=+ tak=(~(get by hit.dom) yon)
?~ tak
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
[~ fod.dom]
?: &(?=(~ ref) =(let.dom yon))
:_ fod.dom :- ~
2020-05-12 10:37:04 +03:00
%+ bind
2022-04-06 22:16:22 +03:00
(~(get ^de ank.dom) pax)
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
|=(a=[p=lobe q=cage] q.a)
2020-05-12 10:37:04 +03:00
=+ yak=(tako-to-yaki u.tak)
=+ lob=(~(get by q.yak) pax)
?~ lob
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
[[~ ~] fod.dom]
2020-05-12 10:37:04 +03:00
=+ mar=(lobe-to-mark u.lob)
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
:: if tombstoned, nothing to return
::
?~ mar
[~ fod.dom]
2020-05-12 10:37:04 +03:00
:: should convert any lobe to cage
::
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
?: ?=(%hoon u.mar)
=/ txt
|- ^- (unit @t)
=+ bol=(lobe-to-blob u.lob)
?- -.bol
%dead ~
%direct `;;(@t q.q.bol)
%delta
=+ txt=$(u.lob q.bol)
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
?~ txt
~
?> ?=(%txt-diff p.r.bol)
=+ dif=;;((urge cord) q.r.bol)
=, format
=+ pac=(of-wain (lurk:differ (to-wain (cat 3 u.txt '\0a')) dif))
:- ~
?~ pac
''
(end [3 (dec (met 3 pac))] pac)
==
:_ fod.dom
?~ txt
~
[~ ~ u.mar [%atom %t ~] u.txt]
=^ peg=(unit page) fod.dom
%- wrap:fusion
(lobe-to-unit-page:(ford:fusion static-ford-args) u.lob)
?~ peg
[~ fod.dom]
=^ =cage fod.dom
%- wrap:fusion
(page-to-cage:(ford:fusion static-ford-args) u.peg)
[``cage fod.dom]
2016-11-24 07:25:07 +03:00
::
:: Gets an arch (directory listing) at a node.
::
++ read-y
|= [yon=aeon pax=path]
^- (unit (unit [%arch (hypo arch)]))
2016-11-24 07:25:07 +03:00
?: =(0 yon)
``[%arch -:!>(*arch) *arch]
=+ tak=(~(get by hit.dom) yon)
?~ tak
~
=+ yak=(tako-to-yaki u.tak)
=+ len=(lent pax)
:^ ~ ~ %arch
:: ~& cy+pax
:- -:!>(*arch)
^- arch
:- (~(get by q.yak) pax)
2018-03-19 07:18:20 +03:00
^- (map knot ~)
%- molt ^- (list (pair knot ~))
2016-11-24 07:25:07 +03:00
%+ turn
^- (list (pair path lobe))
%+ skim ~(tap by (~(del by q.yak) pax))
|= [paf=path lob=lobe]
2016-11-24 07:25:07 +03:00
=(pax (scag len paf))
|= [paf=path lob=lobe]
2016-11-24 07:25:07 +03:00
=+ pat=(slag len paf)
[?>(?=(^ pat) i.pat) ~]
::
:: Gets a recursive hash of a node and all its children.
::
++ read-z
|= [yon=aeon pax=path]
^- (unit (unit [%uvi (hypo @uvI)]))
2016-11-24 07:25:07 +03:00
?: =(0 yon)
``uvi+[-:!>(*@uvI) *@uvI]
=+ tak=(~(get by hit.dom) yon)
?~ tak
~
2020-03-28 06:37:22 +03:00
[~ ~ %uvi -:!>(*@uvI) (content-hash (tako-to-yaki u.tak) pax)]
2016-11-24 07:25:07 +03:00
::
:: Get a value at an aeon.
::
2019-05-23 00:39:12 +03:00
:: Value can be either null, meaning we don't have it yet, [null null],
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
:: meaning we know it doesn't exist, or [null null cage],
2016-11-24 07:25:07 +03:00
:: meaning we either have the value directly or a content hash of the
:: value.
::
++ read-at-aeon :: read-at-aeon:ze
2019-05-03 04:50:20 +03:00
|= [for=(unit ship) yon=aeon mun=mood] :: seek and read
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
^- [(unit (unit cage)) ford-cache]
2019-05-23 00:39:12 +03:00
?. |(?=(~ for) (may-read u.for care.mun yon path.mun))
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
[~ fod.dom]
:: virtualize to catch and produce deterministic failures
::
2021-09-21 00:13:04 +03:00
!:
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
|^ =/ res (mule |.(read))
?: ?=(%& -.res) p.res
%. [[~ ~] fod.dom]
(slog leaf+"clay: read-at-aeon fail {<[desk=syd mun]>}" p.res)
::
++ read
^- [(unit (unit cage)) ford-cache]
?- care.mun
%a (read-a yon path.mun)
%b (read-b yon path.mun)
%c (read-c yon path.mun)
%d [(read-d yon path.mun) fod.dom]
%e (read-e yon path.mun)
%f (read-f yon path.mun)
%p [(read-p path.mun) fod.dom]
%r (read-r yon path.mun)
%s [(read-s yon path.mun) fod.dom]
%t [(read-t yon path.mun) fod.dom]
%u [(read-u yon path.mun) fod.dom]
%v [(read-v yon path.mun) fod.dom]
%w [(read-w case.mun) fod.dom]
%x (read-x yon path.mun)
%y [(read-y yon path.mun) fod.dom]
%z [(read-z yon path.mun) fod.dom]
==
--
2016-11-24 07:25:07 +03:00
--
--
--
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: section 4cA, filesystem vane
::
:: This is the arvo interface vane. Our formal state is a `++raft`, which
:: has five components:
::
2018-12-13 10:42:15 +03:00
:: -- `rom` is the state for all local desks.
2016-11-24 07:25:07 +03:00
:: -- `hoy` is the state for all foreign desks.
:: -- `ran` is the global, hash-addressed object store.
:: -- `mon` is the set of mount points in unix.
:: -- `hez` is the duct to the unix sync.
::
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
=| :: instrument state
$: ver=%12 :: vane version
2019-02-02 00:46:09 +03:00
ruf=raft :: revision tree
2016-11-24 07:25:07 +03:00
== ::
2020-12-06 11:38:37 +03:00
|= [now=@da eny=@uvJ rof=roof] :: current invocation
2021-03-28 10:47:37 +03:00
~% %clay-top ..part ~
2016-11-24 07:25:07 +03:00
|% ::
++ call :: handle request
2021-03-28 10:47:37 +03:00
~/ %clay-call
2018-12-04 00:22:39 +03:00
|= $: hen=duct
2020-02-11 01:03:03 +03:00
dud=(unit goof)
2020-12-08 03:47:06 +03:00
wrapped-task=(hobo task)
2016-11-24 07:25:07 +03:00
==
2019-05-11 00:51:37 +03:00
^- [(list move) _..^$]
2018-12-04 00:22:39 +03:00
::
2020-12-08 03:47:06 +03:00
=/ req=task ((harden task) wrapped-task)
::
:: TODO handle error notifications
::
2020-12-08 05:01:48 +03:00
?^ dud
[[[hen %slip %d %flog %crud [-.req tang.u.dud]] ~] ..^$]
::
2018-02-09 15:33:15 +03:00
?- -.req
2019-05-11 00:51:37 +03:00
%boat
2016-11-24 07:25:07 +03:00
:_ ..^$
[hen %give %hill (turn ~(tap by mon.ruf) head)]~
2019-07-25 01:54:45 +03:00
::
2019-05-11 00:51:37 +03:00
%cred
=. cez.ruf
2018-02-09 15:33:15 +03:00
?~ cew.req (~(del by cez.ruf) nom.req)
(~(put by cez.ruf) nom.req cew.req)
:: wake all desks, a request may have been affected.
=| mos=(list move)
2018-12-13 10:42:15 +03:00
=/ des ~(tap in ~(key by dos.rom.ruf))
|-
2019-06-29 02:43:52 +03:00
?~ des [[[hen %give %done ~] mos] ..^^$]
2020-12-06 11:38:37 +03:00
=/ den ((de now rof hen ruf) our i.des)
=^ mor ruf
=< abet:wake
?: ?=(^ cew.req) den
(forget-crew:den nom.req)
$(des t.des, mos (weld mos mor))
::
2019-05-11 00:51:37 +03:00
%crew
[[hen %give %cruz cez.ruf]~ ..^$]
::
2019-05-11 00:51:37 +03:00
%crow
2018-12-13 10:42:15 +03:00
=/ des ~(tap by dos.rom.ruf)
=| rus=(map desk [r=regs w=regs])
|^
?~ des [[hen %give %croz rus]~ ..^^$]
=+ per=(filter-rules per.q.i.des)
=+ pew=(filter-rules pew.q.i.des)
=? rus |(?=(^ per) ?=(^ pew))
(~(put by rus) p.i.des per pew)
$(des t.des)
::
++ filter-rules
|= pes=regs
^+ pes
=- (~(gas in *regs) -)
%+ skim ~(tap by pes)
|= [p=path r=rule]
(~(has in who.r) |+nom.req)
--
2016-11-24 07:25:07 +03:00
::
2019-05-11 00:51:37 +03:00
%drop
~& %clay-idle
[~ ..^$]
2016-11-24 07:25:07 +03:00
::
2019-05-11 00:51:37 +03:00
%info
2020-05-12 05:26:22 +03:00
?: ?=(%| -.dit.req)
=/ bel=@tas p.dit.req
=/ aey=(unit aeon) q.dit.req
=^ mos ruf
=/ den ((de now rof hen ruf) our des.req)
abet:(label:den bel aey)
[mos ..^$]
2020-05-12 05:26:22 +03:00
=/ [deletes=(set path) changes=(map path cage)]
=/ =soba p.dit.req
=| deletes=(set path)
=| changes=(map path cage)
|- ^+ [deletes changes]
?~ soba
[deletes changes]
?- -.q.i.soba
%del $(soba t.soba, deletes (~(put in deletes) p.i.soba))
%ins $(soba t.soba, changes (~(put by changes) [p p.q]:i.soba))
%mut $(soba t.soba, changes (~(put by changes) [p p.q]:i.soba))
%dif ~|(%dif-not-implemented !!)
==
2016-11-24 07:25:07 +03:00
=^ mos ruf
2020-12-06 11:38:37 +03:00
=/ den ((de now rof hen ruf) our des.req)
2020-05-12 05:26:22 +03:00
abet:(info:den deletes changes)
2016-11-24 07:25:07 +03:00
[mos ..^$]
::
2019-05-11 00:51:37 +03:00
%init
2022-03-26 02:41:54 +03:00
[~ ..^$(hun.rom.ruf hen)]
2016-11-24 07:25:07 +03:00
::
2019-05-11 00:51:37 +03:00
%into
2016-11-24 07:25:07 +03:00
=. hez.ruf `hen
2018-02-09 16:48:08 +03:00
=+ bem=(~(get by mon.ruf) des.req)
?: &(?=(~ bem) !=(%$ des.req))
2018-02-09 16:48:08 +03:00
~|([%bad-mount-point-from-unix des.req] !!)
=/ bem=beam
2016-11-24 07:25:07 +03:00
?^ bem
u.bem
2021-07-14 01:00:13 +03:00
[[our %base %ud 1] ~] :: TODO: remove this fallback?
2018-12-13 10:42:15 +03:00
=/ dos (~(get by dos.rom.ruf) q.bem)
2016-11-24 07:25:07 +03:00
?~ dos
2019-05-11 05:23:13 +03:00
!! :: fire next in queue
2020-04-24 03:12:23 +03:00
=^ mos ruf
2020-12-06 11:38:37 +03:00
=/ den ((de now rof hen ruf) our q.bem)
abet:(into:den s.bem all.req fis.req)
2020-04-24 03:12:23 +03:00
[mos ..^$]
2016-11-24 07:25:07 +03:00
::
2019-05-11 00:51:37 +03:00
%merg :: direct state up
2018-02-09 16:48:08 +03:00
?: =(%$ des.req)
2019-05-03 04:06:31 +03:00
~&(%merg-no-desk !!)
2016-11-24 07:25:07 +03:00
=^ mos ruf
2020-12-06 11:38:37 +03:00
=/ den ((de now rof hen ruf) our des.req)
2020-05-12 10:01:46 +03:00
abet:(start-merge:den her.req dem.req cas.req how.req)
2016-11-24 07:25:07 +03:00
[mos ..^$]
2021-04-20 06:46:46 +03:00
::
%fuse
?: =(%$ des.req)
~&(%fuse-no-desk !!)
=^ mos ruf
=/ den ((de now rof hen ruf) our des.req)
abet:(start-fuse:den bas.req con.req)
2021-04-20 06:46:46 +03:00
[mos ..^$]
2016-11-24 07:25:07 +03:00
::
2019-05-11 00:51:37 +03:00
%mont
2016-11-24 07:25:07 +03:00
=. hez.ruf ?^(hez.ruf hez.ruf `[[%$ %sync ~] ~])
=^ mos ruf
2020-12-06 11:38:37 +03:00
=/ den ((de now rof hen ruf) p.bem.req q.bem.req)
2020-05-12 11:27:19 +03:00
abet:(mount:den pot.req r.bem.req s.bem.req)
2016-11-24 07:25:07 +03:00
[mos ..^$]
2017-01-12 18:50:35 +03:00
::
2019-05-11 00:51:37 +03:00
%dirk
2017-01-12 18:50:35 +03:00
?~ hez.ruf
~& %no-sync-duct
[~ ..^$]
2018-02-09 16:48:08 +03:00
?. (~(has by mon.ruf) des.req)
~& [%not-mounted des.req]
2017-01-12 18:50:35 +03:00
[~ ..^$]
2020-05-12 11:27:19 +03:00
[~[[u.hez.ruf %give %dirk des.req]] ..^$]
2016-11-24 07:25:07 +03:00
::
2019-05-11 00:51:37 +03:00
%ogre
2016-11-24 07:25:07 +03:00
?~ hez.ruf
~& %no-sync-duct
[~ ..^$]
2018-02-09 16:48:08 +03:00
=* pot pot.req
?@ pot
?. (~(has by mon.ruf) pot)
~& [%not-mounted pot]
2016-11-24 07:25:07 +03:00
[~ ..^$]
2018-02-09 16:48:08 +03:00
:_ ..^$(mon.ruf (~(del by mon.ruf) pot))
[u.hez.ruf %give %ogre pot]~
2016-11-24 07:25:07 +03:00
:_ %_ ..^$
mon.ruf
%- molt
%+ skip ~(tap by mon.ruf)
2018-02-09 16:48:08 +03:00
(corl (cury test pot) tail)
2016-11-24 07:25:07 +03:00
==
%+ turn
2018-02-09 16:48:08 +03:00
(skim ~(tap by mon.ruf) (corl (cury test pot) tail))
|= [pon=term bem=beam]
2018-02-09 16:48:08 +03:00
[u.hez.ruf %give %ogre pon]
::
%park
=^ mos ruf
2020-12-06 11:38:37 +03:00
=/ den ((de now rof hen ruf) our des.req)
2020-04-30 03:38:24 +03:00
abet:(park:den | [yok ran]:req)
2020-04-25 03:44:29 +03:00
[mos ..^$]
::
%pork
2020-04-30 03:38:24 +03:00
=/ [syd=desk =yoki] (need pud.ruf)
2020-04-25 03:44:29 +03:00
=. pud.ruf ~
=^ mos ruf
2020-12-06 11:38:37 +03:00
=/ den ((de now rof hen ruf) our syd)
2020-04-30 03:38:24 +03:00
abet:(park:den & yoki *rang)
[mos ..^$]
::
2019-05-11 00:51:37 +03:00
%perm
=^ mos ruf
2020-12-06 11:38:37 +03:00
=/ den ((de now rof hen ruf) our des.req)
2018-02-09 15:33:15 +03:00
abet:(perm:den pax.req rit.req)
[mos ..^$]
2022-03-25 23:13:09 +03:00
::
2022-04-06 10:18:23 +03:00
%tomb (tomb-clue:tomb clue.req)
%trim [~ ..^$]
::
2020-06-12 05:41:22 +03:00
%vega
:: wake all desks, then send pending notifications
::
=^ wake-moves ..^$
=/ desks=(list [=ship =desk])
%+ welp
(turn ~(tap by dos.rom.ruf) |=([=desk *] [our desk]))
%- zing
%+ turn ~(tap by hoy.ruf)
|= [=ship =rung]
%+ turn ~(tap by rus.rung)
|= [=desk *]
[ship desk]
|- ^+ [*(list move) ..^^$]
?~ desks
[~ ..^^$]
=^ moves-1 ..^^$ $(desks t.desks)
2020-12-06 11:38:37 +03:00
=^ moves-2 ruf abet:wake:((de now rof hen ruf) [ship desk]:i.desks)
2020-06-12 05:41:22 +03:00
[(weld moves-1 moves-2) ..^^$]
2020-11-10 09:26:13 +03:00
[wake-moves ..^$]
2016-11-24 07:25:07 +03:00
::
2019-05-11 00:51:37 +03:00
?(%warp %werp)
:: capture whether this read is on behalf of another ship
:: for permissions enforcement
::
2018-02-09 15:33:15 +03:00
=^ for req
2019-05-11 00:51:37 +03:00
?: ?=(%warp -.req)
2018-02-09 15:33:15 +03:00
[~ req]
:: ?: =(our who.req)
:: [~ [%warp wer.req rif.req]]
:- ?:(=(our who.req) ~ `[who.req -.rif.req])
[%warp wer.req riff.rif.req]
::
2019-05-11 00:51:37 +03:00
?> ?=(%warp -.req)
2018-02-09 16:48:08 +03:00
=* rif rif.req
2016-11-24 07:25:07 +03:00
=^ mos ruf
2020-12-06 11:38:37 +03:00
=/ den ((de now rof hen ruf) wer.req p.rif)
2016-11-24 07:25:07 +03:00
=< abet
2018-02-09 15:25:46 +03:00
?~ q.rif
2016-11-24 07:25:07 +03:00
cancel-request:den
2018-02-09 15:25:46 +03:00
(start-request:den for u.q.rif)
2016-11-24 07:25:07 +03:00
[mos ..^$]
::
%plea
2019-06-29 02:43:52 +03:00
=* her ship.req
=* pax path.plea.req
=* res payload.plea.req
2019-06-29 02:43:52 +03:00
::
?: ?=([%backfill *] pax)
=+ ;;(=fill res)
=^ mos ruf
2020-12-06 11:38:37 +03:00
=/ den ((de now rof hen ruf) our desk.fill)
abet:(give-backfill:den lobe.fill)
[[[hen %give %done ~] mos] ..^$]
?> ?=([%question *] pax)
=+ ryf=;;(riff-any res)
2019-06-29 02:43:52 +03:00
:_ ..^$
:~ [hen %give %done ~]
=/ =wire
[%foreign-warp (scot %p her) t.pax]
[hen %pass wire %c %werp her our ryf]
==
2016-11-24 07:25:07 +03:00
==
::
++ load
2021-02-22 22:35:57 +03:00
=> |%
+$ raft-any
$% [%12 raft-12]
[%11 raft-11]
2022-03-26 02:41:54 +03:00
[%10 raft-10]
2021-09-22 02:13:54 +03:00
[%9 raft-9]
2021-06-29 03:01:51 +03:00
[%8 raft-8]
2021-04-20 06:46:46 +03:00
[%7 raft-7]
2021-02-22 22:35:57 +03:00
[%6 raft-6]
==
+$ raft-12 raft
+$ raft-11
$: rom=room-11
hoy=(map ship rung-11)
ran=rang-11
mon=(map term beam)
hez=(unit duct)
cez=(map @ta crew)
pud=(unit [=desk =yoki])
==
+$ rang-11
$: hut=(map tako yaki)
lat=(map lobe blub)
==
+$ room-11
$: hun=duct
dos=(map desk dojo-11)
==
+$ dojo-11
$: qyx=cult
dom=dome-11
per=regs
pew=regs
fiz=melt
==
+$ dome-11
$: ank=ankh
let=aeon
hit=(map aeon tako)
lab=(map @tas aeon)
mim=(map path mime)
fod=ford-cache
==
+$ rung-11
$: rus=(map desk rede-11)
==
+$ rede-11
$: lim=@da
ref=(unit rind-11)
qyx=cult
dom=dome-11
per=regs
pew=regs
fiz=melt
==
+$ rind-11
$: nix=@ud
bom=(map @ud update-state-11)
fod=(map duct @ud)
haw=(map mood (unit cage))
==
+$ update-state-11
$: =duct
=rave
have=(map lobe blub)
need=[let=(list lobe) old=(list lobe)]
nako=(qeu (unit nako))
busy=_|
==
2022-03-26 02:41:54 +03:00
+$ raft-10
$: rom=room-10
hoy=(map ship rung-10)
ran=rang-11
2022-03-26 02:41:54 +03:00
mon=(map term beam)
hez=(unit duct)
cez=(map @ta crew)
pud=(unit [=desk =yoki])
dist-upgraded=_|
==
+$ room-10
$: hun=duct
dos=(map desk dojo-10)
==
+$ dojo-10
$: qyx=cult-10
dom=dome-11
2022-03-26 02:41:54 +03:00
per=regs
pew=regs
fiz=melt
==
+$ cult-10 (jug wove-10 duct)
+$ wove-10 [for=(unit [=ship ver=@ud]) =rove-10]
+$ rove-10
$% [%sing =mood]
[%next =mood aeon=(unit aeon) =cach-10]
$: %mult
=mool
aeon=(unit aeon)
old-cach=(map [=care =path] cach-10)
new-cach=(map [=care =path] cach-10)
==
[%many track=? =moat lobes=(map path lobe)]
==
+$ cach-10 (unit (unit (each cage lobe)))
+$ rung-10
$: rus=(map desk rede-10)
==
+$ rede-10
$: lim=@da
ref=(unit rind-10)
2022-03-26 02:41:54 +03:00
qyx=cult-10
dom=dome-11
2022-03-26 02:41:54 +03:00
per=regs
pew=regs
fiz=melt
==
+$ rind-10
$: nix=@ud
bom=(map @ud update-state-10)
fod=(map duct @ud)
haw=(map mood (unit cage))
==
+$ update-state-10
$: =duct
=rave
have=(map lobe blub)
need=(list lobe)
nako=(qeu (unit nako))
busy=_|
==
2021-09-22 02:13:54 +03:00
+$ raft-9
2022-03-26 02:41:54 +03:00
$: rom=room-10
hoy=(map ship rung-10)
ran=rang-11
2022-03-26 02:41:54 +03:00
mon=(map term beam)
hez=(unit duct)
cez=(map @ta crew)
pud=(unit [=desk =yoki])
==
2021-06-29 03:01:51 +03:00
+$ raft-8
$: rom=room-8
hoy=(map ship rung-8)
ran=rang-11
2021-06-29 03:01:51 +03:00
mon=(map term beam)
hez=(unit duct)
cez=(map @ta crew)
pud=(unit [=desk =yoki])
==
+$ room-8
$: hun=duct
dos=(map desk dojo-8)
==
+$ rung-8
$: rus=(map desk rede-8)
==
+$ dojo-8
2022-03-26 02:41:54 +03:00
$: qyx=cult-10
2021-06-29 03:01:51 +03:00
dom=dome-8
per=regs
pew=regs
fiz=melt
==
+$ dome-8
$: ank=ankh
let=aeon
hit=(map aeon tako)
lab=(map @tas aeon)
mim=(map path mime)
fod=ford-cache
fer=* :: reef cache, obsolete
==
+$ rede-8
$: lim=@da
ref=(unit rind-10)
2022-03-26 02:41:54 +03:00
qyx=cult-10
2021-06-29 03:01:51 +03:00
dom=dome-8
per=regs
pew=regs
fiz=melt
==
2021-04-20 06:46:46 +03:00
+$ raft-7
2021-06-29 03:01:51 +03:00
$: rom=room-7
hoy=(map ship rung-7)
ran=rang-11
2021-06-29 03:01:51 +03:00
mon=(map term beam)
hez=(unit duct)
cez=(map @ta crew)
pud=(unit [=desk =yoki])
==
2021-04-20 06:46:46 +03:00
+$ room-7
2021-06-29 03:01:51 +03:00
$: hun=duct
dos=(map desk dojo-7)
==
2021-04-20 06:46:46 +03:00
+$ rung-7
2021-06-29 03:01:51 +03:00
$: rus=(map desk rede-7)
==
2021-04-20 06:46:46 +03:00
+$ dojo-7
2022-03-26 02:41:54 +03:00
$: qyx=cult-10
2021-06-29 03:01:51 +03:00
dom=dome-8
per=regs
pew=regs
==
2021-04-20 06:46:46 +03:00
+$ rede-7
2021-06-29 03:01:51 +03:00
$: lim=@da
ref=(unit rind-10)
2022-03-26 02:41:54 +03:00
qyx=cult-10
2021-06-29 03:01:51 +03:00
dom=dome-8
per=regs
pew=regs
==
2021-02-22 22:35:57 +03:00
+$ ford-cache-7 ford-cache
+$ raft-6
2022-03-26 02:41:54 +03:00
$: rom=room-6
hoy=(map ship rung-6)
ran=rang-11
2022-03-26 02:41:54 +03:00
mon=(map term beam)
hez=(unit duct)
cez=(map @ta crew)
pud=(unit [=desk =yoki])
2021-02-22 22:35:57 +03:00
== ::
+$ room-6 [hun=duct dos=(map desk dojo-6)]
+$ dojo-6
2022-03-26 02:41:54 +03:00
$: qyx=cult-10
dom=dome-6
per=regs
pew=regs
2021-02-22 22:35:57 +03:00
==
+$ dome-6
2022-03-26 02:41:54 +03:00
$: ank=ankh
let=aeon
hit=(map aeon tako)
lab=(map @tas aeon)
mim=(map path mime)
fod=ford-cache-6
fer=*
2021-02-22 22:35:57 +03:00
==
2021-03-04 07:02:29 +03:00
+$ rung-6
$: rus=(map desk rede-6)
==
+$ rede-6
$: lim=@da
ref=(unit rind-10)
2022-03-26 02:41:54 +03:00
qyx=cult-10
2021-03-04 07:02:29 +03:00
dom=dome-6
per=regs
pew=regs
==
2021-02-22 22:35:57 +03:00
+$ ford-cache-6 * :: discard old cache
--
|= old=raft-any
|^
=? old ?=(%6 -.old) 7+(raft-6-to-7 +.old)
2021-04-25 17:49:30 +03:00
=? old ?=(%7 -.old) 8+(raft-7-to-8 +.old)
2021-06-29 03:01:51 +03:00
=? old ?=(%8 -.old) 9+(raft-8-to-9 +.old)
2021-09-22 02:13:54 +03:00
=? old ?=(%9 -.old) 10+(raft-9-to-10 +.old)
2022-03-26 02:41:54 +03:00
=? old ?=(%10 -.old) 11+(raft-10-to-11 +.old)
=? old ?=(%11 -.old) 12+(raft-11-to-12 +.old)
?> ?=(%12 -.old)
2021-03-04 07:02:29 +03:00
..^^$(ruf +.old)
2021-02-22 22:35:57 +03:00
:: +raft-6-to-7: delete stale ford caches (they could all be invalid)
::
++ raft-6-to-7
|= raf=raft-6
^- raft-7
%= raf
dos.rom
%- ~(run by dos.rom.raf)
|= doj=dojo-6
^- dojo-7
doj(fod.dom *ford-cache-7)
2021-03-04 07:02:29 +03:00
::
hoy
%- ~(run by hoy.raf)
|= =rung-6
%- ~(run by rus.rung-6)
|= =rede-6
rede-6(dom dom.rede-6(fod *ford-cache-7))
2021-02-22 22:35:57 +03:00
==
2021-04-20 06:46:46 +03:00
:: +raft-7-to-8: create bunted melts in each dojo/rede
::
++ raft-7-to-8
|= raf=raft-7
^- raft-8
%= raf
dos.rom
%- ~(run by dos.rom.raf)
|= doj=dojo-7
2021-06-29 03:01:51 +03:00
^- dojo-8
2021-04-20 06:46:46 +03:00
[qyx.doj dom.doj per.doj pew.doj *melt]
::
hoy
%- ~(run by hoy.raf)
|= =rung-7
%- ~(run by rus.rung-7)
|= r=rede-7
2021-06-29 03:01:51 +03:00
^- rede-8
2021-04-20 06:46:46 +03:00
[lim.r ref.r qyx.r dom.r per.r pew.r *melt]
==
2021-06-29 03:01:51 +03:00
:: +raft-8-to-9: remove reef cache
::
++ raft-8-to-9
|= raf=raft-8
^- raft-9
%= raf
dos.rom
%- ~(run by dos.rom.raf)
|= =dojo-8
2022-03-26 02:41:54 +03:00
^- dojo-10
2021-06-29 03:01:51 +03:00
=/ dom dom.dojo-8
dojo-8(dom [ank.dom let.dom hit.dom lab.dom mim.dom *ford-cache])
::
hoy
%- ~(run by hoy.raf)
|= =rung-8
%- ~(run by rus.rung-8)
|= =rede-8
2022-03-26 02:41:54 +03:00
^- rede-10
2021-06-29 03:01:51 +03:00
=/ dom dom.rede-8
rede-8(dom [ank.dom let.dom hit.dom lab.dom mim.dom *ford-cache])
==
2021-09-22 02:13:54 +03:00
:: +raft-9-to-10: add .dist-upgraded
++ raft-9-to-10
|= raf=raft-9
^- raft-10
raf(pud [pud.raf dist-upgraded=|])
2022-03-26 02:41:54 +03:00
::
:: +raft-10-to-11:
::
:: remove .dist-upgraded
:: upgrade +cach
:: upgrade need:update-state
::
2022-03-26 02:41:54 +03:00
++ raft-10-to-11
|= raf=raft-10
|^
^- raft-11
2022-03-26 02:41:54 +03:00
%= raf
|6 pud.raf
dos.rom
^- (map desk dojo-11)
2022-03-26 02:41:54 +03:00
%- ~(run by dos.rom.raf)
|= =dojo-10
^- dojo-11
2022-03-26 02:41:54 +03:00
dojo-10(qyx (cult-10-to-cult qyx.dojo-10))
::
hoy
^- (map ship rung-11)
2022-03-26 02:41:54 +03:00
%- ~(run by hoy.raf)
|= =rung-10
%- ~(run by rus.rung-10)
|= =rede-10
^- rede-11
%= rede-10
qyx (cult-10-to-cult qyx.rede-10)
ref
?~ ref.rede-10
~
%= ref.rede-10
bom.u
%- ~(run by bom.u.ref.rede-10)
|= =update-state-10
^- update-state-11
update-state-10(need [need.update-state-10 ~])
==
==
2022-03-26 02:41:54 +03:00
==
++ cult-10-to-cult
|= qyx=cult-10
^- cult
=/ qux=(list [=wove-10 ducts=(set duct)]) ~(tap by qyx)
%- malt
|- ^- (list [wove (set duct)])
?~ qux
~
:_ $(qux t.qux)
%= i.qux
rove-10.wove-10
?- -.rove-10.wove-10.i.qux
%sing rove-10.wove-10.i.qux
%many rove-10.wove-10.i.qux
%next
%= rove-10.wove-10.i.qux
cach-10 (cach-10-to-cach cach-10.rove-10.wove-10.i.qux)
==
::
%mult
%= rove-10.wove-10.i.qux
old-cach (caches-10-to-caches old-cach.rove-10.wove-10.i.qux)
new-cach (caches-10-to-caches new-cach.rove-10.wove-10.i.qux)
==
==
==
::
++ cach-10-to-cach
|= =cach-10
^- cach
?~ cach-10
~
?~ u.cach-10
[~ ~]
?- -.u.u.cach-10
%& ``p.u.u.cach-10
%| ~
==
::
++ caches-10-to-caches
|= caches-10=(map [=care =path] cach-10)
^- (map [=care =path] cach)
(~(run by caches-10) cach-10-to-cach)
--
::
:: +raft-11-to-12:
::
:: add tom and nor to dome
:: remove parent-mark from delta blobs
::
++ raft-11-to-12
|= raf=raft-11
^- raft-12
=/ keep-all=norm (~(put ^de *norm) / &)
%= raf
lat.ran (~(run by lat.ran.raf) blub-to-blob)
dos.rom
%- ~(run by dos.rom.raf)
|= =dojo-11
^- dojo
dojo-11(|4.dom [~ keep-all |4.dom.dojo-11])
::
hoy
%- ~(run by hoy.raf)
|= =rung-11
%- ~(run by rus.rung-11)
|= =rede-11
^- rede
%= rede-11
|4.dom [~ keep-all |4.dom.rede-11]
ref
?~ ref.rede-11
~
%= ref.rede-11
bom.u
%- ~(run by bom.u.ref.rede-11)
|= =update-state-11
update-state-11(have (~(run by have.update-state-11) blub-to-blob))
==
==
==
2021-02-22 22:35:57 +03:00
--
2016-11-24 07:25:07 +03:00
::
++ scry :: inspect
2021-03-28 10:47:37 +03:00
~/ %clay-scry
2020-12-08 00:52:12 +03:00
^- roon
|= [lyc=gang car=term bem=beam]
2016-11-24 07:25:07 +03:00
^- (unit (unit cage))
2020-12-08 00:52:12 +03:00
=* ren car
2020-11-24 00:06:50 +03:00
=/ why=shop &/p.bem
=* syd q.bem
=/ lot=coin $/r.bem
=* tyl s.bem
::
2018-03-19 06:54:47 +03:00
?. ?=(%& -.why) ~
2016-11-24 07:25:07 +03:00
=* his p.why
2020-05-07 11:51:08 +03:00
?: &(=(ren %$) =(tyl /whey))
``mass+!>(whey)
2016-11-24 07:25:07 +03:00
:: ~& scry+[ren `path`[(scot %p his) syd ~(rent co lot) tyl]]
:: =- ~& %scry-done -
=+ luk=?.(?=(%$ -.lot) ~ ((soft case) p.lot))
2016-11-24 07:25:07 +03:00
?~ luk [~ ~]
?: =(%$ ren)
[~ ~]
=+ run=((soft care) ren)
?~ run [~ ~]
::TODO if it ever gets filled properly, pass in the full fur.
2020-11-24 00:06:50 +03:00
::
=/ for=(unit ship) ?~(lyc ~ ?~(u.lyc ~ `n.u.lyc))
2020-12-06 11:38:37 +03:00
=/ den ((de now rof [/scryduct ~] ruf) his syd)
=/ result (mule |.(-:(aver:den for u.run u.luk tyl)))
?: ?=(%| -.result)
%- (slog >%clay-scry-fail< p.result)
~
?~ p.result ~
?~ u.p.result [~ ~]
clay: Add support for representing tombstones This adds support for tombstoned files to clay. It does not include any way to actually tombstone them; that is left for later. This allows tombstoning at the level of a file. Precisely, this expands +blob:clay by adding a %dead case: +$ blob :: fs blob $% [%delta p=lobe q=[p=mark q=lobe] r=page] :: delta on q [%direct p=lobe q=page] :: immediate [%dead p=lobe ~] :: tombstone == :: Thus, we maintain the invariant that every lobe corresponds to a blob, but now a blob may be an explicit tombstone. Details: - This has not been tested at all, except that it compiles and boots. - This does not have a state adapter from master. The only state change is the definition of +cach. - Additionally, out-of-date ships may unexpectedly receive a %dead blob from a foreign clay which would interfere with their ability to download that desk. No code changes necessary, but sponsors should avoid tombstoning files in %base for a while so their children can get the update. - A merge will only fail if the tombstoned file conflicts with another change. Note that as written, merging from a past desk *can* bring a tombstoned file to the head of a desk. Possibly this shouldn't be allowed. This also includes a couple refactors that were made possible by ford fusion (since everything is synchronous now) but never got done. In both cases we get to remove a monad, which simplifies the code considerably. - refactor +merge's error handling to use !!/mule instead of threading through errors - refactor all +read-* functions and related parts of +try-fill-sub to eagerly convert lobes to cages. We also add support reading %a/b/c/e/f/r/x from past and foreign desks, when possible. Apologies that all of these are in one commit, it was all a single chunk of work. This is a draft until we have a way to tombstone. I suspect we'll want to have a mechanism of keeping track of gc roots and trace to remove, but this PR doesn't suggest any particular strategy.
2021-11-08 09:17:02 +03:00
``u.u.p.result
2016-11-24 07:25:07 +03:00
::
2020-05-16 15:22:39 +03:00
:: We clear the ford cache by replacing it with its bunt as a literal.
:: This nests within +ford-cache without reference to +type, +hoon, or
:: anything else in the sample of cache objects. Otherwise we would be
:: contravariant in the those types, which makes them harder to change.
::
++ stay
:- ver
%= ruf
dos.rom
%- ~(run by dos.rom.ruf)
|= =dojo
2021-01-13 03:10:11 +03:00
dojo(fod.dom [~ ~ ~ ~ ~])
2020-05-16 15:22:39 +03:00
::
hoy
%- ~(run by hoy.ruf)
|= =rung
%= rung
rus
%- ~(run by rus.rung)
|= =rede
2021-01-13 03:10:11 +03:00
rede(fod.dom [~ ~ ~ ~ ~])
2020-05-16 15:22:39 +03:00
==
==
::
2016-11-24 07:25:07 +03:00
++ take :: accept response
2021-03-28 10:47:37 +03:00
~/ %clay-take
2020-12-06 11:38:37 +03:00
|= [tea=wire hen=duct dud=(unit goof) hin=sign]
^+ [*(list move) ..^$]
?^ dud
~|(%clay-take-dud (mean tang.u.dud))
?: ?=([%dist *] tea)
?: ?=(%onto +<.hin)
[~ ..^$]
?> ?=(%unto +<.hin)
?> ?=(%poke-ack -.p.hin)
?~ p.p.hin
[~ ..^$]
=+ ((slog 'clay: dist migration failed' u.p.p.hin) ~)
!!
::
2020-05-12 10:01:46 +03:00
?: ?=([%merge @ @ @ @ ~] tea)
2020-12-06 11:38:37 +03:00
?> ?=(%writ +<.hin)
2019-05-02 04:21:32 +03:00
=* syd i.t.tea
2020-05-12 10:01:46 +03:00
=/ ali-ship (slav %p i.t.t.tea)
=* ali-desk i.t.t.t.tea
=/ germ (germ i.t.t.t.t.tea)
2019-05-02 04:21:32 +03:00
=^ mos ruf
2020-12-06 11:38:37 +03:00
=/ den ((de now rof hen ruf) our i.t.tea)
abet:(merge:den ali-ship ali-desk germ p.hin)
2019-05-02 04:21:32 +03:00
[mos ..^$]
2019-05-17 03:24:10 +03:00
::
?: ?=([%fuse @ @ @ @ ~] tea)
2021-04-20 06:46:46 +03:00
?> ?=(%writ +<.hin)
=* syd i.t.tea
=/ ali-ship=@p (slav %p i.t.t.tea)
=* ali-desk=desk i.t.t.t.tea
=/ ali-case (rash i.t.t.t.t.tea nuck:so)
?> ?=([%$ *] ali-case)
=^ mos ruf
=/ den ((de now rof hen ruf) our i.t.tea)
2021-04-24 00:33:49 +03:00
abet:(take-fuse:den [ali-ship ali-desk (case +.ali-case)] p.hin)
2021-04-20 06:46:46 +03:00
[mos ..^$]
::
2019-06-29 02:43:52 +03:00
?: ?=([%foreign-warp *] tea)
2020-12-06 11:38:37 +03:00
?> ?=(%writ +<.hin)
2019-06-29 02:43:52 +03:00
:_ ..^$
2020-12-06 11:38:37 +03:00
[hen %give %boon `(unit rand)`(bind `riot`p.hin rant-to-rand)]~
2019-06-29 02:43:52 +03:00
::
?: ?=([%warp-index @ @ @ ~] tea)
2020-12-06 11:38:37 +03:00
?+ +<.hin ~| %clay-warp-index-strange !!
%done
2020-12-06 11:38:37 +03:00
?~ error.hin
[~ ..^$]
:: TODO better error handling
::
2020-12-06 11:38:37 +03:00
~& %clay-take-warp-index-error^our^tea^tag.u.error.hin
%- (slog tang.u.error.hin)
[~ ..^$]
::
%lost
~| %clay-take-lost^our
:: TODO better error handling
!!
2019-06-29 02:43:52 +03:00
::
%boon
2020-12-06 11:38:37 +03:00
=+ ;; res=(unit rand) payload.hin
::
=/ her=ship (slav %p i.t.tea)
=/ =desk (slav %tas i.t.t.tea)
=/ index=@ud (slav %ud i.t.t.t.tea)
::
=^ mos ruf
2020-12-06 11:38:37 +03:00
=/ den ((de now rof hen ruf) her desk)
abet:(take-foreign-answer:den index res)
[mos ..^$]
==
2019-06-29 02:43:52 +03:00
::
?: ?=([%back-index @ @ @ ~] tea)
2020-12-06 11:38:37 +03:00
?+ +<.hin ~| %clay-backfill-index-strange !!
%done
2020-12-06 11:38:37 +03:00
?~ error.hin
[~ ..^$]
:: TODO better error handling
::
2020-12-06 11:38:37 +03:00
~& %clay-take-backfill-index-error^our^tea^tag.u.error.hin
%- (slog tang.u.error.hin)
[~ ..^$]
::
%lost
~| %clay-take-backfill-lost^our
:: TODO better error handling
!!
::
%boon
=+ ;; =blub payload.hin
::
=/ her=ship (slav %p i.t.tea)
=/ =desk (slav %tas i.t.t.tea)
=/ index=@ud (slav %ud i.t.t.t.tea)
::
=^ mos ruf
2020-12-06 11:38:37 +03:00
=/ den ((de now rof hen ruf) her desk)
abet:abet:(take-backfill:(foreign-update:den index) blub)
[mos ..^$]
==
::
2019-07-27 06:43:07 +03:00
?: ?=([%sinks ~] tea)
2020-12-06 11:38:37 +03:00
?> ?=(%public-keys +<.hin)
?. ?=(%breach -.public-keys-result.hin)
2019-07-27 06:43:07 +03:00
[~ ..^$]
2020-12-06 11:38:37 +03:00
=/ who who.public-keys-result.hin
2019-07-27 06:43:07 +03:00
?: =(our who)
[~ ..^$]
:: Cancel subscriptions
::
=/ foreign-desk=(unit rung)
(~(get by hoy.ruf) who)
?~ foreign-desk
[~ ..^$]
=/ cancel-ducts=(list duct)
%- zing ^- (list (list duct))
%+ turn ~(tap by rus.u.foreign-desk)
|= [=desk =rede]
^- (list duct) %- zing ^- (list (list duct))
%+ turn ~(tap by qyx.rede)
|= [=wove ducts=(set duct)]
:: ~& [%sunk-wove desk (print-wove wove) ducts]
~(tap in ducts)
=/ cancel-moves=(list move)
%+ turn cancel-ducts
2020-05-21 11:10:39 +03:00
|=(=duct [duct %slip %b %drip !>([%writ ~])])
2019-07-27 06:43:07 +03:00
:: delete local state of foreign desk
::
=. hoy.ruf (~(del by hoy.ruf) who)
2020-05-13 09:36:55 +03:00
[cancel-moves ..^$]
2019-07-27 06:43:07 +03:00
::
2020-12-06 11:38:37 +03:00
?- -.+.hin
2019-07-27 06:43:07 +03:00
%public-keys ~|([%public-keys-raw tea] !!)
2016-11-24 07:25:07 +03:00
::
2019-05-11 00:51:37 +03:00
%mere
2020-12-06 11:38:37 +03:00
?: ?=(%& -.p.+.hin)
2016-11-24 07:25:07 +03:00
~& 'initial merge succeeded'
[~ ..^$]
~> %slog.
:^ 0 %rose [" " "[" "]"]
:^ leaf+"initial merge failed"
leaf+"my most sincere apologies"
2020-12-06 11:38:37 +03:00
>p.p.p.+.hin<
q.p.p.+.hin
2016-11-24 07:25:07 +03:00
[~ ..^$]
::
2020-12-06 11:38:37 +03:00
%note [[hen %give +.hin]~ ..^$]
2019-05-11 00:51:37 +03:00
%wake
:: TODO: handle behn errors
::
2020-12-06 11:38:37 +03:00
?^ error.hin
[[hen %slip %d %flog %crud %wake u.error.hin]~ ..^$]
::
?. ?=([%tyme @ @ ~] tea)
~& [%clay-strange-timer tea]
[~ ..^$]
=/ her (slav %p i.t.tea)
=/ syd (slav %tas i.t.t.tea)
=^ mos ruf
2020-12-06 11:38:37 +03:00
=/ den ((de now rof hen ruf) her syd)
abet:wake:den
[mos ..^$]
2016-11-24 07:25:07 +03:00
::
2019-06-29 02:43:52 +03:00
:: handled in the wire dispatcher
::
%boon !!
%lost !!
%onto !!
%unto !!
%writ
%- (slog leaf+"clay: strange writ (expected on upgrade to Fusion)" ~)
[~ ..^$]
2016-11-24 07:25:07 +03:00
::
2019-06-29 02:43:52 +03:00
%done
2020-12-06 11:38:37 +03:00
?~ error=error.hin
2019-05-11 05:23:13 +03:00
[~ ..^$]
2019-10-30 23:23:37 +03:00
%- (slog >%clay-lost< >tag.u.error< tang.u.error)
2016-11-24 07:25:07 +03:00
[~ ..^$]
==
::
++ rant-to-rand
|= rant
^- rand
[p q [p q.q]:r]
2020-05-07 11:51:08 +03:00
:: +whey: produce memory usage report
::
++ whey
^- (list mass)
=/ domestic
%+ turn (sort ~(tap by dos.rom.ruf) aor)
|= [=desk =dojo]
:+ desk %|
|^
2020-05-07 11:51:08 +03:00
:~ ankh+&+ank.dom.dojo
mime+&+mim.dom.dojo
ford-files+|+files
ford-naves+|+naves
ford-marks+|+marks
ford-casts+|+casts
ford-tubes+|+tubes
2020-05-07 11:51:08 +03:00
==
++ marks
^- (list mass)
%+ turn (sort ~(tap by marks.fod.dom.dojo) aor)
|= [=mark res=*]
^- mass
[mark %& res]
::
++ casts
^- (list mass)
%+ turn (sort ~(tap by casts.fod.dom.dojo) aor)
|= [=mars res=*]
^- mass
[`@t`(rap 3 [a '-' b ~]:mars) %& res]
::
++ naves
^- (list mass)
%+ turn (sort ~(tap by naves.fod.dom.dojo) aor)
|= [=mark res=*]
^- mass
[mark %& res]
::
++ tubes
^- (list mass)
%+ turn (sort ~(tap by tubes.fod.dom.dojo) aor)
|= [=mars res=*]
^- mass
[`@t`(rap 3 [a '-' b ~]:mars) %& res]
::
++ files
^- (list mass)
%+ turn (sort ~(tap by files.fod.dom.dojo) aor)
|= [=path res=*]
^- mass
[(spat path) %& res]
--
2020-05-07 11:51:08 +03:00
:~ domestic+|+domestic
foreign+&+hoy.ruf
:+ %object-store %|
:~ commits+&+hut.ran.ruf
blobs+&+lat.ran.ruf
==
==
::
++ tomb
2022-04-06 10:18:23 +03:00
|%
:: +tomb-clue: safely remove objects
::
++ tomb-clue
|= =clue
^- [(list move) _..^$]
?- -.clue
%lobe `(tomb-lobe lobe.clue &)
%all
=/ lobes=(list [=lobe =blob]) ~(tap by lat.ran.ruf)
|-
?~ lobes
`..^^$
=. ..^^$ (tomb-lobe lobe.i.lobes &)
$(lobes t.lobes)
2022-04-07 09:16:26 +03:00
::
%pick pick
%norm
=^ mos ruf
=/ den ((de now rof *duct ruf) ship.clue desk.clue)
abet:(set-norm:den norm.clue)
[mos ..^$]
2022-04-06 10:18:23 +03:00
==
:: +tomb-lobe: remove specific lobe
::
++ tomb-lobe
|= [lob=lobe veb=?]
2022-04-06 10:18:23 +03:00
^+ ..^$
=/ bol (~(get by lat.ran.ruf) lob)
?~ bol
(noop veb leaf+"clay: lobe doesn't exist" ~)
?: ?=(%dead -.u.bol)
(noop veb leaf+"clay: file already tombstoned" ~)
::
=/ used=(unit beam)
=/ desks=(list [=desk =dojo]) ~(tap by dos.rom.ruf)
|-
=* desk-loop $
?~ desks
~
?: =(0 let.dom.dojo.i.desks)
desk-loop(desks t.desks)
=/ =yaki
%- ~(got by hut.ran.ruf)
%- ~(got by hit.dom.dojo.i.desks)
let.dom.dojo.i.desks
=/ paths=(list [=path =lobe]) ~(tap by q.yaki)
|-
=* path-loop $
?~ paths
desk-loop(desks t.desks)
?: =(lob lobe.i.paths)
`[[our desk.i.desks ud+let.dom.dojo.i.desks] path.i.paths]
path-loop(paths t.paths)
::
?^ used
(noop veb leaf+"clay: file used in {<(en-beam u.used)>}" ~)
::
=/ based=(unit lobe)
=/ lobes=(list [=lobe =blob]) ~(tap by lat.ran.ruf)
|-
?~ lobes
~
?: ?& ?=(%delta -.blob.i.lobes)
=(lob q.blob.i.lobes)
==
`p.blob.i.lobes
$(lobes t.lobes)
::
?^ based
(noop veb leaf+"clay: file is base of delta for {<u.based>}" ~)
::
=. lat.ran.ruf (~(put by lat.ran.ruf) lob %dead lob ~)
(noop veb leaf+"clay: file successfully tombstoned" ~)
::
++ noop
|= [veb=? =tang]
?. veb
2022-04-06 10:18:23 +03:00
..^$
((slog tang) ..^$)
2022-04-07 07:45:30 +03:00
::
2022-04-07 10:44:49 +03:00
++ draw-raft
^- (set [norm yaki])
=/ room-yakis
=/ rooms=(list [=desk =dojo]) ~(tap by dos.rom.ruf)
|- ^- (set [norm yaki])
?~ rooms
~
~& > desk=desk.i.rooms
(~(uni in $(rooms t.rooms)) (draw-dome dom.dojo.i.rooms))
=/ rung-yakis
=/ rungs=(list [=ship =rung]) ~(tap by hoy.ruf)
|- ^- (set [norm yaki])
?~ rungs
~
%- ~(uni in $(rungs t.rungs))
=/ redes=(list [=desk =rede]) ~(tap by rus.rung.i.rungs)
|- ^- (set [norm yaki])
?~ redes
~
~& > [ship=ship.i.rungs desk=desk.i.redes]
(~(uni in $(redes t.redes)) (draw-dome dom.rede.i.redes))
(~(uni in room-yakis) rung-yakis)
::
2022-04-07 09:16:26 +03:00
++ draw-dome
|= =dome
^- (set [norm yaki])
2022-04-07 10:44:49 +03:00
=/ keep-all=norm (~(put ^de *norm) / &)
2022-04-07 09:16:26 +03:00
=/ =aeon 1
|- ^- (set [norm yaki])
?: (lth let.dome aeon)
~
=/ =tako (~(got by hit.dome) aeon)
2022-04-07 10:44:49 +03:00
=/ nor
?: =(let.dome aeon)
keep-all
nor.dome
(~(uni in (draw-tako tom.dome nor tako)) $(aeon +(aeon)))
2022-04-07 09:16:26 +03:00
::
++ draw-tako
|= [tom=(map tako norm) nor=norm =tako]
^- (set [norm yaki])
~+
=/ =norm (~(gut by tom) tako nor)
=/ =yaki (~(got by hut.ran.ruf) tako)
=/ takos
|- ^- (set [^norm ^yaki])
?~ p.yaki
~
(~(uni in $(p.yaki t.p.yaki)) ^$(tako i.p.yaki))
(~(put in takos) norm yaki)
::
:: +pick: copying gc based on norms
2022-04-07 07:45:30 +03:00
::
2022-04-07 10:44:49 +03:00
:: XX somehow this seems to be able to tombstone something in our
:: head!
::
2022-04-07 07:45:30 +03:00
++ pick
2022-04-07 09:16:26 +03:00
=| lat=(map lobe blob)
=| sen=(set (map path lobe))
2022-04-07 07:45:30 +03:00
|^
2022-04-07 09:16:26 +03:00
=. ..pick-raft pick-raft
=. lat.ran.ruf lat
`..^$
::
++ pick-raft
^+ ..pick-raft
2022-04-07 10:44:49 +03:00
=/ yakis=(list [=norm =yaki]) ~(tap in draw-raft)
2022-04-07 09:16:26 +03:00
|- ^+ ..pick-raft
2022-04-07 07:45:30 +03:00
?~ yakis
2022-04-07 09:16:26 +03:00
..pick-raft
$(yakis t.yakis, ..pick-raft (pick-yaki i.yakis))
2022-04-07 07:45:30 +03:00
::
2022-04-07 09:16:26 +03:00
:: NB: recurring in this way with the `sen` cache provides
:: approximately a 100x speedup on a mainnet moon in 4/2022
2022-04-07 07:45:30 +03:00
::
++ pick-yaki
|= [=norm =yaki]
2022-04-07 09:16:26 +03:00
^+ ..pick-raft
|- ^+ ..pick-raft
?~ q.yaki
..pick-raft
?: (~(has in sen) q.yaki)
..pick-raft
=. sen (~(put in sen) q.yaki)
=/ bob (~(get by lat) q.n.q.yaki)
=? lat ?|(?=(~ bob) ?=([~ %dead *] bob))
?: =([~ %&] +:(~(fit ^de norm) p.n.q.yaki))
(uni-blobs lat (pick-lobe q.n.q.yaki))
(~(put by lat) q.n.q.yaki %dead q.n.q.yaki ~)
=. ..pick-raft $(q.yaki l.q.yaki)
$(q.yaki r.q.yaki)
2022-04-07 07:45:30 +03:00
::
++ pick-lobe
|= =lobe
^- (map ^lobe blob)
=/ bob=(unit blob) (~(get by lat.ran.ruf) lobe)
?~ bob
%- (slog leaf+"clay: picking strange lobe {<lobe>}")
~
?- -.u.bob
%dead [[lobe u.bob] ~ ~]
%direct [[lobe u.bob] ~ ~]
2022-04-07 10:44:49 +03:00
%delta (~(put by $(lobe q.u.bob)) lobe u.bob)
2022-04-07 07:45:30 +03:00
==
--
--
2016-11-24 07:25:07 +03:00
--