Commit Graph

485 Commits

Author SHA1 Message Date
Philip Monk
a51152d1a4 clay: address review comments 2022-04-06 12:16:22 -07:00
Philip Monk
4131821b4f clay: unify all trie implementations
We had trie operations independently implemented in +de in arvo,
+an:cloy in zuse, +zu in clay, lib/trie, and app/spider.  This unifies
them all into +de in arvo, aggregating the used operations.
2022-04-06 00:19:38 -07:00
Philip Monk
0f6f2d6321 clay: bump kelvin, refactor 2022-04-06 00:18:23 -07:00
Philip Monk
90a2625016 clay: style 2022-04-05 20:30:52 -07:00
Philip Monk
90098067f2 clay: extend %tomb to delete all unneeded blobs 2022-03-26 14:37:46 -07:00
Philip Monk
ee3951e2df clay: require non-dead files at head of remote 2022-03-25 22:15:53 -07:00
Philip Monk
6e641005a0 clay: add upgrade to %11 2022-03-25 18:02:22 -07:00
Philip Monk
fb3f74f639 clay: prefer life 2022-03-25 15:32:38 -07:00
Philip Monk
ef9521272a Merge remote-tracking branch 'origin/lf/ford-memory-shit' into philip/tomb 2022-03-25 13:13:24 -07:00
Philip Monk
c5fda496ac clay: add basic %tomb 2022-03-25 13:13:09 -07:00
Philip Monk
99a10aa7a8 clay: use XX for things that really need to change 2022-03-25 12:33:48 -07:00
Philip Monk
0c3eee07b5 Merge remote-tracking branch 'origin/master' into philip/tomb 2022-03-25 12:21:56 -07:00
Liam Fitzgerald
dd70856188 clay: be careful with subject in ford results
This fixes a space leak where the entire ford/clay core would be
included in the ford cache. Heavily reduces memory usage by clay,
reducing total usage from 100-200MB to around 10MB.
2022-03-21 13:02:36 -06:00
Liam Fitzgerald
2def946ded clay: break out ford memory usage in more detail
Instead of reporting a single memory size for built files, marks and
conversions, we now report memory size per path, mark name and mark
pair, respectively.
2022-03-21 12:21:38 -06:00
Liam Fitzgerald
0587e0e836 ames: add %pine task
%pine automatically scries for the latest case of a path
2022-03-14 15:04:14 -06:00
Yaseen
70afcbd642 clay: fix bug in fuse
Co-authored-by: Philip Monk <phil@pcmonk.me>
2022-02-25 16:34:44 -05:00
fang
027d80b90c
Merge branch 'lf/content-dist-rc' into m/scry-vane 2022-02-23 16:18:17 -06:00
fang
aaa759268e clay: clean up REVIEW comments 2022-02-23 16:00:55 -06:00
fang
cdff1d91fa clay: scry files at aeons, not timestamps
Turns out we can simply find the corresponding aeon in the nako.
2022-02-23 16:00:55 -06:00
fang
2adc52e03a clay: retry with ames on empty scry response 2022-02-23 16:00:55 -06:00
Liam Fitzgerald
22f3e49823 ames: add |yawn, fix duct wrangling 2022-02-23 16:00:31 -06:00
fang
6b85022a61
clay: properly cancel scry timers and/or requests
Use the correct wires when cancelling the scry request and/or its
timers.

Note that we may produce more %rests and %yawns than strictly necessary,
but these no-op cleanly in those cases.
2022-02-23 15:41:19 -06:00
fang
d8dca44bd2
clay: refactor request wire construction 2022-02-23 15:40:16 -06:00
Yaseen
3d006d60ad clay: add scry for case existence check 2022-02-18 17:43:38 -05:00
fang
2969b1aae1
clay: clean up REVIEW comments 2022-02-17 21:17:45 +01:00
fang
02dec3ef5b
clay: scry files at aeons, not timestamps
Turns out we can simply find the corresponding aeon in the nako.
2022-02-17 21:14:52 +01:00
fang
b91e3f425e
clay: retry with ames on empty scry response 2022-02-17 20:10:38 +01:00
fang
7c0f05f8c3
clay: hook up /back-index %wake
And make sure we handle the timer error case safely.
2022-02-17 16:08:48 +01:00
fang
d6b5633c99
clay: allow for lobe-only need
This can occur after upgrade, or when receiving a %delta blob. We do not
know the path or commit timestamp of a file that resolves to the blob,
so we simply fall back to old ames-style blob-by-lobe fetching.
2022-02-17 16:02:57 +01:00
fang
3473033492
clay: scry when doing file requests outside %sing
In +work, we simply want to fetch file data, regardless of what the
overarching request is. (In fact, %sing would likely never hit this path
anyway.)

In order to be able to make the scry request, we track the timestamp of
the commit that contains a blob we're missing, and scry for that %da
revision.

There's two edge cases where we cannot immediately know the timestamp
that are currently assumed broken. Fix soon.
2022-02-17 15:44:16 +01:00
fang
dd4a115e51
clay: avoid %v scries, cancel scry timer correctly
%v requests are already handled specially for the ames case. We should
continue to respect that, only doing those kinds of requests over ames.

Also cancels the scry timer after receiving a response, instead of not
doing that.
2022-02-17 15:40:44 +01:00
fang
fb4b3c7196
clay: update straggler state version number 2022-02-15 22:35:53 +01:00
fang
0375e0a53f
clay: remove dist-upgraded flag
Ancient version upgraders beware! If you're coming from a home-based
arvo, you must first upgrade to the version prior to this, or else
you'll be in trouble!
2022-02-15 01:02:19 +01:00
fang
c02274dfdd
clay: sync files using remote scry
When clay wants to download blobs from a foreign ship, it attempts doing
this using the new remote scry protocol.

If it doesn't receive a response within ~m1, it falls back to using the
old ames-based syncing instead. We remember this "prefer ames" state for
the specific ship for an hour, after which we'll begin trying the scry
flow again.

Compiles, but untested. Some TODOs and REVIEWs remaining herein.

We intentionally leave the dist-upgraded flag in state to avoid
cluttering the diff here. The next commit will remove it.
2022-02-15 00:51:54 +01:00
Yaseen
ce0fa85e18 story: generators improvements
- use desk parameter instead of %base everywhere
- formatting clean up
- make |story-remove take a case instead of an aeon
- make desk param optional for story-set and story-log
2022-02-05 22:50:10 -05:00
fang
2c5510db64
Merge branch 'jb/next-gen-term' into next/dill
Resolves a good number of conflicts. Most notably, re-propagates removal
of gall's %onto, confirms new /app/herm behavior, coerces hood/drum
state adapters back into place, and updates webterm to use the latest
api.
2022-01-24 14:50:16 +01:00
Yaseen
b37fd09fab clay: implement %base-tako scry under the %s care 2022-01-13 15:56:20 -05:00
Yaseen
163ed030d4 clay: add new %s scry which gives the tako for a given beak 2021-11-25 12:48:24 -05:00
Yaseen
c080eb8336 fix comment styling 2021-11-15 09:07:24 -05:00
Yaseen
ae7cf81c29
clay: keep info in stacktrace when labeling fails
Co-authored-by: Philip Monk <phil@pcmonk.me>
2021-11-15 05:32:59 -05:00
Philip Monk
8af1dd3acc
clay: refactor +wake
+wake had accumulated several layers of abstractions which were later
rendered unnecessary.  This removes those abstractions and should have
no semantic effect.
2021-11-14 19:35:32 -08:00
Yaseen
70a85ccdb0 remove unnecessary comments 2021-11-14 20:41:30 -05:00
Yaseen
17751e7890 clay: enforce referential transparency on labels 2021-11-14 16:49:17 -05:00
Yaseen
25ce97cade clay: add support for labeling commits
- add helper arm `label:de`
- update relevant code to take in arbitrary aeons
2021-11-12 15:35:06 -05:00
Philip Monk
d9276cd51c
clay: Add support for representing tombstones
This adds support for tombstoned files to clay.  It does not include any
way to actually tombstone them; that is left for later.

This allows tombstoning at the level of a file.  Precisely, this expands
+blob:clay by adding a %dead case:

    +$  blob                                              ::  fs blob
      $%  [%delta p=lobe q=[p=mark q=lobe] r=page]        ::  delta on q
          [%direct p=lobe q=page]                         ::  immediate
          [%dead p=lobe ~]                                ::  tombstone
      ==                                                  ::

Thus, we maintain the invariant that every lobe corresponds to a blob,
but now a blob may be an explicit tombstone.

Details:
- This has not been tested at all, except that it compiles and boots.
- This does not have a state adapter from master.  The only state change
  is the definition of +cach.
- Additionally, out-of-date ships may unexpectedly receive a %dead blob
  from a foreign clay which would interfere with their ability to download
  that desk.  No code changes necessary, but sponsors should avoid
  tombstoning files in %base for a while so their children can get the
  update.
- A merge will only fail if the tombstoned file conflicts with another
  change.  Note that as written, merging from a past desk *can* bring a
  tombstoned file to the head of a desk.  Possibly this shouldn't be
  allowed.

This also includes a couple refactors that were made possible by ford
fusion (since everything is synchronous now) but never got done.  In
both cases we get to remove a monad, which simplifies the code
considerably.

- refactor +merge's error handling to use !!/mule instead of threading
  through errors
- refactor all +read-* functions and related parts of +try-fill-sub to
  eagerly convert lobes to cages.

We also add support reading %a/b/c/e/f/r/x from past and foreign desks,
when possible.  Apologies that all of these are in one commit, it was
all a single chunk of work.

This is a draft until we have a way to tombstone.  I suspect we'll want
to have a mechanism of keeping track of gc roots and trace to remove,
but this PR doesn't suggest any particular strategy.
2021-11-07 22:17:02 -08:00
fang
bf4aef198c
clay: give %u answer for non-existent desks
Instead of considering it "in the future", an existence check for paths
on non-existent desks now always produce false.
2021-10-07 21:57:56 +04:00
Ted Blackman
5cce66523c Revert "clay: print cult"
This reverts commit d4bc38f774.
2021-09-29 20:31:57 -04:00
Ted Blackman
86a2715e67 clay: purge %next and %sing on migration 2021-09-29 20:10:53 -04:00
Ted Blackman
d4bc38f774 clay: print cult 2021-09-29 20:01:43 -04:00
Ted Blackman
5e1778ea84 clay: compile failure 2021-09-29 19:43:26 -04:00
Ted Blackman
b16d176863 clay: delete /app/publish from subscription state on migration 2021-09-29 19:36:29 -04:00
Ted Blackman
bd46a367a3 clay: always notify if new read says nonexistent 2021-09-29 17:57:55 -04:00
Ted Blackman
9560fa43b2 gall,clay: fix migration file build dates; add %home as parent desk in jams 2021-09-29 15:00:42 -04:00
Fang
1867209a12
Merge branch 'jm/spider-gall' into dist 2021-09-28 17:12:17 +04:00
Fang
059c2daed3
clay: make distribution desks public on-migrate
To ensure our sponsees (and other ships) can install from them.
2021-09-28 15:59:36 +04:00
ryjm
472f16fd2e garden spider: address comments 2021-09-27 21:23:46 -04:00
ryjm
c630b560ee gall: don't verify spider marks 2021-09-27 21:23:46 -04:00
Ted Blackman
9aa8566a6e -jam-all-desks 2021-09-26 22:03:13 -04:00
Philip Monk
a94e17915f clay: fix failure to load apps
When you loaded an app with an error, then fixed the error, it would
create the main gall %mult subscription at a time in the past.  Then,
clay would never fill the subscription since it couldn't get the old %a
entries for the apps.

This fixes the issue in two ways:  first, don't subscribe in the past.
Second, if clay can't get the old versions, just fire the subscription
anyway.
2021-09-26 00:26:37 -04:00
Ted Blackman
8f115bf059 clay: remote-install desks during migration 2021-09-25 16:53:52 -04:00
ryjm
154deb3281 clay: print stacktrace on build failure 2021-09-25 15:36:00 -04:00
Joe Bryan
5385889ebd Revert "Revert "Merge pull request #4463 from urbit/m/next-gen-term""
This reverts commit 3fba32185d.
2021-09-24 23:14:55 -04:00
Joe Bryan
3fba32185d Revert "Merge pull request #4463 from urbit/m/next-gen-term"
This reverts commit 39667fc071, reversing
changes made to 698fd6f66f.
2021-09-24 16:33:15 -04:00
Fang
ce09681b0a
clay: avoid migrating during boot
We set the "have we migrated yet" flag to true during boot, because
we're starting out in a post-migration world.

Also fixes a state version number.
2021-09-24 20:55:28 +04:00
Ted Blackman
c80fb82318 clay,kiln,landscape/notify: migration runs 2021-09-23 18:07:13 -04:00
Fang
eb1ec5b749
clay: clean up migration error case printfs 2021-09-22 15:45:04 +02:00
Fang
904165f83a
clay: don't forget the extra q
+load still nest-fails though.
2021-09-22 01:39:29 +02:00
Fang
f588ea9feb
clay: grab tmp desk jamfile correctly 2021-09-22 01:16:21 +02:00
Ted Blackman
30364ab29b clay: wip migration 2021-09-21 19:13:54 -04:00
Fang
c0bfab9bf5
clay: wip migrate from desk jams
To work around mark availability issues.
2021-09-21 00:28:31 +02:00
Fang
e9aa3821ef
zuse, clay: further wip migration 2021-09-20 23:13:04 +02:00
Ted Blackman
fb8c73e39a zuse,clay: WIP +migrate-dist 2021-09-20 13:01:48 -04:00
Ted Blackman
84c043d921 lull,clay: WIP migration 2021-08-31 02:25:36 +03:00
Ted Blackman
dda7f13e0e clay: WIP migrate from master 2021-08-26 19:32:52 +03:00
Fang
da85ee2e38
Merge branch 'master' into m/next-gen-term 2021-08-12 15:15:36 +02:00
Ted Blackman
616ab66f26 merge master; loosen :hood's updates 2021-08-11 19:23:36 +03:00
Ted Blackman
d9c9740b8d clay: print desk in +read-a 2021-07-23 14:03:36 +03:00
Ted Blackman
4f5739879e clay: more printing adjustment 2021-07-19 14:01:34 +03:00
Ted Blackman
f521881212 clay: better print 2021-07-18 23:11:13 +03:00
Ted Blackman
4f3e99f797 clay,hood: seems to boot for real now 2021-07-17 02:54:20 +03:00
Ted Blackman
a1b82d4b36 various: can bootstrap off existing ship, but generators are broken 2021-07-16 19:01:55 +03:00
Ted Blackman
5e496e920e clay: invalidate cache on kelvin change 2021-07-14 19:21:00 +03:00
Ted Blackman
f4232ce415 dill,clay: minor printing cleanup 2021-07-14 01:51:35 +03:00
Ted Blackman
529502543b sys,lib: boots! 2021-07-14 01:00:13 +03:00
Ted Blackman
9b968edf75 clay,kiln,/mar/kelvin,/lib/pill: some fixes for pills; not done 2021-07-13 19:02:24 +03:00
Ted Blackman
070bbd0917 clay: use 2021-07-10 08:56:25 -04:00
raghu
ad96d47cdc better formatting in error messages 2021-07-08 22:29:06 -04:00
Ted Blackman
784004cfb6 kiln: +install compiles and loads, untested 2021-06-30 21:03:47 -04:00
Ted Blackman
8c3d14b7a7 clay: check kelvin on non-%base desks 2021-06-28 21:22:55 -04:00
Ted Blackman
40af38c9ee bin/solid.pill 2021-06-28 20:18:37 -04:00
Ted Blackman
501ed4ad82 clay: remove reef 2021-06-28 20:01:51 -04:00
fang
2640a3352a
Merge branch 'master' into m/next-gen-term-real 2021-06-23 11:19:48 +02:00
raghu
5698e2662a Merge remote-tracking branch 'upstream/master' into better-bar-fuse 2021-06-21 20:57:20 -04:00
raghu
a04b318d2d clay: log on strange fuse responses 2021-06-21 20:43:49 -04:00
Philip Monk
85fdd6b190
clay: handle out-of-bounds versions in nako 2021-06-08 15:41:44 -10:00
raghu
b499e1e711 clay: print conflicting source on fuse failure 2021-06-07 20:35:18 -04:00
Philip Monk
bbe3cbe8ee
clay: formatting 2021-05-18 08:32:30 -10:00
Philip Monk
1e6e69a49d
Merge branch 'pr' into release/next-sys 2021-05-18 07:50:26 -10:00
Philip Monk
63ba2823dd
Merge remote-tracking branch 'origin/ted/clay-bunt' into release/next-sys 2021-05-18 07:38:52 -10:00
raghu
44722f3a09 fuse: properly report success 2021-05-11 21:30:44 -04:00
raghu
9db61a1566 fuse: simplify state restoration logic 2021-05-03 22:26:49 -04:00
raghu
2fd8e729e1 fuse: fix indents 2021-05-03 19:29:28 -04:00
raghu
20fb39bba7 fuse: fix comments 2021-05-02 14:15:03 -04:00
raghu
08d3aa2485 fuse: c/r 2021-05-02 14:04:03 -04:00
raghu
75ed2cc0af fuse: c/r 2021-05-02 13:44:18 -04:00
Philip Monk
49bbf37ca2
Merge pull request #4835 from urbit/philip/clay-fixes
clay: fix requests for strange cares over ames
2021-04-29 15:18:05 -07:00
Philip Monk
7b7e9e3104
clay: fix requests for strange cares over ames
Four changes:

- implement +validate-u to allow %u requests over the network
- make +validate-x use our local marks to make %x requests generally
  work over the network
- in +start-request, if a foreign ship is making a request that we
  shouldn't send over the network, ignore it.  This closes a DOS vector.
- in +duce, if we're about to make a request to a foreign ship which
  they won't be able to answer, crash the event.

Combined, these fix many of the common cases of weirdness around foreign
clay requests.  Notably absent is a fix for reading `%a` across the
network, which I still maintain should happen against the foreign
hoon/zuse.

fixes #4834
see also #4307
2021-04-28 23:24:15 -07:00
fang
5d4d7de6a6
Merge pull request #4745 from urbit/m/ford-fassig
clay: add /~ ford rune, as /= for directories
2021-04-26 16:19:19 +02:00
raghu
8cbacb2545 clay: allow successive upgrades 2021-04-25 14:01:55 -04:00
raghu
201c3db193 clay: finalize fuse request - update |fuse 2021-04-25 14:01:45 -04:00
raghu
9d7afd10ee clay: change fuse request 2021-04-25 14:01:33 -04:00
fang
550dee32da
clay: +any:in in place of +lien over +tap:in 2021-04-23 23:05:11 +02:00
fang
dcdf695991
clay: insert %nest-fail into trace conditionally
Previously, we would stack on the ~| for every file we had previously
processed, even though we only want it in the trace for actual failures.
2021-04-23 22:56:24 +02:00
raghu
b2e1008d9a clay: add %fuse 2021-04-19 23:46:46 -04:00
Philip Monk
66b4c3f193
Merge pull request #4677 from urbit/philip/fast-refresh
sys: Speed up things used in Landscape
2021-04-19 15:43:16 -07:00
fang
0570fb5b76
clay: share logic between build-file and directory
By factoring their shared logic out into +build-dependency, which gets
passed the relevant details about how to track the file being built in
the dependency stack.
2021-04-12 13:10:36 +02:00
fang
8b4dba28fa
clay: update dependency logic for /~ imports
To match recent changes to the build logic, where it only includes
top-level hoon files.
2021-04-10 16:22:52 +02:00
fang
a45179412b
clay: for /~, only build top-level files
This is semantically cleaner than arbitrary-depth recursion, and lets us
stop worrying about hyphenated path conflicts.
2021-04-10 02:26:32 +02:00
fang
1411e08944
clay: specify target type for /~ results
Clay will nest-check the results and put a homogenous map into the
subject.
2021-04-10 01:59:26 +02:00
fang
62d7281156
clay: add /~ ford rune, as /= for directories
Hoon files may want to import nouns from all files in a given directory.
/~ lets you do so, importing as a (map @ta *) (but with typed values).

Note the description as "directories" here, instead of "path prefix".
The behavior, as implemented, will not include /path/hoon for /~  /path,
instead only including /path/more/hoon and more deeply nested files.
This seems to be, generally, the behavior you want, for example when
importing from /app/myapp/* for /app/myapp/hoon.

Actually using the resulting map requires some manual casting, which is
not ideal. Some code style improvement work remains to be done as well.
2021-04-09 01:49:14 +02:00
fang
1ec5e5acfb
hoon: add +stap, path parser
And cleans up a bunch of locally hand-written implementations of it.
2021-04-07 21:14:13 +02:00
fang
7e2cc77ae0
clay: explicitly ask dill to print changelist
Instead of emitting a "debug message" gift and hoping dill picks it up
for printing.
2021-03-31 22:28:22 +02:00
Philip Monk
a1647a9767
clay, eyre: add profiling hints 2021-03-28 00:50:25 -07:00
Ted Blackman
b55417a1c3 clay: remove +bunt from marks 2021-03-05 16:53:35 -05:00
Philip Monk
f5ce79da9d
marks: fixup
Many of the marks were broken, and staticizing the marks revealed this.
This makes them build.
2021-03-04 19:21:34 -08:00
Philip Monk
a0146f4a44
clay: add %e, %f requests
For static naves, casts respectively
2021-03-04 18:28:16 -08:00
Philip Monk
4a066c8d4d
clay: fix upgrade 2021-03-04 13:41:07 -08:00
Ted Blackman
9636b889aa clay: speed up +checkout-changes 2021-02-24 14:43:35 -05:00
Ted Blackman
0452eb92f2 Merge branch 'release/next-sys' into HEAD 2021-02-24 13:26:51 -05:00
Ted Blackman
ce61d49ef9 clay: fix |mass; some renaming 2021-02-24 12:59:13 -05:00
Ted Blackman
19bf227428 clay: remove debug artifacts 2021-02-24 12:44:08 -05:00
Ted Blackman
d447e70b09 clay: fix /% parser; clean up unit tests 2021-02-24 12:18:50 -05:00
Ted Blackman
9bf6c6136a clay: fix mark +grad delegation bug 2021-02-22 14:35:57 -05:00
Ted Blackman
674b5edb1f Merge branch 'release/next-sys' into ted/gall-q 2021-02-19 13:42:58 -05:00
Ted Blackman
b702505ac8 clay: print if +read-at-aeon crashes 2021-02-18 21:50:59 -05:00
Ted Blackman
ad20ddb1d4 clay: added mark runes; can boot off solid pill 2021-02-17 14:31:20 -05:00
fang
353e0f7395 clay: condense +pile-rule
Rewrites the `+pile-rule` parsing rule for compactness and legibility.
This is a purely stylistic change.
2021-01-27 12:17:40 -05:00
Ted Blackman
ca1585c841 clay: static clay boots 2021-01-26 22:51:52 -05:00
Ted Blackman
ce9f1eb3da clay: fix +grow in mark casting 2021-01-26 20:43:23 -05:00
Ted Blackman
6be1ebc41a clay,lull: +test-mar-mime passes 2021-01-26 15:59:36 -05:00
Ted Blackman
f469dbf8bc clay,lull: compiles with static marks 2021-01-13 15:20:58 -05:00
Ted Blackman
ece8bbf0a2 clay: WIP static mark and cast builds 2021-01-12 19:10:11 -05:00
fang
1b1b613729
clay: condense +pile-rule
Rewrites the `+pile-rule` parsing rule for compactness and legibility.
This is a purely stylistic change.
2020-12-24 13:03:56 +01:00
Joe Bryan
37a24bb800 clay: build %zuse against %lull 2020-12-07 19:24:34 -08:00
Joe Bryan
aaf506ce8a Merge branch 'jb/motion' into jb/motion-merge
* jb/motion:
  pill: solid
  zuse: remove %crud from vane-task
  arvo: full vane names in $sign
  aqua: build again (still broken)
  arvo: reform of the scry reform
2020-12-07 18:52:51 -08:00
Joe Bryan
5dd3e148b8 zuse: remove %crud from vane-task 2020-12-07 18:01:48 -08:00
fang
d98c59e73a
zuse: lift moves out of +able 2020-12-08 01:47:06 +01:00
Joe Bryan
17736990ed arvo: full vane names in $sign 2020-12-07 16:30:22 -08:00
Joe Bryan
98f2b8c6b3 arvo: reform of the scry reform 2020-12-07 13:52:12 -08:00
Joe Bryan
5bc6d38437 arvo: updates vane interface 2020-12-06 03:33:43 -08:00
Joe Bryan
2c6b1cab44 arvo: removes +sloy-light, updates call sites (%clay, %dill) 2020-12-05 18:45:50 -08:00
Joe Bryan
958bb5acf7 arvo: removes +is and all references to it 2020-12-05 00:51:49 -08:00
Joe Bryan
81b868a364 clay: scry kernel source out of arvo for reef short-circuit 2020-12-03 16:25:44 -08:00
Joe Bryan
83bd3943a8 arvo: adds %lull, vane structures 2020-12-03 15:32:36 -08:00
Joe Bryan
84bec6f7c1 clay: %pass's %home commits to arvo as %what, unconditionally 2020-12-03 15:31:41 -08:00
Joe Bryan
b5ec0c30f0 hoon: renames +new-end, +new-lsh, +new-rsh 2020-12-02 01:00:09 -08:00
Joe Bryan
4f9cba2baf hoon: adds +new-end, switches all +end call sites 2020-12-02 00:21:12 -08:00
Joe Bryan
072e2c8101 Merge branch 'm/vane-unversion' into m/modern-hoon
* m/vane-unversion:
  ames: remove old state versions & conversions
  gall: rename fading styles
  vanes: remove old load types and logic
  gall: refresh clay subscription on fade
  gall: add nonce to agent wires
  gall: add %doze-style fade
  gall: fade initial commit
2020-11-30 21:04:16 -08:00
Joe Bryan
409579398c Merge branch 'philip/clay-version' into m/modern-hoon
* philip/clay-version:
  jael: add version numbers
  gall: add version to over-the-wire protocol
  clay: cleanup protocol versions
2020-11-30 20:54:22 -08:00
Joe Bryan
97cb62bc45 Merge branch 'philip/fade' into m/vane-unversion
* philip/fade:
  gall: rename fading styles
  gall: refresh clay subscription on fade
  gall: add nonce to agent wires
  gall: add %doze-style fade
  gall: fade initial commit
2020-11-30 20:49:04 -08:00
Philip Monk
350521853b
clay: cleanup protocol versions
+riff-any is all clay requests except "backfill" requests.  Change to
`$%` from `$^`, which was used to distinguish originally non-versioned
requests.

+fill is backfill requests and had no version number, so we add one.

We do not have version numbers on responses since those are implied by
the request.  If someone requests at version `n` and you're at `n+1`,
you must respond in the format of `n`.

If someone requests at version `n+1` and you're at `n`, you crash;
though possibly you should be able to respond with message "I only know
up to `n`", in which case they may be able to re-request at `n`.  In
either case, the version of the response is dictated by the request.
2020-11-30 19:25:12 -08:00
fang
732addadd3
various: touch up improperly rewritten comments
Also patches /mar/snip to be more correct, but a &snip [~ ~] in dojo
still does not work.
2020-11-27 13:25:40 +01:00
fang
a539d986a7
various: move away from {type} syntax
In favor of [type] syntax.
Turns a bunch of ++ into +$ along the way.
2020-11-26 17:43:26 +01:00
fang
44fb0cc19d
various: move away from face/type syntax
In favor of face=type.
2020-11-26 17:38:00 +01:00
fang
a6e7af38d5
various: move away from $constant type syntax 2020-11-26 17:31:06 +01:00
Joe Bryan
f95e1f48c2 arvo: scry reform 2020-11-25 16:02:10 -08:00
fang
7956adedee
vanes: remove old load types and logic
Maintains state version numbers to avoid tripping up migration tools.
2020-11-25 16:24:16 +01:00
fang
27d6fc7597
zuse, clay, various: unflop the spur in beams
Unflops the spur in +en-beam, +de-beam, and everything that calls either
of those, or works with the consequences of their output.

This includes clay's interface for mounting and unmounting, which now
no longer expects the arguments to contain an old-style spur.
2020-11-24 15:44:25 +01:00
Philip Monk
a7cc18072a
gall: add %doze-style fade 2020-11-17 12:53:05 -08:00
John Franklin
11f8ba23d7 hoon: bscl to bccl, etc 2020-11-17 01:05:17 -06:00
John Franklin
756c94aeb6 hoon: update glyph names (continued) 2020-11-14 23:08:52 -06:00
Philip Monk
1c8d229fb7
gall: fade initial commit 2020-11-09 23:42:04 -08:00
fang
acf0f8d0e1
clay: %limb instead of %wing 2020-11-03 16:35:28 +01:00
fang
e7afc51018
clay: replace ream with nouns and !, zapcom
Motivation for the change is performance improvements on the un-`^~`d uses of
ream. Parsing turns out to be slow, making ream slow in turn. So we construct
the hoon ast manually instead.

!, is arguably better style than ream, since it doesn't require a ^~ for static
input, and lets syntax highlighting function properly.

For the investigated case, in +get-cast's +grow flow, improves performance by
over 80%.
2020-11-03 00:27:10 +01:00
Philip Monk
8cd72daf20
clay: speed up +find-merge-points
In certain cases +find-merge-points was very slow.  Specifically, the
`done` set was meant to avoid checking the same commit repeatedly, but
it didn't catch the case where a commit was added to the worklist that
was already in that worklist.

Secondly, the worklist was stored as a list but used as a queue, which
resulted in a lot of unnecessary welding.  We change it to a qeu.

Fixes #3735
2020-10-16 11:21:02 -07:00
Joe Bryan
1402f763f3 clay: refactors +cancel-request, consistently accounting for TMI 2020-10-09 10:34:18 -07:00
Philip Monk
f9665c0cbd
Merge pull request #3378 from urbit/philip/base-hash
clay: expose mergebase as base-hash
2020-09-08 20:11:05 -07:00
Philip Monk
c7480c9b07
clay: fix bug in %mate
If both sides changed a file in the same way, %mate used the version in
the mergebase, which is incorrect.  This changes it to use the version
in the destination desk.

An example of this issue:

    > +cat %/test/hoon
    /~zod/home/~2020.9.3..21.41.24..61ed/test/hoon
    first

    > |merge %scratch our %home
    >=
    merged with strategy %fine
    + /~zod/scratch/2/test/hoon

    > +cat /=scratch=/test
    /~zod/scratch/~2020.9.3..21.41.32..408c/test/hoon
    first

    > *%/test/hoon 'second'
    : /~zod/home/3/test/hoon

    > *%%%/scratch=/test/hoon 'second'
    : /~zod/scratch/3/test/hoon

    > |merge %scratch our %home
    >=
    %fine merge failed, trying %meet
    %meet merge failed, trying %mate
    merged with strategy %mate
    : /~zod/scratch/4/test/hoon

    > +cat /=scratch=/test
    /~zod/scratch/~2020.9.3..21.42.25..9e8b/test/hoon
    first
2020-09-03 14:34:30 -07:00
Philip Monk
67c5caada8
clay: add safe merge options
As described in gen/hood/merge/help.txt
2020-08-28 16:36:00 -07:00
Philip Monk
769b43d73a
clay: add comments and cleanup +find-merge-points 2020-08-26 18:32:20 -07:00
Philip Monk
9cd826f8f9
clay: make +find-merge-points faster
The main thing here is that we aggressively check whether we're in
ancestry of another mergebase candidate.  This means we don't have to do
a 2nd pass to eliminate redundant candidates.
2020-08-25 22:54:48 -07:00
Philip Monk
4468d79509
clay: expose mergebase as base-hash
Change the definition of base-hash to be the mergebase of %home with the
OTA source.  This means it's the most recent successfully-applied
update, which is usually the most important information.

Add sour-hash, which is the hash of the most recently *downloaded*
update, regardless of whether it applied successfuly (ie the old
base-hash).

Add a summary of the various hashes at the top of gen/trouble.
2020-08-25 20:25:39 -07:00
Philip Monk
f606d2e3d6
clay: cleaner no-op detection
Only no-op if the incoming commit's parent is the old head of the desk.
Also move the printing near the end so we can know exactly if anything
changed.
2020-08-25 18:15:06 -07:00
Philip Monk
d5596eb1a3
clay: if commit would be a no-op, don't commit it
fixes #3317
2020-08-25 16:24:36 -07:00
Philip Monk
3d1b7f2b35
clay: fix ph tests 2020-07-28 21:44:45 -07:00
Philip Monk
495a6cf004
clay: remove debugging printfs 2020-07-24 22:05:49 -07:00
Philip Monk
51983e5480
clay: rewrite new protocol
Use an explicit state machine to carefully manage pending downloads.
2020-07-24 00:20:22 -07:00
Philip Monk
553a9db843
clay: refactor over-the-wire protocol
Don't send blobs initially; instead, wait for the recipient to ask for
any they need.  This should significantly reduce network and memory usage.
2020-07-23 00:12:33 -07:00
Philip Monk
f4c34c7598
release: urbit-os-v1.0.29 2020-07-21 12:59:12 -07:00
Philip Monk
f4f1d0ab4b
clay: print conflicts 2020-06-30 15:24:42 -07:00
Philip Monk
e7e2c07d6f
clay: remove scaffolding 2020-06-29 14:30:33 -07:00
Philip Monk
40db8ea580
kiln: make otas continue even if they failed to apply 2020-06-29 13:25:10 -07:00
Ted Blackman
ede7105820 clay: flop syntax error trace 2020-06-29 11:01:23 -04:00
Philip Monk
42de999024
kiln: don't implicitly create syncs
Also silence some spurious errors.
2020-06-26 19:21:09 -07:00
Philip Monk
3a662881be
clay: don't be evil 2020-06-24 21:24:27 -07:00
Philip Monk
fc42bf6039
clay: special-case +run-pact for %hoon 2020-06-24 20:26:04 -07:00
Ted Blackman
652dc70304 clay,dojo: clean up stack traces 2020-06-20 03:46:22 -04:00
Ted Blackman
d1b4af89fe sys: s/mure/road 2020-06-19 00:44:17 -04:00
Philip Monk
7494bf57c6
clay: make reachable-takos linear instead of exponential
When merging, +reachable-takos is called roughly once per merge commit
in the ancestry of the new commit.  +reachable-takos was exponential in
the number of merge commits in the ancestry of the commit it's looking
at, due to mishandling of the accumulator.  This makes it linear.

Of course, linear x linear is still quadratic, which is not great.  I
doubt +reachable-takos can be made asymptotically better, but
+reduce-merge-points/+find-merge-points probably can.  50 merge commits
already gives about 14.000 iterations through the loop in
+reachable-takos.  Another option is to try to memoize this somehow, but
a simple ~+ is insufficient since `s` is usually different.

In local tests on macOS with a -L copy of ~wicdev-wisryt, this speeds up
OTAs significantly.  The majority of time was spent on this.
2020-06-18 18:56:31 -07:00
Ted Blackman
1b1d7c9512 Merge remote-tracking branch 'origin/m/clay-fusion-markers' into ota-ford-fusion 2020-06-13 02:09:31 -04:00