sapling/tests
Tony Tung 1a1da38248 [manifestdiskcache] RFC: a disk cache for manifests
Summary:
# Intercept manifest.revision(..).  The cache is checked, and if it's not there, resume the normal path.  Once the normal path is complete, write to the cache.
# Intercept manifest._addrevision(..).  This may be used in bulk operations, such as pull.  Since we don't want to flood the cache with a lot of entries we may not care about, we record all the nodes that are added.  We add an atexit hook to then record the last N nodes to the cache.
# Writes to the cache are done to a temp file, then atomically renamed into place.
# On reads, we run the checkhash function.  This costs us ~100ms per manifest in fbsource, but ensures a corrupt cache doesn't break us.  Love to debate this matter.
# On each batch of writes, we spawn a background copy of ourselves to prune the cache.  We use the mtime of a marker file to determine the last time the prune happened.  We calculate the odds that we should be doing the prune using a couple constants and the time since the last prune.  If another prune happened recently, the dominant factor is a small probability that we run the prune, regardless of the interval.  If no prune has happened recently, the dominant factor is the seconds-since-prune configuration variable.

Some performance numbers (all averaged across 10 runs)
* diff between two revs: 2.29s without caching; 1.75s with caching
* rebasing 4 diffs: 12.1s without caching; 10.7s with caching

Test Plan: passed the rudimentary correctness unit tests.  with fbsource, ran a small handful of commands without anybody tripping and falling.

Reviewers: rmcelroy, ericsumner, mpm, pyd, durham

Reviewed By: durham

Subscribers: akushner, mitrandir, cdelahousse

Differential Revision: https://phabricator.fb.com/D2564490

Signature: t1:2564490:1449075868:70974c62e6bff6521b6f500b5bff3a260ddd6c6d
2015-12-02 11:42:47 -08:00
..
conduithttp.py fix breakages due to fbconduit api changes regarding missing revs 2015-08-10 21:22:21 -07:00
copytrace.sh copytrace: adding ui config to allow incremental deployment 2015-11-20 09:38:21 -08:00
dummyssh Add run-tests back into the repo 2015-11-09 11:33:27 -08:00
histedit-helpers.sh fb-histedit: Add a histedit extension that adds stop 2014-10-09 10:54:43 -07:00
killdaemons.py Add missing killdaemons 2015-11-09 12:21:39 -08:00
printenv.py Add run-tests back into the repo 2015-11-09 11:33:27 -08:00
run-tests.py Add run-tests back into the repo 2015-11-09 11:33:27 -08:00
test-automv.t Adding automatically 'move metadata' if a move is detected 2015-09-30 09:28:40 -07:00
test-bundle2.t copytrace: running the tests through dummy ssh 2015-12-01 11:23:51 -08:00
test-catnotate.t catnotate: add whole extension 2015-10-16 02:45:27 -07:00
test-commitextras.t extras: add commitextras extension 2015-04-27 20:20:48 -07:00
test-copytrace.t copytrace: adding ui config to allow incremental deployment 2015-11-20 09:38:21 -08:00
test-dirsync.t dirsync: fix 'hg commit file.txt' 2015-11-25 11:21:47 -08:00
test-fbamend.t fbamend: fix --logfile for hg amend 2015-10-27 14:37:20 -07:00
test-fbconduit.t changed the port the unit test runs on. 2015-07-15 14:50:46 -07:00
test-fbhistedit-exec-obsolete.t histedit: fix --continue with exec 2015-02-17 20:04:40 -08:00
test-fbhistedit-exec.t Update tests 2015-10-26 15:10:59 -07:00
test-fbhistedit-stop-obsolete.t Update tests 2015-10-26 15:10:59 -07:00
test-fbhistedit-stop.t fbhistedit: Fix abort after stop 2015-11-17 15:07:32 -08:00
test-filldb.t dbutil: manually add missing move data 2015-11-20 09:38:21 -08:00
test-git-getmeta.t Dropped local run-tests in favour of using mercurial's own test runner 2015-10-01 13:03:03 -07:00
test-githelp.t githelp: Handle the "git branch -m new" case correctly 2015-11-21 18:02:15 +00:00
test-gitlikebookmarks.t gitlikebookmark: add a -x option for git like bookmarks for rebase and histedit 2015-11-25 10:09:45 -08:00
test-inhibitwarn.t inhibitwarn: fix broken test 2015-08-06 11:20:20 -07:00
test-manifestdiskcache.t [manifestdiskcache] RFC: a disk cache for manifests 2015-12-02 11:42:47 -08:00
test-mergedriver.t test-mergedriver.t: update output 2015-11-17 16:53:47 -08:00
test-morestatus.t Add interrupted update state to morestatus 2015-11-27 05:16:53 -08:00
test-perftweaks.t perftweaks: change revlog delta heuristic 2015-11-24 15:18:39 -08:00
test-phabdiff.t Added phabdiff template mapping 2014-10-16 17:43:58 -07:00
test-pushrebase-protection.t pushrebase: improve messages when pushing 2015-10-28 18:19:18 -07:00
test-pushrebase-remotenames.t fix tests for new remotenames 2015-11-17 17:54:28 -08:00
test-pushrebase.t update extensions and tests for latest mercurial 2015-11-17 17:54:31 -08:00
test-pushvars-remotenames.t Fix pushvars with remotenames 2015-07-17 15:47:30 -07:00
test-pushvars.t Update to match upstream 2015-05-27 17:37:19 -07:00
test-reflog-remotenames.t reflog: fix devel-warn abotu dirstate.write 2015-10-26 17:51:36 -07:00
test-reflog.t Adding --commits to reflog 2015-11-17 19:03:28 -08:00
test-reset-remotenames.t reset: fix resetting past remote bookmarks 2015-07-10 11:40:30 -07:00
test-reset.t Update tests 2015-10-26 15:10:59 -07:00
test-simplecache.t simplecache: move memcache prefix to mcget/set 2015-06-15 17:19:08 -07:00
test-smartlog-remotenames.t fix tests for new remotenames 2015-11-17 17:54:28 -08:00
test-smartlog.t Fixing smartlog errors on empty repo 2015-09-15 16:13:13 -07:00
test-sparse-extensions.t sparse: move hgwatchman integration test to test-sparse-extensions.t 2015-06-15 17:44:39 -07:00
test-sparse-merges.t sparse: fix temporary includes that are not in working copy 2015-05-19 10:18:50 -07:00
test-sparse-profiles.t Turn debug message into warning; not finding a sparse profile affects the user directly 2015-11-19 08:47:45 +00:00
test-sparse.t sparse: forbid paths starting with / 2015-12-01 13:17:31 -08:00
test-tweakdefaults-ordering.t tweakdefaults: fix ordering issue for real 2015-09-18 10:15:59 -07:00
test-tweakdefaults-remotenames.t tweakdefaults: add default pull destination config 2015-11-25 18:23:07 -08:00
test-tweakdefaults.t update extensions and tests for latest mercurial 2015-11-17 17:54:31 -08:00
test-writecg2.t writecg2: update test, no longer writing CG2 bundles 2015-11-04 13:03:51 -08:00