mirror of
https://github.com/facebook/sapling.git
synced 2024-10-11 01:07:15 +03:00
b4bd534579
Summary: Previously we only wrapped manifestlog.__getitem__ because fastmanifest didn't support the concept of directory manifests. Unfortunately manifestlog.get() is used in some flat manifest cases too, like in changegroup.py where it just passes '' as the directory but still uses .get(). This bug has been present since the manifestlog refactor, but only became exposed now that we've improved our caching and hold on to manifest ctx's for longer. This fix is to just wrap manifestlog.get() too. Fixing this exposed some other inconsistencies with our fastmanifest api. Test Plan: shelve now passes in a repo that was hitting an exception because of this before. I'm still trying to think of how to test this in a test. I've not been able to repro it in my own repo yet. Reviewers: #mercurial, quark Reviewed By: quark Differential Revision: https://phabricator.intern.facebook.com/D4540677 Tasks: 15991119 Signature: t1:4540677:1486690231:07a95b402913d7dcc31633341e235ee1eb0163bc
365 lines
16 KiB
Raku
365 lines
16 KiB
Raku
Setup
|
|
|
|
$ PYTHONPATH=$TESTDIR/..:$PYTHONPATH
|
|
$ export PYTHONPATH
|
|
|
|
Check diagnosis, debugging information
|
|
1) Setup configuration
|
|
$ mkcommit() {
|
|
> echo "$1" > "$1"
|
|
> hg add "$1"
|
|
> echo "add $1" > msg
|
|
> echo "" >> msg
|
|
> hg ci -l msg
|
|
> }
|
|
|
|
2) Basic cache testing
|
|
|
|
$ mkdir cachetesting
|
|
$ cd cachetesting
|
|
$ hg init
|
|
$ cat >> .hg/hgrc << EOF
|
|
> [extensions]
|
|
> fastmanifest=
|
|
> [fastmanifest]
|
|
> cachecutoffdays=-1
|
|
> debugmetrics=True
|
|
> randomorder=False
|
|
> EOF
|
|
|
|
Here we expect a cache hit of 100% because we are based of revision -1 that
|
|
is cached
|
|
$ mkcommit a
|
|
[FM-METRICS] Begin metrics
|
|
[FM-METRICS] kind: cachehitratio, kwargs: [('cachehitratio', -1)]
|
|
[FM-METRICS] kind: diffcachehitratio, kwargs: [('diffcachehitratio', -1)]
|
|
[FM-METRICS] kind: filesnotincachehitratio, kwargs: [('filesnotincachehitratio', -1)]
|
|
[FM-METRICS] End metrics
|
|
[FM-METRICS] Begin metrics
|
|
[FM-METRICS] kind: cachehitratio, kwargs: [('cachehitratio', 100)]
|
|
[FM-METRICS] kind: diffcachehitratio, kwargs: [('diffcachehitratio', -1)]
|
|
[FM-METRICS] kind: filesnotincachehitratio, kwargs: [('filesnotincachehitratio', -1)]
|
|
[FM-METRICS] End metrics
|
|
|
|
For all the next ones, we expect a cache hit of 50% because we are checking the
|
|
manifest of each parent, one is cached (-1) and the other is not:
|
|
$ mkcommit b
|
|
[FM-METRICS] Begin metrics
|
|
[FM-METRICS] kind: cachehitratio, kwargs: [('cachehitratio', -1)]
|
|
[FM-METRICS] kind: diffcachehitratio, kwargs: [('diffcachehitratio', -1)]
|
|
[FM-METRICS] kind: filesnotincachehitratio, kwargs: [('filesnotincachehitratio', -1)]
|
|
[FM-METRICS] End metrics
|
|
[FM-METRICS] Begin metrics
|
|
[FM-METRICS] kind: cachehitratio, kwargs: [('cachehitratio', 33)]
|
|
[FM-METRICS] kind: diffcachehitratio, kwargs: [('diffcachehitratio', -1)]
|
|
[FM-METRICS] kind: filesnotincachehitratio, kwargs: [('filesnotincachehitratio', -1)]
|
|
[FM-METRICS] End metrics
|
|
$ mkcommit c
|
|
[FM-METRICS] Begin metrics
|
|
[FM-METRICS] kind: cachehitratio, kwargs: [('cachehitratio', -1)]
|
|
[FM-METRICS] kind: diffcachehitratio, kwargs: [('diffcachehitratio', -1)]
|
|
[FM-METRICS] kind: filesnotincachehitratio, kwargs: [('filesnotincachehitratio', -1)]
|
|
[FM-METRICS] End metrics
|
|
[FM-METRICS] Begin metrics
|
|
[FM-METRICS] kind: cachehitratio, kwargs: [('cachehitratio', 33)]
|
|
[FM-METRICS] kind: diffcachehitratio, kwargs: [('diffcachehitratio', -1)]
|
|
[FM-METRICS] kind: filesnotincachehitratio, kwargs: [('filesnotincachehitratio', -1)]
|
|
[FM-METRICS] End metrics
|
|
$ mkcommit d
|
|
[FM-METRICS] Begin metrics
|
|
[FM-METRICS] kind: cachehitratio, kwargs: [('cachehitratio', -1)]
|
|
[FM-METRICS] kind: diffcachehitratio, kwargs: [('diffcachehitratio', -1)]
|
|
[FM-METRICS] kind: filesnotincachehitratio, kwargs: [('filesnotincachehitratio', -1)]
|
|
[FM-METRICS] End metrics
|
|
[FM-METRICS] Begin metrics
|
|
[FM-METRICS] kind: cachehitratio, kwargs: [('cachehitratio', 33)]
|
|
[FM-METRICS] kind: diffcachehitratio, kwargs: [('diffcachehitratio', -1)]
|
|
[FM-METRICS] kind: filesnotincachehitratio, kwargs: [('filesnotincachehitratio', -1)]
|
|
[FM-METRICS] End metrics
|
|
$ mkcommit e
|
|
[FM-METRICS] Begin metrics
|
|
[FM-METRICS] kind: cachehitratio, kwargs: [('cachehitratio', -1)]
|
|
[FM-METRICS] kind: diffcachehitratio, kwargs: [('diffcachehitratio', -1)]
|
|
[FM-METRICS] kind: filesnotincachehitratio, kwargs: [('filesnotincachehitratio', -1)]
|
|
[FM-METRICS] End metrics
|
|
[FM-METRICS] Begin metrics
|
|
[FM-METRICS] kind: cachehitratio, kwargs: [('cachehitratio', 33)]
|
|
[FM-METRICS] kind: diffcachehitratio, kwargs: [('diffcachehitratio', -1)]
|
|
[FM-METRICS] kind: filesnotincachehitratio, kwargs: [('filesnotincachehitratio', -1)]
|
|
[FM-METRICS] End metrics
|
|
$ hg diff -c . --debug --nodate
|
|
[FM] cache miss for fastmanifest f064a7f8e3e138341587096641d86e9d23cd9778
|
|
[FM] cache miss for fastmanifest 7ab5760d084a24168f7595c38c00f4bbc2e308d9
|
|
[FM] performing diff
|
|
[FM] diff: other side is hybrid manifest
|
|
[FM] diff: cache and tree miss
|
|
diff -r 47d2a3944de8b013de3be9578e8e344ea2e6c097 -r 9d206ffc875e1bc304590549be293be36821e66c e
|
|
--- /dev/null
|
|
+++ b/e
|
|
@@ -0,0 +1,1 @@
|
|
+e
|
|
[FM-METRICS] Begin metrics
|
|
[FM-METRICS] kind: cachehitratio, kwargs: [('cachehitratio', 0)]
|
|
[FM-METRICS] kind: diffcachehitratio, kwargs: [('diffcachehitratio', 0)]
|
|
[FM-METRICS] kind: filesnotincachehitratio, kwargs: [('filesnotincachehitratio', -1)]
|
|
[FM-METRICS] End metrics
|
|
$ cat >> .hg/hgrc << EOF
|
|
> [fastmanifest]
|
|
> debugmetrics=False
|
|
> EOF
|
|
|
|
$ hg debugcachemanifest -a
|
|
$ hg debugcachemanifest --list
|
|
fast7ab5760d084a24168f7595c38c00f4bbc2e308d9 (size 328 bytes)
|
|
fastf064a7f8e3e138341587096641d86e9d23cd9778 (size 280 bytes)
|
|
faste3738bf5439958f89499a656982023aba57b076e (size 232 bytes)
|
|
fasta539ce0c1a22b0ecf34498f9f5ce8ea56df9ecb7 (size 184 bytes)
|
|
fasta0c8bcbbb45c63b90b70ad007bf38961f64f2af0 (size 136 bytes)
|
|
cache size is: 1.13 KB
|
|
number of entries is: 5
|
|
Most relevant cache entries appear first
|
|
================================================================================
|
|
manifest node |revs
|
|
7ab5760d084a24168f7595c38c00f4bbc2e308d9|4
|
|
f064a7f8e3e138341587096641d86e9d23cd9778|3
|
|
e3738bf5439958f89499a656982023aba57b076e|2
|
|
a539ce0c1a22b0ecf34498f9f5ce8ea56df9ecb7|1
|
|
a0c8bcbbb45c63b90b70ad007bf38961f64f2af0|0
|
|
$ hg diff -c . --debug --nodate
|
|
[FM] cache hit for fastmanifest f064a7f8e3e138341587096641d86e9d23cd9778
|
|
[FM] cache hit for fastmanifest 7ab5760d084a24168f7595c38c00f4bbc2e308d9
|
|
[FM] performing diff
|
|
[FM] diff: other side is hybrid manifest
|
|
diff -r 47d2a3944de8b013de3be9578e8e344ea2e6c097 -r 9d206ffc875e1bc304590549be293be36821e66c e
|
|
--- /dev/null
|
|
+++ b/e
|
|
@@ -0,0 +1,1 @@
|
|
+e
|
|
|
|
Test the --pruneall command to prune all the cached manifests
|
|
$ hg debugcachemanifest --pruneall --debug
|
|
[FM] caching revset: [], pruneall(True), list(False)
|
|
[FM] removing cached manifest fasta0c8bcbbb45c63b90b70ad007bf38961f64f2af0
|
|
[FM] removing cached manifest fasta539ce0c1a22b0ecf34498f9f5ce8ea56df9ecb7
|
|
[FM] removing cached manifest faste3738bf5439958f89499a656982023aba57b076e
|
|
[FM] removing cached manifest fastf064a7f8e3e138341587096641d86e9d23cd9778
|
|
[FM] removing cached manifest fast7ab5760d084a24168f7595c38c00f4bbc2e308d9
|
|
|
|
$ hg diff -c . --debug --nodate
|
|
[FM] cache miss for fastmanifest f064a7f8e3e138341587096641d86e9d23cd9778
|
|
[FM] cache miss for fastmanifest 7ab5760d084a24168f7595c38c00f4bbc2e308d9
|
|
[FM] performing diff
|
|
[FM] diff: other side is hybrid manifest
|
|
[FM] diff: cache and tree miss
|
|
diff -r 47d2a3944de8b013de3be9578e8e344ea2e6c097 -r 9d206ffc875e1bc304590549be293be36821e66c e
|
|
--- /dev/null
|
|
+++ b/e
|
|
@@ -0,0 +1,1 @@
|
|
+e
|
|
|
|
$ cat >> .hg/hgrc << EOF
|
|
> [fastmanifest]
|
|
> cacheonchange=True
|
|
> cacheonchangebackground=False
|
|
> EOF
|
|
$ mkcommit f
|
|
$ hg book --debug foo
|
|
[FM] skipped 1853a742c28c3a531336bbb3d677d2e2d8937027, already cached (fast path)
|
|
[FM] refreshing 1853a742c28c3a531336bbb3d677d2e2d8937027 with delay 0
|
|
[FM] skipped 7ab5760d084a24168f7595c38c00f4bbc2e308d9, already cached (fast path)
|
|
[FM] refreshing 7ab5760d084a24168f7595c38c00f4bbc2e308d9 with delay 0
|
|
[FM] skipped f064a7f8e3e138341587096641d86e9d23cd9778, already cached (fast path)
|
|
[FM] refreshing f064a7f8e3e138341587096641d86e9d23cd9778 with delay 0
|
|
[FM] skipped e3738bf5439958f89499a656982023aba57b076e, already cached (fast path)
|
|
[FM] refreshing e3738bf5439958f89499a656982023aba57b076e with delay 0
|
|
[FM] skipped a539ce0c1a22b0ecf34498f9f5ce8ea56df9ecb7, already cached (fast path)
|
|
[FM] refreshing a539ce0c1a22b0ecf34498f9f5ce8ea56df9ecb7 with delay 0
|
|
[FM] skipped a0c8bcbbb45c63b90b70ad007bf38961f64f2af0, already cached (fast path)
|
|
[FM] refreshing a0c8bcbbb45c63b90b70ad007bf38961f64f2af0 with delay 0
|
|
[FM] refreshing 1853a742c28c3a531336bbb3d677d2e2d8937027 with delay 0
|
|
[FM] refreshing 7ab5760d084a24168f7595c38c00f4bbc2e308d9 with delay 2
|
|
[FM] refreshing f064a7f8e3e138341587096641d86e9d23cd9778 with delay 4
|
|
[FM] refreshing e3738bf5439958f89499a656982023aba57b076e with delay 6
|
|
[FM] refreshing a539ce0c1a22b0ecf34498f9f5ce8ea56df9ecb7 with delay 8
|
|
[FM] refreshing a0c8bcbbb45c63b90b70ad007bf38961f64f2af0 with delay 10
|
|
$ hg diff -c . --debug --nodate
|
|
[FM] cache hit for fastmanifest 7ab5760d084a24168f7595c38c00f4bbc2e308d9
|
|
[FM] cache hit for fastmanifest 1853a742c28c3a531336bbb3d677d2e2d8937027
|
|
[FM] performing diff
|
|
[FM] diff: other side is hybrid manifest
|
|
diff -r 9d206ffc875e1bc304590549be293be36821e66c -r bbc3e467917630e7d77cd77298e1027030972893 f
|
|
--- /dev/null
|
|
+++ b/f
|
|
@@ -0,0 +1,1 @@
|
|
+f
|
|
|
|
$ hg debugcachemanifest --all --debug
|
|
[FM] caching revset: ['fastmanifesttocache()'], pruneall(False), list(False)
|
|
[FM] skipped 1853a742c28c3a531336bbb3d677d2e2d8937027, already cached (fast path)
|
|
[FM] refreshing 1853a742c28c3a531336bbb3d677d2e2d8937027 with delay 0
|
|
[FM] skipped 7ab5760d084a24168f7595c38c00f4bbc2e308d9, already cached (fast path)
|
|
[FM] refreshing 7ab5760d084a24168f7595c38c00f4bbc2e308d9 with delay 0
|
|
[FM] skipped f064a7f8e3e138341587096641d86e9d23cd9778, already cached (fast path)
|
|
[FM] refreshing f064a7f8e3e138341587096641d86e9d23cd9778 with delay 0
|
|
[FM] skipped e3738bf5439958f89499a656982023aba57b076e, already cached (fast path)
|
|
[FM] refreshing e3738bf5439958f89499a656982023aba57b076e with delay 0
|
|
[FM] skipped a539ce0c1a22b0ecf34498f9f5ce8ea56df9ecb7, already cached (fast path)
|
|
[FM] refreshing a539ce0c1a22b0ecf34498f9f5ce8ea56df9ecb7 with delay 0
|
|
[FM] skipped a0c8bcbbb45c63b90b70ad007bf38961f64f2af0, already cached (fast path)
|
|
[FM] refreshing a0c8bcbbb45c63b90b70ad007bf38961f64f2af0 with delay 0
|
|
[FM] refreshing 1853a742c28c3a531336bbb3d677d2e2d8937027 with delay 0
|
|
[FM] refreshing 7ab5760d084a24168f7595c38c00f4bbc2e308d9 with delay 2
|
|
[FM] refreshing f064a7f8e3e138341587096641d86e9d23cd9778 with delay 4
|
|
[FM] refreshing e3738bf5439958f89499a656982023aba57b076e with delay 6
|
|
[FM] refreshing a539ce0c1a22b0ecf34498f9f5ce8ea56df9ecb7 with delay 8
|
|
[FM] refreshing a0c8bcbbb45c63b90b70ad007bf38961f64f2af0 with delay 10
|
|
$ hg debugcachemanifest --pruneall
|
|
$ hg log -r "fastmanifesttocache()" -T '{rev}\n'
|
|
0
|
|
1
|
|
2
|
|
3
|
|
4
|
|
5
|
|
$ hg log -r "fastmanifestcached()" -T '{rev}\n'
|
|
$ hg debugcachemanifest --all --debug
|
|
[FM] caching revset: ['fastmanifesttocache()'], pruneall(False), list(False)
|
|
[FM] cache miss for fastmanifest 1853a742c28c3a531336bbb3d677d2e2d8937027
|
|
[FM] caching revision 1853a742c28c3a531336bbb3d677d2e2d8937027
|
|
[FM] cache miss for fastmanifest 7ab5760d084a24168f7595c38c00f4bbc2e308d9
|
|
[FM] caching revision 7ab5760d084a24168f7595c38c00f4bbc2e308d9
|
|
[FM] cache miss for fastmanifest f064a7f8e3e138341587096641d86e9d23cd9778
|
|
[FM] caching revision f064a7f8e3e138341587096641d86e9d23cd9778
|
|
[FM] cache miss for fastmanifest e3738bf5439958f89499a656982023aba57b076e
|
|
[FM] caching revision e3738bf5439958f89499a656982023aba57b076e
|
|
[FM] cache miss for fastmanifest a539ce0c1a22b0ecf34498f9f5ce8ea56df9ecb7
|
|
[FM] caching revision a539ce0c1a22b0ecf34498f9f5ce8ea56df9ecb7
|
|
[FM] cache miss for fastmanifest a0c8bcbbb45c63b90b70ad007bf38961f64f2af0
|
|
[FM] caching revision a0c8bcbbb45c63b90b70ad007bf38961f64f2af0
|
|
[FM] refreshing 1853a742c28c3a531336bbb3d677d2e2d8937027 with delay 0
|
|
[FM] refreshing 7ab5760d084a24168f7595c38c00f4bbc2e308d9 with delay 2
|
|
[FM] refreshing f064a7f8e3e138341587096641d86e9d23cd9778 with delay 4
|
|
[FM] refreshing e3738bf5439958f89499a656982023aba57b076e with delay 6
|
|
[FM] refreshing a539ce0c1a22b0ecf34498f9f5ce8ea56df9ecb7 with delay 8
|
|
[FM] refreshing a0c8bcbbb45c63b90b70ad007bf38961f64f2af0 with delay 10
|
|
$ hg log -r "fastmanifesttocache()" -T '{rev}\n'
|
|
0
|
|
1
|
|
2
|
|
3
|
|
4
|
|
5
|
|
$ hg log -r "fastmanifestcached()" -T '{rev}\n'
|
|
0
|
|
1
|
|
2
|
|
3
|
|
4
|
|
5
|
|
|
|
$ hg debugcachemanifest --debug --list
|
|
[FM] caching revset: [], pruneall(False), list(True)
|
|
fast1853a742c28c3a531336bbb3d677d2e2d8937027 (size 376 bytes)
|
|
fast7ab5760d084a24168f7595c38c00f4bbc2e308d9 (size 328 bytes)
|
|
fastf064a7f8e3e138341587096641d86e9d23cd9778 (size 280 bytes)
|
|
faste3738bf5439958f89499a656982023aba57b076e (size 232 bytes)
|
|
fasta539ce0c1a22b0ecf34498f9f5ce8ea56df9ecb7 (size 184 bytes)
|
|
fasta0c8bcbbb45c63b90b70ad007bf38961f64f2af0 (size 136 bytes)
|
|
cache size is: 1.50 KB
|
|
number of entries is: 6
|
|
Most relevant cache entries appear first
|
|
================================================================================
|
|
manifest node |revs
|
|
1853a742c28c3a531336bbb3d677d2e2d8937027|5
|
|
7ab5760d084a24168f7595c38c00f4bbc2e308d9|4
|
|
f064a7f8e3e138341587096641d86e9d23cd9778|3
|
|
e3738bf5439958f89499a656982023aba57b076e|2
|
|
a539ce0c1a22b0ecf34498f9f5ce8ea56df9ecb7|1
|
|
a0c8bcbbb45c63b90b70ad007bf38961f64f2af0|0
|
|
|
|
Check that trimming to a limit higher than what is cached does nothing
|
|
$ hg debugcachemanifest --debug --limit=2048
|
|
[FM] caching revset: [], pruneall(False), list(False)
|
|
|
|
Trim the cache to at most 1kb, we should start from the oldest entry to the
|
|
newest ones:
|
|
$ hg debugcachemanifest --debug --limit=1024
|
|
[FM] caching revset: [], pruneall(False), list(False)
|
|
[FM] removing cached manifest fasta0c8bcbbb45c63b90b70ad007bf38961f64f2af0
|
|
[FM] removing cached manifest fasta539ce0c1a22b0ecf34498f9f5ce8ea56df9ecb7
|
|
[FM] removing cached manifest faste3738bf5439958f89499a656982023aba57b076e
|
|
$ hg debugcachemanifest --debug --list
|
|
[FM] caching revset: [], pruneall(False), list(True)
|
|
fast1853a742c28c3a531336bbb3d677d2e2d8937027 (size 376 bytes)
|
|
fast7ab5760d084a24168f7595c38c00f4bbc2e308d9 (size 328 bytes)
|
|
fastf064a7f8e3e138341587096641d86e9d23cd9778 (size 280 bytes)
|
|
cache size is: 984 bytes
|
|
number of entries is: 3
|
|
Most relevant cache entries appear first
|
|
================================================================================
|
|
manifest node |revs
|
|
1853a742c28c3a531336bbb3d677d2e2d8937027|5
|
|
7ab5760d084a24168f7595c38c00f4bbc2e308d9|4
|
|
f064a7f8e3e138341587096641d86e9d23cd9778|3
|
|
$ hg log -r "fastmanifestcached()" -T '{rev}\n'
|
|
3
|
|
4
|
|
5
|
|
|
|
Trim the cache to at most 512 bytes
|
|
$ hg debugcachemanifest --debug --limit=512
|
|
[FM] caching revset: [], pruneall(False), list(False)
|
|
[FM] removing cached manifest fastf064a7f8e3e138341587096641d86e9d23cd9778
|
|
[FM] removing cached manifest fast7ab5760d084a24168f7595c38c00f4bbc2e308d9
|
|
$ hg debugcachemanifest --debug --list
|
|
[FM] caching revset: [], pruneall(False), list(True)
|
|
fast1853a742c28c3a531336bbb3d677d2e2d8937027 (size 376 bytes)
|
|
cache size is: 376 bytes
|
|
number of entries is: 1
|
|
Most relevant cache entries appear first
|
|
================================================================================
|
|
manifest node |revs
|
|
1853a742c28c3a531336bbb3d677d2e2d8937027|5
|
|
$ hg log -r "fastmanifestcached()" -T '{rev}\n'
|
|
5
|
|
|
|
Trim the cache to at most 100 bytes
|
|
$ hg debugcachemanifest --debug --limit=100
|
|
[FM] caching revset: [], pruneall(False), list(False)
|
|
[FM] removing cached manifest fast1853a742c28c3a531336bbb3d677d2e2d8937027
|
|
$ hg debugcachemanifest --debug --list
|
|
[FM] caching revset: [], pruneall(False), list(True)
|
|
cache size is: 0 bytes
|
|
number of entries is: 0
|
|
|
|
Use the cache in a commit.
|
|
$ hg debugcachemanifest -a
|
|
$ mkcommit g
|
|
$ cd ..
|
|
|
|
Amend a changeset which probably requires readdelta to be implemented
|
|
$ hg init amendtest
|
|
$ cd amendtest
|
|
$ cat >> .hg/hgrc << EOF
|
|
> [extensions]
|
|
> fastmanifest=
|
|
> EOF
|
|
$ mkcommit a
|
|
$ hg commit --amend -m amend -q
|
|
|
|
Ensure shelve works (t15991119)
|
|
$ echo >> a
|
|
$ hg commit -m a2
|
|
$ hg up 0
|
|
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
|
|
$ echo b >> a
|
|
$ hg debugcachemanifest -r .
|
|
|
|
# 0 must be public so it doesnt get added to the bundle. That triggers a
|
|
# condition where the manifest ctx cache wont contain the manifest for commit 0,
|
|
# which tests an edge case in fastmanifest.
|
|
$ hg phase -p -r 0
|
|
$ hg shelve --config extensions.shelve=
|
|
shelved as default
|
|
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
|