Summary:
Previously, a local commit would only write data packs, and it just threw away
the history data entirely. Let's add history packs and record them on commit.
Test Plan:
The tests are updated to show these new packs. In some cases the
datapacks got smaller as well, since they can now take advantage of history data
for delta choices.
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4956105
Signature: t1:4956105:1493265399:d3fa1052c207fba0045cbb92b4d833d18d48e099
Summary:
Previously, the logic that added data to a mutable history pack was required to
add it in the correct order (all entries for a certain file at once, and in
newest-first order). This required the callers to jump through weird hoops if
the data came in out of order or at different times in the transaction.
This patch moves the ordering logic to be inside MutableHistoryPack, so callers
can add the data in any order they wish, and it will get sorted before being
serialized.
This does add memory pressure to things that read a lot of history, like repack.
If this becomes a problem we may want to add a 'historypack.flush()' api that
let's us tell the history pack it's ok to flush it's current contents to disk.
Test Plan: Ran the tests
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4956096
Signature: t1:4956096:1493264693:a2275a49e35565d4b11244e3e5dd82c25de7e16e
Summary:
The code that creates new trees when the user commits only creates new trees if
the parent commit has a tree already. If the commit is the very first commit in
the repo (i.e. p1 == nullid), then the parent tree is empty. The `if tree:`
check considered this not-a-tree and therefore the first commit did not generate
a tree. Let's switch to `if tree is None:` to handle this.
A future patch adds tests around peer-to-peer pushing/pulling that encounters
this.
Test Plan: Future patch tests caught this
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4956069
Signature: t1:4956069:1493263952:c3d59f3e48dda623fd8c67630cb4bcd474e63518
Summary:
This adds a remote tree datastore to the store, so if we don't have a tree on
local disk, we can reach out to the server for it.
This also makes a small change to fastmanifest to use get() instead of
getmissing() to test if the tree is available. getmissing() would indicate the
tree is missing if it wasn't local, while get() will attempt to fetch it from
the server.
Test Plan: Added a test
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4864645
Signature: t1:4864645:1492627823:7afff5db54c895507507e57ab7263f4dbf10b5f7
Upstream added __bool__ to manifestdict, and the test checking that
hybridmanifest matches the functions of manifestdict started failing. Let's add
it.
The check-code.py change matches the one in upstream to support the addition of
__bool__.
Summary:
Upstream converted all array.array usages to bytearray, and in doing so they
also changed mdiff.textdiff(array1, array2) to be
mdiff.textdiff(util.buffer(array1), util.buffer(array2)), which we did not do in
our internal implementation.
This fixes that.
Test Plan: Adds a test
Reviewers: #mercurial, quark, simonfar
Reviewed By: quark, simonfar
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4724350
Signature: t1:4724350:1489697253:8f63350bfc30b3079bf38410054b4763942a82f6
Summary:
Upstream has switched the type here for better python 3 compatibility. Let's
match their change.
Test Plan: Tests now pass
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4699371
Signature: t1:4699371:1489429276:9da9f64489ecaf94cd77599a0558c081cfe792c2
Summary:
Upstream has added a new match argument to manifest.diff() and removed the
existing manifest.matches() function, so we need to update our internal usage.
A separate diff will update treemanifest to support the new diff() api.
Test Plan:
Ran the tests, some still fail because of the upstream changes, but
future patches fix those.
Reviewers: #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D4677002
Summary:
Previously, when treemanifest would create packs of trees during pull, we
allowed trees to be delta'd against trees in other packs. This resulted in
smaller packs, but if the other pack disappeared for some reason (since it's a
cache), the chain broke.
This patch ensures that the first version of every tree added to a pack is a
full text.
This temporarily makes repacks worse, since the repacker doesn't know about
history to produce deltas when combining packs. The next patch adds history
awareness which improves the repack deltafication.
Test Plan:
Updated the tests, and inspected the new test results to ensure that
all packs only had deltas within the pack.
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4647348
Signature: t1:4647348:1488882214:e850622a853a534fc60caeef604c88c30740c60d
Summary:
Previously the treemanifest code itself would create the text deltas when
writing a tree out. This meant we couldn't make the delta decision based on
other data, like if the p1 commit was in the same pack file.
This patch removes treemanifest.write() and moves all calls over to
treemanifest.finalize() which gives the python/pack layer control over delta
choices. A future patch will use this to ensure tree packs always contain
complete delta chains.
Test Plan: All tests pass
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4645942
Signature: t1:4645942:1488880851:d0c8c902e7e849072a53344630a9184b6d8e1e7f
Summary:
Previously the treemanifest auto-tree-creation logic only produced data packs
containing the actual contents of the tree blobs. This lost history information
which is important for our ability to efficiently repack the data files.
This patch creates history packs during pull as well. A future patch will also
create history packs for the local tree blob store.
Test Plan: Updated the tests to cover this
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4638865
Signature: t1:4638865:1488449992:48b60961b50b90b6d0e75a64af1f36fb29944e7a
Summary: manifestctx's upstream now accept a manifestlog instead of a repo.
Test Plan: Ran the tests
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4667246
Signature: t1:4667246:1488911047:5340f4dc18183c68dfd47a0c5223b0f804e17806
Summary:
Core mercurial relies on readfast being fast for both iteration and containment
checks. Our previous implementation just returned the full manifest, which was
fast for containment (when a tree was available), but very slow for iteration.
The changegroup code iterates over the result of readfast in an attempt to
record linkrev information as revisions are applied. This resulted in very slow
bundle application in some cases since each full iteration of a treemanifest can
take several seconds.
This fix makes us use tree diffs when possible, which make that iteration very
fast again.
Test Plan:
I had a repro for a slow hg bundle command in our large repo. Before,
each full iteration over the readfast results took 11+ seconds, and now it takes
effectively 0 seconds. The resulting bundles were the same (technically the new
bundle was smaller, since the smarter change detection meant less data was added
to the bundle).
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters, mbolin
Differential Revision: https://phabricator.intern.facebook.com/D4619223
Signature: t1:4619223:1488162752:0a39d7488209cdf74af6ebb6ab27eb1b5873c76a
Summary:
As part of unifying our native store data structures into a single library,
let's move the treemanifest (including the python extension) into py-cstore.
Test Plan:
Built and ran the tests. Verified there was no ctreemanifest.so
dependency in the built cstore.so by using 'ldd cstore.so' on Linux and 'otools
-L cstore.so' on OSX.
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4602484
Signature: t1:4602484:1487842683:964cbb43b7cb20d0db699ef691fe7fcf6bccf2e8
Summary:
Previously we only wrapped manifestlog.__getitem__ because fastmanifest didn't
support the concept of directory manifests. Unfortunately manifestlog.get() is
used in some flat manifest cases too, like in changegroup.py where it just
passes '' as the directory but still uses .get().
This bug has been present since the manifestlog refactor, but only became
exposed now that we've improved our caching and hold on to manifest ctx's for
longer.
This fix is to just wrap manifestlog.get() too. Fixing this exposed some other
inconsistencies with our fastmanifest api.
Test Plan:
shelve now passes in a repo that was hitting an exception because of
this before. I'm still trying to think of how to test this in a test. I've not
been able to repro it in my own repo yet.
Reviewers: #mercurial, quark
Reviewed By: quark
Differential Revision: https://phabricator.intern.facebook.com/D4540677
Tasks: 15991119
Signature: t1:4540677:1486690231:07a95b402913d7dcc31633341e235ee1eb0163bc
Summary: Fastmanifest was not caching manifest contexts. Because of this, we were recreating a manifest dict on every read operation. Merges read the same manifest a lot and benefit heavily from this optimisation.
Test Plan:
```
$ ../../hg/tests/run-tests.py -j28
...................................................................................s..................s.......................................................
Skipped test-infinitepush-sql.t: missing getdb.sh
Skipped test-infinitepush-backup-sql.t: missing getdb.sh
# Ran 156 tests, 2 skipped, 0 warned, 0 failed.
```
Reviewers: #sourcecontrol, ttung, durham
Reviewed By: durham
Subscribers: ttung, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4457102
Signature: t1:4457102:1485449327:16e23c0bb1e7d61fb18c8a0c88a15598603749a8
Summary:
During pull, the reposetup function was updating the static factory ui with the
ui from the remote peer. This didn't have any of the local configs, so it
appeared as if treemanifest was disabled if it had been enabled in the repo
specific hgrc. So let's just not update the ui for remote repos.
This was causing pulls to always generate a full tree manifest instead of only
doing incremental ones.
Test Plan:
Ran 'hg pull' and verified the tree packs created in the hgcache was
small.
Reviewers: #mercurial, mitrandir
Reviewed By: mitrandir
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4456928
Signature: t1:4456928:1485359738:3333e1ad2c5cd68890b3eb5ff80a01ed5584df1e
Summary:
treemanifest requires fastmanifest, and fastmanifest.usetree requires
treemanifest. Let's make these dependencies explicit in the code and error out
if they are incorrect.
Test Plan: Added a test
Reviewers: #mercurial, wez
Reviewed By: wez
Subscribers: wez, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4417512
Signature: t1:4417512:1484347447:7e18340813fac0b298aa51a7cc2f89fc6953680f
Summary:
Treemanifest had a bug where the pack files it created were 400 instead of 444.
This meant people sharing a cache on the same machine couldn't access them. In
most of the remotefilelog code we had set opener.createmode before creating the
pack but we forgot to for treemanifest.
This patch moves the opener creation and createmode setting into the mutable
pack class so we can't screw this up again.
Test Plan:
Tests that wrote to the packs before, now failed and had to be
updated to chmod the packs before writing to them.
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4411580
Tasks: 15469140
Signature: t1:4411580:1484321293:9aa78254677548a6dc2270c58cee0ec6f57dd089
Summary:
Previously treemanifest and fastmanifest were largely unaware of each other. If
the fastmanifest was available, we'd use that. If not, we'd try tree. Then we'd
fall back to flat. When comparing two manifests, this could cause problems since
if one manifest was fast and one was tree, we'd have to fall all the way back to
flat manifests to compare them.
This patch adds the ability to build a treemanifest from a fastmanifest for
inmemory manifests that were copied from a commit that has both a fastmanifest
and a treemanifest. We do it by diff'ing the inmemory fastmanifest with the
commit fastmanifest, then applying that diff to the commit's treemanifest. Then
we can compare that tree with the other tree.
This is particularly useful when doing things like 'hg up master' after a pull.
The new master commit probably has a treemanifest, but doesn't yet have a
fastmanifest, while the commit the user is on probably has both. Now we can do
that hg up without having to parse a full manifest.
Once treemanifest has been enabled, every commit (both pulled and user created)
should have treemanifests. Therefore this case should never happen when
comparing two normal commits. So in theory, the only way to hit this case is
when Mercurial does a manifest.copy() on a manifest that has a fastmanifest
(like when reading a workingctx which must copy the commit manifest and apply
working copy changes), since the copy will choose to copy the fastmanifest and
leave the treemanifest behind. This patch addresses that case, so I think there
shouldn't be other cases.
Test Plan: Adds a test that covers this case.
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4402604
Signature: t1:4402604:1484216866:1c075f314026aaf608095b4aca37c9a4277ababa
Previously we attached the original node to the copy of the manifest we made.
This is almost always incorrect. In the main case, we make a copy of the p1
manifest during commit and then edit that manifest to form the new manifest. By
storing the node (and never invalidating it), there's the opportunity for the
code to accidentally load the old p1 manifest instead of the in memory one.
The fix is to not pass node to the copy. This affects the tests because some
tests relied on the copied manifest being able to access the original tree/cache
later. It just happened to work because we never modified the manifest before
performing those loads.
This is similar to the patch to matches() that made the match result not have
node either.
This patch also cleans up the fastdelta logic that attempted to read the
_treemanifest directly, instead of going through the normal read path. This is
what actually caught the bug, since it read the treemanifest from disk, while
the in memory cachedmanifest had changes.
Previously, when fastmanifest.usecache was set to false, we just had the cache
return False every time we checked if something was in the cache. Unfortunately,
this didn't prevent us from constructing new inmemory fastmanifestdicts, which
didn't play nicely with treemanifest.
Let's make usecache do the exact same thing as usetree, which means storing
'False' in the __cachedmanifest field, and using that to indicate that field is
completely disabled.
There were two openers in use in this code: opener and origself.opener. 'opener'
was only being created when it was the first manifest in the transaction, so the
line at the bottom that depended on 'opener' existing would fail if a second
manifest was created during the transaction (like during an amend). We just need
to fix the last line to use the correct origself.opener, and fix the 'opener'
name to be less accidentaly reusable.
The _treemanifest function was hard coded to return None if self.node was None,
even if self.__treemanifest was present. A future patch removes self.node in
many cases, so this ended up breaking since it could no longer return the
treemanifest.
Summary: This was added in core as well
Test Plan: tests are passing
Reviewers: #sourcecontrol, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4326595
Signature: t1:4326595:1481732429:af2ab66f73e5ef501b4d0480181715c17797b743
Summary:
This adds a few tests that ensure that flat manifests are not being read when
treemanifests are enabled. Any exception to this rule is documented in the test
and commented on.
The test caught a bug where we were not returning a simple treemanifest when the
node was the nullid. While the performance of an empty flat manifest is similar
to the performance of a empty treemanifest, the empty flat manifest may have
forced us to not use the treemanifest in operations that compare two manifests
(like diff).
Test Plan: Ran the test.
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4262975
Signature: t1:4262975:1480706157:4d4913bf032ebf44105a71a27fec5ee1a36b015b
Summary:
When making a commit to the flat manifest revlog, we need to produce a full text
delta. Previously we would have to convert our treemanifest into a text string
so we could produce the delta, which takes many seconds. With this patch, we now
take advantage of the fastdelta code path which can generate a delta given the
parent fulltext and the list of changes.
Test Plan:
Ran rebase with treemanifests and verified it was much faster because
it didn't try to produce a full text from the in memory treemanifest.
Reviewers: #mercurial, ikostia
Reviewed By: ikostia
Subscribers: ikostia, rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4249375
Signature: t1:4249375:1480509343:3a130e8730522e31bd7a08440121a1451046a32c
Summary:
A future patch will make treemanifests also use the fastdelta fast path. To
support that, let's refactor the fast path to separate the fastmanifest specific
part.
Test Plan: Ran the tests
Reviewers: #mercurial, ikostia
Reviewed By: ikostia
Subscribers: rmcelroy, ikostia, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4249369
Signature: t1:4249369:1480509029:b6d08a4f9ef6581a4a2c1bed7f7db8ff084fc35d
Summary:
A future patch will make treemanifests able to use fastdelta as well, so let's
move the function to be global so we can reuse it.
Test Plan: Ran the tests
Reviewers: #mercurial, ikostia
Reviewed By: ikostia
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4249364
Signature: t1:4249364:1480507778:a00bb7b6301ca2f716c542f37f7150f2f6231640
Summary:
Treemanifests are naturally slower to produce full texts than just reading the
text directly from the manifest. So let's change our hybrid manifest
implementation to prefer flat manifests over tree manifests for text(). Fast
manifests are still faster to produce full texts, so we leave those as the
highest preference. This speeds up commit time when using treemanifest since
we don't need to construct the text from the tree for producing the delta that
goes into the revlog.
Test Plan:
Ran rebase with tree manifests enabled and verified it was
significantly faster. Also manually timed the fast vs flat text logic to ensure
fast was actually faster for text() than flat.
Reviewers: #mercurial, ikostia
Reviewed By: ikostia
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4249354
Signature: t1:4249354:1480505430:f9e712a2db49c56680ce34cd3f68c85841393b39
Summary:
Previously, _converttohybridmanifest would always create a new hybrid manifest
with the same node as the original. This meant that some code paths would
attempt to use the treemanifest from the node, instead of the already prepared
matches result. This meant the output could contain all the values from the
original tree, instead of just the matches output.
This is actually a regression from 98ba34a5194c09. Prior to that, matches did
not reuse the node.
Test Plan: Manually inspected the results in the debugger during a rebase.
Reviewers: #mercurial, ikostia
Reviewed By: ikostia
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4247821
Signature: t1:4247821:1480499268:27f4a1b92ecf5d10009996b5b8f22bac02f3f38e
Summary:
Previously, when we wrote each tree entry into a pack file, it wasn't delta'd in
any way. This patch makes it store the delta against p1 in the pack file.
Testing in a large repo shows this reduces tree pack size by about 22x.
Test Plan:
Ran the tests. Did a pull in a large repo and saw the pack file was
22x smaller than before (and still usable).
Reviewers: #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D4202088
The main manifestctx implementation has a find function, so the hybrid one needs
it as well. I tried to add a test for this, but the only known case is in a
weird situation where there are tags, but no tag caches and some other criteria
I haven't figured out yet. So I gave up.
Summary:
hybridmanifest.__nonzero__() was explicitly trying to call __nonzero__() on the
underlying manifest object. This breaks when the underlying manifest is a
manifestdict. manifestdict does not implement __nonzero__ (it instead has a
__len__ method that gets used instead when evaluating the manifestdict as a
boolean).
Because of this issue I was getting a crash in a local script when calling
repo.commitctx() with a memctx object.
Test Plan: Confirmed that calling repo.commitctx() with a memctx no longer crashes.
Reviewers: #sourcecontrol, durham, quark
Reviewed By: quark
Subscribers: net-systems-diffs@, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4191858
Signature: t1:4191858:1479330743:7adb668c9529a8c8abaf9a8a1e64c8ee78b64d86
Summary:
Upstream has refactored the manifest class into several classes, so we need to
update treemanifest to work with the new structure. Notably, the factory add
function previously relied on the ability for the revlog class to create a new
manifestdict (via manifest.maniest.read()), since this isn't possible anymore,
we have to construct the hybridmanifest ourselves and provide an appropriate
loadflat function to get the flat manifest if necessary.
Test Plan: Ran the tests
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4180891
Signature: t1:4180891:1479285991:82bc546a1eb682d3cfd8b4724bda575410405d0f
Summary:
Upstream Mercurial has refactored the manifest to get rid of the manifest class.
This patch updates fastmanifest to work with the new class structure. In
particular, it removes the hacky wrapping of 3 different manifest construction
functions, with a single wrapping of manifestlog.get(), which makes the code
simpler and more robust.
Test Plan: Ran the tests
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4180874
Signature: t1:4180874:1479286125:bbd2a36aa86237d68036b9d7b0a580829219d869
Summary:
fastmanifest makes bundling slower because fastmanifest `readshallowfast` always returns full manifest.
That's a big problem for infinitepush.
Let's copy-paste readshallowfast implementation from upstream. It uses readshallowdelta() if possible.
Test Plan:
1) Run all the tests for fb-hgext
2) Run infinitepush with this extension enabled. Make it is fast
Reviewers: durham, simonfar, rmcelroy, quark
Reviewed By: quark
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4088360
Tasks: 13907166
Signature: t1:4088360:1477580931:746e4054380403abbc52d1922583021b81f31bb6
Summary: Tests are failing because readdelta() in fastmanifest doesn't work after an upstream change. Copy and fix the upstream code to work in this case.
Test Plan: Run the test on my devserver
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4074517
Tasks: 14070483
Signature: t1:4074517:1477412975:bdfb6e81e4ea3bccafa59a5cb0a6e71fcb1e7753
Summary:
This makes local commits get written to trees (as well as flat manifest) when a
commit happens where the parent commit has a tree manifest already. During a
transaction where multiple trees are written (like when rebasing multiple
nodes), we reuse the same pack file for all the trees produced by tieing into
the transaction abort and close hooks.
Test Plan:
Ran the tests. Ran hg commit with the extension enabled. A future
patch will add an integration test for the treemanifest extension.
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4055851
Signature: t1:4055851:1477059659:91b1c2f93ef986e910cea752ebf2466cb20ac921
Summary:
In a future diff we will start allowing local trees to be written as part of
transactions. Part of this involves making those written trees available to
readers in this process before the transaction is committed. Since all manifest
reads go through hybridmanifest, we can insert a cache at this layer (just like
fastmanifest already has a cache at this layer).
The future patch which adds new manifests will populate this cache, and clear
this cache when the transaction completes.
Test Plan:
Ran the tests. Also tested the actual flow with the future patch that
causes commit to write to the cache
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4055838
Signature: t1:4055838:1477059568:0bf1d79cc1ca6f9bcefb881bab35c61c13641f4e
Summary:
It's possible for treemanifests to be created that aren't backed by a flat
manifest (like if you did `foo = treemf.matches(...)`). In these cases, if we
ever need to diff it with a flat text, we need to be able to convert it to flat.
This patches adds the ability to convert it to a flat manifest via the text()
function.
Test Plan: Ran the tests
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4055656
Signature: t1:4055656:1477056921:7417c9ca06d61c5a283c6cf9693d83650e39f196
Summary:
This fixes some (but not all) of the fastmanifest errors. The remaining ones are
from an import * and one that will require slightly more refactoring to fix.
Test Plan: Ran the tests
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4055633
Signature: t1:4055633:1477050763:09e758b1bd72b18799cdabf510eb0b2332077833
Summary:
The fastmanifest code had a race condition where committing quickly in
a loop could cause a stack trace. It was caused by the code check m.incache()
and then using m._cachedmanifest(), and between those two lines a background
process could have cleared the entry fromm the cache. So now we just check if
_cachedmanifest() actually returns a value.
Test Plan:
With and without the fix:
```
cd ~/local && rm -rf temprepo ; hg init temprepo && cd temprepo && for j in `seq -f "%03g" 1 100`; do for i in `seq -f "%03g" 1 10`; do echo "line$j" >> file$i.txt; done; echo "Commit $j done"; hg ci -Am "Commit $j" ; done
```
It would stack trace reliably without the fix, and never with the fix.
Reviewers: ikostia, #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4004376
Tasks: 13828535
Signature: t1:4004376:1476266037:0ff33a43fd35b5c7309fbfbf546799d75cc03ebc
Summary:
Upstream 4718718ed358 requires a `readfast` method for manifestctx.
`hybridmanifestctx` does not have such method so let's add it.
Test Plan: Added a test.
Reviewers: durham, ttung, #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3955478
Signature: t1:3955478:1475582074:f80c9eb97bcd42846625727ef1da4bce3256c88c
Summary:
This fixes a crash that `hybridmanifestctx` does not have `readdelta`
after upstream's b404425704fa. `readdelta` should be fast for vanilla
manifest so just forward the call there.
Test Plan: Modify a file that `.` touched, run `hg amend` and it won't crash.
Reviewers: ttung, #sourcecontrol, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3948989
Signature: t1:3948989:1475198696:d8ae194500fc093c03cc115f4613a9b8c8bfbff2
Summary:
Upstream no longer has manifestctx derive from manifestdict; instead it uses a
read function. Let's update hybridmanifestctx to do the same.
Test Plan: Ran the fastmanifest tests
Reviewers: #sourcecontrol, ttung
Reviewed By: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3863656
Signature: t1:3863656:1473878502:6fb01f9f23fe64aafdc366a7ae98d510f3fb7927
Summary: @durham's changes to manifest.py landed, so we need to update where we plug into fastmanifest.
Test Plan: `PYTHONPATH=~/work/mercurial/facebook-hg-rpms/fb-hgext/:~/work/mercurial/facebook-hg-rpms/lz4revlog/:~/work/mercurial/facebook-hg-rpms/remotenames/ /opt/local/bin/python2.7 ~/work/mercurial/facebook-hg-rpms/hg-crew/tests/run-tests.py -j32 test-fastmanifest*.{py,t}`
Reviewers: rmcelroy, durham
Reviewed By: durham
Subscribers: mitrandir, durham, mjpieters, lcharignon
Differential Revision: https://phabricator.intern.facebook.com/D3838608
Signature: t1:3838608:1473439470:1a689f60e72d351f1cf12d9e2cf1d2da9ab96bd9
