Summary:
This diff adds a config option to tweak deltabase in changegroup. It has 3
options:
- Always null - always use "null" as delta base, effectively make
everything full text
- No external - delta bases cannot be a revision outside the changegroup
- Default - the current behavior: delta bases can be anything that client
thinks the server should have.
This gives Mononoke more time to bake delta related logic, as we can
choose "always null" first, then incrementally increase the complexity.
Reviewed By: phillco
Differential Revision: D7158585
fbshipit-source-id: 5f6d9a78d1108093e8d08b9f296568f4f7e7471b
Summary:
Verify had some logic that checked the length of the changelog and
manifest to decide if either existed. This allowed for simplifying certain error
messages (like not reporting all the broken changelog manifest pointers if the
manifest was simply gone, and just reporting the manifest was gone).
Unfortunately, in future changelog and manifest implementations len() will be an
expensive function, so let's just get rid of that optimization.
This fixes hg verify in a treeonly repository.
Reviewed By: quark-zju
Differential Revision: D7127168
fbshipit-source-id: 8ddc3dfe3c3c913efd4b7af5fc9715a3e48b60a1
Summary:
For large updates, the dirstate update can take a while. Let's show
progress so the user understands what is happening and how long to wait.
Reviewed By: quark-zju
Differential Revision: D6903133
fbshipit-source-id: f7f6c3c14e1d3221a383da4a6e311aa12a8d3a98
Summary:
Let's switch to xdiff for its better diff quality and performance!
The test changes demonstrate xdiff's better diff quality.
Reviewed By: ryanmce
Differential Revision: D7135206
fbshipit-source-id: 1775df6fc0f763df074b4f52779835d6ef0f3a4e
Summary:
Enable the indent heuristic feature, since it provides nice visual
improvements for a wide range of cases. See the added test, and [1].
The only downside is it can slow things down. In a crafted case, this could
make `--indent-heuristic` several times slower than `--no-indent-heuristic`.
```
open('a', 'w').write(" \n" * 1000000)
open('b', 'w').write(" \n" * 1000001)
```
```
git diff --no-indent-heuristic a b 0.21s user 0.03s system 100% cpu 0.239 total
git diff --indent-heuristic a b 0.77s user 0.02s system 99% cpu 0.785 total
```
[1]: 433860f3d0
Reviewed By: ryanmce
Differential Revision: D7135452
fbshipit-source-id: 019b7e89225f288bba0a1d042591b13b5419ad0e
Summary:
Implement a `mercurial.cext.xdiff` module that exposes the xdiff algorithm.
`xdiff.blocks` should be a drop-in replacement for `bdiff.blocks`.
In theory we can change the pure C version of `bdiff.c` directly. However
that means we lose bdiff entirely. It seems more flexible to have both at
the same time so they can be easily switched via Python code. Hence the
Python module approach.
Reviewed By: ryanmce
Differential Revision: D7135205
fbshipit-source-id: 48cd3b5be7fd5ef41b64eab6c76a5c8a6ce99e05
Summary:
verify.skipmanifests was added to let us skip the parts of verify that
were expensive in a lazy-manifest world. We also need to skip .hgsubstate
verification since it requires the manifests.
Reviewed By: singhsrb
Differential Revision: D7127353
fbshipit-source-id: 377fffb8556f7578da3e51c3da53f97554fb5d74
Summary: This change minimally addresses the issue that `debugrebuilddirstate` can crash if the dirstate file is very corrupt.
Reviewed By: markbt
Differential Revision: D7028370
fbshipit-source-id: 72fc7a2900a8bc1bb5f062454530b4fc4c806f09
Summary:
This logic exists for the following case: if becuase of some circumstances (user error, NFS/Samba weirdness) a symlink file becomes corrupted and we check this file in while preserving the 'l' flag in the manifest, the revision becomes un-checoutable on Posix filesystems (according to the original explanations anyway).
This option basically adds an ability to take this risk. I am planning to enable this option for `ovrsource`, since it already has very few symlinks so the probability of corrupting something is pretty low.
Differential Revision: D7127333
fbshipit-source-id: f85d9f3aef676afca641280c9b7f0ecfb87b9fab
Summary: dispatch.lazyaliasentry didn't work with slices i.e. lazyasliasentry[:2] failed. This diff fixes it
Reviewed By: farnz
Differential Revision: D7109779
fbshipit-source-id: c704cd44fea0944ae4be68df36d32df98b7fc09b
Summary:
We were already hacking the ui object onto the manifestlog via the
treemanifest extension, but it didn't cover all cases. Let's just put it on the
manifestlog via the constructor and get rid of the hack. An upcoming change
required this
Reviewed By: phillco
Differential Revision: D7083032
fbshipit-source-id: 4c577cb80193a9c4799853d75a71c26719348e8c
Summary:
As part of moving the manifest mutable pack logic to be used optionally
without transactions, let's move the lifetime logic
out to the repo.transaction() function instead of being part of the primary manifest
write logic.
Differential Revision: D7083035
fbshipit-source-id: 9947d78db61a41896cc8bdfaaa20504ebb03a125
Summary:
Previously the mutable packs were kept on the transaction and had a
lifetime that corresponded with it. In a future patch we want to enable mutable
packs that span for longer than the lifetime of the transaction, so let's move
the mutable pack maintenance on the manifestlog. For now the lifetime is still
maintained by the transaction, but a future diff will change that as well (and
will get rid of _writeclientmanifest entirely).
Differential Revision: D7083034
fbshipit-source-id: 3735eadfc18e5dd1015bfb82dbf5b9e9e6965cdf
Summary:
D6867913 did a bunch of refactoring which can be extended for the write
manifest wrapper. This commit achieves the same.
Reviewed By: singhsrb
Differential Revision: D6867921
fbshipit-source-id: c86cf1d54ecd3503127d91acf7a114d4d99f50ff
Summary:
A future patch will need the ability to convert a linknode to a linkrev and vice
versa without having access to the changelog (when converting a flat manifest
into a treemanifest we need to get the linknode from the flat manifest, and all
we're given is a linkrev). Let's stick the changelog on the manifestlog. Since
they are generally created together and have the same lifetime, this seems safe.
Reviewed By: ryanmce
Differential Revision: D7034091
fbshipit-source-id: df6915f894c6ef975ffd214c8e229f1085141fbc
Summary:
The root issue is described in the new ``workingflags()`` function -- ``workingfilectx.flags()`` ought to read from the working copy, but instead reads from the working copy's p1.
Rather than force a big rewrite to adapt all the old code (which might be a waste if we end up rewriting the context classes), let's add a new function with the behavior we want, and callers can detect for it and call it.
simplemerge is the only callsite that needs this behavior, AFAICT.
Previous attempt: D6694292
Reviewed By: singhsrb
Differential Revision: D7083239
fbshipit-source-id: 6f30b877958d40106203e088fe724ab491b541b9
Summary:
Add a new `--only-files-in-revs` option to `hg diff`. This filters the files
that are considered down to the set of files that are touched in either of the
two revisions.
Reviewed By: ryanmce
Differential Revision: D7070102
fbshipit-source-id: b9140cd3a212af9554ef5b76785210c1dbba8ee0
Summary:
Exposed by UBSAN:
```lang=bash
scm/hg/mercurial/cext/pathencode.c:61:40: runtime error: left shift of 1 by 31 places cannot be represented in type 'int'
#0 0x7f60c372b2e7 in inset scm/hg/mercurial/cext/pathencode.c:61
#1 0x7f60c373022c in _encode scm/hg/mercurial/cext/pathencode.c:424
#2 0x7f60c372a605 in basicencode scm/hg/mercurial/cext/pathencode.c:480
#3 0x7f60c3729f90 in pathencode scm/hg/mercurial/cext/pathencode.c:744
```
Use unsinged 1 in the shift operation to fix the UB
Reviewed By: wez
Differential Revision: D7073320
fbshipit-source-id: 8e714c41e94474c4c62b3b6730df6fe99f9c0391
Summary:
Turns out this function can read from the working copy because it calls `match`. `experimental.merge.checkpathconflicts` does what this function does without that problem and without a perf hit. So, nuke it!
Also add a test.
Previous version: D7070540
Reviewed By: singhsrb
Differential Revision: D7072247
fbshipit-source-id: a217ffd575daf8271411a77c422e1dcdd387dfbb
Summary:
Let's try to be consistent and print relative path names during merge. Note
that it changes test output for mq and subrepos, but we don't use these
features often, so it should be fine.
Differential Revision: D7067050
fbshipit-source-id: 3aba66e67657a1b1b30fcf239c03abd605eb5d9a
Summary:
"hg" only ignored ENOENT errors when including files (so it didn't abort when an included file didn't exist). But it aborted on every other error (e.g EPERM).
This diff makes it ignore any IOError and just print a warning.
Reviewed By: quark-zju
Differential Revision: D7029320
fbshipit-source-id: b1b8137cd575fc89fb5967e8d18fe82ab84a8f85
Summary:
Update the treemanifest extension to also add a 'gettreepack()' function to
localpeer, and add the 'gettreepack' capability to localpeer if the repository
supports acting as a treemanifest server.
Also update the _addmanifestgroup() wrapper to only forbid it from being called
in a treemanifest server repository. When pulling from a local treemanifest
server repository this function may need to be invoked in the client
repository.
Reviewed By: DurhamG
Differential Revision: D6980931
fbshipit-source-id: 61519c7deeaa0347650323a0002b89597e91d9cc
Summary:
Previously some of this code had undefined behavior because it was performing
signed shifts that could overflow into the sign bit, as the result could not be
represented in a signed `int`. This caused failures when built with UBSAN.
This updates the code to consistently use the built-in `_byteswap_*` functions
when compiling with Microsoft Visual Studio and the `__builtin_bswap*` when
compiling with gcc or clang.
Reviewed By: quark-zju
Differential Revision: D7008197
fbshipit-source-id: adbe22032f05a35468195922ee00ccc3ff5efff9
Summary:
Currently, commits adding LFS files are not verified server-side since the LFS
extension explicitly skips hash checking during unbundle, to avoid overhead
downloading/reading LFS objects.
We'd like those commit hashes to be verified. However, the existing verify
command does not scale with a huge repo. So let's add `verify -r REV` support
to be able to incrementally verify a repo.
Reviewed By: quark-zju
Differential Revision: D6947227
fbshipit-source-id: 6ffda3a814822942630339fc7de62c3fcb284fda
Summary:
hgweb's file view must determine if the displayed file revision is
the head of the filelog graph. As remotefilelog does not implement file
revision, we have to check if headrevs is implemented.
Reviewed By: DurhamG
Differential Revision: D6954143
fbshipit-source-id: 0657e58110112537dc5baadf743c657d4ecf372a
Summary:
inhibit has a very hacky way to disable obsmarker cycle check.
Move that to core to clean up the code.
Reviewed By: ryanmce
Differential Revision: D6956217
fbshipit-source-id: 18370721ec80e74008232e09049a057ee080e7d5
Summary:
This is a backport of e49c39ffeac2 from upstream.
It is more efficient to pass down one large string to the output streams
than many small ones. For a ``hg diff`` test case, it improves time from
2m26s to 2m8s.
Reviewed By: quark-zju
Differential Revision: D6951390
fbshipit-source-id: 5f6faacb4c77678a6abca863bf931de4bce2dfde
Summary:
This backports upstream rev a9d07bd8f758. This continues a patch series
that Augie thought was well worth the speed up it provided.
Reviewed By: quark-zju
Differential Revision: D6951332
fbshipit-source-id: a35fce1560a3e2b182faf5871ceb9068a52d697b
Summary:
This check is useful and detects real errors (ex. fbconduit). Unfortunately
`arc lint` will run it with both py2 and py3 so a lot of py2 builtins will
still be warned.
I didn't find a clean way to disable py3 check. So this diff tries to fix them.
For `xrange`, the change was done by a script:
```
import sys
import redbaron
headertypes = {'comment', 'endl', 'from_import', 'import', 'string',
'assignment', 'atomtrailers'}
xrangefix = '''try:
xrange(0)
except NameError:
xrange = range
'''
def isxrange(x):
try:
return x[0].value == 'xrange'
except Exception:
return False
def main(argv):
for i, path in enumerate(argv):
print('(%d/%d) scanning %s' % (i + 1, len(argv), path))
content = open(path).read()
try:
red = redbaron.RedBaron(content)
except Exception:
print(' warning: failed to parse')
continue
hasxrange = red.find('atomtrailersnode', value=isxrange)
hasxrangefix = 'xrange = range' in content
if hasxrangefix or not hasxrange:
print(' no need to change')
continue
# find a place to insert the compatibility statement
changed = False
for node in red:
if node.type in headertypes:
continue
# node.insert_before is an easier API, but it has bugs changing
# other "finally" and "except" positions. So do the insert
# manually.
# # node.insert_before(xrangefix)
line = node.absolute_bounding_box.top_left.line - 1
lines = content.splitlines(1)
content = ''.join(lines[:line]) + xrangefix + ''.join(lines[line:])
changed = True
break
if changed:
# "content" is faster than "red.dumps()"
open(path, 'w').write(content)
print(' updated')
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
```
For other py2 builtins that do not have a py3 equivalent, some `# noqa`
were added as a workaround for now.
Reviewed By: DurhamG
Differential Revision: D6934535
fbshipit-source-id: 546b62830af144bc8b46788d2e0fd00496838939
Summary:
Solves issues below:
```
hgext/backups.py:18:1: F811 redefinition of unused 'registrar' from line 17
hgext/catnotate.py:1:1: F811 redefinition of unused 'util' from line 1
hgext/remotenames.py:57:5: F811 redefinition of unused 'registrar' from line 34
hgsubversion/setup.py:103:5: F401 'mercurial' imported but unused
hgsubversion/setup.py:109:5: F401 'hgsubversion.svnwrap.svn_swig_wrapper' imported but unused
i18n/polib.py:1281:29: F841 local variable 'exc' is assigned to but never used (Python 2)
i18n/polib.py:1427:13: F841 local variable 'typ' is assigned to but never used
i18n/polib.py:28:1: F401 'sys' imported but unused
mercurial/manifest.py:411:5: F811 redefinition of unused '_lazymanifest' from line 168
mercurial/posix.py:419:5: F811 redefinition of unused 'normcasefallback' from line 362
mercurial/posix.py:425:5: F811 redefinition of unused 'checkexec' from line 167
mercurial/posix.py:431:5: F811 redefinition of unused 'checklink' from line 234
mercurial/pycompat.py:29:5: F401 'http.cookiejar as cookielib' imported but unused
mercurial/pycompat.py:30:5: F401 'http.client as httplib' imported but unused
mercurial/pycompat.py:31:5: F401 'pickle' imported but unused
mercurial/pycompat.py:33:5: F401 'socketserver' imported but unused
mercurial/pycompat.py:34:5: F401 'xmlrpc.client as xmlrpclib' imported but unused
mercurial/statprof.py:573:36: F812 list comprehension redefines 'parent' from line 562 (Python 2)
mercurial/util.py:1076:5: F811 redefinition of unused 'nogc' from line 1051
mercurial/util.py:3221:5: F811 redefinition of unused 'dirs' from line 3184
tests/silenttestrunner.py:24:5: F811 redefinition of unused 'main' from line 6
tests/test-context.py:90:1: F811 redefinition of unused 'scmutil' from line 4
tests/test-fb-hgext-cstore-treemanifest.py:146:5: F811 redefinition of unused 'testDeeplyNested' from line 134
tests/test-fb-hgext-extutil.py:46:5: F811 redefinition of unused 'testbgcommandfailure' from line 37
tests/test_hgsubversion_util.py:47:1: F811 redefinition of unused 'svnwrap' from line 31 (Python 2)
tests/test_hgsubversion_util.py:49:1: F811 redefinition of unused 'svnwrap' from line 47 (Python 2)
```
Reviewed By: ryanmce
Differential Revision: D6934533
fbshipit-source-id: 8b51851a76fec88bb59107ed05a901d42c7326f8
Summary: D6937883 turns out to be ineffective. Let's fix it in another way.
Reviewed By: DurhamG
Differential Revision: D6940006
fbshipit-source-id: 082d0ddcf464d7056426bde2feab52a7717eb057
Summary:
It seems the non-standard xar layout could conflict with the foreign
import check somehow.
Reviewed By: singhsrb
Differential Revision: D6937883
fbshipit-source-id: b7c4dfc6836f6f1c8a693775dc42c16c537dc8e6
Summary:
Some actions can take a long time, and not all of them have progress bars.
When this happen the user is left with an empty prompt and doesn't know that
Mercurial is actually working.
This adds a context manager which can be used to give long-running processes a
spinner-style progress bar, indicating that something is happening in the
background.
Reviewed By: farnz
Differential Revision: D6937079
fbshipit-source-id: 396d7b11157b4f13e0c12dac1e2403d9269938ab
Summary:
This allows us to specify `-p 0 --port-file X` to get the port assigned in
an atomic way. So there won't be "server failed to start" caused by race
conditions in tests.
Reviewed By: DurhamG
Differential Revision: D6925397
fbshipit-source-id: 5bbb61b7eb2695f0a673afdb0730d2a61827f8b3
Summary:
Check to see if the module documentation is None before trying to call
`splitlines()` on it.
Reviewed By: singhsrb
Differential Revision: D6919043
fbshipit-source-id: 93c458cde9643a3f1b2d6fe6eb56fa312ae2a192
Summary:
Previosuly, we would miss anything logged in the `ui.atexit`-registered
handler, since the actual logging would happen before those handlers were
called.
Reviewed By: quark-zju
Differential Revision: D6912321
fbshipit-source-id: 77600b7ae535b4da56fef1f92b51998de8e304e2
Summary:
With certain setup, the file history could be incorrect. That makes
`adjustlinknode` much slower since it has to scan the full history.
It is extremely slow if there is a commit with a massive renames.
This is undesirable since `hg log FILE` would give a wrong result.
But it does make the repo usable. Automation that does not care
about `hg log FILE` correctness can probably enable this relatively
safely.
This patch changes both shallow and full repos.
Reviewed By: DurhamG
Differential Revision: D6912051
fbshipit-source-id: 23d6f6c8dd91d4f72b43bc560cf26686bd6c4b47
Summary:
Calling revdiff with non-zero flags is a sign of a hard-to-debug
error. Raise ProgrammingError in this case.
The change is straightforward. Apply it to both shallow and full
repos.
Reviewed By: DurhamG
Differential Revision: D6910080
fbshipit-source-id: cbcf1a444de90e104867cc9f1525629b7edda851
Summary:
This happens when the client with LFS revisions applies a bundle
with a delta base pointing to an LFS revision stored in the repo.
Reviewed By: DurhamG
Differential Revision: D6906210
fbshipit-source-id: 8b47f8304f8ef5ae4b02d7239b680f70106a4d83
Summary:
This is similar to what we have done for changegroups, but for non-changegroup
(addrawrevision) case. This is needed to make sure the delta application code
path can assume deltas are always against vanilla (non-LFS) rawtext so the next
fix becomes possible.
Reviewed By: DurhamG
Differential Revision: D6906202
fbshipit-source-id: a7d62dfed4206d45b42299f1dabf013620ae52b3
Summary:
There is no way to distinguish whether a delta base is LFS or non-LFS.
If the delta is against LFS rawtext, and the client trying to apply it has the
base revision stored as fulltext, the delta (aka. bundle) will fail to apply.
This patch forbids using delta on LFS revisions.
Note: this does not solve the problem entirely. Since the problem could also be
a client with base file revision being LFS tries to apply a non-LFS delta
(bundle).
Reviewed By: DurhamG
Differential Revision: D6878326
fbshipit-source-id: 9c3951e4673b8de61aae73a51e1bfff422f38d0f
