Summary:
Some of our internal hg builds include the Eden extension now. This was
causing the test-help.t test to fail since it included help output for the Eden
extension, but the test code did not expect this.
We unfortunately cannot update the test output to always expect help output for
Eden since the Eden extension is only included in buck-based builds.
Reviewed By: quark-zju, farnz
Differential Revision: D7063937
fbshipit-source-id: e503ddc6889e546b5333a8d9e3555097d689e24c
Summary:
Update run-tests.py to cache the results of hghave checks. Some of the
linematch code can end up calling _hghave() hundreds of times in a row with the
same check.
This is particularly likely to happen on test failures, where a line with an
hghave check does not match the expected output. In this case the same hghave
check will be re-run on each subsequent line of test output.
Reviewed By: quark-zju, farnz
Differential Revision: D7077677
fbshipit-source-id: 768a11df78924b75cde4c8b48c801b79194f7969
Summary:
Add a new `--only-files-in-revs` option to `hg diff`. This filters the files
that are considered down to the set of files that are touched in either of the
two revisions.
Reviewed By: ryanmce
Differential Revision: D7070102
fbshipit-source-id: b9140cd3a212af9554ef5b76785210c1dbba8ee0
Summary:
metaedit when metaediting a commit with obsolete & visible children did
autorebase all of them causing a divergence (which broke the restack)
Reviewed By: ryanmce
Differential Revision: D7067121
fbshipit-source-id: 679bc7acecde1cd648adcdddc9c8ac3ecc101208
Summary:
When compiling with optimization enabled gcc complains that result is not
immediately initialized when created inside a constexpr function.
Differential Revision: D7065250
fbshipit-source-id: 5a829e775ce482c8a5ec9d1cfe279005ff11bb7e
Summary: Including bookmarks in the crdump output
Reviewed By: sfbahr
Differential Revision: D7063784
fbshipit-source-id: 08d38d8a73a8448b3a184bfc3511cdaec01771fb
Summary:
We don't really support files larger than 2G with files larger than
4G remotefilelog crashes badly.
Reviewed By: markbt
Differential Revision: D7066855
fbshipit-source-id: 40cdebbe703a7b3f13ce84174bf6f96565e8c3b7
Summary:
If the user specifies a revision using `-r`, along with
`--since-last-arc-diff`, then show the differences for that revision, rather
than the working copy parent.
Differential Revision: D7068359
fbshipit-source-id: feffadeb45eed4e1d8bb9d8fc7040932ada31736
Summary:
The fbhistedit `exec` verb attempts to create an obsmarker if the working copy
parent is changed as a result of the command execution.
This isn't always valid to do: the command could have created a new commit
entirely, and the old working copy parent could be public, and thus not
obsoletable.
It's also not necessary. If the command that ran during the exec did anything
that obsoleted commits (e.g. amend), then it should have created its own
obsmarker.
Differential Revision: D7066784
fbshipit-source-id: 326396828a06936c88154cfec6705f7834527a5f
Summary:
Inserting a commit after a public commit causes histedit to fail
when it tries to write out the obsmarkers.
Reviewed By: ryanmce
Differential Revision: D6820504
fbshipit-source-id: 1f73bde65eb87485b0712d6a365b846fd00ad48c
Summary:
Let's try to be consistent and print relative path names during merge. Note
that it changes test output for mq and subrepos, but we don't use these
features often, so it should be fine.
Differential Revision: D7067050
fbshipit-source-id: 3aba66e67657a1b1b30fcf239c03abd605eb5d9a
Summary:
`hg debugpathcomplete` is another example where the current implementation
iterates the whole dirstate looking for information that's in one directory.
With treedirstate, this can be made faster by using the tree structure.
Differential Revision: D7024784
fbshipit-source-id: 3c0b403887e85157b7eb2752abce5a0805e6ed1c
Summary:
"hg" only ignored ENOENT errors when including files (so it didn't abort when an included file didn't exist). But it aborted on every other error (e.g EPERM).
This diff makes it ignore any IOError and just print a warning.
Reviewed By: quark-zju
Differential Revision: D7029320
fbshipit-source-id: b1b8137cd575fc89fb5967e8d18fe82ab84a8f85
Summary:
Running `hg log --traceback --template '{file_copies}' -r XXXX` on a file with long history is slow for 2 reasons
- p4 fast importer preserves full history for deleted and re-added files
- p4 fast importer records the wrong parent of a file
This diff tries to fix these two issues.
In mercurial, if a file is added, deleted, and then added back, it should start a new file history when the file is added again.
For example,
commits commit1 commit2 commit3
actions add a.txt delete a.txt add a.txt
timeline ------------X------------X------------------------X------------
`hg debugindex a.txt` at commit3 shows a.txt as a new file without previous history
rev offset length delta linkrev nodeid p1 p2
0 0 3 -1 0 b789fdd96dc2 000000000000 000000000000
However, this is different in p4. `p4 filelog test.txt` gives you
//depot/Software/Apps/Main/Native/.castle/test.txt
... #3 change 523261 add on 2018/01/23 by zhihuih@devbig415 (text) 'test:add-again-same-file'
... #2 change 523254 delete on 2018/01/23 by zhihuih@devbig415 (text) 'testfile:delete'
... #1 change 523253 add on 2018/01/23 by zhihuih@devbig415 (text) 'testfile:add'
Currently, p4 fast importer preserves history the same way as p4, and this causes slowness (even timeout) in hg when it runs `hg log --traceback --template '{file_copies}' -r XXXX` on a revision that contains files with long history in p4 (mostly contributed by automation). To mitigate this, we want the p4 fast importer to behave the same way as hg, and starts a new history for a file that's added again.
Currently, p4 fast importer takes the tip of a filelog and treats that as the parent of the newly written entry diffusion/FBS/browse/master/fbcode/scm/hg/hgext/p4fastimport/importer.py;19ad9b05f50e3ff0265cdc7b4b45174dcf820343$468-469. This can be wrong when there are revisions from branches.
For example, if I edit file a in master in CL1, 2, 4, and I branch at CL3, and edit the file in branch in CL5, the current importer implementation will take filenode at CL4 as the parent of CL3
(CL1,2,3,4,5 corresponds to rev0,1,3,2,4)
{F120393661}
However, the correct behavior is to take filenode at CL2 as the parent of CL3
(CL1,2,3,4,5 corresponds to rev0,1,3,2,4)
{F120393662}
(This is also the example I use in `test-fb-hgext-p4fastimport-import-branch-filelogorder.t`, so if the description here looks confusing, please refer to the test)
Reviewed By: dsp
Differential Revision: D6962019
fbshipit-source-id: 24de76ae009e0d6f976d247087fe4702c99e0f82
Summary:
Update the treemanifest extension to also add a 'gettreepack()' function to
localpeer, and add the 'gettreepack' capability to localpeer if the repository
supports acting as a treemanifest server.
Also update the _addmanifestgroup() wrapper to only forbid it from being called
in a treemanifest server repository. When pulling from a local treemanifest
server repository this function may need to be invoked in the client
repository.
Reviewed By: DurhamG
Differential Revision: D6980931
fbshipit-source-id: 61519c7deeaa0347650323a0002b89597e91d9cc
Summary:
- Add support for RocksDB engine (developed as a drop in replacement for innodb) to hgsql to allow new xdb.hgsql.1-10 shards to host hg repos
- Prefer MySQL test DBs in same region
- Run all hgsql unit tests also for RocksDB engine
- Allow for nested ifs to make that possible (downside if you switch off rockdb tests, innodb tests are run twice)
Reviewed By: quark-zju
Differential Revision: D7014064
fbshipit-source-id: 073c36176aa7eaf74252ef33c3f47da594920b28
Summary: Add some unit tests for the functions in bitmanipulation.h
Reviewed By: quark-zju
Differential Revision: D7018862
fbshipit-source-id: 2ffbb0c6776f702ae1e5f9a19bd6646d1d576af0
Summary: Fixes T25823569. We need to check for mergestate last since some commands leave that file in addition to their own.
Reviewed By: DurhamG
Differential Revision: D7013846
fbshipit-source-id: 42730bbee81b4c3b3aa2280a48d57da703fab5dd
Summary:
Per discussion with @[100000771202578:kulshrax] [1], the new behavior is enabled everywhere, and is also vastly preferable. Let's nuke the old code, as well as the config declaration and all places that manually enabled it.
[1]
> also, it looks like the pre-restack logic is still in that function, and the modern restack code path is still gated by a config option
> we should probably remove that
Reviewed By: kulshrax
Differential Revision: D7031365
fbshipit-source-id: dcc620c292bac03fdc0aa144546d4628bb673046
Summary:
The treemanifest extension now uses a pattern of
treemod._cansendtrees() to detect if it should send trees. pushrebase and
infinitepush were using a different pattern where they were sending trees if
they existed (in infinitepush's case it was ignoring the sendtrees config
entirely). Let's make these consistent.
The new behavior requires that trees exist for the commits we're trying to send.
In order for this to work, we'll need the logic that automatically converts a
flat manifest into a treemanifest so that we're never missing a tree we want to
send.
Reviewed By: markbt
Differential Revision: D7004982
fbshipit-source-id: 7dd6788173a0393d7a129123d5d62256d64b8a64
Summary:
I will be enabling this config on its own more broadly soon. Let's add
a test just to verify it works without treeonly=True being set (which is what
most other tests already test).
Reviewed By: quark-zju
Differential Revision: D7004366
fbshipit-source-id: 5743fbfd23da7327d1779f8555c42a8845d60271
Summary:
There are cases when a local repo communicates to another local repo
where one of the repos has remotenames enabled and the other does not that lead
to crashes. We fix these cases by checking repo objects for _remotenames before
accessing that member variables.
Differential Revision: D7023300
fbshipit-source-id: e4df06671bfb8291f50c01b53750ccaa3f9df968
Summary:
Cases from sandcastle make it clear that there are circumstances where
a one local repo talks to another local repo, one with remotenames enabled and
the other without remotenames enabled, that crashes can happen.
Let's add a test case to find some of these so we can prove they are fixed.
Differential Revision: D7023299
fbshipit-source-id: ddc8a701ac7573bcc15f737df1c5e29ef3581201
Summary:
Previously, if the user had an svn or git push target set up, we would
bail from the normal push --to process even if the users were pushing to a
different location. This is silly. Instead, let's only bail if the user is
actually pushing to an svn or git destination repository.
This bug was discovered while testing the updated svn-to-hg sync scripts.
Differential Revision: D7023173
fbshipit-source-id: bec5e069bc043c2f73fb334bdaeb18a6fad93eef
Summary:
Today, there are cases where this function will be called but bookmarks
will not have been populated. We will eliminate one of these paths next,
but for safety, let's make sure we abort gracefully rather than with a
stacktrace by referencing something that does not exist.
Differential Revision: D7023172
fbshipit-source-id: 8e3b2afe9fe3b9d74d42aa5b947229418ec8c3ad
Summary:
When hg addremove needs to remove a file, remotefilelog tries to create a filenode in order to prefetch the removed files from the server. If the file is not in the parent context manifest, this throws an exception.
To solve this we have to first check if the file exists or not in the parent manifest. If the file does not exist in the manifest then we don't need to prefetch it, and addremove will behave like forget rather than remove.
Reviewed By: markbt
Differential Revision: D7009649
fbshipit-source-id: 0570bc00db546a455b9c2e4628740e24ca819dd6
Summary:
Upon running `p4fastimport` tests locally I found out this test being deleted was not working.
It turns out I was using a newer version of p4d (2017.2) and the clowny behavior
around symlinks that it tests for was fixed in p4d 2017.1
I found out we use 2017.1 in our perforce servers:
diffusion/O/browse/opsfiles/branches/PROD/chef/cookbooks/other/fb_perforce/recipes/perforce_server.rb;1001845$125,125-126,333-334
Therefore, this test is not currently accurate or needed
Differential Revision: D7015278
fbshipit-source-id: 6e0e62371781a06bf194a624f4f557bddd3ccda4
Summary:
Within buck test envirionment there is no guarnatee that `pyflakes` binary is
in `PATH`. This patch builds pyflakes as a test dependency so buck test will
run `test-check-pyflakes.t` instead of skipping it.
Reviewed By: markbt
Differential Revision: D7008100
fbshipit-source-id: eca6f0174dbc9c12d45eca2f17f70a9805f13094
Summary:
Currently, commits adding LFS files are not verified server-side since the LFS
extension explicitly skips hash checking during unbundle, to avoid overhead
downloading/reading LFS objects.
We'd like those commit hashes to be verified. However, the existing verify
command does not scale with a huge repo. So let's add `verify -r REV` support
to be able to incrementally verify a repo.
Reviewed By: quark-zju
Differential Revision: D6947227
fbshipit-source-id: 6ffda3a814822942630339fc7de62c3fcb284fda
Summary:
Previously we would only do ondemand downloading of tree blobs when
accessing data contents. When accessing just history we would just fail if the
data wasn't available locally. This adds the remote store to the history union
store so we can get history remotely.
Reviewed By: singhsrb
Differential Revision: D7003434
fbshipit-source-id: 839f8e84be35779ccb146d13ce3e1d6d1e7f46bd
Summary:
This can be convenient when the size of LFS downloads between r1 and r2 is needed.
NB: this does not pay attention to sparse profiles and acts as if the checkout was full. We probably need to know how to do both.
Reviewed By: quark-zju
Differential Revision: D6938259
fbshipit-source-id: 52ab88be83339472f2eccafc746a191ff26c16c7
Summary:
remotefilelog prefetch was broken in treeonly mode since it referred to
the manifest revlog to get the parents. Let's switch to the more modern
manifestctx way of accessing parent information.
Reviewed By: quark-zju
Differential Revision: D6995267
fbshipit-source-id: e0c11fca0f2156be3f936a6e437e7a4d3dffe75b
Summary:
quark-zju removed mangling of sys.path in D6974954. The `import sys` is now no
longer needed.
Differential Revision: D6998711
fbshipit-source-id: 1d8298ce8d5077e3ae7ae3c88f2a2b7adf9712d3
Summary:
It was used to patch Python's `SocketServer`. Since we no longer use
`SocketServer` in command server's code path. The extension can be removed.
Reviewed By: ryanmce
Differential Revision: D6974976
fbshipit-source-id: 52353b55d812e494756833cefba896b6e64eaf9e
Summary:
Make `library.sh` read `$HGTEST_GETDB_PATH` for `getdb.sh` location. Unify
`getdb.sh` used by infinitepush and hgsql into one.
Reviewed By: ryanmce
Differential Revision: D6974635
fbshipit-source-id: 55bcebfbdaca7f43fb0cfca0d3e089e76cf9f24e
Summary:
Mangling `sys.path` in `.py` tests could break tests running with `buck` opt
build. Because the bundled extensions won't be used, but files in the working
copy will. It's okay-ish for pure Python extensions. But native extensions will
have problems.
Reviewed By: ryanmce
Differential Revision: D6974954
fbshipit-source-id: 8a7f3d86e5c2fbf16cfcc971743d99585fe61b99
Summary:
anydbm uses bsddb by default. bsddb is slower than gdbm and
is not available in buck build. Therefore use gdbm instead.
Differential Revision: D6994367
fbshipit-source-id: 948b991df31436d02f478d7a95f9924e1f8e0ed5
Summary:
As part of producing only one pack file per transaction, we need to
change the mutabledatapack store to allow reads of written-but-not-finalized
data. This will allow us to add the mutabledatapack to the union store, and then
keep it alive for the duration of the transaction.
Reviewed By: quark-zju
Differential Revision: D6944348
fbshipit-source-id: 1e721bd8e07335a9c1f9c6b7595a765ec018c007
Summary:
Let's create a new portability header, which can be used on both Windows and
Posix.
Reviewed By: quark-zju
Differential Revision: D6970928
fbshipit-source-id: a3970c50260f52bfc0a9420a4ff11d93ace304b0
Summary:
We have seen several cases that 00changelog.i got filled with 0s at the end
after a hard reboot. Improve fixcorrupt tool to be able to detect and fix
that. Previously it will not treat node=nullid as corrupted, and it would
also trust linkrev=0, which will truncate the entire changelog.
Reviewed By: ryanmce
Differential Revision: D6952921
fbshipit-source-id: b77e85f23c304ebde3740d6ed70effa1bc792928
Summary:
`test-hgsql-verify.t` occassionally fails, however the error output is lost to
an invocation of grep.
Instead, capture the output of the call to `hg sqlverify`, and if it doesn't
contain the string we're looking for, print the whole output.
Reviewed By: DurhamG
Differential Revision: D6965336
fbshipit-source-id: 72c47bbafcdc2b3ba32dc307670f2d8b0ee09d16
Summary:
inhibit has a very hacky way to disable obsmarker cycle check.
Move that to core to clean up the code.
Reviewed By: ryanmce
Differential Revision: D6956217
fbshipit-source-id: 18370721ec80e74008232e09049a057ee080e7d5
Summary:
This check is useful and detects real errors (ex. fbconduit). Unfortunately
`arc lint` will run it with both py2 and py3 so a lot of py2 builtins will
still be warned.
I didn't find a clean way to disable py3 check. So this diff tries to fix them.
For `xrange`, the change was done by a script:
```
import sys
import redbaron
headertypes = {'comment', 'endl', 'from_import', 'import', 'string',
'assignment', 'atomtrailers'}
xrangefix = '''try:
xrange(0)
except NameError:
xrange = range
'''
def isxrange(x):
try:
return x[0].value == 'xrange'
except Exception:
return False
def main(argv):
for i, path in enumerate(argv):
print('(%d/%d) scanning %s' % (i + 1, len(argv), path))
content = open(path).read()
try:
red = redbaron.RedBaron(content)
except Exception:
print(' warning: failed to parse')
continue
hasxrange = red.find('atomtrailersnode', value=isxrange)
hasxrangefix = 'xrange = range' in content
if hasxrangefix or not hasxrange:
print(' no need to change')
continue
# find a place to insert the compatibility statement
changed = False
for node in red:
if node.type in headertypes:
continue
# node.insert_before is an easier API, but it has bugs changing
# other "finally" and "except" positions. So do the insert
# manually.
# # node.insert_before(xrangefix)
line = node.absolute_bounding_box.top_left.line - 1
lines = content.splitlines(1)
content = ''.join(lines[:line]) + xrangefix + ''.join(lines[line:])
changed = True
break
if changed:
# "content" is faster than "red.dumps()"
open(path, 'w').write(content)
print(' updated')
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
```
For other py2 builtins that do not have a py3 equivalent, some `# noqa`
were added as a workaround for now.
Reviewed By: DurhamG
Differential Revision: D6934535
fbshipit-source-id: 546b62830af144bc8b46788d2e0fd00496838939
Summary:
Solves issues below:
```
hgext/backups.py:18:1: F811 redefinition of unused 'registrar' from line 17
hgext/catnotate.py:1:1: F811 redefinition of unused 'util' from line 1
hgext/remotenames.py:57:5: F811 redefinition of unused 'registrar' from line 34
hgsubversion/setup.py:103:5: F401 'mercurial' imported but unused
hgsubversion/setup.py:109:5: F401 'hgsubversion.svnwrap.svn_swig_wrapper' imported but unused
i18n/polib.py:1281:29: F841 local variable 'exc' is assigned to but never used (Python 2)
i18n/polib.py:1427:13: F841 local variable 'typ' is assigned to but never used
i18n/polib.py:28:1: F401 'sys' imported but unused
mercurial/manifest.py:411:5: F811 redefinition of unused '_lazymanifest' from line 168
mercurial/posix.py:419:5: F811 redefinition of unused 'normcasefallback' from line 362
mercurial/posix.py:425:5: F811 redefinition of unused 'checkexec' from line 167
mercurial/posix.py:431:5: F811 redefinition of unused 'checklink' from line 234
mercurial/pycompat.py:29:5: F401 'http.cookiejar as cookielib' imported but unused
mercurial/pycompat.py:30:5: F401 'http.client as httplib' imported but unused
mercurial/pycompat.py:31:5: F401 'pickle' imported but unused
mercurial/pycompat.py:33:5: F401 'socketserver' imported but unused
mercurial/pycompat.py:34:5: F401 'xmlrpc.client as xmlrpclib' imported but unused
mercurial/statprof.py:573:36: F812 list comprehension redefines 'parent' from line 562 (Python 2)
mercurial/util.py:1076:5: F811 redefinition of unused 'nogc' from line 1051
mercurial/util.py:3221:5: F811 redefinition of unused 'dirs' from line 3184
tests/silenttestrunner.py:24:5: F811 redefinition of unused 'main' from line 6
tests/test-context.py:90:1: F811 redefinition of unused 'scmutil' from line 4
tests/test-fb-hgext-cstore-treemanifest.py:146:5: F811 redefinition of unused 'testDeeplyNested' from line 134
tests/test-fb-hgext-extutil.py:46:5: F811 redefinition of unused 'testbgcommandfailure' from line 37
tests/test_hgsubversion_util.py:47:1: F811 redefinition of unused 'svnwrap' from line 31 (Python 2)
tests/test_hgsubversion_util.py:49:1: F811 redefinition of unused 'svnwrap' from line 47 (Python 2)
```
Reviewed By: ryanmce
Differential Revision: D6934533
fbshipit-source-id: 8b51851a76fec88bb59107ed05a901d42c7326f8
Summary:
When running with a Python runtime with a slightly different zlib module,
some `zlib.compress` outputs are different. Some tests are testing the
length, or the content of `zlib.compress` output, directly or indirectly.
That's causing issues.
This patch adds a `common-zlib` hghave test so it can be used to gate tests
checking zlib output. Some lengths are also changed to glob patterns to be
compatible.
Reviewed By: ryanmce
Differential Revision: D6937735
fbshipit-source-id: 2328a39d7f2022f16d51f61b6178568b26dfe2fb
