Commit Graph

42510 Commits

Author SHA1 Message Date
Jun Wu
78dd086eb5 errorredirect: use absolute import
Summary: This fixes a test-check issue.

Reviewed By: DurhamG

Differential Revision: D6879862

fbshipit-source-id: c6caac6b1d3ed97bb6076d5fb9d0495a41a015c2
2018-04-13 21:51:02 -07:00
Jun Wu
b8f87be31d helpers-usechg: respect CHG environment variable
Summary: This allows CHG to override the chg program being used in tests.

Reviewed By: DurhamG

Differential Revision: D6879866

fbshipit-source-id: c6955609e2b6938d7e26d9b42aa0f6221396a4ad
2018-04-13 21:51:02 -07:00
Jun Wu
8e2207b7e6 run-tests: exit 80 if all tests are skipped
Summary:
This can be used by a wrapper script running a single test
to detect whether the test is skipped or not.

Reviewed By: DurhamG, phillco

Differential Revision: D6879864

fbshipit-source-id: 8bc535fccdbc6ad32eec870bd9e503b6f84f694b
2018-04-13 21:51:02 -07:00
Jun Wu
fbaa53a84a run-tests: create a symlink of hg
Summary:
run-tests.py creates the `python` symlink. Let's do the same for `hg`.

This means tests can just use `hg` and do not worry about it being a
wrong one (like, the system hg). And there is no need to create fragile
shell alias, or set `ui.remotecmd`.

The check to ensure `--with-hg` provides either `hg` or `hg.exe` was
removed since it's no longer necessary.

Reviewed By: DurhamG

Differential Revision: D6879879

fbshipit-source-id: 59255102ea3c4c646afc7900696403242bcfb4db
2018-04-13 21:51:01 -07:00
Jun Wu
ae4b932a4e run-tests: increase default timeout
Summary:
With the upcoming `buck test` change, the default timeout is
not enough because of the python wrapper overhead. Let's double
the limits.

Reviewed By: DurhamG

Differential Revision: D6879874

fbshipit-source-id: 19175be8c4f48dda1fc8917d75798d840d4082da
2018-04-13 21:51:01 -07:00
Jun Wu
30ca26055f run-tests: do not override --with-hg with -l
Summary:
When `--with-hg` and `-l` are both provided. `-l` will override
`--with-hg`, which is unfriendly. Let's do the reverse.

Reviewed By: DurhamG

Differential Revision: D6879863

fbshipit-source-id: 00ea9805431cc7afa6dd521cfb9059c898ed6d1f
2018-04-13 21:51:01 -07:00
Jun Wu
5ae900b05d import-checker: consider lz4 as a system module
Summary:
This fixes `test-check-module-imports.t` running under `buck test`
because `lz4` is not treated as a system module.

Reviewed By: DurhamG

Differential Revision: D6881307

fbshipit-source-id: 26a7e2d36c1ae6815064de15c6bdcb6e762c8ddf
2018-04-13 21:51:01 -07:00
Jun Wu
c03c09633f test-check: use --cwd to avoid issues
Summary:
`test-check-help.t` could run `hg help` and that command shouldn't be affected
by the current repo requirement. So let's add `--cwd /`.

`helpers-testrepo.sh` uses `-R path` and that only works if path is the repo
root. It's not guarnateed now. So let's use `--cwd` instead.

Reviewed By: DurhamG

Differential Revision: D6879867

fbshipit-source-id: 0b40ab71acf7eb215834ce323fc16156da9afcd3
2018-04-13 21:51:01 -07:00
Jun Wu
32aac9256d lazymanifest: do not read uninitialized memory
Summary:
Backported from https://phab.mercurial-scm.org/D1948.
Discovered by clang UBSAN.

Reviewed By: DurhamG, phillco

Differential Revision: D6879871

fbshipit-source-id: c068bdcf5b250c6586515960ef7ed9d0af3a2c59
2018-04-13 21:51:01 -07:00
Jun Wu
cc33c003c2 lz4: import lz4.block
Summary:
`lz4.block` needs to be imported explicitly before being able to
use `lz4.block.compress`.

We didn't notice this because we're using an old version of
`python-lz4`.

Reviewed By: DurhamG

Differential Revision: D6879877

fbshipit-source-id: 37e8fdc00386bef3733753f925ad308f42e5a740
2018-04-13 21:51:01 -07:00
Jun Wu
3a7081dc36 cleanup: remove pythonpath.py
Summary: This should be removed with D6865042.

Reviewed By: phillco

Differential Revision: D6879861

fbshipit-source-id: 960a9ab643f63d6d4a01dfb0ab7af77616cf9ea2
2018-04-13 21:51:01 -07:00
Kostia Balytskyi
84129009ee hg: log time spent waiting for the pager
Summary:
Let's collect how long we wait for a pager so that we can later subtract it
from the elapsed time.
Currently, it's impossible to compute `elapsed - pager - editor - stdio`, which
would include startup time.

Reviewed By: quark-zju

Differential Revision: D6875447

fbshipit-source-id: d055e4ba5bf1ab7ee9e567c99b6708321ab7705b
2018-04-13 21:51:01 -07:00
Durham Goode
01b34be972 hg: prefetch trees before producing changegroup
Summary:
When building a changegroup, sometimes we need to access the list of
files changed by each commit. To do so we need to inspect the manifest.
Previously this would end up downloading each tree one-by-one, producing a bunch
of pack files. With this patch we now do one bulk download at the very
beginning.

Reviewed By: quark-zju

Differential Revision: D6873076

fbshipit-source-id: b916c15efca0770129340f798d3e7b165da6aec9
2018-04-13 21:51:01 -07:00
Durham Goode
57cf49a88d hg: move prefetch onto repo object
Summary:
A future diff will add tree prefetching during changegroup creation. This
requires access to the tree prefetch function from the shallowbundle class.
let's move the prefetch functions onto the repo object in preparation for that.

Reviewed By: quark-zju

Differential Revision: D6873055

fbshipit-source-id: 18de0ee0f6ab566587509f9e23ebb2e5779ed1c8
2018-04-13 21:51:01 -07:00
Liubov Dmitrieva
ddaab68bd5 treedirstate: bugfix
Summary:
treedirstate: bugfix

Incorrect usage of write function that can cause corrupted files in some cases.

Write doen't guarantee all data will be written. Function write_all should be used instead.

https://doc.rust-lang.org/1.0.0/std/io/trait.Write.html#tymethod.write

See: It is not considered an error if the entire buffer could not be written to this writer.

Function returns number of bytes written.

Reviewed By: markbt

Differential Revision: D6883057

fbshipit-source-id: 4bc18cc9fa6e5db0ae8516d67dd6ba92c1facf21
2018-04-13 21:51:01 -07:00
Saurabh Singh
b3c9e0805e treemanifest: store ui once instead of using repo.ui every time
Summary:
setuptreestores caters to a lot of configurations through the `ui`
instance. It makes sense to store the `ui` instance once instead of referring
to it through `repo.ui` every time.

Reviewed By: DurhamG, quark-zju

Differential Revision: D6867843

fbshipit-source-id: 3ffca85332ec4ae3ec8a54bf0dacc7e7cf423b53
2018-04-13 21:51:00 -07:00
Jun Wu
1cde63d99c codemod: drop hacks changing PYTHONPATH in tests
Summary:
Now they are unnecessary since `run-tests.py` will set up `PYTONPATH`
correctly.

Differential Revision: D6865042

fbshipit-source-id: ca95314f725968e14349a9d916434aa832c596f9
2018-04-13 21:51:00 -07:00
Adam Simpkins
d61006b81f update hg rebase -i -s to bail out on unsupported src revs
Summary:
`hg rebase -i` only works if the source revision is ancestor of the working
directory parent.  This updates the code to fail with an error message in this
case instead of rebasing nothing and reporting success.

Note that attempting to change the revset calculation to use `src::` instead
of `src::.` doesn't help fix anything--the histedit code simply rejects the
histedit operation when it finds commits that are not ancestors of the working
directory.

Reviewed By: quark-zju

Differential Revision: D6853548

fbshipit-source-id: 14600f31a5f4ca43c89a81e7b522656b65a056dc
2018-04-13 21:51:00 -07:00
Kaley Huang
b787f74134 p4fastimport: fix badsymlink test
Summary: test-p4fastimport-import-badsymlink.t is failing the regex match P58977427, and this fixes it.

Differential Revision: D6836164

fbshipit-source-id: df43a9e5e30a2745a5dbb36bc6cc07ab81179525
2018-04-13 21:51:00 -07:00
Liubov Dmitrieva
b5b151319c backout: use only first line of commit message for backout default message
Summary: backout: use only first line of original commit message (title) for backout default message.

Reviewed By: phillco

Differential Revision: D6846109

fbshipit-source-id: 68efbff9db698599a12cef45de7ab4af96cf184f
2018-04-13 21:51:00 -07:00
Liubov Dmitrieva
85159c20b5 Improve UX for short hash infinitepush feature
Summary:
Improve UX for short hash infinitepush feature

I would like to provide some suggestions in case there are several with the same prefix,
so we can provide better user experience.

Reviewed By: markbt

Differential Revision: D6845233

fbshipit-source-id: 4868c1dc599f60e4ac495b71a4aed2d57c36ffa4
2018-04-13 21:51:00 -07:00
Mark Thomas
ad1f8c211c fsmonitor: add telemetry for watchman status
Summary:
Adds telemetry showing the status of watchman during an invocation of HG.

The new key, `fsmonitor_status` takes the value:
* `exception` if an exception occurred during the call to watchman.
* `unavailable` if watchman was unavailable.
* `fresh` if a fresh instance of watchman was encountered.
* `normal` if watchman was available normally.

Differential Revision: D6845070

fbshipit-source-id: 5ccf61a1009cdccbc0d1786b2533c3a7fc6ac2c9
2018-04-13 21:51:00 -07:00
Simon Farnsworth
c9914836f9 account for drawdag move in test-doctest.py
Summary:
D6827925 deletes drawdag.py from the tests directory; because the
tests in that revision were run on a repo with .pyc files left behind, the test
failure never happened. Move the test to the new location

Differential Revision: D6845181

fbshipit-source-id: 33c2836d81f66c9cc2a8e82a78b2d6835278a2ac
2018-04-13 21:51:00 -07:00
Jun Wu
e2a5493b04 basepack: workaround Python's mmap fd limit
Summary:
This is a resend of https://phab.mercurial-scm.org/D1430, without breaking
Windows.

I encountered "too many opened files" problem due to treemanifest packs on my
laptop. This patch seems to be the easiest solution without side effects. Other
choices are deleting files (seem like an non-ideal workaround), forcing a
repack (could be slow), and rewriting using Rust (could take too long).

The root cause is Python's `mmap` implementation has to keep a fd internally
to support `mmapobj.resize` API. We only need read-only operation on the
mmap object so the fd is unnecessary. Re-implement a minimal mmap interface
for this purpose.

Reviewed By: DurhamG

Differential Revision: D6835890

fbshipit-source-id: 74c429e957cb8677682604eb02fc38b5b8d13ef7
2018-04-13 21:51:00 -07:00
Jun Wu
e8883f6131 hg: build cstore and cfastmanifest with buck
Reviewed By: DurhamG

Differential Revision: D6828060

fbshipit-source-id: 8af66b61b6bb8d7774e45fd97d5192fedaa03d72
2018-04-13 21:51:00 -07:00
Jun Wu
f0a0b28223 test-infinitepush-bundlestore: stabilize the tests
Summary:
It seems chg might cause troubles like `--bundle-repo` becomes an unknown flag,
if run with other tests. So let's run the tests without chg.

Also avoid using subshells so `exit 80` (in `setup*` shell functions) would correctly
skip the test if `mysql-connector` is not installed.

Reviewed By: singhsrb

Differential Revision: D6841356

fbshipit-source-id: 72d70b8bcc1eac732a2fd89acd9865a8b23be074
2018-04-13 21:51:00 -07:00
Durham Goode
0e21c4e1ab hg: fix treeonly prepushrebase python hooks for treeonly pushes
Summary:
When pushing a treeonly commit to a tree+flat hybrid server, the server
needs to execute the hooks in treeonly mode so we never try to access the flat
manifests. Previously we did this for shell hooks by setting some environment
variables, but we didn't do it for python hooks that ran in process.

This diff makes the python hooks run against a bundle repo that is instantiated
in treeonly mode. No changes to any hooks are required, as the repo object they
are given is already in the correct mode.

Reviewed By: quark-zju

Differential Revision: D6840971

fbshipit-source-id: 9fcb97d972076911b35bccf3f79b60972bcafc33
2018-04-13 21:51:00 -07:00
Jun Wu
96a9afa2a5 drawdag: inline drawdag
Summary:
Since we have `debugbuilddag` as a builtin command, and `drawdag` is
pretty useful for writing clean test cases. Let's inline it. This would
simplify many tests since `drawdag=..." is no longer needed.

Reviewed By: DurhamG

Differential Revision: D6827925

fbshipit-source-id: b749f57fd5c55b2cb000dd2da2c2f9bc2386a519
2018-04-13 21:50:59 -07:00
Durham Goode
d4eed83607 hg: switch remotefilelog-local test to use regex
Summary:
The last attempt did not fix the flakyness. Let's just allow both
results for now.

Reviewed By: quark-zju

Differential Revision: D6839543

fbshipit-source-id: 0d66ee2bbb38aaed67ee2c890c024b3e8eb82a95
2018-04-13 21:50:59 -07:00
Durham Goode
1d3146a75e hg: fix check tests due to string literal issues
Summary:
A recent change added "from __future__ import unicode_literals" to a
few files, this changes the way strings are interpreted so we need to update
them to escape backslashes, otherwise they can be interpreted as unicode
characters and sometimes be invalid.

Reviewed By: quark-zju

Differential Revision: D6838181

fbshipit-source-id: 42fed1edafbb988342f482c3a63b5b235aba7390
2018-04-13 21:50:59 -07:00
Durham Goode
52f0c867dd hg: reduce test flakiness
Summary:
The remotefilelog-local test was notorious for being flaky. I was able
to reproduce the issue pretty consistently for a bit, and this change appears to
have made the problem go away. I can't promise it will work, but it seems to
have helped.

Reviewed By: quark-zju

Differential Revision: D6838177

fbshipit-source-id: d0ac0171fa2a93e28ed4afcabb5b15b5ee5e7d83
2018-04-13 21:50:59 -07:00
Durham Goode
1f577c8417 hg: remove version check from crecord
Summary:
crecord locked out the amend functionality to older versions of
Mercurial. Since it's packaged in core now, let's always enable it.

Reviewed By: quark-zju

Differential Revision: D6837724

fbshipit-source-id: 2a0dae2e03346734f7d8298e5a073f776849005e
2018-04-13 21:50:59 -07:00
Jun Wu
fee9564d3b setup: unbreak setup.py
Summary:
This "fixes" setup.py error:
```
python setup.py  \
  build_py -c -d . \
  build_clib  \
  build_ext  -i \
  build_rust_ext -i \
  build_hgexe  -i \
  build_mo
Traceback (most recent call last):
  File "setup.py", line 969, in <module>
    depends=common_depends),
  File "/usr/lib64/python2.7/distutils/extension.py", line 106, in __init__
    assert type(name) is StringType, "'name' must be a string"
AssertionError: 'name' must be a string
```

While the most correct fix would be possibly prepending prefixes
to strings in `setup.py`, it feels so much churn for so little
gain. So let's just disable unicode literal for now.

Reviewed By: phillco

Differential Revision: D6836945

fbshipit-source-id: 7d44969661c26b681f9d2e12d2e10026c4673a1c
2018-04-13 21:50:59 -07:00
Liubov Dmitrieva
b00894a227 treedirstate: rename of file and class
Summary: treedirstate: rename of file and class

Reviewed By: zertosh

Differential Revision: D6716409

fbshipit-source-id: 83d40103a13771773176c333f1287011d3a42f40
2018-04-13 21:50:59 -07:00
Liubov Dmitrieva
9d7e085b49 Implement support for short hashes
Summary:
[infinitepush] Implement support for short hashes

Also, fix test that was broken and lint errors

Reviewed By: StanislavGlebik

Differential Revision: D6814967

fbshipit-source-id: fef1ae79ef85ebf84aa88cd99367946256049192
2018-04-13 21:50:59 -07:00
Jordi Llull Chavarria
e731a435e4 Add missing column to infinitepush test table
Summary: The time column was missing in the bookmark test table

Reviewed By: StanislavGlebik

Differential Revision: D6819122

fbshipit-source-id: c0b4ede651ba5c598f0384580942cfb727b5fe61
2018-04-13 21:50:59 -07:00
Kostia Balytskyi
e45048e95b hg: use py2exe for packaging built mercurial
Summary:
py2exe allows us to reduce startup time by 0.4s to 1s in some cases.

Depends on D6798345

Reviewed By: DurhamG

Differential Revision: D6798433

fbshipit-source-id: 566cd01f33c7bce32f577bd7b662642a4316c92b
2018-04-13 21:50:59 -07:00
Andrew Regner
537d945c15 Do not probe the home directory because autofs
Summary:
I got tired of my prompt taking a long time to return to me sometimes after a command ended, and started to investigate, and found this:

```
aregner  4069767  0.0  0.0 115052  5076 pts/17   Ss+  12:32   0:00  \_ -bash
aregner   298922  0.0  0.0 115052  3540 pts/17   S+   14:40   0:00  |   \_ -bash
aregner   298923  0.0  0.0 115052  3668 pts/17   S+   14:40   0:00  |       \_ -bash
aregner   273995  0.0  0.0 115052  4824 pts/9    Ss+  14:32   0:00  \_ -bash
aregner   274061  0.0  0.0 115052  3820 pts/9    S+   14:32   0:00      \_ -bash
[devbig308 (77bac96|remote/fbcode/warm):~/2nd-fbcode] strace -p 298923
strace: Process 298923 attached
stat("/home/.hg",
```

Sometimes it will hang ong this for 5+ minutes.  I'm assuming this has something to do with autofs and NFS home dirs or something or another, but it doesn't matter because when it gets to this point it should just stop looking for a source control directory.

Reviewed By: ryanmce

Differential Revision: D6790747

fbshipit-source-id: d32c74476046aa6b62bc26725eea6b60a4eaa9f7
2018-04-13 21:50:59 -07:00
Phil Cohen
1e0dbc1386 contbuild: fix test-shbang.t
Summary:
`build_rpm.py` has a /usr/bin/env shbang so it shouldn't be a failure here.

Although, it's had that shebang since 1/23/18.

Reviewed By: quark-zju

Differential Revision: D6825992

fbshipit-source-id: bf7ff3d61338384d8c6e26a93c71f4361687f1f2
2018-04-13 21:50:58 -07:00
Durham Goode
210bd25b94 hg: fix check code errors
Summary: These snuck in.

Reviewed By: phillco

Differential Revision: D6821043

fbshipit-source-id: bd4d83e2460bebdb2061dfbf84bed0139d1b72e1
2018-04-13 21:50:58 -07:00
Phil Cohen
6e301ce9ab setup: prepend version with 4.4.2
Reviewed By: DurhamG

Differential Revision: D6820709

fbshipit-source-id: 27a9e4a09b2b3cf6107a7dbff981eba0e1cc1d51
2018-04-13 21:50:58 -07:00
Liubov Dmitrieva
68ac2c6418 Improve backout default message
Summary: [backout] Improve backout default message

Reviewed By: phillco

Differential Revision: D6818834

fbshipit-source-id: 3dc2465f64e37ff4bc96eff512ae991cf2a2f96d
2018-04-13 21:50:58 -07:00
Phil Cohen
87bb8c5500 rebase: fix a bug where rebases would wrongly be restarted
Summary:
If `rebase.experimental.inmemory` was enabled, but you ended up merging on-disk (e.g. you rebased the working copy parent), and you got merge conflicts, and didn't immediately solve them, hg would discard any work you did and restart the rebase unnecessarily.

The bug here is basically that `rbsrt.inmemory` field can be set to False later, because we make further calculations once we define the `rebaseset`. In that case `InterventionRequired` will get raised if the user didn't solve conflicts, but we'll catch it, thinking something went wrong "in-memory", and abort/rerun the rebase. I was too hasty in adding the "catch all exceptions and retry" block.

I solved this two ways -- first, move the rbsrt definition here and explicitly check the state in the Exception catch. Second, add an explicit catch for InterventionRequired and explain why with a comment.

Further refactoring of the rebase code will help here, I think, so we can consolidate all of this logic.

Reviewed By: DurhamG

Differential Revision: D6817683

fbshipit-source-id: eb8baba86ec0deb59a1e0b1cf9ffb02f056ecd92
2018-04-13 21:50:58 -07:00
Mark Thomas
086eb81b0d treedirstate: remove unused import
Summary: Remove the unused `os` import that I somehow missed in D6818952.

Reviewed By: ryanmce

Differential Revision: D6819544

fbshipit-source-id: 072be0a956a6ac2bbe96f72c6fc3c79907d22c9a
2018-04-13 21:50:58 -07:00
Simon Farnsworth
30e993a160 template: add functions to truncate blocks and lines
Summary:
smartlog, among other places, would benefit from being able to
strictly limit the size of text output to the terminal. Provide two functions:
one truncates long lines in a filled block, the other limits the number of
lines of output. Both can optionally add a truncation marker.

Reviewed By: markbt

Differential Revision: D6819153

fbshipit-source-id: d95b654f4a6fef49ad4840f26ef74a18c370bdd8
2018-04-13 21:50:58 -07:00
Mark Thomas
11d62f6961 absorb: add -d option to include descriptions in absorb output
Summary:
Add a new `-d` option to `hg absorb`, which includes in the `-p` output the first 50
characters of the first line of the commit description for any commit that will
be absorbed into.

The width is configurable through `absorb.maxdescwidth`.

Reviewed By: ryanmce

Differential Revision: D6819341

fbshipit-source-id: 1a007aa5c15c047563b602559b338e5548bf1989
2018-04-13 21:50:58 -07:00
Mark Thomas
4b7dab149a hg: improve perf interactions with treedirstate
Summary:
There are a couple of ways that dirstate and fsmonitor interact with
treedirstate that are sub-optimal.

When iterating over all files in the dirstate map, use the iterator of keys
(`__iter__` rather than `iteritems`) to hit the treedirstate fastpath that
doesn't build the dirstate tuples.

Use `__getitem__` and catch the `KeyError` rather than `__contains__` followed
by `__getitem__`, as the latter involves two look-ups in the tree.

Skip the indirect call to `hastrackedfile` and `hasremovedfile` as these add
noticable overhead when called many times.

Also fix up the `test-check-code` failure for treedirstate.

Reviewed By: ryanmce

Differential Revision: D6818952

fbshipit-source-id: d49907c282a1b97c66a3d24257bdf54800d7da70
2018-04-13 21:50:58 -07:00
Jun Wu
d942f5a88e hg: basic support for building hg using buck
Summary:
Adds some basic building blocks to build hg using buck.

Header files are cleaned up, so they are relative to the project root.

Some minor changes to C code are made to remove clang build
warnings.

Rust dependencies, fb-hgext C/Python dependencies (ex. cstore,
mysql-connector), and 3rd-party dependencies like python-lz4
are not built yet. But the built hg binary should be able to run
most tests just fine.

Reviewed By: wez

Differential Revision: D6814686

fbshipit-source-id: 59eefd5a3ad86db2ad1c821ed824c9f1878c93e4
2018-04-13 21:50:58 -07:00
Durham Goode
2c5a125383 hg: fix check-code test
Summary: N/A

Reviewed By: quark-zju

Differential Revision: D6814855

fbshipit-source-id: bb98a13974205bdaef90d39db1271a5ed5b9c485
2018-04-13 21:50:58 -07:00
Wojciech Lis
9e3e4e85df Fixing LFS upload
Summary:
This fixes lfs upload by avoiding use of workers. The perfect fix
would be to make handler work with the workers and multithreading, but that
is much more complex. The downlaoding has been working fine for 1.5 months now
without any reported issues so leaving that to use workers.

Reviewed By: quark-zju

Differential Revision: D6804951

fbshipit-source-id: d5ce92b35c5cbcd64174e47df7a19507c98ed64a
2018-04-13 21:50:58 -07:00