Commit Graph

47079 Commits

Author SHA1 Message Date
Jun Wu
78c06706ca shlex: add an expandfunc argument
Summary: This allows customization about how to expand an argument with different quotes.

Reviewed By: xavierd

Differential Revision: D17675350

fbshipit-source-id: 1dc4b2e2b7ea0296dceda0d8654c2278a602e4f8
2019-09-30 16:53:24 -07:00
Jun Wu
b118e96ab2 testutil/dott: vendor Python 3's shlex library
Summary:
We need to get "whether an argument is single-quoted or not" information to
decide whether to run extra logic (ex. expanding globs) on it. `shlex` does not
really have public interface for that and it's hard to `wrapfunction` patch it.
Therefore vendor the script so we can customize it.

Some Python 2 compatibility work was done:
- use `cStringIO` instead of `io`
- mark the file as utf-8
- remove `quote`, which uses an incompatible `re.ASCII`

Reviewed By: xavierd

Differential Revision: D17675349

fbshipit-source-id: eb99a9b69ca15f770c410bd3845052cb0c6f1b17
2019-09-30 16:53:24 -07:00
Xavier Deguillard
f48990a312 tests: fix missing depth initialization in generateworkingcopystates
Summary: The variable was used but sometimes not initialized.

Reviewed By: quark-zju

Differential Revision: D17667256

fbshipit-source-id: 9efa3b160cb23205baaa614827e5990475b956c9
2019-09-30 11:23:55 -07:00
Jun Wu
e4e4cbb639 testutil/dott: avoid crash expanding empty argument
Summary:
`arg` can be empty in this context. Avoid crashing with IndexError.

Also fix test-sparse-issues-t.py so it quotes the pattern correctly.

Reviewed By: xavierd

Differential Revision: D17669371

fbshipit-source-id: ea9e4e7c09a88308a650442c22e133684805738e
2019-09-30 11:14:59 -07:00
Jun Wu
98795c9257 test-import: fix on Windows
Summary: Avoid using `rm -rf` but just create new repos aggressively.

Reviewed By: singhsrb

Differential Revision: D17669063

fbshipit-source-id: f5ea142b08f16dcbdcf99a937fba803e1d8d9958
2019-09-30 10:56:16 -07:00
Zeyi (Rice) Fan
784f741312 hg: make configparser buildable with cmake
Reviewed By: simpkins

Differential Revision: D17505870

fbshipit-source-id: 974ed48424b46a0828133b9fb05da86129c3e21f
2019-09-30 10:41:04 -07:00
Jun Wu
60b58132d1 phases: point out manual edits are no-ops if narrow-heads is enabled
Summary: This makes it clear that the new behavior is different.

Reviewed By: sfilipco

Differential Revision: D17584605

fbshipit-source-id: 59573dfa14b4eb7a4a9c5bce6ae7340f408035da
2019-09-27 17:24:00 -07:00
Jun Wu
640a873e9c narrow-heads: add bi-directional migration
Summary:
When migrating down from narrow-heads, we need to restore phase roots.  Use a
repo requirement to track the migration state, and perform migrations on
demand.

Reviewed By: sfilipco

Differential Revision: D17584604

fbshipit-source-id: 36c04b0385dd94481f5340a3ecd8eeef3506c765
2019-09-27 17:24:00 -07:00
Xavier Deguillard
646314794b test-bisect-t.py: fix it
Reviewed By: quark-zju

Differential Revision: D17620102

fbshipit-source-id: b73398846b38ce13c34d4df5d7c5779c1d8641ad
2019-09-27 17:24:00 -07:00
Xavier Deguillard
f0e98e6104 tests: auto translate "test-bisect.t"
Summary:
This is done by running:

  ./translate "test-bisect.t"

Reviewed By: quark-zju

Differential Revision: D17620101

fbshipit-source-id: 461299cb6b74eac7a4febc2fa495ea65e65eea8e
2019-09-27 17:24:00 -07:00
Saurabh Singh
16da1518e5 perforce: remove p4fastimport
Summary:
I think we moved the last perforce based repository into `fbsource`.
From the Scuba logs, this doesn't seem to be used any more:
https://fburl.com/scuba/lc6vgy3v. We can revive the extension if required for a
future Perforce repository import.

Reviewed By: quark-zju

Differential Revision: D17630314

fbshipit-source-id: e3ca1a97974aff5fc262b8c724b70d4548280325
2019-09-27 16:31:34 -07:00
Andres Suarez
c333ddd733 Enable text linter for toml files
Reviewed By: mzlee

Differential Revision: D17638063

fbshipit-source-id: 9ee13ab9ac2f2202d33028c34922234d51d5dd08
2019-09-27 16:21:47 -07:00
Jun Wu
d9a65157b7 dispatch: do not start ipdb for common errors
Summary: Move ipdb logic so it wouldn't be triggered for common errors like error.Abort.

Reviewed By: xavierd

Differential Revision: D17628968

fbshipit-source-id: fae390d56203d8362a10a75acaa67901279656cf
2019-09-27 15:00:20 -07:00
Stefan Filip
e96a1efb2d tests: migrate test-revert to dott
Summary:
The test is mostly automatic conversion and easy fixes.
No refactoring to simplify code, on the contrary, more
direct printing.

Reviewed By: quark-zju

Differential Revision: D17630869

fbshipit-source-id: 45e57de5fd965d0cc913c2612a6f52cb3c8bd977
2019-09-27 13:44:59 -07:00
Stefan Filip
45784815c1 dott: update shobj to expand filename patterns
Summary:
Filename expansion is probably used a lot in our tests. I think that our
migration path is going to be simplified if we do filename expansion as
part of shobj. The current implementation works for the test that I was
migrating, test-revert.

Reviewed By: quark-zju

Differential Revision: D17630872

fbshipit-source-id: b8312e3d1fee9d331a661849a4cc5ef06ed5b091
2019-09-27 13:44:59 -07:00
Stefan Filip
9454158308 dott: update chmod to support -x
Summary: Following a similar implementation to `chmod +x`.

Reviewed By: quark-zju

Differential Revision: D17630871

fbshipit-source-id: 115fe8fd0cca214f7f739bec66301c9116edb250
2019-09-27 13:44:59 -07:00
Stefan Filip
e16d63bb2d tests: update generateworkingcopystates to work as a module
Summary: I would like to call into this script directly from dott tests.

Reviewed By: quark-zju

Differential Revision: D17630870

fbshipit-source-id: 169ab3f1706d77cb1cdc1461b2a6fce2bc77a877
2019-09-27 13:44:58 -07:00
Stefan Filip
40e989ef9a tests: rename generate-working-copy-states
Summary:
Renaming it to `generateworkingcopystates`. This is a valid module name.
The goal is to convert this script to a module. While we are converting
tests it needs to be accessible in both script form and module.

Reviewed By: singhsrb

Differential Revision: D17611370

fbshipit-source-id: 0b2d0de71f0da7cd6efd7874c505d9aa77f7b296
2019-09-27 13:44:58 -07:00
Stefan Filip
92f92d5138 manifest: add test verifying materialization in finalize
Summary:
This test checks that a directory without modifications between tree
and parent is not going to be materialized.

Reviewed By: quark-zju

Differential Revision: D17540173

fbshipit-source-id: 465f1e0410c42a55665bcd6903d75266c61d5e80
2019-09-27 13:07:21 -07:00
Jun Wu
4b5455424a util: fix hgcmd on buck build
Summary:
`util.hgcmd()` can report an incorrect hg path in the buck build case, since
the buck target is a wrapper script.

Fix it by setting the executable path explicitly.

Reviewed By: simpkins

Differential Revision: D17632684

fbshipit-source-id: d6c208fa1981180f6248bf7b57cc57031f53946e
2019-09-27 11:57:47 -07:00
Arun Kulshreshtha
55d07eeb94 manifest: remove dfs diff
Summary: Remove the DFS diff implementation and replace it with the BFS implementation.

Reviewed By: xavierd

Differential Revision: D17618818

fbshipit-source-id: d486642caae924f866a200d3c82fa5a4cb7d5286
2019-09-27 11:07:23 -07:00
Arun Kulshreshtha
e6f577bf01 treemanifest: enable bfs diff by default
Summary: In order to support efficient data fetching, eventually BFS diff will need to be the default diff algorithm. As an initial step towards this, enable `treemanifest.bfsdiff` by default to validate that the BFS diff algorithm works as a drop-in replacement for the existing diff algorithm. Once we have confidence in this, we can land the next diff in this stack wherein we remove DFS diff entirely.

Reviewed By: xavierd

Differential Revision: D17618819

fbshipit-source-id: 444cb05874913e4eb267a24bd0596e6dde2e099e
2019-09-27 11:07:23 -07:00
Arun Kulshreshtha
323aaa9dd1 manifest: copy dfs diff tests into bfs diff module
Summary: To ensure feature parity between BFS diff and DFS diff, copy the DFS tests into the BFS module and ensure they pass.

Reviewed By: xavierd

Differential Revision: D17618820

fbshipit-source-id: b516abbfa4e231fdc383293d94d8965333f2ab99
2019-09-27 11:07:23 -07:00
Saurabh Singh
7aab6858f0 test-hgsubversion-globalrevs: remove the test
Summary:
This test is currently failing and its not worth fixing since we only
needed this during the `WWW` Subversion deprecation which has already happened.

Reviewed By: quark-zju

Differential Revision: D17629893

fbshipit-source-id: 60f56b84a3cdcd8ddc56d46a155b1f14f2c00646
2019-09-27 09:49:50 -07:00
Jun Wu
7f3b4d93e9 chgserver: deal with fdopen EINVAL case
Summary:
The use of `fdopen` was to change the buffer size. It's not fatal if buffer
size cannot be changed. This fixes a crash in `nohup hg root`.

Reviewed By: aditya7fb

Differential Revision: D17628883

fbshipit-source-id: 58f75e8e99fbe4cea8da2e3b48a092ee1331db6f
2019-09-27 09:45:09 -07:00
Carolyn Busch
6cb28d027c Workingcopy python binding
Summary: Python binding for working copy rust library. Walker is initialized with the root of the repo and the python matcher, iteratively returns the matching files in the working copy. Modified Walker and PythonMatcher to allow Walker to have Send trait.

Reviewed By: xavierd

Differential Revision: D17403235

fbshipit-source-id: b8b84928aac7c79c4388a8ba8aa5475aac0c5219
2019-09-27 09:17:10 -07:00
Carolyn Busch
d90d66fc6e pypathmatcher: Remove python member
Summary:
Remove py from python matcher, because of RC, so python matcher will have send trait.
The python matcher needs to be stored by the python walker, and pyclasses can only
store data that is Send + 'static. Python pathmatcher methods should only be called
by python methods.

Reviewed By: xavierd

Differential Revision: D17511705

fbshipit-source-id: 00a4938fb00c30244ae04cb38362e8875c72fa47
2019-09-27 08:15:54 -07:00
Adam Simpkins
4a0165975b enable chg functionality when building with buck
Summary:
Define the `with_chg` feature when building hg_rust, and link in the chg C
code.

Reviewed By: ikostia

Differential Revision: D17109105

fbshipit-source-id: b8e210df19ff5695553d54047ed561cf7397738f
2019-09-26 20:17:40 -07:00
Adam Simpkins
1baa5c4c07 change the scm/hg:hg target to use a sh_binary() rule
Summary:
Change the `//scm/hg:hg` target to use an `sh_binary()` rule that invokes the
`:hg_rust` binary with the proper environment so it can find its dependencies,
rather than copying the binary and all of its dependencies into a new
subdirectory.

In dev mode builds the `hg_rust` binary isn't guaranteed to work anywhere
other than its original location, due to the way that dev mode builds use
`$ORIGIN` in the binary's `RPATH` setting.  This happened to work up until now
as the hg_rust binary did not have any separate libraries, but I plan to add
one on the `chg` library.

Reviewed By: quark-zju

Differential Revision: D17109104

fbshipit-source-id: ae8bb1126969f012d1d2fb7d04e80867a310b9a8
2019-09-26 20:17:40 -07:00
Tom Hebb
f6df448533 Fall back to project name if path not present
Summary:
According to the [manifest specification][1], the `path` attribute of a
`<project>` is "An optional path ... If not supplied the project name is
used". Respect this when we parse manifests ourselves.

We may want to consider importing the manifest-parsing module from `repo`
itself and using that instead of rolling our own parsing code. That would
prevent bugs like this from happening at all.

[1] https://gerrit.googlesource.com/git-repo/+/master/docs/manifest-format.md

Reviewed By: mwdevine

Differential Revision: D17542715

fbshipit-source-id: 13d963f4d9b8bf30bcd5f5593aef0ad9da90e7ec
2019-09-26 19:35:19 -07:00
Tom Hebb
0a6108156f Fix xml import
Summary:
Python doesn't recursively import submodules, so we need to specify the
exact module we're using if we want to be able to access it. I don't know
why this works in Michael's development environment, but running the
converter fails locally for me unless I do this.

Reviewed By: mwdevine

Differential Revision: D17542716

fbshipit-source-id: 5b769e193a964f8499ee35033e61d71cf0ccef90
2019-09-26 19:35:19 -07:00
Jun Wu
e8ffed0cb3 commitcloud: redirect stdin to /dev/null for background processes
Summary:
Both xavierd and I noticed that characters typed in the terminal sometimes got
eaten.  In my case, killing the background cloud sync processes solves it.

It turns out that the cloud sync erroed out like:

  ServiceError: service error: 500 ...: Mysql error 1062. Duplicate entry 'fbsource-user/quark/default-remote-fbobjc/stable' for key 'PRIMARY' to db ...

And the `cloud sync` command started ipdb, which starts eating characters :/.

D17621846 should have fixed it. Let's just make it safer by redirecting
stdin of background processes to /dev/null.

Reviewed By: xavierd

Differential Revision: D17622250

fbshipit-source-id: d804aea4aa0aeb50bba0307f93a41eb94e8529fc
2019-09-26 19:14:31 -07:00
Jun Wu
aa3170b3af util: redefine some env-sensitive functions on env change
Summary:
xavierd encountered an issue where the user name becomes `test`. It turned out
to be chgserver was started using the `TESTTMP` environment and `util.getuser`
got defined wrong.

Fix it by making `util` redefine those functions for every chg invocation.

Reviewed By: xavierd

Differential Revision: D17622095

fbshipit-source-id: e12665317957eeeb85a1796c2bc252a487805cef
2019-09-26 19:14:31 -07:00
Xavier Deguillard
f8742bd2f0 tests: fix test-graft-t.py
Reviewed By: quark-zju

Differential Revision: D17618836

fbshipit-source-id: c670ceadea3588d4fc7e002dda9855c42c803d4d
2019-09-26 19:11:40 -07:00
Xavier Deguillard
db1d4d419b tests: auto translate "test-graft.t"
Summary:
This is done by running:

  ./translate "test-graft.t"

Reviewed By: quark-zju

Differential Revision: D17618837

fbshipit-source-id: f4b420677c868efd928d38e5d677d82c218b61e5
2019-09-26 19:11:39 -07:00
Jun Wu
b692e72a22 dispatch: do not start ipdb if output is not a tty
Summary:
It turns out that `ui.interactive()` only checks if input is a tty or not,
we also need to check output. That's `ui.formatted()`.

Reviewed By: xavierd

Differential Revision: D17621846

fbshipit-source-id: 5ed30e3dfd59ed8e50261d0b085fbfded43b1c19
2019-09-26 18:43:00 -07:00
Saurabh Singh
b71a5bb279 globalrevs: remove the dependency on 'hgsubversion' extension
Summary:
We want to get rid of `hgsubversion` now that Subversion is almost
dead. Lets move the functionality still provided by the `hgsubversion`
extension to the `globalrevs` extension.

Reviewed By: quark-zju

Differential Revision: D17611443

fbshipit-source-id: 31e1b5a80ecf453e70a7e4ca929d805865579ab3
2019-09-26 17:48:40 -07:00
Jun Wu
31e7c9bc80 test-import: fix the test
Summary:
Make the Python test pass, mainly:
- avoid `grep`, use `hg log -T` instead
- rewrite shell `for` loops
- rewrite `python foo.py` to inline python function

Remove a few less important, and hard-to-translate cases.

Reviewed By: xavierd

Differential Revision: D17611805

fbshipit-source-id: a2cd5de8815b60e632b3f46b8d4493575f9388f8
2019-09-26 16:07:33 -07:00
Jun Wu
248e50dc8a tests: auto translate "test-import.t"
Summary:
This is done by running:

  ./translate "test-import.t"

Reviewed By: xavierd

Differential Revision: D17611799

fbshipit-source-id: 0db12e26dc62abccbf0b4adc2df31da76894aebb
2019-09-26 16:07:33 -07:00
Jun Wu
a3cafcbd18 test-import: only use HGEDITOR=cat
Summary:
The Python test only supports HGEDITOR=cat. Therefore update all HGEDITOR
usecases to `cat`.

Reviewed By: xavierd

Differential Revision: D17611801

fbshipit-source-id: f1fc4ed95bb424fbb1cb717ed8174fee1415ab9f
2019-09-26 16:07:32 -07:00
Jun Wu
e984f3d712 test-import: remove test cases about dirstate visibility in hooks
Summary:
Those tests use HGEDITOR in a weired way that cannot be simply migrated to
Python tests.

Those are subtle cases that don't really belong to the "import" feature.
We already have ideas changing how transaction work, in a way that in-memory
content always gets flushed to disk before running external processes. That
will make the related "in-memory state" less relevant.

Reviewed By: xavierd

Differential Revision: D17611800

fbshipit-source-id: c5bf9dc36362174dc42a16c91b7065a150e93220
2019-09-26 16:07:32 -07:00
Jun Wu
aa14510cbb test-obsmarker-template: migrate to Python
Summary: Migrate to Python so the test runs faster (70s down to 4s).

Reviewed By: singhsrb

Differential Revision: D17608330

fbshipit-source-id: 9a2e21774ba8f2f18a14d45bbd640ada76070481
2019-09-26 16:07:32 -07:00
Jun Wu
fbe8875415 tests: auto translate "test-obsmarker-template.t"
Summary:
This is done by running:

  ./translate "test-obsmarker-template.t"

Reviewed By: singhsrb

Differential Revision: D17609746

fbshipit-source-id: 5e83b33c18aa213fc51116d5e3ad982952b584d4
2019-09-26 16:07:31 -07:00
Saurabh Singh
417fed34fb crdump: remove the dependency on 'hgsubversion'
Summary:
We should be always using `globalrevs` in `crdump` instead of
`hgsubversion` but we had code supporting both the `hgsubversion` and
`globalrevs` during the `WWW` Subversion deprecation.

Reviewed By: quark-zju

Differential Revision: D17610488

fbshipit-source-id: e1cafa25bbc981cf6a2286ee1e6418194fec7853
2019-09-26 16:01:57 -07:00
Zeyi (Rice) Fan
89c898a13e fb-mercurial-rust: make CMake build fetch vendored tp2 first
Summary: We do not have network access in Sandcastle but Mercurial needs some external crates to build. So we download the tp2 crates io tar from LFS and set up cargo to be able to find external crates from it.

Reviewed By: wez

Differential Revision: D17407962

fbshipit-source-id: 4fe6be42d52c79340b33687d13723fcbcd63754e
2019-09-26 15:50:52 -07:00
Zeyi (Rice) Fan
951b9e95f5 getdeps: add rust_static_library to build Rust crate
Reviewed By: simpkins

Differential Revision: D16945510

fbshipit-source-id: a7a88cd94235e3f8c01235d7e7e500c90bde3b38
2019-09-26 15:50:51 -07:00
Xavier Deguillard
2cd2ffbd58 replace use of Queue with pycompat.queue
Summary: The Queue module doesn't exist in Python3, use pycompat.queue instead.

Reviewed By: ikostia

Differential Revision: D17610680

fbshipit-source-id: 7f5bc7fe2b434fd959dc5fd5ed821c7b73b8030d
2019-09-26 15:29:49 -07:00
Xavier Deguillard
2d2d16ba94 replace StringIO with pycompat.StringIO
Summary:
StringIO in Python3 expect strings, while we use it with bytes, the pycompat
module handles the difference properly.

Reviewed By: ikostia

Differential Revision: D17609687

fbshipit-source-id: 16a06c8ffceaed3d3c4acf93b94102e3c9d5c11d
2019-09-26 15:29:49 -07:00
Xavier Deguillard
c5957985a3 use pycompat.range instead of xrange
Summary: The xrange function doesn't exist in python3, use the pycompat version instead.

Reviewed By: ikostia

Differential Revision: D17607522

fbshipit-source-id: c7b992071ad7933372033892ca99e240aa11ccba
2019-09-26 15:29:48 -07:00
Xavier Deguillard
e65af0a803 tests: test-help no longer disable treemanifest
Summary: See title.

Reviewed By: quark-zju

Differential Revision: D17605404

fbshipit-source-id: ea069c0b2bb306b4d44018a9a18a7359bbaf46b0
2019-09-26 14:45:35 -07:00