Commit Graph

47065 Commits

Author SHA1 Message Date
Stefan Filip
45784815c1 dott: update shobj to expand filename patterns
Summary:
Filename expansion is probably used a lot in our tests. I think that our
migration path is going to be simplified if we do filename expansion as
part of shobj. The current implementation works for the test that I was
migrating, test-revert.

Reviewed By: quark-zju

Differential Revision: D17630872

fbshipit-source-id: b8312e3d1fee9d331a661849a4cc5ef06ed5b091
2019-09-27 13:44:59 -07:00
Stefan Filip
9454158308 dott: update chmod to support -x
Summary: Following a similar implementation to `chmod +x`.

Reviewed By: quark-zju

Differential Revision: D17630871

fbshipit-source-id: 115fe8fd0cca214f7f739bec66301c9116edb250
2019-09-27 13:44:59 -07:00
Stefan Filip
e16d63bb2d tests: update generateworkingcopystates to work as a module
Summary: I would like to call into this script directly from dott tests.

Reviewed By: quark-zju

Differential Revision: D17630870

fbshipit-source-id: 169ab3f1706d77cb1cdc1461b2a6fce2bc77a877
2019-09-27 13:44:58 -07:00
Stefan Filip
40e989ef9a tests: rename generate-working-copy-states
Summary:
Renaming it to `generateworkingcopystates`. This is a valid module name.
The goal is to convert this script to a module. While we are converting
tests it needs to be accessible in both script form and module.

Reviewed By: singhsrb

Differential Revision: D17611370

fbshipit-source-id: 0b2d0de71f0da7cd6efd7874c505d9aa77f7b296
2019-09-27 13:44:58 -07:00
Stefan Filip
92f92d5138 manifest: add test verifying materialization in finalize
Summary:
This test checks that a directory without modifications between tree
and parent is not going to be materialized.

Reviewed By: quark-zju

Differential Revision: D17540173

fbshipit-source-id: 465f1e0410c42a55665bcd6903d75266c61d5e80
2019-09-27 13:07:21 -07:00
Jun Wu
4b5455424a util: fix hgcmd on buck build
Summary:
`util.hgcmd()` can report an incorrect hg path in the buck build case, since
the buck target is a wrapper script.

Fix it by setting the executable path explicitly.

Reviewed By: simpkins

Differential Revision: D17632684

fbshipit-source-id: d6c208fa1981180f6248bf7b57cc57031f53946e
2019-09-27 11:57:47 -07:00
Arun Kulshreshtha
55d07eeb94 manifest: remove dfs diff
Summary: Remove the DFS diff implementation and replace it with the BFS implementation.

Reviewed By: xavierd

Differential Revision: D17618818

fbshipit-source-id: d486642caae924f866a200d3c82fa5a4cb7d5286
2019-09-27 11:07:23 -07:00
Arun Kulshreshtha
e6f577bf01 treemanifest: enable bfs diff by default
Summary: In order to support efficient data fetching, eventually BFS diff will need to be the default diff algorithm. As an initial step towards this, enable `treemanifest.bfsdiff` by default to validate that the BFS diff algorithm works as a drop-in replacement for the existing diff algorithm. Once we have confidence in this, we can land the next diff in this stack wherein we remove DFS diff entirely.

Reviewed By: xavierd

Differential Revision: D17618819

fbshipit-source-id: 444cb05874913e4eb267a24bd0596e6dde2e099e
2019-09-27 11:07:23 -07:00
Arun Kulshreshtha
323aaa9dd1 manifest: copy dfs diff tests into bfs diff module
Summary: To ensure feature parity between BFS diff and DFS diff, copy the DFS tests into the BFS module and ensure they pass.

Reviewed By: xavierd

Differential Revision: D17618820

fbshipit-source-id: b516abbfa4e231fdc383293d94d8965333f2ab99
2019-09-27 11:07:23 -07:00
Saurabh Singh
7aab6858f0 test-hgsubversion-globalrevs: remove the test
Summary:
This test is currently failing and its not worth fixing since we only
needed this during the `WWW` Subversion deprecation which has already happened.

Reviewed By: quark-zju

Differential Revision: D17629893

fbshipit-source-id: 60f56b84a3cdcd8ddc56d46a155b1f14f2c00646
2019-09-27 09:49:50 -07:00
Jun Wu
7f3b4d93e9 chgserver: deal with fdopen EINVAL case
Summary:
The use of `fdopen` was to change the buffer size. It's not fatal if buffer
size cannot be changed. This fixes a crash in `nohup hg root`.

Reviewed By: aditya7fb

Differential Revision: D17628883

fbshipit-source-id: 58f75e8e99fbe4cea8da2e3b48a092ee1331db6f
2019-09-27 09:45:09 -07:00
Carolyn Busch
6cb28d027c Workingcopy python binding
Summary: Python binding for working copy rust library. Walker is initialized with the root of the repo and the python matcher, iteratively returns the matching files in the working copy. Modified Walker and PythonMatcher to allow Walker to have Send trait.

Reviewed By: xavierd

Differential Revision: D17403235

fbshipit-source-id: b8b84928aac7c79c4388a8ba8aa5475aac0c5219
2019-09-27 09:17:10 -07:00
Carolyn Busch
d90d66fc6e pypathmatcher: Remove python member
Summary:
Remove py from python matcher, because of RC, so python matcher will have send trait.
The python matcher needs to be stored by the python walker, and pyclasses can only
store data that is Send + 'static. Python pathmatcher methods should only be called
by python methods.

Reviewed By: xavierd

Differential Revision: D17511705

fbshipit-source-id: 00a4938fb00c30244ae04cb38362e8875c72fa47
2019-09-27 08:15:54 -07:00
Adam Simpkins
4a0165975b enable chg functionality when building with buck
Summary:
Define the `with_chg` feature when building hg_rust, and link in the chg C
code.

Reviewed By: ikostia

Differential Revision: D17109105

fbshipit-source-id: b8e210df19ff5695553d54047ed561cf7397738f
2019-09-26 20:17:40 -07:00
Adam Simpkins
1baa5c4c07 change the scm/hg:hg target to use a sh_binary() rule
Summary:
Change the `//scm/hg:hg` target to use an `sh_binary()` rule that invokes the
`:hg_rust` binary with the proper environment so it can find its dependencies,
rather than copying the binary and all of its dependencies into a new
subdirectory.

In dev mode builds the `hg_rust` binary isn't guaranteed to work anywhere
other than its original location, due to the way that dev mode builds use
`$ORIGIN` in the binary's `RPATH` setting.  This happened to work up until now
as the hg_rust binary did not have any separate libraries, but I plan to add
one on the `chg` library.

Reviewed By: quark-zju

Differential Revision: D17109104

fbshipit-source-id: ae8bb1126969f012d1d2fb7d04e80867a310b9a8
2019-09-26 20:17:40 -07:00
Tom Hebb
f6df448533 Fall back to project name if path not present
Summary:
According to the [manifest specification][1], the `path` attribute of a
`<project>` is "An optional path ... If not supplied the project name is
used". Respect this when we parse manifests ourselves.

We may want to consider importing the manifest-parsing module from `repo`
itself and using that instead of rolling our own parsing code. That would
prevent bugs like this from happening at all.

[1] https://gerrit.googlesource.com/git-repo/+/master/docs/manifest-format.md

Reviewed By: mwdevine

Differential Revision: D17542715

fbshipit-source-id: 13d963f4d9b8bf30bcd5f5593aef0ad9da90e7ec
2019-09-26 19:35:19 -07:00
Tom Hebb
0a6108156f Fix xml import
Summary:
Python doesn't recursively import submodules, so we need to specify the
exact module we're using if we want to be able to access it. I don't know
why this works in Michael's development environment, but running the
converter fails locally for me unless I do this.

Reviewed By: mwdevine

Differential Revision: D17542716

fbshipit-source-id: 5b769e193a964f8499ee35033e61d71cf0ccef90
2019-09-26 19:35:19 -07:00
Jun Wu
e8ffed0cb3 commitcloud: redirect stdin to /dev/null for background processes
Summary:
Both xavierd and I noticed that characters typed in the terminal sometimes got
eaten.  In my case, killing the background cloud sync processes solves it.

It turns out that the cloud sync erroed out like:

  ServiceError: service error: 500 ...: Mysql error 1062. Duplicate entry 'fbsource-user/quark/default-remote-fbobjc/stable' for key 'PRIMARY' to db ...

And the `cloud sync` command started ipdb, which starts eating characters :/.

D17621846 should have fixed it. Let's just make it safer by redirecting
stdin of background processes to /dev/null.

Reviewed By: xavierd

Differential Revision: D17622250

fbshipit-source-id: d804aea4aa0aeb50bba0307f93a41eb94e8529fc
2019-09-26 19:14:31 -07:00
Jun Wu
aa3170b3af util: redefine some env-sensitive functions on env change
Summary:
xavierd encountered an issue where the user name becomes `test`. It turned out
to be chgserver was started using the `TESTTMP` environment and `util.getuser`
got defined wrong.

Fix it by making `util` redefine those functions for every chg invocation.

Reviewed By: xavierd

Differential Revision: D17622095

fbshipit-source-id: e12665317957eeeb85a1796c2bc252a487805cef
2019-09-26 19:14:31 -07:00
Xavier Deguillard
f8742bd2f0 tests: fix test-graft-t.py
Reviewed By: quark-zju

Differential Revision: D17618836

fbshipit-source-id: c670ceadea3588d4fc7e002dda9855c42c803d4d
2019-09-26 19:11:40 -07:00
Xavier Deguillard
db1d4d419b tests: auto translate "test-graft.t"
Summary:
This is done by running:

  ./translate "test-graft.t"

Reviewed By: quark-zju

Differential Revision: D17618837

fbshipit-source-id: f4b420677c868efd928d38e5d677d82c218b61e5
2019-09-26 19:11:39 -07:00
Jun Wu
b692e72a22 dispatch: do not start ipdb if output is not a tty
Summary:
It turns out that `ui.interactive()` only checks if input is a tty or not,
we also need to check output. That's `ui.formatted()`.

Reviewed By: xavierd

Differential Revision: D17621846

fbshipit-source-id: 5ed30e3dfd59ed8e50261d0b085fbfded43b1c19
2019-09-26 18:43:00 -07:00
Saurabh Singh
b71a5bb279 globalrevs: remove the dependency on 'hgsubversion' extension
Summary:
We want to get rid of `hgsubversion` now that Subversion is almost
dead. Lets move the functionality still provided by the `hgsubversion`
extension to the `globalrevs` extension.

Reviewed By: quark-zju

Differential Revision: D17611443

fbshipit-source-id: 31e1b5a80ecf453e70a7e4ca929d805865579ab3
2019-09-26 17:48:40 -07:00
Jun Wu
31e7c9bc80 test-import: fix the test
Summary:
Make the Python test pass, mainly:
- avoid `grep`, use `hg log -T` instead
- rewrite shell `for` loops
- rewrite `python foo.py` to inline python function

Remove a few less important, and hard-to-translate cases.

Reviewed By: xavierd

Differential Revision: D17611805

fbshipit-source-id: a2cd5de8815b60e632b3f46b8d4493575f9388f8
2019-09-26 16:07:33 -07:00
Jun Wu
248e50dc8a tests: auto translate "test-import.t"
Summary:
This is done by running:

  ./translate "test-import.t"

Reviewed By: xavierd

Differential Revision: D17611799

fbshipit-source-id: 0db12e26dc62abccbf0b4adc2df31da76894aebb
2019-09-26 16:07:33 -07:00
Jun Wu
a3cafcbd18 test-import: only use HGEDITOR=cat
Summary:
The Python test only supports HGEDITOR=cat. Therefore update all HGEDITOR
usecases to `cat`.

Reviewed By: xavierd

Differential Revision: D17611801

fbshipit-source-id: f1fc4ed95bb424fbb1cb717ed8174fee1415ab9f
2019-09-26 16:07:32 -07:00
Jun Wu
e984f3d712 test-import: remove test cases about dirstate visibility in hooks
Summary:
Those tests use HGEDITOR in a weired way that cannot be simply migrated to
Python tests.

Those are subtle cases that don't really belong to the "import" feature.
We already have ideas changing how transaction work, in a way that in-memory
content always gets flushed to disk before running external processes. That
will make the related "in-memory state" less relevant.

Reviewed By: xavierd

Differential Revision: D17611800

fbshipit-source-id: c5bf9dc36362174dc42a16c91b7065a150e93220
2019-09-26 16:07:32 -07:00
Jun Wu
aa14510cbb test-obsmarker-template: migrate to Python
Summary: Migrate to Python so the test runs faster (70s down to 4s).

Reviewed By: singhsrb

Differential Revision: D17608330

fbshipit-source-id: 9a2e21774ba8f2f18a14d45bbd640ada76070481
2019-09-26 16:07:32 -07:00
Jun Wu
fbe8875415 tests: auto translate "test-obsmarker-template.t"
Summary:
This is done by running:

  ./translate "test-obsmarker-template.t"

Reviewed By: singhsrb

Differential Revision: D17609746

fbshipit-source-id: 5e83b33c18aa213fc51116d5e3ad982952b584d4
2019-09-26 16:07:31 -07:00
Saurabh Singh
417fed34fb crdump: remove the dependency on 'hgsubversion'
Summary:
We should be always using `globalrevs` in `crdump` instead of
`hgsubversion` but we had code supporting both the `hgsubversion` and
`globalrevs` during the `WWW` Subversion deprecation.

Reviewed By: quark-zju

Differential Revision: D17610488

fbshipit-source-id: e1cafa25bbc981cf6a2286ee1e6418194fec7853
2019-09-26 16:01:57 -07:00
Zeyi (Rice) Fan
89c898a13e fb-mercurial-rust: make CMake build fetch vendored tp2 first
Summary: We do not have network access in Sandcastle but Mercurial needs some external crates to build. So we download the tp2 crates io tar from LFS and set up cargo to be able to find external crates from it.

Reviewed By: wez

Differential Revision: D17407962

fbshipit-source-id: 4fe6be42d52c79340b33687d13723fcbcd63754e
2019-09-26 15:50:52 -07:00
Zeyi (Rice) Fan
951b9e95f5 getdeps: add rust_static_library to build Rust crate
Reviewed By: simpkins

Differential Revision: D16945510

fbshipit-source-id: a7a88cd94235e3f8c01235d7e7e500c90bde3b38
2019-09-26 15:50:51 -07:00
Xavier Deguillard
2cd2ffbd58 replace use of Queue with pycompat.queue
Summary: The Queue module doesn't exist in Python3, use pycompat.queue instead.

Reviewed By: ikostia

Differential Revision: D17610680

fbshipit-source-id: 7f5bc7fe2b434fd959dc5fd5ed821c7b73b8030d
2019-09-26 15:29:49 -07:00
Xavier Deguillard
2d2d16ba94 replace StringIO with pycompat.StringIO
Summary:
StringIO in Python3 expect strings, while we use it with bytes, the pycompat
module handles the difference properly.

Reviewed By: ikostia

Differential Revision: D17609687

fbshipit-source-id: 16a06c8ffceaed3d3c4acf93b94102e3c9d5c11d
2019-09-26 15:29:49 -07:00
Xavier Deguillard
c5957985a3 use pycompat.range instead of xrange
Summary: The xrange function doesn't exist in python3, use the pycompat version instead.

Reviewed By: ikostia

Differential Revision: D17607522

fbshipit-source-id: c7b992071ad7933372033892ca99e240aa11ccba
2019-09-26 15:29:48 -07:00
Xavier Deguillard
e65af0a803 tests: test-help no longer disable treemanifest
Summary: See title.

Reviewed By: quark-zju

Differential Revision: D17605404

fbshipit-source-id: ea069c0b2bb306b4d44018a9a18a7359bbaf46b0
2019-09-26 14:45:35 -07:00
Aleksei Kulikov
65c288f459 snapshot: hide and unhide snapshots
Summary:
`hg hide <snapshot id>` should remove it from the list of snapshots (=> no sync and not shown in smartlog).
`hg unhide <snapshot id>` should restore the snapshot in the list, but do not unhide the underlying commit.

Reviewed By: markbt

Differential Revision: D17570798

fbshipit-source-id: 895f7b53464860b80f4f2498ea97f8db270b4d67
2019-09-26 14:20:23 -07:00
Aleksei Kulikov
aaca5459b6 snapshot: add snapshot show command to view individual snapshots
Summary:
Need a cmd which is equivalent to `hg show <commit id>` for snapshots.
```
$ hg snapshot show <snapshot id>
<the regular patch message for commits>
  ? untrackedfile
  @@ -0,0 +1,1 @@  # diff-like content
  +fizz
  ! missingfile
  @@ -1,1 +0,0 @@  # diff-like content
  -foo
  ? .hg/merge/fc4ffdcb8ed23cecd44a0e11d23af83b445179b4
  @@ -0,0 +1,1 @@
  +b

  The snapshot is in an unfinished *merge* state.
```

Reviewed By: mitrandir77

Differential Revision: D17529050

fbshipit-source-id: 331c68f64a5ad71d6d77363a9dd1aed6e569efa5
2019-09-26 14:20:22 -07:00
Aleksei Kulikov
a829fee85c snapshot: show snapshots in smartlog
Summary:
Snapshots can be marked with letter `s`:
```
  s    changeset:   8:510948b1f172
  |\   parent:      7:fdf2c0326bba
  | |  parent:      6:55f800d93e34
  | |  user:        test
  | |  date:        Thu Jan 01 00:00:00 1970 +0000
  | |  summary:     snapshot
  | |
  | @  changeset:   7:fdf2c0326bba
  | |  parent:      2:fa948fa73a59
  | |  user:        test
  | |  date:        Thu Jan 01 00:00:00 1970 +0000
  | |  summary:     merge #2
  | |
  @ |  changeset:   6:55f800d93e34
  | |  user:        test
  | |  date:        Thu Jan 01 00:00:00 1970 +0000
  | |  summary:     merge #1
  | |
  o |  changeset:   5:afed36f05817
  | |  user:        test
  | |  date:        Thu Jan 01 00:00:00 1970 +0000
  | |  summary:     draft2
```

Reviewed By: mitrandir77

Differential Revision: D17533027

fbshipit-source-id: da22fef6c23933e2c91d6f5b4ff6d4adb2714e31
2019-09-26 14:20:22 -07:00
Jun Wu
a381eddae1 tests: add a script to translate tests to Python
Summary:
The script does:
- Run the Python translate script.
  - With `--verify` so the test will be disabled if it does not pass.
  - With `--black` so the test file is formatted.
  - With `--hgmv` so the test is marked as a "rename", and `hg log` on it can
    trace back to the old history.
- Make a commit, so the manual edits are easier to review.

Reviewed By: xavierd

Differential Revision: D17609337

fbshipit-source-id: 0ce61e8a2e1a1d4424b1d3f15426f44a6fcaa739
2019-09-26 13:23:29 -07:00
Jun Wu
6833cec26d test-obsmarker-template: remove use of getid
Summary:
Change `debugobsolete` to take revsets so we don't need to use the shell
substitution feature. This makes it easier to migrate the test to Python.

Reviewed By: singhsrb

Differential Revision: D17608329

fbshipit-source-id: 5f1201b8f16aebe39de14fdbda17a7bfc485dba2
2019-09-26 13:13:44 -07:00
Jun Wu
6d83c91c4d test-obsmarker-template: remove tests about cycles
Summary:
The cycle tests will break with an upcoming change. Our desired cycle
handling behavior is different from upstream, and our main 2nd-order
history storage is the mutation store, which has its own cycle tests
in test-mutation-loops.t. Therefore remove this test section.

Reviewed By: singhsrb

Differential Revision: D17608331

fbshipit-source-id: 658b04cf9c30811a12614cef822a1cdf80551e67
2019-09-26 13:13:44 -07:00
Mateusz Kwapich
114bc9d1aa xdiff: add a friendly wrapper for rust bindings (xdiff)
Summary:
This will make using xdiff much easier.

In the next diff I'm planning to also add a function that converts the output of this function to a textual diff (the one with `+`'s and `-`'s).

Reviewed By: quark-zju

Differential Revision: D17551184

fbshipit-source-id: cda332e817f733d7aa32aeeb7b2d312d971826dd
2019-09-26 11:01:09 -07:00
Mateusz Kwapich
02db823918 xdiff: add rust bindings (xdiff-sys)
Summary:
These are limited (not all features are exposed) bindings for xdiff - the diff library used by git and our version of hg. We need them to be able to generate diffs in Mononoke.

In the next diff I'll add more rust-friendly wrapper library.

Reviewed By: quark-zju

Differential Revision: D17548528

fbshipit-source-id: f23c8a65d11d2c5de8f0456d32883f16b19a98e2
2019-09-26 11:01:09 -07:00
Jun Wu
4afaf065fe indexedlog: add checks about the append-only property
Summary:
Both Index and Log requires their on-disk files to be append-only.
Detect non-append-only changes and return errors. This might help
us get better error messages if the case actually happens.

Reviewed By: markbt

Differential Revision: D17592914

fbshipit-source-id: b12791177ceb04f2373e93a679101e8b96e2bc98
2019-09-26 10:19:44 -07:00
Jeffrey Dunn
727359cd30 Add hash binary config option for hg diff
Summary:
When there's a change in a binary file, it's useful for it to appear
in the output of hg diff in some format. The options today are just to get
a text message saying that there is a change or a large binary blob of the
change. This introduces a middleground where we see the hash of the final
binary.

Reviewed By: quark-zju

Differential Revision: D17579217

fbshipit-source-id: ca204bb0ee28bfef327d57eea8236d7421bf933f
2019-09-26 09:22:18 -07:00
Aleksei Kulikov
08cdcd0ea2 snapshot: store a list of snapshots instead of set
Summary:
Storing nodes in a `set` does not make much sense because of possible aliases etc.
It is possible to remove duplicates via `util.removeduplicates` function.
List, on the other hand, preserves order and there will be less conversions during sync (later in the stack).

Differential Revision: D17397594

fbshipit-source-id: ba76e0a1f53516544f055355356d0beaf1c8bfa8
2019-09-26 04:50:18 -07:00
Aleksei Kulikov
83708239f0 snapshot: store hex nodes of the snapshots
Differential Revision: D17397473

fbshipit-source-id: 36e956fa3031bbc2eb92a75a0c3a18d312df5ef6
2019-09-26 04:50:17 -07:00
Aleksei Kulikov
14d3872615 snapshot: make the snapshots invisible after pull
Summary:
Right now `hg pull -r REV` marks all pulled revs as visible.
This makes sense for regular commits (if they exist in the cloud, then they are not hidden).
For snapshots, it is wrong, thus we need not to mark snapshots as visible when pulling.

Differential Revision: D17289438

fbshipit-source-id: d16237b9e471f49bb44317104bc445472bedc16c
2019-09-26 04:50:17 -07:00
Henri Verroken
836347bf1b sparse: warn when enabling non-existing profile
Summary:
Add a warning when a user enables a sparse profile on the command line that does not exist.

Triggering the check from multiple places, to not have to clutter the `_config` function. The `_config` function is called from other places as well (eg cloning a sparse repo) and I didn't want to add another guard parameter to it for such a benign functionality.

Besides the `_checknonexistingprofiles` function is only called in 3 situtations:

1. `hg sparse {enable|enableprofile}`
2. `hg sparse --enable-profile` (which is actually deprecated)
3. `hg clone --enable-profile`

Special care has been taken when cloning a sparse repo to do the check after the clone.

Reviewed By: markbt

Differential Revision: D17571442

fbshipit-source-id: 882dbe8b142bf192754dd17d0729d1bb0b472a83
2019-09-26 02:23:09 -07:00