Commit Graph

46388 Commits

Author SHA1 Message Date
Jun Wu
ca42b02813 testutil/dott: fix test command
Summary: The "succeed" code path was missing in the `test` command.

Reviewed By: xavierd

Differential Revision: D16452921

fbshipit-source-id: a276b774c528edaf60e0ecbbad00c15ddd8ba1d1
2019-07-24 14:32:23 -07:00
Jun Wu
b88812e338 testutil: stop generating lines with trailing spaces
Summary:
The `_repr` implementation can generate docstrings with trailing spaces if
`indent` is > 0.

Fix it by special handling the blank line case.

Reviewed By: xavierd

Differential Revision: D16452935

fbshipit-source-id: 99704ba3fb30b93aa35ba4fb16c1d853c58c6635
2019-07-24 14:32:23 -07:00
Jun Wu
8beb778255 testutil/dott: fix translator dropping redirections
Summary:
In case there are multiple commands in a line, and the first command uses
redirections:

  foo > bar && baz

The `> bar` got lost after translation. This is because the same `opts` Python
object (including `> bar` information) was returned and it was mutated.

Solve it by returning a copy of `opts` per command.

Reviewed By: xavierd

Differential Revision: D16452936

fbshipit-source-id: a7ab55cc44ebe628817f821412576e17f6f68a6f
2019-07-24 14:32:23 -07:00
Jun Wu
6cd7d1799c encoding: add a way to change HGENCODING at runtime
Summary:
Make it possible to change the internal state of HGENCODING so we can run hg
logic within a single process without shelling out in the new dott test
framework.

Update testutil/dott to use it.

Reviewed By: xavierd

Differential Revision: D16452925

fbshipit-source-id: d21329854eeee171cc5d02d4f42f11dd273f2150
2019-07-24 14:32:23 -07:00
Jun Wu
5dbb19b028 indexedlog: add a binary to repair indexedlog
Summary:
This utility can be useful to unblock people with corrupted indexed logs after
hard reboot.

Reviewed By: xavierd

Differential Revision: D16433835

fbshipit-source-id: 72f0aff4b266072c58d2c574897e580a1529b57f
2019-07-24 14:18:40 -07:00
Jun Wu
f9d6b516e5 indexedlog: add a repair method
Summary:
The `repair` method truncates data to repair the main log. It can be useful to
deal with OS or process crashes.

Reviewed By: xavierd

Differential Revision: D16433832

fbshipit-source-id: 7a7beda0efa30cda36ff039054a8dda650002f79
2019-07-24 14:18:40 -07:00
Jun Wu
58e8de3c7c indexedlog: add an API to rebuild indexes
Summary:
This function rebuilds the indexes in case they are corrupted due to OS crash.
It can also reduce the size of indexes.

Reviewed By: xavierd

Differential Revision: D16433833

fbshipit-source-id: 7a044f4a36024dd81f40b8b8c1ed83daa9d0f0a9
2019-07-24 14:18:39 -07:00
Jun Wu
7f52f7fb6d indexedlog: split an internal method to update a single index
Summary: This will be used by the next diff.

Reviewed By: xavierd

Differential Revision: D16433837

fbshipit-source-id: daf9b3858c4ee595ea762fbbf3a20e29c5fec496
2019-07-24 14:18:39 -07:00
Jun Wu
90dfcbd755 indexedlog: minor tweak to set_index_log_len
Summary:
Change the API to take a mutable iterator so it is more flexible.  Also fix the
actual index_meta data so `len` does not get written repeatively.

Reviewed By: xavierd

Differential Revision: D16433836

fbshipit-source-id: 704e1bfc18e9b20790c3185006c25ea156d15e53
2019-07-24 14:18:39 -07:00
Jun Wu
10b6ebfa39 indexedlog: fix some spelling issues
Summary: Just noticed them via vim spellchecker.

Reviewed By: singhsrb

Differential Revision: D16430541

fbshipit-source-id: ae3e19bf7efc35021f45ff54415f5d9e9be4a56d
2019-07-24 14:18:38 -07:00
Xavier Deguillard
9b21206b39 remotefilelog: remove python mutablehistorypack
Summary: There no users left, let's get rid of it.

Reviewed By: kulshrax

Differential Revision: D16423137

fbshipit-source-id: 6898e681f800ab677010d7b6cdd36d377e3ef644
2019-07-24 10:34:32 -07:00
Xavier Deguillard
0cf51d84e3 revisionstore: copy historypack file format description from python
Summary:
The Rust code didn't have a nice description of the file format, let's copy it
from the Python code.

Reviewed By: kulshrax

Differential Revision: D16423135

fbshipit-source-id: beeae2857fa5e999a3b41e72726b29b865a6a2b4
2019-07-24 10:34:32 -07:00
Xavier Deguillard
10a59c9b75 remotefilelog: remove python mutablehistorypack from mutablestore
Summary: The Rust code is the default.

Reviewed By: kulshrax

Differential Revision: D16423134

fbshipit-source-id: 30baf041a5e7c786a28b31cfe5025e893b34594c
2019-07-24 10:34:32 -07:00
Xavier Deguillard
5d9d464558 tests: remove mutablehistorypack from tests
Summary: Only one place was using it, remove it.

Reviewed By: quark-zju

Differential Revision: D16423138

fbshipit-source-id: 8a679c5d841f9e404099ba69023d98e02fa6a65b
2019-07-24 10:34:31 -07:00
Xavier Deguillard
e86e388661 remotefilelog: remove python mutabledatapack code
Summary: Goodbye.

Reviewed By: kulshrax

Differential Revision: D16392290

fbshipit-source-id: 541b945e274f097b722b5e79ca03e104c4cfd6a0
2019-07-24 10:34:31 -07:00
Xavier Deguillard
c6dccb4955 remotefilelog: all the tests are now using rust mutable datapack
Summary: There is no longer any code depending on the python mutabledatapack.

Reviewed By: kulshrax

Differential Revision: D16392286

fbshipit-source-id: 7fa2a622d1df4e32846c5ab93639fd2d5b509a3e
2019-07-24 10:34:31 -07:00
Xavier Deguillard
bc16992f65 revisionstore: support partial chains in MutableDataPack get_delta_chain
Summary:
The code would wrongly fail if only part of a chain is present in the store.
Let's properly succeed in this case.

Reviewed By: kulshrax

Differential Revision: D16392288

fbshipit-source-id: addaf340f947e48a2747bbe240d6aae081ae165c
2019-07-24 10:34:30 -07:00
Xavier Deguillard
bd88a79858 remotefilelog: remove python mutabledatapack reference from mutablestores
Summary:
The Rust code is now always taken, let's start removing all the references to
the python mutabledatapack code.

Reviewed By: kulshrax

Differential Revision: D16392287

fbshipit-source-id: dfccd4f4ec052a46975b6f9144106b51c3282988
2019-07-24 10:34:30 -07:00
Xavier Deguillard
334e0c0c2e remotefilelog: the rust mutable stores are the default
Summary: This is now enabled for the entire fleet, let's hardcode this in the code now.

Reviewed By: kulshrax

Differential Revision: D16392289

fbshipit-source-id: 462152ded12d00cf8218526d51a911d6fe5975ca
2019-07-24 10:34:30 -07:00
George-Catalin Tintareanu
15f36ceede hg client handler for blacklisted files
Summary: Checks if the retrieved content of a file is equal to a `magic string` representing a blacklisted file. If so, then the content is replaced by a readable text which suggests a `rebase` or `update` to a newer commit.

Reviewed By: ikostia

Differential Revision: D16260011

fbshipit-source-id: ac1d40132b9c947927271d8e6efda98b19dce984
2019-07-24 03:41:20 -07:00
Arun Kulshreshtha
a491bb06ea remotefilelog: only print debug messages during interactive usage
Summary: The debug messages from Eden API provide Source Control team members with useful diagnostic information about HTTP data fetching, but they have the potential to be spammy when written to log files. To prevent log spam, let's only print these messages during interactive usage.

Reviewed By: quark-zju

Differential Revision: D16445346

fbshipit-source-id: 001dc75e440eaf797f4f953648453086421f624e
2019-07-23 15:13:59 -07:00
Zeyi (Rice) Fan
a62bf843cf Back out "RFC: hg: make zsh completion to return list of draft commits"
Summary:
Original commit changeset: 3493895a12ae

This is breaking `arc pull` in fbsource S183062

 #commitClose

Reviewed By: mitrandir77, singhsrb, xavierd

Differential Revision: D16441944

fbshipit-source-id: fd3f5c7027be1468bd8194e6b24dd136b9b767d2
2019-07-23 13:07:14 -07:00
Mark Thomas
d384df8a35 setup.py: always copy library files from fbsource
Summary:
The `fbsourcepylibrary` class only copies the source files if the destination
directory does not exist.  Once the files have been copied, subsequent builds
will not pick up new versions.  This means builds can be stale.

Remove the special case of `_isready`.  This will fall back to the default
implementation which will copy the files each time the build happens.

Reviewed By: quark-zju

Differential Revision: D16438778

fbshipit-source-id: 66dc0b69b427650087cfb822acdbf1fc797babbc
2019-07-23 11:01:08 -07:00
Xavier Deguillard
7478874389 remotefilelog: remove maintenance repack memcache fetch
Summary:
When using fetchpacks, memcache will write to an indexedlog, which can't be
repacked. I'm also suspecting that there is a race between this repack, and
hg_memcache_client opening the newly created packfiles which can cause the
misses to not be sent to memcache.

Reviewed By: kulshrax

Differential Revision: D16432538

fbshipit-source-id: 62362682474883bcd58249791c02b9fed5cb8fea
2019-07-22 22:31:38 -07:00
Shu-Ting Tseng
5c49bedd8b add a trailing newline when committing to svn
Summary:
By adding a newline, we can ensure the meta message is always on its own line.

This is to address the comment in D16340374.

link to comment: https://our.intern.facebook.com/intern/diff/D16340374/?transaction_id=732272950561759&src_number=86745022

Reviewed By: quark-zju

Differential Revision: D16417344

fbshipit-source-id: caaae1748018bbe5ddbb587adacd45b8709da8cc
2019-07-22 14:06:12 -07:00
Stefan Filip
7c511ba610 manifest: add matcher filtering to the files iterator
Summary:
Most operations do not work on iterate on all the files of the repository.
Most operations filter the data set in some way. In many cases this filtering
is to a set of files and in some cases using patterns or subdirectories.

The Python code uses `match.py` to represent this filtering. The parallel
data structure in the rust code is `pathmatcher::Matcher`.

This diff adds `files` integration with `pathmatcher::Matcher`.

Reviewed By: quark-zju

Differential Revision: D16352527

fbshipit-source-id: 8b61ac7399f581773bf61ff648634cbc6e1a27b6
2019-07-22 13:03:02 -07:00
Stefan Filip
3bb2fa1efb bindings: add Macher implementation for PyObj
Summary:
This allows us to use python matcher objects in rust. Particularly we are
looking at using these in the manifest implementation for filtering the
file set that is returned.

Reviewed By: quark-zju

Differential Revision: D16352533

fbshipit-source-id: fd6bde6d9223203c69593d5e8830946170363243
2019-07-22 13:03:01 -07:00
Stefan Filip
e05693d74b manifest: move file description code from lib.rs to file.rs
Summary:
Title. Small clean up. I think that this makes sense because the code in
file.rs is self contained. It provides a distraction free environment for
adding methods to the structures in there.

Reviewed By: quark-zju

Differential Revision: D16352531

fbshipit-source-id: c23e943198e0a4b50aa00c75e67b13bc4c3ee976
2019-07-22 13:03:01 -07:00
Stefan Filip
75fd92d1a2 pathmatcher: add Matcher trait
Summary:
"matcher.py" is used in several places where the the file tree is traversed.
We need to use implement the matcher functionality in the Rust manifest
implementation. We define a common type to be used in our internal code.

In our current state of interfacing a lot with Python, fast paths of matching
full trees is useful so we have 3 states for matching a directory:
 * everything in the directory subtree should be returned
 * nothing in the directory subtree should be returned
 * another state where there is no fast path and the directory should be
 traversed recusively; this state is always valid to return and doe not
 impact correctness

The interface for the Matcher is defined in relationship to RepoPath. We store
paths internally in the same binary format no matter the operating system path
representation. Using std::path would incur a translation cost.

Reviewed By: quark-zju

Differential Revision: D16352528

fbshipit-source-id: 61b259f4347cfaf6f74ee36fa5955e45e4beb739
2019-07-22 13:03:00 -07:00
Stefan Filip
cb57f5d7ac bindings: add manifest python classes
Summary: Bindings so that the rust manifest code can be used in Python.

Reviewed By: quark-zju

Differential Revision: D16352532

fbshipit-source-id: 34d4522f5e084f531f31bcd21770950f15f2fe13
2019-07-22 13:03:00 -07:00
Stefan Filip
69b604e3ec pyrevisionstore: remove unsafe impl Send for PythonDataStore
Summary:
https://doc.rust-lang.org/nomicon/send-and-sync.html
http://dgrunwald.github.io/rust-cpython/doc/cpython/struct.PyObject.html

PyObject implements Send. Because Send is automatically derived when composed
of structures that are Send, PythonDataStore is Send. Similarly
PythonMutableDataPack is Send without the explicit marker.

Reviewed By: xavierd

Differential Revision: D16352526

fbshipit-source-id: cbf9305fcc369716cafcc8adfdf35fd0a9e1fddc
2019-07-22 13:02:59 -07:00
Stefan Filip
414bba325e types: add String/str conversions for RepoPath & friends
Summary:
In cross language code it is useful to convert to a String before giving the
object up. Without this method we would have to copy the string before dropping
the current object. This method provides an efficient method for getting the
String from RepoPath objects.
str conversions show up in similar places places.

Reviewed By: quark-zju

Differential Revision: D16352530

fbshipit-source-id: 159f522d0c19287aed29b8cce99c1675b5d801fc
2019-07-22 13:02:59 -07:00
Stefan Filip
a1a75d5d73 tremanifest: rename tree holding variable to _tree
Summary: Minor rename

Reviewed By: quark-zju

Differential Revision: D16352529

fbshipit-source-id: 658f9843041d858e1471c07554b5715db011d17c
2019-07-22 13:02:58 -07:00
Jared Bosco
089e77642a dispatch: replace final fancyopts call with native rust
Summary: Remove final fancyopts call to have all python parsing being done through native rust codepath, as well as clean-up some deprecated flags that would be special handling.

Reviewed By: quark-zju

Differential Revision: D16156284

fbshipit-source-id: ec5ccaeb982c78426e12ff1d7342b4ea6653e98e
2019-07-20 01:06:35 -07:00
Jared Bosco
f56b7f2a62 cliparser: add more robust error handling for parsing failure cases
Summary: Add errors and map them to python exceptions to emulate the original python error handling behavior.

Reviewed By: quark-zju

Differential Revision: D16136571

fbshipit-source-id: eb999162cad040566e30b460f2b873efb05fc67a
2019-07-20 01:06:35 -07:00
Jared Bosco
939b3f5c57 cliparser: change FlagDefinition to use Cow to support static and dynamic lifetimes
Summary: To support 'static and dynamic lifetimes the FlagDefinition will have a Cow<'a, str> allowing the API to remain the same and still be used with strings passed from python.

Reviewed By: quark-zju

Differential Revision: D16136148

fbshipit-source-id: 670925549919a0965287b264704150f6ab638a18
2019-07-20 01:06:35 -07:00
Jared Bosco
93949f4629 dispatch: replace alias expansion and fancyopts parsing call with native rust
Summary:
Replace the second to last fancyopts call with pure rust code parsing and error handling.

Make slightly nicer help messages for ambiguous commands instead of just saying every possible command possible.

Reviewed By: quark-zju

Differential Revision: D16063049

fbshipit-source-id: bfd9e58649b1de2d3485069ce8d5646927bc77f4
2019-07-20 01:06:34 -07:00
Jared Bosco
252931cc73 bindings: expose native alias expansion to the python world
Summary: Expose the alias expansion native rust code to the Python code to be able to utilize the earlier alias expansion as well as custom Rust errors.

Reviewed By: quark-zju

Differential Revision: D16063040

fbshipit-source-id: 58cd8c9f16e07687545ebf19332b32fce8db278d
2019-07-20 01:06:34 -07:00
Jared Bosco
ae89e32851 cliparser: expose method to perform alias expansion on arguments
Summary:
Alias expansion originally occurs in mercurial through a chain of command handlers that point at their alias.

Now, aliases can be fully expanded early on into parsing to only resolve actual commands and not have to follow a chain of executions.

Reviewed By: quark-zju

Differential Revision: D16059122

fbshipit-source-id: cf28fba4a131ab29ceda87bc3e90d7a434e06625
2019-07-20 01:06:34 -07:00
Jared Bosco
137edc1814 rustshlex: add external source code for posix-style parsing
Summary: Very small library ( one file ) that allows for posix-style splitting.  The library was not vendored in third-party and therefore was just added to unblock as fast as possible.

Reviewed By: quark-zju

Differential Revision: D15911319

fbshipit-source-id: 2820d5beb5b3493a507f00f4b94e93b0405cf991
2019-07-20 01:06:34 -07:00
Jared Bosco
5f02e5cd5c dispatch: replace _parse's call to fancyopts with native code
Summary: Replacing another fancyopts call to be parsed by native Rust code.  This diff introduces slightly hacky feeling behavior in order to handle cycles and resolving aliases, but will be fixed in a follow-up diff where Rust will fully expand the aliases completely removing the need for this confusing alias resolving, chaining, and execution.

Reviewed By: quark-zju

Differential Revision: D15902758

fbshipit-source-id: 11d9a479989a23de09bf96f8020d2fded6c06351
2019-07-20 01:06:33 -07:00
Jared Bosco
3c30d27350 copytrace: remove copytrace flag from mutating global options table
Summary:
Copytrace modified the global definitions table which was making it very difficult to keep track of side-effects as the code was executed, as well as making it harder to replace the fancyopts calls with native Rust.

Since the copytrace behavior can be achieved through a configuration, it now will no longer modify the global definitions table, and will display the correct flag for a user to use in order to get this same behavior.

Reviewed By: quark-zju

Differential Revision: D15902449

fbshipit-source-id: 1c254162d56823e65085b7047bb37513f187b487
2019-07-20 01:06:33 -07:00
Jared Bosco
40ca990aac dispatch: replace a fancyopts call with native code
Summary: Replace the next usage of fancyopts in dispatch.py with native rust code, and ensure all tests pass with this replacement.

Reviewed By: quark-zju

Differential Revision: D15857997

fbshipit-source-id: ec8722bfe661731a14cb324e97846f861bd60bc8
2019-07-20 01:06:33 -07:00
Jared Bosco
ee1fcc3d2d bindings: create cliparser native binding to replace fancyopts early parsing
Summary:
The current Python parsing library fancyopts does an early parse for global flags, as well as slightly different logic for parsing out flags.

Switching this fancyopts call should allow fancyopts to be completely replaced by the native code path, and start using Rust parsing for hg.

This enforces command line arguments to be utf8. At Facebook, our `hg` wrapper already crashes if that is not the case. So it shouldn't cause new issues.

Reviewed By: quark-zju

Differential Revision: D15837079

fbshipit-source-id: 95634ebc814f8865960181f23282e5283068057c
2019-07-20 01:06:32 -07:00
Jared Bosco
95b0be5bdc dispatch: remove strictflags and related tests
Reviewed By: quark-zju

Differential Revision: D15833690

fbshipit-source-id: 3469d8def660b116c0f82c674a19f06b18f29211
2019-07-20 01:06:32 -07:00
Jared Bosco
9984985969 cliparser: port python global flags to rust definition
Summary: The global flags currently in Python should be ported to Rust definitions

Reviewed By: quark-zju

Differential Revision: D15782273

fbshipit-source-id: 1cfdb3bbab946d18dc3c899163286e30fa69c2c7
2019-07-20 01:06:32 -07:00
Jared Bosco
b41f91b70d cliparser: allow flags to be partial matched by prefix
Summary:
Flags should be able to be matched partially by a prefix-match.

If a given prefix returns more than one possible Flag, it is therefore ambiguous and should not choose for the user.

An exact match should always take precedence.  A partial match should be tried in the event the argument is not an exact match.

Reviewed By: quark-zju

Differential Revision: D15749936

fbshipit-source-id: 26e699616a1b3fa6871fb50cc6914f916701004c
2019-07-20 01:06:32 -07:00
Jared Bosco
ef61b6fae9 cliparser: add skeleton code for command and command builder
Summary: Initial design / skeleton code for command::Command and command::CommandBuilder which is unimplemented

Reviewed By: quark-zju

Differential Revision: D15588927

fbshipit-source-id: 193d3d2aaae98a252e13d5b1e655cf93d7ce801e
2019-07-20 01:06:31 -07:00
Xavier Deguillard
8f47102275 fixcorrupt: fix debugfixcorrupt on treeonly repos
Summary:
Treeonly repos doesn't use revlogs, and thus debugfixcorrupt shouldn't try to
fix them.

Reviewed By: quark-zju

Differential Revision: D16373142

fbshipit-source-id: 6517b9516358223bfb6c646fd22aaa99c26b0372
2019-07-19 20:15:36 -07:00
Jun Wu
32b2b5a44e bindings: replace PyResult<String> with PyResult<Bytes>
Summary: This makes sure bytes (Python 2 `str`) is returned as expected.

Reviewed By: xavierd

Differential Revision: D16399693

fbshipit-source-id: c299933cfb5ec57bb46fed1f30a9ad07aa043703
2019-07-19 19:57:18 -07:00