Commit Graph

824 Commits

Author SHA1 Message Date
Xavier Deguillard
cb64df7197 tests: @command expects str command names
Summary: This change all the tests to use strings as the command names.

Reviewed By: simpkins

Differential Revision: D19649479

fbshipit-source-id: 4b2cb9b3af73a7508ef359ee3c407d04a39c7893
2020-01-30 14:55:55 -08:00
Xavier Deguillard
96f5dc6652 localrepo: wwread returns bytes
Summary:
This is directly used in fctx.data() which expects bytes to be returned, encode
the return value of vfs.symlink.

Reviewed By: quark-zju

Differential Revision: D19647846

fbshipit-source-id: 966a5e79db6843c4bc6b802aeb90364e983cd834
2020-01-30 14:55:55 -08:00
Xavier Deguillard
b863a561a4 pytreestate: use PyPath where necessary
Summary:
A previous diff missed these PyPath. As a bonus, replace most of the PyObject
into proper Rust types.

Reviewed By: quark-zju

Differential Revision: D19646139

fbshipit-source-id: a3b55e5a802d8bbef86e9a9431f605bb4e284b32
2020-01-30 14:55:54 -08:00
Durham Goode
ca48fc2665 py3: fix resolving unknown binary nodes in changectx
Summary:
changectx takes an arbitrary identifier and tries to resolve it. If it
takes a binary node and the node is not known, the fallback looksup don't accept
bytes and throw an exception. Let's stop early and raise the RepoLookupError.

Reviewed By: quark-zju

Differential Revision: D19643834

fbshipit-source-id: cfee39e0487f908ebd4ff628b847e53dec13c66e
2020-01-30 14:39:41 -08:00
Durham Goode
cfdf7814f4 py3: fix pipe reads in sshpeer
Summary:
In Python 3, subprocess creates FileIO type pipes, which do one syscall
and are not guaranteed to return 'size' bytes. In Python 2 they're 'file' types
which are guaranteed to return 'size' bytes (via multiple syscalls).

So let's handle that case.

Reviewed By: quark-zju, xavierd

Differential Revision: D19648035

fbshipit-source-id: 7d04a67acc569a55d3a855fc8521f1b8261f4636
2020-01-30 13:55:21 -08:00
Durham Goode
22b2f5becc py3: fix encoding of remotenames at write time
Summary:
remotenames are stored as unicode in memory and bytes on disk. We need
to convert before we write in this case.

Reviewed By: quark-zju

Differential Revision: D19648032

fbshipit-source-id: 4b96cf05eefd66ca031436d9979b94da8e28b8dd
2020-01-30 13:55:20 -08:00
Durham Goode
5311ab2fbb py3: fix encoding of pushkeys
Summary: Let's decode pushkeys into utf8.

Reviewed By: quark-zju

Differential Revision: D19648033

fbshipit-source-id: 9deeeecd53e065df9b344f8edd0183dc7c1511c5
2020-01-30 13:55:20 -08:00
Stefan Filip
843f8a7f50 py3/cython: set the language_level to 3str
Summary:
Follow up to the previous diff setting language_level to 3str for
`edenscmnative/clindex.pyx`. Consistent behavior for all files.

Reviewed By: quark-zju

Differential Revision: D19634907

fbshipit-source-id: 779d6090b5660e109a37f9740299fb9623d8f34d
2020-01-30 12:52:06 -08:00
Mark Thomas
13b7a759a2 cpython-ext: add PyNone, a marker struct for functions that can only return None
Summary: Add `PyNone`.  This is a marker struct that indicates that a python function can only return `PyNone`.

Reviewed By: xavierd

Differential Revision: D19644338

fbshipit-source-id: f846b146237ebf7de996177494934fec662cde0f
2020-01-30 12:28:38 -08:00
Carolyn Busch
6b24e9a69e py3: commandserver write and read bytes
Summary: Make the command server write and read bytes. Add type annotations.

Reviewed By: quark-zju

Differential Revision: D19642862

fbshipit-source-id: 2dfcd8f0c2e6124cc6afe1e85c0f5275e7fe5b1f
2020-01-30 11:12:10 -08:00
Mark Thomas
6b8042662a cpython_ext: rename PyPath to PyPathBuf
Summary:
`PyPath` is the type that owns the data.  Rename it to `PyPathBuf` for analogy
with `PathBuf` and `RepoPathBuf`, and to allow us to introduce a reference type
named `PyPath`.

Reviewed By: xavierd

Differential Revision: D19643797

fbshipit-source-id: 56d80fea5677f7223e967b0723039d1763b26f68
2020-01-30 11:06:24 -08:00
Durham Goode
f170b964be py3: fix known wireproto command
Summary:
It was iterating over what it thought was a string, but in python3 it's
a byte string, which results in iterating over integers. So known always claimed
the server knew every commit.

Reviewed By: quark-zju

Differential Revision: D19644837

fbshipit-source-id: 4e84a33b4d3e436006f93a5d8e1bdabf12e7d120
2020-01-30 10:59:01 -08:00
Xavier Deguillard
f83e1d76b0 pypathmatcher: use PyPath when calling into Python
Summary: The Python code expect paths to be unicode in Python3, not bytes.

Reviewed By: markbt

Differential Revision: D19643701

fbshipit-source-id: fa98da2daccccf70ac047a69b2e2359270f7e3d0
2020-01-30 10:55:34 -08:00
Stefan Filip
45fa522e1c py3/fsmonitor: use next(iter) instead of iter.next()
Summary: iter.next() is gone in Python 3. next(iter) is the Python2 compatible version.

Reviewed By: quark-zju

Differential Revision: D19630455

fbshipit-source-id: e5d4f0a6cc0c6b0489916ca6d177a6831bd7d277
2020-01-30 10:28:02 -08:00
Adam Simpkins
dbaac4f0e7 py3: fix some Python 3 compatibility issues in sshpeer
Summary:
Fix `threadedstderr` to stop directly deriving from `threading.Thread`.
Deriving directly from `threading.Thread` is somewhat tricky.  Classes that
derive from `Thread` are only allowed to implement `__init__()` and `run()`.
This class also defined `close()`, which violates the `Thread` docs.  Changing
this to no longer derive from `Thread` fixes exceptions being thrown when
trying to read from the SSH stderr pipe in this thread.

This also updates `_writessherror()` to decode the stderr data before writing
it to the `ui` object.

Reviewed By: quark-zju

Differential Revision: D19634265

fbshipit-source-id: bddd120c7995f49d3bbebbb171384d5b15b779e9
2020-01-30 10:22:30 -08:00
Durham Goode
3ed7b6e3f2 py3: fix journal to work with utf8
Summary: Various fixes to make it stop crashing

Reviewed By: quark-zju

Differential Revision: D19643879

fbshipit-source-id: 9dd0bcef5369b769a8c968ea86de0e8aaf640650
2020-01-30 09:58:26 -08:00
Durham Goode
9aa1e8d7a3 py3: fix sparse utf8 logging
Summary:
Sparse logs the profile to scuba, but our logging data must be utf8.
Let's encode it first. We probably want to make this "replace" instead of
"strict" for the conversion once D19634265  lands.

Reviewed By: quark-zju

Differential Revision: D19643832

fbshipit-source-id: 550c67f194171ccdbd244bc79edfa1b999467824
2020-01-30 09:58:26 -08:00
Durham Goode
e3992926b7 py3: utf8 fixes for pull and wireproto
Summary: Moves pull closer to working by appropriately encoding and decoding.

Reviewed By: quark-zju

Differential Revision: D19643833

fbshipit-source-id: d311fb897d15594f2cb644b15038866ce506e011
2020-01-30 09:58:26 -08:00
Durham Goode
484a088417 py3: support caching in python3
Summary:
Revlog did a type check against str. In python 3 it accepts bytes
instead, so we need to support that.

Reviewed By: xavierd

Differential Revision: D19634882

fbshipit-source-id: 04ae908b4bfe8ed5de7a6d41cbe748dcb34b2540
2020-01-30 09:12:30 -08:00
Durham Goode
8b942a7166 remotenames: fix broken changelog contains check
Summary:
cl.__contains__ does a "0 < rev <= len(repo)" check, so passing nodes
will always fail. In python 3 this causes a type error, so let's fix this.

Reviewed By: quark-zju

Differential Revision: D19634880

fbshipit-source-id: 011ab2f0c07a16fa45734d360ac952273c4fefc2
2020-01-30 09:12:30 -08:00
Genevieve Helsel
247f686447 izip_longest to zip_longest
Summary:
izip_longest depreciated in python3 in favor of zip_longest, use zip_longest from future

source: https://python-future.org/compatible_idioms.html#itertools-filterfalse-zip-longest

Reviewed By: xavierd

Differential Revision: D19632101

fbshipit-source-id: 2eb3c125d2beaf66045dad304b900fe7928f2066
2020-01-30 08:50:41 -08:00
Xavier Deguillard
a14897afc8 clindex: remove one b
Summary: In `hg sl`, this raises an exception.

Reviewed By: singhsrb

Differential Revision: D19635386

fbshipit-source-id: 28482941bd61b5526993d6d16b2886c7edbe763f
2020-01-30 08:31:23 -08:00
Jun Wu
097503d073 py3: treestate: use IntoAnyhowResult
Summary: This preserves the Python errors across language boundaries.

Reviewed By: xavierd

Differential Revision: D19635355

fbshipit-source-id: 14ab8af38362182ca43951d4c560fa405687bf6e
2020-01-30 08:27:34 -08:00
Jun Wu
002d32e878 py3: treestate: use PyPath
Summary: Use PyPath for input/output and Python callbacks.

Reviewed By: xavierd

Differential Revision: D19635356

fbshipit-source-id: c2d8898ef903fda436277997a842bc837bb1bf2f
2020-01-30 08:27:33 -08:00
Jun Wu
c5dd6829c7 cpython-ext: add more utilities for PyPath
Summary:
Make the type easier to use. Namely, the treestate bindings want PyPath <->
bytes since treestate internally uses bytes.

Reviewed By: xavierd

Differential Revision: D19635357

fbshipit-source-id: 37d1889b5da1d7f3869bb7820de0219b87b71a8b
2020-01-30 08:27:33 -08:00
Jun Wu
cd04a56178 repo: avoid metalog when migrating files from sharedvfs to storevfs
Summary: Otherwise it can trigger metalog ProgrammingErrors.

Reviewed By: markbt

Differential Revision: D19624779

fbshipit-source-id: ba56a9aefa28d56a38ea8d966115647975613c91
2020-01-30 07:25:31 -08:00
Mateusz Kwapich
e4ee93b38b py: fixes for cloud sync
Summary: "hg cloud sync" succeeds when there's nothing to do

Reviewed By: quark-zju

Differential Revision: D19630431

fbshipit-source-id: e2c9859c7660e6283e6d319c634f1cd9053f8b3f
2020-01-29 18:25:41 -08:00
Mateusz Kwapich
892dcb3189 py3: make "hg cloud sl" work in py3
Summary: Some strings had to stay as unicode

Reviewed By: quark-zju

Differential Revision: D19628661

fbshipit-source-id: 6ea191c60a4ef2ed98b2e2d3ab90c875e6eea248
2020-01-29 18:25:41 -08:00
Adam Simpkins
b9790d38d6 py3: explicitly set the language_level to 3str
Summary:
This causes the code to always treat unprefixed string literals as the `str`:
i.e., bytes in Python 2 and Unicode in Python 3.  Without this setting our
unprefixed string literals were always being treated as unicode.

Reviewed By: lisroach

Differential Revision: D19634266

fbshipit-source-id: 770d7d03a96aa92a4e183819d34c72281c6c794b
2020-01-29 17:46:30 -08:00
Zeyi (Rice) Fan
b58fe35549 make run-tests.py py3 compatible
Summary: This diff makes `run-test.py` compatible with Python 3, so we can run this tests both in Python 3 and 2.

Differential Revision: D19611680

fbshipit-source-id: 97571cca7e1310e482ac8c03af73eb345ab825c1
2020-01-29 17:07:33 -08:00
Zeyi (Rice) Fan
e214ad8df9 testing utilities py3 compatibility
Summary:
We temporarily pin these test tools to run with Python 2 so the tests can pass no matter if it is run with 3 or 2.

This diff also makes the fake biggrep client to be Python 3 compatible.

Reviewed By: farnz

Differential Revision: D19611681

fbshipit-source-id: 0c980165934899902b152e27a0a444d6a6c9eb2f
2020-01-29 17:07:32 -08:00
Zeyi (Rice) Fan
05d306bfc3 make test-extension.t py3 compatible
Summary: because of the test `type(b"") != type("")` we actually have different behavior in Python 3 and Python 2. So we remove this test temporarily and we can add it back later.

Reviewed By: sfilipco

Differential Revision: D19611682

fbshipit-source-id: 7bca979f8ef871672ac9666286c8d5e54747e8b3
2020-01-29 17:07:32 -08:00
Zeyi (Rice) Fan
f0e50448b4 make the rest of the tests py3 compatible
Summary: This diff converts the rest of our tests to be compatible with Python 3. There are still a few tests are not fully compatible with Python 3 yet and I'll address these individually as they require major modifications.

Reviewed By: markbt

Differential Revision: D19608299

fbshipit-source-id: 3779dd0d4641ce8470598fa4909306317cf777d8
2020-01-29 17:07:32 -08:00
Stefan Filip
56c0adbc46 py3/match: convert filter result to list
Summary:
In Python3 the result of filter is a class that does not have len
implemented. In Python2 the result of filter is a list.

Reviewed By: quark-zju

Differential Revision: D19629139

fbshipit-source-id: a7a1d9d831d72b2aab8b7a920bf82295c4673603
2020-01-29 17:03:05 -08:00
Stefan Filip
49e90ce242 py3/pypathmatcher: return Str instead of Bytes
Summary:
Globs and patterns generally speaking. They should be typed as Str in
Python3 following how paths are typed.

Reviewed By: quark-zju

Differential Revision: D19628654

fbshipit-source-id: eeaede72f41258d6fe0f3cde26c434fe0b908b8b
2020-01-29 17:03:04 -08:00
Stefan Filip
f80b7bd6c4 py3/sparse: decode sparse profile as utf-8
Summary: File contents are read as bytes but content should be interpreted as utf-8.

Reviewed By: quark-zju

Differential Revision: D19628315

fbshipit-source-id: 523a201ddf57a2aabcd135156e71034c52b59adb
2020-01-29 17:03:04 -08:00
Stefan Filip
ac4aaab5a2 py3/util: treat authentication url as str
Summary: Python3 socke address is str.

Reviewed By: quark-zju

Differential Revision: D19628130

fbshipit-source-id: d6ce93d28588a9ab8829f5b95db30ecee040a160
2020-01-29 17:03:04 -08:00
Mateusz Kwapich
3cfb8a6a7c py3: node: get rid of extra encoding
Summary:
The extra call to encode makes everything a big slower, fortunately python
standard library provides functions that do the right thing.

Reviewed By: xavierd

Differential Revision: D19634034

fbshipit-source-id: 6bb0de26300c41667824422c016e8d4f0aee70d3
2020-01-29 16:48:41 -08:00
Xavier Deguillard
467b3095c1 eden_dirstate_map: encode the path
Summary: This enables `hg add` on EdenFS

Reviewed By: quark-zju

Differential Revision: D19631538

fbshipit-source-id: 1621ab77fef977c9364052bc522df18729face48
2020-01-29 16:45:46 -08:00
Durham Goode
13b96565e5 py3: convert parts of undo.py to specify utf-8
Summary: This makes hg pull make more progress.

Reviewed By: mitrandir77

Differential Revision: D19630124

fbshipit-source-id: 6e8d6bd441353ffbf87076addd2774d5e09dd24a
2020-01-29 16:41:54 -08:00
Jun Wu
c642e72b3e test-rebase-moved: fix up the test
Summary: I think my editor eat the spaces incorrectly. Fix it.

Reviewed By: xavierd

Differential Revision: D19630482

fbshipit-source-id: 3a8b94cdb269b045f678e07d0f21975effbaf7f9
2020-01-29 16:25:02 -08:00
Durham Goode
654ed5e4b1 remotenames: prefer picking default paths
Summary:
S194477 was caused by a config change adding a custom path to
repositories, where the path had the same url as default. This caused all the
remotebookmarks to be associated with the new path instead of default.

This is subtle and undesirable. Let's make it so that the default path takes
precedence.

I also deleted a hggit test because it was causing the entire test to be skipped
when dulwich wasn't available.

Reviewed By: quark-zju

Differential Revision: D19627711

fbshipit-source-id: 55b3ce94f18a26ccb9471c8e96a584d3087dbb82
2020-01-29 15:31:16 -08:00
Simon Farnsworth
c87bac85e1 py3: streamclone: make wireprotocol bytes
Summary: The wireprotocol for streaming clones should be explicitly bytes, encoding filenames as UTF-8 when needed. Make it so

Reviewed By: quark-zju, xavierd

Differential Revision: D19616097

fbshipit-source-id: 3513c801035c8d7ba950777aa119e1d1480013a5
2020-01-29 15:20:56 -08:00
Xavier Deguillard
73d424dd90 pyedenapi: use PyPath and String instead of PyBytes
Summary: This allows edenapi to be able to fetch trees on Python3.

Reviewed By: quark-zju

Differential Revision: D19629411

fbshipit-source-id: f6abd585d8d672854f7219ef8e3ab86d2f659118
2020-01-29 14:51:43 -08:00
Jun Wu
07a1c0a776 py3: connectionpool: fix some Python 3 issues
Summary: `im_self` is gone. Use `__self__` instead.

Reviewed By: xavierd

Differential Revision: D19630121

fbshipit-source-id: aa1af51e89cdffcfc63f3c18eeaf04d35425160e
2020-01-29 14:27:12 -08:00
Jun Wu
552c1dfabd py3: sshpeer: fix some encoding issues
Reviewed By: mitrandir77

Differential Revision: D19630030

fbshipit-source-id: b700187aeb412ac85e81532a525601454c45c426
2020-01-29 14:27:12 -08:00
Jun Wu
b01245ecc1 py3: commitcloud: fix some encoding issues
Reviewed By: mitrandir77, xavierd

Differential Revision: D19630028

fbshipit-source-id: 825bffa5c591f442ddecac4b9b8485dfe273b667
2020-01-29 14:27:12 -08:00
Jun Wu
50f80dff96 py3: clienttelemetry: fix some encoding issues
Reviewed By: xavierd

Differential Revision: D19630029

fbshipit-source-id: e9a9060a1e0cc2745dcb8588a083e8752384cf9f
2020-01-29 14:27:11 -08:00
Jun Wu
77a51f0e3b merge: allow specifying a customized rename script
Summary:
Provide an option to run an external script that calculates the rename
destination. This is especially helpful for cases where development happened in
another branch and *directory* and want to merge changes back.

For example, we recently had an experimental `scm3` branch that copies files
from `scm` to `scm3` to try out changes:

  o master <-- rebase destination
  |
  o (changes in scm/)
  :
  | o (changes that need to backport to scm/) <-- rebase source
  | |
  | o (changes in scm3/ that got implemented differently in scm/)
  | :
  | o (copy scm/ to scm3/ for experiments)
  |/
  o

This allows `--config experimental.rename-cmd='sed "s#scm3/#scm/#"'` to be
used for the above case.

While working on this feature, I noticed a bug that if the "copy" information
is recorded, rebase might lose changes. I added a test to track that bug.

Reviewed By: mitrandir77

Differential Revision: D19609868

fbshipit-source-id: 94462889d3ce9214b48fac793852d16aeacecc77
2020-01-29 14:27:11 -08:00
Jun Wu
0ba90481ab pycompat: add basestring, drop dep on "past"
Summary:
This removes the dependency of `future` (used by `import past`) which is not
available in our Windows tests.

Reviewed By: farnz

Differential Revision: D19626387

fbshipit-source-id: 2b13f08a9dafe77d1e7f6fc1a3cc48b3581681ca
2020-01-29 13:46:04 -08:00
Jun Wu
c18818de39 py3: pyindexes: use str for hex prefix matching
Summary: The hex prefix should be in str, not bytes.

Reviewed By: mitrandir77

Differential Revision: D19627762

fbshipit-source-id: dc9b054198dc61614518fec9bba69f789326ff8b
2020-01-29 13:43:09 -08:00
Jun Wu
282aaf155d py3: remotenames: fix compatibility issues
Reviewed By: mitrandir77

Differential Revision: D19627763

fbshipit-source-id: 3e1396a0686de81e3ed97ea5d3fb14c695bb3b8d
2020-01-29 13:43:09 -08:00
Xavier Deguillard
f2733fb294 pyrevisionstore: use PyPath for key name
Summary:
The names are really RepoPath, and thus let's use PyPath as a way to pass these
in.

Reviewed By: quark-zju, sfilipco

Differential Revision: D19625372

fbshipit-source-id: 4802030b91e6a065d3cb1905c770cad8a86da510
2020-01-29 13:32:46 -08:00
Xavier Deguillard
7c0c6feea1 commitcloud: fix max(None, None) in Python3
Summary:
In Python2, max allows None to be passed in, not in Python3, let's mimic the
Python2 behavior.

Reviewed By: sfilipco

Differential Revision: D19626654

fbshipit-source-id: f4fcaaa0516d898bc747d21735d2c30a4a9b41f4
2020-01-29 13:32:46 -08:00
Mateusz Kwapich
30c0e6482c proper bytes handling for remotefilelog
Summary:
I've inspected how the data flows from struct.unpack through the remotefilelog
and annotated few functions.

In one place I've used encodeutf8 function

Reviewed By: quark-zju

Differential Revision: D19626711

fbshipit-source-id: 77499c567b0cddab17d21b4f11b06f58bf3fcaef
2020-01-29 12:59:44 -08:00
Mateusz Kwapich
c1335eac46 add types module
Summary: Let's have a module that defines types ussed commonly in HG

Reviewed By: quark-zju

Differential Revision: D19625872

fbshipit-source-id: 817535d87acee6cb5647bc8ad25df21a95bec09c
2020-01-29 12:59:44 -08:00
Mateusz Kwapich
13aee5fa2a make all config function params strings
Summary: We want unicode configs.

Reviewed By: quark-zju

Differential Revision: D19612400

fbshipit-source-id: ec9f863c3203be7c956dcd97c735d74a11adce35
2020-01-29 12:09:37 -08:00
Stefan Filip
2dacfad0fc py3/make: rename hg3 to local3 and clean up fix
Summary:
Fixing up cleanup for hg3.
Seems to have a positive impact on regenerating the artifact after doing a rebase.

Reviewed By: quark-zju

Differential Revision: D19626379

fbshipit-source-id: 11caf0826ecb47440bcc532bfb7495771dc9159c
2020-01-29 11:56:38 -08:00
Xavier Deguillard
a80bcd779b phabstatus: make PeekaheadRevsetIter an Iterator
Summary:
Python2 and Python3 have different method names for an iterator, let's inherit
from Iterator so Python (both version) tells us what needs to be implemented.

Reviewed By: mitrandir77

Differential Revision: D19624294

fbshipit-source-id: b9c3d5ae796216b4bcd330c8be6f061e369e2614
2020-01-29 11:07:10 -08:00
Xavier Deguillard
20459835bb clindex: use str as input to vfs
Reviewed By: farnz

Differential Revision: D19624129

fbshipit-source-id: cbee989334d5cf114ef7b74c9143a992cee70370
2020-01-29 11:07:09 -08:00
Xavier Deguillard
776f0fabac pyedenapi: use String for the client constructor
Summary:
Both the repo name and the url come from the config, and thus Unicode values
are passed in in Python3. The String type handles the Python2/Python3
difference, so let's use it instead of PyBytes.

Reviewed By: DurhamG

Differential Revision: D19623202

fbshipit-source-id: 72c72f8338fbb6a2f3e88395f3587a5b59389711
2020-01-29 11:07:09 -08:00
Stefan Filip
11b85a8a33 py3/remotenames: convert to str after reading data
Summary: In Python3, nodes and bookmarks are str.

Reviewed By: DurhamG

Differential Revision: D19624130

fbshipit-source-id: edbb115c7e6fa1232c92ce76e8ee5aa46b3f23db
2020-01-29 11:00:26 -08:00
Genevieve Helsel
7c8128744c remove unsued urllib import
Summary: as title, removes unused import that had python 2/3 compat issues.

Reviewed By: sfilipco

Differential Revision: D19588354

fbshipit-source-id: 05d185d0b1a8fdd1bdb7eef0557362e95d85beb4
2020-01-29 10:45:57 -08:00
Jun Wu
4e7edef4b7 py3: use edenscmdeps3.zip for Python 3 deps
Summary:
Python 2 pyc files are incompatible with Python 3.  Use a different zip file.

This also makes it possible for Python 2 and 3 setup.py build to co-exist
(discovered by markbt).

Reviewed By: markbt

Differential Revision: D19617513

fbshipit-source-id: 47b2a01e905b0cb1ec68e7a7eec40868f09a2167
2020-01-29 10:40:16 -08:00
Durham Goode
38756127f2 hgevents: disable events during status
Summary:
There are times when status updates the dirstate to mark files as
clean or to resolve lookup files. In doing so it takes the wlock, which triggers
a state-enter event which causes external watchers to then run hg status, which
can then trigger another wlock event from status. This causes an infinite loop.

It's not clear why status is taking the wlock every time. It should be a
one-time thing. So more investigation is needed. For now though, we can prevent
this from happening by disabling hgevent signals while status has the wlock.

Reviewed By: quark-zju

Differential Revision: D19602370

fbshipit-source-id: 609316de1441d40be4f81236b8d00ce7dc30a130
2020-01-29 10:30:22 -08:00
Durham Goode
a4453150d0 py3: migrate unittestify to python3
Summary: Fixes some minor issues with unittestify to make it work in python 3.

Reviewed By: quark-zju

Differential Revision: D19613469

fbshipit-source-id: 00ae212d606fbef298eb5467e225341512db1290
2020-01-29 10:11:25 -08:00
Durham Goode
0913d16fb1 py3: support python3 for the builtin interpretter
Summary:
The builtin interpreter currently only responds to "python" as the
input arg. Let's also support python3.

Reviewed By: farnz

Differential Revision: D19613372

fbshipit-source-id: 5d2eed85c2d9546808c1661f12681b03f1edc40f
2020-01-29 10:11:24 -08:00
Jun Wu
8699e769e8 py3: minirst: fix some encoding issues
Summary: This makes `hg3 help` work.

Reviewed By: xavierd

Differential Revision: D19617512

fbshipit-source-id: 6f7621cff36e8f711fb5c7c21c805a1f82f80dfd
2020-01-29 09:00:19 -08:00
Xavier Deguillard
9db47445cd remotefilelog: switch some wireprotocol to use bytes
Summary:
String literals are unicode in Python3, which is not the intention for
over-the-wire streams. I've also added type-checking to get help from pyre to
do the right thing.

Reviewed By: quark-zju

Differential Revision: D19614060

fbshipit-source-id: 33f2b603012481159593426dbb3c18b1614b19f6
2020-01-29 08:50:03 -08:00
Jun Wu
7d8003f36e py3: manifest: drop pure lazymanifest
Summary:
The pure implementation has many encoding issues. Since our code does not build
in the pure Python mode, just drop the pure lazymanifest implementation for
simplicity.

Reviewed By: farnz

Differential Revision: D19613691

fbshipit-source-id: f3f95717bc77a75d4aa7b597a33f8eec745df134
2020-01-29 07:45:23 -08:00
Jun Wu
df746ae467 py3: wireproto: fix some encoding issue
Reviewed By: markbt

Differential Revision: D19615830

fbshipit-source-id: c16efaba5fca65898b24192910b2afd81f77a0b9
2020-01-29 07:45:23 -08:00
Jun Wu
fa999c515f py3: visibility: fix some encoding issue
Reviewed By: farnz

Differential Revision: D19615841

fbshipit-source-id: a88a2f9328dcef77a603d67f29435c6da4956d60
2020-01-29 07:45:23 -08:00
Jun Wu
c048b191ba py3: config: fix a minor encoding issue
Reviewed By: markbt

Differential Revision: D19615812

fbshipit-source-id: 997f4c4111dfecd1c157cd428b5044efb49883f8
2020-01-29 07:45:22 -08:00
Jun Wu
4da8312956 py3: transaction: fix some encoding issues
Summary: Use bytes to write files.

Reviewed By: markbt

Differential Revision: D19615839

fbshipit-source-id: 7881a7078fcfde154ba4b6e58c706064fd8e26b9
2020-01-29 07:45:22 -08:00
Jun Wu
dc01df5bb4 py3: templater: fix some encoding issues
Summary: The template language is in unicode.

Reviewed By: markbt

Differential Revision: D19615823

fbshipit-source-id: 30c6b295b4e4c999480c46aba8b4e8ffed5a4d64
2020-01-29 07:45:21 -08:00
Jun Wu
98a90ec4a0 py3: store: fix some encoding issues
Reviewed By: markbt

Differential Revision: D19615828

fbshipit-source-id: 8adf9b8d5fda818efd7d02d1ce05cb47c56a26da
2020-01-29 07:45:21 -08:00
Jun Wu
4b64b6ce9e py3: scmutil: fix some encoding issues
Reviewed By: farnz

Differential Revision: D19615819

fbshipit-source-id: e0137e15d14cf7fd84fdd015fe1c63dc35195459
2020-01-29 07:45:21 -08:00
Jun Wu
cc6dee9815 py3: revsetlang: fix some encoding issues
Summary: Use unicode in revset expressions.

Reviewed By: farnz

Differential Revision: D19615816

fbshipit-source-id: fc0251ca7108cf3baacd9ec154bc2760657754e0
2020-01-29 07:45:20 -08:00
Jun Wu
1e9cec215e py3: profiling: fix some encoding issues
Summary: Write bytes to files.

Reviewed By: farnz

Differential Revision: D19615833

fbshipit-source-id: 0f5a0df493715fad5723b98f4b4246fc1dec53be
2020-01-29 07:45:20 -08:00
Jun Wu
9cafa05c43 py3: phases: fix some encoding issues
Reviewed By: farnz

Differential Revision: D19615840

fbshipit-source-id: ad9694e736b24254172f732f988230b987597e0e
2020-01-29 07:45:19 -08:00
Jun Wu
573b64085c py3: peer: fix a minor encoding issues
Summary: The "function name" is unicode in Python 3. Put unicode as the dict keys.

Reviewed By: farnz

Differential Revision: D19615814

fbshipit-source-id: 3508f59d11e4f8ab6345c80c6bdfaff150a63d76
2020-01-29 07:45:19 -08:00
Jun Wu
47d36736ae py3: obsolete: fix some encoding issues
Summary: Use bytes to write files.

Reviewed By: farnz

Differential Revision: D19615831

fbshipit-source-id: 0868b9272f3570547326f63e34b9b6a489788464
2020-01-29 07:45:19 -08:00
Jun Wu
330306cfde py3: blackbox: fix some encoding issues
Summary: Use bytes to write files.

Reviewed By: farnz

Differential Revision: D19615824

fbshipit-source-id: fa7b87dea80960dcc7b6586b3d50baef6228db3a
2020-01-29 07:45:18 -08:00
Jun Wu
bfa9485f90 py3: merge: fix some encoding issues
Summary: This affects mergestate and updatestate.

Reviewed By: farnz

Differential Revision: D19615836

fbshipit-source-id: bc549c71a5ba5ba9d2069deb1e77cb580f210ba2
2020-01-29 07:45:18 -08:00
Jun Wu
751d53d4e6 py3: localrepo: fix some encoding issues
Reviewed By: farnz

Differential Revision: D19615825

fbshipit-source-id: 1801652ed93ac9218a66af7f23f5bce9eab3ab10
2020-01-29 07:45:17 -08:00
Jun Wu
456a0ad5e1 py3: json: fix some encoding issues
Reviewed By: farnz

Differential Revision: D19615829

fbshipit-source-id: 3d251b8febedbdfbefe5b1e59ea026de54503663
2020-01-29 07:45:17 -08:00
Jun Wu
29711c1cf0 py3: hg: fix some encoding issues
Reviewed By: farnz

Differential Revision: D19615834

fbshipit-source-id: 8c8aaaf6bca688476662f29375b8c5484ff0765a
2020-01-29 07:45:17 -08:00
Jun Wu
3dea327fba py3: eden_dirstate: use unicode paths on Python 3
Reviewed By: farnz

Differential Revision: D19615837

fbshipit-source-id: 97e514fbc5ab3b077fd9b6c56cd8b5d7b6bc7856
2020-01-29 07:45:16 -08:00
Jun Wu
60733ef7b2 py3: dirstate: fix some encoding issues
Summary: Some assertions were added to make sure paths are unicode.

Reviewed By: farnz

Differential Revision: D19615813

fbshipit-source-id: 7d2135c0ac424fc46edd9d660b072a2a95be7f82
2020-01-29 07:45:16 -08:00
Jun Wu
a3d23bb8d7 py3: context: fix some encoding issues
Reviewed By: farnz

Differential Revision: D19615827

fbshipit-source-id: 79cfab528eeb00048b00be75fa776737b49a53d7
2020-01-29 07:45:15 -08:00
Jun Wu
0ad9daf67f py3: config: fix some encoding issues
Reviewed By: farnz

Differential Revision: D19615815

fbshipit-source-id: 2ae650dc321c776b50f49070b1c8f18dd5a2149f
2020-01-29 07:45:15 -08:00
Jun Wu
b084775bd8 py3: changegroup: fix some encoding issues
Summary:
Changegroup should be using bytes mostly except when it tries to access
files via `repo.file(path)`. The `path` should be unicode on Python 3.
Make it happen. Some assertions are added to help capture issues.

Reviewed By: farnz

Differential Revision: D19615835

fbshipit-source-id: b35d13152ecb835092c7870e88383f484dc5f868
2020-01-29 07:45:15 -08:00
Jun Wu
ef1dd7f142 py3: bundle2: use some bytes
Summary:
The bundle2 layer uses bytes since it's part of the wireprotocol.
Some assertions were added to help detecting issues.

Reviewed By: farnz

Differential Revision: D19615832

fbshipit-source-id: 73ea657b501c7539b17862165c7a5781884013a2
2020-01-29 07:45:14 -08:00
Jun Wu
19c31c1690 py3: remotefilelog: add some type assertions
Summary: This helps detecting type errors at runtime.

Reviewed By: farnz

Differential Revision: D19615821

fbshipit-source-id: 0f45e1271aff50c29858c823366a0a54c89a3903
2020-01-29 07:45:14 -08:00
Jun Wu
e2b48e1122 py3: manifest: use bytes
Summary: The layer interacting with raw manifest text should use bytes.

Reviewed By: farnz

Differential Revision: D19613694

fbshipit-source-id: c5a12235eb50acd90e409c1059a3f9f11665f9f3
2020-01-29 07:45:14 -08:00
Jun Wu
fe28d31b8f py3: lock: drop an unnecessary encoding conversion
Summary: Use native unicode hostname without converting it to bytes.

Reviewed By: farnz

Differential Revision: D19613692

fbshipit-source-id: 0bd7e37890b96566143fb0501c3fa310270844d3
2020-01-29 07:45:13 -08:00
Jun Wu
3add434b57 py3: changelog: use bytes in low-level interfaces
Summary:
Like revlog, the changelog layer should use bytes for parsing its raw content.
`str` is still used for users, messages, etc.

Reviewed By: markbt

Differential Revision: D19613695

fbshipit-source-id: 4f6758f9fca0788bc89ad5c14d9103f66d19b7f2
2020-01-29 07:45:13 -08:00
Jun Wu
63ea4b2ab0 py3: bookmarks: fix encoding
Summary:
Bookmarks use unicode by default. Only need to use bytes when interacting with
vfs.

Reviewed By: farnz

Differential Revision: D19613689

fbshipit-source-id: bbe80e552cf17a023e5905f849a273b4b4299dc5
2020-01-29 07:45:13 -08:00
Jun Wu
47f4f75aba py3: diff: fix patchtext encoding
Summary: The patchtext is about a binary patch. It should use bytes.

Reviewed By: farnz

Differential Revision: D19613688

fbshipit-source-id: 20bddeb17f520f625396b23b5dcead7c7bbd3ede
2020-01-29 07:45:12 -08:00
Jun Wu
4ef1f31745 py3: util: fix some encoding issues
Summary:
Make sure file content and wireproto buffers are using bytes.
Added a utf8 helper function.

Reviewed By: farnz

Differential Revision: D19613686

fbshipit-source-id: 5d67ce48246dc8c3cc47041f461e454c46bc67f9
2020-01-29 07:45:12 -08:00
Jun Wu
32eec0fc37 py3: lock: fix encoding
Reviewed By: farnz

Differential Revision: D19613690

fbshipit-source-id: 617220ac09df7ccbbd1a3bee627b1a796199a33a
2020-01-29 07:45:12 -08:00
Jun Wu
2a28611d8a py3: pycompat: make fsencode/fsdecode do nothing
Summary:
We want to assume almost all paths are unicode and drop support for non-unicode
paths. `os.fsencode` returns bytes that breaks the assumption.

Change fsencode and fsdecode to do nothing so path types remain unicode.

Most use-cases are related to `__file__`, which is already unicode on Python 3.
`posix._checklink` uses it. But it seems making it no-op is okay.

Reviewed By: farnz

Differential Revision: D19613693

fbshipit-source-id: 587dbeef68b23beeaf6bc8861c71ea27b3c512d7
2020-01-29 07:45:11 -08:00
Jun Wu
0e6add4877 py3: repair: fix encoding for SHA1 calculation
Reviewed By: farnz

Differential Revision: D19613687

fbshipit-source-id: e8dbbcc8c1151dd2defe63b1c67df0a3dc598c90
2020-01-29 07:45:11 -08:00
Jun Wu
21b2a06c6f py3: vfs: add helper functions to get utf-8 content directly
Summary: Those will simplify some use-cases.

Reviewed By: farnz

Differential Revision: D19613121

fbshipit-source-id: 85ac4fa0c86afb84e0c9fb78950d3da2c93aa967
2020-01-29 07:45:10 -08:00
Jun Wu
535b35f427 py3: treestate: fix encoding
Summary: The raw blobs are in bytes, while paths and other things are in native string.

Reviewed By: markbt

Differential Revision: D19613122

fbshipit-source-id: e865168b5bc839e85ef843d771d8a6044e35a03a
2020-01-29 07:45:10 -08:00
Jun Wu
a986833f9e py3: ui.edit: fix encoding
Summary:
The commit message should be in native string. The raw file content is bytes.
Add translation for that.

Reviewed By: xavierd

Differential Revision: D19613117

fbshipit-source-id: 1b4145f0de75e48df3416cd7c52c09df1e6c4a07
2020-01-29 07:45:10 -08:00
Jun Wu
96309864ff py3: remotefilelog: use bytes
Summary: Like filelog, the remotefilelog layer should use bytes.

Reviewed By: xavierd

Differential Revision: D19613119

fbshipit-source-id: c8221ac21dc255ad0a837b2bb30fa9966df95390
2020-01-29 07:45:09 -08:00
Jun Wu
0c6dc8d8c9 py3: filelog: use bytes
Summary: Like revlog, the filelog layer should use bytes.

Reviewed By: sfilipco

Differential Revision: D19613116

fbshipit-source-id: 6b07d05a6960c2d1c8b4ebe3001632fd8024c5af
2020-01-29 07:45:09 -08:00
Jun Wu
df36f5c459 py3: revlog: use bytes
Summary: The revlog layer should use bytes.

Reviewed By: sfilipco

Differential Revision: D19613120

fbshipit-source-id: 09f0028cbf810b52694643c79c91997d434cdaf5
2020-01-29 07:45:08 -08:00
Jun Wu
cb289128dc py3: pycompat: add helpers for encode("utf-8")
Summary:
On Python 2 we want bytes.encode("utf-8") to be no-op, while on Python 3 we
want it to return a real `str`. This helper function does that.

Reviewed By: sfilipco

Differential Revision: D19613118

fbshipit-source-id: 1f0fc31bf8aee28cc20694da52b10b12dbdd9000
2020-01-29 07:45:08 -08:00
Jun Wu
9b4d48bcea py3: lz4revlog: fix encoding issues
Summary: The lz4revlog extension needs binary types.

Reviewed By: sfilipco

Differential Revision: D19612879

fbshipit-source-id: c66d3953883d9b06fc9867511843c36d44fe2245
2020-01-29 07:45:08 -08:00
Mark Thomas
1e63f205f4 rust-cpython: allow compilation for both py2 and py3
Summary:
Set up the `cpython-ext` and `hgcommands` libraries so that they can compile
against py2 and py3 versions of rust-cpython.  Make py2 the default so
that cargo test still works.

Reviewed By: singhsrb

Differential Revision: D19615656

fbshipit-source-id: 3403e7077deb3c0a9dfe0e3b7d4f4ad1da73bba3
2020-01-28 20:17:20 -08:00
Genevieve Helsel
ce2bf270b3 fix_map
Summary: Python 3 map call doesn't return list, so wrap in list.

Reviewed By: mitrandir77, xavierd

Differential Revision: D19588352

fbshipit-source-id: 8556f4783f9bedb8a74ec0e99ff226ff96f387a9
2020-01-28 18:24:56 -08:00
Stefan Filip
cdc1efc872 py3: open stdout log file in text mode
Summary: In both py2 and py3 stdout is text mode.

Reviewed By: simpkins

Differential Revision: D19613397

fbshipit-source-id: fe01363836cb5956c7e6d83a0f81488649c262a9
2020-01-28 17:17:45 -08:00
Xavier Deguillard
40d31b3060 traceprof: remove some b
Summary:
The config functions takes Str (ie: bytes in Python2, Unicode in Python3), the
b is uncessary.

Reviewed By: sfilipco

Differential Revision: D19611274

fbshipit-source-id: 0807bd6aed1b5bd0baa0b78d3dfcbb9c46aeff35
2020-01-28 17:07:36 -08:00
Xavier Deguillard
b1e73324ea clindex: remove some b
Summary: All of these should just plain strings.

Reviewed By: sfilipco

Differential Revision: D19611275

fbshipit-source-id: 1d16c10df6f67cee9dadd7480c40950c0abae926
2020-01-28 17:07:36 -08:00
Zeyi (Rice) Fan
09d7bb259c make part of the tests Python 3 compatible
Summary:
Most of these tests use `file` instead of `open` for opening files, which is removed in Python 3. And when we open files in byte mode, Python disallows writing `str` into such files, so we add `b` prefix to write actual bytes instead.

There are some other minor problems like reading from `sys.stdin` gives you `str` while expecting `bytes`. The solution is to use `sys.stdin.buffer` instead. However, Python 2 does not have `sys.stdin.buffer` so we have to do a version check to provide compatibility.

Reviewed By: quark-zju

Differential Revision: D19606357

fbshipit-source-id: e9177134294f03af8b581230a57902c3435b319a
2020-01-28 16:42:11 -08:00
Adam Simpkins
c6b023ce2a py3: remove an incorrect encode() call in python 3
Summary:
We want all config values to always be `str` types, so remove this explicit
`encode()` call that was trying to convert the value to `bytes`.

Reviewed By: sfilipco

Differential Revision: D19610590

fbshipit-source-id: a72f0755cdb12cae13fb9d58f529433d508d46e9
2020-01-28 16:33:38 -08:00
Mark Thomas
ffa61b0945 hggit: allow empty repos when dulwich returns empty FetchPackResult
Summary:
For empty repos, some versions of dulwich return a non-`None` `FetchPackResult`
where `bool(result)` throws an exception.   Handle this by explicitly checking
for `result.refs` being `None`, too.

Reviewed By: sfilipco

Differential Revision: D19611221

fbshipit-source-id: 094e3ec667a484a150eb75596684a120528159e9
2020-01-28 16:27:29 -08:00
Mark Thomas
82715fd2ea py3: iter{keys,values,items} -> pycompat.iter{keys,values,items}
Reviewed By: quark-zju

Differential Revision: D19608323

fbshipit-source-id: dd186ef16d6422a56af41fcaa850d9838ae9a240
2020-01-28 16:27:28 -08:00
Genevieve Helsel
ec51946a13 add pyre ignore for dbm
Summary: this pyre-ignore got lost in translation and caused pyre eden to fail

Reviewed By: sfilipco, mitrandir77, xavierd

Differential Revision: D19610997

fbshipit-source-id: b21faa36ee6b45585a774402d081bb7a3b8731b4
2020-01-28 16:16:40 -08:00
Adam Simpkins
ad957e7803 py3: update Rust hgcommands code to pass argv to python as Str
Summary:
Update the Rust hgcommands code to pass the command line arguments into the
Python logic as `Str` types, so that this will be Unicode `str` objects when
using Python 3.

Reviewed By: xavierd

Differential Revision: D19596739

fbshipit-source-id: 7cdfd44a1c4ce8b0f86d20b634d9b27eab822b2d
2020-01-28 15:58:37 -08:00
Adam Simpkins
2165181562 py3: ensure all command names are str
Summary:
Several of the debug commands explicitly defined their command names as bytes.
Remove the leading `b` prefix so that all command names are uniformly `str`
data types.

Reviewed By: DurhamG, sfilipco

Differential Revision: D19596738

fbshipit-source-id: 3851a32c38009968dfa338877b9ae03015dfca61
2020-01-28 15:56:20 -08:00
Genevieve Helsel
de029e9be7 unicode to pycompat.unicode
Summary: Use pycompat unicode instead of python 2 built in unicode to check type.

Reviewed By: mitrandir77

Differential Revision: D19589969

fbshipit-source-id: 3e3cf2c5cb6fd90cca9c8f803e7bd45f262f96e5
2020-01-28 15:45:17 -08:00
Xavier Deguillard
8697be6910 minirst: use string regexp
Summary: The input of minirst will be unicode, therefore, remove the b prefix.

Reviewed By: sfilipco

Differential Revision: D19610028

fbshipit-source-id: d3b1ebfeb1ba5a30afe250ce5713857dd9ebc359
2020-01-28 15:40:56 -08:00
Xavier Deguillard
ff536a2c24 remotenames: use Mapping instead of DictMixin
Summary: The later is deprecated since python 2.6, and the former works in python3

Reviewed By: simpkins

Differential Revision: D19608560

fbshipit-source-id: c9e45bbc51f0e20416c2640dfc25e4cac4934152
2020-01-28 15:40:56 -08:00
Mateusz Kwapich
7f606baac5 store: remove extra b's
Summary: The fallback filename handling can be utf8 under py3

Reviewed By: sfilipco

Differential Revision: D19610624

fbshipit-source-id: a86ed958edca70597ff5a7c67cbc746c1e325518
2020-01-28 15:37:09 -08:00
Xavier Deguillard
08f8dbab46 pycliparser: use Str instead of Bytes
Reviewed By: quark-zju

Differential Revision: D19607631

fbshipit-source-id: 472b660c9dcd2d044567ce091bf7fa0fdf157121
2020-01-28 15:02:21 -08:00
Mateusz Kwapich
b67b923a59 fbscmquery: fix the urlencode import
Reviewed By: xavierd

Differential Revision: D19609731

fbshipit-source-id: fc666f6238ca62633e0e45351be45f6be78dc012
2020-01-28 14:59:57 -08:00
Mateusz Kwapich
1f10ba1eff copytrace: rename anydbm to dbm
Summary: The module name was changed.

Reviewed By: sfilipco, xavierd

Differential Revision: D19609573

fbshipit-source-id: 7ac812c339b363dab52c942581ee10ca3e607061
2020-01-28 14:59:56 -08:00
Carolyn Busch
d78a41fcfd add past.builtins import basestring
Summary:
Adds from past.builtins import basestring if there is a basestring

libfuturize.fixes.fix_basestring

Reviewed By: quark-zju

Differential Revision: D19588358

fbshipit-source-id: ed2a48372bda3768183e09cd0758ad12471b0cf7
2020-01-28 14:43:19 -08:00
Adam Simpkins
22c3c06c9e py3: some additional string encoding and type issues
Summary:
Deal with a few more minor string/bytes issues in Python 3, and also drop a
reference to the `long` type, which no longer exists in Python 3.

Reviewed By: xavierd

Differential Revision: D19596744

fbshipit-source-id: d7dca9306934aa6bf8c17bada6f6c1a40130027f
2020-01-28 14:32:31 -08:00
Adam Simpkins
68aa9c142e py3: update shellquote() to operate on strings
Summary: Update this function to work on `str` data types in both Python 2 and 3.

Reviewed By: xavierd

Differential Revision: D19596742

fbshipit-source-id: 6140d2a7644dbf60c9a22c4e482757466648d5a1
2020-01-28 14:32:30 -08:00
Adam Simpkins
353c3ae62a py3: drop the use of re.LOCALE for some regexes
Summary:
Drop the use of `re.LOCALE`.  In Python 3.6+ this flag can only be used with
bytes patterns.  None of these regexes actually use the \w, \W, \b, or \B
patterns that would be affected by this setting.

Reviewed By: xavierd

Differential Revision: D19596747

fbshipit-source-id: e26ed5ac6d0378e1830c67ae7c018ccb56b7609a
2020-01-28 14:32:30 -08:00
Genevieve Helsel
e0fac9b23c import print_function to use print as function in python2
Summary: in python3, print becomes a function, import print_fuction to use in python 2.(ran fix_print_with_import)

Reviewed By: lisroach, quark-zju, sfilipco

Differential Revision: D19588356

fbshipit-source-id: b7cf519058ae3909ba7813618719bbd09800e73a
2020-01-28 13:57:59 -08:00
Jun Wu
ed3a2b2247 cpython-ext: add missed types dep
Summary: This is incorrectly removed due to a bad rebase / merge.

Reviewed By: DurhamG

Differential Revision: D19607801

fbshipit-source-id: a6ee7a3f184ff1882eb1f1513f7fed74a7108727
2020-01-28 13:50:14 -08:00
Jun Wu
a6d94a7680 py3: i18n: prefer unicode on Python3
Summary:
See D17098991 for our encoding strategy.

This diff changes the i18n module to prefer unicode on Python 3.

Reviewed By: xavierd

Differential Revision: D19594826

fbshipit-source-id: 011110f3cad7aa89b729f5a86b66168fcd5a7e36
2020-01-28 13:42:22 -08:00
Jun Wu
8703970cea py3: update Cargo.toml to make py3 buildable
Summary:
This makes `make hg3` work. It requires cleaning up the `build` directory when
switching between py2 and py3 build, which will be fixed later.

Reviewed By: DurhamG

Differential Revision: D19604824

fbshipit-source-id: 060ff313420126a5dba935c4451b45dc9af45f13
2020-01-28 13:39:38 -08:00
Xavier Deguillard
fcafda075f pyworkingcopy: use PyPath instead of PyByte
Reviewed By: DurhamG

Differential Revision: D19601926

fbshipit-source-id: 847e1762062c2b12627dd07a50a6d77ada282bdd
2020-01-28 12:40:49 -08:00
Xavier Deguillard
d087e39a34 pypathmatcher: use PyPath instead of PyByte
Reviewed By: DurhamG

Differential Revision: D19592136

fbshipit-source-id: 5db6ca629cd920d52ffbf7f10963c44c8f7b203d
2020-01-28 12:40:48 -08:00
Adam Simpkins
d71b2a2982 py3: drop an unnecessary encode() in ui.traceback()
Summary:
As of D17098991, all the data output data we write should be Unicode, so this
explicit call to `encode()` if we are running Python 3 should no longer be
used.

Reviewed By: xavierd

Differential Revision: D19596741

fbshipit-source-id: b05d94dcb9859c6ceff4bfae130550101ad3f47d
2020-01-28 12:28:41 -08:00
Adam Simpkins
5ae8afd122 py3: update ui.setconfig() to work in Python 3
Summary:
The `ui.setconfig()` code attempts to escape lists, and checks for them by
looking if the config value has an `__iter__` attribute.  However, in Python 3
both `str` and `bytes` types have an `__iter__` method, so the setconfig
code incorrectly treated them as a list.

This re-orders the code so that `str` types are explicitly handled before the
check for iterable types.

Reviewed By: xavierd

Differential Revision: D19596740

fbshipit-source-id: 3c52d3e8d5b68fd2d561651bd1858889cdbe0ccb
2020-01-28 12:25:40 -08:00
Adam Simpkins
f0e087e898 py3: update pyconfigparser bindings to return results as Str
Summary:
Update the pyconfigparser library so that it returns config values as
`unicode` in Python 3.

Most of the Python code already accesses config items with `str` keys,
but a few `.pyx` files were explicitly using bytes and needed to be updated to
use string keys.

Reviewed By: xavierd

Differential Revision: D19596746

fbshipit-source-id: fbeb363639b2256c57311bea996c4b371e068ae8
2020-01-28 12:25:39 -08:00
Adam Simpkins
563ff09e85 py3: update configparser to return errors as Str
Summary:
Update the configparser APIs to return errors as a list of `Str` rather than
`Bytes`.  The Python code already expects the errors as `str`, so this makes
the code do the right thing with Python 3.

Reviewed By: xavierd

Differential Revision: D19596745

fbshipit-source-id: 027c33d76b3a3b800de34366232285591841a900
2020-01-28 12:25:39 -08:00
Adam Simpkins
beff6fdea7 py3: add additional from() conversion methods for Str
Summary: Add methods to convert to a `Str` object from `String` and from `Vec[u8]`

Reviewed By: xavierd

Differential Revision: D19596743

fbshipit-source-id: 6499f7f1b8329f4d14ce8179a41ed46982a85c8e
2020-01-28 12:25:39 -08:00
Zeyi (Rice) Fan
8eca037626 make test-commit-interactive.t py3 compatible
Summary: This test tests Mercurial printing the correct encoding when the system is set to Japanese locale (CP932). The escape.py used in this test isn't Python 3 compatible due to byte-related operations. This diff makes it Python 3 compatible.

Reviewed By: quark-zju

Differential Revision: D19592535

fbshipit-source-id: 114dfcb004805045163be4a636ac703b4d24ea32
2020-01-28 11:24:25 -08:00
Zeyi (Rice) Fan
26859e4907 make test-check-execute.t py3 compatible
Summary: Apparently we tried to open an image file directly with `open()`. This does not work in Python 3 as it tries to read the file as string instead of bytes.

Reviewed By: farnz

Differential Revision: D19589327

fbshipit-source-id: 9c6388b704860f1f5dec199562741ac6e191d928
2020-01-28 11:24:24 -08:00
Zeyi (Rice) Fan
9015eda650 make test-help.t py3 compatible
Reviewed By: quark-zju

Differential Revision: D19589104

fbshipit-source-id: 0db455b2939643c6aa2d7b1ba68ad4bae376bbf4
2020-01-28 11:24:24 -08:00
Zeyi (Rice) Fan
76e325e30d make f py3 compatible
Reviewed By: sfilipco

Differential Revision: D19588237

fbshipit-source-id: 12130d67e59d8d488afb244c89673bd6c0075181
2020-01-28 11:24:24 -08:00
Zeyi (Rice) Fan
c0772d015f make printenv.py py3 compatible
Reviewed By: sfilipco

Differential Revision: D19588063

fbshipit-source-id: 78f2b433bb6a065e63241912b4cc0cf78bcbdd13
2020-01-28 11:24:23 -08:00
Zeyi (Rice) Fan
a334e11c88 make test-clone.t py3 compatible
Reviewed By: sfilipco, singhsrb

Differential Revision: D19587714

fbshipit-source-id: 574e6b7236b0954b621a8e15a36f2f1e50243418
2020-01-28 11:24:23 -08:00
Zeyi (Rice) Fan
f2690f21eb make dummyssh py3 compatible
Summary: This makes the `dummyssh` script Python 3 compatible so we can run it with Python 3 in tests

Reviewed By: sfilipco

Differential Revision: D19587078

fbshipit-source-id: 134b01f4d5e968cd9600c9358c7230a56e11f163
2020-01-28 11:24:23 -08:00
Mark Thomas
4fe02f3607 bindings: update to rust-cpython 0.4
Summary: Update to the new version of rust-cpython.  This supports `list.append`, so make use of it.

Reviewed By: xavierd

Differential Revision: D19590905

fbshipit-source-id: 03609d4f698ae8e4380e82b8144caaa205b4c2d4
2020-01-28 10:46:33 -08:00
Mateusz Kwapich
455fdf8461 py3/cext: be PY_SSIZE_T_CLEAN
Summary:
This is optional for Python 3.6. But Python 3.8 shows a warning at runtime
without the fix.

Reviewed By: quark-zju

Differential Revision: D19581191

fbshipit-source-id: a47453f2fab92faeb6597900b9c38d1239d5ed79
2020-01-28 10:46:33 -08:00
Genevieve Helsel
f0a27646b5 change maxint -> maxsize
Summary: ran print_function, maxint -> maxsize (maxint depreciated)

Reviewed By: lisroach, quark-zju

Differential Revision: D19588359

fbshipit-source-id: fc66b388a1775553a4b8e30a56fd1483800693d6
2020-01-28 10:37:53 -08:00
Genevieve Helsel
514f13056d bool/nonzero aliasing
Summary: most instances of this `__non_zero__` ->` __bool__` were already aliased upstream, just updates some cases in which it was not.

Reviewed By: lisroach, quark-zju, sfilipco

Differential Revision: D19591210

fbshipit-source-id: 624fcfbb35b463f2660a47ca50d28d3dcd9e2cb0
2020-01-28 10:37:53 -08:00
Jun Wu
4561bd91ed py3/cext: pathencode: use unicode paths
Reviewed By: quark-zju

Differential Revision: D19581186

fbshipit-source-id: c0ff8405b73e6348852dc532058b604f04d19eec
2020-01-28 10:35:29 -08:00
Jun Wu
86a147145a py3/cext: osutil: use unicode for paths
Reviewed By: quark-zju

Differential Revision: D19581187

fbshipit-source-id: 00b05c06bc689aa62fed6ad639cf3b59be43d6ff
2020-01-28 10:35:29 -08:00
Jun Wu
d2557dcef3 py3/cext: lazymanifest: use unicode for paths and flags
Summary: Use UTF8 paths and flags. Node, and the manifest text remain bytes.

Reviewed By: xavierd

Differential Revision: D19581190

fbshipit-source-id: f6f3c561bc5d3663f20513654bea65066a669d34
2020-01-28 10:35:29 -08:00
Jun Wu
f3d0077921 py3/cext: revlog: use bytes
Summary:
`Py_BuildValue("s#")` returns str.  Revlog uses it for nodes, which must be
bytes.  Change "s#" to "y#" for bytes.

Reviewed By: xavierd

Differential Revision: D19581197

fbshipit-source-id: b925ea8f26abff49ec4feafe579f00f68cee330e
2020-01-28 10:35:28 -08:00
Jun Wu
2e7d124fd2 py3: setup3.py and make hg3
Summary:
Make `make hg3` run the python3 version of the build.  There are some
Cargo.toml thing to fix. But this is a start.

Reviewed By: xavierd

Differential Revision: D19581181

fbshipit-source-id: 711a02ec0c8384f2ca3e61ead66db1e89a30837c
2020-01-28 10:29:15 -08:00
Jun Wu
917605b7d8 py3: remove b prefix
Summary:
Remove "b" prefix so strings become unicode.

The drawdag module was not affected by the upstream source code transformer [1]
so they added the b prefix. But drawdag is high-level enough that bytes
shouldn't be used.

[1]: https://www.mercurial-scm.org/repo/hg/rev/1c22400db72d

Reviewed By: simpkins

Differential Revision: D19581175

fbshipit-source-id: 7ae5f37af1d5145d8bac4ea82ac5ad856d83ae19
2020-01-28 10:29:15 -08:00
Jun Wu
35250bd8fd py3: miscellaneous fixes
Summary: Some miscellaneous fixes, ex. `map(x)` -> `list(map(x))`.

Reviewed By: farnz

Differential Revision: D19581171

fbshipit-source-id: afbb3b8b798f7878fc3206586497ea771116066b
2020-01-28 10:29:14 -08:00
Jun Wu
811a922e2b py3: urllibcompat: prefer unicode on Python3
Summary:
See D17098991 for our encoding strategy.

This diff changes the urllibcompat module to prefer unicode on Python 3.

Reviewed By: farnz

Differential Revision: D19595738

fbshipit-source-id: 53ce76f8703fa0f4885f3042049d9c5fae4bc47c
2020-01-28 10:29:14 -08:00
Jun Wu
6a37eb5a22 py3: pycompat: prefer unicode on Python3
Summary:
See D17098991 for our encoding strategy.

This diff changes the pycompat module to prefer unicode on Python 3.

Reviewed By: farnz

Differential Revision: D19594828

fbshipit-source-id: 3e6db24a2ebaed69deb68908f19e9b462f62c350
2020-01-28 10:29:13 -08:00
Jun Wu
9db1ad4cf7 py3: encoding: prefer unicode on Python3
Summary:
See D17098991 for our encoding strategy. tl;dr, paths, argv, messages, terminal
(not pipes or files) stdout are utf-8. Non utf-8 paths are unsupported (as they
will have issues on Windows anyway).

This diff changes the encoding module to prefer unicode on Python 3.

Reviewed By: simpkins

Differential Revision: D19594827

fbshipit-source-id: c5c1319d840709bc752b96bed6ca80d10f49669e
2020-01-28 10:29:13 -08:00
Jun Wu
b9133c32f9 py3: node: prepare python3 unicode APIs
Summary:
In Python 3 we'd like hex to return unicode strings. Provide APIs for them.
The old bytes version remain as `bbin` and `bhex`.

Reviewed By: markbt

Differential Revision: D19592521

fbshipit-source-id: 1a06ac82e3061c631c0da5d5124610656bdecaee
2020-01-28 10:29:13 -08:00
Jun Wu
e70c21af9c py3: use pure util.dirs on Python 3
Summary:
The native implementation of "dirs" uses too much PyBytes internals that it's
not easy to migrate to unicode. For now let's disable the native path to
unblock Python 3 process.

Looking at the usage of dirs, it is unclear whether it is still a critical path
for performance, as scaning through all files in manifest even with the C dirs
is still too slow and we probably want time complexity fixes (ex. make it
lazy).

Reviewed By: markbt

Differential Revision: D19581192

fbshipit-source-id: 59bb2a187f675df01b4385bf4bce255f173a0aa3
2020-01-28 10:29:12 -08:00
Jun Wu
234cbf213a py3: disable statprof on Python 3
Summary:
It is optional and has many encoding issues. So let's disable it for now to
unblock progress.

Reviewed By: markbt

Differential Revision: D19581182

fbshipit-source-id: 820da07dfe214a3abcd7cbf819b73dbe493c3bcc
2020-01-28 10:29:12 -08:00
Jun Wu
b93ad9436d py3: use native selectors on Python 3
Summary:
This removes a warning:

	scm3/edenscm/mercurial/thirdparty/selectors2.py:39: DeprecationWarning: Using
	or importing the ABCs from 'collections' instead of from 'collections.abc' is
	deprecated since Python 3.3, and in 3.9 it will stop working

Reviewed By: markbt

Differential Revision: D19581179

fbshipit-source-id: d311b4df4cdb35527c8ab9d4a41ae306ddb2ab03
2020-01-28 10:29:12 -08:00
Stefan Filip
5720b9a2a1 py3/pymanifest: convert path types from PyBytes to PyPath
Reviewed By: xavierd

Differential Revision: D19594134

fbshipit-source-id: e8532a125aa2ed4b7740e669ad572fcbb327692f
2020-01-28 10:29:11 -08:00
Durham Goode
23562ea499 amend: backout transaction change
Summary:
D18213028 made amend and metaedit use a transaction around the
dirstate. This appears to have broken some auto-restack flows (T61073763). Let's
back it out for now then I'll debug later.

Reviewed By: quark-zju

Differential Revision: D19581912

fbshipit-source-id: cce32efdc91be9ca301da26a0e26b50aa6aa92d4
2020-01-28 10:22:25 -08:00
Xavier Deguillard
283b120bb6 pyconfigparser: use PyPath instead of PyByte
Summary:
Also, add a util::path::strip_unc function that is more clear than the
normalize_for_display

Reviewed By: DurhamG

Differential Revision: D19595961

fbshipit-source-id: 330bcb708bf64320a3562d79db685d6cb1e14f16
2020-01-28 10:14:14 -08:00
Xavier Deguillard
f3b73fe8bf pymetalog: remove unecessary encoding deps
Summary: The crate doesn't depend on it, let's remove the dependency

Reviewed By: quark-zju

Differential Revision: D19593138

fbshipit-source-id: ad1e9fc68f8d126b6402efd5b1be4633fc7e5887
2020-01-28 10:01:52 -08:00
Xavier Deguillard
b45d97f7df pybookmarkstore: use PyPath instead of PyByte
Reviewed By: DurhamG

Differential Revision: D19593033

fbshipit-source-id: 5e2d5b9e72c336e6290c8359071d086a15bd05f6
2020-01-28 10:01:52 -08:00
Xavier Deguillard
1ae9196972 pyblackbox: use PyPath instead of PyByte
Reviewed By: DurhamG

Differential Revision: D19592901

fbshipit-source-id: ff875ec30bee5ac160ac701091835adbf3013b90
2020-01-28 10:01:52 -08:00
Xavier Deguillard
2d2d63544d pymutationstore: use PyPath instead of PyByte
Reviewed By: DurhamG

Differential Revision: D19592467

fbshipit-source-id: 2d7df3457a9dd02e839cffb642401575712f0f2c
2020-01-28 10:01:51 -08:00
Xavier Deguillard
962fce975d pytreestate: use PyPath instead of PyBytes
Reviewed By: DurhamG

Differential Revision: D19591844

fbshipit-source-id: be2fcdc872f57307e20e24f714b05a6f461e0594
2020-01-28 10:01:51 -08:00
Xavier Deguillard
3e27c70d42 pynodemap: use PyPath instead of PyBytes
Reviewed By: DurhamG

Differential Revision: D19591313

fbshipit-source-id: bd3a8b04b83c226f23ad6d0e1b9509bf65db2f69
2020-01-28 10:01:51 -08:00
Xavier Deguillard
8d20878c1a pydag: use PyPath instead of PyBytes
Reviewed By: quark-zju

Differential Revision: D19591260

fbshipit-source-id: 72ea5be6f79d02cce22c7a5f389cf13a9c69f969
2020-01-28 10:01:50 -08:00
Xavier Deguillard
a0d934ecc2 pyedenapi: use PyPath instead of PyBytes
Reviewed By: quark-zju

Differential Revision: D19591173

fbshipit-source-id: 1a5362c99c37f342b1aeb2d95573e4d09c8440c5
2020-01-28 10:01:50 -08:00
Xavier Deguillard
61aaf894c3 pyrevisionstore: use PyPath instead of PyBytes
Summary:
For Python3 compatibility, let's use PyPath, it hides the logic of encoding for
Python2

Reviewed By: DurhamG

Differential Revision: D19590024

fbshipit-source-id: 7bed134a500b266837f3cab9b10604e1f34cc4a0
2020-01-28 10:01:50 -08:00
Adam Simpkins
de66c7494d py3: delete an unused variable that causing Python 3 compatibility problems
Summary:
In Python 3 this causes problems trying to mix bytes with str.  This does not
appear to be used anywhere, so delete it.

Reviewed By: farnz

Differential Revision: D19591564

fbshipit-source-id: 42b8720e440e8e7c901b325aecb1343d5b79b7cc
2020-01-27 21:23:19 -08:00
Adam Simpkins
c049799e91 py3: convert some dict.iteritems() calls to items()
Summary:
These call sites are hit during initialization.  They all appear to be
small-ish dictionaries so the performance hit of having this iteration be
non-lazy in Python 2 should not matter much.

Reviewed By: farnz

Differential Revision: D19591565

fbshipit-source-id: 324844d64ba63f6e379aab270e453d44a02063e9
2020-01-27 21:23:19 -08:00
Adam Simpkins
3a71160859 py3: convert some encoding.environ uses to os.environ
Summary:
Our plan for switching to Python 3 is to update many types to use Unicode
strings when appropriate.  In particular, we should just go ahead and use
Unicode for environment variables, since this is the Python 3's default
behavior.

This drops the check-code.py check that warns developers against using
`os.environ` directly, and converts several call sites to use `os.environ`
directly instead of using `encoding.environ`

Reviewed By: markbt

Differential Revision: D19591566

fbshipit-source-id: e4ac44dcd5f19cd0bf65ac904fc85edad95a9aed
2020-01-27 21:23:19 -08:00
Jun Wu
a0bd794d55 run-tests: make --debug add debugger settings
Summary: This makes --debug more useful - crashes in tests trigger ipdb sessions.

Reviewed By: mitrandir77

Differential Revision: D19581174

fbshipit-source-id: 770c7b054dfce88082b67a9347ecaeb24d8583bc
2020-01-27 19:37:21 -08:00
Mateusz Kwapich
2f68d4b0d2 remove py3 code transform
Summary:
We won't need it as we're going straight to dual-compatible code instead of
transforming it.

Reviewed By: quark-zju

Differential Revision: D19588132

fbshipit-source-id: 100f19b3e29874de9e8d48692c42cb2d5856bf5a
2020-01-27 19:00:14 -08:00
Jun Wu
373073df47 py3/rust: cpython-ext: optionally show Python error traceback
Summary:
This is optional, but it helps investigating Python errors chained with other
Rust errors.

For example:

  error.RustError: failed fetching from store (, cc38739855a7f356b4a2aaac0a0a858fd646e6bf)
  Caused by:
    TypeError()
    Traceback (most recent call last):
      File "scm3/edenscm/hgext/remotefilelog/contentstore.py", line 53, in get
        chain = self.getdeltachain(name, node)
      File "scm3/edenscm/hgext/remotefilelog/contentstore.py", line 91, in getdeltachain
        chain = self._getpartialchain(name, node)
      File "scm3/edenscm/hgext/remotefilelog/contentstore.py", line 125, in _getpartialchain
        return store.getdeltachain(name, node)
    TypeError

Without this diff there is only "TypeError()" without the traceback.
This can be turned off by unsetting RUST_BACKTRACE.

Reviewed By: markbt

Differential Revision: D19581173

fbshipit-source-id: 74605b78146b6b1c9ddd5ad720dcd19ff73908a8
2020-01-27 18:56:10 -08:00
Xavier Deguillard
24ae9f9592 cpython-ext: fix python3 compile error
Summary: The format_err is used in shared code too, we need to import it.

Reviewed By: quark-zju

Differential Revision: D19592591

fbshipit-source-id: bd344bf3c295473f4647235a98432d11c9678bf9
2020-01-27 16:58:42 -08:00
Xavier Deguillard
33ea1763ce cpython-ext: add a PyPath type
Summary:
This will be used as an argument to the Rust bindings when using paths. This
type is either a PyBytes in Python2 and uses the various encoding function to
convert into a String, or a PyUnicode in Python3 with no encoding change.

Reviewed By: farnz

Differential Revision: D19587890

fbshipit-source-id: 58903426585693193754691fe3c756b9097b35f6
2020-01-27 16:50:14 -08:00
Xavier Deguillard
e512c370fd py3/rust: cpython-ext: set ob_size on raw PyObject
Summary:
Without this, Rust code using the feature (ex. lz4, used by lz4revlog) will
panic.

Reviewed By: sfilipco

Differential Revision: D19581188

fbshipit-source-id: b499449df4fede27fe66cf8e5af57e8347a0dd48
2020-01-27 16:50:14 -08:00
Xavier Deguillard
f16bb04977 py3/rust: cpython-ext: support memoryview as PySimpleBuf
Summary: Otherwise we got RustPanic when clindex or dagindex reads mmapped changelog.i.

Reviewed By: sfilipco

Differential Revision: D19581189

fbshipit-source-id: 3ee74a1bd000d58272551ae404dcfe7f957bb2c0
2020-01-27 16:50:13 -08:00
Xavier Deguillard
ad58839ca1 py3/rust: use Str type in cliparser and hgcommands
Reviewed By: sfilipco

Differential Revision: D19581176

fbshipit-source-id: e92e5c2538537ec16da25a9819c9a097a24a4d6e
2020-01-27 16:50:13 -08:00
Xavier Deguillard
1c697dbc49 py3/rust: cpython-ext: add Str type
Summary: This converts to bytes on Python 2, but unicode on Python 3.

Reviewed By: markbt

Differential Revision: D19581180

fbshipit-source-id: 0de9056a01ae30810a72352387de5a940b37d7ab
2020-01-27 16:50:13 -08:00
Xavier Deguillard
789d2b5fbb py3/rust: types: add AsRef<str> for RepoPath
Summary:
In a future diff, I have RepoPath in Rust and want to send unicode path to
Python.

Reviewed By: sfilipco

Differential Revision: D19581184

fbshipit-source-id: 73a03707a6bdae4a497a8ee2c14314aa4ffefb6d
2020-01-27 16:50:12 -08:00
Karthikeyan Singaravelan
41cd693b98 Import ABC from collections.abc instead of collections for Python 3.9 compatibility. (#780)
Summary:
Fixes https://github.com/facebook/watchman/issues/778
Pull Request resolved: https://github.com/facebook/watchman/pull/780

Test Plan: sandcastle

Reviewed By: chadaustin

Differential Revision: D19543389

Pulled By: wez

fbshipit-source-id: d9725d2f4325da2152e3c19100a25df0fe6f5dec
2020-01-27 16:35:57 -08:00
Kostia Balytskyi
6bf47a9f5a hgtime: fix corner case of date range parsing
Summary: The docs promise that both `<` and `>` bounds are inclusive, so let's fix that.

Reviewed By: markbt

Differential Revision: D19580840

fbshipit-source-id: 13770a8e9351fe62f58e9a701b526a167752543a
2020-01-27 09:37:00 -08:00
Stefan Filip
d78982a6e8 dag: move iddag to own file
Summary:
Separate Segment and IdDag in two individual files. This is preparation for
refactoring IdDag to be more flexible in terms of storage. That will probably
involve moving stuff out of IdDag into a new file that deals with the storage
abstractions.

Reviewed By: quark-zju

Differential Revision: D19559127

fbshipit-source-id: b3b9b18e2653157e69148b1f29292a57b30016ec
2020-01-24 15:49:54 -08:00
Mark Thomas
d35692711e tests: convert test-rebase-obsolete to mutation
Summary:
Add `test-rebase-mutation.t`, based on `test-rebase-obsolete.t`, but instead
testing specifically for mutation and visibility behaviour.

Reviewed By: xavierd

Differential Revision: D19549473

fbshipit-source-id: c993686eab47985737eef17ab0f52f31d0929159
2020-01-24 14:47:39 -08:00
Mark Thomas
c0e72db0ad tests: enable mutation and visibility in some rebase tests
Summary:
Enable mutation and visibility for the existing tests that use evolution.

Excludes `test-rebase-obsolete.t` which specifically targets obsmarkers - this
will be converted to a new test in D19549473.

Reviewed By: quark-zju

Differential Revision: D19502220

fbshipit-source-id: 70b2828b60220c6887a6210e91bed01c2809cc51
2020-01-24 14:47:39 -08:00
Mark Thomas
f2d5739705 mutation: allow instabilities if mutation is enabled
Summary:
Disallowing instabilities is a feature of evolution.  For mutation tracking
there is no problem with instabilities.

Reviewed By: quark-zju

Differential Revision: D19502221

fbshipit-source-id: c21a5021bf1a8b29abb6635928da50895e2cec33
2020-01-24 14:47:38 -08:00
Mark Thomas
980829029a mutation: include all commits as predecessors in rebase --collapse
Summary:
Rebase with `--collapse` should behave like `fold` and set all the rebased
commits as predecessors of the new commit.

Reviewed By: quark-zju

Differential Revision: D19502223

fbshipit-source-id: a903ee5647bb4ee947a6cc68d8ca791fe0f1299d
2020-01-24 14:47:38 -08:00
Mark Thomas
398e175762 visibility: ensure visibleheads and store requirements are copied on clone
Summary:
When cloning a repo, ensure the store requirements and visibleheads files are
included in the clone.

Reviewed By: quark-zju

Differential Revision: D19502224

fbshipit-source-id: e3ead586fb1fe10def02d55e33810643c0c78bc8
2020-01-24 14:47:38 -08:00
Mark Thomas
845e35ea5f metalog: ensure legacy files are written out atomically
Summary:
`hg clone` can hardlink store files if the repos are on the same device.  This
means we need to write out the legacy files using an atomic temp file, so that
the hardlink is broken.  This also makes sure anyone watching the file sees the
change atomically.

Reviewed By: quark-zju

Differential Revision: D19502222

fbshipit-source-id: 3e9fb35846111c9d924de3c6b6861ce5e295e035
2020-01-24 14:47:37 -08:00
Mark Thomas
39f8b3109f visibility: add visibleheads to journaled files
Summary:
The visibleheads file should be journaled like the other state files, so that
it is rolled back if a transaction aborts.

Reviewed By: quark-zju

Differential Revision: D19502226

fbshipit-source-id: c2b8d926ce97211411ca01f05e83e62cb77625b9
2020-01-24 14:47:37 -08:00
Mark Thomas
4bc06df032 visibility: allow bundlerepo visibleheads to be updated
Summary:
The `incoming` command works by creating a bundlerepo and then updating the
commits as if a pull occurred.  This means the visibleheads can get updated,
which was previously assumed impossible.

We can just accept the new set of visibleheads as-is.  There may be some
unnecessary heads that are covered by other heads, but since the bundlerepo
is short-lived, this isn't a problem.

Reviewed By: quark-zju

Differential Revision: D19502219

fbshipit-source-id: ce9e98c6a40f96f5305bc7580efb476d5264fc88
2020-01-24 14:47:37 -08:00
Mark Thomas
2859b55829 mutation: the workingctx is not obsolete
Summary: It's possible for the workingctx to be queried for obsolescence.  Make sure it is not considered to be obsolete.

Reviewed By: quark-zju

Differential Revision: D19502225

fbshipit-source-id: a1a37afd39a0fc16f95b4eb917372f193ac3fe82
2020-01-24 14:47:36 -08:00
Jia Chen
c717ef0d7d Update pyre version for eden
Summary: Automatic upgrade to remove `version` override and silence errors.

Differential Revision: D19548623

fbshipit-source-id: 12d5e8ec6450de42107d07b230e4b2d0824f7c9d
2020-01-24 12:32:43 -08:00
Lukas Piatkowski
8c7de41d8c arc/licenselint: add license linting for lua and sql
Summary: The fbcode/linttool has been updated to support lua and sql here D19467627, so updating arc rules to cover the new extensions

Reviewed By: zertosh

Differential Revision: D19534326

fbshipit-source-id: 4e615653acedcdf2da88e7dc4b443ad22af1324f
2020-01-24 06:13:28 -08:00
Jun Wu
52af332c28 renderdag: add tests showing how orders affect rendering
Summary:
I wrote it to understand how renderdag draws the same graph with different
orders. It seems useful for future optimization that tries to reduce the number
of columns. So let's check it in.

Reviewed By: xavierd

Differential Revision: D19440713

fbshipit-source-id: 8bc580799f6b24c87886d5ac306020f50bb694e5
2020-01-23 20:50:56 -08:00
Jun Wu
29c749ef7d dag: add fuzz tests on the octopus DAG
Summary: This gives us some confidence about octopus merge handling.

Reviewed By: DurhamG

Differential Revision: D19540726

fbshipit-source-id: e84de74aecae54429483edd185d39fd1bd858f87
2020-01-23 17:58:51 -08:00
Jun Wu
8ac97da54e bindag: make TestContext more flexible
Summary:
TestContext uses ParentRevs. That limits parents to at most 2.

Use a type parameter so we can opt-in Vec<usize> for octopus merge support,
at the cost of worse cache efficiency.

Reviewed By: DurhamG

Differential Revision: D19540727

fbshipit-source-id: f9e8de151b7b296fd6f0fd89be9de2b8996634c7
2020-01-23 17:58:51 -08:00
Jun Wu
df23791d08 bindag: add some octopus examples
Summary:
Our new algorithms support octopus merges. However there were no tests using
octopus merges. This diff adds a simple one.

Reviewed By: DurhamG

Differential Revision: D19540728

fbshipit-source-id: 8411024f0b7e27c2ebfabbe1935496124c25df7b
2020-01-23 17:58:51 -08:00
Jun Wu
494bdae7cc dag: add a fuzz test about range algorithm
Summary:
The test runs the old and new algorithm and compares their result.  This is more
interesting than using random numbers, since the fuzzing framework will try to
explore new code paths.

Reviewed By: sfilipco

Differential Revision: D19511576

fbshipit-source-id: e9a2066769b54a60bb92643e5715f91a6fccbcb5
2020-01-23 17:58:50 -08:00
Jun Wu
78ea96cb9d bindag: port range algorithm from hg
Summary:
The ported algorithm will work as a comparison to verify dag's range
implementation.

Reviewed By: sfilipco

Differential Revision: D19511574

fbshipit-source-id: 589353d6e6c91b8d6707c977eeb8558ac733b525
2020-01-23 17:58:50 -08:00
Stefan Filip
2dc14cf9a6 manifest: use Rust Tree Manifest by default
Summary:
The Rust Manifest is deployed practically everywhere.
Removing the C++ code from being compiled in Mercurial along with all of the cstore extension.

Reviewed By: quark-zju

Differential Revision: D19543248

fbshipit-source-id: d632e171175e6866563c1aa0808a099b67bd937d
2020-01-23 16:06:51 -08:00
Stefan Filip
f5280b75e9 thrift: update thrift generated files
Summary: Commit updates after having ran `make local`

Reviewed By: xavierd

Differential Revision: D19543278

fbshipit-source-id: 00fdc3ebec32e8a3d706b89402dc91f771984c3c
2020-01-23 16:06:51 -08:00
Xavier Deguillard
8041ff9e82 bisect: remove revision number in output
Summary: Revision numbers are deprecated, let's not print them in the bisect output.

Reviewed By: DurhamG

Differential Revision: D19539678

fbshipit-source-id: 29bca3ee38cb390bdef8dd88cb912aa3dc452065
2020-01-23 15:54:00 -08:00
Xavier Deguillard
31b266d32c graft: remove revision number in output
Summary: Revision number are deprecated, remove it from the user output.

Reviewed By: DurhamG

Differential Revision: D19539679

fbshipit-source-id: 7fc618e832e0e5bb3464d2ed04967b062b6f182a
2020-01-23 15:53:59 -08:00
Michael Devine
3f326e911c Repo converter: Add named tuple for commit spec
Summary: The converter has many anonymous tuples being passed around, and it leads to typing error and pain figuring out which field has which data. This diff uses named tuples to implement an important tuple in the code that we use to merge repo history into a single unified history.

Reviewed By: tchebb

Differential Revision: D19377226

fbshipit-source-id: cb7d49227752fd7bc98233f6c0d1b2913cb24b05
2020-01-23 12:36:29 -08:00
Xavier Deguillard
ff09f797f4 demandimport: disable in python3
Summary:
We don't need to it for the initial phases of the conversion. I've seen cases
were exceptions while importing were also hidden and caused weird behavior down
the line.

Reviewed By: DurhamG

Differential Revision: D19539064

fbshipit-source-id: 7415d88ef8fb69c949ed557ab0df7e6f7b39bf86
2020-01-23 11:41:02 -08:00
Xavier Deguillard
fe8855e9fb hgmain: initialize fbinit
Summary:
The memcache code assumes it's initialized, let's actually do it.

The test change is required due to fbinit installing its own SIGTERM handler
that shows a backtrace. Installing our own SIGTERM handler makes it closer to
what Mercurial does already.

Reviewed By: quark-zju

Differential Revision: D19518698

fbshipit-source-id: bc8c2311e65c9c00678756abae3979ccda4453ea
2020-01-23 08:57:22 -08:00
Xavier Deguillard
b6589bde84 revisionstore: prefetch takes &[Key] instead of Vec<Key>
Summary: This can prevent potential moves and clones on the caller of prefetch.

Reviewed By: quark-zju

Differential Revision: D19518697

fbshipit-source-id: 63839fc3f4bb9ca420e290eabaffb481a3584f7b
2020-01-23 08:57:22 -08:00
Jun Wu
fff2cb833f dag: add a fuzz test about gca algorithm
Summary:
The test runs the old and new algorithm and compares their result.  This is more
interesting than using random numbers, since the fuzzing framework will try to
explore new code paths.

This cannot run on stable Rust yet. I added a README for how to run it.

Reviewed By: sfilipco

Differential Revision: D19504096

fbshipit-source-id: 621da02c50a771dee9932f9d7a407cb1f412a543
2020-01-22 19:30:50 -08:00
Jun Wu
af85f4ff3b bindag: add a way to get a subdag of a parsed bindag
Summary: Sometimes the graph is too large. Provide a way to slice it.

Reviewed By: sfilipco

Differential Revision: D19511575

fbshipit-source-id: 504317d6894764043b23ea49dcf09c8cdea96961
2020-01-22 19:30:49 -08:00
Jun Wu
b5482f8976 bindag: add utilities for easier testing
Summary:
As we plan to test the dag crate with some other DAG implementation,
add a convenient structure that setups both DAG implementations.

Reviewed By: sfilipco

Differential Revision: D19503371

fbshipit-source-id: 3e9933ad37301bfac36eb1af6d82b4298af778b6
2020-01-22 19:30:49 -08:00
Jun Wu
7d11508dfa bindag: port GCA algorithm from hg
Summary:
The ported algorithm will work as a comparison to verify dag's gca
implementation.

Reviewed By: sfilipco

Differential Revision: D19503373

fbshipit-source-id: f5253db89fbcdc2fd02f3fdaa0796e24338b1fba
2020-01-22 19:30:49 -08:00
Jun Wu
a98d288938 bindag: apply smallvec optimization
Summary:
This is similar to D17581248. It will make the old linear-scan algorithm (which
will be added later) about 5x faster.

Reviewed By: sfilipco

Differential Revision: D19503372

fbshipit-source-id: c65d7217e7b144603dadd57f54a5e70f513c8e51
2020-01-22 19:30:48 -08:00
Jun Wu
64271f24ba dag: move bindag from benches to a separate crate
Summary: This allows bindag to be used outside benches.

Reviewed By: sfilipco

Differential Revision: D19503374

fbshipit-source-id: 131061f7d1d28125875a86afc330dbb9634249cf
2020-01-22 19:30:48 -08:00
Stanislau Hlebik
f8ba0c99a5 infinitepush: introduce infinitepushbookmark path
Summary:
Previously we had infinitepush path which was preferred over a default path when a
specific revision was pulled or updated to (i.e. "hg pull" would use
paths.default, but "hg pull -r HASH" would use paths.infinitepush if this path
is set in the config).

I'd like to have infinitepushbookmark path which will be preferred for all
cases except for a single commit pulls.

Reviewed By: mitrandir77

Differential Revision: D19513490

fbshipit-source-id: 7f8b809f937aa2e082c2e9cd382709c6db619a4f
2020-01-22 14:05:38 -08:00
Jun Wu
fcaef9ed7e sparse: make sparse profile tracking cheaper
Summary:
Reading all the sparse profiles can be very expensive.
This changes the format we log to the cloud. It seems nobody is depending on the old format.

Reviewed By: wlis

Differential Revision: D19517306

fbshipit-source-id: 256e78acd4c24aceb533aa4d66b90ecf885fcbcd
2020-01-22 13:41:34 -08:00
Xavier Deguillard
d997f31b93 setup: properly install the binaries
Summary:
Recently, the Rust extensions went away (D19340538) but that had the negative
side effect of no longer installing all Rust binaries. Make sure the
{build,install}_rust_ext steps are run properly when Rust binaries are present.

Reviewed By: quark-zju

Differential Revision: D19509526

fbshipit-source-id: 03f62f41595931dd7d5f32544a35d31e20fa3f0c
2020-01-22 09:29:18 -08:00
Harvey Hunt
c5f328be68 hg: Remove purge extension from preimported modules
Summary:
D19506964 moved the purge extension into core, however purge is still
mentioned in dispatch.py's list of external modules to preimport. This causes hg
to crash as such:

```
  Traceback (most recent call last):
    File "/data/users/harveyhunt/fbsource/fbcode/buck-out/dev/gen/eden/scm/__hg__/runtime_resources/__default__/eden/scm/hglib/edenscm/__init__.py", line 61, in run
      dispatch.runchgserver()
    File "/data/users/harveyhunt/fbsource/fbcode/buck-out/dev/gen/eden/scm/__hg__/runtime_resources/__default__/eden/scm/hglib/edenscm/mercurial/dispatch.py", line 335, in runchgserver
      _preimportmodules()
    File "/data/users/harveyhunt/fbsource/fbcode/buck-out/dev/gen/eden/scm/__hg__/runtime_resources/__default__/eden/scm/hglib/edenscm/mercurial/dispatch.py", line 320, in _preimportmodules
      extensions.preimport(extname)
    File "/data/users/harveyhunt/fbsource/fbcode/buck-out/dev/gen/eden/scm/__hg__/runtime_resources/__default__/eden/scm/hglib/edenscm/mercurial/extensions.py", line 180, in preimport
      mod = getattr(__import__("edenscm.hgext.%s" % name).hgext, name)
  ImportError: No module named purge
```

Remove purge from the extmods list.

Reviewed By: ikostia

Differential Revision: D19514862

fbshipit-source-id: 6e16ff160f0bc842038b1481bf23aab9c608685d
2020-01-22 08:18:09 -08:00
Saurabh Singh
17ad617625 purge: move extension to core
Reviewed By: quark-zju

Differential Revision: D19506964

fbshipit-source-id: 173918346e90fadbaa52155786aa30fbd4951c8c
2020-01-21 20:49:08 -08:00
Chad Austin
36954d229d stop calling getManifestEntry and instead look directly in the manifest
Summary:
getManifestEntry has an inefficient implementation. It can result in
multiple round trips to the Mercurial importer process and under heavy
usage it can clog the Thrift worker thread pool. Moreover, it only
provides data that Mercurial already knows. Instead of calling
getManifestEntry, look in the manifest to see if the entry exists.

Reviewed By: quark-zju

Differential Revision: D19426507

fbshipit-source-id: 67bcd7e0ae62f84b313aa6a7203832fb9a10f315
2020-01-21 10:33:10 -08:00
Xavier Deguillard
524c85d711 revisionstore: limit delta chain to 1000 entries
Summary:
We've seen a case where a datapack contains a circular delta chain, causing
Mercurial to fall into a infinite loop when trying to read it. Let's fail when
the chain is over 1000 entries.

Reviewed By: quark-zju

Differential Revision: D19458453

fbshipit-source-id: bfa503f7807122eca72cf94418abda161dafa41c
2020-01-21 08:50:59 -08:00
Jun Wu
b863b99cd2 pullcreatemarkers: print how many commits are marked as landed
Summary:
wez silenced the messages in D18796291. I think it's still useful to show why
commits get disappeared after pull. Let's just print a summary by default, and
print less or more things depending on flags:

- `-v`: print what commits are marked as landed, similar to pre-D18796291.
- `-q`: print nothing, hide the summary line.

Reviewed By: DurhamG

Differential Revision: D19437916

fbshipit-source-id: 130c5a24f0978674d363227f44089d307f7aff72
2020-01-21 07:36:34 -08:00
Jun Wu
0d15eb6227 graphql: translate globalrev to hg node in getlandednodes
Summary:
In www, Phabricator used to return both landed global rev and hg commit hashes.
Now it only returns global revs. The "mark landed" logic ignores global revs
and does not mark them as landed.

This diff changes it so the graphql client translates global revs automatically
so the "mark landed" logic can still work.

Reviewed By: DurhamG

Differential Revision: D19437915

fbshipit-source-id: 0e074bda9a0ee53c594750b2b1fd214f31dec843
2020-01-21 07:36:34 -08:00
Jun Wu
3da70a5d46 graphql: support translating GLOBAL_REVs in getmirroredrevmap
Summary: The function assumes binary nodes. Make it aware of GLOBAL_REVs.

Reviewed By: DurhamG

Differential Revision: D19437914

fbshipit-source-id: aa36b81be2f8e4c38c51667b4cf7048eeb7250da
2020-01-21 07:36:34 -08:00
Mark Thomas
4958c4e342 tests: update tests-fb-*.t to use new tinit features
Reviewed By: quark-zju

Differential Revision: D19457679

fbshipit-source-id: d1010f0d2eee69ecdee5c3feba772bd3684d4cec
2020-01-20 02:45:17 -08:00
Mark Thomas
d2730c3ba0 tests: update tests-[r-x]*.t to use new tinit features
Reviewed By: quark-zju

Differential Revision: D19457676

fbshipit-source-id: 5bbab22ff0d98b725b63b906864037fae5fa5171
2020-01-20 02:45:16 -08:00
Mark Thomas
633382de11 tests: update tests-[i-p]*.t to use new tinit features
Reviewed By: quark-zju

Differential Revision: D19457680

fbshipit-source-id: 2be941f22525ed2d9f5a1aa3c92af38fb612eb46
2020-01-20 02:45:16 -08:00
Mark Thomas
b361a8d1bd tests: update tests-[a-h]*.t to use new tinit features
Reviewed By: quark-zju

Differential Revision: D19457677

fbshipit-source-id: 9f121837dfc6bf98ff4525c6535496b292409844
2020-01-20 02:45:15 -08:00
Mark Thomas
b654a9aae5 tests: use disable treemanifest in place of setconfig
Summary: Use `disable treemanifest` in place of `setconfig extensions.treemanifest=!`

Reviewed By: quark-zju

Differential Revision: D19457678

fbshipit-source-id: 47c8b3c4f4ed5cfc97275ca67afab4a86ffe7f0d
2020-01-20 02:45:15 -08:00
Mark Thomas
212cec117b tinit.sh: add configure for setting up standard configurations
Summary:
Add a new `tinit.sh` function: `configure`, which takes over from the
feature-enabling aspect of `enable`.

This provides a few features that are configurable:

  * `dummyssh` sets `ui.ssh` to `dummyssh`.
  * `mutation` and `mutation-norecord` enable mutation and visibility tracking
  * `evolution` enables evolution and disables mutation and visibility.
  * `noevolution` disables evolution, as well as mutation and visibility.

Since `enable` now only refers to extensions, it is joined by a corresponding `disable` to
disable an extension.

Some tests are updated to take advantage of these new functions.

Reviewed By: quark-zju

Differential Revision: D19427595

fbshipit-source-id: 03cc639918b4e667927330d3f3abf16121ebf161
2020-01-20 02:45:14 -08:00
Jun Wu
124e275377 dag: make NameDag use MultiLog for data consistency
Summary: This ensures IdMap and IdDag are guaranteed consistent in the storage layer.

Reviewed By: DurhamG

Differential Revision: D19432658

fbshipit-source-id: 00f1a9b4c747baa1f14d78c31d925682317463b4
2020-01-17 21:49:57 -08:00
Jun Wu
907aadcdd7 indexedlog: add MultiLog
Summary: The MultiLog holds multiple Logs and can atomically sync them.

Reviewed By: DurhamG

Differential Revision: D19432659

fbshipit-source-id: 6ac7dc6f74468f985c6a6b0c419e888722a80037
2020-01-17 21:49:57 -08:00
Jun Wu
5aa872599c indexedlog: make ScopedDirLock remember which directory gets locked
Summary: This makes it possible to do extra sanity checks.

Reviewed By: DurhamG

Differential Revision: D19443783

fbshipit-source-id: 254c2537a6aadd25a67c5e48a768187ce65aa686
2020-01-17 21:49:56 -08:00
Jun Wu
2c9a3b9c61 indexedlog: use a method to create LogMetadata
Summary: This makes the code overall shorter.

Reviewed By: DurhamG

Differential Revision: D19443552

fbshipit-source-id: abd1db227830a88549c7eca1cfd08b67c4914518
2020-01-17 21:49:56 -08:00
Jun Wu
a78597c92d gitlookup: use nodemap index to speed up lookups
Summary:
Scanning through the plain git mapfile is slow. Use the nodemap to speed it up.
To avoid unnecessary risks, this only replace the "lookup" feature of the mapfile,
other features used by the hggit extension or wire-protocols are unaffected.

Reviewed By: singhsrb

Differential Revision: D19458406

fbshipit-source-id: 665184637d3e62590cc5d12ea3aa2563af4351d1
2020-01-17 19:21:40 -08:00