Commit Graph

343 Commits

Author SHA1 Message Date
Adam Simpkins
2370602802 py3: re-enable demandimport
Summary:
D19539064 disabled demandimport when using Python 3 as a precaution while we
were initially working on Python 3 support.  However, it appears to work
correctly and does not seem to cause obvious problems when enabled.

This re-enables demand import by default.  It can be explicitly disabled at
runtime by specifying `HGDEMANDIMPORT=disable` in the environment.

Reviewed By: xavierd, farnz

Differential Revision: D19655548

fbshipit-source-id: d56814049181da2d68abd09ac7ca772b04e06618
2020-01-30 18:29:50 -08:00
Adam Simpkins
ebf7df9c50 fix the relpath() template to work in hg status
Summary:
When used in an `hg status` command the `relpath()` template function would
throw an unhandled `KeyError` when it tried to access `ctx` in the template
mapping.  This field is only present in `hg log` templates, and is not present
in `hg status` templates.

This updates the `relpath()` template function to look for `repo` in the
template mapping instead.  I also updated the `hg status` code to pass in the
`repo` value now.  The `hg log` code already passes in this field.

Reviewed By: quark-zju

Differential Revision: D19568463

fbshipit-source-id: 3e1c1d6571c73e4b24bfce22ea28f57679d39846
2020-01-30 18:26:36 -08:00
Xavier Deguillard
7c48ba6119 rebase: fix encoding issues
Summary:
Bunch of read/writes to files were using plain strings, use byte strings
instead.

Reviewed By: quark-zju

Differential Revision: D19655101

fbshipit-source-id: 0cb70cbc51630d2c83752846774ff1a1c5ec9f41
2020-01-30 18:12:29 -08:00
Xavier Deguillard
607b2db333 obsutil: decode marker metadata
Summary: These are strings, let's decode them.

Reviewed By: quark-zju

Differential Revision: D19655102

fbshipit-source-id: 18ca5556c76603a53bcb20b58fa8787e2fab70fb
2020-01-30 18:12:29 -08:00
Xavier Deguillard
4eb639c665 amend: write to the obsinhibit with bytes
Summary: This file contains bytes representation of nodes, let's add the b prefix.

Reviewed By: quark-zju

Differential Revision: D19655100

fbshipit-source-id: 2dc163038b17b0d3bce384d9b04f13b1449705fc
2020-01-30 18:12:28 -08:00
Jun Wu
94c6853e8d edenscm: add a main module
Summary:
Make it possible to run edenscm via a "python package".

This makes it possible to run monkeytype [1] via debugpyhon:

    ./hg3 debugpython -- -m monkeytype run edenscm help log --pager=off

[1]: https://github.com/Instagram/MonkeyType

Reviewed By: farnz

Differential Revision: D19655850

fbshipit-source-id: 358f9dcf976141f798af2d57fb6392f2bca6a396
2020-01-30 18:09:14 -08:00
Mateusz Kwapich
040e01d65c filelog: file contents shouldn't be strings
Summary: on file copies instead of contents we were commiting b'contents'

Reviewed By: markbt

Differential Revision: D19656501

fbshipit-source-id: e1cb0bdac79da851494318ce31396a2aca6c5754
2020-01-30 18:05:39 -08:00
Durham Goode
8c211e5224 py3: fix stdout/stderr in dott tests
Summary:
dott tests rig up a custom stdout/stderr. In Python 3 we expect them to
be strings, so we need to use an appropriate io object.

Unfortunatly upstream already created a pycompat.stringio object which uses
bytes for both python 3 and 2, so I had to use a new name for now.

Reviewed By: quark-zju

Differential Revision: D19653469

fbshipit-source-id: a529df5a2ab179c105fa441c4503ed6606e77543
2020-01-30 18:00:34 -08:00
Simon Farnsworth
a579ce59f9 py3: add types to bundle2.py and make wireprotocol exclusively bytes
Reviewed By: xavierd

Differential Revision: D19652082

fbshipit-source-id: b6670845bbd06dc55439ad5223d47bda2ae014b1
2020-01-30 17:56:58 -08:00
Mark Thomas
8e673a1de0 py3: implement util.dirs in Rust
Summary:
The C implementation of util.dirs doesn't work with Python 3.  Instead of trying to port it, write an implementation in Rust that works in both Python 2 and Python 3.

The old implementation had a hack for flat dirstate to omit certain entries.  Remove this hack as we now only use flat dirstate in tests and small repos.  This simplifies the code in the fastpath case.

Reviewed By: quark-zju

Differential Revision: D19634699

fbshipit-source-id: cbb6562bc9a0209b13f67db3dfa5250800a9fcbe
2020-01-30 17:33:36 -08:00
Saurabh Singh
45d5467eec memcommit: add type hints
Summary: This diff just adds the type hints for `memcommit` extension.

Reviewed By: quark-zju

Differential Revision: D19651074

fbshipit-source-id: 834a5980cded92641e4d368ce00d01a3dcaec330
2020-01-30 16:49:50 -08:00
Xavier Deguillard
5ecc59749e treemanifest: hashing requires bytes
Summary: We need to encode the key to be able to hash it.

Reviewed By: genevievehelsel

Differential Revision: D19651373

fbshipit-source-id: 214a3d127eb71a1c9cbf4ed1fec4f5a58f7f5195
2020-01-30 16:29:17 -08:00
Xavier Deguillard
6b328f5f7d remotefilelog: the filelog data is bytes
Reviewed By: DurhamG

Differential Revision: D19651372

fbshipit-source-id: 13bf86235b95ad84beb9fe45e7abc06baec67a47
2020-01-30 16:29:16 -08:00
Xavier Deguillard
4b2b0c6630 remotenames: fix python3 warning
Summary:
Importing directly from collection is deprecated, we should import from
collections.abc in Python3.

Reviewed By: DurhamG

Differential Revision: D19650852

fbshipit-source-id: b2e470e5fde3fe915944ae68a4274e47ed91071a
2020-01-30 16:29:16 -08:00
Xavier Deguillard
8c08a97290 fsmonitor: fix test_symlinks_t
Summary:
A previous change requires the default value to be a tuple with 4 values,
fsmonitor was only passing 1.

Reviewed By: kulshrax

Differential Revision: D19652000

fbshipit-source-id: c9ae04109c61173a1df27ce653622dc4db653977
2020-01-30 16:20:13 -08:00
Carolyn Busch
fca8aafffc py3 chgserver bytes <-> unicode
Summary:
Server reads & writes bytes. Convert between bytes and unicode where
neccessary in chgserver. Add some type annotations.

Reviewed By: xavierd

Differential Revision: D19645028

fbshipit-source-id: 1ad4bec25c398422112ee0ed1780ee9d704e6622
2020-01-30 15:44:42 -08:00
Durham Goode
419858ad5c py3: fix journal writing
Summary:
Writing to the file should happen in bytes, so let's make the
journalentry serializer/deserializer convert from strings to bytes for us.

Reviewed By: xavierd

Differential Revision: D19650468

fbshipit-source-id: 3c13aff032ace1a15611c2370fc7d6ed44a6df34
2020-01-30 15:22:25 -08:00
Durham Goode
c1672ad848 py3: fix treemanifest fetching
Summary:
bundle2 parameter values are bytes, so when we compare the treemanifest
category with the bundle2 part category we need to encode the bundle2 part one.

Also fix wirepack to read utf8 paths.

Reviewed By: xavierd

Differential Revision: D19650197

fbshipit-source-id: 67da3bd8244f7953ec608019844988b20ad1929f
2020-01-30 15:22:25 -08:00
Xavier Deguillard
c1652cad03 drawdag: the commit data should be bytes
Summary:
The commit context requires commit data to be bytes, but drawdag was using str
for it, breaking tests.

Reviewed By: DurhamG

Differential Revision: D19650219

fbshipit-source-id: 1409ce9a277c1c410ede8109989e07595378ff3e
2020-01-30 14:55:55 -08:00
Xavier Deguillard
96f5dc6652 localrepo: wwread returns bytes
Summary:
This is directly used in fctx.data() which expects bytes to be returned, encode
the return value of vfs.symlink.

Reviewed By: quark-zju

Differential Revision: D19647846

fbshipit-source-id: 966a5e79db6843c4bc6b802aeb90364e983cd834
2020-01-30 14:55:55 -08:00
Xavier Deguillard
b863a561a4 pytreestate: use PyPath where necessary
Summary:
A previous diff missed these PyPath. As a bonus, replace most of the PyObject
into proper Rust types.

Reviewed By: quark-zju

Differential Revision: D19646139

fbshipit-source-id: a3b55e5a802d8bbef86e9a9431f605bb4e284b32
2020-01-30 14:55:54 -08:00
Durham Goode
ca48fc2665 py3: fix resolving unknown binary nodes in changectx
Summary:
changectx takes an arbitrary identifier and tries to resolve it. If it
takes a binary node and the node is not known, the fallback looksup don't accept
bytes and throw an exception. Let's stop early and raise the RepoLookupError.

Reviewed By: quark-zju

Differential Revision: D19643834

fbshipit-source-id: cfee39e0487f908ebd4ff628b847e53dec13c66e
2020-01-30 14:39:41 -08:00
Durham Goode
cfdf7814f4 py3: fix pipe reads in sshpeer
Summary:
In Python 3, subprocess creates FileIO type pipes, which do one syscall
and are not guaranteed to return 'size' bytes. In Python 2 they're 'file' types
which are guaranteed to return 'size' bytes (via multiple syscalls).

So let's handle that case.

Reviewed By: quark-zju, xavierd

Differential Revision: D19648035

fbshipit-source-id: 7d04a67acc569a55d3a855fc8521f1b8261f4636
2020-01-30 13:55:21 -08:00
Durham Goode
22b2f5becc py3: fix encoding of remotenames at write time
Summary:
remotenames are stored as unicode in memory and bytes on disk. We need
to convert before we write in this case.

Reviewed By: quark-zju

Differential Revision: D19648032

fbshipit-source-id: 4b96cf05eefd66ca031436d9979b94da8e28b8dd
2020-01-30 13:55:20 -08:00
Durham Goode
5311ab2fbb py3: fix encoding of pushkeys
Summary: Let's decode pushkeys into utf8.

Reviewed By: quark-zju

Differential Revision: D19648033

fbshipit-source-id: 9deeeecd53e065df9b344f8edd0183dc7c1511c5
2020-01-30 13:55:20 -08:00
Stefan Filip
843f8a7f50 py3/cython: set the language_level to 3str
Summary:
Follow up to the previous diff setting language_level to 3str for
`edenscmnative/clindex.pyx`. Consistent behavior for all files.

Reviewed By: quark-zju

Differential Revision: D19634907

fbshipit-source-id: 779d6090b5660e109a37f9740299fb9623d8f34d
2020-01-30 12:52:06 -08:00
Carolyn Busch
6b24e9a69e py3: commandserver write and read bytes
Summary: Make the command server write and read bytes. Add type annotations.

Reviewed By: quark-zju

Differential Revision: D19642862

fbshipit-source-id: 2dfcd8f0c2e6124cc6afe1e85c0f5275e7fe5b1f
2020-01-30 11:12:10 -08:00
Durham Goode
f170b964be py3: fix known wireproto command
Summary:
It was iterating over what it thought was a string, but in python3 it's
a byte string, which results in iterating over integers. So known always claimed
the server knew every commit.

Reviewed By: quark-zju

Differential Revision: D19644837

fbshipit-source-id: 4e84a33b4d3e436006f93a5d8e1bdabf12e7d120
2020-01-30 10:59:01 -08:00
Stefan Filip
45fa522e1c py3/fsmonitor: use next(iter) instead of iter.next()
Summary: iter.next() is gone in Python 3. next(iter) is the Python2 compatible version.

Reviewed By: quark-zju

Differential Revision: D19630455

fbshipit-source-id: e5d4f0a6cc0c6b0489916ca6d177a6831bd7d277
2020-01-30 10:28:02 -08:00
Adam Simpkins
dbaac4f0e7 py3: fix some Python 3 compatibility issues in sshpeer
Summary:
Fix `threadedstderr` to stop directly deriving from `threading.Thread`.
Deriving directly from `threading.Thread` is somewhat tricky.  Classes that
derive from `Thread` are only allowed to implement `__init__()` and `run()`.
This class also defined `close()`, which violates the `Thread` docs.  Changing
this to no longer derive from `Thread` fixes exceptions being thrown when
trying to read from the SSH stderr pipe in this thread.

This also updates `_writessherror()` to decode the stderr data before writing
it to the `ui` object.

Reviewed By: quark-zju

Differential Revision: D19634265

fbshipit-source-id: bddd120c7995f49d3bbebbb171384d5b15b779e9
2020-01-30 10:22:30 -08:00
Durham Goode
3ed7b6e3f2 py3: fix journal to work with utf8
Summary: Various fixes to make it stop crashing

Reviewed By: quark-zju

Differential Revision: D19643879

fbshipit-source-id: 9dd0bcef5369b769a8c968ea86de0e8aaf640650
2020-01-30 09:58:26 -08:00
Durham Goode
9aa1e8d7a3 py3: fix sparse utf8 logging
Summary:
Sparse logs the profile to scuba, but our logging data must be utf8.
Let's encode it first. We probably want to make this "replace" instead of
"strict" for the conversion once D19634265  lands.

Reviewed By: quark-zju

Differential Revision: D19643832

fbshipit-source-id: 550c67f194171ccdbd244bc79edfa1b999467824
2020-01-30 09:58:26 -08:00
Durham Goode
e3992926b7 py3: utf8 fixes for pull and wireproto
Summary: Moves pull closer to working by appropriately encoding and decoding.

Reviewed By: quark-zju

Differential Revision: D19643833

fbshipit-source-id: d311fb897d15594f2cb644b15038866ce506e011
2020-01-30 09:58:26 -08:00
Durham Goode
484a088417 py3: support caching in python3
Summary:
Revlog did a type check against str. In python 3 it accepts bytes
instead, so we need to support that.

Reviewed By: xavierd

Differential Revision: D19634882

fbshipit-source-id: 04ae908b4bfe8ed5de7a6d41cbe748dcb34b2540
2020-01-30 09:12:30 -08:00
Durham Goode
8b942a7166 remotenames: fix broken changelog contains check
Summary:
cl.__contains__ does a "0 < rev <= len(repo)" check, so passing nodes
will always fail. In python 3 this causes a type error, so let's fix this.

Reviewed By: quark-zju

Differential Revision: D19634880

fbshipit-source-id: 011ab2f0c07a16fa45734d360ac952273c4fefc2
2020-01-30 09:12:30 -08:00
Genevieve Helsel
247f686447 izip_longest to zip_longest
Summary:
izip_longest depreciated in python3 in favor of zip_longest, use zip_longest from future

source: https://python-future.org/compatible_idioms.html#itertools-filterfalse-zip-longest

Reviewed By: xavierd

Differential Revision: D19632101

fbshipit-source-id: 2eb3c125d2beaf66045dad304b900fe7928f2066
2020-01-30 08:50:41 -08:00
Jun Wu
cd04a56178 repo: avoid metalog when migrating files from sharedvfs to storevfs
Summary: Otherwise it can trigger metalog ProgrammingErrors.

Reviewed By: markbt

Differential Revision: D19624779

fbshipit-source-id: ba56a9aefa28d56a38ea8d966115647975613c91
2020-01-30 07:25:31 -08:00
Mateusz Kwapich
e4ee93b38b py: fixes for cloud sync
Summary: "hg cloud sync" succeeds when there's nothing to do

Reviewed By: quark-zju

Differential Revision: D19630431

fbshipit-source-id: e2c9859c7660e6283e6d319c634f1cd9053f8b3f
2020-01-29 18:25:41 -08:00
Mateusz Kwapich
892dcb3189 py3: make "hg cloud sl" work in py3
Summary: Some strings had to stay as unicode

Reviewed By: quark-zju

Differential Revision: D19628661

fbshipit-source-id: 6ea191c60a4ef2ed98b2e2d3ab90c875e6eea248
2020-01-29 18:25:41 -08:00
Zeyi (Rice) Fan
05d306bfc3 make test-extension.t py3 compatible
Summary: because of the test `type(b"") != type("")` we actually have different behavior in Python 3 and Python 2. So we remove this test temporarily and we can add it back later.

Reviewed By: sfilipco

Differential Revision: D19611682

fbshipit-source-id: 7bca979f8ef871672ac9666286c8d5e54747e8b3
2020-01-29 17:07:32 -08:00
Stefan Filip
56c0adbc46 py3/match: convert filter result to list
Summary:
In Python3 the result of filter is a class that does not have len
implemented. In Python2 the result of filter is a list.

Reviewed By: quark-zju

Differential Revision: D19629139

fbshipit-source-id: a7a1d9d831d72b2aab8b7a920bf82295c4673603
2020-01-29 17:03:05 -08:00
Stefan Filip
f80b7bd6c4 py3/sparse: decode sparse profile as utf-8
Summary: File contents are read as bytes but content should be interpreted as utf-8.

Reviewed By: quark-zju

Differential Revision: D19628315

fbshipit-source-id: 523a201ddf57a2aabcd135156e71034c52b59adb
2020-01-29 17:03:04 -08:00
Stefan Filip
ac4aaab5a2 py3/util: treat authentication url as str
Summary: Python3 socke address is str.

Reviewed By: quark-zju

Differential Revision: D19628130

fbshipit-source-id: d6ce93d28588a9ab8829f5b95db30ecee040a160
2020-01-29 17:03:04 -08:00
Mateusz Kwapich
3cfb8a6a7c py3: node: get rid of extra encoding
Summary:
The extra call to encode makes everything a big slower, fortunately python
standard library provides functions that do the right thing.

Reviewed By: xavierd

Differential Revision: D19634034

fbshipit-source-id: 6bb0de26300c41667824422c016e8d4f0aee70d3
2020-01-29 16:48:41 -08:00
Xavier Deguillard
467b3095c1 eden_dirstate_map: encode the path
Summary: This enables `hg add` on EdenFS

Reviewed By: quark-zju

Differential Revision: D19631538

fbshipit-source-id: 1621ab77fef977c9364052bc522df18729face48
2020-01-29 16:45:46 -08:00
Durham Goode
13b96565e5 py3: convert parts of undo.py to specify utf-8
Summary: This makes hg pull make more progress.

Reviewed By: mitrandir77

Differential Revision: D19630124

fbshipit-source-id: 6e8d6bd441353ffbf87076addd2774d5e09dd24a
2020-01-29 16:41:54 -08:00
Durham Goode
654ed5e4b1 remotenames: prefer picking default paths
Summary:
S194477 was caused by a config change adding a custom path to
repositories, where the path had the same url as default. This caused all the
remotebookmarks to be associated with the new path instead of default.

This is subtle and undesirable. Let's make it so that the default path takes
precedence.

I also deleted a hggit test because it was causing the entire test to be skipped
when dulwich wasn't available.

Reviewed By: quark-zju

Differential Revision: D19627711

fbshipit-source-id: 55b3ce94f18a26ccb9471c8e96a584d3087dbb82
2020-01-29 15:31:16 -08:00
Simon Farnsworth
c87bac85e1 py3: streamclone: make wireprotocol bytes
Summary: The wireprotocol for streaming clones should be explicitly bytes, encoding filenames as UTF-8 when needed. Make it so

Reviewed By: quark-zju, xavierd

Differential Revision: D19616097

fbshipit-source-id: 3513c801035c8d7ba950777aa119e1d1480013a5
2020-01-29 15:20:56 -08:00
Jun Wu
07a1c0a776 py3: connectionpool: fix some Python 3 issues
Summary: `im_self` is gone. Use `__self__` instead.

Reviewed By: xavierd

Differential Revision: D19630121

fbshipit-source-id: aa1af51e89cdffcfc63f3c18eeaf04d35425160e
2020-01-29 14:27:12 -08:00
Jun Wu
552c1dfabd py3: sshpeer: fix some encoding issues
Reviewed By: mitrandir77

Differential Revision: D19630030

fbshipit-source-id: b700187aeb412ac85e81532a525601454c45c426
2020-01-29 14:27:12 -08:00