Commit Graph

672 Commits

Author SHA1 Message Date
Durham Goode
255b677f91 py3: fix test-fb-hgext-remotefilelog-lfs.t
Reviewed By: quark-zju

Differential Revision: D19772621

fbshipit-source-id: 25fda0f356129b27413b25cfb70e9d08bbc56263
2020-02-17 14:52:35 -08:00
Durham Goode
a8b4789109 py3: fix misc encoding errors that showed up in tests
Summary:
These are a random batch of stack traces that show up in remotefilelog
tests.

Reviewed By: quark-zju

Differential Revision: D19772619

fbshipit-source-id: a8b9ce188cb7a5a2c9ccaeb62f6744f1c4083e38
2020-02-17 14:52:34 -08:00
Durham Goode
41cc67d011 py3: fix remotefilelog repack tests
Reviewed By: quark-zju

Differential Revision: D19751790

fbshipit-source-id: a898a8c37929a73ee2654d2907e9ea5bb2e3fd8a
2020-02-17 14:52:33 -08:00
Durham Goode
3a5728b32f py3: fix archive
Summary:
archive uses a formatter to produce it's metadat file. We need to use a
string io stream instead of bytes.

Reviewed By: quark-zju

Differential Revision: D19748163

fbshipit-source-id: fbd8c32066cfc4a234d9b51691717c7fce4c7c9a
2020-02-17 14:52:33 -08:00
Durham Goode
48aec8ca41 py3: misc fixes to remotefilelog and treemanifest
Reviewed By: quark-zju

Differential Revision: D19747715

fbshipit-source-id: da6716ff46342d777d45bec8e560ab41d544645f
2020-02-17 14:52:33 -08:00
Durham Goode
150aa7be3d py3: fix bundle2 bookmark default values
Summary: They need to be bytes to match the nodes.

Reviewed By: xavierd

Differential Revision: D19746027

fbshipit-source-id: 41e9cc390f4aa97d8c8b378144e64100811665c2
2020-02-17 14:52:32 -08:00
Durham Goode
6d40f2c380 py3: fix occasional ssh hang
Summary:
sshpeer._calltwowaystream used iter(fp, ""), where "" was the sentinel
indicating the fp was empty. Since fp was an iterator of bytes, the sentinel was
never hit and this code kept sending 0\n to each other until the pipes filled
and it hung.

Reviewed By: xavierd

Differential Revision: D19746028

fbshipit-source-id: 0daada0ae2356b5c99cc2c39c121cd88af8f750a
2020-02-17 14:52:32 -08:00
Durham Goode
a3ec80f7f5 py3: various fixes to move bundle2 tests foward
Reviewed By: quark-zju

Differential Revision: D19717253

fbshipit-source-id: 3f07a3c6ff09c1f7081d2dd15659505b4a07a91a
2020-02-17 14:52:32 -08:00
Durham Goode
67ade10253 py3: fix minor bookmark and sparse encoding issues
Reviewed By: quark-zju

Differential Revision: D19717254

fbshipit-source-id: 1ee7fab0bec1bb5b42c2707e7bc07aa8300f4be7
2020-02-17 14:52:31 -08:00
Durham Goode
4d4274ce53 py3: make conduithttp.py python 3 compatible
Reviewed By: quark-zju

Differential Revision: D19717252

fbshipit-source-id: 075aaed642898a3649d40f9cdc9752539b0970aa
2020-02-17 14:52:31 -08:00
Durham Goode
66fd50346c py3: add pycompat buffer alias
Summary: buffer in Python 2 became memoryview in Python 3.

Reviewed By: quark-zju

Differential Revision: D19717251

fbshipit-source-id: f678abfed7e2ec94053329b13015bff3c31eb319
2020-02-17 14:52:30 -08:00
Durham Goode
e4b5247311 py3: fix test-cat.t
Summary:
Fixes test-cat.t by changing some formatter encoding choices. The
formatter is bit awkward here.  hg cat needs to support outputting raw binary,
but also json and templated output.

For now I've set it up so json and templated output can't output non-utf8 data.

Reviewed By: quark-zju

Differential Revision: D19786542

fbshipit-source-id: 84060928103b396b23e3173b715aed996074fa3e
2020-02-17 14:52:30 -08:00
Durham Goode
734ce7959f py3: fix various lfs encoding issues
Reviewed By: quark-zju

Differential Revision: D19717255

fbshipit-source-id: d435a65e566b3a74399632e52c7cd99d0bba7f0a
2020-02-17 14:52:30 -08:00
Durham Goode
9d09b429f0 py3: migrate treedirstate to use utf8 for state and files
Summary:
treedirstate was using bytes for state and for file paths. This does
the appropriate conversions. Note, I don't use strings for all state in rust,
because it's a pain in the butt in some places. We're going to delete
treedirstate eventually anyone, so just I'm getting the test to pass for now.

Reviewed By: quark-zju

Differential Revision: D19715102

fbshipit-source-id: 4f9eac372bee7884d36aa19e3a3ed253392fa7dd
2020-02-17 14:52:29 -08:00
Mark Thomas
892ac0079e rage: add debugmutation for recent draft commits
Summary:
Add `debugmutation` output for recent draft commits to `hg rage`.  This will
allow us to easily see the recent history of the draft commits in a user's
repo.

Reviewed By: DurhamG

Differential Revision: D19903999

fbshipit-source-id: f6ebd729812c63d3760f9dab031414df6b24ab28
2020-02-17 06:41:04 -08:00
Mark Thomas
3690e8ab44 debugmutation: add time range limiting
Summary:
Make it possible to limit the time range of mutation info being displayed by hg
debugmutation.

Reviewed By: DurhamG

Differential Revision: D19904000

fbshipit-source-id: 365f54fdd861661961bba1a0ea96fce772623a23
2020-02-17 06:41:03 -08:00
Mark Thomas
5122958c70 commitcloud: fix program name template in commit cloud warning
Summary: The correct template for the program name is `prog@`.

Reviewed By: HarveyHunt

Differential Revision: D19904938

fbshipit-source-id: 19a3ac18f85e2b090f53d0423fce3c6982cf32fd
2020-02-17 02:37:26 -08:00
Xavier Deguillard
7bb3e384d8 remotefilelog: append the repo name to memcache key
Summary:
Up to now, this has been done in chef, and thus for repos that we do not list,
they may share the memcache keys, with potential unintended consequences. Let's
always add the repo name to the key, so we can simplify the code in chef.

One small negative effect of this change is that while it is being rolled out,
the cache hit rate will be impacted. This should resolve itself quickly.

Reviewed By: DurhamG

Differential Revision: D19885775

fbshipit-source-id: 0b59ce9e378b0ab70f696a39d19d27cd89921098
2020-02-14 14:10:48 -08:00
Mark Thomas
7f71cd1cda commands: pass date-based rev as a hex hash
Summary:
When using the `--date` parameter to `hg update` or `hg revert`,
`cmdutil.finddate` returns the binary node for the target commit.  Passing this
to `scmutil.revsingle` sometimes works, however it's unreliable, as if the
binary node happens to look vaguely like a revset, we will attempt to parse it,
and perhaps fail.

Resolve the ambiguity by converting the binary node to hex.

Reviewed By: quark-zju

Differential Revision: D19902595

fbshipit-source-id: 5eb7b9b029e292a02ccc00f5a465ab7807cd56d5
2020-02-14 09:39:55 -08:00
Jun Wu
d1750883cb py3: sshserver: detect client close correctly
Summary:
This is needed by the next diff. Otherwise Python 3 ssh tests in the next diff
will hang.

Reviewed By: DurhamG

Differential Revision: D19879882

fbshipit-source-id: ecc317d0685993c6b1bef8c72068bf4315030d0f
2020-02-13 12:16:22 -08:00
Jun Wu
594f668295 connectionpool: add a debug note saying connection being reused
Summary:
I'm going to change the connection pool logic but I'm not sure where it gets
used. This change exposes at least one test using it.

Reviewed By: xavierd

Differential Revision: D19872614

fbshipit-source-id: 4921b92c3fe3fd7ba1a72de17eef92604964eb2e
2020-02-13 12:16:21 -08:00
Kostia Balytskyi
a3f1d6ae64 clienttelemetry: report multiple server_hostnames
Summary:
Right now, if the client establishes connection to more than one peer, the
last one to proces `clienttelemetry` wireproto command gets the honor to set
`server_realhostname`. This is not desirable. Specifically, when we have
`fallbackpath` set up for remotefilelog/treemanifest and the prefetch happens
after pulling a commit, we get the hostname of the fallbackpath server,
while losing the hostname of the original `getbundle` server.

Reviewed By: DurhamG

Differential Revision: D19837570

fbshipit-source-id: fdc41565a5dfe670df3caf3b034196c4b7bdf6d9
2020-02-13 00:48:25 -08:00
Stanislau Hlebik
12c2453526 remotenames: log when a node is missing
Reviewed By: DurhamG

Differential Revision: D19832734

fbshipit-source-id: 2861b05cc754b3e3865dee706290fe3bed08d336
2020-02-13 00:23:54 -08:00
Xavier Deguillard
fd5c00465d remotefilelog: re-use the memcachestore
Summary:
Don't rebuild the memcache store everytime. This avoid spawning a thread and
reconnecting to memcache everytime.

Reviewed By: DurhamG

Differential Revision: D19797911

fbshipit-source-id: 7ecfa111a98ab67c4791b91410a22e458fe169f6
2020-02-11 18:05:16 -08:00
Jun Wu
1714cef464 remotefilelog: remove bgprefetch feature
Summary:
We don't use the feature in production. The main motivation is to remove the
flaky test. People wanting this feature can use normal hooks instead.

Reviewed By: xavierd

Differential Revision: D19830033

fbshipit-source-id: 0b1df7bb6c5de116ee27df4c38071912568d893e
2020-02-11 10:25:34 -08:00
Jun Wu
3279babd74 nointerrupt: remove the extension
Summary:
We never actually enabled it in production, and our new storage (ex. metalog,
indexedlog) are SIGKILL-safe. Only the changelog (revlog) might be subject to
double Ctrl+C issues. And we have debugfixcorrupt which can fix changelog
corruptions. So nointerrupt becomes less important.

The direct motivation is the test has been too flaky recently (and it's hard to
reproduce locally).

Differential Revision: D19830032

fbshipit-source-id: 87722d730fab49efbfd231772b1a5dddf8ec0d7b
2020-02-11 10:25:34 -08:00
Stefan Filip
f3b41ee3a9 warn: remove large file warning
Summary: Not very useful on today's hardware.

Reviewed By: quark-zju

Differential Revision: D19783741

fbshipit-source-id: d8ff29486cdc5f8e31d929d9fbba827e37012381
2020-02-11 10:18:06 -08:00
Durham Goode
2a975077f5 py3: don't assert on byte types in python 2
Summary:
We don't have confidence that python 2 is only using bytes (vs unicode
or other byte-like objects). These asserts break users, so let's disable them
except for in tests. We should uncover issues here as we port to python 3.

Reviewed By: xavierd

Differential Revision: D19819699

fbshipit-source-id: 9e3a9c34e7661fac4db9ee2a79f65be3d5e48cb3
2020-02-11 09:38:55 -08:00
Durham Goode
f918387338 py3: fix graphql producing unicode blobs in python 2
Summary:
As part of the py3 migration we've started enforcing that encodeutf8 is
only called on bytes. graphql is returning unicode, so we need to change it to
return bytes on python 2.

Reviewed By: xavierd

Differential Revision: D19819701

fbshipit-source-id: 433ced2da87d522e4234ff453d11cacfaeebdd8d
2020-02-11 09:38:55 -08:00
Durham Goode
0ccfdfdb6e py3: switch blackbox to use pycompat.encodeutf8
Summary:
ui.log apparently receives non-utf8 data occasionally, which makes the
encoding fail. Let's change this to not encode on python2 (by using pycompat).
As we migrate to python 3 all the strings should become unicode, and this will once
again enforce utf8 encoding

Reviewed By: xavierd

Differential Revision: D19819700

fbshipit-source-id: 66da26d00d1b1c4286e325f0574ad7ff478e5d1f
2020-02-11 09:38:54 -08:00
Stanislau Hlebik
430427d51c commitcloud: try to update working copy even if there's nothing to sync
Reviewed By: markbt

Differential Revision: D19814471

fbshipit-source-id: 0c9eb2891a3cf06ddac8d6d492f5487a0c80e969
2020-02-11 03:20:53 -08:00
Lukasz Piatkowski
542d1f93d3 Manual synchronization of fbcode/eden and facebookexperimental/eden
Summary:
This commit manually synchronizes the internal move of
fbcode/scm/mononoke under fbcode/eden/mononoke which couldn't be
performed by ShipIt automatically.

Reviewed By: StanislavGlebik

Differential Revision: D19722832

fbshipit-source-id: 52fbc8bc42a8940b39872dfb8b00ce9c0f6b0800
2020-02-11 11:42:43 +01:00
Jun Wu
3ac689c81f context: do not print verbose KeyError context if the exception is handled
Summary:
Attach the content to the exception object directly so if the exception is
handled, no verbose message will be printed.

Reviewed By: DurhamG

Differential Revision: D19817404

fbshipit-source-id: 9b79952559d07eb62bd7eb2398c6dd022ffe7b28
2020-02-10 22:21:44 -08:00
Durham Goode
8902d630b2 tracing: upload stack traces and link it to our main data
Summary:
Our main data table has no information about the actual failure. Let's
upload a stack trace to blob storage and link to it from the main table.

Reviewed By: quark-zju

Differential Revision: D19780653

fbshipit-source-id: acaab29a47d32dd6ab81fed0c3ae50c8eb44b760
2020-02-07 14:04:38 -08:00
Durham Goode
6e338a97d4 remotefilelog: recreate cache process after fork
Summary:
If the main process forks, the connection to the cache client is likely
unusable. Let's drop that connection and we can recreate it later if needed.

Reviewed By: quark-zju

Differential Revision: D19796387

fbshipit-source-id: 59b0d3db9655d2233b55effcdf18cbd74a3f5edd
2020-02-07 13:20:30 -08:00
Durham Goode
dc9da8ef2e connectionpool: throw away the connection pool after forking
Summary:
We're seeing an error where worker processes fail to fetch data from
the server. Our theory is that the forking messes up the ssh connection, so
let's not reuse ssh connections across forks.

Reviewed By: xavierd

Differential Revision: D19794281

fbshipit-source-id: 3954cd96617fd18ecf0d0aa9b9a6ce774c494067
2020-02-07 11:45:57 -08:00
Genevieve Helsel
0a2fd56bba py3 blame
Summary: plain hg blame for non-binary files

Reviewed By: quark-zju

Differential Revision: D19676073

fbshipit-source-id: 203b0421c6279ea1ca6bb6e253e7952e64b5edeb
2020-02-07 03:19:41 -08:00
Genevieve Helsel
4acc40424c hex printing on revlog match failure
Summary: fixes printing for no match found in revlog match

Reviewed By: quark-zju

Differential Revision: D19652374

fbshipit-source-id: 55bfcbff7919ad817173241e2792df27311b931e
2020-02-07 03:17:06 -08:00
Jun Wu
787cebfe91 context: print more details on fctx.data() key error
Summary:
Print out the name of the commit and the stack.
Hopefully this can help making debugging KeyErrors easier.

Reviewed By: DurhamG

Differential Revision: D19776181

fbshipit-source-id: 2eb985dd5355732a4d7728af68eb16173c48caa5
2020-02-06 15:56:26 -08:00
Durham Goode
309a3a0592 py3: fix commitcloud lock output
Summary:
The use of json meant the progress step was coming out as unicode when
it should be str. Use the mercurial.json functions to solve this for python 2
and 3.

Reviewed By: xavierd

Differential Revision: D19777255

fbshipit-source-id: 15c8e45425fc8742b6e118249104fc1fb2f3345d
2020-02-06 13:23:21 -08:00
Xavier Deguillard
b8947748b5 pyrevisionstore: expose the memcache client to python
Summary:
This allows the Python code to build a memcache client and build ContentStore
and MetadataStore with it.

Reviewed By: DurhamG

Differential Revision: D19518694

fbshipit-source-id: d932fd5223ccfdf37db69cbb54a11a6571312709
2020-02-05 14:01:54 -08:00
Xavier Deguillard
b192e0b54b demandimport: re-enable
Summary:
Partially backport upstream
https://www.mercurial-scm.org/repo/hg/rev/f81c17ec303c to enable lazy loading
of python code contained in edenscmdeps3.zip.

Also, temporarily disabling the demandimport on Python3 is a bit tricky, for
the reasons mentioned in the deactivated function. Thus, instead of using the
disabled function, let's use the deactivated one.

Reviewed By: DurhamG

Differential Revision: D19672866

fbshipit-source-id: c9e39ed044121d962af1cc46745bdec72629c579
2020-02-05 11:23:29 -08:00
Michael Devine
0dceab8e95 New custom immutable revision object to track commit conversions: conversionrevision
Summary:
The Mercurial convert extension passes around parameters to indicate a commit that needs to be converted from source to sink. For existing converters like Git, this is a simple 1:1 conversion: a commit in the source gets mapped to a commit in the sink, and so they use the source commit hash (sometimes called rev or version in the API) to represent the commit to be converted.

Our converter is much more complicated. Source commits get converted multiple times to account for different ways of mounting it into the destination file system and commit history. The commits are also coming from multiple source Git projects. This means that we need multiple pieces of data to represent a single commit conversion action.

Thus far, we've been trying to meet part of this need by using concatenated strings of (variant, commithash). This logic is breaking down as we add more fields. This commit adds a new immutable object called "conversionrevision" that represents the (variant, source commit hash, source project name, destination path) that is the unique identifier for the individual commit conversions we need to perform. This commit also includes logic for serializing and deserializing these objects as strings (useful because the converter seems to require commit IDs to be strings) and unit tests for all of the new logic.

Reviewed By: tchebb

Differential Revision: D19606867

fbshipit-source-id: 77815ca858f841d452874e95dfa3b351bafde306
2020-02-05 10:20:19 -08:00
Jun Wu
1bcfec407d py3: remove pycompat3
Summary:
Use `abc.ABC` from Python 3 stdlib directly. The definition matches
`pycompat3.py`:

  class ABC(metaclass=ABCMeta):
      pass

The following changes are reverted since they're no longer necessary:

  D19732319 "[hg] py3: fix windows build"
  D19703778 "[hg] py3: exclude mercurial/pycompat3.py from Python 2 builds"
  D19703779 "[hg] py3: exclude pycompat3.py from Buck-based Python 2 builds"

Reviewed By: simpkins, singhsrb

Differential Revision: D19739075

fbshipit-source-id: 8c1e3727e8a88ff5f7232270d528d690523b1824
2020-02-04 22:15:16 -08:00
Saurabh Singh
5c37bfd373 nointerrupt: make output more stable
Summary:
Printing via `ui` object instead of directly to the `stderr` makes the
output more consistent.

Reviewed By: quark-zju

Differential Revision: D19738355

fbshipit-source-id: 384bf2de11f2ae7b5264ec5d94b041502c4ef7fb
2020-02-04 20:36:42 -08:00
Carolyn Busch
d49c251d13 py3 iteritems -> pycompat.iteritems
Summary: More replacements of dict.iteritems with pycompat.iteritems(dict) for py3 compatibility.

Reviewed By: singhsrb

Differential Revision: D19704211

fbshipit-source-id: 544cb292c0241a293fc4ab83c3d9472dcf4328b6
2020-02-04 15:29:39 -08:00
Saurabh Singh
3d78cca814 util: remove redundant 'timed' function
Summary:
`timed` has been replaced by the `timefunction` in util.

I have also removed the existing usages because they seemed relevant only
during the development.

Reviewed By: quark-zju

Differential Revision: D19727919

fbshipit-source-id: 58c466f02ac2a5bf7096948b17b49ceb26fc1fd9
2020-02-04 12:33:40 -08:00
Durham Goode
16efd0a0e0 py3: fix osx tests
Summary:
The path normalization OSX logic was using unicode, and when it
switched to use pycompat in D19613690 it stopped using unicode, which broke it.
Let's roll it back.

Reviewed By: singhsrb

Differential Revision: D19726322

fbshipit-source-id: fa7bfacdf41dfedbcfef2fdb5ccb826196101e61
2020-02-04 12:19:13 -08:00
Adam Simpkins
0e1437c823 py3: add some additional type annotations on functions writing file data
Summary:
Add a few additional type annotations on some functions.
This did require some minor code restructuring in `filestore.setfile()` to
work around pyre's unwillingness to unwrap `Optional` member variables.

Reviewed By: quark-zju

Differential Revision: D19715243

fbshipit-source-id: 8078329c1e4a50ad0aa6c765d42a89b5ed58e7bf
2020-02-03 21:36:56 -08:00
Adam Simpkins
0657c35391 py3: add more vfs-related type annotations
Summary:
Add more type annotations to vfs.py and some of the platform-specific
functions that it calls.

This did catch a couple of cases where string/bytes conversion were not being
done properly in Python 3.

Reviewed By: quark-zju

Differential Revision: D19678717

fbshipit-source-id: b3979cc5e0668ddbb93372dee25ef02ea1867d6f
2020-02-03 21:36:56 -08:00
Jun Wu
9602130aab py3: chg: fix compatibility
Summary:
Fix various type issues when running chg under Python 3.
Enable chg in setup3.py build.

This should make tests run faster. For example, test-rebase-detach.t
now completes in 8 seconds, down from 29 seconds.

Reviewed By: xavierd

Differential Revision: D19702535

fbshipit-source-id: 8928b1b920b9b52fd03dc86f996da18f2405f146
2020-02-03 18:26:57 -08:00
Jun Wu
3e0b781197 py3: only use binary stdin/stdout/stderr
Summary:
Drop stdoutbytes/stdinbytes. They make things unnecessarily complicated
(especially for chg / Rust dispatch entry point).

The new idea is IO are using bytes. Text are written in utf-8 (Python 3) or
local encoding (Python 2). To make stdout behave reasonably on systems not
using utf-8 locale (ex. Windows), we might add a Rust binding to Rust's stdout,
which does the right thing:
- When writing to stdout console, expect text to be utf-8 encoded and do proper decoding.
- Wehn writing to stdout file, write the raw bytes without translation.

Note Python's `sys.stdout.buffer` does not do translation when writing to stdout console
like Rust's stdout.

For now, my main motivation of this change is to fix chg on Python 3.

Reviewed By: xavierd

Differential Revision: D19702533

fbshipit-source-id: 74704c83e1b200ff66fb3a2d23d97ff21c7239c8
2020-02-03 18:26:57 -08:00
Durham Goode
16d4739981 fsmonitor: catch watchman issue and fallback
Summary:
My earlier refactor moved a watchmanclient.getcurrentclock() call out
of a "try/except Exception" block. This meant any watchman errors broke the hg
status entirely, instead of falling back to the old status path.

Let's add the logic around this watchman path.

Reviewed By: sfilipco

Differential Revision: D19709391

fbshipit-source-id: bfb0221ba405a926babddaf73c8e70924e4d9b25
2020-02-03 18:04:25 -08:00
Jun Wu
2777fea957 py3: localrepo: manually fixup some type annotations
Summary: This might help pyre check more things.

Reviewed By: farnz

Differential Revision: D19678144

fbshipit-source-id: a04207ccb04ce76796e9a0a7326a8d527d17cff1
2020-02-03 17:07:36 -08:00
Jun Wu
165a8d4f8d py3: add stub for localrepository
Summary:
The "repo" type is a bit hard to type since it might be wrapped by extensions,
and the decorators like "unfilteredmethod" are too hard for type checkers.

Provide a stub for it so typecheck can work. The stub was generated by `pytype`,
I made a small change to `close` to make pyre happy.

Reviewed By: DurhamG

Differential Revision: D19677410

fbshipit-source-id: 71afb3fe78a75ee269c8005d18eee3a807b50df6
2020-02-03 17:07:36 -08:00
Aida Getoeva
38fca29eaf hg-py3: fix restack-multidest/t and fold
Reviewed By: xavierd

Differential Revision: D19700265

fbshipit-source-id: 557cec8388d5c7759d24e1b995f971e72f925a30
2020-02-03 17:05:35 -08:00
Aida Getoeva
acc0425925 hg-py3: make amend-next/prev work
Reviewed By: xavierd

Differential Revision: D19700028

fbshipit-source-id: 9aaab361058a2dfa34f4d0c9218c71c1654aaa8a
2020-02-03 17:05:34 -08:00
Adam Simpkins
b6ce7a08a6 py3: make vfs.join() consistent across subclasses; add type hints
Summary:
Add type hints for `vfs.join()`.  Not all subclasses supported joining
multiple arguments, so fix them all to actually implement the full API.

Reviewed By: farnz

Differential Revision: D19678712

fbshipit-source-id: f7f77eee416cd5edda756a8423cbe3630e195d4a
2020-02-03 15:27:52 -08:00
Sam Uong
974db0c7cd hgext: add hg equivalent for 'git stash clear'
Reviewed By: farnz

Differential Revision: D19669269

fbshipit-source-id: 2d33aae66b1733b9085adb829b38a07afd80fdd8
2020-02-03 12:59:46 -08:00
Aida Getoeva
201a17ca37 hg-py3: make amend-to work
Reviewed By: quark-zju

Differential Revision: D19673900

fbshipit-source-id: cfdc985716aec23335eeddef517855be65505576
2020-02-03 10:04:38 -08:00
Jun Wu
a8f232e0ab py3: drawdag: fix some types
Summary: This fixed using drawdag to create merge commits.

Reviewed By: xavierd

Differential Revision: D19697893

fbshipit-source-id: fac4278a54d2cf2eb2b9d8154a13edd29e1a510a
2020-02-03 09:54:12 -08:00
Simon Farnsworth
ad40d8567f py3: fix up matchers to exclusively use unicode
Summary: Matchers were mixing str and bytes - use decodeutf8 on the place using bytes, and enable all the tests that now pass.

Reviewed By: quark-zju

Differential Revision: D19679076

fbshipit-source-id: 2b90cfbf690b2365a1302efd9db72347a295c977
2020-02-03 08:00:11 -08:00
Jun Wu
18a20f5301 context: add some type annotations
Summary: This gives type checker some hints. Pyre can now look into more areas.

Reviewed By: markbt

Differential Revision: D19676085

fbshipit-source-id: db05b4238051953eb7f8af44da220825ec589c3c
2020-02-03 07:35:32 -08:00
Jun Wu
4e272b6fad revlog: add some type annotations
Summary:
This gives type checker some hints. Pyre was able to find some issues,
which are fixed.

Reviewed By: markbt

Differential Revision: D19674436

fbshipit-source-id: 44d6df6037f27e2edeff4d343a9fdec5e51420b3
2020-02-03 07:35:32 -08:00
Jun Wu
348ce20341 py3: add type annotations to i18n
Summary: This allows type checkers to understand the i18n._ calls.

Reviewed By: markbt

Differential Revision: D19673677

fbshipit-source-id: be2cc72f70e56925007485c786f0d72b5cd40a1a
2020-02-03 07:35:31 -08:00
Durham Goode
14ec2d4b34 py3: enable more tests
Summary: These now pass

Reviewed By: farnz

Differential Revision: D19678869

fbshipit-source-id: 68601ff6a6d33b13947210eecfecd999aa2ddb1a
2020-02-01 10:39:39 -08:00
Adam Simpkins
df74c3e8fa py3: start adding type hints in mercurial/vfs.py
Summary:
Begin adding type hints to many of the vfs methods.

These type hints indicate that the vfs objects are always opened in binary
mode.  While it is possible to open a vfs in text mode if you explicitly
specify the argument `text=True`, this is very rare.  If we run into cases
in the future that do need type hints to indicate TextIO mode I think it would
be better to split those out into completely separate functions, rather than
having functions that sometimes return bytes and sometimes return str based on
the arguments used.

Reviewed By: markbt

Differential Revision: D19673538

fbshipit-source-id: 8683223f28964d4b43fe131d4f16f8877dcbe777
2020-02-01 10:32:51 -08:00
Adam Simpkins
3c8739e179 py3: add type hints for util.atomictempfile
Summary:
Make `atomictempfile` derive from `typing.BinaryIO` so that the type checker
understands it can be used like a file.

This required defining all `BinaryIO` methods on this class.  This does result
in a fair amount of additional boilerplate code, but seems like the easiest
approach for now.

Reviewed By: markbt

Differential Revision: D19673539

fbshipit-source-id: b5775ca79b3af456b45e4ef5480711d7095e7949
2020-02-01 10:32:51 -08:00
Simon Farnsworth
eb5e4fb57e py3: fix share command
Summary: The share command was reaching into the filesystem directly - use the right utility functions instead

Reviewed By: quark-zju

Differential Revision: D19672980

fbshipit-source-id: a14323fd5419c3ea00d9c009b9a77f63862a7b2a
2020-01-31 22:13:04 -08:00
Mark Thomas
b36cdab6f9 py3: use new-style class for fd wrapper
Reviewed By: farnz

Differential Revision: D19672991

fbshipit-source-id: 06ce264f17b395c336c5bd936534e346d0b08a68
2020-01-31 21:28:47 -08:00
Adam Simpkins
7a61371a9d py3: move checkwinfilename() to a separate file and add type hints
Summary:
This moves `checkwinfilename()` from util.py to a new winutil.py module.
This allows the windows.py module to import winutil.py and re-expose
`checkwinfilename()` as `checkosfliename()`.  This makes pyre happier, since
all platform modules now contain a `checkosfilename` method.

This also adds type hints to `checkwinfilename()`, and updates several parts
of the code that were still incorrectly assuming the input was bytes in
Python 3.  However, note that I haven't done much testing of this on Windows.

Reviewed By: quark-zju

Differential Revision: D19672077

fbshipit-source-id: d78ed5b344a2e24c2a361b56465dedc516171d28
2020-01-31 20:18:21 -08:00
Durham Goode
3430fdd32f py3: treat sshserver wireprotocol commands and arguments as strings
Summary:
While the payload of wire commands are bytes, we should treat the
command and args as utf8.

Reviewed By: quark-zju

Differential Revision: D19675217

fbshipit-source-id: df64c842e0c099d77dec1313aa0639e46e539194
2020-01-31 18:39:42 -08:00
Durham Goode
10d673add5 py3: handle utf8 paths properly in remotefilelog
Summary: remotefilelog streaming clones need to handle paths as utf8 strings.

Reviewed By: quark-zju

Differential Revision: D19675219

fbshipit-source-id: dadf54fc67ecb6fbb23c1aeffd0c058ff5804bf5
2020-01-31 18:39:41 -08:00
Durham Goode
d37a246d60 py3: use bytes for stdin for hg serve
Summary: hg serve works strictly with bytes, so let's expose it for stdin.

Reviewed By: quark-zju

Differential Revision: D19672836

fbshipit-source-id: 6abc5385daea81c584a89f19a568e42e4afad48a
2020-01-31 18:39:41 -08:00
Durham Goode
d8fbaa7553 py3: fix sshserver to respond with bytes in most places
Summary: This moves streaming clones a bit further along.

Reviewed By: quark-zju

Differential Revision: D19672546

fbshipit-source-id: 054dd639a8067dce34a5bf667996cc5d30301190
2020-01-31 18:39:41 -08:00
Durham Goode
4e8759a594 py3: various fixes to batch and wireproto encoding
Summary:
We should generally be returning bytes on the wire. This fixes up a
number of places to do so, helping move streaming clone closer.

It also turns out that unescapearg is used in multiple situations.  Usually it's
used for strings, but for batches the actual payloads are also delimited by this
and they are bytes. So let's have a function for each case.

Reviewed By: quark-zju

Differential Revision: D19672545

fbshipit-source-id: 6386ae2f39660b042af1382a1d696903fa1de9ad
2020-01-31 18:39:40 -08:00
Durham Goode
3ce72efbc5 py3: misc fixes around streaming clones
Summary: A couple random stops that need correct encoding.

Reviewed By: quark-zju

Differential Revision: D19672547

fbshipit-source-id: edb0490f70cbf89885fe5fb0a1a8a2472d33e18d
2020-01-31 18:39:40 -08:00
Carolyn Busch
ba8ac287b8 py3 sshserver-test
Summary: Translate between bytes and unicode in sshserver to make test-sshserver pass.

Reviewed By: farnz

Differential Revision: D19672378

fbshipit-source-id: 7e1c0bc2d6fc03fac6ebcf54764862e3ed743f7e
2020-01-31 18:39:40 -08:00
Xavier Deguillard
ae089886fd setdiscovery: use the nodemap __contains__
Summary:
The changelog __contains__ expects rev numbers, not nodes, the nodemap should
be used instead.

Reviewed By: quark-zju

Differential Revision: D19674451

fbshipit-source-id: 8f0ddf3b9cebeea10f2fd639999f8c6c67a0e35d
2020-01-31 18:34:01 -08:00
Aida Getoeva
ad097305f5 hg-py3: make metaedit command work
Reviewed By: quark-zju

Differential Revision: D19666536

fbshipit-source-id: e8d02dd6ff6ab73efd94002b83319cbfe506c21f
2020-01-31 18:29:01 -08:00
Xavier Deguillard
fd2c463b9a rebase: decode rebasestate before parsing it
Summary:
This file contains text data, let's decode it before doing anything else with
it.

Reviewed By: quark-zju

Differential Revision: D19675385

fbshipit-source-id: 8aae08c2400dd7d76c8678d5312bab060563e0ba
2020-01-31 18:12:14 -08:00
Jun Wu
6462672163 pycompat: drop sysbytes, sysstr, strurl, bytesurl
Summary: There are no-ops.  This makes type checkers' life slightly easier.

Reviewed By: farnz

Differential Revision: D19673051

fbshipit-source-id: 363eb8a4e799113b16529825d45645807cafe2fc
2020-01-31 17:59:53 -08:00
Saurabh Singh
185119e3c0 test-extension: fix the test
Summary: This is broken after D19672686.

Reviewed By: quark-zju

Differential Revision: D19674321

fbshipit-source-id: 7f225b7090ce9168e10e02b25ee17ae2b98cd86a
2020-01-31 17:37:43 -08:00
Saurabh Singh
1891591abc unicode: consistently use 'utf-8' everywhere
Summary:
We generally use 'utf-8' but are using 'utf8' at some places. This
diff changes to using 'utf-8' consistently everywhere.

Reviewed By: farnz

Differential Revision: D19674111

fbshipit-source-id: 3be42859a180ded1c234b805259f086f9531ba18
2020-01-31 17:34:53 -08:00
Xavier Deguillard
57d59e5499 remotenames: write bytes to the file
Summary: The write method requires bytes, let's encode what we're writing into it.

Reviewed By: quark-zju

Differential Revision: D19674450

fbshipit-source-id: 9eaa3393d3d0e6324be39f94d7f31f2228b14412
2020-01-31 17:31:24 -08:00
Xavier Deguillard
7182c8c2ca hgdemandimport: use a set for the ignored modules
Summary:
A set gives us constant time test, while list requires a linear search. This
may speedup the importer slightly.

Reviewed By: quark-zju

Differential Revision: D19674449

fbshipit-source-id: 994a5303114d54b9cd40469b7cfe4b35e2515729
2020-01-31 17:31:23 -08:00
Xavier Deguillard
bf3292412a patch: add a missing b""
Summary:
The headers are bytes, therefore startswith needs to be called with a bytes
argument.

Reviewed By: quark-zju

Differential Revision: D19673675

fbshipit-source-id: 9a978ad42bf94a55c984f2ae35c566a59bfd6180
2020-01-31 17:31:23 -08:00
Adam Simpkins
bcad94171a py3: add pycompat.ABC
Summary:
Assigning to `__metaclass__` only works in Python 2.  In Python 3 this
attribute is ignored, and you instead need to specify `metaclass=` in your
class signature.

This adds a new `pycompat.ABC` base class to make it easier to define abstract
classes that are compatible with both Python 2 and 3.

Reviewed By: sfilipco

Differential Revision: D19672078

fbshipit-source-id: c56eb6c7266f962ef9c13c742e6ac0651b9c19c9
2020-01-31 17:05:54 -08:00
Adam Simpkins
3abc65f90d py3: add type annotations to ui.write()
Summary:
Add type annotations to `ui.write()` and the helper functions it calls.
I also removed unused `**opts` arguments on a few of the helper functions that
it calls.

Reviewed By: markbt

Differential Revision: D19657806

fbshipit-source-id: d020396a51d9fc3b07542c65541fca6067addb23
2020-01-31 17:05:54 -08:00
Jun Wu
5d3db79c23 pycompat: drop strkwargs / byteskwargs
Summary: There are no-ops.  This makes type checkers' life slightly easier.

Reviewed By: markbt

Differential Revision: D19673053

fbshipit-source-id: 04f26cd3ca0dbfe579ef939fc39be00035e70cee
2020-01-31 16:38:03 -08:00
Jun Wu
ae22d7d613 pycompat: drop fsencode / fsdecode
Summary:
There are basically no-op (see D19613693).
This makes type checkers' life slightly easier.

Reviewed By: farnz

Differential Revision: D19672686

fbshipit-source-id: 27ca44dd53ec9788fdbb8f31235eae88d1853415
2020-01-31 16:38:03 -08:00
Jun Wu
30f2b80d9f pycompat: drop iterbytestr
Summary: This makes type checkers' life easier.

Reviewed By: markbt

Differential Revision: D19672687

fbshipit-source-id: 335c560289e29ac7c08e36501e9841ddc51232be
2020-01-31 16:38:03 -08:00
Jun Wu
ad0c50a5cb py3: fix some type issues detected by pyre with some type stubs
Summary:
The issues were found by pyre with some type stubs generated via pytype:

  python36 -m pytype edenscm --no-report-errors -j 30

I didn't include the pytype generated stubs because most of them are `Any`.
I'm trying to see if we can get something cleaner.

Reviewed By: markbt

Differential Revision: D19672435

fbshipit-source-id: c57f2ad3a981ddd4a3a267ff1c00e7bdb71e65ca
2020-01-31 16:38:02 -08:00
Mark Thomas
df702d9cb6 py3: encode communication with the worker
Reviewed By: quark-zju

Differential Revision: D19665152

fbshipit-source-id: d4cda1e48bdf0fe64047e12d3011c81b501df113
2020-01-31 15:57:38 -08:00
Mateusz Kwapich
d5a207cf18 py3: fix more diff tests
Summary: This diff enables more test while hopefully not breaking anything

Reviewed By: markbt

Differential Revision: D19671316

fbshipit-source-id: 76ecd22a094c0a40b9724a3070a01e5d294888f3
2020-01-31 15:40:54 -08:00
Mark Thomas
4604332170 py3: cmdutil.export should encode its output when writing to a file
Reviewed By: quark-zju

Differential Revision: D19666638

fbshipit-source-id: 66cc3a9f6efbb421b31e18fb199e6478702eb950
2020-01-31 15:27:31 -08:00
Mark Thomas
ec02a1dc49 py3: decode undolog redonode
Reviewed By: quark-zju

Differential Revision: D19666639

fbshipit-source-id: e9f5760802750c06f965be0f370b9ed185abb79f
2020-01-31 15:27:31 -08:00
Xavier Deguillard
299212da50 checkmessagehook: make it python3 compatible
Summary:
The commit message should already be a utf8 string. Make sure that when it is
read from a file, we decode it properly.

Reviewed By: sfilipco

Differential Revision: D19666371

fbshipit-source-id: ee24fe33b5aaf1ede54286979c9523cedd901b3a
2020-01-31 15:21:55 -08:00
Xavier Deguillard
e83115ecee remotefilelog: fix hg_memcache_client protocol
Summary:
The pipe expects bytes, but the code was passing in strings. Since the protocol
is ascii only, we can encode/decode the str/bytes

Reviewed By: farnz

Differential Revision: D19666372

fbshipit-source-id: 095bf50b9e9529c4ea7d1a06ac01643bac47bca3
2020-01-31 13:51:43 -08:00
Jun Wu
e124dfebc6 pyre: add stub for "bindings"
Summary:
Add a partial stub so pyre stops complaining about "bindings" being unknown.

Pyre's "search_path" was adjusted to read the new "pystubs" directory.

Reviewed By: DurhamG

Differential Revision: D19669131

fbshipit-source-id: a592ed411d1689058405689eb657e543b7172774
2020-01-31 13:18:54 -08:00
Durham Goode
e985d51e02 py3: update simplemerge conflict logic to use bytes
Summary: Patches are bytes, so the lines we add to them need to be bytes

Reviewed By: mitrandir77

Differential Revision: D19668850

fbshipit-source-id: cb521f71d927dbe1cbe3287be37123b609889846
2020-01-31 13:12:15 -08:00
Durham Goode
4f35ef07bf py3: remotenames use writeutf8 for cached distance
Reviewed By: xavierd

Differential Revision: D19668611

fbshipit-source-id: 82d10a29e257a5779ef41675ab60b74cf97adaa3
2020-01-31 13:12:14 -08:00
Durham Goode
2eaaaa65bc py3: fix encodings for magic strings
Summary:
Fixes a few encoding issues for changegroup/bundle2 related issues. In
particular makes magic string comparison use strings.

Reviewed By: xavierd

Differential Revision: D19668613

fbshipit-source-id: c1bc8ab76d13bb04a9c440b5d9d9babfe34c5c92
2020-01-31 13:12:14 -08:00
Durham Goode
ff7266e7f5 py3: fix plural templating
Summary:
It decides whether to render the actual values by using a
isinstance(...,bytes) which isn't correct in python 3. Let's change it to str.

Reviewed By: xavierd

Differential Revision: D19668612

fbshipit-source-id: cf093754707a0a0369cfc3aa74270dd81a6560b5
2020-01-31 13:12:14 -08:00
Durham Goode
fae3d05bdb py3: encode activebookmark before putting it on the wire
Summary: The activebookmark is a string, the wire wants bytes. Encode it!

Reviewed By: farnz

Differential Revision: D19665896

fbshipit-source-id: f5afc1c5b17d90bf3f0d0703cd752d66c6f38778
2020-01-31 13:12:13 -08:00
Durham Goode
a876c6a13b py3: fix pushkey encoding by removing local encoding
Summary:
Now that bundle2 has strings everywhere, pushkey needs to also handle
encoding correctly. Previously it used local encoding, which seems strange.
Let's drop the encoding entirely since we're only dealing with since we're only dealing with utf8 string and bytes.

Reviewed By: farnz

Differential Revision: D19665894

fbshipit-source-id: ed054ab25636ac52de646329f26681ae843b6de3
2020-01-31 13:12:13 -08:00
Mateusz Kwapich
e2dc4e8014 diff: update for py3
Summary:
All diff functions are (bytes, bytes) -> bytes to preserver
the original file encoding.

Because of that I had to add ui.writebytes output function that accepts
bytes for terminal output.

Reviewed By: farnz

Differential Revision: D19656673

fbshipit-source-id: b9a1e4361e825fc8c2313e8402c2bbe00f490dd4
2020-01-31 13:00:23 -08:00
Xavier Deguillard
bf8fbe7d7f grep: make it run with python3
Summary:
The input/output of a subprocess is bytes, let's encode/decode when
writing/reading from it.

Reviewed By: farnz

Differential Revision: D19665636

fbshipit-source-id: 840f6b7a0edbedb7e7de5a65eacab9e1a7f78864
2020-01-31 12:47:04 -08:00
Xavier Deguillard
3bffbe2bf8 ui: remove a b from a regexp
Summary: I've seen this while running some tests, they now go slightly further.

Reviewed By: singhsrb

Differential Revision: D19666665

fbshipit-source-id: 35ad73f383de148f3d53c7f9de24b1a1e1c51f7e
2020-01-31 12:44:09 -08:00
Carolyn Busch
d84dc2c276 py3 cmdutil fix isinstance bytes
Summary:
The msg string isn't bytes in python 3. Hg revert works in both python 2&3
using 'str' instead.

Reviewed By: farnz

Differential Revision: D19664504

fbshipit-source-id: 5e16b637a5fb282eaa97390ef1f7e8912d0359f0
2020-01-31 11:39:13 -08:00
Durham Goode
3eca8e1086 py3: enforce bytes for bundle2 parts
Summary:
We enforced this at send time, but it was hard to debug where it was
coming from. Let's enforce it at part.setdata() time so the stack points out
exactly who the problem is.

Reviewed By: farnz

Differential Revision: D19665895

fbshipit-source-id: 5a6ca0263d52cac472e1ef2aca3ee702f38258e4
2020-01-31 11:29:29 -08:00
Stefan Filip
ae8a2613a6 py3/watchmanclient: use bser encoding
Summary:
The `bser-v1` encoding is not supported for Python 3 so we are forced to
update.

Reviewed By: quark-zju

Differential Revision: D19649948

fbshipit-source-id: 0ae1800942dd967eddcd58183f9899d36256598d
2020-01-31 11:14:47 -08:00
Durham Goode
984192ebef py3: fix hg pull
Summary:
D19657765+D19656773 made bundle2 use the standard urllib which turns
bytes into strings. This messes up bundle2 caps encoding and decoding. Caps
encoding/decoding is a nested mess of strings and bytes, so let's make it all
strings until the last minute it goes over the wire.

Reviewed By: quark-zju

Differential Revision: D19665131

fbshipit-source-id: a2fd6e309ed6da2b927d2f4ead97825a16bfcaac
2020-01-31 10:56:31 -08:00
Durham Goode
3df423a83b py3: fix histedit json parsing
Summary:
fbhistedit adds support for json histedit commands. Unfortunately
json.loads produces unicode strings, which then fail assertions in
pycompat.encodeutf8(...). Let's make sure it produces non-unicode on python2.

Reviewed By: singhsrb

Differential Revision: D19656390

fbshipit-source-id: 11b828319b20a153aa008f03d554087b88b7763c
2020-01-31 10:56:30 -08:00
Durham Goode
850c78d8c9 errors: drop quotes around OSError paths
Summary:
Python 3 has changed which errors are OSError vs IOError. We have
slightly different formatting which makes the test awkward to be py2 and 3
compatible. Let's make the formats the same.

Reviewed By: quark-zju

Differential Revision: D19663964

fbshipit-source-id: 892461049b61d648431f300bea0da2f6f1c70066
2020-01-31 10:56:30 -08:00
Xavier Deguillard
e0e01cbc81 bundle2: backout D19656773
Summary: This broke a bunch of tests. Revert it and re-enable all the tests.

Reviewed By: DurhamG

Differential Revision: D19665042

fbshipit-source-id: c3c17e3ac7e2ea028be5b5836bc8349cdf56184e
2020-01-31 10:48:19 -08:00
Aida Getoeva
af1d91abf0 hg-py3: changegroup - types and encoding
Summary: Added types, added decodeutf8 where needed while packing and unpacking chunks.

Reviewed By: xavierd

Differential Revision: D19626372

fbshipit-source-id: f15cbde5fd7e01a79b8591f0174f266078852a12
2020-01-31 10:10:56 -08:00
Mark Thomas
4c322218db hggit: only test fetch_pack.refs if it exists
Summary:
Different versions of dulwich return different types for
`localclient.fetch_pack`.  If the returned type is a real dict, then we can't
test the inner `refs` attribute, so check if it is there before testing if it
is `None`.

Reviewed By: singhsrb

Differential Revision: D19657503

fbshipit-source-id: 63cce72fa87f615e7b33279726dcb19c2fdf5b9e
2020-01-31 09:44:27 -08:00
Simon Farnsworth
4e10dbb9f6 py3: Back out "[hg] py3: re-enable demandimport"
Summary:
This breaks test-alias.t, so it looks like demandimport no longer behaves properly.

Original commit changeset: d56814049181

Reviewed By: simpkins

Differential Revision: D19658109

fbshipit-source-id: 9fd4865bd00d2307afe0c99f1500b6e08fe3e785
2020-01-31 00:14:04 -08:00
Adam Simpkins
1a359987cd py3: update urllibcompat.quote() to operate on strings again
Summary:
D19652082 changed `urllibcompat.urlreq.quote()` to always return bytes.
This broke most of the code outside of the bundle2 module that was expecting
it to operate on strings.

This changes `quote()` to operate on strings again, since this is what the
Python 3 standard library does.  This updates `bundle2` to handle encoding and
decoding properly when parsing strings that go through `quote()` and
`unquote()`.

This also changes bundle2 to stop going through the `urllibcompat.urlreq`
wrapper class, since that wrapper class prevents type checking from being
performed on its members.

Reviewed By: quark-zju

Differential Revision: D19657765

fbshipit-source-id: 0611838213ec22221e0b8367e1f0d8a6dcfceb15
2020-01-30 23:22:02 -08:00
Adam Simpkins
faf9de663d fix filelog to work with Python 2 again
Summary:
D19656501 added a usage of the `b"%b"` format specifier, which only works in
Python 3.  Change the code to use `b"".join()` for now, which works in both
Python 2 and Python 3.

Reviewed By: farnz

Differential Revision: D19657472

fbshipit-source-id: 22d3123f4939fa984ffd17516e1cc48bad2bb2b7
2020-01-30 20:59:49 -08:00
Saurabh Singh
13a2e1f8ed memcommit: refactor code to be pyre friendly
Summary:
This diff allows us to remove the `type: ignore` specification on the
affected lines.

Reviewed By: farnz

Differential Revision: D19657442

fbshipit-source-id: 4626daf20c57fbb511b203e54f2575edc619f40f
2020-01-30 20:08:34 -08:00
Jun Wu
935cf597c6 py3: hggit: fix absolute import
Reviewed By: singhsrb

Differential Revision: D19657557

fbshipit-source-id: dc0ba4c11f4ad69dc7e5e811875fb78c4fb492b9
2020-01-30 20:05:01 -08:00
Adam Simpkins
fabce3a798 py3: fix typing for journalentry
Summary:
Fix the type annotation for `journalentry.serialize()`: It needs to come
immediately after the method declaration to be recognized, not after the
docstring.

Fixing this causes pyre to produce type errors for this method.  I have
changed `journalentry` to derive from `typing.NamedTuple` to fix this.  This
syntax is somewhat awkward with Python 2, but is much cleaner in Python 3.

Reviewed By: quark-zju

Differential Revision: D19656758

fbshipit-source-id: a97b8a092bff0bdc7e9dee0650c35ef383b32248
2020-01-30 19:57:05 -08:00
Xavier Deguillard
5c00ebfd06 bundle2: fix encoding issue
Summary: The urlreq.quote returns a byte string, let's use b string to manipulate it.

Reviewed By: farnz

Differential Revision: D19656773

fbshipit-source-id: 1cbcc0f19e9463d5eabec2f67f5b00329188777d
2020-01-30 18:45:36 -08:00
Xavier Deguillard
d169de8946 bookmarks: binaryencode should return a byte string
Reviewed By: quark-zju

Differential Revision: D19656320

fbshipit-source-id: 85b0015dd3a8c485bbc5fb6ba854dbb7273ddc11
2020-01-30 18:45:36 -08:00
Xavier Deguillard
1629a2e038 remotenames: encoding fixes
Summary: Reading/writing to files is done via bytes, not strings.

Reviewed By: quark-zju

Differential Revision: D19656321

fbshipit-source-id: 9a9b13b27fc9dd8db6fe757dcd53e4730179832d
2020-01-30 18:45:36 -08:00
Xavier Deguillard
d6d60bda6a clone: make the samplehgrcs strings
Summary: These aren't bytes, they are strings.

Reviewed By: quark-zju

Differential Revision: D19656112

fbshipit-source-id: fcd505e3b4e522154358c80305e39ebca5936395
2020-01-30 18:45:35 -08:00
Adam Simpkins
a4722f9515 py3: fix reading of commitcloud backedupheads files
Summary:
Process the backupstate file as UTF-8.  Previously it was being read as raw
bytes, causing the code to think it was an unknown version.

These files contain the version specifier (pure-ASCII), the remote path
(unicode), followed by a list of hexadecimal node IDs (pure-ASCII).  Therefore
reading the whole file as UTF-8 should be fine.

Reviewed By: farnz

Differential Revision: D19655547

fbshipit-source-id: 6ca536d445cbf81b19122916f1b70b8895304bdb
2020-01-30 18:29:51 -08:00
Adam Simpkins
2370602802 py3: re-enable demandimport
Summary:
D19539064 disabled demandimport when using Python 3 as a precaution while we
were initially working on Python 3 support.  However, it appears to work
correctly and does not seem to cause obvious problems when enabled.

This re-enables demand import by default.  It can be explicitly disabled at
runtime by specifying `HGDEMANDIMPORT=disable` in the environment.

Reviewed By: xavierd, farnz

Differential Revision: D19655548

fbshipit-source-id: d56814049181da2d68abd09ac7ca772b04e06618
2020-01-30 18:29:50 -08:00
Adam Simpkins
ebf7df9c50 fix the relpath() template to work in hg status
Summary:
When used in an `hg status` command the `relpath()` template function would
throw an unhandled `KeyError` when it tried to access `ctx` in the template
mapping.  This field is only present in `hg log` templates, and is not present
in `hg status` templates.

This updates the `relpath()` template function to look for `repo` in the
template mapping instead.  I also updated the `hg status` code to pass in the
`repo` value now.  The `hg log` code already passes in this field.

Reviewed By: quark-zju

Differential Revision: D19568463

fbshipit-source-id: 3e1c1d6571c73e4b24bfce22ea28f57679d39846
2020-01-30 18:26:36 -08:00
Xavier Deguillard
7c48ba6119 rebase: fix encoding issues
Summary:
Bunch of read/writes to files were using plain strings, use byte strings
instead.

Reviewed By: quark-zju

Differential Revision: D19655101

fbshipit-source-id: 0cb70cbc51630d2c83752846774ff1a1c5ec9f41
2020-01-30 18:12:29 -08:00
Xavier Deguillard
607b2db333 obsutil: decode marker metadata
Summary: These are strings, let's decode them.

Reviewed By: quark-zju

Differential Revision: D19655102

fbshipit-source-id: 18ca5556c76603a53bcb20b58fa8787e2fab70fb
2020-01-30 18:12:29 -08:00
Xavier Deguillard
4eb639c665 amend: write to the obsinhibit with bytes
Summary: This file contains bytes representation of nodes, let's add the b prefix.

Reviewed By: quark-zju

Differential Revision: D19655100

fbshipit-source-id: 2dc163038b17b0d3bce384d9b04f13b1449705fc
2020-01-30 18:12:28 -08:00
Jun Wu
94c6853e8d edenscm: add a main module
Summary:
Make it possible to run edenscm via a "python package".

This makes it possible to run monkeytype [1] via debugpyhon:

    ./hg3 debugpython -- -m monkeytype run edenscm help log --pager=off

[1]: https://github.com/Instagram/MonkeyType

Reviewed By: farnz

Differential Revision: D19655850

fbshipit-source-id: 358f9dcf976141f798af2d57fb6392f2bca6a396
2020-01-30 18:09:14 -08:00
Mateusz Kwapich
040e01d65c filelog: file contents shouldn't be strings
Summary: on file copies instead of contents we were commiting b'contents'

Reviewed By: markbt

Differential Revision: D19656501

fbshipit-source-id: e1cb0bdac79da851494318ce31396a2aca6c5754
2020-01-30 18:05:39 -08:00
Durham Goode
8c211e5224 py3: fix stdout/stderr in dott tests
Summary:
dott tests rig up a custom stdout/stderr. In Python 3 we expect them to
be strings, so we need to use an appropriate io object.

Unfortunatly upstream already created a pycompat.stringio object which uses
bytes for both python 3 and 2, so I had to use a new name for now.

Reviewed By: quark-zju

Differential Revision: D19653469

fbshipit-source-id: a529df5a2ab179c105fa441c4503ed6606e77543
2020-01-30 18:00:34 -08:00
Simon Farnsworth
a579ce59f9 py3: add types to bundle2.py and make wireprotocol exclusively bytes
Reviewed By: xavierd

Differential Revision: D19652082

fbshipit-source-id: b6670845bbd06dc55439ad5223d47bda2ae014b1
2020-01-30 17:56:58 -08:00
Mark Thomas
8e673a1de0 py3: implement util.dirs in Rust
Summary:
The C implementation of util.dirs doesn't work with Python 3.  Instead of trying to port it, write an implementation in Rust that works in both Python 2 and Python 3.

The old implementation had a hack for flat dirstate to omit certain entries.  Remove this hack as we now only use flat dirstate in tests and small repos.  This simplifies the code in the fastpath case.

Reviewed By: quark-zju

Differential Revision: D19634699

fbshipit-source-id: cbb6562bc9a0209b13f67db3dfa5250800a9fcbe
2020-01-30 17:33:36 -08:00
Saurabh Singh
45d5467eec memcommit: add type hints
Summary: This diff just adds the type hints for `memcommit` extension.

Reviewed By: quark-zju

Differential Revision: D19651074

fbshipit-source-id: 834a5980cded92641e4d368ce00d01a3dcaec330
2020-01-30 16:49:50 -08:00
Xavier Deguillard
5ecc59749e treemanifest: hashing requires bytes
Summary: We need to encode the key to be able to hash it.

Reviewed By: genevievehelsel

Differential Revision: D19651373

fbshipit-source-id: 214a3d127eb71a1c9cbf4ed1fec4f5a58f7f5195
2020-01-30 16:29:17 -08:00
Xavier Deguillard
6b328f5f7d remotefilelog: the filelog data is bytes
Reviewed By: DurhamG

Differential Revision: D19651372

fbshipit-source-id: 13bf86235b95ad84beb9fe45e7abc06baec67a47
2020-01-30 16:29:16 -08:00
Xavier Deguillard
4b2b0c6630 remotenames: fix python3 warning
Summary:
Importing directly from collection is deprecated, we should import from
collections.abc in Python3.

Reviewed By: DurhamG

Differential Revision: D19650852

fbshipit-source-id: b2e470e5fde3fe915944ae68a4274e47ed91071a
2020-01-30 16:29:16 -08:00
Xavier Deguillard
8c08a97290 fsmonitor: fix test_symlinks_t
Summary:
A previous change requires the default value to be a tuple with 4 values,
fsmonitor was only passing 1.

Reviewed By: kulshrax

Differential Revision: D19652000

fbshipit-source-id: c9ae04109c61173a1df27ce653622dc4db653977
2020-01-30 16:20:13 -08:00
Carolyn Busch
fca8aafffc py3 chgserver bytes <-> unicode
Summary:
Server reads & writes bytes. Convert between bytes and unicode where
neccessary in chgserver. Add some type annotations.

Reviewed By: xavierd

Differential Revision: D19645028

fbshipit-source-id: 1ad4bec25c398422112ee0ed1780ee9d704e6622
2020-01-30 15:44:42 -08:00
Durham Goode
419858ad5c py3: fix journal writing
Summary:
Writing to the file should happen in bytes, so let's make the
journalentry serializer/deserializer convert from strings to bytes for us.

Reviewed By: xavierd

Differential Revision: D19650468

fbshipit-source-id: 3c13aff032ace1a15611c2370fc7d6ed44a6df34
2020-01-30 15:22:25 -08:00
Durham Goode
c1672ad848 py3: fix treemanifest fetching
Summary:
bundle2 parameter values are bytes, so when we compare the treemanifest
category with the bundle2 part category we need to encode the bundle2 part one.

Also fix wirepack to read utf8 paths.

Reviewed By: xavierd

Differential Revision: D19650197

fbshipit-source-id: 67da3bd8244f7953ec608019844988b20ad1929f
2020-01-30 15:22:25 -08:00
Xavier Deguillard
c1652cad03 drawdag: the commit data should be bytes
Summary:
The commit context requires commit data to be bytes, but drawdag was using str
for it, breaking tests.

Reviewed By: DurhamG

Differential Revision: D19650219

fbshipit-source-id: 1409ce9a277c1c410ede8109989e07595378ff3e
2020-01-30 14:55:55 -08:00
Xavier Deguillard
96f5dc6652 localrepo: wwread returns bytes
Summary:
This is directly used in fctx.data() which expects bytes to be returned, encode
the return value of vfs.symlink.

Reviewed By: quark-zju

Differential Revision: D19647846

fbshipit-source-id: 966a5e79db6843c4bc6b802aeb90364e983cd834
2020-01-30 14:55:55 -08:00
Xavier Deguillard
b863a561a4 pytreestate: use PyPath where necessary
Summary:
A previous diff missed these PyPath. As a bonus, replace most of the PyObject
into proper Rust types.

Reviewed By: quark-zju

Differential Revision: D19646139

fbshipit-source-id: a3b55e5a802d8bbef86e9a9431f605bb4e284b32
2020-01-30 14:55:54 -08:00
Durham Goode
ca48fc2665 py3: fix resolving unknown binary nodes in changectx
Summary:
changectx takes an arbitrary identifier and tries to resolve it. If it
takes a binary node and the node is not known, the fallback looksup don't accept
bytes and throw an exception. Let's stop early and raise the RepoLookupError.

Reviewed By: quark-zju

Differential Revision: D19643834

fbshipit-source-id: cfee39e0487f908ebd4ff628b847e53dec13c66e
2020-01-30 14:39:41 -08:00
Durham Goode
cfdf7814f4 py3: fix pipe reads in sshpeer
Summary:
In Python 3, subprocess creates FileIO type pipes, which do one syscall
and are not guaranteed to return 'size' bytes. In Python 2 they're 'file' types
which are guaranteed to return 'size' bytes (via multiple syscalls).

So let's handle that case.

Reviewed By: quark-zju, xavierd

Differential Revision: D19648035

fbshipit-source-id: 7d04a67acc569a55d3a855fc8521f1b8261f4636
2020-01-30 13:55:21 -08:00
Durham Goode
22b2f5becc py3: fix encoding of remotenames at write time
Summary:
remotenames are stored as unicode in memory and bytes on disk. We need
to convert before we write in this case.

Reviewed By: quark-zju

Differential Revision: D19648032

fbshipit-source-id: 4b96cf05eefd66ca031436d9979b94da8e28b8dd
2020-01-30 13:55:20 -08:00
Durham Goode
5311ab2fbb py3: fix encoding of pushkeys
Summary: Let's decode pushkeys into utf8.

Reviewed By: quark-zju

Differential Revision: D19648033

fbshipit-source-id: 9deeeecd53e065df9b344f8edd0183dc7c1511c5
2020-01-30 13:55:20 -08:00
Stefan Filip
843f8a7f50 py3/cython: set the language_level to 3str
Summary:
Follow up to the previous diff setting language_level to 3str for
`edenscmnative/clindex.pyx`. Consistent behavior for all files.

Reviewed By: quark-zju

Differential Revision: D19634907

fbshipit-source-id: 779d6090b5660e109a37f9740299fb9623d8f34d
2020-01-30 12:52:06 -08:00
Carolyn Busch
6b24e9a69e py3: commandserver write and read bytes
Summary: Make the command server write and read bytes. Add type annotations.

Reviewed By: quark-zju

Differential Revision: D19642862

fbshipit-source-id: 2dfcd8f0c2e6124cc6afe1e85c0f5275e7fe5b1f
2020-01-30 11:12:10 -08:00
Durham Goode
f170b964be py3: fix known wireproto command
Summary:
It was iterating over what it thought was a string, but in python3 it's
a byte string, which results in iterating over integers. So known always claimed
the server knew every commit.

Reviewed By: quark-zju

Differential Revision: D19644837

fbshipit-source-id: 4e84a33b4d3e436006f93a5d8e1bdabf12e7d120
2020-01-30 10:59:01 -08:00
Stefan Filip
45fa522e1c py3/fsmonitor: use next(iter) instead of iter.next()
Summary: iter.next() is gone in Python 3. next(iter) is the Python2 compatible version.

Reviewed By: quark-zju

Differential Revision: D19630455

fbshipit-source-id: e5d4f0a6cc0c6b0489916ca6d177a6831bd7d277
2020-01-30 10:28:02 -08:00
Adam Simpkins
dbaac4f0e7 py3: fix some Python 3 compatibility issues in sshpeer
Summary:
Fix `threadedstderr` to stop directly deriving from `threading.Thread`.
Deriving directly from `threading.Thread` is somewhat tricky.  Classes that
derive from `Thread` are only allowed to implement `__init__()` and `run()`.
This class also defined `close()`, which violates the `Thread` docs.  Changing
this to no longer derive from `Thread` fixes exceptions being thrown when
trying to read from the SSH stderr pipe in this thread.

This also updates `_writessherror()` to decode the stderr data before writing
it to the `ui` object.

Reviewed By: quark-zju

Differential Revision: D19634265

fbshipit-source-id: bddd120c7995f49d3bbebbb171384d5b15b779e9
2020-01-30 10:22:30 -08:00
Durham Goode
3ed7b6e3f2 py3: fix journal to work with utf8
Summary: Various fixes to make it stop crashing

Reviewed By: quark-zju

Differential Revision: D19643879

fbshipit-source-id: 9dd0bcef5369b769a8c968ea86de0e8aaf640650
2020-01-30 09:58:26 -08:00
Durham Goode
9aa1e8d7a3 py3: fix sparse utf8 logging
Summary:
Sparse logs the profile to scuba, but our logging data must be utf8.
Let's encode it first. We probably want to make this "replace" instead of
"strict" for the conversion once D19634265  lands.

Reviewed By: quark-zju

Differential Revision: D19643832

fbshipit-source-id: 550c67f194171ccdbd244bc79edfa1b999467824
2020-01-30 09:58:26 -08:00
Durham Goode
e3992926b7 py3: utf8 fixes for pull and wireproto
Summary: Moves pull closer to working by appropriately encoding and decoding.

Reviewed By: quark-zju

Differential Revision: D19643833

fbshipit-source-id: d311fb897d15594f2cb644b15038866ce506e011
2020-01-30 09:58:26 -08:00
Durham Goode
484a088417 py3: support caching in python3
Summary:
Revlog did a type check against str. In python 3 it accepts bytes
instead, so we need to support that.

Reviewed By: xavierd

Differential Revision: D19634882

fbshipit-source-id: 04ae908b4bfe8ed5de7a6d41cbe748dcb34b2540
2020-01-30 09:12:30 -08:00
Durham Goode
8b942a7166 remotenames: fix broken changelog contains check
Summary:
cl.__contains__ does a "0 < rev <= len(repo)" check, so passing nodes
will always fail. In python 3 this causes a type error, so let's fix this.

Reviewed By: quark-zju

Differential Revision: D19634880

fbshipit-source-id: 011ab2f0c07a16fa45734d360ac952273c4fefc2
2020-01-30 09:12:30 -08:00
Genevieve Helsel
247f686447 izip_longest to zip_longest
Summary:
izip_longest depreciated in python3 in favor of zip_longest, use zip_longest from future

source: https://python-future.org/compatible_idioms.html#itertools-filterfalse-zip-longest

Reviewed By: xavierd

Differential Revision: D19632101

fbshipit-source-id: 2eb3c125d2beaf66045dad304b900fe7928f2066
2020-01-30 08:50:41 -08:00
Jun Wu
cd04a56178 repo: avoid metalog when migrating files from sharedvfs to storevfs
Summary: Otherwise it can trigger metalog ProgrammingErrors.

Reviewed By: markbt

Differential Revision: D19624779

fbshipit-source-id: ba56a9aefa28d56a38ea8d966115647975613c91
2020-01-30 07:25:31 -08:00
Mateusz Kwapich
e4ee93b38b py: fixes for cloud sync
Summary: "hg cloud sync" succeeds when there's nothing to do

Reviewed By: quark-zju

Differential Revision: D19630431

fbshipit-source-id: e2c9859c7660e6283e6d319c634f1cd9053f8b3f
2020-01-29 18:25:41 -08:00
Mateusz Kwapich
892dcb3189 py3: make "hg cloud sl" work in py3
Summary: Some strings had to stay as unicode

Reviewed By: quark-zju

Differential Revision: D19628661

fbshipit-source-id: 6ea191c60a4ef2ed98b2e2d3ab90c875e6eea248
2020-01-29 18:25:41 -08:00
Zeyi (Rice) Fan
05d306bfc3 make test-extension.t py3 compatible
Summary: because of the test `type(b"") != type("")` we actually have different behavior in Python 3 and Python 2. So we remove this test temporarily and we can add it back later.

Reviewed By: sfilipco

Differential Revision: D19611682

fbshipit-source-id: 7bca979f8ef871672ac9666286c8d5e54747e8b3
2020-01-29 17:07:32 -08:00
Stefan Filip
56c0adbc46 py3/match: convert filter result to list
Summary:
In Python3 the result of filter is a class that does not have len
implemented. In Python2 the result of filter is a list.

Reviewed By: quark-zju

Differential Revision: D19629139

fbshipit-source-id: a7a1d9d831d72b2aab8b7a920bf82295c4673603
2020-01-29 17:03:05 -08:00
Stefan Filip
f80b7bd6c4 py3/sparse: decode sparse profile as utf-8
Summary: File contents are read as bytes but content should be interpreted as utf-8.

Reviewed By: quark-zju

Differential Revision: D19628315

fbshipit-source-id: 523a201ddf57a2aabcd135156e71034c52b59adb
2020-01-29 17:03:04 -08:00
Stefan Filip
ac4aaab5a2 py3/util: treat authentication url as str
Summary: Python3 socke address is str.

Reviewed By: quark-zju

Differential Revision: D19628130

fbshipit-source-id: d6ce93d28588a9ab8829f5b95db30ecee040a160
2020-01-29 17:03:04 -08:00
Mateusz Kwapich
3cfb8a6a7c py3: node: get rid of extra encoding
Summary:
The extra call to encode makes everything a big slower, fortunately python
standard library provides functions that do the right thing.

Reviewed By: xavierd

Differential Revision: D19634034

fbshipit-source-id: 6bb0de26300c41667824422c016e8d4f0aee70d3
2020-01-29 16:48:41 -08:00
Xavier Deguillard
467b3095c1 eden_dirstate_map: encode the path
Summary: This enables `hg add` on EdenFS

Reviewed By: quark-zju

Differential Revision: D19631538

fbshipit-source-id: 1621ab77fef977c9364052bc522df18729face48
2020-01-29 16:45:46 -08:00
Durham Goode
13b96565e5 py3: convert parts of undo.py to specify utf-8
Summary: This makes hg pull make more progress.

Reviewed By: mitrandir77

Differential Revision: D19630124

fbshipit-source-id: 6e8d6bd441353ffbf87076addd2774d5e09dd24a
2020-01-29 16:41:54 -08:00
Durham Goode
654ed5e4b1 remotenames: prefer picking default paths
Summary:
S194477 was caused by a config change adding a custom path to
repositories, where the path had the same url as default. This caused all the
remotebookmarks to be associated with the new path instead of default.

This is subtle and undesirable. Let's make it so that the default path takes
precedence.

I also deleted a hggit test because it was causing the entire test to be skipped
when dulwich wasn't available.

Reviewed By: quark-zju

Differential Revision: D19627711

fbshipit-source-id: 55b3ce94f18a26ccb9471c8e96a584d3087dbb82
2020-01-29 15:31:16 -08:00
Simon Farnsworth
c87bac85e1 py3: streamclone: make wireprotocol bytes
Summary: The wireprotocol for streaming clones should be explicitly bytes, encoding filenames as UTF-8 when needed. Make it so

Reviewed By: quark-zju, xavierd

Differential Revision: D19616097

fbshipit-source-id: 3513c801035c8d7ba950777aa119e1d1480013a5
2020-01-29 15:20:56 -08:00
Jun Wu
07a1c0a776 py3: connectionpool: fix some Python 3 issues
Summary: `im_self` is gone. Use `__self__` instead.

Reviewed By: xavierd

Differential Revision: D19630121

fbshipit-source-id: aa1af51e89cdffcfc63f3c18eeaf04d35425160e
2020-01-29 14:27:12 -08:00
Jun Wu
552c1dfabd py3: sshpeer: fix some encoding issues
Reviewed By: mitrandir77

Differential Revision: D19630030

fbshipit-source-id: b700187aeb412ac85e81532a525601454c45c426
2020-01-29 14:27:12 -08:00
Jun Wu
b01245ecc1 py3: commitcloud: fix some encoding issues
Reviewed By: mitrandir77, xavierd

Differential Revision: D19630028

fbshipit-source-id: 825bffa5c591f442ddecac4b9b8485dfe273b667
2020-01-29 14:27:12 -08:00
Jun Wu
50f80dff96 py3: clienttelemetry: fix some encoding issues
Reviewed By: xavierd

Differential Revision: D19630029

fbshipit-source-id: e9a9060a1e0cc2745dcb8588a083e8752384cf9f
2020-01-29 14:27:11 -08:00
Jun Wu
77a51f0e3b merge: allow specifying a customized rename script
Summary:
Provide an option to run an external script that calculates the rename
destination. This is especially helpful for cases where development happened in
another branch and *directory* and want to merge changes back.

For example, we recently had an experimental `scm3` branch that copies files
from `scm` to `scm3` to try out changes:

  o master <-- rebase destination
  |
  o (changes in scm/)
  :
  | o (changes that need to backport to scm/) <-- rebase source
  | |
  | o (changes in scm3/ that got implemented differently in scm/)
  | :
  | o (copy scm/ to scm3/ for experiments)
  |/
  o

This allows `--config experimental.rename-cmd='sed "s#scm3/#scm/#"'` to be
used for the above case.

While working on this feature, I noticed a bug that if the "copy" information
is recorded, rebase might lose changes. I added a test to track that bug.

Reviewed By: mitrandir77

Differential Revision: D19609868

fbshipit-source-id: 94462889d3ce9214b48fac793852d16aeacecc77
2020-01-29 14:27:11 -08:00
Jun Wu
0ba90481ab pycompat: add basestring, drop dep on "past"
Summary:
This removes the dependency of `future` (used by `import past`) which is not
available in our Windows tests.

Reviewed By: farnz

Differential Revision: D19626387

fbshipit-source-id: 2b13f08a9dafe77d1e7f6fc1a3cc48b3581681ca
2020-01-29 13:46:04 -08:00
Jun Wu
282aaf155d py3: remotenames: fix compatibility issues
Reviewed By: mitrandir77

Differential Revision: D19627763

fbshipit-source-id: 3e1396a0686de81e3ed97ea5d3fb14c695bb3b8d
2020-01-29 13:43:09 -08:00
Xavier Deguillard
7c0c6feea1 commitcloud: fix max(None, None) in Python3
Summary:
In Python2, max allows None to be passed in, not in Python3, let's mimic the
Python2 behavior.

Reviewed By: sfilipco

Differential Revision: D19626654

fbshipit-source-id: f4fcaaa0516d898bc747d21735d2c30a4a9b41f4
2020-01-29 13:32:46 -08:00
Mateusz Kwapich
30c0e6482c proper bytes handling for remotefilelog
Summary:
I've inspected how the data flows from struct.unpack through the remotefilelog
and annotated few functions.

In one place I've used encodeutf8 function

Reviewed By: quark-zju

Differential Revision: D19626711

fbshipit-source-id: 77499c567b0cddab17d21b4f11b06f58bf3fcaef
2020-01-29 12:59:44 -08:00
Mateusz Kwapich
c1335eac46 add types module
Summary: Let's have a module that defines types ussed commonly in HG

Reviewed By: quark-zju

Differential Revision: D19625872

fbshipit-source-id: 817535d87acee6cb5647bc8ad25df21a95bec09c
2020-01-29 12:59:44 -08:00
Mateusz Kwapich
13aee5fa2a make all config function params strings
Summary: We want unicode configs.

Reviewed By: quark-zju

Differential Revision: D19612400

fbshipit-source-id: ec9f863c3203be7c956dcd97c735d74a11adce35
2020-01-29 12:09:37 -08:00
Xavier Deguillard
a80bcd779b phabstatus: make PeekaheadRevsetIter an Iterator
Summary:
Python2 and Python3 have different method names for an iterator, let's inherit
from Iterator so Python (both version) tells us what needs to be implemented.

Reviewed By: mitrandir77

Differential Revision: D19624294

fbshipit-source-id: b9c3d5ae796216b4bcd330c8be6f061e369e2614
2020-01-29 11:07:10 -08:00
Xavier Deguillard
20459835bb clindex: use str as input to vfs
Reviewed By: farnz

Differential Revision: D19624129

fbshipit-source-id: cbee989334d5cf114ef7b74c9143a992cee70370
2020-01-29 11:07:09 -08:00
Stefan Filip
11b85a8a33 py3/remotenames: convert to str after reading data
Summary: In Python3, nodes and bookmarks are str.

Reviewed By: DurhamG

Differential Revision: D19624130

fbshipit-source-id: edbb115c7e6fa1232c92ce76e8ee5aa46b3f23db
2020-01-29 11:00:26 -08:00
Genevieve Helsel
7c8128744c remove unsued urllib import
Summary: as title, removes unused import that had python 2/3 compat issues.

Reviewed By: sfilipco

Differential Revision: D19588354

fbshipit-source-id: 05d185d0b1a8fdd1bdb7eef0557362e95d85beb4
2020-01-29 10:45:57 -08:00
Jun Wu
4e7edef4b7 py3: use edenscmdeps3.zip for Python 3 deps
Summary:
Python 2 pyc files are incompatible with Python 3.  Use a different zip file.

This also makes it possible for Python 2 and 3 setup.py build to co-exist
(discovered by markbt).

Reviewed By: markbt

Differential Revision: D19617513

fbshipit-source-id: 47b2a01e905b0cb1ec68e7a7eec40868f09a2167
2020-01-29 10:40:16 -08:00
Durham Goode
38756127f2 hgevents: disable events during status
Summary:
There are times when status updates the dirstate to mark files as
clean or to resolve lookup files. In doing so it takes the wlock, which triggers
a state-enter event which causes external watchers to then run hg status, which
can then trigger another wlock event from status. This causes an infinite loop.

It's not clear why status is taking the wlock every time. It should be a
one-time thing. So more investigation is needed. For now though, we can prevent
this from happening by disabling hgevent signals while status has the wlock.

Reviewed By: quark-zju

Differential Revision: D19602370

fbshipit-source-id: 609316de1441d40be4f81236b8d00ce7dc30a130
2020-01-29 10:30:22 -08:00
Jun Wu
8699e769e8 py3: minirst: fix some encoding issues
Summary: This makes `hg3 help` work.

Reviewed By: xavierd

Differential Revision: D19617512

fbshipit-source-id: 6f7621cff36e8f711fb5c7c21c805a1f82f80dfd
2020-01-29 09:00:19 -08:00
Xavier Deguillard
9db47445cd remotefilelog: switch some wireprotocol to use bytes
Summary:
String literals are unicode in Python3, which is not the intention for
over-the-wire streams. I've also added type-checking to get help from pyre to
do the right thing.

Reviewed By: quark-zju

Differential Revision: D19614060

fbshipit-source-id: 33f2b603012481159593426dbb3c18b1614b19f6
2020-01-29 08:50:03 -08:00
Jun Wu
7d8003f36e py3: manifest: drop pure lazymanifest
Summary:
The pure implementation has many encoding issues. Since our code does not build
in the pure Python mode, just drop the pure lazymanifest implementation for
simplicity.

Reviewed By: farnz

Differential Revision: D19613691

fbshipit-source-id: f3f95717bc77a75d4aa7b597a33f8eec745df134
2020-01-29 07:45:23 -08:00
Jun Wu
df746ae467 py3: wireproto: fix some encoding issue
Reviewed By: markbt

Differential Revision: D19615830

fbshipit-source-id: c16efaba5fca65898b24192910b2afd81f77a0b9
2020-01-29 07:45:23 -08:00
Jun Wu
fa999c515f py3: visibility: fix some encoding issue
Reviewed By: farnz

Differential Revision: D19615841

fbshipit-source-id: a88a2f9328dcef77a603d67f29435c6da4956d60
2020-01-29 07:45:23 -08:00
Jun Wu
c048b191ba py3: config: fix a minor encoding issue
Reviewed By: markbt

Differential Revision: D19615812

fbshipit-source-id: 997f4c4111dfecd1c157cd428b5044efb49883f8
2020-01-29 07:45:22 -08:00
Jun Wu
4da8312956 py3: transaction: fix some encoding issues
Summary: Use bytes to write files.

Reviewed By: markbt

Differential Revision: D19615839

fbshipit-source-id: 7881a7078fcfde154ba4b6e58c706064fd8e26b9
2020-01-29 07:45:22 -08:00
Jun Wu
dc01df5bb4 py3: templater: fix some encoding issues
Summary: The template language is in unicode.

Reviewed By: markbt

Differential Revision: D19615823

fbshipit-source-id: 30c6b295b4e4c999480c46aba8b4e8ffed5a4d64
2020-01-29 07:45:21 -08:00
Jun Wu
98a90ec4a0 py3: store: fix some encoding issues
Reviewed By: markbt

Differential Revision: D19615828

fbshipit-source-id: 8adf9b8d5fda818efd7d02d1ce05cb47c56a26da
2020-01-29 07:45:21 -08:00
Jun Wu
4b64b6ce9e py3: scmutil: fix some encoding issues
Reviewed By: farnz

Differential Revision: D19615819

fbshipit-source-id: e0137e15d14cf7fd84fdd015fe1c63dc35195459
2020-01-29 07:45:21 -08:00
Jun Wu
cc6dee9815 py3: revsetlang: fix some encoding issues
Summary: Use unicode in revset expressions.

Reviewed By: farnz

Differential Revision: D19615816

fbshipit-source-id: fc0251ca7108cf3baacd9ec154bc2760657754e0
2020-01-29 07:45:20 -08:00
Jun Wu
1e9cec215e py3: profiling: fix some encoding issues
Summary: Write bytes to files.

Reviewed By: farnz

Differential Revision: D19615833

fbshipit-source-id: 0f5a0df493715fad5723b98f4b4246fc1dec53be
2020-01-29 07:45:20 -08:00
Jun Wu
9cafa05c43 py3: phases: fix some encoding issues
Reviewed By: farnz

Differential Revision: D19615840

fbshipit-source-id: ad9694e736b24254172f732f988230b987597e0e
2020-01-29 07:45:19 -08:00
Jun Wu
573b64085c py3: peer: fix a minor encoding issues
Summary: The "function name" is unicode in Python 3. Put unicode as the dict keys.

Reviewed By: farnz

Differential Revision: D19615814

fbshipit-source-id: 3508f59d11e4f8ab6345c80c6bdfaff150a63d76
2020-01-29 07:45:19 -08:00
Jun Wu
47d36736ae py3: obsolete: fix some encoding issues
Summary: Use bytes to write files.

Reviewed By: farnz

Differential Revision: D19615831

fbshipit-source-id: 0868b9272f3570547326f63e34b9b6a489788464
2020-01-29 07:45:19 -08:00
Jun Wu
330306cfde py3: blackbox: fix some encoding issues
Summary: Use bytes to write files.

Reviewed By: farnz

Differential Revision: D19615824

fbshipit-source-id: fa7b87dea80960dcc7b6586b3d50baef6228db3a
2020-01-29 07:45:18 -08:00
Jun Wu
bfa9485f90 py3: merge: fix some encoding issues
Summary: This affects mergestate and updatestate.

Reviewed By: farnz

Differential Revision: D19615836

fbshipit-source-id: bc549c71a5ba5ba9d2069deb1e77cb580f210ba2
2020-01-29 07:45:18 -08:00
Jun Wu
751d53d4e6 py3: localrepo: fix some encoding issues
Reviewed By: farnz

Differential Revision: D19615825

fbshipit-source-id: 1801652ed93ac9218a66af7f23f5bce9eab3ab10
2020-01-29 07:45:17 -08:00
Jun Wu
456a0ad5e1 py3: json: fix some encoding issues
Reviewed By: farnz

Differential Revision: D19615829

fbshipit-source-id: 3d251b8febedbdfbefe5b1e59ea026de54503663
2020-01-29 07:45:17 -08:00
Jun Wu
29711c1cf0 py3: hg: fix some encoding issues
Reviewed By: farnz

Differential Revision: D19615834

fbshipit-source-id: 8c8aaaf6bca688476662f29375b8c5484ff0765a
2020-01-29 07:45:17 -08:00
Jun Wu
3dea327fba py3: eden_dirstate: use unicode paths on Python 3
Reviewed By: farnz

Differential Revision: D19615837

fbshipit-source-id: 97e514fbc5ab3b077fd9b6c56cd8b5d7b6bc7856
2020-01-29 07:45:16 -08:00
Jun Wu
60733ef7b2 py3: dirstate: fix some encoding issues
Summary: Some assertions were added to make sure paths are unicode.

Reviewed By: farnz

Differential Revision: D19615813

fbshipit-source-id: 7d2135c0ac424fc46edd9d660b072a2a95be7f82
2020-01-29 07:45:16 -08:00
Jun Wu
a3d23bb8d7 py3: context: fix some encoding issues
Reviewed By: farnz

Differential Revision: D19615827

fbshipit-source-id: 79cfab528eeb00048b00be75fa776737b49a53d7
2020-01-29 07:45:15 -08:00
Jun Wu
0ad9daf67f py3: config: fix some encoding issues
Reviewed By: farnz

Differential Revision: D19615815

fbshipit-source-id: 2ae650dc321c776b50f49070b1c8f18dd5a2149f
2020-01-29 07:45:15 -08:00
Jun Wu
b084775bd8 py3: changegroup: fix some encoding issues
Summary:
Changegroup should be using bytes mostly except when it tries to access
files via `repo.file(path)`. The `path` should be unicode on Python 3.
Make it happen. Some assertions are added to help capture issues.

Reviewed By: farnz

Differential Revision: D19615835

fbshipit-source-id: b35d13152ecb835092c7870e88383f484dc5f868
2020-01-29 07:45:15 -08:00
Jun Wu
ef1dd7f142 py3: bundle2: use some bytes
Summary:
The bundle2 layer uses bytes since it's part of the wireprotocol.
Some assertions were added to help detecting issues.

Reviewed By: farnz

Differential Revision: D19615832

fbshipit-source-id: 73ea657b501c7539b17862165c7a5781884013a2
2020-01-29 07:45:14 -08:00
Jun Wu
19c31c1690 py3: remotefilelog: add some type assertions
Summary: This helps detecting type errors at runtime.

Reviewed By: farnz

Differential Revision: D19615821

fbshipit-source-id: 0f45e1271aff50c29858c823366a0a54c89a3903
2020-01-29 07:45:14 -08:00
Jun Wu
e2b48e1122 py3: manifest: use bytes
Summary: The layer interacting with raw manifest text should use bytes.

Reviewed By: farnz

Differential Revision: D19613694

fbshipit-source-id: c5a12235eb50acd90e409c1059a3f9f11665f9f3
2020-01-29 07:45:14 -08:00
Jun Wu
fe28d31b8f py3: lock: drop an unnecessary encoding conversion
Summary: Use native unicode hostname without converting it to bytes.

Reviewed By: farnz

Differential Revision: D19613692

fbshipit-source-id: 0bd7e37890b96566143fb0501c3fa310270844d3
2020-01-29 07:45:13 -08:00
Jun Wu
3add434b57 py3: changelog: use bytes in low-level interfaces
Summary:
Like revlog, the changelog layer should use bytes for parsing its raw content.
`str` is still used for users, messages, etc.

Reviewed By: markbt

Differential Revision: D19613695

fbshipit-source-id: 4f6758f9fca0788bc89ad5c14d9103f66d19b7f2
2020-01-29 07:45:13 -08:00
Jun Wu
63ea4b2ab0 py3: bookmarks: fix encoding
Summary:
Bookmarks use unicode by default. Only need to use bytes when interacting with
vfs.

Reviewed By: farnz

Differential Revision: D19613689

fbshipit-source-id: bbe80e552cf17a023e5905f849a273b4b4299dc5
2020-01-29 07:45:13 -08:00
Jun Wu
47f4f75aba py3: diff: fix patchtext encoding
Summary: The patchtext is about a binary patch. It should use bytes.

Reviewed By: farnz

Differential Revision: D19613688

fbshipit-source-id: 20bddeb17f520f625396b23b5dcead7c7bbd3ede
2020-01-29 07:45:12 -08:00
Jun Wu
4ef1f31745 py3: util: fix some encoding issues
Summary:
Make sure file content and wireproto buffers are using bytes.
Added a utf8 helper function.

Reviewed By: farnz

Differential Revision: D19613686

fbshipit-source-id: 5d67ce48246dc8c3cc47041f461e454c46bc67f9
2020-01-29 07:45:12 -08:00
Jun Wu
32eec0fc37 py3: lock: fix encoding
Reviewed By: farnz

Differential Revision: D19613690

fbshipit-source-id: 617220ac09df7ccbbd1a3bee627b1a796199a33a
2020-01-29 07:45:12 -08:00
Jun Wu
2a28611d8a py3: pycompat: make fsencode/fsdecode do nothing
Summary:
We want to assume almost all paths are unicode and drop support for non-unicode
paths. `os.fsencode` returns bytes that breaks the assumption.

Change fsencode and fsdecode to do nothing so path types remain unicode.

Most use-cases are related to `__file__`, which is already unicode on Python 3.
`posix._checklink` uses it. But it seems making it no-op is okay.

Reviewed By: farnz

Differential Revision: D19613693

fbshipit-source-id: 587dbeef68b23beeaf6bc8861c71ea27b3c512d7
2020-01-29 07:45:11 -08:00
Jun Wu
0e6add4877 py3: repair: fix encoding for SHA1 calculation
Reviewed By: farnz

Differential Revision: D19613687

fbshipit-source-id: e8dbbcc8c1151dd2defe63b1c67df0a3dc598c90
2020-01-29 07:45:11 -08:00
Jun Wu
21b2a06c6f py3: vfs: add helper functions to get utf-8 content directly
Summary: Those will simplify some use-cases.

Reviewed By: farnz

Differential Revision: D19613121

fbshipit-source-id: 85ac4fa0c86afb84e0c9fb78950d3da2c93aa967
2020-01-29 07:45:10 -08:00
Jun Wu
535b35f427 py3: treestate: fix encoding
Summary: The raw blobs are in bytes, while paths and other things are in native string.

Reviewed By: markbt

Differential Revision: D19613122

fbshipit-source-id: e865168b5bc839e85ef843d771d8a6044e35a03a
2020-01-29 07:45:10 -08:00
Jun Wu
a986833f9e py3: ui.edit: fix encoding
Summary:
The commit message should be in native string. The raw file content is bytes.
Add translation for that.

Reviewed By: xavierd

Differential Revision: D19613117

fbshipit-source-id: 1b4145f0de75e48df3416cd7c52c09df1e6c4a07
2020-01-29 07:45:10 -08:00
Jun Wu
96309864ff py3: remotefilelog: use bytes
Summary: Like filelog, the remotefilelog layer should use bytes.

Reviewed By: xavierd

Differential Revision: D19613119

fbshipit-source-id: c8221ac21dc255ad0a837b2bb30fa9966df95390
2020-01-29 07:45:09 -08:00
Jun Wu
0c6dc8d8c9 py3: filelog: use bytes
Summary: Like revlog, the filelog layer should use bytes.

Reviewed By: sfilipco

Differential Revision: D19613116

fbshipit-source-id: 6b07d05a6960c2d1c8b4ebe3001632fd8024c5af
2020-01-29 07:45:09 -08:00
Jun Wu
df36f5c459 py3: revlog: use bytes
Summary: The revlog layer should use bytes.

Reviewed By: sfilipco

Differential Revision: D19613120

fbshipit-source-id: 09f0028cbf810b52694643c79c91997d434cdaf5
2020-01-29 07:45:08 -08:00
Jun Wu
cb289128dc py3: pycompat: add helpers for encode("utf-8")
Summary:
On Python 2 we want bytes.encode("utf-8") to be no-op, while on Python 3 we
want it to return a real `str`. This helper function does that.

Reviewed By: sfilipco

Differential Revision: D19613118

fbshipit-source-id: 1f0fc31bf8aee28cc20694da52b10b12dbdd9000
2020-01-29 07:45:08 -08:00
Jun Wu
9b4d48bcea py3: lz4revlog: fix encoding issues
Summary: The lz4revlog extension needs binary types.

Reviewed By: sfilipco

Differential Revision: D19612879

fbshipit-source-id: c66d3953883d9b06fc9867511843c36d44fe2245
2020-01-29 07:45:08 -08:00
Genevieve Helsel
ce2bf270b3 fix_map
Summary: Python 3 map call doesn't return list, so wrap in list.

Reviewed By: mitrandir77, xavierd

Differential Revision: D19588352

fbshipit-source-id: 8556f4783f9bedb8a74ec0e99ff226ff96f387a9
2020-01-28 18:24:56 -08:00
Stefan Filip
cdc1efc872 py3: open stdout log file in text mode
Summary: In both py2 and py3 stdout is text mode.

Reviewed By: simpkins

Differential Revision: D19613397

fbshipit-source-id: fe01363836cb5956c7e6d83a0f81488649c262a9
2020-01-28 17:17:45 -08:00
Adam Simpkins
c6b023ce2a py3: remove an incorrect encode() call in python 3
Summary:
We want all config values to always be `str` types, so remove this explicit
`encode()` call that was trying to convert the value to `bytes`.

Reviewed By: sfilipco

Differential Revision: D19610590

fbshipit-source-id: a72f0755cdb12cae13fb9d58f529433d508d46e9
2020-01-28 16:33:38 -08:00
Mark Thomas
ffa61b0945 hggit: allow empty repos when dulwich returns empty FetchPackResult
Summary:
For empty repos, some versions of dulwich return a non-`None` `FetchPackResult`
where `bool(result)` throws an exception.   Handle this by explicitly checking
for `result.refs` being `None`, too.

Reviewed By: sfilipco

Differential Revision: D19611221

fbshipit-source-id: 094e3ec667a484a150eb75596684a120528159e9
2020-01-28 16:27:29 -08:00
Mark Thomas
82715fd2ea py3: iter{keys,values,items} -> pycompat.iter{keys,values,items}
Reviewed By: quark-zju

Differential Revision: D19608323

fbshipit-source-id: dd186ef16d6422a56af41fcaa850d9838ae9a240
2020-01-28 16:27:28 -08:00
Genevieve Helsel
ec51946a13 add pyre ignore for dbm
Summary: this pyre-ignore got lost in translation and caused pyre eden to fail

Reviewed By: sfilipco, mitrandir77, xavierd

Differential Revision: D19610997

fbshipit-source-id: b21faa36ee6b45585a774402d081bb7a3b8731b4
2020-01-28 16:16:40 -08:00
Adam Simpkins
2165181562 py3: ensure all command names are str
Summary:
Several of the debug commands explicitly defined their command names as bytes.
Remove the leading `b` prefix so that all command names are uniformly `str`
data types.

Reviewed By: DurhamG, sfilipco

Differential Revision: D19596738

fbshipit-source-id: 3851a32c38009968dfa338877b9ae03015dfca61
2020-01-28 15:56:20 -08:00
Genevieve Helsel
de029e9be7 unicode to pycompat.unicode
Summary: Use pycompat unicode instead of python 2 built in unicode to check type.

Reviewed By: mitrandir77

Differential Revision: D19589969

fbshipit-source-id: 3e3cf2c5cb6fd90cca9c8f803e7bd45f262f96e5
2020-01-28 15:45:17 -08:00
Xavier Deguillard
8697be6910 minirst: use string regexp
Summary: The input of minirst will be unicode, therefore, remove the b prefix.

Reviewed By: sfilipco

Differential Revision: D19610028

fbshipit-source-id: d3b1ebfeb1ba5a30afe250ce5713857dd9ebc359
2020-01-28 15:40:56 -08:00
Xavier Deguillard
ff536a2c24 remotenames: use Mapping instead of DictMixin
Summary: The later is deprecated since python 2.6, and the former works in python3

Reviewed By: simpkins

Differential Revision: D19608560

fbshipit-source-id: c9e45bbc51f0e20416c2640dfc25e4cac4934152
2020-01-28 15:40:56 -08:00
Mateusz Kwapich
7f606baac5 store: remove extra b's
Summary: The fallback filename handling can be utf8 under py3

Reviewed By: sfilipco

Differential Revision: D19610624

fbshipit-source-id: a86ed958edca70597ff5a7c67cbc746c1e325518
2020-01-28 15:37:09 -08:00
Mateusz Kwapich
b67b923a59 fbscmquery: fix the urlencode import
Reviewed By: xavierd

Differential Revision: D19609731

fbshipit-source-id: fc666f6238ca62633e0e45351be45f6be78dc012
2020-01-28 14:59:57 -08:00