Commit Graph

3587 Commits

Author SHA1 Message Date
Harvey Hunt
6a0522aefa mononoke: Add a multiplex ID to blobstore configuration
Summary:
In order to uniquely identify a blobstore multiplexer configuration,
add an ID.

Reviewed By: krallin

Differential Revision: D19770058

fbshipit-source-id: 8e09d5531d1d27b337cf62a6126f88ce15de341b
2020-02-07 07:46:10 -08:00
Genevieve Helsel
0a2fd56bba py3 blame
Summary: plain hg blame for non-binary files

Reviewed By: quark-zju

Differential Revision: D19676073

fbshipit-source-id: 203b0421c6279ea1ca6bb6e253e7952e64b5edeb
2020-02-07 03:19:41 -08:00
Genevieve Helsel
4acc40424c hex printing on revlog match failure
Summary: fixes printing for no match found in revlog match

Reviewed By: quark-zju

Differential Revision: D19652374

fbshipit-source-id: 55bfcbff7919ad817173241e2792df27311b931e
2020-02-07 03:17:06 -08:00
Stanislau Hlebik
ea8bed30a8 mononoke: add new safe sync_commit() method
Summary:
Follow up from D19718839 - let's add a function that will safely sync a commit
from one repo to another. Other function to sync a commit are prefixed with
unsafe

Reviewed By: krallin

Differential Revision: D19769762

fbshipit-source-id: 844da3e2c1cc39ef3cd86d282d275d860be55f44
2020-02-07 01:57:50 -08:00
Thomas Orozco
41216b7530 mononoke/fastreplay: handle JSON-encoded strings in command args
Summary:
If we e.g. a getpack for path like "foo\"bar", then we can't decode it into a
`&str` because we need to allocate a new `String` to hold it. At the same time,
if the path is "foo bar", then having a reference into the JSON we received is
nicer.

Right now, we expect a `&str`, so the latter case. But, if we find command args
from the first case, we can't deserialize them. To fix this, let's use
`Cow<...>`, which lets us either have a referenced or an owned string.

Also, let's add tests to confirm this works.

Reviewed By: ikostia

Differential Revision: D19767689

fbshipit-source-id: bf9e06d4a885638073c819a25a68810ff44f2546
2020-02-07 01:26:19 -08:00
Jun Wu
787cebfe91 context: print more details on fctx.data() key error
Summary:
Print out the name of the commit and the stack.
Hopefully this can help making debugging KeyErrors easier.

Reviewed By: DurhamG

Differential Revision: D19776181

fbshipit-source-id: 2eb985dd5355732a4d7728af68eb16173c48caa5
2020-02-06 15:56:26 -08:00
Jun Wu
69aa37f23b tracing: limit column width on ASCII output
Summary: This makes the output more readable even if the "name" of a span is very long.

Reviewed By: DurhamG

Differential Revision: D19780536

fbshipit-source-id: dce0d3777409c32b0752db51341a572addb823ea
2020-02-06 15:46:53 -08:00
Durham Goode
309a3a0592 py3: fix commitcloud lock output
Summary:
The use of json meant the progress step was coming out as unicode when
it should be str. Use the mercurial.json functions to solve this for python 2
and 3.

Reviewed By: xavierd

Differential Revision: D19777255

fbshipit-source-id: 15c8e45425fc8742b6e118249104fc1fb2f3345d
2020-02-06 13:23:21 -08:00
Thomas Orozco
d39eea991b blobrepo: don't fetch Hg Changeset IDs sequentially
Summary:
Fetching things from MySQL sequentially in a buffered fashion is a bad
practice, since we might end up saturating the underlying MySQL pool, and
starving other MySQL  clients.

Instead, let's make fewer, bigger queries.

Reviewed By: ahornby

Differential Revision: D19766787

fbshipit-source-id: 1cf9102eaca8cc1ab55b7b85039ca99627a86b71
2020-02-06 12:11:22 -08:00
Thomas Orozco
ce8b9a0fbe getbundle_response: don't fetch Hg Changeset IDs sequentially
Summary:
Fetching things from MySQL sequentially in a buffered fashion is a bad
practice, since we might end up saturating the underlying MySQL pool with a lot
of requests. Doing so will result in other queries being delayed as they wait
behind our batch of queries, which results in higher dispatch latency.

Instead, let's make fewer, bigger queries. Also, while we're in here, let's
update blobrepo to have an up-to-date comment.

Reviewed By: StanislavGlebik

Differential Revision: D19766788

fbshipit-source-id: 318ec4778ca259b210d431fc2add8b327bfce99a
2020-02-06 12:11:21 -08:00
Thomas Orozco
4874c1b7ab mononoke/fastreplay: set MultiplexBlob sampling rate
Summary: We don't need to log so many blob fetches. Let's not.

Reviewed By: HarveyHunt

Differential Revision: D19766017

fbshipit-source-id: 674dee276234f96938a9459af18dd78d09243350
2020-02-06 12:08:43 -08:00
Thomas Orozco
8d2b2f7af2 mononoke/multiplexedblob: make sampling rate configurable
Summary: This will let us lower Scuba utilization from Fastreplay.

Reviewed By: HarveyHunt

Differential Revision: D19766018

fbshipit-source-id: 4eac19b929914db910ed13096b2a5910c134ed3a
2020-02-06 12:08:43 -08:00
Mark Thomas
8a829d6f01 scs_server: remove compat around Mononoke::new
Reviewed By: farnz

Differential Revision: D19770416

fbshipit-source-id: 2943d922d0b49a553b410d73c6b520c83f58c35e
2020-02-06 09:14:05 -08:00
Mark Thomas
060eefff74 mononoke_api: requests for blame information for rejected blames are a request error
Summary:
If the user requests blame information for a file where the blame was rejected
(either becuase the file is too big, or because it is binary), this should be
considered a request error.

Reviewed By: farnz

Differential Revision: D19768261

fbshipit-source-id: 7f0d7ba53fe1087b68f4432ec0c6de0353dc3885
2020-02-06 08:33:59 -08:00
Stanislau Hlebik
688a7aeb35 mononoke: remove compat() methods
Summary: They are not used much - let's use new futures instead

Reviewed By: krallin

Differential Revision: D19767952

fbshipit-source-id: c04bcf5efc6f8ee6f1d31254fcb2cb4603769b91
2020-02-06 08:00:16 -08:00
Liubov Dmitrieva
8228f84a60 Short hashes lookup: implement suggestions the same way as in Mercurial.
Summary:
Suggestions come in the error message as it is currently implemented in
Mercurial code. Format of suggestions also stays the same.

We give the hash, time, author and the title.

All suggestions are ordered (most recent go first).

We don't show them if there are two many.

Reviewed By: krallin

Differential Revision: D19732053

fbshipit-source-id: b94154cbc5a4f440a0053fc3fac2bca2ae0b7119
2020-02-06 07:43:51 -08:00
Stanislau Hlebik
af2f50d644 mononoke: add x_repo_lookup to scsc
Summary:
Useful for debugging.

I also fixed how we open a SqlSyncedCommitMapping, because we used incorrect path for that.

Reviewed By: ikostia

Differential Revision: D19767148

fbshipit-source-id: baf67bceceb7b22429b05b41020cf4350e3c87bd
2020-02-06 07:26:14 -08:00
Stanislau Hlebik
c8a4ba5dac mononoke: commit sync thrift api
Summary:
This is the api that will be used by Sandcastle to remap a commit from one repo
to another.
Previously the implementation api was just looking in the commit mapping table,
but that's not enough - draft commit cloud commits are not in this table, so we
actually need to sync them.

There's a caveat though - we allow syncing public commits from a large repo to
a small repo, but not the other way around. Comment in the code has more info
about it.

Reviewed By: ikostia

Differential Revision: D19718839

fbshipit-source-id: 9939530f818fafd22bc3838b4647dd9cbc1c8c07
2020-02-06 07:26:14 -08:00
Stanislau Hlebik
c97ceda175 mononoke: update IncompleteFilenodes to make transition to FilenodesOnlyPublic
Summary:
Jump from "generating filenodes while generating hg changeset" to "generate
filenodes separately" is tricky to do without breaking production. This diff
adds additional logic in IncompleteFilenodes that should make this transition
smoother. See code comment for more details.

Reviewed By: krallin

Differential Revision: D19741913

fbshipit-source-id: 48987c15fc4144c50afcee7ae34072f6cd634271
2020-02-06 07:26:14 -08:00
Lukasz Piatkowski
415bab4e4b Merge mononoke sources into the eden repository
fbshipit-source-id: 43a0252cb3ec42aa365f20d1b6faa4d24d74c9b8
2020-02-06 14:01:58 +01:00
Lukasz Piatkowski
e8d62b64d5 mononoke: move the codebase under eden/ directory
fbshipit-source-id: 43a0252cb3ec42aa365f20d1b6faa4d24d74c9b8
2020-02-06 13:46:04 +01:00
Xavier Deguillard
3c41bd2a92 build: use vendored crate from third-party
Summary:
Now that the source of truth for third-party crates is in fbsource, let's use
it in our cargo build system. This removes the need to fetch a tarball and
untar it, which should have the benefit of speeding up the build.

A small caveat is the first build on EdenFS will be slightly slower, due to
crates needing to be individually fetched, subsequent builds will be faster.

Reviewed By: jsgf

Differential Revision: D19726217

fbshipit-source-id: 24f484d1e3118a76e052f07ff3eea0c66cccce96
2020-02-05 17:49:06 -08:00
Chad Austin
4e1f60fc24 log checkout type and fetch counts to scuba
Summary: In addition to duration and success, log object fetch counts and checkout type to Scuba.

Reviewed By: fanzeyi

Differential Revision: D19334276

fbshipit-source-id: dabf52427f2ebda2b58df93194df39d52f4fcb4f
2020-02-05 16:05:30 -08:00
Chad Austin
cb1c0b45f3 log fetch statistics after checkout
Summary: Log the number of object lookups and cache hit rates for a checkout operation.

Reviewed By: simpkins

Differential Revision: D19191201

fbshipit-source-id: 5e9ad501e704810f072dabcda3fce86d027c452e
2020-02-05 16:05:30 -08:00
Chad Austin
58f352b807 count fetch statistics during diff and checkout
Summary:
During checkout and stats, count every object fetch and which level of
cache it was served from.

Reviewed By: simpkins

Differential Revision: D19186333

fbshipit-source-id: fc0a74db297b9c723682e245996a7befd762f933
2020-02-05 16:05:29 -08:00
Xavier Deguillard
6ea4bb998e revisionstore: move memcache initialization to a background thread
Summary:
As initializing the memcache client takes ~0.7s, let's move it to a background
thread as to not impact Mercurial startup time. This diff uses ArcSwap in
order to reduce the overhead of the very common read paths as much as possible.
Using Mutex or RwLock instead would have caused unecessary contention.

Reviewed By: DurhamG

Differential Revision: D19518693

fbshipit-source-id: 886e9b86813fda6ff005ccce99659890026f643a
2020-02-05 14:01:54 -08:00
Xavier Deguillard
b8947748b5 pyrevisionstore: expose the memcache client to python
Summary:
This allows the Python code to build a memcache client and build ContentStore
and MetadataStore with it.

Reviewed By: DurhamG

Differential Revision: D19518694

fbshipit-source-id: d932fd5223ccfdf37db69cbb54a11a6571312709
2020-02-05 14:01:54 -08:00
Xavier Deguillard
920ea27a17 revisionstore: add memcache client
Summary:
This enables an in-process memcache client for the Rust
ContentStore/MetadataStore. For now, this implementation is lacking several
necessary optimization:
 - Start-up time is always slowed down by ~0.7s, the initialization will be
   moved to a background thread
 - Writing data to memcache is blocking and will be moved to a background
   thread too.
 - Prefetching data does a roundtrip to memcache for every key, batching
   memcache APIs will be added.

Compared to the existing hg_memcache_client, this implementation is both
significantly shorter and do not exhibit some of the pathological behavior of
having to flush the indexedlog for every fetched blob when used in Eden.

Reviewed By: DurhamG

Differential Revision: D19518696

fbshipit-source-id: 4725447d13e7eddd9586135c2511e13ddb921771
2020-02-05 14:01:53 -08:00
Chad Austin
fc07c3b6e6 add an ObjectFetchContext interface
Summary:
Add a fetch context interface to ObjectStore that allows tracing cache
hits, backing store fetches, and fetch durations in the context of a
diff or checkout operation.

Reviewed By: simpkins

Differential Revision: D19135625

fbshipit-source-id: d0d8f134b1c89f7ba4971a404a46a69a1704ba5c
2020-02-05 13:15:01 -08:00
Xavier Deguillard
b192e0b54b demandimport: re-enable
Summary:
Partially backport upstream
https://www.mercurial-scm.org/repo/hg/rev/f81c17ec303c to enable lazy loading
of python code contained in edenscmdeps3.zip.

Also, temporarily disabling the demandimport on Python3 is a bit tricky, for
the reasons mentioned in the deactivated function. Thus, instead of using the
disabled function, let's use the deactivated one.

Reviewed By: DurhamG

Differential Revision: D19672866

fbshipit-source-id: c9e39ed044121d962af1cc46745bdec72629c579
2020-02-05 11:23:29 -08:00
Chad Austin
80b1296a05 add helper for setting default values in toml tables
Summary:
cpptoml has traversal functionality for table reads, but not for
writes. Add a helper function for reading a config value and updating
the TOML table if it's unset.

Reviewed By: fanzeyi

Differential Revision: D19671264

fbshipit-source-id: e2b78d338af35d51fddaa258b7f45f8966d00a26
2020-02-05 11:20:26 -08:00
Puneet Kaushik
9a132380fc Update the state transitions in Windows EdenMount
Summary: This diff updates the states transitions in the EdenMount on Windows. It starts as State::UNINITIALIZED and transitions to State::RUNNING when the start is called. It will transition to State::SHUT_DOWN on stop or destroy. Destroy will put it in State::DESTROYING, from which it should not return.

Reviewed By: chadaustin

Differential Revision: D19559271

fbshipit-source-id: d76983cab610cb9b2c896807cf1fe49c357f8095
2020-02-05 10:37:00 -08:00
Michael Devine
0dceab8e95 New custom immutable revision object to track commit conversions: conversionrevision
Summary:
The Mercurial convert extension passes around parameters to indicate a commit that needs to be converted from source to sink. For existing converters like Git, this is a simple 1:1 conversion: a commit in the source gets mapped to a commit in the sink, and so they use the source commit hash (sometimes called rev or version in the API) to represent the commit to be converted.

Our converter is much more complicated. Source commits get converted multiple times to account for different ways of mounting it into the destination file system and commit history. The commits are also coming from multiple source Git projects. This means that we need multiple pieces of data to represent a single commit conversion action.

Thus far, we've been trying to meet part of this need by using concatenated strings of (variant, commithash). This logic is breaking down as we add more fields. This commit adds a new immutable object called "conversionrevision" that represents the (variant, source commit hash, source project name, destination path) that is the unique identifier for the individual commit conversions we need to perform. This commit also includes logic for serializing and deserializing these objects as strings (useful because the converter seems to require commit IDs to be strings) and unit tests for all of the new logic.

Reviewed By: tchebb

Differential Revision: D19606867

fbshipit-source-id: 77815ca858f841d452874e95dfa3b351bafde306
2020-02-05 10:20:19 -08:00
Saurabh Singh
cbf6db3c58 py3: mark tests failing for Python 3 as requiring Python 2
Reviewed By: xavierd

Differential Revision: D19744346

fbshipit-source-id: b9defccfb396218b2d2fc4d4a2aee438468dcc60
2020-02-05 09:41:55 -08:00
Durham Goode
d0e41ff822 py3: fix test-remotenames-basic.t on Windows
Summary:
When I removed an hggit test case from this test last week, it caused
it to stop being skipped and therefore runs on Windows. The filterpwd magic
doesn't work there, and it's unnecessary, so let's just drop it.

Reviewed By: singhsrb, xavierd

Differential Revision: D19744329

fbshipit-source-id: 21f5c67d4fa7a61f14bbacd78756e5397fd6c819
2020-02-05 09:32:36 -08:00
Saurabh Singh
de006ecee1 test-rebase-inmemory-mergedriver: make the test output more stable
Summary:
Write output using `ui` object instead of using `print` to improve
robustness.

Reviewed By: xavierd

Differential Revision: D19743159

fbshipit-source-id: d0e8d581c65a79784b9843d691c4802ec37d5933
2020-02-05 08:50:54 -08:00
Jun Wu
1bcfec407d py3: remove pycompat3
Summary:
Use `abc.ABC` from Python 3 stdlib directly. The definition matches
`pycompat3.py`:

  class ABC(metaclass=ABCMeta):
      pass

The following changes are reverted since they're no longer necessary:

  D19732319 "[hg] py3: fix windows build"
  D19703778 "[hg] py3: exclude mercurial/pycompat3.py from Python 2 builds"
  D19703779 "[hg] py3: exclude pycompat3.py from Buck-based Python 2 builds"

Reviewed By: simpkins, singhsrb

Differential Revision: D19739075

fbshipit-source-id: 8c1e3727e8a88ff5f7232270d528d690523b1824
2020-02-04 22:15:16 -08:00
Saurabh Singh
8f5ec4b183 test-status-inprocess: make the test output more stable
Summary:
Write output using `ui` object instead of using `print` to improve
robustness.

Reviewed By: quark-zju

Differential Revision: D19738486

fbshipit-source-id: 4da80ce7efef44e9c5694ed71a55b7d64629166c
2020-02-04 20:36:42 -08:00
Saurabh Singh
5c37bfd373 nointerrupt: make output more stable
Summary:
Printing via `ui` object instead of directly to the `stderr` makes the
output more consistent.

Reviewed By: quark-zju

Differential Revision: D19738355

fbshipit-source-id: 384bf2de11f2ae7b5264ec5d94b041502c4ef7fb
2020-02-04 20:36:42 -08:00
Jun Wu
7316c4cc22 cpython-ext: add a way to wrap Rust Write object into a Python object
Summary:
The library already has a way to wrap a Python object into a Rust object that
exposes the Rust Read/Write interface. This is the reverse direction for
the Write interface.

The initial intention is to expose Rust stdout as described in D19702533.
However, I found Python's `sys.stdout.buffer` also enforces utf-8 encoding
on Windows (unless PYTHONLEGACYWINDOWSSTDIO is set). So Python's
stdout actually behaves similarly with Rust's stdout on Windows and is okay
to use. That said, it's still useful to have this abstraction, for streampager [1]
integration.

[1]: https://github.com/markbt/streampager/

Reviewed By: sfilipco

Differential Revision: D19716127

fbshipit-source-id: ba39898122561d9a49b7080ee95d7c940540eb40
2020-02-04 18:41:13 -08:00
Wez Furlong
650575122d eden: fix fuse init on linux kernel 5.4 and up
Summary:
1fb027d759
changed the kernel behavior to reject reads smaller than 8KB,
even for requests that would never need to be that large.

That causes eden to fail to start up on eg: Fedora 31 with a 5.4 kernel.

This commit adds some padding to satisfy this new check.

Reviewed By: chadaustin

Differential Revision: D19736893

fbshipit-source-id: 926456d72124b186976ee9a8a21242e93c26f790
2020-02-04 18:18:44 -08:00
David Tolnay
d358c71e6d Update to Rust 1.41.0
Reviewed By: jsgf

Differential Revision: D19656956

fbshipit-source-id: ef791638021ae55375ca35ca191cf1b8a3cc5d7f
2020-02-04 18:03:03 -08:00
Durham Goode
d3227d34c2 py3: remove forced decoding of environment variable
Summary:
On python 2, os.environ is expected to be bytes, so let's remove the
forced decoding. This broke run-tests.py on Windows.

Reviewed By: singhsrb

Differential Revision: D19737218

fbshipit-source-id: d4782eea279275f57cc91bc412a2d74857002cc1
2020-02-04 17:31:45 -08:00
David Tolnay
34a520536a Update rustfmt and reformat fbsource
Summary:
```
$ tools/third-party/rustfmt/rustfmt --version
rustfmt 1.4.11-nightly (1838235 2019-12-03)
```

Reviewed By: zertosh

Differential Revision: D19704678

fbshipit-source-id: fe8707e964495e76746edcb8b68e34fc1411f52a
2020-02-04 17:14:27 -08:00
Durham Goode
fd47fd9202 py3: fix windows build
Summary:
We added pycompat3.py which only parses in Python 3. We need to
exclude it from the Windows build as well.

Unfortunately, the Windows build users PyZipFile.writepy which writes an entire
directory. Let's copy in the implementation from Python 3 which allows us to
filter out certain files.

Reviewed By: xavierd

Differential Revision: D19732319

fbshipit-source-id: 4cebb434d052c51d9ae0dccdec5eadd1f412f9e5
2020-02-04 15:40:35 -08:00
Carolyn Busch
d49c251d13 py3 iteritems -> pycompat.iteritems
Summary: More replacements of dict.iteritems with pycompat.iteritems(dict) for py3 compatibility.

Reviewed By: singhsrb

Differential Revision: D19704211

fbshipit-source-id: 544cb292c0241a293fc4ab83c3d9472dcf4328b6
2020-02-04 15:29:39 -08:00
Saurabh Singh
3d78cca814 util: remove redundant 'timed' function
Summary:
`timed` has been replaced by the `timefunction` in util.

I have also removed the existing usages because they seemed relevant only
during the development.

Reviewed By: quark-zju

Differential Revision: D19727919

fbshipit-source-id: 58c466f02ac2a5bf7096948b17b49ceb26fc1fd9
2020-02-04 12:33:40 -08:00
Durham Goode
16efd0a0e0 py3: fix osx tests
Summary:
The path normalization OSX logic was using unicode, and when it
switched to use pycompat in D19613690 it stopped using unicode, which broke it.
Let's roll it back.

Reviewed By: singhsrb

Differential Revision: D19726322

fbshipit-source-id: fa7bfacdf41dfedbcfef2fdb5ccb826196101e61
2020-02-04 12:19:13 -08:00
Adam Simpkins
0e1437c823 py3: add some additional type annotations on functions writing file data
Summary:
Add a few additional type annotations on some functions.
This did require some minor code restructuring in `filestore.setfile()` to
work around pyre's unwillingness to unwrap `Optional` member variables.

Reviewed By: quark-zju

Differential Revision: D19715243

fbshipit-source-id: 8078329c1e4a50ad0aa6c765d42a89b5ed58e7bf
2020-02-03 21:36:56 -08:00
Adam Simpkins
0657c35391 py3: add more vfs-related type annotations
Summary:
Add more type annotations to vfs.py and some of the platform-specific
functions that it calls.

This did catch a couple of cases where string/bytes conversion were not being
done properly in Python 3.

Reviewed By: quark-zju

Differential Revision: D19678717

fbshipit-source-id: b3979cc5e0668ddbb93372dee25ef02ea1867d6f
2020-02-03 21:36:56 -08:00
Jun Wu
9602130aab py3: chg: fix compatibility
Summary:
Fix various type issues when running chg under Python 3.
Enable chg in setup3.py build.

This should make tests run faster. For example, test-rebase-detach.t
now completes in 8 seconds, down from 29 seconds.

Reviewed By: xavierd

Differential Revision: D19702535

fbshipit-source-id: 8928b1b920b9b52fd03dc86f996da18f2405f146
2020-02-03 18:26:57 -08:00
Jun Wu
3e0b781197 py3: only use binary stdin/stdout/stderr
Summary:
Drop stdoutbytes/stdinbytes. They make things unnecessarily complicated
(especially for chg / Rust dispatch entry point).

The new idea is IO are using bytes. Text are written in utf-8 (Python 3) or
local encoding (Python 2). To make stdout behave reasonably on systems not
using utf-8 locale (ex. Windows), we might add a Rust binding to Rust's stdout,
which does the right thing:
- When writing to stdout console, expect text to be utf-8 encoded and do proper decoding.
- Wehn writing to stdout file, write the raw bytes without translation.

Note Python's `sys.stdout.buffer` does not do translation when writing to stdout console
like Rust's stdout.

For now, my main motivation of this change is to fix chg on Python 3.

Reviewed By: xavierd

Differential Revision: D19702533

fbshipit-source-id: 74704c83e1b200ff66fb3a2d23d97ff21c7239c8
2020-02-03 18:26:57 -08:00
Durham Goode
16d4739981 fsmonitor: catch watchman issue and fallback
Summary:
My earlier refactor moved a watchmanclient.getcurrentclock() call out
of a "try/except Exception" block. This meant any watchman errors broke the hg
status entirely, instead of falling back to the old status path.

Let's add the logic around this watchman path.

Reviewed By: sfilipco

Differential Revision: D19709391

fbshipit-source-id: bfb0221ba405a926babddaf73c8e70924e4d9b25
2020-02-03 18:04:25 -08:00
Jun Wu
2777fea957 py3: localrepo: manually fixup some type annotations
Summary: This might help pyre check more things.

Reviewed By: farnz

Differential Revision: D19678144

fbshipit-source-id: a04207ccb04ce76796e9a0a7326a8d527d17cff1
2020-02-03 17:07:36 -08:00
Jun Wu
165a8d4f8d py3: add stub for localrepository
Summary:
The "repo" type is a bit hard to type since it might be wrapped by extensions,
and the decorators like "unfilteredmethod" are too hard for type checkers.

Provide a stub for it so typecheck can work. The stub was generated by `pytype`,
I made a small change to `close` to make pyre happy.

Reviewed By: DurhamG

Differential Revision: D19677410

fbshipit-source-id: 71afb3fe78a75ee269c8005d18eee3a807b50df6
2020-02-03 17:07:36 -08:00
Aida Getoeva
38fca29eaf hg-py3: fix restack-multidest/t and fold
Reviewed By: xavierd

Differential Revision: D19700265

fbshipit-source-id: 557cec8388d5c7759d24e1b995f971e72f925a30
2020-02-03 17:05:35 -08:00
Aida Getoeva
acc0425925 hg-py3: make amend-next/prev work
Reviewed By: xavierd

Differential Revision: D19700028

fbshipit-source-id: 9aaab361058a2dfa34f4d0c9218c71c1654aaa8a
2020-02-03 17:05:34 -08:00
Wez Furlong
4c5168658f eden: teach eden redirect to use new apfs helper tool
Summary:
This commit enables mounting and unmounting using the
new helper, and fixes up the behavior of `eden redirect list`
and `eden gc` so that they have reasonable behavior when APFS
is in the mix.

The semantics of the `TARGET` field in the redirect list output
become a bit hazy when using APFS volumes; it's a bit less important
because the data isn't effectively moved to a different part of
the filesystem namespace: we're just mounting a different filesystem
over the top and it can only be accessed through its mountpoint.

Reviewed By: chadaustin

Differential Revision: D19403162

fbshipit-source-id: 136a7b02f605cde0a74016608179924be801eda9
2020-02-03 16:59:18 -08:00
Wez Furlong
30d0a77d53 eden: add helper command for working with apfs
Summary:
We've had a small proportion of our users run into problems
with hdiutil and diskimages-helper, where those components get into
an unhappy state and effectively block operating on the redirections.

This diff introduces a new utility that is intended to replace the
use of disk image files with APFS volumes that we mount in the
appropriate places.

The intention is that we will teach `eden redirect` to use this tool
when available, rather than disk images.

macOS's security model is weird: it is perfectly valid for a non-privileged
user to create and delete APFS volumes in the APFS storage container,
but root privs are required to mount it into the VFS.

The intent is that we deploy this utility setuid root to minimize
the fan out--this way we won't need to teach the priv helper about
this kind of redirection.

There are a couple of subcommands demonstrated in the test plan.

Reviewed By: chadaustin

Differential Revision: D19323850

fbshipit-source-id: 35556f841e49e5c4b77679b756af9093222f4500
2020-02-03 16:59:17 -08:00
Chad Austin
ad6b3f5323 require eden rm be given the checkout root
Summary:
To avoid accidents when people run `eden rm` with a subdirectory of
the mount, require that it be given the checkout root (or a symlink to
it).

Reviewed By: genevievehelsel

Differential Revision: D19549912

fbshipit-source-id: ae0a43e9164f73f5ad9c2f40f1b7226a1e85a567
2020-02-03 16:27:09 -08:00
Adam Simpkins
b6ce7a08a6 py3: make vfs.join() consistent across subclasses; add type hints
Summary:
Add type hints for `vfs.join()`.  Not all subclasses supported joining
multiple arguments, so fix them all to actually implement the full API.

Reviewed By: farnz

Differential Revision: D19678712

fbshipit-source-id: f7f77eee416cd5edda756a8423cbe3630e195d4a
2020-02-03 15:27:52 -08:00
Adam Simpkins
bb9599d1ba py3: exclude mercurial/pycompat3.py from Python 2 builds
Summary:
Update setup.py to exclude `edenscm/mercurial/pycompat3.py` when building
for Python 2.

Reviewed By: quark-zju

Differential Revision: D19703778

fbshipit-source-id: 269f1d4e746eb5112a9e577f43c040f2946b07d1
2020-02-03 15:13:09 -08:00
Yedidya Feldblum
8e6f529d4a Use cacheline_aligned in Tracer
Summary: [Eden] Use `cacheline_aligned` in `Tracer`.

Reviewed By: aary

Differential Revision: D19628287

fbshipit-source-id: 4730c995bc9c11a3d1e485971eef54cb9b329aa8
2020-02-03 14:47:07 -08:00
Sam Uong
974db0c7cd hgext: add hg equivalent for 'git stash clear'
Reviewed By: farnz

Differential Revision: D19669269

fbshipit-source-id: 2d33aae66b1733b9085adb829b38a07afd80fdd8
2020-02-03 12:59:46 -08:00
Chad Austin
49102f1925 switch unsafe folly::collect calls to collectSafe
Summary:
There were some unsafe uses of folly::collect, especially in
the diff code. Switch to collectSafe.

Reviewed By: simpkins

Differential Revision: D19529784

fbshipit-source-id: e0ac920a2d2b44767a453f2c41a9166e72246ab6
2020-02-03 11:30:50 -08:00
Chad Austin
0a28623f9a add a collectSafe function
Summary:
folly::collect's Future is completed immediately when any of its input
future is completed with an exception. This makes forking off subtasks
and then joining with folly::collect dangerous when closures don't
hold pointers or references to parameters, including `this`. Introduce
a collectSafe function, which has the same signature as folly::collect
but unconditionally waits until all futures are completed.

Reviewed By: simpkins

Differential Revision: D19529772

fbshipit-source-id: b3b493fcbb0d9058d2d6e5c8a064dfdae3a43e57
2020-02-03 11:30:50 -08:00
Aida Getoeva
201a17ca37 hg-py3: make amend-to work
Reviewed By: quark-zju

Differential Revision: D19673900

fbshipit-source-id: cfdc985716aec23335eeddef517855be65505576
2020-02-03 10:04:38 -08:00
Jun Wu
a8f232e0ab py3: drawdag: fix some types
Summary: This fixed using drawdag to create merge commits.

Reviewed By: xavierd

Differential Revision: D19697893

fbshipit-source-id: fac4278a54d2cf2eb2b9d8154a13edd29e1a510a
2020-02-03 09:54:12 -08:00
Simon Farnsworth
ad40d8567f py3: fix up matchers to exclusively use unicode
Summary: Matchers were mixing str and bytes - use decodeutf8 on the place using bytes, and enable all the tests that now pass.

Reviewed By: quark-zju

Differential Revision: D19679076

fbshipit-source-id: 2b90cfbf690b2365a1302efd9db72347a295c977
2020-02-03 08:00:11 -08:00
Jun Wu
18a20f5301 context: add some type annotations
Summary: This gives type checker some hints. Pyre can now look into more areas.

Reviewed By: markbt

Differential Revision: D19676085

fbshipit-source-id: db05b4238051953eb7f8af44da220825ec589c3c
2020-02-03 07:35:32 -08:00
Jun Wu
4e272b6fad revlog: add some type annotations
Summary:
This gives type checker some hints. Pyre was able to find some issues,
which are fixed.

Reviewed By: markbt

Differential Revision: D19674436

fbshipit-source-id: 44d6df6037f27e2edeff4d343a9fdec5e51420b3
2020-02-03 07:35:32 -08:00
Jun Wu
348ce20341 py3: add type annotations to i18n
Summary: This allows type checkers to understand the i18n._ calls.

Reviewed By: markbt

Differential Revision: D19673677

fbshipit-source-id: be2cc72f70e56925007485c786f0d72b5cd40a1a
2020-02-03 07:35:31 -08:00
Durham Goode
14ec2d4b34 py3: enable more tests
Summary: These now pass

Reviewed By: farnz

Differential Revision: D19678869

fbshipit-source-id: 68601ff6a6d33b13947210eecfecd999aa2ddb1a
2020-02-01 10:39:39 -08:00
Adam Simpkins
df74c3e8fa py3: start adding type hints in mercurial/vfs.py
Summary:
Begin adding type hints to many of the vfs methods.

These type hints indicate that the vfs objects are always opened in binary
mode.  While it is possible to open a vfs in text mode if you explicitly
specify the argument `text=True`, this is very rare.  If we run into cases
in the future that do need type hints to indicate TextIO mode I think it would
be better to split those out into completely separate functions, rather than
having functions that sometimes return bytes and sometimes return str based on
the arguments used.

Reviewed By: markbt

Differential Revision: D19673538

fbshipit-source-id: 8683223f28964d4b43fe131d4f16f8877dcbe777
2020-02-01 10:32:51 -08:00
Adam Simpkins
3c8739e179 py3: add type hints for util.atomictempfile
Summary:
Make `atomictempfile` derive from `typing.BinaryIO` so that the type checker
understands it can be used like a file.

This required defining all `BinaryIO` methods on this class.  This does result
in a fair amount of additional boilerplate code, but seems like the easiest
approach for now.

Reviewed By: markbt

Differential Revision: D19673539

fbshipit-source-id: b5775ca79b3af456b45e4ef5480711d7095e7949
2020-02-01 10:32:51 -08:00
Simon Farnsworth
eb5e4fb57e py3: fix share command
Summary: The share command was reaching into the filesystem directly - use the right utility functions instead

Reviewed By: quark-zju

Differential Revision: D19672980

fbshipit-source-id: a14323fd5419c3ea00d9c009b9a77f63862a7b2a
2020-01-31 22:13:04 -08:00
Mark Thomas
a6b159822f py3: fix escaping test-unified-test.t
Summary:
Python 3 is more strict about bad escape characters.  As far as I can tell,
this was always wrong, and when corrected, works both in Python 2 and Python 3.

Reviewed By: quark-zju

Differential Revision: D19673184

fbshipit-source-id: b8b358327db8ce2e51761f8f98784bdd4b396423
2020-01-31 22:00:59 -08:00
Mark Thomas
b36cdab6f9 py3: use new-style class for fd wrapper
Reviewed By: farnz

Differential Revision: D19672991

fbshipit-source-id: 06ce264f17b395c336c5bd936534e346d0b08a68
2020-01-31 21:28:47 -08:00
Adam Simpkins
7a61371a9d py3: move checkwinfilename() to a separate file and add type hints
Summary:
This moves `checkwinfilename()` from util.py to a new winutil.py module.
This allows the windows.py module to import winutil.py and re-expose
`checkwinfilename()` as `checkosfliename()`.  This makes pyre happier, since
all platform modules now contain a `checkosfilename` method.

This also adds type hints to `checkwinfilename()`, and updates several parts
of the code that were still incorrectly assuming the input was bytes in
Python 3.  However, note that I haven't done much testing of this on Windows.

Reviewed By: quark-zju

Differential Revision: D19672077

fbshipit-source-id: d78ed5b344a2e24c2a361b56465dedc516171d28
2020-01-31 20:18:21 -08:00
Durham Goode
3430fdd32f py3: treat sshserver wireprotocol commands and arguments as strings
Summary:
While the payload of wire commands are bytes, we should treat the
command and args as utf8.

Reviewed By: quark-zju

Differential Revision: D19675217

fbshipit-source-id: df64c842e0c099d77dec1313aa0639e46e539194
2020-01-31 18:39:42 -08:00
Durham Goode
10d673add5 py3: handle utf8 paths properly in remotefilelog
Summary: remotefilelog streaming clones need to handle paths as utf8 strings.

Reviewed By: quark-zju

Differential Revision: D19675219

fbshipit-source-id: dadf54fc67ecb6fbb23c1aeffd0c058ff5804bf5
2020-01-31 18:39:41 -08:00
Durham Goode
d37a246d60 py3: use bytes for stdin for hg serve
Summary: hg serve works strictly with bytes, so let's expose it for stdin.

Reviewed By: quark-zju

Differential Revision: D19672836

fbshipit-source-id: 6abc5385daea81c584a89f19a568e42e4afad48a
2020-01-31 18:39:41 -08:00
Durham Goode
d8fbaa7553 py3: fix sshserver to respond with bytes in most places
Summary: This moves streaming clones a bit further along.

Reviewed By: quark-zju

Differential Revision: D19672546

fbshipit-source-id: 054dd639a8067dce34a5bf667996cc5d30301190
2020-01-31 18:39:41 -08:00
Durham Goode
4e8759a594 py3: various fixes to batch and wireproto encoding
Summary:
We should generally be returning bytes on the wire. This fixes up a
number of places to do so, helping move streaming clone closer.

It also turns out that unescapearg is used in multiple situations.  Usually it's
used for strings, but for batches the actual payloads are also delimited by this
and they are bytes. So let's have a function for each case.

Reviewed By: quark-zju

Differential Revision: D19672545

fbshipit-source-id: 6386ae2f39660b042af1382a1d696903fa1de9ad
2020-01-31 18:39:40 -08:00
Durham Goode
3ce72efbc5 py3: misc fixes around streaming clones
Summary: A couple random stops that need correct encoding.

Reviewed By: quark-zju

Differential Revision: D19672547

fbshipit-source-id: edb0490f70cbf89885fe5fb0a1a8a2472d33e18d
2020-01-31 18:39:40 -08:00
Carolyn Busch
ba8ac287b8 py3 sshserver-test
Summary: Translate between bytes and unicode in sshserver to make test-sshserver pass.

Reviewed By: farnz

Differential Revision: D19672378

fbshipit-source-id: 7e1c0bc2d6fc03fac6ebcf54764862e3ed743f7e
2020-01-31 18:39:40 -08:00
Xavier Deguillard
ae089886fd setdiscovery: use the nodemap __contains__
Summary:
The changelog __contains__ expects rev numbers, not nodes, the nodemap should
be used instead.

Reviewed By: quark-zju

Differential Revision: D19674451

fbshipit-source-id: 8f0ddf3b9cebeea10f2fd639999f8c6c67a0e35d
2020-01-31 18:34:01 -08:00
Aida Getoeva
ad097305f5 hg-py3: make metaedit command work
Reviewed By: quark-zju

Differential Revision: D19666536

fbshipit-source-id: e8d02dd6ff6ab73efd94002b83319cbfe506c21f
2020-01-31 18:29:01 -08:00
Simon Farnsworth
d1268e8871 py3: ratchet up passing tests
Summary: These four tests pass on master. Let's enable them so that CI keeps us honest

Reviewed By: markbt

Differential Revision: D19672282

fbshipit-source-id: 2d45c3f9840c7323b7bfae8bbd89c4b7ec95f49d
2020-01-31 18:23:58 -08:00
Xavier Deguillard
db4849a0d6 tests: enable passing tests in python3
Summary: These are passing, let's enable them.

Reviewed By: quark-zju, singhsrb

Differential Revision: D19676094

fbshipit-source-id: bcb363382d48a93ac62dceab2dd6349b910bb51a
2020-01-31 18:12:14 -08:00
Xavier Deguillard
fd2c463b9a rebase: decode rebasestate before parsing it
Summary:
This file contains text data, let's decode it before doing anything else with
it.

Reviewed By: quark-zju

Differential Revision: D19675385

fbshipit-source-id: 8aae08c2400dd7d76c8678d5312bab060563e0ba
2020-01-31 18:12:14 -08:00
Xavier Deguillard
06fa75f637 tests: use vfs.writeutf8
Summary: This is the right method to write plain strings data to files.

Reviewed By: quark-zju

Differential Revision: D19675386

fbshipit-source-id: 1f939f79294cbc45f56944058649c384b674b880
2020-01-31 18:12:14 -08:00
Jun Wu
6462672163 pycompat: drop sysbytes, sysstr, strurl, bytesurl
Summary: There are no-ops.  This makes type checkers' life slightly easier.

Reviewed By: farnz

Differential Revision: D19673051

fbshipit-source-id: 363eb8a4e799113b16529825d45645807cafe2fc
2020-01-31 17:59:53 -08:00
Saurabh Singh
185119e3c0 test-extension: fix the test
Summary: This is broken after D19672686.

Reviewed By: quark-zju

Differential Revision: D19674321

fbshipit-source-id: 7f225b7090ce9168e10e02b25ee17ae2b98cd86a
2020-01-31 17:37:43 -08:00
Saurabh Singh
1891591abc unicode: consistently use 'utf-8' everywhere
Summary:
We generally use 'utf-8' but are using 'utf8' at some places. This
diff changes to using 'utf-8' consistently everywhere.

Reviewed By: farnz

Differential Revision: D19674111

fbshipit-source-id: 3be42859a180ded1c234b805259f086f9531ba18
2020-01-31 17:34:53 -08:00
Xavier Deguillard
57d59e5499 remotenames: write bytes to the file
Summary: The write method requires bytes, let's encode what we're writing into it.

Reviewed By: quark-zju

Differential Revision: D19674450

fbshipit-source-id: 9eaa3393d3d0e6324be39f94d7f31f2228b14412
2020-01-31 17:31:24 -08:00
Xavier Deguillard
7182c8c2ca hgdemandimport: use a set for the ignored modules
Summary:
A set gives us constant time test, while list requires a linear search. This
may speedup the importer slightly.

Reviewed By: quark-zju

Differential Revision: D19674449

fbshipit-source-id: 994a5303114d54b9cd40469b7cfe4b35e2515729
2020-01-31 17:31:23 -08:00
Xavier Deguillard
6f3af2ab10 tests: remove iteritems
Summary: The method doesn't exist in python3, let's simply replace it with items.

Reviewed By: quark-zju

Differential Revision: D19673674

fbshipit-source-id: b17eef7d820ae38db3fea1d457b0b8bb409f1b86
2020-01-31 17:31:23 -08:00
Xavier Deguillard
bf3292412a patch: add a missing b""
Summary:
The headers are bytes, therefore startswith needs to be called with a bytes
argument.

Reviewed By: quark-zju

Differential Revision: D19673675

fbshipit-source-id: 9a978ad42bf94a55c984f2ae35c566a59bfd6180
2020-01-31 17:31:23 -08:00
Adam Simpkins
bcad94171a py3: add pycompat.ABC
Summary:
Assigning to `__metaclass__` only works in Python 2.  In Python 3 this
attribute is ignored, and you instead need to specify `metaclass=` in your
class signature.

This adds a new `pycompat.ABC` base class to make it easier to define abstract
classes that are compatible with both Python 2 and 3.

Reviewed By: sfilipco

Differential Revision: D19672078

fbshipit-source-id: c56eb6c7266f962ef9c13c742e6ac0651b9c19c9
2020-01-31 17:05:54 -08:00
Adam Simpkins
3abc65f90d py3: add type annotations to ui.write()
Summary:
Add type annotations to `ui.write()` and the helper functions it calls.
I also removed unused `**opts` arguments on a few of the helper functions that
it calls.

Reviewed By: markbt

Differential Revision: D19657806

fbshipit-source-id: d020396a51d9fc3b07542c65541fca6067addb23
2020-01-31 17:05:54 -08:00
Zeyi (Rice) Fan
fa54b36b93 fix Windows tests
Summary: This fixes the Windows build failure we are seeing. Apparently Python 2 on Windows does not like unicode type in `os.environ`

Reviewed By: markbt

Differential Revision: D19672308

fbshipit-source-id: b45c1ccf47bdd46dc31d6c980d0e70dd2e084c8f
2020-01-31 16:53:12 -08:00
Saurabh Singh
2f3657f342 test-unbundlereplay: fix the test
Reviewed By: farnz

Differential Revision: D19673662

fbshipit-source-id: e2b05e9efb449ffa1af24c5b221eca1e97873b13
2020-01-31 16:47:43 -08:00
Jun Wu
5d3db79c23 pycompat: drop strkwargs / byteskwargs
Summary: There are no-ops.  This makes type checkers' life slightly easier.

Reviewed By: markbt

Differential Revision: D19673053

fbshipit-source-id: 04f26cd3ca0dbfe579ef939fc39be00035e70cee
2020-01-31 16:38:03 -08:00
Jun Wu
ae22d7d613 pycompat: drop fsencode / fsdecode
Summary:
There are basically no-op (see D19613693).
This makes type checkers' life slightly easier.

Reviewed By: farnz

Differential Revision: D19672686

fbshipit-source-id: 27ca44dd53ec9788fdbb8f31235eae88d1853415
2020-01-31 16:38:03 -08:00
Jun Wu
30f2b80d9f pycompat: drop iterbytestr
Summary: This makes type checkers' life easier.

Reviewed By: markbt

Differential Revision: D19672687

fbshipit-source-id: 335c560289e29ac7c08e36501e9841ddc51232be
2020-01-31 16:38:03 -08:00
Jun Wu
ad0c50a5cb py3: fix some type issues detected by pyre with some type stubs
Summary:
The issues were found by pyre with some type stubs generated via pytype:

  python36 -m pytype edenscm --no-report-errors -j 30

I didn't include the pytype generated stubs because most of them are `Any`.
I'm trying to see if we can get something cleaner.

Reviewed By: markbt

Differential Revision: D19672435

fbshipit-source-id: c57f2ad3a981ddd4a3a267ff1c00e7bdb71e65ca
2020-01-31 16:38:02 -08:00
Simon Farnsworth
9266779248 py3: tools to quickly remove the py2 only marking from tests
Summary:
Simple tools, based on D19664298, that use the JSON report to enable py2 only tests for py3 testing, and revert the changes to those tests that fail.

update-successes.py is just xavierd's original code

Use as needed to enable tests on py3 after debugging, A recommended workflow is in the test plan

Reviewed By: singhsrb

Differential Revision: D19670838

fbshipit-source-id: fc525941cb010c9038c1f73d89913420e0c79981
2020-01-31 16:35:39 -08:00
Xavier Deguillard
fe3fc9712f pyrevisionstore: copyfrom should be PyPathBuf, not PyBytes
Summary: This was missed while converting pyrevisionstore to PyPathBuf.

Reviewed By: markbt

Differential Revision: D19672441

fbshipit-source-id: 466a66f2e00c7f73c11a8989c22508560f423e0e
2020-01-31 16:19:25 -08:00
Mark Thomas
df702d9cb6 py3: encode communication with the worker
Reviewed By: quark-zju

Differential Revision: D19665152

fbshipit-source-id: d4cda1e48bdf0fe64047e12d3011c81b501df113
2020-01-31 15:57:38 -08:00
Mateusz Kwapich
d5a207cf18 py3: fix more diff tests
Summary: This diff enables more test while hopefully not breaking anything

Reviewed By: markbt

Differential Revision: D19671316

fbshipit-source-id: 76ecd22a094c0a40b9724a3070a01e5d294888f3
2020-01-31 15:40:54 -08:00
Xavier Deguillard
179aa8ac28 tests: make test-dispatch.py python3 compatible
Summary: Writing to a file opened in "b" mode needs byte strings.

Reviewed By: sfilipco

Differential Revision: D19668666

fbshipit-source-id: 9e8670fa080cc2a7fc611083a4ee2305d8262a3a
2020-01-31 15:31:03 -08:00
Mark Thomas
4604332170 py3: cmdutil.export should encode its output when writing to a file
Reviewed By: quark-zju

Differential Revision: D19666638

fbshipit-source-id: 66cc3a9f6efbb421b31e18fb199e6478702eb950
2020-01-31 15:27:31 -08:00
Mark Thomas
ec02a1dc49 py3: decode undolog redonode
Reviewed By: quark-zju

Differential Revision: D19666639

fbshipit-source-id: e9f5760802750c06f965be0f370b9ed185abb79f
2020-01-31 15:27:31 -08:00
Mark Thomas
f00c23e42e py3: fix encoding in test-visibility.t
Summary: The test itself needs to ensure it decodes the visibleheads file.

Reviewed By: quark-zju

Differential Revision: D19666640

fbshipit-source-id: 999ab3396b0afc7bd86db9ae1ba126730ee3f02d
2020-01-31 15:27:31 -08:00
Xavier Deguillard
299212da50 checkmessagehook: make it python3 compatible
Summary:
The commit message should already be a utf8 string. Make sure that when it is
read from a file, we decode it properly.

Reviewed By: sfilipco

Differential Revision: D19666371

fbshipit-source-id: ee24fe33b5aaf1ede54286979c9523cedd901b3a
2020-01-31 15:21:55 -08:00
Simon Farnsworth
ee586e769a fix test-unified-test.t on python 2
Summary: D19663298 changed the return code, making this test fail

Reviewed By: singhsrb

Differential Revision: D19669105

fbshipit-source-id: c105cd130dcae3889946180761b1f51f4daae527
2020-01-31 14:20:54 -08:00
Durham Goode
1e3ec1859e py3: enable some -t.py tests
Summary: These now pass.

Reviewed By: xavierd

Differential Revision: D19669905

fbshipit-source-id: b141ea6baefa6a6eb92a0c8d70c46214ea24446d
2020-01-31 13:55:11 -08:00
Xavier Deguillard
e83115ecee remotefilelog: fix hg_memcache_client protocol
Summary:
The pipe expects bytes, but the code was passing in strings. Since the protocol
is ascii only, we can encode/decode the str/bytes

Reviewed By: farnz

Differential Revision: D19666372

fbshipit-source-id: 095bf50b9e9529c4ea7d1a06ac01643bac47bca3
2020-01-31 13:51:43 -08:00
Durham Goode
aac5ebea17 py3: remove #require py2 from passing tests
Summary: These now pass

Reviewed By: xavierd

Differential Revision: D19669584

fbshipit-source-id: dd4d374c3b373367138408ae86563ae44849690c
2020-01-31 13:46:59 -08:00
Xavier Deguillard
182e362070 tests: enable some test on Python3
Summary: These are newly passing, let's enable them.

Reviewed By: quark-zju

Differential Revision: D19668516

fbshipit-source-id: d979631203e14ce6e2ec0ab0c45773ee4846ba71
2020-01-31 13:27:46 -08:00
Adam Simpkins
11e1432ff8 update the EdenFS monitor to support self-restarts
Summary:
Update the EdenFS monitor to restart itself when it receives SIGHUP.  This
allows upgrading the monitor daemon to a new version of the software without
disrupting the current running EdenFS daemon.

Reviewed By: wez

Differential Revision: D19588698

fbshipit-source-id: f0303788cee0cc9bf4ce3b7342f10a595c3874b1
2020-01-31 13:22:27 -08:00
Adam Simpkins
b03d2b6c89 update the monitor to poll for successful EdenFS startup
Summary:
Update the SpawnedEdenInstance class to wait for EdenFS to become healthy
before signalling the Future returned by `start()`.

This code works by just polling the `getStatus()` call every few hundred
milliseconds, and waiting until it returns `ALIVE`.

Currently the code will poll forever, until EdenFS either becomes healthy or
exits.  In practice we will likely configure startup timeouts in systemd,
which means that they probably are not necessary in this code.

Reviewed By: wez

Differential Revision: D19588699

fbshipit-source-id: fef57ca23fcc14da908c2e4f7f715db33b7bf841
2020-01-31 13:22:27 -08:00
Adam Simpkins
4f7fd33e29 add a new process to monitor EdenFS
Summary:
Add a simple new wrapper daemon to manage the edenfs daemon.  This is intended
to provide a few different features:

- Perform log rotation for EdenFS's output and the output of any of its
  spawned children processes.
- Help schedule restarts of EdenFS when the system looks idle.
- Provide a single process for the system to manage across graceful EdenFS
  restarts, to make management slightly simpler.

This initial commit does not perform graceful restarts yet, but has the basic
daemon management and log rotation present.

Reviewed By: wez

Differential Revision: D19588700

fbshipit-source-id: bba41c9f7efeb4417753c1d48dd72cf6d191f0c3
2020-01-31 13:22:26 -08:00
Jun Wu
e124dfebc6 pyre: add stub for "bindings"
Summary:
Add a partial stub so pyre stops complaining about "bindings" being unknown.

Pyre's "search_path" was adjusted to read the new "pystubs" directory.

Reviewed By: DurhamG

Differential Revision: D19669131

fbshipit-source-id: a592ed411d1689058405689eb657e543b7172774
2020-01-31 13:18:54 -08:00
Durham Goode
e985d51e02 py3: update simplemerge conflict logic to use bytes
Summary: Patches are bytes, so the lines we add to them need to be bytes

Reviewed By: mitrandir77

Differential Revision: D19668850

fbshipit-source-id: cb521f71d927dbe1cbe3287be37123b609889846
2020-01-31 13:12:15 -08:00
Durham Goode
4f35ef07bf py3: remotenames use writeutf8 for cached distance
Reviewed By: xavierd

Differential Revision: D19668611

fbshipit-source-id: 82d10a29e257a5779ef41675ab60b74cf97adaa3
2020-01-31 13:12:14 -08:00
Durham Goode
2eaaaa65bc py3: fix encodings for magic strings
Summary:
Fixes a few encoding issues for changegroup/bundle2 related issues. In
particular makes magic string comparison use strings.

Reviewed By: xavierd

Differential Revision: D19668613

fbshipit-source-id: c1bc8ab76d13bb04a9c440b5d9d9babfe34c5c92
2020-01-31 13:12:14 -08:00
Durham Goode
ff7266e7f5 py3: fix plural templating
Summary:
It decides whether to render the actual values by using a
isinstance(...,bytes) which isn't correct in python 3. Let's change it to str.

Reviewed By: xavierd

Differential Revision: D19668612

fbshipit-source-id: cf093754707a0a0369cfc3aa74270dd81a6560b5
2020-01-31 13:12:14 -08:00
Durham Goode
fae3d05bdb py3: encode activebookmark before putting it on the wire
Summary: The activebookmark is a string, the wire wants bytes. Encode it!

Reviewed By: farnz

Differential Revision: D19665896

fbshipit-source-id: f5afc1c5b17d90bf3f0d0703cd752d66c6f38778
2020-01-31 13:12:13 -08:00
Durham Goode
a876c6a13b py3: fix pushkey encoding by removing local encoding
Summary:
Now that bundle2 has strings everywhere, pushkey needs to also handle
encoding correctly. Previously it used local encoding, which seems strange.
Let's drop the encoding entirely since we're only dealing with since we're only dealing with utf8 string and bytes.

Reviewed By: farnz

Differential Revision: D19665894

fbshipit-source-id: ed054ab25636ac52de646329f26681ae843b6de3
2020-01-31 13:12:13 -08:00
Mateusz Kwapich
ad3f789bf9 py3: supress the write return value
Summary:
`>>> open(file).write(data)` is a common patten for writing files across the
tests. In py3 such statement return the number of bytes written in py2 the
output is none. Let's make it None in py3 as well.

Reviewed By: xavierd

Differential Revision: D19666339

fbshipit-source-id: 5424287f85f34f3aef2d1596bb476d622464564a
2020-01-31 13:00:23 -08:00
Mateusz Kwapich
e2dc4e8014 diff: update for py3
Summary:
All diff functions are (bytes, bytes) -> bytes to preserver
the original file encoding.

Because of that I had to add ui.writebytes output function that accepts
bytes for terminal output.

Reviewed By: farnz

Differential Revision: D19656673

fbshipit-source-id: b9a1e4361e825fc8c2313e8402c2bbe00f490dd4
2020-01-31 13:00:23 -08:00
Xavier Deguillard
bf8fbe7d7f grep: make it run with python3
Summary:
The input/output of a subprocess is bytes, let's encode/decode when
writing/reading from it.

Reviewed By: farnz

Differential Revision: D19665636

fbshipit-source-id: 840f6b7a0edbedb7e7de5a65eacab9e1a7f78864
2020-01-31 12:47:04 -08:00
Xavier Deguillard
3bffbe2bf8 ui: remove a b from a regexp
Summary: I've seen this while running some tests, they now go slightly further.

Reviewed By: singhsrb

Differential Revision: D19666665

fbshipit-source-id: 35ad73f383de148f3d53c7f9de24b1a1e1c51f7e
2020-01-31 12:44:09 -08:00
Carolyn Busch
d84dc2c276 py3 cmdutil fix isinstance bytes
Summary:
The msg string isn't bytes in python 3. Hg revert works in both python 2&3
using 'str' instead.

Reviewed By: farnz

Differential Revision: D19664504

fbshipit-source-id: 5e16b637a5fb282eaa97390ef1f7e8912d0359f0
2020-01-31 11:39:13 -08:00
Durham Goode
3eca8e1086 py3: enforce bytes for bundle2 parts
Summary:
We enforced this at send time, but it was hard to debug where it was
coming from. Let's enforce it at part.setdata() time so the stack points out
exactly who the problem is.

Reviewed By: farnz

Differential Revision: D19665895

fbshipit-source-id: 5a6ca0263d52cac472e1ef2aca3ee702f38258e4
2020-01-31 11:29:29 -08:00
Stefan Filip
ae8a2613a6 py3/watchmanclient: use bser encoding
Summary:
The `bser-v1` encoding is not supported for Python 3 so we are forced to
update.

Reviewed By: quark-zju

Differential Revision: D19649948

fbshipit-source-id: 0ae1800942dd967eddcd58183f9899d36256598d
2020-01-31 11:14:47 -08:00
Durham Goode
984192ebef py3: fix hg pull
Summary:
D19657765+D19656773 made bundle2 use the standard urllib which turns
bytes into strings. This messes up bundle2 caps encoding and decoding. Caps
encoding/decoding is a nested mess of strings and bytes, so let's make it all
strings until the last minute it goes over the wire.

Reviewed By: quark-zju

Differential Revision: D19665131

fbshipit-source-id: a2fd6e309ed6da2b927d2f4ead97825a16bfcaac
2020-01-31 10:56:31 -08:00
Durham Goode
c572f1e344 py3: update test-argspans.py
Summary:
test-argspans is an awful test that includes it's own line numbers in
the test assertions. When the imports were changed in D19655148, these needed to
be updated.

Reviewed By: quark-zju

Differential Revision: D19664131

fbshipit-source-id: cdd03be2fbbc9d13ac3047f9565a1f68284e9cf0
2020-01-31 10:56:31 -08:00
Durham Goode
3df423a83b py3: fix histedit json parsing
Summary:
fbhistedit adds support for json histedit commands. Unfortunately
json.loads produces unicode strings, which then fail assertions in
pycompat.encodeutf8(...). Let's make sure it produces non-unicode on python2.

Reviewed By: singhsrb

Differential Revision: D19656390

fbshipit-source-id: 11b828319b20a153aa008f03d554087b88b7763c
2020-01-31 10:56:30 -08:00
Durham Goode
850c78d8c9 errors: drop quotes around OSError paths
Summary:
Python 3 has changed which errors are OSError vs IOError. We have
slightly different formatting which makes the test awkward to be py2 and 3
compatible. Let's make the formats the same.

Reviewed By: quark-zju

Differential Revision: D19663964

fbshipit-source-id: 892461049b61d648431f300bea0da2f6f1c70066
2020-01-31 10:56:30 -08:00
Xavier Deguillard
77e09b4e47 test-fb-hgext-extutil.py: fix some python3 warning
Summary:
This is reported while running with buck. assertEquals is renamed to
assertEqual

Reviewed By: quark-zju

Differential Revision: D19665159

fbshipit-source-id: cb7c25901005c6a97f2c703c4773ea01232943b1
2020-01-31 10:53:52 -08:00
Chad Austin
daaeb5012a replace some uses of memset
Summary: Easier to zero initialize structs with braces, which defines that even padding is zeroed.

Reviewed By: wez

Differential Revision: D19655675

fbshipit-source-id: 2fd12383324029646707e93008cf9ad34e9f1dce
2020-01-31 10:50:48 -08:00
Chad Austin
4e1cd5c49a return a valid inode number when looking up an inode fails
Summary:
When importing an object from the backing store failed, we would not
fail lookup requests (e.g. lstat) so that the file could still be
removed with `rm`. Unfortunately that data would get cached, even if
the error was transient. This diff returns a correct inode number for
failed requests and limits the amount of time these fake stat()
results are cached.

Reviewed By: wez

Differential Revision: D19655425

fbshipit-source-id: 1a44ace9217f1ab3f69a2c5a6edc89d0f9c4a941
2020-01-31 10:50:48 -08:00
Xavier Deguillard
e0e01cbc81 bundle2: backout D19656773
Summary: This broke a bunch of tests. Revert it and re-enable all the tests.

Reviewed By: DurhamG

Differential Revision: D19665042

fbshipit-source-id: c3c17e3ac7e2ea028be5b5836bc8349cdf56184e
2020-01-31 10:48:19 -08:00
Chad Austin
7c6efd32e7 rename LogEvent fields so names in code match names in the log
Summary:
Multiple times I've gotten mixed up by the different casing of the
LogEvent fields versus the names in the structured log. Change the
field names to match the log names.

Reviewed By: genevievehelsel

Differential Revision: D19585307

fbshipit-source-id: 3ccbb9ec5e18155367bc85e9db00bc8d556812c4
2020-01-31 10:42:26 -08:00
Chad Austin
e033f203f7 report successful and unsuccessful mounts, including whether the overlay was clean or not
Summary:
I have a suspicion that most edenfs starts are unclean. To test that
hypothesis, add a "mount" structured event type that records whether
the mount was cleanly shut down, how long mounting took, and whether
it's a graceful restart.

Reviewed By: genevievehelsel

Differential Revision: D19585161

fbshipit-source-id: 16eedfeb6181742e64b45bca1ed3cce3edb663cb
2020-01-31 10:42:26 -08:00
Wez Furlong
e739219103 eden: rage: fix ps usage on macOS
Summary:
This has been a minor nit reported by our macOS users for a little
while.

* Skip the code that collects the process tree, as there is no
  out of the box function for this on macOS. (We still run it
  on Linux)
* Adjust the `ps` command arguments for macOS when running on macOS

Reviewed By: chadaustin

Differential Revision: D19649881

fbshipit-source-id: c2e7a434a3264951c3ecbd02dab3459f401190d6
2020-01-31 10:13:45 -08:00
Xavier Deguillard
467f4aef45 tests: mark python3 tests as passing
Summary:
Ran ./run-tests.py --json and used the following script:

  import json
  import subprocess

  with open("report.json", "r") as f:
      tests = json.load(f)
      for name, t in tests.items():
          if t["result"] == "success":
              print("%s successful" % name)
              subprocess.run("sed -i '/#require py2/d' %s" % name, shell=True)
              subprocess.run("sed -i '/require.*py2/d' %s" % name, shell=True)

Reviewed By: singhsrb

Differential Revision: D19664298

fbshipit-source-id: fa67c7c7abd110c9f0df9345daf09f2792aacd44
2020-01-31 10:13:45 -08:00
Xavier Deguillard
3545d26208 tests-rustthreading.py: the thread is renamed to _thread
Summary: The module no longer exist in Python3, it has been renamed.

Reviewed By: quark-zju

Differential Revision: D19663426

fbshipit-source-id: a805e7fa3d15335a0be4f6a8d8ae096328470728
2020-01-31 10:13:45 -08:00
Xavier Deguillard
9643c72d6e run-tests.py: exit 80 on skip
Summary: This allows unittestify to properly report skipped tests.

Reviewed By: quark-zju

Differential Revision: D19663298

fbshipit-source-id: a95d28731fbf324a6f772e9d0b1dd84f2d5968d6
2020-01-31 10:13:44 -08:00
Aida Getoeva
af1d91abf0 hg-py3: changegroup - types and encoding
Summary: Added types, added decodeutf8 where needed while packing and unpacking chunks.

Reviewed By: xavierd

Differential Revision: D19626372

fbshipit-source-id: f15cbde5fd7e01a79b8591f0174f266078852a12
2020-01-31 10:10:56 -08:00
Chad Austin
25b1c4b12e properly escape command line arguments in eden top
Summary: Command lines with newlines were messing up the output of `eden top`, so escape them like a shell would.

Reviewed By: wez

Differential Revision: D19611525

fbshipit-source-id: 96c7ac00d02ac4d0f78f4f9b56fedd71b9728585
2020-01-31 10:07:49 -08:00
Mark Thomas
4c322218db hggit: only test fetch_pack.refs if it exists
Summary:
Different versions of dulwich return different types for
`localclient.fetch_pack`.  If the returned type is a real dict, then we can't
test the inner `refs` attribute, so check if it is there before testing if it
is `None`.

Reviewed By: singhsrb

Differential Revision: D19657503

fbshipit-source-id: 63cce72fa87f615e7b33279726dcb19c2fdf5b9e
2020-01-31 09:44:27 -08:00
Simon Farnsworth
5232dcf9d1 py3: make .py tests continue to run on python 2
Summary:
Use sed to convert testutil.ddot requires to hghave.

ignore-conflict-markers

for test-simplemerge.py

Reviewed By: simpkins

Differential Revision: D19658355

fbshipit-source-id: afae73eb1e43ead79514dfaf9f911f51ac25972e
2020-01-31 00:18:22 -08:00
Simon Farnsworth
4e10dbb9f6 py3: Back out "[hg] py3: re-enable demandimport"
Summary:
This breaks test-alias.t, so it looks like demandimport no longer behaves properly.

Original commit changeset: d56814049181

Reviewed By: simpkins

Differential Revision: D19658109

fbshipit-source-id: 9fd4865bd00d2307afe0c99f1500b6e08fe3e785
2020-01-31 00:14:04 -08:00
Adam Simpkins
1a359987cd py3: update urllibcompat.quote() to operate on strings again
Summary:
D19652082 changed `urllibcompat.urlreq.quote()` to always return bytes.
This broke most of the code outside of the bundle2 module that was expecting
it to operate on strings.

This changes `quote()` to operate on strings again, since this is what the
Python 3 standard library does.  This updates `bundle2` to handle encoding and
decoding properly when parsing strings that go through `quote()` and
`unquote()`.

This also changes bundle2 to stop going through the `urllibcompat.urlreq`
wrapper class, since that wrapper class prevents type checking from being
performed on its members.

Reviewed By: quark-zju

Differential Revision: D19657765

fbshipit-source-id: 0611838213ec22221e0b8367e1f0d8a6dcfceb15
2020-01-30 23:22:02 -08:00
Adam Simpkins
faf9de663d fix filelog to work with Python 2 again
Summary:
D19656501 added a usage of the `b"%b"` format specifier, which only works in
Python 3.  Change the code to use `b"".join()` for now, which works in both
Python 2 and Python 3.

Reviewed By: farnz

Differential Revision: D19657472

fbshipit-source-id: 22d3123f4939fa984ffd17516e1cc48bad2bb2b7
2020-01-30 20:59:49 -08:00
Saurabh Singh
13a2e1f8ed memcommit: refactor code to be pyre friendly
Summary:
This diff allows us to remove the `type: ignore` specification on the
affected lines.

Reviewed By: farnz

Differential Revision: D19657442

fbshipit-source-id: 4626daf20c57fbb511b203e54f2575edc619f40f
2020-01-30 20:08:34 -08:00
Jun Wu
935cf597c6 py3: hggit: fix absolute import
Reviewed By: singhsrb

Differential Revision: D19657557

fbshipit-source-id: dc0ba4c11f4ad69dc7e5e811875fb78c4fb492b9
2020-01-30 20:05:01 -08:00
Adam Simpkins
fabce3a798 py3: fix typing for journalentry
Summary:
Fix the type annotation for `journalentry.serialize()`: It needs to come
immediately after the method declaration to be recognized, not after the
docstring.

Fixing this causes pyre to produce type errors for this method.  I have
changed `journalentry` to derive from `typing.NamedTuple` to fix this.  This
syntax is somewhat awkward with Python 2, but is much cleaner in Python 3.

Reviewed By: quark-zju

Differential Revision: D19656758

fbshipit-source-id: a97b8a092bff0bdc7e9dee0650c35ef383b32248
2020-01-30 19:57:05 -08:00
Mark Thomas
18ecb01b8a mutationstore: update tests so that user is now a string
Summary: D19649887 changed mutation entry users to be strings.  Update the tests accordingly.

Reviewed By: simpkins

Differential Revision: D19656792

fbshipit-source-id: fcff677099dc0200130bf30eadaaf66822c6139c
2020-01-30 19:54:45 -08:00
Zeyi (Rice) Fan
92f6f35e7a mark all tests requiring Python 2
Summary:
This diff marks **ALL** mercurial tests requiring Python 2 feature.

After you fixes some tests, simply remove the `py2` feature requirement and that tests will be continuously run after your diff is landed.

To bypass this feature requirement, run the tests command with `HGTEST_FORCE_PY2=1`. For example:

```
HGTEST_FORCE_PY2=1 buck test //eden/scm/tests:hg_run_tests
```

or

```
HGTEST_FORCE_PY2=1 python run-tests.py
```

----

Basically this diff are created with the following commands:

```
$ sed -i 's/import feature\(.*\)$/import feature\1\n\nfeature.require(["py2"])/' test-*-t.py
$ sed -i '1s/^/#require py2\n/' test-*.t
$ ls | grep -P "^test.*(?<\!-t)\.py$" > list && vim -p $(cat list)
# manually adding feature requires for these Python tests.
```

(Note: this ignores all push blocking failures!)

ignore-conflict-markers

Reviewed By: singhsrb

Differential Revision: D19655148

fbshipit-source-id: 985e3ccb4010cc559049f1d89f8909bc2d9b5e20
2020-01-30 18:49:21 -08:00
Zeyi (Rice) Fan
cdd601bfac change py3k feature to py2
Summary:
This is used in D19655148 to skip all the tests that are not passing in Python 3.

(Note: this ignores all push blocking failures!)

Reviewed By: farnz

Differential Revision: D19655164

fbshipit-source-id: 1a0d26926aa1b764077fa1e5bdddf853621a54f1
2020-01-30 18:49:20 -08:00
Xavier Deguillard
5c00ebfd06 bundle2: fix encoding issue
Summary: The urlreq.quote returns a byte string, let's use b string to manipulate it.

Reviewed By: farnz

Differential Revision: D19656773

fbshipit-source-id: 1cbcc0f19e9463d5eabec2f67f5b00329188777d
2020-01-30 18:45:36 -08:00
Xavier Deguillard
d169de8946 bookmarks: binaryencode should return a byte string
Reviewed By: quark-zju

Differential Revision: D19656320

fbshipit-source-id: 85b0015dd3a8c485bbc5fb6ba854dbb7273ddc11
2020-01-30 18:45:36 -08:00
Xavier Deguillard
1629a2e038 remotenames: encoding fixes
Summary: Reading/writing to files is done via bytes, not strings.

Reviewed By: quark-zju

Differential Revision: D19656321

fbshipit-source-id: 9a9b13b27fc9dd8db6fe757dcd53e4730179832d
2020-01-30 18:45:36 -08:00
Xavier Deguillard
d6d60bda6a clone: make the samplehgrcs strings
Summary: These aren't bytes, they are strings.

Reviewed By: quark-zju

Differential Revision: D19656112

fbshipit-source-id: fcd505e3b4e522154358c80305e39ebca5936395
2020-01-30 18:45:35 -08:00
Adam Simpkins
a4722f9515 py3: fix reading of commitcloud backedupheads files
Summary:
Process the backupstate file as UTF-8.  Previously it was being read as raw
bytes, causing the code to think it was an unknown version.

These files contain the version specifier (pure-ASCII), the remote path
(unicode), followed by a list of hexadecimal node IDs (pure-ASCII).  Therefore
reading the whole file as UTF-8 should be fine.

Reviewed By: farnz

Differential Revision: D19655547

fbshipit-source-id: 6ca536d445cbf81b19122916f1b70b8895304bdb
2020-01-30 18:29:51 -08:00
Adam Simpkins
2370602802 py3: re-enable demandimport
Summary:
D19539064 disabled demandimport when using Python 3 as a precaution while we
were initially working on Python 3 support.  However, it appears to work
correctly and does not seem to cause obvious problems when enabled.

This re-enables demand import by default.  It can be explicitly disabled at
runtime by specifying `HGDEMANDIMPORT=disable` in the environment.

Reviewed By: xavierd, farnz

Differential Revision: D19655548

fbshipit-source-id: d56814049181da2d68abd09ac7ca772b04e06618
2020-01-30 18:29:50 -08:00
Adam Simpkins
ebf7df9c50 fix the relpath() template to work in hg status
Summary:
When used in an `hg status` command the `relpath()` template function would
throw an unhandled `KeyError` when it tried to access `ctx` in the template
mapping.  This field is only present in `hg log` templates, and is not present
in `hg status` templates.

This updates the `relpath()` template function to look for `repo` in the
template mapping instead.  I also updated the `hg status` code to pass in the
`repo` value now.  The `hg log` code already passes in this field.

Reviewed By: quark-zju

Differential Revision: D19568463

fbshipit-source-id: 3e1c1d6571c73e4b24bfce22ea28f57679d39846
2020-01-30 18:26:36 -08:00
Xavier Deguillard
781d0e04ad test-check-code.t: make it run on python3
Summary:
The sources are opened in byte mode, therefore the regexp needs to be a byte
one.

Reviewed By: singhsrb

Differential Revision: D19655928

fbshipit-source-id: 79b54cc5016dc47aee4fe6cf053f28bf04ba630c
2020-01-30 18:12:30 -08:00
Xavier Deguillard
7c48ba6119 rebase: fix encoding issues
Summary:
Bunch of read/writes to files were using plain strings, use byte strings
instead.

Reviewed By: quark-zju

Differential Revision: D19655101

fbshipit-source-id: 0cb70cbc51630d2c83752846774ff1a1c5ec9f41
2020-01-30 18:12:29 -08:00
Xavier Deguillard
607b2db333 obsutil: decode marker metadata
Summary: These are strings, let's decode them.

Reviewed By: quark-zju

Differential Revision: D19655102

fbshipit-source-id: 18ca5556c76603a53bcb20b58fa8787e2fab70fb
2020-01-30 18:12:29 -08:00
Xavier Deguillard
4eb639c665 amend: write to the obsinhibit with bytes
Summary: This file contains bytes representation of nodes, let's add the b prefix.

Reviewed By: quark-zju

Differential Revision: D19655100

fbshipit-source-id: 2dc163038b17b0d3bce384d9b04f13b1449705fc
2020-01-30 18:12:28 -08:00
Xavier Deguillard
1ddfa14e5a tests: fix encoding issue in test-commit-multiple.t
Summary: The file is opened in binary mode, we need to write bytes into it.

Reviewed By: quark-zju

Differential Revision: D19655103

fbshipit-source-id: a2d059adff72db6c65e2933b3c64804325e51a71
2020-01-30 18:12:28 -08:00
Jun Wu
94c6853e8d edenscm: add a main module
Summary:
Make it possible to run edenscm via a "python package".

This makes it possible to run monkeytype [1] via debugpyhon:

    ./hg3 debugpython -- -m monkeytype run edenscm help log --pager=off

[1]: https://github.com/Instagram/MonkeyType

Reviewed By: farnz

Differential Revision: D19655850

fbshipit-source-id: 358f9dcf976141f798af2d57fb6392f2bca6a396
2020-01-30 18:09:14 -08:00
Mateusz Kwapich
040e01d65c filelog: file contents shouldn't be strings
Summary: on file copies instead of contents we were commiting b'contents'

Reviewed By: markbt

Differential Revision: D19656501

fbshipit-source-id: e1cb0bdac79da851494318ce31396a2aca6c5754
2020-01-30 18:05:39 -08:00
Durham Goode
8c211e5224 py3: fix stdout/stderr in dott tests
Summary:
dott tests rig up a custom stdout/stderr. In Python 3 we expect them to
be strings, so we need to use an appropriate io object.

Unfortunatly upstream already created a pycompat.stringio object which uses
bytes for both python 3 and 2, so I had to use a new name for now.

Reviewed By: quark-zju

Differential Revision: D19653469

fbshipit-source-id: a529df5a2ab179c105fa441c4503ed6606e77543
2020-01-30 18:00:34 -08:00
Simon Farnsworth
a579ce59f9 py3: add types to bundle2.py and make wireprotocol exclusively bytes
Reviewed By: xavierd

Differential Revision: D19652082

fbshipit-source-id: b6670845bbd06dc55439ad5223d47bda2ae014b1
2020-01-30 17:56:58 -08:00
Mark Thomas
8e673a1de0 py3: implement util.dirs in Rust
Summary:
The C implementation of util.dirs doesn't work with Python 3.  Instead of trying to port it, write an implementation in Rust that works in both Python 2 and Python 3.

The old implementation had a hack for flat dirstate to omit certain entries.  Remove this hack as we now only use flat dirstate in tests and small repos.  This simplifies the code in the fastpath case.

Reviewed By: quark-zju

Differential Revision: D19634699

fbshipit-source-id: cbb6562bc9a0209b13f67db3dfa5250800a9fcbe
2020-01-30 17:33:36 -08:00
Mark Thomas
914607cac7 cpython-ext: add PyPath for references to paths
Summary: `PyPath` is to `PyPathBuf` as `Path` is to `PathBuf` and `str` is to `String`.

Reviewed By: quark-zju

Differential Revision: D19647995

fbshipit-source-id: 841a5f6fea295bc72b00da028ae256ca38578504
2020-01-30 17:33:35 -08:00
Zeyi (Rice) Fan
a634526801 some fix for run-tests
Summary:
Fix some common errors when run `run-tests` in Python 3.

shlib would be using `str` for stdin/stdout instead of bytes as most of the code in Python 2 operates with that assumption. Getting it to `bytes` will take some work.

Reviewed By: DurhamG, quark-zju

Differential Revision: D19649475

fbshipit-source-id: 1f0190fce4df9a530246774fd17cdfc2d7974a65
2020-01-30 17:24:32 -08:00
Zeyi (Rice) Fan
92e9ae6eea convert print statements in tests
Summary: These are Python 2 style print statements not surfaced during the tests. Mostly because they are used in `hg dbsh` or invoked with Mercurial's Python interrupter. So I did a ripgrep to find these statements and convert them into functions.

Reviewed By: farnz

Differential Revision: D19612968

fbshipit-source-id: 1b07e9c789ab834be692ba00fd58c575bc4a17bf
2020-01-30 17:24:32 -08:00
Arun Kulshreshtha
dd34889d43 py3: make patchrmdir work with Python 3
Summary: With this change `hg rm` works even with extensions enabled. Since this is a Cython extension, I did not bother adding type annotations since IIUC Pyre cannot type check Cython files.

Reviewed By: quark-zju

Differential Revision: D19652178

fbshipit-source-id: 2e03a0f241f80732bdae3f472a1e0576684b1911
2020-01-30 17:22:30 -08:00
Saurabh Singh
45d5467eec memcommit: add type hints
Summary: This diff just adds the type hints for `memcommit` extension.

Reviewed By: quark-zju

Differential Revision: D19651074

fbshipit-source-id: 834a5980cded92641e4d368ce00d01a3dcaec330
2020-01-30 16:49:50 -08:00
Xavier Deguillard
5ecc59749e treemanifest: hashing requires bytes
Summary: We need to encode the key to be able to hash it.

Reviewed By: genevievehelsel

Differential Revision: D19651373

fbshipit-source-id: 214a3d127eb71a1c9cbf4ed1fec4f5a58f7f5195
2020-01-30 16:29:17 -08:00
Xavier Deguillard
6b328f5f7d remotefilelog: the filelog data is bytes
Reviewed By: DurhamG

Differential Revision: D19651372

fbshipit-source-id: 13bf86235b95ad84beb9fe45e7abc06baec67a47
2020-01-30 16:29:16 -08:00
Xavier Deguillard
4b2b0c6630 remotenames: fix python3 warning
Summary:
Importing directly from collection is deprecated, we should import from
collections.abc in Python3.

Reviewed By: DurhamG

Differential Revision: D19650852

fbshipit-source-id: b2e470e5fde3fe915944ae68a4274e47ed91071a
2020-01-30 16:29:16 -08:00
Xavier Deguillard
8c08a97290 fsmonitor: fix test_symlinks_t
Summary:
A previous change requires the default value to be a tuple with 4 values,
fsmonitor was only passing 1.

Reviewed By: kulshrax

Differential Revision: D19652000

fbshipit-source-id: c9ae04109c61173a1df27ce653622dc4db653977
2020-01-30 16:20:13 -08:00
Carolyn Busch
fca8aafffc py3 chgserver bytes <-> unicode
Summary:
Server reads & writes bytes. Convert between bytes and unicode where
neccessary in chgserver. Add some type annotations.

Reviewed By: xavierd

Differential Revision: D19645028

fbshipit-source-id: 1ad4bec25c398422112ee0ed1780ee9d704e6622
2020-01-30 15:44:42 -08:00
Durham Goode
419858ad5c py3: fix journal writing
Summary:
Writing to the file should happen in bytes, so let's make the
journalentry serializer/deserializer convert from strings to bytes for us.

Reviewed By: xavierd

Differential Revision: D19650468

fbshipit-source-id: 3c13aff032ace1a15611c2370fc7d6ed44a6df34
2020-01-30 15:22:25 -08:00
Durham Goode
c1672ad848 py3: fix treemanifest fetching
Summary:
bundle2 parameter values are bytes, so when we compare the treemanifest
category with the bundle2 part category we need to encode the bundle2 part one.

Also fix wirepack to read utf8 paths.

Reviewed By: xavierd

Differential Revision: D19650197

fbshipit-source-id: 67da3bd8244f7953ec608019844988b20ad1929f
2020-01-30 15:22:25 -08:00
Durham Goode
b567c16b60 py3: make mutation markers 'user' utf8
Summary: Username as utf8, so let's make mutationmarker treat them as such.

Reviewed By: xavierd

Differential Revision: D19649887

fbshipit-source-id: 3f8b2db434a57ee8ee3017de8d925c19a2002b20
2020-01-30 15:22:24 -08:00
Xavier Deguillard
c1652cad03 drawdag: the commit data should be bytes
Summary:
The commit context requires commit data to be bytes, but drawdag was using str
for it, breaking tests.

Reviewed By: DurhamG

Differential Revision: D19650219

fbshipit-source-id: 1409ce9a277c1c410ede8109989e07595378ff3e
2020-01-30 14:55:55 -08:00
Xavier Deguillard
cb64df7197 tests: @command expects str command names
Summary: This change all the tests to use strings as the command names.

Reviewed By: simpkins

Differential Revision: D19649479

fbshipit-source-id: 4b2cb9b3af73a7508ef359ee3c407d04a39c7893
2020-01-30 14:55:55 -08:00
Xavier Deguillard
96f5dc6652 localrepo: wwread returns bytes
Summary:
This is directly used in fctx.data() which expects bytes to be returned, encode
the return value of vfs.symlink.

Reviewed By: quark-zju

Differential Revision: D19647846

fbshipit-source-id: 966a5e79db6843c4bc6b802aeb90364e983cd834
2020-01-30 14:55:55 -08:00
Xavier Deguillard
b863a561a4 pytreestate: use PyPath where necessary
Summary:
A previous diff missed these PyPath. As a bonus, replace most of the PyObject
into proper Rust types.

Reviewed By: quark-zju

Differential Revision: D19646139

fbshipit-source-id: a3b55e5a802d8bbef86e9a9431f605bb4e284b32
2020-01-30 14:55:54 -08:00
Durham Goode
ca48fc2665 py3: fix resolving unknown binary nodes in changectx
Summary:
changectx takes an arbitrary identifier and tries to resolve it. If it
takes a binary node and the node is not known, the fallback looksup don't accept
bytes and throw an exception. Let's stop early and raise the RepoLookupError.

Reviewed By: quark-zju

Differential Revision: D19643834

fbshipit-source-id: cfee39e0487f908ebd4ff628b847e53dec13c66e
2020-01-30 14:39:41 -08:00