Summary:
This module previously used to handle deciding how a particular module should
be imported if it had multiple versions (e.g., pure Python or native).
However, as of D18819680 it was changed to always import the native C version.
Lets go ahead and remove it entirely now. Using `policy.importmod` simply
makes it harder for type checkers to figure out the actual module that will be
used.
The only functionality that `policy.importmod()` still provided was verifying
that the module contained a "version" field that looked like what was
expected. In practice these version numbers are not bumped often, so this
doesn't really seem to provide much value in checking that we imported the
correct version that we expected to be shipped with this release.
Reviewed By: xavierd
Differential Revision: D19958227
fbshipit-source-id: 05f1d027d0a41cf99c4aa93cb84a51e830305077
Summary:
Add *.pyi type stub files for most of the native C extensions.
This allows Pyre to type check functions that use these extensions.
These type annotations likely aren't complete, but contain enough information
to allow Pyre to pass cleanly on the existing type-checked locations in the
code using these modules.
Reviewed By: xavierd
Differential Revision: D19958220
fbshipit-source-id: 85dc39a16e595595a174a8e59e419c418d3531be
Summary:
This moves the build rules for the extensions in mercurial/cext into a TARGETS
file in this directory.
This will allow us to start writing `*.pyi` files that contain type
information for these modules, and store them alongside the corresponding `.c`
files. By having the build rules in the top-level `eden/scm` directory we
would have needed to keep the `.pyi` files for these modules directly in the
`eden/scm` directory instead, as the namespace for the `pyi` files is assumed
to be the basemodule plus their path relative to the TARGETS file.
Reviewed By: xavierd
Differential Revision: D19958222
fbshipit-source-id: fdc26ead16663036ffa2562a96eb1649f91cba81
Summary:
The last diff fixed this for fsmonitor. Let's skip these same paths for
non-fsmonitor.
Reviewed By: quark-zju
Differential Revision: D20014808
fbshipit-source-id: 02e3cd9aa29d9c024ba3e8e42a46e21a7c8dfc30
Summary:
Watchman may report invalid utf-8 filenames, even after they've been
deleted. Let's skip them, and print a warning.
Reviewed By: sfilipco
Differential Revision: D20012187
fbshipit-source-id: b13550918a8330ef3eb5c546105d1e054dcb7724
Summary:
This error handling can be extremely slow: calling `self.node()` can end up
triggering a linkrev scan of the changelog, which can take over 5 minutes.
If we did want to add this back in the future we would need some sort of API
on `filectx` to try and get the node ID only if it was cheap, and that would
fail fast if this is using remotefilelog and trying to get the node ID will
require scanning the changelog.
Note that KeyError can occur fairly regularly when invoked in long-lived
commands like `hg debugedenimporthelper`. If we are asked about data in a new
commit that was added since this repository was originally opened a KeyError
will be thrown here (in which case `debugedenimporthelper` will call
`repo.invalidate()` and then retry).
Reviewed By: quark-zju
Differential Revision: D20010279
fbshipit-source-id: 0e9b4c163cb9256de57daa91eed70a3736cb1075
Summary:
On Windows, there are *two* 8-bit encodings for each process.
* The ANSI code page is used for all `...A` system calls, and this is what
Mercurial uses internally. It can be overridden using the `--encoding`
command line option.
* The OEM code page is used when outputing to the console. Mercurial has no
concept of this, and instead renders to the console using the ANSI code page,
which results in mojibake like "Θ" instead of "é".
Add the concept of an `outputencoding`. If this differs from `encoding`, we
convert from the local encoding to the output encoding before writing to the
console.
On non-Windows platforms, this defaults to the same encoding as the local encoding,
so this is a no-op unless `--outputencoding` is manually specified.
On Windows, this defaults to the codepage given by `GetOEMCP`, causing output
to be converted to the OEM codepage before being printed.
For ordinary strings, the local encoded version is wrapped by `localstr` if the
encoding does not round-trip cleanly. This means the output encoding works
even if the character is not represented in the local encoding.
Unfortunately, the templater is not localstr-clean, which means strings can get
flattened down to the local encoding and the original code points are lost. In
this case we can only output characters which are in the intersection of the
encoding and the output encoding.
Most US English Windows systems use cp1252 for the ANSI code page and cp437 for
the OEM code page. These both contain many accented characters, so users with
accented characters in their names will now see them correctly rendered.
All of this only applies to Python 2.7. In Python 3, everything is Unicode,
the `--encoding` and `--outputencoding` options do nothing, and it just works.
Reviewed By: quark-zju, ikostia
Differential Revision: D19951381
fbshipit-source-id: d5cb8b5bfe2bc131b2e6c3b892137a48b2139ca9
Summary:
The Rust graph renderer expects the message to be a unicode string, so ensure
we convert it from the local encoding before passing it to Rust.
Reviewed By: quark-zju
Differential Revision: D19951383
fbshipit-source-id: 644862c63873079364cb9902bd1bb49de8aa1ab9
Summary: This should get rid of the extraneous uninitialized attribute errors related to `setUp` and abstract classes.
Reviewed By: simpkins
Differential Revision: D19964487
fbshipit-source-id: 52d5a6496e372d99d4398473f9ed7672228a76f5
Summary:
Update includes to the third-party xdiff.h file to use absolute includes
from the repository root. This allows many parts of our internal build
tooling to work better, including automatic dependency processing.
Reviewed By: xavierd
Differential Revision: D19958228
fbshipit-source-id: 341dd8c94f1138cf4a387b92e1817b2a286d6aa1
Summary:
Update the C files under edenscm/mercurial/cext to use absolute includes from
the repository root. Also update a few of the libraries in edenscm/mercurial
that the cext code depends on.
This makes these files easier to build with Buck in fbsource, and reduces the
number of places where we have to use deprecated Buck functionality to help
find these headers. This also allows autodeps to work with the build targets
for these rules.
Reviewed By: xavierd
Differential Revision: D19958221
fbshipit-source-id: e6e471583a795ba5773bae5f16ed582c9c5fd57e
Summary:
Remove `thirdparty/pyre2/__init__.py` from the `libhg` sources list.
We don't compile the `thirdparty/pyre2/_re2.cc` file in the fbcode build, so
importing the `__init__.py` module from this package just triggers an
ImportError when the code tries to use it. The code then always falls back to
using the version of pyre2 included from the `fb-re2` wheel.
Dropping the `__init__.py` module from our library should simply trigger an
ImportError earlier when we can't even find this file, and the code will still
fall back to using `fb-re2`.
Including this `__init__.py` file just causes issues for type checking, since
it causes us to try and type check this file even though its dependencies are
not present.
Reviewed By: xavierd
Differential Revision: D19958224
fbshipit-source-id: 34ea8806b6ee9377f17a9318c64c91ec242225df
Summary:
Some of the methods in eden_dirstate_map.py had comments that were close to
type annotations that were added a couple years ago. Update them to proper
type comments that can be recognized by Pyre and mypy.
Also remove the unused create_clone_of_internal_map() method.
Reviewed By: chadaustin, xavierd
Differential Revision: D19958225
fbshipit-source-id: b753c030acb15cf4f8d8c536614e657ee1bcba52
Summary:
Update the `eden_dirstate_map` class to store `dirstatetuple` objects instead
of plain tuples in its `_map` member variable. Without this the `filefoldmap`
code that is used on Windows fails, as it directly accesses `self._map` and
expects it to contain `dirstatetuple` objects.
Reviewed By: DurhamG, pkaush
Differential Revision: D19841881
fbshipit-source-id: ddb7523b598cfd8ec8719a8a74446cefcb411358
Summary:
Commit messages and extras can be unbounded in size. This can cause problems if users create commits with exceptionally large messages or extras. Mercurial will commit these to the changelog, increasing its size. On Mononoke, large commit messages may go over the cacheing threshold, resulting in poor performance for requests involving these commits as Mononoke will need to reload on every access.
Commit messages should not usually be that large. Mostly likely it will happen by accident, e.g. through use of `hg commit -l some-large-file`. Prevent this from happening by accident by adding configuration for soft limits when creating commits.
If a user really does need to create a commit with a very large message or extras, they can override using the config option.
Reviewed By: xavierd
Differential Revision: D19942522
fbshipit-source-id: 09b9fe1f470467237acc1b20286d2b1d2ab25613
Summary:
This parameter was originally removed in D12811551, but re-added in D12855935
due to the fact that at the time the `eden_dirstate.py` and `dirstate.py`
files were deployed in separate RPMs and could not be updated together
atomically. We now deploy these files together, so we can drop this extra
unnecessary argument.
Reviewed By: chadaustin
Differential Revision: D19913057
fbshipit-source-id: 0f0b4fde4b3124a8fc5bb568551b4e67de14d410
Summary:
Sometimes the treestate points to an unknown commit (ex. aborted transaction
might strip commits). While `debugrebuilddirstate -r HASH --hidden` is able to
fix it, it is too slow.
This diff adds treestate repair logic to the `doctor` command. It scans through
the treestate files, find a most recent `Root` entry with `p1` pointing to a
known commit.
This can be much faster than `debugrebuilddirstate` in some cases, because the
watchman clock might still be valid, and the NEED_CHECK file list might still
be small. In that case, `status` can still be fast.
Since treestate atomically updates all information needed for `status`
calculation (parents, need-check-files (or, "non-normal files"), watchman-clock
(only with fsmonitor), and stat for clean files). Reverting to a previous state
is still atomic. Correctness-wise, this is equivalent to aborting a "large"
transaction, and restoring treestate data to the state before the transaction.
It should be consistent, and the next `status` call won't mis-report files like
the dangerous `debugsetparents` command.
Reviewed By: DurhamG
Differential Revision: D19864422
fbshipit-source-id: d5d2f8b43a0c15ea2ac0e3c164edec7deeb8451f
Summary:
See the test change. Without this change repairing the changelog won't give the
user back a working repo.
Reviewed By: markbt
Differential Revision: D19864421
fbshipit-source-id: b84582c5302469828c8cfcb3db362ea82f2eea63
Summary:
Reuse utilities in the fixcorrupt extension to repair changelog.
This is better than fixcorrupt because `hg doctor` does not require a repo
object. Some messages are updated so they become more consistent with the
rest of `hg doctor`.
The main motivation is to get changelog fixed early, so other repair logic can
check if a commit hash is known by changelog or not.
Reviewed By: markbt
Differential Revision: D19864418
fbshipit-source-id: 6f95c6c6191d7db2a474a07a5278a857cf41d8e2
Summary:
Run 'edenfsctl doctor' on an edenfs repo. If there is no current repo, it might
be caused by edenfs daemon stopped running. So let's also run edenfsctl doctor
in that case.
Reviewed By: markbt
Differential Revision: D19864419
fbshipit-source-id: d2a49a126a040845b88b4883d214162326d08d8d
Summary:
These are a random batch of stack traces that show up in remotefilelog
tests.
Reviewed By: quark-zju
Differential Revision: D19772619
fbshipit-source-id: a8b9ce188cb7a5a2c9ccaeb62f6744f1c4083e38
Summary:
archive uses a formatter to produce it's metadat file. We need to use a
string io stream instead of bytes.
Reviewed By: quark-zju
Differential Revision: D19748163
fbshipit-source-id: fbd8c32066cfc4a234d9b51691717c7fce4c7c9a
Summary: They need to be bytes to match the nodes.
Reviewed By: xavierd
Differential Revision: D19746027
fbshipit-source-id: 41e9cc390f4aa97d8c8b378144e64100811665c2
Summary:
sshpeer._calltwowaystream used iter(fp, ""), where "" was the sentinel
indicating the fp was empty. Since fp was an iterator of bytes, the sentinel was
never hit and this code kept sending 0\n to each other until the pipes filled
and it hung.
Reviewed By: xavierd
Differential Revision: D19746028
fbshipit-source-id: 0daada0ae2356b5c99cc2c39c121cd88af8f750a
Summary:
Fixes test-cat.t by changing some formatter encoding choices. The
formatter is bit awkward here. hg cat needs to support outputting raw binary,
but also json and templated output.
For now I've set it up so json and templated output can't output non-utf8 data.
Reviewed By: quark-zju
Differential Revision: D19786542
fbshipit-source-id: 84060928103b396b23e3173b715aed996074fa3e
Summary:
treedirstate was using bytes for state and for file paths. This does
the appropriate conversions. Note, I don't use strings for all state in rust,
because it's a pain in the butt in some places. We're going to delete
treedirstate eventually anyone, so just I'm getting the test to pass for now.
Reviewed By: quark-zju
Differential Revision: D19715102
fbshipit-source-id: 4f9eac372bee7884d36aa19e3a3ed253392fa7dd
Summary:
Make it possible to limit the time range of mutation info being displayed by hg
debugmutation.
Reviewed By: DurhamG
Differential Revision: D19904000
fbshipit-source-id: 365f54fdd861661961bba1a0ea96fce772623a23
Summary:
When using the `--date` parameter to `hg update` or `hg revert`,
`cmdutil.finddate` returns the binary node for the target commit. Passing this
to `scmutil.revsingle` sometimes works, however it's unreliable, as if the
binary node happens to look vaguely like a revset, we will attempt to parse it,
and perhaps fail.
Resolve the ambiguity by converting the binary node to hex.
Reviewed By: quark-zju
Differential Revision: D19902595
fbshipit-source-id: 5eb7b9b029e292a02ccc00f5a465ab7807cd56d5
Summary:
This is needed by the next diff. Otherwise Python 3 ssh tests in the next diff
will hang.
Reviewed By: DurhamG
Differential Revision: D19879882
fbshipit-source-id: ecc317d0685993c6b1bef8c72068bf4315030d0f
Summary:
I'm going to change the connection pool logic but I'm not sure where it gets
used. This change exposes at least one test using it.
Reviewed By: xavierd
Differential Revision: D19872614
fbshipit-source-id: 4921b92c3fe3fd7ba1a72de17eef92604964eb2e
Summary:
Right now, if the client establishes connection to more than one peer, the
last one to proces `clienttelemetry` wireproto command gets the honor to set
`server_realhostname`. This is not desirable. Specifically, when we have
`fallbackpath` set up for remotefilelog/treemanifest and the prefetch happens
after pulling a commit, we get the hostname of the fallbackpath server,
while losing the hostname of the original `getbundle` server.
Reviewed By: DurhamG
Differential Revision: D19837570
fbshipit-source-id: fdc41565a5dfe670df3caf3b034196c4b7bdf6d9
Summary:
We never actually enabled it in production, and our new storage (ex. metalog,
indexedlog) are SIGKILL-safe. Only the changelog (revlog) might be subject to
double Ctrl+C issues. And we have debugfixcorrupt which can fix changelog
corruptions. So nointerrupt becomes less important.
The direct motivation is the test has been too flaky recently (and it's hard to
reproduce locally).
Differential Revision: D19830032
fbshipit-source-id: 87722d730fab49efbfd231772b1a5dddf8ec0d7b
Summary: Not very useful on today's hardware.
Reviewed By: quark-zju
Differential Revision: D19783741
fbshipit-source-id: d8ff29486cdc5f8e31d929d9fbba827e37012381
Summary:
We don't have confidence that python 2 is only using bytes (vs unicode
or other byte-like objects). These asserts break users, so let's disable them
except for in tests. We should uncover issues here as we port to python 3.
Reviewed By: xavierd
Differential Revision: D19819699
fbshipit-source-id: 9e3a9c34e7661fac4db9ee2a79f65be3d5e48cb3
Summary:
As part of the py3 migration we've started enforcing that encodeutf8 is
only called on bytes. graphql is returning unicode, so we need to change it to
return bytes on python 2.
Reviewed By: xavierd
Differential Revision: D19819701
fbshipit-source-id: 433ced2da87d522e4234ff453d11cacfaeebdd8d
Summary:
This commit manually synchronizes the internal move of
fbcode/scm/mononoke under fbcode/eden/mononoke which couldn't be
performed by ShipIt automatically.
Reviewed By: StanislavGlebik
Differential Revision: D19722832
fbshipit-source-id: 52fbc8bc42a8940b39872dfb8b00ce9c0f6b0800
Summary:
Attach the content to the exception object directly so if the exception is
handled, no verbose message will be printed.
Reviewed By: DurhamG
Differential Revision: D19817404
fbshipit-source-id: 9b79952559d07eb62bd7eb2398c6dd022ffe7b28
Summary:
We're seeing an error where worker processes fail to fetch data from
the server. Our theory is that the forking messes up the ssh connection, so
let's not reuse ssh connections across forks.
Reviewed By: xavierd
Differential Revision: D19794281
fbshipit-source-id: 3954cd96617fd18ecf0d0aa9b9a6ce774c494067
Summary: fixes printing for no match found in revlog match
Reviewed By: quark-zju
Differential Revision: D19652374
fbshipit-source-id: 55bfcbff7919ad817173241e2792df27311b931e
Summary:
Print out the name of the commit and the stack.
Hopefully this can help making debugging KeyErrors easier.
Reviewed By: DurhamG
Differential Revision: D19776181
fbshipit-source-id: 2eb985dd5355732a4d7728af68eb16173c48caa5
Summary:
Partially backport upstream
https://www.mercurial-scm.org/repo/hg/rev/f81c17ec303c to enable lazy loading
of python code contained in edenscmdeps3.zip.
Also, temporarily disabling the demandimport on Python3 is a bit tricky, for
the reasons mentioned in the deactivated function. Thus, instead of using the
disabled function, let's use the deactivated one.
Reviewed By: DurhamG
Differential Revision: D19672866
fbshipit-source-id: c9e39ed044121d962af1cc46745bdec72629c579
Summary:
Use `abc.ABC` from Python 3 stdlib directly. The definition matches
`pycompat3.py`:
class ABC(metaclass=ABCMeta):
pass
The following changes are reverted since they're no longer necessary:
D19732319 "[hg] py3: fix windows build"
D19703778 "[hg] py3: exclude mercurial/pycompat3.py from Python 2 builds"
D19703779 "[hg] py3: exclude pycompat3.py from Buck-based Python 2 builds"
Reviewed By: simpkins, singhsrb
Differential Revision: D19739075
fbshipit-source-id: 8c1e3727e8a88ff5f7232270d528d690523b1824
Summary:
`timed` has been replaced by the `timefunction` in util.
I have also removed the existing usages because they seemed relevant only
during the development.
Reviewed By: quark-zju
Differential Revision: D19727919
fbshipit-source-id: 58c466f02ac2a5bf7096948b17b49ceb26fc1fd9
Summary:
The path normalization OSX logic was using unicode, and when it
switched to use pycompat in D19613690 it stopped using unicode, which broke it.
Let's roll it back.
Reviewed By: singhsrb
Differential Revision: D19726322
fbshipit-source-id: fa7bfacdf41dfedbcfef2fdb5ccb826196101e61
Summary:
Add a few additional type annotations on some functions.
This did require some minor code restructuring in `filestore.setfile()` to
work around pyre's unwillingness to unwrap `Optional` member variables.
Reviewed By: quark-zju
Differential Revision: D19715243
fbshipit-source-id: 8078329c1e4a50ad0aa6c765d42a89b5ed58e7bf
Summary:
Add more type annotations to vfs.py and some of the platform-specific
functions that it calls.
This did catch a couple of cases where string/bytes conversion were not being
done properly in Python 3.
Reviewed By: quark-zju
Differential Revision: D19678717
fbshipit-source-id: b3979cc5e0668ddbb93372dee25ef02ea1867d6f
Summary:
Fix various type issues when running chg under Python 3.
Enable chg in setup3.py build.
This should make tests run faster. For example, test-rebase-detach.t
now completes in 8 seconds, down from 29 seconds.
Reviewed By: xavierd
Differential Revision: D19702535
fbshipit-source-id: 8928b1b920b9b52fd03dc86f996da18f2405f146
Summary:
Drop stdoutbytes/stdinbytes. They make things unnecessarily complicated
(especially for chg / Rust dispatch entry point).
The new idea is IO are using bytes. Text are written in utf-8 (Python 3) or
local encoding (Python 2). To make stdout behave reasonably on systems not
using utf-8 locale (ex. Windows), we might add a Rust binding to Rust's stdout,
which does the right thing:
- When writing to stdout console, expect text to be utf-8 encoded and do proper decoding.
- Wehn writing to stdout file, write the raw bytes without translation.
Note Python's `sys.stdout.buffer` does not do translation when writing to stdout console
like Rust's stdout.
For now, my main motivation of this change is to fix chg on Python 3.
Reviewed By: xavierd
Differential Revision: D19702533
fbshipit-source-id: 74704c83e1b200ff66fb3a2d23d97ff21c7239c8
Summary:
The "repo" type is a bit hard to type since it might be wrapped by extensions,
and the decorators like "unfilteredmethod" are too hard for type checkers.
Provide a stub for it so typecheck can work. The stub was generated by `pytype`,
I made a small change to `close` to make pyre happy.
Reviewed By: DurhamG
Differential Revision: D19677410
fbshipit-source-id: 71afb3fe78a75ee269c8005d18eee3a807b50df6
Summary:
Add type hints for `vfs.join()`. Not all subclasses supported joining
multiple arguments, so fix them all to actually implement the full API.
Reviewed By: farnz
Differential Revision: D19678712
fbshipit-source-id: f7f77eee416cd5edda756a8423cbe3630e195d4a
Summary: Matchers were mixing str and bytes - use decodeutf8 on the place using bytes, and enable all the tests that now pass.
Reviewed By: quark-zju
Differential Revision: D19679076
fbshipit-source-id: 2b90cfbf690b2365a1302efd9db72347a295c977
Summary: This gives type checker some hints. Pyre can now look into more areas.
Reviewed By: markbt
Differential Revision: D19676085
fbshipit-source-id: db05b4238051953eb7f8af44da220825ec589c3c
Summary:
This gives type checker some hints. Pyre was able to find some issues,
which are fixed.
Reviewed By: markbt
Differential Revision: D19674436
fbshipit-source-id: 44d6df6037f27e2edeff4d343a9fdec5e51420b3
Summary: This allows type checkers to understand the i18n._ calls.
Reviewed By: markbt
Differential Revision: D19673677
fbshipit-source-id: be2cc72f70e56925007485c786f0d72b5cd40a1a
Summary:
Begin adding type hints to many of the vfs methods.
These type hints indicate that the vfs objects are always opened in binary
mode. While it is possible to open a vfs in text mode if you explicitly
specify the argument `text=True`, this is very rare. If we run into cases
in the future that do need type hints to indicate TextIO mode I think it would
be better to split those out into completely separate functions, rather than
having functions that sometimes return bytes and sometimes return str based on
the arguments used.
Reviewed By: markbt
Differential Revision: D19673538
fbshipit-source-id: 8683223f28964d4b43fe131d4f16f8877dcbe777
Summary:
Make `atomictempfile` derive from `typing.BinaryIO` so that the type checker
understands it can be used like a file.
This required defining all `BinaryIO` methods on this class. This does result
in a fair amount of additional boilerplate code, but seems like the easiest
approach for now.
Reviewed By: markbt
Differential Revision: D19673539
fbshipit-source-id: b5775ca79b3af456b45e4ef5480711d7095e7949
Summary: The share command was reaching into the filesystem directly - use the right utility functions instead
Reviewed By: quark-zju
Differential Revision: D19672980
fbshipit-source-id: a14323fd5419c3ea00d9c009b9a77f63862a7b2a
Summary:
This moves `checkwinfilename()` from util.py to a new winutil.py module.
This allows the windows.py module to import winutil.py and re-expose
`checkwinfilename()` as `checkosfliename()`. This makes pyre happier, since
all platform modules now contain a `checkosfilename` method.
This also adds type hints to `checkwinfilename()`, and updates several parts
of the code that were still incorrectly assuming the input was bytes in
Python 3. However, note that I haven't done much testing of this on Windows.
Reviewed By: quark-zju
Differential Revision: D19672077
fbshipit-source-id: d78ed5b344a2e24c2a361b56465dedc516171d28
Summary:
While the payload of wire commands are bytes, we should treat the
command and args as utf8.
Reviewed By: quark-zju
Differential Revision: D19675217
fbshipit-source-id: df64c842e0c099d77dec1313aa0639e46e539194
Summary: hg serve works strictly with bytes, so let's expose it for stdin.
Reviewed By: quark-zju
Differential Revision: D19672836
fbshipit-source-id: 6abc5385daea81c584a89f19a568e42e4afad48a
Summary: This moves streaming clones a bit further along.
Reviewed By: quark-zju
Differential Revision: D19672546
fbshipit-source-id: 054dd639a8067dce34a5bf667996cc5d30301190
Summary:
We should generally be returning bytes on the wire. This fixes up a
number of places to do so, helping move streaming clone closer.
It also turns out that unescapearg is used in multiple situations. Usually it's
used for strings, but for batches the actual payloads are also delimited by this
and they are bytes. So let's have a function for each case.
Reviewed By: quark-zju
Differential Revision: D19672545
fbshipit-source-id: 6386ae2f39660b042af1382a1d696903fa1de9ad
Summary: A couple random stops that need correct encoding.
Reviewed By: quark-zju
Differential Revision: D19672547
fbshipit-source-id: edb0490f70cbf89885fe5fb0a1a8a2472d33e18d
Summary: Translate between bytes and unicode in sshserver to make test-sshserver pass.
Reviewed By: farnz
Differential Revision: D19672378
fbshipit-source-id: 7e1c0bc2d6fc03fac6ebcf54764862e3ed743f7e
Summary:
The changelog __contains__ expects rev numbers, not nodes, the nodemap should
be used instead.
Reviewed By: quark-zju
Differential Revision: D19674451
fbshipit-source-id: 8f0ddf3b9cebeea10f2fd639999f8c6c67a0e35d
Summary: There are no-ops. This makes type checkers' life slightly easier.
Reviewed By: farnz
Differential Revision: D19673051
fbshipit-source-id: 363eb8a4e799113b16529825d45645807cafe2fc
Summary: This is broken after D19672686.
Reviewed By: quark-zju
Differential Revision: D19674321
fbshipit-source-id: 7f225b7090ce9168e10e02b25ee17ae2b98cd86a
Summary:
We generally use 'utf-8' but are using 'utf8' at some places. This
diff changes to using 'utf-8' consistently everywhere.
Reviewed By: farnz
Differential Revision: D19674111
fbshipit-source-id: 3be42859a180ded1c234b805259f086f9531ba18
Summary:
The headers are bytes, therefore startswith needs to be called with a bytes
argument.
Reviewed By: quark-zju
Differential Revision: D19673675
fbshipit-source-id: 9a978ad42bf94a55c984f2ae35c566a59bfd6180
Summary:
Assigning to `__metaclass__` only works in Python 2. In Python 3 this
attribute is ignored, and you instead need to specify `metaclass=` in your
class signature.
This adds a new `pycompat.ABC` base class to make it easier to define abstract
classes that are compatible with both Python 2 and 3.
Reviewed By: sfilipco
Differential Revision: D19672078
fbshipit-source-id: c56eb6c7266f962ef9c13c742e6ac0651b9c19c9
Summary:
Add type annotations to `ui.write()` and the helper functions it calls.
I also removed unused `**opts` arguments on a few of the helper functions that
it calls.
Reviewed By: markbt
Differential Revision: D19657806
fbshipit-source-id: d020396a51d9fc3b07542c65541fca6067addb23
Summary: There are no-ops. This makes type checkers' life slightly easier.
Reviewed By: markbt
Differential Revision: D19673053
fbshipit-source-id: 04f26cd3ca0dbfe579ef939fc39be00035e70cee
Summary:
There are basically no-op (see D19613693).
This makes type checkers' life slightly easier.
Reviewed By: farnz
Differential Revision: D19672686
fbshipit-source-id: 27ca44dd53ec9788fdbb8f31235eae88d1853415
Summary: This makes type checkers' life easier.
Reviewed By: markbt
Differential Revision: D19672687
fbshipit-source-id: 335c560289e29ac7c08e36501e9841ddc51232be
Summary:
The issues were found by pyre with some type stubs generated via pytype:
python36 -m pytype edenscm --no-report-errors -j 30
I didn't include the pytype generated stubs because most of them are `Any`.
I'm trying to see if we can get something cleaner.
Reviewed By: markbt
Differential Revision: D19672435
fbshipit-source-id: c57f2ad3a981ddd4a3a267ff1c00e7bdb71e65ca
Summary: This diff enables more test while hopefully not breaking anything
Reviewed By: markbt
Differential Revision: D19671316
fbshipit-source-id: 76ecd22a094c0a40b9724a3070a01e5d294888f3
Summary:
The commit message should already be a utf8 string. Make sure that when it is
read from a file, we decode it properly.
Reviewed By: sfilipco
Differential Revision: D19666371
fbshipit-source-id: ee24fe33b5aaf1ede54286979c9523cedd901b3a
Summary:
Add a partial stub so pyre stops complaining about "bindings" being unknown.
Pyre's "search_path" was adjusted to read the new "pystubs" directory.
Reviewed By: DurhamG
Differential Revision: D19669131
fbshipit-source-id: a592ed411d1689058405689eb657e543b7172774
Summary: Patches are bytes, so the lines we add to them need to be bytes
Reviewed By: mitrandir77
Differential Revision: D19668850
fbshipit-source-id: cb521f71d927dbe1cbe3287be37123b609889846
Summary:
Fixes a few encoding issues for changegroup/bundle2 related issues. In
particular makes magic string comparison use strings.
Reviewed By: xavierd
Differential Revision: D19668613
fbshipit-source-id: c1bc8ab76d13bb04a9c440b5d9d9babfe34c5c92
Summary:
It decides whether to render the actual values by using a
isinstance(...,bytes) which isn't correct in python 3. Let's change it to str.
Reviewed By: xavierd
Differential Revision: D19668612
fbshipit-source-id: cf093754707a0a0369cfc3aa74270dd81a6560b5
Summary:
Now that bundle2 has strings everywhere, pushkey needs to also handle
encoding correctly. Previously it used local encoding, which seems strange.
Let's drop the encoding entirely since we're only dealing with since we're only dealing with utf8 string and bytes.
Reviewed By: farnz
Differential Revision: D19665894
fbshipit-source-id: ed054ab25636ac52de646329f26681ae843b6de3
Summary:
All diff functions are (bytes, bytes) -> bytes to preserver
the original file encoding.
Because of that I had to add ui.writebytes output function that accepts
bytes for terminal output.
Reviewed By: farnz
Differential Revision: D19656673
fbshipit-source-id: b9a1e4361e825fc8c2313e8402c2bbe00f490dd4
Summary:
The input/output of a subprocess is bytes, let's encode/decode when
writing/reading from it.
Reviewed By: farnz
Differential Revision: D19665636
fbshipit-source-id: 840f6b7a0edbedb7e7de5a65eacab9e1a7f78864
Summary: I've seen this while running some tests, they now go slightly further.
Reviewed By: singhsrb
Differential Revision: D19666665
fbshipit-source-id: 35ad73f383de148f3d53c7f9de24b1a1e1c51f7e
Summary:
The msg string isn't bytes in python 3. Hg revert works in both python 2&3
using 'str' instead.
Reviewed By: farnz
Differential Revision: D19664504
fbshipit-source-id: 5e16b637a5fb282eaa97390ef1f7e8912d0359f0
Summary:
We enforced this at send time, but it was hard to debug where it was
coming from. Let's enforce it at part.setdata() time so the stack points out
exactly who the problem is.
Reviewed By: farnz
Differential Revision: D19665895
fbshipit-source-id: 5a6ca0263d52cac472e1ef2aca3ee702f38258e4
Summary:
D19657765+D19656773 made bundle2 use the standard urllib which turns
bytes into strings. This messes up bundle2 caps encoding and decoding. Caps
encoding/decoding is a nested mess of strings and bytes, so let's make it all
strings until the last minute it goes over the wire.
Reviewed By: quark-zju
Differential Revision: D19665131
fbshipit-source-id: a2fd6e309ed6da2b927d2f4ead97825a16bfcaac
Summary:
fbhistedit adds support for json histedit commands. Unfortunately
json.loads produces unicode strings, which then fail assertions in
pycompat.encodeutf8(...). Let's make sure it produces non-unicode on python2.
Reviewed By: singhsrb
Differential Revision: D19656390
fbshipit-source-id: 11b828319b20a153aa008f03d554087b88b7763c
Summary:
Python 3 has changed which errors are OSError vs IOError. We have
slightly different formatting which makes the test awkward to be py2 and 3
compatible. Let's make the formats the same.
Reviewed By: quark-zju
Differential Revision: D19663964
fbshipit-source-id: 892461049b61d648431f300bea0da2f6f1c70066
Summary: This broke a bunch of tests. Revert it and re-enable all the tests.
Reviewed By: DurhamG
Differential Revision: D19665042
fbshipit-source-id: c3c17e3ac7e2ea028be5b5836bc8349cdf56184e
Summary:
D19652082 changed `urllibcompat.urlreq.quote()` to always return bytes.
This broke most of the code outside of the bundle2 module that was expecting
it to operate on strings.
This changes `quote()` to operate on strings again, since this is what the
Python 3 standard library does. This updates `bundle2` to handle encoding and
decoding properly when parsing strings that go through `quote()` and
`unquote()`.
This also changes bundle2 to stop going through the `urllibcompat.urlreq`
wrapper class, since that wrapper class prevents type checking from being
performed on its members.
Reviewed By: quark-zju
Differential Revision: D19657765
fbshipit-source-id: 0611838213ec22221e0b8367e1f0d8a6dcfceb15
Summary:
D19656501 added a usage of the `b"%b"` format specifier, which only works in
Python 3. Change the code to use `b"".join()` for now, which works in both
Python 2 and Python 3.
Reviewed By: farnz
Differential Revision: D19657472
fbshipit-source-id: 22d3123f4939fa984ffd17516e1cc48bad2bb2b7
Summary: The urlreq.quote returns a byte string, let's use b string to manipulate it.
Reviewed By: farnz
Differential Revision: D19656773
fbshipit-source-id: 1cbcc0f19e9463d5eabec2f67f5b00329188777d
Summary: These aren't bytes, they are strings.
Reviewed By: quark-zju
Differential Revision: D19656112
fbshipit-source-id: fcd505e3b4e522154358c80305e39ebca5936395
Summary:
When used in an `hg status` command the `relpath()` template function would
throw an unhandled `KeyError` when it tried to access `ctx` in the template
mapping. This field is only present in `hg log` templates, and is not present
in `hg status` templates.
This updates the `relpath()` template function to look for `repo` in the
template mapping instead. I also updated the `hg status` code to pass in the
`repo` value now. The `hg log` code already passes in this field.
Reviewed By: quark-zju
Differential Revision: D19568463
fbshipit-source-id: 3e1c1d6571c73e4b24bfce22ea28f57679d39846
Summary: on file copies instead of contents we were commiting b'contents'
Reviewed By: markbt
Differential Revision: D19656501
fbshipit-source-id: e1cb0bdac79da851494318ce31396a2aca6c5754
Summary:
dott tests rig up a custom stdout/stderr. In Python 3 we expect them to
be strings, so we need to use an appropriate io object.
Unfortunatly upstream already created a pycompat.stringio object which uses
bytes for both python 3 and 2, so I had to use a new name for now.
Reviewed By: quark-zju
Differential Revision: D19653469
fbshipit-source-id: a529df5a2ab179c105fa441c4503ed6606e77543
Summary:
The C implementation of util.dirs doesn't work with Python 3. Instead of trying to port it, write an implementation in Rust that works in both Python 2 and Python 3.
The old implementation had a hack for flat dirstate to omit certain entries. Remove this hack as we now only use flat dirstate in tests and small repos. This simplifies the code in the fastpath case.
Reviewed By: quark-zju
Differential Revision: D19634699
fbshipit-source-id: cbb6562bc9a0209b13f67db3dfa5250800a9fcbe
Summary:
Server reads & writes bytes. Convert between bytes and unicode where
neccessary in chgserver. Add some type annotations.
Reviewed By: xavierd
Differential Revision: D19645028
fbshipit-source-id: 1ad4bec25c398422112ee0ed1780ee9d704e6622
Summary:
The commit context requires commit data to be bytes, but drawdag was using str
for it, breaking tests.
Reviewed By: DurhamG
Differential Revision: D19650219
fbshipit-source-id: 1409ce9a277c1c410ede8109989e07595378ff3e
Summary:
This is directly used in fctx.data() which expects bytes to be returned, encode
the return value of vfs.symlink.
Reviewed By: quark-zju
Differential Revision: D19647846
fbshipit-source-id: 966a5e79db6843c4bc6b802aeb90364e983cd834
Summary:
A previous diff missed these PyPath. As a bonus, replace most of the PyObject
into proper Rust types.
Reviewed By: quark-zju
Differential Revision: D19646139
fbshipit-source-id: a3b55e5a802d8bbef86e9a9431f605bb4e284b32
Summary:
changectx takes an arbitrary identifier and tries to resolve it. If it
takes a binary node and the node is not known, the fallback looksup don't accept
bytes and throw an exception. Let's stop early and raise the RepoLookupError.
Reviewed By: quark-zju
Differential Revision: D19643834
fbshipit-source-id: cfee39e0487f908ebd4ff628b847e53dec13c66e
Summary:
In Python 3, subprocess creates FileIO type pipes, which do one syscall
and are not guaranteed to return 'size' bytes. In Python 2 they're 'file' types
which are guaranteed to return 'size' bytes (via multiple syscalls).
So let's handle that case.
Reviewed By: quark-zju, xavierd
Differential Revision: D19648035
fbshipit-source-id: 7d04a67acc569a55d3a855fc8521f1b8261f4636
Summary: Make the command server write and read bytes. Add type annotations.
Reviewed By: quark-zju
Differential Revision: D19642862
fbshipit-source-id: 2dfcd8f0c2e6124cc6afe1e85c0f5275e7fe5b1f
Summary:
It was iterating over what it thought was a string, but in python3 it's
a byte string, which results in iterating over integers. So known always claimed
the server knew every commit.
Reviewed By: quark-zju
Differential Revision: D19644837
fbshipit-source-id: 4e84a33b4d3e436006f93a5d8e1bdabf12e7d120
Summary:
Fix `threadedstderr` to stop directly deriving from `threading.Thread`.
Deriving directly from `threading.Thread` is somewhat tricky. Classes that
derive from `Thread` are only allowed to implement `__init__()` and `run()`.
This class also defined `close()`, which violates the `Thread` docs. Changing
this to no longer derive from `Thread` fixes exceptions being thrown when
trying to read from the SSH stderr pipe in this thread.
This also updates `_writessherror()` to decode the stderr data before writing
it to the `ui` object.
Reviewed By: quark-zju
Differential Revision: D19634265
fbshipit-source-id: bddd120c7995f49d3bbebbb171384d5b15b779e9
Summary: Moves pull closer to working by appropriately encoding and decoding.
Reviewed By: quark-zju
Differential Revision: D19643833
fbshipit-source-id: d311fb897d15594f2cb644b15038866ce506e011
Summary:
Revlog did a type check against str. In python 3 it accepts bytes
instead, so we need to support that.
Reviewed By: xavierd
Differential Revision: D19634882
fbshipit-source-id: 04ae908b4bfe8ed5de7a6d41cbe748dcb34b2540
Summary: Some strings had to stay as unicode
Reviewed By: quark-zju
Differential Revision: D19628661
fbshipit-source-id: 6ea191c60a4ef2ed98b2e2d3ab90c875e6eea248
Summary: because of the test `type(b"") != type("")` we actually have different behavior in Python 3 and Python 2. So we remove this test temporarily and we can add it back later.
Reviewed By: sfilipco
Differential Revision: D19611682
fbshipit-source-id: 7bca979f8ef871672ac9666286c8d5e54747e8b3
Summary:
In Python3 the result of filter is a class that does not have len
implemented. In Python2 the result of filter is a list.
Reviewed By: quark-zju
Differential Revision: D19629139
fbshipit-source-id: a7a1d9d831d72b2aab8b7a920bf82295c4673603
Summary:
The extra call to encode makes everything a big slower, fortunately python
standard library provides functions that do the right thing.
Reviewed By: xavierd
Differential Revision: D19634034
fbshipit-source-id: 6bb0de26300c41667824422c016e8d4f0aee70d3
Summary: The wireprotocol for streaming clones should be explicitly bytes, encoding filenames as UTF-8 when needed. Make it so
Reviewed By: quark-zju, xavierd
Differential Revision: D19616097
fbshipit-source-id: 3513c801035c8d7ba950777aa119e1d1480013a5
Summary:
Provide an option to run an external script that calculates the rename
destination. This is especially helpful for cases where development happened in
another branch and *directory* and want to merge changes back.
For example, we recently had an experimental `scm3` branch that copies files
from `scm` to `scm3` to try out changes:
o master <-- rebase destination
|
o (changes in scm/)
:
| o (changes that need to backport to scm/) <-- rebase source
| |
| o (changes in scm3/ that got implemented differently in scm/)
| :
| o (copy scm/ to scm3/ for experiments)
|/
o
This allows `--config experimental.rename-cmd='sed "s#scm3/#scm/#"'` to be
used for the above case.
While working on this feature, I noticed a bug that if the "copy" information
is recorded, rebase might lose changes. I added a test to track that bug.
Reviewed By: mitrandir77
Differential Revision: D19609868
fbshipit-source-id: 94462889d3ce9214b48fac793852d16aeacecc77
Summary:
This removes the dependency of `future` (used by `import past`) which is not
available in our Windows tests.
Reviewed By: farnz
Differential Revision: D19626387
fbshipit-source-id: 2b13f08a9dafe77d1e7f6fc1a3cc48b3581681ca
Summary: Let's have a module that defines types ussed commonly in HG
Reviewed By: quark-zju
Differential Revision: D19625872
fbshipit-source-id: 817535d87acee6cb5647bc8ad25df21a95bec09c
Summary:
There are times when status updates the dirstate to mark files as
clean or to resolve lookup files. In doing so it takes the wlock, which triggers
a state-enter event which causes external watchers to then run hg status, which
can then trigger another wlock event from status. This causes an infinite loop.
It's not clear why status is taking the wlock every time. It should be a
one-time thing. So more investigation is needed. For now though, we can prevent
this from happening by disabling hgevent signals while status has the wlock.
Reviewed By: quark-zju
Differential Revision: D19602370
fbshipit-source-id: 609316de1441d40be4f81236b8d00ce7dc30a130
Summary:
The pure implementation has many encoding issues. Since our code does not build
in the pure Python mode, just drop the pure lazymanifest implementation for
simplicity.
Reviewed By: farnz
Differential Revision: D19613691
fbshipit-source-id: f3f95717bc77a75d4aa7b597a33f8eec745df134
Summary: The template language is in unicode.
Reviewed By: markbt
Differential Revision: D19615823
fbshipit-source-id: 30c6b295b4e4c999480c46aba8b4e8ffed5a4d64
Summary: The "function name" is unicode in Python 3. Put unicode as the dict keys.
Reviewed By: farnz
Differential Revision: D19615814
fbshipit-source-id: 3508f59d11e4f8ab6345c80c6bdfaff150a63d76
Summary: Some assertions were added to make sure paths are unicode.
Reviewed By: farnz
Differential Revision: D19615813
fbshipit-source-id: 7d2135c0ac424fc46edd9d660b072a2a95be7f82
Summary:
Changegroup should be using bytes mostly except when it tries to access
files via `repo.file(path)`. The `path` should be unicode on Python 3.
Make it happen. Some assertions are added to help capture issues.
Reviewed By: farnz
Differential Revision: D19615835
fbshipit-source-id: b35d13152ecb835092c7870e88383f484dc5f868
Summary:
The bundle2 layer uses bytes since it's part of the wireprotocol.
Some assertions were added to help detecting issues.
Reviewed By: farnz
Differential Revision: D19615832
fbshipit-source-id: 73ea657b501c7539b17862165c7a5781884013a2
Summary: The layer interacting with raw manifest text should use bytes.
Reviewed By: farnz
Differential Revision: D19613694
fbshipit-source-id: c5a12235eb50acd90e409c1059a3f9f11665f9f3
Summary: Use native unicode hostname without converting it to bytes.
Reviewed By: farnz
Differential Revision: D19613692
fbshipit-source-id: 0bd7e37890b96566143fb0501c3fa310270844d3
Summary:
Like revlog, the changelog layer should use bytes for parsing its raw content.
`str` is still used for users, messages, etc.
Reviewed By: markbt
Differential Revision: D19613695
fbshipit-source-id: 4f6758f9fca0788bc89ad5c14d9103f66d19b7f2
Summary:
Bookmarks use unicode by default. Only need to use bytes when interacting with
vfs.
Reviewed By: farnz
Differential Revision: D19613689
fbshipit-source-id: bbe80e552cf17a023e5905f849a273b4b4299dc5
Summary: The patchtext is about a binary patch. It should use bytes.
Reviewed By: farnz
Differential Revision: D19613688
fbshipit-source-id: 20bddeb17f520f625396b23b5dcead7c7bbd3ede
Summary:
Make sure file content and wireproto buffers are using bytes.
Added a utf8 helper function.
Reviewed By: farnz
Differential Revision: D19613686
fbshipit-source-id: 5d67ce48246dc8c3cc47041f461e454c46bc67f9
Summary:
We want to assume almost all paths are unicode and drop support for non-unicode
paths. `os.fsencode` returns bytes that breaks the assumption.
Change fsencode and fsdecode to do nothing so path types remain unicode.
Most use-cases are related to `__file__`, which is already unicode on Python 3.
`posix._checklink` uses it. But it seems making it no-op is okay.
Reviewed By: farnz
Differential Revision: D19613693
fbshipit-source-id: 587dbeef68b23beeaf6bc8861c71ea27b3c512d7
Summary: Those will simplify some use-cases.
Reviewed By: farnz
Differential Revision: D19613121
fbshipit-source-id: 85ac4fa0c86afb84e0c9fb78950d3da2c93aa967
Summary: The raw blobs are in bytes, while paths and other things are in native string.
Reviewed By: markbt
Differential Revision: D19613122
fbshipit-source-id: e865168b5bc839e85ef843d771d8a6044e35a03a
Summary:
The commit message should be in native string. The raw file content is bytes.
Add translation for that.
Reviewed By: xavierd
Differential Revision: D19613117
fbshipit-source-id: 1b4145f0de75e48df3416cd7c52c09df1e6c4a07
Summary: Like revlog, the filelog layer should use bytes.
Reviewed By: sfilipco
Differential Revision: D19613116
fbshipit-source-id: 6b07d05a6960c2d1c8b4ebe3001632fd8024c5af
Summary: The revlog layer should use bytes.
Reviewed By: sfilipco
Differential Revision: D19613120
fbshipit-source-id: 09f0028cbf810b52694643c79c91997d434cdaf5
Summary:
On Python 2 we want bytes.encode("utf-8") to be no-op, while on Python 3 we
want it to return a real `str`. This helper function does that.
Reviewed By: sfilipco
Differential Revision: D19613118
fbshipit-source-id: 1f0fc31bf8aee28cc20694da52b10b12dbdd9000
Summary: In both py2 and py3 stdout is text mode.
Reviewed By: simpkins
Differential Revision: D19613397
fbshipit-source-id: fe01363836cb5956c7e6d83a0f81488649c262a9
Summary:
We want all config values to always be `str` types, so remove this explicit
`encode()` call that was trying to convert the value to `bytes`.
Reviewed By: sfilipco
Differential Revision: D19610590
fbshipit-source-id: a72f0755cdb12cae13fb9d58f529433d508d46e9
Summary:
Several of the debug commands explicitly defined their command names as bytes.
Remove the leading `b` prefix so that all command names are uniformly `str`
data types.
Reviewed By: DurhamG, sfilipco
Differential Revision: D19596738
fbshipit-source-id: 3851a32c38009968dfa338877b9ae03015dfca61
Summary: Use pycompat unicode instead of python 2 built in unicode to check type.
Reviewed By: mitrandir77
Differential Revision: D19589969
fbshipit-source-id: 3e3cf2c5cb6fd90cca9c8f803e7bd45f262f96e5
Summary: The input of minirst will be unicode, therefore, remove the b prefix.
Reviewed By: sfilipco
Differential Revision: D19610028
fbshipit-source-id: d3b1ebfeb1ba5a30afe250ce5713857dd9ebc359
Summary: The fallback filename handling can be utf8 under py3
Reviewed By: sfilipco
Differential Revision: D19610624
fbshipit-source-id: a86ed958edca70597ff5a7c67cbc746c1e325518
