Commit Graph

17876 Commits

Author SHA1 Message Date
Augie Fackler
585f46ae2f util: use pycompat.bytestr in checkwinfilename
Fixes `hg add` on python3.
2017-03-19 00:16:08 -04:00
Augie Fackler
4eeee9b0e9 dispatch: ensure repr is bytes in _mayberepr
Fixes command line arguments containing spaces on Python 3.
2017-03-19 00:22:04 -04:00
Augie Fackler
ecf3381175 dispatch: extract maybe-use-repr formatting to helper function
I think this makes the code much clearer. I had to think for a bit to
unpack the old-school `condition and if-true or if-false` dance, and
formatting argument lists here shouldn't be performance critical.
2017-03-19 00:21:26 -04:00
Augie Fackler
956c1da8ce dispatch: consolidate formatting of arguments
This was getting done twice, and it's clever enough I'm about to split
it apart and then fix it for Python 3.
2017-03-19 00:18:53 -04:00
Pulkit Goyal
03903626e4 py3: make the regular expression bytes to prevent TypeError 2017-03-17 05:10:58 +05:30
Yuya Nishihara
7701e518b5 pager: flush outputs before firing pager process
So that buffered outputs are always sent to the console.
2017-02-25 17:29:30 +09:00
Yuya Nishihara
e780f7677e httpconnection: make sure to clear progress of httpsendfile at EOF
read() should never raise EOFError. If it did, UnboundLocalError would occur
due to unassigned 'ret' variable.

This issue was originally reported to TortoiseHg:
https://bitbucket.org/tortoisehg/thg/issues/4707/
2017-03-18 16:02:14 +09:00
Yuya Nishihara
7e7dd65819 py3: convert log opts to bytes-key dict
Now simple log command works.
2017-03-14 18:23:59 +09:00
Yuya Nishihara
2b0fe29f27 graphlog: pass function arguments without expansion
It's annoying on Python 3 because keys must be unicode type. Let's stop using
**opts expansion when not necessary.
2017-03-14 18:16:13 +09:00
Yuya Nishihara
687bfe4011 py3: call codecs.escape_decode() directly
The same rule as 9ecf9a0c9837 applies.
2017-03-17 23:48:22 +09:00
Yuya Nishihara
dc88179a4e util: wrap s.decode('string_escape') calls for future py3 compatibility 2017-03-17 23:42:46 +09:00
Augie Fackler
b35d966e25 merge with stable 2017-03-18 12:27:52 -04:00
Gregory Szorc
aff9286eb0 exchange: use v2 bundles for modern compression engines (issue5506)
Previously, `hg bundle zstd` on a non-generaldelta repo would
attempt to use a v1 bundle. This would fail because zstd is not
supported on v1 bundles.

This patch changes the behavior to automatically use a v2 bundle
when the user explicitly requests a bundlespec that is a compression
engine not supported on v1. If the bundlespec is <engine>-v1, it is
still explicitly rejected because that request cannot be fulfilled.
2017-03-16 12:33:15 -07:00
Gregory Szorc
3bbb6e2c52 exchange: reject new compression engines for v1 bundles (issue5506)
Version 1 bundles only support a fixed set of compression engines.
Before this change, we would accept any compression engine for v1
bundles, even those that may not work on v1. This could lead to
an error.

We define a fixed set of compression engines known to work with v1
bundles and we add checking to ensure a newer engine (like zstd)
won't work with v1 bundles.

I also took the liberty of adding test coverage for unknown compression
names because I noticed we didn't have coverage of it before.
2017-03-16 12:23:56 -07:00
Augie Fackler
5811358454 pycompat: verify sys.argv exists before forwarding it (issue5493)
ISAPI_WSGI doesn't set up sys.argv, so we have to look for the
attribute before assuming it exists.
2017-03-07 13:24:24 -05:00
Matt Harbison
27ca0a8a5b hgwebdir: add support for explicit index files
This is useful for when repositories are nested in --web-conf, and in the future
with hosted subrepositories.  The previous behavior was only to render an index
at each virtual directory.  There is now an explicit 'index' child for each
virtual directory.  The name was suggested by Yuya, for consistency with the
other method names.

Additionally, there is now an explicit 'index' child for every repository
directory with a nested repository somewhere below it.  This seems more
consistent with each virtual directory hosting an index, and more discoverable
than to only have an index for a directory that directly hosts a nested
repository.  I couldn't figure out how to close the loop and provide one in each
directory without a deeper nested repository, without blocking a committed
'index' file.  Keeping that seems better than rendering an empty index.
2017-03-05 22:22:32 -05:00
Jun Wu
3d8879dca9 ui: move configlist parser to config.py
The list parser is complex and reusable without ui. Let's move it to
config.py.

This allows us to parse a list from a "pure" config object without going
through ui. Like, we can make "_trustusers" calculated from raw configs,
instead of making sure it's synchronized by calling "fixconfig"s.
2017-03-17 09:19:56 -07:00
Augie Fackler
3839a3e429 pager: skip running the pager if it's set to 'cat'
Avoid useless uses of cat.
2017-03-15 20:34:26 -04:00
Augie Fackler
dd0af0f250 pager: avoid shell=True on subprocess.Popen for better errors (issue5491)
man(1) behaves as poorly as Mercurial without this change. This cribs
from git's run-command[0], which has a list of characters that imply a
string that needs to be run using 'sh -c'. If none of those characters
are present in the command string, we can use shell=False mode on
subprocess and get significantly better error messages (see the test)
when the pager process is invalid. With a complicated pager command
(that contains one of the unsafe characters), we behave as we do today
(which is no worse than git manages.)

I briefly tried tapdancing in a thread to catch early pager exits, but
it's just too perilous: you get races between fd duping operations and
a bad pager exiting, and it's too hard to differentiate between a
slow-bad-pager result and a fast-human-quit-pager-early result.

I've observed some weird variation in exit code handling in the "bad
experience" case in test-pager.t: on my Mac hg predictably exits
nonzero, but on Linux hg always exits zero in that case. For now,
we'll work around it with || true. :(

0: cddbda4bc8/run-command.c (L201)
2017-03-15 20:33:47 -04:00
Rishabh Madan
ea88450bc5 py3: change explicit conversion of config value from str to pycompat.bytestr 2017-03-17 19:12:22 +05:30
Gregory Szorc
5ca0f908bf py3: add __bool__ to every class defining __nonzero__
__nonzero__ was renamed to __bool__ in Python 3. This patch simply
aliases __bool__ to __nonzero__ for every class implementing
__nonzero__.
2017-03-13 12:40:14 -07:00
Martin von Zweigbergk
31bb49fb7d merge: also allow 'e' action with experimental.updatecheck=noconflict
With experimental.updatecheck=noconflict set, if one checks out
e1870a31325e (tests: add execute bit and fix shbang line, 2015-12-22)
and then try to check out its parent, hg will complain about
conflicting changes, even though the working directory is clean. We
need to also allow the 'e' action in merge.py. The 'e' action is used
when moving to a commit where the only change to the file is to its
executable flag, so it's just an optimized 'g' action.

Doesn't seem to be worth writing a test for, since the existing setup
in test-update-branches.t does not set any flags.
2017-03-13 21:58:43 -07:00
Martijn Pieters
0344aed188 config: honour the trusted flag in ui.configbytes 2017-03-12 11:43:31 -07:00
Jun Wu
0c6a9d19a1 osutil: fix potential wrong fd close
According to POSIX closedir [1]:

  If a file descriptor is used to implement type DIR, that file descriptor
  shall be closed.

According to POSIX fdopendir [2]:

  Upon calling closedir() the file descriptor shall be closed.

So we should avoid "close(dfd)" after "closedir(dir)". With threads, there
could be a race where an innocent fd gets closed. But Python GIL seems to
help hiding the issue well.

[1]: http://pubs.opengroup.org/onlinepubs/009695399/functions/closedir.html
[2]: http://pubs.opengroup.org/onlinepubs/9699919799/functions/fdopendir.html
2017-03-15 20:43:12 -07:00
Gregory Szorc
da814fb671 parsers: use Python memory allocator for indexObject->offsets 2017-03-09 12:09:31 -08:00
Gregory Szorc
becdf838b6 parsers: use Python memory allocator in commonancestorsheads() 2017-03-09 12:02:59 -08:00
Gregory Szorc
5deb9bd0a7 osutil: use Python memory allocator in _listdir
The Python memory allocator has performance advantages
for small allocations.
2017-03-09 11:56:47 -08:00
Gregory Szorc
68f48d2e6f bdiff: use Python memory allocator in fixws
Python has its own memory allocation APIs. For allocations
<= 512 bytes, it allocates memory from arenas. This means that
average small allocations don't call the system allocator, which
makes them faster. Also, arena allocations cut down on memory
fragmentation, which can matter for performance in long-running
processes.

Another advantage of using the Python memory allocator is that
allocations are tracked by Python. This is a bigger deal in
Python 3, as modern versions of Python have some decent built-in
tools for examining memory usage, leaks, etc.

This patch converts a trivial malloc() + free() in the bdiff code
to use the Python allocator APIs. Since the object being
operated on is a line, chances are it will use an arena. So,
this could have a net positive impact on performance (although
I didn't measure it).
2017-03-09 11:54:25 -08:00
Pierre-Yves David
3248bb0252 localrepo: fix deprecation warning version of wfile
The patch lingered a bit too long in my local clone and I messed up when I
updated the version number. Since nobody caught it, I'm fixing the version after
the fact.
2017-03-16 11:17:55 -07:00
Pierre-Yves David
fc2b521909 util: explicitly tests for None
Changeset 3b9cdb72931f removed the mutable default value, but did not explicitly
tested for None. Such implicit checking can introduce semantic and performance
issue. We move to an explicit check for None as recommended by PEP8:

https://www.python.org/dev/peps/pep-0008/#programming-recommendations
2017-03-15 15:07:14 -07:00
Pierre-Yves David
8561a8e8ff context: simplify call to icase matcher in 'match()'
The two function takes the very same arguments. We make this clearer and less
error prone by dispatching on the function only and having a single call point
in the code.
2017-03-15 15:38:02 -07:00
Pulkit Goyal
73e25e061d py3: make sure using bytes status char rather than ascii values
'MAR!?IC' is converted to their ascii values when slicing through it. This
patch uses pycompat.iterbytestr() to get bytes value.
2017-03-16 09:13:13 +05:30
Jun Wu
e7a5a37c7c import: get rid of ui.backupconfig 2017-03-16 14:23:49 -07:00
Jun Wu
e7394336ca clone: get rid of ui.backupconfig 2017-03-16 14:18:50 -07:00
Jun Wu
a1b35a50eb commit: get rid of ui.backupconfig 2017-03-16 14:15:20 -07:00
Durham Goode
2e0dd44c19 branchmap: handle nullrev in setcachedata
906be86990 recently changed to switch from:

  self._rbcrevs[rbcrevidx:rbcrevidx + _rbcrecsize] = rec

to

  pack_into(_rbcrecfmt, self._rbcrevs, rbcrevidx, node, branchidx)

This causes an exception if rbcrevidx is -1 (i.e. the nullrev). The old code
handled this because python handles out of bound sets to arrays gracefully. The
new code throws because the self._rbcrevs buffer isn't long enough to write 8
bytes to.  Normally it would've been resized by the immediately preceding line,
but because the 0 length buffer is greater than the idx (-1) times the size, no
resize happens.

Setting the branch for the nullrev doesn't make sense anyway, so let's skip it.
This was caught by external tests in the Facebook extensions repo, but I've
added a test here that catches the issue.
2017-03-15 15:48:57 -07:00
Yuya Nishihara
4744dd2998 py3: call codecs.escape_encode() directly
string_escape doesn't exist on Python 3, but fortunately the undocumented
codecs.escape_encode() function exists on CPython 2.6, 2.7, 3.5 and PyPy 5.6.
So let's use it for now.

http://stackoverflow.com/a/23151714
2017-03-15 23:28:39 +09:00
Yuya Nishihara
5ce2afb81d templatekw: make join() escape values of extras (BC) (issue5504)
Since extras may contain blob, the default template escapes its values:

  'extra': '{key}={value|stringescape}'

join() should follow the output style of the default template.
2017-03-15 23:21:30 +09:00
Yuya Nishihara
02022fc3c5 util: wrap s.encode('string_escape') call for future py3 compatibility 2017-03-15 23:06:50 +09:00
Yuya Nishihara
99a868d61d py3: call strftime() with native str type
Since strftime() may contain non-ascii character if locale set, we use
strfrom/tolocal().

Now "hg tip" works.
2017-03-13 09:19:07 -07:00
Yuya Nishihara
af7f25fdb3 encoding: add converter between native str and byte string
This kind of encoding conversion is unavoidable on Python 3.
2017-03-13 09:12:56 -07:00
Yuya Nishihara
dcade16cf7 encoding: factor out unicode variants of from/tolocal()
Unfortunately, these functions will be commonly used on Python 3.
2017-03-13 09:11:08 -07:00
Yuya Nishihara
ec012a1a69 py3: use next() to obtain next item from inner generator of generatorset
.next attribute does not exist on Python 3. As this function seems to really
care about the overhead of the Python interpreter, I follow the way of micro
optimization.
2017-03-13 08:53:31 -07:00
Yuya Nishihara
02b9a6d074 py3: rewrite itervalues() as values() by importer
I'm not a great fan of these importer magics, but this should be okay since
"itervalues" seems as unique name as "iteritems".
2017-03-13 08:44:57 -07:00
Yuya Nishihara
fad2547154 py3: use portable way to stringify cache key of repoview 2017-03-12 17:20:42 -07:00
Kostia Balytskyi
688382aced exewrapper: prefer HackableMercurial python if availbale
Currently hg.exe will only try to load python27.dll from hg-python
subdir if PYTHONHOME environment variable is not set. I think that
it is better to check whether 'hg-python' subdir exists and load
python from it in that case, regardless of environment. This allows
for reliable approach of distributing Mercurial with its own Python.
2017-03-13 12:44:13 -07:00
Yuya Nishihara
eea0ef4b6e py3: use bytestr wrapper in revsetlang.tokenize()
This backs out a2d8ce9531e1 and wraps program by bytestr() instead.
2017-03-16 21:36:21 +09:00
Yuya Nishihara
a665b1ac40 py3: use bytestr wrapper in revsetlang.formatspec()
This backs out e6d1d689544f and wraps expr by bytestr() instead.
2017-03-16 21:33:25 +09:00
Yuya Nishihara
791afb08eb pycompat: add bytestr wrapper which mostly acts as a Python 2 str
This allows us to handle bytes in mostly the same manner as Python 2 str,
so we can get rid of ugly s[i:i + 1] hacks:

  s = bytestr(s)
  while i < len(s):
      c = s[i]
      ...

This is the simpler version of the previous RFC patch which tried to preserve
the bytestr type if possible. New version simply drops the bytestr wrapping
so we aren't likely to pass a bytestr to a function that expects Python 3
bytes.
2017-03-08 22:48:26 +09:00
Pierre-Yves David
b9d210d680 context: explicitly tests for None
Changeset c832083e5671 removed the mutable default value, but did not explicitly
tested for None. Such implicit testing can introduce semantic and performance
issue. We move to an explicit testing for None as recommended by PEP8:

https://www.python.org/dev/peps/pep-0008/#programming-recommendations
2017-03-15 15:33:24 -07:00