Commit Graph

968 Commits

Author SHA1 Message Date
Augie Fackler
6278186d0b util: pass sysstrs to warnings.filterwarnings
Un-breaks the Python 3 build.
2017-04-13 13:12:49 -04:00
Pierre-Yves David
a185960897 util: add a way to issue deprecation warning without a UI object
Our current deprecation warning mechanism relies on ui object. They are case
where we cannot have access to the UI object. On a general basis we avoid using
the python mechanism for deprecation warning because up to Python 2.6 it is
exposing warning to unsuspecting user who cannot do anything to deal with them.

So we build a "safe" strategy to hide this warnings behind a flag in an
environment variable. The test runner set this flag so that tests show these
warning.  This will help us marker API as deprecated for extensions to update
their code.
2017-04-04 11:03:29 +02:00
Gábor Stefanik
387861cc38 util: fix human-readable printing of negative byte counts
Apply the same human-readable printing rules to negative byte counts as to
positive ones. Fixes output of debugupgraderepo.
2017-04-10 18:16:30 +02:00
Gregory Szorc
3c5a0a039c util: make cookielib module available
In preparation for supporting sending cookies on HTTP requests.
2017-03-09 21:35:21 -08:00
Yuya Nishihara
af3a1d8aae sortdict: fix .pop() to return a value
My future patch will need it.
2017-04-09 11:57:09 +09:00
Pulkit Goyal
04352e4321 py3: replace str() with bytes() 2017-04-07 13:46:35 +05:30
Augie Fackler
f0863383db util: fix %-formatting on docstring by moving a closing parenthesis
We have to do the % formatting over the sysstr, since the things we're
going to splat into it are themselves sysstrs. This is probably
technically wrong-ish, since bt is probably actually a bytestr here,
but this fixes the immediate issue, which was that hg was broken on
Python 3.
2017-04-03 19:03:34 -04:00
Gregory Szorc
252949c70b util: document bundle compression
An upcoming patch will add support for documenting bundle
specifications in more detail. As part of this, we'd like to
enumerate available bundle compression formats. In order to do
this, we need to provide the help mechanism a dict of names
and objects with docstrings.

This patch adds docstrings to compengine.bundletype and adds
a function for retrieving a dict of them. The code is not yet
used.
2017-04-01 13:29:01 -07:00
Yuya Nishihara
40564e2c98 util: add helper to convert between LF and native EOL
See the next patch for why.
2017-03-29 21:40:15 +09:00
Yuya Nishihara
1163409660 util: extract pure tolf/tocrlf() functions from eol extension
This can be used for EOL conversion of text files.
2017-03-29 21:28:54 +09:00
Jun Wu
47e81ec208 hardlink: check directory's st_dev when copying files
Previously, when copying a file, copyfiles will compare src's st_dev with
dirname(dst)'s st_dev, to decide whether to enable hardlink or not.

That could have issues on Linux's overlayfs, where stating directories could
result in different st_dev from st_dev of stating files, even if both the
directories and the files exist in the overlay's upperdir.

This patch fixes it by checking dirname(src) instead. It's more consistent
because we are checking directories for both src and dest.

That fixes test-hardlinks.t running on common Docker setups.
2017-03-29 12:37:03 -07:00
Jun Wu
7621302bee hardlink: duplicate hardlink detection for copying files and directories
A later patch will change one of them so they diverge.
2017-03-29 12:26:46 -07:00
Jun Wu
29358be572 hardlink: extract topic text logic of copyfiles
The topic text shows whether it's "linking" or "copying", based on
"hardlink" value. The function is extracted so a later patch can reuse it.
2017-03-29 12:21:15 -07:00
Yuya Nishihara
91f8f53f3a statfs: make getfstype() raise OSError
It's better for getfstype() function to not suppress an error. Callers can
handle it as necessary. Now "hg debugfsinfo" will report OSError.
2017-03-25 17:25:23 +09:00
Denis Laxalde
a70f2fcec7 revset: factor out linerange processing into a utility function
Similar processing will be done in hgweb.webutil in forthcoming changeset.
2017-02-24 18:39:08 +01:00
Jun Wu
057a1065b9 util: enable hardlink for some BSD-family filesystems
Since we can now detect filesystems on FreeBSD and OSX. Add their major
filesystems (ufs, zfs for FreeBSD; hfs for OSX) to the hardlink whitelist.
2017-03-23 22:31:50 -07:00
Jun Wu
443e227575 util: use util.getfstype 2017-03-23 12:01:18 -07:00
Jun Wu
43dc9d6102 util: add a getfstype method
The util version is a thin wrapper of the osutil version, which is not
always available.
2017-03-23 11:58:45 -07:00
Jun Wu
2761a9ed0c util: enable hardlink for copyfile
This patch removes the global variable "allowhardlinks" that disables
hardlink in all cases, so hardlink gets enabled if the filesystem type is
whitelisted.

Third party extensions wanting to enable hardlink support unconditionally
can replace "_hardlinkfswhitelist.__contains__".
2017-03-12 01:03:23 -08:00
Jun Wu
d726efc89f util: disable hardlink for copyfile if fstype is outside a whitelist
Since osutil.getfstype is available, use it to detect filesystem types. The
whitelist currently includes common local filesystems on Linux where they
should have good hardlink support. We may add new filesystems for other
platforms later.
2017-03-12 00:23:07 -08:00
Gregory Szorc
56e2825efa py3: stop exporting urlparse from pycompat and util (API)
There are no consumers of this in tree.

Functions formerly available on this object/module can now be accessed
via {pycompat,util}.urlreq.
2017-03-21 22:47:49 -07:00
Gregory Szorc
28ebe71a15 util: use urlreq.unquote
pycompat.urlreq.unquote and pycompat.urlunquote effectively alias the
same thing. pycompat.urlunquote is only used once in the code base.
So let's switch to urlreq.unquote.

"Effectively" in the above paragraph is because pycompat.urlreq.unquote
aliases urllib.unquote and pycompat.urlunquote aliases urlparse.unquote
on Python 2. You might think one of urllib.unquote and urlparse.unquote
is an alias to the other, but you would be incorrect. In fact, these
functions are copies of each other. There is even a comment in the
CPython source code saying to keep them in sync. You can't make this
up.
2017-03-21 22:23:11 -07:00
Ryan McElroy
ac0305dcbb util: use tryunlink in unlinkpath
We just introduced a func to attempt a file removal. Start using it.
2017-03-21 06:50:28 -07:00
Ryan McElroy
ff386f9a99 util: add tryunlink function
Throughout mercurial cdoe, there is a common pattern of attempting to remove
a file and ignoring ENOENT errors. Let's move this into a common function to
allow for cleaner code.
2017-03-21 06:50:28 -07:00
Ryan McElroy
4456f7562c util: unify unlinkpath
Previously, there were two slightly different versions of unlinkpath between
windows and posix, but these differences were eliminated in previous patches.
Now we can unify these two code paths inside of the util module.
2017-03-21 06:50:28 -07:00
Augie Fackler
0598bcdf11 util: reference __main__ in sys.modules as a sysstr 2017-03-19 01:19:27 -04:00
Augie Fackler
c476b55519 util: use bytes re on bytes input in fspath
Fixes `hg add` on Python 3.
2017-03-19 00:16:39 -04:00
Augie Fackler
585f46ae2f util: use pycompat.bytestr in checkwinfilename
Fixes `hg add` on python3.
2017-03-19 00:16:08 -04:00
Yuya Nishihara
687bfe4011 py3: call codecs.escape_decode() directly
The same rule as 9ecf9a0c9837 applies.
2017-03-17 23:48:22 +09:00
Yuya Nishihara
dc88179a4e util: wrap s.decode('string_escape') calls for future py3 compatibility 2017-03-17 23:42:46 +09:00
Pierre-Yves David
fc2b521909 util: explicitly tests for None
Changeset 3b9cdb72931f removed the mutable default value, but did not explicitly
tested for None. Such implicit checking can introduce semantic and performance
issue. We move to an explicit check for None as recommended by PEP8:

https://www.python.org/dev/peps/pep-0008/#programming-recommendations
2017-03-15 15:07:14 -07:00
Yuya Nishihara
4744dd2998 py3: call codecs.escape_encode() directly
string_escape doesn't exist on Python 3, but fortunately the undocumented
codecs.escape_encode() function exists on CPython 2.6, 2.7, 3.5 and PyPy 5.6.
So let's use it for now.

http://stackoverflow.com/a/23151714
2017-03-15 23:28:39 +09:00
Yuya Nishihara
02022fc3c5 util: wrap s.encode('string_escape') call for future py3 compatibility 2017-03-15 23:06:50 +09:00
Yuya Nishihara
99a868d61d py3: call strftime() with native str type
Since strftime() may contain non-ascii character if locale set, we use
strfrom/tolocal().

Now "hg tip" works.
2017-03-13 09:19:07 -07:00
Yuya Nishihara
af7f25fdb3 encoding: add converter between native str and byte string
This kind of encoding conversion is unavoidable on Python 3.
2017-03-13 09:12:56 -07:00
Yuya Nishihara
dcade16cf7 encoding: factor out unicode variants of from/tolocal()
Unfortunately, these functions will be commonly used on Python 3.
2017-03-13 09:11:08 -07:00
Rishabh Madan
8a9d68951e py3: use iter() instead of iterkeys() 2017-03-16 04:53:23 +05:30
Gregory Szorc
1c399d6b97 util: make strdate's defaults default value a dict
It was specified to be an empty list in 3d8abfdaa08a in 2007.
It was correct at the time. But when the function was
refactored in 7bca0f2718ab (2010), it started expecting a dict.
I guess this code path is untested?

Thanks to Yuya for spotting this.
2017-03-14 08:51:35 -07:00
Gregory Szorc
fbefc30c75 util: don't use mutable default argument value
I don't think this is any tight loops and we'd need to worry about
PyObject creation overhead. Also, I'm pretty sure strptime()
will be much slower than PyObject creation (date parsing is
surprisingly slow).
2017-03-12 21:54:32 -07:00
Augie Fackler
d5d09dfdf4 util: teach url object about __bytes__
__str__ tries to do something reasonable, but someone else more
familiar with encoding bugs should check my work.
2017-03-12 03:33:22 -04:00
Pulkit Goyal
7deacd3d03 util: pass encoding.[encoding|encodingmode] as unicodes
We need to pass str to encode() and decode().
2017-03-12 07:35:13 +05:30
Mads Kiilerich
6cbae57046 util: add debugstacktrace depth limit
Useful when you don't care about the start of the stack, but only want to see
the last entries.
2015-01-14 01:15:26 +01:00
Mads Kiilerich
97ac40d793 util: strip trailing newline from debugstacktrace message
This makes the function more convenient to use as drop-in replacement for
ui.write & co.
2015-01-16 04:26:40 +01:00
Durham Goode
56acb0faaa util: add allowhardlinks module variable
To enable extensions to enable hardlinks for certain environments, let's move
the 'if False' to be an 'if allowhardlinks' and let extensions modify the
allowhardlinks variable.

Tests on linux ext4 pass with it set to True and to False.
2017-03-02 10:12:40 -08:00
Yuya Nishihara
8072a1daee chg: deduplicate error handling of ui.system()
This moves 'onerr' handling from low-level util.system() to higher level,
which seems better API separation.
2017-02-19 01:16:45 +09:00
Pulkit Goyal
22dbb9466c py3: use pycompat.fsencode() to convert __file__ to bytes
__file__ returns unicodes on Python 3. This patch uses pycompat.fsencode() to
convert them to bytes.
2017-02-20 18:40:42 +05:30
Pulkit Goyal
7f47e65fef py3: convert the mode argument of os.fdopen to unicodes
Couple of these from the earlier series got lost while rebasing. So this patch
converts them again.
2017-02-16 17:30:35 +05:30
Simon Farnsworth
e0b70e4f7f mercurial: switch to util.timer for all interval timings
util.timer is now the best available interval timer, at the expense of not
having a known epoch. Let's use it whenever the epoch is irrelevant.
2017-02-15 13:17:39 -08:00
Simon Farnsworth
9415b63fba util: introduce timer()
As documented for timeit.default_timer, there are better timers available for
performance measures on some platforms. These timers don't have a set epoch,
and thus are only useful for interval measurements, but have higher
resolution, and thus get you a better measurement overall.

Use the same selection logic as Python's timeit.default_timer. This is a
platform clock on Python 2 and early Python 3, and time.perf_counter on Python
3.3 and later (where time.perf_counter is introduced as the best timer to use).
2017-02-15 11:53:59 -08:00
Pulkit Goyal
56031921a5 py3: convert the mode argument of os.fdopen to unicodes (2 of 2) 2017-02-13 22:15:28 +05:30