Experimental because given the possible complexity, it may be worth figuring out
how to load this from a file, similar to the style files for the log command,
instead of trying to stuff it on the command line.
There are no visible changes here.
I'm starting to wonder if adding the '+' to the 'node' line instead of a
separate key line in a4d42f6edc09 was the right thing to do. The '{node}'
keyword never includes '+' elsewhere, and the way setup.py works, it would
truncate it anyway. Additionally, the file is missing '{p2node}' when 'wdir()'
merges are archived. I thought about adding an 'identify' line that would
correspond to `hg id -n`. But the other nodes are the full 40 characters, and
the output most useful for versioning is the short form. All of this cries out
for customization via templating. (Although maybe having the short identify
line by default is still a good idea.)
This is done by a script [2] using RedBaron [1], a tool designed for doing
code refactoring. All "default" values are decided by the script and are
strongly consistent with the existing code.
There are 2 changes done manually to fix tests:
[warn] mercurial/exchange.py: experimental.bundle2-output-capture: default needs manual removal
[warn] mercurial/localrepo.py: experimental.hook-track-tags: default needs manual removal
Since RedBaron is not confident about how to indent things [2].
[1]: https://github.com/PyCQA/redbaron
[2]: https://github.com/PyCQA/redbaron/issues/100
[3]:
#!/usr/bin/env python
# codemod_configitems.py - codemod tool to fill configitems
#
# Copyright 2017 Facebook, Inc.
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.
from __future__ import absolute_import, print_function
import os
import sys
import redbaron
def readpath(path):
with open(path) as f:
return f.read()
def writepath(path, content):
with open(path, 'w') as f:
f.write(content)
_configmethods = {'config', 'configbool', 'configint', 'configbytes',
'configlist', 'configdate'}
def extractstring(rnode):
"""get the string from a RedBaron string or call_argument node"""
while rnode.type != 'string':
rnode = rnode.value
return rnode.value[1:-1] # unquote, "'str'" -> "str"
def uiconfigitems(red):
"""match *.ui.config* pattern, yield (node, method, args, section, name)"""
for node in red.find_all('atomtrailers'):
entry = None
try:
obj = node[-3].value
method = node[-2].value
args = node[-1]
section = args[0].value
name = args[1].value
if (obj in ('ui', 'self') and method in _configmethods
and section.type == 'string' and name.type == 'string'):
entry = (node, method, args, extractstring(section),
extractstring(name))
except Exception:
pass
else:
if entry:
yield entry
def coreconfigitems(red):
"""match coreconfigitem(...) pattern, yield (node, args, section, name)"""
for node in red.find_all('atomtrailers'):
entry = None
try:
args = node[1]
section = args[0].value
name = args[1].value
if (node[0].value == 'coreconfigitem' and section.type == 'string'
and name.type == 'string'):
entry = (node, args, extractstring(section),
extractstring(name))
except Exception:
pass
else:
if entry:
yield entry
def registercoreconfig(cfgred, section, name, defaultrepr):
"""insert coreconfigitem to cfgred AST
section and name are plain string, defaultrepr is a string
"""
# find a place to insert the "coreconfigitem" item
entries = list(coreconfigitems(cfgred))
for node, args, nodesection, nodename in reversed(entries):
if (nodesection, nodename) < (section, name):
# insert after this entry
node.insert_after(
'coreconfigitem(%r, %r,\n'
' default=%s,\n'
')' % (section, name, defaultrepr))
return
def main(argv):
if not argv:
print('Usage: codemod_configitems.py FILES\n'
'For example, FILES could be "{hgext,mercurial}/*/**.py"')
dirname = os.path.dirname
reporoot = dirname(dirname(dirname(os.path.abspath(__file__))))
# register configitems to this destination
cfgpath = os.path.join(reporoot, 'mercurial', 'configitems.py')
cfgred = redbaron.RedBaron(readpath(cfgpath))
# state about what to do
registered = set((s, n) for n, a, s, n in coreconfigitems(cfgred))
toregister = {} # {(section, name): defaultrepr}
coreconfigs = set() # {(section, name)}, whether it's used in core
# first loop: scan all files before taking any action
for i, path in enumerate(argv):
print('(%d/%d) scanning %s' % (i + 1, len(argv), path))
iscore = ('mercurial' in path) and ('hgext' not in path)
red = redbaron.RedBaron(readpath(path))
# find all repo.ui.config* and ui.config* calls, and collect their
# section, name and default value information.
for node, method, args, section, name in uiconfigitems(red):
if section == 'web':
# [web] section has some weirdness, ignore them for now
continue
defaultrepr = None
key = (section, name)
if len(args) == 2:
if key in registered:
continue
if method == 'configlist':
defaultrepr = 'list'
elif method == 'configbool':
defaultrepr = 'False'
else:
defaultrepr = 'None'
elif len(args) >= 3 and (args[2].target is None or
args[2].target.value == 'default'):
# try to understand the "default" value
dnode = args[2].value
if dnode.type == 'name':
if dnode.value in {'None', 'True', 'False'}:
defaultrepr = dnode.value
elif dnode.type == 'string':
defaultrepr = repr(dnode.value[1:-1])
elif dnode.type in ('int', 'float'):
defaultrepr = dnode.value
# inconsistent default
if key in toregister and toregister[key] != defaultrepr:
defaultrepr = None
# interesting to rewrite
if key not in registered:
if defaultrepr is None:
print('[note] %s: %s.%s: unsupported default'
% (path, section, name))
registered.add(key) # skip checking it again
else:
toregister[key] = defaultrepr
if iscore:
coreconfigs.add(key)
# second loop: rewrite files given "toregister" result
for path in argv:
# reconstruct redbaron - trade CPU for memory
red = redbaron.RedBaron(readpath(path))
changed = False
for node, method, args, section, name in uiconfigitems(red):
key = (section, name)
defaultrepr = toregister.get(key)
if defaultrepr is None or key not in coreconfigs:
continue
if len(args) >= 3 and (args[2].target is None or
args[2].target.value == 'default'):
try:
del args[2]
changed = True
except Exception:
# redbaron fails to do the rewrite due to indentation
# see https://github.com/PyCQA/redbaron/issues/100
print('[warn] %s: %s.%s: default needs manual removal'
% (path, section, name))
if key not in registered:
print('registering %s.%s' % (section, name))
registercoreconfig(cfgred, section, name, defaultrepr)
registered.add(key)
if changed:
print('updating %s' % path)
writepath(path, red.dumps())
if toregister:
print('updating configitems.py')
writepath(cfgpath, cfgred.dumps())
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
Since the identify command adds a '+' for missing files, it's reasonable that
this does too. Perhaps the node field's hex value should be p1+p2 for merges?
Now that the 'vfs' classes moved in their own module, lets use the new module
directly. We update code iteratively to help with possible bisect needs in the
future.
The decoders were already run by default for the main repo, so this seemed like
an oversight.
The extdiff extension has been using 'archive' since a80ec1ea2694 to support -S,
and a colleague noticed that after diffing, making changes, and closing it, the
line endings were wrong for the diff-tool modified files in the subrepository.
(Files in the parent repo were correct, with the same .hgeol settings.) The
editor (Visual Studio in this case) reloads the file, but doesn't notice the EOL
change. It still adds new lines with the original EOL setting, and the file
ends up inconsistent.
Without this change, the first file `cat`d in the test prints '\r (esc)' EOL,
but the second doesn't on Windows or Linux.
The home of 'Abort' is 'error' not 'util' however, a lot of code seems to be
confused about that and gives all the credit to 'util' instead of the
hardworking 'error'. In a spirit of equity, we break the cycle of injustice and
give back to 'error' the respect it deserves. And screw that 'util' poser.
For great justice.
Hardcoding '.' is wrong, and yielded strange results when archiving old
revisions. For example, when archiving the cset that adds the signature to 3.4
(a4f6d198e7df), the resulting value was previously 51 (the number of commits on
stable between 3.4 and today), even though it was a direct descendant of a tag,
with a {latesttagdistance} of 2. This still includes all other _ancestor_ paths
not included in {latesttag}.
Note that archiving wdir() currently blows up several lines above this when
building the 'base' variable. Since wdir() isn't documented, ignore that it
needs work to handle wdir() here for now.
Python 2.6 introduced a new octal syntax: "0oXXX", replacing "0XXX". The
old syntax is not recognized in Python 3 and will result in a parse
error.
Mass rewrite all instances of the old octal syntax to the new syntax.
This patch was generated by `2to3 -f numliterals -w -n .` and the diff
was selectively recorded to exclude changes to "<N>l" syntax conversion,
which will be handled separately.
This is a step toward replacing the extdiff internals with archive, in order to
support 'extdiff -S'. Only Mercurial subrepos are supported for now.
If a file is missing from the filesystem, it is silently skipped. Perhaps it
should warn, but it cannot abort when working with extdiff because deleting a
file is a legitimate diff.
All current callers supply some sort of prefix, so the issue was hidden. But if
no parameter was specified, a crash occurred in the write() closure when
concatenating 'prefix' and 'name'.
This is different from latesttagdistance in that while latesttagdistance is
defined to be the length of the longest path to the latest tag,
changessincelatesttag is the number of changes contained in @ that aren't
contained in the latest tag. So, if 't' is the latest tag in the repository
below:
t
|
v
--o--o----o
\ \
..o..o..@
then latesttagdistance is 2, but changessincelatesttag is 4.
Note that changessincelatesttag is always greater than or equal to the
latesttagdistance -- that's because changessincelatesttag counts all the
changes in the longest path since the latest tag, and possibly others. This is
an important fact that we'll take advantage of in upcoming patches.
The current state of subrepo methods is to pass a 'ui' object to some methods,
which has the effect of overriding the subrepo configuration since it is the
root repo's 'ui' that is passed along as deep as there are subrepos. Other
subrepo method are *not* passed the root 'ui', and instead delegate to their
repo object's 'ui'. Even in the former case where the root 'ui' is available,
some methods are inconsistent in their use of both the root 'ui' and the local
repo's 'ui'. (Consider hg._incoming() uses the root 'ui' for path expansion
and some status messages, but also calls bundlerepo.getremotechanges(), which
eventually calls discovery.findcommonincoming(), which calls
setdiscovery.findcommonheads(), which calls status() on the local repo 'ui'.)
This inconsistency with respect to the configured output level is probably
always hidden, because --verbose, --debug and --quiet, along with their 'ui.xxx'
equivalents in the global and user level hgrc files are propagated from the
parent repo to the subrepo via 'baseui'. The 'ui.xxx' settings in the parent
repo hgrc file are not propagated, but that seems like an unusual thing to set
on a per repo config file. Any 'ui.xxx' options changed by --config are also
not propagated, because they are set on repo.ui by dispatch.py, not repo.baseui.
The goal here is to cleanup the subrepo methods by dropping the 'ui' parameter,
which in turn prevents mixing subtly different 'ui' instances on a given subrepo
level. Some methods use more than just the output level settings in 'ui' (add
for example ends up calling scmutil.checkportabilityalert() with both the root
and local repo's 'ui' at different points). This series just goes for the low
hanging fruit and switches methods that only use the output level.
If we really care about not letting a subrepo config override the root repo's
output level, we can propagate the verbose, debug and quiet settings to the
subrepo in the same way 'ui.commitsubrepos' is in hgsubrepo.__init__.
Archive only uses the 'ui' object to call its progress() method, and gitsubrepo
calls status().
Unix utilities like tar will happily prefix the files it packs with './', but
annoyingly, Windows Explorer will not show these packed files when it opens the
archive. Since there doesn't seem to be a point in including './' in the path
names, just drop it. The default 'hg archive' prefix is the basename of the
archive, so specifying '.' allows for that default to be disabled completely.
Note that we could raise this exception even if no pattern were specified, but
the revision contained no files. However this should not happen in practice
since in that case commands.py/archive would exit earlier with an "no working
directory: please specify a revision" error message instead.
Add sorted() in places found by testing with PYTHONHASHSEED=random and code
inspection.
An alternative to sprinkling sorted() all over would be to change substate to a
custom dict with sorted iterators...
Some archive types closed the open file passed to it, some didn't.
This could cause either missing or duplicate close and cause problems in hgweb.
The fix in 4f98880c1b4e should only have closed the compressors and archivers -
not the underlying file itself if no compressor is used.
The existing workaround didn't work when no filename was specified.
If running in a context with warnings enabled and subsecond mtime it gave a
warning:
DeprecationWarning: use the name attribute
Before this patch, zip archives created by "hg archive" are extracted
with unexpected timestamp, if TZ is not configured as GMT.
This patch adds "extended-timestamp" extra block to zip archives, and
unzip will extract such archives with timestamp specified in added
extra block, even though TZ is not configured as GMT.
Please see documents below for detail about specification of zip file
format and "extended-timestamp" extra block:
http://www.pkware.com/documents/casestudies/APPNOTE.TXThttp://www.opensource.apple.com/source/zip/zip-6/unzip/unzip/proginfo/extra.fld
Original implementation of this patch was suggested by "Jun Omae
<jun66j5@gmail.com>".
Add a match object to subrepo.archive(). This will allow the -X and -I
options to be honored inside subrepos when archiving. They formerly
only affect the top level repo.
The usual contract is that close() makes your writes permanent, so
atomictempfile's use of close() to *discard* writes (and rename() to
keep them) is rather unexpected. Thus, change it so close() makes
things permanent and add a new discard() method to throw them away.
discard() is only used internally, in __del__(), to ensure that writes
are discarded when an atomictempfile object goes out of scope.
I audited mercurial.*, hgext.*, and ~80 third-party extensions, and
found no one using the existing semantics of close() to discard
writes, so this should be safe.
Add missing calls to close() to many places where files are
opened. Relying on reference counting to catch them soon-ish is not
portable and fails in environments with a proper GC, such as PyPy.
This mainly affects hgweb, which can generate tar.gz archives without
filenames. Without this change, the header would be set to ".gz",
which can confuse Safari into extracting the file and renaming it to
"gz" when "Open 'safe' files after downloading" is enabled.
file(1) before:
hg-crew-439421eab08d.tar.gz: gzip compressed data, was ".gz", last modified: Thu Dec 2 11:46:20 2010, max compression
after:
hg-crew-439421eab08d.tar.gz: gzip compressed data, last modified: Thu Dec 2 11:46:20 2010, max compression
Previously, branch names were ideally manipulated as UTF-8 strings,
because they were stored as UTF-8 in the dirstate and the changelog
and could not be safely converted to the local encoding and back.
However, only about 80% of branch name code was actually using the
right encoding conventions. This patch uses the localstr addition to
allow working on branch names as local strings, which simplifies
handling so that the previously incorrect code becomes correct.
