The subprocess module is not thread safe. Spawning a thread to read
the output leads to exceptions like this when Mercurial exits:
Exception exceptions.TypeError: TypeError("'NoneType' object is not
callable",) in <bound method Popen.__del__ of <subprocess.Popen
object at 0x9ed0dcc>> ignored
The bug is already reported in the Python bug tracker:
http://bugs.python.org/issue1731717
Some modules (like revlog) would import util.sha1 as _sha1. This
defeats the purpose of having util.sha1 overwrite itself with a faster
version -- revlog would end up always calling the slow version. By
always delegating to util._fastsha1 we avoid this at the cost of an
extra (but unconditional) indirection.
Use a temporary file name as target for a forced rename on Windows. The
target file name is not opened at any time; just renamed into and then
unlinked. Using a temporary instead of a static name is necessary since
otherwise a hg crash can leave the file lying around, blocking future
attempts at renaming.
It seems like the old behaviour with different handling for commands with and
without path was intended, but I think this behaviour of util.find_exe is
better:
* Always returns existing file
* or None if command not found - no default
* Windows: Returned file thus always ends with extension from PATHEXT
This fixes http://www.selenic.com/mercurial/bts/issue1459. The change might
fix other unintended behaviour too.
- simplify version detection code
- move detection code into setup.py
- move version reading function into util.py
- drop version.py code
This makes hg more closely follow its own recommendation of how to deal with
versioning your builds: use hg id in your build script.
Implemented as two functions: diffstat, which yields lines of text,
formatted as a usual diffstat output, and diffstatdata, which is called
inside diffstat to do real performing and yield file names with
appropriate data (numbers of added and removed lines).
collections: direct child repos only
paths *: direct child repos only (like collections)
paths **: recursive discovery
When ** is used, the mq repository (if any) is also shown.
Currently hgweb is not streaming its output -- it accumulates the
entire response before sending it. This patch restores streaming
behaviour. To avoid having to synchronously write many tiny fragments,
this patch also adds buffering to the template generator. Local
testing of a fetch of a 100,000 line file with wget produces a slight
slowdown overall (up from 6.5 seconds to 7.2 seconds), but instead of
waiting 6 seconds for headers to arrive, output begins immediately.
util module implements two versions of statfiles function
_statfiles calls lstat per file
_statfiles_clustered takes advantage of optimizations in osutil.c, stats all
files in directory at once when new directory is hit and caches the results
util.statfiles dispatches to appropriate version during module loading
The speedup on directory tree with 2k directories and 63k files is about
factor of 1.8 (1.3s -> 0.8s for hg diff - hg startup overhead about .2s)
At this point only Win32 now benefit from this patch.
Rest of OSes use the non clustered implementation.
add _checklink var to dirstate
introduce dirstate.flagfunc
switch users of util.execfunc/linkfunc to flagfunc
change manifestdict.set to take a flags string
change ctx.fileflags to ctx.flags
change gitmode func to a dict
remove util.execfunc/linkfunc
The function, given a relative filename and a root, returns the filename
modified to use the case actually stored in the filesystem (or None if the
file does not exist). The returned name is relative to the root, but retains
the path separators used in the input path. (This is not strictly necessary,
but retaining the path separators minimises misleading test suite failures).
A win32-specific implementation (using win32api.FindFiles) is possible, but it
has not been implemented as testing seems to demonstrate that the
win32-specific code is not significantly faster (thanks to the caching of
results in the generic code).
datestr:
- add format specifiers %1 and %2 for timezone hours and minutes
- remove timezone and timezone format options
- correctly find timezone hours and minutes for fractional and negative timezones
- update users
strdate:
- correctly find timezone hours and minutes for fractional and negative timezones
Moves ssh argument building to platform specific utils code.
The win32 version looks for plink in ssh command string and
uses '-P' in lieu of '-p' for specifying a port
Allow adding to dirstate files that clash with previously existing
but marked for removal. Protect from reintroducing clashes by revert.
This change doesn't address related issues with update. Current
workaround is to do "clean" update by manually removing conflicting
files/dirs from working directory.
commit (aborts _after_ typing in a commit message)
backout (aborted after the initial revert)
tag (edited .hgtags and couldn't commit)
import (patch applied, then commit fails)
qnew (aborts on bad dates, but writes any valid date into the # Date header)
qrefresh (like qnew)
sign (like tag)
fetch (merge, merge, merge, merge, abort)
This could happen e.g. in group writable local repositories where a file
should become executable on update.
(Patch by Benoit Boissinot attached to issue530)
On Linux VFAT execution mode can be modified, but changes don't
persist a filesy stem remount. The current test can be trickled by
this. We can help with the det ection of VFAT checking whether new
files get created with the execution bits on
(as usually these partitions are mounted with the exec option, for
convenience)
.
We use a separate cache to avoid problems with
audit = path_auditor(repo.root)
audit("subrepo")
audit("subrepo/file")
whitelisting "subrepo" (which is fine) and then using the same whitelist
with "subrepo/file" (which is not fine).
Since we create a separate path_auditor for every path on the command line,
a "hg add dir/a dir/b dir/c" will still lstat dir 3 times just to audit
the paths.
The following properties of a path are now checked for:
- under top-level .hg
- starts at the root of a windows drive
- contains ".."
- traverses a symlink (e.g. a/symlink_here/b)
- inside a nested repository
If any of these is true, the path is rejected.
The check for traversing a symlink is arguably stricter than necessary;
perhaps we should be checking for symlinks that point outside the
repository.
Simply use find_exe('hg') as the default value for $HG and require to manually
set it if you have special requirements.
While the default will not always be 100% correct (i.e. the identical hg
version) for many users it is and for the others the hg executable found in
the PATH should do most things correctly.
Developers or other users with multiple installs can set $HG or run something
like util.set_hgexecutable in their shell or python scripts.
Additionally util.hgexecutable() is now available so extensions can access
the value with a public interface, too.
Differences from os.symlink:
- the symlink name is relative to the opener base directory
- if a file with that name already exists, it's removed
- if necessary, parent directories are created
- if the system (OS or filesystem) doesn't support symlinks, a
regular file is created. Its contents are the symlink target.
The original idea might have been to prevent circular references, but
as this assignment would have created another reference, this makes
no difference.
In python 2.4+ on darwin, locale.getpreferredencoding() returns
mac-roman regardless of what LC_CTYPE, LANG etc are set to. This can
produce hard-to-notice conversion errors if input text is not in
mac-roman. So this patch overrides it with setlocale/getlocale if the
environment has been customized, on the assumption that the user has
done so deliberately.
The interface provided by opener(atomic=True) is inherently unsafe:
if an exception is raised in the code using the atomic file, the
possibly incomplete file will be renamed to its final destination,
defeating the whole purpose of atomic files.
To get around this, we would either need some bad hacks involving
sys.exc_info (to make sure things work in except: blocks), or an
interface to say "file is complete; rename it".
This is the exact interface provided by atomictempfile. Since there
are no remaining users of the atomicfile class, just remove it.
The behaviour of find_in_path was broken for config options containing
path names, because it always searched the given path, even when not
necessary. The find_exe function is more polite: if the name passed
to it contains a path component, it just returns it.
In cdc7e3627e1b, Benoit made a change that substantially slows matching
when a big .hgignore file is in play, because it calls into the regexp
matching engine potentially hundreds of times per file to be matched.
I've partly rolled back his change, so that we only call into the matcher
once per file, but preserved the ability to report a meaningful error
message if there's a syntax error in the regexp.
Right now, surprisingly enough, if you request an atomic file but the
file still doesn't exist, you get a regular file. AFAICS, the only time
this happens is during the initial creation of the dirstate.
With this change, you have to use "hg locate 'hgweb/**'" to locate
all the files in directories named hgweb. OTOH, "hg locate '*l'"
will locate only files that end with "l" - e.g. a file called "hg.py"
will not be matched just because it's in a directory whose name ends
with "l" (e.g. "mercurial").
With that changeset, it's impossible to use a glob: pattern to match
e.g. all files ending in .py - glob:**.py would also match all files
in a directory called dir.py.
This makes the behaviour of glob: patterns more consistent:
hg status glob:dir and hg status -I glob:dir will match
the same files.
It's also consistent with the fact that {rel,}path patterns
recursively match the contents of a directory.
This will trigger every time somebody runs something like "hg diff"
or "hg status" without any arguments.
The important part here is returning util.always as the match function,
which is a much simpler (and faster) function than the usual return
value, and allows other code to just skip the filtering if it knows
all files will match.
This makes its default behaviour useful again (issue108), and
changes it search the entire repository by default (instead
of just the cwd), just like all other commands.
It also hides issue204 by default, but you'll still see the
same behaviour if you give it a relpath: pattern.
This removes a hack where we appended '/' to a dirname so that:
- it would not appear on the "dc" dict
- it would always be matched by the match function
This was a contorted way of checking if the directory was matched by
some hgignore pattern, and it would still fail with some uses of
--include/--exclude patterns.
Things would still work fine if we removed the check altogether and
just appended things to "work" directly, but then we would end up
walking ignored directories too, which could be quite a bit of work.
This allows further simplification of the match function returned by
util._matcher, and fixes walking the working directory with a
--include pattern that matches only the end of a name.
This should fix issue347.
It also highlights one issue with the directory walking code when
you have an --include pattern that matches the end of a filename.
This is fixed by the next patch.
names=['.'] means "include (recursively) only files from the current subdir";
the function then did a hack to walk the whole tree. Clean that up.
This also fixes a problem where "--include ." works in a subdir, but not
on the tree root.
The main problem was that dirstate.getcwd() returned just "",
which was interpreted as "we're at the repo root". It now returns
an absolute path.
The util.pathto function was also changed to deal with the "cwd is
an absolute path" case.
Changed os_rcpath to combine system_rcpath and user_rcpath. Changed
system_rcpath and user_rcpath to both return a list of paths to add to the
combined rcpath for both Windows NT and other platforms.
Add extended date formats (eg "Dec", "2006")
Add a couple missing basic date formats
Improve default date element scheme to parsedate
Add matchdate function to match a date spec
Add -e switch and range matching to debugdate
The list of trusted users and groups is specified in the [trusted]
section of a hgrc; the current user is always trusted; "*" can be
used to trust all users/groups.
Global hgrc files are always read.
On Windows (and other systems that don't have the pwd and grp modules),
all .hg/hgrc files are read.
This is essentially the same patch that was previously applied as
revision f077d29b114d.
This change has the potential to break existing setups, but the current
behaviour (the keys in configuration files are always lower-cased) can
bite us in a few places:
- no way to use a Command in [defaults]
- hgext.Extension doesn't work in [extensions]
- you can't use an Upper/case/PATH in the [paths] section of hgweb.config
- you can't (easily) protect paths with upper-case letters with the
acl extension
- you can't specify a /Path/TO/a/rEPO in the [reposubs] section for
the notify extension
Vanilla patch chokes on git patches that include files that are copied
or renamed, then modified. So this code detects that case and rewrites
the patch if necessary.
existing clone code uses pull to get changes from remote repo. is very
slow, uses lots of memory and cpu.
new clone code has server write file data straight to client, client
writes file data straight to disk. memory and cpu used are very low,
clone is much faster over lan.
new client can still clone with pull, can still clone from older servers.
new server can still serve older clients.
str.find return -1 when the substring is not found, -1 evaluate
to True and is a valid index, which can lead to bugs.
Using alternatives when possible makes the code clearer and less
prone to bugs. (and __contains__ is faster in microbenchmarks)
When the gpatch fix for solaris was introduced in b67447b909f3 the
patch command was "". For some strange reason windows 2000 is
not happy with those quotes when given in os.popen.
This hides the following traceback (there's a race condition involved,
so you may have to try a few times to hit it):
$ hg sign --key key-that-does-not-exist
Signing 2062:4bad92f4ea65
gpg: skipped "key-that-does-not-exist": secret key not available
gpg: signing failed: secret key not available
Exception in thread Thread-1:
Traceback (most recent call last):
File "threading.py", line 442, in __bootstrap
self.run()
File "threading.py", line 422, in run
self.__target(*self.__args, **self.__kwargs)
File "mercurial/util.py", line 24, in writer
pin.close()
IOError: [Errno 32] Broken pipe
abort: Error while signing
The revlog.checkinlinesize() uses an atomic opener to replace the
index file after converting it from inline to traditional .i and .d
files. If this operation is interrupted, the atomic file class can
overwrite a valid file with a partially written one.
This patch introduces an atomic opener that does not automatically
replace the destination file with the tempfile. This way
an interrupted checkinlinesize() call turns into a noop.
Handle processes that no longer exist and processes that belong to another user.
Enables the lock breaking changes from 0ab4c12ad060 and subsequently "fixes" the left over locks reported in bug 112.
to use, "hg log --style=changelog". makes different output with no
flags, -q, -v, --debug.
templater module has new template filters for this.
email - committer email address
fill68 - refill text to 68 colums
fill76 - refill text to 76 colums
tabindent - prefix every not empty line with tab
shortdate - iso 8631 date, no time zone
stringify - turn template iterator into string
if set, override default hgrc search path.
if empty, only .hg/hgrc of current repo read.
for each element, if directory, all entries in directory with end in
".rc" are added to path. else, element is added to path.
big thing about this change is that user "~/.hgrc" and system hgrc not
longer breaks tests. run-tests makes HGRCPATH empty now.
old style: symlink to pid
new style: symlink to hostname:pid
if lock code finds new-style lock, it breaks lock if locking pid is on
same machine and pid is not alive.
otherwise, lock is left alone. this makes locking code safe with
old-style locks and with locks on other machines.
new code makes server part of mercurial more robust in case machine
crashes, power fails, or crazy user does kill -9.
now it searches <install dir>/etc/mercurial, /etc/mercurial, and user
hgrc.
this allows site-wide configuration to be shared over automounted nfs
partition, instead of chenging on every system. option of having local
configuration on every system remains.
old code for searching /etc/mercurial/hgrc.d never worked, this code
is tested and works.
We used to pass changelog dates around as a "unixtime timezone" string
containing a pair of encoded ints. Now, they get passed around as a
(unixtime, timezone) tuple of numbers, which makes much more sense.
