For a netbeans clone on Windows 7 x64:
Before:
$ hg perffncacheencode
! wall 3.516000 comb 3.525623 user 3.525623 sys 0.000000 (best of 3)
After:
$ hg perffncacheencode
! wall 3.443000 comb 3.447622 user 3.447622 sys 0.000000 (best of 3)
For a netbeans clone on Windows 7 x64:
Before:
$ hg perffncacheload
! wall 0.124000 comb 0.124801 user 0.124801 sys 0.000000 (best of 76)
After:
$ hg perffncacheload
! wall 0.096000 comb 0.093601 user 0.078001 sys 0.015600 (best of 97)
For a netbeans clone on Windows 7 x64:
Before:
$ hg perffncachewrite
! wall 0.210000 comb 0.218401 user 0.202801 sys 0.015600 (best of 47)
After:
$ hg perffncachewrite
! wall 0.104000 comb 0.109201 user 0.078000 sys 0.031200 (best of 95)
I don't think we will ever have anything in the store that resides inside a
directory that ends in .i or .d under store/ that we wouldn't want to have
direncoded. The files not under data/ surely don't need direncoding, but it
doesn't harm to let these few run through it. It hurts more to check whether the
thousands of other files start with 'data/'. They do anyway.
See also 67e6074ba430 (fixed with 0c522fe42894), which moved the direncoding
from filelog into store
by refactoring
for i, n in enumerate(res):
if n:
<main code block>
to
for i, n in enumerate(res):
if not n:
continue
<main code block>
(no functional change)
This change adds `obsstore` to the list of files copied by local clone,
until now changesets were copied without their obsolete markers.
Note: extinct changesets were and are still included by such clones to
enable hardlinking. There is no obvious reason to prevent their exchange
here.
Rebased by Patrick Mezard <patrick@mezard.eu>
In my tests of an fncache containing 300,000 entries, this improves
read time from 567ms to 307, and write time from 1328ms to 533.
These numbers aren't so great, since the fncache file is only 17MB
in size, but they're an improvement.
The usual contract is that close() makes your writes permanent, so
atomictempfile's use of close() to *discard* writes (and rename() to
keep them) is rather unexpected. Thus, change it so close() makes
things permanent and add a new discard() method to throw them away.
discard() is only used internally, in __del__(), to ensure that writes
are discarded when an atomictempfile object goes out of scope.
I audited mercurial.*, hgext.*, and ~80 third-party extensions, and
found no one using the existing semantics of close() to discard
writes, so this should be safe.
as stated at http://mercurial.selenic.com/wiki/CodingStyle
(see also http://selenic.com/pipermail/mercurial-devel/2011-May/031139.html )
name changes done at module scope:
_build_lower_encodefun -> _buildlowerencodefun
_windows_reserved_filenames -> _winreservednames (see 42a6bcfc9d44)
MAX_PATH_LEN_IN_HGSTORE -> _maxstorepathlen
DIR_PREFIX_LEN -> _dirprefixlen
_MAX_SHORTENED_DIRS_LEN -> _maxshortdirslen
(no users of these outside the store module)
changed locals:
win_reserved -> winreserved
space_left -> spaceleft
The 'opener' argument wasn't, in fact, an actual opener instance, but
rather something expected to return an opener. The normal argument,
from localrepository, is the scmutil.opener type; hence 'openertype'.
We can stop pretending that we have to support anything else
than '/' for concatenating path elements anywhere.
Windows deals just fine with '/' in file paths and we already
have plenty of places which produce paths containing '/'
anyway when running on Windows.
Defers updating the fncache file with newly added entries to the end of
the transaction (on e.g. pull), doing a single open call on the fncache
file, instead of opening and closing it each time a new entry is added
to the store.
Implemented by adding a new abstract write() function on store.basicstore
and registering it as a release function on the store lock in
localrepo.lock (compare with dirstate.write).
store.fncachestore overrides write() from basicstore and calls a new
write function on the fncache object, which writes all entries to the
fncache file if it's dirty.
store.fncache.add() now just marks itself as dirty if a new name is added.
Before this patch, the copy order on clone was:
requires
00changelog.i
store\data
store\00manifest.d
store\00manifest.i
store\00changelog.d
store\00changelog.i
store\dh
store\fncache
Which provides a theoretical non-zero probability of a race during clone where
a very early reader might see a repository with missing revlog files if it sees
00changelog.i before all files inside dh have been copied.
The dh directory is similar to the data directory -- just for files with long
names (which are hashed). The manifest refers to files in data *and* dh, so dh
should be copied before the manifest.
This patch improves the copy order to:
requires
00changelog.i
store\data
store\dh
store\fncache
store\00manifest.d
store\00manifest.i
store\00changelog.d
store\00changelog.i
I'm putting fncache to before the manifest while I'm at it, since fncache
provides a mechanism to enumerate all repository files without visiting the
manifest revisions. fncache depends only on data and dh.
Note that data must be copied first, since copying data triggers the creation
of the repository write lock in the destination repo (see hg.clone).
- Mac OS X has problems with filenames starting with '._'
(e.g. '.FOO' -> '._f_o_o' is now encoded as '~2e_f_o_o')
- Explorer of Windows Vista and Windows 7 strip leading spaces of
path elements of filenames when copying trees
Above problems are avoided by encoding the first space (as '~20') or
period (as '~2e') of all path elements.
This introduces a new entry 'dotencode' in .hg/requires, that is,
a new repository filename layout (inside .hg/store).
Newly created repositories require 'dotencode' by default. Specifying
[format]
dotencode = False
in a config file will use the old format instead.
Prior Mercurial versions will abort with the message
abort: requirement 'dotencode' not supported!
when trying to access a local repository that requires 'dotencode'.
New 'dotencode' repositories can be converted to the previous
repository format with
hg --config format.dotencode=0 clone --pull repoA repoB
The three replace calls are slower than this simple __contains__,
and anyway we should not have this many paths ending with .i, .d, or .hg
compared to the normal, un-encoded other paths.
Newly added fncache entries were not added to the in-memory cache,
making it possible for 'hg convert' to cause duplicates in
.hg/store/fncache.
Duplicates in the fncache file are harmless, but excessive numbers
of duplicates from large converted repositories may slow down
execution speed considerably.
