Commit Graph

29950 Commits

Author SHA1 Message Date
Pierre-Yves David
80ed73689f checkcopies: move 'movewithdir' initialisation right before its usage
The 'movewithdir' had a lot of related logic all around the 'mergecopies'.
However it is actually never containing anything until the very last loop in
that function. We move the (simplified) variable definition there for clarity
2016-10-11 02:15:23 +02:00
Mads Kiilerich
4ebd936629 cmdutil: satisfy expections in dirstateguard.__del__, even if __init__ fails
Python "delstructors" are terrible - this one because it assumed that __init__
had completed before it was called. That would not necessarily be the case if
the repository was read only or broken and saving the dirstate thus failed in
unexpected ways. That could give confusing warnings about missing '_active'
after failures.

To fix that, make sure all member variables are "declared" before doing
anything that possibly could fail. [Famous last words.]
2016-10-14 01:53:15 +02:00
Mads Kiilerich
39f2a13215 util: increase filechunkiter size to 128k
util.filechunkiter has been using a chunk size of 64k for more than 10 years,
also in years where Moore's law still was a law. It is probably ok to bump it
now and perhaps get a slight win in some cases.

Also, largefiles have been using 128k for a long time. Specifying that size
multiple times (or forgetting to do it) seems a bit stupid. Decreasing it to
64k also seems unfortunate.

Thus, we will set the default chunksize to 128k and use the default everywhere.
2016-10-14 01:53:15 +02:00
Mads Kiilerich
f926541834 largefiles: always use filechunkiter when iterating files
Before, we would sometimes use the default iterator over large files. That
iterator is line based and would add extra buffering and use odd chunk sizes
which could give some overhead.

copyandhash can't just apply a filechunkiter as it sometimes is passed a
genuine generator when downloading remotely.
2016-10-12 12:22:18 +02:00
Yuya Nishihara
7e790cf836 revset: for x^2, do not take null as a valid p2 revision
Since we don't count null p2 revision as a parent, x^2 should never return
null even if null is explicitly populated.
2016-10-14 23:33:00 +09:00
Yuya Nishihara
e0c2008a2f revset: make follow() reject more than one start revisions
Taking only the last revision is inconsistent because ancestors(set) follows
all revisions given, and theoretically follow(startrev=set) == ancestors(set).
I'm planning to add a support for multiple start revisions, but that won't fit
to the 4.0 time frame. So reject multiple revisions now to avoid future BC.

len(revs) might be slow if revs were large, but we don't care since a valid
revs should have only one element.
2016-10-10 22:30:09 +02:00
Gregory Szorc
d694855e6f bundle2: only emit compressed chunks if they have data
This is similar to 72dcaa40df76. Not all calls into the compressor
return compressed data, as the compressor may buffer compressed
output internally. It is cheaper to check for empty chunks than to
send empty chunks through the generator.

When generating a gzip-v2 bundle of the mozilla-unified repo, this
change results in 50,093 empty chunks not being sent through the
generator (out of 1,902,996 total input chunks).
2016-10-15 17:10:53 -07:00
Danek Duvall
5572888746 color: add some documentation for custom terminfo codes 2016-10-15 15:01:14 -07:00
Danek Duvall
3d2001514d color: debugcolor should emit the user-defined colors
This also fixes a long-standing bug that reversed the sense of the color
name and the label used to print it, which was never relevant before.
2016-10-13 13:10:01 -07:00
Danek Duvall
36e57eb638 color: ignore effects missing from terminfo
If terminfo mode is in effect, and an effect is used which is missing from
the terminfo database, simply silently ignore the request, leaving the
output unaffected rather than causing a crash.
2016-10-13 12:01:41 -07:00
Danek Duvall
fb575a9782 color: allow for user-configurable terminfo codes for effects
If the entry in the terminfo database for your terminal is missing some
attributes, it should be possible to create them on the fly without
resorting to just making them a color.  This change allows you to have

    [color]
    terminfo.<effect> = <code>

where <effect> might be something like "dim" or "bold", and <code> is the
escape sequence that would otherwise have come from a call to tigetstr().
If an escape character is needed, use "\E".  Any such settings will
override attributes that are present in the terminfo database.
2016-10-13 11:48:17 -07:00
Stanislau Hlebik
b83f0c0687 update: warn if cwd was deleted
During update directories are deleted as soon as they have no entries.
But if current working directory is deleted then it cause problems
in complex commands like 'hg split'. This commit adds a warning
that will help users figure the problem faster.
2016-10-04 04:06:48 -07:00
Gregory Szorc
34b225b38d parsers: avoid PySliceObject cast on Python 3
PySlice_GetIndicesEx() accepts a PySliceObject* on Python 2 and a
PyObject* on Python 3. Casting to PySliceObject* on Python 3 was
yielding a compiler warning. So stop doing that.

With this patch, I no longer see any compiler warnings when
building the core extensions for Python 3!
2016-10-13 13:34:53 +02:00
Gregory Szorc
5f9c903265 bdiff: include util.h
Without this, IS_PY3K isn't define and the preprocessor uses the
incorrect module loading code, causing the module fail to load at
run-time.

After this patch, all our C extensions (except for watchman's) appear
to import correctly in Python 3!
2016-10-13 13:27:14 +02:00
Gregory Szorc
b946a5f05c parsers: alias more PyInt* symbols on Python 3
I feel dirty for having to do this. But this is currently our approach
for dealing with PyInt -> PyLong in Python 3 for this file.

This removes a ton of compiler warnings by fixing unresolved symbols.
2016-10-13 13:22:40 +02:00
Gregory Szorc
05e74565d0 manifest: use PyVarObject_HEAD_INIT
More appeasing the Python 3 and compiler overlords. The code is
equivalent.
2016-10-13 13:17:23 +02:00
Gregory Szorc
b5f731482f dirs: use PyVarObject_HEAD_INIT
This makes a compiler warning go away on Python 3.
2016-10-13 13:14:14 +02:00
Martijn Pieters
217717bef0 py3: use namedtuple._replace to produce new tokens 2016-10-13 09:27:37 +01:00
Martijn Pieters
f29fa5e5a1 py3: refactor token parsing to handle call args properly
The token parsing was getting unwieldy and was too naive about accessing
arguments.
2016-10-14 17:55:02 +01:00
Pierre-Yves David
8f6b267699 eol: make sure we always release the wlock when writing cache
If any exception were to happen after we acquired the wlock, we could leave it
unreleased. We move the wlock release in a 'finally:' close as it should be.
2016-10-13 13:47:47 +02:00
Gregory Szorc
44818740cb pathencode: use assert() for PyBytes_Check()
This should have been added in 57bdf32c342e. I sent the patch to the
list prematurely.
2016-10-13 21:42:11 +02:00
Mads Kiilerich
ecb497971e merge: clarify warning for (not) merging flags without ancestor
Give hints why it can't merge and what it will do instead.
2016-10-12 12:22:18 +02:00
Mads Kiilerich
dc79a99eb6 merge: only show "cannot merge flags for %s" warning if flags are different 2016-10-12 12:22:18 +02:00
Mads Kiilerich
bb41b9551b tests: add test coverage of merging x flag without ancestor
It is more noisy than necessary - we will fix that later.
2016-10-12 12:22:18 +02:00
Gregory Szorc
a17dfc705a dirs: document Py_SIZE weirdness
Assigning to what looks like a function is clown shoes. Document that
it is a macro referring to a struct member.
2016-10-08 17:07:43 +02:00
Philippe Pepiot
fd197b867c record: return code from underlying commit 2016-10-12 12:22:54 +02:00
Philippe Pepiot
82dc121fd0 commit: return 1 for interactive commit with no changes (issue5397)
For consistency with non interactive commit
2016-10-14 09:52:38 +02:00
Mads Kiilerich
7f0aee28c1 demandimport: disable lazy import of __builtin__
Demandimport uses the "try to import __builtin__, else use builtins" trick to
handle Python 3. External libraries and extensions might do something similar.
On Fedora 25 subversion-python-1.9.4-4.fc25.x86_64 will do just that (except
the opposite) ... and it failed all subversion convert tests because
demandimport was hiding that it didn't have builtins but should use
__builtin__.

The builtin module has already been imported when demandimport is loaded so
there is no point in trying to import it on demand. Just always ignore both
variants in demandimport.
2016-10-14 03:03:39 +02:00
Gregory Szorc
0ee2ea3be0 changelog: disable delta chains
This patch disables delta chains on changelogs. After this patch, new
entries on changelogs - including existing changelogs - will be stored
as the fulltext of that data (likely compressed). No delta computation
will be performed.

An overview of delta chains and data justifying this change follows.

Revlogs try to store entries as a delta against a previous entry (either
a parent revision in the case of generaldelta or the previous physical
revision when not using generaldelta). Most of the time this is the
correct thing to do: it frequently results in less CPU usage and smaller
storage.

Delta chains are most effective when the base revision being deltad
against is similar to the current data. This tends to occur naturally
for manifests and file data, since only small parts of each tend to
change with each revision. Changelogs, however, are a different story.

Changelog entries represent changesets/commits. And unless commits in a
repository are homogonous (same author, changing same files, similar
commit messages, etc), a delta from one entry to the next tends to be
relatively large compared to the size of the entry. This means that
delta chains tend to be short. How short? Here is the full vs delta
revision breakdown on some real world repos:

Repo             % Full    % Delta   Max Length
hg                45.8       54.2        6
mozilla-central   42.4       57.6        8
mozilla-unified   42.5       57.5       17
pypy              46.1       53.9        6
python-zstandard  46.1       53.9        3

(I threw in python-zstandard as an example of a repo that is homogonous.
It contains a small Python project with changes all from the same
author.)

Contrast this with the manifest revlog for these repos, where 99+% of
revisions are deltas and delta chains run into the thousands.

So delta chains aren't as useful on changelogs. But even a short delta
chain may provide benefits. Let's measure that.

Delta chains may require less CPU to read revisions if the CPU time
spent reading smaller deltas is less than the CPU time used to
decompress larger individual entries. We can measure this via
`hg perfrevlog -c -d 1` to iterate a revlog to resolve each revision's
fulltext. Here are the results of that command on a repo using delta
chains in its changelog and on a repo without delta chains:

hg (forward)
! wall 0.407008 comb 0.410000 user 0.410000 sys 0.000000 (best of 25)
! wall 0.390061 comb 0.390000 user 0.390000 sys 0.000000 (best of 26)

hg (reverse)
! wall 0.515221 comb 0.520000 user 0.520000 sys 0.000000 (best of 19)
! wall 0.400018 comb 0.400000 user 0.390000 sys 0.010000 (best of 25)

mozilla-central (forward)
! wall 4.508296 comb 4.490000 user 4.490000 sys 0.000000 (best of 3)
! wall 4.370222 comb 4.370000 user 4.350000 sys 0.020000 (best of 3)

mozilla-central (reverse)
! wall 5.758995 comb 5.760000 user 5.720000 sys 0.040000 (best of 3)
! wall 4.346503 comb 4.340000 user 4.320000 sys 0.020000 (best of 3)

mozilla-unified (forward)
! wall 4.957088 comb 4.950000 user 4.940000 sys 0.010000 (best of 3)
! wall 4.660528 comb 4.650000 user 4.630000 sys 0.020000 (best of 3)

mozilla-unified (reverse)
! wall 6.119827 comb 6.110000 user 6.090000 sys 0.020000 (best of 3)
! wall 4.675136 comb 4.670000 user 4.670000 sys 0.000000 (best of 3)

pypy (forward)
! wall 1.231122 comb 1.240000 user 1.230000 sys 0.010000 (best of 8)
! wall 1.164896 comb 1.160000 user 1.160000 sys 0.000000 (best of 9)

pypy (reverse)
! wall 1.467049 comb 1.460000 user 1.460000 sys 0.000000 (best of 7)
! wall 1.160200 comb 1.170000 user 1.160000 sys 0.010000 (best of 9)

The data clearly shows that it takes less wall and CPU time to resolve
revisions when there are no delta chains in the changelogs, regardless
of the direction of traversal. Furthermore, not using a delta chain
means that fulltext resolution in reverse is as fast as iterating
forward. So not using delta chains on the changelog is a clear CPU win
for reading operations.

An example of a user-visible operation showing this speed-up is revset
evaluation. Here are results for
`hg perfrevset 'author(gps) or author(mpm)'`:

hg
! wall 1.655506 comb 1.660000 user 1.650000 sys 0.010000 (best of 6)
! wall 1.612723 comb 1.610000 user 1.600000 sys 0.010000 (best of 7)

mozilla-central
! wall 17.629826 comb 17.640000 user 17.600000 sys 0.040000 (best of 3)
! wall 17.311033 comb 17.300000 user 17.260000 sys 0.040000 (best of 3)

What about 00changelog.i size?

Repo                Delta Chains     No Delta Chains
hg                    7,033,250         6,976,771
mozilla-central      82,978,748        81,574,623
mozilla-unified      88,112,349        86,702,162
pypy                 20,740,699        20,659,741

The data shows that removing delta chains from the changelog makes the
changelog smaller.

Delta chains are also used during changegroup generation. This
operation essentially converts a series of revisions to one large
delta chain. And changegroup generation is smart: if the delta in
the revlog matches what the changegroup is emitting, it will reuse
the delta instead of recalculating it. We can measure the impact
removing changelog delta chains has on changegroup generation via
`hg perfchangegroupchangelog`:

hg
! wall 1.589245 comb 1.590000 user 1.590000 sys 0.000000 (best of 7)
! wall 1.788060 comb 1.790000 user 1.790000 sys 0.000000 (best of 6)

mozilla-central
! wall 17.382585 comb 17.380000 user 17.340000 sys 0.040000 (best of 3)
! wall 20.161357 comb 20.160000 user 20.120000 sys 0.040000 (best of 3)

mozilla-unified
! wall 18.722839 comb 18.720000 user 18.680000 sys 0.040000 (best of 3)
! wall 21.168075 comb 21.170000 user 21.130000 sys 0.040000 (best of 3)

pypy
! wall 4.828317 comb 4.830000 user 4.820000 sys 0.010000 (best of 3)
! wall 5.415455 comb 5.420000 user 5.410000 sys 0.010000 (best of 3)

The data shows eliminating delta chains makes the changelog part of
changegroup generation slower. This is expected since we now have to
compute deltas for revisions where we could recycle the delta before.

It is worth putting this regression into context of overall changegroup
times. Here is the rough total CPU time spent in changegroup generation
for various repos while using delta chains on the changelog:

Repo              CPU Time (s)    CPU Time w/ compression
hg                  4.50              7.05
mozilla-central   111.1             222.0
pypy               28.68             75.5

Before compression, removing delta chains from the changegroup adds
~4.4% overhead to hg changegroup generation, 1.3% to mozilla-central,
and 2.0% to pypy. When you factor in zlib compression, these percentages
are roughly divided by 2.

While the increased CPU usage for changegroup generation is unfortunate,
I think it is acceptable because the percentage is small, server
operators (those likely impacted most by this) have other mechanisms
to mitigate CPU consumption (namely reducing zlib compression level and
pre-generated clone bundles), and because there is room to optimize this
in the future. For example, we could use the nullid as the base revision,
effectively encoding the full revision for each entry in the changegroup.
When doing this, `hg perfchangegroupchangelog` nearly halves:

mozilla-unified
! wall 21.168075 comb 21.170000 user 21.130000 sys 0.040000 (best of 3)
! wall 11.196461 comb 11.200000 user 11.190000 sys 0.010000 (best of 3)

This looks very promising as a future optimization opportunity.

It's worth that the changes in test-acl.t to the changegroup part size.
This is because revision 6 in the changegroup had a delta chain of
length 2 before and after this patch the base revision is nullrev.
When the base revision is nullrev, cg2packer.deltaparent() hardcodes
the *previous* revision from the changegroup as the delta parent.
This caused the delta in the changegroup to switch base revisions,
the delta to change, and the size to change accordingly. While the
size increased in this case, I think sizes will remain the same
on average, as the delta base for changelog revisions doesn't matter
too much (as this patch shows). So, I don't consider this a regression.
2016-10-13 12:50:27 +02:00
Gregory Szorc
748ec42334 revlog: add instance variable controlling delta chain use
This is to support disabling delta chains on the changelog in a
subsequent patch.
2016-09-24 12:25:37 -07:00
Gregory Szorc
eb9f859c39 changegroup: document deltaparent's choice of previous revision
As part of debugging low-level changegroup generation, I came across
what I initially thought was a weird behavior: changegroup v2 is
choosing the previous revision in the changegroup as a delta base
instead of p1. I was tempted to rewrite this to use p1, as p1
will delta better than prev in the common case. However, I realized
that taking p1 as the base would potentially require resolving a
revision fulltext and thus require more CPU for e.g. server-side
processing of getbundle requests.

This patch tweaks the code comment to note the choice of behavior.
It also notes there is room for a flag or config option to tweak
this behavior later: using p1 as the delta base would likely make
changegroups smaller at the expense of more CPU, which could be
beneficial for things like clone bundles.
2016-10-13 12:49:47 +02:00
Pierre-Yves David
d482e52866 help: backout 6f89f03ad369 (mark boolean flags with [no-] in help) for now
The ability to negate any boolean flags itself is great, but I think we are not
ready to expose the help side of it yet.

First, while there exist a handful of such flags whose default value can be
changed (eg: git diff, patchwork confirmation), there is only a few of them. The
users who benefit the most from this change are alias users and large
installation that can deploy extension to change behavior (eg: facebook
tweakdefault).  So the majority of user who will be affected by a large change
to command help that is not yet relevant to them. (I expect this to become
relevant when ui.progressive start to exists).

Below is an example of the impact of the new help on 'hg help diff':

  -r --rev REV [+]              revision
  -c --change REV               change made by revision
  -a --[no-]text                treat all files as text
  -g --[no-]git                 use git extended diff format
     --[no-]nodates             omit dates from diff headers
     --[no-]noprefix            omit a/ and b/ prefixes from filenames
  -p --[no-]show-function       show which function each change is in
     --[no-]reverse             produce a diff that undoes the changes
  -w --[no-]ignore-all-space    ignore white space when comparing lines
  -b --[no-]ignore-space-change ignore changes in the amount of white space
  -B --[no-]ignore-blank-lines  ignore changes whose lines are all blank
  -U --unified NUM              number of lines of context to show
     --[no-]stat                output diffstat-style summary of changes
     --root DIR                 produce diffs relative to subdirectory
  -I --include PATTERN [+]      include names matching the given patterns
  -X --exclude PATTERN [+]      exclude names matching the given patterns
  -S --[no-]subrepos            recurse into subrepositories

Another issue with the current state of help, the default value for the
flag is not conveyed to the user. For example in the 'backout' help, there is
no real distinction between "--[no-]backup" (default to True) and "--[no-]keep"
(default) to False:

  --[no-]backup        no backups
  --[no-]keep          do not modify working directory during strip

In addition, I've discussed with Augie Fackler and the last batch of the work on
this have burned him out quite some. Therefore he is not intending to perform
any more work on this topic. Quoting him, he would rather see the help part
backed out than spending more time on it.

I do not think we are ready to expose this to users in 4.0 (freeze in a week),
especially because we cannot expect quick improvement on these aspect as this
topic no longer have an owner. We should be able to reintroduce that change in
the future when someone get back on it and the main issues are solves:

* Introduction of  ui.progressive makes it relevant for a majority of user,
* Current default value are efficiently conveyed to the user.

(In addition, the excerpt from diff help show that we still have some issue with
some negative option like '--nodates' so further improvement are probably
welcome there.)
2016-10-09 03:11:18 +02:00
Augie Fackler
530be6f190 copy: distinguish "file exists" cases and add a hint (BC)
Users that want to add a copy record to an existing commit with 'hg
commit --amend' should be guided towards this workflow, rather than
reaching for some sort of uncommit-recommit flow. As part of this,
distinguish in the top-line error message whether the file merely
already exists (untracked) on disk or the file already exists in
history.

The full list of copy and rename cases and how they interact with
flags are listed below:

target exists  --after  --force  |  action
      n            n      *    |  copy
      n            y      *    |  (1)
  untracked        n      n    |  (4) NEWHINT
  untracked        n      y    |  (3)
  untracked        y      *    |  (2)
      y            n      n    |  (4) NEWHINT
      y            n      y    |  (3)
      y            y      n    |  (2)
      y            y      y    |  (3)
   deleted         n      n    |  copy
   deleted         n      y    |  (3)
   deleted         y      n    |  (1)
   deleted         y      y    |  (1)

* = don't care
(1) <src>: not recording move - <target> does not exist
(2) preserve target contents
(3) replace target contents
(4) <target>: not overwriting - file {exists,already committed}

Credit to Kevin for wholly rewriting my table to cover more cases we
discovered at the sprint.

I think this change gets the hints correct in all cases, but I'd
appreciate close inspection of the test cases to make sure I haven't
gotten turned around in here.
2016-09-19 17:15:39 -04:00
FUJIWARA Katsunori
fc0b0c4b06 perf: make perftags clear tags cache correctly
Before this patch, "hg perftags" command doesn't measure performance
of "repo.tags()" correctly, because it doesn't clear tags cache
correctly.

a043ed82a5cd replaced repo._tags with repo._tagscache, but didn't
change the code path to clear tags cache in perftags() at that time.
BTW, full history of "tags cache" is:

  - b8d757d45f24 (or 0.6) introduced repo.tagscache as the first "tags cache"
  - 4cbf51c74e8c (or 1.4) replaced repo.tagscache with repo._tags
  - a043ed82a5cd (or 2.0) replaced repo._tags with repo._tagscache
  - 04c204f1ed65 (or 2.5) made repo._tagscache filteredpropertycache

To make perftags clear tags cache correctly, and to increase
"historical portability" of perftags, this patch examines existence of
attributes in repo object, and guess appropriate procedure to clear
tags cache.

To avoid examining existence of attributes at each repetition, this
patch makes repocleartagscachefunc() return the function, which
actually clears tags cache.

mozilla-central repo (85 tags on 308365 revs) with each Mercurial
version between before and after this patch.

  ==== ========= =========
  ver  before    after
  ==== ========= =========
  1.9  0.476062  0.466464
       ------- *1 -------
  2.0  0.346309  0.458327
  2.1  0.343106  0.454489
       ------- *2 -------
  2.2  0.069790  0.071263
  2.3  0.067829  0.069340
  2.4  0.068075  0.069573
       ------- *3 -------
  2.5  0.021896  0.022406
  2.6  0.021900  0.022374
  2.7  0.021883  0.022379
  2.8  0.021949  0.022327
  2.9  0.021877  0.022330
  3.0  0.021860  0.022314
  3.1  0.021869  0.022669
  3.2  0.021831  0.022668
  3.3  0.021809  0.022691
  3.4  0.021861  0.022916
  3.5  0.019335  0.020749
  3.6  0.019319  0.020866
  3.7  0.018781  0.020251
       ------- *4 -------
  3.8  0.068262  0.072558
  3.9  0.069682  0.073773
  ==== ========= =========

(*1) repo._tags was replaced with repo._tagscache at this point

     "repo._tags = None" in perftags "before" this patch doesn't clear
     tags cache for Mercurial 2.0 or later. This causes significant
     gap of "before" between 1.9 and 2.0 .

(*2) I'm not sure about significant gap at this point, but release
     note of 2.2 described "a number of significant performance
     improvements for large repositories"

(*3) filtered changelog was cached in repoview as repoview.changelog
     at this point (by 131b01a4654d)

     This avoids calculation of filtered changelog at each repetition
     of t().

(*4) calculation of filtered changelog was included into wall time at
     this point (by adf01efe43a5), again

     See below for detail about this significant gap:

     https://www.mercurial-scm.org/pipermail/mercurial-devel/2016-April/083410.html
2016-10-09 01:03:20 +09:00
FUJIWARA Katsunori
37e273ea35 perf: replace ui.configint() by getint() for Mercurial earlier than 1.9
Before this patch, using ui.configint() prevents perf.py from
measuring performance with Mercurial earlier than 1.9 (or
12e7e9fbf243), because ui.configint() isn't available in such
Mercurial, even though there are some code paths for Mercurial earlier
than 1.9 in perf.py.

For example, setting "_prereadsize" attribute in perfindex() and
perfnodelookup() is effective only with hg earlier than 1.8 (or
1299f0c14572).

This patch replaces ui.configint() invocations by newly introduced
getint().

This patch also adds check-perf-code.py an extra check entry to detect
direct usage of ui.configint() in perf.py.

BTW, this patch doesn't choose adding configint() method at runtime by
replacing ui.__class__ like below, even though this is the recommended
way to modern Mercurial extensions.

    def uisetup(ui):
        if not util.safehasattr(ui, 'configint'):
            class uiwrap(ui.__class__):
                def configint(self, section, name, ....):
                    ....
            ui.__class__ = uiwrap

Because changes to ui.__class__ by uisetup() of loaded extension have
been propagated since 1.6.1 (or 07a6e7bd0cc1), the recommended way
above doesn't work as expected with Mercurial earlier than it.
2016-10-09 01:03:19 +09:00
FUJIWARA Katsunori
87f398d18d perf: omit copying from ui.ferr to ui.fout for Mercurial earlier than 1.9
Before this patch, referring ui.ferr prevents perf.py from measuring
performance with Mercurial earlier than 1.9 (or bac01d164cc1), because
ui.ferr isn't available in such Mercurial, even though there are some
code paths for Mercurial earlier than 1.9 in perf.py.

For example, setting "_prereadsize" attribute in perfindex() and
perfnodelookup() is effective only with hg earlier than 1.8 (or
1299f0c14572).
2016-10-09 01:03:19 +09:00
FUJIWARA Katsunori
31be956823 perf: define formatter locally for Mercurial earlier than 2.2
Before this patch, using ui.formatter() prevents perf.py from
measuring performance with Mercurial earlier than 2.2 (or
045c8375c770), because ui.formatter() isn't available in such
Mercurial, even though there are some code paths for Mercurial earlier
than 2.2 in perf.py.

For example, setting "_prereadsize" attribute in perfindex() and
perfnodelookup() is effective only with hg earlier than 1.8 (or
1299f0c14572).

This patch defines formatter class locally, and use it instead of the
value returned by ui.formatter(), if perf.py is used with Mercurial
earlier than 2.2.

In this case, we don't need to think about -T/--template option for
formatter, because previous patch made -T/--template disabled for
perf.py with Mercurial earlier than 3.2 (or 44a82ed65df7).
2016-10-09 01:03:18 +09:00
FUJIWARA Katsunori
dd96bd471b perf: add functions to get vfs-like object for Mercurial earlier than 2.3
Before this patch, using svfs prevents perf.py from measuring
performance of Mercurial earlier than 2.3 (or 12df7401e8cd), because
svfs isn't available in such Mercurial, even though there are some
code paths for Mercurial earlier than 2.3 in perf.py.

For example, setting "_prereadsize" attribute in perfindex() and
perfnodelookup() is effective only with hg earlier than 1.8 (or
1299f0c14572).

To get appropriate vfs-like object to access files under .hg/store,
this patch adds getsvfs() (and also getvfs(), for future use).

To avoid examining existence of attribute at each repetition while
measuring performance, getsvfs() is invoked outside the function to be
called repeatedly.

This patch also adds check-perf-code.py an extra check entry to detect
direct usage of repo.(vfs|svfs|opener|sopener) in perf.py.
2016-10-09 01:03:18 +09:00
FUJIWARA Katsunori
956b59a347 perf: avoid actual writing branch cache out correctly
Mercurial 2.5 (or 0eb7dcc721cb) introduced "perfbranchmap" command,
and tried to avoid actual writing branch cache out by replacing
write() of branchcache class in branchmap.py with no-op function
(probably, for elimination of noisy and heavy file I/O factor).

But its implementation isn't correct, because 0eb7dcc721cb replaced
not branchmap.branchcache.write() but branchmap.write(). The latter
doesn't exist, even at that change.

To avoid actual writing branch cache out correctly, this patch
replaces branchmap.branchcache.write() with no-op function.

To detect mistake of replacement or change of API in the future
quickly, this patch uses safeattrsetter() instead of direct attribute
assignment. For similarity between replacements, this patch also
changes replacement of branchmap.read().

In this patch, replacement of read()/write() can run safely outside
"try" block, because two safeattrsetter() invocations ensure that
replacement doesn't cause exception.

FYI, the table below compares "base" filter wall time of perfbranchmap
on recent mozilla-central repo with each Mercurial version between
before and after this patch.

  ==== ========= =========
  ver  before    after
  ==== ========= =========
  2.5  18.492334 18.232455
  2.6  18.733858 18.156702
  2.7  18.245598 18.349210
  2.8  18.289070 18.528422
  2.9  17.572742 16.989655
  3.0  17.406953 17.615012
  3.1  17.228419 17.689805
  3.2  17.862961 17.718367
  3.3   2.632110  2.707960
  3.4   3.285683  3.272060
  3.5   3.370141  3.352176
  3.6   3.366939  3.242455
  3.7   3.300778  3.367328
  3.8   3.300132  3.267298
  3.9   3.418996  3.370265
  ==== ========= =========

IMHO, there is no serious overlooking performance regression.
2016-10-09 01:03:17 +09:00
FUJIWARA Katsunori
9d79ac8dbc perf: get subsettable from appropriate module for Mercurial earlier than 2.9
Before this patch, using branchmap.subsettable prevents perfbranchmap
from measuring performance of Mercurial earlier than 2.9 (or
aad678a92970), because aad678a92970 moved subsettable from repoview.py
to branchmap.py, even though there are some code paths for Mercurial
earlier than 2.9 in perf.py.

For example, setting "_prereadsize" attribute in perfindex() and
perfnodelookup() is effective only with hg earlier than 1.8 (or
1299f0c14572).

To get subsettable from appropriate module, this patch examines
existence of subsettable in branchmap and repoview.

This patch also adds check-perf-code.py an extra check entry to detect
direct usage of subsettable attribute in perf.py.
2016-10-09 01:03:17 +09:00
FUJIWARA Katsunori
0f1711011f perf: introduce safeattrsetter to replace direct attribute assignment
Referring not-existing attribute immediately causes failure, but
assigning a value to such attribute doesn't.

For example, perf.py has code paths below, which assign a value to
not-existing attribute. This causes incorrect performance measurement,
but these code paths are executed successfully.

  - "repo._tags = None" in perftags()
    recent Mercurial has tags cache information in repo._tagscache

  - "branchmap.write = lambda repo: None" in perfbranchmap()
    branchmap cache is written out by branchcache.write() in branchmap.py

"util.safehasattr() before assignment" can avoid this issue, but might
increase mistake at "copy & paste" attribute name or so.

To centralize (1) examining existence of, (2) assigning a value to,
and (3) restoring an old value to the attribute, this patch introduces
safeattrsetter(). This is used to replace direct attribute assignment
in subsequent patches.

Encapsulation of restoring is needed to completely remove direct
attribute assignment from perf.py, even though restoring isn't needed
so often.
2016-10-09 01:03:16 +09:00
Mads Kiilerich
7afa73604d largefiles: use context for file closing
Make the code slightly smaller and safer (and more deeply indented).
2016-10-08 00:59:41 +02:00
Mads Kiilerich
73724460ec largefiles: when setting/clearing x bit on largefiles, don't change other bits
It is only the X bit that it matters to copy from the standin to the largefile
in the working directory. While it generally doesn't do any harm to copy the
whole mode, it is also "wrong" to copy more than the X bit we care about. It
can make a difference if someone should try to handle largefiles differently,
such as marking them read-only.

Thus, do similar to what utils.setflags does and set the X bit where there are
R bits and obey umask.
2016-10-08 00:59:40 +02:00
Mads Kiilerich
cb6727b739 eol: on update, only re-check files if filtering changed
Before, update would mark all files as 'normallookup' in dirstate if .hgeol
changed so all files would get the new filtering applied. That takes some time
... and is pointless if the filtering for that file didn't change.

Instead, keep track of the old filtering and only check files where the
filtering is changed.

To keep the old filtering, change to write the applied .hgeol content to
.hg/eol.cache instead of just touching it. That change is backwards/forwards
compatible.

In a real world test, this takes an update that is changing .hgeol and 30000
files from 12s to 4s - where the remaining eol overhead is 1-2s.
2016-10-09 15:54:49 +02:00
Gregory Szorc
48ea4c11ea dirs: add comment about _PyBytes_Resize
So readers have a canonical function to compare this code to.
2016-10-13 10:59:29 +02:00
Pierre-Yves David
163070ae3d checkcopies: extract the '_related' closure
There is not need for it to be a closure.
2016-10-11 01:29:08 +02:00
Pierre-Yves David
2d670597fe checkcopies: add an inline comment about the '_related' call
This helps understanding the flow of the function.
2016-10-08 23:00:55 +02:00
Pierre-Yves David
71b0c4ef9c checkcopies: minor change to comment
This helped me understand the refactoring so this must be helpful.
2016-10-08 19:03:16 +02:00
Pierre-Yves David
1f966aa892 checkcopies: rename 'ca' to 'base'
This variable was named after the common ancestor. It is actually the merge
base that might differ from the common ancestor in the graft case. We rename the
variable before a larger refactoring to clarify the situation.
2016-10-08 18:38:42 +02:00
Pierre-Yves David
023c7518b5 bisect: extra a small initialisation outside of a loop
Having initialisation done during the first iteration is cute, but can be
avoided.
2016-08-24 05:09:46 +02:00