Before this changesets, a literal '.*:foo$' config would match a registered
'.*:foo$' generic. This is wrong since generic should be matched through
regular exception only. This changeset fixes this problem.
Thanks for to Yuya Nishihara for spotting the issue.
When the list of dates is empty, `min` and `max` would raises a ValueError.
Protect against this case by checking that the date list is not empty.
I didn't add a test because I couldn't find a reproducing test case.
Grouping all evolution related-config under the experimental.evolution
namespace would helps the future migration outside [experimental].
Differential Revision: https://phab.mercurial-scm.org/D1155
Stabilization config items were never part of a release, remove them now that
we cleaned up the evolution related configuration.
Differential Revision: https://phab.mercurial-scm.org/D1154
Update the evolution helpers function to support both old-style configuration and
new-style configuration:
experimental.evolution=all is renamed into experimental.evolution=true
experimental.evolution=createmarkers is renamed into
experimental.evolution.createmarkers=true
experimental.evolution=allowunstable is renamed into
experimental.evolution.allowunstable=true
experimental.evolution=exchange is renamed into
experimental.evolution.exchange=true
We choose to not rename individual config options; keeping the same names
would easy the transition for users but it's something that could be easily
done in the future.
Differential Revision: https://phab.mercurial-scm.org/D1147
We want to split the evolution-related configuration and back-out the renaming
from evolution.* to stabilization.*.
First invert the configuration and aliases, so next changesets will be
cleaner.
Differential Revision: https://phab.mercurial-scm.org/D1146
# skip-blame because parsers.c is mechanically rewritten by
clang-format with no semantic change.
Differential Revision: https://phab.mercurial-scm.org/D1170
This gives clang-format the right notion about formatting these struct
initializers, therefore allowing us to automatically format several
additional files.
# skip-blame because this is just a content-free comment addition
Differential Revision: https://phab.mercurial-scm.org/D1169
The original filectx API is kept public since we'll need it to walk ancestor
(rev, match) pairs efficiently. The current implementation scans ancestors
twice for 'hg log -fp FILE'.
This is in preparation to switch this class to inheriting (and being based off
a) `commitctx` instead of a `workingctx`. `filectx` has no `exists` function,
so this is how we'll fall back in that case.
Differential Revision: https://phab.mercurial-scm.org/D1237
With rebase, we will be setting the _wrappedctx at a different point from the
wctx construction (somewhat later, and possibly several times). Move it to a
public function.
Differential Revision: https://phab.mercurial-scm.org/D1231
This was intended to be done by D470. But there was a minor documentation
issue. The feature is quite usable now so it gets formally documented and
enabled.
There is no behavior change for people not using the `SRC` or `ALLSRC` in
rebase destination revset.
.. feature:: Rebase with different destination per source revision
Previously, rebase only supports one unique destination. Now ``SRC`` and
``ALLSRC`` can be used in rebase destination revset to precisely define
destination per each individual source revision.
For example, the following command could move some orphaned changesets to
reasonable new places so they become no longer orphaned::
hg rebase
-r 'orphan()-obsolete()'
-d 'max((successors(max(roots(ALLSRC) & ::SRC)^)-obsolete())::)'
Differential Revision: https://phab.mercurial-scm.org/D1063
Merge conflicts might be supported in the future, but for now are kept out of
scope.
Any places where we used to call `flushall()` should be replaced with some kind
of exception. At this point, IMM M1 is no longer supported.
Differential Revision: https://phab.mercurial-scm.org/D1212
I added `ctx()` to `overlayworkingfilectx`, (and before that, `absentfilectx`),
because `absentfilectx` had reference to this function in its `cmp()` function.
But the standard is actually `changectx()`, and no other class implements
`ctx()`. So let's use the standard name.
(As a result, I'm not sure that part of the `absentfilectx` comparator ever
worked! It was written before I added either function.)
This will be necessary in the next patch.
Differential Revision: https://phab.mercurial-scm.org/D1211
The `diff' command usually writes deletion in red and insertions in green. This
patch adds within-line colors, to highlight which part of the lines differ.
Lines to compare are decided based on their similarity ratio, as computed by
difflib SequenceMatcher, with an arbitrary threshold (0.7) to decide at which
point two lines are considered entirely different (therefore no inline-diff
required).
The current implementation is kept behind an experimental flag in order to test
the effect on performance. In order to activate it, set inline-color-diff to
true in [experimental].
We can't use fctx.linkrev() to sort fctxs coming from multiple files.
This was changed at a5b8b1052ef6 due to performance issue, but we know
we evaluate parent.rev() in revset anyway.
The primary goal of this series is to make follow() support multiple start
revisions.
dagop.filectxancestors() will be extended to take multiple filectxs.
basefilectx.ancestors() is not forwarded to this function because doing that
would resurrect the performance issue fixed by a5b8b1052ef6.
This makes pull consistent with the part used by push and provide us with a
more compact representation of bookmarks.
In addition, this opens the way for smarter bookmark exchanges (e.g. filtering
by names or only sending the bookmark relevant to the pulled set, etc).
In this mode, the bookmarks changes are record in the 'bundleoperation' records
instead of inflicted to the repository. This is necessary to use the part when
pulling.
This new attribute allows the codes requesting an unbundling to pass important
information to individual part handlers. The current target use case is to
allow for receiving 'bookmarks' part without directly updating local
repository, but just recording the received data instead. This is necessary
for pull where the remote bookmarks are processed locally. I expect the
concept to be beneficial to other parts in the future.
To clarify the bookmark behavior on pull, the remote bookmark value are not just
taken -as-is- into the local repository. There is an extra step to detect
bookmark divergence. The remote bookmarks data are stored until this processing
happens.
We use the new binary parts we introduced earlier to exchange bookmark. The
payload is a bit more compact since we use binary and the length of bookmarks
is no longer constrained to 255.
.. fix:: Issue 5165
Bookmark, whose name is longer than 255, can again be exchanged again
between 4.4+ client and servers.
Currently, pushing a bookmark update triggers a pushkey hooks. It is likely
that users in the wild use such hooks to control bookmark movement. Using a non
push-key mechanism to exchange bookmark means these hooks are no longer called,
possibly breaking existing users setup. So we add explicit call to the pushkey
hooks in the handling of the bundle2 part. This behavior can be disabled with a
new config knob: 'server.bookmarks-pushkey-compat'.
This part can carry and apply bookmarks information. We start with adding the
core behavior of the part. In its current form, the part is only suitable for
push since it plain update the bookmark without consideration for the local
state. Support of the behavior needed for pulling will be added in later
changesets.
Before updating the actual bookmark update, we can start with updating the way
we check for push race. Checking bookmarks state earlier is useful even if we
still use pushkey. Aborting before the changegroup is added can save a lot of
time.
This part checks that bookmarks are still at the node they are expected to be.
This allows a pushing client to detect push race where the repository was
updated between the time it discovered the server state and the time it managed
to finish its push.
Such checking already exists when pushing bookmark through pushkey. This new
part can be inserted at the beginning of the bundle, triggering abort earlier.
In addition, we would like to move away from pushey to push bookmark. A step
useful to solve issue5165.
Coming new bundle2 parts related to bookmark will use a binary encoding. It
encodes a series of '(bookmark, node)' pairs. Bookmark name has a high enough
size limit to not be affected by issue5165. (64K length, we are well covered)
Just to give some context to the return values: vertex() needs to return two
HTML elements as strings, <li> to be used as a background and a <li> to be
shown in foreground. The latter was made obsolete recently when changesets
started to be rendered server-side, but background elements are still useful
for now.
This makes sure that dynamically-created class objects are isolated from
local binding of repo instances. The type cache is moved to module level
as it isn't tied to each instance.
This patch adds a new file which will contain utility functions related to
rewritting changesets. It also adds a utility function to check if the
rewritting operation creates new unstable changesets and are we allowed to
create them.
This rewriteutil.py introduced in this patch and the utility functions added in
the upcoming patches exists in the evolve extension are being ported from there.
Differential Revision: https://phab.mercurial-scm.org/D1502
This patch functions which can be used to read remotenames data from
.hg/remotenames/. The logic for the function which reads the remotenames file is
taken from the remotenames extension.
Previously reviewed as D940.
Differential Revision: https://phab.mercurial-scm.org/D1550
This patch moves the functionality from remotenames extension to store
remotenames to core.
Storage format used by remotenames extension:
A single file `.hg/remotenames` with an entry in each line where each line is of
format:
`node nametype remotepath/name`
where nametype is either 'bookmarks' or 'branches'.
This was not the best way to store data, so while moving to core the storage
format was changed but yet not the final format. The storage format used by core
after this patch will be:
* A file for each type of name i.e. bookmarks and branches in .hg/remotenames/
directory
* A version number on the top of the file. The version for current format is 0.
* An entry in each line where each line is of the format
`node\0remotepath\0name`
The logic to sync with existing remotenames file and saving journals and other
related things will be moved to core in next patches incrementally.
Thanks to Ryan, Augie and Durham for suggestions on storage format.
Previously reviewed as D939.
Differential Revision: https://phab.mercurial-scm.org/D1548
This patch is the first patch of the series moving functionality from
hgremotenames extension to core.
There are lot of functionality in the extension which in the end enables us to
store branch heads and bookmarks location on a server from which we are pulling
or cloning from. This will help us in creating a better bookmark workflow where
we can show user that a certain server has this bookmarks at this node. It will
also introduce namespaces related to remote bookmarks and remote branches.
This patch moves the functionality to pull branches and bookmarks from a
server from which we are pulling to core behind config option
`experimental.remotenames`.
This patch adds a test which helps us to analyse whether things are working or
not. We are currently writing things to ui, we will write information to files
in upcoming patches.
Previously reviewed as D937.
Differential Revision: https://phab.mercurial-scm.org/D1547
This prepares for replacing old _early*opt() functions. My initial attempt
was to extend options table to support 'repository|repo' syntax. It worked,
but seemed too invasive. So I decided to add an optional argument to
fancyopts() instead.
This also changes the nevernegate dict to be keyed by a canonical_name,
not by an option-name for clarity.
Historically, client-side graph code was not only rendering the graph itself,
but it was also adding all of the changeset information to the page as well.
It meant that JavaScript code needed to construct valid HTML as a string
(although proper escaping was done server-side). It wasn't too clunky, even
though it meant that a lot of server-side things were duplicated client-side
for no good reason, but the worst thing about it was the data format it used.
It was somewhat future-proof, but not human-friendly, because it was just a
tuple: it was possible to append things to it (as was done in e.g.
4d7cfa1867b5), but you'd then have to remember the indices and reading the
resulting JS code wasn't easy, because cur[8] is not descriptive at all.
So what would need to happen for graph to have more features, such as more
changeset information or a different vertex style (branch-closing, obsolete)?
First you'd need to take some property, process it (e.g. escape and pass
through templatefilters function, and mind the encoding too), append it to
jsdata and remember its index, then go add nearly identical JavaScript code to
4 different hgweb themes that use jsdata to render HTML, and finally try and
forget how brittle it all felt. Oh yeah, and the indices go to double digits if
we add 2 more items, say phase and obsolescence, and there are more to come.
Rendering vertex in a different style would need another property (say,
character "o", "_", or "x"), except if you want to be backwards-compatible, it
would need to go after tags and bookmarks, and that just doesn't feel right.
So here I'm trying to fix both the duplication of code and the data format:
- changesets will be rendered by hgweb templates the same way as changelog and
other such pages, so jsdata won't need any information that's not needed for
rendering the graph itself
- jsdata will be a dict, or an Object in JS, which is a lot nicer to humans and
is a lot more future-proof in the long run, because it doesn't use numeric
indices
What about hgweb themes? Obviously, this will break all hgweb themes that
render graph in JavaScript, including 3rd-party custom ones. But this will also
reduce the size of client-side code and make it more uniform, so that it can be
shared across hgweb themes, further reducing its size. The next few patches
demonstrate that it's not hard to adapt a theme to these changes. And in a
later series, I'm planning to move duplicate JS code from */graph.tmpl to
mercurial.js and leave only 4 lines of code embedded in those <script>
elements, and even that would be just to allow redefining graph.vertex
function. So adapting a custom 3rd-party theme to these changes would mean:
- creating or copying graphnode.tmpl and adding it to the map file (if a theme
doesn't already use __base__)
- modifying one line in graph.tmpl and simply removing the bigger part of
JavaScript code from there
Making these changes in this patch and not updating every hgweb theme that uses
jsdata at the same time is a bit of a cheat to make this series more
manageable: /graph pages that use jsdata are broken by this patch, but since
there are no tests that would detect this, bisect works fine; and themes are
updated separately, in the next 4 patches of this series to ease reviewing.
ajaxScrollInit is a function that loads more elements (e.g. changelog entries)
when browser window is scrolled down to the bottom of the page. It basically
fetches the next page from the server as HTML, finds container element in that
document and "adopts" (essentially, moves) all its child nodes to the container
in the current document.
Currently, hgweb doesn't render any changesets on /graph page (everything is
done in JavaScript), so there are no children to adopt. But there will be soon,
so let's create a reusable function that does it.
Hardcoding #graphnodes selector is suboptimal, but graph code already does this
in two other places.
This makes sure that all columns are aligned. getlogcolumns() is hosted by
templatekw so the namespaces module can see it.
i18n/de.po is updated so test-log.t passes with no error. "obsolete:" and
"instability:" are kept untranslated.
We add a new 'ui.timeout.warn' config to set a grace period before we display
lock related warning:
waiting for lock on PATH held by PROCESS
The config is based on 'ui.timeout' and expresses a number of seconds before
the warning is displayed. Negative values disable the warning altogether.
The messages go to the debug output to help people trouble-shooting deadlocks.
We are about to make the messages around lock more flexible. We move all the
currently logic into a function in the lock module. We'll update the message
scheme in the next changeset.
Without this files in the output archive directory have their mtimes set to
whatever time they were written their. This is in this inconsistent with
the other archivers, eg. zip, which use exactly the same time for all files.
Works on my machine (Windows), but I don't have a *nix box available to run
anything more thorough, unfortunately.
"top: -Xpx" shifts a block up by X pixels, which can be used to visually
compress two lines of text to have less space between them, in this case it's
used for the changesets on /graph page. But not only it's not needed there
(both lines fit fine into their allowed vertical space), but it would also look
better (not as crammed, more vertically centered) without these negative
values.
"position: relative" is needed solely for the "top" property to have effect on
the element, no children of the .info element rely on it, so let's remove it as
well.
When the option is set, the repository will reject any transaction adding
multiple heads to the same named branch.
For now we reject all scenario with multiple heads. One could imagine handling
closed branches differently. We prefer to keep things simple for now. The
feature might get extended later. Branch closing is not the best experience
Mercurial has to offer anyway.
I'd also like --color=always on the command-line to override HGPLAIN=1
et al, but that's more work, and this seems like a better fix. We've
got a fair number of programs that actually want to automate hg and
get colored output to users, so they should set HGPLAINEXCEPT=alias
(what we usually recommend), but this has been breaking them because
they then lose color.
.. feature::
The ``HGPLAINEXCEPT`` environment variable can now include ``color``
to allow automatic output colorization in otherwise automated
environments.
Differential Revision: https://phab.mercurial-scm.org/D1532
The next patch will wrap the conversion code, in order to write out a
requirement for 'lfs' when appropriate. Wrapping convcmd.convertsink() in an
afterloaded callback works fine when the convert extension is enabled by the
user. The problem here is that lfconvert uses the convert extension, whether or
not it was formally enabled by the user.
My first attempt was to have lfs install an afterloaded callback that would wrap
the convert sink if convert was loaded, or wrap lfconvert if it wasn't. Then
the lfconvert override could install an afterloaded callback to try wrapping the
convert sink again, before calling the original lfconvert. But that breaks down
if largefiles can't load the convert extension on the fly. [1] Further, some
tests were failing with an error indicating that the size of the afterloaded
list changed while iterating it.
Yuya mentioned that maybe some bits of convert could be moved into core, but I'm
not sure where to draw that line. The convertsink() method depends on the list
of sinks, which in turn depends on the sink classes.
[1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-November/108038.html
This was added by cb39606d374e, but core matchers support visitdir() of
arbitrary locations since a4236180df5e, and verifier._verifymanifest()
doesn't seem to strictly obey the restriction.
I have no idea how important this API contract is for third-party extensions.
That's why this patch is RFC.
Unfortunately, current version of jshint (2.9.5) doesn't know such a global
variable and complains that it's undefined. Since this line tries to look up
URLSearchParams in a global scope (i.e. window), let's simply preface it with
"window." to work around jshint.
Variables that are used or assigned without any declaration using var (or let,
or const) are considered global. In many cases this is inadvertent and actually
causes a variable leaking to a broader scope, such as a temporary variable used
inside a loop suddenly being accessible in global scope. (This corresponds to
"undef" option of jshint).
So this patch limits the scope of variables that don't need to be global. There
are a lot of helper variables in Graph.render() used in a loop, I've declared
them all on one line to reduce patch size. "radius" is special because it
wasn't passed to graph.vertex, but was used there (it worked because this
variable leaked to global scope). "window.graph" is created by an inline script
in graph.tmpl so that it can be used in ajaxScrollInit() function, this patch
makes this fact explicit by assigning window.graph to a local variable.
"xhr" is a really widespread name for this kind of things, no idea where did
"xfr" come from (the original fbf9645839e4 doesn't explain that). Let's just
change one letter so the name makes more sense.
In JavaScript, using for-in loops to access every property of an object can
have unexpected results when inheritance is involved. For example, if some
piece of code adds a property (it may be a method too) to Object.prototype,
then all for-in loops that iterate over keys of any object (also anything that
inherits Object) will get that property on one of the iterations. To filter out
such unexpected properties for-in loops have to use Object.hasOwnProperty()
method. (This corresponds to "forin" option of jshint).
In the two first cases "data" and "edges" are arrays, to it's simpler to just
switch to using a regular for-with-a-counter loop.
This patch changes "==" (equals operator) to "===" (strict equals operator).
The difference between them is that the latter doesn't do any type coercions.
It's handy to compare string '1' to number 1 sometimes, but most of the time
using "==" is inadvertent and can be replaced by an explicit type conversion.
(This corresponds to "eqeqeq" option of jshint).
Some of the changes in this patch are straightforward, e.g. when comparing
results of typeof (they could only be strings). The same goes for 'none' and
similar strings that can't be sensibly coerced to some other type. Two changes
that compare values to "1" and "0" can be clarified: getAttribute() returns
either a string or null, but comparing null to a string is always false, so no
logic is lost.
The first hunk had a non-breaking space character just before "{", it's not an
error or anything, but let's fix it while we're at it. (This corresponds to
"nonbsp" option of jshint).
Hunks 2 and 3 change "==" (equals operator) to "===" (strict equals operator).
The difference between them is that the latter doesn't do any type coercions.
It's handy to compare string '1' to number 1 sometimes, but most of the time
using "==" is inadvertent and can be replaced by an explicit type conversion.
(This corresponds to "eqeqeq" option of jshint).
Most of this file already uses strict equals operator, and in the code affected
type coercion is not needed, because tagName and selectableTag are both strings
and endId and startId are both numbers.
mercurial.js has a process_dates() function that calculates relative age for a
given date, it works for all elements with "age" css class. If those elements
also have "date" css class, the original text is preserved and age is added at
the end.
This patch adds these two css classes in some pages in gitweb and monoblue that
weren't already using this feature.
Changectx are expensive and not needed there. The use of `repo.set` denote old
code that predate the introduction of `repo.revs` that we now use.
On my mercurial repository 495 draft:
before: 0.054239 second
after: 0.046935 second
On a mercurial repository with 115973 draft:
before: 0.564548 second
after: 0.130534 second
Changectx are expensive and not needed there. The use of `repo.set` denote old
code that predate the introduction of `repo.revs` that we now use.
On my mercurial repository 495 draft:
before: 0.010275 second
after: 0.008832 second
On a mercurial repository with 115973 draft:
before: 0.899255 second
after: 0.397131 second
Clicking on a line link on pages that show any kind of file contents (including
diffs) should highlight that line, and in monoblue it works when there's a
<pre> element (e.g. diff), but pages that use <table> element (annotate and
compare) need this css class. It matches and highlights linked (":target")
table rows. This line is pretty much copied from gitweb theme.
There's a visual difference in hgweb between one changeset that is the tip of
its branch and another that simply belongs to that branch. But paper theme
ignored this difference on changeset page and used to always use "branchname"
css class, be that changeset the tip of its branch or not. That has been
recently fixed, so this piece of css is not needed anymore.
Let's make "instabilities" list contain items with "instability" key as opposed
to "name" key. This way it's more explicit and more consistent with the log
command-line template.
We can avoid a SPACE(len(changelog)) allocation by using a defaultdict.
Test Plan:
python run-tests.py test-bisect*
Differential Revision: https://phab.mercurial-scm.org/D1499
Since we have revsets we can be more concise in doing the ancestor calulcation.
Significant commits are all descendent of the topmost good commits.
Test Plan:
python run-tests.py test-bisect*
Differential Revision: https://phab.mercurial-scm.org/D1498
Pass repo into the bisect function to get more flexibility in what we can call.
This will allow us to use revsets to rewrite parts of the ancestor and children
calculation in later patches.
Test Plan:
python run-tests.py test-bisect*
Differential Revision: https://phab.mercurial-scm.org/D1497
Previously, if the develwarn call site did not specify the category of warning,
and devel.all-warnings was False, it would emit the warning. If it was
intended that this emit a warning if config is unspecified, I would have
expected a comment, so I assumed this was unintentional and am changing the
behavior.
Differential Revision: https://phab.mercurial-scm.org/D1494
hg rm -A option prints the message of every file in the repo. This is not very
user friendly for a big repository with thousands of files. So enabling this
feature only when run in --verbose mode (hg rm -Av)
Differential Revision: https://phab.mercurial-scm.org/D1336
Currently, seekableunbundlepart.seek() will perform a read() during
seek operations. This will allocate a buffer to hold the raw data
over the seek distance. This can lead to very large allocations
and cause performance to suffer.
We change the code to perform read(32768) in a loop to avoid
potentially large allocations.
`hg perfbundleread` on an uncompressed Firefox bundle reveals
a performance impact:
! bundle2 iterparts()
! wall 2.992605 comb 2.990000 user 2.260000 sys 0.730000 (best of 4)
! bundle2 iterparts() seekable
! wall 3.863810 comb 3.860000 user 3.000000 sys 0.860000 (best of 3)
! bundle2 part seek()
! wall 6.213387 comb 6.200000 user 3.350000 sys 2.850000 (best of 3)
! wall 3.820347 comb 3.810000 user 2.980000 sys 0.830000 (best of 3)
Since seekable bundle parts are (only) used by bundlerepo, this /may/
speed up initial loading of bundle-based repos. But any improvement
will likely only be noticed on very large bundles.
Differential Revision: https://phab.mercurial-scm.org/D1394
Before, we were calling struct.unpack() (via an alias) on every
loop iteration. I'm not sure what Python does under the hood, but
it would have to look at the struct format and determine what to
do.
This commit establishes a struct.Struct instance and reuses it for
struct reading.
We can see the impact from running `hg perfbundleread` on a Firefox
bundle:
! read(8k)
! wall 0.679730 comb 0.680000 user 0.140000 sys 0.540000 (best of 15)
! read(16k)
! wall 0.577228 comb 0.570000 user 0.080000 sys 0.490000 (best of 17)
! read(32k)
! wall 0.516060 comb 0.520000 user 0.040000 sys 0.480000 (best of 20)
! read(128k)
! wall 0.496378 comb 0.490000 user 0.010000 sys 0.480000 (best of 20)
! bundle2 iterparts()
! wall 3.056811 comb 3.050000 user 2.340000 sys 0.710000 (best of 4)
! wall 2.992605 comb 2.990000 user 2.260000 sys 0.730000 (best of 4)
! bundle2 iterparts() seekable
! wall 4.007676 comb 4.000000 user 3.170000 sys 0.830000 (best of 3)
! wall 3.863810 comb 3.860000 user 3.000000 sys 0.860000 (best of 3)
! bundle2 part seek()
! wall 6.267110 comb 6.250000 user 3.480000 sys 2.770000 (best of 3)
! wall 6.213387 comb 6.200000 user 3.350000 sys 2.850000 (best of 3)
! bundle2 part read(8k)
! wall 3.404164 comb 3.400000 user 2.650000 sys 0.750000 (best of 3)
! wall 3.241099 comb 3.250000 user 2.560000 sys 0.690000 (best of 3)
! bundle2 part read(16k)
! wall 3.197972 comb 3.200000 user 2.490000 sys 0.710000 (best of 4)
! wall 3.003930 comb 3.000000 user 2.270000 sys 0.730000 (best of 4)
! bundle2 part read(32k)
! wall 3.060557 comb 3.060000 user 2.340000 sys 0.720000 (best of 4)
! wall 2.904695 comb 2.900000 user 2.160000 sys 0.740000 (best of 4)
! bundle2 part read(128k)
! wall 2.952209 comb 2.950000 user 2.230000 sys 0.720000 (best of 4)
! wall 2.776140 comb 2.780000 user 2.070000 sys 0.710000 (best of 4)
Profiling now says most remaining time is spent in util.chunkbuffer.
I already heavily optimized that data structure several releases ago.
So we'll likely get little more performance out of bundle2 reading
while still retaining util.chunkbuffer().
Differential Revision: https://phab.mercurial-scm.org/D1393
Profiling revealed that repeated calls to indebug() were
consuming a fair amount of CPU during bundle2 reading, with
most of the time spent in ui.configbool().
Inlining indebug() and avoiding extra attribute lookups speeds
things up substantially. Using `hg perfbundleread` with a Firefox
bundle:
! read(8k)
! wall 0.679730 comb 0.680000 user 0.140000 sys 0.540000 (best of 15)
! read(16k)
! wall 0.577228 comb 0.570000 user 0.080000 sys 0.490000 (best of 17)
! read(32k)
! wall 0.516060 comb 0.520000 user 0.040000 sys 0.480000 (best of 20)
! read(128k)
! wall 0.496378 comb 0.490000 user 0.010000 sys 0.480000 (best of 20)
! bundle2 iterparts()
! wall 6.983756 comb 6.980000 user 6.220000 sys 0.760000 (best of 3)
! wall 3.460903 comb 3.460000 user 2.760000 sys 0.700000 (best of 3)
! bundle2 iterparts() seekable
! wall 8.132131 comb 8.110000 user 7.160000 sys 0.950000 (best of 3)
! wall 4.312722 comb 4.310000 user 3.480000 sys 0.830000 (best of 3)
! bundle2 part seek()
! wall 10.860942 comb 10.840000 user 7.790000 sys 3.050000 (best of 3)
! wall 6.754764 comb 6.740000 user 3.970000 sys 2.770000 (best of 3)
! bundle2 part read(8k)
! wall 7.258035 comb 7.260000 user 6.470000 sys 0.790000 (best of 3)
! wall 3.668004 comb 3.660000 user 2.960000 sys 0.700000 (best of 3)
! bundle2 part read(16k)
! wall 7.099891 comb 7.080000 user 6.310000 sys 0.770000 (best of 3)
! wall 3.489196 comb 3.480000 user 2.750000 sys 0.730000 (best of 3)
! bundle2 part read(32k)
! wall 6.964685 comb 6.950000 user 6.130000 sys 0.820000 (best of 3)
! wall 3.388569 comb 3.380000 user 2.640000 sys 0.740000 (best of 3)
! bundle2 part read(128k)
! wall 6.852867 comb 6.850000 user 6.060000 sys 0.790000 (best of 3)
! wall 3.276415 comb 3.270000 user 2.560000 sys 0.710000 (best of 4)
Differential Revision: https://phab.mercurial-scm.org/D1391
The last commit removed the last use of the bundle2 part seek() API
in the generic bundle2 part iteration code. This means we can now
switch to using unseekable bundle2 parts by default and have the
special consumers that actually need the behavior request it.
This commit changes unbundle20.iterparts() to expose non-seekable
unbundlepart instances by default. If seekable parts are needed,
callers can pass "seekable=True." The bundlerepo class needs
seekable parts, so it does this.
The interrupt handler is also changed to use a regular unbundlepart.
So, by default, all consumers except bundlerepo will see unseekable
parts.
Because the behavior of the iterparts() benchmark changed, we add
a variation to test seekable parts vs unseekable parts. And because
parts no longer have seek() unless "seekable=True," we update the
"part seek" benchmark.
Speaking of benchmarks, this change has the following impact to
`hg perfbundleread` on an uncompressed bundle of the Firefox repo
(6,070,036,163 bytes):
! read(8k)
! wall 0.722709 comb 0.720000 user 0.150000 sys 0.570000 (best of 14)
! read(16k)
! wall 0.602208 comb 0.590000 user 0.080000 sys 0.510000 (best of 17)
! read(32k)
! wall 0.554018 comb 0.560000 user 0.050000 sys 0.510000 (best of 18)
! read(128k)
! wall 0.520086 comb 0.530000 user 0.020000 sys 0.510000 (best of 20)
! bundle2 forwardchunks()
! wall 2.996329 comb 3.000000 user 2.300000 sys 0.700000 (best of 4)
! bundle2 iterparts()
! wall 8.070791 comb 8.060000 user 7.180000 sys 0.880000 (best of 3)
! wall 6.983756 comb 6.980000 user 6.220000 sys 0.760000 (best of 3)
! bundle2 iterparts() seekable
! wall 8.132131 comb 8.110000 user 7.160000 sys 0.950000 (best of 3)
! bundle2 part seek()
! wall 10.370142 comb 10.350000 user 7.430000 sys 2.920000 (best of 3)
! wall 10.860942 comb 10.840000 user 7.790000 sys 3.050000 (best of 3)
! bundle2 part read(8k)
! wall 8.599892 comb 8.580000 user 7.720000 sys 0.860000 (best of 3)
! wall 7.258035 comb 7.260000 user 6.470000 sys 0.790000 (best of 3)
! bundle2 part read(16k)
! wall 8.265361 comb 8.250000 user 7.360000 sys 0.890000 (best of 3)
! wall 7.099891 comb 7.080000 user 6.310000 sys 0.770000 (best of 3)
! bundle2 part read(32k)
! wall 8.290308 comb 8.280000 user 7.330000 sys 0.950000 (best of 3)
! wall 6.964685 comb 6.950000 user 6.130000 sys 0.820000 (best of 3)
! bundle2 part read(128k)
! wall 8.204900 comb 8.150000 user 7.210000 sys 0.940000 (best of 3)
! wall 6.852867 comb 6.850000 user 6.060000 sys 0.790000 (best of 3)
The significant speedup is due to not incurring the overhead to track
payload offset data. Of course, this overhead is proportional to
bundle2 part size. So a multiple gigabyte changegroup part is on the
extreme side of the spectrum for real-world impact.
In addition to the CPU efficiency wins, not tracking offset data
also means not using memory to hold that data. Using a bundle based on
the example BSD repository in issue 5691, this change has a drastic
impact to memory usage during `hg unbundle` (`hg clone` would behave
similarly). Before, memory usage incrementally increased for the
duration of bundle processing. In other words, as we advanced through
the changegroup and bundle2 part, we kept allocating more memory to
hold offset data. After this change, we still increase memory during
changegroup application. But the rate of increase is significantly
slower. (A bulk of the remaining gradual increase appears to be the
storing of revlog sizes in the transaction object to facilitate
rollback.)
The RSS at the end of filelog application is as follows:
Before: ~752 MB
After: ~567 MB
So, we were storing ~185 MB of offset data that we never even used.
Talk about wasteful!
.. api::
bundle2 parts are no longer seekable by default.
.. perf::
bundle2 read I/O throughput significantly increased.
.. perf::
Significant memory use reductions when reading from bundle2 bundles.
On the BSD repository, peak RSS during changegroup application
decreased by ~185 MB from ~752 MB to ~567 MB.
Differential Revision: https://phab.mercurial-scm.org/D1390
For reasons still not yet fully understood by me, bundlerepo
requires its changegroup bundle2 part to be seeked to beginning
after part iteration. As far as I can tell, it is the only
bundle2 part consumer that relies on this behavior.
This seeking was performed in the generic iterparts() API. Again,
I don't fully understand why it was here and not in bundlerepo.
Probably historical reasons.
What I do know is that all other bundle2 part consumers don't
need this special behavior (assuming the tests are comprehensive).
So, we move the code from bundle2's iterparts() to bundlerepo's
consumption of iterparts().
Differential Revision: https://phab.mercurial-scm.org/D1389
We want bundle parts to not be seekable by default. That means
eliminating the generic seek() method.
A common pattern in bundle2.py is to seek to the end of the part
data. This is mainly used by the part iteration code to ensure
the underlying stream is advanced to the next bundle part.
In this commit, we establish a dedicated API for consuming a
bundle2 part data. We switch users of seek() to it.
The old implementation of seek(0, os.SEEK_END) would effectively
call self.read(). The new implementation calls self.read(32768)
in a loop. The old implementation would therefore assemble a
buffer to hold all remaining data being seeked over. For seeking
over large bundle parts, this would involve a large allocation and
a lot of overhead to collect intermediate data! This overhead can
be seen in the results for `hg perfbundleread`:
! bundle2 iterparts()
! wall 10.891305 comb 10.820000 user 7.990000 sys 2.830000 (best of 3)
! wall 8.070791 comb 8.060000 user 7.180000 sys 0.880000 (best of 3)
! bundle2 part seek()
! wall 12.991478 comb 10.390000 user 7.720000 sys 2.670000 (best of 3)
! wall 10.370142 comb 10.350000 user 7.430000 sys 2.920000 (best of 3)
Of course, skipping over large payload data isn't likely very common.
So I doubt the performance wins will be observed in the wild.
Differential Revision: https://phab.mercurial-scm.org/D1388
The previous commit extracted _payloadchunks() to a new derived class.
There was still a reference to this method in unbundlepart, making
unbundlepart unusable on its own.
This commit implements a generic version of a bundle2 part payload
decoder, without offset tracking. seekableunbundlepart._payloadchunks()
has been refactored to consume it, adding offset tracking like before.
We also implement unbundlepart._payloadchunks(), which is a thin
wrapper for it. Since we never instantiate unbundlepart directly,
this new method is not used. This will be changed in subsequent
commits.
The new implementation also inlines some simple code from unpackermixin
and adds some local variable to prevent extra function calls and
attribute lookups. `hg perfbundleread` on an uncompressed Firefox
bundle seems to show a minor win:
! bundle2 iterparts()
! wall 12.593258 comb 12.250000 user 8.870000 sys 3.380000 (best of 3)
! wall 10.891305 comb 10.820000 user 7.990000 sys 2.830000 (best of 3)
! bundle2 part seek()
! wall 13.173163 comb 11.100000 user 8.390000 sys 2.710000 (best of 3)
! wall 12.991478 comb 10.390000 user 7.720000 sys 2.670000 (best of 3)
! bundle2 part read(8k)
! wall 9.483612 comb 9.480000 user 8.420000 sys 1.060000 (best of 3)
! wall 8.599892 comb 8.580000 user 7.720000 sys 0.860000 (best of 3)
! bundle2 part read(16k)
! wall 9.159815 comb 9.150000 user 8.220000 sys 0.930000 (best of 3)
! wall 8.265361 comb 8.250000 user 7.360000 sys 0.890000 (best of 3)
! bundle2 part read(32k)
! wall 9.141308 comb 9.130000 user 8.220000 sys 0.910000 (best of 3)
! wall 8.290308 comb 8.280000 user 7.330000 sys 0.950000 (best of 3)
! bundle2 part read(128k)
! wall 8.880587 comb 8.850000 user 7.960000 sys 0.890000 (best of 3)
! wall 8.204900 comb 8.150000 user 7.210000 sys 0.940000 (best of 3)
Function call overhead in Python strikes again!
Of course, bundle2 decoding CPU overhead is likely small compared to
decompression and changegroup application. But every little bit helps.
Differential Revision: https://phab.mercurial-scm.org/D1387
Currently, unbundlepart classes support bi-directional seeking.
Most consumers of unbundlepart only ever seek forward - typically
as part of moving to the end of the bundle part so they can move
on to the next one. But regardless of the actual usage of the
part, instances maintain an index mapping offsets within the
underlying raw payload to offsets within the decoded payload.
Maintaining the mapping of offset data can be expensive in terms of
memory use. Furthermore, many bundle2 consumers don't have access
to an underlying seekable stream. This includes all compressed
bundles. So maintaining offset data when the underlying stream
can't be seeked anyway is wasteful. And since many bundle2 streams
can't be seeked, it seems like a bad idea to expose a seek API
in bundle2 parts by default. If you provide them, people will
attempt to use them.
Seekable bundle2 parts should be the exception, not the rule. This
commit starts the process dividing unbundlepart into 2 classes: a
base class that supports linear, one-time reads and a child class
that supports bi-directional seeking. In this first commit, we
split various methods and attributes out into a new
"seekableunbundlepart" class. Previous instantiators of "unbundlepart"
now instantiate "seekableunbundlepart." This preserves backwards
compatibility. The coupling between the classes is still tight:
"unbundlepart" cannot be used on its own. This will be addressed
in subsequent commits.
Differential Revision: https://phab.mercurial-scm.org/D1386
Adding a possibility to configure error hint to be shown in the case of problems with SSH. Example of such hint can be "Please see http://company/internalwiki/ssh.html".
Test Plan:
- Ran hg pull with broken link and verified the output has no hint by default:
```
pulling from ssh://brokenrepository.com//repo
remote: ssh: Could not resolve hostname brokenrepository.com: Name or service not known
abort: no suitable response from remote hg!
```
- Run hg pull --config ui.ssherrorhint="Please see http://company/internalwiki/ssh.html":
```
pulling from ssh://brokenrepository.com//repo
remote: ssh: Could not resolve hostname brokenrepository.com: Name or service not known
abort: no suitable response from remote hg!
(Please see http://company/internalwiki/ssh.html)
```
Differential Revision: https://phab.mercurial-scm.org/D1431
When writing my first extension, I found it hard to figure out these functions.
I figured documenting their inputs/outputs would help future authors who
are new to the codebase.
Differential Revision: https://phab.mercurial-scm.org/D1440
In the previous release we added an argument `cmdtype` to registrar.command()
which is a enum and tells whether the command is read only, recoverable write or
unrecoverable write command. This patch add the value of cmdtype argument for
commands which are read only.
Differential Revision: https://phab.mercurial-scm.org/D1468
We'll raise this exception in the merge code, and in-memory users like rebase
can catch it and retry without IMM.
Differential Revision: https://phab.mercurial-scm.org/D1210
This makes graphdata() simpler by using existing code that gets common
changeset properties for showing in hgweb. graphdata() is a nested function in
graph() that prepares entries for /graph view, but there are two different
lists of changesets prepared: "jsdata" for JavaScript-rendered graph and
"nodes" for everything else.
For "jsdata", properties "node", "user", "age" and "desc" are passed through
various template filters because we don't have these filters in JavaScript, so
the data has to be prepared server-side. But now that commonentry() is used for
producing "nodes" list (and it doesn't apply any filters), these filters need
to be added to the appropriate templates (only raw at this moment, everything
else either doesn't implement graph or uses JavaScript).
This is a bit of refactoring that will hopefully simplify future patches. The
end result is to have /graph that only renders the actual graph with nodes and
vertices in JavaScript, and the rest is done server-side. This way server-side
code can focus on showing a list of changesets, which is easy because we
already have /log, /shortlog, etc, and JavaScript code can be simplified,
making it easier to add obsolescence graph and other features.
This piece of code checks if a changeset is the tip of its branch, but as can
be seen above in the context, "branch" was prepared for being displayed in
hgweb by making it unicode and passing it through url.escape. It's better to
use the original ctx.branch().
In paper, coal, gitweb and monoblue a new "tag" (or multiple, if there are many
instabilities) is added to the same line that has phase, branch, etc of a
changeset; in gitweb and monoblue this element has a light red background, in
paper and coal the element is black and underlined. In spartan theme
instabilities are shown on a separate line.
While test-obsolete.t uses first(phasedivergent()) revset to pick a changeset
to test, that particular changeset is also an orphan, so two different
instability tags are displayed.
This method is now used in webutils.commonentry(), which adds common data items
(commit hash, author, date, etc) for rendering changesets in hgweb. Usually,
commonentry() is given a changectx as ctx; but in views related to files (e.g.
file view, diff, annotate) it's replaced by a filectx, so the latter also needs
to have instabilities() method.
This anchor text is problematic in two ways: first, the "back to" part
assumes that you got to the page showing it via the filelog. This is
not necessarily true, as there are other ways to get to that view
besides the filelog view, such as for example following the history of
lines from a file. Second, it uses "filelog" jargon, which refers to
how each file has its own revlog. This is internal jargon that has no
business being exposed to the end user.
I just reworded this template to improve understanding.
As with phases, spartan theme shows a simple "obsolete: yes" on its own line
(this allows replacing "yes" with something more useful in future, like output
of obsfate* template functions). Everywhere else a new "tag" is added to the
same line that has phase, branch, etc of a changeset; in gitweb and monoblue
the element has gray background, in paper and coal the element is gray with a
dashed underline.
This method is now used in webutils.commonentry(), which adds common data items
(commit hash, author, date, etc) for rendering changesets in hgweb. Usually,
commonentry() is given a changectx as ctx; but in views related to files (e.g.
file view, diff, annotate) it's replaced by a filectx, so the latter also needs
to have obsolete() method.
Other implementations of dirstate will want to replace the class used for the
dirstate map. Allow this to happen by making the class an attribute of the
dirstate.
Differential Revision: https://phab.mercurial-scm.org/D1347
The set-like object returned by dirstate.dirs may be difficult for other
implementations of the dirstate to provide, and is unnecessary as it is
only ever used for __contains__. Instead, provide an explicit method for
testing for a directory.
.. api::
dirstate no longer provides a `dirs()` method. To test for the existence of
a directory in the dirstate, use `dirstate.hasdir(dirname)`.
Differential Revision: https://phab.mercurial-scm.org/D1345
This method now doesn't do very much interesting and can be removed. The
updated files set can be updated where _droppath was originally called.
Differential Revision: https://phab.mercurial-scm.org/D1344
When dropping files from the dirstate, the corresponding entry in the
filefoldmap is also dropped. Move this into the dirstate map object. A future
implementation of the dirstate will maintain the filefoldmap differently.
Differential Revision: https://phab.mercurial-scm.org/D1343
The dirstate dirs object is owned by the map, so move management of that object
there. A future implementation of the dirstate will manage the dirs object
differently.
Differential Revision: https://phab.mercurial-scm.org/D1342
The dirstate map owns the nonnormal sets, and so should be the class to update
them. A future implementation of the dirstate will manage these maps
differently.
The action of clearing ambiguous times is now entirely controlled by the
dirstate map, so it moves there too.
Differential Revision: https://phab.mercurial-scm.org/D1341
Instead of assigning dirstatetuple objects to entries in the dirstate, move
responsibility for creating tuples into the dirstatemap.
Differential Revision: https://phab.mercurial-scm.org/D1340
Strictly speaking, this variable tracks offsets within the
changegroup, not the bundle.
While we're here, mark a class attribute as private because
it is.
.. api::
Rename bundlerepo.bundlerepository.bundlefilespos to
_cgfilespos.
Differential Revision: https://phab.mercurial-scm.org/D1384
"bundle" was appropriate for the bundle1 days where a bundle
was a changegroup. In a bundle2 world, changegroup readers
are referred to as "changegroup unpackers."
Differential Revision: https://phab.mercurial-scm.org/D1383
3649c3f2cd (revert: do not reverse hunks in interactive when REV is not
parent (issue5096)) changed the record "operation" for the text version but
missed the curses version. Without this patch, running
`hg revert -ir '.^' --config ui.interface=curses` would crash with:
ProgrammingError: unexpected operation: apply
Differential Revision: https://phab.mercurial-scm.org/D1381
In spartan theme phase is shown on its own table row, because there's no single
line of "tags". Everywhere else phase is prepended to the list of "tags" of a
changeset. Its element has a purple-ish color in gitweb and monoblue, and a
dotted line under it and no color in paper and coal (as these themes are frugal
with colors).
This patch intentionally doesn't touch graph, because it needs a rewrite. I'll
get to it pretty soon and in the process will add phase and everything that's
still coming (e.g. obsolescence and instabilities).
.. feature::
hgweb now displays phases of non-public changesets
This patch puts all these changeset "tags" into one template shared everywhere
in paper and coal themes. But it should be noted that some of the templates had
different sets of tags, in some cases it was intended, in others - most likely
not.
First, what's up with all these different ways to get changeset's branch. There
are actually 3 ways to do it in hgweb, they can all be seen in this patch;
"branches", "inbranch" and "branch". They are all lists that consist of 1 or 0
items:
- "branches" has ctx.branch() if current changeset is the tip of that branch
- "inbranch" has ctx.branch() if current changeset is _not_ the tip of that
branch and the branch is not "default"
- "branch" aka "changesetbranch" has ctx.branch() if the branch is not
"default"
The majority of cases (7 vs 2 + /graph) in paper theme used only option 3,
which meant that "default" was never displayed. But other parts of the theme
disagreed with this and used option 1 and option 2 together. For example, the
default view (log) displays "default" on the branch tip (can be seen right
about now on m-s.o/repo/hg), but it disappears when you click on the commit.
Also, using option 3 alone meant that there was no way to tell if a changeset
is the tip of its branch or not (it was always assumed that it's not, see how
some css classes change from "branchname" to the correct "branchhead" in tests)
-- so the two different css styles that exist in paper just for this were
underused.
I think this patch improves the situation, even though it changes the old (even
if inconsistent) behavior. The new behavior matches that of gitweb and
monoblue.
I'd like to call the function from an extension, passing both "strip"
and "prefix", but it currently only accepts "strip". The only in-tree
caller seems to be mq.py, which doesn't even pass "strip".
Differential Revision: https://phab.mercurial-scm.org/D1413
Less duplication and it's also easier to add extra "tags" everywhere at once.
These aren't tags as defined `hg help glossary` (hence the quotes), they are
simply called that. They include branch name (in different styles if changeset
is a head of that branch or not), (actual) tags and bookmarks. Good candidates
to add to this list would be changeset phase and obsoletion status.
It is a bit wonky to assign the same object to multiple
attributes and then possibly overwrite them later.
Refactor the code to use a local variable and defer attribute
assignment until the final values are ready.
This required passing the bundle instance to _handlebundle2part().
The only use of this method I could find is Facebook's
treemanifest extension. Since it is a private method, I don't
think it warrants an API callout.
Differential Revision: https://phab.mercurial-scm.org/D1378
These attributes are implementation details and shouldn't be
exposed outside the class.
.. api::
bundlerepo.bundlerepository.bundle and
bundlerepo.bundlerepository.bundlefile are now prefixed with an
underscore.
Differential Revision: https://phab.mercurial-scm.org/D1377
exchange.readbundle() can return a type that represents a stream
clone bundle. Explicitly handle the bundle1 type and raise a
reasonable error message for unhandled bundle types.
Differential Revision: https://phab.mercurial-scm.org/D1376
It looks like the refactor in 0883a2ece555 attempted to establish
this method argument but failed to use it. My editor caught it.
Differential Revision: https://phab.mercurial-scm.org/D1373
My editor was complaining about mismatches between method
signatures.
For methods that are implemented, we change arguments to match
the base. For those that aren't, we use variable arguments
because it shouldn't matter.
Differential Revision: https://phab.mercurial-scm.org/D1372
Constants make code easier to read than magic numbers.
I also threw in an explicit argument in a caller to further
increase code comprehension.
Differential Revision: https://phab.mercurial-scm.org/D1370
This is a short topic to explain how command-line flags can be specified.
Some users have been confused by hg offerring different flag syntax than some
other libraries, so it'd be nice to point them to this rather than explaining
it every time.
Differential Revision: https://phab.mercurial-scm.org/D1270
Everything is pretty self-explanatory except the last hunk, where jshint
complains: "Misleading line break before '||'; readers may interpret this as an
expression boundary."
There is a tweakable called "laxbreak" that allows line breaks before
operators, but I think it's fine to simply join this one line and avoid extra
config for now (we can cook up and add a sensible .jshintrc later).
The internal representation of bookmark value is binary. The fact we stored
'hex' was an implementation detail from using pushkey.
We move the values in 'pullop.remotebookmarks' to binary before adding a way to
exchange bookmarks not based on pushkey.
These new colors and styles are already used in the tooltip that gets shown
when user hovers over line numbers on annotate page. The old styles, replaced
in this patch, look completely unrelated to gitweb or any other hgweb theme.
These new colors and styles are already used in the tooltip that gets shown
when user hovers over line numbers on annotate page. The old styles, replaced
in this patch, look completely unrelated to paper or any other hgweb theme.
The dirstate typically tries to keep the nonnormalset and otherparentset
up-to-date when making changes to the dirstate. In the case of files marked
normallookup, however, it erroneously removes the file from the nonnormalset,
after _addpath had just added it.
This doesn't seem to matter at the moment, as nothing relies on the
nonnormalset being correct at this point, however a future re-implementations
of the dirstate map will require this to be kept up-to-date correctly.
Differential Revision: https://phab.mercurial-scm.org/D1339
dirstatemap.clear should remove all record of the files in the map. This
includes removing caches of values derived from these.
Differential Revision: https://phab.mercurial-scm.org/D1338
This utility function allows clearing of the cached value set up
@propertycache, if there is one.
Differential Revision: https://phab.mercurial-scm.org/D1337
This is fixing quadratic behavior, which is probably not noticeable in the
common case, but if a very large directory gets added here, it can get pretty
bad. This was noticed because we had some pushes that spent >25s in changegroup
generation calling min() here, according to profiling.
The original reasoning for min() being used in 1d71430e1d28 was that, at that
point in the series, we were adding almost everything to tmfnodes during the
first iteration through the loop , so we needed to avoid sending child
directories before parents. Later changes made it so that the child directories
were added only when we visited the parent directory (not all of them on the
first iteration), so this is no longer necessary - there won't be any child
directories in tmfnodes before the parents have been sent.
This does mean that the manifests are now exchanged unordered, whereas
previously we would essentially do [a, b, b/c, b/c/d, e], we now can send a, b,
and e in any order; b/c must still follow b, and b/c/d must still follow b/c.
Differential Revision: https://phab.mercurial-scm.org/D1351
With our treemanifest logic, the manifests are no longer transported as part of
the changegroup and are no longer stored in a revlog. This means the
self.manifestlog line in bundlerepo.filestart no longer calls
_constructmanifest, and therefore does not consume the manifest portion of the
changegroup, which means filestart is not populated and we result in an infinite
loop.
The fix is to make filestart aware that self.manifestlog might not consume the
changegroup part, and consume it manually if necessary.
There's currently no way to test this in core, but our treemanifest extension
has tests to cover this.
Differential Revision: https://phab.mercurial-scm.org/D1329
Right now, successorsetverb only needs successors to compute the verb. But we
will want use additional information (effect-flags and operation) in the near
future to compute a better verb.
Add the markers parameter now so extensions (like Evolve) could wrap the
function and start experimenting around better obsfate verbs.
As successorsetverb now takes both successorset and markers parameters, rename
it to obsfateverb, successorsetandmarkersverb was too long.
Differential Revision: https://phab.mercurial-scm.org/D1191
Previously, the caller was responsible for creating the directory structure of
files written out using a path template. This is onerous, especially if the
exact filenames are not known upfront, but are being accessed via a matcher.
This patch changes things so that hg will attempt to create the appropriate
directories before writing the output file.
Differential Revision: https://phab.mercurial-scm.org/D1332
And introduce a new "apply" operation verb for this case as suggested in
issue5096. This replaces the no longer used "revert" operation.
In interactive revert, when reverting to something else that the parent
revision, display an "apply this change" message with a diff that is not
reversed.
The rationale is that `hg revert -i -r REV` will show hunks of the diff from
the working directory to REV and prompt the user to select them for applying
(to working directory). This contradicts 79cc693b4406 in which it was
decided to have the "direction" of prompted hunks reversed. Later on
[1], there was a broad consensus (but no decision) towards the "as to
be applied direction". Now that --interactive is no longer experimental
(97d754ba45c4), it's time to switch and thus we drop no longer used
"experimental.revertalternateinteractivemode" configuration option.
[1]: https://www.mercurial-scm.org/pipermail/mercurial-devel/2016-November/090142.html
.. feature::
When interactive revert is run against a revision other than the working
directory parent, the diff shown is the diff to *apply* to the working directory,
rather than the diff to *discard* from the working copy. This is in line with
related user experiences with `git` and appears to be less confusing with
`ui.interface=curses`.
This new debugcommand prints the capabilities of any remote in a human friendly
way. Improved bundle2 capabilities support will be introduced in the next
changesets.
We seem to have settled down on behavior changes here (nothing
matching revset `keyword(revert) and keyword(interactive)` since 4.2
was released), so let's go ahead and advertise this excellent feature.
.. feature:: revert --interactive
The revert command now accepts the flag --interactive to allow reverting
only some of the changes to the specified files.
We want a slightly weird format here so that it's easier to read, but
in order to preserve that we need to disable clang-format.
Differential Revision: https://phab.mercurial-scm.org/D1168
Now that all known options are declared, we setup a warning to make sure it will
stay this way.
We disable the warning in two tests checking other behavior with random options.
We add an experimental -L/--line-range option to 'hg log' taking file patterns
along with a line range using the (new) FILE,FROMLINE-TOLINE syntax where FILE
may be a pattern (matching exactly one file). The resulting history is similar
to what the "followlines" revset except that, if --patch is specified,
only diff hunks within specified line range are shown.
Basically, this brings the CLI on par with what currently only exists in hgweb
through line selection in "file" and "annotate" views resulting in a file log
with filtered patch to only display followed line range.
The option may be specified multiple times and can be combined with --rev and
regular file patterns to further restrict revisions. Usage of this option
requires --follow; revisions are shown in descending order and renames are
followed. Only the --graph option is currently not supported.
The UI is the result of a consensus from review feedback at:
https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-October/106749.html
The implementation spreads between commands.log() and cmdutil module.
In commands.log(), the main loop may now use a "hunksfilter" factory (similar
to "filematcher") that, for a given "rev", produces a filtering function
for diff hunks for a given file context object.
The logic to build revisions from -L/--line-range options lives in
cmdutil.getloglinerangerevs() which produces "revs", "filematcher" and
"hunksfilter" information. Revisions obtained by following files' line range
are filtered if they do not match the revset specified by --rev option. If
regular FILE arguments are passed along with -L options, both filematchers are
combined into a new matcher.
.. feature::
Add an experimental -L/--line-range FILE,FROMLINE-TOLINE option to 'hg log'
command to follow the history of files by line range. In combination with
-p/--patch option, only diff hunks within specified line range will be
displayed. Feedback, especially on UX aspects, is welcome.
We add a 'hunksfilterfn' keyword argument in all functions of the call
stack from changeset_printer.show() to patch.diff(). This is a callable
that will be used to filter out hunks by line range and will be used in
the "-L/--line-range" option of "hg log" command introduced in the
following changesets.
We'll need the same logic in forthcoming changeset to handle --line-range
option in 'hg log' command.
The function lives in scmutil.py (rather than util.py) as it uses match and
pathutil modules.
Use the verbosity aware template keyword introduced earlier. It has the nice
property of being verbosity dependent but in order to customize the obsfate
part, users will need to replace the lobsfate definition from default mapfile
with the one using template functions (by copying the one from test-obsmarker-
template.t for example).
As it's a more advanced use-case, I'm more inclined to have the same code for
the {obsfate} keyword, in the changeset printer and in the default mapfile for
consistency.
But, the definition in default mapfile could be replaced with one based on
template filter to obsfate output customization if it is a big need for users.
Having an obsfate by default in log will be useful for users to understand why
they have obsolete and unstable changesets. Obsfate will only be shown for
obsolete changesets, which only happens if people opt-in to experimental feature.
But when obsolete changeset are visible, it is very useful to understand where
they are. Having it in log could be sufficient for most people, so they don't
have to learn a new command (like obslog which is itself useful in case of
divergences).
For example, when pulling and working directory parent become obsolete:
$ hg pull
...
working directory parent is obsolete! (f936c1697205)
This message comes from the Evolve extension.
Obsfate would comes handy:
$ hg log -G
o changeset: 2:6f91013c5136
| tag: tip
| parent: 0:4ef7b558f3ec
| user: Boris Feld <boris.feld@octobus.net>
| date: Mon Oct 09 16:00:27 2017 +0200
| summary: A
|
| @ changeset: 1:f936c1697205
|/ user: Boris Feld <boris.feld@octobus.net>
| date: Mon Oct 09 16:00:27 2017 +0200
| obsfate: rewritten using amend as 2:6f91013c5136
| summary: -A
|
o changeset: 0:feb4dd822b8c
user: Boris Feld <boris.feld@octobus.net>
date: Tue Oct 09 16:00:00 2017 +0200
summary: ROOT
And once we update, we don't have an obsolete changeset in the log anymore so
we don't show obsfate anymore, most users won't see obsfate often if they
don't have obsolete changeset often:
@ changeset: 2:6f91013c5136
| tag: tip
| parent: 0:4ef7b558f3ec
| user: Boris Feld <boris.feld@octobus.net>
| date: Mon Oct 09 16:00:27 2017 +0200
| summary: A
|
o changeset: 0:feb4dd822b8c
user: Boris Feld <boris.feld@octobus.net>
date: Tue Oct 09 16:00:00 2017 +0200
summary: ROOT
Obsolescence is sometimes used only locally so the obs-marker users is always
the same. Showing the user in this case does not bring much values.
In the case where multiple users rewrite the commit, display the full list of
users. Also show all users in verbose mode.
Introduce an obsfate printer that uses all helpers functions defined in
obsutil to get all the obsfate-related data and format a string according to
the current format in test-obsmarker-template.t.
Then, introduce an obsfate templatekw that uses the obsfateprinter to return a
list of strings.
The goal is not to replace existing obsfate template functions but to propose
a default, good-enough and easily usable obsfate definition for end-users that
don't want to customize it. Such output would ultimately get included in the
default log output.
Here are some output examples for a commit amended:
rewritten using amend as 5:a9b1f8652753 by test (at 1970-01-01 00:00 +0000)
Next patches will make the output dependent on the verbosity.
Exemple of use-cases:
For having the obsfate on a single-line between brackets:
{if(obsfate, " [{join(obsfate, "; ")}]")}
For having the obsfate in several lines:
{if(obsfate, "{obsfate % " Obsfate: {fate}\n"}")}
The heuristics algorithm find possible candidates for move/copy and then check
whether they are actually a copy or move. In some cases, there can be lot of
candidates possible which can actually slow down the algorithm.
This patch introduces a config option
`experimental.copytrace.movecandidateslimit` using which one can limit the
candidates to check. The limit defaults to 100.
Thanks to Yuya for suggesting to skip copytracing for that file with a
warning.
Differential Revision: https://phab.mercurial-scm.org/D987
I have spent a lot of time debugging extensions that failed to load
because we don't include a traceback and I didn't realize I could get
traceback for the extension failure with --traceback. Let's just turn
them on by default, since it should be rare that the user sees these
tracebacks anyway (and if they do, it's not so bad if the extra
traceback pushes them a little harder to report the problem).
Since we already had a test case with --traceback and one without, I
just removed the one with the flag.
Differential Revision: https://phab.mercurial-scm.org/D1164
The show extension reads `commands.show.aliasprefix` config in its
`extsetup` and that causes issues with chg. This patch adds that config item
to chg confighash to solve the issue.
Test Plan:
`run-tests.py -l --chg test-show.t`
Differential Revision: https://phab.mercurial-scm.org/D1158
We only want to do I/O in terms of bytes, so lets explode early
instead of recursing forever.
Differential Revision: https://phab.mercurial-scm.org/D1136
The part we are using for pull is now used for push too. As we no longer use
pushkey, pushkey hooks are no longer triggered. This is an obvious backward
incompatible change. We could artificially trigger the pushkey hook within the
bundle2 part, but this seemed too hacky to me.
An option would be to disable by default this new mechanism for a couple of
versions to help people migrate to `txnclose-phase`. I took the liberal and
optimistic path to just turn it on by default directly.
.. bc::
Push no longer triggers a pushkey hook when updating phases. Use the new
`txnclose-phase` and `txnclose-phase` hooks instead.
(Applies when both server and client use version 4.4 or above).
`filecmp` follows symlinks by default, which a `filectx.cmp()` call should not
be doing as it should only compare the requested entry. After this patch, only
the contexts' data are compared, which is the correct contract.
This is a corrected version of D1122.
Differential Revision: https://phab.mercurial-scm.org/D1165
Fix regression introduced in D785.
In some circumstances, context.clearunknown can be called before the path is
audited. Audit the path before deleting any conflicting files.
Differential Revision: https://phab.mercurial-scm.org/D1157
The shelve extensions import and call rebase content without loading the
extension. This is problematic as the config items rebase uses are not
declared and the default value are not set, etc...
The shelve extension should be using core utilities only and the necessary bit
should be moved from rebase into core. In the meantime, I'm taking a small
step to get config registration completed with minimal overhead. The rebase
extension is shipped with core so registering its config option within core is
not a big issue.
This is the last step needed before we can install a warning that enforces all
config to be registered.
We register the merge-tools config section (which has an arbitrary base config
value) and the possible sub-attribute. The sub-attribute has to be registered
first or at the same time otherwise the '.*' item would shadow them.
Merge tools could include "." in their name so we can't constrain any more
than just ".*".
Delta chains can become quite sparse if there is a lot of unrelated data between
relevant pieces. Right now, revlog always reads all the necessary data for the
delta chain in one single read. This can lead to a lot of unrelated data to be
read (see issue5482 for more details).
One can use the `experimental.maxdeltachainspan` option with a large value
(or -1) to easily produce a very sparse delta chain.
This change introduces the ability to slice the chunks retrieval into multiple
reads, skipping large sections of unrelated data. Preliminary testing shows
interesting results. For example the peak memory consumption to read a manifest
on a large repository is reduced from 600MB to 250MB (200MB without
maxdeltachainspan). However, the slicing itself and the multiple reads can have
an negative impact on performance. This is why the new feature is hidden behind
an experimental flag.
Future changesets will add various parameters to control the slicing heuristics.
We hope to experiment a wide variety of repositories during 4.4 and hopefully
turn the feature on by default in 4.5.
As a first try, the algorithm itself is prone to deep changes. However, we wish
to define APIs and have a baseline to work on.
When a merge commit creates an empty diff in the revlog, its offset may still
be quite far from the end of the previous chunk.
Skipping these empty chunks may reduce read size significantly.
In most cases, there is no gain, and in some cases, little gain.
On my clone of pypy, `hg manifest` reads 65% less bytes (96140 i/o 275943) for
revision 4260 by ignoring the only empty trailing diff.
For revision 2229, 35% (34557 i/o 53435)
Sadly, this is difficult to reproduce, as hg clone can make its own different
structure every time.
Move the logic to build bundle2 pushkey part into its dedicated function. It
will help to keep the logic clear when adding support for sending phases change
using 'phase-heads' part.
We are about to switch phase pushing from using pushkey to using a the new
dedicated binary part. We introduce the push race detection on its own to help
detect potential impact.
This part checks if revisions are still in the same phase as when the
bundle was generated. This is similar to what 'check:heads' or
'check:updated-heads' bundle2 part achieves for changesets.
We needs seems before we can move away from pushkey usage from phase since
pushkey has it own built-in push-race detection.
This comment is about:
issue3781: Courtesy Phases synchronisation to publishing server prevent
subrepo push
Not about:
issue3871: Slow hg log when template contains {file_adds}, {file_mods} and
{file_dels}
The on-disk file can contain draft root that are descendants of secret root.
The resulting phase computation is correct, but the phases root content is
not. I will send another series to introduce code that remove some of the
cases where this can happens, but we first need to damage control the existing
case.
After this changeset, we can no longer advertise secret changeset as draft
root.
Previously, only the top level repo was shared, and then any subrepos were
cloned on demand. This is problematic because commits to the parent repo would
write an updated .hgsubstate to the share source, but the corresponding subrepo
commit would be stuck in the local subrepo. That would prevent an update in the
source repo. We already go to great lengths to avoid having inconsistent repos
(e.g., `hg push -r rev` will push _everything_ in a subrepo, even if it isn't
referenced in one of the parent's outgoing commits). Therefore, this seems like
a bug fix, and there's no option to get the old behavior. I can't imagine the
previous behavior was useful to anybody.
There shouldn't be an issue with svn, since it is centralized. Maybe --git-dir
can be used for git subrepos, but I'll leave that to someone more familiar with
git.
An integer was previously being implicitly returned from commands.share(), which
caused dispatch() to start crashing when changing over to returning the shared
repo. All error paths appear to raise, so this can be hardcoded to success.
The clone command checks for 'is None' in a similar pattern, but since
hg.clone() always returns a tuple, that seems wrong?
.. fix:: Issue 5675
Creating a share of a repository with a Mercurial subrepository will now
share the subrepository.
and
.. bc::
Mercurial subrepositories are now shared instead of cloned when the parent
repository is shared. This prevents dangling subrepository references in the
share source. Previously shared repositories with cloned subrepositories
will continue to function unchanged.
__name__ is unicode, but we need bytes. For now, we'll make the
(mostly-safe) assumption that template filter names will be ascii.
Differential Revision: https://phab.mercurial-scm.org/D1137
Stops us from choking the templater on Python 3. With this patch
applied, much of hgweb works correctly in Python 3. The notable
exception is the graph page, which chokes because it gets node IDs as
str instead of bytes.
Differential Revision: https://phab.mercurial-scm.org/D1135
experimental.updatecheck was renamed into commands.update.check, use the
config system to provides the fallback on the old config name instead of
adding more code.
Differential Revision: https://phab.mercurial-scm.org/D1117
I may be a weird person for liking this style, but our C style is
historically nominally the Linux Kernel style, and when you configure
clang-format to be kernel-ish, this is what you get. If we want to
change it, we can do so by tweaking the formatter rules in the future.
Differential Revision: https://phab.mercurial-scm.org/D1132
Depends on D932.
Call the new _onfilemergefailure function when a merge tool reports failure
via a return code.
Differential Revision: https://phab.mercurial-scm.org/D951
Depends on D931.
This patch introduces functions and a config option that will allow a user
to halt the merge if there are failures during a file merge. These functions
will be used in the next patch.
Differential Revision: https://phab.mercurial-scm.org/D932
`commit --amend` and amend command in core and extensions rely on
cmdutil.amend() for amending a commit. So the logic to add a note to amend must
reside here. This patch assumes that note will be passed in opts dictionary to
the function and it will be passed to cleanupnodes and then createmarkers to
store the note in the obsmarker metadata.
After this patch, note can be stored on an amend changeset by passing notes as a
part of opts to cmdutil.amend().
Differential Revision: https://phab.mercurial-scm.org/D1094
This patch adds a metadata argument to cleanupnodes() which will be dict and can
be passed to obsmarker.createmarkers() and can be stored on the obsmarker.
In cases when obsolescence is not enabled, the metadata argument is useless.
This is a step towards storing a note in amend.
Differential Revision: https://phab.mercurial-scm.org/D1093
With in-memory merge, copy information needs to be stored in-memory, not in the
dirstate.
To make this transition easy, move the existing dirstate-based approach to
workingfilectx; that way, other implementations can choose to store it
somewhere else.
Differential Revision: https://phab.mercurial-scm.org/D1106
With in-memory merge, backup files might be overlayworkingfilectxs stored
in memory. But they could also be real files if the user's backup directory is
outside the working dir.
Rather than have two code paths everywhere, let's use arbitraryfilectx so they
can be consistent.
Differential Revision: https://phab.mercurial-scm.org/D1057
