This block was overwriting any result from the previous block anyway. So we move
it first to prove it is possible and we'll extract it in its own function in the
next patch.
One of the main goal of having consolidated destination function is to allow
extension to play with this logic. We extract sub logic to make is wrapping more
practical.
Function in destutil are much simpler to wrap and more flexible than revset.
This also help consistency as 'destupdate' live here and cannot become a pure
revset anyway.
The revset is not ready for prime time yet. However it is useful to have some
version of it exposed to help candidate users to play with it and provide
feedback on what we should aim at.
We add a small test to make sure the code runs.
I'm about to add a cg3, and it seems prudent to annotate what formats
support what features. It strikes me that we may want to consider
moving to a more feature-oriented model in the future, but we'll see
how that looks in a little while I guess.
I'm not sure what to do abstraction-wise here. It might be more
sensible to make a memoryrepo that could apply a bundle in-memory and
then we could make the changegroup data be strictly an applyable
stream, but that's an idea for Later.
The new convention is that bundles contain changegroups. bundle1
happens to *only* be a changegroup, but bundle2 is a more featureful
container that isn't something you can pass to addgroup().
The client now sends a "cbattempted" boolean flag to the "getbundle"
wire protocol command to tell the server whether a clone bundle was
attempted.
The presence of this flag will enable the server to conditionally emit a
bundle2 "output" part advertising the availability of clone bundles to
compatible clients that don't have it enabled.
This is needed so a subsequent patch can conditionally add a bundle2
part to the "getbundle" wire protocol command depending on whether a
clone bundle was attempted.
If a clone bundle persistently fails to apply, users need a way to
disable it so they have a hope of the clone working. Change the hint for
the abort scenario to advertise the config option to disable clone
bundles.
The client represents boolean arguments as '0' and '1'.
bool('0') == bool('1') == True, so a simple bool(val) isn't sufficient
for converting the argument back to a bool type.
Currently, "obsmarkers" is the only boolean argument to getbundle.
I /think/ the only place where we currently set the "obsmarkers"
argument is during bundle2 pulls. As a result of this bug, the server
/might/ be sending obsolete markers bundle2 part(s) to clients that
don't request them. That is why I marked this BC.
Surprisingly there was no test fall out from this change. I suspect a
lapse in test coverage.
We would skip the part if it was fully unknown, so we should also skip it if we
know we won't be able to apply it. This will allow us to produce bundles with
obsolescence markers alongside changegroup while still being able to apply them
on any client.
We want to gracefully handle the read only case in some case (current target:
advisory obsmarkers parts in bundle2). So we expose the attribute in a clean
way.
The 'postincoming' function used by 'hg pull --update' and 'hg unbundle' is
catching 'Abort' exceptions to intercept failed update. This feel a bit too
wide to me, so I'm introducing a more precise exception to specify update
destination issues.
Now that all internal callers pre-compute and set a destination at a higher level
it feels like we can kill this API. This will allow us to simplify this
function. However I feel like this is a bit too central and critical to break
now. I'm adding a devel warning to let extension make catch this in the next
cycle.
Due to how the line links now reside outside of the source lines, hovering over
line numbers doesn't count as hovering over the appropriate source line. It can
be worked around by using a "+" css selector. However, it's necessary to
reorder the elements and put <a> before <span> (which is actually quite
logical). It works without further css tweaks because <a> is already
absolute-positioned and so the order doesn't matter visually.
Due to how the line links now reside outside of the source lines, hovering over
line numbers doesn't count as hovering over the appropriate source line. It can
be worked around by using a "+" css selector. However, it's necessary to
reorder the elements and put <a> before <span> (which is actually quite
logical). It works without further css tweaks because <a> is already
absolute-positioned and so the order doesn't matter visually.
Due to how the line numbers in monoblue are formed (via css counters), the size
of the area with the numbers and the size of the actually clickable links are
not tied together well enough. Before this patch, there were noticeable "gaps"
between line links - clicking on the bottom part of a visible line number did
nothing as opposed to selecting this line.
Let's set font-size for everything in pre.sourcelines so that it also affects
the links and then add a bit of padding to them so compensate for layout
differences.
This way the sizes are still not 100% the same, but should be very close.
In hgweb, some pages have a context of current revision; e.g. changelog and
shortlog show changesets starting from this current revision. However, some
gitweb templates were dropping current revision from some urls _to_ /graph page
and _on_ that page. This patch fixes it.
This has been a repeating source of confusion for users of HackableMercurial.
Note that users of HackableMercurial should *not* and are *not* expected to set
PYTHONHOME.
File/directory case folding collisions cannot be represented on case folding
systems and have to fail.
To detect this and abort early, utilize that for file/directory collisions, a
sorted list of case folded manifest names will have the colliding directory
right after the file.
(This could perhaps be optimized, but this way of doing it also has
directory/directory case folding in mind ... which however not is handled yet.)
A driver-resolved file is a file that's handled specially by the driver. A
common use case for this state would be autogenerated files, the generation of
which should happen only after all source conflicts are resolved.
This is done with an uppercase letter because older versions of Mercurial will
not know how to treat such files at all.
A 'merge driver' is a coordinator for the overall merge process. It will be
able to control:
- tools for individual files, much like the merge-patterns configuration does
today
- tools that can work across groups of files
- the ordering of file resolution
- resolution of automatically generated files
- adding and removing additional files to and from the dirstate
Since it is a critical part of the merge process, it really is part of the
merge state.
This is a lowercase character (i.e. optional) because ignoring this is fine for
older versions of Mercurial -- however, if there are any files that are
specially treated by the driver, we should abort. That will happen in upcoming
patches.
There is a potential security issue with storing the merge driver in the merge
state. See the inline comments for more details.
Not all bundles are appropriate for all clients. For example, someone
with a slow Internet connection may want to prefer bz2 bundles over gzip
bundles because they are smaller and don't take as long to transfer.
This is information that a server cannot know on its own. So, we invent
a mechanism for "preferring" server-advertised URLs based on their
attributes.
We could invent a negotiation between client and server where the client
sends its preferences and the sorting/filtering is done server-side.
However, this feels complex. We can avoid complicating the wire protocol
and exposing ourselves to backwards compatible concerns by performing
the sorting locally.
This patch defines a new config option for expressing preferred
attributes in server-advertised bundles.
At Mozilla, we leverage this feature so clients in fast data centers
prefer uncompressed bundles. (We advertise gzip bundles first because
that is a reasonable default.)
I consider this an advanced feature. I'm on the fence as to whether it
should be documented in `hg help config`.
An upcoming patch will enable clients to prefer certain bundles over
others. The idea is that we define values of attributes from manifests
that are desirable.
The BUNDLESPEC attribute is a complex value consisting of multiple
parts. Clients may wish to only prefer one of these parts. Having to
specify every combination of BUNDLESPEC would be annoying. So, we
extract the components of BUNDLESPEC into their own attributes so
clients can easily filter on a sub-component.
Server Name Indication (SNI) is commonly used in CDNs and other hosted
environments. Unfortunately, Python <2.7.9 does not support SNI and when
these older Python versions attempt to negotiate TLS to an SNI server,
they raise an opaque error like
"_ssl.c:507: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert
handshake failure."
We introduce a manifest attribute to denote the URL requires SNI and
have clients without SNI support filter these entries.
Not all clients are capable of reading every bundle. Currently, content
negotiation to ensure a server sends a client a compatible bundle
format is performed at request time. The response bundle is dynamically
generated at request time, so this works fine.
Clone bundles are statically generated *before* the request. This means
that a modern server could produce bundles that a legacy client isn't
capable of reading. Without some kind of "type hint" in the clone
bundles manifest, a client may attempt to download an incompatible
bundle. Furthermore, a client may not realize a bundle is incompatible
until it has processed part of the bundle (imagine consuming a 1 GB
changegroup bundle2 part only to discover the bundle2 part afterwards is
incompatibl). This would waste time and resources. And it isn't very
user friendly.
Clone bundle manifests thus need to advertise the *exact* format of the
hosted bundles so clients may filter out entries that they don't know
how to read. This patch introduces that mechanism.
We introduce the BUNDLESPEC attribute to declare the "bundle
specification" of the entry. Bundle specifications are parsed using
exchange.parsebundlespecification, which uses the same strings as the
"--type" argument to `hg bundle`. The supported bundle specifications
are well defined and backwards compatible.
When a client encounters a BUNDLESPEC that is invalid or unsupported, it
silently ignores the entry.
exchange.readbundle() can return 2 different types. We weren't handling
the bundle2 case. Handle it.
At some point we'll likely want a generic API for applying a bundle from
a file handle. For now, create another one-off until we figure out what
the unified bundle API should look like (addressing this is a can of
worms I don't want to open right now).
For the same reason, we move the bookmark related update logic into the
'destupdate' function. This requires to extend the returns of the function to
include the bookmark that needs to move (more or less) and the bookmark to
activate at the end of the function. See function documentation for details on
this returns.
The old code was tailored to `hg bundle` usage and not appropriate for
use as a general API, which clone bundles will require. The code has
been rewritten to make it more generally suitable.
We introduce dedicated error types to represent invalid and unsupported
bundle specifications. The reason we need dedicated error types (rather
than error.Abort) is because clone bundles will want to catch these
exception as part of filtering entries. We don't want to swallow
error.Abort on principle.
Clone bundles require a well-defined string to specify the type of
bundle that is listed so clients can filter compatible file types. The
`hg bundle` command and cmdutil.parsebundletype() already establish the
beginnings of a bundle specification format.
As part of formalizing this format specification so it can be used by
clone bundles, we move the specification parsing bits verbatim to
exchange.py, which is a more suitable place than cmdutil.py. A
subsequent patch will refactor this code to make it more appropriate as
a general API.
It's common for GUI or web frontend to fetch chunk of revisions per batch
size. Previously it was possible only if revisions were sorted by revision
number.
$ hg log -r 'limit({revspec} & :{last_known}, 101)'
So this patch introduces a general way to retrieve chunk of revisions after
skipping offset revisions.
$ hg log -r 'limit({revspec}, 100, {last_count})'
This is a dumb implementation. We can optimize it for baseset and spanset
later.
True/False value of '_pendingmode' means whether 'dirstate.pending' is
used to initialize own '_map' and so on. When it is None, neither
'dirstate' nor 'dirstate.pending' is read in yet.
This is used to keep consistent view between '_pl()' and '_read()'.
Once '_pendingmode' is determined by reading one of 'dirstate' or
'dirstate.pending' in, '_pendingmode' is kept even if 'invalidate()'
is invoked. This should be reasonable, because:
- effective 'invalidate()' invocation should occur only in wlock scope, and
- wlock can't be gotten under HG_PENDING mode
'_trypending()' is defined as a normal function to factor similar code
path (in bookmarks and phases) out in the future easily.
This patch delays writing in-memory changes out, if transaction is
running.
'_getfsnow()' is defined as a function, to hook it easily for
ambiguous timestamp tests (see also fakedirstatewritetime.py)
'if tr:' code path in this patch is still disabled at this revision,
because there is no client invoking 'dirstate.write()' with repo
object.
BTW, this patch changes 'dirstate.invalidate()' semantics around
'dirstate.write()' in a transaction scope:
before:
with repo.transaction():
dirstate.CHANGE('A')
dirstate.write() # change for A is written out here
dirstate.CHANGE('B')
dirstate.invalidate() # discards only change for B
after:
with repo.transaction():
dirstate.CHANGE('A')
dirstate.write() # change for A is still kept in memory
dirstate.CHANGE('B')
dirstate.invalidate() # discards changes for A and B
Fortunately, there is no code path expecting the former, at least, in
Mercurial itself, because 'dirstateguard' was introduced to remove
such 'dirstate.invalidate()'.
Some comments in this patch assume that subsequent patch changes
'dirstate.write()' like as below:
def write(self, repo):
if not self._dirty:
return
tr = repo.currenttransaction()
if tr:
tr.addfilegenerator('dirstate', (self._filename,),
self._writedirstate, location='plain')
return # omit actual writing out
st = self._opener('dirstate', "w", atomictemp=True)
self._writedirstate(st)
This patch makes '_savebackup()' write in-memory changes out, and it
causes clearing 'self._dirty'. If dirstate isn't changed after
'_savebackup()', subsequent 'dirstate.write()' never invokes
'tr.addfilegenerator()' because 'not self._dirty' is true.
Then, 'tr.writepending()' unintentionally returns False, if there is
no other (e.g. changelog) changes pending, even though dirstate
changes are already written out at '_savebackup()'.
To avoid such situation, this patch makes '_savebackup()' explicitly
invoke 'tr.addfilegenerator()', if transaction is running.
'_savebackup()' should get awareness of transaction before 'write()',
because the former depends on the behavior of the latter before this
patch.
This can centralize the logic to write in-memory changes out correctly
according to transaction activity into dirstate.
Passing 'repo' object to newly added functions is needed to examine
current transaction activity in subsequent patches, because 'dirstate'
itself doesn't have direct reference to it.
'localrepository.rollback()' explicilty restores dirstate, only if at
least one of current parents of the working directory is removed at
rollbacking (a.k.a "parent-gone").
After DirstateTransactionPlan, 'dirstate.write()' will cause marking
'.hg/dirstate' as a file to be restored at rollbacking.
https://mercurial.selenic.com/wiki/DirstateTransactionPlan
Then, 'transaction.rollback()' restores '.hg/dirstate' regardless of
parents of the working directory at that time, and this causes
unexpected dirstate changes if not "parent-gone" (e.g. "hg update" to
another branch after "hg commit" or so, then "hg rollback").
To avoid such situation, this patch restores dirstate to one before
rollbacking if not "parent-gone".
before:
b1. restore dirstate explicitly, if "parent-gone"
after:
a1. save dirstate before actual rollbacking via dirstateguard
a2. restore dirstate via 'transaction.rollback()'
a3. if "parent-gone"
- discard backup (a1)
- restore dirstate from 'undo.dirstate'
a4. otherwise, restore dirstate from backup (a1)
Even though restoring dirstate at (a3) after (a2) seems redundant,
this patch keeps this existing code path, because:
- it isn't ensured that 'dirstate.write()' was invoked at least once
while transaction running
If not, '.hg/dirstate' isn't restored at (a2).
In addition to it, rude 3rd party extension invoking
'dirstate.write()' without 'repo' while transaction running (see
subsequent patches for detail) may break consistency of a file
backup-ed by transaction.
- this patch mainly focuses on changes for DirstateTransactionPlan
Restoring dirstate at (a3) itself should be cheaper enough than
rollbacking itself. Redundancy will be removed in next step.
Newly added test is almost meaningless at this point. It will be used
to detect regression while implementing delayed dirstate write out.
On recent OS, 'stat.st_mtime' has a double precision floating point
value to represent nano seconds, but it is not wide enough for actual
file timestamp: nowadays, only 52 - 32 = 20 bit width is available for
decimal places in sec.
Therefore, casting it to 'int' may cause unexpected result. See also
changeset 8102a3981272 fixing issue4836 for detail.
For example, changed file A may be treated as "clean" unexpectedly in
steps below. "rounded now" is the value gotten by rounding via
'int(st.st_mtime)' or so.
---------------------+--------------------+------------------------
"now" | | timestamp of A (time_t)
float rounded time_t| action | FS dirstate
------ ------- ------+--------------------+-------- ---------------
N+.nnn N N | | --- ---
| update file A | N
| dirstate.normal(A) | N
N+.999 N+1 N | |
| dirstate.write() | N (*1)
| : |
| change file A | N
| : |
N+1.00 N+1 N+1 | |
| "hg status" (*2) | N N
------ ------- ------+--------------------+-------- ---------------
Timestamp N of A in dirstate isn't dropped at (*1), because "rounded
now" is N+1 at that time, even if 'st_mtime' in 'time_t' is still N.
Then, file A is unexpectedly treated as "clean" at (*2) in this case.
For consistent handling of 'stat.st_mtime', this patch makes
'pack_dirstate()' take 'now' argument not in floating point but in
integer.
This patch makes 'PyArg_ParseTuple()' in 'pack_dirstate()' use format
'i' (= checking type mismatch or overflow), even though it is ensured
that 'now' is in the range of 32bit signed integer by masking with
'_rangemask' (= 0x7fffffff) on caller side.
It should be cheaper enough than packing itself, and useful to
detect that legacy code invokes 'pack_dirstate()' with 'now' in
floating point value.
After moving logic from 'merge.update' into 'destutil.destupdate', we are now
moving logic from 'command.update' in 'destutil.destupdate'. This will make the
function actually useful in predicting (and altering) the update behavior.
We want this function to exactly predict the behavior for update. Moreover, we
would like to remove all high level behavior logic out of the merge module so
this is a step forward.
Now that the 'destupdate' function both compute and validate the destination, we
can directly use it at the command level, ensuring that the 'hg update' command
never call 'merge.update' without a defined destination. This is a first (but
significant) step toward having 'merge.update' always feed with a properly
validated destination and free of high level logic.
On windows, hgrc.d/*.rc would not be read if mercurial.ini was found. That was
far from obvious from the documentation and different from the behavior on
posix systems.
As a consequence of this, TortoiseHg cacert configuration placed in hgrc.d
would not be read if an old global mercurial.ini still existed.
"hg config -g" could also crash when no global configuration files could be
found.
Instead, make windows behave like posix and read all global configuration
files.
The documentation was in a way right that individual config settings in the
global Mercurial.ini would override settings from for example .hgrc.d\*.rc, but
only because the .d files not would be read at all if a Mercurial.ini was
found. The ordering in the documentation is thus changed to match the code.
Cloning can be an expensive operation for servers because the server
generates a bundle from existing repository data at request time. For
a large repository like mozilla-central, this consumes 4+ minutes
of CPU time on the server. It also results in significant network
utilization. Multiplied by hundreds or even thousands of clients and
the ensuing load can result in difficulties scaling the Mercurial server.
Despite generation of bundles being deterministic until the next
changeset is added, the generation of bundles to service a clone request
is not cached. Each clone thus performs redundant work. This is
wasteful.
This patch introduces the "clonebundles" extension and related
client-side functionality to help alleviate this deficiency. The
client-side feature is behind an experimental flag and is not enabled by
default.
It works as follows:
1) Server operator generates a bundle and makes it available on a
server (likely HTTP).
2) Server operator defines the URL of a bundle file in a
.hg/clonebundles.manifest file.
3) Client `hg clone`ing sees the server is advertising bundle URLs.
4) Client fetches and applies the advertised bundle.
5) Client performs equivalent of `hg pull` to fetch changes made since
the bundle was created.
Essentially, the server performs the expensive work of generating a
bundle once and all subsequent clones fetch a static file from
somewhere. Scaling static file serving is a much more manageable
problem than scaling a Python application like Mercurial. Assuming your
repository grows less than 1% per day, the end result is 99+% of CPU
and network load from clones is eliminated, allowing Mercurial servers
to scale more easily. Serving static files also means data can be
transferred to clients as fast as they can consume it, rather than as
fast as servers can generate it. This makes clones faster.
Mozilla has implemented similar functionality of this patch on
hg.mozilla.org using a custom extension. We are hosting bundle files in
Amazon S3 and CloudFront (a CDN) and have successfully offloaded
>1 TB/day in data transfer from hg.mozilla.org, freeing up significant
bandwidth and CPU resources. The positive impact has been stellar and
I believe it has proved its value to be included in Mercurial core. I
feel it is important for the client-side support to be enabled in core
by default because it means that clients will get faster, more reliable
clones and will enable server operators to reduce load without
requiring any client-side configuration changes (assuming clients are
up to date, of course).
The scope of this feature is narrowly and specifically tailored to
cloning, despite "serve pulls from pre-generated bundles" being a valid
and useful feature. I would eventually like for Mercurial servers to
support transferring *all* repository data via statically hosted files.
You could imagine a server that siphons all pushed data to bundle files
and instructs clients to apply a stream of bundles to reconstruct all
repository data. This feature, while useful and powerful, is
significantly more work to implement because it requires the server
component have awareness of discovery and a mapping of which changesets
are in which files. Full, clone bundles, by contrast, are much simpler.
The wire protocol command is named "clonebundles" instead of something
more generic like "staticbundles" to leave the door open for a new, more
powerful and more generic server-side component with minimal backwards
compatibility implications. The name "bundleclone" is used by Mozilla's
extension and would cause problems since there are subtle differences
in Mozilla's extension.
Mozilla's experience with this idea has taught us that some form of
"content negotiation" is required. Not all clients will support all
bundle formats or even URLs (advanced TLS requirements, etc). To ensure
the highest uptake possible, a server needs to advertise multiple
versions of bundles and clients need to be able to choose the most
appropriate from that list one. The "attributes" in each
server-advertised entry facilitate this filtering and sorting. Their
use will become apparent in subsequent patches.
Initial inspiration and credit for the idea of cloning from static files
belongs to Augie Fackler and his "lookaside clone" extension proof of
concept.
We perform all that we can non-interactively before prompting the user for input
via their merge tool. This allows for a maximally consistent state when the user
is first prompted.
The test output changes indicate the actual behavior change happening.
The section of code that writes out the version of the file cached in the merge
state should only be run at preresolve time. This is so that if the premerge
keeps around conflict markers, those don't get overwritten before the main
merge.
The current output for a failed merge with conflict markers looks something like:
merging foo
warning: conflicts during merge.
merging foo incomplete! (edit conflicts, then use 'hg resolve --mark')
merging bar
warning: conflicts during merge.
merging bar incomplete! (edit conflicts, then use 'hg resolve --mark')
We're going to change the way merges are done to perform all premerges before
all merges, so that the output above would look like:
merging foo
merging bar
warning: conflicts during merge.
merging foo incomplete! (edit conflicts, then use 'hg resolve --mark')
warning: conflicts during merge.
merging bar incomplete! (edit conflicts, then use 'hg resolve --mark')
The 'warning: conflicts during merge' line has no context, so is pretty
confusing.
This patch will change the future output to:
merging foo
merging bar
warning: conflicts while merging foo! (edit, then use 'hg resolve --mark')
warning: conflicts while merging bar! (edit, then use 'hg resolve --mark')
The hint on how to resolve the conflicts makes this a bit unwieldy, but solving
that is tricky because we already hint that people run 'hg resolve' to retry
unresolved merges. The 'hg resolve --mark' mostly applies to conflict marker
based resolution.
This means that in ms.resolve we must call merge after calling premerge. This
doesn't yet mean that all premerges happen before any merges -- however, this
does get us closer to our goal.
The output differences are because we recompute the merge tool. The only
user-visible difference caused by this patch is that if the tool is missing
we'll print the warning twice. Not a huge deal, though.
We're soon going to call this function twice, once for premerge and once for
merge. This makes sure the "merging" output only gets printed during the
premerge step.
af5de4d23fd4 introduced nice hexified display of missing nodes. It did however
also make missing 20 character revision specifications be shown as hex - very
confusing.
Users are often wrong and somehow specify revisions that don't exist. Nodes
will however rarely be missing ... and they will only look like a user provided
revision specification and be all ascii in 1 of 4*10**9.
With this change, missing revisions will only be hexified if they really look
like binary nodes. This change will thus improve the error reporting UI in the
common case and only very rarely make it confusing in the opposite direction of
how it was before.
Having the simple and tiny branch of the conditional first help readability. The
"else" that appears after a screen of code is harder to relate to a conditional.
The current setup requires to pass both a packer and, optionally, the version
of the unpacker. This is confusing and error prone as the two value cannot
mismatch. Instead, we simply grab the version from the packer. This fixes a bug
where requesting a cg2 from 'hg bundle' were reported as changegroup 1.
I should have caught that in the initial changeset but I missed it somehow.
The issue4888 was caused by 0-length obsolete marker. If msize is zero,
fm1readmarkers() never ends.
This patch adds several bound checks to fm1readmarker(). Therefore, 0-length
and invalid-size marker should be rejected.
This will make it easy to implement bound checking. Currently fm1readmarker()
has no protection for corrupted obsstore and can cause infinite loop or
out-of-bound reads.
The home of 'Abort' is 'error' not 'util' however, a lot of code seems to be
confused about that and gives all the credit to 'util' instead of the
hardworking 'error'. In a spirit of equity, we break the cycle of injustice and
give back to 'error' the respect it deserves. And screw that 'util' poser.
For great justice.
Before this patch, "hg backout" uses 'begin'/'end'-'parentchange()'
of 'dirstate' class to avoid writing incomplete dirstate changes out
at failure.
But this framework doesn't work as expected, if 'dirstate.write()' is
invoked between them. In fact, in-memory dirstate changes may be
written out at 'repo.status()' implied by 'merge.update()', even
before this patch.
To restore dirstate as expected at failure of "hg backout", this patch
uses 'dirstateguard' instead of 'begin'/'end'-'parentchange()'.
Previous patch made dirstate changes in a transaction scope "all or
nothing". Therefore, 'dirstateguard' is meaningless, if its scope is
as same as one of the related transaction.
Before this patch, "hg import" uses 'dirstateguard' always, but
transaction is also started if '--no-commit' isn't specified.
To avoid redundancy, this patch makes "hg import" use dirstateguard
only if transaction isn't started (= '--no-commit' is specified).
In this patch, 'if dsguard' can be examined safely, because 'dsguard'
is initialized (with None) before outermost 'try'.
There is no user of 'cmdutil.tryimportone()' other than
'commands.import_()', which can restore dirstate at failure of
applying patches by transaction or dirstateguard.
Therefore, it is reasonable to stop 'tryimportone()' from using
redundant 'dirstateguard', even though it changes behavior of
'tryimportone()'.
After this patch, 3rd party extensions should use 'dirstateguard' or
so explicitly, if they want to restore dirstate at failure of
importing a patch.
Previous patch made dirstate changes in a transaction scope "all or
nothing". Therefore, 'dirstateguard' is meaningless, if its scope is
as same as one of the related transaction.
This patch removes such meaningless 'dirstateguard' usage.
Before this patch, in-memory dirstate changes are still kept over a
transaction scope boundary regardless of the result of it.
For "all or nothing" policy of the transaction, in-memory dirstate
changes should be:
- written out at successful closing a transaction, because
subsequent 'dirstate.invalidate()' can lose them
- discarded at failure of a transaction, because outer
'wlock.release()' or so may write them out
To discard all changes in a transaction completely, this patch also
restores '.hg/dirstate' by '.hg/journal.dirstate' at failure, because
'transaction' itself does nothing for files related to '.hg/journal.*'
in such case (therefore, renaming in this patch is safe enough).
This is a part of preparations for "transactional dirstate". See also
the wiki page below for detail about it.
https://mercurial.selenic.com/wiki/DirstateTransactionPlan
This patch also removes redundant 'dirstate.invalidate()' just before
aborting a transaction for shelve/unshelve.
'releasefn' is used by subsequent patch, to do appropriate action
according to the result of it at the end of a transaction scope.
To ensure that 'releasefn' is invoked only once, this patch invokes it
after assignment 'self.journal = None', because such assignment
prevents from invoked 'transaction._abort()' again via '__del__()'.
def __del__(self):
if self.journal:
self._abort()
We ultimately want this to be accessible through a revset, but there is too
much complexity here for that to work. Especially we'll have to return more
than just the destination to control the behavior (eg: bookmarks to activate,
etc).
To prevent cycle, a new module is created, it will receive other
destination/behavior function in the future.
This is a backout of 0f1a7b0ccb69. The stdlib implementation of
multiprocessing.cpu_count() attempts to invoke a process on BSD and
Darwin platforms (at least on 2.7). Under certain conditions (such as
cwd being removed) this could raise. Our old code was silently catching
the exception.
The old code was more robust, so restore it.
The merge code currently does (in pseudocode):
for f in tomerge:
premerge f
merge f
We'd like to change this to look more like:
for f in tomerge:
premerge f
for f in tomerge:
merge f
This makes sure as many files are resolved as possible before prompting for the
others. This restructuring is also necessary for custom merge drivers.
This function separates out the premerge step from the merge step. In future
patches we'll actually turn these into separate steps in the merge driver.
The 'if r:' occurrences will be cleaned up in subsequent patches.
In the external pushrebase extension, it is valuable to be able to do some work
without taking the lock (like running expensive hooks). This enables
significantly higher commit throughput.
This patch adds an option to lazily acquire the lock. It means that all bundle2
part handlers that require writing to the repo must first call
op.gettransction(), when in this mode.
A future patch will allow the bundle2 lock be taken lazily. We need to
introduce transaction-gets to each handler that needs the lock.
The tests caught these issues when I added lazy locking.
It was introduced by 236440938a03, but the important code was removed by
fcf2407610f4. So there was no positive effect other than exhausting memory.
The problem spotted by 236440938a03 is that you can't use a generator keyword
more than once. For example, in hgweb template, "{child} {child}" doesn't work
because the first "{child}" consumes the generator. But as fcf2407610f4 says,
the fix was wrong because it could overwrite a callable keyword that returns
a generator. Also the fix didn't work for a generator of generator such as
"{diff}" keyword. So, the proper fix for that problem would be to not put
a generator in a keyword table. Instead, it should be a factory of a generator.
Note that this should fix the memory issue in hgweb, but my firefox killed by
OOM in place. Be careful to not use a modern web browser to test the issue4868.
As we have a way for extension to add more header, we need a way for them to
actually process them. We add a basic hook point to do extra work after the
import have been committed.
As we have a way for extension to add more header, we need a way for them to
actually process them. We add a basic hook points to alter the changeset
(especially extra) before we commit. There would be more to do for a full
featured hooking, but this currently fit my needs.
The final goal here is to be able to parse, return and process arbitrary data
from patch. This mirror the recently added ability to add arbitrary data to
patch headers.
The first step is to return something more flexible, so we return a dict without
changing any other logic.
Extensions currently have no easy way to add data to exported
patch. This is now fixed using a generic mechanism in the same fashion
used by bundle2. Tests are coming in the next changeset with its first
user.
Incoming was using bundle1 in all cases, as bundle1 is restricted to
changegroup1 and does not support general delta, this can lead to significant
CPU overhead if the server is using general delta storage. We now properly
request and store a bundle2 to disk.
If the server include any output or error in the bundle, they will be stored on
disk and replayed when the bundle is read. As 'hg incoming' is going to read the
bundle right away, we call that 'good' enough and go back to the bigger plan of
having general delta on by default.
This was tracked as 4864
There is use case for directly forward and bundle2 stream from the peer to a
file (eg: 'hg incoming --bundle'), However ssh peers have no way to know the
'getbundle' is over except for actually interpreting the bundle. So we need to
have the unbundle do the interpreting and forward job.
The function is marked as private to highlight that this is terrible and that we
are sorry.
