We're soon going to call this function twice, once for premerge and once for
merge. This makes sure the "merging" output only gets printed during the
premerge step.
af5de4d23fd4 introduced nice hexified display of missing nodes. It did however
also make missing 20 character revision specifications be shown as hex - very
confusing.
Users are often wrong and somehow specify revisions that don't exist. Nodes
will however rarely be missing ... and they will only look like a user provided
revision specification and be all ascii in 1 of 4*10**9.
With this change, missing revisions will only be hexified if they really look
like binary nodes. This change will thus improve the error reporting UI in the
common case and only very rarely make it confusing in the opposite direction of
how it was before.
Having the simple and tiny branch of the conditional first help readability. The
"else" that appears after a screen of code is harder to relate to a conditional.
The current setup requires to pass both a packer and, optionally, the version
of the unpacker. This is confusing and error prone as the two value cannot
mismatch. Instead, we simply grab the version from the packer. This fixes a bug
where requesting a cg2 from 'hg bundle' were reported as changegroup 1.
I should have caught that in the initial changeset but I missed it somehow.
The issue4888 was caused by 0-length obsolete marker. If msize is zero,
fm1readmarkers() never ends.
This patch adds several bound checks to fm1readmarker(). Therefore, 0-length
and invalid-size marker should be rejected.
This will make it easy to implement bound checking. Currently fm1readmarker()
has no protection for corrupted obsstore and can cause infinite loop or
out-of-bound reads.
The home of 'Abort' is 'error' not 'util' however, a lot of code seems to be
confused about that and gives all the credit to 'util' instead of the
hardworking 'error'. In a spirit of equity, we break the cycle of injustice and
give back to 'error' the respect it deserves. And screw that 'util' poser.
For great justice.
Before this patch, "hg backout" uses 'begin'/'end'-'parentchange()'
of 'dirstate' class to avoid writing incomplete dirstate changes out
at failure.
But this framework doesn't work as expected, if 'dirstate.write()' is
invoked between them. In fact, in-memory dirstate changes may be
written out at 'repo.status()' implied by 'merge.update()', even
before this patch.
To restore dirstate as expected at failure of "hg backout", this patch
uses 'dirstateguard' instead of 'begin'/'end'-'parentchange()'.
Previous patch made dirstate changes in a transaction scope "all or
nothing". Therefore, 'dirstateguard' is meaningless, if its scope is
as same as one of the related transaction.
Before this patch, "hg import" uses 'dirstateguard' always, but
transaction is also started if '--no-commit' isn't specified.
To avoid redundancy, this patch makes "hg import" use dirstateguard
only if transaction isn't started (= '--no-commit' is specified).
In this patch, 'if dsguard' can be examined safely, because 'dsguard'
is initialized (with None) before outermost 'try'.
There is no user of 'cmdutil.tryimportone()' other than
'commands.import_()', which can restore dirstate at failure of
applying patches by transaction or dirstateguard.
Therefore, it is reasonable to stop 'tryimportone()' from using
redundant 'dirstateguard', even though it changes behavior of
'tryimportone()'.
After this patch, 3rd party extensions should use 'dirstateguard' or
so explicitly, if they want to restore dirstate at failure of
importing a patch.
Previous patch made dirstate changes in a transaction scope "all or
nothing". Therefore, 'dirstateguard' is meaningless, if its scope is
as same as one of the related transaction.
This patch removes such meaningless 'dirstateguard' usage.
Before this patch, in-memory dirstate changes are still kept over a
transaction scope boundary regardless of the result of it.
For "all or nothing" policy of the transaction, in-memory dirstate
changes should be:
- written out at successful closing a transaction, because
subsequent 'dirstate.invalidate()' can lose them
- discarded at failure of a transaction, because outer
'wlock.release()' or so may write them out
To discard all changes in a transaction completely, this patch also
restores '.hg/dirstate' by '.hg/journal.dirstate' at failure, because
'transaction' itself does nothing for files related to '.hg/journal.*'
in such case (therefore, renaming in this patch is safe enough).
This is a part of preparations for "transactional dirstate". See also
the wiki page below for detail about it.
https://mercurial.selenic.com/wiki/DirstateTransactionPlan
This patch also removes redundant 'dirstate.invalidate()' just before
aborting a transaction for shelve/unshelve.
'releasefn' is used by subsequent patch, to do appropriate action
according to the result of it at the end of a transaction scope.
To ensure that 'releasefn' is invoked only once, this patch invokes it
after assignment 'self.journal = None', because such assignment
prevents from invoked 'transaction._abort()' again via '__del__()'.
def __del__(self):
if self.journal:
self._abort()
We ultimately want this to be accessible through a revset, but there is too
much complexity here for that to work. Especially we'll have to return more
than just the destination to control the behavior (eg: bookmarks to activate,
etc).
To prevent cycle, a new module is created, it will receive other
destination/behavior function in the future.
This is a backout of 0f1a7b0ccb69. The stdlib implementation of
multiprocessing.cpu_count() attempts to invoke a process on BSD and
Darwin platforms (at least on 2.7). Under certain conditions (such as
cwd being removed) this could raise. Our old code was silently catching
the exception.
The old code was more robust, so restore it.
The merge code currently does (in pseudocode):
for f in tomerge:
premerge f
merge f
We'd like to change this to look more like:
for f in tomerge:
premerge f
for f in tomerge:
merge f
This makes sure as many files are resolved as possible before prompting for the
others. This restructuring is also necessary for custom merge drivers.
This function separates out the premerge step from the merge step. In future
patches we'll actually turn these into separate steps in the merge driver.
The 'if r:' occurrences will be cleaned up in subsequent patches.
In the external pushrebase extension, it is valuable to be able to do some work
without taking the lock (like running expensive hooks). This enables
significantly higher commit throughput.
This patch adds an option to lazily acquire the lock. It means that all bundle2
part handlers that require writing to the repo must first call
op.gettransction(), when in this mode.
A future patch will allow the bundle2 lock be taken lazily. We need to
introduce transaction-gets to each handler that needs the lock.
The tests caught these issues when I added lazy locking.
It was introduced by 236440938a03, but the important code was removed by
fcf2407610f4. So there was no positive effect other than exhausting memory.
The problem spotted by 236440938a03 is that you can't use a generator keyword
more than once. For example, in hgweb template, "{child} {child}" doesn't work
because the first "{child}" consumes the generator. But as fcf2407610f4 says,
the fix was wrong because it could overwrite a callable keyword that returns
a generator. Also the fix didn't work for a generator of generator such as
"{diff}" keyword. So, the proper fix for that problem would be to not put
a generator in a keyword table. Instead, it should be a factory of a generator.
Note that this should fix the memory issue in hgweb, but my firefox killed by
OOM in place. Be careful to not use a modern web browser to test the issue4868.
As we have a way for extension to add more header, we need a way for them to
actually process them. We add a basic hook point to do extra work after the
import have been committed.
As we have a way for extension to add more header, we need a way for them to
actually process them. We add a basic hook points to alter the changeset
(especially extra) before we commit. There would be more to do for a full
featured hooking, but this currently fit my needs.
The final goal here is to be able to parse, return and process arbitrary data
from patch. This mirror the recently added ability to add arbitrary data to
patch headers.
The first step is to return something more flexible, so we return a dict without
changing any other logic.
Extensions currently have no easy way to add data to exported
patch. This is now fixed using a generic mechanism in the same fashion
used by bundle2. Tests are coming in the next changeset with its first
user.
Incoming was using bundle1 in all cases, as bundle1 is restricted to
changegroup1 and does not support general delta, this can lead to significant
CPU overhead if the server is using general delta storage. We now properly
request and store a bundle2 to disk.
If the server include any output or error in the bundle, they will be stored on
disk and replayed when the bundle is read. As 'hg incoming' is going to read the
bundle right away, we call that 'good' enough and go back to the bigger plan of
having general delta on by default.
This was tracked as 4864
There is use case for directly forward and bundle2 stream from the peer to a
file (eg: 'hg incoming --bundle'), However ssh peers have no way to know the
'getbundle' is over except for actually interpreting the bundle. So we need to
have the unbundle do the interpreting and forward job.
The function is marked as private to highlight that this is terrible and that we
are sorry.
We want to introduce a simple way to forward the content of a bundle2 stream.
For this purpose, we will need to both yield the parameters block and process it
(to apply any behavior change it might indicate).
The current writebundle function do two things:
- taking a changegroup-packer instance and storing it into a valid bundle with
proper header.
- creating a temporary or requested file to store that bundle
We would like to make it easier to forward bundle stream directly from a remote
peer to a file, so we split the two logic to be able to skip the one about
building a valid bundle (the remote is already sending one).
When users configure the default foreground or background color to
non-default (black on white) values, several hgweb styles lack
contrast for headers and table row items. This patch fixes that by
ensuring that where either foreground or background colors are
specified, both are specified.
We can safely drop this because the very same assignment is enforcement later in
the function. Dropping it will make it simpler to extract the default
destination logic in its own function.
As bundle1 does not support generaldelta, this would mean recomputing delta at
bundle time. This is similar to what we do for strip and shelve and was tracked
as issue4865.
We had some basic undocumented support for uncompressed bundle2 support. We now
have an official extensible syntax to specify both format type and compression
(eg: bzip2-v2).
In practice, this changeset introduce the 'v1' and 'v2' identifier to make it
possible to combine format and compression. The default format is still 'v1'.
We'll care about picking 'v1' or 'v2' in regard with general delta in the next
changesets.
Before this patch, 'bmstore.write()' always write in-memory bookmark
changes into '.hg/bookmarks' regardless of transaction activity.
If 'bmstore.write()' is invoked inside a transaction and it writes
changes into '.hg/bookmarks', then:
- original bookmarks aren't restored at failure of that transaction
This breaks "all or nothing" policy of the transaction.
BTW, "hg rollback" can restore bookmarks successfully even before
this patch, because original bookmarks are saved into
'.hg/journal.bookmarks' at the beginning of the transaction, and
it (actually renamed as '.hg/undo.bookmarks') is used by "hg
rollback".
- uncommitted bookmark changes are visible to other processes
This is a kind of "dirty read"
For example, 'rebase.rebase()' implies 'bmstore.write()', and it may
be executed inside the transaction of "hg unshelve". Then, intentional
aborting at the end of "hg unshelve" transaction doesn't restore
original bookmarks (this is obviously a bug).
This patch uses 'bmstore.recordchange()' instead of actual writing by
'bmstore._writerepo()', if any transaction is active
This patch also removes meaningless restoring bmstore explicitly at
the end of "hg shelve".
This patch doesn't choose fixing each 'bmstore.write()' callers as
like below, because writing similar code here and there is very
redundant.
before:
bmstore.write()
after:
tr = repo.currenttransaction()
if tr:
bmstore.recordchange(tr)
else:
bmstore.write()
Even though 'bmstore.write()' itself may have to be discarded by
putting bookmark operations into transaction scope, this patch chose
fixing it to implement "transactional dirstate" at first.
We're going to separate the pre-merge and merge steps for merge tools. The
merge step will be specific to the tool, but the pre-merge step will be common
to all merge tools that need it.
However, some merge tools run checks *before* the pre-merge step. This callback
will allow that to continue to work.
We are going to introduce significant extensions of the bundle parsing code to
support creation of bundle2 through the bundle command. As an early step, we
extract the logic in its own function.
For some obscure reasons (probably upsetting a Greek goddess),
getchangegroup did not had a 'version' argument to control the changegroup
version. We fixes this to allow cg02 to be used with 'hg bundle' in the future.
For some obscure reasons (probably upsetting a Greek goddess),
getlocalchangegroup did not have a 'version' argument to control the
changegroup version. We fix this to allow cg02 to be used with 'hg
bundle' in the future.
Review feedback from Pierre-Yves David. A separate line of work is working to
ensure that dirstate writes are written to a separate 'pending' file while a
transaction is active. Lock inheritance currently conflicts with that, so dodge
the issue by simply preventing inheritance while a transaction is running.
Custom merge drivers aren't going to run inside a transaction, so this doesn't
affect that.
We want to prevent locks from being inherited sometimes (e.g. when there's a
currently running transaction, which will break a lot of assumptions we're
making in here.)
Because st.st_mtime is computed as 'sec + 1e-9 * nsec' and double is too narrow
to represent nanoseconds, int(st.st_mtime) can be 'sec + 1'. Therefore, that
value could be different from the one got by osutils.listdir().
This patch fixes the problem by accessing to raw st_mtime by tuple index.
It catches TypeError to fall back to st.st_mtime because our osutil.stat does
not support tuple index. In dirstate.normal(), 'st' is always a Python stat,
but in dirstate.status(), it can be either a Python stat or an osutil.stat.
Thanks to vgatien-baron@janestreet.com for finding the root cause of this
subtle problem.
This is part of a series that will allow locks to be inherited by subprocesses
in limited circumstances.
In the future, we'll call this for custom merge drivers.
This is part of a series that will allow locks to be inherited by subprocesses
in limited circumstances.
In an upcoming patch, we'll add an API for the wlock to be inherited.
This will be useful to pass around a reference to the lock to some functions
we're going to add to scmutil. We don't want those functions to live in
localrepo to avoid bloat.
Because _phaserevs and _phasesets cache revision numbers, they must be
invalidated if there are new commits or stripped revisions. We could do
that by calling _phasecache.invalidate(), but it wasn't simple to be
integrated with the filecache mechanism.
So for now, phasecache will be recreated after repo.invalidate() if
00changelog.i was modified before.
