This is based on a patch proposed last year by Mathias De Maré[1], with a few
changes.
- Tags and bookmarks are now formatted lists, for more flexible queries.
- The templater is populated whether or not [-nibtB] is specified. (Plain
output is unchanged.) This seems more consistent with other templated
commands.
- The 'id' property is a string, instead of a list.
- The parents of 'wdir()' have their own list of attributes.
I left 'id' as a string because it seems very useful for generating version
info. It's also a bit strange because the value and meaning changes depending
on whether or not --debug is passed (short vs full hash), whether the revision
is a merge or not (one hash or two, separated by a '+'), the working directory
or not (node vs p1node), and local or not (remote defaults to tip, and never has
'+'). The equivalent string built with {rev} seems much less useful, and I
couldn't think of a reasonable name, so I left it out.
The discussion seemed to be pointing towards having a list of nodes, with more
than one entry for a merge. It seems simpler to give the nodes a name, and use
{node} for the actual commit probed, especially now that there is a virtual node
for 'wdir()'.
Yuya mentioned using fm.nested() in that thread, so I did for the parent nodes.
I'm not sure if the plan is to fill in all of the context attributes in these
items, or if these nested items should simply be made {p1node} and {p1rev}.
I used ':' as the tag separator for consistency with {tags} in the log
templater. Likewise, bookmarks are separated by a space for consistency with
the corresponding log template.
[1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2016-August/087039.html
We are about to remove the branchmap cache update in changegroup application.
There is a debug message alongside this update that we do not want to loose. We
move the message beforehand to simplify the test update in the next changeset.
The message move is quite noisy and isolating that noise is useful.
Most tests update are just line reordering since the message is issued at a
later point during the transaction.
After this changes, the message is displayed in more case since local commit
creation also issue it.
The python hooks have access to the hook type information. There is not reason
for external hook to not be aware of it too.
For the record my use case is to make sure a hook script is configured for the
right type.
Hooks related to the transaction are aware of the transaction id. By definition
this txn-id is unique and different for each transaction. As a result it can
never be predicted in test and always needs matching. As a result, touching any
like with this data is annoying. We solve the problem once and for all by
installing an automatic replacement. In test, this will now show as:
TXNID=TXN:$ID$
Before this patch, checking HG_PENDING for changelog in localrepo.py
might cause unintentional reading unrelated '00changelog.i.a' in,
because HG_PENDING is checked by str.startswith().
An external hook spawned by inner repository in nested ones satisfies
this condition.
This patch uses txnutil.mayhavepending() to check HG_PENDING strictly.
BTW, this patch may cause failure of bisect in the repository of
Mercurial itself, if examination at bisecting assumes that an external
hook can see all pending changes while nested transactions across
repositories.
This invisibility issue will be fixed by subsequent patch, which
allows HG_PENDING to refer multiple repositories.
On Windows platform, invoking printenv.py directly via hook is
problematic, because:
- unless binding between *.py suffix and python runtime, application
selector dialog is displayed, and running test is blocked at each
printenv.py invocations
- it isn't safe to assume binding between *.py suffix and python
runtime, because application binding is easily broken
For example, installing IDE (VisualStudio with Python Tools, or
so) often requires binding between source files and IDE itself.
This patch invokes printenv.py via sh -c for test portability. This is
a kind of follow up for 9e4331825bea, which eliminated explicit
"python" for printenv.py. There are already other 'sh -c "printenv.py"'
in *.t files, and this fix should be reasonable.
This changes were confirmed in cases below:
- without any application binding for *.py suffix
- with binding between *.py suffix and VisualStudio
This patch also replaces "echo + redirection" style with "heredoc"
style, because:
- hook command line is parsed by cmd.exe as shell at first, and
- single quotation can't quote arguments on cmd.exe, therefore,
- "printenv.py foobar" should be quoted by double quotation, but
- nested quoting (or tricky escaping) isn't readable
The partial bundle is not a subset of the full bundle, and the full
bundle is not full in any way that i see. The most obvious
interpretation of "full" I can think of is that it has all commits
back to the null revision, but that is not what the "full" bundle
is. The "full" bundle is simply a backup of what the user asked us to
strip (unless --no-backup). The "partial" bundle contains the
revisions we temporarily stripped because they had higher revision
numbers that some commit that the user asked us to strip.
The "full" bundle is already called "backup" in the code, so let's use
that in user-facing messages too. Let's call the "partial" bundle
"temporary" in the code.
This parameter is slightly confusingly named in wireproto, so it got
mis-specified from the start as 'push' instead of the URL to which we
are pushing. Sigh. I've got a patch for that which I'll mail
separately since it's not really appropriate for stable.
Fixes a regression in bundle2 from bundle1.
getbundle was requesting the "phase" namespace instead of the "phases"
namespace, which led to the client still requesting the phases
separately after getbundle finished.
Before this patch, there was no way for a repository owner to ensure that
validation hooks would be run by people with write access. If someone had write
access but did not trust the user owning the repository, the config and its hook
would simply be ignored.
After this patch, hooks from untrusted configs are taken into account but never
actually run. Instead they are reported as failures right away. This will ensure
validation performed by a hook is not ignored.
As a side effect writer can be forced to trust a repository hgrc by adding a
'pretxnopen.trust=true' hook to the file.
This was discussed during the 3.8 sprint with Matt Mackall, Augie Fackler and
Kevin Bullock.
We had some real-world cases where syntax errors in Python hooks would crash
the whole process and leave it in an indeterminate state. Handle those better.
This matches 'hook failed' warnings.
We're also going to add hints to some of the hook load errors. Without this
change we'd have two pairs of parens for a single error message, which looks
really cluttered.
The new transaction context did not handle the case where an exception during
close should still call release. This cause pretxnclose hooks that failed to
cause the transaction to fail without aborting, thus requiring a hg recover.
I've added a test.
Sometimes a txnclose or changegroup hook wants to iterate through all
the changesets in transaction: in that situation usually the revset
`$HG_NODE:` is used to select the revisions. Unfortunately this revset
sometimes may contain too many changesets because we don't have the
write lock while the hook runs newer changes may be added to
repository in the meantime.
That's why there is a need for extra variable carrying the information about
the last change in the transaction.
is None (issue4983)
Some hooks, such as post-init and post-clone, do not get a repo parameter in
their environment. If there is no repo, there is no repo.currenttransaction();
attempting to retrieve it anyway was causing crashes. Now currenttransaction is
only retrieved and written if the repo is not None.
Changeset e7b51de6e8eb alters the 'HG_PENDING' mechanism to be "always" there.
This change is made under the assumption than we previously did it only when
"writepending() actually wrote something". This assumption was wrong,
'writepending()' informs of pending changes the first time something is written
and for all following calls. We back this change out to restore the former
behavior, which was already correct.
We need to call delayupdate again after writing to the changelog.
Otherwise the prechangegroup hook consumes the delayupdate subscription and
future hooks don't see the pending changes (see issue 4934 for more details).
Adds a test that triggers the prechangegroup hook before the pretxnchangegroup
hook and verifies that the output of pretxnchangegroup doesn't change.
Previously we would only include HG_PENDING in the hook args if the
transaction's writepending() actually wrote something. This is a bad criteria,
since it's possible that a previous call to writepending() wrote stuff and the
hooks want to still see that.
The solution is to always have hooks execute within the scope of the pending
changes by always putting HG_PENDING in the environment.
Before this patch, "hg commit" (process A) executes steps below:
1. get current branch heads via 'repo.branchheads()'
- cache 'repo.changelog'
2. invoke 'repo.commit()'
3. acquire wlock
- invalidate 'repo.dirstate'
4. access 'repo.dirstate'
- re-read '.hg/dirstate'
- check validity of parent revisions with 'repo.changelog'
5. invoke 'repo.commitctx()'
6. acquire store lock (slock)
- invalidate 'repo.changelog'
7. do committing
8. release slock
9. release wlock
10. check new branch head (via 'cmdutil.commitstatus()')
If acquisition of wlock at (3) above waits for another "hg commit"
(process B) or so running parallelly to release wlock, process A
causes creating orphan revision, because:
- '.hg/dirstate' refers the revision, which is newly added by
process B, as its parent
- but already cached 'repo.changelog' doesn't contain such revision
- therefore, validating parents of '.hg/dirstate' at (4) above
replaces such revision with 'nullid'
Then, process A creates "orphan" revision, of which parent is "null"
revision.
In addition to it, "created new head" may be shown at the end of
process A unintentionally, if store is updated parallelly, because
both getting branch heads (1) and checking new branch head (10) are
executed outside slock scope.
To avoid this issue, this patch makes "hg commit" acquire wlock and
slock before processing.
This patch resolves the issue between "hg commit" processes, but not
one between "hg commit" and other commands. Subsequent patches resolve
the latter.
Even after this patch, there are still corner case problems below:
- filecache may overlook changes of '.hg/dirstate', and it causes
similar issue (see below for detail)
https://bz.mercurial-scm.org/show_bug.cgi?id=4368#c10
- 3rd party extension may cause similar issue, if it directly uses
'repo.commit()' without acquisition of wlock and slock
This can be fixed by acquisition of slock at the beginning of
'repo.commit()', but it seems suitable for "default" branch
In fact, acquisition of slock itself is already introduced at
"default" branch by ec227b188932, but acquisition is not at the
beginning of 'repo.commit()'.
This patch also changes some tests:
- test-fncache.t needs this tricky wrapping, to release (= forced
failure of) wlock certainly
- order of "hg commit" output is changed by widening scope of locks,
because some hooks are fired after releasing wlock
c67339617276 (while 3.4 code-freeze) made all 'update' hooks run after
releasing wlock for visibility of in-memory dirstate changes. But this
breaks paired invocation of 'preupdate' and 'update' hooks.
For example, 'hg backout --merge' for TARGET revision, which isn't
parent of CURRENT, consists of steps below:
1. update from CURRENT to TARGET
2. commit BACKOUT revision, which backs TARGET out
3. update from BACKOUT to CURRENT
4. merge TARGET into CURRENT
Then, we expects hooks to run in the order below:
- 'preupdate' on CURRENT for (1)
- 'update' on TARGET for (1)
- 'preupdate' on BACKOUT for (3)
- 'update' on CURRENT for (3)
- 'preupdate' on TARGET for (4)
- 'update' on CURRENT/TARGET for (4)
But hooks actually run in the order below:
- 'preupdate' on CURRENT for (1)
- 'preupdate' on BACKOUT for (3)
- 'preupdate' on TARGET for (4)
- 'update' on TARGET for (1), but actually on CURRENT/TARGET
- 'update' on CURRENT for (3), but actually on CURRENT/TARGET
- 'update' on CURRENT for (4), but actually on CURRENT/TARGET
Root cause of the issue focused by c67339617276 is that external
'update' hook process can't view in-memory changes (especially, of
dirstate), because they aren't written out until the end of
transaction (or wlock).
Now, hooks can be invoked just after updating, because previous
patches made in-memory changes visible to external process.
This patch may break backward compatibility from the point of view of
"scheduling hook execution", but should be reasonable because 'update'
hooks had been executed in this order before 3.4.
This patch tests "hg backout" and "hg unshelve", because the former
activates the transaction before 'update' hook invocation, but the
former doesn't.
The home of 'Abort' is 'error' not 'util' however, a lot of code seems to be
confused about that and gives all the credit to 'util' instead of the
hardworking 'error'. In a spirit of equity, we break the cycle of injustice and
give back to 'error' the respect it deserves. And screw that 'util' poser.
For great justice.
The update command is touching the repository and should lock it for
the length of its operations. Equally importantly, it should lock the
repository when it is writing bookmarks. It wasn't doing so until now,
leaving doors open for all kinds of drunk beaver parties.
This results in some minor tests changes, and the fixing of a couple
of bugs from race conditions.
Code does not receive any changes beside extra indentation.
For some time, bookmark can and should be moved in the transaction. This
changeset migrates the 'hg bookmarks' commands to use a transaction.
Tests regarding rollback and transaction hooks are impacted for
obvious reasons. Some have to be slightly updated to keep testing the
same things. Some can just be dropped because they do not make sense
anymore.
When using bundle2, the bookmark's pushkey parts are now made mandatory. As a
result failure to update the bookmark server side will result in the transaction
being aborted.
We are already fetching remote bookmarks to honor the -B option, we
now pass that data to the pull process so it can reuse it. This
prevents a race condition between the initial looking and the actual
pulling of changesets and bookmarks. Tests are updated to handle this
fact.
Using bundle2 has an effect on which hooks are run when. We turn it on for
test-hooks early to reduce the noise of switching the default exchange to
bundle2.
Before this patch, hook argument `txnid` isn't passed to `pretxnopen`
hooks, even though `hooks` section of `hg help config` describes so.
``pretxnopen``
Run before any new repository transaction is open. The reason for the
transaction will be in ``$HG_TXNNAME`` and a unique identifier for the
transaction will be in ``HG_TXNID``. A non-zero status will prevent the
transaction from being opened.
There were 2 test failures in 3.4-rc when running test-hook.t with the
largefiles extension enabled. For context, the first is a commit hook:
@@ -618,9 +621,9 @@
$ echo 'update = hg id' >> .hg/hgrc
$ echo bb > a
$ hg ci -ma
- 223eafe2750c tip
+ d3354c4310ed+
$ hg up 0
- cb9a9f314b8b
+ 223eafe2750c+ tip
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
make sure --verbose (and --quiet/--debug etc.) are propagated to the local ui
In both cases, largefiles acquires the wlock before calling into core, which
also acquires the wlock. The first case was fixed in 4100e338a886 by ensuring
the hook only runs after the lock has been fully released. The full release is
important, because that is what writes dirstate to the disk, allowing external
hooks to see the result of the update. This simply changes how the update hook
is called, so that it too is deferred until the lock is finally released.
There are many uses of mergemod.update(), but in terms of commands, it looks
like the following commands take wlock while calling mergemod.update(), and
therefore will now have their hook fired at a later time:
backout, fetch, histedit, qpush, rebase, shelve, transplant
Unlike the others, fetch immediately unlocks after calling update(), so for all
intents and purposes, its hook invocation is not deferred (but the external hook
still sees the proper state).
If 'wlock' is taken, we should add 'afterlock' callback to the 'wlock' instead.
Otherwise, running post transaction hook after 'lock' is release but 'wlock' is
still taken lead to a deadlock (eg: 'hg update' during a hook).
This situation is much more common since: b7067abc16c0
push: acquire local 'wlock' if "pushback" is expected (BC) (issue4596)
This hook will be called whenever a transaction is aborted. This will make it
easy for people to clean up temporary content they may have created during a
transaction.
