We have started to isolate extra usecases for developer-only output
that is not a warning. As the section has the fairly generic name
'devel' it makes sense to tuck them there. As a result, 'all' becomes
a bit misleading so we rename it to 'all-warnings'. This will break
some developer setups but the tests are still fine and developers will
likely spot this change.
Before this patch, hook argument `txnid` isn't passed to `pretxnopen`
hooks, even though `hooks` section of `hg help config` describes so.
``pretxnopen``
Run before any new repository transaction is open. The reason for the
transaction will be in ``$HG_TXNNAME`` and a unique identifier for the
transaction will be in ``HG_TXNID``. A non-zero status will prevent the
transaction from being opened.
Before this patch, transaction ID (TXNID) is calculated from
`transaction` object itself by `id()`, but this prevents TXNID from
being passed to `pretxnopen` hooks, which should be executed before
starting transaction processing (also any preparations for it, like
writing journal files out).
As a preparation for passing TXNID to `pretxnopen` hooks, this patch
separates calculation of TXNID from creation of `transaction` object.
This patch uses "random" library for reasonable unique ID. "uuid"
library can't be used, because it was introduced since Python 2.5 and
isn't suitable for Mercurial 3.4.x stable line.
`%f` formatting for `random.random()` is used with explicit precision
number 40, because default precision for `%f` is 6. 40 should be long
enough, even if 10**9 transactions are executed in a short time (a
second or less).
On the other hand, `time.time()` is used to ensures uniqueness of
TXNID in a long time, for safety.
BTW, platform not providing `/dev/urandom` or so may cause failure of
`import random` itself with some Python versions (see Python
issue15340 for detail http://bugs.python.org/issue15340).
But this patch uses "random" without any workaround, because:
- "random" is already used directly in some code paths,
- such platforms are very rare (e.g. Tru64 and HPUX), and
http://bugs.python.org/issue15340#msg170000
- updating Python runtime can avoid this issue
The existing stream_in method assumes a streaming clone is applied via
the wire protocol. Previous patches have enabled streaming clone data to
be produced and consumed outside the context of the wire protocol.
However, the consuming part was incomplete because it didn't deal with
things like updating the branch caches or writing out a requirements
file.
This patch finishes the separation of stream clone handling from the
wire protocol. After this patch, it is possible to consume stream clones
from arbitrary sources, including files. Mozilla plans to leverage this
to serve pre-generated stream clone files to consumers, drastically
reducing the wall and CPU time required to clone large repositories.
This will enable clones to be nearly as fast as `tar`.
For reasons outlined in the previous commit, we want to make the code
for consuming "stream bundles" reusable. This patch extracts the code
into a standalone function.
Before this patch, reverting a file to the revision other than the
parent doesn't update dirstate. This seems to expect that timestamp
and/or size will be changed by reverting.
But if (1) dirstate of file "f" is filled with timestamp before
reverting and (2) size and timestamp of file "f" isn't changed at
reverting, file "f" is recognized as CLEAN unexpectedly.
This patch applies "dirstate.normallookup()" on reverted file, if size
isn't changed.
Making "localrepository.wwrite()" return length of written data is
needed to avoid additional (and redundant) "lstat(2)" on the reverted
file. "filectx.size()" can't be used to know it, because data may be
decoded at being written out.
BTW, interactive reverting may cause similar problem, too. But this
patch doesn't focus on fixing it, because (1) interactive (maybe slow)
reverting changes one (or both) of size/timestamp of reverted files in
many usecases, and (2) changes to fix it seems not suitable for stable
branch.
The pre-pushkey hook will likely validate the pushkey based on element
previously changed in the same transaction. We need to make theses data
available for the hook.
When pushkey is called during a transaction, we include its 'hookargs' when
running the pre-pushkey hooks. Having more data cannot hurt, especially the
transaction ID.
If 'wlock' is taken, we should add 'afterlock' callback to the 'wlock' instead.
Otherwise, running post transaction hook after 'lock' is release but 'wlock' is
still taken lead to a deadlock (eg: 'hg update' during a hook).
This situation is much more common since: b7067abc16c0
push: acquire local 'wlock' if "pushback" is expected (BC) (issue4596)
In case of errors, output parts salvaged from the reply bundle need to be
processed for outputting their content. This concludes our quest for fixing
issue4594.
We are going to add output related logic in this function. We do the
indentation first to help next changeset readability. We need a new try except
because we want to handle output on any exception, including PushRaced ones.
This hook will be called whenever a transaction is aborted. This will make it
easy for people to clean up temporary content they may have created during a
transaction.
We are warning about lock acquired in the wrong order because this can create
dead-lock situation. But non-wait acquisition will not block and therefore not
create a dead-lock. So we do not need to wait in such case.
Lock must be acquired in a specific order to avoid dead-lock. This was
documented on the wiki, but having this information in the docstring is also
useful.
We change the wording of the developer warning:
- "lock" taken before "wlock"
+ "wlock" acquired after "lock"
The goals here are to:
- Put the "subject" as the first word,
- use "acquired" instead of "taken" since it seems more accurate.
Before this change, any call to 'wlock' after we acquired a 'lock' was issuing a
warning. This is wrong as the 'wlock' have been properly acquired before the
'lock' in a previous call.
We move the warning code to only issue such warnings when we are acquiring the
'wlock' -after- acquiring 'lock'. This is the expected behavior of this warning
from the start.
That way, any new server will be ready to accept bundle2 payload. The decision
for the client to use it is still off by default so this is not turning bundle2
everywhere.
We introduce a new kill switch for this in case stuff goes wrong.
It is finally time to freeze the bundle2 format! To do so we:
- rename HG2Y to HG20,
- drop "b2x:" prefix from all part names,
- rename capability to "bundle2-exp" to "bundle2"
- rename the hook flag from 'bundle2-exp' to 'bundle2'
To support multiple bundle2 formats, we will need a function returning
the proper unbundler according to the header. We introduce such aa
function and change the usage in the code base. The function will get
smarter in later changesets.
This is somewhat similar to the dispatching we do for 'HG10' and 'HG11'.
The main target is to allow HG2Y support in an extension to ease transition of
companies using the experimental protocol in production (yeah...) But I've no
doubt this will be useful when playing with a future HG21.
The 'format' argument was not used even when it was added in
05af7fe09faf (getbundle: pass arbitrary arguments all along the call
chain, 2014-04-17).
Note that by removing the argument, if any caller did pass a named
'format' argument, we will now pass that along to exchange.getbundle()
via the kwargs. If the idea was to remove such a key, that should have
been done explicitly.
While it should be safe to switch to the new manifest format on an
existing repo, let's keep it simple for now and make the configuration
have any effect only at repo creation time. If the configuration is
enabled then (at repo creation), we add an entry to requires and read
that instead of the configuration from then on.
With tree manifests, hashes will change anyway, so now is a good time
to also take up the old plans of a new manifest format. While there
should be little or no reason to use tree manifests with the current
manifest format (v1) once the new format (v2) is supported, we'll try
to keep the two dimensions (flat/tree and v1/v2) separate.
In preparation for adding a the new format, let's add configuration
for it and propagate that configuration to the manifest revlog
subclass. The new configuration ("experimental.manifestv2") says in
what format to write the manifest data. We may later add other
configuration to choose how to hash it, either keeping the v1 hash for
BC or hashing the v2 content.
See http://mercurial.selenic.com/wiki/ManifestV2Plan for more details.
This patch newly adds "dirtyreason()" to centralize composing dirty
reason message like "uncommitted changes in subrepository 'xxxx'".
There are 3 similar messages below, and this patch is a part of
preparations for unifying them into (1), too.
1. uncommitted changes in subrepository 'XXXX'
2. uncommitted changes in subrepository XXXX
3. uncommitted changes in subrepo XXXX
This patch chooses adding new method "dirtyreason()" instead of making
"dirty()" return "reason string", because:
- some of existing "dirty()" implementation is too complicated to do
so simply, and
- ill-mannered 3rd party subrepo classes, of which "dirty()" doesn't
return "reason string", cause meaningless message (even though it
is rare case)
Since manifests instances are cached on the manifest log instance, we
can cache directory manifests by caching the directory manifest
logs. The directory manifest log cache is a plain dict, so it never
expires; we assume that we can keep all the directories in memory.
The cache is kept on the root manifestlog, so access to directory
manifest logs now has to go through the root manifest log.
The caching will soon not be only an optimization. When we start
lazily loading directory manifests, we need to make sure we don't
create multiple instances of the log objects. The caching takes care
of that problem.
It should be possible to debug the submanifest revlogs without having
to know where they are stored (in .hg/store/meta/), so let's add a
--dir option for this purpose.
match.ispartial() will return the opposite of match.always() in core, but this
function will be extensible by extensions to produce another result even
if match.always() will be untouched.
This will be useful for narrowhg, where ispartial() will return False even if
the match won't always match. This would happen in the case where the only
time the match function is False is when the path is outside of the narrow
spec.
The new way to allow empty commit is to temporarily set the
'ui.allowemptycommit' config option.
allowemptyback = repo.ui.backupconfig('ui', 'allowemptycommit')
try:
repo.ui.setconfig('ui', 'allowemptycommit', True)
repo.commit(...)
finally:
repo.ui.restoreconfig(allowemptyback)
All known uses of force for allowing empty commits have been removed, so let's
remove it from the allowemptycommits condition.
This adds a config flag that enables a user to make empty commits.
This is useful in a number of cases.
For instance, automation that creates release branches via
bookmarks may want to make empty commits to that release bookmark so that it
can't be fast-forwarded and so it can record information about the release
bookmark's creation. This is already possible with named branches, so making it
possible for bookmarks makes sense.
Another case we've wanted it is for mirroring repositories into Mercurial. We
have automation that syncs commits into hg by running things from the command
line. The ability to produce empty commits is useful for syncing unusual commits
from other VCS's.
In general, allowing the user to create the DAG as they see fit seems useful,
and when I mentioned this in IRC more than one person piped up and said they
were already hacking around this limitation by using mq, import, and
commit-dummy-change-then-amend-the-content-away style solutions.
The empty commit condition was a messy if condition. Let's move it to a new line
and change it to 'or' statements so it's cleaner and more readable. A future
commit will add additional logic to this line.
Before this patch, releasing the store lock implies the actions below, when
the transaction is aborted:
1. "commithook()" scheduled in "localrepository.commit()" is invoked
2. "changectx.__init__()" is invoked via "self.__contains__()"
3. specified ID is examined against "repo.dirstate.p1()"
4. validation function is invoked in "dirstate.p1()"
In subsequent patches, "dirstate.invalidate()" invocations for
discarding changes are replaced with "dirstateguard", but discarding
changes by "dirstateguard" is executed after releasing the store lock:
resources are acquired in "wlock => dirstateguard => store lock" order,
and are released in reverse order.
This may cause that "dirstate.p1()" still refers to the changeset to be
rolled-back at (4) above: pushing multiple patches by "hg qpush" is
a typical case.
When releasing the store lock, such changesets are:
- not contained in "repo.changelog", if it is reloaded from
".hg/00changelog.i", as that file was already truncated by
"transaction.abort()"
- still contained in it, otherwise
(this "dirty read" problem is discussed in "Transaction Plan"
http://mercurial.selenic.com/wiki/TransactionPlan)
Validation function shows "unknown working parent" warning in the
former case, but reloading "repo.changelog" depends on the timestamp
of ".hg/00changelog.i". This causes occasional test failures.
In the case of scheduled "commithook()", it just wants to examine
whether "node ID" of committed changeset is still valid or not. Other
examinations implied in "changectx.__init__()" are meaningless.
To avoid showing the "unknown working parent" warning irregularly, this
patch uses "changelog.hasnode()" instead of "node in self" to examine
existence of committed changeset.
The very next changeset will start writing one revlog per directory
when tree manifests are enabled. That is backwards incompatible, so it
requires .hg/requires to be updated. Just like with generaldelta, we
want to update .hg/requires only when the repo is created. Updating
..hg/requires is bad for repos on shared disk. Instead, those who do
want to upgrade a repo to using treemanifest (or manifestv2, etc) can
run
hg clone --config experimental.treemanifest repo clone
which will create a new repo with the requirement set. Unlike the case
of e.g. generaldelta, it will not rewrite the changesets, since tree
manifests hash differently.
Today, the terms 'active' and 'current' are interchangeably used throughout the
codebase in reference to the active bookmark (the bookmark that will be updated
with the next commit). This leads to confusion among developers and users.
This patch is part of a series to standardize the usage to 'active' throughout
the mercurial codebase and user interface.
Today, the terms 'active' and 'current' are interchangeably used throughout the
codebase in reference to the active bookmark (the bookmark that will be updated
with the next commit). This leads to confusion among developers and users.
This patch is part of a series to standardize the usage to 'active' throughout
the mercurial codebase and user interface.
This change adds boolean configuration option
experimental.treemanifest. When the option is enabled, manifests are
parsed into the new treemanifest type.
Tests can be now run using treemanifest by switching the config option
default in localrepo._applyrequirements(). Tests pass even when made
to randomly choose between manifestdict and treemanifest, suggesting
that the two types produce identical manifests (so e.g. a manifest
revlog entry written from a treemanifest can be parsed by the
manifestdict code).
It is currently only amend and debugbuilddag that will pass a filectx to
localrepo._filecommit. Amend will usually not hit the case where a the filectx
is a parent that just can be reused. Future convert changes will use it more.
Nobody should start a transaction on an unlocked repository. If
developer warnings are enabled this will be reported. This use the
same config as bad locking order since this is closely related.
