I used test-dispatch.py to demonstrate what would happen if
a log file changed from being readonly to writable, by
having it replace a directory (proxy for readonly/not-writable)
with a log file in between transactions of a running python
process (proxy for Mercurial).
This commit makes it easier for people to follow what the test
is doing, by creating a real file that people can read.
The blackbox test rewrites a copy of test-dispatch.py on the fly, and adds
a couple of lines with the s/// command. GNU sed supports the use of the
\n escape to represent a newline, but not Solaris sed. Using a literal
newline, prefixed by a backslash, works with both versions of the utility.
Without this, anyone creating a ui object using: uimod.ui()
skips the blackbox.
Also, anyone doing ui.copy() skipped the blackbox.
Unfortunately, the ui object lifestyle is a bit messy,
the first one that's created is never actually initialized
with subclasses, instead pieces of the subclass are adopted
into the primal ui object. In order to handle this, a
_partialinit method will be called to ensure that the
blackboxui is properly initialized.
Without this, the last logged entry didn't have access to
the repository, and thus couldn't report its version
(and especially that an add or similar dirtied it).
A side-effect is that one repo leaks until process exit...
Without this, while you could see the list of commands run,
it wasn't possible to identify what they were doing, because commads
could rely on revsets (including remote input which varies over time).
A concern around the user experience of Mercurial is user getting stuck on there
own topological branch forever. For example, someone pulling another topological
branch, missing that message in pull asking them to merge and getting stuck on
there own local branch.
The current way to "address" this concern was for bare 'hg update' to target the
tipmost (also latest pulled) changesets and complain when the update was not
linear. That way, failure to merge newly pulled changesets would result in some
kind of failure.
Yet the failure was quite obscure, not working in all cases (eg: commit right
after pull) and the behavior was very impractical in the common case
(eg: issue4673).
To be able to change that behavior, we need to provide other ways to alert a
user stucks on one of many topological head. We do so with an extra message after
bare update:
1 other heads for branch "default"
Bookmark get its own special version:
1 other divergent bookmarks for "foobar"
There is significant room to improve the message itself, and we should augment
it with hint about how to see theses other heads or handle the situation (see
in-line comment). But having "a" message is already a significant improvement
compared to the existing situation. Once we have it we can iterate on a better
version of it. As having such message is an important step toward changing the
default destination for update and other nicety, I would like to move forward
quickly on getting such message.
This was discussed during London - October 2015 Sprint.
With util.getpid, it is now possible to define fixed pids.
Future iterations can define a map of pids on a locked
first come first serve basis to create a more realistic
harness, but for now this is good enough.
This applies to blackbox, but could apply to other
tests as well.
Without this, when there are multiple ui views, each blackbox
will have its own file handle, and the logging will be in
a really bad order.
Also, because of the way blackbox works, it never closes its
file handles, which means the last output before exit is
often lost.
While it is not easy to make a file 000 on Windows, you can
emulate most of the behaviors by replacing the file with a directory.
Also corrects test description to properly indicate that failing to
read from the log is fatal.
c67339617276 (while 3.4 code-freeze) made all 'update' hooks run after
releasing wlock for visibility of in-memory dirstate changes. But this
breaks paired invocation of 'preupdate' and 'update' hooks.
For example, 'hg backout --merge' for TARGET revision, which isn't
parent of CURRENT, consists of steps below:
1. update from CURRENT to TARGET
2. commit BACKOUT revision, which backs TARGET out
3. update from BACKOUT to CURRENT
4. merge TARGET into CURRENT
Then, we expects hooks to run in the order below:
- 'preupdate' on CURRENT for (1)
- 'update' on TARGET for (1)
- 'preupdate' on BACKOUT for (3)
- 'update' on CURRENT for (3)
- 'preupdate' on TARGET for (4)
- 'update' on CURRENT/TARGET for (4)
But hooks actually run in the order below:
- 'preupdate' on CURRENT for (1)
- 'preupdate' on BACKOUT for (3)
- 'preupdate' on TARGET for (4)
- 'update' on TARGET for (1), but actually on CURRENT/TARGET
- 'update' on CURRENT for (3), but actually on CURRENT/TARGET
- 'update' on CURRENT for (4), but actually on CURRENT/TARGET
Root cause of the issue focused by c67339617276 is that external
'update' hook process can't view in-memory changes (especially, of
dirstate), because they aren't written out until the end of
transaction (or wlock).
Now, hooks can be invoked just after updating, because previous
patches made in-memory changes visible to external process.
This patch may break backward compatibility from the point of view of
"scheduling hook execution", but should be reasonable because 'update'
hooks had been executed in this order before 3.4.
This patch tests "hg backout" and "hg unshelve", because the former
activates the transaction before 'update' hook invocation, but the
former doesn't.
This adds the process id to the line header for the blackbox output. This is
useful for distinguishing processes when using the blackbox on a server and many
processes are writing to the blackbox at once.
The update command is touching the repository and should lock it for
the length of its operations. Equally importantly, it should lock the
repository when it is writing bookmarks. It wasn't doing so until now,
leaving doors open for all kinds of drunk beaver parties.
This results in some minor tests changes, and the fixing of a couple
of bugs from race conditions.
Code does not receive any changes beside extra indentation.
We now have multiple tags cache files. Record exactly which file is
being written. This should help aid debugging into performance issues
with any single filter.
.hgtags fnodes are now written to a shared cache file. They don't need
to exist in the per-filter tags cache files. Stop writing them.
The format of the tags cache file has changed in a backwards
incompatible way. This should be acceptable, as we just established
per-filter tags cache files and no client should have per-filter tags
cache files that will need to be read. So no backwards compatbility
concern is present.
The new format has a single header line followed by resolved tags
entries.
The header line is similar to the old first line with a major
difference: we now compute and store a hash of the filtered revisions.
Before, if the set of filtered revs changed, we may return incorrect
results. We now detect that.
A test for verifying filtered rev change is handled properly has been
added.
Having all blackbox log testing in test-blackbox.t isn't scalable. Move
the mock blackbox extension into its own file so we can start to move
blackbox logging into other tests.
This adds a new root hghave to test against. Almost all of these are a
subset of unix-permissions, but that is also used for checking exec
bit handling.
Previously the blackbox wrapped runcommand, but this failed to see the error
codes that were created if an exception occurred. I moved that logging to now
wrap _runcatch, so it can observe and log the actual error code (such as when
a user ctrl+c's during a command).
Updated the tests as well. Tested the change by running all the tests with the
blackbox extension enabled and verifying nothing broke (aside from things that
printed what extensions were enabeld).
The progress tests are affected by calls to time.time() so they needed to be
updated to pass.
The windows and vfat test runs were failing due to read/write permissions not
working the same on those systems. On vfat, permissions can't be changed
at all, and on windows it seems the chmod emulation doesn't remove read
permissions. We could theoretically get the 'cannot write to blacklog.log'
test to pass on windows but there's no #if condition to let us exclude vfat
only.
Verified that test-blackbox passes on windows now.
If enabled, log rotation prevents the amount of space used by the
blackbox log from growing without bound. This becomes important in
cases where there are a lot of busy repositories managed by humans
and automation on many machines.
In large deployments, we cannot reasonably track all the repos where
blackbox logs need to be managed, so it is safer to have blackbox
manage its own logs than to move responsibility to an external tool
such as logrotate.
This change adds two configuration keys:
* blackbox.maxsize is the maximum allowable size of the current log
* blackbox.maxfiles is the number of log files to maintain
Writes the backup bundle paths to the blackbox so it's easy to see which
backup bundle is associated with which command when you are debugging an
issue.
Example output:
2013/03/13 10:39:56 durham> strip tip
2013/03/13 10:39:59 durham> saved backup bundle to /data/users/durham/www-hg/.hg/strip-backup/e5fac262363a-backup.hg
2013/03/13 10:40:03 durham> strip tip exited 0 after 7.97 seconds
The blackbox was logging every head after every incoming group.
Now we only log the heads that have changed.
Added a test. Moved the hooks test to the bottom of the file since
the hooks interfer with the tests after it.
