Summary:
Change the crash header to just a single line without blaming extensions or
Python. This makes the crash log shorter, and easier to read when there are
crashes in run-tests.py diffs.
Remove `ui.supportcontact` since it's no longer used.
Reviewed By: sfilipco
Differential Revision: D14393982
fbshipit-source-id: 3ede8d3d3d8fd5d125944a9a750350d3ce356b14
Summary:
Now that we have perftracing infra, let's trace a bunch of likely
problem spots.
Reviewed By: sfilipco
Differential Revision: D14426367
fbshipit-source-id: 354a241aa9ac5d75d34062a9838d581b4f46746f
Summary:
Move the strip extension to core. Rename the command to `hg debugstrip` as it
is not intended for use by users. Users should use `hg hide` instead.
Reviewed By: quark-zju
Differential Revision: D14185822
fbshipit-source-id: ef096488cb94b72a7bb79f5bf153c064e0555b34
Summary:
This hardcodes several perftweaks configs that have been enabled for major FB
repos for months:
[perftweaks]
disablebranchcache = True
disablebranchcache2 = True
disableresolvingbranches = True
disableupdatebranchcacheoncommit = True
Practically, this means the branchmap is now just `{'default': heads}`. (i.e.
there are no named branches other than `default`), and branchcache is removed
(i.e. `.hg/cache` does not exist without clindex or in-repo tags).
This diff only makes easy-to-verify logic changes by assuming the configs and
removing dead code. Things can be further cleaned up. They will be done by
upcoming changes.
Most test changes are due to the fact that `.hg/cache` is no longer created.
Reviewed By: singhsrb
Differential Revision: D14179858
fbshipit-source-id: 479f7427168eb1d9614a973e273a229e50f5620a
Summary:
D13853115 adds `edenscm/` to `sys.path` and code still uses `import mercurial`.
That has nasty problems if both `import mercurial` and
`import edenscm.mercurial` are used, because Python would think `mercurial.foo`
and `edenscm.mercurial.foo` are different modules so code like
`try: ... except mercurial.error.Foo: ...`, or `isinstance(x, mercurial.foo.Bar)`
would fail to handle the `edenscm.mercurial` version. There are also some
module-level states (ex. `extensions._extensions`) that would cause trouble if
they have multiple versions in a single process.
Change imports to use the `edenscm` so ideally the `mercurial` is no longer
imported at all. Add checks in extensions.py to catch unexpected extensions
importing modules from the old (wrong) locations when running tests.
Reviewed By: phillco
Differential Revision: D13868981
fbshipit-source-id: f4e2513766957fd81d85407994f7521a08e4de48
Summary:
mq is already somehow problematic at D8907646. Without bandwidth supporting
it, let's remove it.
Alternative to mq would be rebase, shelve, unshelve, histedit.
Maintain "--config extensions.mq=" compatibility by marking it builtin so hg4idea
won't break by this change.
Reviewed By: phillco
Differential Revision: D9039741
fbshipit-source-id: a3a1e48a2a982ff8e8b6a6ce659c906a4e2b2b36
Summary: Run tests with watchman built from fbcode master. So we get signals about watchman/hg changes.
Reviewed By: DurhamG
Differential Revision: D8448187
fbshipit-source-id: 8643f139932057f326bc75b0b0c5cc616355eeed
Summary:
Previously, hg returns -1 (255) on "unknown errors". That could conflict with
other things. For example, http://tldp.org/LDP/abs/html/exitcodes.html suggests
1, 2, 126 ... 255 have special meanings defined by a common shell. Namely,
`ssh` also returns 255.
Another long complaints about hg exit code is the use of 1 to indicate
"no changes". Although that's explained at [1], a lot of scripts still find it
surprising.
This diff adds a `ui.exitcodemask` config option, which makes it to alter the
exit code. Set it to 254 will change 1 to 0 for some script use-cases. Set it
to 63 or so will avoid conflicts with software like `ssh`.
Many existing scripts still expect 255 to be the return code. So we cannot
change the default value that easily. To avoid surprises (like, put the config
option in `/etc/mercurial/hgrc.d/exitcode.rc`), the config set by config
files is ignored if `HGPLAIN` is set and `HGPLAINEXCEPT` does not contain
`exitcode`.
[1]: https://www.mercurial-scm.org/pipermail/mercurial-devel/2012-January/037711.html
Reviewed By: DurhamG
Differential Revision: D7921817
fbshipit-source-id: 764b0de030fc727aa5df7305c2b8bc92f576cd33
Summary:
It's legit for a file name to contain `%s` or `%r` strings. Previously,
blackbox always expect the first argument to be the "format string". That's
inconvenient and it's easy to just pass a string containing the troublesome
`%s` to `ui.log`.
Let's just do not even try formatting strings if there is only one argument,
and if there are multiple arguments, fallback to concatenate them if they
cannot be formatted.
Reviewed By: DurhamG
Differential Revision: D8364130
fbshipit-source-id: 75b2d0e0a460b9a86d4ecd6ecfbb77c0c0fbe98c
Summary: This would help labeling interesting commands from callers.
Reviewed By: DurhamG
Differential Revision: D8057148
fbshipit-source-id: 647cc91be3baeaa0362799603a66dab0c6c570b8
Summary:
Some log messages might not end with "\n". Append it automatically so it
looks good in blackbox.log.
Reviewed By: singhsrb
Differential Revision: D7648930
fbshipit-source-id: f5262e0c77a3bde952c3963ac5298b850f38d9db
Summary:
There are certain code paths that passes an empty `msg` that would crash
blackbox code:
```
error in exit handlers:
Traceback (most recent call last):
File "mercurial/dispatch.py", line 73, in _runexithandlers
func(*args, **kwargs)
File "hgext/sampling.py", line 118, in telemetry
generaldelta=str('generaldelta' in repo.requirements).lower())
File "hgext/blackbox.py", line 170, in log
formattedmsg = msg[0] % msg[1:]
IndexError: tuple index out of range
```
Fix it by ignoring those messages.
Reviewed By: singhsrb
Differential Revision: D7648932
fbshipit-source-id: be9f8327fd476399e4b762c92ea5f31cb174e189
Summary: Also change the internal API so it no longer accepts the "heads" argument.
Reviewed By: ryanmce
Differential Revision: D6745865
fbshipit-source-id: 368742be49b192f7630421003552d0a10eb0b76d
# skip-blame because this was mechanically rewritten the following script. I
ran it on both *.t and *.py, but none of the *.py changes were proper. All *.t
ones appear to be, and they run without addition failures on both Windows and
Linux.
import argparse
import os
import re
ap = argparse.ArgumentParser()
ap.add_argument('path', nargs='+')
opts = ap.parse_args()
globre = re.compile(r'^(.*) \(glob\)(.*)$')
for p in opts.path:
tmp = p + '.tmp'
with open(p, 'rb') as src, open(tmp, 'wb') as dst:
for line in src:
m = globre.match(line)
if not m or '$LOCALIP' in line or '*' in line:
dst.write(line)
continue
if '?' in line[:-3] or ('?' in line[:-3] and line[-3:] != '(?)'):
dst.write(line)
continue
dst.write(m.group(1) + m.group(2) + '\n')
os.unlink(p)
os.rename(tmp, p)
Before, early options were stripped from args, and because of this, some
kind of parsing errors weren't reported. For example,
$ hg ci -m -Ra file
would execute "hg ci -m file" in repository "a".
This patch fixes the issue by parsing early options again by real getopt-based
parser, and verifying the results. If the early parsing appears wrong, hg just
aborts. The current error message seems not nice, and should be improved, maybe
in V2 or follow-up.
Note that this isn't a security feature because we can still do anything by
using shell aliases.
Upon pull or unbundle, we display a message with the range of new revisions
fetched. This revision range could readily be used after a pull to look out
what's new with 'hg log'. The algorithm takes care of filtering "obsolete"
revisions that might be present in transaction's "changes" but should not be
displayed to the end user.
Previously, blackbox always appends to blackbox.log and creates the
directory for that file on demand. That could be an issue if:
1. chg starts from `$REPO` directory, so `ui._bbrepo` is set.
2. `rm -rf $REPO`.
3. `chg init $REPO`, blackbox writes something and `init` will fail
because `$REPO` directory is non-empty.
This patch fixes that by verifying whether vfs exists before re-using it.
Differential Revision: https://phab.mercurial-scm.org/D768
I've observed some weirdness around this, and needed to rule some
things out. There aren't any bugs in core around this, but it was nice
to have confirmation.
`lastui` decides where (where is the `.hg`) to use if the current `ui`
object does not have a `_bbrepo` associated. Previously it only gets set in
`ui.log`, which means unless a `ui` with repo associated calls `log` with
tracked event, blackbox does not know where to write its log. This patch
makes `reposetup` set `lastui` so it so we could log some more events (see
test changes).
Differential Revision: https://phab.mercurial-scm.org/D655
The added test will show:
$ $PYTHON showsize.py .hg/blackbox*
.hg/blackbox.log: < 500
.hg/blackbox.log.1: < 500
.hg/blackbox.log.2: < 500
.hg/blackbox.log.3: < 500
.hg/blackbox.log.4: < 500
.hg/blackbox.log.5: >= 500
with previous code.
The issue is caused by blackbox caching file objects *by path*, and the
rotation size check could run on a wrong file object (i.e. it should check
"blackbox.log", but `filehandles["blackbox.log"]` contains a file object
that has been renamed to "blackbox.log.5").
This patch removes the "filehandlers" global cache added by 39bd7b0c79fe to
solve the issue.
I think the original patch was trying to make different ui objects use a same
file object if their blackbox.log path is the same. In theory it could also
be problematic in the rotation case. Anyway, that should become unnecessary
after D650.
Differential Revision: https://phab.mercurial-scm.org/D648
Currently when we have multiple heads on the same branch, update tells us that
there some more heads for the current branch but does not tells us the head to
which the repository has been updated to. It makes more sense showing the
head we updated to and then telling there are some more heads.
Now that we garantee that branchmap cache are updated at the end of the
transaction we can drop that one. This removes a problematic case with nested
transaction where the new cache could be written on disk before the transaction
is finished.
The test change is harmless, since we update the cache at a later point, the
dirstate have been updated in between.
The quoting logic here was actually insufficient, and would have had
bogus b-prefixes on Python 3. shellquote seems more appropriate
anyway. Surprisingly, only two tests have output changes, and both of
them look reasonable to me (both are in blackbox logs).
Spotted by Yuya during review.
This should be extremely useful for helping users debug without having
to see their complete configuration.
Shell aliases do not get their expansion logged, because we don't look
and see if we're in a repo before we dive into the execution of a
shell alias. As a result, the ui object doesn't know where to log.
I used test-dispatch.py to demonstrate what would happen if
a log file changed from being readonly to writable, by
having it replace a directory (proxy for readonly/not-writable)
with a log file in between transactions of a running python
process (proxy for Mercurial).
This commit makes it easier for people to follow what the test
is doing, by creating a real file that people can read.
The blackbox test rewrites a copy of test-dispatch.py on the fly, and adds
a couple of lines with the s/// command. GNU sed supports the use of the
\n escape to represent a newline, but not Solaris sed. Using a literal
newline, prefixed by a backslash, works with both versions of the utility.
Without this, anyone creating a ui object using: uimod.ui()
skips the blackbox.
Also, anyone doing ui.copy() skipped the blackbox.
Unfortunately, the ui object lifestyle is a bit messy,
the first one that's created is never actually initialized
with subclasses, instead pieces of the subclass are adopted
into the primal ui object. In order to handle this, a
_partialinit method will be called to ensure that the
blackboxui is properly initialized.
Without this, the last logged entry didn't have access to
the repository, and thus couldn't report its version
(and especially that an add or similar dirtied it).
A side-effect is that one repo leaks until process exit...
Without this, while you could see the list of commands run,
it wasn't possible to identify what they were doing, because commads
could rely on revsets (including remote input which varies over time).
A concern around the user experience of Mercurial is user getting stuck on there
own topological branch forever. For example, someone pulling another topological
branch, missing that message in pull asking them to merge and getting stuck on
there own local branch.
The current way to "address" this concern was for bare 'hg update' to target the
tipmost (also latest pulled) changesets and complain when the update was not
linear. That way, failure to merge newly pulled changesets would result in some
kind of failure.
Yet the failure was quite obscure, not working in all cases (eg: commit right
after pull) and the behavior was very impractical in the common case
(eg: issue4673).
To be able to change that behavior, we need to provide other ways to alert a
user stucks on one of many topological head. We do so with an extra message after
bare update:
1 other heads for branch "default"
Bookmark get its own special version:
1 other divergent bookmarks for "foobar"
There is significant room to improve the message itself, and we should augment
it with hint about how to see theses other heads or handle the situation (see
in-line comment). But having "a" message is already a significant improvement
compared to the existing situation. Once we have it we can iterate on a better
version of it. As having such message is an important step toward changing the
default destination for update and other nicety, I would like to move forward
quickly on getting such message.
This was discussed during London - October 2015 Sprint.
With util.getpid, it is now possible to define fixed pids.
Future iterations can define a map of pids on a locked
first come first serve basis to create a more realistic
harness, but for now this is good enough.
This applies to blackbox, but could apply to other
tests as well.
Without this, when there are multiple ui views, each blackbox
will have its own file handle, and the logging will be in
a really bad order.
Also, because of the way blackbox works, it never closes its
file handles, which means the last output before exit is
often lost.
While it is not easy to make a file 000 on Windows, you can
emulate most of the behaviors by replacing the file with a directory.
Also corrects test description to properly indicate that failing to
read from the log is fatal.
c67339617276 (while 3.4 code-freeze) made all 'update' hooks run after
releasing wlock for visibility of in-memory dirstate changes. But this
breaks paired invocation of 'preupdate' and 'update' hooks.
For example, 'hg backout --merge' for TARGET revision, which isn't
parent of CURRENT, consists of steps below:
1. update from CURRENT to TARGET
2. commit BACKOUT revision, which backs TARGET out
3. update from BACKOUT to CURRENT
4. merge TARGET into CURRENT
Then, we expects hooks to run in the order below:
- 'preupdate' on CURRENT for (1)
- 'update' on TARGET for (1)
- 'preupdate' on BACKOUT for (3)
- 'update' on CURRENT for (3)
- 'preupdate' on TARGET for (4)
- 'update' on CURRENT/TARGET for (4)
But hooks actually run in the order below:
- 'preupdate' on CURRENT for (1)
- 'preupdate' on BACKOUT for (3)
- 'preupdate' on TARGET for (4)
- 'update' on TARGET for (1), but actually on CURRENT/TARGET
- 'update' on CURRENT for (3), but actually on CURRENT/TARGET
- 'update' on CURRENT for (4), but actually on CURRENT/TARGET
Root cause of the issue focused by c67339617276 is that external
'update' hook process can't view in-memory changes (especially, of
dirstate), because they aren't written out until the end of
transaction (or wlock).
Now, hooks can be invoked just after updating, because previous
patches made in-memory changes visible to external process.
This patch may break backward compatibility from the point of view of
"scheduling hook execution", but should be reasonable because 'update'
hooks had been executed in this order before 3.4.
This patch tests "hg backout" and "hg unshelve", because the former
activates the transaction before 'update' hook invocation, but the
former doesn't.
This adds the process id to the line header for the blackbox output. This is
useful for distinguishing processes when using the blackbox on a server and many
processes are writing to the blackbox at once.
The update command is touching the repository and should lock it for
the length of its operations. Equally importantly, it should lock the
repository when it is writing bookmarks. It wasn't doing so until now,
leaving doors open for all kinds of drunk beaver parties.
This results in some minor tests changes, and the fixing of a couple
of bugs from race conditions.
Code does not receive any changes beside extra indentation.
We now have multiple tags cache files. Record exactly which file is
being written. This should help aid debugging into performance issues
with any single filter.
.hgtags fnodes are now written to a shared cache file. They don't need
to exist in the per-filter tags cache files. Stop writing them.
The format of the tags cache file has changed in a backwards
incompatible way. This should be acceptable, as we just established
per-filter tags cache files and no client should have per-filter tags
cache files that will need to be read. So no backwards compatbility
concern is present.
The new format has a single header line followed by resolved tags
entries.
The header line is similar to the old first line with a major
difference: we now compute and store a hash of the filtered revisions.
Before, if the set of filtered revs changed, we may return incorrect
results. We now detect that.
A test for verifying filtered rev change is handled properly has been
added.
Having all blackbox log testing in test-blackbox.t isn't scalable. Move
the mock blackbox extension into its own file so we can start to move
blackbox logging into other tests.
