Summary:
The blackbox extension uses a global variable for `lastui` in case certain ui
objects do not have an associated repo. That `lastui` reference makes the `ui`
(and `repo`) harder to be dropped. It also sets `ui.repo` which could form a
cycle if `repo.ui` is that `ui`. Fix it by using weakref.
Reviewed By: chadaustin
Differential Revision: D8675908
fbshipit-source-id: 7d0c083cc5c7a3ac134379732de1e0c408a1d5fc
Summary:
It's legit for a file name to contain `%s` or `%r` strings. Previously,
blackbox always expect the first argument to be the "format string". That's
inconvenient and it's easy to just pass a string containing the troublesome
`%s` to `ui.log`.
Let's just do not even try formatting strings if there is only one argument,
and if there are multiple arguments, fallback to concatenate them if they
cannot be formatted.
Reviewed By: DurhamG
Differential Revision: D8364130
fbshipit-source-id: 75b2d0e0a460b9a86d4ecd6ecfbb77c0c0fbe98c
Summary: Mostly empty lines removed and added. A few bugfixes on excessive line splitting.
Reviewed By: quark-zju
Differential Revision: D8199128
fbshipit-source-id: 90c1616061bfd7cfbba0b75f03f89683340374d5
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
Summary: This would help labeling interesting commands from callers.
Reviewed By: DurhamG
Differential Revision: D8057148
fbshipit-source-id: 647cc91be3baeaa0362799603a66dab0c6c570b8
Summary:
Some log messages might not end with "\n". Append it automatically so it
looks good in blackbox.log.
Reviewed By: singhsrb
Differential Revision: D7648930
fbshipit-source-id: f5262e0c77a3bde952c3963ac5298b850f38d9db
Summary:
There are certain code paths that passes an empty `msg` that would crash
blackbox code:
```
error in exit handlers:
Traceback (most recent call last):
File "mercurial/dispatch.py", line 73, in _runexithandlers
func(*args, **kwargs)
File "hgext/sampling.py", line 118, in telemetry
generaldelta=str('generaldelta' in repo.requirements).lower())
File "hgext/blackbox.py", line 170, in log
formattedmsg = msg[0] % msg[1:]
IndexError: tuple index out of range
```
Fix it by ignoring those messages.
Reviewed By: singhsrb
Differential Revision: D7648932
fbshipit-source-id: be9f8327fd476399e4b762c92ea5f31cb174e189
Summary: This change minimally addresses the issue that `debugrebuilddirstate` can crash if the dirstate file is very corrupt.
Reviewed By: markbt
Differential Revision: D7028370
fbshipit-source-id: 72fc7a2900a8bc1bb5f062454530b4fc4c806f09
Summary:
This check is useful and detects real errors (ex. fbconduit). Unfortunately
`arc lint` will run it with both py2 and py3 so a lot of py2 builtins will
still be warned.
I didn't find a clean way to disable py3 check. So this diff tries to fix them.
For `xrange`, the change was done by a script:
```
import sys
import redbaron
headertypes = {'comment', 'endl', 'from_import', 'import', 'string',
'assignment', 'atomtrailers'}
xrangefix = '''try:
xrange(0)
except NameError:
xrange = range
'''
def isxrange(x):
try:
return x[0].value == 'xrange'
except Exception:
return False
def main(argv):
for i, path in enumerate(argv):
print('(%d/%d) scanning %s' % (i + 1, len(argv), path))
content = open(path).read()
try:
red = redbaron.RedBaron(content)
except Exception:
print(' warning: failed to parse')
continue
hasxrange = red.find('atomtrailersnode', value=isxrange)
hasxrangefix = 'xrange = range' in content
if hasxrangefix or not hasxrange:
print(' no need to change')
continue
# find a place to insert the compatibility statement
changed = False
for node in red:
if node.type in headertypes:
continue
# node.insert_before is an easier API, but it has bugs changing
# other "finally" and "except" positions. So do the insert
# manually.
# # node.insert_before(xrangefix)
line = node.absolute_bounding_box.top_left.line - 1
lines = content.splitlines(1)
content = ''.join(lines[:line]) + xrangefix + ''.join(lines[line:])
changed = True
break
if changed:
# "content" is faster than "red.dumps()"
open(path, 'w').write(content)
print(' updated')
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
```
For other py2 builtins that do not have a py3 equivalent, some `# noqa`
were added as a workaround for now.
Reviewed By: DurhamG
Differential Revision: D6934535
fbshipit-source-id: 546b62830af144bc8b46788d2e0fd00496838939
Previously, blackbox always appends to blackbox.log and creates the
directory for that file on demand. That could be an issue if:
1. chg starts from `$REPO` directory, so `ui._bbrepo` is set.
2. `rm -rf $REPO`.
3. `chg init $REPO`, blackbox writes something and `init` will fail
because `$REPO` directory is non-empty.
This patch fixes that by verifying whether vfs exists before re-using it.
Differential Revision: https://phab.mercurial-scm.org/D768
`lastui` decides where (where is the `.hg`) to use if the current `ui`
object does not have a `_bbrepo` associated. Previously it only gets set in
`ui.log`, which means unless a `ui` with repo associated calls `log` with
tracked event, blackbox does not know where to write its log. This patch
makes `reposetup` set `lastui` so it so we could log some more events (see
test changes).
Differential Revision: https://phab.mercurial-scm.org/D655
It seems cleaner to just remove `_partialinit`, `copy`, `__init__`. This
patch makes it so by using `getattr` in `log` so those fields do not need to
be existed.
Differential Revision: https://phab.mercurial-scm.org/D652
Having the blackbox file objects cached in `ui._bbfp` could in theory be
troublesome if multiple processes (ex. chg servers) have file objects
referring to a same file. (Although I spent some time and failed to build a
convincing test case)
This patch makes blackbox re-open the file every time to make the situation
better. Ideally we also need proper locking.
The caching logic traces back to the commit introducing blackbox
(18242716a). That commit does not have details about why caching is
necessary. Consider the fact that blackbox logs are not many, it seems fine
to remove the fp cache to be more confident.
Differential Revision: https://phab.mercurial-scm.org/D650
The added test will show:
$ $PYTHON showsize.py .hg/blackbox*
.hg/blackbox.log: < 500
.hg/blackbox.log.1: < 500
.hg/blackbox.log.2: < 500
.hg/blackbox.log.3: < 500
.hg/blackbox.log.4: < 500
.hg/blackbox.log.5: >= 500
with previous code.
The issue is caused by blackbox caching file objects *by path*, and the
rotation size check could run on a wrong file object (i.e. it should check
"blackbox.log", but `filehandles["blackbox.log"]` contains a file object
that has been renamed to "blackbox.log.5").
This patch removes the "filehandlers" global cache added by 39bd7b0c79fe to
solve the issue.
I think the original patch was trying to make different ui objects use a same
file object if their blackbox.log path is the same. In theory it could also
be problematic in the rotation case. Anyway, that should become unnecessary
after D650.
Differential Revision: https://phab.mercurial-scm.org/D648
When the appropriate developer warnings are enabled, We wrap 'repo.vfs.audit' to
check for locks when accessing file in '.hg' for writing. Another changeset will
add a 'ward' for the store vfs (svfs).
This check system has caught a handful of locking issues that have been fixed
in previous series (mostly in 4.0). I expect another batch to be caught in third
party extensions.
We introduce two real exceptions from extensions 'blackbox.log' (because a lot of
read-only operations add entry to it), and 'last-email.txt' (because 'hg email'
is currently a read only operation and there is value to keep it this way).
In addition we are currently allowing bisect to operate outside of the lock
because the current code is a bit hard to get properly locked for now. Multiple
clean up have been made but there is still a couple of them to do and the freeze
is coming.
Now that the default value is also converted we can use a human readable version
for it. This will be useful if we start to automatically display the default
config value in various place.
cmdutil.command wasn't a member of the registrar framework only for a
historical reason. Let's make that happen. This patch keeps cmdutil.command
as an alias for extension compatibility.
I've caught multiple extensions in the wild lying about being
'internal', so it's time to move the goalposts on people. Goalpost
moving will continue until third party extensions stop trying to
defeat the system.
It's possible for the blackboxui code to do a "del self._bbvfs", then ui.copy()
or similar attempt will fail. It will also fail when constructing a blackboxui
from a non-blackbox ui.
This patch fixes the issue by not assuming any _bb* attr is set.
Without this, anyone creating a ui object using: uimod.ui()
skips the blackbox.
Also, anyone doing ui.copy() skipped the blackbox.
Unfortunately, the ui object lifestyle is a bit messy,
the first one that's created is never actually initialized
with subclasses, instead pieces of the subclass are adopted
into the primal ui object. In order to handle this, a
_partialinit method will be called to ensure that the
blackboxui is properly initialized.
Without this, the last logged entry didn't have access to
the repository, and thus couldn't report its version
(and especially that an add or similar dirtied it).
A side-effect is that one repo leaks until process exit...
Without this, while you could see the list of commands run,
it wasn't possible to identify what they were doing, because commads
could rely on revsets (including remote input which varies over time).
There are multiple ui objects in Mercurial that can relate to a repository,
before this change, each one would have its own file pointer, which
results in unfortunate logging behavior.
Also, any log rotation results would be bad because only the
active blackboxui object's file pointer would be refreshed.
Note that this does not prevent two long running hg commands for the same
repository from causing problems.
Without this, when there are multiple ui views, each blackbox
will have its own file handle, and the logging will be in
a really bad order.
Also, because of the way blackbox works, it never closes its
file handles, which means the last output before exit is
often lost.
This adds the process id to the line header for the blackbox output. This is
useful for distinguishing processes when using the blackbox on a server and many
processes are writing to the blackbox at once.
Python 2.6 introduced the "except type as instance" syntax, replacing
the "except type, instance" syntax that came before. Python 3 dropped
support for the latter syntax. Since we no longer support Python 2.4 or
2.5, we have no need to continue supporting the "except type, instance".
This patch mass rewrites the exception syntax to be Python 2.6+ and
Python 3 compatible.
This patch was produced by running `2to3 -f except -w -n .`.
