There are a number of improvements we want to make to revlogs
that will require a new version - version 2. It is unclear what the
full set of improvements will be or when we'll be done with them.
What I do know is that the process will likely take longer than a
single release, will require input from various stakeholders to
evaluate changes, and will have many contentious debates and
bikeshedding.
It is unrealistic to develop revlog version 2 up front: there
are just too many uncertainties that we won't know until things
are implemented and experiments are run. Some changes will also
be invasive and prone to bit rot, so sitting on dozens of patches
is not practical.
This commit introduces skeleton support for version 2 revlogs in
a way that is flexible and not bound by backwards compatibility
concerns.
An experimental repo requirement for denoting revlog v2 has been
added. The requirement string has a sub-version component to it.
This will allow us to declare multiple requirements in the course
of developing revlog v2. Whenever we change the in-development
revlog v2 format, we can tweak the string, creating a new
requirement and locking out old clients. This will allow us to
make as many backwards incompatible changes and experiments to
revlog v2 as we want. In other words, we can land code and make
meaningful progress towards revlog v2 while still maintaining
extreme format flexibility up until the point we freeze the
format and remove the experimental labels.
To enable the new repo requirement, you must supply an experimental
and undocumented config option. But not just any boolean flag
will do: you need to explicitly use a value that no sane person
should ever type. This is an additional guard against enabling
revlog v2 on an installation it shouldn't be enabled on. The
specific scenario I'm trying to prevent is say a user with a
4.4 client with a frozen format enabling the option but then
downgrading to 4.3 and accidentally creating repos with an
outdated and unsupported repo format. Requiring a "challenge"
string should prevent this.
Because the format is not yet finalized and I don't want to take
any chances, revlog v2's version is currently 0xDEAD. I figure
squatting on a value we're likely never to use as an actual revlog
version to mean "internal testing only" is acceptable. And
"dead" is easily recognized as something meaningful.
There is a bunch of cleanup that is needed before work on revlog
v2 begins in earnest. I plan on doing that work once this patch
is accepted and we're comfortable with the idea of starting down
this path.
Since 348863ccec7e "util: always force line buffered stdout when stdout is
a tty", we have two file objects attached to the same STDOUT_FILENO. If one
is closed, the underlying file descriptor is also closed, and writing to
the other file object would crash the Python interpreter in a hard way, at
least on Windows.
So, it seems safer to not close the standard streams. This also matches
the behavior of the default sys.stdout/stderr.close(), which never close
the FILE* streams in C layer.
https://hg.python.org/cpython/file/v2.7.13/Python/sysmodule.c#l1401
It's been there since 84af5a079c7d (2007-02-19), but seems wrong since any
I/O operations to a closed file would raise ValueError, not IOError. We should
keep the file object open even if the underlying file descriptor is half dead.
The idea is simple. If the given node id prefix is 'ff...f', add +1 to the
number of matches (e.g. ambiguous if partial + maybewdir > 1).
This patch also fixes id() revset and shortest() template since _partialmatch()
can raise WdirUnsupported exception.
The "(not ${revs})" was missing parens around ${revs}, so when revs
was "A + B", it became "(not A + B)" when actually "(not (A + B))" was
intended. Fixing that leads to some more testing of strip.
Similarly, the parens were missing in "${revs}::", making it "A + B::"
instead of "(A + B)::". Thanks to Yuya for noticing this part. This
did not affect any existing tests.
This was the fallout from 87957da1ca0b (on stable). Now that individual lines
can be conditionalized, it seems better to be explicit, rather than mash all of
this into one regex. "getaddrinfo failed" was added in efd8ac31a7e2 to support
Windows.
With #serve enabled on Windows, I was getting occasional stacktraces like this:
Errored test-hgweb-json.t: Traceback (most recent call last):
File "./run-tests.py", line 724, in run
self.tearDown()
File "./run-tests.py", line 805, in tearDown
killdaemons(entry)
File "./run-tests.py", line 540, in killdaemons
logfn=vlog)
File "...\tests\killdaemons.py", line 94, in killdaemons
os.unlink(pidfile)
WindowsError: [Error 32] The process cannot access the file because it is
being used by another process: '...\\hgtests.zmpqj3\\child80\\daemon.pids'
Adrian suggested using util.posixfile, which works. However, the 'mercurial'
package isn't in sys.path when invoking run-tests.py, and it isn't clear that
hacking[1] it in is a good thing (especially for test-run-tests.t, which uses an
installation in a temp folder).
I tried using ProcessMonitor to figure out what the other process is, but that
monitoring slows things down to such a degree that the issue doesn't occur. I
was ready to blame the virus scanner, but it happens without that too.
Looking at the code, I don't see anything that would have the pid file open.
But I was able to get through about 20 full test runs without an issue with this
minor change, whereas before it was pretty certain to hit this at least once in
two or three runs.
[1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-May/097907.html
I'm not sure what the referenced hang specifically was, but the whole test suite
(with #serve) still runs on python 2.7.13. Aside from no longer prepending
"cmd.exe /c", this backs out a2700a095510.
I'm trying to track down a rare failure of TerminateProcess() with an access
error, and I've seen random extra python processes hanging around after running
tests sometimes, so this might help.
However, c9d78bd0980d forces this change. Since the pid object is no longer
converted to a string, the cmd.exe pid was being saved instead of the hg pid,
and none of the daemons were being killed.
The 'discardedheads' return become unused and the relationship between newheads
and newhs can be clarified. Our next goal here is to be able to extract the
_postprocessobsolete call outside of the loop.
We keep returning the 'discardedheads' because we'll start using it again soon
in this series.
Since b5554fb97d46, heads unknown locally no longer get any post processing
from obsolescence markers. We clarify this fact by only feeding the list of
locally known new heads to the function. This simplification of the input will
help moving that post-processing earlier in the function.
The '_headssummary' function is documenting and using list objects in its
return. We now use them in _oldheadssummary too for consistency. This does not
affect any usages of these values.
Support for buffering was added to python in d09d6fe31b61, first released with
python2.7. Without this, the entirety of the response headers is read
byte-by-byte (it does more efficient reads when it gets to the non-header part
of the response).
python2.7's httplib.HTTPResponse takes the arguments in the following order:
sock, debuglevel, strict, method, buffering
This was previously passing them in as positional and skipped strict, so we set
strict=method. I'm explicitly setting strict=True now to preserve the previous
behavior that has been there since this file was created.
This will display color if "color.mode=ansi", and default to 'ansi' if the mode
is set to 'auto'. The 'debugcolor' command also reflects this policy.
Previously, "color.mode=ansi" on Windows printed jibberish around the normal
text. Using ANSI color is better, as it avoids the normal loss of color when
the default pager is enabled on Windows. See also issue5570.
When the underlying function fails (e.g. when run on older Windows), 'auto'
still falls back to 'win32'. Apparently, Microsoft originally had this feature
turned on by default, and then made it opt-in[1]. Therefore, not enabling it
unconditionally seems safer. Instead, only do it after processing the existing
check for support in a Unix-like environment.
[1] https://github.com/symfony/symfony/issues/17499#issuecomment-243481052
SetConsoleMode() fails with an invalid parameter error if given this option
prior to Windows 10, so indicate that to the caller instead of doing explicit
version checks.
This can happen if another process (even another hg process!) comes along and
removes the file at that time.
This partly resolves issue5584, but not completely -- a bogus dirstate update
can still happen. However, the full fix is too involved for stable.
For builds that run on hermetic environments, it's possible that the "less"
package is not installed by default, yet it's needed for tests to pass after
revision ca1519568a93 (which sets less as the fallback pager).
The contrib/zsh_completion file itself says to name it _hg.
With a name like `hg`, if the user has a line like `autoload ${^fpath}/*(N-.:t)`
in their zshrc, it will create a shell function named `hg` that will hide the
actual hg command and make hg unusable.
Separately from that though, the underscore prefix makes it actually work. The
zsh man page states:
The convention for autoloaded functions used in completion is that they
start with an underscore
This does not seem to just be a "convention", though. With the ill-advised line
removed from my zshrc and the file named
`/usr/local/share/zsh/site-functions/hg` (without the underscore), these
completions did not seem to get loaded and the ones from the zsh installation
were loaded instead. If I renamed them to be
`/usr/local/share/zsh/site-functions/_hg`, however, they were loaded.
I manually tested the above statement by starting a new zsh instance with the
file in `/usr/local/share/zsh/site-functions` with the following names:
- As `hg`, `which _hg_labels` did not show anything
- As `_hg`, `which _hg_labels` showed the expected function.
To quote `man 1 pkgbuild`:
--filter filter-expression
By default, --root will include the entire contents of the
given root-path in the package payload, except for any .svn
or CVS directories, and any .DS_Store files. You can override
these default filters by specifying one or more --filter
options. Each filter-expression is an re_format(7)
``extended'' expression: any path in the root which matches
any of the given expressions will be excluded from the pack-
age payload. (Note that specifying even one --filter inhibits
the default filters, so you must respecify the default fil-
ters if you still want them to be used.)
It turns out the default filter these days *also* includes .git and
.hg. Notice how that filter expression is a regular expression? That
(presumably unintentionally) prevents a file named "chg" or "_hg" from
getting included in the distribution. Many many thanks to spectral@
for trying to include a _hg file which led us to figure this bug out.
Bug filed with Apple for this as rdar://problem/32437369, mentioning
both the gap in documentation and the wrong defaults.
Since 1d07d9da84a0, pycompat.bytestr() wrapped by win32mbcs returns
unicode object, if an argument is not byte-str object. And this causes
unexpected failure at colorization.
pycompat.bytestr() is used to convert from color effect "int" value to
byte-str object in color module. Wrapped pycompat.bytestr() returns
unicode object for such "int" value, because it isn't byte-str.
If this returned unicode object is used to colorize non-ASCII byte-str
in cases below, UnicodeDecodeError is raised at an operation between
them.
- colorization uses "ansi" color mode, or
Even though this isn't default on Windows, user might use this
color mode for third party pager.
- ui.write() is buffered with labeled=True
Buffering causes "ansi" color mode internally, regardless of
actual color mode. With "win32" color mode, extra escape sequences
are omitted at writing data out.
For example, with "win32" color mode, "hg status" doesn't fail for
non-ASCII filenames, but "hg log" does for non-ASCII text, because
the latter implies buffered formatter.
There are many "color effect" value lines in color.py, and making them
byte-str objects isn't suitable for fixing on stable. In addition to
it, pycompat.bytestr will be used to get byte-str object from any
types other than int, too.
To resolve this issue, this patch does:
- replace pycompat.bytestr in checkwinfilename() with newly added
hook point util._filenamebytestr, and
- make win32mbcs reverse-wrap util._filenamebytestr
(this is a replacement of 1d07d9da84a0)
This patch does two things above at same time, because separately
applying the former change adds broken revision (from point of view of
win32mbcs) to stable branch.
"_" prefix is added to "filenamebytestr", because it is win32mbcs
specific hook point.
