Before this patch, when printing help text using `hg help`, or `hg log -h`,
the output will wrap at 78 chars even if the user has a bigger terminal width
and there is no config option to change it, making the experience different
from the commonly used `man` tool.
This patch introduces a new config option `ui.textwidth`, which replaces the
hardcoded number. It's set to 78 by default to maintain compatibility. When
set to 0, `hg help` will behave more like `man`.
lazymanifests can compute diffs significantly faster than taking the set
of two manifests and calculating the delta.
when running hg diff --git -c . on Facebook's big repo, this reduces the
run time from 2.1s to 1.5s.
In the 'hg pull --rebase', we don't want to pick a rebase destination unrelated
to the pull, we lay down basic infrastructure to allow such restriction on
stable (before 3.8 release) in this case. See issue 5214 for details.
Actual usage and test will be in the next patch.
This effectively backs out changeset 60b56b3206cc.
The http library behind ui.http2=true isn't specifying the hostname.
It is the day before the expected 3.8 release and we don't want to ship
a regression.
I'll try to restore this requirement in the 3.9 release cycle as part
of planned improvements to Mercurial's SSL/TLS interactions.
The atomictemp.close() file attempts to do a rename, which can fail.
Moving the close inside the exception handler fixes it.
This doesn't fit well with the with: pattern, as it's the finalizer
that's failing.
tryread() doesn't handle "is a directory" errors and presumably
others. We might not want to globally swallow such tryread errors, so
we replace with our own try/except handling.
An upcoming test will use directories as a portable stand-in for
various bizarre circumstances that cache read/write code should be
robust to.
Properly shell quote arguments, to avoid printing commands that won't work when
run literally. For example, a date string with timestamp needs to be quoted:
--date '1456953053 28800'
Initializing a subrepo when one doesn't exist is the right thing to do when the
parent is being updated, but in few other cases. Unfortunately, there isn't
enough context in the subrepo module to distinguish this case. This same issue
can be caused with other subrepo aware commands, so there is a general issue
here beyond the scope of this fix.
A simpler attempt I tried was to add an '_updating' boolean to localrepo, and
set/clear it around the call to mergemod.update() in hg.updaterepo(). That
mostly worked, but doesn't handle the case where archive will clone the subrepo
if it is missing. (I vaguely recall that there may be other commands that will
clone if needed like this, but certainly not all do. It seems both handy, and a
bit surprising for what should be a read only operation. It might be nice if
all commands did this consistently, but we probably need Angel's subrepo caching
first, to not make a mess of the working directory.)
I originally handled 'Exception' in order to pick up the Aborts raised in
subrepo.state(), but this turns out to be unnecessary because that is called
once and cached by ctx.sub() when iterating the subrepos.
It was suggested in the bug discussion to skip looking at the subrepo links
unless -S is specified. I don't really like that idea because missing a subrepo
or (less likely, but worse) a corrupt .hgsubstate is a problem of the parent
repo when checking out a revision. The -S option seems like a better fit for
functionality that would recurse into each subrepo and do a full verification.
Ultimately, the default value for 'allowcreate' should probably be flipped, but
since the default behavior was to allow creation, this is less risky for now.
LoadLibrary() changes behavior depending on whether the argument
passed to it contains a period. From the MSDN docs:
If no file name extension is specified in the lpFileName parameter,
the default library extension .dll is appended. However, the file name
string can include a trailing point character (.) to indicate that the
module name has no extension. When no path is specified, the function
searches for loaded modules whose base name matches the base name of
the module to be loaded. If the name matches, the load succeeds.
Otherwise, the function searches for the file.
As the subsequent patch will show, some environments on Windows
define their Python library as e.g. "libpython2.7.dll." The existing
code would pass "libpython2.7" into LoadLibrary(). It would assume
"7" was the file extension and look for a "libpython2.dll" to load.
By passing ".dll" into LoadLibrary(), we force it to search for the
exact basename we want, even if it contains a period.
The old "update across branches if no uncommitted changes" made
it sound like updating across branches (with no uncommitted changes)
was allowed only with this option, which was not true. Also, the option
did not care whether it was linear or across branches. Instead, it
checked that there were no uncommitted changes. Let's explain what it
does instead of trying to suggest what happens without it.
Update makedirs() to ignore EEXIST in case someone else has already created the
directory in question. Previously the ensuredirs() function existed, and was
nearly identical to makedirs() except that it fixed this race. Unfortunately
ensuredirs() was only used in 3 places, and most code uses the racy makedirs()
function. This fixes makedirs() to be non-racy, and replaces calls to
ensuredirs() with makedirs().
In particular, mercurial.scmutil.origpath() used the racy makedirs() code,
which could cause failures during "hg update" as it tried to create backup
directories.
This does slightly change the behavior of call sites using ensuredirs():
previously ensuredirs() would throw EEXIST if the path existed but was a
regular file instead of a directory. It did this by explicitly checking
os.path.isdir() after getting EEXIST. The makedirs() code did not do this and
swallowed all EEXIST errors. I kept the makedirs() behavior, since it seemed
preferable to avoid the extra stat call in the common case where this directory
already exists. If the path does happen to be a file, the caller will almost
certainly fail with an ENOTDIR error shortly afterwards anyway. I checked
the 3 existing call sites of ensuredirs(), and this seems to be the case for
them.
This causes the longest_match search to limit itself to a window of
30000 lines during search (roughly 1MB), thus avoiding a full O(N*M)
search that might occur in repetitive structured inputs. For a
particular class of many MB pathological test cases, this generated
the following timings:
size before after
10x 1.25s 1.24s
100x 57s 33s
1000x >8400s 400s
The times on the right quickly become much faster and appear more linear.
While windowing means deltas are no longer "optimal", the resulting
deltas were within a couple percent of expected size. While we've yet
to have a report of a file with the level of repetition necessary to
hit this case, some JSON/XML database dump scenario is fairly likely
to hit it.
This may also slightly improve the average-case performance for deltas
of large binaries.
For highly structured files like JSON or XML dumps with large numbers
of duplicate lines (eg braces) and isolated matching lines, bdiff
could find large numbers of equally good spans. Because it prefers
earlier matches, this would result in pathologically unbalance
recursion that resulted in quadratic performance.
This patch makes it prefer matches closer to the middle that tend to
balance recursion. This change improves the speed of a pathological
test case from 1100s to 9s.
Included is a smaller test that has a roughly 50x safety margin on the
performance it accepts. It's likely to fail on pure builds because
difflib also has a recursion-balancing problem.
The longest_match code compares all the possible positions in two
files to find the best match. Given a pair of sequences, it
effectively searches a grid like this:
a b b b c . d e . f
0 1 2 3 4 5 6 7 8 9
a 1 - - - - - - - - -
b - 2 1 1 - - - - - -
b - 1 3 2 - - - - - -
b - 1 2 4 - - - - - -
. - - - - - 1 - - 1 -
Here, the 4 in the middle says "the first four lines of the
file match", which it can compute be comparing the fourth lines and
then adding one to the result found when comparing the third lines in
the entry to the upper left.
We generally avoid the quadratic worst case by only looking at lines
that match, which is precomputed. We also avoid quadratic storage by
only keeping a single column vector and then keeping track of the best
match.
Unfortunately, this can get us into trouble with the sequences above.
Because we want to reuse the '3' value when calculating the '4', we
need to be careful not to overwrite it with the '2' we calculate
immediately before. If we scan left to right, top to bottom, we're
going to have a problem: we'll overwrite our 3 before we use it and
calculate a suboptimal best match.
To address this, we can either keep two column vectors and swap
between them (which significantly complicates bookkeeping), or change
our scanning order. If we instead scan from left to right, bottom to
top, we'll avoid ever overwriting values we'll need in the future.
This unfortunately needs several changes to be made simultaneously:
- change the order we build the initial hash chains for the b sequence
- change the sentinel values from INT_MAX to -1
- change the visit order in the longest_match inner loop
- add a tie-breaker preference for earlier matches
This last is needed because we previously had an implicit tie-breaker
from our visitation order that our test suite relies on. Later matches
can also trigger a bug in the normalization code in diff().
This bug is hidden by the current bias towards matches at the
beginning of the file. When this bias is tweaked later to address
recursion balancing, the normalization code could cause the next block
to shrink to a negative length, thus creating invalid delta chunks. We
add checks here to disallow that.
This bug requires test cases that are an awkwardly large size for the test
suite, but is very rapidly picked up by the included torture tester.
This just makes the code harder to read without any performance
advantage. We're going to make the check here more complex, let's make
it simpler first.
Our invert() function was too clever to not take length into account. I could
fix the problem by appending '\xff' as a terminator (opposite to '\0'), but
it turned out to be slower than simple multi-pass sorting.
New implementation is pretty straightforward, which just calls sort() from the
last key. We can do that since Python sort() is guaranteed to be stable. It
doesn't sound nice to call sort() multiple times, but actually it is faster.
That's probably because we have fewer Python codes in hot loop, and can avoid
heavy string and list manipulation.
revset #0: sort(0:10000, 'branch')
0) 0.412753
1) 0.393254
revset #1: sort(0:10000, '-branch')
0) 0.455377
1) 0.389191 85%
revset #2: sort(0:10000, 'date')
0) 0.408082
1) 0.376332 92%
revset #3: sort(0:10000, '-date')
0) 0.406910
1) 0.380498 93%
revset #4: sort(0:10000, 'desc branch user date rev')
0) 0.542996
1) 0.486397 89%
revset #5: sort(0:10000, '-desc -branch -user -date -rev')
0) 0.965032
1) 0.518426 53%
Before, copied files were assumed as "A" (added) and listed followed by
non-copy added files. This could double entries of a copy if it had "M"
(modified) state.
So, this patch makes the template check if a file is included in copies dict.
This way, entries should never be doubled.
The output of "log -Tstatus -C" does not always agree with "status -C --change"
due to the bug of "status", which is documented in test-status.t. See also
21a68fa3c757.
Produce stable output for tools to rely on by hardcoding all edge styles to
"|". This ensures that any tool parsing the output of hg log -G still gets the
same behaviour as pre-3.8 releases.
When there are more than 2 parents for a given node (in a sparse graph), extra
dummy nodes are inserted to transition the lines more gradually. However, since
the seen state was not updated when yielding the extra nodes, the wrong graph
styles were being applied to the nodes.
Because parsing "$n" requires a crafted tokenizer, it exists only for backward
compatibility (as documented in revset._tokenizealias.) This patch updates the
examples so that users are encouraged to use symbolic names instead of "$n"s.
I'm going to implement alias expansion in templater, which won't support "$n"
parameters to make my life easier. Templater is more complicated than revset
because tokenizer and parser call each other.
Windows command lines use double quotes to quote arguments with spaces.
This change is in a series to unify around using single quotes around
commands, and double quotes around interior arguments.
Windows command lines use double quotes to quote arguments with spaces.
This change is in a series to unify around using single quotes around
commands, and double quotes around interior arguments.
Prevents double usage and helps reduce reference cycles, which
were observed to occur in `hg convert` and other scenarios where
there are multiple transactions per process.
Post release hooks should only be called once. Setting the
list to None after usage will prevent accidental usage after
they are used.
In addition, it is easy for reference cycles to sneak into hook
functions. Clearing the hooks after usage helps prevent these
cycles.
Now template aliases are fully supported in log and formatter templates.
As I said before, aliases are not expanded in map files. This avoids possible
corruption of our stock styles and web templates. This behavior is undocumented
since no map file nor [templates] section are documented at all. Later on,
we might want to add [aliases] section to map files if it appears to be useful.
This function will host loading of template aliases. It is not defined at
templater, but at formatter, since formatter is the module handling ui stuff
in front of templater.
New frommapfile() function will make it clear when template aliases will be
loaded. They should be applied to command arguments and templates in hgrc,
but not to map files. Otherwise, our stock styles and web templates
(i.e map-file templates) could be modified unintentionally.
Future patches will add "aliases" argument to __init__(), but not to
frommapfile().
Before this patch, there was no way for a repository owner to ensure that
validation hooks would be run by people with write access. If someone had write
access but did not trust the user owning the repository, the config and its hook
would simply be ignored.
After this patch, hooks from untrusted configs are taken into account but never
actually run. Instead they are reported as failures right away. This will ensure
validation performed by a hook is not ignored.
As a side effect writer can be forced to trust a repository hgrc by adding a
'pretxnopen.trust=true' hook to the file.
This was discussed during the 3.8 sprint with Matt Mackall, Augie Fackler and
Kevin Bullock.
We want an easy way to fetch the hook config with and without honoring
"trusted" so that we can compare the values. So we extract the part
retrieving raw hook data from the config to be able to call it twice
in the next patch.
We are about to take untrusted hooks into account (to report them as failures)
so we need to rearrange the code a bit to allow config overwriting each other
in a later patch.
