Implement `common.setrevmap` which is used to pass in a file with existing
revision mappings. This functionality is used by `convertcmd.convert` if it
exists and allows implementors such as the p4 converter to make use of an
existing mapping.
We are using the revmap to abort scanning and the repository for more information
if we already have the revision. This means we are allowing incremental imports
in cases where a revmap is provided.
We are using convert_revisions in other importers. In order to unify this
we are also using convert_revision for Perforce in addition to the original
'p4'.
We are iterating over p4changes. Make the continue condition more clear
and easier to add new conditions in future patches, by removing the list
comprehension and move the condition into the existing for-loop.
Previously, changelog.appender.end() would compute the end of the file by
joining all the current appended data and checking the length. This is an O(n)
operation. 449b4adb7d39 introduced a seek call before every revlog write, which
means we are hitting this O(n) behavior n times, which causes changelog writes
during a pull to be n^2.
In our large repo, this caused pulling 100k commits to go from 17s to 130s. With
this fix, it's back to 17s.
Obviously we'd rather patch pure to have the same algorithmic win as
the C code, but this is a quick fix for the pure build since pure
isn't wrong, just not as fast as it could be.
Before this change, you could get in a start where the checker would either
complain about importing local module before stdlib one or complain about the
local one being wrongly lexically sorted with the stdlib one.
We detect the boundary and avoid complaining about lexical sort across it.
ui.load() has been available since d83ca854 and at the time of writing isn't
available on stable branch breaking benchmarking newer stable revisions.
Add historical portability policy note on contrib/benchmarks
The environment variables `HG_*` are usually used by hooks. Unlike `HGPLAIN`
etc, they do not actually affect hg's behavior. So do not include them in
confighash.
This would avoid spawning an unbound number of chg server processes if
commit hook calls hg frequently.
keys of keyword arguments on Python 3 has to be string. We are dealing with
bytes in our codebase so the keys are also bytes. Done that using
pycompat.strkwargs().
Also after this patch, `hg version` now runs on Python 3.5. Hurray!
Binary bookmark format should be used internally. It doesn't make sense to have
optional parameters `srchex` and `dsthex`. This patch removes them. It will
also be useful for `bookmarks` bundle2 part because unnecessary conversions
between hex and bin nodes will be avoided.
Next commit will remove optional parameters from `compare()` function.
Let's rename `compare()` to `comparebookmarks()` to avoid ambiguity from
callers from external extensions.
Previously, the --base option only works with a single "branch" - if there
is one changeset in the "--base" revset whose branching point(s) is/are
different from another changeset in the "--base" revset, "rebase" will error
out with:
abort: source is ancestor of destination
This happens if the user has multiple draft branches, and uses "hg rebase -b
'draft()' -d master", for example. The error message looks cryptic to users
who don't know the implementation detail.
This patch changes the logic to calculate the common ancestor for every
"base" changeset separately so we won't (incorrectly) select "source" which
is an ancestor of the destination.
This patch should not change the behavior where all changesets specified by
"--base" have the same branching point(s).
A new situation is: some of the specified changesets could be rebased, while
some couldn't (because they are descendants of the destination, or they do
not share a common ancestor with the destination). The current behavior is
to show "nothing to rebase" and exits with 1.
This patch maintains the current behavior (show "nothing to rebase") even if
part of the "--base" revset could be rebased. A clearer error message may be
"cannot find branching point for X", or "X is a descendant of destination".
The error message issue is tracked by issue5422 separately.
A test is added with all kinds of tricky cases I could think of for now.
Keys of keyword arguments need to be str(unicodes) on Python 3. We have a lot
of function where we pass keyword arguments. Having utility functions to help
converting keys to unicodes before passing and convert back them to bytes once
passed into the function will be helpful. We now have functions named
pycompat.strkwargs(dic) and pycompat.byteskwargs(dic) to help us.
getopt.getopt() deals with unicodes on Python 3 internally and if bytes
arguments are passed, then it will return TypeError. So we have now
pycompat.getoptb() which takes bytes arguments, convert them to unicode, call
getopt.getopt() and then convert the returned value back to bytes and then
return those value.
All the instances of getopt.getopt() are replaced with pycompat.getoptb().
Previously, the revlog index passed to parse_index2 must be a "string",
which means we have to read the whole revlog index into memory. This patch
makes the code accept a generic Py_buffer, to be more flexible - it could be
a "string", or anything that implements the buffer interface, like a mmap-ed
region.
Note: ideally we want to remove the "data" field. However, it is still used
in parse_index2:
if (idx->inlined) {
cache = Py_BuildValue("iO", 0, idx->data);
....
}
....
tuple = Py_BuildValue("NN", idx, cache);
....
return tuple;
Its only users are revlogio.parseindex and revlog.__init__:
# revlogio.parseindex
index, cache = parsers.parse_index2(data, inline)
return index, getattr(index, 'nodemap', None), cache
# revlog.__init__
d = self._io.parseindex(indexdata, self._inline)
self.index, nodemap, self._chunkcache = d
Maybe we could move the logic (testing inline and returnning "data" object)
to revlog.py. But that should be a separate patch.
In the next patch, we will be creating a bytes version of getopt.getopt() and
doing that will leave getopt as unused import in fancyopts. So before removing
that there are instances in codebase where instead of importing getopt, we
have used fancyopts.getopt. This patch will switch all those cases so that
the next patch can remove the import of getopt from fancyopts without breaking
things.
When we try to pass a bytes argument to a function from imp library, it
returns TypeError as it deals with unicodes internally. So we can't use bytes
with imp.* functions. Hunting through this, I found we were returning bytes
path variable to loadpath() on Python 3.5 (yes most of our codebase is
dealing with bytes on Python 3 especially the path variables). Passing unicode
does not fails the purpose of loding the extensions and a module object is
returned.
This is an example usage of ProgrammingError. Let's start migrating
RuntimeError to ProgrammingError.
The code only runs when devel.all-warnings or devel.check-locks is set, so
it does not affect the end-user experience.
We have requirement to express "this is clearly an error caused by the
programmer". The code base uses RuntimeError for that in some places, not
ideal. So let's add a formal exception for that.
The path variable in localrepository.__init__() has a default value None. So
it gives us a option to create an object to localrespository class without
path variable. But things break if you try to do so. The second line in the
init which will be executed when we try to create a localrepository object
will call os.path.expandvars(path) which returns
TypeError: argument of type 'NoneType' is not iterable
I checked occurrences when it is called and can't find any piece of code
which calls it without path variable. Also if something is calling it, its
should break.
"In this case, result is a source variable of a list to be returned, it
shouldn't be unicode. Hence we can use bytes instead of basestring here." -Yuya
__slots__ in Python 3 accepts only unicodes and there is no harm using
unicodes in __slots__. So just adding u'' is fine. Previous occurences of this
problem are treated the same way.
When we have a lot of files writing a new manifest revision can be expensive.
This commit adds a possibility for memctx to reuse a manifest from a different
commit. This can be beneficial for commands that are creating metadata changes
without any actual files changed like "hg metaedit" in evolve extension.
I will send the change for evolve that leverages this once this is accepted.
In preparation for adding another value to it in a subsequent patch.
While I was here, I added some empty lines because walls of text
are hard to read.
Previously, the capabilities list was protocol agnostic and we
advertised the same capabilities list to all clients, regardless of
transport protocol.
A few capabilities are specific to HTTP. I see no good reason why we
should advertise them to SSH clients. So this patch limits their
advertisement to HTTP clients.
This patch is BC, but SSH clients shouldn't be using the removed
capabilities so there should be no impact.
We add an attribute to the HTTP and SSH protocol implementations
identifying the transport so future patches can conditionally
expose capabilities on a per-transport basis.
It seems quite common that files don't change completely. New lines are often
pretty much appended, and modifications will often only change a small section
of the file which on average will be in the middle.
There can thus be a big win by pruning a common prefix before starting the more
expensive search for longest common substrings.
Worst case, it will scan through a long sequence of similar bytes without
encountering a newline. Splitlines will then have to do the same again ...
twice for each side. If similar lines are found, splitlines will save the
double iteration and hashing of the lines ... plus there will be less lines to
find common substrings in.
This change might in some cases make the algorith pick shorter or less optimal
common substrings. We can't have the cake and eat it.
This make hg --time bundle --base null -r 4.0 go from 14.5 to 15 s - a 3%
increase.
On mozilla-unified:
perfbdiff -m 3041e4d59df2
! wall 0.053088 comb 0.060000 user 0.060000 sys 0.000000 (best of 100) to
! wall 0.024618 comb 0.020000 user 0.020000 sys 0.000000 (best of 116)
perfbdiff 0e9928989e9c --alldata --count 10
! wall 0.702075 comb 0.700000 user 0.700000 sys 0.000000 (best of 15) to
! wall 0.579235 comb 0.580000 user 0.580000 sys 0.000000 (best of 18)
This allows us to write doctests depending on a ui object, but not on global
configs.
ui.load() is a class method so we can do wsgiui.load(). All ui() calls but
for doctests are replaced with ui.load(). Some of them could be changed to
not load configs later.
See the commit message of the previous patch for the reason. In short,
according to the current POSIX standard, "-r" is "removed", and "-R" is the
current standard way to do "copy file hierarchies".
The POSIX documentation about "cp" [1] says:
....
RATIONALE
....
Earlier versions of this standard included support for the -r option to
copy file hierarchies. The -r option is historical practice on BSD and
BSD-derived systems. This option is no longer specified by POSIX.1-2008
but may be present in some implementations. The -R option was added as a
close synonym to the -r option, selected for consistency with all other
options in this volume of POSIX.1-2008 that do recursive directory
descent.
The difference between -R and the removed -r option is in the treatment
by cp of file types other than regular and directory. It was
implementation-defined how the - option treated special files to allow
both historical implementations and those that chose to support -r with
the same abilities as -R defined by this volume of POSIX.1-2008. The
original -r flag, for historic reasons, did not handle special files any
differently from regular files, but always read the file and copied its
contents. This had obvious problems in the presence of special file
types; for example, character devices, FIFOs, and sockets.
....
....
Issue 6
The -r option is marked obsolescent.
....
Issue 7
....
The obsolescent -r option is removed.
....
(No "Issue 8" yet)
Therefore it's clear that "cp -R" is strictly better than "cp -r".
The issue was discovered when running tests on OS X after 2e4d149e62aa.
[1]: pubs.opengroup.org/onlinepubs/9699919799/utilities/cp.html
The NamedTemporaryFile file is cleared up so checklink ends up as a dangling
symlink, causing cp -r in tests to complain on both Solaris and OS X. Use
a permanent file instead when there is a .hg/cache directory.
We are using 'name + ".patch"' pattern throughout the shelve code to
identify the existence of a shelve with a particular name. In two
cases however we use 'name + ".hg"' instead. This commit makes
'patch' be used in all places and "emphasizes" it by moving
'patch' to live in a constant. Also, this allows to extract file
name without extension like this:
f[:-(1 + len(patchextension))]
instead of:
f[:-6]
which is good IMO.
This is a first patch from this initial "obsshelve" series. This
series does not include tests, although locally I have all of
test-shelve.t ported to test obs-shelve as well. I will send tests
later as a separate series.
