See the previous two patches for the reason. The advantage is a simplified
code base and better throughput when starting multiple servers with multiple
confighashes. The disadvantage is starting multiple servers in parallel with
a single confighash will waste some CPU time, which is probably fine in
common use-cases.
This makes it easier to switch to relative paths to support long unix domain
socket paths.
See the previous patch for motivation. Previously, the server is started at
a globally shared address. This patch appends pid to the address so it
becomes unique.
Note: with Linux pid namespace, the address may be non-unique, but it does
not affect correctness of chg - chg client will receive an redirection and
that's it.
Previously, the hash address is just appending "-$HASH" to base address.
This patch makes it truncate the basename address at "." before appending
"-$HASH".
This makes it possible to spawn new servers in a racy situation and the
client could be sure the server it connects is the new server just spawned.
This is a step towards removing the lock.
One of the functionalities of the lock is to make sure the connect will
connect to a server it just created:
1. start server --address foo
2. connect to foo # wish "foo" is the server just started
With this change, the client could do:
1. start server --address foo.tmp$PID
2. connect to foo.tmp$PID # is the server just started
(note: if it is not, it does not affect correctness - linux pid
namespace is not a concern here)
3. rename foo.tmp$PID to foo
Another functionality of the lock is to avoid starting multiple servers with
a same confighash in parallel. But that also prevents starting multiple
servers with different confighashes in parallel.
os.sep returns unicodes on Python 3. We have pycompat.ossep which returns
bytes. This patch is a part of 4 patch series which will replace all the
occurrences of os.sep to pycompat.ossep
os.pathsep returns unicode on Python 3. We already have pycompat.ospathsep
which return bytes on Python 3. This patch replaces all the occurrences of
os.pathsep in the codebase (excluding tests) to pycompat.ospathsep.
Previous, registering different object with the same name would silently
overwrite the first value with the second one. We now detect the situation and
raise an error. No extension in test or core had the issues.
This is the only place where strutil is used. I don't think it's worth to
keep the strutil module, so inline it.
Also, strutil.rfindall() appears to have off-by-one error. 'end = c - 1' is
wrong because 'end' is exclusive.
Source revision data that exists in the revmap are ignored when pulling
data from Perforce as we consider them already imported. In case where
the `convertcmd.convert` algorithm requests a commit object for such
a revision we are creating it. This is usually the case for parent of
the first imported revision.
Split fetching the `describe` form from Perforce and the commit object creation
into two functions. This allows us to reuse the commit construction for
revisions passed from a revmap.
Don't set a head revision in cases where we have a revmap but no
changesets to import, as convertcmd.convert() treats them as heads of
to-imported revisions.
Implement `common.setrevmap` which is used to pass in a file with existing
revision mappings. This functionality is used by `convertcmd.convert` if it
exists and allows implementors such as the p4 converter to make use of an
existing mapping.
We are using the revmap to abort scanning and the repository for more information
if we already have the revision. This means we are allowing incremental imports
in cases where a revmap is provided.
We are using convert_revisions in other importers. In order to unify this
we are also using convert_revision for Perforce in addition to the original
'p4'.
We are iterating over p4changes. Make the continue condition more clear
and easier to add new conditions in future patches, by removing the list
comprehension and move the condition into the existing for-loop.
Previously, changelog.appender.end() would compute the end of the file by
joining all the current appended data and checking the length. This is an O(n)
operation. 449b4adb7d39 introduced a seek call before every revlog write, which
means we are hitting this O(n) behavior n times, which causes changelog writes
during a pull to be n^2.
In our large repo, this caused pulling 100k commits to go from 17s to 130s. With
this fix, it's back to 17s.
Obviously we'd rather patch pure to have the same algorithmic win as
the C code, but this is a quick fix for the pure build since pure
isn't wrong, just not as fast as it could be.
Before this change, you could get in a start where the checker would either
complain about importing local module before stdlib one or complain about the
local one being wrongly lexically sorted with the stdlib one.
We detect the boundary and avoid complaining about lexical sort across it.
ui.load() has been available since d83ca854 and at the time of writing isn't
available on stable branch breaking benchmarking newer stable revisions.
Add historical portability policy note on contrib/benchmarks
The environment variables `HG_*` are usually used by hooks. Unlike `HGPLAIN`
etc, they do not actually affect hg's behavior. So do not include them in
confighash.
This would avoid spawning an unbound number of chg server processes if
commit hook calls hg frequently.
keys of keyword arguments on Python 3 has to be string. We are dealing with
bytes in our codebase so the keys are also bytes. Done that using
pycompat.strkwargs().
Also after this patch, `hg version` now runs on Python 3.5. Hurray!
Binary bookmark format should be used internally. It doesn't make sense to have
optional parameters `srchex` and `dsthex`. This patch removes them. It will
also be useful for `bookmarks` bundle2 part because unnecessary conversions
between hex and bin nodes will be avoided.
Next commit will remove optional parameters from `compare()` function.
Let's rename `compare()` to `comparebookmarks()` to avoid ambiguity from
callers from external extensions.
Previously, the --base option only works with a single "branch" - if there
is one changeset in the "--base" revset whose branching point(s) is/are
different from another changeset in the "--base" revset, "rebase" will error
out with:
abort: source is ancestor of destination
This happens if the user has multiple draft branches, and uses "hg rebase -b
'draft()' -d master", for example. The error message looks cryptic to users
who don't know the implementation detail.
This patch changes the logic to calculate the common ancestor for every
"base" changeset separately so we won't (incorrectly) select "source" which
is an ancestor of the destination.
This patch should not change the behavior where all changesets specified by
"--base" have the same branching point(s).
A new situation is: some of the specified changesets could be rebased, while
some couldn't (because they are descendants of the destination, or they do
not share a common ancestor with the destination). The current behavior is
to show "nothing to rebase" and exits with 1.
This patch maintains the current behavior (show "nothing to rebase") even if
part of the "--base" revset could be rebased. A clearer error message may be
"cannot find branching point for X", or "X is a descendant of destination".
The error message issue is tracked by issue5422 separately.
A test is added with all kinds of tricky cases I could think of for now.
Keys of keyword arguments need to be str(unicodes) on Python 3. We have a lot
of function where we pass keyword arguments. Having utility functions to help
converting keys to unicodes before passing and convert back them to bytes once
passed into the function will be helpful. We now have functions named
pycompat.strkwargs(dic) and pycompat.byteskwargs(dic) to help us.
getopt.getopt() deals with unicodes on Python 3 internally and if bytes
arguments are passed, then it will return TypeError. So we have now
pycompat.getoptb() which takes bytes arguments, convert them to unicode, call
getopt.getopt() and then convert the returned value back to bytes and then
return those value.
All the instances of getopt.getopt() are replaced with pycompat.getoptb().
Previously, the revlog index passed to parse_index2 must be a "string",
which means we have to read the whole revlog index into memory. This patch
makes the code accept a generic Py_buffer, to be more flexible - it could be
a "string", or anything that implements the buffer interface, like a mmap-ed
region.
Note: ideally we want to remove the "data" field. However, it is still used
in parse_index2:
if (idx->inlined) {
cache = Py_BuildValue("iO", 0, idx->data);
....
}
....
tuple = Py_BuildValue("NN", idx, cache);
....
return tuple;
Its only users are revlogio.parseindex and revlog.__init__:
# revlogio.parseindex
index, cache = parsers.parse_index2(data, inline)
return index, getattr(index, 'nodemap', None), cache
# revlog.__init__
d = self._io.parseindex(indexdata, self._inline)
self.index, nodemap, self._chunkcache = d
Maybe we could move the logic (testing inline and returnning "data" object)
to revlog.py. But that should be a separate patch.
In the next patch, we will be creating a bytes version of getopt.getopt() and
doing that will leave getopt as unused import in fancyopts. So before removing
that there are instances in codebase where instead of importing getopt, we
have used fancyopts.getopt. This patch will switch all those cases so that
the next patch can remove the import of getopt from fancyopts without breaking
things.
When we try to pass a bytes argument to a function from imp library, it
returns TypeError as it deals with unicodes internally. So we can't use bytes
with imp.* functions. Hunting through this, I found we were returning bytes
path variable to loadpath() on Python 3.5 (yes most of our codebase is
dealing with bytes on Python 3 especially the path variables). Passing unicode
does not fails the purpose of loding the extensions and a module object is
returned.
