Summary:
Move top-level Python packages `mercurial`, `hgext` and `hgdemandimport` to
a new top-level package `edenscm`. This allows the Python packages provided by
the upstream Mercurial to be installed side-by-side.
To maintain compatibility, `edenscm/` gets added to `sys.path` in
`mercurial/__init__.py`.
Reviewed By: phillco, ikostia
Differential Revision: D13853115
fbshipit-source-id: b296b0673dc54c61ef6a591ebc687057ff53b22e
Summary:
This steals lots of code from [1] and uses dev versions of crates from [2]
as well.
This approach is better than calling things via `Py_Main` directly because
it allows us to pre-populate the container with some data (pre-population
needs to happen after `Py_Initialize` and before passing control to
current Python-based hg).
NB: returning `1` when the exception instance does not have a `code` attribute is an experimentally-determined decision. With `255` tests fail, so `1` just matches Python behavior. I guess we can change it anytime if we come up with exit code globbing for the tests.
Reviewed By: quark-zju
Differential Revision: D9482436
fbshipit-source-id: 9ca63f2358ddc1de60228c489aa9d02b4b7bc482
Summary:
This script takes care of the "one-time" setup. So the following things
would work:
- make local
- locally built "hg"
- tests/run-tests.py
Targing Windows (main goal), OSX, and CentOS.
Other scripts or hacks doing similar things are removed.
Reviewed By: phillco
Differential Revision: D9506352
fbshipit-source-id: dbc47e11f6988224c7c2cb59fb36b75ba7f3704b
Summary:
We want to have two possible Python entry points for Mercurial:
- pure-python one (`hg/hg` script, the one which is renamed into `hg.real` during FB installation)
- the one to be called from a Rust binary
The reason we can't reuse the `hg/hg` to be called form the Rust binary is because this script will eventually not exist. We also need something in the known location, so we chose to put `entrypoint.py` into `mercurial/`.
This means that both the `hg` script and the `hg.rust` binary need to be able to find the `mercurial/` directory on the system. Traditionally, `hg` contained a `LIBDIR@` string, which was replaced with the parent directory of a `mercurial/` dir at install time. Thus we could install Mercurial in non-standard locations if we needed to. We keep this functionality for the `hg` script.
`mercurial/entrypoint.py` however knows that it lives under `mercurial/` and that `hgext/` and `hgdemandimport/` live alongside `mercurial/`. Thus, the parent dir of `mercurial/` is added as a first item of `sys,path` in `entrypoint.py`.
In case `entrypoint.py` is called from a Rust binary, Python import logic also adds the entire `mercurial/` dir to `sys.path`. This is not desired, since we use `absolute_import` and want to only be able to import things as `from mercurial import smth`. A good example is `json`: even with `absolute_import` enabled `import json` loads `mercurial/json.py` if hg is called as a Rust binary. Therefore, we explicitly remove `mercurial/` from `sys.path`.
Reviewed By: quark-zju
Differential Revision: D9336157
fbshipit-source-id: 22f69e7782d549915c91ef9a0ad0ed29f62a9304
Summary:
Motivated by recent D7784903 which kills chg because it holds blackbox.log
file descriptor, and that patch is causing race conditions running with chg
(chg's sock atomic rename might fail if the directory is deleted).
There are other ways to solve the direct issue. This diff takes a more
aggressive but much cleaner approach. Basically, the `hg serve` framework is
too late for chg's usecase - the repo was already loaded, extension
side-effects have been already done at that time - chg has to use
workarounds to be compatible with that. Even with a best effort, it is still
possible to have weird interactions with shared repo because how hg loads
extensions.
The new approach is to pre-import a list of bundled extensions but do not
run their `uisetup`s. This solves a couple of hard problems:
- Compatibility - `uisetup` runs per request. That behaves exactly as what
an extension author expects.
- Less memory usage - there is no `repo` object is loaded in memory.
- Reduced process count - since extension config change does not require a
new chg server, the count of server processes would decrease (ex.
`--config extensions.blackbox=!` won't require a new chg server)
- Not holding fd to edenfs, since neither blackbox nor repo is loaded. This
makes it possible to remove the hacky killing chg logic in D7784903.
The downside is performance, since extension loading, and `uisetup` will run
every time. Benchmark shows that's could be 50ms-ish. But we could move
forward by moving extension logic to core incrementally to get rid of that
cost too.
This is basically a simplified version of my previous stack starting
with [1]. The original commit message was:
This is the beginning of a series that does a re-architect to chg mentioned
in [1], to achieve better compatibility.
The compatibility issues are mainly around "uisetup"s and "reposetup"s:
- Developers are usually unaware that uisetup runs only once per chg
process. We cannot reliably devel-warn them. The result is, potential
broken code are written. For example, it's really hard for chg to deal
with "experimental.evolution" changed from unset to manually set in
config files because setconfig is used if that config option is not set.
- An unnecessary "reposetup" caused by "hg serve" may have unwanted
side effects. This can become troublesome if the repo requires things
like remotefilelog or lz4revlog, and the user sets HGRCPATH to run
tests.
The current chg implementation assumes that "loading" an extension is not
side effect free - if extension related config has changed, a restart is
needed. The new idea is, "loading" = "importing" + "run ui/extsetup", the
"importing" part can be side-effect free for some extensions. And benchmark
shows "import" takes most of the time consumed, while "uisetup" is usually
very fast. We can afford running "uisetup"s per request.
To be able to (pre-)"import" extensions without running any "uisetup"s, a
different entry point is needed. Otherwise as long as we go through the
normal dispatch / runcommand ("hg serve") flow, "uisetup"s cannot be
avoided.
Aside from better compatibility, we can also remove some hacks:
- chg client: no longer needs to extract sensitive argv
- chg server: confighash can be changed to only hash environment variables
(reduce the number of server processes)
- chg server: srcui.walkconfig hack is no longer necessary
This patch adds a new script "chgserve" as the new entry point. Currently,
it is just a minimal implementation that makes "CHGHG=chgserve chg ..."
work, without doing any pre-importing. The change could also be done in the
"hg" script. But since chg is still experimental, let's keep "hg" untouched
for now.
[1]: www.mercurial-scm.org/pipermail/mercurial-devel/2016-July/085965.html
[1]: 6f91a1a69f
Reviewed By: singhsrb
Differential Revision: D7840237
fbshipit-source-id: e3d613b41fe4b721238b86c5bf84434d32cf0609
This uses the new demandimport implementation for Python 3 introduced in
previous patches.
This doesn't yet enhance performance because it isn't integrated with the
custom source file loader we use on Python 3. We'll integrate the two in
upcoming patches.
This lets us easily verify that there are no implicit conversions
between unicodes and bytes in Mercurial's codebase. Based on something
mpm did by hand periodically, but it kept regressing, so just open the
door to running it in a buildbot.
This provides two new features:
- Mercurial may be installed into a non-standard location without
having to set PYTHONPATH.
- Multiple installations can use Mercurial from different locations.
Standard streams are expected to operate in binary mode everywhere, especially with archive, cat, diff and export commands. Rewriting these to separate informational output from binary content is complicated to do and to maintain, nonwithstanding mode switching reliability. Changing all output mode to binary should not have much impact on Windows were stream processing tools are barely used and usually cope with unix style endings.
Streams mode being process wide, the switch is performed in the startup script to avoid polluting existing API users who may have solved this issue already or ignored it at least for the mercurial part.
- move command dispatching functions from commands and cmdutil to dispatch
- change findcmd to take a table argument
- remove circular import of commands in cmdutil
- privatize helper functions in dispatch
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
import and startup cleanups
add commands:run()
add copyright notice to commands
eliminate/reorganize imports to speed up start time:
0.5b:
$ time bash -c 'for i in `seq 100`; do ~/bin/hg > /dev/null; done'
real 0m7.718s
user 0m6.719s
sys 0m0.794s
new:
$ time bash -c 'for i in `seq 100`; do hg > /dev/null; done'
real 0m2.171s
user 0m1.684s
sys 0m0.444s
just python:
$ time bash -c 'for i in `seq 100`; do python -c pass; done'
real 0m0.988s
user 0m0.771s
sys 0m0.207s
Ignoring the fixed cost of loading the Python interpreter, we're 5.6
times faster. With the Python load time, we're still 3.5 times faster.
manifest hash: acce5882a55c76eb165316f5741724c8ce4ef587
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)
iD8DBQFCoihAywK+sNU5EO8RAqMdAJwMe6Ur0R9G6jjayNa5hH2C3c4k/gCeIYvc
N178vaWWGciX9zq+g5qCAls=
=buhv
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
migrate verify
Move the bulk of the verify code into the localrepository class and move
the command into commands.py
manifest hash: 793a8d0094d56ab0a411cd11d7fe7f39c923f209
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)
iD8DBQFCog33ywK+sNU5EO8RApfBAJ4mCmiMmZE1fEfbR6sA+aP1csPvqQCfXHzY
3XK7yc19AivXf5HGKEOL3eM=
=GISf
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
big heap of command clean-up work
Migrate add, forget, remove, commit, diff, addremove, tip, log,
recover, and serve.
Fix up filterfiles, relfilter, and relpath to be a bit more bulletproof
Alphabetize functions and the command table
Make everything in commands.py relative-path aware
manifest hash: f0856031a7be4e49289677b467f29bcf24ebce4a
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)
iD8DBQFCof6gywK+sNU5EO8RAoW1AJsHu8vchPSjls7wVbvsq/UKlGhqtgCgtnnl
xSBxyf/TEVWjHIk3uTa8WSE=
=YPMl
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
move repo.current to dirstate.parents()
dirstate now tracks the parents for the working dir
add a parents command to show them
manifest hash: cd69237838c3f69f7937723c4a6803d47cb27cfa
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)
iD8DBQFCoMGuywK+sNU5EO8RAg5UAKCVLUrsJtkoIOTM+e0BLqEVN3Ni3gCeNDyy
ZF8jD728cl9K7S4sIN4gX4Y=
=P4bu
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
merge: don't bail on outstanding changes
With multiple heads, we don't need to worry about the working dir's
uncommitted changes at pull time
manifest hash: 5b4e024f220fa616732310ce5f48e71abfa910e0
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)
iD8DBQFCoMFQywK+sNU5EO8RApLyAKCoNDF84wFzgnpS+WLuXdkGxeHFPwCdFsMy
CysB458dNcFuB/vDFhgJr58=
=gG+u
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
fix bad assumption about uniqueness of file versions
Mercurial had assumed that a given file hash could show up in only one
changeset, and thus that the mapping from file revision to changeset
was 1-to-1. But if two people perform the same edit with the same
parents, we can get an identical hash in different changesets.
So we've got to loosen up our uniqueness checks in addgroup and in
verify.
manifest hash: 5462003241e7d071ffa1741b87a59f646c9988ed
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)
iD8DBQFCoMDkywK+sNU5EO8RAg9PAJ9YWSknfFBoeYve/+Z5DDGGvytDkwCgoMwj
kT01PcjNzGPr1/Oe5WRvulE=
=HC4t
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
change dircache into dirstate
The dircache now tracks adds and removes directly
diffdir now makes a proper distinction between added and unknown files
Add a forget command to unadd files
Undo tries to fix up the state of just the files in the undone commit
Add and remove complain about files that are not in a proper state of
existence
manifest hash: ca0cd6abc5e119670acf11a54fefa2bc986eadf3
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)
iD8DBQFCn7TRywK+sNU5EO8RAhnSAKC2oHg1HJOCGsvpUYj4SBEq0HmuJQCgr5gl
jEBTs5AFD5IhF73YAgrcnkE=
=prQA
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
hg checkout: refuse to checkout if there are outstanding changes
This is a stop-gap until I make the working dir logic smarter
manifest hash: a3f6adcb7eecec294000039057d59771958f4186
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)
iD8DBQFCnnrKywK+sNU5EO8RAtqBAJwPQQrW5GhjMP9HMkFtfD7qhqxIcgCfXvA4
oXHO13uzBn5JOaTH3KwsMbQ=
=IzTY
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Beginning of new command parsing interface
This adds commands.py, with a primary interface dispatch(args)
Dispatch searches a table of known commands, handles switches, sets up
a repo object if appropriate, and dispatches the command.
It also handles KeyboardInterrupt and can handle similar exceptions in
the future.
If the command is unknown, it falls through to the current command handler.
Commands currently handled by the new scheme: help, init, and annotate
manifest hash: 134cd032c880985e3f92f82efb8b629dd862ba4c
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)
iD8DBQFCnXEGywK+sNU5EO8RAuDAAJ9q7K4w7qGVWv1NWjCPFGO/UJc6VQCdEhMQ
sBBlSRzah9QPy8K94catZyg=
=wuRf
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
hg rawcommit command
From: Christopher Li <hg@chrisli.org>
This allows direct access to the commit command, primarily for
importing from other SCMs.
manifest hash: bea39fa8207582c9fa7ba0904721eb5113c61cf4
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)
iD8DBQFCnUinywK+sNU5EO8RAhWqAJ9PiafRbfEIA3VsO07BbGZr5adNvgCfT2k7
blYTdkrIiRzzCxn6yPq8Yu4=
=o8k0
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Bump the version number to 0.5b for the protocol change
manifest hash: a7930fa15b716eb90613bd761b47c27331ea4b8b
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)
iD8DBQFCmz7pywK+sNU5EO8RAt7dAJ4qmUpDRS7/JP/JpLm8uXZ0c+5W/ACfVb0Q
99rjYslSjJfOWYLCKiAzVyU=
=WVVg
-----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Changes to network protocol
Stream changes at the delta level rather than at whole delta groups
this breaks the protocol - we now send a zero byte delta to indicate
the end of a group rather than sending the entire group length up front
Fix filename length asymmetry while we're breaking things
Fix hidden O(n^2) bug in calculating changegroup
list.append(e) is O(n), list + [element] is not
Decompress chunks on read in revlog.group()
Improve status messages
report bytes transferred
report nothing to do
Deal with /dev/null path brokenness
Remove untriggered patch assertion
manifest hash: 3eedcfe878561f9eb4adedb04f6be618fb8ae8d8
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)
iD8DBQFCmzlqywK+sNU5EO8RAn0KAJ4z4toWSSGjLoZO6FKWLx/3QbZufACglQgd
S48bumc++DnuY1iPSNWKGAI=
=lCjx
-----END PGP SIGNATURE-----
