Summary:
The `argv[0]` is used by libpython to decide the "executable path". However,
libpython cannot handle `argv[0]` being a relative path in the buck test
environment. Looking at the logic, it seems rather fragile. Therefore, use
the Rust stdlib to resolve the full path of the current executable. Pass
it as `argv[0]` to libpython, to make it able to insert the executable path
correctly.
This does affect `sys.argv[0]` seen from Python code. It is changed from a
relative path (ex. `hg`) to a full path (ex. `/bin/hg`). It seems to be
more executable and therefore might avoid some surprises.
Reviewed By: markbt, xavierd
Differential Revision: D15989622
fbshipit-source-id: 5a0d5b2f3e0b9ca1d97c84cb72d5dc079e9d3c1b
Summary:
This just rewrites the code to be shorter by avoiding testing `is_system_exit`
multiple times and avoiding some temporary variables.
Reviewed By: ikostia
Differential Revision: D15924835
fbshipit-source-id: 2bd17538ebbfde2e2865991dd9bfd778774d33b4
Summary:
Switch to the new Python runtime. Remove parts that are incompatible and no
longer necessary, including:
- Building curses, python-lz4, and urllib3 (in build_nupkg.py).
- Mangling sys.path (in hgpython and entrypoint.py).
- Zip-related logic (in hgpython).
- Cargo features controlling build environment (in hgpython and hgmain).
- Zipping Python stdlib (in setup.py).
- Shipping files in `templates`, `help`, and most `contrib` files (in
setup.py).
For the hgpython part, the new expectation is: in all cases (windows, linux,
make local, installed, buck), `edenscm.mercurial.entrypoint` and
`edenscmnative` modules are always directly importable and are always the
expected modules if imported. So the `hg` logic just imports and runs it
without having any `sys.path` related logic.
To explain it further:
- When installed on a POSIX system, the default `sys.path` contains
site-packages. Both edenscm and edenscmnative are in site-packages.
- When installed on Windows, the executable (hg.real.exe), python27.dll,
python27.zip, and edenscmnative/ are all in a same directory. Therefore the
default sys.path (exe dir + python27.zip) would be sufficient.
- When using `make local`, and run via `scm/hg/hg`, the `PySys_SetArgv` API
(called by `hgpython`) inserts the `scm/hg` directory as `sys.path[0]`,
therefore edenscm and edenscmnative in `scm/hg` will be imported as expected.
Since we no longer hard code paths to search for modules, this should fix
issues on systems with a different sys.path, ex. debian and ubuntu uses
"dist-packages" instead of "site-packages".
Note: IPython is broken. It seems to be a combination of newer Python version,
newer compiler and 64 bit (see [1]). It looks like prompt_toolkit incorrectly
uses untyped ctypes APIs where it passes "int/long"s to places expecting a
HANDLE. The ctypes library uses 4-byte integers for plain "int/long"s where a
HANDLE is 8 byte on 64 bit Windows. The new interpreter is stricter somehow
and will error out in this case (also explains why D15912516 is needed).
The fix to prompt_toolkit was sent as
https://github.com/prompt-toolkit/python-prompt-toolkit/pull/930.
[1]: https://github.com/prompt-toolkit/python-prompt-toolkit/issues/406
Reviewed By: ikostia
Differential Revision: D15894694
fbshipit-source-id: 560d11ae28c1e65d58b760eac93701e753bd397e
Summary:
The only use of it is `util.gethgcmd`. With the previous patch,
`util.gethgcmd()` is already correct using `sys.argv`. Therefore
`executionmodel` is no longer necessary.
Reviewed By: ikostia
Differential Revision: D15856611
fbshipit-source-id: 622dc949b4911be44a53e5a2487a0121c781d82b
Summary:
Before `hg.rust`, the pure Python `hg` script uses its executable name in
`sys.argv[0]`, no matter how it gets executed:
% python hg dbsh -c 'print(sys.argv)'
['hg', 'dbsh', '-c', 'print(sys.argv)']
% ./hg dbsh -c 'print(sys.argv)'
['./hg', 'dbsh', '-c', 'print(sys.argv)']
% /bin/hg.real dbsh -c 'print(sys.argv)'
['/bin/hg.real', 'dbsh', '-c', 'print(sys.argv)']
D9482436 made `hg.rust` use the path of `entrypoint.py` for `sys.argv[0]`:
(on Linux, both with and without chg)
% /bin/hg.rust dbsh -c 'print(sys.argv)'
['/usr/lib64/python2.7/site-packages/edenscm/mercurial/entrypoint.py', 'dbsh', '-c', 'print(sys.argv)']
(on OSX)
% hg.rust dbsh -c 'print(sys.argv)'
['/opt/facebook/hg/lib/python2.7/site-packages/edenscm/mercurial/entrypoint.py', 'dbsh', '-c', 'print(sys.argv)']
(on Windows)
> hg dbsh -c "print(sys.argv)"
['\\\\?\\C:\\tools\\hg\\lib\\library.zip\\edenscm\\mercurial\\entrypoint.py', 'dbsh', '-c', 'print(sys.argv)']
It's unclear why that is a desired behavior. This patch makes `hg.rust` just
use the original `argv[0]`. It allows further clean up (in upcoming diffs).
Reviewed By: ikostia
Differential Revision: D15856610
fbshipit-source-id: 6f6811ff8b15109df7d0786676de0095d547510d
Summary: We want to have just one entry point to Mercurial, namely the Rust binary. Getting rid of the `hg` Python script means that we finally can do things that only exist in Rust and not in Python.
Reviewed By: simpkins
Differential Revision: D13186374
fbshipit-source-id: f3c8cfe4beb7bf764172a8af04fd25202eca9af2
Summary:
Update all rust crates that compile on Rust 2018 to use the 2018 edition.
The `commitcloudsubscriber` crate only compiles with Rust 2015, so make that
explicit in `Cargo.toml`.
Reviewed By: farnz
Differential Revision: D15601648
fbshipit-source-id: 7380e6e695fc3049913af91fcbde105dfe1be4bc
Summary:
Move top-level Python packages `mercurial`, `hgext` and `hgdemandimport` to
a new top-level package `edenscm`. This allows the Python packages provided by
the upstream Mercurial to be installed side-by-side.
To maintain compatibility, `edenscm/` gets added to `sys.path` in
`mercurial/__init__.py`.
Reviewed By: phillco, ikostia
Differential Revision: D13853115
fbshipit-source-id: b296b0673dc54c61ef6a591ebc687057ff53b22e
Summary:
Before I implement a proper fix [1], let's just use the correct slashes.
[1]
Correct fix is de-verbatimization of canonicalized paths.
So, if `a-symlink->b` and `"C:\a".canonicalize()` produces `\\?\C:\b`, then doing `.push("c/d")` produces `\\?\C:\b\c/d`, where `c/d` is a *single* path component, becuase the path starts with `\\?\`. If there isn't such prefix, it's fine to push forward-slash-separated things into Windows paths.
Differential Revision: D13234288
fbshipit-source-id: 2ca0326bbd91ddc6ffd259153915037264292dc1
Summary: I only tested the original diff of Windows, aparently.
Reviewed By: mitrandir77
Differential Revision: D13188952
fbshipit-source-id: 9dc33cb0eedb8d3c09cb7a734528f71afd7cbe8a
Summary:
We need to canonicalize `current_exe` to resolve symlinks on OSX.
Unfortunetely, on there's no way to just resolve symlinks and not
turn path into a `\\?\` on Windows, AFAIK.
Once the path is canonicalized on Windows, it starts with `\\?\` and
forward slashes are no longer recognized as valid separators.
Here's a demonstration:
```
> cat src\main.rs
use std::path::Path;
fn main() {
let p = Path::new("\\\\?\\C:\\Code\\fbsource\\fbcode\\scm\\hg\\mercurial/entrypoint.py");
for comp in p.components() { println!("{:?}", comp); }
println!("{:?} exists: {}", p, p.exists());
let p = Path::new("\\\\?\\C:\\Code\\fbsource\\fbcode\\scm\\hg\\mercurial\\entrypoint.py");
for comp in p.components() { println!("{:?}", comp); }
println!("{:?} exists: {}", p, p.exists());
let p = Path::new("C:\\Code\\fbsource\\fbcode\\scm\\hg\\mercurial/entrypoint.py");
for comp in p.components() { println!("{:?}", comp); }
println!("{:?} exists: {}", p, p.exists());
}
> cargo run
Prefix(PrefixComponent { raw: "\\\\?\\C:", parsed: VerbatimDisk(67) })
RootDir
Normal("Code")
Normal("fbsource")
Normal("fbcode")
Normal("scm")
Normal("hg")
Normal("mercurial/entrypoint.py")
"\\\\?\\C:\\Code\\fbsource\\fbcode\\scm\\hg\\mercurial/entrypoint.py" exists: false
Prefix(PrefixComponent { raw: "\\\\?\\C:", parsed: VerbatimDisk(67) })
RootDir
Normal("Code")
Normal("fbsource")
Normal("fbcode")
Normal("scm")
Normal("hg")
Normal("mercurial")
Normal("entrypoint.py")
"\\\\?\\C:\\Code\\fbsource\\fbcode\\scm\\hg\\mercurial\\entrypoint.py" exists: true
Prefix(PrefixComponent { raw: "C:", parsed: Disk(67) })
RootDir
Normal("Code")
Normal("fbsource")
Normal("fbcode")
Normal("scm")
Normal("hg")
Normal("mercurial")
Normal("entrypoint.py")
"C:\\Code\\fbsource\\fbcode\\scm\\hg\\mercurial/entrypoint.py" exists: true
```
Differential Revision: D13176266
fbshipit-source-id: 5f35a3263e058d179b237c80f28e4fdf44105576
Summary:
This is important on OSX where `current_exe` will return the symlink address if `hg.rust` is a symlink.
Therefore, if you create a symlink to the `hg.rust` in the repo (like tests do), repo Python code won't be picked up, and the system code will be.
Reviewed By: mitrandir77
Differential Revision: D13138333
fbshipit-source-id: ffdf27329609d77bee4b8a2eecc47e02cb2dd5c8
Summary: As per quark-zju's request in the earlier diff.
Reviewed By: quark-zju
Differential Revision: D10173168
fbshipit-source-id: 20ab1fbc597b8329bbfec5dabd501d202571bdec
Summary:
Following the conversation with quark-zju, this in future will help us conditionally dynamically load
the `hgpython` `.dll`/`.so` only if we need it.
Reviewed By: quark-zju
Differential Revision: D10084949
fbshipit-source-id: c20ef014ad9922913ee36d1ec28b0555b64f7d1f
