Summary:
The workspace was added by D8741175 mainly to make build artifacts share a same
"target" directory. D14606468 made `setup.py` write a `.cargo/config` that
specifies a sharable "target" directory. Remove the workspace since the crates
already share a "target" directory.
This makes it a bit easier to add new crates.
Reviewed By: singhsrb
Differential Revision: D17053934
fbshipit-source-id: d34781c796356b725ddce3453c1951a4d4133807
Summary:
With fastdatapack gone, the cdatapack bindings are no longer necessary and can
be removed.
Reviewed By: singhsrb
Differential Revision: D16476584
fbshipit-source-id: 130a9c5aed4e4f005876c420961f09d398f6e6aa
Summary:
This utility can be useful to unblock people with corrupted indexed logs after
hard reboot.
Reviewed By: xavierd
Differential Revision: D16433835
fbshipit-source-id: 72f0aff4b266072c58d2c574897e580a1529b57f
Summary:
The `fbsourcepylibrary` class only copies the source files if the destination
directory does not exist. Once the files have been copied, subsequent builds
will not pick up new versions. This means builds can be stale.
Remove the special case of `_isready`. This will fall back to the default
implementation which will copy the files each time the build happens.
Reviewed By: quark-zju
Differential Revision: D16438778
fbshipit-source-id: 66dc0b69b427650087cfb822acdbf1fc797babbc
Summary:
We install the python files under `eden/fs/service` in a package named
`eden/thrift` in the built python binaries that use these modules. This
moves the source files into an `eden/thrift` subdirectory so that the source
directory layout more closely matches the final binary layout.
This will make it easier to run several of Eden's Python-based tools directly
from the source tree, without having to do as much directory layout
transformation. This is particularly helpful on platforms like Windows, which
don't currently have an equivalent of "live PARs" which can be run from the
source tree without requiring a rebuild after each file edit.
Reviewed By: chadaustin
Differential Revision: D16354622
fbshipit-source-id: 4b58cc96451b1ee5441714aaf74f5e3b6ada9eaa
Summary:
We install `eden/py/dirstate.py` as `eden/dirstate.py` in built python
binaries that use this module. This moves the source file into an `eden/`
subdirectory so that the source directory layout more closely matches the
final binary layout.
This will make it easier to run several of Eden's Python-based tools directly
from the source tree, without having to do as much directory layout
transformation. This is particularly helpful on platforms like Windows, which
don't currently have an equivalent of "live PARs" which can be run from the
source tree without requiring a rebuild after each file edit.
Reviewed By: chadaustin
Differential Revision: D16354627
fbshipit-source-id: 578748e76b730db33cf3ea555df48aa94d15019f
Summary: IPython.zip contains not only just IPyhton dependencies. Rename it to clarify.
Reviewed By: xavierd
Differential Revision: D16167579
fbshipit-source-id: c7bbd24c8efd58ed322bfb4520b9b2454b919978
Summary:
A previous change over simplified contrib/ content. `editmergeps.bat` is
necessary. Include it in packaing.
Reviewed By: ikostia
Differential Revision: D16199596
fbshipit-source-id: e48b6ac81d50a02a885723bcd31000b8d983bfef
Summary:
The most important template files are now shipped in the code. Assuming hgweb
is not used, the rest of template files are not necessary. Therefore do not
distribute them. The contributing guide is also heavily outdated and was
removed.
Reviewed By: ikostia
Differential Revision: D15915003
fbshipit-source-id: b080337bbd07775f9ec0643c34d7634ade1afdc7
Summary:
Switch to the new Python runtime. Remove parts that are incompatible and no
longer necessary, including:
- Building curses, python-lz4, and urllib3 (in build_nupkg.py).
- Mangling sys.path (in hgpython and entrypoint.py).
- Zip-related logic (in hgpython).
- Cargo features controlling build environment (in hgpython and hgmain).
- Zipping Python stdlib (in setup.py).
- Shipping files in `templates`, `help`, and most `contrib` files (in
setup.py).
For the hgpython part, the new expectation is: in all cases (windows, linux,
make local, installed, buck), `edenscm.mercurial.entrypoint` and
`edenscmnative` modules are always directly importable and are always the
expected modules if imported. So the `hg` logic just imports and runs it
without having any `sys.path` related logic.
To explain it further:
- When installed on a POSIX system, the default `sys.path` contains
site-packages. Both edenscm and edenscmnative are in site-packages.
- When installed on Windows, the executable (hg.real.exe), python27.dll,
python27.zip, and edenscmnative/ are all in a same directory. Therefore the
default sys.path (exe dir + python27.zip) would be sufficient.
- When using `make local`, and run via `scm/hg/hg`, the `PySys_SetArgv` API
(called by `hgpython`) inserts the `scm/hg` directory as `sys.path[0]`,
therefore edenscm and edenscmnative in `scm/hg` will be imported as expected.
Since we no longer hard code paths to search for modules, this should fix
issues on systems with a different sys.path, ex. debian and ubuntu uses
"dist-packages" instead of "site-packages".
Note: IPython is broken. It seems to be a combination of newer Python version,
newer compiler and 64 bit (see [1]). It looks like prompt_toolkit incorrectly
uses untyped ctypes APIs where it passes "int/long"s to places expecting a
HANDLE. The ctypes library uses 4-byte integers for plain "int/long"s where a
HANDLE is 8 byte on 64 bit Windows. The new interpreter is stricter somehow
and will error out in this case (also explains why D15912516 is needed).
The fix to prompt_toolkit was sent as
https://github.com/prompt-toolkit/python-prompt-toolkit/pull/930.
[1]: https://github.com/prompt-toolkit/python-prompt-toolkit/issues/406
Reviewed By: ikostia
Differential Revision: D15894694
fbshipit-source-id: 560d11ae28c1e65d58b760eac93701e753bd397e
Summary: setuptools is not part of the Python stdlib. Just use distutils.
Reviewed By: ikostia
Differential Revision: D15894697
fbshipit-source-id: a101deaf4c9b5ee73961e4bb46a9d3f506d92a8c
Summary:
The `build_py` command is not expected to compile native extensions, so it
shouldn't test `Python.h`. If `Python.h` is missing, `build_ext` will fail
anyway.
More context:
`build_py` uses `get_python_inc` to test `Python.h`. `get_python_inc` relies
on `sys.prefix`. `sys.prefix` can be empty in some setups (ex. if the Python
runtime from build/ is used on Windows).
Reviewed By: xavierd
Differential Revision: D15894696
fbshipit-source-id: 93bfda90e137b71ac5a778d2d790392cf9fd125b
Summary:
This just moves things around. So native and pure Python modules are split to
different Python packages. This makes it possible to use the standard zip
importer without hacks (ex. `hgdemandimport/embeddedimport`).
This diff is mostly about moving things. While `make local` still works,
it does break nupkg build, which will be fixed in a later diff.
Reviewed By: kulshrax
Differential Revision: D15798642
fbshipit-source-id: 5d83f17099aa198df0acd5b7a99667e2f35fe7b4
Summary: We want to have just one entry point to Mercurial, namely the Rust binary. Getting rid of the `hg` Python script means that we finally can do things that only exist in Rust and not in Python.
Reviewed By: simpkins
Differential Revision: D13186374
fbshipit-source-id: f3c8cfe4beb7bf764172a8af04fd25202eca9af2
Summary: This will hopefully make the `eden` module available on Windows.
Reviewed By: pkaush
Differential Revision: D15526731
fbshipit-source-id: 6c4fe5d6d298ae271fbc86b68ffb9905199cbf7b
Summary:
devservers have both `scratch` and `mkscratch`. `mkscratch` is
the binary that gets distributed with the hgbuild and is the one that is
available on mac (and maybe also windows?).
I plan to kill the old `scratch` package at some point on linux.
Reviewed By: simpkins
Differential Revision: D15479682
fbshipit-source-id: db78cc640ccaeb174b23d808fecbe7875fb656dd
Summary:
After D15266191, the bindings crate directly depends on pyrevisionstore, and
since then we have seen really weird compilation errors that magically go away
when some files are touched.
From my observations, here is what I came up with. The pyrevisionstore crate is
both compiled as a dependency of the bindings crate, and as a library to be
used in python. Therefore, its Cargo.toml contains a '[lib]' section, the
presence of this section forces the crate to be compiled into a
"libpyrevisionstore.{rlib,so}", while all the regular crates have a hashed
suffix like "libedenapi-2b9975ec9b865e87.rlib".
None of this would usually matter, but our build system re-uses the build
directory to then compile the pyrevisionstore library. While doing so, it will
re-create the "libpyrevisionstore.{rlib,so}", but not in an identical fashion.
After this, the rest of the build succeeds.
Once a file in the bindings crate is touched, recompiling will only recompile
its file, and not the pyrevisionstore crate, but since
"libpyrevisionstore.{rlib,so}" is different, it now fails to compile...
A previous effort tried to compile each top level target into a separate
directory, but that directly affected some of our tests. Since the underlying
issue is that pyrevisionstore is compiled twice, let's simply not compile it as
a top-level target and simply fold it into the bindings lib. Ideally, we would
want to do the opposite, but let's do that at a later time.
Reviewed By: DurhamG
Differential Revision: D15381038
fbshipit-source-id: 047cfab794371464942f19084ffb9240e836cb40
Summary: Similar to D7562864, but work with cargo workspaces.
Reviewed By: singhsrb
Differential Revision: D15084323
fbshipit-source-id: 3c15f2ceabb73dd54028523b6da5eb7857e7c842
Summary:
Using cargo for development using mercurial will result in use of random target
directories depending on where cargo is invoked from.
This diff sets the target directory for cargo in the on disk config file so
that all invocations of cargo from our project reuse the previously built
artifacts.
This may make it more problematic to build for multiple targets because cargo
does not recompile when flags change.
Reviewed By: quark-zju
Differential Revision: D14606468
fbshipit-source-id: 56c0b3b192ee94e1ddafe0949cc2267bb995a696
Summary:
Otherwise `hg.rust` might use a stale libchg.a and cannot pass the
client/server version check when installed.
Reviewed By: sfilipco
Differential Revision: D14531275
fbshipit-source-id: 6d8f04a75d91ad2c25ef2002ee3c3f9848094578
Summary:
BuildInfo symbols are generated for fbcode binaries, and scanned by the
internal procprint tool. The procprint tool also scans hg binaries, but cannot
extract any build information.
The procprint data is used by Push4Push to find out job owners running ancient
software.
This diff inserts BuildInfo symbols so Push4Push (and we) can use them to
figure out people running old versions of hg and send tasks to them.
Note rpmbuild strips symbols by default and is hard to customize. For now, I
just disable the debuginfo packages to preserve the symbols. If space usage or
runtime performance becomes an issue, we can try other workarounds (ex. provide
a shim of `%_find_debuginfo_dwz_opts` that skips `hg.rust`, or provide a shim
of `%__strip`, etc).
By default, the symbols only get added when `setup.py` is run by `rpmbuild`.
So `make local` won't rebuild `hg.rust` every time.
Reviewed By: singhsrb
Differential Revision: D14464056
fbshipit-source-id: ac6f9dedf28cee5fc8d65e144e3d7d1c544d1eda
Summary: We don't run this binary anymore, no reason to build and ship it.
Reviewed By: quark-zju
Differential Revision: D14437317
fbshipit-source-id: dd6da521783f18a2a518a7aa042be98950894e89
Summary:
There are not a lot of users of fbcode lfs pointers. Let's just merge the hg
one into it. This makes it easier to upload things without worrying about which
lfs-pointer file to use.
Reviewed By: singhsrb
Differential Revision: D14381672
fbshipit-source-id: f3ecb996e094d41605e3a883a83bd3e96b5bcd55
Summary:
This commit is to start including the memcommit extension in the
Mercurial package.
Reviewed By: quark-zju
Differential Revision: D14293241
fbshipit-source-id: c6e840d66e7e31b35d6c6aea176043cf56a16ad2
Summary: Build openssl using vcpkg. Then zip its build results.
Reviewed By: kulshrax
Differential Revision: D14197306
fbshipit-source-id: 7f019f52fb93118cb7c20c95cf41aedf3a1fbefc
Summary:
My goal is to have `mkscratch` get deployed with the mercurial
release on all platforms. This is currently deployed via packman
as `scratch` on devservers, and that packman build should continue to
succeed because the TARGETS file in here is left unchanged.
That will allow both `scratch` and `mkscratch` to coexist for a while
until we've made it through a few mercurial releases, and then we can
either fixup existing users of `scratch` to use `mkscratch`, or re-do
the `fb-scratch` rpm so that it symlinks to `mkscratch`.
I don't really know if this diff is sufficient to include the binary
in the mercurial package; I'm cargo culting the `scm_daemon` stuff
and that seems to be included in the omnibus, so maybe this is it?
Reviewed By: quark-zju
Differential Revision: D14060864
fbshipit-source-id: 5fd74a877437d84fe7676819c2cefb902fb7002d
Summary: Move the edenapi Python bindings into the common `bindings` crate.
Reviewed By: quark-zju
Differential Revision: D13963179
fbshipit-source-id: 76dead82af992615a9e452ee6fbb9f66639c822c
Summary: This wasn't being included before.
Reviewed By: vipannalla, quark-zju
Differential Revision: D14071697
fbshipit-source-id: 15f78c74a9629a8000ec1125c3e9e30e124c3d51
Summary:
On windows we use zip files to load the mercurial packages, but this
was failing because the edenscm module did not have an __init__.py file. Let's
include that in the build.
Reviewed By: singhsrb
Differential Revision: D13974190
fbshipit-source-id: 90e2cc5256fa08281202e11292e3ddf4cc000c01
Summary: Rename Mononoke API to Eden API, per war room discussion.
Reviewed By: quark-zju
Differential Revision: D13908195
fbshipit-source-id: 94a2fe93f8a89d0c5e9b6a24939cc4760cfaade0
Summary:
As requested by DurhamG, bundle DotSlash in the same way that scmtelemetry
is bundled rather than as directly part of the Mercurial package.
This approach makes it possible for DotSlash to leverage Hg's contbuild
while still being in its own package.
To implement this, I copypasta'd scmtelemetry's setup and modified it
for DotSlash, as appropriate.
Although I tried to port the Windows configuration over in addition to
Mac and Linux, I'm going to disable it for now because I want to limit
how many things I'm going to try to get working on the initial rollout.
Reviewed By: singhsrb
Differential Revision: D13950939
fbshipit-source-id: 5d8393d756e23f1ac228c7315a4b95a36288ac3a
Summary:
Move all Rust bindings to a single python extension, `bindings`. This should
improve compilation time and make things simpler.
Reviewed By: quark-zju
Differential Revision: D13923866
fbshipit-source-id: 560592b5a6c0c4f1b836c755ef123666a1059164
Summary:
This change is to get DotSlash (https://fb.quip.com/7BbiAmxAK09T)
in the Mercurial release. It renames the build rule from `dotslash` to
`dsl` under the assumption that that determines the name of the binary
to be deployed.
Reviewed By: quark-zju
Differential Revision: D13868779
fbshipit-source-id: dd44d874a60ab85b3dcca79ba5076a6f2fa64135
Summary:
`.hg-vendored-crates` is no longer a thing.
Now, the hg vendored crates always match tp2 crates, managed by the
`fbcode/tools/lfs/crates-io.py` script.
Reviewed By: ikostia
Differential Revision: D13873209
fbshipit-source-id: 6b5782b4834c49fa7d88aa1870703138eb9feb00
Summary:
D13853115 adds `edenscm/` to `sys.path` and code still uses `import mercurial`.
That has nasty problems if both `import mercurial` and
`import edenscm.mercurial` are used, because Python would think `mercurial.foo`
and `edenscm.mercurial.foo` are different modules so code like
`try: ... except mercurial.error.Foo: ...`, or `isinstance(x, mercurial.foo.Bar)`
would fail to handle the `edenscm.mercurial` version. There are also some
module-level states (ex. `extensions._extensions`) that would cause trouble if
they have multiple versions in a single process.
Change imports to use the `edenscm` so ideally the `mercurial` is no longer
imported at all. Add checks in extensions.py to catch unexpected extensions
importing modules from the old (wrong) locations when running tests.
Reviewed By: phillco
Differential Revision: D13868981
fbshipit-source-id: f4e2513766957fd81d85407994f7521a08e4de48
Summary:
Move top-level Python packages `mercurial`, `hgext` and `hgdemandimport` to
a new top-level package `edenscm`. This allows the Python packages provided by
the upstream Mercurial to be installed side-by-side.
To maintain compatibility, `edenscm/` gets added to `sys.path` in
`mercurial/__init__.py`.
Reviewed By: phillco, ikostia
Differential Revision: D13853115
fbshipit-source-id: b296b0673dc54c61ef6a591ebc687057ff53b22e
Summary:
When I was debugging an eden importer issue with Puneet, we saw errors caused
by important extensions (ex. remotefilelog, lz4revlog) not being loaded. It
turned out that configpaser was checking the "exe dir" to decide where to
load "system configs". For example, If we run:
C:\open\fbsource\fbcode\scm\hg\build\pythonMSVC2015\python.exe eden_import_helper.py
The "exe dir" is "C:\open\fbsource\fbcode\scm\hg\build", and system config is
not there.
Instead of copying "mercurial.ini" to every possible "exe dir", this diff just
switches to a hard-coded system config path. It's now consistent with what we
do on POSIX systems.
The logic to copy "mercurial.ini" to "C:\open\fbsource\fbcode\scm\hg" or
"C:\tools\hg" become unnecessary and are removed.
Reviewed By: singhsrb
Differential Revision: D13542939
fbshipit-source-id: 5fb50d8e42d36ec6da28af29de89966628fe5549
Summary:
Use an `asset` to download Cython on demand. So we don't need to install Cython
as build dependency on all supported platforms, and maintain the "Cython"
package for those platforms.
Upgrade to the latest Cython by the way.
Reviewed By: singhsrb
Differential Revision: D13513514
fbshipit-source-id: 5ebe9a3e5b785a8f85cd51624663f9cc1e5c66fd
Summary:
This is intended to replace the python-lz4 library so we have a unified code
path.
However, added benchmark indicates the Rust version is significantly slower
than python-lz4:
Benchmarking (easy to compress data)...
pylz4.compress: 10964.14 MB/s
rustlz4.compress_py: 12126.00 MB/s
pylz4.decompress: 3908.29 MB/s
rustlz4.decompress_py: 798.68 MB/s
Benchmarking (hard to compress data)...
pylz4.compress: 5615.86 MB/s
rustlz4.compress_py: 740.32 MB/s
pylz4.decompress: 6145.68 MB/s
rustlz4.decompress_py: 2423.99 MB/s
The only case where the Rust version is fine is when the returned data is
small. That suggests rust-cpython was likely doing some memcpy unnecessarily.
Reviewed By: DurhamG
Differential Revision: D13516207
fbshipit-source-id: 72150b15c38bc8d8c7e7717a56a41f48d114db19
Summary: Adds the current flavor of the eden rust+python deps to LFS and have setup.py bake them into the IPython.zip.
Reviewed By: quark-zju
Differential Revision: D13516179
fbshipit-source-id: 64a0a86bc97b187f35475a3df4b580e0f0bc5deb
Summary:
This is the library shared between the eden cli and the
eden hg extension.
Reviewed By: quark-zju
Differential Revision: D13503730
fbshipit-source-id: 45ab550da3126042cb3baacaf8469b8acd6b1c4a
Summary:
makes it possible to `import thrift` from python2.
This is implemented as a subclass of `asset`, but rather than downloading
files we copy them from their location in fbsource. Extraction
copies them from their fbsource path, optionally excluding specific files
(eg: python3 only files) to the build path.
Reviewed By: quark-zju
Differential Revision: D13503711
fbshipit-source-id: fbe69400de31376ff6135c2c5173a984ff97f282
