Commit Graph

51617 Commits

Author SHA1 Message Date
Xavier Deguillard
b1f3e37e43 tests: the Rust ContentStore always creates .hg/store/packs
Summary:
The Python one only does it when needed, let's fix the 2 tests that were
relying on this.

Reviewed By: quark-zju

Differential Revision: D19393501

fbshipit-source-id: 94e004999045264adce2ffa17b4697583ee0fcbd
2020-01-14 13:46:26 -08:00
Xavier Deguillard
f1c8af3297 remotefilelog: test for phase instead of localcontentstore
Summary:
The phases of a commit is a more precise metric for whether a change is public
or not, let's use this instead of testing for data to be present in the local
contentstore.

Reviewed By: quark-zju

Differential Revision: D19393502

fbshipit-source-id: b1c735c8400e8d5a1a7f7013d9a4a1ad423c5023
2020-01-14 13:46:26 -08:00
Stefan Filip
eca4fbc75d tests: increase timeout for mononoke starting
Summary:
We started to see a lot of tests failing with mononoke timing out at startup.
Increasing the timeout to mitigate the issue.

Reviewed By: krallin

Differential Revision: D19397440

fbshipit-source-id: a8037102b3c8389b98a7c49824fa013fd3e6a080
2020-01-14 12:52:05 -08:00
Stefan Filip
a2ca6bb7dd manifest-tree: move private test function to lib.rs
Summary:
Moving `store_element` and `get_hgid` from testutil.rs to lib.rs in order to
get rid of some buck build warnings. They are only used in lib.rs.

Reviewed By: markbt

Differential Revision: D19350343

fbshipit-source-id: 081783d753425daaae6fadc38589da5e4e8b745d
2020-01-14 11:49:53 -08:00
Stefan Filip
2c9fa6054d manifest: update make_tree to be public
Summary:
It makes sense for manifest-tree to expose a testutil version for building a
TreeManifest. No reason for the testutil function to be limited to the crate.
The only issue is that `make_tree` is to generic so we rename the function to
`make_tree_manifest`.

Reviewed By: markbt

Differential Revision: D19350351

fbshipit-source-id: 06fe04d99bf820d3e81417f5a5514aa4b0d282e6
2020-01-14 11:49:53 -08:00
Stefan Filip
887b776fe1 manifest: add testutil in core package
Summary:
Types that are defined by the manifest core crate should have test utilities
defined in the same crate.

This is motivated by various warning in the buck build.

Reviewed By: markbt

Differential Revision: D19350350

fbshipit-source-id: a6a7c3fb54b465aa09a28ff8b70b49a355b328fc
2020-01-14 11:49:52 -08:00
Jun Wu
1be7e2ea4c hgmain: explicitly flush IO on exit
Summary: Without this, `hg status --print0` can output nothing and break tests.

Reviewed By: xavierd

Differential Revision: D19387317

fbshipit-source-id: 4feb82889df9bd1705526027f491db90a2a5f946
2020-01-14 09:42:48 -08:00
svcscm
c262d1d647 Updating submodules
Summary:
GitHub commits:

cce1e2b815
80362c9c63

Reviewed By: yns88

fbshipit-source-id: 0159101758d94c9c7d179279ab587ae30123c6a0
2020-01-14 09:42:48 -08:00
svcscm
24cab7c3ce Updating submodules
Summary:
GitHub commits:

2156e48924
8c5b4af317
be69716784
4f76ad1fab
0b12b2f13c
0449b53cb1
1481689822
43ffa9bbf0
787d6b6c93

Reviewed By: yns88

fbshipit-source-id: b0080fd1a4c26efbe8f26245fbba7740fbac08f3
2020-01-13 20:16:00 -08:00
Jun Wu
684351a6a9 edenfs-client: fix warnings
Summary:
Fix warnings about unused imports, and fix Cargo.toml to use well-defined
versions.

Reviewed By: singhsrb

Differential Revision: D19386232

fbshipit-source-id: 4eb7478dfb4f34e8338d5090b3ba7225bfcb008f
2020-01-13 18:14:46 -08:00
svcscm
fe9b066361 Updating submodules
Summary:
GitHub commits:

1f8321394d
024c1d0b43
1d57089fc3
3c6f1f782c
21a27b0f8e
23bb716b62
894c6d21af
e3e241d700
ac4e11d84a
c35803ad68
647388f265
50a3288630
b197f0c95a

Reviewed By: yns88

fbshipit-source-id: 1807ac876a126d221c257edbd4732f9a1240e869
2020-01-13 18:07:08 -08:00
Jun Wu
2ade5663b1 mq: remove references about mq
Summary: `mq` was removed. Remove remaining references to it.

Reviewed By: singhsrb

Differential Revision: D19382027

fbshipit-source-id: d194556b63dd6b9a44b949bd24b7e1a7356d5c04
2020-01-13 17:39:43 -08:00
Puneet Kaushik
b01d4a721d Implemented check file name functionality to workaround a Projected FS issue.
Summary:
There is a bug in Projected FS on Windows 1903 and 1909, which BSODs the system on SetInformation call. Projected FS seems to be dereferencing a nullptr while attempting to check if the file is in the backing store. Providing a separate callback to check if the file exists in the backing store will avoid the buggy code (suggested by Microsoft).

This will enable Eden FS on Windows 1903 and 1909.

Reviewed By: simpkins

Differential Revision: D19377522

fbshipit-source-id: ab5e9bc8e2472f3e689d71d05a560bac86211295
2020-01-13 16:15:09 -08:00
Puneet Kaushik
8899c53892 Update Windows EdenMount::getRootTree return a future
Summary: This is a followup on D18666585.

Reviewed By: chadaustin

Differential Revision: D19377523

fbshipit-source-id: f9ab4011d0978a6934de77d1837a37082603cf52
2020-01-13 16:15:09 -08:00
Puneet Kaushik
1350e4f8ce Fixing ObjectStore::create signature in Windows TestMount
Summary: We missed updating this in D18669664.

Reviewed By: simpkins

Differential Revision: D19377524

fbshipit-source-id: e23fb039047855454b4f12353bc375b1a2c8d962
2020-01-13 16:15:08 -08:00
Xavier Deguillard
0be67e6b08 osutil: export O_CLOEXEC
Summary: This is needed in Python3, export it.

Reviewed By: quark-zju

Differential Revision: D19379402

fbshipit-source-id: e5d095005f20c8a940b31e0296293aa8a85c4e98
2020-01-13 16:15:08 -08:00
Adam Simpkins
db881eb225 getdeps: enable the INSTALL_PYTHON_LIB option for Eden builds on Windows
Summary: Enable `INSTALL_PYTHON_LIB=ON` for internal builds of Eden on Windows.

Reviewed By: pkaush

Differential Revision: D19060630

fbshipit-source-id: 40d4f8784958fbeaad4f0d332c55fa0cb810cb5e
2020-01-13 16:10:58 -08:00
Adam Simpkins
6666995b37 add an option to also install the Python DLL on Windows
Summary:
Add an option to CMakeLists.txt so that when building on Windows the Python
DLL will also be installed into Eden's `bin/` directory.  On Windows our
Python-based tools are compiled into `.exe` files that require this DLL.
Installing the Python DLL along side our binaries is an easy way to ensure
that these executables can always find the correct library.

It would perhaps be nicer if we could simply link against Python statically.
Unfortunately the `FindPython3.cmake` file that ships with CMake silently
ignores requests to link statically on Windows.  (They provide a
`Python3_USE_STATIC_LIBS` option, but it is explicitly ignored on Windows.)
In the long run it might be nicer to attempt to fix that.

Reviewed By: pkaush

Differential Revision: D19060632

fbshipit-source-id: 6995fe6b81b820cf8e0713687751c1a811e97f88
2020-01-13 16:10:57 -08:00
Durham Goode
1c573e7569 workingcopy: add error handling and reporting to rust walker
Summary:
Previously, if the rust walker encountered any issues with individual
files it would stop and throw an exception or in some cases panic. To replace
the python walker we need to handle errors more gracefully, so let's add logic
for reporting which files had issues and what those issues were.

This also contains a minor one-line fix to prevent the walker from traversing
directories that contain a '.hg' directory.

Reviewed By: quark-zju

Differential Revision: D19376540

fbshipit-source-id: ede40f2b31aa1e856c564dd9956b9f593f21cf27
2020-01-13 15:26:30 -08:00
Jun Wu
aa64745a35 debugmetalog: add a command to show repo history
Summary:
The command shows how visibility, bookmarks, remotenames change in the repo
over time, using metalog data.

Also add this command to hg rage report.

Reviewed By: markbt

Differential Revision: D19358668

fbshipit-source-id: 9bddc039143b45c29c0c937ee60906ab93a2defe
2020-01-13 14:29:07 -08:00
Jun Wu
cfcf538578 color: define changeset.{public,draft} colors
Summary: The colors are from the our smartlog template. They're used in the next change.

Reviewed By: markbt

Differential Revision: D19358670

fbshipit-source-id: 7476f95767c5fcf1cccdce2119a2ab7ce45d270e
2020-01-13 14:29:06 -08:00
Jun Wu
3895af590a edenfs-client: use clidispatch IO and Error types
Summary:
This simplifies the error handling and makes it more compatible with things that
might capture the output, including the Python testing framework.

Reviewed By: DurhamG

Differential Revision: D19325642

fbshipit-source-id: 53de8b9a8294219e2b8e62831dce236841bd4cbb
2020-01-13 14:23:34 -08:00
Jun Wu
12eaf82291 hgcommands: add edenfs status fast path
Summary: Make `hg status` use the EdenFS Thrift path, similar to the telemetry wrapper.

Reviewed By: DurhamG

Differential Revision: D19325641

fbshipit-source-id: 14777a252d7cb433316511a2a1f1a6649e9cb020
2020-01-13 14:23:34 -08:00
Jun Wu
b9e7f6a995 cliparser: support shared flag definition
Summary:
Make it possible to define reusable flags like walkopts, templateopts that can
be shared among commands.

Reviewed By: DurhamG

Differential Revision: D19325640

fbshipit-source-id: 89a9ad0e59672b1f8cfa22daaacd3c75d8cff61a
2020-01-13 14:23:33 -08:00
Jun Wu
7e9aaf9da5 lib: add edenfs-client
Summary:
Add a crate aiming for talking to EdenFS via Thrift.

This is mostly ported from telemetry. There are lots of things that can be
cleaned up like moving the command portion to hgcommands, or using a better
abstracted rendering layer for outputting, etc. But for now, I just ported
the code as-is.

Reviewed By: DurhamG

Differential Revision: D19325639

fbshipit-source-id: ac70315ed166024857a90aab070e4d521f3e3b58
2020-01-13 14:23:33 -08:00
Jun Wu
b907d121db thrift-types: add a crate exposing EdenFS thrift interface
Summary:
The main motivation is to replace (super slow) Python Thrift serialization.
It also makes it possible to port the native "status" command from telemetry
wrapper to the main executable.

The crate is made to be built with either buck or cargo in fbcode environment:
- Cargo build uses `build.rs` to re-generate (checked-in) Thrift code.
- Buck build uses TARGETS to build Thrift code on the fly.

Reviewed By: DurhamG

Differential Revision: D19325638

fbshipit-source-id: 1d3287d8d6c674065500929827fa13ba9d448708
2020-01-13 14:23:32 -08:00
Durham Goode
e959466b51 automigrate: don't call automigrate if lock is held
Summary:
We don't want automigration to block normal execution of the command,
so let's take the lock with a short timeout, then skip automigration if we don't
get it.

This should solve an issue where debugedenimporthelpers were hanging because
they tried to take a lock that a hg process that was accessing eden held.

Reviewed By: quark-zju

Differential Revision: D19375544

fbshipit-source-id: 6c107aac87b5b839cc2d347a3b40ce67da627b71
2020-01-13 13:52:18 -08:00
svcscm
60ea593f9a Updating submodules
Summary:
GitHub commits:

b5718e35c8
e1af1b0550
8a34e7f444
e9e70ade5b
d9e693ece0
329347c63c
671b5aa064
7f3bb0bf37
6207e92b9b
d4b95d87d4

Reviewed By: yns88

fbshipit-source-id: 3c9131bdee0bf8a8ca5c679a95e8ff8a6f805762
2020-01-13 13:29:34 -08:00
Xavier Deguillard
8c1ef64565 sshpeer: read stderr in a background thread
Summary:
On Windows, select cannot be used for pipes, and thus the doublepipe class
would end up busy spinning trying to read from it. Since stderr is almost
always empty let's move it to a background thread so the main code path
doesn't have to deal with it.

This proves to be a significant win for `hg pull`. For an empty manifest
cache, `hg pull` used to take ~5 minutes, it's now around 1 minute, with
Mononoke taking 40 seconds out of that.

Reviewed By: DurhamG

Differential Revision: D19352144

fbshipit-source-id: 77112c1c18d7dab15f6946a645f73e89908802f2
2020-01-13 12:55:18 -08:00
Mark Thomas
d06144e957 renderdag: test width correctly matches rendered width
Summary:
Enhance the tests to test whether the width of the graph matches the width that
was given by the `width` method before `next_row` was called.

The width we are interested in is the number of cells the string would occupy,
not the string length, so use the `unicode-width` crate to determine this.

Reviewed By: quark-zju

Differential Revision: D19282571

fbshipit-source-id: d9852c4c9e0f76c78db047f0da5dd34723a62a2a
2020-01-13 10:22:43 -08:00
Mark Thomas
aae60162eb renderdag: switch to . for ancestor lines
Summary: Switch from `:` to `.` for the ancestor lines.  I think they look better.

Reviewed By: quark-zju

Differential Revision: D19371721

fbshipit-source-id: e18f1a62e23620a82007e2c377607a0a61623830
2020-01-13 10:22:43 -08:00
Mark Thomas
4683a72a77 cmdutil: convert to local encoding when drawing rust graph
Summary:
Use `encoding.unitolocal` to convert the Rust-generated unicode strings to the
local encoding.

Reviewed By: quark-zju

Differential Revision: D19371723

fbshipit-source-id: 6b37ef51f37f2212ca2d9ab20f9ed4da29b3d63c
2020-01-13 10:22:42 -08:00
Mark Thomas
f395961e33 renderdag: add additional glyph sets for box-drawing
Summary:
Box-drawing characters with curves aren't reliably renderable on Windows.  Add
a "square" glyph set that uses right-angle characters.  These characters are
available in cp437 and cp850, so should be available on most Windows systems.

For completeness, add a "dec" glyph set that uses DEC line drawing characters,
for use in old terminals like xterm.

Reviewed By: quark-zju

Differential Revision: D19371722

fbshipit-source-id: 35887243cceab66c702e2b5278b572f77946805f
2020-01-13 10:22:42 -08:00
Jun Wu
9f02e24674 run-tests: turn on format.use-zstore-commit-data
Summary:
This applies the new feature to all tests.

The tests do not use revlog fallback but we plan to use revlog fallback
in production for faster migration.

Reviewed By: DurhamG

Differential Revision: D18737259

fbshipit-source-id: 69633c7b2026fb3d19ca16d40368bded972efd86
2020-01-13 10:06:10 -08:00
svcscm
4bae966f47 Updating submodules
Summary:
GitHub commits:

accbe8c3f9
50135cdfdd

Reviewed By: yns88

fbshipit-source-id: e3c13fe02e19acca925cfa6a2dbd5bb6602b1c27
2020-01-13 10:06:10 -08:00
Jun Wu
138ad4961f setup: include pdb and py files on Windows package
Summary: Add hg.pdb and pure Python source code on Windows for easier debugging.

Reviewed By: markbt

Differential Revision: D19351559

fbshipit-source-id: 8515a84dc58df7c44fd324cc69b7b142fd063b8f
2020-01-13 09:43:57 -08:00
Thomas Orozco
6e266973a1 chg: don't preload "rust" extension
Summary:
D19340538 deleted `eden/scm/edenscm/mercurial/rust/__init__.py`, but
chg's `_preimportmodules` still tries to import it, which breaks Mononoke

```
   $ hg init repo-hg
+  Traceback (most recent call last):
+    File "/data/users/torozco/fbsource/fbcode/buck-out/dev/gen/eden/scm/__hg__/runtime_resources/__default__/eden/scm/hglib/edenscm/__init__.py", line 61, in run
+      dispatch.runchgserver()
+    File "/data/users/torozco/fbsource/fbcode/buck-out/dev/gen/eden/scm/__hg__/runtime_resources/__default__/eden/scm/hglib/edenscm/mercurial/dispatch.py", line 336, in runchgserver
+      _preimportmodules()
+    File "/data/users/torozco/fbsource/fbcode/buck-out/dev/gen/eden/scm/__hg__/runtime_resources/__default__/eden/scm/hglib/edenscm/mercurial/dispatch.py", line 319, in _preimportmodules
+      __import__(name)
+  ImportError: No module named rust
+  [1]
```

NOTE: I had a `__init__.pyc` left in the directory removed in D19340538, which
I'm guessing is why was still working on a `make local` build. After removing
that directory, a `make local`-built hg crashes consistently with the same
error for me.

Reviewed By: farnz

Differential Revision: D19371748

fbshipit-source-id: 4f57b5e1f75b2bd6e857303111fc01446124417b
2020-01-13 09:41:00 -08:00
svcscm
2dd031af96 Updating submodules
Summary:
GitHub commits:

70c6ce2500
712ddb5628
3aac4d791f

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 8428b6f085dfdb6f8fe3eccc139f2d2f1fa13e9d
2020-01-13 04:23:08 -08:00
svcscm
b0062b0a83 Updating submodules
Summary:
GitHub commits:

d9f0942755
4f9a6d23e8
1fb272ae4d
f7def65369
be7703e85a
af57f10ced

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 4a36353433b9753633cab1bcb9db0a7d1be19d82
2020-01-12 17:22:49 -08:00
svcscm
4b638e682a Updating submodules
Summary:
GitHub commits:

f1709fb5f4
eaf3df607f
cedcd9cb09

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 3849b33057d2700db6461be30c6b807dd2b75863
2020-01-11 22:23:42 -08:00
svcscm
c8e0efb83d Updating submodules
Summary:
GitHub commits:

087f5273b7

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 67c5f4f48622d2a4b3724736904bce37fd58e036
2020-01-10 22:10:18 -08:00
Jun Wu
674b592596 clindex: fix error type in partialmatch
Summary:
The expected error type is RevlogError. Previously that RevlogError
was converted from RuntimeError. However, D19169527 stack changed
RuntimeError to RustError, causing clindex partialmatch to raise
RustError instead, which is not handled by the Python layer.

Convert RustError to RevlogError to solve the issue.

Context:
https://fb.workplace.com/groups/scm/permalink/2591821277534143/

Reviewed By: DurhamG

Differential Revision: D19359074

fbshipit-source-id: 7b7b500cdf01272791a73ea0d94397e1397ff80b
2020-01-10 20:36:59 -08:00
Jun Wu
ffc85f7222 changelog: add a way to migrate up and down with zstore-commit-data format
Summary:
This makes it possible to use zstore instead of 00changelog.d to store commit
data. Right now it double writes to zstore and 00changelog.d if turned on.
In the future we can switch to only writing to zstore if 00changelog.i
alternative is available.

Some related configs were added for fallback strategy: revlog or server.
The revlog fallback allows us to do fast migration. The server fallback
(not implemented in this diff) allows us to stop depending on revlog.d.

Reviewed By: DurhamG

Differential Revision: D18737260

fbshipit-source-id: 3c9605f0babd8a215ee74bdf1275cc4e9dbd766c
2020-01-10 19:01:43 -08:00
Jun Wu
cbb65388aa changelog: add a way to write revlog.d to zstore
Summary:
The `00changelog.d` file contains commit data (user, date, message, but no
ordered parents).

By setting the `zstore` attribute on `changelog`, reads from `00changelog.d`
(aka. the `revision` method) are going to use `zstore` instead. Writes will
go to both `00changelog.d` and `zstore`.

Reviewed By: DurhamG, xavierd

Differential Revision: D18737258

fbshipit-source-id: fd1cf52a77fe396df66b77c4df77089326f5d4c2
2020-01-10 19:01:43 -08:00
svcscm
985e737cf1 Updating submodules
Summary:
GitHub commits:

9d5d3572ff
580df286d1
eff5e076f5
ca8733922f
b71bb506c6
7d2807a5e2
04ee36248f

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 5000cbfa99670c0bceb226bfd3a2f0badc1922f4
2020-01-10 17:41:00 -08:00
Jun Wu
6266e2cad6 tests: improve test reliability for parsing dates
Summary:
We have seen issues parsing '0' as date somehow passes on OSX, and parsing
'Apr 2018' fails on Windows with GitBash. Let's workaround the tests while
I'm going to find better fixes.

Reviewed By: xavierd

Differential Revision: D19353054

fbshipit-source-id: c0302071964191b97eea6572ffd985f831db6791
2020-01-10 15:46:17 -08:00
svcscm
9310643aae Updating submodules
Summary:
GitHub commits:

fc11ff8356
dbec7e7409
0b153ee71a
e9dbd21d73
6548a679a7
772d1e0e30

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: d59e7621ded16afb67e08f989f52833fbc094303
2020-01-10 15:46:17 -08:00
svcscm
6f08cbd328 Updating submodules
Summary:
GitHub commits:

5922872e67
eb1fd069c5
5f89ff6ed3
f62edaf034
a4e4f219e8
51471c3f8a
23960b194b
fd8915edd9

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 4de07f641c5815402fa4b23bcdb75eb9008eab55
2020-01-10 12:13:19 -08:00
Stefan Filip
98fc586755 types: add helper method for generating repo paths in testutil
Summary:
Using RepoPath::arbitrary() has a high chance or producing a lot of different
top level directories. The helper method added by this change,
`generate_repo_paths` simulates directories having various files and several
directories.

Reviewed By: quark-zju

Differential Revision: D19321366

fbshipit-source-id: 5c1aec78b6157f3cbea3d0673b29b3a676de88c0
2020-01-10 11:12:41 -08:00
Stefan Filip
403e4ae594 manifest-tree: add basic benchmark
Summary: Basic benchmark that covers insert, remove, finalize and iterations.

Reviewed By: quark-zju

Differential Revision: D19204893

fbshipit-source-id: 3aafbf6d94a558d2fd4ae59190f6046dc7c5885e
2020-01-10 11:12:41 -08:00