Summary:
Introduce taggederror-util, which provides a new trait `AnyhowEdenExt`, which provides a method `eden_metadata` for anyhow errors and results. This method works much like `AnyhowExt::common_metadata`, but additionally supports extracting default error metadata from known `Tagged` types which are listed explicitly in the method implementation.
Extend `FilteredAnyhow` to support a configuration "metadata function", which allows swapping out `eden_metadata` for the standard `common_metadata`.
Modify Rust dispatch and Python bindings to use `AnyhowEdenExt` for metadata extraction and printing.
Modify `intentional_error` to rely on `AnyhowEdenExt` for tagging (removes `.tagged` call, no tags will be visible if `AnyhowEdenExt` is not used).
Reviewed By: DurhamG
Differential Revision: D22927203
fbshipit-source-id: 04b36fdfaa24af591118acb9e418d1ed7ae33f91
Summary:
Add a `buffered()` method to `AsyncResponse` allowing the user to specify the desired chunk size for the body stream.
(This was already used internally by `CborStream`; this just exposes it in the public interface.)
Reviewed By: quark-zju
Differential Revision: D22935891
fbshipit-source-id: e110e85bf9cb4c7923a8977ea4631ca1cc4cf4cb
Summary: Rename the `cbor` module to `stream` to better indicate that it contains various stream combinators (not all of which are related to CBOR).
Reviewed By: quark-zju
Differential Revision: D22935892
fbshipit-source-id: 3f73aa707ab59c31717c1cf35995ad79946a15c9
Summary:
Previously `eden prefetch` had two subcommands `record-profile` and `finish-profile`, but then when we only want to use `eden prefetch PATTERN`, an error shows up saying that the COMMAND argument is missed.
Since `eden prefetch` has many of its own arguments, and we don't want to use it with `record-profile` and `finish-profile` all the time, we remove those subcommands and create a new `eden debug prefetch_profile` command to get prefetch profiles.
Reviewed By: fanzeyi
Differential Revision: D22959981
fbshipit-source-id: 21b278555fcb56580a62f66a7384b1cff54ba398
Summary:
Redirecting stdout means that ninja/cmake won't act as if it's invoked
interactively, ie: it will buffer the output, show every single files being
compiled (instead of just a line or progress), etc. This results in a fairly
janky UX when getdeps is used on the command line. By not redirecting stdout,
we get immediate feedback about the tests being run, and the files being
compiled.
Reviewed By: wez
Differential Revision: D22967815
fbshipit-source-id: 872ddbf421065686c384a3a876d0acb8832baf2e
Summary: I think that the broken config (wrong relative base for search_path), was what prevented the upgrade from going automatically.
Reviewed By: grievejia
Differential Revision: D22966243
fbshipit-source-id: 4ef42a8e2e6f2c79483301c6876509a3009a83d1
Summary:
On Windows, we were just spawning EdenFS without waiting for it to become
either healthy, or unhealthy. While in most cases EdenFS becomes healthy
shortly after, scripts could race with it and behave weirdly as a consequence.
For instance, `eden clone` would start EdenFS if it isn't started already, and
then immediately clone the repo, that second step may fail and just leave an
empty folder on disk.
On unices, due to EdenFS daemonizing itself, edenfsctl waits for the parent
process to exit which signify that EdenFS is either started, or dead. On
Windows, we can wait on the pid that CreateProcess returns. One drawback is
that the user won't see what `edenfsctl start` is doing while on unices we have
progress bars regarding the repositories being mounted. One approach to
alleviate this would be to use a pipe as the StartupLogger and close it once
EdenFS is initialized.
Reviewed By: fanzeyi
Differential Revision: D22964058
fbshipit-source-id: 4e83c265d22e7e5150ed8b40e2b554cfcd181f8e
Summary:
This provides Ctrl+C/SIGKILL safety. It's needed because we no longer use the
Python transaction framework. If the write is incomplete, the revlog index
logical length will ensure new processes won't see incomplete data.
The length of revlog data is not tracked, as some "unused" in it does not
really matter. Reading the revlog should be still fine.
Reviewed By: sfilipco
Differential Revision: D22914423
fbshipit-source-id: f2f446cde79c7270cbd1ef165f8707368a0a2990
Summary:
This is more complex than previous libraries, mainly because `dag` defines APIs
(traits) used by other code, which might raise error type not interested
by `dag` itself. `BackendError::Other(anyhow::Error)` is currently used to
capture types that do not fit in `dag`'s predefined error types.
Reviewed By: sfilipco
Differential Revision: D22883865
fbshipit-source-id: 3699e14775f335620eec28faa9a05c3cc750e1d1
Summary:
Prefix some `Result` with `dag::Result`. Since `dag::Result` is just
`anyhow::Result` for now, this does not change anything but makes
it more compatible with upcoming changes.
Reviewed By: sfilipco
Differential Revision: D22883864
fbshipit-source-id: 95a26897ed026f1bb8000b7caddeb461dcaad0e7
Summary:
All dependencies of revlogindex have migrated to concreted error types.
Let's migrate revlogindex itself. This allows compile-time type checks
and makes the error returned by revlogindex APIs more predictable.
Reviewed By: sfilipco
Differential Revision: D22857554
fbshipit-source-id: 7d32599508ad682c6e9c827d4599e6ed0769899c
Summary: The `radixbuf` crate already has its own concrete error type. Use it.
Reviewed By: sfilipco
Differential Revision: D22855450
fbshipit-source-id: 307a46ddd79b28a18ee779867ee1e604b531828a
Summary:
`util` as a low-level library should use concrete error types so callsite can
type check their type conversions.
Reviewed By: sfilipco
Differential Revision: D22855448
fbshipit-source-id: 37b3fce36f1ae82a9604ef8ac0dc22c02280ceb2
Summary:
Change the `lz4-pyframe` library to use a concrete error type instead of trait
object. This would allow callsites to type check the error type.
Reviewed By: sfilipco
Differential Revision: D22855449
fbshipit-source-id: 3497b3e0bfb814302fee2f7297b35de8b8a916ed
Summary:
To allow EdenFS to get aux manifest data from Mononoke without needing to derive fsnodes, provide
a mechanism to list a manifest using the hg manifest id that returns the size and content hashes
of each of the files.
NOTE: this is temporary until the EdenAPI server is fully online and serving this data.
Reviewed By: krallin
Differential Revision: D22975967
fbshipit-source-id: 0a25da6d74534d42fc3b5f38ba3b72107b209681
Summary: It served no purpose. Also as a bonus, remove an unecessary std::endl.
Reviewed By: fanzeyi
Differential Revision: D22954057
fbshipit-source-id: bcaca833cdef8b643b16fbda2a0d576b655c2857
Summary: This is unused, no need to add to the build time.
Reviewed By: fanzeyi
Differential Revision: D22967814
fbshipit-source-id: 91a5ed9f03128947af9cb69bca62ed75b75e7e66
Summary:
One of the example is to delete a workspace that doesn't exist.
For service error remove host/port info.
These verbose details are not important for users.
Reviewed By: markbt
Differential Revision: D22976512
fbshipit-source-id: b8437f5b3c0e21e23183270d07ea158404598810
Summary: Previously it was opened twice, even though there were no reason to do it.
Reviewed By: krallin
Differential Revision: D22976149
fbshipit-source-id: 426858da4548f1eaffe1d989e5424937af2583a5
Summary:
Factor out the walkers state internals to BuildStateHasher and StateMap
This change keeps the defaults the same using DashMap and ahash::RandomState and uses the same ahash version that DashMap defaults to internally.
This is in preparation for the next diff the where the ahash dependency is updated to 0.4.4. Though it was clearer not to combine the refactoring and the update of the hasher used in the same diff.
Reviewed By: ikostia
Differential Revision: D22851585
fbshipit-source-id: 84fa0dc73ff9d32f88ad390243903812a4a48406
Summary:
Only emit NodeData from walker if required to save some memory. Each of the walks can now specify which NodeData it is interested in observing in the output stream.
We still need to emit Some as part of the Option<NodeData> in the output stream as it is used in things like the final count of loaded objects. Rather than stream over Option<Option<NodeData>> we instead add a NodeData::NotRequired variant
Reviewed By: markbt
Differential Revision: D22849831
fbshipit-source-id: ef212103ac2deb9d66b017b8febe233eb53c9ed3
Summary:
Extract verify_working_copy_inner function, which lets directly specify
source/target repo, hash and movers. It can be useful to verify equivalence of
two commits even if they are not in commit equivalence mapping.
Reviewed By: krallin
Differential Revision: D22950840
fbshipit-source-id: ab30be7190e29db3343b846b48333d7c7339d043
Summary:
We have reports from users about errors but the errors don't have stack
traces uploaded to scuba because they are considered handled. Let's just upload
traces for every single exception that propagates up to the dispatch layer.
Reviewed By: quark-zju
Differential Revision: D22938883
fbshipit-source-id: 525b6f13422ee2aa79de3beb48e58e13405ed199
Summary: fix the problem of printing 0% fsck progress bars when fsck is actually not happening.
Reviewed By: genevievehelsel
Differential Revision: D22927401
fbshipit-source-id: 868fa188e3c2671b0f6e18e842ec6a23281dc337
Summary: Move it from `'static` BoxFutures to async_trait and lifetimes
Reviewed By: markbt
Differential Revision: D22927171
fbshipit-source-id: 637a983fa6fa91d4cd1e73d822340cb08647c57d
Summary:
Eden runs without a sparse checkout, so "hg prefetch" can be a very expensive
operation on eden. Let's not allow "hg prefetch" on eden if pats are not
specified - this will force users to specify what exactly they want to
prefetch. Obviously they can still request everything, but this diff will make
it harder to do so.
Reviewed By: DurhamG
Differential Revision: D22946092
fbshipit-source-id: 895505bb90980a74b31ded4a75d102c527801652
Summary:
For EdenFS Redirection on Windows, we can simply use symlinks in place of bind mounts on other platforms. This works fine from what I can tell.
NOTE: at this commit EdenFS on Windows is still not able to create these redirections at start up time since that code is located in `EdenMount` and it uses `folly::subprocess`: https://www.internalfb.com/intern/diffusion/FBS/browse/master/fbcode/eden/fs/inodes/EdenMount.cpp. For the same reason we can't enable the EdenFS redirection integration tests.
Reviewed By: xavierd
Differential Revision: D22544200
fbshipit-source-id: b1a33da128c932544660c3e895583574d7891be9
Summary:
The mix use of `os.path` and `pathlib` in `redirect.py` is a little messy, which is making adding Windows support trickier since `os.path` functions do not accept `Path` until 3.8. So before I make the change I think it's better to clean it up.
Since we are target Python 3 nowadays, replacing `os.path` with `pathlib` seems to be better. Basically this diff does the following:
* Replace use of `os.path` functions with the counterpart in `pathlib`.
* Reduce unnecessary conversions from/to `Path` to `str` / `bytes`.
* Only convert `Path` to `str` or `bytes` when interactive with other APIs (Thrift or os)
* Cross-platform APIs: `os.fspath`
* API expecting `str`: `os.fsdecode`
* API expecting `bytes`: `os.fsencode`
Reviewed By: chadaustin
Differential Revision: D22879004
fbshipit-source-id: a247973dc9919c8805daa4046472124310725516
Summary:
This is a backout of D22912569 (34760b5164), which is breaking opt-clang-thinlto builds on platform007 (S206790).
Original commit changeset: 5ffdc48adb1f
Reviewed By: aaronabramov
Differential Revision: D22956288
fbshipit-source-id: 45940c288d6f10dfe5457d295c405b84314e6b21
Summary: Often users run into kerberos certificate issues, which in turn causes permissions issues with mercurial and eden. This issue is not obvious while users are running hg commands, so `eden doctor` should identity this issue and tell the user about it. `hg doctor` runs `eden doctor`, so this should also show itself if a user runs `hg doctor`.
Reviewed By: wez
Differential Revision: D22825882
fbshipit-source-id: 5f51901934862336b0ebc2996da6e1168ea8d8a3
Summary:
Added more logs when running the binary to be able to track the progress more easily.
Saved bonsai hashes into a file. In case we fail at deriving data types, we can still try to derive them manually with the saves hashes and avoid running the whole tool again.
Reviewed By: StanislavGlebik
Differential Revision: D22943309
fbshipit-source-id: e03a74207d76823f6a2a3d92a1e31929a39f39a5
Summary:
Large commits and many hooks can mean checking 100 commits at a time overload
the system. Reduce the default concurrency to something more reasonable.
While we're here, lets use the proper mechanism for default values in clap.
Reviewed By: ikostia
Differential Revision: D22945597
fbshipit-source-id: 0f0a086c3b74bec614ada44a66409c8d2b91fe69
Summary:
Argument names should be `snake_case`. Long options should be `--kebab-case`.
Retain the old long options as aliases for compatibility.
Reviewed By: HarveyHunt
Differential Revision: D22945600
fbshipit-source-id: a290b3dc4d9908eb61b2f597f101b4abaf3a1c13
