Summary:
The `debugfsync` command calls fsync on newly modified files in svfs.
Right now it only includes locations that we know have constant number
of files.
The fsync logic is put in a separate crate to avoid slow compiles.
Reviewed By: DurhamG
Differential Revision: D22992103
fbshipit-source-id: b5503e498d5216d4ba19701ecd5582387e4f45f5
Summary: This allows callsites to get access to the storage.
Reviewed By: DurhamG
Differential Revision: D22992104
fbshipit-source-id: c72fa313be1468170c9728d3856f822bb6385dc8
Summary:
This makes the main command table cleaner.
I dropped the `indexedlogrepair` command as it cannot rebuild indexes. `hg
doctor` is a better replacement. Some debug commands are renamed so they
no longer have `-` in the command name.
Reviewed By: DurhamG
Differential Revision: D22992107
fbshipit-source-id: f65d74e36fb971e592ad0cc8be9a94e245c39662
Summary:
If every command lives in their module, then we can define the "module" interface:
- run(...): run the command
- doc(): the help text
- name(): command name, with aliases
Then the macro would make command registration look simpler.
This diff changes `status` to use the pattern as an example.
Reviewed By: DurhamG
Differential Revision: D22992109
fbshipit-source-id: eaf589863092ec2eb1f8c24c1c7e425492fe1e3a
Summary:
As the number of commands grows, it starts making sense to move them to
individual files. Let's create a directory for them.
Reviewed By: DurhamG
Differential Revision: D22992108
fbshipit-source-id: a0556be602b832579a8e027342d5b86d9d84d257
Summary:
EdenFS on Windows is a bit weird as ProjectedFS is implemented as a filter
driver that adds reparse point to all the files/directories to get notified of
filesystem operations on them. It then hides these reparse points from the
outside which means that the dwAttributes of a file in EdenFS will not claim
that a reparse point is attached to it. On top of this, newly created
files/directories won't have any reparse points attached to them, until they
start being tracked by EdenFS.
While the first issue can be solved by always querying the reparse tags, I'm
not entirely sure how to solve the second one. That second issue causes
Mercurial to always try to create hardlink in the .hg directory, while it shouldn't.
Reviewed By: DurhamG
Differential Revision: D22937788
fbshipit-source-id: 5d90cd37d40858ed60103ff2d17c2cef16472b38
Summary:
When running `edenfsctl prefetch **/BUCK` with an empty hgcache, EdenFS ends up
asking mercurial for every manifest one by one. Unfortunately, every manifest
fetched also causes the packfile to be flushed to disk, which then leads EdenFS
to rescan the filesystem for the new packfile. Once too many packfiles are
present on disk, Mercurial triggers a repack. Effectively, that means we have a
quadratic complexity both on Mercurial, and on EdenFS's side.
While this has been a long standing issue, we've so far avoided falling into
this complexity for a number of reason. The main one being that the hgcache is
very rarely empty, and thus the quadratic complexity is usually on low number
of files. Users also rarely run a prefetch of all the files for the entire
repo. However, on repositories with long standing branches, the hgcache is
effectively cold and thus any prefetch would trigger the pathological behavior.
To solve this, we take the same approach taken for files: sending the raw
manifest to EdenFS, which will then take care of deserializing it properly.
Reviewed By: DurhamG
Differential Revision: D23035335
fbshipit-source-id: 855e6fb4fabf81c427fad6c9f17d05f95c47e9ae
Summary:
On macOS, it appears that ssh has a ~1% chance of never being able to connect
too the server and just hang. This caused mactest to be completely unhealthy
for a couple of days and a similar hotfix was applied to mitigate the issue.
Since it proved to be working, let's now backport this hotfix in the actual
code.
Reviewed By: DurhamG
Differential Revision: D22953230
fbshipit-source-id: ead7662ea6d0a33efaa5c4044c9391b2835ee421
Summary: Client portion for the commit/revlog_data endpoint that was added to the server.
Reviewed By: kulshrax
Differential Revision: D23065989
fbshipit-source-id: 3115ad2b426daca22472e2106fcd293f3ccd70f3
Summary:
Pyre now has improved support for decorators and descriptors, which makes it
possible for us to add type annotations to `dirstate.py` without needing lots
of `pyre-ignore` comments everywhere. (Previously Pyre could not handle the
`propertycache` decorator, causing it to be confused about the type of
various dirstate members, like `_map`).
Reviewed By: mrkmndz
Differential Revision: D22969757
fbshipit-source-id: 1b54f1edfb56c20c237a34f14a47404d10605240
Summary: Begin adding some initial type annotations for the Rust Python bindings.
Reviewed By: quark-zju
Differential Revision: D22993222
fbshipit-source-id: 2073db93b22f6bb04e30b767594d435c36ddb17f
Summary:
EdenFS may spawn several Mercurial process concurrently and they would all try
to take the wlock at startup time, more often than not, one of these process
would die early due to the tmplock not being present on disk. This is due to
the other Mercurial process removing it, let's have a 10s grace period where
temporary locks aren't removed to avoid this race.
Reviewed By: DurhamG
Differential Revision: D22954997
fbshipit-source-id: ce191265c03a7042d9c6e45db0dc44a688fa204c
Summary:
When doing large clones or checkouts the amount of data we add to an
indexedlog can be many GB. On a laptop we don't have much memory, so let's set a
max memory threshold for the file data/history indexedlogs.
Reviewed By: xavierd
Differential Revision: D23046489
fbshipit-source-id: 43b7686b11fe05e4c074bcb02c475ebf8cf14ab1
Summary: Dump the text into file as it is
Reviewed By: markbt
Differential Revision: D23039839
fbshipit-source-id: 966d6c5e90f020efbb8123704f5c2749596fbab5
Summary:
There are two different magic background syncing that can be enabled. The first
is triggered by commit or any other local changes. The second is triggered by
SCM Daemon by any remote change in this workspace.
I would like to explain it a bit better in `hg cloud status` command.
This will also offer some reassurance to clients.
For example, assume they run `hg cloud disable` command that should disable all background Commit Cloud traffic for some time, so then they can run `hg cloud status` and verify that neither local changes, nor remote changes trigger any commit cloud traffic on this machine.
I also provide full log path to Scm Daemon logs if it is enabled.
Reviewed By: markbt
Differential Revision: D23038954
fbshipit-source-id: c3a5b8f58df729ee3f1c7f15da44ad6e6e0b98f6
Summary:
Matches the `getcommitdata` SSH endpoint.
This is going to be used to remove the requirement that client repostories
need to have all commits locally.
Reviewed By: krallin
Differential Revision: D22979458
fbshipit-source-id: 75d7265daf4e51d3b32d76aeac12207f553f8f61
Summary:
All of these are already valid utf-8 characters, no need to dance to
decode/encode them again.
Reviewed By: DurhamG
Differential Revision: D22978828
fbshipit-source-id: c5f6e25e71cdcaa1c0558d4a1181b667ffe379fb
Summary:
When computing which heads to remove because of bookmark removal,
ignore any heads that are not in the smartlog dag.
Heads might be missing from the smartlog dag if they're not
available on the server.
Reviewed By: quark-zju
Differential Revision: D22980810
fbshipit-source-id: f002eece8567aaf57780f592aaf29a790b8314ce
Summary:
A commit doesn't show up after `hg pull -r` command if it's known locally.
This is a bug that the test demonstrates.
Reviewed By: quark-zju
Differential Revision: D22977182
fbshipit-source-id: 428094568140892fc8a13004f3395371d8b55ebf
Summary:
Introduce taggederror-util, which provides a new trait `AnyhowEdenExt`, which provides a method `eden_metadata` for anyhow errors and results. This method works much like `AnyhowExt::common_metadata`, but additionally supports extracting default error metadata from known `Tagged` types which are listed explicitly in the method implementation.
Extend `FilteredAnyhow` to support a configuration "metadata function", which allows swapping out `eden_metadata` for the standard `common_metadata`.
Modify Rust dispatch and Python bindings to use `AnyhowEdenExt` for metadata extraction and printing.
Modify `intentional_error` to rely on `AnyhowEdenExt` for tagging (removes `.tagged` call, no tags will be visible if `AnyhowEdenExt` is not used).
Reviewed By: DurhamG
Differential Revision: D22927203
fbshipit-source-id: 04b36fdfaa24af591118acb9e418d1ed7ae33f91
Summary:
Add a `buffered()` method to `AsyncResponse` allowing the user to specify the desired chunk size for the body stream.
(This was already used internally by `CborStream`; this just exposes it in the public interface.)
Reviewed By: quark-zju
Differential Revision: D22935891
fbshipit-source-id: e110e85bf9cb4c7923a8977ea4631ca1cc4cf4cb
Summary: Rename the `cbor` module to `stream` to better indicate that it contains various stream combinators (not all of which are related to CBOR).
Reviewed By: quark-zju
Differential Revision: D22935892
fbshipit-source-id: 3f73aa707ab59c31717c1cf35995ad79946a15c9
Summary: I think that the broken config (wrong relative base for search_path), was what prevented the upgrade from going automatically.
Reviewed By: grievejia
Differential Revision: D22966243
fbshipit-source-id: 4ef42a8e2e6f2c79483301c6876509a3009a83d1
Summary:
This provides Ctrl+C/SIGKILL safety. It's needed because we no longer use the
Python transaction framework. If the write is incomplete, the revlog index
logical length will ensure new processes won't see incomplete data.
The length of revlog data is not tracked, as some "unused" in it does not
really matter. Reading the revlog should be still fine.
Reviewed By: sfilipco
Differential Revision: D22914423
fbshipit-source-id: f2f446cde79c7270cbd1ef165f8707368a0a2990
Summary:
This is more complex than previous libraries, mainly because `dag` defines APIs
(traits) used by other code, which might raise error type not interested
by `dag` itself. `BackendError::Other(anyhow::Error)` is currently used to
capture types that do not fit in `dag`'s predefined error types.
Reviewed By: sfilipco
Differential Revision: D22883865
fbshipit-source-id: 3699e14775f335620eec28faa9a05c3cc750e1d1
Summary:
Prefix some `Result` with `dag::Result`. Since `dag::Result` is just
`anyhow::Result` for now, this does not change anything but makes
it more compatible with upcoming changes.
Reviewed By: sfilipco
Differential Revision: D22883864
fbshipit-source-id: 95a26897ed026f1bb8000b7caddeb461dcaad0e7
Summary:
All dependencies of revlogindex have migrated to concreted error types.
Let's migrate revlogindex itself. This allows compile-time type checks
and makes the error returned by revlogindex APIs more predictable.
Reviewed By: sfilipco
Differential Revision: D22857554
fbshipit-source-id: 7d32599508ad682c6e9c827d4599e6ed0769899c
Summary: The `radixbuf` crate already has its own concrete error type. Use it.
Reviewed By: sfilipco
Differential Revision: D22855450
fbshipit-source-id: 307a46ddd79b28a18ee779867ee1e604b531828a
Summary:
`util` as a low-level library should use concrete error types so callsite can
type check their type conversions.
Reviewed By: sfilipco
Differential Revision: D22855448
fbshipit-source-id: 37b3fce36f1ae82a9604ef8ac0dc22c02280ceb2
Summary:
Change the `lz4-pyframe` library to use a concrete error type instead of trait
object. This would allow callsites to type check the error type.
Reviewed By: sfilipco
Differential Revision: D22855449
fbshipit-source-id: 3497b3e0bfb814302fee2f7297b35de8b8a916ed
Summary: This is unused, no need to add to the build time.
Reviewed By: fanzeyi
Differential Revision: D22967814
fbshipit-source-id: 91a5ed9f03128947af9cb69bca62ed75b75e7e66
Summary:
One of the example is to delete a workspace that doesn't exist.
For service error remove host/port info.
These verbose details are not important for users.
Reviewed By: markbt
Differential Revision: D22976512
fbshipit-source-id: b8437f5b3c0e21e23183270d07ea158404598810
Summary:
We have reports from users about errors but the errors don't have stack
traces uploaded to scuba because they are considered handled. Let's just upload
traces for every single exception that propagates up to the dispatch layer.
Reviewed By: quark-zju
Differential Revision: D22938883
fbshipit-source-id: 525b6f13422ee2aa79de3beb48e58e13405ed199
Summary:
Eden runs without a sparse checkout, so "hg prefetch" can be a very expensive
operation on eden. Let's not allow "hg prefetch" on eden if pats are not
specified - this will force users to specify what exactly they want to
prefetch. Obviously they can still request everything, but this diff will make
it harder to do so.
Reviewed By: DurhamG
Differential Revision: D22946092
fbshipit-source-id: 895505bb90980a74b31ded4a75d102c527801652
Summary:
We were experiencing hangs during lfs http fetches. We've seen similar
issues before when using Hyper, which Reqwest is based off of. Let's switch to
the new Curl-based http_client instead.
Note, this does get rid of the ability to obey hg config http proxy settings.
That said, they shouldn't be used much, and the http proxy environment variables
are respected by libcurl.
Reviewed By: xavierd
Differential Revision: D22935348
fbshipit-source-id: 1c61c04bbb4043e3bde592251f12bf846ab3afd4
Summary:
A future diff does LFS fetches via http_client. Curl has some default
behavior of adding the "Expect: 100-continue" header which causes the server to
send a 100 status code response after the headers have been received but before
the the payload has been received. Since the http_client model only expects a
single response, this breaks the model and we're unable to read the second
response. Let's disable this behavior by manually setting the header to empty
string, which appears to be the official way to handle this.
Add it early so callers can overwrite it.
Reviewed By: quark-zju
Differential Revision: D22935349
fbshipit-source-id: 3009a5eb72f40584b846510f34f121e0e821a2bc
Summary:
hg rage had a couple string issues with Python 3 and subprocess output.
This fixes them.
Reviewed By: quark-zju
Differential Revision: D22937629
fbshipit-source-id: 2b90ada536f152afb2d2662eb7f8e12d787f9544
Summary:
In python 3, curses.unctrl(...) expects a single character string. If
we pass a longer string it will throw an exception. Let's handle that.
Reviewed By: singhsrb
Differential Revision: D22937223
fbshipit-source-id: 00d1a38e48ee7d6bc0aa43eb771619027dc3d802
Summary:
On Windows, this environment variable is required to be set for less to
properly recognize that the input encoding is utf-8.
Reviewed By: kulshrax
Differential Revision: D22925498
fbshipit-source-id: 7969441792e0c1d9e2b93a690f658431f0acf59c
Summary:
One repack code path would return an error if the pack was already
deleted before the repack started. This is a fine situation, so let's just eat
the error and repack what can be repacked.
Reviewed By: xavierd
Differential Revision: D22873219
fbshipit-source-id: c716a5f0cd6106fd3464702753fb79df0bc7d13f
Summary:
Eden api endpoint for segmented changelog. It translates a path in the
graph to the hash corresponding to that commit that the path lands on.
It is expected that paths point to unique commits.
This change looks to go through the plumbing of getting the request from
the edenapi side through mononoke internals and to the segmented changelog
crate. The request used is an example. Follow up changes will look more at
what shape the request and reponse should have.
Reviewed By: kulshrax
Differential Revision: D22702016
fbshipit-source-id: 9615a0571f31a8819acd2b4dc548f49e36f44ab2
Summary: The original function is changing behavior on non-UNIX platforms. Let's honor it in tests too.
Reviewed By: wez
Differential Revision: D22901850
fbshipit-source-id: 7c4e7dbfa0526d482f86794758e7ad39feacce10
Summary:
Add a way to actually try out segmented changelog by setting
`format.use-segmented-changelog`.
The main compatibility issues are assumptions that `0 .. len(repo)` are valid
revision numbers. Segmented changelog uses 2 spans `0 .. len(master_group)`
and `<a large 64-bit int> ..` for the non-master group. Some assumptions
about `len(repo)` were removed.
Segmented changelog also does not support linkrevs since revs can be
re-assigned. Some logic (ex. in treemanifest) that relies on linkrev being
`len(repo)` is changed.
There might be other issues, which will be fixed once we discover them.
Reviewed By: sfilipco
Differential Revision: D22657187
fbshipit-source-id: 688ad718741de0ca15e6aeaf84aced24a20c6b09
Summary:
For example, `master~1000000` should not trigger auto pull of the name
`1000000`. `x~y` is parsed as `(ancestor x y)`.
Reviewed By: singhsrb
Differential Revision: D22905438
fbshipit-source-id: 757ae8856f28126bc6e988d9989a894f83d83eb4
