Summary:
When running `eden info`, users probably want the commit hash WITHOUT filter information attached to it. This diff removes filter information (and therefore fixes the `eden info` test).
I can consider adding a "Filter" field to Eden info. However, this is low priority right now since I want to get other tests fixed first.
Reviewed By: kmancini
Differential Revision: D51926557
fbshipit-source-id: 98480a252816384681008e9059b479e48639f9a2
Summary:
When using FilteredBackingStores, Eden now stores FilteredRootIds in the Snapshot file. FilteredRootIds consist of the original underlying BackingStore's RootId, the length of the FilteredRootId (in varint format), and the filter that was applied to the repo during the last checkout.
To properly parse out the original underlying RootId (in most cases a Mercurial commit hash), we need to parse the varint that describes the length of the underlying RootId and parse the Snapshot file accordingly.
This diff introduces the utils that will be used to do that varint parsing logic.
Reviewed By: kmancini
Differential Revision: D51926558
fbshipit-source-id: f35ba4af4cf59e593fe86b4411d7f94cd722946d
Summary: Readdir tests previously failed because the render method tried to pass a FilteredObjectID into the underlying BackingStore's renderObjectId() method. This diff parses the underlying ObjectID out and passes that into the underlying BackingStore instead of passing in an uninterpretable FilteredObjectId.
Reviewed By: kmancini
Differential Revision: D51829836
fbshipit-source-id: 865cbf8f5e5f73bce77fd8ab32a315c307a741e9
Summary:
If a new repo was to be cloned with symlinks disabled on Windows, this would make the repo have a wrong config for symlinks.
In order for symlinks to be enabled for a working copy, they have to be enabled both in the EdenFS config for that checkout as well as having them inabled in hg by having `windowssymlinks` in `.hg/requires`.
Having symlinks enabled **in only one of the two** can cause really wonky behavior.
Some users were reporting having issues with symlinks being partially enabled on their working copies. This was caused by the backing repo having `windowssymlinks` enabled on the backing repo, since the requires is copied from there into the newly cloned repo.
This diff makes changes to that, preventing copying the requires file into a new one for both symlinks and FilteredFS, since both can be affected by this kind of behavior.
Reviewed By: MichaelCuevas
Differential Revision: D52311410
fbshipit-source-id: bca76757f1b9b6f2cafd4e024b8eafc0735e7e89
Summary:
Generated changes from {D52231478}.
NOTE: This needs to be folded into the MSDK-built autocargo binary diff. https://www.internalfb.com/intern/msdk/bump/?schedule_fbid=342556550408072
Reviewed By: zertosh
Differential Revision: D52231497
fbshipit-source-id: baadabc9dfabeb2ab0232568c2e4bf4df55e26eb
Summary:
This diff adds oncall to Buck TARGETS files in fbcode repository based on their contbuild information.
This diff was generated by this command:
./fbcode/ownership/coverage/submit_diff_with_file_oncall_mapping.sh /data/users/bayarmunkh/target_oncalls_26
see more context here: https://fb.workplace.com/groups/fbcode/posts/6551897871513663 and https://fb.workplace.com/groups/fbcode.devx/permalink/3149549092006627/
drop_conflicts
Reviewed By: markisaa
Differential Revision: D52166498
fbshipit-source-id: 5bf25a26cbd700b32bbb585d987d21137c8fc545
Summary:
`None` means "remove this dep from `Cargo.toml`". These deps are either
not autocargo managed or not actually used, so there is nothing to
remove.
Reviewed By: dtolnay
Differential Revision: D52216173
fbshipit-source-id: e94473d247f19b01be9099a50d97712144b40299
Summary:
These were missed before because they're listed in exclusions or are in
non-standard locations (e.g. `xplat/`).
Reviewed By: dtolnay
Differential Revision: D52217525
fbshipit-source-id: 0bd5a9832ead34869c46b7396521ecb364787001
Summary: There are some users reporting issues with symlinks again, so let's disable them for now. This diff essentially undoes the changes done on D51284233 without actually reverting most of the code in case we want to permanently disable them again
Reviewed By: jdelliot
Differential Revision: D52211990
fbshipit-source-id: 50f541ee984793c13c45b093377a4caeb11b7fc8
Summary:
It's a pain to add and remove extensions from our warn and blocklists in
eden doctor. It it takes a while for things to rollout since the change
is dependent on the eden package which only rolls out once a week.
Let's move these to eden configs so that we can change them with chef and
the new configerator configs.
Reviewed By: genevievehelsel
Differential Revision: D52140236
fbshipit-source-id: 737595f4fdc964b334cc3479a3a4a6369251306d
Summary: HgImporter was the only way to get the repo name. With the removal of it, we a new way. Adding that to SaplingNativeBackingStore.
Reviewed By: xavierd
Differential Revision: D51911195
fbshipit-source-id: 7fcb61d72219d4e5851b31507c53c62f68df8476
Summary: The rollout of this feature has completed, but users need to reclone. Let's update the users checkout config to set the inode catalog type to be inmemory on Windows.
Reviewed By: kmancini
Differential Revision: D52005759
fbshipit-source-id: 0e3c6948fb57496bae87e71eb25e5eb1c88979f4
Summary: This is the Sapling form of EDENSCM_LOG.
Reviewed By: kmancini
Differential Revision: D52001131
fbshipit-source-id: 5932f96d0647e064cffa6279adebbf1c16a521d6
Summary:
`-Wextra-semi` or `-Wextra-semi-stmt`
If the code compiles, this is safe to land.
Reviewed By: palmje
Differential Revision: D51995015
fbshipit-source-id: c92e3c06ae30b98afdcb204c52db8ea3a4e28729
Summary: We have counters for retry success, but not for failure. Now that failure will not always result in a fallback to HgImporter we should add them.
Reviewed By: genevievehelsel
Differential Revision: D51794512
fbshipit-source-id: 108463e55c4feb20c19f7ab602365c089930601f
Summary:
To get a full picture of the kinds of misses we have at each step adding in the backingstore misses on retry.
NOTE: we could go in a refactor HgImporter to return folly::Try<[Object]Ptr>, clean up the code a bit and unify the retry reporting here, but we are planning to remove that code soon.
Reviewed By: genevievehelsel
Differential Revision: D51784901
fbshipit-source-id: b0cb876c892ba2de077d38df49e73fa4785069b1
Summary:
The test for configuration falling back to HgImporter was placed earlier in the flow preventing using this test after falling back to backingstore. This change moves that test to between the fallback to backingstore and the actual fallback.
This is done so that we can add a GK (D51784706) to control rolling out configuration changes to stop the fallback path - the final step before removing it completly.
To ensure we only apply this new behavior to the versions of the code that include these changes, we are creating a new config setting (hgImporterFetchFallback) to replace the previous, similar settings. This will ensure that during rollout previous versions of EdenFS are not affected.
Reviewed By: genevievehelsel
Differential Revision: D51784900
fbshipit-source-id: fd3e6a160ceaaf2a125812bb42f2382324239e83
Summary: D51538112 increased the stack size on Windows for the `eden/fs/edenfs-oss` target, which is only actually used for development. We should have done the same for the non-oss target, which this diff does.
Reviewed By: jdelliot
Differential Revision: D51929793
fbshipit-source-id: 3b749584718ac3211e9a2b2e9e3a564a249575d1
Summary:
Sometimes when there are two EdenFS's attempting to start at the same time
we run into flock error. It's possible we need to take the lock earlier to
avoid two eden's from getting this far in the process. Seems like this
locking code throws instead of returning false, so maybe this error should just
be swallowed and we fall back to the "another eden is running code path".
While we sort that out let's at least give the user a more informative error
message.
Reviewed By: genevievehelsel
Differential Revision: D51903606
fbshipit-source-id: 062921a6899a9e62b7770a199d329bf233f1168e
Summary: I am planning on moving these into `edenfs.rc`, so in order to avoid the confusing parse error startup message, this adds them.
Reviewed By: jdelliot
Differential Revision: D51829423
fbshipit-source-id: 7d8a45b33aaeed92a6f81c1b8819985d9ff5d992
Summary:
SaplingNativeBackingStore "eats" errors for single object fetches. This prevents certain kinds of logging to scuba and other actions that might be taken by higher layers of code.
Eventually, we wil changes these APIs to return futures, but in the meantime, we will switch them to return folly::Try's.
The propagation goes from SaplingNativeBackingStore to HgDatapackStore to HgBackingStore and all of their callers.
Reviewed By: kmancini
Differential Revision: D51777559
fbshipit-source-id: 177e97fc12a5438abe0ba83f97aa3b1367074b3a
Summary:
Our object fetch miss reporting done via two different events: EdenApiMiss and HgImportFailure. EdenApiMiss is misleading (no pun intented) as it represents backingstore misses (local and remote). Neither of these events capture all that we want and EdenApiMiss had some wonky names.
This change fixes that, replacing both with ObjectFetchMiss. Additonally, it adds miss events for BlobMetadata.
Up next: we need to expose the failure causes from single object fetches through HgDatapackStore->SaplingNativeStore. Today they are eaten and only a pointer is returned.
Reviewed By: genevievehelsel
Differential Revision: D51733914
fbshipit-source-id: c592520da56a58a1d3e5a6198695857fe6c19451
Summary:
We were reporting EdenApiMiss events if a) there was an error and b) we had set the hg<Tree|Blob>FetchFallback flag.
We should have been reporting the even if there was an error and falling back if both conditions are true.
Reviewed By: genevievehelsel
Differential Revision: D51730692
fbshipit-source-id: e6471a706d4263321bfb4a0c56c97224c4365d5c
Summary: Users may still want to run debug commands that involve sparse profiles on EdenFS repos, so we cannot really disable sparse profiles everywhere. This diff re-enables them for all non-filteredhg EdenFS repos.
Reviewed By: MichaelCuevas
Differential Revision: D51772393
fbshipit-source-id: 30052aebcf0f16e90de50ee5a2dccc0e380e8245
Summary: I'm not sure this matters or not, but to clean up `zbgs`, replace `experimental_systemd` with a different bool config
Reviewed By: kmancini
Differential Revision: D51728414
fbshipit-source-id: 3b6c6756e84ccee16e7c8222f46f06c816616197
Summary: Changes the config source to the configerator made before. It'll be easier to maintain and change
Reviewed By: genevievehelsel
Differential Revision: D51720288
fbshipit-source-id: 7854f324c694f7850454fafddb2eb403b1c22d98
Summary:
Release notes: https://blog.rust-lang.org/2023/10/05/Rust-1.73.0.html
This release is coupled with an update of the `anyhow` and `thiserror` crates because the unstable standard library API for backtraces has changed.
Fbcode changes:
- `feature(default_free_fn)` deleted (D50300881)
- `noop_method_call` lint becomes warn-by-default (D50486032, D50516201)
- stronger `invalid_reference_casting` detection (D50488164)
- `feature(unix_chown)` has been stabilized
- `feature(provide_any)` and `std::provider` deleted
- `clippy::unwrap_or_else_default` renamed to `clippy::unwrap_or_default`
- type inference ambiguities (D51780425)
- `nu-command` build error (D51779062)
Reviewed By: AndreasBackx, shayne-fletcher
Differential Revision: D50294321
fbshipit-source-id: 0fac87f6ba072ad029f9ce41ce94ed813e855b20
Summary:
`-Wextra-semi` or `-Wextra-semi-stmt`
If the code compiles, this is safe to land.
Reviewed By: dmm-fb
Differential Revision: D51777971
fbshipit-source-id: 773e23e38af97640a244306fa4205d0f2c9e40c5
Summary: Having this extension enabled by default breaks a bunch of tests and leads to wonky behavior on non-EdenFS repos, so let's only enable it on EdenFS repos
Reviewed By: zzl0
Differential Revision: D51760161
fbshipit-source-id: 09b84d2304df88b44962c1eb9fa39c9cc3a141d6
Summary:
Eden rage pastes are craeted with eden configs as ODs run hg rage --preview.
This was caused by eden rage creating pastes for eden configs without checking the dry run path
This diff is not creating pastes anymore .
Reviewed By: genevievehelsel
Differential Revision: D51682394
fbshipit-source-id: 97de6a0874c9c8a6b9be85ccb44605d9099e075f
Summary:
In order to remove all use of unsafe inline futures from EdenFS, we also need
to remove all uses of `folly::collectUnsafe`, this one is already used as a
`folly::SemiFuture` higher in the stack so it should be safe to use
`folly::collect` instead.
Reviewed By: chadaustin
Differential Revision: D51393144
fbshipit-source-id: 16213d5c86fc9092782da71db462983b924b9ca1
Summary:
Most of it was replaced with a ManualExecutor. Some care had to be taken around
the use of StoredObject as triggering an error on them when no future was
obtained from it leads to the error not being triggered when the future is then
retrieved from it.
The only place where QueuedImmediateExecutor is still present is in the
ImmediateFutureTest to demonstrate the lack of safety with it.
Reviewed By: genevievehelsel
Differential Revision: D51302393
fbshipit-source-id: fb66a9426e0ae7672799bd997898e8b525ff112c
Summary:
This removes the last unsafe QueuedImmediateExecutor from the code (excluding
tests).
In the tests, a global CPU executor had to be used instead of a ManualExecutor
due to the FuseChannel code relying on threads to read from the FUSE
filedescriptor, the timing of which would make tests flaky.
Reviewed By: jdelliot
Differential Revision: D51302394
fbshipit-source-id: f520bf4d50ce8228a2470e21b74b4a713ab7e710
Summary:
This diff adds the ability to clone a "FILTEREDHG" repository that is backed by a FilteredBackingStore that wraps a normal HgQueuedBackingStore.
To enable this, we need a few additions:
1) A new BackingStoreType, FILTEREDHG.
2) Support for creating/registering a new mount that uses the FilteredHg BackingStoreType
3) Logic in the Rust and Python CLI to recognize "filteredhg" as a valid scm/repo type.
4) Modifications to `eden clone` logic to allow cloning a repo that uses the new BackingStoreType
Reviewed By: xavierd
Differential Revision: D50754003
fbshipit-source-id: b58944d0868da8857f324544641ba28af0044565
Summary: This makes the edenfs_ffi OSS target actually work. This is needed because our integration tests use the OSS version of Eden, and we need to add FilteredFS integration tests before rolling out to users.
Reviewed By: jdelliot
Differential Revision: D51512912
fbshipit-source-id: 01e41aaa53221190588e68fe0b17d10db563db34
Summary:
Backingstore can return misses under certain conditions - where new objects have not yet been landed, but are commited; a rebase can race and the fetch of the new objects miss.
Refreshing the backing store (forecing a rescan of pack files and or local indexes resolves this.
Reviewed By: xavierd
Differential Revision: D51695578
fbshipit-source-id: fce2fe691b147d4cd5ea5b949a24d405386fc962
Summary:
FWICT, co_download* method return DigestStatusTracker only when the method doesn't throw an exception. And this method throws an exception whenever status of any digest is not OK (i.e. we failed to download it). Ergo, the only result we can get in digest status tracker is OK. Ergo, we don't really even need to return DigestStatusTracker in the first place.
This is what this diff does - removes DigestStatusTracker from DownloadResult.
Differential Revision: D51558546
fbshipit-source-id: 1cc2016fb56d49571c65021dbb2f8a7658c0b475
Summary:
When testing this initally on macOS, I was using `buck run eden/fs/config/facebook:edenfs_config_manager`, which runs as the correct user, but when this will be ran by systemd timers, the current user would be `root`. This changes the gk checker to check for the configured user instead
The behavior on Windows should be the same because the config manager already just runs as the user. I will retest it anyway though
Reviewed By: kmancini
Differential Revision: D51437085
fbshipit-source-id: b9a454c3210ace4e48e5580f5fd504448f56d620
Summary:
We had a crash in the hgDatapackStore because we held a reference into
an object in the eden config. I fixed it in D51133312, but I had punted on
writing a test since it needed some plumbing.
We finally have all the plumbing we need so now we can write a test that
causes the edenconfig to be reloaded while a reference is being held to a
config value.
Reviewed By: genevievehelsel
Differential Revision: D51621134
fbshipit-source-id: 5b7afd0e7971b58adb3ed6e51c8895f0d95275f6
Summary:
We had a crash in the hgDatapackStore because we held a reference into
an object in the eden config. I fixed it in D51133312, but I had punted on
writing a test since it needed some plumbing.
The next piece I need is to be able to trigger an EdenConfig reload. The
inode test stuff already has TestConfigSource that allows doing that. Pulling
this code out of cpp files so I can reuse it in my test.
Reviewed By: genevievehelsel
Differential Revision: D51621102
fbshipit-source-id: 2f07325b2c476d8c188af977455166709dcc9619
Summary:
We had a crash in the HgDatapackStore because we held a reference into
an object in the eden config. I fixed it in D51133312, but I had punted on
writing a test since it needed some plumbing.
The first piece of that plumbing is being able to inject an artifical delay
from when the reference into the EdenConfig is aquired and used.
FaultInjector can help us do that. Plumbing that down into the hgDatapackStore
in this diff.
Reviewed By: genevievehelsel
Differential Revision: D51621073
fbshipit-source-id: fbdd29d8bebadec5ca932e3bc7e9b6c8c2654352
Summary: This was followup code review from the initial implementation, to improve readability. There is no behavior change here, just moving/renaming some things.
Reviewed By: kmancini
Differential Revision: D51375399
fbshipit-source-id: ebfde4a3bbb67f1efb421c7e385430626b00d212
Summary: This is causing the wrong config to be generated when eden is started
Reviewed By: rafael-munoz
Differential Revision:
D51666121
Privacy Context Container: L1153010
fbshipit-source-id: fd9a3919d73cd8659acd33aa6423c4735dcb8822
