Summary:
A primary advantage of existentials over sum types is that they allow
they breaking dependencies. Now that channel_ is an FsChannelPtr, we
no longer need to include PrjfsChannel.h in EdenMount.h.
Reviewed By: kmancini
Differential Revision: D45109931
fbshipit-source-id: 7774804fe4056ef5b897de294707496ff722db28
Summary:
On Windows, macOS, and Linux, EdenMount::channel_ is an
FsChannelPtr. Code that wants to branch on a specific implementation
can use getFuseChannel or getNfsChannel or getPrjfsChannel, though
those will be removed later.
Reviewed By: kmancini
Differential Revision: D45109907
fbshipit-source-id: d4b3b46bd78b9487909bf1cb7c1c754e48b6e9a9
Summary:
Our build signals for the Python CLI are currently fragile because other code
pulls main_lib into complex test cases, when it really only wants the CLI's
return code constants. This decouples main_lib from these external tests.
Reviewed By: genevievehelsel
Differential Revision: D45404900
fbshipit-source-id: 4adb18030ac0ca934c6ceab7cbcd50ebd20c707d
Summary:
Fixes `buck2 run edenfsctl` by ensuring daemon.py knows how to use the
`__EDENFSCTL_RUST` environment variable, which we already plumb through our run
target, to lookup the path to the CLI. This is needed so that `edenfsctl
start` can perform mounts at startup on macOS, for example.
Reviewed By: chadaustin
Differential Revision: D45533101
fbshipit-source-id: c9df700e506409f734ca98a4fb205f962fd8ed78
Summary:
The shell scripts and awk fu did not always work correctly, so use
Python to parse the Linux mount table.
Reviewed By: mshroyer
Differential Revision: D45238375
fbshipit-source-id: 6d8918b8ecbdfe37e8ceba2b0962e5f821794140
Summary: importstack is for automation so let's skip the messages for human.
Reviewed By: muirdm
Differential Revision: D45530213
fbshipit-source-id: b9fde9e24b5d53a77af29aa03c0c685db791d9d1
Summary: Wrong indentation. `cleanupnodes` should be called once for N commits.
Reviewed By: muirdm
Differential Revision: D45530212
fbshipit-source-id: 6cd21a6b0de3e91bc9c7a2c64333540f2b886cf3
Summary: With all the building blocks in place, this diff implements chunking support in the core sharding framework. Because the framework level contract is being modified, it affects all the services and jobs using sharding. In usual cases the changes are minimal but in x-repo and chunked jobs, there are some additional modifications. Note that these changes are not backward compatible so I will be stopping the existing chunked jobs (`AliasVerifyProd` and `AliasVerifyBackup`) before rolling out these changes. I plan to test these changes on the test sharding job.
Reviewed By: mitrandir77
Differential Revision: D45479019
fbshipit-source-id: 6e2290de356642a2dc33e35b56c9ab25c2571769
Summary:
This test have been broken because it relied on deprecated openssl binary from
platform 009.
Differential Revision: D45505489
fbshipit-source-id: b3c52c5b22fb7f77fd0812ec4a2d4d6337b71b51
Summary:
As a follow-up to a code review comment in a previous diff, move
acceptStopped into the EventBaseState to make it clear it's only
accessed on the EventBase thread.
Reviewed By: kmancini
Differential Revision: D45106980
fbshipit-source-id: 5bd201a0652fa2b539e18a7584fae48fd3abd54f
Summary:
FuseChannel already had an asynchronous initialize(). Lift that to
FsChannel and unify initialization of Prjfs and NFS.
Reviewed By: kmancini
Differential Revision: D45106889
fbshipit-source-id: 632ddfa275c732fc30efc1984bda43f61c37fd5e
Summary:
EdenMount shouldn't know about the vagaries of FsChannel teardown. In
fact, FuseChannel already has its own requirements, hidden behind a
unique_ptr deleter. Use the same mechanism for NFS, allowing us to
remove a .via() special-case.
Reviewed By: kmancini
Differential Revision: D44810223
fbshipit-source-id: db2f88bc3843b3842de60c5a993dfd9890dc3c15
Summary:
mshroyer suggested renaming run_with_fault in D45135531 as the previous name
was confusing and he expected IO to fail, not to be blocked.
Reviewed By: fanzeyi
Differential Revision: D45459918
fbshipit-source-id: c39e5d414a6c0fda4eeff8314e6a266fa8eee185
Summary:
When unmount is called, the mount is kept alive until all the pending
ProjectedFS callbacks have completed (D45135531) to avoid use-after-free.
However, the notification callback is handled entirely in the background and
would thus not prevent the mount from being unmounted, and not being freed.
In order to fix this, we need to keep the PrjfsChannelInner alive for the
duration of the notifcation handling, which as a side effect will keep the
mount alive.
Reviewed By: mshroyer
Differential Revision: D45251808
fbshipit-source-id: e78afbc388cdf75f6d8d4e774bced8fdf841db5e
Summary: Will look at seeing if the script exists in the repo to determine whether to use absolute path to run a script.
Reviewed By: muirdm
Differential Revision: D45106495
fbshipit-source-id: 88393b6563470e273218849cb71d41ab05a27df1
Summary: As we need to add some functionality to `mononoke newadmin` that is not ready to be shared with `open-source`, extend the `subcommands` macro and adapt the `mononoke newadmin`'s `main` function to allow loading subcommands that are private to `facebook`.
Reviewed By: RajivTS
Differential Revision: D45398027
fbshipit-source-id: 493b24920c5310d67417274ab96e434a2bc0fbac
Summary:
I propose that instead of logging from pushredirection code we log from
backsyncer code. In that case it won't matter how the commits made it to
large repo, they'll be logged in small repo regardless.
Differential Revision: D45401754
fbshipit-source-id: 471cf6c3a4afd3da2f6a01729b7443f506e0d092
Summary: This code will be useful for anybody doing the logging.
Differential Revision: D45401755
fbshipit-source-id: 0d1e4f933e241cce9d8078ded8963ee0f74080a3
Summary:
I lied in the previous diff. Without some extra scribe configuration the extra
logs wouldn't make it to fake scribe output in the test
Differential Revision: D45401756
fbshipit-source-id: f582dc6cb8692b0fd4ff2fec5da95f17b59366a4
Summary:
I needed to use a bit different repo object than blobrepo and it's been frustating that I can't use it here.
This is not a complete deblobrepoification yet but it's definitely step in the right direction.
Reviewed By: YousefSalama
Differential Revision: D45356780
fbshipit-source-id: f8a9698e3afbcf4f08c6b3db485466ab19fc154d
Summary:
I needed to use a bit different repo object than blobrepo and it's been
frustating that I can't use it here.
Reviewed By: YousefSalama
Differential Revision: D45356781
fbshipit-source-id: b7effca4d547975bf085e58d704247f48dae9593
Summary: Phabricator uses the `commit_file_diffs` method to get metadata diff which it then uses to render information on the diff view page. With `ContentMetadataV2`, there are opportunities to avoid loading the files in the commit in memory cause we already have the necessary metadata in hand. However, this optimization will only come in play when the file is non-textual (i.e. binary or non-UTF8). This diff adds logging changes to identify such requests from clients.
Reviewed By: YousefSalama
Differential Revision: D45395092
fbshipit-source-id: 19d2e7370adafff6e626387e1480249349da9be1
Summary: We know there are some `ContentMetdataV2` blobs that haven't been generated with the correct format and when `ContentMetadataV2` gets generated directly on user request path, it doesn't generate the necessary aliases. This diff adds the necessary support for that use-case.
Reviewed By: Croohand
Differential Revision: D44197817
fbshipit-source-id: d8dd654228618232ed1e24d6755c673322fc18db
Summary: Now that `ContentMetadataV2` is exposed via `FileStore` and all the relevant metadata methods, we no longer need to validate or scrub `ContentMetadata` blobs. Once we remove support from walker, we can then go ahead and delete the unused `ContentMetadata` blobs.
Reviewed By: mitrandir77, yancouto
Differential Revision: D43905796
fbshipit-source-id: 20c98a65192244546b7212b82eb24612abc92c4a
Summary: We were displaying a line's current line number instead of the original line number when that line was first introduced.
Reviewed By: quark-zju
Differential Revision: D45458176
fbshipit-source-id: 75c6a43f0d015bdea6585058ebec74526104dbf6
Summary:
Now if the user requests a blame including working copy changes, we still use EdenAPI as long as the file is clean.
For a real fix we can use the server data as a base and then augment it as required. That is non-trivial, so let's do that later.
Note that I noticed we are displaying the wrong line number in blame output. Will fix in following diff.
Reviewed By: quark-zju
Differential Revision: D45415546
fbshipit-source-id: 47f412c835d20107ad945dc030cb8b3eb541bf6e
Summary: Before we do a slower search for all matching paths, do an optimistic "get()" for the exact path. If it exists and is EXIST_NEXT, we can confidently return it early. I think this is the normal case in practice because case insensitive filesystems are case preserving, so things normally match up between dirstate and filesystem.
Reviewed By: quark-zju
Differential Revision: D45378856
fbshipit-source-id: 7fb5fdadd417d95a39bb559d703dc792e336bca0
Summary: I don't think we need directory results from watchman, and in particular they were triggering use of the GitIgnorematcher because we were stupidly considering them as untracked files.
Reviewed By: quark-zju
Differential Revision: D45378855
fbshipit-source-id: c3dce4212801749ac25cc4f7ccd5c86cf8d8ee76
Summary:
We were using an ExactMatcher to walk the treestate and discover tracked files that werent returned by watchman after fresh instance (and hence must have been deleted). This was slow, so replace with a simple HashSet check.
With this change, however, we lose some case-insensitive correctness (although I have discovered things didn't work that well before anyway). Improve things by ignoring file deletions if they case insentitively match other files we got from watchman. This handles two cases:
1. File "foo" is renamed to "FOO" and watchman sends us both "foo deleted" and "FOO updated" (now we ignore the "foo" deletion). This issue was pre-existing.
2. Watchman restarts and "foo" has been changed to "FOO". Watchman only reports "FOO", and we need to ignore the "foo" deletion we infer when walking the treestate. This issue was introduced by this commit's perf optimization.
Reviewed By: quark-zju
Differential Revision: D45378862
fbshipit-source-id: 3dd56ef2ff3baf6115333326d4c8386c91715ab5
Summary: Break down the "status" work into more spans so I can tell what is going on in traceprof output for slow commands.
Reviewed By: quark-zju
Differential Revision: D45378863
fbshipit-source-id: a9604d888b3c9ca37e284a023c33105404df8707
Summary:
I noticed the parallel file detector was slower when there aren't many dirty files that need content checks. Now that we use the watchman file metadata, we don't have to "stat" clean files at all, so the parallel overhead outweighs the the light bookkeeping work.
Address by combining the non-parallel detector and parallel detectors. The metadata comparison is done without parallelism since we always have the metadata in-hand. Any content comparisons are still done in parallel since that work is IO heavy.
Reviewed By: quark-zju
Differential Revision: D45378865
fbshipit-source-id: 31d0890e88183b09b7703d66f69be86f86a9b0e8
Summary:
Per Microsoft (Christian Allred), calling any Prj functions after
`PrjStopVirtualizing` will lead to use-after-free due to the internal
virtualization context being freed on `PrjStopVirtualizing`.
In EdenFS, there is a subtle cases where this may happen, in particular, if a
callback is ongoing and detached to a background thread, while
`PrjStopVirtualizing` is called as a result of `PrjfsChannel::stop`. In that case,
the callback will attempt to use a freed virtualization context and crash
EdenFS!
To prevent this, we need to make sure that `PrjStopVirtualizing` is only called
when no in-flight callbacks exits. This can be achieved by simply moving the
`PrjStopVirtualizing` into the `PrjfsChannelInner` destructor: a `shared_ptr` of
`PrjfsChannelInner` is held alive for the duration of callbacks.
After `PrjfsChannel::stop` is called, no new callbacks will be accepted due to
the `PrjfsChannelInner` field being `nullptr` thus failing IO early ensuring
that the lifetime of the mount won't be extended indefinitively.
Reviewed By: mshroyer
Differential Revision: D45135531
fbshipit-source-id: 15b25efc121ab1577c4355e07357148d122cc39f
Summary:
The issue is that revset parser raised an error for `.^::`, the user's
expectation was something like `(.^)::`, but the current parser thinks `^` is
an infix in `.^::` [2], and then it tried to parse `::` as a prefix, and
eventually got the "not a prefix: end" error. I would agree with the user and
think `.^::` should be parsed as `(.^)::`.
I asked Yuya this question, the current parser can't recover from
the error while parsing. He suggested to backport a workaround from
https://www.mercurial-scm.org/repo/hg/rev/beb667c9880f.
This diff is to introduce that change to Sapling. Another potential fix might be
resolving this ambiguity in an ealier stage by checking token type after ^, so
we don't need the `_fixops` function to rewrite the tree, but that would need
more code.
Related issue: https://github.com/facebook/sapling/issues/612
Reviewed By: muirdm
Differential Revision: D45443282
fbshipit-source-id: 8eba6fbba76bdde5051342abc1d1c739eced2661
Summary:
Loop vars have been out of favor since 3.7. in 3.10 they are not acceptable in most places.
#buildmore
Reviewed By: itamaro
Differential Revision: D45072252
fbshipit-source-id: 667af4b1d44e709cec84e9f5e4ef173063b0cb9e
Summary:
Previously we prefetched the trees for all relevant commits so that fetching the fctx via ctx[path] was fast. Fetching the trees was still slow, though, since it takes O(path depth) round trips. However, I've realized we don't actually need the file context since all the info used to display the blame output comes from the change context.
Tweak things so that annotate can provide either the change context and path, or the file context.
This gets rid of the slowest remaining part of edenapi assisted blame. This reduces "hg blame" latency of edenscm/merge.py from 8s to 1.5s.
Reviewed By: quark-zju
Differential Revision: D45405272
fbshipit-source-id: b65813f958645009baf41240cae15f0f60e2bed5
Summary: We're seeing even more users that are crashing due to win32 errors. It seems like these sorts of E-Menu issues shouldn't crash Eden entirely. A non-functional E-Menu is better than crashing Eden entirely. Therefore, let's log these as errors for now.
Reviewed By: xavierd
Differential Revision: D45365471
fbshipit-source-id: f4092a9d44c1c5e6749e52d7e656c5de339551d3
Summary:
join expensive remote bookmarks fetching into a single fetch in cloud sync
This can be done because streaming has been implemented for public pulls!
And if we join it will be faster overall to pull.
Although, this is very rare, I've seen a workspace with oculus-10, oculus-12, and portal-pie branches together.
Reviewed By: mitrandir77
Differential Revision: D45394191
fbshipit-source-id: 901370f4be8b846b1124894554f17cf8b0067d1e
Summary: Directs RangeNodeStream usages to use commit_graph().range_stream instead (conditional on a tunable (enable_new_commit_graph_range_stream) for critical ones).
Differential Revision: D45356044
fbshipit-source-id: 379f214e8a6da1dc69d8744950e0e313abe8d813
Summary: Migrating from using the Changesets facet to the CommitGraph facet would otherwise require deblobrepoifying too much stuff.
Differential Revision: D45319163
fbshipit-source-id: f93c091befc5b3419a7d1364107cc717c9d19303
Summary:
The previous diff introduced `RepoShard` as the struct containing the parsed information from the passed in string shard-id. This diff adds relevant test cases to ensure that the logic for parsing the string based ID is accurate.
This diff also modifies the original parsing logic in response to the errors discovered during unit testing.
Reviewed By: mitrandir77
Differential Revision: D45355079
fbshipit-source-id: 78f28fa80bf4a3186ab66b08b29f20d39230094d
Summary: make mutation tracing message similar to other requests
Reviewed By: mitrandir77
Differential Revision: D45392969
fbshipit-source-id: 9ba180962fbde4607b777273837d41aeefe30263
Summary: clean up left over legacy code
Reviewed By: quark-zju
Differential Revision: D45368971
fbshipit-source-id: 2cbba53b571f1cfc004ec59e44871ea3e8343b2c
Summary:
This will allow tests to easily block some methods to perform concurrent
operations to them. For instance, this will allow that unmount concurrently to
read will not crash EdenFS.
Reviewed By: mshroyer
Differential Revision: D45186931
fbshipit-source-id: 3bd1fa9191d6e18cec985159914c0ee03a22a107
Summary:
Background runs of eden rage are making tracing pastes. We should not make these
pastes in dry-run automation runs.
I should have caught this in code review, my bad.
This uses the same fix as D28541898. Perhaps the paste method should not work in automation.
Reviewed By: xavierd
Differential Revision: D45362138
fbshipit-source-id: c28a18bc739921b3dc6e5b906c2491427f501644
Summary:
EdenFS doesn't care about the min/max definition from Windows.h, thus define
NOMINMAX. This allows removal of a lone `#define NOMINMAX` in FaultInjector.h.
Reviewed By: janondrusek
Differential Revision: D45359806
fbshipit-source-id: 7f24bda46e9720fdac471c24ff05a32c504d3eff
Summary: Add GitRenameFinder, this diff only consider the files that are exactly the same as source file as a rename. Will add content similarity check later, I am considering using [similar](https://github.com/mitsuhiko/similar) crate for this, since it has a nice API: https://docs.rs/similar/latest/similar/fn.get_diff_ratio.html
Reviewed By: quark-zju
Differential Revision: D45232985
fbshipit-source-id: d462063aff74103f8332c838b137df9dabc01501
Summary:
streamimg support for public pulls
If all heads in the request are public, use the streaming API, so that we are not
limited to any keep alive timeouts on HTTP side, since we can return things more gradually.
This is to be used for hg pulls in not segmented changelog repos, pulling of
expensive branches, etc
Commit Cloud would still use the API where we calculate the whole graph before returning it but this is fine because:
* we need to derive filenodes and calculate phases in parallel
* that function is a little bit more optimised since better batching of hg <=> bonsai conversions, for example.
Reviewed By: mitrandir77
Differential Revision: D45352368
fbshipit-source-id: 7549396d249a50adc163b5de1586d8789776b0df
Summary:
eliminate lifetimes from ancestors_difference_stream method since they
create unnecessary complexity and some consumers require a static stream.
self (arc under the hood) and ctx is cheap to clone there
Reviewed By: markbt
Differential Revision: D45352367
fbshipit-source-id: 6eca62838d1e31ff046ce987c52afced84227a6c
Summary: This diff introduces the `RepoShard` type which will serve as the holder of shard-level state information. Currently, we keep the information encoded in the shard_id (e.g. `fbsource_TO_ovrsource_CHUNK_1-11_SIZE_1000`) and expose it directly to the clients who then have to parse it themselves to get the relevant pieces of information. To avoid this duplication and to centralize the parsing and error handling logic, we will use the `RepoShard` struct. I have added it in `sharding_ext` crate so the type itself can be used independently of the sharding framework (for lightweight dependencies).
Reviewed By: markbt
Differential Revision: D45277195
fbshipit-source-id: 6ae9717fdcf312f0546e7b75d530e8dec8f69d32
Summary:
The `sharding_lib_ext` crate was created to contain the add-on logic that is used by `ShardedProcessManager` but also by other components (e.g. encoding, decoding repo-name for shard-aware routing). I plan to use this crate further for adding chunking support. Before I do that, I decided to clean things up a bit. Particularly,
- Move `sharding_lib_ext` crate under `cmdlib` instead of keeping it under `mononoke` since its a library
- Renamed the crate to `sharding_ext` removing the `lib` since all crates under `cmdlib` are libraries anyway
Reviewed By: markbt
Differential Revision: D45272976
fbshipit-source-id: 90be4f1f4659a03a591a8acbb04fdcbf06234a74
Summary: Similar to direct `gitimport`, this diff adds support in `remote-gitimport` to upload raw git annotated tags when pushed to the remote git server. Note that this only uploads annotated tags since basic tags get directly converted into branches while annotated tags need a corresponding changeset to be represented in the Mononoke data model. This diff doesn't delete existing tags when deletions are pushed to the server and modifications of the raw git tag are prevented by the no-update model in Mononoke blobstores.
Reviewed By: markbt
Differential Revision: D45271703
fbshipit-source-id: f7cdbed7fb665c9d32b08e26d7e9f404175b6eb9
Summary:
The `criterion` crate is a heavyweight crate that shouldn't be part of `types`
dependency tree.
Reviewed By: muirdm
Differential Revision: D45314783
fbshipit-source-id: 6476539c8d95927a6e6fa821e312632c6fffcced
Summary:
xdiff has logic to "cleanup" (force "unmatch" lines that are unique on one
side) with the intention to speed things up. However, in the `edit_cost`
case this might actually slow things down. So let's skip the cleanup logic
accordingly.
Using the test case of `test_edit_cost_large_diff()`, the `edit_cost(100)`
now completes in ~0.6s, down from ~1.5s.
In [7]: %time bindings.xdiff.edit_cost(a,b,100)
CPU times: user 253 ms, sys: 347 ms, total: 599 ms
Wall time: 597 ms
Out[7]: 100
In [6]: %time bindings.xdiff.edit_cost(a,b,1000)
CPU times: user 282 ms, sys: 351 ms, total: 633 ms
Wall time: 632 ms
Out[6]: 1000
Reviewed By: muirdm
Differential Revision: D45305089
fbshipit-source-id: 20f49bce742f7ef02f17fc5a0d0a2008cb1eecbd
Summary: This makes it possible to use `edit_cost` in Python.
Reviewed By: muirdm
Differential Revision: D45303297
fbshipit-source-id: f948f317f907f9e808aec66f5e388abe632c768a
Summary:
`edit_cost` returns the edit cost with a maximum cap, without producing actual
diffs.
This can be useful for similarity check without calculating the full diff.
See `test_edit_cost_large_diff` for performance comparsion.
Reviewed By: muirdm
Differential Revision: D45303299
fbshipit-source-id: c33bb6c186c2a5281910530988397ec00e37769d
Summary:
Update the bindgen logic so it skips C stdlib items and impls Default.
This makes the next change smaller.
Reviewed By: muirdm
Differential Revision: D45305448
fbshipit-source-id: dc3462c3ff037bafc3ebc867350c28aeda81c3a3
Summary:
In this code:
ps1 = msmall.ptr + msmall.size - 1
In case an empty slice is passed from Rust,
- `msmall.ptr` is 0x1, since Rust uses pointer 0x1 for empty slices.
- `msmall.size - 1` is `-1` (overflow).
- ps1 will be 1 - 1 = NULL, which makes UBSAN unhappy.
Resolve it by skipping the code block for `size == 0` cases.
This issue wasn't found before, because previously `xdiff` tests do not use empty slices.
Reviewed By: muirdm
Differential Revision: D45305368
fbshipit-source-id: 2b9ca3dd8dd48e9a3c23e41c556f63af0a59fc6c
Summary:
xdiff has internal logic to optimized out lines unique on one side, since they
never match the other side. We want to track how many lines are optimized out
for bounded similarity check.
The value will be used by the next change.
Reviewed By: muirdm
Differential Revision: D45303298
fbshipit-source-id: a9c58197664800013473f59dead76852a10c0e4a
Summary: Replace the C xdiff extension with the Rust one.
Reviewed By: muirdm
Differential Revision: D45303295
fbshipit-source-id: a278a0c7a26d6d7f73cc1aaf70c1ccfcf5e10f3b
Summary: This will replace the C Python `xdiff` extension.
Reviewed By: muirdm
Differential Revision: D45303296
fbshipit-source-id: 29a16b8c75f2109fa3b5bb5f4406a34ff880c160
Summary:
Implement the `blocks` API in xdiff Rust wrapper so it can serve mdiff.py
use-cases.
Reviewed By: muirdm
Differential Revision: D45303294
fbshipit-source-id: 8eae63a52fa2e7ec825d55b140f598a06e068c0a
Summary: These no longer apply. This change should have been part of D44715340.
Reviewed By: sggutier
Differential Revision: D45339827
fbshipit-source-id: f112625dfae011c70dfb17c3f9c13501b96428e7
Summary:
We've seen at least 1 user that crashed due to:
```
E0417 09:41:18.412118 20840 WindowsNotifier.cpp:68] TrackPopupMenuEx failed: Error (0x5a6) Popup menu already active.
F0417 09:41:18.434692 20840 WindowsNotifier.cpp:454] Exception occurred in E-Menu WndProc: TrackPopupMenuEx failed: Error (0x5a6) Popup menu already active.
```
Failures like this really shouldn't cause the entirety of Eden to crash, so let's ignore it for now.
Reviewed By: chadaustin
Differential Revision: D45130458
fbshipit-source-id: f906a17ac359f2a3c48a852d0668f70014ccd63d
Summary:
Add a new commit resolution step that checks if unknown hashes are from another repo, and if so translates to corresponding hash of current repo.
This behavior is enabled by configuring the other repo name under the megarepo.transparent-lookup config.
Reviewed By: sggutier
Differential Revision: D45092192
fbshipit-source-id: 9434a21d258cdfda153eb81e2d1eb0149f8a063e
Summary: Allow specifying repo name for the http edenapi client. This makes it easier to do edenapi queries for other repos.
Reviewed By: quark-zju
Differential Revision: D45092190
fbshipit-source-id: 77d09eba0cb4f1cbf169d24bdf424cf1e1519844
Summary:
Respect the optional "from" and/or "to" repo names to convert the commit ids from and/or to different repos. Only applies for repos that are megarepo'd together.
I added a get_repo method to the EdenApiContext object to facilitate the xrepo behavior.
I thought about disallowing requests where both "from" and "to" are different from the "subject" repo in the path, but it seems harmless.
Reviewed By: quark-zju
Differential Revision: D45019173
fbshipit-source-id: 8a25f5506521765296142553963b19f47570e865
Summary: Previously any outer error returned by an EdenApiHandler got turned into a 500. With this change, we no longer allow the generic `Into<anyhow::Error>` propagation. This allows for specialized HttpError propagation that retains response code.
Reviewed By: quark-zju
Differential Revision: D45296090
fbshipit-source-id: bb21518e5f8c77f578eec6d42a2582c474b5d1e9
Summary:
Use the CommitId directly as HashMap key to get rid of some per-type bookkeeping.
This minor change allows my next diff for xrepo translation to be significantly simpler.
Reviewed By: markbt
Differential Revision: D45019175
fbshipit-source-id: eb369b9e2d51345482590bbbbd9a1758b140b309
Summary:
I want to enable EdenApiHandlers to fetch other repo objects, but that requires various other objects not currently available. I don't want to complicate the handler signature with multiple additional args, so introduce a context object that can contain the dependent objects (i.e. RequestContext and ServerContext).
I tucked the path and query extractors into the context since they are barely used by any handlers (in fact, the path extractor is not used at all).
Reviewed By: markbt
Differential Revision: D45019176
fbshipit-source-id: ec4427d2f134277e89a8c48018f29c5f56cdc60d
Summary: Implements a stream that yields all changesets that are simulatenously a descendant of one changeset and a descendant of another, in topological order.
Reviewed By: markbt
Differential Revision: D45156176
fbshipit-source-id: 6c70d40252b5e2402ec1718e6cbe4c35ec8e9c1e
Summary: Make this help text more helpful. Also record that `--bookmark-info` and `--path` are mutually exclusive.
Reviewed By: MichaelCuevas
Differential Revision: D45315761
fbshipit-source-id: 150c80e1e1369c3f0c76d6708e9453fb64e3e7e0
Summary: `boost/regex.hpp` probably includes `windows.h` at some point, this causes a `min()` / `max()` macro conflict
Reviewed By: ispeters
Differential Revision: D45318842
fbshipit-source-id: 19fc29b03ca7dbf75fb6bfaef1787d8e73c329c2
Summary:
Context: https://fb.workplace.com/groups/learningrust/permalink/3405570553032635/
The previous diff makes use of `//third-party/rust:blake3` (or `//third-party/blake3:blake3-rust`) in `revisionstore` for computing the `seeded_blake3` hash of raw file content in `LazyFile`. `revisionstore` is a dependency for `edenfs` which also depends on the C++ library `eden/fs/digest:blake3_hash`.
When building `edenfs`, the linker comes across multiple definitions of the same symbol coming from the C++ and the Rust Blake3 library. Minimum repro in D45176271. Since there is no known solution for this, this diff uses the `blake3_c_ffi` crate which is a FFI wrapper (instead of native rust implementation) over the same C blake3 code as used by `eden/fs/digest:blake3_hash`. This solves the build problem while giving the same behavior in code. Note that the OSS version of the code continues to use the open-source Blake3 crate since the `duplicate symbols` error is not observed in OSS builds.
Reviewed By: mitrandir77
Differential Revision: D45310838
fbshipit-source-id: 9273bcc64871d5a86d07e23cc0200e305be7f4fd
Summary:
The `ContentMetadataV2` provides the seeded blake3 hash for each created file. This information needs to be passed to `buck` which interacts with `EdenAPI` through `EdenFS`. The relevant endpoints in `EdenAPI` are `trees` and `files` which were modified in the previous diff to support the new hash and are used here.
NOTE: This diff was already landed (D44010967) but then had to be reverted (D45156142) since it was causing build issues for EdenFS on ARM64 MacOS. More context [here](https://fb.workplace.com/groups/learningrust/permalink/3405570553032635/)
Reviewed By: quark-zju
Differential Revision: D45226440
fbshipit-source-id: 051dee9898d2441d6a03c10809988c6b99f36ab4
Summary:
We want `py2` to be a feature that returns False instead of crashing the tests
complaining about unknown feature.
Reviewed By: sggutier
Differential Revision: D45321821
fbshipit-source-id: 80a2a0077a50b29284a8db06f16def5e16081f14
Summary:
Fix issue:
AttributeError: module 'edenscm.ext.extlib.pywatchman.compat' has no attribute 'PYTHON3'
in various tests caused by D44715340.
Reviewed By: sggutier
Differential Revision: D45321822
fbshipit-source-id: 49025b54b04b819804be58ee0265d2530b67de66
Summary:
prefetch expensive bookmarks separately
this is to avoid timeouts until we have streaming support and ublock rollout
for AOSP repo
also, this provides better ux since we notify users that slow fetch is being executed
I also noticed that the commitcloud.remotebookmarkssync config option wasn't mentioned in the help. So, add it to avoid confusion.
Reviewed By: quark-zju, zzl0
Differential Revision: D44998087
fbshipit-source-id: f24bd959f02380ff0d7ff4f30dbe7e1647403329
Summary:
The goal is to avoid any serial fetching per commit/path in the blame history.
To get there, we now prefetch:
- Relevant commits and their parents. Parent hashes are used to verify commit hashes in repo.changelog.revision().
- Trees for all relevant paths for all relevant commits. This makes use of the newly improved manifest BFS iter. Note this is still the slowest part since it has a round trip for each directory as it walks the tree. I want to make the trees endpoint to support deep fetches to avoid these round trips.
Reviewed By: quark-zju
Differential Revision: D45273889
fbshipit-source-id: 7935874da178d9f39d9aab3cc9e026503db919ab
Summary:
This extension is not used and depends on server side features Mononoke doesn't implement.
Server assisted blame is instead achieved via an EdenAPI blame endpoint that fetches precomputed blame data.
Reviewed By: sggutier
Differential Revision: D45273888
fbshipit-source-id: 4efd269ad16f52b3df98e558b1fded7135c87419
Summary: Tweak the manifest BFS iter to take a list of initial nodes. I'm going to use this to allow prefetching trees for multiple commits at once.
Reviewed By: quark-zju
Differential Revision: D45273890
fbshipit-source-id: d3487a1a2ee71f602e535b8b7f673aa5d6a91aea
Summary:
First, kill the "depth" and "path" args since neither were set by any callers.
Then, use the bfs_iter to walk the tree instead of the naive level-by-level walk. This should be much faster, but more importantly is a step towards making prefetch support fetching multiple root trees/paths at once.
Reviewed By: quark-zju
Differential Revision: D45273891
fbshipit-source-id: 8b93ac1f5a327555d80c22410c7c95ab56af9984
Summary: removed python 2 specific references along with sys.version.info and dependencies.
Reviewed By: sggutier
Differential Revision: D44715340
fbshipit-source-id: 0dae8cf1bd9cb21a8ca336d22efecf796930bccc
Summary: I will reuse this `SearchDirection` in GitRenameFinder in the following diff.
Reviewed By: sggutier
Differential Revision: D45233573
fbshipit-source-id: 4885fc5df8beda729d23843471d63aa8e870c8f4
Summary:
Code42 was known to crawl the repository without taking
into account that files are virtual. It seems that this has been corrected in the new CrashPlan 11.0.1 agent, but we will land this diff as a safety precaution to not allow them to read the repository
Reviewed By: xavierd
Differential Revision: D45282206
fbshipit-source-id: 32f77aa96745bccb36e43d4df8ff47755a7670bc
Summary:
This data may be useful for analysing the blame history of the file.
* `approx_commit_count` is the total number of commits in the blame history of the file. It is approximate as some merged in commits may be omitted.
* `distinct_range_count` is the number of distinct ranges in the blame, which may indicate the file's complexity.
Reviewed By: RajivTS
Differential Revision: D45218574
fbshipit-source-id: c0c399f525f8c047a0415cf9d6f6e3483c033700
Summary: bunch of unit tests that relied on the specific error formatting weren't caught by CI and are failing. this fixes them by changing their error formatting to use `{:#}`.
Reviewed By: markbt
Differential Revision: D45245281
fbshipit-source-id: 6b0c9c5f3a3ecb594292d2a658ecce457f007e7e
Summary: Directs DifferenceOfUnionsOfAncestorsNodeStream callsites to use commit_graph().ancestors_difference_stream instead conditional on a tunable (enable_new_commit_graph_ancestors_difference_stream) being set to true.
Reviewed By: markbt
Differential Revision: D45042733
fbshipit-source-id: ca508d9b1a2e9efd7b45d1ee834f7587a7d886c3
Summary: Implements a streamable variant of ancestors_difference to be used as a replacement for DifferenceOfUnionsOfAncestorsStream
Reviewed By: Croohand
Differential Revision: D45042732
fbshipit-source-id: aa7377512866038172899545ee9c43a2d6db39f7
Summary: Use ChangesetsCommitGraphCompat when constructing Changesets in TestRepoFactory to enable double writing to the new commit graph
Reviewed By: mitrandir77, singhsrb
Differential Revision: D45227317
fbshipit-source-id: 6ce1809a2e30d930f8edef1efda0f9bef53b2a7c
Summary: Replaced by using TestRepoFactory directly and adding the extra config to the default config.
Reviewed By: RajivTS
Differential Revision: D45227320
fbshipit-source-id: bd93d1a2fd3a10398ca1cfd4b647a32627b83179
Summary: Has the same behaviour as the contruction of warm bookmarks cache in Repo::new_test_common. To keep the same behaviour this required making TestRepoFactory into an async builder and modifying all of its callsites.
Reviewed By: RajivTS
Differential Revision: D45227318
fbshipit-source-id: 51303d48bebc0c4b915a359a7d0942bf485e8db5
Summary: Adds a sync method that awaits the completion of all ongoing updates. Mostly useful for tests to avoid adding artificial sleeps.
Reviewed By: RajivTS
Differential Revision: D45227319
fbshipit-source-id: 9f3a9c7d063ad065b913db521a27624041b2400a
Summary:
fix 'repo changed while backing up'
the current logic is still incorrect for non besteffort (regular) mode
we introduced a high level transaction in D44664729, so now it is safe to drop
the check for non besteffort (regular) mode entirely
Why is it incorrect?
This is because the check is inside the retry loop (have lost the race),
inside the loop we apply commit cloud changes from the currect sync,
so basically if we haven't synced yet at this stage we will always fail the check.
We are loosing our retry logic that we should keep for the regular mode.
After a high level transaction has been added, we can drop the check and
resuscitate our retry logic.
The only possibility for another transaction to change the repo is that the transaction belongs to the previous iteration of the loop in the current cloud sync.
Reviewed By: quark-zju
Differential Revision: D45271716
fbshipit-source-id: 78de00c512f6ac737b107053563bd74b43080d32
Summary:
✨(clone): set publicheads to remote HEAD when cloning
c8b66c6899 added support for cloning the remote git repo's HEAD (in this
case `develop`), instead of main/master by default.
However, this doesn't mark `remote/develop` and its ancestors as public
commits.
This commit will also set remote git repo's HEAD as a public commit
Closes https://github.com/facebook/sapling/issues/600
Pull Request resolved: https://github.com/facebook/sapling/issues/607
Pull Request resolved: https://github.com/facebook/sapling/pull/607
Test Plan: - Added test `test-git-clone-sets-publicheads.t`
Reviewed By: sggutier
Differential Revision: D45249747
Pulled By: zzl0
fbshipit-source-id: 8c3a36bd21a25a856d13cc3b2c78f9cbdd9a94c5
Summary:
When creating a stack, make it possible to request that a derived data type is prepared as part of creating the stack. Since the data for the stack is in local cache, this should be faster than it would otherwise be.
This is a simple initial implementation. It requires that the data is already derived for the parents of the stack, and does not support parallel derivation of the different types, nor types that include interdependencies. In fact, we only support fsnodes and skeleton manifests right now.
Differential Revision: D45216982
fbshipit-source-id: c6ca7b265b1e6219a1193c51d0a984794aa34965
Summary:
Add support for skeleton manifests as a type of derived data that can be prepared.
The current enum for derived data types does not follow best practices and uses zero. Fix this now by adding a new variant for `FSNODE`. Once server support is available, we will remove `FSNODE_NEW` and switch to that value for `FSNODE`.
Differential Revision: D45216983
fbshipit-source-id: b0714ef6f9121139cc8d4602a5a8d1beaac00a3f
Summary: Move the preparation of derived data into a `mononoke_api` method in preparation for re-use.
Differential Revision: D45216984
fbshipit-source-id: ee99379685a202018b4c32b4b56e2930755d723a
Summary:
After my previous refactoring, the memory safety bug in Nfsd3 no
longer manifests, and we can remove the unnecessary defer call.
Reviewed By: kmancini
Differential Revision: D44988749
fbshipit-source-id: 2ebd5fbf96d05ce6850fc248a642409f827ffeec
Summary:
All of the comments in RpcServer are helpful. I found a couple
opportunities to improve their clarity.
It also helps to decouple RpcStopReason from the connection status.
Reviewed By: kmancini
Differential Revision: D44988608
fbshipit-source-id: 967f35ba09533a9c8c953ef33cd595b51e7c3111
Summary:
I'm carefully reading the takeover code and noticed a few things to
improve.
Reviewed By: kmancini
Differential Revision: D44946740
fbshipit-source-id: 558441a47425b15c19ca2a6816ae2e8bf3d68019
Summary: When tests are run in parallel, it can take more than a couple of seconds for the server update to become visible. Increase the wait to up to to 10 seconds.
Reviewed By: RajivTS
Differential Revision: D45229687
fbshipit-source-id: 24bc9607c285a4098c35a4105b8deac7916a7591
Summary:
add `debugcloudstatus` command
this command is to be used by automation, for example, on demand to get a name
of commit cloud workspace
currently they have to parse the name from the text output of the `hg cloud status` but providing json output is less error prune.
Reviewed By: quark-zju
Differential Revision: D45227570
fbshipit-source-id: 9e1a67edcefa7a2f7f11c7a80e99d061e5ca4bad
Summary:
Fix a typo. The word capabilities hurts my eyes when I look at it too
long, but it's the correct spelling, so sed it into place.
Reviewed By: kmancini
Differential Revision: D44945525
fbshipit-source-id: 05e82c058c3015b0590b07d6458840b4c56730b6
Summary:
Out of curiosity, I'd removed a .via(getEventBase()) from
TakeoverServer, and it manifested as hard-to-understand segmentation
faults in the integration tests. The issue is that FutureUnixSocket
has EventBase affinity and must only be accessed from it.
To make this safer and more obvious, put FutureUnixSocket and the
other takeover connection state in EventBaseState.
Reviewed By: kmancini
Differential Revision: D44945469
fbshipit-source-id: add3af84a35d451385a73afb622f9b63c7cdb217
Summary:
I want to use EventBaseState in TakeoverServer, so move it from
RpcServer to eden/fs/utils.
Reviewed By: kmancini
Differential Revision: D44945418
fbshipit-source-id: b9738162c84ca20da800d3662be6271f4b492acc
Summary:
Some dead code, a judicious use of makeFutureWith, and replacement of
thenTryInline+makeAsyncTask with via+thenTry makes the implementation
of TakeoverServer easier to follow.
Reviewed By: kmancini
Differential Revision: D44945165
fbshipit-source-id: 67c7b68b892c5112620bdc1ab2a0feb7df4d1864
Summary:
For small handle types, passing by value is the correct choice,
because it indicates unconditional ownership transfer. (It also
generates slightly smaller code and is less typing.)
Reviewed By: xavierd
Differential Revision: D44927835
fbshipit-source-id: 5fefee7889e8216e46548f4801e83f51cde8b8c4
Summary:
Simplify ServerState a bit by returning a const reference to the
member shared_ptr.
Reviewed By: xavierd
Differential Revision: D44924418
fbshipit-source-id: cd5f3a17d92b677ec7be43d9a07ed38dca1a3c24
Summary:
No need to have an intermediate object to forward callbacks from
ReadCallback to RpcConnectionHandler. This simplifies both memory
management and data flow.
Reviewed By: kmancini
Differential Revision: D44860344
fbshipit-source-id: af02f7d7f331d6ff3a8c4f49abb5ba1b95db5f85
Summary:
Adding the below cli outputs to eden rage
`eden trace inode --retroactive` (this one is empty though because we don't publish events right now)
`eden trace thrift --retroactive`
`eden trace hg --retroactive`
Reviewed By: kmancini
Differential Revision: D45165616
fbshipit-source-id: f4a89377fdc489e10bd78dc4175bdf89e2992cbf
Summary: Backing out the revert of D44173515 to unblock RE with their development. I will revert my own diff instead.
Reviewed By: genevievehelsel
Differential Revision: D45190063
fbshipit-source-id: 2b14fdc2fa118719aed0f3215393d172a162639f
Summary:
Based on statiscal data, most renames have either same basename or
same directory, so we will use this information when checking potential
renamed-to files.
Reviewed By: quark-zju
Differential Revision: D45184854
fbshipit-source-id: 7b4ed45d27781725c372beb33d24a0960d442816
Summary:
add file_path_similarity function, we will use this to reduce the
number of files to compare for Git use case; and also use it for sorting
the potential files for Sapling use case.
Reviewed By: quark-zju
Differential Revision: D45157537
fbshipit-source-id: 476c9fa07f2409a2cf0d1a6a72c57069a42ad347
Summary:
UDS don't have port numbers, so trying to register their port with the
portmapper is going to fail. This can cause eden to fail to start when we
enable uds. The current portmapper version we use can't support
this, so let's just skip attempting to register uds servers with the
portmapper.
Reviewed By: xavierd
Differential Revision: D45196379
fbshipit-source-id: 584e6e04129d1ddbcc756ed66c55a03c3955705a
Summary:
The case that I met was like the following DAG, and I'd like to skip those `Landed` commits when running `hg ab` since they already landed.
```
╷ o zzz Apr 21 at 18:18 zhaolong
╷ │ commit zzz
╷ │
╷ o yyy Apr 21 at 17:18 zhaolong
╷ │ commit yyy
╷ │
╷ x ccc [Landed as ddd] Apr 21 at 9:00 zhaolong
╷ ╷ commit bbb
╷ ╷
╷ x aaa [Landed as bbb] Apr 12 at 08:00 zhaolong
╭─╯ commit aaa
```
Reviewed By: sggutier
Differential Revision: D45184140
fbshipit-source-id: a539468843032d41ab4d61e5c461b9a1de53cb40
Summary: Like in the previous diffs, this diff uploads the raw git tags during repo import through `gitimport`.
Differential Revision: D45118571
fbshipit-source-id: f5a6bb9b5805235e9f66f97ab3a58bd165ef475c
Summary: The previous diff uploaded raw git commit data while the commit gets imported in Mononoke through gitimport or remote-gitimport. This diff ensures that the git `tree` associated with the commit also gets uploaded in Mononoke.
Differential Revision: D45085959
fbshipit-source-id: cf4f4a5ebd0d077972f75fe9c6a8d1dfc5067180
Summary:
With the `upload_object` method in place, this diff utilizes the new uploader method to upload the raw git commit in Mononoke store before creating its corresponding Changeset at Mononoke end.
This diff has the following changes:
- Modified the `get_object` method in `GitReader` to return the raw git content (i.e. header + body bytes of git object) along with the parsed `GitObject`
- Added the `read_raw_commit` method in the `gitimport_objects` library to read the raw commit bytes
- Modified the `ExtractedCommit` type to contain the raw bytes of the commit as well
- Modified all modes of `gitimport` execution to upload the raw git commit before creating the corresponding changeset at Mononoke end
- Made changes in integration test to validate the commit upload behavior
Differential Revision: D45082135
fbshipit-source-id: f339b2851646996be7f43f02458ea5afabb9ca48
Summary:
This diff includes the following changes:
- Introduces the `upload_object` method in the `GitUploader` trait for uploading raw git-objects (e.g. commit, tree, tag) in the mercurial mirror of the repo
- Implements the `upload_object` method in `Direct` and `Remote` implementations of the `GitUploader` trait allowing the objects to uploaded from both `gitimport` and `remote-gitimport`
This diff just introduces this method, follow-up diffs will use it for uploading different commit objects.
Differential Revision: D45046811
fbshipit-source-id: 9b4e6a295ec22e06294e410735e67877e6d0ef10
Summary:
The existing `mononoke_api` git methods were defined under `RepoContext` which relies on a specific type of `Repo` facet container as defined in `mononoke_api` crate. These methods can easily be used through SCS which serves as an external entry point into the `mononoke_api` crate. `remote-gitimport` uses SCS and thus can leverage the necessary endpoints.
However, `gitimport` (or `direct-gitimport`) directly references the logic behind these methods. To ensure that we don't repeat ourselves, there are two alternatives.
- Make `gitimport` use the same `Repo` facet container as the one in `mononoke_api` crate
- Decouple the git methods from `RepoContext` and define them with explicit `facet` dependencies
I chose the later since using the same facet container would have made `gitimport` use a much bulkier `Repo` definition than is necessary. I also made some changes in the `create_annotated_tag` method which are as follows:
- Removed the `start_write()` statement which was essentially an authorization check to validate if writing to the repo is allowed. This check makes sense when validating an external caller (e.g. SCS) but direct references of the library should be allowed
- Removed `CreateChangeset` permission check since we are not actually creating a changeset that corresponds to a commit. Additionally, any auth checks need to be performed at the call-site (e.g. SCS) and not in the API
- Removed the `allowed_no_parents` check since any changeset created for representing a `Tag` object will not have a parent. Infact the concept of parent is invalid for a tag.
- Replaced the returned `MononokeError` into `GitError` to maintain consistency with other methods
- Replaced the instance `save_changeset` method with the struct-level `save_changeset` method. The new method doesn't log to the scribe category thus preventing users tailing it from receiving an invalid commit creation notification. Additionally, the new method doesn't consider any bubble changes but they are anyway irrelevant in the context of these git methods.
Differential Revision: D45046639
fbshipit-source-id: 7ff02cde3fd356eb7a18f5ca12e6c478e23ee315
Summary:
Previously, the nfs.allow-apple-double was tentatively rolled out but had to be
rolled back as XCode disliked when an AppleDouble file was present on disk but
a new one couldn't be created. To avoid this issue, let's prevent AppleDouble
from being loaded from the Overlay and re-write the Overlay for directories
that contains one at the same time.
As a slight behavior change, the nfs.allow-apple-double is no longer read on
the fly but only read at startup time to avoid cases where a TreeInode is
already loaded with an AppleDouble and the config is flipped which would cause
XCode to fail to save files.
Reviewed By: chadaustin
Differential Revision: D44854105
fbshipit-source-id: 2c263411719d19eb1d457e26661620e8d6c905ce
Summary: In preparation for deleting blame v1, make blame v2 the default.
Differential Revision: D45044338
fbshipit-source-id: c9c71a148b42f4a3301d6c4fd0de5ad7ed321a3e
Summary:
Allow the walker to walk the blame v2 graph.
Since the walker is the main user of blame in the integration tests, we can make v2 the default for the integration tests, too.
Differential Revision: D45044294
fbshipit-source-id: 52c991dd81be78053ffc0a6953caf8a068ef59e3
Summary: This is a benchmark, so put it with the other benchmarks.
Differential Revision: D45114045
fbshipit-source-id: c524c76671d4183f6baa71ca9cd4c3cdfc402dac
Summary:
We don't need folly::Synchronized for the RpcServer state
machine. It's only touched from the EventBase, so make it
EventBaseState.
Reviewed By: xavierd
Differential Revision: D44860240
fbshipit-source-id: bcbc3d02e7cc5f2dc377718065c7f95eed30202d
Summary:
There's nothing TCP-specific about RpcTcpHandler, so give it a more
accurate name.
Reviewed By: xavierd
Differential Revision: D44860198
fbshipit-source-id: 55b9dc95b2caa9d87179a17435313b91bacfee49
Summary:
RpcServer can handle the accept callbacks directly. There's no need
for an intermediate object.
Reviewed By: xavierd
Differential Revision: D44860163
fbshipit-source-id: fd2d96735810a52cd287c357534c8f5bae9eee90
Summary:
I plan to use StateWrapper elsewhere so rename it to EventBaseState
and move it to the top level.
Reviewed By: xavierd
Differential Revision: D44860040
fbshipit-source-id: fb03b3b726df50099bcce144e343be0caeeaf1be
Summary:
There is no need for an intermediate object: RpcTcpHandler can handle
the write callback methods directly.
Reviewed By: xavierd
Differential Revision: D44860027
fbshipit-source-id: fe1f44b83a371291a21a3b1b26422bef5cbbf2a0
Summary:
To simplify RpcServer further, assert that removal of the accept
callback is a synchronous operation. Previously, it wasn't, because
specifying any EventBase in addAcceptCallback created a RemoteAcceptor
which scheduled operations on the other EventBase.
Reviewed By: xavierd
Differential Revision: D44853408
fbshipit-source-id: 9369cdcb0bd8450d42e195c15738362d4a53d2e7
Summary:
I'm trying to track down a subtle lifetime rule violation in
EdenMount. While reading RpcServer, I noticed a few possible
simplifications.
Reviewed By: xavierd
Differential Revision: D44849773
fbshipit-source-id: 4c27c47a7e2c211dcd040acce955e9b2f617b55b
Summary:
This removes an ifdef _WIN32 in mount shutdown by standardizing the
takeover logic on all three platforms.
Projected FS doesn't support takeover, but removing ifdefs allows
simplifying the control flow and unblocking NFS or other FsChannel
implementations on Windows.
Reviewed By: kmancini
Differential Revision: D44737273
fbshipit-source-id: fc6b718e01698ef3700b5c198750c0e3b1f62063
Summary:
By moving NFS's StopData behind FsStopData, the std::variant and
runtime checks during takeover can be unified into one path. The next
diff will do the same for Windows.
Reviewed By: kmancini
Differential Revision: D44737160
fbshipit-source-id: 19ca623f90b367d25385f177cafa44d7bf7ebb62
Summary:
Adds a buck target to drop us into a Python repl, to ease interactive
development of Python commands in the CLI.
Reviewed By: genevievehelsel
Differential Revision: D45159857
fbshipit-source-id: 31ff61ea29ab080cc1131905c6cf363368f3c617
Summary:
Original commit changeset: a755f0750ed3
Original Phabricator Diff: D44010967
need to revert to fix our release
https://www.internalfb.com/intern/sandcastle/job/36028797946731051/insights
Reviewed By: xavierd
Differential Revision: D45156142
fbshipit-source-id: 6e5362cf609f69cdbfeb7da30981227f93d42b65
Summary: The original `find_renames` finds all the renames of the commit and checks if it contains the path that we are interested in. This diff optimizes that logic to make it return early when found the target rename. In order to do that we split `find_renames` into two functions `find_rename_forward` and `find_rename_backward`, since they will have different optimization logics.
Reviewed By: quark-zju
Differential Revision: D45149506
fbshipit-source-id: b77b3b9ef4700581e6d044644d40bf405d66d263
Summary: fix a typo in the doc of read_file_contents function
Reviewed By: quark-zju
Differential Revision: D45126147
fbshipit-source-id: 6f541acb41c5de71f56c5573a5f56b536b669959
Summary:
Practically the graph log and the find_renames can use different impls
independently and form different combinations. This will be used for suppoting
Git use case in following diffs.
Reviewed By: quark-zju
Differential Revision: D45125851
fbshipit-source-id: 785800a0e4b1c74c48b97cf47132b45b268dc0bb
Summary: This tool was used for the original import of globalrevs. We are now the source of truth for globalrevs, so it is no longer needed.
Differential Revision: D45112829
fbshipit-source-id: 8ff106c66de74a79e6cd963ca48bd1875d6ee44c
Summary: This tool was used to rechunk the filestore. The filestore is always chunked now, so this tool is no longer needed.
Differential Revision: D45112830
fbshipit-source-id: cd03dd06f7dc55a6b47b8fc338accf9d300977d2
Summary: This tool was used for the initial population of the XDB blobstore. It has been superseded by the walker, and can now be deleted.
Differential Revision: D45112828
fbshipit-source-id: 134e2c2b761fd27a9b096df4d64404bc14c89c43
Summary: This tool lints toml-format configs which we don't use in production any more, and as such the tool is unused.
Differential Revision: D45112827
fbshipit-source-id: a4f7783144ab782adba5f0a0fb346e8cb542ad97
Summary: This tool was used early on to compute commit stats, but is no longer needed as it has been superseded by other data.
Differential Revision: D45112824
fbshipit-source-id: e88282647e8e3ed85a95ff2a9b15d9212851b8ff
Summary: This tool was introduced to fix up the problem with linknodes pointing at commit cloud commits, however that has long since been resolved, so the tool is no longer needed.
Differential Revision: D45112822
fbshipit-source-id: 350b770e9a3499a8bad8aca29badc3e20a02f168
Summary: General-purpose running of hooks has been superseded by the source control service `run_hooks` method.
Differential Revision: D45112823
fbshipit-source-id: 0093953da8fbf4289260058e7db076c2537bf262
Summary: This was used for an experiment with very large repos which has now concluded. Remove the code so that it doesn't need to be maintained.
Differential Revision: D45112825
fbshipit-source-id: 2591ab469ddf40ab8155c9d3713208f9bc3e5463
Summary:
When a user has more than 8 mounts we summarize the output of eden start. But
we have an underflow in the number of "finished" mounts.
The issue is that we do:
"number of successfully mounted mounts" - (
"number of mounts that have not yet started that were already printed" +
"number of mounts that failed that were already printed" +
"number of mounts that successfully mounted that were already printed")
and stick that in an unsigned int.
When there are not yet started mounts or failed mounts this number underflows.
This makes the output confusing.
This diff introduces a new category for failed mounts and displays this in the
summary.
And not yet starting mounts are considered "in progress".
With these two fixes that number that unerflowed is now just:
"number of successfully mounted mounts" -
"number of mounts that successfully mounted that were already printed"\
which should not underflow.
Additionally, while testing I noticed we have some output issues with the
errors we display when a mount fails. So I fixed the ordering of our output.
Reviewed By: chadaustin
Differential Revision: D45128100
fbshipit-source-id: 181adaaa7c9dbaba367cb6793ecf648515f4d511
Summary:
`eden config` prints the CLI configuration, which is different from
the daemon configuration.
The fact that both exist is confusing. Nonetheless, add an `fsconfig`
command which prints the current value of every config setting, as
well as the file that sets it.
Reviewed By: kmancini
Differential Revision: D41896506
fbshipit-source-id: 7a32daf04c5f4da31d50508ba4012585b0b6c069
Summary:
There are cases where the case of a file changes but not changes are done to
it, in which case this doctor check would fail.
Reviewed By: mshroyer
Differential Revision: D45129119
fbshipit-source-id: b0473485b760127715ec4372e8511936189c30a3
Summary:
We overuse make*FutureWith. As a rule, leaf functions should not need
it, because callers must already consider the possibility that an
exception is thrown prior to the asynchronous function's first
`await`. (The same rules apply to Rust async functions, C++
coroutines, Future chains, and Python async.)
Remove an unnecessary makeSemiFutureWith in performBindMounts..
Reviewed By: kmancini
Differential Revision: D44932401
fbshipit-source-id: 5a2a2773cc92faa59a981ee7b1e01324b5d3d5d5
Summary:
Adds a doctor check to warn the user if they're running a Windows build and
revision older than KB5022906, which has the fix for a known issue with ProjFS.
Reviewed By: xavierd
Differential Revision: D45104879
fbshipit-source-id: 57cae39c0f488f7b5c295b642a96f76bccbbb3ce
Summary:
Since EdenFS doesn't explicitely sets the timestamps of files/directory when
writing them on disk, ProjectedFS sets these to the current time. Prior to
running the background GC, this was an appropriate behavior: the timestamps of
files would only get changed on checkout when the file changed. However this
behavior changed slightly since background GC got introduced as placeholder
will get invalidated when not accessed for a while. The next time the
placeholder is written on disk, it's timestamps would now be different from
what it was before GC.
To avoid this issue, we need to consistently write the same timestamp before
and after GC. This is more or less the last checkout time. The one current
downside is that the last checkout time is reset across restarts, to fix this,
we could write the last checkout time to disk and restore it at startup time.
This is left as a future improvement.
Reviewed By: MichaelCuevas
Differential Revision: D44936789
fbshipit-source-id: a21ef3f2f0ef1c0d7ecb57658ae99647dc2bd99b
Summary:
StopData is slightly different for each filesystem implementation. But
it's used in the same way. Introduce a standard interface and port
FUSE to it.
Reviewed By: kmancini
Differential Revision: D44737143
fbshipit-source-id: 93ea567b6bc4d20a89ea1e285ab7240f533f6854
Summary:
It's been a long time since EdenFS has managed bind mounts in the
daemon. We no longer need to populate the bindmounts field in the
takeover protocol.
This is a small simplification for later changes.
Reviewed By: kmancini
Differential Revision: D44737123
fbshipit-source-id: d9127445083d83626d214efbbf271cd58df96ac6
Summary: Switch from the deprecated cmdlib to `mononoke_app` as a dependency. The type originates in `mononoke_app` and is simply re-exported from cmdlib, so this is a no-op change.
Reviewed By: RajivTS
Differential Revision: D45092104
fbshipit-source-id: ff8999a68c9707131a5e499b67f6d96492066b28
Summary: This library doesn't seem to be used anymore, so let's delete it.
Reviewed By: RajivTS
Differential Revision: D45092106
fbshipit-source-id: 46d477e6847b876ee2ece74d5e7ca69c787ce32a
Summary: The `--skip-caching` arg is replaced by `--cache-mode=disabled`.
Differential Revision: D45089768
fbshipit-source-id: 22b176b992351368fe96865c8559f83c75015409
Summary:
Now that Windows is going to support NFS. We need a reliable way to check at
runtime if a file should be read from disk (Prjfs) or it should be read from
the overlay (nfs).
We read inodes during fsck which is before the mount is fully initialized, so
we need to be able to detect the mount type earlier than how we do right now.
Here I am moving the NFS detection before eden mount initialization.
While moving this I realized that the InodeMap's view if NFS is incorrect
in the takeover case, so I am also fixing that here.
Reviewed By: chadaustin
Differential Revision: D45020849
fbshipit-source-id: b0a8fd431a37174c81b0d053d92b8bac026bd0f1
