Summary:
Currently, we take a `&mut (dyn Buf + Send)` as input when writng to Manifold.
This has a couple of downsides:
- It's not very ergonomic. From the API it's not obvious what mutations are
going to be done on your `Buf` exactly (it's going to be consumed entirely),
and it often results in code that's a little clumsy (see what I had to change
here), where make calls like `write(&key, &mut mydata.clone())`.
- It limits what we can do with it. If you already have a `IOBufShared` on
hand, you shouldn't need to copy it in order to pass it to Manifold, but
currently the Manifold client code does have to copy it because all it sees
is a `dyn Buf`.
This diff updates the client to take a `IOBufShared` directly, which has plenty
of convenient `From<...>` conversions (and some efficient ones like
`From<Bytes>`, which doesn't copy the `Bytes` at all).
Reviewed By: ahornby, Imxset21
Differential Revision: D28535539
fbshipit-source-id: bba1b963a96350ad57cc0fbfcc31f7e1eb36c317
Summary: Adding tests to verify that NDP and ARP entry resolution happens correctly on Aggregate ports. Recently we have seen sevs where ARP resolution happens incorrectly on subports
Reviewed By: jasmeetbagga
Differential Revision: D28156706
fbshipit-source-id: 3066620e9ed0d8805cf0d2733ea6fdece47b4254
Summary:
The goal of this diff is to make gettreepack requests that need to fetch a lot
of entries from blobstore faster.
Some of the gettreepack requests take really long time if they need to fetch,
say, 60K blobs. While this number of blobs is relatively high, it certainly
shouldn't take 1.5 mins to finish.
Upon closer inspection it looks like the reason for this slowness is a
combination of tokio coop mechanism and extremely large buffer size. Disabling
tokio coop and/or decreasing the buffer size makes requests significantly
faster (though I have to admin I didn't dig deeper to understand why they cause
this slowness).
To mitigate the issue I decided to decrease the buffer size first. This is a small
and predictable change that will be very easy to revert with tunable. Besides
having a buffer size of >1K doesn't make a lot of sense anyway, because we have
1K cachelib shards. Also this 10K buffer size was set a long time ago
(D8610582 (3dcf0048d8)), and there weren't a lot of measurements done to justify this size.
From the summary it seems that the problem was in slow manifold fetches from
remote regions, but as I said above, setting it highger than 1K is not helpful
anyway.
Reviewed By: krallin
Differential Revision: D28536686
fbshipit-source-id: 5f0ea04af8ce0c54c79d9668d95f2a1ece10f047
Summary: If you have checked out a shared workspace or other user workspace, this part of hg doctor could hide incorrectly, so, should be skipped.
Reviewed By: markbt
Differential Revision: D28505928
fbshipit-source-id: 65e1b3978a916fad2a33bb4f81ff1b75cd657567
Summary:
Paying the setup and teardown overhead of multiple processes seems silly, when we can pack in parallel in a single process.
Make it possible to run multiple packing runs from a single packer process
Reviewed By: ahornby
Differential Revision: D28508527
fbshipit-source-id: eab07d028db46d62731f06effbde2f5bc5579000
Summary:
Improve git-import speed by grouping many BonsaiChangests into each call to save_bonsai_changesets.
During earlier profiling noticed that the speed of gitimport was dictated by the save_bonsai_changesets, by doing this we can split the speed more evenly between save_bonsai_changesets and the steps to derive the diff and upload the missing file-blobs.
For profiling and performance analysis details please see
https://fb.workplace.com/groups/1619247701796399/permalink/1709890502732118/
Reviewed By: StanislavGlebik
Differential Revision: D28497973
fbshipit-source-id: 0fbcf37535554dd96664da4906633eeb07c58f7c
Summary:
In D28287486 (b9b5e16dcf) we added logic that should prevent some of the failures on
is_present() checks. However some parts of the codebase rely on `get()`
returning None and they don't use is_present() at all - in particular, that's
what configerator-a is doing.
Given that we have failures of our derived data tailer, let's try to replicate
the logic from is_present() to get().
Reviewed By: krallin
Differential Revision: D28506739
fbshipit-source-id: a37d6c4b5a43aa9a1284499831fcdc3ee5605b9e
Summary: Fetch bookmarks via the http edenapi protocol in the bookmark command with the --list-remote option when all bookmark patterns are full bookmark names (not prefixes).
Reviewed By: kulshrax
Differential Revision: D27331526
fbshipit-source-id: 4f4eda255c551c9b55c6966569755f493335b458
Summary:
This is currently calling `Buf::bytes()` in order to get a `&[u8]` out of
`Bytes`, but this method isn't actually guaranteed to return all data in the
`Buf`: https://docs.rs/bytes/0.5.6/bytes/trait.Buf.html#tymethod.bytes
> Note that this can return shorter slice (this allows non-continuous internal representation)
In practice this is fine because `Bytes` does in fact have a contiguous
internal representation (which is why we can call `as_ref()` on it), but let's
do the more correct thing here.
(I happened to spot this while looking into the feasibility of a Bytes 0.5 ->
Bytes 1.x upgrade, where this `bytes()` method was renamed to `chunk()`)
Reviewed By: farnz
Differential Revision: D28507885
fbshipit-source-id: 73e5f1ba587292f772c702127a3933ea76fceb9f
Summary:
The --workspace-version option is currently ignored by interactive history.
Allow it to be used to specify the initial version. This makes jumping back to
a much older version easier.
Reviewed By: liubov-dmitrieva
Differential Revision: D28478194
fbshipit-source-id: f4f121d919e89c298677256f227f2e96d63ef644
Summary: For whatever reason some of my debug logs got landed :(
Reviewed By: fanzeyi
Differential Revision: D28510825
fbshipit-source-id: 65bd9165020bfe3afff54109a9f550a460266737
Summary:
This test is a bit flaky because its execution depends on the other in which 2
futures that are spawned execute, which isn't entirely deterministic (those
tests were using a multi-threaded runtime, I'm also fixing this here).
Reviewed By: ahornby
Differential Revision: D28507337
fbshipit-source-id: 3c33dc7ffe73de0c6696523ed49d3b30ceda82c0
Summary:
Simple test that syncs a single commit from a source to a target.
A few notes
1) I want to use test Mononoke instance, but I can't initialize them in another
crate because of cfg(test). I removed cfg(test) to make it possible.
2) I initialized megarepo mapping in TestRepoFactory so that all tables
were in the same db. It's not strictly necessary, but makes initialization a bit simpler.
Reviewed By: mitrandir77
Differential Revision: D28444865
fbshipit-source-id: f39c917806709606ce8e5a1c1158f186d239d8b8
Summary:
In order to sync changesets we need to know what was the latest synced
changeset from a given source. Previously we wanted to store this data in sql,
however later we decided to store it in the file in the repo. The main reason
for doing that is that storing data inside the repo gives us both blame and
history, while database can't provide that.
This diff implements that - removes all the sql-related code for storing latest
synced changesets and instead puts the data inside the file in the repository.
Note that we still using sql to store mapping between (source, target)
changesets.
Reviewed By: mitrandir77
Differential Revision: D28470102
fbshipit-source-id: 996905b17ad4b0d5b0ea1e40c73d762850e113a8
Summary: Now it's at least possible to add new configs
Reviewed By: mitrandir77
Differential Revision: D28444866
fbshipit-source-id: cf0e2e737a125bdbd3b7eff55e8ee3f1d5a193d2
Summary:
Like it says in the title, this updates bookmarks to track the log id of their last update.
This will let us do incremental queries from the WBC. See the design doc linked to the
task here for details.
Reviewed By: farnz
Differential Revision: D28444852
fbshipit-source-id: e966ced8e136ad6c4306e977e21670caa457ea71
Summary: This diff introduces some currently failing test case to demonstrate the issue of files being created when EdenFS is not running on Windows. The tests are disabled for now since they are still failing. Later diffs should fix the issue this demonstrated and we can enable these tests on Windows at then.
Reviewed By: chadaustin
Differential Revision: D25285548
fbshipit-source-id: f0738bca05cfc82e5bf7b8238d009dc59bce93ca
Summary:
This diff implements FSCK for EdenFS Windows.
On Windows, users can still modify EdenFS mounts even when EdenFS is not
running, which may cause mismatch between EdenFS state and on-disk view. This
subsequently may cause issues when EdenFS is running again (such as permission
error, not seeing added entries in `hg status`, etc..).
This diff adds FSCK to EdenFS Windows. It is not exactly same as what fsck
traditionally do on *NIX systems. We are still dubbing it as FSCK since it
works at the same place as eden fsck.
At startup, FSCK will crawl the EdenFS mount to compare the overlay state with
disk state. It then synchronizes the overlay view with what the user has on
disk. Note Windows does not always permit user to modify the mount, it only
allows changes in certain situation. In particular, when the directory is in
Full state, this diff takes advantage of that so we can finish the scanning by
only scans such directories.
One limitation of Windows FSCK is that, it cannot reliably tell if the user
deleted a directory or file from dirty placeholder directories. This is because
ProjectedFS will hide untouched entries under dirty placeholder directory when
EdenFS is not running, and there is no way we can tell if the entry is gone
because of user deletion or hid by ProjectedFS.
This is not perfect but acceptable to some extent. One possible failure scenario is:
1. User creates a directory named `foo`.
2. User writes a file to that directory (`foo/bar`).
3. EdenFS then stops running.
4. User then deletes the entire `foo` directory on disk.
5. EdenFS starts again, `foo` will be recrated with an empty `bar` file. This
will still correctly show in `hg status`, and the user is able to delete
them again.
Reviewed By: xavierd
Differential Revision: D27872753
fbshipit-source-id: c553db568379062ff4504204c1a1785664f87c00
Summary:
We want eden doctor to report a problem about mismatched running and installed version only if the running version is more than 2 weeks older than installed.
Please see task for more context
Reviewed By: fanzeyi
Differential Revision: D28464491
fbshipit-source-id: 34b9b4cf533482f3006100bbf675c89ea7ee6fff
