Summary:
This just makes it more convenient to use in circumstances when fns expects
`impl LCAHint` and we have `Arc<dyn LCAHint`.
Reviewed By: farnz
Differential Revision: D23650380
fbshipit-source-id: dacbfcafe6f3806317a81ed4677af190027f010b
Summary:
We noticed spurious config file reloads, so add some logging to help
track that down.
Reviewed By: xavierd
Differential Revision: D23644447
fbshipit-source-id: 9953a17de402660c7f6491fb9abd8d702fa290e8
Summary:
GNU `df` (and any other coreutil that relies on gnulib's ME_REMOTE
flag) detects remote filesystems with some heuristics. One of which is
whether the device type contains a colon. Since edenfs is a remote
filesystem, include a colon, so it's properly detected as such.
Reviewed By: genevievehelsel
Differential Revision: D23520233
fbshipit-source-id: f4816b303e198d4e2a446efdcc5b49a593e09a05
Summary:
We intend to rename the edenfs device type to include a colon (and
possibly the backing repo basename). In preparation, update code that
detects edenfs mounts to include anything that starts with "edenfs:".
Reviewed By: genevievehelsel
Differential Revision: D23520008
fbshipit-source-id: 280f7617d5c96e23d548041b3482bca388076a7b
Summary:
For some unknown reason, we weren't setting this on Windows, which meant that
whenever edenfs would need to call edenfsctl (like at mount time when fixing up
redirections), it would always use the binary found in the PATH. While in most
cases this is OK, this is not the intended behavior for tests that are expected
to use the just compiled binary, not the one in the PATH.
Reviewed By: genevievehelsel
Differential Revision: D23653027
fbshipit-source-id: f1cc977e44b10c379d2b90bc7972bfec1fccad23
Summary: Previously, `SignalStream` would assume that an error would always end the `Stream`, and would therefore stop and report the amount of transferred data upon encountering any error. This isn't always the desired behavior, as it is possible for `TryStream`s to return mid-stream errors without immediately ending. `SignalStream` should allow for this kind of usage.
Reviewed By: farnz
Differential Revision: D23643337
fbshipit-source-id: 2c7ffd9d02c05bc09c6ec0e282c0b2cca166e079
Summary:
Similarly to FuseRequestContext, this will be allocated whenever ProjectedFS
calls onto EdenFS to keep track of timing, stats, etc.
For now, this just holds the callback data passed in, we need to copy it as
ProjectedFS will deallocate it when the EdenFS callback returns, but since we
intend to complete these asynchronously, the callback data needs to outlive the
callback, hence the copy. It's likely that this is copying too much, and only
part of it actually needs to be copied, this will be tackled later.
Reviewed By: wez
Differential Revision: D23505511
fbshipit-source-id: ece00183e3194611d3d63465878470d6e53b790c
Summary: A following diff will make use of the channel.
Reviewed By: wez
Differential Revision: D23505510
fbshipit-source-id: c044fff51c8771b1ead86333317e5c617184075c
Summary:
Currently getbundle implementation works this way: it accepts two list of
changesets, a list of `heads` and a list of `common`. getbundle needs to return
enough changesets to the client so that client has `heads` and all of its
ancestors. `common` is a hint to the server that tells which nodes client
already has and doesn't need sending.
Current getbundle implementation is inefficient when any of the heads have a low
generation number but also we have a lot excludes which have a high generation
number. Current implementation needs to find ancestors of excludes that are
equal or lower than generation of this head with a low generation number.
This diff is hacky attempt to improve it. It does it by splitting the heads
into heads with high generation numbers and heads with low generation numbers,
sending separate getbundle requests for each of them and then combining the
results.
Example
```
O <- head 1
|
O <- exclude 1
|
... O <- head 2 <- low generation number
| |
O O <- exclude 2
```
If we have a request like `(heads: [head1, head2], excludes: [exclude1, exclude2])` then this optimization splits it into two requests: `(heads: [head1], excludes: [exclude1, exclude2] )` and `(heads:[head2], excludes:[exclude2])`, and then combines the results. Note that it might result in overfetching i.e. returning much more commits to the client then it requested. This might make a request much slower, and to combat that I suggest to have a tunable getbundle_low_gen_optimization_max_traversal_limit that prevents overfetching.
Reviewed By: krallin
Differential Revision: D23599866
fbshipit-source-id: fcd773eb6a0fb4e8d2128b819f7a13595aca65fa
Summary:
This completely converts mercurial changeset to be an instance of derived data:
- Custom lease logic is removed
- Custom changeset traversal logic is removed
Naming scheme of keys for leases has been changed to conform with other derived data types. This might cause temporary spike of cpu usage during rollout.
Reviewed By: farnz
Differential Revision: D23575777
fbshipit-source-id: 8eb878b2b0a57312c69f865f4c5395d98df7141c
Summary:
shed/sql library used mainly to communicate with Mysql db and to have a nice abstraction layer around mysql (which is used in production) and sqlite (integration tests). The library provided an interface, that was backed up from Mysql side my raw connections and by MyRouter.
This diff introduces a new backend - new Mysql client for Rust.
New backend is exposed as a third variant for the current model: sqlite, mysql (raw conn and myrouter) and mysql2 (new client). The main reason for that is the fact that the current shed/sql interface for Mysql
(1) heavily depends on mysql_async crate, (2) introduces much more complexity than needed for the new client and (3) it seems like this will be refactored and cleaned up later, old things will be deprecated.
So to not overcomplicate things by trying to implement the given interface for the new Mysql client, I tried to simplify things by adding it as a third backend option.
Reviewed By: farnz
Differential Revision: D22458189
fbshipit-source-id: 4a484b5201a38cc017023c4086e9f57544de68b8
Summary:
Pull Request resolved: https://github.com/facebookexperimental/eden/pull/56
Additionally to adding a flag as in the commit title the "test-scs*" tests are being excluded via pattern rather than listed out. This will help in future when more tests using scs_server will be added as the author won't have to list it in exclusion list.
Reviewed By: farnz
Differential Revision: D23647298
fbshipit-source-id: f5c263b9f68c59f4abf9f672c7fe73b63fa74102
Summary:
lookup has special support for requests that start with _gitlookup. It remaps
hg commit to git commit and vice versa. Let's support that in mononoke as well.
Reviewed By: krallin
Differential Revision: D23646778
fbshipit-source-id: fcde58500b5956201e718b0a609fc3fee1bbdd28
Summary:
`hg cloud rejoin` is used in fbclone
By providing a bit more information about the workspaces available we can improve user
experience and try to eliminate the confusion multiple workspaces cause.
Reviewed By: mitrandir77
Differential Revision: D23623063
fbshipit-source-id: 7598c1b58597032c9cfcef0b44b0ec1b00510ffa
Summary: This diff refactors the tool to use changeset_ids instead of changesets, since the functionalities of the tool always use the ids over the changesets. For process recovery (next steps) it's more ideal to save the changeset_ids, since they are simpler to store. However, if we used changesets in the tool, we would need to load them with the saved ids just to only use the ids throughout the process. Therefore, this loading step becomes redundant and we should simply use the ids.
Reviewed By: krallin
Differential Revision: D23624639
fbshipit-source-id: d9b558ebb46c0670bd09556783060f12d3a279ed
Summary: Add a Scuba handler for the EdenAPI server, which will allow the server to log custom columns to Scuba via `ScubaMiddleware`. For now, the only application specific columns are `repo` and `method`.
Reviewed By: krallin
Differential Revision: D23619437
fbshipit-source-id: f08aaf9c84657b4d92f1a1cfe5f0cb5ccd408e5e
Summary: Add `OdsMiddleware` to the EdenAPI server to log various aggregate request stats to ODS. This middleware is directly based on the `OdsMiddleware` from the LFS server (which unfortunately can't be easily generalized due to the way in which the `stats` crate works).
Reviewed By: krallin
Differential Revision: D23619075
fbshipit-source-id: d361c73d18e0d1cb57347fd24c43bdb68fb7819d
Summary:
Add a new `HandlerInfo` struct that can be inserted into the request's `State` by each handler to provide information about the handler to post-request middleware. (This includes things like which handler ran, which repo was queried, etc.)
A note on design: I did consider adding these fields to `RequestContext` (similar to how it's done in the LFS server), but that proved somewhat problematic in that it would require using interior mutability to avoid forcing each handler to borrow the `RequestContext` mutably (and therefore prevent it from borrowing other state data). The current approach seems preferable to adding a `Mutex` inside of `RequestContext`.
Reviewed By: krallin
Differential Revision: D23619077
fbshipit-source-id: 911806b60126d41e2132d1ca62ba863c929d4dc9
Summary:
The corpus rev that biggrep has indexed may not be available in the
local client. Later on in the function it will pull that revision, but earlier
in the function the new logic I added a few weeks ago is just crashing.
That logic was trying to diff against the earlier revision, but that's pretty
arbitrary. Let's just diff against one of the revs at random
(deterministically) and get rid of the need for the hash to exist in the repo
early in the command.
Reviewed By: sfilipco
Differential Revision: D23635801
fbshipit-source-id: 1c284d710b8df9539a696e900183bc10d5d71869
Summary:
The AccessCount fields were recently renamed to start with fsChannel instead of
fuse to be more platform independant. This broke edenfsctl due to it using the
old names.
Reviewed By: wez
Differential Revision: D23633574
fbshipit-source-id: 2a5fc73c47d2f0a6db407ecfeaf85992b7932c10
Summary:
Readdir tries to be smart and prefetch the metadata for each of the children.
But this uses the old path to read metadata for files which can cause eden to
download the blob. When metadata prefetching is turned on in the backing store
it is better to leave this to metadata prefetching to the backing store.
Reviewed By: wez
Differential Revision: D23476876
fbshipit-source-id: 41cc5e6f423f19adb18581564c069c12621b6c1b
Summary:
This diff fixes a bug on Windows when the redirection target is a non-empty directory. As seen in P141872812
This doesn't make the exception go away but generate a more meaningful error message so the user can act on it.
Reviewed By: xavierd
Differential Revision: D23605233
fbshipit-source-id: 2d2bde0e9cd94323a6537ebcec29a4c15868806d
Summary: Once we start to move the bookmark for the large repo commits, small repo commits should also start to appear for the dependent systems (e.g. Phabricator) through back-syncing. This diff adds this functionality to see if the commits have been recognised by the tools.
Reviewed By: StanislavGlebik
Differential Revision: D23566994
fbshipit-source-id: 2f6f3b9099bb864fec6a488064abe1abe7f06813
Summary: Useful when looking into blobstore corruption - you can compare all the blobstore versions by manual fetchees.
Reviewed By: krallin
Differential Revision: D23604436
fbshipit-source-id: 7b56947b0188536499514bae6615c6e81b9106c3
Summary: Going to add more features, so simplify by asyncifying first
Reviewed By: krallin
Differential Revision: D23604437
fbshipit-source-id: 52b2b372e4d3fbf1d59168c6c11311d9edf4ff0f
Summary: When we're scrubbing blobstores, it's not actually a success state if a scrub fails to write. Report this back to the caller - no-one will usually be scrubbing unless they expect repair writes to succeed, and a failure is a sign that we need to investigate further
Reviewed By: mitrandir77
Differential Revision: D23601541
fbshipit-source-id: d328935af9999c944719a6b863d0c86b28c54f59
Summary:
One test was fixed earlier by switching MacOS to use modern version of bash, the other is fixed here by installing "nmap" and using "ncat" from within it on both linux and mac.
Pull Request resolved: https://github.com/facebookexperimental/eden/pull/50
Reviewed By: krallin
Differential Revision: D23599695
Pulled By: lukaspiatkowski
fbshipit-source-id: e2736cee62e82d1e9da6eaf16ef0f2c65d3d8930
Summary:
Fixes a few issues with Mononoke tests in Python 3.
1. We need to use different APIs to account for the unicode vs bytes difference
for path hash encoding.
2. We need to set the language environment for tests that create utf8 file
paths.
3. We need the redaction message and marker to be bytes. Oddly this test still
fails with jq CLI errors, but it makes it past the original error.
Reviewed By: quark-zju
Differential Revision: D23582976
fbshipit-source-id: 44959903aedc5dc9c492ec09a17b9c8e3bdf9457
