Summary: It's useful to know the max number of connections to a shard that it is sensible to open. Used in next diff in stack.
Differential Revision: D26543419
fbshipit-source-id: 57e7c3295a5b5db1572f26954ae0dfb04c84b374
Summary:
The walker mostly checks for duplicates before emitting a new edge, at the same time recording the edge as visited to prevent duplicate edges.
However for derived data where the node may or may not be present, the node isn't considered visited until the node data is successfully loaded and seen in state.rs record_resolved_visit().
In such cases multiple copies of a node could be enqueued, and then we need to run each one.
With this change, where the walker can detect that such a step has completed previously, it will now short circuit the step and return None.
Differential Revision: D26369917
fbshipit-source-id: c2bdbbabfaa80dbb7cc7d2bc25a17230531ae111
Summary:
LOOKUP is the next RPC that EdenFS needs to implement, let's start by adding
the RPC types.
Reviewed By: kmancini
Differential Revision: D26562919
fbshipit-source-id: 3a4ce1a039b6405df9ff94757963aa46acf56d92
Summary:
The ACCESS RPC merely tries to see if a client can perform some action on a
file/directory. Until we have a better story around checking the RPC
credentials and passing it around, let's just return the requested access
rights as-is.
Reviewed By: kmancini
Differential Revision: D26562918
fbshipit-source-id: 468e038cc063c289b5554f3faa6a7f99be2731e4
Summary: This merely add the types necessary to implement the ACCESS RPC
Reviewed By: kmancini
Differential Revision: D26562920
fbshipit-source-id: 2f3238e723e97a54896fabb026adf9b92ab4fcd3
Summary:
Take 2 of fixing the lifetime issues of the RpcServer, this time, the
RpcAcceptCallback needs to live for longer than RpcServer itself. This is due
to the acceptStopped callback being called after RpcServer is being destroyed.
Since the RpcServer needs to be destroyed in the EventBase thread, the
acceptStopped callback is delayed, causing it to be invoked after the RpcServer
has freed its memory.
To solve this, we simply need to make the RpcAcceptCallback a delayed
destructor, and hold a reference to itself that goes away once the
acceptStopped callback is called.
Reviewed By: kmancini
Differential Revision: D26556644
fbshipit-source-id: 32cd80f1664649c1ad96f88c4eec6dd2ed6e8c12
Summary:
Now that the privhelper has the necessary code to unmount an NFS mount, let's
do it. With this in place, this enables `eden mount` and `eden unmount` for NFS
mounts. Unfortunately, it appears that during unmount, releasing the Nfsd3
object triggers a use-after-free in the rpc code
Reviewed By: kmancini
Differential Revision: D26500109
fbshipit-source-id: a210761f818b28b1eb0044f7314a0e2b9017848c
Summary:
This is merely fixing a typo that I made previous where I reused the prjfs
request timeout instead of adding a new one.
Reviewed By: kmancini
Differential Revision: D26500110
fbshipit-source-id: b9bf0cbc0d74866cdc2471f126751d6d8e514e21
Summary:
Now that `mount(2)` complete, we can start plumbing the rest of the mount code
to accomodate for NFS. Trying to find a common ground between Prjfs, FUSE and
Prjfs is harder than I wish it would be and thus I wasn't able to find a
satisfatory solution. For now, I went with a std::variant that stores either a
FuseChannel, or the Nfsd3. In the future once Nfs is stabilized and we have a
good understanding of how it differs (and where it doesn't), EdenMount would
probably need a good refactoring.
Reviewed By: kmancini
Differential Revision: D26500111
fbshipit-source-id: f02a2eaf8890261f150d7cdd2cdfd134aa62c2aa
Summary:
Adding a new configuration that instantiates SegmentedChangelog by downloading
a dag from a prebuilt blob. It then updates in process.
Reviewed By: krallin
Differential Revision: D26508428
fbshipit-source-id: 09166a3c6de499d8813a29afafd4dfe19a19a2a5
Summary:
I am not sure if this too abstract. It might be. This however has separation of
concerns :)
The goal here is to end up with an in memory IdMap that we write to and read
from first. For things that are not found in the in memory IdMap we fall back
to the SqlIdMap. We'll end up with something like:
`OverlayIdMap(ConcurrentMemIdMap, SqlIdMap)`
Reviewed By: quark-zju
Differential Revision: D26417642
fbshipit-source-id: b2b310306db4dc9fc3427bbf50b19366160882a9
Summary:
The `MemIdMap` is not a valid `IdMap` implementation because it takes `&mut
self` when doing inserts. Wrapping the IdMap in a `RwLock` allows us to
implement the `IdMap` trait.
Reviewed By: krallin
Differential Revision: D26417643
fbshipit-source-id: cb5e3513841fa1dd7c8b8004ce7b2fe1467983d7
Summary:
The on demand update code we have is the most basic logic that we could have.
The main problem is that it has long and redundant write locks. This change
reduces the write lock strictly to the section that has to update the in memory
IdDag.
Updating the Dag has 3 phases:
* loading the data that is required for the update;
* updating the IdMap;
* updating the IdDag;
The Dag can function well for serving requests as long as the commits involved
have been built so we want to have easy read access to both the IdMap and the
IdDag. The IdMap is a very simple structure and because it's described as an
Arc<dyn IdMap> we push the update locking logic to the storage. The IdDag is a
complicated structure that we ask to update itself. Those functions take
mutable references. Updating the storage of the iddag to hide the complexities
of locking is more difficult. We deal with the IdDag directly by wrapping it in
a RwLock. The RwLock allows for easy read access which we expect to be the
predominant access pattern.
Updates to the dag are not completely stable so racing updates can have
conflicting results. In case of conflics one of the update processes would have
to restart. It's easier to reason about the process if we just allow one
"thread" to start an update process. The update process is locked by a sync
mutex. The "threads" that fail the race to update are asked to wait until the
ongoing update is complete. The waiters will poll on a shared future that
tracks the ongoing dag update. After the update is complete the waiters will go
back to checking if the data they have is available in the dag. It is possible
that the dag is updated in between determining that the an update is needed and
acquiring the ongoing_update lock. This is fine because the update building
process checks the state of dag before the dag and updates only what is
necessary if necessary.
Reviewed By: krallin
Differential Revision: D26508430
fbshipit-source-id: cd3bceed7e0ffb00aee64433816b5a23c0508d3c
Summary:
This structure is going to be useful to implement the SegmentedChangelog
functionlity for the OnDemandDag as we move forward with separate objects for
the iddag and the idmap rather than a direct dependency on a Dag object.
Reviewed By: quark-zju
Differential Revision: D26508429
fbshipit-source-id: 9116f1c82d301e8e5b726966abd2add2e32765d6
Summary:
Moving all update logic to `dag::update`.
Additional minor changes: removing Dag::build and spliting build_incremental
around the mutable update section of iddag.
Reviewed By: krallin
Differential Revision: D26508427
fbshipit-source-id: 984259d2f199792fcf0635dd3100ec39260fd3ed
Summary:
This is going to be useful when we update the InProcessDag on the server. As
opposed to taking a lock for everything we will be able to take a short lock
to update the IdMap and then a short lock to update the IdDag.
Reviewed By: krallin
Differential Revision: D26417621
fbshipit-source-id: 43f342d384f1be80dcfe721de659ac3ce9dd0e7b
Summary:
Some features in debushell, like `%time` uses `locals()`. They currently cannot
use common variables like `cl` like:
In [1]: %time len(cl)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<timed eval> in <module>
NameError: name 'cl' is not defined
Fix it by updating `locals()`.
In [1]: %time len(cl)
CPU times: user 700 µs, sys: 130 µs, total: 830 µs
Wall time: 1.28 ms
Reviewed By: ikostia
Differential Revision: D26590943
fbshipit-source-id: 2b7fd3274f1ea41f7996edbbdae0af8e103a9c0a
Summary:
The EdenFS import helper's `_fetch_tree_impl` method works by either calling `repo._prefetchtrees` or `repo.prefetchtrees`, both of which are methods from `treemanifest` extension's `treerepository` class. Unfortunately, these methods are lower-level than their names would suggest, and will always perform gettreepack style fetching (see [1]).
If HTTP fetching is enabled, this means that EdenFS will always query EdenAPI's `/complete_trees` endpoint instead of `/trees`, which is needlessly expensive given that EdenFS just wants the exact set of trees specified. As a somewhat hacky workaround, this diff just checks whether HTTP fetching is enabled, and if so, directly calls the appropriate HTTP fetch method.
This solution isn't ideal; it would be better for the import helper to request the trees from Mercurial's `remotetreestore` instead of via methods from `treerepository`. Unfortunately, with the current structure of Mercurial's storage layer, the `remotetreestore` isn't readily accessible because it gets passed into the Rust API's storage hierarchy upon construction.
---
[1]: The various `*prefetchtrees` methods are usually called from Mercurial's `remotetreestore`, which is where the choice of tree fetching strategy is made (i.e., designated nodes vs gettreepack). The `remotetreestore` then calls `*prefetchtrees` for gettreepack-style fetching, or `*getdesignatednodes` for on-demand fetching. As such, a call to `*prefetchtrees` generally implies gettreepack-style fetching.
Reviewed By: quark-zju
Differential Revision: D26560451
fbshipit-source-id: 2eedf50a6e66fac78df77214b777544eb8049714
Summary:
Comments affecting runtime behavior disturbs me, so use explicit
structopt annotations on help text. And move those annotations to the
command implementations.
Reviewed By: fanzeyi
Differential Revision: D26412452
fbshipit-source-id: 066dfdd1c54254bae4bd2af65031487b7a1094da
Summary: clap/structopt adds a -V flag to every subcommand by default. Disable that.
Reviewed By: fanzeyi
Differential Revision: D26412093
fbshipit-source-id: 03a0320fd15444f700b359f5ed0ca8c40b10ae1c
Summary:
I don't want the fallback when testing, so add an environment variable
for bypassing it.
Reviewed By: fanzeyi
Differential Revision: D26411754
fbshipit-source-id: f2aea82bf3e79db11e72ad5f5ce33513cfc2f05b
Summary:
Rather than offloading dealing with unexpected errors to each
subcommand, allow them to propogate out. Subcommands can still be
responsible for handling expected "errors" like EdenFS not running.
Reviewed By: fanzeyi
Differential Revision: D26411186
fbshipit-source-id: 4e1c5fb1d7bed495e3e22cca44d3f84f7f4c7f14
Summary:
We should be consistent about how we render mode_t across all of the
FUSE requests.
Reviewed By: singhsrb
Differential Revision: D26286527
fbshipit-source-id: aadf247c0621be79525c4df2da2940c02ee27719
Summary:
In EdenAPI this is logged as a vector (and in all our other services), but in
Mononoke Server we log it as a string. Let's fix this up. This is worth doing
now since right now we end up logging to 2 columns with the same name and a
different type.
Reviewed By: ahornby
Differential Revision: D26542737
fbshipit-source-id: 2f12c9e475061b1c21c71bade99b83cc070006e8
Summary:
The earlier diffs in this stack have removed all our dependencies on the Tokio
0.1 runtime environment (so, basically, `tokio-executor` and `tokio-timer`), so
we don't need this anymore.
We do still have some deps on `tokio-io`, but this is just traits + helpers,
so this doesn't actually prevent us from removing the 0.1 runtime!
Note that we still have a few transitive dependencies on Tokio 0.1:
- async-unit uses tokio-compat
- hg depends on tokio-compat too, and we depend on it in tests
This isn't the end of the world though, we can live with that :)
Reviewed By: ahornby
Differential Revision: D26544410
fbshipit-source-id: 24789be2402c3f48220dcaad110e8246ef02ecd8
Summary:
This was a thing that was only ever used in Mononoke, and we don't think it's
usable and haven't been using it. Let's get rid of it. As-is, it won't even work
for most people due to its (indirect) dependency on Tokio 0.1.
Reviewed By: StanislavGlebik
Differential Revision: D26512243
fbshipit-source-id: faa16683f2adb20dfba43c4768486b982bc02ff9
Summary:
By passing the argument to the channel, we can make it so that the NFS code
correctly replies to whether it is case sensitive or not.
Reviewed By: kmancini
Differential Revision: D26500112
fbshipit-source-id: 2988eae403ff3648b50a1a8f0c978be2828ba568
Summary: This code has been used to transfer an existing repo to remotenames. After D26460435 (84280e36c3) it doesn't work as designed and should be removed as well.
Reviewed By: quark-zju
Differential Revision: D26544739
fbshipit-source-id: 34c08d4b9997c0b1f298ee1ecd0e8af24f4d8a39
Summary:
Support for unsubscribing from a scratch remote bookmarks in `hg hide` command.
This is support if you hide via a revision. Hiding by its name will be another change.
Reviewed By: quark-zju
Differential Revision: D26544305
fbshipit-source-id: d10372513dda88903e2cc031ff16883a001c8e34
Summary: We don't log this information if a request failed. Let's start doing that.
Reviewed By: farnz
Differential Revision: D26577832
fbshipit-source-id: acfac1c57364eeb457a81ff4bbeddc5407f3a985
Summary: Can remove Clone constraint and no need for Arc with some code tidyups
Reviewed By: farnz
Differential Revision: D26575852
fbshipit-source-id: 0cd31879c6d72cc7242e1c34ef5df07c96ac6e6c
Summary:
The `commit` or `amend` command might not have been the first item in the
command list (e.g. there could have been a custom config option). Expand the
heuristics for detecting the `commit` and `amend` cases to account for this.
Reviewed By: quark-zju
Differential Revision: D26545339
fbshipit-source-id: a5b1fc8ccc87989e742fce1fa79273266892ed79
Summary: Dynamic config only gets applied once the local repo is instantiated, but source peer is created long before that. This causes it to have missing configuration when being instantiated. Apply dynamic config much earlier in case of a clone so that srcpeer can use dynamicconf as well.
Reviewed By: DurhamG
Differential Revision: D26543635
fbshipit-source-id: 88c16d345f5116bfaaf57c593eb09145df81b4fb
