Summary:
Migrate away from streaming Thrift Python, and instead implement `eden
trace hg` in C++, which will eventually let us support Windows.
This also fixes an unexplained crash in `eden trace hg` on macOS.
Reviewed By: xavierd
Differential Revision: D25515377
fbshipit-source-id: 1916aa7f09df37e9dee3479a858932c724f8aed4
Summary: This diff sets up the ability for us to track requests as they are shuffled around in the `std::vector<> queue`. Since the queue is a vector, and since it is sorted everytime a new element is added or removed, we cannot keep track of elements in the queue with indices or references. Instead, we will store the requests in a shared_ptr so we can maintain a pointer to the request no matter where the request is moved around to.
Reviewed By: kmancini
Differential Revision: D26355907
fbshipit-source-id: d714d689963106a4f495221dbcfcbab758ffc7b2
Summary:
NFS clients will act differently depending on what kind of error is returned
from an RPC. A NFS3ERR_NOENT in response to a LOOKUP RPC may for instance
populate a negative lookup cache to avoid re-querying the same file again.
Thus, we need to translate the various errors with their corresponding
nfsstat3 values.
Reviewed By: genevievehelsel
Differential Revision: D26597788
fbshipit-source-id: d22a552196a378d6d7e9cd374e63bb8bcf436ce4
Summary:
It's silly to use `eden prefetch --no-prefetch` to efficiently glob
for filenames. Introduce an `eden glob` command which resolves a glob
relative to the current working directory.
Reviewed By: genevievehelsel
Differential Revision: D25450358
fbshipit-source-id: 45d6dc870d21510e51d5662c75e80385886899fc
Summary:
I want to use EdenError from some code outside of service/, so move
EdenError into utils.
Reviewed By: genevievehelsel
Differential Revision: D25447438
fbshipit-source-id: 2d1ddfa379238369679e84708518a9ba106f76b9
Summary:
As useful as `eden strace` can be, it's hard to correlate inode
numbers to their filenames via the mkdir/create/lookup request that
returned the inode. Add logging for result codes.
Reviewed By: kmancini
Differential Revision: D26287958
fbshipit-source-id: dae4b56513831185b038546f238938b0d21a6bad
Summary:
The world has moved on utf-8 as the default encoding for files and data, but
EdenFS still accepts non utf-8 filenames to be written to it. In fact, most of
the time when a non utf-8 file is written to the working copy, and even though
EdenFS handles it properly, Mercurial ends up freaking out and crash. In all of
these cases, non-utf8 files were not intentional, and thus refusing to create
them wouldn't be a loss of functionality.
Note that this diff makes the asumption that Mercurial's manifest only accept
utf8 path, and thus we only have to protect against files being created in the
working copy that aren't utf8.
The unfortunate part of this diff is that it makes importing trees a bit more
expensive as testing that a path is utf8 valid is not free.
Reviewed By: chadaustin
Differential Revision: D25442975
fbshipit-source-id: 89341a004272736a61639751da43c2e9c673d5b3
Summary:
LOOKUP is the next RPC that EdenFS needs to implement, let's start by adding
the RPC types.
Reviewed By: kmancini
Differential Revision: D26562919
fbshipit-source-id: 3a4ce1a039b6405df9ff94757963aa46acf56d92
Summary:
The ACCESS RPC merely tries to see if a client can perform some action on a
file/directory. Until we have a better story around checking the RPC
credentials and passing it around, let's just return the requested access
rights as-is.
Reviewed By: kmancini
Differential Revision: D26562918
fbshipit-source-id: 468e038cc063c289b5554f3faa6a7f99be2731e4
Summary: This merely add the types necessary to implement the ACCESS RPC
Reviewed By: kmancini
Differential Revision: D26562920
fbshipit-source-id: 2f3238e723e97a54896fabb026adf9b92ab4fcd3
Summary:
Take 2 of fixing the lifetime issues of the RpcServer, this time, the
RpcAcceptCallback needs to live for longer than RpcServer itself. This is due
to the acceptStopped callback being called after RpcServer is being destroyed.
Since the RpcServer needs to be destroyed in the EventBase thread, the
acceptStopped callback is delayed, causing it to be invoked after the RpcServer
has freed its memory.
To solve this, we simply need to make the RpcAcceptCallback a delayed
destructor, and hold a reference to itself that goes away once the
acceptStopped callback is called.
Reviewed By: kmancini
Differential Revision: D26556644
fbshipit-source-id: 32cd80f1664649c1ad96f88c4eec6dd2ed6e8c12
Summary:
Now that the privhelper has the necessary code to unmount an NFS mount, let's
do it. With this in place, this enables `eden mount` and `eden unmount` for NFS
mounts. Unfortunately, it appears that during unmount, releasing the Nfsd3
object triggers a use-after-free in the rpc code
Reviewed By: kmancini
Differential Revision: D26500109
fbshipit-source-id: a210761f818b28b1eb0044f7314a0e2b9017848c
Summary:
This is merely fixing a typo that I made previous where I reused the prjfs
request timeout instead of adding a new one.
Reviewed By: kmancini
Differential Revision: D26500110
fbshipit-source-id: b9bf0cbc0d74866cdc2471f126751d6d8e514e21
Summary:
Now that `mount(2)` complete, we can start plumbing the rest of the mount code
to accomodate for NFS. Trying to find a common ground between Prjfs, FUSE and
Prjfs is harder than I wish it would be and thus I wasn't able to find a
satisfatory solution. For now, I went with a std::variant that stores either a
FuseChannel, or the Nfsd3. In the future once Nfs is stabilized and we have a
good understanding of how it differs (and where it doesn't), EdenMount would
probably need a good refactoring.
Reviewed By: kmancini
Differential Revision: D26500111
fbshipit-source-id: f02a2eaf8890261f150d7cdd2cdfd134aa62c2aa
Summary:
Comments affecting runtime behavior disturbs me, so use explicit
structopt annotations on help text. And move those annotations to the
command implementations.
Reviewed By: fanzeyi
Differential Revision: D26412452
fbshipit-source-id: 066dfdd1c54254bae4bd2af65031487b7a1094da
Summary: clap/structopt adds a -V flag to every subcommand by default. Disable that.
Reviewed By: fanzeyi
Differential Revision: D26412093
fbshipit-source-id: 03a0320fd15444f700b359f5ed0ca8c40b10ae1c
Summary:
I don't want the fallback when testing, so add an environment variable
for bypassing it.
Reviewed By: fanzeyi
Differential Revision: D26411754
fbshipit-source-id: f2aea82bf3e79db11e72ad5f5ce33513cfc2f05b
Summary:
Rather than offloading dealing with unexpected errors to each
subcommand, allow them to propogate out. Subcommands can still be
responsible for handling expected "errors" like EdenFS not running.
Reviewed By: fanzeyi
Differential Revision: D26411186
fbshipit-source-id: 4e1c5fb1d7bed495e3e22cca44d3f84f7f4c7f14
Summary:
We should be consistent about how we render mode_t across all of the
FUSE requests.
Reviewed By: singhsrb
Differential Revision: D26286527
fbshipit-source-id: aadf247c0621be79525c4df2da2940c02ee27719
Summary:
By passing the argument to the channel, we can make it so that the NFS code
correctly replies to whether it is case sensitive or not.
Reviewed By: kmancini
Differential Revision: D26500112
fbshipit-source-id: 2988eae403ff3648b50a1a8f0c978be2828ba568
Summary:
By moving the createChannel outside of the EdenMount class, we no longer need
the channelType alias, so let's do it.
Reviewed By: kmancini
Differential Revision: D26500113
fbshipit-source-id: f992ed2aaac5d692d316d06340bf9b219a2d7006
Summary: The RPC just translate the various `struct stat` fields into the fattr3 ones.
Reviewed By: kmancini
Differential Revision: D26389793
fbshipit-source-id: 48c6e109d5d2cb62cab096114c37314d7833035f
Summary:
Similarly to what is done for FUSE and ProjectedFS, the dispatcher is the glue
that sits in between the protocol specific bits and the inodes layer.
For now, this only implements "getattr" but it will be filled overtime as more
RPC can be answered properly.
Reviewed By: kmancini
Differential Revision: D26389795
fbshipit-source-id: 19cf3457feec2ebc100e632cb28c20b11fdde26d
Summary: This merely adds the various datastructures needed to implement GETATTR.
Reviewed By: kmancini
Differential Revision: D26389794
fbshipit-source-id: bda557a21386483449c18aa9e52f4f626b73e69f
Summary:
Timeseries is memory intensive and not really required in the current context
it is being used.
Reviewed By: chadaustin
Differential Revision: D26315632
fbshipit-source-id: ee51c3ad8bef6fce152aa787c8c4602f0b499f92
Summary: This implements the same logic as the `edenfsctl uptime`.
Reviewed By: fanzeyi
Differential Revision: D26412789
fbshipit-source-id: ebcf5f0b4767025ea210f7e9c69116b79436d5d0
Summary:
In Python, no passed in timeout means a 3s connection timeout, let's do the
same in Rust.
Reviewed By: chadaustin
Differential Revision: D26407991
fbshipit-source-id: ad2919e2cb72e5a113499d7e036ae285ecf9ae34
Summary: You can use `instance.get_config()` to get access to global EdenFS configurations
Reviewed By: chadaustin
Differential Revision: D26407350
fbshipit-source-id: 022cc59fd86b2711c15cfd781872465c6ada9081
Summary:
This diff adds `edenfs-config` for loading EdenFS configurations from various
locations.
Reviewed By: xavierd
Differential Revision: D26391272
fbshipit-source-id: 3d34da98b2d530e13cdd831d3dc204e44304c486
Summary:
**Problem:** EdenFS has the classic hierarchical configuration design. We load from `/etc/eden/edenfs.rc` first, then `/etc/eden/conf.d/*` then `$HOME/.edenrc`. The latter can overwrite the former. At the end we have a complete view of the configurations.
`serde` is great, but it can't give us the information of whether it generated a field from `serde(default)` or from de-serialization. So we can't just deserialize then merge the configuration files. We need that information, and nor serde should give us that functionality.
`stack-config` is created to load configurations with serde. It automatically generates code with the intermediate data structure and taking care of merging of multiple configuration files. It derives a data type based on the original struct but wrap each field with `Option<T>` and mark it with Serde. Then it generates the code to merge and build the final concrete configuration data type.
It does not care what kind of data format the configuration file is, as long as it can deserialize into the generate optional type, it accepts it.
Example, say I have a file with this structure:
```
#[derive(StackConfig)]
struct Config {
path: String,
}
```
Then this crate will generate:
```
mod __stack_config_private {
#[derive(serde::Deserialize)]
pub(super) struct ConfigOpt {
path: Option<String>
}
...
}
struct ConfigLoader {
layers: Vec<__stack_config_private::ConfigOpt>,
}
impl ConfigLoader {
fn new() { ... }
fn load(&mut self, layer: __stack_config_private::ConfigOpt> { ... }
fn build(self) -> Config { ... }
}
```
See `examples/parse.rs` for usage.
Reviewed By: xavierd
Differential Revision: D26377547
fbshipit-source-id: 1e07d9867742913fd76ed4f765160f0035a2f2a3
Summary: This diff sets up debug logging for EdenFS CLI with tracing.
Reviewed By: chadaustin
Differential Revision: D26354205
fbshipit-source-id: bcc89fe3eaf4c7ae7642b8c20fd74fd3ea6dd4ee
Summary:
While I was fixing the cli_test I realized that the rest of the tests didn't
enforce strict type checking, let's make sure it is enabled so `pyre -l eden`
works for all the tests now and in the future.
Reviewed By: chadaustin
Differential Revision: D26356267
fbshipit-source-id: 4f026b6f96c410115a6a38d772f8e06ab977293b
Summary:
The default filesystem on macOS (APFS) is case-insensitive, but EdenFS has so
far been case-sensitive except on Windows. Some of the native tools (Unity for
instance), expect macOS to always be case-insensitive, and is thus breaking due
to that.
The safe behavior would be to have EdenFS behave exactly the same as APFS: be
case insensitive. For now, to avoid breaking users this will be done on new
mounts only, and once fully validated, this will be made the default and forced
on.
Reviewed By: chadaustin
Differential Revision: D26356269
fbshipit-source-id: 96ca331d8c9726213520dff3e3563019d0400a95
