Summary: This implements the same logic as the `edenfsctl uptime`.
Reviewed By: fanzeyi
Differential Revision: D26412789
fbshipit-source-id: ebcf5f0b4767025ea210f7e9c69116b79436d5d0
Summary:
In Python, no passed in timeout means a 3s connection timeout, let's do the
same in Rust.
Reviewed By: chadaustin
Differential Revision: D26407991
fbshipit-source-id: ad2919e2cb72e5a113499d7e036ae285ecf9ae34
Summary: You can use `instance.get_config()` to get access to global EdenFS configurations
Reviewed By: chadaustin
Differential Revision: D26407350
fbshipit-source-id: 022cc59fd86b2711c15cfd781872465c6ada9081
Summary:
This diff adds `edenfs-config` for loading EdenFS configurations from various
locations.
Reviewed By: xavierd
Differential Revision: D26391272
fbshipit-source-id: 3d34da98b2d530e13cdd831d3dc204e44304c486
Summary:
**Problem:** EdenFS has the classic hierarchical configuration design. We load from `/etc/eden/edenfs.rc` first, then `/etc/eden/conf.d/*` then `$HOME/.edenrc`. The latter can overwrite the former. At the end we have a complete view of the configurations.
`serde` is great, but it can't give us the information of whether it generated a field from `serde(default)` or from de-serialization. So we can't just deserialize then merge the configuration files. We need that information, and nor serde should give us that functionality.
`stack-config` is created to load configurations with serde. It automatically generates code with the intermediate data structure and taking care of merging of multiple configuration files. It derives a data type based on the original struct but wrap each field with `Option<T>` and mark it with Serde. Then it generates the code to merge and build the final concrete configuration data type.
It does not care what kind of data format the configuration file is, as long as it can deserialize into the generate optional type, it accepts it.
Example, say I have a file with this structure:
```
#[derive(StackConfig)]
struct Config {
path: String,
}
```
Then this crate will generate:
```
mod __stack_config_private {
#[derive(serde::Deserialize)]
pub(super) struct ConfigOpt {
path: Option<String>
}
...
}
struct ConfigLoader {
layers: Vec<__stack_config_private::ConfigOpt>,
}
impl ConfigLoader {
fn new() { ... }
fn load(&mut self, layer: __stack_config_private::ConfigOpt> { ... }
fn build(self) -> Config { ... }
}
```
See `examples/parse.rs` for usage.
Reviewed By: xavierd
Differential Revision: D26377547
fbshipit-source-id: 1e07d9867742913fd76ed4f765160f0035a2f2a3
Summary: This diff sets up debug logging for EdenFS CLI with tracing.
Reviewed By: chadaustin
Differential Revision: D26354205
fbshipit-source-id: bcc89fe3eaf4c7ae7642b8c20fd74fd3ea6dd4ee
Summary:
While I was fixing the cli_test I realized that the rest of the tests didn't
enforce strict type checking, let's make sure it is enabled so `pyre -l eden`
works for all the tests now and in the future.
Reviewed By: chadaustin
Differential Revision: D26356267
fbshipit-source-id: 4f026b6f96c410115a6a38d772f8e06ab977293b
Summary:
The default filesystem on macOS (APFS) is case-insensitive, but EdenFS has so
far been case-sensitive except on Windows. Some of the native tools (Unity for
instance), expect macOS to always be case-insensitive, and is thus breaking due
to that.
The safe behavior would be to have EdenFS behave exactly the same as APFS: be
case insensitive. For now, to avoid breaking users this will be done on new
mounts only, and once fully validated, this will be made the default and forced
on.
Reviewed By: chadaustin
Differential Revision: D26356269
fbshipit-source-id: 96ca331d8c9726213520dff3e3563019d0400a95
Summary:
This diff adds a Rust wrapper that will forward any command it fails to parse
to the companion binary `edenfsctl.real` at the same directory. This will allow
us to rewrite some of the subcommands in Rust while keeps the original
functionality.
With `EDENFSCTL_SKIP_RUST` environment variable set, it will skip the argument
parsing process in case of any emergency situation.
Reviewed By: xavierd
Differential Revision: D25758959
fbshipit-source-id: 0fbc69d0e0733ad2608622417be93dc7db2d6fa6
Summary:
My understanding of C++ not being perfect, I thought that these 2 fields would
be default initialized, but it turns out this is wrong, and the values of these
would be whatever is on the stack at the time a non-initialized variant is
used. The XdrSerialize variant test exercise this case and fails when optimized
due to that, this change fixes it.
Reviewed By: chadaustin, fanzeyi
Differential Revision: D26371695
fbshipit-source-id: 489cac846b51508809783bb577c7a97c4e6c060a
Summary:
Instead of having one "Dispatcher" type that the various backend overload,
let's simply have a per-mount type dispatcher type. The previous model worked
fine when EdenFS supported only one way of mounting a repository, but with NFS
coming, unix platform will support both FUSE and NFS, making the Dispatcher
overload nonsensical.
As a behavioral change, the dispatcher lifetime and ownership is changed a bit.
It used to live for the duration of the EdenMount object, but is now tied to
the channel lifetime, as it is now owned by it.
Reviewed By: kmancini
Differential Revision: D26329477
fbshipit-source-id: 3959b90a4909e3ab0898caa308f54686f59a943c
Summary:
As I'm modifying the definition for CheckoutConfig, I'm heavily relying on
`pyre` to tell me about the places I need to fix, but this test not being type
checked means that pyre misses it.
Reviewed By: chadaustin
Differential Revision: D26356268
fbshipit-source-id: 2139c0053b182e656074de3117ee1950e519f19a
Summary: The PATHCONF call gather various POSIX attribute for the passed in file-handle.
Reviewed By: kmancini
Differential Revision: D26270239
fbshipit-source-id: 96a46c58402ce169cacc789c1df3f05f969dcd33
Summary:
The FSINFO procedure is used to collect information about this mount point and
its limits. For now, I've just used random values for most of these, I'll come
back to it later to figure out the ones that makes sense.
Reviewed By: kmancini
Differential Revision: D26269947
fbshipit-source-id: e5fdd961ea7772926208916de0179721930daeab
Summary: Instead of using a plan DBG7, let's use the strace logger to log the RPC calls.
Reviewed By: kmancini
Differential Revision: D26269946
fbshipit-source-id: e8ab7acc658463b9f0f1d4c420f63b2e0c88f46a
Summary: These will be needed when implementing FSINFO.
Reviewed By: kmancini
Differential Revision: D26268588
fbshipit-source-id: 8160f00e988a088769ad682a6de062f6c0443c73
Summary:
Variant are default constructed with the first variant alternative, which means
that when deserialization would not deserialize anything, the first variant
would be initialized. This would mean that deserialize(serialize(myVariant))
would be different from myVariant, which is unexpected.
Reviewed By: kmancini
Differential Revision: D26268586
fbshipit-source-id: ebfd7f9e473509b2a16acdbb6914e828c67c5005
Summary:
With the PrivHelper now being able to try to NFS mount, we can just plug it in
the EdenMount code to actually attempt to mount EdenFS via NFS.
The mount will expectedly fail due to the FSINFO RPC not being implemented just
yet, but that will be the next step.
Reviewed By: kmancini
Differential Revision: D26266143
fbshipit-source-id: a44ae98af76b55a0c24c89e766c072c1a2e1b4fd
Summary:
This will be the main entry point for everything related to the various NFS
programs. For now, it simply starts an nfsd program when registering a new
mount point and returns the ports that mount(2) should use to connect to it.
Reviewed By: kmancini
Differential Revision: D26204060
fbshipit-source-id: 115d2d206109499d5b49c529ebf1f739fc364c23
Summary:
Using the current EventBase can lead to unexpected behavior when the RpcServer
is created on the wrong thread. For instance, if it is created on a thread
pool, and another future is running on that same thread, the RpcServer won't be
able to accept and service new connections, potentially leading to deadlocks.
To avoid this, EdenFS's EventBase is excplicitely passed in, this will ensure
that the RpcServer is created on the proper thread.
Reviewed By: kmancini
Differential Revision: D26266145
fbshipit-source-id: 23211e3aa200c32d2f6fbbfd9ae6fb307896a873
Summary:
MSVC is broken as it doesn't understand the various macros used to generate
XdrTrait for specific types, and I can't figure out a way to make that work.
Even if I provide dummy version of these, the buck build breaks due to glog
trying to redefine the ERROR symbol that some Microsoft headers contain. For
now, let's just ifdef it out since it's unlikely that we're going to be using
NFS on Windows anytime soon.
Reviewed By: singhsrb
Differential Revision: D26293519
fbshipit-source-id: bbaf325c7d1f1688327708360244797a6d48179e
Summary:
Since mounting EdenFS via NFS requires the same privilege as mounting EdenFS
with FUSE, let's re-use the PrivHelper infrastructure that FUSE already uses.
The macOS bit isn't yet implemented as developing on Linux is for now easier,
the mount args are also overly complicated on macOS and not well documented.
Reviewed By: fanzeyi
Differential Revision: D26255629
fbshipit-source-id: 295261dd40442fe7e0f9439c4f4c25e0d50211a3
Summary: This enables the repository to be mounted via NFS, and not FUSE.
Reviewed By: chadaustin
Differential Revision: D26229827
fbshipit-source-id: 5af5a47ebe5f1dd54df7707bf57d9b7476921f29
Summary:
Accumulate every commit transition in the journal, rather than
treating every merged range of journal deltas as having a "from"
commit and a "to" commit. This makes it possible for Watchman to
accurately report the set of changed files across an ABA situation or
rapid `hg next` and `hg prev` operations.
Reviewed By: genevievehelsel
Differential Revision: D26193478
fbshipit-source-id: 8b54b9d5bcefa1811008a3b6e9c3aa25a69471ca
Summary:
For now, they all pretend to not be available, except the null one which does
nothing as per the RFC.
Reviewed By: genevievehelsel
Differential Revision: D26159846
fbshipit-source-id: 8d0f43f6bacc5c5a93e883e527769cb7a3b6e22b
Summary: That way the NFS procedure won't clash with it.
Reviewed By: genevievehelsel
Differential Revision: D26159845
fbshipit-source-id: 22ce07326f9ec42aa9d44352ae5bb71368337c03
Summary:
For now, this only registers itself against rpcbind and always reply that the
procedure is unavailable. In the future, this will service all the procedures
and forward them to a Dispatcher.
Reviewed By: genevievehelsel
Differential Revision: D26159844
fbshipit-source-id: 21908f1333ed41b3eea3fb5ce19c8e68391df103
Summary:
We do have logs for what we receive, let's do the same for what we are sending
back.
Reviewed By: genevievehelsel
Differential Revision: D26152811
fbshipit-source-id: 8e605f78a8c849f3bd65b70be51617fc058330ff
Summary:
In the NFS spec, the fhandle3 is defined as an opaque byte array, and thus its
size must preceed its content. Let's also move it to an NfsdRpc.h as this type
will be predominantly used by the Nfsd RPC program.
Reviewed By: chadaustin
Differential Revision: D26152812
fbshipit-source-id: 0cc37325078a2c7b58551eaa5177436b21e03838
Summary:
Now that we have the list of available mount points, we can properly reply to
the MNT RPC call.
Reviewed By: kmancini
Differential Revision: D26152810
fbshipit-source-id: f364292ead321689fb20f858aee45efdfbd7287d
Summary:
For a proper NFS server, rpcbind is used to query and collect the port/protocol
of the mountd and nfs programs at mount time. In EdenFS, the mount.nfs command
will be run by EdenFS directly, against itself, and we can manually specify
what the ports are for both the mountd and nfs programs. Thus we can use these
directly to avoid polluting the rpcbind database, and potentially conflicting
with a real NFS server.
This will be particularly useful when testing EdenFS as this allows multiple
tests to run at the same time.
Reviewed By: kmancini
Differential Revision: D26140416
fbshipit-source-id: 6b8aa22e69f580a9d6f6edf775d268809a909118
Summary:
Once everything is plugged in, EdenFS will first register the mount point with
Mountd, and then execute the mount.nfs CLI against itself to mount the
repository with the OS.
For now, this does the first part by simply informing Mountd of a new mount
point during mount.
Reviewed By: kmancini
Differential Revision: D26139990
fbshipit-source-id: e5097b8e90c032c8c7bdd2cd03b69695074d7874
Summary:
When NFS is enabled, we will need to have one mountd RPC program running to
be able to serve the mount requests, let's make sure that we have one by adding
it to the ServerState.
On top of the per-mount protocol setting added previously, I've also added a
global setting to make sure that we're not advertising ourself as an NFS server
before the code is more functional.
Reviewed By: genevievehelsel
Differential Revision: D26139991
fbshipit-source-id: 0d9e02618c931a0401394888ea8f3680974966b5
Summary:
As the NFS bits are integrated into the rest of EdenFS, we need to make sure
the code compiles properly. The only major change was the removal of the
//common dependencies as these aren't available to OSS builds.
Reviewed By: genevievehelsel
Differential Revision: D26181750
fbshipit-source-id: 44f8ff335e543ab282f40fa7f8d28fc19cf4581c
Summary:
On macOS, EdenFS will soon be exposed as an NFS server. To ease the transition,
let's have a per-mount config to decide whether this repository should be
mounted via FUSE or NFS
Reviewed By: genevievehelsel
Differential Revision: D26139992
fbshipit-source-id: d28b0e2327c1101f3555ab0bbc9a5b03d82fd7ff
Summary:
The portmap protocol is limited to IPv4, while the rpcbind not only does
support IPv4 and IPv6, but also unix sockets. While we still bind only to an
ipv4, this should allow us to support IPv6 in the future.
Reviewed By: kmancini
Differential Revision: D26114218
fbshipit-source-id: 46e4dfe60a9c1ddbccbb586bf4e3d24012fd26ee
Summary:
As I'm going through the various NFS specs, it becomes clear that variant are
widely used, and the macros to generate them is awkward and making working with
them harder than it should be. Instead writing their deserialization would be
more flexible.
While doing this, I also revamped how serialization/deserialization is done by
using a trait that contains 2 static function: serialize and deserialize, new
types are expected to specialize the XdrTrait with their own
serialization/deserialization.
Reviewed By: kmancini
Differential Revision: D26160026
fbshipit-source-id: 951bdd20620cc5715e780e99b64e15397688570d
Summary: `hg gc` is deprecated and does nothing anymore. Lets just remove this section talking about running `hg gc`
Reviewed By: chadaustin
Differential Revision: D26178391
fbshipit-source-id: 2bb16bbac38465a524ed50454d548564324d35bf
Summary:
The folly::async bits are a bit complicated to get right in particular around
its lifetime management. The solution that is used in our codebase is to make
use of a folly::DelayedDestruction and the DestructorGuard. As long as a
DestructorGuard is alive, the object cannot be destroyed, and thus we can
simply hold a guard in the Reader, and make sure that a guard is held during
the future callbacks.
The only thing to be aware of is that the DestructorGuard is not thread safe,
but since all the callbacks and futures are running in the context of the
single threaded event base, this is not an issue.
Reviewed By: chadaustin
Differential Revision: D26105288
fbshipit-source-id: 4c7b91eca2bc1cae66d8d1724931b20340740574
Summary:
When the RPC server starts, it registers itself against the global rpcbind
daemon to allow RPC clients to reach it. However, it never unregisters itself
when it is destructed, leading to the rpcbind daemon having outdated
information. It also prevents a future RPC server for the same program and
version to register itself.
To fix this, we can simply unregister on destruction. The MountdUtil binary is
also fixed to be able to interrupt it with ctrl+c.
Reviewed By: chadaustin
Differential Revision: D26105289
fbshipit-source-id: c9c3d552ec04a78025d6ff715a425f9ec3aa1328
Summary:
In NFSv3, mounting is not part of the NFS program but is done in a separate
program. This adds the basic scaffolding to reply to the NULL procedure for the
mount protocol. Later, this will be enhanced by having `eden mount` first
register a mount point to Mountd, then trigger a `mount.nfs` against localhost
where Mountd will be able to reply properly.
Similarly to PortMapUtil, I've added a small binary that allows for a "mountd"
process to be spawned. Note that since the code doesn't yet answer properly to
the mount procedure, nfusr closes the connection and retries, causing mountd to
crash due to RpcServer being unsafe on error, this will be fixed separately.
Reviewed By: chadaustin
Differential Revision: D26033504
fbshipit-source-id: 27f90d9072a93460a3a383aadde38e50801e3e87
