Summary:
Since an EdenFS mount can only be mounted in a single place, we don't need to
support the NLM protocol, thus let's inform Linux to not use it.
Reviewed By: kmancini
Differential Revision: D26771237
fbshipit-source-id: bc534a938ad6348bf10a553c6c4f283f043da4f5
Summary:
The SETATTR RPC allows for changing various attributes of the file, like it's
mode, uid, gid, etc. The one piece of the NFS RFC that isn't implemented is
that NFS allows for a client to pass a ctime to the server that it needs to
check prior to setting the attributes. This is done to avoid concurrent
operations on the file conflicting with each other. This is not implemented for
now as Mercurial appears to not be using it.
Reviewed By: kmancini
Differential Revision: D26760073
fbshipit-source-id: 3474665fcf1b089ef6f7de4a6c45a26ef324240e
Summary:
Fast-track some tricky-to-avoid getxattr requests from the kernel
itself. It would be best to avoid the context switch entirely, but in
lieu of that, at least don't log anything or enter our more expensive
FUSE handling path.
This is follow-up to D24039130 (37f030ba72), which apparently did not completely
eliminate POSIX ACL lookups.
Reviewed By: xavierd
Differential Revision: D26803589
fbshipit-source-id: 18cba8e3ffc45516e6458872e408ed29a562c7a8
Summary:
All the callers of these methods have a real context in hand, let's use it
instead of creating a null one.
Reviewed By: genevievehelsel
Differential Revision: D26699271
fbshipit-source-id: 9fb268c9b3194d7e951e30ab5e90c4f2f0388e81
Summary:
`folly::format` is a problematic API because it returns a `Formatter` object that may store references to arguments as its comment warns:
```
* Formatter class.
*
* Note that this class is tricky, as it keeps *references* to its lvalue
* arguments (while it takes ownership of the temporaries), and it doesn't
* copy the passed-in format string. Thankfully, you can't use this
* directly, you have to use format(...) below.
```
This has negative safety and performance (encourages reuse of the same formatter) implications because contrary to what the comment says you can use the object directly since it's returned from `folly::format`. For example
```
auto f = folly::format(std::string("{}"), 42);
f.str();
```
is a UB with no compile-time diagnostic (only caught by ASAN).
It's also unnecessary because the `Formatter` object is usually converted to string straight away via `str()` or written to an output stream. Reusing the formatter doesn't make much sense either because the expensive part is formatting, not capturing arguments.
This diff deprecates `folly::format` suggesting `fmt::format` as a potential replacement. `fmt::format` doesn't have the above problem because arguments are always in scope. It also has the following advantages:
* Better compile times.
* Compile-time format string checks.
* Compatibility with C++20 `std::format`.
* Better performance, particularly with format string compilation which eliminates format string processing overhead altogether.
* Smaller binary footprint.
Also remove `folly::writeTo` which is no longer used and the `format_nested_fbstrings` benchmark which is identical to `format_nested_strings` since there is no `fbstr()` any more.
Reviewed By: yfeldblum
Differential Revision: D26391489
fbshipit-source-id: f0309e78db0eb6d8c22b426d4cc333a17c53f73e
Summary:
As useful as `eden strace` can be, it's hard to correlate inode
numbers to their filenames via the mkdir/create/lookup request that
returned the inode. Add logging for result codes.
Reviewed By: kmancini
Differential Revision: D26287958
fbshipit-source-id: dae4b56513831185b038546f238938b0d21a6bad
Summary:
The world has moved on utf-8 as the default encoding for files and data, but
EdenFS still accepts non utf-8 filenames to be written to it. In fact, most of
the time when a non utf-8 file is written to the working copy, and even though
EdenFS handles it properly, Mercurial ends up freaking out and crash. In all of
these cases, non-utf8 files were not intentional, and thus refusing to create
them wouldn't be a loss of functionality.
Note that this diff makes the asumption that Mercurial's manifest only accept
utf8 path, and thus we only have to protect against files being created in the
working copy that aren't utf8.
The unfortunate part of this diff is that it makes importing trees a bit more
expensive as testing that a path is utf8 valid is not free.
Reviewed By: chadaustin
Differential Revision: D25442975
fbshipit-source-id: 89341a004272736a61639751da43c2e9c673d5b3
Summary:
We should be consistent about how we render mode_t across all of the
FUSE requests.
Reviewed By: singhsrb
Differential Revision: D26286527
fbshipit-source-id: aadf247c0621be79525c4df2da2940c02ee27719
Summary:
By passing the argument to the channel, we can make it so that the NFS code
correctly replies to whether it is case sensitive or not.
Reviewed By: kmancini
Differential Revision: D26500112
fbshipit-source-id: 2988eae403ff3648b50a1a8f0c978be2828ba568
Summary:
By moving the createChannel outside of the EdenMount class, we no longer need
the channelType alias, so let's do it.
Reviewed By: kmancini
Differential Revision: D26500113
fbshipit-source-id: f992ed2aaac5d692d316d06340bf9b219a2d7006
Summary:
Instead of having one "Dispatcher" type that the various backend overload,
let's simply have a per-mount type dispatcher type. The previous model worked
fine when EdenFS supported only one way of mounting a repository, but with NFS
coming, unix platform will support both FUSE and NFS, making the Dispatcher
overload nonsensical.
As a behavioral change, the dispatcher lifetime and ownership is changed a bit.
It used to live for the duration of the EdenMount object, but is now tied to
the channel lifetime, as it is now owned by it.
Reviewed By: kmancini
Differential Revision: D26329477
fbshipit-source-id: 3959b90a4909e3ab0898caa308f54686f59a943c
Summary:
Since mounting EdenFS via NFS requires the same privilege as mounting EdenFS
with FUSE, let's re-use the PrivHelper infrastructure that FUSE already uses.
The macOS bit isn't yet implemented as developing on Linux is for now easier,
the mount args are also overly complicated on macOS and not well documented.
Reviewed By: fanzeyi
Differential Revision: D26255629
fbshipit-source-id: 295261dd40442fe7e0f9439c4f4c25e0d50211a3
Summary: Like `strace`, show the return codes from FUSE requests.
Reviewed By: kmancini
Differential Revision: D26033195
fbshipit-source-id: 2347129ce480e50a3b0f588937e535e0df45dfbd
Summary:
Correct an oversight that did not show the name of the created file in
a FUSE_CREATE request.
Reviewed By: kmancini
Differential Revision: D26033166
fbshipit-source-id: 35eb8a844aff29b519318109c918f2a700692835
Summary:
Some linkers (lld being one) use fallocate() or posix_fallocate() on
the output file before writing its contents. EdenFS would return
ENOSYS or ENOTSUP so glibc would fall back and write a single byte to
every 512 byte block, which is terribly slow and generates a bunch of
fake traffic in the Watchman journal.
This diff implements basic support for FUSE_FALLOCATE, avoiding this
slow emulation.
Reviewed By: xavierd
Differential Revision: D25934694
fbshipit-source-id: c6c90ea2b517d4dbedce29d9a4340870c8c177c3
Summary: Starting from 3.11.1, OSXFUSE switched into using macOS's major version number for different system versions. So we need to consider that when calculating path to the kernel extensions on macOS.
Reviewed By: xavierd
Differential Revision: D25675984
fbshipit-source-id: ea8c76ce7204ba5da3ca98ceca2cfbeb9c84fa8f
Summary:
This commit adds a new eden configuration option that
controls whether we try to load our edenfs.kext in preference to
an alternative fuse implementation on macOS.
The majority of this diff is plumbing to convey the configuration
value through to the privhelper, which is relatively restrictive
due to its root-ness.
I've also updated watchman and mercurial to be aware of the new
filesystem type that shows up in the mount table.
Reviewed By: genevievehelsel
Differential Revision: D25065462
fbshipit-source-id: 4f35b9440654298e2706a0d0613d97eb63451999
Summary:
osxfuse is rebranding as macfuse in 4.x.
That has ripple effects through how the filesystem is mounted and shows up in
the system.
This commit adjusts for the new opaque and undocumented mount procedure and
speculatively updates a couple of other code locations that were sensitive to
looking for "osxfuse" when examining filesystems.
Reviewed By: genevievehelsel
Differential Revision: D24769826
fbshipit-source-id: dab81256a31702587b0683079806558e891bd1d2
Summary:
The EdenFS codebase uses folly/logging/xlog to log, but we were still relying
on glog for the various CHECK macros. Since xlog also contains equivalent CHECK
macros, let's just rely on them instead.
This is mostly codemodded + arc lint + various fixes to get it compile.
Reviewed By: chadaustin
Differential Revision: D24871174
fbshipit-source-id: 4d2a691df235d6dbd0fbd8f7c19d5a956e86b31c
Summary:
We had some unnamed threads that made profiling performance on macOS a
bit harder. Give them a semblance of a useful name, at least.
Reviewed By: kmancini
Differential Revision: D24640223
fbshipit-source-id: 7dd74b30a081753006df681bf97ac96147b1896c
Summary:
Sometimes you only want to trace writes into the EdenFS checkout, so
add the ability to run `eden strace` with `--reads`, `--writes`, or
both in order to see only those events.
Reviewed By: wez
Differential Revision: D24468539
fbshipit-source-id: a1b3c730987cf86ce3d39952c6a5e9c5edaddfc2
Summary:
FUSE_ACCESS was incorrectly categorized as an 'other' FUSE request
type. Categorize it as 'read'.
Reviewed By: wez
Differential Revision: D24437458
fbshipit-source-id: 5ea0a7d5a0fd57b9230f37674fd2568a4917bab5
Summary:
I noticed while playing with `eden strace` that, on my Linux machine,
running `ls -lA` would cause a pile of FUSE_GETXATTR requests for
`system.posix_acl_access` and `system.posix_acl_default`.
FUSE does not cache getxattr yet, but the kernel can cache ACLs if we
pretend to support them. Thus, advertise FUSE_POSIX_ACL support, but
continue returning ENOSYS for all setxattr calls.
Reviewed By: wez
Differential Revision: D24039130
fbshipit-source-id: a2e69c43b728fb51fb1d1b41ca9d31eba8efaa19
Summary:
Include opcode name and process name in eden.FuseCall Thrift structs
so we can use eden.FuseCall in a later diff in the stack.
Reviewed By: kmancini
Differential Revision: D24036420
fbshipit-source-id: fc6d8f3d174b85e07fac299a6f86b2b2d24f301d
Summary:
Provide an API that allows a FuseChannel TraceBus subscriber to
request population of rendered argument strings. This is not on by
default to avoid the cost in the common case that there are no
subscribers that use the field.
Reviewed By: kmancini
Differential Revision: D24036018
fbshipit-source-id: ff12252de3777606360efc75814149931637aa9b
Summary:
Provide a standard argument renderer per HandlerEntry so it can be
used outside of the strace logger.
Reviewed By: kmancini
Differential Revision: D24035882
fbshipit-source-id: f3f1d918b1a7c937777e2c5fc0de981b5ca3286c
Summary:
Instead of logging in the Dispatcher, move strace logging to
FuseChannel where it can be standardized for all FUSE request types.
Reviewed By: wez
Differential Revision: D24035838
fbshipit-source-id: c84d8c27b62f9944e2d26a35a7ed7bbbeeb5bf0e
Summary:
GNU `df` (and any other coreutil that relies on gnulib's ME_REMOTE
flag) detects remote filesystems with some heuristics. One of which is
whether the device type contains a colon. Since edenfs is a remote
filesystem, include a colon, so it's properly detected as such.
Reviewed By: genevievehelsel
Differential Revision: D24174015
fbshipit-source-id: 4b1f2f49c6eee6e690a6f570924274060a66eee7
Summary:
These don't compile on Windows, and in order to get mode/win to compile, we
need to avoid compiling their contents. Ideally, we could do that in the
TARGETS files with the select statement, but that's not available in fbcode.
Thus, we do the next best thing: ifdef the file entirely.
Reviewed By: wez
Differential Revision: D23871728
fbshipit-source-id: b4d9df6503eaa008e649afd7bdc665cd37a9585d
Summary:
on macOS we cannot safely use `fork`.
This commit replaces the use of `fork` in the startup logger subsystem.
This was a little tricky to untangle; originally (prior to any of
the `fork` removal efforts in this diff stack), the startup flow was
to spawn a set of processes via fork:
```
edenfs (setuid)
\-----edenfs (privhelper, as root)
\------edenfs (daemonized)
```
The forked children take advantage of being able to implicitly pass state to
the child processes from the parent. That data flow needs to become explicit
when removing the fork which makes some things a little awkward.
With fork removed:
* `edenfs` unconditionally spawns `edenfs_privhelper` while it has
root privs and before most of the process has been initialized.
* That same `edenfs` process will then spawn a child `edenfs`
process which starts from scratch, but that which needs to
run as the real server instance
* The original `edenfs` instance needs to linger for a while
to remain connected to the controlling tty to pass back the
startup state to the user, before terminating.
This commit deletes the check that `edenfs` is started originally
as root; previously the logic relied on the forked startup logger
continuing past the `daemonizeIfRequested` call and simply deferring
the check until after folly::init. With these changes we can't
easily perform such a check without adding some extra gymnastics
to pass the state around; the place where that is checked is in
the spawned child of the original edenfs, which is not a privileged
process and doesn't know the original euid. I don't believe this
to be a great loss as we tuck `edenfs` away under the libexec dir.
Reviewed By: chadaustin
Differential Revision: D23696569
fbshipit-source-id: 55b95daf022601a4699274d696af419f0a11f6f2
Summary:
On macOS we cannot safely use `fork` to spawn processes while other threads may initialize objc classes.
This commit replaces the use of `fork` in the privhelper startup with
`SpawnedProcess` instead. We need to take care with this as we are generally
installed setuid root and we'd like to avoid being tricked into running an
arbitrary child process as root.
This commit defines a separate executable called `edenfs_privhelper` that
contains just the privhelper server code.
We need to be careful about locating this executable; to avoid invoking an
arbitrary process while we have root privileges we require that the privhelper
be a sibling to the edenfs executable and carry out some additional ownership
verification so that we can tell that the owner of edenfs also controls
edenfs_privhelper.
To facilitate this, I've added an `executablePath` function to PathFuncs; it
returns the path to the current executable image.
To make the integration test scenario simpler, I've added the edenfs_executable
binary definition alongside that of the edenfs binary in the buck and cmake
build systems. This causes the binaries to be siblings in-situ in the build
tree and avoids the need to move things into place in the test harness.
Reviewed By: chadaustin
Differential Revision: D23653343
fbshipit-source-id: 3c2539a5e0e11cee88960db49c885ce0366d314e
Summary:
This diff teaches the CheckoutConfig how to determine
whether a given checkout should be case-sensitive (the default)
or case-insensitive-case-preserving.
This option is passed through to the fuse channel initialization,
so that the kernel will respect it, however, our DirEntry layer
doesn't yet know that it should respect this.
There's currently no UI to set this option. My game plan
is to suggest the following steps to folks that want to try
this out:
```
$ eden stop
$ vim ~/local/.eden/clients/ovrsource/config.toml
```
and then add this line to the `[repository]` section:
```
case-sensitive = false
```
and finally:
```
$ eden start
```
Reviewed By: xavierd
Differential Revision: D23751184
fbshipit-source-id: 6facb23c460cfff6e37d0091b51b97ab06f62c91
Summary:
Original commit changeset: f4816b303e19
The newer version of Watchman that understands the new mount type name
won't release in time, so back this out for now.
Reviewed By: fanzeyi
Differential Revision: D23720167
fbshipit-source-id: 588541e1d9093533611d1f32b319d2562318506a
Summary:
GNU `df` (and any other coreutil that relies on gnulib's ME_REMOTE
flag) detects remote filesystems with some heuristics. One of which is
whether the device type contains a colon. Since edenfs is a remote
filesystem, include a colon, so it's properly detected as such.
Reviewed By: genevievehelsel
Differential Revision: D23520233
fbshipit-source-id: f4816b303e198d4e2a446efdcc5b49a593e09a05
Summary:
Most of the RequestData code is platform generic, but bits of it are currently
strongly tied to FUSE. By splitting these 2 parts, we will be able to use the
RequestContext class in Windows too and not having to re-implement all the
logic there.
Reviewed By: chadaustin
Differential Revision: D23482072
fbshipit-source-id: 857fd9ca4264d0f308ec10cc487e9ff3eeb5ee16
Summary:
Removing Fuse from the enum name makes it non tied to Fuse and thus makes it
more portable. This also eliminates the last platform specific bit from
RequestData.
Reviewed By: chadaustin
Differential Revision: D23467773
fbshipit-source-id: 52515522c8ac51d0c4b56dc5e42d4b6593df6623
Summary:
This helps make RequestData slightly more generic by depending less on Fuse
specific types/code.
Reviewed By: chadaustin
Differential Revision: D23467487
fbshipit-source-id: 830f8269e2114c2968dcc49d3b5574e687191e4d
Summary:
This is unused and is sadly Fuse specific, making it harder to move to inodes/,
thus, let's remove it.
Reviewed By: chadaustin
Differential Revision: D23467486
fbshipit-source-id: c34e1abe245cfb79b9414002fd1055b8c0a5f1d3
Summary:
None of these were used, let's remove them.
ps: I thought we had a system to detect unused headers and lint about them?
Reviewed By: chadaustin
Differential Revision: D23465783
fbshipit-source-id: c21a34c9838db29f4fd0057d3be4e0fcb527cd6d
Summary:
This is not per-se fuse related, thus move it to a common location and remove
the duplicated define in FileInode.h
Reviewed By: chadaustin
Differential Revision: D23465192
fbshipit-source-id: 5fa7709f127c2d3372ee5ea3aeb89e793ea5b9f7
Summary: This file is not fuse specific, therefore, let's move it to a non-fuse folder.
Reviewed By: chadaustin
Differential Revision: D23464460
fbshipit-source-id: f70e94bb0ecc37bd74798fd230dee2058484f31b
Summary:
In the code base, "channel" is used to denote the OS mechanism that sends
EdenFS notifications. In macOS and Linux, that's Fuse, on Windows, that's
ProjectedFS. To avoid platform specific naming in ObjectedFetchContext, let's
rename the fetch cause enum.
Reviewed By: kmancini
Differential Revision: D23462460
fbshipit-source-id: 3ac68cdf4999e6a3b4ff4ee266f94e1f9736df39
Summary:
This commit introduces a new process spawning class derived
from the ChildProcess class in the watchman codebase.
`SpawnedProcess` is similar to folly::Subprocess but is designed around the
idea that we will use a system provided spawning API to start a process, rather
than assuming the use of `fork`.
`fork` is to be avoided because it can be expensive for processes with large
address spaces and also because it interacts poorly with threads on macOS. In
particular, we see the objC runtime terminating our process in some scenarios
where fork and threads are mixed.
There are some important differences from `folly::Subprocess` and that means
that some assumptions and uses need to be altered slightly from their prior
workings. For example, detaching a SpawnedProcess moves the responsibility of
waiting on the child to a periodic task as there is no way to detach via
posix_spawn without also using fork.
On the plus side, this commit allows unifying spawning between posix and
windows systems, which simplifies the code!
Reviewed By: xavierd
Differential Revision: D23287763
fbshipit-source-id: b662af1d7eaaa9ed445c42f6c5765ae9af975eea
Summary:
I'm not sure if that's the right thing to do, so I'd like feedback on that.
Check for opcode was added in D22050919 (fdb1af8bc9), but there was also a comment from
chadaustin
```
Why do you think fuseHeader isn't staying valid? It's a struct that's copied
into the RequestData. It does look like stealReq() sets the opcode to 0, but I
don't see anything that would affect the pid field.
```
So I wonder if this code can be responsible for some of the blind spots in
logging?
Reviewed By: chadaustin
Differential Revision: D23422635
fbshipit-source-id: 4d9a7171d685eb3f6f69da7b8a191df2f65ad897
Summary:
Setting up, tearing down, and querying RequestContext has some
overhead that I would like to avoid in the inner FUSE loop, so replace
RequestData with a single class that's heap-allocated at the start of
a request and is guaranteed to survive until the request ends, and is
otherwise explicitly passed where it's needed.
Reviewed By: kmancini
Differential Revision: D22712310
fbshipit-source-id: fc30d0b0f7e22b39306b857194ea07a913110b0f
