Summary:
Avoid this compilation error:
```
[ 23%] Building CXX object eden/fs/fuse/CMakeFiles/eden_fuse.dir/Dispatcher.cpp.o
In file included from /Users/wez/fbsource/fbcode/eden/oss/eden/fs/fuse/Dispatcher.cpp:10:
In file included from /Users/wez/fbsource/fbcode/eden/oss/eden/fs/fuse/Dispatcher.h:12:
In file included from /Users/wez/fbsource/fbcode/eden/oss/external/install/include/folly/Portability.h:19:
In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/cstddef:110:
/Library/Developer/CommandLineTools/usr/include/c++/v1/type_traits:3141:38: error: incomplete type 'facebook::eden::DirList' used in type trait expression
: public integral_constant<bool, __is_constructible(_Tp, _Args...)>
^
/Users/wez/fbsource/fbcode/eden/oss/external/install/include/folly/futures/Future.h:146:36: note: in instantiation of template class 'std::__1::is_constructible<facebook::eden::DirList>' requested here
typename std::enable_if<std::is_constructible<T, Args&&...>::value, int>::
^
/Users/wez/fbsource/fbcode/eden/oss/external/install/include/folly/futures/Future.h:148:12: note: while substituting prior template arguments into non-type template parameter [with Args = <>]
explicit FutureBase(in_place_t, Args&&... args);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/wez/fbsource/fbcode/eden/oss/external/install/include/folly/futures/Future.h:115:7: note: while substituting deduced template arguments into function template 'FutureBase' [with Args = <>, $1 = (no value)]
class FutureBase {
^
/Users/wez/fbsource/fbcode/eden/oss/eden/fs/fuse/Dispatcher.cpp:175:13: note: in instantiation of template class 'folly::Future<facebook::eden::DirList>' requested here
Dispatcher::readdir(InodeNumber, DirList&&, off_t, uint64_t) {
^
/Users/wez/fbsource/fbcode/eden/oss/eden/fs/fuse/Dispatcher.h:35:7: note: forward declaration of 'facebook::eden::DirList'
class DirList;
^
1 error generated.
```
Reviewed By: chadaustin
Differential Revision: D13475720
fbshipit-source-id: 2da1692010a82b73cbed71d996993cf5fc13af0e
Summary:
It appears as though osxfuse is a bit eager to re-use `header->unique`
values, as this DCHECK to ensure that we erased our request at the end of a
request is frequently triggered. It's not for 100% of requests, but I did
notice that the `header->unique` values tend to be in the range 1-6.
We don't check for collisions when we emplace requests in the dispatching
logic, and this particular check happens after we've acked the kernel, so it
doesn't seem like a safe thing to check and abort on really, so let's remove
it.
Reviewed By: chadaustin
Differential Revision: D13479516
fbshipit-source-id: a01e6c3e47b78b651f65fc3f5138168c71968030
Summary:
ported forward from D4209167, add a couple of helpers
to access these fields on mac and linux, centralizing/minimizing ifdefs.
Simplify some of the logic in FileChangeMonitor.
Reviewed By: chadaustin
Differential Revision: D13475717
fbshipit-source-id: d7b39999808bc41a6dc17a87189501cb34e68b30
Summary: only include and use it on linux
Reviewed By: simpkins
Differential Revision: D13475715
fbshipit-source-id: 6b0b9da1b32088e01cbb932f9b3ed62532dfe00f
Summary: only add the selinux deps if we found selinux
Reviewed By: simpkins
Differential Revision: D13475711
fbshipit-source-id: c3375282b61881317f9a6c4c8e321ce717d1f9ab
Summary: overlooked because there is no CI exercising this today
Reviewed By: simpkins
Differential Revision: D13475721
fbshipit-source-id: 3e8fe280ab73d249da374129b37d32cd7e17f472
Summary:
Eden no longer tracks any state in file handles, and has no plans to in the future.
Therefore, remove all related code.
Reviewed By: strager
Differential Revision: D13354307
fbshipit-source-id: 341d081f64c6c8fb2b4b1b5a5ff42f2cc7d38039
Summary:
Now that all file access in Eden is stateless, we no longer need to handle open() or release().
If the kernel advertises FUSE_NO_OPEN_SUPPORT, return ENOSYS from open().
Reviewed By: simpkins
Differential Revision: D13325759
fbshipit-source-id: 38486848f27ffeb005f74407888e94d891496f98
Summary:
D13325746 changed `EdenDispatcher::open()` to no longer return a file handle.
However the code in `FuseChannel::fuseOpen()` throws an exception if the
dispatcher does not create a file handle. This breaks most file operations in
Eden.
The Dispatcher check is removed in D13354307, but that hasn't landed yet.
That change probably just needed to be part of D13325746.
Reviewed By: chadaustin
Differential Revision: D13445570
fbshipit-source-id: 70d639142057740766bcbe02a0df50b14f7c9937
Summary:
Previously, a file handle must have been held for the entirety of a write operation. That is no
longer true. Stop looking up file handles on write.
Reviewed By: strager
Differential Revision: D13325662
fbshipit-source-id: 9ae31b467d17d633c388917d18098e6e5a620b89
Summary:
Write tests for readdir's semantics (we really do want to return . and
..) and simplify the logic. It's still quadratic in large directories,
but there aren't any allocations anymore.
Reviewed By: strager
Differential Revision: D13287764
fbshipit-source-id: 5e0d4b86eb16dbd7a16cdeb324e4b43363512e25
Summary:
Send readdir requests to TreeInode. This may not sound like a good
idea: the FUSE documentation suggests that stateful directory handles
are required to implement correct readdir semantics under concurrent
deletes and renames. However, the 63-bit offset value is treated as a
cookie that is passed from one readdir call into the next, and 63 bits
should be sufficient to implement readdir concurrent with
rename/unlink. So move readdir's implementation into TreeInode in
preparation for the complete removal of TreeInodeDirHandle.
Reviewed By: strager
Differential Revision: D13287664
fbshipit-source-id: c0d615675edd9b83353534468a69b89068bba923
Summary:
Send fsyncdir straight through the inode rather than going through
DirHandle. This is the better design anyway, since the DirHandle does
not receive directory-mutating requests like mkdir.
Reviewed By: strager
Differential Revision: D13287610
fbshipit-source-id: 154fa32a3877c89a204a2d10b4e2b637410d9486
Summary:
FUSE_NO_OPEN_SUPPORT is better than ATOMIC_O_TRUNC for Eden's use
case. Remove the code that pretended we might support ATOMIC_O_TRUNC
again someday.
(Note: this ignores all push blocking failures!)
Reviewed By: strager
Differential Revision: D13163382
fbshipit-source-id: 948d701571a8d2977da3d2532fdc9538c5011636
Summary:
The new blob cache wants to know, given a request, whether the blob is
expected to be needed or not. The answer, in general, is yes if the
request came from Thrift and no if it came from FUSE, because the kernel
will cache the result of the request in its own page and dentry caches.
Propagate this information through FileInode.
Reviewed By: strager
Differential Revision: D12813838
fbshipit-source-id: 7a359686149cd4daff41630c94085b680c448c4f
Summary:
Sandcastle has several cases where we chown the entire
repository which performs terribly on Eden. As a workaround we have a
command to do this in eden without loading all the files.
Reviewed By: chadaustin
Differential Revision: D12857956
fbshipit-source-id: 36cebcc710fbcf4e1eb265df901513cf50a227b9
Summary: This option was added upstream and to our internal 4.16 kernel series
Reviewed By: chadaustin
Differential Revision: D12823476
fbshipit-source-id: 54d241b77eff92f4083c0f1d4c98b47a6a098e3f
Summary:
D9029272 (github commit fae2056037) accidentally broke the CMake-based
build of Eden when SELinux is available: it defined `PRIVHELPER_LIBS` but
later used `PRIVHELPER_LIBRARIES` (LIBS vs LIBRARIES).
This fixes that error by simply removing this intermediate variable. We
can just use SELINUX_INCLUDE_DIR and SELINUX_LIBRARIES directly, as they
will simply be empty if SELinux is not available.
Reviewed By: quark-zju
Differential Revision: D10503905
fbshipit-source-id: 1c12bb1cad0351e4e0a77d0c7e8a83086209efee
Summary:
Eden supports reading the SHA-1 of a file via getxattr. Unfortunately,
it returned ENOSYS if you called getxattr on a directory inode. This
caused FUSE to fail all future getxattr requests with EOPNOTSUPP.
In addition to fixing that, this diff makes our xattr handling a
little more consistent across inodes:
- setxattr() always fails with ENOSYS
- removexattr() always fails with ENOSYS
- listxattr() is always handled by the corresponding inode class
- getxattr() is always handled by the corresponding inode class
Differential Revision: D10437723
fbshipit-source-id: a1ea1e92d3412abb15e91057becea3160a17f1e2
Summary:
This diff dynamically detects whether eden is running on an older
kernel and sends back a smaller fuse_init_out response.
Reviewed By: simpkins
Differential Revision: D10282016
fbshipit-source-id: ce9701e8f39defd4b90b15fc941ad2e243ac61e9
Summary: Always send write requests straight to the inode rather than going through FileHandle.
Reviewed By: wez
Differential Revision: D10220619
fbshipit-source-id: 9ce328583cf0fa9d7d8850d92d9e15ddc382d6a3
Summary:
Always send read requests straight to the inode rather than going
through the FileHandle.
Reviewed By: wez
Differential Revision: D10220604
fbshipit-source-id: 6aa5d20f3ce09696a29bd5c1cb95d0b987ab213c
Summary:
Always send setattr and getattr straight to the inode rather than
going through the FileHandle.
Reviewed By: wez
Differential Revision: D10187876
fbshipit-source-id: 4c3aaa977cd568d5f9cc4b28583e164119c07c1b
Summary: Eliminate the need to look up an InodeNumber from a FileHandle. Instead, simply preserve the mapping when it's created.
Reviewed By: wez
Differential Revision: D10187120
fbshipit-source-id: dc47f7776294871ff2398f33c31bd85d240ead50
Summary:
In order to implement FUSE_NO_OPEN_SUPPORT, we must eliminate
FileHandle and FileHandleBase. They didn't add any value anyway. Start
clipping.
Reviewed By: wez
Differential Revision: D10187103
fbshipit-source-id: 81e226f9c12486e0bbbde99b798b169fa31740c2
Summary:
Aborting the process with self-access is overlay aggressive. We
already respond with EIO, and the calling syscall should handle the
error. Continue logging at CRITICAL.
Reviewed By: wez
Differential Revision: D10134987
fbshipit-source-id: a4c4eed5e5de20698e95f442b3e063a09db311e6
Summary: We've diverged in a few places from clang-format, so run it across the entirety of Eden.
Reviewed By: wez
Differential Revision: D10137785
fbshipit-source-id: 9603c2eeddc7472c33041ae60e3e280065095eb7
Summary:
If an edenfs thread accesses an Eden mount, it trips SIGABRT from the
DFATAL, but the SIGABRT can't end the process because one thread is
stuck in uninterruptible sleep. So respond to the FUSE request before
aborting the process.
Reviewed By: strager
Differential Revision: D10118625
fbshipit-source-id: b84063e0b186d6a464531284b70c25b0b6a710ce
Summary: I wanted to see more details from the fuse requests, so log them.
Reviewed By: strager
Differential Revision: D9944649
fbshipit-source-id: 143703528fa029ed51e6cb42a5f6d8b3b0230ca3
Summary:
Add a Thrift API for reading the pid access logs from each
EdenMount/FuseChannel. Used in a future diff.
Reviewed By: strager
Differential Revision: D9477867
fbshipit-source-id: 0897a915ca654bca952aecc123ea40105830a75b
Summary: Begin tracking pids passed into FUSE in the ProcessAccessLog.
Reviewed By: strager
Differential Revision: D9595795
fbshipit-source-id: 02e5fefebcd0de860274409ba6258f14fa050b55
Summary: While looking at FuseChannel I noticed an unnecessary Future.
Reviewed By: strager
Differential Revision: D9595672
fbshipit-source-id: 5c84822c4f2c4c3c78b88456e44728e463d5a1e8
Summary:
By passing a locked ptr instead of a reference directly to
the locked object, the check and update methods can drop the lock
early
Reviewed By: chadaustin
Differential Revision: D9635507
fbshipit-source-id: a881043cfd2c28f6f53eb12e1494fcbc5f7f8e08
Summary:
Since the privhelper process is often not our direct child
we both cannot and do not need to wait for it, so treat ECHILD as
a successful exit status to prevent bubbling up an exception that
blocks graceful restart.
Reviewed By: simpkins
Differential Revision: D9581473
fbshipit-source-id: 6d53bbb6ee2043de76df9c6870028a1c15571dac
Summary:
Overall plan to modify Future<T>::then to be r-value qualified and use Future<T>::thenTry or Future<T>::thenValue.
The goal is to disambiguate folly::Future and to improve type and lifetime safety of Future and its methods.
Codemod:
* future<T>.then(callable with operator()(not-a-try)) to future<T>.thenValue(callable with operator()(not-a-try)).
* future<T>.then(callable with operator()()) to future<T>.thenValue(callable with operator()(auto&&)).
* future<T>.then(callable with operator()(auto)) to future<T>.thenValue(callable with operator()(auto)).
Reviewed By: yfeldblum
Differential Revision: D9512177
fbshipit-source-id: daa3581611dcd9f32d9314bae1c5fa0f966613f3
Summary:
Goal: change catchErrors() so its param is an rvalue-ref (`template <typename T> ... catchErrors(Future<T>&& fut)`) instead of a forwarding ref (and, if necessary, adjust callsites that passed an lvalue-ref).
Rationale: this is part of T33085035 which is migrating Future::onError() to be rvalue-only. D9505757 codemodded `RequestData::catchErrors()` from `fut.onError()` to `std::move(fut).onError()`. However `fut` was a forwarding-ref so it can actually hold an lvalue-ref.
Timeline/urgency: this diff is a prereq to removing the lvalue-qual overload of Future::onError() (T33085035). (Technically the only part of this that is a prereq to T33085035 is to adjust callers of catchErrors() that pass lvalue Futures, but changing the catchErrors() signature is righteous since it will force a more sensible error-message on any future callers which erroneously pass an lvalue.)
Note: D9505757 landed since it didn't break anything, even though it used `std::move()` on a forwarding-ref. (The lvalue-qual and rvalue-qual overloads of Future::onError() currently have identical semantics; neither invalidates / moves-out its Future object.)
Reviewed By: yfeldblum
Differential Revision: D9510204
fbshipit-source-id: 5c1ff2f36c5d84c583268f5e952fd322ea8e9327
Summary:
This is part of "the great r-valuification of folly::Future":
* This is something we should do for safety in general.
* Several of folly::Future's methods are lvalue-qualified even though they act as though they are rvalue-qualified, that is, they provide a postcondition that says, in effect, callers should act as though the method invalidated its `this` object (regardless of whether that invalidation was actual or logical).
* This violates the C++ principle to "Express ideas directly in code" (see Core Guidelines), and generally makes it more confusing for callers as well as hiding the actual semantics from tools (linters, compilers, etc.).
* This dichotomy and confusion has manifested itself by some failures around D7840699 since lvalue-qualification hides that operation's move-out semantics - leads to some use of future operations that are really not correct, but are not obviously incorrect.
* The goal of rvalueification is to make sure methods that are logically rvalue-qualified are actually rvalue-qualified, which forces callsites to acknowledge that rvalueification, e.g., `std::move(f).onError(...)` instead of `f.onError(...)`. This syntactic change in the callsites forces callers to acknowledge the method's rvalue semantics.
Codemod changes:
* expr.onError(...) ==> std::move(expr).onError(...) // if expr is not already an xvalue
* expr->onError(...) ==> std::move(*expr).onError(...)
Note: operator precedence of that last step is safe - no need to parenthesize `expr`. Reason: `->` binds more tightly than unary `*`.
Reviewed By: yfeldblum
Differential Revision: D9505757
fbshipit-source-id: de666f3a877313526d10f5d3569a1bbb2203f066
Summary:
This is part of "the great r-valuification of folly::Future":
* This is something we should do for safety in general.
* Several of folly::Future's methods are lvalue-qualified even though they act as though they are rvalue-qualified, that is, they provide a postcondition that says, in effect, callers should act as though the method invalidated its `this` object (regardless of whether that invalidation was actual or logical).
* This violates the C++ principle to "Express ideas directly in code" (see Core Guidelines), and generally makes it more confusing for callers as well as hiding the actual semantics from tools (linters, compilers, etc.).
* This dichotomy and confusion has manifested itself by some failures around D7840699 since lvalue-qualification hides that operation's move-out semantics - leads to some use of future operations that are really not correct, but are not obviously incorrect.
* The goal of rvalueification is to make sure methods that are logically rvalue-qualified are actually rvalue-qualified, which forces callsites to acknowledge that rvalueification, e.g., `std::move(f).ensure(...)` instead of `f.ensure(...)`. This syntactic change in the callsites forces callers to acknowledge the method's rvalue semantics.
This diff started as a Codemod, then required manual fixes. Here were the codemod steps:
* expr.ensure(...) ==> std::move(expr).ensure(...) // if expr is not already an xvalue
* expr->ensure(...) ==> std::move(*expr).ensure(...)
Note: operator precedence of that last step is safe - no need to parenthesize `expr`. Reason: `->` binds more tightly than unary `*`.
Reviewed By: yfeldblum
Differential Revision: D9332070
fbshipit-source-id: 882121fe82c05fdb196ce676db686b6bc254974b
Summary:
The setcon() failure is not actionable or interesting, so don't log it
to stdout at Eden startup.
Reviewed By: wez
Differential Revision: D9344467
fbshipit-source-id: 68435c8f22c228f2fbb86f37c2b1874723934169
Summary:
Overall plan to modify Future<T>::then to be r-value qualified and use Future<T>::thenTry or Future<T>::thenValue.
The goal is to disambiguate folly::Future and to improve type and lifetime safety of Future and its methods.
6/n: Codemod rvalue-future<T>.then(...) to rvalue-future<T>.then(...).
Reviewed By: yfeldblum
Differential Revision: D9152002
fbshipit-source-id: 166475c1dcafb29a11154cbfbdf7e2e1feaf745b
Summary:
Fixed a data race in Eden's `FuseChannel` implementation that could cause a crash if a `FUSE_INTERRUPT` request was concurrently on a different thread while a first thread was still launching the original request.
Modified `FakeFuse` to use `SOCK_SEQPACKET` instead of `SOCK_STREAM` so that tests can submit several requests in a pipelined fashion without having to first wait until the previous request was received. This requires the `recvResponse()` function to first read the header with `MSG_PEEK` to determine the response size and then subsequently read the entire message atomically to avoid reading a truncated message.
Added a new unit-test that exercises the `FUSE_INTERRUPT` race condition by sending a series of alternating `FUSE_LOOKUP`/`FUSE_INTERRUPT` requests.
Reviewed By: simpkins
Differential Revision: D9023654
fbshipit-source-id: 0eb44669ea8c4f58cf4313adf6ceb11098115a70
Summary:
I tried to build eden using GitHub version and ran into SELinux issues.
This patch fixed related issues, namely:
- `set(X Y)` sets X to string literal Y. Change it to `set(X ${Y})`.
- `SELINUX_INCLUDE_DIR` could be undefined. Check before use it.
- `./eden/fs/eden-config.h` and `./build/eden/fs/eden-config.h` both exist and
the former is used. Set include dirs so the latter gets used.
Reviewed By: bolinfest
Differential Revision: D9029272
fbshipit-source-id: 0c94bbe2f9e3fa90973802ddde16ad4d9ddfc0e8
