Summary:
We use privhelper for NFS too, so move fs/fuse/privhelper into
fs/privhelper.
Reviewed By: kmancini
Differential Revision: D45243273
fbshipit-source-id: cfc93b324768809c87075b0085905e873c7ea88d
Summary:
waitForPendingWrites is general to any userspace filesystem with
asynchronous write notifications. This could even include FUSE in
writeback caching mode. To remove another ifdef, move
waitForPendingWrites into FsChannel.
Reviewed By: kmancini
Differential Revision: D45167206
fbshipit-source-id: 4daec70845864d90b94739fe2011e8ed90a6a200
Summary:
FuseChannel already had an asynchronous initialize(). Lift that to
FsChannel and unify initialization of Prjfs and NFS.
Reviewed By: kmancini
Differential Revision: D45106889
fbshipit-source-id: 632ddfa275c732fc30efc1984bda43f61c37fd5e
Summary:
EdenMount shouldn't know about the vagaries of FsChannel teardown. In
fact, FuseChannel already has its own requirements, hidden behind a
unique_ptr deleter. Use the same mechanism for NFS, allowing us to
remove a .via() special-case.
Reviewed By: kmancini
Differential Revision: D44810223
fbshipit-source-id: db2f88bc3843b3842de60c5a993dfd9890dc3c15
Summary:
I'm carefully reading the takeover code and noticed a few things to
improve.
Reviewed By: kmancini
Differential Revision: D44946740
fbshipit-source-id: 558441a47425b15c19ca2a6816ae2e8bf3d68019
Summary:
For small handle types, passing by value is the correct choice,
because it indicates unconditional ownership transfer. (It also
generates slightly smaller code and is less typing.)
Reviewed By: xavierd
Differential Revision: D44927835
fbshipit-source-id: 5fefee7889e8216e46548f4801e83f51cde8b8c4
Summary:
StopData is slightly different for each filesystem implementation. But
it's used in the same way. Introduce a standard interface and port
FUSE to it.
Reviewed By: kmancini
Differential Revision: D44737143
fbshipit-source-id: 93ea567b6bc4d20a89ea1e285ab7240f533f6854
Summary:
A common FsChannel base class allows removing some ifdefs from
EdenServiceHandler. With some effort, we should be able to find the
correct abstractions that allow us to entirely decouple particular
FsChannel implementations from the rest of the inode layer.
Reviewed By: kmancini
Differential Revision: D44657395
fbshipit-source-id: 9702186f1d0c9ce76a11e766010f5899e2b517af
Summary:
DurationScope has a pair of atomic reference count operations at the
beginning and end of every recorded duration. To avoid that overhead,
reference EdenStats with RefPtr and, in production EdenFS, store a
global, unowned EdenStats object.
This gives us the benefits of explicit dependencies and no global
state in tests, while avoiding atomic RMWs in the release build.
Reviewed By: xavierd
Differential Revision: D44323723
fbshipit-source-id: 1b3384d2e6a0a2959fd79774a8ba46afc4c176ca
Summary:
This makes lifetime easier to reason about, and allows easier use of
DurationScope (see next diff).
Reviewed By: chadaustin
Differential Revision: D44311712
fbshipit-source-id: 1f9f4cbcc59bafeb464e2b39a248958cedaf9ca6
Summary:
This copies the new `fuse_rename_in` fields from the final public release of osxfuse's `fuse_kernel.h` into our macOS FUSE header file, so that our header is now identical to the public release (aside from fixing a shift overflow issue).
This breaks compatibility with old versions of osxfuse prior to the ABI change.
With the header updated to match macFUSE's ABI, we can also remove the runtime check for an extended `fuse_rename_in` in FuseChannel.cpp.
Reviewed By: chadaustin
Differential Revision: D42617239
fbshipit-source-id: 2cf3aaec87488f53dc5f3fec439b7c34c2e5a6b5
Summary: It's much easier to read throwf vs throw_. Let's move all uses over to throwf.
Reviewed By: chadaustin
Differential Revision: D41698085
fbshipit-source-id: 20ff0aece7dff0abce8c196ae06274d4dc4f3f90
Summary:
When starting developer instances of EdenFS, we're forced to use `sudo` to start developer instances privhelper daemons. This is an issue because `sudo` is not available on macOS sandcastle hosts, and therefore we cannot run integration tests on macOS (all integration tests for macOS are currently disabled).
Instead, we can use the system privhelper by default (instead of using developer instances of privhelper). The system privhelper is installed with SUID root, which means we can leverage its privileges to run integration tests on macOS without `sudo`!
Reviewed By: chadaustin, xavierd
Differential Revision: D41020919
fbshipit-source-id: ef26f95d673b9290c62c7d0755d580f30eb43645
Summary: We moved away from forking the privhelper due to security reasons. The logic is now only used in 1 test, so it's worth deleting this code now.
Reviewed By: chadaustin
Differential Revision: D41697605
fbshipit-source-id: 5e9beae1689d72cd6dda2abd19d81077255017b4
Summary:
In some cases, we may want a developer-instance of EdenFS to communicate with a production privhelper daemon. However, this can cause issues if the protocol between the edenfs-daemon and privhelper-daemon change in the developer version of EdenFS (or vice versa).
To ensure protocol versioning issues aren't a problem, we need to add versioning info/logic to the privhelper server/conn code. This will ensure that we can keep the protocols backwards/forwards compatible in the future.
Reviewed By: chadaustin
Differential Revision: D40960617
fbshipit-source-id: e862f041a3d31efd68dae6a52146e68742b1405b
Summary:
We've had a couple of failures after landing D40818724 (f199e93924) due to some places still
building AbsolutePath from a literal. On Windows, this is almost always a bug
due to paths usually not being UNC.
To avoid this issue, we can simply forbid an AbsolutePath from being built in
the first place from a path, instead forcing the use of canonicalPath. This was
suggeted by mshroyer in the orignal diff.
Reviewed By: chadaustin
Differential Revision: D41229194
fbshipit-source-id: 3451bdcba276c87f98326b025e05f6a4eddbd1b7
Summary:
std::string_view has noexcept accessors and folly::Range doesn't, so
this allows us to make Path and PathPiece noexcept.
Reviewed By: kmancini
Differential Revision: D41145426
fbshipit-source-id: 046f6f6a532d8d0da8508ccf7896c914e19a25ec
Summary:
Requiring that callers ensure that ObjectFetchContext outlives the
asynchronous result is a burden and a memory-safety risk. Without
Rust's borrow checker, we can't feel confident this is safe under all
error and timeout conditions. It also requires we use
folly::collectAll and collectAllSafe, which means that an error during
fanout has to wait until the fanout completes before we can report the
error to our caller.
Moreover, tying the ObjectFetchContext's lifetime to the
FUSE/NFS/Thrift/etc. request's lifetime gives us incorrect statistics
about how long the request takes. I consider this an API design
mistake (on my part).
Correct the situation by decoupling ObjectFetchContext from
RequestContext, and begin reference-counting ObjectFetchContext.
This does increase the number of atomic RMW operations, but we can
alleviate those where they matter by manually checking
*Future::isReady() before calling .then*.
Reviewed By: xavierd
Differential Revision: D40744706
fbshipit-source-id: 5baf654f6d65790ed0cafb45c0bc7f2aaa8ecec2
Summary:
At API boundaries, it's common to want to convert a Path or PathPiece
to std::string. Previously, code would call
`.stringPiece().str()`. Add an `asString()` to make this explicit.
Reviewed By: xavierd
Differential Revision: D41140757
fbshipit-source-id: 72003dc9a353cdfa4632a7d1ab433122211186e0
Summary:
We are trying to enable rootsquash on our netapp volume mounts (eg. userA sudos to root and does ls on some nfs mount -> the nfs server maps that root uid to some facade uid)
and we see that eden clone fails either the access, or stat checks. I tested an `access` performing binary and stat, and both let my personal user access the mount that fails in clone, so I'd like to add this extra int in the debug to clear it up. It seems as though sudo (and hence the facade account) is invoked since the facade account does show these errors.
Reviewed By: xavierd
Differential Revision: D40764092
fbshipit-source-id: 8df782b2f4c10cfa51186074753c1b0f75275409
Summary:
Separately allocate ObjectFetchContext in RequestContext so that its
lifetime can extend longer than the request itself.
Reviewed By: kmancini
Differential Revision: D40349610
fbshipit-source-id: abc1785f194abb06383a2e72cb2ca71c266ae42b
Summary:
Now that finishRequest is moved into the destructor, we no longer need
a custom shared_ptr constructor function.
Reviewed By: genevievehelsel
Differential Revision: D40349291
fbshipit-source-id: 1ac8c745cab4a09c7c5cf899803e5683d2aeedf7
Summary:
I noticed security.selinux getxattr calls when ls-ing around in an
EdenFS mount. Skip almost all work for those.
Reviewed By: kmancini
Differential Revision: D40603477
fbshipit-source-id: 9e983a7edab9bb5c15a25189c4591175229b2a1e
Summary: It is no longer necessary to pass a reference to the caught exception along with `std::current_exception()`.
Reviewed By: xavierd
Differential Revision: D40252106
fbshipit-source-id: 08bfbea675c60097050f3f026af308fcf0a240df
Summary:
In advance of some trickier changes, make a few small improvements to
ObjectFetchContext.
Reviewed By: genevievehelsel
Differential Revision: D39901208
fbshipit-source-id: ed50141558201dd2666d55a21bf63db5bf2ffd43
Summary:
The main non-mechanical conversion from folly/Conv to fmt is replacing
folly::to<std:string>'s variadic concatenation operation with
fmt::join. Now that that's solved, we can slowly migrate the rest.
Reviewed By: genevievehelsel
Differential Revision: D39992102
fbshipit-source-id: 81df8e805c78ea5ca80ad5977f20b19e5a7467a3
Summary:
This allows us to remove ifdefs and begin enforcing consistent
prefixes in StatsGroup.
Reviewed By: genevievehelsel
Differential Revision: D39782327
fbshipit-source-id: c453e0ada5d5d9d3b221c707f67dce7d1e0a970d
Summary:
The public EdenStats API was both too chatty and exposed too many
implementation details. Don't mention thread-locals in the public API,
and replace uses of `addValue` with explicit `addDuration` on Duration
and `increment` on Counter.
Reviewed By: genevievehelsel
Differential Revision: D39661497
fbshipit-source-id: 8f6f2a9d72a4c385cde32e72fba0cebf8f5b4274
Summary:
Rename "channel" to "fs channel", rename StatPtr to DurationStatPtr,
and generalize DurationStatPtr to every stat category.
Reviewed By: xavierd
Differential Revision: D39631597
fbshipit-source-id: 147f9a9d9fe97a39c0d38ccecf57e73c93af0d8c
Summary:
It's not possible to differentiate NFS m1 and intel users in edenfs_events
right now. We are currently rolling out to intel users and it would be good
to understand the real number of intel users on NFS vs fuse.
Let's log the cpu architecture so that we can differentiate them.
Reviewed By: mshroyer
Differential Revision: D39139996
fbshipit-source-id: 582804707a7816c41aa6c69594f7198d3ce5dfe1
Summary:
Fixes https://github.com/facebookexperimental/edencommon/issues/3
`utils` is a bit too generic and Gentoo seems to be building edencommon into a shared library.
Reviewed By: chadaustin
Differential Revision: D38719753
fbshipit-source-id: fb46b6a7c9d3bcc3034765cb47e997a80c646b3d
Summary:
From the folly format docs
> Use fmt::format instead of folly::format for better performance, build times and compatibility with std::format
eden build times have gotten a bit high, cutting out folly format will help reduce build time, so let's start
by banishing it from eden
see the last change in this group for the difference
Reviewed By: xavierd
Differential Revision: D38251001
fbshipit-source-id: 0e6c4f7403a6962ef1e5fc6adba4a991dcdf6a5d
Summary:
On macOS Ventura, Apple seems to not loading this kernel extension with mount syscall.
After some debugging it turned out we need to load this kernel extension manually if it is not loaded. This diff does that.
Reviewed By: xavierd
Differential Revision: D38258142
fbshipit-source-id: 7c8d52d758626679345aea5c7f4940c5c0ea8b64
Summary: In a previous change (D37961108 (18bc0ee56a)), we added path information to the `eden trace inode` live and retroactive commands. In that change, the path information was computed only when needed in the thrift Eden service handler using the current EdenMount's InodeMap. However, when doing this, retroactive path information may not always be available (or correctly representative of the past path) if an inode in the path has since been removed (or changed), and live path information may not always be available in the InodeMap in the case of start events for Inode creations. This diff addresses this issue by adding in the current inode's filename (but not the full path) to each tracebus InodeEvent when the InodeEvent occurs and defaults to displaying just this filename if the full path info isn't available. An exception occurs when new inodes are created and we compute the full path anyway in which case we do add the full path to the tracebus InodeEvent.
Reviewed By: kmancini
Differential Revision: D38000772
fbshipit-source-id: 7a718989c397425cc0b0c96def21fd09b5b55fc9
Summary:
To simplify the build and fall in line with Folly's conventions,
remove several conditional includes. Instead, headers conditionally
define symbols.
Reviewed By: xavierd
Differential Revision: D37897850
fbshipit-source-id: 8b52d5310f5cd8cb17fdc013117c271ae09dd5d6
Summary:
LockedPtrs used as temporaries within range expressions may go out of scope
before the end of the loop.
https://en.cppreference.com/w/cpp/language/range-for:
> If range-expression returns a temporary, its lifetime is extended until the
> end of the loop, as indicated by binding to the forwarding reference __range,
> but beware that the lifetime of any temporary within range-expression is not
> extended.
Reviewed By: xavierd
Differential Revision: D37904242
fbshipit-source-id: 915812e7bb4d19c333dc4c309261610884f8e61d
Summary:
In debug builds, PathComponent will no longer be noexcept moveable, thus the
static_assert fires, failing the build. By fully specifying the noexcept
qualifier, the code stays correct, but the build succeeds.
Reviewed By: kmancini
Differential Revision: D37840355
fbshipit-source-id: a1a862ebe4060983fea8d7a94ca68fec307f91bd
Summary:
I've always been annoyed at needing to include folly/Conv.h or fmt to
throw a formatted exception. And while exceptions are rarely in the
hot path, it's still worth having the option to optimize the
formatting and exception allocation step at some point in the future.
I'm sure I missed some, but this diff migrates most of our
`throw T{fmt::format(...)}` patterns to `throwf` and
`throw T{folly::to<std::string>(...)}` to `throw_`.
Reviewed By: genevievehelsel
Differential Revision: D37229822
fbshipit-source-id: c053b5cdaed99e58c485afd2a99be98828f07657
Summary:
The former is problematic on Windows, let's thus be consistent and always use
Folly's GFlags header.
Reviewed By: chadaustin
Differential Revision: D37216406
fbshipit-source-id: d2e527ffc77bed626e282b37f3bde1b04b919b11
