Commit Graph

200 Commits

Author SHA1 Message Date
Chad Austin
d8400c6faa small RpcServer simplifications
Summary:
I'm trying to track down a subtle lifetime rule violation in
EdenMount. While reading RpcServer, I noticed a few possible
simplifications.

Reviewed By: xavierd

Differential Revision: D44849773

fbshipit-source-id: 4c27c47a7e2c211dcd040acce955e9b2f617b55b
2023-04-20 18:49:40 -07:00
Chad Austin
2c30d06196 unify FUSE and NFS takeover handling
Summary:
By moving NFS's StopData behind FsStopData, the std::variant and
runtime checks during takeover can be unified into one path. The next
diff will do the same for Windows.

Reviewed By: kmancini

Differential Revision: D44737160

fbshipit-source-id: 19ca623f90b367d25385f177cafa44d7bf7ebb62
2023-04-20 18:49:40 -07:00
Katie Mancini
d9737647bc register rpc servers with port mapper
Summary:
Now that we have the plumbing we can start registering our NFS RPC servers (
mountd and nfsd) with the portmapper/rpcbind.

I am taing a short cut for prototyping. I am skipping general purpose
rpc registration (i.e. making these rpc calls to register a service). This is
because (1) I am trying to prototype quickly, so I don't want to implement the
portmapper endpoints that allow registering/ unregistering a server and (2)
there are maybe security considerations, and I don't want to think about those
rn.

Reviewed By: chadaustin

Differential Revision: D44987884

fbshipit-source-id: 4b849d1dd060d2e033ab48df883e228015906a7d
2023-04-18 17:50:02 -07:00
Katie Mancini
901f989bc7 portmapper reply to getport
Summary:
getport is the primary endpoint on the rpcbind/portmap server. NFS clients
make a request on this endpoint to find the NFS RPC servers.

In this diff we keep a mapping of RPC servers and respond to the getport
request based on this mapping.

Currently no rpc severs are registered, so we reply no port to all
requests.

Reviewed By: chadaustin

Differential Revision: D44987893

fbshipit-source-id: 920171c6007b82609ae84fb7a9aff43d898e43dd
2023-04-18 17:50:02 -07:00
Katie Mancini
4ee4fa5e6b skeleton of portmapper server
Summary:
The msft client can't be pointed at an NFS server on a particular port, so
we have to register our RPC servers (mountd and nfsd) with the port mapper
service.

There is no portmapper service running on Windows by default.

There is a port mapper implementation from msft that can be installed on
Windows Server, but not on Windows desktop.

There are some third-party implementations of the portmapper for Windows. Most
of them are part of an NFS Server implementation. But none that I have found
both run on modern Windows versions, are a general purpose port mapper, and
support port mapper v2. General purpose means they allow processes to set a
port mapping. Most the port mappers I found must register their NFS server in
process, because they don't support calling the set mapping endpoint. The rest
only run on Windows versions from before I was born or only run v4.

I tried porting the linux gnu implementation of port mapper to Windows. But despite
a few hours of trying I could not even get it to build even on Linux. Writing our own port mapper is faster.

This is the skeleton of the server. In the next diffs I will register our
RPC servers and support telling the msft NFS client about our servers.

Reviewed By: chadaustin

Differential Revision: D44987863

fbshipit-source-id: bff065795a9f4b7b6c13ef3e3ce603646e1ce364
2023-04-18 17:50:02 -07:00
Katie Mancini
90741e9e30 introduce portmapper protocol v2
Summary:
The msft NFS client likes to speak portmapper v2. It does not seem to know v4.
Perhaps there is a way to configure that, but I have not found it yet.

v4 is the latest version of the protocol and so what we should prefer using on
mac and linux.

Here I am introducing the v2 protocol and renaming all the v4 types to
explicitly reference v4.

Reviewed By: chadaustin

Differential Revision: D44987848

fbshipit-source-id: 97f9e2365736b49a5888af204609598b2566de55
2023-04-18 17:50:02 -07:00
Katie Mancini
974bd46a9d don't start portmapper client for now on Windows
Summary:
There is no portmapper service running on Windows by default, so we should
not be trying to connect to one by default on Windows.

Though I am adding a port mapper service that eden it self runs in the coming
diffs it is not compatible with this client. The msft NFS client likes to
speak portmapper v2 and this client only knows portmapper v4 (otherwise known
as rpc bind). So I am stubbing these out for now.

Reviewed By: chadaustin

Differential Revision: D44987834

fbshipit-source-id: 078ed1761cffbe9621ab39892c2135504a6aeb1b
2023-04-18 17:50:02 -07:00
Chad Austin
774494ba52 introduce FsChannel with a completeInvalidations method
Summary:
A common FsChannel base class allows removing some ifdefs from
EdenServiceHandler. With some effort, we should be able to find the
correct abstractions that allow us to entirely decouple particular
FsChannel implementations from the rest of the inode layer.

Reviewed By: kmancini

Differential Revision: D44657395

fbshipit-source-id: 9702186f1d0c9ce76a11e766010f5899e2b517af
2023-04-14 00:44:11 -07:00
Katie Mancini
bca274d519 make NFS Server run on windows
Summary:
I'm getting nfs to run on windows to prototype it and see how feasible it
might be as an option on Windows. PrjFS has a very different model than EdenFS,
and that has made EdenFS correctness on Windows very difficult. NFS may be
easier to get correct, though the performance is suspect. Just exploring
options here.

After this change the NFS Server opens a socket to accept incoming connections!

Reviewed By: xavierd

Differential Revision: D44154505

fbshipit-source-id: 10c679cb390c2bab99591656d6fe96ccdf4443fe
2023-04-06 13:00:32 -07:00
Katie Mancini
04c2b8b54c get NfsDispatcherImpl building on Windows
Summary:
I'm getting nfs to build on windows to prototype it and see how feasible it
might be as an option on Windows. PrjFS has a very different model than EdenFS,
and that has made EdenFS correctness on Windows very difficult. NFS may be
easier to get correct, though the performance is suspect. Just exploring
options here.

After this change NFS builds on Windows!

NOTE: this one is more than removing ifdefs, probably the most non trivial in the stack, please review carefully

Reviewed By: xavierd

Differential Revision: D44153443

fbshipit-source-id: b07e19f8bde1aa455eec97647ea450849133041f
2023-04-06 13:00:32 -07:00
Chad Austin
7d4a54eb11 DirList -> {Fuse,Nfs}DirList
Reviewed By: xavierd

Differential Revision: D44716052

fbshipit-source-id: 45bd5d4b1e55792a3f43853dd4decc1faf4a1a9e
2023-04-05 16:35:43 -07:00
Katie Mancini
c76ead3eda get mountd building on windows
Summary:
I'm getting nfs to build on windows to prototype it and see how feasible it
might be as an option on Windows. PrjFS has a very different model than EdenFS,
and that has made EdenFS correctness on Windows very difficult. NFS may be
easier to get correct, though the performance is suspect. Just exploring
options here.

Reviewed By: xavierd

Differential Revision: D44153341

fbshipit-source-id: 8351c72e1ad4dd41090a6289aadebc6749bdf056
2023-03-29 12:52:44 -07:00
Katie Mancini
9265f1e7df get mount rpc building on Widnows
Summary:
I'm getting nfs to build on windows to prototype it and see how feasible it
might be as an option on Windows. PrjFS has a very different model than EdenFS,
and that has made EdenFS correctness on Windows very difficult. NFS may be
easier to get correct, though the performance is suspect. Just exploring
options here.

Reviewed By: xavierd

Differential Revision: D44153119

fbshipit-source-id: 4d6034ea3c7c417d69da5ce54256ae8751ba2a73
2023-03-29 12:52:44 -07:00
Katie Mancini
3c3dc3ae07 get nfsd3 building on windows
Summary:
I'm getting nfs to build on windows to prototype it and see how feasible it
might be as an option on Windows. PrjFS has a very different model than EdenFS,
and that has made EdenFS correctness on Windows very difficult. NFS may be
easier to get correct, though the performance is suspect. Just exploring
options here.

Reviewed By: xavierd

Differential Revision: D44153042

fbshipit-source-id: 012adea3c89e9c4b37c08cff685e3937d2c6678b
2023-03-29 12:52:44 -07:00
Katie Mancini
6cf70b37d5 get rpc server building on Windows
Summary:
I'm getting nfs to build on windows to prototype it and see how feasible it
might be as an option on Windows. PrjFS has a very different model than EdenFS,
and that has made EdenFS correctness on Windows very difficult. NFS may be
easier to get correct, though the performance is suspect. Just exploring
options here.

Reviewed By: xavierd

Differential Revision: D44152440

fbshipit-source-id: 70de1a7dfd986b60e57079dd9f1d20a6b63dbc0f
2023-03-29 12:52:44 -07:00
Katie Mancini
ae9b25a51b get portmapper client building on Windows
Summary:
I'm getting nfs to build on windows to prototype it and see how feasible it
might be as an option on Windows. PrjFS has a very different model than EdenFS,
and that has made EdenFS correctness on Windows very difficult. NFS may be
easier to get correct, though the performance is suspect. Just exploring
options here.

Reviewed By: xavierd

Differential Revision: D44152981

fbshipit-source-id: fb2788fd2334ad0debf3d45bcb6bb04ca5667b1f
2023-03-29 12:52:44 -07:00
Katie Mancini
7ff6f41868 make stream client build on Windows
Summary:
I'm getting nfs to build on windows to prototype it and see how feasible it
might be as an option on Windows. PrjFS has a very different model than EdenFS,
and that has made EdenFS correctness on Windows very difficult. NFS may be
easier to get correct, though the performance is suspect. Just exploring
options here.

Reviewed By: xavierd

Differential Revision: D44152888

fbshipit-source-id: d213f87921d4505429aea42bfa1007a97f89f9dd
2023-03-29 12:52:44 -07:00
Katie Mancini
f18a34cc0d Getting NfsDispatcher building on Windows
Summary:
I'm getting nfs to build on windows to prototype it and see how feasible it
might be as an option on Windows. PrjFS has a very different model than EdenFS,
and that has made EdenFS correctness on Windows very difficult. NFS may be
easier to get correct, though the performance is suspect. Just exploring
options here.

NOTE: this one is more than removing ifdefs, please review carefully

Reviewed By: xavierd

Differential Revision: D44152305

fbshipit-source-id: ee39db08a53762f23bf184cc6dec41ec28627cda
2023-03-29 12:52:44 -07:00
Katie Mancini
d15b6ccd81 get nfs_utils building on Windows
Summary:
I'm getting nfs to build on windows to prototype it and see how feasible it
might be as an option on Windows. PrjFS has a very different model than EdenFS,
and that has made EdenFS correctness on Windows very difficult. NFS may be
easier to get correct, though the performance is suspect. Just exploring
options here.

NOTE: this one is more than removing ifdefs, please review carefully

Reviewed By: xavierd

Differential Revision: D44152041

fbshipit-source-id: 21affcc417dce492c2846902225686334eb2dc30
2023-03-29 12:52:44 -07:00
Katie Mancini
6ccae0c2f5 Get nfs_rpc building on Windows
Summary:
I'm getting nfs to build on windows to prototype it and see how feasible it
might be as an option on Windows. PrjFS has a very different model than EdenFS,
and that has made EdenFS correctness on Windows very difficult. NFS may be
easier to get correct, though the performance is suspect. Just exploring
options here.

Reviewed By: xavierd

Differential Revision: D44151904

fbshipit-source-id: c14fa313b2817c3d55f616f4252a0611965ead78
2023-03-29 12:52:44 -07:00
Katie Mancini
bdeeebcd24 get rpc building on Widnows
Summary:
I'm getting nfs to build on windows to prototype it and see how feasible it
might be as an option on Windows. PrjFS has a very different model than EdenFS,
and that has made EdenFS correctness on Windows very difficult. NFS may be
easier to get correct, though the performance is suspect. Just exploring
options here.

Reviewed By: xavierd

Differential Revision: D44151833

fbshipit-source-id: 5d39f9d3de527cd21506b8473555f9dbd2a587bb
2023-03-29 12:52:44 -07:00
Katie Mancini
444a19e5e9 get xdr building on Windows
Summary:
I'm getting nfs to build on windows to prototype it and see how feasible it
might be as an option on Windows. PrjFS has a very different model than EdenFS,
and that has made EdenFS correctness on Windows very difficult. NFS may be
easier to get correct, though the performance is suspect. Just exploring
options here.

Reviewed By: xavierd

Differential Revision: D44151636

fbshipit-source-id: 0f13b997064d96995f48b658ccf0ecac6477dad8
2023-03-29 12:52:44 -07:00
Chad Austin
e419dd2799 remove the reference-counting overhead from DurationScope
Summary:
DurationScope has a pair of atomic reference count operations at the
beginning and end of every recorded duration. To avoid that overhead,
reference EdenStats with RefPtr and, in production EdenFS, store a
global, unowned EdenStats object.

This gives us the benefits of explicit dependencies and no global
state in tests, while avoiding atomic RMWs in the release build.

Reviewed By: xavierd

Differential Revision: D44323723

fbshipit-source-id: 1b3384d2e6a0a2959fd79774a8ba46afc4c176ca
2023-03-24 13:50:40 -07:00
Xavier Deguillard
317e55ca91 inodes: use EdenStats with shared_ptr, not raw pointers
Summary:
This makes lifetime easier to reason about, and allows easier use of
DurationScope (see next diff).

Reviewed By: chadaustin

Differential Revision: D44311712

fbshipit-source-id: 1f9f4cbcc59bafeb464e2b39a248958cedaf9ca6
2023-03-22 20:04:41 -07:00
Xavier Deguillard
553b4278a0 main: move folly::init default configuration to main
Reviewed By: fanzeyi

Differential Revision: D44263797

fbshipit-source-id: 61a219e98b96f61495eb549c0e01ca635ab29e9f
2023-03-21 14:17:32 -07:00
Mark Shroyer
58fa5c3541 Fix 100072 errors when copying file into EdenFS in Finder
Summary:
# Problem

In macOS Ventura, copying a file into an EdenFS NFS mount in Finder only creates an empty file, producing the error message:

> The operation can’t be completed because an unexpected error occurred (error code 100072)

This error code indicates a bad RPC struct, and it happens because we're sending a malformed response to some SETATTR requests.

# Analysis

In D40798840 (34884008f5) we added code to ignore two types of setattr requests:

- On NFS, requests that are no-op aside from updating an `atime`.
- Everywhere, requests that are entirely no-op.

Contrary to the description of D40798840 (34884008f5), it's the former type of request that can be created by `mmap`-ing a file over NFS. The latter appears to be generated at various points by macOS, including when copying a file into an NFS share with Finder.

We currently return an empty response body to that second type of request, without even a response status. As of macOS Ventura this produces an error message.

# Fix

Here we modify the `is_nop` check at the NFS level so that SETATTRs that only update `atime` become fully no-op, falling through to the FileInode check (which doesn't ignore `atime`) which then considers the request no-op.  This returns up through the NFS stack to produce a well-formed NFS response.

This leaves in place the `atime`-sensitive behavior at the FileInode level.

This does not affect the invalidation behavior introduced in D35435764 (8f3f873874), which sets `mode`.

Reviewed By: xavierd

Differential Revision: D42785781

fbshipit-source-id: af174abd61b4927040f481d0a177443eb64ae8da
2023-01-27 10:47:02 -08:00
Mark Shroyer
a9322aeae7 Restore metadata size mismatch logging
Summary:
This re=implements the backed-out D41393677 (6384a0f39a) with some changes:

  - Only log if the size metadata is _less than_ the size of the file contents
    we're returning.  This way we avoid erroneously logging partial reads, as
    happened in T139036919.

  - Only log once per Nfsd3ServerProcessor instance.

Reviewed By: chadaustin

Differential Revision: D41751040

fbshipit-source-id: 2503ecb488b0600bdeabbb11e532e4590edbd280
2022-12-14 03:18:23 -08:00
Mark Shroyer
83bb997ee9 Back out "Log metadata size mismatch on NFS READ"
Summary:
Original commit changeset: 968c2866d8fd

Original Phabricator Diff: D41393677 (6384a0f39a)

Reviewed By: xavierd

Differential Revision: D41624043

fbshipit-source-id: 051981d201d28905d97385012eecf58e75ce8634
2022-11-30 14:56:35 -08:00
Mark Shroyer
6384a0f39a Log metadata size mismatch on NFS READ
Summary:
Log to text logs and Scuba if there's an NFS READ where the size of the file
contents didn't match the size in metadata.  This way we can see whether size
mismatches are still happening much in practice, in order to know how to
prioritize further investigation.

Based on kmancini's D41239395, but adds Scuba logging too.

Reviewed By: chadaustin, kmancini

Differential Revision: D41393677

fbshipit-source-id: 968c2866d8fd353e89ad871080a57308cacdc102
2022-11-28 14:42:03 -08:00
Xavier Deguillard
b7d87d807a utils: deny building an AbsolutePath from a literal
Summary:
We've had a couple of failures after landing D40818724 (f199e93924) due to some places still
building AbsolutePath from a literal. On Windows, this is almost always a bug
due to paths usually not being UNC.

To avoid this issue, we can simply forbid an AbsolutePath from being built in
the first place from a path, instead forcing the use of canonicalPath. This was
suggeted by mshroyer in the orignal diff.

Reviewed By: chadaustin

Differential Revision: D41229194

fbshipit-source-id: 3451bdcba276c87f98326b025e05f6a4eddbd1b7
2022-11-22 16:20:30 -08:00
Mark Shroyer
615748c257 Log if there's an error calling getattr during NFS read
Summary:
During NFS reads we call getattr and lookupInode to get post-operation
attributes, but we currently silently return empty attributes if lookupInode
fails but the read otherwise succeeds.  This adds logging so we can check if
this is happening when we reproduce NFS returning truncated file contensts.

Reviewed By: xavierd

Differential Revision: D41364371

fbshipit-source-id: 8f85ae6ed767ad880ec5cbc863b2d573fdc341f2
2022-11-17 14:20:46 -08:00
Chad Austin
6f60c48ea8 decouple ObjectFetchContext lifetime from RequestContext
Summary:
Requiring that callers ensure that ObjectFetchContext outlives the
asynchronous result is a burden and a memory-safety risk. Without
Rust's borrow checker, we can't feel confident this is safe under all
error and timeout conditions. It also requires we use
folly::collectAll and collectAllSafe, which means that an error during
fanout has to wait until the fanout completes before we can report the
error to our caller.

Moreover, tying the ObjectFetchContext's lifetime to the
FUSE/NFS/Thrift/etc. request's lifetime gives us incorrect statistics
about how long the request takes. I consider this an API design
mistake (on my part).

Correct the situation by decoupling ObjectFetchContext from
RequestContext, and begin reference-counting ObjectFetchContext.

This does increase the number of atomic RMW operations, but we can
alleviate those where they matter by manually checking
*Future::isReady() before calling .then*.

Reviewed By: xavierd

Differential Revision: D40744706

fbshipit-source-id: 5baf654f6d65790ed0cafb45c0bc7f2aaa8ecec2
2022-11-15 13:35:45 -08:00
Di Xiao
5777b33e4d Make NFS Tracebus size configurable by EdenConfig
Summary: Adding an ConfigSetting "telemetry:nfs-tracebus-capacity" to allow dynamically readjusting NFS Tracebus buffer capacity from outside EdenFS.

Reviewed By: genevievehelsel

Differential Revision: D40917315

fbshipit-source-id: 159b3b2747702d812cf1891b09b1905c2c177508
2022-11-02 21:02:48 -07:00
Milen Dzhumerov
34884008f5 Fix spurious notifications on ARM64
Summary:
# Problem

On ARM64, macOS will send nop/empty `setattr` requests. If we don't filter them, we end up incorrectly changing the `ctime` of the file and thus generating spurious/invalid change notifications.

This ends up slowing down buck2 builds as as part of each build, we end generating lots of spurious file change events and it invalidates buck2 daemon state incorrectly (as files haven't changed)

# Investigation

I set a breakpoint in `Nfsd3ServerProcessor` to figure out what was happening. As suspected, it was a completely empty/nop request P542900494

```
(facebook::eden::DesiredMetadata) desired = {
  size =  Has Value=false  {}
  mode =  Has Value=false  {}
  uid =  Has Value=false  {}
  gid =  Has Value=false  {}
  atime =  Has Value=false  {}
  mtime =  Has Value=false  {}
}
```

# Solution

Filter out empty `setattr` requests. As per discussion with chadaustin, we do not need to care about `atime` changes, so those are dropped.

Reviewed By: chadaustin

Differential Revision: D40798840

fbshipit-source-id: ef463d581bae89f905aa8ada3f6452e6eeb744d2
2022-10-28 16:38:40 -07:00
Chad Austin
53b9dc5d3f Back out "decouple ObjectFetchContext lifetime from RequestContext"
Summary:
Original commit changeset: abc1785f194a

Original Phabricator Diff: D40349610 (1eed4f0063)

Reviewed By: xavierd

Differential Revision: D40731896

fbshipit-source-id: 632c08fd3e44e8d2a126fdf688f41803354641a4
2022-10-26 15:07:38 -07:00
Chad Austin
1eed4f0063 decouple ObjectFetchContext lifetime from RequestContext
Summary:
Separately allocate ObjectFetchContext in RequestContext so that its
lifetime can extend longer than the request itself.

Reviewed By: kmancini

Differential Revision: D40349610

fbshipit-source-id: abc1785f194abb06383a2e72cb2ca71c266ae42b
2022-10-25 20:16:34 -07:00
Chad Austin
7ab1054530 simplify Nfsd3ServerProcessor request handling
Summary:
Remove an unnecessary context capture in NFS request handling. This is
a small simplification in preparation for changes to how
ObjectFetchContext is managed.

Reviewed By: genevievehelsel

Differential Revision: D40349525

fbshipit-source-id: 0b013d5485302b46959f6535692e116cb0aeea72
2022-10-25 20:16:34 -07:00
Chad Austin
4f6b77af28 remove RequestContext::makeSharedRequestContext
Summary:
Now that finishRequest is moved into the destructor, we no longer need
a custom shared_ptr constructor function.

Reviewed By: genevievehelsel

Differential Revision: D40349291

fbshipit-source-id: 1ac8c745cab4a09c7c5cf899803e5683d2aeedf7
2022-10-25 20:16:34 -07:00
Chad Austin
e120426cea minor ObjectFetchContext refactoring
Summary:
In advance of some trickier changes, make a few small improvements to
ObjectFetchContext.

Reviewed By: genevievehelsel

Differential Revision: D39901208

fbshipit-source-id: ed50141558201dd2666d55a21bf63db5bf2ffd43
2022-10-12 11:43:39 -07:00
Chad Austin
08086fafc0 migrate utils/Throw.h to fmt
Summary:
The main non-mechanical conversion from folly/Conv to fmt is replacing
folly::to<std:string>'s variadic concatenation operation with
fmt::join. Now that that's solved, we can slowly migrate the rest.

Reviewed By: genevievehelsel

Differential Revision: D39992102

fbshipit-source-id: 81df8e805c78ea5ca80ad5977f20b19e5a7467a3
2022-10-04 21:42:44 -07:00
Chad Austin
6857ef5676 split FsChannelStats into FuseStats, NfsStats, PrjfsStats
Summary:
This allows us to remove ifdefs and begin enforcing consistent
prefixes in StatsGroup.

Reviewed By: genevievehelsel

Differential Revision: D39782327

fbshipit-source-id: c453e0ada5d5d9d3b221c707f67dce7d1e0a970d
2022-09-30 16:48:23 -07:00
Chad Austin
50b9eafa6b remove the thread-local implementation detail from the public EdenStats API
Summary:
The public EdenStats API was both too chatty and exposed too many
implementation details. Don't mention thread-locals in the public API,
and replace uses of `addValue` with explicit `addDuration` on Duration
and `increment` on Counter.

Reviewed By: genevievehelsel

Differential Revision: D39661497

fbshipit-source-id: 8f6f2a9d72a4c385cde32e72fba0cebf8f5b4274
2022-09-30 11:35:05 -07:00
Mark Shroyer
4ca1838e1c Fix incorrect EOF value in TreeInode::readdirImpl
Summary:
If we run out of NFS buffer space while trying to add the final directory entry
in TreeInode::readdirImpl we would incorrectly return true, indicating that
we've returned the entire listing.

This means that on NFS we would incorrectly send an EOF to the client, which
would then not send a subsequent, offset readdir request.  Because of this, for
certain NFS-mounted Eden directory contents, calling `ls` would result in a
single file being missing from the listing.

Reviewed By: chadaustin

Differential Revision: D39911610

fbshipit-source-id: 96a8fc7958a8e800ff7f2489c968f2dde23a86da
2022-09-30 11:32:31 -07:00
Chad Austin
142166abaf light EdenStats refactoring
Summary:
Rename "channel" to "fs channel", rename StatPtr to DurationStatPtr,
and generalize DurationStatPtr to every stat category.

Reviewed By: xavierd

Differential Revision: D39631597

fbshipit-source-id: 147f9a9d9fe97a39c0d38ccecf57e73c93af0d8c
2022-09-23 17:45:28 -07:00
Katie Mancini
b44082e107 remove folly::Format InodeNumber
Summary:
From the folly format docs
> Use fmt::format instead of folly::format for better performance, build times and compatibility with std::format

eden build times have gotten a bit high, cutting out folly format will help reduce build time, so let's start
by banishing it from eden

buck2 build times have been un-reliable, I have been trying `--local-only` and `buck clean` but the build times seem too goo for clean builds and are all over the place. Will try more throughly clearing caches. I think that folly/format.h is still being linked  due to recursive dependencies, so I am looking into making sure it at least isn't being recursively included in any of our headers.

Reviewed By: xavierd

Differential Revision: D38250995

fbshipit-source-id: b02d8c458fe92a068426031cffd20ef01ae0edf7
2022-08-02 11:44:19 -07:00
Chad Austin
265128f4f4 remove some conditional includes
Summary:
To simplify the build and fall in line with Folly's conventions,
remove several conditional includes. Instead, headers conditionally
define symbols.

Reviewed By: xavierd

Differential Revision: D37897850

fbshipit-source-id: 8b52d5310f5cd8cb17fdc013117c271ae09dd5d6
2022-07-19 12:37:57 -07:00
Mark Shroyer
b02c805973 Fix incorrect lock use in range-based for loops
Summary:
LockedPtrs used as temporaries within range expressions may go out of scope
before the end of the loop.

https://en.cppreference.com/w/cpp/language/range-for:

> If range-expression returns a temporary, its lifetime is extended until the
> end of the loop, as indicated by binding to the forwarding reference __range,
> but beware that the lifetime of any temporary within range-expression is not
> extended.

Reviewed By: xavierd

Differential Revision: D37904242

fbshipit-source-id: 915812e7bb4d19c333dc4c309261610884f8e61d
2022-07-18 10:22:18 -07:00
Chad Austin
b12c397618 compilation fixes for gcc
Summary:
GCC provides some useful warnings and errors when compiling EdenFS,
and I noticed our open source build was failing. Fix some of the
issues.

Reviewed By: xavierd

Differential Revision: D37157074

fbshipit-source-id: de5705ec2eac18cd269143739289031ca5d478fc
2022-06-23 15:45:36 -07:00
Chad Austin
9fa292b9ed standardize namespaces on C++17 syntax
Reviewed By: genevievehelsel

Differential Revision: D36429182

fbshipit-source-id: 7d355917abf463493c37139856810de13e1090ff
2022-05-17 10:12:56 -07:00
Katie Mancini
d1953ac1e3 fix IOBuf manipulations
Summary:
We trim off the fragment header from the NFS requests we get. Sometimes during
the test_takeover_with_io tests this was crashing in this code with:

```
F0418 20:55:24.216660 25733 IOBuf.h:765] Check failed: amount <= length_ (4 vs. 1)
*** Check failure stack trace: ***
*** Aborted at 1650340524 (Unix time, try 'date -d 1650340524') ***
*** Signal 6 (SIGABRT) (0x50aa) received by PID 20650 (pthread TID 0x7f5f83aa9700) (linux TID 25733) (maybe from PID 20650, UID 0) (code: -6), stack trace: ***
    @ 00000000000192fd folly::symbolizer::(anonymous namespace)::innerSignalHandler(int, siginfo_t*, void*)
                       ./fbcode/folly/experimental/symbolizer/SignalHandler.cpp:449
    @ 00000000000102ab folly::symbolizer::(anonymous namespace)::signalHandler(int, siginfo_t*, void*)
                       ./fbcode/folly/experimental/symbolizer/SignalHandler.cpp:470
    @ 0000000000000000 (unknown)
    @ 000000000003e530 __GI_raise
    @ 000000000002551c __GI_abort
    @ 000000000000dbac google::LogMessage::Fail()
                       /home/engshare/third-party2/glog/0.3.2_fb/src/glog-0.3.2/src/logging.cc:1519
    @ 00000000000109f2 google::LogMessage::SendToLog()
                       /home/engshare/third-party2/glog/0.3.2_fb/src/glog-0.3.2/src/logging.cc:1473
    @ 000000000000d7f2 google::LogMessage::Flush()
                       /home/engshare/third-party2/glog/0.3.2_fb/src/glog-0.3.2/src/logging.cc:1346
    @ 000000000000eaa8 google::LogMessageFatal::~LogMessageFatal()
                       /home/engshare/third-party2/glog/0.3.2_fb/src/glog-0.3.2/src/logging.cc:2013
    @ 000000000009dead folly::IOBuf::trimStart(unsigned long)
                       ./buck-out/v2/gen/fbcode/8107dfcc75db0fad/folly/io/__iobuf__/headers/folly/io/IOBuf.h:765
                       -> ./fbcode/eden/fs/nfs/rpc/Server.cpp
    @ 000000000009ce3f facebook::eden::RpcTcpHandler::tryConsumeReadBuffer()::$_0::operator()()
                       ./fbcode/eden/fs/nfs/rpc/Server.cpp:243
    @ 000000000009bedc void folly::detail::function::FunctionTraits<void ()>::callSmall<facebook::eden::RpcTcpHandler::tryConsumeReadBuffer()::$_0>(folly::detail::function::Data&)
                       ./buck-out/v2/gen/fbcode/8107dfcc75db0fad/folly/__function__/headers/folly/Function.h:364
                       -> ./fbcode/eden/fs/nfs
...
...
...
y/__named_thread_factory__/headers/folly/executors/thread_factory/NamedThreadFactory.h:40
                       -> ./fbcode/configerator/distribution/api/ConfigeratorStaticData.cpp
    @ 000000000033b32c void std::__invoke_impl<void, folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::{lambda()#1}>(std::__invoke_other, folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::{lambda()#1}&&)
                       ./fbcode/third-party-buck/platform009/build/libgcc/include/c++/trunk/bits/invoke.h:60
                       -> ./fbcode/configerator/distribution/api/ConfigeratorStaticData.cpp
    @ 000000000033b2bc std::__invoke_result<folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::{lambda()#1}>::type std::__invoke<folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::{lambda()#1}>(std::__invoke_result&&, (folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::{lambda()#1}&&)...)
                       ./fbcode/third-party-buck/platform009/build/libgcc/include/c++/trunk/bits/invoke.h:95
                       -> ./fbcode/configerator/distribution/api/ConfigeratorStaticData.cpp
    @ 000000000033b294 void std:🧵:_Invoker<std::tuple<folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::{lambda()#1}> >::_M_invoke<0ul>(std::_Index_tuple<0ul>)
                       ./fbcode/third-party-buck/platform009/build/libgcc/include/c++/trunk/thread:244
                       -> ./fbcode/configerator/distribution/api/ConfigeratorStaticData.cpp
    @ 000000000033b264 std:🧵:_Invoker<std::tuple<folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::{lambda()#1}> >::operator()()
                       ./fbcode/third-party-buck/platform009/build/libgcc/include/c++/trunk/thread:251
                       -> ./fbcode/configerator/distribution/api/ConfigeratorStaticData.cpp
    @ 000000000033b0ed std:🧵:_State_impl<std:🧵:_Invoker<std::tuple<folly::NamedThreadFactory::newThread(folly::Function<void ()>&&)::{lambda()#1}> > >::_M_run()
                       ./fbcode/third-party-buck/platform009/build/libgcc/include/c++/trunk/thread:195
                       -> ./fbcode/configerator/distribution/api/ConfigeratorStaticData.cpp
    @ 00000000000d9660 execute_native_thread_routine
    @ 000000000000920b start_thread
    @ 000000000011816e __GI___clone
```

The reason we are crashing is that the first IOBuf in the chain does not
necessarily contain the full fragment header. So we might attempt to trim off
more data than is in the IOBuf. We can fix this my trimming from the IOBuf chain
instead.

TBH idk why only this test triggers the problem, it seems like eden should
crash more often than that. But it should be fixed now anyways.

Reviewed By: xavierd

Differential Revision: D35863737

fbshipit-source-id: 21640b252a703fe4fa52f66111e6ae50a94bc347
2022-05-06 22:31:51 -07:00