Summary:
To support better telemetry and logging in watchman we want to use Eden's components. Lets migrate and detangle the needed pieces.
This change moves SystemError from eden to edencommon.
Reviewed By: MichaelCuevas
Differential Revision: D54343729
fbshipit-source-id: 7861e3effc9d242fbeda34333078c14c4d021a80
Summary:
To support better telemetry and logging in watchman we want to use Eden's components. Lets migrate and detangle the needed pieces.
This change moves CaseSensitvity from eden to edencommon.
Reviewed By: fanzeyi
Differential Revision: D54339283
fbshipit-source-id: f96a421f4390578e5d2281b307532c62e22935d3
Summary:
To support better telemetry and logging in watchman we want to use Eden's components. Lets migrate and detangle the needed pieces.
This change moves ImmediateFuture from eden to edencommon.
Reviewed By: fanzeyi
Differential Revision: D54337631
fbshipit-source-id: 2e09a9e244ef8b8b662959c1b14db3e103f559d0
Summary:
To support better telemetry and logging in watchman we want to use Eden's components. Lets migrate and detangle the needed pieces.
This change moves Throw.h and it's related tests from eden to edencommon.
Reviewed By: genevievehelsel
Differential Revision: D54046153
fbshipit-source-id: 669d702c13e70536d9c0b58ff8ff17f826237851
Summary:
`-Wextra-semi` or `-Wextra-semi-stmt` found an extra semi
If the code compiles, this is safe to land.
Reviewed By: MichaelCuevas
Differential Revision: D53776196
fbshipit-source-id: f83dda6df32622f1ced24d051eadeaa8ebfda4e8
Summary:
Most of it was replaced with a ManualExecutor. Some care had to be taken around
the use of StoredObject as triggering an error on them when no future was
obtained from it leads to the error not being triggered when the future is then
retrieved from it.
The only place where QueuedImmediateExecutor is still present is in the
ImmediateFutureTest to demonstrate the lack of safety with it.
Reviewed By: genevievehelsel
Differential Revision: D51302393
fbshipit-source-id: fb66a9426e0ae7672799bd997898e8b525ff112c
Summary:
I'm planning on re-using this thread pool in both PrjFS and FUSE, let's thus
move it to a more generic location.
Reviewed By: genevievehelsel
Differential Revision: D51302395
fbshipit-source-id: 39a503fd31a7b7464fc0a38f8d4152e33808293f
Summary:
This is unecessary, the threadPool executor can be used instead to achieve the
same outcome.
Reviewed By: kmancini
Differential Revision: D50720495
fbshipit-source-id: 4661c1a2433e90cd1d0e0a953aea181c22c8b2cf
The internal and external repositories are out of sync. This Pull Request attempts to brings them back in sync by patching the GitHub repository. Please carefully review this patch. You must disable ShipIt for your project in order to merge this pull request. DO NOT IMPORT this pull request. Instead, merge it directly on GitHub using the MERGE BUTTON. Re-enable ShipIt after merging.
Summary: In preparation for merging ProcessSimpleName and ProcessName functionality into ProcessNameCache renaming now The new cache will cache ProcessInfo object which will include both names, and the parent pid.
Reviewed By: genevievehelsel
Differential Revision: D48855222
fbshipit-source-id: 4cb10df7b6cc32efc0d655771d01d58e7ba57dd5
Summary:
While investigating T153138578, I hoped to find an obvious place where
we called a function from the wrong thread. I couldn't find any, but
make sure we document the functions that we expect to be called from
the EventBase thread.
Reviewed By: xavierd
Differential Revision: D47682474
fbshipit-source-id: 372123f06eb9efa9c97c2af8fb63ae50fe9054fc
Summary:
Server.h is such a generic name to call up in myles, and now the class
name matches the header name.
While I was touching these files, I broke a couple #include dependencies.
Reviewed By: xavierd
Differential Revision: D47681923
fbshipit-source-id: 46c1b8ce130ba63c7bd6e44db31d0719f57e81bf
Summary:
Looking at the root cause of recent crashes, one of the main one appears to be
a use after free in RpcServer::takeoverStop. The local root cause of which is
caused by the use of a `this` capture, moving code around makes the use after
free disappear.
Taking a step back and trying to understand what the actual root cause of the
bug is is somewhat tricky, but my suspicion has to do with
`EdenMount::fsChanelInitSuccessful` where the `FsChannel` is being destroyed
manually. In the case where `Nfsd3::destroy` enqueues the `delete this` before
the `takeoverStop` enqueues the lambda we could end up in a situation where
Nfsd3 is freed (and thus the RpcServer destroyed) before `takeoverStop`
completes and thus causing this bug.
Reviewed By: kmancini
Differential Revision: D46912289
fbshipit-source-id: f0fd1c6daf897d5fdd8169bc1cfd9fb667d97afd
Summary:
WIN32 is defined by windows.h. _WIN32 is defined by the compiler. The
latter is more reliable under refactoring.
Reviewed By: xavierd
Differential Revision: D47528663
fbshipit-source-id: 494f18d94968d9c8ea38fe395881e87c0368728e
Summary:
off_t is not suitable on Windows because it is a 32-bit signed
type. Introduce a new FileOffset type for use in EdenFS.
Reviewed By: kmancini
Differential Revision: D47406784
fbshipit-source-id: 9cc9585b4b118c54a1ca148c2ba7e7ccfb6f278c
Summary:
Partially enables symlink support on Windows by making symlinks appear on Windows as actual symlinks as opposed as regular files containing only the place where the symlink would point in other systems.
Creating new commits with symlinks also works (after editing `fscap.py` on hg's side as well as the requirements file for the current directory for enabling symlinks on Windows EdenFS) when the symlinks are in the same directory.
Reviewed By: xavierd
Differential Revision: D44218035
fbshipit-source-id: 0e3094dc5a13cabef1cd24f8fe18cc73ca40d4a8
Summary:
Aiming towards moving unmount() into FsChannel, move the knowledge of
how to unmount an NFS mount into Nfsd3.
To support unmounting on Windows, we can eventually add an invocation
of unmount.exe to the Windows PrivHelper implementation.
Reviewed By: kmancini
Differential Revision: D45296963
fbshipit-source-id: 55fa7fe0f6190d3708caa21a0cd4b3868f464f8b
Summary:
Provided implementation for EdenMount::chown on NFS (MacOS). Utilized exisiting code to get all referencered Inodes and then loaded and invaliated them.
Ran into some issues compiling Ranges - something to learn more about - where certain iterator traits needed to be defined. In subsquent diff should be able to address (if desired), but for this diff just used a vector instead.
Reviewed By: MichaelCuevas
Differential Revision: D46946414
fbshipit-source-id: b405f16712d6d6315d43d6973e7615aced7e5c7d
Summary:
waitForPendingWrites is general to any userspace filesystem with
asynchronous write notifications. This could even include FUSE in
writeback caching mode. To remove another ifdef, move
waitForPendingWrites into FsChannel.
Reviewed By: kmancini
Differential Revision: D45167206
fbshipit-source-id: 4daec70845864d90b94739fe2011e8ed90a6a200
Summary:
We almost never intend to include folly/String.h, especially in
headers, and it's a somewhat expensive header.
We sometimes use the string transformation functions in .cpp files,
but rarely in .h. Therefore, remove that dependency where we can.
Reviewed By: genevievehelsel
Differential Revision: D45672957
fbshipit-source-id: 11743156388aff5c61cfe6b46e385a95687a7a31
Summary: This include is no longer necessary, so we can remove it.
Reviewed By: genevievehelsel
Differential Revision: D45620895
fbshipit-source-id: d8d9ae00cc38ad1331f8f6fb1f1c55b9d4276095
Summary: There's nothing unix-specific about these tests, so enable them on Windows.
Reviewed By: genevievehelsel
Differential Revision: D45586168
fbshipit-source-id: 5acbcbe81c5c03b16e43fc561762dbafb47696b8
Summary:
After a takeover, the new socket was initialized with a "remote
EventBase", causing stopAccepting to be asynchronous, tripping an
assertion.
There was a latent bug here: it might have been possible for a new
connection to be accepted by the old process after takeover begun.
Reviewed By: xavierd
Differential Revision: D45584154
fbshipit-source-id: cdd8118d82100bc1e19f6f19676cc2fa412b8775
Summary:
As a follow-up to a code review comment in a previous diff, move
acceptStopped into the EventBaseState to make it clear it's only
accessed on the EventBase thread.
Reviewed By: kmancini
Differential Revision: D45106980
fbshipit-source-id: 5bd201a0652fa2b539e18a7584fae48fd3abd54f
Summary:
FuseChannel already had an asynchronous initialize(). Lift that to
FsChannel and unify initialization of Prjfs and NFS.
Reviewed By: kmancini
Differential Revision: D45106889
fbshipit-source-id: 632ddfa275c732fc30efc1984bda43f61c37fd5e
Summary:
EdenMount shouldn't know about the vagaries of FsChannel teardown. In
fact, FuseChannel already has its own requirements, hidden behind a
unique_ptr deleter. Use the same mechanism for NFS, allowing us to
remove a .via() special-case.
Reviewed By: kmancini
Differential Revision: D44810223
fbshipit-source-id: db2f88bc3843b3842de60c5a993dfd9890dc3c15
Summary:
After my previous refactoring, the memory safety bug in Nfsd3 no
longer manifests, and we can remove the unnecessary defer call.
Reviewed By: kmancini
Differential Revision: D44988749
fbshipit-source-id: 2ebd5fbf96d05ce6850fc248a642409f827ffeec
Summary:
All of the comments in RpcServer are helpful. I found a couple
opportunities to improve their clarity.
It also helps to decouple RpcStopReason from the connection status.
Reviewed By: kmancini
Differential Revision: D44988608
fbshipit-source-id: 967f35ba09533a9c8c953ef33cd595b51e7c3111
Summary:
I'm carefully reading the takeover code and noticed a few things to
improve.
Reviewed By: kmancini
Differential Revision: D44946740
fbshipit-source-id: 558441a47425b15c19ca2a6816ae2e8bf3d68019
Summary:
I want to use EventBaseState in TakeoverServer, so move it from
RpcServer to eden/fs/utils.
Reviewed By: kmancini
Differential Revision: D44945418
fbshipit-source-id: b9738162c84ca20da800d3662be6271f4b492acc
Summary:
For small handle types, passing by value is the correct choice,
because it indicates unconditional ownership transfer. (It also
generates slightly smaller code and is less typing.)
Reviewed By: xavierd
Differential Revision: D44927835
fbshipit-source-id: 5fefee7889e8216e46548f4801e83f51cde8b8c4
Summary:
No need to have an intermediate object to forward callbacks from
ReadCallback to RpcConnectionHandler. This simplifies both memory
management and data flow.
Reviewed By: kmancini
Differential Revision: D44860344
fbshipit-source-id: af02f7d7f331d6ff3a8c4f49abb5ba1b95db5f85
Summary:
UDS don't have port numbers, so trying to register their port with the
portmapper is going to fail. This can cause eden to fail to start when we
enable uds. The current portmapper version we use can't support
this, so let's just skip attempting to register uds servers with the
portmapper.
Reviewed By: xavierd
Differential Revision: D45196379
fbshipit-source-id: 584e6e04129d1ddbcc756ed66c55a03c3955705a
Summary:
We don't need folly::Synchronized for the RpcServer state
machine. It's only touched from the EventBase, so make it
EventBaseState.
Reviewed By: xavierd
Differential Revision: D44860240
fbshipit-source-id: bcbc3d02e7cc5f2dc377718065c7f95eed30202d
Summary:
There's nothing TCP-specific about RpcTcpHandler, so give it a more
accurate name.
Reviewed By: xavierd
Differential Revision: D44860198
fbshipit-source-id: 55b9dc95b2caa9d87179a17435313b91bacfee49
Summary:
RpcServer can handle the accept callbacks directly. There's no need
for an intermediate object.
Reviewed By: xavierd
Differential Revision: D44860163
fbshipit-source-id: fd2d96735810a52cd287c357534c8f5bae9eee90
Summary:
I plan to use StateWrapper elsewhere so rename it to EventBaseState
and move it to the top level.
Reviewed By: xavierd
Differential Revision: D44860040
fbshipit-source-id: fb03b3b726df50099bcce144e343be0caeeaf1be
Summary:
There is no need for an intermediate object: RpcTcpHandler can handle
the write callback methods directly.
Reviewed By: xavierd
Differential Revision: D44860027
fbshipit-source-id: fe1f44b83a371291a21a3b1b26422bef5cbbf2a0
Summary:
To simplify RpcServer further, assert that removal of the accept
callback is a synchronous operation. Previously, it wasn't, because
specifying any EventBase in addAcceptCallback created a RemoteAcceptor
which scheduled operations on the other EventBase.
Reviewed By: xavierd
Differential Revision: D44853408
fbshipit-source-id: 9369cdcb0bd8450d42e195c15738362d4a53d2e7
Summary:
I'm trying to track down a subtle lifetime rule violation in
EdenMount. While reading RpcServer, I noticed a few possible
simplifications.
Reviewed By: xavierd
Differential Revision: D44849773
fbshipit-source-id: 4c27c47a7e2c211dcd040acce955e9b2f617b55b
Summary:
By moving NFS's StopData behind FsStopData, the std::variant and
runtime checks during takeover can be unified into one path. The next
diff will do the same for Windows.
Reviewed By: kmancini
Differential Revision: D44737160
fbshipit-source-id: 19ca623f90b367d25385f177cafa44d7bf7ebb62
Summary:
Now that we have the plumbing we can start registering our NFS RPC servers (
mountd and nfsd) with the portmapper/rpcbind.
I am taing a short cut for prototyping. I am skipping general purpose
rpc registration (i.e. making these rpc calls to register a service). This is
because (1) I am trying to prototype quickly, so I don't want to implement the
portmapper endpoints that allow registering/ unregistering a server and (2)
there are maybe security considerations, and I don't want to think about those
rn.
Reviewed By: chadaustin
Differential Revision: D44987884
fbshipit-source-id: 4b849d1dd060d2e033ab48df883e228015906a7d
Summary:
getport is the primary endpoint on the rpcbind/portmap server. NFS clients
make a request on this endpoint to find the NFS RPC servers.
In this diff we keep a mapping of RPC servers and respond to the getport
request based on this mapping.
Currently no rpc severs are registered, so we reply no port to all
requests.
Reviewed By: chadaustin
Differential Revision: D44987893
fbshipit-source-id: 920171c6007b82609ae84fb7a9aff43d898e43dd
Summary:
The msft client can't be pointed at an NFS server on a particular port, so
we have to register our RPC servers (mountd and nfsd) with the port mapper
service.
There is no portmapper service running on Windows by default.
There is a port mapper implementation from msft that can be installed on
Windows Server, but not on Windows desktop.
There are some third-party implementations of the portmapper for Windows. Most
of them are part of an NFS Server implementation. But none that I have found
both run on modern Windows versions, are a general purpose port mapper, and
support port mapper v2. General purpose means they allow processes to set a
port mapping. Most the port mappers I found must register their NFS server in
process, because they don't support calling the set mapping endpoint. The rest
only run on Windows versions from before I was born or only run v4.
I tried porting the linux gnu implementation of port mapper to Windows. But despite
a few hours of trying I could not even get it to build even on Linux. Writing our own port mapper is faster.
This is the skeleton of the server. In the next diffs I will register our
RPC servers and support telling the msft NFS client about our servers.
Reviewed By: chadaustin
Differential Revision: D44987863
fbshipit-source-id: bff065795a9f4b7b6c13ef3e3ce603646e1ce364
Summary:
The msft NFS client likes to speak portmapper v2. It does not seem to know v4.
Perhaps there is a way to configure that, but I have not found it yet.
v4 is the latest version of the protocol and so what we should prefer using on
mac and linux.
Here I am introducing the v2 protocol and renaming all the v4 types to
explicitly reference v4.
Reviewed By: chadaustin
Differential Revision: D44987848
fbshipit-source-id: 97f9e2365736b49a5888af204609598b2566de55
