Summary:
There is no good reason for this function to be in a header, let's move it to
StringConv.cpp
Reviewed By: genevievehelsel
Differential Revision: D24000882
fbshipit-source-id: 5bb6bc3b9ef37232d38b8e35da693e12c0453ea1
Summary:
Replace it by simply using std::chrono facilities to achieve the exact same
behavior.
Reviewed By: chadaustin
Differential Revision: D24027637
fbshipit-source-id: d22eefec5d408ead0b4cfaa20e716f4c10cce0b5
Summary:
These don't compile on Windows, and in order to get mode/win to compile, we
need to avoid compiling their contents. Ideally, we could do that in the
TARGETS files with the select statement, but that's not available in fbcode.
Thus, we do the next best thing: ifdef the file entirely.
Reviewed By: wez
Differential Revision: D23871728
fbshipit-source-id: b4d9df6503eaa008e649afd7bdc665cd37a9585d
Summary:
*shakes fist at C++ copy-constructors*
We weren't guaranteeing that case-insensitive status was being propagated on
copies or moves, which meant that eg: `lookup("workspace")` would be treated as
case-sensitive when mounted case insensitively.
Reviewed By: xavierd
Differential Revision: D23857218
fbshipit-source-id: 67e33a8455a0a85e5885389b5bb38b20ef043894
Summary: This will make it easier to build with Buck.
Reviewed By: fanzeyi
Differential Revision: D23827754
fbshipit-source-id: bf3bf4d607a08b9831f9dfea172b2e923a219561
Summary:
on macOS we cannot safely use `fork`.
This commit replaces the use of `fork` in the startup logger subsystem.
This was a little tricky to untangle; originally (prior to any of
the `fork` removal efforts in this diff stack), the startup flow was
to spawn a set of processes via fork:
```
edenfs (setuid)
\-----edenfs (privhelper, as root)
\------edenfs (daemonized)
```
The forked children take advantage of being able to implicitly pass state to
the child processes from the parent. That data flow needs to become explicit
when removing the fork which makes some things a little awkward.
With fork removed:
* `edenfs` unconditionally spawns `edenfs_privhelper` while it has
root privs and before most of the process has been initialized.
* That same `edenfs` process will then spawn a child `edenfs`
process which starts from scratch, but that which needs to
run as the real server instance
* The original `edenfs` instance needs to linger for a while
to remain connected to the controlling tty to pass back the
startup state to the user, before terminating.
This commit deletes the check that `edenfs` is started originally
as root; previously the logic relied on the forked startup logger
continuing past the `daemonizeIfRequested` call and simply deferring
the check until after folly::init. With these changes we can't
easily perform such a check without adding some extra gymnastics
to pass the state around; the place where that is checked is in
the spawned child of the original edenfs, which is not a privileged
process and doesn't know the original euid. I don't believe this
to be a great loss as we tuck `edenfs` away under the libexec dir.
Reviewed By: chadaustin
Differential Revision: D23696569
fbshipit-source-id: 55b95daf022601a4699274d696af419f0a11f6f2
Summary:
On macOS we cannot safely use `fork` to spawn processes while other threads may initialize objc classes.
This commit replaces the use of `fork` in the privhelper startup with
`SpawnedProcess` instead. We need to take care with this as we are generally
installed setuid root and we'd like to avoid being tricked into running an
arbitrary child process as root.
This commit defines a separate executable called `edenfs_privhelper` that
contains just the privhelper server code.
We need to be careful about locating this executable; to avoid invoking an
arbitrary process while we have root privileges we require that the privhelper
be a sibling to the edenfs executable and carry out some additional ownership
verification so that we can tell that the owner of edenfs also controls
edenfs_privhelper.
To facilitate this, I've added an `executablePath` function to PathFuncs; it
returns the path to the current executable image.
To make the integration test scenario simpler, I've added the edenfs_executable
binary definition alongside that of the edenfs binary in the buck and cmake
build systems. This causes the binaries to be siblings in-situ in the build
tree and avoids the need to move things into place in the test harness.
Reviewed By: chadaustin
Differential Revision: D23653343
fbshipit-source-id: 3c2539a5e0e11cee88960db49c885ce0366d314e
Summary:
This commit moves a compile-time template parameter
to be a runtime boolean parameter.
There's a bit of fan-out that, while I don't think it is
super awesome, isn't super terrible either.
The case sensitivity value is read from the checkout config
added in the prior diff in this stack.
Reviewed By: xavierd
Differential Revision: D23751192
fbshipit-source-id: 46f6fe25bfa6666305096ad9c416b510cd3aac8f
Summary:
I had originally make the logic around ECHILD very strict,
thinking it impossible to have a situation where it may arise,
but it turns out that our daemonization makes this happen all
the time.
This commit treats an ECHILD return from waitpid as equivalent
to a success waitpid result and success child process termination.
Reviewed By: chadaustin
Differential Revision: D23683107
fbshipit-source-id: 7867d636afd8ee79b9f100454f84e7ef480109d8
Summary:
Since the Stub.h now only contains NOT_IMPLEMENTED, let's move it to its own
header outside of the win directory.
Reviewed By: genevievehelsel
Differential Revision: D23696244
fbshipit-source-id: 2dfc3204707e043ee6c89595668c484e0fa8c0d0
Summary:
While the code isn't compiled, this makes the thrift definition available to
the rest of the code, eliminating the need for having a stub for
SerializedInodeMap on Windows.
Reviewed By: genevievehelsel
Differential Revision: D23696242
fbshipit-source-id: 8a42dd2ed16887f3b7d161511e07aaa35fd1b968
Summary:
While hacking on some code, I ran into a situation where some
zero-initialized stat structs weren't actually being zeroed. This was
either a compiler bug or a situation where the build system was not
correctly rebuilding everything after my changes, and I did not have
enough disassembly available to investigate.
Either way, since this code assumes zero bits in some nonobvious ways,
explicitly assert they are.
Reviewed By: xavierd
Differential Revision: D23644819
fbshipit-source-id: eb6bff9ff997379113db1e1bf9d6a0a538f10f0b
Summary: This fixes the `The program issued a command but the command length is incorrect` error we have been seeing in EdenFS Windows.
Reviewed By: wez
Differential Revision: D23589264
fbshipit-source-id: 72478a653ff07fca81c163bab12c0fd33f6a0a67
Summary:
Even though these might not be fully ported on Windows, they do compile and
tests are passing, so let's compile them.
Reviewed By: chadaustin
Differential Revision: D23505509
fbshipit-source-id: 567e8668ca489daf89c1c6576973bbaaabbb6c88
Summary:
Most of the RequestData code is platform generic, but bits of it are currently
strongly tied to FUSE. By splitting these 2 parts, we will be able to use the
RequestContext class in Windows too and not having to re-implement all the
logic there.
Reviewed By: chadaustin
Differential Revision: D23482072
fbshipit-source-id: 857fd9ca4264d0f308ec10cc487e9ff3eeb5ee16
Summary:
Removing Fuse from the enum name makes it non tied to Fuse and thus makes it
more portable. This also eliminates the last platform specific bit from
RequestData.
Reviewed By: chadaustin
Differential Revision: D23467773
fbshipit-source-id: 52515522c8ac51d0c4b56dc5e42d4b6593df6623
Summary:
The loop took care to advance `b` to match the amount
of data that it had processed, but was still passing `buf`
(the unadjusted start of the buffer) to the syscalls.
This meant that in situations where a `readFull` might
encounter a partial read, it would scribble over the start
of the buffer and leave junk at the end.
For example:
write("hell");
write("o");
could produce "oell?" in the buffer when `readFull` consumes
the other end of the pipe.
Reviewed By: xavierd
Differential Revision: D23486270
fbshipit-source-id: 0848f6789b44421b609b91fe08890768ff59f7f5
Summary: The latter will not strip new lines from the system error message, while the former does.
Reviewed By: genevievehelsel
Differential Revision: D23480435
fbshipit-source-id: 44742b960935552fa1781ed19f38ff446a8c9403
Summary:
This is not per-se fuse related, thus move it to a common location and remove
the duplicated define in FileInode.h
Reviewed By: chadaustin
Differential Revision: D23465192
fbshipit-source-id: 5fa7709f127c2d3372ee5ea3aeb89e793ea5b9f7
Summary:
This commit adds a compile time option to select
between case-sensitive and case-insensitive-but-case-preserving
mode for `PathMap`.
This replaces the `ifdef _WIN32` preprocessor conditional
that was inline in a couple of the methods and allows
explicitly testing the behavior in both modes of operation.
The unit tests have been expanded and rounded out to catch
some inconsistent behavior; insertion wasn't respecting
case insensitivity in all ... cases.
Hopefully we not relying on that behavior in the windows
flavor of the build; let's see what our CI says.
Reviewed By: genevievehelsel
Differential Revision: D23232629
fbshipit-source-id: 96e752e501d0398ec2bed5879f7c11c7ab6e1d70
Summary:
I switched these from ERR to DBG2 thinking that would
take them out of the log by default, but we log DBG2 by default
so that didn't have the desired result.
This changes the log level to DBG6 which is an arbitrary log
level that isn't included by default.
Reviewed By: genevievehelsel
Differential Revision: D23467738
fbshipit-source-id: dd0c75f86318ece27313c237938f24f55758eec1
Summary:
This commit introduces a new process spawning class derived
from the ChildProcess class in the watchman codebase.
`SpawnedProcess` is similar to folly::Subprocess but is designed around the
idea that we will use a system provided spawning API to start a process, rather
than assuming the use of `fork`.
`fork` is to be avoided because it can be expensive for processes with large
address spaces and also because it interacts poorly with threads on macOS. In
particular, we see the objC runtime terminating our process in some scenarios
where fork and threads are mixed.
There are some important differences from `folly::Subprocess` and that means
that some assumptions and uses need to be altered slightly from their prior
workings. For example, detaching a SpawnedProcess moves the responsibility of
waiting on the child to a periodic task as there is no way to detach via
posix_spawn without also using fork.
On the plus side, this commit allows unifying spawning between posix and
windows systems, which simplifies the code!
Reviewed By: xavierd
Differential Revision: D23287763
fbshipit-source-id: b662af1d7eaaa9ed445c42f6c5765ae9af975eea
Summary:
This commit introduces a few types from the watchman codebase:
`FileDescriptor` which is on posix systems represents a file descriptor,
and on Windows is a HANDLE (which can be a file, pipe or socket descriptor).
`Pipe` is a convenience struct that holds the read and write ends of a Pipe.
Note that we have a conceptual class with a windows specific Pipe type under
eden/fs/win/utils/Pipe.h; I remove that in the next diff in the stack.
There are a couple of differences from the watchman code
Reviewed By: chadaustin
Differential Revision: D23287819
fbshipit-source-id: 6ca90ba345037c6c3e308f588d690a899c9866a5
Summary:
std::iterator is deprecated in C++17. Removing it fixes warnings in
the Clang/Windows build.
Reviewed By: genevievehelsel
Differential Revision: D23352927
fbshipit-source-id: 293e30909eaa8a7c4856a91930a3886ad0b19364
Summary:
Up to now, Windows had to have its own version of folly::{readFile, writeFile,
writeFileAtomic} as these only operate on `char *` path, which can only
represent ascii paths on Windows. Since the Windows version is slightly
different from folly, this forced the code to either ifdef _WIN32, or use the
folly version pretending that it would be OK. The Windows version was also
behaving slightly differently from folly. For instance, where folly would
return a boolean to indicate success, on Windows we would throw an exception.
To simplify our code, add type safety and unify both, we can implement our own
wrappers on top of either folly or Windows APIs.
We still have some code that uses folly::readFile but these should only be
dealing with filedescriptors. As a following step, we may want to have our own
File class that wraps a file descriptor/HANDLE so we can completely remove all
uses of folly::readFile.
Reviewed By: wez
Differential Revision: D23037325
fbshipit-source-id: 2b9a026f3ee6220ef55097abe649b23e38d9fe91
Summary:
Instead of modifying the existing APIs and marking all callsites as deprecated in one diff, I am going to take the "add and remove" approach, where I will add the deprecated version of methods first, then mark all callsites, and finally remove the existing ones. This provides some backward compatibility without breaking things.
This diff is to add deprecated getSelection APIs to ServiceSelectorCache.
Differential Revision: D22981269
fbshipit-source-id: 6e3025e7f7df6ee7f9e1cba9dc036ca84adbe49a
Summary:
The StringConv.h header contains many functions to convert from Windows paths
to Eden's path (and vice versa) to workaround the fact that Eden's path don't
support wide strings that Windows uses. Let's simply add support for these wide
strings in PathFuncs so we can greatly simplify all the call sites. Instead of
calling "edenToWinName(winstr)", "PathComponent(winstr)" is both more
descriptive and more idiomatic.
To be fair, I'm not entirely a fan of the approach taken in this diff, as this
adds Windows specific code to PathFuncs.h, but I feel that the benefit is too
big to not do that.
Reviewed By: chadaustin
Differential Revision: D23004523
fbshipit-source-id: 3a1507e398a66909773251907db01e06603b91dd
Summary:
Previously, fetch heavy event's cmdline was delimited by '\x00' when logged to Scuba. (for example: `grep--color=auto-rtest.`)
Now we replace \x00 with a space, so command name and args will be separated by space. ( `grep --color=auto -r test .` )
Reviewed By: kmancini
Differential Revision: D22772868
fbshipit-source-id: 4ab42e78c7bc786767eee3413b9586739a12e8ac
Summary:
We are unifying C++ APIs for accessing optional and unqualified fields:
https://fb.workplace.com/groups/1730279463893632/permalink/2541675446087359/.
This diff migrates code from accessing data members generated from unqualified
Thrift fields directly to the `field_ref` API, i.e. replacing
```
thrift_obj.field
```
with
```
*thrift_obj.field_ref()
```
The `_ref` suffixes will be removed in the future once data members are private
and names can be reclaimed.
The output of this codemod has been reviewed in D20039637.
The new API is documented in
https://our.intern.facebook.com/intern/wiki/Thrift/FieldAccess/.
drop-conflicts
Reviewed By: yfeldblum
Differential Revision: D22631599
fbshipit-source-id: 9bfcaeb636f34a32fd871c7cd6a2db4a7ace30bf
Summary:
While there isn't a direct equivalent to the unix RSS and VSZ, the working set
size and the page file usage can be used instead, they roughly have the same
meaning and that's probably good enough.
I've also moved the private bytes into the MemoryStats as it's a memory stat
too.
Reviewed By: chadaustin
Differential Revision: D22081006
fbshipit-source-id: d597d523eee21e7651a14e60b1fd6dc152a61185
Summary:
Currently when we are resolving the full command line for a client pid, we only
read the first 256 bytes of the command.
This means that some commands will be truncated, this has come up in some
of our recently added logs. This ups the buffer size so that we can
hopefully get the full command line.
The longer term solution would be to implement the something fancier mentioned
in the comment in the code copied below, but also has drawbacks as mentioned.
> // Could do something fancy if the entire buffer is filled, but it's better
// if this code does as few syscalls as possible, so just truncate the
// result
Reviewed By: wez
Differential Revision: D22436219
fbshipit-source-id: 80a9aecfe148aa3e333ca480c6a8cb8b9c5c86f2
Summary:
We have seen that eden will unexpectedly fetch data, we want to know why.
This adds the plumbing to interact with edens current logging to be able to
log when eden fetches data from the server and what caused eden to do this
fetch. Later changes will use the classes created here to log the cause of data
fetches.
Reviewed By: chadaustin
Differential Revision: D22051013
fbshipit-source-id: 27d377d7057e66f3e7a304cd7004f8aa44f8ba62
Summary:
Add some functions for validating and producing valid UTF-8 to be used
in an upcoming diff.
Reviewed By: fanzeyi
Differential Revision: D21890510
fbshipit-source-id: b25144a34f1df91c72e8ed776b1ee7c1d68344c8
Summary:
- Added uptime field to DaemonInfo thrift struct
- Created startTime member variable in EdenServer
- Made appropriate refactoring changes to EdenMain and EdenServer
- Changed main.py and util.py to use the new uptime value
Reviewed By: genevievehelsel
Differential Revision: D21471140
fbshipit-source-id: 8868de667dfb95de93e3e71b90c0412fb3825388
Summary:
In glibc, pthread cancellation support adds two atomic CAS operations
to each "cancellation point" syscall (see pthreads(7)). This includes
read() and write(). We can avoid that overhead by disabling pthread
cancellation at the start of the FUSE worker threads.
This saves two CAS operations (~40 ns) in the critical FUSE request
processing loop.
Reviewed By: simpkins
Differential Revision: D21469690
fbshipit-source-id: 7f28a2a8e831006351657981e901dc572c58cf48
Summary:
All of these were simply NOT_IMPLEMENTED on Windows, but the code compiles
and doesn't break any existing tests. The underlying called functions might
have been implemented already, or are NOT_IMPLEMENTED, either way, this reduces
the amount of `#ifdef _WIN32`.
Reviewed By: chadaustin
Differential Revision: D21405622
fbshipit-source-id: bdc2de41d6a57e1c0b532e76eeb2c0c86180d558
Summary:
This brings it closer to folly::writeFile which should help in avoiding ifdef
whenever we want to use it.
Reviewed By: wez
Differential Revision: D21319020
fbshipit-source-id: 80fbf7fba671b18b5ef68375910e1a2a8869f590
Summary:
Google Benchmark is easier to use, has more built-in functionality,
and more accurate default behavior than Folly Benchmark, so switch
EdenFS to use it.;
Reviewed By: simpkins
Differential Revision: D20273672
fbshipit-source-id: c90c49878592620a83d2821ed4bc75c20e599a75
Summary:
Move the `UserInfo` code from `fuse/privhelper` to `utils`, and also unify the
POSIX and Windows implementations of this class.
This code was originally put in the `fuse/privhelper` directory since it was
written at the same time as the privhelper. However, it's really a
lower-level library that doesn't depend on FUSE or any of the other code in
the `fuse/` subdirectory.
Reviewed By: wez
Differential Revision: D21296594
fbshipit-source-id: f58682f6ce86bba0328472c491bb4c0dc3370319
Summary:
Enable the unit tests under eden/fs/utils on Windows.
This does comment out a few tests related to `AbsolutePath` that are broken on
Windows. The AbsolutePath constructor does not enforce that the input path is
actually absolute. Additionally the `canonicalPath()` function ends up doing
weird things and returning paths that start with `/` followed by a drive
letter (e.g., `/C:/foo`). These issues should ideally be addressed in
subsequent diffs.
Reviewed By: xavierd
Differential Revision: D21239886
fbshipit-source-id: ef08d62353ba83b96d9fd79bd4636f4a0f961373
Summary:
Previously, for logging the value of unexpected enum values, we wrote
`static_cast<int>`. Given enumerations can have any integral type
backing them, this was somewhat inaccurate. Instead, introduce an
explicit enumValue function which returns a value of the appropriate
underlying type.
Reviewed By: genevievehelsel
Differential Revision: D20975176
fbshipit-source-id: 0bb5b0d2f68f8fe9d68e4c6a847d59ae0997d0df
Summary:
It turns out that macOS, FreeBSD, and Linux all agree on the values
for dtype. If we assume dtype is just the high nibble of mode_t
shifted right by 12, then the Windows CRT agrees too. So hardcode the
values in eden.thrift and add appropriate static_asserts. This opens
the possibility of simply static_cast'ing the numeric values in
Watchman.
Reviewed By: simpkins
Differential Revision: D20975101
fbshipit-source-id: 354ffcbdf3f1d5f8b1715abf1026eaea429d16cf
Summary:
Add variants of the helpers that return by value rather
than by reference so that we can use those on Windows (which
doesn't embed a `timespec` in a `struct stat`).
Reviewed By: simpkins
Differential Revision: D20562242
fbshipit-source-id: e4769fccb40229765bbf99d0967708cc864db6c3
Summary:
This diff updates the EdenDispatcher to fetch/update the FS info in EdenMount which is backed by InodeTree. This helps to store the FS state in the Inode structure, which are used for source control status and update operations.
Also added a custom formatter definition for RelativePathPiece to make it easy to log relative paths.
Reviewed By: simpkins
Differential Revision: D20480861
fbshipit-source-id: b4bf1da3eeebeaee46a4a187eea9193302182068
Summary: This diff is merging the Windows version of MountTest with the POSIX version. The merged MountTest is build and tested later in this stack with the Merged EdenMount.
Reviewed By: simpkins
Differential Revision: D20480864
fbshipit-source-id: 65e9402f1b03c81166835a6a605053a1bf011ddc
