Summary:
Since the Stub.h now only contains NOT_IMPLEMENTED, let's move it to its own
header outside of the win directory.
Reviewed By: genevievehelsel
Differential Revision: D23696244
fbshipit-source-id: 2dfc3204707e043ee6c89595668c484e0fa8c0d0
Summary:
While the code isn't compiled, this makes the thrift definition available to
the rest of the code, eliminating the need for having a stub for
SerializedInodeMap on Windows.
Reviewed By: genevievehelsel
Differential Revision: D23696242
fbshipit-source-id: 8a42dd2ed16887f3b7d161511e07aaa35fd1b968
Summary:
While hacking on some code, I ran into a situation where some
zero-initialized stat structs weren't actually being zeroed. This was
either a compiler bug or a situation where the build system was not
correctly rebuilding everything after my changes, and I did not have
enough disassembly available to investigate.
Either way, since this code assumes zero bits in some nonobvious ways,
explicitly assert they are.
Reviewed By: xavierd
Differential Revision: D23644819
fbshipit-source-id: eb6bff9ff997379113db1e1bf9d6a0a538f10f0b
Summary: This fixes the `The program issued a command but the command length is incorrect` error we have been seeing in EdenFS Windows.
Reviewed By: wez
Differential Revision: D23589264
fbshipit-source-id: 72478a653ff07fca81c163bab12c0fd33f6a0a67
Summary:
Even though these might not be fully ported on Windows, they do compile and
tests are passing, so let's compile them.
Reviewed By: chadaustin
Differential Revision: D23505509
fbshipit-source-id: 567e8668ca489daf89c1c6576973bbaaabbb6c88
Summary:
Most of the RequestData code is platform generic, but bits of it are currently
strongly tied to FUSE. By splitting these 2 parts, we will be able to use the
RequestContext class in Windows too and not having to re-implement all the
logic there.
Reviewed By: chadaustin
Differential Revision: D23482072
fbshipit-source-id: 857fd9ca4264d0f308ec10cc487e9ff3eeb5ee16
Summary:
Removing Fuse from the enum name makes it non tied to Fuse and thus makes it
more portable. This also eliminates the last platform specific bit from
RequestData.
Reviewed By: chadaustin
Differential Revision: D23467773
fbshipit-source-id: 52515522c8ac51d0c4b56dc5e42d4b6593df6623
Summary:
The loop took care to advance `b` to match the amount
of data that it had processed, but was still passing `buf`
(the unadjusted start of the buffer) to the syscalls.
This meant that in situations where a `readFull` might
encounter a partial read, it would scribble over the start
of the buffer and leave junk at the end.
For example:
write("hell");
write("o");
could produce "oell?" in the buffer when `readFull` consumes
the other end of the pipe.
Reviewed By: xavierd
Differential Revision: D23486270
fbshipit-source-id: 0848f6789b44421b609b91fe08890768ff59f7f5
Summary: The latter will not strip new lines from the system error message, while the former does.
Reviewed By: genevievehelsel
Differential Revision: D23480435
fbshipit-source-id: 44742b960935552fa1781ed19f38ff446a8c9403
Summary:
This is not per-se fuse related, thus move it to a common location and remove
the duplicated define in FileInode.h
Reviewed By: chadaustin
Differential Revision: D23465192
fbshipit-source-id: 5fa7709f127c2d3372ee5ea3aeb89e793ea5b9f7
Summary:
This commit adds a compile time option to select
between case-sensitive and case-insensitive-but-case-preserving
mode for `PathMap`.
This replaces the `ifdef _WIN32` preprocessor conditional
that was inline in a couple of the methods and allows
explicitly testing the behavior in both modes of operation.
The unit tests have been expanded and rounded out to catch
some inconsistent behavior; insertion wasn't respecting
case insensitivity in all ... cases.
Hopefully we not relying on that behavior in the windows
flavor of the build; let's see what our CI says.
Reviewed By: genevievehelsel
Differential Revision: D23232629
fbshipit-source-id: 96e752e501d0398ec2bed5879f7c11c7ab6e1d70
Summary:
I switched these from ERR to DBG2 thinking that would
take them out of the log by default, but we log DBG2 by default
so that didn't have the desired result.
This changes the log level to DBG6 which is an arbitrary log
level that isn't included by default.
Reviewed By: genevievehelsel
Differential Revision: D23467738
fbshipit-source-id: dd0c75f86318ece27313c237938f24f55758eec1
Summary:
This commit introduces a new process spawning class derived
from the ChildProcess class in the watchman codebase.
`SpawnedProcess` is similar to folly::Subprocess but is designed around the
idea that we will use a system provided spawning API to start a process, rather
than assuming the use of `fork`.
`fork` is to be avoided because it can be expensive for processes with large
address spaces and also because it interacts poorly with threads on macOS. In
particular, we see the objC runtime terminating our process in some scenarios
where fork and threads are mixed.
There are some important differences from `folly::Subprocess` and that means
that some assumptions and uses need to be altered slightly from their prior
workings. For example, detaching a SpawnedProcess moves the responsibility of
waiting on the child to a periodic task as there is no way to detach via
posix_spawn without also using fork.
On the plus side, this commit allows unifying spawning between posix and
windows systems, which simplifies the code!
Reviewed By: xavierd
Differential Revision: D23287763
fbshipit-source-id: b662af1d7eaaa9ed445c42f6c5765ae9af975eea
Summary:
This commit introduces a few types from the watchman codebase:
`FileDescriptor` which is on posix systems represents a file descriptor,
and on Windows is a HANDLE (which can be a file, pipe or socket descriptor).
`Pipe` is a convenience struct that holds the read and write ends of a Pipe.
Note that we have a conceptual class with a windows specific Pipe type under
eden/fs/win/utils/Pipe.h; I remove that in the next diff in the stack.
There are a couple of differences from the watchman code
Reviewed By: chadaustin
Differential Revision: D23287819
fbshipit-source-id: 6ca90ba345037c6c3e308f588d690a899c9866a5
Summary:
std::iterator is deprecated in C++17. Removing it fixes warnings in
the Clang/Windows build.
Reviewed By: genevievehelsel
Differential Revision: D23352927
fbshipit-source-id: 293e30909eaa8a7c4856a91930a3886ad0b19364
Summary:
Up to now, Windows had to have its own version of folly::{readFile, writeFile,
writeFileAtomic} as these only operate on `char *` path, which can only
represent ascii paths on Windows. Since the Windows version is slightly
different from folly, this forced the code to either ifdef _WIN32, or use the
folly version pretending that it would be OK. The Windows version was also
behaving slightly differently from folly. For instance, where folly would
return a boolean to indicate success, on Windows we would throw an exception.
To simplify our code, add type safety and unify both, we can implement our own
wrappers on top of either folly or Windows APIs.
We still have some code that uses folly::readFile but these should only be
dealing with filedescriptors. As a following step, we may want to have our own
File class that wraps a file descriptor/HANDLE so we can completely remove all
uses of folly::readFile.
Reviewed By: wez
Differential Revision: D23037325
fbshipit-source-id: 2b9a026f3ee6220ef55097abe649b23e38d9fe91
Summary:
Instead of modifying the existing APIs and marking all callsites as deprecated in one diff, I am going to take the "add and remove" approach, where I will add the deprecated version of methods first, then mark all callsites, and finally remove the existing ones. This provides some backward compatibility without breaking things.
This diff is to add deprecated getSelection APIs to ServiceSelectorCache.
Differential Revision: D22981269
fbshipit-source-id: 6e3025e7f7df6ee7f9e1cba9dc036ca84adbe49a
Summary:
The StringConv.h header contains many functions to convert from Windows paths
to Eden's path (and vice versa) to workaround the fact that Eden's path don't
support wide strings that Windows uses. Let's simply add support for these wide
strings in PathFuncs so we can greatly simplify all the call sites. Instead of
calling "edenToWinName(winstr)", "PathComponent(winstr)" is both more
descriptive and more idiomatic.
To be fair, I'm not entirely a fan of the approach taken in this diff, as this
adds Windows specific code to PathFuncs.h, but I feel that the benefit is too
big to not do that.
Reviewed By: chadaustin
Differential Revision: D23004523
fbshipit-source-id: 3a1507e398a66909773251907db01e06603b91dd
Summary:
Previously, fetch heavy event's cmdline was delimited by '\x00' when logged to Scuba. (for example: `grep--color=auto-rtest.`)
Now we replace \x00 with a space, so command name and args will be separated by space. ( `grep --color=auto -r test .` )
Reviewed By: kmancini
Differential Revision: D22772868
fbshipit-source-id: 4ab42e78c7bc786767eee3413b9586739a12e8ac
Summary:
We are unifying C++ APIs for accessing optional and unqualified fields:
https://fb.workplace.com/groups/1730279463893632/permalink/2541675446087359/.
This diff migrates code from accessing data members generated from unqualified
Thrift fields directly to the `field_ref` API, i.e. replacing
```
thrift_obj.field
```
with
```
*thrift_obj.field_ref()
```
The `_ref` suffixes will be removed in the future once data members are private
and names can be reclaimed.
The output of this codemod has been reviewed in D20039637.
The new API is documented in
https://our.intern.facebook.com/intern/wiki/Thrift/FieldAccess/.
drop-conflicts
Reviewed By: yfeldblum
Differential Revision: D22631599
fbshipit-source-id: 9bfcaeb636f34a32fd871c7cd6a2db4a7ace30bf
Summary:
While there isn't a direct equivalent to the unix RSS and VSZ, the working set
size and the page file usage can be used instead, they roughly have the same
meaning and that's probably good enough.
I've also moved the private bytes into the MemoryStats as it's a memory stat
too.
Reviewed By: chadaustin
Differential Revision: D22081006
fbshipit-source-id: d597d523eee21e7651a14e60b1fd6dc152a61185
Summary:
Currently when we are resolving the full command line for a client pid, we only
read the first 256 bytes of the command.
This means that some commands will be truncated, this has come up in some
of our recently added logs. This ups the buffer size so that we can
hopefully get the full command line.
The longer term solution would be to implement the something fancier mentioned
in the comment in the code copied below, but also has drawbacks as mentioned.
> // Could do something fancy if the entire buffer is filled, but it's better
// if this code does as few syscalls as possible, so just truncate the
// result
Reviewed By: wez
Differential Revision: D22436219
fbshipit-source-id: 80a9aecfe148aa3e333ca480c6a8cb8b9c5c86f2
Summary:
We have seen that eden will unexpectedly fetch data, we want to know why.
This adds the plumbing to interact with edens current logging to be able to
log when eden fetches data from the server and what caused eden to do this
fetch. Later changes will use the classes created here to log the cause of data
fetches.
Reviewed By: chadaustin
Differential Revision: D22051013
fbshipit-source-id: 27d377d7057e66f3e7a304cd7004f8aa44f8ba62
Summary:
Add some functions for validating and producing valid UTF-8 to be used
in an upcoming diff.
Reviewed By: fanzeyi
Differential Revision: D21890510
fbshipit-source-id: b25144a34f1df91c72e8ed776b1ee7c1d68344c8
Summary:
- Added uptime field to DaemonInfo thrift struct
- Created startTime member variable in EdenServer
- Made appropriate refactoring changes to EdenMain and EdenServer
- Changed main.py and util.py to use the new uptime value
Reviewed By: genevievehelsel
Differential Revision: D21471140
fbshipit-source-id: 8868de667dfb95de93e3e71b90c0412fb3825388
Summary:
In glibc, pthread cancellation support adds two atomic CAS operations
to each "cancellation point" syscall (see pthreads(7)). This includes
read() and write(). We can avoid that overhead by disabling pthread
cancellation at the start of the FUSE worker threads.
This saves two CAS operations (~40 ns) in the critical FUSE request
processing loop.
Reviewed By: simpkins
Differential Revision: D21469690
fbshipit-source-id: 7f28a2a8e831006351657981e901dc572c58cf48
Summary:
All of these were simply NOT_IMPLEMENTED on Windows, but the code compiles
and doesn't break any existing tests. The underlying called functions might
have been implemented already, or are NOT_IMPLEMENTED, either way, this reduces
the amount of `#ifdef _WIN32`.
Reviewed By: chadaustin
Differential Revision: D21405622
fbshipit-source-id: bdc2de41d6a57e1c0b532e76eeb2c0c86180d558
Summary:
This brings it closer to folly::writeFile which should help in avoiding ifdef
whenever we want to use it.
Reviewed By: wez
Differential Revision: D21319020
fbshipit-source-id: 80fbf7fba671b18b5ef68375910e1a2a8869f590
Summary:
Google Benchmark is easier to use, has more built-in functionality,
and more accurate default behavior than Folly Benchmark, so switch
EdenFS to use it.;
Reviewed By: simpkins
Differential Revision: D20273672
fbshipit-source-id: c90c49878592620a83d2821ed4bc75c20e599a75
Summary:
Move the `UserInfo` code from `fuse/privhelper` to `utils`, and also unify the
POSIX and Windows implementations of this class.
This code was originally put in the `fuse/privhelper` directory since it was
written at the same time as the privhelper. However, it's really a
lower-level library that doesn't depend on FUSE or any of the other code in
the `fuse/` subdirectory.
Reviewed By: wez
Differential Revision: D21296594
fbshipit-source-id: f58682f6ce86bba0328472c491bb4c0dc3370319
Summary:
Enable the unit tests under eden/fs/utils on Windows.
This does comment out a few tests related to `AbsolutePath` that are broken on
Windows. The AbsolutePath constructor does not enforce that the input path is
actually absolute. Additionally the `canonicalPath()` function ends up doing
weird things and returning paths that start with `/` followed by a drive
letter (e.g., `/C:/foo`). These issues should ideally be addressed in
subsequent diffs.
Reviewed By: xavierd
Differential Revision: D21239886
fbshipit-source-id: ef08d62353ba83b96d9fd79bd4636f4a0f961373
Summary:
Previously, for logging the value of unexpected enum values, we wrote
`static_cast<int>`. Given enumerations can have any integral type
backing them, this was somewhat inaccurate. Instead, introduce an
explicit enumValue function which returns a value of the appropriate
underlying type.
Reviewed By: genevievehelsel
Differential Revision: D20975176
fbshipit-source-id: 0bb5b0d2f68f8fe9d68e4c6a847d59ae0997d0df
Summary:
It turns out that macOS, FreeBSD, and Linux all agree on the values
for dtype. If we assume dtype is just the high nibble of mode_t
shifted right by 12, then the Windows CRT agrees too. So hardcode the
values in eden.thrift and add appropriate static_asserts. This opens
the possibility of simply static_cast'ing the numeric values in
Watchman.
Reviewed By: simpkins
Differential Revision: D20975101
fbshipit-source-id: 354ffcbdf3f1d5f8b1715abf1026eaea429d16cf
Summary:
Add variants of the helpers that return by value rather
than by reference so that we can use those on Windows (which
doesn't embed a `timespec` in a `struct stat`).
Reviewed By: simpkins
Differential Revision: D20562242
fbshipit-source-id: e4769fccb40229765bbf99d0967708cc864db6c3
Summary:
This diff updates the EdenDispatcher to fetch/update the FS info in EdenMount which is backed by InodeTree. This helps to store the FS state in the Inode structure, which are used for source control status and update operations.
Also added a custom formatter definition for RelativePathPiece to make it easy to log relative paths.
Reviewed By: simpkins
Differential Revision: D20480861
fbshipit-source-id: b4bf1da3eeebeaee46a4a187eea9193302182068
Summary: This diff is merging the Windows version of MountTest with the POSIX version. The merged MountTest is build and tested later in this stack with the Merged EdenMount.
Reviewed By: simpkins
Differential Revision: D20480864
fbshipit-source-id: 65e9402f1b03c81166835a6a605053a1bf011ddc
Summary:
This updates the top-level CMakeLists.txt file to compute package version
information, and expose this to C++ code in `eden-config.h`, and to Python
code in a new `eden/config.py` module.
Previously we exposed an `EDEN_VERSION` macro for the C++ code in
`eden-config.h`, but this was not initialized or used anywhere. Now the
top-level CMakeLists.txt file computes appropriate version information and
exposes the package name, version, release, commit ID, and build time in these
configuration files.
The version selection logic in CMakeLists.txt based largely on the code that
wez wrote for watchman in D20636833.
Reviewed By: wez
Differential Revision: D21000164
fbshipit-source-id: db1a1035f1eefec058bbad558d35e113005e454e
Summary:
Build and run the fuse privhelper unit tests, as well as the `drop_privs`
helper program used by some of the integration tests.
Reviewed By: wez
Differential Revision: D21004425
fbshipit-source-id: 650e0729909f4753095e19fba4f01c02d516713b
Summary:
Some of our types were vulnerable to the issue described in
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p1008r0.pdf so
make all deleted default constructors explicit.
Reviewed By: simpkins
Differential Revision: D21008976
fbshipit-source-id: 5b21923f25121dabf4bb0ea55f94536fb3532e6b
Summary: Missed cleaning up the other definition in D19956272.
Reviewed By: chadaustin
Differential Revision: D20199689
fbshipit-source-id: 9a00cb7b640f71edc034632e354fb61112a74bc1
Summary: Windows SDK doesn't define dirent. Defining it here for adding Inodes support on Edenfs on Windows.
Reviewed By: simpkins
Differential Revision: D19956272
fbshipit-source-id: 1bdf9a7563c194fe38008741b09668242ffa64ee
Summary: On Windows we will verify both Windows and POSIX path separators, because we have both types of paths.
Reviewed By: simpkins
Differential Revision: D19562772
fbshipit-source-id: a6d8280e3bae4f6cd32b1f379bb59e22dd584211
Summary:
folly::collect's Future is completed immediately when any of its input
future is completed with an exception. This makes forking off subtasks
and then joining with folly::collect dangerous when closures don't
hold pointers or references to parameters, including `this`. Introduce
a collectSafe function, which has the same signature as folly::collect
but unconditionally waits until all futures are completed.
Reviewed By: simpkins
Differential Revision: D19529772
fbshipit-source-id: b3b493fcbb0d9058d2d6e5c8a064dfdae3a43e57
Summary: `folly::cold_detail::ColdClass` was marking things (like `folly::Unexpected`) cold, but at the cost of inhibiting the inliner from doing its job. This is leading to bad codegen, which offsets any small wins we mind get for the `cold` attribute.
Reviewed By: yfeldblum
Differential Revision: D19324159
fbshipit-source-id: 7ed431b6c9d6e963c3bf438c707fa6cf6a38bf9d