Summary:
This timeout appears to be too short for the last step of the test that sends
32MB of data in multiple send calls of no more than 1000 bytes each. On
stress test runs this timeout has occasionally been firing and causing the
test to fail.
Reviewed By: chadaustin
Differential Revision: D15221306
fbshipit-source-id: d1f7ed2d58b47512a8f31f8324c61372394a6709
Summary:
beholdunittests
This enables some plumbing for running some of the
tests using the gtest/gmock machinery in cmake.
Part of this diff is removing the FindGMock.cmake file from the
eden repo; we now pull this in from the shared cmake library
that is populated by shipit.
Reviewed By: simpkins
Differential Revision: D14993344
fbshipit-source-id: 51caf9518c7f3a083a3b90cda10324c3a8170359
Summary:
Interestingly, the request doesn't fail, but does leave
our stack garbage intact. Let's be sure to zero it out to make
it more obvious what is happening.
While we're in here, let's make sure that we get the same
results from both ends of the socket pair.
Reviewed By: simpkins
Differential Revision: D14994593
fbshipit-source-id: 9aec957dfcd80d88c3d8fbce6bf45480502ea812
Summary:
UnixSocketTest would block forever on macos if the message
size was larger than approx 1kb. It turns out that MSG_DONTWAIT
isn't documented in the `sendmsg` man pages and apparently has
no effect at all. Instead, the socket must be placed in non-blocking
mode. This appears to be the case for the client side of the sockets
but I need to follow up for the server side.
Reviewed By: simpkins, strager
Differential Revision: D14994588
fbshipit-source-id: 2a0a1c26a7b45ece82f8f79c15fb7756844bc86f
Summary:
On macOS, the atime, mtime, and ctime fields are named st_atimespec,
etc. Add some portability functions so the inode tests compile on
macOS.
Reviewed By: simpkins
Differential Revision: D15048758
fbshipit-source-id: 6f85b8eb7f7da2f9bd21c9034296cde96b68f740
Summary:
Let's catch more Xcode and upstream regressions by enabling more
builds and tests on macOS.
Reviewed By: simpkins
Differential Revision: D15003839
fbshipit-source-id: 257086d578f28198c7718906e1ec06395a55896e
Summary:
Add a helper function to recursively remove a directory from the local
filesystem, given an `AbsolutePath`.
Reviewed By: wez
Differential Revision: D14954737
fbshipit-source-id: a3f6b60e610c18eabea2a7c507661a43502a2554
Summary: This diff adds `ServiceAddress` to utils. This is a class that encapsulates an address of a service that could be a traditional hostname & port pair as well as an SMC tier name. This eliminates the needs of the manually resolving network address for remote services.
Reviewed By: chadaustin
Differential Revision: D14845257
fbshipit-source-id: 9fe9847cca4bba0170be94b9c209247342708574
Summary: This is a helper function to help lazily initializing Mononoke backing stores only when necessary and release it when it is not used any more. See next diff in the stack for detail usage.
Reviewed By: chadaustin
Differential Revision: D14791528
fbshipit-source-id: c26811bc5c7aebcd02f704f10ad19bc35f8b9a21
Summary:
While testing out the new getdeps code I found that none
of the include directories from the probed libraries were being used.
The new getdeps installs each dep into its own prefix, whereas the
existing getdeps script installed them into the installation
prefix for eden itself. That meant that they were being implicitly
found from a single include directory.
In addition to this, I encountered linker failures for the pretty
printers; the solution to those was to add appropriate deps for
the modules that depend upon the pretty printers.
Reviewed By: pkaush
Differential Revision: D14638758
fbshipit-source-id: a4c2b4c79603c268e1b1c707a05c3cb0e3f2757b
Summary: Fix some missing includes that showed up in my mode/mac builds of parts of Eden.
Reviewed By: simpkins, strager
Differential Revision: D14213572
fbshipit-source-id: 54e070e89afdfb8479abdaa122ba76ca1d2d9ba9
Summary:
futures::sleep returning a Future leads to continuations easily being run on
the Timekeeper's callback. The goal is to change the return type so that
futures::sleep returns a folly::SemiFuture.
This codemod is part of the first stage:
* Migrate all call sites off of futures::sleep and onto futures::sleepUnsafe.
This will be followed by:
* Changing the return type of futures::sleep to folly::SemiFuture.
* Migrating callers, where clearly safe such as when they follow with .via or
.semi() from sleepUnsafe to sleep.
* Migrating remaining callers.
Reviewed By: yfeldblum
Differential Revision: D14152623
fbshipit-source-id: bc6874e4320e4a7352ac61b20d9750458e2cbbff
Summary:
Add a new class for injecting faults into Eden. This will make it easier to
build tests that exercise specific error conditions or specifically control
the ordering of operations that might otherwise race.
To minimize the performance impact, the fault injection framework is disabled
by default and must be explicitly enabled at process startup using a command
line flag. If this command line flag was not specified all fault injection
checks short-circuit into a no-op. The `checkAsync()` version does still add
some extra overhead due to the addition of a new `folly::SemiFuture` into the
code path.
Reviewed By: wez
Differential Revision: D14079489
fbshipit-source-id: 3e8725d2e51e5c829199a602e90574424ea52636
Summary:
Replace Future::onError with Future::thenError:
* to remove ambiguous typing
* to ensure that the executor is not lost and the returned Future is still bound to an executor
See:
https://fb.workplace.com/groups/fbcode/permalink/2002251863144976/
for details.
Reviewed By: yfeldblum
Differential Revision: D13784772
fbshipit-source-id: 1d3ede848b7d31c7a197a21b4ff2b31e840040a5
Summary:
Now that the deadlock in ProcessNameCache has been fixed, bring it
back.
Reviewed By: strager
Differential Revision: D13742965
fbshipit-source-id: f407105e06b9954766bdb48ef1303e2003c07284
Summary:
This fixes a deadlock where the kernel held the memory manager lock while satisfying a page fault resulting in a call into Eden, which tried to read /proc/pid/cmdline, acquiring the memory manager lock again.
The fix is to never read /proc/pid/cmdline while handling a request. That work is now done on a background thread.
Reviewed By: strager
Differential Revision: D13685318
fbshipit-source-id: c4e17de3358b668638db0c2dcfba8536d05e5b7c
Summary:
To mitigate a deadlock, I want to make ProcessAccessLog not access /proc/. Allow this by making ProcessNameCache optional.
This diff should not change behavior.
Reviewed By: simpkins
Differential Revision: D13540948
fbshipit-source-id: 4c5d68c972c04122de1d2414084debfec078dd4c
Summary:
ported forward from D4209167, add a couple of helpers
to access these fields on mac and linux, centralizing/minimizing ifdefs.
Simplify some of the logic in FileChangeMonitor.
Reviewed By: chadaustin
Differential Revision: D13475717
fbshipit-source-id: d7b39999808bc41a6dc17a87189501cb34e68b30
Summary:
on macOS the LHS is `unsigned long` and the RHS is `unsigned long long`.
Introduce a cast for consistency.
Reviewed By: chadaustin
Differential Revision: D13470036
fbshipit-source-id: f726507013e4ed9f123ced474299bb2d6818732f
Summary: Even in C++17 mode, this isn't available, so work without it
Reviewed By: chadaustin
Differential Revision: D13470034
fbshipit-source-id: c917cc011aaabc2cfcf79e801bb6870482302fd8
Summary: this is required for the build on macos
Reviewed By: chadaustin
Differential Revision: D13470035
fbshipit-source-id: 066cb5b2ea86ffddb9c8cf6f7ae50da90b62a5bc
Summary:
Drop interest in cached blobs at various points in the FileInode
lifecycle.
Reviewed By: strager
Differential Revision: D12991762
fbshipit-source-id: 19fd94938c96485160d547ecbd259ffeb39b2341
Summary:
Frequently, non-eden-owner-owned processes show up as `err:13` in
the `eden top` output which is a bit of a PITA. This diff pulls the data
from `/proc/PID/cmdline` instead of `/proc/PID/exe`; the former is world
readable always while the latter is restricted to process owner readable.
Reviewed By: chadaustin
Differential Revision: D13342789
fbshipit-source-id: d4e395b318107e873189a1e2039329015c4c38f8
Summary: Add a CoverageSet type that tracks non-overlapping intervals so FileInode will be able to tell when the entire blob has been read by FUSE.
Reviewed By: strager
Differential Revision: D12947146
fbshipit-source-id: fbcfd9c19b09d4a7b5364671dcdbd39b53e6d186
Summary: No need for String to import fbvector.
Reviewed By: yfeldblum
Differential Revision: D13007715
fbshipit-source-id: c61639a04273f14dd3daf230ee6ed9ade93d058e
Summary:
Backing store works with eden strings(UTF8 + Unix path separator). The path strings we receive on Windows from FS and cli are Windows paths
(Win path separator and/or UTF16). Adding the functions to convert one to another.
Reviewed By: strager
Differential Revision: D13091788
fbshipit-source-id: f7fc8a79e360e964cf4619dfa540b57f1f18d283
Summary:
Now that we have a standardized StartingGate for benchmarks, use it
everywhere we used to use folly::Baton.
(Note: this ignores all push blocking failures!)
Differential Revision: D13012244
fbshipit-source-id: 5841ab74cfa408e87d021fe5591557e79e677e5c
Summary:
As we start to build out both FUSE and Thrift benchmarks, we'll want a
standard library. Introduce a benchharness and have both the thrift
sha-1 and parallel_open_close benchmarks use it.
(Note: this ignores all push blocking failures!)
Differential Revision: D12969306
fbshipit-source-id: 89c8bbcc37d53560decffb9281af4aba20345787
Summary:
yfeldblum were talking about whether this code might make sense in
folly. That led to polishing it a bit more. The hot path is only six
instructions now. It's not any faster in a tight generateUniqueID loop
but uses only one thread_local slot and a bit less global storage.
Reviewed By: strager
Differential Revision: D12927275
fbshipit-source-id: 94a5872c61dfe9dd441f1f34fab65f93c12997d8
Summary:
In a later diff, I needed generateUniqueID to be
noexcept. folly::ThreadLocal does not guarantee that (and it allocates
the first time a thread calls get()), so use C++ thread_local
instead. Bonus: it's about half a nanosecond faster now.
Reviewed By: strager
Differential Revision: D12914625
fbshipit-source-id: 9ddbe65d0ba1d317907f821c03dea5a207a73a68
Summary:
Update all of the C++ unit tests that create temporary files and directories
to use the new `facebook::eden::makeTempFile()` and
`facebook::eden::makeTempDir()` functions.
Note that this likely changes the behavior of some code paths in meaningful
ways: `/dev/shm` normally does not support `getxattr()`, and Eden's overlay
code attempts to store the SHA-1 for materialized files as using extended
attributes. This means that the tests will now typically hit the fallback
code path and will not store SHA-1 data in the overlay.
Reviewed By: chadaustin, strager
Differential Revision: D12971162
fbshipit-source-id: 6cc5eba2e04be7e9a13a30e90883feadfb78f9ce
Summary:
This function behaves similarly to the python `os.path.expanduser`
function, except that it is restricted to expanding only information for the
current user.
Use this new function to process the hgcache path returned via the
rust config parsing code.
In the longer term I want to centralize and add accessors for
the rust config and move this stuff into EdenConfig, but for
now this is the one place that does this and it is OK to
process it this way here.
Reviewed By: strager
Differential Revision: D12988902
fbshipit-source-id: 96b10640359c3b8c0adac1e14cd42dd592023c3d
Summary:
In a later diff, I needed to make generateUniqueID() noexcept. This
made me think that C++'s thread_local might be a little faster than
folly::ThreadLocal. So I added this benchmark.
Reviewed By: yfeldblum
Differential Revision: D12914249
fbshipit-source-id: 2c4acfff2162f66d13f456439d91df2ecb4167e3
Summary:
In order to get better insights into where time is spent on
various requests, some tracing infrastructure is helpful. This will
record tracepoints into a per-thread ring-buffer which can then be
dumped out-of-band.
The trace format is quite simple, a trace is composed of blocks with
begin and end points and each block may have a parent block. We can
then translate this format to many other formats as we desire.
Reviewed By: chadaustin, strager
Differential Revision: D10384071
fbshipit-source-id: f9e0f11521c68b96ab40d517c7a83cf89375b101
Summary:
Add a very cheap unique ID allocator designed for extremely
low-latency uses like D10384071.
Differential Revision: D10501712
fbshipit-source-id: 15a0b7a2d344c0f6100082850dd967c585f3e2da
Summary: D3376727 removed the `+1` from `AbsolutePathBase::paths`, but did not remove the comment referring to the `+1`. Remove the stale comment.
Reviewed By: pkaush
Differential Revision: D10394249
fbshipit-source-id: 3b2dc27f7083fe8a8706bac38275249dd4969d11
Summary:
relativize() returns an AbsolutePathPiece into its argument which is a
temporary AbsolutePath. The tests and ASAN didn't catch this because
they asserted the result's value before the temporary was destroyed, I
assume. Also the test strings fit into the SSO so there was no actual
memory allocation.
Reviewed By: simpkins
Differential Revision: D10261332
fbshipit-source-id: 912ffe6a46845cdd625e6dffbf65f0a0f8f89d0f
Summary:
Add a splitFirst() function that, if you think about a path as a linked list,
splits the head and the tail. Normally you could use the components
iterator, but split() is easier when the iteration is asynchronous and
requires lambda capture.
Reviewed By: pkaush
Differential Revision: D10154746
fbshipit-source-id: 6372a0bbde27f6596659ad1244cc96fa1f9eb6ff
Summary:
It looks like D9477181 conflicted with D9635507, causing a compile error:
```
eden/fs/utils/ProcessNameCache.cpp:74:15: error: no member named 'names' in 'folly::LockedPtr<folly::Synchronized<facebook::eden::ProcessNameCache::State, folly::SharedMutexImpl<false> >, folly::LockPolicyExclusive>'
state.names.emplace(pid, ProcessName{detail::readPidName(pid), now});
~~~~~ ^
eden/fs/utils/Synchronized.h:48:10: note: in instantiation of function template specialization 'facebook::eden::ProcessNameCache::add(pid_t)::(anonymous class)::operator()<folly::LockedPtr<folly::Synchronized<facebook::eden::ProcessNameCache::State, folly::SharedMutexImpl<false> >, folly::LockPolicyExclusive> >' requested here
return update(wlock);
^
eden/fs/utils/ProcessNameCache.cpp:58:3: note: in instantiation of function template specialization 'facebook::eden::tryRlockCheckBeforeUpdate<folly::Unit, facebook::eden::ProcessNameCache::State, (lambda), (lambda)>' requested here
tryRlockCheckBeforeUpdate<folly::Unit>(
^
eden/fs/utils/ProcessNameCache.cpp:81:15: error: no member named 'waterLevel' in 'folly::LockedPtr<folly::Synchronized<facebook::eden::ProcessNameCache::State, folly::SharedMutexImpl<false> >, folly::LockPolicyExclusive>'
state.waterLevel += 2;
~~~~~ ^
eden/fs/utils/ProcessNameCache.cpp:82:19: error: no member named 'waterLevel' in 'folly::LockedPtr<folly::Synchronized<facebook::eden::ProcessNameCache::State, folly::SharedMutexImpl<false> >, folly::LockPolicyExclusive>'
if (state.waterLevel > state.names.size()) {
~~~~~ ^
3 errors generated.
```
Dereference `state` to fix the error.
Reviewed By: chadaustin
Differential Revision: D9673324
fbshipit-source-id: 5143cd22baf66307e5aacb0a04669de69dde99e8
