Summary:
folly::collect's Future is completed immediately when any of its input
future is completed with an exception. This makes forking off subtasks
and then joining with folly::collect dangerous when closures don't
hold pointers or references to parameters, including `this`. Introduce
a collectSafe function, which has the same signature as folly::collect
but unconditionally waits until all futures are completed.
Reviewed By: simpkins
Differential Revision: D19529772
fbshipit-source-id: b3b493fcbb0d9058d2d6e5c8a064dfdae3a43e57
Summary:
This splits `EDEN_BUG()` into three separate version. All three crash in
debug mode builds, but in release builds they behave differently:
- `EDEN_BUG()` throws an exception
- `EDEN_BUG_FUTURE(Type)` returns a `folly::Future<Type>` that has been
fulfilled with an exception.
- `EDEN_BUG_EXCEPTION()` returns a `folly::exception_wrapper`.
The main advantage of this is that this allows the compiler to detect that
`EDEN_BUG()` can never return. Previously `EDEN_BUG()` was used for all 3 of
these different cases, and its behavior depended on whether `toException()`
was ever called. As a result we could not easily get the compiler to identify
code paths where we know at compile time that it will never return.
Reviewed By: chadaustin
Differential Revision: D18652103
fbshipit-source-id: 070107c7520f51b05696905fa243de5f8df15958
Summary:
Add code on macOS to collect the process's resident set size and
virtual size so the memory usage counter shows up.
Reviewed By: wez
Differential Revision: D18142618
fbshipit-source-id: b42206017ace5309b7bc379f042c1cd6e455f574
Summary: EdenFS recently is unable to import data from API Server on devservers. This is due to the missing of service router support during the build time. This diff fixes this issue.
Reviewed By: wez
Differential Revision: D18042201
fbshipit-source-id: 15868d8e81b9403218e481b3fbf0a7ca1279b023
Summary:
D17623550 deleted `ServiceCacheIf::invalidateSelection()` but did not remove
this override, which broke the build for EdenFS's tests.
Reviewed By: wez
Differential Revision: D17638351
fbshipit-source-id: 200ead0f79a2252dc7ac2318745cd98e520737a4
Summary:
On a machine with one core, it's quite possible for the process to
freeze for a second, resulting in a flaky test.
Reviewed By: wez
Differential Revision: D17579535
fbshipit-source-id: a3e158b5895c06bdf560ebecd849a74e4efa591e
Summary:
StartingGate was similar to std::barrier except that it guaranteed the
coordinating thread woke the worker threads. While this symmetry is
nice in concept, given enough threads, I doubt it pays for the
duplication.
Just adopt folly::test::Barrier (which is basically std::barrier)
instead.
Reviewed By: strager
Differential Revision: D16942136
fbshipit-source-id: 1691d8ef72b88c8867df74c8ed938f6c8d6ee094
Summary: Now that StartingGate is in a more accessible dependency, use it in Eden's benchmarks.
Reviewed By: strager
Differential Revision: D16705853
fbshipit-source-id: 79ab9e9b32ab93f5b50a06a4889edb5e7b0d7e8b
Summary: Uses the existing RequestData class to make calls to static functions to set and get the `didImportFromBackingStore` flag.
Reviewed By: simpkins
Differential Revision: D16461868
fbshipit-source-id: e3ed39249f5dd1a842ad06a204b5933014b12f7f
Summary:
Update the copyright & license headers in CMake files to reflect the
relicensing to GPLv2
Reviewed By: wez
Differential Revision: D15487079
fbshipit-source-id: 715e559464c19a0070d6e55a095b3fc7d61ad2f8
Summary:
Update the copyright & license headers in C++ files to reflect the
relicensing to GPLv2
Reviewed By: wez
Differential Revision: D15487078
fbshipit-source-id: 19f24c933a64ecad0d3a692d0f8d2a38b4194b1d
Summary:
Add a `durationToString()` function. This converts a duration value to a
string that can be parsed back with `stringToDuration()`.
Reviewed By: wez, strager
Differential Revision: D15428935
fbshipit-source-id: 2840663385434bc35f04c16d2570fd86e234cb3f
Summary:
Add a `stringToDuration()` function to parse a string into a
`std::chrono::nanoseconds` value.
Strings are parsed as a sequence of `[number] [unit]` pairs.
For instance:
`1m30s` --> 90 seconds
`5s10ms` --> 5010 milliseconds
`1d3h10m` --> 1 day, 3 hours, 10 minutes (1630 minutes total)
Reviewed By: strager
Differential Revision: D15428934
fbshipit-source-id: 63f6614595c1aaa08c0d03633c6d1b53ca5bf3d5
Summary:
Read information from the machine-readable `/proc/self/statm` rather than
attempting to parse the human-readable `/proc/self/status` file.
This simplifies the parsing logic overall.
Reviewed By: wez
Differential Revision: D15466907
fbshipit-source-id: c0184b0faa0a18cf0101828c3822b546ca7a677e
Summary:
Increase the default send timeout from 100ms to 250ms.
Increse the timeout in the `UnixSocket.sendDataAndFiles` test to 10s.
We have seen some failures in the `UnixSocket.sendDataAndFiles()` test on
our continuous build infrastructure when testing debug builds on Mac OS.
These failed due to hitting the send timeout on the 4MB and 32MB sends. Bump
up the timeout for this test to 10s. (I initially tried 250ms, and this was
still not sufficient.)
I increased the default send timeout to 250ms just to help ensure that we
don't hit it unexpectedly during real production code when running on Mac OS.
Reviewed By: wez
Differential Revision: D15401154
fbshipit-source-id: c8be68a77231e90d37c9b5b4f349379bdf2b15ed
Summary:
This timeout appears to be too short for the last step of the test that sends
32MB of data in multiple send calls of no more than 1000 bytes each. On
stress test runs this timeout has occasionally been firing and causing the
test to fail.
Reviewed By: chadaustin
Differential Revision: D15221306
fbshipit-source-id: d1f7ed2d58b47512a8f31f8324c61372394a6709
Summary:
beholdunittests
This enables some plumbing for running some of the
tests using the gtest/gmock machinery in cmake.
Part of this diff is removing the FindGMock.cmake file from the
eden repo; we now pull this in from the shared cmake library
that is populated by shipit.
Reviewed By: simpkins
Differential Revision: D14993344
fbshipit-source-id: 51caf9518c7f3a083a3b90cda10324c3a8170359
Summary:
Interestingly, the request doesn't fail, but does leave
our stack garbage intact. Let's be sure to zero it out to make
it more obvious what is happening.
While we're in here, let's make sure that we get the same
results from both ends of the socket pair.
Reviewed By: simpkins
Differential Revision: D14994593
fbshipit-source-id: 9aec957dfcd80d88c3d8fbce6bf45480502ea812
Summary:
Let's catch more Xcode and upstream regressions by enabling more
builds and tests on macOS.
Reviewed By: simpkins
Differential Revision: D15003839
fbshipit-source-id: 257086d578f28198c7718906e1ec06395a55896e
Summary:
Add a helper function to recursively remove a directory from the local
filesystem, given an `AbsolutePath`.
Reviewed By: wez
Differential Revision: D14954737
fbshipit-source-id: a3f6b60e610c18eabea2a7c507661a43502a2554
Summary: This diff adds `ServiceAddress` to utils. This is a class that encapsulates an address of a service that could be a traditional hostname & port pair as well as an SMC tier name. This eliminates the needs of the manually resolving network address for remote services.
Reviewed By: chadaustin
Differential Revision: D14845257
fbshipit-source-id: 9fe9847cca4bba0170be94b9c209247342708574
Summary: This is a helper function to help lazily initializing Mononoke backing stores only when necessary and release it when it is not used any more. See next diff in the stack for detail usage.
Reviewed By: chadaustin
Differential Revision: D14791528
fbshipit-source-id: c26811bc5c7aebcd02f704f10ad19bc35f8b9a21
Summary:
Add a new class for injecting faults into Eden. This will make it easier to
build tests that exercise specific error conditions or specifically control
the ordering of operations that might otherwise race.
To minimize the performance impact, the fault injection framework is disabled
by default and must be explicitly enabled at process startup using a command
line flag. If this command line flag was not specified all fault injection
checks short-circuit into a no-op. The `checkAsync()` version does still add
some extra overhead due to the addition of a new `folly::SemiFuture` into the
code path.
Reviewed By: wez
Differential Revision: D14079489
fbshipit-source-id: 3e8725d2e51e5c829199a602e90574424ea52636
Summary:
Replace Future::onError with Future::thenError:
* to remove ambiguous typing
* to ensure that the executor is not lost and the returned Future is still bound to an executor
See:
https://fb.workplace.com/groups/fbcode/permalink/2002251863144976/
for details.
Reviewed By: yfeldblum
Differential Revision: D13784772
fbshipit-source-id: 1d3ede848b7d31c7a197a21b4ff2b31e840040a5
Summary:
Now that the deadlock in ProcessNameCache has been fixed, bring it
back.
Reviewed By: strager
Differential Revision: D13742965
fbshipit-source-id: f407105e06b9954766bdb48ef1303e2003c07284
Summary:
This fixes a deadlock where the kernel held the memory manager lock while satisfying a page fault resulting in a call into Eden, which tried to read /proc/pid/cmdline, acquiring the memory manager lock again.
The fix is to never read /proc/pid/cmdline while handling a request. That work is now done on a background thread.
Reviewed By: strager
Differential Revision: D13685318
fbshipit-source-id: c4e17de3358b668638db0c2dcfba8536d05e5b7c
Summary:
To mitigate a deadlock, I want to make ProcessAccessLog not access /proc/. Allow this by making ProcessNameCache optional.
This diff should not change behavior.
Reviewed By: simpkins
Differential Revision: D13540948
fbshipit-source-id: 4c5d68c972c04122de1d2414084debfec078dd4c
Summary:
Drop interest in cached blobs at various points in the FileInode
lifecycle.
Reviewed By: strager
Differential Revision: D12991762
fbshipit-source-id: 19fd94938c96485160d547ecbd259ffeb39b2341
Summary:
Frequently, non-eden-owner-owned processes show up as `err:13` in
the `eden top` output which is a bit of a PITA. This diff pulls the data
from `/proc/PID/cmdline` instead of `/proc/PID/exe`; the former is world
readable always while the latter is restricted to process owner readable.
Reviewed By: chadaustin
Differential Revision: D13342789
fbshipit-source-id: d4e395b318107e873189a1e2039329015c4c38f8
Summary: Add a CoverageSet type that tracks non-overlapping intervals so FileInode will be able to tell when the entire blob has been read by FUSE.
Reviewed By: strager
Differential Revision: D12947146
fbshipit-source-id: fbcfd9c19b09d4a7b5364671dcdbd39b53e6d186
Summary:
Now that we have a standardized StartingGate for benchmarks, use it
everywhere we used to use folly::Baton.
(Note: this ignores all push blocking failures!)
Differential Revision: D13012244
fbshipit-source-id: 5841ab74cfa408e87d021fe5591557e79e677e5c
Summary:
As we start to build out both FUSE and Thrift benchmarks, we'll want a
standard library. Introduce a benchharness and have both the thrift
sha-1 and parallel_open_close benchmarks use it.
(Note: this ignores all push blocking failures!)
Differential Revision: D12969306
fbshipit-source-id: 89c8bbcc37d53560decffb9281af4aba20345787
Summary:
yfeldblum were talking about whether this code might make sense in
folly. That led to polishing it a bit more. The hot path is only six
instructions now. It's not any faster in a tight generateUniqueID loop
but uses only one thread_local slot and a bit less global storage.
Reviewed By: strager
Differential Revision: D12927275
fbshipit-source-id: 94a5872c61dfe9dd441f1f34fab65f93c12997d8
Summary:
Update all of the C++ unit tests that create temporary files and directories
to use the new `facebook::eden::makeTempFile()` and
`facebook::eden::makeTempDir()` functions.
Note that this likely changes the behavior of some code paths in meaningful
ways: `/dev/shm` normally does not support `getxattr()`, and Eden's overlay
code attempts to store the SHA-1 for materialized files as using extended
attributes. This means that the tests will now typically hit the fallback
code path and will not store SHA-1 data in the overlay.
Reviewed By: chadaustin, strager
Differential Revision: D12971162
fbshipit-source-id: 6cc5eba2e04be7e9a13a30e90883feadfb78f9ce
Summary:
This function behaves similarly to the python `os.path.expanduser`
function, except that it is restricted to expanding only information for the
current user.
Use this new function to process the hgcache path returned via the
rust config parsing code.
In the longer term I want to centralize and add accessors for
the rust config and move this stuff into EdenConfig, but for
now this is the one place that does this and it is OK to
process it this way here.
Reviewed By: strager
Differential Revision: D12988902
fbshipit-source-id: 96b10640359c3b8c0adac1e14cd42dd592023c3d
Summary:
In a later diff, I needed to make generateUniqueID() noexcept. This
made me think that C++'s thread_local might be a little faster than
folly::ThreadLocal. So I added this benchmark.
Reviewed By: yfeldblum
Differential Revision: D12914249
fbshipit-source-id: 2c4acfff2162f66d13f456439d91df2ecb4167e3
Summary:
Add a very cheap unique ID allocator designed for extremely
low-latency uses like D10384071.
Differential Revision: D10501712
fbshipit-source-id: 15a0b7a2d344c0f6100082850dd967c585f3e2da
Summary:
relativize() returns an AbsolutePathPiece into its argument which is a
temporary AbsolutePath. The tests and ASAN didn't catch this because
they asserted the result's value before the temporary was destroyed, I
assume. Also the test strings fit into the SSO so there was no actual
memory allocation.
Reviewed By: simpkins
Differential Revision: D10261332
fbshipit-source-id: 912ffe6a46845cdd625e6dffbf65f0a0f8f89d0f
Summary:
Add a splitFirst() function that, if you think about a path as a linked list,
splits the head and the tail. Normally you could use the components
iterator, but split() is easier when the iteration is asynchronous and
requires lambda capture.
Reviewed By: pkaush
Differential Revision: D10154746
fbshipit-source-id: 6372a0bbde27f6596659ad1244cc96fa1f9eb6ff
Summary:
Add a ProcessAccessLog that supports cheaply tracking pids passed
into FUSE. It's a write-many, read-maybe access pattern, so the code
is careful to add as little latency as possible on the hot path.
Reviewed By: strager
Differential Revision: D9477839
fbshipit-source-id: 6928c1d09d55c2b0c0958ac2cb0dc91ec21b370c
