Summary: Several tests create a temporary directory and destroy it. Consolidate the temporary directories and their cleanup code to reduce code duplication.
Reviewed By: simpkins
Differential Revision: D10251110
fbshipit-source-id: caa5b5ad49fcb7925f63094594af4a20009d790d
Summary:
This diff clarifies that the errors are a result of speaking with
mononoke in their exception reason, and includes the original error context
in the case of: eg: a connection failure.
Reviewed By: chadaustin
Differential Revision: D10298353
fbshipit-source-id: fbccc39706819270cf19ef6ca8397734440f525e
Summary:
This comment is instructional but is better situated in the
header file.
Reviewed By: singhsrb
Differential Revision: D10298365
fbshipit-source-id: 07ad9757a6be06a97f6aae81cff94088b776c11f
Summary: This was previously hard coded and making it configurable is operationally desirable.
Reviewed By: chadaustin
Differential Revision: D10298279
fbshipit-source-id: 124c3fa319c44aacb3d17ef6e380e9dec49db8be
Summary:
This little function helps reduce the amount of boilerplate
and casting in the main body of the code. It should not change
and functionality.
Reviewed By: chadaustin
Differential Revision: D10297205
fbshipit-source-id: ea74c294cf2ca258e3e9abb0e47209626d58f2a5
Summary:
We were catching exceptions that occurred during setup of
the mononoke futures rather than catching errors that occurred during
processing of the mononoke futures themselves.
This diff switches the try/catch blocks to `onError` handlers on
the future chains, as well as improves an ambiguous error message.
Reviewed By: chadaustin
Differential Revision: D10297428
fbshipit-source-id: c87e4a23a70e010646b07d9d8728851bdfdbcc2a
Summary:
This diff dynamically detects whether eden is running on an older
kernel and sends back a smaller fuse_init_out response.
Reviewed By: simpkins
Differential Revision: D10282016
fbshipit-source-id: ce9701e8f39defd4b90b15fc941ad2e243ac61e9
Summary:
We've been shipping our RPMs with toml config support turned on for a
while now. Remove support for the old config file format. Continue to
ship the old format configurations. We'll remove those in a later
diff.
Reviewed By: strager
Differential Revision: D10020958
fbshipit-source-id: 11c2ca3b5da086b142042496a2814699880c4f81
Summary:
The logic we care about has been moved to core so we can delete the
`inhibit` extension.
Reviewed By: markbt
Differential Revision: D10276448
fbshipit-source-id: 44cabe5d561344cf8f196127ce52491e4654b598
Summary:
Using boost::filesystem to create the bind mount directories causes
edenfs to communicate with itself via FUSE, which causes edenfs to die
at startup (sometimes?). Instead, create the directories by traversing
the TreeInodes.
Reviewed By: simpkins
Differential Revision: D10154921
fbshipit-source-id: 4d8be71911e8a988b6fc5796904856f58a9a5153
Summary:
relativize() returns an AbsolutePathPiece into its argument which is a
temporary AbsolutePath. The tests and ASAN didn't catch this because
they asserted the result's value before the temporary was destroyed, I
assume. Also the test strings fit into the SSO so there was no actual
memory allocation.
Reviewed By: simpkins
Differential Revision: D10261332
fbshipit-source-id: 912ffe6a46845cdd625e6dffbf65f0a0f8f89d0f
Summary:
Add a splitFirst() function that, if you think about a path as a linked list,
splits the head and the tail. Normally you could use the components
iterator, but split() is easier when the iteration is asynchronous and
requires lambda capture.
Reviewed By: pkaush
Differential Revision: D10154746
fbshipit-source-id: 6372a0bbde27f6596659ad1244cc96fa1f9eb6ff
Summary:
This explains why some people ended up with broken bind mounts. If
bind mount creation ever failed, eden start would silently ignore the
error, leaving buck-out as a mutable directory in the Eden overlay
itself.
Reviewed By: wez, strager
Differential Revision: D10118612
fbshipit-source-id: ce99df4216f6815c24dc8283e55a9a30d58d29a1
Summary:
If 'eden stop' tries to tell edenfs to cleanly shut down, and edenfs never responds, 'eden stop' just hangs. Make 'eden stop' time out CLI -> edenfs Thrift calls. After a timeout, 'eden stop' should by stop edenfs via SIGKILL.
Note: This diff does not affect 'eden restart'.
Reviewed By: chadaustin
Differential Revision: D9982372
fbshipit-source-id: 42e498fb93fd97a2ec963e97c25f4b4101c7d524
Summary:
pexpect prints program output while the program is running, which is very useful when debugging. Make the tests in stop_test.py use this feature.
Also, add some assertions to some of the tests.
Reviewed By: chadaustin
Differential Revision: D10157354
fbshipit-source-id: 210fcde79d5d21c6ee34fad3813e7c56eb298b9b
Summary:
Added checks for kernel version and system type (eg. Linux, MacOS etc).
A check for running on linux has been added to ensure we're not comparing kernel versions from other unixy systems (eg. my mac kernel is currently version 17.7.0). OS tests are not run for non-linux kernels.
The configuration parameters (in doctor.py) MIN_KERNEL_VERSION should be set to the lowest supported kernel version. For bad kernel versions, add them to KNOWN_BAD_KERNEL_VERSIONS as strings or regexes.
Reviewed By: chadaustin
Differential Revision: D10126747
fbshipit-source-id: a0c60351377a9b85e9d76376c7c61fbf63f74a35
Summary: If edenfs is unresponsive, 'eden status' hangs forever. Add a timeout to turn the hang into a user-friendly error.
Reviewed By: chadaustin
Differential Revision: D10156229
fbshipit-source-id: 9186826ae6b131a193b1499c8baac616d131357f
Summary: Small change that makes the use case more explicit.
Reviewed By: wez
Differential Revision: D10245746
fbshipit-source-id: cb8312e5f5eddf2dfe0a4c8b98748932201ece18
Summary:
Add a direct getSize() accessor to Blob. The thinking here is that all
of this information is known and in cache when the Blob is
constructed, so there's no need to walk a list later on.
Reviewed By: simpkins
Differential Revision: D10245695
fbshipit-source-id: f6d5abbae75d468085dcc02bbbac8aa6239a7c70
Summary:
The eden server library used to rely on a symbol being defined by the
executable that linked it. Pass runServer as a parameter to
EdenServer::run to allow us to leave undefined_symbols=False in the
library rule.
Reviewed By: wez
Differential Revision: D10246972
fbshipit-source-id: d4c47e58947a2d57c4163134db2455fe9c815344
Summary:
I want to test more conditions for 'eden stop', such as the edenfs process taking a long time to respond to the Thrift stop request. Refactor how fake_edenfs decides to handle stop requests to make it easy to add new behavior.
Aside from changing the log messages, this diff should not change behavior.
Reviewed By: wez
Differential Revision: D10019459
fbshipit-source-id: f6d5f514a67435a446d8d672bf7d21af77d76c56
Summary:
* Add type annotations to struct.unpack results to prevent leaking Any.
* Make the type signature for read and write as similar as possible (without changing behavior). This will help a future diff (D10224078) which makes the signatures match exactly.
This diff should not change behavior.
Reviewed By: chadaustin
Differential Revision: D10224079
fbshipit-source-id: 5d43d6fbbc989f71dae006dc3d378600d49272d3
Summary:
Instead of calling getBlobMetadata in multiple places and only using
the .sha1 field, add a getSha1 function directly to ObjectStore. This
gives ObjectStore the latitude to fetch it and store it in different ways.
Reviewed By: wez
Differential Revision: D10227935
fbshipit-source-id: 180830534db3c42c07f04216599e496406af5ced
Summary: Always send write requests straight to the inode rather than going through FileHandle.
Reviewed By: wez
Differential Revision: D10220619
fbshipit-source-id: 9ce328583cf0fa9d7d8850d92d9e15ddc382d6a3
Summary:
Always send read requests straight to the inode rather than going
through the FileHandle.
Reviewed By: wez
Differential Revision: D10220604
fbshipit-source-id: 6aa5d20f3ce09696a29bd5c1cb95d0b987ab213c
Summary: I want to make minor changes to eden.dirstate. Add some basic tests to increase code coverage from 0%.
Reviewed By: chadaustin
Differential Revision: D10223605
fbshipit-source-id: 09835d8052b1460915b4091488f525d4268b3e51
Summary:
Always send setattr and getattr straight to the inode rather than
going through the FileHandle.
Reviewed By: wez
Differential Revision: D10187876
fbshipit-source-id: 4c3aaa977cd568d5f9cc4b28583e164119c07c1b
Summary:
EdenFileHandle is just a thin wrapper around a FileInodePtr, so remove
the last vestiges of interesting logic from it.
Reviewed By: wez
Differential Revision: D10187221
fbshipit-source-id: 327e99ae0d860bcc010e31753e7226f2a6f953fd
Summary: Eliminate the need to look up an InodeNumber from a FileHandle. Instead, simply preserve the mapping when it's created.
Reviewed By: wez
Differential Revision: D10187120
fbshipit-source-id: dc47f7776294871ff2398f33c31bd85d240ead50
Summary:
In order to implement FUSE_NO_OPEN_SUPPORT, we must eliminate
FileHandle and FileHandleBase. They didn't add any value anyway. Start
clipping.
Reviewed By: wez
Differential Revision: D10187103
fbshipit-source-id: 81e226f9c12486e0bbbde99b798b169fa31740c2
Summary:
Part of the larger project to modify Future<T>::then to be r-value qualified and use Future<T>::thenTry or Future<T>::thenValue.
The goal is to disambiguate folly::Future and to improve type and lifetime safety of Future and its methods.
Codemod:
future<T>.then(callable with operator()(not-a-try)) to future<T>.thenValue(callable with operator()(not-a-try)).
future<T>.then(callable with operator()()) to future<T>.thenValue(callable with operator()(auto&&)).
future<T>.then(callable with operator()(auto)) to future<T>.thenValue(callable with operator()(auto)).
future<T>.then(callable with operator()(folly::Try<T>)) to future<T>.thenTry(callable)
Reviewed By: Orvid
Differential Revision: D10227086
fbshipit-source-id: 1bb31c91cc65e28291e39302627f97801bfde15c
Summary:
Aborting the process with self-access is overlay aggressive. We
already respond with EIO, and the calling syscall should handle the
error. Continue logging at CRITICAL.
Reviewed By: wez
Differential Revision: D10134987
fbshipit-source-id: a4c4eed5e5de20698e95f442b3e063a09db311e6
Summary: We've diverged in a few places from clang-format, so run it across the entirety of Eden.
Reviewed By: wez
Differential Revision: D10137785
fbshipit-source-id: 9603c2eeddc7472c33041ae60e3e280065095eb7
Summary:
If an edenfs thread accesses an Eden mount, it trips SIGABRT from the
DFATAL, but the SIGABRT can't end the process because one thread is
stuck in uninterruptible sleep. So respond to the FUSE request before
aborting the process.
Reviewed By: strager
Differential Revision: D10118625
fbshipit-source-id: b84063e0b186d6a464531284b70c25b0b6a710ce
Summary:
When running the watchman integration tests, I saw that some of them were
fataling here:
diffusion/FBS/browse/master/fbcode/thrift/lib/cpp2/async/StreamPublisher-inl.h$71
In this situation we're tearing down all the subscribers via
`Journal::cancelAllSubscribers` and this causes the stream be destroyed and
trigger this FATAL check.
We need to explicitly complete it before we can proceeed, so we do the gross
thing and call this in the destructor of a little wrapper.
This is made a little more complex because the disconnection handler has to
inform the Journal that it was torn down in the case where the client has
gone away. The result is that we need to be careful to avoid the destructor
on Journal callback from running while we hold the Journal subscriber lock.
Reviewed By: strager
Differential Revision: D10024271
fbshipit-source-id: 06a9d40f7f6e46fe35ffcedba2669e27e6624427
Summary: Reformat all opted-in python code with version `18.9b0` of Black.
Reviewed By: ambv
Differential Revision: D10121017
fbshipit-source-id: 08404dba959bc63bcd7eee7eafe1753c9cfb58ee
Summary:
In D6766900, we turned on automatic background inode
unloading. However, there's a bug somewhere in InodeMap or the loading
sequence that causes occasional crashes for users, so turn it back off
for now.
Reviewed By: pkaush
Differential Revision: D10114443
fbshipit-source-id: fd35fd31b5731aa9bf26004fca8aa73930bb2344
Summary:
Restructured the Windows code to align with the eden code layout. Plus changed the build location to eden/win/build directory, which is generated by the Windows build script.
eden
\_fs
\_ ...
\_ win
\_fs
\_service
\_utils
\_build (generated by the build script)
Reviewed By: strager
Differential Revision: D10081143
fbshipit-source-id: db9fb25f963d1a9cccb8a8f83646e7e45c87d409
Summary: Taking (#ifdef'ing) out some of the mononoke code from the Windows builds. Mononoke support will come once we compile the proxygen on Windows.
Reviewed By: strager
Differential Revision: D10079379
fbshipit-source-id: df431f9c57832d43811af41d4512674f1108cacf
Summary:
I noticed that, when Eden hangs, an importer is usually stuck for a
long time too. Add information about edenfs's child processes in
`eden rage`.
Reviewed By: strager
Differential Revision: D10020317
fbshipit-source-id: afe82f559ea0905f10c757fc0b05c3ff64f5ab41
Summary: HypothesisSimpleTest.test_create is timing out on Sandcastle (Facebook's CI). It looks like the hypothesis tests run indefinitely on Sandcastle. Set a time limit so the tests complete and provide signal.
Reviewed By: wez
Differential Revision: D10053452
fbshipit-source-id: 8c8cb2c375a73b7727e827ea2a9e7d56ab390e54
Summary:
'eden stop' has poor test coverage. Write tests for the common case and some edge cases which 'eden stop' already handles.
This diff should not change behavior.
Reviewed By: wez
Differential Revision: D10017819
fbshipit-source-id: 4d9f5db52187c34c62a9379a6b3dd62f62894233
Summary:
When a file's contents are not cached by the kernel, open(), read(),
and close() are all piped into the FUSE daemon. But when a file's
contents ARE cached by the kernel, only open() and close() are. Thus,
in the common case, the kernel will notify us that a file is being
opened, but read() will be served out of cache.
In that case, prefetching the blob is not beneficial, because it will
be dropped anyway upon close().
In situations where the VFS cache is hot but eden's own caches are
cold, this is probably a win.
Reviewed By: strager
Differential Revision: D10044546
fbshipit-source-id: eeb0854dbff021b2c73f1a42f31a94dd9fcf0837
Summary:
Previously, Eden would log `UnloadInodeScheduler Unloading Free
Inodes` every 10 minutes. Now it only logs if it actually unloads an
inode for a mount.
Reviewed By: strager
Differential Revision: D9999029
fbshipit-source-id: f01d0e7b1341bef230bcb5a327c31ffc8e3aeb2c
Summary:
We're looking into doing some performance optimizations in Eden to the
open() and close() calls, since they are never handled by the kernel
and always pass through to the underlying user-space FUSE daemon. But
first, let's capture some baseline numbers on Eden and non-Eden
checkouts.
Reviewed By: strager
Differential Revision: D10036941
fbshipit-source-id: 64f3414a4572fd963017491db37d70e6b5ae4f24
Summary:
My eden crashed with an invariant violation in InodeMap. It looks
related to background unloading, so I attempted to reproduce the
sequence in a unit test.
These tests pass but they're worth checking in I think.
Reviewed By: strager
Differential Revision: D9998970
fbshipit-source-id: d182fc8d5b185c286082320adda3ea4c862f2f18
Summary:
There's a lot of logic here that need not be offloaded to the
importer threads
Reviewed By: chadaustin
Differential Revision: D9539811
fbshipit-source-id: 1379eef390df6400291a61263d003b493a474b81
Summary: This function is never used.
Reviewed By: strager
Differential Revision: D9998991
fbshipit-source-id: 27a8f5180d7516c3bf61b11192672142f77abccc
Summary: D6612669 introduced a workaround for edenfs not running as a child of sudo if you run Eden's integration tests as root. D9980225 fixed the underlying bug in 'eden stop' which necessitated the workaround. Remove the now-unnecessary workaround.
Reviewed By: wez
Differential Revision: D9984524
fbshipit-source-id: b9ed7114cd7385a13899d16b4da0d40e6e9fd704
