Summary:
Calls to ObjectStore::create and EdenMount::create are duplicated in the various TestMount::initialize overloads. Fix the duplication by moving the two calls into their own function.
This diff should not change behavior.
Reviewed By: chadaustin
Differential Revision: D14058972
fbshipit-source-id: b3e79756793292b59f09f53c05649a637c8af07e
Summary:
getInitialMode and getModeUnsafe were not inherently different, so
remove getModeUnsafe and write an accurate comment in Overlay's
serialization code about why the initial mode is saved and how we
could remove that logic in the future.
Reviewed By: strager
Differential Revision: D14007488
fbshipit-source-id: db42f45f00dcd213fabd9575360da1261931778b
Summary:
It's common for code to use InodeNumber without needing to include the
main FUSE headers or vice versa. Split them into two separate headers.
Reviewed By: strager
Differential Revision: D13979868
fbshipit-source-id: c5eeb6a3697bb538729a403434dc4f0f7408cda0
Summary:
Previously the default ignore path was `~/ignore`. This doesn't really seem
like a great path choice, and I suspect that no one is actually using an
ignore file at this location. This default was set in D8594297, but I it
looks like this was mostly accidental from just not separating out data for
the system ignore path (`/etc/eden/ignore`) vs the user ignore path.
Using `~/.edenignore` seems reasonable for now, and unlikely to conflict with
other paths. We used to use `~/.gitignore` as the default before D8906226,
but this did cause problems for a handful of users that treated their entire
home directory as a git repository, and had a `~/.gitignore` file that was
supposed to apply only to their home directory.
Reviewed By: wez
Differential Revision: D13984153
fbshipit-source-id: 887528372b9be789317933f7026dfcbde8cd4539
Summary:
Update FileChangeMonitor to stop logging messages about files not being
present. We normally use FileChangeMonitor to monitor the user's `edenrc`
configuration files and their personal and system-wide ignore files. It is
normal for these files to not exist in many cases, so there is no need to log
a warning message in this case.
Reviewed By: chadaustin
Differential Revision: D13981278
fbshipit-source-id: fa5f8d42980fd8683f55d3d51f5375a72f511dfe
Summary:
This diff enables building the eden watcher by linking in thrift and
its various dependencies.
To support building in-fbsource and in the github repo, a `maybe_shipit_dir`
function is used to setup a symlink to the `eden` and `fboss/common` dirs (this
mirrors the shipit configuration for this project: we cannot simply run shipit
because we have to build on mac and windows and shipit requires Hack, and that
does not support those platforms).
I tried to persuade cmake to let me build this without the use of a symlink but
found it too difficult to teach everything about the path mapping. The
symlinks aren't terrible, but are the reason why this diff also updates some
`.gitinore` files that are seemingly unrelated to this diff.
This diff changes a couple of build/link options: without them the end product
fails to link either due to implicit/unilateral enablement of UBSAN in some of
the deps, or because warning->error promotion is turned on.
This diff includes a copy of the `ThriftCppLibrary.cmake` file from the fboss
repo. This should get centralized and shipit'ed out into the places that
consume it. That can be done when someone gets around to doing the same for
the `FindGlog.cmake` file and doesn't need to hold up this diff.
Reviewed By: simpkins
Differential Revision: D13486486
fbshipit-source-id: 3bb5b011771b2a87618147ca019b4e50a8e0aaf2
Summary:
Replace Future::onError with Future::thenError:
* to remove ambiguous typing
* to ensure that the executor is not lost and the returned Future is still bound to an executor
See:
https://fb.workplace.com/groups/fbcode/permalink/2002251863144976/
for details.
Reviewed By: yfeldblum
Differential Revision: D13908257
fbshipit-source-id: 8b5315b019290f1c60087ca5716c31ebbf1f1be5
Summary: FuseTypes.h defines the InodeNumber struct, but also has a comment which implies that InodeNumber is an alias of uint64_t. Delete this misleading commented-out definition of InodeNumber.
Reviewed By: chadaustin
Differential Revision: D13892907
fbshipit-source-id: 8e9710db29d8e0d708ee1785956f6953a0b2a49f
Summary:
When debugging Eden's behavior under a kernel that caches readdir
calls, I observed that directories were not returning the correct
entries after a checkout. The issue is that Eden calls INVAL_ENTRY
when new entries are added, but INVAL_ENTRY is a no-op and returns
ENOENT if the kernel doesn't know about that entry.
Reviewed By: strager
Differential Revision: D13864281
fbshipit-source-id: 56b3a67d0bc6f8ed1733acaf5e889414b753cbc7
Summary: `hgext` was moved to `edenscm.hgext`.
Reviewed By: simpkins
Differential Revision: D13884629
fbshipit-source-id: 4bf69720cccbd43665cd350ac8a3ba4d46b10b93
Summary:
Replace Future::onError with Future::thenError:
* to remove ambiguous typing
* to ensure that the executor is not lost and the returned Future is still bound to an executor
See:
https://fb.workplace.com/groups/fbcode/permalink/2002251863144976/
for details.
Reviewed By: yfeldblum
Differential Revision: D13784772
fbshipit-source-id: 1d3ede848b7d31c7a197a21b4ff2b31e840040a5
Summary:
D13853115 adds `edenscm/` to `sys.path` and code still uses `import mercurial`.
That has nasty problems if both `import mercurial` and
`import edenscm.mercurial` are used, because Python would think `mercurial.foo`
and `edenscm.mercurial.foo` are different modules so code like
`try: ... except mercurial.error.Foo: ...`, or `isinstance(x, mercurial.foo.Bar)`
would fail to handle the `edenscm.mercurial` version. There are also some
module-level states (ex. `extensions._extensions`) that would cause trouble if
they have multiple versions in a single process.
Change imports to use the `edenscm` so ideally the `mercurial` is no longer
imported at all. Add checks in extensions.py to catch unexpected extensions
importing modules from the old (wrong) locations when running tests.
Reviewed By: phillco
Differential Revision: D13868981
fbshipit-source-id: f4e2513766957fd81d85407994f7521a08e4de48
Summary:
Move top-level Python packages `mercurial`, `hgext` and `hgdemandimport` to
a new top-level package `edenscm`. This allows the Python packages provided by
the upstream Mercurial to be installed side-by-side.
To maintain compatibility, `edenscm/` gets added to `sys.path` in
`mercurial/__init__.py`.
Reviewed By: phillco, ikostia
Differential Revision: D13853115
fbshipit-source-id: b296b0673dc54c61ef6a591ebc687057ff53b22e
Summary: using upgrade script to clear out all remaining version-set configs
Reviewed By: dark
Differential Revision: D13832474
fbshipit-source-id: 52c280cbd79b1410821ed829465b1c0907b50a86
Summary: PrivHelperServer::initLogging is declared in the .h file, but it's never defined. Remove the useless declaration.
Reviewed By: simpkins
Differential Revision: D13814416
fbshipit-source-id: 25cb47442a19947da08d13d9bed9b4631a1c9739
Summary: Currently, `runOnDestruction` aims to be thread-safe; new callbacks are added to the `onDestructionCallbacks_` list while the associated mutex is held. However, the caller may own the `LoopCallback` and wish to destroy/cancel it before the `EventBase` destructor runs, and this callback cancellation is not thread-safe, since unlinking does not happen under the lock protecting `onDestructionCallbacks_`. The primary motivation of this diff is to make on-destruction callback cancellation thread-safe; in particular, it is safe to cancel an on-destruction callback concurrently with `~EventBase()`.
Reviewed By: spalamarchuk
Differential Revision: D13440552
fbshipit-source-id: 65cee1e361d37647920baaad4490dd26b791315d
Summary:
Now that the deadlock in ProcessNameCache has been fixed, bring it
back.
Reviewed By: strager
Differential Revision: D13742965
fbshipit-source-id: f407105e06b9954766bdb48ef1303e2003c07284
Summary:
This fixes a deadlock where the kernel held the memory manager lock while satisfying a page fault resulting in a call into Eden, which tried to read /proc/pid/cmdline, acquiring the memory manager lock again.
The fix is to never read /proc/pid/cmdline while handling a request. That work is now done on a background thread.
Reviewed By: strager
Differential Revision: D13685318
fbshipit-source-id: c4e17de3358b668638db0c2dcfba8536d05e5b7c
Summary:
Previously edenfs always returned `ALIVE` in response to the fb303
`getStatus()` call. This fixes the code to return `STARTING` while Eden is
still starting and `STOPPING` while Eden is shutting down.
Reviewed By: strager
Differential Revision: D13515856
fbshipit-source-id: 91fe52e5f7f0a9e2c48ca1dd7663c99319afa8ad
Summary:
When graceful restart was first implemented we forgot to update the
lock file with the new pid, resulting in occasional unexpected output
from tools like eden doctor.
Reviewed By: simpkins
Differential Revision: D13744411
fbshipit-source-id: cdc758ed6ac1201fd2ff3e9d7805bb5ab6f83e8a
Summary:
This is something that is not needed on linux because
the kernel module is typically already loaded (at least on the systems
on which we run).
On macos, since fuse is not part of the kernel, libfuse has some code
that loads it when needed.
This diff performs the equivalent actions for eden.
Reviewed By: simpkins
Differential Revision: D13721489
fbshipit-source-id: 627bc90681141d0e7da3d5b5e06756a36839958c
Summary:
This requires our mercurial repo to be available during
the build; I symlink it in alongside `common` in the `oss` dir,
and point it up to `scm/hg`.
This has partial support for mononoke too, but will need to add
logic to getdeps to pull down the proxygen repo and build that.
Reviewed By: simpkins
Differential Revision: D13480146
fbshipit-source-id: 54874245015af83a259f56944d2e5f87615baee7
Summary:
Note that the concept of bind mounts doesn't exist on macos, so that
portion of the server just throws.
Reviewed By: simpkins
Differential Revision: D13480147
fbshipit-source-id: 92225188c0af42574d090004490f3926d393747b
Summary:
In our linux deployments it was relatively straightforward
to import the mercurial runtime from a python process running the
system python executable. Our macOS deployments are a lot more
complex because they do not use the system python and do not install
the mercurial python packages in the python path of the target
python executable.
It is simpler to move the import helper functional into a mercurial
command that we can invoke instead of our own helper program.
This diff moves the script to be a debug command and adjusts its
argument parsing to match the mercurial dispatcher requirements.
There are some stylistic mismatches between this code and the
rest of mercurial; I'm suggesting that we ignore those as the
medium term solution is that this command is replaced by eden
directly consuming the rust config parsing code and by native
rust code to perform the data fetching that we need.
Reviewed By: pkaush
Differential Revision: D13522225
fbshipit-source-id: 28d751c5de4228491924df4df88ab382cfbf146a
Summary:
This updates the logic in EdenServer to add the EdenMount to the mountPoints_
map as soon as it is created, so that we track mount points as they are
initializing.
I don't expect this change to have any major impact in functionality yet. In
a subsequent diff I also plan have EdenServer keep mount points in the
mountPoints_ map longer while they are shutting down. I expect that change to
matter a bit more, as that will allow us to do a better job reporting and
debugging when mount points are taking a non-trivial amount of time to become
unreferenced and fully shut down.
Reviewed By: strager
Differential Revision: D13503050
fbshipit-source-id: 2e0e8dfde64c6a005efd6dcf503ad7577f314356
Summary:
This is really a continuation of D13479516; the issue is that
the osxfuse kernel module is very eager to recycle `unique` request
id values, recycling them before our code has had a chance to update
internal state.
This diff re-keys the requests map so that we generate our own sequence
of identifiers to use as the key rather than the fuse protocol `unique`
value.
Because we cannot reliably track by `unique` value we also cannot
reliably implement interrupt support. We've never really tested
interrupt support, and it relies on functionality in folly futures
that hasn't really been tested or proven either, so I've removed
that functionality as part of this diff.
That allows simplifying some code in RequestData and FuseChannel;
we're now able to simply tack an `.ensure` on the end of the
future chain to ensure that we remove the entry from the map
once the future is resolved, successfully or otherwise.
Reviewed By: chadaustin
Differential Revision: D13679964
fbshipit-source-id: c1081a868c4061de2a725589ec1614959a8e9316
Summary:
Migrate the code from accessing optional Thrift fields directly to a safer
`optional_field_ref` API. See https://fburl.com/safe for more details.
The output of this codemod has been reviewed in D13259011.
To preserve semantics, each unchecked access is replaced with an explicit call
to `value_unchecked()`. If you are sure that accessing a field is safe (the
field is marked as set), you can later replace `value_unchecked()` with
`value()` or dereferencing (`operator *`):
```
ThriftStruct s = ...
- auto foo = s.foo_ref().value_unchecked();
+ auto foo = *s.foo_ref(); // will throw if s.foo is unset
```
Reviewed By: chadaustin
Differential Revision: D13684410
fbshipit-source-id: 919de4ddf89e7f0463f2614baba4bfbac1c8255c
Summary: the build breaks when making clean unless we declare this dep
Reviewed By: simpkins
Differential Revision: D13679633
fbshipit-source-id: f23a533eab9e37fdeab839e4f5e1b6b312ea10b0
Summary: This just makes the debug a little easier to follow.
Reviewed By: chadaustin
Differential Revision: D13680020
fbshipit-source-id: e4045822e56ba42a831ccb0ceaa9baaba5b79a10
Summary:
the osxfuse implementation includes some additional
fields in the `fuse_attr` struct. One of those fields is
`flags`. We were not initializing this field when converting
the stat data to fuse attributes which resulted in it holding
"random" data. In debug builds this seemed to usually end up
zeroed out but in release builds it was usually a bit pattern
that caused the kernel to respond with EPERM when attempting
to access the files. I didn't capture exactly what that
bit pattern was, just that initializing the struct to zeroes
reliably fixed up the EPERM issues in the Release build.
Reviewed By: chadaustin
Differential Revision: D13680004
fbshipit-source-id: 6b2ce6c10ef8f7db4a8a50bd3f2ddcfdddc3bb45
Summary: Small things I noticed while working on other stuff.
Reviewed By: strager
Differential Revision: D10055671
fbshipit-source-id: de8c3b04928567a821172e6fa7ee0e056958e1e7
Summary:
Address this error with clang:
```
In file included from /Users/wez/fbsource/fbcode/eden/oss/eden/fs/service/main.cpp:25:
/Users/wez/fbsource/fbcode/eden/oss/eden/fs/fuse/privhelper/PrivHelper.h:22:1: warning: class 'Unit' was previously declared as a struct [-Wmismatched-tags]
class Unit;
^
/Users/wez/fbsource/fbcode/eden/oss/external/install/include/folly/Unit.h:36:8: note: previous use is here
struct Unit {
^
/Users/wez/fbsource/fbcode/eden/oss/eden/fs/fuse/privhelper/PrivHelper.h:22:1: note: did you mean struct here?
class Unit;
^~~~~
struct
```
Reviewed By: strager
Differential Revision: D13602383
fbshipit-source-id: 6e69716498680660181ab441c3c007b074ec1d40
Summary:
If a file in the overlay is truncated unexpectedly, reads on it would
fail with EIO and a log message of "std::out_of_range: string
underflow" which wasn't helpful. Instead, use the same truncated file
detection as directories use.
Reviewed By: strager
Differential Revision: D13627565
fbshipit-source-id: 246f2659ba139e8f7adb7d556719e5ead9d84ebd
Summary:
Add `ensureStateTransition()` and `unconditionalStateTransition()` helper
methods to EdenMount, for performing state transitions where we want to verify
the current state and where we do not care about the current state,
respectively. Update most call sites that change the state to use these
methods.
Also change the code to use acq_rel semantics for updating the state,
rather than seq_cst, which is not necessary in this case. We do not care
about having a consistent visible ordering of change to the state variable
relative to other atomic variables.
Reviewed By: strager
Differential Revision: D13579333
fbshipit-source-id: 5fd62e740a7ea2777f79f722bbde7f5b65255cb6
Summary:
To determine the precise behavior of unmounting an Eden mount with
MNT_FORCE, I needed to see whether ENODEV was being returned from the
FUSE socket.
This helps us clarify the documentation in libfuse
https://github.com/libfuse/libfuse/issues/333
Reviewed By: strager
Differential Revision: D13630289
fbshipit-source-id: 90e6f0afc927c042a24cd6c82deac644c15ed066
Summary:
In the future, we will likely coalesce redundant fetches at the
BlobAccess/ObjectStore layer. To measure what we actually want,
populate a normal ObjectStore with a NullLocalStore and add counters
to FakeBackingStore.
Reviewed By: wez
Differential Revision: D13454331
fbshipit-source-id: 2fbf393d159f94e84c24ac53ccc207162fa754b7
Summary:
This changes prefetch so that it loads all of the direct children of
the tree. This improves `time ls -l bigdir` performance by 2x.
Reviewed By: wez
Differential Revision: D12888690
fbshipit-source-id: eb8c8274bd9c5b5edc94d7092a5feb492fad6d66
Summary:
We think that it shouldn't really be needed to perform
the prefetch call during lookup; for file inodes it doesn't buy
us much, and it should only really help for readdir.
This removes the prefetch call from lookup, instead prefetching
upon the first readdir() of a loaded TreeInode.
Reviewed By: simpkins
Differential Revision: D12896022
fbshipit-source-id: 0209eb64bd522daf5f7461dffccd1312d32a1554
Summary:
Update the `FuseTest.destroyWithInitRace()` test to succeed even if the future
returned by `startFuse()` completes with an exception.
Even though the test waits to see the `FUSE_INIT` response sent back to the
kernel, there is more initialization work performed by EdenMount after the
`FUSE_RESPONSE` is sent back. This initialization code can potentially fail.
At the moment the initialization code generally succeeds even if the
`EdenMount` has already transitioned to the `SHUT_DOWN` state. However, I
plan to change the `EdenMount` code soon to error out in this case. This
currently will cause this test to fail with its existing behavior.
Reviewed By: strager
Differential Revision: D13503048
fbshipit-source-id: 6ff147d8679559f0520f5e6091291c3a07bba3ed
Summary:
Update the `listMounts()` thrift API to also report the current mount point
state. This will allow us to do a better job of reporting mount points that
are in the process of initializing or shutting down.
This change splits the `MountInfo` thrift type into two distinct types for
the `listMounts()` vs `mount()` APIs. However this change should be
completely backwards compatible at the wire protocol level for older client
and server code.
Reviewed By: strager
Differential Revision: D13503049
fbshipit-source-id: 68e7ca708b956991c8fd93bbf8973d90650aced9
Summary:
Some people encounter system-wide hangs on their Linux machines. Debugging points to a deadlock related to EdenFS' process name lookup code. Disable the process name cache during FUSE dispatch to avoid the deadlock.
Effects:
* Hopefully, the deadlock no longer happens.
* 'eden top' will no longer report process names. (It should work otherwise, though.)
Reviewed By: simpkins
Differential Revision: D13540947
fbshipit-source-id: 595c36150a5f8ff1b8e7cd81d8f61ee1463d96eb
Summary:
To mitigate a deadlock, I want to make ProcessAccessLog not access /proc/. Allow this by making ProcessNameCache optional.
This diff should not change behavior.
Reviewed By: simpkins
Differential Revision: D13540948
fbshipit-source-id: 4c5d68c972c04122de1d2414084debfec078dd4c
Summary:
Eden can handle parallel readdir and lookup so don't require the
kernel to acquire a mutex just to serialize the requests.
5c672ab3f0
Reviewed By: strager
Differential Revision: D13386133
fbshipit-source-id: aa935af941ff2901b07b63751c97052c295f7076
Summary:
The fuse opcodes are defined as an enum so we have to use
the relatively coarse and indirect apple vs linux preprocessor
checks in the maps for the opcode names.
The osxfuse implementation branched off from the 7.19 fuse
implementation, so add a light dusting of some preprocessor
checks around enabling the performance optimization features
we desire on Linux.
We also need to relax the compile time check for the min
fuse version; I've constrained this to be apple specific,
although I suppose it wouldn't hurt to make it more broadly
applicable.
Reviewed By: chadaustin
Differential Revision: D13480145
fbshipit-source-id: 010ac114e22ea942dfcebf1105cb1f01b766f297
Summary:
There's nothing nice about this; the full set of kernel headers are
not installed with the binary distribution, and since the kernel distribution
has to be signed to be loaded on osx, there's no benefit to us building it
for ourselves.
This diff adds a nop builder and tweaks the cmake to point into the osxfuse
repo.
The osxfuse repo aggregates a couple of related repos using the git
submodule feature, so trigger that from getdeps.py too.
Reviewed By: strager
Differential Revision: D13480148
fbshipit-source-id: 84e09a86f6a83f83ffd1e3fe113dc7b15b3ea208
Summary:
a couple of parts of the rust, datapack and mononoke code
didn't have appropriate definitions.
Allow `useDatapackGetBlob_` to be default initialized per its
definition in the header file rather than replicating the relatively
complex ifdef logic in the implementation.
Reviewed By: strager
Differential Revision: D13475710
fbshipit-source-id: d2955c9b22f1186f5897aa8bdbd9046b8f6b5f7a
Summary:
handle this best-effort by setting this bit on each fd after
allocating them.
Reviewed By: strager
Differential Revision: D13475712
fbshipit-source-id: 46be80f025b21967f75822f983bc327c5e2d20af
Summary:
ScopedHTTPServer allows callers to specify a simple request handler function
instead of having to define their own handler factory. This updates the
MononokeBackingStore tests to change the `Handler` class in to a simpler
functor object, and delete the `HandlerFactory` class.
Reviewed By: wez
Differential Revision: D13476411
fbshipit-source-id: 0ede232ff9570c95e877b689272ea8eb26d97d83
Summary:
Change the code to use ScopedHTTPServer instead of its own custom logic that
starts a server and runs all of the test logic inside the `onSuccess()`
callback.
Doing all of the test logic inside `onSuccess()` is generally bad, as most of
the tests perform blocking `get()` calls on `folly::Future` objects. Blocking
work generally should not be done from inside EventBase threads.
When built with ASAN these tests occasionally crash with heap-use-after-free
errors during the HTTP server shutdown. I didn't track down the exact bad
behavior that was causing this, but attempting to stop the server from inside
the `onSuccess()` callback does seem a little fishy. Hopefully simply
switching to `ScopedHTTPServer` will fix this issue.
Reviewed By: wez
Differential Revision: D13476413
fbshipit-source-id: ab92cc16a5bf99a5e7b52529012a03786495c319
Summary:
Switch to ScopedEventBaseThread for the standalone EventBase thread create in
the unit tests. This way the test does not need its own custom logic for
starting and joining this thread. `ScopedEventBaseThread` also ensures that
the thread is actually running and ready to process events before its
constructor returns.
Reviewed By: wez
Differential Revision: D13476412
fbshipit-source-id: 260a40d93050e2e9b92ef9efd1549633679f36f7
Summary:
Update the Hash constructors that accept a `ByteRange` and a `StringPiece` to
be `constexpr` so that all Hash constructors are now `constexpr`.
This probably shouldn't really make a big difference in practice. I added
this since I wanted to define some static `Hash` constants in some tests, and
didn't want to worry about SIOF issues.
Reviewed By: chadaustin
Differential Revision: D13475781
fbshipit-source-id: fc1ce91c998f1badadbd6becd525458c25dd30de
Summary:
Avoid this compilation error:
```
[ 23%] Building CXX object eden/fs/fuse/CMakeFiles/eden_fuse.dir/Dispatcher.cpp.o
In file included from /Users/wez/fbsource/fbcode/eden/oss/eden/fs/fuse/Dispatcher.cpp:10:
In file included from /Users/wez/fbsource/fbcode/eden/oss/eden/fs/fuse/Dispatcher.h:12:
In file included from /Users/wez/fbsource/fbcode/eden/oss/external/install/include/folly/Portability.h:19:
In file included from /Library/Developer/CommandLineTools/usr/include/c++/v1/cstddef:110:
/Library/Developer/CommandLineTools/usr/include/c++/v1/type_traits:3141:38: error: incomplete type 'facebook::eden::DirList' used in type trait expression
: public integral_constant<bool, __is_constructible(_Tp, _Args...)>
^
/Users/wez/fbsource/fbcode/eden/oss/external/install/include/folly/futures/Future.h:146:36: note: in instantiation of template class 'std::__1::is_constructible<facebook::eden::DirList>' requested here
typename std::enable_if<std::is_constructible<T, Args&&...>::value, int>::
^
/Users/wez/fbsource/fbcode/eden/oss/external/install/include/folly/futures/Future.h:148:12: note: while substituting prior template arguments into non-type template parameter [with Args = <>]
explicit FutureBase(in_place_t, Args&&... args);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/wez/fbsource/fbcode/eden/oss/external/install/include/folly/futures/Future.h:115:7: note: while substituting deduced template arguments into function template 'FutureBase' [with Args = <>, $1 = (no value)]
class FutureBase {
^
/Users/wez/fbsource/fbcode/eden/oss/eden/fs/fuse/Dispatcher.cpp:175:13: note: in instantiation of template class 'folly::Future<facebook::eden::DirList>' requested here
Dispatcher::readdir(InodeNumber, DirList&&, off_t, uint64_t) {
^
/Users/wez/fbsource/fbcode/eden/oss/eden/fs/fuse/Dispatcher.h:35:7: note: forward declaration of 'facebook::eden::DirList'
class DirList;
^
1 error generated.
```
Reviewed By: chadaustin
Differential Revision: D13475720
fbshipit-source-id: 2da1692010a82b73cbed71d996993cf5fc13af0e
Summary:
It appears as though osxfuse is a bit eager to re-use `header->unique`
values, as this DCHECK to ensure that we erased our request at the end of a
request is frequently triggered. It's not for 100% of requests, but I did
notice that the `header->unique` values tend to be in the range 1-6.
We don't check for collisions when we emplace requests in the dispatching
logic, and this particular check happens after we've acked the kernel, so it
doesn't seem like a safe thing to check and abort on really, so let's remove
it.
Reviewed By: chadaustin
Differential Revision: D13479516
fbshipit-source-id: a01e6c3e47b78b651f65fc3f5138168c71968030
Summary:
When run inside the systemd service (fb-edenfs@.service), edenfs' logs are written to `/var/log/messages` (on Facebook dev servers). This is undesirable, since those logs have a bunch of noise.
Make systemd-managed edenfs log to `~/local/.eden/logs/edenfs.log` instead, matching the behavior of custom-managed edenfs.
---
I considered using systemd's StandardOutput= and StandardError= directives [1], but they have limitations:
* **StandardOutput=file:%f/logs/edenfs.log**: When the `logs` directory is missing, systemd does not create it. In this case, systemd fails when it opens the log file, so systemd refuses to start the service.
* **StandardOutput=journal** [2]: journald and journalctl are broken for user services. Logging to journald only works with persistent journal storage [3][4], but Facebook uses volatile journal storage.
* **StandardOutput=syslog** [5]: rsyslog seems designed for system administrators, not users. I didn't investigate much, but I suspect it's impossible to make rsyslog write to a user-controlled path such as `~/local/.eden/logs/edenfs.log`.
* **LogsDirectory=%f/logs and StandardOutput=file:%L/edenfs.log** [6][7]: LogsDirectory= does exactly what we need, except it only supports paths relative to `/var/log` or `~/.config/log/`. `LogsDirectory=%f/logs` does not work, and systemd will ignore such a directive.
* **StandardOutput=file:%f/logs/edenfs.log and a `mkdir` service**: If we create a service which just creates the `logs` directory, and make fb-edenfs@.service depend upon that service, systemd can successfully open the log file [8]. In theory, using StandardOutput= would cause errors like "could not set resource limits" to be logged to `edenfs.log`. In practice, systemd does not respect the service's logging configuration when reporting such errors [9]. Therefore, this solution is no better than the manual redirect.
[1] https://www.freedesktop.org/software/systemd/man/systemd.exec.html#StandardOutput=
[2] https://www.freedesktop.org/software/systemd/man/systemd-journald.service.html#
[3] https://www.freedesktop.org/software/systemd/man/journald.conf.html#SplitMode=
[4] https://lists.freedesktop.org/archives/systemd-devel/2016-October/037554.html
[5] https://www.rsyslog.com/
[6] https://www.freedesktop.org/software/systemd/man/systemd.exec.html#RuntimeDirectory=
[7] https://www.freedesktop.org/software/systemd/man/systemd.unit.html#Specifiers
[8]
```name=fb-edenfs-logs@.service
[Service]
Environment=EDENFS_CONFIG_DIR=%f
ExecStart=/bin/sh -c ' \
set -e; \
set -u; \
\
/bin/mkdir -p -- "$${EDENFS_CONFIG_DIR}/logs""; \
'
```
```name=fb-edenfs@.service
[Unit]
After=fb-edenfs-logs@%i.service
Requires=fb-edenfs-logs@%i.service
```
[9] fd0ec39d38/src/basic/log.c (L560-L639)
Reviewed By: simpkins
Differential Revision: D13422459
fbshipit-source-id: 57c575a6f377812caa2a79168778576c6ccff33e
Summary:
ported forward from D4209167, add a couple of helpers
to access these fields on mac and linux, centralizing/minimizing ifdefs.
Simplify some of the logic in FileChangeMonitor.
Reviewed By: chadaustin
Differential Revision: D13475717
fbshipit-source-id: d7b39999808bc41a6dc17a87189501cb34e68b30
Summary:
Thankfully, we can simply remove it; it is really just a performance
optimization that we can enable for linux.
Reviewed By: simpkins
Differential Revision: D13475719
fbshipit-source-id: 09b60dcf995c2c5390b91aa316c62ca1b4d3f944
Summary:
It is only 16 bits wide on this system, so we don't need
to borrow bits and there may not be enough bits to borrow from
anyway.
Reviewed By: simpkins
Differential Revision: D13475714
fbshipit-source-id: 1d342c89a3037abd744d97fef21ad421b5e60356
Summary: only include and use it on linux
Reviewed By: simpkins
Differential Revision: D13475715
fbshipit-source-id: 6b0b9da1b32088e01cbb932f9b3ed62532dfe00f
Summary: only add the selinux deps if we found selinux
Reviewed By: simpkins
Differential Revision: D13475711
fbshipit-source-id: c3375282b61881317f9a6c4c8e321ce717d1f9ab
Summary: overlooked because there is no CI exercising this today
Reviewed By: simpkins
Differential Revision: D13475721
fbshipit-source-id: 3e8fe280ab73d249da374129b37d32cd7e17f472
Summary:
on macOS the LHS is `unsigned long` and the RHS is `unsigned long long`.
Introduce a cast for consistency.
Reviewed By: chadaustin
Differential Revision: D13470036
fbshipit-source-id: f726507013e4ed9f123ced474299bb2d6818732f
Summary: Even in C++17 mode, this isn't available, so work without it
Reviewed By: chadaustin
Differential Revision: D13470034
fbshipit-source-id: c917cc011aaabc2cfcf79e801bb6870482302fd8
Summary: this is required for the build on macos
Reviewed By: chadaustin
Differential Revision: D13470035
fbshipit-source-id: 066cb5b2ea86ffddb9c8cf6f7ae50da90b62a5bc
Summary:
I want to use fake_edenfs to test logging via EdenFS' systemd service. Make fake_edenfs and the real edenfs use similar logic to determine the log file path.
This diff should not change behavior for the real edenfs.
Reviewed By: simpkins
Differential Revision: D13424470
fbshipit-source-id: d0c2e035fdb5884dbd2d9704c7e0244d35e052f2
Summary:
Eden no longer tracks any state in file handles, and has no plans to in the future.
Therefore, remove all related code.
Reviewed By: strager
Differential Revision: D13354307
fbshipit-source-id: 341d081f64c6c8fb2b4b1b5a5ff42f2cc7d38039
Summary:
Now that all file access in Eden is stateless, we no longer need to handle open() or release().
If the kernel advertises FUSE_NO_OPEN_SUPPORT, return ENOSYS from open().
Reviewed By: simpkins
Differential Revision: D13325759
fbshipit-source-id: 38486848f27ffeb005f74407888e94d891496f98
Summary:
D13325746 changed `EdenDispatcher::open()` to no longer return a file handle.
However the code in `FuseChannel::fuseOpen()` throws an exception if the
dispatcher does not create a file handle. This breaks most file operations in
Eden.
The Dispatcher check is removed in D13354307, but that hasn't landed yet.
That change probably just needed to be part of D13325746.
Reviewed By: chadaustin
Differential Revision: D13445570
fbshipit-source-id: 70d639142057740766bcbe02a0df50b14f7c9937
Summary:
Remove some more of the double-checking code left over from debugging
T24406969. We fixed the root cause of that task and haven't seen any errors
reported by this code. The higher log levels used for reporting information
about empty files just causes extra noise in the logs.
Reviewed By: strager
Differential Revision: D13403531
fbshipit-source-id: bcb7b29a1d687ffcefd6ecf586130b96f87b4820
Summary:
Previously, a file handle must have been held for the entirety of a write operation. That is no
longer true. Stop looking up file handles on write.
Reviewed By: strager
Differential Revision: D13325662
fbshipit-source-id: 9ae31b467d17d633c388917d18098e6e5a620b89
Summary:
Now that FileInode read and write operations are stateless via BlobAccess and OverlayFileAccess,
EdenFileHandle no longer provides any value. Remove it. This also fixes eden's shutdown timeout
when a file handle is open and paves the way for FUSE_NO_OPEN_SUPPORT.
Reviewed By: strager
Differential Revision: D13325137
fbshipit-source-id: 71ed47a7c997f5035b4394ccb311f94332ecd8c2
Summary:
Have FileInode use OverlayFileAccess instead of using the Overlay directly.
This allows IO on materialized files to be stateless and pave the way for
eliminating EdenFileHandle. It also paves the way for performance improvements
such as nicer SHA-1 caching.
Reviewed By: strager
Differential Revision: D13325079
fbshipit-source-id: fb27d48b5dc9196dc6e36557596f601194a56aa9
Summary:
Move the overlay IO logic out of FileInode into a centralized OverlayFileAccess class. It keeps the last 100 overlay file handles open.
FileInode's changes are coming in D13325079.
Reviewed By: strager
Differential Revision: D13324995
fbshipit-source-id: 04fb3fe50114b0f19b78acd17a9684c92f8e8072
Summary:
Add a fuzz test that randomly modifies a directory during a series of
readdir calls and verifies each unmodified entry is returned once.
Reviewed By: simpkins
Differential Revision: D13371162
fbshipit-source-id: 252b03ab0288e82b56a33c347955de129e61ae42
Summary:
If a blob was partially read, evicted from cache, and then read again,
the readByteRanges coverage set was not being cleared. Always clear it
in startLoadingData.
Reviewed By: strager
Differential Revision: D13405267
fbshipit-source-id: 6f60e6e80662fd470fe4ddbc833fc8efd8850686
Summary:
There was a bug in BlobCache where, if you had an interest handle to a
blob, but that blob was evicted anyway and then something else caused
it to be reloaded, dropping your interest handle would cause the blob
to be incorrectly evicted since the reference counts were no longer
compatible. Add a version to cache items and only decrement the
reference count on an item if the interest handle and item agree.
Reviewed By: strager
Differential Revision: D13405144
fbshipit-source-id: aee052bf777e7225551c3ae2b8b69a99f4f77691
Summary:
When you run 'eden start' without systemd integration, edenfs writes startup logs to the terminal to let users know that stuff is happening:
```
$ eden start
Starting edenfs (dev build), pid 2792025
Opening local RocksDB store...
Opened RocksDB store in 0.95 seconds.
Remounting 1 mount points...
Successfully remounted /data/users/strager/fbsource-dev
Started edenfs (pid 2792025)
Logs available at /data/users/strager/.eden-dev/logs/edenfs.log
```
These startup logs are also used by various tests (especially 'eden restart's tests).
Make the same thing happen when running 'eden start' with systemd integration, improving the user experience and making some tests work:
```
$ EDEN_EXPERIMENTAL_SYSTEMD=1 \
./buck-out/gen/eden/cli/eden.par start \
--daemon-binary "${PWD}/buck-out/gen/eden/fs/service/edenfs"
Starting edenfs (dev build), pid 2800760
Opening local RocksDB store...
Opened RocksDB store in 0.693 seconds.
Remounting 1 mount points...
Successfully remounted /data/users/strager/fbsource-dev
Started edenfs (pid 2800760)
```
Reviewed By: wez
Differential Revision: D13241979
fbshipit-source-id: de79b714e42b690fdab7c21d9add46bc2da35328
Summary: Several tests for 'eden start', 'eden stop', and 'eden status' need to pass command-line arguments to fake_edenfs. With systemd support enabled, make 'eden start' forward daemon arguments to fake_edenfs, making these tests pass.
Reviewed By: wez
Differential Revision: D13249891
fbshipit-source-id: 9008a361fce7a5629535cc9d245b86073ee70826
Summary:
getSHA1 is only handling std::system_error. If another kind of exception is thrown, it's never caught and edenfs crashes.
Calling EdenMount::getInode with a path such as "./hello" will cause a std::domain_error to be thrown. Since std::domain_error is not derived from std::system_error, `getSHA1ForPathDefensively(["./hello"])` crashes edenfs.
Fix the crash by forwarding all exceptions over Thrift, not just std::system_error-s.
Reviewed By: simpkins
Differential Revision: D13386450
fbshipit-source-id: 06262dad30a5508ed482c9e8979b61aa9643280a
Summary:
As I'm working on ripping out file handle support, I wanted to ensure
that we had a path towards a correct and stateless readdir()
implementation. Stateful file handles require extra care during
graceful restart, and it's nice that we can avoid them. In fact, this
work paves the way towards a possible FUSE_NO_OPENDIR_SUPPORT feature.
This diff fixes concurrent rm -rf in the same directory.
Reviewed By: simpkins
Differential Revision: D13370898
fbshipit-source-id: eba650e673a7cb9559e04ba28417980f6d0c65cb
Summary:
Have `eden stats` print the size of the blob cache if the running
edenfs has information about it.
Reviewed By: strager
Differential Revision: D13349220
fbshipit-source-id: 9f59f4399f2d4283aa80bcb54ba73c51d555d502
Summary:
edenfs's privhelper process needs the CAP_SYS_ADMIN capability [1] in order to manage mount points. Give it this capability by invoking edenfs as the `root` user using `sudo`.
Known issues:
* `sudo` must be passwordless in order for fb-edenfs@.service to start.
* systemd can't kill all of fb-edenfs@.service's processes, so `systemctl stop` is unreliable for example.
[1] https://manpage.me/index.cgi?q=capabilities&apropos=0&sektion=0&manpath=CentOS+7.1&arch=default&format=html
Reviewed By: chadaustin
Differential Revision: D13113450
fbshipit-source-id: 01b89521cab371b5017fab6fbd38d55eea599c46
Summary:
I want to add some non-trivial systemd-specific logic to fb-edenfs@.service's ExecStart command. Refactor the existing command to use sh, giving us access to `if` and other useful utilities.
Also, the shell language is likely more familiar (e.g. `${foo}` and `$foo` behave the same in sh, but behave differently in systemd), so using sh should lead to fewer headaches.
This diff should not change behavior.
Reviewed By: chadaustin
Differential Revision: D13165377
fbshipit-source-id: 2e500abf75c6f79a8b174a848f7eeb1cfaebb96c
Summary:
Add the plumbing necessary to make 'eden start' start a systemd user service. This is only enabled if you opt in using EDEN_EXPERIMENTAL_SYSTEMD.
Currently, only fake_edenfs works. The real edenfs doesn't work yet because it needs root access to configure mount points.
'eden restart', 'eden stop', etc. are not affected by this diff (and are probably broken with EDEN_EXPERIMENTAL_SYSTEMD enabled).
Reviewed By: simpkins
Differential Revision: D10849390
fbshipit-source-id: c087a6498951ff100e5c80bd07ad869b2709e1b3
Summary:
Drop interest in cached blobs at various points in the FileInode
lifecycle.
Reviewed By: strager
Differential Revision: D12991762
fbshipit-source-id: 19fd94938c96485160d547ecbd259ffeb39b2341
Summary:
Frequently, non-eden-owner-owned processes show up as `err:13` in
the `eden top` output which is a bit of a PITA. This diff pulls the data
from `/proc/PID/cmdline` instead of `/proc/PID/exe`; the former is world
readable always while the latter is restricted to process owner readable.
Reviewed By: chadaustin
Differential Revision: D13342789
fbshipit-source-id: d4e395b318107e873189a1e2039329015c4c38f8
Summary: Add a CoverageSet type that tracks non-overlapping intervals so FileInode will be able to tell when the entire blob has been read by FUSE.
Reviewed By: strager
Differential Revision: D12947146
fbshipit-source-id: fbcfd9c19b09d4a7b5364671dcdbd39b53e6d186
Summary:
Write tests for readdir's semantics (we really do want to return . and
..) and simplify the logic. It's still quadratic in large directories,
but there aren't any allocations anymore.
Reviewed By: strager
Differential Revision: D13287764
fbshipit-source-id: 5e0d4b86eb16dbd7a16cdeb324e4b43363512e25
Summary: This diff also includes build script changes to EdenDispatcher, FsChannel and EdenMount.
Reviewed By: strager
Differential Revision: D13091786
fbshipit-source-id: cecc8d849fcb9ebc8fa718e1011ef8931bebc279
Summary:
Stop holding a reference count to the TreeInode while a directory
handle is open. This allows eden to shut down while a directory handle
is open.
Reviewed By: strager
Differential Revision: D13287701
fbshipit-source-id: a24f32a1ac40b6c19bc5864aa5f5785f3016361b
Summary:
Send readdir requests to TreeInode. This may not sound like a good
idea: the FUSE documentation suggests that stateful directory handles
are required to implement correct readdir semantics under concurrent
deletes and renames. However, the 63-bit offset value is treated as a
cookie that is passed from one readdir call into the next, and 63 bits
should be sufficient to implement readdir concurrent with
rename/unlink. So move readdir's implementation into TreeInode in
preparation for the complete removal of TreeInodeDirHandle.
Reviewed By: strager
Differential Revision: D13287664
fbshipit-source-id: c0d615675edd9b83353534468a69b89068bba923
Summary:
If a file was partially truncated, it would not always be marked as
materialized. During materialization, the SHA-1 would be cached,
but not invalidated after the truncation.
Write tests that ensure that both ftruncate and O_TRUNC mark files as
modified.
Reviewed By: simpkins
Differential Revision: D13329102
fbshipit-source-id: f09fdc5f11f1da25e1b4453de1b29d1390b3dc71
Summary:
Send fsyncdir straight through the inode rather than going through
DirHandle. This is the better design anyway, since the DirHandle does
not receive directory-mutating requests like mkdir.
Reviewed By: strager
Differential Revision: D13287610
fbshipit-source-id: 154fa32a3877c89a204a2d10b4e2b637410d9486
Summary: No need for String to import fbvector.
Reviewed By: yfeldblum
Differential Revision: D13007715
fbshipit-source-id: c61639a04273f14dd3daf230ee6ed9ade93d058e
Summary:
FUSE_NO_OPEN_SUPPORT is better than ATOMIC_O_TRUNC for Eden's use
case. Remove the code that pretended we might support ATOMIC_O_TRUNC
again someday.
(Note: this ignores all push blocking failures!)
Reviewed By: strager
Differential Revision: D13163382
fbshipit-source-id: 948d701571a8d2977da3d2532fdc9538c5011636
Summary:
It's not clear that this code is a win and either way it will be a
no-op when FUSE_NO_OPEN_SUPPORT is enabled so just remove the prefetch
in open().
(Note: this ignores all push blocking failures!)
Reviewed By: strager
Differential Revision: D13162205
fbshipit-source-id: a3161c0d042e13bd092fc9589e851be78552fa7a
Summary:
FileInode no longer has a strong reference to a blob. Instead, all accesses go through the blob cache. This diff changes the caching behavior for blobs.
The previous behavior was:
When a file's contents are needed in any way, the blob is loaded and referenced by the inode. When the number of open file handles goes to zero, the blob is dropped. The blob is also dropped when the inode is unloaded. Future inode loads or open requests, in that situation, require the blob to be reloaded from the LocalStore.
The new behavior is:
When a file's contents are needed, the blob is loaded and stored into the BlobCache, evicting any if necessary. Future blob requests are satisfied from the BlobCache, pushing it to the back of the eviction queue. When the inode is materialized or unloaded, the blob will be evicted from cache if no other blob has interest in it.
(Note: this ignores all push blocking failures!)
Reviewed By: strager
Differential Revision: D12813912
fbshipit-source-id: 20d20807d2e4a6c37cddab38562fdb7456316aac
Summary:
A later diff needed a constant for the SHA-1 of an empty buffer. While
I'm at it, I made Hash a little bit nicer to use.
Reviewed By: strager
Differential Revision: D13224195
fbshipit-source-id: b2fb1437be042215b5b398a8c7fc9fc5dd115e9e
Summary:
Now that the Overlay no longer serializes timestamps, remove all of
the special-case migration logic.
Reviewed By: strager
Differential Revision: D13144764
fbshipit-source-id: 713a4bfcde9003a8d5a28837cb530b05a9017c22
Summary:
The linter tripped on D12813838 even though the warning was unrelated
to my changes.
Reviewed By: simpkins
Differential Revision: D13167184
fbshipit-source-id: 555691f00d113ed2bff9488b61392cc92f4395e3
Summary:
Eden has used the InodeMetadataTable as the authoritative source for
timestamp metadata for over six months. I think we can safely assume
that anyone at this point who has old inodes in the overlay that don't
have corresponding entries in the inode metadata table are fine with
those timestamps being reset to the last checkout time.
Reviewed By: strager
Differential Revision: D13144735
fbshipit-source-id: 06a9a8835ea83c98fb6a165e4c8d5c3c6b28ad84
Summary:
Eden has used the InodeMetadataTable as the primary source of
timestamp data for more than six months. Stop writing timestamps into
the overlay, since they will never be used.
Reviewed By: strager
Differential Revision: D13144696
fbshipit-source-id: e36423036228e89dd2a986e6bacfa74553c17a92
Summary: Instantiate a blob cache in the EdenServer with configurable settings.
Reviewed By: strager
Differential Revision: D12813880
fbshipit-source-id: 8ccc89826f04aca78964230374dea48abf05e05e
Summary: In support of making Eden's file access stateless, add a facade that connects the BlobCache and ObjectStore, allowing FileInode to fetch blobs, minimizing reloads and memory usage.
Reviewed By: strager
Differential Revision: D10850143
fbshipit-source-id: 4307f7c1143aecad1284ea3cadf3e4efca9a3925
Summary: Change the Windows pipe read and write functions to read/write in a loop. Plus changed the functions prototype to match the POSIX version.
Reviewed By: strager
Differential Revision: D13091785
fbshipit-source-id: 375b22bd9e62d371a78d410f42068945b966a743
Summary:
Backing store works with eden strings(UTF8 + Unix path separator). The path strings we receive on Windows from FS and cli are Windows paths
(Win path separator and/or UTF16). Adding the functions to convert one to another.
Reviewed By: strager
Differential Revision: D13091788
fbshipit-source-id: f7fc8a79e360e964cf4619dfa540b57f1f18d283
Summary:
The new blob cache wants to know, given a request, whether the blob is
expected to be needed or not. The answer, in general, is yes if the
request came from Thrift and no if it came from FUSE, because the kernel
will cache the result of the request in its own page and dentry caches.
Propagate this information through FileInode.
Reviewed By: strager
Differential Revision: D12813838
fbshipit-source-id: 7a359686149cd4daff41630c94085b680c448c4f
Summary: Add a BlobCache with a maximum cache size and a minimum entry count and interest-based eviction.
Reviewed By: strager
Differential Revision: D12972062
fbshipit-source-id: 1958f7f500c051a5bc0b39b5b89a6f0fc1774b0f
Summary: Create a platform independent function to compare the stats and use that to check if the file contents have changed.
Reviewed By: strager
Differential Revision: D13046269
fbshipit-source-id: c4f5bc88cec3df5cb6555d13cea2f23fd4eeb7ce
Summary: Add type annotations to all functions in eden/fs/service/client.py
Reviewed By: strager
Differential Revision: D13051091
fbshipit-source-id: 23b93008352664336ad155a7f5cc281bd5529702
Summary:
Now that we have a standardized StartingGate for benchmarks, use it
everywhere we used to use folly::Baton.
(Note: this ignores all push blocking failures!)
Differential Revision: D13012244
fbshipit-source-id: 5841ab74cfa408e87d021fe5591557e79e677e5c
Summary:
As we start to build out both FUSE and Thrift benchmarks, we'll want a
standard library. Introduce a benchharness and have both the thrift
sha-1 and parallel_open_close benchmarks use it.
(Note: this ignores all push blocking failures!)
Differential Revision: D12969306
fbshipit-source-id: 89c8bbcc37d53560decffb9281af4aba20345787
Summary: Update the CLI code to make the pyre type checker happy.
Reviewed By: wez
Differential Revision: D13035413
fbshipit-source-id: d201f2e65667e0ce1bf4a73fbb05878e8711ad16
Summary:
chadaustin noticed this as part of fixing up the ESTALE
handling. The issue is that we were using `inodeMap->lookupInode` and
assuming that it will always return one of our magical inodes. This
isn't guaranteed so it is better to resolve the inode by name from
the root, so that's what this diff does.
Reviewed By: chadaustin
Differential Revision: D12970034
fbshipit-source-id: 8207660cbc71577b276cb092d1ef19e1076b4946
Summary:
yfeldblum were talking about whether this code might make sense in
folly. That led to polishing it a bit more. The hot path is only six
instructions now. It's not any faster in a tight generateUniqueID loop
but uses only one thread_local slot and a bit less global storage.
Reviewed By: strager
Differential Revision: D12927275
fbshipit-source-id: 94a5872c61dfe9dd441f1f34fab65f93c12997d8
Summary:
In a later diff, I needed generateUniqueID to be
noexcept. folly::ThreadLocal does not guarantee that (and it allocates
the first time a thread calls get()), so use C++ thread_local
instead. Bonus: it's about half a nanosecond faster now.
Reviewed By: strager
Differential Revision: D12914625
fbshipit-source-id: 9ddbe65d0ba1d317907f821c03dea5a207a73a68
Summary:
Update the HgUI object used by hg_import_helper to always return true from the
`plain()` function, regardless of whether HGPLAIN is set in the environment or
not.
This should help ensure that this script is never affected by user-defined
configuration settings. This should also help ensure that mercurial won't try
to print progress bars or other strange things.
Reviewed By: quark-zju
Differential Revision: D13008639
fbshipit-source-id: afe581958470c4c4b89825a259c460ece4e20fe7
Summary:
It is desirable to be able to reference the same variable
multiple times in the RHS of a config setting. This diff makes that
possible.
Reviewed By: strager
Differential Revision: D12906500
fbshipit-source-id: 4277f12105d0a0fb3dca880d3dad6b0238acedc0
Summary:
With the mononoke service being configured with proxygen
in front of the servers we need to ensure that we're correctly
setting up our http request if we want them not to fail with a
400 bad request error.
This diff allows setting mononoke.hostname to a DNS resolvable
name rather than an IP address and passing that hostname through
so that we can set it in the HTTP request.
Reviewed By: strager
Differential Revision: D12906498
fbshipit-source-id: b5aaabfd6f2f4c48d45128eaad8406e648477f75
Summary:
Update all of the C++ unit tests that create temporary files and directories
to use the new `facebook::eden::makeTempFile()` and
`facebook::eden::makeTempDir()` functions.
Note that this likely changes the behavior of some code paths in meaningful
ways: `/dev/shm` normally does not support `getxattr()`, and Eden's overlay
code attempts to store the SHA-1 for materialized files as using extended
attributes. This means that the tests will now typically hit the fallback
code path and will not store SHA-1 data in the overlay.
Reviewed By: chadaustin, strager
Differential Revision: D12971162
fbshipit-source-id: 6cc5eba2e04be7e9a13a30e90883feadfb78f9ce
Summary:
Add new helper files for creating temporary files and directories.
The main advantage of these functions is that they prefer creating files in
`/dev/shm` by default instead of `/tmp`. Some of the eden unit tests are
fairly I/O intensive (particularly the checkout tests, which create and
destroy many test mounts). This can be quite slow on hosts where `/tmp` is a
a spinning hard disk. Putting the temporary files in a ramdisk greatly speeds
up the tests in this case.
These test functions also default to using the prefix `eden_test.` for the
temporary file names.
This does not yet change any of the test code to use these functions. I will
do that in a subsequent diff.
Reviewed By: chadaustin, strager
Differential Revision: D12971161
fbshipit-source-id: 3f74be7a467e8080185d4d97d114288b4528755a
Summary:
This function behaves similarly to the python `os.path.expanduser`
function, except that it is restricted to expanding only information for the
current user.
Use this new function to process the hgcache path returned via the
rust config parsing code.
In the longer term I want to centralize and add accessors for
the rust config and move this stuff into EdenConfig, but for
now this is the one place that does this and it is OK to
process it this way here.
Reviewed By: strager
Differential Revision: D12988902
fbshipit-source-id: 96b10640359c3b8c0adac1e14cd42dd592023c3d
Summary:
This makes it possible to change configuration options
for the LocalStore while the server is running.
As you'll see in the next diff, our current layering makes using
the config a bit more awkward, but at least this diff doesn't
look gross :-p
This diff doesn't introduce any new functionality or configuration.
Reviewed By: strager
Differential Revision: D12949577
fbshipit-source-id: cf897ba676b9359f92865170faa42ff17329b85f
Summary:
Sometimes users accidentally run `edenfs start` or `edenfs restart` instead of
`eden start` or `eden restart`. This adds a new `--edenfs` flag to the
`edenfs` binary, and asks users if they meant to run `eden` instead if they do
not pass in this flag.
This used to be less of a problem since `edenfs` required users to also
explicitly specify several other configuration flags (like `--edenDir`).
However `edenfs` can not automatically figure out these settings, so these
flags are no longer required. Therefore `edenfs` would still try to start
normally when invoked with `edenfs restart`, since it did not require these
flags and it did not complain about unhandled command line arguments.
Reviewed By: wez
Differential Revision: D12927803
fbshipit-source-id: dbf7ce2449c391ca218652439eb68ff43c2ebd46
Summary: folly/Subprocess is not compatible with Windows and has broken the Windows build. It's not used so removing it.
Reviewed By: wez
Differential Revision: D12967451
fbshipit-source-id: 54d33bf6fe2ec3ede9d68eccd99e53c5eb6ed53d
Summary:
In a later diff, I needed to make generateUniqueID() noexcept. This
made me think that C++'s thread_local might be a little faster than
folly::ThreadLocal. So I added this benchmark.
Reviewed By: yfeldblum
Differential Revision: D12914249
fbshipit-source-id: 2c4acfff2162f66d13f456439d91df2ecb4167e3
Summary:
Thanks to some bpf tracing by strager, we traced the ESTALE response to
`d_splice_alias` and noted this comment above the implementation in the kernel:
> If a non-IS_ROOT directory is found, the filesystem is corrupt, and
> we should error out: directories can't have multiple aliases.
Well, our magic `.eden` directory is a directory with aliases and we were
seeing the error trigger on that dir. So, this diff replaces hardlinking
directories into each tree with a hardlink to a symlink in each tree!
At mount time we create `.eden/this-dir` as a symlink to `/abs/path/to/mount/.eden`
so that `readlink("/abs/path/to/mount/sub/dir/.eden/socket")` still
resolves as it did prior to this diff.
Reviewed By: strager
Differential Revision: D12954819
fbshipit-source-id: 7f3b1b53f2bd5b9c51e64055fc34110657a19110
Summary:
Sandcastle has several cases where we chown the entire
repository which performs terribly on Eden. As a workaround we have a
command to do this in eden without loading all the files.
Reviewed By: chadaustin
Differential Revision: D12857956
fbshipit-source-id: 36cebcc710fbcf4e1eb265df901513cf50a227b9
Summary:
I've included a missing `.via` for the mononoke tree import path. It's hard to
measure the difference here, but it is more technically correct than letting
that second stage run in an event base thread. desired.
Reviewed By: chadaustin
Differential Revision: D12896023
fbshipit-source-id: 0af0f7f94c3274a3e4b570fed0af4d4d3c783bc3
Summary:
FileInode::prefetch was entirely redundant - it queried for metadata
upon inode lookup after getattr() was called already (which requires
the blob metadata to be loaded).
Reviewed By: wez
Differential Revision: D12896473
fbshipit-source-id: 9ba5104a43860e1f22b88726b9e3e977d0b50e89
Summary:
Eden used to load materialized entries at startup. It no longer does
and this code is dead.
Reviewed By: wez
Differential Revision: D12896210
fbshipit-source-id: 398363724a661f87208cf05313e61755a451edb7
Summary:
With this the eden cli can dump tracepoints and translate
them to various formats or perform any processing
Reviewed By: chadaustin
Differential Revision: D10384072
fbshipit-source-id: 8b38e7f6b551a2bd98b3e748ba1cceafeceeec8c
Summary:
This uses the tracing infrastructure to create blocks around
the getSHA1 invocation as well as the getOrLoadChild (called for each
element of the path-walk) method.
Reviewed By: chadaustin
Differential Revision: D10384074
fbshipit-source-id: a06fbe38e8d3f35fcb248e6bd724e5572724d27d
Summary:
In order to get better insights into where time is spent on
various requests, some tracing infrastructure is helpful. This will
record tracepoints into a per-thread ring-buffer which can then be
dumped out-of-band.
The trace format is quite simple, a trace is composed of blocks with
begin and end points and each block may have a parent block. We can
then translate this format to many other formats as we desire.
Reviewed By: chadaustin, strager
Differential Revision: D10384071
fbshipit-source-id: f9e0f11521c68b96ab40d517c7a83cf89375b101
Summary:
This diff implements getBlob on top of the mercurial rust
datapack code. It adds a C++ binding on top of the rust code to
make it easier to use and hooks it up in the hg backing store.
Need to figure this out for our opensource and windows builds:
* Need to teach them how to build and link the rust code
* need to add a windows version of the methods that accept paths;
this is just a matter of adding a WCHAR version of the functions.
Reviewed By: strager
Differential Revision: D10433450
fbshipit-source-id: 45ce34fb9c383ea6018a0ca858581e0fe11ef3b5
Summary:
Nothing populates these fields yet today, but will in a later diff.
If present in the json data for an entry, populate them in our TreeEntry.
When processing trees from mononoke, check for these fields being set
and emit them into the current writeBatch so that we set the BlobMetadata.
Reviewed By: chadaustin
Differential Revision: D12814793
fbshipit-source-id: cd19d3f553c22462adc58c024a90cfeb5b8da224
Summary:
We'll start populating these in the mononoke client.
They are currently unused.
Reviewed By: chadaustin
Differential Revision: D12814791
fbshipit-source-id: f5407de5cdb9f1f3ad6ee2befed50e2a7562ec97
Summary:
I want to reuse this outside the LocalStore implementation
in a later diff. This has no functional changes.
Reviewed By: chadaustin
Differential Revision: D12814792
fbshipit-source-id: 2cc34b449a93afb2e71bd49ed6587009ebf3e419
Summary:
While this doesn't deliver any gains immediately, it is desirable
because the shortest path to getting trees with pre-populated size and
content sha1 information is via the mononoke api server fetch.
If we use the local trees we will populate the basic metadata slightly
faster but will then pay more to fetch and hash the data.
The calculus is a little more complex really, as having the contents of
a dir locally in a local datapack is probably faster, but we don't have
a good way to know if that is the case right now.
Reviewed By: strager
Differential Revision: D12813490
fbshipit-source-id: 545b90bf20940dab193bef8120712dd9227033e1
Summary:
This partially wires up handling mononoke related config
changes if they are made after starting the server. We don't currently
have logic to enable mononoke after starting the server, but can disable/enable
it if it was configured at process start.
Reviewed By: strager
Differential Revision: D12813346
fbshipit-source-id: ee29e91fe714a65a3b61c4e23929b67d6b7cb521
Summary:
To facilitate testing against a test instance rather than selecting
from a tier, add a config option to return the host override.
The semantics are that a hostname string will override the tier name setting.
An empty host name string is treated as missing and will not replace the
tier name.
Reviewed By: chadaustin
Differential Revision: D12813050
fbshipit-source-id: 67773d772d78d3f7882825ce54f4313c97ab0a33
Summary:
Remove the handful of (but growing) parameters that
are threaded down to the HgBackingStore in favor of the ReloadableConfig
object and use that to configure mononoke support.
This doesn't notice changes to the config on the fly today, but
does simplify the amount of stuff we need to pass around.
Reviewed By: chadaustin
Differential Revision: D12813053
fbshipit-source-id: b8309328fab4980889624257cfd387c8c1335001
Summary:
This allows detecting changes to the edenrc configuration
and adjusting how we read data accordingly.
To avoid a huge gnarly diff with a lot of dependencies, this diff
introduces a little interface class that defines the method to obtain
a possibly reloaded EdenConfig instance.
This diff shouldn't change any functionality.
Reviewed By: chadaustin
Differential Revision: D12813051
fbshipit-source-id: 35390dc28671ba46a03d098c4f280e2f567dbdc9
Summary: This option was added upstream and to our internal 4.16 kernel series
Reviewed By: chadaustin
Differential Revision: D12823476
fbshipit-source-id: 54d241b77eff92f4083c0f1d4c98b47a6a098e3f
Summary:
Checkout.modifyFile is the long pole when running the Eden unit
tests. Splitting it into three runs it across multiple threads. It
could be split further if needed.
Reviewed By: strager
Differential Revision: D12813684
fbshipit-source-id: 49e847d46bc54f16fcb2b273ec81e92eaf756dbb
Summary: If an error happens when opening the log file, edenfs should stop immediately and report the error. Write a test to verify that this happens with DaemonStartupLogger.
Reviewed By: chadaustin
Differential Revision: D10436926
fbshipit-source-id: 3f33ed276a668b7ecda8176a55b114b5878304e3
Summary:
When running edenfs in daemonize mode (i.e. when --foreground is not specified), the daemon process inherits stdin from the parent and never closes it. This has two consequences:
* The daemon process holds a reference to stdin even after the parent exits. This causes tests to hang if they wait for EPIPE on stdin after waiting for the parent process to exit. (I discovered this issue when adding tests for 'eden start' in D10434379.)
* edenfs could potentially read from the terminal after 'eden start' has returned. I think this behavior is surprising and could potentially cause problems.
Promptly close stdin after forking the daemon process, since nothing inside the daemon needs stdin.
Reviewed By: simpkins
Differential Revision: D10442346
fbshipit-source-id: f3d287e8b48e121807f845b73a1a550e63efae25
Summary:
getLogPath in eden/fs/service/main.cpp ensures that a non-empty log path is given to DaemonStartupLogger::daemonize. An empty log path is not possible. DaemonStartupLogger behaves differently with an empty log path, but that behavior has caused problems (see D10439299).
Make DaemonStartupLogger require a (non-empty) log path, simplifying its implementation.
This diff should not change behavior.
Reviewed By: chadaustin
Differential Revision: D10227597
fbshipit-source-id: 37931528add51ab3e6e1b941478000297cb8b693
Summary:
There may be a metadata header in the contents and we currently
do not have any code to process this, so we want to disable this.
Add a constructor boolean to disable it and default to off while we
work up a proper solution to this.
Reviewed By: chadaustin
Differential Revision: D10523393
fbshipit-source-id: ec96b63b6897841ee5a6b848834b5a0ada145223
Summary:
The daemonizeIfRequested is implemented in the main edenfs process as well as in fake_edenfs. Consolidate the almost-identical implementations.
This diff should not change behavior.
Reviewed By: chadaustin
Differential Revision: D10439300
fbshipit-source-id: 6253668896de0f8393b92f4aa5ee8a8a2fcbfb31
Summary:
D9029272 (github commit fae2056037) accidentally broke the CMake-based
build of Eden when SELinux is available: it defined `PRIVHELPER_LIBS` but
later used `PRIVHELPER_LIBRARIES` (LIBS vs LIBRARIES).
This fixes that error by simply removing this intermediate variable. We
can just use SELINUX_INCLUDE_DIR and SELINUX_LIBRARIES directly, as they
will simply be empty if SELinux is not available.
Reviewed By: quark-zju
Differential Revision: D10503905
fbshipit-source-id: 1c12bb1cad0351e4e0a77d0c7e8a83086209efee
Summary:
Our CMake-based build had gotten somewhat out of date. This fixes several
issues that were preventing it from building:
- wangle now depends on fizz, which in turn depends on libsodium
- fbthrift now depends on yarpl for the streaming functionality
- Eden now depends on libsqlite3
- The eden/fs/inode library now depends on eden/fs/config
Reviewed By: strager
Differential Revision: D10436531
fbshipit-source-id: f41065da2edb909e8eb94e03d0e62ae19ad3a84f
Summary:
Add a very cheap unique ID allocator designed for extremely
low-latency uses like D10384071.
Differential Revision: D10501712
fbshipit-source-id: 15a0b7a2d344c0f6100082850dd967c585f3e2da
Summary:
Because Mercurial blob IDs change without the contents changing, and
because files get unloaded upon checkout, rebasing across a large
distance in history can result in status fetching a lot of
metadata. Keep a smallish LRU cache for SHA-1 and size by blob ID.
Reviewed By: strager
Differential Revision: D10419965
fbshipit-source-id: 81499573814775471913db05f924767c3bab300e
Summary:
Now that the import bug has been fixed for some time, it's likely few
people have cached empty files. And if they do, `eden gc; eden
restart` is a fine workaround.
On the other hand, reimporting empty files gums up the importer, and
I've seen several people recently complaining about performance that
could be partially attributed to the fact that their Eden needed to
verify empty files. simpkins has talked about adding a bit to the
cached blob to determine whether it needs reimporting or not, but it's
probably going to take a while for anyone to implement that, and this
reverification logic is hurting people today.
Reviewed By: strager
Differential Revision: D10456519
fbshipit-source-id: 657bc377ee16ce93494075bde4388aed59dceecf
Summary:
Noticed that the filenames in the logging in the error
case were sometimes garbled; capture a copy of the filename
rather than a reference to it.
Reviewed By: bolinfest
Differential Revision: D10473798
fbshipit-source-id: ee218a2ad9d6edfad6ba7246107c152e79a9efd8
Summary: When debugging a failure in DaemonStartupLoggerTest.closePipeWhileStillRunning, I was confused because the pipe which is closed in the test (readPipe/writePipe) is unrelated to the DaemonStartupLogger. Eventually I learned that the pipe referred to by the test's name is a pipe internal to DaemonStartupLogger. Rename the test to express the high-level scenario instead of mentioning pipes (which are an implementation detail).
Reviewed By: simpkins
Differential Revision: D10247254
fbshipit-source-id: f5f5b358adad0b656db27ea75fec65fe0331317b
Summary:
This uses the existing unionstore to attempt to load blobs directly from
the hgcache. This is a pretty simplistic implementation:
1. We unconditionally add the filepack path into the list of treepack paths
in the import helper startup. This is not 100% good, but should be good
enough in practice. This does mean that we'll only successfully find
a filepack if we have enabled treemanifests.
2. The load is performed in context of the calling thread, which means that
we may block. Not sure if that is critical at this point.
In the longer run, we want to migrate this to the newer rust code; this is
just early stage proof of concept.
Reviewed By: strager
Differential Revision: D10277985
fbshipit-source-id: 9b953bbf3983bdb777635c6ff0b78efc101a17cc
Summary:
FileInode::stat is called very often, even by FUSE operations
as common as lookup(). stat() requires the size, which we don't in
general know until the blob's been imported. That said, if the blob
has been imported once, we don't actually need to decode the entire
blob out of RocksDB - we can much more cheaply read the cached blob
metadata to get the size.
Differential Revision: D10441161
fbshipit-source-id: aafc52b54aca9ba30248420fbc4f2ccf1ec0bed8
Summary:
Eden supports reading the SHA-1 of a file via getxattr. Unfortunately,
it returned ENOSYS if you called getxattr on a directory inode. This
caused FUSE to fail all future getxattr requests with EOPNOTSUPP.
In addition to fixing that, this diff makes our xattr handling a
little more consistent across inodes:
- setxattr() always fails with ENOSYS
- removexattr() always fails with ENOSYS
- listxattr() is always handled by the corresponding inode class
- getxattr() is always handled by the corresponding inode class
Differential Revision: D10437723
fbshipit-source-id: a1ea1e92d3412abb15e91057becea3160a17f1e2
Summary:
StartupLogger handles two independent scenarios: edenfs in daemon mode, and edenfs in foreground mode. I need to add a third for systemd support: edenfs in foreground mode but with startup logs written to a file.
Refactor the existing StartupLogger into an abstract base class with two concrete derived classes: DaemonStartupLogger and ForegroundStartupLogger. This will make adding the systemd feature easy without confusing the logic for the existing scenarios.
Aside from dropping the setsid() call for ForegroundStartupLogger, this diff should not change behavior.
Reviewed By: simpkins
Differential Revision: D10228156
fbshipit-source-id: fbc2c771c6fbb675779ef05ca3b7395b70b67485
Summary:
This code to recompute the SHA-1 in fsync is probably unnecessary. It
shifts some work to the writer of the file from Buck's getSHA1 query,
which may not even occur and also overlaps with a lot of other work
Buck is doing. In addition, computing SHA-1 during fsync is an O(file)
operation, so a series of writes and fsyncs would result in quadratic
behavior.
Differential Revision: D10436219
fbshipit-source-id: 9ea9b027e7676181478c4ffc9d791fed8033255c
Summary:
We weren't doing anything that interesting in flush()
anyway. Precomputing the SHA-1 for materialized files optimizes for a
relatively rare situation that penalizes the writer of large files for
the possibility that Buck might read the files later.
Differential Revision: D10435552
fbshipit-source-id: 24aa8f7d9ec5094b084ebd02964840b4b01ad48b
Summary:
Make sure all code that requires treemanifest is properly gated with
`EDEN_HAVE_HG_TREEMANIFEST` checks.
Reviewed By: chadaustin
Differential Revision: D10436528
fbshipit-source-id: b57005980242a3152f08b654306bff6e9c3ea4d3
Summary: Update some deprecated `Future::then()` calls to the new methods.
Reviewed By: chadaustin
Differential Revision: D10436529
fbshipit-source-id: 357ebd21606af6273ff0d622d6badfca6d8fb903
Summary: Convert then to thenValue or thenTry as appropriate.
Reviewed By: Orvid
Differential Revision: D10413847
fbshipit-source-id: 834756c2bcb4722fbe3729ce54aece0482b0a7ff
Summary:
To ease the migration from the current custom daemon setup to a systemd-managed daemon, add a way to enable the to-be-implemented systemd behavior.
* To the edenfs daemon, support the --experimentalSystemd option.
* To the Eden CLI, support the EDEN_EXPERIMENTAL_SYSTEMD environment variable. When set, invoke the edenfs daemon with --experimentalSystemd.
Aside from adding a log message, this diff does not change any behavior. This diff just sets up scaffolding for future diffs.
Reviewed By: simpkins
Differential Revision: D10248055
fbshipit-source-id: 30ca5a4bfde00ff43fd6f2a5d0282ced4f177fed
Summary:
D10370931 made all logs from StartupLogger use origStderr_ and never origStdout_. origStdout_ is thus unused. Delete origStdout_.
This diff should not change behavior.
Reviewed By: pkaush
Differential Revision: D10371294
fbshipit-source-id: 5dc1049192fea68a064a81143ea140c07452ecef
Summary:
Logging some messages to stdout and some messages to stderr has a few problems:
* stdout and stderr can get interleaved. For pipes, interleaving is avoidable for messages smaller than PIPE_BUF (512 bytes), but there's no guarantee that stdout and stderr are pipes or that the reader of the pipe reads PIPE_BUF bytes at a time.
* For the systemd integration I am developing, I need one file for stderr logs and another file for stdout logs. This is more complicated than having a single file for all logs.
* The distinction between stdout and stderr is unclear. No messages from StartupLogger are meant to be piped to another program.
Address these problems by only logging to stderr.
From a user's point of view, this diff should not change behavior.
Reviewed By: chadaustin
Differential Revision: D10370931
fbshipit-source-id: b063ed3a66cefc846d30ce1486759cbd3f3ea29e
Summary: D3376727 removed the `+1` from `AbsolutePathBase::paths`, but did not remove the comment referring to the `+1`. Remove the stale comment.
Reviewed By: pkaush
Differential Revision: D10394249
fbshipit-source-id: 3b2dc27f7083fe8a8706bac38275249dd4969d11
Summary:
Eden performance can degrade when something (a status operation in
particular) queues a ton of work on the hg importer queue. In that
case, the 12 RocksDB threads are all blocked waiting to insert more
work into the importer queue. Thus, even RocksDB queries that would
hit cache cannot make progress until the importer queue clears up.
In the long term, we need a more comprehensive approach to managing
our concurrency and boundedness of scheduled work. In the short term,
though, just make the importer queue unbounded.
Reviewed By: bolinfest, strager
Differential Revision: D10368779
fbshipit-source-id: 9200002338bea091fb353b2a96ac543df98701c2
Summary:
In D10384953 I learned that std::unordered_map<Hash, V> V for any V
would waste storage by storing the hash result in each hash node. This
is especially silly because the hash function of std::Hash is as
simple as loading the first word from memory.
Marking the hash function as noexcept reduces the node size for
std::unordered_map<Hash, BlobMetadata> from 72 to 64, making it fit
cleanly in one of jemalloc's slabs.
Reviewed By: strager
Differential Revision: D10385972
fbshipit-source-id: 7a3da5d09be0650ad43849e6d82334b44f3582a3
Summary:
The next version of LLVM that will be released to fbcode will include two new
warnings: `-Wreturn-std-move` and `-Wreturn-std-move-in-c++11`. Although these
are disabled globally in fbcode (see T34353622), `eden/BUILD_MODE.bzl`
re-enables these due to its `-Wextra` option.
Fix the `-Wreturn-std-move` violations.
Alternatively, I considered having `eden/BUILD_MODE.bzl` explicitly disable
these warnings, but simply fixing the warnings seemed like the path of least
resistance.
Reviewed By: taewookoh
Differential Revision: D10376364
fbshipit-source-id: 376dc3a216e2c7eb71934bb0015e852dda37bdfd
Summary:
I want to split the StartupLogger class (see D10228156). The split will make logPath_ inaccessible from the success() function. I see two solutions:
* Make successImpl() (called by success()) write the entire message. This means each implementation of successImpl() needs to log the same "Started edenfs" message.
* Make success() write just the "Started edenfs" message, and have successImpl() write the "Logs available at" message (if possible). This means each implementation of successImpl() which has a log file needs to log the same "Logs available at" message, but in the short term there will be only one such implementation.
Implement the latter solution to minimize source code size.
Reviewed By: simpkins
Differential Revision: D10228164
fbshipit-source-id: aaa03e9d8c1d08c00a388a7fcaa76aa3f629decf
Summary: Ensure the "Started edenfs" log message is shown to the parent process when running in daemon mode. Also ensure the log message is shown at all in foreground mode.
Reviewed By: simpkins
Differential Revision: D10227185
fbshipit-source-id: c13987100c4b591218bbf1aa80828a8490188434
Summary: Existing tests for StartupLogger only test the daemon (not --foreground) case. Test the --foreground case as well.
Reviewed By: simpkins
Differential Revision: D10225293
fbshipit-source-id: b99d3ed9457d2d57cefd648d5e9513d12189d0b7
Summary:
This diff clarifies that the errors are a result of speaking with
mononoke in their exception reason, and includes the original error context
in the case of: eg: a connection failure.
Reviewed By: chadaustin
Differential Revision: D10298353
fbshipit-source-id: fbccc39706819270cf19ef6ca8397734440f525e
Summary:
This comment is instructional but is better situated in the
header file.
Reviewed By: singhsrb
Differential Revision: D10298365
fbshipit-source-id: 07ad9757a6be06a97f6aae81cff94088b776c11f
Summary: This was previously hard coded and making it configurable is operationally desirable.
Reviewed By: chadaustin
Differential Revision: D10298279
fbshipit-source-id: 124c3fa319c44aacb3d17ef6e380e9dec49db8be
Summary:
This little function helps reduce the amount of boilerplate
and casting in the main body of the code. It should not change
and functionality.
Reviewed By: chadaustin
Differential Revision: D10297205
fbshipit-source-id: ea74c294cf2ca258e3e9abb0e47209626d58f2a5
