Summary: Updating Eden to send its blob cache memory usage to ODS. This will be useful in seeing how much of our overall memory usage is coming from the Blob Cache and debugging what is using large portions of Eden's memory.
Reviewed By: chadaustin
Differential Revision: D16157709
fbshipit-source-id: abae8e8da22369c7757187f998a1f6bef31d74e9
Summary: Running `eden debug journal -f` will print and follow the eden journal in a similar style to the unix `tail -f` command.
Reviewed By: chadaustin
Differential Revision: D16112458
fbshipit-source-id: 5304cd0f857bdbeca41c2591e98920f4f1fc8f42
Summary: Sets up a thrift interface to set the size of the journal (until truncation is added in the size field in the journal currently does nothing other than being viewable from getMemoryLimit)
Reviewed By: chadaustin
Differential Revision: D16042286
fbshipit-source-id: bc0acdf4ac5516cfac66fa0fbd87254d08ad479b
Summary: Added the cli command `eden stats object-store` for querying the counts on what part of the object store was responsible for finding the blob or blob size (local store or backing store). This will tell us how well local and in-memory caching works for different workflows.
Reviewed By: chadaustin
Differential Revision: D15934535
fbshipit-source-id: 70345f11a51c3c6996dc001d4101744395a3d182
Summary: eden_win_mount, eden_win_utils and eden_win_store include journal which depend on streamingeden_thrift. Adding the dependency to fix the build order.
Reviewed By: fanzeyi
Differential Revision: D16065869
fbshipit-source-id: d3bd5887e6c675885d284671bef7e514515d5fc0
Summary: Journal not specifying template argument for optional was causing eden builds to fail for old versions of C++. See D15978960.
Reviewed By: chadaustin
Differential Revision: D16059722
fbshipit-source-id: fb9903e304d9c5f79fc2d34f7763c3ada6ae4553
Summary: Create a counter for the end-to-end duration of the journal.
Reviewed By: chadaustin
Differential Revision: D15993675
fbshipit-source-id: 260a58a7b37dfa16b82b3df187ad7f7263ff67cf
Summary: Shows the end-to-end duration of the journal in "eden stats"
Reviewed By: chadaustin
Differential Revision: D15993261
fbshipit-source-id: 46471faca17d4f12ccdd8cea55b2722e33519a74
Summary:
Moving from the previous linked list approach to having a deque in the Journal.
The memory used by the deque is not tracked currently, the difference might be negligible though. I ran a script that touched a file 62 million times and here is the result comparing rss_memory used in eden and the journal's estimated memory: https://fburl.com/ods/6sq8g5vc and it still seems to closely estimate the right amount.
Reviewed By: strager
Differential Revision: D15944614
fbshipit-source-id: 6a1ac34ecd80c0eecb80411984f88f62ae712e91
Summary: Change getLatest such that it just returns Info about the latest delta and not the delta itself.
Reviewed By: strager
Differential Revision: D15931214
fbshipit-source-id: c7a1cf4d62cdd4f9396fab46354eabcbb31f32c0
Summary: Combine the backend function that iterates over the deltas into one function so we do not need to modify the code in multiple places as we start to modify the internal structure (Additionally pulled the one use of the code outside of the journal into the journal)
Reviewed By: strager
Differential Revision: D15912907
fbshipit-source-id: 2f59ff9e7f33ffa5b420859153a609e68bda10b4
Summary: Prior to this diff "eden debug journal" got the latest journal entry and manually seeked for the one "limit" prior to it, this has been updated to just passing the limit to the journal. [This will make the queries always happen from the tip of the journal, since in theory currently something can be added to the journal after the python script gets the latest delta]
Reviewed By: strager
Differential Revision: D15972018
fbshipit-source-id: 0ee0dd88a1e9edef5ccce3b3da2dbc09aa64f8a9
Summary:
Implements size-only local storage, as opposed to storing metadata. This is useful when the blob's SHA-1 is not needed. This diff prevents SHA-1 computations, which can be especially expensive for large blobs.
From D15934535, operations such as `ls -l` and `stat` will get the size of a blob in two ways:
1) The blob's size is already stored locally, so it will be deserialized and returned.
2) The blob is fetched from the backing store, stored, and its size is returned.
This diff optimizes the second case, because SHA-1 is no longer computed.
Reviewed By: strager
Differential Revision: D15723239
fbshipit-source-id: a868f3bf6b68a83ddafb057dc3e4e65f0a2dd989
Summary:
These allow the cli to setup and tear down mounts and
have the eden server keep track of them.
Reviewed By: strager
Differential Revision: D15707318
fbshipit-source-id: abdb8eaa28c8c67c8211a8af1647efe3a083e998
Summary: We need --edenDir support to run muliple instance of Edenfs, which is required to run the integration tests.
Reviewed By: simpkins
Differential Revision: D15951483
fbshipit-source-id: a516159cdeb5f046f795fc28399a2af5fe8a9f95
Summary: Adding a nullptr check for the case when the journal is empty and removing the use of default parameters.
Reviewed By: strager
Differential Revision: D15907329
fbshipit-source-id: 787b4a44f835fd8d128496ee6655e02987db98a7
Summary:
This contains several fixes to LocalStore handling during shutdown.
- Have EdenServer explicitly call localStore_->close() during shutdown.
This ensures that the local store really gets close, just in case some other
part of the code somehow still has an outstanding shared_ptr reference to
it.
- Add synchronization around internal access to the RocksDB object in
RocksDbLocalStore. This ensures that calling `close()` is safe even if
there happens to still be some outstanding I/O operations. In particular
this helps ensure that if background GC operation is in progress that
`close()` will wait until it completes before destroying the DB object.
This also improves the code so that calling subsequent methods on a closed
RocksDbLocalStore throws an exception, instead of simply crashing.
I don't believe the additional synchronization in RocksDbLocalStore should
have much impact on performance: the synchronization overhead should be very
low compared to the cost of the RocksDB reads/writes.
Ideally some of this synchronization logic should perhaps be moved into the
base `LocalStore` class: all of the different `LocalStore` implementations
should ideally ensure that `close()` is thread-safe and blocks until other
pending I/O operations are complete. However, that requires a bigger
refactoring. I may attempt that in a subsequent diff, but for now I mainly
want to address this problem just for RocksDbLocalStore.
Reviewed By: strager
Differential Revision: D15948382
fbshipit-source-id: 96d633ac0879b3321f596224907fcfe72691b3f0
Summary: This diff takes care of importing blob from Mononoke and Mercurial at the same time, also improves the name situation in the statistics counters.
Reviewed By: strager
Differential Revision: D15768557
fbshipit-source-id: 10cf831b1ae6dc9e6b91f1e96508c4fa92583743
Summary:
watchman/eden have been using the new Thrift streaming for nearly a year now and are the last users of the old
Thrift streaming. This diff kills the fallback `legacySubscribe` method, which complete's migration to the new Thrift
streaming and unblocks the complete removal of wangle-based streaming from Thrift and fbcode.
Reviewed By: wez
Differential Revision: D15878892
fbshipit-source-id: ec0d270dba79e56c7e41afbf36669a08e5a15518
Summary: This code calls close on fd while exiting. If the file doesn't exist the open will return -1 and close (-1) will be called.
Reviewed By: strager
Differential Revision: D15951485
fbshipit-source-id: ea3a52517847d75e9a822e51f360be7cb2c411da
Summary: Disabling this task because it's not implemented and will throw and crash the fs.
Reviewed By: strager
Differential Revision: D15951484
fbshipit-source-id: 0819219d621edfad488707b709705f636358ede8
Summary:
D15901794 deleted the move `ThriftLogHelper` move constructor, to suppress
some warnings from clang on Mac OS. However, this causes build failures on
Windows, which needs the move constructor. Clang does not need the move
constructor due to copy elision, which is guaranteed by C++17.
This also changes the itcLogger_ member variable to be a copy instead of a
reference: `folly::Logger` is just a simple wrapper around a
`folly::LogCategory*`, so making a copy should be just as efficient than using
a reference (if not more, since we no longer have to traverse the reference).
Reviewed By: pkaush
Differential Revision: D15936517
fbshipit-source-id: 92ab47ac9985a04e360f529bc99705a7cad5c5b2
Summary:
If edenfs was started using `sudo`, the `$USER` environment variable will be
set to `root` rather than the actual user. When we drop privileges make sure
we restore the value of `$USER` as well.
The `$USER` variable isn't checked anywhere else in edenfs itself, but it
matters for subprocesses we spawn, like `hg debugedenimporthelper`.
I also changed the code to clear the `SUDO_*` variables as well, mostly
just for good measure.
Reviewed By: kulshrax
Differential Revision: D15929539
fbshipit-source-id: e022c7ae762e2a5e86d0227058bb476aff17cf55
Summary:
Add a periodic task for performing LocalStore management tasks. For now only
the RocksDBLocalStore class implements this management task.
When this periodic task runs the RocksDBLocalStore object computes how much
space each of the column families are using and publishes this as fb303
counters. If the total size of the ephemeral column families exceeds a
configurable limit it then triggers a background garbage collection task.
I also added a new `edenfsctl stats local_store` command that reports the new
counters added by this diff.
Reviewed By: chadaustin, strager
Differential Revision: D15798505
fbshipit-source-id: 25ca4ba80f5a9c4a1a09dc08633c7b3af363d7ff
Summary:
This diff makes eden to import tree from Mercurial and Mononoke at the same time.
Since both futures will finish even if the other one finishes early, this change will make sure:
1. Mercurial fills its cache
2. User gets the fastest experience
3. Content SHA1 and file size still get fetched from Mononoke API Server since we store metadata in a separate space:
Reviewed By: chadaustin
Differential Revision: D15768555
fbshipit-source-id: 5407b5e1e9b0ea6c90905de5adcdfbbf09aa6469
Summary: This diff clears the use of `LocalStore::WriteBatch` in `HgBackingStore` so operations does not share the same write batch. Sharing write batch is causing segfault when we switch to importing concurrently from Mercurial and Mononoke.
Reviewed By: chadaustin
Differential Revision: D15768556
fbshipit-source-id: 1b0baee3a561d79b55d455c1dcb0d66475a1ea8d
Summary: EdenMount.cpp includes folly/Subprocess.h, but uses none of the included symbols. Drop the unnecessary #include.
Reviewed By: simpkins
Differential Revision: D15773136
fbshipit-source-id: 4f30af9a1afcc3258aaa00d519d2aceeda72824f
Summary: Update the license headers in the remaining Eden files.
Reviewed By: wez
Differential Revision: D15487083
fbshipit-source-id: cea9cc133907eadf5afc069f5d420f686b4c1886
Summary:
Update the copyright & license headers in thrift files to reflect the
relicensing to GPLv2+
Reviewed By: wez
Differential Revision: D15487082
fbshipit-source-id: 33f68617037f36c07075fb962a16a4d8f55bd6a6
Summary:
Update the copyright & license headers in CMake files to reflect the
relicensing to GPLv2
Reviewed By: wez
Differential Revision: D15487079
fbshipit-source-id: 715e559464c19a0070d6e55a095b3fc7d61ad2f8
Summary:
Update the copyright & license headers in Python files to reflect the
relicensing to GPLv2
Reviewed By: wez
Differential Revision: D15487088
fbshipit-source-id: 9f2138dff41048d2c35f15e09a04ae5a9c9c80dd
Summary:
Update the copyright & license headers in C++ files to reflect the
relicensing to GPLv2
Reviewed By: wez
Differential Revision: D15487078
fbshipit-source-id: 19f24c933a64ecad0d3a692d0f8d2a38b4194b1d
Summary: Merge was a function on a JournalDelta that created a new JournalDelta and optionally left it connected to old JournalDeltas. AccumulateRange is a new function on the Journal itself (acting on the latest delta) that creates a JournalDeltaSum (which can't be connected to older Deltas)
Reviewed By: chadaustin
Differential Revision: D15881452
fbshipit-source-id: 573505c1171f78d46afc98f1db9b5b9ee2fff60f
Summary: Making addDelta private and giving users a more user-friendly way of appending entries to the journal.
Reviewed By: chadaustin, strager
Differential Revision: D15868089
fbshipit-source-id: 00c8a3066f0e4483e3c792651ade5f6a7ea05eed
Summary:
Some newer versions of `clang` (such as Apple's version 11) will warn/error out if a constructor or assignment operator
marked `default` is implicitly deleted (e.g., if the object contains a non-moveable/non-copyable member). This diff
removes all such defaulted constructors/assignment operators, which I ran into while building `edenfs` on my Macbook Pro.
Reviewed By: chadaustin, strager
Differential Revision: D15901794
fbshipit-source-id: 794ed8377693a6735bb567635dc919bc678751a4
Summary: This diff allows watchman to connect to `EdenServer` over an rsocket stream.
Reviewed By: simpkins, strager
Differential Revision: D15900246
fbshipit-source-id: e4a99fcd7b9c75b6ad2c331c1bbf13d22f78cfd7
Summary:
Update `RocksDbLocalStore::getApproximateSize()` to use `DB::GetIntProperty()`
rather than `DB::GetApproximateSizes()`.
The `GetIntProperty()` call reports properties about the overall column
family, as opposed to `GetApproximateSizes()` which supports querying size
information for specific key ranges. We always want data about the full
table, so the `GetIntProperty()` call seems to make more sense for us.
Also change the code to include both the SST file size (which we were querying
before) and the memtable size (which was not included before). The
`GetApproximateSizes()` call allow this to be queried too, but using
`GetIntProperty()` still seems better since we don't care about a specific key
range.
Reviewed By: wez
Differential Revision: D15798506
fbshipit-source-id: 86d087876d838c0ff4ae517ccac2e8b8bf31e304
Summary:
Add a debug log message at the end of each PeriodicTask iteration, reporting
how long the task took.
Reviewed By: wez
Differential Revision: D15798504
fbshipit-source-id: 3f33877cb4a9ddb8d966877b005e6d4c60d7dbe0
Summary:
Add a periodic task to reload the configuration file from disk. By default
this runs once every 5 minutes, but this interval can be controlled from the
config file.
At the moment reloading the config file does not do much other than update the
interval for how frequently the config file is reloaded. However, I plan to
add additional periodic tasks shortly that are controlled by this config
setting.
This will also make it possible for other parts of the code to
access the config settings in the `ServerState` and use them as-is without
checking to see if they reloaded. Currently all of the code that accesses
config values performs a check to see if the config needs to be reloaded. If
we want to switch to Mercurial-style configs in the future that check will be
substantially more expensive.
This diff also includes a new thrift call to force the config file to be
reloaded immediately. This can be used to restart automatic config reloading
if it is ever disabled in the config file.
Reviewed By: wez
Differential Revision: D15756357
fbshipit-source-id: 1999f4730903633ce838842932a6ae6a65eda4e6
Summary:
Migrate the code from accessing optional Thrift fields directly to a safer
`optional_field_ref` API. See https://fburl.com/safe for more details.
The output of this codemod has been reviewed in D13259011.
To preserve semantics, each unchecked access is replaced with an explicit call
to `value_unchecked()`. If you are sure that accessing a field is safe (the
field is marked as set), you can later replace `value_unchecked()` with
`value()` or dereferencing (`operator *`):
```
ThriftStruct s = ...
- auto foo = s.foo_ref().value_unchecked();
+ auto foo = *s.foo_ref(); // will throw if s.foo is unset
```
drop-conflicts
Reviewed By: stevegury
Differential Revision: D15786632
fbshipit-source-id: 7c3f089dfa79597b189da6996cb76dcdbd5ed612
Summary: The first step towards decoupling size from the metadata object in the local store. This is necessary for the next step, which is to serialize size separately from the combined metadata object.
Reviewed By: chadaustin
Differential Revision: D15719637
fbshipit-source-id: 979555e3e362f2b698b3af3a0b6db1e350e07dcd
Summary: Wrote tests for retrieving hashes from the object store and refactored existing tests. Includes positive and negative tests.
Reviewed By: strager, pkaush
Differential Revision: D15697488
fbshipit-source-id: 44aa8a36fbc1d1e56dcbbb6bcb665d3784cbb476
Summary: `getSize` and `getSha1` were misleading function names, since the functions refer to the size and hash of a given blob and not the object store itself. These functions have been renamed to `getBlobSize` and `getBlobSha1`.
Reviewed By: chadaustin
Differential Revision: D15696510
fbshipit-source-id: 4dd31659f60969fa90d8e2b39f43c46a2b7dff7c
Summary: I decoupled the getSize() function from the getMetadata() function, using a naive implementation for now. This was necessary because I want to add support for fetching only the size of a blob during a request like `ls -l`. Right now, the size and hash of a blob are coupled in a Metadata object, so if the size is requested, the whole file must be downloaded to calculate the hash, which is expensive for large files.
Reviewed By: chadaustin, strager
Differential Revision: D15678216
fbshipit-source-id: 8f68692768faaae0e65373ffe608d09ae49bbc42
Summary: HgCommand.h and related code isn't used anywhere. Delete it.
Reviewed By: simpkins
Differential Revision: D15773223
fbshipit-source-id: c4c116150bb74e29e4f220d3b0cbf988d4a1b1c5
Summary:
The eden/fs/store library depends on eden/fs/config. The changes in
D15428932 exposed a build failure due to this missing dependency.
Reviewed By: strager
Differential Revision: D15768197
fbshipit-source-id: 4482e3a9027fd49163af39d6887da709be810b0a
Summary:
The `FieldConverter<bool>` code previously relied on `folly::to<string>()` for
all arithmetic types, including `bool`. Unfortunately `folly::to<string>()`
returns `"1"` and `"0"` for booleans, which isn't terribly human-friendly.
This updates the code to explicitly return `"true"` and `"false"` as the
output for converting boolean values to strings.
Reviewed By: strager
Differential Revision: D15550014
fbshipit-source-id: 69a3385e529e2940a8be20dbcbeda92e5836d969
Summary:
Add a `FieldConvertor` implementation allowing a config field to be parsed
into a `std::chrono::nanoseconds` value.
Reviewed By: strager
Differential Revision: D15428932
fbshipit-source-id: d3ace45a2986717c798897534e7b3015eda35fa9
Summary:
The `ConfigSetting` code previously used `folly::to<string>()` to convert
field values back to strings for debugging. However, this only works for
values that actually are supported by `folly::to()`
This changes the code to use the `FieldConverter` to perform the conversion
back to string. The `FieldConverter` class now contains the logic for doing
conversion both from strings and back to strings.
Reviewed By: strager
Differential Revision: D15428933
fbshipit-source-id: a37a978d151bf0ba79ca2a2b9107401c847b6591
Summary:
Add a `durationToString()` function. This converts a duration value to a
string that can be parsed back with `stringToDuration()`.
Reviewed By: wez, strager
Differential Revision: D15428935
fbshipit-source-id: 2840663385434bc35f04c16d2570fd86e234cb3f
Summary:
Add a `stringToDuration()` function to parse a string into a
`std::chrono::nanoseconds` value.
Strings are parsed as a sequence of `[number] [unit]` pairs.
For instance:
`1m30s` --> 90 seconds
`5s10ms` --> 5010 milliseconds
`1d3h10m` --> 1 day, 3 hours, 10 minutes (1630 minutes total)
Reviewed By: strager
Differential Revision: D15428934
fbshipit-source-id: 63f6614595c1aaa08c0d03633c6d1b53ca5bf3d5
Summary:
Update the `FieldConverter` code to support parsing any arithmetic type
(integers and floating point) using `folly::to<T>()`. Also change the `bool`
conversion implementation to use the same code.
This does change the `bool` implementation to accept any string accepted by
`folly::to<bool>(string)`. The old code only allowed `true` and `false`. The
new code allows other strings: `yes`, `no`, `y`, `n`, `0`, `1`, `on`, `off`
This also makes the implementation more efficient by eliminating some
unnecessary string copies and avoiding throwing exceptions internally on
error.
Reviewed By: wez
Differential Revision: D15428700
fbshipit-source-id: feb73a1f53775fbf3d3acb8585fa689f79c0db81
Summary:
Add a thrift call to get the current config settings.
My primary use case for this method at the moment is to make it possible to
build integration tests that check the config behavior. However in the future
this will probably also be useful for building CLI commands to report the
current config values to allow debugging if there are ever issues. This API
can also be used to force EdenFS to immediately reload the config from disk.
Reviewed By: strager
Differential Revision: D15572124
fbshipit-source-id: da3bc982f9c419b3314a8b0560c9bd327760d429
Summary:
Change the `ReloadableConfig::getEdenConfig()` method to accept an enum to
control whether it should reload the configuration, rather than the simpler
bool that it used previously.
The primary motivation for this is to allow us to force a reload to occur in
unit tests and integration tests, even if this operation would otherwise be
within the normal reload throttling time limit.
Reviewed By: strager
Differential Revision: D15572122
fbshipit-source-id: 9c638a415a451f26e7f5923f544724594ed4e0f1
Summary: Previously functions were defined per string type but since SSO detection is now templatized so can path memory estimation.
Reviewed By: chadaustin
Differential Revision: D15723098
fbshipit-source-id: df8dcd359a0f4f704eba337ebf5e382d87ca3abe
Summary: Previously hard-coded values for how long a string has to be for SSO, now just check if data pointer is inside string stuct
Reviewed By: chadaustin
Differential Revision: D15720268
fbshipit-source-id: 988e45648e8b96332587f8d2f021642407d3dac3
Summary:
Update the EdenServer::unmountAll() code to make sure to keep a shared_ptr
reference to the EdenMount while the EdenMount::unmount() call is running.
Previously this could cause use-after-free errors due to the EdenMount
object being destroyed while the `unmount()` logic was still running. In
particular, as soon as we call `privhelper->fuseUnmount()` the
fuseCompletionFuture could complete in a different thread. The
fuseCompletionFuture's callback (normally the lambda inside
`EdenServer::mount()` which triggers the call to
`EdenServer::mountFinished()`) may have the last outstanding shared_ptr
refcount to the `EdenMount` object. When it finishes in this other thread
it will then destroy the EdenMount, possibly before the
`EdenMount::unmount()` logic has completed.
Reviewed By: strager
Differential Revision: D15688674
fbshipit-source-id: 47b8b4fe022d2bca112558f9ef32bcdd169fefb0
Summary: JournalStats is currently O(# of deltas), updating it to be O(1)
Reviewed By: chadaustin
Differential Revision: D15718255
fbshipit-source-id: 1fb3f0b76d736bfa22195231c21d5f8b742fa1f7
Summary: Setting up structure to get the stats from Eden's Journal to ODS
Reviewed By: chadaustin
Differential Revision: D15684380
fbshipit-source-id: 90debc3f2a09d497201e5e73f85400a994b7afba
Summary: Replace inclusions of `folly/futures/helpers.h` with `folly/futures/Future.h` to avoid the cyclic include trap.
Differential Revision: D15600549
fbshipit-source-id: 19950be24a7437fb1fbec293e24058adf17343ca
Summary:
The `testSplayOn()` and `testSplayOff()` tests have been failing occasionally
in continuous integration tests. In every test failure I have looked at, the
only reported failures were for iteration 0: the very first time the task was
scheduled after the server started. I believe the server start is simply
taking a non-zero amount of time, delaying how long it takes for the first
iteration of the tasks to be run.
This updates the tests to wait for 3 iterations of the main event base loop
before recording the test start time and scheduling the tasks.
Reviewed By: strager
Differential Revision: D15618328
fbshipit-source-id: 2b119700662457c0fcdf38cc231c6c7d26ade248
Summary:
Move the ConfigSource enum definition to a thrift file. This will let us
return ConfigSource values over thrift APIs in the future. This also allows
us to use thrift's `TEnumTraits` functionality to determine the maximum enum
value, rather than having to maintain a separate `kConfigSourceLastIndex`
variable.
As part of this change I also renamed the enum values to be CamelCase to match
our current C++ style recommendations and to avoid possibly conflicting with
macros defined in other headers (`DEFAULT` seemed particularly susceptible to
collision).
Reviewed By: strager
Differential Revision: D15572120
fbshipit-source-id: 8fbd03da221a9f75ef670dee1eb250eb198a5bd0
Summary:
Update the thrift definition for `initiateShutdown()` to indicate that it may
throw `EdenError` on failure. Without this all application-level errors that
occur will be translated to generic thrift `TApplicationException` objects on
the wire.
Also move this method declaration out of the "debugging APIs" section of the
file.
Reviewed By: strager
Differential Revision: D15572121
fbshipit-source-id: 7e621a24abd4347cedbb1bcce1ae9c2b70f991fd
Summary:
Don't honor the older (and deprecated) fb303 shutdown() call. The
`initiateShutdown()` call should always be used instead now, which includes a
bit more diagnostic information that we log to describe why the shutdown was
requested.
Reviewed By: wez, strager
Differential Revision: D15572123
fbshipit-source-id: 413be611313c9ae490673a2b34742e512cf8e5ea
Summary:
This was accidentally left behind from my original version of this code that
did not use `TEST_F()` but instead had a local `TestServer` variable in each
test function.
Reviewed By: wez
Differential Revision: D15618327
fbshipit-source-id: 5dd1252c77c286d22a53dd5cb4708a0b890f887c
Summary: This turned out to be a lot simpler than I thought it would!
Reviewed By: chadaustin
Differential Revision: D15630515
fbshipit-source-id: 51aeb8b6739cb886c3bca23ab441874ea9ac819c
Summary: These functions are part of the blocking API, which is being replaced by the Future-based API. They are not being called, so they are unneeded.
Reviewed By: chadaustin, strager
Differential Revision: D15582641
fbshipit-source-id: 11f693040124846515aef7c7ee6ad750e1b5c164
Summary: Developers should be encouraged to use Future-based functions instead of blocking functions in production code. Removing this blocking function makes it less convenient for developers to write blocking code in the future.
Reviewed By: strager
Differential Revision: D15564952
fbshipit-source-id: 3f62db472a59a6487ffe12e48c04178cad84ca61
Summary: The new Windows CMake targets were failing to include the generated thrift headers on Sandcastle machines. Making eden_service_thrift a dependecy will make sure that the thrift generated files are created before we compile the code.
Reviewed By: simpkins
Differential Revision: D15536743
fbshipit-source-id: 4bedb8f33cddf5f7eb8f5b5ce52b2013728b38f7
Summary: This is a stop gap solution to get the Windows build working with the getdeps, until we fix the SDK.
Reviewed By: chadaustin
Differential Revision: D15536740
fbshipit-source-id: 77cc6ea80c304a6cfcd0180bb28f63ce4dac2988
Summary:
Update EdenServer to use the new PeriodicTask class for flushing thread local
stats and collecting memory usage stats.
Reviewed By: strager
Differential Revision: D15492169
fbshipit-source-id: 41c23630ffe2f8365a0234a252821d07d48c8b29
Summary:
Add a new class for running periodic tasks on the main EventBase thread.
We already have several tasks that we run periodically:
- Flushing thread local stats (every 1s)
- Reporting memory stats (every 30s)
- Unloading unused inodes (configurable, but disabled by default)
I plan to add some new periodic tasks soon:
- Report LocalStore stats and perform garbage collection if needed
- Check if the config files have been updated and need to be re-read
This new class should make it easier to add periodic tasks with configurable
intervals, and make the intervals at which they run controllable from the
configuration file.
Reviewed By: strager
Differential Revision: D15492165
fbshipit-source-id: c1d2663da3c08851b60ba64d7a1e38ccca709b8a
Summary:
This diff teach Eden's Python client to use Windows' Unix Domain Socket support to connect to Eden.
#delayed_publish
Reviewed By: chadaustin
Differential Revision: D15454514
fbshipit-source-id: 86aa39f8aa13ae789efabda3fa88635f471a1a7a
Summary:
Some threads update FUSE counters, some threads update HgBackingStore counters, and some threads update HgImporter counters. No thread updates all of FUSE counters, HgBackingStore counters, and HgImporter counters. FUSE threads and HgImporter threads allocate the full set of counters (EdenThreadStats), even though many of the allocated counters can never be used during the thread's lifetime.
Reduce redundant allocation (and overhead during counter aggregation): split EdenThreadStats into three classes (FuseThreadStats, HgBackingStoreThreadStats, and HgImporterThreadStats) so different threads can allocate different sets of counters.
This diff should not change observable behavior.
Reviewed By: simpkins
Differential Revision: D14822273
fbshipit-source-id: cfd238187d20a0b8d3959673401ecad894e2095b
Summary: This diff pushes service resolution down to the `CurlClient` so that when a host is gone it can avoid sending more requests to a dead server.
Reviewed By: strager
Differential Revision: D15385384
fbshipit-source-id: a47e80f796dcd93d662750e3426abcb7cfb79607
Summary: Enable rocket server for services that switch to rocket/rsocket transport *in the future*. This diff should be a no-op for existing services that take rocket/rsocket traffic.
Reviewed By: yfeldblum
Differential Revision: D15511722
fbshipit-source-id: e2ba1ed9a32c18eca68b5d06722edbf029244f05
Summary:
Add a helper class for creating an EdenServer in a unit test.
Most of our existing unit tests only create EdenMount objects, without a full
EdenServer. This new class will make it easier to write tests for
functionality that does require a full EdenServer object.
Reviewed By: chadaustin
Differential Revision: D15492166
fbshipit-source-id: f8b1ce3b78a1160a5d55d305e6bf4b5305cca509
Summary:
In dev builds I have seen the `DaemonStartupLoggerTest.crashWithNoResult()`
test sometimes fail with the message
`edenfs is still running but did not report its initialization status`
instead of the expected error `edenfs crashed with signal 9`.
It appears that when killing the child process with `SIGKILL` we can see the
pipe close before the child is actually waitable. Update the `StartupLogger`
code to sleep briefly and retry the wait a few times before giving up in this
case. This allows us to determine the child's exit status correctly in this
situation.
Reviewed By: wez
Differential Revision: D15507309
fbshipit-source-id: 1f169c3061d1b2bd4154849c5dc7249e8f8c8e0c
Summary:
Read information from the machine-readable `/proc/self/statm` rather than
attempting to parse the human-readable `/proc/self/status` file.
This simplifies the parsing logic overall.
Reviewed By: wez
Differential Revision: D15466907
fbshipit-source-id: c0184b0faa0a18cf0101828c3822b546ca7a677e
Summary:
HgBackingStore has two constructors. The constructor used in production code accepts an EdenStats object, but the constructor used in test code does not. This means that `HgBackingStore::stats_` could be null, thus tracking statistics in HgBackingStore is more work than it should be.
Make HgBackingStore require an EdenStats object so `HgBackingStore::stats_` is never null.
In production code, this diff should not change behavior.
Reviewed By: simpkins
Differential Revision: D15256266
fbshipit-source-id: f74137bed3e83e1b82d68b840fa55f3a3f0bf0cc
Summary:
`HgBackingStore::getBlob` calls `HgImporter::importFileContents` in two different branches. The calls are identical, and have an identical comment explaining the use of `folly::SemiFuture::via`. This duplication makes it annoying to add instrumentation and other changes.
Factor each call into a new `HgBackingStore::getBlobFromHgImporter` function, deduplicating the code.
This diff should not change behavior.
Reviewed By: simpkins
Differential Revision: D15256267
fbshipit-source-id: cad0566e9dab15775ee0d711e452f3e60fa6e645
Summary:
Remove calls to Future::then(executor, callback).
This form of Future::then is ambiguous, and does not yet implement the stronger
typing of thenValue and thenTry. It is also tempting to use instead of via,
where it is not obvious that it has the behaviour of wrapping a via call in a
push and pop of the current executor:
.pushCurrentExecutor().via(executor).then(callback).popCurrentExecutor().
With the addition of inline continuations, we can instead make the nesting
explicit at low cost by making it an inline continuation that launches an
asynchronous task on the passed executor.
Reviewed By: yfeldblum
Differential Revision: D15487604
fbshipit-source-id: 24b83f56c4f3e8843a6ce8339c70ff4794500d5b
Summary:
D15456687 didn't catch this due a problem with the CI
not showing the build failure in the code review tool.
Reviewed By: chadaustin
Differential Revision: D15472747
fbshipit-source-id: 5ad739926c0c01faaddd0d5c9631cdc8aafd33bd
Summary:
Computing the private memory usage from /proc/self/smaps is quite expensive.
On my system this takes around 90ms on opt mode builds, and 2s on
dev mode builds. (I believe dev mode builds are much worse because they use
lots of small shared libraries.)
In practice the RSS memory usage tracks relatively closely to the private
bytes count, and should good enough for most performance investigation and
monitoring.
We do still report the private memory usage in the `getStatInfo()` thrift call
results.
Reviewed By: chadaustin
Differential Revision: D15466906
fbshipit-source-id: 80ebce25046dfb705e37f50d1ec0270698718576
Summary:
This utility was initially used for testing flatmanifest and treemanifest
import. It's less important now that we are planning to drop flatmanifest
import.
The recent changes to this code have mostly just been keeping up with changes
to Eden's initialization code. I plan to continue making some additional
changes to Eden's config and local store initialization. Rather than
continuing to keep this code up-to-date it seems simpler to just delete it.
We can always resurrect this later if we find a need for it again.
Reviewed By: chadaustin, strager
Differential Revision: D15456687
fbshipit-source-id: dd65f93012530392b399a353f5b092192666f102
Summary:
Some of our tests create Mercurial repositories with default settings
which means they don't support treemanifest. Flip treemanifest on for
them.
Reviewed By: wez
Differential Revision: D15302347
fbshipit-source-id: 34f98b88976b5de1bf2ad07c5272e217e3fe3b0a
Summary:
Add a mechanism to open the RocksDbLocalStore in read-only mode. This is
primarily helpful to allow the `eden_store_util` process to examine a DB
without making many modifications, including automatic compactions that might
happen otherwise.
Note that this doesn't appear to be entirely read-only from a file contents
perspective: RocksDB does still rewrite the MANIFEST and OPTIONS files when
opening the DB in read-only mode, and does rotate the logs and write a new LOG
file.
Reviewed By: wez
Differential Revision: D15350216
fbshipit-source-id: 797498f9575a05334b741322654843d6545ca35c
Summary: Use `folly::prettyPrint()` when printing the sizes.
Reviewed By: chadaustin
Differential Revision: D15398827
fbshipit-source-id: 17b1bee7b78f63c5b766aad8a65f1a8f776bf6b4
Summary:
Increase the default send timeout from 100ms to 250ms.
Increse the timeout in the `UnixSocket.sendDataAndFiles` test to 10s.
We have seen some failures in the `UnixSocket.sendDataAndFiles()` test on
our continuous build infrastructure when testing debug builds on Mac OS.
These failed due to hitting the send timeout on the 4MB and 32MB sends. Bump
up the timeout for this test to 10s. (I initially tried 250ms, and this was
still not sufficient.)
I increased the default send timeout to 250ms just to help ensure that we
don't hit it unexpectedly during real production code when running on Mac OS.
Reviewed By: wez
Differential Revision: D15401154
fbshipit-source-id: c8be68a77231e90d37c9b5b4f349379bdf2b15ed
Summary: getEntryPtr() does a case sensitive lookup because of which few Ovrsource builds were failing. Ovrsource code is including header files with the wrong case.
Reviewed By: strager
Differential Revision: D15344850
fbshipit-source-id: 3d5d658a49cdafc07dc9a18a2f3d2073306e8f40
Summary: Small parts of our code have diverged from our clang-format rules.
Reviewed By: strager
Differential Revision: D15380260
fbshipit-source-id: f668ac22d6c0c5f2468549f2a94dd1c9bb22ce3d
Summary: 2 seconds is apparently too short for some of the big directory and files we have in our repository.
Reviewed By: wez
Differential Revision: D15387381
fbshipit-source-id: 6ebf850c5fbad8d60c2e8c9f8cb19b1c28297d47
Summary:
The `LocalStore` constructor was updated to accept a `ReloadableConfig`
argument in D12949577, but this was never used anywhere. Remove it for now to
help simplify the code. If we do want to add it back in the future I think we
should make it required, rather than allowing a null config to be specified.
Reviewed By: wez
Differential Revision: D15350217
fbshipit-source-id: 8571b48dff8c8d079ba6b25821dd0b1d77ffe791
Summary:
Implement the `show_sizes` function which reports the approximate size on disk
used by each column family.
The initial skeleton for this command was accidentally included in D15162813.
Reviewed By: wez
Differential Revision: D15307391
fbshipit-source-id: 03efee939d65500e17b48047552dec1dbb7aff7e
Summary:
Add a `name` field to KeySpaceRecord. This allows the `RocksDbLocalStore` and
`SqliteLocalStore` code to use the name from the `kKeySpaceRecords` data
structure rather than keeping their own array of names that must be maintained
in sync with the `LocalStore` code.
Reviewed By: wez
Differential Revision: D15307390
fbshipit-source-id: b53bca7d53d630f9527a810f145380288c54198e
Summary:
Update the KeySpace enum values to start at 0 instead of 1. This simplifies
the code to avoid having to skip over 0 in a few places.
This also makes the `kKeySpaceRecords` array slightly less confusing. Unlike
the `columns` array used by `RocksDbLocalStore` and the `tableNames` array
used by `SqliteLocalStore`, the `kKeySpaceRecords` array was not previously
indexed by the `KeySpace` enum values.
Reviewed By: wez, strager
Differential Revision: D15307393
fbshipit-source-id: ae8392d02396b4dc3c18e9ee94b198fcbb9b1a34
Summary:
Move the KeySpace record definitions out of LocalStore.cpp and into a header
file. This will make it possible to access the key space information from
other files in the future.
I would like to have access to this data directly in the various LocalStore
implementations, as well as in eden_store_util.cpp. Currently both
RocksDbLocalStore.cpp and SqliteLocalStore.cpp have their own copies of key
space information that should really be shared.
Reviewed By: chadaustin, strager
Differential Revision: D15307392
fbshipit-source-id: ad7cdc705d3dc9834592fe0f14b38d82f91a3b77
Summary:
Now Eden on Windows could be build using
python.exe .\opensource\fbcode_builder\getdeps.py build eden
Reviewed By: strager
Differential Revision: D15251103
fbshipit-source-id: 8ac1097b234bb27ce478101bfbc9591251e2e359
Summary: MSVC doesn't like the ImporterOptions declared as class and defined as struct.
Reviewed By: wez
Differential Revision: D15251102
fbshipit-source-id: e7725df5db8ee39a20a1940a8d045369800aa4c3
Summary: tar is in /usr/bin on macOS.
Reviewed By: simpkins
Differential Revision: D15052024
fbshipit-source-id: 5388b2d53f876717310d0f9726b74e8a283d1a89
Summary:
This solves the client certificate mystery. We creates a new `EdenConfig` object
whenever there is a change detected and these two fields are lost during the
copying. Causing the variable substitution not working as expected.
Reviewed By: wez
Differential Revision: D15221180
fbshipit-source-id: 3fba36c8f261cb3dbe25e2e363d65661240be8cf
Summary: Change `ConfigSource` from a plain `enum` to an `enum class`.
Reviewed By: chadaustin
Differential Revision: D15162811
fbshipit-source-id: 680ec1e785de1a54b623d99477914afa5d0931ca
Summary:
Move the `ConfigSetting` template and related classes from `EdenConfig.h` to a
new `ConfigSetting.h` header file.
Reviewed By: chadaustin
Differential Revision: D15162815
fbshipit-source-id: 34fba0b8b0e8e50714f622bae642d20d77050673
Summary:
Update the `isValidAbsolutePath()` implementation to use `kDirSeparator`, and
update the TODO comment.
Reviewed By: chadaustin
Differential Revision: D15162812
fbshipit-source-id: f52f0e85c5eef342723947c6efaa52992da1660b
Summary:
Move the `FieldConverter` logic out of EdenConfig.h/cpp and into its own
module.
Reviewed By: chadaustin
Differential Revision: D15162816
fbshipit-source-id: f30d81329139e5827b81a609e83baddf71b6af5e
Summary:
Rename the `ClientConfig` class to `CheckoutConfig`, to help further progress
on updating our terminology from "client" to "checkout".
Reviewed By: chadaustin
Differential Revision: D15162814
fbshipit-source-id: 3575958f1161d5842c6f0ee9e2d2d20ab20b7372
Summary:
Move the `ReloadableConfig` implementation out of `ServerState` and into the
`ReloadableConfig` class itself. This also updates `ServerState` to simply
contain a `ReloadableConfig` member variable rather than inheriting from
`ReloadableConfig`.
This makes it easier to build other utilities that need a `ReloadableConfig`
object but do not have a full `ServerState` object, such as the
`eden_store_util` class and the `zero_blob` helper tool used by the
integration tests.
Reviewed By: chadaustin
Differential Revision: D15162813
fbshipit-source-id: 1ebc0ac6c475f4aa1090dd933ebce16bc222674c
Summary:
This timeout appears to be too short for the last step of the test that sends
32MB of data in multiple send calls of no more than 1000 bytes each. On
stress test runs this timeout has occasionally been firing and causing the
test to fail.
Reviewed By: chadaustin
Differential Revision: D15221306
fbshipit-source-id: d1f7ed2d58b47512a8f31f8324c61372394a6709
Summary:
beholdunittests
This enables some plumbing for running some of the
tests using the gtest/gmock machinery in cmake.
Part of this diff is removing the FindGMock.cmake file from the
eden repo; we now pull this in from the shared cmake library
that is populated by shipit.
Reviewed By: simpkins
Differential Revision: D14993344
fbshipit-source-id: 51caf9518c7f3a083a3b90cda10324c3a8170359
Summary:
This argument is unnecessary since D14222323, which should be deployed
everywhere by now. Passing in the extra repo argument forces hg to open the
repository object twice, which is unnecessary.
We already set the current working directory to the repository when invoking
the import helper, so hg will find the repository correctly from that.
Reviewed By: pkaush
Differential Revision: D15179667
fbshipit-source-id: 838cbee91748c41c713731187608a9823b971a53
Summary:
Update HgImporter to use `hg debugedenimporthelper` instead of the separate
`hg_import_helper.py` script by default on all platforms now. The
`debugedenimporthelper` subcommand has existed in our deployed versions of hg
for a while now.
Reviewed By: pkaush
Differential Revision: D15179668
fbshipit-source-id: 2fb8c4c9f92aed54c84899d6643f746baac73327
Summary:
This diff tries to improve the Thrift Mononoke backing store implementation by:
* Re-creating thrift client for every request since thrift has builtin connection pooling
* Using Eden's CPU thread pool for processing data fetched from remote
* Calling thrift methods within correct executor
Reviewed By: wez
Differential Revision: D15170818
fbshipit-source-id: c8be70755a851f63fb62e4d4f36833703283565e
Summary: We intend to break support for flatmanifest hg, so require that treemanifest is available in the build.
Reviewed By: wez
Differential Revision: D15057150
fbshipit-source-id: 449399cfb9d018f3b722598091eead1bd5d7c77d
Summary: Flatmanifest is on its way out. Remove support for falling back to it if a tree import fails.
Reviewed By: pkaush
Differential Revision: D15056459
fbshipit-source-id: a4df820322ee354d77f50a0ec92e9705d0f152ec
Summary:
Interestingly, the request doesn't fail, but does leave
our stack garbage intact. Let's be sure to zero it out to make
it more obvious what is happening.
While we're in here, let's make sure that we get the same
results from both ends of the socket pair.
Reviewed By: simpkins
Differential Revision: D14994593
fbshipit-source-id: 9aec957dfcd80d88c3d8fbce6bf45480502ea812
Summary: this fixes linking the tests when building with cmake in D14993344
Reviewed By: chadaustin
Differential Revision: D15167223
fbshipit-source-id: 1dda3beed3b6ff3788f3992783c58b4b92f697f2
Summary:
D15043209 missed fixing up the name of the variable
containing the list of sources.
The result is that we compile the `main` from `eden_store_util`
in to the tests and break gtest discovery in D14993344
Reviewed By: chadaustin
Differential Revision: D15167219
fbshipit-source-id: 95e9dc8ba74965f41bfebed1c1776605d8c9443a
Summary:
UnixSocketTest would block forever on macos if the message
size was larger than approx 1kb. It turns out that MSG_DONTWAIT
isn't documented in the `sendmsg` man pages and apparently has
no effect at all. Instead, the socket must be placed in non-blocking
mode. This appears to be the case for the client side of the sockets
but I need to follow up for the server side.
Reviewed By: simpkins, strager
Differential Revision: D14994588
fbshipit-source-id: 2a0a1c26a7b45ece82f8f79c15fb7756844bc86f
Summary:
on macOS `/var/tmp` is a symlink to `/private/var/tmp`
which means that our tests for path canonicalization end up
trying to compare the non-canonical `/var/tmp` prefix of the
temporary paths generated by the tests with `/private/var/tmp`
and failing.
Reviewed By: simpkins
Differential Revision: D14994584
fbshipit-source-id: e69addca8e315855d3ca87feb576ca7c54567c31
Summary:
D14677339 added tracing to all HgImporter requests. Each trace loads a thread-local variable through EdenStats::getStatsForCurrentThread(). Each of these loads has a small but non-trivial cost.
Because each instance of HgImporter is used only on one thread, each getStatsForCurrentThread() call returns the same EdenThreadStats object. Avoid the thread-local-variable lookups by caching the result of getStatsForCurrentThread() in HgImporter.
This diff should not change behavior.
Reviewed By: simpkins
Differential Revision: D14794284
fbshipit-source-id: d1609a1720d44c680dc0ebaa2536779def2a6f37
Summary:
If EdenFS' globFiles API is given two patterns, and one pattern is a prefix of the other, EdenFS effectively ignores the longer pattern. Given the patterns `project/src/*` and `project/src/*/*`, when EdenFS encounters `project/src/dir/`, it generates a result (because `project/src/dir/` matches the first pattern) but does not recurse into its children.
The problem caused by an incorrect understanding of the GlobNode::isLeaf_ flag. isLeaf_ means "this GlobNode should generate results", but GlobNode::evaluateImpl understands it to mean "this GlobNode should generate results, and this GlobNode has no children". Given the patterns `project/src/*` and `project/src/*/*`, the GlobNode representing `project/src/*` has isLeaf_=true but also has children.
Fix the bug by not using isLeaf_ for determining whether to recurse, and instead relying on the presence of child GlobNode-s.
Reviewed By: chadaustin
Differential Revision: D15078089
fbshipit-source-id: 1c480d11361f89193b35965266e6873c57181113
Summary:
If globbing returns a directory, that directory doesn't need to be loaded. GlobNode::evaluateImpl is careful to only load directories/trees if necessary. When fixing a bug in GlobNode::evaluateImpl, I accidentally broke this optimization. No test failed, though.
Ensure this optimization works and doesn't regress by writing a simple test.
This diff should not change behavior.
Reviewed By: chadaustin
Differential Revision: D15090386
fbshipit-source-id: fe932e4a7ad517ff4c4523d58668f05fe8b0cafb
Summary:
In GlobNode::evaluateImpl, the logic for recursing into child GlobNode-s is duplicated token-for-token for hasSpecials_ and !hasSpecials_. While fixing a bug in GlobNode::evaluateImpl, I was forced to update both copies, which is prone to mistakes.
Fix the duplication by factoring the code into a function.
This diff should not change behavior.
Reviewed By: chadaustin
Differential Revision: D15090688
fbshipit-source-id: 65eec62214074139cb8af75026758b631eb89bb2
Summary:
gdb and lldb have a hard time printing recursive structures involving std::vector and std::unique_ptr. This makes debugging GlobNode a pain.
Add a function to print a GlobNode recursively to stderr. This makes it easier to debug GlobNode using printf.
Reviewed By: simpkins
Differential Revision: D15078111
fbshipit-source-id: 3ba211026c7c894676435ade278ae859e1b5d1f6
Summary:
Sandcastle's [1] code coverage builds compile EdenFS and Hg with Clang's -fprofile-generate flag (or a related flag). By default, running a program with with that flag creates a `default.profraw` file in the current directory. This causes some tests to fail, such as HgBackingStoreTest.getTreeForCommit_reimports_tree_if_it_was_deleted_after_import, because `default.profraw` is unintentionally committed to the test repo:
```
[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from HgBackingStoreTest
[ RUN ] HgBackingStoreTest.getTreeForCommit_reimports_tree_if_it_was_deleted_after_import
eden/fs/store/hg/test/HgBackingStoreTest.cpp:64: Failure
Value of: tree1->getEntryNames()
Expected: has 2 elements where
element #0 is equal to foo,
element #1 is equal to src
Actual: { default.profraw, foo, src }, which has 3 elements
[ FAILED ] HgBackingStoreTest.getTreeForCommit_reimports_tree_if_it_was_deleted_after_import (1563 ms)
[----------] 1 test from HgBackingStoreTest (1563 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test case ran. (1563 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] HgBackingStoreTest.getTreeForCommit_reimports_tree_if_it_was_deleted_after_import
```
When Sandcastle runs the tests, it sets the `LLVM_PROFILE_FILE` environment variable, which configures the path to `default.profraw`. EdenFS' HgRepo class is stripping this variable when invoking hg, so hg uses the default path and not the configured path.
Make HgRepo pass `LLVM_PROFILE_FILE` through to hg. This fixes some of EdenFS' tests when run by Sandcastle for code coverage purposes. (This does *not* fix running the tests manually (without setting `LLVM_PROFILE_FILE`), though.)
[1] Sandcastle is Facebook's continuous integration system.
Reviewed By: simpkins
Differential Revision: D15104832
fbshipit-source-id: 3929b9b0ab0904f2552ace7d8e4e4f4a4221192c
Summary:
Migrate the code from accessing optional Thrift fields directly to a safer
`optional_field_ref` API. See https://fburl.com/safe for more details.
The output of this codemod has been reviewed in D13259011.
To preserve semantics, each unchecked access is replaced with an explicit call
to `value_unchecked()`. If you are sure that accessing a field is safe (the
field is marked as set), you can later replace `value_unchecked()` with
`value()` or dereferencing (`operator *`):
```
ThriftStruct s = ...
- auto foo = s.foo_ref().value_unchecked();
+ auto foo = *s.foo_ref(); // will throw if s.foo is unset
```
Reviewed By: fanzeyi
Differential Revision: D15078040
fbshipit-source-id: 8ef270e8c687535e9d4aa2208beff4517e121814
Summary:
Add a command to force a call to RepairDB() on the local store.
This is similar to using `ldb repair`, but invokes `RepairDB()` with the same
set of column family options as normally used by edenfs.
Reviewed By: chadaustin
Differential Revision: D15043210
fbshipit-source-id: 2c4c0e2d3410a50cb1e523611f569f1701604ae6
Summary:
Change eden_store_util to take a subcommand argument, and define 3 separate
subcommands: "gc", "clear", and "compact"
The "gc" was the old default behavior. The "clear" and "compact" subcommands
simply clear cached entries and compact the storage, respectively.
This also adds a `--keySpace` argument. If this argument is specified the
command will process only this single key space, instead of all key spaces.
Reviewed By: chadaustin
Differential Revision: D15043207
fbshipit-source-id: 7e76bc89862d9647edde631ed3d82c9afda09f41
Summary:
This moves some logic from the RocksHandles class up to RocksDbLocalStore.
The main thing moved here is the logic to automatically try and repair the DB
if opening fails. This will make it easier in a subsequent diff to make the
repair logic a bit smarter and more aware of our column family semantics.
This keeps RocksHandles a pretty dumb wrapper around the RocksDB object and
column family handles, whose only purpose is to manage destroying these two
things in the correct order.
Reviewed By: chadaustin
Differential Revision: D15043208
fbshipit-source-id: ee2d5619ac7781a892e1ba151712eee9e3ebfb14
Summary:
Move this utility from eden/fs/service to eden/fs/store, and rename it to
`eden_store_util`.
I originally put it in the `service` subdirectory since it depends on library
code from `service/`, and most of the other code in `store/` shouldn't depend
on code in `service/`. However logically it makes sense to be in the `store/`
directory.
I plan to update in the future to be able to perform some additionally utility
operations besides just garbage collection, so rename it to the more generic
`eden_store_util` name.
Reviewed By: chadaustin
Differential Revision: D15043209
fbshipit-source-id: e6c4a874ddd05bd1576ea2e8a6c6156ccdf4628f
Summary:
On macOS, the atime, mtime, and ctime fields are named st_atimespec,
etc. Add some portability functions so the inode tests compile on
macOS.
Reviewed By: simpkins
Differential Revision: D15048758
fbshipit-source-id: 6f85b8eb7f7da2f9bd21c9034296cde96b68f740
Summary: Add some includes that are required on macOS.
Reviewed By: simpkins
Differential Revision: D15048681
fbshipit-source-id: d8642dc0977dc8cd8aadc83fc750400a8f917b7a
Summary:
Add a standalone program that can be used to garbage collect the local store
data when edenfs is not currently running.
Reviewed By: wez
Differential Revision: D14910392
fbshipit-source-id: 1cf8f959e5967cfbb797752653cc72223c87134f
Summary:
Let's catch more Xcode and upstream regressions by enabling more
builds and tests on macOS.
Reviewed By: simpkins
Differential Revision: D15003839
fbshipit-source-id: 257086d578f28198c7718906e1ec06395a55896e
Summary:
Build the store tests on mode/mac. RocksDB doesn't build yet (and we
will likely move away from it anyway) so split it into its own target.
Reviewed By: simpkins
Differential Revision: D15002840
fbshipit-source-id: fa0fea1d52df06526032a6fd585e2da7273910ef
Summary:
The hg store implementation depends on Rust code that doesn't build on
mode/mac yet, but TestMount's dependency on hg was false anyway.
Reviewed By: simpkins
Differential Revision: D15001077
fbshipit-source-id: 840fdf565093ad7a6eb95e001a74ae95eecbf1db
Summary: These allow more tests to compile
Reviewed By: chadaustin
Differential Revision: D14994582
fbshipit-source-id: 6b2a0b276fda64c7f27e28ea9e6d548aaaa1db7c
Summary: This diff add four metrics to the stats we are collecting in Eden. Both time cost on calling `get_tree` and `get_blob` from hg and mononoke backing store.
Reviewed By: chadaustin
Differential Revision: D14780349
fbshipit-source-id: f0e902c7f7523fac2ae576a67f0c85555064e26c
Summary:
The Finder shows the volume name instead of the directory
name when it renders the directory containing a mount, and the default
name of `OSXFuse Volume X` makes it hard to distinguish between multiple
mounts in the same folder.
This changes the volume name to be the basename of the mounted path so
that the finder renders things more reasonably.
Reviewed By: chadaustin
Differential Revision: D14969807
fbshipit-source-id: de9d8469360712828598fed124ed02ee5d6dd44a
Summary:
Unfortunately, some node modules packages include `.DS_Store` files
and since we disable apple double, the kext refuses to allow them to be
unpacked in the repo. Ideally they wouldn't be in the repo, but we need to
unblock that workflow.
Reviewed By: chadaustin
Differential Revision: D14969808
fbshipit-source-id: f9262294a223fd5cc7929a2b324f2401e1fed083
Summary:
Replace Future::onError with Future::thenError:
* to remove ambiguous typing
* to ensure that the executor is not lost and the returned Future is still bound to an executor
See:
https://fb.workplace.com/groups/fbcode/permalink/2002251863144976/
for details.
Reviewed By: Orvid
Differential Revision: D14975808
fbshipit-source-id: b7790bf0e8965c9ffb3e63901bcdaf108480d4ca
Summary:
Add a helper function to recursively remove a directory from the local
filesystem, given an `AbsolutePath`.
Reviewed By: wez
Differential Revision: D14954737
fbshipit-source-id: a3f6b60e610c18eabea2a7c507661a43502a2554
Summary:
Add a new class to manage access to the .eden state directory, and move some
logic for managing this director from the `EdenServer` class and into this new
class.
This is largely just a first step. In the future I think we should move
additional functionality into this class. For instance, logic to determine
the correct local store type and logic to load the checkout map from
config.json. In the future it probably also make sense to move this class
from the `service` library into a slightly lower-level library (perhaps
`config` or `store`).
Reviewed By: chadaustin
Differential Revision: D14889617
fbshipit-source-id: 15cb7daa9b97500124152d016bfdd686d3039771
Summary:
Move some of the argument parsing and config setup code out into a new
EdenInit.h header file. This makes it possible to re-use this logic for other
standalone utilities that want to be able to find the Eden state directory and
config information.
For now I have updated the `fake_edenfs` helper tool used by the integration
tests to use this. This may also be useful for writing standalone tools that
can perform garbage collection of the LocalStore or checking of the overlay
state.
Reviewed By: chadaustin
Differential Revision: D14889616
fbshipit-source-id: b0b193a42cb2f52177d0c44592426b42e27242aa
Summary:
This effectively reverts D14452214, which caused Eden to write an `id` entry
to each RocksDB column family and then flush the column family each time
edenfs started.
There was relatively little benefit to this in practice. It only matters in
cases where the RocksDB column families never had enough data written to them
to get flushed automatically and then a repair is required.
On the other hand it does have some material downsides: it flushing the column
families can be fairly expensive, and can require a substantial amount of free
disk space. This flush caused some users to not be able to start up edenfs
when they did not have enough free disk space.
Reviewed By: chadaustin
Differential Revision: D14947235
fbshipit-source-id: a29f98163fa87185b028bb47945b6fab75700fd6
Summary: Fallout from another diff that never got landed.
Reviewed By: simpkins
Differential Revision: D14938191
fbshipit-source-id: 980728ca6a1d4c4f90fea46aac9525a03d9809e5
Summary:
Change the `eden gc` implementation to use RocksDB's `DeleteFilesInRange()`
function, as well as its new-ish `DeleteRange()` method.
This makes the garbage collection much faster, and also require much less free
disk space than previously.
`DeleteFilesInRange()` asks RocksDB to simply delete all SST files from disk
if they only contain keys in the specified range. Since the range we specify
should include all keys in the DB this should simply drop all SST files for
this column family.
We also call `DeleteRange()` after this, just in case. This API is relatively
recent, and writes a single tombstone saying that the specified range has been
deleted.
Reviewed By: wez
Differential Revision: D14910345
fbshipit-source-id: c76bdc1c8e07cb2def66673ea892e7f455c9dc7a
Summary:
EdenStats is currently an alias for ThreadLocal<EdenThreadStats>. I want to split EdenThreadStats into two structs which are allocated independently, but EdenStats's interface makes this impossible.
Refactor EdenStats from an alias to a class and encapsulate the underlying ThreadLocal<EdenThreadStats> member.
This diff should not change behavior.
Reviewed By: simpkins
Differential Revision: D14822272
fbshipit-source-id: 691f4731aa22ecbdcd3535ee0bb0b99c816ffc3d
Summary:
I'm confused by the naming of EdenThreadStats and ThreadLocalEdenStats. For example, when I see "ThreadLocalEdenStats", I think that such objects can only be accessed by one thread. That's definitely not what "ThreadLocalEdenStats" means!
I think the following naming scheme makes more sense:
* **EdenThreadStats**: Statistics which are updated by one thread. Currently called EdenThreadStats.
* **EdenStats**: Statistics for all threads. Provides access to EdenThreadStats. Currently called ThreadLocalEdenStats.
Implement my preferred scheme: rename ThreadLocalEdenStats to EdenStats.
This diff should not change behavior.
Note: Prior to D14822274, EdenThreadStats was called EdenStats.
Reviewed By: simpkins
Differential Revision: D14822271
fbshipit-source-id: bd20179b1010588e3fc16dc9ed0657d458606f16
Summary:
I'm confused by the naming of EdenStats and ThreadLocalEdenStats. For example, when I see "ThreadLocalEdenStats", I think that such objects can only be accessed by one thread. That's definitely not what "ThreadLocalEdenStats" means!
I think the following naming scheme makes more sense:
* **EdenThreadStats**: Statistics which are updated by one thread. Currently called EdenStats.
* **EdenStats**: Statistics for all threads. Provides access to EdenThreadStats. Currently called ThreadLocalEdenStats.
Implement half of my preferred scheme: rename EdenStats to EdenThreadStats. (Make this diff easier to read by renaming ThreadLocalEdenStats to EdenStats in a separate diff.) In effect, implement the following naming scheme:
* **EdenThreadStats**: Statistics which are updated by one thread. (was EdenStats)
* **ThreadLocalEdenStats**: Statistics for all threads. Provides access to EdenThreadStats. (no change)
This diff should not change behavior.
Reviewed By: simpkins
Differential Revision: D14822274
fbshipit-source-id: 236cd9878cd249a06d14e124050ee01329667a18
Summary: This diff makes `MononokeHttpBackingStore` uses `ServiceAddress` to manage remote address of the Mononoke API Server.
Reviewed By: chadaustin
Differential Revision: D14855661
fbshipit-source-id: 1c102f7b2c547dc6a5b5c2cba65a513313d90bdc
Summary: This diff adds `ServiceAddress` to utils. This is a class that encapsulates an address of a service that could be a traditional hostname & port pair as well as an SMC tier name. This eliminates the needs of the manually resolving network address for remote services.
Reviewed By: chadaustin
Differential Revision: D14845257
fbshipit-source-id: 9fe9847cca4bba0170be94b9c209247342708574
Summary:
Sometimes, EdenFS goes bonkers and talks to 'hg debugedenimporthelper' a lot for seemingly no reason.
Make these situations easier to debug by counting how many requests EdenFS makes to 'hg debugedenimporthelper'. These counters let us answer questions such as the following:
* When performance sucks, is EdenFS is making a lot of requests or only a few requests? I.e. is Hg slow at responding or is EdenFS very demanding?
* Does a recent performance issue correlate with EdenFS communicating with 'hg debugedenimporthelper'?
* Which engineers are outliers having orders of magnitude more 'hg debugedenimporthelper' requests than p50 engineers?
We could get fancier with these counters and include the number of bytes received, the duration of the request, etc. For now, just having a request count is useful.
Reviewed By: simpkins
Differential Revision: D14677339
fbshipit-source-id: 7f8f394fb0096aef65d6a8a45d7da5936db539a0
Summary:
D8559702 changed `folly::IOBuf::computeChainDataLength()` to return a `size_t`
Update our format specifier to match to avoid compiler warnings on Mac.
Reviewed By: chadaustin
Differential Revision: D14878220
fbshipit-source-id: 19e96bea07c57bb542a848b3688d65143db51d13
Summary:
The file descriptor API here needs to go away, so switch this API to NetworkSocket
It is expected that this commit will cause a number of Open Source projects to temporarily show up as broken. This is due to the fact that not all projects get synced to Github at the exact same time, so the builds may temporarily be fetching an older version of it's dependencies than it needs to :) It should fix itself quickly.
Reviewed By: yfeldblum
Differential Revision: D14673328
fbshipit-source-id: c5842fa5dc383d50043e0d8228e35d03b10a1c6b
Summary:
Ensure that the `TreeInode::createImpl()` code always releases the contents
lock at the end of the scope that was intended to signify the critical
section. Previously this was only unlocked when returning from this scope
successfully.
In particular, the old behavior could deadlock:
- We first created the new child inode.
- If saveOverlayDir() failed, we would throw an exception, causing us to not
release the directory lock yet.
- As we returned in the exception case, the local `inode` variable would be
destroyed before the `contents` argument, causing us to release the child
InodePtr's refcount before releasing the parent directory's lock. Because
we were the only user of the child inode we would end up needing to acquire
the parent inode's content's lock to check if it was unlinked.
This fixes the code to ensure that we always release the tree's contents lock
before the child inode becomes unreferenced.
Reviewed By: chadaustin
Differential Revision: D14848807
fbshipit-source-id: 28dcde842072b9ee1e7c63d54456d849e31af8fe
Summary: Move Mononoke backing store initialization process to every request so we can adjust the configuration value without restarting the process.
Reviewed By: strager
Differential Revision: D14583357
fbshipit-source-id: 49ce2736229ce3062d34337757ebda6bb6eae16a
Summary: This is a helper function to help lazily initializing Mononoke backing stores only when necessary and release it when it is not used any more. See next diff in the stack for detail usage.
Reviewed By: chadaustin
Differential Revision: D14791528
fbshipit-source-id: c26811bc5c7aebcd02f704f10ad19bc35f8b9a21
Summary:
We updating the encoding that mercurial uses for storing paths to UTF-8.
The current step is preventing files that have invalid names from being
committed. To that end we are updating the HgPrefetch test to submit
valid file names (valid UTF-8). This allows us to start using UTF-8
encoded string for the internal representation of file paths.
Doing encoding transformation between internal storage and system locale will
be implemented later. Normalization should also be handled at that point.
Reviewed By: simpkins
Differential Revision: D14812780
fbshipit-source-id: 1689910500391eed941df185ba92aa2d2f3f0960
Summary:
I want to use EdenStats in eden/fs/store/. EdenStats currently lives in eden/fs/fuse/, and making eden/fs/store/ depend upon eden/fs/fuse/ is confusing. (It's also confusing that some code in eden/fs/fuse/ is used on Windows.)
Reorganize the code: move EdenStats into eden/fs/tracing/.
This diff should not change behavior.
Reviewed By: chadaustin
Differential Revision: D14677337
fbshipit-source-id: af26d214bcc3a9919920fbd4e59e6098fe4e3834
Summary:
This message was added in D14337058. It is logged at the `INFO` level, which
is enabled by default, but doesn't seem to add much value to normal production
logs.
Reviewed By: chadaustin
Differential Revision: D14712654
fbshipit-source-id: 5a86d883ace30e22d299046e33a6cd6247432857
Summary:
we need to be explicit about pulling in the dep for the
case where folly is not installed into a default installation
prefix.
Reviewed By: strager, pkaush
Differential Revision: D14683955
fbshipit-source-id: 0f302877674fb744ef8076641cd3fa72de74efe4
Summary:
This isn't a complete install but it is sufficient
to generate an `install` target for the getdeps build to run.
Reviewed By: strager
Differential Revision: D14680670
fbshipit-source-id: 9de1caa24c25702795842fe5b1b1f4d82aef24d8
Summary:
While testing out the new getdeps code I found that none
of the include directories from the probed libraries were being used.
The new getdeps installs each dep into its own prefix, whereas the
existing getdeps script installed them into the installation
prefix for eden itself. That meant that they were being implicitly
found from a single include directory.
In addition to this, I encountered linker failures for the pretty
printers; the solution to those was to add appropriate deps for
the modules that depend upon the pretty printers.
Reviewed By: pkaush
Differential Revision: D14638758
fbshipit-source-id: a4c2b4c79603c268e1b1c707a05c3cb0e3f2757b
Summary:
This function depends on things that are not available
when EDEN_HAVE_HG_TREEMANIFEST is not enabled.
Ensure that the entire function is disabled if that is the case.
Reviewed By: chadaustin
Differential Revision: D14638759
fbshipit-source-id: 3fe83b7b42357c2b818469214b016ff591058e35
Summary:
The issue is that the compiler needs an `else` to see
that we can only reach the throw if none of the other paths are
taken; with that satisfied it believes that we are legitimately
constexpr.
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67371
Reviewed By: chadaustin
Differential Revision: D14638234
fbshipit-source-id: f9524d2816580f41842a40e30118b03998c3660a
Summary: This is a largely automated codemod that shifts `folly::AsyncServerSocket::AcceptCallback::connectionAccepted` to taking a `NetworkSocket` rather than a file descriptor. This needs to be done as a single atomic change to avoid breaking things.
Reviewed By: yfeldblum
Differential Revision: D13966702
fbshipit-source-id: 415622dc347de53368c404dfbe9a2deae5b75e18
Summary: This diff enables thrift as a type of connection when talks to Mononoke API Server.
Reviewed By: strager, pkaush
Differential Revision: D14508956
fbshipit-source-id: 32e130c714be164f19452dec554976ea1cb594d3
Summary: Implement a backing store that uses thrift protocol to communicate with Mononoke API Server.
Reviewed By: strager
Differential Revision: D13966575
fbshipit-source-id: 66f66dda6b17aecd6c6b4475ab6b004c608f457f
Summary:
When Eden runs on kernel 4.20 (has readdir caching but not
FUSE_NO_OPENDIR_SUPPORT) indicate to the kernel that Eden wants
readdir results to be cached.
Reviewed By: wez
Differential Revision: D13893922
fbshipit-source-id: 3092adb16dabc4273bdba1e6e9f15e9e3bd7bb87
Summary:
Eden requires no state in its directory handles, so tell the kernel it
doesn't need to send opendir() and releasedir() requests, provided it
has FUSE_NO_OPENDIR_SUPPORT.
Reviewed By: strager
Differential Revision: D13594734
fbshipit-source-id: ebd4b69f4efcd1428a69024c4bdffb1ae455fa40
Summary:
After the kernel added readdir caching, my testing uncovered that Eden
was invalidating TreeInode entries incorrectly when new entries were
added. Change TreeInode to distinguish between directory entry changes
and removals (FUSE_NOTIFY_INVAL_ENTRY) and additions
(FUSE_NOTIFY_INVAL_INODE).
Reviewed By: strager
Differential Revision: D13870422
fbshipit-source-id: 2a6f25bfd9e77436a5aae639fedbfd8a445b2e05
Summary:
Prior to D14398507, two calls to EdenMount::unmount would result in two calls to fuseMountPromise->getFuture(). D14398507 made two calls to EdenMount::unmount result in only one call to fuseMountPromise->getFuture().
Since fuseMountPromise->getFuture() is only called at most once, turn fuseMountPromise into a Promise instead of a SharedPromise.
Reviewed By: simpkins
Differential Revision: D14515070
fbshipit-source-id: 3ecaf9b55f3274046bc840bd5a65481cc552df72
Summary:
I want to make EdenMount robust if mounting and unmounting are requested concurrently. This will make it safer to call EdenMount::destroy in different situations.
Take a step toward that goal by making EdenMount::unmount idempotent; calling EdenMount::unmount multiple times should return the same future.
Changed behaviors:
* If EdenMount::unmount is called multiple times, EdenMount::fuseMount is called at most once, and each call to EdenMount::unmount waits for the EdenMount::fuseMount call to finish.
* **Old behavior**: One of the calls to unmount succeeds, and the remaining calls to unmount fail with an exception from PrivHelper.
Unchanged behaviors:
* If EdenMount::unmount was ever called, EdenMount::startFuse fails with EdenMountCancelled.
* If EdenMount::unmount was ever called, EdenMount::takeoverFuse fails with EdenMountCancelled.
* If EdenMount::startFuse is in progress, EdenMount::unmount causes the concurrent startFuse call to fail with FuseDeviceUnmountedDuringInitialization.
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount waits for fuseMount to finish before calling PrivHelper::fuseUnmount.
* If EdenMount::startFuse -> PrivHelper::fuseMount is complete, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::takeoverFuse is in progress, EdenMount::unmount does not cause EdenMount::takeoverFuse to fail.
* If EdenMount::takeoverFuse was called, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If neither EdenMount::startFuse nor EdenMount::takeoverFuse was never called, EdenMount::unmount succeeds and does not call PrivHelper::fuseUnmount.
Reviewed By: simpkins
Differential Revision: D14398507
fbshipit-source-id: 1da5424abc47341e4db3da1c50d425711e177366
Summary:
I want to make EdenMount robust if mounting and unmounting are requested concurrently. This will make it safer to call EdenMount::destroy in different situations.
Take a step toward that goal by making EdenMount::unmount succeed if EdenMount::startFuse was never called.
Changed behaviors:
* If neither EdenMount::startFuse nor EdenMount::takeoverFuse was never called, EdenMount::unmount succeeds and does not call PrivHelper::fuseUnmount.
* **Old behavior**: EdenMount::unmount fails with an exception from PrivHelper.
Unchanged behaviors:
* If EdenMount::unmount was ever called, EdenMount::startFuse fails with EdenMountCancelled.
* If EdenMount::unmount was ever called, EdenMount::takeoverFuse fails with EdenMountCancelled.
* If EdenMount::startFuse is in progress, EdenMount::unmount causes the concurrent startFuse call to fail with FuseDeviceUnmountedDuringInitialization.
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount waits for fuseMount to finish before calling PrivHelper::fuseUnmount.
* If EdenMount::startFuse -> PrivHelper::fuseMount is complete, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::takeoverFuse is in progress, EdenMount::unmount does not cause EdenMount::takeoverFuse to fail.
* if EdenMount::takeoverFuse was called, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::unmount is called multiple times, one of the calls succeeds, and the remaining calls to unmount fail with an exception from PrivHelper.
Reviewed By: simpkins
Differential Revision: D14398537
fbshipit-source-id: 9209aad8b2980045fd0f89a5ba149a833579c2cf
Summary:
I want to make EdenMount robust if mounting and unmounting are requested concurrently. This will make it safer to call EdenMount::destroy in different situations.
Take a step toward that goal by making EdenMount::unmount complete only after a concurrent PrivHelper::fuseMount call finishes.
Changed behaviors:
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount waits for fuseMount to finish before calling PrivHelper::fuseUnmount.
* In other words, if EdenMount::startFuse and EdenMount::unmount execute concurrently, either:
* mount(2) is never called (if EdenMount::unmount won the race), or
* mount(2) is called then umount(2) is called (if EdenMount::startFuse won the race).
* **Old behavior**: EdenMount::unmount calls PrivHelper::fuseUnmount immediately and either fails or succeeds.
Unchanged behaviors:
* If EdenMount::unmount was ever called, EdenMount::startFuse fails with EdenMountCancelled.
* If EdenMount::unmount was ever called, EdenMount::takeoverFuse fails with EdenMountCancelled.
* If EdenMount::startFuse is in progress, EdenMount::unmount causes the concurrent startFuse call to fail with FuseDeviceUnmountedDuringInitialization.
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount causes the fuseMount to be rolled back (by calling PrivHelper::fuseUnmount) after fuseMount completes.
* If EdenMount::startFuse -> PrivHelper::fuseMount is complete, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* if EdenMount::takeoverFuse was called, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::unmount is called multiple times, one of the calls succeeds, and the remaining calls to unmount fail with an exception from PrivHelper.
Reviewed By: simpkins
Differential Revision: D14398508
fbshipit-source-id: 6f01ce9e36b5802045acf54afe756b80fe102b1e
Summary:
D14398423 made EdenMount::startFuse fail immediately if EdenMount::unmount was ever called. EdenMount::takeoverFuse behaves differently; any prior EdenMount::unmount calls don't cause EdenMount::takeoverFuse to fail.
For consistency, make EdenMount::takeoverFuse fail in the same situations that make EdenMount::startFuse fail.
Changed behaviors:
* If EdenMount::unmount was ever called, EdenMount::takeoverFuse fails with EdenMountCancelled.
* **Old behavior**: takeoverFuse succeeds.
* **Note**: EdenMount::unmount still fails with an exception from PrivHelper.
Unchanged behaviors:
* If EdenMount::unmount was ever called, EdenMount::startFuse fails with EdenMountCancelled.
* If EdenMount::startFuse is in progress, EdenMount::unmount causes the concurrent startFuse call to fail with FuseDeviceUnmountedDuringInitialization.
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount causes the fuseMount to be rolled back (by calling PrivHelper::fuseUnmount) after fuseMount completes.
* If EdenMount::startFuse -> PrivHelper::fuseMount is complete, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::takeoverFuse is in progress, EdenMount::unmount does not cause EdenMount::takeoverFuse to fail.
* if EdenMount::takeoverFuse was called, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::unmount is called multiple times, one of the calls succeeds, and the remaining calls to unmount fail with an exception from PrivHelper.
Reviewed By: simpkins
Differential Revision: D14511630
fbshipit-source-id: b77ca5ee39c888e2d50547f76977db7aa5a0331a
Summary:
I want to make EdenMount robust if mounting and unmounting are requested concurrently. This will make it safer to call EdenMount::destroy in different situations.
Take a step toward that goal by making EdenMount::unmount cancel a concurrent EdenMount::startFuse call.
Changed behaviors:
* If EdenMount::startFuse is in progress, EdenMount::unmount causes the concurrent startFuse call to fail with FuseDeviceUnmountedDuringInitialization.
* **Old behavior**: startFuse might succeed or might fail with FuseDeviceUnmountedDuringInitialization.
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount causes the fuseMount to be rolled back (by calling PrivHelper::fuseUnmount) after fuseMount completes.
* **Old behavior**: EdenMount::unmount does not change the behavior of EdenMount::startFuse.
* **Note**: EdenMount::unmount still fails with an exception from PrivHelper, despite having an effect on EdenMount::startFuse. This quirk will be addressed by a future diff.
* If EdenMount::unmount was ever called, EdenMount::startFuse fails with EdenMountCancelled.
* **Old behavior**: startFuse succeeds.
* **Note**: EdenMount::unmount still fails with an exception from PrivHelper, despite having an effect on EdenMount::startFuse. This quirk will be addressed by a future diff.
Unchanged behaviors:
* If neither EdenMount::startFuse nor EdenMount::takeoverFuse was never called, EdenMount::unmount fails with an exception from PrivHelper.
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount calls PrivHelper::fuseUnmount immediately and either fails or succeeds.
* If EdenMount::startFuse -> PrivHelper::fuseMount is complete, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::takeoverFuse is in progress, EdenMount::unmount does not cause EdenMount::takeoverFuse to fail.
* If EdenMount::takeoverFuse was called, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::unmount was ever called, EdenMount::takeoverFuse succeeds.
* If EdenMount::unmount is called multiple times, one of the calls succeeds, and the remaining calls to unmount fail with an exception from PrivHelper.
Reviewed By: simpkins
Differential Revision: D14398423
fbshipit-source-id: eb86a237859ad2d992de5c901e63dda80d5ea6d5
Summary:
In future diffs, I will introduce some EdenMount tests which need to mock PrivHelper in more ways than are currently supported. For example, I want to mock PrivHelper::fuseUnmount and check how many times fuseUmount is called. Extend the existing MockMountDelegate and FakeFuseMountDelegate classes to support these to-be-written tests.
This diff should not change behavior.
Reviewed By: simpkins
Differential Revision: D14418206
fbshipit-source-id: a9b1257bb62d9a46fae0eef89c3fa42718bec142
Summary:
Eden needs the osxfuse version of the FUSE protocol on mac. The open
source cmake build pulls in osxfuse with getdeps.py, but that doesn't
work in the Buck build, and all we need is this one (old) copy of the
Linux kernel headers.
Reviewed By: strager
Differential Revision: D14506747
fbshipit-source-id: 028ddbaf80be9cad412462f3338c30b8f2a70087
Summary:
If EdenMount::takeoverFuse is called and createFuseChannel fails, the EdenMount reports that it's in the STARTING state. This is misleading and doesn't match how EdenMount::startFuse handles failures in createFuseChannel.
Extend the `try` block in EdenMount::takeoverFuse to handle exceptions raised from createFuseChannel.
Reviewed By: simpkins
Differential Revision: D14511564
fbshipit-source-id: 8f791efe1221bcfc1042f95054d181c303564086
Summary:
Update `RocksHandles` to call `RepairDB()` if we get an error opening the
database.
We have seen errors opening the DB in some cases after hard server reboots.
In every case so far `ldb repair` has been able to repair it with no adverse
effects. This simply makes `edenfs` automatically attempt to perform this DB
repair step.
Reviewed By: chadaustin, strager
Differential Revision: D14452216
fbshipit-source-id: 10c0cb0ff9cea3c3bbe485a4e489e4a6df640803
Summary:
When opening the RocksDB, write one entry to each column family and then
explicitly flush that column family. This ensures that the column family
information has actually been flushed to an SST file. Without this some
column families may only have been written out to the write-ahead log files.
(Even calling `db->Flush()` does not appear to be sufficient; each column
family has to be explicitly flushed.)
The RocksDB' `RepairDB()` function (used by `ldb repair`) currently ends up
deleting column families that do not have any data defined in an SST file.
The repair tool ends up deleting column families that only have data in log
files.
The fact that we haven't been doing these explicit flushes previously probably
isn't too much of a concern in practice: once we write out enough data RocksDB
will automatically trigger a flush. This only matters in cases where we have
not yet written out enough data to trigger an automatic flush.
Note that with this change we re-write these `id` keys each time we open the
RocksDB store, even if they were already present.
Reviewed By: chadaustin, strager
Differential Revision: D14452214
fbshipit-source-id: 3f1b17e240cc89fe00e3d31105d16452795e754d
Summary:
folly::readFile() doesn't work correctly on Windows. It was returning the incorrect size. I didn't check if the read contents were correct, because I was testing it on a binary data file.
Moreover we would need wide char API which won't be a part of Folly for sometime.
Reviewed By: strager
Differential Revision: D14205306
fbshipit-source-id: 55859dca1a6399d5b5f106b4c6757e582ba1375d
Summary:
Rename `main.cpp` to `EdenMain.cpp` now that this contains the `EdenMain`
class rather than the top-level `main()` function, and rename `RunServer.cpp`
to `main.cpp`
Reviewed By: pkaush
Differential Revision: D14435306
fbshipit-source-id: e1528a773e0724c6bd50e31f6b33a1762d7bd49e
Summary:
This restructures `main.cpp` to turn our current top-level `main()` into a
method of a virtual class.
This allows us to eliminate the slightly awkward way that `main.cpp` depends
on `RunServer.cpp`. This required us to list `main.cpp` in the sources
parameter of multiple buck targets rules, which confuses some other tools like
autodeps. With this new change, `RunServer.cpp` depends on `main.cpp` instead
of the other way around. The real top-level `main()` function now lives in
`RunServer.cpp`
Reviewed By: pkaush
Differential Revision: D14435309
fbshipit-source-id: 402d00db0d8aa8d473d51a4f0e9d9d80c97a0134
Summary:
The systemctl command requires XDG_RUNTIME_DIR to be configured. If it's not configured, 'eden start' should pick a sane default. The sane default includes the user's UID (e.g. /run/user/6986). I want this default to be configurable via Eden's config files.
Expose the ${USER_ID} token to Eden configs. This will let administrators can customize XDG_RUNTIME_DIR's fallback value in the future.
Reviewed By: wez
Differential Revision: D13811732
fbshipit-source-id: 7933e078dd5f2b3bbbb0299730220a129c257256
Summary:
Implement `flush()` and `fsyncdir()` in `EdenDispatcher` and explicitly return
ENOSYS. We intentionally do not need to do any work for these methods.
Providing our an implementation that returns ENOSYS avoids the error log
message in the default `Dispatcher` implementation about the these calls being
unimplemented.
Reviewed By: chadaustin
Differential Revision: D14453245
fbshipit-source-id: 71efe6de6af73a5d705dace0f3439ba2466a50a8
Summary:
Update the FuseChannel code to explicitly handle `FUSE_LSEEK` and `FUSE_POLL`
to avoid emitting error log messages about these calls not being implemented.
We intentionally do not implement these operations at the moment.
Also drop the log about other unknown opcodes from an error log to a warning
message.
Reviewed By: chadaustin
Differential Revision: D14453246
fbshipit-source-id: 2605cf7e5c160cda92460a80187438ac3549ade5
Summary:
I want to change some of the StartupLogger behavior to automatically show all
INFO and higher level log messages to the user during start-up. These two
messages from the FUSE code are currently logged at the INFO level, but don't
really seem important enough to show to the user.
This drops their level to DBG1. These messages will therefore still be shown
in the normal `edenfs.log` file, since it includes everything from DBG2 and
up.
Reviewed By: chadaustin
Differential Revision: D14453247
fbshipit-source-id: 5d79766f87e658b807029d82ac035cb94ae68832
Summary:
The Windows headers and source files currently aren't mentioned in any buck
build rule, which confuses autodeps. Mark them all `manual` to tell
autodeps to simply ignore them.
Reviewed By: strager
Differential Revision: D14435307
fbshipit-source-id: a10b9be779f232cc5ae74356d7f54e77ca5414cb
Summary:
- Wait to call `curl_global_init()` until after we have dropped root
privileges. `curl_global_init()` performs a non-trivial amount of work,
including loading and processing OpenSSL configuration files from disk. To
guard against any possible security issues in this code we should wait until
after we have dropped root privileges to do this.
- Move the call to `curl_global_cleanup()` to after the main `EdenServer`
object has been destroyed. Since the EdenServer object will likely contain
backing store objects that use curl it seems like we want to make sure we
wait to clean up the curl library until after the `EdenServer` has been
destroyed. This change uses `SCOPE_EXIT` to perform the cleanup as one of
the last steps of `main()`, and this also reduces the number of `#ifdef`
blocks that we need.
Reviewed By: strager, fanzeyi
Differential Revision: D14435308
fbshipit-source-id: 6c277e4471f0f93decebd4fc741639c6a047d62a
Summary: We don't run this binary anymore, no reason to build and ship it.
Reviewed By: quark-zju
Differential Revision: D14437317
fbshipit-source-id: dd6da521783f18a2a518a7aa042be98950894e89
Summary:
The future returned by EdenMount::shutdown() can continue on an arbitrary thread. This has caused at least one deadlock in the past (D14337058).
Prevent users of EdenMount::shutdown() from continuing on the wrong thread by making shutdown return a SemiFuture instead of a Future.
This diff should not change behavior for production code, since all users of EdenMount::shutdown() already use Future<>::via().
Reviewed By: simpkins
Differential Revision: D14389607
fbshipit-source-id: 821d206d01ea3dcb0261cd0c7ca2d591b2c84e7a
Summary:
If TreeInode::startLoadingInode() is in progress, and EdenServer::startTakeoverShutdown() is called, edenfs can deadlock:
1. Thread A: A FUSE request calls TreeInode::readdir() -> TreeInode::prefetch() -> TreeInode::startLoadingInode() on the children TreeInode-s -> RocksDbLocalStore::getFuture().
2. Thread B: A takeover request calls EdenServer::performTakeoverShutdown() -> InodeMap::shutdown().
3. Thread C: RocksDbLocalStore::getFuture() (called in step 1) completes -> TreeInode::inodeLoadComplete(). (The inodeLoadComplete continuation was registered by TreeInode::registerInodeLoadComplete().)
4. Thread C: After TreeInode::inodeLoadComplete() returns, the TreeInode's InodePtr is destructed, dropping the reference count to 0.
5. Thread C: InodeMap::onInodeUnreferenced() -> InodeMap::shutdownComplete() -> EdenMount::shutdown() (called in step 2) completes -> EdenServer::performTakeoverShutdown().
6. Thread C: EdenServer::performTakeoverShutdown() -> localStore_.reset() -> RocksDbLocalStore::~RocksDbLocalStore().
7. Thread C: RocksDbLocalStore::~RocksDbLocalStore() signals the thread pool to exit and waits for the pool's threads to exit. Because thread C is one of the threads managed by RocksDbLocalStore's thread pool, the signal is never handled and RocksDbLocalStore::~RocksDbLocalStore() never finishes.
Fix this deadlock by executing EdenServer::shutdown()'s callback (in EdenServer::performTakeoverShutdown()) on a different thread.
Reviewed By: simpkins
Differential Revision: D14337058
fbshipit-source-id: 1d63b4e7d8f5103a2dde31e329150bf763be3db7
Summary:
FuseChannel::initialize throws runtime_error when the FUSE connection is interrupted. I want EdenMount::startFuse to handle unexpected-unmount errors from FuseChannel::initialize, but catching runtime_error would catch unrelated errors too.
When the FUSE connection is interrupted during initialization, make FuseChannel throw FuseDeviceUnmountedDuringInitialization instead of runtime_error.
Reviewed By: chadaustin
Differential Revision: D14077848
fbshipit-source-id: ed7b7d370a83ed1a9c36a443d8bb06ba940dc032
Summary:
In this change, we separate the low-level code that manipulates the overlay
into the FsOverlay class. The Overlay class makes use of the FsOverlay and
InodeMetaData table to support its Overlay interfaces. The FsOverlay class
is decoupled from the Overlay class, allowing other classes to manipulate
the overlay independently. We have a need for this in order to add
fsck to the c++ code base : described in T40728883.
Reviewed By: simpkins
Differential Revision: D14218281
fbshipit-source-id: 66c587f2b341579b8075ca5e5eeb4da6ffadf6f5
Summary: Since `CurlClient` is going to be used in multiple threads and each thread has its own instance of CurlClient, we are now able to assign one curl_easy handle for each of the client. So curl would be able to reuse the existing connections to Mononoke API Server.
Reviewed By: chadaustin
Differential Revision: D14334486
fbshipit-source-id: aacd210773c44dea6f1d9dd7f9b09d765a9c7a0a
Summary: Put `CurlClient` into its own thread pool (like how `HgBackingStore` uses `HgImporter`)
Reviewed By: chadaustin
Differential Revision: D14334483
fbshipit-source-id: 3ce2a8245761c11bfa0f66691d10a66004b46fe6
Summary: This diff enables `MononokeCurlBackingStore` in HgBackingStore. It also adds a new config value `mononoke:connection-type` (values: `http`, `curl` and `thrift`).
Reviewed By: chadaustin
Differential Revision: D14135028
fbshipit-source-id: cc6fb95d94b3d98996d872d22ad1a95a97d562ab
Summary: This diff add a MononokeBackingStore implemented based on libcurl so we could add Mononoke support on macOS.
Reviewed By: chadaustin
Differential Revision: D14096313
fbshipit-source-id: 2bcb0c49002dcbb0a0190ba505a6e814e74db747
Summary:
This diff added a configurable value "mononoke:connector" to EdenConfig, so that we can specify a particular type of connection we need when using Mononoke as backing store.
This diff also moved the logic of initializing the existing `MononokeHttpBackingStore` to `initializeHttpMononokeBackingStore`.
Reviewed By: wez, strager
Differential Revision: D14141725
fbshipit-source-id: c7208295d7b3853740d7b0e5166f8854457fcf8e
Summary:
Not sure why gflags is automatically included on Linux and not in
mode/mac, but add the includes nonetheless!
Reviewed By: strager
Differential Revision: D14389458
fbshipit-source-id: 27811ec4bb65b03e73b15bb51de1264dbfe053dc
Summary: Internal Facebook infrastructure is nagging me about some files not having a Facebook copyright notice. Add a notice to these files to make the nagging stop.
Reviewed By: simpkins
Differential Revision: D14173944
fbshipit-source-id: 7234431224fcf4f86ea56ca2f9108f47ef959d87
Summary:
This updates `edenfs` to automatically create the mount point directory
if it does not exist.
Previously the `eden mount` CLI command would automatically create the mount
directory in the Python logic. This adds similar logic to the C++ code, which
handles more situations. In particular this makes it so that `eden start`
will now automatically create missing mount point directories.
Note that the C++ code does not create the `README_EDEN.txt` symlink inside
the mount point if it is missing. We could move that functionality into the
C++ code in the future if needed.
Reviewed By: strager
Differential Revision: D14254699
fbshipit-source-id: bad5634f57fba6e7af3b6a3830eb51ac099b435e
Summary:
Update `Overlay::initialization()` to perform the bulk of the initialization
logic in the GC thread. We re-use the GC thread simply for convenience, since
it already exists and won't have any work to do until initialization has
completed anyway.
After an unclean shutdown this allows edenfs to start and perform the overlay
scans for all of its checkouts in parallel rather than serially.
Reviewed By: wez
Differential Revision: D14154868
fbshipit-source-id: c177cb9f950a6317fd7ea06888bd5b326a55ace7
Summary:
Add an Overlay::initialize() function, and consolidate all Overlay
initializtion logic in this function. Previously some initialization was done
by the Overlay constructor, and some was done in `EdenMount::initialize()`
Overlay::initialize() returns a folly::SemiFuture as it may perform a
non-trivial amount of disk I/O. However, at the moment it currently performs
all I/O in the current thread before it returns. I plan to move this work to
a separate thread in a subsequent diff.
Reviewed By: strager
Differential Revision: D13981140
fbshipit-source-id: b59eaef88012a8e74fcb770a9c93ca3f9bde32a0
Summary:
This updates the `EdenServer` class so that the existing `getMount()` and
`getMountPoints()` APIs only return mounts that have finished initializing.
These APIs are primarily used by the thrift interfaces. In most cases the
callers did not intend to operate on mounts that were still initializing, and
doing so was unsafe. The code could potentially dereference a null pointer if
it tried to access the mount's root inode before the root inode object had
been created.
New `getMountUnsafe()` and `getAllMountPoints()` APIs have been added for call
sites that explicitly want to be able to access mounts that may still be
initializing. Currently the `listMounts()` thrift API is the only location
that needs this.
Reviewed By: strager
Differential Revision: D13981139
fbshipit-source-id: e6168d7a15694c79ca2bcc129dda46f82382e8e9
Summary:
Add a flag to tell edenfs to report successful start-up as soon as the thrift
server is running, without waiting for all mount points to finish being
remounted.
In the future I plan to have edenfs automatically perform an fsck scan of the
overlay for checkouts that were not shut down cleanly. This may cause the
remount to take a significant amount of extra start-up time in some cases.
(This is already true today in some cases even with the simpler scan we do to
re-compute the max inode number.)
I think we will probably want to have systemd invoke edenfs with this option,
so that we do not time out during system start up if some mount points need to
be rescanned.
Reviewed By: strager
Differential Revision: D13522040
fbshipit-source-id: 6f183770c25efee34c4805c9bad42a9cce51039e
Summary:
In the future, I want some tests to mock PrivHelper::fuseUnmount. The existing classes for mocking PrivHelper::fuseMount, MountPromiseDelegate and FailingMountDelegate, work poorly for fuseUnmount.
Combine MountPromiseDelegate and FailingMountDelegate to create MockMountDelegate, which is more flexible and allows a mocking fuseUnmount to be implemented easily.
This diff should not change behavior.
Reviewed By: simpkins
Differential Revision: D14319712
fbshipit-source-id: 6eecf121e3b44f39cd5dffbeafaf0f3aab75cb76
Summary: Switch to using cpptoml from third-party on all platforms.
Reviewed By: igorsugak
Differential Revision: D14351179
fbshipit-source-id: 183bedc7d27e9c0f9216f3d0cca58ada75ac74e7
Summary: Ensure that an EdenMount is RUNNING after it takes over an EdenMount from another process.
Reviewed By: simpkins
Differential Revision: D14178500
fbshipit-source-id: 2813b024b6815dc5d31f3e3bf89cffa98ad0f1c3
Summary:
I want manipulate a FakeFuseMountDelegate in a test, but FakeFuseMountDelegate is private to FakePrivHelper.cpp. Make FakeFuseMountDelegate usable outside FakePrivHelper.cpp by putting its declaration in FakePrivHelper.h.
This diff should not change behavior.
Reviewed By: simpkins
Differential Revision: D14319711
fbshipit-source-id: d7af18c316f63febe1d60f6e5aadcd4f4827cca5
Summary:
Update `edenfs` to automatically create the bind mount source directories if
they are missing. Previously Eden would report an error and would not be able
to mount the checkout if some of the bind mount source directories were
missing.
Reviewed By: strager
Differential Revision: D14253771
fbshipit-source-id: 87ad091ccf2c0f0f72aebb50437fd7680ddbfd1c
Summary:
[Folly] Stop checking `EventBase::runInEventBaseThread` result, as the function will soon be changed not to return any result.
It returned `false` when failing to enqueue a task. But it cannot really fail anyway besides allocation failure, unless in the `EventBase` destructor and while draining and the `AlwaysEnqueue` variant is called.
Reviewed By: andriigrynenko
Differential Revision: D14254969
fbshipit-source-id: a6a9199cbafa18b61488a240e4318ce946953f51
Summary:
We're not that far from building privhelper on mode/mac but it does
require figuring out how to depend on osxfuse from the Buck build, so
bypass that by breaking the inodes target's dependency on privhelper's
<fuse_ioctl.h> include.
Reviewed By: strager
Differential Revision: D14218709
fbshipit-source-id: edbb2a21df06d6f2a4f860ef13718ad05d445e98
Summary: Fix some missing includes that showed up in my mode/mac builds of parts of Eden.
Reviewed By: simpkins, strager
Differential Revision: D14213572
fbshipit-source-id: 54e070e89afdfb8479abdaa122ba76ca1d2d9ba9
Summary:
Throw a slightly nicer error message if the hg import helper exits before
returning a `CMD_STARTED` response or an error response.
When switching from the `hg_import_helper.py` script to the
`hg debugedenimporthelper` subcommand we are slightly more likely to encounter
errors during startup. For instance, if the repository path was not a valid
repository the `hg_import_helper.py` code would send an error message back
that the `HgImporter.cpp` code could parse. However, we unfortunately can't
catch errors loading the repository the same way when using an `hg`
subcommand. If it fails to load the repository it will simply print an error
message to stderr and then bail out before it invokes the
`debugedenimporthelper` code.
Reviewed By: chadaustin
Differential Revision: D14222266
fbshipit-source-id: cd60e5a61725d45a816b74f63b9969b29ade2160
Summary:
EdenMount calls fuseMount, but EdenServer calls fuseUnmount. I think only one of EdenMount or EdenServer should call fuseMount/fuseUnmount. Splitting the responsibility is confusing.
Move the call to fuseUnmount into EdenMount by creating a member function called unmount.
This diff should not change behavior.
Reviewed By: simpkins
Differential Revision: D14115385
fbshipit-source-id: 845c4a87cdba9f270c0eacaf01916ad91c3b7c0e
Summary:
Update `EdenMount::initialize()` to perform a fault injection check. This
allows test code to inject delays and errors into the mount initialization
flow.
Reviewed By: strager
Differential Revision: D14079491
fbshipit-source-id: be80135b0833c8f0300104524473cc3e949fec34
Summary:
This reverts the change in D14188312 that moved `hg_import_helper.py` from
`bin/` to `libexec/eden/`, and instead updates edenfs to look for
`hg_import_helper.py` at `../../bin/hg_import_helper.py`
We cannot remove `hg_import_helper.py` from `/usr/local/bin` without breaking
existing running `edenfs` daemons as they will continue to look for
`hg_import_helper.py` in this location.
Rather than keeping a copy in both `bin/` and `libexec/eden` it seems better
to simply update edenfs so that it can correctly find `hg_import_helper.py`
even when edenfs itself has been installed in `libexec/eden/`
Reviewed By: strager
Differential Revision: D14197390
fbshipit-source-id: 2845bd79343bc29af9d3acbacecac4501cdb9032
Summary: We should only attempt to include selinux on Linux, not macOS.
Reviewed By: strager
Differential Revision: D14181109
fbshipit-source-id: be47b7bdadc3409577fa114559e905214848ebd8
