Summary:
EdenMount calls fuseMount, but EdenServer calls fuseUnmount. I think only one of EdenMount or EdenServer should call fuseMount/fuseUnmount. Splitting the responsibility is confusing.
Move the call to fuseUnmount into EdenMount by creating a member function called unmount.
This diff should not change behavior.
Reviewed By: simpkins
Differential Revision: D14115385
fbshipit-source-id: 845c4a87cdba9f270c0eacaf01916ad91c3b7c0e
Summary:
Update `EdenMount::initialize()` to perform a fault injection check. This
allows test code to inject delays and errors into the mount initialization
flow.
Reviewed By: strager
Differential Revision: D14079491
fbshipit-source-id: be80135b0833c8f0300104524473cc3e949fec34
Summary:
This reverts the change in D14188312 that moved `hg_import_helper.py` from
`bin/` to `libexec/eden/`, and instead updates edenfs to look for
`hg_import_helper.py` at `../../bin/hg_import_helper.py`
We cannot remove `hg_import_helper.py` from `/usr/local/bin` without breaking
existing running `edenfs` daemons as they will continue to look for
`hg_import_helper.py` in this location.
Rather than keeping a copy in both `bin/` and `libexec/eden` it seems better
to simply update edenfs so that it can correctly find `hg_import_helper.py`
even when edenfs itself has been installed in `libexec/eden/`
Reviewed By: strager
Differential Revision: D14197390
fbshipit-source-id: 2845bd79343bc29af9d3acbacecac4501cdb9032
Summary: We should only attempt to include selinux on Linux, not macOS.
Reviewed By: strager
Differential Revision: D14181109
fbshipit-source-id: be47b7bdadc3409577fa114559e905214848ebd8
Summary:
The Eden CLI tool is really a control program for `edenfs`. Rename it to
`edenfsctl` to free up the `eden` name for future use.
The Eden daemon shouldn't really be on the user's path, and instead belongs in
`libexec`.
For transition compatibility, `eden` is symlinked to `edenfsctl`.
Reviewed By: simpkins
Differential Revision: D13888875
fbshipit-source-id: 435cc63e92b85b1f28b8691e4846fbcb05bc450e
.pyre_configuration.local is a file specific to Facebook's
infrastructure. Committing this file to the public Eden repo was
unintentional. Delete this file to reduce confusion.
Summary:
futures::sleep returning a Future leads to continuations easily being run on
the Timekeeper's callback. The goal is to change the return type so that
futures::sleep returns a folly::SemiFuture.
This codemod is part of the first stage:
* Migrate all call sites off of futures::sleep and onto futures::sleepUnsafe.
This will be followed by:
* Changing the return type of futures::sleep to folly::SemiFuture.
* Migrating callers, where clearly safe such as when they follow with .via or
.semi() from sleepUnsafe to sleep.
* Migrating remaining callers.
Reviewed By: yfeldblum
Differential Revision: D14152623
fbshipit-source-id: bc6874e4320e4a7352ac61b20d9750458e2cbbff
Summary:
If EdenMount::shutdown is in progress and EdenMount::destroy is called [1], the EdenMount object won't ever be deleted. A comment in EdenMount::destroy claims that EdenMount::shutdown will delete the EdenMount object, but the comment is wrong:
case State::SHUTTING_DOWN:
// Nothing else to do. shutdown() will destroy us when it completes.
return;
This leak is probably benign, but the leak checker can cause tests to fail spuriously. I think the leak currently can't happen for EdenServer's mounts, but in the future the leak might occur and cause problems.
Make EdenMount::shutdown delete the EdenMount if EdenMount::destroy was called. This fixes the leak.
[1] EdenMount::destroy is called by EdenMountDeleter when the last std::shared_ptr<EdenMount> is destroyed.
Reviewed By: simpkins
Differential Revision: D14015024
fbshipit-source-id: 39a0ba78bb00afb5f0363fe5c74dbec00f9d68e3
Summary:
If EdenMount::shutdownImpl and EdenMount::startFuse execute concurrently, the mount's `state_` might end up inconsistent. For example, for the following sequence of events, EdenMount::shutdownImpl is erroneously called twice:
1. EdenMount::initialize transitions the mount's state to INITIALIZED.
2. EdenMount::startFuse begins and transitions the mount's state to STARTING.
3. EdenMount::shutdown begins and transitions the mount's state to SHUTTING_DOWN.
4. EdenMount::shutdown finishes and transitions the mount's state to SHUT_DOWN.
5. EdenMount::startFuse finishes due to FuseChannel::initialize failing with an error and transitions the mount's state to FUSE_ERROR.
6. EdenMountDeleter calls EdenMount::destroy, which observes that the mount's state is FUSE_ERROR. EdenMount::destroy proceeds by calling EdenMount::shutdownImpl.
This specific scenario can't happen in production right now because step 3 requires allowFuseNotStarted to be true (and it's never true in production).
A similar scenario can happen with EdenMount::startFuse+EdenMount::destroy running currently, but the effects are benign. However, in a future diff (D14015024), I am changing how EdenMount::destroy works, and the SHUT_DOWN -> FUSE_ERROR transition causes problems.
When FuseChannel::initialize returns an error, transition the mount to the FUSE_ERROR state only if we are not already shutting down (instead of unconditionally). This fixes the [impossible] scenario above, and also fixes EdenMount::destroy in a future diff.
Reviewed By: simpkins
Differential Revision: D14030898
fbshipit-source-id: fe5a749b8798749734e0de91eb3b71b601ebb64b
Summary:
This diff adds the dtype field to the glob results;
this will help to reduce the cost of some watchman queries by avoiding a
getFileInformation call that instantiates inodes.
As part of this, I added a bunch of unit test coverage.
Reviewed By: strager
Differential Revision: D8779149
fbshipit-source-id: 3064a3e42be55ec576fed9e0f7112edef426f32d
Summary:
This diff updates our internal eden packaging to correctly declare
a dep on python3, and to specify that python3 interpreter when building
the CLI.
It also allows importing of `readline` to fail, which is important because
the python3 on the system may not have a functioning readline extension.
Reviewed By: strager
Differential Revision: D14098276
fbshipit-source-id: ad1174e46b9b1c0ec1e602ecaeb59c1f0835472a
Summary: SDL checks generate errors for using negatives with unsigned ints (for ex x = -uint_var), which we have at few places in Folly and Thrift.
Reviewed By: wez
Differential Revision: D14125841
fbshipit-source-id: 3a9dd0fc36ecde4cdd75473c893c5f7adb450740
Summary: This would use all the available CPUs and speed up the build process.
Reviewed By: strager
Differential Revision: D13563433
fbshipit-source-id: 18b3862ae0c56ae3865c56864b58cf749c844bd4
Summary:
Add a new class for injecting faults into Eden. This will make it easier to
build tests that exercise specific error conditions or specifically control
the ordering of operations that might otherwise race.
To minimize the performance impact, the fault injection framework is disabled
by default and must be explicitly enabled at process startup using a command
line flag. If this command line flag was not specified all fault injection
checks short-circuit into a no-op. The `checkAsync()` version does still add
some extra overhead due to the addition of a new `folly::SemiFuture` into the
code path.
Reviewed By: wez
Differential Revision: D14079489
fbshipit-source-id: 3e8725d2e51e5c829199a602e90574424ea52636
Summary:
If PrivHelper::fuseMount fails with an error, EdenMount::getState returns STARTING. This is confusing. Make EdenMount::getState instead return FUSE_ERROR in this case.
Aside from changing the output of 'eden list' in some cases, this diff should not change behavior.
Reviewed By: chadaustin
Differential Revision: D14060982
fbshipit-source-id: ae9c2c532b1654cce36b806971053d89231058c3
Summary:
While moving code around in EdenMount::startFuse, EdenMount::shutdown, and EdenMount::destroy, I accidentally changed the result of EdenMount::getState in some circumstances. I noticed that we don't have any tests for EdenMount::getState, so it could have been broken even before my changes!
Write some tests for EdenMount::getState to verify the existing behavior and to prevent future regressions. Only add tests which pass with the current implementation; future diffs will fixe cases where EdenMount::getState returns the wrong value.
Reviewed By: chadaustin
Differential Revision: D14058334
fbshipit-source-id: 68b60ac92f94d336fcfb8f11e571379573bb3f42
Summary:
Some tests want to create a TestMount but don't care what files are in the mount. The following code is intuitively correct. However, TestMount's constructor takes a FakeTreeBuilder by non-const reference, so it fails to compile:
auto testMount = TestMount{FakeTreeBuilder{}};
The following code works, but more verbose:
auto builder = FakeTreeBuilder{};
auto testMount = TestMount{builder};
Overload TestMount's constructor to allow the former form in addition to the latter form. This neatly shrinks some test code.
This diff should not change behavior.
Reviewed By: chadaustin
Differential Revision: D14060110
fbshipit-source-id: c6a5d4b2c5859812efff279dedbc1fe690c8f8ad
Summary:
Make it easier to write tests involving a FakeFuse-mounted EdenMount by creating the TestMount::startFuseAndWait function which initializes EdenMount.
This diff should not change behavior.
Reviewed By: chadaustin
Differential Revision: D14059808
fbshipit-source-id: ce63bbce7f2c6e710910ad530aa1b4efefe80272
Summary:
I want to test EdenMount::startFuse when PrivHelper::fuseMount fails. FakePrivHelper makes this inconvenient. Extend FakePrivHelper so a test can execute arbitrary code for fuseMount calls.
This diff should not change behavior.
Reviewed By: chadaustin
Differential Revision: D14058663
fbshipit-source-id: 27ff32a75adde8aff952ac50b3f1977cf0b59e93
Summary: It's been a few weeks since we dropped support for `eden config --get` (D13737807), and we've received no reports of this being a problem. Delete the code which handles `eden config --get`.
Reviewed By: chadaustin
Differential Revision: D13772873
fbshipit-source-id: 04e2aa7f193806ee26c2f0ab1fe4246815c581a6
Summary:
Now that `hg debugedenimporthelper` has been released for
a little while, we can remove the bundled implementation of it from
the eden release.
However, we cannot remove the script itself as there are users
with long running edenfs instances that pre-date the knowledge
of `hg debugedenimporthelper`. So, as a compatibility shim,
this diff redirects `hg_import_helper.py` and has it exec the
command in mercurial.
For extra fun, our own integration tests rely on being able
to import `hg_import_helper.py` and override portions of it,
so we cannot remove its implementation from the codebase just
yet either.
So this diff:
* Introduces `proxy_import_helper.py` which execs `hg debugedenimporthelper`
* Installs `proxy_import_helper.py` as `hg_import_helper.py` in the rpm
* Leave `hg_import_helper.py` as-is in the tree for now.
Reviewed By: simpkins
Differential Revision: D13970332
fbshipit-source-id: 717dc86a880fbbbe4a7e801a8b748abd053c7f7c
Summary:
in D13970332 we want to allow using `hg.real` to run the
import helper that is embedded into mercurial, but we don't know
the full path, so we need to resolve it from the environment.
`folly::Subprocess` has `usePath` option, but it is not compatible
with also overriding the environment for a child process.
The easiest short term thing to do is to resolve the path ourselves
using this simple function call.
I've also added the path to `hg.real` that is used on our mac systems
to the PATH we pass down to `edenfs` when starting eden.
Longer term we want to better encapsulate how we resolve and start
the import helper, but for now this unblocks starting eden on the
mac.
Reviewed By: strager
Differential Revision: D14078612
fbshipit-source-id: bc6787bbe99cd87224b783d2fb9d3255b825b976
Summary:
Differ.h was including an overly-broad header when all it needed was
the type definitions. This makes building the inodes target with
mode/mac easier.
Reviewed By: simpkins, strager
Differential Revision: D14036477
fbshipit-source-id: 99f3a55d0523c179f6cf2aa4210119ae6f8e51f8
Summary:
I don't have RocksDB building in mode/mac yet, and the inode code
only depends on the store interface, not all of the specific
implementations.
Reviewed By: simpkins, strager
Differential Revision: D14036390
fbshipit-source-id: d0b8d5c4d4f2f38dbc9bc2476c24ac99398d37b7
Summary:
Some tests set up fuse_init_in manually then call FakeFuse::sendRequest. The specific fuse_init_in setup is benign and already exists in FakeFuse::sendInitRequest. Shorten these tests by making them call FakeFuse::sendInitRequest instead.
This diff should not change behavior.
Reviewed By: chadaustin
Differential Revision: D14059700
fbshipit-source-id: 210dcfdc4ae1c74e10c50dbc3f3044c3b908da0a
Summary:
Calls to ObjectStore::create and EdenMount::create are duplicated in the various TestMount::initialize overloads. Fix the duplication by moving the two calls into their own function.
This diff should not change behavior.
Reviewed By: chadaustin
Differential Revision: D14058972
fbshipit-source-id: b3e79756793292b59f09f53c05649a637c8af07e
Summary:
getInitialMode and getModeUnsafe were not inherently different, so
remove getModeUnsafe and write an accurate comment in Overlay's
serialization code about why the initial mode is saved and how we
could remove that logic in the future.
Reviewed By: strager
Differential Revision: D14007488
fbshipit-source-id: db42f45f00dcd213fabd9575360da1261931778b
Summary: Branches are going away. Remove the use of them.
Reviewed By: strager
Differential Revision: D14062107
fbshipit-source-id: 00f6d3666eb3cb6900cd570fa3fcf12ba75c2ae0
Summary:
This fixes a bug where "eden doctor" would incorrectly report that some
checkouts were not mounted when the version of `edenfs` was too old. It would
think it needed to remount the checkouts even though they were in fact working
correctly.
Older versions of `edenfs` did not return a `state` field in the
`listMounts()` reply. This caused the newer `eden doctor` logic in D13526754
to incorrectly think the mounts were not running. This fixes the code to
assume the state is `RUNNING` in this case.
Reviewed By: wez
Differential Revision: D14017363
fbshipit-source-id: 020135f386aad84b1a92f2699a2706ba73141ec7
Summary:
It's common for code to use InodeNumber without needing to include the
main FUSE headers or vice versa. Split them into two separate headers.
Reviewed By: strager
Differential Revision: D13979868
fbshipit-source-id: c5eeb6a3697bb538729a403434dc4f0f7408cda0
Summary:
Previously the default ignore path was `~/ignore`. This doesn't really seem
like a great path choice, and I suspect that no one is actually using an
ignore file at this location. This default was set in D8594297, but I it
looks like this was mostly accidental from just not separating out data for
the system ignore path (`/etc/eden/ignore`) vs the user ignore path.
Using `~/.edenignore` seems reasonable for now, and unlikely to conflict with
other paths. We used to use `~/.gitignore` as the default before D8906226,
but this did cause problems for a handful of users that treated their entire
home directory as a git repository, and had a `~/.gitignore` file that was
supposed to apply only to their home directory.
Reviewed By: wez
Differential Revision: D13984153
fbshipit-source-id: 887528372b9be789317933f7026dfcbde8cd4539
Summary:
Update FileChangeMonitor to stop logging messages about files not being
present. We normally use FileChangeMonitor to monitor the user's `edenrc`
configuration files and their personal and system-wide ignore files. It is
normal for these files to not exist in many cases, so there is no need to log
a warning message in this case.
Reviewed By: chadaustin
Differential Revision: D13981278
fbshipit-source-id: fa5f8d42980fd8683f55d3d51f5375a72f511dfe
Summary:
Update the `check_hg()` code to verify all required files in the `.hg/`
directory, and recreate them if necessary.
Reviewed By: chadaustin
Differential Revision: D13935005
fbshipit-source-id: aceb1aebcc54d039cb0a6a19a44e867424f15bf0
Summary:
This renames the CheckoutConfig `path` member variable to `backing_repo`.
The value in this variable is actually the path to the source control backing
repository, and not the path to the Eden checkout, which was confusing. (The
CLI tests were actually setting this up incorrectly in `create_test_mount()`,
but until now nothing in the tests had been using it, so we hadn't noticed
this problem earlier.)
I also changed `CheckoutConfig` to be a `typing.NamedTuple` as part of this
change.
Reviewed By: strager
Differential Revision: D13935006
fbshipit-source-id: 43b6ba4efa34f8c6fb8264de8bb31a33a6d73036
Summary:
Update the code in `hg_util` to take advantage of the `EdenCheckout` methods
for getting config data rather than passing around an `EdenInstance` and a
config object separately.
Reviewed By: strager
Differential Revision: D13935007
fbshipit-source-id: 7c4079fa9b6c1a56f15006f44da994bf69b0f050
Summary:
Make `build-oss.sh` actually run the cmake build.
At the moment this is only run from nbtd for mac platforms,
so that is all that I've tested with it.
This is similar to the the equivalent scripts for watchman.
In a later diff we can centralize and clean up the duplication.
Reviewed By: simpkins
Differential Revision: D13910031
fbshipit-source-id: 6a7250c22033e913d999cf928fed27dada0647ef
Summary:
This diff enables building the eden watcher by linking in thrift and
its various dependencies.
To support building in-fbsource and in the github repo, a `maybe_shipit_dir`
function is used to setup a symlink to the `eden` and `fboss/common` dirs (this
mirrors the shipit configuration for this project: we cannot simply run shipit
because we have to build on mac and windows and shipit requires Hack, and that
does not support those platforms).
I tried to persuade cmake to let me build this without the use of a symlink but
found it too difficult to teach everything about the path mapping. The
symlinks aren't terrible, but are the reason why this diff also updates some
`.gitinore` files that are seemingly unrelated to this diff.
This diff changes a couple of build/link options: without them the end product
fails to link either due to implicit/unilateral enablement of UBSAN in some of
the deps, or because warning->error promotion is turned on.
This diff includes a copy of the `ThriftCppLibrary.cmake` file from the fboss
repo. This should get centralized and shipit'ed out into the places that
consume it. That can be done when someone gets around to doing the same for
the `FindGlog.cmake` file and doesn't need to hold up this diff.
Reviewed By: simpkins
Differential Revision: D13486486
fbshipit-source-id: 3bb5b011771b2a87618147ca019b4e50a8e0aaf2
Summary:
Move code for loading and saving checkout configs and snapshot files from
EdenInstance to EdenCheckout.
Reviewed By: chadaustin
Differential Revision: D13932141
fbshipit-source-id: aa5cf56ed6f2ac7d3e5529eee353e1dd0ea12a21
Summary:
Refactor the checks that verify the contents of `.hg/dirstate`, in preparation
for teaching `eden doctor` how to repair the other required files in the
`.hg/` directory.
This creates a new `HgChecker` class, which I plan to use in the future for
adding checks for each specific file that needs to be checked. This also
consolidates all issues with the .hg directory into a new `HgDirectoryError`
class. This is mainly to keep the error reporting simpler: if the entire .hg
directory is missing we will report this as a single problem rather than a
separate problem for each file in the .hg directory that we will recreate.
Reviewed By: chadaustin
Differential Revision: D13928667
fbshipit-source-id: fb78f014d44e85b9ac07048cbe641e9e8be4ed4c
Summary:
Update the `create_test_mount()` helper function to return an `EdenCheckout`
object instead of simply the mount path as a string. This will make it easier
to write some upcoming tests that need a full `EdenCheckout` object.
Since `EdenCheckout` stores the mount path as a `pathlib.Path` instead of as a
plain `str`, this also updates a few more locations in the test code to use
`Path`.
Reviewed By: chadaustin
Differential Revision: D13928666
fbshipit-source-id: 585f27c344e2d0283bf5f8562e17ac3fd356c539
Summary:
Replace Future::onError with Future::thenError:
* to remove ambiguous typing
* to ensure that the executor is not lost and the returned Future is still bound to an executor
See:
https://fb.workplace.com/groups/fbcode/permalink/2002251863144976/
for details.
Reviewed By: yfeldblum
Differential Revision: D13908257
fbshipit-source-id: 8b5315b019290f1c60087ca5716c31ebbf1f1be5
