Commit Graph

2187 Commits

Author SHA1 Message Date
Chad Austin
5a532b216c remove some unnecessary includes
Summary: Noticed these includes aren't necessary.

Reviewed By: simpkins

Differential Revision: D14574423

fbshipit-source-id: 672e886841c64312e54baf98ebe808c4c654815a
2019-03-21 20:51:55 -07:00
Matt Glazar
3d89f8b1e6 Refactor fuseMount SharedPromise into Promise
Summary:
Prior to D14398507, two calls to EdenMount::unmount would result in two calls to fuseMountPromise->getFuture(). D14398507 made two calls to EdenMount::unmount result in only one call to fuseMountPromise->getFuture().

Since fuseMountPromise->getFuture() is only called at most once, turn fuseMountPromise into a Promise instead of a SharedPromise.

Reviewed By: simpkins

Differential Revision: D14515070

fbshipit-source-id: 3ecaf9b55f3274046bc840bd5a65481cc552df72
2019-03-21 13:51:16 -07:00
Matt Glazar
7da8f80758 Synchronize mount+unmount 4/4: Make unmount idempotent
Summary:
I want to make EdenMount robust if mounting and unmounting are requested concurrently. This will make it safer to call EdenMount::destroy in different situations.

Take a step toward that goal by making EdenMount::unmount idempotent; calling EdenMount::unmount multiple times should return the same future.

Changed behaviors:

* If EdenMount::unmount is called multiple times, EdenMount::fuseMount is called at most once, and each call to EdenMount::unmount waits for the EdenMount::fuseMount call to finish.
  * **Old behavior**: One of the calls to unmount succeeds, and the remaining calls to unmount fail with an exception from PrivHelper.

Unchanged behaviors:

* If EdenMount::unmount was ever called, EdenMount::startFuse fails with EdenMountCancelled.
* If EdenMount::unmount was ever called, EdenMount::takeoverFuse fails with EdenMountCancelled.
* If EdenMount::startFuse is in progress, EdenMount::unmount causes the concurrent startFuse call to fail with FuseDeviceUnmountedDuringInitialization.
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount waits for fuseMount to finish before calling PrivHelper::fuseUnmount.
* If EdenMount::startFuse -> PrivHelper::fuseMount is complete, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::takeoverFuse is in progress, EdenMount::unmount does not cause EdenMount::takeoverFuse to fail.
* If EdenMount::takeoverFuse was called, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If neither EdenMount::startFuse nor EdenMount::takeoverFuse was never called, EdenMount::unmount succeeds and does not call PrivHelper::fuseUnmount.

Reviewed By: simpkins

Differential Revision: D14398507

fbshipit-source-id: 1da5424abc47341e4db3da1c50d425711e177366
2019-03-21 13:51:16 -07:00
Matt Glazar
7697a170be Synchronize mount+unmount 3/4: Make unmount succeed if not mounted
Summary:
I want to make EdenMount robust if mounting and unmounting are requested concurrently. This will make it safer to call EdenMount::destroy in different situations.

Take a step toward that goal by making EdenMount::unmount succeed if EdenMount::startFuse was never called.

Changed behaviors:

* If neither EdenMount::startFuse nor EdenMount::takeoverFuse was never called, EdenMount::unmount succeeds and does not call PrivHelper::fuseUnmount.
  * **Old behavior**: EdenMount::unmount fails with an exception from PrivHelper.

Unchanged behaviors:

* If EdenMount::unmount was ever called, EdenMount::startFuse fails with EdenMountCancelled.
* If EdenMount::unmount was ever called, EdenMount::takeoverFuse fails with EdenMountCancelled.
* If EdenMount::startFuse is in progress, EdenMount::unmount causes the concurrent startFuse call to fail with FuseDeviceUnmountedDuringInitialization.
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount waits for fuseMount to finish before calling PrivHelper::fuseUnmount.
* If EdenMount::startFuse -> PrivHelper::fuseMount is complete, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::takeoverFuse is in progress, EdenMount::unmount does not cause EdenMount::takeoverFuse to fail.
* if EdenMount::takeoverFuse was called, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::unmount is called multiple times, one of the calls succeeds, and the remaining calls to unmount fail with an exception from PrivHelper.

Reviewed By: simpkins

Differential Revision: D14398537

fbshipit-source-id: 9209aad8b2980045fd0f89a5ba149a833579c2cf
2019-03-21 13:51:15 -07:00
Matt Glazar
f0a6413df7 Synchronize mount+unmount 2/4: Make unmount wait for pending mount
Summary:
I want to make EdenMount robust if mounting and unmounting are requested concurrently. This will make it safer to call EdenMount::destroy in different situations.

Take a step toward that goal by making EdenMount::unmount complete only after a concurrent PrivHelper::fuseMount call finishes.

Changed behaviors:

* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount waits for fuseMount to finish before calling PrivHelper::fuseUnmount.
  * In other words, if EdenMount::startFuse and EdenMount::unmount execute concurrently, either:
    * mount(2) is never called (if EdenMount::unmount won the race), or
    * mount(2) is called then umount(2) is called (if EdenMount::startFuse won the race).
  * **Old behavior**: EdenMount::unmount calls PrivHelper::fuseUnmount immediately and either fails or succeeds.

Unchanged behaviors:

* If EdenMount::unmount was ever called, EdenMount::startFuse fails with EdenMountCancelled.
* If EdenMount::unmount was ever called, EdenMount::takeoverFuse fails with EdenMountCancelled.
* If EdenMount::startFuse is in progress, EdenMount::unmount causes the concurrent startFuse call to fail with FuseDeviceUnmountedDuringInitialization.
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount causes the fuseMount to be rolled back (by calling PrivHelper::fuseUnmount) after fuseMount completes.
* If EdenMount::startFuse -> PrivHelper::fuseMount is complete, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* if EdenMount::takeoverFuse was called, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::unmount is called multiple times, one of the calls succeeds, and the remaining calls to unmount fail with an exception from PrivHelper.

Reviewed By: simpkins

Differential Revision: D14398508

fbshipit-source-id: 6f01ce9e36b5802045acf54afe756b80fe102b1e
2019-03-21 13:51:15 -07:00
Matt Glazar
a3c26e2ae5 Synchronize mount+unmount 1.5/4: Make unmount cancel takeoverFuse
Summary:
D14398423 made EdenMount::startFuse fail immediately if EdenMount::unmount was ever called. EdenMount::takeoverFuse behaves differently; any prior EdenMount::unmount calls don't cause EdenMount::takeoverFuse to fail.

For consistency, make EdenMount::takeoverFuse fail in the same situations that make EdenMount::startFuse fail.

Changed behaviors:

* If EdenMount::unmount was ever called, EdenMount::takeoverFuse fails with EdenMountCancelled.
  * **Old behavior**: takeoverFuse succeeds.
  * **Note**: EdenMount::unmount still fails with an exception from PrivHelper.

Unchanged behaviors:

* If EdenMount::unmount was ever called, EdenMount::startFuse fails with EdenMountCancelled.
* If EdenMount::startFuse is in progress, EdenMount::unmount causes the concurrent startFuse call to fail with FuseDeviceUnmountedDuringInitialization.
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount causes the fuseMount to be rolled back (by calling PrivHelper::fuseUnmount) after fuseMount completes.
* If EdenMount::startFuse -> PrivHelper::fuseMount is complete, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::takeoverFuse is in progress, EdenMount::unmount does not cause EdenMount::takeoverFuse to fail.
* if EdenMount::takeoverFuse was called, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::unmount is called multiple times, one of the calls succeeds, and the remaining calls to unmount fail with an exception from PrivHelper.

Reviewed By: simpkins

Differential Revision: D14511630

fbshipit-source-id: b77ca5ee39c888e2d50547f76977db7aa5a0331a
2019-03-21 13:51:15 -07:00
Matt Glazar
142415554e Synchronize mount+unmount 1/4: Make unmount cancel mounting
Summary:
I want to make EdenMount robust if mounting and unmounting are requested concurrently. This will make it safer to call EdenMount::destroy in different situations.

Take a step toward that goal by making EdenMount::unmount cancel a concurrent EdenMount::startFuse call.

Changed behaviors:

* If EdenMount::startFuse is in progress, EdenMount::unmount causes the concurrent startFuse call to fail with FuseDeviceUnmountedDuringInitialization.
  * **Old behavior**: startFuse might succeed or might fail with FuseDeviceUnmountedDuringInitialization.
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount causes the fuseMount to be rolled back (by calling PrivHelper::fuseUnmount) after fuseMount completes.
  * **Old behavior**: EdenMount::unmount does not change the behavior of EdenMount::startFuse.
  * **Note**: EdenMount::unmount still fails with an exception from PrivHelper, despite having an effect on EdenMount::startFuse. This quirk will be addressed by a future diff.
* If EdenMount::unmount was ever called, EdenMount::startFuse fails with EdenMountCancelled.
  * **Old behavior**: startFuse succeeds.
  * **Note**: EdenMount::unmount still fails with an exception from PrivHelper, despite having an effect on EdenMount::startFuse. This quirk will be addressed by a future diff.

Unchanged behaviors:

* If neither EdenMount::startFuse nor EdenMount::takeoverFuse was never called, EdenMount::unmount fails with an exception from PrivHelper.
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount calls PrivHelper::fuseUnmount immediately and either fails or succeeds.
* If EdenMount::startFuse -> PrivHelper::fuseMount is complete, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::takeoverFuse is in progress, EdenMount::unmount does not cause EdenMount::takeoverFuse to fail.
* If EdenMount::takeoverFuse was called, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::unmount was ever called, EdenMount::takeoverFuse succeeds.
* If EdenMount::unmount is called multiple times, one of the calls succeeds, and the remaining calls to unmount fail with an exception from PrivHelper.

Reviewed By: simpkins

Differential Revision: D14398423

fbshipit-source-id: eb86a237859ad2d992de5c901e63dda80d5ea6d5
2019-03-21 13:51:15 -07:00
Matt Glazar
f91ef8092c Synchronize mount+unmount 0/4: Add features to MountDelegate test helpers
Summary:
In future diffs, I will introduce some EdenMount tests which need to mock PrivHelper in more ways than are currently supported. For example, I want to mock PrivHelper::fuseUnmount and check how many times fuseUmount is called. Extend the existing MockMountDelegate and FakeFuseMountDelegate classes to support these to-be-written tests.

This diff should not change behavior.

Reviewed By: simpkins

Differential Revision: D14418206

fbshipit-source-id: a9b1257bb62d9a46fae0eef89c3fa42718bec142
2019-03-21 13:51:15 -07:00
Puneet Kaushik
d42b639e3e Use hg import subcommand on Windows
Reviewed By: chadaustin

Differential Revision: D14314008

fbshipit-source-id: edfb600d4a9fa921bbcb4204b9a0a87ef042c185
2019-03-20 21:50:50 -07:00
Puneet Kaushik
1315b73a59 Move Handle class inside Eden namespace.
Summary: Somehow, I missed adding the namespace to the Handle class - fixing it now.

Reviewed By: simpkins

Differential Revision: D14545345

fbshipit-source-id: b6bc8ae83703b04399a73dce3dd63a3aceae7331
2019-03-20 21:37:51 -07:00
Puneet Kaushik
4cdb779016 Added UserInfo for Windows
Reviewed By: strager

Differential Revision: D13360664

fbshipit-source-id: b234fc70e8487b1e38b6b0bdceabbd0bdecf5bb4
2019-03-20 14:16:50 -07:00
generatedunixname89002005289445
1f369ed67b Update pyre version for eden
Summary: Automatic upgrade to remove `version` override and silence errors.

Reviewed By: shannonzhu

Differential Revision: D14525100

fbshipit-source-id: ca27023b89da9eb8f983caaceb78e9fb4fea7bfe
2019-03-19 13:15:29 -07:00
Chad Austin
1a1dbccb1d add osxfuse kernel header
Summary:
Eden needs the osxfuse version of the FUSE protocol on mac. The open
source cmake build pulls in osxfuse with getdeps.py, but that doesn't
work in the Buck build, and all we need is this one (old) copy of the
Linux kernel headers.

Reviewed By: strager

Differential Revision: D14506747

fbshipit-source-id: 028ddbaf80be9cad412462f3338c30b8f2a70087
2019-03-19 10:26:24 -07:00
Matt Glazar
2921ce998d Report FUSE_ERROR if creating FuseChannel fails
Summary:
If EdenMount::takeoverFuse is called and createFuseChannel fails, the EdenMount reports that it's in the STARTING state. This is misleading and doesn't match how EdenMount::startFuse handles failures in createFuseChannel.

Extend the `try` block in EdenMount::takeoverFuse to handle exceptions raised from createFuseChannel.

Reviewed By: simpkins

Differential Revision: D14511564

fbshipit-source-id: 8f791efe1221bcfc1042f95054d181c303564086
2019-03-18 19:11:32 -07:00
Zeyi Fan
2b086dfe3e fix oss macOS build
Summary: This diff fixes Eden build on macOS.

Reviewed By: chadaustin

Differential Revision: D14507115

fbshipit-source-id: fdf460ebadc2e69b1cf3fc40a6fd3e104dbc2833
2019-03-18 14:12:57 -07:00
Adam Simpkins
981952749d attempt to repair the RocksDB if we fail to open it
Summary:
Update `RocksHandles` to call `RepairDB()` if we get an error opening the
database.

We have seen errors opening the DB in some cases after hard server reboots.
In every case so far `ldb repair` has been able to repair it with no adverse
effects.  This simply makes `edenfs` automatically attempt to perform this DB
repair step.

Reviewed By: chadaustin, strager

Differential Revision: D14452216

fbshipit-source-id: 10c0cb0ff9cea3c3bbe485a4e489e4a6df640803
2019-03-18 11:36:42 -07:00
Adam Simpkins
a328624182 generate a new version of the "basic" snapshot
Summary:
This checks in a new version of the "basic" snapshot file.

The change I am most interested in picking up is the recent change to make
sure that we always flush column family information in the RocksDB local
store.

I don't think there were too many other consequential changes to our on-disk
data structures since we last regenerated the snapshot in November.  The main
other change is that we no longer make the checkout mount point directories
read-only (D13515854).

Reviewed By: chadaustin

Differential Revision: D14452215

fbshipit-source-id: 87f1cdb61a91ed0f71333f2566ab6b5dac79f628
2019-03-18 11:36:42 -07:00
Adam Simpkins
9e1805c9e0 explicitly flush all column families when opening the RocksDB
Summary:
When opening the RocksDB, write one entry to each column family and then
explicitly flush that column family.  This ensures that the column family
information has actually been flushed to an SST file.  Without this some
column families may only have been written out to the write-ahead log files.
(Even calling `db->Flush()` does not appear to be sufficient; each column
family has to be explicitly flushed.)

The RocksDB' `RepairDB()` function (used by `ldb repair`) currently ends up
deleting column families that do not have any data defined in an SST file.
The repair tool ends up deleting column families that only have data in log
files.

The fact that we haven't been doing these explicit flushes previously probably
isn't too much of a concern in practice: once we write out enough data RocksDB
will automatically trigger a flush.  This only matters in cases where we have
not yet written out enough data to trigger an automatic flush.

Note that with this change we re-write these `id` keys each time we open the
RocksDB store, even if they were already present.

Reviewed By: chadaustin, strager

Differential Revision: D14452214

fbshipit-source-id: 3f1b17e240cc89fe00e3d31105d16452795e754d
2019-03-18 11:36:42 -07:00
Puneet Kaushik
44644df968 Use FileUtil APIs on Windows
Summary:
folly::readFile() doesn't work correctly on Windows. It was returning the incorrect size. I didn't check if the read contents were correct, because I was testing it on a binary data file.

Moreover we would need wide char API which won't be a part of Folly for sometime.

Reviewed By: strager

Differential Revision: D14205306

fbshipit-source-id: 55859dca1a6399d5b5f106b4c6757e582ba1375d
2019-03-18 10:00:32 -07:00
Puneet Kaushik
6471e739f4 Add Windows version of FileUtils
Reviewed By: strager

Differential Revision: D14205304

fbshipit-source-id: 964e6b5bf4c456d468b9624ab32ef104fa53c3a8
2019-03-18 10:00:32 -07:00
Adam Simpkins
84349add33 rename some of the files related to main()
Summary:
Rename `main.cpp` to `EdenMain.cpp` now that this contains the `EdenMain`
class rather than the top-level `main()` function, and rename `RunServer.cpp`
to `main.cpp`

Reviewed By: pkaush

Differential Revision: D14435306

fbshipit-source-id: e1528a773e0724c6bd50e31f6b33a1762d7bd49e
2019-03-15 18:13:58 -07:00
Adam Simpkins
be2474ee41 restructure main() link dependency ordering
Summary:
This restructures `main.cpp` to turn our current top-level `main()` into a
method of a virtual class.

This allows us to eliminate the slightly awkward way that `main.cpp` depends
on `RunServer.cpp`.  This required us to list `main.cpp` in the sources
parameter of multiple buck targets rules, which confuses some other tools like
autodeps.  With this new change, `RunServer.cpp` depends on `main.cpp` instead
of the other way around.  The real top-level `main()` function now lives in
`RunServer.cpp`

Reviewed By: pkaush

Differential Revision: D14435309

fbshipit-source-id: 402d00db0d8aa8d473d51a4f0e9d9d80c97a0134
2019-03-15 18:13:58 -07:00
Matt Glazar
d65aa6ad8b Infer XDG_RUNTIME_DIR if unset
Summary:
Sometimes, the XDG_RUNTIME_DIR environment variable isn't set. If this happens, 'eden start' fails because systemctl uses XDG_RUNTIME_DIR to talk to systemd. We still want 'eden start' to work in these cases, so guess what XDG_RUNTIME_DIR should be and use that guess if the variable isn't set.

If XDG_RUNTIME_DIR is set in the environment, its value should still be used.

Reviewed By: chadaustin

Differential Revision: D13811813

fbshipit-source-id: bb44d99e585bbe7a4341087c5cb4644c606fc441
2019-03-14 18:19:41 -07:00
Matt Glazar
65f80da385 Add USER_ID variable for config options
Summary:
The systemctl command requires XDG_RUNTIME_DIR to be configured. If it's not configured, 'eden start' should pick a sane default. The sane default includes the user's UID (e.g. /run/user/6986). I want this default to be configurable via Eden's config files.

Expose the ${USER_ID} token to Eden configs. This will let administrators can customize XDG_RUNTIME_DIR's fallback value in the future.

Reviewed By: wez

Differential Revision: D13811732

fbshipit-source-id: 7933e078dd5f2b3bbbb0299730220a129c257256
2019-03-14 18:19:41 -07:00
Matt Glazar
9cbf2baf9b Fix systemd tests on CI servers
Summary:
Sometimes, Facebook's CI servers might not have a /run/systemd directory. This causes EdenFS' systemd tests to fail, because daemon-respawn can't access that directory [1].

Fix the tests on CI by creating /run/systemd.

Why did the tests only start failing recently? I'm not sure. I think we were just lucky in the past; tests in other projects seem to create /run/systemd (e.g. using the systemd-nspawn command), and it looks like this state persists across CI jobs.

[1] https://github.com/systemd/systemd/blob/v239/src/core/dbus-manager.c#L1277

Reviewed By: simpkins

Differential Revision: D14436098

fbshipit-source-id: eb48abeb1ce38ea4ae760192db37bb1910efff99
2019-03-14 13:48:20 -07:00
Matt Glazar
5e3e446aa6 Fix error message in tests
Summary:
If sanity_check_enabled_unit_fragment fails, the error message is unhelpful:

```
Exception: Enabled unit's FragmentPath does not match unit file
Expected: {repr(expected_unit_file)}
Actual:   {repr(actual_unit_file)}
```

Use format strings as was originally intended, and drop repr to make the paths easier to read:

```
Exception: Enabled unit's FragmentPath does not match unit file
Expected: /data/users/strager/fbsource/fbcode/eden/fs/service/fb-edenfs@.service
Actual:   /usr/lib/systemd/user/fb-edenfs@.service
```

Reviewed By: simpkins

Differential Revision: D13372758

fbshipit-source-id: 0f12cc7a6f63fc53d72ce92b265e0ccbcc26d394
2019-03-14 13:41:17 -07:00
Adam Simpkins
d91561800e stop emitting error logs about flush() and fsyncdir()
Summary:
Implement `flush()` and `fsyncdir()` in `EdenDispatcher` and explicitly return
ENOSYS.  We intentionally do not need to do any work for these methods.
Providing our an implementation that returns ENOSYS avoids the error log
message in the default `Dispatcher` implementation about the these calls being
unimplemented.

Reviewed By: chadaustin

Differential Revision: D14453245

fbshipit-source-id: 71efe6de6af73a5d705dace0f3439ba2466a50a8
2019-03-14 13:07:09 -07:00
Adam Simpkins
0e9d62783d tweak error log messages for unhandled fuse opcodes.
Summary:
Update the FuseChannel code to explicitly handle `FUSE_LSEEK` and `FUSE_POLL`
to avoid emitting error log messages about these calls not being implemented.
We intentionally do not implement these operations at the moment.

Also drop the log about other unknown opcodes from an error log to a warning
message.

Reviewed By: chadaustin

Differential Revision: D14453246

fbshipit-source-id: 2605cf7e5c160cda92460a80187438ac3549ade5
2019-03-14 13:07:09 -07:00
Adam Simpkins
d529f75e9e tweak the log levels of some FUSE log messages
Summary:
I want to change some of the StartupLogger behavior to automatically show all
INFO and higher level log messages to the user during start-up.  These two
messages from the FUSE code are currently logged at the INFO level, but don't
really seem important enough to show to the user.

This drops their level to DBG1.  These messages will therefore still be shown
in the normal `edenfs.log` file, since it includes everything from DBG2 and
up.

Reviewed By: chadaustin

Differential Revision: D14453247

fbshipit-source-id: 5d79766f87e658b807029d82ac035cb94ae68832
2019-03-14 13:07:09 -07:00
Adam Simpkins
b96dd3e764 add @manual tags to Windows includes to make autodeps happy
Summary:
The Windows headers and source files currently aren't mentioned in any buck
build rule, which confuses autodeps.  Mark them all `manual` to tell
autodeps to simply ignore them.

Reviewed By: strager

Differential Revision: D14435307

fbshipit-source-id: a10b9be779f232cc5ae74356d7f54e77ca5414cb
2019-03-14 10:53:07 -07:00
Adam Simpkins
6e1f185084 fix some issues with curl library initialization and cleanup
Summary:
- Wait to call `curl_global_init()` until after we have dropped root
  privileges.  `curl_global_init()` performs a non-trivial amount of work,
  including loading and processing OpenSSL configuration files from disk.  To
  guard against any possible security issues in this code we should wait until
  after we have dropped root privileges to do this.

- Move the call to `curl_global_cleanup()` to after the main `EdenServer`
  object has been destroyed.  Since the EdenServer object will likely contain
  backing store objects that use curl it seems like we want to make sure we
  wait to clean up the curl library until after the `EdenServer` has been
  destroyed.  This change uses `SCOPE_EXIT` to perform the cleanup as one of
  the last steps of `main()`, and this also reduces the number of `#ifdef`
  blocks that we need.

Reviewed By: strager, fanzeyi

Differential Revision: D14435308

fbshipit-source-id: 6c277e4471f0f93decebd4fc741639c6a047d62a
2019-03-14 10:53:07 -07:00
Kostia Balytskyi
9cdaa234b6 chg: remove the ability to produce an actual binary
Summary: We don't run this binary anymore, no reason to build and ship it.

Reviewed By: quark-zju

Differential Revision: D14437317

fbshipit-source-id: dd6da521783f18a2a518a7aa042be98950894e89
2019-03-14 06:35:40 -07:00
Matt Glazar
52e6a3a364 Use SemiFuture for EdenMount::shutdown
Summary:
The future returned by EdenMount::shutdown() can continue on an arbitrary thread. This has caused at least one deadlock in the past (D14337058).

Prevent users of EdenMount::shutdown() from continuing on the wrong thread by making shutdown return a SemiFuture instead of a Future.

This diff should not change behavior for production code, since all users of EdenMount::shutdown() already use Future<>::via().

Reviewed By: simpkins

Differential Revision: D14389607

fbshipit-source-id: 821d206d01ea3dcb0261cd0c7ca2d591b2c84e7a
2019-03-12 19:29:35 -07:00
Matt Glazar
3b9a0310a1 Fix deadlock when restarting during RocksDbLocalStore::get()
Summary:
If TreeInode::startLoadingInode() is in progress, and EdenServer::startTakeoverShutdown() is called, edenfs can deadlock:

1. Thread A: A FUSE request calls TreeInode::readdir() -> TreeInode::prefetch() -> TreeInode::startLoadingInode() on the children TreeInode-s -> RocksDbLocalStore::getFuture().
2. Thread B: A takeover request calls EdenServer::performTakeoverShutdown() -> InodeMap::shutdown().
3. Thread C: RocksDbLocalStore::getFuture() (called in step 1) completes -> TreeInode::inodeLoadComplete(). (The inodeLoadComplete continuation was registered by TreeInode::registerInodeLoadComplete().)
4. Thread C: After TreeInode::inodeLoadComplete() returns, the TreeInode's InodePtr is destructed, dropping the reference count to 0.
5. Thread C: InodeMap::onInodeUnreferenced() -> InodeMap::shutdownComplete() -> EdenMount::shutdown() (called in step 2) completes -> EdenServer::performTakeoverShutdown().
6. Thread C: EdenServer::performTakeoverShutdown() -> localStore_.reset() -> RocksDbLocalStore::~RocksDbLocalStore().
7. Thread C: RocksDbLocalStore::~RocksDbLocalStore() signals the thread pool to exit and waits for the pool's threads to exit. Because thread C is one of the threads managed by RocksDbLocalStore's thread pool, the signal is never handled and RocksDbLocalStore::~RocksDbLocalStore() never finishes.

Fix this deadlock by executing EdenServer::shutdown()'s callback (in EdenServer::performTakeoverShutdown()) on a different thread.

Reviewed By: simpkins

Differential Revision: D14337058

fbshipit-source-id: 1d63b4e7d8f5103a2dde31e329150bf763be3db7
2019-03-12 19:29:35 -07:00
Puneet Kaushik
e7a40683c8 Created HResultError class and fixed the exception handling.
Reviewed By: strager

Differential Revision: D13289036

fbshipit-source-id: fa34f5a19916c06b11bafb5162dd4e9d9ebe4543
2019-03-12 16:30:44 -07:00
generatedunixname89002005289445
519fbabc9b Update pyre version for eden
Summary: Automatic upgrade to remove `version` override and silence errors.

Reviewed By: shannonzhu

Differential Revision: D14426843

fbshipit-source-id: bda3fa62d5af6d4c6c5120e793e0da79c4a145f0
2019-03-12 13:56:33 -07:00
Matt Glazar
201fef1c2e Create exception type for FUSE unmount-during-init
Summary:
FuseChannel::initialize throws runtime_error when the FUSE connection is interrupted. I want EdenMount::startFuse to handle unexpected-unmount errors from FuseChannel::initialize, but catching runtime_error would catch unrelated errors too.

When the FUSE connection is interrupted during initialization, make FuseChannel throw FuseDeviceUnmountedDuringInitialization instead of runtime_error.

Reviewed By: chadaustin

Differential Revision: D14077848

fbshipit-source-id: ed7b7d370a83ed1a9c36a443d8bb06ba940dc032
2019-03-11 20:48:27 -07:00
Eamonn Kent
e8a8830e72 Isolate low-level overlay logic into FsOverlay
Summary:
In this change, we separate the low-level code that manipulates the overlay
into the FsOverlay class. The Overlay class makes use of the FsOverlay and
InodeMetaData table to support its Overlay interfaces. The FsOverlay class
is decoupled from the Overlay class, allowing other classes to manipulate
the overlay independently. We have a need for this in order to add
fsck to the c++ code base : described in T40728883.

Reviewed By: simpkins

Differential Revision: D14218281

fbshipit-source-id: 66c587f2b341579b8075ca5e5eeb4da6ffadf6f5
2019-03-11 17:30:21 -07:00
Zeyi Fan
fa10788fce Reuse CURL* handle from curl_easy interface
Summary: Since `CurlClient` is going to be used in multiple threads and each thread has its own instance of CurlClient, we are now able to assign one curl_easy handle for each of the client. So curl would be able to reuse the existing connections to Mononoke API Server.

Reviewed By: chadaustin

Differential Revision: D14334486

fbshipit-source-id: aacd210773c44dea6f1d9dd7f9b09d765a9c7a0a
2019-03-11 14:34:09 -07:00
Zeyi Fan
16687decb2 Use thread factory for curl clients
Summary: Put `CurlClient` into its own thread pool (like how `HgBackingStore` uses `HgImporter`)

Reviewed By: chadaustin

Differential Revision: D14334483

fbshipit-source-id: 3ce2a8245761c11bfa0f66691d10a66004b46fe6
2019-03-11 14:34:09 -07:00
Zeyi Fan
199eb0ed42 Use MononokeCurlBackingStore in HgBackingStore
Summary: This diff enables `MononokeCurlBackingStore` in HgBackingStore. It also adds a new config value `mononoke:connection-type` (values: `http`, `curl` and `thrift`).

Reviewed By: chadaustin

Differential Revision: D14135028

fbshipit-source-id: cc6fb95d94b3d98996d872d22ad1a95a97d562ab
2019-03-11 14:34:09 -07:00
Zeyi Fan
21a6606e1f Add MononokeCurlBackingStore
Summary: This diff add a MononokeBackingStore implemented based on libcurl so we could add Mononoke support on macOS.

Reviewed By: chadaustin

Differential Revision: D14096313

fbshipit-source-id: 2bcb0c49002dcbb0a0190ba505a6e814e74db747
2019-03-11 14:34:08 -07:00
Zeyi Fan
d9f1beb1a1 add mononoke:connector config and create initializeHttpMononokeBackingStore
Summary:
This diff added a configurable value "mononoke:connector" to EdenConfig, so that we can specify a particular type of connection we need when using Mononoke as backing store.

This diff also moved the logic of initializing the existing `MononokeHttpBackingStore` to `initializeHttpMononokeBackingStore`.

Reviewed By: wez, strager

Differential Revision: D14141725

fbshipit-source-id: c7208295d7b3853740d7b0e5166f8854457fcf8e
2019-03-11 14:34:08 -07:00
Shrikrishna Khare
ec85a8b308 spawn thread to publish stats
Summary:
wedge_agent uses ThreadLocal stats update API to avoid the overhead of
acquiring lock for each stat update. Prior to this patch, wedge_agent used its
own StatsPublisher class which called publishStats every second. StatsPublisher
also asserted if startPublishThread is not running as StatsPublisher was doing
the stats publish.

However, there is no reason for wedge_agent to publish stats on its own.
Instead, the wedge_agent could use startPublishThread mechanism.

With D14255180, servicerouter/dispatcher now calls startPublishThread, which
started hitting in the aforementioned ASSERT in StatsPublisher.

This patch fixes it by having wedge_agent start the publish thread by calling
startPublishThread. Among other things, StatsPublisher also updated route stats
and port info. That functionality is moved to updateStats.

Differential Revision: D14388119

fbshipit-source-id: 503d08f12131317592be64e46243401d8a2e5bae
2019-03-09 10:14:46 -08:00
Jun Wu
1f30afc972 config: enable merge.printcandidatecommmits
Summary:
The feature was completed by Phil in D9816270. It's handy and can probably
reduce user support burden like: https://fb.intern.facebook.com/groups/scm/permalink/2039619916087618/

Therefore let's enable it.

Reviewed By: DurhamG

Differential Revision: D14293405

fbshipit-source-id: 54e934e0bf495c090109462e4f743d427df39380
2019-03-08 19:56:07 -08:00
Adam Simpkins
574c34ffbd fix "eden start --if-necessary" when using systemd
Summary:
Update `eden start` to honor the `--if-necessary` flag even when starting
with systemd.

Reviewed By: chadaustin, strager

Differential Revision: D14356541

fbshipit-source-id: b7c12b11a2fd63eeee84026bf4418702f794cbf0
2019-03-08 19:06:04 -08:00
Adam Simpkins
1b7f39fc56 convert most the CLI config code from str to Path
Summary:
Update most of the `eden/cli/config.py` to use `Path` instead of `str` where
appropriate.  This also updates several of the APIs in `util.py` that were
affected as well.

Reviewed By: chadaustin

Differential Revision: D14356543

fbshipit-source-id: a8f6d15b8870bf689eeb78f9fc0e9a0c65c97218
2019-03-08 19:06:04 -08:00
Chad Austin
eafd6107de add missing gflags headers
Summary:
Not sure why gflags is automatically included on Linux and not in
mode/mac, but add the includes nonetheless!

Reviewed By: strager

Differential Revision: D14389458

fbshipit-source-id: 27811ec4bb65b03e73b15bb51de1264dbfe053dc
2019-03-08 14:43:24 -08:00
Adam Simpkins
98ef256f4c remove CheckoutConfig.hooks_path
Summary:
Remove the `CheckoutConfig.hooks_path` attribute which is no longer used
anywhere.

Reviewed By: strager

Differential Revision: D14356542

fbshipit-source-id: 28b7e7aeb197aeb5d04edf8adfdcfe7946ffa0c7
2019-03-08 11:10:01 -08:00
Adam Simpkins
524bf4243d remove the old hooks configuration file
Summary:
This file has not been used D13447272, which moved creation of the `.hg`
directory from a separate hook into the CLI itself.  Remove it from the
packaging.

Reviewed By: strager

Differential Revision: D14356544

fbshipit-source-id: 18232c0e7e80e8cb255dd66f70b2a21b4abe6ca7
2019-03-08 11:10:01 -08:00