Commit Graph

2059 Commits

Author SHA1 Message Date
Adam Simpkins
b41bba16e4 fix the config for bad kernel versions
Summary:
Remove the incorrect regex for `^4.*_fbk13`.  From T34471852 this was
originally only intended to flag 4.11.3-52_fbk13 as bad.  It is incorrectly
flagging `4.16.18-151_fbk13` as a bad release.

This regex isn't needed at all since that release is older than the
`minimum-kernel-version` setting anyway.

It really seems like this config setting should probably just be a single
regex, and shouldn't be split on `,` since you can just use `|` in the regex
instead.  However that seems like a minor issue that should probably be
addressed in a separate diff.

Reviewed By: chadaustin

Differential Revision: D14815156

fbshipit-source-id: b00dd45cb212f6c3e33c02b6216c57d1510a233b
2019-04-05 17:08:04 -07:00
Stefan Filip
a2a0658ebe hg: update prefetch test with utf8 filenames
Summary:
We updating the encoding that mercurial uses for storing paths to UTF-8.
The current step is preventing files that have invalid names from being
committed. To that end we are updating the HgPrefetch test to submit
valid file names (valid UTF-8). This allows us to start using UTF-8
encoded string for the internal representation of file paths.

Doing encoding transformation between internal storage and system locale will
be implemented later. Normalization should also be handled at that point.

Reviewed By: simpkins

Differential Revision: D14812780

fbshipit-source-id: 1689910500391eed941df185ba92aa2d2f3f0960
2019-04-05 16:29:50 -07:00
Matt Glazar
d9e4eabc9d Move EdenStats into eden/fs/tracing/
Summary:
I want to use EdenStats in eden/fs/store/. EdenStats currently lives in eden/fs/fuse/, and making eden/fs/store/ depend upon eden/fs/fuse/ is confusing. (It's also confusing that some code in eden/fs/fuse/ is used on Windows.)

Reorganize the code: move EdenStats into eden/fs/tracing/.

This diff should not change behavior.

Reviewed By: chadaustin

Differential Revision: D14677337

fbshipit-source-id: af26d214bcc3a9919920fbd4e59e6098fe4e3834
2019-04-01 17:41:57 -07:00
Adam Simpkins
5eb2009088 drop the log level of a message in RocksDbLocalStore
Summary:
This message was added in D14337058.  It is logged at the `INFO` level, which
is enabled by default, but doesn't seem to add much value to normal production
logs.

Reviewed By: chadaustin

Differential Revision: D14712654

fbshipit-source-id: 5a86d883ace30e22d299046e33a6cd6247432857
2019-04-01 14:53:02 -07:00
Wez Furlong
3f307498ec eden: cmake: add missing folly dep for the rocksdb component
Summary:
we need to be explicit about pulling in the dep for the
case where folly is not installed into a default installation
prefix.

Reviewed By: strager, pkaush

Differential Revision: D14683955

fbshipit-source-id: 0f302877674fb744ef8076641cd3fa72de74efe4
2019-03-29 15:02:05 -07:00
Wez Furlong
fc2e7feec8 eden: cmake: probe for osxfuse headers
Summary:
the new getdeps script places the osxfuse/common dir into
its own install prefix, rather than dropping them into `external/osxfuse`.
This configures cmake to check in the installation prefixes known
to cmake.

Reviewed By: strager

Differential Revision: D14683956

fbshipit-source-id: 6e8a73341f8ddc21fef4b40b1f18a4a5128810e3
2019-03-29 15:02:04 -07:00
Wez Furlong
de7f624497 eden: cmake: fixup adding libgit2 options to target
Summary:
`target_link_libraries` only allows passing things that
are libraries and expressly forbids passing in `-framework Foo`,
which is the sort of thing we get back from pkg-config on macos.
The result of misusing this is that cmake would add `["-framework", "-lFoo"]`
to the argv for the linker, which is totally broken.

Instead, we should use `target_link_options`.

Unfortunately, cmake seems to fail to do the right thing with the
` -framework CoreFoundation -framework Security` flags returned
from libgit2 on my system even using `target_link_options`; it somehow
ends up with a bare `Security` and fails to link.  meh.

Reviewed By: strager

Differential Revision: D14680672

fbshipit-source-id: 62f65ddb4d07c8194cfc453cef1349b01be6c8b3
2019-03-29 15:02:04 -07:00
Wez Furlong
a13cc66d0f eden: add simple install rule for the cmake build
Summary:
This isn't a complete install but it is sufficient
to generate an `install` target for the getdeps build to run.

Reviewed By: strager

Differential Revision: D14680670

fbshipit-source-id: 9de1caa24c25702795842fe5b1b1f4d82aef24d8
2019-03-29 15:02:04 -07:00
Wez Furlong
f8099e5129 eden: cmake fixup include directories and deps
Summary:
While testing out the new getdeps code I found that none
of the include directories from the probed libraries were being used.

The new getdeps installs each dep into its own prefix, whereas the
existing getdeps script installed them into the installation
prefix for eden itself.  That meant that they were being implicitly
found from a single include directory.

In addition to this, I encountered linker failures for the pretty
printers; the solution to those was to add appropriate deps for
the modules that depend upon the pretty printers.

Reviewed By: pkaush

Differential Revision: D14638758

fbshipit-source-id: a4c2b4c79603c268e1b1c707a05c3cb0e3f2757b
2019-03-28 20:57:17 -07:00
Wez Furlong
df5f035b7f eden: fixup mononoke related opensource linux build
Summary:
This function depends on things that are not available
when EDEN_HAVE_HG_TREEMANIFEST is not enabled.

Ensure that the entire function is disabled if that is the case.

Reviewed By: chadaustin

Differential Revision: D14638759

fbshipit-source-id: 3fe83b7b42357c2b818469214b016ff591058e35
2019-03-28 20:57:17 -07:00
Chad Austin
e56081a893 fix static_assert in DirEntry on macOS
Summary: bits -> bytes

Reviewed By: wez

Differential Revision: D14673692

fbshipit-source-id: 739b5e8989d88a34e2c1bb49687d7d96b1422db4
2019-03-28 17:44:53 -07:00
Chad Austin
11175b7e85 set executable bit on force-unmount-all.sh
Summary: Must have forgotten this when originally added.

Reviewed By: wez

Differential Revision: D14672454

fbshipit-source-id: 1702054e92ae6c88e974fc7d32e49835cca4bbbc
2019-03-28 14:45:57 -07:00
Wez Furlong
0f376fc904 eden: workaround constexpr throw problem with gcc
Summary:
The issue is that the compiler needs an `else` to see
that we can only reach the throw if none of the other paths are
taken; with that satisfied it believes that we are legitimately
constexpr.

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67371

Reviewed By: chadaustin

Differential Revision: D14638234

fbshipit-source-id: f9524d2816580f41842a40e30118b03998c3660a
2019-03-27 12:47:46 -07:00
Fred Emmott
9f28060b95 Revert D13966702: [fbcode] AsyncServerSocket::AcceptCallback::connectionAccepted to NetworkSocket
Differential Revision:
D13966702

Original commit changeset: 415622dc347d

fbshipit-source-id: 11ff9cac08174cfaefe20e7e4c5e08dc005aaa39
2019-03-27 08:48:30 -07:00
Orvid King
730bc71c82 AsyncServerSocket::AcceptCallback::connectionAccepted to NetworkSocket
Summary: This is a largely automated codemod that shifts `folly::AsyncServerSocket::AcceptCallback::connectionAccepted` to taking a `NetworkSocket` rather than a file descriptor. This needs to be done as a single atomic change to avoid breaking things.

Reviewed By: yfeldblum

Differential Revision: D13966702

fbshipit-source-id: 415622dc347de53368c404dfbe9a2deae5b75e18
2019-03-26 15:09:39 -07:00
Zeyi Fan
31186d275d enable Mononoke thrift backing store
Summary: This diff enables thrift as a type of connection when talks to Mononoke API Server.

Reviewed By: strager, pkaush

Differential Revision: D14508956

fbshipit-source-id: 32e130c714be164f19452dec554976ea1cb594d3
2019-03-26 13:47:15 -07:00
Zeyi Fan
84b535fe3c Add MononokeThriftBackingStore
Summary: Implement a backing store that uses thrift protocol to communicate with Mononoke API Server.

Reviewed By: strager

Differential Revision: D13966575

fbshipit-source-id: 66f66dda6b17aecd6c6b4475ab6b004c608f457f
2019-03-26 13:47:15 -07:00
Matt Glazar
c1c72692a4 Improve messaging of disk space doctor check
Summary: When `eden doctor` reports that your machine is low on free disk space, people don't know what to do next and ask for help in support groups. Make the message site-configurable so we can guide users to fixing this problem on their own.

Reviewed By: chadaustin

Differential Revision: D14583025

fbshipit-source-id: 96c6ae3da3879a7a82501c22ebc2fa8ccc4097c2
2019-03-25 14:27:18 -07:00
Matt Glazar
003414bad9 Refactor DiskUsageTest ProblemTracker mocking
Summary:
DiskUsageTest mocks the eden.cli.doctor.ProblemFixer class. It could just as easily create its own subclass of ProblemTracker. Refactor the tests to use ProblemCollector, which adds problems into a list, instead of mocking ProblemFixer globally.

This diff should not change behavior.

Reviewed By: chadaustin

Differential Revision: D14589366

fbshipit-source-id: 0f0603779683ab5b7b78327cba065239ca2692e4
2019-03-25 14:27:18 -07:00
Chad Austin
c804232e52 opt into readdir caching if the kernel supports it but not FUSE_NO_OPENDIR_SUPPORT
Summary:
When Eden runs on kernel 4.20 (has readdir caching but not
FUSE_NO_OPENDIR_SUPPORT) indicate to the kernel that Eden wants
readdir results to be cached.

Reviewed By: wez

Differential Revision: D13893922

fbshipit-source-id: 3092adb16dabc4273bdba1e6e9f15e9e3bd7bb87
2019-03-22 15:57:33 -07:00
Chad Austin
cc1c841004 stop handling opendir and releasedir on kernels with FUSE_NO_OPENDIR_SUPPORT
Summary:
Eden requires no state in its directory handles, so tell the kernel it
doesn't need to send opendir() and releasedir() requests, provided it
has FUSE_NO_OPENDIR_SUPPORT.

Reviewed By: strager

Differential Revision: D13594734

fbshipit-source-id: ebd4b69f4efcd1428a69024c4bdffb1ae455fa40
2019-03-22 15:57:33 -07:00
Chad Austin
a9d9689d3d invalidate directory inodes properly on checkout
Summary:
After the kernel added readdir caching, my testing uncovered that Eden
was invalidating TreeInode entries incorrectly when new entries were
added. Change TreeInode to distinguish between directory entry changes
and removals (FUSE_NOTIFY_INVAL_ENTRY) and additions
(FUSE_NOTIFY_INVAL_INODE).

Reviewed By: strager

Differential Revision: D13870422

fbshipit-source-id: 2a6f25bfd9e77436a5aae639fedbfd8a445b2e05
2019-03-22 15:57:33 -07:00
Chad Austin
5a532b216c remove some unnecessary includes
Summary: Noticed these includes aren't necessary.

Reviewed By: simpkins

Differential Revision: D14574423

fbshipit-source-id: 672e886841c64312e54baf98ebe808c4c654815a
2019-03-21 20:51:55 -07:00
Matt Glazar
3d89f8b1e6 Refactor fuseMount SharedPromise into Promise
Summary:
Prior to D14398507, two calls to EdenMount::unmount would result in two calls to fuseMountPromise->getFuture(). D14398507 made two calls to EdenMount::unmount result in only one call to fuseMountPromise->getFuture().

Since fuseMountPromise->getFuture() is only called at most once, turn fuseMountPromise into a Promise instead of a SharedPromise.

Reviewed By: simpkins

Differential Revision: D14515070

fbshipit-source-id: 3ecaf9b55f3274046bc840bd5a65481cc552df72
2019-03-21 13:51:16 -07:00
Matt Glazar
7da8f80758 Synchronize mount+unmount 4/4: Make unmount idempotent
Summary:
I want to make EdenMount robust if mounting and unmounting are requested concurrently. This will make it safer to call EdenMount::destroy in different situations.

Take a step toward that goal by making EdenMount::unmount idempotent; calling EdenMount::unmount multiple times should return the same future.

Changed behaviors:

* If EdenMount::unmount is called multiple times, EdenMount::fuseMount is called at most once, and each call to EdenMount::unmount waits for the EdenMount::fuseMount call to finish.
  * **Old behavior**: One of the calls to unmount succeeds, and the remaining calls to unmount fail with an exception from PrivHelper.

Unchanged behaviors:

* If EdenMount::unmount was ever called, EdenMount::startFuse fails with EdenMountCancelled.
* If EdenMount::unmount was ever called, EdenMount::takeoverFuse fails with EdenMountCancelled.
* If EdenMount::startFuse is in progress, EdenMount::unmount causes the concurrent startFuse call to fail with FuseDeviceUnmountedDuringInitialization.
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount waits for fuseMount to finish before calling PrivHelper::fuseUnmount.
* If EdenMount::startFuse -> PrivHelper::fuseMount is complete, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::takeoverFuse is in progress, EdenMount::unmount does not cause EdenMount::takeoverFuse to fail.
* If EdenMount::takeoverFuse was called, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If neither EdenMount::startFuse nor EdenMount::takeoverFuse was never called, EdenMount::unmount succeeds and does not call PrivHelper::fuseUnmount.

Reviewed By: simpkins

Differential Revision: D14398507

fbshipit-source-id: 1da5424abc47341e4db3da1c50d425711e177366
2019-03-21 13:51:16 -07:00
Matt Glazar
7697a170be Synchronize mount+unmount 3/4: Make unmount succeed if not mounted
Summary:
I want to make EdenMount robust if mounting and unmounting are requested concurrently. This will make it safer to call EdenMount::destroy in different situations.

Take a step toward that goal by making EdenMount::unmount succeed if EdenMount::startFuse was never called.

Changed behaviors:

* If neither EdenMount::startFuse nor EdenMount::takeoverFuse was never called, EdenMount::unmount succeeds and does not call PrivHelper::fuseUnmount.
  * **Old behavior**: EdenMount::unmount fails with an exception from PrivHelper.

Unchanged behaviors:

* If EdenMount::unmount was ever called, EdenMount::startFuse fails with EdenMountCancelled.
* If EdenMount::unmount was ever called, EdenMount::takeoverFuse fails with EdenMountCancelled.
* If EdenMount::startFuse is in progress, EdenMount::unmount causes the concurrent startFuse call to fail with FuseDeviceUnmountedDuringInitialization.
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount waits for fuseMount to finish before calling PrivHelper::fuseUnmount.
* If EdenMount::startFuse -> PrivHelper::fuseMount is complete, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::takeoverFuse is in progress, EdenMount::unmount does not cause EdenMount::takeoverFuse to fail.
* if EdenMount::takeoverFuse was called, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::unmount is called multiple times, one of the calls succeeds, and the remaining calls to unmount fail with an exception from PrivHelper.

Reviewed By: simpkins

Differential Revision: D14398537

fbshipit-source-id: 9209aad8b2980045fd0f89a5ba149a833579c2cf
2019-03-21 13:51:15 -07:00
Matt Glazar
f0a6413df7 Synchronize mount+unmount 2/4: Make unmount wait for pending mount
Summary:
I want to make EdenMount robust if mounting and unmounting are requested concurrently. This will make it safer to call EdenMount::destroy in different situations.

Take a step toward that goal by making EdenMount::unmount complete only after a concurrent PrivHelper::fuseMount call finishes.

Changed behaviors:

* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount waits for fuseMount to finish before calling PrivHelper::fuseUnmount.
  * In other words, if EdenMount::startFuse and EdenMount::unmount execute concurrently, either:
    * mount(2) is never called (if EdenMount::unmount won the race), or
    * mount(2) is called then umount(2) is called (if EdenMount::startFuse won the race).
  * **Old behavior**: EdenMount::unmount calls PrivHelper::fuseUnmount immediately and either fails or succeeds.

Unchanged behaviors:

* If EdenMount::unmount was ever called, EdenMount::startFuse fails with EdenMountCancelled.
* If EdenMount::unmount was ever called, EdenMount::takeoverFuse fails with EdenMountCancelled.
* If EdenMount::startFuse is in progress, EdenMount::unmount causes the concurrent startFuse call to fail with FuseDeviceUnmountedDuringInitialization.
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount causes the fuseMount to be rolled back (by calling PrivHelper::fuseUnmount) after fuseMount completes.
* If EdenMount::startFuse -> PrivHelper::fuseMount is complete, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* if EdenMount::takeoverFuse was called, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::unmount is called multiple times, one of the calls succeeds, and the remaining calls to unmount fail with an exception from PrivHelper.

Reviewed By: simpkins

Differential Revision: D14398508

fbshipit-source-id: 6f01ce9e36b5802045acf54afe756b80fe102b1e
2019-03-21 13:51:15 -07:00
Matt Glazar
a3c26e2ae5 Synchronize mount+unmount 1.5/4: Make unmount cancel takeoverFuse
Summary:
D14398423 made EdenMount::startFuse fail immediately if EdenMount::unmount was ever called. EdenMount::takeoverFuse behaves differently; any prior EdenMount::unmount calls don't cause EdenMount::takeoverFuse to fail.

For consistency, make EdenMount::takeoverFuse fail in the same situations that make EdenMount::startFuse fail.

Changed behaviors:

* If EdenMount::unmount was ever called, EdenMount::takeoverFuse fails with EdenMountCancelled.
  * **Old behavior**: takeoverFuse succeeds.
  * **Note**: EdenMount::unmount still fails with an exception from PrivHelper.

Unchanged behaviors:

* If EdenMount::unmount was ever called, EdenMount::startFuse fails with EdenMountCancelled.
* If EdenMount::startFuse is in progress, EdenMount::unmount causes the concurrent startFuse call to fail with FuseDeviceUnmountedDuringInitialization.
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount causes the fuseMount to be rolled back (by calling PrivHelper::fuseUnmount) after fuseMount completes.
* If EdenMount::startFuse -> PrivHelper::fuseMount is complete, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::takeoverFuse is in progress, EdenMount::unmount does not cause EdenMount::takeoverFuse to fail.
* if EdenMount::takeoverFuse was called, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::unmount is called multiple times, one of the calls succeeds, and the remaining calls to unmount fail with an exception from PrivHelper.

Reviewed By: simpkins

Differential Revision: D14511630

fbshipit-source-id: b77ca5ee39c888e2d50547f76977db7aa5a0331a
2019-03-21 13:51:15 -07:00
Matt Glazar
142415554e Synchronize mount+unmount 1/4: Make unmount cancel mounting
Summary:
I want to make EdenMount robust if mounting and unmounting are requested concurrently. This will make it safer to call EdenMount::destroy in different situations.

Take a step toward that goal by making EdenMount::unmount cancel a concurrent EdenMount::startFuse call.

Changed behaviors:

* If EdenMount::startFuse is in progress, EdenMount::unmount causes the concurrent startFuse call to fail with FuseDeviceUnmountedDuringInitialization.
  * **Old behavior**: startFuse might succeed or might fail with FuseDeviceUnmountedDuringInitialization.
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount causes the fuseMount to be rolled back (by calling PrivHelper::fuseUnmount) after fuseMount completes.
  * **Old behavior**: EdenMount::unmount does not change the behavior of EdenMount::startFuse.
  * **Note**: EdenMount::unmount still fails with an exception from PrivHelper, despite having an effect on EdenMount::startFuse. This quirk will be addressed by a future diff.
* If EdenMount::unmount was ever called, EdenMount::startFuse fails with EdenMountCancelled.
  * **Old behavior**: startFuse succeeds.
  * **Note**: EdenMount::unmount still fails with an exception from PrivHelper, despite having an effect on EdenMount::startFuse. This quirk will be addressed by a future diff.

Unchanged behaviors:

* If neither EdenMount::startFuse nor EdenMount::takeoverFuse was never called, EdenMount::unmount fails with an exception from PrivHelper.
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount calls PrivHelper::fuseUnmount immediately and either fails or succeeds.
* If EdenMount::startFuse -> PrivHelper::fuseMount is complete, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::takeoverFuse is in progress, EdenMount::unmount does not cause EdenMount::takeoverFuse to fail.
* If EdenMount::takeoverFuse was called, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::unmount was ever called, EdenMount::takeoverFuse succeeds.
* If EdenMount::unmount is called multiple times, one of the calls succeeds, and the remaining calls to unmount fail with an exception from PrivHelper.

Reviewed By: simpkins

Differential Revision: D14398423

fbshipit-source-id: eb86a237859ad2d992de5c901e63dda80d5ea6d5
2019-03-21 13:51:15 -07:00
Matt Glazar
f91ef8092c Synchronize mount+unmount 0/4: Add features to MountDelegate test helpers
Summary:
In future diffs, I will introduce some EdenMount tests which need to mock PrivHelper in more ways than are currently supported. For example, I want to mock PrivHelper::fuseUnmount and check how many times fuseUmount is called. Extend the existing MockMountDelegate and FakeFuseMountDelegate classes to support these to-be-written tests.

This diff should not change behavior.

Reviewed By: simpkins

Differential Revision: D14418206

fbshipit-source-id: a9b1257bb62d9a46fae0eef89c3fa42718bec142
2019-03-21 13:51:15 -07:00
Puneet Kaushik
d42b639e3e Use hg import subcommand on Windows
Reviewed By: chadaustin

Differential Revision: D14314008

fbshipit-source-id: edfb600d4a9fa921bbcb4204b9a0a87ef042c185
2019-03-20 21:50:50 -07:00
Puneet Kaushik
1315b73a59 Move Handle class inside Eden namespace.
Summary: Somehow, I missed adding the namespace to the Handle class - fixing it now.

Reviewed By: simpkins

Differential Revision: D14545345

fbshipit-source-id: b6bc8ae83703b04399a73dce3dd63a3aceae7331
2019-03-20 21:37:51 -07:00
Puneet Kaushik
4cdb779016 Added UserInfo for Windows
Reviewed By: strager

Differential Revision: D13360664

fbshipit-source-id: b234fc70e8487b1e38b6b0bdceabbd0bdecf5bb4
2019-03-20 14:16:50 -07:00
generatedunixname89002005289445
1f369ed67b Update pyre version for eden
Summary: Automatic upgrade to remove `version` override and silence errors.

Reviewed By: shannonzhu

Differential Revision: D14525100

fbshipit-source-id: ca27023b89da9eb8f983caaceb78e9fb4fea7bfe
2019-03-19 13:15:29 -07:00
Chad Austin
1a1dbccb1d add osxfuse kernel header
Summary:
Eden needs the osxfuse version of the FUSE protocol on mac. The open
source cmake build pulls in osxfuse with getdeps.py, but that doesn't
work in the Buck build, and all we need is this one (old) copy of the
Linux kernel headers.

Reviewed By: strager

Differential Revision: D14506747

fbshipit-source-id: 028ddbaf80be9cad412462f3338c30b8f2a70087
2019-03-19 10:26:24 -07:00
Matt Glazar
2921ce998d Report FUSE_ERROR if creating FuseChannel fails
Summary:
If EdenMount::takeoverFuse is called and createFuseChannel fails, the EdenMount reports that it's in the STARTING state. This is misleading and doesn't match how EdenMount::startFuse handles failures in createFuseChannel.

Extend the `try` block in EdenMount::takeoverFuse to handle exceptions raised from createFuseChannel.

Reviewed By: simpkins

Differential Revision: D14511564

fbshipit-source-id: 8f791efe1221bcfc1042f95054d181c303564086
2019-03-18 19:11:32 -07:00
Zeyi Fan
2b086dfe3e fix oss macOS build
Summary: This diff fixes Eden build on macOS.

Reviewed By: chadaustin

Differential Revision: D14507115

fbshipit-source-id: fdf460ebadc2e69b1cf3fc40a6fd3e104dbc2833
2019-03-18 14:12:57 -07:00
Adam Simpkins
981952749d attempt to repair the RocksDB if we fail to open it
Summary:
Update `RocksHandles` to call `RepairDB()` if we get an error opening the
database.

We have seen errors opening the DB in some cases after hard server reboots.
In every case so far `ldb repair` has been able to repair it with no adverse
effects.  This simply makes `edenfs` automatically attempt to perform this DB
repair step.

Reviewed By: chadaustin, strager

Differential Revision: D14452216

fbshipit-source-id: 10c0cb0ff9cea3c3bbe485a4e489e4a6df640803
2019-03-18 11:36:42 -07:00
Adam Simpkins
a328624182 generate a new version of the "basic" snapshot
Summary:
This checks in a new version of the "basic" snapshot file.

The change I am most interested in picking up is the recent change to make
sure that we always flush column family information in the RocksDB local
store.

I don't think there were too many other consequential changes to our on-disk
data structures since we last regenerated the snapshot in November.  The main
other change is that we no longer make the checkout mount point directories
read-only (D13515854).

Reviewed By: chadaustin

Differential Revision: D14452215

fbshipit-source-id: 87f1cdb61a91ed0f71333f2566ab6b5dac79f628
2019-03-18 11:36:42 -07:00
Adam Simpkins
9e1805c9e0 explicitly flush all column families when opening the RocksDB
Summary:
When opening the RocksDB, write one entry to each column family and then
explicitly flush that column family.  This ensures that the column family
information has actually been flushed to an SST file.  Without this some
column families may only have been written out to the write-ahead log files.
(Even calling `db->Flush()` does not appear to be sufficient; each column
family has to be explicitly flushed.)

The RocksDB' `RepairDB()` function (used by `ldb repair`) currently ends up
deleting column families that do not have any data defined in an SST file.
The repair tool ends up deleting column families that only have data in log
files.

The fact that we haven't been doing these explicit flushes previously probably
isn't too much of a concern in practice: once we write out enough data RocksDB
will automatically trigger a flush.  This only matters in cases where we have
not yet written out enough data to trigger an automatic flush.

Note that with this change we re-write these `id` keys each time we open the
RocksDB store, even if they were already present.

Reviewed By: chadaustin, strager

Differential Revision: D14452214

fbshipit-source-id: 3f1b17e240cc89fe00e3d31105d16452795e754d
2019-03-18 11:36:42 -07:00
Puneet Kaushik
44644df968 Use FileUtil APIs on Windows
Summary:
folly::readFile() doesn't work correctly on Windows. It was returning the incorrect size. I didn't check if the read contents were correct, because I was testing it on a binary data file.

Moreover we would need wide char API which won't be a part of Folly for sometime.

Reviewed By: strager

Differential Revision: D14205306

fbshipit-source-id: 55859dca1a6399d5b5f106b4c6757e582ba1375d
2019-03-18 10:00:32 -07:00
Puneet Kaushik
6471e739f4 Add Windows version of FileUtils
Reviewed By: strager

Differential Revision: D14205304

fbshipit-source-id: 964e6b5bf4c456d468b9624ab32ef104fa53c3a8
2019-03-18 10:00:32 -07:00
Adam Simpkins
84349add33 rename some of the files related to main()
Summary:
Rename `main.cpp` to `EdenMain.cpp` now that this contains the `EdenMain`
class rather than the top-level `main()` function, and rename `RunServer.cpp`
to `main.cpp`

Reviewed By: pkaush

Differential Revision: D14435306

fbshipit-source-id: e1528a773e0724c6bd50e31f6b33a1762d7bd49e
2019-03-15 18:13:58 -07:00
Adam Simpkins
be2474ee41 restructure main() link dependency ordering
Summary:
This restructures `main.cpp` to turn our current top-level `main()` into a
method of a virtual class.

This allows us to eliminate the slightly awkward way that `main.cpp` depends
on `RunServer.cpp`.  This required us to list `main.cpp` in the sources
parameter of multiple buck targets rules, which confuses some other tools like
autodeps.  With this new change, `RunServer.cpp` depends on `main.cpp` instead
of the other way around.  The real top-level `main()` function now lives in
`RunServer.cpp`

Reviewed By: pkaush

Differential Revision: D14435309

fbshipit-source-id: 402d00db0d8aa8d473d51a4f0e9d9d80c97a0134
2019-03-15 18:13:58 -07:00
Matt Glazar
d65aa6ad8b Infer XDG_RUNTIME_DIR if unset
Summary:
Sometimes, the XDG_RUNTIME_DIR environment variable isn't set. If this happens, 'eden start' fails because systemctl uses XDG_RUNTIME_DIR to talk to systemd. We still want 'eden start' to work in these cases, so guess what XDG_RUNTIME_DIR should be and use that guess if the variable isn't set.

If XDG_RUNTIME_DIR is set in the environment, its value should still be used.

Reviewed By: chadaustin

Differential Revision: D13811813

fbshipit-source-id: bb44d99e585bbe7a4341087c5cb4644c606fc441
2019-03-14 18:19:41 -07:00
Matt Glazar
65f80da385 Add USER_ID variable for config options
Summary:
The systemctl command requires XDG_RUNTIME_DIR to be configured. If it's not configured, 'eden start' should pick a sane default. The sane default includes the user's UID (e.g. /run/user/6986). I want this default to be configurable via Eden's config files.

Expose the ${USER_ID} token to Eden configs. This will let administrators can customize XDG_RUNTIME_DIR's fallback value in the future.

Reviewed By: wez

Differential Revision: D13811732

fbshipit-source-id: 7933e078dd5f2b3bbbb0299730220a129c257256
2019-03-14 18:19:41 -07:00
Matt Glazar
9cbf2baf9b Fix systemd tests on CI servers
Summary:
Sometimes, Facebook's CI servers might not have a /run/systemd directory. This causes EdenFS' systemd tests to fail, because daemon-respawn can't access that directory [1].

Fix the tests on CI by creating /run/systemd.

Why did the tests only start failing recently? I'm not sure. I think we were just lucky in the past; tests in other projects seem to create /run/systemd (e.g. using the systemd-nspawn command), and it looks like this state persists across CI jobs.

[1] https://github.com/systemd/systemd/blob/v239/src/core/dbus-manager.c#L1277

Reviewed By: simpkins

Differential Revision: D14436098

fbshipit-source-id: eb48abeb1ce38ea4ae760192db37bb1910efff99
2019-03-14 13:48:20 -07:00
Matt Glazar
5e3e446aa6 Fix error message in tests
Summary:
If sanity_check_enabled_unit_fragment fails, the error message is unhelpful:

```
Exception: Enabled unit's FragmentPath does not match unit file
Expected: {repr(expected_unit_file)}
Actual:   {repr(actual_unit_file)}
```

Use format strings as was originally intended, and drop repr to make the paths easier to read:

```
Exception: Enabled unit's FragmentPath does not match unit file
Expected: /data/users/strager/fbsource/fbcode/eden/fs/service/fb-edenfs@.service
Actual:   /usr/lib/systemd/user/fb-edenfs@.service
```

Reviewed By: simpkins

Differential Revision: D13372758

fbshipit-source-id: 0f12cc7a6f63fc53d72ce92b265e0ccbcc26d394
2019-03-14 13:41:17 -07:00
Adam Simpkins
d91561800e stop emitting error logs about flush() and fsyncdir()
Summary:
Implement `flush()` and `fsyncdir()` in `EdenDispatcher` and explicitly return
ENOSYS.  We intentionally do not need to do any work for these methods.
Providing our an implementation that returns ENOSYS avoids the error log
message in the default `Dispatcher` implementation about the these calls being
unimplemented.

Reviewed By: chadaustin

Differential Revision: D14453245

fbshipit-source-id: 71efe6de6af73a5d705dace0f3439ba2466a50a8
2019-03-14 13:07:09 -07:00
Adam Simpkins
0e9d62783d tweak error log messages for unhandled fuse opcodes.
Summary:
Update the FuseChannel code to explicitly handle `FUSE_LSEEK` and `FUSE_POLL`
to avoid emitting error log messages about these calls not being implemented.
We intentionally do not implement these operations at the moment.

Also drop the log about other unknown opcodes from an error log to a warning
message.

Reviewed By: chadaustin

Differential Revision: D14453246

fbshipit-source-id: 2605cf7e5c160cda92460a80187438ac3549ade5
2019-03-14 13:07:09 -07:00