Summary: This diff makes `MononokeHttpBackingStore` uses `ServiceAddress` to manage remote address of the Mononoke API Server.
Reviewed By: chadaustin
Differential Revision: D14855661
fbshipit-source-id: 1c102f7b2c547dc6a5b5c2cba65a513313d90bdc
Summary: This diff adds `ServiceAddress` to utils. This is a class that encapsulates an address of a service that could be a traditional hostname & port pair as well as an SMC tier name. This eliminates the needs of the manually resolving network address for remote services.
Reviewed By: chadaustin
Differential Revision: D14845257
fbshipit-source-id: 9fe9847cca4bba0170be94b9c209247342708574
Summary:
Sometimes, EdenFS goes bonkers and talks to 'hg debugedenimporthelper' a lot for seemingly no reason.
Make these situations easier to debug by counting how many requests EdenFS makes to 'hg debugedenimporthelper'. These counters let us answer questions such as the following:
* When performance sucks, is EdenFS is making a lot of requests or only a few requests? I.e. is Hg slow at responding or is EdenFS very demanding?
* Does a recent performance issue correlate with EdenFS communicating with 'hg debugedenimporthelper'?
* Which engineers are outliers having orders of magnitude more 'hg debugedenimporthelper' requests than p50 engineers?
We could get fancier with these counters and include the number of bytes received, the duration of the request, etc. For now, just having a request count is useful.
Reviewed By: simpkins
Differential Revision: D14677339
fbshipit-source-id: 7f8f394fb0096aef65d6a8a45d7da5936db539a0
Summary:
D8559702 changed `folly::IOBuf::computeChainDataLength()` to return a `size_t`
Update our format specifier to match to avoid compiler warnings on Mac.
Reviewed By: chadaustin
Differential Revision: D14878220
fbshipit-source-id: 19e96bea07c57bb542a848b3688d65143db51d13
Summary:
The file descriptor API here needs to go away, so switch this API to NetworkSocket
It is expected that this commit will cause a number of Open Source projects to temporarily show up as broken. This is due to the fact that not all projects get synced to Github at the exact same time, so the builds may temporarily be fetching an older version of it's dependencies than it needs to :) It should fix itself quickly.
Reviewed By: yfeldblum
Differential Revision: D14673328
fbshipit-source-id: c5842fa5dc383d50043e0d8228e35d03b10a1c6b
Summary:
Ensure that the `TreeInode::createImpl()` code always releases the contents
lock at the end of the scope that was intended to signify the critical
section. Previously this was only unlocked when returning from this scope
successfully.
In particular, the old behavior could deadlock:
- We first created the new child inode.
- If saveOverlayDir() failed, we would throw an exception, causing us to not
release the directory lock yet.
- As we returned in the exception case, the local `inode` variable would be
destroyed before the `contents` argument, causing us to release the child
InodePtr's refcount before releasing the parent directory's lock. Because
we were the only user of the child inode we would end up needing to acquire
the parent inode's content's lock to check if it was unlinked.
This fixes the code to ensure that we always release the tree's contents lock
before the child inode becomes unreferenced.
Reviewed By: chadaustin
Differential Revision: D14848807
fbshipit-source-id: 28dcde842072b9ee1e7c63d54456d849e31af8fe
Summary: My editor configuration automatically format on save and I don't want to mix formatting with other meaningful changes.
Reviewed By: simpkins, strager
Differential Revision: D14859060
fbshipit-source-id: 8c906096b5c98a7f680bd2aeac9a7a5133da1ebc
Summary: Move Mononoke backing store initialization process to every request so we can adjust the configuration value without restarting the process.
Reviewed By: strager
Differential Revision: D14583357
fbshipit-source-id: 49ce2736229ce3062d34337757ebda6bb6eae16a
Summary: This is a helper function to help lazily initializing Mononoke backing stores only when necessary and release it when it is not used any more. See next diff in the stack for detail usage.
Reviewed By: chadaustin
Differential Revision: D14791528
fbshipit-source-id: c26811bc5c7aebcd02f704f10ad19bc35f8b9a21
Summary: The data structure is being modifed from multiple threads and should be synchronized.
Reviewed By: chadaustin
Differential Revision: D14712053
fbshipit-source-id: 25bbd45695895e6d85a6cc0c0d5321f9ef914d3f
Summary:
Remove the incorrect regex for `^4.*_fbk13`. From T34471852 this was
originally only intended to flag 4.11.3-52_fbk13 as bad. It is incorrectly
flagging `4.16.18-151_fbk13` as a bad release.
This regex isn't needed at all since that release is older than the
`minimum-kernel-version` setting anyway.
It really seems like this config setting should probably just be a single
regex, and shouldn't be split on `,` since you can just use `|` in the regex
instead. However that seems like a minor issue that should probably be
addressed in a separate diff.
Reviewed By: chadaustin
Differential Revision: D14815156
fbshipit-source-id: b00dd45cb212f6c3e33c02b6216c57d1510a233b
Summary:
We updating the encoding that mercurial uses for storing paths to UTF-8.
The current step is preventing files that have invalid names from being
committed. To that end we are updating the HgPrefetch test to submit
valid file names (valid UTF-8). This allows us to start using UTF-8
encoded string for the internal representation of file paths.
Doing encoding transformation between internal storage and system locale will
be implemented later. Normalization should also be handled at that point.
Reviewed By: simpkins
Differential Revision: D14812780
fbshipit-source-id: 1689910500391eed941df185ba92aa2d2f3f0960
Summary:
I want to use EdenStats in eden/fs/store/. EdenStats currently lives in eden/fs/fuse/, and making eden/fs/store/ depend upon eden/fs/fuse/ is confusing. (It's also confusing that some code in eden/fs/fuse/ is used on Windows.)
Reorganize the code: move EdenStats into eden/fs/tracing/.
This diff should not change behavior.
Reviewed By: chadaustin
Differential Revision: D14677337
fbshipit-source-id: af26d214bcc3a9919920fbd4e59e6098fe4e3834
Summary:
This message was added in D14337058. It is logged at the `INFO` level, which
is enabled by default, but doesn't seem to add much value to normal production
logs.
Reviewed By: chadaustin
Differential Revision: D14712654
fbshipit-source-id: 5a86d883ace30e22d299046e33a6cd6247432857
Summary:
we need to be explicit about pulling in the dep for the
case where folly is not installed into a default installation
prefix.
Reviewed By: strager, pkaush
Differential Revision: D14683955
fbshipit-source-id: 0f302877674fb744ef8076641cd3fa72de74efe4
Summary:
the new getdeps script places the osxfuse/common dir into
its own install prefix, rather than dropping them into `external/osxfuse`.
This configures cmake to check in the installation prefixes known
to cmake.
Reviewed By: strager
Differential Revision: D14683956
fbshipit-source-id: 6e8a73341f8ddc21fef4b40b1f18a4a5128810e3
Summary:
`target_link_libraries` only allows passing things that
are libraries and expressly forbids passing in `-framework Foo`,
which is the sort of thing we get back from pkg-config on macos.
The result of misusing this is that cmake would add `["-framework", "-lFoo"]`
to the argv for the linker, which is totally broken.
Instead, we should use `target_link_options`.
Unfortunately, cmake seems to fail to do the right thing with the
` -framework CoreFoundation -framework Security` flags returned
from libgit2 on my system even using `target_link_options`; it somehow
ends up with a bare `Security` and fails to link. meh.
Reviewed By: strager
Differential Revision: D14680672
fbshipit-source-id: 62f65ddb4d07c8194cfc453cef1349b01be6c8b3
Summary:
This isn't a complete install but it is sufficient
to generate an `install` target for the getdeps build to run.
Reviewed By: strager
Differential Revision: D14680670
fbshipit-source-id: 9de1caa24c25702795842fe5b1b1f4d82aef24d8
Summary:
While testing out the new getdeps code I found that none
of the include directories from the probed libraries were being used.
The new getdeps installs each dep into its own prefix, whereas the
existing getdeps script installed them into the installation
prefix for eden itself. That meant that they were being implicitly
found from a single include directory.
In addition to this, I encountered linker failures for the pretty
printers; the solution to those was to add appropriate deps for
the modules that depend upon the pretty printers.
Reviewed By: pkaush
Differential Revision: D14638758
fbshipit-source-id: a4c2b4c79603c268e1b1c707a05c3cb0e3f2757b
Summary:
This function depends on things that are not available
when EDEN_HAVE_HG_TREEMANIFEST is not enabled.
Ensure that the entire function is disabled if that is the case.
Reviewed By: chadaustin
Differential Revision: D14638759
fbshipit-source-id: 3fe83b7b42357c2b818469214b016ff591058e35
Summary: Must have forgotten this when originally added.
Reviewed By: wez
Differential Revision: D14672454
fbshipit-source-id: 1702054e92ae6c88e974fc7d32e49835cca4bbbc
Summary:
The issue is that the compiler needs an `else` to see
that we can only reach the throw if none of the other paths are
taken; with that satisfied it believes that we are legitimately
constexpr.
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67371
Reviewed By: chadaustin
Differential Revision: D14638234
fbshipit-source-id: f9524d2816580f41842a40e30118b03998c3660a
Summary: This is a largely automated codemod that shifts `folly::AsyncServerSocket::AcceptCallback::connectionAccepted` to taking a `NetworkSocket` rather than a file descriptor. This needs to be done as a single atomic change to avoid breaking things.
Reviewed By: yfeldblum
Differential Revision: D13966702
fbshipit-source-id: 415622dc347de53368c404dfbe9a2deae5b75e18
Summary: This diff enables thrift as a type of connection when talks to Mononoke API Server.
Reviewed By: strager, pkaush
Differential Revision: D14508956
fbshipit-source-id: 32e130c714be164f19452dec554976ea1cb594d3
Summary: Implement a backing store that uses thrift protocol to communicate with Mononoke API Server.
Reviewed By: strager
Differential Revision: D13966575
fbshipit-source-id: 66f66dda6b17aecd6c6b4475ab6b004c608f457f
Summary: When `eden doctor` reports that your machine is low on free disk space, people don't know what to do next and ask for help in support groups. Make the message site-configurable so we can guide users to fixing this problem on their own.
Reviewed By: chadaustin
Differential Revision: D14583025
fbshipit-source-id: 96c6ae3da3879a7a82501c22ebc2fa8ccc4097c2
Summary:
DiskUsageTest mocks the eden.cli.doctor.ProblemFixer class. It could just as easily create its own subclass of ProblemTracker. Refactor the tests to use ProblemCollector, which adds problems into a list, instead of mocking ProblemFixer globally.
This diff should not change behavior.
Reviewed By: chadaustin
Differential Revision: D14589366
fbshipit-source-id: 0f0603779683ab5b7b78327cba065239ca2692e4
Summary:
When Eden runs on kernel 4.20 (has readdir caching but not
FUSE_NO_OPENDIR_SUPPORT) indicate to the kernel that Eden wants
readdir results to be cached.
Reviewed By: wez
Differential Revision: D13893922
fbshipit-source-id: 3092adb16dabc4273bdba1e6e9f15e9e3bd7bb87
Summary:
Eden requires no state in its directory handles, so tell the kernel it
doesn't need to send opendir() and releasedir() requests, provided it
has FUSE_NO_OPENDIR_SUPPORT.
Reviewed By: strager
Differential Revision: D13594734
fbshipit-source-id: ebd4b69f4efcd1428a69024c4bdffb1ae455fa40
Summary:
After the kernel added readdir caching, my testing uncovered that Eden
was invalidating TreeInode entries incorrectly when new entries were
added. Change TreeInode to distinguish between directory entry changes
and removals (FUSE_NOTIFY_INVAL_ENTRY) and additions
(FUSE_NOTIFY_INVAL_INODE).
Reviewed By: strager
Differential Revision: D13870422
fbshipit-source-id: 2a6f25bfd9e77436a5aae639fedbfd8a445b2e05
Summary:
Prior to D14398507, two calls to EdenMount::unmount would result in two calls to fuseMountPromise->getFuture(). D14398507 made two calls to EdenMount::unmount result in only one call to fuseMountPromise->getFuture().
Since fuseMountPromise->getFuture() is only called at most once, turn fuseMountPromise into a Promise instead of a SharedPromise.
Reviewed By: simpkins
Differential Revision: D14515070
fbshipit-source-id: 3ecaf9b55f3274046bc840bd5a65481cc552df72
Summary:
I want to make EdenMount robust if mounting and unmounting are requested concurrently. This will make it safer to call EdenMount::destroy in different situations.
Take a step toward that goal by making EdenMount::unmount idempotent; calling EdenMount::unmount multiple times should return the same future.
Changed behaviors:
* If EdenMount::unmount is called multiple times, EdenMount::fuseMount is called at most once, and each call to EdenMount::unmount waits for the EdenMount::fuseMount call to finish.
* **Old behavior**: One of the calls to unmount succeeds, and the remaining calls to unmount fail with an exception from PrivHelper.
Unchanged behaviors:
* If EdenMount::unmount was ever called, EdenMount::startFuse fails with EdenMountCancelled.
* If EdenMount::unmount was ever called, EdenMount::takeoverFuse fails with EdenMountCancelled.
* If EdenMount::startFuse is in progress, EdenMount::unmount causes the concurrent startFuse call to fail with FuseDeviceUnmountedDuringInitialization.
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount waits for fuseMount to finish before calling PrivHelper::fuseUnmount.
* If EdenMount::startFuse -> PrivHelper::fuseMount is complete, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::takeoverFuse is in progress, EdenMount::unmount does not cause EdenMount::takeoverFuse to fail.
* If EdenMount::takeoverFuse was called, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If neither EdenMount::startFuse nor EdenMount::takeoverFuse was never called, EdenMount::unmount succeeds and does not call PrivHelper::fuseUnmount.
Reviewed By: simpkins
Differential Revision: D14398507
fbshipit-source-id: 1da5424abc47341e4db3da1c50d425711e177366
Summary:
I want to make EdenMount robust if mounting and unmounting are requested concurrently. This will make it safer to call EdenMount::destroy in different situations.
Take a step toward that goal by making EdenMount::unmount succeed if EdenMount::startFuse was never called.
Changed behaviors:
* If neither EdenMount::startFuse nor EdenMount::takeoverFuse was never called, EdenMount::unmount succeeds and does not call PrivHelper::fuseUnmount.
* **Old behavior**: EdenMount::unmount fails with an exception from PrivHelper.
Unchanged behaviors:
* If EdenMount::unmount was ever called, EdenMount::startFuse fails with EdenMountCancelled.
* If EdenMount::unmount was ever called, EdenMount::takeoverFuse fails with EdenMountCancelled.
* If EdenMount::startFuse is in progress, EdenMount::unmount causes the concurrent startFuse call to fail with FuseDeviceUnmountedDuringInitialization.
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount waits for fuseMount to finish before calling PrivHelper::fuseUnmount.
* If EdenMount::startFuse -> PrivHelper::fuseMount is complete, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::takeoverFuse is in progress, EdenMount::unmount does not cause EdenMount::takeoverFuse to fail.
* if EdenMount::takeoverFuse was called, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::unmount is called multiple times, one of the calls succeeds, and the remaining calls to unmount fail with an exception from PrivHelper.
Reviewed By: simpkins
Differential Revision: D14398537
fbshipit-source-id: 9209aad8b2980045fd0f89a5ba149a833579c2cf
Summary:
I want to make EdenMount robust if mounting and unmounting are requested concurrently. This will make it safer to call EdenMount::destroy in different situations.
Take a step toward that goal by making EdenMount::unmount complete only after a concurrent PrivHelper::fuseMount call finishes.
Changed behaviors:
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount waits for fuseMount to finish before calling PrivHelper::fuseUnmount.
* In other words, if EdenMount::startFuse and EdenMount::unmount execute concurrently, either:
* mount(2) is never called (if EdenMount::unmount won the race), or
* mount(2) is called then umount(2) is called (if EdenMount::startFuse won the race).
* **Old behavior**: EdenMount::unmount calls PrivHelper::fuseUnmount immediately and either fails or succeeds.
Unchanged behaviors:
* If EdenMount::unmount was ever called, EdenMount::startFuse fails with EdenMountCancelled.
* If EdenMount::unmount was ever called, EdenMount::takeoverFuse fails with EdenMountCancelled.
* If EdenMount::startFuse is in progress, EdenMount::unmount causes the concurrent startFuse call to fail with FuseDeviceUnmountedDuringInitialization.
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount causes the fuseMount to be rolled back (by calling PrivHelper::fuseUnmount) after fuseMount completes.
* If EdenMount::startFuse -> PrivHelper::fuseMount is complete, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* if EdenMount::takeoverFuse was called, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::unmount is called multiple times, one of the calls succeeds, and the remaining calls to unmount fail with an exception from PrivHelper.
Reviewed By: simpkins
Differential Revision: D14398508
fbshipit-source-id: 6f01ce9e36b5802045acf54afe756b80fe102b1e
Summary:
D14398423 made EdenMount::startFuse fail immediately if EdenMount::unmount was ever called. EdenMount::takeoverFuse behaves differently; any prior EdenMount::unmount calls don't cause EdenMount::takeoverFuse to fail.
For consistency, make EdenMount::takeoverFuse fail in the same situations that make EdenMount::startFuse fail.
Changed behaviors:
* If EdenMount::unmount was ever called, EdenMount::takeoverFuse fails with EdenMountCancelled.
* **Old behavior**: takeoverFuse succeeds.
* **Note**: EdenMount::unmount still fails with an exception from PrivHelper.
Unchanged behaviors:
* If EdenMount::unmount was ever called, EdenMount::startFuse fails with EdenMountCancelled.
* If EdenMount::startFuse is in progress, EdenMount::unmount causes the concurrent startFuse call to fail with FuseDeviceUnmountedDuringInitialization.
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount causes the fuseMount to be rolled back (by calling PrivHelper::fuseUnmount) after fuseMount completes.
* If EdenMount::startFuse -> PrivHelper::fuseMount is complete, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::takeoverFuse is in progress, EdenMount::unmount does not cause EdenMount::takeoverFuse to fail.
* if EdenMount::takeoverFuse was called, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::unmount is called multiple times, one of the calls succeeds, and the remaining calls to unmount fail with an exception from PrivHelper.
Reviewed By: simpkins
Differential Revision: D14511630
fbshipit-source-id: b77ca5ee39c888e2d50547f76977db7aa5a0331a
Summary:
I want to make EdenMount robust if mounting and unmounting are requested concurrently. This will make it safer to call EdenMount::destroy in different situations.
Take a step toward that goal by making EdenMount::unmount cancel a concurrent EdenMount::startFuse call.
Changed behaviors:
* If EdenMount::startFuse is in progress, EdenMount::unmount causes the concurrent startFuse call to fail with FuseDeviceUnmountedDuringInitialization.
* **Old behavior**: startFuse might succeed or might fail with FuseDeviceUnmountedDuringInitialization.
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount causes the fuseMount to be rolled back (by calling PrivHelper::fuseUnmount) after fuseMount completes.
* **Old behavior**: EdenMount::unmount does not change the behavior of EdenMount::startFuse.
* **Note**: EdenMount::unmount still fails with an exception from PrivHelper, despite having an effect on EdenMount::startFuse. This quirk will be addressed by a future diff.
* If EdenMount::unmount was ever called, EdenMount::startFuse fails with EdenMountCancelled.
* **Old behavior**: startFuse succeeds.
* **Note**: EdenMount::unmount still fails with an exception from PrivHelper, despite having an effect on EdenMount::startFuse. This quirk will be addressed by a future diff.
Unchanged behaviors:
* If neither EdenMount::startFuse nor EdenMount::takeoverFuse was never called, EdenMount::unmount fails with an exception from PrivHelper.
* If EdenMount::startFuse -> PrivHelper::fuseMount is in progress, EdenMount::unmount calls PrivHelper::fuseUnmount immediately and either fails or succeeds.
* If EdenMount::startFuse -> PrivHelper::fuseMount is complete, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::takeoverFuse is in progress, EdenMount::unmount does not cause EdenMount::takeoverFuse to fail.
* If EdenMount::takeoverFuse was called, EdenMount::unmount calls PrivHelper::fuseUnmount immediately.
* If EdenMount::unmount was ever called, EdenMount::takeoverFuse succeeds.
* If EdenMount::unmount is called multiple times, one of the calls succeeds, and the remaining calls to unmount fail with an exception from PrivHelper.
Reviewed By: simpkins
Differential Revision: D14398423
fbshipit-source-id: eb86a237859ad2d992de5c901e63dda80d5ea6d5
Summary:
In future diffs, I will introduce some EdenMount tests which need to mock PrivHelper in more ways than are currently supported. For example, I want to mock PrivHelper::fuseUnmount and check how many times fuseUmount is called. Extend the existing MockMountDelegate and FakeFuseMountDelegate classes to support these to-be-written tests.
This diff should not change behavior.
Reviewed By: simpkins
Differential Revision: D14418206
fbshipit-source-id: a9b1257bb62d9a46fae0eef89c3fa42718bec142
Summary: Somehow, I missed adding the namespace to the Handle class - fixing it now.
Reviewed By: simpkins
Differential Revision: D14545345
fbshipit-source-id: b6bc8ae83703b04399a73dce3dd63a3aceae7331
Summary:
Eden needs the osxfuse version of the FUSE protocol on mac. The open
source cmake build pulls in osxfuse with getdeps.py, but that doesn't
work in the Buck build, and all we need is this one (old) copy of the
Linux kernel headers.
Reviewed By: strager
Differential Revision: D14506747
fbshipit-source-id: 028ddbaf80be9cad412462f3338c30b8f2a70087
Summary:
If EdenMount::takeoverFuse is called and createFuseChannel fails, the EdenMount reports that it's in the STARTING state. This is misleading and doesn't match how EdenMount::startFuse handles failures in createFuseChannel.
Extend the `try` block in EdenMount::takeoverFuse to handle exceptions raised from createFuseChannel.
Reviewed By: simpkins
Differential Revision: D14511564
fbshipit-source-id: 8f791efe1221bcfc1042f95054d181c303564086
Summary:
Update `RocksHandles` to call `RepairDB()` if we get an error opening the
database.
We have seen errors opening the DB in some cases after hard server reboots.
In every case so far `ldb repair` has been able to repair it with no adverse
effects. This simply makes `edenfs` automatically attempt to perform this DB
repair step.
Reviewed By: chadaustin, strager
Differential Revision: D14452216
fbshipit-source-id: 10c0cb0ff9cea3c3bbe485a4e489e4a6df640803
