Summary:
Some Unix applications (notably, nfsd) create regular files using vfs_create, which ends up invoking the `mknod` system call rather than `open`, which for historical reasons only supported socket creation with Eden. However, since Eden supports regular files, we can broaden the FUSE mknod handler to support regular files as well.
For context, see https://github.com/GoogleCloudPlatform/gcsfuse/issues/137#issuecomment-155273363
Reviewed By: chadaustin
Differential Revision: D17792424
fbshipit-source-id: 466fcbcb3bcb587e731bc8b2a3e0f1508ff1f4e4
Summary:
Add test cases to ensure behavior of files that exist in the source control tree but match a gitignore pattern (in that case we should report file modification/removal if such happens and should not consult the gitignore)
from `man 5 gitignore` : Note that all the gitignore files really concern only files that are not already tracked by git
Reviewed By: chadaustin
Differential Revision: D17606492
fbshipit-source-id: 1da9f0617ecf2201eb6e548561c2de170f6184fa
Summary:
We make use of the KERN_PROCARGS2 MIB data that we can
retrieve via `sysctl`.
If we can't retrieve that data then we fall back to libproc as
we were doing previously. From my testing so far it seems like
the main reason for failure is that the target process is a
protected system process.
Reviewed By: chadaustin
Differential Revision: D17724101
fbshipit-source-id: 8de1a978e6f89612bfe247e0fd540d9078f50746
Summary: This will help us identify the calling process for the FS request. This was helpful while testing watchman crawl in Eden repo.
Reviewed By: chadaustin
Differential Revision: D17719028
fbshipit-source-id: 0d646d7949a6e26268b71287b26d63473a9a87f8
Summary:
We are seeing random segment fault originating from OpenSSL on macOS when
Mononoke fetching is enabled.
The cause is that on macOS we are actually linking against libcurl shipped with
the system instead of ours. That copy of libcurl is linked with macOS's
libcrypto instead of the one we compiles during Eden's build, and it seems that
version of libcrypto does not provide concurrency safety.
The solution is to build curl on macOS and make sure it is linked to our
OpenSSL that has the concurrency callbacks registered.
Reviewed By: wez
Differential Revision: D17657822
fbshipit-source-id: 85abdf3be10b3903a5efc6b3a91624c7258de790
Summary: Moved the code from vcproj testutils folder to test folder and enabled the Cmake build on it.
Reviewed By: wez
Differential Revision: D17738227
fbshipit-source-id: bcba41cb29b7e24782c36899c0977c5592888ff5
Summary:
Make it possible for any user on the system to connect to Eden's unix
domain socket. Our own permission checking is implemented in D17459971.
Reviewed By: wez
Differential Revision: D17460113
fbshipit-source-id: 29eba62010a58837df0376c66e252f53314a25dd
Summary:
To allow other user accounts to access fb303 counters, we plan to open
our unix domain socket to the world. But that requires implementing
our own permission checks, so do that first.
Reviewed By: wez
Differential Revision: D17459971
fbshipit-source-id: 3cc512da90f725d1c93f25f2a7a5ea7c72fb2b96
Summary:
Previously, readdir-triggered prefetches would merely look up and
cache the inodes for entries. Given a lookup also needs to return stat
information, including file sizes, prefetch blob sizes too.
Reviewed By: wez
Differential Revision: D17728752
fbshipit-source-id: 1c555c123fb7405f82885b427b32cd007fdb16ef
Summary:
Converting the const ptr in EdenDispatcher because we need to call non-const function (like update Journal). Also storing the reference in EdenDispatcher and WinStore instead of ptr.
This diff should not change the behavior of the Eden.
Reviewed By: chadaustin
Differential Revision: D17719606
fbshipit-source-id: 6c371c38045c760837130bb607163c2bc469b17a
Summary: This allows us to test the datapack code easier without rebuilding Eden.
Reviewed By: wez
Differential Revision: D17468473
fbshipit-source-id: a6807b4d6e747ae8557ae51fdf798de2a54fd4f1
Summary:
When Eden is competing with hundreds of build processes using about
the same amount of RAM, it's less painful for the OOM killer to kill
the compilers, not the edenfs daemon. Bias its OOM score a little.
Reviewed By: wez
Differential Revision: D17701071
fbshipit-source-id: 5ee1078e78e5aa6251692944ca12d9a892f86a6f
Summary: Now that Eden has dropped support for flatmanifest, allow GC'ing trees in the local storage.
Reviewed By: simpkins
Differential Revision: D17236345
fbshipit-source-id: d40ec0c2e61c9971ba1d976572d002c48c7bfc20
Summary:
it turns out that buck was using this to avoid making a thrift
call to get the sha1 of files.
Let's restore the method but have it return an empty list until buck
has migrated away from that.
asp2insp is looking at fixing up that aspect of buck
Reviewed By: chadaustin
Differential Revision: D17694987
fbshipit-source-id: 9ab0771c00fc64ed3c3395948650189d50d99180
Summary: Keep track of `ResponseChannelRequest` in diff() in order to short circut function if client request is no longer active.
Reviewed By: wez, simpkins
Differential Revision: D17531102
fbshipit-source-id: 662333e5fbc2ed9b3b32b619b8d15db618ad06ec
Summary:
Force a reference to the edenfsctlPath flag, otherwise the
linker will discard it and a large number of tests will fail.
Reviewed By: simpkins
Differential Revision: D17683222
fbshipit-source-id: b7cb29e74af85b544f45a228770ad2613c8e6efc
Summary:
This diff removes the logic that consumes the legacy bind
mount list and mounts them on startup. That functionality has been
replaced with the eden redirect command.
Instead of performing the bind mounts in the server, the server will
now run `eden redirect fixup` to apply that configuration.
This diff also changes the behavior of performBindMounts: previously, if the
bind mount setup failed, we would tear down the entire repo mount. Since we're
now spawning an external process, it is much more likely that something might
fail and result in a bad experience, so we no longer bail out in that case:
we'll continue and leave the bind mounts as-is. The user can then use `eden
doctor` or `eden redirect fixup` to sort things out.
Reviewed By: simpkins
Differential Revision: D17236366
fbshipit-source-id: 8b004551a076216f0e5448942f00b5195ee18803
Summary:
The RocksDbLocalStore.cpp code updates counters, so it depends on the fb303
library.
Reviewed By: fanzeyi
Differential Revision: D17669619
fbshipit-source-id: d05bee82b05743c2ba292808ce3d77812413d2eb
Summary: `folly::ssl::init` will initialize some of the random number generator and callbacks needed for concurrency.
Reviewed By: wez
Differential Revision: D17644835
fbshipit-source-id: 8b0b4ec98b119298053d7362339ebb8bfbd30586
Summary:
Update the `Future` based version of the `ThriftLogHelper` class to log how
long the call took, even if it failed.
The code already logged on error for non-Future thrift handler functions, but
for `future_*()` calls we only logged when producing a successful result.
This switches the code from `thenValue()` to `thenTry()` so that we log on
both successful and failed results.
Reviewed By: wez
Differential Revision: D17636471
fbshipit-source-id: 859a326e5bc54183c80f16bf463ba6d451f624e5
Summary:
D17623550 deleted `ServiceCacheIf::invalidateSelection()` but did not remove
this override, which broke the build for EdenFS's tests.
Reviewed By: wez
Differential Revision: D17638351
fbshipit-source-id: 200ead0f79a2252dc7ac2318745cd98e520737a4
Summary:
Update the logging for `getFileInformation()`, `glob()`, `globFiles()`, and
`getSHA1()` to log at most 5 paths, plus the number of remaining paths.
Watchman can sometimes call `getFileInformation()` with tens of thousands of
paths. We don't enable these log messages by default, but if someone has
turned up the log levels to help debug, this can easily fill up the log files
with spam.
Reviewed By: wez
Differential Revision: D17623234
fbshipit-source-id: 3ea8ace294fb8f042f1c53382c76148ba4af95f6
Summary:
Add a CMake option to control whether or not we should build support for
fb-mercurial (aka Eden SCM). If this is disabled we avoid building anything
under eden/fs/store/hg and drop support for the "hg" backing store.
Reviewed By: wez
Differential Revision: D15980320
fbshipit-source-id: 23a49d3e5cf89199666ff4a0bf46626502c12171
Summary:
On a machine with one core, it's quite possible for the process to
freeze for a second, resulting in a flaky test.
Reviewed By: wez
Differential Revision: D17579535
fbshipit-source-id: a3e158b5895c06bdf560ebecd849a74e4efa591e
Summary: Use callback to save ScmStatus instead of storing status inside a `TreeDiffer` object. Involes a bit of restructuring of some code to avoid circular dependencies and library creation. Mostly renames and file moves, some funtion moves as well.
Reviewed By: simpkins
Differential Revision: D17400466
fbshipit-source-id: fcd194a4c20204dffd3d11cd4a083564dc0ea938
Summary:
Increase the log level if a FUSE request timeout occurs, so we can tell if
this has occurred in the logs when debugging user problems. Rate limit the
messages to at most once per second to avoid lots of log spam when a problem
occurs that causes lots of FUSE requests to time out.
Reviewed By: chadaustin
Differential Revision: D17506837
fbshipit-source-id: b84bd094034f1b31ef74038a9fd3828d4db9acf2
Summary: Use callback to save ScmStatus instead of storing status inside a `TreeDiffer` object.
Reviewed By: simpkins
Differential Revision: D17379318
fbshipit-source-id: b5f56c937509f0861f5dc06c742ebc3e1f85c695
Summary:
Add a CMake option to control whether or not we should build support for
Git. If this is disabled we avoid building anything under eden/fs/store/git
and drop support for the "git" backing store.
Reviewed By: wez
Differential Revision: D15980321
fbshipit-source-id: 434364d81b44935ce86fdf4d66697ee21ff2992f
Summary:
The folly's version of fileutil's throw's "permission denied" error on Windows while creating the config file. Updating the code to call the Win32 version from eden/fs/win/utils.
We do eventually plan to fix Folly by moving the Eden Windows's FileUtils code to it. Until then we will call it from Eden's utils.
Reviewed By: fanzeyi
Differential Revision: D17490623
fbshipit-source-id: 948c488f1dfddd7b12eb8623e056a7680fd5aa73
Summary:
The code in `EdenConfigChecks.cmake` had two separate `find_package(cpptoml)`
calls, one using its installed CMake config file, and one using a custom
`Findcpptoml.cmake` module.
This removes the custom `Findcpptoml.cmake` code, and updates everything to
only used cpptoml's installed CMake configuration.
Reviewed By: chadaustin
Differential Revision: D17401220
fbshipit-source-id: 3789703cdfc029049db3b1bd9f5751fa2a60a8d4
Summary:
I was experimenting with using direct mononoke fetches from my mac but
was hitting 400 errors. It turned out that we were resolving the VIP name and
using the resolved host in the generated URL. The VIP configuration doesn't
know how to route such a request and yields a 400 response.
The SocketAddressWithHostname type has enough information to allow for handling
this correctly, but we were discarding the hostname component!
This diff captures the entire SocketAddressWithHostname type in the `address_`
member and uses the hostname portion when generating the URL.
Reviewed By: fanzeyi
Differential Revision: D17470256
fbshipit-source-id: 14ebd517eac12e07248a64fb37c0623ecbe37e89
Summary: Remove all of the importer code that supported flatmanifest repositories.
Reviewed By: simpkins
Differential Revision: D15057249
fbshipit-source-id: 1581712c3ab9a5daeb70e61cceb5de013f16f5e1
Summary:
The Catalina beta reports `10.15` and we were throwing an
exception when we didn't match three version number components.
Allow for 2 or more components to match when scanning the string.
Reviewed By: fanzeyi
Differential Revision: D17490838
fbshipit-source-id: 240513f21c0f57e629a300a35a815e7a572e93ff
Summary:
Use the correct inode pointer when creating an `InodeError`. Previously this
passed in an inode pointer that was always null.
Reviewed By: wez
Differential Revision: D17438864
fbshipit-source-id: 37b7e40723afb9335b7616b585eed3b6cf5b3c81
Summary:
Increase the tolerance values in the PeriodicTask tests from 20ms to 40ms.
In practice we occasionally did see flaky test failures in our continuous
integration environment due to events happening taking slightly longer than
allowed with our given timeout tolerances.
This also increases the overall timeout intervals, to help ensure that the
increased tolerance isn't lowering our confidence in the correctness of the
timing behavior. This does make the tests take about twice as long, but this
doesn't seem like a huge problem overall: making sure the tests are reliable
seems more important. The `testSplayOn()` test went from about 800ms to
around 1.7s, and `testSplayOff()` went from about 700ms to about 1.3s.
Reviewed By: wez
Differential Revision: D17436700
fbshipit-source-id: a64666a2850d6a704cbc6e535be9fdf191647b3b
Summary:
Fix `GlobNode::evaluate()` so that it only generates a result once all
processing is complete. The glob evaluation walks down the inode tree in
parallel. Previously if an error occurred while processing one TreeInode it
would return that exception to the caller immediately, even if work was still
being done to evaluate the glob on other parts of the inode tree. This
behavior is problematic, since the caller may destroy the GlobNode immediately
once a result is generated, and the caller has no way to know that other
children GlobNodes may still be using some of the data.
This fixes the code to wait until all evaluation sub-futures have completed
processing even on error, so that we know it is safe to destroy the GlobNode
before we generate a result.
Reviewed By: wez
Differential Revision: D17434327
fbshipit-source-id: ef70dc82281eeb3540840202ec5e704afe2d29eb
Summary:
Building on D17288111, this allows the systemd integration
to start eden without using sudo.
Reviewed By: chadaustin
Differential Revision: D17362519
fbshipit-source-id: 4a18380ff8de02e659bf4f906ecfffbf25ce7ad7
Summary:
Make sure the contents of the special `.eden/` subdirectory are correct each
time we mount a checkout. Before we would generally only set up the contents
of this directory if it didn't previously exist.
Now the code verifies that the contents of each symlink in this directory are
correct, and recreates the symlink if needed.
This allows EdenFS to automatically repair the contents of this directory even
if the checkout or its `clients` directory has been manually moved.
Reviewed By: wez
Differential Revision: D17279413
fbshipit-source-id: e24e7530f44fff94ebb6f67174aaf78c9b498d6b
Summary:
Update the CMakeLists.txt to also build the Python-based `edenfsctl` command
line tool.
This requires switching most of the thrift rules to generate both C++ and
Python sources.
Note that one missing feature at this point is that this does not package
external dependencies into the binary. Currently `edenfsctl` depends on both
`six` and `toml` as external dependencies. For now these must be available in
your `PYTHONPATH` in order to run the generated `edenfsctl` binary.
Reviewed By: chadaustin
Differential Revision: D17127615
fbshipit-source-id: fc138ab39e75c6a5bbd39e3f527d4e9f7f420e46
Summary:
The Windows library definitions were missing many dependencies. They happened
to link for now, but this was fragile and it was easy to cause this to break
by making changes to the dependency structure of our other libraries.
Additionally the files in eden/fs/win/store and eden/fs/win/mount also have
circular dependencies between each other that cannot easily be resolved in one
direction or the other. Therefore this combines the two `eden_win_mount` and
`eden_win_store` libraries into a single `eden_win` library to eliminate this
circular library dependency.
Reviewed By: wez
Differential Revision: D17309104
fbshipit-source-id: c95d0cfb8979e2b68d4947884acd13f859a4e01b
Summary:
In the CMake-based build, split EdenError.cpp and PrettyPrinters.cpp out into
a separate library from `eden_service`. Even though these are in the
`eden/fs/service/` subdirectory, these are lower-level utilities, and other
libraries in other `eden/fs/` subdirectories depend on them. Having them be
part of the `eden_service` library incorrectly causes circular library
dependencies. This splits the libraries out to make our CMake-based build
more closely match our Buck-based build.
Reviewed By: wez
Differential Revision: D17308275
fbshipit-source-id: 557af2ab54fb2a8baa44d30bcef85e4f76ab7cf0
Summary:
This silences a compiler warning about reaching the end of a non-void function
without returning a value.
Reviewed By: chadaustin
Differential Revision: D17308290
fbshipit-source-id: 95cdb3353364a36dcd2295b19bf745a941e5e3cf
Summary:
This diff passes the free space information through from
the overlay lock file descriptor back up to the kernel when the
filesystem stats are requested.
This makes the user experience with eg: Finder on macOS nicer.
Reviewed By: chadaustin
Differential Revision: D17255859
fbshipit-source-id: ed9f3b9fd386c5706539879513854a9cd1550d8a
Summary:
The macos fuse kext will block umount(2) until it gets
a reply to the FUSE_DESTROY opcode, causing `eden stop` to always
end up SIGKILL'ing the process, and thus causing an fsck on the next
startup.
So: let's respond to it after dispatching to the destroy method.
Reviewed By: chadaustin
Differential Revision: D17251386
fbshipit-source-id: 947d2aeb65edadbfbbac8b2673933ade283215b0
Summary:
Build the hg store on macOS with Buck. This primarily involves putting
ServiceRouter and Rust datapack support behind build switches.
Reviewed By: fanzeyi
Differential Revision: D17106241
fbshipit-source-id: da4643a11d6e5d0a6df23bff1d5e1454ab351abe
Summary:
The Eden macOS build has regressed in a few ways since it's been
broken due to RocksDB changes. Fix those small issues.
Reviewed By: fanzeyi
Differential Revision: D17107209
fbshipit-source-id: ef2e97a84e2403e7b7f36a4a01e2ceecab69106e
Summary:
Update the fsck code to save any orphaned symlink inodes that it finds as
symlinks in the repair archive directory, rather than saving the contents as a
regular file.
Reviewed By: wez
Differential Revision: D17170346
fbshipit-source-id: 4cba8b27233b728114a80a327ab519b039297aea
Summary:
Use the new `OverlayChecker` class to automatically scan for errors and
attempt to repair them if the overlay was not shutdown cleanly the previous
time it was used.
Reviewed By: wez
Differential Revision: D16596601
fbshipit-source-id: 9923565b101ba953e92909e502be6ef5895c5cbd
Summary:
Update the OverlayChecker's `inodes_` map to store `InodeInfo` objects
directly, rather than pointers to `InodeInfo` objects allocated on the heap.
This tweaks some of the error handling code to look up `InodeInfo` objects in
the map in a few places, to avoid storing raw pointers to `InodeInfo` objects.
The `InodeInfo` objects generally shouldn't move once we start performing
error checking, but it seems better to be conservative here and avoid using
raw pointers that are only loosely coupled with the original object's
lifetime.
This probably shouldn't really make much of a performance difference in
practice, since this code is likely to be disk I/O bound anyway. (It does
appear to make a small difference in performance for ASAN-enabled builds when
the inode files are warmed up in the kernel cache.)
Reviewed By: wez
Differential Revision: D16750736
fbshipit-source-id: b089a259cc83ffd2fda6f83b617d95ce4ac467f9
Summary:
We are probably going to add more inodemap counters, so add a prefix
now.
Reviewed By: fanzeyi
Differential Revision: D17142015
fbshipit-source-id: 4bd3cd4fd9234d8766864f364fef0b0d963f03b6
Summary:
This was causing problems on macos where various tools
would enumerate and helpfully try to preserve attributes across
copies. On macos this would result in appledouble metadata files
being created to track the metadata in the destination file,
which clutters up the repo and has surprising secondary effects
such as being picked up by glob operations in cmake build rules.
This diff simply stops enumerating the extended attribute.
Reviewed By: fanzeyi
Differential Revision: D17140414
fbshipit-source-id: 2924657dc75b900baf70595edfa72e5d0521a697
Summary:
This fixes a few issues with the library dependencies:
- The `eden_utils` library depends on `eden_service_thrift`, not
`eden_service`. By incorrectly depending on `eden_service` this introduced
a circular dependency which would cause a build failure, depending on which
order CMake chose to try and emit the link line.
- The `eden_config` library depends on code from `eden_model` (for `Hash` and
`ParentCommits`)
- The `eden_inodes` library depends on `eden_model_git` for the `GitIgnore`
logic. I also alphabetized the dependency list.
Reviewed By: wez
Differential Revision: D17124930
fbshipit-source-id: 70cbe81081fc1dc807cca13a93edc25ba270b01f
Summary:
Rename the `ThriftCppLibrary.cmake` file to `FBThriftCppLibrary.cmake`, and
also rename `add_thrift_cpp2_library()` to `add_fbthrift_cpp_library()`.
Explicitly calling this `fbthrift` helps clearly distinguish that this is
intended for use with fbthrift (https://github.com/facebook/fbthrift/), as
opposed to Apache thrift.
Reviewed By: wez
Differential Revision: D16738440
fbshipit-source-id: 9b255e06b71c98ad74a34989f564a211958dcdd5
Summary:
Implement the repair() logic for InodeDataError, MissingMaterializedInode, and
OrphanInode.
The most common errors we see after an unclean system reboot is an inode file
not being present at all (`MissingMaterializedInode`), or it being present but
empty (`InodeDataError`). When either of these errors occurs for a directory
it also produces `OrphanInode` errors, as all of the children entries in the
directory are no longer part of the directory hierarchy.
This implements repair logic for these three error types. The repair behavior
is largely similar to the Python version in `eden/cli/fsck.py`, with some
minor changes to the output paths in the `lost+found/` repair directory. This
also includes more unit test to better exercise handling when `InodeDataError`
and `MissingMaterializedInode` errors occur inside an orphan subdirectory that
needs to be extracted to `lost+found`
Reviewed By: strager
Differential Revision: D16577696
fbshipit-source-id: 948158c5a0c32b31574d93011281c42051645ad9
Summary:
If the hg_import_helper (really `hg debugedenimporthelper`) command
claims to not support treemanifest, then immediately fail.
Reviewed By: strager
Differential Revision: D15302513
fbshipit-source-id: 044a9cba877d65b6dab0ede6b431146a361d2801
Summary: Create config.toml in first Eden start up and store initial local store flag.
Reviewed By: chadaustin
Differential Revision: D16993074
fbshipit-source-id: dca8dbe60a892b7402ebf2f6c17df715dd2e8792
Summary:
This diff adds a fuse request timeout configuration setting
and threads it through to both the FuseChannel for our internal processing
and to the privhelper so that we can set an appropriate (slightly larger)
value for the kernel level daemon_timeout setting.
Reviewed By: chadaustin
Differential Revision: D16957552
fbshipit-source-id: a0fecc691d72914b5aebaed8a006dc0d6b0d7d12
Summary:
The macOS FUSE implementation has the concept of a daemon timeout,
which is used to set an upper bound on the length of time that the kernel will
wait for a request to be satisfied. If a request takes too long, the kernel
will shutdown the fuse device and the user is left with a relatively broken
experience in their repo until they restart the eden server. Critically, when
it is left in the broken state it is hard for the user to realize that they
need to restart the server because the kernel will still respond with EIO to
file accesses within the mount; even though it has been stopped, it isn't
fully unmounted until the fuse process stops.
This diff introduces a self imposed timeout in the fuse processing flow
and emits an ETIMEDOUT error when it is reached.
The intent is that we'll configure this timeout to be smaller than the
macOS daemon_timeout (which will be adjusted in a separate diff) so
that we don't trigger the problematic behavior when the kernel decides
that we've timedout.
This change is made for all fuse implementations, not just macOS, for
the sake of consistency: there's value to doing this on Linux as well,
to avoid some deadlock like scenarios: this should put an upper bound
on blocking in certain situations.
I've made the timeout default to 60 seconds, but haven't added any
configuration code for this yet; will do that in a follow on diff.
Reviewed By: chadaustin
Differential Revision: D16917954
fbshipit-source-id: 675539c43cf7f0009fd65d138081b9126464b7e0
Summary:
on linux kernel 5.1 we noticed that some caps were requested but not
known to the diagnostic code. This little diff adds a missing entry.
There is still one more that is unknown, but I think that we'll need to update
our copy of the fuse header to get that handled here.
Reviewed By: chadaustin
Differential Revision: D16953397
fbshipit-source-id: 30d19843a2cf1ec22fd469654773afaafa8e3b9b
Summary:
StartingGate was similar to std::barrier except that it guaranteed the
coordinating thread woke the worker threads. While this symmetry is
nice in concept, given enough threads, I doubt it pays for the
duplication.
Just adopt folly::test::Barrier (which is basically std::barrier)
instead.
Reviewed By: strager
Differential Revision: D16942136
fbshipit-source-id: 1691d8ef72b88c8867df74c8ed938f6c8d6ee094
Summary: Using a positive meaning rather than a double negative makes the build a tad simpler.
Reviewed By: wez
Differential Revision: D17000782
fbshipit-source-id: ef6c7b64708aa9b1f50c7ad4086c492a90c944f4
Summary: Using a positive meaning rather than a double negative makes the build a tad simpler.
Reviewed By: strager
Differential Revision: D17000620
fbshipit-source-id: ff27eb8098786b8ed6ed1ba81166b51e29e62d47
Summary:
The definition of RLockedPtr should use a non-const Subclass type, and not a
const one.
Reviewed By: ot
Differential Revision: D15356827
fbshipit-source-id: b8ad41e263f0e15ffa25b0698aa85eab8ca2ccb8
Summary:
The Java environment on Sandcastle macOS does not match corp laptops,
so allow disabling javadeprecated Thrift bindings on macOS for
now. Long-term, the better fix is to standardize the Java runtime and
JDK deployments across both environments so fbcode scripts can rely on
/usr/local/java-runtimes. See discussion in D16656570.
Reviewed By: strager
Differential Revision: D16676813
fbshipit-source-id: c21420bad554402c4c1e0881cabb962856719f5f
Summary: Allow building fuse_tester with open source gflags and glog.
Reviewed By: strager
Differential Revision: D16636221
fbshipit-source-id: 5128e936d5a6b6d0ed33bf34b770ee661728af3f
Summary: Now that StartingGate is in a more accessible dependency, use it in Eden's benchmarks.
Reviewed By: strager
Differential Revision: D16705853
fbshipit-source-id: 79ab9e9b32ab93f5b50a06a4889edb5e7b0d7e8b
Summary: On Linux and Mac, we provision .eden inside every directory, which contains symlinks to other locations. On Windows this won't work as well. First, Projected FS will cache all the .eden and will bump up the number of files on the disk. Second, it would be hard to simulate symlinks from Projected FS.
Reviewed By: strager
Differential Revision: D16724354
fbshipit-source-id: a560698fae8c7f14e475f1c1fb9ecc53e60c189f
Summary:
Implement OverlayChecker::computePath().
This also updates it to return a structure which can better represent
information if the inode is part of an orphaned subtree, which prevents us
from figuring out the full path to the root of the checkout.
Reviewed By: chadaustin
Differential Revision: D16577697
fbshipit-source-id: b543e2210fe8bc561d78194852962bf57ad9644a
Summary:
This ports much of the Python logic in `eden/cli/fsck.py` to C++, so that
the edenfs daemon can invoke it directly during mount initialization when
necessary.
This initial diff just contains the code to scan for errors, without any
repair logic.
This also includes some improvements to the logic for efficiency. For
instance, we compute less state during the initial scan phase. This makes the
scan phase faster, at the expense of making some of the repair steps slightly
more expensive if errors are found. This newer C++ scan logic is
significantly faster than the old Python version: scanning 260,000 inode files
took around 1m20 second for the Python code, and runs in under 3 seconds with
the new C++ code. (On an SSD with the files in question already warm in the
kernel cache.)
Reviewed By: strager
Differential Revision: D16577699
fbshipit-source-id: 2f8713a219ad882848cee0c57dca34f2c7808340
Summary: Redirecting the stdout and stderr to the eden log file. The log categories are not working as expected on Windows and will need to be fixed. At this point this doesn't log the "eden=DBG2" on Windows. One workaround for now is to remove "FOLLY_INIT_LOGGING_CONFIG()" to get the default logs.
Reviewed By: chadaustin
Differential Revision: D16653632
fbshipit-source-id: 028cdf1d698798ff3fb0284b3f0fa2df5570eaf0
Summary: Moving these functions to EdenInit so we could use them on Windows. This diff doesn't change the behavior.
Reviewed By: chadaustin
Differential Revision: D16653498
fbshipit-source-id: 5cc8c2f4b12f54035fb86ae20f3274245642d032
Summary:
Drop the `2` from the `fb303_thrift_cpp2` rule name.
The old `cpp` version of fbthrift is gone, `cpp2` is the only supported C++
version, so lets drop the `2` from this build rule name.
I plan to change some of the CMake thrift rule generation code soon, to
support multiple languages (namely Python). As part of this I plan to
automatically add a language-specific suffix(`_py` or `_cpp`) to the build
rules. It seems better to use `_cpp` rather than `_cpp2` for C++ libraries.
Changing the name for this fb303 rule will make it fit this planned naming
scheme.
Reviewed By: chadaustin
Differential Revision: D16653264
fbshipit-source-id: e4da70c79ff14e9981717ab909d982770d104512
Summary:
Now that we have unsafe access to the contents of a
folly::Synchronized, InodeMap::updateOverlayForUnload can just access
the storage because it knows nobody else has access.
Reviewed By: simpkins
Differential Revision: D16632778
fbshipit-source-id: 4b2c1d6679e2655671d43899407bac1a19f2cab9
Summary:
`StartupLogger::exitUnsuccessfully()` is declared as `noreturn`, and its last
statement is to call another `noreturn` method. However, this other method is
virtual, and the behavior of the `noreturn` attribute on virtual functions
doesn't seem consistently implemented across compilers. gcc doesn't seem to
treat the virtual method as `noreturn`, and therefore still warns that
`exitUnsuccessfully()` may return.
This suppresses that warning by adding a `folly::assume_unreachable()`
statement.
Reviewed By: fanzeyi
Differential Revision: D16653184
fbshipit-source-id: e0a2d8a4420d3582f849e93568a5081827a788f5
Summary:
The toHash field is unnecessary in HashUpdateDeltas since we only ever iterate in reverse. We instead keep track of the current hash in the journal itself instead of looking at the toHash of the latest HashUpdateDelta.
Delta Struct Sizes:
> File Change: 88
> Hash Update: 96
Reviewed By: strager
Differential Revision: D16522519
fbshipit-source-id: 43baccc8ef2579f72609cc84e81e218794b11725
Summary: Create the second delta type HashUpdateJournalDelta that keeps track of changes to the hash [and the unclean files associated with that hash change]. Journal methods were updated to account for the different delta types.
Reviewed By: strager
Differential Revision: D16520444
fbshipit-source-id: 2a5cea11c9e70e30f6db55d9c8e33f9322ae91fc
Summary: Add in two tests for deltas that change commit hashes since a test for this did not exist before. Previously a bug in setting the fromHash of a result only showed up as a failure in Watchman's integration tests with Eden.
Reviewed By: chadaustin
Differential Revision: D16528248
fbshipit-source-id: 56eede749ef2da4dc492a1f7376dc07ca8aa3050
Summary: Deltas are stores in a deque and therefore not malloced independently so therefore there is no reason to be getting the goodMallocSize of them. We also need to do accounting for the buckets in the deque that the deltas fall into.
Reviewed By: strager
Differential Revision: D16566675
fbshipit-source-id: 4506fbbcc2044b8fdfe6244313ef7480cfa5151e
Summary: Record a rolling sum of the time taken by any FUSE call on a per-process basis
Reviewed By: strager
Differential Revision: D16553149
fbshipit-source-id: 54f1e453916727a40f245b294239dc1b232a8967
Summary:
Change the behavior of `newEdenError()` to simply join its message arguments
together using `folly::to<std::string>()` rather than using
`folly::sformat()`.
Only a couple locations in the code were actually passing in a format string
correctly. Several other locations in the code were incorrectly passing in
arguments that they expected to be joined. This resulted in the error
messages being truncated, as the initial string did not contain any format
placeholders, so the trailing arguments were simply ignored.
Reviewed By: strager
Differential Revision: D16592709
fbshipit-source-id: d56f5387e05da2a68d94e6badac28ffcf68d6eab
Summary: Uses the existing RequestData class to make calls to static functions to set and get the `didImportFromBackingStore` flag.
Reviewed By: simpkins
Differential Revision: D16461868
fbshipit-source-id: e3ed39249f5dd1a842ad06a204b5933014b12f7f
Summary: hg debugedenimporthelper should either have the repo path as argument or the current working directory should be inside a repo. Setting the current working directory for the process.
Reviewed By: strager
Differential Revision: D16565042
fbshipit-source-id: d6e826bebf58fd58f5250e6c477849ca147628c8
Summary:
Update `EdenServer::mount()` to correctly handle errors that occur during the
mount `INITIALIZING` phase. Previously the code did not add error callbacks
to the `Future` result to handle errors during initialization. As a result we
would propagate the exception back to the thrift caller, but the `EdenMount`
object would remain in our mount point list, stuck forever in the
`INITIALIZING` state.
Reviewed By: strager
Differential Revision: D16590032
fbshipit-source-id: 9adbdf05441dad815096b195ece36f3d958c96a9
Summary:
Update `FsOverlay::validateHeader()` to throw `EdenError` exceptions rather
than `std::system_error`. These exceptions are generated when we find invalid
data, rather than from system calls that return errno values. Previously the
code was using `folly::throwSystemErrorExplicit()` and
`folly:throwSystemError()`, and passing in a made-up `EIO` value.
However, the latter two calls were incorrectly using
`folly::throwSystemError()` rather than `throwSystemErrorExplicit()`. As a
result the `EIO` parameter was being treated as part of the error message, and
the code was appending the error description for whatever value `errno`
happened to be set to at the moment.
This fixes the code to just throw `EdenError` exception types, since these
errors aren't caused by a real `errno` value.
Reviewed By: strager
Differential Revision: D16577698
fbshipit-source-id: 9487c34f04da99d397611f005f00f02114b12094
Summary:
Fix `FsOverlay::initOverlay()` to also create the `tmp` subdirectory, which is
required for the `FsOverlay` code to function properly. Previously this
directory was created by the `Overlay` class. However, `FsOverlay` should be
responsible for creating it so it can be used from other locations besides
just `Overlay`.
I also removed the code that checks if the `tmp` directory needs to be created
on restart. The code to create `tmp` has been present for over a year
(D8330070), and we no longer have anyone using EdenFS versions older than
this.
Reviewed By: strager
Differential Revision: D16577700
fbshipit-source-id: 3eaa28003159a03084c4f367c7b96d69fc1702d0
Summary:
There is no point to call `readBE<uint64_t>()` here. The only way these calls
can fail is if the buffer is too short for the data, but we already checked
the buffer length at the top of the function.
Reviewed By: strager
Differential Revision: D16577701
fbshipit-source-id: 8b57cec05453b9b7224470671e032e3aefe41033
Summary: Replace the uses of std::unique_ptr<JournalDelta> with just the JournalDelta itself to avoid an extra allocation.
Reviewed By: chadaustin, strager
Differential Revision: D16572089
fbshipit-source-id: be080b2fb9096f6c8783e2ecae21a99466336f6f
Summary:
Adds a debug command to eden such that users can flush the journal and cause the subscribers to get a truncated result, this will be useful for debugging [to debug we won't have to lower the memory limit and repeatedly touching files to fill up the journal].
Can be used with "eden debug flush_journal"
Reviewed By: chadaustin
Differential Revision: D16348811
fbshipit-source-id: fdbe6729d0393c424addcd42a091de440383035b
Summary: Refactoring the addDelta function to allow other Journal functions to add deltas while holding the deltaState lock. This refactoring will be used to create the flush function for the journal (which needs to hold the lock to empty the journal but wants to add a delta before releasing the lock).
Reviewed By: chadaustin
Differential Revision: D16341196
fbshipit-source-id: 7b7b1d933802b466efe624378206c72c71469129
Summary: Delete tests that rely on flatmanifest support in Mercurial.
Reviewed By: strager
Differential Revision: D15302489
fbshipit-source-id: 7e46f21bfe3ae910e205e73bca95d9a33f4a6816
Summary: D16461081 broke the windows build of Eden by not updating the Windows mount code, this diff makes the same change as that diff in the Windows portion of the code.
Reviewed By: chadaustin
Differential Revision: D16549392
fbshipit-source-id: eee1bede08c1b27329b026746dda6863519ed10b
