Commit Graph

1033 Commits

Author SHA1 Message Date
Lee Howes
6e7b264295 Future<T>::then Future<T>::then(not-try-task) -> Future<T>::thenValue(task).
Summary:
Overall plan to modify Future<T>::then to be r-value qualified and use Future<T>::thenTry or Future<T>::thenValue.

The goal is to disambiguate folly::Future and to improve type and lifetime safety of Future and its methods.

Codemod:
 * future<T>.then(callable with operator()(not-a-try)) to future<T>.thenValue(callable with operator()(not-a-try)).
 * future<T>.then(callable with operator()()) to future<T>.thenValue(callable with operator()(auto&&)).
 * future<T>.then(callable with operator()(auto)) to future<T>.thenValue(callable with operator()(auto)).

Reviewed By: chadaustin

Differential Revision: D9443286

fbshipit-source-id: be712b58b92dc7422f128713deaf6f46b29b36ce
2018-08-22 12:07:41 -07:00
Dan Schatzberg
111c7fc399 Increase timer precision
Summary:
Existing timer precision (milliseconds) is too coarse to
debug many operations, it will often round down to 0ms. I didn't see
any rationale for not using something more precise here.

Reviewed By: chadaustin

Differential Revision: D9437556

fbshipit-source-id: 6a79431c5a976e933ecda228aa5386c07b8fb668
2018-08-22 11:22:39 -07:00
Puneet Kaushik
15998182c4 Added Windows handle support in hg_import_helper
Summary: If the OS is Windows it will expect the in-fd and out-fd as Windows handles and convert them to CRT fds. This will not change the behavior if it's not Windows.

Reviewed By: wez

Differential Revision: D9344414

fbshipit-source-id: 8b30b57d979dc45af06676979e75e00b02d2b89b
2018-08-22 10:52:13 -07:00
Puneet Kaushik
eeaf1c0025 Started Eden for Windows and integrated hg store with it.
Summary:
This diff is first in the series to make Eden work on Windows. It includes:

1. HG backing store and Object store, which provides the capability to talk to mercurial and fetch the file and folder contents on Windows.
2. Subprocess and Pipe definition for Windows.
3. The Visual studio solution and projects files to compile Eden and scm datapack.

Few Important points:

1. Most of the changes to existing code is done under a macro EDEN_WIN so that it doesn't impact on other platform.
2. Sqlite is used for caching the fetched contents. We are not using Rocksdb on Windows.
3. The main function only calls some test code and exit after printing the output.
4. The initializeMononoke code is disabled for Windows because it needs Proxygen to talk HTTP. Will enable this once I get Proxygen and other dependencies working.
5. HgImporter pass Windows handles to hg_import_helper as command line args. The code to convert these handles into fds is in a separate diff.

Reviewed By: wez

Differential Revision: D8653992

fbshipit-source-id: 52a3c3750425fb92c2a7158c2c214a9372661e13
2018-08-21 17:53:38 -07:00
Chad Austin
bfc189cc92 Use a deterministic executor in TestMount
Summary:
As a prerequisite to running inode unloading code in a background
queue, use a deterministic executor in TestMount to make sequencing in
unit tests reliable.

Reviewed By: simpkins

Differential Revision: D9323878

fbshipit-source-id: 0b85632c1637a8cf83d6f238675e5b6bbb6923c7
2018-08-21 12:23:00 -07:00
Adam Simpkins
bac93b71cf be more careful about how we re-open the repository
Summary:
Update the hg_import_helper.py code to catch exceptions from `repo.close()`
when we are re-opening the repository after an error.  Also set `self.repo` to
`None` before we start, so that it will be left as None if anything goes wrong
either closing or re-opening the repository.

I ran into a situation where `repo.close()` threw an error, and previously
this would leave `hg_import_helper.py` stuck in a bad state since it would
still be pointing to the old repo object.  The next time it received a command
it would fail and try to call `self.repo.close()` again, which would still
fail since the repository was already in a halfway closed state.

Now the code will always forget about the old repository object and create a
new object.

Reviewed By: wez

Differential Revision: D9419109

fbshipit-source-id: 15bb296ba19d9d3d2a2b90169bf25b0e8e197c1f
2018-08-21 11:53:26 -07:00
Zeyi Fan
96c7bb1c5f read use-mononoke and client-certificate in edenfs
Summary: This commit let `HgImporter` pick up configuration values from `EdenConfig`.

Reviewed By: chadaustin

Differential Revision: D9346293

fbshipit-source-id: cb63f7d13a86058e9bf076eddb52212560a64cb1
2018-08-20 11:22:14 -07:00
Chad Austin
e6ab9d58a2 Fix stack overflow when destroying long Journal
Summary:
While destroying JournalDelta entries, unzip the chain as we go and
destroy each entry one by one. This prevents the stack from
overflowing on long JournalDelta chains and should fix some flaky
tests.

Reviewed By: wez

Differential Revision: D9355365

fbshipit-source-id: 31af124d318ca5d7a84314b707e1b3c71b2ccaa9
2018-08-17 21:06:53 -07:00
Chad Austin
8b4841ef82 Replace shared_ptr<JournalDelta> with a custom intrusive JournalDeltaPtr
Summary:
For O(1) stack space JournalDelta destruction, we need to check
whether a reference-counted pointer has a unique refcount. It looks at
first glance like shared_ptr<T>::unique() gives us that, but it uses a
relaxed load and is now deprecated.

Sadly, the easiest thing is to make a new smart pointer class. (It
happens to be slightly more efficient, requiring one fewer heap
allocation per entry in the chain.)

Reviewed By: simpkins

Differential Revision: D9355314

fbshipit-source-id: 8c782ba9e0ec27fae90325079c199e1b82df88fa
2018-08-17 21:06:53 -07:00
Marshall Cline
dfac9dda4a use rvalue-qual Future::ensure(): pass 2
Summary:
This is part of "the great r-valuification of folly::Future":

* This is something we should do for safety in general.
* Several of folly::Future's methods are lvalue-qualified even though they act as though they are rvalue-qualified, that is, they provide a postcondition that says, in effect, callers should act as though the method invalidated its `this` object (regardless of whether that invalidation was actual or logical).
* This violates the C++ principle to "Express ideas directly in code" (see Core Guidelines), and generally makes it more confusing for callers as well as hiding the actual semantics from tools (linters, compilers, etc.).
* This dichotomy and confusion has manifested itself by some failures around D7840699 since lvalue-qualification hides that operation's move-out semantics - leads to some use of future operations that are really not correct, but are not obviously incorrect.
* The goal of rvalueification is to make sure methods that are logically rvalue-qualified are actually rvalue-qualified, which forces callsites to acknowledge that rvalueification, e.g., `std::move(f).ensure(...)` instead of `f.ensure(...)`. This syntactic change in the callsites forces callers to acknowledge the method's rvalue semantics.

This diff started as a Codemod, then required manual fixes. Here were the codemod steps:

* expr.ensure(...) ==> std::move(expr).ensure(...)  // if expr is not already an xvalue
* expr->ensure(...) ==> std::move(*expr).ensure(...)

Note: operator precedence of that last step is safe - no need to parenthesize `expr`. Reason: `->` binds more tightly than unary `*`.

Reviewed By: yfeldblum

Differential Revision: D9332070

fbshipit-source-id: 882121fe82c05fdb196ce676db686b6bc254974b
2018-08-16 20:37:45 -07:00
Chad Austin
de9416e41c stop writing to the overlay header on unload
Summary:
Now that timestamps are read from the inode metadata table, and users
aren't likely to run a pre-metadata-table version, the timestamp data
in the overlay header's no longer needs to be written. So remove that
code which has the bonus of making unloading faster.

Reviewed By: wez

Differential Revision: D9318044

fbshipit-source-id: 27a9a9ee954003940209819466932237a81f8929
2018-08-16 11:52:29 -07:00
Marshall Cline
b430d2c094 use rvalue-qual Future::unit(): pass 2
Summary:
This is part of "the great r-valuification of folly::Future":

* This is something we should do for safety in general.
* Several of folly::Future's methods are lvalue-qualified even though they act as though they are rvalue-qualified, that is, they provide a postcondition that says, in effect, callers should act as though the method invalidated its `this` object (regardless of whether that invalidation was actual or logical).
* This violates the C++ principle to "Express ideas directly in code" (see Core Guidelines), and generally makes it more confusing for callers as well as hiding the actual semantics from tools (linters, compilers, etc.).
* This dichotomy and confusion has manifested itself by some failures around D7840699 since lvalue-qualification hides that operation's move-out semantics - leads to some use of future operations that are really not correct, but are not obviously incorrect.
* The goal of rvalueification is to make sure methods that are logically rvalue-qualified are actually rvalue-qualified, which forces callsites to acknowledge that rvalueification, e.g., `std::move(f).unit(...)` instead of `f.unit(...)`. This syntactic change in the callsites forces callers to acknowledge the method's rvalue semantics.

Codemod changes:

* expr.unit(...) ==> std::move(expr).unit(...)  // if expr is not already an xvalue
* expr->unit(...) ==> std::move(*expr).unit(...)

Note: operator precedence of that last step is safe - no need to parenthesize `expr`. Reason: `->` binds more tightly than unary `*`.

Reviewed By: LeeHowes

Differential Revision: D9347419

fbshipit-source-id: 2773365f3793d977f2bad1c0a85ef394633e7d2c
2018-08-15 17:06:31 -07:00
Chad Austin
6394450579 restructure JournalDelta and fix Watchman subscription race
Summary:
Watchman's Eden integration has a bug where the combination of
Watchman querying Eden for overlapping delta ranges ("give me changes
between X and Y, now changes between X+1 and Y+1") and Eden eliding
redundant change events ("add-modify-remove" -> []) results in
Watchman sometimes reporting that a file exists in its final
subscription update when it no longer does.

The fix is to never elide events, even for files that were added and
removed in the same sequence. To continue to support Watchman's `new`
flag, track whether a file existed at the beginning and end of a
journal delta.

Reviewed By: wez

Differential Revision: D9304964

fbshipit-source-id: f34c12b25f2b24e3a0d46fc94aa428528f4c5098
2018-08-15 14:52:06 -07:00
Chad Austin
a541b15818 hide setcon() failure messages at startup by default
Summary:
The setcon() failure is not actionable or interesting, so don't log it
to stdout at Eden startup.

Reviewed By: wez

Differential Revision: D9344467

fbshipit-source-id: 68435c8f22c228f2fbb86f37c2b1874723934169
2018-08-15 13:52:06 -07:00
Zeyi Fan
10dfcf8da4 add two new configuration value for Mononoke
Summary: This commit adds two configurable value for setting path to client certificate and a flag to control Mononoke integration.

Reviewed By: chadaustin

Differential Revision: D9303157

fbshipit-source-id: 2f44d55d17b567655157a5f4b6f52e9468dda234
2018-08-15 10:51:44 -07:00
Wez Furlong
cfde0c0717 define paths as binary rather than strings in the thrift interface
Summary:
This prevents `hg status` from blowing up with a UTF-8 decode
error inside the generated thrift code.

Push safety concerns:
* This doesn't change the wire representation of the data
* Existing clients that believe it to be a string will continue to have
  the same behavior
* Buck has its own copy of an older version of the thrift spec, so it will
  continue to work "OK".
* When buck resyncs with our thrift file, some changes will likely be needed
  to convert the byte arrays to strings or paths or whatever is appropriate
  for bucks internal API

Work "OK" above means that clients that currently believe that `string` is
utf-8 encoded will have a runtime error if we ever send them a path that
is not utf-8.  This is the behavior prior to this diff and will continue
to be the behavior for clients (like buck) that have an older version
of the thrift file.

Reviewed By: simpkins

Differential Revision: D9270843

fbshipit-source-id: b01135aec9152aaf5199e1c654ddd7f61c03717e
2018-08-11 01:35:49 -07:00
Adam Simpkins
ee03bf2f70 always call fdatasync() when writing overlay data for the root inode
Summary:
The root inode is particularly important, so always call `fdatasync()` on its
changes in the overlay before committing them.

This will hopefully reduce the number of cases we see where users have empty
or corrupt data for the root inode after hard rebooting their server.  My
guess is that the root directory is being modified by hg or other tools
creating and removing temporary files in the root directory.  If a change like
this is in progress when a hard reboot has been performed we risk data loss
without the `fdatasync()` call.

While Eden can still mostly serve the checkout data if other files or
directories are corrupt/missing in the overlay it currently is completely
unable to mount the checkout if the root overlay is corrupt.  Therefore it
seems worth being more cautious about making sure that the root overlay data
is updated atomically.

Reviewed By: chadaustin, wez

Differential Revision: D9275852

fbshipit-source-id: b1e3eeb94ba670d0e2b52da4af7143d3ddbc919b
2018-08-10 14:57:57 -07:00
Adam Simpkins
d752d72fc8 add some additional verification when importing empty files
Summary:
We have seen issues with Eden occasionally getting empty file contents back
from Mercurial.  We have not been able to reproduce this issue directly
yet--the overall incidence rate relative to the number of users is fairly
small, but the problem is pretty problematic when it does occur.

This updates the hg_import_helper.py script to perform additional checking
when it finds a file that is empty.

Empty files that have never been modified or renamed all have the same file
revlog hash.  If the rev hash in question is this known hash, we know that the
file is in fact empty.  Otherwise, if remotefilelog is in use we check the
remotefilelog metadata to confirm the size, and we log an error if it is
non-zero.  We then try re-opening the repository and re-importing the file.

These additional checks and log messages should hopefully help narrow down the
problem the next time we see a report of the issue.  This should help us
identify if the remotefilelog metadata has the correct file size or not.

Reviewed By: wez

Differential Revision: D9260788

fbshipit-source-id: 29615b32632946cd319aa837bec3c68b757d3ee0
2018-08-10 11:38:20 -07:00
Chad Austin
fae4229ff2 add eden gc command
Summary:
Add the beginnings of an eden gc command. Today it's equivalent to
`eden debug clear_local_caches` followed by `eden
debug_compact_local_storage`, except that it compacts each column as
they're cleared to minimize peak disk consumption.

Eventually, it will also unload in-memory inodes, flush data from the
overlay, and clear the kernel's VFS cache too.

Reviewed By: wez

Differential Revision: D9138305

fbshipit-source-id: b303a63f601014cf38ca94c9e6f7c04394159ea8
2018-08-10 11:38:20 -07:00
Zeyi Fan
6f997c4d3f Use MononokeBackingStore when fetching trees
Summary:
This commit integrates Mononoke API Server with eden:

* Added two new command line options: `--client_certificate` and `--use_mononoke`.
* Added two new config values: `ssl.client-certificate` and `mononoke.use-mononoke`.
* Made `HgImporter` return the repo name along with treemanifest options.
* Make `HgImporter` ask Mononoke API Server for tree data when the desired tree is not present in local Mercurial store.

Reviewed By: chadaustin

Differential Revision: D9183035

fbshipit-source-id: e328fb3237d10c545c8af71f856007ad6c487061
2018-08-07 13:52:29 -07:00
Harsh Poddar
6289c96520 Add function name property to LogMessage
Summary: This update adds the name of the function which called the LOG() to the `LogMessage` object so that a custom formatter could use it to show method name with logs.

Reviewed By: simpkins

Differential Revision: D9071926

fbshipit-source-id: 9d5359d84cf2544371c6f1826f99d8b47a272dd1
2018-08-07 11:51:36 -07:00
Eamonn Kent
b857159384 Allow USER and HOME to be interpolated in Eden c++ code
Summary:
The python code supports expansion of ${HOME} and ${USER} in the config files.
Some of the config files use these values. This adds support for their
interpolation.

Reviewed By: chadaustin

Differential Revision: D9160960

fbshipit-source-id: 9133ee247c17937c9d8d548b9bfd26559a1459e9
2018-08-06 13:53:07 -07:00
Zeyi Fan
417fc46ce4 use ServiceRouter and SSL Connection in MononokeBackingStore
Reviewed By: chadaustin

Differential Revision: D9143949

fbshipit-source-id: 5e2b89579a19c1f5cca673fda476b74891bb6447
2018-08-06 11:51:36 -07:00
Lee Howes
69d8203162 Future<T>::then 6/n: Future<T>::then() & -> Future<T>::then() &&.
Summary:
Overall plan to modify Future<T>::then to be r-value qualified and use Future<T>::thenTry or Future<T>::thenValue.

The goal is to disambiguate folly::Future and to improve type and lifetime safety of Future and its methods.

6/n: Codemod rvalue-future<T>.then(...) to rvalue-future<T>.then(...).

Reviewed By: yfeldblum

Differential Revision: D9152002

fbshipit-source-id: 166475c1dcafb29a11154cbfbdf7e2e1feaf745b
2018-08-03 23:36:16 -07:00
Chad Austin
5d54a799c9 Rename UnboundedQueueThreadPool to UnboundedQueueExecutor and add a ManualExecutor variant
Summary:
To improve the determinism of our C++ tests, I am planning to switch
TestMount to a ManualExecutor. This adds a ManualExecutor constructor
to UnboundedQueueExecutor.

In Rust, I'd use a trait, but a simple class with two constructors works fine.

Reviewed By: strager

Differential Revision: D8846553

fbshipit-source-id: c52752105255503d26f1e65494c32b3f62882e44
2018-08-03 13:21:59 -07:00
Eamonn Kent
836600476b Remove unused code for ingesting system and user ignore files
Summary:
This code is no longer necessary since introduction of FileChangeMonitor and
CachedParsedFileMonitor.

Reviewed By: simpkins

Differential Revision: D8915599

fbshipit-source-id: 7871dac1f7985968bd4d04a4dfb42684892a2e72
2018-07-30 19:06:33 -07:00
Eamonn Kent
d39ca39b85 EdenMount should load/use ignore file list from configuration
Summary:
[Eden] EdenMount should make use of EdenConfig in order to determine the user and system ignore files to load.

This commit does the following:
- adds EdenConfig to ServerState that gets passed to the MountPoint.
- removes hard-coded system ignore settings.
- allows the ignore files to be updated in the user and system config.
- has the diff context loaded with actual ignore file contents.
- adds the FileChangeMonitor class to efficiently (throttled) identify when file changes have occured and reload as necessary.

Reviewed By: simpkins

Differential Revision: D8876214

fbshipit-source-id: d2697c130d3d4960c7f645ace226e5ce6b772048
2018-07-30 17:20:45 -07:00
Eamonn Kent
1c18cf23e5 FileChangeMonitor optimizations and stat changes
Summary:
This diff includes the following changes:

- Typically, we stat the monitored file to check for changes. Then, if changes
  are detected, we open/fstat the file.  If open fails, on subsequent
  operations, we can skip the stat check.

- We only update the stat struct of the FileChangeMonitor member when the file
  is successfully opened/processed. This means that all file change checks are
  done relative to the processed file (rather than the results of the previous
  stat).

- We still skip open/fstat if the isChanged() check indicates a file change
  because of a different stat error code. This will still result in the
  call-back being called.

Reviewed By: simpkins

Differential Revision: D9030118

fbshipit-source-id: def735fffe913f84b210cbf646f2849ab130bda4
2018-07-30 13:38:14 -07:00
Adam Simpkins
19d922b0e6 add integration tests for CLI behavior inside a stale mount
Summary:
Add some integration tests that invoked "eden doctor" and "eden list" inside a
stale mount point, and confirms that they work correctly.  These commands do
fail in dev builds, since python itself does not handle this error.
Fortunately XAR-based python archives avoids this error, so these commands
work in opt mode builds.

Reviewed By: strager

Differential Revision: D8905489

fbshipit-source-id: 0666f8cb0715d61697253bee9fe0bde2f4a3fa15
2018-07-27 18:36:55 -07:00
Lewis Baker
34b48e102d Fix data race in Eden fuse driver handling of FUSE_INTERRUPT
Summary:
Fixed a data race in Eden's `FuseChannel` implementation that could cause a crash if a `FUSE_INTERRUPT` request was concurrently on a different thread while a first thread was still launching the original request.

Modified `FakeFuse` to use `SOCK_SEQPACKET` instead of `SOCK_STREAM` so that tests can submit several requests in a pipelined fashion without having to first wait until the previous request was received. This requires the `recvResponse()` function to first read the header with `MSG_PEEK` to determine the response size and then subsequently read the entire message atomically to avoid reading a truncated message.

Added a new unit-test that exercises the `FUSE_INTERRUPT` race condition by sending a series of alternating `FUSE_LOOKUP`/`FUSE_INTERRUPT` requests.

Reviewed By: simpkins

Differential Revision: D9023654

fbshipit-source-id: 0eb44669ea8c4f58cf4313adf6ceb11098115a70
2018-07-27 14:36:42 -07:00
Adam Simpkins
ee88d00389 rename updateOverlayHeader() to updateOverlayHeaderIfNeeded()
Summary:
The `updateOverlayHeader()` only updates the overlay data if the inode is
materialized.  This updates the name to clarify that.

(This function name change was previously part of D8884795, and I'm just
splitting it into its own separate diff.)

Reviewed By: bolinfest

Differential Revision: D9011358

fbshipit-source-id: 6024d64a1dee0b5d741bec32ed88f6c8f8dd8a9a
2018-07-27 13:22:58 -07:00
Jun Wu
fae2056037 do not require SELinux in GitHub build
Summary:
I tried to build eden using GitHub version and ran into SELinux issues.
This patch fixed related issues, namely:

- `set(X Y)` sets X to string literal Y. Change it to `set(X ${Y})`.
- `SELINUX_INCLUDE_DIR` could be undefined. Check before use it.
- `./eden/fs/eden-config.h` and `./build/eden/fs/eden-config.h` both exist and
  the former is used. Set include dirs so the latter gets used.

Reviewed By: bolinfest

Differential Revision: D9029272

fbshipit-source-id: 0c94bbe2f9e3fa90973802ddde16ad4d9ddfc0e8
2018-07-27 13:22:57 -07:00
Adam Simpkins
4442ab5e54 increase the timeouts in FuseChannelTest to 1 second
Summary:
Many of the tests in FuseChannelTest.cpp perform blocking waits on Futures.
Previously they used a 100ms timeout.  This could sometimes result in flaky
test failures if the system was under heavy CPU load.  Bumping up the timeout
to 1 second appears to avoid this issue.

Reviewed By: lewissbaker

Differential Revision: D9024730

fbshipit-source-id: 7af78c61008c6b4c1e5e130b3d37b2f3ac787a01
2018-07-26 21:21:36 -07:00
Adam Simpkins
3b31841c4b fix crash when unmounting with corrupt overlay
Summary:
Update `InodeMap::updateOverlayForUnload()` to catch exceptions that occur
trying to save state to the overlay.  If something goes wrong when saving
state there is not much we can do other than log an error.  We still want to
unload the inode, and code that is unloading inodes generally cannot deal with
exceptions at this point.  In particular if this error occurred while trying
to shut down an EdenMount the code would crash.

Reviewed By: chadaustin

Differential Revision: D8884795

fbshipit-source-id: c2f850f13d775be4b0a0a10f9df3948c7b2c8f4a
2018-07-26 17:06:49 -07:00
Zeyi Fan
4b6aa1bf5c use new Mononoke API Server for MononokeBackingStore
Summary: This commit changes MononokeBackingStore to use new APIs provided by Mononoke API Server.

Reviewed By: chadaustin

Differential Revision: D8882789

fbshipit-source-id: 0f06ca5f850af9fb52f1d593b9abd715a541488a
2018-07-25 21:52:09 -07:00
Adam Simpkins
5da49afee8 change EDEN_BUG() to a plain exception on short overlay file
Summary:
Encountering a truncated overlay file doesn't necessarily indicate a software
bug in Eden.  Depending on the underlying filesystem this often happens after
a hard system reboot since we write the overlay files without an `fdatasync()`
call.

Change the code to simply log an error and throw an exception rather than
using `EDEN_BUG()`.  This makes it possible to exercise this code path in
tests without having it crash in debug builds.

Reviewed By: chadaustin

Differential Revision: D8988209

fbshipit-source-id: 8c0fe1dae692f4c493413d3939d2e4c21e0da596
2018-07-25 21:07:12 -07:00
Adam Simpkins
ea2a6034d4 Allow rm of files with corrupt overlay
Summary:
Sometimes, Eden's overlay (in `$client_dir/local/`) gets corrupt. In
particular, sometimes overlay files can be truncated or missing after a hard
reboot where the underlying filesystem state was not flushed to disk.

For such files, open(), stat(), unlink(), etc. from Eden report ENOENT, yet
readdir() on the containing directory shows that the file does exist.

In other words, the problematic file is undeletable:

```
$ ls -la dir/
/bin/ls: cannot access dir/corrupt_file: No such file or directory
total 0
drwxr-xr-x.  3 strager 0 Jul 10 21:41 .
drwxr-xr-x. 48 strager 0 Jul 10 21:41 ..
-??????????  ? ?       ?            ? corrupt_file
$ rm dir/corrupt_file
rm: cannot remove ‘dir/corrupt_file’: No such file or directory
```

Allow users to delete these problematic files (if the file was a regular file
and not a directory) by doing the following:

* Allow corrupt regular files to be unlink()d successfully.
* Allow corrupt regular files to be stat()d.

Making stat() succeed is a requirement by FUSE:

* For unlink(), FUSE performs FUSE_LOOKUP before FUSE_UNLINK. If FUSE_LOOKUP
  fails, unlink() fails. Therefore, we must make FUSE_LOOKUP succeed for
  corrupt files.
* For stat(), FUSE performs FUSE_LOOKUP and sometimes FUSE_GETATTR. Since we
  must make FUSE_LOOKUP succeed (for unlink()), it's natural to make
  FUSE_GETATTR succeed too.

A future diff will fix corrupted directories.

Reviewed By: chadaustin

Differential Revision: D8884793

fbshipit-source-id: 1100037bf52475fcca66f39946b917ce604f12dc
2018-07-25 21:07:12 -07:00
Eamonn Kent
4620a3af43 Added class CachedParsedFileMonitor and tests
Summary:
CachedParsedFileMonitor provides cached access to an object created by parsing
a data file. The object can be accessed through "getFileContents().
"getFileContents()" will check if the file has changed and reload/parse if
necessary. A throttle is applied to limit change checks to at most to 1
per throttleMilliSeconds.

Reviewed By: simpkins

Differential Revision: D8903257

fbshipit-source-id: 7ed28610a3e9b40b98b0a9ffaff7b307bbcd32a8
2018-07-25 20:07:23 -07:00
Eamonn Kent
f471204190 FileChangeMonitor interface changes.
Summary:
This commit improves the FileChangeMonitor interfaces. Changes are:
- on-demand check initiates a call-back
- FileChangeMonitor does not rely on user to call-back to update stat
- updated tests for new api
- fixed issue for determining if file changed (on error)

Reviewed By: simpkins

Differential Revision: D8989042

fbshipit-source-id: cf7a03b2bc8679891c8b9c22a24eed980eeb4060
2018-07-25 19:08:33 -07:00
Lee Howes
ac9b8ac398 Future<T>::then 3/n: Future<T>::then(not-try-task) -> Future<T>::thenValue(task).
Summary:
Overall plan to modify Future<T>::then to be r-value qualified and use Future<T>::thenTry or Future<T>::thenValue.

The goal is to disambiguate folly::Future and to improve type and lifetime safety of Future and its methods.

3/n: Codemod rvalue-future<T>.then(callable with operator()(not-a-try)) to rvalue-future<T>.thenValue(callable with operator()(not-a-try)).

Reviewed By: yfeldblum

Differential Revision: D8986716

fbshipit-source-id: 906339d9ffb90b3c38a24ce8bf0cef7be318d946
2018-07-24 23:36:23 -07:00
Lee Howes
2b4e4f861e Future<T>::then 1/n: Future<T>::then(try-task) -> Future<T>::thenTry(task).
Summary:
Overall plan to modify Future<T>::then to be r-value qualified and use
Future<T>::thenTry or Future<T>::thenValue.

1/n: Codemod rvalue-future<T>.then(callable with operator()(Try<T>)) to rvalue-future<T>.thenTry(callable with operator()(Try<T>)).

Reviewed By: simpkins

Differential Revision: D8961903

fbshipit-source-id: ff17b7833d240c221197cdf0bf914b8a39f80b07
2018-07-23 18:51:38 -07:00
Eamonn Kent
d176764379 FileChangeMonitor to monitor for file changes
Summary:
Add the FileChangeMonitor class to monitor for a file for changes. It provides
an interface "isChanged()" to check if the file has changed. It limits checks
by doing them on demand and by throttling. The throtte limits checks to 1 per
throttleMilliSeconds. It uses Stat information to determine if the file has
changed.

Reviewed By: simpkins

Differential Revision: D8898783

fbshipit-source-id: 067781489a9a91e5fe77cb613fba9d35b8348548
2018-07-20 17:52:36 -07:00
Matt Glazar
ae291d0a5d Refactor Overlay::InodePath
Summary:
InodePath is difficult to work with because it's just an alias to std::array. Make InodePath look more like std::string so using InodePath instances is more intuitive.

This diff should not change behaviour.

Reviewed By: chadaustin

Differential Revision: D8923073

fbshipit-source-id: 85a2548f0cfa61e50b6590048084076b9bece3da
2018-07-20 11:50:51 -07:00
Eamonn Kent
2f39c5297e DiffContext changes to use TopLevelIgnores
Summary:
Make use of the TopLevelIgnores class to hold the system and user
GitIgnoreStack details. This is a cleaner implementation making ownership
semantics more intuitive. In later commits we will provide acess to the
TopLevelIgnores as part of the ServerState. It will be dynamically loaded.

Reviewed By: simpkins

Differential Revision: D8906226

fbshipit-source-id: d955436582498861ac4b4113a47f357432c8f32e
2018-07-19 17:21:38 -07:00
Chad Austin
5ee9cbdabd clarify DirEntry's ownership of InodeBase*
Summary:
The raw Inode pointer in a DirEntry is more of an association than
ownership, so add comments and have clearInode return the old value.

Reviewed By: strager

Differential Revision: D8842315

fbshipit-source-id: d401dcdf4955ea335b39c2a57b0bedb1f83fdf9b
2018-07-17 21:37:35 -07:00
Eamonn Kent
60fec6bc5c Add TopLevelIgnores class for system/user ignore file details
Summary:
Add a class to encapsulate details of top-level system and user ignore files.
This approach has the following benefits:
- simplifies memory management of the GitIgnore;
- separate structure that can be created and provided as part of ServerState
  interface;

We will make use of it later in this commit stack.

Reviewed By: simpkins

Differential Revision: D8876064

fbshipit-source-id: 35c82b918c09e58068370401883edd8474dd3fbf
2018-07-17 18:27:07 -07:00
Eamonn Kent
00ad72e97c GitIgnore/GitIgnoreStack add copy constructors and assignment operators
Summary: GitIgnore and GitIgnoreStack encapsulate details of ignore files. Further changes (as part of this commit stack) require copy constructors and assignment operators. Also, compiler wants destructors in .cpp file (since otherwise, it cannot tell the object size).Keeping this as a separate commit to simplify review process.

Reviewed By: simpkins

Differential Revision: D8730299

fbshipit-source-id: 7cb443906143c80209278b85fc5ad4dc5ea0bf56
2018-07-17 17:22:10 -07:00
Aaryaman Sagar
d2a15f5aff Remove const from shutdownSubscribers() and stop()
Summary:
The methods were marked const when they were actually mutating the inner
objects, this was working because the pointed to objects were referenced via
`shared_ptr` handles

Reviewed By: yfeldblum

Differential Revision: D8851578

fbshipit-source-id: 4ca49390daf46f5c8681149da4d1e9eeab5aacb5
2018-07-16 16:10:38 -07:00
Chad Austin
51ed99000d Remove the redundant path_ member of EdenMount
Summary:
While trying to make destroyWithInitRace non-flaky, it uncovered an
ASAN violation in the case that fuseCompleteFuture finishes during
EdenMount destruction. In that case, path_ gets destroyed prior to the
executor, so it's illegal to construct TakeoverData::MountInfo from
path_. This diff removes path_ entirely and reads it from
ClientConfig.

Reviewed By: simpkins, strager

Differential Revision: D8848663

fbshipit-source-id: f9368aa9eec7dfa8f2897cce55fad6d19723e30c
2018-07-16 13:52:01 -07:00
Wez Furlong
eda02cdfab fix logic bug in glob prefetching
Summary:
We should only kick off prefetching for the files
that matched the glob.  We were prefetching files that
didn't match the glob.

facepalms

Reviewed By: strager

Differential Revision: D8846994

fbshipit-source-id: 593e85d843ffa1cc0707ed1dc86f1385262821f5
2018-07-16 11:38:15 -07:00