Summary:
Fix the arguments to `doctor.cure_what_ails_you()`, which were updated in
D8870494.
Reviewed By: chadaustin
Differential Revision: D8940413
fbshipit-source-id: 21d45036f668c9bae9a2002c360532918eb157a5
Summary:
If the `nuclide-connections` command is not installed
`subprocess.check_output()` throws a `FileNotFound` exception. Previously the
code only caught `CalledProcessError` and `ValueError` exception. This
updates the code to catch any Exception type, just in case some other
unexpected exception type can also be thrown too.
Reviewed By: chadaustin
Differential Revision: D8945774
fbshipit-source-id: 5e6f63981c3ed6695a687b4bfbaac97ea440b537
Summary:
Add a new `eden debug logging` command to show and update the current edenfs
logging settings.
I have deprecated the `eden debug set_logging_command`, but not removed it
entirely yet. It now longer shows up in the `eden debug --help` output, and
if run it prints the `eden debug logging` command that the user should run
instead.
Reviewed By: chadaustin
Differential Revision: D8907310
fbshipit-source-id: 7ed056637b236db5a50c619023bd8025b787e227
Summary:
This returns a `Repository` object and not a `Repository` subclass type.
This type information has been broken since D7512320 replaced the
`get_repo_class()` method with `create_repo()`.
Reviewed By: strager
Differential Revision: D8940415
fbshipit-source-id: 7711a2a33bc12fc361dfd0ff3f47eba1e587e6e7
Summary:
Add the FileChangeMonitor class to monitor for a file for changes. It provides
an interface "isChanged()" to check if the file has changed. It limits checks
by doing them on demand and by throttling. The throtte limits checks to 1 per
throttleMilliSeconds. It uses Stat information to determine if the file has
changed.
Reviewed By: simpkins
Differential Revision: D8898783
fbshipit-source-id: 067781489a9a91e5fe77cb613fba9d35b8348548
Summary:
InodePath is difficult to work with because it's just an alias to std::array. Make InodePath look more like std::string so using InodePath instances is more intuitive.
This diff should not change behaviour.
Reviewed By: chadaustin
Differential Revision: D8923073
fbshipit-source-id: 85a2548f0cfa61e50b6590048084076b9bece3da
Summary:
Make use of the TopLevelIgnores class to hold the system and user
GitIgnoreStack details. This is a cleaner implementation making ownership
semantics more intuitive. In later commits we will provide acess to the
TopLevelIgnores as part of the ServerState. It will be dynamically loaded.
Reviewed By: simpkins
Differential Revision: D8906226
fbshipit-source-id: d955436582498861ac4b4113a47f357432c8f32e
Summary:
Update `EdenFS.run_unchecked()` to return the full
`subprocess.CompletedProcess` argument rather than just the return code.
This will make it possible for future tests to check the output of the command
in addition to just its return code.
Reviewed By: strager
Differential Revision: D8905491
fbshipit-source-id: 7f17f5d103a06aeb56a3be0618683bd1c017fdf0
Summary:
D8508489 updated the integration test code so that the `capture_output`
argument to `EdenFS.run_cmd()` was no longer necessary and was not used.
However I forgot to actually remove the argument in that diff. This deletes
the argument.
Reviewed By: strager
Differential Revision: D8905492
fbshipit-source-id: d7a691f7d628a9fd5daa6723575e4454480f6aa1
Summary:
As of D8631281 `eden clone` now makes the mount point directory read-only and
puts a readme file inside of it. Since the directory is read-only the
integration tests are unable to clean it up using a normal `shutil.rmtree()`
call.
This fixes the integration tests to try making the directory writable first
if it encounters an error during cleanup.
Reviewed By: strager
Differential Revision: D8905488
fbshipit-source-id: c6b9d0b12c2b1dcb4a0839757cbcc3aad179bfc1
Summary:
This replaces the `StdoutPrinter` class with a different `Output` class
interface that combines both the `StdoutPrinter` functionality and the
underlying I/O object being printed to.
Previously the doctor code had to pass around two separate `out` and
`printer` objects. This simplifies things so that only one object has to be
passed around.
This also adds some additional error checking around the curses
initialization, and falls back to a plain output object if anything goes wrong
during setup. Previously we had seen some reports of `eden doctor` failing
due to errors in the terminal setup code.
Reviewed By: strager
Differential Revision: D8870494
fbshipit-source-id: a6dbafdde9aa5e1a0336fa7d723d83fa7d2426ca
Summary:
The raw Inode pointer in a DirEntry is more of an association than
ownership, so add comments and have clearInode return the old value.
Reviewed By: strager
Differential Revision: D8842315
fbshipit-source-id: d401dcdf4955ea335b39c2a57b0bedb1f83fdf9b
Summary:
When users run into "Transport endpoint not connected" errors they sometimes
attempt to run `eden status` or `eden doctor` to fix the problem. However
previously the CLI did not deal with being run from an invalid working
directory very well.
This updates the CLI to explicitly check if the current working directory is
valid before doing anything else. If it gets an ENOTCONN error it prints a
message telling the user how to update their shell's working directory. If
the `$PWD` environment variable points to a valid directory it attempts to
continue using that as the working directory. It records the fact that the
original working directory was invalid so that `eden doctor` can report this
error if that is the command being run.
Reviewed By: wez, strager
Differential Revision: D8731005
fbshipit-source-id: 4be55ccb43e22ed878114268e4dcc4c90c20e8e2
Summary:
Add a class to encapsulate details of top-level system and user ignore files.
This approach has the following benefits:
- simplifies memory management of the GitIgnore;
- separate structure that can be created and provided as part of ServerState
interface;
We will make use of it later in this commit stack.
Reviewed By: simpkins
Differential Revision: D8876064
fbshipit-source-id: 35c82b918c09e58068370401883edd8474dd3fbf
Summary: GitIgnore and GitIgnoreStack encapsulate details of ignore files. Further changes (as part of this commit stack) require copy constructors and assignment operators. Also, compiler wants destructors in .cpp file (since otherwise, it cannot tell the object size).Keeping this as a separate commit to simplify review process.
Reviewed By: simpkins
Differential Revision: D8730299
fbshipit-source-id: 7cb443906143c80209278b85fc5ad4dc5ea0bf56
Summary:
This adds initial code for an `eden debug fsck` command.
At the moment this simply scans the overlay and reports errors, but does not
correct any issues. In subsequent diffs I'll update it to support correcting
some problems.
There are also several other issues that I'll improve in subsequent diffs as
well. At the moment this code uses the same functions as other CLI commands
to find the client directory, and these only work if the checkout is already
mounted. For `fsck` we generally want the checkout to be unmounted before
checking it. I also plan to clean up some of the output handling as well.
Reviewed By: strager
Differential Revision: D8813714
fbshipit-source-id: 90238e94540db1cc06c17eabf8ce3a1bb808d4b3
Summary:
The methods were marked const when they were actually mutating the inner
objects, this was working because the pointed to objects were referenced via
`shared_ptr` handles
Reviewed By: yfeldblum
Differential Revision: D8851578
fbshipit-source-id: 4ca49390daf46f5c8681149da4d1e9eeab5aacb5
Summary:
While trying to make destroyWithInitRace non-flaky, it uncovered an
ASAN violation in the case that fuseCompleteFuture finishes during
EdenMount destruction. In that case, path_ gets destroyed prior to the
executor, so it's illegal to construct TakeoverData::MountInfo from
path_. This diff removes path_ entirely and reads it from
ClientConfig.
Reviewed By: simpkins, strager
Differential Revision: D8848663
fbshipit-source-id: f9368aa9eec7dfa8f2897cce55fad6d19723e30c
Summary:
We should only kick off prefetching for the files
that matched the glob. We were prefetching files that
didn't match the glob.
facepalms
Reviewed By: strager
Differential Revision: D8846994
fbshipit-source-id: 593e85d843ffa1cc0707ed1dc86f1385262821f5
Summary:
This makes it easier to add some test coverage.
There's no real functional change in this diff; the only code change is to
throw a system_error instead of a thrift eden error wrapper class from the core
globbing code. There's a little bit of code to restore this exception type in
the callers in EdenServiceHandler; this is covered by existing integration
tests, but I've also expanded that coverage to cover both variants of the glob
thrift calls.
Reviewed By: strager
Differential Revision: D8776767
fbshipit-source-id: 3ea4ea642ae5108aa4b0153541bd3604f010b54c
Summary:
Let's use consistent naming for the user ignore file. We will go with
'userIgnoreFile' as recomended. There is now symetry since we have
'systemIgnoreFile'.
I have another diff (later) that already uses the 'userIgnoreFile'.
Reviewed By: chadaustin
Differential Revision: D8823906
fbshipit-source-id: 47594e8971fa4db809821fc819da05e9eb2c5277
Summary:
Some of the heavier watchman->eden queries in www today
trigger calls to `getFileInformation` with very large numbers
of paths.
For a set of paths with a lot of common prefixes there is a lot
of repeated work to load same inodes over and over again.
eg: `a/b/c/d/A` and `a/b/c/d/B` both have to load `a -> b -> c -> d` before
they can load the leaf node.
This diff pre-processes the list of paths and builds a tree-shaped plan
so that we won't need to load any inode referenced by any of the paths
more than once.
The `getSHA1` method could benefit from a similar change; I'll do that in
a follow-up diff.
Reviewed By: strager
Differential Revision: D8578261
fbshipit-source-id: e899640a2ef6dcdb6b903d2a2735e9240496d74b
Summary:
The `eden debug overlay` code has some logic in python for parsing overlay
files. This moves that logic into a new module so I can re-use it for an
`fsck` command in a subsequent diff.
Reviewed By: wez
Differential Revision: D8813715
fbshipit-source-id: 6c32561db95f5da112cfd3e06b9e4653f431b94a
Summary:
This class name looks like it was copy-and-pasted incorrectly, probably from
the `eden debug set_log_level` code. Update the name to specify "Journal"
rather than "Level"
Reviewed By: strager
Differential Revision: D8813713
fbshipit-source-id: 2a5edd7f2de8279ee70dcc72b200c2b86b60f376
Summary:
Update `eden restart` to explain that the restart will be disruptive when
performing a full restart, and prompt the user for confirmation. This prompt
can be overridden with the `--force` option. Also print more messaging after
the restart completes telling users what to do if they see "Transport endpoint
not connected" errors from programs still using the old mount points.
This also updates `eden restart` to more carefully handle cases where there is
an existing edenfs daemon running but it is not fully healthy.
Reviewed By: wez
Differential Revision: D8731004
fbshipit-source-id: 05762187b47057b2930d0a6b71b0a6fdbd4aa7e5
Summary:
Fix two debug subcommands that were calling `os.getcwd()` while setting up the
argument parser. This caused the eden CLI to throw an exception of the
current working directory was invalid (for instance, if it was inside a stale
Eden mount point).
Reviewed By: strager
Differential Revision: D8731006
fbshipit-source-id: 8b7a67815d61b3476109aad646d21af8282ac68c
Summary:
EdenConfig uses stat to determine if config files have change. However, it
should use st_mtim.tv_nsec rather than st_mtime because this provides
nano-second resolution.
Reviewed By: simpkins
Differential Revision: D8797143
fbshipit-source-id: aadf114f509116d497a4c237ecaf108eb3824d5f
Summary:
This commit focuses on the following functionality:
- Eden configuration should have reasonable default values.
- Eden should allow configuration to be specified in various manners: default, cli, user and system configuration files.
- Configuration values should have the following precedence (hi to low): cli,
user config, system config, default.
- A light-weight mechanism is required to detect and update configuration changes.
Reviewed By: simpkins
Differential Revision: D8594297
fbshipit-source-id: ed195212669b18b450a1eae359e4d23905beadb4
Summary:
This refactors `HgBackingStore::getTreeForCommit()` so that it no longer invokes
`importThreadPool_.get()` if the `LocalStore` contains the `Tree` for the
specified `commitID` (which is commonly the case). When the `LocalStore` has a
cache hit, there is no need to import any data from Mercurial, so there is no
need to involve the `importThreadPool_`.
As explained in T30958650, `importThreadPool_.get()` can be relatively expensive
because it can have the side-effect of invoking
`HgImporterThreadFactory::newThread()`, which creates a new `HgImporter`, which
in turn does some potentially expensive work (it spawns a new subprocess for the
import helper and does some treemanifest-related I/O, if appropriate).
Because `HgBackingStore::getTreeForCommit()` is called as part of `hg status`,
the existing implementation was preventing us from reliably providing <30ms
`hg status` calls because we would frequently get "unlucky" and unnecessarily
create a new `HgImporter`.
Reviewed By: wez
Differential Revision: D8764805
fbshipit-source-id: 0acf7779560ddf182e7eb50daea54c5ee57030d0
Summary:
This is part of "the great r-valuification of folly::Future":
* This is something we should do for safety in general.
* Context: `Future::get(...)` means both `Future::get()` and `Future::get(Duration)`
* Using lvalue-qualified `Future::get(...)` has caused some failures around D7840699 since lvalue-qualification hides that operation's move-out semantics - leads to some use of future operations that are really not correct, but are not obviously incorrect.
* Problems with `Future::get(...) &`: it moves-out the result but doesn't invalidate the Future - the Future remains (technically) valid even though it actually is partially moved-out. Callers can subsequently access that moved-out result via things like `future.get(...)`, `future.result()`, `future.value()`, etc. - these access an already-moved-out result which is/can be surprising.
* Reasons `Future::get(...) &&` is better: its semantics are more obvious and user-testable. It moves-out the Future, leaving it with `future.valid() == false`.
Reviewed By: LeeHowes
Differential Revision: D8756764
fbshipit-source-id: 5c75c79cebcec77658a3a4605087716e969a376d
Summary:
This is part of "the great r-valuification of folly::Future":
* This is something we should do for safety in general.
* Context: `Future::get(...)` means both `Future::get()` and `Future::get(Duration)`
* Using lvalue-qualified `Future::get(...)` has caused some failures around D7840699 since lvalue-qualification hides that operation's move-out semantics - leads to some use of future operations that are really not correct, but are not obviously incorrect.
* Problems with `Future::get(...) &`: it moves-out the result but doesn't invalidate the Future - the Future remains (technically) valid even though it actually is partially moved-out. Callers can subsequently access that moved-out result via things like `future.get(...)`, `future.result()`, `future.value()`, etc. - these access an already-moved-out result which is/can be surprising.
* Reasons `Future::get(...) &&` is better: its semantics are more obvious and user-testable. It moves-out the Future, leaving it with `future.valid() == false`.
Reviewed By: yfeldblum
Differential Revision: D8711368
fbshipit-source-id: fbfcb731097cdf9d8d98583956bc7fe614157a6b
Summary:
This moves the fsattr helper program into the helpers/ directory
created in D8373672.
Reviewed By: chadaustin
Differential Revision: D8705391
fbshipit-source-id: 43e04d8b06b5d17caefafdfc750e3e60cad1b0cc
Summary:
An upcoming change to mercurial will make trees only serialize to disk at the
end of the command. Since Eden is bypassing commands and calling prefetch
directly, it needs to also manually tell Mercurial to serialize.
Reviewed By: chadaustin
Differential Revision: D8679211
fbshipit-source-id: 865ecaf340fa6003c48a6563f713479f07a29f16
Summary:
This is part of "the great r-valuification of folly::Future":
* https://fb.facebook.com/groups/fbcode/permalink/1726007767436055/
* This is something we should do for safety in general.
* Using lvalue-qualified `Future::get()` has caused some failures around D7840699 since lvalue-qualification hides that operation's move-out semantics - leads to some use of future operations that are really not correct, but are not obviously incorrect.
* Drill-down: `Future::get() &` moves-out the result but doesn't invalidate the Future - the Future remains (technically) valid even though it actually is partially moved-out. Callers can subsequently access that moved-out result via things like `future.get()`, `future.result()`, etc., which will cause surprising results - accessing an already-moved-out result. The semantics of `Future::get() &&` are more obvious: it moves-out the Future, leaving it with `future.valid() == false`.
Codemod steps (where `get(...)` means both `Future::get()` and `Future::get(Duration)`):
* expr.get(...) ==> std::move(expr).get(...) // if expr is not already an xvalue
* expr->get(...) ==> std::move(*expr).get(...)
Note: operator precedence of that last step is safe - no need to parenthesize `expr`. Reason: `->` binds more tightly than unary `*`.
Reviewed By: yfeldblum
Differential Revision: D8625157
fbshipit-source-id: 70323ff3189631ce5597d3c9b17629c99f5d457b
Summary:
Update "eden clone" to create a README file inside the mount point directory
before the checkout is mounted. When Eden is running normally this file will
not be visible because the Eden checkout has been mounted on top of this
directory. However if the checkout is not currently mounted users will see
this file instead.
This should hopefully make it easier for users to figure out how to remount
their checkout on their own, and should prevent them from worrying that their
files have been deleted when the mount point simply is not mounted.
Reviewed By: chadaustin, wez
Differential Revision: D8631281
fbshipit-source-id: 7610450983687a839bdbc3558ff35e0422edc3be
Summary:
The eden integration tests currently run hg through the telemetry wrapper.
This wrapper script adds profiling arguments to a percentage of the commands
it runs. The extra arguments that it adds are not entirely transparent to the
user, and are unfortunately reported in the output of "hg journal".
This was causing occasional test failures in the Eden integration tests that
check the output of "hg journal".
This diff changes the Eden code that checks the journal output to try to strip
off these flags before checking the command output.
Reviewed By: wez
Differential Revision: D8631180
fbshipit-source-id: 004d3bcce291ffc2208f85a29cb34a921f16fdde
Summary:
Update the python CLI code to let the C++ edenfs code handle daemonization
rather than daemonizing in the python code.
This means that we now daemonize after the `sudo` invocation rather than
before it. This allow's sudo's wrapper process to exit once edenfs starts,
rather than remaining around for the lifetime of the edenfs daemon. This
means that the original process' stdout and stderr file descriptors are
closed properly rather than being held open by the sudo wrapper.
This also removes the logic in the CLI that waits for edenfs startup and can
potentially time out and give up.
Reviewed By: strager
Differential Revision: D8508489
fbshipit-source-id: 6a38439d5596049d6f40cb6589b6865c229f736e
Summary:
Update the C++ edenfs code to ensure that the .eden and
.eden/storage/rocks-db directories exist, rather than requiring the python CLI
code create these directories as part of `eden start`
Reviewed By: strager
Differential Revision: D8508488
fbshipit-source-id: 358521b4f5eed1d19bf37903900ca50718e2c35c
Summary:
Update edenfs to print version information in its startup messages.
More detailed version information will be printed to the log file, but it
seems nice to also include this in user-facing messages that will be printed
to the user's terminal in most situations.
Reviewed By: chadaustin
Differential Revision: D8508486
fbshipit-source-id: 9364290ed470375120acd74cfaef1ccde41fd746
Summary:
This updates edenfs to be able to daemonize itself, and moves the
daemonization logic from the python CLI code into C++.
The main benefit of this is that we can now do a better job of reporting
messages to the user during start-up. We can log around potentially slow
operations (opening the RocksDB local store), and we can print messages
directly to the user if startup fails. Previously most failure messages would
go only to the eden log and would not be printed to the user's terminal.
This also fixes some issues where stdin and stdout were not closed properly
when daemonization was performed by the CLI. sudo needed access to these file
descriptors in case it needed to prompt for a password, and it would then hold
the descriptors open until edenfs exited.
Reviewed By: wez
Differential Revision: D8373672
fbshipit-source-id: 3272bff2208596f41d26e479c82c700d6c1efe11
Summary:
This updates the `eden doctor` code to define a `Problem` class, and to update
all of the checks to generate `Problem` objects. Problems that can be
automatically fixed derive from the `FixableProblem` subclass and implement a
`perform_fix()` method.
This provides a more consistent API to report which problems were found and
which ones can be automatically fixed. This also simplifies handling of the
`--dry-run` option, as each check no longer needs to individually check if
they are running in dry-run mode or not.
I also removed the `Check` API, since it only had a single `do_check()`
method. The code now simply uses a few separate helper functions that perform
various checks and generate `Problem` objects.
Reviewed By: wez
Differential Revision: D8617702
fbshipit-source-id: f7da2b7c17799b2844ae9416e827fd59b99d93c5
Summary:
Add a helper function for defining mount points during the test, rather than
requiring each test function to manually set up all of the individual
configurations for each mount point.
Reviewed By: wez
Differential Revision: D8581496
fbshipit-source-id: 41c8f1099292f7ebdbad30278ed530c01d083543
Summary:
Add a helper function that uses `self.addCleanup()` to destroy the temporary
directory rather than having to use `try...finally` blocks.
Reviewed By: strager
Differential Revision: D8581494
fbshipit-source-id: 3e1368c876b305c1528c5ed1de948a65a3da8199
Summary:
This updates the CLI code to put a timeout on `unmount()` thrift calls to
edenfs. We've had a few reports of situations where the unmount thrift call
has hung while edenfs was waiting for all inodes to be released.
Eventually we should fix the C++ code to put a hard timeout on the operation,
(and forcibly remove the mount point from its list in EdenServer even if it is
still waiting for some inodes to be released). However as an interim solution
timing out on the CLI is better than hanging forever. In particular the old
behavior made `eden rm` unable to clean up some mount points.
Reviewed By: pkaush
Differential Revision: D8558767
fbshipit-source-id: 942e5c7bcc147229f040f377ce59fcc2565276e0
Summary:
Previously `eden rm` tried to look at `<path>/.eden/root` to find the root of
the mount point. This failed if the checkout was not currently mounted.
This updates the code to try looking for the exact input path in the config
file in this case. It also avoids sending an `unmount()` thrift call to
edenfs in this case as well.
Reviewed By: strager
Differential Revision: D8558765
fbshipit-source-id: a5d9db7eee702817713b089a457f25710438661e
Summary:
Update `eden doctor` to automatically remount any checkouts that have been
unmounted.
In general we don't expect users to need to unmount checkouts. If they have
gotten into this state it is most likely unexpected, so have `eden doctor` fix
it for them. This is also consistent with how `edenfs` automatically remounts
all configured mount points on startup.
Reviewed By: chadaustin
Differential Revision: D8558766
fbshipit-source-id: 2e22cccead69d01530c6f646a958fc435ccce6c4
Summary:
When running the integration tests in parallel in a dev mode build I
occasionally see timeouts in the shutdown code in some tests. (Usually this
seems to happen in some of the remount tests that create multiple mount points
and shutdown and restart edenfs multiple times.)
Usually when I see this error edenfs does exit successfully on its own just
after this 15 second timeout has expired.
This bumps up the timeout to 30 seconds to avoid spurious failures.
Reviewed By: pkaush
Differential Revision: D8559052
fbshipit-source-id: 567062b08b34eced8f05538607ae9f4773c4aa9c
Summary:
We provide average values for vmRSS and memory (dirty bytes) on linux systems
by parsing the proc file system. This change allows the current values
(non-average) to be queried and calculated from the thrift API. The cli has
been updated to present these values (and no longer consumes the parsed smaps
files from EdenServer).
Reviewed By: chadaustin
Differential Revision: D8549305
fbshipit-source-id: 77f6838f39784e7ebeda11d8c66dba1fa9f10591
Summary:
This adds a small helper class for printing startup status messages to the
user, and for signaling startup status over a pipe.
This will make it possible in a future diff for edenfs to daemonize, but still
communicate messages back to the parent process to report what it is doing
until it finishes the start-up steps.
Reviewed By: strager
Differential Revision: D8372250
fbshipit-source-id: 53a897944beeb1582a090a2b69afbc2b41408d52
Summary:
Previously `eden list` reported normally running active checkouts with an
`(active)` suffix, and non-mounted checkouts with no suffix.
The non-mounted situation is rare. This diff changes the command to report
non-mounted checkouts with a `(not mounted)` suffix. I removed the `(active)`
suffix from normally-running checkouts now, but let me know if people think it
is still valuable to keep this annotation in the output.
Reviewed By: chadaustin
Differential Revision: D8558770
fbshipit-source-id: ccce0f7c2031e50f2847089e49007274c31a69a7
