Summary:
This moves some things around in order to facilitate adding the migration
command in a separate file.
Reviewed By: bolinfest
Differential Revision: D7946842
fbshipit-source-id: 54a554fb02e83a12f1d626b81377bc042fac41aa
Summary:
Before I fixed an issue with `nuclide-connections` in D7833410, it could throw
an error. Although there was a try/except around the `check_output` call in
`eden doctor`, it caught `OSError` rather than `CalledProcessError`, which
seems like a mistake (or maybe an inadvertent evolution of the code).
While there, I also extended it to catch `ValueError` in case the stdout of the
subprocess is not valid JSON, in which case `json.loads()` will raise a
`ValueError`.
Reviewed By: chadaustin
Differential Revision: D7890571
fbshipit-source-id: 184f6f669e9d62a5fb04db29bcbab450defc226e
Summary:
For sure, there is still more we can do to improve the output of `eden doctor`
to make it easier to scan, but I thought some color to classify the status
would be helpful. With this change, `eden doctor`:
* Prints green when no issues are encountered.
* Prints red when there are issues that could not be fixed.
* Prints yellow when there were issues that were fixed.
* Prints yellow when there were issues that were not fixed because it was a dry run.
In making this change, I took `StdoutPrinter` from `eden/cli/debug.py` and moved
it into its own file, `eden/cli/stdout_printer.py`. While there, I introduced an
`AnsiEscapeCodes` class that can be passed to the constructor of `StdoutPrinter`
so the client can specify its own ANSI escape codes. The unit test uses these to
ensure the test output will be the same independent of the value of
`sys.stdout.isatty()` when the test is run.
Reviewed By: chadaustin
Differential Revision: D7890525
fbshipit-source-id: a95ff8c1685b48c2d239923cf08456ec6de757fe
Summary:
This diff works a little harder to be able to successfully
stop buck in a repo. It does so by performing a single level glob
to find the main buckconfig files and then invoking buck kill in
each of those locations.
The output from buck is suppressed as we've had reports that it
was confusing.
I've removed the code that shutdown chg; it's been causing us
problems in our integration tests, and the problematic behavior
will soon be addressed in chg itself.
Reviewed By: chadaustin
Differential Revision: D7874975
fbshipit-source-id: e9755099b1d22f2b4e3684280eb95cb9c9d11a41
Summary:
I have an upcoming change that touches this file and I don't want to pull in
unrelated edits. (I thought we had already autoformatted all of this code?)
Reviewed By: chadaustin
Differential Revision: D7855072
fbshipit-source-id: 2d2ab2ce19d438af73c30471199d15db98fa4e3a
Summary:
While testing the fbclone --eden changes I found that
we were failing to pick up the bind mounts. The root cause of
this is that the default head rev for hg repos is `.`, so if
the source repo is bare then we won't find an `.arcconfig` file
and thus won't be able to set up the bind mounts.
Reviewed By: simpkins
Differential Revision: D7782489
fbshipit-source-id: f41d3a7daf39ecd0946707cb2c3211d70c36eea2
Summary:
These daemon processes tend to keep file handles open
and block shutdown in various annoying ways, so we terminate
them prior to stopping the eden server.
Note that the chg shutdown is linux specific. We need a supported
command that does the right thing because this is fragile and won't
work on macos. cc: quark-zju.
We'd talked about prompting the user about this; I can add prompting
as a follow on diff.
Reviewed By: chadaustin
Differential Revision: D7784903
fbshipit-source-id: 47a2f6395f99b2bfde3dbd049b6ca4804d011373
Summary:
We've been talking about this for a little while;
this changes the terminology to refer to checkouts rather
than clients because that concept feels more intuitive.
Reviewed By: simpkins
Differential Revision: D7784902
fbshipit-source-id: 5e1e9be9438eed7915a95ea4141c4452e86adbfc
Summary: We set `metavar=''` to suppress the default and make things look nicer.
Reviewed By: simpkins
Differential Revision: D7783417
fbshipit-source-id: 05f970e8f964cb38e153c509cd5c71dcaa3143d0
Summary:
I found this while testing the fbclone script; clone was
not performing realpath on the destination path, so we ended up
creating and registering a mount under a non-canonical name.
That doesn't sound so bad except that unmount always canonicalizes
the path to be unmounted and this made it impossible to unmount
such a repo.
Reviewed By: chadaustin
Differential Revision: D7766366
fbshipit-source-id: cbfd484a1481d5082969fc97eaf211c18c0d30be
Summary:
Only treat edenfs mount points as stale if attempting to access them fails with
ENOTCONN. If we can successfully access the mount point then it cannot be
stale and does not need to be unmounted.
This prevents `eden doctor` from incorrectly unmounting mount points owned by
another edenfs process running on this system.
Reviewed By: bolinfest
Differential Revision: D7683503
fbshipit-source-id: 412cc17442fac1e8030dd40de0e0f642c15d1253
Summary:
This command directly invoked Eden's `checkOutRevision()` thrift call. This
bypasses mercurial, so mercurial would not know about the working directory
parent change.
If we need to we could add this back as a debug command in the future.
However, this isn't something that normal users should ever invoke directly.
Reviewed By: chadaustin
Differential Revision: D7754529
fbshipit-source-id: 8d6aad70da37e56b3b8ced1ade4c8537141ab239
Summary:
Update the Eden CLI command names to more closely match the command names used
by other Unix tools like systemctl and runit.
* `eden daemon` --> `eden start`
* `eden shutdown` --> `eden stop`
* `eden health` --> `eden status`
I have added aliases for the old command names so that they still work.
Reviewed By: chadaustin
Differential Revision: D7753780
fbshipit-source-id: af2c95dd5bcad09c395496f1cd5def082bccdc5c
Summary:
Try treating the argument as a repository path first, rather than a well-known
repository name. If we can find a mercurial or git repository at this path,
look for a .arcconfig file in the repository, and try to identify the
repository project type from this file. Use this project name to load our
default bind-mount and hooks configuration for the repository.
If the argument is not a valid repository path, fall back to treating it as a
well-known project name in our repository configs.
This also updates `eden clone` to print a few more diagnostic messages as it
works. (For instance, printing when it is starting the edenfs daemon, and
reporting the commit ID that was checked out.)
Reviewed By: wez
Differential Revision: D7739917
fbshipit-source-id: ac927b9e93039e4d1b8afa80466c2eee3a8829e9
Summary:
Define an `add_mount()` method to simplify the process of defining mounts in
the FakeMountTable. Also simplify the `lstat()` code to only try lookups using
strings. The keys in the dictionary are always strings, so there is no need to
ever try looking up entries using bytes.
Reviewed By: chadaustin
Differential Revision: D7704335
fbshipit-source-id: 9c81b43c0a196287254bcd749d096f9b148af8ac
Summary:
Add a pretty basic `restart` command to the Eden CLI.
This is relatively simple at the moment. It has `--graceful` and `--full`
options, and defaults to a full restart at the moment if neither of these
arguments is specified. We probably could add most of the arguments from the
`daemon` command to `restart`, but I'm not sure if we really want to support
that complexity in this command.
In the future we can make this command smarter and have it do a better job of
reporting the status as Eden restarts.
Reviewed By: chadaustin
Differential Revision: D7682838
fbshipit-source-id: 80c8d6604c3ca120fb71d6b8a388309218b5500c
Summary:
Refactor the Eden CLI command so that all subcommands are implemented as
subclasses. This helps keep the command line argument definitions together
with the logic for the command.
This is primarily just a code refactoring change, but I did include a few minor
behavioral changes to the help output:
- The command list is now always sorted alphabetically in the help output.
- The "help" subcommand can now show help for more than just one subcommand
deep. (e.g., `eden help stats io` now works correctly)
- I made some minor improvements to a few of the help strings.
Reviewed By: chadaustin
Differential Revision: D7673021
fbshipit-source-id: dc4c6db20a0fe7452d38bdafc6273e234dba8e4e
Summary:
Temporary fix to bump the takeover timeout to five minutes in the cli
code. I messed up and forgot to ship these changes with D7660642.
Reviewed By: simpkins
Differential Revision: D7676402
fbshipit-source-id: a22963fbf89663e44537e7946b3be5eb870f930c
Summary:
In practice, we have received a number of user reports where Nuclide had lost
track of its Watchman subscriptions and `eden doctor` failed to report it.
When the `NuclideHasExpectedWatchmanSubscriptions` check was first added to
`eden doctor`, in the repro case we had, it seemed that Nuclide would lose its
`filewatcher-<root-folder>` subscription, but would retain its
`hg-repository-watchman-subscription-primary` subscription. As such, we encoded
a heuristic where the presence of the `hg-repository-watchman-subscription-primary`
subscription without the accompanying `filewatcher-<root-folder>` subscription
was taken as a sign that (1) the user was connected to the directory via
Nuclide and (2) Nuclide's Watchman subscriptions had gotten corrupted.
Unfortunately, this was not a sufficient heuristic because it would not detect
the error case where Nuclide had lost //all// of its Watchman subscriptions!
Because `eden doctor` did not flag this, users kept using Nuclide even though
things were in a bad state, sometimes losing data as a result.
To fix that, we introduced a new tool, `nuclide-connections`, in D7598900. This
executable prints the list of local folders used as root folders in Nuclide
clients connected to the local Nuclide server. Now we no longer have to rely on
the presence of the `hg-repository-watchman-subscription-primary` Watchman
subscription as a proxy for a Nuclide connection.
This commit updates the `NuclideHasExpectedWatchmanSubscriptions` check in
`eden doctor` to use `nuclide-connections`. For every Nuclide connection that is
also an Eden mount, it verifies that the expected Watchman subscriptions exist.
As noted in the TODO in `NuclideHasExpectedWatchmanSubscriptions`, this
check is a bit brittle, but it is intended as a stopgap until we find the root cause
of the missing Watchman subscriptions and fix it.
Reviewed By: simpkins
Differential Revision: D7663214
fbshipit-source-id: 200b78ce25010222e59f3296f2412161086aa7ce
Summary: Materialized entries are stored with an unset hash. Fix the CLI.
Reviewed By: simpkins
Differential Revision: D7465192
fbshipit-source-id: 41e50c7214feb8afb1f3bf47252e4538945a5737
Summary:
My muscle memory doesn't quite know where the edenfs log file
is, so add a shortcut command to view it.
Reviewed By: simpkins
Differential Revision: D7418743
fbshipit-source-id: 290b51b63fc3573663ad53f44e50dfd073c211c8
Summary:
Update the Eden CLI to preserve the `ASAN_OPTIONS` environment variable when
starting edenfs.
Reviewed By: wez
Differential Revision: D7440148
fbshipit-source-id: c9932c5c14b5a2e040309268f95229c04715bb28
Summary:
Update the `eden clone` command to accept extra command line flags telling it
where to find the edenfs executable, as well as additional arguments to pass
to edenfs.
This enables the `test_clone_should_start_daemon()` integration test to pass in
flags to tell it how to start edenfs correctly.
Reviewed By: chadaustin
Differential Revision: D7433367
fbshipit-source-id: 0eddd93d1332e113fb85fa4b8fc87ba51d7eab2c
Summary:
Update most of the Eden CLI code to include python type annotations.
I believe the stats.py and stats_print.py are the only CLI files that do not
have complete typing information now.
Reviewed By: chadaustin
Differential Revision: D7433368
fbshipit-source-id: dfd6a064cacffeeed9147739da7064f3303de789
Summary:
Request data was coming back as null because of a race between ctx->getContextData() in FuseChannel::getOutstandingRequests() and the it initialization.
Also, added a check to verify rdata is valid before dereferencing it.
Plus, fixed debug.py where output gave an error.
Reviewed By: chadaustin
Differential Revision: D7377993
fbshipit-source-id: 8343119983c74185fd5d8cc05c2f5af63dcff99e
Summary:
Added a thrift call to return the outstanding FUSE requests.
Cli will call the thrift and print the output.
Added a unit test to test getOutstandingRequests().
Reviewed By: simpkins
Differential Revision: D7314584
fbshipit-source-id: 420790405babdb734f598e19719b487096ec53ca
Summary:
If you run `eden daemon` on a machine where sudo needs input, sudo's
output would get redirected to edenfs.log and eden daemon would appear
to hang. Worse, if you pressed ctrl-C, sudo would remain in the
background and continue to swallow keypresses from the tty, appearing
to somewhat break your shell until it gave up. The fix is to stop
redirecting stdout and stderr from Python and instead have edenfs
redirect itself into the log if given.
Reviewed By: simpkins
Differential Revision: D7223399
fbshipit-source-id: bae17b150b6594a3dd87bb88741c8dcefd1c5213
Summary:
This removes the TARGETS files from the eden github repository. The
open source buck build has been failing for several months, since buck
removed support for the thrift_library() rule.
I will potentially take a stab at adding CMake build support for Eden
at some point in the future.
Reviewed By: chadaustin
Differential Revision: D6893233
fbshipit-source-id: e6023094a807cf481ac49998c6f21b213be6c288
Summary:
We would immediately satisfy the health check and
tell the user that the system is healthy and show the pid of
the prior incarnation rather than the one for the instance
that we just launched.
This diff refactors the health checking code so that we can
share the implementation between the cli and the integration
tests; the integration tests already had code to do the right
thing for this.
Reviewed By: simpkins
Differential Revision: D6944989
fbshipit-source-id: 7c0f02c875b1b81f8f1b7521add67928200b27ed
Summary:
If we fail to lstat a mount because the FUSE daemon is
unconnected, go ahead and unmount it.
Reviewed By: simpkins
Differential Revision: D6848115
fbshipit-source-id: df797b15956db24c4ac87dc6d4cd0a7177dd20ef
Summary:
This is a fairly minimal attempt at asking sandcastle to
run our tests as root. This is inspired by D6608746 but is for
the moment pretty coarse; I just want to see how successful this
is before finishing this off nicely, which means splitting our
contbuild config into two pieces along similar lines as D6608746;
one piece would run the tests as usual without root privileges
whilst the other would run just the integration tests with root
privs.
Reviewed By: chadaustin
Differential Revision: D6612669
fbshipit-source-id: 9a730b02424e939dd3509310b6265d50d1102e77
Summary:
in our sandcastle environment, eden is not currently
installed and the tests that look at the version are throwing
an error when the rpm query fails.
Don't do that!
Reviewed By: simpkins
Differential Revision: D6853305
fbshipit-source-id: d17cb1fb0253ef5ae5000695b6ec5bcda4a6a448
Summary: A minor convenience for mass mounting and unmounting.
Reviewed By: simpkins
Differential Revision: D6803003
fbshipit-source-id: 16c0d6982ba0ce2dba9900ee15013fcec8bc5ad5
Summary:
My stale mounts check regressed eden doctor's handling of
unconfigured mounts. For now, just skip them, and add a test that
eden doctor doesn't crash.
Reviewed By: simpkins
Differential Revision: D6801445
fbshipit-source-id: 65d2f9028af651ef487a7ce6c334f387b541492d
Summary:
The prior implementation of StaleMountsCheck filtered by path
and did not correctly handle seeing the same FUSE mount multiple times
in the mount table. This occurred when an Eden mount was created
underneath a bind mount.
Now it only unmounts mounts where st_dev does not match the st_dev of
any active mounts, and where st_uid matches the current user.
Reviewed By: simpkins
Differential Revision: D6787618
fbshipit-source-id: 24e0f156cb74822500d91205349c0e6638c0340c
Summary:
It's interesting to see the total number of loaded files
vs. trees when the loaded inode count is high.
Reviewed By: wez
Differential Revision: D6765874
fbshipit-source-id: 178b30184428bd5cf5e005eb475e4f5a1476c385
Summary:
I'm not sure what was wrong with the old code, but I
simplified and clarified all of the time math and now `eden debug
unload` behaves as I'd expect it should.
Reviewed By: simpkins
Differential Revision: D6764962
fbshipit-source-id: 3ed359d4ab4652e95d1538a0982c24185999351c
Summary:
bolinfest has observed `eden doctor` unmounting active mounts, so disable
that check until this is resolved.
Reviewed By: bolinfest
Differential Revision: D6768284
fbshipit-source-id: 21add1fa283f8631055019dae99819d04e179d5e
Summary:
On closer investigation, I realized that the typechecker is
complaining about a real problem here, that was apparently hidden by
the stubs. The problem is explained in
https://our.intern.facebook.com/intern/dex/qa/1596320410425583/
It's not safe to `import thrift` followed by accessing
`thrift.Thrift.TException`; it only happens to work at runtime if some
intervening code has explicitly imported `thrift.Thrift` submodule for
you.
This diff fixes the code to be safer and also make mypy happy.
Reviewed By: chadaustin
Differential Revision: D6761089
fbshipit-source-id: 5c6fdfa0bef893c4aea17ade8e7bfacaf7c0c8cc
Summary:
As noted by the TODO, enough time has passed since the migration to the new
directory layout has been available that we should feel comfortable deleting the
migration code.
Reviewed By: wez
Differential Revision: D6762457
fbshipit-source-id: 9b4a9ad1a0204878a419362c8cfbdcbb500b1aee
Summary:
If the Eden process dies or malfunctions, it's possible to end up with stale
edenfs mounts on the system. Change `eden doctor` to correct them.
Reviewed By: simpkins
Differential Revision: D6659193
fbshipit-source-id: d9fcf2e68663295e4f43b2c11fd4503a1dfac038
Summary:
Added --version to main.py, including -v and version.
It prints both installed and running version (as per Wez's suggestion), and accounts for the possibilities that eden may not be running at all, or dev version may be running.
Reviewed By: wez
Differential Revision: D6724204
fbshipit-source-id: 5085f53a00a557f759a23fe41fb57189c9ad6a7e
Summary:
D6738525 and D6731821 (https://github.com/python/typeshed/pull/1827)
broke type checking on Eden's Python. This works around removing the thrift
stubs.
Reviewed By: carljm
Differential Revision: D6745233
fbshipit-source-id: 760f7ee199698a0a570a993f16b45c833910aade
Summary:
Changed `eden clone` to check out master commit in both git and hg.
Previously, it checked out the current commit for the repo.
Reviewed By: simpkins
Differential Revision: D6663754
fbshipit-source-id: 92b185ccca5d082dc2bde9c8b191c82a2a4f06b4
Summary:
`eden list` enumerates all of the mounts in the Eden config,
whether or not the daemon is running and whether or not the mount is
currently active or not. This diff adds an (active) suffix on a mount
if the daemon is running and the mount is currently active.
Reviewed By: simpkins
Differential Revision: D6661303
fbshipit-source-id: c098e90fc9a77f16c723c707cc4da3ee3d4c5abb
