Summary:
This diff works a little harder to be able to successfully
stop buck in a repo. It does so by performing a single level glob
to find the main buckconfig files and then invoking buck kill in
each of those locations.
The output from buck is suppressed as we've had reports that it
was confusing.
I've removed the code that shutdown chg; it's been causing us
problems in our integration tests, and the problematic behavior
will soon be addressed in chg itself.
Reviewed By: chadaustin
Differential Revision: D7874975
fbshipit-source-id: e9755099b1d22f2b4e3684280eb95cb9c9d11a41
Summary:
While testing the fbclone --eden changes I found that
we were failing to pick up the bind mounts. The root cause of
this is that the default head rev for hg repos is `.`, so if
the source repo is bare then we won't find an `.arcconfig` file
and thus won't be able to set up the bind mounts.
Reviewed By: simpkins
Differential Revision: D7782489
fbshipit-source-id: f41d3a7daf39ecd0946707cb2c3211d70c36eea2
Summary:
These daemon processes tend to keep file handles open
and block shutdown in various annoying ways, so we terminate
them prior to stopping the eden server.
Note that the chg shutdown is linux specific. We need a supported
command that does the right thing because this is fragile and won't
work on macos. cc: quark-zju.
We'd talked about prompting the user about this; I can add prompting
as a follow on diff.
Reviewed By: chadaustin
Differential Revision: D7784903
fbshipit-source-id: 47a2f6395f99b2bfde3dbd049b6ca4804d011373
Summary:
We've been talking about this for a little while;
this changes the terminology to refer to checkouts rather
than clients because that concept feels more intuitive.
Reviewed By: simpkins
Differential Revision: D7784902
fbshipit-source-id: 5e1e9be9438eed7915a95ea4141c4452e86adbfc
Summary:
I found this while testing the fbclone script; clone was
not performing realpath on the destination path, so we ended up
creating and registering a mount under a non-canonical name.
That doesn't sound so bad except that unmount always canonicalizes
the path to be unmounted and this made it impossible to unmount
such a repo.
Reviewed By: chadaustin
Differential Revision: D7766366
fbshipit-source-id: cbfd484a1481d5082969fc97eaf211c18c0d30be
Summary:
This command directly invoked Eden's `checkOutRevision()` thrift call. This
bypasses mercurial, so mercurial would not know about the working directory
parent change.
If we need to we could add this back as a debug command in the future.
However, this isn't something that normal users should ever invoke directly.
Reviewed By: chadaustin
Differential Revision: D7754529
fbshipit-source-id: 8d6aad70da37e56b3b8ced1ade4c8537141ab239
Summary:
Update the Eden CLI command names to more closely match the command names used
by other Unix tools like systemctl and runit.
* `eden daemon` --> `eden start`
* `eden shutdown` --> `eden stop`
* `eden health` --> `eden status`
I have added aliases for the old command names so that they still work.
Reviewed By: chadaustin
Differential Revision: D7753780
fbshipit-source-id: af2c95dd5bcad09c395496f1cd5def082bccdc5c
Summary:
Try treating the argument as a repository path first, rather than a well-known
repository name. If we can find a mercurial or git repository at this path,
look for a .arcconfig file in the repository, and try to identify the
repository project type from this file. Use this project name to load our
default bind-mount and hooks configuration for the repository.
If the argument is not a valid repository path, fall back to treating it as a
well-known project name in our repository configs.
This also updates `eden clone` to print a few more diagnostic messages as it
works. (For instance, printing when it is starting the edenfs daemon, and
reporting the commit ID that was checked out.)
Reviewed By: wez
Differential Revision: D7739917
fbshipit-source-id: ac927b9e93039e4d1b8afa80466c2eee3a8829e9
Summary:
Add a pretty basic `restart` command to the Eden CLI.
This is relatively simple at the moment. It has `--graceful` and `--full`
options, and defaults to a full restart at the moment if neither of these
arguments is specified. We probably could add most of the arguments from the
`daemon` command to `restart`, but I'm not sure if we really want to support
that complexity in this command.
In the future we can make this command smarter and have it do a better job of
reporting the status as Eden restarts.
Reviewed By: chadaustin
Differential Revision: D7682838
fbshipit-source-id: 80c8d6604c3ca120fb71d6b8a388309218b5500c
Summary:
Refactor the Eden CLI command so that all subcommands are implemented as
subclasses. This helps keep the command line argument definitions together
with the logic for the command.
This is primarily just a code refactoring change, but I did include a few minor
behavioral changes to the help output:
- The command list is now always sorted alphabetically in the help output.
- The "help" subcommand can now show help for more than just one subcommand
deep. (e.g., `eden help stats io` now works correctly)
- I made some minor improvements to a few of the help strings.
Reviewed By: chadaustin
Differential Revision: D7673021
fbshipit-source-id: dc4c6db20a0fe7452d38bdafc6273e234dba8e4e
Summary:
Temporary fix to bump the takeover timeout to five minutes in the cli
code. I messed up and forgot to ship these changes with D7660642.
Reviewed By: simpkins
Differential Revision: D7676402
fbshipit-source-id: a22963fbf89663e44537e7946b3be5eb870f930c
Summary:
Update the `eden clone` command to accept extra command line flags telling it
where to find the edenfs executable, as well as additional arguments to pass
to edenfs.
This enables the `test_clone_should_start_daemon()` integration test to pass in
flags to tell it how to start edenfs correctly.
Reviewed By: chadaustin
Differential Revision: D7433367
fbshipit-source-id: 0eddd93d1332e113fb85fa4b8fc87ba51d7eab2c
Summary:
Update most of the Eden CLI code to include python type annotations.
I believe the stats.py and stats_print.py are the only CLI files that do not
have complete typing information now.
Reviewed By: chadaustin
Differential Revision: D7433368
fbshipit-source-id: dfd6a064cacffeeed9147739da7064f3303de789
Summary: A minor convenience for mass mounting and unmounting.
Reviewed By: simpkins
Differential Revision: D6803003
fbshipit-source-id: 16c0d6982ba0ce2dba9900ee15013fcec8bc5ad5
Summary:
As noted by the TODO, enough time has passed since the migration to the new
directory layout has been available that we should feel comfortable deleting the
migration code.
Reviewed By: wez
Differential Revision: D6762457
fbshipit-source-id: 9b4a9ad1a0204878a419362c8cfbdcbb500b1aee
Summary:
If the Eden process dies or malfunctions, it's possible to end up with stale
edenfs mounts on the system. Change `eden doctor` to correct them.
Reviewed By: simpkins
Differential Revision: D6659193
fbshipit-source-id: d9fcf2e68663295e4f43b2c11fd4503a1dfac038
Summary:
Added --version to main.py, including -v and version.
It prints both installed and running version (as per Wez's suggestion), and accounts for the possibilities that eden may not be running at all, or dev version may be running.
Reviewed By: wez
Differential Revision: D6724204
fbshipit-source-id: 5085f53a00a557f759a23fe41fb57189c9ad6a7e
Summary:
Changed `eden clone` to check out master commit in both git and hg.
Previously, it checked out the current commit for the repo.
Reviewed By: simpkins
Differential Revision: D6663754
fbshipit-source-id: 92b185ccca5d082dc2bde9c8b191c82a2a4f06b4
Summary:
`eden list` enumerates all of the mounts in the Eden config,
whether or not the daemon is running and whether or not the mount is
currently active or not. This diff adds an (active) suffix on a mount
if the daemon is running and the mount is currently active.
Reviewed By: simpkins
Differential Revision: D6661303
fbshipit-source-id: c098e90fc9a77f16c723c707cc4da3ee3d4c5abb
Summary:
`eden doctor` is a new subcommand that attempts to diagnose
issues with Eden and autofix them, as appropriate. There is also
a `--dry-run` flag that will tell the user about issues without attempting
to fix them.
Initially, `eden doctor` checks for the following:
- If Watchman has an `inotify` watcher, `eden doctor` identifies this and
will attempt to replace it with an `eden` watcher.
- If there are some active Watchman subscriptions that appear to come
from Nuclide, warn the user if the mission-critical `filewatcher-` subscription
that watches the entire repo is missing.
- If p1 in `SNAPSHOT` and `.hg/dirstate` do not match, warn the user.
The code for `eden doctor` is organized so that it should be relatively
easy to add new conditions to check for going forward. Admittedly, the
UX could be better by formatting the output (color, boldness, etc.) to
make certain information stand out, but we can improve that in
subsequent revisions.
Note that I had to do a bit of cleanup in `eden/cli/TARGETS` as part of
this revision and I created `eden/cli/test/TARGETS` so the tests have
their own build file.
Reviewed By: chadaustin
Differential Revision: D6446057
fbshipit-source-id: ae23c5996dba4f7f70f118179e5556efc29c31c3
Summary:
Increase the eden daemon start timeout from 5 to 60 seconds.
With the 5 second timeout I sometimes see `buck test eden/...` time out in the
clone integration test which waits for eden to start. This likely takes longer
than normal when lots of other integration tests are also running in parallel.
Even when simply manually starting eden I have seen it take nearly 60 seconds
to open the RocksDB database. (I suspect RocksDB performs some extra checking
when opening a DB created by an older version of the code.) Eden requires the
RocksDB be open before it starts responding successfully to health check
requests.
Reviewed By: wez
Differential Revision: D6434357
fbshipit-source-id: 5f62ff821ecd64f7a1345e611f2299177513e411
Summary:
In onboarding users, we usually tell them to run `eden clone fbsource`,
but that fails because users generally have not run `eden daemon` yet.
The simplest thing is to do it for them when they run `eden clone` when
the daemon is not running.
Reviewed By: wez
Differential Revision: D6357249
fbshipit-source-id: dc112c1efe214485e3c5c8e06522d299a100d3a0
Summary:
This begins implementing the "client-side" portion of graceful takeover in
edenfs. When the --takeover flag is specified, if edenfs finds that another
edenfs process is already running it will attempt to gracefully take over its
mount points rather than exiting with an error.
This does not yet actually take over the mount points themselves--it still
sends dummy mount information during shutdown, and does not use this data
during startup. However, we do perform takeover of the eden lock file and the
thrift server socket.
Reviewed By: bolinfest
Differential Revision: D6038944
fbshipit-source-id: 42406a0559367cec79af088b4ca84c22de3f3ef3
Summary:
When cloning an existing Eden mount, we should be smart and inherit its
underlying config so that we inherit properties such as its bind mounts.
Reviewed By: wez
Differential Revision: D6322002
fbshipit-source-id: 3f5ba135b12ad7dcecef6676d27495cfbf0ce97b
Summary:
Previously, a user had to define a config for a repo in a file like `~/.edenrc`
in order to create a new Eden mount via `eden clone`. In practice, the
information that is hardcoded in the config can generally be inferred from an
existing repo, so this expands `eden clone` to support both modes of operation.
Note this made it possible to finally unify the `RepoConfig` and `ClientConfig`
types. This revision removes `RepoConfig`, so I dutifully renamed every
local variable named `repo_config` to `client_config`.
Reviewed By: wez
Differential Revision: D6314115
fbshipit-source-id: 9625a5fbe35b30f76b6099180580c64435a4cf72
Summary:
The headline changes of this revision are:
- Changes the format of the config file from INI to TOML
(the `edenrc` file under `~/local/.eden` has been replaced
with `config.toml`). This revision includes logic for automatically
performing the migration when Eden is restarted.
- Inlines data from `/etc/eden/config.d` into the TOML file.
Historically, the `edenrc` file for a client would contain the
name of the "configuration alias" defined in a config file like
`~/.edenrc` or `/etc/eden/config.d/00-defaults`. When Eden
loaded a client, it would have to first read the `edenrc` and
then reconstitute the rest of the client configuration by
looking up the alias in the set of config files that were used to
create the client in the first place.
This changes things so that all of the data that was being
cross-referenced is now inlined in the client's config file.
This makes loading a config considerably simpler at the cost
of no longer being able to change the config for multiple clients
that were cloned from the same configuration alias in one place.
It was questionable whether being able to modify a client from
a foreign config after it was created was a safe thing to do, anyway.
Eliminating the need for a historic link to the configuration alias
will make it easier to support running `eden clone` on an arbitrary
local Hg or Git repo. So long as `eden clone` can extract enough
information from the local repo to create an appropriate config file
for the new Eden client, there is no need for a configuration alias
to exist a priori.
Since we were already changing the data in the config file, this
seemed like an appropriate time to make the switch from INI to
TOML, as this was something we wanted to do, anyway.
In testing, I discovered a discrepancy between how boost's
`boost::property_tree::ptree` and Python's `ConfigParser` handled
the following section heading:
```
[repository ZtmpZsillyZeden-clone.LIkh32]
```
Apparently `hasSection("repository ZtmpZsillyZeden-clone.LIkh32")`
in boost would fail to find this section. Because
[[https://stackoverflow.com/questions/13109506/are-hyphens-allowed-in-section-definitions-in-ini-files | there is no spec for INI]],
it is not that surprising that boost and `ConfigParser` do not 100% agree
on what they accept. Moving to TOML means we have a configuration
language with the following desirable properties:
- It has a formal spec, unlike INI. This is important because there are parsers
in a wide range of programming languages that, in theory, accept a consistent
input language.
- It is reasonable for humans to write, as it supports comments, unlike JSON.
- It supports nested structures, like maps and arrays, without going crazy
on the input language it supports, unlike YAML.
Eden now depends on the following third-party TOML parsers:
* C++ https://github.com/skystrife/cpptoml
* Python https://github.com/uiri/toml
This revision also changes the organization of `~/local/.eden` slightly. For now,
there is still a `config.json` file, but the values are no longer hashes of the realpath
of the mount. Instead, we take the basename of the realpath and use that as the
name of the directory under `~/local/.eden/clients`. If there is a naming collision, we
add the first available integral suffix. Using the basename makes it easier to
navigate the `~/local/.eden/clients` directory.
Although the `edenrc` file under `~/local/.eden/clients` has been switched from INI
to TOML, the other Eden config files (`~/.edenrc` and `/etc/eden/config.d/*`) still use
INI. Migrating those to TOML will be done in a future revision.
Note this revision allowed us to eliminate `facebook::eden::InterpolatedPropertyTree`
as well as a number of uses of boost due to the elimination of
`ClientConfig::loadConfigData()` in the C++ code. Because `ClientConfig`
no longer does interpolation, a bit of `ClientConfigTest` was deleted as part of
this revision because it is no longer relevant.
Reviewed By: wez
Differential Revision: D6310325
fbshipit-source-id: 2548149c064cdf8e78a3b3ce6fe667ff70f94f84
Summary:
This flips the sense of the unmount command; previously
we would default to destroying the mount and associated state,
but this was a bit of a massive sharp edge to our UX.
Now the default is non-destructive and you have to explicitly
pass in `--destroy` to enable destructive mode.
Reviewed By: chadaustin
Differential Revision: D6346013
fbshipit-source-id: ce612e7d8a70540d63217a97f96bc5760f3951af
Summary:
This will facilitate an upcoming change where I include the `hooks` information
as part of the data returned by `get_repo_config()`
(formerly `get_repo_data()`).
Reviewed By: chadaustin
Differential Revision: D6311536
fbshipit-source-id: 2e1bc063c4a12772c9806f6880822fb15e18fc92
Summary: Adds an `eden stats thrift` command that prints call counts for each Thrift call, similar to `eden stats io` printing FUSE counts.
Reviewed By: bolinfest
Differential Revision: D6189543
fbshipit-source-id: 59c29a4036687e1bf75b1c0ca2a7d311b9d1399f
Summary:
Using this option prints the rage report to stdout rather than to the default
reporter. Sometimes it's easier to get the report on stdout so you can pipe it
to other commands.
Reviewed By: simpkins
Differential Revision: D6195684
fbshipit-source-id: 59e1ae6b19b951eac67377cdd5f8c0560e0d181a
Summary:
This was an error that an end-user ran into. Previously, we did not fail
gracefully and the user was faced with an intimidating stacktrace.
Reviewed By: simpkins
Differential Revision: D6195529
fbshipit-source-id: bde3c2a3e6f49457a4c6ac5c87103cf52cd227c2
Summary:
In practice, we have seen cases where `eden shutdown` does not succeed within
15 seconds (the default timeout), and then we see the following message printed
to the console:
error: sent shutdown request, but edenfs did not exit within 15.0 seconds
Sometimes we end up in a state where the Thrift server has successfully shutdown
but the `edenfs` process is still running. Because `eden health` is implemented
using a Thrift call, it reports that `edenfs` is not running even though it
still is. This is confusing because a subsequent `eden daemon` request will
likely fail with something like:
```
terminate called after throwing an instance of 'std::runtime_error'
what(): another instance of Eden appears to be running for /home/mbolin/local/.eden
```
To address this, we make `eden shutdown` more aggressive in killing `edenfs`
by sending `SIGKILL`.
Follow-up work:
- Make Eden's shutdown process more robust so that `SIGKILL` is rarely, if ever,
necessary.
- Store the PID of the `edenfs` process in the lock file so that Eden does not
rely upon the Thrift server to be running in order to check whether the
`edenfs` process is running.
Reviewed By: wez
Differential Revision: D6080929
fbshipit-source-id: 462a6ec506cdb6a51211d2c64b777eadbcee9fc1
Summary:
This adds support for specifying where the output of `eden rage` should be sent
by adding a `reporter` property to the `[rage]` section of `~/.edenrc` like so:
```
[rage]
reporter = arc paste
```
If specified, Eden will take the command as an opaque string and use it as an
argument to `sh -c <reporter_from_edenrc>` when spawning a subprocess that will
receive the output of `eden rage` on its stdin. If no reporter is specified,
then the stream will be written directly to stdout.
Reviewed By: wez
Differential Revision: D6049355
fbshipit-source-id: beafe37eed8c48dac7499a6978cacfb44d8f253e
Summary:
Add an `--strace <FILE>` flag to the `eden daemon` CLI command to run eden
under strace, saving the strace output to the specified path.
Reviewed By: wez
Differential Revision: D5771462
fbshipit-source-id: fe4bf18f372f3276400bee624e906ed4f3569735
Summary:
Added new tool to report stat information of EdenFs like fuse counters, Memory counters, latencies, Inode status for all the mount points etc.
eden stat : Prints the general information about eden like list of mount points, loaded unloaded and materialized inodes in each mount point. Also this reports how well periodic unload job is doing by reporting the number of unloaded inodes by periodic job.
eden stat io : Prints how many number of calls made to a system call in Edenfs.
eden stat memory : returns the memory stat for edenfs.
eden stat latency : reports the latencies of system calls in Edenfs.
Reviewed By: bolinfest
Differential Revision: D5660345
fbshipit-source-id: 97a1c2b83a6d8df0cd1b82c4d54b52d7ebd126bd
Summary:
1.Added new rage command to command line tools.
2.Eden rage command currently shows Logs,package version,rpm version,build revision,build upstream revision,list of running eden processes and their info,list of mount points and their info.
Reviewed By: simpkins
Differential Revision: D5220250
fbshipit-source-id: 357f46d8d08d4a1f197b705dfd1a28668dd180f0
Summary: We overlooked this in the recent move around of the cli code
Reviewed By: bolinfest
Differential Revision: D4894230
fbshipit-source-id: 5c3ae274a182e5cc41df47100959f8cfbcb59c2d
Summary:
Move the code for the command-line tool up one directory, out of eden/fs.
This better separates the code so that eden/fs contains code for the edenfs
daemon, while eden/cli contains code for the command line tool.
Reviewed By: bolinfest
Differential Revision: D4888633
fbshipit-source-id: 5041e292c5353d05122eefe5db3257289e31239a
