Summary:
In D20130406 I updated the CLI to call `getDaemonInfo()` to check on the
server status. However, some very old EdenFS instances do not have have this
method. These instances should all be gone shortly, but for now update the
code to handle the unknown method error and fall back to calling `getPid()`
and `getStatus()` separately.
I implemented this in our `EdenClient` wrapper class, similar to our existing
wrapper for `getPid()`.
Reviewed By: fanzeyi
Differential Revision: D20212518
fbshipit-source-id: 9d48bdd26822802a7e9776128c5567436d4bb445
Summary:
Update the thrift `getDaemonInfo()` call to also return the fb303 status.
This allows the CLI to make a single thrift call instead of 2 when checking if
the EdenFS daemon is healthy.
Reviewed By: genevievehelsel
Differential Revision: D20130406
fbshipit-source-id: 9d25341e1d5f82fb1a921e1d7b1ebd34bcf19dc8
Summary:
Fix the `check_health()` function to always set a timeout when querying for
EdenFS's health. Originally we used to always set a default timeout of 60
seconds when creating thrift connections to EdenFS, but this was removed in
D5942205. In practice we ideally really want a handful of specific thrift
calls (e.g., 'checkOutRevision()`, `getScmStatusV2()`) to have extremely high
timeouts, but most other calls should have fairly short timeouts.
For now this ensures that we apply a 3 second timeout by default when checking
for EdenFS health. The `edenfsctl status` call did explicitly set a 15 second
timeout, but other commands like `edenfsctl clone` and `edenfsctl restart`
would also check for health and were not applying their own timeout.
Also add thrift timeout for the `initiateShutdown()` call when doing a full
restart in `edenfsctl restart`
Reviewed By: chadaustin
Differential Revision: D20130405
fbshipit-source-id: c59118dbcafc2ed0d29206e33891f1a58da8c05f
Summary:
When checking if a commit is valid explicitly check against the backing
repository rather than the Eden checkout. This makes the commit work
correctly if the Eden checkout's `.hg` directory has been corrupted but the
backing repository is still fine.
Reviewed By: genevievehelsel
Differential Revision: D19629959
fbshipit-source-id: 57992260332cbc1d6868813263fb3768b50db07e
Summary: There is one instace of `getScmStatusBetweenRevisions` in use - it is used in the eden cli in a hacky way to check if a commit hash is valid. Since this is not used anywhere else in a meaningful way, this replaces that use case with a hg call and depreciates `getScmStatusBetweenRevisions`
Reviewed By: simpkins
Differential Revision: D18690026
fbshipit-source-id: 02bd2c20a0f631ec41116f9fd4e18d14369298ef
Summary: The file_size attribute for an inode is not always defined because it can refer to a directory. However, since `LoadedNode`s are always regular files, we can enforce file_size to be always an `int`.
Reviewed By: chadaustin
Differential Revision: D17683020
fbshipit-source-id: b79275c2146e5304bdef02c3228cc0bf35456095
Summary:
Use fdatasync (or fsync on macOS and Windows) and os.replace to write
Eden's config.json.
Reviewed By: wez
Differential Revision: D17465682
fbshipit-source-id: 0e92a58653befc84d45eb815bc0a02b8c2df3d39
Summary: On Windows, we create a .eden on the root of the repo as compared to in each directory. Updating get_eden_mount_name() to look for .eden in the current parent path.
Reviewed By: strager
Differential Revision: D16742335
fbshipit-source-id: 2d27c54895917524d3d9c233ccc723b51f33f032
Summary:
Basic tool to collect read/written files on eden
This is v0. The next steps are:
* Add collection of sizes for all paths
* Add as an optional step in duplo/jobs
* Add some smarter analysis options as we discover need
Reviewed By: strager
Differential Revision: D16663820
fbshipit-source-id: 1c25289c9840fc23318e0653bc632b097256a5bd
Summary: Add a dependency from the eden open source build to the fb303 open source build and switch EdenServiceHandler to BaseService.
Reviewed By: simpkins
Differential Revision: D15528156
fbshipit-source-id: 2ca5c31dd9fcc9bac43fd399b27f33b6f2c5ebfc
Summary:
I made the mistake of migrating the Thrift API for edenfs process
checks without a fallback for older edenfs running processes.
If getDaemonInfo() is not available, fall back on getPid().
Reviewed By: strager, pkaush
Differential Revision: D16365848
fbshipit-source-id: aee5c43c8c40db9a38fef7ca601f7639daa4adb8
Summary:
Open source fb303 will not have getPid() or getCommandLine(), so
introduce a new method for Eden's tests.
Reviewed By: fanzeyi
Differential Revision: D16292993
fbshipit-source-id: 5cdc006ec0ee15f50a3e1cebe9b46a3ea275ff78
Summary: The pwd module doesn't exit on Windows. getpass.getuser() will first check the environment varible to get the username first.
Reviewed By: chadaustin
Differential Revision: D16135009
fbshipit-source-id: 3ccc281e9a1725e1384d20c36f467a7e68ef6367
Summary:
This is part of the effort to make our bind-mount configuration more
visible and easier to change.
The idea is to generalize the concept of redirection and add a command to help
manage it.
The `eden redirect add` subcommand allows creating one of two different kinds
of redirection:
* `bind` - allocate some space using `mkscratch` and mount it into the repo
* `symlink` - allocate some space using `mkscratch` and create a symlink
that points to it from the repo
On Linux we use bind mounts to implement `bind` but on macOS, which doesn't
have a bind mount concept, we create a sparse disk image file that can grow
to match the size of the disk on which it is created (in practice these are a
7-15MB in size to start and grow as the user stores data into them).
The `eden redirect del` subcommand allows removing a redirection, including
the legacy `bind-mounts` configuration from `.eden/client/config.toml`.
The `eden redirect list` subcommand lists the effective set of redirections,
both from the new redirections configuration and the legacy `bind-mounts`
configuration, along with their state.
The `eden redirect fixup` subcommand iterates over the effective set of
redirections and can remove and reinstate any that are in a broken state.
Reviewed By: strager
Differential Revision: D15707319
fbshipit-source-id: a5dd8c44c9f748482d7b48855b1305d44267885c
Summary:
Update the copyright & license headers in Python files to reflect the
relicensing to GPLv2
Reviewed By: wez
Differential Revision: D15487088
fbshipit-source-id: 9f2138dff41048d2c35f15e09a04ae5a9c9c80dd
Summary:
Update most of the `eden/cli/config.py` to use `Path` instead of `str` where
appropriate. This also updates several of the APIs in `util.py` that were
affected as well.
Reviewed By: chadaustin
Differential Revision: D14356543
fbshipit-source-id: a8f6d15b8870bf689eeb78f9fc0e9a0c65c97218
Summary:
When graceful restart was first implemented we forgot to update the
lock file with the new pid, resulting in occasional unexpected output
from tools like eden doctor.
Reviewed By: simpkins
Differential Revision: D13744411
fbshipit-source-id: cdc758ed6ac1201fd2ff3e9d7805bb5ab6f83e8a
Summary:
Update some of the test code to use `Path.read_text()`, so we can eliminate
`eden.cli.util.read_all()`
Reviewed By: strager
Differential Revision: D13374245
fbshipit-source-id: 3399923b60ae78a4f7ea57367d097697c8b9c1cb
Summary:
Since we can't seem to find a way to prevent ESTALE
from being returned by the kernel, and that only seems to occur
with readlink, add a retry to our readlink calls.
This feels gross, but not sure how we can do better :/
Reviewed By: strager
Differential Revision: D12239143
fbshipit-source-id: 04ecbdeac3b3ea3969e779da4f50cf91ccc5cd00
Summary: Doctor now detects if multiple edenfs are running for the same edenDir and recommends to kill the rogue one.
Reviewed By: strager
Differential Revision: D10098157
fbshipit-source-id: 8a785a9fee0c768696f897a4950552d141012b2a
Summary:
If 'eden stop' tries to tell edenfs to cleanly shut down, and edenfs never responds, 'eden stop' just hangs. Make 'eden stop' time out CLI -> edenfs Thrift calls. After a timeout, 'eden stop' should by stop edenfs via SIGKILL.
Note: This diff does not affect 'eden restart'.
Reviewed By: chadaustin
Differential Revision: D9982372
fbshipit-source-id: 42e498fb93fd97a2ec963e97c25f4b4101c7d524
Summary: If edenfs is unresponsive, 'eden status' hangs forever. Add a timeout to turn the hang into a user-friendly error.
Reviewed By: chadaustin
Differential Revision: D10156229
fbshipit-source-id: 9186826ae6b131a193b1499c8baac616d131357f
Summary:
Update `eden restart` to explain that the restart will be disruptive when
performing a full restart, and prompt the user for confirmation. This prompt
can be overridden with the `--force` option. Also print more messaging after
the restart completes telling users what to do if they see "Transport endpoint
not connected" errors from programs still using the old mount points.
This also updates `eden restart` to more carefully handle cases where there is
an existing edenfs daemon running but it is not fully healthy.
Reviewed By: wez
Differential Revision: D8731004
fbshipit-source-id: 05762187b47057b2930d0a6b71b0a6fdbd4aa7e5
Summary:
D7946842 moved some of the daemon management code from util.py to daemon.py
but accidentally broke handling of TimeoutError exceptions. The code in
daemon.py was trying to catch python's built-in `TimeoutError` type, while the
code in util.py throws its own `util.TimeoutError` type.
This removes the custom `TimeoutError` type in util.py and just uses the
built-in python type to be consistent everywhere.
Reviewed By: wez
Differential Revision: D8373739
fbshipit-source-id: 8523c444fef74b0758c38b4fe2644d4f68d38442
Summary: Mostly empty lines removed and added. A few bugfixes on excessive line splitting.
Reviewed By: cooperlees
Differential Revision: D8198776
fbshipit-source-id: 4361faf4a2b9347d57fb6e1342c494575f2beb67
Summary:
When calling mercurial from inside the Eden CLI, use `$EDEN_HG_BINARY` as the
path to hg if this variable is set. If it is not set, continue the current
behavior of using `hg` from the user's `$PATH`. This is primarily used during
the `eden clone` command.
This makes sure the Eden integration tests use `hg` built from the local
repository rather than the version of `hg` installed on the system. Most
locations in the integration tests were already doing so, but `eden clone` was
one place that still ended up using the system hg binary.
Reviewed By: wez
Differential Revision: D7839850
fbshipit-source-id: da801fad0767a111b3e3dfd393d82da8e2147e22
Summary:
Add a `remove` command to the Eden CLI. This behaves like
`eden unmount --destroy`, but calling this "remove" is hopefully a more
intuitive UI. If stdin is a TTY this command also prompts the user for
confirmation before removing the checkout.
I plan to deprecate the `eden unmount --destroy` command in a subsequent
diff.
Reviewed By: wez
Differential Revision: D8086823
fbshipit-source-id: 562cf0f998eea416b80589b188eee255a10b9699
Summary:
This is stacked on top of Black 18.5b0.
allow-large-files
Reviewed By: carljm
Differential Revision: D8061834
fbshipit-source-id: 92e3645e159b60d77cf7e0bec64a8262ca4e88c2
Summary:
This moves some things around in order to facilitate adding the migration
command in a separate file.
Reviewed By: bolinfest
Differential Revision: D7946842
fbshipit-source-id: 54a554fb02e83a12f1d626b81377bc042fac41aa
Summary:
While testing the fbclone --eden changes I found that
we were failing to pick up the bind mounts. The root cause of
this is that the default head rev for hg repos is `.`, so if
the source repo is bare then we won't find an `.arcconfig` file
and thus won't be able to set up the bind mounts.
Reviewed By: simpkins
Differential Revision: D7782489
fbshipit-source-id: f41d3a7daf39ecd0946707cb2c3211d70c36eea2
Summary:
Try treating the argument as a repository path first, rather than a well-known
repository name. If we can find a mercurial or git repository at this path,
look for a .arcconfig file in the repository, and try to identify the
repository project type from this file. Use this project name to load our
default bind-mount and hooks configuration for the repository.
If the argument is not a valid repository path, fall back to treating it as a
well-known project name in our repository configs.
This also updates `eden clone` to print a few more diagnostic messages as it
works. (For instance, printing when it is starting the edenfs daemon, and
reporting the commit ID that was checked out.)
Reviewed By: wez
Differential Revision: D7739917
fbshipit-source-id: ac927b9e93039e4d1b8afa80466c2eee3a8829e9
Summary:
Update most of the Eden CLI code to include python type annotations.
I believe the stats.py and stats_print.py are the only CLI files that do not
have complete typing information now.
Reviewed By: chadaustin
Differential Revision: D7433368
fbshipit-source-id: dfd6a064cacffeeed9147739da7064f3303de789
Summary:
We would immediately satisfy the health check and
tell the user that the system is healthy and show the pid of
the prior incarnation rather than the one for the instance
that we just launched.
This diff refactors the health checking code so that we can
share the implementation between the cli and the integration
tests; the integration tests already had code to do the right
thing for this.
Reviewed By: simpkins
Differential Revision: D6944989
fbshipit-source-id: 7c0f02c875b1b81f8f1b7521add67928200b27ed
Summary:
Changed `eden clone` to check out master commit in both git and hg.
Previously, it checked out the current commit for the repo.
Reviewed By: simpkins
Differential Revision: D6663754
fbshipit-source-id: 92b185ccca5d082dc2bde9c8b191c82a2a4f06b4
Summary: Update the functions in cli/util.py with type annotations.
Reviewed By: bolinfest
Differential Revision: D6434356
fbshipit-source-id: 054d81427132229a390b8a133d5180be5f70bd20
Summary:
It seemed unnecessary to include a `\n` in the template only to strip it.
Similarly, we should be strict when parsing the output since we expect it to be
a string of hex characters.
Reviewed By: wez
Differential Revision: D6322129
fbshipit-source-id: 61c483badfd7b68ed012310360aa582d6bdf5181
Summary:
This was an error that an end-user ran into. Previously, we did not fail
gracefully and the user was faced with an intimidating stacktrace.
Reviewed By: simpkins
Differential Revision: D6195529
fbshipit-source-id: bde3c2a3e6f49457a4c6ac5c87103cf52cd227c2
Summary:
Move the code for the command-line tool up one directory, out of eden/fs.
This better separates the code so that eden/fs contains code for the edenfs
daemon, while eden/cli contains code for the command line tool.
Reviewed By: bolinfest
Differential Revision: D4888633
fbshipit-source-id: 5041e292c5353d05122eefe5db3257289e31239a
