Summary: Remove a number of unused imports detected by the linter.
Reviewed By: wez
Differential Revision: D15776268
fbshipit-source-id: 221f45d275664d037bbabcac9858b40266b4833e
Summary:
get_edenfs_start_cmd checks if edenfs is already running, and fails if it is. Most of the time, this check is redundant. This function is effectively called in five places:
* `edenfsctl start`
* `edenfsctl clone` if not running
* `edenfsctl restart` (`_graceful_restart`) with takeover=True
* `edenfsctl restart` (`_start`) if not running
* `edenfsctl restart` (`_finish_restart`) after stopping edenfs
The check is only necessary in one of these places (`edenfsctl start`). Move the check.
This refactor will allow us to run the check if systemd is enabled without duplicating code.
This diff should not change behavior (ignoring unimportant race conditions).
Reviewed By: simpkins
Differential Revision: D15703308
fbshipit-source-id: cbd4a70a3e0eb23da0d65d641d89a5ac022cd79f
Summary:
This diff teach Eden's Python client to use Windows' Unix Domain Socket support to connect to Eden.
#delayed_publish
Reviewed By: chadaustin
Differential Revision: D15454514
fbshipit-source-id: 86aa39f8aa13ae789efabda3fa88635f471a1a7a
Summary:
This diff just stubs out some functions that were either
not present or that were doing linux specific things that would
error out when attempted on macOS.
This diff doesn't implement macOS equivalents of the functions;
each of those can be tackled separately in follow up diffs.
Reviewed By: simpkins
Differential Revision: D15453050
fbshipit-source-id: 228044fc416d9170a4ba3aef6cd9b40194707f17
Summary:
Rename the `ClientConfig` class to `CheckoutConfig`, to help further progress
on updating our terminology from "client" to "checkout".
Reviewed By: chadaustin
Differential Revision: D15162814
fbshipit-source-id: 3575958f1161d5842c6f0ee9e2d2d20ab20b7372
Summary:
Sometimes, the XDG_RUNTIME_DIR environment variable isn't set. If this happens, 'eden start' fails because systemctl uses XDG_RUNTIME_DIR to talk to systemd. We still want 'eden start' to work in these cases, so guess what XDG_RUNTIME_DIR should be and use that guess if the variable isn't set.
If XDG_RUNTIME_DIR is set in the environment, its value should still be used.
Reviewed By: chadaustin
Differential Revision: D13811813
fbshipit-source-id: bb44d99e585bbe7a4341087c5cb4644c606fc441
Summary:
The systemctl command requires XDG_RUNTIME_DIR to be configured. If it's not configured, 'eden start' should pick a sane default. The sane default includes the user's UID (e.g. /run/user/6986). I want this default to be configurable via Eden's config files.
Expose the ${USER_ID} token to Eden configs. This will let administrators can customize XDG_RUNTIME_DIR's fallback value in the future.
Reviewed By: wez
Differential Revision: D13811732
fbshipit-source-id: 7933e078dd5f2b3bbbb0299730220a129c257256
Summary:
Update most of the `eden/cli/config.py` to use `Path` instead of `str` where
appropriate. This also updates several of the APIs in `util.py` that were
affected as well.
Reviewed By: chadaustin
Differential Revision: D14356543
fbshipit-source-id: a8f6d15b8870bf689eeb78f9fc0e9a0c65c97218
Summary:
Remove the `CheckoutConfig.hooks_path` attribute which is no longer used
anywhere.
Reviewed By: strager
Differential Revision: D14356542
fbshipit-source-id: 28b7e7aeb197aeb5d04edf8adfdcfe7946ffa0c7
Summary:
Pyre complains about this cast being redundant. It was needed at one point
(in D9144139) to make either pyre or mypy happy, but now neither seem to need
it and pyre actually seems to complain about it.
Reviewed By: chadaustin
Differential Revision: D14319772
fbshipit-source-id: ed252763acfe9c3c6326c6712e30d2840080454c
Summary:
in D13970332 we want to allow using `hg.real` to run the
import helper that is embedded into mercurial, but we don't know
the full path, so we need to resolve it from the environment.
`folly::Subprocess` has `usePath` option, but it is not compatible
with also overriding the environment for a child process.
The easiest short term thing to do is to resolve the path ourselves
using this simple function call.
I've also added the path to `hg.real` that is used on our mac systems
to the PATH we pass down to `edenfs` when starting eden.
Longer term we want to better encapsulate how we resolve and start
the import helper, but for now this unblocks starting eden on the
mac.
Reviewed By: strager
Differential Revision: D14078612
fbshipit-source-id: bc6787bbe99cd87224b783d2fb9d3255b825b976
Summary:
This renames the CheckoutConfig `path` member variable to `backing_repo`.
The value in this variable is actually the path to the source control backing
repository, and not the path to the Eden checkout, which was confusing. (The
CLI tests were actually setting this up incorrectly in `create_test_mount()`,
but until now nothing in the tests had been using it, so we hadn't noticed
this problem earlier.)
I also changed `CheckoutConfig` to be a `typing.NamedTuple` as part of this
change.
Reviewed By: strager
Differential Revision: D13935006
fbshipit-source-id: 43b6ba4efa34f8c6fb8264de8bb31a33a6d73036
Summary:
Update the code in `hg_util` to take advantage of the `EdenCheckout` methods
for getting config data rather than passing around an `EdenInstance` and a
config object separately.
Reviewed By: strager
Differential Revision: D13935007
fbshipit-source-id: 7c4079fa9b6c1a56f15006f44da994bf69b0f050
Summary:
Move code for loading and saving checkout configs and snapshot files from
EdenInstance to EdenCheckout.
Reviewed By: chadaustin
Differential Revision: D13932141
fbshipit-source-id: aa5cf56ed6f2ac7d3e5529eee353e1dd0ea12a21
Summary:
The service.experimental_systemd config option is implemented hackily: the value is parsed as a boolean, then converted to a string (with `str`), then compared with `"True"`. Remove this ugliness by not converting all config options to strings in EdenConfigParser, and adding a type-safe `get_bool` accessor.
Aside from disallowing `experimental_systemd = "True"` in configs and changing the output of `eden config`, this diff should not change behavior.
Reviewed By: simpkins
Differential Revision: D13666331
fbshipit-source-id: 65b7d233c43db08bc7e0288f248041ca4a18a5fd
Summary:
This adds a new `CheckoutInfo` class in the `eden doctor` code to consolidates
some information about a particular checkout. This should make it easier to
pass around this single object to various check functions, instead of several
separate arguments.
I did make a few functional changes as part of this refactoring:
- All checkouts are now checked in order sorted by their mount path.
Previously non-mounted checkouts were checked before running checkouts.
- The code now reports additional errors about checkouts if they are running
but not listed in the on-disk configuration, or if their running state
differs from the on-disk configuration in some cases.
Reviewed By: strager
Differential Revision: D13526754
fbshipit-source-id: 81802f96a3a3c9f2aaad9a4468fb666ed8b2e47e
Summary:
In preparation for proper support for booleans and arrays in configs, refactor the interface of `EdenConfigParser`:
* Rename `get` to `get_str` so callers explicitly mention the expected type. This change will make sense in the future when `EdenConfigParser` supports boolean and array options in addition to string options. Also, rename the `fallback` parameter to `default`.
* Move `config_to_raw_dict` into `EdenConfigParser` and remove the `raw` boolean parameter from `items`.
* Rename `items` to `get_section_str_to_str`, and change its return type to `Mapping` (instead of `List[Tuple]`).
This diff should not change behavior.
Reviewed By: simpkins
Differential Revision: D13661228
fbshipit-source-id: fc93560f95b75ade78044d8146e92c6f65dfc404
Summary:
Python's `configparser` module requires values to be strings. Values in TOML can have other types, such as boolean and array, and `configparser` gets in the way of parsing such values.
Allow fixing `configparser`'s limitations by rewriting parts of it. Instead of using `configparser.ConfigParser`, create an `EdenConfigParser` class (which implements a subset of `configparser.ConfigParser`'s API) and use that instead.
This diff should not change behavior. A future diff will enable non-string config values.
Reviewed By: simpkins
Differential Revision: D13646873
fbshipit-source-id: e26a886789cf00dc363f3a97fe0febe6f5010f8e
Summary:
`EdenInstance.get_config_value` raises a KeyError if the option or section is not set in any config files. Every caller of `EdenInstance.get_config_value` handles KeyError manually. Simplify code by letting callers specify a default value.
Aside from treating some options with an empty value as if they were unset, this diff should not change behavior.
Reviewed By: chadaustin
Differential Revision: D13737806
fbshipit-source-id: fdb7fa75d601de4644704813db38b671b27f73d7
Summary:
Update the `eden list` command to also report the current state for each
checkout if it is not running normally. Also added a `--json` flag to
print information as JSON so it can be consumed programmatically.
Reviewed By: strager
Differential Revision: D13503053
fbshipit-source-id: 4ef366f5bf4a1157036fdfd7ff1056079588e802
Summary:
I plan on replacing Eden's use of `configparser` [1]. Prepare for that change by reducing the set of methods Eden uses from `configparser`. In particular, avoid using `configparser.SectionProxy` and `configparser.ConfigParser.__getitem__`.
This diff should not change behavior.
[1] Python's `configparser` module requires values to be strings. Values in TOML can have other types, such as boolean and array, and `configparser` gets in the way of using such values.
Reviewed By: simpkins
Differential Revision: D13646807
fbshipit-source-id: 12f9875b86e5fb639e7e6da5ef05bb9a1e4fc543
Summary:
As of D13737806, for most config options, an option with an empty value is treated as if it is unset. The `type` and `path` options in repository sections behave differently: an empty value is different from no value. Make `type` and `path` consistent with other config options.
This diff should not change behavior given valid config files.
Reviewed By: chadaustin
Differential Revision: D13737986
fbshipit-source-id: f1d2044229d828cc3a04ce46acc31e90bee76c39
Summary:
The 'eden config' command prints output naively. This leads to confusing output when using the hg.extra_hgrc option. For example, in the following output, `mode = off` is part of hg.extra_hgrc's value, but it looks like it's a separate option in Eden's config:
```
$ eden config
[clone]
default-revision=master
[rage]
reporter=pastry --title "eden rage from $(hostname)"
[hg]
extra_hgrc=[fsmonitor]
mode = off
%include /etc/mercurial/repo-specific/eden.rc
[service]
experimental_systemd=True
```
Fix this issue by making 'eden config' output valid TOML:
```
[clone]
default-revision = "master"
[rage]
reporter = "pastry --title \"eden rage from $(hostname)\""
[hg]
extra_hgrc = "[fsmonitor]\nmode = off\n\n%include /etc/mercurial/repo-specific/eden.rc\n"
[service]
experimental_systemd = "True"
```
Reviewed By: simpkins
Differential Revision: D13661229
fbshipit-source-id: 76e4fa83ad186d04451623e3d8d87a78e4b821d8
Summary: The create_no_such_repository_exception method is not referenced anywhere. Delete it.
Reviewed By: chadaustin
Differential Revision: D13646638
fbshipit-source-id: 631c9230e6242bd858c30eec56e000ce99fbf2d8
Summary:
Update the Eden CLI to use `os.path.realpath()` to resolve symlinks in the
Eden config directory before using it.
In most situations today it is common for the default path
(`$HOME/local/.eden`) to traverse as symlink in the user's home directory.
For users that are still using NFS home directories we can sometimes read this
symlink when running as the user, but we cannot read the symlink as root (for
instance, from inside the privhelper process).
Resolving symlinks in the CLI code ensures that the `edenfs` daemon will see
the final resolved path and will not need to traverse symlinks in the user's
home directory in this situation.
Reviewed By: strager
Differential Revision: D13515871
fbshipit-source-id: 0602389492afc0b542e089bb002534f3d714882e
Summary:
Update the `listMounts()` thrift API to also report the current mount point
state. This will allow us to do a better job of reporting mount points that
are in the process of initializing or shutting down.
This change splits the `MountInfo` thrift type into two distinct types for
the `listMounts()` vs `mount()` APIs. However this change should be
completely backwards compatible at the wire protocol level for older client
and server code.
Reviewed By: strager
Differential Revision: D13503049
fbshipit-source-id: 68e7ca708b956991c8fd93bbf8973d90650aced9
Summary:
Change `eden clone` so that it no longer makes the underlying mount point
directory read-only.
The original intention of making the directory read-only was to prevent users
from accidentally removing the `README_EDEN.txt` file that tells them how to
recover if edenfs is not running. However in practice this ends up confusing
users that are legitimately trying to remove checkouts. (At the moment
`eden rm` can sometimes time out trying to unmount the checkout, in which case
it can leave this directory behind.
Reviewed By: strager
Differential Revision: D13515854
fbshipit-source-id: c2e86632c1ab8a92d54179abb58839c68a8b23c6
Summary:
Update the `eden clone` command to automatically create the `.hg` directory
when creating a checkout for a Mercurial repository.
Previously this logic was performed by a separate post-clone hook that was
invoked by `eden clone`. Having this logic in a separate script made the code
slightly more complicated, and meant that configuring Eden was also more
complicated, as the hook also needed to be installed and configured. Moving
the logic into the Eden CLI will make it easier to re-use this code in
`eden doctor` if the `.hg` directory needs to be repaired.
Reviewed By: wez
Differential Revision: D13447272
fbshipit-source-id: 11c4f8e389aead151dd235eff95c860a326967af
Summary:
Rename `ClientConfig` to `CheckoutConfig` to move slightly closer towards
standardizing on the term "checkout" rather than "client" to refer to an Eden
mount point directory.
Reviewed By: chadaustin
Differential Revision: D13437952
fbshipit-source-id: 5086b4187eb975c860d48cadf0ccd4e5dce78e9a
Summary:
Currently, users can opt into systemd for edenfs by setting EDEN_EXPERIMENTAL_SYSTEMD=1 every time they use Eden's CLI. Make it easier to opt in: use systemd if service.experimental_systemd is set in ~/.edenrc. Also allow opting out for a single command via EDEN_EXPERIMENTAL_SYSTEMD=0.
For users who don't set service.experimental_systemd or use EDEN_EXPERIMENTAL_SYSTEMD, this diff should not change behavior. (The EDEN_EXPERIMENTAL_SYSTEMD environment variable is still respected.)
Reviewed By: simpkins
Differential Revision: D13371186
fbshipit-source-id: d7b533ea3583f4b6c8a8f63c6a74eace2be7d783
Summary:
I want to allow opting into systemd using a setting in ~/.edenrc. Since should_use_experimental_systemd_mode is a global function, it can't read any configs. Make the config file visible to should_use_experimental_systemd_mode by moving it into EdenInstance.
This diff should not change behavior.
Reviewed By: simpkins
Differential Revision: D13370823
fbshipit-source-id: b604db66954d0a08973030daae38bf6b1433e821
Summary:
Add the plumbing necessary to make 'eden start' start a systemd user service. This is only enabled if you opt in using EDEN_EXPERIMENTAL_SYSTEMD.
Currently, only fake_edenfs works. The real edenfs doesn't work yet because it needs root access to configure mount points.
'eden restart', 'eden stop', etc. are not affected by this diff (and are probably broken with EDEN_EXPERIMENTAL_SYSTEMD enabled).
Reviewed By: simpkins
Differential Revision: D10849390
fbshipit-source-id: c087a6498951ff100e5c80bd07ad869b2709e1b3
Summary:
The thrift APIs accept path names and commit IDs as binary data (python bytes)
rather than unicode strings. Our python code got this wrong in several
locations. It looks like mypy didn't previously flag this since mypy doesn't
actually figure out the correct type for the thrift `client` object, and
seemed to just be largely ignoring it. I plan to update the code so that mypy
can figure out out the client type correctly. Fixing these type errors is
required to make sure we won't get type errors once that is changed.
This just simply uses `encode("utf-8")` for now. In the future the path
arguments should be converted to `pathlib.Path`, which will do a slightly
smarter conversion, and avoid errors on non-UTF-8 binary data. In the
meantime, I believe that just using `encode("utf-8")` preserves what the
thrift code was doing implicitly before, and does not make handling of
non-UTF-8 data any worse than it was before.
Reviewed By: strager
Differential Revision: D13051094
fbshipit-source-id: 94cb62f3dd78b8e854a72a392fe8fdfad5ffd4cb
Summary: Update the CLI code to make the pyre type checker happy.
Reviewed By: wez
Differential Revision: D13035413
fbshipit-source-id: d201f2e65667e0ce1bf4a73fbb05878e8711ad16
Summary:
The `path` parameter to `cmd_util.find_checkout()` is optional, so make sure
the typing documents that correctly.
Also use the `CLIENTS_DIR` symbolic constant consistently in `config.py`
Reviewed By: wez
Differential Revision: D12955042
fbshipit-source-id: da8bd94dbabc2a7c0e2f9991a4513f1e6be45899
Summary:
Sometimes users accidentally run `edenfs start` or `edenfs restart` instead of
`eden start` or `eden restart`. This adds a new `--edenfs` flag to the
`edenfs` binary, and asks users if they meant to run `eden` instead if they do
not pass in this flag.
This used to be less of a problem since `edenfs` required users to also
explicitly specify several other configuration flags (like `--edenDir`).
However `edenfs` can not automatically figure out these settings, so these
flags are no longer required. Therefore `edenfs` would still try to start
normally when invoked with `edenfs restart`, since it did not require these
flags and it did not complain about unhandled command line arguments.
Reviewed By: wez
Differential Revision: D12927803
fbshipit-source-id: dbf7ce2449c391ca218652439eb68ff43c2ebd46
Summary:
- Update mypy in fbcode to upstream version 0.641.
- Add/fix some type stubs
- Disabled some pieces that break mypy (T35976411 and T35973384)
This diff will likely cause some unit test failures, due to incorrect type annotations that should be fixed by the team owning the relevant code.
mathchecksout
Reviewed By: jeffmo
Differential Revision: D10414250
fbshipit-source-id: 282ea62aef6793a2f4baef78afd58ddd4ddf60f5
Summary:
Since we can't seem to find a way to prevent ESTALE
from being returned by the kernel, and that only seems to occur
with readlink, add a retry to our readlink calls.
This feels gross, but not sure how we can do better :/
Reviewed By: strager
Differential Revision: D12239143
fbshipit-source-id: 04ecbdeac3b3ea3969e779da4f50cf91ccc5cd00
Summary:
To ease the migration from the current custom daemon setup to a systemd-managed daemon, add a way to enable the to-be-implemented systemd behavior.
* To the edenfs daemon, support the --experimentalSystemd option.
* To the Eden CLI, support the EDEN_EXPERIMENTAL_SYSTEMD environment variable. When set, invoke the edenfs daemon with --experimentalSystemd.
Aside from adding a log message, this diff does not change any behavior. This diff just sets up scaffolding for future diffs.
Reviewed By: simpkins
Differential Revision: D10248055
fbshipit-source-id: 30ca5a4bfde00ff43fd6f2a5d0282ced4f177fed
Summary:
We've been shipping our RPMs with toml config support turned on for a
while now. Remove support for the old config file format. Continue to
ship the old format configurations. We'll remove those in a later
diff.
Reviewed By: strager
Differential Revision: D10020958
fbshipit-source-id: 11c2ca3b5da086b142042496a2814699880c4f81
Summary: If edenfs is unresponsive, 'eden status' hangs forever. Add a timeout to turn the hang into a user-friendly error.
Reviewed By: chadaustin
Differential Revision: D10156229
fbshipit-source-id: 9186826ae6b131a193b1499c8baac616d131357f
Summary:
This adds a new `find_eden()` method to the Eden CLI code which can look up
information about the correct EdenCheckout and EdenInstance given a path.
In the future most CLI commands should switch to use this function over the
current `get_eden_instance()` and `get_mount_path()` methods. These older
APIs only work to find currently mounted checkouts, which is inadequate for
commands like `fsck` and some other debug commands that want to be able to
operate on unmounted checkouts.
This new logic is able to correctly find checkout information even for
unmounted checkouts. If the checkout is managed by an edenfs instance running
out of a non-default location it also correctly finds the state information
for Eden, rather than potentially using the wrong Eden state directory.
Reviewed By: wez
Differential Revision: D9385821
fbshipit-source-id: a6638f3c3817a595a7b7979c5cd218a2d7400f51
Summary: This commit let `HgImporter` pick up configuration values from `EdenConfig`.
Reviewed By: chadaustin
Differential Revision: D9346293
fbshipit-source-id: cb63f7d13a86058e9bf076eddb52212560a64cb1
Summary:
In the CLI, rename the "Config" class to "EdenInstance". This class
represents all state about a particular edenfs instance. It provides APIs for
making thrift calls to edenfs, for creating and destroying checkouts, and
generally does much more than just managing configuration.
Renaming it to "EdenInstance" also helps clarify that it is distinct from the
configuration tracked in the user's ~/.edenrc and the /etc/eden directory.
This change purely renames internal class and variable names and should not
affect any user-visible functionality.
Reviewed By: wez
Differential Revision: D9355515
fbshipit-source-id: ba5d4c3b753c6eb12a3783306dcd29e85fea3f52
Summary:
This prevents `hg status` from blowing up with a UTF-8 decode
error inside the generated thrift code.
Push safety concerns:
* This doesn't change the wire representation of the data
* Existing clients that believe it to be a string will continue to have
the same behavior
* Buck has its own copy of an older version of the thrift spec, so it will
continue to work "OK".
* When buck resyncs with our thrift file, some changes will likely be needed
to convert the byte arrays to strings or paths or whatever is appropriate
for bucks internal API
Work "OK" above means that clients that currently believe that `string` is
utf-8 encoded will have a runtime error if we ever send them a path that
is not utf-8. This is the behavior prior to this diff and will continue
to be the behavior for clients (like buck) that have an older version
of the thrift file.
Reviewed By: simpkins
Differential Revision: D9270843
fbshipit-source-id: b01135aec9152aaf5199e1c654ddd7f61c03717e
Summary:
I noticed while making some changes to eden doctor that our
typechecker wasn't working. check_types in python_library became a
no-op, so set typing instead and fix recently-introduced type-checking
failures.
Reviewed By: simpkins
Differential Revision: D9225840
fbshipit-source-id: 32edd60be862b70d97ca48dc9e1929b9add9f166
Summary:
This commit integrates Mononoke API Server with eden:
* Added two new command line options: `--client_certificate` and `--use_mononoke`.
* Added two new config values: `ssl.client-certificate` and `mononoke.use-mononoke`.
* Made `HgImporter` return the repo name along with treemanifest options.
* Make `HgImporter` ask Mononoke API Server for tree data when the desired tree is not present in local Mercurial store.
Reviewed By: chadaustin
Differential Revision: D9183035
fbshipit-source-id: e328fb3237d10c545c8af71f856007ad6c487061
Summary:
Eden should load its configuration from toml files. This change includes:
- ConfigParser is the class that parses and provides access to the configuration. We load the configuration from toml files and pass to ConfigParser in order to preserve existing behavior.
- The toml configuration format differs from the Config format. Thus, we use toml.load() and toml.dump() to load/save toml format configuration.
- Tests
- configuration is properly loaded from a group of config files
- toml configuration (if present) takes precedence
- missing configuration files (/home/user/.edenrc) are ignored
- invalid configuration files are fatal
- update of configuration works (adding repo)
- toml parsing handles over-rides of properties in the same way as cfg files
Deployment Strategy:
- This code determines whether to use toml configuration based on the presence
of atleast 1 '.toml' file in the etc-eden-dir/config.d.
Reviewed By: chadaustin
Differential Revision: D9023232
fbshipit-source-id: 6734a9d91eca92b05872a758c764546451dd2d51
