Summary:
On Windows, `pathlib.Path.resolve` has an use of uninitialized memory bug in Python 3.6.2. Lego Windows has this version of Python 3 installed.
When `eden prefetch` runs, it will first attempt to read the config file at the root of the repository (`$REPOROOT/.eden/config`). Because of this bug, when we normalize the path in edenfsctl, it will get some random bytes as a result. This subsequently causes `toml.read` to fail as it is unable to read a path containing NUL byte.
As you can see, this is the same exception as we see on Windows:
```
In [1]: open("test\x00")
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-1-57764b20d660> in <module>
----> 1 open("test\x00")
ValueError: embedded null byte
```
Reviewed By: xavierd
Differential Revision: D22177997
fbshipit-source-id: ab2565c8946d809bc15bc1597b749fb5e9440ca0
Summary:
On Windows, we can't rely on the .eden directory to not be present to decide if
the repository is mounted. Instead, we can just let edenfs decide and catch the
exception if it tells us that it's already mounted.
Reviewed By: fanzeyi
Differential Revision: D21934538
fbshipit-source-id: 60ed9f530456b627091f95f7387dc4b8f3a8dc5c
Summary:
The Python 2-and-3 Thrift API is sort of deprecated and does not
handle binary data in `binary` fields. In advance of migrating to the
modern Python 3 API, remane eden.thrift to eden.thrift.legacy.
Reviewed By: fanzeyi
Differential Revision: D21889697
fbshipit-source-id: a745ee8977999acbfb383a4edebe81d8deb1734e
Summary:
I am planning to start migrating Eden's CLI to the new Python 3 Thrift
implementation. In preparation, slightly clean up the interface and
implementation of our Python 2 Thrift wrapper.
Reviewed By: genevievehelsel
Differential Revision: D21854539
fbshipit-source-id: d398dd3f324c12288871cf0c9db41e64ed4cf7ed
Summary:
This diff is auto-generated to upgrade the Pyre version and suppress errors in eden. The upgrade will affect Pyre local configurations in the following directories:
```
eden
```
Differential Revision: D21687853
fbshipit-source-id: baf0d9bc33f86da63ea289690faca6cf4d566588
Summary:
- Added uptime field to DaemonInfo thrift struct
- Created startTime member variable in EdenServer
- Made appropriate refactoring changes to EdenMain and EdenServer
- Changed main.py and util.py to use the new uptime value
Reviewed By: genevievehelsel
Differential Revision: D21471140
fbshipit-source-id: 8868de667dfb95de93e3e71b90c0412fb3825388
Summary:
As a top layer of reliability in graceful restart, we'd like the CLI to enforce that something has started after a graceful restart call. There are a few things that can happen
1) The restart short circuts before attempting to shutdown (version mismatch etc). The old process will continue (`success == False`)
2) The restart is successful, the new process will continue (`success == True`)
3) The restart is not successful (failed ping), the old process will recover (`success ==
False`)
(note here that the `success` field is the return code of the `StartupLogger` process, not the edenfs process itself).
In case any of these fail, we'd like a final line of defence with the CLI.
First, in the case of a successful start from the `StartupLogger` report, we consider this successful and exit. If we do not have a successful (`success == False`) graceful restart, we will wait for the new process to recover (since the old process will go from `ALIVE` to `STOPPED` to `ALIVE`). I do not expect recovery to take long since we are only remounting, not waiting on RocksDB. Here though, we give a fairly long timeout (1hr) to ensure we're not stuck on an in process thrift call, since we would not like to force kill the process if we're writing to the overlay.
If we time out waiting, we will just kick off a force restart and assume the old process is stuck.
Reviewed By: simpkins
Differential Revision: D20673460
fbshipit-source-id: 215e06fa8aa76dbe5dae6602f82ce5f05968e538
Summary:
- Catch the Windows-specific exceptions types thrown when attempting to open
the socket if EdenFS is not running, and convert this to an EdenNotRunning
exception.
- Update EdenFS to write its pid to a separate file, in addition to the normal
lock file, and have `edenfsctl` read this file instead of the lock file on
Windows. Other processes cannot read the lock file while EdenFS is running
and holding the lock, so it is nice to have the pid written in an alternate
location.
Reviewed By: pkaush
Differential Revision: D20854615
fbshipit-source-id: 1c8e8f402eb17dd012d03e11fbb662f493d9362d
Summary:
Add methods to check if a process ID is alive, and if it looks like an EdenFS
process.
This also adds an initial version of ProcUtils for Windows, and implements
these two methods on Windows. I have moved parts of the `winproc.py` module
to the new `proc_utils_win.py` module, to help better manage dependencies
between our modules. This keeps all of the Windows-specific `ctypes` code
together in `proc_utils_win.py`. The functionality that is still left on
`winproc.py` depends on `config.py`, and the `proc_utils` code should not
depend on `config.py` to help avoid circular dependencies.
Reviewed By: wez
Differential Revision: D20833245
fbshipit-source-id: 43e9b6dd1b520dcb6b2da7701de885058f0f7ea2
Summary:
Update out platform checks to use `sys.platform` instead of `os.name`
Python type checkers (mypy and Pyre) currently understand checks against
`sys.platform`, and know that code guarded by these checks is platform
specific. They don't understand the `os.name` checks.
This code should behave the same, but allows type checkers to do a better job
checking this code.
Note that there is also `platform.system()`, but this has drawbacks that it
apparently does runtime-checks to determine the platform, whereas `sys.platform`
is baked-in at compile time. Additionally, the typecheckers do not appear to
support checking based on `platform.system()` for now. There is an open
feature request for this for mypy, but it is not implemented yet.
One caveat for `sys.platform` is that the results on Linux are not consistent
across Python versions: older versions of Python used to report `linux2` while
new versions report simply `linux`.
Reviewed By: chadaustin
Differential Revision: D20830149
fbshipit-source-id: d173e41f1ae84951a84b87e2dc05787fe8b01407
Summary:
Move the code for the command-line tool up one directory, out of eden/fs.
This better separates the code so that eden/fs contains code for the edenfs
daemon, while eden/cli contains code for the command line tool.
Reviewed By: bolinfest
Differential Revision: D4888633
fbshipit-source-id: 5041e292c5353d05122eefe5db3257289e31239a
Summary:
Update copyright statements to "2016-present". This makes our updated lint
rules happy and complies with the recommended license header statement.
Reviewed By: wez, bolinfest
Differential Revision: D4433594
fbshipit-source-id: e9ecb1c1fc66e4ec49c1f046c6a98d425b13bc27
Summary:
This design is inspired by that of Git hooks:
https://git-scm.com/docs/githooks
By default, `/etc/eden/hooks` should be the place where Eden looks for
hooks; however, this can be overridden in `~/.edenrc` on a per-`repository` basis.
This directory should be installed as part of installing Eden.
There is information in `eden/hooks/README.md` about this.
The first hook that is supported is for post-clone logic for a repository.
This change demonstrates the need for an `eden config --get <value>`
analogous to what Git has, as hooks should be able to leverage this in their
own scripts. There introduces a `TODO` in `post-clone.py` where such a
feature would be useful, so that I could add the following to my `~/.edenrc`
to develop the Eden extension for Hg:
```
[hooks]
hg.edenextension = /data/users/mbolin/fbsource/fbcode/eden/hg/eden
[repository fbsource]
path = /data/users/mbolin/fbsource
type = hg
hooks = /data/users/mbolin/eden-hooks
```
Note that this revision also introduces a `generate-hooks-dir` script that can be
used to generate the standard `/etc/eden/hooks` directory that we intend to
distribute with Eden. This is also useful in creating the basis for a custom `hooks`
directory that can be specified as shown above in an `~/.edenrc` file.
Reviewed By: simpkins
Differential Revision: D3858635
fbshipit-source-id: 215ca26379a4b3b0a07d50845fd645b4d9ccf0f2
Summary: Restructure the current logic used for loading the config data into a ClientConfig object. Rather than having loadFromClientDirectory iterate through all the config files and parse them to find the necessary information, abstract that logic out into a new method that compiles all of the relevant data so that all loadFromClientDirectory has to do is pull out the needed information. Since this change separates the two steps, this will make it easier to move the first step of compiling config information outside of ClientConfig - the goal here is to have the eden server load all of the config data at start up and cache it in memory so that it doesn't need to be done every time a ClientConfig object is created, and this change is an intermediate step.
Reviewed By: simpkins
Differential Revision: D3580757
fbshipit-source-id: c340a0fe715856066a554238249574f8177bc4d7
Summary: Move the _get_or_create_dir() method from config.py to util.py and rename as mkdir_p(). This change replaces all of the os.makedirs call sites with the new mkdir_p() method.
Reviewed By: bolinfest
Differential Revision: D3512570
fbshipit-source-id: a867049f9af22076934390061f09070bf9ee6397
Summary: Add a simple eden repository command that takes no arguments. The behavior mimics that of git remote and lists all repositories by parsing both the global and local INI config files. The INI file currently needs to be manually added and updated to maintain information about existing repositories.
Reviewed By: bolinfest
Differential Revision: D3479543
fbshipit-source-id: b0d36f33c9b8e0c7b9fe20ec2f362ca15c0adeb8
Summary:
- Update "eden daemon" to wait until edenfs is healthy before returning.
- Accept additional command line arguments and pass them through to edenfs.
- When starting edenfs as a background daemon, direct stdout and stderr to a
log file in the eden config directory.
- Update "eden shutdown" command to wait for edenfs to exit.
- Update "eden shutdown" command to print an error message if edenfs was not
running.
Reviewed By: bolinfest
Differential Revision: D3446403
fbshipit-source-id: 2ca1874256b7d124a85092b886548bea8c198327
