Summary: To be able to quickly find all activity related to given Sandcastle job as discussed in [this workplace post](https://fb.workplace.com/groups/2120196508269853/permalink/2899040243718805).
Reviewed By: kmancini
Differential Revision: D27541803
fbshipit-source-id: a55900064bbee92da902de785ebe0c0e8738c3a2
Summary:
One of the unforseen effect of D24393690 (c5d631fd09) is that EdenFS was tied to the console
that ran `edenfsctl start`. As a result, killing that console would also kill
EdenFS, which is a bit unexpected.
Console in Windows are a bit complicated, and in theory, EdenFS could simply
call `FreeConsole` once it is started, but the side effect of this is that any
process started by EdenFS (say Mercurial), would create its own console as its
parent doesn't have one, specifying CREATE_NO_WINDOW when creating these would
lead to not having any output in the edenfs logs from Mercurial (D21820195 (223846d313))...
We could solve this if we could tell Windows to allocate a hidden console
window, but that API unfortunately doesn't exist.
So, the best we can do for now is to simply start EdenFS with the
CREATE_NO_WINDOW flag, this fixes the above issue, allows for the original
console to be killed without affecting EdenFS, but it brings another issue:
`eden start` no longer shows what is going on...
For now, this is the least worst solution, so we'll go with that. In the
future, we can imagine fixing EdenFS startup code to send its output to a pipe
that edenfsctl would read and print to its standard output.
Reviewed By: chadaustin
Differential Revision: D24607540
fbshipit-source-id: c18590052a96b7dd2938b589e92e808f38b5ef59
Summary:
Now that Linux/macOS startup no longer uses fork it becomes trivial to share
the same code with Windows. This improves Windows startup in several different ways:
- `edenfsctl start` now displays the status of the start process in the console
- `edenfsctl start` no longer has an arbitrary timeout after which it reports
a timeout even though EdenFS is still starting up.
This also kills a bunch of Windows specific code that is no longer needed.
Reviewed By: fanzeyi
Differential Revision: D24393690
fbshipit-source-id: 28100aec96da81c92d5b592353edceed332e2364
Summary:
For some unknown reason, we weren't setting this on Windows, which meant that
whenever edenfs would need to call edenfsctl (like at mount time when fixing up
redirections), it would always use the binary found in the PATH. While in most
cases this is OK, this is not the intended behavior for tests that are expected
to use the just compiled binary, not the one in the PATH.
Reviewed By: genevievehelsel
Differential Revision: D23653027
fbshipit-source-id: f1cc977e44b10c379d2b90bc7972bfec1fccad23
Summary:
On Windows, we were just spawning EdenFS without waiting for it to become
either healthy, or unhealthy. While in most cases EdenFS becomes healthy
shortly after, scripts could race with it and behave weirdly as a consequence.
For instance, `eden clone` would start EdenFS if it isn't started already, and
then immediately clone the repo, that second step may fail and just leave an
empty folder on disk.
On unices, due to EdenFS daemonizing itself, edenfsctl waits for the parent
process to exit which signify that EdenFS is either started, or dead. On
Windows, we can wait on the pid that CreateProcess returns. One drawback is
that the user won't see what `edenfsctl start` is doing while on unices we have
progress bars regarding the repositories being mounted. One approach to
alleviate this would be to use a pipe as the StartupLogger and close it once
EdenFS is initialized.
Reviewed By: fanzeyi
Differential Revision: D22964058
fbshipit-source-id: 4e83c265d22e7e5150ed8b40e2b554cfcd181f8e
Summary:
This extends the metadata importer to work on corp.
Prefetching metadata for the entries in a tree when we fetch it saves us
an extra round trip to the server to fetch a blob when only the metadata
for that blob is fetched. This is particularly important on corp where the
latencies to the server are larger.
Requesting metadata with out the blob itself happens often in the parsing
phase of buck commands, thus this metadata prefetch should improve
the build times, and other work flows that rely heavily on buck queries.
Reviewed By: chadaustin
Differential Revision: D21698728
fbshipit-source-id: 4072be23f2fa7df33cf46879e8c1d8ddd6c316ba
Summary:
Rather than dynamically allocating an event loop in the systemd async
code, make all the corresponding functions async, so the caller is
responsible for threading an event loop down.
Reviewed By: genevievehelsel
Differential Revision: D21894106
fbshipit-source-id: 398c769c30c85a3bb210dbc209f34f9f7336996c
Summary: When calling `eden start` from the CLI layer, make sure to redirect stdin in order to daemonize.
Reviewed By: chadaustin
Differential Revision: D21675707
fbshipit-source-id: 26085cc2ff7774e86f03872030c8885bc3c3b949
Summary:
Currently when you run commands on eden on mac that both fetch data for commit
and update to that commit there is a multithreading bug. This could be a new bug
or an old bug that was not causing problems before, but is showing up a lot more
recently.
See http://sealiesoftware.com/blog/archive/2017/6/5/Objective-C_and_fork_in_macOS_1013.html
for a nice explanation of this.
The root cause of this is still unknown, but this flag should bring back the
old behavior where we were not seeing this bug.
Reviewed By: fanzeyi
Differential Revision: D21744987
fbshipit-source-id: 85092e32212e83b16bc00dc0188f03b643c48eea
Summary:
This script deals with obtaining the right privilege to start the EdenFS
service. This effectively enables `edenfsctl start`.
Reviewed By: wez
Differential Revision: D21585739
fbshipit-source-id: 7b835434d865fa4c4c8473e13665ae669fd86108
Summary:
This makes it a bit easier to write integration tests
that run an isolated EdenFS instance complete with its own
unique scratch path environment.
Reviewed By: genevievehelsel
Differential Revision: D21258455
fbshipit-source-id: 3ea13b6337bb8bb58a237e67488b116c423bbe6c
Summary:
This refactors the `edenfsctl start` command so that we more clearly split the
functionality into two pieces:
* Starting EdenFS as a service
* Running EdenFS in the foreground
In most normal situations in production the `start`, `restart`, and `stop`
commands are used to manage running EdenFS as a service. In the future I
believe our service management logic will start to diverge a bit more on Linux
vs Mac vs Windows, and this should help isolate the service-management code a
bit more cleanly.
The foreground behavior is mainly only used by developers during testing and
during the integration tests. Several options like `--gdb` and `--strace` are
only allowed in foreground mode, and this refactoring makes that clearer. In
the future we may also want to further restrict this, to allow only
specifying additional custom arguments and a custom binary path when running
in foreground mode. However, for now I have not updated that as I believe
some of our integration tests may be exercising this behavior today.
This change also cleans up some of the platform-specific code, and lets them
share more of the logic to construct arguments for edenfs. With this change
`edenfsctl start --foreground` now works on Windows.
Reviewed By: pkaush
Differential Revision: D20833244
fbshipit-source-id: 0f09d59702d8b64ca8f4fedccbc30da1c858afb4
Summary:
Update the `edenfsctl` and `edenfs` code so that `edenfsctl stop` works on
Windows.
Reviewed By: pkaush
Differential Revision: D20854616
fbshipit-source-id: 9a51cdc1f1d1838c5089b945df1fb624a3eaedb7
Summary:
Move the `start_systemd_service()` function from `daemon.py` to a separate
module. This allows `daemon.py` to be imported on Windows without causing any
of the systemd logic to be imported.
I kept this separate from the existing `systemd.py` module for now just to
help keep the dependencies a little bit cleaner. The `systemd.py` module
currently does not depend on any other logic from other edenfsctl modules.
Reviewed By: pkaush
Differential Revision: D20854612
fbshipit-source-id: dbdbf0fec90682e3b7d16a7532aefdd3f186bf80
Summary:
Add methods to check if a process ID is alive, and if it looks like an EdenFS
process.
This also adds an initial version of ProcUtils for Windows, and implements
these two methods on Windows. I have moved parts of the `winproc.py` module
to the new `proc_utils_win.py` module, to help better manage dependencies
between our modules. This keeps all of the Windows-specific `ctypes` code
together in `proc_utils_win.py`. The functionality that is still left on
`winproc.py` depends on `config.py`, and the `proc_utils` code should not
depend on `config.py` to help avoid circular dependencies.
Reviewed By: wez
Differential Revision: D20833245
fbshipit-source-id: 43e9b6dd1b520dcb6b2da7701de885058f0f7ea2
Summary:
Update the `edenfsctl.exe` binary to find `edenfs.exe` relative to itself.
This ensures that when you run `edenfsctl.exe` from a development build that it
finds the associated development build of `edenfs.exe`, rather than the current
system-installed version.
Reviewed By: wez
Differential Revision: D20821419
fbshipit-source-id: 29ff0b587800f1b76c0b729512bc19f5aef648a7
