sapling/eden/fs/monitor
Wez Furlong 154d7309c9 eden: introduce SpawnedProcess
Summary:
This commit introduces a new process spawning class derived
from the ChildProcess class in the watchman codebase.

`SpawnedProcess` is similar to folly::Subprocess but is designed around the
idea that we will use a system provided spawning API to start a process, rather
than assuming the use of `fork`.

`fork` is to be avoided because it can be expensive for processes with large
address spaces and also because it interacts poorly with threads on macOS.  In
particular, we see the objC runtime terminating our process in some scenarios
where fork and threads are mixed.

There are some important differences from `folly::Subprocess` and that means
that some assumptions and uses need to be altered slightly from their prior
workings.  For example, detaching a SpawnedProcess moves the responsibility of
waiting on the child to a periodic task as there is no way to detach via
posix_spawn without also using fork.

On the plus side, this commit allows unifying spawning between posix and
windows systems, which simplifies the code!

Reviewed By: xavierd

Differential Revision: D23287763

fbshipit-source-id: b662af1d7eaaa9ed445c42f6c5765ae9af975eea
2020-09-01 13:31:32 -07:00
..
test add log rotation support to edenfs_monitor 2020-05-07 20:05:45 -07:00
EdenInstance.cpp eden: introduce SpawnedProcess 2020-09-01 13:31:32 -07:00
EdenInstance.h eden: introduce SpawnedProcess 2020-09-01 13:31:32 -07:00
EdenMonitor.cpp enable log rotation in edenfs_monitor 2020-05-07 20:05:46 -07:00
EdenMonitor.h enable log rotation in edenfs_monitor 2020-05-07 20:05:46 -07:00
LogFile.cpp enable log rotation in edenfs_monitor 2020-05-07 20:05:46 -07:00
LogFile.h enable log rotation in edenfs_monitor 2020-05-07 20:05:46 -07:00
LogRotation.cpp add log rotation support to edenfs_monitor 2020-05-07 20:05:45 -07:00
LogRotation.h add log rotation support to edenfs_monitor 2020-05-07 20:05:45 -07:00
main.cpp enable log rotation in edenfs_monitor 2020-05-07 20:05:46 -07:00
README.md add a new process to monitor EdenFS 2020-01-31 13:22:26 -08:00

This directory contains a wrapper process that monitors the EdenFS daemon. This wrapper process serves a few purposes:

Simplifies management of EdenFS across graceful restarts

This monitoring process provides a single parent process that can be monitored by systemd and other system management daemons, even across EdenFS graceful restarts. When a graceful restart is desired this wrapper daemon can spawn the new EdenFS instance, so that the new EdenFS instance is still part of the original service process hierarchy.

Note that using a wrapper for this purpose is not strictly required with systemd (it is possible to inform systemd that the main process ID has changed and it should monitor a new process moving forward). However, this wrapper provides us a bit more flexibility and control around the restart mechanism, and also makes it easier to monitor EdenFS with other service management frameworks on other platforms.

Log file management and rotation

This process reads all messages printed by EdenFS to stdout and stderr, and writes them to a log file, performing log rotation when appropriate.

Implementing log rotation properly is tricky otherwise, as there are many different sources that can end up writing data to EdenFS's stdout/stderr descriptors, including separate processes like the privhelper process and spawned Python subprocesses.

Intelligent Restarting of EdenFS when it is Idle

This wrapper process supports requests to trigger a restart at some point in the future when EdenFS appears to be idle.

While graceful restart should minimize user-visible disruption, it can still introduce a delay for I/O operations while the restart is in progress. Therefore it is still desirable to try and perform the restart while users are not actively accessing the file system, if possible.

This functionality is provided by the wrapper primarily because the wrapper provides a convenient location to centralize this management in case multiple restart attempts are requested before EdenFS becomes idle.