Summary: In its current state, FORCE is a restart mode that cannot be combined with other modes (specifically `graceful`). Removing this coupling from `force` will allow us to at first attempt a graceful restart, but if the daemon is not healthy, then do a force restart, all within one call.
Reviewed By: wez
Differential Revision: D21873166
fbshipit-source-id: 079a777ff6e7b219198cbef15335244b9865f5f3
Summary:
The Python 2-and-3 Thrift API is sort of deprecated and does not
handle binary data in `binary` fields. In advance of migrating to the
modern Python 3 API, remane eden.thrift to eden.thrift.legacy.
Reviewed By: fanzeyi
Differential Revision: D21889697
fbshipit-source-id: a745ee8977999acbfb383a4edebe81d8deb1734e
Summary:
Under heavy parallelism or system load, our tests could trigger
short-ish timeouts and cause tests to flake. The stats test in
particular often failed in continuous integration. It looks like
opening a unix domain Thrift socket early and holding onto it can
cause it to sometimes hit ThriftServer's default idle timeout of 60
seconds, which results in the test failing with BrokenPipeError
(EPIPE).
Reviewed By: simpkins
Differential Revision: D21780023
fbshipit-source-id: 7e8838429475c2a322d836b9a497411199948cce
Summary:
This diff is auto-generated to upgrade the Pyre version and suppress errors in eden. The upgrade will affect Pyre local configurations in the following directories:
```
eden
```
Differential Revision: D21687853
fbshipit-source-id: baf0d9bc33f86da63ea289690faca6cf4d566588
Summary:
Update the integration test code that uses pexpect to use the more modern
pexpect APIs. The top-level `pexpect.spawn()` function is considered legacy
now, and is only provided for backwards compatibility on Unix platforms.
Reviewed By: wez
Differential Revision: D21214640
fbshipit-source-id: 941da5435c4f8afaf22e8053f4c344175d7b1a7f
Summary:
systemd is Linux-specific, so it does not make sense to try and run these
tests on Linux and Mac. Additionally, the pystemd module currently used by
the edenfsctl CLI is not currently open source, and therefore only works in
our internal Linux builds.
This updates the integration test code to only include systemd-related tests
if we are actually in an environment where we support systemd.
Reviewed By: wez
Differential Revision: D21084095
fbshipit-source-id: fb6c9a066d9dcb741bddf3cdcf9decdb763a1d05
Summary:
This updates the various systemd-related tests to use EdenTestCaseBase,
and allows us to delete the EnvironmentVariableMixin and
SystemdUserServiceManagerMixin classes.
The main goal of this clean-up is to make it easier to consolidate most of the
systemd-related code into just a few locations, so that we can more easily
disable the systemd-related tests in build environments where systemd is not
supported.
Reviewed By: wez
Differential Revision: D21084098
fbshipit-source-id: d5e05254c689c28751fe48d2dc38d722c7e77ed3
Summary:
Replace explicit casts working around T38947910 (Pyre not understanding
decorators) with `pyre-ignore` comments. This will hopefully help ensure that
these are tracked more visibly, and are removed when this task is fixed.
Reviewed By: wez
Differential Revision: D20434082
fbshipit-source-id: da4e4d11e4e029a11984c0efcad2aecd0d3094bf
Summary:
Update the copyright & license headers in Python files to reflect the
relicensing to GPLv2
Reviewed By: wez
Differential Revision: D15487088
fbshipit-source-id: 9f2138dff41048d2c35f15e09a04ae5a9c9c80dd
Summary: Remove a number of unused imports detected by the linter.
Reviewed By: wez
Differential Revision: D15776268
fbshipit-source-id: 221f45d275664d037bbabcac9858b40266b4833e
Summary:
A bug in Pyre causes the properties of FindEXE to have an incorrect type. We currently work around this bug by silencing type errors. Unfortunately, this might silence legitimate errors too.
Instead of silencing type errors, using `typing.cast` to tell Pyre the correct type. This should expose legitimate errors if they exist.
This diff should not change behavior.
Reviewed By: chadaustin
Differential Revision: D13709138
fbshipit-source-id: 55f47f47062a35911c6bbe03ffd7b02a90a5107f
Summary:
Update the ServiceTestCaseBase code so that each test case keeps its state in
a single top-level temporary directory. This makes it a little easier to
figure out which directory is which when debugging a test. I also plan to
write a new test soon that creates some additional directories, and having one
top-level temporary directory avoids needing to create new TemporaryDirectory
objects.
Reviewed By: strager
Differential Revision: D13522026
fbshipit-source-id: 95a3d268d267a107bbf5e405839d64afd6afdb03
Summary:
D13422460 made TemporaryDirectoryMixin required for classes deriving from ServiceTestCaseBase, but ServiceTestCaseBase expressed this requirement poorly. Make the dependency explicit by making ServiceTestCaseBase derive from TemporaryDirectoryMixin.
This diff should not change behavior.
Reviewed By: simpkins
Differential Revision: D13454141
fbshipit-source-id: e07745cfd2a364da5011371fe8671d94d32c6798
Summary:
I noticed some Managed service tests behaving strangely on my dev server. They behave as if they are being managed by systemd.
I noticed that `systemctl --user status 'fb-edenfs@*.service'` showed a bunch of tmp-eden_test services. Since the Managed tests don't create an isolated systemd instance, they are starting and stopping services on my real systemd!
This behavior is caused by me setting service.experimental_systemd=true in my ~/.edenrc (D13371186).
Fix the odd behavior of these tests by preventing them from reading ~/.edenrc. Also do the same for /etc/eden.
Reviewed By: simpkins
Differential Revision: D13422460
fbshipit-source-id: b8a4cbabe55b75b34729d4122ba804cd7d3297a2
Summary:
If the user opted into systemd integration, make 'eden restart' restart the systemd service.
Don't support --graceful yet. --graceful will be implemented in a later diff. Mark it as explicitly unsupported for now.
Reviewed By: chadaustin
Differential Revision: D13271438
fbshipit-source-id: e505f00cbc337a2bf4da77bdea4b8faba063607c
Summary:
Add the plumbing necessary to make 'eden start' start a systemd user service. This is only enabled if you opt in using EDEN_EXPERIMENTAL_SYSTEMD.
Currently, only fake_edenfs works. The real edenfs doesn't work yet because it needs root access to configure mount points.
'eden restart', 'eden stop', etc. are not affected by this diff (and are probably broken with EDEN_EXPERIMENTAL_SYSTEMD enabled).
Reviewed By: simpkins
Differential Revision: D10849390
fbshipit-source-id: c087a6498951ff100e5c80bd07ad869b2709e1b3
Summary: CLI tests use shutil.rmtree to clean up temporary directories. Reuse TemporaryDirectoryMixin which is more robust against errors and supports EDEN_TEST_NO_CLEANUP.
Reviewed By: chadaustin
Differential Revision: D13268108
fbshipit-source-id: d77e95a2def0dceb34cf14e19c0c0c0e3aeef3f2
Summary: While creating a systemd service unit for EdenFS, I noticed a few features of the current implementation of 'eden start', 'eden stop', and 'eden restart' which are not covered by tests. Write tests for these features to ensure they don't regress with systemd integration.
Reviewed By: wez
Differential Revision: D10434379
fbshipit-source-id: 6c828a85d59179bbc4beda87e1bf2534543b60b2
Summary:
Right now, 'eden start' spawns edenfs directly (using `os.exec`). In the future, 'eden start' will spawn edenfs using systemd. 'eden restart', 'eden stop', and 'eden status' might behave differently depending on how the edenfs process was spawned.
Parameterize some tests on how fake_edenfs was originally started. This flexibility lets us easily spawn fake_edenfs with 'systemctl start' (systemd) in the future. For now, spawn fake_edenfs in ad-hoc mode (i.e. directly without 'eden start') or in managed mode (i.e. indirectly with 'eden start').
Reviewed By: chadaustin
Differential Revision: D10414995
fbshipit-source-id: 2ee0d5df8a605ca9d7da8f6eeca1fc171a8342e8
Summary:
RestartTest uses 'eden restart' during setup to start the fake_edenfs process. There are two issues with doing this:
* I want to extend the tests in RestartTest so they also tests with an ad-hoc fake_edenfs process (D10414995). Restarting edenfs during setup makes no sense in that context.
* I think using 'eden restart' during setup is clunky and unintuitive. 'eden start' is more appropriate.
Refactor RestartTest's setup so it uses 'eden start' instead. Also add a test which explicitly verifies that 'eden restart' starts fake_edenfs if no daemon is already running.
Reviewed By: chadaustin
Differential Revision: D10417304
fbshipit-source-id: 20f8190026cd153dd9b539067f6f63b6bd27abed
Summary:
In Python 3, `pexpect.spawn` objects come in two flavors: bytes and unicode. The flavor is selected using `pexpect.spawn.__init__`'s `encoding` parameter.
For the bytes flavor, `pexpect.spawn`'s instance attributes generally use `bytes`:
class spawn:
args: Sequence[bytes]
before: bytes
command: str
For the unicode flavor, `pexpect.spawn`'s instance attributes generally use `str`:
class spawn:
args: Sequence[str]
before: str
command: str
Teach the type checker (mypy) about these two flavors to reduce false positives in calling code.
Reviewed By: simpkins
Differential Revision: D10280499
fbshipit-source-id: b4475226423ef37ff5bf2bbae0eb9deac6a1099a
Summary:
RestartTest._spawn_restart dynamically types its kwargs parameter, hiding potential type errors. The kwargs parameter is unused anyway, so delete it and tidy up the related code.
This diff should not change behavior.
Reviewed By: pkaush
Differential Revision: D10281769
fbshipit-source-id: 3e7f7078edf9e47013a9eb9ba42cf51fa8dd0a16
Summary: Several tests create a temporary directory and destroy it. Consolidate the temporary directories and their cleanup code to reduce code duplication.
Reviewed By: simpkins
Differential Revision: D10251110
fbshipit-source-id: caa5b5ad49fcb7925f63094594af4a20009d790d
Summary:
Update `eden restart` to explain that the restart will be disruptive when
performing a full restart, and prompt the user for confirmation. This prompt
can be overridden with the `--force` option. Also print more messaging after
the restart completes telling users what to do if they see "Transport endpoint
not connected" errors from programs still using the old mount points.
This also updates `eden restart` to more carefully handle cases where there is
an existing edenfs daemon running but it is not fully healthy.
Reviewed By: wez
Differential Revision: D8731004
fbshipit-source-id: 05762187b47057b2930d0a6b71b0a6fdbd4aa7e5
Summary:
To me, restart implied shutting down the daemon and restarting it
again. Perhaps instead of `eden daemon --takeover` we should have
`eden restart`. But if people typed `eden restart` I imagine they're
trying to debug a problem, so that's probably not the right verb.
Reviewed By: wez
Differential Revision: D6929166
fbshipit-source-id: d568a1940d67f755e4c3656098c58fc81e0a3156
Summary:
This implements a TODO/FATAL that is important for
graceful restarts to be useful in my "acid test" scenario,
which is to perform a graceful restart while buck build is
running.
Reviewed By: simpkins
Differential Revision: D6700189
fbshipit-source-id: dec1b818ebc9e907841bc127ee08c953b59d6487
Summary:
I'm so-so on a bit of the implementation here, but it works!
I had to change the `takeoverPromise` from the `pair<fuseDevice, connInfo>`
to a new helper struct because we now have three distinct pieces of data
to pass out of EdenMount to build up the overall TakeoverData.
The key change in this diff is that we have to release all of the file handles
we're maintaining in the `FileHandleMap` prior to shutting down the `InodeMap`,
otherwise the `InodeMap` will never complete (it waits for all inodes to be
unreferenced, and that cannot happen while there are open file handles). I've
made the `FileHandleMap` serialization and clearing contingent on performing a
takeover shutdown because that feels like the safest thing to do wrt. not
losing any pending writes.
Reviewed By: simpkins
Differential Revision: D6672437
fbshipit-source-id: 7b1f0f8e7ff09dbed850c7737383ecdf1e5ff0c7
Summary:
This is the key portion that makes the graceful restart
function. This diff connects almost all of the moving pieces together;
it informs the priv helper about the takeover mount and transfers
the InodeMap information into the new generation of the eden server.
It doesn't yet load the fileHandleMap (will tackle that in a follow up diff)
Reviewed By: simpkins
Differential Revision: D6670903
fbshipit-source-id: 1770d99eb1477440a6c1deed83b0da55b9c1bbe4
Summary:
This begins implementing the "client-side" portion of graceful takeover in
edenfs. When the --takeover flag is specified, if edenfs finds that another
edenfs process is already running it will attempt to gracefully take over its
mount points rather than exiting with an error.
This does not yet actually take over the mount points themselves--it still
sends dummy mount information during shutdown, and does not use this data
during startup. However, we do perform takeover of the eden lock file and the
thrift server socket.
Reviewed By: bolinfest
Differential Revision: D6038944
fbshipit-source-id: 42406a0559367cec79af088b4ca84c22de3f3ef3
