Summary:
This is another case where we had issues with the long default temp path on
macOS. Here, fake_edenfs was failing with SIGABRT on startup due to trying to
open a unix-domain socket with too long of a path.
D43925614 addressed this for tests based on EdenTestCase, but this doesn't help
with StartFakeEdenFSTestBase which only inherits from EdenTestCaseBase. So
this diff moves the environment variable override one level up in the type
hierarchy to cover more test cases.
Reviewed By: kmancini
Differential Revision: D43988120
fbshipit-source-id: 9eca299c42aea819d3e93ed5bb408b4a4783d8ee
Summary:
Applies new import merging and sorting from µsort v1.0.
When merging imports, µsort will make a best-effort to move associated
comments to match merged elements, but there are known limitations due to
the diynamic nature of Python and developer tooling. These changes should
not produce any dangerous runtime changes, but may require touch-ups to
satisfy linters and other tooling.
Note that µsort uses case-insensitive, lexicographical sorting, which
results in a different ordering compared to isort. This provides a more
consistent sorting order, matching the case-insensitive order used when
sorting import statements by module name, and ensures that "frog", "FROG",
and "Frog" always sort next to each other.
For details on µsort's sorting and merging semantics, see the user guide:
https://usort.readthedocs.io/en/stable/guide.html#sorting
Reviewed By: lisroach
Differential Revision: D36402162
fbshipit-source-id: 6d180e9003d466c4f866fc9d454c6531766ca1dd
Summary:
We've seen cases in CI where the length of the path to the socket is just over
108 characters, breaking tests. Let's shave 10 characters to be under the
limit.
Reviewed By: fanzeyi
Differential Revision: D34046658
fbshipit-source-id: 249b003db2a1f18f53c28702d81577ff55dcb237
Summary:
With Facebook having been renamed Meta Platforms, we need to change the license
headers.
Reviewed By: fanzeyi
Differential Revision: D33407812
fbshipit-source-id: b11bfbbf13a48873f0cea75f212cc7b07a68fb2e
Summary:
With D30320515, EdenFS internally canonicalize all the mount paths passed to
it. As a result, the output of `eden list` may not match the path given to
`eden mount` if one of the directory leading to the mount point was a symlink.
Since some tests are comparing both, this can lead to some test failures.
To solve this, we simply need to make sure that the temporary directory for the
test is canonicalized.
Reviewed By: fanzeyi
Differential Revision: D30349411
fbshipit-source-id: 139d4be02b5783c6a439270845239acab6a6c955
Summary:
This applies the formatting changes from black v21.4b2 to all covered
projects in fbsource. Most changes are to single line docstrings, as black
will now remove leading and trailing whitespace to match PEP8. Any other
formatting changes are likely due to files that landed without formatting,
or files that previously triggered errors in black.
Any changes to code should be AST identical. Any test failures are likely
due to bad tests, or testing against the output of pyfmt.
Reviewed By: thatch
Differential Revision: D28204910
fbshipit-source-id: 804725bcd14f763e90c5ddff1d0418117c15809a
Summary:
Somehow, rmtree can fail to open the file being removed, in which case the
function is `os.open` that requires a flag on top of the path. When that
happens a `TypeError` will be raised:
```
✗ Fail: eden/integration:integration - test_eden_start_fails_if_edenfs_fails_during_startup (eden.integration.start_test.StartFakeEdenFSTestSystemdEdenCLI) (4.363)
Test output:
> TypeError: open() missing required argument 'flags' (pos 2)
File "/usr/local/fbcode/platform009/lib/python3.8/unittest/case.py", line 60, in testPartExecutor
yield
File "/usr/local/fbcode/platform009/lib/python3.8/unittest/case.py", line 717, in doCleanups
self._callCleanup(function, *args, **kwargs)
File "/usr/local/fbcode/platform009/lib/python3.8/unittest/async_case.py", line 72, in _callCleanup
self._callMaybeAsync(function, *args, **kwargs)
File "/usr/local/fbcode/platform009/lib/python3.8/unittest/async_case.py", line 84, in _callMaybeAsync
ret = func(*args, **kwargs)
File "/usr/local/fbcode/platform009/lib/python3.8/contextlib.py", line 533, in close
self.__exit__(None, None, None)
File "/usr/local/fbcode/platform009/lib/python3.8/contextlib.py", line 525, in __exit__
raise exc_details[1]
File "/usr/local/fbcode/platform009/lib/python3.8/contextlib.py", line 510, in __exit__
if cb(*exc_details):
File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/aab7ed39/eden/integration/integration#binary,link-tree/eden/test_support/temporary_directory.py", line 104, in __exit__
self.cleanup(exc_type is not None)
File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/aab7ed39/eden/integration/integration#binary,link-tree/eden/test_support/temporary_directory.py", line 117, in cleanup
cleanup_tmp_dir(temp_dir)
File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/aab7ed39/eden/integration/integration#binary,link-tree/eden/test_support/temporary_directory.py", line 71, in cleanup_tmp_dir
shutil.rmtree(tmp_dir, onerror=_remove_readonly)
File "/usr/local/fbcode/platform009/lib/python3.8/shutil.py", line 715, in rmtree
_rmtree_safe_fd(fd, path, onerror)
File "/usr/local/fbcode/platform009/lib/python3.8/shutil.py", line 652, in _rmtree_safe_fd
_rmtree_safe_fd(dirfd, fullname, onerror)
File "/usr/local/fbcode/platform009/lib/python3.8/shutil.py", line 652, in _rmtree_safe_fd
_rmtree_safe_fd(dirfd, fullname, onerror)
File "/usr/local/fbcode/platform009/lib/python3.8/shutil.py", line 652, in _rmtree_safe_fd
_rmtree_safe_fd(dirfd, fullname, onerror)
File "/usr/local/fbcode/platform009/lib/python3.8/shutil.py", line 648, in _rmtree_safe_fd
onerror(os.open, fullname, sys.exc_info())
File "/data/users/xavierd/fbsource/fbcode/buck-out/dev/gen/aab7ed39/eden/integration/integration#binary,link-tree/eden/test_support/temporary_directory.py", line 66, in _remove_readonly
func(path)
```
To bypass this, let's simply test if the function is os.unlink or os.rmdir
first.
Reviewed By: chadaustin
Differential Revision: D27654126
fbshipit-source-id: 318c547f363c020e98e21d5f07a9c7a9b2e5102d
Summary: I think that the broken config (wrong relative base for search_path), was what prevented the upgrade from going automatically.
Reviewed By: grievejia
Differential Revision: D22966243
fbshipit-source-id: 4ef42a8e2e6f2c79483301c6876509a3009a83d1
Summary:
Python 3.8 provides the (poorly named)
unittest.IsolatedAsyncioTestCase base class that gives each test an
asyncio event loop and runs it. This allows us to write async
integration tests, required to use modern Thrift Python.
Reviewed By: genevievehelsel
Differential Revision: D21955584
fbshipit-source-id: 8366dc7c72b31e1667e72fe1e359587b74e2c614
Summary:
Unawaited coroutines are pernicious and not errors by default. In
advance of enabling asynchronous tests, fail our tests when Python
warnings are shown.
We record warnings and report them after the test body instead of
enabling an "error" warning filter because exceptions thrown by
finalizers are silently swallowed, and most important warnings occur
during finalization.
Reviewed By: genevievehelsel
Differential Revision: D21955210
fbshipit-source-id: b1fc52ddfa9f9befbde6800f85f090862af0ef15
Summary:
It looks like a refactoring broke the EDEN_TEST_NO_CLEANUP feature
which was useful when trying to debug the eden state directory after a
failure.
While fixing it, I went ahead and added support for only saving the
state directory upon test failure.
Reviewed By: simpkins
Differential Revision: D21780122
fbshipit-source-id: 0cd6e9f274601eebd9b4a6978c0cf61fb1b85545
Summary:
The new temporary file code added in D20885160 was missing a call to close the
underlying file in the `__exit__()` function.
Reviewed By: xavierd
Differential Revision: D21271901
fbshipit-source-id: 653258d471421edd08b473654ae3d0503f3c6829
Summary:
This updates the various systemd-related tests to use EdenTestCaseBase,
and allows us to delete the EnvironmentVariableMixin and
SystemdUserServiceManagerMixin classes.
The main goal of this clean-up is to make it easier to consolidate most of the
systemd-related code into just a few locations, so that we can more easily
disable the systemd-related tests in build environments where systemd is not
supported.
Reviewed By: wez
Differential Revision: D21084098
fbshipit-source-id: d5e05254c689c28751fe48d2dc38d722c7e77ed3
Summary:
This adds a new `EdenTestCaseBase` class to serve as the base class across a
number of our integration tests and some of our CLI tests.
The main goal of this is to allow eliminating many of the annoying `*Mixin`
classes used in a lot of our integration tests. These mixin classes are
annoying since they result in complicated multiple inheritance, and it can be
tricky to ensure that the method resolution order behaves the way you want.
The systemd tests in particular use a lot of mixins, which gets complicated.
These mixin classes are also awkward from a Python typing perspective, and the
systemd tests end up resorting to just declaring different APIs in several
places when `typing.TYPE_CHECKING` is True.
The fact that `EdenTestCaseBase` has a `contextlib.ExitStack` member variable
should make it easier for us to eliminate these mixins moving forward: rather
than using mixins that use inheritance and assume a `self.cleanUp()` method
exists, we can transition this code to standalone functions or context
managers, and they can take the `ExitStack` variable as an argument if
necessary.
Reviewed By: wez
Differential Revision: D21084097
fbshipit-source-id: 77ee457b7debe6f584f630e3e30f79fe634a2026
Summary:
Unfortunately the Python standard library's `tempfile.NamedTemporaryFile`
class does not work very well on Windows: even though the temporary files are
named, they cannot actually be opened by other processes. Attempting to open
the file fails with a permission denied error.
This code extends our existing temporary directory management logic to also
add APIs for creating individual temporary files. This has the advantage of
better grouping all temporary files for a given test in the same directory. I
also updated the code to include the test function name in the temporary
directory prefix. This should make it a little easier to identify which test
is at fault when there are temporary files left behind after a test. (This
can happen if the test gets killed or if it leaves behind files or mount
points that cannot be removed normally).
Reviewed By: wez
Differential Revision: D20885160
fbshipit-source-id: 7267b2352e51214354eab7ead839bf166d690974
Summary:
Sadly, we didn't derive as much value from these as we would
have liked, and now they are a source of flakeyness in our tests.
Reviewed By: genevievehelsel
Differential Revision: D19460792
fbshipit-source-id: 48c82cc2d1fdbd81bece4057e93799fbcc4f4725
Summary:
Add initial support for building and running some of the integration tests
with CMake. For now this just runs the tests from basic_test.py, just to
confirm that most of the framework code works in CMake-based builds.
Many of the other tests should also work as well, but a few of them we may
want to disable for CMake-based builds. e.g., a couple of the tests depend on
hypothesis, and we would need to include hypothesis as a dependency. Some of
the tests that use systemd might also require a little more work to get
working.
Reviewed By: fanzeyi
Differential Revision: D17659026
fbshipit-source-id: 67420fda9e1021a0cddee2d385fd21e34fb2fd70
Summary:
Update the copyright & license headers in Python files to reflect the
relicensing to GPLv2
Reviewed By: wez
Differential Revision: D15487088
fbshipit-source-id: 9f2138dff41048d2c35f15e09a04ae5a9c9c80dd
Summary:
Currently, users can opt into systemd for edenfs by setting EDEN_EXPERIMENTAL_SYSTEMD=1 every time they use Eden's CLI. Make it easier to opt in: use systemd if service.experimental_systemd is set in ~/.edenrc. Also allow opting out for a single command via EDEN_EXPERIMENTAL_SYSTEMD=0.
For users who don't set service.experimental_systemd or use EDEN_EXPERIMENTAL_SYSTEMD, this diff should not change behavior. (The EDEN_EXPERIMENTAL_SYSTEMD environment variable is still respected.)
Reviewed By: simpkins
Differential Revision: D13371186
fbshipit-source-id: d7b533ea3583f4b6c8a8f63c6a74eace2be7d783
Summary:
I want to allow opting into systemd using a setting in ~/.edenrc. Since should_use_experimental_systemd_mode is a global function, it can't read any configs. Make the config file visible to should_use_experimental_systemd_mode by moving it into EdenInstance.
This diff should not change behavior.
Reviewed By: simpkins
Differential Revision: D13370823
fbshipit-source-id: b604db66954d0a08973030daae38bf6b1433e821
Summary: CLI tests use shutil.rmtree to clean up temporary directories. Reuse TemporaryDirectoryMixin which is more robust against errors and supports EDEN_TEST_NO_CLEANUP.
Reviewed By: chadaustin
Differential Revision: D13268108
fbshipit-source-id: d77e95a2def0dceb34cf14e19c0c0c0e3aeef3f2
Summary:
EdenFS' systemd service will be configurable using config files. The Eden CLI will communicate information, such as extra daemon arguments, to systemd using a file specified using EnvironmentFile= [1]. Write code to serialize and deserialize these files.
The code introduced by this diff isn't used yet, but it will be used in future diffs.
[1] https://www.freedesktop.org/software/systemd/man/systemd.exec.html#EnvironmentFile=
Reviewed By: chadaustin
Differential Revision: D12066543
fbshipit-source-id: 28ba38d823b92f7a3a7ad97c416120a7f95be122
Summary:
I want to use Hypothesis to fuzz some CLI code. Move EdenFS' Hypothesis configuration out of eden.integration.lib.testcase and into a place where both CLI tests and integration tests can use it.
This diff should not change behavior.
Reviewed By: wez
Differential Revision: D12813285
fbshipit-source-id: 3a1badd1e18b0e070295ea03dcb24be166cd42c1
