Summary:
I observed an integration test fail because Mercurial aborted with an
error about not handling EINTR, but it had no traceback. Always run
Mercurial with --traceback to try to debug these.
Reviewed By: xavierd
Differential Revision: D24204308
fbshipit-source-id: 44960bc645e5f97f61761e511b372328430fcda7
Summary:
Formats a subset of opted-in Python files in fbsource.
Black formatting was applied first, which is guaranteed
safe as the AST will not have changed during formatting.
Pyfmt was then run, which also includes import sorting.
The changes from isort were manually reviewed, and
some potentially dangerous changes were reverted,
and the directive was added to those
files. A final run of pyfmt shows no more changes to
be applied.
Reviewed By: zertosh
Differential Revision: D24101830
fbshipit-source-id: 0f2616873117a821dbc6cfb6d8e4f64f4420312b
Summary:
Now that `hg whereami` properly reads the SNAPSHOT file on Windows, the doctor
tests properly detect that Mercurial and EdenFS disagree about the current
commit, thus we can enable the remaining 2 tests.
Reviewed By: genevievehelsel
Differential Revision: D23819924
fbshipit-source-id: 21be19aff913e5e485d72e8cd730e6851ecaba2e
Summary:
on macOS we cannot safely use `fork`.
This commit replaces the use of `fork` in the startup logger subsystem.
This was a little tricky to untangle; originally (prior to any of
the `fork` removal efforts in this diff stack), the startup flow was
to spawn a set of processes via fork:
```
edenfs (setuid)
\-----edenfs (privhelper, as root)
\------edenfs (daemonized)
```
The forked children take advantage of being able to implicitly pass state to
the child processes from the parent. That data flow needs to become explicit
when removing the fork which makes some things a little awkward.
With fork removed:
* `edenfs` unconditionally spawns `edenfs_privhelper` while it has
root privs and before most of the process has been initialized.
* That same `edenfs` process will then spawn a child `edenfs`
process which starts from scratch, but that which needs to
run as the real server instance
* The original `edenfs` instance needs to linger for a while
to remain connected to the controlling tty to pass back the
startup state to the user, before terminating.
This commit deletes the check that `edenfs` is started originally
as root; previously the logic relied on the forked startup logger
continuing past the `daemonizeIfRequested` call and simply deferring
the check until after folly::init. With these changes we can't
easily perform such a check without adding some extra gymnastics
to pass the state around; the place where that is checked is in
the spawned child of the original edenfs, which is not a privileged
process and doesn't know the original euid. I don't believe this
to be a great loss as we tuck `edenfs` away under the libexec dir.
Reviewed By: chadaustin
Differential Revision: D23696569
fbshipit-source-id: 55b95daf022601a4699274d696af419f0a11f6f2
Summary: This diff fixes the eden redirection tests so it runs on Windows.
Reviewed By: xavierd
Differential Revision: D22958766
fbshipit-source-id: 45d26587831ed74d6bd7912b22c7c955b077f571
Summary:
tpx doesn't support heavyweight tags or rate limiting, and integration
tests regularly fail with timeouts on my devbig, so bump the process
start and process stop timeouts.
Reviewed By: genevievehelsel
Differential Revision: D23553924
fbshipit-source-id: fa9b8710395d61b087963d18718137e4525ae03d
Summary:
30 seconds is not enough time on heavily contented systems, including
CI. Bump the shutdown timeout to 120.
Also, correctly send SIGKILL to the daemon process when it's been
started with sudo.
Reviewed By: simpkins
Differential Revision: D22422784
fbshipit-source-id: dc7be0962705f1feb9643990309f570e352b68a0
Summary:
This test can't be non-flaky, because it relies on the kernel deciding
when to drop inodes from cache, and we've investigated it multiple
times. Given it tests a rarely-used function that would be better
expressed as a unit test in C++, just remove it for now.
Reviewed By: wez
Differential Revision: D23665455
fbshipit-source-id: 522e47113857eff399be4f2bb60e26e801d61e9a
Summary:
We intend to rename the edenfs device type to include a colon (and
possibly the backing repo basename). In preparation, update code that
detects edenfs mounts to include anything that starts with "edenfs:".
Reviewed By: genevievehelsel
Differential Revision: D23520008
fbshipit-source-id: 280f7617d5c96e23d548041b3482bca388076a7b
Summary:
We are trying to get eden running in our atypical EDA corp environment.
When testing it out on one of these machines, we got things sorted out to the point where the only test failures were coming from this file.
chadaustin identified this as being a test of dead code, and so we decided to go for a deletion of it.
If this work resumes, these tests can be retrieved from version control and then be made to work on Centos7 (hopefully at that point we'll also have contbuild/utd magic set-up to have that re-enabling automatically trigger the build/test).
Reviewed By: genevievehelsel
Differential Revision: D23463831
fbshipit-source-id: 7714547c04573b94dbb2d9acf7906734d853c5aa
Summary:
Most of the fixes are pretty trivial as the code was using functions not
present on Windows, either work around them, or switch to ones that are
multi-platform.
Of note, it looks like `hg doctor` doesn't properly detect when Mercurial and
EdenFS are out of sync, disabling the tests until we figure out why.
Reviewed By: genevievehelsel, fanzeyi
Differential Revision: D23409708
fbshipit-source-id: 3314c197d43364dda13891a6874caab4c29e76ca
Summary:
From a quick experiment, this greatly cuts down on the amount requests to
nonexistent files. For instance, the .hg directory in folders is now only
looked up once and no longer afterwards.
Reviewed By: wez
Differential Revision: D23112343
fbshipit-source-id: 223134ca591054ae9ac2e839033bbd1b714443da
Summary:
On Windows, unmounting the repo isn't sufficient to remove all traces of the
repo as ProjectedFS uses it as a cache, and thus all the files that were
touched by the user are still present on disk after unmounting it.
Reviewed By: wez
Differential Revision: D22915349
fbshipit-source-id: 445379fef57160b5da39a298ca4518e2662d16e3
Summary:
On Windows, this has the benefit of automatically converting \r\n into \n,
which allows more tests to pass.
Reviewed By: chadaustin
Differential Revision: D22871408
fbshipit-source-id: 02ec1d21dc236175c3b0f3176db9b8c91dee21a4
Summary:
On Windows, paths are separated by \, but the test was comparing them against
/. We can simply ask Mercurial to return / with the slashpath template filter.
Reviewed By: chadaustin
Differential Revision: D22871407
fbshipit-source-id: 421bd14f752f29265b12eb25609d4f65e593dda8
Summary:
Cache invalidation is hard, and on Windows we avoided doing a lot of them. It
turns out, this was the wrong decision as it's fairly easy to find cases where
the filesystem view is different from the manifest state.
Since the Linux code is most likely correct in where the invalidation is done,
let's also do the same on Windows, removing a whole lot of #ifdef. It is very
likely that as a result of this diff we end up invalidating more than needed,
thus slowing down EdenFS, but at this point I'd prefer to err on the side of
correctness, performance will come later.
While invalidating files should use PrjDeleteFile, for directories, we simply
need to mark them as placeholder, as directories created by a user won't have a
placeholder, thus ProjectedFS would bypass EdenFS when listing in.
Reviewed By: chadaustin
Differential Revision: D22833202
fbshipit-source-id: d807557f5e44279c49ab701b7a797253ef1f0717
Summary: The test is passing, enable it.
Reviewed By: genevievehelsel
Differential Revision: D22798421
fbshipit-source-id: aec5302aad38d3413385bf5f0242800d685fb5ef
Summary: The test is passing, enable it.
Reviewed By: genevievehelsel
Differential Revision: D22798424
fbshipit-source-id: 76b99457aacf5a81c2b9b3ebaedd0e6e1cf2a1e8
Summary: The test is passing, enable it.
Reviewed By: genevievehelsel
Differential Revision: D22798422
fbshipit-source-id: 0f5a106be51e319a0d317900cc88de1131b95e4d
Summary: Out of the 5 tests, 3 are passing, let's enable them.
Reviewed By: genevievehelsel
Differential Revision: D22798418
fbshipit-source-id: 5bd3fd90945e5556de7838a5dc61ad00865a6d55
Summary: Half of them are passing, let's make sure they run.
Reviewed By: genevievehelsel
Differential Revision: D22798423
fbshipit-source-id: b762213ffad431de6f54acb8ceffa54f28f5909d
Summary: The test is passing, enable it.
Reviewed By: genevievehelsel
Differential Revision: D22798417
fbshipit-source-id: c45531a75a3db61e33f95ef80eb007f12dd02f2a
Summary: The test is passing, enable it.
Reviewed By: genevievehelsel
Differential Revision: D22794679
fbshipit-source-id: 8d5f5a9b9fd7750034baeda814bfeff29882d409
Summary: These are all passing, enable them.
Reviewed By: genevievehelsel
Differential Revision: D22794158
fbshipit-source-id: c381a906e096c3642248a2521e8e7772d74f992e
Summary: All of the tests are passing, enable them.
Reviewed By: genevievehelsel
Differential Revision: D22794159
fbshipit-source-id: 8b7e33f3abbde88e06488f7fe9ea7217d204e70e
Summary: The test is passing, enable it.
Reviewed By: chadaustin
Differential Revision: D22794161
fbshipit-source-id: 94d167b88782d386ca60b9215f1c2aef56a88a4d
Summary: The only ones not passing are the FUSE ones, which is expected. Enable all the others.
Reviewed By: genevievehelsel
Differential Revision: D22794162
fbshipit-source-id: 31f51fd5fff8e3ced75924533fa536208dabf11d
Summary: The test is passing, enable it.
Reviewed By: chadaustin
Differential Revision: D22794155
fbshipit-source-id: 286c9f1129d279487019206d58521951c768cbb1
Summary: 2 out of 3 tests are passing, let's run these.
Reviewed By: chadaustin
Differential Revision: D22794160
fbshipit-source-id: e62e1956980689b083107ebb3b8340880e0d72a6
Summary:
Besides the 3 listed, all the other 13 are passing, let's make sure we run them
to not regress.
Reviewed By: chadaustin
Differential Revision: D22794164
fbshipit-source-id: 5660cc36365de96a2b9e62e3462c01b39904d2f3
Summary:
we've seen flaky tests here on stress test runs, lets subtract time.time() by an epsilon of one to ensure that the comparison is only made within a second precision (time.time() worst case precision)
an example error: `AssertionError: 1595414015.5022793 not greater than or equal to 1595414015.5025299`
`AssertionError: 1595759609.439305 not greater than or equal to 1595759609.441682`
Reviewed By: chadaustin
Differential Revision: D22687496
fbshipit-source-id: 8e87148d620577e3198d2845d785a87a909cd1d3
Summary:
Disregard two default systemd services on Fedora 32 that cause our
systemd tests to fail.
I believe genevievehelsel is planning on removing this code soon anyway.
Reviewed By: genevievehelsel
Differential Revision: D22713393
fbshipit-source-id: b703b23a3158cb007dc2e1cb53fae36be7282719
Summary:
We are unifying C++ APIs for accessing optional and unqualified fields:
https://fb.workplace.com/groups/1730279463893632/permalink/2541675446087359/.
This diff migrates code from accessing data members generated from unqualified
Thrift fields directly to the `field_ref` API, i.e. replacing
```
thrift_obj.field
```
with
```
*thrift_obj.field_ref()
```
The `_ref` suffixes will be removed in the future once data members are private
and names can be reclaimed.
The output of this codemod has been reviewed in D20039637.
The new API is documented in
https://our.intern.facebook.com/intern/wiki/Thrift/FieldAccess/.
drop-conflicts
Reviewed By: yfeldblum
Differential Revision: D22631599
fbshipit-source-id: 9bfcaeb636f34a32fd871c7cd6a2db4a7ace30bf
Summary:
Bypass truncation-based transaction if narrow-heads is on.
The transaction abort still works logically because commit references stay
unchanged on abort.
Related EdenFS and Mononoke tests are updated. Mononoke tests probably
shouldn't rely on revlog / fncache implementation details in hg.
Reviewed By: DurhamG
Differential Revision: D22240186
fbshipit-source-id: f97efd60855467b52c9fb83e7c794ded269e9617
Summary:
An assertion error is raised if `eden doctor` is in the middle of a merge. This is because we enter a specific "if" condition in the case that mercurial has two parent commits, and EdenFS only ever tracks `p0`, so EdenFS simply sets `p1` to the null commit in `_select_new_parents()`. Specifically, this is in the case in which both `_old_dirstate_parents` and `_old_snapshot` are not None.
Because `_old_dirstate_parents` has `p1` set to nonnull, and Eden thinks it is null , the check `self._new_parents != self._old_dirstate_parents` would be `True` even though there was actually no error.
Reviewed By: chadaustin
Differential Revision: D22048525
fbshipit-source-id: 9a19cc092e2bd80db0e01fb38533a1007640bee6
Summary:
On Windows, paths are case insensitive (but the filesystem is case preserving),
and thus `open("FILE.TXT")` and `open("file.txt")` refer to the same file. When
that file is not materialized and its parent directory isn't yet enumerated,
PrjFS will call the PRJ_GET_PLACEHOLDER_INFO_CB with the file name passed in to
the `open` call. In this callback, if the passed in name refers to a valid
file, it needs to call PrjWritePlaceholderInfo to populate the directory entry.
Here is what the documentation for that function states:
"For example, if the PRJ_GET_PLACEHOLDER_INFO_CB callback specifies
dir1\dir1\FILE.TXT in callbackData->FilePathName, and the provider's backing
store contains a file called File.txt in the dir1\dir2 directory, and
PrjFileNameCompare returns 0 when comparing the names FILE.TXT and
File.txt, then the provider specifies dir1\dir2\File.txt as the value of
this parameter."
While the documentation doesn't state how that name is used internally, we can
infer (and test) that the returned case will be used as the canonical
representation of that file, ie: the one that a directory listing will see.
Since the PathMap code already does a case insensitive search, we just need to
make sure to use what it returns instead of re-using the name used for the search.
The only caveat to all of this is the original comment that describe that
`metadata.name` can't be used as it causes crashes. From what I can tell, this
was written in later 2018, and I believe is no longer relevant: the
`metadata.name` field was simply not populated.
Reviewed By: wez
Differential Revision: D21799627
fbshipit-source-id: aee877cc2d5f057944fcd39b1d59f0e97de6315c
Summary:
updated `eden top` to:
- obtain PID-fetchCounts data from the updated -`getAccessCounts` thrift call in the previous diff
- display that data in a new column `FUSE FETCH`
Reviewed By: kmancini
Differential Revision: D22101430
fbshipit-source-id: 6584e71ce3a4629c73469607ca0a4c6ffd63e46f
Summary: The word "disabled" better express the intent behind it.
Reviewed By: fanzeyi
Differential Revision: D21945736
fbshipit-source-id: a61a3df4f1b704224e7f4d4a7f1a43f4d2b5b2aa
Summary:
Removing a directory with unlink raises different errors on Windows: EACCES
instead of EISDIR.
Reviewed By: fanzeyi
Differential Revision: D21944455
fbshipit-source-id: 251ceb305a46c01ece78f45b0fa8000778c6c650
