Commit Graph

633 Commits

Author SHA1 Message Date
Xavier Deguillard
aeb1fb2079 integration: enable storage_engine_test on Windows
Summary: These 3 are passing on Windows, let's enable them.

Reviewed By: genevievehelsel

Differential Revision: D22798419

fbshipit-source-id: af399325d482a249530d81b1d04da7ce4af1277d
2020-07-28 16:44:10 -07:00
Xavier Deguillard
29e8b1ee18 integration: enable sparse_test on Windows
Summary: The test is passing, enable it.

Reviewed By: genevievehelsel

Differential Revision: D22798424

fbshipit-source-id: 76b99457aacf5a81c2b9b3ebaedd0e6e1cf2a1e8
2020-07-28 16:44:10 -07:00
Xavier Deguillard
0bb37fbdce integration: enable rollback_test on Windows
Summary: The test is passing, enable it.

Reviewed By: genevievehelsel

Differential Revision: D22798422

fbshipit-source-id: 0f5a106be51e319a0d317900cc88de1131b95e4d
2020-07-28 16:44:10 -07:00
Xavier Deguillard
0d670a2bdb integration: enable rm_test on Windows
Summary: Out of the 5 tests, 3 are passing, let's enable them.

Reviewed By: genevievehelsel

Differential Revision: D22798418

fbshipit-source-id: 5bd3fd90945e5556de7838a5dc61ad00865a6d55
2020-07-28 16:44:09 -07:00
Xavier Deguillard
4f57c681e3 integration: enable revert_test on Windows
Summary: They are passing, enable them.

Reviewed By: genevievehelsel

Differential Revision: D22798420

fbshipit-source-id: 27c8e18195a514f4c564481334e8ae65eba65215
2020-07-28 16:44:09 -07:00
Xavier Deguillard
b8ca9cde60 integration: enable rebase_test on Windows
Summary: Half of them are passing, let's make sure they run.

Reviewed By: genevievehelsel

Differential Revision: D22798423

fbshipit-source-id: b762213ffad431de6f54acb8ceffa54f28f5909d
2020-07-28 16:44:09 -07:00
Xavier Deguillard
541024b73f integration: enable non_eden_operation_test in Windows
Summary: The test is passing, enable it.

Reviewed By: genevievehelsel

Differential Revision: D22798417

fbshipit-source-id: c45531a75a3db61e33f95ef80eb007f12dd02f2a
2020-07-28 16:44:09 -07:00
Xavier Deguillard
8312b66d82 integration: enable merge_test on Windows
Summary: The 5 tests are passing, enable them.

Reviewed By: genevievehelsel

Differential Revision: D22794678

fbshipit-source-id: 5547be906b9ad34539b61dfada2a1f25b7385468
2020-07-28 16:19:33 -07:00
Xavier Deguillard
3c1bf6f757 integration: enable journal_test on Windows
Summary: The test is passing, enable it.

Reviewed By: genevievehelsel

Differential Revision: D22794679

fbshipit-source-id: 8d5f5a9b9fd7750034baeda814bfeff29882d409
2020-07-28 16:19:33 -07:00
Xavier Deguillard
f19f2b5389 integration: enable histedit_test for Windows
Summary: The 5 tests are passing, enable them.

Reviewed By: genevievehelsel

Differential Revision: D22794677

fbshipit-source-id: 186190a03bcd1646b0219cb574cc1969138a45e7
2020-07-28 16:19:33 -07:00
Xavier Deguillard
22c6079f9d integration: enable grep_test on Windows
Summary: 3 out of 5 tests are passing, let's enable them.

Reviewed By: genevievehelsel

Differential Revision: D22794675

fbshipit-source-id: 2834a512ccb89618247cbf5d420b13d1f7e9c70c
2020-07-28 16:19:33 -07:00
Xavier Deguillard
9cea00657d integration: enable graft_test on Windows
Summary: These are all passing, enable them.

Reviewed By: genevievehelsel

Differential Revision: D22794158

fbshipit-source-id: c381a906e096c3642248a2521e8e7772d74f992e
2020-07-28 16:19:32 -07:00
Xavier Deguillard
0df75ca9f4 integration: enable commit_test on Windows
Summary: All of the tests are passing, enable them.

Reviewed By: genevievehelsel

Differential Revision: D22794159

fbshipit-source-id: 8b7e33f3abbde88e06488f7fe9ea7217d204e70e
2020-07-28 16:19:32 -07:00
Xavier Deguillard
a36e862f3c integration: enable add_test on Windows
Summary: Out of 7 tests, 4 are passing, enable them.

Reviewed By: genevievehelsel

Differential Revision: D22794163

fbshipit-source-id: ad24ce0fcb8cabcfb9039228042cdab448e8273b
2020-07-28 16:19:32 -07:00
Xavier Deguillard
c46418e1fa integration: enable absorb_test for Windows
Summary: The test is passing, enable it.

Reviewed By: chadaustin

Differential Revision: D22794161

fbshipit-source-id: 94d167b88782d386ca60b9215f1c2aef56a88a4d
2020-07-28 16:19:32 -07:00
Xavier Deguillard
d28fd83ff1 integration: enable most thrift_test on Windows
Summary: Out of 14 tests, 9 are passing, enable them.

Reviewed By: chadaustin

Differential Revision: D22794157

fbshipit-source-id: 7856ac0449d73155903bded2d3adf84e45d38c11
2020-07-28 16:19:32 -07:00
Xavier Deguillard
f4fa401cdc integration: enable most stats_test on Windows
Summary: The only ones not passing are the FUSE ones, which is expected. Enable all the others.

Reviewed By: genevievehelsel

Differential Revision: D22794162

fbshipit-source-id: 31f51fd5fff8e3ced75924533fa536208dabf11d
2020-07-28 16:19:32 -07:00
Xavier Deguillard
844a13f27d integration: enablle rocksdb_store_test on Windows
Summary: The test is passing, enable it.

Reviewed By: chadaustin

Differential Revision: D22794155

fbshipit-source-id: 286c9f1129d279487019206d58521951c768cbb1
2020-07-28 16:19:32 -07:00
Xavier Deguillard
9b6e7f19c6 integration: enable persistence_test on Windows
Summary: 2 out of 3 tests are passing, let's run these.

Reviewed By: chadaustin

Differential Revision: D22794160

fbshipit-source-id: e62e1956980689b083107ebb3b8340880e0d72a6
2020-07-28 16:19:32 -07:00
Xavier Deguillard
3e8ebf86ab integration: enable glob_test on Windows
Summary: The 11 tests are passing, let's run them.

Reviewed By: genevievehelsel

Differential Revision: D22794156

fbshipit-source-id: 6f35a538276ed147a312e01d9031af62809d3b3b
2020-07-28 16:19:32 -07:00
Xavier Deguillard
189cca1227 integration: enable passing update_tests
Summary:
Besides the 3 listed, all the other 13 are passing, let's make sure we run them
to not regress.

Reviewed By: chadaustin

Differential Revision: D22794164

fbshipit-source-id: 5660cc36365de96a2b9e62e3462c01b39904d2f3
2020-07-28 16:19:32 -07:00
Genevieve Helsel
247965ae33 fix flaky time test by rounding to nearest second
Summary:
we've seen flaky tests here on stress test runs, lets subtract time.time() by an epsilon of one to ensure that the comparison is only made within a second precision (time.time() worst case precision)

an example error: `AssertionError: 1595414015.5022793 not greater than or equal to 1595414015.5025299`
`AssertionError: 1595759609.439305 not greater than or equal to 1595759609.441682`

Reviewed By: chadaustin

Differential Revision: D22687496

fbshipit-source-id: 8e87148d620577e3198d2845d785a87a909cd1d3
2020-07-27 13:04:13 -07:00
Chad Austin
02c180b1db fix systemd tests on fedora 32
Summary:
Disregard two default systemd services on Fedora 32 that cause our
systemd tests to fail.

I believe genevievehelsel is planning on removing this code soon anyway.

Reviewed By: genevievehelsel

Differential Revision: D22713393

fbshipit-source-id: b703b23a3158cb007dc2e1cb53fae36be7282719
2020-07-24 10:21:35 -07:00
Victor Zverovich
e3f4a56f6b Migrate to field_ref Thrift API
Summary:
We are unifying C++ APIs for accessing optional and unqualified fields:
https://fb.workplace.com/groups/1730279463893632/permalink/2541675446087359/.

This diff migrates code from accessing data members generated from unqualified
Thrift fields directly to the `field_ref` API, i.e. replacing

```
thrift_obj.field
```

with

```
*thrift_obj.field_ref()
```

The `_ref` suffixes will be removed in the future once data members are private
and names can be reclaimed.

The output of this codemod has been reviewed in D20039637.

The new API is documented in
https://our.intern.facebook.com/intern/wiki/Thrift/FieldAccess/.

drop-conflicts

Reviewed By: yfeldblum

Differential Revision: D22631599

fbshipit-source-id: 9bfcaeb636f34a32fd871c7cd6a2db4a7ace30bf
2020-07-21 11:23:35 -07:00
generatedunixname89002005307016
6527db2809 suppress errors in eden - batch 1
Reviewed By: pradeep90

Differential Revision: D22628389

fbshipit-source-id: c109334243a92269a7fb9a9b0515565ac22ecfac
2020-07-20 13:29:51 -07:00
Jun Wu
b80966f93c revlog: turn on head-based-commit-transaction for tests
Summary:
Bypass truncation-based transaction if narrow-heads is on.

The transaction abort still works logically because commit references stay
unchanged on abort.

Related EdenFS and Mononoke tests are updated. Mononoke tests probably
shouldn't rely on revlog / fncache implementation details in hg.

Reviewed By: DurhamG

Differential Revision: D22240186

fbshipit-source-id: f97efd60855467b52c9fb83e7c794ded269e9617
2020-07-08 14:33:58 -07:00
Genevieve Helsel
da7b453306 fix AssertionError if a checkout is in the middle of a merge
Summary:
An assertion error is raised if `eden doctor` is in the middle of a merge. This is because we enter a specific "if" condition in the case that mercurial has two parent commits, and EdenFS only ever tracks `p0`, so EdenFS simply sets `p1` to the null commit in `_select_new_parents()`. Specifically, this is in the case in which both `_old_dirstate_parents` and `_old_snapshot` are not None.

Because `_old_dirstate_parents` has `p1` set to nonnull, and Eden thinks it is null , the check `self._new_parents != self._old_dirstate_parents` would be `True` even though there was actually no error.

Reviewed By: chadaustin

Differential Revision: D22048525

fbshipit-source-id: 9a19cc092e2bd80db0e01fb38533a1007640bee6
2020-06-30 13:58:21 -07:00
Xavier Deguillard
a935fc38b4 inodes: fix casing issue on Windows
Summary:
On Windows, paths are case insensitive (but the filesystem is case preserving),
and thus `open("FILE.TXT")` and `open("file.txt")` refer to the same file. When
that file is not materialized and its parent directory isn't yet enumerated,
PrjFS will call the PRJ_GET_PLACEHOLDER_INFO_CB with the file name passed in to
the `open` call. In this callback, if the passed in name refers to a valid
file, it needs to call PrjWritePlaceholderInfo to populate the directory entry.
Here is what the documentation for that function states:

"For example, if the PRJ_GET_PLACEHOLDER_INFO_CB callback specifies
dir1\dir1\FILE.TXT in callbackData->FilePathName, and the provider's backing
store contains a file called File.txt in the dir1\dir2 directory, and
PrjFileNameCompare returns 0 when comparing the names FILE.TXT and
File.txt, then the provider specifies dir1\dir2\File.txt as the value of
this parameter."

While the documentation doesn't state how that name is used internally, we can
infer (and test) that the returned case will be used as the canonical
representation of that file, ie: the one that a directory listing will see.

Since the PathMap code already does a case insensitive search, we just need to
make sure to use what it returns instead of re-using the name used for the search.

The only caveat to all of this is the original comment that describe that
`metadata.name` can't be used as it causes crashes. From what I can tell, this
was written in later 2018, and I believe is no longer relevant: the
`metadata.name` field was simply not populated.

Reviewed By: wez

Differential Revision: D21799627

fbshipit-source-id: aee877cc2d5f057944fcd39b1d59f0e97de6315c
2020-06-24 18:59:16 -07:00
Ailin Zhang
dde49c1c44 add a new column FUSE FETCH to eden top to display fetchCounts
Summary:
updated `eden top` to:
- obtain PID-fetchCounts data from the updated -`getAccessCounts` thrift call in the previous diff
- display that data in a new column `FUSE FETCH`

Reviewed By: kmancini

Differential Revision: D22101430

fbshipit-source-id: 6584e71ce3a4629c73469607ca0a4c6ffd63e46f
2020-06-24 15:56:14 -07:00
Xavier Deguillard
2e184e2a1a integration: rename blacklist.py
Summary: The word "disabled" better express the intent behind it.

Reviewed By: fanzeyi

Differential Revision: D21945736

fbshipit-source-id: a61a3df4f1b704224e7f4d4a7f1a43f4d2b5b2aa
2020-06-19 18:13:20 -07:00
Xavier Deguillard
8883532338 tests: enable unlink_test on Windows
Summary:
Removing a directory with unlink raises different errors on Windows: EACCES
instead of EISDIR.

Reviewed By: fanzeyi

Differential Revision: D21944455

fbshipit-source-id: 251ceb305a46c01ece78f45b0fa8000778c6c650
2020-06-19 18:13:20 -07:00
Xavier Deguillard
5d96bcae3c tests: enable some integration tests
Summary: These appear to be passing, let's enable them.

Reviewed By: genevievehelsel

Differential Revision: D21820196

fbshipit-source-id: bb36e48d02411b1dc94e3d1e499f892145cee6ae
2020-06-19 18:13:19 -07:00
Chad Austin
c13386ead4 fix test
Summary:
test_no_units_are_active was failing on CentOS 8. Presumably systemd
has some new default services. I believe genevievehelsel is planning on
replacing all of this code, so, rather than understand the failure,
just ignore the additional service names.

Reviewed By: genevievehelsel

Differential Revision: D22055992

fbshipit-source-id: b6f313350d0e1b107fe6ea3d7ed7f5b4eb025ef3
2020-06-15 18:37:01 -07:00
Xavier Deguillard
42fb95adc2 integration: write files to the repo in binary mode
Summary:
In text mode, Python will use '\r\n' as line endings instead of '\n' on
Windows. This has the negative side effect of making some tests fails due to
the file content not being exactly what is being written.

Reviewed By: fanzeyi

Differential Revision: D21944456

fbshipit-source-id: f7f271e2ef9e2f21bd78c76233e644ce411b63df
2020-06-15 15:10:20 -07:00
Genevieve Helsel
ca1da512bf store force restart action as its own flag
Summary: In its current state, FORCE is a restart mode that cannot be combined with other modes (specifically `graceful`). Removing this coupling from `force` will allow us to at first attempt a graceful restart, but if the daemon is not healthy, then do a force restart, all within one call.

Reviewed By: wez

Differential Revision: D21873166

fbshipit-source-id: 079a777ff6e7b219198cbef15335244b9865f5f3
2020-06-11 17:40:51 -07:00
Chad Austin
597c209fa1 disallow warnings in integration tests
Summary:
Unawaited coroutines are pernicious and not errors by default. In
advance of enabling asynchronous tests, fail our tests when Python
warnings are shown.

We record warnings and report them after the test body instead of
enabling an "error" warning filter because exceptions thrown by
finalizers are silently swallowed, and most important warnings occur
during finalization.

Reviewed By: genevievehelsel

Differential Revision: D21955210

fbshipit-source-id: b1fc52ddfa9f9befbde6800f85f090862af0ef15
2020-06-10 23:23:34 -07:00
Chad Austin
e1e4bdee40 add a (failing) test of binary blob contents over thrift
Summary:
Add a disabled test that illustrates that binary data fails to
serialize between edenfs and the CLI with the current Thrift
implementation.

Reviewed By: fanzeyi

Differential Revision: D21890379

fbshipit-source-id: 725e99751e1d62c39f4059ec0f1197857c47e3b4
2020-06-10 19:29:42 -07:00
Chad Austin
7e4835f677 rename eden.thrift Python module to eden.thrift.legacy
Summary:
The Python 2-and-3 Thrift API is sort of deprecated and does not
handle binary data in `binary` fields. In advance of migrating to the
modern Python 3 API, remane eden.thrift to eden.thrift.legacy.

Reviewed By: fanzeyi

Differential Revision: D21889697

fbshipit-source-id: a745ee8977999acbfb383a4edebe81d8deb1734e
2020-06-10 19:29:42 -07:00
Chad Austin
11d569b04d refactor EdenClient construction
Summary:
I am planning to start migrating Eden's CLI to the new Python 3 Thrift
implementation. In preparation, slightly clean up the interface and
implementation of our Python 2 Thrift wrapper.

Reviewed By: genevievehelsel

Differential Revision: D21854539

fbshipit-source-id: d398dd3f324c12288871cf0c9db41e64ed4cf7ed
2020-06-10 19:29:23 -07:00
Chad Austin
3b04c02b29 fix flaky integration tests
Summary:
Under heavy parallelism or system load, our tests could trigger
short-ish timeouts and cause tests to flake. The stats test in
particular often failed in continuous integration. It looks like
opening a unix domain Thrift socket early and holding onto it can
cause it to sometimes hit ThriftServer's default idle timeout of 60
seconds, which results in the test failing with BrokenPipeError
(EPIPE).

Reviewed By: simpkins

Differential Revision: D21780023

fbshipit-source-id: 7e8838429475c2a322d836b9a497411199948cce
2020-05-29 11:49:37 -07:00
Xavier Deguillard
0cb5ceedfe unify the EdenMain code on Windows
Summary:
Update EdenFS on Windows to use EdenMain.cpp, the same as on POSIX.

This reduces the amount of code divergence, and also brings several
improvements to Windows: exposing version information over fb303, support for
dynamically changing logging configuration over fb303, etc.

Reviewed By: simpkins

Differential Revision: D21332569

fbshipit-source-id: dd2da7c385e96f65fc3927511c9f84b96bec9e2b
2020-05-28 09:01:03 -07:00
generatedunixname89002005307016
f9358e566a suppress errors in eden - batch 1
Summary:
This diff is auto-generated to upgrade the Pyre version and suppress errors in eden. The upgrade will affect Pyre local configurations in the following directories:
```
eden
```

Differential Revision: D21687853

fbshipit-source-id: baf0d9bc33f86da63ea289690faca6cf4d566588
2020-05-21 19:32:35 -07:00
Xavier Deguillard
cd92c10363 integration: symlinks are not supported on Windows
Summary:
Now that regular users can create symlinks, a handful of tests were failing
due to the symlink call no longer raising an exception. For now, let's always
consider that symlinks aren't supported on Windows.

Reviewed By: fanzeyi

Differential Revision: D21664213

fbshipit-source-id: c55a99d1cb92e68b9861701b5517b1d5db2d40c6
2020-05-20 13:11:10 -07:00
Shannon Zhu
096b35c70b Convert type check targets in eden to use configuration
Summary:
Migrating buck integration to use configurations, which is more performant and is the primary supported way to run Pyre.
Mypy is no longer supported at Facebook, so mypy buck integration is covered in this migration as well.

Configuration target automatically expanded to include all subtargets, expanding type coverage while introducing no more than 2 fixmes per file.

Reviewed By: dkgi

Differential Revision: D21474723

fbshipit-source-id: 2501af8250b4998fe331350b80c9dc8b4ba2dd1a
2020-05-08 15:24:57 -07:00
Chad Austin
c4db2f39a9 remove some internal uses of the term 'client'
Summary:
Our use of the term "client" to refer to a checkout is
deprecated. Rename some internal functions that use the term client.

Reviewed By: simpkins

Differential Revision: D21395159

fbshipit-source-id: fa96ba593f53b493e5ae816fa686f333a132c232
2020-05-07 22:08:10 -07:00
Adam Simpkins
13a8f17325 de-blacklist most of the hg status related tests on Windows
Summary: All of the tests in hg/status_test.py pass except for these 3.

Reviewed By: genevievehelsel

Differential Revision: D21410570

fbshipit-source-id: cf9a87ce0f025860c1e95927a780757fda6801ad
2020-05-07 16:41:30 -07:00
Adam Simpkins
ccdbc2279b update most of the integration/basic_test.py tests to work on Windows
Summary:
- If we fail to create symlinks ignore this error on Windows
- Using a file as a directory produces an ENOENT error instead of ENOTDIR
  on Windows.
- Deal with some issues around CR -> CRLF translation when writing files in
  non-binary mode on Windows.
- Move several POSIX-specific tests out into a separate test class.

Reviewed By: wez

Differential Revision: D21214598

fbshipit-source-id: 587a755bd06990bf88d26017254a69ff8869edbc
2020-05-07 16:41:30 -07:00
Adam Simpkins
54af4a5770 update can_run_eden() to check for ProjectedFSLib.dll on Windows
Summary:
On Windows we can only run EdenFS if ProjectedFSLib.dll is available.  It
turns out that some Windows systems have ProjectedFSLib.lib available, so we
can build `edenfs.exe`, but are missing ProjectedFSLib.dll, so we cannot
actually run it.

Reviewed By: wez

Differential Revision: D21443255

fbshipit-source-id: 6396fce8bd71adc0695468a2df3fd5f584909138
2020-05-07 16:41:30 -07:00
Shannon Zhu
83dc7b9129 Custom pyre upgrade for eden
Summary:
Addressing issues simpkins brought up on D21207287 when we upgraded and introduced some pyre bugs.

Temporarily upgrading just this project, once we resolve some sandcastle capacity issues we'll release this via another global upgrade in fbcode.

Reviewed By: simpkins

Differential Revision: D21316793

fbshipit-source-id: f0c79f53d97f7182e7d8fe6e081c58ef53ce0c9a
2020-05-04 21:19:20 -07:00
Adam Simpkins
c0fb9cea2d enable all integration test sources in CMake builds
Summary:
Previously we only included `basic_test.py` and `hg/status_test.py` in the
integration tests during CMake-based builds.  This updates the code to now
include all of the test files, with just a few exclusions based on platform
type and what dependencies were available at build time.

Reviewed By: wez

Differential Revision: D21239912

fbshipit-source-id: b8826d249a6323ac3bcc555c9ceba54a4cbcfde9
2020-05-04 11:46:09 -07:00