Commit Graph

309 Commits

Author SHA1 Message Date
Xavier Deguillard
0fb5fa7846 thrift: remove getManifestEntry
Summary:
This is unused, and the Thrift file mentions that it should be fine to remove
in July 2020. It's now March 2021, time to kill it.

Reviewed By: chadaustin

Differential Revision: D26852134

fbshipit-source-id: 2872185edd834f889b78802210071d16b881e14c
2021-03-19 09:25:05 -07:00
Zeyi (Rice) Fan
f3f3fc546f integration: improve error reporting
Summary:
Previously the code would result an exception raised while handling another
exception which is a little confusing. This diff fixes that.

Reviewed By: chadaustin

Differential Revision: D27100659

fbshipit-source-id: 8c6be4df62214c8e8d778478de66f271f7b84d3c
2021-03-17 11:06:21 -07:00
Chad Austin
99c59c7e2b add some status edge case integration tests
Summary:
As Rice is replacing the overlay, introduce some integration tests
around the subtler interactions of mode bits and status.

Reviewed By: genevievehelsel

Differential Revision: D26707622

fbshipit-source-id: 429ba404c1c0655f8fff393664bc5ba9e3b0301f
2021-03-01 19:49:59 -08:00
generatedunixname89002005307016
428d0f602e Add annotations to eden/integration/lib/fake_edenfs.py
Reviewed By: xavierd

Differential Revision: D26710952

fbshipit-source-id: 207169a4f283ded2d4a33f46d585aa83af50e863
2021-03-01 10:58:17 -08:00
Xavier Deguillard
be8a46ce6e Add annotations to eden/integration/lib/edenclient.py
Reviewed By: fanzeyi

Differential Revision: D26692495

fbshipit-source-id: dac2a484bbf1f6e1c8e3e258bdb27849720c9882
2021-02-26 11:01:05 -08:00
generatedunixname89002005307016
a301311ce8 Add annotations to eden/integration/lib/pexpect.py
Reviewed By: xavierd

Differential Revision: D26486496

fbshipit-source-id: 16c4b0cd1cc7a657b8575adc7d45e2c1f96ed3a1
2021-02-17 10:30:15 -08:00
Xavier Deguillard
cc1fd73793 test: add a way to add edenfsrc config in tests
Summary:
Previously, for configs that are only read once, EdenFS would have to be
stopped, the config written, and then EdenFS would be restarted. For Mercurial,
this increases the test time significantly as starting EdenFS takes ~20s.

Reviewed By: fanzeyi

Differential Revision: D26258174

fbshipit-source-id: a74d1e5be35044e95e5a7403f1bf28d557b613d2
2021-02-04 20:10:38 -08:00
Xavier Deguillard
17c99d69de cli: add a --nfs argument to eden clone
Summary: This enables the repository to be mounted via NFS, and not FUSE.

Reviewed By: chadaustin

Differential Revision: D26229827

fbshipit-source-id: 5af5a47ebe5f1dd54df7707bf57d9b7476921f29
2021-02-04 20:10:37 -08:00
Genevieve Helsel
55da8ffcbc run linter in eden/integration
Summary: just running the linter!

Reviewed By: chadaustin

Differential Revision: D26000269

fbshipit-source-id: 184eb962fa7b88eb9b8b6bd22ae76477cbb6a06c
2021-01-25 16:13:54 -08:00
Xavier Deguillard
4cdbffe823 integration: allow OSS integration test to build on Windows
Summary:
On Windows, since Mercurial doesn't yet build with Buck, we need to test
against the system Mercurial, thus remove the dependencies to //eden/scm:hg for
the tests. Also remove various dependencies that don't build yet on Windows.

This allows for the tests to run, but fail while trying to execute edenfsctl.par.

Reviewed By: kmancini

Differential Revision: D25807727

fbshipit-source-id: c2533eedc361cc6db9fdf2190476c3d52833139d
2021-01-11 12:06:14 -08:00
Jun Wu
ec0b533381 ui: make ui.debug write to stderr
Summary:
Debug output belongs to stderr.

This makes it possible to turn on debug output without breaking programs
parsing stdout.

Reviewed By: singhsrb

Differential Revision: D25315954

fbshipit-source-id: c7813a824fbf6640cb5b80b5ed2d947e7059d53e
2020-12-15 12:07:47 -08:00
John Reese
d13e0b137f apply pyfmt with usort to opted-in sources
Reviewed By: zertosh

Differential Revision: D24880203

fbshipit-source-id: 2034cdfc2712209e86d3d05c119c58f979b05c52
2020-11-10 21:25:54 -08:00
Chad Austin
20c77da782 implement debugInodeStatus with traverseObservedInodes
Summary:
Replace the old implementation of debugInodeStatus with the more
general traverseObservedInodes functionality, and add the ability to
customize its results with flags.

Reviewed By: xavierd

Differential Revision: D24300122

fbshipit-source-id: 0fbd3aa02575faa515fd7852441547d7de13426d
2020-11-02 13:52:37 -08:00
John Reese
737849c9dd apply black 20.8b1 formatting update
Summary:
allow-large-files

black_any_style

Reviewed By: zertosh

Differential Revision: D24325133

fbshipit-source-id: b4afe80d1e8b2bc993f4b8e3822c02964df47462
2020-10-14 20:21:52 -07:00
Xavier Deguillard
001c399d6f tests: enable remaining update_tests on Windows
Summary: Only 2 tests aren't passing on Windows.

Reviewed By: genevievehelsel

Differential Revision: D24257788

fbshipit-source-id: 42a0169955bbff6e05e02374c3b3e434bf849ba0
2020-10-12 17:09:41 -07:00
Chad Austin
f11c8893c0 always run hg in integration tests with --traceback
Summary:
I observed an integration test fail because Mercurial aborted with an
error about not handling EINTR, but it had no traceback. Always run
Mercurial with --traceback to try to debug these.

Reviewed By: xavierd

Differential Revision: D24204308

fbshipit-source-id: 44960bc645e5f97f61761e511b372328430fcda7
2020-10-09 17:18:12 -07:00
Xavier Deguillard
e2a4bcf917 tests: enable all doctor tests on Windows
Summary:
Now that `hg whereami` properly reads the SNAPSHOT file on Windows, the doctor
tests properly detect that Mercurial and EdenFS disagree about the current
commit, thus we can enable the remaining 2 tests.

Reviewed By: genevievehelsel

Differential Revision: D23819924

fbshipit-source-id: 21be19aff913e5e485d72e8cd730e6851ecaba2e
2020-09-21 13:12:11 -07:00
Wez Furlong
b39f678b85 edenfs: remove use of fork from StartupLogger
Summary:
on macOS we cannot safely use `fork`.

This commit replaces the use of `fork` in the startup logger subsystem.
This was a little tricky to untangle; originally (prior to any of
the `fork` removal efforts in this diff stack), the startup flow was
to spawn a set of processes via fork:

```
edenfs (setuid)
 \-----edenfs (privhelper, as root)
  \------edenfs (daemonized)
```

The forked children take advantage of being able to implicitly pass state to
the child processes from the parent.  That data flow needs to become explicit
when removing the fork which makes some things a little awkward.

With fork removed:

* `edenfs` unconditionally spawns `edenfs_privhelper` while it has
  root privs and before most of the process has been initialized.
* That same `edenfs` process will then spawn a child `edenfs`
  process which starts from scratch, but that which needs to
  run as the real server instance
* The original `edenfs` instance needs to linger for a while
  to remain connected to the controlling tty to pass back the
  startup state to the user, before terminating.

This commit deletes the check that `edenfs` is started originally
as root; previously the logic relied on the forked startup logger
continuing past the `daemonizeIfRequested` call and simply deferring
the check until after folly::init.  With these changes we can't
easily perform such a check without adding some extra gymnastics
to pass the state around; the place where that is checked is in
the spawned child of the original edenfs, which is not a privileged
process and doesn't know the original euid.  I don't believe this
to be a great loss as we tuck `edenfs` away under the libexec dir.

Reviewed By: chadaustin

Differential Revision: D23696569

fbshipit-source-id: 55b95daf022601a4699274d696af419f0a11f6f2
2020-09-18 17:22:39 -07:00
Zeyi (Rice) Fan
5fedaa71b8 enable redirection tests
Summary: This diff fixes the eden redirection tests so it runs on Windows.

Reviewed By: xavierd

Differential Revision: D22958766

fbshipit-source-id: 45d26587831ed74d6bd7912b22c7c955b077f571
2020-09-18 16:39:52 -07:00
Chad Austin
55f9f07c7d increase timeout in graceful restart tests
Summary:
tpx doesn't support heavyweight tags or rate limiting, and integration
tests regularly fail with timeouts on my devbig, so bump the process
start and process stop timeouts.

Reviewed By: genevievehelsel

Differential Revision: D23553924

fbshipit-source-id: fa9b8710395d61b087963d18718137e4525ae03d
2020-09-15 17:37:50 -07:00
Chad Austin
a303d6a399 increase shutdown timeout in integration tests and avoid EPERM when started with sudo
Summary:
30 seconds is not enough time on heavily contented systems, including
CI. Bump the shutdown timeout to 120.

Also, correctly send SIGKILL to the daemon process when it's been
started with sudo.

Reviewed By: simpkins

Differential Revision: D22422784

fbshipit-source-id: dc7be0962705f1feb9643990309f570e352b68a0
2020-09-15 17:37:50 -07:00
Chad Austin
5f8fde4570 remove an irredeemably flaky test
Summary:
This test can't be non-flaky, because it relies on the kernel deciding
when to drop inodes from cache, and we've investigated it multiple
times. Given it tests a rarely-used function that would be better
expressed as a unit test in C++, just remove it for now.

Reviewed By: wez

Differential Revision: D23665455

fbshipit-source-id: 522e47113857eff399be4f2bb60e26e801d61e9a
2020-09-14 14:57:44 -07:00
Chad Austin
850999df92 handle mounts with device type starting with edenfs:
Summary:
We intend to rename the edenfs device type to include a colon (and
possibly the backing repo basename). In preparation, update code that
detects edenfs mounts to include anything that starts with "edenfs:".

Reviewed By: genevievehelsel

Differential Revision: D23520008

fbshipit-source-id: 280f7617d5c96e23d548041b3482bca388076a7b
2020-09-11 13:35:03 -07:00
Mark Mendoza
53e1072e7d Deleting failing eden systemd_fixture_test
Summary:
We are trying to get eden running in our atypical EDA corp environment.
When testing it out on one of these machines, we got things sorted out to the point where the only test failures were coming from this file.
chadaustin identified this as being a test of dead code, and so we decided to go for a deletion of it.
If this work resumes, these tests can be retrieved from version control and then be made to work on Centos7 (hopefully at that point we'll also have contbuild/utd magic set-up to have that re-enabling automatically trigger the build/test).

Reviewed By: genevievehelsel

Differential Revision: D23463831

fbshipit-source-id: 7714547c04573b94dbb2d9acf7906734d853c5aa
2020-09-01 22:39:10 -07:00
Xavier Deguillard
4f9e1750c2 cli: enable doctor on Windows
Summary:
Most of the fixes are pretty trivial as the code was using functions not
present on Windows, either work around them, or switch to ones that are
multi-platform.

Of note, it looks like `hg doctor` doesn't properly detect when Mercurial and
EdenFS are out of sync, disabling the  tests until we figure out why.

Reviewed By: genevievehelsel, fanzeyi

Differential Revision: D23409708

fbshipit-source-id: 3314c197d43364dda13891a6874caab4c29e76ca
2020-08-28 19:49:37 -07:00
Xavier Deguillard
76ea3054de win: conditionally enable negative path caching
Summary:
From a quick experiment, this greatly cuts down on the amount requests to
nonexistent files. For instance, the .hg directory in folders is now only
looked up once and no longer afterwards.

Reviewed By: wez

Differential Revision: D23112343

fbshipit-source-id: 223134ca591054ae9ac2e839033bbd1b714443da
2020-08-14 17:35:50 -07:00
Xavier Deguillard
5ef7260cc4 cli: remove the repo on edenfsctl remove
Summary:
On Windows, unmounting the repo isn't sufficient to remove all traces of the
repo as ProjectedFS uses it as a cache, and thus all the files that were
touched by the user are still present on disk after unmounting it.

Reviewed By: wez

Differential Revision: D22915349

fbshipit-source-id: 445379fef57160b5da39a298ca4518e2662d16e3
2020-08-13 08:28:45 -07:00
Xavier Deguillard
740d414184 integration: set the encoding to utf-8 on Popen
Summary:
On Windows, this has the benefit of automatically converting \r\n into \n,
which allows more tests to pass.

Reviewed By: chadaustin

Differential Revision: D22871408

fbshipit-source-id: 02ec1d21dc236175c3b0f3176db9b8c91dee21a4
2020-08-04 12:05:43 -07:00
Xavier Deguillard
165199b58c integration: enable files_test on Windows
Summary:
On Windows, paths are separated by \, but the test was comparing them against
/. We can simply ask Mercurial to return / with the slashpath template filter.

Reviewed By: chadaustin

Differential Revision: D22871407

fbshipit-source-id: 421bd14f752f29265b12eb25609d4f65e593dda8
2020-08-03 11:26:31 -07:00
Xavier Deguillard
ef4db32904 inodes: invalidate more on Windows
Summary:
Cache invalidation is hard, and on Windows we avoided doing a lot of them. It
turns out, this was the wrong decision as it's fairly easy to find cases where
the filesystem view is different from the manifest state.

Since the Linux code is most likely correct in where the invalidation is done,
let's also do the same on Windows, removing a whole lot of #ifdef. It is very
likely that as a result of this diff we end up invalidating more than needed,
thus slowing down EdenFS, but at this point I'd prefer to err on the side of
correctness, performance will come later.

While invalidating files should use PrjDeleteFile, for directories, we simply
need to mark them as placeholder, as directories created by a user won't have a
placeholder, thus ProjectedFS would bypass EdenFS when listing in.

Reviewed By: chadaustin

Differential Revision: D22833202

fbshipit-source-id: d807557f5e44279c49ab701b7a797253ef1f0717
2020-08-03 11:26:31 -07:00
Xavier Deguillard
2065e76451 integration: enable undo_test on Windows
Summary: The test is passing, enable it.

Reviewed By: genevievehelsel

Differential Revision: D22798421

fbshipit-source-id: aec5302aad38d3413385bf5f0242800d685fb5ef
2020-07-28 16:44:10 -07:00
Xavier Deguillard
aeb1fb2079 integration: enable storage_engine_test on Windows
Summary: These 3 are passing on Windows, let's enable them.

Reviewed By: genevievehelsel

Differential Revision: D22798419

fbshipit-source-id: af399325d482a249530d81b1d04da7ce4af1277d
2020-07-28 16:44:10 -07:00
Xavier Deguillard
29e8b1ee18 integration: enable sparse_test on Windows
Summary: The test is passing, enable it.

Reviewed By: genevievehelsel

Differential Revision: D22798424

fbshipit-source-id: 76b99457aacf5a81c2b9b3ebaedd0e6e1cf2a1e8
2020-07-28 16:44:10 -07:00
Xavier Deguillard
0bb37fbdce integration: enable rollback_test on Windows
Summary: The test is passing, enable it.

Reviewed By: genevievehelsel

Differential Revision: D22798422

fbshipit-source-id: 0f5a106be51e319a0d317900cc88de1131b95e4d
2020-07-28 16:44:10 -07:00
Xavier Deguillard
0d670a2bdb integration: enable rm_test on Windows
Summary: Out of the 5 tests, 3 are passing, let's enable them.

Reviewed By: genevievehelsel

Differential Revision: D22798418

fbshipit-source-id: 5bd3fd90945e5556de7838a5dc61ad00865a6d55
2020-07-28 16:44:09 -07:00
Xavier Deguillard
4f57c681e3 integration: enable revert_test on Windows
Summary: They are passing, enable them.

Reviewed By: genevievehelsel

Differential Revision: D22798420

fbshipit-source-id: 27c8e18195a514f4c564481334e8ae65eba65215
2020-07-28 16:44:09 -07:00
Xavier Deguillard
b8ca9cde60 integration: enable rebase_test on Windows
Summary: Half of them are passing, let's make sure they run.

Reviewed By: genevievehelsel

Differential Revision: D22798423

fbshipit-source-id: b762213ffad431de6f54acb8ceffa54f28f5909d
2020-07-28 16:44:09 -07:00
Xavier Deguillard
541024b73f integration: enable non_eden_operation_test in Windows
Summary: The test is passing, enable it.

Reviewed By: genevievehelsel

Differential Revision: D22798417

fbshipit-source-id: c45531a75a3db61e33f95ef80eb007f12dd02f2a
2020-07-28 16:44:09 -07:00
Xavier Deguillard
8312b66d82 integration: enable merge_test on Windows
Summary: The 5 tests are passing, enable them.

Reviewed By: genevievehelsel

Differential Revision: D22794678

fbshipit-source-id: 5547be906b9ad34539b61dfada2a1f25b7385468
2020-07-28 16:19:33 -07:00
Xavier Deguillard
3c1bf6f757 integration: enable journal_test on Windows
Summary: The test is passing, enable it.

Reviewed By: genevievehelsel

Differential Revision: D22794679

fbshipit-source-id: 8d5f5a9b9fd7750034baeda814bfeff29882d409
2020-07-28 16:19:33 -07:00
Xavier Deguillard
f19f2b5389 integration: enable histedit_test for Windows
Summary: The 5 tests are passing, enable them.

Reviewed By: genevievehelsel

Differential Revision: D22794677

fbshipit-source-id: 186190a03bcd1646b0219cb574cc1969138a45e7
2020-07-28 16:19:33 -07:00
Xavier Deguillard
22c6079f9d integration: enable grep_test on Windows
Summary: 3 out of 5 tests are passing, let's enable them.

Reviewed By: genevievehelsel

Differential Revision: D22794675

fbshipit-source-id: 2834a512ccb89618247cbf5d420b13d1f7e9c70c
2020-07-28 16:19:33 -07:00
Xavier Deguillard
9cea00657d integration: enable graft_test on Windows
Summary: These are all passing, enable them.

Reviewed By: genevievehelsel

Differential Revision: D22794158

fbshipit-source-id: c381a906e096c3642248a2521e8e7772d74f992e
2020-07-28 16:19:32 -07:00
Xavier Deguillard
0df75ca9f4 integration: enable commit_test on Windows
Summary: All of the tests are passing, enable them.

Reviewed By: genevievehelsel

Differential Revision: D22794159

fbshipit-source-id: 8b7e33f3abbde88e06488f7fe9ea7217d204e70e
2020-07-28 16:19:32 -07:00
Xavier Deguillard
a36e862f3c integration: enable add_test on Windows
Summary: Out of 7 tests, 4 are passing, enable them.

Reviewed By: genevievehelsel

Differential Revision: D22794163

fbshipit-source-id: ad24ce0fcb8cabcfb9039228042cdab448e8273b
2020-07-28 16:19:32 -07:00
Xavier Deguillard
c46418e1fa integration: enable absorb_test for Windows
Summary: The test is passing, enable it.

Reviewed By: chadaustin

Differential Revision: D22794161

fbshipit-source-id: 94d167b88782d386ca60b9215f1c2aef56a88a4d
2020-07-28 16:19:32 -07:00
Xavier Deguillard
d28fd83ff1 integration: enable most thrift_test on Windows
Summary: Out of 14 tests, 9 are passing, enable them.

Reviewed By: chadaustin

Differential Revision: D22794157

fbshipit-source-id: 7856ac0449d73155903bded2d3adf84e45d38c11
2020-07-28 16:19:32 -07:00
Xavier Deguillard
f4fa401cdc integration: enable most stats_test on Windows
Summary: The only ones not passing are the FUSE ones, which is expected. Enable all the others.

Reviewed By: genevievehelsel

Differential Revision: D22794162

fbshipit-source-id: 31f51fd5fff8e3ced75924533fa536208dabf11d
2020-07-28 16:19:32 -07:00
Xavier Deguillard
844a13f27d integration: enablle rocksdb_store_test on Windows
Summary: The test is passing, enable it.

Reviewed By: chadaustin

Differential Revision: D22794155

fbshipit-source-id: 286c9f1129d279487019206d58521951c768cbb1
2020-07-28 16:19:32 -07:00
Xavier Deguillard
9b6e7f19c6 integration: enable persistence_test on Windows
Summary: 2 out of 3 tests are passing, let's run these.

Reviewed By: chadaustin

Differential Revision: D22794160

fbshipit-source-id: e62e1956980689b083107ebb3b8340880e0d72a6
2020-07-28 16:19:32 -07:00