Commit Graph

51374 Commits

Author SHA1 Message Date
Mark Thomas
23fe5a7780 reset: add visibility tracking support
Summary:
The `reset` extension does its own obsmarker creation, rather than going
through `scmutil.cleanupnodes`.  This means it doesn't support new-style
visibility tracking.

Fix this by making it use `scmutil.cleanupnodes`.  This isn't completely
straightforward:

* The revset it uses to work out what to prune might accidentally include
  public commits (e.g. when you reset to a different public branch).  Make sure
  these are filtered.
* The tests originally had strip and obsmarker based tests.  The strip tests,
  when converted to obsmarkers and using `scmutil.cleanupnodes` stop working
  because they strip and revive the same commit over and over, which is an edge
  case that obsmarkers can't handle well.  Fix this by restoring the strip tests
  as strip tests.  A separate test handles the new-style visibility.
* Reset's behaviour is still a bit wonky.  If an ancestor of the source commit
  has other (non-bookmarked) descendants, then reset will try to prune those
  commits.  New-style visibility will ignore this, and this is tested in the
  new test.

Reviewed By: farnz

Differential Revision: D18912817

fbshipit-source-id: cc115333407cf67d339c24fcd0807ddedce2660d
2019-12-11 03:49:38 -08:00
Adam Simpkins
d0ba23f47a decrease the log level of globFiles() thrift calls
Summary:
D18820280 increased the log level for `globFiles()` calls from DBG3 to DBG2.
However, buck and other tools often end up huge numbers of glob calls, and
this ends up causing log spam.

This change reduces the log level of `globFiles()` back to DBG3.  In a
subsequent diff I will update the code to increase the log level for slow
calls.  This does mean that we will only get a log message for slow glob calls
after the call completes, but this is better than not getting a message at
all.

Reviewed By: pkaush

Differential Revision: D18924780

fbshipit-source-id: 86e83614f51424aca078e4de8a56c45018a6d286
2019-12-10 20:18:36 -08:00
Genevieve Helsel
8ff8d6217e setParentCommits after checkout while holding parentsLock_
Summary: was looking in this area and saw this TODO. moves the `setParentCommits()` call into `CheckoutConflict::finish()` so it is called while we are still holding the parents lock.

Reviewed By: chadaustin

Differential Revision: D18668329

fbshipit-source-id: 8415c792193e5b89737f15141f33c5a9799f527b
2019-12-10 18:13:26 -08:00
Jun Wu
20c87ae2db demandimport: blacklist windll
Summary:
This solve the following issue when running on devserver with demandimport enabled:

  In [1]: ui.config?
  abort: No module named windll!

Reviewed By: xavierd

Differential Revision: D18898009

fbshipit-source-id: bf4f3b12ac6f0a1628ed8ef805f6843868363542
2019-12-10 17:53:39 -08:00
svcscm
53a2bb126e Updating submodules
Summary:
GitHub commits:

5c811f4b3a
ad7bdb3133
849da74e65

Reviewed By: zpao

fbshipit-source-id: c6e56b1eb144948f545be0bea68809035d7319f0
2019-12-10 17:53:39 -08:00
svcscm
de14bc1837 Updating submodules
Summary:
GitHub commits:

c6506e2698
4427c1a832
a653857178
558f42bd6c
3839cbaf52

Reviewed By: zpao

fbshipit-source-id: 4a253bba6de9a2c2a11a82e33809a370e1b4fd04
2019-12-10 16:58:05 -08:00
Chad Austin
d0ce25afba log to scuba upon automatic gc
Summary:
A spike in automatic GCs usually implies something has gone wrong. Log
an event for each one, recording the cache size prior to the GC and
the cache size after.

Reviewed By: simpkins

Differential Revision: D18902580

fbshipit-source-id: 158b2635733a415a9fcc7c412b2c0f44ed04aa01
2019-12-10 16:16:35 -08:00
Jun Wu
27a1ffa9d1 tests: fix test-doctor.t on Windows
Summary:
On Windows the error message is different. In this test we only need to verify
a command fails. Therefore remove the output and only keep the exit code.

Reviewed By: xavierd

Differential Revision: D18915580

fbshipit-source-id: bd1031170335918d507e6225b02541ec52bc62fa
2019-12-10 14:45:43 -08:00
svcscm
72a5354e39 Updating submodules
Summary:
GitHub commits:

1d7a0e1a4b
b8031f09d7
7fd86a8f64

Reviewed By: zpao

fbshipit-source-id: 98b2487b39fb56641641c0947ed09f883755126a
2019-12-10 14:19:36 -08:00
Puneet Kaushik
eb17a5bde8 Enable getScmStatusBetweenRevisions
Summary: Enabling getScmStatusBetweenRevisions to work on Windows and its tests.

Reviewed By: simpkins

Differential Revision: D18431271

fbshipit-source-id: eee82538e2fc3d7e371c96fc271cd9662ea6d737
2019-12-10 14:07:26 -08:00
Puneet Kaushik
4c720d12c1 hg status tests for Eden Windows
Summary: This diff contains the tests for hg status. This is not comprehensive list of tests but the tests that we have working at this time. More tests will be added as we test and fix issues.

Reviewed By: simpkins

Differential Revision: D18454851

fbshipit-source-id: 80e3ff4d24873016dc420f202fbfe53fcffc24f2
2019-12-10 14:07:26 -08:00
Puneet Kaushik
8b0feed6fd Move EdenDispatcher under PrjfsChannel
Summary: The current implementation of EdenDispatcher is projectedfs specific and was linking against its dll. This was adding the need to add Prjfs dll for running our tests. Moving it to prjfs channel should fix that.

Reviewed By: simpkins

Differential Revision: D18462171

fbshipit-source-id: ff7bfcdd14d19552d49b61ff03cdd770badfcd68
2019-12-10 14:07:26 -08:00
Puneet Kaushik
35accc34bc Add TestMount class to test EdenMount
Summary: This class provides an interface to create an EdenMount for testing and use it through the APIs. It's similar to TestMount for Eden on Posix.

Reviewed By: simpkins

Differential Revision: D18454852

fbshipit-source-id: 91d64b2c535060979ef634fe4e7779f50f169fc5
2019-12-10 14:07:25 -08:00
Puneet Kaushik
f54831e526 Teach WinStore to work with std::wstring_view to avoid unnecessary copy
Summary: WinStore interface was taking wstring for path and we were converting wchar pointers to wstring which were later converted to RelativePath. In this diff we pass the path as is and save one copy per operation.

Reviewed By: simpkins

Differential Revision: D18444773

fbshipit-source-id: fa6a3efb6a7e91cf9b0dca919e80870e1bcf0804
2019-12-10 14:07:25 -08:00
Puneet Kaushik
abfdda5447 Create TestFsChannel class
Summary: Moved the current FsChannel implementation in to PrjfsChannel and converted the FsChannel.h into an abstract class. Created a TestFsChannel class for testing.

Reviewed By: simpkins

Differential Revision: D18440401

fbshipit-source-id: 30d6742cd18b4b76b664e3af7ed1a528cf876046
2019-12-10 14:07:25 -08:00
Puneet Kaushik
e19f84aa65 Add resetParents to enable hg commit
Summary:
A successful run of hg commit ends in call to resetParents() to reset the current parent commit.

We also need resetParent to run the Status tests.

Reviewed By: genevievehelsel

Differential Revision: D18439954

fbshipit-source-id: 44f7a009e050bd5e2130d8c4fc3a85a8ec3bfad1
2019-12-10 14:07:24 -08:00
Puneet Kaushik
86f5c839d1 Add EdenMount diff to generate the ScmStatus
Summary: This implements diff() inside EdenMount which uses GenerateStatus to compute the ScmStatus.

Reviewed By: simpkins

Differential Revision: D18195574

fbshipit-source-id: 16cfbda35796d1cfd4e13e6566b7ac222163aea4
2019-12-10 14:07:24 -08:00
Puneet Kaushik
7e2de5609a Add GenerateStatus to compute the status change
Summary:
GenerateStatus will parse the CurrentState and compute the status. The algorithm here is little different than of Eden with FUSE, Eden with Prjfs will loop over the CurrentState and only compare the entries present in it. CurrentState has the list of all the entries modified and a complete status could be generated from it.

This doesn't solve all the scenarios. I have few unit tests running at the top of this stack and those cases are working. I need to test the other cases and make them work. One case that I already know doesn't work is when a user resets the parent and doesn't make any change. I will work on fixing the other cases and put up diffs for that.

Reviewed By: simpkins

Differential Revision: D18195573

fbshipit-source-id: 3650f26387f1a5f8d467fd81148c266fa1307a3c
2019-12-10 14:07:24 -08:00
Zeyi (Rice) Fan
ecfc387fe6 backingstore: optionally create EdenApiRemoteStore
Summary: This diff makes EdenFS to be able to use EdenApi to import blobs and trees when requested objects are not present in local hgcache.

Reviewed By: chadaustin

Differential Revision: D18605547

fbshipit-source-id: 4acd2e07cfd9de6b6775ded30ea22a4478b9f1e4
2019-12-10 13:40:56 -08:00
Zeyi (Rice) Fan
2f15f80957 revisionstore: make EdenApiRemoteStore to be able to fetch trees from remote
Summary: This will allow us to use EdenApi in EdenFS to fetch trees and blobs.

Reviewed By: xavierd

Differential Revision: D18622844

fbshipit-source-id: 59a9091e9f2fdbcae078da2fb24ee9c0dd18505b
2019-12-10 13:40:55 -08:00
Zeyi (Rice) Fan
ae1dae6b96 eden: add experimental:use-edenapi for optionally turn on EdenApi importing
Summary:
Add an option `experimental:use-edenapi` to `EdenConfig`.

See the next diff for usage.

Reviewed By: chadaustin

Differential Revision: D18605549

fbshipit-source-id: 2786c21bb38a76229078662cc5c1ddf906d1be4a
2019-12-10 13:40:55 -08:00
Zeyi (Rice) Fan
ed9e7abca2 eden: add FakeEdenConfig for testing
Summary:
I think we need to make `EdenConfig` available for testing as more things are using `EdenConfig` to configure behaviors.

Right now it does not do much things other than just construct an `EdenConfig` instance without manually providing these parameters.

Reviewed By: chadaustin

Differential Revision: D18798484

fbshipit-source-id: b4a85d966a9b4f425c547bef9eb6e7570d7e2170
2019-12-10 13:40:55 -08:00
svcscm
4fe7ad486c Updating submodules
Summary:
GitHub commits:

b459fcc89f
2b060c1498
13a2c072c4

Reviewed By: zpao

fbshipit-source-id: 59fb11a977dcb7b2c09acb7fe997b0d5e52f27c4
2019-12-10 12:50:56 -08:00
Xavier Deguillard
d21a108280 tests: fix test-atomictempfile.py
Summary:
os.stat uses float values, use util.stat instead.

(Note: this ignores all push blocking failures!)

Reviewed By: singhsrb

Differential Revision: D18913982

fbshipit-source-id: 2df5ad0efbba3b18ef90669b4d1910ab39730e18
2019-12-10 11:57:12 -08:00
svcscm
8df1bf08d1 Updating submodules
Summary:
GitHub commits:

fd8771904e
6bf51e234f
6380df5e10
696c2a2359

Reviewed By: zpao

fbshipit-source-id: 188670fcdc50ccf060eea137698ecfb45484e059
2019-12-10 11:23:05 -08:00
Chad Austin
b47bf0289e ensure fuse capability flags survive graceful restart
Summary:
Two bugs conspired to cause edenfs after a graceful restart to think
the kernel supported FUSE_NO_OPENDIR_SUPPORT when it didn't: the
connection info struct wasn't zeroed, and FUSE connection capabilities
weren't properly mirrored into the Dispatcher upon graceful
restart. Fix both and add an integration test.

Reviewed By: simpkins

Differential Revision: D18903761

fbshipit-source-id: 23f4db3e240ee7d035f707820072c606a45f1138
2019-12-10 10:25:30 -08:00
Stefan Filip
1d61f2dbfd manifest: add TODOs with ideas about improvements on Manifest
Summary: Ideas for improvements. Put them in the code for future consideration.

Reviewed By: quark-zju

Differential Revision: D18870140

fbshipit-source-id: a2a84d94d72303adc64614b06e36588ff7778aab
2019-12-10 10:18:19 -08:00
Stefan Filip
ef220f3f7c manifest: add the list method to the Manifest trait
Summary:
The list functionality is required by EdenFs. We want this functionality
to be well supported by the Manifest.

Reviewed By: quark-zju

Differential Revision: D18870143

fbshipit-source-id: 1ebaa713ff521226e6ace22cbd35cc841d967298
2019-12-10 10:18:19 -08:00
Stefan Filip
67b0e3a600 manifest: rename FsNode to FsNodeMetadata
Summary:
Consistency in naming. The general idea is to have Metadata types that don't
contain paths. Then we will have File, Directory and eventually FsNode that
will contain paths.

Reviewed By: quark-zju

Differential Revision: D18870141

fbshipit-source-id: a1f09add7f1c3dd4fa0348693cd3ce2fd5767fa7
2019-12-10 10:18:18 -08:00
Stefan Filip
05aa630a18 manifest-tree: rename Tree to TreeManifest
Summary:
This rename is going to make it easier to import and use outside of
manifest specific crates.

Reviewed By: quark-zju

Differential Revision: D18870142

fbshipit-source-id: 2f3ea460170308162ee834efc038b2dcedd9e233
2019-12-10 10:18:18 -08:00
svcscm
2cdda7f4d8 Updating submodules
Summary:
GitHub commits:

e63b40cb4b
cb5f0670a6
470a664def
6e8f70b2d9
0fb026ca58
3595e0cf38
79b171ffa3
fb5322d98d
cd48fc606b

Reviewed By: zpao

fbshipit-source-id: 99bee659ea0fca0247d67d2dac12a821e1bd402d
2019-12-10 10:18:18 -08:00
svcscm
b774054dee Updating submodules
Summary:
GitHub commits:

6c87dc4d3c
5ec43afc1d
1e3cb8283f
3af1c72471
dc8e6e6e68
405e596d50
f40ae54a52
479a143912

Reviewed By: zpao

fbshipit-source-id: 932226ea90129f6c337d9fe833e2c3e4e1f955ee
2019-12-10 05:58:56 -08:00
Lukasz Piatkowski
d2520b62e2 Fixes for travic CI build (#3)
Summary:
This should fix the Travis CI builds. It adds rust toolchain support inside docker and sets the required THRIFT env variable.
Pull Request resolved: https://github.com/facebookexperimental/rust-shed/pull/3

Reviewed By: krallin

Differential Revision: D18905608

Pulled By: lukaspiatkowski

fbshipit-source-id: 5db1eff6f215a6617d8acaa0c99a62d45225956b
2019-12-10 04:56:56 -08:00
svcscm
49c3720863 Updating submodules
Summary:
GitHub commits:

aab1edd0df
ed15255982

Reviewed By: zpao

fbshipit-source-id: 613c5af2a8601afbd65c11eef8c8087d95420942
2019-12-10 04:56:56 -08:00
svcscm
f887d792a7 Updating submodules
Summary:
GitHub commits:

1e7ac08e2a

Reviewed By: zpao

fbshipit-source-id: 8fcf7b1799d6c639178a375d0306363662883209
2019-12-09 20:58:11 -08:00
Jun Wu
2781801365 doctor: attempt to fix mutationstore, metalog and allheads
Summary:
Since we got `repair` APIs in the Python land, use them to attempt to repair
things.

This would hopefully be able to fix some user-reported issues, usually
caused by hard reboots.

Reviewed By: xavierd

Differential Revision: D18737907

fbshipit-source-id: ef3deccbf4a1c959cff7e3eb78771acd9ae9bd01
2019-12-09 20:02:09 -08:00
Chad Austin
40e6e4bde2 enable eden debug logging in open source and macOS builds
Summary:
Register the logging and logging_full option handlers in the open
source and macOS builds of Eden.

Reviewed By: simpkins

Differential Revision: D18870545

fbshipit-source-id: 140debe77be071a9fa2d628ad305517273e095e3
2019-12-09 19:41:18 -08:00
svcscm
ba49b7f7f6 Updating submodules
Summary:
GitHub commits:

b221a6b5bb

Reviewed By: zpao

fbshipit-source-id: 6dd4252a106f8c6f6cb35613a1d10d346006d8a1
2019-12-09 18:22:40 -08:00
Stefan Filip
ef336812a7 hggit: update dulwich usage to match 0.19+
Summary:
From Dulwich changelog:
0.19.0 2018-03-10
API CHANGES
GitClient.send_pack now accepts a generate_pack_data rather than a
generate_pack_contents function for performance reasons.

(Note: this ignores all push blocking failures!)

Reviewed By: quark-zju

Differential Revision: D18901269

fbshipit-source-id: 9510b3a516cb2fcbeae9f1f2072d8445e442a9ed
2019-12-09 17:59:14 -08:00
svcscm
61a9c9c6f8 Updating submodules
Summary:
GitHub commits:

8b2d8468c9
840bcfbdb8
0a702dcd44
e45e35f7e1

Reviewed By: zpao

fbshipit-source-id: 524ed22a0723ac5b669ef4c1b00b6b4e511a2d58
2019-12-09 17:59:14 -08:00
svcscm
d2999e8999 Updating submodules
Summary:
GitHub commits:

a072a44b4d
59c0f06905
c39a3e42b8
a68dff5c35
f4b1e6532c
d0790d3a38

Reviewed By: zpao

fbshipit-source-id: 29e7660c1d2981dcd61821dc79c8cc088f96ae5c
2019-12-09 16:46:21 -08:00
Genevieve Helsel
504a255355 add getTreeForManifest
Summary: Adds a function which takes both the manifestID and the commitID to get a Tree. This will be used in `checkOutRevision()` and this allows us to skip looking up the manifestID since the caller can just pass it in themselves.

Reviewed By: wez

Differential Revision: D18719405

fbshipit-source-id: 919f0a7c84bff4a2f0bc20110c45bd272f9e9107
2019-12-09 16:25:27 -08:00
Chad Austin
4963352d64 temporarily mark trees persistent
Summary:
It is very painful for Eden's automatic GC to delete trees from the
local store. An unfortunate number of programs scan the entire repo,
causing every tree to be loaded. If Eden's automatic GC kicks in, it
causes all trees to be downloaded again, blocking both FUSE and Thrift
requests.

Temporarily mark trees as persistent.

Reviewed By: fanzeyi

Differential Revision: D18842066

fbshipit-source-id: 4a22defcb1151912337d4167ac49852519c7789d
2019-12-09 16:11:36 -08:00
Jun Wu
a3129104fe tests: fix buck run :run_tests
Summary:
`buck run :run_tests -- ...` was added to be a buck version of `run-tests.py`.
However it does not work because the watchman path provided was invalided.

Reviewed By: xavierd

Differential Revision: D18892361

fbshipit-source-id: 8745a33e0d19b14f042ef1cd255d10c5737dffb1
2019-12-09 15:26:30 -08:00
Jun Wu
75a8173a10 tests: enable chg for 572 tests
Summary:
Add `#chg-compatible` to 572 tests that seem to pass with chg enabled.
This should make them run faster.

Reviewed By: xavierd

Differential Revision: D18870507

fbshipit-source-id: fe895e733efffc9286cd3d17c7a156c803124395
2019-12-09 15:26:29 -08:00
Jun Wu
84a4da85ca metalog: implement bi-directional migration
Summary:
Previously, it's fine to migrate up to metalog, and fine to migrate down once
since we double write data. However, re-enabling metalog can be problematic
since there is no code path to do "migrate up" again.

This diff fixes the issue by tracking what files (or "keys") are using metalog
as the source of truth in metalog. So we can accurately figure out whether to
migrate svfs files to metalog on demand.

Reviewed By: xavierd

Differential Revision: D18864424

fbshipit-source-id: e61e1790c231f9c88de869f413f27bb954a29920
2019-12-09 14:18:18 -08:00
Jun Wu
d754747d01 bindings: add repair to structures
Summary: Expose the repair API to the Python world.

Reviewed By: xavierd

Differential Revision: D18737913

fbshipit-source-id: c31085727589b6938c2fafb28897925aea617bc4
2019-12-09 14:15:47 -08:00
Jun Wu
02b4902319 metalog: implement Repair
Summary:
Provide a way to repair the MetaLog. It's not just repairing 2 indexedlog
structures, but also checking the relationships between them.

Reviewed By: xavierd

Differential Revision: D18737904

fbshipit-source-id: fcf8ae56a1fdbb0561765701d962dfad4a8b5bd4
2019-12-09 14:15:47 -08:00
svcscm
039e5a1f9d Updating submodules
Summary:
GitHub commits:

aba4498a76
3fa5699d02

Reviewed By: zpao

fbshipit-source-id: 0109e104147cf3eeff04bcc79e72542a1bb89056
2019-12-09 14:15:47 -08:00
svcscm
fa464c1e21 Updating submodules
Summary:
GitHub commits:

cfa22893ac

Reviewed By: zpao

fbshipit-source-id: 2b1b1ec6c532eef5f54c04fc50a57f1644972524
2019-12-09 13:39:34 -08:00