Commit Graph

191 Commits

Author SHA1 Message Date
Jun Wu
780faa2c10 tests: replace print in debugshell with ui.write
Summary:
Similar to D18926784.

Mercurial makes sure content written by `ui.write` is flushed, while there is
no such guarantee for `print`. When running using chg, it's more likely that
`print` content gets disappeared. Therefore let's use `ui.write` instead.

Reviewed By: ikostia

Differential Revision: D18926983

fbshipit-source-id: 960c189cc48bc3c89bb4f7993d2033dd45c52a67
2019-12-13 10:30:39 -08:00
Mateusz Kwapich
7e0e170db9 rename fbconduit extension to fbscmquery
Summary:
This is mostly result of:
```
  find . -type f -exec sed -i "s/fbconduit/fbscmquery/g" {} \;

```
`fbconduit` extension doesn't use conduit anymore so the name is just misleading.

Reviewed By: ikostia

Differential Revision: D18748843

fbshipit-source-id: 0d59e61ba7a96d86d9d1333d81255108cc3141bc
2019-12-13 03:23:25 -08:00
Xavier Deguillard
828f5b487d tests-flagprocessor.t: fix it
Summary: This looks like a fallout of D18666054

Reviewed By: quark-zju

Differential Revision: D18992910

fbshipit-source-id: c2392553a0489097f82b4073bbbca37b43f53bba
2019-12-12 20:36:43 -08:00
Xavier Deguillard
e7a3859828 convert: disable "getting files" progress bar
Summary:
This is showing in profile for about 10% of the time, let's just disable it as
getting files should be fairly quick.

Reviewed By: quark-zju

Differential Revision: D17841315

fbshipit-source-id: 08a61e19351fb5217431a3ca8993b8074f6d6c74
2019-12-12 14:15:05 -08:00
Xavier Deguillard
81365e9e9f localrepo: use repo.commitpending in repo.commitnotransaction
Summary:
There is no need to duplicate the code.

The test change are due to commitpending being called 3 times, vs 1 before. The
total amount of times {fileslog,manifestlog}.commitpending is called is unchanged.

Reviewed By: quark-zju

Differential Revision: D17841316

fbshipit-source-id: be33a126f913af8824a6d5ade17351707d39ed2f
2019-12-12 14:15:05 -08:00
Jun Wu
cf8efdb0d2 ui: use smarttraceback
Summary: The smart traceback is generally more useful.

Reviewed By: markbt

Differential Revision: D18666054

fbshipit-source-id: a96bb3e2919ed6692c4a7b965ad74cd2c8a66241
2019-12-12 13:47:49 -08:00
Adam Simpkins
1ec027f5bc add type annotations to mercurial/commands/eden.py
Summary: Add type annotations to all functions in this file.

Reviewed By: xavierd

Differential Revision: D18949804

fbshipit-source-id: 4ac1789aa086e9e5f2397bc005a33cfb2f6c0b56
2019-12-12 12:48:50 -08:00
Xavier Deguillard
f0bf034842 test-check-code: use glob for st_*time error
Summary: This keep failing due to single line changes. Let's make it more robust.

Reviewed By: kulshrax

Differential Revision: D18963816

fbshipit-source-id: c56748560b619c4c682e8cc9e5710c607aa363fb
2019-12-12 11:41:22 -08:00
Adam Simpkins
264f6565f6 make the EdenFS tree prefetch depth configurable
Summary:
D18263067 updated the `CMD_FETCH_TREE` handler to fetch tree with a depth
of 1, rather than using fetching the entire recursive tree structure.
I think this has contributed to some of the recent user-reported slowness for
fetch operations, as fetching a large directory structure now triggers many
more individual requests for each separate tree.

This diff adds a configuration parameter to control the tree fetch depth.  I
have set the default value at 3, which should hopefully provide a reasonable
level of tree prefetching without completely fetching all contents of deep
trees.

Reviewed By: wez

Differential Revision: D18942585

fbshipit-source-id: 6a8d749434520baee25a4277712c44b916adcb3f
2019-12-11 20:54:55 -08:00
Jun Wu
c745d644d9 doctor: fix incorrect repair on indexedloghistorystore
Summary:
I made a mistake when editing the last version of the code. It uses
indexedlogdatastore to repair indexedloghistorystore, which is incorrect.
Fix it.

Reviewed By: xavierd

Differential Revision: D18945952

fbshipit-source-id: addec020c761e6c11d2fe485eb408f5b200afb08
2019-12-11 17:34:44 -08:00
Jun Wu
ab69de6a14 util: improve smarttraceback
Summary:
Make util.smarttraceback:
- Support `traceback` object.
- Look more similar to normal traceback (headers, and indentation).
- Hide boring content.
- Show binary hashes in hex form.

Reviewed By: markbt

Differential Revision: D18666055

fbshipit-source-id: d61446604c673ec5e1a96912fa29935d4238ecc9
2019-12-11 16:34:33 -08:00
Jun Wu
efa601b19e snapshot: make import side-effect free for chg compatibility
Summary:
See also D5271419. Decorators in `bundle2` and `exchange` are not side-effect
free. Move them to `uisetup` for better chg compatibility.

Reviewed By: StanislavGlebik

Differential Revision: D18926786

fbshipit-source-id: 2a3fa6ed4cef42c9bad4666ff75496d483991f88
2019-12-11 15:45:23 -08:00
Jun Wu
b72f3479fd infinitepush: make import side-effect free for chg compatibility
Summary:
See also D5271419. Decorators in `bundle2` and `exchange` are not side-effect
free. Move them to `uisetup` for better chg compatibility.

Reviewed By: StanislavGlebik

Differential Revision: D18926782

fbshipit-source-id: b99e2882cb6530a648a065c6d739ee9a7aebb851
2019-12-11 15:45:23 -08:00
Jun Wu
652cc9c160 gitlookup: make import side-effect free for chg compatibility
Summary:
See also D5271419. Decorators in `bundle2` and `exchange` are not side-effect
free. Move them to `uisetup` for better chg compatibility.

Reviewed By: StanislavGlebik

Differential Revision: D18926783

fbshipit-source-id: 9855b8debe3f809139a9fc3c6eea2775a1562cfd
2019-12-11 15:45:23 -08:00
Jun Wu
cdc42a94d1 pushrebase: make import side-effect free for chg compatibility
Summary:
See also D5271419. Decorators in `bundle2` and `exchange` are not side-effect
free. Move them to `uisetup` for better chg compatibility.

Reviewed By: StanislavGlebik

Differential Revision: D18926787

fbshipit-source-id: cfbf8de6d8c841478d7aa3de180a0816349fa4e4
2019-12-11 15:45:22 -08:00
Xavier Deguillard
291db4b116 tests: fix test-fb-hgext-treemanifest.t
Summary: This looks like an omission from D18912817

Reviewed By: mitrandir77, ikostia

Differential Revision: D18935510

fbshipit-source-id: f72d95155c54d7eb36ff7c868f5e6b0ff6d6e39c
2019-12-11 10:20:59 -08:00
Mark Thomas
23fe5a7780 reset: add visibility tracking support
Summary:
The `reset` extension does its own obsmarker creation, rather than going
through `scmutil.cleanupnodes`.  This means it doesn't support new-style
visibility tracking.

Fix this by making it use `scmutil.cleanupnodes`.  This isn't completely
straightforward:

* The revset it uses to work out what to prune might accidentally include
  public commits (e.g. when you reset to a different public branch).  Make sure
  these are filtered.
* The tests originally had strip and obsmarker based tests.  The strip tests,
  when converted to obsmarkers and using `scmutil.cleanupnodes` stop working
  because they strip and revive the same commit over and over, which is an edge
  case that obsmarkers can't handle well.  Fix this by restoring the strip tests
  as strip tests.  A separate test handles the new-style visibility.
* Reset's behaviour is still a bit wonky.  If an ancestor of the source commit
  has other (non-bookmarked) descendants, then reset will try to prune those
  commits.  New-style visibility will ignore this, and this is tested in the
  new test.

Reviewed By: farnz

Differential Revision: D18912817

fbshipit-source-id: cc115333407cf67d339c24fcd0807ddedce2660d
2019-12-11 03:49:38 -08:00
Jun Wu
20c87ae2db demandimport: blacklist windll
Summary:
This solve the following issue when running on devserver with demandimport enabled:

  In [1]: ui.config?
  abort: No module named windll!

Reviewed By: xavierd

Differential Revision: D18898009

fbshipit-source-id: bf4f3b12ac6f0a1628ed8ef805f6843868363542
2019-12-10 17:53:39 -08:00
Jun Wu
27a1ffa9d1 tests: fix test-doctor.t on Windows
Summary:
On Windows the error message is different. In this test we only need to verify
a command fails. Therefore remove the output and only keep the exit code.

Reviewed By: xavierd

Differential Revision: D18915580

fbshipit-source-id: bd1031170335918d507e6225b02541ec52bc62fa
2019-12-10 14:45:43 -08:00
Zeyi (Rice) Fan
ecfc387fe6 backingstore: optionally create EdenApiRemoteStore
Summary: This diff makes EdenFS to be able to use EdenApi to import blobs and trees when requested objects are not present in local hgcache.

Reviewed By: chadaustin

Differential Revision: D18605547

fbshipit-source-id: 4acd2e07cfd9de6b6775ded30ea22a4478b9f1e4
2019-12-10 13:40:56 -08:00
Zeyi (Rice) Fan
2f15f80957 revisionstore: make EdenApiRemoteStore to be able to fetch trees from remote
Summary: This will allow us to use EdenApi in EdenFS to fetch trees and blobs.

Reviewed By: xavierd

Differential Revision: D18622844

fbshipit-source-id: 59a9091e9f2fdbcae078da2fb24ee9c0dd18505b
2019-12-10 13:40:55 -08:00
Zeyi (Rice) Fan
ae1dae6b96 eden: add experimental:use-edenapi for optionally turn on EdenApi importing
Summary:
Add an option `experimental:use-edenapi` to `EdenConfig`.

See the next diff for usage.

Reviewed By: chadaustin

Differential Revision: D18605549

fbshipit-source-id: 2786c21bb38a76229078662cc5c1ddf906d1be4a
2019-12-10 13:40:55 -08:00
Xavier Deguillard
d21a108280 tests: fix test-atomictempfile.py
Summary:
os.stat uses float values, use util.stat instead.

(Note: this ignores all push blocking failures!)

Reviewed By: singhsrb

Differential Revision: D18913982

fbshipit-source-id: 2df5ad0efbba3b18ef90669b4d1910ab39730e18
2019-12-10 11:57:12 -08:00
Stefan Filip
1d61f2dbfd manifest: add TODOs with ideas about improvements on Manifest
Summary: Ideas for improvements. Put them in the code for future consideration.

Reviewed By: quark-zju

Differential Revision: D18870140

fbshipit-source-id: a2a84d94d72303adc64614b06e36588ff7778aab
2019-12-10 10:18:19 -08:00
Stefan Filip
ef220f3f7c manifest: add the list method to the Manifest trait
Summary:
The list functionality is required by EdenFs. We want this functionality
to be well supported by the Manifest.

Reviewed By: quark-zju

Differential Revision: D18870143

fbshipit-source-id: 1ebaa713ff521226e6ace22cbd35cc841d967298
2019-12-10 10:18:19 -08:00
Stefan Filip
67b0e3a600 manifest: rename FsNode to FsNodeMetadata
Summary:
Consistency in naming. The general idea is to have Metadata types that don't
contain paths. Then we will have File, Directory and eventually FsNode that
will contain paths.

Reviewed By: quark-zju

Differential Revision: D18870141

fbshipit-source-id: a1f09add7f1c3dd4fa0348693cd3ce2fd5767fa7
2019-12-10 10:18:18 -08:00
Stefan Filip
05aa630a18 manifest-tree: rename Tree to TreeManifest
Summary:
This rename is going to make it easier to import and use outside of
manifest specific crates.

Reviewed By: quark-zju

Differential Revision: D18870142

fbshipit-source-id: 2f3ea460170308162ee834efc038b2dcedd9e233
2019-12-10 10:18:18 -08:00
Jun Wu
2781801365 doctor: attempt to fix mutationstore, metalog and allheads
Summary:
Since we got `repair` APIs in the Python land, use them to attempt to repair
things.

This would hopefully be able to fix some user-reported issues, usually
caused by hard reboots.

Reviewed By: xavierd

Differential Revision: D18737907

fbshipit-source-id: ef3deccbf4a1c959cff7e3eb78771acd9ae9bd01
2019-12-09 20:02:09 -08:00
Stefan Filip
ef336812a7 hggit: update dulwich usage to match 0.19+
Summary:
From Dulwich changelog:
0.19.0 2018-03-10
API CHANGES
GitClient.send_pack now accepts a generate_pack_data rather than a
generate_pack_contents function for performance reasons.

(Note: this ignores all push blocking failures!)

Reviewed By: quark-zju

Differential Revision: D18901269

fbshipit-source-id: 9510b3a516cb2fcbeae9f1f2072d8445e442a9ed
2019-12-09 17:59:14 -08:00
Jun Wu
a3129104fe tests: fix buck run :run_tests
Summary:
`buck run :run_tests -- ...` was added to be a buck version of `run-tests.py`.
However it does not work because the watchman path provided was invalided.

Reviewed By: xavierd

Differential Revision: D18892361

fbshipit-source-id: 8745a33e0d19b14f042ef1cd255d10c5737dffb1
2019-12-09 15:26:30 -08:00
Jun Wu
75a8173a10 tests: enable chg for 572 tests
Summary:
Add `#chg-compatible` to 572 tests that seem to pass with chg enabled.
This should make them run faster.

Reviewed By: xavierd

Differential Revision: D18870507

fbshipit-source-id: fe895e733efffc9286cd3d17c7a156c803124395
2019-12-09 15:26:29 -08:00
Jun Wu
84a4da85ca metalog: implement bi-directional migration
Summary:
Previously, it's fine to migrate up to metalog, and fine to migrate down once
since we double write data. However, re-enabling metalog can be problematic
since there is no code path to do "migrate up" again.

This diff fixes the issue by tracking what files (or "keys") are using metalog
as the source of truth in metalog. So we can accurately figure out whether to
migrate svfs files to metalog on demand.

Reviewed By: xavierd

Differential Revision: D18864424

fbshipit-source-id: e61e1790c231f9c88de869f413f27bb954a29920
2019-12-09 14:18:18 -08:00
Jun Wu
d754747d01 bindings: add repair to structures
Summary: Expose the repair API to the Python world.

Reviewed By: xavierd

Differential Revision: D18737913

fbshipit-source-id: c31085727589b6938c2fafb28897925aea617bc4
2019-12-09 14:15:47 -08:00
Jun Wu
02b4902319 metalog: implement Repair
Summary:
Provide a way to repair the MetaLog. It's not just repairing 2 indexedlog
structures, but also checking the relationships between them.

Reviewed By: xavierd

Differential Revision: D18737904

fbshipit-source-id: fcf8ae56a1fdbb0561765701d962dfad4a8b5bd4
2019-12-09 14:15:47 -08:00
Jun Wu
083bf3f1c3 run-tests: support '#chg-compatible' header
Summary:
Detect `#chg-compatible` in test header and force enable chg. This would allow
us to mark tests as chg-compatible to get some speed ups.

Reviewed By: xavierd

Differential Revision: D18870508

fbshipit-source-id: 923d6cb10b0c621d9995809b60ae535126f6f2a5
2019-12-09 13:20:25 -08:00
Jun Wu
fb13bc5ecf run-tests: make --chg work again
Summary:
The chg logic is inlined in the "hg" binary so the old logic running "chg" as
an external binary needs to be changed.

The version check part can be annoying - it requires more frequent `make local`
to run the tests while `make local` is actually not necessary. So let's just
skip the version check if running in tests.

Reviewed By: ahornby

Differential Revision: D18870510

fbshipit-source-id: 2343549c4080a0425098698b5423c6cfc1f60c77
2019-12-09 13:20:24 -08:00
Xavier Deguillard
82082bfaf8 policy: simplify it
Summary:
We only support the "C" policy, therefore let's inline it and simplify the code
significantly.

Reviewed By: quark-zju

Differential Revision: D18819680

fbshipit-source-id: 0bc0a12b85ddc42af007d3a20c0f4e8497bedb21
2019-12-09 13:13:27 -08:00
Xavier Deguillard
24d99a9178 util: remove os.stat_float_times(False)
Summary:
This is not available on Python3, instead stat.ST_MTIME and friends should be
used to get an integer time. This is achieved by wrapping the output of the
stat function and overriding the st_[acm]time properties.

Reviewed By: quark-zju

Differential Revision: D18819679

fbshipit-source-id: 2911f3d47506456e927872d57f69253d903618e2
2019-12-09 13:13:26 -08:00
Jun Wu
6da344eee2 hgcommands: avoid forked processes to override trace output
Summary:
The environment variable `EDENSCM_TRACE_OUTPUT` specifies where to write the
tracing output for the current command. Environment variables are inherited
by subprocesses by default. That is undesirable because another hg command
(triggered by hook, or background maintaince) will rewrite the trace output.

Avoid it by unsetting EDENSCM_TRACE_OUTPUT for subprocesses.

Reviewed By: xavierd

Differential Revision: D18892973

fbshipit-source-id: 575b6c0df2e7a0775172893e4aa72ca33fa4658c
2019-12-09 13:09:25 -08:00
Durham Goode
dd6f76863b lfs: use unfiltered repo for lfs gc
Summary:
Some users were reporting hg gc failures because it couldn't access a
hidden commit.  It looks like there's a `repo.unfiltered().revs("draft()")`
line to get the list of draft commits, so we need to also use an unfiltered repo
when extracting all the pointers.

Reviewed By: quark-zju

Differential Revision: D18863471

fbshipit-source-id: 61220c1ec3dcbf55879211bb271b4d9b686d66a1
2019-12-09 12:34:22 -08:00
Durham Goode
ad43fdcaf6 fastlog: fix fastlog directory histories
Summary:
D18732257 changed our fastlog integration from using conduit to using
scmquery. This changed the result type from a hash to a dictionary of {"hash":
"xxx"}, but the consumer code wasn't updated appropriately. This fixes it.

Reviewed By: mitrandir77

Differential Revision: D18833566

fbshipit-source-id: 2573cbf4f93c5e4aeb2515d42696526ceb51428e
2019-12-09 12:32:07 -08:00
Jun Wu
8f749d32f2 metalog: handle missing Root blob errors correctly
Summary:
The previously behavior silently uses the empty Root upon "id not found" cases,
which hides data corruption issues captured by the next changes.

Reviewed By: xavierd

Differential Revision: D18737906

fbshipit-source-id: 0f2cfe1b05f11e9d2584d96a14afcb1aed80da4d
2019-12-06 21:08:33 -08:00
Jun Wu
91884d1fab tests: mark tests as skipped on MySQL error
Summary:
Recently many tests become unreliable due to MySQL errors like:

  OperationalError: 2055: Lost connection to MySQL server at '<host:port>', system error: 32 Broken pipe

That makes the testing infra think our tests are broken and cause:

  # Test Failure Bot
  Test is now disabled because it has been consistently broken

Avoid that by detecting MySQL errors and mark tests as skipped.

Reviewed By: simpkins

Differential Revision: D18668303

fbshipit-source-id: 02cb09f31c712fed41903f1e8c4ad45a1d277c30
2019-12-06 21:01:44 -08:00
Jun Wu
35f85f7864 morecolors: remove extension
Summary:
It was used to highlight "more interesting" (aka. extension) code. Nowadays the
non-extension part of code is also interesting. Therefore drop the extension.

Reviewed By: simpkins

Differential Revision: D18666053

fbshipit-source-id: 027190f742015bdb9e7dba109e7a28987af3cf68
2019-12-06 21:01:44 -08:00
Jun Wu
b8c04b5a97 indexedlog: do not repair non-existent directory
Summary: Otherwise the repair will fail with "unable to lock directory" errors.

Reviewed By: xavierd

Differential Revision: D18737911

fbshipit-source-id: 2c47b5b9a071e0db2d20fc385ef466aef1e732f6
2019-12-06 19:35:06 -08:00
Jun Wu
a1f08d6628 indexedlog: make it possible to construct Error from other errors
Summary: This makes it possible for other crates to implement the `Repair` trait.

Reviewed By: xavierd

Differential Revision: D18737909

fbshipit-source-id: 3d098e583b45e84eb4bd01ef45569e9694b63ed6
2019-12-06 19:35:05 -08:00
Jun Wu
d4c4ccde40 zstore: implement indexedlog::DefaultOpenOptions
Summary: This implements `Zstore::repair` for free.

Reviewed By: xavierd

Differential Revision: D18737910

fbshipit-source-id: 7db2032ba4722b7073e777fbad7229a801724f71
2019-12-06 19:35:05 -08:00
Jun Wu
237a9708f6 nodemap: implement indexedlog::DefaultOpenOptions
Summary: This implements `Node{Map,Set}::repair` for free.

Reviewed By: xavierd

Differential Revision: D18737905

fbshipit-source-id: f0a4a64fddeb895d979bee3c51478c138c8235fe
2019-12-06 19:35:05 -08:00
Jun Wu
b0f061f66b mutationstore: implement indexedlog::DefaultOpenOptions
Summary: This implements `MutationStore::repair` for free.

Reviewed By: xavierd

Differential Revision: D18737912

fbshipit-source-id: 098e22bf4ab94af96be9b0d54914abf880c91f74
2019-12-06 19:35:04 -08:00
Jun Wu
7a16996ce0 revisionstore: implement indexedlog::DefaultOpenOptions
Summary:
This simplifies the logic a bit by getting a free `repair` method from
indexedlog.

Reviewed By: xavierd

Differential Revision: D18737908

fbshipit-source-id: 4988c1a83b7709b751cd1899c5663acc0c42e313
2019-12-06 19:35:04 -08:00