Commit Graph

51186 Commits

Author SHA1 Message Date
Jun Wu
6a9f476980 sparse: apply sparse matcher for merge.update
Summary: This reduces directories needed to be visited.

Reviewed By: sfilipco

Differential Revision: D18405646

fbshipit-source-id: 7d78bb997bd0a8148cabc04329dabf59fbc191a6
2019-11-12 15:35:46 -08:00
Jun Wu
57de4307d5 sparse: add a test showing suboptimal fetches
Summary: The test uses the tracing framework to examine paths being prefetched.

Reviewed By: sfilipco

Differential Revision: D18405644

fbshipit-source-id: 6f3e72c1c084e0bb4950b050e3217bf811e15dc1
2019-11-12 15:35:46 -08:00
Jun Wu
0cb781519d matcher: use tree matcher if possible
Summary:
For glob patterns, tree matcher has a better 'visitdir' implementation that
can greatly speed up some pattern matching. So try to use it instead.

This should make things like `hg status -n -c "glob:path/to/something**"` much
faster.

With this change:

  % ~/hg/hg files 'glob:skycastle/src/test/resources**' --time --pager=off | wc -l                                                                                                                                                         :(
  time: real 0.110 secs (user 0.070+0.000 sys 0.020+0.000)
  118

Without this change:

  % hg files 'glob:skycastle/src/test/resources**' --time --pager=off | wc -l
  time: real 16.700 secs (user 15.660+0.000 sys 1.310+0.000)
  118

A config was added so we can turn this off if it causes issues.

The fsmonitor code was slightly changed since treematcher.visitdir does not
like trailing slash.

Reviewed By: matthewdippel

Differential Revision: D18305373

fbshipit-source-id: 2ff927041ba8bab6912f125a7ae465be1a0ea659
2019-11-12 15:35:46 -08:00
Jun Wu
04613287e2 merge: do disablecasecheck config check earlier
Summary:
The new code skips potentially expensive `wc.dirty`, which might trigger
unnecessary tree fetches.

Reviewed By: sfilipco

Differential Revision: D18430900

fbshipit-source-id: 563d506f4b8785c6cc6f7d91662d78fc865cea47
2019-11-12 15:35:45 -08:00
Jun Wu
32ceffb794 sparse: fix up some issues with the forceincludematcher
Summary:
Two small issues:

Use `matchfn` instead of `__call__`, since `basematcher` has:

    def __call__(self, fn):
        return self.matchfn(fn)

    def matchfn(self, f):
        return False

Avoid using forceincludematcher when unnecessary.

Reviewed By: sfilipco

Differential Revision: D18414143

fbshipit-source-id: f7a81f88ac922f75b5a145e8094fab4dc8bd1681
2019-11-12 15:35:45 -08:00
Jun Wu
253daad18b sparse: use always matcher more consistently
Summary:
The `repo.sparsematch(*revs)` API is expected to return a union matcher that
unions sparse configurations from multiple revs.

I expect `repo.sparsematch(a, b, c)` to match more files than
`repo.sparsematch(a, b)`. However, that's not true if all sparse profiles in
`repo.sparsematch(a, b)` are empty (which, turns it into an always matcher),
and `repo.sparsematch(c)` is not empty.

Fix it by using the alwaysmatcher if any of the revs passed to `sparsematch`
has an empty profiler, instead of requiring all revs to all being empty
profiles.

Reviewed By: sfilipco

Differential Revision: D18414145

fbshipit-source-id: 357e90961acb9b2ae59fbdf84509f8201f629d44
2019-11-12 15:35:44 -08:00
Jun Wu
7ccd021436 manifest: add a debug command to show treemanifest ids
Summary: This is used in an upcoming diff.

Reviewed By: sfilipco

Differential Revision: D18414144

fbshipit-source-id: 92a68b36f24b01d87c55d4a5f52de4efcc50221e
2019-11-12 15:35:44 -08:00
Jun Wu
df37eae50a manifest: add an API to iterate directories
Summary:
Right now, there is no read-only API to get directory information.
In an upcoming change I'd like to get a mapping between tree ids and paths.
So add an `tree.dirs()` API to do that.

Reviewed By: sfilipco

Differential Revision: D18414146

fbshipit-source-id: bf47071b052e512e65fdaafe71742274bed4131f
2019-11-12 15:35:43 -08:00
Jun Wu
a9f0edda60 manifest: add tracing span for prefetch
Summary: This gives us a way to check what exactly are being fetched.

Reviewed By: sfilipco

Differential Revision: D18405645

fbshipit-source-id: 9f1c928eaee521562d7f40dae56d477ad501aae1
2019-11-12 15:35:43 -08:00
Jun Wu
3158354519 pathmatcher: make TreeMatcher non-recursive
Summary:
Previously I thought it's possible to use `[pattern, "!%s/*" % pattern]` to
emulate non-recursive rules. However, that's not true.

Consider this rule `**/a`, if we translate it to `["**/a", "!**/a/*"]`, then
consider this path `a/b/a`:
- Its parent directory `a/b` matches the negative pattern `**/a/*` so the
  directory wouldn't be visited.
- If `a/b` is not visited, then `a/b/a` might be missed.
- But `a/b/a` does match the pattern `**/a`.

I couldn't find a way to express the non-recursive `**a` pattern using
recursive patterns.  Therefore,  change TreeMatcher APIs to do non-recursive by
default. The callsite can still append `/**` to all rules to get the old
behavior.

Reviewed By: sfilipco

Differential Revision: D18342228

fbshipit-source-id: a97144c2a73c6d35fbfbd0d55ea3661ad4a9ac12
2019-11-12 15:35:43 -08:00
Jun Wu
098c5a767c matcher: add Python wrappers for TreeMatcher
Summary: Add a Python binding so it can be used in the Python world.

Reviewed By: DurhamG

Differential Revision: D10861609

fbshipit-source-id: c70ed514229fbcdf99556dd26760f76e027d8aa2
2019-11-12 15:35:42 -08:00
Mark Thomas
ac08b49411 commitcloud: add extras method to FakeCtx
Summary:
The `snapshot` extension expects to be able to call `extra()` on the smartlog
context, so make the method available.

Reviewed By: quark-zju

Differential Revision: D18456550

fbshipit-source-id: 9d1620e1b5fbb0866e728c7b378d37b4de9dbe21
2019-11-12 13:55:31 -08:00
Adam Simpkins
9fed2c8215 fix some auto-fixable lint formatting errors
Summary:
Apply some automatic formatting fixes to a couple files that disagreed from
the automatic clang-format and black style.

Reviewed By: sfilipco

Differential Revision: D18446028

fbshipit-source-id: 48e4a82b5b8975b930f08ae56fa449e80b0439c8
2019-11-12 13:20:16 -08:00
Jun Wu
bd75c77669 fbconduit: make gitnode much faster
Summary:
`subset.filter` has the time complexity `O(len(subset))`, which can be huge.
Since we already know the one single revision, use `baseset` to construct the
resulting revset directly. This makes a huge difference in performance.

Reviewed By: sfilipco

Differential Revision: D18441368

fbshipit-source-id: 5094803e8968a5ddd4aaf4909a72f239325b5fff
2019-11-12 12:22:26 -08:00
Jun Wu
8b0916e317 pullcreatemarkers: do not crash if Phabricator client cannot be created
Summary:
If there is no `.arcconfig`, the Phabricator client will fail to be created.
Let's just wrap it in the `try` block to ignore the issue.

Reviewed By: sfilipco

Differential Revision: D18440072

fbshipit-source-id: b1141df2513991c8cd23817ae5257aa740ce0c3a
2019-11-12 12:22:26 -08:00
Jun Wu
de4ab2952f blackbox: increase buffer time for better performance, esp on Windows
Summary:
blackbox sync shows up as taking 100ms multiple times in tracing profiling
result on Windows. In particular, `atomicwrite` takes about 20ms.  `sync()`
calls `atomicwrite` multiple times. They add up to 100ms.

For now, improve it by increasing the buffer time so `sync()` gets called less
frequently. If it continues to be a problem, we might move the blackbox logic
to a separate thread.

See also https://gregoryszorc.com/blog/2015/10/22/append-i/o-performance-on-windows/
where Greg moved `close` to background threads on Windows to improve performance.

Reviewed By: sfilipco

Differential Revision: D18452761

fbshipit-source-id: 53901611fd9538e3c5e897431fdfcab3d7e6f909
2019-11-12 12:10:50 -08:00
Mateusz Kwapich
9668315bb2 diff command for diffing two commmits
Summary: Command will be used only in the tests for now.

Reviewed By: markbt

Differential Revision: D18303171

fbshipit-source-id: 4938cca6b0ac0fa1868ab75a64db6d23c201a4f8
2019-11-12 11:16:58 -08:00
Mateusz Kwapich
6d0f21c5bb fix headers for metadata-only changes
Summary:
Previously we returned early on no content changes but in case of metadata-only
changes (like file mode or copy info) we should output some header indicating those.

Reviewed By: markbt

Differential Revision: D18303168

fbshipit-source-id: b715a2254799f76a8e9ca87196865269c53e4830
2019-11-12 11:16:58 -08:00
svcscm
2f526e9753 Updating submodules
Summary:
GitHub commits:

56c338eee0

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: fdb23afb0d4656399082564c8e986d7e95e0b0d0
2019-11-12 03:19:14 -08:00
svcscm
b630f7ad8b Updating submodules
Summary:
GitHub commits:

a5381c4d13
a19de78da5
b5a7d0259c
8c4e217115

Reviewed By: yns88

fbshipit-source-id: 26a91452c36caab109dad713fb04b71551f36a90
2019-11-12 03:19:13 -08:00
svcscm
abcdf32d2f Updating submodules
Summary:
GitHub commits:

2bdb5a4a7c
dfd5219816
66f868b745
0c4130d051
c912150192
c17384fea4
e0b2156829
7aef78fb2e

Reviewed By: yns88

fbshipit-source-id: 82552466afa665f0e335d5dce385dfcae9247b0b
2019-11-11 16:20:23 -08:00
Jun Wu
ccea9853ff restack: do not select "ancestors(predecessors(stack))" as rebase source
Summary:
Both simpkins and I encountered cases where `restack` selects too many
commits.

Reviewed By: kulshrax

Differential Revision: D18344037

fbshipit-source-id: 4e0883ee149bcf62ab368ee129e49f3e2eb9c971
2019-11-11 12:08:28 -08:00
Jun Wu
0d6a3c26c0 tests: add a test showing restack selects commits sub-optimally
Summary:
The test shows that restack rebases some ancient commits to old stacks where
the user probably doesn't care about.

Reviewed By: kulshrax

Differential Revision: D18344038

fbshipit-source-id: 65fa1b809b8515a9546a03eb30b2a6304538cc83
2019-11-11 12:08:27 -08:00
Arun Kulshreshtha
1c2026e006 eden: set depth to 1 when fetching trees over SSH
Summary: Previously, whenever EdenFS needed to fetch a single tree node, the import helper would run a prefetch for the entire tree, resulting in an excessively large fetch. Let's limit the fetch to depth 1 to just get the desired node.

Reviewed By: singhsrb

Differential Revision: D18263067

fbshipit-source-id: edd6ac56940a2ade64d95e78154db5c46eedccea
2019-11-11 11:55:40 -08:00
Arun Kulshreshtha
4d38f38dc3 remotefilelog: add depth parameter to prefetchtrees
Summary: The top-level prefetchtrees function didn't support a depth parameter, while the underlying implementation does. Let's add an argument to pass on so that callers of the top-level function can configure the depth. The particular use case I have in mind is EdenFS's import helper, which needs to be able to limit the fetch depth to 1.

Reviewed By: quark-zju

Differential Revision: D18263661

fbshipit-source-id: 59991da2c30a5107df86c5a4b9614f113327f468
2019-11-11 11:55:40 -08:00
Arun Kulshreshtha
e64f743070 treemanifest: factor out HTTP fallback logic into new method
Summary: There are several places where we need to perform an HTTP tree fetch, and fall back to SSH if it fails. So far, every callsite has had to wrap the fetch function with a try block and manually print an error message and log the exception. Let's put all this functionality into a single wrapper function that catches all exceptions and returns a boolean indicating success or failure.

Reviewed By: quark-zju

Differential Revision: D18262915

fbshipit-source-id: b5c3395509eb90f70311f439129a6d60089bfe6b
2019-11-11 11:55:39 -08:00
Jatin Kumar
5c859df62d Move simple getClientUnique call sites to getSRClientUnique
Summary:
Applied codemod output from D18378597
```
clangr patch GAGSkATyRG7lVpsBANZ6tBguOgNZbthBAAAz
arc lint
```

Differential Revision: D18412196

fbshipit-source-id: 91ed7b35b01e5f4e0b2ff6ff27d76fc8a98e7a5b
2019-11-11 11:18:37 -08:00
Genevieve Helsel
f23c60212d support closing the Overlay class even if it is still in use
Summary:
Throw errors if a read or write is called on the `Overlay` class after `close()` has been called. Adds IO reference counting to know when it is safe to release the overlay lock after calling `close()`

This takes care to catch errors that could be thrown around overlay IO and makes sure to correctly decrement the IO count in the case of errors.

Reviewed By: chadaustin

Differential Revision: D17957360

fbshipit-source-id: af5e3c3f98c77800914d03cd748fb2f1c59ae264
2019-11-11 10:56:15 -08:00
svcscm
60d252f6c3 Updating submodules
Summary:
GitHub commits:

c10f5bf664
b6d8b723b7
35f57f4100
286e860fbc
8996c5393f

Reviewed By: yns88

fbshipit-source-id: b8cea63518aa662ae640c6d5e4a7c8c14cf0690f
2019-11-11 10:56:15 -08:00
Mark Thomas
e3c4eaf5d9 TARGETS: add pytz dependency
Summary:
Smartlog depends on `pytz` to render times in timezones other than the default
timezone.  Add this as a dependency to `libhg`.

Reviewed By: xavierd, farnz

Differential Revision: D18425630

fbshipit-source-id: 16bc147cbea6cdf34dda73eee17946186c9277b6
2019-11-11 10:12:34 -08:00
svcscm
bf47c6c0ab Updating submodules
Summary:
GitHub commits:

84ff1378e7

Reviewed By: yns88

fbshipit-source-id: 8f5c6e268f983b8c62b088dbd7fc10457bfab46b
2019-11-11 07:24:40 -08:00
Jun Wu
af54a4730f demandimport: make pyreadline non-lazy
Summary:
This makes `hg dbsh` work on Windows because IPython can now detect missing
`pyreadline` correctly.

Reviewed By: DurhamG

Differential Revision: D18388302

fbshipit-source-id: 07dcdd8886316f577249ea42b8229de30ae2c96f
2019-11-11 07:18:21 -08:00
svcscm
ffa7b12bd7 Updating submodules
Summary:
GitHub commits:

3b56786f08
aabaf3c3d7

Reviewed By: yns88

fbshipit-source-id: 88eec06a375a1789471d110918a088a84994f206
2019-11-10 12:22:16 -08:00
svcscm
c85be4ccff Updating submodules
Summary:
GitHub commits:

8d9e9cd41f
4c3495f81b

Reviewed By: yns88

fbshipit-source-id: b9575d2deb624b369e5626805244e7bb0bafb559
2019-11-09 15:23:31 -08:00
Jun Wu
704053a550 Partially back out "[hg] test-narrow-heads-migration: add some tests about secret commits"
Summary:
Backout unnecessary changes to unblock tests.

Original commit changeset: 117b02447f37

Reviewed By: sfilipco

Differential Revision: D18416800

fbshipit-source-id: 14a26daef31f1d41b901212ff097259f475f8fb6
2019-11-09 12:49:18 -08:00
Jun Wu
d16a8d3cf8 pushrebase: do push in one transaction
Summary:
The push operation has 2 phases: push, then move bookmarks, etc.  Make that in
one transaction so it wouldn't end up with an inconsistent state.

Reviewed By: sfilipco

Differential Revision: D18362363

fbshipit-source-id: 338ef1b088975a9d1b043ccef81782e14c77d8e1
2019-11-08 22:29:37 -08:00
Jun Wu
d1d23d69b8 continue: reset merge state if all files are resolved
Summary: This makes `hg continue` the way to exit the "unfinished merge" state.

Reviewed By: sfilipco

Differential Revision: D18110070

fbshipit-source-id: cf22242a8dc47603d9cc3a92cf544959330636c7
2019-11-08 19:32:41 -08:00
Jun Wu
419b664a5d test-narrow-heads-migration: add some tests about secret commits
Summary: The current behavior is actually okay. So let's add a test to protect it.

Reviewed By: sfilipco

Differential Revision: D18329288

fbshipit-source-id: 117b02447f371f1c369f897b136c1a97607bb93e
2019-11-08 19:30:16 -08:00
svcscm
24480ecb63 Updating submodules
Summary:
GitHub commits:

1f41d7adb1
c8fcfd74b5
72de842ac9

Reviewed By: yns88

fbshipit-source-id: 599e746838cc24a74e6f0dbaa1a03673c9d51bac
2019-11-08 18:30:30 -08:00
Stefan Filip
04aeb57df7 context: gracefully handle broken dirstate
Summary:
We want to parse revsets even when dirstate is broken.
We need this code to work in as many scenarios as possible os that we can start
to fix broken repositories.

Reviewed By: quark-zju

Differential Revision: D18385918

fbshipit-source-id: d161b8e3041f503a4662eddeb1a11e76f730e66d
2019-11-08 12:13:02 -08:00
Jun Wu
db819b4af9 pullcreatemarkers: skip creating "landed as" relationship if the new commit is not public
Summary:
This avoids some buggy cases. For example, the client repetitively marks X as
"landed as X".

Reported By: simpkins

Reviewed By: simpkins

Differential Revision: D18389747

fbshipit-source-id: 2a66730b5df1cb7971bce6c7ace04c55426f77c1
2019-11-07 21:47:25 -08:00
svcscm
89aa512894 Updating submodules
Summary:
GitHub commits:

403ca8188d
74c43417f7
0e5d305664

Reviewed By: yns88

fbshipit-source-id: 1db292dabf5341cdeb5d866ef3ca4c7129b24566
2019-11-07 17:52:53 -08:00
Jun Wu
b4bb1fb3eb phabricator: ignore errors calculating landed commits
Summary:
Not getting the landed commits is not a fatal error. So let's just ignore them
with some warning messages.

Reviewed By: DurhamG

Differential Revision: D18384062

fbshipit-source-id: bef01fdd3303ab66170175f5f5ed690182464df2
2019-11-07 17:31:39 -08:00
Jun Wu
00e6c6c080 phabricator: ignore subversion revisions in commit identifiers
Summary:
The "commit identifier" returned via Phabricator GraphQL can include subversion
numbers that cannot be converted to commit hashes. Ignore them.

Reviewed By: singhsrb

Differential Revision: D18376641

fbshipit-source-id: e62ba41a486fc91910928e4f5d66c5430bf8c142
2019-11-07 17:31:39 -08:00
Chad Austin
1206ab8ae3 write SNAPSHOT atomically during clone
Summary:
To avoid scenarios where hard reboots prevent the SNAPSHOT page from
getting flushed, atomically write the file upon clone.

Reviewed By: simpkins

Differential Revision: D18382583

fbshipit-source-id: 1c4de46165587635f71744daf4d2a8c71c58cc5f
2019-11-07 17:26:10 -08:00
Chad Austin
8bf6a06bfa fail with a more sensible error if a SNAPSHOT file is missing
Summary:
If a checkout's SNAPSHOT file disappeared, the mount error given would
be that a Hash had an invalid size. Instead, throw a file not found
error on mount.

Reviewed By: genevievehelsel

Differential Revision: D18381074

fbshipit-source-id: 35282e0990189d4084a2a64330a5733561cf88c2
2019-11-07 17:26:09 -08:00
Chad Austin
7ef7154001 improve error message when invalid hashes are given
Summary:
Looking at a log, it wasn't immediately obvious what might have passed
an invalid hash into the Hash constructor. Improve the error message
to make the cause clearer.

Reviewed By: genevievehelsel

Differential Revision: D18380916

fbshipit-source-id: 620b8fa902a87496b87a5aa0ff304e6991585864
2019-11-07 17:26:09 -08:00
svcscm
b73994cc00 Updating submodules
Summary:
GitHub commits:

d752e52a31
d7cc18d7c7
6e26fa9d03
76fcc9469a
1da1f04231
17ff03e136
d8fde1c7fc
8a07c4270c
50fdf05973
affb36ec7c

Reviewed By: yns88

fbshipit-source-id: dfcf050e5eaa6e5077ea9b4c21326f127ec6066c
2019-11-07 17:26:08 -08:00
James Donald
1056d968d8 Fix corner-case directory listings with Windows hg rage
Summary:
We now list some directories with `hg rage` on Windows, but a few fail with:
```
Abort: dir /o-s C:\open\myrepo\.hg/store/packs\ error: Parameter format not correct - "store\packs\".
```
This is due to the forward-slash after .hg, so replace it with a backslash on Windows.

Reviewed By: quark-zju

Differential Revision: D18378509

fbshipit-source-id: 3ff1a8ba9f400a7330fece4dd83e29efa7ffebc1
2019-11-07 17:07:05 -08:00
James Donald
adb4e71c54 hg rage on Windows doesn't need time /T
Summary: The uptime on Windows originally ran `time /T` to get the start time for comparison. By itself this would be insufficient because it doesn't include the date, and there is actually a portable `time.ctime()` step above in the log so this extra command is unnecessary.

Reviewed By: quark-zju

Differential Revision: D18376448

fbshipit-source-id: 06d5b4a87baec2352b6a07ac264161c6e43b5965
2019-11-07 16:01:21 -08:00