Summary:
When building a changegroup, sometimes we need to access the list of
files changed by each commit. To do so we need to inspect the manifest.
Previously this would end up downloading each tree one-by-one, producing a bunch
of pack files. With this patch we now do one bulk download at the very
beginning.
Reviewed By: quark-zju
Differential Revision: D6873076
fbshipit-source-id: b916c15efca0770129340f798d3e7b165da6aec9
Summary:
A future diff will add tree prefetching during changegroup creation. This
requires access to the tree prefetch function from the shallowbundle class.
let's move the prefetch functions onto the repo object in preparation for that.
Reviewed By: quark-zju
Differential Revision: D6873055
fbshipit-source-id: 18de0ee0f6ab566587509f9e23ebb2e5779ed1c8
Summary:
treedirstate: bugfix
Incorrect usage of write function that can cause corrupted files in some cases.
Write doen't guarantee all data will be written. Function write_all should be used instead.
https://doc.rust-lang.org/1.0.0/std/io/trait.Write.html#tymethod.write
See: It is not considered an error if the entire buffer could not be written to this writer.
Function returns number of bytes written.
Reviewed By: markbt
Differential Revision: D6883057
fbshipit-source-id: 4bc18cc9fa6e5db0ae8516d67dd6ba92c1facf21
Summary:
setuptreestores caters to a lot of configurations through the `ui`
instance. It makes sense to store the `ui` instance once instead of referring
to it through `repo.ui` every time.
Reviewed By: DurhamG, quark-zju
Differential Revision: D6867843
fbshipit-source-id: 3ffca85332ec4ae3ec8a54bf0dacc7e7cf423b53
Summary:
`hg rebase -i` only works if the source revision is ancestor of the working
directory parent. This updates the code to fail with an error message in this
case instead of rebasing nothing and reporting success.
Note that attempting to change the revset calculation to use `src::` instead
of `src::.` doesn't help fix anything--the histedit code simply rejects the
histedit operation when it finds commits that are not ancestors of the working
directory.
Reviewed By: quark-zju
Differential Revision: D6853548
fbshipit-source-id: 14600f31a5f4ca43c89a81e7b522656b65a056dc
Summary:
Improve UX for short hash infinitepush feature
I would like to provide some suggestions in case there are several with the same prefix,
so we can provide better user experience.
Reviewed By: markbt
Differential Revision: D6845233
fbshipit-source-id: 4868c1dc599f60e4ac495b71a4aed2d57c36ffa4
Summary:
Adds telemetry showing the status of watchman during an invocation of HG.
The new key, `fsmonitor_status` takes the value:
* `exception` if an exception occurred during the call to watchman.
* `unavailable` if watchman was unavailable.
* `fresh` if a fresh instance of watchman was encountered.
* `normal` if watchman was available normally.
Differential Revision: D6845070
fbshipit-source-id: 5ccf61a1009cdccbc0d1786b2533c3a7fc6ac2c9
Summary:
This is a resend of https://phab.mercurial-scm.org/D1430, without breaking
Windows.
I encountered "too many opened files" problem due to treemanifest packs on my
laptop. This patch seems to be the easiest solution without side effects. Other
choices are deleting files (seem like an non-ideal workaround), forcing a
repack (could be slow), and rewriting using Rust (could take too long).
The root cause is Python's `mmap` implementation has to keep a fd internally
to support `mmapobj.resize` API. We only need read-only operation on the
mmap object so the fd is unnecessary. Re-implement a minimal mmap interface
for this purpose.
Reviewed By: DurhamG
Differential Revision: D6835890
fbshipit-source-id: 74c429e957cb8677682604eb02fc38b5b8d13ef7
Summary:
When pushing a treeonly commit to a tree+flat hybrid server, the server
needs to execute the hooks in treeonly mode so we never try to access the flat
manifests. Previously we did this for shell hooks by setting some environment
variables, but we didn't do it for python hooks that ran in process.
This diff makes the python hooks run against a bundle repo that is instantiated
in treeonly mode. No changes to any hooks are required, as the repo object they
are given is already in the correct mode.
Reviewed By: quark-zju
Differential Revision: D6840971
fbshipit-source-id: 9fcb97d972076911b35bccf3f79b60972bcafc33
Summary: treedirstate: rename of file and class
Reviewed By: zertosh
Differential Revision: D6716409
fbshipit-source-id: 83d40103a13771773176c333f1287011d3a42f40
Summary:
[infinitepush] Implement support for short hashes
Also, fix test that was broken and lint errors
Reviewed By: StanislavGlebik
Differential Revision: D6814967
fbshipit-source-id: fef1ae79ef85ebf84aa88cd99367946256049192
Summary:
If `rebase.experimental.inmemory` was enabled, but you ended up merging on-disk (e.g. you rebased the working copy parent), and you got merge conflicts, and didn't immediately solve them, hg would discard any work you did and restart the rebase unnecessarily.
The bug here is basically that `rbsrt.inmemory` field can be set to False later, because we make further calculations once we define the `rebaseset`. In that case `InterventionRequired` will get raised if the user didn't solve conflicts, but we'll catch it, thinking something went wrong "in-memory", and abort/rerun the rebase. I was too hasty in adding the "catch all exceptions and retry" block.
I solved this two ways -- first, move the rbsrt definition here and explicitly check the state in the Exception catch. Second, add an explicit catch for InterventionRequired and explain why with a comment.
Further refactoring of the rebase code will help here, I think, so we can consolidate all of this logic.
Reviewed By: DurhamG
Differential Revision: D6817683
fbshipit-source-id: eb8baba86ec0deb59a1e0b1cf9ffb02f056ecd92
Summary: Remove the unused `os` import that I somehow missed in D6818952.
Reviewed By: ryanmce
Differential Revision: D6819544
fbshipit-source-id: 072be0a956a6ac2bbe96f72c6fc3c79907d22c9a
Summary:
Add a new `-d` option to `hg absorb`, which includes in the `-p` output the first 50
characters of the first line of the commit description for any commit that will
be absorbed into.
The width is configurable through `absorb.maxdescwidth`.
Reviewed By: ryanmce
Differential Revision: D6819341
fbshipit-source-id: 1a007aa5c15c047563b602559b338e5548bf1989
Summary:
There are a couple of ways that dirstate and fsmonitor interact with
treedirstate that are sub-optimal.
When iterating over all files in the dirstate map, use the iterator of keys
(`__iter__` rather than `iteritems`) to hit the treedirstate fastpath that
doesn't build the dirstate tuples.
Use `__getitem__` and catch the `KeyError` rather than `__contains__` followed
by `__getitem__`, as the latter involves two look-ups in the tree.
Skip the indirect call to `hastrackedfile` and `hasremovedfile` as these add
noticable overhead when called many times.
Also fix up the `test-check-code` failure for treedirstate.
Reviewed By: ryanmce
Differential Revision: D6818952
fbshipit-source-id: d49907c282a1b97c66a3d24257bdf54800d7da70
Summary:
Adds some basic building blocks to build hg using buck.
Header files are cleaned up, so they are relative to the project root.
Some minor changes to C code are made to remove clang build
warnings.
Rust dependencies, fb-hgext C/Python dependencies (ex. cstore,
mysql-connector), and 3rd-party dependencies like python-lz4
are not built yet. But the built hg binary should be able to run
most tests just fine.
Reviewed By: wez
Differential Revision: D6814686
fbshipit-source-id: 59eefd5a3ad86db2ad1c821ed824c9f1878c93e4
Summary:
This fixes lfs upload by avoiding use of workers. The perfect fix
would be to make handler work with the workers and multithreading, but that
is much more complex. The downlaoding has been working fine for 1.5 months now
without any reported issues so leaving that to use workers.
Reviewed By: quark-zju
Differential Revision: D6804951
fbshipit-source-id: d5ce92b35c5cbcd64174e47df7a19507c98ed64a
Summary:
pull in the following revisions from the copy of this code
we had under fbsource/scm/hgext:
2f7e4f11e002cf33e4878df77d6a0472adf31245 D6099388
e2a5a711e36c7392129b8753bea37c89a5d73a9c D6099754
77b975dcde28cd7c3d4ae2302bddb625682d1994 D6099753
Reviewed By: simpkins
Differential Revision: D6792967
fbshipit-source-id: e91a74329cddaf322172d3c7d9e1a05b3b6cba02
Summary:
Many processes may be reading and writing from this cache at once, so
let's be sure to use atomic temp.
Reviewed By: quark-zju
Differential Revision: D6811428
fbshipit-source-id: 5c6c157a60e294ab77cc5cb735c5705b7cddff87
Summary:
unbundle should use raw revisions and keep flags as-is,
instead of using high-level `filelog.add` API.
Reviewed By: ryanmce
Differential Revision: D6806031
fbshipit-source-id: 3e1819a91ee869ac8023eefb3f3aa7542f770539
Summary:
We're seeing high load on the server when many clients are fetching treemanifest
entries. A lot of this stems from the cost of opening revlogs. Let's introduce a
simple on-disk cache that let's us avoid opening revlogs as much as possible.
Reviewed By: quark-zju
Differential Revision: D6797698
fbshipit-source-id: c131aa57be333dcb2e4453c6a1f9f4e24532a7ee
Summary: Pyflakes complains it's not used, but it's actually a reference to the native class. Make that obvious.
Reviewed By: quark-zju
Differential Revision: D6799935
fbshipit-source-id: 16060839385ab52d53e43b9b4b202b5771d01dc2
Summary:
Port of D6798134 to fbsource. It eliminates module-import failures as well as errors like this:
```
mercurial.error.ForeignImportError: hgext.extlib.treedirstate: /home/phillco/.local/lib/python2.7/site-packages/hgext/extlib/treedirstate.so lives outside /..../hg
```
....that block other tests, like test-help.t
(Note: this ignores all push blocking failures!)
Reviewed By: quark-zju
Differential Revision: D6799259
fbshipit-source-id: b77d1b565dbf52165e0847002be498648658e064
Summary:
`hg version`, as of the last path, currently prints:
```
Mercurial Distributed SCM (version 20180123_174746_5cb750b72031)
Facebook Mercurial release: UNKNOWN-RELEASE
```
There's no need for us to print the second line now that the first represents the internal version ID.
(Note: this ignores all push blocking failures!)
Reviewed By: quark-zju
Differential Revision: D6794821
fbshipit-source-id: 499828832a0acb066adc260f852c3a743e41a563
Summary:
When used with hggit, simplecache would treat all hggit overlayctxs as having the `nullid` node. This is used in the cache key for the result of `status()`; this caused the return value of all hggit's changectxs' `status()` to be intermingled.
There are arguably two issues here:
- hggit's `overlaychangectx` simply implemented `node()` and didn't set a `_node` like changectx does. Changectx refers to this property internally (and other external callers might do so too) so we should probably set it.
- simplecache was reading the `changectx._node` property instead of the more public `changectx.node()`. This returned `nullid` because of the above.
Either change would fix it but I sent both to be safe. I look forward to tightening the context class interfaces in the future.
(Note: this ignores all push blocking failures!)
Reviewed By: quark-zju
Differential Revision: D6788335
fbshipit-source-id: eb4b3cebc1b6ee6011405cdb7d702cdf5cac0904
Summary:
Same fix as D6788335, though not blocking anything. Use the public `node()` function
on contexts and exclude None, nullid, and wdirid from caching.
Reviewed By: DurhamG
Differential Revision: D6790845
fbshipit-source-id: 4ccdc6889c993bb1a8379d50ecc92cb2aa03513d
Summary:
This fixes an obvious mistake. It wasn't noticed because it's part of the last
"cleanup" before landing the change. And it runs on a big repo which hides
the problem.
On smaller repos, changelog is inlined and clindex won't be used because the
Rust index does not support inlined revlogs. `repo.updatecaches()` will raise
`AttributeError` since the original `self.changelog.index` object does not have
`updatecaches()`. In that case, it's obvious that we should not skip the
original `repo.updatecaches()` call. The same applies to `repo.destorying()`.
Since after the fix, tests are passing. Let's enable clindex by default. As
explained above, clindex is not actually well exercised because of inlined
revlog. We should probably special case changelog to be always non-inlined.
That's probably a separate diff.
Reviewed By: phillco
Differential Revision: D6767200
fbshipit-source-id: c8432eec6c02c06c4d49c13df150e1b25bbaa1aa
Summary:
This extension can be a bit noisy when defaulted-on. Let's disable its debug output in
tests (except fot its own tests).
Differential Revision: D6777765
fbshipit-source-id: e2a6c6b7527336a1fac24078d2b49eb1e75ecabf
Summary:
The keyword extension is rarely used. And it breaks sshaskpass because
Python standard library `collections` (which does not use absolute import)
will import the wrong `keyword` module:
$ hg pull ssh://localhost/foo --debug
pulling from ssh://localhost/foo
running SSH_ASKPASS=/tmp/tmpj_6pDJttysrv/2991364.sh DISPLAY=localhost:10.0 TTYSOCK=/tmp/tmpj_6pDJttysrv/2991364 ssh -oControlMaster=no localhost 'hg -R foo serve --stdio'
sending hello command
sending between command
remote: Traceback (most recent call last):
remote: File "/usr/lib64/python2.7/site-packages/hgext/sshaskpass.py", line 43, in <module>
remote: import collections
remote: File "/usr/lib64/python2.7/collections.py", line 10, in <module>
remote: from keyword import iskeyword as _iskeyword
remote: File "/usr/lib64/python2.7/site-packages/hgext/keyword.py", line 93, in <module>
remote: from mercurial.i18n import _
remote: File "/usr/lib64/python2.7/site-packages/mercurial/i18n.py", line 15, in <module>
remote: from . import (
remote: File "/usr/lib64/python2.7/site-packages/mercurial/encoding.py", line 15, in <module>
remote: from . import (
remote: File "/usr/lib64/python2.7/site-packages/mercurial/error.py", line 17, in <module>
remote: from . import pycompat
remote: File "/usr/lib64/python2.7/site-packages/mercurial/pycompat.py", line 22, in <module>
remote: import cookielib
remote: File "/usr/lib64/python2.7/cookielib.py", line 32, in <module>
remote: import re, urlparse, copy, time, urllib
remote: File "/usr/lib64/python2.7/urlparse.py", line 123, in <module>
remote: from collections import namedtuple
remote: ImportError: cannot import name namedtuple
Reviewed By: singhsrb
Differential Revision: D6766686
fbshipit-source-id: af67a4d8c736158951c737b790904ba11419d432
Summary:
Added options supported by diff:
```
-w --ignore-all-space ignore white space when comparing lines
-b --ignore-space-change ignore changes in the amount of white space
-B --ignore-blank-lines ignore changes whose lines are all blank
-Z --ignore-space-at-eol ignore changes in whitespace at EOL
```
Differential Revision: D6738122
fbshipit-source-id: bd4236ce7cb9d5f803f932cfec267de71b0067c2
Summary:
Some consumers still rely on enabling 'sparse=' so let's add a module
that just redirects to the real fbsparse.py.
Also updates configerator to use the newer name.
Reviewed By: markbt, quark-zju
Differential Revision: D6755971
fbshipit-source-id: 3a67f029045dacf927742a616a714fe632b97fea
Summary:
The isgooddelta tweak was introduced in D2693043 (perftweaks: change revlog
delta heuristic, 2015-11-24). Comparing with the existing version, the only
change is that we removed `dist > maxdist` check.
Note that the upstream commit 895ecec31 (revlog: add an experimental option
to mitigated delta issues (issue5480), 2017-06-23) also introduces a config
option to override `maxdist` to make the condition fail, which basically does
a same thing.
Instead of introducing new config options or adding more "if"s to the
codebase to make it more obscure, let's just simplify it by disabling the
check entirely, and removing the `dist` concept, removing two config
options: `experimental.maxdeltachainspan` and `perftweaks.preferdeltas`.
The `chainlen > self._maxchainlen` check should be enough for keeping
delta chain length bounded.
Reviewed By: DurhamG
Differential Revision: D6752529
fbshipit-source-id: e8fd8ec39240191db5fb274190fc661e97087a78
Summary:
This enables clindex for its nodemap. Verification is turned off by default
for the performance win since we have been running verification in
production for a while.
Reviewed By: phillco
Differential Revision: D6751412
fbshipit-source-id: bc3e87df86e86a758392bdd4aef3e282f397fe04
Summary: Also change the internal API so it no longer accepts the "heads" argument.
Reviewed By: ryanmce
Differential Revision: D6745865
fbshipit-source-id: 368742be49b192f7630421003552d0a10eb0b76d
Summary: This removes the effectflag logic from both core and perftweaks.
Reviewed By: ryanmce
Differential Revision: D6745769
fbshipit-source-id: 55ed1676e7117bca358471c256805ded7bc83f3c
Summary:
It's pretty handy. The implementation is simple and clean. So move it from
contrib and enable it by default. "import"s are adjusted to make the module
checker happy.
Security and UX wise, since we have `--debugger` already, adding another
REPL seems fine.
Test Plan: Ran all tests
Reviewers: phillco, #mercurial
Reviewed By: phillco
Differential Revision: https://phabricator.intern.facebook.com/D6741283
Signature: 6741283:1516225662:ddc19a663e7ecef2a1fdaa5041f308dc838a8471
Summary:
`lfs` is the better large file solution. `largefiles` is rarely used, and
its implementation is less clean. So let's remove it.
Test Plan:
Ran all tests. A subrepo test was removed instead of cleaned up since the
longer term plan is to also drop subrepo support.
Reviewers: phillco, #mercurial
Reviewed By: phillco
Differential Revision: https://phabricator.intern.facebook.com/D6740361
Signature: 6740361:1516225594:555e3803571ad05e0434021897a2823ac99347ae
There's no need to do this and not doing so speeds up IMM tremendously when the sparse profile had to be adjusted.
(This is the grafted version from the fb-hgext repo)
Test Plan:
I ran an IMM before and after this change. Both succeeded but the later version was
much faster.
Differential Revision: https://phab.mercurial-scm.org/D1805
Summary: This is a quick way to ensure that if IMM goes wrong in a way that isn't conflict related (e.g. an uncaught exception) we restart and log to scuba.
Test Plan: ./run-tests.py
Reviewers: durham, quark, #mercurial
Reviewed By: durham, quark
Differential Revision: https://phabricator.intern.facebook.com/D6734117
Signature: 6734117:1516152432:5620b12579d41dca2ea8f15b08a8007e2f6467fc
