Summary:
btrfs has a bug involving truncate not returning the correct error
code. We already handle it in other cases, so let's extend it to these uses of
truncate. Hopefully the need for this disappears soon.
Reviewed By: quark-zju
Differential Revision: D8152297
fbshipit-source-id: f55602ff5e0ec36346c547bfd4b6d0f6e4127500
Summary: Mostly empty lines removed and added. A few bugfixes on excessive line splitting.
Reviewed By: quark-zju
Differential Revision: D8199128
fbshipit-source-id: 90c1616061bfd7cfbba0b75f03f89683340374d5
Summary:
We're seeing "Permission denied" errors on some of our automation, but
it's difficult to track down where it's coming from. Let's make each message
more descriptive.
Also updates, hgsql to handle the hg-rsh hook, which I noticed while
investigating.
Reviewed By: phillco, farnz
Differential Revision: D8188414
fbshipit-source-id: 5f8c99e8ba896c2636b1a04716125bc6a9df0591
Summary:
Add a new debug command to check whether any of the provided files
casecollide with any file in a revision's manifest.
Reviewed By: quark-zju
Differential Revision: D8165659
fbshipit-source-id: 9315ff052c9996888202961d168d20b834c22834
Summary:
Add an internal `get_dir` API to return aggregated states. It is exposed via
`.get('dir/')` python interface.
This is useful for implementing `hastrackeddir` of the dirstatemap class.
Reviewed By: markbt
Differential Revision: D7909173
fbshipit-source-id: 100a8f36237a6b911a4bfb4afbb4c63b98611317
Summary:
Going to make changes to `mercurial/` for cleaner fsmonitor support
directly. So let's move the Rust python bridge there first.
Reviewed By: markbt
Differential Revision: D7909174
fbshipit-source-id: 454d784b5dca18a3af9328fc7b2f342cd4188cf6
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
Summary:
The clang-format file in scm/hg basically only applies to things in the
scm/hg/mercurial directory.
There are 180 C and C++ files under scm/hg, and the vast majority of them do
not follow the style specified in this clang-format file. All but 11 of these
files were present in scm/hg/contrib/clang-format-blacklist.
Of the 11 files that do follow this style, 10 are in the scm/hg/mercurial/
directory. (10 of the 21 files in this directory use this style.) The 1
other file is in scm/hg/contrib/xdiff.
The majority of the C/C++ files in scm/hg/hgext/extlib and scm/hg/lib/ follow
a style closer to Facebook and Google's C++ style guidelines.
Therefore this moves the .clang-format file to scm/hg/mercurial, and lets the
main fbsource clang-format file apply to the other files under scm/hg
Reviewed By: quark-zju
Differential Revision: D8131512
fbshipit-source-id: 622a33abc39eb240eff4ca27f69a675a7ed54a89
Summary:
Add the filestat template keyword, which expands to a list of file status
information for each file modified in a commit.
Reviewed By: mitrandir77
Differential Revision: D8161706
fbshipit-source-id: 1dc41acebbcb081581f6b227facc8228375a320e
Summary:
Make template keywords that expand to lists of things that aren't strings, and
don't have a template to render the items in the list, render as the number of
items.
Reviewed By: mitrandir77
Differential Revision: D8161705
fbshipit-source-id: 19ca1cba88c0ce75c0ba358cd7e6f27c7ac61c34
Summary:
Checkheads is a legacy feature that is less useful in our setups, namely:
- In commit cloud / Mononoke's world, it's intentional to have many heads.
Pushing a new head is a normal operation that should not be forbidden.
- With remotenames, remotenames performs the check and checkheads is
redundant, as shown by the added test.
So let's add a config option to turn it off first. Later we can remove the
feature and update all the tests.
Reviewed By: ryanmce
Differential Revision: D8148016
fbshipit-source-id: 71684f20b9ca37902440eae331292679b0feb4c6
Summary:
Make it show "added", "removed", "changed" so it looks similar to the
original commit template.
Reviewed By: singhsrb
Differential Revision: D8158006
fbshipit-source-id: 1a1578bd263e7870c27e7169537e808ffd8add9f
Summary:
Generate a `u64` integer about the "version" at build time, and make chg
client check the version before connecting to the server.
This would ensure a chg client would only connect to a matched version of
the server.
- In setup.py, compute the "versionhash", write it as
`mercurial.__version__.versionhash`.
- In dispatch.py, `mercurial.__version__` needs to be explicitly loaded
before forking.
- In commandserver.py, send the versionhash to the client with the "hello"
message.
- In chg.c, verify the versionhash. If it does not match, unlink the socket
path and reconnect.
Reviewed By: farnz
Differential Revision: D7978131
fbshipit-source-id: 50acc923e72e40a4f66a96f01a194cf1a57fe832
Summary:
Mercurial help syntax requires that the line is the same length as the line it
is under.
Reviewed By: mitrandir77
Differential Revision: D8140787
fbshipit-source-id: 460b6e1503d7a42f35b8205376ec6819144d9c0b
Summary:
In previous diff we've added the ability to log wireproto requests.
However getfiles wireproto request is special, because it sends arguments in a different way.
Reviewed By: farnz
Differential Revision: D8097782
fbshipit-source-id: f598acd155e842e90d14a295fc71b1886e5a7fb0
Summary:
Make it possible to log wireproto requests. It will be used by Mononoke to
replay the traffic sent to hg. It can also be used to analyze what expensive
wireproto requests we have.
Reviewed By: farnz
Differential Revision: D8042236
fbshipit-source-id: 7b2b762a51bcbfad30309d15e58afd99618b7af9
Summary:
When reverting to non-parent revision, the reverted files should be marked
as "need content check" anyway. The code should not depend on the fact the
mtime of the reverted files is usually "fsnow".
Reviewed By: DurhamG
Differential Revision: D8123136
fbshipit-source-id: d602ee06408701c0190c142f1ea4e0ff6247fe62
Summary:
Improve `makedirs` error message so it would be more helpful when there are
broken symlinks.
Differential Revision: D8108794
fbshipit-source-id: 08013022642efde946ef9d5c6b06b4763f4ad68f
Summary:
This will be used by the next change testing whether `{files|count}` is
exceeding a threshold for deciding which commit template to use.
Reviewed By: phillco
Differential Revision: D8101518
fbshipit-source-id: 51e918c6d8ab7d6e8b71708d9291945b2a09a632
Summary:
D5521665 added a `{stat}` template that can be used to show lines changes.
That gives more information than a plain template with just filenames, and
could serve as a hint about whether to split a diff or not.
Reviewed By: farnz
Differential Revision: D7960550
fbshipit-source-id: 5cb151b5d7ff72ce6260a7a76f15d7c17bd3bbd4
Summary:
This splits out the logic in the `fsmonitor` extension that is responsible for
publishing `hg.filemerge` and `hg.update` state changes to Watchman into
its own extension, `hgevents`. This is because we want the behavior of
`hgevents` when Hg is running in Eden, but we do not want the remaining
behavior of `fsmonitor` when Hg is running in Eden, so splitting the logic
into separate extensions is the most straightforward way to achieve that.
To achieve the split, we move some more logic that is common to both
`fsmonitor` and `hgevents` out of `hgext/fsmonitor/__init__.py` and into
`hgext/extlib/watchmanclient/__init__.py`. Then we move these lines
out of `extsetup()` in `fsmonitor` to create `extsetup()` in `hgevents`:
```
extensions.wrapfunction(merge, 'update', wrapupdate)
extensions.wrapfunction(filemerge, '_xmerge', _xmerge)
```
We also have to pull all of the transitive dependencies for this logic
into `hgevents`.
Finally, we also have to define a `reposetup()` function in `hgevents`
that does a subset of what `reposetup()` does in `fsmonitor`. Specifically,
it ensures that a Watchman client is created for a `repo`, as appropriate,
so that it can be used to dispatch state changes to Watchman in
`state_filemerge` and `state_update`.
Note that the utility functions `createclientforrepo()` and
`getclientforrepo()` have been added to ensure that only one
Watchman client is created (and shared) when both `fsmonitor`
and `hgevents` are enabled.
Today, when an Hg repo is created in Eden, we set `extensions.fsmonitor=!`
in the `.hg/hgrc`:
diffusion/FBS/browse/master/fbcode/eden/hooks/hg/post-clone.py$69
Therefore, to get existing repos (both Eden and non-Eden) to pick up
the `hgevents` extension automatically, we add it to the list of
`[extensions]` in `common.rc`:
diffusion/FBS/browse/master/fbcode/scm/hg/fb/staticfiles/etc/mercurial/repo-specific/common.rc$53-60
as this is where `fsmonitor` is configured. We do not enable it in
`scm/hg/fb/staticfiles/etc/mercurial/facebook.rc` because
there is no reason to enable `hgevents` on Hg servers. Therefore, we
also decline to add `hgevents` to the set of `DEFAULT_EXTENSIONS` in
`scm/hg/mercurial/extensions.py`.
Reviewed By: quark-zju
Differential Revision: D8003628
fbshipit-source-id: 4f23881f8c25f4638f5475c292537b0352ae8d15
Summary:
wjb has seen some weird truncate issues, which is caused by `ftruncate (2)`
returning 1 while its manpage says it returns <= 0. The recent contbuilds
also failed due to this.
While we're waiting for the upstream fix [1], let's workaround it from the
userspace to unblock the contbuild.
[1]: https://www.spinics.net/lists/linux-btrfs/msg78417.html
Reviewed By: DurhamG
Differential Revision: D8064611
fbshipit-source-id: df6c44255de47efa25a4c2b713a159ecd61b7478
Summary: Didn't realize it was possible to land with a failure.
Reviewed By: DurhamG
Differential Revision: D8065037
fbshipit-source-id: abc34efeb4898f2068e57691adb0ddec8a64eabd
Summary: Ran into an ImportError with the old layout.
Reviewed By: quark-zju
Differential Revision: D8061476
fbshipit-source-id: 6fc0b0c3a426e49eb5a43d59e13c69efd19fd453
Summary:
Add a new config section, `[hint-definitions]` that describes additional hint
messages. This can also be used to override built-in hint definitions.
Add a new template function `triggerhint(name)` that triggers the given hint.
Reviewed By: quark-zju
Differential Revision: D8056847
fbshipit-source-id: 5ffc945343133eb635ae0820190ecad9f16bc731
Summary:
This is a precursor to splitting the fsmonitor extension, as both
it and the new extension will use pywatchman.
Reviewed By: quark-zju
Differential Revision: D8002713
fbshipit-source-id: 37983fe2898d23223d1178eb3f15685f17ff8868
Summary:
Adds rust bindings around the existing mpatch c library.
Also fixes a bug in mpatch where it could reference uninitialized memory.
Reviewed By: quark-zju
Differential Revision: D7769299
fbshipit-source-id: bcc21df85c97ef6f5537ebff8fbf1b350ee64fc3
Summary:
This could be useful for implementing logic like "if this commit is by the
current user, show something differently".
Reviewed By: markbt
Differential Revision: D7964292
fbshipit-source-id: de1ac0b5edde838dbaae646a88ebf636b4925b22
Summary:
`pkill -f chgunix` is a convenient, but not a safe way to invalidate chg
servers. We're going to do the server invalidation in a better way. So just
revert the change.
Reviewed By: singhsrb
Differential Revision: D7978130
fbshipit-source-id: c13b75204ae1097ffe992b2e26d80d028022ff0d
Summary:
hgsql depends on repositories being byte for byte identical, but the
current pull after a streaming clone can cause the repository to be different
(like if different delta decisions were made, or the commits were ordered
slightly differently). Let's disable that pull when the repository is an hgsql
repo.
Reviewed By: ryanmce
Differential Revision: D7925300
fbshipit-source-id: 6eba7ad4ccdd37f6d7c5090522867d1a54f722b7
Summary:
The "sensitive config sections" was used because "hg serve" loading
different extensions are incompatible with each other. Now we neither load
extensions nor run their uisetups, and just use one chg server. So
sensitive config sections can be removed.
Reviewed By: singhsrb
Differential Revision: D7847149
fbshipit-source-id: 758c1df21d280bf0f88d91432e1201c8417df532
Summary:
`pkill -f chgunix` was used to kill chg server processes. That could kill
worker processes if the worker hasn't changed its process name. Let's change
the process name early after forking to reduce the window a chg worker can
be killed by `pkill -f chgunix`.
Reviewed By: singhsrb
Differential Revision: D7845335
fbshipit-source-id: 3b3fc64c4328a058e0c6aad3cb11b0bc4efd9d13
Summary:
It was used to reduce process count on laptops. Now chg server can be
reused across different `--config`, `HGPLAIN` settings, it's no longer
necessary to keep the hack.
Reviewed By: singhsrb
Differential Revision: D7847150
fbshipit-source-id: d4f98debb5e9eb4f2c3f8575532cc833c61e4b1d
Summary:
This makes it further harder to have multiple chg servers.
- MODULEPOLICY: We effectively only support 'c' now.
- LD_: It might be intentionally to trace chg server but not the client.
Assuming the user knows what they are doing.
- PATH: It's rare that `PATH` change can cause issues.
- PYTHON: Assume the current Python works, it's reasonable to not start a
new Python process.
- PYTHONPATH: Since we now bundle most extensions, side effects on
extensions should be considered rare.
Reviewed By: singhsrb
Differential Revision: D7845333
fbshipit-source-id: 264d67b687173fb4f2bdef1ef45937d8f098ed3d
Summary:
Previously, i18n are initialized at module import time, and cannot be
reinitialized if HGPLAIN is changed. That makes chg server has to hash
HGPLAIN, and use different servers if HGPLAIN is set.
Change i18n to be only initialized at `dispatch._dispatch` time, after chg
server updating environment variables. So a same chg server could serve
different languages or encodings.
Reviewed By: singhsrb
Differential Revision: D7845334
fbshipit-source-id: 6fba19cc07efdfa60a0e328cf2cc981cfed4bcc8
Summary:
Pre-import modules so when they are imported by `extensions.py`, they will
get a cache hit.
Note: use a dict explicitly since it's much faster than re-`import`:
In [2]: %timeit import hgext.rebase
The slowest run took 13.43 times longer than the fastest. This could mean that an intermediate result is being cached.
1000000 loops, best of 3: 444 ns per loop
In [3]: d=sys.modules
In [4]: %timeit d['hgext.rebase']
10000000 loops, best of 3: 38 ns per loop
In [7]: %timeit sys.modules['hgext.rebase']
The slowest run took 31.89 times longer than the fastest. This could mean that an intermediate result is being cached.
10000000 loops, best of 3: 67.3 ns per loop
Even importing two modules and constructing a tuple is 2.5x faster than directly
"import" one module:
In [8]: %timeit (sys.modules['hgext.rebase'], sys.modules['hgext.absorb'])
The slowest run took 10.78 times longer than the fastest. This could mean that an intermediate result is being cached.
10000000 loops, best of 3: 177 ns per loop
Reviewed By: singhsrb
Differential Revision: D7840236
fbshipit-source-id: ca351c61ec63ffdaf401e561944e97963b434b3c
Summary:
Motivated by recent D7784903 which kills chg because it holds blackbox.log
file descriptor, and that patch is causing race conditions running with chg
(chg's sock atomic rename might fail if the directory is deleted).
There are other ways to solve the direct issue. This diff takes a more
aggressive but much cleaner approach. Basically, the `hg serve` framework is
too late for chg's usecase - the repo was already loaded, extension
side-effects have been already done at that time - chg has to use
workarounds to be compatible with that. Even with a best effort, it is still
possible to have weird interactions with shared repo because how hg loads
extensions.
The new approach is to pre-import a list of bundled extensions but do not
run their `uisetup`s. This solves a couple of hard problems:
- Compatibility - `uisetup` runs per request. That behaves exactly as what
an extension author expects.
- Less memory usage - there is no `repo` object is loaded in memory.
- Reduced process count - since extension config change does not require a
new chg server, the count of server processes would decrease (ex.
`--config extensions.blackbox=!` won't require a new chg server)
- Not holding fd to edenfs, since neither blackbox nor repo is loaded. This
makes it possible to remove the hacky killing chg logic in D7784903.
The downside is performance, since extension loading, and `uisetup` will run
every time. Benchmark shows that's could be 50ms-ish. But we could move
forward by moving extension logic to core incrementally to get rid of that
cost too.
This is basically a simplified version of my previous stack starting
with [1]. The original commit message was:
This is the beginning of a series that does a re-architect to chg mentioned
in [1], to achieve better compatibility.
The compatibility issues are mainly around "uisetup"s and "reposetup"s:
- Developers are usually unaware that uisetup runs only once per chg
process. We cannot reliably devel-warn them. The result is, potential
broken code are written. For example, it's really hard for chg to deal
with "experimental.evolution" changed from unset to manually set in
config files because setconfig is used if that config option is not set.
- An unnecessary "reposetup" caused by "hg serve" may have unwanted
side effects. This can become troublesome if the repo requires things
like remotefilelog or lz4revlog, and the user sets HGRCPATH to run
tests.
The current chg implementation assumes that "loading" an extension is not
side effect free - if extension related config has changed, a restart is
needed. The new idea is, "loading" = "importing" + "run ui/extsetup", the
"importing" part can be side-effect free for some extensions. And benchmark
shows "import" takes most of the time consumed, while "uisetup" is usually
very fast. We can afford running "uisetup"s per request.
To be able to (pre-)"import" extensions without running any "uisetup"s, a
different entry point is needed. Otherwise as long as we go through the
normal dispatch / runcommand ("hg serve") flow, "uisetup"s cannot be
avoided.
Aside from better compatibility, we can also remove some hacks:
- chg client: no longer needs to extract sensitive argv
- chg server: confighash can be changed to only hash environment variables
(reduce the number of server processes)
- chg server: srcui.walkconfig hack is no longer necessary
This patch adds a new script "chgserve" as the new entry point. Currently,
it is just a minimal implementation that makes "CHGHG=chgserve chg ..."
work, without doing any pre-importing. The change could also be done in the
"hg" script. But since chg is still experimental, let's keep "hg" untouched
for now.
[1]: www.mercurial-scm.org/pipermail/mercurial-devel/2016-July/085965.html
[1]: 6f91a1a69f
Reviewed By: singhsrb
Differential Revision: D7840237
fbshipit-source-id: e3d613b41fe4b721238b86c5bf84434d32cf0609
Summary:
This is an attempt to fix the wrong status issue. A formal fix would be
D7909172, which merges fsmonitor.state into treestate. See the comment in
`__init__.py` change for why status can be wrong.
Although it fixes the "status" path, I'm not sure whether other code paths
writing dirstates could be problematic or not. Anyway, D7909172 should be the
preferred final fix.
Reviewed By: r4-in
Differential Revision: D7916344
fbshipit-source-id: beab1c825b970bb47ffe03617a14eb6f203feafa
Summary:
This helps to avoid the following problem:
1. hg creates a temporary lock file, writes some stuff there
2. os writes this stuff into its buffer
3. hg closes the file, the metadata is written out (or journaled)
4. hg renames the file, which is again a metadata-only operation
5. the buffer is still not flushed
6. the OS crashes
7. upon reload, the os has a file with a correct name and a correct length,
but unexpected contents
Reviewed By: quark-zju
Differential Revision: D7889111
fbshipit-source-id: a0a152c9e7efef34847fa2d2ab9b94191bde43f4
Summary:
While establishing an ssh connection, ssh may need to interact with the user
(e.g. to collect passwords). Suspend the progress bar for the duration of
this, so it doesn't interfere.
Reviewed By: quark-zju
Differential Revision: D7876414
fbshipit-source-id: 5fa82f0f40fcffa6b94fa0210d102c76d3618a1d
Summary:
Add generalised support for subcommands. This is similar to the monkey-patched
version in `fbsparse`, but fully supported by the command infrastructure.
Subcommands are the same structure as normal commands, but are attached to a
table in the `subcommands` attribute of the main command. Normally, if no
subcommand is provided, the normal command function is called. This can be
made into an error by setting `subonly` on the top-level command.
In order to make `fbsparse` continue to work, I've temporarily hacked how it
handles help text. This will be fixed in a later diff that switches fbsparse
to use this infrastructure.
Reviewed By: mjpieters
Differential Revision: D7849476
fbshipit-source-id: b988edabb17da77bf91a278e0faa2feecd0c1db9
Summary: D7840688 broke building RPM on OSX. This commit fixes the same.
Reviewed By: quark-zju
Differential Revision: D7854180
fbshipit-source-id: 4d3e4c87da777930780ad53888c13a41aab6c6e4
Summary: This is a hint for performing certain fast paths.
Reviewed By: mitrandir77
Differential Revision: D7818730
fbshipit-source-id: 4adcf8724b462d8d652e8e580d6a36eebc46a0f8
Summary:
`GetFileInformationByHandle` returns a `BY_HANDLE_FILE_INFORMATION` structure,
which is similar to what a `stat` call returns. In particular, this structure
contains:
- the `VolumeSerialNumber` field
- the `CreationTime` fields
- the `LastWriteTime` fields
- the `FileSize` field
- the `FileIndex` fields
All of these are self-explanatory, except for the `FileIndex`. Here's what MSDN says:
```
The identifier that is stored in the nFileIndexHigh and nFileIndexLow members is called the file ID.
...
In the NTFS file system, a file keeps the same file ID until it is deleted.
You can replace one file with another file without changing the file ID
by using the ReplaceFile function. However, the file ID of the replacement file,
not the replaced file, is retained as the file ID of the resulting file.
```
Basically, every change to a file, except replacing it with some other file,
results in a changed file Id. Calling `ReplaceFile` however results in
`CreationTime` preserved from the replaced file and `LastWriteTime` preserved
from the replacement file:
```
C:\Code\tries\windowstries
λ python fileinfo.py
1.txt: Attr;32;Create;4064609256;30663014;Write;3046340864;30663166;Volume;1792064959;Size;0;5;Idx;655360;547898
2.txt: Attr;32;Create;3030045984;30663166;Write;3030172944;30663166;Volume;1792064959;Size;0;5;Idx;786432;565725
Replacing 1.txt with 2.txt; result is: 1
1.txt: Attr;32;Create;4064609256;30663014;Write;3030172944;30663166;Volume;1792064959;Size;0;5;Idx;786432;565725
```
Thus comparing all of these fields seems to be enough to replicate the `cachestat` beharior from `posix.py` (We
cache the `stat` of a file, which we almost always expect to change by renaming into it. We only use this `cachestat`
while our process is alive. One notable exception is the `.hgignore` file, which the user can change as they please,
but which we still `cachestat`.)
This change has performance implications for `status` if we use `.hgignore`: it's nearly 0.1s faster.
If we use `.gitignore`, there are no performance implications (at least I did not find any), but I'd still like
to land it for the sake of feature parity between Posix and Windows.
Reviewed By: quark-zju
Differential Revision: D7843746
fbshipit-source-id: f6f69ee12bdce054d7ea77917e83a95bcec17f83
Summary: Next diff in the stack accesses it from `windows.py`.
Reviewed By: quark-zju
Differential Revision: D7843747
fbshipit-source-id: fa9458a3ac4e66013f61c92d8ebc41b0859d4e37
Summary:
Update the buck build rules to depend on the pyre2 third-party library, and to
try importing it using the module name `re2`.
Reviewed By: ryanmce
Differential Revision: D7840688
fbshipit-source-id: 21156958f42cdcf61f4dfdb2c6eccf95e657fcd1
Summary:
This would make tests run on treedirstate.
To avoid issues with Eden pulling from a non-eden treedirstate repo,
treedirstate is changed to be "always on" and disables itself on an eden repo.
The extension list is changed to a set for efficient `__contains__` test.
Reviewed By: phillco
Differential Revision: D7769804
fbshipit-source-id: d328fe51ef67c4730cfc53f43bdfc48c2765c541
Summary:
"\n" will flush the line and that would probably solve the OSX test failure.
```
--- tests/test-clone-uncompressed.t
+++ tests/test-clone-uncompressed.t.err
@@ -37,8 +37,8 @@
> EOF
$ hg clone --stream -U ssh://user@dummy/server blockedclone
streaming all changes
- remote: unable to perform an implicit streaming clone - make sure remotefilelog is enabled
abort: locking the remote repository failed
+ remote: unable to perform an implicit streaming clone - make sure remotefilelog is enabled (no-eol)
[255]
$ hg clone --stream --config clone.requestfullclone=True -U ssh://user@dummy/server blockedclone
streaming all changes
ok
```
Reviewed By: DurhamG
Differential Revision: D7776998
fbshipit-source-id: f21c26e1bf7aa547cd79892f66521fb27cb2e77f
Summary:
We want to allow blocking full repo streaming clones in certain
repositories (since they can take the lock and take a very long time) unless the
client has explicitly asked for it. The existing stream_out wire protocol has no
way of passing an option, so let's create a new endpoint.
Reviewed By: quark-zju
Differential Revision: D7763717
fbshipit-source-id: eace47143f8fdcc4c6e302b5c26678ccf56ca5d4
Summary:
This logs timing data for:
- update
- merge.update
- mergedriver
- applyupdates
- workers
- fetches
- pulls
- remotefilelog prefetches
- treemanifest prefetches
- status
- dirstate walks
- fsmonitor walks
- watchman queries
Hopefully this will let us narrow down where time is going in a number of cases
where the profile data is ambigiuous or hard to come by.
Reviewed By: quark-zju
Differential Revision: D7681026
fbshipit-source-id: e6fe65c9a4d2f4e128f62ccb767a7cbe73b2649a
Summary: A stub class in metrics.py can be overwritten by dedicated implementations.
Reviewed By: quark-zju
Differential Revision: D7673553
fbshipit-source-id: f713abb3203d393e356f96fb834111ec2c37498a
Summary:
In an upcoming diff I want to add more timing measurements for various
parts of the Mercurial code (like how long status takes, vs checkout, vs
prefetch, etc). Let's rename the logblockedtimes logic to be more generic, since
it is doing basically the same thing.
Reviewed By: singhsrb
Differential Revision: D7676406
fbshipit-source-id: 9aa8c90ce562fa3ad5b654f7b3191b2c16a440c2
Summary:
Some services like sandcastle want to be able to correlate their job
with specific hg commands in our scuba data. Let's allow them to specify a job
id we can filter on.
I considered having multiple columns, like job type and job id, but I figured
the service can use whatever format they like, and we can used derived columns
to filter on different parts of the string as needed.
Reviewed By: mjpieters
Differential Revision: D7672264
fbshipit-source-id: e4723274bad2812358176e71da791e759b346ac0
Summary:
The document was for ".title()". The code finally uses ".capitalize()" so
it's not capitalizing each word.
Quoting is changed to be consistent with the surrounding style.
Reviewed By: ryanmce
Differential Revision: D7658780
fbshipit-source-id: 4ad99d41cd8104dd382058a50752f88aa2116a0d
Summary: Let's report that this lock cannot be read by Mercurial
Reviewed By: markbt
Differential Revision: D7653196
fbshipit-source-id: c5b7889cdde9c0ecc03a8c961aeba92f426648b1
Summary:
Quick experimentation shows that existing lock file logic is not enoug for
frequently run and killed Mercurial processes (Mercurial run by tools, such as
Nuclide is an example of such scenario)
I wrote the following two files:
```
c:\Code\tries\pythontries λ cat lockcreator.py
import os, random
def makelock(info, pathname):
ld = os.open(pathname, os.O_CREAT | os.O_WRONLY | os.O_EXCL)
os.write(ld, info)
# os.fsync(ld)
os.close(ld)
name = os.path.join('locks', 'lock.pid' + str(os.getpid()) + ".rand" + str(random.randint(0, 10000)))
makelock('contents', name)
```
and
```
c:\Code\tries\pythontries λ cat lockracer.py
import os, subprocess, time, random
for i in xrange(10000):
proc = subprocess.Popen('python lockcreator.py')
time.sleep(0.001*random.randint(0, 500))
proc.terminate()
```
After runnning `python lockracer.py`, I did `ls -l locks | grep "0 Apr"`, this way it showed all the 0-byte files created in April. This shows a non-empty output. Uncommenting the `os.fsync` line does not help much.
Rewriting `lockcreator.py` to use temp lock file approach helps greatly.
Reviewed By: quark-zju
Differential Revision: D7653186
fbshipit-source-id: 48e9eeeca34075ea2ec78f3319491bcebc0e88c7
Summary:
This avoids potential SEGSEGVs when the system re2 is ABI incompatible.
See D7622774 for more details.
Reviewed By: markbt
Differential Revision: D7622771
fbshipit-source-id: bae1500e7468881a7a19d3cb9074db583b2444a6
Summary:
We recently found issues where it seems "hg" in some environment uses an
ABI-incompatible re2 library and segfaults. pyre2 is small and was
originally written by Facebook for Mercurial use-case. So let's just vendor
it.
Reviewed By: markbt
Differential Revision: D7622774
fbshipit-source-id: db01183c9881bf9c3ed21fceed195b2059d40208
Summary:
Suspending the progress bar is achieved by taking the progress lock. This has
a cost, which reduces performance even when there are no progress bars to
suspend. This is particularly bad for commands that output lots of lines, e.g.
`hg files`.
Don't take the lock during suspend when there aren't any progress bars to
suspend.
Reviewed By: mjpieters
Differential Revision: D7652389
fbshipit-source-id: 627d893261228087b4cc84b90bf05b034cc60a40
Summary:
Similar to the previous patch, this allows us to get more information about
conflicts being resolved.
Reviewed By: phillco
Differential Revision: D7648933
fbshipit-source-id: 5b8c3dde385ace4c2a7cff7455e11436e5afd8cd
Summary: Commit 23f4cd4264ef broke a bunch of tests. Let's back it out.
Reviewed By: singhsrb
Differential Revision: D7646736
fbshipit-source-id: ace9b7be2c50f64ea382eaa72a350ead67b2a2a6
Summary: Let's report that this lock cannot be read by Mercurial
Differential Revision: D7617178
fbshipit-source-id: 23f4cd4264ef6c685762d8f89e2c24ed2f224211
Summary:
The recent changes to Mercurial made us less tolerant to wrongly-formatted
lock files.
Reviewed By: farnz
Differential Revision: D7586684
fbshipit-source-id: d7fa2151b8bb20f49c83941f7d3ef751fdbab2de
Summary:
{F123775861}
Improve smartlog extension for Commit Cloud Users.
The difference with the current smartlog implementation is that in Commit Cloud world
a commit can be modified by a sequence of operations on another host and smartlog doesn't handle it.
After `hg cloudsync` run command we should be able to 'explain' in smartlog what has happened
this is important for user who opt out from 'updateonmove' feature because they will see two version of their stack all the time.
Reviewed By: DurhamG, quark-zju
Differential Revision: D7492969
fbshipit-source-id: 9ac2180f9abaa9ae596620b7f25d9ad8212deb28
Summary:
fsmonitor relies on `repr(match)` to check ignore changes. `__repr__` is not
defined by `negatematcher`. That leads to unnecessary fsmonitor state
invalidation.
This diff fixed that by defining `__repr__` on the base matcher class.
Reviewed By: singhsrb
Differential Revision: D7619774
fbshipit-source-id: df55390411cdb2d8e22e65efdeeb3b1db3bfa284
Summary:
Removes bundlev1 from the supported outgoing versions, but keeps a flag
around to force enable it if tests need it.
Reviewed By: quark-zju
Differential Revision: D7591176
fbshipit-source-id: 280cbbbe87848e3d6c9d448ce4f87c5eadeff720
Summary:
Bundlev1 is old and really shouldn't be used anywhere. Let's default to
v2.
Reviewed By: quark-zju
Differential Revision: D7591172
fbshipit-source-id: 2699e0b4dd8d1c1951f9dd92f0d9d300d935a04b
Summary:
We want to deprecate the bundlev1 format, so let's start by adding a
develwarn. Later diffs will update the tests to not use v1, then remove v1 as a
supported outgoing bundle entirely.
Reviewed By: quark-zju
Differential Revision: D7591166
fbshipit-source-id: 143ad029bfe4d141f91d6d5077342dfa44ad2944
Summary:
We would like to know how much time is spent setting up ssh connections. Add a
timeblockedsection to sshpeer that records the amount of time between starting
the ssh command, and getting (and validating) the response for the `hello` and
`between` commands.
Reviewed By: ryanmce, farnz
Differential Revision: D7584383
fbshipit-source-id: fd3c48dc57e0ebbafc191c235355ce2330c6bd61
Summary: We've seen a case when a malformed `wlock.break` file prevented the stale lock file from being deleted. It seems unsafe to just delete the `wlock.break`, but we can add more debug messaging before it, so that rerunning a command with `--debug` would tell us what is going on.
Reviewed By: quark-zju
Differential Revision: D7572510
fbshipit-source-id: 5456ae6dbff3721bbd40c6ed55e173beabac3f65
Summary:
Backport from:
```
# HG changeset patch
# User Yuya Nishihara <yuya@tcha.org>
# Date 1520766638 -32400
# Sun Mar 11 20:10:38 2018 +0900
# Branch stable
# Node ID eeb87b24aea7f547f6d95b812dd080dc6e9ab194
# Parent 9639c433be54191b4136b48fe70fc8344d2b5db2
amend: abort if unresolved merge conflicts found (issue5805)
It was checked by repo.commit() before e8a7c1a0565a "cmdutil: remove the
redundant commit during amend."
```
Differential Revision: D7517547
fbshipit-source-id: fd585bc4dceb837e8486b1285ed7ff14d24795d3
Summary:
The progress bar thread is a daemon thread, so that it doesn't need to be
deactivated manually. However, this means the thread terminates and the lock
is released during interpreter shutdown. Because Python clears module
attributes at the start of interpreter shutdown (see [1]), releasing the lock
fails as it can no longer get the thread identity.
To mitigate this, we register an atexit handler to terminate and join to the
progress bar thread.
[1] https://stackoverflow.com/questions/25649676/where-is-pythons-shutdown-procedure-documented
Differential Revision: D7568155
fbshipit-source-id: 85ef10af6c1576d5beceb78f8514e0e440cdab7f
Summary: We want to know what is the reason why we think the lock file can't be removed.
Reviewed By: farnz
Differential Revision: D7513954
fbshipit-source-id: fd1668e7a614e5e24e250018fbc880ba87821aa8
Summary:
This helps with PID reuse, which can cause false positives when checking
whether the lock-owning process is still alive.
Reviewed By: mjpieters
Differential Revision: D7513955
fbshipit-source-id: d3df4b4afa53cd1e9633d71c294cf7014b7b65c5
Summary:
The goal of this diff is to introduce a central place for all logic related
to parsing and handling the contents of the lock files. The current state of
things is hard to manage and even harder to extend.
NB: the stack is not complete, the changes that actually use this refactoring are yet to come.
Reviewed By: quark-zju
Differential Revision: D7489326
fbshipit-source-id: aa79d964411e3f5b61e24aa9babece05d4f0bd60
Summary:
This will later be used to add a bit of uniqueness to PIDs on Windows, to
address the stale lock problem.
Reviewed By: quark-zju
Differential Revision: D7489325
fbshipit-source-id: b866bd239eacfc7fe64433f0989a73dff1246644
Summary:
This allows us to turn on and off hgignore support directly without changing
files in the working copy (which could be hard to revert cleanly).
Reviewed By: mjpieters
Differential Revision: D7544529
fbshipit-source-id: 14cc41e2ae361070f91bf3b8aa28dd5808e7fe99
Summary:
This avoids building shared dependencies (ex. regex) over and over. The only
downside is cargo will take a lock and cannot build projects in parallel.
But `setup.py` does not support building extensions in parallel. So it's
fine.
Changed `matcher` to also enable lto like existing extensions, so `cpython`
build result can be reused.
Before (on devserver):
$ time python setup.py build_rust_ext
real 2m19.401s
user 3m35.118s
sys 0m8.277s
$ du -hs build/temp.linux-x86_64-2.7/
115M build/temp.linux-x86_64-2.7/
After:
$ time python setup.py build_rust_ext
real 2m4.371s
user 2m25.864s
sys 0m5.198s
$ du -hs build/temp.linux-x86_64-2.7/
58M build/temp.linux-x86_64-2.7/
`cargo` builds things in parallel. The speed improvement would be more
significant on laptops.
Differential Revision: D7512429
fbshipit-source-id: 378e721890bdfe53c8adbe364ad5f0b374023ff5
Summary:
Previously `hg graft -r 'ancestor::descendant` printed no error message at all.
This diff fixes it.
Reviewed By: mjpieters
Differential Revision: D7533498
fbshipit-source-id: 5c4e41ecc3178495ad2f41ef53ef65f7fbb70212
Summary:
This allows people to silence hints as they like. It's done by modifying
user hgrc.
Reviewed By: markbt
Differential Revision: D7392133
fbshipit-source-id: 1365294217db92dfb3a0c81332a9fefd164795d4
Summary: This allows us to change configs using code.
Reviewed By: mjpieters
Differential Revision: D7392129
fbshipit-source-id: 0960d4a322d6906469a79976031ab2b619dfa183
Summary:
This allows us to have a unified way to print hint messages at the end of a
command. It would be helpful for feature discovery in general.
Reviewed By: mjpieters
Differential Revision: D7392132
fbshipit-source-id: 8b4e94cc2176266652459ecca3428bd86d95bfe2
Summary:
I did some extra xdiff changes in upstream, namely:
- Remove unused features
- Replace "long" (32-bit in MSVC) with int64_t to support large files
- Add comment on some key variables
This backports them. It also includes Matt's fixes about Windows compatibility.
Reviewed By: ryanmce
Differential Revision: D7223939
fbshipit-source-id: 9287d5be22dae4ab41b05b3a4c160d836b5714a6
Summary:
By using low-level APIs, `environ` could contain ill-formed entries that do
not have the `NAME=VAL` form:
#define _GNU_SOURCE
#include <unistd.h>
char *envp[] = { "X", 0 };
char *argv[] = { "hg", "log", 0 };
int main() {
return execvpe("chg", argv, envp);
}
Ignore them silently so they won't break chg.
Reviewed By: DurhamG
Differential Revision: D7537406
fbshipit-source-id: 3fef3d656383723d451fbfa29ba9a9fa170311d0
Summary:
Using windows libs like `_kernel32 = ctypes.windll.kernel32` means that
the `GetLastError` calls are useless.
(1317 is a non-existent process)
```
In [44]: k32 = ctypes.WinDLL('kernel32', use_last_error=True)
In [52]: ctypes.windll.kernel32.OpenProcess(0x0001, False, 1317)
Out[52]: 0
In [53]: ctypes.windll.kernel32.GetLastError()
Out[53]: 0
In [54]: k32.OpenProcess(0x0001, False, 1317)
Out[54]: 0
In [55]: k32.GetLastError()
Out[55]: 5
```
This behavior can explain why we can't auto-delete the stale lock file. See the `testpid` code:
```
def testpid(pid):
'''return True if pid is still running or unable to
determine, False otherwise'''
h = _kernel32.OpenProcess(_PROCESS_QUERY_INFORMATION, False, pid)
if h:
try:
status = _DWORD()
if _kernel32.GetExitCodeProcess(h, ctypes.byref(status)):
return status.value == _STILL_ACTIVE
finally:
_kernel32.CloseHandle(h)
return _kernel32.GetLastError() != _ERROR_INVALID_PARAMETER
```
The non-existent `pid` causes `h` to be `0` and `_kernel.GetLastError()` to also be `0` (see above why). This means that we think that this process exists!
(Note: this ignores all push blocking failures!)
Reviewed By: quark-zju
Differential Revision: D7519457
fbshipit-source-id: c08f727228073962359e93db13f1fdb76f3699e6
Summary:
When a .backupfiles file is invalid, hg recover gives a not very
useful stack trace. This makes it a bit easier to debug.
Reviewed By: DurhamG
Differential Revision: D7482578
fbshipit-source-id: e8c7ad73bb14a38d2d43b636d60a3b77cd947331
Summary:
This makes hg pull use the connectionpool. This means prefetches can
reuse the existing ssh connection when appropriate. This both speeds up
prefetches, and also means they will speak to the same server that served the
pull.
Reviewed By: ryanmce
Differential Revision: D7481107
fbshipit-source-id: f9a3670527cb7e8956029c86d50d8e030dd3cc01
Summary:
Previously the connectionpool was a remotefilelog specific concept. We
want to start sharing connections between pull and prefetches, so let's move it
to core Mercurial.
Reviewed By: ryanmce, phillco
Differential Revision: D7480670
fbshipit-source-id: 1b2eff3b0e61a815709ffaec35df802eeda0c24b
Summary:
`hg debugcolor --style` shows the component parts of each style individually,
however this doesn't work if the styles are defined as the new fallback styles
(separated by colons). This is because the fallback is only implemented for
actual style names - it doesn't work for `ui.label('brightblue:blue', 'text')`.
It's usefule to see what the fallbacks are (even if they're not necessary on
your own system), so change debugcolor to split the elements of the fallback
style and show them separately.
Reviewed By: quark-zju
Differential Revision: D7485545
fbshipit-source-id: dce7204c9f0a98bb730b3ba864db28a9ec52a339
Summary:
gitignore could have performance issues stating .gitignore files everywhere.
That happens if watchman returns O(working copy) files. Add a config to
disable it as we're finding solutions.
Reviewed By: DurhamG
Differential Revision: D7482499
fbshipit-source-id: 4c9247b0318bf034c8e9af4b74c21110cc598714
Summary:
After testing locally, I couldn't conclusively prove if rebasing a single change with IMM was any faster or slower than on disk.
Using IMM on the working copy will definitely be better for rebasing stacks, and it's just nicer to not have the working copy thrash around as much. It also might be interesting to (possibly) let you work while the rebase is running, too.* So I've added the code that will let us enable this more widely (as a subset of IMM) to experiment.
*I've made it so that if you make any changes during the rebase (causing the last update to fail), we just print a nice message telling you to checkout the new rebased working copy commit, instead of failing/aborting. TBD whether this is something we want to encourage people to do, however. I've kept the existing up-front check for uncommited changes when rebasing the WCP with IMM for now.
Reviewed By: DurhamG
Differential Revision: D7051282
fbshipit-source-id: c04302539021f481c17e47c23d3f4d8b3ed59db6
Summary:
Since error parts pass the message and hint as parameters instead of
payload, they are limited to 255 characters. Let's add a helper function to
enforce this.
Reviewed By: quark-zju
Differential Revision: D7448104
fbshipit-source-id: 33d47a21e7159b6c4bd72cad9669568b92a51e34
Summary:
In an in-memory merge, if a commit only changed the flags of a file, and that file also never got written to during the merge, the IMM could fail and cause it to restart.
The reason is pretty simple: `setflags()` sets `cache[flags]` but not `cache[data]`, as it doesn't have any new data to store. In that case, calls to read the data should to fall-through to the underlying `p1` context.
Indeed, proper logic to do that already exists in `overlayworkingctx.data(path)` and `flags(path)`. The problem is that `tomemctx()` was reading from the cache directly, which is problematic and unhygenic. So let's just change it to call the proper functions, which also fixes the bug.
Reviewed By: DurhamG
Differential Revision: D7447640
fbshipit-source-id: 1625ef82ad2683c6a72059a0944fd5e336d3ec3a
Summary:
Use the new gitignore matcher powered by Rust.
The hgignore matcher has some laziness, but is not tree-aware - with N
"hgignore" files loaded, it needs O(N) time to match. The gitignore matcher
is tree-aware and backed by native code with decent time complexity.
We have been maintaining a translation script that collects all gitignores,
generate hgignore files with very long regexp for them. That script has
issues with sparse recently. This diff allows us to remove those generated
hgignore files from the repo.
Note: fsmonitor state does not contain ignored files. And ignore
invalidation is generally broken in fsmonitor (it only checks top-level
.hgignore). That means, once a file is ignored, it cannot be "unignored" by
just removing the matched pattern from ".gitignore". The file has to be
"touched" or so.
Reviewed By: markbt
Differential Revision: D7319608
fbshipit-source-id: 1763544aedb44676413efb6d14ffd3917ed3b1cd
Summary: Build the new Rust matcher with both buck and setup.py
Reviewed By: markbt
Differential Revision: D7319607
fbshipit-source-id: c5944a28602495a9127acb20b59eb95632a9a1f5
Summary:
It only contains a `gitignorematcher` which exposes `GitignoreMatcher`
features to Python.
Reviewed By: markbt
Differential Revision: D7319605
fbshipit-source-id: 846964a551813f9b0933bc30f4a0ba3f85362944
Summary:
- Support 8 color mode, since some terminals (emacs) do not support 16
colors (i.e. "bright*" colors have no effects).
- Detect emacs and use 8 color mode automatically.
- Read terminfo colors. Respect it if it says the terminal supports more
colors. Suggested by wez.
- Read HGCOLORS. Respect it unconditionally. Change run-tests.py to set it.
- Change diff related colors to also fallback to 8 colors since "bright"
colors are not guaranteed available.
Differential Revision: D7432965
fbshipit-source-id: da75829f856b4de737b946af72d24ff5351026cb
Summary:
Add the ability to set a `formatfunc` on a progres bar, which formats the
numbers used. Use `util.bytecount` as a format function in the places
where we have progress bars in numbers of bytes.
Reviewed By: quark-zju
Differential Revision: D7355232
fbshipit-source-id: 117c7035d46d47259cdfd70b80438cc6f4615977
Summary:
Nothing currently displays a progress bar while the curses interface is shown,
but in the future something may be added which does, so ensure the progress bar
is suspended when showing the curses interface.
Reviewed By: ryanmce
Differential Revision: D7417435
fbshipit-source-id: 6b91b17ee5390cbde6e983081a0940051ab865c8
Summary:
I want to add to helpcmd but with nested functions that's a lot more
complicated then it needs to be.
Reviewed By: ryanmce
Differential Revision: D7365391
fbshipit-source-id: 02092dd55f8f9521324b8ed51fa3134817454d36
Summary:
Raise this instead of RepoError. This way commands can catch and auto-recover the
repo, if appropriate. Right now I only do this inside megarepo commands.
Reviewed By: DurhamG
Differential Revision: D7363245
fbshipit-source-id: 0a6cab92a4ff6e50cd269bdddadee47be6199ebd
Summary: Use 256 colors if supported, or fallback to 16 colors.
Reviewed By: singhsrb
Differential Revision: D7388275
fbshipit-source-id: e7be447f2b900e384e9280f92ea8f881747493e4
Summary:
Allow color config to be something like `color132:red` that is to use 256
color if supported, but fallback to `red`.
To get a consistent test output. `run-tests.py` is changed to disable 256
color support explicitly.
Reviewed By: singhsrb
Differential Revision: D7388277
fbshipit-source-id: da74ae8fc70c971901d56a6985976db44cbec0d9
Summary:
This would allow us to use 256 colors, which seems to be widely available
on modern terminals (including mosh and tmux).
Reviewed By: singhsrb
Differential Revision: D7388279
fbshipit-source-id: 726a364ed5d3acc449f0d7ada14c42e4b68424ec
Summary:
This allows us to treat terminals with 256 color support differently.
The motivation is, the bright colors seem to work fine on Linux. But they
appear to have minor differences on OSX terminals (iTerm2 or Terminal.app).
Using explicit 256 colors is better.
Reviewed By: singhsrb
Differential Revision: D7388276
fbshipit-source-id: 970ee18f0f5bb9cc4bb5c39b2b46b354525b3d55
Summary:
As suggested by wez, "dim" is less supported than the bright colors.
So let's use the bright versions instead. Basically, replacing "red",
"red dim" with "brightred" and "red".
Reviewed By: singhsrb
Differential Revision: D7387254
fbshipit-source-id: f741d9e1c12115acdd1a3a2cae1658d8fae534bf
Summary:
Most terminals support 16 colors these days. This is manually tested
with: tmux (2.2), mosh (1.3.0), screen (4.04) from Linux (xfce) and
OS X Terminal.app and iTerm 2.
Note: aside from "screen" with default configuration, the terminals
also have 256 colors support. "screen" can support 256 colors if it's
started with TERM=xterm-256color.
See https://en.wikipedia.org/wiki/ANSI_escape_code for details.
Reviewed By: singhsrb
Differential Revision: D7387258
fbshipit-source-id: e7b2437fee2b4e593077bb3f44143c86ece93a8f
Summary:
The progress bar runs in another thread and may draw " <=> " which pollutes the
current screen.
Make `ui.system` suspend the progress bar automatically. This should work for
the "invoking editor" case. I think sshaskpass and curses might also need to
suspend the progress bar but those are trickier.
Reviewed By: singhsrb
Differential Revision: D7377492
fbshipit-source-id: 833ac724bbe4f9c630ca37567dc088f7279dd67e
Summary:
Previously we were storing the changelog on the manifestlog and using
it to resolve linkrevs before serializing them. It turns out the changelog can
be invalidated at a different rate than the manifestlog, so we could encounter
issues where the manifestlog held a reference to the old changelog.
To fix this, let's hold a reference to the repo and access the changelog from
there when we need it. This introduces a circular reference between the
manifestlog and the repo, but it's probably fine for now until we can get rid of
the need for changelog invalidation.
Reviewed By: singhsrb
Differential Revision: D7360321
fbshipit-source-id: 2317c7fcd6b307a50b64f0c5df97dda2955f3e21
Summary:
Remove ui.progress as a method of updating progress. All progress bars now go
through new-style progress bars.
This also splits out the rendering of progress bars from the reporting of
progress. All tests are updated to use new-style debug progress bars, which
simply report the position of the progress bar. Rendering of progress bars
will be tested separately once the progress bar engine has been rewritten.
Reviewed By: quark-zju
Differential Revision: D7329488
fbshipit-source-id: 14f8ab67365ddd98b74986aa25d9abc7a0546144
Summary:
httpconnection uses old-style progress bars in a way that isn't easy to
replicate using new-style progress bars. Remove it for now, it can be added
back in later in the right place.
Reviewed By: quark-zju
Differential Revision: D7329486
fbshipit-source-id: 814c55d7a5bb5c97fc6b3967510ed656110038c8
Summary:
The recursive function _verifymanifests behaves differently for the root and
non-root manifests in treemanifest. Split out the behaviour into separate
functions in preparation for reworking the progress bar to the new style.
Reviewed By: quark-zju
Differential Revision: D7329518
fbshipit-source-id: 4f499354060d7397a6a9781d15df26a0343c8fca
Summary:
Update hg.copystore and util.copyfiles to use the new-style progress bar
context manager.
Reviewed By: ryanmce
Differential Revision: D7329482
fbshipit-source-id: 4ac1def57e0ae4e7819c7c0c4d6f1715b8cbb625
Summary:
New-style progress bars should only update if there has been any progress since
the last refresh. Otherwise, stalls in progress cause weird behaviour for time
estimates.
Reviewed By: DurhamG, quark-zju
Differential Revision: D7329494
fbshipit-source-id: 43b9fdd5573b34174673db838e53a0d54504eefd
Summary:
`progress.debugbar` is an implementation of the `progress.bar` context that
behaves like the old progress bar implementation when used in tests.
Reviewed By: DurhamG
Differential Revision: D7329500
fbshipit-source-id: 28c96c1ddf096efbec6994cf5b4dea04f88d55d3
Summary:
We have some repo's whose first commit didn't not add any files, which
means they have nullid as their manifest pointer which breaks backfilltrees.
This patch fixes that.
Reviewed By: singhsrb
Differential Revision: D7340678
fbshipit-source-id: 85b048b81c1861fd48d5cc082d414271aee7283b
Summary:
This adds the ability to specify a config file to be used during the
command. This is useful during clones for letting the clone command use the
given repositories system specified repo-specific hgrc file.
Reviewed By: quark-zju
Differential Revision: D7311576
fbshipit-source-id: a97d8ebada2e0bea27c75a7650df8ede00dc10c6
Summary:
worddiff needs extra computation and does not have effect if color is
disabled. It should also be disabled with HGPLAIN=1.
Reviewed By: ryanmce
Differential Revision: D7315276
fbshipit-source-id: b59255bc708713186bd4ddafdc469763e1a66ea6
Summary:
There were recent complains about both quality [1] [2] and performance [3]
of the current word diff algorithm.
The current algorithm is actually bad in various ways:
- Lines could be matched across hunks, which is confusing (report [1]).
- For short lines, they can fail "similarity" check, which means they
won't be highlighted when they are expected to be (report [2]).
- Various performance issues:
- Using difflib implemented by pure Python, which is both slow and
suboptimal comparing with xdiff.
- Searching for matched lines across hunks could be O(N^2) if there are
no match found.
Thinking it in a "highlight" way is actually tricky, consider the following
change:
```
# before
foo = 10
# after
if True:
foo = 21 + 3
```
It's obvious that "10" and "21 + 3" need highlighting because they are
different. But what about "if True:"? In theory it's also "different" and
need highlighting. How about purely inserted or deleted hunks then?
Highlighting all of them would be too noisy.
This diff rewrites the word diff algorithm. It differs in multiple ways:
1. Get rid of "matching lines by similarity" step.
2. Only diff words within a same hunk.
3. Dim unchanged words. Instead of highlighting changed words.
4. Treat pure insertion or deletion hunks differently - do not dim or
highlight words in them.
5. Use xdiff instead.
6. Use a better regexp to split words. This reduces the number of tokens sent
to the diff algorithm.
1, 2, 5, 6 help performance. 1, 2, 3, 4 make the result more predictable and
trustworthy. 3 avoids the nasty question about what to highlight. 3 and 4 makes
it more flexible for people to tweak colors. 6 makes the result better since it
merges multiple space tokens into one so xdiff will less likely miss important
matches (than meaningless matches like spaces).
"bold" and "underline" were removed so the changed words will have regular
red/green colors. The output won't be too "noisy" even in cases where code are
changed in a way that inline word matching is meaningless. For people who want
more contrast, they can set:
[color]
diff.inserted.changed = green bold
diff.deleted.changed = red bold
Practically, when diffing D7319718, the old code spends 4 seconds on finding
matched lines preparing for worddiff:
```
| diffordiffstat cmdutil.py:1522
\ difflabel (17467 times) patch.py:2471
....
> 3927 \ _findmatches (22 times) patch.py:2537
348 \ __init__ (8158 times) difflib.py:154
340 | set_seqs (8158 times) difflib.py:223
328 | set_seq2 (8158 times) difflib.py:261
322 | __chain_b (8158 times) difflib.py:306
1818 \ ratio (8158 times) difflib.py:636
1777 | get_matching_blocks (8158 times) difflib.py:460
1605 \ find_longest_match (51966 times) difflib.py:350
38 | __new__ (51966 times) <string>:8
29 \ _make (36035 times) <string>:12
143 \ write (17466 times) ui.py:883
```
The new code takes 0.14 seconds:
```
| diffordiffstat cmdutil.py:1522
\ difflabel (23401 times) patch.py:2562
....
> 140 \ consumehunkbuffer (23346 times) patch.py:2585
130 | diffsinglehunkinline (23240 times) patch.py:2496
215 \ write (23400 times) ui.py:883
118 \ flush cmdutil.py:1606
118 | write ui.py:883
```
[1]: https://fburl.com/lkb9rc9m
[2]: https://fburl.com/0r9bqf0e
[3]: https://fburl.com/pxqznw31
Reviewed By: ryanmce
Differential Revision: D7314726
fbshipit-source-id: becd979cb9ac3fd3f4adae11cb10804d535f58df
Summary:
Instead of yielding tokens directly, buffer them if they belong to a same
hunk. This makes it easier for the upcoming new worddiff algorithm to only
focus on the diff hunk, instead of having to worry about other contents.
This breaks how the existing experimental worddiff algorithm works, so the
algorithm was removed, and related tests are disabled for now.
Reviewed By: ryanmce
Differential Revision: D7314725
fbshipit-source-id: 344e502cd185b2412fbd2ee299131bbb4560ac17
Summary:
The original logic makes it harder to reason about - it yields the "\n"
character belonging to the last line in the next loop iteration.
The new code is in theory a little bit slower. But is more readable. It
makes the following changes easier to read.
Reviewed By: ryanmce
Differential Revision: D7314727
fbshipit-source-id: cb792577adc8f7629bc2e0fe8f0688e705bf4b30
Summary:
Previously, during a push the changegroup packer had no information about the
destination server, aside from what changegroup version it supported. In a
future diff we'll want to package certain files and trees based on if the server
is a main server or not, so let's pass thread the bundle2caps down to the
changegroup packer.
The changegroup already had the old style bundlecaps on it, but those
capabilities don't actually exist on the client side during a push, so let's
just use the bundle2 ones, which are more modern anyway.
Differential Revision: D7297899
fbshipit-source-id: 74ab529baccbdc999267555779a10c0d27f8bd47
Summary: Help people debug issues by providing useful hints when templates fail to parse.
Differential Revision: D7148780
fbshipit-source-id: bd2b0f2bdeb0c970732fcaf53854d4a003b08571
Summary:
This logic is largely based on the similar logic added to template error
messages in D2608 and D2609, but with a few tweaks based on how revsets
actually work.
Differential Revision: D7148779
fbshipit-source-id: fb435788711a2c4ef881dfaeac5176fc1972c07a
Summary:
Previously we were just putting nullid as the linknode in client side
trees, because when the trees were added the changelog hadn't been written yet,
so we didn't know the linknode. This diff updates manifestlog.add to pass
linkrevs to mutablehistorypack, which get resolved at serialization time into
the appropriate linknode.
Reviewed By: ryanmce
Differential Revision: D7280104
fbshipit-source-id: bbc8a7bfad31da296a1b65973875ba2e1e1f7a95
Summary:
In a future diff we'll want to allow converting flat manifests to tree
manifests during infinitepush. To do so, let's allow writing to in memory packs
on bundlerepositories.
Reviewed By: StanislavGlebik
Differential Revision: D7256563
fbshipit-source-id: 10ec58d1171b7882d6db9a916c50a19bc11dbcb4
