Summary:
The previous logic restarted the *entire* rebase if one commit had merge conflicts or needed a mergedriver run. This was unnecessarily slow, plus the abort triggered a full repo scan.
Also, this logic is actually simpler and easier to reason about when done here, vs. `rebase()`. We can delete all the wrapping that function does after this rolls out.
This is config-gated, so we can roll it out gradually. Old config = old behavior.
Reviewed By: quark-zju
Differential Revision: D8272566
fbshipit-source-id: 9e598eea10c8a634497cd211a468dd2f58c5ff47
Summary:
The fancyrenderer is a little loud for progress bars with no totals and
spinners. Tone it down a little.
Reviewed By: phillco
Differential Revision: D8403456
fbshipit-source-id: ff8316f4098a02fbafb083e5ae7baa4a243c93bd
Summary:
After `fork()`, the chg worker no longer needs to detect incoming
connections using the selector, or `accept()` a connection. So let's just
close those fds explicitly.
Reviewed By: phillco
Differential Revision: D8384549
fbshipit-source-id: 9bd43692a575d425b61da8ad9c6b069113b2ae4b
Summary:
Avoid writing `clock` and other debug values if clock is empty.
This helps generate stable output in tests.
Reviewed By: markbt
Differential Revision: D8394734
fbshipit-source-id: 95c6fdc1448eb382c51303ac6ccf400c2c6e578a
Summary: This makes it easier to test whether a "dirstate" file is treestate or not.
Reviewed By: markbt
Differential Revision: D8394737
fbshipit-source-id: 07066da1fc0478cbf460e5b0d46800c7ba20b2ed
Summary:
This helps expose internal flags like NEED_CHECK etc. They would be useful
for precisely testing treestate.
Reviewed By: markbt
Differential Revision: D7935821
fbshipit-source-id: 21117cec148c113ec24f94d9fae849cb7a924284
Summary: This allows us to dogfood the new treestate.
Reviewed By: markbt
Differential Revision: D7912559
fbshipit-source-id: bfb30cd9989460b0eae859aeed402bd92776ab0a
Summary:
Previously, there are 2 configs: `treedirstate.useinnewrepos` and
`format.usetreestate`. They are both related to dirstate format and conflict
with each other. This patch unifies them into a single config
`format.dirstate`.
As we're here, merge `test-fb-hgext-treedirstate-x.t` to `test-dirstate-x.t`
if they were previously copied from `test-dirstate-x.t`
Reviewed By: markbt
Differential Revision: D8393878
fbshipit-source-id: 57abeea22ce732d93205e4d4308923afa90693f4
Summary:
Importing via Python is too slow. Therefore let's implement a native code
path for importing from other dirstate maps.
As we're here, modify treedirstate's import implementation so it skips
untracked files.
Reviewed By: markbt
Differential Revision: D7912557
fbshipit-source-id: ff1673ece2396a89fcc01e8b04041a6754cad885
Summary:
Previously, hg returns -1 (255) on "unknown errors". That could conflict with
other things. For example, http://tldp.org/LDP/abs/html/exitcodes.html suggests
1, 2, 126 ... 255 have special meanings defined by a common shell. Namely,
`ssh` also returns 255.
Another long complaints about hg exit code is the use of 1 to indicate
"no changes". Although that's explained at [1], a lot of scripts still find it
surprising.
This diff adds a `ui.exitcodemask` config option, which makes it to alter the
exit code. Set it to 254 will change 1 to 0 for some script use-cases. Set it
to 63 or so will avoid conflicts with software like `ssh`.
Many existing scripts still expect 255 to be the return code. So we cannot
change the default value that easily. To avoid surprises (like, put the config
option in `/etc/mercurial/hgrc.d/exitcode.rc`), the config set by config
files is ignored if `HGPLAIN` is set and `HGPLAINEXCEPT` does not contain
`exitcode`.
[1]: https://www.mercurial-scm.org/pipermail/mercurial-devel/2012-January/037711.html
Reviewed By: DurhamG
Differential Revision: D7921817
fbshipit-source-id: 764b0de030fc727aa5df7305c2b8bc92f576cd33
Summary:
When requesting trees from the server we try to provide a base tree
that we already have. In the hybrid manifest days, when we were fetching
manifest X, we would look up the linknode for X in the flat manifest revlog,
then scan the changelog up and down from that linknode. In the treeonly world we
don't have that revlog anymore though, so the current algorithm just searches from
the tip. This ended up being a really bad algorithm, since if you have tip and you
request tip~1000, then tip~2000, then tip~3000, you end up basing them all
against tip and redownloading a lot of data. This makes hg blame and hg log -p
super slow.
Let's change the algorithm to base off of a hint linknode, and let's change
changectx to set that hint every time it attempts to read a manifest. In manual
testing this significantly sped up hg log -p
Reviewed By: phillco
Differential Revision: D8399613
fbshipit-source-id: 771a94ee8b82be682ea0091b8d6c0fcd5f4e6646
Summary:
This allows us to remove the Python binding without breaking existing zstd
users (commitcloud bundles). It might also make the future Rust migration
easier.
Using `zstd` create for its streaming APIs. `zstdelta`'s APIs cannot be used
since it requires decompressed length to be known, which wouldn't work for
streaming compressed data.
Note: For easier implementation, the Python land no longer processes data
in a streaming way. This is probably fine for the current bundle use-case.
In the long term, we might want to revisit the bundle format entirely.
As we're here, also expose zstdelta's APIs and add a test for it.
Reviewed By: DurhamG
Differential Revision: D8342421
fbshipit-source-id: 89902d551f4616469d6e1bc9b334a1c37c884775
Summary: This should be self._repo.ui.
Reviewed By: quark-zju
Differential Revision: D8382375
fbshipit-source-id: b3269d8fdac3d2159f547b6220280109d8400dcb
Summary:
If the repo has treemanifests, we can use this to do the case check faster.
Note that with treemanifests we only find the first path component that
conflicts (subsequent conflicts won't be found), as these conflicts occur in
different trees.
Reviewed By: farnz
Differential Revision: D8332426
fbshipit-source-id: 243dab9bf711c6fdb62a0e0ba4346faea3b6b0ad
Summary:
This is clearly the right thing to do, given what the other `lookup` function in this file does.
Without `None` as a default `get` argument, OSX tests fail with `missing "default" argument1 error.
Reviewed By: ryanmce
Differential Revision: D8379304
fbshipit-source-id: ff66551221eb95cd5e59dfb5a5ab66b15211ad7f
Summary:
Remove the special-case code path where m.always() is true. The non-always
code path also behaves correctly with always matchers and should perform just
as efficiently. The only two additional things it does is to customize the
matcher's `bad()` callback (which should never be invoked for matchers that
match everything) and to loop through `m.files()`, which is guaranteed to be
empty for matchers where `m.always()` is true.
Reviewed By: quark-zju
Differential Revision: D8229267
fbshipit-source-id: adb9f7cfe269febd53ff417ec0d3817833ac5338
Summary:
The new treestate could track untracked files, and casecollisionauditor
should not iterate all files in the treestate. So let's implement a lazy
version of casecollisionauditor for treestate similar to what was done in
treedirstate.
Reviewed By: markbt
Differential Revision: D7912558
fbshipit-source-id: 7a95275a430aed109a49c4cdcd3a474c5961470b
Summary:
Add a config option `format.usetreestate`, which would turn on the new
treestate for new repos.
Treestate does not need manually maintaining "copymap", "otherparentset",
"nonnormalset". The one single tree is the source of truth of everything.
So some code like `copymap.pop` etc. are disabled on treestate path.
Reviewed By: markbt
Differential Revision: D7909175
fbshipit-source-id: 96a405fccd4099cefc152ec17204cb5407efbe7e
Summary:
This is subtle. If visitor changes file state, `Node.id` should be set to
`None` to mark it as "changed".
In practise, treedirstate uses visitor to rewrite mtime to -1 if mtime is
"fsnow". Those rewritten mtime all belong to "changed" nodes (because "fsnow"
can only increase, and on-disk entries cannot have "mtime == fsnow" because
they would be written to -1 during the previous write), so it's not a problem
yet.
It is safer to not depend on the fact that "visitor" can only change "changed"
nodes. On the other hand, detecting changes for all filestate fields could be
undesirably expensive. So let's make the visitor provide the "changed or not"
information. Surely the visitor knows what it does.
Reviewed By: markbt
Differential Revision: D7909167
fbshipit-source-id: 21e71302cf1db86c1330b294baddd51cc8a96026
Summary:
The reason caching is disabled seems to be that symlinks can change where
paths points to, while some properties of that path are cached.
Sometimes we know that symlinks are not used in the repo, so we'd like hg
to let us shoot ourselves in the foot.
The motivation here is that I have a specially crafted rebase operation,
which makes on the order of 400000 stat-like syscalls without caching and
on the order of 6000 with caching. This can save ~20s on Windows.
Note that this setting is disabled by default so we can't break anything.
FWIW, we already have an unsafe config option, which allows us to opt into the "we don't do crazy stuff with symlinks" behavior, it's called `unsafe.filtersuspectsymlink`, setting it to false speeds up Windows `status` and we do so for `ovrsource` on Windows.
Reviewed By: quark-zju
Differential Revision: D8297129
fbshipit-source-id: ec751d18b50b042f28f68c9e1a3634ab94a4e7c3
Summary:
An updated version of D8174246 with a test. Max memory usage is very useful to us to understand which commands are using too much memory, and the existing wrapper metrics don't work when chg is used (they measure chg's usage).
Windows support will come later as it is a bit more involved.
Reviewed By: DurhamG
Differential Revision: D8318584
fbshipit-source-id: 323450bc7ab376014d70106beb5d4fdcc7fba0c8
Summary:
Needed for windows to allow us write to a tempfile created in Rust. Should do it in a different way?
Context: we create a NameTempFile from Rust that we pass as argument to hg as profiling.output=named_temp_file. When open with 'w', the mercurial command fails with "abort: Permission denied". As far as I read, it might be because of how the Win32 API works. Under the hood, Python's open function is calling the CreateFile function, and if that fails, it translates the Windows error code into a Python IOError. From CreateFile documentation:
If CREATE_ALWAYS and FILE_ATTRIBUTE_NORMAL are specified, CreateFile fails and sets the last error to ERROR_ACCESS_DENIED if the file exists and has the FILE_ATTRIBUTE_HIDDEN or FILE_ATTRIBUTE_SYSTEM attribute. To avoid the error, specify the same attributes as the existing file.
I might be wrong though... any other suggestions?
Reviewed By: ikostia
Differential Revision: D8099420
fbshipit-source-id: c3077cc5d7bc03c8f2eeafdac3467db62c20a669
Summary:
Template function requires input arguments. They do not always exist.
If any argument is missing, just return an empty value, instead of crash.
Reviewed By: mitrandir77
Differential Revision: D8221246
fbshipit-source-id: ff3839398bdc620bff61d1b19b1512010c06388e
Summary: Will be used by the next change.
Reviewed By: mitrandir77
Differential Revision: D8221077
fbshipit-source-id: c9713175dfaab0b090f81d2c9d4f4d9d63a0771c
Summary:
There mercurial template language is a powerful language. But it has some
limitations right now:
- It's focused on "commit". If there is no commit, things become harder.
- The related formatter concept is coupled with "a list of things". If there
is no list, things become impossible.
- Some commands use formatter/template partially, and that does not play
very well with `ui.write`, and is difficult to document.
This patch starts to solve all the issues, by introducing command-level
template state, and with the vision:
- Anything could be used as an "environment" passed to the template
language, a commit, a file, a line, a "command" with customized
properties, etc.
- If a template keyword function is incompatible with the environment, it
should fail gracefully.
- The template language is a super set of other things. `-Tjson` should not
be magical, but an alias of `{some complex expression | json}`.
- Discourage the use of formatter.
- Template keywords can be registered and documented.
This would remove the limitations of the template language and unleash its
full potential.
Note there are still some cases where the formatter is desirable. For example,
`hg log` with a large revset should be able to stream its output. The command
template feature added right now only renders things at the end of a command,
and is not (and requires significant effort to be) streaming-friendly.
Reviewed By: mitrandir77
Differential Revision: D8221078
fbshipit-source-id: 26fed50025c4676af277686382ce2d36f1faed97
Summary:
The added method takes an environment (`props`), and a program written in
the template language, execute the program, and pass output to `ui.write`.
The API is similar to the upstream [1]. But it avoids `ctx` intentionally.
Since the "environment" should be more flexible than having to have a `ctx`.
[1]: aa32940279
Reviewed By: mitrandir77
Differential Revision: D8221079
fbshipit-source-id: 94ee5c563f943330ef91966a4883c5733cc8dac9
Summary: Implement most methods, similar to treedirstatemap.
Reviewed By: markbt
Differential Revision: D7909176
fbshipit-source-id: d20e4642ee381cc1cfe7c774071c8c3c1e493f45
Summary:
Add a class with methods defined in `dirstate.dirstatemap` so it could be
used as a drop-in replacement. The actual logic is not implemented yet.
Reviewed By: markbt
Differential Revision: D7909170
fbshipit-source-id: 5c1ae1bf4341d6368f5c9824487e7702f93043a7
Summary:
They're actively fighting against the clang-format config
and don't have an auto-fix.
Reviewed By: quark-zju
Differential Revision: D8283622
fbshipit-source-id: 2de45f50e6370a5ed14915c6ff23dc843ff14e8a
Summary:
Re-using long-running connections can cause problems, as the `hg serve` process
on the server may be using large amounts of resources. Add a new configuration
option `connectionpool.lifetime`. Connections that are older than that time
will be closed and not re-used.
Reviewed By: DurhamG
Differential Revision: D8204415
fbshipit-source-id: ac937778e3dc79e77e367017db2141066bd17263
Summary:
A new renderer that doesn't print anything. This ensures the progress engine
continues to run (other ways of disabling progress completely disable progress
calculations), and can be used in conjunction with the progressfile extension
to give progress output there without anything on stdout.
Reviewed By: phillco, farnz
Differential Revision: D8258548
fbshipit-source-id: 78cb78d84c6d1dd22744d84d34bf0f43f7962ace
Summary:
The default value of `rev` for `debugcheckcasecollisions` should be `""`, not
`[]` (the latter causes an exception).
Differential Revision: D8236465
fbshipit-source-id: 841e0218664c46e70abd5b774463d4b4e40fdfb8
Summary:
When running `hg hint --ack ...`, pick the file that exists to write,
instead of using the first default one.
Reviewed By: singhsrb
Differential Revision: D8230173
fbshipit-source-id: 90375653ffa16cfbf77eed4f967b5250b649a532
Summary:
When called with any matcher parameters, `hg revert` would first walk both the
working directory and the commit in question to find all files that match the
pattern. Only after it found all matching files did it check to see which
ones actually need updating.
This made revert performance `O(matched files)` rather than
`O(files needing revert)`. This makes operations like
`hg revert large_directory` quite expensive, especially when only a handful of
files need to be updated.
This should help address the performance of `hg revert .`, which previously
was much slower than `hg revert --all`. Previously it enumerated every file
in the repository twice (once in the working directory and once in the current
commit) before checking which ones needed reverting. This should also help
fix the performance of `hg revert <directory>` in Eden as well.
Reviewed By: quark-zju
Differential Revision: D8217972
fbshipit-source-id: c18e72c8a5ed0b4c3a5ed69e79f6a88d3ea04a9f
Summary:
Calling workingctx.status() ignored the input matcher's bad() function and
always replaced it with its own function intended to filter out bad() calls
for directories that do exist on disk. However for paths that were
legitimately bad it directly printed a message using ui.warn() rather than
calling the original input matcher's bad() function.
This updates workingctx.status() to invoke the input bad() callback so that it
behaves like the `status()` method on other context object types.
Reviewed By: quark-zju
Differential Revision: D8217973
fbshipit-source-id: 9dd458b96bc43e326791873e599dc38a93898c58
Summary:
btrfs has a bug involving truncate not returning the correct error
code. We already handle it in other cases, so let's extend it to these uses of
truncate. Hopefully the need for this disappears soon.
Reviewed By: quark-zju
Differential Revision: D8152297
fbshipit-source-id: f55602ff5e0ec36346c547bfd4b6d0f6e4127500
Summary: Mostly empty lines removed and added. A few bugfixes on excessive line splitting.
Reviewed By: quark-zju
Differential Revision: D8199128
fbshipit-source-id: 90c1616061bfd7cfbba0b75f03f89683340374d5
Summary:
We're seeing "Permission denied" errors on some of our automation, but
it's difficult to track down where it's coming from. Let's make each message
more descriptive.
Also updates, hgsql to handle the hg-rsh hook, which I noticed while
investigating.
Reviewed By: phillco, farnz
Differential Revision: D8188414
fbshipit-source-id: 5f8c99e8ba896c2636b1a04716125bc6a9df0591
Summary:
Add a new debug command to check whether any of the provided files
casecollide with any file in a revision's manifest.
Reviewed By: quark-zju
Differential Revision: D8165659
fbshipit-source-id: 9315ff052c9996888202961d168d20b834c22834
Summary:
Add an internal `get_dir` API to return aggregated states. It is exposed via
`.get('dir/')` python interface.
This is useful for implementing `hastrackeddir` of the dirstatemap class.
Reviewed By: markbt
Differential Revision: D7909173
fbshipit-source-id: 100a8f36237a6b911a4bfb4afbb4c63b98611317
Summary:
Going to make changes to `mercurial/` for cleaner fsmonitor support
directly. So let's move the Rust python bridge there first.
Reviewed By: markbt
Differential Revision: D7909174
fbshipit-source-id: 454d784b5dca18a3af9328fc7b2f342cd4188cf6
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
Summary:
The clang-format file in scm/hg basically only applies to things in the
scm/hg/mercurial directory.
There are 180 C and C++ files under scm/hg, and the vast majority of them do
not follow the style specified in this clang-format file. All but 11 of these
files were present in scm/hg/contrib/clang-format-blacklist.
Of the 11 files that do follow this style, 10 are in the scm/hg/mercurial/
directory. (10 of the 21 files in this directory use this style.) The 1
other file is in scm/hg/contrib/xdiff.
The majority of the C/C++ files in scm/hg/hgext/extlib and scm/hg/lib/ follow
a style closer to Facebook and Google's C++ style guidelines.
Therefore this moves the .clang-format file to scm/hg/mercurial, and lets the
main fbsource clang-format file apply to the other files under scm/hg
Reviewed By: quark-zju
Differential Revision: D8131512
fbshipit-source-id: 622a33abc39eb240eff4ca27f69a675a7ed54a89
Summary:
Add the filestat template keyword, which expands to a list of file status
information for each file modified in a commit.
Reviewed By: mitrandir77
Differential Revision: D8161706
fbshipit-source-id: 1dc41acebbcb081581f6b227facc8228375a320e
Summary:
Make template keywords that expand to lists of things that aren't strings, and
don't have a template to render the items in the list, render as the number of
items.
Reviewed By: mitrandir77
Differential Revision: D8161705
fbshipit-source-id: 19ca1cba88c0ce75c0ba358cd7e6f27c7ac61c34
Summary:
Checkheads is a legacy feature that is less useful in our setups, namely:
- In commit cloud / Mononoke's world, it's intentional to have many heads.
Pushing a new head is a normal operation that should not be forbidden.
- With remotenames, remotenames performs the check and checkheads is
redundant, as shown by the added test.
So let's add a config option to turn it off first. Later we can remove the
feature and update all the tests.
Reviewed By: ryanmce
Differential Revision: D8148016
fbshipit-source-id: 71684f20b9ca37902440eae331292679b0feb4c6
Summary:
Make it show "added", "removed", "changed" so it looks similar to the
original commit template.
Reviewed By: singhsrb
Differential Revision: D8158006
fbshipit-source-id: 1a1578bd263e7870c27e7169537e808ffd8add9f
Summary:
Generate a `u64` integer about the "version" at build time, and make chg
client check the version before connecting to the server.
This would ensure a chg client would only connect to a matched version of
the server.
- In setup.py, compute the "versionhash", write it as
`mercurial.__version__.versionhash`.
- In dispatch.py, `mercurial.__version__` needs to be explicitly loaded
before forking.
- In commandserver.py, send the versionhash to the client with the "hello"
message.
- In chg.c, verify the versionhash. If it does not match, unlink the socket
path and reconnect.
Reviewed By: farnz
Differential Revision: D7978131
fbshipit-source-id: 50acc923e72e40a4f66a96f01a194cf1a57fe832
