Summary:
Add a config option `fsmonitor.detectrace` and environment variable
`HGDETECTRACE`. Once set, perform an additional query to check if there are
writes to the working copy while `wctx._buildstatus()` is being calculated.
Also enable the race detector for all fsmonitor tests.
Reviewed By: wez
Differential Revision: D8597957
fbshipit-source-id: 5d83d529641325dc8d5e72ad059eed8db27d1f2a
Summary:
2f205fdc948fbea made remote stderr get written to local stderr, but
didn't catch this spot where when a connection closes we read the remainder of
stderr and write it out.
Reviewed By: quark-zju
Differential Revision: D8627272
fbshipit-source-id: c846ac6bb7425114fb29c6374c74b35057c14e63
Summary: D8601652 fixed a bug which broke this test. So let's update the test.
Reviewed By: quark-zju
Differential Revision: D8629814
fbshipit-source-id: 661a406f6b978836644f79ae80d007be1ff98ffa
Summary:
For some reason, some users still have `perftweaks.cachenoderevs` enabled.
That actually slows down things because `clindex` is expected to make node
lookup faster without special hacks. So let's disable
`perftweaks.cachenoderevs` if clindex is enabled in config. This would also
allow us to clean up related configs in Chef (blocking D7826336).
Reviewed By: DurhamG, singhsrb
Differential Revision: D8626199
fbshipit-source-id: 69a6f22ee25ff0a00e870f7d07c73ca0431720c9
Summary:
use -w option instead of -p for keychain command security add-generic-password
-p Specify password to be added (legacy option, equivalent to -w)
-w Specify password to be added
options are the same, but -p seems to be legacy one
Reviewed By: DurhamG
Differential Revision: D8481013
fbshipit-source-id: 7c2c47f0b4e4737612416449c2a7c000ca09ab29
Summary:
Fix the `_collectimport()` code to more accurately figure out module paths.
When used from a statement like `import foo.bar` the `__import__()` function
returns the `foo` module. However, if used like `from foo.bar import baz` it
returns the `foo.bar` module. The `_collectimport()` code did not expect this
behavior and assumed that it always returned `foo`. This resulted in it doing
the wrong thing when trying to resolve nested imports.
In most cases it would just fail with an attribute error and it would then
swallow this exceptions. However in some cases it was able to find a
non-module object. For instance, when processing the `mercurial.extensions`
module it would find the `extensions()` function inside this module. Calling
`inspect.getabsfile()` on this function doesn't always produce the correct
result. If loaded from a pre-compiled .pyc file the function may have a
relative path baked in as its function name. The `inspect.getabsfile()` code
then simply prepends this with the current working directory, despite the fact
that this wasn't actually imported from the current directory.
I also changed the code to use inspect.getfile() rather than
inspect.getabsfile(). The getabsfile() function attempts to do other
normalization and string manipulation that we don't need. (It tries to find
`*.py` names rather than `*.pyc` or `*.pyo` names, and it normalizes case.)
Reviewed By: ryanmce, quark-zju
Differential Revision: D8601652
fbshipit-source-id: db9be3c5cbbea83a880851036093aaa28b5b9b19
Summary:
The chg code now always attempts to import __version__.py
Previously this was not generated during buck-based builds. This updates the
buck build to generate a __version__.py file now.
Reviewed By: ryanmce
Differential Revision: D8601650
fbshipit-source-id: 59bd0605394feeabcc41766a7e55cbdd253f07ad
Summary:
`size = -1` does not have special meaning in dirstate. Only `size = -2` is
special.
Reviewed By: DurhamG
Differential Revision: D8623318
fbshipit-source-id: 7293a7b492ba44cb8ffc49ce54a908e9612148e2
Summary:
The previous version of this tool raised on all merges, even non-conflicting ones.
We need to attempt a three-way merge first, _then_ raise an exception if it produces conflicts.
Reviewed By: DurhamG
Differential Revision: D8607184
fbshipit-source-id: 3f208caf54fa1ace28e1ee9011b34f5ec7bacec4
Summary:
For treestate, `copies` is more expensive as it's not a cache but calculates
from the source of truth. So avoid using `copies` but read the copy
information directly from the tree.
Reviewed By: DurhamG
Differential Revision: D8595357
fbshipit-source-id: bcea254358d66c9fa9f933d73221bbeb0bd8d5b2
Summary:
Let's advertise sparse to all the people who still need to hear about this.
This is behind config know. I think it's reasonable to enable it for ovrsource
and fbsource (possibly with different tresholds).
Reviewed By: quark-zju
Differential Revision: D8591973
fbshipit-source-id: 99d22705eb81e0b646b0c1537ddec395322069e6
Summary:
Some scripts are parsing all hg outputs. The remote stderr output could
contain random noise that break the scripts. Therefore let's forward
remote stderr to local stderr, instead of local stdout.
Reviewed By: DurhamG
Differential Revision: D8514952
fbshipit-source-id: 2f205fdc948fbeacd20b5af9d6d52eaa8212e90e
Summary: This will allow us to trigger hints from functions that wrap runcommand itself.
Reviewed By: quark-zju
Differential Revision: D8591974
fbshipit-source-id: e8e4742d6728ca347b85d10adc2e2ab7ab06ef16
Summary:
Using `wdir()` in dagrange expressions (e.g. `master::wdir()`) causes Mercurial
to crash with `IndexError: head out of range`. This is because the dagrange
operations can't use the wdir rev number as it is out of range for the
changelog.
Instead, modify the revisions that we are passing in to calculate the matching
revs, and add the wdirrev at the last moment if appropriate.
Reviewed By: ryanmce
Differential Revision: D8592129
fbshipit-source-id: 8742cb72c22d1ba26f5d03f9fcf56bfdb785c25a
Summary:
provide sockpath in the config, rather than calling `watchman get-sockname` command from python.
the config variable was already deployed on all mac
this improves hg st by 250-300 ms
(the logic gives more priority when env var if specified (the env var override is usually used for testing))
the logic has a retry mechanism if we can't connect to the socket path specified in the config
next step: deploy config on dev servers (but we will have only few ms speed up there)
we use %i (i for identity) notation for user
[298] → hg conf | grep sockpath
fsmonitor.sockpath=/opt/facebook/watchman/var/run/watchman/%i-state/sock
Reviewed By: farnz
Differential Revision: D8549130
fbshipit-source-id: 31df9ebf3eb5133d3e6f64794022cda1b0fadec4
Summary:
This config has been True everwhere for a while, and seems uncontroversial. Let's fold it in everywhere as True.
This simplifies a test case.
Reviewed By: DurhamG
Differential Revision: D8578289
fbshipit-source-id: f66a6427e435d80750b4da14358c5e4f0528b6f1
Summary:
It's exceeding unix domain socket length limit:
AssertionError: 2018-06-22T13:48:50,820: [] /tmp/hgtests.8M8JR4/child3/test-fb-hgext-treemanifest-prefetch.t-remotefilelog.true.shallowrepo.true.watchman/sock: path is too long
Talked with wez. Watchman does not want to workaround the issue by using
chdir like chg[1]. The main reason is changing cwd could break
thread-safety, especially when used as a library. chg is single-threaded and
does not have the concern.
[1]: c80c16a8a0
Reviewed By: DurhamG, singhsrb
Differential Revision: D8596333
fbshipit-source-id: 347a42fcd40d4ab075ca9c24f7c9ce4c8fa519b0
Summary:
D8560049 made `{reviewers}` work with commit template, but it breaks empty
`committemplate.reviewers` with `abort: "" not in template map`. This diff
fixes it.
Reviewed By: DurhamG
Differential Revision: D8596389
fbshipit-source-id: e1c046d59097181ef5c8d56a1cffc7c2407f0ba5
Summary:
Treestate tracks untracked files while treedirstate doesn't. The
treedirstate importmap logic changed by D7912557 will remove untracked files
from the tree. But it missed other places like nonnormalset, otherparentset
and copymap. This patch changes it to make sure paths in those sets are
tracked.
Reviewed By: DurhamG
Differential Revision: D8560022
fbshipit-source-id: 8ac39bfe634804ea94125fe42e11382acae35d94
Summary:
Previously `state._usetreestate` is a value set in `state.__init__`. In the
treestate migration case, `state._usetreestate` needs to change accordingly.
Therefore let's use a property to always calculate that value from the
source of truth.
Reviewed By: DurhamG
Differential Revision: D8560019
fbshipit-source-id: a313f603275321b959502e7f2c528c4b54a11ce9
Summary:
D8576826 broke the tests by allowing the importer to use lfs objects,
which therefore causes the lfs objects directory to exist, which breaks the
tests. Let's update the tests.
Reviewed By: quark-zju
Differential Revision: D8593012
fbshipit-source-id: 2897f55a8d75c3784d67f0d0e11870092f7dad71
Summary:
Revset predicates that work substantially through `subset.filter` should have a
higher weight than the default value of 1, to prevent the optimizer incorrectly
optimizing operations like:
```
reverse(ancestors(node, limit)) & filterpredicate()
```
in a way that filters all the revisions in the repo according the predicate
before performing the dag search.
Reviewed By: ikostia
Differential Revision: D8588027
fbshipit-source-id: 57be642761af013be8c8aa27ec126b3919c34256
Summary:
The expression `::revs & age("<x")` can be slow, as all ancestors must be
analyzed in case there are commits with ancestors that have later dates. Most
of this time this is unnecessary, as commits are usually monotonic in their
dates.
The `ancestorsaged` revsetpredicate evaluates to all ancestors within an age
range, optimized as if commit dates are monotonic.
Reviewed By: ikostia
Differential Revision: D8489319
fbshipit-source-id: 9b9055019c2afa70497a9df02c6fe3504e2c381a
Summary:
The `public()` revsetpredicate has the default weight of 1, which means
expressions like `::x & public()` are optimized to `('andsmally', ::x, public())`,
which will actually result in iterating over all commits in the repo.
Increase the weight to 3 to ensure it's optimized in a way that favours
following ancestry links and then checking the phase.
Reviewed By: ikostia
Differential Revision: D8549645
fbshipit-source-id: f58750ecb1918079b7e350ce27b7a61771489233
Summary:
Upstream Mercurial renamed peer.cleanup to peer._cleanup last August,
but our connectionpool logic wasn't updated. This meant that expired connections
were left alive and just hanging there until the process died. This defeated the
entire purpose of expiring connections (to free memory on the server).
Reviewed By: singhsrb
Differential Revision: D8583327
fbshipit-source-id: 561b6ed8b76247d996051b0536c5f58050fb83a2
Summary:
In the recent refactor we changed the definition of "is this tree
local" on the server to also include if the tree didn't exist in the store,
since then it might be an ondemand conversion from a flat manifest in a bundle.
This caused issues on peer-to-peer pulls, since most trees don't exist locally.
Let's fix it by only considering a missing tree to be a local tree if we're on
the server.
Reviewed By: quark-zju
Differential Revision: D8586779
fbshipit-source-id: ea8c8ee3e2a4dc054fae9caa122cfc733df1ab81
Summary:
The treemanifest manifest revlog store was keeping all the revlogs it
had ever opened. This was causing unnnecessary memory usage. Let's use an lru
dict instead.
Reviewed By: singhsrb
Differential Revision: D8579992
fbshipit-source-id: a0df30401714c93eebe0448b0ce92571f704bd96
Summary:
It seems the datapackstore c++ implementation is very leaky, which is
causing memory issues on the server. We're in the process of rewriting the
datapack logic to be pure rust, so let's just disable this feature for now.
Disabling this slows down tree downloads by 25%.
Reviewed By: quark-zju
Differential Revision: D8580519
fbshipit-source-id: 7e42e7ff211008d5461f95386da019ba5696e07b
Summary: It turned out that a waiting loop is necessary, at least on my laptop.
Reviewed By: DurhamG
Differential Revision: D8560023
fbshipit-source-id: 8efd8dca4c630a30e5ccb140eb4f2da8a0cc5717
Summary:
While writing the test, I noticed the test didn't error out as expected:
```
% ./run-tests.py test-treestate-downgrade.t -i --watchman
E
Errored test-treestate-downgrade.t: Traceback (most recent call last):
File "./run-tests.py", line 1064, in run
self.setUp()
File "./run-tests.py", line 1053, in setUp
raise RuntimeError("timed out waiting for watchman")
RuntimeError: timed out waiting for watchman
(exit code 0)
```
This means buck test which relies on exit code will treat those tests as
"successful". That's incorrect. So let's change the exit code.
Reviewed By: DurhamG
Differential Revision: D8560021
fbshipit-source-id: 4f98772099c559e59288e42165386f1069086377
Summary:
The template keyword only works for committed changesets. That conflicts
with committemplate. So special case the working copy case and make it work
if committemplate.reviewers is set.
Reviewed By: liubov-dmitrieva
Differential Revision: D8560049
fbshipit-source-id: 1427d0b72e023741be7012dd6eb4d51b0b595f39
Summary:
This avoids potential issues when `fork()` inheriting the socket fd could
cause unresponsive servers.
Reviewed By: simpkins
Differential Revision: D8384547
fbshipit-source-id: 91aeb76f4a7b526a39bbbf846a9e5c0546348f6c
Summary: Run tests with watchman built from fbcode master. So we get signals about watchman/hg changes.
Reviewed By: DurhamG
Differential Revision: D8448187
fbshipit-source-id: 8643f139932057f326bc75b0b0c5cc616355eeed
Summary:
After spawning the watchman process, there is a small window before the
watchman process listening on the specified socket file. If hg tests run
during that window, they might fail with watchman unavailable problems.
Therefore, change run-tests.py to wait for watchman by testing
`watchman version`.
Reviewed By: DurhamG
Differential Revision: D8523989
fbshipit-source-id: c5cb2366f938c0617fdd14d232f879b7cdc46b30
Summary:
The `repos` file contains paths to repos so `hg gc` can scan them. The
current implementation basically writes a line per `hg pull`, and `repos`
can grow unbounded. Let's do a check before appending to that file.
Reviewed By: ryanmce
Differential Revision: D8416210
fbshipit-source-id: 0624bfbd6eb7d71dd749626e96063cb10f55bb7f
Summary: Commit cloud state is stored in the store, so the wlock is not necessary.
Reviewed By: liubov-dmitrieva
Differential Revision: D8121600
fbshipit-source-id: f96f4764e3cf0637cd45a0bc83d1227318ac9083
Summary:
Analogous to `localrepo._wlockfreeprefix`, this is the prefix of files in the
store that can be safely accessed without the store lock.
Reviewed By: mitrandir77
Differential Revision: D8480907
fbshipit-source-id: 6d6c625c7f2230384ee361ff6b585f42feea6d10
Summary:
`/` in this case is semantically not a dir seaparator, but a bookmark namespace
separator, so let's use `posixpath`, rather than `os.path` to combine parts.
Reviewed By: StanislavGlebik
Differential Revision: D8477029
fbshipit-source-id: 964b7d3f8a560fb73dfaaa39757c091908af77c5
Summary:
Flat manifest backfilling (which is used when reverting from treeonly
mode) was failing when the client had public heads that didn't exist on the
server (for instance, if they pulled from svn directly or if they had old
commits that had been stripped from the server). This was breaking the pull.
Let's fix it so we never send unknown heads to the server during the backfill
pull.
Reviewed By: phillco
Differential Revision: D8526027
fbshipit-source-id: 8d5bf67d2022dbc24facd0b96e001c8c81a0a2da
Summary: Added to commonancestorshead function explaining how the algorithm works.
Reviewed By: DurhamG
Differential Revision: D8415752
fbshipit-source-id: 68e77b0b1f7afb54b88b77084eb371abeeae8ab0
Summary:
test-fb-hgext-pushrebase-manifests.t is flaky, with occasional blank lines
sometimes appearing in the output. This is another location that recently
caused a build failure.
Reviewed By: ryanmce
Differential Revision: D8517151
fbshipit-source-id: 38e7e87745a971d9734b86bf83b3c0a86d3ba797
Summary: This adds a merge tool to only rebase if there aren't conflicts, which I think will be widely useful. It's for IMM only at the moment.
Reviewed By: DurhamG, quark-zju
Differential Revision: D8493065
fbshipit-source-id: 32ccc0e53af27d275a56bcac80861a421d7abfc6
Summary:
hg filename encoding will double the length of upper-case paths. That could
be problematic if there is a long file name with many upper-case characters.
Workaround it by detecting the long name case, and keep upper-case
unchanged.
This unblocks a production issue where a long upper-case file name was
committed to the source of truth and the hg mirror cannot import it. But it
makes Windows hosting less "correct".
Reviewed By: DurhamG
Differential Revision: D8527475
fbshipit-source-id: da25fcce7e3173769788bd164629c28466158cd2
Summary:
This was added to ease the IMM rollout, but today IMM rarely raises new exception, and this code fires on real errors that should be raised -- you can see here this lets us remove many hacky workarounds.
Some of the exceptions that didn't have carve-outs were:
- interrupted update
- source and destination form a cycle
- rebase in progress
Reviewed By: DurhamG, quark-zju
Differential Revision: D8522466
fbshipit-source-id: b407807ea059a0861fb3caca1db8699242342075
