This means that the state will not be written if:
(1) either the wlock can't be obtained
(2) something else came along and changed the dirstate while we were in the
middle of a status run.
fsmonitor and debugignore currently access matcher fields that I would
consider implementation details, namely patternspat, includepat, and
excludepat. Let' instead implement __repr__() and have the few users
use that instead.
Marked (API) because the fields can now be None.
Everyone knows that it's supposed to be spelled with two asterisks.
It started failing in 7013de107975 (update: accept --merge to allow
merging across topo branches (issue5125), 2017-02-13) because until
then there was only one argument that was covered by the kwargs, so
*kwargs or **kwargs both worked (or at least that's what I think with
my limited understanding of Python).
The state-enter command may not have been successful; for example, the watchman
client session may have timed out if the user was busy/idle for a long period
during a merge conflict resolution earlier in processing a rebase for a stack
of diffs.
It's cleaner (from the perspective of the watchman logs) to avoid issuing the
state-leave command in these cases.
Test Plan:
ran
`hg rebase --tool :merge -r '(draft() & date(-14)) - master::' -d master`
and didn't observe any errors in the watchman logs or in the output from
`watchman -p -j <<<'["subscribe", "/data/users/wez/fbsource", "wez", {"expression": ["name", ".hg/updatestate"]}]'`
we see some weird things in the watchman logs where the mercurial
process is seemingly confused about which hg.update state it is publishing
through watchman.
On closer examination, we're seeing conflicting pids for the clients involved
and this implies a race.
To resolve this, we extend the wlock around the state-enter/state-leave
events that are emitted to watchman.
Test Plan:
Some manual testing:
In one window, run this, and then checkout a different rev:
```
$ watchman -p -j <<<'["subscribe", "/data/users/wez/fbsource", "wez", {"expression": ["name", ".hg/updatestate"]}]'
{
"version": "4.9.0",
"subscribe": "wez",
"clock": "c:1495034090:814028:1:312576"
}
{
"state-enter": "hg.update",
"version": "4.9.0",
"clock": "c:1495034090:814028:1:312596",
"unilateral": true,
"subscription": "wez",
"metadata": {
"status": "ok",
"distance": 125,
"rev": "a1275d79ffa6c58b53116c8ec401c275ca6c1e2a",
"partial": false
},
"root": "/data/users/wez/fbsource"
}
{
"root": "/data/users/wez/fbsource",
"metadata": {
"status": "ok",
"distance": 125,
"rev": "a1275d79ffa6c58b53116c8ec401c275ca6c1e2a",
"partial": false
},
"subscription": "wez",
"unilateral": true,
"version": "4.9.0",
"clock": "c:1495034090:814028:1:312627",
"state-leave": "hg.update"
}
```
Tailed the watchman log file and looked for invalid state assertion errors,
then ran my `rebase-all` script to update/rebase all of my heads.
Didn't trigger the error condition (but couldn't reliably trigger it previously
anyway), and the output captured above shows that the states are being emitted
correctly.
It seems like fsmonitor/__init__.py was based on a pretty old version
of dirstate.py. Let's copy over the changes from the following two
commits:
1161b515cc4d (match: add isexact() method to hide internals, 2014-10-29)
b1d8372ab1d0 (dirstate: avoid match.files() in walk(), 2015-05-19)
In the future, chg may prefill repo's dirstate filecache so it's valuable
and should be kept. Previously we drop both filecache and property cache for
dirstate during fsmonitor reposetup, this patch changes it to only drop
property cache but keep the filecache.
watchman's paths encoding can differ from filesystem encoding. For example,
on Windows, it's always utf-8.
Before this patch, on Windows, mismatch in path comparison between fsmonitor
state and osutil.statfiles would yield a clean status for added/modified files.
In addition to status reporting wrong results, this leads to files being
discarded from changesets while doing history editing operations such as rebase.
Benchmark:
There is a little overhead at module import:
python -m timeit "import hgext.fsmonitor"
Windows before patch: 1000000 loops, best of 3: 0.563 usec per loop
Windows after patch: 1000000 loops, best of 3: 0.583 usec per loop
Linx before patch: 1000000 loops, best of 3: 0.579 usec per loop
Linux after patch: 1000000 loops, best of 3: 0.588 usec per loop
10000 calls to _watchmantofsencoding:
python -m timeit -s "from hgext.fsmonitor import _watchmantofsencoding, _fixencoding" "fname = '/path/to/file'" "for i in range(10000):" " if _fixencoding: fname = _watchmantofsencoding(fname)"
Windows (_fixencoding is True): 100 loops, best of 3: 19.5 msec per loop
Linux (_fixencoding is False): 100 loops, best of 3: 3.08 msec per loop
Earlier this was left thinking that its part of pywatchman package.
This patch replaces variables os.sep, sys.platform and os.envrion with their
py3 compatible ones.
I've caught multiple extensions in the wild lying about being
'internal', so it's time to move the goalposts on people. Goalpost
moving will continue until third party extensions stop trying to
defeat the system.
All versions of Python we support or hope to support make the hash
functions available in the same way under the same name, so we may as
well drop the util forwards.
Since (b) is banned, we should do the same for (a) for consistency.
a) from mercurial import hg
from mercurial.i18n import _
b) from . import hg
from .i18n import _
Keeping the codebase in sync with upstream:
Watchman 4.4 introduced an advanced settling feature that allows publishing
tools to notify subscribing tools of the boundaries for important filesystem
operations.
https://facebook.github.io/watchman/docs/cmd/subscribe.html#advanced-settling
has more information about how this feature works.
This diff connects a signal that we're calling `hg.update` to the mercurial
update function so that mercurial can indirectly notify tools (such as IDEs or
build machinery) when it is changing the working copy. This will allow those
tools to pause their normal actions as the files are changing and defer them
until the end of the operation.
In addition to sending the enter/leave signals for the state, we are able to
publish useful metadata along the same channel. In this case we are passing
the following pieces of information:
1. destination revision hash
2. An estimate of the distance between the current state and the target state
3. A success indicator.
4. Whether it is a partial update
The distance is estimate may be useful to tools that wish to change their
strategy after the update has complete. For example, a large update may be
efficient to deal with by walking some internal state in the subscriber rather
than feeding every individual file notification through its normal (small)
delta mechanism.
We estimate the distance by comparing the repository revision number. In some
cases we cannot come up with a number so we report 0. This is ok; we're
offering this for informational purposes only and don't guarantee its accuracy.
The success indicator is only really meaningful when we generate the
state-leave notification; it indicates the overall success of the update.
Extension to plug into a Watchman daemon, speeding up hg status calls by
relying on OS events to tell us what files have changed.
Originally developed at https://bitbucket.org/facebook/hgwatchman
