match.match already interprets "!bool(patterns)" as matching
everything (but includes and excludes still apply). We might as well
allow None, which lets us simplify some callers a bit.
I originally wrote this patch while trying to change
match.match(patterns=[]) to mean to match no patterns. This patch is
one step towards that goal. I'm not sure it'll be worth the effort to
go all the way there, but I think this patch still makes sense on its
own.
Most of it does the same as its superclass, so it can simply be
removed. It also seems to make more sense for it to use relative
paths, as we do for everything except alwaysmatcher, although
nevermatcher.uipath() will probably never get called anyway, so it
won't matter.
c01965ab5195 introduced a deterministic `__repr__` for ignores. However, it
didn't account for when ignore was `util.never`. This broke fsmonitor's ignore
change detection -- with an empty hgignore, it would kick in all the time.
Introduce `nevermatcher` and switch to it. This neatly parallels
`alwaysmatcher`.
This moves the optimization for patterns that match everything to the
caller, so we can remove it from patternmatcher.
Note that we need to teach alwaysmatcher to use relative paths now in
cases like "hg files .." from inside mercurial/, because while it
still matches everything, paths should be printed relative to the
working directory.
By passing in the result of the normalize() call, we prepare for
moving the special handling of patterns that always match out of the
patternmatcher.
It also lets us remove many of the arguments from the matcher, because
they were passed only the the normalize function (we could have
removed the arguments by binding them to the function instead of
moving the normalize() call out).
Since the caller now deals with empty pattern lists, we can drop that
support in the patternmatcher. It now gets the more logical behavior
of matching nothing when no patterns are given (although there is no
in-core caller that will pass no patterns).
In patternmatcher, we used to say that all directories should be
visited if no explicit files were listed, because the case of empty
_files usually implied that no patterns were given (which in turns
meant that everything should match). However, this made e.g. "hg files
-r . rootfilesin:." slower than necessary, because that also ended
up with an empty list in _files. Now that patternmatcher does not
handle includes, the only remaining case where its _files/_fileset
fields will be empty is when it's matching everything. We can
therefore treat the always-case specially and stop treating the empty
_files case specially. This makes the case mentioned above faster on
treemanifest repos.
Having a special matcher that always matches seems to make more sense
than making one of the other matchers handle the case. For now, we
just use this new matcher when no patterns were provided.
Should at least be useful for debugging. Would matter for correctness
too if fsmonitor or Facebook's sparse extension worked with subrepos
(which I don't know if they do).
The includematcher will always get at least one include pattern and
will never get any non-include patterns, so we can remove most of the
code in it. This patch does mostly straight-forward deletions of
code. We will clean up further later.
At this point the includematcher is an exact copy of the main matcher
class. We will specialize and simplify both classes in the following
patches. This initial unmodified copy is just to make the differences
clearer. We also rename the main matcher to "patternmatcher" for
consistency.
I may eventually merge this new includematcher back into the main
matcher, but I think doing it this way makes the intermediate steps
clearer regardless.
Exact matching is now handled by the exactmatcher class.
We can safely remove _files from the __repr__() implementation,
because even though the field is set, the patternspat field is enough
for the representation to be unambiguous (which was not the case when
the matcher could handle exact matches).
Even though most matchers will always want to use the relative path in
uipath(), when we add support for intersecting matcher, we will want
to control which form to use for any kind of matcher without knowing
the type (see next patch), so we need the implementation on the base
class.
Also rename the attribute from "pathrestricted" to "relativeuipath"
since there actually are cases where we match everything but still use
relative paths (like when the user runs "hg files .." from inside
mercurial/).
As I've said on earlier patches, I'm hoping to use more composition of
simpler matchers instead of the single complex matcher we currently
have. This extracts a first new matcher that composes two other
matchers. It matches if the first matcher matches but the second does
not. As such, we can use it for excludes, which this patch also
does. We'll remove the now-unncessary code for excludes in the next
patch.
I'm hoping to rewrite the matcher so excludes are handled by
composition of one matcher with another matcher where the second
matcher has only includes. For that to work, we need to make
visitdir() to return 'all' for directory 'foo' for a '-I foo' matcher.
This makes the subdirmatcher not depend on the main matcher, giving us
more freedom to modify that (specifically, it will lose it _always
field in a while).
We will soon start splitting up the current matcher class into more
specialized classes, so we'll want a base class for all the things
that don't vary much between different matchers.
Exact matchers are only created internally (as opposed to from user
input) based on a set of files that the caller collected before, so
they should always match the list exactly (i.e. case-sensitively).
fsmonitor and debugignore currently access matcher fields that I would
consider implementation details, namely patternspat, includepat, and
excludepat. Let' instead implement __repr__() and have the few users
use that instead.
Marked (API) because the fields can now be None.
match() will soon gain more logic and we don't want to duplicate that
in icasefsmatch(), so merge the two functions instead and use a flag
to get case-insensitive behavior.
On case-insensitive file systems, if file A exists and you try to
remove it (or add, etc.) by specifying a different case, you will see
something like this:
$ hg rm a
removing file A
I honestly found this surprising because it seems to me like it was
explicitly listed by the user. Still, there is a comment in the code
describing it, so it is very clearly intentional. The code was added
in d70aa474bd84 (match: add a subclass for dirstate normalizing of the
matched patterns, 2015-04-12).
I'm going to do a lot of refactoring to matchers and the feature
mentioned above is going to get in my way. I'm therefore removing it
for the time being and we can hopefully add it back when I'm done.
matcher._normalize() no longer depends on any of the matcher's state,
and making it static will enable further refactoring. Note that the
subdirmatcher subclass calls _normalize(), so we can't remove it
completely.
No other arguments are passed via the matcher's state, so we should
treat 'warn' the consistently. More importantly, this will let us make
it a static function, which will help with further refactoring.
The matcher class is getting hard to understand. It will be easier to
follow if we can break it up into simpler matchers that we then
compose. I'm hoping to have one matcher that accepts regular
(non-include) patterns, one for exact file matches, one that always
matches (and maybe one that never does) and then compose them by
intersection and difference.
This patch takes a simple but important step towards that goal by
making match.match() a function (and renaming the matcher class itself
from "match" to "matcher"). The new function will eventually be
responsible for creating the simple matchers and composing them.
icasefsmatcher similarly gets a factory function (called
"icasefsmatch"). I also moved the other factory functions nearby.
It seems like the subdirmatcher should be checking if the matcher it's
based on is matching prefixes. It was effectively doing that already
because "prefix() == not always() and not anypats() and not
isexact()", subdirmatcher was checking the first two parts of that
condition and I don't think it will ever be given an "exact" matcher
with it's directory name (because exact matchers are for matching
files, not directories). Still, let's switch to using prefix() for
clarity (and because I'm trying to remove code that reaches for
matchers internals).
Because _rootsanddirs() returns a list of directories to visit
recursively and a list of directories to visit non-recursively. For
patterns such as 'rootfilesin:foo/bar', we clearly need to visit the
directory foo/bar, but we also need to visit its parents. The method
therefore uses util.dirs() to find the parent directories of
'foo/bar'. That method does not include the root directory, but since
we obviously need to visit the root directory, we always added '.' to
the set of directories to visit non-recursively.
The visitdir() method had special handling to consider set(['.']) to
mean that no includes had been specified and would thus visit all
directories. However, when the pattern is 'rootfilesin:.', set(['.'])
is actually the real set of directories to visit and the special
handling of that set meant that all directories got visited instead of
just the root directory.
The fix is simple: add '.' to the set of parent directories in
_rootsanddirs() and stop treating set(['.']) specially. This makes
hg files -r . -I rootfilesin:.
in a treemanifest version of the Firefox repo go from 1.5s to 0.26s on
warm disk (and a *much* bigger improvement on cold disk).
Note that the -I is necessary for no good reason. We just haven't
optimized visitdir() for regular (non-include, non-exclude) patterns
yet.
The matcher subinclude functionality allows us to have .hgignore files that
include subdirectory hgignore files. Today it parses the entire repo at once,
even if we only need to test a file in one subdirectory. This patch makes the
subinclude tree creation lazy, which speeds up matcher creation significantly in
large repos with very large trees of ignore patterns.
Changeset ba7f2a1cc2d2 removed the mutable default value, but did not explicitly
tested for None. Such implicit testing can introduce semantic and performance
issue. We move to an explicit testing for None as recommended by PEP8:
https://www.python.org/dev/peps/pep-0008/#programming-recommendations
There shouldn't be a big perf hit creating a new object because
this function is complicated and does things that dwarf the cost
of creating a new PyObject.
