Followup 814eb5a11da4 to provide complete context for proper localization.
Also update cmdutil.recordfilter docstring to remove recommendation that
"operation" argument should be translated. Indeed, for record/revert, we
either go to patch.filterpatch or crecord.filterpatch (in curses mode) ; the
former now build the full ui message from the operation parameter and the
latter does not use this parameter (removing in a followup patch). For shelve,
operation is not specified and this thus falls back to "record".
The cPickle is renamed to _pickle in python3 and this C extension is available
in pickle which was not included in earlier versions. So imports are conditionalized
to import cPickle in py2 and pickle in py3. Moreover the use of pickle in py2 is
switched to cPickle as the C extension is faster. The hack is added in util.py and
the modules import util.pickle
9981d464ac53 fixed matching() to preserve the order of the input set, but
the test was incorrect. Given "A and B", "A" should be the input set to "B".
But thanks to our optimizer, the test expression was rewritten as
"(2 or 3 or 1) and matching(1 or 2 or 3)", therefore it was working well.
Since I'm going to fix the overall ordering issue, the test needs to be
adjusted to do the right thing.
This fix allows __nonzero__ to respect the direction of iteration of the
whole filteredset. Here's the case when it matters. Imagine that we have a
very large repository and we want to execute a command like:
$ hg log --rev '(tip:0) and user(ikostia)' --limit 1
(we want to get the latest commit by me).
Mercurial will evaluate a filteredset lazy data structure, an
instance of the filteredset class, which will know that it has to iterate
in a descending order (isdescending() will return True if called). This
means that when some code iterates over the instance of this filteredset,
the 'and user(ikostia)' condition will be first checked on the latest
revision, then on the second latest and so on, allowing Mercurial to
print matches as it founds them. However, cmdutil.getgraphlogrevs
contains the following code:
revs = _logrevs(repo, opts)
if not revs:
return revset.baseset(), None, None
The "not revs" expression is evaluated by calling filteredset.__nonzero__,
which in its current implementation will try to iterate the filteredset
in ascending order until it finds a revision that matches the 'and user(..'
condition. If the condition is only true on late revisions, a lot of
useless iterations will be done. These iterations could be avoided if
__nonzero__ followed the order of the filteredset, which in my opinion
is a sensible thing to do here.
The problem gets even worse when instead of 'user(ikostia)' some more
expensive check is performed, like grepping the commit diff.
I tested this fix on a very large repo where tip is my commit and my very
first commit comes fairly late in the revision history. Results of timing
of the above command on that very large repo.
-with my fix:
real 0m1.795s
user 0m1.657s
sys 0m0.135s
-without my fix:
real 1m29.245s
user 1m28.223s
sys 0m0.929s
I understand that this is a very specific kind of problem that presents
itself very rarely, only on very big repositories and with expensive
checks and so on. But I don't see any disadvantages to this kind of fix
either.
Cached attribute repo._phasecache uses stat of '.hg/phaseroots' file
to examine validity of cached contents. If writing '.hg/phaseroots'
file out keeps ctime, mtime and size of it, change is overlooked, and
old contents cached before change isn't invalidated as expected.
To avoid ambiguity of file stat, this patch writes '.hg/phaseroots'
file out with checkambig=True.
This patch is a part of "Exact Cache Validation Plan":
https://www.mercurial-scm.org/wiki/ExactCacheValidationPlan
Cached attribute dirstate._branch uses stat of '.hg/branch' file to
examine validity of cached contents. If writing '.hg/branch' file out
keeps ctime, mtime and size of it, change is overlooked, and old
contents cached before change isn't invalidated as expected.
To avoid ambiguity of file stat, this patch writes '.hg/branch' file
out with checkambig=True.
This patch is a part of "Exact Cache Validation Plan":
https://www.mercurial-scm.org/wiki/ExactCacheValidationPlan
Cached attribute repo.dirstate uses stat of '.hg/dirstate' file to
examine validity of cached contents. If writing '.hg/dirstate' file
out keeps ctime, mtime and size of it, change is overlooked, and old
contents cached before change isn't invalidated as expected.
To avoid ambiguity of file stat, this patch writes '.hg/dirstate' file
out with checkambig=True.
The former diff hunk changes the code path for "dirstate.write()", and
the latter changes the code path for "dirstate.savebackup()".
This patch is a part of "Exact Cache Validation Plan":
https://www.mercurial-scm.org/wiki/ExactCacheValidationPlan
Cached attribute repo._bookmarks uses stat of '.hg/bookmarks' and
'.hg/bookmarks.current' files to examine validity of cached
contents. If writing these files out keeps ctime, mtime and size of
them, change is overlooked, and old contents cached before change
isn't invalidated as expected.
To avoid ambiguity of file stat, this patch writes '.hg/bookmarks' and
'.hg/bookmarks.current' files out with checkambig=True.
This patch is a part of "Exact Cache Validation Plan":
https://www.mercurial-scm.org/wiki/ExactCacheValidationPlan
Files below, which might be changed at closing transaction, are used
to examine validity of cached properties. If changing keeps ctime,
mtime and size of a file, change is overlooked, and old contents
cached before change isn't invalidated as expected.
- .hg/bookmarks
- .hg/dirstate
- .hg/phaseroots
To avoid ambiguity of file stat, this patch writes files out with
checkambig=True at closing transaction.
checkambig becomes True only at closing (= 'not suffix'), because stat
information of '.pending' file isn't used to examine validity of
cached properties.
This patch is a part of "Exact Cache Validation Plan":
https://www.mercurial-scm.org/wiki/ExactCacheValidationPlan
Before, we would always print the unprefixed SHA-1 fingerprint when
fingerprint comparison failed. Now, we print the fingerprint of the
last hash used, including the prefix if necessary. This helps ensure
that the printed hash type matches what is in the user configuration.
There are still some cases where this can print a mismatched hash type.
e.g. if there are both SHA-1 and SHA-256 fingerprints in the config,
we could print a SHA-1 hash if it comes after the SHA-256 hash. But
I'm inclined to ignore this edge case.
While I was here, the "section" variable assignment has been moved to
just above where it is used because it is now only needed for this
error message and it makes the code easier to read.
The previous warning and abort messages were difficult to understand.
This patch makes them slightly better.
I think there is still room to tweak the messaging. And as we adopt
new security defaults, these messages will certainly change again.
But at least this takes us a step in the right direction.
References to "section" have been removed because if no fingerprint
is defined, "section" can never be "hostfingerprints." So just print
"hostsecurity" every time.
We didn't need to use a temporary variable to indicate success because
we just return anyway.
This refactor makes the code simpler. While we're here, we also call
into formatfingerprint() to ensure the fingerprint from the proper
hashing algorithm is logged.
The world is starting to move on from SHA-1. A few commits ago, we
gained the ability to define certificate fingerprints using SHA-256
and SHA-512.
Let's start printing the SHA-256 fingerprint instead of the SHA-1
fingerprint to encourage people to pin with a more secure hashing
algorithm.
There is still a bit of work to be done around the fingerprint
messaging. This will be addressed in subsequent commits.
A short time ago, validatesocket() didn't know the reasons why
cert verification was disabled. Multiple code paths could lead
to cert verification being disabled. e.g. --insecure and lack
of loaded CAs.
With the recent refactorings to sslutil.py, we now know the reasons
behind security settings. This means we can recognize when the user
requested security be disabled (as opposed to being unable to provide
certificate verification due to lack of CAs).
This patch moves the check for certificate verification being disabled
and changes the wording to distinguish it from other states. The
warning message is purposefully more dangerous sounding in order
to help discourage people from disabling security outright.
We may want to add a URL or hint to this message. I'm going to wait
until additional changes to security defaults before committing to
something.
There are various tests for behavior when CA certs aren't loaded.
Previously, we would pass --insecure to disable loading of CA
certs. This has worked up to this point because the error message
for --insecure and no CAs loaded is the same. Upcoming commits will
change the error message for --insecure and will change behavior
when CAs aren't loaded.
This commit introduces the ability to disable loading of CA certs
by setting devel.disableloaddefaultcerts. This allows a testing
backdoor to disable loading of CA certs even if system/default
CA certs are available. The flag is purposefully not exposed to
end-users because there should not be a need for this in the wild:
certificate pinning and --insecure provide workarounds to disable
cert loading/validation.
Tests have been updated to use the new method. The variable used
to disable CA certs has been renamed because the method is not
OS X specific.
This patch effectively moves the ui.insecureconnections check to
_hostsettings(). After this patch, validatesocket() no longer uses the
ui instance for anything except writing messages.
This patch also enables us to introduce a per-host config option
for disabling certificate verification.
smtp.verifycert was accidentally broken by 799db3fe9866. And,
I believe the "loose" value has been broken for longer than that.
The current code refuses to talk to a remote server unless the
CA is trusted or the fingerprint is validated. In other words,
we lost the ability for smtp.verifycert to lower/disable security.
There are special considerations for smtp.verifycert in
sslutil.validatesocket() (the "strict" argument). This violates
the direction sslutil is evolving towards, which has all security
options determined at wrapsocket() time and a unified code path and
configs for determining security options.
Since smtp.verifycert is broken and since we'll soon have new
security defaults and new mechanisms for controlling host security,
this patch formally deprecates smtp.verifycert. With this patch,
the socket security code in mail.py now effectively mirrors code
in url.py and other places we're doing socket security.
For the record, removing smtp.verifycert because it was accidentally
broken is a poor excuse to remove it. However, I would have done this
anyway because smtp.verifycert is a one-off likely used by few people
(users of the patchbomb extension) and I don't think the existence
of this seldom-used one-off in security code can be justified,
especially when you consider that better mechanisms are right around
the corner.
Before this commit bare update --clean on newly created branch
updates to the parent commit, even if there are later commits
on the parent commit's branch. Update to the latest head on the
parent commit's branch instead.
This seems reasonable as clean should discard uncommited changes,
branch is one of them.
Instead of "record this change to 'FILE'?" now prompt with:
* "discard this change to 'FILE'?" when reverting to the parent of working
directory, and,
* "revert this change to 'FILE'?" otherwise.
This patch makes repquote() distinguish more characters below, as a
preparation for exact detection in subsequent patch.
- "%" as "%"
- "\\" as "b"(ackslash)
- "*" as "A"(sterisk)
- "+" as "P"(lus)
- "-" as "M"(inus)
Characters other than "%" don't use itself as replacement, because
they are treated as special ones in regexp.
d19c9c93ad10 tried to detect '.. note::' more exactly. But
implementation of it seems not correct, because:
- fromc.find(c) returns -1 for other than "." and ":"
- tochr[-1] returns "q" for such characters, but
- expected result for them is "o"
This patch uses dict to manage replacement instead of replacing
str.find() by str.index(), for improvement/refactoring in subsequent
patches. Examination by fixedmap is placed just after examination for
' ' and '\n', because subsequent patch will integrate the latter into
the former.
This patch also changes regexp for 'string join across lines with no
space' rule, and adds detailed test for it, because d19c9c93ad10 did:
- make repquote() distinguish "." (as "p") and ":" (as "q") from
others (as "o"), but
- not change this regexp without any reason (in commit log, at
least), even though this regexp depends on what "o" means
This patch doesn't focuses on deciding whether "." and/or ":" should
be followed by whitespace or not in translatable messages.
It doesn't make sense to run test-chg.t without chg, so ignore it with vanilla
hg, and specify chg executable explicitly.
test-chg.t can host chg-specific tests.
Those assertions will prevent the backup functions from overwriting
the dirstate file in case both: suffix and prefix are empty.
(foozy suggested making that change and I agree with him)
