Summary:
Remove ui.progress as a method of updating progress. All progress bars now go
through new-style progress bars.
This also splits out the rendering of progress bars from the reporting of
progress. All tests are updated to use new-style debug progress bars, which
simply report the position of the progress bar. Rendering of progress bars
will be tested separately once the progress bar engine has been rewritten.
Reviewed By: quark-zju
Differential Revision: D7329488
fbshipit-source-id: 14f8ab67365ddd98b74986aa25d9abc7a0546144
Summary:
When running with a Python runtime with a slightly different zlib module,
some `zlib.compress` outputs are different. Some tests are testing the
length, or the content of `zlib.compress` output, directly or indirectly.
That's causing issues.
This patch adds a `common-zlib` hghave test so it can be used to gate tests
checking zlib output. Some lengths are also changed to glob patterns to be
compatible.
Reviewed By: ryanmce
Differential Revision: D6937735
fbshipit-source-id: 2328a39d7f2022f16d51f61b6178568b26dfe2fb
Summary: Also change the internal API so it no longer accepts the "heads" argument.
Reviewed By: ryanmce
Differential Revision: D6745865
fbshipit-source-id: 368742be49b192f7630421003552d0a10eb0b76d
# skip-blame because this was mechanically rewritten the following script. I
ran it on both *.t and *.py, but none of the *.py changes were proper. All *.t
ones appear to be, and they run without addition failures on both Windows and
Linux.
import argparse
import os
import re
ap = argparse.ArgumentParser()
ap.add_argument('path', nargs='+')
opts = ap.parse_args()
globre = re.compile(r'^(.*) \(glob\)(.*)$')
for p in opts.path:
tmp = p + '.tmp'
with open(p, 'rb') as src, open(tmp, 'wb') as dst:
for line in src:
m = globre.match(line)
if not m or '$LOCALIP' in line or '*' in line:
dst.write(line)
continue
if '?' in line[:-3] or ('?' in line[:-3] and line[-3:] != '(?)'):
dst.write(line)
continue
dst.write(m.group(1) + m.group(2) + '\n')
os.unlink(p)
os.rename(tmp, p)
Upon pull or unbundle, we display a message with the range of new revisions
fetched. This revision range could readily be used after a pull to look out
what's new with 'hg log'. The algorithm takes care of filtering "obsolete"
revisions that might be present in transaction's "changes" but should not be
displayed to the end user.
This seems like a prudent thing to do. As the inline comment says,
we may want to make this abort once the functionality is stabilized
as part of `hg bundle`. Let's save that debate for another day.
The python hooks have access to the hook type information. There is not reason
for external hook to not be aware of it too.
For the record my use case is to make sure a hook script is configured for the
right type.
Hooks related to the transaction are aware of the transaction id. By definition
this txn-id is unique and different for each transaction. As a result it can
never be predicted in test and always needs matching. As a result, touching any
like with this data is annoying. We solve the problem once and for all by
installing an automatic replacement. In test, this will now show as:
TXNID=TXN:$ID$
revlog.compress() compares the compressed size to the input size
and throws away the compressed data if it is larger than the input.
This is the correct thing to do, as storing compressed data that
is larger than the input takes up more storage space and makes reading
slower.
However, the comparison was implemented inconsistently. For the
streaming compression mode, we threw away the result if it was
greater than or equal to the input size. But for the one-shot
compression, we threw away the compression only if it was greater
than the input size!
This patch changes the comparison for the simple case so it is
consistent with the streaming case.
As a few tests demonstrate, this adds 1 byte to some revlog entries.
This is because of an added 'u' header on the chunk. It seems
somewhat wrong to increase the revlog size here. However, IMO the cost
of 1 byte in storage is insignificant compared to the performance gains
of avoiding decompression. This patch should invite questions around
the heuristic for throwing away compressed data. For example, I'd argue
we should be more liberal about rejecting compressed data, additionally
doing so where the number of bytes saved fails to reach a threshold.
But we can have this discussion another time.
If environment variable looks like PATH or so (e.g. any of components
joined by ":" contains "/"), ":" in it is replaced with ";" by MinGW
at spawning Windows native process, to follow path concatenation style
of Windows.
Therefore, "bundle:../full.hg" is converted into "bundle;..\full.hg"
on MinGW.
Difference between "/" and "\" is automatically ignored by "(glob)",
but difference between ":" and ";" should be globed explicitly.
On Windows platform, invoking printenv.py directly via hook is
problematic, because:
- unless binding between *.py suffix and python runtime, application
selector dialog is displayed, and running test is blocked at each
printenv.py invocations
- it isn't safe to assume binding between *.py suffix and python
runtime, because application binding is easily broken
For example, installing IDE (VisualStudio with Python Tools, or
so) often requires binding between source files and IDE itself.
This patch invokes printenv.py via sh -c for test portability. This is
a kind of follow up for 9e4331825bea, which eliminated explicit
"python" for printenv.py. There are already other 'sh -c "printenv.py"'
in *.t files, and this fix should be reasonable.
This changes were confirmed in cases below:
- without any application binding for *.py suffix
- with binding between *.py suffix and VisualStudio
This patch also replaces "echo + redirection" style with "heredoc"
style, because:
- hook command line is parsed by cmd.exe as shell at first, and
- single quotation can't quote arguments on cmd.exe, therefore,
- "printenv.py foobar" should be quoted by double quotation, but
- nested quoting (or tricky escaping) isn't readable
Before this patch, streamclone-ed changes are invisible via @filecache
properties to in-process procedures before closing transaction
(e.g. pretxnclose python hook), if corresponded property is cached
before consumev1(). Strictly speaking, caching should occur inside
(store) lock for transaction.
repo.invalidate() after closing transaction is too late to force
@filecache properties to be reloaded from changed files at next
access.
For visibility of streamclone-ed changes to in-process procedures
before closing transaction, this patch clears caches just after
writing changes into files.
BTW, regardless of changing in this patch, clearing cached properties
in consumev1() causes inconsistency, if (1) transaction is started and
(2) any @filecache property is changed before consumev1().
This patch also adds the comment to fix this (potential) inconsistency
in the future.
The bundle2 changegroup part has an advisory param saying how many
changesets are in the part. Before this patch, we were setting
this part when generating bundle2 parts via the wire protocol but
not when generating local bundle2 files.
A side effect of not setting the changeset count part is that progress
bars don't work when applying changesets. As the tests show, this
impacted clone bundles, shelve, backup bundles, `hg unbundle`, and
anything touching bundle2 files.
This patch adds a backdoor to allow us to pass state from
changegroup generation into the unbundler. We store the number
of changesets in the changegroup in this state and use it to
populate the aforementioned advisory part parameter when generating
the bundle2 bundle.
I concede that I'm not thrilled by how state is being passed in
changegroup.py (it feels a bit hacky). I would love to overhaul the
rather confusing set of functions in changegroup.py with something that
passes rich objects around instead of e.g. low-level generators.
However, given the code freeze for 3.9 is imminent, I'd rather not
undertake this endeavor right now. This feels like the easiest way
to get the parameter added to the changegroup part.
Windows command lines use double quotes to quote arguments with spaces.
This change is in a series to unify around using single quotes around
commands, and double quotes around interior arguments.
Now its done silently, so unless user really knows what he is doing
will be suprised to find that after update 'hg status' doesn't work.
This commit makes also merge operation warns about missing parent when
revision to merge exists only in the bundle.
Previously, if you passed a revset that resolved to no nodes, it would get
interpreted by the changegroup discovery logic as 'bundle all my heads', which
is not what the user asked.
Let's exit early when we notice this case.
It could be argued that the changeset discovery logic should be smarter and only
assume 'all heads' if the incoming heads parameter is None, but that's a much
riskier change.
Sometimes a txnclose or changegroup hook wants to iterate through all
the changesets in transaction: in that situation usually the revset
`$HG_NODE:` is used to select the revisions. Unfortunately this revset
sometimes may contain too many changesets because we don't have the
write lock while the hook runs newer changes may be added to
repository in the meantime.
That's why there is a need for extra variable carrying the information about
the last change in the transaction.
In the most complex case, we try using the incoming delta base, then
we try both parents, and then we try the previous revlog entry. If
none of these result in a good delta, we natually use the null
revision as base. However, we sometimes consider the nullrev before we
have exhausted our other options. Specifically, when both parents are
null, we use the nullrev as delta base if it produces a good delta
(according to _isgooddelta()), and we fail to try the previous revlog
entry as delta base. After e60126c6093d (addrevision: use general
delta when the incoming base delta is bad, 2015-12-01), it can also
happen for non-merge commits when the incoming delta is not good.
The Firefox repo (from many months back) shrinks a tiny bit with this
patch: from 1.855GB to 1.830GB (1.4%). The hg repo itself shrinks even
less: by less than 0.1%. There may be repos that get larger instead.
This undoes the unexplained test change in e60126c6093d.
We unify the delta selection process to be a simple three options process:
- try to use the incoming delta (if lazydeltabase is on)
- try to find a suitable parents to delta against (if gd is on)
- try to delta against the tipmost revision
The first of this option that yield a valid delta will be used.
The test change in 'test-generaldelta.t' show this behavior as we use a delta
against the parent instead of a full delta when the incoming delta is not
suitable.
This as some impact on 'test-bundle.t' because a delta somewhere changes. It
does not seems to change the test semantic and have been ignored.
This impacts tests of the 'packed' feature. We can safely accept the new output
because the 'v1' format is not restricted to old revlog format, the requirements
are properly advertised to the client.
For the same reasons that we don't produce stream clone bundles with `hg
bundle`, we don't support consuming stream clone bundles with `hg
unbundle`. We introduce a complementary debug command for applying
stream clone bundles. This command is mostly to facilitate testing.
Although it may be used to manually apply stream clone bundles until a
more formal mechanism is (possibly) adopted.
Now that we have support for recognizing the streaming clone bundle
type, add a debug command for creating them.
I decided to create a new debug command instead of adding support to `hg
bundle` because stream clone bundles are not exactly used the same way
as normal bundle files and I don't want to commit to supporting them
through the official `hg bundle` command forever. A debug command,
however, can be changed without as much concern for backwards
compatibility.
As part of this, `hg bundle` will explicitly reject requests to produce
stream bundles.
This command will be required by server operators using stream clone
bundles with the clone bundles feature.
When the progress extension is not enabled, each call to 'ui.progress' used to
issue a debug message. This results is a very verbose output and often redundant
in tests. Dropping it makes tests less volatile to factor they do not meant to
test.
We had to alter the sed trick in 'test-rename-merge2.t'. Sed is used to drop all
output from a certain point and hidding the progress output remove its anchor.
So we anchor on something else.
This patch makes bundrepo retract the phase boundary for new commits to 'draft'
status, which is consistent with the behavior of 'hg unbundle'. The old
behavior was for commits to appear with the same phase as their nearest
ancestor in the base repository.
This affects several classes of operation:
* Inspecting a bundle with the -B flag
* Treating a bundle file as a peer (old: everything public, new: everything draft)
* Incoming command (neither old or new behavior is sensible -- fixed in next patch)
addchangegroup creates a runhook function that is used to invoke the
changegroup and incoming hooks, but at the time the function is called,
the contents of hookargs associated with the transaction may have been
modified externally. For instance, bundle2 code affects it with
obsolescence markers and bookmarks info.
It also creates problems when a single transaction is used with multiple
changegroups added (as per an upcoming change), whereby the contents
of hookargs are that of after adding a latter changegroup when invoking
the hook for the first changegroup.
When pulling changes from a compressed bundle Mercurial first uncompresses it
to a temporary file in .hg directory. This file will not be deleted unless
the bundlerepo (other) is explicitly closed.
This is similar to cleanup that occurs after incoming.
Before this patch, "hg bundle --branch foo other" fails to create
bundle file, if specified "foo" branch is created newly on the local
repository.
"hg bundle" uses "hg.addbranchrevs(repo, other, ...)" to look branch
names up, even though other outgoing-like implementation uses
"hg.addbranchrevs(repo, repo, ...)". In the former invocation, "other"
repository recognizes such branches as unknown, so execution is
aborted.
This patch uses "hg.addbranchrevs(repo, repo, ..)" in "hg bundle" to
bundle revisions on such branches correctly.
bundle() revset expression returns all changes that are present
in the bundle file (no matter whether they are in the repo or not).
Bundle file should be specified via -R option.
As a part of migration to vfs, this patch uses "self.root", which can
be recognized as the path relative to "self.vfs", instead of "path"
argument.
This fix allows to make invocations of "util.makedirs()" and
"os.path.exists()" while ensuring repository directory in
"localrepository.__init__()" ones indirectly via vfs.
But this fix also raises issue 2528: "hg clone" with empty destination.
"path" argument is empty in many cases, so this issue can't be fixed
in the view of "localrepository.__init__()".
Before this patch, it is fixed by empty-ness check ("not name") of
exception handler in "util.makedirs()".
try:
os.mkdir(name)
except OSError, err:
if err.errno == errno.EEXIST:
return
if err.errno != errno.ENOENT or not name:
raise
This requires "localrepository.__init__()" to invoke "util.makedirs()"
with "path" instead of "self.root", because empty "path" is treated as
"current directory" and "self.root" becomes valid path.
But "hg clone" with empty destination can be detected also in
"hg.clone()" before "localrepository.__init__()" invocation, so this
patch re-fixes issue2528 by checking it in "hg.clone()".
Many tests didn't change back from subdirectories at the end of the tests ...
and they don't have to. The missing 'cd ..' could always be added when another
test case is added to the test file.
This change do that tests (99.5%) consistently end up in $TESTDIR where they
started, thus making it simpler to extend them or move them around.
This is achieved by acting as if the user had given -r<rev> for each head rev
of outgoing changesets on the command line, as well as appropriate
--base <rev>.
The discovery information is computed as normal, and then adjusted as above.
Currently we have the following return codes if nothing is found:
commit incoming outgoing pull push
intended 1 1 1 1 1
documented 1 1 1 0 1
actual 1 1 1 0 1
This makes pull agree with the rest of the table and makes it easy to
detect "nothing was pulled" in scripts.
