Before, we would sometimes use the default iterator over large files. That
iterator is line based and would add extra buffering and use odd chunk sizes
which could give some overhead.
copyandhash can't just apply a filechunkiter as it sometimes is passed a
genuine generator when downloading remotely.
Files that are already in local store should be checked locally. The problem
with this implementation is how difference in messages between local and remote
checks should look like. For now local errors for file missing and content
corrupted looks like this:
'changeset cset: filename references missing storepath\n'
'changeset cset: filename references corrupted storepath\n'
for remote it looks like:
'changeset cset: filename missing\n'
'changeset cset: filename: contents differ\n'
Contents differ error for remote calls is never raised currently - for now
statlfile implementation lacks checking file content.
Methods _put, _get, _stat were used in remotestore class as
abstract expecting that subclass would implement them. This
commit makes this fact explicit.
The home of 'Abort' is 'error' not 'util' however, a lot of code seems to be
confused about that and gives all the credit to 'util' instead of the
hardworking 'error'. In a spirit of equity, we break the cycle of injustice and
give back to 'error' the respect it deserves. And screw that 'util' poser.
For great justice.
Python 2.6 introduced the "except type as instance" syntax, replacing
the "except type, instance" syntax that came before. Python 3 dropped
support for the latter syntax. Since we no longer support Python 2.4 or
2.5, we have no need to continue supporting the "except type, instance".
This patch mass rewrites the exception syntax to be Python 2.6+ and
Python 3 compatible.
This patch was produced by running `2to3 -f except -w -n .`.
This avoids a lot of expensive roundtrips to remote repositories ... but might
be slightly slower for local operations.
This will also change some aborts on missing files to warnings. That will in
some situations make it possible to continue working on a repository with
missing largefiles.
Avoid the intermediate limitreader and filechunkiter between getlfile and
copyandhash - return the right protocol and put the complexity where it better
can be managed.
6fb54510b150 introduced batching of statlfile, but not all codepaths got
converted.
_getfile gave _stat garbage and got garbage back. The garbage didn't match the
expected error codes and was thus interpreted as success. It could thus end up
trying to fetch a largefile that didn't exist.
Instead we now pass _stat valid input and handle both correct and invalid
output correctly.
This makes the code work as intended ... but it would probably be better if it
didn't abort on missing largefiles, just like it happened to do before.
6fb54510b150 introduced batching of statlfile, but not all codepaths got
converted.
'hg verify' with a remotestore could thus crash with
TypeError: 'builtin_function_or_method' object is not iterable
Also, the 'hash' variable was used without assigning to it. Don't use variable
names that collide with Python built-in functions. Instead we use 'expecthash'
as in localstore.
The tests for this issue covers an untested area. The tests happens to also
reveal incorrect attempts at getting non-existing largefiles, bad server side
handling of that, and corruption issues - all to be fixed later.
This implements a part of issue 3386. It batches the request for the status of
all largefiles in the revisions that are about to be pushed into a single
request, instead of doing N separate requests.
In a real world test case, this change was verified to save 1,116 round-trips to
the server. It only requires a client-side change; it is backwards-compatible
with an older version of the server.
- tweak wording of some error messages
- use consistent capitalization
- always say 'largefile', not 'lfile'
- fix I18N problems
- only raise Abort for errors the user can do something about
- fix some ungrammatical/unclear/incorrect comments/docstrings
- rewrite some really unclear comments/docstrings
- make formatting/style more consistent with the rest of Mercurial
(lowercase without period unless it's really multiple sentences)
- wrap to 75 columns
- always say "largefile(s)", not "lfile(s)" (or "big files")
- one space between sentences, not two
