Largefiles revert do for some reason have two lfdirstates and lfdirstatestatus
invocations in one function. The result from the first lfdirstate check was
however not written back to the lfdirstate, and some files was thus checked
twice.
Before this patch, largefiles extension uses "largefiles: No remote
repo" message not only for "outgoing" as status report, but also for
"summary" as summarized information.
This sharing prevents message translators from inserting white spaces
between "largefiles:" and "No remote repo" in translated message to
align column position of summarized information.
This patch changes output of largefiles for summary to distinguish
from one for outgoing.
This patch puts "no remote repo" into parentheses, because this is not
summarized information.
Previously, if one or more largefiles for a repo being converted were not in the
usercache, the convert would abort with a reference to the largefile being
missing (as opposed to the previous patch, where the standin was referenced as
missing). This is because commitctx() tries to copy all largefiles to the
local store, first from the user cache, and if the file isn't found there, from
the working directory. No files will exist in the working directory during a
convert, however. It is not sufficient to force the source repo to be local
before proceeding, because clone and pull do not download largefiles by default.
This is slightly less than ideal because while the conversion will now complete,
it won't be possible to update to revs with missing largefiles unless the user
intervenes manually, because there is no default path pointing back to the
source repo. Ideally these files would be cached during the conversion.
This check could have been done in reposetup.commitctx() instead, but this
ensures the local store directory is created, which is necessary to enable the
standin matcher.
The rm -> 'rm -f' change in the test is to temporarily suppress an error
clearing the cache- as noted, the cache is is not repopulated during convert.
When that is fixed, this can be changed back and the verification errors will
disappear too.
For some yet-unkown reason, largefile status does not work on
repoproxy. As status is not affected by filtering, we run it unfiltered.
Na'Tosha Bard's view on this issue:
"but, well, largefiles status is kind of an unholy piece of code"
This only applies to downloading largefiles, and only when no source for the
pull is explicitly provided. The repository itself was properly being pulled
via 'default' previously.
Using --all-largefiles is not necessary on a bare pull to test this (this
existing test is merely a convenience), but it is required to test pulling on
the rebase path.
Note that the errors generated in the --rebase case are because the repo
specified doesn't have the largefiles in its cache (though they are in the user
cache), so the errors are misleading. Specifying --all-largefiles when cloning
to 'b' fixes this, but instead of errors, it reports caching only 5 largefiles
instead of the 9 that come up missing. Likely this is because the largefile
download procedure tries to download missing files for each rev, and some of the
files have standins in more than one rev that gets pulled.
Before this patch, when no files to upload, "hg outgoing --large" and
"hg summary --large" show "no remote repo", even though valid remote
repository is specified.
It is because that "getoutgoinglfiles()" returns None, not only if no
valid remote repository is specified, but also if no files to upload.
This patch makes "getoutgoinglfiles()" return empty list when no files
to upload, and makes largefiles show "no files to upload" message at
that time.
This patch doesn't test "if toupload is None" route in
"overrideoutgoing()", because this route is not executed unless remote
repository becomes inaccessible just before largefiles specific
processing: successful execution of "orig()" means that at least one
of "default", "default-push" or dest is valid one, and that
"getoutgoinglfiles()" never returns None in such cases.
At "hg summary --large" invocation, this patch shows message below:
largefiles: (no files to upload)
This follows the message shown by "hg summary" with MQ:
mq: (empty queue)
The standin matcher only works if the .hglf directory exists (and it won't exist
if 'clone -U' is used, unless --all-largefiles is also specified). Since not
even 'update -r null' will get rid of the standin directory, this ensures that
the standin directory always exists if the repo has the 'largefiles'
requirement. This requirement is only set after a largefile is committed, so
these directories will not be created for repos that have the extension enabled
but have not committed a largefile.
With the standin directory in place, 'lfconvert --to-normal' will now be able to
download the required largefiles when converting a repo that was created with
'clone -U', and whose files are not in the usercache.
The downloadlfiles command could probably be put inside the 'largefiles'
requirement conditional too, but given that the user specified --all-largefiles,
there is likely an expectation to print out the number of largefiles downloaded,
even if it is 0.
This allows the wrapped command's validation code to run (which is currently
only to ensure 'noupdate' and 'updaterev' aren't both specified), the
copy/pasted unpacking of hg.clone() args to be removed, and any future changes
to the base command (however unlikely) to be inherited by largefiles.
Unfortunately, the command override can't be swapped entirely for an hg.clone()
override because the extra --all-largefiles arg needs to be injected. It also
isn't enough to call the wrapped clone command and leave the caching code after
it, because the file caching code needs access to the destination repo, which is
only available from hg.clone(). An alternative would be to use the dest path in
the clone command override to re-obtain a reference to the repo.
A slight deviation from the regular hg.clone() function is that the repo is NOT
deleted if the caching fails, but that was also the previous behavior. Maybe it
should for consistency?
A status message is output if hg.clone() determines the default destination
because None was provided. The previous code never passed None to hg.clone().
Previously, tip would be checked out regardless of the -u or -U parameter. I'm
not sure what the 'required for successful walkchangerevs' comment meant, but it
appears to reference code which has since moved to downloadlfiles() in
5c06bddf85b8. Perhaps it was to force caching when the -U parameter is given?
The price of this change is that -U --all-largefiles won't cache anything. That
will be fixed next.
Note that X + Y in the 'X largefiles updated, n removed' and 'Y additional
largefiles cached' lines do not add up to the same values in these tests, but
all of the largefiles have been downloaded. The reason being that several
largefiles have the same content (eb7338044 is pointed to by sub/large2, large3
and sub/large4). In the 'clone -u 1' operation, this largefile is cached to
populate the working directory, even without --all-largefiles. That means the
file isn't downloaded again and cached in the rev where large3 and sub/large4
both point to this file. Downloading that one file in that one rev seems to be
counted twice with 'clone -u 0'. (Maybe it is also being downloaded twice?)
Previous to this, 'commit -A' would add as normal files, files that were already
committed as largefiles, resulting in files being listed twice by 'status -A'.
It also missed when (only) a largefile was deleted, even though status reported
it as '!'. This also has the side effect of properly reporting the state of the
affected largefiles in the post commit hook after a remove that also affected a
normal file (the largefiles used to be 'R', now are properly absent).
Since scmutil.addremove() is called both by the ui command (after some trivial
argument validation) and during the commit process when -A is specified, it
seems like a more appropriate method to wrap than the addremove command.
Currently, a repo is only enabled to use largefiles after an add that explicitly
identifies some file as large, and a subsequent commit. Therefore, this patch
only changes behavior after such a largefile enabling commit.
Note that in the test, if the final commit had a '-v', 'removing large8' would
be printed twice. Both of these originate in removelargefiles(). The first
print is in verbose mode after traversing remove + forget, the second is because
the '_isaddremove' attr is set and 'after' is not.
Previously, a copy or a move of a largefile only worked if the cwd was the root
of the repository. The first issue was that the destination path passed to
os.mkdirs() chopped the absolute path to the standin after '.hglf/', which
essentially created a path relative to the repository root. Similarly, the
second issue was that the source and dest paths for copyfile() were relative to
the repo root. This converts these three paths to absolute paths.
Some notable issues, regardless of the directory in which the cp/mv is executed:
1) The copy is not being recorded in lfdirstate, but it is in dirstate for the
standins. I'm not sure if this is by design (i.e. minimal info in lfdirstate).
2) status -C doesn't behave as expected. Using the testcase as an example:
# after mv + ci
$ hg status -C -v --rev '.^' # expected to see 'A' and ' ' lines too
R dira\dirb\largefile
$ hg status -C -v --rev '.^' foo/largefile
# no output # expected to see 'A' and ' ' lines only
$ hg status -C -v --rev '.^' foo/
# no output # expected to see 'A', ' ' and 'R' lines
$ hg status -C -v --rev '.^' ./ # expected to see 'A' and ' ' lines too
R dirb\largefile
$ hg status -C -v --rev '.^' ../.hglf/dira/foo/largefile
A ..\.hglf\dira\foo\largefile
..\.hglf\dira\dirb\largefile # no 'R' expected when new file is specified
$ hg status -C -v --rev '.^' ../.hglf # OK
A ..\.hglf\dira\foo\largefile
..\.hglf\dira\dirb\largefile
R ..\.hglf\dira\dirb\largefile
An easy way to force this (and cause a traceback) prior to the fix for 3507 was
$ touch large
$ hg add --large large
$ hg ci -m "add"
$ hg remove large
$ touch large
$ hg addremove --config largefiles.patterns=**large
This patch also detected (and corrected) a previous test where a standin got
added as a largefile (without a traceback).
The problem only occurred if a file was removed with 'hg rm' (as opposed to the
OS utilities), and then addremove was run before a commit. Both normal and
large files were affected.
Ensuring that the file exists prior to an lstat() for size seems like the Right
Thing. But oddly enough, the missing file that was causing lstat() to blow up
was a standin when a largefile was removed, which seems fishy, because a standin
should never be added as a largefile. I was then able to get a standin added as
a largefile (whose name is 'large') with
hg addremove --config largefiles.patterns=**large
which also causes a backtrace. That will be fixed next.
If a file was missing, the missing list contained a path relative to the repo.
When building the matcher from that list, the file name ended up concatenated to
cwd, causing the command to abort with '<file> not under root'. This rebuilds
the missing list with paths relative to cwd.
This introduces a peer method into all repository classes, which currently
simply returns self. It also changes hg.repository so it now raises an
exception if the supplied paths does not resolve to a localrepo or descendant.
Finally, all call sites are changed to use the peer and local methods as
appropriate, where peer is used whenever the code is dealing with a remote
repository (even if it's on local disk).
Add a match object to subrepo.archive(). This will allow the -X and -I
options to be honored inside subrepos when archiving. They formerly
only affect the top level repo.
