Commit Graph

39 Commits

Author SHA1 Message Date
Matt Harbison
f6129be11b test-largefiles: stabilize output for Windows
Systems with unix-permissions have a test above this that adds an additional
head.
2016-03-15 21:56:01 -04:00
Mads Kiilerich
069bac6a3e largefiles: introduce push --lfrev to control which revisions are pushed
The default of pushing all largefiles referenced in outgoing revisions is safe,
but also expensive and sometimes not what is needed. We thus introduce a
--lfrev option, similar to what pull already has.

By specifying an empty set of revisions (or null), it is possible to get lazy
(and insecure!) pushes of revisions without referenced largefiles, similar to
how pull works.
2016-03-27 13:00:28 -07:00
Pierre-Yves David
2dcf98d614 update: change default destination to tipmost descendant (issue4673) (BC)
Bare 'hg update' now brings you to the tipmost descendant (on the same branch).
Leaving the user on the same topological branch. The previous behavior, updating
to the tipmost changeset on the same branch could lead to jump from a
topological branch to another. This was confusing and impractical. As the only
conceivable reason for the old behavior have been address by the recently
introduce message about other heads, we can "safely" change this behavior

All test changes have been reviewed and seen a valid consequences.
2016-02-02 15:24:11 +00:00
Pierre-Yves David
2e0675043e update: warn about other topological heads on bare update
A concern around the user experience of Mercurial is user getting stuck on there
own topological branch forever. For example, someone pulling another topological
branch, missing that message in pull asking them to merge and getting stuck on
there own local branch.

The current way to "address" this concern was for bare 'hg update' to target the
tipmost (also latest pulled) changesets and complain when the update was not
linear. That way, failure to merge newly pulled changesets would result in some
kind of failure.

Yet the failure was quite obscure, not working in all cases (eg: commit right
after pull) and the behavior was very impractical in the common case
(eg: issue4673).

To be able to change that behavior, we need to provide other ways to alert a
user stucks on one of many topological head. We do so with an extra message after
bare update:

  1 other heads for branch "default"

Bookmark get its own special version:

  1 other divergent bookmarks for "foobar"

There is significant room to improve the message itself, and we should augment
it with hint about how to see theses other heads or handle the situation (see
in-line comment). But having "a" message is already a significant improvement
compared to the existing situation. Once we have it we can iterate on a better
version of it. As having such message is an important step toward changing the
default destination for update and other nicety, I would like to move forward
quickly on getting such message.

This was discussed during London - October 2015 Sprint.
2016-02-02 14:49:02 +00:00
Matt Harbison
6d46368119 largefiles: prevent committing a missing largefile
Previously, if the largefile was deleted at the time of a commit, the standin
was silently not updated and its current state (possibly garbage) was recorded.
The test makes it look like this is somewhat of an edge case, but the same thing
happens when an `hg revert` followed by `rm` changes the standin.

Aside from the second invocation of this in lfutil.updatestandinsbymatch()
(which is what triggers this test case), the three other uses are guarded by
dirstate checks for added or modified, or an existence check in the filesystem.
So aborting in lfutil.updatestandins() should be safe, and will avoid silent
skips in the future if this is used elsewhere.
2016-01-24 00:10:19 -05:00
Matt Harbison
9906cb44b6 largefiles: fix an explicit largefile commit after a remove (issue4969)
The change in 6fce9a02f069 to handle a normal -> largefile switch was too
aggressive in preserving the original matcher names.  If a largefile is
explicitly provided by the user, but only the standin exists in dirstate, then
only the standin can be committed.

There's still maybe an issue when the largefile is deleted outside of Mercurial:

  $ rm large
  $ hg ci -m "oops" large
  large: The system cannot find the file specified
  nothing changed
  [1]
2016-01-23 20:51:17 -05:00
Matt Harbison
e6074e31cf test-largefiles-cache: add glob for Windows 2015-11-09 12:51:58 -05:00
Mads Kiilerich
f3cdae8f42 largefiles: don't mute and obfuscate http errors when putlfile fails
'unexpected putlfile response: None' when an http error occurs is not very
helpful.

Instead, leave the handling of urllib2.HTTPError exceptions to other layers.
2015-10-23 21:27:29 +02:00
Mads Kiilerich
d6bc0b0c94 largefiles: test coverage of error handling from putlfile
This reveals a wrong and unhelpful 'unexpected putlfile response'.
2015-10-23 21:27:29 +02:00
Mads Kiilerich
24ee58b9f9 largefiles: check hash of files in the store before copying to working dir
If the store somehow got corrupted, users could end up in weird situations that
were very hard to recover from or lead to propagation of the corruption.

Instead, spend the extra time checking the hash when copying to the working
directory. If it doesn't match, emit a warning, and don't put wrong content in
the working directory.
2015-10-23 21:27:29 +02:00
Mads Kiilerich
d9326efafd largefiles: test coverage of handling of store corruption
This reveals that update might put a corrupted largefile in the working
directory where it will show up as modified and ready for commit.
2015-10-23 21:27:29 +02:00
Matt Harbison
673b9701b1 largefiles: use the share source as the primary local store (issue4471)
The benefit of retargeting the local store to the share source is that all
shares will always have access to the largefiles any one of them commit, even if
the user cache is deleted (which is documented to be OK to do).  Further, any
push into the source (and now any shares), will likewise make the largefile(s)
visible to all related repositories.

In order to maintain compatibility with existing repos, where the largefiles
would be cached only in the local share, fallback to searching the local share
if it isn't found at the share source.

The unshare command should probably be taught to copy the source store into the
store for the repo being unshared to complete the loop.


This patch changes the test like this:

  @@ -159,6 +159,5 @@
     $ hg share -q src share_dst --config extensions.share=
     $ hg -R share_dst update -r0
     getting changed largefiles
  -  large: largefile $HASH not available from file:///$TESTTMP\share_dst
  -  0 largefiles updated, 0 removed
  +  1 largefiles updated, 0 removed
     1 files updated, 0 files merged, 0 files removed, 0 files unresolved


The issue writeup mentions pushing a largefile from a remote repo to the main
local repo, and the largefile is then not available in any shares.  Since the
push doesn't cache the largefile in $USERCACHE, the trashed $USERCACHE in this
test is equivalent.
2015-04-04 19:06:43 -04:00
Mads Kiilerich
b2b60414f6 spelling: fixes from proofreading of spell checker issues 2015-01-18 02:38:57 +01:00
Matt Harbison
93382de245 largefiles: fix a spurious missing file warning with forget (issue4053)
If an uncommitted and deleted file was forgotten, a warning would be emitted,
even though the operation was successful.  See the previous patch for
'remove -A' for the exact circumstances, and details about the cause.
2014-12-21 15:06:54 -05:00
Matt Harbison
8edc086e01 largefiles: fix a spurious missing file warning with 'remove -A' (issue4053)
The bug report doesn't mention largefiles, but the given recipe doesn't fail
unless the largefiles extension is loaded.  The problem only affected normal
files, whether or not any largefiles are committed, and only files that have
not been committed yet.  (Files with an 'a' state are dropped from dirstate,
not marked removed.)  Further, if the named normal file never existed, the
warning would be printed out twice.

The problem is that the core implementation of remove() calls repo.status(),
which eventually triggers a dirstate.walk().  When the file isn't seen in the
filesystem during the walk, the exception handling finds the file in
dirstate, so it doesn't complain.  However, the largefiles implementation
called status() again with all of the original files (including the normal
ones, just dropped).  This time, the exception handler doesn't find the file
in dirstate and does complain.  This simply excludes the normal files from
the second repo.status() call, which the largefiles extension has no interest
is processing anyway.
2014-12-21 15:04:13 -05:00
Mads Kiilerich
0e8795ccd6 spelling: fixes from spell checker 2014-04-13 19:01:00 +02:00
Simon Heimberg
305a187eff tests: lines with largefile .* file://$TESTTMP also match on windows
on windows, largefile paths are written as "file:///C:/temp/...", corresponding
to "file:///$TESTTMP/..." (all three slashes shown).
But on posix systems they are written as "file:///tmp/..." corresponding to
"file://$TESTTMP/..." (only two slashes shown).

Write the glob "file:/*/" to match both versions.
2014-01-27 21:39:25 +01:00
Mads Kiilerich
63e01aeb0a largefiles: hide passwords in URLs in ui messages 2013-10-17 16:13:15 +08:00
Brendan Cully
5fda2ba23f tests: sprinkle globs over largefiles/subrepo tests for Windows 2013-05-02 11:26:43 -07:00
Mads Kiilerich
208aa05eca largefiles: stat all largefiles in one batch before downloading
This avoids a lot of expensive roundtrips to remote repositories ... but might
be slightly slower for local operations.

This will also change some aborts on missing files to warnings. That will in
some situations make it possible to continue working on a repository with
missing largefiles.
2013-04-15 23:37:43 +02:00
Mads Kiilerich
00a100f5a5 merge with stable 2013-02-28 14:51:59 +01:00
Mads Kiilerich
8c5a6710d0 largefiles: don't assume that .hg/largefiles/ still exists
It might not have been created and it might have been removed.
2013-02-28 13:45:18 +01:00
Mads Kiilerich
34d52d8c6e largefiles: getstandinmatcher should not depend on existence of directories
Looking for a (potentially empty) directory was not reliable - both because it
is a reasonable assumption that empty directories can be removed and because it
wasn't created in all cases ... such as when pulling to an existing repository.
2013-02-28 13:45:18 +01:00
Mads Kiilerich
3e416221ec tests: don't rely on broken behaviour in test-largefiles-cache.t
The test relied on the bug that 'pull largefiles from branchheads' didn't pull
any largefiles from tip revision when it seemed like no largefiles had been
checked out before.
2013-02-28 13:44:59 +01:00
Na'Tosha Bard
f1bd9d017e largefiles: don't cache largefiles for pulled heads by default
After discussion, we've agreed that largefiles for newly pulled heads should
not be cached by default.  The use case for this is using largefiles repos
with multiple remote servers (and therefore multiple remote largefiles caches),
where users will be pulling from non-default locations on a regular basis.  We
think this use case will be significantly less common than the use case where
all largefiles are stored on the same central server, so the default should be
no caching.

The old behavior can be obtained by passing the --cache-largefiles flag to
pull.
2013-02-09 21:07:42 +00:00
Wagner Bruna
30efeaefbe largefiles: enhance error message to make it more i18n-friendly 2013-01-21 13:42:04 -02:00
Benoit Boissinot
8f1400147f largefiles: fix largefiles+subrepo update (issue3752)
Override updaterepo() instead of individual methods that may not be called for
each subrepo. Add test.
Based on patch from Matt Harbison.

Changes the order of update-related messages (now largefiles comes before the
global status).
2013-01-23 00:51:53 +01:00
Na'Tosha Bard
7b4ce40b7d largefiles: fix test and check code 2013-02-09 16:02:01 +00:00
Mads Kiilerich
6376600846 tests: fix windows test failures 2012-12-28 14:22:24 +01:00
Mads Kiilerich
9302004c43 largefiles: cleanup of warnings on errors getting largefiles
Especially the "no default or default-push path set in hgrc" was often very
misleading and didn't give any hint where it actually was looking.

A long error messages is better than several multi-line messages.
2012-12-13 19:19:06 +01:00
Mads Kiilerich
f7215835a1 tests: convert some hghave unix-permissions to #if 2012-06-19 01:43:41 +02:00
Martin Geisler
643b04fbb7 largefiles: lowercase messages 2012-06-12 14:18:18 +02:00
Mads Kiilerich
fa1c4e5ebe tests: add missing trailing 'cd ..'
Many tests didn't change back from subdirectories at the end of the tests ...
and they don't have to. The missing 'cd ..' could always be added when another
test case is added to the test file.

This change do that tests (99.5%) consistently end up in $TESTDIR where they
started, thus making it simpler to extend them or move them around.
2012-06-11 01:40:51 +02:00
Martin Geisler
b8759cea3b largefiles: respect store.createmode and avoid extra file copy
Before, a tempfile was used to create a temp file was created with 600
permissions and the uploaded data was written into it. This file was
then *copied* to .hg/largefiles/<hash>.

We now simply use atomictempfile to write the data to a temp file with
the right permissions and then rename that into place.
2012-02-23 13:37:10 +01:00
Martin Geisler
61a444869a largefiles: respect store.createmode in basestore.get
This replaces another use of tempfile with atomictempfile. The problem
with tempfile is that it creates files with 600 permissions instead of
respecting repo.store.createmode.
2012-02-23 13:37:10 +01:00
Martin Geisler
c3f611d55c largefiles: use repo.store.createmode for new files in .hg/largefiles
Before, the mode was copied from the file in the working copy. This is
inconsistent with how Mercurial normally creates files inside the .hg
folder. This can lead to a situation where you can read the files
under .hg/store but not under .hg/largefiles and so a clone will fail
if it needs to access a largefile.
2012-02-23 13:22:55 +01:00
Na'Tosha Bard
1e5b58f8a8 largefiles: cache new largefiles for new heads when pulling
When the user pulls from a remote repository that is not his default repo, it
is quite likely that he will pull a new head.  This means that if he tries to
merge or rebase with the other head, he will run into a problem becuase
largefiles has no way of tracking where the remote repository for this other
head is, so it cannot download the largefiles from this other remote repository.
It will attempt to download them from its default remote repository, which will
not yet contain the largefiles.

This patch solves this problem by caching any new largefiles for all heads
directly into the system cache at the time of the pull, so they are available
later.

This behavior is actually more in line with Mercurial's distributed nature,
because pulling already implies we have a connection to the remote server, but
merging or rebasing does not.
2012-01-18 11:33:14 +01:00
Matt Mackall
71fadf8296 tests: fix trailing whitespace 2011-11-10 17:06:12 -06:00
FUJIWARA Katsunori
970e57770c largefiles: treat status of cache missed largefiles as "missing" correctly
"hg status" may treat cache missed largefiles as "removed" incorrectly.

assumptions for problem case:

  - there is no cache for largefile "L"
  - at first, update working directory to the revision in which "L" is
    not yet added,
  - then, update working directory to the revision in which "L" is
    already added

and now, "hg status" treats "L" as "removed".

current implementation does not allocate entry for cache missed
largefile in ".hg/largefiles/dirstate", but files without
".hg/largefiles/dirstate" entry are treated as "removed" by largefiles
extension.

"hg revert" can not recover from this situation, but "rm -rf
.hg/largefiles", because it causes dirstate rebuilding.

this patch invokes normallookup() for cache missed largefiles to
allocate entry in ".hg/largefiles/dirstate", so "hg status" can treat
it as "missing" correctly.
2011-11-11 02:33:01 +09:00