Commit Graph

8796 Commits

Author SHA1 Message Date
Anton Shestakov
dc749de111 hgweb: provide links to branches, tags and bookmarks by name (paper and coal)
It's sometimes handy to, say, have a url always point to branch head, not just
at the current branch head by node hash. Previously, this was only possible by
manually editing url and replacing node hash with branch/tag/bookmark name. It
wasn't very convenient, or easy - in case the name contained special
characters that needed to be urlencoded.

Let's have /branches, /tags and /bookmarks pages in paper and coal style
provide links both to symbolic revisions and to node hashes.

This feature was wished for in issue3594.
2015-07-12 18:04:48 +08:00
Anton Shestakov
59f182075a hgweb: allow symbolic revisions with forward slashes in urls
It's possible to have a branch/tag/bookmark with all kinds of special
characters, such as {}/\!?. While not very conveniently, symbolic revisions
with such characters work from command line if user correctly quotes the
characters. These characters also work in hgweb, when they are properly
encoded, with one exception: '/' (forward slash, urlencoded as '%2F'), which
was getting decoded before hgweb could parse it as a part of PATH_INFO.
Because of that, hgweb was seeing it as any other forward slash, that is, as
just another url parts separator.

For example, if user wanted to see the content of dir/file at bookmark
'feature/eggs', url could be: '/file/feature%2Feggs/dir/file'. But hgweb tried
to find a revision 'feature' and get contents of 'eggs/dir/file'.

To fix this, let's assume forward slashes are doubly-urlencoded (%252F), so
CGI/WSGI server decodes it into %2F. Then we can decode %2F in the revision
part of the url into an actual '/' character.

Making hgweb produce such urls will be done in the next 2 patches.
2015-07-12 16:06:57 +08:00
FUJIWARA Katsunori
ec38bf98b8 shelve: keep old backups if timestamp can't decide exact order of them
Before this patch, backups to be discarded are decided by steps below
at 'hg unshelve' or so:

  1. list '(st_mtime, filename)' tuples of each backups up
  2. sort list of these tuples, and
  3. discard backups other than 'maxbackups' ones at the end of list

This doesn't work well in the case below:

  - "sort by name" order differs from actual backup-ing order, and
  - some of backups have same timestamp

For example, 'test-shelve.t' satisfies the former condition:

  - 'default-01' < 'default-1' in "sort by name" order
  - 'default-1'  < 'default-01' in actual backup-ing order

Then, 'default-01' is discarded instead of 'default-1' unexpectedly,
if they have same timestamp. This failure appears occasionally,
because the most important condition "same timestamp" is timing
critical.

To avoid such unexpected discarding, this patch keeps old backups if
timestamp can't decide exact order of them.

Timestamp of the border backup (= the oldest one of recent
'maxbackups' ones) as 'bordermtime' is used to examine whether
timestamp can decide exact order of backups.
2015-07-13 23:34:12 +09:00
FUJIWARA Katsunori
78eff9a5a8 subrepo: use repo.pathto instead of util.pathto to simplify invocation
This centralization into 'repo.pathto()' should reduce the cost of vfs
migration around 'getcwd()' and so on in the future.
2015-07-10 00:59:51 +09:00
Yuya Nishihara
33dcb19532 revset: work around x:y range where x or y is wdir()
All revisions must be contiguous in spanset, so we need the special case
for the wdir revision.
2015-06-28 16:08:07 +09:00
Yuya Nishihara
3732960ab3 revset: use integer representation of wdir() in revset
This is the simplest way to handle wdir() revision in revset. None didn't
work well because revset heavily depends on integer operations such as min(),
max(), sorted(), x:y, etc.

One downside is that we cannot do "wctx.rev() in set" because wctx.rev() is
still None. We could wrap the result set by wdirproxyset that translates None
to wdirrev, but it seems overengineered at this point.

    result = getset(repo, subset, tree)
    if 'wdir' in funcsused(tree):
        result = wdirproxyset(result)

Test cases need the '(all() + wdir()) &' hack because we have yet to fix the
bootstrapping issue of null and wdir.
2015-03-16 16:17:06 +09:00
Yuya Nishihara
0cf602f409 changeset_printer: display wdirrev/wdirnode values for workingctx
Because we want to eliminate "if"s in the default template, it makes sense to
display wdirrev/wdirnode values for now. wdir() is still experimental, so the
output of "log -r'wdir()'" may change in future.
2015-07-04 17:19:49 +09:00
Gregory Szorc
c9f7c94c30 hg: support for auto sharing stores when cloning
Many 3rd party consumers of Mercurial have created wrappers to
essentially perform clone+share as a single operation. This is
especially popular in automated processes like continuous integration
systems. The Jenkins CI software and Mozilla's Firefox release
automation infrastructure have both implemented custom code that
effectively perform clone+share. The common use case here is that
clients want to obtain N>1 checkouts while minimizing disk space and
network requirements. Furthermore, they often don't care that a clone
is an exact mirror of a remote: they are simply looking to obtain
checkouts of specific revisions.

When multiple third parties implement a similar feature, it's a good
sign that the feature is worth adding to the core product. This patch
adds support for an easy-to-use clone+share feature.

The internal "clone" function now accepts options to control auto
sharing during clone. When the auto share mode is active, a store will
be created/updated under the base directory specified and a new
repository pointing to the shared store will be created at the path
specified by the user.

The share extension has grown the ability to pass these options into
the clone command/function.

No command line options for this feature are added because we don't
feel the feature will be popular enough to warrant their existence.

There are two modes for auto share mode. In the default mode, the shared
repo is derived from the first changeset (rev 0) in the remote
repository. This enables related repositories existing at different URLs
to automatically use the same storage. In environments that operate
several repositories (separate repo for branch/head/bookmark or separate
repo per user), this has the potential to drastically reduce storage
and network requirements. In the other mode, the name is derived from the
remote's path/URL.
2015-07-08 16:19:09 -07:00
FUJIWARA Katsunori
ec0ec0f1d2 cmdutil: apply dirstate.normallookup on (maybe partially) committed files
To detect change of a file without redundant comparison of file
content, dirstate recognizes a file as certainly clean, if:

  (1) it is already known as "normal",
  (2) dirstate entry for it has valid (= not "-1") timestamp, and
  (3) mode, size and timestamp of it on the filesystem are as same as
      ones expected in dirstate

This works as expected in many cases, but doesn't in the corner case
that changing a file keeps mode, size and timestamp of it on the
filesystem.

The timetable below shows steps in one of typical such situations:

  ---- ----------------------------------- ----------------
                                           timestamp of "f"
                                           ----------------
                                           dirstate   file-
  time          action                     mem  file  system
  ---- ----------------------------------- ---- ----- -----
  N                                              ***   ***
       - change "f"                                     N

       - execute 'hg commit -i'
         - backup "f" with timestamp N
         - revert "f" by 'merge.update()'               N
           with 'partially'
         - apply selected hunks                         N
           by 'patch.patch()'

         - 'repo.commit()'
           - 'dirstate.normal("f")'         N
  N+1
           - 'dirstate.write()'             N    N

         - restore "f"                                  N+1
         - restore timestamp of "f"                     N

       - 'hg status' shows "f" as "clean"   N    N      N
  ---- ----------------------------------- ---- ----- -----

The most important point is that 'dirstate.write()' is executed at N+1
or later. This causes writing dirstate timestamp N of "f" out
successfully. If it is executed at N, 'parsers.pack_dirstate()'
replaces timestamp N with "-1" before actual writing dirstate out.

This issue can occur when 'hg commit -i' satisfies conditions below:

  - the file is committed partially, and
  - mode and size of the file aren't changed before and after committing

The root cause of this issue is that (maybe partially changed) files
are restored with original timestamp but dirstate isn't updated for
them.

To detect changes of files correctly, this patch applies
'dirstate.normallookup()' on restored files. Status check is needed
before 'dirstate.normallookup()', because status other than "n(ormal)"
should be kept at failure of committing.

This patch doesn't examine whether each files are committed fully or
partially, because interactive hunk selection makes it difficult.

After this change, timetable is changed as below:

  ---- ----------------------------------- ----------------
                                           timestamp of "f"
                                           ----------------
                                           dirstate   file-
  time          action                     mem  file  system
  ---- ----------------------------------- ---- ----- -----
  N                                              ***   ***
       - change "f"                                     N

       - execute 'hg commit -i'
         - backup "f" with timestamp N
         - revert "f" by 'merge.update()'               N
           with 'partially'
         - apply selected hunks                         N
           by 'patch.internalpatch()'

         - 'repo.commit()'
           - 'dirstate.normal("f")'         N
  N+1
           - 'dirstate.write()'             N    N

         - restore "f"                                  N+1
         - restore timestamp of "f"                     N
       ----------------------------------- ---- ----- -----
         - normallookup("f")                -1
         - release wlock
           - 'dirstate.write()'             -1   -1     N
       ----------------------------------- ---- ----- -----

       - 'hg status' shows "f" as "clean"   -1   -1     N
  ---- ----------------------------------- ---- ----- -----

To reproduce this issue in tests certainly, this patch emulates some
timing critical actions as below:

  - change "f" at N

    'touch -t 200001010000' before command invocation changes mtime of
    "f" to "2000-01-01 00:00" (= N).

  - apply selected hunks at N

    'patch.internalpatch()' with 'fakepatchtime.py' explicitly changes
    mtime of patched files to "2000-01-01 00:00" (= N).

  - 'dirstate.write()' at N+1 (or "not at N")

    'pack_dirstate()' uses actual timestamp at runtime as "now", and
    it should be different from the "2000-01-01 00:00" of "f".

BTW, in 'test-commit-interactive.t', files are sometimes treated as
modified , even though they are just committed fully via 'hg commit
-i' and 'hg diff' shows nothing for them.

Enabling win32text causes EOL style mismatching below:

  - files are changed in LF style EOL

    => files restored after committing uses LF style EOL (1)

  - 'merge.update()' reverts files in CRLF style EOL
  - 'patch.internalpatch()' changes files in CRLF style EOL

    => 'dirstate.normal()' via 'repo.commit()' uses the size of files
       in CRLF style EOL (2)

Therefore, fully committed files are treated as "modified", because
'lstat()' returns size of (1) restored files in LF style EOL, but
dirstate expects size of (2) committed files in CRLF style EOL.

After this patch, 'dirstate.normallookup()' on committed files forces
subsequent 'hg status' to examine changes exactly, and fully committed
files are treated as clean as expected.

This is reason why this patch also does:

  - add some 'hg status' checking status of fully committed files
  - clear win32text configuration before size/timestamp sensitive examination
2015-07-08 17:07:45 +09:00
FUJIWARA Katsunori
1dcc27a649 context: write dirstate out explicitly at the end of markcommitted
To detect change of a file without redundant comparison of file
content, dirstate recognizes a file as certainly clean, if:

  (1) it is already known as "normal",
  (2) dirstate entry for it has valid (= not "-1") timestamp, and
  (3) mode, size and timestamp of it on the filesystem are as same as
      ones expected in dirstate

This works as expected in many cases, but doesn't in the corner case
that changing a file keeps mode, size and timestamp of it on the
filesystem.

The timetable below shows steps in one of typical such situations:

  ---- ----------------------------------- ----------------
                                           timestamp of "f"
                                           ----------------
                                           dirstate   file-
  time          action                     mem  file  system
  ---- ----------------------------------- ---- ----- -----
   *                                             ***    ***
       - 'hg transplant REV1 REV2 ...'
         - transplanting REV1
           ....
   N
           - change "f", but keep size                   N
             (via 'patch.patch()')
           - 'dirstate.normal("f")'          N   ***
             (via 'repo.commit()')

         - transplanting REV2
           - change "f", but keep size                   N
             (via 'patch.patch()')
           - aborted while patching
  N+1
         - release wlock
           - 'dirstate.write()'              N    N      N

       - 'hg status' shows "r1" as "clean"   N    N      N
  ---- ----------------------------------- ---- ----- -----

The most important point is that 'dirstate.write()' is executed at N+1
or later. This causes writing dirstate timestamp N of "f" out
successfully. If it is executed at N, 'parsers.pack_dirstate()'
replaces timestamp N with "-1" before actual writing dirstate out.

This issue can occur when 'hg transplant' satisfies conditions below:

  - multiple revisions to be transplanted change the same file
  - those revisions don't change mode and size of the file, and
  - the 2nd or later revision of them fails after changing the file

The root cause of this issue is that files are changed without
flushing in-memory dirstate changes via 'repo.commit()' (even though
omitting 'dirstate.normallookup()' on files changed by 'patch.patch()'
for efficiency also causes this issue).

To detect changes of files correctly, this patch writes in-memory
dirstate changes out explicitly after marking files as clean in
'committablectx.markcommitted()', which is invoked via
'repo.commit()'.

After this change, timetable is changed as below:

  ---- ----------------------------------- ----------------
                                           timestamp of "f"
                                           ----------------
                                           dirstate   file-
  time          action                     mem  file  system
  ---- ----------------------------------- ---- ----- -----
   *                                             ***    ***
       - 'hg transplant REV1 REV2 ...'
         - transplanting REV1
           ....
   N
           - change "f", but keep size                   N
             (via 'patch.patch()')
           - 'dirstate.normal("f")'          N    ***
             (via 'repo.commit()')
       ----------------------------------- ---- ----- -----
           - 'dirsttate.write()'            -1    -1
       ----------------------------------- ---- ----- -----
         - transplanting REV2
           - change "f", but keep size                   N
             (via 'patch.patch()')
           - aborted while patching
  N+1
         - release wlock
           - 'dirstate.write()'              -1   -1     N

       - 'hg status' shows "r1" as "clean"   -1   -1     N
  ---- ----------------------------------- ---- ----- -----

To reproduce this issue in tests certainly, this patch emulates some
timing critical actions as below:

  - change "f" at N

    'patch.patch()' with 'fakepatchtime.py' explicitly changes mtime
    of patched files to "2000-01-01 00:00" (= N).

  - 'dirstate.write()' via 'repo.commit()' at N

    'fakedirstatewritetime.py' forces 'pack_dirstate()' to use
    "2000-01-01 00:00" as "now", only if 'pack_dirstate()' is invoked
    via 'committablectx.markcommitted()'.

  - 'dirstate.write()' via releasing wlock at N+1 (or "not at N")

    'pack_dirstate()' via releasing wlock uses actual timestamp at
    runtime as "now", and it should be different from the "2000-01-01
    00:00" of "f".

BTW, this patch doesn't test cases below, even though 'patch.patch()'
is used similarly in these cases:

  1. failure of 'hg import' or 'hg qpush'

  2. success of 'hg import', 'hg qpush' or 'hg transplant'

Case (1) above doesn't cause this kind of issue, because:

  - if patching is aborted by conflicts, changed files are committed

    changed files are marked as CLEAN, even though they are partially
    patched.

  - otherwise, dirstate are fully restored by 'dirstateguard'

    For example in timetable above, timestamp of "f" in .hg/dirstate
    is restored to -1 (or less than N), and subsequent 'hg status' can
    detect changes correctly.

Case (2) always causes 'repo.status()' invocation via 'repo.commit()'
just after changing files inside same wlock scope.

  ---- ----------------------------------- ----------------
                                           timestamp of "f"
                                           ----------------
                                           dirstate   file-
  time          action                     mem  file  system
  ---- ----------------------------------- ---- ----- -----
  N                                              ***   ***
       - make file "f" clean                            N

       - execute 'hg foobar'
         ....
         - 'dirstate.normal("f")'           N    ***
           (e.g. via dirty check
            or previous 'repo.commit()')

         - change "f", but keep size                    N
         - 'repo.status()' (*1)
           (via 'repo.commit()')
  ---- ----------------------------------- ---- ----- -----

At a glance, 'repo.status()' at (*1) seems to cause similar issue (=
"changed files are treated as clean"), but actually doesn't.

'dirstate._lastnormaltime' should be N at (*1) above, because
'dirstate.normal()' via dirty check is finished at N.

Therefore, "f" changed at N (= 'dirstate._lastnormaltime') is forcibly
treated as "unsure" at (*1), and changes are detected as expected (see
'dirstate.status()' for detail).

If 'hg import' is executed with '--no-commit', 'repo.status()' isn't
invoked just after changing files inside same wlock scope.

But preceding 'dirstate.normal()' is invoked inside another wlock
scope via 'cmdutil.bailifchanged()', and in-memory changes should be
flushed at the end of that scope.

Therefore, timestamp N of clean "f" should be replaced by -1, if
'dirstate.write()' is invoked at N. It means that condition of this
issue isn't satisfied.
2015-07-08 17:01:09 +09:00
FUJIWARA Katsunori
0a80b7e42b tests: add extension to emulate invoking internalpatch at the specific time
This extension fakes "mtime" of patched files to emulate invoking
'patch.internalpatch()' at the specific time.

This is useful to reproduce timing critical issues fixed in subsequent
patches.
2015-07-08 17:01:09 +09:00
FUJIWARA Katsunori
51754ba82b context: write dirstate out explicitly after marking files as clean
To detect change of a file without redundant comparison of file
content, dirstate recognizes a file as certainly clean, if:

  (1) it is already known as "normal",
  (2) dirstate entry for it has valid (= not "-1") timestamp, and
  (3) mode, size and timestamp of it on the filesystem are as same as
      ones expected in dirstate

This works as expected in many cases, but doesn't in the corner case
that changing a file keeps mode, size and timestamp of it on the
filesystem.

The timetable below shows steps in one of typical such situations:

  ---- ----------------------------------- ----------------
                                           timestamp of "f"
                                           ----------------
                                           dirstate   file-
  time          action                     mem  file  system
  ---- ----------------------------------- ---- ----- -----
  N                                              -1    ***
       - make file "f" clean                            N

       - execute 'hg foobar'
         - instantiate 'dirstate'           -1   -1
         - 'dirstate.normal("f")'           N    -1
           (e.g. via dirty check)
         - change "f", but keep size                    N
  N+1
         - release wlock
           - 'dirstate.write()'             N    N

       - 'hg status' shows "f" as "clean"   N    N      N
  ---- ----------------------------------- ---- ----- -----

The most important point is that 'dirstate.write()' is executed at N+1
or later. This causes writing dirstate timestamp N of "f" out
successfully. If it is executed at N, 'parsers.pack_dirstate()'
replaces timestamp N with "-1" before actual writing dirstate out.

Occasional test failure for unexpected file status is typical example
of this corner case. Batch execution with small working directory is
finished in no time, and rarely satisfies condition (2) above.

This issue can occur in cases below;

  - 'hg revert --rev REV' for revisions other than the parent
  - failure of 'merge.update()' before 'merge.recordupdates()'

The root cause of this issue is that files are changed without
flushing in-memory dirstate changes via 'repo.commit()' (even though
omitting 'dirstate.normallookup()' on changed files also causes this
issue).

To detect changes of files correctly, this patch writes in-memory
dirstate changes out explicitly after marking files as clean in
'workingctx._checklookup()', which is invoked via 'repo.status()'.

After this change, timetable is changed as below:

  ---- ----------------------------------- ----------------
                                           timestamp of "f"
                                           ----------------
                                           dirstate   file-
  time          action                     mem  file  system
  ---- ----------------------------------- ---- ----- -----
  N                                              -1    ***
       - make file "f" clean                            N

       - execute 'hg foobar'
         - instantiate 'dirstate'           -1   -1
         - 'dirstate.normal("f")'           N    -1
           (e.g. via dirty check)
       ----------------------------------- ---- ----- -----
         - 'dirsttate.write()'              -1   -1
       ----------------------------------- ---- ----- -----
         - change "f", but keep size                    N
  N+1
         - release wlock
           - 'dirstate.write()'             -1   -1

       - 'hg status'                        -1   -1      N
  ---- ----------------------------------- ---- ----- -----

To reproduce this issue in tests certainly, this patch emulates some
timing critical actions as below:

  - timestamp of "f" in '.hg/dirstate' is -1 at the beginning

    'hg debugrebuildstate' before command invocation ensures it.

  - make file "f" clean at N
  - change "f" at N

    'touch -t 200001010000' before and after command invocation
    changes mtime of "f" to "2000-01-01 00:00" (= N).

  - invoke 'dirstate.write()' via 'repo.status()' at N

    'fakedirstatewritetime.py' forces 'pack_dirstate()' to use
    "2000-01-01 00:00" as "now", only if 'pack_dirstate()' is invoked
    via 'workingctx._checklookup()'.

  - invoke 'dirstate.write()' via releasing wlock at N+1 (or "not at N")

    'pack_dirstate()' via releasing wlock uses actual timestamp at
    runtime as "now", and it should be different from the "2000-01-01
    00:00" of "f".

BTW, this patch also changes 'test-largefiles-misc.t', because adding
'dirstate.write()' makes recent dirstate changes visible to external
process.
2015-07-08 17:01:09 +09:00
FUJIWARA Katsunori
4414fdbe75 tests: add extension to emulate invoking dirstate.write at the specific time
This extension fakes 'now' for 'parsers.pack_dirstate()' to emulate
invoking 'dirstate.write()' at the specific time, only when
'dirstate.write()' is invoked via functions below:

  - 'workingctx._checklookup()' (= 'repo.status()')
  - 'committablectx.markcommitted()'

This is useful to reproduce timing critical issues fixed in subsequent
patches.
2015-07-08 17:01:09 +09:00
Durham Goode
9f87b2f455 convert: add config for recording the source name
This creates the convert.hg.sourcename config option which will embed a user
defined name into each commit created by the convert. This is useful when using
the convert extension to merge several repositories together and we want to
record where each commit came from.
2015-07-08 10:31:09 -07:00
Durham Goode
54fa7659ef convert: support multiple specifed revs in git source
This allows specifying multiple revs/branches to convert from a git repo.
2015-07-08 10:29:11 -07:00
Durham Goode
0f795691b8 convert: add support for specifying multiple revs
Previously convert could only take one '--rev'. This change allows the user to
specify multiple --rev entries. For instance, this could allow converting
multiple branches (but not all branches) at once from git.

In this first patch, we disable support for this for all sources.  Future
patches will enable it for select sources (like git).
2015-07-08 10:27:43 -07:00
Pierre-Yves David
5e9a275007 bookmarks: change bookmark within a transaction
For some time, bookmark can and should be moved in the transaction. This
changeset migrates the 'hg bookmarks' commands to use a transaction.

Tests regarding rollback and transaction hooks are impacted for
obvious reasons. Some have to be slightly updated to keep testing the
same things. Some can just be dropped because they do not make sense
anymore.
2014-09-28 00:49:36 -07:00
Durham Goode
d70241d991 convert: add config to not convert tags
In some cases we do not want to convert tags from the source repo to be tags in
the target repo (for instance, in a large repository, hgtags cause scaling
issues so we want to avoid them). This adds a config option to disable
converting tags.
2015-06-29 13:40:20 -07:00
Yuya Nishihara
2f4ec7df91 templatekw: make {rev} return wdirrev instead of None
wdirrev/wdirnode identifiers are still experimental, but {node} is mapped to
wdirnode. So {rev} should do the same for consistency.

I'm not sure if templatekw can import scmutil. If not, we should move intrev()
to node module.
2015-07-02 22:18:21 +09:00
Yuya Nishihara
0ce55c7cf2 changeset_printer: use node.wdirrev to calculate meaningful parentrevs
Because we defined the working-directory revision is INT_MAX, it makes sense
that "hg log -r 'wdir()'" displays the "parent:" field. This is the same for
two revisions that are semantically contiguous but the intermediate revisions
are hidden.
2015-07-02 22:03:06 +09:00
Yuya Nishihara
950db6aab2 templatekw: apply manifest template only if ctx.manifestnode() exists
This will prevent crash by "hg log -r 'wdir()' -Tdefault". We could use the
pseudo ff... hash introduced by 187c3ec3d83f, but it isn't proven idea yet.
For now, I want to make "hg log" just works in order to test 'wdir()' revset.

Note that unlike its name, "{manifest}" is not a list of files in that
revision, but a pair of (manifestrev, manifestnode).
2015-03-14 17:58:18 +09:00
FUJIWARA Katsunori
3c2a14f766 hghave: allow adding customized features at runtime
Before this patch, there is no way to add customized features to
`hghave` without changing `hghave` and `hghave.py` themselves.

This decreases reusability of `run-tests.py` framework for third party
tools, because they may want to examine custom features at runtime
(e.g.  existence of some external tools).

To allow adding customized features at runtime, this patch makes
`hghave` import `hghaveaddon` module, only when `hghaveaddon.py` file
can be found in directories below:

  - `TESTDIR` for invocation via `run-tests.py`
  - `.` for invocation via command line

The path to the directory where `hghaveaddon.py` should be placed is
added to `sys.path` only while importing `hghaveaddon`, because:

  - `.` may not be added to `PYTHONPATH`

  - adding additional path to `sys.path` may change behavior of
    subsequent `import` for other features

`hghave` is terminated with exit code '2' at failure of `import
hghaveaddon`, because exit code '2' terminates `run-tests.py`
immediately.

This is a one of preparations for issue4677.
2015-07-03 06:56:03 +09:00
FUJIWARA Katsunori
4bae7bb820 import-checker.py: exit with code 0 if no error is detected
Before this patch, `import-checker.py` exits with non-0 code, if no
error is detected. This is unusual as Unix command.

This change may be a one of preparations for issue4677, because this
can avoid extra explanation about unusual exit code of
`import-checker.py` for third party tool developers.
2015-07-03 06:56:03 +09:00
FUJIWARA Katsunori
4a75be9bfa run-tests.py: add TESTDIR to PATH if it differs from RUNTESTDIR
Before this patch, `RUNTESTDIR` is added to `PATH`, but `TESTDIR`
isn't.

This doesn't cause any problems, if `run-tests.py` runs in `tests`
directory of Mercurial source tree. In this case, `RUNTESTDIR` should
be equal to `TESTDIR`.

On the other hand, if `run-tests.py` runs in `tests` of third party
tools, commands in that directory should be executed with explicit
`$TESTDIR/` prefix in `*.t` test scripts. This isn't suitable for the
policy "drop explicit $TESTDIR from executables" of Mercurial itself
(see fcb1c7d8c36e).

BTW, fcb1c7d8c36e describes that "$TESTDIR is added to the path" even
though `TESTDIR` isn't added to `PATH` exactly speaking, because
`TESTDIR` and `RUNTESTDIR` weren't yet distinguished from each other
at that time.

This is a one of preparations for issue4677.
2015-07-03 06:56:03 +09:00
FUJIWARA Katsunori
70980837e8 run-tests.py: add RUNTESTDIR to refer tests of Mercurial
Before this patch, there is no way to refer files under `tests` or so
of Mercurial source tree, when `run-tests.py` runs in `tests` of third
party tools. In this case, `TESTDIR` refers the latter `tests`.

This prevents third party tools from using useful tools in Mercurial
source tree (e.g. `contrib/check-code.py`).

This patch adds `RUNTESTDIR` environment variable to refer `tests` of
Mercurial source tree, in which `run-tests.py` now running is
placed. For example, tests of third party tools can refer
`contrib/check-code.py` in Mercurial source tree as
`$RUNTESTDIR/../contrib/check-code.py`.

BTW, for similarity with `TESTDIR` referring `test*s*` directory,
newly added environment variable isn't named as `RUNTEST*S*DIR`. In
addition to it, the corresponded local variable is also named as
`runtestdir`.

This is a one of preparations for issue4677.
2015-07-03 06:56:03 +09:00
FUJIWARA Katsunori
c5c4f4ba0d run-tests.py: execute hghave by the path relative to run-tests.py
Before this patch, `run-tests.py` executes `hghave` by the path
relative to `TESTDIR` (= cwd of `run-tests.py` running).

This prevents third party tools for Mercurial from running
`run-tests.py`, which is placed in `tests` of Mercurial source tree,
in `tests` of own source tree. In such cases, `TESTDIR` refers the
latter `tests`, and `hghave` doesn't exist in it.

This is a one of preparations for issue4677.
2015-07-03 06:56:03 +09:00
Matt Harbison
edbaac787d templatekw: make {latesttag} a hybrid list
This maintains the previous behavior of expanding {latesttag} to a string
containing all of the tags, joined by ':'.  But now it also allows list type
operations.

I'm unsure if the plural handling is correct (i.e. it seems like it is usually
"{foos % '{foo}'}"), but I guess we are stuck with this because the singular
form previously existed.
2015-07-06 23:23:22 -04:00
Matt Harbison
9e00498f91 templatekw: introduce the changessincelatesttag keyword
Archive is putting a value with the same name in the metadata file, to count all
of the changes not covered by the latest tag, instead of just along the longest
path.  It seems that this would be useful to have on the command line as well.
It might be nice for the name to start with 'latesttag' so that it is grouped
with the other tag keywords, but I can't think of a better name.

The initial version of this counted a clean wdir() and '.' as the same value,
and a dirty wdir() as the same value after it is committed.  Yuya objected on
the grounds of consistency [1].  Since revsets can be used to conditionally
select a dirty wdir() or '.' when clean, I can build the version string I need
and will defer to him on this.

[1] https://www.selenic.com/pipermail/mercurial-devel/2015-June/071588.html
2015-06-26 23:11:05 -04:00
Matt Harbison
1ea0ae2f3c help: support 'hg help template.somekeyword'
Previously the output was simply 'abort: help section not found'.
2015-07-04 23:11:32 -04:00
Matt Harbison
10b3c14d0d test-convert-git: use a relative gitmodule url
The absolute URL was causing this error with 1.9.5 on Windows, which had a
cascading effect:

  @@ -466,22 +466,24 @@
     >   url = $TESTTMP/git-repo5
     > EOF
     $ git commit -a -m "weird white space submodule"
  -  [master *] weird white space submodule (glob)
  -   Author: nottest <test@example.org>
  -   1 file changed, 3 insertions(+)
  +  fatal: bad config file line 6 in $TESTTMP/git-repo6/.gitmodules
  +  [128]
     $ cd ..
     $ hg convert git-repo6 hg-repo6
     initializing destination hg-repo6 repository
     scanning source...

For reasons unknown, there is still this delta on Windows:

  @@ -490,7 +490,6 @@
     $ git commit -q -m "missing .gitmodules"
     $ cd ..
     $ hg convert git-repo6 hg-repo6 --traceback
  -  fatal: Path '.gitmodules' does not exist in '*' (glob)
     initializing destination hg-repo6 repository
     scanning source...
     sorting...
2015-07-02 00:04:08 -04:00
Matt Harbison
95581f5463 test-convert-git: stablize for git 1.7.7.6
The output has apparently changed slightly since this version.  Since they are
just commits without any obvious importance to the test, and I can't figure out
how to glob them away, silence them.

Sample diffs were like this:

  @@ -468,7 +468,7 @@
     $ git commit -a -m "weird white space submodule"
     [master *] weird white space submodule (glob)
      Author: nottest <test@example.org>
  -   1 file changed, 3 insertions(+)
  +   1 files changed, 3 insertions(+), 0 deletions(-)
     $ cd ..
     $ hg convert git-repo6 hg-repo6
     initializing destination hg-repo6 repository
2015-07-01 20:53:12 -04:00
Colin Chan
f6f6547f72 shelve: only keep the latest N shelve backups
This will keep the backup directory from growing indefinitely. The number of
backups to keep can be set using the shelve.maxbackups config option (defaults
to 10 backups).
2015-07-01 13:14:03 -07:00
Colin Chan
cdd7ed7476 shelve: always backup shelves instead of deleting them
Instead of being deleted, shelve files are now moved into the .hg/shelve-backup
directory. This is designed similarly to how strip saves backups into
.ht/strip-backup. The goal is to prevent data loss especially when using
unshelve. There are cases in which a user can complete an unshelve but lose
some of the data that was shelved by, for example, resolving merge conflicts
incorrectly. Storing backups will allow the user to recover the data that was
shelved, at the expense of using more disk space over time.
2015-07-01 13:13:02 -07:00
Augie Fackler
8642271531 wireproto: correctly escape batched args and responses (issue4739)
This issue appears to be as old as wireproto batching itself: I can
reproduce the failure as far back as 6afda0a50a20 trivially by
rebasing the test changes in this patch, which was back in the 1.9
era. I didn't test before that change, because prior to that the
testfile has a different name and I'm lazy.

Note that the test thought it was checking this case, but it actually
wasn't: it put a literal ; in the arg and response for its greet
command, but the mangle/unmangle step defined in the test meant that
instead of "Fo, =;o" going over the wire, "Gp-!><p" went instead,
which doesn't contain any special characters (those being [.=;]) and
thus not exercising the escaping. The test has been updated to use
pre-unmangled special characters, so the request is now "Fo+<:o",
which mangles to "Gp,=;p". I have confirmed that the test fails
without the adjustment to the escaping rules in wireproto.py.

No existing clients of RPC batching were depending on the old behavior
in any way. The only *actual* users of batchable RPCs in core were:

1) largefiles, wherein it batches up many statlfile calls. It sends
hexlified hashes over the wire and gets a 0, 1, or 2 back as a
response. No risk of special characters.

2) setdiscovery, which was using heads() and known(), both of which
communicate via hexlified nodes. Again, no risk of special characters.

Since the escaping functionality has been completely broken since it
was introduced, we know that it has no users. As such, we can change
the escaping mechanism without having to worry about backwards
compatibility issues.

For the curious, this was detected by chance: it happens that the
lz4-compressed text of a test file for remotefilelog compressed to
something containing a ;, which then caused the failure when I moved
remotefilelog to using batching for file content fetching.
2015-06-30 19:19:17 -04:00
Yuya Nishihara
329cd61d62 revset: port extra() to support keyword arguments
This is an example to show how keyword arguments are processed.
2015-06-28 22:57:33 +09:00
Yuya Nishihara
411c9c1693 revset: add parsing rule for key=value pair
It will be used as an keyword argument.

Note that our "=" operator is left-associative. In general, the assignment
operator is right-associative, but we don't care because it isn't allowed to
chain "=" operations.
2015-06-27 17:05:28 +09:00
Gregory Szorc
5fe42b2975 import-checker: establish modern import convention
We introduce a new convention for declaring imports and enforce it via
the import checker script.

The new convention is only active when absolute imports are used, which is
currently nowhere. Keying off "from __future__ import absolute_import" to
engage the new import convention seems like the easiest solution. It is
also beneficial for Mercurial to use this mode because it means less work
and ambiguity for the importer and potentially better performance due to
fewer stat() system calls because the importer won't look for modules in
relative paths unless explicitly asked.

Once all files are converted to use absolute import, we can refactor
this code to again only have a single import convention and we can
require use of absolute import in the style checker.

The rules for the new convention are documented in the docstring of the
added function. Tests have been added to test-module-imports.t. Some
tests are sensitive to newlines and source column position, which makes
docstring testing difficult and/or impossible.
2015-06-28 12:46:34 -07:00
Matt Harbison
1147e2bdb5 templatekw: use a list of tags in getlatesttags() instead of joining them
This will be used in the next patch.

It also points out that the documentation for '{latesttag}' is not quite
accurate, since it says "most recent global tag" (singular).  I assume it is too
radical of a change to convert it to a list of strings.  At least ':' is
currently a reserved character in tag names.
2015-06-26 23:23:10 -04:00
Durham Goode
bc1c03c315 convert: improve support for unusual .gitmodules
Previously convert would throw an exception if it encountered a git commit with
a .gitmodules file that was malformed (i.e. was missing, but had submodule
files, or was malformed).

Instead of breaking the convert entirely, let's print error messages and move
on.
2015-06-29 17:19:58 -07:00
Durham Goode
8785908f11 convert: handle .gitmodules with non-tab whitespaces
The old implementation assumed .gitmodules file lines always began
with tabs. It can be any whitespace, so lets trim the lines
appropriately.
2015-06-29 17:19:18 -07:00
Durham Goode
782f192338 convert: fix bug with converting the same commit twice
Convert had a bug where it relied on repo.tip() to be the newly committed
commit. This was not the case if the commit already existed in the repository
(since repo.commitctx() did nothing, the tip() referenced some random other
commit and the revmap got corrupted).

This fixes it by using the node returned by repo.commitctx().
2015-06-29 13:39:05 -07:00
Yuya Nishihara
2dde4312da templater: remove workaround for escaped quoted string in quoted template
This patch backs out 297d563e92af which should no longer be needed.

The test for '{\"invalid\"}' is removed because the parser is permissive for
\"...\" literal.
2015-06-27 15:28:46 +09:00
Matt Mackall
b050378961 merge with stable 2015-07-01 16:33:31 -05:00
Pierre-Yves David
a7b191b4b3 transplant: only pull the transplanted revision (issue4692)
For some reason, transplant was pulling all remote revisions when transplanting
from a remote repository (unless --branch was
specified).
2015-06-29 19:09:42 -07:00
Pierre-Yves David
2558bde9af transplant: update test to use hash for remote transplant
Transplant is apparently allowing using revision numbers when transplanting
through http. I call this nonsense and update the test to use an explicit hash.

This "feature" will break in the next change fixing another bug.
2015-06-29 19:18:51 -07:00
Augie Fackler
c7fd94e3df sshserver: drop ancient do_{lock,unlock,addchangegroup} methods
These were marked as deprecated and dangerous way back in
f8e443eb02c9, which was first included in Mercurial 0.9.1. While it's
possible that clients from that long ago are still around somewhere,
they're risky for servers in that they want to lock the repo, and then
might leave it locked if they died before finishing their transaction.

Given that it's been 9 years, let's go ahead and cut this last
lingering tie with a basically-untested protocol.
2015-06-29 17:10:36 -04:00
Pierre-Yves David
b05f468b44 changegroup: properly compute common base in changeggroupsubset (issue4736)
The computation of roots was buggy, any ancestor of a bundled merge which was
also a descendant of the parents of a bundled revision were included as part of
the bundle. We fix it and add a test for strip (which revealed the problem).

Check the test for a practical usecase.
2015-06-29 11:20:09 -07:00
Matt Mackall
174e090e10 archive: fix changesincelatesttag with wdir() 2015-07-01 15:12:45 -05:00
Matt Harbison
b41110155b revset: fix a crash in parents() when 'wdir()' is in the set
The crash was "TypeError: expected string or Unicode object, NoneType found"
down in revlog.parentrevs().  This fixes heads() too (which is where I found
it.)
2015-06-29 10:34:56 -04:00
Matt Harbison
9f8b7aa09e workingctx: don't report the tags for its parents
This fixes the bad distance calculation for '{latesttagdistance}' mentioned in
the previous patch.
2015-06-28 13:38:03 -04:00