Commit Graph

12315 Commits

Author SHA1 Message Date
Augie Fackler
ee6c7f7907 tests: test-pathconflicts-merge.t requires symlinks
Once we're ready to turn this functionality on more widely, we might
want to write a symlink-free version of the test that can run on
Windows, but for now we'll just leave it disabled there.

Differential Revision: https://phab.mercurial-scm.org/D1710
2017-12-15 17:52:38 -05:00
Kostia Balytskyi
7405833d16 sshpeer: allow for additional environment passing to ssh exe
We already have the ability to customize the ssh command line arguments, let's
add the ability to customize its environment as well.

Example use-case is ssh.exe from Git on Windows. If `HOME` enviroment variable
is present and has some non-empty value, ssh.exe will try to access that
location for some stuff (for example, it seems for resolving `~` in
`.ssh/config`). Git for Windows seems to sometimess set this variable to the
value of `/home/username` which probably works under Git Bash, but does not
work in a native `cmd.exe` or `powershell`. Whatever the root cause, setting
`HOME` to be an empty string heals things. Therefore, some distributors
might want to set `sshenv.HOME=` in the configuration (seems less intrusive
that forcing everyone to tweak their env).

Test Plan:
- rt

Differential Revision: https://phab.mercurial-scm.org/D1683
2017-12-14 14:31:57 +00:00
Martin von Zweigbergk
0c2857a328 unamend: allow unamending if allowunstable is set
I don't see why unamend should be disallowed when allowunstable is
set. By switching to rewriteutil.precheck() we fix that and get more
consistent error messages (and some additional ones).

Differential Revision: https://phab.mercurial-scm.org/D1682
2017-12-13 17:03:39 -08:00
Wojciech Lis
f2ebe55279 lfs: using workers in lfs prefetch
This significantly speeds up lfs prefetch. With fast network we are
seeing ~50% improvement of overall prefetch times
Because of worker's API in posix we do lose finegrained progress update and only
see progress when a file finished downloading.

Test Plan:
Run tests:
./run-tests.py -l test-lfs*
....
# Ran 4 tests, 0 skipped, 0 failed.
Run commands resulting in lfs prefetch e.g. hg sparse --enable-profile

Differential Revision: https://phab.mercurial-scm.org/D1568
2017-12-11 17:02:02 -08:00
Martin von Zweigbergk
d28cfbc9a5 tests: avoid echo with backslash escapes
Some shells' echo (e.g. Bash's) do not interpret backslash escapes by
default, while others shells' do (e.g. Dash's). This led to a failure
in test-run-tests.t. We could switch to the more compatible printf,
but using a heredoc seems simpler, so that's what this patch does.

For anyone who wants to test this for themselves, remove the "unset
HGTEST_SHELL" line in helper-runtests.sh and pass
e.g. "--shell=/bin/dash" to the test runner.

Differential Revision: https://phab.mercurial-scm.org/D1699
2017-12-14 22:30:03 -08:00
Anton Shestakov
03a4817df8 hgweb: stop using HTML comments in <script>
Once upon a time, in 1995, there were browsers that didn't understand <script>
tags and they would simply show the code inside as text. This started a
tradition of wrapping everything inside <script> in <!-- HTML comments -->.
Nowadays, it's not only not needed, but can be considered harmful[1]:

- within XHTML documents, the source will actually be hidden from all browsers
  and rendered useless

- `--` is not allowed within HTML comments, so any decrement operations in
  script are invalid

[1]: http://www.javascripttoolbox.com/bestpractices/#comments
2017-12-15 12:15:58 +08:00
Augie Fackler
96c1d16249 merge with stable 2017-12-14 16:01:28 -05:00
Boris Feld
18540b6753 histedit: preserve active branch while histediting
The branch information was properly preserved in the changeset, but the
"active" branch of the working copy could be lost (the branch of the base
being used).

Histedit used to behave properly in this regard but the case was not tested
and regressed 4 years ago in c038baa4b6f0.
2017-12-12 18:22:11 +01:00
Augie Fackler
6f9e783026 tests: write and use a custom helper script to avoid find's -printf
-printf on find is a GNU-ism and will be banned in an upcoming
check-code change.

Differential Revision: https://phab.mercurial-scm.org/D1597
2017-12-05 16:55:41 -05:00
Yuya Nishihara
5ff5d9b38c dispatch: add HGPLAIN=+strictflags to restrict early parsing of global options
If this feature is enabled, early options are parsed using the global options
table. As the parser stops processing options when non/unknown option is
encountered, it won't mistakenly take an option value as a new early option.
Still "--" can be injected to terminate the parsing (e.g. "hg -R -- log"), I
think it's unlikely to lead to an RCE.

To minimize a risk of this change, new fancyopts.earlygetopt() path is enabled
only when +strictflags is set. Also the strict parser doesn't support '--repo',
a short for '--repository' yet. This limitation will be removed later.

As this feature is backward incompatible, I decided to add a new opt-in
mechanism to HGPLAIN. I'm not pretty sure if this is the right choice, but
I'm thinking of adding +feature/-feature syntax to HGPLAIN. Alternatively,
we could add a new environment variable. Any bikeshedding is welcome.

Note that HGPLAIN=+strictflags doesn't work correctly in chg session since
command arguments are pre-processed in C. This wouldn't be easily fixed.
2017-11-23 22:17:03 +09:00
Yuya Nishihara
b0d1fec38b fancyopts: add early-options parser compatible with getopt()
The next patch will add a flag for strict parsing of early options, where
we'll have to parse all early options at once instead of processing them
one-by-one by dispatch._earlygetopt(). That's why I decided to hook
fancyopts().

All dispatch._early*opt() functions is planned to be replaced with this
function. But in this stable series, only the strict mode will be handled
by fancyopts.earlygetopt().
2017-11-23 22:04:53 +09:00
Matt Harbison
c1ef7c5a7a largefiles: pay attention to dropped standin files when updating largefiles
Previously, the largefile for a dropped standin would be deleted here, and then
restored from the cache.  This had the effect of clobbering uncommitted changes
if a revert caused the file to be forgotten, which is not what happens with a
normal file.  Now the removal and update is skipped for dropped largefiles, and
the corresponding standin is deleted from disk.

This was noticed when working on issue5738 because the forgotten standin files
were left behind, and that changes the behavior of the next rename to that
directory.  My first attempt was to cleanup the standins before calling this.
That failed, because this function deletes the largefile if the corresponding
standin is missing.

This function is called by the revert command, merge (and therefore update), and
patch, via the scmutil.marktouched() override.  So it should be pretty narrow in
scope.

I didn't mark issue5738 as fixed because the move related issues can still
happen if the main tree and the .hglf subtree get out of sync somehow.  I don't
see an easy fix for that, but that should be an edge case.  If whoever queues
this thinks it is good enough to close out the bug and can cram it into the
summary, go for it.
2017-11-12 23:45:14 -05:00
Matt Harbison
9d58e7d4a4 test-largefiles: demonstrate problems with renaming and reverting a directory
These things were uncovered looking at issue5738.

First, if the destination directory exists under .hglf, the source is moved
under the destination instead of renaming the last component for `hg mv srcdir
dstdir`.  This is extra confusing, because it occurs even if the user visible
destination (i.e. the path _not_ under .hglf) does not exist.

Additionally, when a largefile is forgotten via revert, any modifications end up
getting clobbered.  For normal files, the forgotten file is left unchanged, as
shown by test-import.t.  The forget command on a largefile will correctly leave
the file unmodified.
2017-11-12 00:24:38 -05:00
Matt Harbison
d913b0c5b9 tests: add globs for Windows 2017-11-11 12:37:05 -05:00
Yuya Nishihara
d46d87090c cat: record the current behavior of wildcard matches in subrepos
Mercurial subrepos support any match patterns.
2017-11-25 15:29:34 +09:00
Matt Harbison
40cf71ae24 convert: avoid wrong lfconvert defaults by moving configitems to core
The `hg lfconvert --to-normal` command uses the convert extension internally to
work its magic, but that produced devel-warn messages if the convert extension
wasn't loaded by the user.  The test in 658e7a6d93e0 (modified here) wasn't
showing the warnings because the convert extension was loaded via $HGRCPATH.
Most of the config options default to None/False, but 'hg.usebranchnames' and
'hg.tagsbranch' are supposed to default to True and 'default' respectively.

The first iteration of this was to ui.setconfig() inside lfconvert, to force the
convert extension to load.  But there really is no precedent for doing this, and
check-config complained that 'extensions.convert' isn't documented.  Yuya
suggested this alternative.

This partially backs out 448e09d8859d.
2017-11-28 23:20:08 -05:00
Yuya Nishihara
52b06fe73d dispatch: verify result of early command parsing
Before, early options were stripped from args, and because of this, some
kind of parsing errors weren't reported. For example,

  $ hg ci -m -Ra file

would execute "hg ci -m file" in repository "a".

This patch fixes the issue by parsing early options again by real getopt-based
parser, and verifying the results. If the early parsing appears wrong, hg just
aborts. The current error message seems not nice, and should be improved, maybe
in V2 or follow-up.

Note that this isn't a security feature because we can still do anything by
using shell aliases.
2017-11-11 12:40:13 +09:00
Yuya Nishihara
82ae4401fe dispatch: abort if early boolean options can't be parsed
Perhaps we'll need to restrict the parsing rules of --debugger and --profile,
where this patch will help us know why the --debugger option doesn't work.

I have another series to extend this feature to --config/--cwd/-R, but even
with that, shell aliases can be used to get around the restriction.
2017-11-11 14:02:41 +09:00
Yuya Nishihara
8f01395e02 dispatch: stop parsing of early boolean option at "--" 2017-11-10 22:27:26 +09:00
Yuya Nishihara
7137d3d976 dispatch: do not drop unpaired argument at _earlygetopt()
Before, "hg log -R" just worked.
2017-11-11 12:09:19 +09:00
Yuya Nishihara
0704342965 amend: update .hgsubstate before committing a memctx (issue5677)
This is a minimal copy of localrepo.commit(). As the current amend() function
heavily depends on the wctx API, it wasn't easy to port it to use a separate
status tuple. So for now, wctx._status is updated in-place.
2017-11-04 20:07:40 +09:00
Yuya Nishihara
ec502c74f8 tests: demonstrate that .hgsubstate isn't updated on amend
Loosely based on test-amend.t and test-subrepo.t
2017-11-04 18:54:57 +09:00
Yuya Nishihara
df1f119682 amend: do not drop missing files (issue5732)
samefile() can be wrong since wctx.manifest() does not include missing files,
whereas missing files should be preserved on commit.
2017-11-04 23:48:19 +09:00
Yuya Nishihara
e6c4026850 amend: do not take untracked files as modified or clean (issue5732)
fctx.exists() shouldn't be used here as it checks if a file physically exists,
which may disagree with the dirstate.
2017-11-04 23:45:59 +09:00
Yuya Nishihara
5ea9a0ac48 tests: add more complete test for status changes on amend
This demonstrates that missing/untracked files are handled incorrectly. The
correct outputs are suppressed by (false !), and wrong outputs are added with
(true !) instead.

The setup code is copied from test-status-rev.t.
2017-11-04 23:40:26 +09:00
Mark Thomas
ce1d136e58 scmutil: don't try to delete origbackup symlinks to directories (issue5731)
When origbackuppath is set, when looking to see if a file we are backing up
conflicts with a directory in the origbackuppath, we incorrectly match on
symlinks to directories.  This means we try to call vfs.rmtree on the
symlink, which fails.

Differential Revision: https://phab.mercurial-scm.org/D1311
2017-11-03 09:27:36 -07:00
Mark Thomas
63c7a46713 tests: add a test demonstrating issue5731
If origbackups are in use, a symlink to a valid directory is backed up, and an
update is made that attempts to backup a file or link over that symlink, we
abort with a bad error message instead of successfully updating.

Differential Revision: https://phab.mercurial-scm.org/D1310
2017-11-03 09:27:35 -07:00
Gregory Szorc
bcd6d015d2 subrepo: use per-type config options to enable subrepos
We change subrepos.allowed from a list of allowed subrepo types to
a combination of a master switch and per-type boolean flag.

If the master switch is set, subrepos can be disabled wholesale.

If subrepos are globally enabled, then per-type options are
consulted. Mercurial repos are enabled by default. Everything else
is disabled by default.
2017-11-06 22:32:41 -08:00
Yuya Nishihara
e5730ba521 subrepo: disable git and svn subrepos by default (BC) (SEC)
We have a security issue with git subrepos. I'm not sure if svn subrepo is
vulnerable, but it seems not 100% safe to allow writing arbitrary data into
a metadata directory. So for now, only hg subrepo is enabled by default.

Maybe we should improve the help to describe why git/svn subrepos are
disabled.
2017-11-05 21:51:42 +09:00
Yuya Nishihara
9752ef714d subrepo: extend config option to disable subrepos by type (SEC)
This allows us to minimize the behavior change introduced by the next patch.

I have no idea which config style is preferred in UX POV, but I decided to
get things done.

 a) list: 'allowed = hg, git, svn'
 b) sub option: 'allowed.hg = True' or 'allowed:hg = True'
 c) per-type action: 'hg = allow', 'git = abort'
2017-11-05 21:48:58 +09:00
Yuya Nishihara
f864544d81 subrepo: add config option to reject any subrepo operations (SEC)
This is an alternative workaround for the issue5730.

Perhaps this is the simplest way of disabling subrepo operations. It does
nothing clever, but just aborts if Mercurial starts accessing to a subrepo.

I think Greg's patch is more useful since it allows us to at least check
out the parent repository. However, that would be confusing if the default
is flipped to checkout=False and subrepos are silently ignored.

I don't like the config name 'allowed', but I couldn't get any better name.
2017-11-05 21:22:07 +09:00
Yuya Nishihara
dac75264c0 subrepo: disallow symlink traversal across subrepo mount point (SEC)
It wasn't easy to extend the pathauditor to check symlink traversal across
subrepos because pathauditor._checkfs() rejects a directory having ".hg"
directory. That's why I added the explicit islink() check.

No idea if this patch is necessary after we've fixed the issue5730 by
splitting submerge() into planning and execution phases.
2017-11-03 20:12:50 +09:00
Yuya Nishihara
ace48cdb7b tests: show symlink traversal across subrepo mount point (SEC)
Also adds a couple of tests where the auditor does work as expected.
2017-11-03 19:17:25 +09:00
Jun Wu
04a084983c test-dispatch: stabilize the test
When cwd is removed and `hg` is executed, some shells may run `getcwd`
before forking and executing, some may not do it, some may print a
different error message.

The test should be shell-independent so let's just avoid checking the error
message.

Differential Revision: https://phab.mercurial-scm.org/D1282
2017-11-01 14:22:26 -07:00
Kevin Bullock
0c315becfe internals: update test-help.t for config registrar copy-edit 2017-11-01 15:34:22 -05:00
Matt Harbison
ba66d4513f help: minor copy editing for grammar 2017-10-31 23:09:29 -04:00
Yuya Nishihara
21161a9bda configitems: relax warning about unwanted default value
The original condition was a bit harsh for extension authors since third-party
extensions need to preserve compatibility with older Mercurial versions, where
no defaults would be loaded from the configtable. So let's silence the warning
if the given default value matches, which should be harmless.
2017-10-31 22:37:30 +09:00
Yuya Nishihara
1fa3521832 test-static-http: flush access log per request
It appears that stderr is fully buffered on Windows.

# no-check-commit because of log_message() function
2017-10-29 17:53:52 +09:00
Siddharth Agarwal
2d9d1fcc19 merge: disable path conflict checking by default (issue5716)
We shouldn't ship a severe perf regression in hg update for 4.4.

Differential Revision: https://phab.mercurial-scm.org/D1223
2017-10-24 11:15:30 -07:00
Mark Thomas
1d4d162882 dirstate: clean up when restoring identical backups
When a dirstate backup is restored, it is possible that no actual changes to
the dirstate have been made.  In this case, the backup is still a hardlink
to the original dirstate.

Unfortunately, `os.rename` silently fails (nothing happens, and no error
occurs) when `src` and `dst` are hardlinks to the same file.  As a result,
the backup is left lying around.  Over time, these files accumulate.

When restoring dirstate backups, check if the backup and the dirstate are
the same file, and if so, just delete the backup.

Differential Revision: https://phab.mercurial-scm.org/D1201
2017-10-20 05:53:35 -07:00
Mark Thomas
0a3c5d4762 tests: add a test demonstrating failure to clean up dirstate backups
Differential Revision: https://phab.mercurial-scm.org/D1200
2017-10-20 05:53:33 -07:00
Matt Harbison
bd8eaff510 tests: adjust hooks for Windows
I'm not sure why these weren't working on Windows.  The failures were generally
in the style of:

  -  remote: phase-move: cb9a9f314b8b07ba71012fcdbc544b5a4d82ff5b:  1 -> 0
  +  remote: "phase-move: $HG_NODE:  $HG_OLDPHASE -> $HG_PHASE"

and

  -  abort: pretxnclose-bookmark.force-forward hook exited with status 1
  -  [255]
  +  abort: pretxnclose-bookmark.force-public hook exited with status 255
  +  [255]

These failures originated in 5625d0ddc285::6e3e88681b23.
2017-10-20 23:01:56 -04:00
Matt Harbison
0aa0daf3f0 test-log-linerange: open binary file in binary mode
The '\n' characters were being translated on Windows, throwing off hg and git
hashes, as well as the file content in the diff.
2017-10-20 21:29:15 -04:00
Matt Harbison
80170c3cd7 test-arbitraryfilectx: stabilize for Windows
Previously, the second last test (context.arbitraryfilectx(..)) returned True on
Windows.  I changed the repo setup sequence to import a patch, so that way the
repo would have a proper symlink.  That made the last test fail, since it is
comparing files in wdir(), one of which is not the expected symlink.

Apparently the (feature !) line matching doesn't work well with (no-eol), so I
had to conditionalize the test instead of the output.
2017-10-19 23:16:48 -04:00
Boris Feld
2d8cec51a9 internal-doc: document the config register mechanism
This explains the various usage and feature of the config register introduced
in Mercurial 4.3 and 4.4.
2017-10-27 18:19:07 +02:00
Yuya Nishihara
73e9f96212 statichttprepo: prevent loading dirstate over HTTP on node lookup (issue5717)
This seems a bit hacky, but works well. There should be no reason that
static-http repo had to load dirstate.

Initially I tried to proxy os.stat() call through vfs so that statichttpvfs
could hook it, but there wasn't a good error value which the statichttpvfs
could return to get around the util.filestat issue.
2017-10-25 21:58:03 +09:00
Yuya Nishihara
e368124f71 test-static-http: show all files accessed over HTTP
This provides some confidence on files that should be loadable over HTTP.
Hopefully it will prevent future bugs.
2017-10-25 21:21:42 +09:00
muxator
aaf12eb558 commands: be more uniform in an "hg update" error message
"can only specify one of -C/--clean, -c/--check, or -m/merge"
becomes
"can only specify one of -C/--clean, -c/--check, or -m/--merge"
2017-10-23 22:45:24 +02:00
Yuya Nishihara
feaa3f8a8c mq: copy pager attributes back to qrepo.ui
If the legacy pager extension is enabled, a pager is started through repo.ui
at dispatch._runcommand(). After that, mqcommand() creates a qrepo with a
fresh repo.baseui, at which point pager information was lost and another pager
would be spawned by the modern pager interface.

This is a minimal workaround for the problem.
2017-10-21 13:42:43 +09:00
Joe Blaylock
f9b24c83ac help: fix typo in hg merge documentation 2017-10-20 14:15:46 -07:00