Summary:
The helper could be used in individual tests to enable chg if chg exists.
This allows us to have more precise control on what tests to use chg instead
of using a global flag in run-tests.py.
This makes certain tests containing many hg commands much faster. For example,
`test-revset.t` took 99 seconds before:
% ./run-tests.py test-revset.t --time
.
# Ran 1 tests, 0 skipped, 0 failed.
# Producing time report
start end cuser csys real Test
0.000 99.990 86.410 12.000 99.990 test-revset.t
And 10 seconds after:
% ./run-tests.py test-revset.t --time
.
# Ran 1 tests, 0 skipped, 0 failed.
# Producing time report
start end cuser csys real Test
0.000 10.080 0.380 0.130 10.080 test-revset.t
Also enable it for some other tests. Note the whitelist is not complete. We
probably want to whitelist more tests in the future.
The feature could be opted out by deleting `contrib/chg/chg`.
Reviewed By: phillco
Differential Revision: D6767036
fbshipit-source-id: 8220cf408aa198d5d8e2ca5127ca60e2070d3444
Summary: Also change the internal API so it no longer accepts the "heads" argument.
Reviewed By: ryanmce
Differential Revision: D6745865
fbshipit-source-id: 368742be49b192f7630421003552d0a10eb0b76d
# skip-blame because this was mechanically rewritten the following script. I
ran it on both *.t and *.py, but none of the *.py changes were proper. All *.t
ones appear to be, and they run without addition failures on both Windows and
Linux.
import argparse
import os
import re
ap = argparse.ArgumentParser()
ap.add_argument('path', nargs='+')
opts = ap.parse_args()
globre = re.compile(r'^(.*) \(glob\)(.*)$')
for p in opts.path:
tmp = p + '.tmp'
with open(p, 'rb') as src, open(tmp, 'wb') as dst:
for line in src:
m = globre.match(line)
if not m or '$LOCALIP' in line or '*' in line:
dst.write(line)
continue
if '?' in line[:-3] or ('?' in line[:-3] and line[-3:] != '(?)'):
dst.write(line)
continue
dst.write(m.group(1) + m.group(2) + '\n')
os.unlink(p)
os.rename(tmp, p)
When ui.origbackuppath is set, .orig files are stored outside of the working
copy, however they still have a .orig suffix appended to them. This can cause
unexpected conflicts, particularly when tracked files or directories have .orig
at the end.
This change removes the .orig suffix from files stored in an out-of-tree
origbackuppath.
Test Plan:
Update and run unit tests.
Differential Revision: https://phab.mercurial-scm.org/D679
The POSIX documentation about "cp" [1] says:
....
RATIONALE
....
Earlier versions of this standard included support for the -r option to
copy file hierarchies. The -r option is historical practice on BSD and
BSD-derived systems. This option is no longer specified by POSIX.1-2008
but may be present in some implementations. The -R option was added as a
close synonym to the -r option, selected for consistency with all other
options in this volume of POSIX.1-2008 that do recursive directory
descent.
The difference between -R and the removed -r option is in the treatment
by cp of file types other than regular and directory. It was
implementation-defined how the - option treated special files to allow
both historical implementations and those that chose to support -r with
the same abilities as -R defined by this volume of POSIX.1-2008. The
original -r flag, for historic reasons, did not handle special files any
differently from regular files, but always read the file and copied its
contents. This had obvious problems in the presence of special file
types; for example, character devices, FIFOs, and sockets.
....
....
Issue 6
The -r option is marked obsolescent.
....
Issue 7
....
The obsolescent -r option is removed.
....
(No "Issue 8" yet)
Therefore it's clear that "cp -R" is strictly better than "cp -r".
The issue was discovered when running tests on OS X after 2e4d149e62aa.
[1]: pubs.opengroup.org/onlinepubs/9699919799/utilities/cp.html
During merge, added (from one perspective) file can be reported as "modified".
To work around that, revert was testing if modified file were present in the
parent manifest and marking them as "added" in this case. However, we should be
checking against the target revision manifest instead. Otherwise see file as
"newly added" even if they exist in the target revision.
That revert behavior regressed in 3657ae7519b7.
To detect change of a file without redundant comparison of file
content, dirstate recognizes a file as certainly clean, if:
(1) it is already known as "normal",
(2) dirstate entry for it has valid (= not "-1") timestamp, and
(3) mode, size and timestamp of it on the filesystem are as same as
ones expected in dirstate
This works as expected in many cases, but doesn't in the corner case
that changing a file keeps mode, size and timestamp of it on the
filesystem.
The timetable below shows steps in one of typical such situations:
---- ----------------------------------- ----------------
timestamp of "f"
----------------
dirstate file-
time action mem file system
---- ----------------------------------- ---- ----- -----
N -1 ***
- make file "f" clean N
- execute 'hg foobar'
- instantiate 'dirstate' -1 -1
- 'dirstate.normal("f")' N -1
(e.g. via dirty check)
- change "f", but keep size N
N+1
- release wlock
- 'dirstate.write()' N N
- 'hg status' shows "f" as "clean" N N N
---- ----------------------------------- ---- ----- -----
The most important point is that 'dirstate.write()' is executed at N+1
or later. This causes writing dirstate timestamp N of "f" out
successfully. If it is executed at N, 'parsers.pack_dirstate()'
replaces timestamp N with "-1" before actual writing dirstate out.
Occasional test failure for unexpected file status is typical example
of this corner case. Batch execution with small working directory is
finished in no time, and rarely satisfies condition (2) above.
This issue can occur in cases below;
- 'hg revert --rev REV' for revisions other than the parent
- failure of 'merge.update()' before 'merge.recordupdates()'
The root cause of this issue is that files are changed without
flushing in-memory dirstate changes via 'repo.commit()' (even though
omitting 'dirstate.normallookup()' on changed files also causes this
issue).
To detect changes of files correctly, this patch writes in-memory
dirstate changes out explicitly after marking files as clean in
'workingctx._checklookup()', which is invoked via 'repo.status()'.
After this change, timetable is changed as below:
---- ----------------------------------- ----------------
timestamp of "f"
----------------
dirstate file-
time action mem file system
---- ----------------------------------- ---- ----- -----
N -1 ***
- make file "f" clean N
- execute 'hg foobar'
- instantiate 'dirstate' -1 -1
- 'dirstate.normal("f")' N -1
(e.g. via dirty check)
----------------------------------- ---- ----- -----
- 'dirsttate.write()' -1 -1
----------------------------------- ---- ----- -----
- change "f", but keep size N
N+1
- release wlock
- 'dirstate.write()' -1 -1
- 'hg status' -1 -1 N
---- ----------------------------------- ---- ----- -----
To reproduce this issue in tests certainly, this patch emulates some
timing critical actions as below:
- timestamp of "f" in '.hg/dirstate' is -1 at the beginning
'hg debugrebuildstate' before command invocation ensures it.
- make file "f" clean at N
- change "f" at N
'touch -t 200001010000' before and after command invocation
changes mtime of "f" to "2000-01-01 00:00" (= N).
- invoke 'dirstate.write()' via 'repo.status()' at N
'fakedirstatewritetime.py' forces 'pack_dirstate()' to use
"2000-01-01 00:00" as "now", only if 'pack_dirstate()' is invoked
via 'workingctx._checklookup()'.
- invoke 'dirstate.write()' via releasing wlock at N+1 (or "not at N")
'pack_dirstate()' via releasing wlock uses actual timestamp at
runtime as "now", and it should be different from the "2000-01-01
00:00" of "f".
BTW, this patch also changes 'test-largefiles-misc.t', because adding
'dirstate.write()' makes recent dirstate changes visible to external
process.
The phase of the pending commit depends on the parent of the working directory
and on the phases.newcommit configuration.
First, this information rather depend on the commit line which describe the
pending commit.
Then, we only want to be advertised when the pending phase is going to be higher
than the default new commit phase.
So the format will change from
$ hg summary
parent: 2:ab91dfabc5ad
foo
parent: 3:24f1031ad244 tip
bar
branch: default
commit: 1 modified, 1 unknown, 1 unresolved (merge)
update: (current)
phases: 1 secret (secret)
to
parent: 2:ab91dfabc5ad
foo
parent: 3:24f1031ad244 tip
bar
branch: default
commit: 1 modified, 1 unknown, 1 unresolved (merge) (secret)
update: (current)
phases: 1 secret
The number of draft and secret changesets are currently not summarized.
This is an important information because the number of drafts give some rough
idea of the number of outgoing changesets in typical workflows, without needing
to probe a remote repository. And a non-zero number of secrets means that
those changeset will not be pushed.
If the repository is "dirty" - some draft or secret changesets exists - then
summary will display a line like:
phases: X draft, Y secret (public)
The phase in parenthesis corresponds to the highest phase of the parents of
the working directory, i.e. the current phase.
By default, the line is not printed if the repository is "clean" - all
changesets are public - but if verbose is activated, it will display:
phases: (public)
On the other hand, nothing will be printed if quiet is in action.
A few tests have been added in test-phases.t to cover the -v and -q cases.
'hg revert -I foo' currently fails with
abort: no files or directories specified
(use --all to revert all files, or 'hg update 1' to update)
It doesn't seem intentional that -I/-X without other paths or
--all/--interactive should fail, and it doesn't seem that harmful to
allow it either, so let's just do that.
When calculating status involving the working copy and a revision
other than the parent of the working copy, the files that are not in
the working context manifest ('mf2' in the basectx._buildstatus())
will be reported as removed (note that deleted files _are_ in the
working context manifest). However, if the file is reported as deleted
in the dirstate, it will get that status too (as shown by failing
tests).
Fix by removing deleted files from the 'removed' list after the main
loop in _buildstatus().
Add a parameter to generate-working-copy-states.py that indicates
how many changesets are wanted. This number impacts all the
subcommands. The current 'filelist' subcommand becomes 'filelist 2',
the current 'base' and 'parent' subcommands become 'state 2 1' and
'state 2 2' respectively, while 'wc' becomes 'state 2 wc'.
See earlier patch for motivation.
With the recent change in naming of the generated files, it becomes
much easier to generate the files by iterating over all the possible
states than over the state transitions.
When a file is missing in the 'parent' version and is tracked but
missing in the working directory, which happens by the 'missing' or
'removed' types, and the 'clean' type in the working directory, the
file does not exist in the working directory (unlike it would had the
'deleted' type been used). Thus, the *_missing_missing_tracked are not
actually tracked and they end up testing the same state as
*_missing_missing_untracked. To make them tracked, add a temporary
file, just like we do for the delete case. For simplicity's sake,
let's make sure the gen-revert-cases.py script always puts a file in
the working directory, whether or not it's going to be deleted.
Future patches will change how the output of 'gen-revert-cases.py
filelist' is generated, so now we want the order to depend on just the
filename again.
This is the main patch in a series. See motivation in earlier patch.
In this patch, we actually change the names of the generated
files. For example, the file that is currently called missing_clean
becomes missing_missing_missing-tracked and it's clearer that it
should be tracked. It turns out that since the state was not
previously clear, it ended up testing an untracked state, which was
the same as for missing_clean. We'll fix this in a later patch.
Let's also change the content from (base,parent,wc) to
(content1,content2,content3) to make them all the same length so they
line up when displayed.
The next patch will change the names of the files produced by the
script in test-revert. In order to reduce the size and increase the
clarity of the next patch, make the order produced by the internal
'gen-revert-cases.py filelist' command independent of the filenames.
By putting the file content rather than keys in the 'combination'
list, we restrict the knowledge of 'ctxcontent' and 'wccontent' to the
loop generating the combinations. That will make it easier to replace
the generation code.
The 'wccontent' variable has eight different states, four of them
tracked, and the other four untracked (at least when the file existed
in the parent revision). Among these eight states, 'removed' sticks
out by lacking the 'untracked-' prefix despite resulting in an
untracked state. To make the symmetry clearer, and to prepare for
future patches, rename 'removed' to 'untracked-deleted', which is
exactly what it is.
Note that, unlike 'remove', 'deleted' is configured in
gen-revert-cases.py to have content in the working directory and that
that content is instead expected to be removed in the test script.
However, no changes are needed to the test script, since it already
contains 'hg forget *untracked*' and 'rm *deleted*', which together
have the same effect as 'hg remove'.
See additional motivation in earlier patch.
The tests for removed_deleted and removed_removed test the same state
as removed_clean and removed_untracked-clean, respectively. Drop the
duplicate tests.
See additional motivation in earlier patch.
The tests for added_revert and added_untracked-revert test the same
state as added_deleted and added_removed, respectively. Drop the
duplicate tests.
See additional motivation in earlier patch.
This is the first step in a series that aims to put the state, not the
state transitions, in the filenames of the files generated by the
gen-revert-cases.py script. The possible state of a file in a revision
and in the working copy is only whether it exists and what its content
is (the tests don't care check flags). In the dirstate, the only state
is whether it's tracked or not. With the new naming, the file that is
currently called modified_untracked-clean now becomes
content1_content2_content2-untracked, for example.
By putting these states in the filename, it becomes easier to see that
we're not missing or duplicating any state, and to check that the
state is what we think it is. For example, the file that is currently
called missing_clean becomes missing_missing_missing-tracked and it's
clearer that it should be tracked.
Putting the content in the filename will also make the tests of file
content (e.g. "cat ../content-parent.txt") very obvious.
When we put the state in the filename, the filenames clearly need to
be unique. However, it turns out that some states are currently tested
multiple times. The 'revert' transition in the script means to take
the content from the grandparent. If the parent is the same as the
grandparent, there is no change compared to the parent, which is
exactly what 'clean' means. Avoid testing the same state twice.
It seems like the last known misbehvior that the comment was referring
to was dealt with in 708ed580cbea (revert: properly back up added
files with local modification, 2014-08-31).
These files were previously not backed up because the backup mechanism was not
smart enough. This leads to data lose for the user since uncommitted contents
were discarded.
We now properly move the modified version to <filename>.orig before deleting it.
We have to use a small hack to do a different action if "--no-backup" is
specified. This is needed because the backup process is actually a move (not a
copy) so the file is already missing when we backup. The internet kitten is a
bit disapointed about that, but such is life.
This patch concludes the "lets refactor revert" phases. We can now open the
"Lets find stupid bug with renames and merge" phases.
I'm sure that now that the code is clearer we could do it in another simpler
way, but I consider the current improvement good enough for now.
There are multiple possible cases for added files. But it's all handled by magic
much lower in the stack. We document them, simplify the codes and move on.
Using status information against the target ensures we are catching all
files with modifications that need reverting.
We still need to distinguish fresh modifications for backup purpose.
test-largefile is affected because it reverted a file that needs no content
change.
Tracking clean files is the simplest way to be able to reports files that need
no changes. So we explicitly retrieve them.
This fixes a couple of test outputs where the lack of changes was not reported.
When a file was marked as removed in the working copy and did not existed in the
target of the revert, we did not issued any message pointing that no change was
needed to the file (implicitly saying that revert had changed the file).
We now properly issue a message in this situation. Tests change in and handful
of case where the message was documented as missing.
Unlike untracked, the file is also missing from the working directory.
This test highlights a small misbehavior in output when reverting to another
revision.
We now also test reverting file to another revision's content. However
this differs from previously introduced test by using the explicit path
of each "case file" when calling revert. This should result in the
same result regarding file content and backup creation, but the output
of the `hg revert` call should differ.
We now also test reverting file to the working directory parent
content. However this differs from the previously introduced test by using
the explicit path of each "case file" when calling revert. This should
result in the same result regarding file content and backup creation,
but the output of the `hg revert` call should differ.