Summary: The template keyword for split was not producing output because obsmarkers with multiple successors were being filtered out.
Test Plan: Correct "Split into XXX, YYY" labels appear in smartlog. Tests have been added for all of the obsolescence-related smartlog template keywords.
Reviewers: #sourcecontrol, rmcelroy, durham
Reviewed By: durham
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4396453
Signature: t1:4396453:1484880307:bfdc60385e42fee169ff2f0ec4f85dd0747b55c9
Summary: Automatically rebase unstable commits left behind after a split or fold in the middle of a stack. Previously, users had to run `hg restack` to fix their stacks after the fact. This change also fixes the issue identified in t14560234 whereby the old stack (entirely obsolete) stack would be left visible after splitting or folding a head commit due to the inhibit extension not correctly deinhibiting the old stack.
Test Plan: Create a stack of commits and perform `hg split` or `hg fold` on a commit (or set of commits) that have descendants. After the operation these commits should be rebased, preserving the linearity of the stack. See unit tests for specific test cases.
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4432833
Tasks: 14560234, 15266009
Signature: t1:4432833:1484880689:1e07e421c8871dfef8bd016454aaed5a88a4a7d1
Summary:
After my fix in D4427670 the test-check* tests now use the correct config when
running hg commands in the main repository. The scripts/utils.py code no
longer needs to explicitly enable the lz4revlog extension when running the unit
tests in a repository using lz4revlog.
Test Plan: Confirmed that "arc unit" still passes in a repository with lz4revlog enabled.
Reviewers: #sourcecontrol, quark, stash, rmcelroy
Reviewed By: rmcelroy
Subscribers: net-systems-diffs@, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4431170
Signature: t1:4431170:1484831433:a69be31042572ba6d0f5f3efd4775a260fda1414
Summary:
Many of the template keywords in our extensions were being registered
incorrectly, causing their help output to be rendered incorrectly in the
"hg help templates" output. The ones in smartlog.py were particularly bad, as
most of them showed only their description, without displaying the name of the
template. In smartlog.py only singlepublicsuccessor was being displayed
correctly, because it's docstring explicitly included it's own name at the
start.
This fixes all of our extensions to consistently use the
registrar.templatekeyword() decorator to register the keywords. This decorator
automatically prefixes the help message with the keyword name. The
mercurial/extensions.py code will explicitly check to see if an extension
contains an "templatekeyword" attribute, and if so it will register any
keywords contained in this registry after calling extsetup().
Test Plan:
Added new unit tests to check the output of "hg help templates" for the
affected keywords.
Reviewers: #sourcecontrol, kulshrax, ikostia, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, net-systems-diffs@, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4427729
Signature: t1:4427729:1484831476:17b478a5e867dfc3f85402588c381bf8b1831107
Summary:
Update scripts/unit.py with an option to skip running all of the test-check*
tests. This makes it easier to run just a single specific test when developing
a unit test itself.
Test Plan:
Used "./scripts/unit.py --skip-checks tests/test-smartlog.t" when adding new
tests to test-smartlog.g
Reviewers: #sourcecontrol, stash, mjpieters, rmcelroy
Reviewed By: rmcelroy
Subscribers: net-systems-diffs@, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4430919
Signature: t1:4430919:1484820777:08da5f8fd4144bf05c7596db846d827b3680bc3e
Summary:
This requirement was preventing the pyflakes test from running. In development
builds our mercurial reports its version as
"Mercurial Distributed SCM (version +0-)", which causes this check to fail.
Everyone using this repository should be on a version of mercurial greater than
1.0 now, so it seems like it should be fine to drop this check.
Test Plan: Ran "arc unit" and confirmed the pyflakes check was not skipped.
Reviewers: #sourcecontrol, durham, rmcelroy
Reviewed By: rmcelroy
Subscribers: net-systems-diffs@, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4427727
Signature: t1:4427727:1484733735:5bd6359b351df5ada917ccc7bac0fa2fae065cee
Summary:
All of the test-check*.t tests cd out of the test repository and into the main
normal fb-hgext repository, and attempt to run hg commands there to examine the
repository contents. Unfortunately they were doing so still using the HGRCPATH
settings configured for the test repository. In practice this causes the tests
in many situations, because extensions enabled in the system configuration and
required for the main repository may not have been enabled.
Test Plan:
Ran "arc unit" in a repository with sqldirstate enabled and confirmed the tests
pass now instead of bombing out with screenfuls of error messages.
Reviewers: #sourcecontrol, stash, durham, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, net-systems-diffs@, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4427670
Signature: t1:4427670:1484733701:f6c859c29d34eb33b1280e798ab3398a03a3a461
Summary:
treemanifest requires fastmanifest, and fastmanifest.usetree requires
treemanifest. Let's make these dependencies explicit in the code and error out
if they are incorrect.
Test Plan: Added a test
Reviewers: #mercurial, wez
Reviewed By: wez
Subscribers: wez, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4417512
Signature: t1:4417512:1484347447:7e18340813fac0b298aa51a7cc2f89fc6953680f
Summary:
Previously the find function would return None if the given file was not present
in the tree. The other manifest implementations throw a KeyError here instead.
This affects things like sparse looking for files in the null revision, where it
was receiving a None and crashing because it expected a KeyError in this case.
Test Plan: Added a test. Also, clones were working again.
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4412592
Signature: t1:4412592:1484321389:690e6cd7b3d1599714102ebf361e59fcb0ed59ef
Summary:
Treemanifest had a bug where the pack files it created were 400 instead of 444.
This meant people sharing a cache on the same machine couldn't access them. In
most of the remotefilelog code we had set opener.createmode before creating the
pack but we forgot to for treemanifest.
This patch moves the opener creation and createmode setting into the mutable
pack class so we can't screw this up again.
Test Plan:
Tests that wrote to the packs before, now failed and had to be
updated to chmod the packs before writing to them.
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4411580
Tasks: 15469140
Signature: t1:4411580:1484321293:9aa78254677548a6dc2270c58cee0ec6f57dd089
Summary: Let people put #, t, commas or no, etc!
Test Plan: existing and new tests
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters, ullrich
Differential Revision: https://phabricator.intern.facebook.com/D4411027
Tasks: 15467918
Signature: t1:4411027:1484264937:cdc2452ac3ba920c153d3e0317972433da188a8e
Summary:
Previously treemanifest and fastmanifest were largely unaware of each other. If
the fastmanifest was available, we'd use that. If not, we'd try tree. Then we'd
fall back to flat. When comparing two manifests, this could cause problems since
if one manifest was fast and one was tree, we'd have to fall all the way back to
flat manifests to compare them.
This patch adds the ability to build a treemanifest from a fastmanifest for
inmemory manifests that were copied from a commit that has both a fastmanifest
and a treemanifest. We do it by diff'ing the inmemory fastmanifest with the
commit fastmanifest, then applying that diff to the commit's treemanifest. Then
we can compare that tree with the other tree.
This is particularly useful when doing things like 'hg up master' after a pull.
The new master commit probably has a treemanifest, but doesn't yet have a
fastmanifest, while the commit the user is on probably has both. Now we can do
that hg up without having to parse a full manifest.
Once treemanifest has been enabled, every commit (both pulled and user created)
should have treemanifests. Therefore this case should never happen when
comparing two normal commits. So in theory, the only way to hit this case is
when Mercurial does a manifest.copy() on a manifest that has a fastmanifest
(like when reading a workingctx which must copy the commit manifest and apply
working copy changes), since the copy will choose to copy the fastmanifest and
leave the treemanifest behind. This patch addresses that case, so I think there
shouldn't be other cases.
Test Plan: Adds a test that covers this case.
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4402604
Signature: t1:4402604:1484216866:1c075f314026aaf608095b4aca37c9a4277ababa
Summary:
We have run into some cases where users ended up with empty pack file in their
packs directory. Just log a warning in this case and skip this pack file,
rather than letting the exception propagate up and crashing the command.
Test Plan:
Created empty 0000000000000000000000000000000000000000.histpack and
0000000000000000000000000000000000000000.histidx files in my repository's
hgcache directory, and confirmed that "hg log" now simply warns about them
instead of crashing.
I didn't really test the perftest.py or treemanifest_correctness.py extensions
much. They seem to throw exceptions, and look like they have maybe gotten a
bit stale. I fixed one minor typo, but didn't dig into the other exceptions
too much.
Reviewers: durham
Reviewed By: durham
Subscribers: net-systems-diffs@, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4402516
Tasks: 15428659
Signature: t1:4402516:1484155254:96d2980efcec2d735257b08910e1ca437ef1dad6
Summary: I somehow missed this test when updating all of them with fbamend's new error message.
Test Plan: Test now passes.
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4399110
Signature: t1:4399110:1484094504:2c7a8124849c9fd32fb57162dfb1ba7a90b1efbc
Previously we attached the original node to the copy of the manifest we made.
This is almost always incorrect. In the main case, we make a copy of the p1
manifest during commit and then edit that manifest to form the new manifest. By
storing the node (and never invalidating it), there's the opportunity for the
code to accidentally load the old p1 manifest instead of the in memory one.
The fix is to not pass node to the copy. This affects the tests because some
tests relied on the copied manifest being able to access the original tree/cache
later. It just happened to work because we never modified the manifest before
performing those loads.
This is similar to the patch to matches() that made the match result not have
node either.
This patch also cleans up the fastdelta logic that attempted to read the
_treemanifest directly, instead of going through the normal read path. This is
what actually caught the bug, since it read the treemanifest from disk, while
the in memory cachedmanifest had changes.
Previously, when fastmanifest.usecache was set to false, we just had the cache
return False every time we checked if something was in the cache. Unfortunately,
this didn't prevent us from constructing new inmemory fastmanifestdicts, which
didn't play nicely with treemanifest.
Let's make usecache do the exact same thing as usetree, which means storing
'False' in the __cachedmanifest field, and using that to indicate that field is
completely disabled.
There were two openers in use in this code: opener and origself.opener. 'opener'
was only being created when it was the first manifest in the transaction, so the
line at the bottom that depended on 'opener' existing would fail if a second
manifest was created during the transaction (like during an amend). We just need
to fix the last line to use the correct origself.opener, and fix the 'opener'
name to be less accidentaly reusable.
The _treemanifest function was hard coded to return None if self.node was None,
even if self.__treemanifest was present. A future patch removes self.node in
many cases, so this ended up breaking since it could no longer return the
treemanifest.
Summary:
The remotenames extension automatically adds a --dest option to `hg rebase`, which causes
restack to abort. Instead, we should just drop the 'dest' key and proceed.
Test Plan: Attempt to use `hg rebase --restack` in fbsource when the current bookmark is tracking a remote bookmark.
Reviewers: #mercurial, durham, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4399846
Tasks: 15422257
Signature: t1:4399846:1484084585:4a6ec907032e7d8b1388980870b6d72cb560829f
Summary: Adding progress bar to slow prefetch process.
Test Plan: Perform a prefetch with :-10 or :-100 and see a progress bar displayed.
Reviewers: hanw, mjpieters
Reviewed By: mjpieters
Subscribers: mjpieters, hanw
Differential Revision: https://phabricator.intern.facebook.com/D4398078
Tasks: 9993344
Signature: t1:4398078:1484071212:f906493464340bea103be66133aa7026284143c7
Summary: Upstream Mercurial now has native support for statprof, rendering this file useless.
Test Plan: Remove statprofext.py* from my installed hg, confirm it still works. Build fb-hgext and run tests, confirm no new failures
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4397926
Signature: t1:4397926:1484066120:0c1469d838a91a65fb57ecc858675256087e2c48
Summary: Added some tests for this also
Test Plan: arc unit in fb-hgext
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4397881
Tasks: 15360841
Signature: t1:4397881:1484066348:1b9fb24a30a9962de0dad858f4d57956a2c91dc4
Summary: To encourage people to learn about and use `hg restack` (which is a more general fixup tool), tell users about it instead of `hg amend --fixup`. I wasn't sure whether to advertise it as `hg rebase --restack` or just `hg restack`, so I've included both.
Test Plan: Do an amend in the middle of a stack and observe the new wording. Unit tests reflect changed wording.
Reviewers: durham, #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4390111
Signature: t1:4390111:1483962208:46951c8f5e4c499840b0b4b0ee8ac1e1b6c14ca9
Summary:
On a big repo with many old heads bundle that is sent to the server can become
big (in some cases even 500 Mb). That looks like a waste of bundlestore space
and will probably make backup and restore slower. Most of the space is taken
by manifest deltas because it prefers to diff manifest against previous commit
in the bundle. There are two possible approaches to reduce the size:
1) Send many small bundles (for example, one bundle per head)
2) Wrap deltaparent function and diff against actual parent previous commit in the bundle.
I chose the second approach for the following reasons:
1) It's easier to implement (main reason)
2) Many bundles probably means slower restore because there will be many requests to the bundlestore instead of just one
With this diff bundle size was reduced from 500 Mb to 8 Mb.
It can potentially increase CPU usage. I'm not sure how bad is it and will investigate it more.
Test Plan: Run `test-infinitepush-*`
Reviewers: rmcelroy, mitrandir, durham
Reviewed By: durham
Subscribers: simpkins, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4384053
Tasks: 12479677
Signature: t1:4384053:1483664446:62ec30fad433e8d279758926199a8330cb73ed2b
Summary:
If client pulled a commit without filelogs (because of remotefilelog) and
this commit was later stripped on the server, then attempt to backup this
commit fails because of missing filelogs. It's a rare issue but unfortunately
it happens sometimes. To fix it let's exclude from the backup all the
commits and their descendants that doesn't have all necessary filelogs.
Test Plan: Run infinitepush tests
Reviewers: mitrandir, rmcelroy, durham
Reviewed By: rmcelroy
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4365747
Signature: t1:4365747:1483466164:560cdf5cd2369cd2603dfd0fe6b30d7a70951f00
Summary:
Test was flakey because wait_for_background_backup.py waited for change in
backup state file but then logfile was checked. In this case it was possible
that backup state file was already synced while logfile was not.
Let's not use logfile at all and just check backup file.
Test Plan: Run `test-infinitepush-*`
Reviewers: rmcelroy, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4383660
Tasks: 12479677
Signature: t1:4383660:1483727504:2d738ef5ea8d016a8df71cbc6a99d44f24a77f3e
Summary:
I got the following profile results when profiling smartlog:
{P56936293}
It suggests that regex compiling takes too much time. I tend to think that
profiling is inaccurate in this case because with this diff I didn't get any
noticable speed up. But this diff won't do any harm.
Test Plan: arc unit
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4387265
Signature: t1:4387265:1483724595:43de3c694109be2d4343d8ebdbc7ab79aa9edb04
Summary:
`pushbackup` command became huge. This diff splits it into a few
smaller functions
Test Plan: Run `test-infinitepush-*`
Reviewers: rmcelroy, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4384008
Tasks: 12479677
Signature: t1:4384008:1483659549:a97b63a38b702a55d19cf6b47a6fd6b2547b9168
Summary: __init__.py became huge and I suggest to split it
Test Plan: Run `test-infinitepush-*`
Reviewers: rmcelroy, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4383661
Tasks: 12479677
Signature: t1:4383661:1483659134:eaf3d7e217633895a65ad568831d9c21ea0f18d8
The sshaskpass is not only a mercurial extension, but also a script that
runs separately, in which case it cannot import mercurial modules. So let's
fallback to os.environ.
Summary:
wait_for_background_backup.py does sleep internally and stops as soon as file
is present (with 0.1 sec precision). Additional sleeps shouldn't be necessary
Test Plan:
Run `test-infinitepush-*` on devserver
PYTHONPATH=/opt/facebook/hg/lib/python2.7/site-packages/:/opt/homebrew/lib/python2.7/site-packages/ /opt/homebrew/opt/python27/bin/python2.7 ../../hg-crew/tests/run-tests.py
on mac
Reviewers: rmcelroy, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4381211
Signature: t1:4381211:1483551806:bbefc6632d7aed83f4a1caf8ad2ec5ba2b8f12d7
Summary:
`pushbackup` failed when there are hidden heads in the repo but no visible
heads (see test for example). This diff fixes it.
Test Plan: Run `test-infinitepush-*`
Reviewers: durham, mitrandir, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4365677
Tasks: 12479677
Signature: t1:4365677:1483465909:fd1d96211909d3eeccab4fd17a364af4a1997f85
Summary:
Recently I had this scm prompt:
{F64910275}
This is not user friendly. I suggest to output at most one remote bookmark
Also this diff copies `seq.py` from upstream hg repo
Test Plan: Run scm prompt tests on devserver and mac and with '--shell=zsh'
Reviewers: rmcelroy
Reviewed By: rmcelroy
Subscribers: simonfar, wez, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4197242
Signature: t1:4197242:1483466446:cd1fe943dc99fde0600a4e20994fa4b0f971f72a
Summary:
The '&>>' operator only works in linux, not OSX. So let's replace it with a more
verbose equivalent. 'man bash' says these two are equivalent.
Test Plan:
Running the tests normally on OSX don't actually catch this, because
this particular test is skipped because it doesn't see that evolve is installed.
We need to run the tests with:
PYTHONPATH=/opt/facebook/hg/lib/python2.7/site-packages/:/opt/homebrew/lib/python2.7/site-packages/ /opt/homebrew/opt/python27/bin/python2.7 ../../facebook-hg-rpms/hg-crew/tests/run-tests.py test-infinitepush-backup.t
To get coverage. And the test is now passing on OSX and linux.
Reviewers: stash, #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4379593
Signature: t1:4379593:1483488383:ce8a045bbc29d719eafa372db56453b83f1d8df8
Summary:
Some recent commits introduced non-OSX compatible test output. This patch fixes
that.
- 'ls' has different output text and error codes for missing files on osx
- 'sync' just waits for an in memory write to be flushed to disk, which
shouldn't actually be visible to any user land process, so it seems there's a
deeper race condition here, like wait_for_background_backup.py not working
correctly. Adding a sleep here to work around it.
Test Plan: Ran the tests on linux and osx
Reviewers: stash, #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4378618
Signature: t1:4378618:1483477739:5cd8bf82368cc417fbc7ee22410e6d962ba10342
Summary:
Without `--clean`, the user may not be able to abort cleanly.
Also, change "erase" to "discard" to be more consistent with the official
language. "changed" is replaced by "changes" as it looks like a grammar
mistake.
Test Plan: Updated existing tests
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4377376
Signature: t1:4377376:1483464655:f695fd643509d7fabb88298bd1004727ee64ef76
Summary: bundle file should be closed to ensure that all the temporary files were deleted
Test Plan: arc unit
Reviewers: #sourcecontrol, ikostia
Reviewed By: ikostia
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4377148
Signature: t1:4377148:1483451990:2a1362d9e524a54ffedad8f2bc09fc622efe8ad4
Summary:
test-check-code.t was updated in core mercurial so we have to fix it in
fb-hgext
Test Plan: arc unit
Reviewers: #sourcecontrol, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4377208
Signature: t1:4377208:1483451688:bc382cdab07c6d6127d67c902458a2ccfa1373d1
Summary:
In 676596f945ea2166820ef92e692ef7fe6a6247f0 were added comments with
lines > 80. In aec81a9a80d22989bbdc8c74c1dfec9dcbbe6866 default config value
was changed.
Test Plan: arc unit
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4377188
We need to keep the path in sync with the stack, and since we've already popped
the stack here, we also need to pop the path in this short-circuit case.
Previously we were relying on mfrevlog.revdiff() to produce the delta for us.
This only showed us what files were added/modified, as compared to p1, and we
had to use a heuristic to know what files were deleted (by looking at the list
of files in the commit metadata). Merge commits have a different criteria for
what is in the commit metadata (it only contains entries where the file is
different from both parents), so we can't use it for the same heuristic. So
let's fall back to a normal manifest diff for merge commits, since they are
rare.
Adds a test for verifying that conversion of merge commits into a tree works.
Summary: The --limit option was carried over from the option list for `hg log`. However, it doesn't do anything in smartlog, so this diff removes it from the option list.
Test Plan: `hg smartlog --help` no longer shows `--limit`.
Reviewers: #mercurial
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4363187
Tasks: 15186221
