Summary:
This diff makes the following modifications to the (former) `hiddenhash` extension:
- Makes the error messages configurable, allowing us to easily change the error messages in `facebook.rc`.
- Renames the extension to `hiddenerror`, since the primary feature is now the configurable error messages rather than using a hash in the message.
- Changes the method by which the extension wraps the `changectx` constructor. The old method of wrapping would result in errors in code paths that in `context.py` that performed `isinstance()` checks on contexts.
Test Plan: To test arc patch this extension into your `facebook-hg-rpms` and enable it for a test repo. Then, attempt to access a hidden node. You should see the default error message defined in the extension. Setting `hiddenerror.message` and `hiddenerror.hint` should allow you to customize the messages. See test file for examples.
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4548140
Tasks: 15896355
Signature: t1:4548140:1487063552:06395c1eaaa83b00ead8e80e2cd8f9d13f7d08e2
I would like to make an extension that replaces one of the layers (by replacing
makecachestores) with a layer that reads from an existing machine-local caching
system. This would speed up my accesses a lot and reduce a lot of duplication
on my disk.
Summary: Sometimes you want to be able to justify why a line is in a sparse profile. Support empty lines and comments (starting with a hash).
Test Plan: Run the test runner
Reviewers: #mercurial, rmcelroy, durham
Reviewed By: durham
Differential Revision: https://phabricator.intern.facebook.com/D4537617
Tasks: 15993301
Signature: t1:4537617:1486687441:18a00b65df244b573bab095809fb1fbb57e21d71
Summary: If `hg debuginhibit` is run with no arguments, print out a list of inhibited nodes. This is similar to how `hg debugobsolete` works and will assist with debugging.
Test Plan: Run `hg debuginhibit` in a repo with inhibited nodes. (This can be tested by either stripping some test commits and then updating back to them, or by manually inhibiting the commits with `hg debuginhibit`.) A list of inhibited nodes should be printed. See test file for an example.
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Differential Revision: https://phabricator.intern.facebook.com/D4541514
Tasks: 16004717
Signature: t1:4541514:1486722366:10b4232830ed88931bd6ac610972114743797192
Revert Plan:
Summary:
Let's pull node during update if not found locally.
This is a part of selectivepull functionality.
See remotenames extensions for details about selectivepull.
Test Plan: arc unit
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: indragie, mjpieters, sergeyb
Differential Revision: https://phabricator.intern.facebook.com/D4536129
Tasks: 12479658
Signature: t1:4536129:1486667537:3d1df30cb5d1db0dd7451756102ccafee20789d5
When remotefilelog moved from its own repo, the tests needed to be updated to
adjust the PYTHONPATH to ensure the in-repo remotefilelog was loaded instead of
the system one.
This meant any local runs of remotefilelog tests would've been using the system
remotefilelog unless the user had manually set the PYTHONPATH themselves.
Summary:
Previously we only wrapped manifestlog.__getitem__ because fastmanifest didn't
support the concept of directory manifests. Unfortunately manifestlog.get() is
used in some flat manifest cases too, like in changegroup.py where it just
passes '' as the directory but still uses .get().
This bug has been present since the manifestlog refactor, but only became
exposed now that we've improved our caching and hold on to manifest ctx's for
longer.
This fix is to just wrap manifestlog.get() too. Fixing this exposed some other
inconsistencies with our fastmanifest api.
Test Plan:
shelve now passes in a repo that was hitting an exception because of
this before. I'm still trying to think of how to test this in a test. I've not
been able to repro it in my own repo yet.
Reviewers: #mercurial, quark
Reviewed By: quark
Differential Revision: https://phabricator.intern.facebook.com/D4540677
Tasks: 15991119
Signature: t1:4540677:1486690231:07a95b402913d7dcc31633341e235ee1eb0163bc
Summary: Many operations that did not require access to histpacks were constructing them regardless. By postponing histpack initialization until we require access to it, we save up to 55ms on such operations.
Test Plan:
1. Stat profiler no longer shows ~6.5% time spent in basepack constructor during `hg book`. Also no longer shows up in the `hg show` profile.
2. Timed how much time was spent in basepack constructor, this was up to 55ms (mostly around 35ms).
3. Ran tests, they all pass.
Reviewers: #sourcecontrol, durham, quark, simonfar
Reviewed By: simonfar
Subscribers: simonfar, quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4500999
Signature: t1:4500999:1486502571:cda6cfb7be54eacb341b58fbff209113ea9cf670
Summary:
Sometimes we pass in two revs because we're creating a merge commit.
Let's handle that case better.
Test Plan: tests still pass
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4528594
Signature: t1:4528594:1486576897:6bd80017dfe2933e4c1a9685ba35bb1af0440eb0
Summary:
See comment in code for details. This addresses a performance
problem that arises when the same remotefilelog repo makes two changes
to a file far apart in history with no other repo making changes to the
same file in between.
Test Plan: new test verifies that linkrevs are fixed up using remote blob
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: stash, quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4475183
Tasks: 12958591
Signature: t1:4475183:1486576777:81f4f9815cb677c94954c23ecd8aa7edb006ffd9
Summary:
We will be reusing this logic in the next patch; for now we're simply
factoring it out of the main loop
Test Plan: tests continue to pass
Reviewers: #mercurial, durham, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4528290
Tasks: 12958591
Signature: t1:4528290:1486567826:cb17e2fa1ab71f491482eba1a88c107a18333781
Summary:
To get rid of this DeprecationWarning:
--- /data/hgbuild/facebook-hg-rpms/fb-hgext/tests/test-hiddenhash.t
+++ /data/hgbuild/facebook-hg-rpms/fb-hgext/tests/test-hiddenhash.t.err
@@ -10,5 +10,7 @@
$ hg debugbuilddag +1
$ hg debugobsolete 1ea73414a91b0920940797d8fc6a11e447f8ea1e
$ hg log -r 0
+ $TESTTMP/hiddenhash.py:33: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
+ match = re.match(r"hidden revision '(\d+)'", e.message)
abort: hidden changeset 1ea73414a91b!
[255]
let's not access `.message` and use str(e) instead
Test Plan:
arc unit,
Will test on centos6 soon
Reviewers: #sourcecontrol, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4521398
Signature: t1:4521398:1486470861:e39e4639346d0f1f2e4a69fc38ccf34fbc432df6
Summary:
HG_SAMPLING_FILEPATH was renamed to SCM_SAMPLING_FILEPATH long time ago.
Let's update the comment
Test Plan: Not tested
Reviewers: #sourcecontrol, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4507708
Signature: t1:4507708:1486141731:8a96ac74006cf9d240ad4cbbff928fe49fe71cb3
Summary:
Previously, when a command tried to access a hidden revision without the `--hidden` flag, the user would get the following error:
```
abort: hidden revision '0'!
(use --hidden to access hidden revisions)
```
With this extension, the error is now:
```
abort: hidden commit b8144197c244!
```
This way, novice users aren't tempted to immediately re-run the command with `--hidden`, which may have undesired results. It also makes it much easier to find the commit in question in `hg sl --hidden` since by default smartlog does not show rev numbers.
Test Plan: Attempt to access a hidden commit, observe new error message as seen above. Note that in the test file the word "changeset" is present instead of "commit" due to the lack of the `dialect` extension in the test.
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4506065
Tasks: 15747879
Signature: t1:4506065:1486122372:d82ebd488a00d1958c235f733a9455111954c116
Summary:
turned out that transaction can be None in some cases.
Let's just write to the cache in this case.
Test Plan: arc unit
Reviewers: #sourcecontrol, durham, andrasbelo
Reviewed By: andrasbelo
Subscribers: andrasbelo, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4507653
Tasks: 15862906
Signature: t1:4507653:1486133933:de0997ef1724ba1082c3597ea64aab760f670b47
Summary:
Recently @rmcelroy discovered that attempting to rebase a single commit in the middle of a stack results in the rest of the stack being marked as unstable. See https://fb.facebook.com/groups/sourcecontrol/permalink/1207767375939547/ for full example.
It turns out that the inhibit extension's post-transaction hook that would ordinarily inhibit the rebased (and thus obsolete) commit, and thereby hide the instability, was not firing in this case. This was similar to a previous problem with `hg next --rebase` wherein `--continue`'ing a rebase with conflicts would result in visible instability for the same reason. In both cases, the problem appears to be that `rebase` doesn't actually create a transaction, so of course the post-transaction hook doesn't fire.
The solution for `hg next --rebase` was to add a (nearly) empty transaction after the rebase call to trigger the hook. This diff just makes this the default behavior for `hg rebase` so that the hook is always called.
Test Plan:
See new test file. Basically, create a situation like:
```
o e5d56d debugbuilddag
| r3
|
| o c175ba debugbuilddag
| | r2
| |
| o 220949 debugbuilddag
|/ r1
|
o 1ad88b debugbuilddag
r0
```
Perform `hg rebase -r 1 -d 3`, which rebases commit "r1" onto "r3", which would ordinarily make "r2" unstable. Instead, the result should be as below, with no visible instability because "r1" is inhibited.
```
o 738e8e debugbuilddag
| r1
|
o e5d56d debugbuilddag
| r3
|
| o c175ba debugbuilddag
| | r2
| |
| x 220949 (Rebased as 738e8e) debugbuilddag
|/ r1
|
o 1ad88b debugbuilddag
r0
```
Reviewers: rmcelroy, #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters, rmcelroy
Differential Revision: https://phabricator.intern.facebook.com/D4491870
Tasks: 15697758
Signature: t1:4491870:1486033128:45b8e9a72af1e88ab78484ddef47a788d1aca7b5
Summary: Sort the output to fix tests on different platforms
Test Plan: arc unit on mac
Reviewers: #sourcecontrol, simonfar, ikostia
Reviewed By: simonfar, ikostia
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4501449
Signature: t1:4501449:1486050577:3041dc3c3691f48e39a0e9e05b1f97954a6d88e2
Summary:
If we have too many unsorted entries then finding matching revisions is
slower. To fix it we will rebuild partial index from time to time.
_findcandidates function marks index as need-to-be-rebuilt if it notices
index file with too many unsorted entries. Then _changegrouphook rebuilds
the index.
Test Plan: arc unit
Reviewers: #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4494617
Summary:
Since we have some of the nodes sorted we can now use bisect to quickly find
matching nodes.
Test Plan: arc unit
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4494419
Summary:
Append header to each fast partial match index file. For now it contains
version number and number of sorted entries in the file.
File is ignored if version number from file is different.
Test Plan: arc unit
Reviewers: #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4488417
Summary:
We'll add bisect algorithm soon.
Let's sort the nodes first during index rebuilding
Test Plan: arc unit
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4488263
Summary:
strip may change revision numbers of many nodes at once.
It's better to rebuild index in case of strip.
Test Plan: arc unit
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4474710
Summary:
This hook builds index if it's not present and it appends new entries
from the changegroup.
Test Plan: arc unit
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4474704
Summary: These two functions actually use fastpartial index to resolve revision for a node.
Test Plan: arc unit
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4474658
Summary:
Add commit hook to append new entries to the index.
It uses the same transactions as revlogs.
Do not append entries if index is not built (i.e.
if _partialindex does not exist). This is to ensure that we either
have complete index or none at all. Besides next diff will add
hooks to build index after pull if it doesn't exist.
Test Plan: arc unit
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4474654
Summary:
Index to make node partial match fast i. e. speed up matching of partial hashes like f9bbd5.
This is an initial diff and it doesn't add any useful functionality, just debug commands.
Storage format is simple. There are a few files (at most 256).
Each file contains entries:
<20-byte node hash><4 byte encoded rev>
Each entry represents a commit (node).
Name of the file is the first two letters of the hex node hash. Nodes with the
same first two letters go to the same file. Nodes are NOT sorted inside the file
to make appends of new nodes easier.
Partial index should always be correct i.e. it should contain only nodes that
are present in the repo (regardless of whether they are visible or not) and
rev numbers for nodes should be correct too.
Test Plan: arc unit
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4474642
Summary: Add a new extension to assist with debugging issues with the inhibit extension. Adds the `hg debuginhibit` and `hg debugdeinhibit` commands to manually inhibit/deinhibit commits as well as config options to print out the nodes being inhibited and truncated stack traces of each call site. I mostly made this into an extension so that it would be easy to debug inhibit issues from any user by just asking them to run their command with the appropriate `--config` flag.
Test Plan: See test file. Enabling the `debuginhibit.printnodes` and `debuginhibit.printstack` functions should result in the relevant nodes and stack traces to be printed whenever nodes are inhibited and deinhibited.
Reviewers: durham, rmcelroy, #mercurial, mitrandir, simonfar
Reviewed By: simonfar
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4485659
Signature: t1:4485659:1485985964:e6920b67450942c706c9523e9df48b4093b26d69
Summary: Although the split and fold commands usually abort on error, in some cases they simply write to stderr and return a nonzero exit status. Previously, the wrappers in fbamend would expect failures to result in an abort, and therefore didn't return early. This resulted in restack errors when this was not the case -- notably in the case of folding a commit with itself. This diff fixes the problem by returning early on error.
Test Plan: Attempting to fold a commit with itself (i.e., `hg fold --exact . .`) fails with an error message instead of crashing.
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4493029
Tasks: 15704338
Signature: t1:4493029:1485986404:17186b47786cf79c735b33adbb19de79ac75a5be
Summary: Previously, if a user ran split or fold on commits with bookmarks on them, those bookmarks would remain on the original commit instead of the new split or folded commits. This behavior was unexpected and confusing because it meant the obsolete commit would remain visible and the user would be forced to manually move the bookmark. This change makes split and fold automatically move any bookmarks on the old commits onto the newly created commits.
Test Plan: Run `hg split` or `hg fold` on a commit or set of commits with one or more bookmarks, and observe that the bookmarks are updated to point to the newly created commits. See tests for examples.
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4490655
Tasks: 15595449
Signature: t1:4490655:1485986009:858f099d3f95c6a45fe8b119d4f8d376b87652b7
Summary: Fastmanifest was not caching manifest contexts. Because of this, we were recreating a manifest dict on every read operation. Merges read the same manifest a lot and benefit heavily from this optimisation.
Test Plan:
```
$ ../../hg/tests/run-tests.py -j28
...................................................................................s..................s.......................................................
Skipped test-infinitepush-sql.t: missing getdb.sh
Skipped test-infinitepush-backup-sql.t: missing getdb.sh
# Ran 156 tests, 2 skipped, 0 warned, 0 failed.
```
Reviewers: #sourcecontrol, ttung, durham
Reviewed By: durham
Subscribers: ttung, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4457102
Signature: t1:4457102:1485449327:16e23c0bb1e7d61fb18c8a0c88a15598603749a8
Summary:
In certain situations, like infinitepush, the noderev cache builders would try
to save a node that didn't exist in the repo. This caused an exception. Now we
check each node before we attempt to resolve it to a rev.
Test Plan: Added a test. It failed before, and passes now.
Reviewers: #mercurial, andrasbelo
Reviewed By: andrasbelo
Subscribers: andrasbelo, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4494848
Signature: t1:4494848:1485964835:986d4fe3e29404cac22e4acf7d88c3715fcb51b9
This is an import of a bugfix in upstream evolve by Martin.
As reported in the issue, "hg uncommit" would lose copy information of
the files that remained in the commit. The problem was simply that the
a dict of copies was iterated as "src, dst in copies.items()" where it
should have been "dst, src ...".
Summary: Minor stylistic cleanup for this test.
Test Plan: Test still passes.
Reviewers: #sourcecontrol, mitrandir
Reviewed By: mitrandir
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4486391
Signature: t1:4486391:1485876954:0d800a4ba3858b23560d7ae9d3ba5a9981e9eb6a
Summary:
Previously the noderev cache only covered the branchmap. These days remotenames
and phases are also doing a lot of node->rev look ups, so let's add them to the
noderev cache as well.
Test Plan:
Compared hg book before and after the change. Noticed a significant
perf increase.
Reviewers: #mercurial, ikostia
Reviewed By: ikostia
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4487962
Signature: t1:4487962:1485871615:634d95f61e916d81440a0f440010912af5ae6a13
Summary:
Previously, fastannotate requires `.hg/fastannotate` to be writable. However,
this cannot be guaranteed in cases like running hg in other's repo.
This patch adds some exception handling to make fastannotate running in "fctx"
mode just work even if `.hg/fastannotate` is not writable.
Note: the `fastannotate` command ("fastannotate" mode) is not changed as if
that command is triggered explicitly, the user should be aware of it works
differently from the vanilla annotate command.
Test Plan: Added a test
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4481427
Tasks: 15489272
Signature: t1:4481427:1485775004:ccc0b99b38146034c5b235445d762e55dbb78dbe
Summary:
The remotefilelog extension previously unconditionally added a sparsematch()
method to repository objects, which just called super.sparsematch() if this is
a sparse repository, and returned None if this isn't a sparse repository.
However, defining a sparsematch() method that returns None confuses the sparse
extension. The sparse extension expects that all repositories that define
sparesmatch() are actually sparse repositories, and never expects this method
to return null.
This updates the remotefilelog code to call its method maybesparsematch()
instead.
Test Plan:
Confirmed existing remotefilelog unit tests pass, and that the sparse extension
no longer crashes when it is used with non-sparse repositories. (This happens
when using the share extension, if the current working directory is non-sparse
but the sharedpath repository configuration loads the sparse extension.)
Reviewers: tja, durham
Reviewed By: durham
Subscribers: net-systems-diffs@, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4466108
Signature: t1:4466108:1485451483:287a519151e35bdb99f5be0d9287b4698386183e
Summary: This is basically the same as `hg amend -i` but since allowunstable operates at the command function level we need to explicitly whitelist this command. Note that I've set the obsmarker metadata to "amend" so that commits affected will be marked as "Amended as XXX" in smartlog. I don't think any other `hg record` flags other than --amend create obsmarkers so this should be OK.
Test Plan: Run `hg record --amend` in the middle of a stack and observe that it works now.
Reviewers: durham, #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters, reedriley
Differential Revision: https://phabricator.intern.facebook.com/D4466338
Tasks: 15705764
Signature: t1:4466338:1485424744:442604ce2089298f0724c27025502a3602113401
Summary:
On Windows, the default SSH client shipped with TortoiseHg is TortoisePlink.
It is very slow and for some reason, we did not experiment deadlocks using this
client. (My unverified hypothesis is that it may have an internal buffer to
manage large requests).
Using another SSH client such as MSYS ssh.exe (shipped with git Windows
installer) dramatically speeds up tranfers (~8 times). However, using this client
yields the deadlock problem.
NOTE: I don't know why it's not a problem on linux boxes.
The fix is to issue requests from a separate thread, so we don't enter a
deadlock on the client.
Summary: There is no valid reason to run `hg restack --hidden` (and this will cause unexpected behavior) so disable it.
Test Plan: `hg restack --hidden` now aborts. Added check to unit test.
Reviewers: rmcelroy, #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4463934
Tasks: 15615216
Signature: t1:4463934:1485386049:9836ec970d7c3e11c7dfaed21aa0ce8b58013294
Summary:
During pull, the reposetup function was updating the static factory ui with the
ui from the remote peer. This didn't have any of the local configs, so it
appeared as if treemanifest was disabled if it had been enabled in the repo
specific hgrc. So let's just not update the ui for remote repos.
This was causing pulls to always generate a full tree manifest instead of only
doing incremental ones.
Test Plan:
Ran 'hg pull' and verified the tree packs created in the hgcache was
small.
Reviewers: #mercurial, mitrandir
Reviewed By: mitrandir
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4456928
Signature: t1:4456928:1485359738:3333e1ad2c5cd68890b3eb5ff80a01ed5584df1e
Summary:
this is neccessary for the new metaedit to not spend time in
remotefilelog
Test Plan: tests are passing
Reviewers: #sourcecontrol, rmcelroy, durham
Reviewed By: durham
Subscribers: durham, rmcelroy, quark, jeroenv, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4326589
Tasks: 14548013
Signature: t1:4326589:1485343332:df8f7f6169e148b4724b088878ca8dfff34e9275
Summary:
Default lock timeout value is very small (around couple of secs).
Let's set it for something bigger by default
Test Plan: arc unit
Reviewers: rmcelroy, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4435236
Tasks: 15389402
Signature: t1:4435236:1484880747:d46276a82782ef551cda0165dea686f267c13ff9
