Commit Graph

2390 Commits

Author SHA1 Message Date
Jun Wu
5269322567 fastannotate: convert revision number to hex
Summary:
Previously, the revision number was converted to binary, and parsed using the
revset language. It worked fine in some cases but could be broken when it
matches other bits of the revset language occasionally.

This patch converts the revision number to hex to address the issue.

Test Plan: `arc unit`

Reviewers: #sourcecontrol, ikostia

Reviewed By: ikostia

Subscribers: ikostia, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4327054

Signature: t1:4327054:1481904759:52170ba0ec7baf284b489f0d5a164153a7e8f25f
2016-12-14 16:46:28 +00:00
Jun Wu
9a67f6b7f1 fastannotate: do not cache lastnode
Summary:
When the server was handling a "getannotate" request, it may read lastnode,
update the revmap, then read lastnode again. So lastnode should not be cached.

Test Plan: `arc unit`

Reviewers: #sourcecontrol, ikostia

Reviewed By: ikostia

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4327046

Signature: t1:4327046:1481904335:2a236091b5d36872df5703e4bb9f3651012a2f02
2016-12-14 16:45:55 +00:00
Jun Wu
9fe5387619 fastannotate: add a forcetext option
Summary:
The default annotate command will do a "isbinary" check which reads fctx.data()
and contributes to a remotefilelog fetch round-trip. Since the "fastannotate"
command does not do binary check, and we want to minimize the round-trips,
disable the binary check by default.

Test Plan: `arc unit`

Reviewers: #sourcecontrol, ikostia

Reviewed By: ikostia

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4327028

Signature: t1:4327028:1481904237:72a59dfb622459cd961b3da615053f80b55f1a7a
2016-12-14 16:44:45 +00:00
Stanislau Hlebik
feecb32f3a infinitepush: pullbackup command
Summary:
As the name suggest it will restore backup made by `hg pushbackup`.

If user has only one backup for the `dest` repo then it will be restored.
But user may have backed up many local repos that points to `dest` repo.
These local repos may reside on different hosts or in different
repo roots. It makes restore ambiguous; `--reporoot` and `--hostname`
options are used to disambiguate.

Example situation:
1) User has only one laptop with mercurial repo `repo` that was cloned
from remote server. He or she run `hg pushbackup`. Then laptop breaks
and user gets a new one, clones the `repo` again and runs `hg restore`.
It automatically restores the backup.
2) User has devserver and laptop and backups were made from both.
Then if user decides to switch devserver and run `hg restore` on the new
devserver he or she has to specify `--hostname`.


Future plans:
1) Add `--user` option to make it possible to restore another user's backup

Test Plan: Run `test-infinitepush-*`

Reviewers: rmcelroy, mitrandir, durham

Reviewed By: durham

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4280832

Tasks: 12479677

Signature: t1:4280832:1481565335:2a21ceafa2ff80242076a79693046514434afb40
2016-12-16 08:29:08 -08:00
Durham Goode
73c92dfa70 pushrebase: update test for upstream change
Upstream changed when http headers are advertised (75deec4e372a). So we need to
update our tests.
2016-12-15 14:16:00 -08:00
Stanislau Hlebik
db0892583c infinitepush: send cg v2
Test Plan: Run `test-infinitepush-*`

Reviewers: durham, rmcelroy, mitrandir, mjpieters

Reviewed By: mjpieters

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4299163

Tasks: 12479677

Signature: t1:4299163:1481212693:21120d98330b9a4fa0aced0523cc389eaa54551b
2016-12-15 10:02:59 -08:00
Stanislau Hlebik
06231c630a infinitepush: add cleanup
Summary:
`bundlerepo` should be closed to ensure that temp file is deleted.
`bundlerepo` creates temp file when bundle is compressed. This is *not* the
case in infinitepush (all our bundle are uncompressed). But seems that
upstream cg1unpacker.compressed() returns True even if bundle is uncompressed.

Also this diff makes `_getoutputbundleraw()` read output bundle in memory
instead of returning a generator. The reason for doing it is because generator
becomes invalid as soon as `bundlerepo` is closed.
This approach will obviously increase memory usage but it shouldn't be a
problem since bundles are small (no more than 10s Mb).

#thanks @quark for reporting it

Test Plan: Run `test-infinitepush-*`

Reviewers: durham, rmcelroy, mitrandir, quark

Reviewed By: quark

Subscribers: quark, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4327186

Tasks: 12479677

Signature: t1:4327186:1481738559:2c6b22c305da4d572da9de21dfdf1179f7281744
2016-12-14 12:26:30 -08:00
Mateusz Kwapich
b453f14bb2 fastmanifest: add the new parents method to fastmanifest
Summary: This was added in core as well

Test Plan: tests are passing

Reviewers: #sourcecontrol, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4326595

Signature: t1:4326595:1481732429:af2ab66f73e5ef501b4d0480181715c17797b743
2016-12-14 17:41:15 +00:00
Jun Wu
2e622df71d fastannotate: deal with absolute paths in perfhack
Summary:
Previously, when perfhack is enabled, `_matchpaths` will use slow code for
absolute paths. This diff makes it use the fast code for absolute paths.

Test Plan: `arc diff`

Reviewers: #sourcecontrol, mjpieters

Reviewed By: mjpieters

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4318343

Signature: t1:4318343:1481724095:9d21a5135486c9d503bee3d8f93a23fa15fa647a
2016-12-13 02:40:27 +00:00
Jun Wu
76fcdc8d6d fastannotate: remove mtime check for isuptodate
Summary:
The mtime check seems to cause more trouble than benefit - often the
annotate cache was treated "not up-to-date", while it is up-to-date
actually.

Test Plan: `arc unit`

Reviewers: #sourcecontrol, mjpieters

Reviewed By: mjpieters

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4318077

Signature: t1:4318077:1481723495:d4e0ab5e7ef5a520520c07fb7246230639b9b9b4
2016-12-13 01:52:53 +00:00
Stanislau Hlebik
b1506f0277 infinitepush: fix tests
Summary:
stderr/stdout order is different on mac. We don't need stderr at all on this
test so let's just disable it

Test Plan: Run `test-infinitepush-*`

Reviewers: #sourcecontrol, simonfar, jeroenv

Reviewed By: simonfar, jeroenv

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4326319

Tasks: 12479677

Signature: t1:4326319:1481711140:443ea2ab92391fa6e4671280441b8f71769650d8
2016-12-14 02:36:58 -08:00
zphricz
3e1da40cda Change setup.py to build modules for windows
Summary: The situation here is that our FB C extensions just won't compile on MSVC at the moment. This simply strips them out of consideration for building on Windows until they can be compiled on Windows later.

Test Plan: Built fb.hg, it didn't include these c extensions

Reviewers: #idi, durham, #sourcecontrol, rmcelroy, davidsp

Reviewed By: rmcelroy, davidsp

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4270189

Signature: t1:4270189:1481612143:9c035b4c6eb4af6b542795a64b24dc274454843a
2016-12-13 17:20:13 -08:00
Jun Wu
e4e12d3f95 test-infinishpush: some cleanups on setupdb
Summary:
The "parsing getdb.sh's output and cut -d" approach is not the best practise of
passing multiple values between shell scripts. Use "source" and setting shell
variables instead.

Besides, skip the test (exit 80) if getdb.sh does not exist

Test Plan:
Provide a good getdb.sh and test on my devserver.
Delete getdb.sh and make sure the test is skipped.

Reviewers: #sourcecontrol, stash

Reviewed By: stash

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4262570

Signature: t1:4262570:1480668031:cd5371c620889856ff892d671d1ee498438b8487
2016-12-13 13:43:58 -08:00
Stanislau Hlebik
3cbecabdb8 infinitepush: add logging
Summary: Let's add logging to infinitepush.

Test Plan: Run `test-infinitepush-*`

Reviewers: rmcelroy, mitrandir, durham

Reviewed By: durham

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4306193

Tasks: 12479677

Signature: t1:4306193:1481312610:1c3f6af7fedbaca51ef81291a471047c7e5931ab
2016-12-13 10:44:40 -08:00
Stanislau Hlebik
c014561bb5 infinitepush: accept different cg versions server-side
Summary:
infinitepush has been storing bundle1 with changegroups v1. Turned out that in
some cases generated bundles can become huge because v1 changegroups don't
support generaldelta. It makes sense to support all changegroup versions (v1 for BC,
v2 for new infinitepush bundles, and maybe later we'll switch to v3).
To do this we have to store bundle2 instead of bundle1 because bundle1
supports only changegroups v1.

Test Plan: Run `test-infinitepush-*`

Reviewers: rmcelroy, mitrandir, quark, durham

Reviewed By: durham

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4299151

Tasks: 12479677

Signature: t1:4299151:1481564714:810be69447d0b35aa57328c60aab72ad374e994d
2016-12-13 06:32:31 -08:00
Durham Goode
bab8d2e0a5 treemanifest: remove allfiles optimization
Commit 24515b72d5 attempted to optimize writes by checking if the file in the
delta was also in the list of files in the commit metadata. This doesn't work for merge
commits since the only files the commit metadata are the ones that differ from
both (so set(changes in metadata) != set(changes in node-diff-against-p1)). This
caused the verify code to catch the issue. The fix is to just remove the
allfiles optimization.
2016-12-12 18:59:45 -08:00
Stanislau Hlebik
e44e5d8bc8 infinitepush: move writing to store outside of indexapi transaction
Summary:
Writing to bundlestore can take a lot of time. It doesn't make sense to do it
inside indexapi transaction

Test Plan: Run `test-infinitepush-*`

Reviewers: rmcelroy, mitrandir, durham

Reviewed By: durham

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4305903

Tasks: 12479677

Signature: t1:4305903:1481312165:6aca8c908367d9715f8ba3f5643cd56650b4456f
2016-12-10 13:37:48 -08:00
Adam Simpkins
859693151a [treemanifest] improve test glob patterns
Summary:
Fix the tests so they pass when running from an install directory.  The files
may be inside python/site-packages/ rather than an fb-hgext/ directory.

Test Plan:
Successfully ran the tests from inside an RPM install directory as part of an
RPM build.

Reviewers: ikostia, stash, durham, mjpieters

Reviewed By: mjpieters

Subscribers: net-systems-diffs@, yogeshwer, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4298095

Signature: t1:4298095:1481190643:79e9d612cbc8021c292d131bcdad1a6749e63f80
2016-12-08 11:47:37 -08:00
Stanislau Hlebik
01528f5840 infinitepush: rename debugbackup to pushbackup
Summary:
@simon suggested to rename `debugbackup` because `debug-` prefix
implies that command is internal only and it's not safe for user to run it.
This is not the case for the backup because user can safely run it.

Test Plan: Run `test-infinitepush-*`

Reviewers: simon, rmcelroy, mitrandir, durham, mjpieters

Reviewed By: mjpieters

Subscribers: simon, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4292516

Tasks: 12479677

Signature: t1:4292516:1481213170:a03c3eaebd3459887bbb3ff1c80f20977139cc90
2016-12-08 09:24:19 -08:00
Stanislau Hlebik
09d6fda2a0 infinitepush: avoid using push during debugbackup
Summary:
`hg push` takes a lock because it updates phases and may receive other bundle2
parts from remote server. We don't need it in debugbackup. And since
debugbackup is intended to run in background we want to avoid taking locks.
Instead let's use raw `unbundle()` wireproto method and disable pushback
bundle2 parts.

Test Plan: Run `test-infinitepush-*`

Reviewers: durham, rmcelroy, mitrandir, quark

Reviewed By: quark

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4291740

Tasks: 12479677

Signature: t1:4291740:1481118369:1b20f771c5db6e4fed4fc18681cc66ef4e91dd3a
2016-12-07 06:05:29 -08:00
Arun Kulshreshtha
bfa9bd7a91 fbamend: remove allowunstable hack
Summary: Move the enabling of unstable changesets from fbamend to allowunstable, keeping all of the wrapping in one place.

Test Plan: Unit test still passes. No change in functionality.

Reviewers: #sourcecontrol, quark, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4288233

Signature: t1:4288233:1481073858:a7432a9941e93fcadf5ad6502630a49c4e1207f7
2016-12-06 17:56:19 -08:00
Arun Kulshreshtha
e7a2ea79bc allowunstable: allow rebasing in the middle of a stack without --keep
Summary: This change enables "allowunstable" for the `hg rebase` command, which lets it create unstable changesets without full evolution enabled.

Test Plan: Create a stack of commits and attempt to rebase a commit in the middle of the stack, using a command like `hg rebase -r REV1 -d REV2` without the `--keep` flag. This will now work, and the old version of the rebased commit (with descendants) will render as an X in smartlog.

Reviewers: #sourcecontrol, quark, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4287989

Signature: t1:4287989:1481074215:80103f6bfd150e51cc152bc257146d3f9369b175
2016-12-06 17:56:17 -08:00
Jun Wu
5a61921433 fastannotate: resolve revision number to node
Summary:
fastannotate has a fastpath, when the node (commit hash) exists in revmap,
it would generate the output instantly from the linelog without going
through the history.

In the "fctx" mode, the vanilla annotate command would pass the revision
number instead of changeset node to fastannotate, which prevents
fastannotate from using the fast path.

This diff converts the revision number to node before testing the fast path
to solve the problem.

Test Plan: Modified existing tests.

Reviewers: #sourcecontrol, durham

Reviewed By: durham

Subscribers: durham, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4288927

Signature: t1:4288927:1481073363:3330d60878161139ee195bc968c161d6c0d95ace
2016-12-07 00:09:04 +00:00
Jun Wu
d3857c8f92 fastannotate: add a config option to disable up-to-date check server-side
Summary:
The `isuptodate` check could still be expensive because of reading the manifest
takes time. Let's add a config option to skip the up-to-date checking to avoid
expensive manifest reading server-side.

Test Plan: Added a new test

Reviewers: #sourcecontrol, durham

Reviewed By: durham

Subscribers: durham, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4287289

Signature: t1:4287289:1481071706:5a3100f1513cd01ae7f78c1dfc9da9f5a3ba79cd
2016-12-06 21:19:59 +00:00
Stanislau Hlebik
83c0fd8ccd infinitepush: support pull from many scratch bookmarks
Summary:
We could've avoided creating this diff if bundlerepo supported many bundles.
Unfortunately it doesn't. Adding support requires changes in upstream mercurial and it can take a lot of time.
This implementation wraps `listkeys` to set phases properly and `exchange._changegrouppart` to send more
than one bundle at a time.
It has unsolved problem with phases. For example in this situation


  # Publishing server creates commit A (it is marked as draft on the server).
  # Client pulls commit A from publishing server (in this case commit A is marked on the client but it's still draft on the server).
  # Another client make pull from scratch bookmark. Commit A will be draft on client because we delete publishing = True from listkeys.

We assume that this case is quite rare.

Test Plan: Run `test-infinitepush-*`

Reviewers: mitrandir, rmcelroy, quark, durham

Reviewed By: durham

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4074889

Tasks: 12479677

Signature: t1:4074889:1480446669:97e7f1d8ad23d457d7984a4cde0efb2b6e89eb2e
2016-12-06 07:41:04 -08:00
Jun Wu
d633064ec0 remotefilelog: correct ResponseError usage
Summary:
This fixed a user report (P56867550) where the error message does not get
shown correctly, because ResponseError takes two parameters.

Also did a minor change for the first ResponseError to comply the format.

Test Plan: `arc diff`

Reviewers: durham, #sourcecontrol, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4280775

Signature: t1:4280775:1481016196:4762161be699e5d215ec86b2d1385493d977a2b6
2016-12-05 23:02:22 +00:00
Alexander Mols
8391d808d0 myparent: commit template keywords based on your previous commit
Summary:
An extension to pre-fill your commit template based on your own
previous commit. Useful for stacked diffs where the commits are mostly related
and will have mostly the same reviewers, tasks, etc.

Test Plan: dogfooding + tests

Reviewers: #sourcecontrol, mjpieters, quark

Reviewed By: quark

Subscribers: quark, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4265616

Tasks: 12218809

Signature: t1:4265616:1480946835:ed18fadbd9068f2d2ec4799fa1ac176132b41e4b
2016-12-05 06:27:29 -08:00
Martijn Pieters
9ce3cb6435 fastmanifest: glob out paths
Summary: Looks like these paths were meant to be globbed out anyway.

Test Plan: run-tests -l test-treemanifest-noflat.t

Reviewers: rmcelroy, durham, ikostia, stash

Reviewed By: ikostia, stash

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4276576

Signature: t1:4276576:1480944202:c88d8aa8bd4ceda0b02f0babcc719b9d9ae3eb05
2016-12-05 05:24:20 -08:00
Kostia Balytskyi
14c482a18a tests_: make test-sqldirstate-shelve.t check for old shelves in a new way
Summary:
With the default shelve extension being unified as .patch,
cleanupoldbackups now also checks for times on .patch files instead
of .hg ones. Thus, we need to adjust tests to touch the right files.

Test Plan: run tests locally, see that they pass.

Reviewers: #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4276322
2016-12-05 03:41:20 -08:00
Stanislau Hlebik
f0a4bf7c7b infinitepush: do not create transcations if there are no bookmarks to update
Summary:
There are lots of breakages if upstream tests are run with infinitepush
enabled. This patch fixes one issue in test-pull-update.t test.

Test Plan:
Run `test-infinitepush-*`
Run test-pull-update.t with --extra-config-opt=extensions.infinitepush=PATH_TO_INFINITEPUSH.
Make sure that it works fine up to line 115 (later lines may still be broken)

Reviewers: durham, rmcelroy, mitrandir

Reviewed By: mitrandir

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4250678

Tasks: 12479677

Signature: t1:4250678:1480689565:792f3d77e08c969a9ffed904815fe7e9aa72f61d
2016-12-05 03:31:06 -08:00
Stanislau Hlebik
bfe26123d5 infinitepush: abort if no default path
Summary:
There are lots of breakages if upstream tests are run with infinitepush.
It fixes test-default-push.t upstream test.

Test Plan:
Run `test-infinitepush-*`
Run `test-default-push.t` upstream test

Reviewers: durham, rmcelroy, mitrandir

Reviewed By: mitrandir

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4250684

Tasks: 12479677

Signature: t1:4250684:1480689556:7d6bc4788cfebb5aa62c631a104e9d735eca3e47
2016-12-05 03:28:23 -08:00
Stanislau Hlebik
8121ee7328 infinitepush: backup bookmarks even if no commits were created
Summary:
We want to backup bookmarks even if no new commits were added. To do this let's
save the hash of all the backuped bookmarks and backup only if saved hash is
different from the hash of the current bookmarks.

In the case when no new commits need to be backuped revs=['null'] is passed
(remotenames does the same in `push --delete`).

Test Plan: Run `test-infinitepush-*`

Reviewers: durham, rmcelroy, mitrandir, quark

Reviewed By: quark

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4251589

Tasks: 12479677

Signature: t1:4251589:1480611524:6faaa9d6d1b1aba9f7a45a4bf9a4c0df3d039ebe
2016-12-05 03:25:34 -08:00
Durham Goode
5ceef3bdc0 treemanifest: add test to verify flat manifests are not being read
Summary:
This adds a few tests that ensure that flat manifests are not being read when
treemanifests are enabled. Any exception to this rule is documented in the test
and commented on.

The test caught a bug where we were not returning a simple treemanifest when the
node was the nullid. While the performance of an empty flat manifest is similar
to the performance of a empty treemanifest, the empty flat manifest may have
forced us to not use the treemanifest in operations that compare two manifests
(like diff).

Test Plan: Ran the test.

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4262975

Signature: t1:4262975:1480706157:4d4913bf032ebf44105a71a27fec5ee1a36b015b
2016-12-02 14:38:02 -08:00
Durham Goode
67f6d86cd7 treemanifest: add test for incremental tree repack
Summary:
This adds a test for hg repack --incremental handling tree packs. It fixes a few
bugs that were caught in the process:

1. Since remotefilelog was being imported via it's file path, it was being
loaded into the process as hgext_remotefilelog. When treemanifest loaded it into
the process via 'import remotefilelog' it was being imported as 'remotefilelog'.
This meant we had the same types imported into the same process with two
different names, which meant 'isinstance()' checks could fail when they
shouldn't (which affects incremental repacks). So we just drop the filepath.

2. We need to allow repacking local tree manifest data even if the full delta
chain isn't available (since part of the delta chain may be in the cache). So we
add allowincomplete to the data pack in this case.

Test Plan: Ran it

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4262412

Signature: t1:4262412:1480706110:45bb0a1e1b031f9cfd4658a5071bbac5df2f6543
2016-12-02 14:38:00 -08:00
Durham Goode
21f0c7615a treemanifest: add test for repacking local packs
Summary:
Adds a test for making sure hg repack handles local pack files in
.hg/store/packs

Test Plan: Ran it

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4261788

Signature: t1:4261788:1480705839:efb9657654bf6d31a3ec1d1125277174cbc93276
2016-12-02 14:37:57 -08:00
Durham Goode
51ae957ffc treemanifest: add simple test for tree repack
Summary:
This adds a simple test that verifies hg repack will pack two tree manifest
packs into one.

It caught a bug where creating a treemanifest for a commit with a null parent
produced incorrect output because it constructed an empty tree and tried to use
it's node as the parent of the delta, when there should not have been any delta
in the first place. This is fixed by this diff as well.

Test Plan: Ran the new test

Reviewers: #mercurial, dsyang, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4261591

Signature: t1:4261591:1480705822:ef21fb8cebd8b89f92f58f11bb1dab59bf97664d
2016-12-02 14:37:55 -08:00
Durham Goode
9514bca8ce packs: make debug*pack commands take more paths
Summary:
Previously 'hg debug[data|hist]pack' required passing the filepath without the
suffix (just up to the hash). This was kind of a pain when using tab complete,
and a pain in tests when trying to use 'find' to run hg debugdatapack on a
number of files. Let's make the debug commands more forgiving.

Test Plan: Manual verification

Reviewers: #mercurial, dsyang, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4261586

Signature: t1:4261586:1480705548:1aa5dbe44ed5e29c28a29a8256de12ffee8d7387
2016-12-02 14:37:53 -08:00
Durham Goode
fe149d0caa treemanifest: set repo.name
Summary:
Since treemanifest uses the same storage locations as remotefilelog, it needs
access to the repo name as well. If a given repo has treemanifest enabled but is
not a remotefilelog repo, it won't have the repo.name member already. So let's
add it ourselves in treemanifest.

Eventually we should probably refactor this out to be a more global concept of
repo name.

Test Plan: A future patch adds a test that caught this

Reviewers: #mercurial, dsyang, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4261581

Signature: t1:4261581:1480705457:8fb4b86ce8abeed62cc7c8f787868359c3cf8abc
2016-12-02 14:37:51 -08:00
Durham Goode
27e07eeed4 remotefilelog: make repack work for non-remotefilelog repos
Summary:
treemanifest also has the concept of repack and shares the same code as
remotefilelog's repack. We want treemanifest to be usable even without
remotefilelog, so let's update the repack code to not require the presence of
remotefilelog configured members on the repo object.

In the long term we'll probably move the repack and pack code out of
remotefilelog entirely.

Test Plan: A future patch adds a test that caught this

Reviewers: #mercurial, dsyang, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4261571

Signature: t1:4261571:1480705343:2ceabbbbdeaf7408ef8b94ad3e670b9423162399
2016-12-02 14:37:49 -08:00
Durham Goode
edd8b25c30 repack: add incremental tree repacking
Summary:
Previously we only supported full repacks of the manifest stores. This patch
adds incremental repacking for both the shared manifest store and the local
manifest store.

Test Plan:
Did a few pulls to produce a few treemanifest pack files. Ran hg
repack --incremental after each one and verified it was a no-op until there were
enough pack files to trigger the repack.

Will add tests later today.

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4260602

Signature: t1:4260602:1480705304:b17d3ec5ff8d4caafe935b2c4e941454052fe3ec
2016-12-02 14:37:47 -08:00
Durham Goode
046fb2ca08 repack: enable repacking of local manifest stores
Summary:
Previous hg repack would only repack the shared manifest cache store. This patch
makes it also repack the local manifest store too.

Test Plan:
Made a local commit in my test repo with treemanifests enabled.
Rebased the commit to master so now there were two local tree packs. Ran hg
repack and verified they were combined into one pack in
.hg/store/packs/manifests

I'll add a test later today

Reviewers: #mercurial, mjpieters

Reviewed By: mjpieters

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4260580

Signature: t1:4260580:1480696151:8f989e299dda50281ca63489e870202eb195d714
2016-12-02 14:37:45 -08:00
Durham Goode
26e1154b71 repack: make repack more generic
Summary:
Previously the repack logic was hardcoded to only work on the shared cache
stores. This patch refactors that knowledge to a higher level so a future patch
can also repack the local stores as well.

Test Plan: Ran the tests

Reviewers: #mercurial, mjpieters

Reviewed By: mjpieters

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4260569

Signature: t1:4260569:1480694802:30ab24dd031661227b4da1febc3d3daf9ad598fb
2016-12-02 14:37:42 -08:00
Durham Goode
569e3f4d38 treemanifest: support fastdelta fast path
Summary:
When making a commit to the flat manifest revlog, we need to produce a full text
delta. Previously we would have to convert our treemanifest into a text string
so we could produce the delta, which takes many seconds. With this patch, we now
take advantage of the fastdelta code path which can generate a delta given the
parent fulltext and the list of changes.

Test Plan:
Ran rebase with treemanifests and verified it was much faster because
it didn't try to produce a full text from the in memory treemanifest.

Reviewers: #mercurial, ikostia

Reviewed By: ikostia

Subscribers: ikostia, rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4249375

Signature: t1:4249375:1480509343:3a130e8730522e31bd7a08440121a1451046a32c
2016-12-02 14:37:40 -08:00
Durham Goode
52a399a6a9 fastmanifest: refactor add()'s fastdelta usage
Summary:
A future patch will make treemanifests also use the fastdelta fast path. To
support that, let's refactor the fast path to separate the fastmanifest specific
part.

Test Plan: Ran the tests

Reviewers: #mercurial, ikostia

Reviewed By: ikostia

Subscribers: rmcelroy, ikostia, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4249369

Signature: t1:4249369:1480509029:b6d08a4f9ef6581a4a2c1bed7f7db8ff084fc35d
2016-12-02 14:37:37 -08:00
Durham Goode
d4cfb31a2b fastmanifest: move fastdelta to a top level function
Summary:
A future patch will make treemanifests able to use fastdelta as well, so let's
move the function to be global so we can reuse it.

Test Plan: Ran the tests

Reviewers: #mercurial, ikostia

Reviewed By: ikostia

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4249364

Signature: t1:4249364:1480507778:a00bb7b6301ca2f716c542f37f7150f2f6231640
2016-12-02 14:37:35 -08:00
Durham Goode
3de4f9ec15 treemanifest: always use flat manifest to get full text
Summary:
Treemanifests are naturally slower to produce full texts than just reading the
text directly from the manifest. So let's change our hybrid manifest
implementation to prefer flat manifests over tree manifests for text(). Fast
manifests are still faster to produce full texts, so we leave those as the
highest preference. This speeds up commit time when using treemanifest since
we don't need to construct the text from the tree for producing the delta that
goes into the revlog.

Test Plan:
Ran rebase with tree manifests enabled and verified it was
significantly faster. Also manually timed the fast vs flat text logic to ensure
fast was actually faster for text() than flat.

Reviewers: #mercurial, ikostia

Reviewed By: ikostia

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4249354

Signature: t1:4249354:1480505430:f9e712a2db49c56680ce34cd3f68c85841393b39
2016-12-02 14:37:33 -08:00
Durham Goode
7a71d2e604 ctreemanifest: implement fast path for matches
Summary:
The original python code for manifest matches has a fast path for when the
matcher contains a specific list of files, this let it check those specific
files instead of iterating over the entire manifest. Let's add this same
optimization to our ctreemanifest implementation.

This greatly speeds up copies._computeforwardmissing() during rebases.

Test Plan: Ran rebase and verified it was much faster

Reviewers: #mercurial, ikostia

Reviewed By: ikostia

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4248075

Signature: t1:4248075:1480505248:964cbdd701fe29c38a7a0d9bf0752fb5d4bb0ed0
2016-12-02 14:37:31 -08:00
Durham Goode
6492dbbf03 hybridmanifest: don't reuse node in match results
Summary:
Previously, _converttohybridmanifest would always create a new hybrid manifest
with the same node as the original. This meant that some code paths would
attempt to use the treemanifest from the node, instead of the already prepared
matches result. This meant the output could contain all the values from the
original tree, instead of just the matches output.

This is actually a regression from 98ba34a5194c09. Prior to that, matches did
not reuse the node.

Test Plan: Manually inspected the results in the debugger during a rebase.

Reviewers: #mercurial, ikostia

Reviewed By: ikostia

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4247821

Signature: t1:4247821:1480499268:27f4a1b92ecf5d10009996b5b8f22bac02f3f38e
2016-12-02 14:37:28 -08:00
Jun Wu
1a05e790ca linkrevcache: update documentation
Summary:
This patch changes docstring to make it clearer about how to use the
extensions. It provides some possibly useful details.

Besides, update `testedwith` to be consistent with other extensions.

Test Plan: My vim spellchecker seems happy.

Reviewers: #sourcecontrol, stash

Reviewed By: stash

Subscribers: stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4263123

Signature: t1:4263123:1480668147:d042e9b52d0a2759ac360c87a2c87d2d3b0aa0f9
2016-12-02 00:59:48 +00:00
Pascal Hartig
a7cffaf493 hgext: githelp support for commit -C
Summary:
Add support for `git commit -C <commit>`/`git commit --reuse-message <commit>`.

E.g.

```
hg githelp -- git commit -C deadbeef
hg commit -M deadbeef
```

Test Plan: Added test case for the new feature

Reviewers: rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4258167

Signature: t1:4258167:1480674937:4f21fd43272462c8c5eee40a02d0db7014c834d4
2016-12-02 10:41:54 +00:00