Commit Graph

911 Commits

Author SHA1 Message Date
Jun Wu
5a5a7662cd tests.t: clean up the shell code a bit
Summary:
Simplifies some shell code, namely `$extpath` is duplicated with `$TESTDIR/..`
so remove one of them.

Change `library.sh` (used by remotefilelog tests) to reuse `$HGRCPATH` instead
of overriding the environment variable, because it never gets restored and the
current code would assume `$HGRCPATH` to have `.hgrc` as its basename, which is
not always true.

Test Plan: `arc unit`

Reviewers: #sourcecontrol, stash

Reviewed By: stash

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4256506

Signature: t1:4256506:1480612191:73832c427b6562b9000a6df006226653157a7156
2016-12-01 02:06:57 +00:00
Jun Wu
786439a1c0 fastannotate: skip downloading linelog if the file history is short
Summary:
If the history of a file is short, it would be easier to just build the
linelog locally, instead of downloading it from the server.

This diff adds a "threshold" config option to make it possible for the client
to skip downloading linelogs.

A special case is, the file only has a single revision, where we can just avoid
all SSH connections - no fileservice peer, and no annotate peer.

Test Plan:
Add a test case for testing filelog repo.
Note: remotefilelog repo is not tested for this feature yet because there is
other planned changes related to remotefilelog.

Reviewers: #sourcecontrol, stash

Reviewed By: stash

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4255483

Signature: t1:4255483:1480611801:4eb79c66eaf1fb5436d550f13c7a78d09b9a1ee2
2016-12-01 00:23:24 +00:00
Jun Wu
bdf8638228 fastannotate: allow the client to build up annotate cache on its own
Summary:
Previously fastannotate client will always fetch linelog from the server,
regardless of how long the history of a file is.

It actually makes sense for fastannotate client to build up the annotate
cache on its own, if, for example, the file only has one single revision,
where we don't even need any kind of SSH connection.

Test Plan: Run existing tests

Reviewers: #sourcecontrol, stash

Reviewed By: stash

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4254602

Signature: t1:4254602:1480611745:a44e70495771cb4f5c508ebb05fcfe8b0bb4d53f
2016-12-01 00:15:51 +00:00
Stanislau Hlebik
7dc24c7c82 infinitepush: backup local heads and bookmarks
Summary:
Backuping changesets is not enougn for restore. We also want to backup heads
and bookmarks to completely capture the state of the repo.

This diff is the first step in implementing this functionality. It adds new
bundle2 part which contains encoded dict (simple json encoding is used).
If value in the dict is empty then key is the bookmark pattern to delete.
If value is not empty then key is the bookmark name to save and value is a
node hash. The reason to put them in the same part is to make it possible to
delete and insert into indexapi in one transaction. It's also possible to pass
patterns to delete in part parameters but there is a bug in upstream hg that
limits parameters' size to 256 and we can potentially have longer bookmarks.

Local bookmarks are saved in infinitepush in the following form:
  infinitepush/backups/USERNAME/HOSTNAME/REPOROOT/bookmarks/LOCAL_BOOKMARK_NAME

Local heads are saved in infintiepush in the following form:
  infinitepush/backups/USERNAME/HOSTNAME/REPOROOTheads/HEAD_HASH

Hostname, username and repo root is necessary to distinguish different backups.

Test Plan: Run `test-infinitepush-*`

Reviewers: durham, rmcelroy, mitrandir, quark

Reviewed By: quark

Subscribers: quark, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4245080

Tasks: 12479677

Signature: t1:4245080:1480518959:aa199d67fac4e2cd2f543651ff56fdd649dac729
2016-12-01 08:44:57 -08:00
Stanislau Hlebik
35b49c7546 infinitepush: set push defaults in debugbackup
Summary:
Not setting all default options may result in KeyValue errors.
For example pushvars extensions does
  repo._shellvars = opts['pushvars']

It results in failure if 'pushvars' is not set.
Let's fix it by explicitly setting default values.

Also we need to set `allow_anon` to remotenames because otherwise
debugbackup will fail.

Test Plan:
Run `test-infinitepush-*`

Run `hg debugbackup` inside fbsource make sure there were no failures

Reviewers: rmcelroy, mitrandir, durham

Reviewed By: durham

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4237230

Tasks: 12479677

Signature: t1:4237230:1480446802:36e0630ce3e60c947be47d83bbc6deff8624048e
2016-11-30 02:11:28 -08:00
Durham Goode
fe6f541ed8 remotefilelog: reuse deltabase when history is not available
Summary:
Previously, if there was no history available for a given revision, we would
just store the full text in the pack file. This patch makes it attempt to reuse
the existing delta base instead. This will be useful for repacking
treemanifests, since we currently don't have histpacks for them (we just delta
them efficiently when they are first added to the repo).

Test Plan: Ran tests

Reviewers: #mercurial

Differential Revision: https://phabricator.intern.facebook.com/D4240705
2016-11-29 15:37:58 -08:00
Durham Goode
a0dc16174d treemanifest: write deltas for trees
Summary:
Previously, when we wrote each tree entry into a pack file, it wasn't delta'd in
any way. This patch makes it store the delta against p1 in the pack file.

Testing in a large repo shows this reduces tree pack size by about 22x.

Test Plan:
Ran the tests. Did a pull in a large repo and saw the pack file was
22x smaller than before (and still usable).

Reviewers: #mercurial

Differential Revision: https://phabricator.intern.facebook.com/D4202088
2016-11-29 15:37:58 -08:00
Durham Goode
b179a3593c treemanifest: sort parent nodes for hash computation
Summary:
Core mercurial sorts p1 and p2 before computing the hash, so it's deterministic.
We need to do the same.

Test Plan: Ran the tests, saw a hash changed

Reviewers: #mercurial

Differential Revision: https://phabricator.intern.facebook.com/D4202063
2016-11-29 15:37:58 -08:00
Jun Wu
1c63f171b5 fastannotate: prefetch cache from server in fctx mode
Summary:
Previously, all prefetching work is done by the "fastannotate" command and
we may replace the "annotate" command with "fastannotate" and we run into
issues dealing with command line option differences.

Now that we have the "fctx" mode, do not redirect "annotate" to
"fastannotate" but wrap "annotate" in a lightweight way so it's responsible
for downloading the annotate cache.

Test Plan: Added a new test case

Reviewers: #sourcecontrol, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4239607

Signature: t1:4239607:1480447754:a84268a9e64c678c1f275bc04c44a6f4138eafe6
2016-11-29 11:51:08 +00:00
Jun Wu
a3dc8b960d fastannotate: support replacing fctx.annotate directly
Summary:
Previously, I have performance concern about the signature of
`fctx.annotate`: it returns fctxs, not ideal for performance since my
initial goal is to get rid of reading revlogs in the best case. And letting
the low-level `fctx.annotate` have some side effects writing to the disk is
not that pretty.

Because of that, fastannotate had re-invent part of the formatter (also
optimized for performance somehow), and still cannot support all the
features the original annotate supports, namely, templates.

Now it makes sense to just replace `fctx.annotate` with a sub-optimal
implementation, just for the flexibility of the template support of the
original annotate command. We can use a "fake" or "lazy" fctx object to
minimal the performance impact when people only need to print changeset
nodes and line numbers - in which case we don't read revlog.

Actually, the hgweb support already did a similar thing - converting
fastannotate output to annotate output. So we can reuse some code.

The next planned steps are:

  - Make the original "annotate" command aware of fastannotate protocol, do
    the pre-download stuff (downloading cache files at `fctx.annotate` is
    possible but inefficient because of not being batched).
  - "fastannotate" command remains a separated command optimized for perf
    and provides extra features like "--deleted".

Because of the plan, the "commands" option does not make much sense -
"fastannotate" command will not replace "annotate" directly thus dropped
from the config interface. A new "modes" config option is added to control
how fastannotate works.

Test Plan: Modified existing tests

Reviewers: #sourcecontrol, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4238998

Signature: t1:4238998:1480447679:c48e0e565663c086293265e104d9cf414d913aa7
2016-11-29 11:51:08 +00:00
Adam Simpkins
d3a77bd1f3 [infinitepush tests] don't hang forever if getdb.sh does not exist
Summary:
Fix the test to abort if getdb.sh does not exist (or otherwise fails), rather
than ignoring the failure and continuing to try and run the remainder of the
test anyway.  Otherwise mysql will hang forever.

The run-tests.py code unfortunately can't even kill the test properly in this
situation, so even with a --timeout argument it still hangs forever.

Test Plan:
Ran the tests without getdb.sh, and confirmed it failed quickly instead of
hanging.

Reviewers: durham, rmcelroy, stash

Reviewed By: stash

Subscribers: net-systems-diffs@, yogeshwer, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4242014

Signature: t1:4242014:1480408476:8e8765ca4567a0436fba0778938f935f87287747
2016-11-29 10:26:27 -08:00
Jun Wu
f08e17d3ed testedwith: change testedwith to "ships-with-fb-hgext"
Summary:
Using `testedwith = 'internal'` is not a good habit [1]. Having it
auto-updated in batch would also introduce a lot of churn. This diff makes
them "ships-with-fb-hgext". If we do want to fill the ideal "testedwith"
information, we could put it in a centric place, like a "fbtestedwith"
extension rewriting those "ships-with-fb-hgext" on the fly.

Maybe having in-repo tags for tested Mercurial releases is also a good idea.

[1]: www.mercurial-scm.org/repo/hg/rev/2af1014c2534

Test Plan: `arc lint`

Reviewers: #sourcecontrol, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4244689

Signature: t1:4244689:1480440027:3dc18d017b48beba1176fbfd120351889259eb4b
2016-11-29 13:24:07 +00:00
Martijn Pieters
69898cdd44 testing: insert missing space
Summary: Attack of the trivial missing space.

Test Plan: Run run-tests.py -l test-sqldirstate-shelve.t to see it pass

Reviewers: rmcelroy, simonfar, quark, jeroenv

Reviewed By: quark, jeroenv

Subscribers: mjpieters, #mercurial

Differential Revision: https://phabricator.intern.facebook.com/D4244663

Signature: t1:4244663:1480439601:3ab7e306e618f7e6e2372046f00e85cf1eb3826d
2016-11-29 09:15:29 -08:00
Jun Wu
c030cc61cb tweakdefaults: do not use template for annotate if unnecessary (BC)
Summary:
Template lacks of the "padding" feature. There is a hard-coded 13-char wide
username, 5-char wide line number, which does not look well if the usernames
are too long or too short.

This patch restores the original annotate format which is different from
the current behavior. For example, given "-uc", it shows "user node" instead
of "node:user". It also tries to make the template version as close as the
non-template one, by adjusting the order, changing separators, and making
"-c" optional (could be hidden from the output).

Test Plan: Modified existing tests

Reviewers: #sourcecontrol, ikostia

Reviewed By: ikostia

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4243830

Signature: t1:4243830:1480437984:1c679c9db516bab905d027061326a7a1524922c6
2016-11-29 11:18:00 +00:00
Martijn Pieters
14dc38a7ba testing: adjust for new home for cmdutil.service
Summary: See rd9d8d78e in the hg repository; cmdutil.service moved to server.runservice

Test Plan: Run test-fbconduit.t and observe it passing once more.

Reviewers: simonfar, rmcelroy, quark

Reviewed By: quark

Subscribers: mjpieters, #mercurial

Differential Revision: https://phabricator.intern.facebook.com/D4244268

Signature: t1:4244268:1480432981:8e06beadba912294e6d1a01d93b13848cb16a096
2016-11-29 07:51:16 -08:00
Martijn Pieters
b302fcecbc testing: fix spacing for merge conflict markers
Summary: Yay for arbitrary single space changes.

Test Plan: Run the two affected tests to show they now pass again.

Reviewers: simonfar, rmcelroy, quark, stash

Reviewed By: stash

Subscribers: mjpieters, #mercurial

Differential Revision: https://phabricator.intern.facebook.com/D4244213

Signature: t1:4244213:1480431919:ba4837619652023005faeb00a93a351c7ca0411a
2016-11-29 07:23:32 -08:00
Jun Wu
c7946d4233 fastannotate: auto rebuild broken cache server-side via wireprotocol
Summary:
D4233433 changed commands to rebuild the broken cache but missed
wireprotocol. Let's do it for wireprotocol as well.

Test Plan: Added a new test

Reviewers: #sourcecontrol, stash

Reviewed By: stash

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4237855

Signature: t1:4237855:1480421375:d5715d61a3c66ec8be5a6a80041ee6204e5e4378
2016-11-29 12:14:30 +00:00
Arun Kulshreshtha
73155a224e allowunstable: add allowunstable extension
Summary: Currently, the reason we can't perform operations such as histedit, split, and fold in the middle of a stack is that we do not have the experimental.evolution=allowunstable option enabled. It turns out all of these commands will do the "right thing" if this option is enabled. However, since we are uncertain of the ramifications of enabling this option globally, this new extension turns the option on for the duration of split, fold, and histedit, allowing those commands to be used in the middle of a stack without config changes.

Test Plan: See unit tests. Basically, create a stack and then attempt to use split, fold, or histedit somewhere in the middle.

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4209315

Tasks: 14598405

Signature: t1:4209315:1479744780:811e8bef02149a055376c0f105ac2d389dd60970
2016-11-28 10:43:34 -08:00
Jun Wu
db7dd53ae9 fastannotate: skip downloading cache if client's master is behind
Summary:
It's common that the client's "master" ("main branch") falls behind the
server's, in which case the "is-up-to-date" check will fail and trigger a
re-download while it's unnecessary. Add some checks to avoid downloading
the cache for the client in this case.

Test Plan: Added a test

Reviewers: #sourcecontrol, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4233534

Signature: t1:4233534:1480330716:db995d0fa5c6a2371894fc2a7871277da23608a9
2016-11-25 22:06:18 +00:00
Jun Wu
b4efcc56cf fastannotate: deal with corrupted files gracefully
Summary:
Previously, fastannotate's annotatecontext would delete individual cache
files (linelog and revmap) if it meets Exception. While the "fastannotate"
command has some retry logic, "debugbuildannotatecache" doesn't, which means
users have to run "debugbuildannotatecache" O(number-of-corrupted-files)
times to unblock. And "fastannotate" does not handle AssertionError which
could be seen as "linelog is corrupted" nicely - it crashes without retry.

Addressing all the above issues by replacing the "assert" to
"CorruptedFileError", catching them and retry properly.

Test Plan: Added a new test.

Reviewers: #sourcecontrol, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4233433

Signature: t1:4233433:1480330548:618e80762af2209784ad72ad0658f649b050a080
2016-11-25 19:49:13 +00:00
Jun Wu
d203784d68 pyflakes: fix all pyflakes issues
Summary: As the title.

Test Plan: `arc unit`

Reviewers: #sourcecontrol, stash, rmcelroy

Reviewed By: stash, rmcelroy

Subscribers: rmcelroy, stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4232321

Signature: t1:4232321:1480067588:54e91ece8fa6b5ff13b3ebc9279217c76bf96a24
2016-11-25 00:23:21 +00:00
Jun Wu
311a3fc653 test-fastannotate-hg: fix compatibility with upstream
Summary:
The recent bdiff behavior change affects us. Document why and update the test
so it works with both the old and new (less optimal imo) bdiff behavior.

Test Plan: Updated the test

Reviewers: #sourcecontrol, simonfar, rmcelroy

Reviewed By: simonfar, rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4231966

Signature: t1:4231966:1480011763:d116799ded3514d644e4b80296ffc71a8d712a43
2016-11-24 18:07:32 +00:00
Stanislau Hlebik
024d254256 infinitepush: increase test timeout and add logging
Summary:
Tests failed on mac. My guess is that timeout is too low.
Let's increase it to 10 seconds (it's fine because we want wait for 10 sec
if debugbackup finishes faster).

Also let's add logging

Test Plan: Run `test-infinitepush-*`

Reviewers: #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4213330

Tasks: 12479677
2016-11-21 08:25:10 -08:00
Stanislau Hlebik
24dc1a5878 infinitepush: fix tests
Summary:
`backup` was renamed to `debugbackup` a few commits ago.

Rename it there too

Test Plan: Run `test-infinitepush-*`

Reviewers: #sourcecontrol, andrasbelo

Reviewed By: andrasbelo

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4212539

Tasks: 12479677

Signature: t1:4212539:1479721307:1ac25c7dc8951d655910e2b0c41b2c9555353f74
2016-11-21 01:44:54 -08:00
Stanislau Hlebik
0a536f70fd infinitepush: add background backup
Summary:
`hg backup --background` will be used as a `txnclose` hook to backup all of the
local commits to infinitepush.

Test Plan: Run `test-infinitepush-*`

Reviewers: rmcelroy, mitrandir, durham

Reviewed By: durham

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4175953

Tasks: 12479677

Signature: t1:4175953:1479145307:e698903b519361b376f6e182db7c49869c992617
2016-11-21 00:53:37 -08:00
Stanislau Hlebik
565b70f23e infinitepush: support bundle with many heads
Summary:
During `hg backup` bundle with many heads may be pushed. Let's support it too.
Many heads bundle is not allowed only when we are pushing it with `--to`
because in this case we don't know the node for the bookmark.

Test Plan: Run `test-infinitepush-*`

Reviewers: rmcelroy, durham

Reviewed By: durham

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4175280

Tasks: 12479677

Signature: t1:4175280:1479231056:fb0c47a7752319d77a6cfc83a29c57e9e2dced16
2016-11-21 00:48:09 -08:00
Durham Goode
06d00be6b9 remotefilelog: make remotefilelog sparse test more robust
Summary:
We were seeing some flakiness with remotefilelog prefetching in the sparse
integration test. Let's make the test more explicit by having it explicitly
prefetch exactly the files it cares about, so we shouldn't see any more
flakiness.

Also have the test refer to the in-repo sparse extension instead of the system
one, now that remotefilelog is in the same repo as sparse.

Test Plan: Ran it

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4192463

Signature: t1:4192463:1479334442:f91a99a4a026a0f71f9519155185fb484d8d68ec
2016-11-16 14:15:35 -08:00
Durham Goode
dcc5006448 check: update check code override
For some reason the .cpp files now show up. They should probably be ignored,
same as the .hg files already are.
2016-11-16 13:51:48 -08:00
Durham Goode
3ad41b8578 treemanifest: limit autocreatetrees to commits on master
Summary:
Creating a tree for a commit whose parent is not already a tree is expensive.
Let's optimize the autocreatetrees option (which converts manifests to trees
during pull) to only create trees if A) the parent is already a tree, or B) the
parent is master. This way we only pay the expensive part once. It also means
that as new branches fork off master, they will be trees too, since all commits
in the new branch will meet criteria (A).

Test Plan:
Ran hg pull in a large repo over a large pull with 40 different
branches and verified it only paused for the initial master commit.

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4129010

Signature: t1:4129010:1478260239:1698ada4e3c6a38ab77a94317e75daee4812276f
2016-11-16 13:51:48 -08:00
Stanislau Hlebik
fd8f0625ef infinitepush: add hg backup command
Summary:
Backups all new non-extinct [1] commits to bundlestore.

When it is called for the first time `hg backup` will backup all draft
visible commits. Next backups will save only new commits since the last
backup (it is recorded in `.hg/store/infinitepushbackuptip`[2]).

It's an initial implementation. Later the following features will be added:
1) It will be called automatically whenever user creates or strips commits
or even creates or deletes bookmarks
2) It will also save all local bookmarks and all local heads
(probably only visible).

Note: calling `pushcmd` directly does not set default values for opts. That means that `--to` will be None and `_scratchbranchmatcher` will throw exception. Let's add a check to ensure that `--to` is never None.

[1] I also want to backup extinct commits (i.e. obsolete invisible commits).
But it will require bigger changes in discovery algorithm, so I'd leave it
for later.
[2] The name is a bit verbose. But I want to keep `infinitepush` part to make
it easier to debug problems.

Test Plan: Run `test-infinitepush-*`

Reviewers: rmcelroy, mitrandir, durham

Reviewed By: durham

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4147651

Tasks: 12479677

Signature: t1:4147651:1479229440:3eb38880c14f18e9a2fb4eaba44bedf079bca506
2016-11-16 02:13:17 -08:00
Arun Kulshreshtha
97ed6a8afa fbamend: add new implementations of hg previous and hg next
Summary:
This diff replaces the `hg previous` and `hg next` commands from the evolve extension with new implementations. The new commands have several features not found in evolve:

  - Users can now move by multiple commits at a time. Example: `hg next 3`
  - Users can move all the way to the top or bottom of the current stack with the following new flags: `hg next --top` and `hg prev --bottom`
  - Users can rebase child commits on demand with `hg next --rebase`. This flag can be combined with the others to specify exactly how many commits to rebase. Example: `hg next --rebase 4` (move 4 commits up the stack, rebasing as needed) `hg next --rebase --top` (rebase all the way to the top of the stack)
  - When a user lands on a particular commit, if there is a bookmark it will be automatically activated. The user can also move to the previous or next commit with a bookmark with the `--bookmark` flag.
  - The user can pass the `--newest` flag to resolve ambiguity in situations where history is nonlinear. With the flag the command will always choose the commit with the higher rev number at each step.
  - The commands now show commit hashes instead of rev numbers when showing the user what commit they landed on.

Apologies in advance for the large diff. I've tried splitting it up as best as I could, but the unit tests introduced here significantly increase the size. I figured the unit tests should be part of this diff instead of being added in a separate one. This diff is almost entirely new code, so I hope that helps a little bit. Let me know if there's a better way to split this up.

Test Plan: See the new unit tests for an extensive look at the new commands in action. Essentially, all of the features mentioned above should work as expected.

Reviewers: quark, #sourcecontrol, durham

Reviewed By: durham

Subscribers: stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4121778

Tasks: 14119420

Signature: t1:4121778:1478774243:a7131593222bc329f541e77e1d3ebd8222e47e79
2016-11-15 18:46:56 -08:00
Jun Wu
127bd207b5 linkrevcache: wrap _adjustlinkrev to take advantage of the linkrev database
Summary:
Wrap `_adjustlinkrev` so it first tries the possibly faster path (linkrev
database). Then fallback to the original linkrev algorithm.

Also fixes an issue that `{k : v ...}` syntax does not work in Python 2.6.

Test Plan: Added a very simple test

Reviewers: #sourcecontrol, durham

Reviewed By: durham

Subscribers: durham, stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4143315

Signature: t1:4143315:1479234110:07e87c7584e0cfd627b3f9e2fae1a2a34beb390f
2016-11-14 18:04:29 +00:00
Wez Furlong
9b5499d572 phabdiff: don't require a space when parsing the Tasks field
Summary: Allow for any number of spaces after the colon

Test Plan: arc unit

Reviewers: #sourcecontrol

Subscribers: markisaa

Differential Revision: https://phabricator.intern.facebook.com/D4179619

Tasks: 14532985
2016-11-15 12:28:21 -08:00
Jun Wu
6860161db4 linkrevcache: add a command to verify the linkrev database
Summary:
Adds a `debugverifylinkrevcache` to verify the linkrevs in the database,
against the vanilla adjustlinkrev algorithm.

Test Plan: A simple test is added. The code is also manually tested on my hg repo.

Reviewers: #sourcecontrol, stash

Reviewed By: stash

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4143240

Signature: t1:4143240:1479217639:bb9b5814b405b9e60a0722b01fc2c918c5a24b35
2016-11-14 18:04:29 +00:00
Jun Wu
dd91b969e6 linkrevcache: implement the database
Summary:
This diff implements some simple linkrev database classes which uses dbm to
store linkrev candidates.

Why not sqlite? sqlite's insertion seems to be much slower with index being
present, so we have to do manual buffering and do insertion in batch, not to
say parsing SQL itself can introduce overhead. If we really want sqlite for
some reason (i.e. its support for concurrent writer and reader seems to be
better), we can still implement it as a dbm engine.

Test Plan: A test will be added when there is something usable.

Reviewers: #sourcecontrol, stash

Reviewed By: stash

Subscribers: jsgf, stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4143119

Signature: t1:4143119:1479131795:5f6e368ebd1798b874cec28ac05b137eb3bea86e
2016-11-14 18:03:15 +00:00
Arun Kulshreshtha
1c6b719130 fbamend: make restack start from bottom of the stack
Summary:
Previously, `hg rebase --restack` would find the closest stable base commit with unstable descendants on its precursors, and begin the restack operation from this commit. What this meant is that it would not resolve all instability in the current stack -- only instability above the first instance of instability detected by searching downward from the current changeset.

Given that we will soon be able to perform local fixup operations with something like `hg next --rebase --top`, it no longer makes sense to keep `hg rebase --restack` as a local operation. Instead, it should resolve all instability in the current stack. This diff changes the behavior to always begin restacking from the bottom of the stack.

One thing to note about this diff is that it completely removes divergence detection from restack. This is because in the process of making this change, I discovered a bug in the original version of restack whereby divergence would only be detected if it involved a precursor of the base changeset. This is because the check was only happening in _latest(), which after several rounds of changes to this code, was only being used in _findstablebase(). In all other cases, the implicit behavior was that unstable children are rebased onto the newest visible successor of their parent, as determined by rev number.

It turns out that altering this behavior to skip rebasing unstable descendants of divergent changesets is nontrivial, and furthermore, one could argue that the current behavior is actually desirable. (Furthermore, this change removes the O(n^2) behavior from _latest().) Regardless of whether it is desirable or not, this diff makes that implicit behavior explicit, and modifies the tests to explicitly check for the new behavior. If we do decide to skip divergent changesets, this will need to happen in another diff as it will require some restructuring of the restack target finding logic.

Test Plan:
See unit tests.

1. Create a stack of commits.
2. Pick several commits in the stack, and amend them without the `--rebase` flag, thereby leaving their respective descendants unrebased.
3. Update to the top of the stack.
4. Run `hg rebase --restack`. All instability will be resolved.

Reviewers: quark, #sourcecontrol, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4144899

Tasks: 14041544

Signature: t1:4144899:1478781779:1a94dbc1c6b69d7116f1d40593a83684ed44942a
2016-11-10 11:25:19 -08:00
Stanislau Hlebik
1b8b5987b2 infinitepush: overwrite bundle for old revs
Summary:
Previously if bundle has contained nodes that are already saved in bundlestore
then we wouldn't update bundleid value for these nodes.
That means that we can never delete a bundle because there will always be a
node that points to it.

I suggest to always update bundleid value for all nodes in the bundle.

Test Plan: Run `test-infinitepush-*`

Reviewers: durham, mitrandir, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4130662

Tasks: 12479677

Signature: t1:4130662:1478565392:682fe39c5832d5edc84c84541f0ba6c1f2e1daf8
2016-11-08 02:27:25 -08:00
Jun Wu
77d215b481 remotefilelog: check file existence in _revertprefetch
Summary: This is to address a crash report: P56804194.

Test Plan: `arc unit`

Reviewers: durham, #sourcecontrol, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4141002

Signature: t1:4141002:1478547571:2a06a48c3687ce91f6a52b59ddc01a17466782a8
2016-11-07 18:30:10 +00:00
Jun Wu
855af710bb fastannotate: support remotefilelog
Summary:
In order to support remotefilelog, do not construct
`mercurial.context.context` object directly. This reverts a previous attempt
to use the fastest manifest look up - `ctx[path]` actually seems good
enough.

A minor change is made in `protocol.py` to ignore failed requests.

Test Plan: Run existing tests

Reviewers: #sourcecontrol, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4127025

Signature: t1:4127025:1478541481:c15477332582e2c9b22e2732e9e76bf37b5fba74
2016-11-03 20:55:16 +00:00
Stanislau Hlebik
e61c86753e infinitepush: add tests for sqlindexapi
Test Plan: Run `test-infinitepush-*`

Reviewers: durham, mitrandir, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4124846

Tasks: 12479677

Signature: t1:4124846:1478537233:afb783ba195fd8f0d6e55d0fb8666220705575cd
2016-11-07 12:35:34 -08:00
Arun Kulshreshtha
cdfe81c7d3 fbamend: change test for restack to use globs for rev numbers
Summary: One of the new features I'm adding to `hg previous` and `hg next` is replacing the rev numbers with commit hashes, since these are more familiar and intuitive to users. As a first step, update this test to glob out the rev numbers.

Test Plan: Test still passes.

Reviewers: durham, quark, #sourcecontrol, stash

Reviewed By: stash

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4121748

Tasks: 14119420

Signature: t1:4121748:1478162085:c7bc64f133dcfd7af67f349a10c0e0a587a8aa19
2016-11-03 11:11:37 -07:00
Jun Wu
5f83e80668 fastannotate: add a debug command to update cache
Summary:
The `debugbuildannotatecache` is to pre-populate the annotate cache
server-side, or pre-fetch the annotate cache client-side.

To avoid import cycle, `cmdtable` is moved from `__init__.py` to
`commands.py`.

Test Plan: Modified existing tests.

Reviewers: #sourcecontrol, stash

Reviewed By: stash

Subscribers: stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4103932

Signature: t1:4103932:1478094621:e82192a12fe4b0b3ac732dbe4c127a2e93fc6b4c
2016-11-01 15:10:21 +00:00
Jun Wu
93c918c92e absorb: fix an issue where changesets do not get deleted correctly
Summary:
The old code didn't test if the new commit will become empty correctly, if its
"memworkingcopy" contains changes inherited from its parent. This diff fixes
the problem.

Thanks @stash for discovering the issue!

Test Plan: Added a test case for this issue. Make sure the old code cannot pass.

Reviewers: #mercurial, stash

Reviewed By: stash

Subscribers: mjpieters, stash

Differential Revision: https://phabricator.intern.facebook.com/D4113438

Signature: t1:4113438:1478077373:194104b902bdfe6827858c2916a7f3f0cd83ee6c
2016-11-01 21:16:13 +00:00
Jun Wu
21bbbebffe fastannotate: implement downloading logic for clients
Summary:
This diff makes the client `fastannotate.client=1`  download
linelogs and revmaps from the server on demand.

Test Plan: Added a new test

Reviewers: #sourcecontrol, stash

Reviewed By: stash

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4103929

Signature: t1:4103929:1477997846:608cf58c090806dafa21ae01feab90a6552100fc
2016-10-29 23:44:34 +01:00
Jun Wu
57890ff99f fastannotate: add a method to quickly get the last node of the revmap
Summary:
Previously, to get changeset hashes in the revmap, we have to construct
the `revmap` object, which reads all content of a revmap.

In the following patches, we want to quickly get the last hash of the
revmap to test if the linelog / revmap is up-to-date or not. This patch
adds a method to do so.

Test Plan: Modified existing tests.

Reviewers: #sourcecontrol, stash

Reviewed By: stash

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4103898

Signature: t1:4103898:1477988583:15869dfe9028068563b03d1f06d2337562049751
2016-10-29 19:19:18 +01:00
Stanislau Hlebik
f4f6b2411a infinitepush: inhibit pulled revs
Summary:
Pulling scratch commits that were stripped before results in
abort: 00changelog.i@c153241c1ea22cbce8a229dccdf7f46de827d962: filtered node!

The reason is that upstream pull does
checkout = str(repo.changelog.rev(checkout))

and it fails if revision is filtered.
I suggest to inhibit filtered revisions that are present in the local repo.
It makes sense because if user tries to pull scratch branch he or she
probably expects to see them in the repo even though these revisions were
stripped before.

Also we can avoid doing any pull if all revisions are present in the repo.
I decided not to do this because it complicates code (for example,
we need to ensure that bookmarks are updated correctly) and it doesn't give
any significant benefits.

Test Plan: arc unit

Reviewers: durham, rmcelroy, mitrandir

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4103869

Tasks: 12479677
2016-11-01 03:42:26 -07:00
Stanislau Hlebik
cf1214923e infinitepush: disable pushes to svn repo
Summary:
During infinitepush test we noticed that `hg push -r . --to SCRATCHBRANCH`
fails with stack trace in www. This is expected because infinitepush doesn't work
with svn servers. But it's better to print a nice error message instead of stack trace.

Test Plan:
Try to push to scratch bookmark from www without specifying path:
stash@dev1918 ~/www (344c6bf) $ hg push -r . --to hack/durhamstash
abort: infinite push does not work with svn repo
(Did you forget to `hg push default`?)
[Exit: 255]

Got nice error instead of stack trace.

Push to scratch bookmark from www with path specified works fine

Reviewers: rmcelroy, mitrandir, durham

Reviewed By: durham

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4103745

Tasks: 12479677

Signature: t1:4103745:1477950448:14b5ae8484f3e65cb88c83b2628f73f3a670b1a0
2016-11-01 01:50:34 -07:00
Stanislau Hlebik
563a57128e infinitepush: add mysql waittimeout server-side
Summary:
Saw a few timeout errors during infinitepush test.
Previous timeout was 10 secs, it makes sense to increase it.

There was a unit test failures because of the config options.
So this diff also adds missing config docs and copyrights headers.

Test Plan: Build the package, install it locally and run integrationtests

Reviewers: durham, rmcelroy, mitrandir, andrasbelo

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4103689

Tasks: 12479677
2016-11-01 01:44:38 -07:00
Durham Goode
201d5f2149 remotefilelog: fix getchangegroup signature
Summary:
The upstream getchangegroup function changed back in August and our attempt at
fixing it here did not correclty handle the case where remotefilelog is loaded
but not enabled for this repository.

This patches fixes the signature, and makes our wrapping more resitant to future
signature changes.

Test Plan: Added a test.

Reviewers: #mercurial, jsgf

Reviewed By: jsgf

Subscribers: jsgf, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4099519

Signature: t1:4099519:1477698801:adcb406f1908c4f7e508e5f99e126f383c6f2574
2016-10-31 13:40:33 -07:00
Durham Goode
922c03630a remotefilelog: make test less flaky
Summary:
The stderr and stdout in this test has a flaky ordering issue when run in
parallel with other tests. Let's just eat the stderr since all we care about is
the stdout message from the remote and the error code.

Test Plan: Ran it

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4106366

Signature: t1:4106366:1477946237:2e188a1abde93a06ec97d99c237851955a23f20a
2016-10-31 13:39:17 -07:00