Commit Graph

721 Commits

Author SHA1 Message Date
Fano Yong
050a2705e7 pushrebase.py modification to skip cache when _data is not found
Summary: pushrebase.py modification to skip cache when _data is not found

Test Plan: hg-dev rt pushrebase*

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: mitrandir, medson, mjpieters, #mercurial

Differential Revision: https://phabricator.intern.facebook.com/D5367059

Tasks: 19603920

Tags: hg

Signature: t1:5367059:1500457213:d9870e1af74a4e27c1be2b08f440dc5e3f905947

Blame Revision: n/a
2017-07-19 09:09:04 -07:00
Ryan McElroy
f95c4e8f49 drop: fix check-code failures
Summary: These issues broke test-check-code-hg.t

Test Plan: test-check-code-hg.t now passes

Reviewers: #fbhgext, stash, simonfar

Reviewed By: #fbhgext, stash, simonfar

Differential Revision: https://phab.mercurial-scm.org/D145
2017-07-19 07:08:21 -07:00
Ola Kopczynska-Dobosz
379f8596da Adding drop command
Summary: Adding a drop extension which drops specified commit from the stack.

Test Plan: Unit tests which are part of this diff and manual testing.

Reviewers: durham, gshanbhag, #mercurial, kulshrax

Reviewed By: kulshrax

Subscribers: mitrandir, quark, #mercurial, phillco, avig, medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5346014

Tasks: 19602078

Signature: t1:5346014:1500438430:b38acce97940fdeb7103ae70844cce71fdd79698
2017-07-18 21:57:03 -07:00
Arun Kulshreshtha
79baca1fcf fbamend: add --clean flag to hg prev/next
Summary: This patch adds a `--clean` flag to `hg prev` and `next` that discards any pending changes, similar to `hg update --clean`.

Test Plan: See included test changes.

Reviewers: #fbhgext

Differential Revision: https://phab.mercurial-scm.org/D140
2017-07-18 20:13:25 -07:00
Jun Wu
0e6e370774 hiddenoverride: avoid race condition updating the state file
Previously there is a race condition:

    origpinned = loadpinnednodes(repo)
    newpinned = ....
    with repo.lock(): # the lock might be taken by process X
        # get the lock after some time. at this time, the state file might
        # be updated by process X but we are not aware of it.
        savepinnednodes(...) # process X's change gets discarded

This patch solves that by making the write function takes a delta instead of
full content, and apply that delta inside a lock. Using repo lock is
expensive so we use a single-file lightweight flock instead.

Differential Revision: https://phab.mercurial-scm.org/D139
2017-07-18 16:07:33 -07:00
Jun Wu
2211d677b5 pushrebase: move working copy parent
When pushing the current changeset like `hg push -r . --to x`, try to update
the working copy parent.

Differential Revision: https://phab.mercurial-scm.org/D123
2017-07-18 12:16:04 -07:00
Jun Wu
c1a9fbc79e undo: move undolog to vfs and mark as lock-safe
The upstream has added some warning around repo.svfs/vfs write operations
without taking locks. For undo, we don't want to take the repo lock (since
that means read-only operations like `hg log` will be blocked by another
`hg commit`). So undo has its own locking and own transaction (for lower
overhead). Therefore move all files we may write (including lock and
transaction files) to `.hg/undolog` and whitelist the directory at
`localrepository._wlockfreeprefix` to bypass the checks.

Differential Revision: https://phab.mercurial-scm.org/D112
2017-07-18 11:30:05 -07:00
Ryan McElroy
1ba39b35d7 copytrace: remove unused error module import
Summary: This broke the check-code linter.

Test Plan: run linters and unit tests

Reviewers: #fbhgext, stash

Reviewed By: #fbhgext, stash

Differential Revision: https://phab.mercurial-scm.org/D137
2017-07-18 10:11:06 -07:00
Ryan McElroy
7a5f7434d2 config: do not pass defaults to registered config items
Summary: This prevents some recently-introduced devel-warnings from breaking the tests.

Test Plan: fewer devel-warns while running tests

Reviewers: #fbhgext, mitrandir

Reviewed By: #fbhgext, mitrandir

Differential Revision: https://phab.mercurial-scm.org/D131
2017-07-18 10:09:28 -07:00
Ryan McElroy
55611588d1 bookmarks: use new applychanges api in all extensions
Summary: This eliminates the recently-added devel-warnings.

Test Plan: fewer `devel-warn`s during tests

Reviewers: #fbhgext, mitrandir

Reviewed By: #fbhgext, mitrandir

Differential Revision: https://phab.mercurial-scm.org/D130
2017-07-18 10:09:28 -07:00
Ryan McElroy
b4c0bebf74 perftweaks: start using cachevfs
Summary:
Without this, we get develwarnings about vfs being deprecated for accessing
files under .hg/cache/.

Test Plan: run-tests, more tests pass

Reviewers: #fbhgext, mitrandir

Reviewed By: #fbhgext, mitrandir

Differential Revision: https://phab.mercurial-scm.org/D129
2017-07-18 10:09:28 -07:00
Mark Thomas
9d389170da copytrace: add fast copytracing for amends 2017-07-18 08:52:20 +01:00
Felix Merk
4fdf562a93 undo: improved performance and prep for hg undo
Implements lighttransaction(repo) which improves performance of hg undo and
prevents infinite loops caused by hooks that run true hg commands.  Also adds
_getrevlog and _invertindex commands, factoring out code that will be reused.
Changes output for gaps when typing hg debugundohistory -l for slightly
better ui and clearer testing. Lastly, changes lock from repo lock to new
undolog lock and tests this.  In the future, with some changes to undo.log,
we may be able to take the lock later, bypassing the need to take it for
read-only operations.

Differential Revision: https://phab.mercurial-scm.org/D50
2017-07-17 13:18:54 -07:00
Jun Wu
285b5a92e6 codemod: rewrite nested with
This is the result of running a codemod script:

  python ~/hg/contrib/codemod/codemod_nestedwith.py **/*.py

Plus a manual cleanup that removes the comment in `absorb/__init.py`.

See D76 for the codemod script.

Differential Revision: https://phab.mercurial-scm.org/D78
2017-07-17 12:02:08 -07:00
Stanislau Hlebik
196a16380c fastpartialmatch: fix devel.warn
Summary:
Upstream hg recently introduced new developer warning if files inside .hg are
modified without taking a lock. The usual fix is to add file in
localrepository._wlockfreeprefix list, but it doesn't work for "needrebuild"
fastpartialmatch file because it's stored inside .hg/store.
So to get rid of developer warning let's move needrebuild file to .hg/.
And since we are detecting that partial index needs to be rebuilt in revlog class,
I also had to wrap `runcommand()` and create needrebuild file there, because
`runcommand()` has vfs object, while revlog class doesn't.

Test Plan: Run tests

Reviewers: #fbhgext, durham

Reviewed By: #fbhgext, durham

Subscribers: durham

Differential Revision: https://phab.mercurial-scm.org/D99
2017-07-17 04:03:15 -07:00
Durham Goode
655c9a4f08 obsshelve: update savebackup signature
Summary:
Upstream has changed the signature of saveback to include the full name. Let's
update to match it.

Test Plan: The test now passes

Reviewers: #fbhgext, stash

Reviewed By: #fbhgext, stash

Differential Revision: https://phab.mercurial-scm.org/D105
2017-07-17 04:02:43 -07:00
Durham Goode
d651fab9a8 tweakdefaults: take wlock when changing bookmarks
Summary:
Upstream has added devel-warnings when writing to vfs files without taking the
wlock. We need to take the wlock here.

Test Plan: The test now passes

Reviewers: #fbhgext, simonfar

Reviewed By: #fbhgext, simonfar

Subscribers: simonfar

Differential Revision: https://phab.mercurial-scm.org/D103
2017-07-17 04:02:43 -07:00
Durham Goode
0afce8a31b cleanobsstore: fix devel-warning
Summary:
Upstream now warns when writing a vfs file outside of a lock. For this
particular file, it's safe to write it outside the lock, so let's add it to the
whitelist.

Test Plan: Test now passes

Reviewers: #fbhgext, simonfar

Reviewed By: #fbhgext, simonfar

Differential Revision: https://phab.mercurial-scm.org/D102
2017-07-17 04:02:43 -07:00
Durham Goode
3c1c4327b8 mergedriver: move state writing into wlock
Summary:
Upstream has added devel-warnings for writing files without the wlock. This was
one instance.

Test Plan: Test now passes

Reviewers: #fbhgext, simonfar

Reviewed By: #fbhgext, simonfar

Differential Revision: https://phab.mercurial-scm.org/D100
2017-07-17 04:02:43 -07:00
Jun Wu
cd181c7ec8 fastannotate: mark as wlock safe
fastannotate has its own file-level flock implementation and does not need
to use the repo lock.

Differential Revision: https://phab.mercurial-scm.org/D91
2017-07-14 16:11:55 -07:00
Durham Goode
fd07b800c7 morestatus: add wlock around updateargs file
Summary:
Upstream has added a warning around writing files with locks. Since this file is
written as part of an update, let's just take the wlock.

Test Plan: Ran the tests

Reviewers: #fbhgext, quark

Differential Revision: https://phab.mercurial-scm.org/D90
2017-07-14 15:55:26 -07:00
Durham Goode
9c003295c7 gitlookup: move writes into wlock
Summary:
Upstream has introduced a new warning when writing to a repo outside of the
lock. gitlookup triggers this during getgitmeta. Let's take the wlock.

Test Plan: Ran the tests. They now pass

Reviewers: #fbhgext, quark

Differential Revision: https://phab.mercurial-scm.org/D88
2017-07-14 15:55:26 -07:00
Jun Wu
4e032f7260 hiddenoverride: improve writing logic
Avoid writing if nothing changes, and write inside a lock.

Differential Revision: https://phab.mercurial-scm.org/D86
2017-07-14 15:00:18 -07:00
Sen Li
b974a7cf2c adding hg amend --to option so it amends to a specific commit
Summary: adding hg amend --to option so it can amends to a specific commit

Test Plan: will add a test for this

Reviewers: juehui, durham

Reviewed By: durham

Subscribers: medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5396805

Signature: t1:5396805:1499986779:d555d75c636e37adbe484b5b77c1844fcacfdc86
2017-07-14 12:06:22 -07:00
Durham Goode
e3a87e44e3 pushrebase: add pack files to storage during hooks
Summary:
Hooks should run with awareness of the tree pack data, so we need to pass the
temporary pack location as part of the hook environment.

Test Plan: Added a test

Reviewers: #fbhgext, simpkins

Reviewed By: #fbhgext, simpkins

Subscribers: simpkins

Differential Revision: https://phab.mercurial-scm.org/D8
2017-07-14 11:47:12 -07:00
Durham Goode
e1b7318640 pushrebase: add config for switching to treemanifest reading
Summary:
Adds functionality and a test for when clients pushes a tree pack to the server
without any flat manifest data, and the server writes the correct flat and tree
manifests during pushrebase.

Test Plan: Added tests

Reviewers: #fbhgext, quark

Reviewed By: #fbhgext, quark

Subscribers: phillco, quark, dsp

Differential Revision: https://phab.mercurial-scm.org/D5
2017-07-14 11:47:12 -07:00
Durham Goode
937ee01b7d sparse: fix interaction with treemanifest
Summary:
Treemanifest relies on the matcher visitdir function, which was not implemented
in our special sparse matchers. Long term we should migrate to using the core
sparse extension. For now let's just implement visitdir().

Test Plan: Added a test. Previously the test would throw a big exception.

Reviewers: #fbhgext, quark

Reviewed By: #fbhgext, quark

Differential Revision: https://phab.mercurial-scm.org/D84
2017-07-14 11:13:39 -07:00
Durham Goode
3102e456ab infinitepush: backout lfs fix
Summary: This broke hg bundle on non-generaldelta repositories.

Test Plan: Ran the tests

Reviewers: #fbhgext, stash, quark

Reviewed By: #fbhgext, quark

Differential Revision: https://phab.mercurial-scm.org/D74
2017-07-13 13:53:40 -07:00
Jun Wu
0b38d6d44e hiddenoverride: unhide permanently by bookmark and working parent
We want these behaviors:

  - Checking out an obsoleted revision makes it visible even if another
    revision was checked out later
  - Putting a bookmark on an obsoleted revision makes it visible even if the
    bookmark was removed later
  - Strip/prune could make them hidden again

This is basically a lightweight version of independent hidden.

Differential Revision: https://phab.mercurial-scm.org/D70
2017-07-13 11:04:56 -07:00
Yan Yan
8e95eaafb0 Let non-existing client fail the import early
Summary: Check if the client exists, and if not, exit the import. Added a test for non-regression.

Test Plan: running test-p4fastimport-import-badclient.t

Reviewers: davidsp

Reviewed By: davidsp

Subscribers: medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5405340

Signature: t1:5405340:1499965824:394077f0d148be2e5fef15902c69703eaf360bbb
2017-07-13 10:18:39 -07:00
stash@fb.com
35d988093a infinitepush: handle lfs correctly
Summary:
Make sure that infinitepush work fine with lfs. To achieve it we try to use
changegroup3 if it's available and also run prepush hook before infinitepush.

Test Plan: Run `arc unit`

Reviewers: #fbhgext, quark, dsp

Reviewed By: #fbhgext, dsp

Differential Revision: https://phab.mercurial-scm.org/D17
2017-07-12 09:52:03 -07:00
Jun Wu
73aa574018 smartlog: remove temporary hack for old revset compatibility
The old smartlog revset is no longer used.

Test Plan:
Confirmed no user uses `smartlog('all')` or `smartlog('recent')` by searching
`smartlog(` globally.

Differential Revision: https://phab.mercurial-scm.org/D48
2017-07-12 02:25:01 +00:00
Durham Goode
5fe640c293 pushrebase: add comments and basic cleanup
Summary:
As part of refactoring the main pushrebase function, let's add some more
comments and move some declarations closer to where they are used (or delete
them entirely if the initial declaration is not used).

Test Plan: Ran the tests

Reviewers: #mercurial, mitrandir

Reviewed By: mitrandir

Subscribers: mitrandir, medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5340604

Signature: t1:5340604:1498751551:714faa78b8a3a01e165d7f5335484b89b7205995
2017-07-10 15:53:12 -07:00
Durham Goode
f9d954a907 pushrebase: refactor post rebase hooks to a function
Summary:
The pushrebase main function is too huge. Let's refactor it into smaller
functions so we can add treemanifest support safely later.

Test Plan: Ran the tests

Reviewers: #mercurial, mitrandir

Reviewed By: mitrandir

Subscribers: mitrandir, medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5340597

Signature: t1:5340597:1498751423:8b7ddce604ff7c9bfeea897afecaa7e7cfa8beb4
2017-07-10 15:53:12 -07:00
Durham Goode
c9fba7ef9e pushrebase: refactor rebase into a function
Summary:
The pushrebase main function is too huge. Let's refactor it into smaller
functions so we can add treemanifest support safely later.

Test Plan: Ran the tests

Reviewers: #mercurial, mitrandir

Reviewed By: mitrandir

Subscribers: mitrandir, medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5340592

Signature: t1:5340592:1498751349:cf7172ac9c603db0d18a189dc02394f2f0a547f3
2017-07-10 15:53:12 -07:00
Durham Goode
4fc972bfe9 pushrebase: refactor push messaging to a function
Summary:
The pushrebase main function is too huge. Let's refactor it into smaller
functions so we can add treemanifest support safely later.

Test Plan: Ran the tests

Reviewers: #mercurial, mitrandir

Reviewed By: mitrandir

Subscribers: mitrandir, medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5340586

Signature: t1:5340586:1498751296:f931fb170e0092222a125dc34e2f6ffac74dcfb7
2017-07-10 15:53:12 -07:00
Durham Goode
433f6f1cd4 pushrebase: refactor onto resolution to a function
Summary:
The pushrebase main function is too huge. Let's refactor it into smaller
functions so we can add treemanifest support safely later.

Test Plan: Ran the tests

Reviewers: #mercurial, mitrandir

Reviewed By: mitrandir

Subscribers: mitrandir, medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5340576

Signature: t1:5340576:1498751222:7ba075ef49ea9d0ad368e6402fb9e61f0d1d0c94
2017-07-10 15:53:12 -07:00
Durham Goode
81959671dd pushrebase: refactor post-lock cache filling to be a function
Summary:
The pushrebase main function is too huge. Let's refactor it into smaller
functions so we can add treemanifest support safely later.

Test Plan: Ran the tests

Reviewers: #mercurial, mitrandir

Reviewed By: mitrandir

Subscribers: mitrandir, medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5340571

Signature: t1:5340571:1498751172:4d926642c68611989ee439fbb0327d6e403cb555
2017-07-10 15:53:12 -07:00
Durham Goode
02bd7b1d57 pushrebase: refactor pre rebase cache prefetching to a function
Summary:
The pushrebase main function is too huge. Let's refactor it into smaller
functions so we can add treemanifest support safely later.

Test Plan: Ran the tests

Reviewers: #mercurial, mitrandir

Reviewed By: mitrandir

Subscribers: mitrandir, medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5340565

Signature: t1:5340565:1498751154:81c70d75131977defdaa7f92ef0a3b2214b86381
2017-07-10 15:53:12 -07:00
Durham Goode
dee8ff410d pushrebase: refactor prepushrebase hooks to function
Summary:
The pushrebase main function is too huge. Let's refactor it into smaller
functions so we can add treemanifest support safely later.

Test Plan: Ran the tests

Reviewers: #mercurial, mitrandir

Reviewed By: mitrandir

Subscribers: mitrandir, medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5340559

Signature: t1:5340559:1498751124:9c4542e0bbd041fcc584a8eb1122269135fe92c8
2017-07-10 15:53:12 -07:00
Durham Goode
474765111d pushrebase: handle incoming pack parts
Summary:
Now that packs are being sent across the wire during push, pushrebase needs to
handle those packs. It does so by adding them to the store list and replacing
the bundlerepo's data and history store with new ones that include the temporary
packs.

Test Plan: Future patch adds test around receiving trees in pushrebase.

Reviewers: #mercurial, mjpieters

Reviewed By: mjpieters

Subscribers: mitrandir, medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5329893

Signature: t1:5329893:1499706143:44afeacf594a4f87cdc0d96c3f2700dbd4aadd75
2017-07-10 15:53:12 -07:00
Durham Goode
f762741367 pushrebase: generate tree pack parts during push
Summary:
We want to enable pushing tree packs during pushrebase, and eventually we can
stop including flat manifest information entirely. The first step is to generate
the tree packs during push. A future patch will handle the incoming parts.

Test Plan: Ran the tests. A future patch adds pack file handling to test this

Reviewers: #mercurial, mjpieters

Reviewed By: mjpieters

Subscribers: mitrandir, medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5329775

Signature: t1:5329775:1499705168:43ea77c9a5c85c09b40910e102535d89a0504a4d
2017-07-10 15:53:12 -07:00
Jun Wu
18c1406b18 inhibitold: add back the old extension
Summary:
This allows us to quickly switch back to the old behavior somehow by using
Chef overrides.

Test Plan: The file was copied from the old revision. Therefore not tested.

Reviewers: #mercurial, kulshrax

Reviewed By: kulshrax

Subscribers: medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5392986

Signature: t1:5392986:1499722593:a895f94b3281ccba709fc219f927e8f6c770054a
2017-07-10 15:45:52 -07:00
Jun Wu
99386e40fd inhibit: update existing code to work with the rewritten version
Summary:
debuginhibit was removed since we no longer have a separate inhibit state.

smartlog hack about changing "o" to "x" was removed since "obsolete()"
revset is correct and the hack is unnecessary now.

directaccess was removed from tests since inhibit does not depend on it.

`- obsolete()` was added to some revsets to avoid divergence and other
surprises.

Use `inhibit.revive` API in infinitepush and reset to revive changesets
properly.

Remove various hacky code that mangle inhibit state in corner cases.

Most test changes are `o` changed to `x` in output since we draw
`obsolete()` state correctly now. `test-infinitepush-backup-remotefilelog.t`
change was because output could be `bytes/sec` instead of `KB/sec`.

Test Plan: arc unit

Reviewers: #mercurial, kulshrax

Reviewed By: kulshrax

Subscribers: kulshrax, medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5391361

Signature: t1:5391361:1499722618:d3c1cf629f0c59ecdf1dfd5e653c1eb6176646b8
2017-07-10 15:45:31 -07:00
Jun Wu
40503c20fb inhibit: completely rewrite the extension
Summary:
The main feature we want is to be able to unobsolete an already obsoleted
changeset. The old inhibit code is causing all kinds of unsolvable weird
cases and is hard to deubg or maintain.

This patch rewrites it completely. Basically, we now require people to use
obsmarkers to "unobsolete" changesets. We treat cycles in obsstore as a
normal case and break the cycle using date information.

It should be a neat and correct solution until we want marker exchange.

A "revive" API was provided for other extensions to use.

Tests and other code changes will be fixed in a follow up.

Test Plan:
`test-inhibit.t` was rewritten to test the new features.

Other tests are broken and skipped for now. The next diff will fix them.

Reviewers: #mercurial, kulshrax

Reviewed By: kulshrax

Subscribers: medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5391320

Signature: t1:5391320:1499716172:a946381421cc242411f5175ee3b7a3a0bc5a4f07
2017-07-10 15:45:07 -07:00
Jun Wu
b757e7498c codemod: use obsutil
Upstream has moved APIs to obsutil, and removed some APIs from obsolete.py
2017-07-07 20:54:19 -07:00
Adam Simpkins
4a7be070d7 phabstatus: fix interaction with the smartlog extension
Summary:
Previously phabstatus.extsetup() would throw an exception if the smartlog
extension was not in use and already loaded.

This fixes the code to use extensions.afterloaded() to ensure that we try to
wrap smartlog functions only after smartlog is loaded.  It also handles the
lookup error if smartlog is not in use at all.

Test Plan: Included a new unit test.

Reviewers: #mercurial, phillco

Reviewed By: phillco

Subscribers: medson, mjpieters, net-systems-diffs@fb.com

Differential Revision: https://phabricator.intern.facebook.com/D5375301

Tasks: 16939242

Signature: t1:5375301:1499323251:4bc00c9cba79d60cbbb6738e8f1d648e75055cdb
2017-07-06 09:38:14 -07:00
Felix Merk
5ddcdfffe8 undo: add olddraft revset to return previous drafts
Summary: Register `olddraft([index])` revset returning draft changesets using undo data.

Test Plan: unit tests

Reviewers: stash, #mercurial, quark

Reviewed By: quark

Subscribers: mitrandir, quark, medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5359322

Tasks: 19646287, 19261492

Signature: t1:5359322:1499122771:0a1e952a9bb4cd6b31d95a229ccb398cd6aafd1f
2017-07-03 19:01:09 -07:00
Felix Merk
ba1dce8f88 undo: deal with --hidden flag edge case
Summary:
Makes sure repo is filtered by the "visible" filter which hides hidden commits.
Otherwise --hidden flag can mass up "draftheads" storage.

Test Plan: Added a new test case.

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: quark, medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5357222

Tasks: 19822306

Signature: t1:5357222:1499102417:ce78065d36841601fe0dfb1d615d7b3be28c5870
2017-07-03 19:01:09 -07:00
Felix Merk
955a9bfcce undo: allow user to list or see details about undoable commands run
Summary: add hg debugundohistory -l to view recent undoable transactions

Test Plan: unit test

Reviewers: #mercurial, stash, quark

Reviewed By: quark

Subscribers: quark, mitrandir, durham, stash, medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5325772

Tasks: 19646287

Signature: t1:5325772:1499122626:48dd25cee8ed8912c07a6c1dabb3d4286483a8b7
2017-07-03 19:01:09 -07:00
Jun Wu
605fa025ff morecolors: colorize uncaught exceptions
Summary:
Previously morecolors only changes "ui.traceback". This diff makes it replace
uncaught exceptions. It now conflicts with errorredirect (whichever loads last
will be effective) but that's probably fine since morecolors is intended for
the team only.

Test Plan: Added a test case.

Reviewers: #mercurial, mitrandir

Reviewed By: mitrandir

Subscribers: mitrandir, medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5347030

Signature: t1:5347030:1498751985:b8c750c5bfe4a25f1e0a4dd1cebcfa74fa8739b0
2017-06-29 09:55:25 -07:00
Kostia Balytskyi
d9a8476274 obsshelve: adopt upsteam changes regarding applybundle
Summary:
Upstream made a coupld of changes (including 4fadfa7f05a2a49a2554f0d70f9a35e085109d44), which changed how bundles are supposed to be applied. This breaks obsshelve's compatibility with old shelves.

This commit just ports the aformentioned upstream change into obsshevlve.

Test Plan: rt

Reviewers: quark, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5347598
2017-06-29 09:52:11 -07:00
Felix Merk
1abcb63689 undo: amend transacts twice
Summary: amend performs two transactions, second without command which breaks our assertion, but possibly isn't an issue. This fixes the assertion.

Test Plan: unit tests

Reviewers: #mercurial, stash, quark

Reviewed By: quark

Subscribers: quark, stash, medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5326358

Tasks: 19261492

Signature: t1:5326358:1498684945:0fde1df243cdbac67ceb092d30599ebd56a731a5
2017-06-28 15:28:40 -07:00
Sen Li
3a87ac6ec2 errorredirect: modify errorredirect extension to send crash log to scuba table
Summary: Modify error redirect extension to send crash log, type and traceback to scuba table hgerrors

Test Plan:
Sample that is being sent to Scuba:
```
{
"int":{"time":1498528572},
"normal":{
  "host":"devvm26497.prn1",
  "metrics_type":"hgerrors",
  "msg":"exception has occurred: ** unknown exception encountered, please report by visiting\n** Source Control @ FB group\n** Python 2.7.5 (default, Nov  6 2016, 00:28:07) [GCC 4.8.5 20150623 (Red Hat 4.8.5-11)]\n** Mercurial Distributed SCM (version 4.2.1+842-79c3f212a9c9)\n** Extensions loaded: tweakdefaults, hgk, absorb, arcdiff, automv, blackbox, chistedit, color, debugcommitmessage, debuginhibit, evolve, fbhistedit, githelp, hiddenerror, histedit, inhibit, journal, logginghelper, lz4revlog, morestatus, moreversion, myparent, pager, patchrmdir, perftweaks, phabdiff, phabstatus, phrevset, pullcreatemarkers, purge, rage, rebase, record, remotefilelog, remotenames, reset, sampling, fbshow, sigtrace, simplecache, smartlog, sshaskpass, strip, directaccess, uncommit, conflictinfo, extorder, obsshelve, sqldirstate, fastmanifest, hgsubversion, traceprof, dialect, grpcheck, errorredirect, fbamend, pushvars, pushrebase\n",
  "traceback":"Traceback (most recent call last):\n  File \"\/usr\/lib64\/python2.7\/site-packages\/mercurial\/dispatch.py\", line 326, in _callcatch\n    return scmutil.callcatch(ui, func)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/mercurial\/scmutil.py\", line 145, in callcatch\n    return func()\n  File \"\/usr\/lib64\/python2.7\/site-packages\/mercurial\/dispatch.py\", line 308, in _runcatchfunc\n    return _dispatch(req)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/mercurial\/dispatch.py\", line 938, in _dispatch\n    cmdpats, cmdoptions)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/mercurial\/extensions.py\", line 273, in closure\n    return func(*(args + a), **kw)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/fastmanifest\/cachemanager.py\", line 318, in runcommandtrigger\n    result = orig(*args, **kwargs)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/mercurial\/extensions.py\", line 273, in closure\n    return func(*(args + a), **kw)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/fastmanifest\/__init__.py\", line 175, in _logonexit\n    r = orig(ui, repo, cmd, fullargs, *args)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/mercurial\/extensions.py\", line 273, in closure\n    return func(*(args + a), **kw)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/hgext3rd\/perftweaks.py\", line 263, in _tracksparseprofiles\n    res = runcommand(lui, repo, *args)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/mercurial\/extensions.py\", line 273, in closure\n    return func(*(args + a), **kw)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/hgext3rd\/perftweaks.py\", line 248, in _trackdirstatesizes\n    res = runcommand(lui, repo, *args)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/mercurial\/extensions.py\", line 273, in closure\n    return func(*(args + a), **kw)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/hgext\/journal.py\", line 76, in runcommand\n    return orig(lui, repo, cmd, fullargs, *args)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/mercurial\/dispatch.py\", line 674, in runcommand\n    ret = _runcommand(ui, options, cmd, d)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/mercurial\/extensions.py\", line 273, in closure\n    return func(*(args + a), **kw)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/hgext\/pager.py\", line 69, in pagecmd\n    return orig(ui, options, cmd, cmdfunc)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/mercurial\/dispatch.py\", line 946, in _runcommand\n    return cmdfunc()\n  File \"\/usr\/lib64\/python2.7\/site-packages\/mercurial\/dispatch.py\", line 935, in <lambda>\n    d = lambda: util.checksignature(func)(ui, *args, **strcmdopt)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/mercurial\/util.py\", line 1056, in check\n    return func(*args, **kwargs)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/mercurial\/extensions.py\", line 273, in closure\n    return func(*(args + a), **kw)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/mercurial\/util.py\", line 1056, in check\n    return func(*args, **kwargs)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/mercurial\/extensions.py\", line 273, in closure\n    return func(*(args + a), **kw)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/hgext3rd\/absorb\/__init__.py\", line 977, in _amendcmd\n    return orig(ui, repo, *pats, **opts)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/mercurial\/util.py\", line 1056, in check\n    return func(*args, **kwargs)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/mercurial\/extensions.py\", line 273, in closure\n    return func(*(args + a), **kw)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/mercurial\/util.py\", line 1056, in check\n    return func(*args, **kwargs)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/hgext3rd\/tweakdefaults.py\", line 627, in cmd\n    return origcmd(*args, **kwargs)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/hgext3rd\/tweakdefaults.py\", line 694, in amendcmd\n    return orig(ui, repo, *pats, **opts)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/mercurial\/util.py\", line 1056, in check\n    return func(*args, **kwargs)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/mercurial\/extensions.py\", line 273, in closure\n    return func(*(args + a), **kw)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/mercurial\/util.py\", line 1056, in check\n    return func(*args, **kwargs)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/hgext\/automv.py\", line 63, in mvcheck\n    return orig(ui, repo, *pats, **opts)\n  File \"\/usr\/lib64\/python2.7\/site-packages\/mercurial\/util.py\", line 1056, in check\n    return func(*args, **kwargs)\n  File \"\/data\/users\/lsen\/facebook-hg-rpms\/fb-hgext\/hgext3rd\/fbamend\/__init__.py\", line 169, in amend\n    raise Exception(\"testing\")\nException: testing\n",
  "type":"<type 'exceptions.Exception'>"}
}
```

Reviewers: juehui, akushner, durham

Reviewed By: durham

Subscribers: medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5326253

Tasks: 19602850

Signature: t1:5326253:1498577444:c62977ec782519a22b9dc64b33f2a575c544b83c
2017-06-27 09:44:29 -07:00
Jun Wu
e7998929ab inhibitwarn: temporarily add the extension back
Summary:
This extension is still enabled by other code that has a longer release
cycle. Add back the placeholder extension temporarily until we confirm their
release removed the config.

Test Plan: Adding an empty file should be harmless.

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: durham, medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5321153

Signature: t1:5321153:1498500260:941dd1738a2ab3d3389c4251d86bf7c2ccc17648
2017-06-26 11:08:30 -07:00
Therin Irwin
66b64d8fb6 fbamend: change hint text to suggest running restack
Summary: Instead of referencing `hg rebase` --restack option, just reference the `restack` subcommand.

Test Plan:
1. Make a few commits 1, 2, 3, 4 on any branch
2. `hg update 1`
3. Try to amend commit 1
4. Warning message should appear about children left behind
expected: Hint text does not mention `hg rebase --restack`

Reviewers: durham, kulshrax

Reviewed By: kulshrax

Subscribers: medson, mjpieters, kkrewink

Differential Revision: https://phabricator.intern.facebook.com/D5313108

Tasks: 19601780

Signature: t1:5313108:1498249314:98f31f4424341ef51997687293bf99b7297b6cd0
2017-06-26 07:31:28 -07:00
Jun Wu
14440f1fcf tweakdefaults: add bookmark -D support
Summary:
It got removed from inhibit by D5258813. It's a useful feature so let's
re-implement it in tweakdefaults.

Test Plan: Added test cases with plain strip and fbamend's safe strip.

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5313291

Signature: t1:5313291:1498321590:03e61a5b1acc2df44afcfd6ceb723e17c648ce79
2017-06-24 23:18:52 -07:00
Jun Wu
17886e370b fbamend: remove troublesome tr.release()
Summary:
`tr` could be None and `tr.release()` there is causing issues.

The next line actually checks `tr` and call `tr.release()` if `tr` is not None.

Test Plan: arc unit

Reviewers: #mercurial, wez

Reviewed By: wez

Subscribers: medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5314538

Signature: t1:5314538:1498255892:084e1f5f1521ff7794be43b24510d2b54bf2148e
2017-06-23 15:12:46 -07:00
Felix Merk
fbd87d6923 undo: store op history
Summary: Store basic operational history: bookmarks, commands, private heads, date and working copy parent

Test Plan: Unit tests

Reviewers: durham, #mercurial, mitrandir, quark

Reviewed By: quark

Subscribers: stash, mitrandir

Differential Revision: https://phabricator.intern.facebook.com/D5293596

Tasks: 19261492, 19563903, 19450726

Tags: Hg Undo

Signature: t1:5293596:1498245082:264e344054ee5061a439dec1c6071c2a60ffcf55
2017-06-23 12:19:09 -07:00
Jun Wu
c35c433212 smartlog: be graceful when ancestorcache db is corrupted
Summary:
We have seem some errors when closing the database during `db.close()`. This
diff nukes the database if that happens.

Test Plan: arc unit

Reviewers: #mercurial, mjpieters

Reviewed By: mjpieters

Subscribers: medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5305123

Signature: t1:5305123:1498166056:8355e861c18c670e6342e541f2bc55d65bc7b1c8
2017-06-22 14:15:56 -07:00
Mateusz Kwapich
afebcb5512 phabdiff: add singlepublicbase revset
Summary:
This revset will be used by jellyfish to set the proper base in sandcastle. Anyway: a public ancestor is
a valuable commit information so there should be a template for it.

Test Plan: see test

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: durham, quark, medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5275811

Tasks: 19186426

Signature: t1:5275811:1497979632:9589491be723daf0b127703a7bc4571f7539bd84
2017-06-21 17:03:50 +01:00
Jun Wu
ce21516db2 smartlog: use dagop.reachableroots
97fbbbc2ac35 moved reachableroots from revset to dagop.
2017-06-20 13:58:40 -07:00
Jun Wu
2df622a724 cleanup: remove fbmetaedit and inhibitwarn
Summary: We no longer use those extensions. Remove them.

Test Plan: arc unit

Reviewers: #mercurial

Differential Revision: https://phabricator.intern.facebook.com/D5286709
2017-06-20 13:58:40 -07:00
Durham Goode
55f677869b remotefilelog: use a connection pool instead of a long running connection
Summary:
Previously remotefilelog would open a connection and leave the getfiles command
running on that connection, so it didn't have to reopen the ssh connection each
time. We want to reuse this ssh connection for treemanifest and fastannotate, so
let's switch it to a pool model where the connection is kept open but the
getfiles command is not left open.

If an exception happens while the connection is out of the pool, it is discarded
instead of being added back to the pool.

Test Plan:
Ran the tests. The fastannotate tests changed to reflect the new way
the connectionpool allows use.

Reviewers: quark, #mercurial, mitrandir

Reviewed By: mitrandir

Subscribers: mitrandir, medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5280323

Signature: t1:5280323:1497975420:e3ae1ee854a1afc90816502543a19ff36f59b497
2017-06-20 11:08:15 -07:00
Jun Wu
32dd7f7841 fbamend: fix tests
Fix test issues caused by importing 0e0ff0ef4ad6.
2017-06-19 13:24:41 -07:00
Adam Simpkins
13bd099e74 pushrebase: improve blocknonpushrebase error message
Summary:
Improve the error message from the blocknonpushrebase hook to inform the user
that they need the pushrebase extension enabled.

Previously the message indicated that you mush push using
`hg push --to <bookmark>`, but this is not sufficient.  Enabling the pushrebase
hook is the most important step here.  (The `--to` flag is not part of vanilla
mercurial, and is added by pushrebase, but other extensions like remotenames
also add a `--to` flag.)

Test Plan:
Added a unit tests for `hg push --to <bookmark>` with remotenames enabled but
pushrebase disabled.

Reviewers: #mercurial, davidsp, zhihuih

Reviewed By: zhihuih

Subscribers: quark, medson, mjpieters, net-systems-diffs@fb.com

Differential Revision: https://phabricator.intern.facebook.com/D5276550

Signature: t1:5276550:1497899421:3656bcc3e7358f97fc852e14307b34d8aa4d9951
2017-06-19 12:48:46 -07:00
Zhihui Huang
5e0994a87f p4fastimport: handle keyword extension
Summary:
Perforce has a feature called [[ http://answers.perforce.com/articles/KB/3482 | keyword expansion ]].

When a file is marked +k or +ko and you have a string like $Id$, it get's expanded to $Id: SOMETHING$ on Perforce. When importing a file like this from Perforce into Mercurial we need to do the reverse, renaming $Id: SOMETHING$ to $Id$ (and equivalent for all other expansion types).


Test Plan:
$ cd ~/facebook-hg-rpms/fb-hgext/tests/
$ python ../../hg-crew/tests/run-tests.py test-p4fastimport-import.t
$ python ../../hg-crew/tests/run-tests.py test-p4*

Reviewers: #idi, davidsp

Reviewed By: davidsp

Subscribers: davidsp

Differential Revision: https://phabricator.intern.facebook.com/D5266323

Tasks: 19211063

Signature: t1:5266323:1497890444:a1cced5faa5e48c708c2721510952ca4d27d1bd4
2017-06-19 11:54:10 -07:00
Tony Tung
6c4e900989 add date=implicitupdate option to automagically update the commit date 2017-06-19 09:50:32 -07:00
Tony Tung
c4d7df0a0f add the --date and --user parameters that commit supports 2017-06-19 09:50:32 -07:00
Mateusz Kwapich
250c2c7c4a metaedit: fix editing multiple commits
Summary: The recent refactoring brought back the original condition from metaedit that was preventing multi-commit metaedits.

Test Plan: see test

Reviewers: #mercurial, quark, stash

Reviewed By: stash

Subscribers: medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5274562

Signature: t1:5274562:1497884504:f46988cd1926f52227a907e4e081e65de24080ad
2017-06-19 16:03:17 +01:00
Jun Wu
3d461ae600 check-ext: make checks stricter
Summary:
Enhance check-ext script to be more strict:

 - Only one foreign extension is allowed: `remotenames`
 - Require explicit path for in-repo extensions to avoid wrong extensions
   being tested

This would make the test more predicatable since system extensions
will be less likely to be imported. Explicit path is better than
setting `PYTHONPATH` since `hgext/name.py` could override
`hgext3rd/name.py` regardless of `PYTHONPATH`.

Test Plan: arc unit

Reviewers: phillco, durham, ikostia, #mercurial, stash

Reviewed By: stash

Subscribers: medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5271430

Signature: t1:5271430:1497861776:7dd35ec7c522cd9b26aa0871cb4306b4f1b8993a
2017-06-19 08:02:38 -07:00
Stanislau Hlebik
7b8cb0004c debugcommitmessage: fix tests
Upstream 608e426d0577d44f4d5030dc6d71e57d691ca3da changed the API. Now template
ref should be passed instead of template itself. This diff fixes it.
2017-06-19 03:01:18 -07:00
Jun Wu
a6c7fc128f allowunstable: remove the extension
Summary: allowunstable is no longer used. So remove it.

Test Plan: arc unit

Reviewers: #mercurial, stash

Reviewed By: stash

Subscribers: mjpieters, medson

Differential Revision: https://phabricator.intern.facebook.com/D5258845

Signature: t1:5258845:1497602890:4b9c7cffc2132df0a7059fe2fdbeddf3e21c1875
2017-06-16 15:54:15 -07:00
Jun Wu
be7ac42b50 codemod: replace evolve by fbamend
Summary:
Now that fbamend has most of the features we want. Replace evolve
used by tests.

Minor changes are made to some extensions to make test pass:
- reset: drop commit reviving support (was provided by "hg touch")
- smartlog: do not assume who provides "allsuccessors" revset
- fbamend: return 1 on "nothing changed"
- fbamend: write operation data for split and fold
- fbamend: do not print incompatibility notice
- fbamend: do not disable itself when rebase is not enabled
- inhibit: remove `bookmark -D` option - use `prune -B` instead

Test Plan: arc unit

Reviewers: #mercurial, stash

Reviewed By: stash

Subscribers: stash, mjpieters, medson

Differential Revision: https://phabricator.intern.facebook.com/D5258813

Signature: t1:5258813:1497604051:0c7334f9a6554cf31aaaaa2a4995b17d4e445847
2017-06-16 15:50:01 -07:00
Jun Wu
2dc367dac3 inhibitwarn: nuke the extension
Summary:
We are going to remove evolve and inhibit. So the warning is no longer
useful.

Test Plan: arc unit

Reviewers: #mercurial, ikostia

Reviewed By: ikostia

Subscribers: ikostia, mjpieters, medson

Differential Revision: https://phabricator.intern.facebook.com/D5254984

Signature: t1:5254984:1497518474:80649544fef3aee90bf9a407d45ddeb9f6d85dba
2017-06-16 14:40:31 -07:00
Jun Wu
d06133e837 fbamend: backport prune
Summary:
Backport the prune command and its strip wrapper from mutable-history changeset
cb0d62ed5e2a using GPL2 license.

There are some adjustments to make the ported code work well with existing
code better, namely:

  - Add a `safestrip` option to control whether `strip` is replaced to `prune`
    or not.


Test Plan: arc unit

Reviewers: #mercurial, ikostia

Reviewed By: ikostia

Subscribers: mjpieters, medson

Differential Revision: https://phabricator.intern.facebook.com/D5254961

Signature: t1:5254961:1497518704:91b200bf791297ff797cc0ff28ab31d7eb294963
2017-06-16 14:39:04 -07:00
Jun Wu
2852dabbc8 fbamend: merge fbmetaedit
Summary:
Merge fbmetaedit which uses a fast path to change commit messages.

Make fbmetaedit a no-op.

Test Plan: arc unit

Reviewers: #mercurial, mitrandir

Reviewed By: mitrandir

Subscribers: mjpieters, medson

Differential Revision: https://phabricator.intern.facebook.com/D5254867

Signature: t1:5254867:1497524975:c7cc3ad9d1162332fbdd665eb5a2f6d7ce87a585
2017-06-16 14:12:32 -07:00
Jun Wu
87b248dd27 fbamend: backport metaedit command
Summary:
Backport the metaedit command and its test from mutable-history changeset
cb0d62ed5e2a using GPL2 license.

There are some adjustments to make the ported code work well with existing
code better, namely:

  - Removed allowunstable check to simplify the code.
  - Make "nothing changed" return 1.
  - Minor i18n and check code fixes.

Note: this backport does not include fbmetaedit changes.

Test Plan: arc unit

Reviewers: #mercurial, mitrandir

Reviewed By: mitrandir

Subscribers: mjpieters, medson

Differential Revision: https://phabricator.intern.facebook.com/D5254807

Signature: t1:5254807:1497524996:abf856dc9bb8225316ca6c17f41ffa2c9d8d9964
2017-06-16 14:11:43 -07:00
Jun Wu
04579ab177 fbmetaedit: do not assume evolve provides metaedit
Summary: This is temporary as fbmetaedit will be merged into fbamend.

Test Plan: arc unit

Reviewers: #mercurial, ikostia

Reviewed By: ikostia

Subscribers: mjpieters, medson

Differential Revision: https://phabricator.intern.facebook.com/D5254803

Signature: t1:5254803:1497520047:2c6eb22ce8cb80e4c785fd0b82311706487f6c7e
2017-06-16 14:07:02 -07:00
Jun Wu
c7c3280b44 fbamend: backport fold command
Summary:
Backport the fold command and its dependencies (`rewrite`) from mutable-history
changeset cb0d62ed5e2a using GPL2 license.

There are some adjustments to make the ported code work well with existing
code better, namely:

 - The `--norebase` flag was renamed as `--no-rebase` and moved to the
   backported command so we no longer wrap the split command.
 - Rebase now runs inside a same transaction.
 - `bookmarksupdater` now accepts multiple revs so bookmarks on all changesets
   being folded will be updated correctly. It is a bugfix to the ported code.

Test Plan:
`arc unit`.
Some tests are changed. It seems the new behavior looks better.

Reviewers: #mercurial, ikostia

Reviewed By: ikostia

Subscribers: ikostia, mjpieters, medson

Differential Revision: https://phabricator.intern.facebook.com/D5254681

Signature: t1:5254681:1497520003:f7ec3622286804fead059ab9a8021bb8f3067e64
2017-06-16 14:06:24 -07:00
Jun Wu
020ff84b79 fbamend: backport split command
Summary:
Backport the split command and its dependencies (namely, `bookmarksupdater`)
from mutable-history changeset cb0d62ed5e2a using GPL2 license.

There are some adjustments to make the ported code work well with existing
code, namely:

 - The `--norebase` flag was renamed as `--no-rebase` and moved to the
   backported command so we no longer wrap the split command.
 - Rebase now runs inside a same transaction.

The glob change in tests is because `(tip)` shows up.


Test Plan: arc unit

Reviewers: #mercurial, ikostia

Reviewed By: ikostia

Subscribers: ikostia, mjpieters, medson

Differential Revision: https://phabricator.intern.facebook.com/D5254544

Signature: t1:5254544:1497519470:fe8a7308b35b578fd4f1257cebba15062e752c2f
2017-06-16 13:59:12 -07:00
Jun Wu
08e9bc0bbf fbamend: make restack take inhibithack
Summary:
This hack is needed to make split + rebase work well inside a same transaction.

inhibit does special things at the end of a transaction which may override deinhibit.
Sometimes we want deinhibit to override the transaction close handler.

The hack will be removed once inhibit gets removed.

Test Plan: arc unit

Reviewers: #mercurial, ikostia

Reviewed By: ikostia

Subscribers: ikostia, mjpieters, medson

Differential Revision: https://phabricator.intern.facebook.com/D5254517

Signature: t1:5254517:1497519294:89b6de077514252c42fb128760cf9f9241180242
2017-06-16 13:55:29 -07:00
Jun Wu
30b417ea27 allowunstable: do not assume evolve provides split and fold
Summary:
split and fold will be backported and removed from evolve. So do not assume
evolve provides them.

Test Plan: Not breaking existing tests

Reviewers: #mercurial, ikostia

Reviewed By: ikostia

Subscribers: mjpieters, medson

Differential Revision: https://phabricator.intern.facebook.com/D5254455

Signature: t1:5254455:1497519005:aca0373cc61ca5438cee1c9d85f2df77f33d8455
2017-06-16 13:54:03 -07:00
Jun Wu
edf220db3a inhibit: allow _deinhibitmarkers inside a transaction
Summary:
inhibit hacks transactioncallback so _deinhibitmarkers might be ineffective
inside a transaction. This diff adds a flag to allow _deinhibitmarkers
effective inside a transaction. It will be used by the split backport.

Test Plan: No new test being added since inhibit is a hack and is going to be removed.

Reviewers: #mercurial, ikostia

Reviewed By: ikostia

Subscribers: mjpieters, medson

Differential Revision: https://phabricator.intern.facebook.com/D5254416

Signature: t1:5254416:1497519161:b01f3fa59004ded5beaf5dcfc7d18eb2b5227130
2017-06-16 13:52:29 -07:00
Jun Wu
200a9b16fb fbamend: implement successors and precursors revsets
Summary:
We are depending on `allprecursors` and `allsuccessors` revsets in multiple
places. Re-implement them in fbamend.

Also reimplement `successors` and `precursors` since they are used by some
other code.

Test Plan: arc unit

Reviewers: #mercurial, ikostia

Reviewed By: ikostia

Subscribers: mjpieters, medson

Differential Revision: https://phabricator.intern.facebook.com/D5254403

Signature: t1:5254403:1497519041:8c6f001a6acf8056b5f795575b42922a476ac34b
2017-06-16 13:48:14 -07:00
Jun Wu
7e8b62aef6 lfs: add retry logic for transferring a single object
Summary:
We got TCP RESETs frequently. It's hard to pinpoint the root cause so let's
just add retry for now.

Test Plan: eyes

Reviewers: #mercurial, #ovrsource_warroom, steaphan

Reviewed By: steaphan

Subscribers: steaphan, mjpieters, medson

Differential Revision: https://phabricator.intern.facebook.com/D5265928

Tasks: 19419154

Signature: t1:5265928:1497635499:17cf5d5cb69b406330326d693a9eceb1d22861f8
2017-06-16 10:54:07 -07:00
Jun Wu
3556ad01c5 fbamend: rename conflicted evolve commands instead of removing them
Summary: This unbreaks `experimental.evolutioncommands`.

Test Plan: arc unit

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: mjpieters, medson

Differential Revision: https://phabricator.intern.facebook.com/D5256903

Signature: t1:5256903:1497547902:9af6930776fc272db0c7d1bbfca2f4e76342255a
2017-06-15 10:35:28 -07:00
Felix Merk
88a744be9e smartlog: let smartlog revset take recentday argument
Summary: Add an optional `recentdays` parameter to smartlog revset.

Test Plan: Added test to test-smartlog.t

Reviewers: #mercurial, durham, quark

Reviewed By: quark

Subscribers: durham, stash, mjpieters, medson

Differential Revision: https://phabricator.intern.facebook.com/D5211513

Tasks: 19179819

Signature: t1:5211513:1497487196:f7d59ec32c09680ed769977890f3031369f7652e
2017-06-15 10:06:09 -07:00
Stanislau Hlebik
6c09d4ce19 rage: add a hack to make rage work on osx
Summary: Add a config option to specify rpm binary

Test Plan: arc unit

Reviewers: #mercurial, quark, ikostia, luk

Reviewed By: ikostia, luk

Subscribers: luk, mjpieters, medson, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5255174

Tasks: 19361425

Signature: t1:5255174:1497518216:d78a66e27a1073921f06d9dd0e7036bab3e4010a
2017-06-15 04:20:52 -07:00
Stanislau Hlebik
31e9c3d80b copytrace: fix logging
ctx.node() is None, so it's bettter to use ctx.hex()
2017-06-15 04:08:49 -07:00
Felix Merk
5c4ff4f39f tweakdefaults: add --inactive flag to update command
Summary:
People want update to optionally not activate a bookmark.
This diff adds an `--inactive` flag to do that.

Test Plan: Added test-update-i.t

Reviewers: durham, #mercurial, quark

Reviewed By: quark

Subscribers: mitrandir, mjpieters, medson, durham, akushner, quark

Differential Revision: https://phabricator.intern.facebook.com/D5228552

Tasks: 8097957

Signature: t1:5228552:1497396566:16322e1a2dfdab30a213ddc37cbde4fc71bffcc0
2017-06-14 16:42:39 -07:00
Olivier Trempe
7b57890313 pushrebase: make pretxnchangegroup and changegroup hook arguments consistent with mercurial hook documentation
Before this patch, the pretxnchangegroup 'node' argument would be the last
added node whereas mercurial documention specifies that it should be the first
new changeset. The 'node_last' argument has also been added to both
pretxnchangegroup and changegroup hooks as per mercurial documentation.
2017-06-14 16:29:16 -07:00
Olivier Trempe
602a765fe6 pushrebase: bring variable initialization closer to first usage 2017-06-14 16:29:16 -07:00
Jun Wu
8d58d00052 fbamend: move restack to a separate file
Summary: This makes `__init__.py` smaller.

Test Plan: arc unit

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: mjpieters, medson

Differential Revision: https://phabricator.intern.facebook.com/D5223421

Signature: t1:5223421:1497387183:9ce70419c3c63ca55f1a225192ae7a6556316ea3
2017-06-13 17:09:34 -07:00
Jun Wu
3ab3ee018a fbamend: move next, previous to a movement.py
Summary:
Move movement commands (next, previous) to `movement.py`. Some common utilities
were moved to `common.py`. I did some minor cleanups on the help output.

Test Plan: arc unit

Reviewers: #mercurial, ikostia

Reviewed By: ikostia

Subscribers: mjpieters, medson

Differential Revision: https://phabricator.intern.facebook.com/D5222981

Signature: t1:5222981:1497265815:5dbba53fe7a16dbf924898331496a5c50b54b9f2
2017-06-13 17:09:16 -07:00
Jun Wu
e1f58d87e3 fbamend: move unamend to a separate file
Summary:
Going to split fbamend into individual files by commands. `unamend` seems
self-contained to be moved out.

Test Plan: arc unit

Reviewers: #mercurial, ikostia

Reviewed By: ikostia

Subscribers: mjpieters, medson

Differential Revision: https://phabricator.intern.facebook.com/D5221159

Signature: t1:5221159:1497261013:d371bcf4406277228fc4f3f6c6aaa4600dab4823
2017-06-13 17:08:50 -07:00
Jun Wu
f1b6191c18 fbamend: move to a package
Summary: `fbamend.py` is already long. It will become more complex so let's move it.

Test Plan: arc unit

Reviewers: #mercurial, ikostia

Reviewed By: ikostia

Differential Revision: https://phabricator.intern.facebook.com/D5209475

Signature: t1:5209475:1497260957:e84aec4a2f90c8dd1d247b2b0caef48cfbb59c55
2017-06-13 17:08:20 -07:00
Olivier Trempe
0e2e9efbc7 pushrebase: add node_onto argument to prepushrebase hook
This helps doing some validation before accepting a pushrebase.
For example, if you want to prevent users to accidentally pushrebase
changesets to the wrong branch by checking if all pushed changesets are
on the same branch as the destination (--to)
2017-06-13 16:32:25 -07:00
Stanislau Hlebik
defa69c1d8 extorder: fix test
Summary:
There was a change in core mercurial b319e3173a9534a2b7750b69ef446a803c3145ff
that catches all exception from uisetup and extsetup. Let's throw subclass of
BaseException in this case.

Test Plan: arc unit

Reviewers: #mercurial, mjpieters

Reviewed By: mjpieters

Subscribers: mjpieters, medson, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5236650

Signature: t1:5236650:1497363414:9bbde3cabb696bb77a7ffddcb576923e5743c993
2017-06-13 09:13:41 -07:00
Stanislau Hlebik
300bc72abd absorb: fix test
Upstream e0fc50267c46 removed makememctx() function and requires filectxfn to
be an instance of patch.filestore.
2017-06-13 07:17:36 -07:00
Stanislau Hlebik
b82acd87f7 conflictinfo: fix test
There was a bug that was introduced upstream in 469914605447 and fixed in
758d59a5f3515b18c767af69f4ed28060f8af56c. This diff reflects the changes.

Also filestat was changed in 4091c920ac07e27ee8536715297127e56d536dab.
2017-06-13 07:15:54 -07:00
Stanislau Hlebik
5a69e3d88a tweakdefaults: dump per file diff statistic in json
Summary:
Upstream mercurial --stat option shows only aggregated statistics for all
files. This diff make it possible to get diff statistics per file.
It can be used by automation.

Test Plan: arc unit

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: quark, mjpieters, medson, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5203835

Tasks: 18508761

Signature: t1:5203835:1496981602:c8a649158892d05442d8d0aa717589974b92ff6b
2017-06-13 03:38:54 -07:00
Stanislau Hlebik
0077886854 copytrace: do not log if file contexts were the same
Summary:
One one more improvement to logging to better measure the impact. Do not log if
file contents are the same as it is explained in the comment.

Depends on D5217007

Test Plan: arc unit

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: quark, mjpieters, medson, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5218112

Tasks: 18508761

Signature: t1:5218112:1497317521:e73e8ebe01a4a8ca73ad9fc87b5608d4c9f5484c
2017-06-13 01:57:07 -07:00
Stanislau Hlebik
cb9f3c9a22 copytrace: log file paths and hashes
Test Plan: arc unit

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters, medson

Differential Revision: https://phabricator.intern.facebook.com/D5217007

Tasks: 18508761

Signature: t1:5217007:1497020693:a42d44bf26ca991e3143f41b77148e3803c0267d
2017-06-12 02:49:20 -07:00
Stanislau Hlebik
84ab17a89a copytrace: exlude added files
Summary:
Previously if file was added in source then it would be considered missing in
destination. This behaviour won't add incorrect copies, but it would do
additional work of checking move candidates for the added file. This diff fixes
it.

Test Plan: arc unit

Reviewers: #mercurial

Subscribers: #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5190369

Tasks: 18508761
2017-06-07 03:40:54 -07:00
David Soria Parra
b0d59e858a p4fastimport: implement transaction.find and .replace
Summary:
revlog uses transaction.find and transaction.replace in cases where a
datafile is separated from an index (>= revlog._maxinline). In these cases the
importer broke as we did not implement find and replace in filetransaction.

We implement the two methods. However we ned a save way to store the data
portion which find must correclty return. We cannot store json as some
serializations aren't bidirecitonal. We can't store pickle as \n is a common
separator in pickle, and we use that to separate our entries. So we use pickle
and then base64 encode it. This is cumbersome but works.

The fix is aimed to make things "work" and I want to explore options to better
implement find and replace.

Test Plan:
- Working on a test that correclty triggers inline -> separation.
- Deployed it to our test system with test data and see commits comming in.

Reviewers: #idi, #ovrsource_warroom, #mercurial

Differential Revision: https://phabricator.intern.facebook.com/D5189308
2017-06-06 19:54:29 -07:00
Kostia Balytskyi
afad4c3c15 inhibitwarn: stop using strftime('%s') because it is wrong
Summary:
%s is not supported, works on Linux by accident, does not work on Windows.
https://stackoverflow.com/questions/11743019/convert-python-datetime-to-epoch-with-strftime

Test Plan: - run tests

Reviewers: #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5191450
2017-06-06 12:02:00 -07:00
Kostia Balytskyi
666b06e763 obsshelve: get rid of isobsshelve function and its uses
Summary:
Since it is now an independent extension, we don't need to have a knob
to enable/disable obs-based shelve, we can just enable/disable the entire
extension itself.

Test Plan: - run a test

Reviewers: #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5181001
2017-06-06 05:50:19 -07:00
Kostia Balytskyi
cc087222de obsshelve: make sure shelve and obsshelve cannot be enabled same time
Summary: Just for an early detection of bad config being distributed.

Test Plan: - add a test case

Reviewers: #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5180986
2017-06-06 05:48:28 -07:00
Kostia Balytskyi
222003b2ac obsshelve: copy the patched shelve.py from core
Summary:
Grand goal is to remove all of the shelve-related hotfixes and just have
a separate extension, obsshelve, which we could roll out to all FB users
without worrying about upstream.

This commit just copies current version of core's shelve.py with FB hotfixes
applied on top of it. The only changes are changes in testedwith and a
description at the top of the file and an import of rebase from hgext rather
than '.'.

Test Plan: - it is not enabled and tests are added in a later commit in this stack

Reviewers: #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5180947
2017-06-06 05:43:39 -07:00
Stanislau Hlebik
b03156c294 tweakdefault-blame: fix test
Upstream 135567a9d3e46a5686343e3a40f15482f7787a48 changed blame formatting.
This diff fixes tweakdefault to use this new formatting.
2017-06-05 14:56:36 -07:00
Stanislau Hlebik
0aa84957c7 copytrace: remove useless and add useful logging
Summary:
Previously we logged every time there is a changed/deleted file conflict.
This generates lots of useless log entries. Most of these entries are valid
conflicts where one file was deleted and another was added. Besides analyzing
these log entries is manual and very time consuming. Let's instead log cases
where copytracing actually helped.

Test Plan:
arc unit
Run with hg-crew test suite with --extra-config-opt

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5175656

Tasks: 18508761

Signature: t1:5175656:1496651487:fe710731213b21aefd1b957d259dc93470c9545c
2017-06-05 02:17:54 -07:00
Stanislau Hlebik
ce077253b5 copytracing: trace copies in draft commits
Summary:
Simple copy tracking for draft commits. It doesn't use db to save move info
because getting move info is quite fast already.

Depends on D5137372

Test Plan: Run tests

Reviewers: #mercurial, durham, quark, rmcelroy

Reviewed By: quark

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5137886

Tasks: 18508761

Signature: t1:5137886:1496348869:8f4a9761fdeb26ddee5e18a318cc85582ef4adbb
2017-06-02 01:27:13 -07:00
Jun Wu
767de0233b annotate: use keyword arguments for skiprevs and diffopts
Summary: This will be safer if upstream API ever changed again.

Test Plan: arc unit

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5147881

Signature: t1:5147881:1496326141:c70559ffe2bb43953a1a0584163ca24c3235c5f3
2017-06-01 08:58:00 -07:00
Stanislau Hlebik
9e8ca3ae43 copytrace: use filename heuristics to quickly find moves
Summary:
Copytracing that is based on a simple idea: most moves are either directory
moves or moves of the file inside the same directory. That means that either
basename of the moved file or the dirname of the moved file is the same.
More details in the comments.

Test Plan: Run unit-tests

Reviewers: #mercurial, durham, quark, rmcelroy

Reviewed By: quark, rmcelroy

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5137372

Tasks: 18508761

Signature: t1:5137372:1496243148:8d229c1593da196b674318ee8b37af15a60831c8
2017-06-01 04:39:27 -07:00
Adam Simpkins
1b9a5fdc5c tweakdefaults: fix hg pull --rebase with a --tool argument
Summary:
Fix the tweakdefaults code to extract the --tool argument when extracting
--rebase, and to pass it in to the rebasemodule.rebase() function correctly.

Test Plan: Included a new test.

Reviewers: #mercurial, rmcelroy, quark

Reviewed By: quark

Subscribers: quark, net-systems-diffs@fb.com, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5134466

Tasks: 18770938

Signature: t1:5134466:1495848690:308a8d3fbad39e6ecfeaee04b1b3267c1fa1118c
2017-05-31 10:58:49 -07:00
Stanislau Hlebik
72641e2afc copytrace: remove useless sorting
Summary: It was added by mistake, let's remove it.

Test Plan: arc unit

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Differential Revision: https://phabricator.intern.facebook.com/D5155301

Tasks: 18508761

Signature: t1:5155301:1496246623:a650cbffa7edf4311f96071e57c61697dae8014e
2017-05-31 09:04:59 -07:00
Stanislau Hlebik
699e45aef6 copytrace: better copytracing logging
Summary:
1) Log commit hashes. Since we have infinitepush we may see the actual commits that caused the problems.
2) Log reponame
3) Wrap logging code in try/catch so that any errors in this code won't affect clients

Test Plan: arc unit

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5146976

Tasks: 18508761

Signature: t1:5146976:1496241350:1d8a921527a0e2d984f6a46452697428df1757be
2017-05-31 07:51:29 -07:00
Kostia Balytskyi
44cd7a5717 smartlog: use dbm implementation, available on the system
Summary:
`gdbm` is unavailable on Windows. Python provides module called `anydbm` which
abstracts the implementation of `dbm` and is even capable of using a dump
Python-only implementation if nothing is installed.

Test Plan: - rt

Reviewers: #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5154620
2017-05-31 04:11:53 -07:00
Phil Cohen
45b775ae72 pushrebase: exchange -> bundle2
Summary: Broken by an upstream refactor

Test Plan: Fixes test

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5149611
2017-05-30 13:13:36 -07:00
Jun Wu
cc95aa58ff test-inhibit: fix the test
Summary: Fix lint errors and make the test pass.

Test Plan: arc unit

Reviewers: #sourcecontrol, sid0

Reviewed By: sid0

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5142477

Signature: t1:5142477:1496173824:bf941d895d3259fc788c09d4fefca7dddd11a33b
2017-05-30 12:56:20 -07:00
Jun Wu
e3e36e37ec inhibit: vendor inhibit and directaccess from mutable-history
Summary:
Vendored using GPL2 license from https://bitbucket.org/marmoute/mutable-history
changeset c9f1118b33d60f8faa4b89988a8155c544f5bb0d without modification.

Test Plan: N/A. The code was imported as is.

Reviewers: #sourcecontrol, sid0

Reviewed By: sid0

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5142063

Signature: t1:5142063:1496173803:d0a9fa50a8423a531432ac3cff378b50952684b1
2017-05-30 12:55:33 -07:00
Jun Wu
78c76613b9 annotate: add skiprevs parameter
Summary:
This adds the `skiprevs` parameter to all `filectx.annotate` methods
to match the upstream change.

For fastannotate, things is a bit more complex since it's a mix of two
algorithms. For now we just fallback to the slow path for correctness.

I'll think about adding back a fast path later.

Test Plan: arc unit

Reviewers: sid0, #mercurial, quark

Subscribers: stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5128392
2017-05-30 09:29:33 -07:00
Stanislau Hlebik
23e41d98e5 copytrace: move to hgext3rd
Summary:
copytracing extension is going to be simpler than I expected in the beginning.
There is no need to separate folder, let's move it to hgext3rd

Test Plan: arc unit

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5146585

Tasks: 18508761

Signature: t1:5146585:1496147733:ad6ad14f663a8b9bf4b687f0767395321f9840b7
2017-05-30 06:39:31 -07:00
Stanislau Hlebik
bbe014cb96 perftweaks: remove newhiddencache
Summary: Upstream dropped hidden cache in 2f38ac4af04e70af3ffce4700c5a15252391e391, let's drop newhiddencache too

Test Plan: arc unit

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: simonfar, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5146661

Signature: t1:5146661:1496140799:a5a4ddeac577f58a7e1998a5f9fb2eac305cc9d2
2017-05-30 03:42:56 -07:00
Jun Wu
3ed8d7b512 smartlog: add a simple ancestor cache
Summary:
`revlog.ancestor` is expensive on long changelog, but is only called a few
(about 30) times for smartlog usage. Therefore we could simply cache the
result in a key-value database.

This speeds up smartlog by about 200ms.

Test Plan: Added a new test

Reviewers: #mercurial, stash

Reviewed By: stash

Subscribers: stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5135746

Signature: t1:5135746:1495783503:411260551fd29fda42c9fc809b56f8f77fb2eaf4
2017-05-26 09:03:58 -07:00
Durham Goode
5caecec1ee sparse: switch to using repr for hash computation
Upstream is refactoring the matcher api (952017471f9) and recommends using repr
for hash computation now. This is how fsmonitor in core now does it.
2017-05-24 09:24:19 -07:00
Durham Goode
0d916abbed perftweaks: rename _getdynamicblocks to revealedrevs
Upstream has renamed this function.
2017-05-24 09:18:45 -07:00
Durham Goode
56be761d2e chistedit: fix lint error 2017-05-23 15:38:09 -07:00
Jun Wu
3b9d8cf876 traceprof: store raw Python code object
Summary:
Previously, the tracer extracts file name, function name from Python code
objects and stores them as std::string. And frame the de-duplication logic
hashes those strings, which is actually quite expensive.

`hg id` takes 1.0 seconds without frame de-duplication, and 2.5 seconds with
de-duplication.

This diff assumes Python code object is unique (i.e. Python won't generate two
code objects for a single code segment). That seems reasonable and `lsprof`
seems to make a same assumption.

With that assumption, just store `PyCodeObject` (contains file, method name and
line number) instead of `std::string` (for file, method names) and line numbers
for code identity and use the raw address of `PyCodeObject` to do frame
de-duplication.

That is helpful during profiling and de-duplication:

  - During profiling, no need to copy strings (convert to std::string),
    therefore less overhead
  - During frame de-duplication, no need to read long strings and hash them,
    therefore much faster

Debug flag during compilation is added to make debugging easier as the code
starts to mix of C++ and Python objects, which means mistakes could be made
more easily.

Test Plan:
With this patch, `hg id` with frame de-duplication takes 1.0 seconds.
There is no visible overhead doing de-duplication. Cheers!

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5112636

Signature: t1:5112636:1495562232:dd0724198ebb7f2eb4e9fd3a83517fc9160fa01e
2017-05-23 11:51:10 -07:00
Jun Wu
85be37794d traceprof: implement frame deduplication
Summary:
Previously, traceprof hashes frames using their raw memory addresses, which
causes a same frame (same traceback) being recorded multiple times because
of memory address change. That's for low overhead because hashing all the
way back to the top frame is slow.

This patch implements slow frame hashing and de-duplication so the output
could look tidier in some cases. The slow de-duplication runs after
profiling ends so it won't affect profiling accuracy.

Test Plan:
Trace IPython, before:

```
  6211          \ run                           posix.py:50
  6152            \ call_inputhook (10 times)   inputhook.py:60
                    ....
  6147              \ select_fds (10 times)     select.py:194
     6                \ select                  select.py:85
     6                 | select                 select.py:179
   501                \ select                  select.py:85
*  501                 | select                 select.py:179
  3775                \ select                  select.py:85
* 3775                 | select                 select.py:179
*  627                \ select (2 times)        select.py:85
   736                \ select (3 times)        select.py:85
*  736                 | select (6 times)       select.py:179
   501                \ select                  select.py:85
*  501                 | select (2 times)       select.py:179
```

After:

```
  4397          \ run                           posix.py:50
  4350            \ call_inputhook (4 times)    inputhook.py:60
  4349             | select_fds (4 times)       select.py:194
  4348             | select (4 times)           select.py:85
* 4348             | select (8 times)           select.py:179
```

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5112561

Signature: t1:5112561:1495562165:bf86c3283156d3b67918706dc875e6d3e28eac18
2017-05-23 11:50:33 -07:00
Jun Wu
c41d432bb1 fastannotate: move to hgext3rd
Summary: Extensions should belong there.

Test Plan: Updated existing tests

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5113146

Signature: t1:5113146:1495562344:47bae69e6b22c14b32a8a9511878e88e89759d8f
2017-05-23 11:47:22 -07:00
Jun Wu
a97e026674 remoteid: move to hgext3rd
Summary: This is what extensions should belong to.

Test Plan: Changed existing test.

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5113042

Signature: t1:5113042:1495561987:d1af0f7cebfdb8b0c1a6bb71cdc826d817a4e441
2017-05-23 11:47:04 -07:00
Martin von Zweigbergk
050cb5d1db chistedit: fail only when attempting to run if curses is not found
If curses is not available, we can still be nice and not fail until
the user actually tries to run the chistedit command.
2017-05-22 23:00:54 -07:00
Jun Wu
f1be725a71 traceprof: new extension to perform accurate profiling
Summary:
Instead of sampling periodically like the default profiler `statprof`, trace
every function call so the report is accurate. The output is designed to be
similar to `statprof`.

Written in C++ and inline assembly to minimize overhead. Cython is used as a
glue layer.

Comparing to `lsprof`, the overhead is similarly small. The major difference is
this profiler cares about "frame object" and records `frame->f_back` (parent
frame) so it could draw a statprof-like tree of the callgraph. `lsprof` records
"code object" instead and could have difficulty to deal with recursive calls.

The following code could demostrate the difference:

```
def fib(x):
    if x < 2:
        return x
    return fib(x - 1) + fib(x - 2)

# lsprof - recorded 1 code object for fib and no way to figure out callgraph
import _lsprof
p = _lsprof.Profiler()
p.enable(subcalls=True)
fib(5)
p.disable()
for s in p.getstats():
    print(s)

'''output:
_lsprof.profiler_entry(code="<method 'disable' of '_lsprof.Profiler' objects>", callcount=1, reccallcount=0, totaltime=0.0, inlinetime=0.0, calls=None)
_lsprof.profiler_entry(code=<code object fib at 0x7f27ca587530, file "a.py", line 1>, callcount=15, reccallcount=14, totaltime=8e-06, inlinetime=8e-06, calls=[_lsprof.profiler_subentry(code=<code object fib at 0x7f27ca587530, file "a.py", line 1>, callcount=14, reccallcount=12, totaltime=6e-06, inlinetime=6e-06)])
'''

# traceprof (with timethreshold set to 0) - callgraph drawn correctly
import sys
from hgext3rd import traceprof
with traceprof.profile(None, sys.stderr):
    fib(5)

'''output:
    | <module>                                  a.py:1
  0  \ fib                                      a.py:1
  0    \ fib                                    a.py:1
  0      \ fib                                  a.py:1
  0        \ fib                                a.py:1
  0         | fib                               a.py:1
  0        \ fib                                a.py:1
  0      \ fib                                  a.py:1
  0       | fib                                 a.py:1
  0    \ fib                                    a.py:1
  0     | fib                                   a.py:1
     \ __exit__                                 contextlib.py:21
'''
```

There are some performance / correctness tradeoffs. The current implementation
takes `frame` address and several (but not theoretically complete) fields to
hash a frame.  That could in theory lead to inaccurate frame information being
recorded. But hashing all fields recursively incurs significant overhead. The
related logic is the `hashframe` function, which could be tweaked later.

This is an initial version. It probably does not cover all corner cases like
exception handling well. And it may duplicate a same frame multiple times
because of the way it hashes a frame. The duplicated frame issue could be
solved by adding post-profiling filtering though.

Here is a real-world example of `hg log -r .` on `hg-committed`:

```
      | <module>                                hg:10
      | run                                     dispatch.py:81
      | dispatch                                dispatch.py:132
      | _runcatch                               dispatch.py:202
      | _callcatch                              dispatch.py:309
      | callcatch                               scmutil.py:133
      | _runcatchfunc                           dispatch.py:215
      | _dispatch                               dispatch.py:744
    2  \ repository                             hg.py:166
    2   | _peerorrepo                           hg.py:151
    2   | instance                              localrepo.py:2025
    2   | __init__                              localrepo.py:275
  863  \ runcommand                             dispatch.py:656
  863   | _runcommand                           dispatch.py:926
  863   | <lambda>                              dispatch.py:918
  863   | check                                 util.py:1059
  863   | log                                   commands.py:3262
  480    \ getlogrevs                           cmdutil.py:2236
  479     | _logrevs                            cmdutil.py:2190
  479     | revrange                            scmutil.py:429
  479     | anyrevs                             localrepo.py:622
  478     | mfunc                               revset.py:2275
  478     | __init__ (4 times)                  smartset.py:1033
  478     | __init__ (4 times)                  smartset.py:919
  478     | __len__ (4 times)                   localrepo.py:587
  478     | changelog                           repoview.py:309
  477     | filterrevs (11 times)               repoview.py:260
  477     | computehidden                       repoview.py:171
  455      \ hideablerevs                       repoview.py:24
  455       | getrevs                           obsolete.py:1114
  455       | _computeobsoleteset               obsolete.py:1143
    3        \ __get__ (2 times)                localrepo.py:76
    7        \ node (5760 times)                changelog.py:359
    4         | node (8938 times)               revlog.py:482
    9        \ __get__ (5760 times)             localrepo.py:76
  428        \ __get__                          util.py:798
  428         | successors                      obsolete.py:672
  225          \ __get__                        util.py:798
  224           | _all                          obsolete.py:662
    6            \ tryread                      vfs.py:32
    6             | read                        vfs.py:60
  200            \ _readmarkers                 obsolete.py:442
* 199             | _fm1readmarkers             obsolete.py:430
   15            \ _checkinvalidmarkers         obsolete.py:523
* 203          \ _addsuccessors                 obsolete.py:504
    3      \ tryreadcache                       repoview.py:166
    3       | cachehash                         repoview.py:98
    3       | heads                             localrepo.py:1884
   17      \ <genexpr> (5225 times)             repoview.py:191
   15       | __contains__ (5226 times)         ancestor.py:334
    5       | parentrevs (4010 times)           changelog.py:371
    2       | parentrevs (4332 times)           revlog.py:479
    9    \ pager                                ui.py:843
    9     | _runpager                           ui.py:906
    8     | __init__                            subprocess.py:330
    8     | _execute_child                      subprocess.py:880
    5     | _eintr_retry_call                   subprocess.py:118
  374    \ show                                 cmdutil.py:1308
  374     | _show                               cmdutil.py:1316
  365      \ _changesetlabels                   cmdutil.py:1266
  365       | troubled                          context.py:225
   12        \ unstable (2 times)               context.py:207
   12         | getrevs                         obsolete.py:1114
   12         | _computeunstableset             obsolete.py:1154
    3          \ set (322 times)                localrepo.py:610
    3            \ revs (3 times)               localrepo.py:593
    8            \ __getitem__ (963 times)      localrepo.py:566
    4          \ parents (321 times)            context.py:246
    4           | __get__ (321 times)           util.py:798
    4           | _parents (321 times)          context.py:562
    3          \ <genexpr> (642 times)          obsolete.py:1164
    3           | obsolete (321 times)          context.py:199
    2           | getrevs (321 times)           obsolete.py:1114
  319        \ bumped (2 times)                 context.py:211
  319         | getrevs                         obsolete.py:1114
  319         | _computebumpedset               obsolete.py:1181
    4          \ set (322 times)                localrepo.py:610
  312          \ allprecursors (2021 times)     obsolete.py:850
  308           | __get__                       util.py:798
  308           | precursors                    obsolete.py:678
* 308           | _addprecursors                obsolete.py:509
   34        \ divergent (2 times)              context.py:218
   34         | getrevs                         obsolete.py:1114
   33         | _computedivergentset            obsolete.py:1204
    4          \ set (322 times)                localrepo.py:610
   24          \ successorssets (1368 times)    obsolete.py:899
    3            \ __get__ (1368 times)         localrepo.py:76
    4            \ __contains__ (200 times)     localrepo.py:575
    9      \ names (2 times)                    namespaces.py:184
    9       | <lambda>                          namespaces.py:43
    9       | nodetags                          localrepo.py:741
    8       | __get__                           util.py:798
    8       | _tagscache                        localrepo.py:646
    8       | _findtags                         localrepo.py:685
    7       | findglobaltags                    tags.py:170
    7       | _readtagcache                     tags.py:370
    6       | filteredhash                      scmutil.py:307
       \ __exit__                               contextlib.py:21
        | maybeprofile                          profiling.py:199
        | __exit__                              contextlib.py:21
        | profile                               profiling.py:148
        | __exit__                              contextlib.py:21
Total time: 867 ms
```

Some example conclusions from reading the above output include:

  1. `_fm1readmarkers`, `_addsuccessors`, `_addprecursors` are slow
  2. `changelog.node` and `changelog.parentrevs` are called thousands of times,
     but they are just a few milliseconds.
  3. Reading the `obsstore` file (`vfs.read`) takes 5ms exactly

While `statprof` may get 1 right, `lsprof` may get 2 right, neither of them is
helpful for 3. `statprof` is not accurate enough to be confident about 5ms,
`lsprof` will include other `vfs.read` calls so it's impossible to see just the
read for obsstore.


Test Plan:
With the upstream [patch](https://patchwork.mercurial-scm.org/patch/20822/), run:

```
 HGPROF=traceprof hg id --config extensions.traceprof= --profile
```

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters, jiangyuan

Differential Revision: https://phabricator.intern.facebook.com/D5104851

Signature: t1:5104851:1495480386:666847ea08bb6a94658bc10d7f0e91c225d56836
2017-05-22 15:49:45 -07:00
Durham Goode
dcaee39e39 obsmarkers: update obsmarker operation wrapping to work with upstream
Summary:
Upstream has added an optional operation argument to createmarkers, which we
need to also accept. This argument is currently only processed if a config
option is set, so for now we will still put the operation in the metadata dict
manually.

Test Plan: Ran the tests

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5106511

Signature: t1:5106511:1495485044:23d45d9ba389cf0d49b31705ae36902310c5056c
2017-05-22 13:38:37 -07:00
Durham Goode
0766a3469c dirstate: switch to new parentchange context
Summary: Upstream has deprecated begin/endparentchange in favor of a context object.

Test Plan: Ran the tests

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5106500

Signature: t1:5106500:1495485058:8377335a0f678de7bb50cb93fff29bcfeb53fe1d
2017-05-22 13:38:37 -07:00
Durham Goode
e34660b057 commands: update to use registrar instead of cmdutil
Summary: Upstream has deprecated cmdutil.commands() in favor of registrar.commands()

Test Plan: Ran the tests

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5106486

Signature: t1:5106486:1495485074:0e20f00622cc651e8c9dda837f84dd84cc51099e
2017-05-22 13:38:37 -07:00
Jun Wu
f532dddaa4 lfs: implement byte-level progress bar
Summary:
Previously, the progress bar is file-level - it moves when a file is transferred.

When uploading or downloading a single giant file, progress bar matters. And
this diff adds it to make people more patient.

Test Plan:
Manually upload and download several big files.
Make sure the progress bar appears in both cases.

Also make sure the output is sane with `-v`.

Reviewers: #mercurial, davidsp, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5100014

Signature: t1:5100014:1495316719:c500ccd63d3a495e41575cadd72419a41f5fb259
2017-05-22 11:03:14 -07:00
Durham Goode
f02e3e0686 revlog: rename revlog constants
Upstream has renamed some revlog constants, so we need to update our code
accordingly.
2017-05-21 17:09:08 -07:00
Jun Wu
65b1cff91e dirsync: use _fileset
Upstream has renamed _fileroots to _fileset.
2017-05-19 21:39:47 -07:00
Jun Wu
0fd94f75db fixcorrupt: use REVLOGV1
Upstream has renamed REVLOGNG to REVLOGV1.
2017-05-19 21:32:02 -07:00
Jun Wu
3e43e09a97 lfs: add a lfs_files template
Summary:
This allows automation to know which modified or added files are LFS for
specific changesets.

Test Plan: Added a test

Reviewers: #mercurial, davidsp

Reviewed By: davidsp

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5091591

Signature: t1:5091591:1495163606:138638d8ccc57b8ed6c1e324750ec5dc15c198a0
2017-05-18 22:18:20 -07:00
Jun Wu
546cff40f8 lfs: skip uploading when remotestore is a null store
Summary:
When remotestore is a null store, uploading is a no-op and can be skipped.
This diff makes it so.

Test Plan: arc unit

Reviewers: #mercurial, davidsp

Reviewed By: davidsp

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5090686

Signature: t1:5090686:1495144964:956549307912f4a44201aa305959bf8de26ac028
2017-05-18 15:10:39 -07:00
Misha Shneerson
cbdaa23f88 conduit: added correct Content-Type header for conduit queries
Summary:
conduit HTTP POST request did not have correct content-type header set.
Now they do. Ideally we should use python-request package to do HTTP
but I do not know enough how to import these dependencies so I just went
with the minimal changes.

Test Plan:
ran hg log from dev environment and eventually got it all working.
{P57411895}

Reviewers: simpkins

Reviewed By: simpkins

Subscribers: net-systems-diffs@fb.com, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5080725

Tasks: 18294945

Signature: t1:5080725:1495052201:1bbd6edad4c1c4b3482a7479259460a815947630
2017-05-17 13:50:37 -07:00
Rainer Alves
37fdd68496 morestatus: make conflict paths relative
Summary: morestatus: make conflict paths relative

Test Plan: Output: https://phabricator.intern.facebook.com/P57402806

Reviewers: rmcelroy, mgorven

Differential Revision: https://phabricator.intern.facebook.com/D5062074

Tasks: 10366065
2017-05-17 07:13:41 -07:00
Jun Wu
24d8640017 absorb: do not strip innocent children
Summary:
`repair.strip` removes required revisions and their children unconditionally.
That means innocent children may be removed. This patch fixes it.

We may want to have an in-core function for this feature to not strip innocent
children.

Thanks Zibi Braniecki from Mozilla for reporting [1]!

[1]: https://bitbucket.org/facebook/hg-experimental/issues/6/hg-absorb-merges-diverged-commits

Test Plan: Added a test

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5073342

Signature: t1:5073342:1494972231:ca2a9881415119047771c8f35e059f4bfb1749e5
2017-05-16 16:25:00 -07:00
Jun Wu
a82574317b lfs: upload blobs during 'hg bundle'
Summary:
When a bundle is created, the bundle could be exchanged via copy and possibly
eventually reach to a publishing repo. If we don't upload LFS blobs, hg server
could have revisions that can never be checked out or verified.

So let's just assume bundles generated by `hg bundle` will be public, and
upload LFS blobs automatically, without depending on other code review tooling.

Note: there is a `preoutgoing` hook which will be triggered in this case,
however it's not useful since it does not have the `outgoing` information.

Test Plan: Will add a new test

Reviewers: davidsp, #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5050945

Signature: t1:5050945:1494958648:0c46ab1d85755838ba189bbb1e0673882922bb58
2017-05-16 15:49:32 -07:00
Jun Wu
7098094e7e lfs: cleanup file headers
Summary:
This diff removes `# coding=UTF-8` and adds standard GPL2 headers to lfs
files.

Test Plan: arc unit

Reviewers: davidsp, #mercurial, simonfar, rmcelroy

Reviewed By: simonfar, rmcelroy

Subscribers: simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5067881

Signature: t1:5067881:1494947941:e669d4d9485c390138f76031608a7f727c57bf55
2017-05-16 15:44:06 -07:00
Jun Wu
f0c7a3cef9 lfs: remove util.py
Summary:
`util.py` only contains 2 small things: `lfsvfs` and `sha256`. Both of them
only have one user (in terms of files). Therefore just move the code to
related files to make it simpler.

Test Plan: arc unit

Reviewers: davidsp, #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5067862

Signature: t1:5067862:1494955824:b99021f1bb44568dc0c738b0e472eb732f2bc91a
2017-05-16 15:43:36 -07:00
Jun Wu
ae8a1ccb8f lfs: update old PointerDeserializationError error handling
Summary:
The error was renamed to `InvalidPointer`. This diff updates error handling
to catch that instead.

Test Plan: arc unit

Reviewers: davidsp, #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5067496

Signature: t1:5067496:1494955510:5f0162a8cc6f1d0d83e3ab6319ec3202028684d8
2017-05-16 15:42:30 -07:00
Jun Wu
acfcc44094 lfs: add a fctx.islfs method
Summary:
This simplifies code a bit, and could be useful for 3rd party code. We already
have `isbinary`, `islink`, `isexec`, `isabsent`. So another `is` method looks
fine.

Test Plan: Run existing tests

Reviewers: davidsp, #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5067369

Signature: t1:5067369:1494955466:ae17e310e743c704cc0d9bb73e0d2e82adcffff8
2017-05-16 15:41:39 -07:00
Jun Wu
2eef188bc0 lfs: add a filectx.cmp fast path
Summary:
The `filectx.cmp` fast path allows us to show "binary file changed" diff output
when two binaries have the same size but different content.

Test Plan: Added a test case

Reviewers: davidsp, #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5067266

Signature: t1:5067266:1494954783:a41e8213782afe24618ef2930c9576f21610fd3e
2017-05-16 15:38:51 -07:00
Jun Wu
9eedb98b61 lfs: downgrade "computing set of blobs to upload" to debug message
Summary:
Calculating what blobs to upload should be fast since we use changelog `files`
information. Users will see `lfs: uploading ...` very soon. So let's downgrade
`lfs: computing set of blobs to upload` to a debug message to make `-v`
cleaner.

Test Plan: Updated existing test.

Reviewers: davidsp, #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5066600

Signature: t1:5066600:1494954115:c00925a1930f9b53e914078d32b6c4e4161099ee
2017-05-16 15:27:25 -07:00
Jun Wu
ae541c9f80 lfs: show status per object when ui.verbose is set
Summary:
Downloading or uploading a single LFS object could take long. Instead of
showing a summary of what objects are downloaded or uploaded, print a message
per object if --verbose is provided.

Test Plan: Added a test

Reviewers: davidsp, #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5066327

Signature: t1:5066327:1494953894:3593436ff53eaff117a2932476dfc4b867fb9f17
2017-05-16 15:24:04 -07:00
Jun Wu
28de141267 lfs: handle "wrong action" error earlier
Summary:
Previously, when an object is not found server-side, we error out during the
basic transfer API. This diffs move the check to the batch metadata API so the
check is preformed earlier.

That also means, when batch fetching many LFS objects, we could report multiple
objects being missing instead of just one.

Test Plan: Modified an existing test

Reviewers: davidsp, #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5065623

Signature: t1:5065623:1494952131:cb1df73e1cea21e07bfd0ec630bacec60e5b385c
2017-05-16 15:19:40 -07:00
David Soria Parra
aca5b8465c p4fastimport: filelog transaction wtih support for concurrent access
Summary:
We are adding a simple transaction that works only with filelogs but
allows for concurrent access from multiple workers. This allows for a proper
rollback in case of a failure in a worker process, which previously would result
in bad data in the repositoriy.

Test Plan: rt test-p4* test-check*

Reviewers: #mercurial, durham, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5070340

Signature: t1:5070340:1494958313:b10b1eac5b42b36d1a587c4ae1c95fc2f8b5ad35
2017-05-16 12:36:23 -07:00
Jun Wu
b78dbc3dad lfs: check object errors from batch API early
Summary:
The batch API allows server to return errors per object, handling them early
seems to be a good idea.

This is an attempt to distinguish "cannot upload - no permission" from
"no need to upload - already exist".

Unfortunately, `lfs-test-server` reference implementation does not provide
the error message so it's not reflected in tests.

Reviewers: #mercurial, davidsp, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5057816

Signature: t1:5057816:1494951879:07f2b67408e9d77255104b932baae2e50af9890b
2017-05-16 12:30:23 -07:00
Jun Wu
adb61e1976 lfs: split basic transfer API out from _batch
Summary:
The basic transfer API [1] worths a separate method.

[1]: https://github.com/git-lfs/git-lfs/blob/master/docs/api/basic-transfers.md

Reviewers: #mercurial, davidsp, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5057683

Signature: t1:5057683:1494951784:de3d3a28a69366fbbc4546f9e68b4d664ba00dfa
2017-05-16 12:26:30 -07:00
Jun Wu
d5a9c91ff9 lfs: move batch API to a separate method
Summary:
The LFS uploading or downloading process consists of 2 parts: a batch API to
get metadata about objects, followed by `len(objects)` requests.

This diff moves the first metadata API out from a giant `_batch` method to
make the code structure easier to maintain. Later diffs will further split
`_batch` method.

A side effect is `lfs: mapping blobs to #{action} URLs` message is removed.
More user-friendly message will be added back in a later patch.

Reviewers: #mercurial, davidsp, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5057541

Signature: t1:5057541:1494949989:d510c3d9369744e7ee776c0d0a26d263dfaf8352
2017-05-16 12:24:20 -07:00
Jun Wu
31882a9791 lfs: remove total parameters from remote store APIs
Summary:
Displaying total bytes to upload is currently inaccurate - the server could
already have some objects so only a subset of selected objects will be
uploaded.

Besides, we pass `pointers` to related upload and download APIs so `total`
could be calculated from them.

This diff removes the inaccurate "need to upload" message and unnecessary
parameters. An accurate message will be added in a later patch.

Reviewers: #mercurial, davidsp, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5057470

Signature: t1:5057470:1494949554:07e637a8b58c894598b3f0c2dd53f80a5ade25a8
2017-05-16 11:19:39 -07:00
Don Stewart
bc85e8a4cf hgext: accept a --unified param for lines of diff context
Summary: Added -U/--unified NUM to fbshow.py, and used that to set diff.unified=NUM.

Test Plan: hg show --help; hg show -U 1; hg show -U 2; hg show -U by induction.

Reviewers: mburman, simonfar, ikostia

Differential Revision: https://phabricator.intern.facebook.com/D5052156

Tasks: 10006036

Tags: python
2017-05-15 05:59:55 -07:00
Jun Wu
f484dba4a0 lfs: narrow down try block
Summary:
A try block is only effective for the HTTP error. Let's narrow it down to
just the HTTP request.

Test Plan: arc unit

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5057107

Signature: t1:5057107:1494634308:584877babc6a4705fa0740788dcf58affdf51bec
2017-05-12 17:28:45 -07:00
Jun Wu
853b228a8a lfs: handle batch API's HTTP and JSON error
Summary:
There are `1+n` HTTP requests. `n` of them is handled. But the first one
about JSON metadata is not. Let's handle it.

Test Plan: arc unit

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5057053

Signature: t1:5057053:1494634184:fa0e562dc7879379106bf37915a2e5ea3cc01a4d
2017-05-12 17:26:06 -07:00
Jun Wu
d5e501b573 lfs: provide more detailed message on network error
Summary:
The server or the HTTP library may provide more detailed error message.
Let's use them. `RequestFailedError` is also renamed to `LfsRemoteError` to
make it clear it's related to LFS and network.

Test Plan: arc unit

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5057014

Signature: t1:5057014:1494634052:00c1cb1b337a0e4cb92828cd49e95b7b0ff7e540
2017-05-12 17:22:02 -07:00
Jun Wu
b808e0f70f lfs: remove UnavailableBatchOperationError
Summary:
It's caught, and re-raise with RequestFailedError. We can just raise
RequestFailedError directly.

Test Plan: arc unit

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5056886

Signature: t1:5056886:1494633307:3fdf2fab13e783224d343266a1278c41ed4394e8
2017-05-12 17:18:05 -07:00
Jun Wu
34ffa6d4ce lfs: use ProgrammingError for invalid action in _batch
Summary:
_batch is called by programmers. If action is not expected, it's a
ProgrammingError.

Test Plan: arc unit

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5056822

Signature: t1:5056822:1494634121:59737eb0d3f42d706ee8b0dc18ec7a4b49ada34f
2017-05-12 17:14:52 -07:00
David Soria Parra
07292913da p4fastimport: move p4fastimport under hgext3rd
Summary:
move p4fastimport under hgext3rd as that's where it belongs.
Also has the benefit that we package it up.

Test Plan: rt test-p4*

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5056664

Signature: t1:5056664:1494631758:4892f47922d8fbbd2c1f7793f3f29ee73fa8fa42
2017-05-12 17:03:20 -07:00
Jun Wu
d3b28f4b42 lfs: remove StoreID
Summary:
For long I have been wondering - we have `oid` already and an `oid` could
identify an object already, why do we need a `StoreID` which contains both
`oid` and `size`? It does not seem necessary?

I figured out the answer - the batch download or upload API *requires* size
information per object [1].

Since we will have LFS pointers during upload or download, we can just use
pointers. This diff removes `StoreID`, changes remote blob store to take
`pointers` directly, and local blob store to take `oid` directly.

As a side effect, `debuglfsupload` no longer takes `-o`, which seems fine
because `-r` should be more useful.

[1] https://github.com/git-lfs/git-lfs/blob/master/docs/api/batch.md

Test Plan: `arc unit`

Reviewers: davidsp, #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy

Differential Revision: https://phabricator.intern.facebook.com/D5043694

Signature: t1:5043694:1494496522:ea10f3dd12fc2ae3ab77b8f623eeead84dcc25fa
2017-05-11 17:57:42 -07:00
Jun Wu
8641f3cc37 lfs: rename GithubPointer to gitlfspointer
Summary:
It's Git-LFS, not GitHub.
Besides, mercurial uses lowercase for class names.

Test Plan: Run existing tests

Reviewers: davidsp, #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5043570

Signature: t1:5043570:1494493515:1df24d12d8b9be51480909ef75cad56cd9e22a2e
2017-05-11 17:54:33 -07:00
Jun Wu
a024f96812 lfs: add validation to pointer
Summary:
Read the Git-LFS specification [1] and implement most checks to prevent
programming error and detect data corruption.

The new code should be stronger than what was before the refactoring, since the
old code only checks keys but not values.

[1]: https://github.com/git-lfs/git-lfs/blob/master/docs/spec.md

Test Plan: Added a new test

Reviewers: #mercurial, davidsp, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5046740

Signature: t1:5046740:1494543072:2f0dbfc07cd52a10572550a277d850bdf3f78d27
2017-05-11 17:51:44 -07:00
Jun Wu
143b8c67d9 lfs: rewrite pointer logic
Summary:
Since we decided to only support GitHub's Git LFS specification, there is no
need to support multiple pointer types, so the code could be simplified.

The old code special case keys like `version`, `oid`, `hashalgo`, which makes
it longer than necessary. The new code is a rewrite treating everything as a
normal dict entry so the pointer class is much shorter: 76 -> 21 lines.

Data validation is temporary lost, which will be added back (and stronger) by
the next diff. It is separated to make review easier.


Test Plan: Run existing tests

Reviewers: #mercurial, davidsp, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy

Differential Revision: https://phabricator.intern.facebook.com/D5043547

Signature: t1:5043547:1494543031:ac1100939a10a79dfd749ab6ac9c3bb7fcd84dbf
2017-05-11 17:48:21 -07:00
Rodrigo Damazio Bovendorp
641bb65374 githelp: making the unknown message footer configurable
Facebook probably doesn't want external users reaching out to them directly
for support, so this should be customizable to specify other possible courses
of action (especially in cases where githelp is further extended to document
git-related company-internal commands in other companies).
2017-05-11 07:19:32 -07:00
Jun Wu
ec852f8f5f lfs: disable lfs code path if filenode is None
Summary:
`filelog.filenode` could be `None` when it's unknown - like working copy. That
breaks `isbinary` test. Let's modify `_islfs` check to return False to disable
lfs code path in that case.

Test Plan: Added a new test.

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5034058

Signature: t1:5034058:1494431790:a02b69addf5d2f9b6b9a27ad71ed4f136b2bfd2b
2017-05-10 13:28:39 -07:00
Jun Wu
ee34d8ca1e lfs: remove lfs.bypass config option
Summary:
The config option was designed to be used server-side to avoid accessing remote
LFS blob stores. However, we now have `lfs.url=null:` as a clean alternative,
which will explicitly raise if the server ever tries to download remote
content. So `bypass` is no longer needed.

`bypass` could also be used for displaying raw content. That is doable using
`debugdata`.

Test Plan: Modified existing tests

Reviewers: #mercurial, rmcelroy, davidsp

Reviewed By: davidsp

Subscribers: davidsp, rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5009725

Signature: t1:5009725:1494260384:332d7bd658359c004342efd01e4f13b6fc5499b1
2017-05-08 11:22:01 -07:00
Jun Wu
8909f8ded8 lfs: remove lfs.blobstore option
Summary:
`lfs.blobstore` is a path used to store local blobs. Some of the blobs are not
uploaded yet so they couldn't be easily discarded. Although blobs downloaded
from the server could be removed if necessary.

It does not make much sense to make this a config option - the data should be
stored reliably. It's also dangerous to allow write to arbitrary paths under
`repo.vfs`.

This diff makes the local blob store fixed path at `.hg/store/lfs/objects`.
The choice is similar to `.git/lfs/objects`, but with `store`, shared repo
could share their lfs stores.

Test Plan: Modified existing test.

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5009664

Signature: t1:5009664:1493978440:c3f6351d0ea0cca2ce3caa9f7260c5d65bcc0e5f
2017-05-08 11:21:34 -07:00
Jun Wu
186df9d185 lfs: add a command to upload lfs blobs
Summary: This makes it possible to upload lfs blobs without going through push.

Test Plan: The feature will be used in the next change.

Reviewers: #mercurial, davidsp, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5009622

Signature: t1:5009622:1494230385:0df33ed12c96bc91f6252398ce5091057fe32a21
2017-05-08 11:21:11 -07:00
Jun Wu
4532efb04d lfs: split prepush hook into individual functions
Summary:
This makes it possible to reuse part of them - like uploading blobs for
given revisions without going through prepush hook.

`pointer.tostoreids()` was changed to `pointer.tostoreid()` to simplify
things a bit.

Unnecessary remoterepo assignment was removed.

Test Plan: `arc unit`

Reviewers: #mercurial, davidsp, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5009560

Signature: t1:5009560:1494230285:6469a2701baa8cfa4511a08149a37fc429733343
2017-05-08 11:20:50 -07:00
Jun Wu
3793eee754 lfs: remove 40 char length limit
Summary:
The `lfs-test-server` reference implementation [1] validates the hash, and
40-byte sha256 hexdigest will not pass that check.

[1]: https://github.com/git-lfs/lfs-test-server

Test Plan: A test will be added in the next diff.

Reviewers: #mercurial, davidsp

Reviewed By: davidsp

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5009319

Signature: t1:5009319:1494006078:32950c5490935d9786553fc07d01f1fc92aacf25
2017-05-08 11:12:11 -07:00
Jun Wu
3f004c2689 lfs: ensure storeid.size is an integer
Summary:
The request JSON data used in the batch API should have `size` field as an
integer, not a string. This was discovered when testing against GitHub's
`lfs-test-server` implementation [1]. The latter was written in Golang and has
strong type requirement, it will treat `{"size": "42"}` as `{"size": 0}`.

[1]: https://github.com/git-lfs/lfs-test-server

Test Plan:
An integration test with `lfs-test-server` will be added once remaining issues
are resolved.

Reviewers: #mercurial, ikostia, davidsp

Reviewed By: ikostia, davidsp

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5009248

Signature: t1:5009248:1493977299:5daf8d32cd8c933be71a41afcc7ff832eb7edb5a
2017-05-08 11:11:47 -07:00
Jun Wu
fd5d8b9996 lfs: simplify blobstore config options
Summary:
This diff simplifies lfs remote server configs to a single item: `url`,
similar to what git-lfs has.

Compare:

```
  Before                        | After
 -------------------------------+-----------------------------------
  remoteurl = http://a.com/lfs  | url = http://foo:pass@a.com/lfs
  remoteuser = foo              |
  remotepassword = pass         |
  remotestore = git-lfs         |
  ------------------------------+-----------------------------------
  remotepath = /tmp/lfs-test    | url = file:///tmp/lfs-test
  remotestore = dummy           |
```

Test Plan: Modified existing cases.

Reviewers: #mercurial, davidsp, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5008882

Signature: t1:5008882:1494230125:822b0e92f45dff2a37e26e6b3e44b559b4a47e6d
2017-05-08 11:11:14 -07:00
Jun Wu
2baf329dc2 tweakdefaults: let commit -M support revset
Summary: This will allow `-M` to have complex queries like `REV^'.

Test Plan: Test `scmutil.revsingle` in debugshell.

Reviewers: #mercurial, ikostia, rmcelroy

Reviewed By: ikostia, rmcelroy

Subscribers: ikostia, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5008983

Signature: t1:5008983:1493977159:7e105be95330b498d25033565be8594789406cae
2017-05-06 10:26:20 -07:00
David Soria Parra
cd7eed4dd0 lfs: ensure svfs is set
Summary:
Some remotestores (in particular null) do not have a svfs.
Check for it before we set it.

Test Plan:
Used a nullstore with a threshold set during push and saw it
now worked.

Reviewers: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4984664
2017-05-04 16:18:32 -07:00
Jun Wu
64daf52ec8 lfs: use lfsvfs in dummy store
Summary: This simplifies code a lot.

Test Plan: Updated existing test

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5001837

Signature: t1:5001837:1493885337:eea8b5cba7234453b32eba46fd58b4d5904367bf
2017-05-04 09:25:20 -07:00
Jun Wu
fb4cd9891d lfs: remove chunking feature
Summary:
Per discussion with @davidsp, we want to stick to Git-LFS specification and
avoid non-standard behavior. The chunking behavior will happen at LFS server
transparently.

The direct motivation for this is to make it possible to implement an
efficient `filectx.cmp` that just compares hashes.

Test Plan: Updated existing test

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: mjpieters, davidsp

Differential Revision: https://phabricator.intern.facebook.com/D5001827

Signature: t1:5001827:1493914639:c58694873e79a8ca910bb8ee01bf593885896664
2017-05-04 09:20:34 -07:00
Jun Wu
3d56581781 lfs: store isbinary information in LFS metadata
Summary:
Usually LFS files are binary files. But there could be exceptions. This diff
adds a new customized field `x-is-binary` to record those exceptions.

The `filectx.isbinary` API is changed to use that metadata as a fast path.

This allows us to provide a transparent user experience (whether a file is
stored in LFS or not does not affect its original `isbinary` property),
while still being able to skip loading the LFS blob if the file is binary.

Test Plan: Added a new test case

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5001117

Signature: t1:5001117:1493892406:2a8ebd42d0ae0dbae39b87b9ea16db22b99f3d12
2017-05-04 09:18:04 -07:00
Martijn Pieters
372c6c955f sparse: restore sparse profile on exception during import
Summary: Like _config, _import needs to restore the pre-existing profile if refreshing failed. Otherwise we have marked new rules as imported and applying without that actually being true.

Test Plan: rt test-sparse*

Reviewers: #sourcecontrol, rmcelroy, durham, simonfar

Reviewed By: simonfar

Subscribers: simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4985415

Tasks: 17793415

Tags: autosparse

Signature: t1:4985415:1493893462:ee4bcbf558706c3d2f58367dd7bf2533ccf1b824
2017-05-04 04:58:43 -07:00
Jun Wu
4131d4ebcd pushrebase: fix a typo in comment 2017-05-03 16:19:11 -07:00
Jun Wu
d82b8b1495 pushrebase: do not require exchange for getting markers
Summary:
D4865150 and D4934720 aren't effective in our current setup. The direct
cause in the code is because the server couldn't find common marker version:

```
  # old server-side code, returns empty in our current setup
  obsolete.commonversion(bundle2.obsmarkersversion(reply.capabilities))
```

Upon investigation, it's because there is no `exchange` enabled client-side.

But we do want one-way (server->client) markers for the rebased commits, as
long as obsstore is enabled (createmarkers is set, without exchange).

The upstream expects the server to have obsstore enabled, and exchange
enabled, to send markers. Since we are generating markers without an
obsstore (see D4865150), we are on our own way. This diff makes it one step
further.

This diff adds an explicit parameter to the `b2x:rebase` part to tell the
server what obsmarker format the client supports so the server could make a
right decision without relying on the "standard" `reply.capabilities`, which
is affected by the exchange option.

Test Plan: Change the existing test, make sure the old code fails.

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: durham, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4997972

Signature: t1:4997972:1493848751:14c29654b2e8246bd12a8de8820af5b3773e2fb7
2017-05-03 16:08:35 -07:00
Jun Wu
d3f8c4e9dc lfs: override filelog.renamed code path
Summary:
An `hg pull` test triggers the following code path server-side when using
remotefilelog:

  ...
  remote:   File "/usr/lib64/python2.7/site-packages/remotefilelog/remotefilelogserver.py", line 308, in streamer
  remote:     text = _loadfileblob(repo, cachepath, path, node)
  remote:   File "/usr/lib64/python2.7/site-packages/remotefilelog/remotefilelogserver.py", line 223, in _loadfileblob
  remote:     text = createfileblob(filectx)
  remote:   File "/usr/lib64/python2.7/site-packages/remotefilelog/remotefilelogserver.py", line 348, in createfileblob
  remote:     ancestors.extend([f for f in filectx.ancestors()])
  remote:   File "/usr/lib64/python2.7/site-packages/mercurial/context.py", line 1072, in ancestors
  remote:     for parent in c.parents()[:cut]:
  remote:   File "/usr/lib64/python2.7/site-packages/mercurial/context.py", line 923, in parents
  remote:     r = fl.renamed(self._filenode)
  remote:   File "/usr/lib64/python2.7/site-packages/mercurial/filelog.py", line 62, in renamed
  remote:     t = self.revision(node)
  ...

That triggers downloading a blob. We don't want to do that server-side. So
override the `renamed` method to use LFS fast path to test rename.

Practically, this reverts part of D4906074.

Test Plan: Run existing tests. This change was made on the server.

Reviewers: davidsp, #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4992421

Signature: t1:4992421:1493802628:2bf2cf819bfed2aa61ea1c2323c03ab428732815
2017-05-03 11:10:48 -07:00
Augie Fackler
f198dbbc1e morestatus: add config knob to allow some states to be skipped
The main rationale for this change is that I've got years of use
patterns on bisect where I *start* bisect by running 'hg bisect
--reset', not *finish* with that command, so `hg status` with
morestatus enabled is spewing annoyances in nearly every repository
I've ever used. It wasn't any harder to allow people to ignore
anything they want, but (at least for now) bisect is the only state
that seems "special" in that it doesn't have a definite "end" that
Mercurial can detect and automatically clean up. I'd also be fine with
adding something like morestatus.ignorebisect if we want to avoid a
potential footgun where a user could ignore rebase state.
2017-05-03 13:34:36 -04:00
Jun Wu
4b467776ad pushrebase: accept arbitary keywords for _peerorrepo
Summary:
stateful chg will add a new parameter to repo initialization. This diff makes
sure pushrebase works with that change.

Test Plan: codereview

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: durham, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4978687

Signature: t1:4978687:1493689190:7119205d76a8ac26dbf2bcf0be9c74ced9fc16b4
2017-05-01 19:11:08 -07:00
David Soria Parra
f8bb4e2122 lfs: add a 'null' store that doesn't require any setup
Summary:
We have to use the dummy store to ensure we don't try to open an HTTP
connection during push, which can fail and must not happen during p4fastimport.
However `dummy` requires some configuration. So just add an internal `null`
store that allows us to operate LFS in a mode that will ignore all large files
completely.

Test Plan: used it in p4fastimport

Reviewers: #sourcecontrol, quark

Reviewed By: quark

Subscribers: quark, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4963638

Signature: t1:4963638:1493316573:c012bd97794f9a57c3cf8c15d868a67ae3c03c31
2017-04-27 11:13:54 -07:00
Martijn Pieters
05d97a9461 sparse: provide a change summary on file and rule count deltas
Summary:
This allows a third-party tool like Buck report more detail on what a sparse
command actually achieved.

Test Plan: Run the test suite.

Reviewers: #sourcecontrol, durham, rmcelroy, ikostia

Reviewed By: ikostia

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4962550

Tasks: 17150508

Tags: autosparse

Signature: t1:4962550:1493307403:d14e16d079121688a4eb69f0d69cd02ab55c4936
2017-04-27 08:37:13 -07:00
Jun Wu
0d6151f530 remotefilelog: add lfs integration test
Summary:
The test covers common workflows like clone, commit, push, update, pull. It
exercises the remotefilelog plain store and Python datapack store to make sure
they won't lose the revlog flag. The test also tries to verify rename works
correctly.

Since the lfs extension may be eventually upstreamed, it seems a good idea to
make remotefilelog call `lfs.wrapfilelog` so lfs is free from remotefilelog
code.

Test Plan: Added a test

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4904281

Signature: t1:4904281:1492560308:5fd9f214ada6de795735ea7d737d30c1bf39812a
2017-04-26 19:55:02 -07:00
Phil Cohen
cf88ecb166 progressfile: fix trailing space
Summary:
To my deep shame, I discovered today that `arc land` does not rerun `arc unit` if you introduce post-review changes.

Sorry!

Test Plan: Ran arc unit this time :)

Reviewers: quark, #mercurial, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4957528

Signature: t1:4957528:1493241560:9208e745ac92c0e771b0a4b4bac91d6c1feed025
2017-04-26 14:19:39 -07:00
Phil Cohen
22969a6bf8 progress: add progress.statefile config
Summary:
allows users to have JSON progress bar information written to a path

Controlled by the `ui.progressfile` config. Mercurial will overwrite this file
each time the progress bar is updated.

The schema of this file is (JSON):

- topics: array of topics from oldest to newest. (last is always the active one)
- state: map of topic names to objects with keys:
    - topic (e.g. "changesets", "manifests")
    - pos: which item number out of <total> we're processing
    - total: total number of items
    - unit: name of the type of unit being processed (e.g., "changeset")
    - item: the active item being processed (e.g., "changeset #5")
    - active: whether this is the currently active progress bar
    - units_per_sec: if active, how many <unit>s per sec we're processing
    - speed_str: if active, a human-readable string of how many <unit>s per sec
        we're processing
    - estimate_sec: an estimate of how much time is left, in seconds
    - estimate_str: if active, a human-readable string estimate of how much time
        is left (e.g. "2m30s")


Test Plan: added a test

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: rmcelroy, quark, stash, asriram, mjpieters, durham

Differential Revision: https://phabricator.intern.facebook.com/D4752788

Signature: t1:4752788:1493233878:b49095237d32233c78cd0e0aaaa7b94e0e9e7011
2017-04-26 14:05:44 -07:00
Stanislau Hlebik
64ecdf2691 fastpartialmatch: handle nullid correctly
Summary: Special case for nullid

Test Plan: arc unit

Reviewers: #mercurial, simpkins, ikostia

Reviewed By: ikostia

Subscribers: net-systems-diffs@fb.com, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4945927

Tasks: 17622253

Signature: t1:4945927:1493123209:712990c1e883241d99c0aeb4af8d48e18e1baa3d
2017-04-26 03:55:59 -07:00
Stanislau Hlebik
c48413d98b cleanobsstore: write cleanobsstore output to stderr
Summary: As it was noted in D4851138, writing to stdout may break extensions.

Test Plan: arc unit

Reviewers: #mercurial, simpkins

Reviewed By: simpkins

Subscribers: net-systems-diffs@fb.com, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4948924

Tasks: 17453793

Signature: t1:4948924:1493151885:4bcb971e65ac623aa5f91cf80fee5af6902c9226
2017-04-26 03:55:16 -07:00
Jun Wu
6c6ebce1f6 morecolors: new extension to add more colors to outputs
Summary:
The extension was a quick hack from me to make it easier to locate non-core
code when doing extension development. It seems a good idea to add colors in
general so it's named "morecolors" with the hope that we can colorize more
places in the future.

Test Plan: Added a test

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: durham, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4884999

Signature: t1:4884999:1492473666:89a6756b231eac4274fa05d489480229146bdab8
2017-04-25 17:03:26 -07:00
Jun Wu
44c4a80415 pushrebase: deal with phase move correctly
Summary:
Previously we ignore "draft -> public" phase move, and new nodes returned by
the server may remain "draft" until the next pull. This diff records the
obsmarkers and updates phases of the new nodes correctly so they become
public without an extra hg pull.

Test Plan: Modified an existing testcase

Reviewers: #mercurial, durham, rmcelroy

Reviewed By: durham

Subscribers: durham, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4934720

Signature: t1:4934720:1493076241:9b23da1140b68aece9e838c604e64611b3498794
2017-04-25 14:46:36 -07:00
Phil Cohen
eb2814985f resolve: add internal:dumpjson as an internal merge tool (as an extension, for now)
Summary:
This basically takes the last upstream version of my conflictinfo patch and makes it an extension, so we can get it out to FB/Nuclide users
a bit faster. @asriram has already been developing with this version as a personal extension for a couple of weeks.

I still plan to ship the upstream version through after the freeze ends -- when that happens, we can delete this extension. During the time the two
versions overlap, they shouldn't conflict. (The extension version will still run over the internal version until it's disabled, though.)

Review-wise, this is similar to the last upstream version, except it adds the "command" key that indicates which command generated the conflicts (based on which mutually-exclusive state file exists), and incorporates most of the feedback.

Test Plan: Added a test and ran associated tests.

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, asriram, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4944709

Signature: t1:4944709:1493148790:a4e798f5bd17ada767ae6c96fe8c8ab973960383
2017-04-25 12:46:28 -07:00
Stanislau Hlebik
8595e6160c logginghelper: log obsstore size
Summary: That's an interesting piece of information that we'd like to know.

Test Plan: Enable patched logginghelper.py extension and check that new field is logged

Reviewers: #mercurial, simonfar

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4936862

Tasks: 17453793
2017-04-25 01:09:28 -07:00
Rodrigo Damazio Bovendorp
31b8b83011 chistedit: choose default revision just like histedit
This makes the default behavior consistent between "histedit" and "chistedit"
in picking the ancestor revision to use when none is specified.
2017-04-24 10:00:31 -07:00
Rodrigo Damazio Bovendorp
7f02e97a65 nointerrupt: allowing attendance to default to True
This lets us prevent interrupts by default, and only allow them for
the few commands we know are safe.
2017-04-24 10:00:31 -07:00
Rodrigo Damazio Bovendorp
fe81fd7cdc backups: small output improvements for backups extension
This allows us to turn off the obsolescence warning altogether, since we still
use strip together with obsolescence in some situations (e.g. changing the
narrowspec for narrowhg, in which case we "insert" revisions which were
previously not relevant in the middle of the change log and then have to
strip and re-create their children to repoint p1).
2017-04-24 10:00:31 -07:00
Stanislau Hlebik
fceac7d85b phrevset: return baseset instead of list
Summary:
There was a recent change in mercurial and revsets
shouldn't return lists anymore. This diff fixes it.

This diff also fixes git diff handling. It returned not even a list,
but a single rev number. Unfortunately it's not tested because we need to
mock conduit response.

Test Plan: arc unit

Reviewers: #mercurial, ikostia

Reviewed By: ikostia

Subscribers: ikostia, kulshrax, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4938930

Signature: t1:4938930:1493051696:cfb278773e5932d5f38942b0a0ca49f018ec5083
2017-04-24 09:47:52 -07:00
Jun Wu
00a9465989 lfs: be compatible with filelog metadata
Summary:
This diff changes lfs `revision(raw=False)` output to include hg filelog
metadata. The LFS blob does not contain filelog metadata as before.

This hurts performance if there is a rename, or the binary starts with the
magic `\1\n`. But compatibility is greatly improved - it's now possible to swap
a non-lfs revision with mercurial rename to a lfs revision, and easier to be
compatible with remotefilelog (namely, remotefilelog defers filelog.add until
commit hash is known).

Test Plan: Modified existing test.

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: rmcelroy, durham, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4906074

Signature: t1:4906074:1492559513:09b25fc1026d4ce8fd784a044d6724f12e8bda45
2017-04-21 19:56:27 -07:00
Jun Wu
6cef30df92 absorb: do not invalidate fsmonitorstate
Summary:
@sid0 noticed absorb invalidating fsmonitorstate, causing `hg status` to be
slow. Unfortunately, `dirstate.rebuild` seems to be the only way to move
working directory parent without changing working copy, which is also used
by `hg reset` and removing state invalidation will cause `test-reset.t` to
fail. Fortunately, in absorb's case, it's fine to not invalidate fsmonitorstate
because no new status will occur outside ctx.files (`hg status` after absorb is
a subset of `hg status` before absorb).

Test Plan: `rt --extra-config-opt=extensions.fsmonitor= test-absorb*.t --chg`

Reviewers: #mercurial, sid0

Reviewed By: sid0

Subscribers: mjpieters, sid0

Differential Revision: https://phabricator.intern.facebook.com/D4925426

Signature: t1:4925426:1492796143:0d6cd4fd46ce130286d0b6ea5a01eadf350397b8
2017-04-21 19:47:00 -07:00
Durham Goode
cd53821465 warnings: remove deprecated nested usages
Summary:
Upstream Mercurial has enabled python deprecation warning in the tests, so
python 2.6 runs now complain about our use of context.nested(). The current fix
of marking all those functions with suppression decorators breaks the tests in
some environments since the suppression module can't be found unless it's
installed or the environment has been set up correctly.

Let's just get rid of our use of nested() and the nodeprecate module.

Test Plan: Ran the tests on centos7 with python 2.7

Reviewers: wez, #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4920337

Signature: t1:4920337:1492676584:1c26f2673c14e79793a3ee80c3281fd1bc76a313
2017-04-20 10:58:50 -07:00
Arun Kulshreshtha
02e5cc57ba tweakdefaults: allow hg update --merge without --nocheck
Summary: The --merge flag is relatively new; we shouldn't add --check if it is specified.

Test Plan: Unit test

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4916396

Tasks: 17490374

Signature: t1:4916396:1492647440:d9955931cb9177719a58325c08a715d4d51d1226
2017-04-19 17:20:23 -07:00
Arun Kulshreshtha
6065ceef3f fbamend: use unfiltered repo when clearing preamend bookmarks
Summary: Sometimes restack attempts to clear preamend bookmarks on hidden commits. This only happens when the user has a configuration error (typically inhibit is disabled) which would mess up other parts of restack. Nonetheless, there is no reason to crash here.

Test Plan: Disable inhibit, perform an amend, and run `hg restack`. Unit test is forthcoming.

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4856640

Tasks: 16726367

Signature: t1:4856640:1491816803:56beeb11360a7a2f6bc6243069c562b08a015afc
2017-04-19 15:52:45 -07:00
Wez Furlong
04ee6433d4 warnings: squelch contextlib.nested DeprecationWarnings
Summary:
deals with the tests failing like this on my python 2.7 system:

```
+  /data/users/wez/facebook-hg-rpms/fb-hgext/hgext3rd/uncommit.py:144: DeprecationWarning: With-statements now directly support multiple context managers
+    with nested(repo.wlock(), repo.lock()):
```

Test Plan: arc unit

Reviewers: #mercurial, stash, simonfar, wez

Reviewed By: stash, simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4910040

Signature: t1:4910040:1492593520:ff815c5dbed4a341e0a313cd7160b700d878ee2c
2017-04-19 03:28:23 -07:00
Simon Farnsworth
e31246ce99 rage: include klist output
Summary:
We're seeing a rise in the number of auth issues. As a first step,
let's have rage grab klist, so we can see if the user has a valid Kerberos
ticket for hg.vvv or not.

Test Plan: Run it and see https://phabricator.intern.facebook.com/P57315650

Reviewers: #sourcecontrol, ikostia

Reviewed By: ikostia

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4907615

Signature: t1:4907615:1492546856:969cab7b8deacf220c374d4c9e1b6e99f84ca059
2017-04-19 03:18:04 -07:00
Kostia Balytskyi
aeef0ad5cc compatibility: migrate from scmutil.vfs to mercurial.vfs.vfs
Differential Revision: https://phabricator.intern.facebook.com/D4908906
2017-04-18 14:42:33 -07:00
Wez Furlong
934dfaf08b templater: fixup for templater changes upstream
Summary:
The function signature for a number of functions relating to templating changed.

In addition, the templater layer now requires that template functions have been
registered via the decorator, otherwise a require attribute is missing and
things blow up.

Test Plan: arc unit.  rt.

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4908220

Signature: t1:4908220:1492545033:66f089e03a6064ae96ca36f3abf99ecb6d48d422
2017-04-18 13:04:11 -07:00
Wez Furlong
7170cd6411 tweakdefaults: switch hg grep to use dirstate.walk
Summary:
previously, this was using .match() to find candidate names
and then reaching inside the dirstate._map to check the status of
the file.

This implementation uses the dirstate.walk instead, which returns a
map of filename -> stat information.

This approach works better with custom dirstate implementations, such
as eden, where a local map of the required information is not available
and where it would be suboptimal to request that information on
demand on a per-file basis.

Test Plan:
testing before and after this change:

`time hg grep eden foo` has a runtime of ~1.2 seconds.

If you have sqldirstate enabled, this change makes the runtime 3 seconds
compared to 2 with sqldirstate on and without this change.

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: mjpieters, simpkins

Differential Revision: https://phabricator.intern.facebook.com/D4903110

Signature: t1:4903110:1492473495:ebe375f3e2533f6ae4ca7ea9a494f6feb9704f9e
2017-04-18 10:05:31 -07:00
Arun Kulshreshtha
a72a47d515 absorb: s/chunks/chunk
Summary: Noticed this minor gramatical error when using absorb.

Test Plan: Unit tests.

Reviewers: quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4887040

Signature: t1:4887040:1492123047:dcae53cabd64f1df13d80a20953c712ce0aaaa09
2017-04-13 16:00:51 -07:00
Adam Simpkins
642dea7e2c fixcorrupt: automatically look back as far as necessary
Summary:
Update the fixcorrupt extension to look back as far as necessary in the revlog
to find a good entry.  Previously the user had to supply a good value using the
--checklen argument.  If --checklen was too small, fixcorrupt could end up
trying to truncate in the middle of a corrupted section of the revlog, which
would cause problems as it was still using corrupted data.

This change ensures that fixcorrupt always looks back far enough to find a good
entry.  We start by looking at the last 10 entries, and double that amount each
time the first item in the list is still bad.

Test Plan:
Used it to fix a corrupted fbsource repository where the corruption was 36
commits back.

Reviewers: #mercurial, quark, stash

Reviewed By: stash

Subscribers: stash, net-systems-diffs@fb.com, yogeshwer, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4874243

Signature: t1:4874243:1492007518:7406b03a9967815a496a8c2fae394c5f21f7e60a
2017-04-12 14:09:22 -07:00
Durham Goode
4da0040f03 fastlog: fix ScmQuery request parameter name
Summary:
This was totally wrong and has been forever. I think we got lucky because the
server was serving from memcache mostly, which did not validate this parameter.

Test Plan:
Ran log before and after the change.

hg.real log wdc --config extensions.fastlog=../../facebook-hg-rpms/fb-hgext/hgext3rd/fastlog.py

Reviewers: #mercurial, mitrandir

Reviewed By: mitrandir

Subscribers: mjpieters, jeroenv

Differential Revision: https://phabricator.intern.facebook.com/D4878633

Tasks: 17327887

Signature: t1:4878633:1492030224:cf1b802345c7e00d1134ee81fbe9f271cf1a6752
2017-04-12 13:51:18 -07:00
Stanislau Hlebik
8f8cdc2d49 hgext3rd: cleanobsstore extension
Summary: See comments and docstring

Test Plan: arc unit

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: quark, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4851138

Signature: t1:4851138:1491589247:5ca750e45bcc9fc996c9d7c14e31cb5f24ea983f
2017-04-12 01:16:01 -07:00
Jun Wu
340750c4f4 pushrebase: send obsmarkers to client if client supports it
Summary:
Previously when the client has obsstore enabled, and the server has obsstore
disabled, clients won't get the obsmarkers, which is suboptimal. This diff
makes the server send obsmarkers in that case.

Practically, this means people will no longer need to run `strip` after landing
a diff in this repo.

Note: `test-pushrebase-manifests.t` is somehow flaky about the empty lines. I
guess they are not related to the change. But the test change is not 100%
reproducible.

Test Plan: Added a new test case

Reviewers: #mercurial, stash

Reviewed By: stash

Subscribers: rmcelroy, ikostia, stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4865150

Signature: t1:4865150:1491895844:afbc3079a40a9a9fa9af1eab4eeaca91091e8d2d
2017-04-11 13:21:27 -07:00
Jun Wu
11cd7af681 lfs: add a fast path for filelog.size
Summary:
Previously, `filelog.size` requires loading the lfs blob in memory. This
makes it unnecessary.

Test Plan: Run existing tests.

Reviewers: #mercurial, mitrandir

Reviewed By: mitrandir

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4861358

Signature: t1:4861358:1491923349:f1d2fe5656158854e1dd50987ea3db64a9793db6
2017-04-11 13:20:58 -07:00
Jun Wu
de5e7c6545 fixcorrupt: new extension to fix corrupted repo
Summary:
We sometimes get reports about corrupted repos. Usually the corrupted part
is just at the end of changelog or manifest.

Truncating them manually works but people need to be very careful. This
extension is like the manual fix but automatized.

Test Plan:
Run `hg debugfixcorrupt --no-dryrun` on a reported corrupted repo and check
it truncates files correctly and the repo passes `hg verify` afterwards.

Reviewers: #mercurial, durham, stash

Reviewed By: stash

Subscribers: stash, rmcelroy, durham, lcharignon, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3408396

Signature: t1:3408396:1491897232:fc17a105124b568963441adfec97e26735df3258
2017-04-11 13:19:42 -07:00
Adam Simpkins
fdc2c3e6f9 extutil: add unit tests for runbgcommand
Summary:
Add some basic unit tests for extutil.runbgcommand().

This also changes the behavior to throw an OSError if we fail to execute the
process, instead of a subprocess.CalledProcessError.  This matches the behavior
of the subprocess module when it fails to execute the command.

Test Plan: Ran the tests.

Reviewers: stash, quark

Reviewed By: quark

Subscribers: net-systems-diffs@fb.com, yogeshwer, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4864999

Signature: t1:4864999:1491874918:c03edafe02af217e41c28a770137bfd72bcbba9b
2017-04-11 11:25:40 -07:00
Pallav Shinghal
3ded80a5c7 rage: fix hg sparse entry in hg rage output
Summary: The `hg sparse` entry in `hg rage` output is consistently `(Failed. See footnote [1])` due to the `sparse` function being called with the wrong signature.

Test Plan:
Ran `hg rage` in <reponame>. Output contained:

```
hg sparse:
---------------------------
%include .hgsparse-<reponame>
[include]

[exclude]

```

Reviewers: #sourcecontrol, quark

Reviewed By: quark

Subscribers: quark, mjpieters, dawidp

Differential Revision: https://phabricator.intern.facebook.com/D4861832

Tasks: 17269067

Signature: t1:4861832:1491869628:0abc55eb5308a83effe78943dba8d13e9f12d616
2017-04-11 07:19:26 -07:00
Jun Wu
dc051c4791 tweakdefaults: fix common incorrect PAGER config automatically
Summary:
People may mistakenly set PAGER=less, which will lose colors and does not
have the friendly "exit if there is less than one screen" feature.

Detect that and add "-FRQX" automatically.

Test Plan:
Unset all my `pager.pager` configs, and export `PAGER=less`, verify `hg sl` has
the desired pager and colors are fine.

Reviewers: wez, #mercurial, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4864822

Signature: t1:4864822:1491870110:246f906d41b5ac72ecfdda487aa8f412a90cf2cb
2017-04-10 17:38:22 -07:00
Adam Simpkins
df34a531e6 extutil: add a new runbgcommand() function
Summary:
Add a new `runbgcommand()` function, which is similar to `runshellcommand()`
but does not require running the command through a subshell.  Running commands
through a shell is a very common place for security vulnerabilities if care is
not taken to correctly escape all arguments.

This changes `runshellcommand()` to be implemented using `runbgcommand()`.
This does change the behavior of `runshellcommand()` slightly:
`runshellcommand()` now throws an exception if it failed to execute the
command.  Previously the Unix implementation silently swallowed most
exceptions.  This also fixes the unix implementation to properly wait on the
intermediate child process.  Previously it would remain as a zombie process.

Test Plan: Ran all unit tests.

Reviewers: #mercurial, stash

Reviewed By: stash

Subscribers: quark, net-systems-diffs@fb.com, yogeshwer, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4856672

Signature: t1:4856672:1491813879:367e1f42b9aa7797542f0d02da99f46683b80df8
2017-04-10 11:44:32 -07:00
Jun Wu
2f1756608e lfs: remove manifest walk in prepush hook
Summary:
`ctx.files` should contain changed files from both p1 and p2. There is no
need to walk the manifest, which could also be painfully slow for large
repos. So the manifest walk got removed.

Test Plan: `arc unit`

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4856552

Signature: t1:4856552:1491639067:8080f9bc7e246a8d36fcc3504716e9c95ee24fcf
2017-04-10 11:22:54 -07:00
Jun Wu
46d44c8b07 lfs: use non-chunking spec format if possible
Summary:
The "chunking" feature is not specified by [the current Git LFS standard](21e1695220/docs/spec.md).

Therefore avoid using it if possible - if there is only one chunk, use the
standard specification (`https://git-lfs.github.com/spec/v1`).

An upload message is slightly changed to be more accurate.

Test Plan: Changed existing tests

Reviewers: davidsp, #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4856527

Signature: t1:4856527:1491638822:1c6f555b706e7bb22dd9090afa156f2161bf9f7f
2017-04-10 11:17:48 -07:00
Jun Wu
38632025a9 lfs: add bundle support
Summary:
This diff adds bundle support for lfs:

  - Let `hg bundle` use changegroup3 instead of changegroup2 to record revlog
    flags.
  - Hook related functions so `hg -R bundle.hg` works with LFS.

Test Plan: Added a test

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4856335

Signature: t1:4856335:1491638660:d56ba54ea5f59262f009418b9c7e48c8f2a25ed6
2017-04-10 11:11:37 -07:00
Jun Wu
cdc9accca5 lfs: do not store mercurial filelog metadata in lfs blobs
Summary:
Per discussion with @davidsp, it's better for LFS to not store Mercurial
filelog metadata, which is currently used to store rename information. That has
many advantages:

  - Large blobs could be reused across renames
  - No need to special handle files starting with `\1\n`
  - P4 LFS server implementation is much easier
  - remotefilelog LFS support is easier and cleaner

That said, the rename information is stored as lfs metadata using the
non-standard `x-hg-copy`, `x-hg-copyrev` keys. So they still exist and are
functional.

The disadvantage is that rename gets no longer hashed, which is probably fine.


Test Plan: Added a test

Reviewers: davidsp, #sourcecontrol, rmcelroy

Reviewed By: rmcelroy

Subscribers: jsgf, rmcelroy, stash, mjpieters, davidsp

Differential Revision: https://phabricator.intern.facebook.com/D4849764

Signature: t1:4849764:1491580506:1d80ad476b9cbd6773843cb52aee6745f478a0b0
2017-04-07 18:29:35 -07:00
Stanislau Hlebik
1c6b1b495d fastpartialmatch: catch write exceptions
Summary:
We saw a couple of weird exceptions during clone. Since read path is wrapped
in try/catch, let's wrap write path too. If exception happened then let's
remove index completely since it will be rebuilt on the next pull.

Test Plan:
arc unit

Unit-tests are actually almost useless in this case.
So I added fake exception raises and checked that it behaves as expected.

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4843497

Tasks: 17153898

Signature: t1:4843497:1491536062:8e72a4ad4fffe5625866a90b3b89acda75211ce8
2017-04-07 02:21:54 -07:00
TJ
9b91e50f77 edrecord: allow specifying a different editor for chunk selection
Summary:
a user may want to use a special-purpose 'editor' to do chunk selection, but not
have to use that same 'editor' to write commit messages and resolve merge conflicts

This is hacked together to make it work.  I'm not super familiar with the correct way to do this in python.

Test Plan: added test case

Reviewers: rmcelroy, simonfar, most, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4832207

Tasks: 15728734

Signature: t1:4832207:1491519433:82a3e62d5ae03f456abe97caf1bd352a02fe35c3
2017-04-06 16:49:27 -07:00
Jeroen Vaelen
8bc1e221b6 hgext: rename show.py to fbshow.py to prevent upstream collision
Summary:
Upstream recently introduced show as an extension. Because the file
name is the same and it lives in core, extensions.show now points to it.

It behaves differently. Rename our extension. In the next diff I'll update
facebook.rc to make extensions.show point to our version.

Test Plan: Ran tests

Reviewers: durham, #sourcecontrol

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4845959
2017-04-06 13:36:18 -07:00
Jun Wu
de84869150 lfs: cleanup user-facing messages
Summary:
The diff cleans up messages shown to the user. It makes verbose messages
gated by `if ui.verbose`, and simplifies some words.

The resulting user experience is, when there is no large file involved, lfs
shows nothing. When there are largefiles being downloaded or uploaded, show
progress bar if it takes long. The progress bar is the only user visible
output from lfs by default.

Test Plan: `rt test-lfs.t`

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4813297

Signature: t1:4813297:1491213089:c9ec363eb65df4c85282c90b230da3321b93b5e0
2017-04-05 15:58:56 -07:00
Jun Wu
b1cdf3f0e6 lfs: remove setup.py
Summary:
The logic in `setup.py` is now simple enough to be moved to `reposetup`.
`setup.py` becomes unused and gets removed.

Test Plan: `rt test-lfs.t`

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4813193

Signature: t1:4813193:1491212154:0238ea448269b0e5516c927c1fa989d6d9130d4a
2017-04-05 15:49:08 -07:00
Jun Wu
f0b6b81970 lfs: simplify remote blobstore.logic
Summary:
Previously, the remote blobstore could be either `git-lfs` or `dummy`. The
application code does not really care, it only wants a "remote" blobstore.

This diff adds a factory method and makes `git-lfs` and `dummy` stores private.

The `@staticmethod get(vfs)` interface is also removed as it's duplicated and
unnecessary - as long as mercurial calls `reposetup`, the blob store objects
are set, and they cannot be missing.

The error message about an unsupported store is also changed to be consistent
with mercurial style. A test was added to test the error.

Test Plan: `rt test-lfs.t`

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4813023

Signature: t1:4813023:1491382755:b9d8ec6518141d0ba8263e16c53f430ce80c39f0
2017-04-05 15:48:10 -07:00
Jun Wu
039e232a0f lfs: fix an error caused by a wrong merge resolution
The change should belong to D4812131. But the change gets lost during a wrong
merge resolution. This patch fixes it.
2017-04-04 16:22:31 -07:00
Jun Wu
00e6156a07 lfs: let blobstore.local and blobstore.remote take an repo argument
Summary:
Previously local takes a path, remote takes a ui. This diff makes it more
consistent.

Test Plan: `rt test-lfs.t`

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4812884

Signature: t1:4812884:1491211049:6b3709e3b7e054b11d4a3f9a92e7921fe55de9fa
2017-04-04 16:13:16 -07:00
Jun Wu
ead899b26b lfs: cleanup option passing
Summary:
For options passed to revlog, use `repo.svfs.options`. For objects, use
`repo.svfs.objname`.

Unused assignments and unused functions are removed.

Test Plan: `arc unit`

Reviewers: #mercurial, simonfar, rmcelroy

Reviewed By: simonfar, rmcelroy

Subscribers: rmcelroy, simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4812131

Signature: t1:4812131:1491210581:004297885b8b09d23b58e0a3bee147910615ce62
2017-04-04 16:10:29 -07:00
Jun Wu
6d19f83af9 lfs: add a "bypass" config option
Summary:
The bypass option limits lfs's functionality to only skip hash checks. It is
intended to be used server-side, to make it more predictable - the server
never interacts with the lfs blob service.

Test Plan: Added a test case

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4811828

Signature: t1:4811828:1491301114:c2e3c4200ce4cc84b9c5872a8b9a040176bb002a
2017-04-04 16:08:36 -07:00
Jun Wu
6c51d725bd errorredirect: log and fallback to old error handler when appropriate
Summary:
Previously errorredirect swallows errors in some cases. This diff makes it
more robust.

  - Always log commandexception. `blackbox.log` will probably catch it.
  - Print the trace if the handler script cannot be executed.
  - Print the trace if the handler script exits with non-zero.
  - Add exception for the Ctrl+C case, to not pollute user's terminal.

Test Plan: Added some test cases

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4824767

Signature: t1:4824767:1491299807:3fd863cb99ce71a6c2b59643b0c89eece7985d4b
2017-04-04 10:27:30 -07:00
Stanislau Hlebik
ebde6a9a79 gitlookup: fix lookup if not string is passed
Test Plan:
arc unit

Note: making sure that int is passed in repo.lookup(...) is not easy.
For example `hg log -r 1` does not pass an int value to repo.lookup().
But `hg bundle` actually does it so use `hg bundle` to verify that fix works.

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4825996

Signature: t1:4825996:1491292671:9bd0897205dff93cb6dee75498827b3826f894da
2017-04-04 01:04:40 -07:00
Jun Wu
08d017f89e absorb: move to a package
Summary:
The ideal interactive mode couldn't be implemented trivially. Move `absorb`
to a directory so we can add related, but decoupled components as separate
files.

Test Plan: `make local`

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4813909

Signature: t1:4813909:1491211561:e9c40b1242c9b74230c0b8937723a2d4548e22c3
2017-04-03 10:40:31 -07:00
Jun Wu
abb7a83354 lfs: make tests stronger
Summary:
This diff makes the `test-lfs.t` much more stronger. It reveals a lot of core
hg issues in this area. I'll send patches to fix them all.

Regarding on lfs, there are some changes:

  - An existence check in its push hook was added. Otherwise pushing a revision
    with rename will cause crash.
  - The "read" processor is responsible for downloading blobs, and translate
    raw revision to lfs text. It should always return lfs text. But it may
    return raw revision text on error currently. That error handler was
    removed to avoid further damage.


Test Plan: Added new test cases. I also added `hg verify` to sanity check things are good.

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4792641

Signature: t1:4792641:1490868946:8f09c84dc9ebda3889d6a1ea04c49a06acbf38a3
2017-03-30 16:41:46 -07:00
Adam Simpkins
36dcf86559 phabstatus: fail gracefully if necessary arcrc settings are missing
Summary:
If the user does not have necesary credentials defined in their arc
configuration, catch the KeyError and convert it into an ArcConfigError.

The existing call sites in the phabstatus and arcdiff extensions catch and
handle ArcConfigError, but not generic KeyErrors.

This also fixes the phabstatus warning messages to end with a newline.

Test Plan: Added a unit test.

Reviewers: #sourcecontrol, quark, simonfar, wez, rmcelroy

Reviewed By: wez, rmcelroy

Subscribers: rmcelroy, net-systems-diffs@fb.com, yogeshwer, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4800977

Tasks: 17002914

Signature: t1:4800977:1490847078:e18bba042e3ff57100e0a7b25c610b5cad17fa2e
2017-03-30 11:55:39 -07:00
Stanislau Hlebik
2fc45e80f1 gitlookup: fix gitnode() template and revset
Summary:
Neither gitnode() template nor gitnode() revset worked on the server. They
both called `hg.peer(repo, {}, 'default)` and since 'default' is not a valid
url these calls failed. To fix it let's wrap `localrepo.lookup()` and call
local repo instead of peer repo if we can't find default path.

Test Plan: arc unit

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4794488

Tasks: 16848812

Signature: t1:4794488:1490795600:24269507f24613883406f0c1838fce302f86df58
2017-03-29 07:13:41 -07:00
Phil Cohen
0e0f112b48 pushrebase: prepopulate the post-lock bundle manifest cache
Summary:
Previously we copied the old bundle dirmancache entries into the new bundle
dirmancache entries, but that missed any manifests that weren't already in the
cache. Since we have a list of bundle revs that are guaranteed to be needed
during the rebase, lets read them before the lock and force them into the cache
after the lock.

Test Plan:
Ran all pushrebase tests with a block like:

```
bundle.manifestlog[mfnode].read()
realmf = newdirmancache[''][mfnode].read()
if realmf.text() != mfdict.text():
     raise Exception("cache invalid")
```

...to verify the manifest we cached is the same as can be read from disk.

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4774728

Signature: t1:4774728:1490736995:6e20d69181b4df76aeb800ea246a0dae0a4b802d
2017-03-28 17:18:59 -07:00
Phil Cohen
5afc9c89dd pushrebase: set the manifest revlog fulltextcache
Summary:
This fetches the old master's full text before the lock is taken, and puts it in
the revlog._cache field so we don't have to read the full delta chain when we go
to read the latest master revision. This minimizes the amount of time we have to
spend processing the manifest during the first write.

Test Plan:
Ran all tests using a block like the following:

```
        for mfnode, mfdict in bundlerepocache.iteritems():
            bundle.manifestlog[mfnode].read()
            existing = newdirmancache[""][mfnode].read().text()
            new = mfdict.text()
            if new != existing:
                raise Exception("Cache failed")
```

...which raises if the data we were about to pull from the cache was different from what we were reading from the manifest before.

Reviewers: #mercurial, durham, rmcelroy

Reviewed By: durham

Subscribers: rmcelroy, quark, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4774716

Signature: t1:4774716:1490735677:85bac79999d607f5554d090978d0ee6787f5e11c
2017-03-28 17:18:59 -07:00
Phil Cohen
948d47064a pushrebase: avoid diff'ing manifests for conflict detection
Summary:
Previously, pushrebase detected if it was ok to rebase a commit by diff'ing the
original common ancestor of the bundle and master with the new master. This
required reading both manifests. To improve performance, let's read the
changelog files list instead, which is much, much faster.

Merge commits don't have a files list that is representative of the actual diff,
so fall back to manifest diff in that case.

Test Plan: ran tests

Reviewers: #mercurial, stash, durham

Reviewed By: stash, durham

Subscribers: stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4774694

Signature: t1:4774694:1490601056:18f78ab7c3e35a1beb2a7775160271a3311e38bb
2017-03-28 17:18:59 -07:00
Jun Wu
07d5d0f836 lfs: fix a pyflakes issue 2017-03-28 15:43:52 -07:00
Jun Wu
eeeb0210ae lfs: allow blobstore to be outside the repo
Summary:
Previously `lfs.blobstore` must be a relative path. If an absolute
blobstore path is set, it will traceback because vfs audit fails:

```
  File "hg/mercurial/revlog.py", line 1356, in _processflags
    text, vhash = writetransform(self, text)
  File "fb-hgext/hgext3rd/lfs/wrapper.py", line 69, in writetostore
    blobstore.local.get(self.opener).write(storeid, chunk)
  File "fb-hgext/hgext3rd/lfs/blobstore.py", line 41, in write
    fp = self._opener(self.filename(storeid), 'w+', atomictemp=True)
  File "hg/mercurial/vfs.py", line 344, in __call__
    self.audit(path)
  File "hg/mercurial/pathutil.py", line 64, in __call__
    raise error.Abort(_("path contains illegal component: %s") % path)
  Abort: path contains illegal component: /home/quark/lfslocalblobstore/d7/dbc611df1fe7dfacfe267a2bfd32ba8fc27ad16aa72af7e6c553a120b92f18
```

That was because the code was using `repo.vfs`. This diff adds a new `lfsvfs`
to avoid the issue. The `lfsvfs` also did the correct filename check (the
old `re.match` check will not match the whole string), so `blobstore.local`
could be simplified a lot.


Test Plan:
A new test case was added to make sure absolute blobstore path works. I also
did some cleanups for the test file to de-dup hgrc, and avoid writing files
outside `$TESTTMP`.

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters, remi

Differential Revision: https://phabricator.intern.facebook.com/D4785084

Signature: t1:4785084:1490693306:b42eef8e53af107897c2e1bc2984c090bdd2e465
2017-03-28 15:41:59 -07:00
Jun Wu
adfd01303d perftweaks: remove hardlink tweak
Summary:
The core mercurial could now detect filesystem for Linux and OSX, and enable
real hardlink support for their major filesystems. So the hardlink tweak is no
longer needed.

Test Plan: `arc unit`

Reviewers: #mercurial, stash

Reviewed By: stash

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4776139

Signature: t1:4776139:1490601097:0be66e2a0111b71a932b453ef4e1e75fed8276e6
2017-03-27 10:43:01 -07:00
Jun Wu
c3fd525bf8 lfs: fix tests
Summary:
A bunch of modifications to get the test pass with the new lfs code.

- Move `lfs` to `hgext3rd`. The code was supposed for hg-core. For now, we do
  them in fb-hgext to speed up the process
- Remove the windows test, which is not supported by `run-tests.py` and is
  duplicated with `test-lfs.t`.
- Do import `mercurial.i18n._` correctly.
- Change some i18n logic a bit so it's more translator-friendly.
- Change `revlog.RevlogError` to `error.RevlogError`.
- Avoid direct symbol import of `mercurial.util.bytecount`, which will fail the
  upstream importchecker test.
- Fix various lint issues like lines being too long etc.
- Document lfs config options.

Test Plan: `arc unit`

Reviewers: #sourcecontrol, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4772216

Signature: t1:4772216:1490401458:1ad3c18ab80e1d31085d0b6b4c630e62a7dc7930
2017-03-24 19:01:42 -07:00
Simon Farnsworth
ccf6655f3a fb-hgext: fix test-check-code.t failures
Summary: test-check-code.t now dislikes our docstrings. Fix them up

Test Plan: Run the test locally on my devserver

Reviewers: #sourcecontrol, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4773908

Signature: t1:4773908:1490394131:9d9e5f85b4243119a3615044605bcc5abdecbe4f
2017-03-24 15:22:31 -07:00
Kostia Balytskyi
b6a03bd647 pushrebase: get page size form mmap instead of resource
Summary: `resource` is UNIX-only module, we can't use it if we want `pushrebase` to work on Windows. As we only use `getpagesize()`, we can also do `mmap.PAGESIZE` instead.

Test Plan: - apply change on Linux, see that they do not introduce any new failures

Reviewers: #sourcecontrol, durham, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4762553

Signature: t1:4762553:1490294453:fdcb2e9caa38f5d14eafc6ff36a1bb7d9429b6e9
2017-03-23 11:51:56 -07:00
Jeroen Vaelen
b7e3010b10 perftweaks: enable hardlink backups
Summary:
Hardlink backups are significantly cheaper than file copies. They are
disabled in core because they are problematic on CIFS.

Test Plan: Ran the tests.

Reviewers: #sourcecontrol

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4652490

Tasks: 16406729
2017-03-21 13:05:28 -08:00
Jun Wu
1e30f781e0 dirsync: rename in-repo config file to .hgdirsync
Summary:
It will allow sparse to not ignore the file, and is more consistent with
other special files.

Test Plan: Updated existing tests

Reviewers: rmcelroy, #sourcecontrol, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4738766

Signature: t1:4738766:1490032493:199f3fef9c74a137b16ae7637b87de625ca5115d
2017-03-20 10:55:43 -07:00
Stanislau Hlebik
7f8418f995 newhiddencache: record obsinhibit
Summary:
We forgot one more store file: obsinhibit.
This diff adds it

Test Plan: arc unit

Reviewers: #mercurial, mjpieters

Reviewed By: mjpieters

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4714522

Tasks: 16558840

Signature: t1:4714522:1489717585:fdabf3c5f0c11310708ba3d78df39cbaa04a3afa
2017-03-20 01:38:06 -07:00
Stanislau Hlebik
f0b136f93b newhiddencache: add knob to enable fasthiddencache
Summary: Add a knob to easily disable/enable fasthiddencache functionality

Test Plan: arc unit

Reviewers: #mercurial, mjpieters

Reviewed By: mjpieters

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4714509

Tasks: 16558840

Signature: t1:4714509:1489717542:d022201fd7c17863ffe664ae4115b8d1d896da3c
2017-03-20 01:38:06 -07:00
Stanislau Hlebik
32199b95b4 newhiddencache: restore back
Summary:
This diff just restores the code that was reverted before. Next diffs will fix
it

Test Plan: arc unit

Reviewers: #mercurial, mjpieters

Reviewed By: mjpieters

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4714397

Tasks: 16558840

Signature: t1:4714397:1489717436:9777d783d1443dea46460450e8f3da862c147483
2017-03-20 01:38:06 -07:00
Jun Wu
5404cd9a3c codemod: get rid of ui.backupconfig
Summary:
The upstream will remove `ui.backupconfig` soon(tm). Let's migrate our code
to `ui.configoverride` to avoid future breakages.

Test Plan: `arc unit`

Reviewers: #sourcecontrol, simonfar

Reviewed By: simonfar

Subscribers: simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4734300

Signature: t1:4734300:1489803960:a4101d6bc95ab62af67f0076b8e21f57cf926e13
2017-03-17 19:42:50 -07:00
Jun Wu
c133169b8b dirsync: read ".dirsync" in the working copy as part of its config
Summary:
It'd be nice to have dirsync config in the repo, since it's better synced, and
easier to be modified.

Test Plan: Added a test case

Reviewers: #sourcecontrol, simonfar

Reviewed By: simonfar

Subscribers: simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4732054

Signature: t1:4732054:1489793650:ca6c63ef47a1d9ebd181fbaee5554975801324eb
2017-03-17 16:35:08 -07:00
Jun Wu
ca4723ad22 patchrmdir: new extension to workaround rmdir kernel issues
Summary:
We have encountered a kernel issue where `rmdir` a non-empty directory may race
with other things and hang in kernel for a long time.

This patch changes `os.rmdir` to avoid `rmdir` non-empty directories. It is
written in Cython calling the low-level `readdir` libc friends to make overhead
minimal.

Test Plan: Added a new test

Reviewers: #sourcecontrol, clm, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, simpkins, osandov, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4716711

Tasks: 16647532

Signature: t1:4716711:1489627923:7c7432748c1fd8c070ce257bd172feebd3807f65
2017-03-15 18:55:48 -07:00
Mateusz Kwapich
1bcc10919d fbmetaedit: intoduce the extension
Summary:
It's just copy-paste from my mutable-history patch series that ended up as
persistent hotfix.

The wrapping logic is very simple: it redirects '--fold' calls to original
metaedit and handles all the rest.

Test Plan:
Included a big copy-pase from test-evolve.t preserving only cases that were
executing metaedit and modified for the smaller test repo (the metaedit tests
were at the end of evolve test file).

Reviewers: #sourcecontrol, quark, simonfar

Reviewed By: quark, simonfar

Subscribers: rmcelroy, quark, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4708017

Signature: t1:4708017:1489524494:d682aab857e8422bb1a23adc22dddfb9f505b6b3
2017-03-15 07:40:26 -07:00
Jun Wu
02d79f722e codemod: replace repo.join to repo.vfs.join
Summary: Upstream has deprecated `repo.join`. Let's use `repo.vfs.join` instead.

Test Plan: `arc unit`

Reviewers: #sourcecontrol, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4704018

Signature: t1:4704018:1489462391:532b15c24faa33d584f25bb9382c1ff4b2c0c483
2017-03-13 20:51:37 -07:00
Jun Wu
3440ef0a0d chistedit: use an alternative way to disable color
Summary:
The upstream has changed how color works. `chistedit` seems to use a very hacky
way to disable color. Let's use a safer and simpler approach.

Test Plan: Run chistedit with hg-committed.

Reviewers: #sourcecontrol, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4702229

Signature: t1:4702229:1489446376:f0c7b4d31eb62bb6aea4cfeb13d78be8c219374c
2017-03-13 16:07:21 -07:00
TJ
10a278c273 edrecord: fix binary change comment bug
Summary: The poorly formatted docstring was causing an extra blank line to be inserted both before and after the content

Test Plan: Changed a binary in an hg repo. Committed interactively using the editor. Made sure that there were no blank lines

Reviewers: rmcelroy, quark, simonfar

Reviewed By: simonfar

Subscribers: most, asriram, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4684549

Tasks: 15728734

Signature: t1:4684549:1489269897:faecd381c98e8c2b9884f9bacd18dfc32ee77758
2017-03-13 10:34:38 -07:00
Durham Goode
b0c4b9a793 fbamend: unlink rebasestate after transaction
Summary:
Previously, the rebase logic itself would unlink its state file. Now that the
state file is part of the transaction, it gets serialized at the end of the
transaction, even if it was unlinked midway through the transaction. This
affects fbamend because it wraps various rebases in a higher transaction.

Ideally we would add support to the mercurial transaction framework to allow
marking a file as deleted halfway through the transaction, but this patch will
fix the tests until we get that upstream.

Test Plan: Tests now pass

Reviewers: kulshrax, #mercurial, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4696578

Signature: t1:4696578:1489348040:b12104b2519d69dab98d2076f640f9f3157036e2
2017-03-12 13:11:47 -07:00
Durham Goode
932cc0e5b1 grepdiff: fix to match upstream
Summary:
Upstream changed the format from ['diff ...\n', 'file1\nfile2\nchunks...'] to be
['diff ...\nfile1\nfile2\n', 'chunks...'] in 199440c7d1f49752. So grepdiff needs
to be updated.

Test Plan: Tests now pass

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4695911

Signature: t1:4695911:1489282635:920e88fecc5d6d4660d1d21289d1d07e26c93315
2017-03-12 12:49:18 -07:00
Stanislau Hlebik
4b2050d789 logginghelper: add logging.configoptions
Summary: Let's add option that contains a list of other config options to log.

Test Plan: arc unit

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: durham, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4681336

Signature: t1:4681336:1489171115:dddd27032e945f37fcc846a511e3aaa9e83d29d3
2017-03-12 12:01:45 -07:00
Durham Goode
33a1d6afdc fix: update manifest.diff usages to handle match arg
Summary:
Upstream has added a new match argument to manifest.diff() and removed the
existing manifest.matches() function, so we need to update our internal usage.

A separate diff will update treemanifest to support the new diff() api.

Test Plan:
Ran the tests, some still fail because of the upstream changes, but
future patches fix those.

Reviewers: #mercurial

Differential Revision: https://phabricator.intern.facebook.com/D4677002
2017-03-12 10:14:35 -08:00
Arun Kulshreshtha
7174632e3c fbamend: add --towards flag to hg next
Summary: This diff adds a `--towards` flag to `hg next`. This flag tells `hg next` to always proceed in a linear fashion towards the specified commit. This is useful in situations where the user has a very branchy stack. For example, if the user has a local stack with several branches, each with a feature bookmark at the top, if the user wants to go up a particular stack they can do so with something like `hg next 3 --towards bookmark`.

Test Plan:
Here is an example of basic usage. Also see test file.

```
$ hg debugbuilddag "+5 *2 +2"
$ hg book -r 4 feature1
$ hg book -r 7 feature2
$ hg up 0
$ hg sl
o  92eaf3  debugbuilddag  feature2
|  r7
|
o  8bbe84  debugbuilddag
|  r6
|
o  914970  debugbuilddag
|  r5
|
| o  bebd16  debugbuilddag  feature1
| |  r4
| |
| o  2dc09a  debugbuilddag
| |  r3
| |
| o  012414  debugbuilddag
|/   r2
|
o  66f7d4  debugbuilddag
|  r1
|
@  1ea734  debugbuilddag
   r0
$ hg next 3 --towards feature1
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
[2dc09a] r3
$ hg sl
o  92eaf3  debugbuilddag  feature2
|  r7
|
o  8bbe84  debugbuilddag
|  r6
|
o  914970  debugbuilddag
|  r5
|
| o  bebd16  debugbuilddag  feature1
| |  r4
| |
| @  2dc09a  debugbuilddag
| |  r3
| |
| o  012414  debugbuilddag
|/   r2
|
o  66f7d4  debugbuilddag
|  r1
|
o  1ea734  debugbuilddag
   r0
```

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4695861

Signature: t1:4695861:1489325959:1145a29ba87f99d07ede1e81415804723dc48818
2017-03-12 10:09:12 -07:00
David Soria Parra
83b308aa75 chistedit: make chistedit compatible with current tip 2017-03-10 14:11:11 -08:00
Adam Simpkins
143d13f6e3 perftweaks: revert new computehidden cache functionality
Summary:
The computehidden caching functionality doesn't seem to be correct, and breaks
my repository.  "hg log -r head()" now incorrectly reports some obsolete
commits, which causes various mercurial commands to crash.  (They incorrectly
think that these commits aren't obsolete, but their parents are obsolete, which
breaks things.)

This diff reverts the new functionality until it gets fixed.

Test Plan: Confirmed that "hg-dev pull" no longer crashes in my repository.

Reviewers: #mercurial, stash, durham, quark

Reviewed By: quark

Subscribers: net-systems-diffs@fb.com, yogeshwer, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4676573

Tasks: 16558840

Signature: t1:4676573:1489008220:f730da69ec0f8d7220cad16ea6956944cfeb670d
2017-03-08 13:42:01 -08:00
Arun Kulshreshtha
9df08f6f42 fbamend: ensure that restack obsmarkers have correct metadata
Summary: Previously, we were relying on the `allowunstable` extension to wrap various instability-inducing commands and set the appropriate config option to cause the `tweakdefaults` extension to write the correct operation metadata into the obsmarkers produced by each respective command, causing commits to show up as "amended as", "rebased as", etc, in `hg sl`. Since the operation name was based on the command being run, if a user ran `hg amend --fixup` instead of `hg restack`, for example, smartlog would say "amended as" instead of "rebased as". This diff fixes the problem by always setting the operation to "rebase" during a restack.

Test Plan:
1. Create a stack of two commits.
2. Amend the bottom one without `--rebase`.
3. Run `hg amend --fixup` with the `fbamend.userestack` config option enabled.
4. Run `hg sl --hidden` and observe that the rebased commit is marked as "rebased".

See test file for an example.

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4670066

Tasks: 16529478

Signature: t1:4670066:1488991076:58c139c50a1aab2aa4fc7e28b1efd57a67bdfb08
2017-03-08 11:30:14 -08:00
Jun Wu
84143416ed sigtrace: new extension provides stack traces on signal
Summary:
Sometimes the hg process gets stuck, and we want to get the stack traces to
learn what it's doing. It's not a big issue if gdb can be used with python
debugging support [1]. However, that feature could not be easily set up on
OS X. This extension will make debugging on OS X easier by providing the
Python stack traces on SIGUSR1.

It's similar to OpenStack's "GuruMeditationReport" feature [2].

Unlike `contrib/showsstack.py`, this extension uses SIGUSR1 instead of SIGQUIT,
and writes to a file, instead of stderr, and prints all threads, instead of
just the current one. So it's more practically useful. We may want to
replace `showstack.py` eventually.

[1]: https://wiki.python.org/moin/DebuggingWithGdb
[2]: https://wiki.openstack.org/wiki/GuruMeditationReport

Test Plan: Added a new test

Reviewers: #sourcecontrol, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4660938

Signature: t1:4660938:1488927878:c751856681816a739160c361ed5cc10bab325000
2017-03-07 17:25:40 -08:00
Durham Goode
6434f61b55 sparse: update hg.updaterepo wrapper to match upstream
Summary:
Upstream changed the api of this function. Let's add args and kwargs to cover
future api changes.

Test Plan: Ran the tests

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: quark, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4667282

Signature: t1:4667282:1488910879:7db5764704dd345946f7f933e7e2c4a8fd77b699
2017-03-07 10:27:05 -08:00
Durham Goode
1cb65599a2 opener: replace repo.*opener with repo.*vfs
Summary:
Upstream has deprecated repo.*opener properties, so we need to switch to the
equivalent vfs apis to prevent us from being spammed with deprecation warnings.

Test Plan: Ran the tests

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4667252

Signature: t1:4667252:1488911019:efc3adf21f723151f1ee3b21b91fe62ea3c7b259
2017-03-07 10:27:05 -08:00
Arun Kulshreshtha
6b191cfb3e rage: fix and update 'hg rage' output
Differential Revision: https://phabricator.intern.facebook.com/D4664999
2017-03-07 01:27:12 -08:00
Simon Farnsworth
21802b82da histedit: set blockedtag when invoking user process
Summary: Improve dev_command_timers by removing some unknown_system sources

Test Plan:
testinprod

But also run-tests.py -j40

Reviewers: #sourcecontrol, mitrandir

Reviewed By: mitrandir

Subscribers: mitrandir, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4658516

Signature: t1:4658516:1488812254:474cf4033fc8e59c3eadf41bf9b3218a6ff4c236
2017-03-06 06:58:53 -08:00
Stanislau Hlebik
ba7fe90440 hgext3rd: newhiddencachekey extension
Summary:
Computing cache key can be quite slow (100ms and even more). Instead let's use
a simpler cache key: size + mtime of phaseroots, obsstore and changelod and
whether changelog write was delayed.

Test Plan:
1) arc unit

2) Run test-newhiddencache.t

3) Run upstream unittest with extension enabled.
python run-tests.py --extra-config-opt extensions.newhiddencachekey=/home/stash/facebook-hg-rpms/fb-hgext/hgext3rd/newhiddencachekey.py -j20

Verify that there are no failures except for those that show that new extension were added.
Note: also need to comment out debug output in the extension

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: durham, quark, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4604811

Signature: t1:4604811:1487958882:702ea8fdee16def674db97344832936d53f0cd9d
2017-03-06 01:18:58 -08:00
Stanislau Hlebik
68889ae8f4 fastpartialmatch: add amended commits to the index
Summary:
Upstream mercurial doesn't run hooks for temporary amended commits
(see https://bz.mercurial-scm.org/show_bug.cgi?id=3501 and
c2ca20984e3c9f30d73f0f35c35904b64edbd692). No other hook contains info about
temporary amended commits.

Let's wrap localrepository.commit() function and add missing commits inside
this wrapper.

Test Plan: arc unit

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: durham, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4634950

Tasks: 16387175

Signature: t1:4634950:1488399457:1d299c6ad3cfd418dda32a7ec20c60290f8d7778
2017-03-02 01:14:09 -08:00
Stanislau Hlebik
0e16b9b84d checkmessagehook: use reposetup to setup a hook
Summary:
It makes it easier to set up a hook - just enabling the extension will enable
the hook.

Test Plan:
Run perl script



  system("echo 1 >> 1");
  system("hg add 1");
  system("hg ci -m"."\x80");
  system("hg ci -m"."\x01");
  system("hg ci -m ok");

make sure only one commit was created

Reviewers: #sourcecontrol, mjpieters

Reviewed By: mjpieters

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4604980

Tasks: 16212973

Signature: t1:4604980:1487944300:8e4e7f340c746237b47773fc81759025e97b919e
2017-03-02 01:05:19 -08:00
Stanislau Hlebik
f008b056b4 fastpartialmatch: do not use fastpartialmatch in bundlerepos
Summary:
bundlerepo adds new commits that are not present in fastpartialmatch index.
Without this diff any access to the commit from bundle fails if
fastpartialmatch.raiseifinconsistent config option is set.

Let's not use fastpartialmatch in bundlerepos

Test Plan: arc unit

Reviewers: #mercurial, mjpieters

Reviewed By: mjpieters

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4635368

Tasks: 16394593

Signature: t1:4635368:1488393506:15b8f6a9fddabe681de7fd1296bbfafb9512f558
2017-03-02 00:53:09 -08:00
Stanislau Hlebik
098f1ee4b6 fastpartialmatch: remove changectx.__init__() wrapper
Summary:
Turned out that wrapping `changectx.__init__()` doesn't give any performance
benefits but instead it breaks initializing of changectx. For example, if
there is a commit which revision number N is a prefix of another commit A then
`hg log -r N` outputs commit A instead of commit N. Let's get rid of this
wrapping.

Test Plan: arc unit

Reviewers: #mercurial, mjpieters

Reviewed By: mjpieters

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4635311

Tasks: 16394593

Signature: t1:4635311:1488393427:eae091a0d155df336adbb122f2881a28e95ba6b6
2017-03-02 00:37:17 -08:00
Scott Ragan
598c76eb7b creates hg sparse --clear-rules option
Summary:
The sparse profile in .hg/sparse currently only can grow, removing rules from this must either be done manually (by editing the file), or one by one using the the -d/--delete command line switch.

To mirror the hg sparse --import-rules command line, hg sparse --clear-rules removes local includes/excludes rules.

Test Plan: fb-hgext/tests/test-sparse-clear.t

Reviewers: marianomartin, mjpieters

Reviewed By: mjpieters

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4609102

Tasks: 16073155

Signature: t1:4609102:1488028035:f5f7b7420d911667fb4a07ac770b7e0480a76977
2017-02-27 10:53:50 -08:00
Durham Goode
f1d376b16f debuginhibit: remove warning if inhibit is not enabled
Summary:
We have users who don't use inhibit, so printing this message clutters their UI.
Let's just move this debug.

Test Plan: yolo

Reviewers: kulshrax, #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4614036

Signature: t1:4614036:1487986024:6945b057aae62fc7a9fded9474d47ba134496e5a
2017-02-25 15:28:15 -08:00
Durham Goode
2bc65bd9db hiddenerror: fix out of bounds error
Summary:
The current code checks if the filter error reports a rev number, and if it does
tries to convert it to a node. If the 'number' is actually just a short form of
a hash that happens to be all digits, then it may be a number greater than the
size of the changelog, so we need to not attempt to access it in the changelog,
otherwise we get an IndexError.

Test Plan: Added a test, it failed before with a weird error

Reviewers: #mercurial, kulshrax

Reviewed By: kulshrax

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4613988

Signature: t1:4613988:1488027938:4f5ffcb07aeecbaf59cad20fb3a592dd92ed2beb
2017-02-25 15:26:19 -08:00
Stanislau Hlebik
fa422ea77d fastpartialmatch: remove useless code
Summary:
`changectx.__init__()` always returns None since it's an `__init__()` func.
Next comparison is useless

Test Plan: arc unit

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4608204

Signature: t1:4608204:1487928626:7205b9e7d9c931692337c3a1aa33451104b05fcd
2017-02-24 05:34:22 -08:00
Jun Wu
635683876d codemod: fix compatibility with the upstream revset refactoring
Summary:
`mercurial.revset` was recently split into `revset + revsetlang + smartset`.
Update our code accordingly.

D4604848 has fixed the `revsetlang` part. This patch fixes the remaining
`smartset` part.

Also fixes some test failures introduced by D4547080.

Test Plan: `arc unit`

Reviewers: #mercurial

Subscribers: jeroenv, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4595417
2017-02-23 13:09:59 -08:00
TJ
01fdb5bf0d edrecord: introduce interactive commits with an editor interface
Summary: creating a mercurial extension so that `hg record` functionality can be used via a text editor (instead of interactive command prompt or curses menu)

Test Plan: `$ run-tests.py test-edrecord.t`

Reviewers: durham, skreyen, most, #sourcecontrol, asriram, simonfar

Reviewed By: simonfar

Subscribers: quark, rmcelroy, simonfar, mjpieters, #nuclide

Differential Revision: https://phabricator.intern.facebook.com/D4547080

Tasks: 15728734

Signature: t1:4547080:1487842609:8855e35010cac3b6d2b136a80d7effa369eb1da8
2017-02-23 11:27:52 -08:00
Stanislau Hlebik
31e3ca4aa3 fb-hgext: make compatible to upstream changes
Summary:
c6cf3ac8c9bb811afa5b6eb5b69b29d128a7476a made revset API change.
This diff fixes our extensions

Test Plan: arc unit

Reviewers: #sourcecontrol, mjpieters, ikostia

Reviewed By: ikostia

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4604848

Signature: t1:4604848:1487851636:1c7618e9050f607e2d8b47ed495a349b877ed48a
2017-02-23 04:08:18 -08:00
Martijn Pieters
1752a2ba84 Send a single string (normal) to scuba
Summary: The data was being sent to scuba, but then ignored altogether. Turn this into a single string instead.

Test Plan: arc unit

Reviewers: #sourcecontrol, andrasbelo

Reviewed By: andrasbelo

Subscribers: mjpieters, andrasbelo

Differential Revision: https://phabricator.intern.facebook.com/D4598196

Signature: t1:4598196:1487784582:8952e0e386325d1635029aab2b9d97e9284376ea
2017-02-22 14:36:28 -08:00
Stanislau Hlebik
47e9e5588c hgext3rd: add checkcommitmessage
Summary:
Let's add a script that can be used as a commit hook to prevent bad commit
messages from happenning.

Test Plan:
Run the following perl script:
  system("echo 1 >> 1");
  system("hg add 1");
  system("hg ci -m"."\x80");
  system("hg ci -m"."\x01");
  system("hg ci -m ok");

Output:
  fbcode/1 already tracked!
  transaction abort!
  rollback completed
  abort: decoding near '': 'utf8' codec can't decode byte 0x80 in position 0: invalid start byte!
  non-printable characters in commit message
  transaction abort!
  rollback completed
  abort: pretxncommit hook failed

Make sure that commit is made with "ok" message, but there are no commits with
bad commit messages

Reviewers: #sourcecontrol, tja

Reviewed By: tja

Subscribers: tja, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4597432

Tasks: 16212973

Signature: t1:4597432:1487761552:cf38eabf93374b0ec2feb653dd70033de25e6e0e
2017-02-22 04:50:24 -08:00
Jun Wu
8a407d22ea hiddenerror: fix Python 2.6 compatibility
Python 2.6 does not support `'{}' % x`, so change it to `'{0}' % x`.
2017-02-17 16:34:59 -08:00
Jun Wu
78b2c91c65 profiling: remove the extension
Summary:
The profiling extension is to log interactive time.

The upstream change 459366b580cf breaks the profiling extension as it could
no longer wrap `maybeprofile` in time. And @simonfar's
`ui.timeblockedsection` is going to be a superior solution. So just remove
the profiling extension and its tests.

Test Plan: Checked that the extension is not enabled in our config files.

Reviewers: simonfar, #mercurial

Subscribers: mjpieters, simonfar

Differential Revision: https://phabricator.intern.facebook.com/D4583087
2017-02-17 16:27:19 -08:00
Jeroen Vaelen
0ecf1322c6 fastlog-hg: use smartset to merge multi-path results
Summary:
97d0be4019ac in core broke the fastlog extension for multi-path
queries. We did not have a test to catch it. It was reported as broken by a
user.

Test Plan:
```
$ ../../hg/tests/run-tests.py test-fastlog.t
```

Reviewers: #sourcecontrol

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4579133

Tasks: 16154586
2017-02-17 09:01:50 -08:00
Mateusz Kwapich
7b859c681d whereami: add extension
Summary:
This command will allow us to expose simple api to all automation checking
working copy parent.  It's meant to be overridden by our wrapper to read the
dirsate directly without shelling out to hg (see D4454504 )

Test Plan: see tests

Reviewers: #mercurial, durham, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4454502

Signature: t1:4454502:1485441135:09902c6437a7af55dfa5c97e3165681de687231f
2017-02-17 16:01:02 +00:00