Commit Graph

2507 Commits

Author SHA1 Message Date
Jun Wu
c133169b8b dirsync: read ".dirsync" in the working copy as part of its config
Summary:
It'd be nice to have dirsync config in the repo, since it's better synced, and
easier to be modified.

Test Plan: Added a test case

Reviewers: #sourcecontrol, simonfar

Reviewed By: simonfar

Subscribers: simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4732054

Signature: t1:4732054:1489793650:ca6c63ef47a1d9ebd181fbaee5554975801324eb
2017-03-17 16:35:08 -07:00
Kostia Balytskyi
0c2d706810 remotefilelog: wrap lz4 imports for compatibility
Differential Revision: https://phabricator.intern.facebook.com/D4707115
2017-03-17 14:02:26 -07:00
Kostia Balytskyi
b96abc88d5 setup: on windows only try to build python stuff
Differential Revision: https://phabricator.intern.facebook.com/D4727605
2017-03-17 09:04:19 -07:00
Durham Goode
b7b2cf134a fastmanifest: fix bytearray issue with mdiff.textdiff
Summary:
Upstream converted all array.array usages to bytearray, and in doing so they
also changed mdiff.textdiff(array1, array2) to be
mdiff.textdiff(util.buffer(array1), util.buffer(array2)), which we did not do in
our internal implementation.

This fixes that.

Test Plan: Adds a test

Reviewers: #mercurial, quark, simonfar

Reviewed By: quark, simonfar

Subscribers: quark, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4724350

Signature: t1:4724350:1489697253:8f63350bfc30b3079bf38410054b4763942a82f6
2017-03-16 13:48:48 -07:00
stash@fb.com
d36fb477cb infinitepush: fix the build
Summary:
On macos sshpeer flushes remote output later and it causes tests to fail. Let's
add a call to cleanup() to make sure the output is flushed.

This diff also makes remotefilelog test more robust.

Test Plan: Run tests on macos

Reviewers: #mercurial, tja, mitrandir

Reviewed By: mitrandir

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4721276

Tasks: 15389402

Signature: t1:4721276:1489669109:4ce59f4a1d224d57dbb7c1eb341c4e6a659d2e8c
2017-03-16 08:35:15 -07:00
Stanislau Hlebik
6b7e94a53f infinitepush: add --all option to debugcheckbackup
Summary:
By default debugcheckbackup checks only one backup for the user.
With --all option it checks all backups for the user from all host/reporoots

Test Plan: arc unit

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: durham, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4700993

Tasks: 15389402

Signature: t1:4700993:1489548921:800d08f420acc8ef4f807ffd17b31837dbb3fe82
2017-03-16 02:22:39 -07:00
Stanislau Hlebik
3553b5e9bb infinitepush: do not restore bookmark if node is not present
Test Plan: arc unit

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4700979

Tasks: 15389402

Signature: t1:4700979:1489548965:bab126d96adc827be84a2fc8939f6c03b0ef6599
2017-03-16 02:22:39 -07:00
Stanislau Hlebik
280028dd61 infinitepush: add maxheadstobackup option
Summary:
Backing up very big repo can cause problems. And very often old commits are
not necessary at all. Let's add a config option to limit the number of heads
that are being backed up.

Test Plan: arc unit

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: durham, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4700965

Tasks: 15389402

Signature: t1:4700965:1489549163:4e2c121f01debd7b495486a1f3b062926873399d
2017-03-16 02:22:39 -07:00
Stanislau Hlebik
2faa6709ca infinitepush: change backup state file format
Summary:
Previously we saved bookmark hash and revision number as state of
the last backup. Storing last backed up revision is error-prone. I saw
a few corrupted backups where bookmarks pointed to non-backed up
nodes. Also this approach puts more pressure on mysql bundle index.
On every backup it first deletes all existing backup bookmarks and then it
writes new bookmarks. Even if user changed one head or one bookmark
(and that is the usual case) current approach still deletes bookmarks first
and then rewrites them.

Instead let's store all backed up heads and bookmarks locally in json format.
Json was chosen because it's simple and human-readable. Then during backup
let's compare last backed up state to the current state of the repo, and send
to the server only the difference.

Test Plan: arc unit

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: durham, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4700930

Tasks: 15389402

Signature: t1:4700930:1489613249:a34369bf53e0718c22258304493dfa27b578857f
2017-03-16 02:22:39 -07:00
Stanislau Hlebik
897faf2d06 infinitepush: refactor and rename _getbackupstate
Summary:
Let's rename _getbackupstate() to _downloadbackupstate() because it actually
downloads it from the server. Also return class instead of tuple

Test Plan: arc unit

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4713633

Tasks: 15389402

Signature: t1:4713633:1489612870:3439a6a99ef311bf96784b3b1e53d7e9b94c0713
2017-03-16 02:22:39 -07:00
Stanislau Hlebik
5ee9ad7feb infinitepush: add timestamp to the logs
Test Plan: arc unit

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: durham, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4700897

Tasks: 16474976

Signature: t1:4700897:1489550188:1c6346ff158cec32df92934b585fc9cb983397c6
2017-03-16 02:20:02 -07:00
Durham Goode
5903428735 remotefilelog: add metric logging for prefetch
Summary:
This adds ui.log() output for prefetch statistics. Extensions who hook into
ui.log() can now log this data to external metrics systems.

Test Plan:
Ran a hg prefetch with the config flags enabled, while ptailing the dev command
timer. Verified the result contained remotefilelogfetches*

```
CHGDISABLE=1 FB_HG_DIAGS=1 hg --config
extensions.remotefilelog=../fb-hgext/remotefilelog/ --config
sampling.key.remotefilelog.prefetch=perfpipe_dev_command_timers prefetch -r .~9
ptail -f perfpipe_dev_command_timers | grep durham
```

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4711096

Signature: t1:4711096:1489591144:1c91a4fbd118a3c10c2a2c68391c9f5b0dbcedf3
2017-03-15 20:57:32 -07:00
Jun Wu
ca4723ad22 patchrmdir: new extension to workaround rmdir kernel issues
Summary:
We have encountered a kernel issue where `rmdir` a non-empty directory may race
with other things and hang in kernel for a long time.

This patch changes `os.rmdir` to avoid `rmdir` non-empty directories. It is
written in Cython calling the low-level `readdir` libc friends to make overhead
minimal.

Test Plan: Added a new test

Reviewers: #sourcecontrol, clm, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, simpkins, osandov, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4716711

Tasks: 16647532

Signature: t1:4716711:1489627923:7c7432748c1fd8c070ce257bd172feebd3807f65
2017-03-15 18:55:48 -07:00
Durham Goode
91dc0a18cc fbmetaedit: fix lint breakage
test-check-code doesn't like sed -i, so let's switch to a tiny batch script.
Also fix the test importing the extension directly instead of relying on the
path to be right.
2017-03-15 14:17:28 -07:00
Mateusz Kwapich
1bcc10919d fbmetaedit: intoduce the extension
Summary:
It's just copy-paste from my mutable-history patch series that ended up as
persistent hotfix.

The wrapping logic is very simple: it redirects '--fold' calls to original
metaedit and handles all the rest.

Test Plan:
Included a big copy-pase from test-evolve.t preserving only cases that were
executing metaedit and modified for the smaller test repo (the metaedit tests
were at the end of evolve test file).

Reviewers: #sourcecontrol, quark, simonfar

Reviewed By: quark, simonfar

Subscribers: rmcelroy, quark, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4708017

Signature: t1:4708017:1489524494:d682aab857e8422bb1a23adc22dddfb9f505b6b3
2017-03-15 07:40:26 -07:00
Augie Fackler
fde49a1a56 datapack: don't depend on demandimport when cstore isn't available
We've got a goofy test binary that doesn't use demandimport, and this
was tripping it up.
2017-03-14 12:44:43 -07:00
Stanislau Hlebik
314a434baf infinitepush: better client-side logging
Summary: New logging will have rotation and will separate different users and repos

Test Plan: arc unit

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: durham, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4689191

Tasks: 16474976

Signature: t1:4689191:1489172091:d304d4b19ae6cf52f86c11f442fa760b50d1fdf9
2017-03-14 02:13:16 -07:00
Jun Wu
02d79f722e codemod: replace repo.join to repo.vfs.join
Summary: Upstream has deprecated `repo.join`. Let's use `repo.vfs.join` instead.

Test Plan: `arc unit`

Reviewers: #sourcecontrol, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4704018

Signature: t1:4704018:1489462391:532b15c24faa33d584f25bb9382c1ff4b2c0c483
2017-03-13 20:51:37 -07:00
Jun Wu
3440ef0a0d chistedit: use an alternative way to disable color
Summary:
The upstream has changed how color works. `chistedit` seems to use a very hacky
way to disable color. Let's use a safer and simpler approach.

Test Plan: Run chistedit with hg-committed.

Reviewers: #sourcecontrol, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4702229

Signature: t1:4702229:1489446376:f0c7b4d31eb62bb6aea4cfeb13d78be8c219374c
2017-03-13 16:07:21 -07:00
Durham Goode
ebaf947a6c fastmanifest: switch array.array to bytearray
Summary:
Upstream has switched the type here for better python 3 compatibility. Let's
match their change.

Test Plan: Tests now pass

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: quark, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4699371

Signature: t1:4699371:1489429276:9da9f64489ecaf94cd77599a0558c081cfe792c2
2017-03-13 11:22:18 -07:00
TJ
10a278c273 edrecord: fix binary change comment bug
Summary: The poorly formatted docstring was causing an extra blank line to be inserted both before and after the content

Test Plan: Changed a binary in an hg repo. Committed interactively using the editor. Made sure that there were no blank lines

Reviewers: rmcelroy, quark, simonfar

Reviewed By: simonfar

Subscribers: most, asriram, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4684549

Tasks: 15728734

Signature: t1:4684549:1489269897:faecd381c98e8c2b9884f9bacd18dfc32ee77758
2017-03-13 10:34:38 -07:00
Stanislau Hlebik
ddf10d0f6e infinitepush: add debugwaitbackup command
Summary:
For now these command will be used in test because
`wait_for_background_backup.py` tied to the backup state file. And every change
to this file requires change to wait_for_background_backup.py. Let's create
simple internal backup command.

Test Plan: arc unit

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: durham, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4667056

Tasks: 15389402

Signature: t1:4667056:1489024344:6c43241e42b7c418baa4e1542f303239ba887c45
2017-03-13 01:35:30 -07:00
Stanislau Hlebik
af7ad809cc infinitepush: add lock for backup command
Summary:
There can be multiple backup processes running at the same time. Since these
processes can be quite heavy it makes sense to limit them. Let's use lock file
to do that.

Test Plan: arc unit

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: durham, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4667020

Tasks: 15389402

Signature: t1:4667020:1489170813:f2a685b1c224c553d3ee004d89d3eeeca816e824
2017-03-13 01:32:32 -07:00
Durham Goode
b0c4b9a793 fbamend: unlink rebasestate after transaction
Summary:
Previously, the rebase logic itself would unlink its state file. Now that the
state file is part of the transaction, it gets serialized at the end of the
transaction, even if it was unlinked midway through the transaction. This
affects fbamend because it wraps various rebases in a higher transaction.

Ideally we would add support to the mercurial transaction framework to allow
marking a file as deleted halfway through the transaction, but this patch will
fix the tests until we get that upstream.

Test Plan: Tests now pass

Reviewers: kulshrax, #mercurial, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4696578

Signature: t1:4696578:1489348040:b12104b2519d69dab98d2076f640f9f3157036e2
2017-03-12 13:11:47 -07:00
Durham Goode
1dec377020 sqldirstate: add _otherparentset to match upstream
Summary: Upstream has added a new attribute to dirstate. Let's add it too.

Test Plan: Tests now pass

Reviewers: mitrandir, #mercurial, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4695923

Signature: t1:4695923:1489283439:c7876832af2764d2bbe9cd0f9e47199c84799609
2017-03-12 12:49:18 -07:00
Durham Goode
932cc0e5b1 grepdiff: fix to match upstream
Summary:
Upstream changed the format from ['diff ...\n', 'file1\nfile2\nchunks...'] to be
['diff ...\nfile1\nfile2\n', 'chunks...'] in 199440c7d1f49752. So grepdiff needs
to be updated.

Test Plan: Tests now pass

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4695911

Signature: t1:4695911:1489282635:920e88fecc5d6d4660d1d21289d1d07e26c93315
2017-03-12 12:49:18 -07:00
Durham Goode
c8fb813c3d treemanifest: add matcher arg to filesnotin
Summary:
Upstream has added a matcher arg to manifest.filesnotin, so let's support it in
our native treemanifest implementation.

Test Plan: Added unit tests for filesnotin with and without matchers.

Reviewers: #mercurial, stash

Reviewed By: stash

Subscribers: stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4677030

Signature: t1:4677030:1489076599:23ab67d85bb4fa689f05349253a4d2c4e7d496cc
2017-03-12 12:49:18 -07:00
Durham Goode
84a862e6f3 treemanifest: support matcher in diff
Summary:
Upstream has added a matcher argument to the diff API which allows diff to avoid
traversing certain parts of the tree. This adds support for that to our native
treemanifest implementation.

Test Plan: Added tests for diff with matches

Reviewers: #mercurial, stash

Reviewed By: stash

Subscribers: stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4677023

Signature: t1:4677023:1489076627:dbcea209d300a68fa050f68c52b4fd9949b85302
2017-03-12 12:49:18 -07:00
Durham Goode
a365f1d7b2 treemanifest: refactor path manipulation in diff
Summary:
A bunch of if statements were doing the same thing every time. This moves that
logic out of the individual if statements, which makes it cleaner and will make
a subsequent patch that runs a matcher against the paths easier.

Test Plan: Ran the tests

Reviewers: #mercurial, stash

Reviewed By: stash

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4677011

Signature: t1:4677011:1489075541:9379597b8866358ad5dad3f1f9ae00a0a0b523f9
2017-03-12 12:49:18 -07:00
Stanislau Hlebik
4b2050d789 logginghelper: add logging.configoptions
Summary: Let's add option that contains a list of other config options to log.

Test Plan: arc unit

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: durham, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4681336

Signature: t1:4681336:1489171115:dddd27032e945f37fcc846a511e3aaa9e83d29d3
2017-03-12 12:01:45 -07:00
Durham Goode
33a1d6afdc fix: update manifest.diff usages to handle match arg
Summary:
Upstream has added a new match argument to manifest.diff() and removed the
existing manifest.matches() function, so we need to update our internal usage.

A separate diff will update treemanifest to support the new diff() api.

Test Plan:
Ran the tests, some still fail because of the upstream changes, but
future patches fix those.

Reviewers: #mercurial

Differential Revision: https://phabricator.intern.facebook.com/D4677002
2017-03-12 10:14:35 -08:00
Arun Kulshreshtha
7174632e3c fbamend: add --towards flag to hg next
Summary: This diff adds a `--towards` flag to `hg next`. This flag tells `hg next` to always proceed in a linear fashion towards the specified commit. This is useful in situations where the user has a very branchy stack. For example, if the user has a local stack with several branches, each with a feature bookmark at the top, if the user wants to go up a particular stack they can do so with something like `hg next 3 --towards bookmark`.

Test Plan:
Here is an example of basic usage. Also see test file.

```
$ hg debugbuilddag "+5 *2 +2"
$ hg book -r 4 feature1
$ hg book -r 7 feature2
$ hg up 0
$ hg sl
o  92eaf3  debugbuilddag  feature2
|  r7
|
o  8bbe84  debugbuilddag
|  r6
|
o  914970  debugbuilddag
|  r5
|
| o  bebd16  debugbuilddag  feature1
| |  r4
| |
| o  2dc09a  debugbuilddag
| |  r3
| |
| o  012414  debugbuilddag
|/   r2
|
o  66f7d4  debugbuilddag
|  r1
|
@  1ea734  debugbuilddag
   r0
$ hg next 3 --towards feature1
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
[2dc09a] r3
$ hg sl
o  92eaf3  debugbuilddag  feature2
|  r7
|
o  8bbe84  debugbuilddag
|  r6
|
o  914970  debugbuilddag
|  r5
|
| o  bebd16  debugbuilddag  feature1
| |  r4
| |
| @  2dc09a  debugbuilddag
| |  r3
| |
| o  012414  debugbuilddag
|/   r2
|
o  66f7d4  debugbuilddag
|  r1
|
o  1ea734  debugbuilddag
   r0
```

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4695861

Signature: t1:4695861:1489325959:1145a29ba87f99d07ede1e81415804723dc48818
2017-03-12 10:09:12 -07:00
Stanislau Hlebik
2d7e54cf5a infinitepush: rename nodes to hexnodes
Summary: These are actually hexnodes

Test Plan: arc unit

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4667011

Tasks: 15389402

Signature: t1:4667011:1488991240:e0ba5e0bd58f753235e600ed31858bb565c62e73
2017-03-11 09:22:50 -08:00
David Soria Parra
83b308aa75 chistedit: make chistedit compatible with current tip 2017-03-10 14:11:11 -08:00
Durham Goode
67da0498fb treemanifest: block non-pushrebase pushes to treemanifest servers
Summary:
We are relying on pushrebase to replay each of the incoming commits so they
produce both flat and treemanifests. This patch blocks any commits that don't
come through pushrebase, by intercepting manifest.addgroup calls (which is what
a normal push would use) and throwing an exception.

Test Plan: Added a test

Reviewers: #mercurial, stash

Reviewed By: stash

Subscribers: stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4093745

Signature: t1:4093745:1478100170:79b1eea6668cd3efb27742323e11a45bde930440
2017-03-09 14:45:23 -08:00
Durham Goode
e2b422dc98 treemanifest: server - write trees at the same time as flat manifests
Summary:
When a server is a treemanifest enabled server, we want to ensure that every
commit that comes in will be added to the treemanifest as well as the flat
manifest. This patch accomplishes that by hooking into the addmanifest logic and
applying the new manifest to the tree as well.

Test Plan: Added a test

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4093738

Signature: t1:4093738:1478260442:58f77bd88ab09ccda4ac238a065388be8f4a271d
2017-03-09 14:45:23 -08:00
Durham Goode
911ea4ace5 treemanifest: add backfilltree command
Summary:
This adds a command for producing tree manifests for existing manifests in the
repository. The tree manifests are stored similar to how vanilla mercurial
stores them, but the root revs are in 00manifestree.i instead of 00manifest.i
and we fake the root node to be the same as the flat manifest node, so we can
reference either the flat or the tree easily.

Test Plan: Added a test. Also began converting fbsource

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4092252

Signature: t1:4092252:1478260394:a55360a8ba83ad85b1694cb93f0788f1f672ef61
2017-03-09 14:45:23 -08:00
Durham Goode
29dc858324 treemanifest: create a separate manifest revlog on the server
Summary:
As part of the migration, we need both flat and tree manifests to exist at once
on the server. To do this, we need to store the tree manifest root revisions
somewhere other than 00manifest.i.

This patch divides the treemanifest logic into client and server bits, and on
the server creates a repo.manifesttree property that contains the tree manifest.
Future patches will write to this to fill in new tree entries, and eventually
read from it to seed clients with tree data.

Test Plan:
Future patches add tests that for features that use this
functionality

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4092217

Signature: t1:4092217:1478260318:7edada2ea39a5de565955a4180bfad62f6126352
2017-03-09 14:45:23 -08:00
Jun Wu
5c350fb239 test-sigtrace: fix OSX compatibility
"ls" has different exit code on OSX, add "|| false" to normalize it.
2017-03-08 22:38:23 -08:00
Adam Simpkins
143d13f6e3 perftweaks: revert new computehidden cache functionality
Summary:
The computehidden caching functionality doesn't seem to be correct, and breaks
my repository.  "hg log -r head()" now incorrectly reports some obsolete
commits, which causes various mercurial commands to crash.  (They incorrectly
think that these commits aren't obsolete, but their parents are obsolete, which
breaks things.)

This diff reverts the new functionality until it gets fixed.

Test Plan: Confirmed that "hg-dev pull" no longer crashes in my repository.

Reviewers: #mercurial, stash, durham, quark

Reviewed By: quark

Subscribers: net-systems-diffs@fb.com, yogeshwer, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4676573

Tasks: 16558840

Signature: t1:4676573:1489008220:f730da69ec0f8d7220cad16ea6956944cfeb670d
2017-03-08 13:42:01 -08:00
Arun Kulshreshtha
9df08f6f42 fbamend: ensure that restack obsmarkers have correct metadata
Summary: Previously, we were relying on the `allowunstable` extension to wrap various instability-inducing commands and set the appropriate config option to cause the `tweakdefaults` extension to write the correct operation metadata into the obsmarkers produced by each respective command, causing commits to show up as "amended as", "rebased as", etc, in `hg sl`. Since the operation name was based on the command being run, if a user ran `hg amend --fixup` instead of `hg restack`, for example, smartlog would say "amended as" instead of "rebased as". This diff fixes the problem by always setting the operation to "rebase" during a restack.

Test Plan:
1. Create a stack of two commits.
2. Amend the bottom one without `--rebase`.
3. Run `hg amend --fixup` with the `fbamend.userestack` config option enabled.
4. Run `hg sl --hidden` and observe that the rebased commit is marked as "rebased".

See test file for an example.

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4670066

Tasks: 16529478

Signature: t1:4670066:1488991076:58c139c50a1aab2aa4fc7e28b1efd57a67bdfb08
2017-03-08 11:30:14 -08:00
Jun Wu
84143416ed sigtrace: new extension provides stack traces on signal
Summary:
Sometimes the hg process gets stuck, and we want to get the stack traces to
learn what it's doing. It's not a big issue if gdb can be used with python
debugging support [1]. However, that feature could not be easily set up on
OS X. This extension will make debugging on OS X easier by providing the
Python stack traces on SIGUSR1.

It's similar to OpenStack's "GuruMeditationReport" feature [2].

Unlike `contrib/showsstack.py`, this extension uses SIGUSR1 instead of SIGQUIT,
and writes to a file, instead of stderr, and prints all threads, instead of
just the current one. So it's more practically useful. We may want to
replace `showstack.py` eventually.

[1]: https://wiki.python.org/moin/DebuggingWithGdb
[2]: https://wiki.openstack.org/wiki/GuruMeditationReport

Test Plan: Added a new test

Reviewers: #sourcecontrol, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4660938

Signature: t1:4660938:1488927878:c751856681816a739160c361ed5cc10bab325000
2017-03-07 17:25:40 -08:00
Durham Goode
5dac028b0e cstore: move pythonutil to cstore
Summary:
Now that ctreemanifest no longer depends on python.h, let's move pythonutil over
to cstore where all the python code is.

Test Plan: Ran the build and the tests

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4663988

Signature: t1:4663988:1488895919:652b3fc35a2dd12c51a9f70e32997c7b4d037c95
2017-03-07 11:39:46 -08:00
Durham Goode
9b3457e955 ctreemanifest: replace PythonObj with DiffResult in treemanifest
Summary:
This is the last piece of removing the Python dependency from the core
treemanifest code. This replaces the old PythonObj diff dictionary with a new
DiffResult class that has a PythonDiffResult implementation.

Test Plan:
Ran the test suite, including the unit tests that explicitly cover
treemanifest diff from python.

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4663984

Signature: t1:4663984:1488889615:62f064924b0d6b45dfa7e490d19418060c374f40
2017-03-07 11:39:46 -08:00
Durham Goode
8eede45c7f cstore: add Matcher class
Summary:
As part of breaking the native cstore implementation away from Python, let's
create a Matcher class that can be used to perform path match testing. Initially
the only implementation is the PythonMatcher which just wraps a python match
object.

Test Plan: Covered by existing matcher tests in cstore-treemanifest.py

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4663354

Signature: t1:4663354:1488914384:2c33c7e0e7f2eade0786b6ff41503317989fd1e5
2017-03-07 11:39:46 -08:00
Durham Goode
9dbf0996eb treemanifest: add visitdir check during matches
Summary:
The upstream matcher has a 'visitdir' function which indicates whether a walk
should traverse into a directory. Let's support this in treemanifest so we can
skip a ton of lookups for parts of the tree that aren't relevant to the match.

Test Plan:
Used gdb to verify commands like 'hg diff -r .^ -r .
--include foo' only traversed into directory foo, and not directory bar. I'm not
sure how to write a test for this since it doesn't actually affect user facing
output.

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4648252

Signature: t1:4648252:1488882860:b5e1b738fef2f63336c80cd7c0eb59c2fcf16d0f
2017-03-07 11:15:26 -08:00
Durham Goode
70ce116529 treemanifest: add history data to tree repacks
Summary:
Previously, tree repacks did not take into account tree history. It would just
look at the delta base and if the base existed, it would just reuse the delta.
This would A) result in very long chains, and B) result in chains where the full
text was the oldest version, instead of the newest (recent full texts means
faster access to recent versions).

This patch threads tree history into the repacker, which already knows how to
use history for repacks.

Test Plan:
Updated the tests, and inspected the new test results to ensure tree
entries that were not deltas before the repack became reverse deltas during the
repack.

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4647359

Signature: t1:4647359:1488882710:dba72cf488766ce827b7641735164fa0efc9a303
2017-03-07 11:15:26 -08:00
Durham Goode
3f65bd2532 treemanifest: don't build pack files that depend on other pack files
Summary:
Previously, when treemanifest would create packs of trees during pull, we
allowed trees to be delta'd against trees in other packs. This resulted in
smaller packs, but if the other pack disappeared for some reason (since it's a
cache), the chain broke.

This patch ensures that the first version of every tree added to a pack is a
full text.

This temporarily makes repacks worse, since the repacker doesn't know about
history to produce deltas when combining packs. The next patch adds history
awareness which improves the repack deltafication.

Test Plan:
Updated the tests, and inspected the new test results to ensure that
all packs only had deltas within the pack.

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4647348

Signature: t1:4647348:1488882214:e850622a853a534fc60caeef604c88c30740c60d
2017-03-07 11:15:25 -08:00
Durham Goode
5bc368c71d treemanifest: move tree delta logic up to python layer
Summary:
Previously the treemanifest code itself would create the text deltas when
writing a tree out. This meant we couldn't make the delta decision based on
other data, like if the p1 commit was in the same pack file.

This patch removes treemanifest.write() and moves all calls over to
treemanifest.finalize() which gives the python/pack layer control over delta
choices. A future patch will use this to ensure tree packs always contain
complete delta chains.

Test Plan: All tests pass

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4645942

Signature: t1:4645942:1488880851:d0c8c902e7e849072a53344630a9184b6d8e1e7f
2017-03-07 11:15:25 -08:00
Durham Goode
cc11141ce5 treemanifest: add new finalize method
Summary:
Previously, to serialize a tree we would call treemanifest.write() which would
take the stores and call the appropriate add() functions. This meant the
treemanifest code controlled when deltas were used, which makes it hard to
decide deltas based on external factors, like which pack file the parent is in.

As a first step to fixing this, we're adding a new finalize function that
returns a python iterator that allows iterating over the new parts of the tree.
In a future diff we will use this to do serialization and delta decisions at the
python/pack layer instead.

The same future diff will also cover this in tests, as it moves all calls to
treemanifest.write() to use treemanifest.finalize().

Test Plan: The next diff switches all write() calls and tests to finalize()

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4645922

Signature: t1:4645922:1488880722:574a37ba800f4b78faf8bad3979b447f4278cc55
2017-03-07 11:15:25 -08:00