Commit Graph

43127 Commits

Author SHA1 Message Date
Mateusz Kwapich
c74ac7db7f print the first line of hidden commit message
Summary: This way it's even more clear

Reviewed By: ryanmce

Differential Revision: D7585130

fbshipit-source-id: ce9e535f88f794fd19d7fd3db5171263fcd7865a
2018-04-13 21:51:54 -07:00
Adrian Catană
4bfc90f422 hg: log generaldelta and remotefilelog in sampling
Summary: Mercurial adds new requirements (generaldata + remotefilelog) to the temp file created by the wrapper. Needed for Rust migration

Differential Revision: D7586451

fbshipit-source-id: d9c3454055a76cafa7a0f93238bc837eba8e4a48
2018-04-13 21:51:54 -07:00
Martijn Pieters
1e0608239d fbclone: add sparse stats output
Summary:
These stats are hard-coded for now but will in a future diff be fetched from a regularly-updated source.

```
fbclone fbsource --dry-run
usage: fbclone REPO [options]
fbclone: error: fbsource is too large to clone without a sparse profile
Please use one of the following profile switches:

--fbandroid: The full FBAndroid profile, for general development on Android apps.
    file count: 644324 (35.83%)  size 10.3 GB

--fbcode: The full FBCode profile, for general development on backend code.
    file count: 950490 (52.85%)  size 14.8 GB

--fbobjc: The full FBObjC profile, for general development on iOS apps.
    file count: 620154 (34.48%)  size 9.18 GB

--xplat: The base profile for xplat development, separate from other profiles that include xplat
    file count: 269858 (15.01%)  size 4.41 GB

Or use --everything to get a full-size working copy
```

Differential Revision: D7502448

fbshipit-source-id: e37e4e31d355251e9dd2c390e3de7643fa38b80b
2018-04-13 21:51:54 -07:00
Martijn Pieters
abd2175e71 hg: actually run the tests in fb/tests
Summary:
There are tests in fb/tests we don't run as part of the test suite. Update the makefile and add buck infrastructure to run them.

 - Update the test-common-commands-hg.t test
 - Re-generate the fbcode/.hgignore file from the updated .gitignore

Reviewed By: quark-zju

Differential Revision: D7584511

fbshipit-source-id: d85800dccf0eb569a68db4b9e1d9796e3d7ac957
2018-04-13 21:51:54 -07:00
Phil Cohen
487a3fb991 repogenerator: add new files and directories over time
Summary:
The megarepo generator now adds new directories and files automatically. We now pick folders and paths randomly, with a configurable number of subdirectories and a configurable length for the leaf filename, and a fixed alphabet.

The array of directories and leaf filenames are prorated: they start out constrained and grow as we approach our desired number of commits to generate (`repogenerator.numcommits`), so as not to be too sparse initially.

The directory array is randomized initially, so the above proration doesn't lead to a trivial edit pattern (e.g. everything in a/* initially.)

Lastly, we now stop generation when we hit our goal.

Reviewed By: quark-zju

Differential Revision: D7572968

fbshipit-source-id: b60d60b6262962ee114cb19b0989e6a1af8c0549
2018-04-13 21:51:54 -07:00
Durham Goode
3e7392d050 hg: fix authormap name corruption in hgsubversion
Summary:
The AuthorMap in hgsubversion is currently derived from BaseMap which
has some strange behavior. During long running conversions, this caused some
names to be mangled.  For instance, if a long running conversion encounter a
commit with author 'abc' with converted name 'abc@uuid' and then later an author
'abcx' came along, it would convert the later name to 'abc@uuidx' because
BaseMap would match the prefix 'abc' with the beginning of the author name and
then reuse 'abc@uuid'. It then swapped the substring 'abc' from 'abcx' with
'abc@uuid' which resulted in the corrupt 'abc@uuidx'.

The fix is to add beginning and end markers to the regex name. The regex support
seems to be based around the ability to define custom regular expressions in
files to manipulate author/branch/tag names. In those cases the dictionary
__setitem__ call is passed a regular expression as a value. Since this patch
only affects the case where a string is passed, the regular expression scenario
will still work.

Reviewed By: singhsrb

Differential Revision: D7592383

fbshipit-source-id: 250b32d56782a6ec905cbd6620e0289fbdb523f0
2018-04-13 21:51:53 -07:00
Durham Goode
c679adb7d0 hg: remove bundlev1 from the supported outgoing versions
Summary:
Removes bundlev1 from the supported outgoing versions, but keeps a flag
around to force enable it if tests need it.

Reviewed By: quark-zju

Differential Revision: D7591176

fbshipit-source-id: 280cbbbe87848e3d6c9d448ce4f87c5eadeff720
2018-04-13 21:51:53 -07:00
Durham Goode
0f716a4f57 hg: remove test dependency on bundle1
Summary:
Bundle1 is being deprecated. These tests use it but don't need to, so
let's change them to use bundle2.

Reviewed By: quark-zju

Differential Revision: D7591178

fbshipit-source-id: 3fd46d3bbab9f7acd2f63dfb4f10c21b6d2c35e9
2018-04-13 21:51:53 -07:00
Durham Goode
a2f2208f71 hg: update commit interactive test to not use bundle1
Summary:
The commit interactive test was creating a bundle to test committing
binary files. It had no reason to need bundle1, so let's update it to bundle2.

Reviewed By: quark-zju

Differential Revision: D7591170

fbshipit-source-id: 964cd2cfc4368a89c956b30389ca74ab0b87a486
2018-04-13 21:51:53 -07:00
Durham Goode
6750ab25b0 hg: allow bundle1 for hg web tests
Summary:
These tests use hard coded url's that don't include bundlecaps, so it's
difficult to convert them to use the right bundlecaps for bundle2.

Reviewed By: quark-zju

Differential Revision: D7591165

fbshipit-source-id: 3424c448be8c713cac5b876438745414a59b7230
2018-04-13 21:51:53 -07:00
Durham Goode
f96219ef87 hg: update test-patchbomb to not use bundle1
Summary: We're deprecating bundlev1, so let's update this test to not use it.

Reviewed By: quark-zju

Differential Revision: D7591169

fbshipit-source-id: c3bd3496e46c57365beb6cc8380ed6d13110f7b3
2018-04-13 21:51:53 -07:00
Durham Goode
a54b05ee58 hg: use bundle2 for treemanifest backfill
Summary:
We weren't passing bundlecaps to the backfill pull, so we were getting
bundle1. Let's fix that

Reviewed By: quark-zju

Differential Revision: D7591167

fbshipit-source-id: 111e2967190dca17764c84e6d7be850644f222fd
2018-04-13 21:51:53 -07:00
Durham Goode
ba5c4cc4e9 hg: update tests to not use bundle1
Summary: We're deprecating bundle1, so let's update some tests.

Reviewed By: quark-zju

Differential Revision: D7591173

fbshipit-source-id: 806dfab377d2416e9fc8fe34bfebcaaf854e5c11
2018-04-13 21:51:53 -07:00
Durham Goode
db081e2c0c hg: update test-bundle2-format to not use bundlev1
Summary: We're deprecating bundlev1. Let's update this test.

Reviewed By: quark-zju

Differential Revision: D7591171

fbshipit-source-id: 0f840d318937016bec22d42577ab882eef9bedc9
2018-04-13 21:51:53 -07:00
Durham Goode
0ae182a740 hg: default to using bundle2 for pushrebase parts
Summary:
Pushrebase previously defaulted to bundlev1. Now that the server can
support v2, let's default to v2.

Reviewed By: quark-zju

Differential Revision: D7591174

fbshipit-source-id: bc5448490ff319543baedf3f5a1aab160a73ed27
2018-04-13 21:51:53 -07:00
Durham Goode
e02f8031d7 hg: change default bundle version to v2
Summary:
Bundlev1 is old and really shouldn't be used anywhere. Let's default to
v2.

Reviewed By: quark-zju

Differential Revision: D7591172

fbshipit-source-id: 2699e0b4dd8d1c1951f9dd92f0d9d300d935a04b
2018-04-13 21:51:53 -07:00
Durham Goode
24653e5e32 hg: add develwarn for bundle1 format
Summary:
We want to deprecate the bundlev1 format, so let's start by adding a
develwarn. Later diffs will update the tests to not use v1, then remove v1 as a
supported outgoing bundle entirely.

Reviewed By: quark-zju

Differential Revision: D7591166

fbshipit-source-id: 143ad029bfe4d141f91d6d5077342dfa44ad2944
2018-04-13 21:51:52 -07:00
Jun Wu
8e1acfaaa4 sshaskpass: disable echo more aggressively
Summary: Disable echo for "Enter passphrase for key".

Reviewed By: phillco

Differential Revision: D7592208

fbshipit-source-id: 79afb7bb49d952239571c1ddb8d3f5c32071d49b
2018-04-13 21:51:52 -07:00
Liubov Dmitrieva
4affcfaa2a infinitepush: MRU order of listing hosts and repo roots
Summary:
infinitepush: MRU order of listing hosts and repo roots to show host/repos with most recent first

this is for improving ux for those who have a lot of backups (T27713908)

Reviewed By: StanislavGlebik

Differential Revision: D7568128

fbshipit-source-id: 65bb6a4759298268fbd3109df57a2b9e198bd4c0
2018-04-13 21:51:52 -07:00
Mateusz Kwapich
e6ed96017c add --cleanup option
Summary:
This is meant to replace `arc cleanup`. Normally the commits corresponding
to landed Differential Revisions disappear on `hg pull` unless:
 * there's a bookmark pointing to a commit
 * the commit is currently checkout out

This command makes it easy to deal with such commits. Just run `hg hide -c`

Reviewed By: ryanmce

Differential Revision: D7350794

fbshipit-source-id: 3b77acbca01edc5bffda53023687f268dc8edb08
2018-04-13 21:51:52 -07:00
Mateusz Kwapich
323157e712 add a hint about hg undo to hide
Summary:
I know that `hg undo` is useful for undoing most of the commands
but hide is the one that a bit more sensitive because it's
the command to remove commits from your smartlog.

Also I'm planning on introducing additional, more powerful options
for hide and that could be useful for undoing them.

Reviewed By: ryanmce

Differential Revision: D7350795

fbshipit-source-id: fbc42674fe0ca4ac6300c08095897f624d41a2f3
2018-04-13 21:51:52 -07:00
Mateusz Kwapich
d2c8503e6f make hide more verbose
Summary:
Users would feel more comfortable using this command if they knew what it's
doing for them. Use `--quiet` to disable verbosity

Reviewed By: quark-zju

Differential Revision: D7380617

fbshipit-source-id: 0de46a858ec9c1c4a8cd9a55a5760bdcd8723424
2018-04-13 21:51:52 -07:00
Liubov Dmitrieva
e745c7b870 commitcloud: add config option that enables logging of requests
Summary:
add config option for very verbose output (print the requests)

this option is intended for troubleshooting

and it can greatly simplify troubleshooting

Reviewed By: StanislavGlebik

Differential Revision: D7572794

fbshipit-source-id: c0c5cfcc22ebc7799b02abd7bb08dc6a9bc4c6cd
2018-04-13 21:51:52 -07:00
Mark Thomas
9979530751 sshpeer: add timeblockedsection for ssh setup
Summary:
We would like to know how much time is spent setting up ssh connections.  Add a
timeblockedsection to sshpeer that records the amount of time between starting
the ssh command, and getting (and validating) the response for the `hello` and
`between` commands.

Reviewed By: ryanmce, farnz

Differential Revision: D7584383

fbshipit-source-id: fd3c48dc57e0ebbafc191c235355ce2330c6bd61
2018-04-13 21:51:52 -07:00
Kostia Balytskyi
e53224ec32 hg: add even more debug messaging around locks
Summary: We've seen a case when a malformed `wlock.break` file prevented the stale lock file from being deleted. It seems unsafe to just delete the `wlock.break`, but we can add more debug messaging before it, so that rerunning a command with `--debug` would tell us what is going on.

Reviewed By: quark-zju

Differential Revision: D7572510

fbshipit-source-id: 5456ae6dbff3721bbd40c6ed55e173beabac3f65
2018-04-13 21:51:52 -07:00
Alexandre Marin
c8a36c3d0c rm scm/hg/hgext/p4fastimport/filetransaction.py
Summary:
We haven't used this for over a month, we're alive and well.
The raccoon asked me to remove this:
codecleanupracoon

Differential Revision: D7578280

fbshipit-source-id: ee522203095a213dc90d6146ce720b473ab5afe5
2018-04-13 21:51:52 -07:00
Alexandre Marin
dcfdac9963 seqimport - test moving files in/out of client view
Summary: Adds a more thorough test for D7574415

Differential Revision: D7575302

fbshipit-source-id: cdd98f738da5d78331f7e761b940897a8761bfcc
2018-04-13 21:51:52 -07:00
Alexandre Marin
1fde98d0db seqimport - make it survive CLs that have moves outside of clientspec
Summary:
seqimport currently has a gap: if a changelist touches files outside of
clientspec, it will blow up when trying to get the move info for it. Even if that
was not the case, it could blow up if the file was moved into the clientspec.

This change makes it resilient to that by providing the same list created by
`parse_fstat` to prefilter files we check for move info, and then making
`parse_where` take an optional parameter saying it is fine if file is not on
client (i.e. moved into clientspec)

Differential Revision: D7574415

fbshipit-source-id: 63f6a32436d3d53d6f9402575a9a13bb4187b76c
2018-04-13 21:51:52 -07:00
Jun Wu
5346a46bb5 rebase: on abort, don't strip commits that didn't need rebased (issue5822)
Summary:
Backports:

```
  # HG changeset patch
  # User Martin von Zweigbergk <martinvonz@google.com>
  # Date 1521175251 25200
  #      Thu Mar 15 21:40:51 2018 -0700
  # Branch stable
  # Node ID a046d6890761b6a99475f0e5bf9cce74715897ec
  # Parent  b9a6ee2066f9636265d812a3a8b82b02bc3223aa
  rebase: avoid defining two lists with the same contents

  In abort(), there's "dstates" and "rebased" that are identical, which
  they seem to have been since 0806823370d8 (rebase: properly calculate
  descendant set when aborting (issue3332), 2012-03-22). Let's
  de-duplicate. I don't know what "dstates" means, but "rebased" makes
  sense (it's the list of rebased revisions), so let's pick that.

  Differential Revision: https://phab.mercurial-scm.org/D2878

  # HG changeset patch
  # User Martin von Zweigbergk <martinvonz@google.com>
  # Date 1521178507 25200
  #      Thu Mar 15 22:35:07 2018 -0700
  # Branch stable
  # Node ID 177f3b90335fc49c23a8880de57dc865f0832849
  # Parent  a046d6890761b6a99475f0e5bf9cce74715897ec
  rebase: on abort, don't strip commits that didn't need rebased (issue5822)

  I clearly missed adding this condition in 78496ac30025 (rebase: allow
  rebase even if some revisions need no rebase (BC) (issue5422),
  2017-05-11). Perhaps I should have opted for the "revdone" solution I
  mentioned there...

  Differential Revision: https://phab.mercurial-scm.org/D2879
```

Differential Revision: D7517675

fbshipit-source-id: c08206700f2a26a36785adf9df93176449a58376
2018-04-13 21:51:51 -07:00
Jun Wu
1caa774e95 amend: abort if unresolved merge conflicts found (issue5805)
Summary:
Backport from:

```
  # HG changeset patch
  # User Yuya Nishihara <yuya@tcha.org>
  # Date 1520766638 -32400
  #      Sun Mar 11 20:10:38 2018 +0900
  # Branch stable
  # Node ID eeb87b24aea7f547f6d95b812dd080dc6e9ab194
  # Parent  9639c433be54191b4136b48fe70fc8344d2b5db2
  amend: abort if unresolved merge conflicts found (issue5805)

  It was checked by repo.commit() before e8a7c1a0565a "cmdutil: remove the
  redundant commit during amend."
```

Differential Revision: D7517547

fbshipit-source-id: fd585bc4dceb837e8486b1285ed7ff14d24795d3
2018-04-13 21:51:51 -07:00
Liubov Dmitrieva
da07ca7373 commitcloudcommands: fix bookmarks, request is expected a list
Summary:
fix bookmarks

the thrift service expect list of names not dict, so removed bookmarks were just skipped when we transfer json request to thrift reuqest

Reviewed By: markbt

Differential Revision: D7570412

fbshipit-source-id: ae7d84db528fe5a76644ca6108be51e8e6b12e13
2018-04-13 21:51:51 -07:00
Kostia Balytskyi
004ebe3a7c hg: build fastmanifest on Windows
Differential Revision: D7568271

fbshipit-source-id: 399b56af0b0a7bd205a030ceaa2b0255006630dd
2018-04-13 21:51:51 -07:00
Kostia Balytskyi
64b26c3c94 hg: migrate cfastmanifest code from system to portability headers
Summary: To allow for Windows compilation.

Differential Revision: D7568270

fbshipit-source-id: eeef1b3712a30da3263ac2e2554d26f2aa48ff1a
2018-04-13 21:51:51 -07:00
Kostia Balytskyi
446d302f33 hg: do not use void* arithmetics not supported by the standard
Summary:
Yet another gcc-specific behavior that we're using.

FYI: https://stackoverflow.com/a/3524270/440677

Differential Revision: D7568272

fbshipit-source-id: ffa033ee9ec521ca0b4ac4befe447929c93db273
2018-04-13 21:51:51 -07:00
Kostia Balytskyi
dc1583e347 hg: remove extra semicolons in structs
Summary:
`PyObject_HEAD` is macro that ends with a semicolon when is is resolved. Thus `PyObject_HEAD;` ends in two semicolons.
Tolerating extra semicolons is a gcc feature, not a standard one.
I am adding the explanation for `// clang-format off` only once per file.

https://stackoverflow.com/questions/49529130/are-extra-semicolons-allowed-inside-struct-declarations?rq=1&utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa

Differential Revision: D7568269

fbshipit-source-id: 5afcfd7d96dc638cd0673d25809c9c43d58d00de
2018-04-13 21:51:51 -07:00
Mark Thomas
f1ebfba3dc progress: deactivate the progress thread when shutting down
Summary:
The progress bar thread is a daemon thread, so that it doesn't need to be
deactivated manually.  However, this means the thread terminates and the lock
is released during interpreter shutdown.  Because Python clears module
attributes at the start of interpreter shutdown (see [1]), releasing the lock
fails as it can no longer get the thread identity.

To mitigate this, we register an atexit handler to terminate and join to the
progress bar thread.

[1] https://stackoverflow.com/questions/25649676/where-is-pythons-shutdown-procedure-documented

Differential Revision: D7568155

fbshipit-source-id: 85ef10af6c1576d5beceb78f8514e0e440cdab7f
2018-04-13 21:51:51 -07:00
Kaley Huang
667f296601 bring back mercurial worker
Summary:
We disabled `runworker` for every importer (fastimport, syncimport, seqimport) when we hotfixed a customized file transaction breakage in fastimport D7108127.

Since
- we're not using fastimport, which relies on the customized file transaction
- seqimport does not rely on the customized file transaction mechanism
- syncimport is super slow without `runworker` and syncimport does not rely on the customized file transaction mechanism
- the manual work involved in adding mapping to ovrsource (we have to do that if we don't have syncimport) is not trivial

I think it's a good idea to bring back `runworker`, which is a configurable config, and it's explicitly set to `false` for fastimport and seqimport, `force` for syncimport.

This diff
- make the `useworker` config available again
- remove the unnecessary test on the customized file transaction, which is still broken with `runworker`
- log the time it takes to finish one sync import

Differential Revision: D7557856

fbshipit-source-id: 6d4105cc38b182e027512730901ce3b2a4e1d449
2018-04-13 21:51:51 -07:00
Kostia Balytskyi
88b296f51b hg: add more debug output to the _testlock function
Summary: We want to know what is the reason why we think the lock file can't be removed.

Reviewed By: farnz

Differential Revision: D7513954

fbshipit-source-id: fd1668e7a614e5e24e250018fbc880ba87821aa8
2018-04-13 21:51:51 -07:00
Kostia Balytskyi
14e5b61eaf hg: add process startime to lock unique id
Summary:
This helps with PID reuse, which can cause false positives when checking
whether the lock-owning process is still alive.

Reviewed By: mjpieters

Differential Revision: D7513955

fbshipit-source-id: d3df4b4afa53cd1e9633d71c294cf7014b7b65c5
2018-04-13 21:51:51 -07:00
Kostia Balytskyi
e516de21d5 hg: introduce a Locker class for lock-owner processes
Summary:
The goal of this diff is to introduce a central place for all logic related
to parsing and handling the contents of the lock files. The current state of
things is hard to manage and even harder to extend.

NB: the stack is not complete, the changes that actually use this refactoring are yet to come.

Reviewed By: quark-zju

Differential Revision: D7489326

fbshipit-source-id: aa79d964411e3f5b61e24aa9babece05d4f0bd60
2018-04-13 21:51:51 -07:00
Kostia Balytskyi
d612e92d9f hg: add process start time query function
Summary:
This will later be used to add a bit of uniqueness to PIDs on Windows, to
address the stale lock problem.

Reviewed By: quark-zju

Differential Revision: D7489325

fbshipit-source-id: b866bd239eacfc7fe64433f0989a73dff1246644
2018-04-13 21:51:51 -07:00
Martijn Pieters
3f7758625a sparse: Add support for specifying a revision to sparse explain
Summary: We need this to support producing sparse profile stats in a null-revision working copy.

Reviewed By: quark-zju

Differential Revision: D7553022

fbshipit-source-id: b1643f443de5a748bf81ff2df16ace984d59a2c2
2018-04-13 21:51:50 -07:00
Martijn Pieters
6e097ff6cd sparse: add support for per-subcommand switches
Summary: A followup diff will add more in the way of testing; this is easier with an actual usecase.

Reviewed By: quark-zju

Differential Revision: D7553020

fbshipit-source-id: a6e8b86154d802f8cdbc846f5ea14fa809ac5264
2018-04-13 21:51:50 -07:00
Martijn Pieters
440232e553 sparse: fix the accidental recursive handling of profiles
Summary:
hg sparse explain wasn't supposed to recurse, not without additional support.

It was the process of gathering stats that overwrote shared mutable structures.
Prevent this from happening again by making these structures immutable.

Differential Revision: D7516570

fbshipit-source-id: 247c60b4bd8d2d22b42c41871e1eb15988820025
2018-04-13 21:51:50 -07:00
Martijn Pieters
62ec76397b sparse: fix broken signature calculation for sparse profiles
Summary:
We were ignoring the actual profile data, and all profiles were given the same .hg/sparse-based hash signature instead.

Error introduced in D7415720

Differential Revision: D7514087

fbshipit-source-id: 56288aaaa2065b031318e7c065ec6310f6cecd37
2018-04-13 21:51:50 -07:00
Kostia Balytskyi
fa03821500 hg: fix test-checks broken in windows treemanifest stack
Reviewed By: quark-zju

Differential Revision: D7560133

fbshipit-source-id: 98b016d0911aaecc1058263c134a5e4ecd0be9e5
2018-04-13 21:51:50 -07:00
Kostia Balytskyi
2c984a6b29 hg: build cstore on Windows
Summary: This is not conditional on Posix now!

Differential Revision: D7555762

fbshipit-source-id: 63976ea24f28fbe3f84140a3246afa159af650b0
2018-04-13 21:51:50 -07:00
Kostia Balytskyi
546943a322 hg: fix how setup.py treats macros
Differential Revision: D7555763

fbshipit-source-id: 899d714ca3a68872532fd0c7a299d88181cd48cd
2018-04-13 21:51:50 -07:00
Kostia Balytskyi
3fd779e8be hg: migrate datapackstore to portability's dirent.h
Summary: Just start including a different header.

Differential Revision: D7555760

fbshipit-source-id: 7b0cd273a01db29cc31044b0cc0cff144b13a22b
2018-04-13 21:51:50 -07:00
Kostia Balytskyi
171ca13ebe hg: add a dirent.h portability header
Summary:
This will either include system dirent on POSIX or a vendored dirent from
folly on Windows.

`/no-check-code` is here because it's everywhere across hg's .c codebase.

Differential Revision: D7555759

fbshipit-source-id: dc55926e83e17976930522277ed7fe6ce41f32f7
2018-04-13 21:51:50 -07:00