Commit Graph

43092 Commits

Author SHA1 Message Date
Jun Wu
f0b5cd6eae indexedlog: add simple DirtyOffset abstraction
Summary: The `DirtyOffset` enum converts between array indexes and u64.

Reviewed By: DurhamG

Differential Revision: D7377215

fbshipit-source-id: 29d4f7d74f15523034c11abcc09329a1b21142b1
2018-04-13 21:51:39 -07:00
Martijn Pieters
bf9eb9018e sparse: add the option to hide certain profiles
Summary:
Some profiles are not for general consumption; they are usually profiles aimed at CI subsystems or similar. These can be hidden from listings by default using a `hidden` key.

The value of the key doesn't matter but can be used to explain why it is hidden.

Reviewed By: quark-zju

Differential Revision: D7433781

fbshipit-source-id: 877cd8698d50dc64cec8da706ab005e1fd786de4
2018-04-13 21:51:39 -07:00
Martijn Pieters
b9cca4a62b sparse: rework help information
Summary:
The hg sparse help page was overwhelming, and out of date. With subcommands, we
can break up the information a bit more.

* Subcommand help is the place for details on that command. Reduce the main
  help info to an overview.
* Move configuration switches to a `verbose` container to hide it by default.
* Add documentation on the sparse profile format.
* Mark the command-line switches with subcommands as deprecated so they are
  hidden by default.

Reviewed By: quark-zju

Differential Revision: D7432862

fbshipit-source-id: ffddd27e0ee4216f7e743e63e8efaed4eeaac582
2018-04-13 21:51:38 -07:00
Martijn Pieters
a4952f4bfa sparse: add a hg sparse files command
Summary: This command mostly echoes how `hg files` works, albeit simplified somewhat. Given a profile and optional patterns, the files that profile matches are listed.

Reviewed By: quark-zju

Differential Revision: D7431522

fbshipit-source-id: 8e9d1f0d2aa31335f53269f3005875a0cc2e65bf
2018-04-13 21:51:38 -07:00
Martijn Pieters
4c93c15316 sparse: include sparse profile impact
Summary:
Show the 'impact' of a profile, relative to a non-sparse working copy.

By default, this is the percentage of the total file count; adding --verbose will show the file sizes too.

The defined matchers have been refactored to reuse more of mercurial.match.basematcher, making it easier to reuse these in a wider mercurial context and avoiding repetition of common methods.

Reviewed By: quark-zju

Differential Revision: D7415720

fbshipit-source-id: 4ac3492c61aa70ee71d4bdf8c201b905a345a9d1
2018-04-13 21:51:38 -07:00
Martijn Pieters
aacd20f02b remotefilelog: add a spinner when finding outgoing revisions
Summary: On my system, the `outgoing()` revset could easily take several seconds. A spinner here helps dispell the notion that mercurial has hung.

Reviewed By: quark-zju

Differential Revision: D7429086

fbshipit-source-id: 28908a34798d985dad3120647c3a5f474ca8a746
2018-04-13 21:51:38 -07:00
Durham Goode
6ada5becf6 hg: fix stripping trees that were changed in merge commits
Summary:
Previously we relied on the list of files created by strip to decide
what directories to strip. It turns out this list wasn't adequate though, since
it's possible for directories to change even when files do not. For example,
during a merge if different files were added on different sides of the merge,
the files don't change during the merge but their containing directory does.

This diff special cases merge commits to make sure all the directories affected
by the merge are included in the strip.

Reviewed By: quark-zju

Differential Revision: D7409156

fbshipit-source-id: 9bf67eefb70189300c29db60d9945a7f608dfdda
2018-04-13 21:51:38 -07:00
Durham Goode
f5b8d91797 hg: fix sending trees from the server to non-tree clients
Summary:
Previously we were trying to send trees to all clients during an
infinitepull, even ones that didn't support treemanifest. This caused
infinitepulls that required rebundling to fail for non-tree clients.

The fix is to just not send them unless the client is advertising the
capability.

Reviewed By: phillco

Differential Revision: D7432374

fbshipit-source-id: 1fae14a158ef56fe39439a718b1b98928f4e07b0
2018-04-13 21:51:38 -07:00
Martijn Pieters
78965e53ca sparse: fix flakey test
Summary: The order of the two reported files can differ, depending on the random hash seed.

Reviewed By: ryanmce

Differential Revision: D7427725

fbshipit-source-id: 9ec76867877553a02d1a19642d98d626b0ba91cc
2018-04-13 21:51:38 -07:00
Martijn Pieters
fcdfdd9931 simplecache: refactor to allow just getting or setting, split out jsonserializer
Summary:
I need to be able to test for the presence of a cache without the ability to use a function, and I rather not use a sentinel value or have func raise an exception.

Note that the extension already is vulnerable to race conditions (there is plenty of time between the get, the func() call, and the set as it is).

Also, make it trivial to use JSON as a serializer.

Reviewed By: quark-zju

Differential Revision: D7415721

fbshipit-source-id: dc06e8f3efe725858a7960f7acfecc5a60390b85
2018-04-13 21:51:38 -07:00
Martijn Pieters
96a6a5773d sparse: make hg help sparse <subcommand> work
Summary:
This gives us access to the specifics of a subcommand, including switches
(still stubbed out). Note: `hg sparse <subcommand> -h` doesn't work yet, as the
`-h` command line switch would need intercepting as well.

Reviewed By: quark-zju

Differential Revision: D7413859

fbshipit-source-id: 02534f1255a11985ebe8636b26946fbeffc6fe09
2018-04-13 21:51:38 -07:00
Mark Thomas
449b58b48d progress: implement formatting of bytes values
Summary:
Add the ability to set a `formatfunc` on a progres bar, which formats the
numbers used.  Use `util.bytecount` as a format function in the places
where we have progress bars in numbers of bytes.

Reviewed By: quark-zju

Differential Revision: D7355232

fbshipit-source-id: 117c7035d46d47259cdfd70b80438cc6f4615977
2018-04-13 21:51:38 -07:00
Aida Getoeva
c752c3c3d5 Obsolete markers inside crdump
Summary: Added obsolete markers for each revision into `hg debugcrdump` output

Reviewed By: ryanmce

Differential Revision: D7416991

fbshipit-source-id: 9cca7339a4c465c5df81f2646e2ad0ce87bd2c3d
2018-04-13 21:51:38 -07:00
Mark Thomas
dbf70dc3fd crecord: suspend progress bar when displaying the curses interface
Summary:
Nothing currently displays a progress bar while the curses interface is shown,
but in the future something may be added which does, so ensure the progress bar
is suspended when showing the curses interface.

Reviewed By: ryanmce

Differential Revision: D7417435

fbshipit-source-id: 6b91b17ee5390cbde6e983081a0940051ab865c8
2018-04-13 21:51:38 -07:00
Mark Thomas
633d10fe2b ui: suspend progress bars when prompting the user
Differential Revision: D7416038

fbshipit-source-id: dbf4132c646c9b70a8d240cf2aa6e9eea53501a9
2018-04-13 21:51:37 -07:00
Martijn Pieters
84bfcf3162 RFC: sparse: ignore case collisions outside the sparse profile
Summary:
When updating a working copy that is limited by a sparse profile, do not abort on case collisions outside of the current working copy view.

Performance-wise, this isn't a big change; previously `_checkcollision()` would convert the whole manifest to a set, now we filter first with the matcher, then convert the remainder to a set. Either the original set conversion or the filtering can take O(size-of-manifest) time.

Reviewed By: ryanmce

Differential Revision: D7350251

fbshipit-source-id: efb8e270631f0aaa75c34c7b68189c60efe45984
2018-04-13 21:51:37 -07:00
David Soria Parra
f2f32df45b p4fastimport: pass ui to progress.bar
Summary: progress.bar() is incorrectly called without passing ui.

Differential Revision: D7415250

fbshipit-source-id: 22c7419561879ed9293e2c79cc9d4271e805be76
2018-04-13 21:51:37 -07:00
Liubov Dmitrieva
3ab4155a04 obsshelve: bugfix with old format shelves
Summary:
fix bug with new shelved() revsetpredicate

it failed if old format shelve files were present

Differential Revision: D7414883

fbshipit-source-id: 7b96abe7abf950ba339c22d0e9025f7423004911
2018-04-13 21:51:37 -07:00
Saurabh Singh
4c60397d1f test-fb-hgext-obsshelve: use shell function instead of alias
Summary:
D7397947 broke `test-check-code.t` which complains about the use of
alias instead of function. This is blocking our continuous builds. This commit
addresses the same.

Reviewed By: quark-zju

Differential Revision: D7404361

fbshipit-source-id: 7907e913327c77861258a37ff8b87d467a7a15c6
2018-04-13 21:51:37 -07:00
Phil Cohen
26476a8a58 add megarepo generation script
Summary: This script can be used to generate a large number of commits.

Reviewed By: DurhamG

Differential Revision: D7363243

fbshipit-source-id: 1a1e3ba7fca29dc695446eeffb715474ea2943ad
2018-04-13 21:51:37 -07:00
Durham Goode
78f6364280 hg: check if treemanifest is enabled in more places
Summary:
In uisetup we wrap a bunch of functions. We need to make sure they each
handle the case where treemanifest is not actually enabled for that repo. This
can happen in cases like a peer-to-peer pull where one repo has treemanifest
enabled and one does not.

Reviewed By: quark-zju

Differential Revision: D7368084

fbshipit-source-id: 152a448fb8a902e8c8f8ac4e806ab53514fadaa7
2018-04-13 21:51:37 -07:00
Martijn Pieters
6414c84571 sparse: show individual profile details
Summary: This shows basic details about a template, including base metadata. Once we have support for per-subcommand switches we can add further filtering.

Reviewed By: ryanmce

Differential Revision: D7365794

fbshipit-source-id: 3dd362f099e7f4d6db73c9bda7c24a9f4f2f90ee
2018-04-13 21:51:37 -07:00
Martijn Pieters
422a95de10 sparse: provide subcommands for other common tasks
Summary: Now all commands can be subcommands. The command-line switches will be hidden in a next commit (but remain available for backwards compatibility).

Reviewed By: ryanmce

Differential Revision: D7365393

fbshipit-source-id: 29b8ed2ac6bffccda41ed3f738087dde23312a8c
2018-04-13 21:51:37 -07:00
Martijn Pieters
5296ef905b sparse: make the help system print our subcommands
Summary:
This means we can

- count on the minirst module to format longer lines correctly,
- show subcommands when an error occurs (so full is false),
- add verbose help when the -v switch

Reviewed By: ryanmce

Differential Revision: D7365392

fbshipit-source-id: c8711bd19a6d992ad86f24f3c07b381f69d02121
2018-04-13 21:51:37 -07:00
Martijn Pieters
b8110f185c Allow for better help handling and multiple registrations for a single function.
Summary:
We take help info from the docstring but that'd disallow multiple registrations for a single function that can handle multiple subcommands. Refactor to let the decorator set an override.

So before you'd do this:

```
subcmd('foo')
def _foocmd(...):
    "Help for foo command"
    # implementation for foo

subcmd('bar')
def _barcmd(...):
    "Help for bar command"
    # implementation for bar
```
but you could not *combine* the implementations into one function and decorate both, because there is only the single docstring.

Now you can do:

```
subcmd('foo', help="Help for foo command")
subcmd('bar', help="Help for bar command")
def _fooandbarcmd(cmd, ...):
    "This docstring is no longer used"
    # implementation for foo and bar, combined, switching on the value of cmd
```

Reviewed By: ryanmce

Differential Revision: D7365390

fbshipit-source-id: 3d41542f1f197137ef13458e8d850cda8f53da74
2018-04-13 21:51:37 -07:00
Martijn Pieters
c9dd45c81f help: refactor to allow for patching
Summary:
I want to add to helpcmd but with nested functions that's a lot more
complicated then it needs to be.

Reviewed By: ryanmce

Differential Revision: D7365391

fbshipit-source-id: 02092dd55f8f9521324b8ed51fa3134817454d36
2018-04-13 21:51:37 -07:00
Liubov Dmitrieva
ee6457a0fb obsshelve: add highlight
Summary:
obsshelve: highlight shelved changes in smartlog (you can see with --hidden)

we highlighted only active shelved changes, those that has been unshelved are just ordinary hidden changes.

Reviewed By: ryanmce

Differential Revision: D7383800

fbshipit-source-id: 2df4092514d58315a6a204411feed99819df5c93
2018-04-13 21:51:36 -07:00
Liubov Dmitrieva
58b4fc9380 obsshelve: add tests for new revsetpredicate
Summary: obsshelve: add tests for new revsetpredicate

Differential Revision: D7397947

fbshipit-source-id: f2940ef20e594aea93832860a3dccfa2c4cf787a
2018-04-13 21:51:36 -07:00
Liubov Dmitrieva
d0972892e5 shelve: add revsetpredicate to select shelved commits
Summary:
add revsetpredicate to select shelved commits

if we are going to backup shelved commits, we should be able to select them

Reviewed By: ryanmce

Differential Revision: D7381561

fbshipit-source-id: 35aa0ad9a77fa9df94e91b571279eaa4301c85f3
2018-04-13 21:51:36 -07:00
Phil Cohen
2c66516728 add subclass for AbandonedTransactionFoundError
Summary:
Raise this instead of RepoError. This way commands can catch and auto-recover the
repo, if appropriate. Right now I only do this inside megarepo commands.

Reviewed By: DurhamG

Differential Revision: D7363245

fbshipit-source-id: 0a6cab92a4ff6e50cd269bdddadee47be6199ebd
2018-04-13 21:51:36 -07:00
Phil Cohen
2e5e764e25 a basic perf suite implementation
Reviewed By: DurhamG

Differential Revision: D7341042

fbshipit-source-id: e87bd1bfe198fc4c5e1e2239b761c543d6acbf01
2018-04-13 21:51:36 -07:00
Jun Wu
81b678d400 color: change diff related colors to 256 colors
Summary: Use 256 colors if supported, or fallback to 16 colors.

Reviewed By: singhsrb

Differential Revision: D7388275

fbshipit-source-id: e7be447f2b900e384e9280f92ea8f881747493e4
2018-04-13 21:51:36 -07:00
Jun Wu
3d8a2aa583 color: support fallback configs
Summary:
Allow color config to be something like `color132:red` that is to use 256
color if supported, but fallback to `red`.

To get a consistent test output. `run-tests.py` is changed to disable 256
color support explicitly.

Reviewed By: singhsrb

Differential Revision: D7388277

fbshipit-source-id: da74ae8fc70c971901d56a6985976db44cbec0d9
2018-04-13 21:51:36 -07:00
Jun Wu
7f793ca8d3 color: define 256 colors on supported terminals
Summary:
This would allow us to use 256 colors, which seems to be widely available
on modern terminals (including mosh and tmux).

Reviewed By: singhsrb

Differential Revision: D7388279

fbshipit-source-id: 726a364ed5d3acc449f0d7ada14c42e4b68424ec
2018-04-13 21:51:36 -07:00
Jun Wu
7afcefd83c color: add a method to detect 256color support
Summary:
This allows us to treat terminals with 256 color support differently.

The motivation is, the bright colors seem to work fine on Linux. But they
appear to have minor differences on OSX terminals (iTerm2 or Terminal.app).
Using explicit 256 colors is better.

Reviewed By: singhsrb

Differential Revision: D7388276

fbshipit-source-id: 970ee18f0f5bb9cc4bb5c39b2b46b354525b3d55
2018-04-13 21:51:36 -07:00
Jun Wu
97584a7de7 color: change diff related colors to bright versions
Summary:
As suggested by wez, "dim" is less supported than the bright colors.
So let's use the bright versions instead. Basically, replacing "red",
"red dim" with "brightred" and "red".

Reviewed By: singhsrb

Differential Revision: D7387254

fbshipit-source-id: f741d9e1c12115acdd1a3a2cae1658d8fae534bf
2018-04-13 21:51:36 -07:00
Jun Wu
c93ed71dcd color: add bright colors
Summary:
Most terminals support 16 colors these days. This is manually tested
with: tmux (2.2), mosh (1.3.0), screen (4.04) from Linux (xfce) and
OS X Terminal.app and iTerm 2.

Note: aside from "screen" with default configuration, the terminals
also have 256 colors support. "screen" can support 256 colors if it's
started with TERM=xterm-256color.

See https://en.wikipedia.org/wiki/ANSI_escape_code for details.

Reviewed By: singhsrb

Differential Revision: D7387258

fbshipit-source-id: e7b2437fee2b4e593077bb3f44143c86ece93a8f
2018-04-13 21:51:36 -07:00
Liubov Dmitrieva
f776a334b5 obsshelve: use real user for shelve commits
Summary:
The reason why we would like to have a real user there is that

we may want to start push shelved changes in commit cloud, so

we should have correct user on the commit

Reviewed By: ryanmce

Differential Revision: D7380732

fbshipit-source-id: 27b07f992a3e394016ce69e1dc9694a4dd1c336b
2018-04-13 21:51:36 -07:00
Liubov Dmitrieva
ce46d0b01f fix import warnings and add log request time
Reviewed By: StanislavGlebik

Differential Revision: D7365309

fbshipit-source-id: 5984fa811b8d1cb9eb996700e6352ec9966b18ad
2018-04-13 21:51:35 -07:00
Liubov Dmitrieva
8eb1968255 obsshelve: do not use secret phase
Summary: obsshelve: do not use secret phase

Reviewed By: ryanmce

Differential Revision: D7380531

fbshipit-source-id: 21538d42a43f019b894dd1a77ba0adc67798936a
2018-04-13 21:51:35 -07:00
Jun Wu
ae0142473d progress: suspend progress when shelling out
Summary:
The progress bar runs in another thread and may draw " <=> " which pollutes the
current screen.

Make `ui.system` suspend the progress bar automatically. This should work for
the "invoking editor" case. I think sshaskpass and curses might also need to
suspend the progress bar but those are trickier.

Reviewed By: singhsrb

Differential Revision: D7377492

fbshipit-source-id: 833ac724bbe4f9c630ca37567dc088f7279dd67e
2018-04-13 21:51:35 -07:00
Jun Wu
3859d00394 indexedlog: implement flush for the main index
Summary:
The flush method will write buffered data to disk.

A mistake in Root entry serialization is fixed - it needs to translate dirty
offsets to non-dirty ones.

Reviewed By: DurhamG

Differential Revision: D7223729

fbshipit-source-id: baeaab27627d6cfb7c5798d3a39be4d2b8811e5f
2018-04-13 21:51:35 -07:00
Jun Wu
8f5c35c8d2 indexedlog: initial main index structure
Summary:
Add the main `Index` structure and its constructor.

The structure focus on the index logic itself. It does not have the checksum
part yet.

Some notes about choices made:
- The use of mmap: mmap is good for random I/O, and has the benefit of
  sharing buffers between processes reading the same file. We may be able to
  do good user-space caching for the random I/O part. But it's harder to
  share the buffers between processes.
- The "read_only" auto decision. Common "open" pattern requires the caller
  to pass whether they want to read or write. The index makes the decision
  for the caller for convenience (ex. running "hg log" on somebody else's
  repo).
- The "load root entry from the end of the file" feature. It's just for
  convenience for users wanting to use the Index in a standalone way. We
  probably

Reviewed By: DurhamG

Differential Revision: D7208358

fbshipit-source-id: 14b74d7e32ef28bd5bc3483fd560c489d36bf8e5
2018-04-13 21:51:35 -07:00
Martijn Pieters
85eb89e007 sparse: make listing profiles a subcommand
Summary:
This is based on the hg show implementation. hg sparse --list-profiles is not yet widely known, so now is the time to move it to a subcommand.

This is the first step in untangling the mess that is the `hg sparse` forest of options.

Currently, all switches on the command are mutually exclusive, except for `—force` and `—template`, which each only apply to a subset of the actions the other switches affect.

Subcommands are the right pattern for mutually-exclusive actions that can accept their own individual switches.

Reviewed By: quark-zju

Differential Revision: D7350928

fbshipit-source-id: d03014cf7edd2f089f670d11465c70940d96c070
2018-04-13 21:51:35 -07:00
Kostia Balytskyi
6f265bb8d5 hg: when shelling our from rage, pass shell=True to subprocess.Popen
Summary:
Without `shell=True`, `Popen` does not find `.bat` files, just `.exe` files.
The way `arc` is deployed (to at least some people) does not create a `.exe`
shim and only has a `.bat` in the PATH.

FYI: https://stackoverflow.com/questions/41860668/why-does-this-python-subprocess-command-only-work-when-shell-true-on-windows?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa

Reviewed By: quark-zju

Differential Revision: D7370386

fbshipit-source-id: ad9e2006893e71b2a9e8cd1a480f9188e412a931
2018-04-13 21:51:35 -07:00
Jun Wu
545f670504 pathencoding: utility for converting between bytes and paths
Summary:
A simple utility that does paths <-> local bytes conversion. It's needed
since Mercurial stores paths using local encoding in manifests.

For POSIX, the code is zero-cost - no real conversion or error can happen.
This is in theory cheaper than what treedirstate does.

For Windows, the "local_encoding" crate is selected as Yuya suggested the
`MultiByteToWideChar` Win32 API [1] and "local_encoding" uses it. It does
the right thing given my experiment with GBK (Chinese, simplified) encoding.

```
  ....
  C:\Users\quark\enc>hg debugshell --config extensions.debugshell=
  >>> repo[0].manifest().text()
  '\xc4\xbf\xc2\xbc1/\xce\xc4\xbc\xfe1\x00b80de5d138758541c5f05265ad144ab9fa86d1db\n'
  >>> repo[0].files()
  ['\xc4\xbf\xc2\xbc1/\xce\xc4\xbc\xfe1']
  extern crate local_encoding;
  use std::path::PathBuf;
  use local_encoding::{Encoder, Encoding};
  const mpath: &[u8] = b"\xc4\xbf\xc2\xbc1/\xce\xc4\xbc\xfe1";
  fn main() {
      let p = PathBuf::from(Encoding::OEM.to_string(mpath).unwrap());
      println!("exists: {}", p.exists());
      println!("mpath len: {}, osstr len: {}", mpath.len(), p.as_path().as_os_str().len());
  }
  exists: true
  mpath len: 11, osstr len: 15
```

In the future, we might normalize the paths to UTF-8 before storing them in
manifest to avoid issues.

Differential Revision: D7319604

fbshipit-source-id: a7ed5284be116c4176598b4c742e8228abcc3b02
2018-04-13 21:51:35 -07:00
Kostia Balytskyi
731f87492f hg: make rage errors more friendly
Summary:
We've seen cases when `suprocess.Popen` fails [1], [2]. It might happen if
`arc` binary is not in `$PATH` env var. Let's fail gracefully.

[1] https://fburl.com/7hcq73as
[2] https://fburl.com/u1v0es8q

Differential Revision: D7365459

fbshipit-source-id: ce838f434cb81211c334d6f170fd25ef30edaabd
2018-04-13 21:51:35 -07:00
Liubov Dmitrieva
1c8877987d https connector
Summary: https connector for commit cloud

Reviewed By: mjpieters

Differential Revision: D7306672

fbshipit-source-id: c95388964a1aced82579c0d55ccdd0a9ef391e81
2018-04-13 21:51:35 -07:00
Durham Goode
d87789e3e7 hg: store repo on manifestlog instead of changelog
Summary:
Previously we were storing the changelog on the manifestlog and using
it to resolve linkrevs before serializing them. It turns out the changelog can
be invalidated at a different rate than the manifestlog, so we could encounter
issues where the manifestlog held a reference to the old changelog.

To fix this, let's hold a reference to the repo and access the changelog from
there when we need it. This introduces a circular reference between the
manifestlog and the repo, but it's probably fine for now until we can get rid of
the need for changelog invalidation.

Reviewed By: singhsrb

Differential Revision: D7360321

fbshipit-source-id: 2317c7fcd6b307a50b64f0c5df97dda2955f3e21
2018-04-13 21:51:35 -07:00
Mark Thomas
0827683bf0 remotefilelog: show names of files being downloaded in the progress bar
Summary:
When remotefilelog downloads filelog information for a particular file, set the
progress bar item to that file.  This means if we get stuck on a particular
file, there is feedback to the user as to which file that is.

Reviewed By: quark-zju

Differential Revision: D7329503

fbshipit-source-id: 94416962cdc4c97994f76e8ed9203823aeca3d64
2018-04-13 21:51:35 -07:00