Commit Graph

1959 Commits

Author SHA1 Message Date
Steve Borho
d7b9ecc0ba wix: only one KeyPath is allowed per Component 2017-05-06 10:18:34 -05:00
Augie Fackler
c1e3251ee2 contrib: add check-code rule to enforce non-use of python in tests 2017-06-20 09:49:43 -04:00
Boris Feld
e2b921724b packaging: add make target for linux wheels
Having linux wheels is going to helps system without compiler or python-dev
plus speed up the installation for everyone.

I followed the manylinux example repository
https://github.com/pypa/python-manylinux-demo
to add a make target (build-linux-wheels) using
official docker image to build python 2 linux wheels
for mercurial. It generates Python 2.6 and Python 2.7 for both
32 and 64 bits architectures.

I had to blacklist several test cases for various reasons:
* test-convert-git.t and test-subrepo-git.t because of the git version
* test-patchbomb-tls.t because of warning using tls 1.0
  It's likely because the docker image is based on centos 5.0 and
  openssl is outdated.
2017-04-25 16:50:01 +02:00
Augie Fackler
f1dbb5d073 debian: configure editor and pager to sensible-\1 by default
Now that environment variables override system-wide hgrc settings, we
can default Mercurial to sensible-editor and sensible-pager by default
for debian users.
2017-04-22 12:29:17 -04:00
Augie Fackler
3d0bf6c892 freeze: merge default into stable for 4.2 code freeze 2017-04-18 12:24:34 -04:00
Augie Fackler
b2bd435955 dispatch: protect against malicious 'hg serve --stdio' invocations (sec)
Some shared-ssh installations assume that 'hg serve --stdio' is a safe
command to run for minimally trusted users. Unfortunately, the messy
implementation of argument parsing here meant that trying to access a
repo named '--debugger' would give the user a pdb prompt, thereby
sidestepping any hoped-for sandboxing. Serving repositories over HTTP(S)
is unaffected.

We're not currently hardening any subcommands other than 'serve'. If
your service exposes other commands to users with arbitrary repository
names, it is imperative that you defend against repository names of
'--debugger' and anything starting with '--config'.

The read-only mode of hg-ssh stopped working because it provided its hook
configuration to "hg serve --stdio" via --config parameter. This is banned for
security reasons now. This patch switches it to directly call ui.setconfig().
If your custom hosting infrastructure relies on passing --config to
"hg serve --stdio", you'll need to find a different way to get that configuration
into Mercurial, either by using ui.setconfig() as hg-ssh does in this patch,
or by placing an hgrc file someplace where Mercurial will read it.

mitrandir@fb.com provided some extra fixes for the dispatch code and
for hg-ssh in places that I overlooked.
2017-04-12 11:23:55 -07:00
Bryan O'Sullivan
287bd28acf atexit: switch to home-grown implementation 2017-04-11 14:54:12 -07:00
Jun Wu
488ba17f87 chg: respect environment variables for pager
Previously chg runs the pager command without respecting its environment
variables being told to use. This patch makes it so.
2017-04-12 16:50:23 -07:00
FUJIWARA Katsunori
2afd920706 misc: update year in copyright lines
This patch also makes some expected output lines in tests glob-ed for
persistence of them.

BTW, files below aren't yet changed in 2017, but this patch also
updates copyright of them, because:

    - mercurial/help/hg.1.txt

      almost all of "man hg" output comes from online help of hg
      command, and is already changed in 2017

    - mercurial/help/hgignore.5.txt
    - mercurial/help/hgrc.5

      "copyright 2005-201X Matt Mackall" in them mentions about
      copyright of Mercurial itself
2017-02-12 02:23:33 +09:00
FUJIWARA Katsunori
f778068b44 misc: replace domain of mercurial-devel ML address by mercurial-scm.org
This patch also adds new check-code.py pattern to detect invalid usage
of "mercurial-devel@selenic.com".
2017-02-11 00:23:55 +09:00
FUJIWARA Katsunori
e6f91a13e7 misc: replace domain of mercurial ML address by mercurial-scm.org
This patch also adds new check-code.py pattern to detect invalid usage
of "mercurial@selenic.com".

Change for test-convert-tla.t is tested, but similar change for almost
same test-convert-baz.t isn't yet tested actually, because I couldn't
find out the way to get "GNU Arch baz client".

AFAIK, buildbot skips test-convert-baz.t, too. Does anybody have
appropriate environment for testing?
2017-02-11 00:23:53 +09:00
Anton Shestakov
d3b5c285e9 debian: update copyright years 2017-02-04 20:29:34 +08:00
Anton Shestakov
dcf9dc5b3d debian: update mailing list address 2017-02-04 20:29:13 +08:00
Augie Fackler
cde5195cd5 contrib: fix check-commit to not reject commits from hg sign and hg tag
I'm tired of having a spurious red build every time we do a
release. Fix it once and for all.
2017-01-18 23:34:35 -05:00
Gregory Szorc
4225a4e399 zstd: prevent potential free() of uninitialized memory
This is a cherry pick of an upstream fix. The free() of uninitialed
memory could likely only occur if a malloc() inside zstd fails.

The patched functions aren't currently used by Mercurial. But I don't
like leaving footguns sitting around.
2017-01-17 10:17:13 -08:00
Jun Wu
c81e982932 chg: always wait for pager
Previously, when runcommand raises, chg aborts with, and does not wait for
pager. The call stack is like:

  hgc_runcommand -> handleresponse -> readchannel -> debugmsg("failed to
  read channel") -> exit(255)

That means, chg returns to the shell, then both the pager and the shell will
read from the terminal at the same time, causing problems.

This patch fixes that by using "atexit" to register the pager cleanup
function so chg will always wait for pager even if runcommand raises.
2017-04-11 18:31:40 -07:00
Gregory Szorc
c3cb00b3e9 zstd: vendor python-zstandard 0.6.0
Commit 63c68d6f5fc8de4afd9bde81b13b537beb4e47e8 from
https://github.com/indygreg/python-zstandard is imported without
modifications (other than removing unwanted files).

This includes minor performance and feature improvements. It also
changes the vendored zstd library from 1.1.1 to 1.1.2.

# no-check-commit
2017-01-14 19:41:43 -08:00
Pulkit Goyal
3c7388da12 py3: replace pycompat.getenv with encoding.environ.get
pycompat.getenv returns os.getenvb on py3 which is not available on Windows.
This patch replaces them with encoding.environ.get and checks to ensure no
new instances of os.getenv or os.setenv are introduced.
2017-01-15 13:17:05 +05:30
Martin von Zweigbergk
631c635657 check-code: reject module-level @cachefunc
Module-level @cachefunc usage is risky because it can easily create a
memory "leak". Let's reject it completely for now. If a valid usage
comes up in the future, we can always improve the check or reconsider.
2017-01-13 10:11:37 -08:00
Gregory Szorc
b2e18c67d3 perf: support multiple compression engines in perfrevlogchunks
Now that the revlog has a reference to a compressor, it is
possible to swap in other compression engines. So, teach
`hg perfrevlogchunks` to do that.

The default behavior of `hg perfrevlogchunks` is now to measure the
compression performance of all compression engines implementing the
revlog compressor API. This effectively adds the no-op "none"
compressor and zstd (when available) into the default set.

While we can't yet plug alternate compressors into revlogs, this
command gives us a preview of the performance. On the mozilla-unified
repository:

$ hg perfrevlogchunks -c
! compress w/ none
! wall 0.115159 comb 0.110000 user 0.110000 sys 0.000000 (best of 86)
! compress w/ zlib
! wall 5.681406 comb 5.680000 user 5.680000 sys 0.000000 (best of 3)
! compress w/ zstd
! wall 2.624781 comb 2.620000 user 2.620000 sys 0.000000 (best of 4)

$ hg perfrevlogchunks -m
! compress w/ none
! wall 0.124486 comb 0.120000 user 0.120000 sys 0.000000 (best of 79)
! compress w/ zlib
! wall 10.144701 comb 10.150000 user 10.150000 sys 0.000000 (best of 3)
! compress w/ zstd
! wall 4.383118 comb 4.390000 user 4.390000 sys 0.000000 (best of 3)

Those numbers for zstd look promising. But they aren't the full story.
For that, we'll need to look at decompression times and storage sizes.
Stay tuned...
2017-01-02 12:02:08 -08:00
Gregory Szorc
1a6670d670 revlog: move decompress() from module to revlog class (API)
Upcoming patches will convert revlogs to use the compression engine
APIs to perform all things compression. The yet-to-be-introduced
APIs support a persistent "compressor" object so the same object
can be reused for multiple compression operations, leading to
better performance. In addition, compression engines like zstd
may wish to tweak compression engine state based on the revlog
(e.g. per-revlog compression dictionaries).

A global and shared decompress() function will shortly no longer
make much sense. So, we move decompress() to be a method of the
revlog class. It joins compress() there.

On the mozilla-unified repo, we can measure the impact of this change
on reading performance:

$ hg perfrevlogchunks -c
! chunk
! wall 1.932573 comb 1.930000 user 1.900000 sys 0.030000 (best of 6)
! wall 1.955183 comb 1.960000 user 1.930000 sys 0.030000 (best of 6)
! chunk batch
! wall 1.787879 comb 1.780000 user 1.770000 sys 0.010000 (best of 6
! wall 1.774444 comb 1.770000 user 1.750000 sys 0.020000 (best of 6)

"chunk" appeared to become slower but "chunk batch" got faster. Upon
further examination by running both sets multiple times, the numbers
appear to converge across all runs. This tells me that there is no
perceived performance impact to this refactor.
2017-01-02 13:00:16 -08:00
Martin von Zweigbergk
1840263a8c help: merge revsets.txt into revisions.txt
Selecting single and multiple revisions is closely related, so let's
put it in one place, so users can easily find it. We actually did not
even point to "hg help revsets" from "hg help revisions", but now that
they're on a single page, that won't be necessary.
2017-01-11 11:37:38 -08:00
Jun Wu
402691b892 chg: check snprintf result strictly
This makes the program more robust when somebody changes hgclient's
maxdatasize in the future.
2017-01-11 23:39:24 +08:00
Jun Wu
bb277b3ff3 chg: change server's process title
This patch uses the newly introduced "setprocname" interface to update the
process title server-side, to make it easier to tell what a worker is actually
doing.

The new title is "chg[worker/$PID]", where PID is the process ID of the
connected client. It can be directly observed using "ps -AF" under Linux, or
"ps -A" under FreeBSD.
2017-01-11 07:40:52 +08:00
Jun Wu
b61b02a865 chg: remove getpager support
We have enough bits to switch to the new chg pager code path in runcommand.
So just remove the legacy getpager support.

This is a red-only patch, and will break chg's pager support temporarily.
2017-01-10 06:59:39 +08:00
Jun Wu
5ae59a4110 chg: handle pager request client-side
This patch implements the simple S-channel pager handling at chg
client-side.

Note: It does not deal with environ and cwd currently for simplicity, which
will be fixed later.
2017-01-10 06:59:03 +08:00
Jun Wu
923ee6957d chg: check type read from S channel
The previous patch added the check server-side. This patch added it
client-side.
2017-01-06 16:14:52 +00:00
Jun Wu
734e02b02d chg: send type information via S channel (BC)
Previously S channel is only used to send system commands. It will also be
used to send pager commands. So add a type parameter.

This breaks older chg clients. But chg and hg should always come from a
single commit and be packed into a single package. Supporting running
inconsistent versions of chg and hg seems to be unnecessarily complicated
with little benefit. So just make the change and assume people won't use
inconsistent chg with hg.
2017-01-06 16:11:03 +00:00
Pulkit Goyal
baf5959e2c py3: pass the path in hg.repository() as bytes
This make test-ancestor.py pass on Python 3.
2017-06-17 14:39:10 +05:30
Matt Harbison
09ec25b785 perf: ensure HGRCPATH is exported on Windows
After dropping the garbage collector hack, `hg perfstartup` started yelling
about not being able to import the evolve extension, which I have in my user
config.  Launching `env` shows that an empty HGRCPATH isn't exported to the
environment.  Since `env` doesn't quote, I have no idea if the variable is
trimmed, but Mercurial doesn't complain when processing it.
2017-06-16 20:49:32 -04:00
Augie Fackler
6faf0dceff contrib: add a ratchet for tests in Python 3
This gives us an easy way to automatically update passing tests in Python 3.
2017-06-15 11:00:29 -04:00
Augie Fackler
9dcef694f2 contrib: check in a whitelist of passing tests in Python 3 2017-06-15 10:59:48 -04:00
Gregory Szorc
31b468ec11 check-config: look for ui.configwith
We previously weren't looking for this config helper. And, surprise,
profiling.py references config options without docs.

If I tried hard enough, I could have combined the regexps using a
positive lookbehind assertion or something. But I didn't want to make
my brain explode.

At some point, we should probably do this linting at the tokenizer or
ast layer. I'm not willing to open that can of worms right now.
2017-06-15 10:58:36 -07:00
Gregory Szorc
ea2746aaed check-config: use named groups in regexp
In preparation for making this regexp a bit more complicated.
2017-06-15 10:38:19 -07:00
Gregory Szorc
6f1aea28c8 check-config: use compiled regexp
And split the regexp across multiple lines to make it easier to read.
2017-06-15 10:36:23 -07:00
Pierre-Yves David
8ab26d7bb8 checkcode: only match pushd/popd as word
otherwise it partially match common words like "pushdestination".
2017-04-10 17:05:54 +02:00
Gregory Szorc
c70926c31e zstd: vendor python-zstandard 0.8.1
This contains a fix for compilation on BSDs.

#no-check-commit
2017-04-08 14:33:20 -07:00
Jun Wu
f809188b4c check-code: use "-" to specify a list of files from stdin
This will be used by the next patch.
2017-04-06 22:08:23 -07:00
Philippe Pepiot
e5957f96bd perf: add historical portability for util.timer
util.timer has been introduced in ae5d60bb and used in perf.py since 22fbca1d.
For historical portability, forcibly define util.timer in perf.py
2017-04-06 14:41:42 +02:00
Augie Fackler
dcec340a8b check-code: update test IP address enforcement checks
Instead of mentioning 127.0.0.1, we should use $LOCALIP. Anytime
$LOCALIP appears in output, we should make sure we use (glob) on that
line of output so that weird environments that do remapping jiggery
pokery (such as our FreeBSD buildbot that's in a jail) don't get
spurious test failures.
2017-04-03 18:58:00 -04:00
Gregory Szorc
6758d40c63 zstd: vendor python-zstandard 0.8.0
Commit 81e1f5bbf1fc54808649562d3ed829730765c540 from
https://github.com/indygreg/python-zstandard is imported without
modifications (other than removing unwanted files).

Updates relevant to Mercurial include:

* Support for multi-threaded compression (we can use this for
  bundle and wire protocol compression).
* APIs for batch compression and decompression operations using
  multiple threads and optimal memory allocation mechanism. (Can
  be useful for revlog perf improvements.)
* A ``BufferWithSegments`` type that models a single memory buffer
  containing N discrete items of known lengths. This type can be
  used for very efficient 0-copy data operations.

# no-check-commit
2017-04-01 15:24:03 -07:00
Gregory Szorc
2f6dff3311 help: document bundle specifications
I softly formalized the concept of a "bundle specification" a while
ago when I was working on clone bundles and stream clone bundles and
wanted a more robust way to define what exactly is in a bundle file.

The concept has existed for a while. Since it is part of the clone
bundles feature and exposed to the user via the "-t" argument to
`hg bundle`, it is something we need to support for the long haul.

After the 4.1 release, I heard a few people comment that they didn't
realize you could generate zstd bundles with `hg bundle`. I'm
partially to blame for not documenting it in bundle's docstring.

Additionally, I added a hacky, experimental feature for controlling
the compression level of bundles in 054e64c4d837. As the commit
message says, I went with a quick and dirty solution out of time
constraints. Furthermore, I wanted to eventually store this
configuration in the "bundlespec" so it could be made more flexible.

Given:

a) bundlespecs are here to stay
b) we don't have great documentation over what they are, despite being
   a user-facing feature
c) the list of available compression engines and their behavior isn't
   exposed
d) we need an extensible place to modify behavior of compression
   engines

I want to move forward with formalizing bundlespecs as a user-facing
feature. This commit does that by introducing a "bundlespec" help
page. Leaning on the just-added compression engine documentation
and API, the topic also conveniently lists available compression
engines and details about them. This makes features like zstd
bundle compression more discoverable. e.g. you can now
`hg help -k zstd` and it lists the "bundlespec" topic.
2017-04-01 13:42:06 -07:00
Martin von Zweigbergk
6b2860bef0 check-code: fix "covert" typo 2017-04-03 11:30:51 -07:00
Denis Laxalde
b2e35d013c hgweb: rename linerangelog.js as followlines.js
So that the file name matches both the feature name and user facing vocabulary
(e.g. the revset function).
2017-04-03 10:02:55 +02:00
Gregory Szorc
62d4252847 show: new extension for displaying various repository data
Currently, Mercurial has a number of commands to show information. And,
there are features coming down the pipe that will introduce more
commands for showing information.

Currently, when introducing a new class of data or a view that we
wish to expose to the user, the strategy is to introduce a new command
or overload an existing command, sometimes both. For example, there is
a desire to formalize the wip/smartlog/underway/mine functionality that
many have devised. There is also a desire to introduce a "topics"
concept. Others would like views of "the current stack." In the
current model, we'd need a new command for wip/smartlog/etc (that
behaves a lot like a pre-defined alias of `hg log`). For topics,
we'd likely overload `hg topic[s]` to both display and manipulate
topics.

Adding new commands for every pre-defined query doesn't scale well
and pollutes `hg help`. Overloading commands to perform read-only and
write operations is arguably an UX anti-pattern: while having all
functionality for a given concept in one command is nice, having a
single command doing multiple discrete operations is not. Furthermore,
a user may be surprised that a command they thought was read-only
actually changes something.

We discussed this at the Mercurial 4.0 Sprint in Paris and decided that
having a single command where we could hang pre-defined views of
various data would be a good idea. Having such a command would:

* Help prevent an explosion of new query-related commands
* Create a clear separation between read and write operations
  (mitigates footguns)
* Avoids overloading the meaning of commands that manipulate data
  (bookmark, tag, branch, etc) (while we can't take away the
  existing behavior for BC reasons, we now won't introduce this
  behavior on new commands)
* Allows users to discover informational views more easily by
  aggregating them in a single location
* Lowers the barrier to creating the new views (since the barrier
  to creating a top-level command is relatively high)

So, this commit introduces the `hg show` command via the "show"
extension. This command accepts a positional argument of the
"view" to show. New views can be registered with a decorator. To
prove it works, we implement the "bookmarks" view, which shows a
table of bookmarks and their associated nodes.

We introduce a new style to hold everything used by `hg show`.

For our initial bookmarks view, the output varies from `hg bookmarks`:

* Padding is performed in the template itself as opposed to Python
* Revision integers are not shown
* shortest() is used to display a 5 character node by default (as
  opposed to static 12 characters)

I chose to implement the "bookmarks" view first because it is simple
and shouldn't invite too much bikeshedding that detracts from the
evaluation of `hg show` itself. But there is an important point
to consider: we now have 2 ways to show a list of bookmarks. I'm not
a fan of introducing multiple ways to do very similar things. So it
might be worth discussing how we wish to tackle this issue for
bookmarks, tags, branches, MQ series, etc.

I also made the choice of explicitly declaring the default show
template not part of the standard BC guarantees. History has shown
that we make mistakes and poor choices with output formatting but
can't fix these mistakes later because random tools are parsing
output and we don't want to break these tools. Optimizing for human
consumption is one of my goals for `hg show`. So, by not covering
the formatting as part of BC, the barrier to future change is much
lower and humans benefit.

There are some improvements that can be made to formatting. For
example, we don't yet use label() in the templates. We obviously
want this for color. But I'm not sure if we should reuse the existing
log.* labels or invent new ones. I figure we can punt that to a
follow-up.

At the aforementioned Sprint, we discussed and discarded various
alternatives to `hg show`.

We considered making `hg log <view>` perform this behavior. The main
reason we can't do this is because a positional argument to `hg log`
can be a file path and if there is a conflict between a path name and
a view name, behavior is ambiguous. We could have introduced
`hg log --view` or similar, but we felt that required too much typing
(we don't want to require a command flag to show a view) and wasn't
very discoverable. Furthermore, `hg log` is optimized for showing
changelog data and there are things that `hg display` could display
that aren't changelog centric.

There were concerns about using "show" as the command name.

Some users already have a "show" alias that is similar to `hg export`.

There were also concerns that Git users adapted to `git show` would
be confused by `hg show`'s different behavior. The main difference
here is `git show` prints an `hg export` like view of the current
commit by default and `hg show` requires an argument. `git show`
can also display any Git object. `git show` does not support
displaying more complex views: just single objects. If we
implemented `hg show <hash>` or `hg show <identifier>`, `hg show`
would be a superset of `git show`. Although, I'm hesitant to do that
at this time because I view `hg show` as a higher-level querying
command and there are namespace collisions between valid identifiers
and registered views.

There is also a prefix collision with `hg showconfig`, which is an
alias of `hg config`.

We also considered `hg view`, but that is already used by the "hgk"
extension.

`hg display` was also proposed at one point. It has a prefix collision
with `hg diff`. General consensus was "show" or "view" are the best
verbs. And since "view" was taken, "show" was chosen.

There are a number of inline TODOs in this patch. Some of these
represent decisions yet to be made. Others represent features
requiring non-trivial complexity. Rather than bloat the patch or
invite additional bikeshedding, I figured I'd document future
enhancements via TODO so we can get a minimal implmentation landed.
Something is better than nothing.
2017-03-24 19:19:00 -07:00
Denis Laxalde
c2ed8e445d hgweb: expose a followlines UI in filerevision view
In filerevision view (/file/<rev>/<fname>) we add some event listeners on
mouse clicks of <span> elements in the <pre class="sourcelines"> block.
Those listeners will capture a range of lines selected between two mouse
clicks and a box inviting to follow the history of selected lines will then
show up. Selected lines (i.e. the block of lines) get a CSS class which make
them highlighted. Selection can be cancelled (and restarted) by either
clicking on the cancel ("x") button in the invite box or clicking on any other
source line. Also clicking twice on the same line will abort the selection and
reset event listeners to restart the process.

As a first step, this action is only advertised by the "cursor: cell" CSS rule
on source lines elements as any other mechanisms would make the code
significantly more complicated. This might be improved later.

All JavaScript code lives in a new "linerangelog.js" file, sourced in
filerevision template (only in "paper" style for now).
2017-03-29 22:26:16 +02:00
Jun Wu
5dda7e9e48 check-code: detect r.revision(r.node(rev))
revlog.revision takes either node or rev, but taking a rev is more
efficient, because converting rev to node is just a seek and read.
That's cheaper than converting node to rev, which may require O(n) walk in
revlog index for the first times, and then triggering building the radix
tree index. Even with the radix tree built, rev -> node is still faster than
node -> rev because the radix tree requires more jumps in memory.

So r.revision(r.node(rev)) should be changed to r.revision(rev). This patch
adds a check-code rule to detect that.
2017-03-29 16:46:57 -07:00
Jun Wu
35b00f22fa runtests: change local IP glob pattern from "127.0.0.1" to "$LOCALIP"
This is similar to what 9704c8e70d2d does. Since 1363aaf74791 has changed
"127.0.0.1" to "$LOCALIP". The glob pattern needs update accordingly. It is
expected to fix tests running in some BSD jails.
2017-03-26 19:57:17 -07:00
Jun Wu
7fa1b063a1 checkcode: enforce lowercase for extension docstring title
This will ensure new extensions are consistent and `hg help -e` has a
consistent output.

I have to add a new group since the normal "pypats" will be filtered by
"pyfilters", which will remove comments and docstrings.
2017-03-23 21:23:21 -07:00
Jun Wu
dfb8c98044 chg: add procutil.h
This patch adds a formal header procutil.h for procutil.c, and changes
Makefile to build procutil.c independently.
2017-01-02 14:57:14 +00:00
Jun Wu
8e6abc8b48 chg: let procutil maintain its own pagerpid
Previously, chg.c maintains the pagerpid. Let's move it to procutil.c.

Note: chg.c still have a pagerpid to decide whether to call attachio or not.
In the future, attachio may be moved from hgc_open to hgc_runcommand, and
hgc_runcommand handles both pager and attachio so we don't need to run
attachio twice. And chg.c will be free of pagerpid.
2017-01-02 14:43:37 +00:00
Jun Wu
4321520c94 chg: decouple hgclient from setuppager
procutil should not depend on hgclient. This patch makes the pager handling
part independent from hgclient.
2017-01-02 14:10:32 +00:00
Jun Wu
efbbb12a58 chg: decouple hgclient from setupsignalhandler
procutil should not depend on hgclient. This patch makes the signal handling
part independent from hgclient.
2017-01-02 14:04:35 +00:00
Jun Wu
fc44f74541 chg: move signal and pager handling to a separate file
In the future hgclient will deal with pager directly inside runcommand, so
related signal handling stuff needs to be decoupled from chg.c.

The signal handling and pager logic are coupled because we need to forward
SIGPIPE when pager exits. So they are moved together, otherwise a global
variable (pagerpid) is inevitable.

This patch moves related functions from chg.c to procutil.c, which was
marked as copied to maintain annotate history.

The move is done without code modification for easy review, therefore
`#include "procutil.c"` was introduced temporarily.
2017-01-02 14:02:47 +00:00
Jun Wu
2419c6679a chg: respect XDG_RUNTIME_DIR
$XDG_RUNTIME_DIR [1] is a better place for user daemons. Let's use it and
fallback to $TMPDIR.

After this patch, chg will try socket paths in the following order:

  1. $CHGSOCKNAME
  2. $XDG_RUNTIME_DIR/chg/server
  3. ${TMPDIR:-tmp}/chg$UID/server

[1]: https://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html
2016-12-26 00:02:42 +00:00
Jun Wu
4548a63b44 chg: make "get default sockdir" a separate method
The logic to get a default socket directory will become longer in the next
patch. So let's move it out.
2016-12-25 23:49:54 +00:00
Jun Wu
f437cb2c85 chg: handle connect failure before errno gets overridden
This patch moves the error handling logic up so that errno after connect
won't be overridden.
2016-12-25 23:32:11 +00:00
Jun Wu
d23c117db4 chg: support long socket path
This patch replaces UNIX_PATH_MAX (108) with PATH_MAX (4096) so we can have
long unix path.
2016-12-23 16:26:40 +00:00
Jun Wu
c5374c4a97 chg: remove sockdirfd
See the previous patch for the reason.
2016-12-23 16:16:44 +00:00
Jun Wu
8499ade54b chg: let hgc_open support long path
"sizeof(sun_path)" is too small. Use the chdir trick to support long socket
path, like "mercurial.util.bindunixsocket".

It's useful for cases where TMPDIR is long. Modern OS X rewrites TMPDIR to a
long value. And we probably want to use XDG_RUNTIME_DIR [2] for Linux.

The approach is a bit different from the previous plan, where we will have
hgc_openat and pass cmdserveropts.sockdirfd to it. That's because the
current change is easier: chg has to pass a full path to "hg" as the
"--address" parameter. There is no "--address-basename" or "--address-dirfd"
flags. The next patch will remove "sockdirfd".

Note: It'd be nice if we can use a native "connectat" implementation.
However, that's not available everywhere. Some platform (namely FreeBSD)
does support it, but the implementation has bugs so it cannot be used [2].

[1]: https://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html
[2]: https://www.mercurial-scm.org/pipermail/mercurial-devel/2016-April/082892.html
2016-12-23 16:37:00 +00:00
Pulkit Goyal
2207f24f07 py3: add warnings in check-code related to py3
We have our own bytes versions of things like, getopt.getopt, os.sep, os.name,
sys.executable, os.environ and few more for python 3 portability. Its better
to come up with warnings if someone breaks the things which we have fixed.

After this patch, check-code will warn us to use our bytes version.
These checks run on mercurial/ and hgext/ and pycompat.py is excluded.
2016-12-21 22:42:31 +05:30
Jun Wu
e912a360fc chg: remove locks
See the previous two patches for the reason. The advantage is a simplified
code base and better throughput when starting multiple servers with multiple
confighashes. The disadvantage is starting multiple servers in parallel with
a single confighash will waste some CPU time, which is probably fine in
common use-cases.

This makes it easier to switch to relative paths to support long unix domain
socket paths.
2016-12-19 22:15:00 +00:00
Jun Wu
24cb48448f chg: start server at a unique address
See the previous patch for motivation. Previously, the server is started at
a globally shared address. This patch appends pid to the address so it
becomes unique.

Note: with Linux pid namespace, the address may be non-unique, but it does
not affect correctness of chg - chg client will receive an redirection and
that's it.
2016-12-19 22:09:49 +00:00
Martin von Zweigbergk
8f2ed099c1 help: make multirevs just an alias for revsets
The multirevs topis seems to be covered well by the revsets topic, so
just make it an alias and remove multirevs.txt.
2016-12-16 09:48:14 -08:00
Pierre-Yves David
54022676d2 import-checker: do not enforce lexical sort accross stdlib/local boundary
Before this change, you could get in a start where the checker would either
complain about importing local module before stdlib one or complain about the
local one being wrongly lexically sorted with the stdlib one.

We detect the boundary and avoid complaining about lexical sort across it.
2016-12-15 19:56:48 +01:00
Philippe Pepiot
396c998f12 perf: add historical support of ui.load()
ui.load() has been available since d83ca854 and at the time of writing isn't
available on stable branch breaking benchmarking newer stable revisions.

Add historical portability policy note on contrib/benchmarks
2016-12-15 12:17:08 +01:00
Remi Chaintron
cc88d4a3c4 revlog: merge hash checking subfunctions
This patch factors the behavior of both methods into 'checkhash'.
2016-12-13 14:21:36 +00:00
Pulkit Goyal
9523fa9b6b fancyopts: switch from fancyopts.getopt.* to getopt.*
In the next patch, we will be creating a bytes version of getopt.getopt() and
doing that will leave getopt as unused import in fancyopts. So before removing
that there are instances in codebase where instead of importing getopt, we
have used fancyopts.getopt. This patch will switch all those cases so that
the next patch can remove the import of getopt from fancyopts without breaking
things.
2016-12-06 06:27:58 +05:30
Yuya Nishihara
1d44bd2bbb ui: factor out ui.load() to create a ui without loading configs (API)
This allows us to write doctests depending on a ui object, but not on global
configs.

ui.load() is a class method so we can do wsgiui.load(). All ui() calls but
for doctests are replaced with ui.load(). Some of them could be changed to
not load configs later.
2016-10-22 14:35:10 +09:00
Jun Wu
35a3247fa1 check-code: add a rule to forbid "cp -r"
See the commit message of the previous patch for the reason. In short,
according to the current POSIX standard, "-r" is "removed", and "-R" is the
current standard way to do "copy file hierarchies".
2016-11-30 19:23:04 +00:00
Yuya Nishihara
30fe4722fb chgserver: make it a core module and drop extension flags
It was an extension just because there were several dependency cycles I
needed to address.

I don't add 'chgserver' to extensions._builtin since chgserver is considered
an internal extension so nobody should enable it by their config.
2016-10-15 14:30:16 +09:00
Danek Duvall
267cb097d4 zstd: fix compilation with Solaris Studio
Without these changes, Solaris Studio (12.4) gives us "syntax error: empty
declaration" on these two lines.
2016-11-22 13:32:05 -08:00
Gregory Szorc
67e45331a8 perf: add command for measuring revlog chunk operations
Upcoming commits will teach revlogs to leverage the new compression
engine API so that new compression formats can more easily be
leveraged in revlogs. We want to be sure this refactoring doesn't
regress performance. So this commit introduces "perfrevchunks" to
explicitly test performance of reading, decompressing, and
recompressing revlog chunks.

Here is output when run on the mozilla-unified repo:

$ hg perfrevlogchunks -c
! read
! wall 0.346603 comb 0.350000 user 0.340000 sys 0.010000 (best of 28)
! read w/ reused fd
! wall 0.337707 comb 0.340000 user 0.320000 sys 0.020000 (best of 30)
! read batch
! wall 0.013206 comb 0.020000 user 0.000000 sys 0.020000 (best of 221)
! read batch w/ reused fd
! wall 0.013259 comb 0.030000 user 0.010000 sys 0.020000 (best of 222)
! chunk
! wall 1.909939 comb 1.910000 user 1.900000 sys 0.010000 (best of 6)
! chunk batch
! wall 1.750677 comb 1.760000 user 1.740000 sys 0.020000 (best of 6)
! compress
! wall 5.668004 comb 5.670000 user 5.670000 sys 0.000000 (best of 3)

$ hg perfrevlogchunks -m
! read
! wall 0.365834 comb 0.370000 user 0.350000 sys 0.020000 (best of 26)
! read w/ reused fd
! wall 0.350160 comb 0.350000 user 0.320000 sys 0.030000 (best of 28)
! read batch
! wall 0.024777 comb 0.020000 user 0.000000 sys 0.020000 (best of 119)
! read batch w/ reused fd
! wall 0.024895 comb 0.030000 user 0.000000 sys 0.030000 (best of 118)
! chunk
! wall 2.514061 comb 2.520000 user 2.480000 sys 0.040000 (best of 4)
! chunk batch
! wall 2.380788 comb 2.380000 user 2.360000 sys 0.020000 (best of 5)
! compress
! wall 9.815297 comb 9.820000 user 9.820000 sys 0.000000 (best of 3)

We already see some interesting data, such as how much slower
non-batched chunk reading is and that zlib compression appears to be
>2x slower than decompression.

I didn't have the data when I wrote this commit message, but I ran this
on Mozilla's NFS-based Mercurial server and the time for reading with a
reused file descriptor was faster. So I think it is worth testing both
with and without file descriptor reuse so we can make informed
decisions about recycling file descriptors.
2016-11-17 20:17:51 -08:00
Toshi MARUYAMA
8227d06f8d packagelib: use LANGUAGE=C for "hg version"
If "hg version" does not contain "version" (e.g. Japanese),
$hgversion was empty and rpmbuild failed.
2017-06-12 03:23:58 +09:00
Toshi MARUYAMA
ea4c5fa07c rpms: add hgdemandimport in files 2017-06-12 03:23:56 +09:00
Toshi MARUYAMA
905b1512b3 rpms: remove '%if "%{?pythonver}" != "2.4"'
Mercurial requires python >= 2.7.
2017-06-12 03:22:45 +09:00
Pierre-Yves David
78ab1d5356 perf: add a perfbookmarks command
A new command dedicated to benchmark of bookmark initialization.
2017-06-07 18:22:11 +01:00
Pierre-Yves David
43a12c3e5f perfphases: add a flag to also include file access time
The flag purges all phases data so we'll have to read the file from disk again.
2017-05-23 02:27:41 +02:00
Pierre-Yves David
667fac2e9e perf: have a generic "clearstorecache" function
There are multiple places where we will want to purge some store cache. So we
promote the existing _clearobsstore function to a something reusable.
2017-06-07 17:31:30 +01:00
Pierre-Yves David
3324c6b39d perfbranchmap: add an option to purge the revbranch cache
The perf extension needs to be able to reflect this reality too. (eg: 4s vs 60s
on a Million-ish revisions repository).
2017-06-05 16:24:01 +01:00
Danek Duvall
e90fa83ca6 check-code: ban grep's context flags (-A/-B/-C) since they're not on Solaris 2017-06-06 08:58:27 -07:00
Augie Fackler
9a722d4382 merge with stable 2017-06-03 16:33:28 -04:00
Jun Wu
d4a87ea37f check-code: suggest policy.importmod
This forbids patterns like `from mercurial.cext import parsers` which breaks
pure.
2017-05-31 11:41:54 -07:00
Siddharth Agarwal
0c09474fcb import-checker: add hgdemandimport to local modules
This fixes test-check-module-imports.t.
2017-05-31 20:07:08 -07:00
Gregory Szorc
4f8c92c87a check-code: recommend util.urlreq when importing urlparse 2017-03-21 22:46:17 -07:00
Gregory Szorc
5ca0f908bf py3: add __bool__ to every class defining __nonzero__
__nonzero__ was renamed to __bool__ in Python 3. This patch simply
aliases __bool__ to __nonzero__ for every class implementing
__nonzero__.
2017-03-13 12:40:14 -07:00
Gregory Szorc
ddf0e60af7 perf: perform a garbage collection before each iteration
Currently, no explicit garbage collection is performed when running
the microbenchmarks in `hg perf`. I think this is wrong because
garbage collection can have a significant impact on execution times.
And, if gc is triggered via the default heuristics, it will
fire effectively randomly during subsequent benchmark iterations
due to variable amount of garbage left over from previous runs.

Running a gc before invoking the measured function will help ensure
state is more consistent across all iterations.
2017-03-13 18:16:42 -07:00
Kostia Balytskyi
06571f213d contrib: remove unnecessary debug output from editmergeps.ps1 2017-05-30 06:22:14 -07:00
Kostia Balytskyi
07d2531f6b contrib: fix a bug preventing editmergeps.ps1 from running unknonw editors
'$ executable' is not a way to run executable in powershell, '& executable'
is instead. Found this when testing with regular Windows notepad.
2017-05-30 06:17:51 -07:00
Kostia Balytskyi
63f283180f contrib: make editmergeps use -NoNewWindow option in Start-Process cmdlet
Running 'Start-Process -Wait "vim" "+10" "filename"' from PowerShell
actually spawns a separate cmd window to run vim in. This looks ugly
and in most cases not what user wants. During my initial testing I was
using the Cmder app, which made me not notice this (it captures new
windows as new tabs).
2017-05-30 06:02:31 -07:00
Kostia Balytskyi
b7b5dbd3c4 contrib: run editmergeps.ps1 from the same location as editmergeps.bat
This change is needed for cases when user does not put editmergeps.bat
directly into PATH, but rather uses 'merge-tools.editmergeps.executable'
config option to provide a full path to editmergeps.bat. In such cases,
editmergeps.ps1 cannot be run simply by name, it needs a full path. In
BATCH file %~dp0 stands for the directory in which the original file
is located.
2017-05-30 05:56:48 -07:00
Gregory Szorc
21a83ae27a perf: benchmark command for revlog indexes
We didn't have explicit microbenchmark coverage for loading revlog
indexes. That seems like a useful thing to have, so let's add it.

We currently measure the low-level nodemap APIs. There is room to
hook in at the actual revlog layer. This could be done as a follow-up.

The hackiest thing about this patch is specifying revlog paths.
Other commands have arguments that allow resolution of changelog,
manifest, and filelog. I needed to hook in at a lower level of
the revlog API than what the existing helper functions to resolve
revlogs allowed. I was too lazy to write some new APIs. This could
be done as a follow-up easily enough.

Example output for `hg perfrevlogindex 00changelog.i` on my
Firefox repo (404418 revisions):

! revlog constructor
! wall 0.003106 comb 0.000000 user 0.000000 sys 0.000000 (best of 912)
! read
! wall 0.003077 comb 0.000000 user 0.000000 sys 0.000000 (best of 924)
! create index object
! wall 0.000000 comb 0.000000 user 0.000000 sys 0.000000 (best of 1803994)
! retrieve index entry for rev 0
! wall 0.000193 comb 0.000000 user 0.000000 sys 0.000000 (best of 14037)
! look up missing node
! wall 0.003313 comb 0.000000 user 0.000000 sys 0.000000 (best of 865)
! look up node at rev 0
! wall 0.003295 comb 0.010000 user 0.010000 sys 0.000000 (best of 858)
! look up node at 1/4 len
! wall 0.002598 comb 0.010000 user 0.010000 sys 0.000000 (best of 1103)
! look up node at 1/2 len
! wall 0.001909 comb 0.000000 user 0.000000 sys 0.000000 (best of 1507)
! look up node at 3/4 len
! wall 0.001213 comb 0.000000 user 0.000000 sys 0.000000 (best of 2275)
! look up node at tip
! wall 0.000453 comb 0.000000 user 0.000000 sys 0.000000 (best of 5697)
! look up all nodes (forward)
! wall 0.094615 comb 0.100000 user 0.100000 sys 0.000000 (best of 100)
! look up all nodes (reverse)
! wall 0.045889 comb 0.050000 user 0.050000 sys 0.000000 (best of 100)
! retrieve all index entries (forward)
! wall 0.078398 comb 0.080000 user 0.060000 sys 0.020000 (best of 100)
! retrieve all index entries (reverse)
! wall 0.079376 comb 0.080000 user 0.070000 sys 0.010000 (best of 100)
2017-05-28 11:13:10 -07:00
Gregory Szorc
845664d2aa perf: rename perfrevlog to perfrevlogrevisions
We have a couple of commands beginning with "perfrevlog." The
actual "perfrevlog" command actually measures resolving the fulltext
of multiple revisions. So let's rename it to reflect what it actually
does.
2017-05-28 10:56:28 -07:00
Yuya Nishihara
b44d32013d import-checker: guess names of C extension modules
Since extension modules aren't included in the list of source files, they
need to be populated somehow. Otherwise the import from cext/cffi would be
treated as a global one.
2017-05-02 22:28:18 +09:00
Yuya Nishihara
a603182b94 import-checker: convert localmods to a set of module names
This makes it easy to add a source-less module name to the set.
2017-05-28 15:21:18 +09:00
Yuya Nishihara
b28bfa196a import-checker: allow importing symbols from pure modules
This allows us to re-export pure functions from cffi modules:

  # mercurial/cffi/base85.py
  from ..pure.base85 import *
2017-05-02 22:24:57 +09:00
Pierre-Yves David
41d67dfdab perfphases: add 'perfphases' command
This commands benchmark the time spend computing the data in a repository.
2017-05-23 02:23:07 +02:00
Siddharth Agarwal
1ce1a31421 check: check modules in hgdemandimport
A few places only check modules in mercurial and hgext. Add
hgdemandimport to the list in those places.
2017-05-21 13:44:26 -07:00
Siddharth Agarwal
a41c665cd6 demandimport: move to separate package
In Python 3, demand loading is per-package. Keeping demandimport in the
mercurial package would disable demand loading for any modules in
mercurial.
2017-05-21 12:10:53 -07:00
Siddharth Agarwal
59a34a8229 import-checker: add a way to directly import certain symbols
We'll use this for the 'demandimport' symbol in an upcoming patch.
2017-05-21 12:09:01 -07:00