Some platforms (ex. FreeBSD) do not have `bash` by default. Therefore it
should not be used in test scripts.
Differential Revision: https://phab.mercurial-scm.org/D609
This would make the checker more friendly for 3rd-party code. For example,
In remotefilelog/x.py, it may have:
from . import shallowutils
That could trigger "relative import of stdlib module" if
"remotefilelog" was installed in the system. If the module being checked
conflicts with the system module, it makes sense to not treat that module as
system module. This patch makes it so.
Differential Revision: https://phab.mercurial-scm.org/D552
Often people running `phabsend --confirm` just want to check whether a
commit will trigger a creation of new Differential Revision, or update an
existing one. This patch implements that. The `--confirm` message was
changed to use node instead of revision number to be consistent with what
`phabsend` outputs.
An example output looks like:
D487 - a80f447973a0 test-extension: enable demandimport explicitly
D494 - cf440ea6e47e test-casecollision-merge: fix the test
NEW - 0a6b97147128 phabsend: polish the docstring a bit
Send the above changes to https://phab.mercurial-scm.org/ (yn)?
Differential Revision: https://phab.mercurial-scm.org/D514
Sometimes people have multiple Phabricator endpoints set in multiple repos.
It seems better for `--confirm` to prompt about the Phabricator endpoint
patches being sent to.
Differential Revision: https://phab.mercurial-scm.org/D513
Previously phabsend has an optimization that will skip uploading a diff if
the patch (with context line number = 1) remains unchanged. That could be
confusing:
Aug 24 15:52:28 <martinvonz> phillco: something is wrong with phabricator'your patches/
Aug 24 15:52:45 <martinvonz> ... with phabricator's view of your patches again
Aug 24 15:53:38 <martinvonz> if i phabread D388 and then D399, i get a version of filemerge.py with "a, b, c" somewhere on line 344, which is not what phabricator shows for D399
Aug 24 15:53:51 <martinvonz> junw: maybe that's more for you ^
Fix that by checking context with 32767 lines, which is the same as what
will be actually sent.
Differential Revision: https://phab.mercurial-scm.org/D512
The local tag feature was intended to make `phabsend` closer to `email`
workflow. But its experience is not great in multiple ways:
- after rebase, obsoleted changesets are still visible because of tags
- without obsstore, the association information will get lost
- even with obsstore, things could go wrong with graft, export+import
- no easy way to tell which Differential Revision a commit is associated
Therefore make `--amend` the default. People wanting the old behavior can
use `--no-amend`.
Differential Revision: https://phab.mercurial-scm.org/D511
run-tests.py and import-checker.py take wholly different approaches to
.t-tests, and the result was that they had different notions about
when a Python heredoc had ended, leading to conflicts. This resolves
part of the issue, and the rest I can deal with by tweaking the one
offending test file.
`simplemerge()` will soon require context-like objects to work. Create a simple
context-like object that wraps the requested files and can be passed to the new
API.
Differential Revision: https://phab.mercurial-scm.org/D378
I didn't do anything to ensure correctness here, just enough to avoid
tracebacks in the import checker, which uses the native ast module to
try and parse all our Python files.
Changing status (accept, etc) on the webpage is not very convenient -
currently there is no way to accept (or abandon etc.) a stack using a single
click or command.
This patch adds a `phabupdate` command that could be used to change status
in batch. It also supports `--comment` which will write a comment on the
last revision, which is similar to what we do using emails.
Differential Revision: https://phab.mercurial-scm.org/D127
This patch adds status words (ex. `abandoned`, `accepted`, `needsreview`,
`needsrevision`, `closed`) to the revision query language so people can
select revision in a more flexible way.
Test Plan:
Try something like `phabread ':2 & accepted'`, `phabread ':105 - closed` and
make sure they have desired outputs.
Differential Revision: https://phab.mercurial-scm.org/D126
Previously, `phabread` can only be used to read a single patch, or a single
stack of patches. In the future, we want to have more complex queries like
filtering with status (open, accepted, closed, etc), or maybe more complex
like filtering by reviewers etc. The command line flag approach won't scale
with that.
Besides, we might want to have other commands to update Differential
Revision status in batch, like accepting a stack using a single command.
Therefore, this patch adds a small language. It has basic set operations:
`&`, `+`, `-` and an ancestor operator to support `--stack`.
Test Plan:
Try querying this Phabricator instance:
hg phabread 1+2 # 1, 2
hg phabread D2+D1 # 2, 1
hg phabread ':118-115+:2-1' # 114, 116, 117, 118, 2
hg phabread '((:118-(D115+117)))&:117' # 114, 116
hg phabread ':2&:117' --debug # differential.query is called only once
Make sure the output is expected and prefetch works.
Differential Revision: https://phab.mercurial-scm.org/D125
Previously, `readpatch` and `querydrev` take a same `params` and `stack`
parameters. This patch changes `readpatch` so it takes the output of
`querydrev`, not the input of `querydrev`. This makes the code more
flexible and cleaner.
Differential Revision: https://phab.mercurial-scm.org/D124
Previously `hg phabsend` was imitating `hg email` and won't mutate
changesets. That works fine with reviewer-push workflow, reviewers run
`phabread`, `import`.
However, it does not work well with author-push workflow. Namely, the author
needs to run extra commands to get the right commit message, and remove the
local tag after push.
This patch solves those issues by adding the `--amend` option, so local
changesets will have the right commit message, and tags become unnecessary.
Test Plan:
Given the following DAG:
o 17
o 16
| o 15
| @ 14
|/
o 13
o 12
Run `hg phabsend '(13::)-17' --amend`, check the new DAG looks like:
o 21
| o 20
| @ 19
|/
o 18
| o 17
| x 16
| x 13
|/
o 12
And commit messages are updated to contain the `Differential Revision` lines.
Use `phabread` to make sure Phabricator has the amended node recorded.
Also check `phabsend .` followed by a `phabsend . --amend`, the commit
message will be updated and the tag will be removed.
Differential Revision: https://phab.mercurial-scm.org/D122
Without this flag, compilation fails with:
hgclient.c: In function 'hgc_open':
hgclient.c:466: error: 'O_DIRECTORY' undeclared (first use in this function)
hgclient.c:466: error: (Each undeclared identifier is reported only once
hgclient.c:466: error: for each function it appears in.)
Differential Revision: https://phab.mercurial-scm.org/D260
CentOS 5 has reached end-of-life. A working build for it is still
possible using 'vault.centos.org'.
Differential Revision: https://phab.mercurial-scm.org/D261
This is a quick fix to make `hg phabread D189` work.
It seems we might want to replace all `r''` to `u''`, and add more
`encoding.*to*` to be more explicit when interacting with `json` module.
Differential Revision: https://phab.mercurial-scm.org/D192
If you're shipping prerelease or rc packages using Munki, you'll
eventually discover that Munki's version comparison logic is not as
good as pip's. In theory we should be able to fix Munki, but it seems
entirely reasonable to produce version strings that sort reasonably
under these conditions. Since the requried logic not brief, add a new
script and some tests of that logic.
A followup change will wire this into the Makefile.
5ba3f753c9b1 introduced a use of 'hg revert --no-backup' which is
currently broken. Rather than try and do a fix on the (complicated)
revert code during the freeze, let's just give up on this test under
Python 3 until later. Once we do fix things, the test ratchet script
will re-enable it.
I used this to more quickly track down a failing test-check-config.t issue
in another repo. I thought it might be useful more generally, so I'm sending
it out in case others think it's a worthwhile change.
Previously, we trust Differential Revision in commit message blindly. This
patch adds sanity check so a host name change will be detected and the
commit message will be ignored.
Differential Revision: https://phab.mercurial-scm.org/D35
The diff property contains metadata like "HG Node". Previously we skip
uploading a new diff if we are sure that the old patch and new patch have a
same content. That has issues when a pusher adds an obsmarker using the old
"HG Node" stored in the old diff.
This patch adds logic to update the diff property so "HG Node" gets updated
to prevent that issue.
Differential Revision: https://phab.mercurial-scm.org/D229
This makes it more strict when checking whether or not we should update a
Differential Revision. For example,
a) Alice updates D1 to content 1.
b) Bob updates D1 to content 2.
c) Alice tries to update D1 to content 1.
Previously, `c)` will do nothing because `phabsend` detects the patch is not
changed. A more correct behavior is to override Bob's update here, hence the
patch.
This also makes it possible to return a reaonsable "last node" when there is
no tags but only `Differential Revision` commit messages.
Test Plan:
```
for i in A B C; do echo $i > $i; hg ci -m $i -A $i; done
hg phabsend 0::
# D40: created
# D41: created
# D42: created
echo 3 >> C; hg amend; hg phabsend .
# D42: updated
hg tag --local --hidden -r 2 -f D42
# move tag to the previous version
hg phabsend .
# D42: skipped (previously it would be "updated")
rm -rf .hg; hg init
hg phabread --stack D42 | hg import -
hg phabsend .
# D42: updated
hg tag --local --remove D42
hg commit --amend
hg phabsend .
# D42: updated (no new diff uploaded, previously it will upload a new diff)
```
The old diff object is now returned, which could be useful in the next
patch.
Differential Revision: https://phab.mercurial-scm.org/D121
This adds a --confirm flag similar to the confirm flag of `hg email` using which
one can confirm the changesets before they get emailed. The confirm flag will
show the changesets and ask for confirmation before sending them.
Differential Revision: https://phab.mercurial-scm.org/D218
For example, if "hg log" was defined as an alias:
# /etc/mercurial/hgrc
[alias]
log = log --graph
the buildrpm script would be surprised by log messages formatted in
unexpected ways, and bail out.
This patch sets HGPLAIN, effectively resetting all the user configs,
including log output, to a common state, making the build more
predictable across all the possible environments.
I'm not sure if there's a reason chg is not added by default.
If not, I would like to propose adding in this patch.
Differential Revision: https://phab.mercurial-scm.org/D220
Sometimes people want to specify reviewer explicitly for a stack. The
webpage only allows changing reviewer for one revision at a time. This patch
adds a `--reviewer` flag to make it easier to specify reviewers.
Test Plan:
On a test Phabricator instance, enable `differential.allow-self-accept`,
assign myself as a reviewer and make sure it works. Also try an invalid
username and make sure it raises.
Differential Revision: https://phab.mercurial-scm.org/D38
Previously we trust local tags blindly and that could cause wrong
Differential Revision to be updated, when people switch between Phabricator
instances.
This patch adds verification logic to detect such issue and remove
problematic tags. For example, a tag "D19" was on node "X", the code will
fetch all diffs attached to D19, and check if nodes server-side overlaps
with nodes in precursors. If they do not overlap, create a new Differential
Revision.
Test Plan:
Use a test Phabricator instance, send patches using `hg phabsend`, then
change the local tag manually to a wrong Differential Revision number.
Amend the patch and send again. Make sure the tag gets ignored and deleted.
Differential Revision: https://phab.mercurial-scm.org/D36
This allows us to do extra sanity checks using batch APIs to prevent
updating a wrong revision, which could happen when people switch Phabricator
instances and having stale tags living in the repo.
Differential Revision: https://phab.mercurial-scm.org/D34
Previously we only respect hg:meta sent by phabsend. This patch makes it
respect local:commits sent by arc as well. This avoids issues where phabread
could lose the author information.
Test Plan:
Commit using a customized user, send the patch using arc to a test
Phabricator instance, and then read the patch using phabread. Make sure it
preserves the user information.
Differential Revision: https://phab.mercurial-scm.org/D33
This was meant as a substitute for Python's "with" with multiple
context managers before we moved to Python 2.7. We're now on 2.7, so
we should have no reason to keep ctxmanager. "hg grep --all
ctxmanager" says that it was never used anyway.
Differential Revision: https://phab.mercurial-scm.org/D73
When using the 'import' command running ZSH together with its
Mercurial-specific completions the flag '--partial' (introduced in Mercurial
3.1, 2014-08-01) is not offered as completion option.
This patch adds it to the list of completions for the 'import' command.
I'm still cleaning this up, but it's easier to do in bite-size chunks
like this than all at once. The negative lookahead avoids one false
positive category from some output related to finding Subversion
bindings.
Differential Revision: https://phab.mercurial-scm.org/D13
It's possible to set up non-linear dependencies in Phabricator like:
o D4
|\
| o D3
| |
o | D2
|/
o D1
The old `phabread` code will print D1 twice. This patch adds de-duplication
to prevent that.
Test Plan:
Construct the above dependencies in a Phabricator test instance and make
sure the old code prints D1 twice while the new code won't.
Previously, we read Differential Revisions one by one by calling
`differential.query`.
Fetching them one by one is suboptimal. Unfortunately, there is no Conduit
API that allows us to get a stack of diffids using a single API call.
This patch tries to be smarter using a simple heuristic: when fetching D59
as a stack, previous IDs like D51, D52, D53, ..., D58 are likely belonging
to a same stack so just fetch them as well. Since `differential.query` only
returns cheap metadata without expensive diff content, it shouldn't be a big
problem for the server.
Using a test Phabricator instance, this patch reduces `phabread` reading a
10 patch stack from about 13 to 30 seconds to 8 seconds.
Previously, we call differential.getcommitmessage API to get commit
messages. Now we read that from "Differential Revision" object fetched
via "differential.query" API.
This removes one API call per patch.
This patch reworked phabread a bit so it fetches the lightweight
"Differential Revision" metadata for a stack first. Then read other data.
This allows the code to:
a) send 1 request to get all `hg:meta` data instead of N requests
b) patches are read in desired order, no need to buffer the output
"b)" reduces the memory usage from O(N^2) to O(N) since we no longer keep
old patch contents in memory.
The above `N` means the number of patches in the stack.
Previously we didn't abort. Now we abort if revs is empty. This is
consistent with "hg export" behavior. Maybe "return 1" is also a reasonable
behavior, but that's inconsistent with the existing "hg export".
Previously, `phabsend` uploads new diffs as long as the commit hash changes.
That's suboptimal because sometimes the diff is exactly the same as before,
the commit hash change is caused by a parent hash change, or commit message
change which do not affect diff content.
This patch adds a check examining actual diff contents to skip uploading new
diffs in that case.
The "hg:meta" property is for extra metadata to reconstruct the patch.
Previously it does not have node or parent information since I think by
reading the patch again, the commit message will be mangled (like, added the
"Differential Revision" line) and we cannot preserve the commit hash.
However, the "parent" information could be useful. It could be helpful to
locate the "base revision" so in case of a conflict applying the patch
directly, we might be able to use 3-way merge to resolve it correctly.
Note: "local:commits" is an existing "property" used by Phabricator that has
the node and parent information. However, it lacks of timezone information
and requires "author" and "authorEmail" to be separated. So we are using a
different "property" - "hg:meta" to be distinguished from "local:commits".
Previously, only tags can "associate" a changeset to a Differential
Revision. But the usual pattern (arc patch or hg phabread) is to put the
Differential Revision URL in commit message.
This patch makes the code read commit message to find associated
Differential Revision if associated tags are not found.
This makes some workflows possible. For example, if the author loses their
repo, or switch to another computer, they can continue download their own
patches from Phabricator and update them without needing to manually create
tags.
This patch adds a `phabread` command generating plain-text patches from
Phabricator, suitable for `hg import`. It respects `hg:meta` so user and
date information might be preserved. And it removes `Summary:` field name
which makes the commit message a bit tidier.
To support stacked diffs, a `--stack` flag was added to read dependent
patches recursively.
The `phabsend` command is intended to provide `hg email`-like experience -
sending a stack, setup dependency information and do not amend existing
changesets.
It uses differential.createrawdiff and differential.revision.edit Conduit
API to create or update a Differential Revision.
Local tags like `D123` are written indicating certain changesets were sent
to Phabricator. The `phabsend` command will use obsstore and tags
information to decide whether to update or create Differential Revisions.
The default Phabricator client arcanist is not friendly to send a stack of
changesets. It works better when a feature branch is reviewed as a single
review unit. However, we want multiple revisions per feature branch.
To be able to have an `hg email`-like UX to send and receive a stack of
commits easily, it seems we have to re-invent things. This patch adds
`phabricator.py` speaking Conduit API [1] in `contrib` as the first step.
This may also be an option for people who don't want to run PHP.
Config could be done in `hgrc` (instead of `arcrc` or `arcconfig`):
[phabricator]
# API token. Get it from https://phab.mercurial-scm.org/conduit/login/
token = cli-xxxxxxxxxxxxxxxxxxxxxxxxxxxx
url = https://phab.mercurial-scm.org/
# callsign is used by the next patch
callsign = HG
This patch only adds a single command: `debugcallconduit` to keep the patch
size small. To test it, having the above config, and run:
$ hg debugcallconduit diffusion.repository.search <<EOF
> {"constraints": {"callsigns": ["HG"]}}
> EOF
The result will be printed in prettified JSON format.
[1]: Conduit APIs are listed at https://phab.mercurial-scm.org/conduit/
The ignore regular expression has been updated to detect
"inconsistent config." If present, we track which configs have
that set and we suppress the conflicting defaults error for those
options.
I also added named groups to the regexp to aid readability.
A comment was added to profiling.py to make a desired inconsistent
value error go away.
For builds that run on hermetic environments, it's possible that the "less"
package is not installed by default, yet it's needed for tests to pass after
revision ca1519568a93 (which sets less as the fallback pager).
When trying to create a zstd bundle, the MSI based install said:
abort: compression engine zstd could not be loaded
The Inno installer is unaffected. The name will need to be updated to include
'cext' when merging into default.
Having linux wheels is going to helps system without compiler or python-dev
plus speed up the installation for everyone.
I followed the manylinux example repository
https://github.com/pypa/python-manylinux-demo
to add a make target (build-linux-wheels) using
official docker image to build python 2 linux wheels
for mercurial. It generates Python 2.6 and Python 2.7 for both
32 and 64 bits architectures.
I had to blacklist several test cases for various reasons:
* test-convert-git.t and test-subrepo-git.t because of the git version
* test-patchbomb-tls.t because of warning using tls 1.0
It's likely because the docker image is based on centos 5.0 and
openssl is outdated.
Now that environment variables override system-wide hgrc settings, we
can default Mercurial to sensible-editor and sensible-pager by default
for debian users.
Some shared-ssh installations assume that 'hg serve --stdio' is a safe
command to run for minimally trusted users. Unfortunately, the messy
implementation of argument parsing here meant that trying to access a
repo named '--debugger' would give the user a pdb prompt, thereby
sidestepping any hoped-for sandboxing. Serving repositories over HTTP(S)
is unaffected.
We're not currently hardening any subcommands other than 'serve'. If
your service exposes other commands to users with arbitrary repository
names, it is imperative that you defend against repository names of
'--debugger' and anything starting with '--config'.
The read-only mode of hg-ssh stopped working because it provided its hook
configuration to "hg serve --stdio" via --config parameter. This is banned for
security reasons now. This patch switches it to directly call ui.setconfig().
If your custom hosting infrastructure relies on passing --config to
"hg serve --stdio", you'll need to find a different way to get that configuration
into Mercurial, either by using ui.setconfig() as hg-ssh does in this patch,
or by placing an hgrc file someplace where Mercurial will read it.
mitrandir@fb.com provided some extra fixes for the dispatch code and
for hg-ssh in places that I overlooked.
This patch also makes some expected output lines in tests glob-ed for
persistence of them.
BTW, files below aren't yet changed in 2017, but this patch also
updates copyright of them, because:
- mercurial/help/hg.1.txt
almost all of "man hg" output comes from online help of hg
command, and is already changed in 2017
- mercurial/help/hgignore.5.txt
- mercurial/help/hgrc.5
"copyright 2005-201X Matt Mackall" in them mentions about
copyright of Mercurial itself
This patch also adds new check-code.py pattern to detect invalid usage
of "mercurial@selenic.com".
Change for test-convert-tla.t is tested, but similar change for almost
same test-convert-baz.t isn't yet tested actually, because I couldn't
find out the way to get "GNU Arch baz client".
AFAIK, buildbot skips test-convert-baz.t, too. Does anybody have
appropriate environment for testing?
This is a cherry pick of an upstream fix. The free() of uninitialed
memory could likely only occur if a malloc() inside zstd fails.
The patched functions aren't currently used by Mercurial. But I don't
like leaving footguns sitting around.
Previously, when runcommand raises, chg aborts with, and does not wait for
pager. The call stack is like:
hgc_runcommand -> handleresponse -> readchannel -> debugmsg("failed to
read channel") -> exit(255)
That means, chg returns to the shell, then both the pager and the shell will
read from the terminal at the same time, causing problems.
This patch fixes that by using "atexit" to register the pager cleanup
function so chg will always wait for pager even if runcommand raises.
Commit 63c68d6f5fc8de4afd9bde81b13b537beb4e47e8 from
https://github.com/indygreg/python-zstandard is imported without
modifications (other than removing unwanted files).
This includes minor performance and feature improvements. It also
changes the vendored zstd library from 1.1.1 to 1.1.2.
# no-check-commit
pycompat.getenv returns os.getenvb on py3 which is not available on Windows.
This patch replaces them with encoding.environ.get and checks to ensure no
new instances of os.getenv or os.setenv are introduced.
Module-level @cachefunc usage is risky because it can easily create a
memory "leak". Let's reject it completely for now. If a valid usage
comes up in the future, we can always improve the check or reconsider.
Now that the revlog has a reference to a compressor, it is
possible to swap in other compression engines. So, teach
`hg perfrevlogchunks` to do that.
The default behavior of `hg perfrevlogchunks` is now to measure the
compression performance of all compression engines implementing the
revlog compressor API. This effectively adds the no-op "none"
compressor and zstd (when available) into the default set.
While we can't yet plug alternate compressors into revlogs, this
command gives us a preview of the performance. On the mozilla-unified
repository:
$ hg perfrevlogchunks -c
! compress w/ none
! wall 0.115159 comb 0.110000 user 0.110000 sys 0.000000 (best of 86)
! compress w/ zlib
! wall 5.681406 comb 5.680000 user 5.680000 sys 0.000000 (best of 3)
! compress w/ zstd
! wall 2.624781 comb 2.620000 user 2.620000 sys 0.000000 (best of 4)
$ hg perfrevlogchunks -m
! compress w/ none
! wall 0.124486 comb 0.120000 user 0.120000 sys 0.000000 (best of 79)
! compress w/ zlib
! wall 10.144701 comb 10.150000 user 10.150000 sys 0.000000 (best of 3)
! compress w/ zstd
! wall 4.383118 comb 4.390000 user 4.390000 sys 0.000000 (best of 3)
Those numbers for zstd look promising. But they aren't the full story.
For that, we'll need to look at decompression times and storage sizes.
Stay tuned...
Upcoming patches will convert revlogs to use the compression engine
APIs to perform all things compression. The yet-to-be-introduced
APIs support a persistent "compressor" object so the same object
can be reused for multiple compression operations, leading to
better performance. In addition, compression engines like zstd
may wish to tweak compression engine state based on the revlog
(e.g. per-revlog compression dictionaries).
A global and shared decompress() function will shortly no longer
make much sense. So, we move decompress() to be a method of the
revlog class. It joins compress() there.
On the mozilla-unified repo, we can measure the impact of this change
on reading performance:
$ hg perfrevlogchunks -c
! chunk
! wall 1.932573 comb 1.930000 user 1.900000 sys 0.030000 (best of 6)
! wall 1.955183 comb 1.960000 user 1.930000 sys 0.030000 (best of 6)
! chunk batch
! wall 1.787879 comb 1.780000 user 1.770000 sys 0.010000 (best of 6
! wall 1.774444 comb 1.770000 user 1.750000 sys 0.020000 (best of 6)
"chunk" appeared to become slower but "chunk batch" got faster. Upon
further examination by running both sets multiple times, the numbers
appear to converge across all runs. This tells me that there is no
perceived performance impact to this refactor.
Selecting single and multiple revisions is closely related, so let's
put it in one place, so users can easily find it. We actually did not
even point to "hg help revsets" from "hg help revisions", but now that
they're on a single page, that won't be necessary.
This patch uses the newly introduced "setprocname" interface to update the
process title server-side, to make it easier to tell what a worker is actually
doing.
The new title is "chg[worker/$PID]", where PID is the process ID of the
connected client. It can be directly observed using "ps -AF" under Linux, or
"ps -A" under FreeBSD.
We have enough bits to switch to the new chg pager code path in runcommand.
So just remove the legacy getpager support.
This is a red-only patch, and will break chg's pager support temporarily.
This patch implements the simple S-channel pager handling at chg
client-side.
Note: It does not deal with environ and cwd currently for simplicity, which
will be fixed later.
Previously S channel is only used to send system commands. It will also be
used to send pager commands. So add a type parameter.
This breaks older chg clients. But chg and hg should always come from a
single commit and be packed into a single package. Supporting running
inconsistent versions of chg and hg seems to be unnecessarily complicated
with little benefit. So just make the change and assume people won't use
inconsistent chg with hg.
After dropping the garbage collector hack, `hg perfstartup` started yelling
about not being able to import the evolve extension, which I have in my user
config. Launching `env` shows that an empty HGRCPATH isn't exported to the
environment. Since `env` doesn't quote, I have no idea if the variable is
trimmed, but Mercurial doesn't complain when processing it.
We previously weren't looking for this config helper. And, surprise,
profiling.py references config options without docs.
If I tried hard enough, I could have combined the regexps using a
positive lookbehind assertion or something. But I didn't want to make
my brain explode.
At some point, we should probably do this linting at the tokenizer or
ast layer. I'm not willing to open that can of worms right now.
Instead of mentioning 127.0.0.1, we should use $LOCALIP. Anytime
$LOCALIP appears in output, we should make sure we use (glob) on that
line of output so that weird environments that do remapping jiggery
pokery (such as our FreeBSD buildbot that's in a jail) don't get
spurious test failures.
Commit 81e1f5bbf1fc54808649562d3ed829730765c540 from
https://github.com/indygreg/python-zstandard is imported without
modifications (other than removing unwanted files).
Updates relevant to Mercurial include:
* Support for multi-threaded compression (we can use this for
bundle and wire protocol compression).
* APIs for batch compression and decompression operations using
multiple threads and optimal memory allocation mechanism. (Can
be useful for revlog perf improvements.)
* A ``BufferWithSegments`` type that models a single memory buffer
containing N discrete items of known lengths. This type can be
used for very efficient 0-copy data operations.
# no-check-commit
I softly formalized the concept of a "bundle specification" a while
ago when I was working on clone bundles and stream clone bundles and
wanted a more robust way to define what exactly is in a bundle file.
The concept has existed for a while. Since it is part of the clone
bundles feature and exposed to the user via the "-t" argument to
`hg bundle`, it is something we need to support for the long haul.
After the 4.1 release, I heard a few people comment that they didn't
realize you could generate zstd bundles with `hg bundle`. I'm
partially to blame for not documenting it in bundle's docstring.
Additionally, I added a hacky, experimental feature for controlling
the compression level of bundles in 054e64c4d837. As the commit
message says, I went with a quick and dirty solution out of time
constraints. Furthermore, I wanted to eventually store this
configuration in the "bundlespec" so it could be made more flexible.
Given:
a) bundlespecs are here to stay
b) we don't have great documentation over what they are, despite being
a user-facing feature
c) the list of available compression engines and their behavior isn't
exposed
d) we need an extensible place to modify behavior of compression
engines
I want to move forward with formalizing bundlespecs as a user-facing
feature. This commit does that by introducing a "bundlespec" help
page. Leaning on the just-added compression engine documentation
and API, the topic also conveniently lists available compression
engines and details about them. This makes features like zstd
bundle compression more discoverable. e.g. you can now
`hg help -k zstd` and it lists the "bundlespec" topic.
Currently, Mercurial has a number of commands to show information. And,
there are features coming down the pipe that will introduce more
commands for showing information.
Currently, when introducing a new class of data or a view that we
wish to expose to the user, the strategy is to introduce a new command
or overload an existing command, sometimes both. For example, there is
a desire to formalize the wip/smartlog/underway/mine functionality that
many have devised. There is also a desire to introduce a "topics"
concept. Others would like views of "the current stack." In the
current model, we'd need a new command for wip/smartlog/etc (that
behaves a lot like a pre-defined alias of `hg log`). For topics,
we'd likely overload `hg topic[s]` to both display and manipulate
topics.
Adding new commands for every pre-defined query doesn't scale well
and pollutes `hg help`. Overloading commands to perform read-only and
write operations is arguably an UX anti-pattern: while having all
functionality for a given concept in one command is nice, having a
single command doing multiple discrete operations is not. Furthermore,
a user may be surprised that a command they thought was read-only
actually changes something.
We discussed this at the Mercurial 4.0 Sprint in Paris and decided that
having a single command where we could hang pre-defined views of
various data would be a good idea. Having such a command would:
* Help prevent an explosion of new query-related commands
* Create a clear separation between read and write operations
(mitigates footguns)
* Avoids overloading the meaning of commands that manipulate data
(bookmark, tag, branch, etc) (while we can't take away the
existing behavior for BC reasons, we now won't introduce this
behavior on new commands)
* Allows users to discover informational views more easily by
aggregating them in a single location
* Lowers the barrier to creating the new views (since the barrier
to creating a top-level command is relatively high)
So, this commit introduces the `hg show` command via the "show"
extension. This command accepts a positional argument of the
"view" to show. New views can be registered with a decorator. To
prove it works, we implement the "bookmarks" view, which shows a
table of bookmarks and their associated nodes.
We introduce a new style to hold everything used by `hg show`.
For our initial bookmarks view, the output varies from `hg bookmarks`:
* Padding is performed in the template itself as opposed to Python
* Revision integers are not shown
* shortest() is used to display a 5 character node by default (as
opposed to static 12 characters)
I chose to implement the "bookmarks" view first because it is simple
and shouldn't invite too much bikeshedding that detracts from the
evaluation of `hg show` itself. But there is an important point
to consider: we now have 2 ways to show a list of bookmarks. I'm not
a fan of introducing multiple ways to do very similar things. So it
might be worth discussing how we wish to tackle this issue for
bookmarks, tags, branches, MQ series, etc.
I also made the choice of explicitly declaring the default show
template not part of the standard BC guarantees. History has shown
that we make mistakes and poor choices with output formatting but
can't fix these mistakes later because random tools are parsing
output and we don't want to break these tools. Optimizing for human
consumption is one of my goals for `hg show`. So, by not covering
the formatting as part of BC, the barrier to future change is much
lower and humans benefit.
There are some improvements that can be made to formatting. For
example, we don't yet use label() in the templates. We obviously
want this for color. But I'm not sure if we should reuse the existing
log.* labels or invent new ones. I figure we can punt that to a
follow-up.
At the aforementioned Sprint, we discussed and discarded various
alternatives to `hg show`.
We considered making `hg log <view>` perform this behavior. The main
reason we can't do this is because a positional argument to `hg log`
can be a file path and if there is a conflict between a path name and
a view name, behavior is ambiguous. We could have introduced
`hg log --view` or similar, but we felt that required too much typing
(we don't want to require a command flag to show a view) and wasn't
very discoverable. Furthermore, `hg log` is optimized for showing
changelog data and there are things that `hg display` could display
that aren't changelog centric.
There were concerns about using "show" as the command name.
Some users already have a "show" alias that is similar to `hg export`.
There were also concerns that Git users adapted to `git show` would
be confused by `hg show`'s different behavior. The main difference
here is `git show` prints an `hg export` like view of the current
commit by default and `hg show` requires an argument. `git show`
can also display any Git object. `git show` does not support
displaying more complex views: just single objects. If we
implemented `hg show <hash>` or `hg show <identifier>`, `hg show`
would be a superset of `git show`. Although, I'm hesitant to do that
at this time because I view `hg show` as a higher-level querying
command and there are namespace collisions between valid identifiers
and registered views.
There is also a prefix collision with `hg showconfig`, which is an
alias of `hg config`.
We also considered `hg view`, but that is already used by the "hgk"
extension.
`hg display` was also proposed at one point. It has a prefix collision
with `hg diff`. General consensus was "show" or "view" are the best
verbs. And since "view" was taken, "show" was chosen.
There are a number of inline TODOs in this patch. Some of these
represent decisions yet to be made. Others represent features
requiring non-trivial complexity. Rather than bloat the patch or
invite additional bikeshedding, I figured I'd document future
enhancements via TODO so we can get a minimal implmentation landed.
Something is better than nothing.
In filerevision view (/file/<rev>/<fname>) we add some event listeners on
mouse clicks of <span> elements in the <pre class="sourcelines"> block.
Those listeners will capture a range of lines selected between two mouse
clicks and a box inviting to follow the history of selected lines will then
show up. Selected lines (i.e. the block of lines) get a CSS class which make
them highlighted. Selection can be cancelled (and restarted) by either
clicking on the cancel ("x") button in the invite box or clicking on any other
source line. Also clicking twice on the same line will abort the selection and
reset event listeners to restart the process.
As a first step, this action is only advertised by the "cursor: cell" CSS rule
on source lines elements as any other mechanisms would make the code
significantly more complicated. This might be improved later.
All JavaScript code lives in a new "linerangelog.js" file, sourced in
filerevision template (only in "paper" style for now).
revlog.revision takes either node or rev, but taking a rev is more
efficient, because converting rev to node is just a seek and read.
That's cheaper than converting node to rev, which may require O(n) walk in
revlog index for the first times, and then triggering building the radix
tree index. Even with the radix tree built, rev -> node is still faster than
node -> rev because the radix tree requires more jumps in memory.
So r.revision(r.node(rev)) should be changed to r.revision(rev). This patch
adds a check-code rule to detect that.
This is similar to what 9704c8e70d2d does. Since 1363aaf74791 has changed
"127.0.0.1" to "$LOCALIP". The glob pattern needs update accordingly. It is
expected to fix tests running in some BSD jails.
This will ensure new extensions are consistent and `hg help -e` has a
consistent output.
I have to add a new group since the normal "pypats" will be filtered by
"pyfilters", which will remove comments and docstrings.
Previously, chg.c maintains the pagerpid. Let's move it to procutil.c.
Note: chg.c still have a pagerpid to decide whether to call attachio or not.
In the future, attachio may be moved from hgc_open to hgc_runcommand, and
hgc_runcommand handles both pager and attachio so we don't need to run
attachio twice. And chg.c will be free of pagerpid.
In the future hgclient will deal with pager directly inside runcommand, so
related signal handling stuff needs to be decoupled from chg.c.
The signal handling and pager logic are coupled because we need to forward
SIGPIPE when pager exits. So they are moved together, otherwise a global
variable (pagerpid) is inevitable.
This patch moves related functions from chg.c to procutil.c, which was
marked as copied to maintain annotate history.
The move is done without code modification for easy review, therefore
`#include "procutil.c"` was introduced temporarily.
$XDG_RUNTIME_DIR [1] is a better place for user daemons. Let's use it and
fallback to $TMPDIR.
After this patch, chg will try socket paths in the following order:
1. $CHGSOCKNAME
2. $XDG_RUNTIME_DIR/chg/server
3. ${TMPDIR:-tmp}/chg$UID/server
[1]: https://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html
"sizeof(sun_path)" is too small. Use the chdir trick to support long socket
path, like "mercurial.util.bindunixsocket".
It's useful for cases where TMPDIR is long. Modern OS X rewrites TMPDIR to a
long value. And we probably want to use XDG_RUNTIME_DIR [2] for Linux.
The approach is a bit different from the previous plan, where we will have
hgc_openat and pass cmdserveropts.sockdirfd to it. That's because the
current change is easier: chg has to pass a full path to "hg" as the
"--address" parameter. There is no "--address-basename" or "--address-dirfd"
flags. The next patch will remove "sockdirfd".
Note: It'd be nice if we can use a native "connectat" implementation.
However, that's not available everywhere. Some platform (namely FreeBSD)
does support it, but the implementation has bugs so it cannot be used [2].
[1]: https://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html
[2]: https://www.mercurial-scm.org/pipermail/mercurial-devel/2016-April/082892.html
We have our own bytes versions of things like, getopt.getopt, os.sep, os.name,
sys.executable, os.environ and few more for python 3 portability. Its better
to come up with warnings if someone breaks the things which we have fixed.
After this patch, check-code will warn us to use our bytes version.
These checks run on mercurial/ and hgext/ and pycompat.py is excluded.
See the previous two patches for the reason. The advantage is a simplified
code base and better throughput when starting multiple servers with multiple
confighashes. The disadvantage is starting multiple servers in parallel with
a single confighash will waste some CPU time, which is probably fine in
common use-cases.
This makes it easier to switch to relative paths to support long unix domain
socket paths.
See the previous patch for motivation. Previously, the server is started at
a globally shared address. This patch appends pid to the address so it
becomes unique.
Note: with Linux pid namespace, the address may be non-unique, but it does
not affect correctness of chg - chg client will receive an redirection and
that's it.
