splitblock() was added to handle blocks returned by bdiff.blocks() which differ
only by blank lines but are not made only of blank lines. I do not know exactly
how it could happen but mdiff.blocks() threshold behaviour makes me think it
can if those blocks are made of very popular lines mixed with popular blank
lines. If it is proven to be wrong, the function can be dropped.
The first implementation made annotate share diff configuration entries. But it
looks like users will user -w/b for annotate but not for diff, on both the
command line and hgweb. Since the latter cannot use command line entries, we
introduce a new [annotate] section duplicating the diff whitespace options.
Older clients will still print the provided error message and not much else:
over ssh, this will be each line prefixed with 'remote: ' in addition to an
"abort: unexpected response: '\n'"; over http, this will be the '---%<---'
banners in addition to the 'does not appear to be a repository' message.
Currently, clients with this patch will display 'abort: remote error:\n' and
the provided error text, but it is trivial to style the error text however is
deemed appropriate.
Before: hgweb made it possible to download file content with a content type
detected from the file extension. It would serve .html files as text/html and
could thus cause XSS vulnerabilities if the web site had any kind of session
authorization and the repository content wasn't fully trusted.
Now: all files default to "application/binary", which all important
browsers will refuse to treat as text/html. See the table here:
https://code.google.com/p/browsersec/wiki/Part2#Survey_of_content_sniffing_behaviors
In the branches page, branches that are closed and are merged into another
branch are displayed as `inactive'. This patch changes that behaviour to
show these branches as `closed'.
For me, the `closed' attribute is more important than the `inactive'
attribute.
Branches that are not closed, and are merged into other branches will still
be shown as `inactive'.
Branches that are closed, and are not merged into other branches will still
be shown as `closed'.
This is the same message displayed at the end of the "diff --stat" command.
For example, "9 files changed, 1651 insertions(+), 2 deletions(-)".
The webutil.diffstatgen function allows the diffstat data to be lazily
calculated only once and then re-used.
This allows the diffstat templates to link into the diff output. For example,
the URLs of the first three files within the diff are #l1.1, #l2.1, #l3.1.
The webutil.diffstat function now returns a diffstat template for each file
in the diff. It previously returned a template for each file returned by
ctx.files() which did not work well for merge changesets.
Remove the lambda used to wrap webutil.diffstat because:
- functions passed to the templater must accept keyword arguments
- webutil.diffstat is a generator, so already calculates the diffstat lazily
This reverts the changes made to a4067e29e29d after it was submitted to the
mailing list but before it was queued.
This includes all affected files, so it can be used for an extended view of
the files or as a replacement for the filenodelink and filenolink templates.
This change complements the existing web/logourl setting, and lets the user
customize the logo image that is shown on many of the hg server pages.
If this setting is not set, hglogo.png is used.
Send the command arguments in the HTTP headers. The command is still part
of the URL. If the server does not have the 'httpheader' capability, the
client will send the command arguments in the URL as it did previously.
Web servers typically allow more data to be placed within the headers than
in the URL, so this approach will:
- Avoid HTTP errors due to using a URL that is too large.
- Allow Mercurial to implement a more efficient wire protocol.
An alternate approach is to send the arguments as part of the request body.
This approach has been rejected because it requires the use of POST
requests, so it would break any existing configuration that relies on the
request type for authentication or caching.
Extensibility:
- The header size is provided by the server, which makes it possible to
introduce an hgrc setting for it.
- The client ignores the capability value after the first comma, which
allows more information to be included in the future.
The introduction of the new URL parsing code has created a startup
time regression. This is mainly due to the use of url.hasscheme() in
the ui class. It ends up importing many libraries that the url module
requires.
This fix helps marginally, but if we can get rid of the urllib import
in the URL parser all together, startup time will go back to normal.
perfstartup time before the URL refactoring (707e4b1e8064):
! wall 0.050692 comb 0.000000 user 0.000000 sys 0.000000 (best of 100)
current startup time (9ad1dce9e7f4):
! wall 0.070685 comb 0.000000 user 0.000000 sys 0.000000 (best of 100)
after this change:
! wall 0.064667 comb 0.000000 user 0.000000 sys 0.000000 (best of 100)
By default, hgweb_mod supports caching via the ETag header. This can
cause some confusion with browsers which cache aggressively. This change
preserves existing behavior while giving the administrator a knob to
disable the ETag header.
Clicking on the logo image/text in the hgweb interface brings the
user to the Mercurial project page. The majority of users expect that
this would bring them to the top level index. I have added a new template
variable named `logourl' which allows an administrator to change this
behavior. To stay compatible with existing behavior, `logourl' will
default to http://mercurial.selenic.com/. This change is very useful in
large installations where jumping to the index is common.
Repository() raises a number of IOErrors in addition to RepoErrors.
these are just as uninteresting as RepoErrors and should be ignored
The easiest case of this is a repo whose .hg/ directory is -rx
Invalid requests could give an unhandled ErrorResponse.
Now this ErrorResponse is handled like other ErrorResponses so the client gets
an error message which also is logged on the server.
Other exceptions than StandardExceptions were left to the default error handler
which was muted when running in daemon mode.
Now all Exceptions are handled and logged to the log file.
The archive list generator was holding a reference to each
temporary ui copy passed by rawentries(), so the memory
usage for index generation growed proportionally to the
ui object size and the amount of repositories. By returning a
list instead, the temporary reference is dropped immediately.
Add missing calls to close() to many places where files are
opened. Relying on reference counting to catch them soon-ish is not
portable and fails in environments with a proper GC, such as PyPy.
The only revision information yielded by the annotate view was the revision
number itself. The patch allows the use of per-line revision dates in the
corresponding templates.
8aa8db6deb47 moved the subdirectory match inside the repository match
loop. A virtual path existing/path/invalid/path would then match
existing/path, and generate a wrong index page.
PROTOCOL_SSLv3 on the server side doesn't work everywhere. Sometimes the client
reports "EOF occurred in violation of protocol" (for example on Mac and Solaris).
The more compatible PROTOCOL_SSLv23 is now used instead. It works but is less
"secure" for some OpenSSL versions as it can fall back to weak encryption.
pyOpenSSL apparently doesn't work for Python 2.7 and isn't very actively
maintained.
The built-in ssl module seems like a long-term winner, so we now use that with
Python 2.6 and higher.
The https mode failed in super because BaseRequestHandler is an old-style
class.
This introduces the first test of https client/server functionality - and
"hghave ssl". The test is currently only run on Python 2.6.
Without this fix, mod_wsgi and spawning get in a wedged state after
sending a 304 response. Not sending a body fixed that problem. The
header change was discovered by using wsgiref.validate.validator to
check for other errors.
The behaviour between http and ssh still differ:
- the "unsynced changes" is seen as a remote output in the http cases
- but it is correctly seen as a push error for ssh
The content type for both .tar.gz and .tar.bz2 downloads was
application/x-tar, which is correct for .tar files when no
Content-Encoding is present, but is not correct for .tar.gz and .tar.bz2
files unless Content-Encoding is set to gzip or x-bzip2, respectively.
However, setting Content-Encoding causes browsers to undo that encoding
during download, when a .gz or .bz2 file is usually the desired
artifact. Omitting the Content-Encoding header is preferred to avoid
having browsers uncompress non-render-able files.
Additionally, the Content-Disposition line indicates a final desired
filename with .tar.gz or .tar.bz2 extension which makes providing a
Content-Encoding header inappropriate.
With the current configuration browsers (Chrome and Firefox thus far)
are registering the application/x-tar Content-Type and not .tar
extension and appending that extension, yielding filename.tar.gz.tar as
a final on-disk artifact. This was originally reported here:
http://stackoverflow.com/questions/3753659
I've changed the .tar.gz and .tar.bz2 Content-Type values to
application/x-gzip and application/x-bzip2, respectively. Which yields
correctly named download artifacts on Firefox, Chrome, and IE.
This adds util.getport(port) which tries to parse port as an int, and
failing that, looks it up using socket.getservbyname(). Thus, the
following will work:
[smtp]
port = submission
[web]
port = http
This does not apply to ports in URLs used in clone, pull, etc.
If hgweb.config contains wrong path mapping, hgweb causes internal server
error on repository index page.
This patch changes makeindex() to ignore RepoError, because it looks to be
designed to suppress configuration error.
This allows the (to me, in some setups) obvious configuration:
[paths]
/ = *
or
/ = **
Relative paths used to work with [collections] even though it isn't documented.
Perhaps it should be documented?
getbundle(common, heads) -> bundle
Returns the changegroup for all ancestors of heads which are not ancestors of common. For both
sets, the heads are included in the set.
Intended to eventually supercede changegroupsubset and changegroup. Uses heads of common region
to exclude unwanted changesets instead of bases of desired region, which is more useful and
easier to implement.
Designed to be extensible with new optional arguments (which will have to be guarded by
corresponding capabilities).
Clients that send 100-continue should make sure they really support
continue intelligently. In a later patch we'll introduce a capability so
new clients don't pay a performance penalty talking to old servers.
Currently, callers of addchangegroup first acquire the repository
lock, usually to check that an unbundle request isn't racing. This
means that changegroup hook actions that might write to a repo get
stuck waiting for a lock. Here, we add a new optional lock parameter
and update all the callers. Post-1.6 we may make it non-optional.
Quoting PEP 333 (WSGI):
"If the iterable returned by the application has a close() method, the server
or gateway must call that method upon completion of the current request,
whether the request was completed normally, or terminated early due to
an error. (This is to support resource release by the application.
This protocol is intended to complement PEP 325's generator support,
and other common iterables with close() methods."
This fixes doubled URL, e.g. http://example.orghttp://example.org/...,
which appears on RSS/Atom feeds served by hgwebdir.
It splits baseurl to update SERVER_NAME, SERVER_PORT and SCRIPT_NAME,
according to RFC 3875.
Updated the test output since SCRIPT_NAME becomes not to contain
http://host:port part.
Entering "<correct_cset_hash> " in the search form was not returning anything.
This happens relatively often, due to HTML formatting: when copy/pasting a cset
hash from the web, selection might contain surrounding spaces.
This allows using
application = hgwebdir(...)
application.motd = (string or object with __str__ method)
in WSGI (like it is possible in CGI).
Changed web.motd in the config file is still read with this, because
hgwebdir.templater.motd() does not store the config value.
Fixes a bug in protocol which caused an exception during exception handling in
some cases on Windows. Also makes sure the server error message is correctly
propagated to the client, instead of being thrown away.
Previously, the acl extension just read the current system user, which
is fine for direct file system access and SSH, but will not work for
HTTP(S) as that would return the web server process user identity
rather than the authenticated user. An empty user is returned if the
user is not authenticated.
If DNS lookups are turned off on the web server, REMOTE_HOST may be
populated with REMOTE_ADDR, which, if the remote is an IPv6 hosts will
contain colons, thus interfering with the separator character. This is
solved by URL quoting the REMOTE_HOST string.
Changes graph() to colorededges(), which operates on the new
generic DAG walks and adds color and edge information needed
by the web graph.
This is in preparation of adding DAG walk filters, like the
linear run collapser in the next patch. The idea is to have
a bunch of changelog walkers that return basic data. Then we
can filter this data. Finally we add edge and formatting info
suitable for the output media we want to target (glog, hgweb).
- repository heads are not associated with the closed attribute, so
remove it making the code in line with the concept.
- Fix functions that were calling heads with the parameter.
- Adjust webcommands.branches to include the concept of inactive
as well as open and closed branches
- Fix code and docstrings in commands to make the correct use of
closed branches & branch heads clearer
- Improve grammar of 'hg heads' help text (2nd submission)
this does not alter the cli for hg branches, that work is
still to be done
The repository command, 'branchmap', returns a dictionary, branchname
-> [branchheads], and will be implemented for localrepo, httprepo and
sshrepo.
The following wire format is used for returning data:
branchname1 branch1head2 branch1head2 ...
branchname2 ...
...
Branch names are URL encoded to escape white space, and branch heads
are sent as hex encoded node ids. All branches and all their heads are
sent.
The background and motivation for this command is the desire for a
richer named branch semantics when pushing changesets. The details are
explained in the original proposal which is included below.
1. BACKGROUND
The algorithm currently implemented in Mercurial only considers the
graph theoretical heads when determining whether new heads are
created, rather than using the branch heads as a count (the algorithm
considers a branch head effectively closed when it is merged into
another branch or a new named branch is started from that point
onward).
Our particular problem with the algorithm is that we'd like to see the
following case working without forcing a push:
Upsteam has:
(0:dev) ---- (1:dev)
\
`--- (2:stable)
Someone merges stable into dev:
(0:dev) ---- (1:dev) ------(3:dev)
\ /
`--- (2:stable) --------´
This can be pushed without --force (as it should).
Now someone else does some coding on stable (a bug fix, say):
(0:dev) ---- (1:dev) ------(3:dev)
\ /
`--- (2:stable) ---------´---------(4:stable)
This time we need --force to push.
We allow this to be pushed without using --force by getting all the
remote branch heads (by extending the wire protocol with a new
function).
We would, furthermore, also prefer if it is impossible to push a new
branch without --force (or a later --newbranch option so --force isn't
shoe-horned into too many disparate functions, if need be), except of
course in the case where the remote repository is empty.
This is what our patches accomplish.
2. ALTERNATIVES
We have, of course, considered some alternatives to reconstructing
enough information to decide whether we are creating new remote branch
heads, before we added the new wire protocol command.
2.1. LOOKUP ON REMOTE
The main alternative is to use the information from remote.heads() and
remote.lookup() to try to reconstruct enough graph information to
decide whether we are creating new heads. This is not adequate as
illustrated below.
Remember that each lookup is typically a request-response pair over
SSH or HTTP(S).
If we have a simple repository at the remote end like this:
(0:dev) ---- (1:dev) ---- (3:stable)
\
`--- (2:dev)
then remote.heads() will yield [2, 3]. Assume we have nodes [0, 1, 2]
locally and want to create a new node, 4:dev, as a descendant from
(1:dev), which should be OK as 1:dev is a branch head.
If we do remote.lookup('dev') we will get [2]. Thus, we can get
information about whether a branch exists on the remote server or not,
but this does not solve our problem of figuring out whether we are
creating new heads or not.
Pushing 4:dev ought to be OK, since after the push, we still only have
two heads on branch a.
Using remote.lookup() and remote.heads() is thus not adequate to
consistently decide whether we are creating new remote heads (e.g. in
this situation the latter would never return 1:dev).
2.2. USING INCOMING TO RECONSTRUCT THE GRAPH
An alternative would be to use information equivalent to hg incoming
to get the full remote graph in addition to the local graph.
To do this, we would have to get a changegroup(subset) bundle
representing the remote end (which may be a substantial amount of
data), getting the branch heads from an instantiated bundlerepository,
deleting the bundle, and finally, we can compute the prepush logic.
While this is backwards compatible, it will cause a possibly
substantial slowdown of the push command as it first needs to pull in
all changes.
3. FURTHER ARGUMENTS IN FAVOUR OF THE BRANCHMAP WIRE-PROTOCOL EXTENSION
Currently, the commands incoming and pull, work based on the tip of a
given branch if used with "-r branchname", making it hard to get all
revisions of a certain branch only (if it has multiple heads). This
can be solved by requesting the remote's branchheads and letting the
revisions to be used with the command be these heads. This can be done
by extending the commands with a new option, e.g.:
hg pull -b branchname
which will be turned into the equivalent of:
hg pull -r branchhead1 -r branchhead2 -r branchhead3
We have a simple follow-up patch that can do this ready as well
(although not submitted yet as it is pending the acceptance of the
branch patch).
4. WRAP-UP
We generally find that the branchmap wire protocol extension can
provide better named branch support to Mercurial. Currently, some
things, like the initial push scenario in this mail, are fairly
counter-intuitive, and the more often you have to force push, the more
it is likely you will get a lot of spurious and unnecessary merge
nodes. Also, restricting incoming and pull to all changes on a branch
rather than changes on the tip-most head would be a sensible extension
to making named branches a first class citizen in Mercurial.
Currently, named branches sometimes feel like a late-coming unwanted
step-child.
We have run it in a production environment for a while, with fewer
multiple heads occurring in our repositories and fewer confused users
as a result.
Also, it fixes the long-standing issue 736.
Co-contributor: Sune Foldager <cryo@cyanite.org>
This allows extensions to hook into permission checking, providing both
authentication and authorization as needed. The existing authorization
function has been changed to a hook, which is added by default.
The property returns os.environ by default, and is propagated by ui.copy.
During hgweb processing, ui.environ is set to the proper WSGI-request
environment, as contained in wsgirequest.environ. For CGI, this is the
same as os.environ.
The property is meant to be read-only, as with os.environ (generally).
Before this patch, the only way to get hgwebdir to honor the recursive paths
was to create a config object or a config file with the recursive paths in it.
This patch makes hgwebdir treat paths the same whether passed in as a list,
tuple, config or however hgwebdir supports passing paths.
The old default behaviour of hgwebdir was to maintain a list of
repositories permanently. This interacted badly with persistent
application hosting software such as WSGI containers. If a new repository
was published, it would potentially never appear in the top-level list
of repositories.
This change causes the hgwebdir configuration and list of repositories
served to be refreshed periodically (at most every 20 seconds).
Before this patch, hgwebdir suddenly showed up empty for a webdir-conf file
that was for some reason not trusted. This was previously not a problem because
the trust mechanism wasn't required for the very separate webdir-conf.
This cleans up code and allows specification of values more globally. For
example, it's now possible to specify web.contact in webdir-conf for all
repositories without a specified contact set.
This avoids hard-coding the '(none)' string in the JavaScript for each
style and avoids the {nonexisting|nonempty} hack which only works
since the template system don't complain about undefined variables.
