Summary:
Upstream changed the signature of computenonoverlap. Let's change our wrapping
of it to be more robust to signature changes.
Test Plan: ./run-tests.py test-remotefilelog* test-check*
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4062705
Tasks: 14037455
Signature: t1:4062705:1477096049:5a011a7a5edf9bb01475694777c738cdb02453f5
Summary: These variables were unused, causing test-check-pyflakes.t to fail whenever fbamend.py is modified in a way that changes the line numbers of those assignments.
Test Plan: Unit tests still pass (except test-fbamend.t, which fails for reasons unrelated to this change)
Reviewers: durham, #sourcecontrol, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4055599
Signature: t1:4055599:1477049676:84ec9a1036bb6d6873d497caed9feff701a8169b
Summary:
This makes local commits get written to trees (as well as flat manifest) when a
commit happens where the parent commit has a tree manifest already. During a
transaction where multiple trees are written (like when rebasing multiple
nodes), we reuse the same pack file for all the trees produced by tieing into
the transaction abort and close hooks.
Test Plan:
Ran the tests. Ran hg commit with the extension enabled. A future
patch will add an integration test for the treemanifest extension.
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4055851
Signature: t1:4055851:1477059659:91b1c2f93ef986e910cea752ebf2466cb20ac921
Summary:
In a future diff we will start allowing local trees to be written as part of
transactions. Part of this involves making those written trees available to
readers in this process before the transaction is committed. Since all manifest
reads go through hybridmanifest, we can insert a cache at this layer (just like
fastmanifest already has a cache at this layer).
The future patch which adds new manifests will populate this cache, and clear
this cache when the transaction completes.
Test Plan:
Ran the tests. Also tested the actual flow with the future patch that
causes commit to write to the cache
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4055838
Signature: t1:4055838:1477059568:0bf1d79cc1ca6f9bcefb881bab35c61c13641f4e
Summary:
In a future patch we will start writing user local tree data into a local
directory. This patches add the local store to the union store so the contents
will be accessible once we start writing it.
It also renames manifest to manifests for consistency with the other store names
(like 'packs').
Test Plan: Ran the tests
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4055827
Signature: t1:4055827:1477059457:0d5b0d999b47d88c73f5ab2721d8d27deacc01bc
Summary:
In a future diff we will be introducing packs into .hg/store, so we need to
differentiate between cache packs and local packs. This patch renames
getpackpath to getcachepackpath. A future diff will add getlocalpackpath.
This exposed a pyflakes error, so we fix that too.
Test Plan: Ran the tests
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4055815
Signature: t1:4055815:1477059415:e0221557bbeec6701820c826f00390d3a71cd2d3
Summary:
This makes mutablepacks close and abort function accessible from outside the
__exit__ logic. This will be useful later when we tie the lifetime of a
mutablepack to a transaction.
Test Plan: Ran the tests
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4055730
Signature: t1:4055730:1477059602:ffdfd66e65279ddf3ff43d7c2ee65b00f1fd2600
Summary:
This improves the performance of treemanifest.text() from 5s to 3s. This
function is used when converting a treemanifest to a full text.
Test Plan:
Ran the unit tests.
This was visible when running hg commit with the extension enabled. I verified
the time went down from 5 to 3. A future diff will add an integration test suite
for the treemanifest extension.
Reviewers: simpkins, #mercurial, quark
Reviewed By: quark
Subscribers: quark, net-systems-diffs@, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4055693
Signature: t1:4055693:1477067172:47a61fe98e57b4b10e329b7f53f41b80bf15d205
Summary:
It's possible for treemanifests to be created that aren't backed by a flat
manifest (like if you did `foo = treemf.matches(...)`). In these cases, if we
ever need to diff it with a flat text, we need to be able to convert it to flat.
This patches adds the ability to convert it to a flat manifest via the text()
function.
Test Plan: Ran the tests
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4055656
Signature: t1:4055656:1477056921:7417c9ca06d61c5a283c6cf9693d83650e39f196
Summary:
This fixes some (but not all) of the fastmanifest errors. The remaining ones are
from an import * and one that will require slightly more refactoring to fix.
Test Plan: Ran the tests
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4055633
Signature: t1:4055633:1477050763:09e758b1bd72b18799cdabf510eb0b2332077833
Summary:
This fixes all the pyflaks and module errors for the main remotefilelog
code base.
Test Plan: ./run-tests.py test-check* test-remotefilelog*
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4055537
Signature: t1:4055537:1477049663:ee904d311d17d3659e055e2c109c68c9023cfd1f
Summary:
The test generate different outputs on different platforms (devserver,
laptops), possibly related to system python path. Let's remove it until
we have a better solution.
Besides, fix an issue of the arcanist test engine: for skipped tests,
no "duration" information available.
Test Plan:
The test is not referenced explicitly anywhere.
Check "Skip test-check-module-imports-hg.t" shows up in the review page.
Reviewers: #sourcecontrol, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4058782
Signature: t1:4058782:1477067103:789deda43ddb7f439e7c98d574c8c7524e329e45
Summary: Upstream's done a few changes that we need to catch up with.
Test Plan: Run tests locally on my devserver
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4058640
Signature: t1:4058640:1477065516:e2ab2483d054e47b4fe75c6eba185dfd5179f57a
Summary:
User should see no difference between scratch and non-scratch remote bookmarks.
So let's make scratch bookmarks remote if remotenames is enabled.
Unfortunately it requires saving scratch remote bookmarks before push/pull and
restoring it afterwards because `remotenames` extension doesn't know about
them.
Test Plan: Run `test-infinitepush-*`
Reviewers: rmcelroy, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4036653
Tasks: 12479677
Signature: t1:4036653:1476985129:b5c1afdab4c7585c8b12c2962a4b95a56751091b
Summary:
This diff was created when I was investigating hgamend exec not working
with histedit which resulted with [1].
This also brings some very small fixes for sqldirstate and fbamend that
I was too lazy to diff separately.
[1] https://patchwork.mercurial-scm.org/patch/16451/
Test Plan: ran tests
Reviewers: #mercurial, ttung, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3777784
Signature: t1:3777784:1472227075:f2f8a9cad22bf0c45d4554709846b8aafb61d9a0
Summary: We are using scripts/lint.py instead of this one.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4051121
Signature: t1:4051121:1476985509:f2c362a8efe82387bbedd55b126279a1ca5fd8b4
Summary:
"arc lint" runs the lint script for each file in parallel:
>>> [5, pid=27257] <exec> $ scripts/lint.py 'scripts/lint.py'
>>> [6, pid=27257] <exec> $ scripts/lint.py 'scripts/unit.py'
>>> [7, pid=27257] <exec> $ scripts/lint.py 'tests/test-check-code-hg.t'
>>> [8, pid=27257] <exec> $ scripts/lint.py 'tests/test-check-pyflakes-hg.t'
That looks fine but `lint.py` runs `test-check*.t` which will check all
files. This is a typical `N^2` behavior and needs to be fixed.
This diff changes the linter and test code so the test only checks the
single selected file.
Test Plan:
Run `arc lint --trace` and check `lint.py` time usage. It went down from
10 seconds to 2 seconds for this diff.
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4051080
Signature: t1:4051080:1476985355:45845c6eb0e66cbfa000e61b7b496f2d00aeb042
Summary:
Previously, `arc unit` does not run `test-check-code-hg.t` and if people
have ignored lint errors, they may be ignored forever.
Since test-check-code-hg.t is relatively fast (3 seconds), let's also
run it during `arc unit`.
Test Plan: `arc unit` and confirm test-check-code-hg shows up.
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4051073
Signature: t1:4051073:1476985307:5448827437ecee9add534855a0f87ac3c3bd1f84
Summary: now when we always have the fakedirstate it's no longer neccessary
Test Plan: .
Reviewers: ikostia
Reviewed By: ikostia
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4051368
Signature: t1:4051368:1476981079:99caa6e209822aa9529327e15cb66f651f8c3da3
Before this patch, if sqldirstate.fakedirstate configuration is false
value, (fake) dirstate isn't written out (or explicitly unlinked),
In such case, @filecache never detects change of dirstate by another
process/thread, because not-existing file never change its file stat.
To achieve checking cache validity without existence of .hg/dirstate
file, we should use another cache annotation instead of @filecache.
This patch removes code paths to avoid writing (or unlinking) fake
dirstate, and writes fake dirstate always to detect changes correctly.
Just removing these code paths seems reasonable enough, because it was
already confirmed at Mercurial 4.0 sprint that there is no concrete
usecase for sqldirstate.fakedirstate configuration.
With sqldirstate extension, size of .hg/dirstate is always fixed: node
ID x 2 + dummy fixed text.
This increases occurrence of file stat ambiguity, and
processes/threads running parallelly might overlook change of
dirstate, because reloading @filecache-ed property repo.dirstate is
avoided in such case. See wiki page below for detail about file stat
ambiguity.
https://www.mercurial-scm.org/wiki/ExactCacheValidationPlan
This patch specifies checkambig=True at opening .hg/dirstate file for
writing, to avoid file stat ambiguity.
Strictly speaking, to enable sqldirstate with Mercurial earlier than
upcoming 4.0 (or ccbeace526ad), it should be examined whether
checkambig optional argument is available for opener or not, because
it has been available since 4.0 (or ccbeace526ad).
But current sqldirstate implementation depends on 2369a09e89de of
Mercurial, which has introduced dirstate._origpl after ccbeace526ad.
Therefor, changes in this patch are safe enough, unless removing
dependence on dirstate._origpl from sqldirstate.
Summary:
This diff tries to fix `test-check-*` issues for fastannotate.
One issue cannot be fixed:
fastannotate/context.py:31: imports not lexically sorted: linelog < os
If these two imports are swapped, the checker will report:
stdlib import "os" follows local import: linelog
instead. So leave it as-is for now until we get the checker fixed upstream.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4044157
Signature: t1:4044157:1476949486:13052173bc694edd99d21affa336d252bc2aae88
Summary:
Errors like "direct symbol import ... from mercurial/hgext" do not make
sense to this repo. And there is no easy way to extend the whitelist
used by the checker to include "mercurial". So filter them out using
`grep` for now.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4044039
Signature: t1:4044039:1476949701:36ad9f611b6844c1794efe09c426ee3a47a33fec
Summary:
This is the last one of the test-check series.
`test-check-py3-compat.t` was ignored intentionally as we do not care about
Python 3 support right now.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4039479
Signature: t1:4039479:1476876594:5b95aad9be245a86dc9327770502830896273131
Summary:
Hopefully this will prevent new undocumented config options.
Issues about fastannotate are fixed. We still have many other issues, which
are ignored by this diff for now.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4037173
Signature: t1:4037173:1476876491:280e839a8e3d4f4dfbdf6b6953ab860bb7cd3ed4
Summary:
Hopefully this would rule out things like double blank lines and
inconsistent commit title.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4036978
Signature: t1:4036978:1476807264:414cd17672f59675be104c0026a8e50efac8b15d
The package now has a dependency on a .so, so it's
architecture-specific now. Technically we should split the
architecture-specific and architecture-independent bits into separate
packages, but I don't want to bother with that for a one-off package
script like this.
Summary: Benchmarking `hg rebase --restack` against `hg evolve --all` showed that `hg rebase --restack` was spending a lot of time on a single revset to identify the current stack of commits. This diff changes the revset to an equivalent but more efficient one.
Test Plan:
All unit tests still pass.
For benchmarking, I created a stack of 4 commits on top of remote/master in fbsource/fbcode. I amended the second commit and then ran the command. Here is profiler output for both before and after the change:
Before (10.87 seconds): P56747821
After (7.58 seconds): P56747927
Based on the profiler output, this change results in a 30% performance improvement in this case.
Reviewers: durham, #sourcecontrol, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4038630
Tasks: 13976698
Signature: t1:4038630:1476825563:02da1d4e459f780de41d852f88da789513aba7dc
Summary:
If param is set to '0', then `params.get(PARAMNAME)` is True.
This is not what's expected and can lead to subtle bugs in future.
Instead do not send anything if param is not set.
Test Plan: Run `test-infinitepush-*`
Reviewers: durham, mitrandir, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4036504
Tasks: 12479677
Signature: t1:4036504:1476797842:112bae5c318922795c8be5bb5e65458fe9cf034e
Summary:
`timeout` is not available in OSX. Use `gtimeout` instead.
`gtimeout` can usually be installed using `brew install coreutils`.
Test Plan: Run `arc unit`.
Reviewers: simonfar, #sourcecontrol, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4029462
Tasks: 13945749
Signature: t1:4029462:1476784209:5a5d87a2b82314da1c1a0d189dd71d36e16d8c49
Summary:
The current `sshaskpass` implementation relies on `stdin` and `stderr` being
TTYs. That can not be guaranteed if pager is used. With chg enabled, the chg
client will call `attachio` when starting the pager, and it which will override
both `stdin` and `stderr` to the pipes created by the chg pager logic. This
causes failures when a paged command (like `hg diff`) needs interactive ssh
authentication (happens when remotefilelog cache misses).
This patch fixes it by backing up TTY fds inside the `sshaskpass` module. With
chg enabled, the first `attachio` (before starting pager) can provide TTY fds,
where we just back them up. The second `attachio` (after pager started) will
override `ui.fin/fout`, `sys.stdin/stdout` to non-tty fds but it does not matter
since we have backups.
Test Plan:
Run the following command in a remotefilelog repo:
```
rm /tmp/chg$UID /tmp/r -rf; chg diff --config remotefilelog.cachepath=/tmp/r --debug --pager=on
```
And confirm password prompt works as expected.
Reviewers: rmcelroy, #sourcecontrol, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4032236
Signature: t1:4032236:1476755400:80fc6007b6fb2f9fe83cf6570d80a2b431fc069f
Summary:
If `branchpattern` is not set then it matches empty bookmark.
It causes regressions like this t13808497. Let's ignore empty `branchpattern`
Test Plan: Run `test-infinitepush-*`
Reviewers: mitrandir, rmcelroy, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4031365
Tasks: 12479677
Signature: t1:4031365:1476755541:3d3781ef92f1701500d74ce7b98b4d82b5a1228e
Summary:
Remotefilelog can't find file revisions outside of the repo.
Because of it user can't update to the revisions that was pulled from
bundlestore. This diff downloads all filelogs for revisions
from bundlestore.
There were two options to pass info about changed files to remotefilelog:
through config or through bundlecaps. I decided to use config because
it's not really a bundle capability.
Test Plan: Run `test-infinitepush-*`
Reviewers: rmcelroy, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4030065
Tasks: 12479677
Signature: t1:4030065:1476755932:4aa02e540f13d3221163d906524fb9ce97f9c7e8
Summary:
Added the following flags to statprof.py:
- `-f --file` Location of the .data file from the profiler.
- `-s --script-path` Location of the flamegraph.pl script.
- `-o --output-file` Path to write the flamegraph SVG output.
Test Plan: Ran the statprof.py command with various permutations of the aforementioned flags. Running without these flags produces the same behavior as before.
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4034242
Tasks: 13961909
Signature: t1:4034242:1476755279:23d2f3ea2850626ea9afbf60e712bc2b93c2fec0
Summary: "%d" is the wrong format specifier for size_t, and thus this does not build.
Test Plan: make local and see build succeed
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4030507
Signature: t1:4030507:1476726943:44c480a1e231fe98354d386116b112d508436093
Summary:
POSIX requires that the `sed` "insert" command to be followed by a backslash
and a new line character. Let's make the test more POSIX compliant.
Test Plan: Run the test on both OS X and Linux.
Reviewers: simonfar, #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4029954
Signature: t1:4029954:1476720499:0a56da77fa89229ed9606eb9ac3748264fdf12fd
Summary:
Tests doesn't require mysql since it uses fileindexapi.
Let's move sqlindexapi and fileindexapi to different files and include
only necessary file.
Test Plan: Run test-infinitepush-*
Reviewers: durham, simonfar, quark
Reviewed By: quark
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4029460
Tasks: 12479677
Signature: t1:4029460:1476709161:556f80605db9bd3c18dce1e16e996b13bbe96229
Summary:
Previously we can only answer the "path" information when the revision is in
the linelog revmap, and the code would crash if a revision is in a side
branch, and the user requests path information. This diff fixes it.
Besides, this diff improves rename handling. For example, given the following
chart:
```
o---o -o file name: a
/
o---o- file name: b
^ ^ ^
1 2 3 revisions
```
Depending on the position of the `main branch` reference, fastannotate may
or may not use linelog:
- main branch is at rev 2, annotate a -r 3 will not take advantage of linelog
(fallback to slow annotate)
- main branch is at rev 3, annotate a -r 2 will not take advantage of linelog
This is not ideal, but seems to be the best we can do for now.
Test Plan:
Added a new test, updated existing relevant tests. Some debug messages are
changed to reflect internals more precisely.
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4010964
Signature: t1:4010964:1476458201:79875d96399d023d0000d0c4bb8b8d40ea43eef0
Summary:
Fix infinitepush tests for Mac. Let's split test in two.
Now if remotenames extension is unavailable most of the tests will still run.
Test Plan: Run `python run-tests.py test-infinitepush-*`
Reviewers: simonfar, durham, rmcelroy, mitrandir, quark
Reviewed By: quark
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4029394
Tasks: 12479677
Signature: t1:4029394:1476706777:95473f493e10d3f7f21f4679976b33d75d8821f2
