Summary: Attack of the trivial missing space.
Test Plan: Run run-tests.py -l test-sqldirstate-shelve.t to see it pass
Reviewers: rmcelroy, simonfar, quark, jeroenv
Reviewed By: quark, jeroenv
Subscribers: mjpieters, #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D4244663
Signature: t1:4244663:1480439601:3ab7e306e618f7e6e2372046f00e85cf1eb3826d
Summary:
Template lacks of the "padding" feature. There is a hard-coded 13-char wide
username, 5-char wide line number, which does not look well if the usernames
are too long or too short.
This patch restores the original annotate format which is different from
the current behavior. For example, given "-uc", it shows "user node" instead
of "node:user". It also tries to make the template version as close as the
non-template one, by adjusting the order, changing separators, and making
"-c" optional (could be hidden from the output).
Test Plan: Modified existing tests
Reviewers: #sourcecontrol, ikostia
Reviewed By: ikostia
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4243830
Signature: t1:4243830:1480437984:1c679c9db516bab905d027061326a7a1524922c6
Summary: See rd9d8d78e in the hg repository; cmdutil.service moved to server.runservice
Test Plan: Run test-fbconduit.t and observe it passing once more.
Reviewers: simonfar, rmcelroy, quark
Reviewed By: quark
Subscribers: mjpieters, #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D4244268
Signature: t1:4244268:1480432981:8e06beadba912294e6d1a01d93b13848cb16a096
Summary: Yay for arbitrary single space changes.
Test Plan: Run the two affected tests to show they now pass again.
Reviewers: simonfar, rmcelroy, quark, stash
Reviewed By: stash
Subscribers: mjpieters, #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D4244213
Signature: t1:4244213:1480431919:ba4837619652023005faeb00a93a351c7ca0411a
Summary:
D4233433 changed commands to rebuild the broken cache but missed
wireprotocol. Let's do it for wireprotocol as well.
Test Plan: Added a new test
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4237855
Signature: t1:4237855:1480421375:d5715d61a3c66ec8be5a6a80041ee6204e5e4378
Summary: Currently, the reason we can't perform operations such as histedit, split, and fold in the middle of a stack is that we do not have the experimental.evolution=allowunstable option enabled. It turns out all of these commands will do the "right thing" if this option is enabled. However, since we are uncertain of the ramifications of enabling this option globally, this new extension turns the option on for the duration of split, fold, and histedit, allowing those commands to be used in the middle of a stack without config changes.
Test Plan: See unit tests. Basically, create a stack and then attempt to use split, fold, or histedit somewhere in the middle.
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4209315
Tasks: 14598405
Signature: t1:4209315:1479744780:811e8bef02149a055376c0f105ac2d389dd60970
Summary:
It's common that the client's "master" ("main branch") falls behind the
server's, in which case the "is-up-to-date" check will fail and trigger a
re-download while it's unnecessary. Add some checks to avoid downloading
the cache for the client in this case.
Test Plan: Added a test
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4233534
Signature: t1:4233534:1480330716:db995d0fa5c6a2371894fc2a7871277da23608a9
Summary:
Previously, fastannotate's annotatecontext would delete individual cache
files (linelog and revmap) if it meets Exception. While the "fastannotate"
command has some retry logic, "debugbuildannotatecache" doesn't, which means
users have to run "debugbuildannotatecache" O(number-of-corrupted-files)
times to unblock. And "fastannotate" does not handle AssertionError which
could be seen as "linelog is corrupted" nicely - it crashes without retry.
Addressing all the above issues by replacing the "assert" to
"CorruptedFileError", catching them and retry properly.
Test Plan: Added a new test.
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4233433
Signature: t1:4233433:1480330548:618e80762af2209784ad72ad0658f649b050a080
Summary:
The recent bdiff behavior change affects us. Document why and update the test
so it works with both the old and new (less optimal imo) bdiff behavior.
Test Plan: Updated the test
Reviewers: #sourcecontrol, simonfar, rmcelroy
Reviewed By: simonfar, rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4231966
Signature: t1:4231966:1480011763:d116799ded3514d644e4b80296ffc71a8d712a43
Summary:
Tests failed on mac. My guess is that timeout is too low.
Let's increase it to 10 seconds (it's fine because we want wait for 10 sec
if debugbackup finishes faster).
Also let's add logging
Test Plan: Run `test-infinitepush-*`
Reviewers: #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4213330
Tasks: 12479677
Summary:
`backup` was renamed to `debugbackup` a few commits ago.
Rename it there too
Test Plan: Run `test-infinitepush-*`
Reviewers: #sourcecontrol, andrasbelo
Reviewed By: andrasbelo
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4212539
Tasks: 12479677
Signature: t1:4212539:1479721307:1ac25c7dc8951d655910e2b0c41b2c9555353f74
Summary:
`hg backup --background` will be used as a `txnclose` hook to backup all of the
local commits to infinitepush.
Test Plan: Run `test-infinitepush-*`
Reviewers: rmcelroy, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4175953
Tasks: 12479677
Signature: t1:4175953:1479145307:e698903b519361b376f6e182db7c49869c992617
Summary:
During `hg backup` bundle with many heads may be pushed. Let's support it too.
Many heads bundle is not allowed only when we are pushing it with `--to`
because in this case we don't know the node for the bookmark.
Test Plan: Run `test-infinitepush-*`
Reviewers: rmcelroy, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4175280
Tasks: 12479677
Signature: t1:4175280:1479231056:fb0c47a7752319d77a6cfc83a29c57e9e2dced16
Summary:
We were seeing some flakiness with remotefilelog prefetching in the sparse
integration test. Let's make the test more explicit by having it explicitly
prefetch exactly the files it cares about, so we shouldn't see any more
flakiness.
Also have the test refer to the in-repo sparse extension instead of the system
one, now that remotefilelog is in the same repo as sparse.
Test Plan: Ran it
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4192463
Signature: t1:4192463:1479334442:f91a99a4a026a0f71f9519155185fb484d8d68ec
Summary:
Creating a tree for a commit whose parent is not already a tree is expensive.
Let's optimize the autocreatetrees option (which converts manifests to trees
during pull) to only create trees if A) the parent is already a tree, or B) the
parent is master. This way we only pay the expensive part once. It also means
that as new branches fork off master, they will be trees too, since all commits
in the new branch will meet criteria (A).
Test Plan:
Ran hg pull in a large repo over a large pull with 40 different
branches and verified it only paused for the initial master commit.
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4129010
Signature: t1:4129010:1478260239:1698ada4e3c6a38ab77a94317e75daee4812276f
Summary:
Backups all new non-extinct [1] commits to bundlestore.
When it is called for the first time `hg backup` will backup all draft
visible commits. Next backups will save only new commits since the last
backup (it is recorded in `.hg/store/infinitepushbackuptip`[2]).
It's an initial implementation. Later the following features will be added:
1) It will be called automatically whenever user creates or strips commits
or even creates or deletes bookmarks
2) It will also save all local bookmarks and all local heads
(probably only visible).
Note: calling `pushcmd` directly does not set default values for opts. That means that `--to` will be None and `_scratchbranchmatcher` will throw exception. Let's add a check to ensure that `--to` is never None.
[1] I also want to backup extinct commits (i.e. obsolete invisible commits).
But it will require bigger changes in discovery algorithm, so I'd leave it
for later.
[2] The name is a bit verbose. But I want to keep `infinitepush` part to make
it easier to debug problems.
Test Plan: Run `test-infinitepush-*`
Reviewers: rmcelroy, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4147651
Tasks: 12479677
Signature: t1:4147651:1479229440:3eb38880c14f18e9a2fb4eaba44bedf079bca506
Summary:
This diff replaces the `hg previous` and `hg next` commands from the evolve extension with new implementations. The new commands have several features not found in evolve:
- Users can now move by multiple commits at a time. Example: `hg next 3`
- Users can move all the way to the top or bottom of the current stack with the following new flags: `hg next --top` and `hg prev --bottom`
- Users can rebase child commits on demand with `hg next --rebase`. This flag can be combined with the others to specify exactly how many commits to rebase. Example: `hg next --rebase 4` (move 4 commits up the stack, rebasing as needed) `hg next --rebase --top` (rebase all the way to the top of the stack)
- When a user lands on a particular commit, if there is a bookmark it will be automatically activated. The user can also move to the previous or next commit with a bookmark with the `--bookmark` flag.
- The user can pass the `--newest` flag to resolve ambiguity in situations where history is nonlinear. With the flag the command will always choose the commit with the higher rev number at each step.
- The commands now show commit hashes instead of rev numbers when showing the user what commit they landed on.
Apologies in advance for the large diff. I've tried splitting it up as best as I could, but the unit tests introduced here significantly increase the size. I figured the unit tests should be part of this diff instead of being added in a separate one. This diff is almost entirely new code, so I hope that helps a little bit. Let me know if there's a better way to split this up.
Test Plan: See the new unit tests for an extensive look at the new commands in action. Essentially, all of the features mentioned above should work as expected.
Reviewers: quark, #sourcecontrol, durham
Reviewed By: durham
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4121778
Tasks: 14119420
Signature: t1:4121778:1478774243:a7131593222bc329f541e77e1d3ebd8222e47e79
Summary:
Wrap `_adjustlinkrev` so it first tries the possibly faster path (linkrev
database). Then fallback to the original linkrev algorithm.
Also fixes an issue that `{k : v ...}` syntax does not work in Python 2.6.
Test Plan: Added a very simple test
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: durham, stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4143315
Signature: t1:4143315:1479234110:07e87c7584e0cfd627b3f9e2fae1a2a34beb390f
Summary: Allow for any number of spaces after the colon
Test Plan: arc unit
Reviewers: #sourcecontrol
Subscribers: markisaa
Differential Revision: https://phabricator.intern.facebook.com/D4179619
Tasks: 14532985
Summary:
Adds a `debugverifylinkrevcache` to verify the linkrevs in the database,
against the vanilla adjustlinkrev algorithm.
Test Plan: A simple test is added. The code is also manually tested on my hg repo.
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4143240
Signature: t1:4143240:1479217639:bb9b5814b405b9e60a0722b01fc2c918c5a24b35
Summary:
This diff implements some simple linkrev database classes which uses dbm to
store linkrev candidates.
Why not sqlite? sqlite's insertion seems to be much slower with index being
present, so we have to do manual buffering and do insertion in batch, not to
say parsing SQL itself can introduce overhead. If we really want sqlite for
some reason (i.e. its support for concurrent writer and reader seems to be
better), we can still implement it as a dbm engine.
Test Plan: A test will be added when there is something usable.
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: jsgf, stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4143119
Signature: t1:4143119:1479131795:5f6e368ebd1798b874cec28ac05b137eb3bea86e
Summary:
Previously, `hg rebase --restack` would find the closest stable base commit with unstable descendants on its precursors, and begin the restack operation from this commit. What this meant is that it would not resolve all instability in the current stack -- only instability above the first instance of instability detected by searching downward from the current changeset.
Given that we will soon be able to perform local fixup operations with something like `hg next --rebase --top`, it no longer makes sense to keep `hg rebase --restack` as a local operation. Instead, it should resolve all instability in the current stack. This diff changes the behavior to always begin restacking from the bottom of the stack.
One thing to note about this diff is that it completely removes divergence detection from restack. This is because in the process of making this change, I discovered a bug in the original version of restack whereby divergence would only be detected if it involved a precursor of the base changeset. This is because the check was only happening in _latest(), which after several rounds of changes to this code, was only being used in _findstablebase(). In all other cases, the implicit behavior was that unstable children are rebased onto the newest visible successor of their parent, as determined by rev number.
It turns out that altering this behavior to skip rebasing unstable descendants of divergent changesets is nontrivial, and furthermore, one could argue that the current behavior is actually desirable. (Furthermore, this change removes the O(n^2) behavior from _latest().) Regardless of whether it is desirable or not, this diff makes that implicit behavior explicit, and modifies the tests to explicitly check for the new behavior. If we do decide to skip divergent changesets, this will need to happen in another diff as it will require some restructuring of the restack target finding logic.
Test Plan:
See unit tests.
1. Create a stack of commits.
2. Pick several commits in the stack, and amend them without the `--rebase` flag, thereby leaving their respective descendants unrebased.
3. Update to the top of the stack.
4. Run `hg rebase --restack`. All instability will be resolved.
Reviewers: quark, #sourcecontrol, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4144899
Tasks: 14041544
Signature: t1:4144899:1478781779:1a94dbc1c6b69d7116f1d40593a83684ed44942a
Summary:
Previously if bundle has contained nodes that are already saved in bundlestore
then we wouldn't update bundleid value for these nodes.
That means that we can never delete a bundle because there will always be a
node that points to it.
I suggest to always update bundleid value for all nodes in the bundle.
Test Plan: Run `test-infinitepush-*`
Reviewers: durham, mitrandir, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4130662
Tasks: 12479677
Signature: t1:4130662:1478565392:682fe39c5832d5edc84c84541f0ba6c1f2e1daf8
Summary:
In order to support remotefilelog, do not construct
`mercurial.context.context` object directly. This reverts a previous attempt
to use the fastest manifest look up - `ctx[path]` actually seems good
enough.
A minor change is made in `protocol.py` to ignore failed requests.
Test Plan: Run existing tests
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4127025
Signature: t1:4127025:1478541481:c15477332582e2c9b22e2732e9e76bf37b5fba74
Summary: One of the new features I'm adding to `hg previous` and `hg next` is replacing the rev numbers with commit hashes, since these are more familiar and intuitive to users. As a first step, update this test to glob out the rev numbers.
Test Plan: Test still passes.
Reviewers: durham, quark, #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4121748
Tasks: 14119420
Signature: t1:4121748:1478162085:c7bc64f133dcfd7af67f349a10c0e0a587a8aa19
Summary:
The `debugbuildannotatecache` is to pre-populate the annotate cache
server-side, or pre-fetch the annotate cache client-side.
To avoid import cycle, `cmdtable` is moved from `__init__.py` to
`commands.py`.
Test Plan: Modified existing tests.
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4103932
Signature: t1:4103932:1478094621:e82192a12fe4b0b3ac732dbe4c127a2e93fc6b4c
Summary:
The old code didn't test if the new commit will become empty correctly, if its
"memworkingcopy" contains changes inherited from its parent. This diff fixes
the problem.
Thanks @stash for discovering the issue!
Test Plan: Added a test case for this issue. Make sure the old code cannot pass.
Reviewers: #mercurial, stash
Reviewed By: stash
Subscribers: mjpieters, stash
Differential Revision: https://phabricator.intern.facebook.com/D4113438
Signature: t1:4113438:1478077373:194104b902bdfe6827858c2916a7f3f0cd83ee6c
Summary:
This diff makes the client `fastannotate.client=1` download
linelogs and revmaps from the server on demand.
Test Plan: Added a new test
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4103929
Signature: t1:4103929:1477997846:608cf58c090806dafa21ae01feab90a6552100fc
Summary:
Previously, to get changeset hashes in the revmap, we have to construct
the `revmap` object, which reads all content of a revmap.
In the following patches, we want to quickly get the last hash of the
revmap to test if the linelog / revmap is up-to-date or not. This patch
adds a method to do so.
Test Plan: Modified existing tests.
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4103898
Signature: t1:4103898:1477988583:15869dfe9028068563b03d1f06d2337562049751
Summary:
Pulling scratch commits that were stripped before results in
abort: 00changelog.i@c153241c1ea22cbce8a229dccdf7f46de827d962: filtered node!
The reason is that upstream pull does
checkout = str(repo.changelog.rev(checkout))
and it fails if revision is filtered.
I suggest to inhibit filtered revisions that are present in the local repo.
It makes sense because if user tries to pull scratch branch he or she
probably expects to see them in the repo even though these revisions were
stripped before.
Also we can avoid doing any pull if all revisions are present in the repo.
I decided not to do this because it complicates code (for example,
we need to ensure that bookmarks are updated correctly) and it doesn't give
any significant benefits.
Test Plan: arc unit
Reviewers: durham, rmcelroy, mitrandir
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4103869
Tasks: 12479677
Summary:
During infinitepush test we noticed that `hg push -r . --to SCRATCHBRANCH`
fails with stack trace in www. This is expected because infinitepush doesn't work
with svn servers. But it's better to print a nice error message instead of stack trace.
Test Plan:
Try to push to scratch bookmark from www without specifying path:
stash@dev1918 ~/www (344c6bf) $ hg push -r . --to hack/durhamstash
abort: infinite push does not work with svn repo
(Did you forget to `hg push default`?)
[Exit: 255]
Got nice error instead of stack trace.
Push to scratch bookmark from www with path specified works fine
Reviewers: rmcelroy, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4103745
Tasks: 12479677
Signature: t1:4103745:1477950448:14b5ae8484f3e65cb88c83b2628f73f3a670b1a0
Summary:
Saw a few timeout errors during infinitepush test.
Previous timeout was 10 secs, it makes sense to increase it.
There was a unit test failures because of the config options.
So this diff also adds missing config docs and copyrights headers.
Test Plan: Build the package, install it locally and run integrationtests
Reviewers: durham, rmcelroy, mitrandir, andrasbelo
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4103689
Tasks: 12479677
Summary:
The upstream getchangegroup function changed back in August and our attempt at
fixing it here did not correclty handle the case where remotefilelog is loaded
but not enabled for this repository.
This patches fixes the signature, and makes our wrapping more resitant to future
signature changes.
Test Plan: Added a test.
Reviewers: #mercurial, jsgf
Reviewed By: jsgf
Subscribers: jsgf, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4099519
Signature: t1:4099519:1477698801:adcb406f1908c4f7e508e5f99e126f383c6f2574
Summary:
The stderr and stdout in this test has a flaky ordering issue when run in
parallel with other tests. Let's just eat the stderr since all we care about is
the stdout message from the remote and the error code.
Test Plan: Ran it
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4106366
Signature: t1:4106366:1477946237:2e188a1abde93a06ec97d99c237851955a23f20a
Summary: Adds a test for verifying that hg commit adds a tree pack to local storage.
Test Plan: Ran it
Reviewers: #mercurial, zamsden
Reviewed By: zamsden
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4082915
Signature: t1:4082915:1477590191:6637b9ab0fdad27a7d4933934168b57620dee235
Summary:
Adds a simple test for checking that trees are created during pull when
autocreatetrees is enabled.
Test Plan: Ran it
Reviewers: #mercurial, zamsden
Reviewed By: zamsden
Subscribers: zamsden, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4082871
Signature: t1:4082871:1477590276:09e594f4b87628bae8654ffdfc0bba18beb7ccad
Summary:
Since ctreemanifest is no longer in remotefilelog, let's rename the test to drop
the remotefilelog reference.
Test Plan: Ran it and check code
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4082288
Signature: t1:4082288:1477506197:a1f591b98a087ad838fef5cc0a17e8352b6bc6e3
Summary:
There are some minor differences between the two annotate algorithm.
Both are reasonable in theory. The differences are explained in the test.
The `badfn` change is needed to make exit code correct.
Test Plan: Run the modified test
Reviewers: #mercurial, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4060263
Signature: t1:4060263:1477401942:64a08d2fa28ea0fa9b54fa0f0ea48ca489c308df
Summary:
Now fastannotate have almost all the features the vanilla annotate has.
Backport the test case from upstream commit 89ed2b82ff93.
Test Plan: Will modify this test to use fastannotate in the next patch
Reviewers: #mercurial, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4060203
Signature: t1:4060203:1477402099:4a866ca7a7fbab086a1f3f86c5f4c7dc16744165
Summary:
Although `wdir()` is an undocumented feature, it exists in upstream annotate
tests. Let's support it so it is more similar to the vanilla annotate.
Test Plan: Added a new test
Reviewers: #mercurial, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4060169
Signature: t1:4060169:1477402083:021cae092e695a72fff65cd07640c9f7984b2a60
Summary: The vanilla annotate command supports `-Tjson`. Let's implement it as well.
Test Plan: Added a new test
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4059577
Signature: t1:4059577:1477074802:09b0b3ea0769d480eb3a2e42308636ff2b8d40d2
Summary:
Add an option to list remote and scratch bookmarks by pattern.
Only very simple pattern is allowed: either the literal string
or a prefix (like 'scratch/stash/*'). It was made intentionally
to make sure that pattern requests are fast in mysql.
Mysql tests will be added to the integration tests
Test Plan: Run `test-infinitepush-*`
Reviewers: mitrandir, andrasbelo, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4074409
Tasks: 12479677
Signature: t1:4074409:1477500968:e91cd2505d61a2f1db30c7f00cdcfc949e433507
Summary: Make nodes with visible successors render as 'x' in hg smartlog output. Previously these nodes were rendered as 'o' because the inhibit extension made them appear to be non-obsolete.
Test Plan:
1. Create a stack of commits.
2. Amend a commit in the middle of the stack.
3. Run `hg sl`, observe that although the precursor of the amended commit renders as an 'x'.
4. Run `hg unamend`, then update to a different commit (so the precursor is not the current commit and thus not rendered as '@').
5. Observe that the precursor now appears as 'o' again because it has no visible successors.
Reviewers: #sourcecontrol, quark, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4071058
Tasks: 13652083
Signature: t1:4071058:1477434742:f57bf5a07b6b9497472deb8445f5fc01a5379f8c
Summary:
The new tree walk did not check if the compare entry was actually a directory
before traversing it. This caused problems for commits that deleted a file and
replaced it with a directory, since it attempted to recurse down the file, which
had no treemanifest.
Test Plan: Added a test
Reviewers: simpkins, #mercurial, zamsden, rmcelroy
Reviewed By: rmcelroy
Subscribers: net-systems-diffs@, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4055678
Signature: t1:4055678:1477430721:ec679afcb4ff6ea2bbf44927e04f2bfcdee8cc03
Summary: Added a configuration option to make fbamend use the functionality from `hg rebase --restack` instead of the previous bookmark-based approach in `hg amend`.
Test Plan: Enable the `fbamend.userestack` config option and attempt to use `hg amend`, `hg amend --fixup`, and `hg amend --rebase`. These should all work just as before, except that `hg amend` will no longer create .preamend bookmarks.
Reviewers: #sourcecontrol, quark, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4062925
Tasks: 13839177
Signature: t1:4062925:1477359251:0ece10599987d6b8e39384e9530866ffb2e50e58
Summary: This test fails because Macs sometimes take 0.1 seconds, not the Linux 0.0 seconds, to do the transfer. Give them some flexibility
Test Plan: Run test on devserver and laptop
Reviewers: #sourcecontrol, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4074654
Signature: t1:4074654:1477411436:b18bde2e35025c5d9801e15ff0fd5c07261a7d40
Summary: Make the test pass on OS X with conditionals on the "only works on case-insensitive FS" block and a wider regex for output
Test Plan: Run on devserver and laptop
Reviewers: #sourcecontrol, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4074471
Signature: t1:4074471:1477410666:f0fb89060fd7c78e246e0772c4509334090c2d83
Summary: On Mac OS X, xargs only ever returns 1 for failure. On Linux, it returns 123 for "xargs succeeded, but the command(s) it invoked did not". Make the grep test pass on OS X.
Test Plan: Run the tests on devserver, see them pass. Try again on laptop, see them continue to pass
Reviewers: #sourcecontrol, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4074442
Signature: t1:4074442:1477407528:809b64f7478ff0df0d2a39b745a2b9602d5f22ee
Summary: We don't use memcache on Macs, so don't test it
Test Plan:
Run on my devserver and my laptop, confirm that it passes in both places.
Use --debug to ensure that the right commands are run on each host
Reviewers: #sourcecontrol, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4069194
Signature: t1:4069194:1477334983:bc18f8dc9487f2aca3874d22191cb8bc0a44295d
Summary:
Tests were broken because there were internal API changes in
upstream mercurial:
1) `hg book` outputs nothing instead of `no bookmarks set`
2) `getbundle` function was removed, new `getbundlechunks` function was added
Test Plan: Run `test-infinitepush-*` and run integration tests
Reviewers: durham, rmcelroy, mitrandir, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4067223
Tasks: 14035546
Signature: t1:4067223:1477305056:f705764ff3cdd70fcc61e677ccb184870077ec86
Summary: These variables were unused, causing test-check-pyflakes.t to fail whenever fbamend.py is modified in a way that changes the line numbers of those assignments.
Test Plan: Unit tests still pass (except test-fbamend.t, which fails for reasons unrelated to this change)
Reviewers: durham, #sourcecontrol, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4055599
Signature: t1:4055599:1477049676:84ec9a1036bb6d6873d497caed9feff701a8169b
Summary:
In a future diff we will be introducing packs into .hg/store, so we need to
differentiate between cache packs and local packs. This patch renames
getpackpath to getcachepackpath. A future diff will add getlocalpackpath.
This exposed a pyflakes error, so we fix that too.
Test Plan: Ran the tests
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4055815
Signature: t1:4055815:1477059415:e0221557bbeec6701820c826f00390d3a71cd2d3
Summary:
This fixes some (but not all) of the fastmanifest errors. The remaining ones are
from an import * and one that will require slightly more refactoring to fix.
Test Plan: Ran the tests
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4055633
Signature: t1:4055633:1477050763:09e758b1bd72b18799cdabf510eb0b2332077833
Summary:
This fixes all the pyflaks and module errors for the main remotefilelog
code base.
Test Plan: ./run-tests.py test-check* test-remotefilelog*
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4055537
Signature: t1:4055537:1477049663:ee904d311d17d3659e055e2c109c68c9023cfd1f
Summary:
The test generate different outputs on different platforms (devserver,
laptops), possibly related to system python path. Let's remove it until
we have a better solution.
Besides, fix an issue of the arcanist test engine: for skipped tests,
no "duration" information available.
Test Plan:
The test is not referenced explicitly anywhere.
Check "Skip test-check-module-imports-hg.t" shows up in the review page.
Reviewers: #sourcecontrol, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4058782
Signature: t1:4058782:1477067103:789deda43ddb7f439e7c98d574c8c7524e329e45
Summary: Upstream's done a few changes that we need to catch up with.
Test Plan: Run tests locally on my devserver
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4058640
Signature: t1:4058640:1477065516:e2ab2483d054e47b4fe75c6eba185dfd5179f57a
Summary:
User should see no difference between scratch and non-scratch remote bookmarks.
So let's make scratch bookmarks remote if remotenames is enabled.
Unfortunately it requires saving scratch remote bookmarks before push/pull and
restoring it afterwards because `remotenames` extension doesn't know about
them.
Test Plan: Run `test-infinitepush-*`
Reviewers: rmcelroy, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4036653
Tasks: 12479677
Signature: t1:4036653:1476985129:b5c1afdab4c7585c8b12c2962a4b95a56751091b
Summary:
This diff was created when I was investigating hgamend exec not working
with histedit which resulted with [1].
This also brings some very small fixes for sqldirstate and fbamend that
I was too lazy to diff separately.
[1] https://patchwork.mercurial-scm.org/patch/16451/
Test Plan: ran tests
Reviewers: #mercurial, ttung, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3777784
Signature: t1:3777784:1472227075:f2f8a9cad22bf0c45d4554709846b8aafb61d9a0
Summary: We are using scripts/lint.py instead of this one.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4051121
Signature: t1:4051121:1476985509:f2c362a8efe82387bbedd55b126279a1ca5fd8b4
Summary:
"arc lint" runs the lint script for each file in parallel:
>>> [5, pid=27257] <exec> $ scripts/lint.py 'scripts/lint.py'
>>> [6, pid=27257] <exec> $ scripts/lint.py 'scripts/unit.py'
>>> [7, pid=27257] <exec> $ scripts/lint.py 'tests/test-check-code-hg.t'
>>> [8, pid=27257] <exec> $ scripts/lint.py 'tests/test-check-pyflakes-hg.t'
That looks fine but `lint.py` runs `test-check*.t` which will check all
files. This is a typical `N^2` behavior and needs to be fixed.
This diff changes the linter and test code so the test only checks the
single selected file.
Test Plan:
Run `arc lint --trace` and check `lint.py` time usage. It went down from
10 seconds to 2 seconds for this diff.
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4051080
Signature: t1:4051080:1476985355:45845c6eb0e66cbfa000e61b7b496f2d00aeb042
Summary:
Previously, `arc unit` does not run `test-check-code-hg.t` and if people
have ignored lint errors, they may be ignored forever.
Since test-check-code-hg.t is relatively fast (3 seconds), let's also
run it during `arc unit`.
Test Plan: `arc unit` and confirm test-check-code-hg shows up.
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4051073
Signature: t1:4051073:1476985307:5448827437ecee9add534855a0f87ac3c3bd1f84
Summary:
This diff tries to fix `test-check-*` issues for fastannotate.
One issue cannot be fixed:
fastannotate/context.py:31: imports not lexically sorted: linelog < os
If these two imports are swapped, the checker will report:
stdlib import "os" follows local import: linelog
instead. So leave it as-is for now until we get the checker fixed upstream.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4044157
Signature: t1:4044157:1476949486:13052173bc694edd99d21affa336d252bc2aae88
Summary:
Errors like "direct symbol import ... from mercurial/hgext" do not make
sense to this repo. And there is no easy way to extend the whitelist
used by the checker to include "mercurial". So filter them out using
`grep` for now.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4044039
Signature: t1:4044039:1476949701:36ad9f611b6844c1794efe09c426ee3a47a33fec
Summary:
This is the last one of the test-check series.
`test-check-py3-compat.t` was ignored intentionally as we do not care about
Python 3 support right now.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4039479
Signature: t1:4039479:1476876594:5b95aad9be245a86dc9327770502830896273131
Summary:
Hopefully this will prevent new undocumented config options.
Issues about fastannotate are fixed. We still have many other issues, which
are ignored by this diff for now.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4037173
Signature: t1:4037173:1476876491:280e839a8e3d4f4dfbdf6b6953ab860bb7cd3ed4
Summary:
Hopefully this would rule out things like double blank lines and
inconsistent commit title.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4036978
Signature: t1:4036978:1476807264:414cd17672f59675be104c0026a8e50efac8b15d
Summary:
`timeout` is not available in OSX. Use `gtimeout` instead.
`gtimeout` can usually be installed using `brew install coreutils`.
Test Plan: Run `arc unit`.
Reviewers: simonfar, #sourcecontrol, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4029462
Tasks: 13945749
Signature: t1:4029462:1476784209:5a5d87a2b82314da1c1a0d189dd71d36e16d8c49
Summary:
If `branchpattern` is not set then it matches empty bookmark.
It causes regressions like this t13808497. Let's ignore empty `branchpattern`
Test Plan: Run `test-infinitepush-*`
Reviewers: mitrandir, rmcelroy, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4031365
Tasks: 12479677
Signature: t1:4031365:1476755541:3d3781ef92f1701500d74ce7b98b4d82b5a1228e
Summary:
Remotefilelog can't find file revisions outside of the repo.
Because of it user can't update to the revisions that was pulled from
bundlestore. This diff downloads all filelogs for revisions
from bundlestore.
There were two options to pass info about changed files to remotefilelog:
through config or through bundlecaps. I decided to use config because
it's not really a bundle capability.
Test Plan: Run `test-infinitepush-*`
Reviewers: rmcelroy, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4030065
Tasks: 12479677
Signature: t1:4030065:1476755932:4aa02e540f13d3221163d906524fb9ce97f9c7e8
Summary:
POSIX requires that the `sed` "insert" command to be followed by a backslash
and a new line character. Let's make the test more POSIX compliant.
Test Plan: Run the test on both OS X and Linux.
Reviewers: simonfar, #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4029954
Signature: t1:4029954:1476720499:0a56da77fa89229ed9606eb9ac3748264fdf12fd
Summary:
Tests doesn't require mysql since it uses fileindexapi.
Let's move sqlindexapi and fileindexapi to different files and include
only necessary file.
Test Plan: Run test-infinitepush-*
Reviewers: durham, simonfar, quark
Reviewed By: quark
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4029460
Tasks: 12479677
Signature: t1:4029460:1476709161:556f80605db9bd3c18dce1e16e996b13bbe96229
Summary:
Previously we can only answer the "path" information when the revision is in
the linelog revmap, and the code would crash if a revision is in a side
branch, and the user requests path information. This diff fixes it.
Besides, this diff improves rename handling. For example, given the following
chart:
```
o---o -o file name: a
/
o---o- file name: b
^ ^ ^
1 2 3 revisions
```
Depending on the position of the `main branch` reference, fastannotate may
or may not use linelog:
- main branch is at rev 2, annotate a -r 3 will not take advantage of linelog
(fallback to slow annotate)
- main branch is at rev 3, annotate a -r 2 will not take advantage of linelog
This is not ideal, but seems to be the best we can do for now.
Test Plan:
Added a new test, updated existing relevant tests. Some debug messages are
changed to reflect internals more precisely.
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4010964
Signature: t1:4010964:1476458201:79875d96399d023d0000d0c4bb8b8d40ea43eef0
Summary:
Fix infinitepush tests for Mac. Let's split test in two.
Now if remotenames extension is unavailable most of the tests will still run.
Test Plan: Run `python run-tests.py test-infinitepush-*`
Reviewers: simonfar, durham, rmcelroy, mitrandir, quark
Reviewed By: quark
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4029394
Tasks: 12479677
Signature: t1:4029394:1476706777:95473f493e10d3f7f21f4679976b33d75d8821f2
manifest.flags() actually returns the default value if the filename doesn't
exist. So we need to replicate that behavior.
As part of this fix, I changed treemanifest.get() to return a boolean indicating
whether the file was found or not.
This implements the dirs function, which returns a collection set that can
answer the question of if a directory is in the manifest. Currently we do a
naive solution of using util.dirs(), which iterates over all the files. Given
that we have a tree already, we should be able to return something smarter in
the future.
The diff algorithm assumed every tree already had a node. If we are iterating
over an uncommitted tree, it may have tree entries with NULL as their node. We
need to always recurse in these cases.
hg has an optional 'clean' arg on diff, which causes it to also return files
that aren't different between the two diffs. This implements it on our
treemanifest diff algorithm.
Summary:
Previously `--to` bookmark was not set locally.
This diff fixes it by setting this bookmark to the head after push happened.
This is correct because it's guaranteed that we'll push only one head.
Test Plan: Run `test-infinitepush-bundlestore.t`
Reviewers: durham, rmcelroy, mitrandir
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4001168
Tasks: 12479677
Summary: `_` does not take two arguments while `ui.status` does.
Test Plan: Added a test case
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4010079
Signature: t1:4010079:1476298648:81e678fbe433aa8cca8b03c9eba50d9b78bbc9bb
Blame Revision: D2811887
Summary:
This change adds a `--restack` option to `hg rebase`. When invoked, the command will rebase all descendants of precursors of the current changeset onto the current changeset. This is similar to the behavior of `hg evolve --all`, except it only handles unstable changesets, and not other issues that can arise from shared mutable history such as divergence or bumping.
I've been playing around with some of the more advanced features (such as allowing the command to be run from anywhere in the old stack or new stack, as well as allowing the user to specify the number of changesets to rebase), but I wanted to upload the most simple iteration of this command for feedback.
Test Plan:
See unit tests for complete commands.
1. Create a stack of commits.
2. Somewhere in the middle of the stack, amend a commit, potentially several times.
3. Run `hg rebase --restack`.
4. The top half of the stack should be rebased onto the amended commit, and the preamend bookmark should be gone.
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: rmcelroy, quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3972103
Tasks: 13651947
Signature: t1:3972103:1476230834:8f77eac4e8d8681dd9f8125747c1ff75c8da1ad8
Summary:
Non-fast-forward pushes were disallowed. But having a
mutable shared history for scratch branches is a very
nice feature. Let's use --force option to force
non-fast forward push.
Test Plan: Run test-infinitepush-bundlestore.t
Reviewers: durham, rmcelroy, mitrandir
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3994813
Tasks: 12479677
Summary:
The vanilla annotate command takes diffopts, let's add it to fastannotate.
This is also useful to support hgweb correctly.
Test Plan: Added a new test
Reviewers: #sourcecontrol, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3992672
Signature: t1:3992672:1475944204:2fbdf6a90d8c965775e3b7d5bc55fdb37c8909e7
Summary:
Upstream 4718718ed358 requires a `readfast` method for manifestctx.
`hybridmanifestctx` does not have such method so let's add it.
Test Plan: Added a test.
Reviewers: durham, ttung, #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3955478
Signature: t1:3955478:1475582074:f80c9eb97bcd42846625727ef1da4bce3256c88c
Summary:
Previously error was thrown if mysql.connector wasn't present on client hosts.
But there is no need for mysql.connector to be installed on clients.
Remove global imports to fix it and create bundlestore() only for server repo.
P. S.
Changes in test is required because bundlestore() is not created if
infinitepush.server=True is not present in config.
Test Plan:
Uninstall mysql.connector: `sudo yum remove mysql-connector-python`
Enable infinitepush extension in any repo and run any hg command (hg st,
for example). Make sure it doesn't throw exceptions.
Install mysql.connector: `sudo yum install mysql-connector-python`.
Run `test-infinitepush-bundlestore.t`
Reviewers: durham, mitrandir, quark, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3967189
Tasks: 12479677
Signature: t1:3967189:1475581802:fbaafceac298242d0a449267ec0d68e290c3f262
Summary: Seems the output has changed somehow and needs an update.
Test Plan: Run the test
Reviewers: #sourcecontrol, rmcelroy, mjpieters
Reviewed By: rmcelroy
Subscribers: mitrandir, rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3955443
Signature: t1:3955443:1475332010:d2a0e33b509d198312d71458aaf0dfd7f5ce73d7
Summary:
the nuclide team would like to be able to get the commit template
message without actually invoking a commit. This small extensions allows them
to do this.
Caveats: ignores subrepositories.
Test Plan: new test
Reviewers: #mercurial, most
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3960557
Tasks: 12771006
Summary:
Hack to fix interaction with remotenames. Remotenames push '--to' bookmark to the server but we don't want to push scratch bookmark to the server. Let's delete '--to' and '--create' and also set allow_anon to True (because if --to is not set remotenames will think that we are pushing anonymoush head).
Test Plan: Run `test-infinitepush-bundlestore.t`
Reviewers: durham, mitrandir, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3937202
Tasks: 12479677
Signature: t1:3937202:1475170335:fa4d39e1e17bea67b62b99dfc4cd2ffa4e5d6bea
Summary:
It's possible to push new scratch bookmark which points to existing scratch
commit. Previously it has failed with a stack trace.
Unfortunately it still doesn't allow to push scratch bookmark that points
to non-scratch commit. It requires more complex changes so
it will be fixed later if necessary.
Test Plan: Run `test-infinitepush-bundlestore.t`
Reviewers: rmcelroy, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3936728
Tasks: 12479677
Signature: t1:3936728:1475078059:ac4588e086b0344070a90f806b6f518c6e187c3b
Summary:
infinitepush bundle2 part should be run before pushrebase bundle2 part. This is a bit tricky since both pushrebase and infinitepush change the `extensions._order` and both of them try to load before bundle2 `changeset` part.
I suggest to load before 'changeset' or pushrebase part whichever is earlier. Then we have two situations:
1) pushrebase comes first. It sets it's part before 'changeset' part, then infinitepush sets it's part before pushrebase part.
2) infinitepush comes first. It sets part before 'changeset' and pushrebase part, then pushrebase sets it's part before 'changeset' part. Since infinitepush part comes before 'changeset' part it still be before pushrebase part.
This solution is not robust but it should be fine for now. The proper solution requires changes upstream that will introduce dependencies between bundle2 parts.
Test Plan: Run `test-infinitepush-bundlestore.t`
Reviewers: rmcelroy, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3930697
Tasks: 12479677
Signature: t1:3930697:1475079064:1b5851df4d2067ec21ec01e7510ddf83ff1684ad
Summary:
Use wrappers around ui functions to exclude interactive
and log the results to scuba.
Test Plan:
USE_DIST_HG= FB_HG_DIAGS= CHGDISABLE= hg --config 'extensions.profiling=~/facebook-hg-rpms/fb-hgext/hgext3rd/profiling.py' --pager=off --config profiling.enabled=False amend --edit
chg: disabled by CHGDISABLE
internal stats file: /tmp/scm-internal-statsx9EXSw
hg profiling mode: SKIPPED
stats: {
"int": {
"builddate": 1471281577,
"cachehitratio": 75,
"consumed": 425,
"diffcachehitratio": -1,
"elapsed": 1921,
"errorcode": 0,
"filesnotincachehitratio": -1,
"interactive_time": 1534,
"internal_time": 1701,
"time": 1473459054
},
Tested both with and without chg to make sure both modes work, otherwise
there are some subtle bugs that can come up under hg (timer becomes persistent
if invoked too early in uisetup).
Check out scuba table, new columns are showing up. I still need to clean up the table as I accidentally added data as string first.
Reviewers: #sourcecontrol, ttung, quark, durham
Reviewed By: durham
Subscribers: giorgioc, mjpieters, akushner
Differential Revision: https://phabricator.intern.facebook.com/D3844697
Tasks: 12879683
Signature: t1:3844697:1475250469:a46aff22781e0e07596e2880d6477a62754f1397
Summary:
This change updates the behavior hg next --rebase. Specifically:
- Only one changeset can be rebased at a time. If there are multiple candidate changesets, the command aborts.
- Each time a changeset is rebased, its precursor is marked as obsolete, inhibition markers are stripped from it and its ancestors, and its preamend bookmark is deleted, if one exists.
- The result of this is that if no non-obsolete changesets depend on the existence of the pre-rebased changeset, that changeset and its ancestors will be stripped, resulting in a cleaner user experience.
- This change also adds back the --evolve flag, but makes it show in error instead of working. It turns out that removing the flag outright breaks the evolve extension.
Test Plan:
See updated unit tests for the exact commands to run to test this, as well as an overview of all of the new situations where behavior was changed.
A basic test plan would be:
1. Initialize a new repository, and create a stack of 4 commits.
2. Amend the second commit in the stack.
3. Do `hg next --rebase`. It should work as before.
4. Do `hg next --rebase` again. This time, the entire old stack should "disappear" from hg sl.
Additionally, attempting to run `hg next --rebase` when there are multiple possible child changesets should fail.
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3941922
Tasks: 13570554
Signature: t1:3941922:1475205056:58a8d1726cfcccbf14a38727be0220a09532ec97
Summary:
This fixes a crash that `hybridmanifestctx` does not have `readdelta`
after upstream's b404425704fa. `readdelta` should be fast for vanilla
manifest so just forward the call there.
Test Plan: Modify a file that `.` touched, run `hg amend` and it won't crash.
Reviewers: ttung, #sourcecontrol, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3948989
Signature: t1:3948989:1475198696:d8ae194500fc093c03cc115f4613a9b8c8bfbff2
Summary:
The test is just a copy of the existing one but with perfhack enabled. It's
expected to generate the same test result.
Test Plan: Run the newly added tests
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3937109
Signature: t1:3937109:1475166970:4c39d9b15878458cd8b4a26323749224a4c3c746
Summary:
`_decorate` backported from upstream does not handle empty file correctly.
It would raise an assertion error when annotating an empty file:
File "fastannotate/commands.py", line 138, in fastannotate
not showdeleted))
File "fastannotate/context.py", line 331, in annotate
return self._refineannotateresult(result, revfctx, showpath, showlines)
File "fastannotate/context.py", line 503, in _refineannotateresult
if len(lines) != len(result):
AssertionError
This patch fixes it.
Test Plan: Run the modified test.
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3944132
Signature: t1:3944132:1475166894:e2610c6364806b77c8533315a1a0a08b6c158fe5
Summary:
We need to reserve ".l", ".m" and ".lock". So encode directory names to
avoid collision.
Test Plan: Run the modified test
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3944069
Signature: t1:3944069:1475166648:055811239514cb699a0ebe1cfab809b661c6cfd2
Summary:
This feature uses the linelog to show all lines ever existed (even deleted) in
a file. Helpful to see the history all the way back to the beginning.
Sadly it has to be inefficient currently as we have chosen to not store line
content (but only numbers) in linelog. Calculating the revisions and line
numbers is very fast because of linelog but resolving the line contents is
painfully slow. We may want a key-value database in the future, answering the
query:
(path, node, linenum) -> content
How slow is it? With the linelog pre-built, generating the output for
`mercurial/commands.py` needs resolving 400+ revisions and is about 10+
seconds.
Test Plan: Run the changed `test-fastannotate.t`
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3849503
Signature: t1:3849503:1475086235:83077c571746a7515b5ba75c4df37a1a400d9232
Summary:
This diff adds a config option to replace the default annotate command using
fastannotate.
Test Plan: Run the modified test
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3837499
Signature: t1:3837499:1475076620:65dc656c74e9c8a969f68cc4a2480f5dcbeb6361
Summary: This diff adds a simple test for the `fastannotate` command.
Test Plan: Run the newly added test.
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3836677
Signature: t1:3836677:1475076179:fa8b8ff3711a8dad9410a15f77e23adaeb676d5e
Summary:
This patch removes the unused line and space so it looks a bit better:
```
.
.
.
| o o
| | |
| | |
| o o
|/ /
| |
o o
. .
. .
(before) (after)
```
Due to the current implementation of graphlog, it's a bit hacky to achieve the
above effect.
Test Plan: Run `test-smartlog*.t`
Reviewers: #mercurial, ttung, mjpieters, durham
Reviewed By: durham
Differential Revision: https://phabricator.intern.facebook.com/D3881516
Signature: t1:3881516:1475170264:477c66b2372d04c4e5d7c8fbb69de30599706e5d
Summary:
Previously `absorb` does not care too much about copies or renames and
assumesa file path only exists in every changeset of `stack[k:len(stack)]`,
but does not exist in any of `stack[0:k]`. If this assumption is not true,
absorb will likely crash.
This patch implements copy or rename handling correctly. After this patch,
renames happens to a single file is tracked correctly. Copies are ignored
by default. For the "double move" case, like:
hg cp a a1
hg cp a a2
hg rm a
We only follow the first path (sorted alphabetically). In this case, changes
to "a1" could affect the content of "a" in a previous changeset, while changes
to "a2" couldn't.
Test Plan: Added a new test
Reviewers: durham, #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3941438
Signature: t1:3941438:1475166850:97461174619d469750146e25f6bff16dae19bd3d
Summary:
--bundle-store option makes push go to bundle store
even if bookmark is not specified or bookmark doesn't match scratch branch pattern.
The main reason for the `--bundle-store` is to be able to push not a scratch bookmark, but a scratch commit, for example, `hg push -r . --bundle-store`.
It will be useful when we will save all the local commits in the bundlestore.
To do this:
1) Wrap `discovery.checkheads()` to allow push to scratch branch even if new head is created.
2) Make scratchbranchpat params advisory because now bookmark may not be specified.
Test Plan: Run `test-infinitepush-bundlestore.t`
Reviewers: mitrandir, rmcelroy, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3862665
Tasks: 12479677
Signature: t1:3862665:1475077612:7e02a5bdc1da28161e45029b1e3a38fe8223a305
Summary:
bc720745268b added a ui argument to the mutabledatapack() and
mutablehistorypack() constructors, but did not update the tests to pass in a ui
object, causing the tests to fail.
Test Plan: Ran the tests.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: net-systems-diffs@, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3934065
Signature: t1:3934065:1475077643:4c0c32f5efdb25d482128cc58f4ba903611cc959
Summary:
The order of `find` output is not guaranteed, so sort them and make the
output determined.
Test Plan: Run the test.
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3925502
Summary:
`mysql` is required for the test. Otherwise it would error out:
```
$ hg init repo
+ abort: No module named mysql!
+ [255]
$ cd repo
```
Test Plan: Remove mysql python package and run the test, make sure it's skipped.
Reviewers: stash, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D3925440
Summary:
This diff adds the --rebase option to the `hg next` command by extending
the evolve extension in the fbamend extension.
The --rebase option finds any child commits of the obsolete precursor to the
current commit, and rebases them onto the current commit before performing
`hg next`. This way, one can be in the middle of a stack of commits, amend one
in the middle, and do `hg next` without running `hg amend --fixup`. The next
commit will be correctly found, rebased, and updated to.
This is equivalent to:
hg rebase -d . -r "children(OLD_HASH)" -k && hg next
Test Plan:
I tested this command by hand and created some unit tests. The basic way to test it is:
1. `hg init` a new repository.
2. Create a chain of 3 commits.
3. Amend the 2nd commit.
4. Run `hg next --rebase` and confirm that the 3rd commit was rebased and switched to.
The exact commands to do this are in tests/test-fbamend-nextrebase.t. This file also
contains test cases in which there are multiple child commits on the precursor, and cases
where there is a rebase conflict.
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: mjpieters, nain
Differential Revision: https://phabricator.intern.facebook.com/D3910568
Tasks: 13347995
Signature: t1:3910568:1474650102:8c7dd712afb9d36f29e1e73bb781e1d8e0870371
Summary: We need to start fixing test failures to test hgbuild e2e :)
Test Plan: run-tests.py against changed files shows that these ones were fixed
Reviewers: quark, mjpieters, rmcelroy, simonfar
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D3907217
Previously, the test would attempt to import the system ctreemanifest, which is
not what we should be testing. This fix adjusts the PYTHONPATH so that we load
the extensions from our repo, instead of the system.
Summary:
Now that the graph module can draw "..." for grand parents, it's no longer
to have fake nodes for that purpose. Remove most fake node related logic.
This will make the output more compact.
Note that the fake node is still used for "indentnonpublic" and its output
is slightly changed to display an "o" node, which will be fixed in the next
diff.
Since we now rely on a recent version of mercurial, remove some "inspect"
check for ancient mercurial support.
Test Plan: Run the modified `test-smartlog.t` and other `test-smartlog*.t`.
Reviewers: #mercurial, ttung, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3880771
Signature: t1:3880771:1474306496:3400c70efbb569c5096e724720ead1992b87f160
Summary:
One thing that may confuse new users is that when draft changesets are on top
of public changesets:
```
o draft
|
o remote/master
```
It's unclear whether the draft changeset is already in the master branch, or not.
This diff adds an option to move the drafts to the second column to indicate they
are not part of master.
Test Plan: Run `test-smartlog.t`
Reviewers: ttung, durham, #mercurial, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3876951
Signature: t1:3876951:1474043568:c97b26b49917f36a5197af93ea386658b91d56a4
Summary:
"changeset" is a more official term and let's use it. Note that this patch
only changes documentation / i18n messages visible to the users and header
comment blocks to developers. Other places like comments in the code are
untouched.
With the "dialect" extension enabled, users will still see the more friendly
term - "commit".
Test Plan:
`arc unit`. Note the remotefilelog failure is probably unrelated - seems
related to ongoing / upcoming manifest refactoring upstream.
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3900394
Signature: t1:3900394:1474470348:6a1b5691e2599cc47df18b227d56d1f9d3c7c906
Summary:
As discussed before, we think terms like "changeset" is more official to hg,
while less known as "commit" is used more widely.
This extension replaces "changeset" with "commit" so users will get a
consistent view about the term being used, and we can use "changeset" in our
code to be more official while don't need to worry about user friendliness.
Test Plan: Run the newly added test
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3900240
Signature: t1:3900240:1474470482:44ac6e09d58e624c539bb1336842ce8911c9dcee
Summary:
We have seen issues that users have outdated groups when running hg commands,
which will probably always cause issues:
- Authentication issue, unable to ssh
- Filesystem permission issue, unable to write hgcache
- Even worse with chg server since the long-running server process will keep
the wrong groups information
This extension is to address the above issues. It allows us to print a message
to let the user know they have group issues. Besides, it allows us to override
configs like `chgserver.idletimeout` so chg servers with wrong groups can have
a much smaller TTL and won't be long-running and causing issues.
Test Plan: Run the newly added test
Reviewers: #sourcecontrol, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3896628
Signature: t1:3896628:1474454162:22785ff23e3ada75013ce5f1eead3407068ba172
Summary:
This adds a test for checking if the write function correctly writes to the
store, and can be read back exactly.
Test Plan: Ran it
Reviewers: #fastmanifest
Differential Revision: https://phabricator.intern.facebook.com/D3890469
Summary:
When diffing a with b, if a is empty, we don't insert a "dummyendline" (see
line 214) because we do not have the last line to copy. In this case, if an
insertion happens, we have no "nearbylinenums" (see line 274) to visit.
The current code assumes "nearbylinenums" have at least one element, which
is wrong in the above case and will crash the program.
The fix is to add a check about whether "annotated" (aka. "a") is empty or
not.
Test Plan: Run `test-absorb-filefixupstate.py`
Reviewers: #sourcecontrol, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3867686
Tasks: 13371265
Signature: t1:3867686:1474289020:a183e6837e6e0afb8a5d56641689e240e8cd78c9
Summary:
This feature allows the user to choose what lines exist in what changesets
precisely, for example:
HG: editing a
HG: "y" means the line to the right exists in the changeset to the top
HG:
HG: /---- 991104c93ef1 commit 1
HG: |/--- aa7e21df9b0b commit 2
HG: ||/-- 05b39f5d9141 commit 3
HG: |||
yyy : 1
yy : 2
y : 3
The feature can be used to "move" lines among changesets, or generally
editing the contents in a new way. So it's not limited in the "absorb"
use-case. We may want to give it another name in the future.
Test Plan: Added a new test case
Reviewers: #mercurial, rmcelroy, ttung
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3841494
Signature: t1:3841494:1473769708:1bb2a46c010222438ceaae69ea0dca39e37581d2
Summary: The test covers all features implemented prior to this diff.
Test Plan: Run the newly added test
Reviewers: #sourcecontrol, zamsden
Reviewed By: zamsden
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3849347
Signature: t1:3849347:1473722759:6d5e62733a841ef7b55bfc2254187bdee0eaf005
Summary:
This is required to get the proper sorting with characters with the MSb set.
Wrote unit test to cover it.
Test Plan:
1. passed that unit test.
2. `PYTHONPATH=~/work/mercurial/facebook-hg-rpms/fb-hgext/:~/work/mercurial/facebook-hg-rpms/remotenames/:~/work/mercurial/facebook-hg-rpms/lz4revlog/ /opt/local/bin/python2.7 ~/work/mercurial/facebook-hg-rpms/hg-crew/hg --config extensions.treemanifest_correctness=~/work/mercurial/facebook-hg-rpms/fb-hgext/tests/treemanifest_correctness.py --config remotefilelog.fastdatapack=True testtree --build "master~50000::master" --revs 'master + master~5000'` is clean now!
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3869292
Signature: t1:3869292:1473922557:4c691f696ea991a578a151b8091ae44beff528df
Summary: This is a mod of perftest.py, where we remove the unneeded components and distill it into a simple correctness test. This is exposing bugs in our code which I will fix in subsequent diffs.
Test Plan: well, it runs. it doesn't produce clean output yet though. :)
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: durham, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3866674
Signature: t1:3866674:1474225197:d147b18548e9e62c3b54e1ab1506141f3d52a07f
Summary:
Mercurial core does this sometimes where it sets the new entries before it removes the old entries. In that case, you might have:
old manifest:
```
abc
```
operations:
```
+ abc/def
- abc
```
If you add `abc/def` to a manifest already containing `abc`, it will throw a `TypeError` today. With this change, we accept this weird state.
Test Plan: Update the tests to cover both versions of this scenario (directory was there first vs file was there first). Pass all tests.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: durham, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3865617
Signature: t1:3865617:1473897481:4cf4f122dfd3a3759fe84ea167b8cb0e78238bc2
Summary:
fileiter/stackframe gain a `sorted` field, which it uses to determine whether to use a ManifestIterator or a SortedManifestIterator.
add a unit test to test this scenario
Test Plan: pass unit tests.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: durham, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3868088
Signature: t1:3868088:1474225427:64d19f3f58376682850e46a3547e7a6ce1de97f4
Summary: I wanted scopes, but python's contextmanager doesn't really provide a real scope. Derp.
Test Plan: run the test.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3862225
Signature: t1:3862225:1473895847:c2e437b6cb8ea2de9a94c6243de1cf731906d995
Summary:
When we remove a file, we need to remove all the empty directories that lead to that file. This test ensures that we are accomplishing this.
Note that when we put in the fix for '/' "feature", this test may no longer work.
Test Plan: pass unit tests.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: durham, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3855366
Signature: t1:3855366:1473807483:66bd4650ee014e7da3fa62ef529af0467462aeb9
Summary: This way, we can have a true nullable (null, a.k.a., not present, and any character).
Test Plan: make local and pass tests
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3855340
Signature: t1:3855340:1473807404:54c1ae17c735665935a52b0140b58094f0b45a26
Summary:
There is no point in disallowing it.
The next step is to allow to fetch arbitrary many scratch bookmarks with one command.
Test Plan: Run `test-infinitepush-bundlestore.t`
Reviewers: durham, rmcelroy, mitrandir
Differential Revision: https://phabricator.intern.facebook.com/D3835502
Tasks: 12479677
Summary:
scratchbookmarks are not automatically pulled (because they are not showed in
listkeys). Let's record them after the actual pull. It introduces a race
condition. But this race condition is unlikely and shouldn't be harmful.
Test Plan: Run `test-infinitepush-bundlestore.t`
Reviewers: mjpieters, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3835414
Tasks: 12479677
Signature: t1:3835414:1473703998:554c890be50f732f34b2793160ee1abb26561e39
Summary: I could have sworn I ran this before I committed, but clearly not. Sadface.
Test Plan: actually ran them.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3853260
Signature: t1:3853260:1473723548:f360c6c4559641deb157e570df08a51a7edf8e21
Summary:
Removes are treated as set to None.
The API is not exactly correct, as manifestdict needs to implement size/getitem/setitem, but we can refactor in a subsequent diff.
Test Plan: pass unit test
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3836197
Signature: t1:3836197:1473363870:21a447e4bc17148193fde7ed4887927089e54930
Summary:
Python binding is pretty straightforward. The test exercises a few simple paths: set, update, and conflict detection.
Not yet done: tests to set a deeply nested directory structure.
Test Plan: PASSED THE SIMPLE TESTS. YES!
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: durham, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3832682
Signature: t1:3832682:1473363415:60240c6bbbcec82184b4588b83bbda750e8b77f1
Summary:
The root node is now optional. If it's not specified, an empty tree is constructed.
Added an unit test to cover this code path.
Test Plan: `PYTHONPATH=~/work/mercurial/facebook-hg-rpms/fb-hgext/:~/work/mercurial/facebook-hg-rpms/remotenames/:~/work/mercurial/facebook-hg-rpms/lz4revlog/ /opt/local/bin/python2.7 ~/work/mercurial/facebook-hg-rpms/hg-crew/hg --config extensions.perftest=~/work/mercurial/facebook-hg-rpms/fb-hgext/tests/perftest.py --config remotefilelog.fastdatapack=True testtree --kind flat,ctree,fast --test fulliter,diff,find --build "master~5::master"` still runs.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3803569
Signature: t1:3803569:1472759620:02c1f723fe3b7f4cdca9da6399fc2296c6d5a022
getfile should return the file mode as (islink, isexec),
but it was returned as (islink, isbinary).
As a result, using 'hg absorb' removed executable mode from
the relevant changeset so the mode change showed up in the working directory.
Issue reported with test case by Thomas De Schampheleire.
Summary: Most of this is pretty self-explanatory. We're creating two different test cases that invokes slightly different APIs to read the datapack (conventional datapack vs fastdatapack). The quirk is that fastdatapack does not expose the fanout prefix, so we suppress the test involving that for fastdatapack.
Test Plan: `PYTHONPATH=~/work/mercurial/facebook-hg-rpms/fb-hgext/:~/work/mercurial/facebook-hg-rpms/lz4revlog/ /opt/local/bin/python2.7 ~/work/mercurial/facebook-hg-rpms/hg-crew/tests/run-tests.py test-remotefilelog-datapack.py`
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3792849
Tasks: 12932864
Signature: t1:3792849:1472597547:35503ff5ef0c4910f3ae7f49b1268d3305a0b6c9
Summary: Doesn't seem to make sense to keep two of them.
Test Plan: meh.
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3789029
Signature: t1:3789029:1472521395:db958a5f050319dfa7c608cc448e2d38acbb8c0c
Summary: This removes the hard coded revs and paths used in the perf test script.
Test Plan: Ran the perf test
Reviewers: #fastmanifest, ttung
Reviewed By: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3755335
Signature: t1:3755335:1471930070:e90a242a8f73d2fc32ea68bc99cae88b00af14c9
Summary:
buffer.h gained the ability to deal with non-char-sized buffers when I built cdatapack. We need to update the callers in ctreemanifest to be aware of this. Most of this is done with macro magic.
Some functionality was dropped from cdatapack's buffer.h (macro definitions to deal with paths). Those are moved to path_buffer.h
Test Plan:
make local && clion build.
pass cfastmanifest unit tests.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3780767
Signature: t1:3780767:1472255278:40a19edfd171df5804e9cdfa4444d5c6386f00e8
Summary:
I've seen the failures like https://phabricator.intern.facebook.com/P56599278
which were the result of "enforce_root_files" being set in our global watchman
config.
It appears that the global configs are being picked up only on server start.
Killing server during the test won't help because watchman is automagically
restarted in our dev environment.
The way to do it properly is to mimic
https://github.com/facebook/watchman/blob/master/tests/integration/WatchmanInstance.py
until we do that I've just put a .watchmanconfig to fix the test.
Test Plan: tests is passing now
Reviewers: #mercurial, ttung, durham, wez, zamsden
Reviewed By: wez, zamsden
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3772968
Signature: t1:3772968:1472158987:0e08c5e9f862ba3d74d016d051b852512d06e399
Summary: I ran tests with --noskip and found one more failure
Test Plan: test passing now
Reviewers: #mercurial, ttung
Reviewed By: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3772978
Signature: t1:3772978:1472155449:1c5d8e16bd51d2b06f15432dd47c6fba48da3abd
Summary:
This makes it possible to run all remotefilelog tests without others
It also avoids some issues with name collisions in the upcoming merge.
Test Plan: next commit is a merge and no conflicts in tests/
Reviewers: #sourcecontrol, ttung, durham, mitrandir, simonfar
Reviewed By: mitrandir, simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3764379
Tasks: 12855049
Signature: t1:3764379:1472217061:67a0cc8f1fc29f991be08fe965679535ff6df27a
Summary:
764cd9916c94 recently introduced code that was unconditionally checking the
repo.includepattern and repo.excludepattern attributes on a local repository
without first checking if this is a shallow repository. These attributes only
exist on shallow repositories, causing "hg pull" to crash on non-shallow
repositories. This crash wouldn't happen in simple circumstances, since the
remotefilelog extension only gets fully set up once a shallow repository object
has been created, however when using chg you can end up with scenarios where a
non-shallow repository is used in the same hg process after a shallow one.
This refactors the code to now store the local repository object on the remote
peer rather than trying to store the individual shallow, includepattern, and
excludepattern attributes.
Overall this code does still feel a bit janky to me -- the rest of the peer API
is independent of the local repository, but the _callstream() wrapper cares
about the local repository being referenced. It seems like we should ideally
redesign the APIs so that _callstream() receives the local repository data as
an argument (or we should make the peer <--> local repository assocation more
formal and explicit if think it's better to force an association here).
Test Plan: Added a new test which triggered the crash, but passes with these changes.
Reviewers: ttung, mitrandir, durham
Reviewed By: durham
Subscribers: net-systems-diffs@, yogeshwer
Differential Revision: https://phabricator.intern.facebook.com/D3756493
Tasks: 12823586
Signature: t1:3756493:1471971600:9666e9c31bf59070c3ace0821d47d322671eb5b1
Summary:
We need to fix tests due to de-facto HG_URL variable semantics change in
https://selenic.com/hg/rev/a5fd89db5549
Test Plan: run-tests.py test-pushrebase.t
Reviewers: #mercurial, ttung, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3771208
Signature: t1:3771208:1472141889:562c84ca92b91d2dbe60decea53349f53a9676c4
Summary:
The built linelog.so cannot be discovered automatically. Change `PYTHONPATH`
to make sure it can be found.
Also fix the issue that the rename change misses some places in the test.
Test Plan: `unset PYTHONPATH` and run the touched files.
Reviewers: #sourcecontrol, ttung, mitrandir
Reviewed By: mitrandir
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3771583
Signature: t1:3771583:1472145156:d05a1bf0d5920dfd0131acd1998698f2dddae1f8
Blame Revision: D3763992
Summary:
This assert is meant to check that at least one source was provided to the
hybridmanifest. The old version was broken though.
The test requires an update because it attempts to construct a hybridmanifest
with no source.
Test Plan: Ran the tests
Reviewers: #fastmanifest, ttung
Reviewed By: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3760229
Signature: t1:3760229:1472062113:695085444915cb15a93db7f317580dcffc2f115d
Summary:
This seems to be an elegant choice. It does not conflict with other names,
and it does explain the concept to some extent vividly.
Credit goes to [trousers at lobste.rs](https://lobste.rs/s/nws1uj/help_us_name_new_mercurial_feature/comments/u4tsw7#c_u4tsw7).
Test Plan: Run the existing tests.
Reviewers: #sourcecontrol, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3763992
Signature: t1:3763992:1472049269:5bb9de3b7b3a0f0dd9587b952151fc38351bc822
Summary:
Some people think this command should be a part of "amend". If you check the
meaning of the "amend" English word, that makes sense. But if you look at
the actual "amend" command, there are significant differences:
1. smartfixup rewrites a stack of changesets, and can even delete
changesets, where amend only rewrites the working directory parent.
2. smartfixup is best-effort, i.e. does not guarantee that 100% of
user-requested modifications will be included, where amend will just take
100% (with "-i", it's 100% chunks selected by the user).
3. a lot of "amend" flags do not make much sense to smartfixup, like message
editing (designed to edit a single changeset), "--addremove", "--secret",
etc.
4. literally, smartfixup shares little code with the existing "amend" logic.
"amend" is part of "fbamend" or "evolve". this extension should not
depend on any of them.
So it's cleaner to be a separate command, not a part of `amend`.
However, I think it makes sense to have an option to satisfy those who want
to use "amend". So they can use "amend --related", "amend --fixups",
"amend --stack", "amend --auto" or whatever they choose. This diff adds such
a config option. We may also ship such a config option to make the command
easier for discovery.
Note the "amend" version is slightly different from the original smartfixup
command. The former targets basic users who expect amend to take all of
their changes, while the latter targets power users understanding what's
going on.
Therefore, the "amend" version will print extra information about what
changes are ignored, for example:
```
# changes not applied and left in working directory:
# M a : 1 modified chunks were ignored
# M c : unsupported file type (ex. binary or link)
# R b : removed files were ignored
```
To support the above change, `fixupstate.status` was added to avoid a second
"status" run and handles the "status" with interactive mode correctly. An
issue about symbolic links being added to `fixupstate.paths` was fixed by
the way.
Test Plan: Run the newly changed test.
Reviewers: #sourcecontrol, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3760498
Signature: t1:3760498:1472052376:7ddbfe763c7327d044b0d471c31a58fcb1e21dac
Summary:
The `--print-changes` option will show which commit, a changed lines will be
applied to. Note it is a bit expensive so we only calculate the content when
being asked.
The `--dry-run` option will prevent the final commit operation.
Users can now use `-pn` to preview changes to understand what will be changed.
Test Plan: Run `test-smartfixup.t`
Reviewers: #mercurial, ttung, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3719673
Signature: t1:3719673:1471582117:7f3be3c3bf3829148c3d096d2aae6e7382449b98
Summary:
This is the last piece of the initial smartfixup implementation: the command.
A new test was added to test the command.
Test Plan: Run the newly added test
Reviewers: #mercurial, ttung, jbower, durham
Reviewed By: durham
Subscribers: akushner, simonfar, durham, jbower, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3717354
Signature: t1:3717354:1471629588:202e4a5820de472eb131eb968ec73f194d2c077d
Summary:
The `filefixupstate` object maintains states and contains logic related to
do "smartfixup" to a single file.
Test Plan: Added a new test
Reviewers: #mercurial, ttung, durham
Reviewed By: durham
Subscribers: durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3716761
Signature: t1:3716761:1471919159:033e79e9d12307785863959dd5d82a7c3c647ff3
Summary:
I like many small files.
There is one place where I'm making a functional change (convert.h) to satisfy angry compilers.
Test Plan: make local.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir
Differential Revision: https://phabricator.intern.facebook.com/D3732584
Signature: t1:3732584:1471542758:d0b7804753ea4fd39a507090338ae3c5104dc7fa
Summary:
More extensive test to make sure existing behavior isn't broken.
What is broken currently - globs.
Test Plan: ../../hg/run-tests.py test-fastlog.t
Reviewers: durham, quark, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3743677
Tasks: 12341014
Signature: t1:3743677:1471852994:7684524f1879e6ac16d473cabd53e95adf73d888
Summary:
This diff together with patches introducing the mentioned callback in upstream
will fix the journal under sqldirstate.
Test Plan: journal tests passed, the rest is currently running - will update the test plan soon
Reviewers: #mercurial, ttung, durham
Reviewed By: durham
Subscribers: durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3690878
Signature: t1:3690878:1470800077:3fe48e03bb65eaeed3a7326b6e7d392abe7efd8d
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
Summary:
The test will call "replacelines" with random arguments, keeps track of the
content of "lines" on its own. Then it compares its own "lines" with linelog's
annotateresult - should be the same. After that, it verifies the content of
old revisions can be retrieved by using "annotate" correctly.
Test Plan: Run this test
Reviewers: #mercurial, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3709431
Signature: t1:3709431:1471012434:68ca06c0b3b2705740375c32acac8012ded404a5
Summary:
We are seeing perf issues with hidden/obsolete handling.
`hg bookmark` is a frequently used command and by making it use the unfiltered
repo, it could be 200ms-300ms faster.
Test Plan: Added a new test
Reviewers: #mercurial, mitrandir, ikostia, ttung
Reviewed By: mitrandir, ikostia
Subscribers: mitrandir, rmcelroy, akushner, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3692968
Signature: t1:3692968:1470777864:72ad5d0ffb52ecfcaaa607082693b88319d778fd
Summary:
This happens when pressing Ctrl+C during "hg rebase". We should show
"interrupted rebase" instead of "interrupted update".
Test Plan: Added a new test
Reviewers: ttung, #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3697984
Signature: t1:3697984:1470863868:acd67eeb5099c73ea7df174b3e1dbcbf9ac54944
Summary:
fastdatapack is the same as datapack. add selector in datapackstore to determine which datapack to create.
test-datapack-fast.t is the same as tset-datapack.t, except it enables fastdatapack
Test Plan: pass test-datapack.t test-datapack-fast.t
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir
Differential Revision: https://phabricator.intern.facebook.com/D3666932
Signature: t1:3666932:1470426499:45292064e2868caab152d9a5b788840c5f63e4e4
Summary: Now that we report uncompressed lengths, the test output needs to be updated.
Test Plan: pass `PYTHONPATH=~/work/mercurial/facebook-hg-rpms/fb-hgext/:~/work/mercurial/facebook-hg-rpms/lz4revlog/:~/work/mercurial/facebook-hg-rpms/remotefilelog/ python ~/work/mercurial/facebook-hg-rpms/hg-crew/tests/run-tests.py -j32 test-repack.t`
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir
Differential Revision: https://phabricator.intern.facebook.com/D3656796
Signature: t1:3656796:1470175464:56bf12516710cef8e8aaa7e7b3e0dbdfa220d797
Summary:
1) Add sync status
2) Combine sync status and phab status to use one unified conduit call,
i.e. differential.querydiffhashes
Test Plan:
cd hg-crew
make local
cd ../fb-hgext
make local
cd tests
../../hg-crew/tests/run-tests.py test-phabstatus.t
../../hg-crew/tests/run-tests.py test-syncstatus.t
Reviewers: wqfish, lcharignon, #sourcecontrol, wez, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3651915
Tasks: 10100400
Signature: t1:3651915:1470340328:bf003006f6afe9b86a40f204e150e0d12350c21d
Summary:
This is mainly to address the same issue as D3609747: prevent issues caused
by running hg as root. The difference is this extension reads owner and uid
dynamically without config when initializing of a localrepo object.
As a side effect it covers more situations like running hg in others' repos, while
being less restrictive for some other commands like `sudo hg version`.
Test Plan: Added a new test
Reviewers: #mercurial, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: ikostia, rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3619997
Tasks: 11723374
Signature: t1:3619997:1470176448:beaf53c09fca498206767641ffa4315a744ee07e
Test Plan: run tree_dump on a fastmanifest file. did some spot checks against the text manifest with the same sha (`hg debugdata -m`)
Reviewers: lcharignon, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3585843
Tasks: 12119301
Signature: t1:3585843:1469050470:70729d5601f935f2de80f89a7ad83590bcb8dd00
Summary:
Because we now prune in parallel to priming the cache, we no longer need the prune at the end. The only scenario where we still need to prune is where we never enter the priming loop, i.e., when the revset is empty.
Depends on D3545267, D3544997
Test Plan: since we don't prune in many circumstances any more, the test output is slighty affected. otherwise, the tests pass.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: durham, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3547219
Signature: t1:3547219:1468373775:e4535e3a66fb362385a23b992749eb7c3fcf7ea1
Summary:
During chg pull or push over ssh, ssh is started by chgserver which does not
have a controlling tty. Therefore the ssh process won't be able to ask for
passwords interactively.
This is actually a hard issue because an unprivileged process without a ctty
cannot attach to a ctty of another process.
The discussion at upstream tends to make it clear it's part of limitations
of chg. Therefore if we decide to workaround it, it has to live outside core,
thus fb-hgext.
GUI ssh-askpass is actually a good and clean choice. See D3510178 and D3515604.
However, they are for OS X but not Linux.
This diff is a very hacky solution to make ssh-askpass works in terminal.
It starts a "tty server" providing tty I/O fds and set `SSH_ASKPASS` to use a
custom script talking to the "tty server".
Test Plan:
Run the new test. Start a sshd locally and try:
```
$ hg push ssh://root@localhost/tmp
pushing to ssh://root@localhost/tmp
root@localhost's password:
remote: Permission denied (publickey,password).
abort: no suitable response from remote hg!
$ chg push ssh://root@localhost/tmp
pushing to ssh://root@localhost/tmp
==== SSH Authenticating ====
root@localhost's password:
remote: Permission denied (publickey,password).
abort: no suitable response from remote hg!
```
Reviewers: #mercurial, ttung, mpm
Reviewed By: mpm
Subscribers: durham, mpm, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3577509
Tasks: 12029680
Signature: t1:3577509:1469467700:cd93565bd47e535bb4cb41fcdaa39e45dddfae28
Summary:
All of hg journal is now fully upstreamed to mercurial core and remotenames.
Remove the outdated copy here.
Test Plan: --
Reviewers: #mercurial, ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3611123
Tasks: 10804988
Summary:
The old scm-prompt uses this longer, less awesome name.
Let's be backwards compatible with this and all that jazz.
Test Plan: tests pass under bash and zsh
Reviewers: #sourcecontrol, ttung, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3597315
Signature: t1:3597315:1469153052:98600400bb3cabf561294fd62f7cc601c280a1bd
Summary:
Previously, aliases could screw up scm-prompt. We had seen earlier
issues with this (thus the --color=never), but this is a more generic and
robust solution, applied everywhere.
We can't use full paths because they differ on different hosts, so we still
rely on a reasonable $PATH, as we always have.
Test Plan: ran scm-prompt tests under bash and zsh
Reviewers: #sourcecontrol, ttung, zamsden
Reviewed By: zamsden
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3597300
Tasks: 12298139
Signature: t1:3597300:1469060969:e88be6bd79234167f039f2964117a738ffbb95f5
Summary:
The `.py` tests are different from `.t` ones. They need special care about
`import`.
Also adds unlink `socketpath` to make sure the `.py` file runs directly
with python with additional effort to clean up the test dir.
Test Plan:
```
unset PYTHONPATH
run-tests.py test-patchpython.py
python2 test-patchpython.py
```
Reviewers: ttung, #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3579579
Signature: t1:3579579:1468848206:3711a4714080fbcc2d4360ded8316603be48fa25
Summary:
The tests that are modifying the hgrc are blacklisted for
sqldirstate because they are effectively switching exitension off.
Test Plan: tests are passing
Reviewers: #mercurial, ttung, durham, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3559268
Signature: t1:3559268:1468453765:38140f2ab4c392dc2aeefec230cbd1d4bb1b7170
Summary:
It's a common mistake that our tests require foreign extensions (namely evolve
and remotenames) without checking them first.
This diff adds checks to catch these mistakes, adds missing checks, and unifies
our checking logic using `require-ext.sh`, which is aware of `hgext3rd` and
prints skip message.
This affects `arc lint` so hopefully our new testing code would be free of this
kind of mistakes.
Test Plan: `arc lint` would catch errors
Reviewers: #mercurial, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3550977
Signature: t1:3550977:1468455857:e849dfd9e3cbc446cc6e6c662050ee88a3366e6c
Summary: `prune` is essentially `makeroomfor(0, set())`.
Test Plan: pass existing unit tests. output is slightly different, but that's just because we no longer output that debugging line.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3547069
Signature: t1:3547069:1468352305:6185132648f871e26d3cfd449c059523b5eb6d6b
Summary: It's caught by the contbuild script. I forgot to change this test.
Test Plan: Run `test-patchpython.t`.
Reviewers: #mercurial, ttung, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3550459
Signature: t1:3550459:1468349937:07123651fb103d283a382d1323d9f69a5c5d81b7
Blame Revision: D3534311
Summary:
Be a better citizen under system python path.
Fix all tests issues and change setup.py to use glob pattern to include
all extensions.
Test Plan:
Run tests and `make local`.
Also build and install the package and run `hg sl` in major repos.
Reviewers: #mercurial, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3534311
Signature: t1:3534311:1468275426:fe122646c8bd6c541e1889e73e9df28f86747ff2
Summary: Currently, this test breaks! This is awful because that means for users, when the cache becomes full, we never accept new manifests. :( :( :(
Test Plan: it breaks. :( did i say :( ?
Reviewers: lcharignon, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3545060
Tasks: 12136039
Signature: t1:3545060:1468279919:ff8928557c8ad03e06370ee75f2386ffb46f54fa
Summary:
Previously, depending on the code path, the limit specified would not actually take effect. For instance, if we came in from debugmanifestcache, and attempted to populated the cache, we would use `systemawarecachelimit` when filling the cache, and the fixedsize limit specified by the user when pruning.
With this change, we unify the all the cache limit decisions to `fastmanifestcache`. If the user actually overrides the limit, we set the limit in `fastmanifestcache` and let that make the decisions.
We also change the definitions of limit in `hg debugcachemanifest` to:
1) >0 => it's the limit.
2) =0 => use systemawarecachelimit
3) <0 => no limit!
Test Plan: pass existing unit tests. there's a small change in the test output, because we always evaluate the limit now, plus we remove the test for limit=0, since it means something different now.
Reviewers: lcharignon, durham
Reviewed By: durham
Subscribers: trunkagent, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3544997
Signature: t1:3544997:1468281604:8f78f00ebf2afd8f3f1fbefbd82316b97cc4b193
Summary:
We need a fastmanifest object in order to size it. Once we know its size, we can make room in the cache.
This slightly affects one of the tests, as we request the manifest text earlier than we previously did.
Depends on D3537904
Test Plan: used in later diff.
Reviewers: lcharignon, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters, quark
Differential Revision: https://phabricator.intern.facebook.com/D3538991
Signature: t1:3538991:1468280648:41c65d91529babe0559eac7b75509481adf2765f
Summary: We don't need this since we loosely lock the cache worker.
Test Plan: pass existing tests.
Reviewers: lcharignon, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters, quark
Differential Revision: https://phabricator.intern.facebook.com/D3537355
Signature: t1:3537355:1468268248:ebf7de5bf5030b90039823e1111a073d7eacc447
Summary:
The shared cache needs to be completely g+ws so that all members of the group
can write to each directory in it. The old code only applied g+ws to the leaf
directories, so other users aren't able to write to non leaf directories (like
hgcache/7a/83beca8.../ others couldn't write to 7a/)
Test Plan:
Updated a test to view group permissions for the intermediate
directories
Reviewers: #mercurial, ttung, simpkins
Reviewed By: simpkins
Subscribers: lcharignon, net-systems-diffs@, simpkins, mbolin
Differential Revision: https://phabricator.intern.facebook.com/D3523918
Signature: t1:3523918:1467930221:452b11b56a2e69896bf8d2cd0acd7131b41f90d8
Summary:
Previously, the history repack logic would stop traversing history for a given
filename once it encountered a rename. This isn't quite right, since the history
could eventually be traversed back to the original file, where we'd need to
continue processing. So now we check for when the copyfrom becomes the filename.
Also, if the copy source file and the copy target file have two nodes with the
same value, we would not process the one in the copy target (since it was marked
do not process). We fix this by explicitly checking if the node is one of the
known entries in the file being processed.
Test Plan: Added a test
Reviewers: #mercurial, ttung, mitrandir, rmcelroy
Reviewed By: mitrandir, rmcelroy
Differential Revision: https://phabricator.intern.facebook.com/D3523215
Signature: t1:3523215:1467828169:bd487c8f296352c1a1b9355cb55f9001bd5e19a9
Summary:
Before this patch we were doing two mistakes in the ratio computation.
- We were not recording global cache hit or miss, it was always a hit because
a function is truthy is python
- We were not deduping cache miss for the same manifest multiple times
This patch fixes these two mistakes.
It also changes the logging of cache hit and miss to include the name of the
operation that triggered the cache hit or miss (diff or filesnotin).
Test Plan: Test output changed and commented
Reviewers: durham, ttung
Differential Revision: https://phabricator.intern.facebook.com/D3484658
Summary:
Before this patch we were using config instead of configbool for reading
the debugmetrics config causing "False" to be evaluated as a truthy value
for the config. This patch fixes the issue and sets the config to false for
some of the tests to reduce the noise of the output.
Test Plan: Tests pass
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3524501
Summary:
There are two parts to add logging to blackbox. Logging events
and changing the configuration of blackbox. This diff does the former.
We don't reuse all the metrics event as:
- blackbox does not support wildcard for events like fastmanifest-*
- they are not meant to be human readable, they are for performance logging
We instead log a few new events:
- What triggered a caching operation?
- What command we run to cache the manifest?
- What revisions are to be cached?
- What revisions are actuallt cached?
- Do we overflow the cache?
This will allow us to understand all the cases where caching didn't trigger.
Like in t11877434.
Test Plan:
Test output does not change
Verified manually by changing the blackbox configuration that events get logged:
CHGDISABLE=1 hg book food --config blackbox.track='command, commandfinish, commandexception, exthook, fsmonitor, pythonhook, fastmanifest' --config extensions.fastmanifest=/home/lcharignon/facebook-hg-rpms/fb-hgext/fastmanifest --config extensions.blackbox=/home/lcharignon/facebook-hg-rpms/hg-crew/hgext/blackbox.py
Reviewers: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3504158
Tasks: 11877434
Summary: D3505464 modified the kwargs, but the test output was not updated.
Test Plan: passed tests.
Reviewers: lcharignon, #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3506179
Signature: t1:3506179:1467353060:7c7cf9fd51257f5f5aa037a779618cf6512c1dd0
Summary:
reflog is dead, long live journal! journal is at feature parity and reflog can
go. I've marked this as a move to help find what happened in history.
Test Plan:
Run all the tests to see if anything broke:
$HGSRC/tests/run-tests.py -l
Reviewers: #sourcecontrol, ttung
Reviewed By: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3498613
Tasks: 10804988
Signature: t1:3498613:1467232574:75c80b0223c101236b4d4e0f27f46f72797ebe90
Summary:
chg runs extsetup and uisetup with dummy ui objects. Before this
patch we were relying on ui objects in extsetup and uisetup. After this
patch, we move the logic to reposetup. We do things a little differently
because reposetup can be called multiple times and extsetup and uisetup are
called only once.
This caused one problem for `hg clone` in the test as reposetup is not
called for clone operation. I adapted the test a little bit to cover this case.
Test Plan: Modified tests pass
Reviewers: mitrandir, quark, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3499176
Signature: t1:3499176:1467244335:a70d718725c1d19ab6b6feb4558a8ffd1a020ab2
Summary:
Rather than put everything into one journal file, split entries up in *shared*
and *local* entries. Working copy changes are local to a specific working copy,
so should remain local only. Other entries are shared with the source if so
configured when the share was created.
When unsharing, any shared journale entries are copied across.
Test Plan:
Run all journal tests:
$HGSRC/tests/run-tests.py -l test-journal*
Reviewers: #sourcecontrol, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3481745
Tasks: 10804988
Signature: t1:3481745:1467215718:3b6666c2b908cea279f181f5a9310c0093d0c0fe
Summary:
Note that now the default action for `hg journal` is to list the working copy
history, not all bookmarks. In its place is the `--all` switch which lists all
name changes recorded, including the name for which the change was recorded on
each line.
Depends on D3458115
Test Plan:
Run all journal tests:
$HGSRC/tests/run-tests.py -l test-journal*
Reviewers: #sourcecontrol, rmcelroy, durham
Reviewed By: durham
Subscribers: durham, mitrandir, rmcelroy, ttung, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3462793
Tasks: 10804988
Signature: t1:3462793:1466703746:8ee2b9822e39a458d64d30b160de3ecb77c623e7
Summary:
This change would have to go into remotenames directly when hg journal itself
is added to Mercurial core.
Depends on D3458077
Test Plan:
Run the included test-journal*.t tests
$HGSRC/tests/run-tests.py -l test-journal*
Reviewers: #sourcecontrol, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3458115
Tasks: 10804988
Signature: t1:3458115:1467217834:1f8421da38280ab333faa7142118de1ef0e1aed4
Summary:
This is slated to be the upstream version of reflog. It is being hashed out
here to accellerate moving from reflog to journal.
This initial revision only supports bookmarks; later diffs add remote bookmarks
and working copy changes. As such the storage format is built to support those
usecases but the commandline usage is still geared towards only needing to list
bookmarks for now.
We'll replace reflog.py with journal.py altogether, but due to the fine details
of our RPM / chef config duality reflog.py needs to stick around until we
rolled out the RPM everywhere and updated chef to follow.
Test Plan:
Run the included test-journal.t test
$HGSRC/tests/run-tests.py -l test-journal.t
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3458077
Tasks: 10804988
Signature: t1:3458077:1467215042:d3fbba283901de6abee5d2b28a43ed9293627795
Summary: ui.log expects msg[0] to be the format string, and msg[1:] to be the arguments.
Test Plan: extend test-sampling to cover the case of a formatting string + args. pass test-sampling.t
Reviewers: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3479711
Summary:
Since metrics doesn't send a string, let's just stub out the call.
The alternative is to put a '' in every `recordsample()` call.
Test Plan: pass existing tests.
Reviewers: lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3479138
Tasks: 11900487
Signature: t1:3479138:1466787619:910917093cf0361ed181af5597ab702b936f8d1e
Summary:
1. verify that we cache entries on background.
2. verify that we don't fill the cache when we can't take the lock.
Test Plan: passed existing unit tests.
Reviewers: durham, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3471355
Signature: t1:3471355:1466719450:191070cd2d5ef93e0e1a6925a04d1a885431cbb4
Summary: --created is the opposite of --deleted.
Test Plan: meh. used in later diff.
Reviewers: lcharignon, mjpieters
Reviewed By: mjpieters
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3478780
Signature: t1:3478780:1466759469:47a3af68df2c8d550aa654f135d4f0fcf6df7851
Summary: It's actually most relevant to least relevant. Add a docblock.
Test Plan: pass unit tests
Reviewers: lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3478616
Signature: t1:3478616:1466787566:b38eceddf91c3e88341f9173e07244ac33ab6345
Summary: This allows us to build tests to verify the behavior of background cache fills.
Test Plan: used in next diff.
Reviewers: mjpieters, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters, #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D3471343
Tasks: 11683504
Signature: t1:3471343:1466698914:93d15b2d4879b9d8a91aeab62fcf119f24de3c1c
Summary:
Everything will be synchronous within the process. We will do background work in a separate process.
Depends on D3468827
Test Plan: updated tests to remove the one background task.
Reviewers: durham, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3468828
Tasks: 11683504
Signature: t1:3468828:1466694717:556968a10717c3dd7ce403593cc6a1e97067633f
Summary:
This extensions prevents mercurial from being ctrl-c'ed by warning the
user on the first SIGINT. The subsequent interrupts are handled normally.
Test Plan:
see tests
(the tests are lacking the scenario of double ctrl-c - any suggestions on how
to do it in nice way inside the test framework welcome)
Reviewers: #sourcecontrol, durham, wez, quark, akushner, rmcelroy
Subscribers: lcharignon, quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D2936488
Summary:
Directly execute the xargs command, rather than using a shell. This is
slightly more efficient, but also ensures we won't have security problems or
other issues based on the user's grep.command config setting. Even if it
contains shell redirects, semicolons, or other special characters, we will no
longer pass them to a shell to interpret them.
We do use shlex.split() go process the grep.command setting, to allow the user
to specify a command plus arguments.
Test Plan: Included new unit tests.
Reviewers: #sourcecontrol, durham, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: akushner, rmcelroy, sid0, net-systems-diffs@, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3376617
Signature: t1:3376617:1464841542:f58fc0a05d62b649edc70bc261009875385186c9
Summary:
Previously grep only searched in the current directory, even if you gave it a
pattern like "../". This fixes the code to correctly honor the arguments, and
to accept mercurial file patterns just like most other mercurial commands.
This does change the file parsing behavior somewhat: previously "hg grep" would
treat all arguments as glob patterns. Now you explicitly have to say
"glob:<pattern>" to get glob matching. However, I suspect this won't impact
most users, since they would normally let their shell perform glob expansion
before the arguments are given to mercurial.
Test Plan:
Included new unit tests for grep, and updated the existing tests that were
affected by the change in glob behavior.
Reviewers: #sourcecontrol, durham, carenthomas, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: quark, mwilliams, matthieu, meyering, yfeldblum, rmcelroy, net-systems-diffs@, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3376605
Signature: t1:3376605:1464841648:98bfb47866dec802ca34f55c821796f36532211f
Summary:
This patch fixes various things around the sampling extension to
match what our wrapper expects to see. See detail in the test.
Test Plan:
lcharignon@XXX fbsource cat ~/.hgrc
...
[extensions]
sampling=
[sampling]
key.fastmanifest-cachehitratio=table_blah
key.fastmanifest-trigger=table_blah
filepath=/dev/shm/samppath
lcharignon@XXX fbsource hhg book foo
lcharignon@XXX fbsource cat /dev/shm/samppath
{"category": "table_blah", "data": {"source": "bookmark", "metrics_type":
"fastmanifest-trigger"}}\0{"category": "table_blah", "data": {"ratio": -1, "metrics_type":
"fastmanifest-cachehitratio"}}\0
Reviewers: ttung, durham
Differential Revision: https://phabricator.intern.facebook.com/D3466719
Summary:
This patch adds a test that uncovers logic bugs in the cache. It also
adds fixes for these bugs.
Test Plan:
Add a new test and ran existing tests, observed no change for
exisiting test.
Reviewers: durham, ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3450430
Summary: This patch fixes the torulemethod of the exec action to match upstream.
Test Plan: Add a test that failed before the codechange
Reviewers: mitrandir
Differential Revision: https://phabricator.intern.facebook.com/D3459588
Summary:
The upstream change has now been *properly* landed at
https://selenic.com/repo/hg//rev/6ee50bdeeb32.
Once again, Mea Culpa, yesterday I apparently was in clowntown mode with these
patches.
Test Plan:
Run all affected tests:
$HGSRC/tests/run-tests.py -l test-pull-createmarkers* test-pushrebase* test-pushvars*
Reviewers: #sourcecontrol, ttung, quark
Reviewed By: quark
Subscribers: mjpieters, durham
Differential Revision: https://phabricator.intern.facebook.com/D3462832
Signature: t1:3462832:1466513182:58d6187c04e0e567a026aba90bf753c76f4949a7
Blame Revision: D3460645
Summary:
e61bd9d5e8ca requires a change to core that has been accepted but not yet
pushed. This breaks the build, so let's revert it until upstream pushes the
dependency.
Test Plan: Ran the tests
Reviewers: ttung, mjpieters, #sourcecontrol, mitrandir
Reviewed By: mitrandir
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3460645
Signature: t1:3460645:1466465904:cd521a989e475f9987237307a7d4d672d58ba615
Summary:
Before this patch, we were not importing mercurial.error, this was
causing a crash when calling error.Abort. This patch adds the missing import.
Test Plan: Tests pass, and add a new test
Reviewers: durham
Differential Revision: https://phabricator.intern.facebook.com/D3457086
Summary:
Now that my upstream patch for filedescriptor class access has landed the helper function to bypass the descriptor protocol can go altogether.
Depends on D3454878
Test Plan:
Run affected extension tests:
$HGSRC/tests/run-tests.py -l test-sqldirstate-* test-reflog* test-sparse*
Reviewers: #sourcecontrol, ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3456368
Summary:
There were 3 clowntowny versions of the wrapfiledescriptor function, 2 being
dangerously wrong about how they scan the class hierarchy. All because
`classobj.descriptorname` invokes the descriptor protocol and the filecache
descriptor is not set up for that access.
Instead of one function to do it all, use one utility function to retrieve
the descriptor object from the class __dict__ to avoid the descriptor protocol
kicking in, and use mercurial.extensions.wrapfunction to do the actual
wrapping.
I've sent a patch upstream to fix accessing the filecache descriptor on the
class; if accepted we can drop the getfilecache utility function altogether,
leaving only the wrapfunction calls.
Test Plan:
Run affected extension tests:
$HGSRC/tests/run-tests.py -l test-sqldirstate-* test-reflog* test-sparse*
Reviewers: #sourcecontrol, ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3454878
In this race condition test, occasionally the second invocation would actually
obtain the lock before the first. This meant that the first repack would fail
with an error message while the second would exit with 0, resulting in the test
output changing slightly. Let's introduce a slight delay before the second
invocation to prevent this from happening.
Summary: This will be useful to refactor the cache
Test Plan:
We change the python test to give a vfs that makes sense, it will
be useful in the next patch when we refactor the cache. Indeed, with the new
cache we use systemawarecachelimit by default instead of no limit
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3427796
Summary:
Before this patch, if we decided to remove entries A,B,C where A is
more recent than B, which in turn is more recent than C. We were removing
in order A,B and C. This patch changes the order to C,B and A and will make
the next patches simpler.
Test Plan: Tests are updated accordingly
Reviewers: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3427795
Summary:
When refreshing cache entries, it is useful to know what delay we
introduce. This patch adds a debug line showing that information.
Test Plan: tests are updated accordingly
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3427794
Summary:
Before this patch, it wasn't clear from hg debugcachemanifest --list
which manifest were older and newer or what rev were actually cached. This
patch improves the output of hg debugcachemanifest --list --debug to show
this information.
Test Plan: add debug output to the test
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3427791
Summary:
Before this patch we were returning the cache entries alphabetically.
This patch changes the behavior to return the entries sorted by date and
alphabetically. This will be used to simplify the code for pruning entries
in furter patches.
Test Plan: test output changes
Reviewers: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3427789
Summary:
Before this patch we were stopping execution when the lock couldn't
be taken. This was problematic when running hg command for someone else's repo
(for example to debug problems). This patch allows us to continue the execution
of the command when that happen, except we won't use fastmanifest.
Test Plan: This is reflected in the test output
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3422458
Summary: This could break people using fastmanifest in someone else's repo
Test Plan: tests pass, not sure how we normally test permission stuff
Reviewers: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3420870
Summary: It can't find those extensions now that we removed the extensions from the system pythonpath. Therefore, we need to load it from the source directory.
Test Plan: pass PYTHONPATH=~/work/mercurial/facebook-hg-rpms/remotenames/:~/work/mercurial/facebook-hg-rpms/mutable-history/hgext python ~/work/mercurial/facebook-hg-rpms/hg-crew/tests/run-tests.py -j32 test-fbhistedit-stop.t
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3419503
Signature: t1:3419503:1465590764:23e348efee30cabbbff232b7457d5820125bd6ad
Summary: printf appears to be more platform agnostic than echo -n, which is not supported on bsd/mac.
Test Plan: run PYTHONPATH=~/work/mercurial/facebook-hg-rpms/remotenames/:~/work/mercurial/facebook-hg-rpms/mutable-history/hgext python ~/work/mercurial/facebook-hg-rpms/hg-crew/tests/run-tests.py -j32 test-catnotate.t
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3419500
Signature: t1:3419500:1465590785:a34da9af73c64d9edef66a7b4a6ed3abd885e5c7
The previous commit fixed a bug where copyfrom data was represented incorrectly
in the local .hg/store/data remotefilelog blobs when the ancestor data was read
from a pack file. This commit adds a test for that situation.
Summary:
Now that repack can clean up old remotefilelog blobs, let's have it also delete
any empty directories that get left behind.
Test Plan: Updated an existing test to cover it
Reviewers: mitrandir, lcharignon, #sourcecontrol, ttung, simonfar
Reviewed By: simonfar
Subscribers: simonfar
Differential Revision: https://phabricator.intern.facebook.com/D3385546
Signature: t1:3385546:1464972782:5ca63cf0a5589bb8a537957f50b4bc5ec4e0f0f5
Summary:
Previously a bunch of different places accessed the cachepath through ui.config
directly. This is a problem because we need to resolve any environment variables
in the path, and some spots didn't do this. So let's unify all accesses through
a helper function that takes care of the environment variables.
Test Plan: Added a test
Reviewers: mitrandir, lcharignon, #sourcecontrol, ttung, simonfar
Reviewed By: simonfar
Subscribers: simonfar
Differential Revision: https://phabricator.intern.facebook.com/D3385583
Signature: t1:3385583:1464971813:5b9ee5ed3d6ff9f1a78cb9e0269e433844758c9d
Summary:
As discussed, we want to make sure we have a good SocketServer for Python 2.6,
which is unsupported by PSF but still being widely used by our users.
This diff adds a new extension that tries to fix buggy Python standard modules,
namely the SocketServer. It fixes a critical race condition for Python 2.6
(https://bugs.python.org/issue21491) and the zombie process issue for both
Python 2.6 and 2.7 (https://bugs.python.org/issue11109).
A simple test is added to check zombie processes are no more.
Test Plan:
Run the newly added test, `test-patchpython.py` with both Python 2.6 and 2.7.
Comment out the `import patchpython` line in the test and make sure it fails
on Python 2.6 and 2.7 with `AssertionError: zombie process detected`.
Reviewers: #mercurial, ttung, mjpieters
Reviewed By: mjpieters
Subscribers: simonfar, durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3384909
Signature: t1:3384909:1465319491:0ba64b496186712651b7ee6bd502d1e2ccac57d1
Summary: Add code to log metrics for fastmanifest, also handles aggregate
metrics. We reuse ui.log for logtoprocess. We also refactor diffs and
filesnotin to make it easier to log cache hit and miss ratio.
Test Plan:
Replaced ui.log by ui.status and aggregated the field, check
that I see metrics getting logged.
Reviewers: ttung, durham
Differential Revision: https://phabricator.intern.facebook.com/D3368504
Summary: This mostly replicates the logic in `manifest.py::manifest::add(..)`, except it interfaces with the fastmanifest cache instead.
Test Plan:
used `hg debugcachedmanifest` to populate the cache, then did a commit with `pdb.set_trace()`.
added a correctness test that exercises the code path.
Reviewers: simonfar, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3397910
Tasks: 11294278
Signature: t1:3397910:1465407138:60c7b696b4bafa153c8076ab58c53bf71a9fcfd4
Summary:
Before this patch, all the events logged by the sampling extensions
were dumped to the log file. With this patch, one can specify the relevant
events to log as well as a value to be added for each event. This will be
useful to specify what logging table to use for each of the logged events.
More information added to the documentation accordingly.
Test Plan:
The test changes accordingly and we add events that gets stopped
by the filter to make sure that the filter works.
Reviewers: ttung, durham
Differential Revision: https://phabricator.intern.facebook.com/D3387446
Summary:
We used to have test code to make the test deterministic. Now we
enforce the same constraint in code and that code is not necessary anymore.
We change the test output to match the new, deterministic output.
Test Plan: Ran the tests 80 times and checked that the output is stable
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3387259
Summary:
This will allow us to know what revisions are actually cached vs
revisions that would be cached by a trigger.
Test Plan: Add new tests
Reviewers: ttung, durham
Differential Revision: https://phabricator.intern.facebook.com/D3385955
Summary:
`lrucachedict`'s `get` method is a lower-level API that retrieves a _lrucachenode object, not what we want. the idiomatic construction is:
```
if key in lrucachedictobj:
do_something(lrucachedictobj[key])
```
Test Plan: hg amend with fastmanifest works. wrote a test that failed before this diff, passed after.
Reviewers: #mercurial, lcharignon, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3384502
Tasks: 11569523
Signature: t1:3384502:1465004304:8dfee6a4d0b2f6bb39262a310744458838cb0bf3
Summary:
This patch adds a new extension: sampling. It allows us to collect
aggregated data to a log file for each command that runs.
We will use it with our wrapper to collect data on sqldirstate and fastmanifest.
The workflow:
- Wrapper creates a log file, passes it to mercurial through an environment variable
- Mercurial writes data to this log file
- The wrapper parses the log file and upload its content to a data logging backend
Test Plan: Add a test
Reviewers: mitrandir, wez, ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3360988
Summary:
BSD version of head does not support -n0 and it looks like the only purpose of
using it here is to hide stdout output (and leave stderr).
Test Plan:
Ran test-tweakdefaults.t on both Linux and OSX, made sure that this particular
part of the test does not fail anymore.
That said, test-tweakdefaults.t still fails on OSX, but that requires more
investigation and my 5-minute look did not reveal the reason.
Reviewers: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3378544
Summary:
Before this patch, we were passing None as ui for hybridmanifest's
constructor in test-fastmanifest.py. With this patch, we pass a dummy ui object.
Therefore, it removes the need for None guard in hybridmanifest and simplifies
the code.
Test Plan: Tests pass
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3374069
Summary: We got all we wanted from this.
Test Plan: meh.
Reviewers: #mercurial, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3373068
Signature: t1:3373068:1464810628:ebfa809a0a02d6814c153c1b47354aae9a98dde1
Previously, background repacks would only repack pack files, which meant there
was no automated way to repack loose remotefilelog files without manually
running 'hg repack'. This allows incremental repacks to also pack the loose
files.
It also changes the config knob for background repacks, so we can enable pack
file usage without the server having to support it just yet.
Summary:
Instead of refreshing the cache at the time of the trigger, accumulate the repos we need to refresh, and do so once at the end.
This also allows us to remove the hacky once-per-process gating.
Test Plan: pass most of the existing tests, update the remotenames test.
Reviewers: quark, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters, #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D3363061
Tasks: 11475606
Signature: t1:3363061:1464465376:574c694d345b61109dca2c9e0b9fee0d8f742e46
Summary: dirstate changes are overly broad. we really only want the trigger if we make a new commit.
Test Plan: will update tests after this series of commits, as the tests are not really exercising the code paths we want.
Reviewers: lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters, #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D3363013
Tasks: 11475606
Signature: t1:3363013:1464465629:8aad5cbb5f357e751a11aab593347327c50314b5
Summary:
This is neccessary for scmprompt to work. I added a config knob because we
may want to disable it if we want to do paranoid mode at some point.
Test Plan: ran tests
Reviewers: #mercurial, durham
Differential Revision: https://phabricator.intern.facebook.com/D3356646
Summary: I based my code on the generaldelta upgrade which also occured on pull.
Test Plan: test will follow in next comit
Reviewers: #mercurial, durham, ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3356590
Summary:
This is somewhat insulated against potential future design changes in chg, as we record the last pid to execute `cachemanifestfillandtrim`, and only suppress it if the pid is unchanged.
Notice the unit test no longer does extra calls to populate the cache.
Test Plan: passed unit tests
Reviewers: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3357697
Summary: When --debug is turned on for commands, there's a ton of spew. This makes it easier to spot the parts I care about.
Test Plan: update tests
Reviewers: lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3357974
Signature: t1:3357974:1464365624:83edc815109db8533465991413a7c7741f1056a8
Summary: We want to wrap the fastmanifest cache priming code in a lock, but we want it to be easily stealable. If the lock is more than X seconds in age, we just assume it's stale and steal it. We remember that we stole it so we don't blow away the lock, but we do update the time of the lock so another process doesn't try to steal the lock.
Test Plan: write test to cover basic functionality (mutual exclusivity). write test to cover stealing, and that only one thief is permitted at a time (subject to race conditions).
Reviewers: durham, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters, #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D3343478
Tasks: 11385124
Signature: t1:3343478:1464304447:18658ebea60f98bfda0f034414991ffd6c334ca7
Summary:
This test is mostly a copy of test-shelve.t from mercurial with
minor changes (removing the hooks usage).
The normal tests for shelve unfortunately are using out of proces
hooks which doesn't work good with sqldirstate. As shelve is
important extension for us we want to test is with shelve anyway.
Test Plan:
test passes on clowncopter with this patch applied:
http://patchwork.serpentine.com/patch/15207/
Reviewers: #mercurial, wez, quark, durham
Reviewed By: durham
Subscribers: quark
Differential Revision: https://phabricator.intern.facebook.com/D3350194
Signature: t1:3350194:1464260281:c85a1d5ae50d9488a8bcdb552343f466bebb9e05
Summary: This combines D3351047, D3351086, and D3351137 into one diff. This allows mercurial to better remember where things came from since these are all hg cp commands followed by some edits. However, they were separated into multiple diffs for ease of reviews.
Test Plan: run unit tests
Reviewers: lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3354405
Signature: t1:3354405:1464291057:da140a02b9c87186b1dd9f03f8236d5996319fff
Summary:
Before this patch, we were caching in the background in remotenames
test. This could result in a race condition making the test non deterministic.
Test Plan: Test changed
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3353406
Summary:
As discussed it is not relevant as the tests cache so little it should
not make the test non-deterministic.
Test Plan: The test change accordingly since the limit is no longer None
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3353394
Summary: This allows us to use fastmanifest as a directory to drop in the python module.
Test Plan: compiles, passes existing tests.
Reviewers: lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3351021
Signature: t1:3351021:1464284417:6cbcde514ab1fd7b5caa6c83cb5577f3502dbc58
Summary:
Before this patch, we didn't test that caching could be triggered by
remotename changes. When I tried testing it, I noticed that it wasn't working
because manifest for revisions with remotenames were not cached. I added
remote/master as a revision to cache to the fastmanifesttocache revset and
included a test.
Test Plan: Added a new test
Reviewers: simonfar, quark, ttung, durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3335931
Summary: This will limit what we cache in the first place
Test Plan:
No test yet, I will add a test in a diff later checking cache
eviction.
Reviewers: ttung, durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3346515
Summary:
As discussed in the group, we want to shuffle by batches to keep
an approximate ordering and still avoid caching process fighting for the
same entries.
Test Plan: Add a new test
Reviewers: durham, ttung
Differential Revision: https://phabricator.intern.facebook.com/D3344144
Summary:
If we want to cache revision [300, 350, 320, 380], we would first order the
revisions to be [380, 350, 320, 300] and cache them in that order. Assuming that
only 3 revisions can fit in the cache, we write all revisions and prune the
one that overflew. Afterwards, we change the mtime of all the cached entries
to ensure that for each of the newly cached entries, more recent mtime == more
relevant entry.
Test Plan: Test output changes slightly, as expected
Reviewers: mitrandir, ttung, simonfar
Differential Revision: https://phabricator.intern.facebook.com/D3344125
Summary: also remove some tests that are passing now from the list
Test Plan: ran mercurial tests with sqldirstate
Reviewers: #mercurial, ttung, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3345259
Signature: t1:3345259:1464168975:1f6e26ce730b32e5b0542ab3ef7e1dc7088c0b66
Summary: Interpret revrange inside the worker process. The reason this is needed is that `triggercacheondirstatechange` happens in the wlock release callback. However, the lock is not released at this point. Interpreting the revrange requires wlock, so then we try to acquire it but we'll never succeed (because we need to finish the wlock release callback). If we do it in the scope of the worker process, it'll try to acquire it, find that the primary process owns it, and wait to acquire it.
Test Plan: run unit tests
Reviewers: durham, lcharignon
Reviewed By: lcharignon
Subscribers: quark, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3339805
Tasks: 11385141
Signature: t1:3339805:1464209208:ec2424f84570d489aa5c0c629a15b69b6b126c60
Summary: This could be useful to have the number of valid entries in the cache
Test Plan: Test output changes accordingly
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3343488
Summary:
Before this patch with fastmanifest.cacheonchange on, we would cache
manifest on change to dirstate, bookmarks and remotenames. This patch makes
these operation in the background by default. Also it changes the cache strategy
used by default to be the new strategy that adapts based on the system resources.
Test Plan: Tests does not change
Reviewers: ttung, simonfar, quark
Differential Revision: https://phabricator.intern.facebook.com/D3341681
Summary:
I didn't fix the non determinism properly before, this patch fixes
the issue with a more defensive code.
Test Plan: Ran the test 32 times and the result didn't vary
Reviewers: ttung, simonfar, quark
Differential Revision: https://phabricator.intern.facebook.com/D3341032
Summary: If we don't, then a lazily-evaluated set is returned. That lazily-evaluated set will suppress the hidden bit when dirstate changes are involved.
Test Plan: on a dirstate change with a lot of hidden revs, observe that only a small set of revs are cached.
Reviewers: lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters, #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D3338252
Signature: t1:3338252:1464107195:fd0e342fb4e4eb485057b7fda9679f7ff18f4e7b
Summary:
Now that we have a mutablebasepack base class, we can get rid of all the
redundant logic in mutablehistorypack. This also has the side effect and making
the historypack's fanout table dynamically size, just like the datapack already
does. That required a few changes to the historypack reader class as well.
Test Plan: Ran the tests
Reviewers: mitrandir, #mercurial, ttung
Differential Revision: https://phabricator.intern.facebook.com/D3306546
Summary:
Before this patch, we were iterating over the cache entries in a non-
deterministic fashion. This patch makes the iteration stable by ordering the
entries by filename.
Test Plan: Changed the tests accordingly
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3336880
Summary:
We order by mtime and then by filename and not just by mtime. Otherwise, we
wrongfully relied on os.listdir to return the same thing on different platforms.
Test Plan: Change the test
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3336874
Summary:
When we start worker process to cache revisions, since we don't use
a lock, it is better to have the workers cache revisions in a random order to
avoid duplicated effort as much as possible.
Test Plan: No test for now, not sure how to test that.
Reviewers: simonfar, quark, ttung
Differential Revision: https://phabricator.intern.facebook.com/D3335918
Summary:
This adds a new strategy for the cache limit that depends on the free
space availabe in the system. For system with more than 100GB of free space
we allocate 5GB for the cache. With more than 20 GB of free space we allocate
2GB of free space and otherwise 10% of the free space at most.
Test Plan: Add a new test
Reviewers: ttung, durham, rmcelroy
Differential Revision: https://phabricator.intern.facebook.com/D3330894
Summary:
This adds the first, basic implementation of size limitation for
the fastmanifest cache
Test Plan: Add a few tests that cover edge cases
Reviewers: ttung, simonfar, durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3326149
Summary:
This patch makes it possible to cache fastmanifest asynchronously. I added
a test an also tested with fbsource:
hg debugcachemanifest --debug --config extensions.fastmanifest=/data/users/lcharignon/facebook-hg-rpms/fb-hgext/fastmanifest.py -R ~/fbsource --all
hg debugcachemanifest --debug --config extensions.fastmanifest=/data/users/lcharignon/facebook-hg-rpms/fb-hgext/fastmanifest.py -R ~/fbsource --list
This is a tricky code, and I think @quark would be best to review it.
Test Plan: Added a new test
Reviewers: quark, durham, ttung
Subscribers: quark
Differential Revision: https://phabricator.intern.facebook.com/D3323540
Summary: This is a refactoring of the _cachemanifest function
Test Plan: test output does not change
Reviewers: ttung, simonfar
Differential Revision: https://phabricator.intern.facebook.com/D3330501
Summary:
We use the manifestnode from the changelog and avoid having to reach the manifest
completely to check if it is already in cache.
This goes with a refactoring of the contains method. Before this patch, the
contains method was misleading because an entry could be contained in the cache
but not show up when you iterate over the cache. This happened because contains
was operating on nodes and iterating showed filesnames (so keys + prefix). To
make it clear that contains operates on nodes, we change its name to containsnode
We also rename key to hexnode to make it clearer that the keys are hexnodes.
Test Plan: Existing tests pass. We also add a new test to show that the fast
pass is hit.
Reviewers: durham, ttung
Differential Revision: https://phabricator.intern.facebook.com/D3325335
Summary: This will be useful for implementing garbage collection later.
Test Plan: Add a new test that exercises the new function
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3322426
Summary:
This patch changes fastmanifest to add automatic caching of relevant
manifest when bookmarks change or if the parent of the working copy changes.
Test Plan: Added a test
Reviewers: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3318775
Summary:
We move wrapfilecache from reflog to extutil to be able to reuse it
in fastmanifest
Test Plan: Tests pass
Reviewers: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3318762
Summary:
Adding a pruneall method to prune all cache manifest. This will be
useful to test the caching logic without having to run shell commands to remove
manifests.
Test Plan: Add a new test
Reviewers: ttung, durham
Differential Revision: https://phabricator.intern.facebook.com/D3314158
Summary:
When people combine --root-relative with a file pattern, they would probably
expect the output to be relative to the root -- but it's not! In this case, it
would fall back to the default file pattern behavior, which is cwd-relative.
Instead of confusing the user with apprarently errorneous output, let's abort
explicitly and provide a helptful hint.
Test Plan: added a new test, existing tests still pass
Reviewers: #mercurial, simpkins, ttung, quark
Reviewed By: quark
Subscribers: quark, net-systems-diffs@, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3319366
Signature: t1:3319366:1463610688:3f129c97f68f43ac85d2b31b55fac5c859e85c04
The iterbatch() handling added in f93aa99d4f1e (fileserverclient: use
new iterbatch() method, 2016-03-22) was broken by 31e88bf6faf0 (store:
change fileserviceclient to write via new store, 2016-04-04). Fix it
by copying the pattern introduced elsewhere in that change.
Summary:
mutabledatapack and mutablehistorypack share a lot of common code, especially
around the index and fanout table. Let's move much of the code to a common
mutablebasepack class and out of mutabledatapack. In the next patch I will make
mutablehistorypack a subclass of mutablebasepack and delete all the duplicate
logic.
Test Plan: Ran the tests
Reviewers: mitrandir, #mercurial, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: quark, rmcelroy
Differential Revision: https://phabricator.intern.facebook.com/D3306542
Signature: t1:3306542:1463611860:16bc68416c9bbed87748a50f55a3bac7c618fdf1
Summary:
In normal Mercurial, the filelog entry's contents contains extra metadata that
stores the copy source. In the new pack format, that information is stored in
the history store, not in the data store. Therefore we need to change the server
side logic that responds to requests for packs to move that information over to
the history side before it sends the data.
Test Plan: Added a test
Reviewers: ttung, mitrandir, #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy
Differential Revision: https://phabricator.intern.facebook.com/D3306539
Signature: t1:3306539:1463609462:0c1e33e0892f96effcc96c8f78401cf0d8ab5cbd
Summary:
Previously we only had incremental repacking for data packs. This patch adds it
for history packs as well. The algorithm here is simpler, since the amount of
history data is generally much smaller than the amount of delta data.
The algorithm is basically: if there are 2 things bigger than 100MB, repack
them; else repack up to 100MB of smaller things.
The datapack hashes changed because having the history available during a repack
allows us to make different decisions about delta ordering, etc.
Test Plan: Updated the tsets
Reviewers: mitrandir, #mercurial, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy
Differential Revision: https://phabricator.intern.facebook.com/D3306535
Signature: t1:3306535:1463696613:f40ed10c9dfed40d7bc455582592a7aed108ec3a
Summary:
This runs the incremental background repacking logic after hg pull.
As part of adding tests, I also added a 'hg debugwaitonrepack' function that
will wait until any pending repack is done before returning, so the tests can
wait on repacks without so many sleeps.
Test Plan: Adds a test
Reviewers: mitrandir, #mercurial, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy
Differential Revision: https://phabricator.intern.facebook.com/D3306526
Signature: t1:3306526:1463696933:9e27daf0c08076468e8f365a3c372fa7d4f56bde
Summary:
This adds a --incremental flag to the hg repack command. This flag causes repack
to look at the distribution of pack files in the repo and performs the most
minimal repack to keep the repo in good shape. Currently it's only implemented
for datapacks.
The new remotefilelog.datagenerations config contains a list of the sizes for
the different generations of pack files. For instance:
[remotefilelog]
datagenerations=1GB
100MB
1MB
Designates 4 generations - packs over 1GB, packs over 100MB, packs over 1MB, and
implicitly packs undex 1MB. The incremental algorithm will try to keep each
generation to less than 3 pack files (prioritizing the larger generations
first). When performing a repack it will grab at least 2 packs, and will grab
more if the total pack size is less than 100MB (since repacking at that level is
pretty cheap).
I have no idea if this is a good algorithm. We'll how to see and iterate.
Test Plan: Adds a test
Reviewers: mitrandir, #mercurial, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy
Differential Revision: https://phabricator.intern.facebook.com/D3306523
Signature: t1:3306523:1463697129:c87f4a397ef357b5ca4a80d01e9a6ca4d61f9d3d
Summary:
A future patch will be adding incremental repack, so let's move our repack logic
to the repack module so it's easier to refactor and extend.
Also adds a message for when a background repack kicks off (since we'll be
calling that from other places eventually).
Test Plan: Adds a test
Reviewers: mitrandir, #mercurial, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy
Differential Revision: https://phabricator.intern.facebook.com/D3306521
Signature: t1:3306521:1463602886:cece3d517f0672b829702866482c902812f9ae27
Summary:
* `_manifest()` should attempt to retrieve a fastmanifest if possible.
* `self.incache` represents a tristate indicating True if the fastmanifest is available, False if the fastmanifest is not, and None if we haven't tried to determine its availablility.
* clarified the debug messages a bit
* re-introduce the test removed in D3277498 (since it actually can work now).
Test Plan: pass unit tests!
Reviewers: lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3288967
Signature: t1:3288967:1463416544:416f28ecdc5e6a26545f3d4215fc8baa9222af7c
Summary: since it's a singleton, we can have one instance of the code for retrieving the instance.
Test Plan: run tests/test-fastmanifest.t
Reviewers: lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3284113
Signature: t1:3284113:1463416610:207c7e9954cc0b830e77f2fecc0a65acfcfd1096
Summary: Flat manifests are cached in memory by manifest.py. Unclear whether there's any reason for us to cache flat manifests on disk.
Test Plan: ran test-fastmanifest.py
Reviewers: durham, lcharignon
Reviewed By: lcharignon
Subscribers: trunkagent, mitrandir, mjpieters, #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D3277498
Tasks: 10589051
Signature: t1:3277498:1463416673:74cab287b9045166dc0322a45f8a325f65da5d1f
Summary:
I want to re-use this elsewhere, so robustify it a bit
and move it to phabricator.diffprops.
Test Plan: run-tests.py, also verified in my www repo.
Reviewers: #sourcecontrol, ttung
Reviewed By: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3230147
Signature: t1:3230147:1463081787:799f232e2ce73395218db3a0fff37dec9a0b02e0
Summary: Previously, we were just reading the first rev we accessed. This is modified to read the entire file.
Test Plan:
I don't see any non-determinism any more. hammered with
```
while true; do python ~/work/mercurial/facebook-hg-rpms/hg-crew/tests/run-tests.py --keep-tmpdir test-fastmanifest.t; if [ $? -ne 0 ]; then break; fi; done
```
Reviewers: durham, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters, #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D3278649
Tasks: 10604335
Signature: t1:3278649:1463063754:1905372c2da75dcc42d09a3a8cda4ff5a93f1c04
Summary:
The manifests consumed by mercurial does not appear to be nondeterministic, which is reassuring. There is, however, a small bug in the test code. That is resolved in the next diff.
This is *not* a straight revert, as this file has changed. The minimal set of changes were done merge the two.
Test Plan: next diff, sorry.
Reviewers: durham, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters, #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D3278623
Signature: t1:3278623:1463067584:30ea81ac48fb4c8df1dcf7bf6d17233369b47cc3
Summary:
An extension replacing dirstate file with sqlite database so we can have incremental changes and we don't have to read the whole dirstate on every op. This makes sense only when hgwatchan/fsmonitor is on so we don't iterate through whole dirstate.
This is also using the sqlite transactions to handle dirstate transactions instead of copying db around. As a result of that "hg rollback" doesn't work anymore. You can fall back to copying things by setting sqldirstate.skipbackups to False.
Needs those to go to upstream to work: https://phabricator.intern.facebook.com/P56319612
(will send them once the freeze is over)
To use make sure that the extension is loaded *before* hgwatchman (watchman
should be outmost layer).
Test Plan:
Passing all but few mercurial tests (when running with skipbackups=False)
The failures are described in blacklist file.
Reviewers: lcharignon, wez, quark, durham
Reviewed By: durham
Subscribers: laurent, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D3242547
Signature: t1:3242547:1462577481:fdbfb5287fb8d3e58f7b4d587c01de79ce6b78df
Summary:
When running with chg, `sys.argv` is not a reliable source to get command line
arguments. Hacking chgserver with setting `sys.argv` is possible but that
is not welcomed by upstream (http://patchwork.serpentine.com/patch/14318/)
Instead, wrap `dispatch.runcommand` to get the arguments for now. We will
lose all the `--config` arguments but that seems okay.
Test Plan: Run test-reflog.t
Reviewers: #mercurial, ttung, durham
Reviewed By: durham
Subscribers: durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3264615
Tasks: 11114741
Signature: t1:3264615:1462487500:44e5cd91b5d4b5bc60a7d01a816fb91ed54aa6bb
Summary:
Shelve and unshelve use commits under the hood to bundle up data. dirsync sees
these commits happening and performs mirrors, which then shows up at unbundle
time, making it impossible to mirror the change again. Example: if I change X,
and it gets mirrored to Y, if I go back and change X later after a unshelve, hg
commit now fails because Y has pending changes and differs from X.
Test Plan: Added a test
Reviewers: #mercurial, ttung, quark
Reviewed By: quark
Subscribers: quark, wez, mjpieters, frantic, wluh
Differential Revision: https://phabricator.intern.facebook.com/D3266602
Signature: t1:3266602:1462484017:738f67c0ab4b5af999819d3855c1f4ba6b2ea338
Summary:
API to write trees to disk and read trees from disk.
Some interesting things to keep in mind:
* To keep valgrind happy, we have to clear all the memory we allocate but never initialize. That can be padding between fields in structs, fields that don't make sense to initialize, the extra byte in checksums that doesn't get used normally, or portions of bitfields that are never used. All this logic is kept compartmentalized in `initialize_unused_bytes(..)`. Normal runtime does not invoke this code, though I suspect the cost is probably not significant.
* A significant chunk of the write path is made up of the `CHECKED_WRITE_XXX` macros. They're calls to ensure that writes succeed. If not, they jump to a fixed exit point.
* The write path is not very optimized. We can use a bottom-up traversal, similar to tree_convert, to make the write less costly. However, our design doesn't require a fast write, so tentatively, I'm going to reuse `tree_copy(..)` to compact a tree.
Depends on D3255048
Test Plan: pass unit tests!
Reviewers: lcharignon, wez
Reviewed By: wez
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3255656
Tasks: 10589048
Signature: t1:3255656:1462342423:c28d32c610b2351614d7648c03f35f931370a770
Summary:
this reduces some of the grovelling around that we're doing
in the full diff structure, and is a smaller amount of data to receive
and process in any case.
Test Plan:
test-diff-since-last-arc-diff.t
```
PYTHONPATH=/data/users/wez/facebook-hg-rpms/fb-hgext /data/users/wez/facebook-hg-rpms/hg-crew/hg --config extensions.publish=/data/users/wez/facebook-hg-rpms/fb-hgext/arcdiff.py --config extensions.errorredirect=! diff --since-last-arc-diff
```
Reviewers: #sourcecontrol, ttung, mitrandir
Reviewed By: mitrandir
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3230136
Signature: t1:3230136:1461810350:54cea1026e86ef33b398ee070eb8bbe7a8667ed6
Summary: We want out phabricator diff parsers to recognize both https://phabricator.intern.facebook.com/ and https://phabricator.fb.com/ URLs (and a bigger class as well).
Test Plan:
- updated some tests (for `phabdiff` and `pullcreatemarkers` other files don't even seem to be used)
- ran tests
- tested phabdiff manually as well:
{F60696023}
Reviewers: #sourcecontrol, andersonmat, mitrandir, simpkins, lcharignon, quark, ttung, ikostia, rmcelroy
Reviewed By: ikostia, rmcelroy
Subscribers: wez, rmcelroy, net-systems-diffs@, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3229776
Tasks: 11013909, 11017978
Signature: t1:3229776:1461839346:08b9b3414e43ff9966bc05591ca5662ef9691aac
Summary:
In addition to being duplicated between these places,
I'd like to re-use this elsewhere.
Test Plan: run-tests continues to pass
Reviewers: #sourcecontrol, ttung, ikostia
Reviewed By: ikostia
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3230128
Tasks: 11013909
Signature: t1:3230128:1461775513:6fc79fda68cd15ded7fb11d52024b5aab56ee880
Summary:
I pretty much stole this from our libfb.py.conduit client, but
removed the python 2.7 and 3 specific aspects of it.
This is an HTTP client for conduit, rather than shelling out to arcanist.
I've added a very simple mechanism for replaying conduit results in the
test harness and used this to build out some tests for the `arcdiff.py`
and `phabstatus.py` extensions.
Test Plan:
```
$ ../../hg-crew/tests/run-tests.py -j8
```
In addition to the new tests, manually tested the actual HTTP functionality:
```
$ /data/users/wez/facebook-hg-rpms/hg-crew/hg --config extensions.phabstatus=/data/users/wez/facebook-hg-rpms/fb-hgext/phabstatus.py --config extensions.errorredirect=! ssl
```
Does not error out and shows the diff status.
Reviewers: #sourcecontrol, ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.fb.com/D3200713
Summary:
This is here to support other arcanist/phabrication integration
modules.
Note: this adds a new package. I'm not sure what the packaging ramifications
are exactly, but surely there are some as other extensions start to depend on
this.
Test Plan:
integration test is provided:
```
$ ../../hg-crew/tests/run-tests.py -j8
....................................................
# Ran 52 tests, 0 skipped, 0 warned, 0 failed.
```
Reviewers: #sourcecontrol, ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.fb.com/D3215615
Summary: We want to be able to see the rebase and amend targets in our smartlog. In order to do so, we need to add correspondin template keywords.
Test Plan:
- wrote a test
- tested manually
Reviewers: #sourcecontrol, durham, ttung
Reviewed By: durham
Subscribers: durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3224240
Tasks: 10950209
Signature: t1:3224240:1461711953:0625e4e1ac8597670d433b14a5b21a2e041b3ef5
Summary:
It should have been done in D3191840, which removes the extension without
dealing with the test.
Test Plan: Code Review.
Reviewers: #sourcecontrol, ttung, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.fb.com/D3199159
Tasks: 10906130
Signature: t1:3199159:1461128102:38e5c093fa8e36d842936f60bcde90f0846f8d82
Blame Revision: D3191840
Summary:
Core changed the quoting convention in some of the warning messages,
this patch makes fb-hgext up to date with these new conventions.
Test Plan: the tests failing before are passing now
Reviewers: durham, quark, mitrandir
Subscribers: mjpieters
Differential Revision: https://phabricator.fb.com/D3198601
Tasks: 10906174
Summary:
Since `handlecommandexception` is in core, wrap it to suppress original
crash text, instead of doing an ugly `sys.exit()`. It's much simpler and
cleaner than before. It also fixes an issue with current chg since chgserver
will print an traceback when it encounters SystemExit.
Although `logtoprocess` are doing very similar things with `errorredirect`,
it currently cannot replace `errorredirect` because we need synchronized
behavior to return the user to their shell prompt correctly. If we can get
the synchronized behavior and a way to disable builtin warning (may be tricky)
in `logtoprocess` upstream, the `errorredirect` extension could be just a thin
wrapper around `logtoprocess` using `setconfig`.
Besides, we no longer need `suppresscommandfailure` since the feature will
be either in `errorredirect` or `logtoprocess`.
Test Plan:
Run `hg crash` with `errorredirect.script` set to confirm it is compatible
with the old behavior.
Also run unittest `test-errorredirect.t`
Reviewers: #sourcecontrol, ttung, mjpieters
Reviewed By: mjpieters
Subscribers: rmcelroy, durham, lcharignon, mjpieters
Differential Revision: https://phabricator.fb.com/D3191840
Signature: t1:3191840:1461088280:77febd539c2f38a55778af3977738c045f63a0d8
Summary:
This change fixes up tests to track graphmod changes.
The change for the smartlog test is a temporary stop-gap measure, as smartlog
itself will be updated soon to track the changes in better ways.
Test Plan:
Run the test-suite, ignore unrelated failures in test-morestatus.t, test-mergedriver.t
and test-fastmanifest.t
Reviewers: #sourcecontrol, ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.fb.com/D3191336
Tasks: 10804976
Summary: It's too common of a name.
Test Plan: compiles && unit tests pass.
Reviewed By: lcharignon
Differential Revision: https://phabricator.fb.com/D3192722
Summary: logtoprocess is now in core mercurial as an experimental extension.
Test Plan: -
Reviewers: #sourcecontrol, ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.fb.com/D3191121
Tasks: 10805001
Summary:
Support both fsmonitor and hgwatchman for now, until fully rolled over to
fsmonitor.
Test Plan:
Run $HG/tests/run-tests.py test-sparse-extensions.t (disable simplecache lines
if not installed to ensure the fsmonitor test at least runs).
Reviewers: #sourcecontrol, ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.fb.com/D3174946
Tasks: 10804941
Summary: This adds end to end proof of concept of manifest caching supporting
different manifest type
Test Plan: Added a test
Reviewers: ttung, durham
Differential Revision: https://phabricator.fb.com/D3129530
Summary:
We got the diagnosis information confirming our assumption but the
test is flaky and outputs different results every time. Let's remove the flaky
part of the test.
Test Plan: The test pass always
Reviewers: durham, rmcelroy, quark
Subscribers: mjpieters
Differential Revision: https://phabricator.fb.com/D3163545
Summary:
The automv extension is now in mercurial core. Both dev servers and laptops
already use the upstream extension.
Test Plan:
run `hg debugextensions --config "extensions.automv=" --debug | grep automv` to
confirm that the extension is loaded from `hgext`.
Reviewers: #sourcecontrol, ttung, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.fb.com/D3162775
Signature: t1:3162775:1460388914:20c5afff1dbf191fde476b6f8d2a210e83d33742
Summary:
It was too easy to check in code that was not passing check-code. I
got tired of fixing it so let's add a linter :)
Test Plan:
Test that the linter works as expected and asks the user to provide
the location of run-tests.py if not set.
Reviewers: #sourcecontrol, ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.fb.com/D3153365
Summary:
Before this patch the test output wasn't great looking, with this patch
one can easily see if our caching strategy worked
Test Plan: This is only a test change
Reviewers: #sourcecontrol, ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.fb.com/D3129319
Tasks: 10604335
Summary: This adds the CPythong interface layer for fastmanifest
Test Plan: Compiles
Reviewers: ttung, durham
Differential Revision: https://phabricator.fb.com/D3157095
Summary:
This patch adds a wrapper that allows us to instantiate either flat
or fast manifest when a manifest is read.
This needs a change in core to work. In manifest.manifest in core we are
currently using two abstractions to create manifests: calling manifestdict()
or _newmanifest(). This was a hack added for the developement of treemanifest
and needs to be changed to make things saner.
Test Plan:
I added a new test. I also ran the whole test suite in core with
the extension, all the tests passed except test-treemanifest.t, failing on the
assert in _flatmanifest as expected.
python run-tests.py -j32 -l
--extra-config-opt=extensions.fastmanifest=/Users/lcharignon/facebook-hg-rpms/fb-hgext/fastmanifest.py
Reviewers: rmcelroy, ttung, durham
Subscribers: mjpieters
Differential Revision: https://phabricator.fb.com/D3115446
Tasks: 10589064
Summary:
The test requires remotenames and hg-new-workdir. Check them before running
the test.
Test Plan:
Run the test and make sure it is skipped if either of the dependencies is
missing
Reviewers: #sourcecontrol, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.fb.com/D3153904
Tasks: 10749152
Signature: t1:3153904:1460072643:b79eb2d161330ef7a6270711590f24b57b1e768f
