Summary: the '.linkrev()' made downloading stuff one by one and took ages if it were not already downloaded (remotefilelog). this fixes the stuff
Test Plan: former tests still pass and testing this manually in fbsource during rebases, the one-by-one downloading stopped as far as I could see
Reviewers: #sourcecontrol, rmcelroy, ttung
Differential Revision: https://phabricator.fb.com/D2730826
Tasks: 8660367
Summary: In order not to spend too much time manually calculating moves, falls back without any user message if too many are not present in the server or local database
Test Plan: tested manually after removing the database, it still prints a message in the blackbox
Reviewers: #sourcecontrol, rmcelroy, ttung
Differential Revision: https://phabricator.fb.com/D2729800
Tasks: 8660367
Summary: Copytracing works fine unless the ancestor is on the p2() branch, in that case fall back to old copytracing without bothering the user with an error
Test Plan: Former tests still pass and merge cases say the same with or without copytrace the extension
Reviewers: #sourcecontrol, rmcelroy, ttung
Differential Revision: https://phabricator.fb.com/D2729604
Tasks: 8660367
Summary: Only looking up at the renames files, instead of all the files in the manifest
Test Plan: former tests still pass and this speeds copytrace on a high range of commits
Reviewers: #sourcecontrol, rmcelroy, ttung
Differential Revision: https://phabricator.fb.com/D2728529
Tasks: 8660367
Summary: template names were not showing up in the documentation
Test Plan:
12/08 11:20 cdelahousse@dev4253 /data/users/cdelahousse/fb-hgext
$ hg help template --config extensions.phabdiff=phabdiff.py --config extensions.phabstatus=phabstatus.py
phabdiff String. Return the phabricator diff id for a given hg rev
phabstatus String. Return the diff approval status for a given hg rev
Reviewers: #sourcecontrol, ttung
Differential Revision: https://phabricator.fb.com/D2735148
Tasks: 9287658
Summary:
Commit 2f72e7519155 in core hg added an extra 'am' action to the merge state.
This broke sparse. This change simply adds it to the list of actions to apply.
Test Plan:
Ran the sparse tests
$ ./run-tests.py -j 40 test-sparse* --with-hg=~/local/hg-repo/hg
....
# Ran 4 tests, 0 skipped, 0 warned, 0 failed.
Where the mercurial's tip is 27ad998ef93f
Reviewers: #sourcecontrol, durham, sid0, ttung
Differential Revision: https://phabricator.fb.com/D2732213
Summary:
In case of crash, instead of printing an ugly stack trace, we want to
paste the message to some pastebin-like places. This extension allows
us to do so.
Test Plan:
1. Add `raise 1` to a random hg command, for example, `hg status`
2. Modify `~/.hgrc`:
```
[errorredirect]
script = (echo HEADER; cat; echo FOOTER) | cat > /tmp/a && echo '/tmp/a written'
```
3. Run `hg status` and confirm the shell script works and no stack trace is printed
Also, run `test-errorredirect.t`
Reviewers: rmcelroy, #sourcecontrol, ttung
Reviewed By: ttung
Differential Revision: https://phabricator.fb.com/D2715697
Tasks: 9259456
Signature: t1:2715697:1449173573:7a5abdb3cf3087128bdf688a596d97bb13f27783
Summary: Queries were too big
Test Plan: Former tests still pass, copytracing doesn't raise an exception on 10000 anymore.
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2719221
Tasks: 8660367
Summary: Using it on 10000 commits makes SQL crashs because the queries are too big. Modifies buildstate to check if the oldest and most recent commits on the public branch, rebased to, are in the database, if yes: Assumes all of the nodes in between are. If not: goes through all the nodes by 1000 to check/retrieve them from the server. To avoid double checking if the hash is present in the database, during a retrievepkg, missing hashes are manually added and never asked to the server (this should mostly never happem)
Test Plan: The former tests still pass and buildstate doesn't raise an exception anymore when rebasing over 10000 commits
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2718840
Tasks: 8660367
Summary: Only add a copy in the dictionary is that file was modified in the other branch or in some weird case useful when rebasing several commits at a time. This avoids unnecessary merges.
Test Plan: Old tests still pass
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2718786
Tasks: 8659945
Summary:
This allows us to target the statprof data at a file of the
wrapper's choosing.
Test Plan: HGPROF=stat STATPROF_DEST=/tmp/statprof.data PYTHONPATH=/Users/tonytung/work/mercurial/facebook-hg-rpms/fb-hgext hg.real --profile status
Reviewers: durham, ericsumner, rmcelroy
Reviewed By: rmcelroy
Subscribers: mitrandir
Differential Revision: https://phabricator.fb.com/D2715997
Signature: t1:2715997:1449105707:c07c83c0889c33cc693836917100d3007d0b631b
Summary:
`hg rage` is useful collecting information suitable for error reports.
Currently it is hardcoded to create a new task. This patch adds an
`--preview` option to print the information without creating tasks.
Test Plan:
1. `hg rage --preview` prints expected information
2. Mock `oncalls`, `arc`, `tasks` and confirm `hg rage` works
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Differential Revision: https://phabricator.fb.com/D2716395
Tasks: 9259456
Signature: t1:2716395:1449110560:efa372f5aab0cf2c586339413f4a0ebf27ad9014
Summary: adding comments
Test Plan: former tests still pass
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2713383
Summary: the option copytrace.pushmvdb allows to use the push command (with nothing to push) to send the whole database to the server and fill it
Test Plan: I tested it manually with an XDB. I am not sure how to automate that see what's present in the XDB
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2713157
Tasks: 8660367
Summary: adds a 'fillmvdb' command to backfill the move db with --start and --stop options
Test Plan: tests with several --start nodes and one --stop revision
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2713126
Tasks: 8659945
Summary: The interactive rebase which is using new histedit base keyword underneath.
Test Plan: see tests
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: durham, rmcelroy
Differential Revision: https://phabricator.fb.com/D2705834
Signature: t1:2705834:1449024840:c29ee246492c5e7c9e595b65ba256e5326ef631d
Summary:
Resolve the revset configured, and use that to insulate revs from pruning.
Built a unit test.
Test Plan: passed unit tests.
Reviewers: rmcelroy, mpm, pyd, durham
Reviewed By: durham
Subscribers: mitrandir, akushner
Differential Revision: https://phabricator.fb.com/D2573162
Signature: t1:2573162:1449076016:9353a80b8801accfe634990285496fab44a66d6e
Summary:
# Intercept manifest.revision(..). The cache is checked, and if it's not there, resume the normal path. Once the normal path is complete, write to the cache.
# Intercept manifest._addrevision(..). This may be used in bulk operations, such as pull. Since we don't want to flood the cache with a lot of entries we may not care about, we record all the nodes that are added. We add an atexit hook to then record the last N nodes to the cache.
# Writes to the cache are done to a temp file, then atomically renamed into place.
# On reads, we run the checkhash function. This costs us ~100ms per manifest in fbsource, but ensures a corrupt cache doesn't break us. Love to debate this matter.
# On each batch of writes, we spawn a background copy of ourselves to prune the cache. We use the mtime of a marker file to determine the last time the prune happened. We calculate the odds that we should be doing the prune using a couple constants and the time since the last prune. If another prune happened recently, the dominant factor is a small probability that we run the prune, regardless of the interval. If no prune has happened recently, the dominant factor is the seconds-since-prune configuration variable.
Some performance numbers (all averaged across 10 runs)
* diff between two revs: 2.29s without caching; 1.75s with caching
* rebasing 4 diffs: 12.1s without caching; 10.7s with caching
Test Plan: passed the rudimentary correctness unit tests. with fbsource, ran a small handful of commands without anybody tripping and falling.
Reviewers: rmcelroy, ericsumner, mpm, pyd, durham
Reviewed By: durham
Subscribers: akushner, mitrandir, cdelahousse
Differential Revision: https://phabricator.fb.com/D2564490
Signature: t1:2564490:1449075868:70974c62e6bff6521b6f500b5bff3a260ddd6c6d
Summary:
Forbit paths starting with / in sparse file and using the command
line arguments. Also fix a typo (util.abort vs util.Abort)
Test Plan: added test
Reviewers: durham
Differential Revision: https://phabricator.fb.com/D2696541
Summary: running the tests through dummy ssh to test bundle2
Test Plan: the old tests pass, the modified one now uses remote
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2705240
Tasks: 8660367
Summary:
Something changed in upstream hg to make mergestate.active blow up without when
not no instantiated with `.read(repo)`
Test Plan:
Ran tests using appropriate hg (release)
11/30 17:35 cdelahousse@dev4253 ~/local/fb-hgext/tests
$ ./run-tests.py test-morestatus.t --with-hg=~/facebook-hg-rpms/hg-crew/hg
.
# Ran 1 tests, 0 skipped, 0 warned, 0 failed.
Reviewers: #sourcecontrol, rmcelroy, ericsumner
Differential Revision: https://phabricator.fb.com/D2706230
Tasks: 9259195
Summary: People complain that `hg status` won't tell them if the repo is in a interrupted update state. This diff adds it.
Test Plan:
1. `cd` to some big project
2. Pick a random old revision from `hg log`, say `fdaa3224d9bd`.
3. `hg update fdaa3224d9bd`
4. Shortly after, in another terminal, `pkill -9 hg`
5. Check `hg status` will print:
```
# The repository is in an unfinished *update* state.
# To continue: hg update fdaa3224d9bd
# To abort: hg update .
```
Also run `run-tests.py test-morestatus.t`.
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: durham, pyd, cdelahousse, lcharignon
Differential Revision: https://phabricator.fb.com/D2691730
Tasks: 9200036
Signature: t1:2691730:1448496639:135542e2698e44a824debd74d6393b98cc74e014
Summary: people have complained that this did not have a sensible default
Test Plan: added a new test
Reviewers: #sourcecontrol, cdelahousse
Reviewed By: cdelahousse
Subscribers: cdelahousse
Differential Revision: https://phabricator.fb.com/D2698778
Tasks: 9175076
Signature: t1:2698778:1448503081:defe2ec4a79b0a899107948472815a54a98d1fff
Summary: Making retrievedatapkg use checkpresence to check if the hashes are in the local database before it tries to remove the data. This avoids having to recurse on the function and having custom "ask to server"/"add data" for that function. Plus, if the node is not public, no data is requested from the server.
Test Plan: The former tests still pass, with one request to the server less for the modified test, since the node is draft.
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2697356
Tasks: 8660367
Summary:
During a rebase, if the data requested by checkpresence is not present in the server, manually add it to the local database to avoid reasking it during retrievedatapkg
This case should mostly never happen though.
Test Plan: The former tests still work. I added one for which the server has no data. It requests the data to the server twice: once for the branch it is rebased to (check presence) and once for the rebased branch (retrievedatapkg) instead of thrice (check presence and retrievedatapkg for the branch it is rebased to)
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2696664
Tasks: 8660367
Summary:
Using the real bundle2 wireproto, 'movedatareq' needs to be added to the allowed getbundle() options
Also some issues due to the fact that the hash may not be present in the remote repo so I send the hash instead of the node
Test Plan: The former tests still pass, and from what I've manually tested, push, pull, rebases, commit, ... work without bug with fbjava_test
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2696452
Tasks: 8660367
Summary:
`os.path.dirname('/')` returns `'/'`, so for an absolute path you end up with
infinite recursion here. Switch to testing for a basename instead (works on Windows with drive letters too). Absolute directories outside of the repository are rejected elsewhere.
Added a testcase that demonstrates absolute paths now work and paths outside the repository are rejected to detect future regressions.
Test Plan:
Run the test suite:
cd tests && python run-tests.py test-sparse*
Reviewers: rmcelroy, durham
Reviewed By: rmcelroy
Differential Revision: https://phabricator.fb.com/D2690822
Tasks: 9156031
Signature: t1:2690822:1448475411:108ac635951cdb9d05329e0b91a9db6c24e8a75f
Summary:
Previously, only commiting/amending a certain file would break the dirsync,
since it would apply the mirror to the working copy but it would not perform the
amend.
The fix is to extend the commit's matcher to include the files that are mirrors
of what is already being included in the commit.
Test Plan: Added tests
Reviewers: #sourcecontrol
Differential Revision: https://phabricator.fb.com/D2696486
Summary:
This diff adds a -x flag to the rebase and histedits commands to only move the
current bookmark. It shows an issue with inhibit that we have to address and
can be extended to other commands.
Test Plan:
Adds a test, shows an issue with the way inhibit handles the
rebase set
Reviewers: pyd, mitrandir, durham
Differential Revision: https://phabricator.fb.com/D2680168
Summary:
The existing upstream heurestic for determining when to accept a delta and when
not to takes into account the total ondisk distance from the chain base to the
current rev. In a large repo with 10+ active branches, its pretty easy to get
into a situation where a few full text entries are written, and suddenly new
entries are deciding they need to be full text too because they are too far from
their root, which in turn causes future entries to also choose full texts,
spiraling into huge revlogs.
The fix is to remove the condition around total ondisk distance. The new
heuristic only takes into account the delta chain size and distance, so it
should make the same choice each time, regardless of the amount of branching
happening or the full text choices of other branches.
Test Plan:
Did an hg pull in a repo that was known to be slow before. It was
veyr fast.
Reviewers: mpm, pyd, #sourcecontrol
Differential Revision: https://phabricator.fb.com/D2693043
Summary: retrieving the mapping of oldhashes/newhashes from pushrebase and applying those changes to the raw move data before the server inserts them in the MySQL database
Test Plan:
the former tests still pass.
I modified manually 'mapping' with pdb and 'insertrawdata' applied the changes correctly.
I will test if it retrieves the data from pushrebase correctly on fbjava_test when this is deployed to the servers
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2689509
Tasks: 8660367
Summary: making the new parts in the bundle non mandatory
Test Plan: the former tests still pass. I pushed to fbjava_test where the part is not handled yet, and the bundle doesn't fail anymore
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2689306
Tasks: 8660367
Summary:
"git branch -m new" renames the *current* branch. Add a shell command to read
the current bookmark when omitted from the git branch -m command line.
Test Plan:
Run tests with:
cd tests && python run-tests.py test-githelp.t
Reviewers: mitrandir, rmcelroy, durham
Reviewed By: rmcelroy, durham
Subscribers: mitrandir
Differential Revision: https://phabricator.fb.com/D2679470
Tasks: 8905356
Signature: t1:2679470:1448062781:d877b60bc2d675efac8d4814f9f89713e1c2630d
Summary: loading copytrace
Test Plan: i built the rpm locally
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2682418
Tasks: 8501037
Summary: returning after exception in bundle2 instead of continuing to run without some args
Test Plan: the former tests still pass. when manually removing getdb.sh, the tests fail gracefully
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2681872
Tasks: 8660367
Summary: When only filldb was enabled, there was an infinite loop, trying to add missing '0' data. so the '0' case was removed and a flag avoiding infinite loop was added
Test Plan: The tests still passes and the loop doesn't appear again during rebases
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2680808
Tasks: 8660367
Summary: Two error modules were imported in bundle2.py and there were some _fail leftovers
Test Plan: the tests still pass
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2680415
Tasks: 8660367
Summary: calculate and add move data when its not in the local or move repo. In theory this should only be called for a few old draft commits, prior to the filldb.py extensions
Test Plan: the old tests still pass and i tested manually a rebase when the local database was erased
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2678603
Tasks: 8659945
Summary: adding the INDEX to the databases. Removed the 'asking server' part of retrieverawdata which is never used since this is to send data to the serve
Test Plan: The former tests still pass
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2678151
Tasks: 8660367
Summary: moved the global remote variable to a repo argument
Test Plan: the former tests still pass
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2677830
Tasks: 8660367
Summary: On exception, pass and print a message asking to report the bug with the metadata.
Test Plan: The former tests still pass and i manually tested the _fail(repo) function
Reviewers: #sourcecontrol, lcharignon, rmcelroy
Subscribers: lcharignon
Differential Revision: https://phabricator.fb.com/D2676065
Tasks: 8659945
