Summary: adds a 'fillmvdb' command to backfill the move db with --start and --stop options
Test Plan: tests with several --start nodes and one --stop revision
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2713126
Tasks: 8659945
Summary: The interactive rebase which is using new histedit base keyword underneath.
Test Plan: see tests
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: durham, rmcelroy
Differential Revision: https://phabricator.fb.com/D2705834
Signature: t1:2705834:1449024840:c29ee246492c5e7c9e595b65ba256e5326ef631d
Summary:
Resolve the revset configured, and use that to insulate revs from pruning.
Built a unit test.
Test Plan: passed unit tests.
Reviewers: rmcelroy, mpm, pyd, durham
Reviewed By: durham
Subscribers: mitrandir, akushner
Differential Revision: https://phabricator.fb.com/D2573162
Signature: t1:2573162:1449076016:9353a80b8801accfe634990285496fab44a66d6e
Summary:
# Intercept manifest.revision(..). The cache is checked, and if it's not there, resume the normal path. Once the normal path is complete, write to the cache.
# Intercept manifest._addrevision(..). This may be used in bulk operations, such as pull. Since we don't want to flood the cache with a lot of entries we may not care about, we record all the nodes that are added. We add an atexit hook to then record the last N nodes to the cache.
# Writes to the cache are done to a temp file, then atomically renamed into place.
# On reads, we run the checkhash function. This costs us ~100ms per manifest in fbsource, but ensures a corrupt cache doesn't break us. Love to debate this matter.
# On each batch of writes, we spawn a background copy of ourselves to prune the cache. We use the mtime of a marker file to determine the last time the prune happened. We calculate the odds that we should be doing the prune using a couple constants and the time since the last prune. If another prune happened recently, the dominant factor is a small probability that we run the prune, regardless of the interval. If no prune has happened recently, the dominant factor is the seconds-since-prune configuration variable.
Some performance numbers (all averaged across 10 runs)
* diff between two revs: 2.29s without caching; 1.75s with caching
* rebasing 4 diffs: 12.1s without caching; 10.7s with caching
Test Plan: passed the rudimentary correctness unit tests. with fbsource, ran a small handful of commands without anybody tripping and falling.
Reviewers: rmcelroy, ericsumner, mpm, pyd, durham
Reviewed By: durham
Subscribers: akushner, mitrandir, cdelahousse
Differential Revision: https://phabricator.fb.com/D2564490
Signature: t1:2564490:1449075868:70974c62e6bff6521b6f500b5bff3a260ddd6c6d
Summary:
Forbit paths starting with / in sparse file and using the command
line arguments. Also fix a typo (util.abort vs util.Abort)
Test Plan: added test
Reviewers: durham
Differential Revision: https://phabricator.fb.com/D2696541
Summary: running the tests through dummy ssh to test bundle2
Test Plan: the old tests pass, the modified one now uses remote
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2705240
Tasks: 8660367
Summary:
Something changed in upstream hg to make mergestate.active blow up without when
not no instantiated with `.read(repo)`
Test Plan:
Ran tests using appropriate hg (release)
11/30 17:35 cdelahousse@dev4253 ~/local/fb-hgext/tests
$ ./run-tests.py test-morestatus.t --with-hg=~/facebook-hg-rpms/hg-crew/hg
.
# Ran 1 tests, 0 skipped, 0 warned, 0 failed.
Reviewers: #sourcecontrol, rmcelroy, ericsumner
Differential Revision: https://phabricator.fb.com/D2706230
Tasks: 9259195
Summary: People complain that `hg status` won't tell them if the repo is in a interrupted update state. This diff adds it.
Test Plan:
1. `cd` to some big project
2. Pick a random old revision from `hg log`, say `fdaa3224d9bd`.
3. `hg update fdaa3224d9bd`
4. Shortly after, in another terminal, `pkill -9 hg`
5. Check `hg status` will print:
```
# The repository is in an unfinished *update* state.
# To continue: hg update fdaa3224d9bd
# To abort: hg update .
```
Also run `run-tests.py test-morestatus.t`.
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: durham, pyd, cdelahousse, lcharignon
Differential Revision: https://phabricator.fb.com/D2691730
Tasks: 9200036
Signature: t1:2691730:1448496639:135542e2698e44a824debd74d6393b98cc74e014
Summary: people have complained that this did not have a sensible default
Test Plan: added a new test
Reviewers: #sourcecontrol, cdelahousse
Reviewed By: cdelahousse
Subscribers: cdelahousse
Differential Revision: https://phabricator.fb.com/D2698778
Tasks: 9175076
Signature: t1:2698778:1448503081:defe2ec4a79b0a899107948472815a54a98d1fff
Summary: Making retrievedatapkg use checkpresence to check if the hashes are in the local database before it tries to remove the data. This avoids having to recurse on the function and having custom "ask to server"/"add data" for that function. Plus, if the node is not public, no data is requested from the server.
Test Plan: The former tests still pass, with one request to the server less for the modified test, since the node is draft.
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2697356
Tasks: 8660367
Summary:
During a rebase, if the data requested by checkpresence is not present in the server, manually add it to the local database to avoid reasking it during retrievedatapkg
This case should mostly never happen though.
Test Plan: The former tests still work. I added one for which the server has no data. It requests the data to the server twice: once for the branch it is rebased to (check presence) and once for the rebased branch (retrievedatapkg) instead of thrice (check presence and retrievedatapkg for the branch it is rebased to)
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2696664
Tasks: 8660367
Summary:
Using the real bundle2 wireproto, 'movedatareq' needs to be added to the allowed getbundle() options
Also some issues due to the fact that the hash may not be present in the remote repo so I send the hash instead of the node
Test Plan: The former tests still pass, and from what I've manually tested, push, pull, rebases, commit, ... work without bug with fbjava_test
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2696452
Tasks: 8660367
Summary:
`os.path.dirname('/')` returns `'/'`, so for an absolute path you end up with
infinite recursion here. Switch to testing for a basename instead (works on Windows with drive letters too). Absolute directories outside of the repository are rejected elsewhere.
Added a testcase that demonstrates absolute paths now work and paths outside the repository are rejected to detect future regressions.
Test Plan:
Run the test suite:
cd tests && python run-tests.py test-sparse*
Reviewers: rmcelroy, durham
Reviewed By: rmcelroy
Differential Revision: https://phabricator.fb.com/D2690822
Tasks: 9156031
Signature: t1:2690822:1448475411:108ac635951cdb9d05329e0b91a9db6c24e8a75f
Summary:
Previously, only commiting/amending a certain file would break the dirsync,
since it would apply the mirror to the working copy but it would not perform the
amend.
The fix is to extend the commit's matcher to include the files that are mirrors
of what is already being included in the commit.
Test Plan: Added tests
Reviewers: #sourcecontrol
Differential Revision: https://phabricator.fb.com/D2696486
Summary:
This diff adds a -x flag to the rebase and histedits commands to only move the
current bookmark. It shows an issue with inhibit that we have to address and
can be extended to other commands.
Test Plan:
Adds a test, shows an issue with the way inhibit handles the
rebase set
Reviewers: pyd, mitrandir, durham
Differential Revision: https://phabricator.fb.com/D2680168
Summary:
The existing upstream heurestic for determining when to accept a delta and when
not to takes into account the total ondisk distance from the chain base to the
current rev. In a large repo with 10+ active branches, its pretty easy to get
into a situation where a few full text entries are written, and suddenly new
entries are deciding they need to be full text too because they are too far from
their root, which in turn causes future entries to also choose full texts,
spiraling into huge revlogs.
The fix is to remove the condition around total ondisk distance. The new
heuristic only takes into account the delta chain size and distance, so it
should make the same choice each time, regardless of the amount of branching
happening or the full text choices of other branches.
Test Plan:
Did an hg pull in a repo that was known to be slow before. It was
veyr fast.
Reviewers: mpm, pyd, #sourcecontrol
Differential Revision: https://phabricator.fb.com/D2693043
Summary: retrieving the mapping of oldhashes/newhashes from pushrebase and applying those changes to the raw move data before the server inserts them in the MySQL database
Test Plan:
the former tests still pass.
I modified manually 'mapping' with pdb and 'insertrawdata' applied the changes correctly.
I will test if it retrieves the data from pushrebase correctly on fbjava_test when this is deployed to the servers
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2689509
Tasks: 8660367
Summary: making the new parts in the bundle non mandatory
Test Plan: the former tests still pass. I pushed to fbjava_test where the part is not handled yet, and the bundle doesn't fail anymore
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2689306
Tasks: 8660367
Summary:
"git branch -m new" renames the *current* branch. Add a shell command to read
the current bookmark when omitted from the git branch -m command line.
Test Plan:
Run tests with:
cd tests && python run-tests.py test-githelp.t
Reviewers: mitrandir, rmcelroy, durham
Reviewed By: rmcelroy, durham
Subscribers: mitrandir
Differential Revision: https://phabricator.fb.com/D2679470
Tasks: 8905356
Signature: t1:2679470:1448062781:d877b60bc2d675efac8d4814f9f89713e1c2630d
Summary: loading copytrace
Test Plan: i built the rpm locally
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2682418
Tasks: 8501037
Summary: returning after exception in bundle2 instead of continuing to run without some args
Test Plan: the former tests still pass. when manually removing getdb.sh, the tests fail gracefully
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2681872
Tasks: 8660367
Summary: When only filldb was enabled, there was an infinite loop, trying to add missing '0' data. so the '0' case was removed and a flag avoiding infinite loop was added
Test Plan: The tests still passes and the loop doesn't appear again during rebases
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2680808
Tasks: 8660367
Summary: Two error modules were imported in bundle2.py and there were some _fail leftovers
Test Plan: the tests still pass
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2680415
Tasks: 8660367
Summary: calculate and add move data when its not in the local or move repo. In theory this should only be called for a few old draft commits, prior to the filldb.py extensions
Test Plan: the old tests still pass and i tested manually a rebase when the local database was erased
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2678603
Tasks: 8659945
Summary: adding the INDEX to the databases. Removed the 'asking server' part of retrieverawdata which is never used since this is to send data to the serve
Test Plan: The former tests still pass
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2678151
Tasks: 8660367
Summary: moved the global remote variable to a repo argument
Test Plan: the former tests still pass
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2677830
Tasks: 8660367
Summary: On exception, pass and print a message asking to report the bug with the metadata.
Test Plan: The former tests still pass and i manually tested the _fail(repo) function
Reviewers: #sourcecontrol, lcharignon, rmcelroy
Subscribers: lcharignon
Differential Revision: https://phabricator.fb.com/D2676065
Tasks: 8659945
Summary: adding ui config to allow deploying first the 'filling the client database' then 'pushing to and pushing from the server database' and finally the 'copytracing' part
Test Plan: adding those ui configs, the former tests still pass
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2672217
Tasks: 8660367
Summary: moving remote to a ui config. servers are True, clients are False.
Test Plan: the former tests still pass
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2671855
Tasks: 8660367
Summary: Added all the sql commands for the MySQL remote database, and gathered all the commands in a single function
Test Plan: Created a serverrepo, two client ones, did push and pulls between them
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Differential Revision: https://phabricator.fb.com/D2661606
Tasks: 8660367
Signature: t1:2661606:1447875185:0e9d3876b2f509028585349a9d05b447876b3da0
Summary:
When a sparse profile is not found (perhaps because the user used an incomplete
or misspelled path), warn the user that a profile is not found.
There was already a debug message for this, elevate it to a warning.
Test Plan:
Run the tests:
cd tests
python run-tests.py test-sparse*.t
Manual test:
hg init test
cd test
cat > .hg/hgrc <<EOF
[extensions]
sparse=$(dirname $TESTDIR)/sparse.py
EOF
touch testfile
hg add testfile
hg ci -m 'initial commit'
hg sparse --enable-profile foo/bar
hg sparse --disable-profile foo/bar
and the following warning is printed twice:
warning: sparse profile 'foo/bar' not found in rev <hash> - ignoring it
Disabling the warning with:
cat >> .hg/hgrc <<EOF
[sparse]
missingwarning=false
EOF
hg sparse --enable-profile foo/bar
hg sparse --disable-profile foo/bar
Reviewers: rmcelroy, durham
Reviewed By: durham
Differential Revision: https://phabricator.fb.com/D2668406
Tasks: 8994281
Signature: t1:2668406:1447948981:66d16b94ea7ea682b9e0c3faaccc1a364283364d
Summary: centralizing the sqlite commands in a function - adding a 'repo' column to the sqlite database so that it is similar to the future XDB
Test Plan: the former tests still pass
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2665931
Tasks: 8660367
Summary: Part of the delay in the hg ssl call was because phabstatus was actually making the network call for each time it was called in the template condition, and we can cache the results
Test Plan: Ran hg ssl, so much faster
Reviewers: durham, rmcelroy
Differential Revision: https://phabricator.fb.com/D2652050
Summary:
Remove the need to ommit the rREPO prefix when copy pasting from phabricator and
looking up a commit.
Test Plan:
Ran the following in phabricator enabled HG repos
11/13 12:15 cdelahousse@dev4253 ~/fbsource/fbcode
$ hg log -r rFBS27aa00fb74d9a3b82756dad6ff26fe253f1e9a70 --config extensions.fbconduit=~/local/fb-hgext/fbconduit.py
changeset: 992495:27aa00fb74d9
user: Peng Li <pengli@fb.com>
date: Tue Nov 03 10:29:01 2015 -0800
summary: Add a simple root dir arc library
11/13 12:21 cdelahousse@dev4253 ~/fbjava
$ hg log -r rFBA8f1335e6d588 --config extensions.fbconduit=~/local/fb-hgext/fbconduit.py
changeset: 18654:8f1335e6d588
user: Ryan Menezes <ryandm@fb.com>
date: Thu Oct 22 09:59:27 2015 -0700
summary: move jenkins hook into fbjava/arcanist (fbjava changes)
Tried the same on a repo with a git mirror that exists in phabricator:
11/16 11:40 cdelahousse@dev4253 ~/fbsource/fbcode
$ hg log -r rFBCODE8272d25d65869ce059024ff38c7051388ad7b802 --config extensions.fbconduit=~/local/fb-hgext/fbconduit.py
changeset: 1027287:f26a9c32ae08
user: Oleksandr Kuvshynov <oleksandr@fb.com>
date: Sun Nov 15 19:16:39 2015 -0800
summary: [mf] simple cleanup of feed story view
If a git hash is too small, abort:
$ hg log -r rFBCODE8272d25d65869ce059024f --config extensions.fbconduit=~/local/fb-hgext/fbconduit.py
abort: git hash must be 40 characters
The previous commands depends on a list of repos set in my hgrc. See (D2660070)
[fbconduit]
gitcallsigns=LK, CFPUMA, CFSV, CFGK, CFMF, CFGMON, CF, RSIGMA, FA,·
WAWEBCLIENT, SKY, FBCODE, FBOBJC, WAWP, SV, OVRMOBILEMAIN, WAANDROID, IGSRV,
CPE, MSS, ANDROIDSDK, BUCK, IT-CHEF, WABB, ITINFRA, SIGMA, IOSSDK
Reviewers: #sourcecontrol, durham, ericsumner
Reviewed By: durham, ericsumner
Subscribers: rmcelroy, ericsumner
Differential Revision: https://phabricator.fb.com/D2653497
Tasks: 8361368
Signature: t1:2653497:1447480768:295079a7793e182ddea3aeece9cfaead1bfc1a57
Summary:
replacements for the newly-picked node were not present in the
histedit state before successfully continuing which made --abort
unaware of them.
This isn't clean solution. But the stop command (with node) will cease to
exist in near future (it will be replaced by the normal pick and just "stop"
in following line for which we will have better solution). This should suffice
until then.
Test Plan:
test attached
tested manually too
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Differential Revision: https://phabricator.fb.com/D2661831
Tasks: 9111207
Signature: t1:2661831:1447790670:a26ee58cacb77e6cf675a1e99d2eadcbffb684d9
Summary: Instead of retrieving moves one by one during copytracing, retrieves all the moves from the ctxstack at a time
Test Plan: The former tests still work
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2661088
Tasks: 8660367
Summary: The regular expression was matching hg hashes that started with a digit.
Test Plan:
Ran it in different repoes:
This should fail (fbcode)
11/13 11:15 cdelahousse@dev4253 ~/fbsource/fbcode
$ hg up rFBCODE8a781801149 --config extensions.fbconduit=~/local/fb-hgext/fbconduit.py
abort: unknown revision 'rFBCODE8a781801149'!
These should succeed (WWW and opsfiles)
11/13 11:16 cdelahousse@dev4253 ~/www
$ hg log -r rE1502460 --config
extensions.fbconduit=~/local/fb-hgext/fbconduit.py
changeset: 808001:34d7f02c58d6
user: cdelahousse@2c7ba8d8-a2f7-0310-a573-de162e16dcc7
date: Thu Nov 20 04:45:21 2014 +0000
summary: CRMEmailHarvester Script runner for raw email processor
11/13 11:29 cdelahousse@dev4253 ~/opsfiles
$ hg log -r rO766855 --config
extensions.fbconduit=~/local/fb-hgext/fbconduit.py
changeset: 758197:db438a952785
user: twsvcscm@914ed892-1039-11dd-9db8-c56a9f3689f4
date: Sat Oct 31 01:13:33 2015 +0000
summary: [Storage] netgroup_gen AUTOCOMMIT
Reviewers: #sourcecontrol, rmcelroy
Subscribers: cdown
Differential Revision: https://phabricator.fb.com/D2653121
Tasks: 9071883
The no-date-rewrite-on-rebase test had output that assumed date rewriting. It
should be outputting the first commit which has a recent date (commit 7) instead
of the current commit (commit 12) which did not have it's date rewritten.
