Summary: adds a 'fillmvdb' command to backfill the move db with --start and --stop options
Test Plan: tests with several --start nodes and one --stop revision
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2713126
Tasks: 8659945
Summary: The interactive rebase which is using new histedit base keyword underneath.
Test Plan: see tests
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: durham, rmcelroy
Differential Revision: https://phabricator.fb.com/D2705834
Signature: t1:2705834:1449024840:c29ee246492c5e7c9e595b65ba256e5326ef631d
Summary:
Resolve the revset configured, and use that to insulate revs from pruning.
Built a unit test.
Test Plan: passed unit tests.
Reviewers: rmcelroy, mpm, pyd, durham
Reviewed By: durham
Subscribers: mitrandir, akushner
Differential Revision: https://phabricator.fb.com/D2573162
Signature: t1:2573162:1449076016:9353a80b8801accfe634990285496fab44a66d6e
Summary:
# Intercept manifest.revision(..). The cache is checked, and if it's not there, resume the normal path. Once the normal path is complete, write to the cache.
# Intercept manifest._addrevision(..). This may be used in bulk operations, such as pull. Since we don't want to flood the cache with a lot of entries we may not care about, we record all the nodes that are added. We add an atexit hook to then record the last N nodes to the cache.
# Writes to the cache are done to a temp file, then atomically renamed into place.
# On reads, we run the checkhash function. This costs us ~100ms per manifest in fbsource, but ensures a corrupt cache doesn't break us. Love to debate this matter.
# On each batch of writes, we spawn a background copy of ourselves to prune the cache. We use the mtime of a marker file to determine the last time the prune happened. We calculate the odds that we should be doing the prune using a couple constants and the time since the last prune. If another prune happened recently, the dominant factor is a small probability that we run the prune, regardless of the interval. If no prune has happened recently, the dominant factor is the seconds-since-prune configuration variable.
Some performance numbers (all averaged across 10 runs)
* diff between two revs: 2.29s without caching; 1.75s with caching
* rebasing 4 diffs: 12.1s without caching; 10.7s with caching
Test Plan: passed the rudimentary correctness unit tests. with fbsource, ran a small handful of commands without anybody tripping and falling.
Reviewers: rmcelroy, ericsumner, mpm, pyd, durham
Reviewed By: durham
Subscribers: akushner, mitrandir, cdelahousse
Differential Revision: https://phabricator.fb.com/D2564490
Signature: t1:2564490:1449075868:70974c62e6bff6521b6f500b5bff3a260ddd6c6d
Summary:
Forbit paths starting with / in sparse file and using the command
line arguments. Also fix a typo (util.abort vs util.Abort)
Test Plan: added test
Reviewers: durham
Differential Revision: https://phabricator.fb.com/D2696541
Summary: running the tests through dummy ssh to test bundle2
Test Plan: the old tests pass, the modified one now uses remote
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2705240
Tasks: 8660367
Summary: People complain that `hg status` won't tell them if the repo is in a interrupted update state. This diff adds it.
Test Plan:
1. `cd` to some big project
2. Pick a random old revision from `hg log`, say `fdaa3224d9bd`.
3. `hg update fdaa3224d9bd`
4. Shortly after, in another terminal, `pkill -9 hg`
5. Check `hg status` will print:
```
# The repository is in an unfinished *update* state.
# To continue: hg update fdaa3224d9bd
# To abort: hg update .
```
Also run `run-tests.py test-morestatus.t`.
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: durham, pyd, cdelahousse, lcharignon
Differential Revision: https://phabricator.fb.com/D2691730
Tasks: 9200036
Signature: t1:2691730:1448496639:135542e2698e44a824debd74d6393b98cc74e014
Summary: people have complained that this did not have a sensible default
Test Plan: added a new test
Reviewers: #sourcecontrol, cdelahousse
Reviewed By: cdelahousse
Subscribers: cdelahousse
Differential Revision: https://phabricator.fb.com/D2698778
Tasks: 9175076
Signature: t1:2698778:1448503081:defe2ec4a79b0a899107948472815a54a98d1fff
Summary: Making retrievedatapkg use checkpresence to check if the hashes are in the local database before it tries to remove the data. This avoids having to recurse on the function and having custom "ask to server"/"add data" for that function. Plus, if the node is not public, no data is requested from the server.
Test Plan: The former tests still pass, with one request to the server less for the modified test, since the node is draft.
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2697356
Tasks: 8660367
Summary:
During a rebase, if the data requested by checkpresence is not present in the server, manually add it to the local database to avoid reasking it during retrievedatapkg
This case should mostly never happen though.
Test Plan: The former tests still work. I added one for which the server has no data. It requests the data to the server twice: once for the branch it is rebased to (check presence) and once for the rebased branch (retrievedatapkg) instead of thrice (check presence and retrievedatapkg for the branch it is rebased to)
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2696664
Tasks: 8660367
Summary:
`os.path.dirname('/')` returns `'/'`, so for an absolute path you end up with
infinite recursion here. Switch to testing for a basename instead (works on Windows with drive letters too). Absolute directories outside of the repository are rejected elsewhere.
Added a testcase that demonstrates absolute paths now work and paths outside the repository are rejected to detect future regressions.
Test Plan:
Run the test suite:
cd tests && python run-tests.py test-sparse*
Reviewers: rmcelroy, durham
Reviewed By: rmcelroy
Differential Revision: https://phabricator.fb.com/D2690822
Tasks: 9156031
Signature: t1:2690822:1448475411:108ac635951cdb9d05329e0b91a9db6c24e8a75f
Summary:
Previously, only commiting/amending a certain file would break the dirsync,
since it would apply the mirror to the working copy but it would not perform the
amend.
The fix is to extend the commit's matcher to include the files that are mirrors
of what is already being included in the commit.
Test Plan: Added tests
Reviewers: #sourcecontrol
Differential Revision: https://phabricator.fb.com/D2696486
Summary:
This diff adds a -x flag to the rebase and histedits commands to only move the
current bookmark. It shows an issue with inhibit that we have to address and
can be extended to other commands.
Test Plan:
Adds a test, shows an issue with the way inhibit handles the
rebase set
Reviewers: pyd, mitrandir, durham
Differential Revision: https://phabricator.fb.com/D2680168
Summary:
The existing upstream heurestic for determining when to accept a delta and when
not to takes into account the total ondisk distance from the chain base to the
current rev. In a large repo with 10+ active branches, its pretty easy to get
into a situation where a few full text entries are written, and suddenly new
entries are deciding they need to be full text too because they are too far from
their root, which in turn causes future entries to also choose full texts,
spiraling into huge revlogs.
The fix is to remove the condition around total ondisk distance. The new
heuristic only takes into account the delta chain size and distance, so it
should make the same choice each time, regardless of the amount of branching
happening or the full text choices of other branches.
Test Plan:
Did an hg pull in a repo that was known to be slow before. It was
veyr fast.
Reviewers: mpm, pyd, #sourcecontrol
Differential Revision: https://phabricator.fb.com/D2693043
Summary:
"git branch -m new" renames the *current* branch. Add a shell command to read
the current bookmark when omitted from the git branch -m command line.
Test Plan:
Run tests with:
cd tests && python run-tests.py test-githelp.t
Reviewers: mitrandir, rmcelroy, durham
Reviewed By: rmcelroy, durham
Subscribers: mitrandir
Differential Revision: https://phabricator.fb.com/D2679470
Tasks: 8905356
Signature: t1:2679470:1448062781:d877b60bc2d675efac8d4814f9f89713e1c2630d
Summary: calculate and add move data when its not in the local or move repo. In theory this should only be called for a few old draft commits, prior to the filldb.py extensions
Test Plan: the old tests still pass and i tested manually a rebase when the local database was erased
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2678603
Tasks: 8659945
Summary: adding ui config to allow deploying first the 'filling the client database' then 'pushing to and pushing from the server database' and finally the 'copytracing' part
Test Plan: adding those ui configs, the former tests still pass
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2672217
Tasks: 8660367
Summary: moving remote to a ui config. servers are True, clients are False.
Test Plan: the former tests still pass
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2671855
Tasks: 8660367
Summary: Added all the sql commands for the MySQL remote database, and gathered all the commands in a single function
Test Plan: Created a serverrepo, two client ones, did push and pulls between them
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Differential Revision: https://phabricator.fb.com/D2661606
Tasks: 8660367
Signature: t1:2661606:1447875185:0e9d3876b2f509028585349a9d05b447876b3da0
Summary:
When a sparse profile is not found (perhaps because the user used an incomplete
or misspelled path), warn the user that a profile is not found.
There was already a debug message for this, elevate it to a warning.
Test Plan:
Run the tests:
cd tests
python run-tests.py test-sparse*.t
Manual test:
hg init test
cd test
cat > .hg/hgrc <<EOF
[extensions]
sparse=$(dirname $TESTDIR)/sparse.py
EOF
touch testfile
hg add testfile
hg ci -m 'initial commit'
hg sparse --enable-profile foo/bar
hg sparse --disable-profile foo/bar
and the following warning is printed twice:
warning: sparse profile 'foo/bar' not found in rev <hash> - ignoring it
Disabling the warning with:
cat >> .hg/hgrc <<EOF
[sparse]
missingwarning=false
EOF
hg sparse --enable-profile foo/bar
hg sparse --disable-profile foo/bar
Reviewers: rmcelroy, durham
Reviewed By: durham
Differential Revision: https://phabricator.fb.com/D2668406
Tasks: 8994281
Signature: t1:2668406:1447948981:66d16b94ea7ea682b9e0c3faaccc1a364283364d
Summary: centralizing the sqlite commands in a function - adding a 'repo' column to the sqlite database so that it is similar to the future XDB
Test Plan: the former tests still pass
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2665931
Tasks: 8660367
Summary:
replacements for the newly-picked node were not present in the
histedit state before successfully continuing which made --abort
unaware of them.
This isn't clean solution. But the stop command (with node) will cease to
exist in near future (it will be replaced by the normal pick and just "stop"
in following line for which we will have better solution). This should suffice
until then.
Test Plan:
test attached
tested manually too
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Differential Revision: https://phabricator.fb.com/D2661831
Tasks: 9111207
Signature: t1:2661831:1447790670:a26ee58cacb77e6cf675a1e99d2eadcbffb684d9
The no-date-rewrite-on-rebase test had output that assumed date rewriting. It
should be outputting the first commit which has a recent date (commit 7) instead
of the current commit (commit 12) which did not have it's date rewritten.
Summary:
There was a bug where if you had a sparse profile and updated across revs that
both A) add file X to the sparse profile, and B) changed the permissions on file
X (ex: to make it executable), the update would fail.
This was caused by the edit permission action being present in the action list
as an 'edit' action, so the profile changing code did not add the 'get' action
that would actually put it on disk.
The fixes is to just add the 'get' action for every file that wasn't previously
in the profile (overwriting any action that had already been computed).
Test Plan: Added a test, and verified it against the reported repro case
Reviewers: #sourcecontrol
Differential Revision: https://phabricator.fb.com/D2652793
Summary: the move data concerning the ctx is pushed to the remote sqlite database at the same time as hg push
Test Plan: testing a push between a client and server repo
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2643067
Tasks: 8660367
Summary: If some contexts are not present in the local database during a retrieve, a request is send to the server for those moves one time through a 'pull(revs)' call. When doing a rebase, a check is performed beforehand to see if the local database contains the contexts of the rebased-to branch.
Test Plan: Rebasing in the middle of a branch for which the moves are not known
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2634365
Tasks: 8660367
Summary: Changed the format under which the move dictionaries are sent by bundle2 and sorted the sql test results so that it is constant
Test Plan: The former tests (fill-db.t, copytrace.t and bundle2.t) still pass
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2634262
Tasks: 8660367
Summary:
Previously rebase retained the commit date from the original. Now it uses current time.
Added config option 'tweakdefaults.rebasekeepdate' to save original date in graft.
Test Plan: ./run-tests.py test-tweakdefaults.t
Reviewers: rmcelroy
Reviewed By: rmcelroy
Differential Revision: https://phabricator.fb.com/D2646736
Tasks: 8770194
Signature: t1:2646736:1447368541:e9735f75b394a25b72d1a5cc4cd26454c2231c91
Summary:
If tracking was set up, 'hg push' was not invoking the pushrebase logic, like it
would if the user ran 'hg push --to master'.
This is required for us to unify all pushes through pushrebase, so we can move
the hooks out of the lock and into prepushrebase.
Test Plan: Added a test
Reviewers: #sourcecontrol
Differential Revision: https://phabricator.fb.com/D2635253
Summary:
Requiring that developers use the run-tests from the core hg repo has
some downsides:
1) It forces the tests to run against whatever version of Mercurial I have
checked out right now. I usually want to run tests against my installed
Mercurial (since we're likely to deploy a new fb-hgext without a new hg).
2) It makes it harder for people to contribute. Someone wrote a new extension,
and in order for them to test it, I have to explain how to checkout hg and how
to run tests using the hg run-tests and first they must checkout the rev we
have deployed, not @.
Test Plan: Ran the tests
Reviewers: pyd, cdelahousse, #sourcecontrol
Subscribers: ps
Differential Revision: https://phabricator.fb.com/D2632574
Summary:
The branch cache is used to store a cache of each branches heads. If your
Mercurial repo doesn't have any branches though, this can be slow.
This tweak allows simplifying the branchcache computation to just say every head
in the repo belongs to the 'default' branch. This shaves 600ms off of some
commit commands.
I tested it by running the entire Mercurial test suite with it enabled, and
scanning the failures to make sure they were only related to repos that had
non-default branches.
Test Plan:
$ cd ~/hg/tests
$ ./run-tests.py -S -j 48 --extra-config-opt=perftweaks.disablebranchcache=True --extra-config-opt=extensions.perftweaks=/data/users/durham/fb-hgext/perftweaks.py
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy
Differential Revision: https://phabricator.fb.com/D2630360
Signature: t1:2630360:1447092762:adab0456ff842cf508b7f46e988ef9d865fa3988
Summary: Uses the local sqlite database to check for moves during the rebase - Wraps and copies the mergecopies function
Test Plan: Tests the different cases: renamed/renamed renamed/modified renamed/deleted deleted/renamed modified/renamed
Reviewers: #sourcecontrol, rmcelroy
Differential Revision: https://phabricator.fb.com/D2552629
Tasks: 8659945
Summary:
This adds an extension that allows configuring the repository such that any
commit that touches a file in directory foo/ will also have the change applied
to directory bar/. It will also block commits if they try to make divergent
changes to mirrored directories.
Test Plan: Added a test suite
Reviewers: #sourcecontrol
Subscribers: chip, aponomarenko
Differential Revision: https://phabricator.fb.com/D2602592
Summary:
The previous implementation relied on the python serialization of lists. This
resulted in some truncation due to line length in some infrastructure.
This patch makes it prettier.
Test Plan: Tests
Reviewers: #sourcecontrol, mitrandir
Reviewed By: mitrandir
Subscribers: mitrandir
Differential Revision: https://phabricator.fb.com/D2594066
Signature: t1:2594066:1446079418:cd27cea1f44a9d432984b8dc37b547450ff076ab
Summary:
Previously the output just said "searching for changes" then "updating
bookmark", which was kind of terse. It also showed the "adding XX changesets"
output without any context, which is kind of scary to see "adding 100
changesets" when I'm only pushing one commit.
Now we mention how many commits we're pushing, what the hash range is, and how
many commits are being downloaded.
Test Plan: Ran ze tests
Reviewers: #sourcecontrol, mitrandir
Reviewed By: mitrandir
Subscribers: mitrandir
Differential Revision: https://phabricator.fb.com/D2594007
Signature: t1:2594007:1446081446:99b77f4982acafc1f1940ba07b094a27dd05e354
Summary:
This adds an option for blocking non-pushrebase pushes. This is important
because it allows us to force every push through the pushrebase logic and
therefore allows us to put all the hooks in the prepushrebase hook (so people
can't avoid those hooks).
This is important for performance, since running the hooks in prepushrebase
allows us to run them while outside the lock.
Test Plan: Added a test
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy
Differential Revision: https://phabricator.fb.com/D2593512
Signature: t1:2593512:1446076882:9f5ee5d8bccd884f34c8284fbacb7d6f8e96f62f
Summary:
Since pushrebase allows running hooks outside of the lock, we want to force all
pushes to go through pushrebase. Step #1 of that is making push --force go
through the pushrebase flow.
The initial version is pretty naive. A future version might avoid the whole
graft process for performance sake.
Test Plan:
Added a test that creates a new bookmark. I also tested the same test
doing a force push.
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy
Differential Revision: https://phabricator.fb.com/D2588894
Signature: t1:2588894:1446076583:12b58048742b112a5d7cab839204c6219414f9ee
