Summary:
It'd be nice to have dirsync config in the repo, since it's better synced, and
easier to be modified.
Test Plan: Added a test case
Reviewers: #sourcecontrol, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4732054
Signature: t1:4732054:1489793650:ca6c63ef47a1d9ebd181fbaee5554975801324eb
Summary:
Upstream converted all array.array usages to bytearray, and in doing so they
also changed mdiff.textdiff(array1, array2) to be
mdiff.textdiff(util.buffer(array1), util.buffer(array2)), which we did not do in
our internal implementation.
This fixes that.
Test Plan: Adds a test
Reviewers: #mercurial, quark, simonfar
Reviewed By: quark, simonfar
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4724350
Signature: t1:4724350:1489697253:8f63350bfc30b3079bf38410054b4763942a82f6
Summary:
On macos sshpeer flushes remote output later and it causes tests to fail. Let's
add a call to cleanup() to make sure the output is flushed.
This diff also makes remotefilelog test more robust.
Test Plan: Run tests on macos
Reviewers: #mercurial, tja, mitrandir
Reviewed By: mitrandir
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4721276
Tasks: 15389402
Signature: t1:4721276:1489669109:4ce59f4a1d224d57dbb7c1eb341c4e6a659d2e8c
Summary:
By default debugcheckbackup checks only one backup for the user.
With --all option it checks all backups for the user from all host/reporoots
Test Plan: arc unit
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4700993
Tasks: 15389402
Signature: t1:4700993:1489548921:800d08f420acc8ef4f807ffd17b31837dbb3fe82
Summary:
Backing up very big repo can cause problems. And very often old commits are
not necessary at all. Let's add a config option to limit the number of heads
that are being backed up.
Test Plan: arc unit
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4700965
Tasks: 15389402
Signature: t1:4700965:1489549163:4e2c121f01debd7b495486a1f3b062926873399d
Summary:
Previously we saved bookmark hash and revision number as state of
the last backup. Storing last backed up revision is error-prone. I saw
a few corrupted backups where bookmarks pointed to non-backed up
nodes. Also this approach puts more pressure on mysql bundle index.
On every backup it first deletes all existing backup bookmarks and then it
writes new bookmarks. Even if user changed one head or one bookmark
(and that is the usual case) current approach still deletes bookmarks first
and then rewrites them.
Instead let's store all backed up heads and bookmarks locally in json format.
Json was chosen because it's simple and human-readable. Then during backup
let's compare last backed up state to the current state of the repo, and send
to the server only the difference.
Test Plan: arc unit
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4700930
Tasks: 15389402
Signature: t1:4700930:1489613249:a34369bf53e0718c22258304493dfa27b578857f
Summary:
We have encountered a kernel issue where `rmdir` a non-empty directory may race
with other things and hang in kernel for a long time.
This patch changes `os.rmdir` to avoid `rmdir` non-empty directories. It is
written in Cython calling the low-level `readdir` libc friends to make overhead
minimal.
Test Plan: Added a new test
Reviewers: #sourcecontrol, clm, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, simpkins, osandov, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4716711
Tasks: 16647532
Signature: t1:4716711:1489627923:7c7432748c1fd8c070ce257bd172feebd3807f65
test-check-code doesn't like sed -i, so let's switch to a tiny batch script.
Also fix the test importing the extension directly instead of relying on the
path to be right.
Summary:
It's just copy-paste from my mutable-history patch series that ended up as
persistent hotfix.
The wrapping logic is very simple: it redirects '--fold' calls to original
metaedit and handles all the rest.
Test Plan:
Included a big copy-pase from test-evolve.t preserving only cases that were
executing metaedit and modified for the smaller test repo (the metaedit tests
were at the end of evolve test file).
Reviewers: #sourcecontrol, quark, simonfar
Reviewed By: quark, simonfar
Subscribers: rmcelroy, quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4708017
Signature: t1:4708017:1489524494:d682aab857e8422bb1a23adc22dddfb9f505b6b3
Summary: New logging will have rotation and will separate different users and repos
Test Plan: arc unit
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4689191
Tasks: 16474976
Signature: t1:4689191:1489172091:d304d4b19ae6cf52f86c11f442fa760b50d1fdf9
Summary:
For now these command will be used in test because
`wait_for_background_backup.py` tied to the backup state file. And every change
to this file requires change to wait_for_background_backup.py. Let's create
simple internal backup command.
Test Plan: arc unit
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4667056
Tasks: 15389402
Signature: t1:4667056:1489024344:6c43241e42b7c418baa4e1542f303239ba887c45
Summary:
Upstream has added a matcher arg to manifest.filesnotin, so let's support it in
our native treemanifest implementation.
Test Plan: Added unit tests for filesnotin with and without matchers.
Reviewers: #mercurial, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4677030
Signature: t1:4677030:1489076599:23ab67d85bb4fa689f05349253a4d2c4e7d496cc
Summary:
Upstream has added a matcher argument to the diff API which allows diff to avoid
traversing certain parts of the tree. This adds support for that to our native
treemanifest implementation.
Test Plan: Added tests for diff with matches
Reviewers: #mercurial, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4677023
Signature: t1:4677023:1489076627:dbcea209d300a68fa050f68c52b4fd9949b85302
Summary:
Upstream has added a new match argument to manifest.diff() and removed the
existing manifest.matches() function, so we need to update our internal usage.
A separate diff will update treemanifest to support the new diff() api.
Test Plan:
Ran the tests, some still fail because of the upstream changes, but
future patches fix those.
Reviewers: #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D4677002
Summary: This diff adds a `--towards` flag to `hg next`. This flag tells `hg next` to always proceed in a linear fashion towards the specified commit. This is useful in situations where the user has a very branchy stack. For example, if the user has a local stack with several branches, each with a feature bookmark at the top, if the user wants to go up a particular stack they can do so with something like `hg next 3 --towards bookmark`.
Test Plan:
Here is an example of basic usage. Also see test file.
```
$ hg debugbuilddag "+5 *2 +2"
$ hg book -r 4 feature1
$ hg book -r 7 feature2
$ hg up 0
$ hg sl
o 92eaf3 debugbuilddag feature2
| r7
|
o 8bbe84 debugbuilddag
| r6
|
o 914970 debugbuilddag
| r5
|
| o bebd16 debugbuilddag feature1
| | r4
| |
| o 2dc09a debugbuilddag
| | r3
| |
| o 012414 debugbuilddag
|/ r2
|
o 66f7d4 debugbuilddag
| r1
|
@ 1ea734 debugbuilddag
r0
$ hg next 3 --towards feature1
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
[2dc09a] r3
$ hg sl
o 92eaf3 debugbuilddag feature2
| r7
|
o 8bbe84 debugbuilddag
| r6
|
o 914970 debugbuilddag
| r5
|
| o bebd16 debugbuilddag feature1
| | r4
| |
| @ 2dc09a debugbuilddag
| | r3
| |
| o 012414 debugbuilddag
|/ r2
|
o 66f7d4 debugbuilddag
| r1
|
o 1ea734 debugbuilddag
r0
```
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4695861
Signature: t1:4695861:1489325959:1145a29ba87f99d07ede1e81415804723dc48818
Summary:
We are relying on pushrebase to replay each of the incoming commits so they
produce both flat and treemanifests. This patch blocks any commits that don't
come through pushrebase, by intercepting manifest.addgroup calls (which is what
a normal push would use) and throwing an exception.
Test Plan: Added a test
Reviewers: #mercurial, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4093745
Signature: t1:4093745:1478100170:79b1eea6668cd3efb27742323e11a45bde930440
Summary:
When a server is a treemanifest enabled server, we want to ensure that every
commit that comes in will be added to the treemanifest as well as the flat
manifest. This patch accomplishes that by hooking into the addmanifest logic and
applying the new manifest to the tree as well.
Test Plan: Added a test
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4093738
Signature: t1:4093738:1478260442:58f77bd88ab09ccda4ac238a065388be8f4a271d
Summary:
This adds a command for producing tree manifests for existing manifests in the
repository. The tree manifests are stored similar to how vanilla mercurial
stores them, but the root revs are in 00manifestree.i instead of 00manifest.i
and we fake the root node to be the same as the flat manifest node, so we can
reference either the flat or the tree easily.
Test Plan: Added a test. Also began converting fbsource
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4092252
Signature: t1:4092252:1478260394:a55360a8ba83ad85b1694cb93f0788f1f672ef61
Summary:
The computehidden caching functionality doesn't seem to be correct, and breaks
my repository. "hg log -r head()" now incorrectly reports some obsolete
commits, which causes various mercurial commands to crash. (They incorrectly
think that these commits aren't obsolete, but their parents are obsolete, which
breaks things.)
This diff reverts the new functionality until it gets fixed.
Test Plan: Confirmed that "hg-dev pull" no longer crashes in my repository.
Reviewers: #mercurial, stash, durham, quark
Reviewed By: quark
Subscribers: net-systems-diffs@fb.com, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4676573
Tasks: 16558840
Signature: t1:4676573:1489008220:f730da69ec0f8d7220cad16ea6956944cfeb670d
Summary: Previously, we were relying on the `allowunstable` extension to wrap various instability-inducing commands and set the appropriate config option to cause the `tweakdefaults` extension to write the correct operation metadata into the obsmarkers produced by each respective command, causing commits to show up as "amended as", "rebased as", etc, in `hg sl`. Since the operation name was based on the command being run, if a user ran `hg amend --fixup` instead of `hg restack`, for example, smartlog would say "amended as" instead of "rebased as". This diff fixes the problem by always setting the operation to "rebase" during a restack.
Test Plan:
1. Create a stack of two commits.
2. Amend the bottom one without `--rebase`.
3. Run `hg amend --fixup` with the `fbamend.userestack` config option enabled.
4. Run `hg sl --hidden` and observe that the rebased commit is marked as "rebased".
See test file for an example.
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4670066
Tasks: 16529478
Signature: t1:4670066:1488991076:58c139c50a1aab2aa4fc7e28b1efd57a67bdfb08
Summary:
Sometimes the hg process gets stuck, and we want to get the stack traces to
learn what it's doing. It's not a big issue if gdb can be used with python
debugging support [1]. However, that feature could not be easily set up on
OS X. This extension will make debugging on OS X easier by providing the
Python stack traces on SIGUSR1.
It's similar to OpenStack's "GuruMeditationReport" feature [2].
Unlike `contrib/showsstack.py`, this extension uses SIGUSR1 instead of SIGQUIT,
and writes to a file, instead of stderr, and prints all threads, instead of
just the current one. So it's more practically useful. We may want to
replace `showstack.py` eventually.
[1]: https://wiki.python.org/moin/DebuggingWithGdb
[2]: https://wiki.openstack.org/wiki/GuruMeditationReport
Test Plan: Added a new test
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4660938
Signature: t1:4660938:1488927878:c751856681816a739160c361ed5cc10bab325000
Summary:
Now that ctreemanifest no longer depends on python.h, let's move pythonutil over
to cstore where all the python code is.
Test Plan: Ran the build and the tests
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4663988
Signature: t1:4663988:1488895919:652b3fc35a2dd12c51a9f70e32997c7b4d037c95
Summary:
As part of breaking the native cstore implementation away from Python, let's
create a Matcher class that can be used to perform path match testing. Initially
the only implementation is the PythonMatcher which just wraps a python match
object.
Test Plan: Covered by existing matcher tests in cstore-treemanifest.py
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4663354
Signature: t1:4663354:1488914384:2c33c7e0e7f2eade0786b6ff41503317989fd1e5
Summary:
Previously, tree repacks did not take into account tree history. It would just
look at the delta base and if the base existed, it would just reuse the delta.
This would A) result in very long chains, and B) result in chains where the full
text was the oldest version, instead of the newest (recent full texts means
faster access to recent versions).
This patch threads tree history into the repacker, which already knows how to
use history for repacks.
Test Plan:
Updated the tests, and inspected the new test results to ensure tree
entries that were not deltas before the repack became reverse deltas during the
repack.
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4647359
Signature: t1:4647359:1488882710:dba72cf488766ce827b7641735164fa0efc9a303
Summary:
Previously, when treemanifest would create packs of trees during pull, we
allowed trees to be delta'd against trees in other packs. This resulted in
smaller packs, but if the other pack disappeared for some reason (since it's a
cache), the chain broke.
This patch ensures that the first version of every tree added to a pack is a
full text.
This temporarily makes repacks worse, since the repacker doesn't know about
history to produce deltas when combining packs. The next patch adds history
awareness which improves the repack deltafication.
Test Plan:
Updated the tests, and inspected the new test results to ensure that
all packs only had deltas within the pack.
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4647348
Signature: t1:4647348:1488882214:e850622a853a534fc60caeef604c88c30740c60d
Summary:
Previously the treemanifest code itself would create the text deltas when
writing a tree out. This meant we couldn't make the delta decision based on
other data, like if the p1 commit was in the same pack file.
This patch removes treemanifest.write() and moves all calls over to
treemanifest.finalize() which gives the python/pack layer control over delta
choices. A future patch will use this to ensure tree packs always contain
complete delta chains.
Test Plan: All tests pass
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4645942
Signature: t1:4645942:1488880851:d0c8c902e7e849072a53344630a9184b6d8e1e7f
Summary:
Previously the treemanifest auto-tree-creation logic only produced data packs
containing the actual contents of the tree blobs. This lost history information
which is important for our ability to efficiently repack the data files.
This patch creates history packs during pull as well. A future patch will also
create history packs for the local tree blob store.
Test Plan: Updated the tests to cover this
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4638865
Signature: t1:4638865:1488449992:48b60961b50b90b6d0e75a64af1f36fb29944e7a
Summary:
Command that checks that every head and bookmark node is present in the
bundlestore
Test Plan: arc unit
Reviewers: #mercurial
Subscribers: #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4658436
Tasks: 15389402
Summary:
Computing cache key can be quite slow (100ms and even more). Instead let's use
a simpler cache key: size + mtime of phaseroots, obsstore and changelod and
whether changelog write was delayed.
Test Plan:
1) arc unit
2) Run test-newhiddencache.t
3) Run upstream unittest with extension enabled.
python run-tests.py --extra-config-opt extensions.newhiddencachekey=/home/stash/facebook-hg-rpms/fb-hgext/hgext3rd/newhiddencachekey.py -j20
Verify that there are no failures except for those that show that new extension were added.
Note: also need to comment out debug output in the extension
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4604811
Signature: t1:4604811:1487958882:702ea8fdee16def674db97344832936d53f0cd9d
Summary:
This diff is part of the series to avoid downloading the same bundle
a few times.
Finally reuse the same bundlerepo.
Test Plan: arc unit
Reviewers: #mercurial
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4612094
Tasks: 15389402
Summary:
There are a couple of reasons to avoid using ui.username():
1) If config option ui.username is set on the server then it will use it
instead of the name of the user that does push
2) It prints confusing warning `no username found, using '....' instead`
3) In some cases it fails (probably because it calls socket.getfqdn() which
does network request).
Let's use simple util.getuser() function and set username to 'unknown' if
it fails.
Test Plan: arc unit
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: azich, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4642457
Signature: t1:4642457:1488458040:5dfac435bf44dfa6ab7725d2f5800963e8c860ec
Summary:
Upstream mercurial doesn't run hooks for temporary amended commits
(see https://bz.mercurial-scm.org/show_bug.cgi?id=3501 and
c2ca20984e3c9f30d73f0f35c35904b64edbd692). No other hook contains info about
temporary amended commits.
Let's wrap localrepository.commit() function and add missing commits inside
this wrapper.
Test Plan: arc unit
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4634950
Tasks: 16387175
Signature: t1:4634950:1488399457:1d299c6ad3cfd418dda32a7ec20c60290f8d7778
Summary:
bundlerepo adds new commits that are not present in fastpartialmatch index.
Without this diff any access to the commit from bundle fails if
fastpartialmatch.raiseifinconsistent config option is set.
Let's not use fastpartialmatch in bundlerepos
Test Plan: arc unit
Reviewers: #mercurial, mjpieters
Reviewed By: mjpieters
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4635368
Tasks: 16394593
Signature: t1:4635368:1488393506:15b8f6a9fddabe681de7fd1296bbfafb9512f558
Summary:
Turned out that wrapping `changectx.__init__()` doesn't give any performance
benefits but instead it breaks initializing of changectx. For example, if
there is a commit which revision number N is a prefix of another commit A then
`hg log -r N` outputs commit A instead of commit N. Let's get rid of this
wrapping.
Test Plan: arc unit
Reviewers: #mercurial, mjpieters
Reviewed By: mjpieters
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4635311
Tasks: 16394593
Signature: t1:4635311:1488393427:eae091a0d155df336adbb122f2881a28e95ba6b6
Summary:
Adds a treemanifest.usecunionstore config flag for enabling and disabling use of
the native code uniondatapackstore.
Since we haven't implemented the repack APIs on the native datapack stores, we
currently have to force repack to use the old python implementations. Instead of
trying to expose just the appropriate APIs through the python interface, I think
we'll rewrite all of repack to be in C++ at a future time, since we can take
advantage of parallelism, etc.
Test Plan:
Updated test-treemanifest.t to use the c datapackstore. Also run all
the tests with --extra-config-opt=treemanifest.usecunionstore=True.
These tests caught a missing null check in the C++ code as well.
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4609795
Signature: t1:4609795:1488365341:203362db5f470b613c4d6484686cd32c3fa8458f
Summary:
The markForRefresh function allows callers to tell the datapackstore that new
pack files have been written. This is important for being able to support
generation of treemanifests during pull and is a blocker for using the native
datapackstore in production.
Test Plan: Added a test
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4609076
Signature: t1:4609076:1488365022:d5480897f2b7674b4d07243e20a495551f985321
Summary: Make `test-show.t` pass again!
Test Plan: - run test locally, see it passing
Reviewers: #sourcecontrol, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4636012
Signature: t1:4636012:1488401012:da051bd55c91890c637f954aaf151d88ea37a978
Summary: As per upstream's https://www.mercurial-scm.org/repo/hg-committed/rev/8fb2c95d816a, we need to add `--color` to help messages.
Test Plan: - run test locally, see it passing
Reviewers: #sourcecontrol, mitrandir, simonfar
Reviewed By: mitrandir, simonfar
Subscribers: simonfar, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4627818
Signature: t1:4627818:1488296052:eff62a80e71c24965072d13e2f67b0e78b3f4ea3
Summary:
This makes the `_getfiles` batch size configurable. Let me know if some other config name will serve this purpose better.
**Reason for this fix**
Currently, `_getfile` will write 10000 lines of text into a pipe and only upon the success of this operation, will read file blobs from another pipe. Serving process will start writing file blobs into a second pipe as soon as it sees something in the first pipe. Second pipe's buffer will fill up as it is not read from by the client until client writes 10K file requests. 10K file requests fill the buffer of the first pipe and we have a deadlock.
Ideally, we should make client check whether it can write to the first pipe and if not, go and read from the second pipe, but that is a bigger fix.
Test Plan:
- run local tests, see them all passing
- except for `test-cstore.t`, but it fails for me without my changes as well
- this generally makes sense
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4620152
Signature: t1:4620152:1488221258:04555177926d129c6ba41bc982ad4e913cb31b20
Summary:
The sparse profile in .hg/sparse currently only can grow, removing rules from this must either be done manually (by editing the file), or one by one using the the -d/--delete command line switch.
To mirror the hg sparse --import-rules command line, hg sparse --clear-rules removes local includes/excludes rules.
Test Plan: fb-hgext/tests/test-sparse-clear.t
Reviewers: marianomartin, mjpieters
Reviewed By: mjpieters
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4609102
Tasks: 16073155
Signature: t1:4609102:1488028035:f5f7b7420d911667fb4a07ac770b7e0480a76977
Summary:
The current code checks if the filter error reports a rev number, and if it does
tries to convert it to a node. If the 'number' is actually just a short form of
a hash that happens to be all digits, then it may be a number greater than the
size of the changelog, so we need to not attempt to access it in the changelog,
otherwise we get an IndexError.
Test Plan: Added a test, it failed before with a weird error
Reviewers: #mercurial, kulshrax
Reviewed By: kulshrax
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4613988
Signature: t1:4613988:1488027938:4f5ffcb07aeecbaf59cad20fb3a592dd92ed2beb
Summary: Get rid of hardcoded devserver name
Test Plan: arc unit
Reviewers: #sourcecontrol, andrasbelo
Reviewed By: andrasbelo
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4611903
Signature: t1:4611903:1487951306:00465fe52de92c71f85c3706594d1a128d4b9628
Summary:
Previously --list-remote wasn't user friendly. User have to always specify a
pattern, and specifying * just won't work because of bash globbing. Also
it wasn't possible to specify many patterns at once.
This diff allows specifying many patterns, it allows to not specify
patterns at all and also it allows to specify default scratch patterns to fetch
if no patterns were specified. This is useful if there are many auto-generated
bookmarks under the same scratch bookmark pattern.
Test Plan: arc unit
Reviewers: #mercurial, mjpieters
Reviewed By: mjpieters
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4605548
Signature: t1:4605548:1487934838:066a12c28dda16fa8f90674d736d21345631ef7e
Summary:
discovery.findcommonoutgoing() explicitly skips secret commits. Because of it
we also need to skip bookmarks that point to secret commits, otherwise they
will point to non-existent nodes.
Test Plan: arc unit
Reviewers: #mercurial, mjpieters
Reviewed By: mjpieters
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4611393
Tasks: 15389402
Signature: t1:4611393:1487933210:72e79923c944b13204f4cde64d415076703bbe47
Summary:
A future patch will want to import just the cstore py headers into
py-ctreemanifest, so let's move those out to their own header file.
Test Plan: Ran the tests
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4571320
Signature: t1:4571320:1487774077:64b4c5053b223f075a1c9f73ddcd92bd73b53e23
Summary:
To begin the process of weaning treemanifest off of Python and on to the new
cstore, let's introduce a base Store class that implements a simple get()
function. UnionDatapackStore already implements this. Future patches will make a
Python version that implements this interface as well, so we can swap between
them easily.
Test Plan: Built and ran the tests.
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4569870
Signature: t1:4569870:1487846232:aa816311ec37a520a2405b68e3c02ad744613049
Summary:
This implements the get function of UnionDatapackStore which iterates over
deltas and produces a full text. Unfortunately the delta chains currently come
out in reverse order (full text last), so we have to read all of them before we
can produce the delta.
Since mpatch takes a char*, and we want to avoid copying the array into a string
array for every read, we event a ref counted string reference type which let's
us transfer ownership of the char* to the caller.
Test Plan: Added a test
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: stash, simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4556547
Signature: t1:4556547:1487200302:c73b8f464f6ce413f1c22c0125a91fbece701f08
Summary:
Implements the getdeltachain function on the new UnionDatapackStore class.
This required some modifications to the DeltaChainIterator. Since the results of
the iterator may cross multiple different chains, we need to keep each chain
alive until the iterator is destructed, so we need to keep a reference to each
chain. We also had to remove the size() property from the iterator since the
fact that the chain spans chains means we don't know the size up front.
Test Plan: Adds a test
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4556458
Signature: t1:4556458:1487199872:07dffa3121acfbeb6d6993b518e6f4887122d4d5
Summary:
This adds a new C++ UnionDatapackStore implementation that only has the
getmissing() function at the moment.
Test Plan: Adds a test
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4556052
Signature: t1:4556052:1487161607:664752df19c63c06819ee2af5b4c436f1b76609d
Summary:
This implements a C++ version of the datapackstore getmissing() function and
adds a test.
Test Plan: Added a test
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: simonfar, ikostia, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4552853
Signature: t1:4552853:1487191581:c108be4a85f8eacafddbea8086d5d22fdcabdacf
Summary:
The remaining python parts of the store are a perf bottleneck when accessing
hundreds of thousands of pack file entries (like in treemanifest). Let's
implement them in C++.
This first patch just add the basic boiler plate, and implements a single
function getdeltachain(), with a test. Future patches will add more
functionality and other parts of the store.
Since cstore depends on cdatapack and ctreemanifest (the pythonutils.h part for
now), we need to tweak our setup.py to enforce a certain build order too.
Test Plan: Added a test, yo
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: simonfar, stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4547929
Signature: t1:4547929:1487181318:21c146cf370d26cb97efe6a883868b85b4e32f49
Summary:
As part of unifying our native store data structures into a single library,
let's move the treemanifest (including the python extension) into py-cstore.
Test Plan:
Built and ran the tests. Verified there was no ctreemanifest.so
dependency in the built cstore.so by using 'ldd cstore.so' on Linux and 'otools
-L cstore.so' on OSX.
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4602484
Signature: t1:4602484:1487842683:964cbb43b7cb20d0db699ef691fe7fcf6bccf2e8
Summary:
'python setup.py build' puts the build artifacts in build/lib.../, but
test-ctreemanifest.py was not configured to look there so it was actually
running against the system ctreemanifest in that case.
The old development workflow was to use make local, which put it at the root of
the repo, which is why this used to work. Let's move the ctreemanifest to be
with the rest of the tests in test-cstore.t so we can set the environment up
correctly.
Test Plan:
Some future diffs broke the test without me knowing. I happened to
catch it via other means, so I came back and fixed the test framework.
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: simonfar, quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4570449
Signature: t1:4570449:1487763880:682800284c01ac25bd99cbe811a8d98d34caf79b
Summary:
As part of unifying our storage layer into a single library, let's move
py-cdatapack into the new cstore directory. Future patches will move
ctreemanifest and the upcoming datapackstore into here as well.
py-cdatapack.h required some reordering since it seems forward declarations work
a little differently between C and C++. There were no code changes though,
except one int->size_t fix.
Test Plan: Ran the tests
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4581320
Signature: t1:4581320:1487788968:e8a34c7a03a16db282214c7dd476b749b92a1bfa
Summary:
A future patch which renamed cdatapack.so to cstore.so pointed out that our
test-remotefilelog*.py tests are running against the system remotefilelog. Let's
move them to a .t test that sets the appropriate PYTHONPATH.
Test Plan: Ran the tests
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4581281
Signature: t1:4581281:1487788843:365c6adf41ba51bdbe7394c1ab423fcc5a63bdbc
Summary: creating a mercurial extension so that `hg record` functionality can be used via a text editor (instead of interactive command prompt or curses menu)
Test Plan: `$ run-tests.py test-edrecord.t`
Reviewers: durham, skreyen, most, #sourcecontrol, asriram, simonfar
Reviewed By: simonfar
Subscribers: quark, rmcelroy, simonfar, mjpieters, #nuclide
Differential Revision: https://phabricator.intern.facebook.com/D4547080
Tasks: 15728734
Signature: t1:4547080:1487842609:8855e35010cac3b6d2b136a80d7effa369eb1da8
Summary:
listkeyspatterns support batching. Let's use and avoid making many network
calls to the hg server.
I had to modify tests because scratchbookmarks are passed to the listkeyspatterns,
and scratchbookmarks is dict and the order of the keys is unpredictable.
Because of it bundles from the server will be sent in unpredictable order, and log
output may be different.
Test Plan: arc unit
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4590914
Signature: t1:4590914:1487788586:dd5ba0dd41b0cd19f790755d7b3c1fc5ba4ade61
Summary: The data was being sent to scuba, but then ignored altogether. Turn this into a single string instead.
Test Plan: arc unit
Reviewers: #sourcecontrol, andrasbelo
Reviewed By: andrasbelo
Subscribers: mjpieters, andrasbelo
Differential Revision: https://phabricator.intern.facebook.com/D4598196
Signature: t1:4598196:1487784582:8952e0e386325d1635029aab2b9d97e9284376ea
Summary:
The profiling extension is to log interactive time.
The upstream change 459366b580cf breaks the profiling extension as it could
no longer wrap `maybeprofile` in time. And @simonfar's
`ui.timeblockedsection` is going to be a superior solution. So just remove
the profiling extension and its tests.
Test Plan: Checked that the extension is not enabled in our config files.
Reviewers: simonfar, #mercurial
Subscribers: mjpieters, simonfar
Differential Revision: https://phabricator.intern.facebook.com/D4583087
Summary:
1196262099b5ff1f8529afc8055b1bc2c7e41658 upstream change cause test to fail
Fix it
Test Plan: arc unit
Reviewers: #sourcecontrol, mitrandir
Reviewed By: mitrandir
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4581218
Signature: t1:4581218:1487366936:330f5982edcd03e7e8f4b4a514ea3b285713795b
Summary:
97d0be4019ac in core broke the fastlog extension for multi-path
queries. We did not have a test to catch it. It was reported as broken by a
user.
Test Plan:
```
$ ../../hg/tests/run-tests.py test-fastlog.t
```
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4579133
Tasks: 16154586
Summary:
This command will allow us to expose simple api to all automation checking
working copy parent. It's meant to be overridden by our wrapper to read the
dirsate directly without shelling out to hg (see D4454504 )
Test Plan: see tests
Reviewers: #mercurial, durham, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4454502
Signature: t1:4454502:1485441135:09902c6437a7af55dfa5c97e3165681de687231f
Summary:
fastpartialmatch index may be incorrect if fastpartialmatch extension was
disabled, then new commits were added and then fastpartialmatch was
enabled back. We need to detect this situation and rebuild the index in
this case.
One approach would be to use _handleinconsistentindex() function to mark index
as invalid. There are a couple of drawbacks to this approach:
1) It requires more significant code changes
2) The output may still be incorrect if hash with the same prefix was added while fastpartialmatch was disabled.
Another approach would be to use header version from index files
(see class _header). That's even worse because we'll need to bump index
version in the code whenever we disable fastpartialmatch.
Third approach would be to use changelog mtime + size as a validation key but
then we need to ensure partialmatch index is written after changelog.
Instead I suggest to use generation numbers. It's just a number in the config
and in the special file inside partialmatch index directory. If the
two doesn't match then index is incorrect and needs to be deleted. The biggest
drawback to this approach is that we need to bump generation number each time
the extension is enabled. I think it should be fine because the one who enables
extensions usually understands it and he or she will bump the generation number.
Test Plan: arc unit
Reviewers: #sourcecontrol, durham, mjpieters
Reviewed By: mjpieters
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4571926
Signature: t1:4571926:1487263035:53419c9ca4e344127f391ac19c1071a826fea3f1
Summary:
User reported failures because revlog.opener doesn't have `exists` function.
The reason is that sometimes opener is wrapped inside the function.
Let's use _realopener instead
Test Plan: arc unit
Reviewers: #sourcecontrol, mjpieters
Reviewed By: mjpieters
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4571659
Signature: t1:4571659:1487267797:a950ab7c88d75573df64ec1964755c054307c7ea
Summary:
This implements the get function of UnionDatapackStore which iterates over
deltas and produces a full text. Unfortunately the delta chains currently come
out in reverse order (full text last), so we have to read all of them before we
can produce the delta.
Since mpatch takes a char*, and we want to avoid copying the array into a string
array for every read, we event a ref counted string reference type which let's
us transfer ownership of the char* to the caller.
Test Plan: Added a test
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: stash, simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4556547
Signature: t1:4556547:1487200302:c73b8f464f6ce413f1c22c0125a91fbece701f08
Summary:
Implements the getdeltachain function on the new UnionDatapackStore class.
This required some modifications to the DeltaChainIterator. Since the results of
the iterator may cross multiple different chains, we need to keep each chain
alive until the iterator is destructed, so we need to keep a reference to each
chain. We also had to remove the size() property from the iterator since the
fact that the chain spans chains means we don't know the size up front.
Test Plan: Adds a test
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4556458
Signature: t1:4556458:1487199872:07dffa3121acfbeb6d6993b518e6f4887122d4d5
Summary:
This adds a new C++ UnionDatapackStore implementation that only has the
getmissing() function at the moment.
Test Plan: Adds a test
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4556052
Signature: t1:4556052:1487161607:664752df19c63c06819ee2af5b4c436f1b76609d
Summary:
This implements a C++ version of the datapackstore getmissing() function and
adds a test.
Test Plan: Added a test
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: simonfar, ikostia, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4552853
Signature: t1:4552853:1487191581:c108be4a85f8eacafddbea8086d5d22fdcabdacf
Summary:
The remaining python parts of the store are a perf bottleneck when accessing
hundreds of thousands of pack file entries (like in treemanifest). Let's
implement them in C++.
This first patch just add the basic boiler plate, and implements a single
function getdeltachain(), with a test. Future patches will add more
functionality and other parts of the store.
Since cstore depends on cdatapack and ctreemanifest (the pythonutils.h part for
now), we need to tweak our setup.py to enforce a certain build order too.
Test Plan: Added a test, yo
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: simonfar, stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4547929
Signature: t1:4547929:1487181318:21c146cf370d26cb97efe6a883868b85b4e32f49
Summary:
Don't let errors propagate out of the gitnode() revset. Always report errors
in gitnode() as a translation failure, rather than letting exceptions propagate
up and crash mercurial. The code was previously only catching internally
generated ConduitError exceptions, but it can also throw HttpError exceptions,
and the underlying httplib code can throw its own exceptions as well as
socket.error exceptions.
This also fixes the test code to use an ephemeral TCP port rather than assuming
port 8543 will always be available. Using a fixed TCP port in test code is a
very common way to cause bogus failures if the tests are run in parallel.
(For instance, testing multiple repositories in parallel on the same build
host.)
Test Plan:
Added unit tests that check the behavior when the server returns a 500 error,
and when the server refuses the connection entirely.
Reviewers: quark, durham, rmcelroy
Reviewed By: rmcelroy
Subscribers: net-systems-diffs@fb.com, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4556871
Signature: t1:4556871:1487062142:b58d770d46c975d44933bec08cfce8acb25ff16b
Summary:
Previously, smartlog use `tip` as the fallback master, and prompts:
warning: there is no master changeset locally, try pulling from server
if master is `tip`.
That makes smartlog less friendly for external users. This patch removes the
prompt and changes the fallback master to the last public revision so it
works out-of-box.
Test Plan: `arc unit`
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4555346
Signature: t1:4555346:1487036530:35a5fa65d5419e7324d3203fa52de9a82b8132e3
Summary:
This diff makes the following modifications to the (former) `hiddenhash` extension:
- Makes the error messages configurable, allowing us to easily change the error messages in `facebook.rc`.
- Renames the extension to `hiddenerror`, since the primary feature is now the configurable error messages rather than using a hash in the message.
- Changes the method by which the extension wraps the `changectx` constructor. The old method of wrapping would result in errors in code paths that in `context.py` that performed `isinstance()` checks on contexts.
Test Plan: To test arc patch this extension into your `facebook-hg-rpms` and enable it for a test repo. Then, attempt to access a hidden node. You should see the default error message defined in the extension. Setting `hiddenerror.message` and `hiddenerror.hint` should allow you to customize the messages. See test file for examples.
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4548140
Tasks: 15896355
Signature: t1:4548140:1487063552:06395c1eaaa83b00ead8e80e2cd8f9d13f7d08e2
Summary: Sometimes you want to be able to justify why a line is in a sparse profile. Support empty lines and comments (starting with a hash).
Test Plan: Run the test runner
Reviewers: #mercurial, rmcelroy, durham
Reviewed By: durham
Differential Revision: https://phabricator.intern.facebook.com/D4537617
Tasks: 15993301
Signature: t1:4537617:1486687441:18a00b65df244b573bab095809fb1fbb57e21d71
Summary: If `hg debuginhibit` is run with no arguments, print out a list of inhibited nodes. This is similar to how `hg debugobsolete` works and will assist with debugging.
Test Plan: Run `hg debuginhibit` in a repo with inhibited nodes. (This can be tested by either stripping some test commits and then updating back to them, or by manually inhibiting the commits with `hg debuginhibit`.) A list of inhibited nodes should be printed. See test file for an example.
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Differential Revision: https://phabricator.intern.facebook.com/D4541514
Tasks: 16004717
Signature: t1:4541514:1486722366:10b4232830ed88931bd6ac610972114743797192
Revert Plan:
Summary:
Let's pull node during update if not found locally.
This is a part of selectivepull functionality.
See remotenames extensions for details about selectivepull.
Test Plan: arc unit
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: indragie, mjpieters, sergeyb
Differential Revision: https://phabricator.intern.facebook.com/D4536129
Tasks: 12479658
Signature: t1:4536129:1486667537:3d1df30cb5d1db0dd7451756102ccafee20789d5
When remotefilelog moved from its own repo, the tests needed to be updated to
adjust the PYTHONPATH to ensure the in-repo remotefilelog was loaded instead of
the system one.
This meant any local runs of remotefilelog tests would've been using the system
remotefilelog unless the user had manually set the PYTHONPATH themselves.
Summary:
Previously we only wrapped manifestlog.__getitem__ because fastmanifest didn't
support the concept of directory manifests. Unfortunately manifestlog.get() is
used in some flat manifest cases too, like in changegroup.py where it just
passes '' as the directory but still uses .get().
This bug has been present since the manifestlog refactor, but only became
exposed now that we've improved our caching and hold on to manifest ctx's for
longer.
This fix is to just wrap manifestlog.get() too. Fixing this exposed some other
inconsistencies with our fastmanifest api.
Test Plan:
shelve now passes in a repo that was hitting an exception because of
this before. I'm still trying to think of how to test this in a test. I've not
been able to repro it in my own repo yet.
Reviewers: #mercurial, quark
Reviewed By: quark
Differential Revision: https://phabricator.intern.facebook.com/D4540677
Tasks: 15991119
Signature: t1:4540677:1486690231:07a95b402913d7dcc31633341e235ee1eb0163bc
Summary:
See comment in code for details. This addresses a performance
problem that arises when the same remotefilelog repo makes two changes
to a file far apart in history with no other repo making changes to the
same file in between.
Test Plan: new test verifies that linkrevs are fixed up using remote blob
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: stash, quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4475183
Tasks: 12958591
Signature: t1:4475183:1486576777:81f4f9815cb677c94954c23ecd8aa7edb006ffd9
Summary:
Previously, when a command tried to access a hidden revision without the `--hidden` flag, the user would get the following error:
```
abort: hidden revision '0'!
(use --hidden to access hidden revisions)
```
With this extension, the error is now:
```
abort: hidden commit b8144197c244!
```
This way, novice users aren't tempted to immediately re-run the command with `--hidden`, which may have undesired results. It also makes it much easier to find the commit in question in `hg sl --hidden` since by default smartlog does not show rev numbers.
Test Plan: Attempt to access a hidden commit, observe new error message as seen above. Note that in the test file the word "changeset" is present instead of "commit" due to the lack of the `dialect` extension in the test.
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4506065
Tasks: 15747879
Signature: t1:4506065:1486122372:d82ebd488a00d1958c235f733a9455111954c116
Summary:
Recently @rmcelroy discovered that attempting to rebase a single commit in the middle of a stack results in the rest of the stack being marked as unstable. See https://fb.facebook.com/groups/sourcecontrol/permalink/1207767375939547/ for full example.
It turns out that the inhibit extension's post-transaction hook that would ordinarily inhibit the rebased (and thus obsolete) commit, and thereby hide the instability, was not firing in this case. This was similar to a previous problem with `hg next --rebase` wherein `--continue`'ing a rebase with conflicts would result in visible instability for the same reason. In both cases, the problem appears to be that `rebase` doesn't actually create a transaction, so of course the post-transaction hook doesn't fire.
The solution for `hg next --rebase` was to add a (nearly) empty transaction after the rebase call to trigger the hook. This diff just makes this the default behavior for `hg rebase` so that the hook is always called.
Test Plan:
See new test file. Basically, create a situation like:
```
o e5d56d debugbuilddag
| r3
|
| o c175ba debugbuilddag
| | r2
| |
| o 220949 debugbuilddag
|/ r1
|
o 1ad88b debugbuilddag
r0
```
Perform `hg rebase -r 1 -d 3`, which rebases commit "r1" onto "r3", which would ordinarily make "r2" unstable. Instead, the result should be as below, with no visible instability because "r1" is inhibited.
```
o 738e8e debugbuilddag
| r1
|
o e5d56d debugbuilddag
| r3
|
| o c175ba debugbuilddag
| | r2
| |
| x 220949 (Rebased as 738e8e) debugbuilddag
|/ r1
|
o 1ad88b debugbuilddag
r0
```
Reviewers: rmcelroy, #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters, rmcelroy
Differential Revision: https://phabricator.intern.facebook.com/D4491870
Tasks: 15697758
Signature: t1:4491870:1486033128:45b8e9a72af1e88ab78484ddef47a788d1aca7b5
Summary: Sort the output to fix tests on different platforms
Test Plan: arc unit on mac
Reviewers: #sourcecontrol, simonfar, ikostia
Reviewed By: simonfar, ikostia
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4501449
Signature: t1:4501449:1486050577:3041dc3c3691f48e39a0e9e05b1f97954a6d88e2
Summary:
If we have too many unsorted entries then finding matching revisions is
slower. To fix it we will rebuild partial index from time to time.
_findcandidates function marks index as need-to-be-rebuilt if it notices
index file with too many unsorted entries. Then _changegrouphook rebuilds
the index.
Test Plan: arc unit
Reviewers: #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4494617
Summary:
Since we have some of the nodes sorted we can now use bisect to quickly find
matching nodes.
Test Plan: arc unit
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4494419
Summary:
Append header to each fast partial match index file. For now it contains
version number and number of sorted entries in the file.
File is ignored if version number from file is different.
Test Plan: arc unit
Reviewers: #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4488417
Summary:
We'll add bisect algorithm soon.
Let's sort the nodes first during index rebuilding
Test Plan: arc unit
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4488263
Summary:
strip may change revision numbers of many nodes at once.
It's better to rebuild index in case of strip.
Test Plan: arc unit
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4474710
Summary:
This hook builds index if it's not present and it appends new entries
from the changegroup.
Test Plan: arc unit
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4474704
Summary: These two functions actually use fastpartial index to resolve revision for a node.
Test Plan: arc unit
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4474658
Summary:
Add commit hook to append new entries to the index.
It uses the same transactions as revlogs.
Do not append entries if index is not built (i.e.
if _partialindex does not exist). This is to ensure that we either
have complete index or none at all. Besides next diff will add
hooks to build index after pull if it doesn't exist.
Test Plan: arc unit
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4474654
Summary:
Index to make node partial match fast i. e. speed up matching of partial hashes like f9bbd5.
This is an initial diff and it doesn't add any useful functionality, just debug commands.
Storage format is simple. There are a few files (at most 256).
Each file contains entries:
<20-byte node hash><4 byte encoded rev>
Each entry represents a commit (node).
Name of the file is the first two letters of the hex node hash. Nodes with the
same first two letters go to the same file. Nodes are NOT sorted inside the file
to make appends of new nodes easier.
Partial index should always be correct i.e. it should contain only nodes that
are present in the repo (regardless of whether they are visible or not) and
rev numbers for nodes should be correct too.
Test Plan: arc unit
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4474642
Summary: Add a new extension to assist with debugging issues with the inhibit extension. Adds the `hg debuginhibit` and `hg debugdeinhibit` commands to manually inhibit/deinhibit commits as well as config options to print out the nodes being inhibited and truncated stack traces of each call site. I mostly made this into an extension so that it would be easy to debug inhibit issues from any user by just asking them to run their command with the appropriate `--config` flag.
Test Plan: See test file. Enabling the `debuginhibit.printnodes` and `debuginhibit.printstack` functions should result in the relevant nodes and stack traces to be printed whenever nodes are inhibited and deinhibited.
Reviewers: durham, rmcelroy, #mercurial, mitrandir, simonfar
Reviewed By: simonfar
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4485659
Signature: t1:4485659:1485985964:e6920b67450942c706c9523e9df48b4093b26d69
Summary: Although the split and fold commands usually abort on error, in some cases they simply write to stderr and return a nonzero exit status. Previously, the wrappers in fbamend would expect failures to result in an abort, and therefore didn't return early. This resulted in restack errors when this was not the case -- notably in the case of folding a commit with itself. This diff fixes the problem by returning early on error.
Test Plan: Attempting to fold a commit with itself (i.e., `hg fold --exact . .`) fails with an error message instead of crashing.
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4493029
Tasks: 15704338
Signature: t1:4493029:1485986404:17186b47786cf79c735b33adbb19de79ac75a5be
Summary: Previously, if a user ran split or fold on commits with bookmarks on them, those bookmarks would remain on the original commit instead of the new split or folded commits. This behavior was unexpected and confusing because it meant the obsolete commit would remain visible and the user would be forced to manually move the bookmark. This change makes split and fold automatically move any bookmarks on the old commits onto the newly created commits.
Test Plan: Run `hg split` or `hg fold` on a commit or set of commits with one or more bookmarks, and observe that the bookmarks are updated to point to the newly created commits. See tests for examples.
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4490655
Tasks: 15595449
Signature: t1:4490655:1485986009:858f099d3f95c6a45fe8b119d4f8d376b87652b7
Summary:
In certain situations, like infinitepush, the noderev cache builders would try
to save a node that didn't exist in the repo. This caused an exception. Now we
check each node before we attempt to resolve it to a rev.
Test Plan: Added a test. It failed before, and passes now.
Reviewers: #mercurial, andrasbelo
Reviewed By: andrasbelo
Subscribers: andrasbelo, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4494848
Signature: t1:4494848:1485964835:986d4fe3e29404cac22e4acf7d88c3715fcb51b9
Summary: Minor stylistic cleanup for this test.
Test Plan: Test still passes.
Reviewers: #sourcecontrol, mitrandir
Reviewed By: mitrandir
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4486391
Signature: t1:4486391:1485876954:0d800a4ba3858b23560d7ae9d3ba5a9981e9eb6a
Summary:
Previously the noderev cache only covered the branchmap. These days remotenames
and phases are also doing a lot of node->rev look ups, so let's add them to the
noderev cache as well.
Test Plan:
Compared hg book before and after the change. Noticed a significant
perf increase.
Reviewers: #mercurial, ikostia
Reviewed By: ikostia
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4487962
Signature: t1:4487962:1485871615:634d95f61e916d81440a0f440010912af5ae6a13
Summary:
Previously, fastannotate requires `.hg/fastannotate` to be writable. However,
this cannot be guaranteed in cases like running hg in other's repo.
This patch adds some exception handling to make fastannotate running in "fctx"
mode just work even if `.hg/fastannotate` is not writable.
Note: the `fastannotate` command ("fastannotate" mode) is not changed as if
that command is triggered explicitly, the user should be aware of it works
differently from the vanilla annotate command.
Test Plan: Added a test
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4481427
Tasks: 15489272
Signature: t1:4481427:1485775004:ccc0b99b38146034c5b235445d762e55dbb78dbe
Summary: This is basically the same as `hg amend -i` but since allowunstable operates at the command function level we need to explicitly whitelist this command. Note that I've set the obsmarker metadata to "amend" so that commits affected will be marked as "Amended as XXX" in smartlog. I don't think any other `hg record` flags other than --amend create obsmarkers so this should be OK.
Test Plan: Run `hg record --amend` in the middle of a stack and observe that it works now.
Reviewers: durham, #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters, reedriley
Differential Revision: https://phabricator.intern.facebook.com/D4466338
Tasks: 15705764
Signature: t1:4466338:1485424744:442604ce2089298f0724c27025502a3602113401
Summary: There is no valid reason to run `hg restack --hidden` (and this will cause unexpected behavior) so disable it.
Test Plan: `hg restack --hidden` now aborts. Added check to unit test.
Reviewers: rmcelroy, #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4463934
Tasks: 15615216
Signature: t1:4463934:1485386049:9836ec970d7c3e11c7dfaed21aa0ce8b58013294
Summary: fold doesn't work without --from or --exact anymore
Test Plan: all tests are passing now
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4441874
Signature: t1:4441874:1484922064:1616827e2308b4b9d43e3ef1bd90f51c35a2a513
Summary: The template keyword for split was not producing output because obsmarkers with multiple successors were being filtered out.
Test Plan: Correct "Split into XXX, YYY" labels appear in smartlog. Tests have been added for all of the obsolescence-related smartlog template keywords.
Reviewers: #sourcecontrol, rmcelroy, durham
Reviewed By: durham
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4396453
Signature: t1:4396453:1484880307:bfdc60385e42fee169ff2f0ec4f85dd0747b55c9
Summary: Automatically rebase unstable commits left behind after a split or fold in the middle of a stack. Previously, users had to run `hg restack` to fix their stacks after the fact. This change also fixes the issue identified in t14560234 whereby the old stack (entirely obsolete) stack would be left visible after splitting or folding a head commit due to the inhibit extension not correctly deinhibiting the old stack.
Test Plan: Create a stack of commits and perform `hg split` or `hg fold` on a commit (or set of commits) that have descendants. After the operation these commits should be rebased, preserving the linearity of the stack. See unit tests for specific test cases.
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4432833
Tasks: 14560234, 15266009
Signature: t1:4432833:1484880689:1e07e421c8871dfef8bd016454aaed5a88a4a7d1
Summary:
Many of the template keywords in our extensions were being registered
incorrectly, causing their help output to be rendered incorrectly in the
"hg help templates" output. The ones in smartlog.py were particularly bad, as
most of them showed only their description, without displaying the name of the
template. In smartlog.py only singlepublicsuccessor was being displayed
correctly, because it's docstring explicitly included it's own name at the
start.
This fixes all of our extensions to consistently use the
registrar.templatekeyword() decorator to register the keywords. This decorator
automatically prefixes the help message with the keyword name. The
mercurial/extensions.py code will explicitly check to see if an extension
contains an "templatekeyword" attribute, and if so it will register any
keywords contained in this registry after calling extsetup().
Test Plan:
Added new unit tests to check the output of "hg help templates" for the
affected keywords.
Reviewers: #sourcecontrol, kulshrax, ikostia, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, net-systems-diffs@, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4427729
Signature: t1:4427729:1484831476:17b478a5e867dfc3f85402588c381bf8b1831107
Summary:
This requirement was preventing the pyflakes test from running. In development
builds our mercurial reports its version as
"Mercurial Distributed SCM (version +0-)", which causes this check to fail.
Everyone using this repository should be on a version of mercurial greater than
1.0 now, so it seems like it should be fine to drop this check.
Test Plan: Ran "arc unit" and confirmed the pyflakes check was not skipped.
Reviewers: #sourcecontrol, durham, rmcelroy
Reviewed By: rmcelroy
Subscribers: net-systems-diffs@, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4427727
Signature: t1:4427727:1484733735:5bd6359b351df5ada917ccc7bac0fa2fae065cee
Summary:
All of the test-check*.t tests cd out of the test repository and into the main
normal fb-hgext repository, and attempt to run hg commands there to examine the
repository contents. Unfortunately they were doing so still using the HGRCPATH
settings configured for the test repository. In practice this causes the tests
in many situations, because extensions enabled in the system configuration and
required for the main repository may not have been enabled.
Test Plan:
Ran "arc unit" in a repository with sqldirstate enabled and confirmed the tests
pass now instead of bombing out with screenfuls of error messages.
Reviewers: #sourcecontrol, stash, durham, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, net-systems-diffs@, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4427670
Signature: t1:4427670:1484733701:f6c859c29d34eb33b1280e798ab3398a03a3a461
Summary:
treemanifest requires fastmanifest, and fastmanifest.usetree requires
treemanifest. Let's make these dependencies explicit in the code and error out
if they are incorrect.
Test Plan: Added a test
Reviewers: #mercurial, wez
Reviewed By: wez
Subscribers: wez, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4417512
Signature: t1:4417512:1484347447:7e18340813fac0b298aa51a7cc2f89fc6953680f
Summary:
Previously the find function would return None if the given file was not present
in the tree. The other manifest implementations throw a KeyError here instead.
This affects things like sparse looking for files in the null revision, where it
was receiving a None and crashing because it expected a KeyError in this case.
Test Plan: Added a test. Also, clones were working again.
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4412592
Signature: t1:4412592:1484321389:690e6cd7b3d1599714102ebf361e59fcb0ed59ef
Summary:
Treemanifest had a bug where the pack files it created were 400 instead of 444.
This meant people sharing a cache on the same machine couldn't access them. In
most of the remotefilelog code we had set opener.createmode before creating the
pack but we forgot to for treemanifest.
This patch moves the opener creation and createmode setting into the mutable
pack class so we can't screw this up again.
Test Plan:
Tests that wrote to the packs before, now failed and had to be
updated to chmod the packs before writing to them.
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4411580
Tasks: 15469140
Signature: t1:4411580:1484321293:9aa78254677548a6dc2270c58cee0ec6f57dd089
Summary: Let people put #, t, commas or no, etc!
Test Plan: existing and new tests
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters, ullrich
Differential Revision: https://phabricator.intern.facebook.com/D4411027
Tasks: 15467918
Signature: t1:4411027:1484264937:cdc2452ac3ba920c153d3e0317972433da188a8e
Summary:
Previously treemanifest and fastmanifest were largely unaware of each other. If
the fastmanifest was available, we'd use that. If not, we'd try tree. Then we'd
fall back to flat. When comparing two manifests, this could cause problems since
if one manifest was fast and one was tree, we'd have to fall all the way back to
flat manifests to compare them.
This patch adds the ability to build a treemanifest from a fastmanifest for
inmemory manifests that were copied from a commit that has both a fastmanifest
and a treemanifest. We do it by diff'ing the inmemory fastmanifest with the
commit fastmanifest, then applying that diff to the commit's treemanifest. Then
we can compare that tree with the other tree.
This is particularly useful when doing things like 'hg up master' after a pull.
The new master commit probably has a treemanifest, but doesn't yet have a
fastmanifest, while the commit the user is on probably has both. Now we can do
that hg up without having to parse a full manifest.
Once treemanifest has been enabled, every commit (both pulled and user created)
should have treemanifests. Therefore this case should never happen when
comparing two normal commits. So in theory, the only way to hit this case is
when Mercurial does a manifest.copy() on a manifest that has a fastmanifest
(like when reading a workingctx which must copy the commit manifest and apply
working copy changes), since the copy will choose to copy the fastmanifest and
leave the treemanifest behind. This patch addresses that case, so I think there
shouldn't be other cases.
Test Plan: Adds a test that covers this case.
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4402604
Signature: t1:4402604:1484216866:1c075f314026aaf608095b4aca37c9a4277ababa
Summary:
We have run into some cases where users ended up with empty pack file in their
packs directory. Just log a warning in this case and skip this pack file,
rather than letting the exception propagate up and crashing the command.
Test Plan:
Created empty 0000000000000000000000000000000000000000.histpack and
0000000000000000000000000000000000000000.histidx files in my repository's
hgcache directory, and confirmed that "hg log" now simply warns about them
instead of crashing.
I didn't really test the perftest.py or treemanifest_correctness.py extensions
much. They seem to throw exceptions, and look like they have maybe gotten a
bit stale. I fixed one minor typo, but didn't dig into the other exceptions
too much.
Reviewers: durham
Reviewed By: durham
Subscribers: net-systems-diffs@, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4402516
Tasks: 15428659
Signature: t1:4402516:1484155254:96d2980efcec2d735257b08910e1ca437ef1dad6
Summary: I somehow missed this test when updating all of them with fbamend's new error message.
Test Plan: Test now passes.
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4399110
Signature: t1:4399110:1484094504:2c7a8124849c9fd32fb57162dfb1ba7a90b1efbc
Previously we attached the original node to the copy of the manifest we made.
This is almost always incorrect. In the main case, we make a copy of the p1
manifest during commit and then edit that manifest to form the new manifest. By
storing the node (and never invalidating it), there's the opportunity for the
code to accidentally load the old p1 manifest instead of the in memory one.
The fix is to not pass node to the copy. This affects the tests because some
tests relied on the copied manifest being able to access the original tree/cache
later. It just happened to work because we never modified the manifest before
performing those loads.
This is similar to the patch to matches() that made the match result not have
node either.
This patch also cleans up the fastdelta logic that attempted to read the
_treemanifest directly, instead of going through the normal read path. This is
what actually caught the bug, since it read the treemanifest from disk, while
the in memory cachedmanifest had changes.
Summary:
The remotenames extension automatically adds a --dest option to `hg rebase`, which causes
restack to abort. Instead, we should just drop the 'dest' key and proceed.
Test Plan: Attempt to use `hg rebase --restack` in fbsource when the current bookmark is tracking a remote bookmark.
Reviewers: #mercurial, durham, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4399846
Tasks: 15422257
Signature: t1:4399846:1484084585:4a6ec907032e7d8b1388980870b6d72cb560829f
Summary: Upstream Mercurial now has native support for statprof, rendering this file useless.
Test Plan: Remove statprofext.py* from my installed hg, confirm it still works. Build fb-hgext and run tests, confirm no new failures
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4397926
Signature: t1:4397926:1484066120:0c1469d838a91a65fb57ecc858675256087e2c48
Summary: Added some tests for this also
Test Plan: arc unit in fb-hgext
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4397881
Tasks: 15360841
Signature: t1:4397881:1484066348:1b9fb24a30a9962de0dad858f4d57956a2c91dc4
Summary: To encourage people to learn about and use `hg restack` (which is a more general fixup tool), tell users about it instead of `hg amend --fixup`. I wasn't sure whether to advertise it as `hg rebase --restack` or just `hg restack`, so I've included both.
Test Plan: Do an amend in the middle of a stack and observe the new wording. Unit tests reflect changed wording.
Reviewers: durham, #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4390111
Signature: t1:4390111:1483962208:46951c8f5e4c499840b0b4b0ee8ac1e1b6c14ca9
Summary:
If client pulled a commit without filelogs (because of remotefilelog) and
this commit was later stripped on the server, then attempt to backup this
commit fails because of missing filelogs. It's a rare issue but unfortunately
it happens sometimes. To fix it let's exclude from the backup all the
commits and their descendants that doesn't have all necessary filelogs.
Test Plan: Run infinitepush tests
Reviewers: mitrandir, rmcelroy, durham
Reviewed By: rmcelroy
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4365747
Signature: t1:4365747:1483466164:560cdf5cd2369cd2603dfd0fe6b30d7a70951f00
Summary:
Test was flakey because wait_for_background_backup.py waited for change in
backup state file but then logfile was checked. In this case it was possible
that backup state file was already synced while logfile was not.
Let's not use logfile at all and just check backup file.
Test Plan: Run `test-infinitepush-*`
Reviewers: rmcelroy, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4383660
Tasks: 12479677
Signature: t1:4383660:1483727504:2d738ef5ea8d016a8df71cbc6a99d44f24a77f3e
Summary:
wait_for_background_backup.py does sleep internally and stops as soon as file
is present (with 0.1 sec precision). Additional sleeps shouldn't be necessary
Test Plan:
Run `test-infinitepush-*` on devserver
PYTHONPATH=/opt/facebook/hg/lib/python2.7/site-packages/:/opt/homebrew/lib/python2.7/site-packages/ /opt/homebrew/opt/python27/bin/python2.7 ../../hg-crew/tests/run-tests.py
on mac
Reviewers: rmcelroy, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4381211
Signature: t1:4381211:1483551806:bbefc6632d7aed83f4a1caf8ad2ec5ba2b8f12d7
Summary:
`pushbackup` failed when there are hidden heads in the repo but no visible
heads (see test for example). This diff fixes it.
Test Plan: Run `test-infinitepush-*`
Reviewers: durham, mitrandir, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4365677
Tasks: 12479677
Signature: t1:4365677:1483465909:fd1d96211909d3eeccab4fd17a364af4a1997f85
Summary:
Recently I had this scm prompt:
{F64910275}
This is not user friendly. I suggest to output at most one remote bookmark
Also this diff copies `seq.py` from upstream hg repo
Test Plan: Run scm prompt tests on devserver and mac and with '--shell=zsh'
Reviewers: rmcelroy
Reviewed By: rmcelroy
Subscribers: simonfar, wez, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4197242
Signature: t1:4197242:1483466446:cd1fe943dc99fde0600a4e20994fa4b0f971f72a
Summary:
Some recent commits introduced non-OSX compatible test output. This patch fixes
that.
- 'ls' has different output text and error codes for missing files on osx
- 'sync' just waits for an in memory write to be flushed to disk, which
shouldn't actually be visible to any user land process, so it seems there's a
deeper race condition here, like wait_for_background_backup.py not working
correctly. Adding a sleep here to work around it.
Test Plan: Ran the tests on linux and osx
Reviewers: stash, #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4378618
Signature: t1:4378618:1483477739:5cd8bf82368cc417fbc7ee22410e6d962ba10342
Summary:
Without `--clean`, the user may not be able to abort cleanly.
Also, change "erase" to "discard" to be more consistent with the official
language. "changed" is replaced by "changes" as it looks like a grammar
mistake.
Test Plan: Updated existing tests
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4377376
Signature: t1:4377376:1483464655:f695fd643509d7fabb88298bd1004727ee64ef76
Summary: bundle file should be closed to ensure that all the temporary files were deleted
Test Plan: arc unit
Reviewers: #sourcecontrol, ikostia
Reviewed By: ikostia
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4377148
Signature: t1:4377148:1483451990:2a1362d9e524a54ffedad8f2bc09fc622efe8ad4
Summary:
In 676596f945ea2166820ef92e692ef7fe6a6247f0 were added comments with
lines > 80. In aec81a9a80d22989bbdc8c74c1dfec9dcbbe6866 default config value
was changed.
Test Plan: arc unit
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4377188
Previously we were relying on mfrevlog.revdiff() to produce the delta for us.
This only showed us what files were added/modified, as compared to p1, and we
had to use a heuristic to know what files were deleted (by looking at the list
of files in the commit metadata). Merge commits have a different criteria for
what is in the commit metadata (it only contains entries where the file is
different from both parents), so we can't use it for the same heuristic. So
let's fall back to a normal manifest diff for merge commits, since they are
rare.
Adds a test for verifying that conversion of merge commits into a tree works.
Summary:
Remove hard-coded devserver name and use `sync` to make sure data is written
by the background process
Test Plan: Run `test-infinitepush-*`
Reviewers: #sourcecontrol, mjpieters
Reviewed By: mjpieters
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4361991
Tasks: 12479677
Signature: t1:4361991:1482423111:81e69044caaa608736e03cc3746cf1ff1f6625a5
Summary:
Make adding a set of rules efficient.
Currently buck autosparse writes directly to .hg/sparse, clobbering any
existing manual rules or profile settings. This is not optimal.
The command-line of hg sparse is too limited at the moment to add rules in any
other way (only one new rule per call!). The --import-rules option lets us
bulk-import and efficiently decide wether or not a refresh is needed.
Test Plan: arc unit
Reviewers: #sourcecontrol, andrasbelo
Reviewed By: andrasbelo
Subscribers: andrasbelo, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4358087
Signature: t1:4358087:1482409901:42b2d32dc624579a3d35875ba1bb126d9539cc69
Summary:
Default path was used during background backup even if non-default path was
passed on the command line. So `hg push somepath --background` is
equivalent to `hg push --background`. This is not correct and this diff fixes it
Test Plan: Run `test-infinitepush-*`
Reviewers: durham, rmcelroy, mitrandir
Reviewed By: mitrandir
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4357295
Tasks: 12479677
Signature: t1:4357295:1482330125:d2b043f8035046ab43667b4387bf642e53261681
Summary: Log active profiles for any command that uses the repository.
Test Plan: arc unit
Reviewers: #sourcecontrol, mitrandir
Reviewed By: mitrandir
Subscribers: andrasbelo, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4346795
Signature: t1:4346795:1482251224:a9b3568eeb7e57c61273ce0dc9f4d41abcd2888e
Summary:
fastannotate will tell remotefilelog what nodes of a file is already known in
linelog + revmap, so remotefilelog will not prefetch those files. Previously,
fastannotate either prevents all prefetching or allows all prefetching, which
is sub-optimal.
fastannotate could now donate its sshpeer to remotefilelog, so remotefilelog
won't need to start another one (assuming they can share a same sshpeer,
could be turned off via config options). This should reduce run time if SSH
handshake is expensive.
fastannotate could now also steal sshpeer from remotefilelog, so fastannotate
won't need to start another one. Combined with the above change, there would
always be only one sshpeer shared by fastannotate and remotefilelog.
Test Plan: Modified existing tests
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: ikostia, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4325382
Signature: t1:4325382:1481933531:39d6344b2c076fbbff1f07997cd268e7cee25e80
Summary:
SCM_SAMPLING_FILEPATH is set by arcanist, and thus the tests fail when you run
arc unit.
Test Plan: Run arc unit
Reviewers: #sourcecontrol, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4342459
Signature: t1:4342459:1481929811:1ac8db74cfd86be7ad35466e6eaf2f7e35ad9140
Summary:
Logging dirstate sizes gives us a way to track sparse profile use; how big a
working copy do people actually have?
The logged info can be directed to Scuba via the sampling extension.
Test Plan: Run on a local test repo with the debugger attached. Goal
Reviewers: #sourcecontrol, quark
Reviewed By: quark
Subscribers: mitrandir, quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4340367
Signature: t1:4340367:1481920819:d7358e0bdc2cc1422b36266374344e7d2b644ae5
Summary:
Previously, if during a pushrebaes a merge commit was rebased and the
parent commit that contained the original rebase destinations content (i.e.
master's content) was on the p2 side, it would all be thrown away.
This is because the pushrebase commit works by copying the destination-repo-p1, then
applying changes from the diff of the original-repo-p1 vs original-repo-commit.
Since all the rebase destination contents was in destination-repo-p2, it is all
lost here.
The solution is to track which commit is the latest one to contain the
destination bookmark contents, and make sure it is always p1 when rebasing merge
commits. This will ensure that the destination bookmark contents are preserved
all the way up to the new commit for that bookmark.
Test Plan:
Added a test. It fails before the change and passes after. Another
test was also affected, since the p1/p2 ordering is different.
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4336427
Tasks: 15092414
Signature: t1:4336427:1481881555:a22b8c363f2f7a197cef192abf34bfca926ecdc2
Summary:
As the name suggest it will restore backup made by `hg pushbackup`.
If user has only one backup for the `dest` repo then it will be restored.
But user may have backed up many local repos that points to `dest` repo.
These local repos may reside on different hosts or in different
repo roots. It makes restore ambiguous; `--reporoot` and `--hostname`
options are used to disambiguate.
Example situation:
1) User has only one laptop with mercurial repo `repo` that was cloned
from remote server. He or she run `hg pushbackup`. Then laptop breaks
and user gets a new one, clones the `repo` again and runs `hg restore`.
It automatically restores the backup.
2) User has devserver and laptop and backups were made from both.
Then if user decides to switch devserver and run `hg restore` on the new
devserver he or she has to specify `--hostname`.
Future plans:
1) Add `--user` option to make it possible to restore another user's backup
Test Plan: Run `test-infinitepush-*`
Reviewers: rmcelroy, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4280832
Tasks: 12479677
Signature: t1:4280832:1481565335:2a21ceafa2ff80242076a79693046514434afb40
Summary:
stderr/stdout order is different on mac. We don't need stderr at all on this
test so let's just disable it
Test Plan: Run `test-infinitepush-*`
Reviewers: #sourcecontrol, simonfar, jeroenv
Reviewed By: simonfar, jeroenv
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4326319
Tasks: 12479677
Signature: t1:4326319:1481711140:443ea2ab92391fa6e4671280441b8f71769650d8
Summary:
The "parsing getdb.sh's output and cut -d" approach is not the best practise of
passing multiple values between shell scripts. Use "source" and setting shell
variables instead.
Besides, skip the test (exit 80) if getdb.sh does not exist
Test Plan:
Provide a good getdb.sh and test on my devserver.
Delete getdb.sh and make sure the test is skipped.
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4262570
Signature: t1:4262570:1480668031:cd5371c620889856ff892d671d1ee498438b8487
Summary:
infinitepush has been storing bundle1 with changegroups v1. Turned out that in
some cases generated bundles can become huge because v1 changegroups don't
support generaldelta. It makes sense to support all changegroup versions (v1 for BC,
v2 for new infinitepush bundles, and maybe later we'll switch to v3).
To do this we have to store bundle2 instead of bundle1 because bundle1
supports only changegroups v1.
Test Plan: Run `test-infinitepush-*`
Reviewers: rmcelroy, mitrandir, quark, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4299151
Tasks: 12479677
Signature: t1:4299151:1481564714:810be69447d0b35aa57328c60aab72ad374e994d
Summary:
Fix the tests so they pass when running from an install directory. The files
may be inside python/site-packages/ rather than an fb-hgext/ directory.
Test Plan:
Successfully ran the tests from inside an RPM install directory as part of an
RPM build.
Reviewers: ikostia, stash, durham, mjpieters
Reviewed By: mjpieters
Subscribers: net-systems-diffs@, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4298095
Signature: t1:4298095:1481190643:79e9d612cbc8021c292d131bcdad1a6749e63f80
Summary:
@simon suggested to rename `debugbackup` because `debug-` prefix
implies that command is internal only and it's not safe for user to run it.
This is not the case for the backup because user can safely run it.
Test Plan: Run `test-infinitepush-*`
Reviewers: simon, rmcelroy, mitrandir, durham, mjpieters
Reviewed By: mjpieters
Subscribers: simon, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4292516
Tasks: 12479677
Signature: t1:4292516:1481213170:a03c3eaebd3459887bbb3ff1c80f20977139cc90
Summary:
`hg push` takes a lock because it updates phases and may receive other bundle2
parts from remote server. We don't need it in debugbackup. And since
debugbackup is intended to run in background we want to avoid taking locks.
Instead let's use raw `unbundle()` wireproto method and disable pushback
bundle2 parts.
Test Plan: Run `test-infinitepush-*`
Reviewers: durham, rmcelroy, mitrandir, quark
Reviewed By: quark
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4291740
Tasks: 12479677
Signature: t1:4291740:1481118369:1b20f771c5db6e4fed4fc18681cc66ef4e91dd3a
Summary: Move the enabling of unstable changesets from fbamend to allowunstable, keeping all of the wrapping in one place.
Test Plan: Unit test still passes. No change in functionality.
Reviewers: #sourcecontrol, quark, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4288233
Signature: t1:4288233:1481073858:a7432a9941e93fcadf5ad6502630a49c4e1207f7
Summary: This change enables "allowunstable" for the `hg rebase` command, which lets it create unstable changesets without full evolution enabled.
Test Plan: Create a stack of commits and attempt to rebase a commit in the middle of the stack, using a command like `hg rebase -r REV1 -d REV2` without the `--keep` flag. This will now work, and the old version of the rebased commit (with descendants) will render as an X in smartlog.
Reviewers: #sourcecontrol, quark, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4287989
Signature: t1:4287989:1481074215:80103f6bfd150e51cc152bc257146d3f9369b175
Summary:
fastannotate has a fastpath, when the node (commit hash) exists in revmap,
it would generate the output instantly from the linelog without going
through the history.
In the "fctx" mode, the vanilla annotate command would pass the revision
number instead of changeset node to fastannotate, which prevents
fastannotate from using the fast path.
This diff converts the revision number to node before testing the fast path
to solve the problem.
Test Plan: Modified existing tests.
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4288927
Signature: t1:4288927:1481073363:3330d60878161139ee195bc968c161d6c0d95ace
Summary:
The `isuptodate` check could still be expensive because of reading the manifest
takes time. Let's add a config option to skip the up-to-date checking to avoid
expensive manifest reading server-side.
Test Plan: Added a new test
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4287289
Signature: t1:4287289:1481071706:5a3100f1513cd01ae7f78c1dfc9da9f5a3ba79cd
Summary:
We could've avoided creating this diff if bundlerepo supported many bundles.
Unfortunately it doesn't. Adding support requires changes in upstream mercurial and it can take a lot of time.
This implementation wraps `listkeys` to set phases properly and `exchange._changegrouppart` to send more
than one bundle at a time.
It has unsolved problem with phases. For example in this situation
# Publishing server creates commit A (it is marked as draft on the server).
# Client pulls commit A from publishing server (in this case commit A is marked on the client but it's still draft on the server).
# Another client make pull from scratch bookmark. Commit A will be draft on client because we delete publishing = True from listkeys.
We assume that this case is quite rare.
Test Plan: Run `test-infinitepush-*`
Reviewers: mitrandir, rmcelroy, quark, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4074889
Tasks: 12479677
Signature: t1:4074889:1480446669:97e7f1d8ad23d457d7984a4cde0efb2b6e89eb2e
Summary:
An extension to pre-fill your commit template based on your own
previous commit. Useful for stacked diffs where the commits are mostly related
and will have mostly the same reviewers, tasks, etc.
Test Plan: dogfooding + tests
Reviewers: #sourcecontrol, mjpieters, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4265616
Tasks: 12218809
Signature: t1:4265616:1480946835:ed18fadbd9068f2d2ec4799fa1ac176132b41e4b
Summary: Looks like these paths were meant to be globbed out anyway.
Test Plan: run-tests -l test-treemanifest-noflat.t
Reviewers: rmcelroy, durham, ikostia, stash
Reviewed By: ikostia, stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4276576
Signature: t1:4276576:1480944202:c88d8aa8bd4ceda0b02f0babcc719b9d9ae3eb05
Summary:
With the default shelve extension being unified as .patch,
cleanupoldbackups now also checks for times on .patch files instead
of .hg ones. Thus, we need to adjust tests to touch the right files.
Test Plan: run tests locally, see that they pass.
Reviewers: #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4276322
Summary:
We want to backup bookmarks even if no new commits were added. To do this let's
save the hash of all the backuped bookmarks and backup only if saved hash is
different from the hash of the current bookmarks.
In the case when no new commits need to be backuped revs=['null'] is passed
(remotenames does the same in `push --delete`).
Test Plan: Run `test-infinitepush-*`
Reviewers: durham, rmcelroy, mitrandir, quark
Reviewed By: quark
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4251589
Tasks: 12479677
Signature: t1:4251589:1480611524:6faaa9d6d1b1aba9f7a45a4bf9a4c0df3d039ebe
Summary:
This adds a few tests that ensure that flat manifests are not being read when
treemanifests are enabled. Any exception to this rule is documented in the test
and commented on.
The test caught a bug where we were not returning a simple treemanifest when the
node was the nullid. While the performance of an empty flat manifest is similar
to the performance of a empty treemanifest, the empty flat manifest may have
forced us to not use the treemanifest in operations that compare two manifests
(like diff).
Test Plan: Ran the test.
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4262975
Signature: t1:4262975:1480706157:4d4913bf032ebf44105a71a27fec5ee1a36b015b
Summary:
This adds a test for hg repack --incremental handling tree packs. It fixes a few
bugs that were caught in the process:
1. Since remotefilelog was being imported via it's file path, it was being
loaded into the process as hgext_remotefilelog. When treemanifest loaded it into
the process via 'import remotefilelog' it was being imported as 'remotefilelog'.
This meant we had the same types imported into the same process with two
different names, which meant 'isinstance()' checks could fail when they
shouldn't (which affects incremental repacks). So we just drop the filepath.
2. We need to allow repacking local tree manifest data even if the full delta
chain isn't available (since part of the delta chain may be in the cache). So we
add allowincomplete to the data pack in this case.
Test Plan: Ran it
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4262412
Signature: t1:4262412:1480706110:45bb0a1e1b031f9cfd4658a5071bbac5df2f6543
Summary:
Adds a test for making sure hg repack handles local pack files in
.hg/store/packs
Test Plan: Ran it
Reviewers: #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4261788
Signature: t1:4261788:1480705839:efb9657654bf6d31a3ec1d1125277174cbc93276
Summary:
This adds a simple test that verifies hg repack will pack two tree manifest
packs into one.
It caught a bug where creating a treemanifest for a commit with a null parent
produced incorrect output because it constructed an empty tree and tried to use
it's node as the parent of the delta, when there should not have been any delta
in the first place. This is fixed by this diff as well.
Test Plan: Ran the new test
Reviewers: #mercurial, dsyang, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4261591
Signature: t1:4261591:1480705822:ef21fb8cebd8b89f92f58f11bb1dab59bf97664d
Summary:
Previously, _converttohybridmanifest would always create a new hybrid manifest
with the same node as the original. This meant that some code paths would
attempt to use the treemanifest from the node, instead of the already prepared
matches result. This meant the output could contain all the values from the
original tree, instead of just the matches output.
This is actually a regression from 98ba34a5194c09. Prior to that, matches did
not reuse the node.
Test Plan: Manually inspected the results in the debugger during a rebase.
Reviewers: #mercurial, ikostia
Reviewed By: ikostia
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4247821
Signature: t1:4247821:1480499268:27f4a1b92ecf5d10009996b5b8f22bac02f3f38e
Summary:
This fixes test-check-config-hg.t for tweakdefaults. And did some clean-up
for other minor issues.
I was trying to implement another feature (along with the clean-up) in
tweakdefaults and finally realized it's infeasible and drop the feature. But
the clean-up seems useful thus sent here.
Also change `cp -r` to `cp -R` to pass the usptream check-code test.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, mitrandir
Reviewed By: mitrandir
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4253852
Signature: t1:4253852:1480613350:398e9b234fcc2360dcb8a3e3ad4e5bc5c4377857
Summary:
If the annotate cache is up-to-date on the main branch, there is likely no need
to prefetch file contents, unless the user is annotating a side branch, which
requires a deeper integration between fastannotate and remotefilelog to just
prefetch the missing part correctly - I'll think about it later while this diff seems
good enough for common cases.
Test Plan: Added a new test
Reviewers: durham, #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4256370
Signature: t1:4256370:1480612767:c2e2a9225fb63ff36ffb579f9641851eb8cd8b39
Summary:
Simplifies some shell code, namely `$extpath` is duplicated with `$TESTDIR/..`
so remove one of them.
Change `library.sh` (used by remotefilelog tests) to reuse `$HGRCPATH` instead
of overriding the environment variable, because it never gets restored and the
current code would assume `$HGRCPATH` to have `.hgrc` as its basename, which is
not always true.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4256506
Signature: t1:4256506:1480612191:73832c427b6562b9000a6df006226653157a7156
Summary:
If the history of a file is short, it would be easier to just build the
linelog locally, instead of downloading it from the server.
This diff adds a "threshold" config option to make it possible for the client
to skip downloading linelogs.
A special case is, the file only has a single revision, where we can just avoid
all SSH connections - no fileservice peer, and no annotate peer.
Test Plan:
Add a test case for testing filelog repo.
Note: remotefilelog repo is not tested for this feature yet because there is
other planned changes related to remotefilelog.
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4255483
Signature: t1:4255483:1480611801:4eb79c66eaf1fb5436d550f13c7a78d09b9a1ee2
Summary:
Previously fastannotate client will always fetch linelog from the server,
regardless of how long the history of a file is.
It actually makes sense for fastannotate client to build up the annotate
cache on its own, if, for example, the file only has one single revision,
where we don't even need any kind of SSH connection.
Test Plan: Run existing tests
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4254602
Signature: t1:4254602:1480611745:a44e70495771cb4f5c508ebb05fcfe8b0bb4d53f
Summary:
Backuping changesets is not enougn for restore. We also want to backup heads
and bookmarks to completely capture the state of the repo.
This diff is the first step in implementing this functionality. It adds new
bundle2 part which contains encoded dict (simple json encoding is used).
If value in the dict is empty then key is the bookmark pattern to delete.
If value is not empty then key is the bookmark name to save and value is a
node hash. The reason to put them in the same part is to make it possible to
delete and insert into indexapi in one transaction. It's also possible to pass
patterns to delete in part parameters but there is a bug in upstream hg that
limits parameters' size to 256 and we can potentially have longer bookmarks.
Local bookmarks are saved in infinitepush in the following form:
infinitepush/backups/USERNAME/HOSTNAME/REPOROOT/bookmarks/LOCAL_BOOKMARK_NAME
Local heads are saved in infintiepush in the following form:
infinitepush/backups/USERNAME/HOSTNAME/REPOROOTheads/HEAD_HASH
Hostname, username and repo root is necessary to distinguish different backups.
Test Plan: Run `test-infinitepush-*`
Reviewers: durham, rmcelroy, mitrandir, quark
Reviewed By: quark
Subscribers: quark, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4245080
Tasks: 12479677
Signature: t1:4245080:1480518959:aa199d67fac4e2cd2f543651ff56fdd649dac729
Summary:
Not setting all default options may result in KeyValue errors.
For example pushvars extensions does
repo._shellvars = opts['pushvars']
It results in failure if 'pushvars' is not set.
Let's fix it by explicitly setting default values.
Also we need to set `allow_anon` to remotenames because otherwise
debugbackup will fail.
Test Plan:
Run `test-infinitepush-*`
Run `hg debugbackup` inside fbsource make sure there were no failures
Reviewers: rmcelroy, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4237230
Tasks: 12479677
Signature: t1:4237230:1480446802:36e0630ce3e60c947be47d83bbc6deff8624048e
Summary:
Previously, if there was no history available for a given revision, we would
just store the full text in the pack file. This patch makes it attempt to reuse
the existing delta base instead. This will be useful for repacking
treemanifests, since we currently don't have histpacks for them (we just delta
them efficiently when they are first added to the repo).
Test Plan: Ran tests
Reviewers: #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D4240705
Summary:
Previously, when we wrote each tree entry into a pack file, it wasn't delta'd in
any way. This patch makes it store the delta against p1 in the pack file.
Testing in a large repo shows this reduces tree pack size by about 22x.
Test Plan:
Ran the tests. Did a pull in a large repo and saw the pack file was
22x smaller than before (and still usable).
Reviewers: #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D4202088
Summary:
Core mercurial sorts p1 and p2 before computing the hash, so it's deterministic.
We need to do the same.
Test Plan: Ran the tests, saw a hash changed
Reviewers: #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D4202063
Summary:
Previously, all prefetching work is done by the "fastannotate" command and
we may replace the "annotate" command with "fastannotate" and we run into
issues dealing with command line option differences.
Now that we have the "fctx" mode, do not redirect "annotate" to
"fastannotate" but wrap "annotate" in a lightweight way so it's responsible
for downloading the annotate cache.
Test Plan: Added a new test case
Reviewers: #sourcecontrol, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4239607
Signature: t1:4239607:1480447754:a84268a9e64c678c1f275bc04c44a6f4138eafe6
Summary:
Previously, I have performance concern about the signature of
`fctx.annotate`: it returns fctxs, not ideal for performance since my
initial goal is to get rid of reading revlogs in the best case. And letting
the low-level `fctx.annotate` have some side effects writing to the disk is
not that pretty.
Because of that, fastannotate had re-invent part of the formatter (also
optimized for performance somehow), and still cannot support all the
features the original annotate supports, namely, templates.
Now it makes sense to just replace `fctx.annotate` with a sub-optimal
implementation, just for the flexibility of the template support of the
original annotate command. We can use a "fake" or "lazy" fctx object to
minimal the performance impact when people only need to print changeset
nodes and line numbers - in which case we don't read revlog.
Actually, the hgweb support already did a similar thing - converting
fastannotate output to annotate output. So we can reuse some code.
The next planned steps are:
- Make the original "annotate" command aware of fastannotate protocol, do
the pre-download stuff (downloading cache files at `fctx.annotate` is
possible but inefficient because of not being batched).
- "fastannotate" command remains a separated command optimized for perf
and provides extra features like "--deleted".
Because of the plan, the "commands" option does not make much sense -
"fastannotate" command will not replace "annotate" directly thus dropped
from the config interface. A new "modes" config option is added to control
how fastannotate works.
Test Plan: Modified existing tests
Reviewers: #sourcecontrol, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4238998
Signature: t1:4238998:1480447679:c48e0e565663c086293265e104d9cf414d913aa7
Summary:
Fix the test to abort if getdb.sh does not exist (or otherwise fails), rather
than ignoring the failure and continuing to try and run the remainder of the
test anyway. Otherwise mysql will hang forever.
The run-tests.py code unfortunately can't even kill the test properly in this
situation, so even with a --timeout argument it still hangs forever.
Test Plan:
Ran the tests without getdb.sh, and confirmed it failed quickly instead of
hanging.
Reviewers: durham, rmcelroy, stash
Reviewed By: stash
Subscribers: net-systems-diffs@, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4242014
Signature: t1:4242014:1480408476:8e8765ca4567a0436fba0778938f935f87287747
Summary:
Using `testedwith = 'internal'` is not a good habit [1]. Having it
auto-updated in batch would also introduce a lot of churn. This diff makes
them "ships-with-fb-hgext". If we do want to fill the ideal "testedwith"
information, we could put it in a centric place, like a "fbtestedwith"
extension rewriting those "ships-with-fb-hgext" on the fly.
Maybe having in-repo tags for tested Mercurial releases is also a good idea.
[1]: www.mercurial-scm.org/repo/hg/rev/2af1014c2534
Test Plan: `arc lint`
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4244689
Signature: t1:4244689:1480440027:3dc18d017b48beba1176fbfd120351889259eb4b
