Summary:
This fixes test-check-config-hg.t for tweakdefaults. And did some clean-up
for other minor issues.
I was trying to implement another feature (along with the clean-up) in
tweakdefaults and finally realized it's infeasible and drop the feature. But
the clean-up seems useful thus sent here.
Also change `cp -r` to `cp -R` to pass the usptream check-code test.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, mitrandir
Reviewed By: mitrandir
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4253852
Signature: t1:4253852:1480613350:398e9b234fcc2360dcb8a3e3ad4e5bc5c4377857
Summary:
If the annotate cache is up-to-date on the main branch, there is likely no need
to prefetch file contents, unless the user is annotating a side branch, which
requires a deeper integration between fastannotate and remotefilelog to just
prefetch the missing part correctly - I'll think about it later while this diff seems
good enough for common cases.
Test Plan: Added a new test
Reviewers: durham, #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4256370
Signature: t1:4256370:1480612767:c2e2a9225fb63ff36ffb579f9641851eb8cd8b39
Summary:
Simplifies some shell code, namely `$extpath` is duplicated with `$TESTDIR/..`
so remove one of them.
Change `library.sh` (used by remotefilelog tests) to reuse `$HGRCPATH` instead
of overriding the environment variable, because it never gets restored and the
current code would assume `$HGRCPATH` to have `.hgrc` as its basename, which is
not always true.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4256506
Signature: t1:4256506:1480612191:73832c427b6562b9000a6df006226653157a7156
Summary:
The upcoming remotefilelog change would require some cleanups to avoid code
duplication. Namely:
- Add a `fctxannotatecontext` so we can easily get a `annotatecontext` from
within `filectx.annotate` or `remotefilectx.annotate`.
- Print something instead of ignoring errors silently in `_fctxannotate`.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4256469
Signature: t1:4256469:1480612106:63756d64014a5b9e798bce52eda0d3e7368d6cba
Summary:
If the history of a file is short, it would be easier to just build the
linelog locally, instead of downloading it from the server.
This diff adds a "threshold" config option to make it possible for the client
to skip downloading linelogs.
A special case is, the file only has a single revision, where we can just avoid
all SSH connections - no fileservice peer, and no annotate peer.
Test Plan:
Add a test case for testing filelog repo.
Note: remotefilelog repo is not tested for this feature yet because there is
other planned changes related to remotefilelog.
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4255483
Signature: t1:4255483:1480611801:4eb79c66eaf1fb5436d550f13c7a78d09b9a1ee2
Summary:
Previously fastannotate client will always fetch linelog from the server,
regardless of how long the history of a file is.
It actually makes sense for fastannotate client to build up the annotate
cache on its own, if, for example, the file only has one single revision,
where we don't even need any kind of SSH connection.
Test Plan: Run existing tests
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4254602
Signature: t1:4254602:1480611745:a44e70495771cb4f5c508ebb05fcfe8b0bb4d53f
Summary:
Backuping changesets is not enougn for restore. We also want to backup heads
and bookmarks to completely capture the state of the repo.
This diff is the first step in implementing this functionality. It adds new
bundle2 part which contains encoded dict (simple json encoding is used).
If value in the dict is empty then key is the bookmark pattern to delete.
If value is not empty then key is the bookmark name to save and value is a
node hash. The reason to put them in the same part is to make it possible to
delete and insert into indexapi in one transaction. It's also possible to pass
patterns to delete in part parameters but there is a bug in upstream hg that
limits parameters' size to 256 and we can potentially have longer bookmarks.
Local bookmarks are saved in infinitepush in the following form:
infinitepush/backups/USERNAME/HOSTNAME/REPOROOT/bookmarks/LOCAL_BOOKMARK_NAME
Local heads are saved in infintiepush in the following form:
infinitepush/backups/USERNAME/HOSTNAME/REPOROOTheads/HEAD_HASH
Hostname, username and repo root is necessary to distinguish different backups.
Test Plan: Run `test-infinitepush-*`
Reviewers: durham, rmcelroy, mitrandir, quark
Reviewed By: quark
Subscribers: quark, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4245080
Tasks: 12479677
Signature: t1:4245080:1480518959:aa199d67fac4e2cd2f543651ff56fdd649dac729
Summary:
Right now `hg debugbackup` backups only changesets. We also want to backup
bookmarks. Since user may have many bookmarks let's add a separate method
that will save many bookmarks at once.
Test Plan: Will be tested in next diffs
Reviewers: durham, rmcelroy, mitrandir, quark
Reviewed By: quark
Subscribers: quark, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4244823
Tasks: 12479677
Signature: t1:4244823:1480596116:7114128dafe2b8a10f599d05d68c1a6dce522f4a
Summary:
This methods accepts list of bookmark patterns to delete (patterns are not
supported yet, but they will be supported).
Test Plan: Will be tested in subsquent diffs
Reviewers: andrasbelo, rmcelroy, durham
Reviewed By: rmcelroy, durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4176016
Tasks: 12479677
Signature: t1:4176016:1479144865:8c9121970878d4d462cc2c679a414999dfdce1f4
Summary:
Not setting all default options may result in KeyValue errors.
For example pushvars extensions does
repo._shellvars = opts['pushvars']
It results in failure if 'pushvars' is not set.
Let's fix it by explicitly setting default values.
Also we need to set `allow_anon` to remotenames because otherwise
debugbackup will fail.
Test Plan:
Run `test-infinitepush-*`
Run `hg debugbackup` inside fbsource make sure there were no failures
Reviewers: rmcelroy, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4237230
Tasks: 12479677
Signature: t1:4237230:1480446802:36e0630ce3e60c947be47d83bbc6deff8624048e
Test Plan:
Without this change, running `hg commit` in Eden was failing with the following:
```
AttributeError: 'shallowrepository' object has no attribute 'getsparsepatterns'
```
This is because our Eden mount is *shallow*, but not *sparse*, and
`shallowrepository` has a `sparsematch` method, so it yielded a false positive
for this check in `sparse.py`:
```
if util.safehasattr(repo, 'sparsematch'):
```
Because the first thing that is called after this check is:
```
_, _, profiles = repo.getsparsepatterns(ctx.rev())
```
it seems more appropriate to test for the `getsparsepatterns` attribute, anyway.
Reviewers: simpkins, durham
Differential Revision: https://phabricator.intern.facebook.com/D4248940
Summary:
Previously hg repack would only repack file content pack files. This patch makes
it also repack tree manifest pack files.
Test Plan:
Ran pull repack in a repo and verified the manifest packs were
repacked. I'll add some tests around this at some point.
Reviewers: #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D4240723
Summary:
Previously, if there was no history available for a given revision, we would
just store the full text in the pack file. This patch makes it attempt to reuse
the existing delta base instead. This will be useful for repacking
treemanifests, since we currently don't have histpacks for them (we just delta
them efficiently when they are first added to the repo).
Test Plan: Ran tests
Reviewers: #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D4240705
Summary:
The recently added __nonzero__ function was not handling the possibility of
pyexceptions. This adds the appropriate catch.
A pyexception is where somewhere up the stack something has set the python error
string, and we just need to return the appropriate error value from the top
level c api to indicate an error happened.
Test Plan: none. I just caught this while debugging
Reviewers: #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D4202090
Summary:
Previously, when we wrote each tree entry into a pack file, it wasn't delta'd in
any way. This patch makes it store the delta against p1 in the pack file.
Testing in a large repo shows this reduces tree pack size by about 22x.
Test Plan:
Ran the tests. Did a pull in a large repo and saw the pack file was
22x smaller than before (and still usable).
Reviewers: #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D4202088
Summary:
As part of enabling deltas in our tree pack files, we need NewTreeIter to return
the p1 and p2 Manifests as well. To do this we need to return a Manifest/Node
tuple. Since this is becoming a common structure, let's define a type for it and
change NewTreeIter to use it for it's return values.
Test Plan:
Ran the tests. With a future diff I built a pack file and verified it
was small because of deltas.
Reviewers: #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D4202080
Summary:
Core mercurial sorts p1 and p2 before computing the hash, so it's deterministic.
We need to do the same.
Test Plan: Ran the tests, saw a hash changed
Reviewers: #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D4202063
Summary:
Previously, all prefetching work is done by the "fastannotate" command and
we may replace the "annotate" command with "fastannotate" and we run into
issues dealing with command line option differences.
Now that we have the "fctx" mode, do not redirect "annotate" to
"fastannotate" but wrap "annotate" in a lightweight way so it's responsible
for downloading the annotate cache.
Test Plan: Added a new test case
Reviewers: #sourcecontrol, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4239607
Signature: t1:4239607:1480447754:a84268a9e64c678c1f275bc04c44a6f4138eafe6
Summary:
Previously, I have performance concern about the signature of
`fctx.annotate`: it returns fctxs, not ideal for performance since my
initial goal is to get rid of reading revlogs in the best case. And letting
the low-level `fctx.annotate` have some side effects writing to the disk is
not that pretty.
Because of that, fastannotate had re-invent part of the formatter (also
optimized for performance somehow), and still cannot support all the
features the original annotate supports, namely, templates.
Now it makes sense to just replace `fctx.annotate` with a sub-optimal
implementation, just for the flexibility of the template support of the
original annotate command. We can use a "fake" or "lazy" fctx object to
minimal the performance impact when people only need to print changeset
nodes and line numbers - in which case we don't read revlog.
Actually, the hgweb support already did a similar thing - converting
fastannotate output to annotate output. So we can reuse some code.
The next planned steps are:
- Make the original "annotate" command aware of fastannotate protocol, do
the pre-download stuff (downloading cache files at `fctx.annotate` is
possible but inefficient because of not being batched).
- "fastannotate" command remains a separated command optimized for perf
and provides extra features like "--deleted".
Because of the plan, the "commands" option does not make much sense -
"fastannotate" command will not replace "annotate" directly thus dropped
from the config interface. A new "modes" config option is added to control
how fastannotate works.
Test Plan: Modified existing tests
Reviewers: #sourcecontrol, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4238998
Signature: t1:4238998:1480447679:c48e0e565663c086293265e104d9cf414d913aa7
Summary:
Fix the test to abort if getdb.sh does not exist (or otherwise fails), rather
than ignoring the failure and continuing to try and run the remainder of the
test anyway. Otherwise mysql will hang forever.
The run-tests.py code unfortunately can't even kill the test properly in this
situation, so even with a --timeout argument it still hangs forever.
Test Plan:
Ran the tests without getdb.sh, and confirmed it failed quickly instead of
hanging.
Reviewers: durham, rmcelroy, stash
Reviewed By: stash
Subscribers: net-systems-diffs@, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4242014
Signature: t1:4242014:1480408476:8e8765ca4567a0436fba0778938f935f87287747
Summary: In my rush to get the allowunstable extension landed before the holidays, I made several embarrassing typos in the doc comment at the top of the extension. This diff fixes the typos and makes the comment more clear.
Test Plan: The only change was in a comment, so this is a no-op. Unit tests still pass for good measure.
Reviewers: durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4242510
Signature: t1:4242510:1480399263:a9822b5e762f1a94c322b15c2ddec9c1acb7df78
Summary:
Using `testedwith = 'internal'` is not a good habit [1]. Having it
auto-updated in batch would also introduce a lot of churn. This diff makes
them "ships-with-fb-hgext". If we do want to fill the ideal "testedwith"
information, we could put it in a centric place, like a "fbtestedwith"
extension rewriting those "ships-with-fb-hgext" on the fly.
Maybe having in-repo tags for tested Mercurial releases is also a good idea.
[1]: www.mercurial-scm.org/repo/hg/rev/2af1014c2534
Test Plan: `arc lint`
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4244689
Signature: t1:4244689:1480440027:3dc18d017b48beba1176fbfd120351889259eb4b
Summary: Attack of the trivial missing space.
Test Plan: Run run-tests.py -l test-sqldirstate-shelve.t to see it pass
Reviewers: rmcelroy, simonfar, quark, jeroenv
Reviewed By: quark, jeroenv
Subscribers: mjpieters, #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D4244663
Signature: t1:4244663:1480439601:3ab7e306e618f7e6e2372046f00e85cf1eb3826d
Summary:
Template lacks of the "padding" feature. There is a hard-coded 13-char wide
username, 5-char wide line number, which does not look well if the usernames
are too long or too short.
This patch restores the original annotate format which is different from
the current behavior. For example, given "-uc", it shows "user node" instead
of "node:user". It also tries to make the template version as close as the
non-template one, by adjusting the order, changing separators, and making
"-c" optional (could be hidden from the output).
Test Plan: Modified existing tests
Reviewers: #sourcecontrol, ikostia
Reviewed By: ikostia
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4243830
Signature: t1:4243830:1480437984:1c679c9db516bab905d027061326a7a1524922c6
Summary: See rd9d8d78e in the hg repository; cmdutil.service moved to server.runservice
Test Plan: Run test-fbconduit.t and observe it passing once more.
Reviewers: simonfar, rmcelroy, quark
Reviewed By: quark
Subscribers: mjpieters, #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D4244268
Signature: t1:4244268:1480432981:8e06beadba912294e6d1a01d93b13848cb16a096
Summary: Yay for arbitrary single space changes.
Test Plan: Run the two affected tests to show they now pass again.
Reviewers: simonfar, rmcelroy, quark, stash
Reviewed By: stash
Subscribers: mjpieters, #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D4244213
Signature: t1:4244213:1480431919:ba4837619652023005faeb00a93a351c7ca0411a
Summary:
The debugindex and debugindexdot commands have moved and are not registered
unless you import the new mercurial.debugcommands module.
Test Plan:
Run
hg --config=extensions.remotefilelog=fb-hgext/remotefilelog help
and observe that you get help info, rather that the error
hg: unknown command 'debugindex'
then run the fb-hgext test suite.
Reviewers: rmcelroy, quark, simonfar
Reviewed By: simonfar
Subscribers: mjpieters, #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D4244047
Signature: t1:4244047:1480427216:dcaa1ca441ea189bdf68f1f619b4078d8c1d09dc
Summary:
D4233433 changed commands to rebuild the broken cache but missed
wireprotocol. Let's do it for wireprotocol as well.
Test Plan: Added a new test
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4237855
Signature: t1:4237855:1480421375:d5715d61a3c66ec8be5a6a80041ee6204e5e4378
Summary: Currently, the reason we can't perform operations such as histedit, split, and fold in the middle of a stack is that we do not have the experimental.evolution=allowunstable option enabled. It turns out all of these commands will do the "right thing" if this option is enabled. However, since we are uncertain of the ramifications of enabling this option globally, this new extension turns the option on for the duration of split, fold, and histedit, allowing those commands to be used in the middle of a stack without config changes.
Test Plan: See unit tests. Basically, create a stack and then attempt to use split, fold, or histedit somewhere in the middle.
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4209315
Tasks: 14598405
Signature: t1:4209315:1479744780:811e8bef02149a055376c0f105ac2d389dd60970
Summary: Add template keywords for split, folded, and histedited successors. This will enable us to show "(Histedited as XXX)" annotations in smartlog.
Test Plan: By itself this should be a no-op. All unit tests still pass. The effects of this aren't visible unless a user uses a version of our facebook.rc config file that contains updated smartlog templates. See D4209113.
Reviewers: durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4209314
Signature: t1:4209314:1479744837:8eeeeb7a031abfabeb302abde9e4b1f3d789a479
Summary:
It's common that the client's "master" ("main branch") falls behind the
server's, in which case the "is-up-to-date" check will fail and trigger a
re-download while it's unnecessary. Add some checks to avoid downloading
the cache for the client in this case.
Test Plan: Added a test
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4233534
Signature: t1:4233534:1480330716:db995d0fa5c6a2371894fc2a7871277da23608a9
Summary:
Previously, fastannotate's annotatecontext would delete individual cache
files (linelog and revmap) if it meets Exception. While the "fastannotate"
command has some retry logic, "debugbuildannotatecache" doesn't, which means
users have to run "debugbuildannotatecache" O(number-of-corrupted-files)
times to unblock. And "fastannotate" does not handle AssertionError which
could be seen as "linelog is corrupted" nicely - it crashes without retry.
Addressing all the above issues by replacing the "assert" to
"CorruptedFileError", catching them and retry properly.
Test Plan: Added a new test.
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4233433
Signature: t1:4233433:1480330548:618e80762af2209784ad72ad0658f649b050a080
Summary:
The recent bdiff behavior change affects us. Document why and update the test
so it works with both the old and new (less optimal imo) bdiff behavior.
Test Plan: Updated the test
Reviewers: #sourcecontrol, simonfar, rmcelroy
Reviewed By: simonfar, rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4231966
Signature: t1:4231966:1480011763:d116799ded3514d644e4b80296ffc71a8d712a43
Summary:
Tests failed on mac. My guess is that timeout is too low.
Let's increase it to 10 seconds (it's fine because we want wait for 10 sec
if debugbackup finishes faster).
Also let's add logging
Test Plan: Run `test-infinitepush-*`
Reviewers: #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4213330
Tasks: 12479677
Summary:
`backup` was renamed to `debugbackup` a few commits ago.
Rename it there too
Test Plan: Run `test-infinitepush-*`
Reviewers: #sourcecontrol, andrasbelo
Reviewed By: andrasbelo
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4212539
Tasks: 12479677
Signature: t1:4212539:1479721307:1ac25c7dc8951d655910e2b0c41b2c9555353f74
Summary:
`hg backup --background` will be used as a `txnclose` hook to backup all of the
local commits to infinitepush.
Test Plan: Run `test-infinitepush-*`
Reviewers: rmcelroy, mitrandir, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4175953
Tasks: 12479677
Signature: t1:4175953:1479145307:e698903b519361b376f6e182db7c49869c992617
Summary:
I'm going to use runshellfast in infinitepush.
To avoid copy-paste let's move it to the separate package.
Note: fastmanifest also has runshellfast but it's implementation
is a bit different and seems that it doesn't work with remotefilelog.
So I'll leave it alone for now.
Test Plan: python run-tests.py -j20
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4175836
Summary:
During `hg backup` bundle with many heads may be pushed. Let's support it too.
Many heads bundle is not allowed only when we are pushing it with `--to`
because in this case we don't know the node for the bookmark.
Test Plan: Run `test-infinitepush-*`
Reviewers: rmcelroy, durham
Reviewed By: durham
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4175280
Tasks: 12479677
Signature: t1:4175280:1479231056:fb0c47a7752319d77a6cfc83a29c57e9e2dced16
The main manifestctx implementation has a find function, so the hybrid one needs
it as well. I tried to add a test for this, but the only known case is in a
weird situation where there are tags, but no tag caches and some other criteria
I haven't figured out yet. So I gave up.
Summary:
This could help people understand what's different essentially, and make better
decision on what to use.
Test Plan: Seems the vim spell checker is happy.
Reviewers: #sourcecontrol, rmcelroy, stash
Reviewed By: rmcelroy, stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4203039
Signature: t1:4203039:1479461745:9cce0239088707bd8b66055ba66c420e97a3bd03
This is to fix the following error discovered by pyflakes:
fastannotate/context.py:407: list comprehension redefines 'f' from line 352
The change is necessary after upstream changeset 4173d21a33de, which enables
the check.
Summary:
We were seeing some flakiness with remotefilelog prefetching in the sparse
integration test. Let's make the test more explicit by having it explicitly
prefetch exactly the files it cares about, so we shouldn't see any more
flakiness.
Also have the test refer to the in-repo sparse extension instead of the system
one, now that remotefilelog is in the same repo as sparse.
Test Plan: Ran it
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4192463
Signature: t1:4192463:1479334442:f91a99a4a026a0f71f9519155185fb484d8d68ec
