Summary:
It's a common mistake that our tests require foreign extensions (namely evolve
and remotenames) without checking them first.
This diff adds checks to catch these mistakes, adds missing checks, and unifies
our checking logic using `require-ext.sh`, which is aware of `hgext3rd` and
prints skip message.
This affects `arc lint` so hopefully our new testing code would be free of this
kind of mistakes.
Test Plan: `arc lint` would catch errors
Reviewers: #mercurial, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3550977
Signature: t1:3550977:1468455857:e849dfd9e3cbc446cc6e6c662050ee88a3366e6c
Summary: Rather than have fastmanifestcache as a singleton, we attach it to each opener.
Test Plan: pass existing unit tests.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3553542
Tasks: 12169797
Signature: t1:3553542:1468373683:b0a7fcdf5dbf12046e5b4be9aa0bc5d46e55ce06
Summary: `prune` is essentially `makeroomfor(0, set())`.
Test Plan: pass existing unit tests. output is slightly different, but that's just because we no longer output that debugging line.
Reviewers: #fastmanifest, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3547069
Signature: t1:3547069:1468352305:6185132648f871e26d3cfd449c059523b5eb6d6b
Summary: It's caught by the contbuild script. I forgot to change this test.
Test Plan: Run `test-patchpython.t`.
Reviewers: #mercurial, ttung, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3550459
Signature: t1:3550459:1468349937:07123651fb103d283a382d1323d9f69a5c5d81b7
Blame Revision: D3534311
Summary:
Be a better citizen under system python path.
Fix all tests issues and change setup.py to use glob pattern to include
all extensions.
Test Plan:
Run tests and `make local`.
Also build and install the package and run `hg sl` in major repos.
Reviewers: #mercurial, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3534311
Signature: t1:3534311:1468275426:fe122646c8bd6c541e1889e73e9df28f86747ff2
Summary:
It's kind of silly to write the FM cache entry, and then immediately blow away the entry if it doesn't fit in the limit. Instead, just calculate if we have enough space, and raise CacheFullException if we don't.
Depends on D3538944
Test Plan: pass existing tests.
Reviewers: lcharignon, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters, quark
Differential Revision: https://phabricator.intern.facebook.com/D3545446
Signature: t1:3545446:1468283437:7e4167e3b5a13bf3d01fca45a965d7aaa8843166
Summary:
There's two issues with `setwithlimit`:
1) It does not have a docblock, nor an API that fully covers all the possible conditions.
2) It returns None if the cache entry is already present.
This modifies `setwithlimit` to return True if the cache entry ultimately makes it (whether it previously existed or the write was successful), False if the write fails, and raises CacheFullException if the cache is full.
Test Plan: pass existing tests. the already-in-cache case is a race condition that is not practical to reproduce.
Reviewers: lcharignon, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3545407
Signature: t1:3545407:1468283390:64d6cdba7a17cca9bd4465c6e03b9143e4b4cc4b
Summary:
Instead of pruning at the end, we need to prune as we are writing entries.
Depends on D3545060
Test Plan: pass test in D3545060
Reviewers: lcharignon, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3545267
Signature: t1:3545267:1468279898:5332838efa52bb55733d0e6dcf613b27e71e9d3e
Summary: Currently, this test breaks! This is awful because that means for users, when the cache becomes full, we never accept new manifests. :( :( :(
Test Plan: it breaks. :( did i say :( ?
Reviewers: lcharignon, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3545060
Tasks: 12136039
Signature: t1:3545060:1468279919:ff8928557c8ad03e06370ee75f2386ffb46f54fa
Summary:
Previously, depending on the code path, the limit specified would not actually take effect. For instance, if we came in from debugmanifestcache, and attempted to populated the cache, we would use `systemawarecachelimit` when filling the cache, and the fixedsize limit specified by the user when pruning.
With this change, we unify the all the cache limit decisions to `fastmanifestcache`. If the user actually overrides the limit, we set the limit in `fastmanifestcache` and let that make the decisions.
We also change the definitions of limit in `hg debugcachemanifest` to:
1) >0 => it's the limit.
2) =0 => use systemawarecachelimit
3) <0 => no limit!
Test Plan: pass existing unit tests. there's a small change in the test output, because we always evaluate the limit now, plus we remove the test for limit=0, since it means something different now.
Reviewers: lcharignon, durham
Reviewed By: durham
Subscribers: trunkagent, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3544997
Signature: t1:3544997:1468281604:8f78f00ebf2afd8f3f1fbefbd82316b97cc4b193
Summary:
We need a fastmanifest object in order to size it. Once we know its size, we can make room in the cache.
This slightly affects one of the tests, as we request the manifest text earlier than we previously did.
Depends on D3537904
Test Plan: used in later diff.
Reviewers: lcharignon, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters, quark
Differential Revision: https://phabricator.intern.facebook.com/D3538991
Signature: t1:3538991:1468280648:41c65d91529babe0559eac7b75509481adf2765f
Summary: This is actually, strictly speaking, an upper bound. This is because we haven't gone and done the work of compacting the tree. Since the tree may have undergone deletions, the directory nodes may be oversized.
Test Plan: used in later diff.
Reviewers: lcharignon, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters, quark
Differential Revision: https://phabricator.intern.facebook.com/D3538944
Signature: t1:3538944:1468268102:45f2cbaa3dc156352ff3f71d0c53393f2445813a
Summary: This allows us to convert a manifest once, and try to repeatedly jam it into the cache (presumably while evicting things).
Test Plan: pass existing tests.
Reviewers: lcharignon, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters, quark
Differential Revision: https://phabricator.intern.facebook.com/D3537904
Signature: t1:3537904:1468268209:6f1746d3cee5071321d830bb91efd591a31a8616
Summary: We don't need this since we loosely lock the cache worker.
Test Plan: pass existing tests.
Reviewers: lcharignon, durham
Reviewed By: durham
Subscribers: mitrandir, mjpieters, quark
Differential Revision: https://phabricator.intern.facebook.com/D3537355
Signature: t1:3537355:1468268248:ebf7de5bf5030b90039823e1111a073d7eacc447
Test Plan: ran tests, ran it with chg
Reviewers: durham, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3533231
Signature: t1:3533231:1467986141:dcc020b8cd29dc58c532690a73737a5ce149da51
Summary: @quark suggested it in D3519183. It does not work in both places because the other is not an immediate return.
Test Plan: make local && pass fastmanifest unit tests
Reviewers: lcharignon, quark
Reviewed By: quark
Subscribers: mitrandir, mjpieters, quark
Differential Revision: https://phabricator.intern.facebook.com/D3530652
Signature: t1:3530652:1467927107:99c9aeb6b873ab3cdca974f526c3984f4256e201
Summary:
Before this patch we were doing two mistakes in the ratio computation.
- We were not recording global cache hit or miss, it was always a hit because
a function is truthy is python
- We were not deduping cache miss for the same manifest multiple times
This patch fixes these two mistakes.
It also changes the logging of cache hit and miss to include the name of the
operation that triggered the cache hit or miss (diff or filesnotin).
Test Plan: Test output changed and commented
Reviewers: durham, ttung
Differential Revision: https://phabricator.intern.facebook.com/D3484658
Summary:
Before this patch we were using config instead of configbool for reading
the debugmetrics config causing "False" to be evaluated as a truthy value
for the config. This patch fixes the issue and sets the config to false for
some of the tests to reduce the noise of the output.
Test Plan: Tests pass
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3524501
Summary:
There's no need for the `setup()` to be called in `reposetup()` any more.
Depends on D3510966
Test Plan:
1) pass existing unit tests.
2) kill chg server instances, then run `USE_CHG= hg diff -c . --debug` repeatedly. (USE_CHG tells my hg wrapper to use chg)
Reviewers: quark, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3511004
Signature: t1:3511004:1467410145:0761c80da77d5a193fbdb08527fcc4aae9f8065e
Summary: Whenever fastmanifest needs an ui object, we pass it a global proxy object. Whenever someone calls `reposetup(..)`, we update the backing object.
Test Plan:
1) pass existing unit tests
2) enable chg, then run `hg diff -c . --debug`. saw fastmanifest debug output in the console.
Reviewers: lcharignon, quark
Reviewed By: lcharignon
Subscribers: trunkagent, mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3510966
Signature: t1:3510966:1467410140:88fe810d03b9a9aa6818f17eb605a70eee931364
Summary:
There are two parts to add logging to blackbox. Logging events
and changing the configuration of blackbox. This diff does the former.
We don't reuse all the metrics event as:
- blackbox does not support wildcard for events like fastmanifest-*
- they are not meant to be human readable, they are for performance logging
We instead log a few new events:
- What triggered a caching operation?
- What command we run to cache the manifest?
- What revisions are to be cached?
- What revisions are actuallt cached?
- Do we overflow the cache?
This will allow us to understand all the cases where caching didn't trigger.
Like in t11877434.
Test Plan:
Test output does not change
Verified manually by changing the blackbox configuration that events get logged:
CHGDISABLE=1 hg book food --config blackbox.track='command, commandfinish, commandexception, exthook, fsmonitor, pythonhook, fastmanifest' --config extensions.fastmanifest=/home/lcharignon/facebook-hg-rpms/fb-hgext/fastmanifest --config extensions.blackbox=/home/lcharignon/facebook-hg-rpms/hg-crew/hgext/blackbox.py
Reviewers: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3504158
Tasks: 11877434
Summary:
A hg command should have a short description, which starts with a lower case
letter and does not end with a period.
Test Plan: Run `hg` and see they are consistent with others.
Reviewers: #mercurial, ttung, mjpieters
Reviewed By: mjpieters
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3509524
Signature: t1:3509524:1467409590:c1fe1b9421598e791a7245a130306fb05ebd8763
Summary: This allows us to pass the check-code test.
Test Plan: run all tests
Reviewers: #mercurial, ttung, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3514693
Signature: t1:3514693:1467624953:99a15876c9eb78f79a05ad315f3eb9dbd0b70fb8
Summary: D3505464 modified the kwargs, but the test output was not updated.
Test Plan: passed tests.
Reviewers: lcharignon, #mercurial, rmcelroy
Reviewed By: rmcelroy
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3506179
Signature: t1:3506179:1467353060:7c7cf9fd51257f5f5aa037a779618cf6512c1dd0
Summary: If the key as always 'ratio', then all the ratios are aliased when we send off aggregate stats. That means we end up sending the last entry, rather than each individual entry.
Test Plan:
along with D3504940
run FB_HG_DIAGS= hg diff -c .:
```
"int": {
"builddate": 1467289392,
"cachehitratio": 100,
"consumed": 489,
"diffcachehitratio": -1,
"elapsed": 280,
"errorcode": 0,
"filesnotincachehitratio": 0,
"time": 1467318481
},
```
Reviewers: lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3505464
Tasks: 12019647
Signature: t1:3505464:1467320144:327e0d306b9afa90ed9fc2d704a8ca02a79f3038
Summary:
This diff removes `self.ui = ui` in metricscollector to address stale ui
issues with chg. Also clean up unnecessary sample merges since there is
only one collector per process.
Test Plan:
Run with `fastmanifest.debugmetrics=1` and confirm see lines like:
[FM-METRICS] Begin metrics
[FM-METRICS] kind: cachehitratio, kwargs: [('ratio', 100.0)]
[FM-METRICS] kind: diffcachehitratio, kwargs: [('ratio', -1)]
[FM-METRICS] kind: filesnotincachehitratio, kwargs: [('ratio', 0.0)]
[FM-METRICS] End metrics
Reviewers: #mercurial, ttung, lcharignon
Reviewed By: lcharignon
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3504442
Signature: t1:3504442:1467311563:5904bd06b210fa8550b2e1f7e85c2133ad7a1a0e
Summary:
Silence the test-check-code-hg test failure by using the new
mercurial.util.urlparse location (used to bridge Python 2 and 3).
Test Plan:
Ran all phabricator tests:
$HG/tests/run-tests.py -l test-phab*
Reviewers: #mercurial, ttung, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3503125
Signature: t1:3503125:1467283251:dd77c1dc7037d35c80c17ccf89d96c16168aa117
Summary:
With reflog replaced by journal, rename all references to it in documentation
and tools.
Test Plan: Ran both hg githelp reflog and hg backup commands
Reviewers: #sourcecontrol, ttung
Reviewed By: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3498673
Tasks: 10804988
Signature: t1:3498673:1467232649:b250f945c806a06b064c96b1c2e364294632656a
Summary:
reflog is dead, long live journal! journal is at feature parity and reflog can
go. I've marked this as a move to help find what happened in history.
Test Plan:
Run all the tests to see if anything broke:
$HGSRC/tests/run-tests.py -l
Reviewers: #sourcecontrol, ttung
Reviewed By: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3498613
Tasks: 10804988
Signature: t1:3498613:1467232574:75c80b0223c101236b4d4e0f27f46f72797ebe90
Summary:
chg runs extsetup and uisetup with dummy ui objects. Before this
patch we were relying on ui objects in extsetup and uisetup. After this
patch, we move the logic to reposetup. We do things a little differently
because reposetup can be called multiple times and extsetup and uisetup are
called only once.
This caused one problem for `hg clone` in the test as reposetup is not
called for clone operation. I adapted the test a little bit to cover this case.
Test Plan: Modified tests pass
Reviewers: mitrandir, quark, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3499176
Signature: t1:3499176:1467244335:a70d718725c1d19ab6b6feb4558a8ffd1a020ab2
Summary: Rebasing left a stray self._write() call that should have been removed.
Test Plan:
Run all journal tests:
$HGSRC/tests/run-tests.py -l test-journal*
Reviewers: #sourcecontrol, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3497981
Tasks: 10804988
Signature: t1:3497981:1467223172:686be42ad92cd11a3e105fe2bb71f2bb7ae1c7e5
Summary:
Rather than put everything into one journal file, split entries up in *shared*
and *local* entries. Working copy changes are local to a specific working copy,
so should remain local only. Other entries are shared with the source if so
configured when the share was created.
When unsharing, any shared journale entries are copied across.
Test Plan:
Run all journal tests:
$HGSRC/tests/run-tests.py -l test-journal*
Reviewers: #sourcecontrol, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3481745
Tasks: 10804988
Signature: t1:3481745:1467215718:3b6666c2b908cea279f181f5a9310c0093d0c0fe
Summary:
Note that now the default action for `hg journal` is to list the working copy
history, not all bookmarks. In its place is the `--all` switch which lists all
name changes recorded, including the name for which the change was recorded on
each line.
Depends on D3458115
Test Plan:
Run all journal tests:
$HGSRC/tests/run-tests.py -l test-journal*
Reviewers: #sourcecontrol, rmcelroy, durham
Reviewed By: durham
Subscribers: durham, mitrandir, rmcelroy, ttung, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3462793
Tasks: 10804988
Signature: t1:3462793:1466703746:8ee2b9822e39a458d64d30b160de3ecb77c623e7
Summary:
This change would have to go into remotenames directly when hg journal itself
is added to Mercurial core.
Depends on D3458077
Test Plan:
Run the included test-journal*.t tests
$HGSRC/tests/run-tests.py -l test-journal*
Reviewers: #sourcecontrol, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3458115
Tasks: 10804988
Signature: t1:3458115:1467217834:1f8421da38280ab333faa7142118de1ef0e1aed4
Summary:
This is slated to be the upstream version of reflog. It is being hashed out
here to accellerate moving from reflog to journal.
This initial revision only supports bookmarks; later diffs add remote bookmarks
and working copy changes. As such the storage format is built to support those
usecases but the commandline usage is still geared towards only needing to list
bookmarks for now.
We'll replace reflog.py with journal.py altogether, but due to the fine details
of our RPM / chef config duality reflog.py needs to stick around until we
rolled out the RPM everywhere and updated chef to follow.
Test Plan:
Run the included test-journal.t test
$HGSRC/tests/run-tests.py -l test-journal.t
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3458077
Tasks: 10804988
Signature: t1:3458077:1467215042:d3fbba283901de6abee5d2b28a43ed9293627795
Summary: ui.log expects msg[0] to be the format string, and msg[1:] to be the arguments.
Test Plan: extend test-sampling to cover the case of a formatting string + args. pass test-sampling.t
Reviewers: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3479711
