Summary:
Since metrics doesn't send a string, let's just stub out the call.
The alternative is to put a '' in every `recordsample()` call.
Test Plan: pass existing tests.
Reviewers: lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3479138
Tasks: 11900487
Signature: t1:3479138:1466787619:910917093cf0361ed181af5597ab702b936f8d1e
Summary:
1. verify that we cache entries on background.
2. verify that we don't fill the cache when we can't take the lock.
Test Plan: passed existing unit tests.
Reviewers: durham, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3471355
Signature: t1:3471355:1466719450:191070cd2d5ef93e0e1a6925a04d1a885431cbb4
Summary: --created is the opposite of --deleted.
Test Plan: meh. used in later diff.
Reviewers: lcharignon, mjpieters
Reviewed By: mjpieters
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3478780
Signature: t1:3478780:1466759469:47a3af68df2c8d550aa654f135d4f0fcf6df7851
Summary: It's actually most relevant to least relevant. Add a docblock.
Test Plan: pass unit tests
Reviewers: lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3478616
Signature: t1:3478616:1466787566:b38eceddf91c3e88341f9173e07244ac33ab6345
Summary:
To fix sqldirstate perf in case of iterating through whole
dirstate we are introducing the possibility to cache it in memory when
neccessary.
Currently we are doing it only in the case when dirstate.walk() is called.
Test Plan:
./run-sqlitedirstate-test.py --hg ~/clowncopter/ -j 30 -t 1200
~/clowncopter/tests/run-tests.py -j 30
Reviewers: #mercurial, durham, ttung, lcharignon
Reviewed By: lcharignon
Subscribers: lcharignon, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3472402
Signature: t1:3472402:1466807592:a28bc4662cc157547b8b747c6be98faae35c63b5
Summary: This allows us to build tests to verify the behavior of background cache fills.
Test Plan: used in next diff.
Reviewers: mjpieters, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters, #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D3471343
Tasks: 11683504
Signature: t1:3471343:1466698914:93d15b2d4879b9d8a91aeab62fcf119f24de3c1c
Summary:
Since we no longer need to unlock in only one fork, we can use the `with lock() as xxx:` syntax
Depends on D3468830
Test Plan: pass existing unit tests
Reviewers: durham, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3468832
Tasks: 11683504
Signature: t1:3468832:1466694908:6e1d3beff673df5b70ece250af72ad9be66545a6
Summary:
Introduce a cacher class, which encapsulates all the logic for actually caching manifests. Triggers can either directly invoke cacher, or invoke `hg cachemanifest`, which then calls cacher.
Depends on D3468828
Test Plan: pass existing unit tests
Reviewers: durham, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3468830
Tasks: 11683504
Signature: t1:3468830:1466695022:9a470fcb06badd9e90931cd931a945e8039d71ea
Summary:
Everything will be synchronous within the process. We will do background work in a separate process.
Depends on D3468827
Test Plan: updated tests to remove the one background task.
Reviewers: durham, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3468828
Tasks: 11683504
Signature: t1:3468828:1466694717:556968a10717c3dd7ce403593cc6a1e97067633f
Summary:
We want to be able to dump debugging output for the workers.
Depends on D3468818
Test Plan: used in later diffs.
Reviewers: durham, lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters, #mercurial
Differential Revision: https://phabricator.intern.facebook.com/D3468827
Tasks: 11683504
Signature: t1:3468827:1466695135:bb930ac6e34f3654a7006634c22dc99b3d2e75bf
Summary: That is neccesary for this to work with extorder.
Test Plan: ~/clowncopter/tests/run-tests.py -j 30
Reviewers: #mercurial, ttung, durham, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3472367
Signature: t1:3472367:1466633447:8b5af3f546be9d1f3634882f9638cfe5feb6c181
Summary: I plan to use it to ensure proper loading ordr in case of sqldirstate
Test Plan:
~/clowncopter/tests/run-tests.py -j 30
Reviewers: #mercurial, ttung, durham, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3472358
Signature: t1:3472358:1466637677:3b8b5a89ef7dfbcde4a1e7b8ea58348b07a6a969
Summary: The iteritems makes sure that we do one big query instead of a query per item.
Test Plan:
./run-sqlitedirstate-test.py --hg ~/clowncopter/ -j 30 -t 1200
~/clowncopter/tests/run-tests.py -j 30
Reviewers: #mercurial, durham, ttung
Reviewed By: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3472343
Signature: t1:3472343:1466642757:d29c569b718fbfbf2c9b030467f9a262c0c4078b
Summary:
This extensions prevents mercurial from being ctrl-c'ed by warning the
user on the first SIGINT. The subsequent interrupts are handled normally.
Test Plan:
see tests
(the tests are lacking the scenario of double ctrl-c - any suggestions on how
to do it in nice way inside the test framework welcome)
Reviewers: #sourcecontrol, durham, wez, quark, akushner, rmcelroy
Subscribers: lcharignon, quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D2936488
Summary: The scm wrappers is agnostic of which scm. Technically, we could implement the same functionality in git.
Test Plan: run with the wrappers
Reviewers: lcharignon
Reviewed By: lcharignon
Subscribers: mitrandir, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3465938
Signature: t1:3465938:1466550871:912a627d114961b76ef605c89ccfcc75b6665664
Summary:
Directly execute the xargs command, rather than using a shell. This is
slightly more efficient, but also ensures we won't have security problems or
other issues based on the user's grep.command config setting. Even if it
contains shell redirects, semicolons, or other special characters, we will no
longer pass them to a shell to interpret them.
We do use shlex.split() go process the grep.command setting, to allow the user
to specify a command plus arguments.
Test Plan: Included new unit tests.
Reviewers: #sourcecontrol, durham, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: akushner, rmcelroy, sid0, net-systems-diffs@, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3376617
Signature: t1:3376617:1464841542:f58fc0a05d62b649edc70bc261009875385186c9
Summary:
Previously grep only searched in the current directory, even if you gave it a
pattern like "../". This fixes the code to correctly honor the arguments, and
to accept mercurial file patterns just like most other mercurial commands.
This does change the file parsing behavior somewhat: previously "hg grep" would
treat all arguments as glob patterns. Now you explicitly have to say
"glob:<pattern>" to get glob matching. However, I suspect this won't impact
most users, since they would normally let their shell perform glob expansion
before the arguments are given to mercurial.
Test Plan:
Included new unit tests for grep, and updated the existing tests that were
affected by the change in glob behavior.
Reviewers: #sourcecontrol, durham, carenthomas, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: quark, mwilliams, matthieu, meyering, yfeldblum, rmcelroy, net-systems-diffs@, yogeshwer, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3376605
Signature: t1:3376605:1464841648:98bfb47866dec802ca34f55c821796f36532211f
Summary:
This patch fixes various things around the sampling extension to
match what our wrapper expects to see. See detail in the test.
Test Plan:
lcharignon@XXX fbsource cat ~/.hgrc
...
[extensions]
sampling=
[sampling]
key.fastmanifest-cachehitratio=table_blah
key.fastmanifest-trigger=table_blah
filepath=/dev/shm/samppath
lcharignon@XXX fbsource hhg book foo
lcharignon@XXX fbsource cat /dev/shm/samppath
{"category": "table_blah", "data": {"source": "bookmark", "metrics_type":
"fastmanifest-trigger"}}\0{"category": "table_blah", "data": {"ratio": -1, "metrics_type":
"fastmanifest-cachehitratio"}}\0
Reviewers: ttung, durham
Differential Revision: https://phabricator.intern.facebook.com/D3466719
Summary:
This patch adds a test that uncovers logic bugs in the cache. It also
adds fixes for these bugs.
Test Plan:
Add a new test and ran existing tests, observed no change for
exisiting test.
Reviewers: durham, ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3450430
Summary: This patch fixes the torulemethod of the exec action to match upstream.
Test Plan: Add a test that failed before the codechange
Reviewers: mitrandir
Differential Revision: https://phabricator.intern.facebook.com/D3459588
Summary:
The upstream change has now been *properly* landed at
https://selenic.com/repo/hg//rev/6ee50bdeeb32.
Once again, Mea Culpa, yesterday I apparently was in clowntown mode with these
patches.
Test Plan:
Run all affected tests:
$HGSRC/tests/run-tests.py -l test-pull-createmarkers* test-pushrebase* test-pushvars*
Reviewers: #sourcecontrol, ttung, quark
Reviewed By: quark
Subscribers: mjpieters, durham
Differential Revision: https://phabricator.intern.facebook.com/D3462832
Signature: t1:3462832:1466513182:58d6187c04e0e567a026aba90bf753c76f4949a7
Blame Revision: D3460645
Summary:
e61bd9d5e8ca requires a change to core that has been accepted but not yet
pushed. This breaks the build, so let's revert it until upstream pushes the
dependency.
Test Plan: Ran the tests
Reviewers: ttung, mjpieters, #sourcecontrol, mitrandir
Reviewed By: mitrandir
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3460645
Signature: t1:3460645:1466465904:cd521a989e475f9987237307a7d4d672d58ba615
Summary: We need to add `patchpython` to `setup.py` so it can get built.
Test Plan: `make local`
Reviewers: #sourcecontrol, ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3456715
Summary:
This file clearly predates linting, if `hookargs` is ever set to `None` a
`NameError` would be triggered here.
Test Plan:
Run all affected tests:
$HGSRC/tests/run-tests.py -l test-pull-createmarkers* test-pushrebase* test-pushvars*
Reviewers: #sourcecontrol, ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3456417
Summary:
Now that my upstream patch for filedescriptor class access has landed the helper function to bypass the descriptor protocol can go altogether.
Depends on D3454878
Test Plan:
Run affected extension tests:
$HGSRC/tests/run-tests.py -l test-sqldirstate-* test-reflog* test-sparse*
Reviewers: #sourcecontrol, ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3456368
Summary:
There were 3 clowntowny versions of the wrapfiledescriptor function, 2 being
dangerously wrong about how they scan the class hierarchy. All because
`classobj.descriptorname` invokes the descriptor protocol and the filecache
descriptor is not set up for that access.
Instead of one function to do it all, use one utility function to retrieve
the descriptor object from the class __dict__ to avoid the descriptor protocol
kicking in, and use mercurial.extensions.wrapfunction to do the actual
wrapping.
I've sent a patch upstream to fix accessing the filecache descriptor on the
class; if accepted we can drop the getfilecache utility function altogether,
leaving only the wrapfunction calls.
Test Plan:
Run affected extension tests:
$HGSRC/tests/run-tests.py -l test-sqldirstate-* test-reflog* test-sparse*
Reviewers: #sourcecontrol, ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3454878
Summary:
This diff refactors the fastmanifest cache. Main differences introduced:
- We use python magic methods to access the cache (__getitem__, __contains__, ...)
- The cache is separated in two classes: ondiskcache and fastmanifestcache, the
former handles all the I/O, the latter handles the cache hierarchy and logic.
This allows to better group I/O related operations and make it easier to reason
about the code.
- By default we use systemawarecachelimit when creating a cache instead of no
limit before.
- The code is simpler to understand, the interface is cleaner: we only use
manifest nodes not a mix of filepath, manifestnodes and cachekeys.
Test Plan: test output does not change
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3427797
Summary: This will be useful to refactor the cache
Test Plan:
We change the python test to give a vfs that makes sense, it will
be useful in the next patch when we refactor the cache. Indeed, with the new
cache we use systemawarecachelimit by default instead of no limit
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3427796
Summary:
Before this patch, if we decided to remove entries A,B,C where A is
more recent than B, which in turn is more recent than C. We were removing
in order A,B and C. This patch changes the order to C,B and A and will make
the next patches simpler.
Test Plan: Tests are updated accordingly
Reviewers: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3427795
Summary:
When refreshing cache entries, it is useful to know what delay we
introduce. This patch adds a debug line showing that information.
Test Plan: tests are updated accordingly
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3427794
Summary:
Before this patch, it wasn't clear from hg debugcachemanifest --list
which manifest were older and newer or what rev were actually cached. This
patch improves the output of hg debugcachemanifest --list --debug to show
this information.
Test Plan: add debug output to the test
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3427791
Summary:
Before this patch we were returning the cache entries alphabetically.
This patch changes the behavior to return the entries sorted by date and
alphabetically. This will be used to simplify the code for pruning entries
in furter patches.
Test Plan: test output changes
Reviewers: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3427789
Summary:
The existing statprof uses signals to perform stack sampling. Those signals can
interfere with system calls though. This patch introduces a way of using python
threads to perform sampling from a background thread.
This also changes the default to be the threaded profiler instead of the
sampling one, since the sampling can crash the proces sometimes.
In some situations this produces a more accurate stack trace.
Test Plan:
```
sudo cp statprof.py /usr/lib64/python2.6/site-packages/statprof.py
hg pull --profile
# Verified I got output and that it was different from the old statprof (it was
# more accurate, as compared with time.time() measurements in the code)
```
Reviewers: #mercurial, ttung, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3402864
Signature: t1:3402864:1465978363:c8455d119cc03c2c475e190aee28587a9d434c4d
Summary:
Before this patch if the path was not accessible we would fail to
compute the limit.
Test Plan:
We should test systematically all these kind of issues, I am doing
a refactoring that will allow us to do that.
Reviewers: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3424996
Summary:
Before this patch we were stopping execution when the lock couldn't
be taken. This was problematic when running hg command for someone else's repo
(for example to debug problems). This patch allows us to continue the execution
of the command when that happen, except we won't use fastmanifest.
Test Plan: This is reflected in the test output
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3422458
Summary: This could break people using fastmanifest in someone else's repo
Test Plan: tests pass, not sure how we normally test permission stuff
Reviewers: ttung
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3420870
Summary:
Before this patch we would have error using fastmanifest on repos you
didn't own.
Test Plan: tests pass
Reviewers: ttung
Differential Revision: https://phabricator.intern.facebook.com/D3420863
